IoT Media Stack on Yocto Project*-Based

Metadata and Source Code for Intel

Atom® Processor E3900 Series
White Paper

March 2017

Revision 001

Ajit Kumar K. P
Graphics Platform Application Engineer
Internet of Things Group, Intel

Document Number: 335260-001US

You may not use or facilitate the use of this document in connection with any infringement or other legal analysis concerning Intel
products described herein. You agree to grant Intel a non-exclusive, royalty-free license to any patent claim thereafter drafted
which includes subject matter disclosed herein

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel
product specifications and roadmaps.

The products described may contain design defects or errors known as errata which may cause the product to deviate from
published specifications. Current characterized errata are available on request.

Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725
or by visiting: http://www.intel.com/design/literature.htm

Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service
activation. Learn more at http://www.intel.com/ or from the OEM or retailer.

No computer system can be absolutely secure.

Intel, Intel Atom, and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.

*Other names and brands may be claimed as the property of others.

Copyright © 2017, Intel Corporation. All rights reserved.

IoT Media Stack for Intel Atom® Processor E3900 Series

White Paper March 2017
2 Document Number: 335260-001US
Contents
1.0 Introduction ............................................................................................................................................ 5
1.1 Terminology .................................................................................................................................................... 5
1.2 Reference Documents .............................................................................................................................. 5

2.0 Default Yocto Project* Media Stack ......................................................................................... 6

2.1 Ingredients of the Default Yocto Project Media Stack ....................................................... 7
2.2 Usage Models on the Default Yocto Project Media Stack ................................................ 8

3.0 IoT Media Stack .................................................................................................................................... 9

3.1 Usage Models on the IoT Media Stack ........................................................................................12
3.2 Customizing the IoT Media Stack ..................................................................................................12
4.0 Summary ................................................................................................................................................. 14

Figures
Figure 1. Default Yocto Project Media Stack (Open Source) ................................................................. 7
Figure 2. Options when Compiling a BSP Image ........................................................................................... 8
Figure 3. IoT Media Stack (Proprietary) ............................................................................................................10

Tables
Table 1. Terminology ...................................................................................................................................................... 5
Table 2. Reference Documents ................................................................................................................................ 5

IoT Media Stack for Intel Atom® Processor E3900 Series

March 2017 White Paper
Document Number: 335260-001US 3
Revision History

Date Revision Description

March 2017 001 Initial release.

IoT Media Stack for Intel Atom® Processor E3900 Series

White Paper March 2017
4 Document Number: 335260-001US
Introduction

1.0 Introduction
This document provides an overview of software components that comprise the IoT
Media Stack on Yocto Project*-based metadata and source code for Intel Atom®
Processor E3900 Series. This document also introduces and provides a distinction
between the IoT Media Stack and the default Yocto Project Media Stack. This document
does not detail the architectural level of the software components.

The complete graphics stack for Intel Atom® processor E3900 series includes additional
user space drivers as part of the Intel® Unified 3D Library package that includes the
OpenCL*, direct rendering infrastructure (DRI), and video acceleration (VA) drivers.

For more information on how to integrate Intel Unified 3D Library with the board
support package (BSP), refer to the Intel Unified 3D Library package, as stated in Table 2.

1.1 Terminology
Table 1. Terminology

Term Description

API Application Programming Interface

BSP Board Support Package

GPU Graphics Processing Unit

IoT Internet of Things

POR Plan of Record

1.2 Reference Documents

Table 2. Reference Documents

Document Document
No./Location

Intel Unified 3D Library package http://www.intel.com/content/www/us/en/embedded/products/ap

ollo-lake/software-and-drivers.html

IoT Media Stack for Intel Atom® Processor E3900 Series

March 2017 White Paper
Document Number: 335260-001US 5
 Default Yocto Project* Media Stack

2.0 Default Yocto Project* Media Stack

The default Yocto Project* Media stack is available upon compiling the BSP without any
additional Yocto Project recipes. This media stack consists of fundamental building
blocks of a typical Yocto Project-based metadata and source code that are mostly open
source.

The default Yocto Project Media Stack for Intel Atom® Processor E3900 Series consists
of the following software components shown in Figure 1.

IoT Media Stack for Intel Atom® Processor E3900 Series

White Paper March 2017
6 Document Number: 335260-001US
Default Yocto Project* Media Stack

Figure 1. Default Yocto Project Media Stack (Open Source)

Video Applications

2
GStreamer/FFMPEG
Framework
1

Open Source LibVA

i965 VA Driver

LibDRM

Base BSP
User Space

Linux Kernel i915 DRM driver

Hardware Intel® GPU

2.1 Ingredients of the Default Yocto Project Media Stack

While compiling (using the BitBake build engine) a BSP image for Intel Atom Processor
E3900 Series, the default “core-image-sato-sdk” option generates an image that
contains the default Yocto Project Media Stack.

IoT Media Stack for Intel Atom® Processor E3900 Series

March 2017 White Paper
Document Number: 335260-001US 7
 Default Yocto Project* Media Stack

Figure 2. Options when Compiling a BSP Image

The base BSP provides necessary recipes for the kernel space graphics components
(including the i915 DRM driver). The “meta-intel-middleware” layer provides additional
recipes for user space middleware components that includes, but not limited to,
connectivity, security, graphics, and multimedia.

“recipe-graphics” and “recipe-multimedia” supply the ingredients specifically for user

space graphics and multimedia stack. Notable user space graphics and multimedia
stack are as follows:
• gstreamer framework (refer https://gstreamer.freedesktop.org/src/)
• Open Source LibVA (refer
https://www.freedesktop.org/software/vaapi/releases/libva/)
• Open Source i965 VA driver (refer
https://www.freedesktop.org/software/vaapi/releases/libva-intel-driver/)
• LibDRM (refer https://cgit.freedesktop.org/mesa/drm/)

Note: Refer to the graphics and multimedia recipes in “meta-intel-middleware” for updated
links.

2.2 Usage Models on the Default Yocto Project Media Stack

Figure 1 shows two hardware acceleration paths for video applications:

Path 1 is likely the most common path where customers use well-established
multimedia frameworks such as GStreamer SDK (or FFmpeg* framework) to develop
their applications.

Path 2 is adopted usually by customers who have a good understanding of LibVA and
prefer to use VA-API directly in their applications.

IoT Media Stack for Intel Atom® Processor E3900 Series

White Paper March 2017
8 Document Number: 335260-001US
IoT Media Stack

3.0 IoT Media Stack

The IoT Media Stack (shown in Figure 3), replaces certain user-space components in the
default media stack, and also includes certain additional middleware components that
make the video acceleration framework on Intel Atom® Processor E3900 series
effective and efficient. The following section provides details of the components that
have been replaced and newly added.

IoT Media Stack for Intel Atom® Processor E3900 Series

March 2017 White Paper
Document Number: 335260-001US 9
 IoT Media Stack

Figure 3. IoT Media Stack (Proprietary)

Video Applications
3

GStreamer Framework
3

GStreamer-Media SDK 4 5

Plugins
3

Intel® Media SDK

Libraries Plugins Samples Tools

LibVA staging

iHD VA Driver
meta-intel-msdk

LibDRM
User Space

Linux Kernel i915 DRM driver

Hardware Intel® GPU

IoT Media Stack for Intel Atom® Processor E3900 Series

White Paper March 2017
10 Document Number: 335260-001US
IoT Media Stack

The “meta-intel-msdk” layer supplies the following middleware components that make
up the IoT Media Stack:
1. GStreamer-Intel® Media SDK Plugins

These plugins are based on the GStreamer-VAAPI architecture and provides a

set of GStreamer plugins for Intel Media SDK. The primary purpose of these
plugins is to make the GStreamer framework compatible with Intel Media SDK.
These plugins enable customers familiar with the GStreamer framework but not
the Intel Media SDK APIs, to receive the benefits of hardware acceleration
offered by Intel Media SDK. These plugins are available at
https://github.com/01org/gstreamer-media-SDK.

2. Intel® Media SDK

Intel® Media SDK offers a set of software development libraries that expose the
media acceleration capabilities of Intel® platforms for various media workloads
such as decoding, encoding, and video preprocessing. Intel Media SDK consists
of hardware acceleration libraries, plugins, media sample applications, and
debug tools. The Intel Media SDK package can be installed from
https://software.intel.com/en-us/media-sdk

3. LibVA staging

This dedicated LibVA library works in tandem with Intel Media SDK. This open
source library is maintained at https://github.com/01org/iotg-lin-gfx-libva. The
compiled LibVA-staging binaries are also available in the Intel® Unified 3D
Library package.

4. iHD VA driver

This is an Intel-proprietary VA driver custom-built for Intel Media SDK and

LibVA staging. Unlike the i965 VA driver on the default stack (see Figure 1), the
iHD VA driver is proprietary. The compiled driver binary (.so) is packaged in the
Intel Unified 3D Library package.

Note: Besides LibVA–staging and iHD VA driver, the Intel Unified 3D Library package also
includes the UFO-DRI and OpenCL drivers (available at
http://www.intel.com/content/www/us/en/embedded/products/apollo-lake/software-
and-drivers.html).

Note: OpenCL and UFO-DRI drivers do not have any dependencies on the LibVA-staging
library or iHD VA driver. Therefore, OpenCL and UFO-DRI drivers can work with either
the default Yocto Project Media Stack or the IoT Media Stack.

Note: “meta-intel-ufo” provides the necessary recipes to pre-install UFO-DRI and OpenCL
drivers during the BitBake build. Therefore, users who need UFO-DRI and OpenCL

IoT Media Stack for Intel Atom® Processor E3900 Series

March 2017 White Paper
Document Number: 335260-001US 11
 IoT Media Stack

drivers, and not Intel Media SDK, only need to use “meta-intel-ufo” during the BitBake
build. The users can exclude “meta-intel-msdk” during the BitBake build if they do not
need the IOT media stack.

3.1 Usage Models on the IoT Media Stack

In Figure 3, Intel recommends three paths for customer applications.

Through Path 3, customers who are familiar with the GStreamer framework can
seamlessly integrate their applications using the GStreamer-Intel Media SDK plugins.
These plugins allow customers to easily adopt Intel Media SDK and benefit from the
hardware acceleration capabilities offered from the proprietary iHD VA driver.

Note: Customer applications developed based on the GStreamer framework can only work on
the IoT Media Stack if the GStreamer-Intel Media SDK plugins are used.

Path 4 is recommended for customers who directly use Intel Media SDK APIs to
develop their video applications. For details, refer to the Intel Media SDK website at
https://software.intel.com/en-us/media-sdk

Path 5 is for customers who understand LibVA well and who want to use VA-APIs
directly in their applications, because LibVA staging is open source.

3.2 Customizing the IoT Media Stack

Intel recommends the following two methods to build the IoT Media Stack shown in
Figure 3.
1. Manually Install IoT Media Stack (recommended for first-time users):

Build (using the BitBake build engine) the default Yocto Project*-based metadata
and source code BSP and manually install each software component on the IoT
Media Stack on the target machine at run time.

2. Pre-Install IoT Media Stack (recommended for advanced users):

Build a bootable image using the base BSP and “meta-intel-msdk” to include all
software components on the IoT Media Stack at compilation time (during the
BitBake process).

Customers who do not wish to install certain components can exclude those
recipes from “bblayers.conf”, e.g:

To exclude GStreamer-Intel Media SDK plugins, remove “gstreamer-msdk”.

IoT Media Stack for Intel Atom® Processor E3900 Series

White Paper March 2017
12 Document Number: 335260-001US
IoT Media Stack

The Intel Media SDK Getting Started Guide (available at

http://www.intel.com/content/www/us/en/embedded/products/apollo-lake/software-
and-drivers.html) provides detailed procedures for these two methods.

IoT Media Stack for Intel Atom® Processor E3900 Series

March 2017 White Paper
Document Number: 335260-001US 13
 Summary

4.0 Summary
This whitepaper has provided an overview of the two software stacks supported on
Intel Atom® Processor E3900 Series, namely the default Yocto Project Media Stack and
the IoT Media Stack. This whitepaper has also highlighted the different paths video
applications can take to achieve hardware acceleration. Customers should carefully
consider the pros and cons of the two media stacks and their video acceleration paths
before developing their solutions on Intel Atom® Processor E3900 Series.

To get the optimum hardware acceleration performance out of the Intel Atom®
Processor E3900 Series, Intel recommends customers to adopt the IoT Media Stack.
The software ingredients on the IoT Media Stack also have long-life support from Intel.

Intel does not recommend customers to mix and match software ingredients between
the Yocto Project* Media Stack and the IoT Media Stack.

Note: For benchmarking information, please contact your Intel representatives.

IoT Media Stack for Intel Atom® Processor E3900 Series

White Paper March 2017
14 Document Number: 335260-001US