Open Standards for Today’s Gaming

Industry
Erik Noreke
VP of Business Development, Khronos Group

© Copyright Khronos Group 2013 - Page 1

Gaming Evolution

DESKTOP MOBILE

New high Mobile is the new platform for game innovation.

performance games Diverse platforms – mobile, TV, embedded
Mobile APIs unlock hardware acceleration and
continue to evolve on – means HTML5 will become
conserve battery life, making portable gaming
high-end platforms increasingly important as a universal
possible
gaming platform

© Copyright Khronos Group 2013 - Page 3

Courtesy of PC Gaming Alliance
www.pcgamingalliance.org
© Copyright Khronos Group 2013 - Page 4
Courtesy of PC Gaming Alliance
www.pcgamingalliance.org
© Copyright Khronos Group 2013 - Page 5
 US$28.7B gaming revenues generated on platforms supporting
Khronos native graphics and compute APIs

Courtesy of PC Gaming Alliance

www.pcgamingalliance.org
© Copyright Khronos Group 2013 - Page 6
 OpenGL 3D API Family Tree
ES3 is backward compatible
Fixed function Programmable vertex so new features can be
3D Pipeline and fragment shaders added incrementally

OpenGL ES 1.1 OpenGL ES 2.0 OpenGL ES 3.0

Content Content Content

Mobile 3D
WebGL 1.0 WebGL-Next

OpenGL ES 1.1 OpenGL ES 2.0 OpenGL ES 3.0 ES-Next

OpenGL ES 1.0

OpenGL 1.3 OpenGL 1.5 OpenGL 2.0 OpenGL 2.1 OpenGL 3.1 OpenGL 3.3 OpenGL 4.2 OpenGL 4.3 OpenGL 4.4 GL-Next
OpenGL 3.0 OpenGL 3.2 OpenGL 4.0
OpenGL 4.1

OpenGL 4.4 is a
superset of DX11
Desktop 3D
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

© Copyright Khronos Group 2013 - Page 7

Mobile OpenCL Shipping
• Android ICD extension released in latest extension specification
- OpenCL implementations can be discovered and loaded as a shared object
• Multiple implementations shipping in Android NDK
- ARM, Imagination, Vivante, Qualcomm, Samsung …

© Copyright Khronos Group 2013 - Page 8

 Active Gamers by Platform

Courtesy of PC Gaming Alliance CAMERA

www.pcgamingalliance.org
© Copyright Khronos Group 2013 - Page 9
Sensors & Cameras
• Single sensor RGB cameras are just the start of the mobile visual revolution
- IR sensors – LEAP Motion, eye-trackers
• Multi-sensors: Stereo pairs -> Plenoptic array -> Depth cameras
- Stereo pair can enable object scaling and enhanced depth extraction
• Hybrid visual sensing solutions
- Different sensors mixed for different distances and lighting conditions

Dual Camera Plenoptic Array Capri Structured Light 3D Camera

LG Electronics Pelican imaging PrimeSense
© Copyright Khronos Group 2013 - Page 10
Typical Imaging Pipeline
• Pre- and Post-processing can be done on CPU, GPU, DSP…
• ISP controls camera via 3A algorithms
Auto Exposure (AE), Auto White Balance (AWB), Auto Focus (AF)
• ISP may be a separate chip or within Application Processor

Lens, sensor, aperture control 3A

Bayer RGB/YUV
Image Signal Processor Post-
Pre-processing App
(ISP) processing

CMOS sensor Need for advanced

Color Filter Array camera control API!
Lens
CAMERA
© Copyright Khronos Group 2013 - Page 11
Low Power Environment Scanning
• Many sensor use cases would consume too much power to be running 24/7
- Environment aware use cases have to be very low power
• ‘Scanners’ - very low power, always on, detect things in the environment
- Trigger the next level of processing capability

ARM 7 DSP GPU

1 MIP and accelerometers can Low power activation of camera GPU acceleration for precision
detect someone in the vicinity to detect someone in field of view gesture processing

© Copyright Khronos Group 2013 - Page 12

StreamInput Sensor Fusion Stack
Platforms can provide
increased access to
improved sensor data stream
– driving faster, deeper
sensor usage by applications
StreamInput implementations
compete on sensor stream quality,
reduced power consumption,
environment triggering and context
detection – enabling sensor
subsystem vendors to increased
ADDED VALUE
Applications
Middleware engines need platform-
portable access to native, low-level
OS Sensor OS APIs Middleware sensor data stream
(E.g. Android SensorManager or (E.g. Augmented Reality engines,
iOS CoreMotion) gaming engines)
Mobile or embedded
platforms without sensor
fusion APIs can provide
direct application access
Low-level native API defines access to to StreamInput
fused sensor data stream and context-awareness
Sensor
Sensor
… Sensor
Sensor
Hub
Hardware transport
interfaces are defined
by each system, e.g.
Hub IIO or HID sensor
© Copyright Khronos Group 2013 - Page 13
 Worldwide Game Users – 2012 (Active and Casual)
WORLDWIDE GAME USERS
BY REGION AND PLATFORM (IN MILLIONS)
500

400

300

200

100

0
N.America Europe Asia Other
Console PC Dedicated Handheld Mobile

Courtesy of PC Gaming Alliance

www.pcgamingalliance.org
© Copyright Khronos Group 2013 - Page 14
Cross-OS, Cross Platform Portability

HTML5 provides cross

HTML/CSS HTML/CSS HTML/CSS platform portability. GPU
accessibility through
WebGL available soon on
~90% mobile systems

Dalvik Preferred development

SDK Objective C C# environments not
(Java) designed for portability

Native code is portable-

but apps must cope with
C/C++ DirectX different available APIs
and libraries

© Copyright Khronos Group 2013 - Page 15

WebGL Availability in Browsers

Much WebGL content uses

three.js library:
http://threejs.org/

- Microsoft – “where you have IE11, you have WebGL – turned on by default and working all the time”
- Microsoft - WebGL also enabled for Windows applications - web app framework and web view
- Apple - WebGL must be explicitly turned on MAC Safari and only exposed on iOS for iAds
- Chrome OS - WebGL is the only cross-platform API to program the GPU
- Google IO announcement - Chrome on Android will soon launch with WebGL
© Copyright Khronos Group 2013 - Page 16
WebCL – Parallel Computing for the Web
• JavaScript bindings to hardware acceleration
- Giving performance gaming in a web browser

http://www.youtube.com/user/SamsungSISA#p/a/u/1/9Ttux1A-Nuc © Copyright Khronos Group 2013 - Page 17

Remember
© Copyright Me imageGroup
Khronos courtesy
2013of- Page
CapCom
18
3D Needs a Transmission Format!
• Compression and streaming of 3D assets becoming essential
- Mobile and connected devices need access to increasingly large asset databases
• 3D is the last media type to define a compressed format
- 3D is more complex – diverse asset types and use cases
• Needs to be royalty-free
- Avoid an ‘internet video codec war’ scenario
• Eventually enable hardware implementations of successful codecs
- High-performance and low power – but pragmatic adoption strategy is key

Audio Video Images 3D

MP3 H.264 JPEG ?
!
An effective and widely adopted codec ignites previously
unimagined opportunities for a media type
© Copyright Khronos Group 2013 - Page 19
glTF Goals
• Binary file format for efficient transmission for 3D assets
- Reduce network bandwidth and minimize client processing overhead
• Run-time neutral - DO NOT IMPLY OR MANDATE ANY RUN-TIME BEHAVIOR
- Can be used by any app or run-time – usually WebGL accelerated
• Scalable to handle compression and streaming
- Though baseline format does not include compression
• ‘Direct load efficiency’ for WebGL
- Little or NO processing to drop glTF data into WebGL client
• Carry conditioned data from any authoring format
- Prototyping and optimizing efficient handling of COLLADA assets

A standards-based
content pipeline for
rich native and Web 3D
Authoring Playback applications
© Copyright Khronos Group 2013 - Page 20
Game On!
• Gaming and Animation is a complex and expanding application domain and
coherent multiple standards across multiple domains are needed to enable the
market
• Khronos family of standards allow you to focus your efforts on game
development, not porting between platforms
• Now is a great time to get involved with the standards initiatives
that effect your business and game development

• www.khronos.org
• erik.noreke@khronos.org

© Copyright Khronos Group 2013 - Page 21

© Copyright Khronos Group 2013 - Page 22
WebGL First Wave Application Categories
• Maps and Navigation
• Modeling Tools and Repositories
• Games
• 3D Printing
• Visualization
• Music Videos and Promotion
• Education
• Photo Editors
• Music Visualizers
• Vision/Video Processing

© Copyright Khronos Group 2013 - Page 23

COLLADA and glTF Open Source Ecosystem
OpenCOLLADA Tool Interop COLLADA2GLTF
Importer/Exporter
Translator
and COLLADA Other
Conformance Tests authoring
On GitHUB formats

https://github.com/KhronosGroup/glTF
Web-based https://github.com/KhronosGroup/OpenCOLLADA
Tools
https://github.com/KhronosGroup/COLLADA-CTS

Pervasive WebGL
deployment Three.js glTF Importer.
Rest3D initiative

© Copyright Khronos Group 2013 - Page 24

Google Maps
• All rendering (2D and 3D) in Google Maps uses WebGL

© Copyright Khronos Group 2013 - Page 25

WebCL – Parallel Computing for the Web
• JavaScript bindings to OpenCL APIs
- Enables initiation of Kernels written in OpenCL C within the browser
• Bindings stay close to the OpenCL standard
- Maximum flexibility to provide a foundation for higher-level middleware
- Minimal language modifications for 100% security and app portability
- E.g. Mapping of CL memory objects into host memory space is not supported
• API definition underway – public draft released
- https://cvs.khronos.org/svn/repos/registry/trunk/public/webcl/spec/latest/index.html
• Compelling use cases
- Physics engines for WebGL games
River Trail
- Image and video editing in browser
JavaScript binding to OpenCL Data-parallel language
APIs to enable initiation of extensions to JavaScript
OpenCL C kernels for offload at run-time

© Copyright Khronos Group 2013 - Page 26

Leveraging Proven Native APIs into HTML5
• Khronos and W3C liaison
- Leverage proven native API investments into the Web
- Fast API development and deployment
- Designed by the hardware community
- Familiar foundation reduces developer learning curve

HTML WebCAM(!)
Canvas WebVX? Camera
Vision
WebStream?
Sensor Fusion
control and JavaScript
Processing video
processing

Camera
Control
Native
Path Rendering

JavaScript API shipping, Possible future

Native APIs shipping
acceleration being developed JavaScript APIs or
or Khronos working group
or work underway acceleration

© Copyright Khronos Group 2013 - Page 27

OpenGL ES 3.0 Highlights
• Better looking, faster performing games and apps – at lower power
- Incorporates proven features from OpenGL 3.3 / 4.x
- 32-bit integers and floats in shader programs
- NPOT, 3D textures, depth textures, texture arrays
- Multiple Render Targets for deferred rendering, Occlusion Queries
- Instanced Rendering, Transform Feedback …
• Make life better for the programmer
- Tighter requirements for supported features to reduce implementation variability
• Backward compatible with OpenGL ES 2.0
- OpenGL ES 2.0 apps continue to run unmodified
• Standardized Texture Compression
- #1 developer request!

© Copyright Khronos Group 2013 - Page 32

Accelerating OpenGL Innovation

Bringing state-of-the-
art functionality to
cross-platform OpenGL 4.4
graphics
OpenGL 4.3
OpenGL 4.2
OpenGL 4.1
OpenGL 3.3/4.0
OpenGL 3.2
OpenGL 3.1
OpenGL 2.0 OpenGL 2.1 OpenGL 3.0

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

DirectX 9.0c DirectX 10.0 DirectX 10.1 DirectX 11 DirectX 11.1

© Copyright Khronos Group 2013 - Page 33

OpenGL 4.3 Compute Shaders
• Execute algorithmically general-purpose GLSL shaders
- Can operate on uniforms, images and textures
• Process graphics data in the context of the graphics pipeline
- Easier than interoperating with a compute API IF processing ‘close to the pixel’
• Standard part of all OpenGL 4.3 implementations
- Matches DX11 DirectCompute functionality

Physics AI Simulation Ray Tracing Imaging Global Illumination

© Copyright Khronos Group 2013 - Page 34

OpenGL Conformance Test Suite released!
 Conformance submissions are required for GL 4.4 implementations
 encouraged for earlier driver versions

 Shared codebase with OpenGL ES 3.0 CTS

 additional desktop-specific tests

 Core profile functionality

 Enhancements underway to add more coverage

© Copyright Khronos Group 2013 - Page 35

OpenCL and OpenGL Compute Shaders
• OpenGL compute shaders and OpenCL support distinctly different use cases
- OpenCL provides a significantly more powerful and complete compute solution

1. Fine grain compute operations 1. Full ANSI C programming of

inside OpenGL heterogeneous CPUs and GPUs
2. GLSL Shading Language 2. Utilize multiple processors
3. Execute on single GPU only 3. Precisely defined IEEE accuracy

Enhanced 3D Imaging Pure compute

Graphics apps Video apps touching
“Shaders++” Physics no pixels
AI

Compute Shaders
© Copyright Khronos Group 2013 - Page 36
Gaming Evolution

DESKTOP MOBILE

New high Mobile is the new platform for game innovation.

performance games Diverse platforms – mobile, TV, embedded
Mobile APIs unlock hardware acceleration and
still evolve on high- – means HTML5 will become
conserve battery life, making portable gaming
end platforms increasingly important as a universal
possible
gaming platform

© Copyright Khronos Group 2013 - Page 37