Professional Documents
Culture Documents
Qualcomm AGENDA
Qualcomm AGENDA
+ AI
The essential role of AI in the 5G future
+ AI
2
Unifying connectivity fabric that can efficiently Learning platform that can make virtually
connect virtually everything around us everything around us intelligent
5G
Extreme capacity
better AI
Contextual awareness
Multi-Gbps speeds Personalization at scale
Ultra-high reliability and low latency Intelligent, intuitive, and automated actions
Robust end-to-end security and privacy Continual improvement through self-learning
New and diverse services, spectrum, deployments Solving seemingly impossible-to-model problems
5G better
Elevated level of performance
AI better
Responsive user experiences and services
More efficient resource utilization Lifelong learning
Energy reduction for longer battery life Flexibility for distributed functionality across devices
Personalized security and privacy On-device intelligence assisted by cloud
Continuous enhancements over time Scale intelligence through distributed learning
New and enhanced system capabilities Massive data aggregation for improved AI models
5
Transformation of the
Connected Intelligent Public network
Edge has begun at scale
Processing data closer to devices at the edge derives
new system values (e.g., lower latency, enhanced privacy)
Private networks
Past Today Future
Cloud-centric AI Partially-distributed AI Fully-distributed AI
AI training and Power-efficient With lifelong on-
inference in the on-device AI inference device learning
central cloud
6
Local network
analytics
Connected
Voice Object
activation classification
Low-latency
interactive content
Intelligent Edge Text
Fingerprint
recognition
brings new and enhanced services
Boundless XR
Face On-device
detection security
On-demand
computing
Industrial
Edge
Cloud AI + On-device
AI
Voice
recognition
Gesture/
hand tracking
automation
and control Computational Contextual
photography awareness
Enterprise data
7
Federated learning brings on-device learning to new level
Coordinator
5G
Model connectivity
updates
Training on
local data
Training on
local data
Worker Worker
Data
Locally adapt once to a few samples Aggregate model updates across
(e.g., few shot learning) or continuously multiple users to globally improve model
(e.g., unsupervised learning) from more diverse data
8
Our research focus to improve 5G with AI
With a cloud-to-network-to-device approach for data collection and learning
10
Wireless challenges AI strengths
Hard-to-model problems Determining appropriate
representations for hard-to-
Computational model problems
infeasibility
of optimal solution Finding near-ideal and
computationally realizable
Efficient modem solutions
parameter optimization
Modeling non-linear functions
Dealing with non-linearity
AI-enhanced
wireless communications
12
Spectrum sensing and access
Predict activities of other devices for more efficient
access and better scheduling to improve 5G
system performance
Contextual awareness
Radio awareness Use device context (e.g., position, velocity,
or in-car) derived from RF, sensors, traffic
achieved by advanced on-device
activities to improve device experience
AI algorithms
13
On-device AI
enhances 5G
device experience Agent
State
M strongest
RSRPs
14
On-device AI
improves 5G system performance
AI inference result
15
Demo: Mobile 5G mmWave beam prediction
16
Improving system spectral efficiency
Implementing a neural network framework for CSI1 on non-linear temporal encoding and decoding
Applying AI
for enhanced
Improving device power efficiency
5G air interface Optimizing transmit waveform to reduce Allowing receiver to recover signal from a
efficiency peak-to-average power ratio (PAPR) device operating in the non-linear PA region
NLOS
Multi-path detection
Blockage
LOS
RF sensing
Learning device position over time without prior Sensing changes in environment to infer location
knowledge with RF sensing — complementing and type of motion for a wide range of use cases
existing positioning methodologies1 (e.g., vital sign tracking, fall detection)
1. For example, Observed Time Difference of Arrival (OTDOA), Multiple Round Trip Time (Multi-RTT), Angle of Arrival (AoA)
Sensor information
performance
Channel multi-path profile Improved device
(e.g., NLOS) position estimate
Neural Network
20
Demo: Precise 5G positioning with machine learning fusion
21
Demo: Wireless AI-assisted indoor positioning
Leveraging unsupervised/weakly supervised learning — also applicable to 5G RF sensing (e.g., for positioning, motion and gesture detection)
22
AI enables intelligent 5G network management
23
A more intelligent way
to deploy 5G mmWave
Create a digital twin of targeted deployment
based on readily available sources/databases
(e.g., Google Street/Aerial view, GIS, …)
24
Distributed topology enables more efficient deployments
Standardization in e.g., 3GPP, O-RAN Alliance
Integrated Access/
Backhaul (IAB)
In-band or
out of-band wireless
Fiber
Centralized backhaul
Small Cell
coordination
Fiber
fronthaul Simple Repeater
ORAN Radio Unit
Diversity of node types e.g., Diversity of interconnectivity Many potential radio locations
small cells, IAB, repeaters, … e.g., fiber, out-of-band wireless, … e.g., for different objectives
25
Creating a digital twin of targeted deployment
using readily available databases
Example in Tokyo, Japan
27
5G mmWave topology optimization example
Design with integrated access/backhaul, repeaters, small cells with
in-band and out-of-band backhaul
Existing Newly added Unused Device Repeaters Integrated In-band Out-of-band In-band
small cells small cells poles hotspots access/ backhaul backhaul access
backhaul
28
Expanding design
to larger deployment
area with diverse
mmWave topologies
29
Machine learning
plays an expanding role in the evolution
of 5G towards 6G
30
Advancing 5G to fulfill its full promise
Enhanced mobile experiences, new capabilities, and expansion to diverse verticals
31
Standardizing AI/ML in cellular communication systems
Broad range of work across standards and industry organizations
Developing AI use cases Developing an AI Mobile Device Requirement Spec Data collection for network performance
Architectural framework for ML (TS.47) enhancements (RAN2/3 — Rel-16/17)
Framework for evaluating intelligence level Focusing on AI mobile phone and tablet (may extend Study on AI/ML functional frameworks and use cases
to IoT/wearable in future releases) (RAN3 — Rel-17)
Framework for data handling to enable ML
Network data analytic function for core AI/ML use
AI for autonomous and assisted driving cases (SA2 — Rel-16/17)
Management data analytic service, autonomous
network (SA5 — R17)
Study on AI/ML model transfer performance
requirements over 5G (SA1 — Rel-17)
Defining reference architecture for Radio Intelligence Network automation and autonomy based on AI
Controller (RIC) and interfaces Defining requirements and platform recommendations for
Developing technical report: “AI/ML Workflow Description reference implementation and interface standards
and Requirement”
32
Driving the 5G technology evolution in the new decade
Rel-20+ evolution
Longer-term evolution to
deliver on the 5G vision Rel-191
New verticals, deployments,
use cases, spectrum Rel-181
Rel-161
Rel-15
2018 2019 2020 2021 2022 2023 2024 2025 2026 2027+
Rel-15 eMBB focus Rel-16 industry expansion Rel-17 continued expansion Rel-18+ 5G-Advanced
• 5G NR foundation • eURLLC and TSN for IIoT • Lower complexity NR-Light • Next set of 5G releases
• Smartphones, FWA, PC • NR in unlicensed • Higher precision positioning (i.e., 18, 19, 20, …)
• Expanding to venues, • 5G V2X sidelink multicast • Improved IIoT, V2X, IAB, and more… • Potential projects in discussions
enterprises • In-band eMTC/NB-IoT • Rel-18 expected to start in 2022
• Positioning
1. 3GPP start date indicates approval of study package (study item->work item->specifications), previous release continues beyond start of next release with functional freezes and ASN.1 33
Data collection for network performance enhancements
Part of 3GPP Release 16
Enhanced Network Automation (eNA) Minimization of Drive Testing (MDT) Self Organizing Network (SON)
Data
repositories
Delivery of
activity data Measurements, e.g., average
and local Delivery of delay, sensor information
analytics analytics data
NWDAF
OAM
New enhanced core network function for Logged and immediate MDT, mobility Mobility robust optimization (MRO),
data collection and exposure history information, accessibility and L2 mobility load balancing (MLB), and
Expanding NWDAF1 from providing network slice
measurements3 RACH optimization
analysis in Rel-15 to data collection and exposure Specifying features for identified use cases, including Specifying device reporting needed to enhance network
from/to 5G core NF, AF, OAM2, data repositories coverage, optimization, QoS verification, location configurations and inter-node information exchange
information reporting, sensor data collection (e.g., enhancements to interfaces like N2, Xn)
1 NWDAF — Network Data Analytics Function; 2 Network Function, Application Function, Operations Administration and Maintenance; 3 For standalone and dual connected 5G NR systems 34
Expanding 5G system support
for wireless machine learning
Part of 3GPP Release 17
1 Such as multicast, broadcast, V2X, sidelink, multi-SIM, RAN slicing, and more
35
Network architecture AI/ML procedure Data management New and expanded
enhancements enhancements enhancements use cases
Allowing for machine Optimizing system for model Standardizing ML data storage/ Supporting traffic/mobility
learning to run over different management, training (e.g., access, data registration/ prediction, coverage/capacity
HW/SW and future RAN federated and reinforcement discovery, and data request/ optimization, massive MIMO,
function split to improve learning), and inference subscription SON, CSI feedback, beam
flexibility and efficiency management, and other PHY/MAC
and upper layer improvements
37
Machine learning
is a key technology
vector on the path to 6G
Wireless AI/ML Communications resiliency
Data-driven communication and network design, End-to-end configurable security, post
with joint training, model sharing and distributed quantum security, robust networks tolerant
interference across networks and devices to failures and attacks
38
Longer term R&D direction
E2E approach with machine learning to improve 5G system performance and efficiency
39
Signal intelligence, Network intelligence Device intelligence Vertical intelligence
baseband and and system optimization and optimization and other capabilities
medium access
ML-based channel feedback Coverage and capacity optimization Digital front-end optimization High-precision positioning
Channel estimation & pilot optimization Traffic and mobility prediction Antenna and RF optimization Environmental sensing
MIMO detection Energy saving Full duplex Contextual awareness
Link prediction & adaptation Cooperative edge caching Battery saving Sensor fusion
Beam management and optimization Content-aware X-layer optimization Reflective intelligent surface Vehicular communication
Spectrum sensing and sharing Enhanced personalized security
Radio resource scheduling TCP optimization
Gray Buildings ©
2008 ZENRIN
Research face MWC demo showcasing Collaboration Brain Corp Qualcomm Power efficiency Mobile AI Enablement Center
detection with photo sorting and with Google on raises $114M Artificial Intelligence gains through compression, in Taiwan
deep learning handwriting recognition TensorFlow Research initiated quantization,
Deep-learning
and compilation
based AlexNet wins ImageNet
competition
Research in spiking Research artificial neural Opened Qualcomm Announced Qualcomm Gauge AIMET
neural networks processing architectures Research Netherlands Facebook Caffe2 Technologies ships equivariant Model Zoo
support ONNX supported CNNs open sourced
by Microsoft,
Facebook, Amazon
Acquired
EuVision
Qualcomm
2007 Opened joint
research lab
Technologies
researchers AI Model Efficiency
Investment and
Qualcomm Research collaboration with Completed Brain Corp with University Acquired win best paper Toolkit (AIMET)
initiates first AI project Brain Corp joint research of Amsterdam Scyfer at ICLR open sourced
Consistent AI R&D
Qualcomm® Qualcomm® 3rd Gen
Neural Vision Snapdragon Snapdragon
Processing Snapdragon 660 Intelligence Automotive 665, 730, Snapdragon Ride™
SDK Snapdragon 630 Platform Cockpit 730G Platform
investment is the foundation for product leadership 2nd Gen 3rd Gen 4th Gen Qualcomm 5th Gen 6th Gen
Qualcomm AI Engine Qualcomm AI Engine Snapdragon AI Engine Qualcomm® Qualcomm AI Engine Qualcomm AI Engine
(Snapdragon 835) (Snapdragon 845) 710 (Snapdragon 855) Cloud AI 100 (Snapdragon 865) (Snapdragon 888)
Qualcomm Artificial Intelligence Research is an initiative of Qualcomm Technologies, Inc.
Snapdragon, Qualcomm Neural Processing SDK, Qualcomm Vision Intelligence Platform, 1st Gen Qualcomm® AI Engine Qualcomm® QCS400 Qualcomm® Robotics
Qualcomm AI Engine, Qualcomm Cloud AI, Snapdragon Ride, Qualcomm Robotics RB3 (Snapdragon® 820 Mobile Platform) (First audio SoC) RB5 Platform
Platform, and Qualcomm QCS400I are products of Qualcomm Technologies, Inc. and/or
its subsidiaries. 42
AIMET and AIMET Model Zoo are products of Qualcomm Innovation Center, Inc.
2nd gen Qualcomm® Qualcomm AI
Sensing Hub Engine direct
Dedicated AI accelerator Easier and faster access
to the entire AI Engine
First to support TensorFlow Micro
Qualcomm® Hexagon™
Industry leading 780 Processor
AI use cases
AI
Fused AI Accelerators
• Tensor - 2X compute capacity
Super movie with Tetras.AI
• Scalar - 50% performance improvement
Snapchat lenses accelration • Vector – Support for additional data types
NLP with Hugging Face
Highlights 3X performance per watt improvement
Skin condition detection with trinamiX 16X dedicated memory
Up to 1000X hand off time improvement
in certain use cases
TVM Opensource
More efficient coding
*Compared to previous generations
43
Qualcomm Sensing Hub and Qualcomm Hexagon are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
Leading research and development
Across the entire spectrum of AI
Fundamental Applied
research research
44
Advancing AI Perception
Object detection, speech IoT/IIoT
research to make
recognition, contextual fusion
45
Quantization Relaxed Quantization Data-free Quantization AdaRound Bayesian Bits Transformer Quantization
research (ICLR 2019) (ICCV 2019) (ICML 2020) (NeurIPS 2020) (EMNLP 2021)
Quantization
open-sourcing
AIMET and AIMET Model Zoo are products of Qualcomm Innovation Center, Inc.
Compression
Quantization
TensorFlow or PyTorch
47
Benefits Features
Lower Quantization
power
State-of-the-art INT8 and Quantization-aware training
INT4 performance Quantization simulation
Post-training quantization methods,
Lower memory
bandwidth including Data-Free Quantization
and Adaptive Rounding (AdaRound) —
Higher
Visualization
performance Analysis tools for drawing insights Weight ranges
for quantization and compression Per-layer compression sensitivity
Simple
ease of use
48
Image Semantic Super
AIMET
classification segmentation resolution
Model Zoo
Accurate pre-trained 8-bit
quantized models
49
5G and AI are two synergistic, Applying AI techniques to solve Machine learning plays an
essential ingredients that are difficult wireless challenges and expanding role in the evolution of
fueling future innovations deliver new values 5G towards 6G
Nothing in these materials is an offer to sell any of the References in this presentation to “Qualcomm” may mean Qualcomm
components or devices referenced herein. Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries
or business units within the Qualcomm corporate structure, as
©2018-2021 Qualcomm Technologies, Inc. and/or its
applicable. Qualcomm Incorporated includes our licensing business,
affiliated companies. All Rights Reserved.
QTL, and the vast majority of our patent portfolio. Qualcomm
Qualcomm Hexagon and Snapdragon are trademarks or Technologies, Inc., a subsidiary of Qualcomm Incorporated, operates,
registered trademarks of Qualcomm Incorporated. Other along with its subsidiaries, substantially all of our engineering,
products and brand names may be trademarks or research and development functions, and substantially all of our
registered trademarks of their respective owners. products and services businesses, including our QCT semiconductor
business.