STM32 Artificial

Intelligence Solutions

MDG Korea
문현수 과장
 A new way to add environment awareness
to your products
From rule-based engineering to data-driven engineering

Standard programming Machine Learning

Handcrafted rules based on experience Rules learnt from real-world data

Input Data Input Data

▪ Requires domain ▪ Generate code from
expertise to code Desired output
real-world observations
Rules
▪ Need to rewrite if from the system ▪ Re-learn from data if
adapt the environment evolves collect environment evolves
rules more data

Desired output
Rules
from the system

2
 Distributed Artificial Intelligence approach

Leverage billions of devices at the Edge!

Data Center Cloud

Thousands Analytics, storage, compute

Edge Nodes
Millions IoT gateways, micro datacenters

Ultra-low-power Edge Things

100 Billions devices and sensors Real time, local processing

3
 Artificial intelligence at the deep edge
Moving part of Artificial Intelligence closer to the data acquisition
brings several benefits

Better user Optimized Cloud

experience usage

No latency Privacy by design

(real-time) (GDPR compliant)

Sustainable on
More reliability Add new functions and energy
services with Embedded AI

4
 STM32 comprehensive AI ecosystem
Applications Frameworks

Applicative
Examples
(Function Packs)

AI Model convertor
Pre and post
Processing Graph Memory
Quantizer
libraries optimizer optimizer

Edge Hardware

STM32 series Discovery kit STM32 Nucleo board Camera add-on

5
STM32 Cube.AI
 For teams with AI expertise
Accelerate your embedded development

Convert models into optimized

Train your own AI models Deploy on optimized runtime
code
1 2 3

Neural Machine
And more… Networks Learning

Iterate fast and increase productivity

7

Graph optimizer
Automatically improve performance through
graph simplifications & optimizations that
benefit STM32 target HW architectures
• Auto graph rewrite
• Node/operator fusion
• Layout optimization
• Constant-folding…
• Operator-level info to fine-tune
memory footprint and computation
STM32Cube.AI is free of charge, available both in graphical interface and in command line.
The 3 pillars of STM32Cube.AI
Quantized model support
Import your quantized ANN to be compatible
with STM32 embedded architectures while
keeping their performance
• From FP32 to Int8
• Minimum loss of accuracy
• Code validation on target
o Latency
o Accuracy
o Memory usage
Memory optimizer
Optimize memory allocation to get the best
performance while respecting the constraints
of your embedded design
• Memory allocation
• Internal/external memory repartition
• Model-only update option
8
 STM32Cube.AI
Get the best performance on STM32
Image Classif v0.5 Visual Wake Word v0.5

325
KB

96
KB

77
KB 214
KB
61
53
49 KB HW Target: STM32H723
KB
Flash: 1Mbyte
37 RAM: 564 Kbytes
98
KB Freq: 550 MHz

SW Version:
42 37 X-Cube.AI v 7.0.0
31
KB TFLm v2.5.0
Latency (ms) Flash RAM Latency (ms) Flash RAM
* the lower the better

9
 Making deep edge AI accessible to all portfolio
All STM32 are compatible with NanoEdge™Studio ecosystem

STM32 compatible with STM32Cube.AI ecosystem (Extension down to M0 since 7.1.0 version)

STM32MP1
4158 CoreMark
MPU 800 MHz Cortex -A7
209 MHz Cortex -M4

STM32F2 STM32F4 STM32H7 STM32F7

High Perf 398 CoreMark 608 CoreMark 3224 CoreMark 1082 CoreMark
MCUs 120 MHz 180 MHz 240 MHz Cortex -M4 216 MHz
480 MHz Cortex -M7

Mainstream STM32F0 STM32G0 STM32F1 STM32F3 STM32G4

106 CoreMark 142 CoreMark 177 CoreMark 245 CoreMark 550 CoreMark
MCUs 48 MHz 64 MHz 72 MHz 72 MHz 170 MHz

Ultra-low Power STM32L0 STM32L1 STM32L5 STM32U5 STM32L4 STM32L4+

75 CoreMark 93 CoreMark 424 CoreMark 651 CoreMark 273 CoreMark 409 CoreMark
MCUs 32 MHz 32 MHz 110 MHz 160 MHz 80 MHz 120 MHz

Wireless STM32WL STM32WB

161 CoreMark 216 CoreMark
MCUs 48 MHz 64 MHz

Arm® Cortex® core -M0 -M0+ -M3 -M33 -M4 -M7 dual -A7& -M4 10
 STM32Cube.AI
The STM32CubeMX expansion pack for ML
Power Consumption
MCU Selector
Calculator

Code Generation Pinout Configuration

Middleware Parameters Clock Tree Initialization

Peripherals Configuration

11
 Integrate your ML models more easily
with our application-oriented code examples
Time series-based
Audio and sensing Computer vision Face recognition
monitoring

FP-AI-MONITOR1 FP-AI-SENSING1 FP-AI-VISION1 FP-AI-FACEREC1

• Predictive maintenance and • Human Activity Recognition • Food recognition (CNN) • Face detection and
much more sensor- • Acoustic Scene • Person presence detection recognition
monitoring apps Classification (CNN) • Fully functional without
• Runs Libraries from • Data logging, labeling and • People counting (Object cloud connection
NanoEdge AI Studio result on BLE applications detection NN)
• Image processing Library

12
 We provide everything to kick off your project
Design documentation Hardware and software tools Support & Updates

Getting started Development zone

Be guided step-by- Get started on
step to learn application
STM32 ecosystem development and
project sharing

• Wiki by ST is a great forum to learn and • Evaluation platforms for STM32 • ST Community: STM32 ML & AI group
start developing AI on STM32! MCU/MPU • Distributor certified FAE
• Videos of application examples • Extra sensor boards • Support center
• Massive Open Online Course (MOOC) • Full software suite • Newsletter

13
STM32Cube.AI Deeper dive

Simple, fast, optimized

 The key steps behind Neural Networks
Neural Network (NN) Model Creation Operating Mode

Process & analyze new

Capture data Train NN Model data using trained NN

2 4

1 3 5
Clean, label data Convert NN into
Build NN topology optimized code for MCU 15
 STM32Cube.AI main features
STM32Cube.AI is available both as graphical and command line interface

✓ Quickly assess model footprint requirements

✓ Select and configure MCU in STM32CubeMX
Dimension ✓ Review model layers in STM32Cube.AI

✓ Generate C-code for pre-trained model

✓ Support quantized models to reduce RAM, flash
Optimize and latency with minimal loss of accuracy
✓ Use light run-time libraries
✓ Optimize for performance

✓ Optimize memory allocation

Fine Tune ✓ Fine control of weight mapping
✓ Split between internal and external memory
✓ Update model without full FW update

And quickly iterate thanks to on-target validation 16

 STM32Cube.AI user flow (1/3)

• Load model and select an AI runtime

1 Select MCU • Analyze minimal required footprint
• Select corresponding STM32 MCUs

2 Optimize and validate

Generate project and

3
integrate

17
 STM32Cube.AI user flow (2/3)
• Model complexity and footprint analysis
• Fine tune memory allocation with optimizations and GUI
• Optimize system parameters and clock tree
• Extend model with your own customer layers
1 Select MCU

2 Optimize and validate

Generate project and

3
integrate
• Validate on desktop with your own dataset
• Validate on target and check inference time

18
 STM32Cube.AI user flow (3/3)

1 Select MCU
• Generate Application Template
• Integrate with your application-
specific code in your favorite IDE
• Perform system tests
2 Optimize and validate

Generate project and

3
integrate

19
 Fine-tuning memory allocation
Model RAM consumption can be
displayed layer per layer Model memory allocation is easily controlled
✓ Easily identify most critical layers ✓ Make use of external memory based on model
requirements and application footprint
✓ Mapped in non-contiguous internal flash section
✓ Mapping partially in internal and external flash memories

Model input buffer can be re-used to

store activation at run-time
Enable relocatable network ✓ Minimize RAM requirements
✓ A separate binary is generated for the library and the ✓ Possible if input and activation buffers are in
network to enable model upgrade same memory

20
 Possible conversion strategies:
Network code generation and interpreter
More Flexible: More optimized:
TensorFlow Lite interpreter mode Optimized C code generated by

Flat buffer NN C files

.tflite
converter converter STM32.AI lib
Pre-trained Pre-trained
model model

Flat buffer .tflite NN C files

Model is Model is pre- User app
interpreted and User compiled and STM32.AI lib
TFLite interpreter
app
executed by built-in custom linked only with
STM32 BSP
pre-built ops ops ops used ops
STM32 BSP STM32 device

STM32 device

run-time

run-time on
21
 STM32Cube.AI with ML extension

Leverage NN and ML in a unique tool optimized for STM32

Train Neural Network using Train ML model using SciKit

any major AI frameworks Learn

▪ Efficiently develop your ML

solution on STM32
▪ Add ML layers to
Export to complement Neural
Network model
And more
▪ Implement pre-processing
steps based on ML

22
 STM32Cube.AI with ML extension

Classification Regression Clustering Dimensionality reduction

Preprocessing Model selection

23
NanoEdge AI Studio
 NanoEdge AI Studio V3
Same easiness, more power
ON THE PC ON THE MCU

1 Create the library, ONCE. 2 Use the library, MANY TIMES.

Create and embed

a self learning engine

Standalone PC (Win/Linux) solution For anomaly detection, the model is

self-trained at the Edge.

25
 New user interface,
more functions, better user experience
New families of Machine New Datalogging
Learning algorithms experience

Get inspired by
Multiple uses cases
Easily retrieve or create
projects

26
Our customers have increasingly ambitious use cases
for ever smarter products
"My pumps are installed in a variety of environments that I can't
anticipate.
I want them to autonomously adapt to their target environment
and detect anomalies by themselves."

"I know exactly how my pumps behave.

I want to detect any outliers."

"I know the signals when a pump is experiencing, for example, ball
bearing or cavitation problems.
I want to know by name what problems are occurring.“

“I know several vibration values of my machine.

I want to anticipate when a specific vibration level will be
reached so that I have time to take corrective actions before
reaching that limit.”
27
 Anomaly detection use-case
ON THE PC ON THE MCU

1 Creation of an ANOMALY DETECTION 2 Use of an ANOMALY DETECTION

Machine Learning library Machine Learning library

Infer Infer

Model A Model B

ML library
Contextual
Signals Learn Learn

28
 One class classification use-case
ON THE PC ON THE MCU

1 Creation of a ONE CLASS CLASSIFICATION 2 Use of an ONE CLASS CLASSIFICATION

Machine Learning library Machine Learning library

Infer

Embedded
Static
Model

Normal ML library
Condition
Signal

29
 n Class classification use-case
ON THE PC ON THE MCU

1 Creation of a n CLASS CLASSIFICATION 2 Use of an n CLASS CLASSIFICATION

Machine Learning library Machine Learning library

Bearing
Problem
Signals Bearing 90% Bearing problem
Misalignment 0%
Misalignment Cavitation 3%
Problem Shaft Imbalance 7%
Signals ML library

Cavitation Classification
Problem
Signals

Shaft imbalance
Problem signals

30
 Extrapolation use-case
ON THE PC ON THE MCU

1 Creation of an EXTRAPOLATION 2 Use of an EXTRAPOLATION

Machine Learning library Machine Learning library

SPEED 10%
Signals
Vibration level 80% Vibration level 87%
Vibration level 65%
SPEED 25%
Vibration level 25%
Signals Vibration level 10%
ML library

SPEED 65% Classification

Signals

SPEED 80%
Signals

31
 From idea to datalogging
in a matter of minutes
• Streamlined data logging process
• No code
• All settings done using a graphic interface

The STWIN SensorTile wireless industrial node (STEVAL-

STWINKT1B) is a development kit and reference design
that simplifies prototyping and testing of advanced
industrial IoT applications such as condition
monitoring and predictive maintenance

The kit features a core system board

with a range of embedded industrial-
grade sensors and an ultra-low-power
microcontroller

32
 How NanoEdgeTM can grow your business

• Full AI prototype for less than 3-month data scientist salary

Save time, resources • No AI experts required
& money • Works on M0+
• Fast time to market: your AI solution in a few weeks

• Set yourself apart from competition

Raise customer
• Position your company as an innovative business
satisfaction & loyalty
• Improve level of service or offer a new service

33
 And many more

https://DATA.CARTESIAM.AI 34
STM32 EVK Board for AI

Running AI on ST
Microprocessors
 Example form factor hardware
to capture and process data
SensorTile

Capture Data Inference on STM32L476

STM32L4

Motion MEMS Motion MEMS

www.st.com/SensorTile
36
www.st.com/SensorTile-edu
 Fast go to market module
to capture data with more accuracy
SensorTile.Box

Capture Data Inference on STM32L4R9

Microsoft IoT
Services ready

More advanced, high accuracy and low power sensors

• First Inertial module with Machine Learning capabilities.
• Motion (accelerometer and gyroscope, magnetometer) and slow motion (inclinometer)
• Altitude (pressure), environment (pressure, temperature, humidity, compass) and sound
(sound and ultrasound analog microphone)
• Microsoft IoT services ready to make available on a web dashboard the result of the
embedded processing
37
www.st.com/SensorTileBox
 Form factor hardware
AI IoT node for more connectivity
B-L475E-IOT01A
Sub-1GHz Dynamic NFC Tag
+ Wi-Fi
Capture Data Inference on STM32L4

More debug capabilities

• Integrated ST-Link/V2.1
• PMOD extension connector
• Arduino Uno extension connectors

38
www.st.com/IoTnode
 Wireless Industrial node
to capture data at industrial grade
STWIN

Capture Data Inference on STM32L4R9

Industrial-grade sensors
• Industrial scale 9-DoF motion sensors including accelerometer, gyrometer
and an ultra wide-bandwidth vibrometer with ultra low noise
• Very high frequency audio and ultrasound microphone
• High precision temperature and environmental monitoring
• Micro SD card for standalone data logging
• BLE5.0 connectivity and WiFi expansion board
• USART
www.st.com/stwin 39
 STM32H7 discovery boards with camera
STM32H747I-DISCO
with B-CAMS-OMV

Capture Data Inference on STM32H747

Computer Vision on microcontroller

• STM32H747 high-performance and DSP with DP-FPU, Arm Cortex-M7 at
480 MHz + Cortex-M4 MCU with 2MB internal Flash, 1MB internal RAM,
Chrom-ART Accelerator
• External memory 2x64MB Quad-SPI NOR Flash and 32MB SDRAM
• 4” capacitive touch LCD display module with MIPI® DSI interface
• Camera module adapter board and camera module based on OV5640
5MPx 8b color rolling shutter
• ST-MEMS digital microphones
• Ethernet RJ45 and WiFi / cellular expansion boards 40
 OpenMV integration
Fast machine vision prototyping

Configure Machine Vision in

real-time over USB in Python

Run and validate optimized

Neural Network
OpenMV CAM
Running MicroPython over STM32

41
https://github.com/openmv/openmv
AI Application demos

Running AI on ST
Microprocessors
STM32 AI Application demos

43
FP-AI-SENSING1
Human activity motion recognition (HAR)

5 classes example
Motion Data Capture Labelling controlled Data stored on the device Stationary, walking, running,
by smartphone application SD card for future learning biking, driving labelling

NN & example
dataset provided

Embedded motion Inferences running Inference result

pre-processing on the microcontroller displayed on mobile app
44
 Human Activity Recognition

Classify human activity in wearables

Use cases Demo overview

• Adapt behavior based on user’s activity • Activity is displayed in BLE mobile app
environment: switch peripherals to low • Ultralow-power always-on Human Activity
power modes exploits 3-axis accelerometer data
• Basic heath assistant measuring time • 5 classes: stationary, walking, running,
spent moving per day biking, driving
• Pre/Post-processing: filtering gravity,
reference rotation, temporal filter
Smartwatch enables BLE
when user is driving
SensorTile.Box

Sport watch monitor

time spent on sport Neural Network on STM32L4
activities

45
FP-AI-SENSING1
Audio scene classification (ASC)

3 classes
Audio Data capture Labelling controlled Data stored on the device Indoor, Outdoor, In vehicle
by smartphone application SD card for future learning labelling

NN & example
dataset provided

Embedded audio Inferences running Inference result

pre-processing on the microcontroller displayed on mobile app
46
 Acoustic Scene Classification

Classify acoustic scene

Use cases Demo overview

• Adapt behavior based on user’s • Exploits 16kHz microphone data
environment: switch peripherals to low • 3 classes: indoor, outdoor, in-vehicle
power modes • Pre-processing: LogMel spectrogram on-
• Detect time spent driving per day board

Smartwatch turns off WIFI

when not indoor

SensorTile.Box

Sport watch turns GPS

when outdoor
Neural Network on STM32L4

47
► Demo available on FP-AI-SENSING1
 Handwritten character recognition

Character recognition on touchscreen for wearable HMI

Use cases Demo overview

• Neural Network for wearables HMI • Handwritten character recognition from
improvement touch screen panel
• Each character or word can be • Screen panel screenshots 32x32
associated to actions images provided to the Neural network

Access menus with

natural machine interface STM32L5 DK
STM32L496 DK
Control functions
with handwritten STM32L4
shortcuts
Neural Network on STM32L5

48
 Computer Vision for STM32

Give vision to your STM32 product for

new features and add-on services And more
User 2

Optimized NN Customer
files application
Food Person presence Face People
classification detection recognition counting STM32.Vision
STM32.AI lib
lib
FP-AI- FP-AI- FP-AI- FP-AI-
VISION1 v1.0 VISION1 v2.0 FACEREC v1.0 VISION1 v3.0
run-time
49
 Food classification

Classify live image stream in real-time

Use cases Demo overview

• Classify image content to trigger custom
actions or content-specific notifications
• Runs in medium FPS allowing real-time
actions
• Validated with RGB camera, can be
extended to other types
• Camera acquisition to image pre-
processing before feeding the NN
• Multiple memory mapping possibilities
to optimize / test impact on perf
• Quantize your trained network to
optimized inference time and memory
usage

Route packages in supply

chain based on their STM32H747 Discovery
standardized appearance + F4DIS-CAM

Sort waste and Neural Network on STM32H7

assign to correct
containers
50
 Facial expression recognition

Live face expression recognition from a camera video stream

Sad (85%) Disgusted (82%)
Use cases Demo overview
• Build interactive experience in toys • Multiple face detection with labeled
• Optimize advertising experience based bounding box displayed in real-time
on customer emotional response • Recognize 7 expressions: angry,
• Only a low-resolution camera is required disgusted, scared, happy, neutral,
sad, surprised
• Processing time: 4.8 FPS for 1 face,
3.4 FPS for 2 faces
Assert customer interest at Surprised (92%) Happy (91%)
point of sales, like vending STM32H747 Discovery
machines + VG5661 (or F4DIS-CAM)

Improve toys Neural Network on STM32H7

reaction based on
user’s expression
51
 Person presence detection

Detect person presence even without movement

Use cases L4R Demo overview

• Visual wake word for Smart home or
cities security cameras
• Reduce false alarms due to object
movement detection
Trigger alarm​ on person
detection instead of
movement​
Visual wake word
for more advanced
person identification
• Person Presence Detection model
from tensorflow.org​ (L4R and H7) and
MobileNet v2 (H7 only)
• QVGA 320x240 color image on the
LCD
• Can adapt camera flipping depending
on which side camera is placed
STM32L4R DK
STM32F4DIS-CAM​
Neural Network on STM32L4R

52
 Anomaly Detection using vibration

Detect anomaly in industrial equipment using monitoring device

Use cases L5R Demo overview

• Key component of condition monitoring system
for industrial equipment with rotating parts:
motors, fans, pumps, bearings, spindles…
• Detect anomalies in complex system based on
3-axis accelerometer sensor
• No training on abnormal state is required, on-
chip anomaly detection is done in real-time
Real-time detection of
anomalies of industrial
pumps
• Function Pack provides logging capabilities,
display and local control via USB CLI.
• NanoEdge AITM Studio by Cartesiam to
generate custom machine learning library
able to learn and infer on STM32
• Learn normal state on STM32 during onsite
initialization phase, then detect anomalies
from vibration
Motor Control Kit G4
STM32L5R Discovery Kit

Real-time detection of
shocks, misalignment Neural Network on STM32L5R
or motor unbalance on
industrial appliances

53
 Multiple object detection on STM32MP1 MPU
Advanced object detection among 90 different objects using
TensorFlow Lite on STM32MP1
Use cases Demo overview
• Detect object and its position. • integrated via C++
• 90 objects can be detected runtime implementation on dual-core A7
• Object detection is done in real time for • COCO ssd MobileNet v1: 90 classes
fast interaction with user • CPU load balanced on the 2 cores
• Requires only a low resolution camera • Processing time: 1.1 FPS

Identify and locate

potential instances Arrow Avenger96 board
of plant disease Parallel interface Camera Display

Neural Network on STM32MP1

54
 Aftermarket wireless digit reader for metering

Equip meters with aftermarket Wireless & Low power reader

Use case Demo overview

• Equip meter with ad-on SPI low-cost • QQVGA input @4fps
camera • Proprietary Neural Network trained on
• Boost ROI avoiding onsite visit SCUT WMN dataset
implementing long range wireless ▪ Accuracy: 95%
reader ▪ Inference 84 ms / digit
• Electrical, gaz, water meters supported ▪ Flash: 16 kB, RAM : 8 kB

• Reading sent over LoRAWan • LoRaWAN stack on LoRa SoC

▪ Flash: 65KB, RAM: 6KB
STM32WL55JC
Arducam mini 5MP plus

Neural Network on STM32WL

55
Why choose STMicroelectronics?
A complete ST AI solution stack

Simple, fast, optimized

 A full development ecosystem
Create your AI application on STM32
Various application examples to
get started rapidly
Computer Vision MCU
applications
FP-AI-VISION1
Data processing libraries to
speed up development
Audio and Sensing MCU
applications
FP-AI-SENSING1 Step-by-step guides to extend
examples or benchmark ML
Time series-based MCU performance
monitoring applications
FP-AI-MONITOR1 STM32 Community with
dedicated Neural Networks topic
AI frameworks on STM32 and AI expert partners
Microcontrollers
X-LINUX-AI
Trainings, hands on, MOOCs and
partners videos

57
 Whatever your company's profile,
you will find an AI solution suited to your needs
USE CASES
COMPANY’S PROFILE
Anomaly detection Classification Deep Learning

Engineering
Services
Embedded developers
▪ No dataset available
▪ No dedicated AI Team

Team with AI expertise

▪ Dataset available
▪ AI Team

58
 Don't go alone

A network of companies is there to support you

Trust our authorized partners to ensure the success of your

project. Learn more at st.com/stm32ai

Would you like to discuss a co-development partnership

for ML/AI projects? Contact us at edge.ai@st.com

59
Releasing your creativity
/STM32

@ST_World

community.st.com

www.st.com/STM32ai

wiki.st.com/stm32

github.com/STMicroelectronics

Videos

STM32Cube.AI blog articles

60
 Find out more at www.st.com

© STMicroelectronics - All rights reserved.

ST logo is a trademark or a registered trademark of STMicroelectronics International NV or its affiliates in the EU and/or other countries.
For additional information about ST trademarks, please refer to www.st.com/trademarks.
All other product or service names are the property of their respective owners.