Design Industrial Robot Applications from

Perception to Motion
 Authorized
As the Sole Reseller in Southeast Asia for MathWorks Inc,
Distr ibuto
developer of the MATLAB® and Simulink® family of products since
1996, we provide organizations and businesses with a wide spectrum
of the best tools, products and services to facilitate innovations.

2
 DATA QUADRANT AWARDS 2022
Machine Learning

MATLAB® is the enterprise engineering platform for

AI.
•Empower your team, including those with limited AI or data science
experience
•Apply complete workflows for data preparation, AI modeling, system
design, and production
•Deploy AI models on embedded devices, edge, enterprise systems, and
the cloud
•Tackle integration challenges and reduce risk in designing AI-driven
systems with Simulink®

*Top Machine Learning Software Awards 2022 | Software Reviews 4

 Our Solutions

Image
Artificial Wireless Computational Processing & Predictive Machine
Control Systems
Intelligence Systems Biology Computer Vision Maintenance Learning

Robotics
Robotics Power Electronics Internet of Power System Signal
Data Science Mechatronics
Control Design Things (IoT) Analysis & Design Processing

Automated Test & Embedded FPGA, ASIC & SoC Mixed Signal Enterprise &
Driving Systems Deep Learning Measurement Systems Development System IT Systems

5
Agenda

▪ Introduction
– Pick-and-Place Manipulation Application
▪ Trends in Industrial Robotics
– Landscape of AI adoption in Robotics
– Why Use AI for Robotics?
▪ Autonomous Robotics System Development
– How to get data for training & labeling
– AI modeling
– AI within entire systems
– Deployment
▪ Robotics and Controls Made Easier
▪ Resources for further learning

7
Autonomous Robot Development with MATLAB & Simulink

Platform design tools for industrial

robots

Tools for autonomous industrial

robot applications

Deployment and hardware interface

with MATLAB and Simulink

Integrated workflows enabled

by MATLAB and Simulink

8
Trends in Industrial Robotics

Factories are:
• Incorporating more advanced systems
• Taking advantage of new technology
• Increasing connectivity between components
• Enhancing infrastructure to reach Industrial
IoT

Flexible manufacturing and autonomous production

Collaborative Robots
AI-based Robots

9
What we will discuss today
01
Advanced Robots in
Smart Factory
Use case: Pick-and-
03
Place Manipulation
Application using
MATLAB and Simulink
Autonomous Robotics
02
System Development
• Platform Design & Simulation
• Autonomous Application
• Hardware connection & deployment
04
Concluding Remarks
10
 Smart Factory Concept: Asset, Operation, & Workforce Optimization

IoT
Factory Cloud + AI

Local 5G Operation
Big Data
Process Management・Quality Increase Management

Robot Optimization
✔?
✘?
OT
Worker AMR
Optimization

Autonomy Monitor
Update
11
 Smart Factory Concept: Asset, Operation, & Workforce Optimization
What we are talking about today

IoT
Factory Cloud + AI

Local 5G Operation
Big Data
Process Management・Quality Increase Management

Robot Optimization
✔?
✘?
OT
Worker AMR
Optimization

Autonomy Monitor
Update
12
 Core of autonomous system design

Sense & Plan &

Control
Perceive Decide

13 13
Core of autonomous system design

Sense & Plan &

Control
Perceive Decide

Computer vision /
Mission planning Motion control
Image processing

Localization, Path tracking

mapping / SLAM Motion planning

Sensor fusion and Behavior planning Control logic

tracking

Deep learning Reinforcement

networks Learning

14
But there are challenges…..

Too much data Incomplete No data Poor ROI

tools scientists
Problem is a
poor fit for AI
Can’t integrate
Not enough No AI
with other
data expertise Problem is
systems
unsolvable

15
 Enables
Robotics AI

Full Autonomy
• AI in robots increase the capabilities
High Autonomy AI-powered robots of robots in performing complex
tasks
Conditional
Autonomy
• Cobots combine the individual strengths
Cobots of humans and robots for flexible and
Partial Autonomy
agile manufacturing

Human • Not all robots are artificially intelligent

Assistant
- Automation
Traditional Industrial • Non-intelligent robots are mostly
Robots restricted to performing a repetitive
series of movements
No Automation

16
There are two ways to get a robot to do what you want

Traditional Programming

Data

Output

Program

17
There are two ways to get a robot to do what you want

Data

Program

Output

18
There are two ways to get a robot to do what you want

Data

Model

Output

19
 There are two ways to get a robot to do what you want

Machine Learning
Artificial Intelligence

Deep Learning Data

Model

Reinforcement
Learning Output

20
Autonomous Robot Development with MATLAB & Simulink

Autonomous
Platform C/C++ Code
Applications
Perception Environment
ROS Node

GPU Code
Planning
Robot Model

Control

Development Environment
Design Simulate Analyze Implement Test
21
Autonomous Robot Development with MATLAB & Simulink

Autonomous
Platform C/C++ Code
Applications
Perception Environment
ROS Node

GPU Code
Planning
Robot Model

Control

Development Environment
Design Simulate Analyze Implement Test
22
 Physical & Kinematic Modeling

Creating a physical model Automatic import from CAD Tools URDF / Robot Library
%% Import robot from URDF
smimport('GEN3_FOR_URDF_ARM_V11_MOD.urdf');
CAD Model

Multibody
Model
%% Use robot library
Robot = loadrobot('abbYuMi’);
Show(robot);

Simscape Multibody, Robotics System Toolbox 23

Landscape of AI adoption in Robotics
❑Computer Vision
Object Identification for
Pixels Deep Learning Labels
Robots

24
Landscape of AI adoption in Robotics
❑Computer Vision
Image Industrial Inspection
Pixels Deep Learning
Abnormal with Robots / UAVs

25
Landscape of AI adoption in Robotics
❑Point Cloud Processing
Point Pose Estimation
Deep Learning Structure Localization
Clouds for Robots

26
 Landscape of AI adoption in Robotics
Sense & Plan &
Control
Perceive Decide
Voice Command to
Robots

Object Identification for

Robots

Industrial Inspection
with Robots / UAVs

Pose Estimation
Localization
for Robots

27
 Landscape of AI adoption in Robotics
Sense & Plan &
Control
Perceive Decide
Voice Command to
Robots

Object Identification for

Robots
Adopting
Industrial Inspection Deep Learning for
with Robots / UAVs
Perception

Pose Estimation
Localization
for Robots

28
Landscape of AI adoption in Robotics

❑Learn a behavior or accomplish a task through trial & error

Reinforcement Learning

Look-up table

Interactions / Make Decision

Polynomials Action
Observations
(Deep) neural
networks

Control
29
Landscape of AI adoption in Robotics

❑Learn a behavior or accomplish a task through trial & error

Sense & Plan &

Control
Perceive Decide

Exploring
Make Decision
Reinforcement
Learning for
Making Decision &
Control

Control Simulation and Virtual Models are a Key

Aspect of Reinforcement Learning!
30
Landscape of AI adoption in Robotics

• Voice recognition for service robots

• Deep learning for part identification and
Adopting fault detection
DL for Perception • Aerial/satellite image classification
• …

• Navigation and motion control in

Exploring autonomous systems
RL for Making • Robotics process automation
Decision & Control • …

31
Case study: developing pick & place robot arm

32
AI-driven robotics system
design workflow

Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
AI-driven robotics system
design workflow

Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
 How to get data for training?

Data
1
Preparation • •
AI
• •
2 • •
Modeling

Simulation
3
and Test

4 Deployment

Synthetic data generation with simulator Data acquisition with hardware

35
 Generate synthetic data to improve your datasets

Data
1
Preparation Semantic Segmentation from
Unreal for UAV
AI UAV Toolbox
2
Modeling Simulink Lidar Sensor Model: Simulate
lidar sensor and generate
Simulation
3 point cloud data
and Test
Lidar Toolbox Gazebo Co-simulation with a
Pretrained Deep Learning
4 Deployment Model to Detect Recyclable
Parts
Robotics System Toolbox
ROS Toolbox

36
 Automated labeling apps save you weeks to months

Synthetic data generation

Train object detector
Data with CAD models
1
Preparation

AI
2
Modeling

Simulation
3
and Test

4 Deployment

Manually correct Automated labeling with

wrong labels actual images
37
AI-driven robotics system
design workflow

Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
 Start with a complete set of algorithms and pre-built models

Object Detection
with YOLOv4

Instance Segmentation
with Mask R-CNN
Data
1
Preparation
Semantic Segmentation
2
AI with U-Net
Modeling

Simulation
3
and Test

4 Deployment

39
 AI modeling Apps automate training, tuning, visualization...

Data
1
Preparation
Deep Network Designer app to
AI build, visualize, and edit deep Classification Learner app to
2
Modeling learning networks. try different classifiers and find Experiment Manager app to
the best fit for data sets. run deep learning experiments
Simulation to train networks and compare
3
and Test results.

4 Deployment

40
 Access AI models from the broader AI community​

TensorFlow Importer
Data
1
Preparation Caffe Importer
AI
2
Modeling

Simulation
3
and Test

4 Deployment Other
Frameworks

41
AI-driven robotics system
design workflow

Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
 Use AI within entire systems Robot Platform
(Simulation or
Real HW)

Sensing

Data AI model
1
Preparation
Perception
AI
2
Modeling

Simulation
3
and Test
Control
4 Deployment
Planning

43
 Use AI within entire systems

Data AI model
1
Preparation

AI
2
Modeling

Simulation
3
and Test

4 Deployment

44
 Use AI within entire systems

Data AI model
1
Preparation

AI
2
Modeling

Simulation
3
and Test

4 Deployment

45
AI-driven robotics system
design workflow

Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test
 AI models are useful everywhere

Data
1
Preparation

AI
2
Modeling

Simulation
3
and Test

4 Deployment

47
 Deploy to any processor with zero coding errors

CPU

Data
1
Preparation Code
Generation GPU
AI
2
Modeling

Simulation
3
and Test

4 Deployment FPGA

48
 Deploy to Jetson device as CUDA ROS node

Other ROS
nodes

CUDA ROS node

Data
1 CUDA
Preparation
ROS Code .cpp CMakeLists.txt

AI Generation
2
Modeling

Simulation
3
and Test

4 Deployment

NVIDIA ® Jetson™
49
Key Takeaways

Advanced robotics systems in the factory of the future

 Collaborative Robots
 AI-enabled Robots

Three pillars to develop autonomous robotics systems

 Platform design with environment models
 Autonomous application design
 Deployment

MATLAB and Simulink as a unified development environment

 Develop autonomous robotics applications from perception to motion
 Hardware interface

50
Why Use AI for Robotics?

AI helps AI helps AI helps AI

robots robots robots powered
to sense to think to act robot

• Natural Language • Reasoning • Robotics process • Pick and place robot

• Machine Vision • Planning and automation and • Welding robot
• Detection and scheduling control • Assembly robot
identification • Collaborative and • Agri AI robot
adaptive system
• Service robot
…

51
 Why MATLAB and Simulink for AI in Robotics?
Data AI Simulation
1 2 3 4 Deployment
Preparation Modeling and Test

▪ Create more and ▪ Use a compete set of ▪ Integrate AI into ▪ Make deployment
better data sets AI algorithms and pre- system-wide context, of AI models easy
▪ Speed up data built models simulate, and verify to accelerate AI
labeling effectiveness system

Empower robotics domain experts, including ones with limited AI experience

52
Get Started with AI in MATLAB
No download, no installation, just your browser and you…

53
Industry application examples – User Stories / Presentation

Robotic Arm for Removing Verification of Picking System for Autonomous Humanoid Robot with
Nuclear Fuel Debris Industrial Robot Using ROS and Model-Based Design
Mitsubishi Heavy Industries MATLAB German Aerospace Center (DLR)
Yaskawa Electric Corp. Robotics and Mechatronics Center

54
54
Resources
Examples Videos Webinars

Deep Learning Tech Talk AI for Simulink Users

Reinforcement Learning
Tech Talk Deploy Deep Neural Networks

55
Line official to get a trial license...

www.techsource-asia.com/events /ascendassystems /techsourcesystem

56