1

Course Introduction

Course: RO47002
Lecturer: Jens Kober
 2

Plan for today

• 13:45 – ca. 14:30: Course Introduction
– Course Information
– Robot Learning
– Types of Machine Learning
• Ca. 14:45 – 16:30
– Q&A on Installation and Practicum 1
 3

Team
• Lecturers

Holger Caesar Jens Kober

 4

Team
• TAs

Anna Mészáros Cong Wang Rodrigo Pérez Dattari Shiming Wang Zimin Xia

• SAs

Georg Strunck Runyu Ma

 5

Learning objectives
Presupposes
• Familiarity with programming (e.g. Matlab, Python), linear algebra, probability theory

After completing this course, students will be able to

• Explain the role of ML in robotics, and describe possible use cases, and when not to use ML.
• Explain the main concepts in ML
• Explain the principles of well-established ML methods
• Collect and preprocess data for ML
• Analyze a ML problem, consider available data and computational resources, and select the
appropriate ML methods to apply
• Perform experiments, evaluate the results, and draw sound conclusions
• Present the findings of a ML experiment clearly in a report using graphs and scientific English
• Write python code to visualize data and implement ML methods using popular
python frameworks, such as pandas, numpy, matplotlib, jupyter notebook, sklearn, keras
• Understand ML experiments descriptions found in scientific literature, and reproduce results
 6

Practical things:
Get the book!
– Legal digital copy
via TUD library

A bit rusty?
➢ learnpython.org
➢ https://docs.python.org/3/tutorial/
 7

Structure topic blocks

• Tuesday 18:00 - Practicum and pre-recorded
lectures released
• Friday 13:45-15:45 - Practicum Q&A
• Monday 23:59 - Practicum deadline
• Tuesday 15:45-17:45 - Discussion session

Exceptions - First and last week

 8

Flipped Classroom
• Pre-recorded lecture
– Ca. 2 hours
– Presentation of new material
• Q&A session on practica
• Discussion session
– Ca. 2 hours
– About lecture & practicum of previous week
– Classroom discussion
– Study the material before the discussion session!
 9

Practica
There are 6 obligatory practica: hands-on robotic
applications of the topics of the week
• Groups: Assignments must be made in pairs.
• Formative feedback, i.e., does not directly count
towards final course grade
• Knockout criteria: You must pass at least 4 practica.
• Deadline: Monday after the practicum is published.
The next practicum is released on Tuesday.
• No Late Submission
• No resit, valid for 2 years
 10

Final assignment
1 obligatory final assignment: integrate all the knowledge
you gained in a mini-project
• Groups: Same as practica
• Summative feedback: 30% of final grade
• Knockout criteria: You must get at least a 5
• Deadline: 13 days after the assignment is published.
• Late Submission: -1 grade point per day
• No resit, valid for 2 years
 11

Exam / Resit
• Individual
• Summative feedback: 70% of final grade
• Knockout criteria: You must get at least a 5
• Written exam: On-campus, closed-book
– Exam: Wed Nov 8th, 13:30-16:30
– Resit: Thu Jan 24th, 13:30-16:30
 12

Time investment from students

• 5 ECTS x 28 hours/ECTS = 140 hours of study
– 8 normal weeks (lectures, practica)

• Breakdown of total
– 14.5 lectures à 2h = 29 contact hours
– 8 practicum sessions à 2h = 16 contact hours
– → 95 hours self study ...

• Breakdown of week
– 140 hours / 9 weeks = 15.5 hours per week
– 2h lecture + 2h discussion + 2h practicum
= 6 contact hours per normal week
– → about 9.5 hours self study per normal week
 13

Important: get ready for practica

• Enroll on Brightspace in Collaboration → Groups
deadline September 11th
• Attend the Q&A (2nd half of lecture)
• How to find a lab partner?
– Find somebody in this lecture room
– Use the Brightspace “find a lab partner” discussion forum
– Enroll in a Brightspace group with a single person
 14

Collaboration Do’s
• For assignments you must collaborate in pairs
• Use the practicum sessions for help on the assignments

• TU Delft ethical guidelines apply to reports, code, etc.

• If your name is on it, you claim it is your work
(and not another group’s, StackExchange’s, ChatGPT’s, etc.)
 15

Collaboration Don’ts
• Do NOT copy parts of other student group’s code or ChatGPT or …
• You are NOT allowed to share your code/solutions with anybody outside
your group
• Generally, do NOT show your group’s work to other students

• Do NOT submit late

– Unless you provide in time a notice of a family or medical emergency (requires
doctor’s notice)
– Practica: no late submission (= fail)
– Final assignment: -1 grade point per day for the final assignment

• Plagiarism will be reported to the exam committee

 16

Ask for help/hints,

not for solutions!
 17

Practica submission - Do’s and Don’ts

1. Don't delete any cells.
2. Don't copy cells entirely (only the code you would
like to use).
3. Don't change the name of the notebook, and
the folder.
4. Do have look at the upload instruction video
for how to format your answer upload (you
should upload the zip file at the end of the video).
5. Do clear all outputs before uploading.
6. Do submit as a zip file (not rar, 7z, tar.gz, or
similar).
7. Do be careful when re-submitting (older
version gets irrevocably overwritten).
 18

Housekeeping Rules
• Questions outside of contact hours?
➢ BrightSpace forums

• Personal question?
➢ Email with both Jens & Holger as recipients.
 19

We need your help!

• Forth iteration, first time with Holger
• Talk to us
 20

Introduction to
Robot Learning
 21

Today’s topics
• Machine Learning in Robotics
• Understanding the goal of machine learning,
fundamental problems, high-level overview
• Supervised vs.
Unsupervised vs.
Reinforcement learning
 22

Robot Learning
 https://youtu.be/Ry8H75X0CGw 23
Zabri Zakaria

Traditional Robot Programming

 CC BY Steve Jurvetson
CC BY-SA Mixabest
What’s the problem?

CC BY-NC-SA Brandon Heyer

24
 25

What’s the problem?

• Uncertainties and variations
– Objects
– Environment
– Tasks
– Human behavior
• Occurring in all robot domains!
 26

Traditional Development Cycle

Build & Program

Study Task Test Launch
System

Analyze Problems
 27

Machine Learning Cycle

Update Data

Data

Study Task ML Test Launch

Analyze Problems
 28

Summary & Outlook

• Summary
– Robot learning
– Machine learning cycle
• Outlook
– Definition of Machine Learning
– Types of Machine Learning
 29

Types of Machine Learning

 30

What is Machine Learning?

Machine Learning is the study of computer
algorithms that improve automatically
through experience.

A computer program is said to learn from

experience E with respect to some task T
and some performance measure P, if its
performance on T, as measured by P,
improves with experience E.

—Tom Mitchell, 1997

 31

Spectrum of Learning Techniques

Reinforcement learning
(learn directly to control)

Supervised learning Unsupervised learning

(with ‘teacher’) (without ‘teacher’)
 32

Supervised Learning
• Construct a model from data
• Teacher:
Input and output given

• Types
– Regression
– Classification
Maarten Vaandrager
http://intelligent-vehicles.org/research/topics/perception-modeling/ 33
IDIAP https://youtu.be/w0in0Lr0FbU 34
 35

Spectrum of Learning Techniques

Reinforcement learning
(learn directly to control)

Supervised learning Unsupervised learning

(with ‘teacher’) (without ‘teacher’)
 36

Unsupervised Learning - Clustering

• Discover automatically
groups and structures in
data

• Applications:
– Robot perception, vision
– Data-driven construction of
dynamic models
 37

Construction of Nonlinear Models

Experimental data

Nonlinear and uncertain behavior

 38
HRI-EU
MSc thesis Radojević 2017
Clustering Objects
BBC https://youtu.be/QfPkHU_36Cs 39
 40

Clustering Jump Styles

• controlled takeoff
and landing
• stiff legged landing
• explosive takeoff
• hesitant landing

Westermann et al., Scientific reports, 2020

 41

Spectrum of Learning Techniques

Reinforcement learning
(learn directly to control)

Supervised learning Unsupervised learning

(with ‘teacher’) (without ‘teacher’)
 42

Reinforcement Learning
• Inspiration - animal learning (reward desired behavior)
Goals

reward Performance
evaluation

state

Adaptive
controller System

action
• Goal: Adapt the control strategy so that the
sum of rewards over time is maximal.
 43

Simple example
Haarnoja et al., 2019 https://youtu.be/n2gE7n11h1Y 44
Kober & Peters, 2008 https://youtu.be/cNyoMVZQdYM 45
 46

Types of Machine Learning

Supervised Unsupervised
Learning Learning
Machine
Learning

Reinforcement
Learning
 47

Model-driven vs. Data-driven

• White-box model
– Purely based on (physical) knowledge
– Derive equations, use known constants
– E.g., ridgid-body robot dynamics model based on CAD data
• Grey-box model
– Fix model structure based on prior knowledge
– Estimate parameters based on data
– E.g., estimate friction coefficients and inertia in robot dynamics model based on
data
• Black-box model
– General purpose model structure
– Rely exclusively on data
– E.g., neural network for robot dynamics model
 48

Summary & Outlook

• Summary
– General definition of machine learning
– Supervised Learning
– Unsupervised Learning
– Reinforcement learning
– Model-driven vs. data-driven
• Outlook
– Fundamental problems
– Machine learning formalized