Université d’Ottawa University of Ottawa

Faculté de génie Faculty of Engineering

Département de Department of
génie mécanique Mechanical Engineering

ELG7113/MCG5470: Machine Learning for Adaptive and Intelligent Control Systems

Winter 2024
Last update: January 9, 2024

Lectures: Lectures are delivered in person in the classroom stated below. Additionally, they are recorded in MS Teams,
as an option for the students that occasionally may not be able to attend. See Virtual Classroom below for more
information.
Classroom: 145 Jean-Jacques Lussier (LMX) 219
Schedule: Wed 2:30 PM - 5:20 PM
Virtual Classroom: Recorded lectures and some tutorial material will be available in MS Teams. You should have
access to a Team for MCG5470/ELG7113 (link). The course’s Team also hosts a OneNote class notebook, which
you should consult after each lecture if you want to access the instructor’s class notes.
One lecture meeting is scheduled in Teams for the duration of the semester, occurring on Wed between 2:30 PM -
5:30 PM.
Instructor: Dr. Davide Spinello
email: dspinell@uottawa.ca
office: CBY A612
phone: 613.562.5800 ext. 2460
office hours: Take an appointment by email
Teaching Assistant: Mohammad Savari
email: mohammad.savari@uottawa.ca

Course Outline
Linear Feedback Structures. Effects of Process Variations. Adaptive Schemes. Recursive Least Squares and Regression
Models. Self-Tuning Regulators. Model Following. Model-Reference Adaptive Systems. Lyapunov Stability Theory.
Output Feedback. Bellman’s optimality. Dynamic Programming. Reinforcement Learning and Q-Learning. Temporal
Difference Learning. Approximate Dynamic Programming. Adaptive Critics. Multi-Agent Adaptive Reinforcement
Learning Systems. Integral Reinforcement Learning.

Course pre-requisites
Theoretical: Typical undergraduate linear control and mathematics backgrounds are sufficient.
Coding: Basic familiarity with any language (examples: Python, C, C++) and/or a scientific computing environment
(examples: Wolfram Mathematica, Maple, SageMath, GNU Octave, Matlab) as tools for homework assignments,
project, and final exam. In this course, coding is applied, and not the focus.

Course description and expected outcomes

• The course covers contemporary concepts related to machine learning, model-reference control, system identification,
and optimal control systems, which are needed in particular to understand and implement modern adaptive learning
control paradigms.
• It teaches students different techniques on how to control dynamical systems with various degrees of uncertainties
using analytical as well as machine learning processes.

ELG7113/MCG5470 - Winter 2024 Page 1 of 4 Syllabus

 • The course prepares the students to conduct graduate-level research in the fields of dynamics and modern control
engineering, covering practical and theoretical tools to design intelligent adaptive controllers using the state-of-the-
art machine learning methods.
• It is complemented with hands on practice emphasized through a series of assignments, exams, and a project, with a
focus on autonomous and intelligent applications, such as the control of under-actuated systems, autonomous driving
vehicles, unmanned aerial/underwater systems, robot manipulators, and the guidance and coordination of swarms
of mobile robots.

Course evaluation
Please consult Academic Regulation A-8.6 for rules pertaining “Justification of absence from an examination or of late
submission of assignments” for policies that pertain late submissions and absences from exams and in class tests. Absences
need to be communicated via the absence portal.

Assignments (45%) Assignments are individual. They consist of structured problems that comprehensively cover topics
addressed in class.
Group project (30%) Groups will be formed at the beginning of the semester. Each group has to choose a system that
will be the object of the project, and discuss it with the course instructor. The system can be from a research paper,
from a thesis project (if there is one), or it can generally originate from a problem that is relevant for the group
members. The system chosen can be modified/simplified to make it manageable within the course. The project will
be due the last day of classes; a revision will be conducted after reading week.
Final exam (25%) The final exam will be take-home, with 24 hours duration. It will be due on the last day of final
exams.

Reading and tutorial material

At the beginning of the course the students are assumed to be familiar with standard undergraduate mathematics and
linear control theory. You are required to review the related material from pre-requisite courses.

Textbooks:

1. K. J. Åström, B. Wittenmark: Adaptive Control. 2nd edition. Dover, 2008.

2. F. L. Lewis, D. L. Vrabie, and V. L. Syrmos: Optimal Control. 3rd Edition. Wiley, 2012.

The following books are only for reference and further readings:

3. R. S. Sutton, A. G. Barto: Reinforcement Learning: An Introduction. MIT Press, 2018

4. H. Butler: Model-Reference Adaptive Control - From Theory to Practice. Prentice-Hall, 1992
5. K. S. Narendra, A. M. Annaswamy: Stable Adaptive Systems. Prentice Hall, 1989

Some of the course material will be developed in Wolfram Mathematica:

• Tutorials: http://www.wolfram.com/broadcast/video.php?c=89&v=1119
• Wolfram cloud (free online interface with limitations): https://www.wolframcloud.com/

Lecture content
The content listed below will be covered sequentially through the lectures. Topics are mapped to the two textbooks
referenced above, with “Textbook” abbreviated to “Tb”.

Part I - Introduction and Parameter Estimation

1. Introduction to Adaptive Control Systems [Tb 1, Ch. 1]

a. Linear Feedback Structures
b. Effects of Process Variations
c. Adaptive Schemes
2. Real-Time Parameter Estimation [Tb 1, Ch. 2]

ELG7113/MCG5470 - Winter 2024 Page 2 of 4 Syllabus

 a. Least Squares and Regression Models
b. Recursive Least Squares
c. Parameter Estimation in Dynamic Systems

Part II - Bellman’s Optimality and Machine Learning Solutions

1. Dynamic Programming[Tb 2, Ch. 6]

a. Bellman’s principle of optimality
b. Optimal control of discrete time systems
c. Optimal control of continuous time systems
2. Reinforcement Learning and Optimal Adaptive Control [Tb 2, Ch. 11]
a. Dynamic Programming - Markov Decision Processes
b. Approximate Dynamic Programming
c. Temporal Difference Learning
d. Reinforcement Learning and Q-Learning
e. Policy Evaluation and Policy Improvement
f. Integral Reinforcement Learning
g. Adaptive Critics
h. Multi-Agent Adaptive Reinforcement Learning Systems
i. Neural Networks for Nonlinear Adaptive Control

Part III - Adaptive Techniques for Systems Identification and Control

1. Self-Tuning Regulators [Tb 1, Ch. 3]

a. Direct Self-Tuning Regulators
b. Indirect Self-Tuning Regulators
c. Model Following
2. Model-Reference Adaptive Systems [Tb 1, Ch. 5(5.1 to 5.4)]
a. The MIT Rule
b. Lyapunov Stability Theory
c. Design of MRAS Using Lyapunov Stability Theory
d. Design of MRAS Using Output Feedback

ELG7113/MCG5470 - Winter 2024 Page 3 of 4 Syllabus

Resources for students
uOSatisfACTION: a comprehensive portal collecting services by the University of Ottawa, to be used as a reference for a
variety of issues related to academic life.
Academic integrity: rights and responsibilities related to academic integrity.
Academic accommodation: support for students with temporary or permanent disability.
Health and wellness.

Academic regulations:

• A-4 for academic integrity and academic misconduct.

• A-8 for evaluation of student learning.
– A-8.6 for Justification of absence from an examination or of late submission of assignments. Absences need
to be communicated via the absence portal.
• A-9 for revision of grades and appeal.

ELG7113/MCG5470 - Winter 2024 Page 4 of 4 Syllabus