DGMD S-17 Robotics, Autonomous Vehicles, Drones, and Artificial Intelligence Summer 2019 (updated 2019-03-27)

DGMD S-17 Robotics, Autonomous Vehicles, Drones, and Artificial Intelligence

Course Description: Practical advances in machine learning and artificial intelligence (AI) are ushering in a
new era of digital automation. In the next ten to fifteen years, drones, driverless vehicles, and artificial
intelligence will be used to transport goods, send packages, perform agricultural tasks, and transport people
in an efficient and safe way. In this course students learn the algorithms that underlie an autonomous
vehicle's understanding of itself and the world around it. They learn how a car can use unreliable sensor data
to make accurate predictions of its location in the world. This algorithm, called SLAM (simultaneous
localization and mapping), relies on Bayesian inference, tracking algorithms, Kalman filtering, and sensor
fusion. Students learn how to use an algorithm that employs a map and traffic information to find the quickest
route between two points. Students also use code that helps them simulate, visualize, test, and debug the
trajectories that comes from the search and control algorithms using the most popular tools and libraries.
Key Info
Term: Summer 2019 (JUNE 24–AUG 9, 2019)
Course Format: Online option-Zoom/HELIX.
Credits: Graduate, Undergraduate and Non-credit
Course website: https://canvas.harvard.edu/courses/56771
Location: 1 Story Street Room 304
Meeting time: Mondays and Wednesdays, 6:30 - 9:30 pm
Instructor: Jose Luis Ramirez Herrán, A.L.M. in Information Technology. Email:jlr682@g.harvard.edu
Office hours: by appointment (Zoom)
Email: jlr682@g.harvard.edu
Teaching Assistants: Ricardo Villamil, Soulemane Konate, David Lavy Brizuela, Akane Wakai
Prerequisites: Willingness to build things step-by-step and persistence when the things you have just created do
not yet work as you expected and you need to trouble shoot them.
Textbooks: No textbooks are required for this course. Supplemental online resources will be distributed as
necessary throughout the semester.
Learning outcomes
By immersing themselves in this course, students will be able to:
• Learn the fundamentals of autonomy, from a systems perspective
• Learn Vision-based perception (localization, mapping, object detection and tracking), mission/motion
planning, control, resource management. And how things fit together.
• Get excited about the beauty, the fun, the importance, and the challenges of robotics and artificial
intelligence, through learning experiences that are tangible, accessible, and inclusive.
• Participate of a complete undergraduate or graduate level class on AI and robotics; from the off-the-shelf
hardware plans, open source software, to the weekly lecture materials.
• Learn Methods and Tools for the development of a modern robotic systems, source code control,
branching, versioning, middleware, logs management, regression tests, continuous integration, and
configuration management. And the currently popular stack (ROS/Python/OpenCV/Git/Github/Circle).
• Learn the tools that academy and industry uses in the research and development and use them to prepare
for their future projects and/or career goals.

Assessment methods
Hands-on work is the core of this course. Focus on breadth, rather than depth. There will be tutorials, which could
be easily completed in the first 3-5 weeks. Please note that all students are expected to do and complete all the
tutorials. Tutorials are a very effective tool to easily empower the student and built confidence so students can take
the challenge of executing they own projects. This class is mainly build around the Dockietown.org project and
inspired on the Autonomous Navigation for Flying Robots class of the technical University of Munich. My approach
is to exploit the practical and advanced knowledge of Autonomous Navigation and apply it to this two different kind
of robots. It is mainly inspired in the Duckietown project because it is a project that offer unlimited possibilities in the
future and it provide an environment in which students can make part of the Autonomous navigation community
that grows and provides a great support. Because real implementation of this systems are more likely happen first

Jose Luis Ramirez Herran, ALM. Harvard University.

DGMD S-17 Robotics, Autonomous Vehicles, Drones, and Artificial Intelligence Summer 2019 (updated 2019-03-27)

in the area of the autonomous cars, we give emphasis to this subject. The same knowledge can be applied to
quad-copters and a special course for that is being developed.
Each tutorial will be accompanied with a quiz or a demo due the class after. This weekly quiz or demo will cover
reflection questions and specific concept questions to provide us with feedback of your progress. A demo will be
required to show with a small video that your system is working correctly. The pre-requisite knowledge also be
covered during lecture and will be evaluated with a quiz to be announced in our website. Note that Assignment 0 is
not graded but is mandatory. A writing assignment will be posted in the second week, to help us detect any
language barriers.

Projects: The objective of the project is to take a deep dive in one topic of interest. You’ll practice: project
development, collaboration, documentation, system integration.

We can identify 4 areas for research projects:

Area 1. Your project is on the applying the fundamentals of navigation based in vision applied to autonomous cars,
including: feature extraction, image filtering, line detection, place recognition, camera calibration, object recognition,
object tracking, line following, navigation, localization.
Area 2. You are interested in solving the mapping problem (notice that you can navigate without constructing a map
necessarily) including: SLAM, Motion planning, testing, validation and verification.
Area 3. You want to work in a problem involving multiple vehicles including: Fleet level planning, multi-vehicle
coordination.
Area 4. You want to implement one of the vision-based navigation task from areas 1-3 for an Autonomous flying
robots (drone).

Notes: we intend to present all the areas of research and demonstrate them, if possible, for you. Finally, if you
already been working with a different robotics platform, in which you are an expert, you’ll be allow it to use it for
your project assignment and make a demo for us, but no support will be provided. The project will be approved
depending on the viability to be completed in the limited time allowed and the current performance of the student at
the time of the proposal. Someone interested in area 4 still will do all the assignments with the Duckiebot and city
before moving to work with the drone.

Grading

Description Undergraduate (%) Graduate (%)

Autonomy Tutorials 60 50
Quizzes/Demos 10 10
Participation (class, forums, discussions, sections, office hours) 20 20
Undergraduate/Graduate Project 10 20

Workload: Students expected to do at least 8–12 hours of work per week outside of class time.

Additional graduate-credit requirements: For graduate students, the project has more value than for
undergraduates.

Attendance and participation policy: On Campus with Online Option

These courses give you the choice of attending on campus, participating live online, viewing class recordings on
your own time, or doing a combination of all three.

Classes are recorded, then posted to the course website within 24 hours of the end of class.

Jose Luis Ramirez Herran, ALM. Harvard University.

DGMD S-17 Robotics, Autonomous Vehicles, Drones, and Artificial Intelligence Summer 2019 (updated 2019-03-27)

Tentative Syllabus
Please use the course calendar for up to date information. Exact topic schedule is subject to change!

Tentative Weekly Topics:

Week Date Description Assignments Due Date
0 Present- Preparations – software and hardware, VMs, dual boots, Get Equipment/Quiz Jun 26
June 24 etc.
1 Jan 24 Introduction, class plans, state-of-the-art Quiz/ Demo Jun 26
Technical Overview of Autonomy
Software Architectures and Middle-wares for Robotics
1 July 26 Modeling Kinematics and Dynamics Quiz/ Demo Jul 1
2 July 1 Sensors, motors, motor controllers Writing assignment Jul 3
April tag detection and relative pose estimation, lane required (not graded)
following
2 July 3 Probabilistic state estimation, Bayes and Kalman filters Jul 8
3 July. 8 single robot behavior, navigation, localization Quiz/ Demo Jul 10
3 July. 10 Planning Control Quiz / Demo Jul 15
4 July 15 Multi-vehicle navigation and behavior Quiz / Demo Jul 17
Human-Machine Interaction and Safety, Advanced
Perception
4 July 17 ML and Reinforcement Learning for Autonomous Vehicles Quiz / Demo Jul 22
5 July 22 Project pitch Quiz / Demo Jul 24
5 July 24 Project proposal presentations Project work plan Jul 29
6 July 29 Engineering meetings Quiz / Demo Jul 31
6 July 31 Engineering meetings Progress presentation Aug 5
7 Aug 5 Engineering meetings Submit digital Aug 7
Presentation
7 Aug 7 Final presentations Presentation Noting
due

Important Dates
Term start date: 06/24/2019
Term end date: 08/09/2019
100% drop deadline: 06/26/2019
Withdrawal deadline: 07/26/2019
Grades due: 08/14/2019
Last day to drop courses for 50% tuition refund 07/03/2019
Independence Day (No class) 07/04/2019

Required Resources
1. Equipment (order at : https://www.duckietown.org/about/hardware)
The following is the list of items to order for the class.
There are 3 main tracks on the hands-on activities and projects you can do:
Area 1 - April tag detection and relative pose estimation, lane following with at least a 3x3 road loop
-> Starter kit robot + road loop at $349 + shipping
Area 2 - Intersection navigation, stop signs, traffic lights: one robot, bigger and more complex city (starter
kit ($349) + navigation expansion pack ($449) = 798 + shipping)
Area 3 - Traffic management, coordination: same city (big or small depending on navigation tasks), more
Duckiebots (at least 2 total).

Jose Luis Ramirez Herran, ALM. Harvard University.

DGMD S-17 Robotics, Autonomous Vehicles, Drones, and Artificial Intelligence Summer 2019 (updated 2019-03-27)

Area 4. AR drone 2.0 autonomous Navigation. You’ll have to acquire the AR drone 2.0 robot online. Need
approval of the instructor since is a more advanced topic and may require more dedication for the student.
Notes:
You can decide how you invest in these learning technologies and your project will depend on that choice.
This is a great investment and will allow you to take leadership and maybe create a community in your
locality towards promoting this new technologies for the future and for your career.
2. Zoom
• Laptop computer with Linux, Windows 10 or Mac OSX High Sierra.
• High speed Internet connection.
• Students are required to use a webcam and headset during class. We recommend a Logitech H390
headset, but any USB headset should be adequate. It’s not recommended to use a microphone that is
built into a laptop.

Accessibility
The Summer School is committed to providing an accessible academic community. The Accessibility
Office covers a variety of accommodations and services to students with documented disabilities. More in:
http://www.summer.harvard.edu/accessibility-services

Important Notes for Registered Students

We will be using a course management web service called Canvas for all course communication. Please be sure
that you get a Harvard e-mail account and access to Canvas. More importantly, it is critical that you check the e-
mail registered with Canvas, monitor course announcements and participate in discussions on our forum.
http://www.summer.harvard.edu/resources-policies/student-responsibilities

Official Harvard Summer School Policies

You are responsible for understanding Harvard Summer School policies on academic integrity and how to use
sources responsibly (https://www.summer.harvard.edu/resources-policies). Not knowing the rules,
misunderstanding the rules, running out of time, submitting the wrong draft, or being overwhelmed with multiple
demands are not acceptable excuses. There are no excuses for failure to uphold academic integrity.

To support your learning about academic citation rules, please visit the Harvard Summer School Tips to Avoid
Plagiarism (https://www.summer.harvard.edu/resources-policies/resources-support-academic-integrity), where
you'll find links to the Harvard Guide to Using Sources (https://www.extension.harvard.edu/resources-
policies/resources/tips-avoid-plagiarism) and two free online 15-minute tutorials to test your knowledge of academic
citation policy. The tutorials are anonymous open-learning tools.

Please note that academic integrity issues should be handled by the Dean of Students Office. Any cases of
suspected cheating or plagiarism should be brought to the attention of the Associate Dean of Students, Shirley
Greene (shirley_greene@harvard.edu).

Jose Luis Ramirez Herran, ALM. Harvard University.

DGMD S-17 Robotics, Autonomous Vehicles, Drones, and Artificial Intelligence Summer 2019 (updated 2019-03-27)

Appendix 1. Equipment (order at: https://get.duckietown.org/)

1. Starter Kit: Duckiebot + 3x3 city loop ($349)

This option provides: April tag detection and relative pose estimation, lane following
1.1 Duckiebot

1.2 City loop

Jose Luis Ramirez Herran, ALM. Harvard University.

DGMD S-17 Robotics, Autonomous Vehicles, Drones, and Artificial Intelligence Summer 2019 (updated 2019-03-27)

2. Starter kit ($349) + city navigation pack ($449)

This option provides: single robot behavior + navigation + localization

3. Traffic management, coordination: same city (big or small depending on navigation tasks), more
Duckiebots.
This option provides shared autonomy.

Jose Luis Ramirez Herran, ALM. Harvard University.