General: Places to Learn Robotics

For the uneducated, here are some good places to learn the background of the field. None of
this is really a pre-req, but I think it’s helpful if you are interested in it as a career option.

My past crash courses/link pages:

- Legged Robotics:
https://docs.google.com/document/d/1hRomxCYV8-QkmUAbEtQ45mRZrWe32xureQwel
EtpGFc/edit?usp=sharing
- Robot Arms:
https://docs.google.com/document/d/1dQq4EfBD3MWvAWnioHKCsFHYMW_WQ9pfJIT
spgXiqNc/edit?usp=sharing

UPenn Open Source

- Source:
https://www.youtube.com/playlist?list=PL_onPhFCkVQhuPiUxUW2lFHB39QsavEEA
- A phenomenal cohesive robotics playlist. It’s nicely broken down so you can revisit
easily.

Underactuated Robotics & Drake:

- Source: ​http://underactuated.mit.edu/underactuated.html
- https://drake.guzhaoyuan.com/
- Read their ipynb tutorials as well:
https://github.com/RobotLocomotion/drake/tree/master/tutorials
- Source:
https://www.youtube.com/watch?v=VeEqtTgDXFc&list=PLkx8KyIQkMfVG-tWyV3CcQbo
n0Mh5zYaj
https://www.youtube.com/watch?v=ZTizHbj339w&list=PLkx8KyIQkMfX1WpWYqtep7TO
mboZeDtev
- Probably the best controls focused robotics course. It really puts hard robotics problems
into the perspective of a controls engineer.

Recommended Books and Sharepoints:

- Modern Robotics, Lynch
- (Free pdf here): ​http://hades.mech.northwestern.edu/images/7/7f/MR.pdf
- Github Link Dumps:
- https://github.com/kiloreux/awesome-robotics
- https://github.com/ahundt/awesome-robotics
- https://github.com/jslee02/awesome-robotics-libraries
- https://github.com/fkromer/awesome-ros2​:
- Controls Engineering in FRC: ​https://github.com/calcmogul/controls-engineering-in-frc
Bipedal Robotics: General
Legged robotics is a particularly interesting and difficult topic because it is dictated by ​contact
mechanics​. Control in the context of contact mechanics deals with the study of multibody
systems (legged robots) that can only be controlled when there is contact with the ground and
subject to friction constraints. Fundamentally this makes the problem within the realm of a
Hybrid System​, which means that it exhibits both discrete and continuous behavior. The
second key idea is the concept of ​limit cycle stability​, which can be made analogous to
walking stability, and finding a valid poincare map. One of the most common examples of a limit
cycle is the Van der Pol oscillator. ​Hybrid Zero Dynamics (HZD)​ is the process of analyzing
whether or not the desired output of the system leads to a stable periodic orbit
(​http://ece.umich.edu/faculty/grizzle/papers/Grizzle_Westervelt_HZD_IsidoriFest.pdf​).

We’ve been doing solid legged robotic solutions since 1980’s, when Marc Raibert (founder of
Boston Dynamics) first began experimenting with legged systems with simplified dynamic
representations (​https://www.youtube.com/watch?v=XFXj81mvInc​). Nowadays legged studies
have evolved from exclusively simplified models/control to also include more complicated
formulations such as Model Predictive Control.

Watch Lectures 11-16 of Underactuated Robotics for a more in depth understanding of some of
these concepts.

One of the best resources out there:

https://groups.csail.mit.edu/robotics-center/public_papers/Wieber15.pdf

Design for Legged Locomotion

1) Try to keep all of the mass in the body of the robot. Legs should be kept as light as
physically possible to better reflect control models and simplify nonlinearities. This also
improves the mechanical advantage, motion control, and backdrivability of the leg
actuators.
2) Sensing and control bandwidth are extremely important to deal with impacts. If you are
working with force control (many modern balance controllers) then you need high
bandwidth for impedance control.
3) High Torque Density Motors (BLDC/Permanent Magnet Synchronous Motor)
4) Backdrivability is a very nice design choice you can do it, because the system will be a
bit safer to work with around humans and won’t be as fragile. There are other nice
motion control benefits of backdrivable systems as well. Ballscrews and
planetary/harmonic gearboxes with gear ratios < 30:1 are great examples for
 backdrivable systems. However it can also be done with smart motor control and good
torque sensors on higher geared systems.
5) Keep it as lightweight as possible. Better for motion control and will save you from nasty
impact forces and torques.
6) More than anything​ legged robots is about form factor. How big and how heavy relative
to the actuator strength.

https://www.researchgate.net/publication/315471039_Design_of_Dynamic_Legged_Rob
ots/figures?lo=1
 https://www.ri.cmu.edu/pub_files/2016/8/kaloucheThesis.pdf

Design Ideologies
● High torque, low inertia: ​https://ieeexplore.ieee.org/abstract/document/9035049
Contact Models
When a foot comes into contact with the ground, there will be a returning ground reaction force
that acts on the rest of the robot. This force can be represented a few different ways with
differing results from a control perspective.

Soft Contact​ models use a virtual spring/damper model between the foot and the ground to
establish the ground reaction force (F=kx+damping). Soft models are typically the best ways to
represent what will happen in the real world. The downside of good soft contact models is that
they are more difficult to analyze because they lead to “stiff” differential equations and require
large spring constant values.

Hard Contact​ models have no spring/damper representation. They are worse at reflecting
reality, but are far better for numerical analysis.

Running/Hopping: Simple Model Controllers

A controller can be called a running controller if two concepts are present:
1) The existence of an “aerial” phase
2) Exchange of energy (kinematic/potential)
An even bigger simplification between running and walking controllers can be made based on
pendulum motions.

For the most part, biology adheres to these two models for locomotion. Some animals exhibit
just spring-mass models, such as birds, while other animals can exhibit a mixture. Elephants are
funky because their front legs “walk” and their rear legs “run” according to these models.
Spring Loaded Inverted Pendulum (SLIP)
SLIP is probably the most famous robot locomotion algorithm. It was originally coined by Raibert
in his PhD from his experimentation with his single legged “Raibert Hoppers.” After studying
nature, it was found that spring-mass templates represent animals moving in nature pretty
accurately. Fundamentally, this algorithm approaches legged locomotion as an open loop
process, where the only closed loop process is the leg touchdown angle around a neutral point
and sometimes the pitch of the robot (for rigid bodies and the classical raibert hoppers).

SLIP runs on a few underlying assumptions:

1) A massless leg (zero effort leg commands and instant angle adjustments)
2) Perfectly elastic collisions (energy is always conserved)
3) The foot acts like a pin joint (infinite friction) when it makes contact with the ground
These assumptions can fall apart in quite a few scenarios, obviously, whether it is because of
design flaws of poor environmental conditions. However the model still holds up very well today
because of its robustness.

The algorithm can be summarized as three distinct control phases:

Landing Flight Phase -> Stance Phase -> Take-off Flight Phase​.

A few different ideas for controlling each of these phases:

1) LQR
2) PID
3) Deadbeat control [1]
a) Finite control inputs can be done to move from the state to the origin in one step.
There is always a touchdown angle you can choose so you reach the fixed point
in one hop.
b) Advantage of this is that you can control the system with no knowledge of the
terrain.
- [1] ​https://link.springer.com/referenceworkentry/10.1007%2F978-94-007-7194-9_43-1
Walking Controllers: Simple Model Controllers

Linear Inverted Pendulum (LIP)

The linear inverted pendulum is probably the most basic form of underactuated
balancing. Don’t get this confused with the regular Inverted Pendulum (IP) Model! It was the first
major simplified model that could represent a legged underactuated system. LIP models
represent walking locomotion quite nicely, as opposed to SLIP, which does not work that well
with “walking speeds.” Even today, there is new research that contributes ways to use this
model in legged robots, such as Hybrid Passive-LIP (HLIP)[A. Ames 2019].

More specifically, the cart-pole problem can instead be represented as a “cart-table” model,
which can represent the dynamics more accurately. The cart-table model also makes a direct
relation to what is known as the Zero Moment Point (ZMP), which is one of the most common
stable algorithms.
Cart Pole Inverted
● https://www.youtube.com/watch?v=qjhAAQexzLg
● https://www.youtube.com/watch?v=M_jchYsTZvM
● https://www.youtube.com/watch?v=1_UobILf3cc
Inverted Pendulum for Walking
● https://arxiv.org/abs/1910.00684
● https://ieeexplore.ieee.org/document/5651082
● https://www.cs.cmu.edu/~hgeyer/Teaching/R16-899B/Papers/KajiitaEA01IEEE_ICIRS.p
df
● https://www.techunited.nl/media/files/humanoid/SwanVanDalen_GRAD2012_A_Linear_I
nverted_Pendulum_Walk_Implemented_on_TUlip.pdf
Zero Moment Point (ZMP)
In the last decade, Zero Moment Point (ZMP) has emerged as one of the main control
methods to maintain stability. Asimo from Honda [1], HRP-3 from AIST [2], KHR-3 (HUBO) from
KAIST [3], and Nao from Aldebaran-Robotics [4] all use this method. ZMP was originally
discovered by Vukobratovic and Juricic in [5].
The ZMP is the point on the ground around which the sum of all the moments of the
active forces equals zero. Under the assumption that no external forces exists, the ZMP position
can be computed rather easily based on the forces on the feet about the CoM [6].
ZMP was the main algorithm used for walking in the 2015 DRC and can definitely be
considered a “standard” by many accounts in the walking community. It has a requirement to
know more about the environment than other, more passive, methods, but works great in flat
areas and steps. Part of what made Asimo awkward at first was the fact that it maintained a
constant Center of Mass (CoM) height, with no impact events, and linear CoM dynamics.

Cons:
- This model does not have good control of angular momentum, which can make it fragile
in very unstructured environments. However it works great for flat surface environments
and stepping.

1. T. Hirose, M. Takenaka. Development of humanoid robot asimo. Honda R&D Technical

Review, 12(1):1–6, 2001
2. K. Kaneko, K. Harada, F. Kanehiro, G. Miyamori, and K. Akachi. Humanoid robot hrp-3.
In IROS 2008. IEEE/RSJ International Conference on Intelligent Robots and Systems,
sept. 2008.
3. I.W. Park, J.Y. Kim, J. Lee, and J.H. Oh. Mechanical design of humanoid robot platform
khr-3 (kaist humanoid robot 3: Hubo). In 5th IEEE-RAS International Conference on
Humanoid Robots, 2005., pages 321–326. Ieee, 2005.
4. D. Gouaillier, V. Hugel, P. Blazevic, C. Kilner, J. Monceaux, P. Lafourcade, B. Marnier, J.
Serre, and B. Maisonnier. Mechatronic design of nao humanoid. In ICRA’09. IEEE
International Conference on Robotics and Automation, 2009., pages 769–774. IEEE,
2009.
5. G.A. Cavagna, H. Thys, and A. Zamboni. The sources of external work in level walking
and running. The Journal of physiology, 262(3):639, 1976.
6. http://ai.stanford.edu/~rxzhang/Motion%20Planning%20of%20Biped%20Robot%20Clim
bing.pdf

Feedback Linearization
Pros:
- Easy control of a biped (originally done with Asimo in 1990’s)
Cons:
 - Not natural and energy inefficient

Divergent Component of Motion / Capture Point

The divergent component of motion is an extension of the Capture Point Method and is a very
common method for safe control of legged robots. Capture point is a nice standard for push
recovery.

Hybrid Zero Dynamics

This is meant for planar walking mostly. However there are 3D implementations of it. It is
a very popular algorithm and still has research interest, especially with neural nets. To learn
more: ​https://web.eecs.umich.edu/~grizzle/papers/Grizzle_Westervelt_HZD_IsidoriFest.pdf

Expanding Simple Models

A SLIP model is not exclusive to running locomotion. ​A SLIP controller can be used
for both walking and running for a bipedal robot​. A bipedal robot can be represented as two
SLIP models (one for each leg). An example of this can be seen in this video, when I worked on
Blackbird with Gabrael Levine: ​https://www.youtube.com/watch?v=ZTL4L9i174Y​.
Bipedal SLIP models can be broken down as follows:
Low Spring Stiffness -> Running Gait | High Spring Stiffness -> Walking Gait

- [2] ​https://link.springer.com/referenceworkentry/10.1007%2F978-94-007-7194-9_43-1
Bipedal SLIP
ATRIAS (Hubicki/Hurst) was developed to embody the SLIP template model. It was one
of the first big projects to make a case for ​Passive Dynamics​. Zero Moment Pole strategies rely
on planning with respect to the environment to ensure ZMQ criteria. A passive dynamic system
is one that locomotes as a self-stable phenomenon; one that rejects disturbances without formal
feedback.

One of the other interesting contributions that was made clear with ATRIAS was the
effects of the​ “toe off force”​ or “energy injection” to improve the efficiency of the system. The
simple process of ​adding a little extra force in the last moments of the take-off flight stage
will dramatically improve the efficiency​ of locomotion (10x the energy efficiency in some
cases). For a SLIP model, energy injection can be done by simply extending the leg.
- [3] ​https://mime.oregonstate.edu/research/drl/publications/_documents/hubicki_2018.pdf

Actuated SLIP (aSLIP)

CASSIE was developed based on the concept of the SLIP model as a descendent of
ATRIAS. Actuated SLIP (aSLIP) is a method is a dynamic hybrid method, the domains of which
differentiate by the number of contacts. Walking is described as periodic alternation of a Single
Support Phase (SSP) and Double Support Phase (DSP) of spring loaded legs.
 Trajectory optimization via Direct Collocation can be used to find optimal periodic
solutions for the aSLIP walking method. Energy injection is encoded implicitly through the
optimized leg length actuation. The Leg Length and Leg Angle (minimal coordinates) are the
outputs of the reduced order model.
Feedback control on aSLIP involves the stabilization of the optimized trajectories, which
is commonly done by feedback control on the output dynamics. With underactuated walking,
periodic stability can be checked through Poincare Maps.
LIP techniques can be used for stability against lateral rolling of the robot during walking.
The LIP dynamics are purely passive and have a closed form solution, which can be used to
keep the system in the correct state-space during the SSP.
- [4] ​https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8625066

Variable Body Inertial (Reaction Mass Pendulum)

https://www.researchgate.net/publication/262673894_Dynamics_and_Balance_Control_of_the_
Reaction_Mass_Pendulum_A_3D_Multibody_Pendulum_With_Variable_Body_Inertia
https://hybrid-robotics.berkeley.edu/publications/Dissertation2019_Avinash_Siravuru.pdf
Cost of Transport
Cost of Transport (CoT) is a non-dimensional number that defines efficiency of a legged system.

Where ​E​ is the energy input to the system, with a mass ​m​, that is used to move the system a
distance ​d,​ and ​g​ is gravity. Another definition uses the power input to the system ​P​ used to
move the system at a constant velocity ​v​.
https://www.researchgate.net/publication/315471039_Design_of_Dynamic_Legged_Robots/figu
res?lo=1

Manipulator Equations (Inverse Dynamics)

The manipulator equation is a common and generalized way to represent rigid body dynamics in
a system. Any rigid body can be broken down and represented using a ​Intertia/Mass Matrix​,
Coriolis Term​, ​Gravity Torque/Force​, and a ​Control Input​. These terms can be found using
the ​Lagrangian Dynamic Equation​ or ​Newton Euler Method

For a more detailed info:

https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-832-underactuated-r
obotics-spring-2009/readings/MIT6_832s09_read_appA.pdf
Admittance and Impedance Control
In robotics, ​Admittance​ ​Control ​or ​Impedance Control​ are commonly used to control both the
forces and positions of a multibody system. They are especially helpful in applications where
humans need to interact with robots. For legged robots it is used for handling abstract
disturbances and unexpected impacts (e.g feet making contact earlier or later than expected).

Impedance - ​Mechanical Impedance is the ratio of force output to motion input. A “spring
constant” defines the force output for a displacement of a spring. A “damping constant” defines
the force output for a velocity input. These constants can thus be adjusted as a gain to ​control
the force of resistance ​to external disturbances.

Admittance - ​The inverse of Impedance. It defines ​the motion that results from a force
input​. For example, if a mechanism applies a force to the environment, the environment may, or
may not, move depending on the force.

Impedance and Admittance Control can be applied at subsystem levels (e.g Impedance
Control of just the legs or arms to respond to external forces) or at the Whole-Body Level (e.g
Admittance Control to make sure the CoM doesn’t move too far after disturbed). Especially as
systems evolve to embrace more hierarchical control schemes with varying levels of
optimization.
Both Impedance and Admittance Control expand the Manipulator Equation to
incorporate disturbances and control inputs. For example, the MIT Cheetah 3 [1] uses a
feedforward PD impedance controller.
- https://www.researchgate.net/publication/330591547_Dynamic_Locomotion_in_th
e_MIT_Cheetah_3_Through_Convex_Model-Predictive_Control

Whole Body Control

WBC aims to i) define a small set of simple, low-dimensional rules (e.g., equilibrium, self
collision avoidance, etc.) ii) that are sufficient to guarantee the correct execution of any single
task, whenever feasible (e.g., reaching for an object with one end-effector), and of simultaneous
multiple tasks (e.g., reaching for an object with one end-effector, while reaching for a second
object with another end-effector), iii) exploiting the full capabilities of the entire body of
redundant, floating-based robots in compliant multi-contact interaction with the environment.
Therefore, a control system that is specifically designed to guarantee the execution of a single
task, even if it uses all the joints of a robot, cannot be considered WBC. A WBC system should
necessarily be general enough to allow the execution of several (ideally any) tasks.
Optimization Approaches

Kinematic/Dynamic Based Optimizations

We know that ​baking underactuated and simplified models into higher level optimizations
has the potential to be far more versatile. There are multiple solutions evolving to solve the
online Optimal Control Problem (OCP). Most algorithms will generalize the ZMP point mass
model with additional environmental and joint considerations. [2] Uses iLQR, a quasi-Differential
Dynamic Programming (DDP) method descent method that uses sparsity of derivatives to
minimize computation costs and still use box constraints on control variables. [3] Solves WBC
motion planning using Mixed-Integer MPC optimization for local stabilization across multiple
control modes (​https://www.youtube.com/watch?v=2Vry-th8g2s​). This was because locally valid
approximations resemble a piecewise affine system. [4] Introduces a trajectory optimization
algorithm (DIRCON) that extends the direct collocation method to third order trajectories
(important for reliable motion control) (​https://www.youtube.com/watch?v=62GIUJC60P4​)

1. http://www.roboticsproceedings.org/rss14/p54.pdf
2. https://homes.cs.washington.edu/~todorov/papers/KoenemannIROS15.pdf
3. https://www.researchgate.net/publication/281374074_Whole-body_Motion_Planning_wit
h_Centroidal_Dynamics_and_Full_Kinematics
4. https://ieeexplore.ieee.org/document/7487270
5. https://arxiv.org/pdf/1901.08505.pdf
6. https://ieeexplore.ieee.org/document/7384453
7. https://arxiv.org/pdf/1702.08742.pdf
8. https://ieeexplore.ieee.org/document/8246930
9. https://ieeexplore.ieee.org/document/7384453
10. https://ieeexplore.ieee.org/document/7353843
11. https://ieeexplore.ieee.org/document/4651055
12. https://ieeexplore.ieee.org/document/4115592
Small Notes on LQR
LQR is a great tool to work with legged robots so they work how you want. They can avoid the
common pitfalls of feedback linearized systems (like Asimo in the early 2000’s that made it walk
funny).
Consider the following guidelines:
- Minimal Coordinates -> LQR
- Maximal Coodrinates -> Equality Constrained LQR
Problems with LQR:
- Can’t handle push recovery that well. And we are kinda stuck there as well. Constraints
change. You have to take extra steps to recover. LQR and Constrained LQR can’t do
that.
An equivalent system of linearized LQR can be considered Mixed Integer MPC. 2015 MIT
Darpa team used MI MPC to great success. Specifically try to linearize the centroidal dynamics

Momentum Optimizations (Standard practice)

In WBC, the momentum of the entire system can be seen as the principle control for an entire
body. Simply put, Euler’s laws of motion state that the rate of change of centroidal momentum is
equal to the sum of all external wrenches applied to the robot. Orin and Goswami showed that
there is a linear relationship between the joint velocity vector​ ​and the robot’s centroidal
momentum. The Matrix for this relationship, A, is called the ​Centroidal Momentum Matrix
(CMM).

Using the CMM, A, an affine relationship between joint acceleration and external wrenches
applied to the robot can be defined.
A high level behavior, such as a walking or driving behavior, sets up a quadratic program (QP)
with the following data: (i) desired motions in the form of motion tasks, (ii) information about
available contacts, and (iii) a rate of change of centroidal momentum objective. The QP
reconciles the motion tasks with the available contact by exploiting the CMM and Euler Law
relationship. The QP solver outputs joint accelerations and ground reaction wrenches exerted at
the robot links in contact to calculate desired joint torques via Inverse Dynamics

● https://www.researchgate.net/publication/280839675_Design_of_a_Momentum-Based_
Control_Framework_and_Application_to_the_Humanoid_Robot_Atlas
● https://arxiv.org/pdf/2004.12083.pdf
● https://hal.archives-ouvertes.fr/hal-01875387/document

Step Planning
Periodic Gaits - Locomotion gaits that have somewhat consistent contact timing and positioning
Non Periodic Gaits - Locomotion gaits that do not follow consistent contact timing or positioning

Typically you can solve footstep placement using optimization. Define a convex approximation
of the reachable sets of footstep positions.

Be on the lookout for proper footstep planners!!! There are some dead giveaways of good/bad
planners/controllers. ​Look for someone pushing the robot hard to the left when the right
foot is in the air. Typically the robot will fail because the legs get crossed, causing the
robot to fail.
 - Boston Dynamics gets around the “Crossed Leg” problem by “hopping” as it moves
sideways

Be aware of partial footholds and limited contact. Alot of the time you wont have a perfectly flat
surface, causing uneven forces and/or narrow pressure points on the robot.

General Recipe:
1) Find abstract regions for hand/feet placement
2) On a second pass, fill in the details for how/where feet and hands will move to those
areas.
Best References
- https://hal.laas.fr/hal-01859108/document
- https://www.youtube.com/watch?v=PqLZP8TANlg
- https://arxiv.org/abs/1907.08673

ATLAS - Boston Dynamics

https://www.youtube.com/watch?v=EGABAx52GKI