Professional Documents
Culture Documents
LQR Control of A Humanoid Robot
LQR Control of A Humanoid Robot
The conditioning of the system is now much better and Using Eq. (21) and following the transformation method in
we find in practice that the condition number of A is ∼ Eq. (17) and Eq. (18), we can choose Q1 such that
1e3. Because of this explicit projection, it is now easy to
1
find a minimal realization for the LTI system using Kalman Q1 = ATG1 Qcom AG1 (23)
decomposition. To find the optimal feedback gain matrix K m2
we use an infinite-horizon linear quadratic regulator. The cost giving
function to be minimized is defined as
Z ∞
T 1
J= (xT Qx + uT Ru)dt (15) (∆θ) Q1 (∆θ) = (∆xcom )T Qcom (∆xcom ) (24)
m2
0
and the resulting controller for the humanoid robot is where Qcom is a 3 × 3 matrix. Along the same lines, using
Eq. (19) we choose Q2 such that
τ = τ0 − Kx. (16)
D. LQR Control in Task Space Q2 = ATG QG AG (25)
In humanoid whole-body control problems, it is often more giving us
desirable to directly regulate task space variables, such as the θ̇T Q2 θ̇ = hTG QG hG (26)
CoM or the angular momentum of the robot than to regulate
joint states. We show how it is possible to explicitly regulate where QG is a 6 × 6 matrix. This allows one to weight
such task space variables. We simply rewrite the matrix Q the CoM movement and total robot momentum directly
of the cost function of Eq. (15) such that we optimize for through the cost matrices Qcom and QG . To ensure positive
elements in the task space defined as a linear combination definiteness of Q we also add a regularization term QR that
of the original state vector x. If one would like to put costs directly specifies costs for joint tracking.
on the task space states x̂ defined as
Q1 0
x̂ = T x Q= + QR (27)
0 Q2
rather than transforming the states, we can equivalently
Note that QR should be chosen to be appropriately small
transform the cost function such that we specify Q = T T Q̂T . such that it does not dominate the weighting. For the rest of
The cost function is equivalently written the paper the controller that uses this type of cost function
Z ∞
manipulation will be referred to as the LQR-momentum
J= x̂T Q̂x̂ + uT Ru (17)
0
controller.
376
III. E XPERIMENTS Sagittal Push Experiment - LQR Controller (Joint Response)
real hardware. This paper will focus extensively on the tests 0.00
conducted on the real system and use the simulation results to 0.02 LQR HFE
support future work. The hardware used is the lower body of 0.04 HAA
HFR
the hydraulically actuated torque controlled Sarcos humanoid 0.06
KFE
(Fig. 1). We use the low level feedback force controller 0.08
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5AR
described in [5] which controls the hydraulic actuation to 0.04 AFE
ensure a desired torque profile. Base state estimations were AAA
0.02
made using a Kalman filter that fuses IMU data with robot 0.00
kinematics as described in [14]. With the goal of balancing 0.02
LQR momentum
the humanoid in mind, the cost matrix was tuned such 0.04
that the heaviest cost was placed on the error in the base 0.06
horizontal positions for the LQR controller and the heaviest 0.08
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5
cost on the horizontal CoM error for the LQR-momentum Time (s)
controller. To test the disturbance rejection capabilities of the
two controllers, push tests were conducted in both the sagittal Fig. 3: Joint error response to the push shown in Fig. 2. The
(Fig. 2) and frontal (Fig. 4) plane. In addition, the response joints are abbriviated by concatenating the specific link (Hip,
for the sagittal push in joint space is shown in Fig. 3. To Knee, and Ankle) and motion (Flexion-Extension, Abduction-
measure the force of the push a 6D force torque sensor was Adduction, and Rotation).
mounted to the end of push stick to record time-synchronized
data. Frontal Push Experiments
200
Sagittal Push Experiments Force (N) 150
LQR
700 LQR momentum
100
600
Force (N)
LQR
500 50
400 LQR momentum
300 0
0.0 0.5 1.0 1.5 2.0 2.5
200
100 0.04
COM (m)
0
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 0.02
0.04 0.00
COM (m)
0.04 Z 0.00
COM (m)
z
plot is omitted. The two gain matrices will diverge more when
x
y
the full humanoid is used and the joint coupling becomes
more complex. Using this visualization, one can easily see
the extensive coupling that results. The gain matrix is far
from a diagonal matrix, meaning that there is a significant
amount of coupling generated. Note that this coupling is not
intuitive and thus near impossible to be replicated by a hand
tuned PD controller. Furthermore, natural balancing strategies
are visible in this figure. In the columns corresponding to the
base, we can see that all the ankle joints and the knee joint
are inversely coupled to the ankle AA joints when rejecting
disturbances in the frontal plane (base x). Secondly, the ankle
and knee FE joints are used almost exclusively for rejecting
disturbances in the sagittal plane (base y). Finally, the base
vertical position holds no importance for the given cost values
(this was expected due to our prescribed cost function). Other
interesting couplings are also visible for all the other joints.
LEFT RIGHT
K K
HIP N ANKLE HIP N ANKLE Torso BASE
E E
150
FE AA R E R FE AA FE AA R E R FE AA R AA X Y Z a b g
FE
Tracking 1Hz squat − LQR Controller H
AA
I
P 100
actual R
Base X (m)
0.25 L
desired E KNEE FE
0.2 F
T A R
N
0.15 K FE
L 50
0.1 E AA
FE
−0.4 H
I AA
Base Y (m)
−0.45 actual
R P
desired R 0
I
−0.5 G KNEE FE
H R
−0.55 T A
N
K FE
L
AA
−50
E
0.05 T
O R
actual
Base Z (m)
0 R
S AA Matrix Diagonal
desired O
−0.05
−0.1
Fig. 7: Visualization of gain matrix for LQR controller
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Time (s)
corresponding to the position error (torques on left, states on
top), where variables a,b,g are the cardan angles of the base.
Fig. 6: Tracking a 1 Hz squatting trajectory along the Z-axis The gains for the velocity terms are omitted because they
without relinearization (i.e. constant gains and feed-forward have a very similar structure with values ranging between
torques). -20 and 40.
378
The LQR-momentum was much easier to tune (i.e. fewer as the computation of the rotation resulting from a small
tuning parameters with more intuitive effects on the CoM) disturbance ω around the rotation q1 . It allows us to compute
and produced a similar performance in terms of impulses an infinitesimal perturbation of a quaternion along a rotation
that could be rejected (Figs. 2, 4) . For both the LQR and axis ei . A more detailed description of differential calculus on
LQR-momentum controller formulations, the humanoid was SO(3) and the operator can be found in [16]. Numerical
able to reject pushes of 10.4 Ns - 10.7 Ns with a peak force differentiation for the quaternion becomes
of approximately 650 N in the sagittal plane and 5.2 Ns ∂f f (q ei ) − f (q (−ei ))
- 6.5 Ns pushes with a peak force of approximately 200 = , i = 1, . . . , 3. (30)
N in the frontal plane. In [2], test results were reported ∂q 2
for an experiment where a pendulum was used to impact ACKNOWLEDGEMENT
the robot with about 5.8J in the frontal plane. This would We would like to thank Umashankar Nagarajan for provid-
equivalently deliver approximately a 7.6 Ns impulse assuming ing both guidance and data that proved useful for debugging
that all the momentum of the ball was transferred to the this formulation in the early stages of the project.
robot. In [5], impulses between 4.5 Ns and 5.8 Ns with peak
forces up to 150N were rejected while in single support R EFERENCES
balancing. Our results suggests that a simple LQR controller [1] G. Cheng, S.-H. Hyon, A. Ude, J. Morimoto, J. Hale, J. Hart,
is capable of maintaining balance and rejecting perturbations J. Nakanishi, D. Bentivegna, J. Hodgins, C. Atkeson, M. Mistry,
S. Schaal, and M. Kawato, “Cb: Exploring neuroscience with a
at levels comparable with more complicated, state of the humanoid research platform,” in Robotics and Automation, 2008. ICRA
art, methods. This result raises the following question: how 2008. IEEE International Conference on, May 2008, pp. 1772–1773.
much improvement is made by using more advanced and [2] C. Ott, M. Roa, and G. Hirzinger, “Posture and balance control for
biped robots based on contact force optimization,” in Humanoid Robots
computationally complex controllers? It would be interesting (Humanoids), 2011 11th IEEE-RAS International Conference on, Oct
to systematically compare these different controllers in order 2011, pp. 26–33.
to experimentally measure their relative performance. [3] L. Righetti, J. Buchli, M. Mistry, and S. Schaal, “Inverse dynamics
control of floating-base robots with external constraints: A unified
V. CONCLUSION view,” in Robotics and Automation (ICRA), 2011 IEEE International
Conference on, May 2011, pp. 1085–1090.
In this contribution, we experimentally tested the perfor- [4] B. Stephens and C. Atkeson, “Dynamic balance force control for
compliant humanoid robots,” in Intelligent Robots and Systems (IROS),
mance of a LQR controller for a humanoid robot using 2010 IEEE/RSJ International Conference on, Oct 2010, pp. 1248–1255.
a linearization of the full robot dynamics together with [5] A. Herzog, L. Righetti, F. Grimminger, P. Pastor, and S. Schaal,
the contact constraints. In these experiments, balancing and “Balancing experiments on a torque-controlled humanoid with
hierarchical inverse dynamics,” in IEEE/RSJ Intl Conf on Intelligent
tracking performance was evaluated. We also proposed a Robots and Systems, 2014. [Online]. Available: http://www-clmc.usc.
way to incorporate task space objectives into the LQR cost edu/publications/H/herzog-IROS2014.pdf
function. It has been successfully demonstrated that with [6] S. Hyon, J. Hale, and G. Cheng, “Full-body compliant human-
humanoid interaction: Balancing in the presence of unknown external
this simple control law both good disturbance rejection and forces,” Robotics, IEEE Transactions on, vol. 23, no. 5, pp. 884–898,
tracking can be achieved on a physical robot in the presence Oct 2007.
of real conditions (i.e. model uncertainty, sensor noise, [7] B. Anderson and J. Moore, Optimal control: linear quadratic methods,
ser. Prentice-Hall information and system sciences series. Prentice
imperfect state estimation, ground contact compliance, etc.). Hall, 1990. [Online]. Available: http://books.google.com/books?id=
Furthermore, we analyzed the natural balancing strategies LgUqAQAAMAAJ
that emerge from the coupled gain matrix. In the future, [8] S.-H. Lee and A. Goswami, “A momentum-based balance controller
for humanoid robots on non-level and non-stationary ground,”
we will investigate the improvements that relinearization, Autonomous Robots, vol. 33, no. 4, pp. 399–414, 2012. [Online].
calculated either online or offline (potentially making use Available: http://dx.doi.org/10.1007/s10514-012-9294-z
of methods such as [15]), contributes to the tracking per- [9] C. Atkeson and B. Stephens, “Multiple balance strategies from one
optimization criterion,” in Humanoid Robots, 2007 7th IEEE-RAS
formance. Alternatively, this formulation could be used International Conference on, Nov 2007, pp. 57–64.
with classical receding time horizon control method used [10] K. Yamane, “Systematic derivation of simplified dynamics for hu-
with LQR controllers. Finally, we will extend the approach manoid robots,” in Humanoid Robots (Humanoids), 2012 12th IEEE-
RAS International Conference on, 2012, pp. 28–35.
to include contact switching and single support balancing, [11] O. C. Garofalo, G. and A. Albu-Schaffer, “On the closed form
making it possible to test more complicated tasks involving computation of the dynamic matrices and their differentiations,” in
a larger range of motion (e.g. walking). International Conference on Intelligent Robots and Systems(IROS),
2013, pp. 2364–2369.
[12] M. Popovic, A. Hofmann, and H. Herr, “Angular momentum regulation
A PPENDIX during human walking: biomechanics and control,” in Robotics and
For the numerical differentiation, we use a central finite Automation, 2004. Proceedings. ICRA ’04. 2004 IEEE International
Conference on, vol. 3, April 2004, pp. 2405–2411 Vol.3.
difference method [13] D. Orin and A. Goswami, “Centroidal momentum matrix of a humanoid
robot: Structure and properties,” in Intelligent Robots and Systems,
∂f (x) f (x + ) − f (x − ) 2008. IROS 2008. IEEE/RSJ International Conference on, Sept 2008,
≈ . (28)
∂x 2 pp. 653–659.
[14] N. Rotella, M. Bloesch, L. Righetti, and S. Schaal, “State
Special care needs to be taken for the orientation of the base estimation for a humanoid robot,” in IEEE/RSJ Intl Conf
which is an element of SO(3). To account for this the box on Intelligent Robots and Systems, 2014. [Online]. Available:
http://www-clmc.usc.edu/∼nrotella/IROS 2014 Submission.pdf
plus operator is introduced : SO(3) × so(3) → SO(3) [15] P. Reist and R. Tedrake, “Simulation-based lqr-trees with input and
such that state constraints,” in Robotics and Automation (ICRA), 2010 IEEE
q1 ω = exp(ω) ⊗ q1 . (29) International Conference on, May 2010, pp. 5504–5510.
[16] C. Hertzberg, R. Wagner, U. Frese, and L. SchröDer, “Integrating
generic sensor fusion algorithms with sound state representations
where the exponential map is the map from an infinitely through encapsulation of manifolds,” Inf. Fusion, vol. 14, no. 1,
small rotation ω to its corresponding rotation and ⊗ is the pp. 57–77, Jan. 2013. [Online]. Available: http://dx.doi.org/10.1016/j.
quaternion multiplication. The box plus operator can be seen inffus.2011.08.003
379