Double inverted pendulum linked with a spring

dynamics

1 Constants
∆t = 0.1 : Time step
g = 9.81 : Gravity
L = 1.0 : Half-length of the track
l = 0.5 : Half-length of a pole
mc = 1.0 : Mass of a cart
mp = 0.1 : Mass of a pole
µc = 0.0005 : Friction coefficient of a cart
µp = 0.000002 : Friction coefficient of a pole
K = 2.0 : Coefficient K of the spring
ls = 0.5 : Relaxed length of the spring
lsmin = 0.1 : Minimum length of the spring before deformation
lsmax = 1.5 : Maximum length of the spring before deformation
ẍmax = 2.0 : Maximum acceleration applied to a cart
ẋmax = 15.0 : Maximum velocity supported by the system
θ̇max = 10.0 : Maximum angular velocity supported by the system

1
2 State variables
x1 : Position of the first cart
ẋ1 : Velocity of the first cart
θ1 : Angular position of the first pole in [0, 2 ∗ π]
θ̇1 : Angular velocity of the first pole

x2 : Position of the second cart

ẋ2 : Velocity of the second cart
θ2 : Angular position of the second pole in [0, 2 ∗ π]
θ̇2 : Angular velocity of the second pole

3 Actions
a1 = ( −ẍmax , −ẍmax )
a2 = ( ẍmax , ẍmax )
a3 = ( −ẍmax , ẍmax )
a4 = ( ẍmax , −ẍmax )

4 Dynamics
f~1 = aj 1 + {K ∗ (ls − |x2 − x1 |)} with j ∈ [1, 4]
f~2 = aj 2 + {K ∗ (ls − |x2 − x1 |)} with j ∈ [1, 4]
a11 = (4 ∗ l) /3
a22 = − (mc + mp )

∀i ∈ {1, 2}
a12
i = − cos (θi )
a21
i = l ∗ mp ∗ cos (θni ) o
b1i = g ∗ sin (θi ) − µp ∗ θ˙i / (l ∗ mp )
2
 
b2i = l ∗ mp ∗ θ˙i ∗ sin (θi ) − f~i + µc ∗ sign (ẋi )
 2 12

22 1

 12 21

11 22


θ̈i = n bi ∗ ai − ao ∗ bi
/ ai ∗ ai − a ∗ a
ẍi = b1i − a11 ∗ θ¨i / a12i

     
ẋi ẋi ∆t ∗ ẍi
= +
θ̇ θ̇ ∆ ∗ θ̈
 i   i   t i 
xi xi ∆t ∗ ẋi
= +
θi θi ∆t ∗ θ̇i

5 Reward
r(xt , at ) = 0 if the resulting state variables agree with one of the following
statements :

2
 |xi | > L
x2 <= x1
|x2 − x1 | < lsmin
|x2 − x1 | > lsmax
else r(xt , at ) is equal to result computed with the state variable of the next state
:
{(1 + cos (θ1 )) /4} + {(1 + cos (θ2 )) /4}

6 Initial state
x1 = 0.0
ẋ1 = 0.0
θ1 = π
θ̇1 = 0.0

x 2 = x 1 + ls
ẋ2 = 0.0
θ2 = π
θ̇2 = 0.0