Professional Documents
Culture Documents
Automatica: Abhinav Kumar Singh Bikash C. Pal
Automatica: Abhinav Kumar Singh Bikash C. Pal
Automatica
journal homepage: www.elsevier.com/locate/automatica
Brief paper
1. Introduction Hardouin, and Ferrier (2000) and Yang and Kwak (2010, 2011),
wherein such inputs are called ‘uncontrollable inputs’. Another
Optimal control of a dynamic system involves cost effective recent example of exogenous inputs can be found in power system
operation of the system through constrained control effort. The literature in Singh and Pal (2014), wherein they are referred to as
particular case of optimal control in which system dynamics are ‘pseudo-inputs’.
described by linear differential equations, and the associated cost is The problem of linear quadratic control of LTI systems
a quadratic function of system states and control-effort, is termed with external disturbances has been studied in past (see, for
as linear quadratic (LQ) problem (Kwakernaak & Sivan, 1972; Ogata instance, Cheok & Loh, 1987; Johnson, 1968, 1971; Ostertag, 2011).
& Yang, 2001). Solution to the LQ problem is provided by linear In these studies it is assumed that the controller can either
quadratic regulator (LQR), a state feedback controller (Kalman, measure or estimate the external disturbances using information
1960). of the model which governs these disturbances. Thereby, the
It is assumed in the LQ problem that all of the inputs given to disturbances are eliminated using a control input which has two
the system are control inputs, which means that each of the input components; one component exactly cancels out the disturbance,
can be manipulated by the controller. But, this may not be the case while the other provides optimal state feedback. In case the
for a general LTI system, and some of the inputs to the system disturbance cannot be exactly canceled out (and this is practically
may be external disturbances, which means that such inputs can the most likely case), such a component of the control input
neither be disabled nor be manipulated by the controller. Such is found which is smallest in magnitude and also minimizes
inputs are also termed as exogenous inputs, and the terms ‘external the effect of the disturbance on the system. This technique of
disturbance’ and ‘exogenous input’ are used interchangeably in accommodating the external disturbance in the LQ problem by
this paper. Some examples of dynamic systems with exogenous minimizing its effect has a drawback: it does not guarantee the
inputs can be found in control literature in Menguy, Boimond, minimization of the net costs associated with control-efforts and
state-deviations. This drawback is also demonstrated in Section 5
of this paper. Thus, this technique fails to achieve the main
objective of linear quadratic control.
✩ This work was supported by EPSRC, U.K., under Grants EP/K036173/1 and EESC-
This paper proposes a new solution to the problem of optimal
P55251. The material in this paper was not presented at any conference. This paper
control of LTI systems with external disturbances or exogenous
was recommended for publication in revised form by Associate Editor Delin Chu
under the direction of Editor Ian R. Petersen. inputs, and the solution not only optimally accommodates the
E-mail addresses: a.singh11@imperial.ac.uk (A.K. Singh), b.pal@imperial.ac.uk disturbance, but also minimizes the overall quadratic costs of
(B.C. Pal). control-efforts and state deviations. The solution does not make
http://dx.doi.org/10.1016/j.automatica.2016.10.014
0005-1098/© 2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
A.K. Singh, B.C. Pal / Automatica 76 (2017) 10–16 11
Thus, the aim of the above problem is to control the system via
Nomenclature its control inputs under the constraints of exogenous inputs. This
problem is termed as the extended linear quadratic (ELQ) problem
k Denotes the kth time sample
in this paper.
T Denotes the transpose of a matrix or a vector
T0 The sampling period for the system in s
0a×b Denotes a zero matrix of size (a × b) 3. Classical LQR control
N The final time sample at which a closed-loop system
with LQR or ELQR reaches steady state A discrete-time open-loop LTI system without any exogenous
x The vector of states of the system input is represented by the following equation.
u The vector of control inputs to the system
u′ The vector of exogenous inputs to the system xk+1 = Axk + Buk . (1)
A The state matrix
B The input matrix corresponding to control inputs The quadratic cost function for (1) for N + 1 samples is given by:
B′ The input matrix corresponding to exogenous
N
inputs
F The state-feedback gain in the LQR and the ELQR
J= [xTk Qxk + uTk Ruk ] where Q ≥ 0, R > 0. (2)
k=0
solutions
G The feedback gain corresponding to u′ in the ELQR Minimizing J with respect to uk gives the following LQR solution.
solution
G′ The supplementary feedback quantity in the ELQR uk = −Fk xk , k = 0, 1, . . . , (N − 1), uN = 0; where, (3)
solution Fk−1 = (R + B Pk B)T
B Pk A,
−1 T
P N = Q, and, (4)
Ic Denotes an identity matrix of size (c × c )
J The quadratic cost function for a discrete LTI system Pk−1 = Q + A [Pk − Pk B(R + B Pk B)
T T
B Pk ]A.
−1 T
(5)
without any exogenous input
J′ The quadratic cost function for a discrete LTI system If N is finite then the above optimal control policy is called as finite
with both control inputs and exogenous inputs horizon LQR; otherwise it is infinite horizon LQR. Moreover, Pk and
P The positive-definite matrix corresponding to F Fk for the infinite horizon case are bounded and have a steady-
Q The cost weighting matrix corresponding to x state solution iff the pair (A, B) is stabilizable, and the steady-state
R The cost weighting matrix corresponding to u solution is found by solving the following discrete-time algebraic
R′ The cost weighting matrix corresponding to u′ Riccati equation (ARE).
S The matrix corresponding to G in the ELQR solution
S′ The matrix corresponding to G′ in the ELQR solution P = Q + AT [P − PB(R + BT PB)−1 BT P]A; (6)
F = (R + B PB) T
B PA.
−1 T
(7)
The above control solution in (9) does not consider the quadratic-
Definition 2. An ‘exogenous input’ given to an LTI system is an
cost for the discrete system given by (8). This system has an extra
input which cannot be removed from the system and whose
term (corresponding to the exogenous inputs) as compared to the
magnitude cannot be decided or changed. This input is an
system given by (1). Thus the quadratic-cost for this system gets
unavoidable quantity which cannot be used as a control input in
modified. For N + 1 samples it is given by:
corrective actions, although it may be possible to find a control
input which cancels out or minimizes the effect of the exogenous N
input. J′ = [xTk Qxk + uTk Ruk + u′kT R′ u′k ],
k=0
Using the above two definitions, the control problem is stated as
follows: where, Q ≥ 0, R > 0, R′ ≥ 0. (10)
For a discrete-time open-loop LTI system in which both control inputs
and exogenous inputs are present, find an optimal control law such In order to find the optimal control policy for (8), J′ in (10) needs
that the sum of the quadratic costs associated with the system state to be minimized with respect to uk . This minimization gives the
deviations, the exogenous inputs and the control inputs is minimized. following theorem.
12 A.K. Singh, B.C. Pal / Automatica 76 (2017) 10–16
Theorem 1. For an LTI system with pseudo-inputs (as given by (8)), Finding the partial derivative of J′N −1 in above equation with
provided u′k = 0 ∀ k ≥ N, the optimal control policy for 0 ≤ k < N respect to uN −1 , ∂ J′N −1 /∂ uN −1 comes as:
is given by (11)–(13) (and for k ≥ N , uk = 0).
∂ J′N −1 /∂ uN −1
uk = −(Fk xk + Gk u′k + G′k ); (11) = 2[RuN −1 + BT PN (AxN −1 + BuN −1 + B1 vN −1 )] (22)
Gk = Fk (Pk − Q) −1
Sk , Gk = Fk (Pk − Q)
′ −1 ′
Sk ; (12) ∵ ∂ JN −1 /∂ uN −1 = 0, for uN −1 =
′ opt
uN −1 , (23)
SN = 0, SN = 0,
′
Sk = (A − BFk ) (Pk+1 B + Sk+1 ),
T ′
∴
opt
RuN −1 + BT PN (AxN −1 + opt
BuN −1 + B1 vN −1 ) = 0, (24)
Sk = (A − BFk ) (Sk+1 (uk+1 − uk ) + Sk+1 ).
′ T ′ ′ ′
(13) opt
⇒ uN −1 = −(FN −1 xN −1 + HN −1 vN −1 ), (25)
Fk and Pk remain same as the LQR case (given by (4)–(5)). where, FN −1 = (R + B PN B) T −1 T
B PN A, (26)
(31)
Here a is the number of elements in xk and b is the number of
elements in u′k . It should be understood that because of (16), the Again, the combined quadratic cost for k = (N − 2), (N − 1) and N,
above modification has no effect on the dynamics of the original provided that the combined cost for k = (N − 1) and N is optimal
′opt
system. The modification is needed to get an iterative expression (which is JN −1 ), is given by J′N −2 = xTN −2 QxN −2 + uTN −2 RuN −2 +
′opt
for the optimal control policy. On its own, u′k cannot be expressed vTN −2 R1 vN −2 + JN −1 , and following the same aforementioned steps
in terms of u′k−1 . But when a new pseudo-input vector vk is defined ′opt
applied to find JN −1 , the values of uN −2 and JN −2 come as:
opt ′opt
J′N −1 = xTN −1 QxN −1 + uTN −1 RuN −1 + vTN −1 R1 vN −1 Hk = (R + BT Pk+1 B)−1 BT (Pk+1 B1 + Uk+1 Lk+1 ) (41)
+ BuN −1 + B1 vN −1 ). (21) Uk = FTk RHk + (A − BFk ) [Pk+1 (B1 − BHk ) + Uk+1 Lk+1 ].
T
(43)
A.K. Singh, B.C. Pal / Automatica 76 (2017) 10–16 13
opt
It may be noted that Wk has no role in deciding uk . Also, Pk (using solutions for Pk , Fk , Gk and Sk exist, and are given by F, P as in
(42)) can be rewritten as: (6)–(7), and S, G as in (57)–(58).
Pk = Q + FTk (R + BT Pk+1 B)Fk − FTk BT Pk+1 A S = (A − BF)T (PB′ + S) = (P − Q)A−1 (B′ + P−1 S),
+ AT Pk+1 (A − BFk ) (44) (since (A − BF)T = (P − Q)A−1 P−1 from (49))
∵ FTk (R + BT Pk+1 B)Fk = FTk BT Pk+1 A (from (40)) ⇒ S = (A(P − Q)−1 − P−1 )−1 B′ (57)
G = F(P − Q)−1 S, substituting S from (57) :
∴ Pk = Q + AT Pk+1 (A − BFk ). (45)
⇒ G = F(A − P−1 (P − Q))−1 B′ . (58)
Substituting Fk from (40) in (45) gives:
Although the terms Fk and Pk for the ELQR case remain same
Pk = Q + A (Pk+1 B(R + B Pk+1 B)
T T −1 T
B Pk+1 )A. (46) as the LQR case, this needs to be mathematically derived and
hence this derivation is an important contribution of this paper.
Similarly, Uk (using (43)) can be rewritten as: The other terms Gk and Sk are independent of the sequence of
u′k , and hence they can be easily calculated if A, B, B′ , Q and R
Uk = (A − BFk )T (Pk+1 B1 + Uk+1 Lk+1 ) are known. On the other hand, the terms G′k and S′k require the
+ FTk (R + BT Pk+1 B)Hk − AT Pk+1 BHk , (47) knowledge of the sequence of u′k for all the future and present
samples. Thus, if the sequence of exogenous inputs is not known,
∵ FTk (R + BT Pk+1 B)Hk = AT Pk+1 BHk (using (40)), only the terms Fk , Pk , Gk and Sk can be accurately calculated,
and the terms G′k and S′k can only be estimated/predicted
∴ Uk = (A − BFk )T (Pk+1 B1 + Uk+1 Lk+1 ). (48)
based on the estimated/predicted values of u′k . If u′k cannot be
Also, from (45): estimated/predicted then the term G′k should be ignored while
finding the ELQR policy.
(A − BFk )T = (Pk − Q)A−1 P−
k+1 .
1
(49)
5. Implementation example: A third-order LTI system
Substituting (A − BFk )T from (49) in (48):
The ELQR control can be implemented on any system whose
Uk = (Pk − Q)A−1 (B1 + P−
k+1 Uk+1 Lk+1 ).
1
(50)
equations can be reduced to the form given by (8). An illustrative
Using (40), Hk in (41) can be rewritten as: example has been presented as follows, in which a simple third-
order LTI system is controlled using the ELQR methodology.
Hk = Fk A−1 (B1 + P−
k+1 Uk+1 Lk+1 );
1
and using (50), The various state-space matrices of the test system, the
equation of which is given by (8), are as follows.
⇒ Hk = Fk (Pk − Q)−1 Uk . (51)
1.4 0.2 −0.1 0.1 0.8
Sk , Sk ∈ R
′
, Sk ∈ R
a×b ′ a ×1
Partitioning Uk in (48) as Sk : A= −0.2 0.8 −0.3 , B= 1.1 0.3 and
0.1 0.1 0.9 0.9 0.5
Sk = (A − BFk ) (Pk+1 B1 + Sk+1
′ T
S′k+1 Lk+1 )
Sk
1.2
B′ = 0.1 .
S′k = (A − BFk )T Pk+1 B′ 0a×1
⇒ Sk 0.2
∆u′k+1
Ib Hence, the above system has three states, two control inputs and
+ Sk+1 S′k+1
(52) one exogenous input. Initially, all the states and inputs are zero,
01×b 1
that is, x0 = 03×1 , u0 = 02×1 , and u′0 = 0. Following three cases
⇒ Sk = (A − BFk )T (Pk+1 B′ + Sk+1 ), and, (53) are considered for the exogenous input.
⇒ uopt
k = −(Fk xk + Gk u′k + G′k ) (55) Although in the above example system a vanishing exogenous
input has been given (which vanishes when k → ∞), any other
and using (51), Gk Gk = Fk (Pk − Q)−1 Sk
′ ′
Sk ⇒
sequence of exogenous input(s) can be given to the system (which
Gk = Fk (Pk − Q) −1
Sk ; Gk = Fk (Pk − Q)
′
Sk .
−1 ′
(56) may or may not vanish) and subsequently ELQR solution can be
applied.
Hence, with (40), (46), (53)–(56), Theorem 1 stands proved.
The optimal control solution in Theorem 1 is termed as the 5.1.1. ELQR policy
extended linear quadratic regulator (ELQR) solution. In this As u′k in (59) is an exponentially decreasing function of time-
solution, finite horizon case is applicable only when the sequence sample, and it becomes zero only when k → ∞, therefore
of exogenous inputs is known to be finite, otherwise if the the infinite horizon case of ELQR needs to be used to optimally
sequence is not known or if it is infinite then infinite horizon case control this system. Using equations (6), (7), (57) and (58), and cost
is applicable. If the pair (A, B) is stabilizable, then infinite horizon weighting matrices Q and R as I3 and I2 , respectively, the following
14 A.K. Singh, B.C. Pal / Automatica 76 (2017) 10–16
−0.67 −0.67
Sk = −1.11 × (0.95) = −1.11 u′k ,
′ k
and,
2.46 2.46 (61)
0.27
G′k = u′ .
−0.02 k
Substituting the values of F, G, and G′k from (60) and (61) in (11),
the optimal control policy for ELQR comes as:
5.1.3. Disturbance accommodating LQR policy 5.2. Known and stochastic model of the exogenous input
The disturbance accommodating LQR (DALQR) policy given by
(9) is also applied on the test system. After substituting the values
for F, B and B′ in (9), the DALQR policy comes as: In this case, the exogenous input is stochastic with known
model, and the rest of the system is same as in Case A. For k ≥ 1,
−0.42 0.11 0.60 −0.43 ′
the exogenous input is given as follows:
uk .
DALQR
uk =− x − (64)
1.14 0.12 0.05 k 1.49
u′k = (Xk )k ; Xk ∼ N (0.9, 0.01); Xk ∈ (0.85, 0.95). (65)
5.1.4. Comparison of control performance
Thus, Xk in the above model is a random variable with a truncated
The weighted norms of the states and the control inputs (given
by xTk Qxk and uTk Ruk , respectively) can be used as measures of normal distribution with mean = 0.9, variance = 0.01, upper limit
control performance of a control method. These weighted norms = 0.95, and lower limit = 0.85.
are also the quadratic costs associated with the control method As A, B and B′ remain same as in Case A, the values of P, F, S
for the kth sample, as can be inferred from the constituent terms and G also remain unchanged and are given by (60). An exact value
of J′ in (10). The cost associated with the exogenous inputs, given of S′k for this case cannot be found as the sequence of u′k is non-
by u′kT R′ u′k remains independent of the control method. This is deterministic. But the expected value of S′k can be evaluated by
because u′k is not dependent on the control method. substituting ∆u′k+1 with its expected value (which is −0.1(0.9)k )
The test system has been simulated in MATLAB, and the and replacing Pk and Sk with P and S, respectively, in (13). Finally,
weighted norms of states and control inputs have been plotted in S′k is solved iteratively and substituted in (12) to find G′k . The final
Fig. 1. It should be noted that xTk Qxk = xTk xk and uTk Ruk = uTk uk solutions for the expected values of S′k and G′k are given as:
for the test system. Table 1 presents a comparison of quadratic
costs associated with the states and the control inputs for the three −1.16
0.46
methods. It may be inferred from Fig. 1 and Table 1 that ELQR is Sk = −1.87 × (0.9)k ;
′ ′
Gk = × (0.9)k . (66)
−0.06
much more efficient than both DALQR and classical LQR in presence 4.15
A.K. Singh, B.C. Pal / Automatica 76 (2017) 10–16 15
Table 2
Comparison of mean quadratic costs for 1000 simulations for Case B. Table 3
Mean costs (p.u.) ELQR DALQR LQR Comparison of mean quadratic costs for 1000 simulations for Case C.
Mean costs (p.u.) ELQR DALQR LQR
State-cost ( xTk Qxk )
6.8 9.0 28.5
Control-cost ( uTk Ruk ) 21.8 24.3 40.8 State-cost ( xTk Qxk )
93.8 145.4 366.5
Total costs 28.6 33.3 69.3 Control-cost ( uTk Ruk ) 163.4 338.6 534.8
Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Abhinav Kumar Singh received Bachelor of Technology
(B.Tech.) degree from Indian Institute of Technology,
Journal of Basic Engineering, 82(1), 35–45.
New Delhi, India, and Ph.D. degree from Imperial College
Kwakernaak, H., & Sivan, R. (1972). Linear optimal control systems, vol. 1. New York:
London, London, U.K., in Aug. 2010 and Jan. 2015,
Wiley-Interscience.
respectively, all in electrical engineering.
Menguy, E., Boimond, J. L., Hardouin, L., & Ferrier, J. L. (2000). Just in time control
Currently, he is working as a Research Associate at
of timed event graphs: update of reference input, presence of uncontrollable Imperial College London. His research interests include
input. IEEE Transactions on Automatic Control, 45(11), 2155–2159. estimation, control and communication aspects of power
Ogata, K., & Yang, Y. (2001). Modern control engineering (4 edition). New Jersey: systems.
Prentice-Hall PTR.
Ostertag, E. (2011). Mono-and multivariable control and estimation: linear, quadratic
and LMI methods, vol. 2. (pp. 115–121). Berlin: Springer Science & Business
Media, chapter Observer theory: Deterministic disturbances compensation. Bikash C. Pal received B.E.E. (with honors) degree from
disturbance observer. Jadavpur University, Calcutta, India; M.E. degree from the
Singh, A. K., & Pal, B. C. (2014). Decentralized dynamic state estimation in power Indian Institute of Science, Bangalore, India; and Ph.D. de-
systems using unscented transformation. IEEE Transactions on Power Systems, gree from Imperial College London, London, U.K., in 1990,
29(2), 794–804. 1992, and 1999, respectively, all in electrical engineering.
Yang, J. M., & Kwak, S. W. (2010). Corrective control of asynchronous machines Currently, he is a Professor in the Department of Elec-
with uncontrollable inputs: application to single-event-upset error counters. trical and Electronic Engineering, Imperial College London.
IET Control Theory & Applications, 4(11), 2454–2462. He is Editor-in-Chief of IEEE Transactions on Sustainable
Yang, J. M., & Kwak, S. W. (2011). Model matching for asynchronous sequential Energy and Fellow of IEEE for his contribution to power
machines with uncontrollable inputs. IEEE Transactions on Automatic Control, system stability and control. His current research interests
56(9), 2140–2145. include state estimation, and power system dynamics.