Application of penalty function method to computation of reachable sets for control

systems with state constraints

M. I. Gusev

Citation: AIP Conference Proceedings 1773, 050003 (2016); doi: 10.1063/1.4964973

View online: http://dx.doi.org/10.1063/1.4964973
View Table of Contents: http://scitation.aip.org/content/aip/proceeding/aipcp/1773?ver=pdfcov
Published by the AIP Publishing

Articles you may be interested in

On some questions in computer modeling of the reachability sets constructing problems
AIP Conf. Proc. 1773, 110015 (2016); 10.1063/1.4965019

Estimates of reachable sets of impulsive control problems with special nonlinearity

AIP Conf. Proc. 1773, 100004 (2016); 10.1063/1.4964998

Internal ellipsoidal estimates of reachable set of impulsive control systems

AIP Conf. Proc. 1631, 238 (2014); 10.1063/1.4902482

Numerical methods for construction reachability sets of dynamical systems

AIP Conf. Proc. 1497, 144 (2012); 10.1063/1.4766779

Optimizing Structural Active Control Force Using the Exterior Penalty Function Method
AIP Conf. Proc. 1020, 1245 (2008); 10.1063/1.2963747

Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 Application of Penalty Function Method to Computation of
Reachable Sets for Control Systems with State Constraints
M. I. Gusev

N.N.Krasovskii Institute of Mathematics and Mechanics, 16 Sofia Kovalevskaya str., 620099 Ekaterinburg, Russia

gmi@imm.uran.ru
URL: http://www.imm.uran.ru

Abstract. We study the penalty function type methods for computing the reachable sets of nonlinear control systems with state
constraints. The state constraints are given by a finite system of smooth inequalities. The proposed methods are based on removing
the state constraints by replacing the original system with an auxiliary system without constraints. This auxiliary system is obtained
by modifying the set of velocities of the original system around the boundary of constraints. The right-hand side of the system
depends on a penalty parameter. We prove that the reachable sets of the auxiliary system approximate in the Hausdorff metric the
reachable set of the original system with state constraints as the penalty parameter tends to zero (infinity) and give the estimates
of the rate of convergence. The numerical algorithms for computing the reachable sets, based on Pontryagin’s maximum principle,
are also considered.

INTRODUCTION

Reachable sets and their analogs play an important role in the solution of various control problems, problems of esti-
mation and control under uncertainty, and differential games [1, 2]. In the present paper, we consider the algorithms
for calculating the external and internal estimates for the reachable sets of a control system under state constraints.
Different approaches to computing the reachable sets and their estimates, including those for systems with state con-
straints, are presented in [3, 4, 5, 6, 7, 8, 9]. In this paper we assume that constraints are given as a level set of a
piecewise smooth function. The considered algorithms involve removal of the state constraints by replacing the orig-
inal system with auxiliary constraint-free systems, depending on a penalty parameters. These system are obtained by
modifying the set of velocities of the original system near the constraint boundary. The method of removing the state
constraints in the construction of reachable sets for differential inclusions was proposed in [10]. In this paper the tube
of trajectories of the differential inclusion with convex state constraint was approximated by the solutions of a family
of differential inclusions without constraints depending on a matrix penalty parameter in theirs right-hand sides.
In [11] the state constraints were removed by restricting velocities of the original system near the constraint
border. The right-hand side of the approximating system here depends on a scalar penalty, and the reachable sets of
this system tends from the inside to the reachable set of the original system when the penalty tends to infinity.
In this paper we consider another procedure for removing state constraints. The procedure is based on construc-
tion of an auxiliary control system without state constraints with the right-hand side depending on a small scalar
parameter [12]. This construction assumes the existence of Lipschitz continuous feedback control, which ensures the
viability of the trajectories of closed system in a small neighborhood of state constraints. The existence of such control
was established earlier for the case of control-affine nonlinear system with a smooth state constraint boundary.
Here we extend these results to a broader class of control systems and constraints with a piecewise smooth
boundary. We prove the convergence of reachable sets of auxiliary systems in the Hausdorff metric to the reachable
set of the original system when the small parameter tends to zero. The estimate of the rate of convergence is also given
.

Application of Mathematics in Technical and Natural Sciences

AIP Conf. Proc. 1773, 050003-1–050003-9; doi: 10.1063/1.4964973
Published by AIP Publishing. 978-0-7354-1431-0/$30.00

050003-1
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 CONTROL SYSTEM AND STATE CONSTRAINTS
Consider the control system
ẋ(t) = f (x(t), u(t)), t0 ≤ t ≤ θ, x(t0 ) = x0 (1)
where x(t) ∈ Rn is a state vector and u(t) is a control. The constraints on the control have the form

u(t) ∈ U, a.e. t ∈ [t0 , θ], (2)

where U is a compact set in Rr , the controls are measurable functions u : [t0 , θ] → U, we use U to denote the set of
the controls U = {u(·) ∈ L∞ [t0 , θ] : u(t) ∈ U, a.e. t ∈ [t0 , θ]}.
We suppose that the mapping f (x, u) : Rn × U → Rn satisfies the following conditions
1) f (x, u) is continuous in x, u and locally Lipschitz in x uniformly in u ∈ U;
2) linear growth condition: there exists C > 0 such that  f (x, u) ≤ C(1 + x), (x, u) ∈ Rn × U.
Further we use the following notation. By A we denote the transpose of a real matrix A, 0 stands for a zero
1
vector of appropriate dimension. For x, y ∈ Rn let (x, y) = x y be an inner product, x = (x1 , . . . , xn ), x = (x, x) 2 be
an Euclidean norm, and Br ( x̄): Br ( x̄) = {x ∈ R : x − x̄ ≤ r} be a ball of radius r > 0 centered at x̄. For a set S ⊂ Rn
n

let ∂S , intS , clS , coS be a boundary, an interior, a closure, and a convex hull of S respectively; ∇g(x) is the gradient
of a function g(x) at the point x, h(A, B) is the Hausdorff distance between two sets A, B ⊂ Rn , comp(Rn ), conv(Rn )
denote the families of compact and compact convex subsets of Rn respectively.
The control system (1) is equivalent to the differential inclusion

ẋ(t) ∈ F(x(t)), x(t0 ) = x0 , (3)

where set-valued mapping F : Rn → comp(Rn ), defined by the equality F(x) = f (x, U), is locally Lipschitz with re-
spect to the Hausdorff distance. Let x(t, u(·), x0 ) denote the solution of system (1)— an absolutely continuous function
satisfying (1) a.e. in [t0 , θ] and the initial condition x(t0 ) = x0 .
The state constraints are given by the inclusion

x(t) ∈ S , t ∈ [t0 , θ], (4)

S ⊂ Rn is a given set, further we assume for simplicity that S is a compact set.

The reachable set G0 (θ) of the system (1) with state constraints (4) at time θ is the set of all states x that can be
reached in prescribed time interval [t0 , θ] along the trajectories satisfying the state constraints:

G0 (θ) = {x(θ, u(·), x0 ) : u(·) ∈ U, x(t, u(·), x0 ) ∈ S , t ∈ [t0 , θ]}.

State constraints
We will consider the state constraints of the following form S = {x ∈ Rn : g(x) ≤ 0}. Assume that g(x) =
max0≤i≤m gi (x), where gi (x), i = 1, ..., m, are continuously differentiable functions with Lipschitz continuous gra-
dients, and x0 ∈ S . For x ∈ Rn , we set I(x) = {i ∈ {1, . . . m} : gi (x) = g(x)}. Further, we assume that
Assumption 1 At the points x ∈ ∂S , the gradients ∇gi (x), i ∈ I(x), are positively linearly independent.
We assume also the following inward-pointing condition [13, 14]

∀x ∈ ∂S coF(x) ∩ intT S (x)  ∅. (5)

Here T S (x) = {d ∈ Rn : limξ→+0 ξ−1 d(x + ξd, S ) = 0} is a tangent cone to S at point x, d(x, S ) = miny∈S x − y. This
ensures that G0 (θ)  ∅ for any x0 ∈ S . Under Assumption 1 condition (5) can be written in the form [11]

m
max min ( λi ∇gi (x), f ) < 0 (6)
λ∈Λ(x) f ∈coF(x)
i=1

for all {x : g(x) = 0}, where Λ(x) = {λ ∈ Rm : λi ≥ 0, m i=1 λi = 1; λi = 0, i  I(x)}. Note, that Assumption 1 and (6)
imply that ∂S = {x ∈ Rn : g(x) = 0}. Inequality (6) is equivalent to the following [11]

050003-2
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 Assumption 2
min max(∇gi (x), f ) < 0, ∀x : g(x) = 0. (7)
f ∈coF(x) i∈I(x)

By standard reasoning, using compactness of S and Lipschitz continuity of ∇gi (x), i = 1, ..., m, we prove the following

Assertion 1 If condition (6) holds , then there exist σ > 0, ρ > 0 such that the inequality

min max(∇gi (x), f ) < −ρ, (8)

f ∈coF(x) i∈I(x)

is valid for all points of the set S σ = {x : 0 ≤ g(x) ≤ σ}.

In what follows, we will use the following extension of condition (8).

Assumption 3 There exist σ > 0, ρ > 0 and a Lipschitz continuous function f¯(x) defined on the set S σ such that

max(∇gi (x), f¯(x)) < −ρ, f¯(x) ∈ coF(x), (9)

i∈I(x)

for all x ∈ S σ .

PROBLEM STATEMENT AND MAIN RESULTS

We consider here the following problems.

Problem 1 (Internal approximation). Construct the differential inclusion (control system)

ẋ ∈ Fk (x) (10)

such that

1) Fk (x) is locally Lipschitz in x;

2) Fk (x) ⊂ F(x), x ∈ S ;
3) all the trajectories of differential inclusion starting from x0 ∈ S satisfy the state constraint x(t) ∈ S ;
4) Gk (θ) ⊂ G0 (θ), h(Gk (θ), G0 (θ)) → 0, k → ∞.

Here k > 0 is a penalty coefficient, Gk (θ) is the reachable set of system (10) without state constraints.

Problem 2 (External approximation). Construct the control system

ẋ(t) = fε (x(t), u(t)), x(t0 ) = x0 , (11)

depending on a small parameter ε > 0 such that

1) the mapping fε (x, u) is defined for x from a certain neighborhood of S and for u from U; fε (x, u) is continuous
in (x, u) and locally Lipschitz in x uniformly in u ∈ U;
2) fε (x, U) ⊂ f (x, U), fε (x, U) = f (x, U) for x ∈ S ;
3) Gε (θ) → G0 (θ) in the Hausdorff metric as ε → 0, where Gε (θ) is the reachable set of system (11) without state
constraints.

Thus, the original control system is substituted by a family of control systems without state constraints those reachable
sets approximate the reachable set of the original system. We say that systems (10), (11) are the approximating systems
for (1).

050003-3
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 Internal approximation of reachable sets
The considered here method of approximation of reachable sets is close to the barrier function method in constrained
optimization. State constraints are eliminated due to the restriction of the set of velocities of the original system near
the boundary of the constraints. This does not let the trajectories of the approximating system intersect this boundary,
and the reachable set of the approximating system approximates the reachable set of the system with state constraints
from inside.
Define the set-valued mapping Fk (x) by the system of inequalities Fk (x) = { f ∈ f (x, U) : ∇gi (x) f ≤ −kgi (x), i =
1, ..., m}, where k > 0 is a penalty parameter.
Theorem 1 [11] Let gi (x), i = 1, ..., m, are convex functions, and F(x) is a convex set for x ∈ S . Let Assumptions
1, 2 are satisfied and x0 is an interior point of S . Then the following holds true:
There exists k0 > 0 such that
1. k ≥ k0 implies Fk (x)  ∅ ∀x ∈ S , and Fk (x) is Lipschitz in x on S .
2. Gk (θ)  ∅, Gk1 (θ) ⊂ Gk2 (θ) ⊂ G0 (θ), k0 ≤ k1 < k2 .

3. There exists M > 0 such that for k > k0 h(Gk (θ), G0 (θ))  M 1k .

In studying the properties of the value function in problems of optimal control with state constraints and its
connection with generalized solutions of Hamilton–Jacobi equations, the theorems on approximation of admissible
trajectories are used (see, e.g., [13, 15, 14]). In the proof of Theorem 1, we use the results of [15], which establish
the possibility of approximation of the trajectories of a system with state constraints by the trajectories of the same
system lying strictly inside the state constraints.

External approximation of reachable sets

Assuming Assumption 3 to be fulfilled, we define right-hand side fε (x, u) of control system (11) on the set {x ∈ Rn :
g(x) ≤ σ} as follows. Choose 0 < ε < σ. Let hε (τ) : R → R be a continuously differentiable function such that
0 ≤ hε (τ) ≤ 1, hε (τ) = 1 for τ < 0, hε (τ) = 0 for τ > ε and pε (x) = hε (g(x)). Define

pε (x) f (x, u) + (1 − pε (x)) f¯(x) if g(x) > 0,
fε (x, u) =
f (x, u) if g(x) ≤ 0.

As hε (τ) we can take, for example, a linear-quadratic function.

Theorem 2 Let f (x, u) and the constraints of the problem satisfy Assumptions 1, 3. Then
1) for 0 < ε < σ the mapping fε (x, u) is continuous on {x ∈ Rn : g(x) ≤ σ} × U and Lipschitz in x uniformly in
u ∈ U;
2) for any u(·) ∈ U the solution xε (t) of system (11) with the initial state xε (t0 ) = x0 ∈ S is defined on [t0 , θ] and
satisfies the inequality g(xε (t)) ≤ ε, t ∈ [t0 , θ].
Fix ε > 0. On the set S 1 × U, where S 1 = {x : g(x) ≤ 0}, fε (x, u) coincides with f (x, u); hence it is continuous.
For (x, u) ∈ S 2 × U, S 2 = {x : 0 ≤ g(x) ≤ σ}, fε (x, u) is continuous as the superposition of continuous functions.
For x ∈ S 1 ∩ S 2 , continuity of fε (x, u) is obvious. To prove the Lipschitz condition for fε (x, u), note, that there exist
constants L1 , L2 > 0, independent of u, such that ∀i = 1, 2 | fε (x, u) − fε (y, u)| ≤ Li x − y, ∀x, y ∈ S i , ∀u ∈ U. For
x, y ∈ S 1 (S 2 ) the inequality follows from the Lipschitz continuity in x of f, f¯. We take x ∈ S 1 , y ∈ S 2 , and connect
x, y by a line segment. At the end points of the segment the function g takes values of different signs, hence, on this
segment there exists z, such that g(x) = 0. The inclusions z ∈ S i , i = 1, 2 imply

| fε (x, u) − fε (y, u)| ≤ | fε (x, u) − fε (z, u)| + | fε (z, u) − fε (y, u)|

≤ L1 x − z + L2 y − z ≤ max{L1 , L2 }(x − z + y − z) = max{L1 , L2 }x − y, ∀u ∈ U.

Consider the solution xε (t) of system (11), corresponding to u(·) ∈ U. Since fε (x, u) is a convex combination of
vectors f (x, u) and f¯(x) from coF(x), we have the inclusion ẋε (t) ∈ coF(xε (t)). Let us prove that this trajectory does

050003-4
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 not leave the set {x : g(x) ≤ σ}. Let γ∗ be the maximum among numbers γ not exceeding θ such that the solution
xε (t) is defined on [t0 , γ]. Let us show that the inequality g(xε (t)) ≤ ε is fulfilled at all points [t0 , γ∗ ]. Assume by
contradiction that g(xε (t∗ )) > ε for some t∗ ∈ [t0 , γ∗ ]. Let t∗ = max{t : t ∈ [t0 , t∗ ], g(xε(t)) = ε}. The function g(xε (t)) is
Lipschitz and, hence, is differentiable almost everywhere. We estimate the quantity dtd g(xε (t)) at the points t ∈ [t∗ , t∗ ],
where this derivative and the derivative ẋε (t) exist. Denote Δxε (t) = xε (t + δt) − xε (t), then

gi (xε (t + δt)) − gi (xε (t)) = (∇gi (xε (t)), Δxε (t)) + oi (Δxε (t)), i = 1, . . . , k, (12)

where oi (η)/η → 0 as η → 0. Substituting Δxε (t) = ẋε (t)Δt + o(Δt) (o(Δt)/Δt → 0, Δt → 0) into equality (12), we get

gi (xε (t + Δt)) − gi (xε (t)) = (∇gi (xε (t)), ẋε (t))Δt + αi (Δt), i = 1, . . . , k, (13)

where αi (Δt) = (∇gi (xε (t), o(Δt)) + oi ( ẋε (t)Δt + o(Δt)). Obviously, αi (Δt)/Δt → 0 as Δt → 0.
For i ∈ I(xε (t)), we have gi (xε (t)) = g(xε (t)). If i  I(xε (t)), then gi (xε (t)) < g(xε (t)), consequently, by the
continuity of g(x), gi (x), xε (t) for sufficiently small Δt, we have gi (xε (t + Δt)) < g(xε (t + Δt)). Thus, g(xε (t + Δt)) =
maxi∈I(xε (t)) gi (xε (t + Δt)). In view of the above, passing in both sides of equality (13) to the maximum over i ∈ I(xε (t)),
we obtain g(xε (t + Δt)) − g(xε (t)) ≤ Δt maxi∈I(xε (t)) (∇gi (xε (t)), ẋε (t)) + maxi∈I(xε (t)) αi (Δt). In the limit as Δt → 0, we
obtain from the above inequality that dtd g(xε (t)) ≤ maxi∈I(xε (t)) (∇gi (xε (t)), ẋε (t)). Since g(xε (t)) ≥ ε on the interval
[t∗ , t∗ ], we have hε (g(xε (t)) = 0 and, consequently, ẋε (t)) = fε (xε (t), u(t)) = f¯(xε (t)). From the definition of f¯(x),
we find that dtd g(xε (t)) ≤ −ρ < 0 for almost all t ∈ [t∗ , t∗ ], which implies that g(xε (t∗ )) > g(xε (t∗ )) contrary to the
assumption. The theorem is proved.
Denote by Gε (θ)) the reachable set of system (11). The following theorem states that this reachable set provides
an external estimate for G0 (θ) and approximate it under ε → 0.
Theorem 3 Let f (x, u) and the constraints of the problem satisfy Assumptions 1, 3 and the functions gi (x), i =
1, ..., m are convex. Then, for any 0 < ε < σ we have the inclusion G0 (θ) ⊂ Gε (θ). There exists a constant L > 0 such
that h(G0 (θ), Gε (θ)) ≤ Lε.
The principal components of the proof are as follows. First, the trajectories of auxiliary system (11) are the trajectories
of the differential inclusion
ẋ(t) ∈ coF(x(t)), x(t0 ) = x0 , (14)
where F(x) = f (x, U) is a compact-valued Lipschitz set-valued mapping with a Lipschitz constant not dependent on
ε. By the neighboring feasible trajectory theorem [14] Assumption 2 implies the existence of a constant L with the
following property. For any trajectory x(t) of system (14) starting from the point x(t0 ) = x0 ∈ S there exists a trajectory
x̂(t), x̂(t0 ) = x0 , that satisfies the state constraints and the inequality maxt0 ≤t≤θ x(t)− x̂(t) ≤ L maxt0 ≤t≤θ max{g(x(t)), 0}.
To complete the proof, we apply the following
Lemma 1 Let S = {x ∈ Rn : g(x) ≤ 0, i = 1, ..., m} where gi (x), i = 1, ..., m, are convex continuously differen-
tiable functions satisfying Assumption 2. Let D be a bounded subset of Rn .Then there exists a constant M > 0 such
that
d(x∗ , S ) ≤ M max{ max gi (x∗ ), 0} ∀x∗ ∈ D.
i=1,...,m

The existence of Lipschitz continuous feedback control

The construction of approximating system ẋ(t) = fε (x(t), u(t)), x(t0 ) = x0 , is based on the assumption of existence of
Lipschitz feedback control that ensures the viability of trajectories of closed system in a small neighborhood of state
constraints. Here we prove the existence of this feedback control under Assumptions 1,2. Observe that Assumptions
1,2 imply the existence of σ0 > 0, ρ0 > 0 such that inequality

min max(∇gi (x), f ) < −ρ0 , (15)

f ∈coF(x) i∈I(x)

is valid for all points of the set S σ0 = {x ∈ Rn : 0 ≤ g(x) ≤ σ0 }. Let inequality (6) be fulfilled. For any σ > 0 the set
S σ  ∅ is compact. For k > 0, ρ > 0 define the following set-valued mapping
ρ
Fk (x) = { f ∈ coF(x) : (∇gi (x), f ) < −ρ − k(gi (x) − g(x)), i = 1, ..., m}.

050003-5
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 Lemma 2 Let inequality (6) be fulfilled and 0 < ρ ≤ ρ0 /4. There exist k > 0, 0 < σ̄ ≤ σ0 such that ∀σ ≤ σ̄,
Fkρ (x)  ∅, x ∈ S σ .
Denote F ρ (x) = { f ∈ coF(x) : (∇gi (x), f ) < −ρ, i ∈ I(x)}.
Lemma 3 For any 0 < σ ≤ σ̄, 0 < ρ < ρ0 /4, k > 0 the following inclusion is true Fkρ (x) ⊂ F ρ (x), x ∈ S σ,
ρ
hence, F (x)  ∅.

Lemma 4 Let inequality (6) be fulfilled. There exist k > 0, 0 < σ1 ≤ σ0 , ρ1 > 0 such that for all σ ≤ σ1 , ρ < ρ1
the set-valued mapping Fkρ (x) with nonempty compact convex values satisfies a Lipschitz condition on S σ .

The proof of this Lemma is by induction in m, it is based on [11, Lemma 7].

Theorem 4 Let Assumptions 1,2 be satisfied. There exist σ > 0, ρ > 0 and a Lipschitz selector f¯(x) ∈ coF(x), x ∈
S σ , such that maxi∈I(x) (∇gi (x), f¯(x)) < −ρ, ∀x ∈ S σ . Any trajectory x(t) of differential equation ẋ = f¯(x) with initial
state x(t0 ) ∈ S σ remains (is viable) in S σ till the moment it hits the set ∂S , the function g(x(t)) is decreasing.
The proof is based on the theorem on parametrization of Lipschitz multi-functions [16].

Lipschitz continuous feedback for control-affine system

Consider the control-affine system ẋ(t) = f (x(t), u(t)) = f1 (x(t)) + f2 (x(t))u(t), x(t0 ) = x0 , where
f1 : Rn → Rn , f2 : Rn → Rn×r are continuously differentiable mappings, Rn×r is a linear space of n × r real
matrices. Assume that the constraints on control u(t) ∈ U, are given by the convex compact set U ⊂ Rr . In this case
coF(x) = f1 (x) + f2 (x)U. Inequality (8) for this system can be written as

min max[(∇gi (x), f1 (x)) + ∇gi (x) f2 (x)u] < −ρ0 . (16)
u∈U i∈I(x)

for x ∈ S σ0 . From Theorem 4 follows

Theorem 5 Let Assumption 3 and inequality (16) be fulfilled. There exist σ > 0, ρ > 0 and a Lipschitz continuous
feedback control ū(x) ∈ U, x ∈ S σ , such that

max[(∇gi (x), f1 (x)) + ∇gi (x) f2 (x)ū(x)] < −ρ0 .

i∈I(x)

Any trajectory x(t) of the system ẋ = f1 (x) + f2 (x)u(x) with initial condition x(t0 ) ∈ S σ remains (is viable) in S σ till
the moment it hits the set ∂S , the function g(x(t)) is decreasing.
Consider a special case when Lipschitz feedback ū(x) can be calculated explicitly. Let us assume that the values of
the control u belongs to a non-degenerate ellipsoid : U = {u ∈ Rr : (u − û) Q(u − û) ≤ 1}, where Q is a positive
definite r × r matrix, û ∈ Rr is a center of the ellipsoid. Assume that m = 1, the function g1 (x) we will denote as g(x).
Assumption 2 ((5)) for this system takes the form

∇g(x) f1 (x) + ∇g(x) f2 (x)û + min ∇g(x) f2 (x)v < 0

v∈V


for x ∈ S σ . Here V = {v : v Qv ≤ 1} is an ellipsoid centered at the origin. Assumption 1 in this case follows from
Assumption 2.
   
 following notation a(x)1 = ∇g(x)1 f1 (x) + ∇g(x) f2 (x)û, b (x) = ∇g(x) f2 (x), we get
Introducing the
minv∈V b (x)v = − b (x)Q b(x), where Q = (Q ) is the square root of a positive definite matrix Q−1 . Given
  −1 −2 −1 2

this notation condition (5) takes the form


a(x) + min b (x)v = a(x) − b (x)Q−1 b(x) < 0. (17)
v∈V

The minimum in (17) is attained on a vector v = v(x), where

Q−1 b(x)
v(x) =  . (18)
b (x)Q−1 b(x)

050003-6
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 Thus, for the function ū(x) = v(x) + û the inequality (∇g(x), f (x, ū(x))) < 0 holds. However, this function may not be
Lipschitz continuous. Modifying formula (18), we can ensure Lipschitz continuity of the function ū(x).
1
Inequality (17) implies that a(x) < 0 at the points x, where b(x) = 0. We substitute w = Q 2 v, then a(x) + b (x)v =
1
a(x)+b1 (x)w, where b1 (x) = Q− 2 b(x). The ellipsoid V transforms to the ball {w : (w, w) ≤ 1}. Consider a non-negative
function p(x) defined on S σ by the equality
⎧ √2
⎪
⎪
 2
⎨ a(x)+ a (x)+(b1 (x)b1 (x)) if b1 (x)  0,
p(x) = ⎪⎪ b1 (x)b1 (x)
⎩ 0 if b1 (x) = 0.
Since a(x) < 0 for b1 (x) = 0, the function p(x) is continuously differentiable [17]. Sontag feedback control
1
w̄(x) = −p(x)b1 (x) satisfies the inequality a(x) + b1 (x)w̄(x) < 0, hence for ū(x) = Q− 2 w̄(x) + û the inequality
(∇g(x), f (x, ū(x))) < 0 holds. The function ū(x) is obviously Lipschitz. To ensure the condition ū(x) ∈ U let us modify
w̄(x) as follows. Let π(w) be the operator of metric projection onto the unit Euclidean ball in Rr , π(w) satisfies the
Lipschitz condition with the Lipschitz constant equal to unit. Setting w̄(x) = π(−p(x)b1(x)) we get

a(x) + b1 (x)w̄(x) = a(x) − b1 (x) = a(x) − b (x)Q−1 b(x) < 0
1 1
due to (17). So ū(x) = Q− 2 π(−p(x)Q− 2 f2 (x)∇g(x)) + û is the required control function.

NUMERICAL EXAMPLES
Consider a double integrator system
ẋ1 = x2 , ẋ2 = u, x1 (0) = 0, x2 (0) = 0, |u| ≤ 1, 0 ≤ t ≤ 2, (19)
2 
with the state constrains S = {x ∈ R : g(x) = x22 − 1 ≤ 0}, x = (x1 , x2 ) .
Let us take the function hε (τ) in the form
⎧
⎪
⎪
⎪ 1 τ < 0,
⎪
⎪
⎪ 2
⎨ 1 − 2τ ε2
2 0 ≤ τ ≤ ε/2,
hε (τ) = ⎪
⎪
⎪
⎪
⎪
2(τ−ε)
ε/2 ≤ τ ≤ ε,
⎪
⎩ ε2
0 τ > ε.
There exists a control ū(x) such that ∇g(x)(x2 , ū) = g x2 (x2 )ū(x) < 0 in the neighborhood of ∂S = {x ∈ R2 : |x2 | = 1},
e.g., ⎧
⎪
⎪ −1 if x2 ≥ 1,
⎪
⎨
ū(x) = ⎪ ⎪ −x2 if −1 < x2 < 1,
⎪
⎩ 1 if x2 ≤ −1.
The auxiliary system takes the form
ẋ1 = x2 , ẋ2 = pε (x2 , u), (20)
where pε (x2 , u) = hε (x22 − 1) u + (1 − hε(x22 − 1)) ū(x). Note that the first equation of the system (19) remains unchanged
as this equation does not contain u. We apply the Pontryagin maximum principle to compute the control functions that
steers the trajectory of the system to the boundary of the reachable set. Figure 1 (left) shows the results of numerical
simulation for the given algorithms. The boundaries of external approximating sets for different values of ε are shown
by thick lines. The thin lines here indicate the boundaries of internal approximating sets for some values of penalty
parameter.
Consider another example. Let the control system be a triple integrator
ẋ1 = x2 , ẋ2 = x3 , ẋ3 = u, xi (0) = 0, i = 1, 2, 3, |u| ≤ 1, 0 ≤ t ≤ 2, (21)
under the state constraints S = {x ∈ R3 : g(x) = x23 − 1 ≤ 0}, x = (x1 , x2 , x3 ) . As a control ū(x) we can take, for
example, ū(x) = − π2 arctang(x3 ). The auxiliary system is as follows
ẋ1 = x2 , ẋ2 = x3 , ẋ3 = pε (x3 , u), (22)
where pε (x3 , u) = hε (x23
− 1) u + (1 − hε (x23
− 1)) ū(x). Figure 1 (right) shows the external estimate for the reachable
set, given by the proposed algorithm under ε = 0.1.

050003-7
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 x_2
x_1 -1.0 -1.0
0.0
1.5 1.0
0.0
1.0

1 1.0

0.5

0
x2

x_3 0.0

−0.5

-1.0
−1
-1.0
0.0
x_2 1.0
−1.5
−2 −1.5 −1 −0.5 0 0.5 1 1.5 2
x
1

FIGURE 1: Internal and external approximating reachable sets

CONCLUSION
The algorithms for removing state constraints in the construction of reachable sets of nonlinear control system are pro-
posed. In this paper we assume that constraints are given as a level set of a piecewise smooth function. The considered
algorithms involves removal of the state constraints by replacing the original system with an auxiliary constraint-
free one, depending on a scalar penalty parameter. These algorithms are the analogies of the barrier and the external
penalty functions methods. The analog of the external penalty functions methods uses Lipschitz continuous feedback
control, which ensures the viability of trajectories of closed system in a small neighborhood of state constraint. Ear-
lier the existence of such control was established for the case of control-affine nonlinear system with a smooth state
constraint boundary. In this paper we extend these results to a broader class of control systems and constraints with a
piecewise smooth boundary. We prove the convergence of reachable sets of auxiliary systems in the Hausdorff metric
to the reachable set of the original system when the small parameter tends to zero and give the estimate of the rate of
convergence.

ACKNOWLEDGEMENTS
The research is supported by Russian Science Foundation, project No 16-11-10146.

REFERENCES
[1] N. N. Krasovskii and A. I. Subbotin, Game-Theoretical Control Problems (Springer-Verlag, New York,
1988), pp. 25–30.
[2] A. B. Kurzhanski and I. Valyi, Ellipsoidal Calculus for Estimation and Control (Birkhäuser, Boston, 1997),
pp. 25–30
[3] A. V. Lotov (1975) Z. Vycisl. Mat. i Mat. Fiz 15, 67–78. [in Russian]
[4] F. Lempio and V. M. Veliov (1998) GAMM Mitt. Ges. Angew. Math. Mech. 21, 103–135.
[5] A. B. Kurzhanski, I. M. Mitchell, and P. Varaiya (2006) J. Optim. Theory Appl. 128, 499–521.
[6] S. V. Grigor’eva, V. Y. Pakhotinskikh, A. A. Uspenskii, and V. N. Ushakov (2005) Sbornik Mathematics 196,
513–539. [in Russian]
[7] R. Baier, I. A. Chahma, and F. Lempio (2007) SIAM J. Optim. 18, 1004–1026.
[8] M. I. Gusev (2012) Automation and Remote Control 73, 450–461.
[9] E. K. Kostousova (2011) Automation and Remote Control 72, 1841–1851.
[10] A. B. Kurzhanskii and T. F. Filippova (1987) Differents. Uravneniya 23, 1303–1315. [in Russian]

050003-8
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40
 [11] M. I. Gusev (2014) Proceedings of the Steklov Institute of Mathematics 287, Suppl. 1, 77–92.
[12] M. I. Gusev (2016) Proceedings of the Steklov Institute of Mathematics 293, Suppl. 1, 66–74.
[13] F. Forcellini and F. Rampazzo (1999) J. Differential Integral Equations 12, 471–497.
[14] A. Bressan and G. Facchi (2011) J. Differential Equations 250, 2267–2281.
[15] R. J. Stern 2004) SIAM J. Control and Optim. 43, 697–707.
[16] A. Ornelas (1990) Rendiconti del Seminario Matematico della Università di Padova 83, 33–44.
[17] E. D. Sontag (1989) System and Control Letters 13, 117–123.

050003-9
Reuse of AIP Publishing content is subject to the terms at: https://publishing.aip.org/authors/rights-and-permissions IP: 195.19.132.64 On: Mon, 24 Oct 2016 11:08:40