Optimal Control Problems Arising in Forest Management: Alexander J. Zaslavski

SPRINGER BRIEFS IN OPTIMIZATION
Alexander J. Zaslavski
Optimal Control
Problems
Arising in Forest
Management
123
SpringerBriefs in Optimization
Series Editors
Sergiy Butenko
Mirjam Dür
Panos M. Pardalos
János D. Pintér
Stephen M. Robinson
Tamás Terlaky
My T. Thai
SpringerBriefs in Optimization showcases algorithmic and theoretical tech-

niques, case studies, and applications within the broad-based field of optimization.
Manuscripts related to the ever-growing applications of optimization in applied
mathematics, engineering, medicine, economics, and other applied sciences are
encouraged.
More information about this series at http://www.springer.com/series/8918

Optimal Control Problems

Arising in Forest
Management
123
Department of Mathematics
The Technion – Israel Institute of Techn
Rishon LeZion, Israel
ISSN 2190-8354 ISSN 2191-575X (electronic)

SpringerBriefs in Optimization
ISBN 978-3-030-23586-4 ISBN 978-3-030-23587-1 (eBook)
https://doi.org/10.1007/978-3-030-23587-1
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AG.
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
The growing importance of optimal control has been recognized in recent years.
This is due not only to impressive theoretical developments but also because of
numerous applications to engineering, economics, life sciences, etc. This book
is devoted to the study of a class of optimal control problems arising in forest
management. The forest management problem is an important and interesting topic
in mathematical economics that was studied by many researchers including Nobel
laureate P.A. Samuelson [68]. As usual, for this problem, the existence of optimal
solutions over infinite horizon and the structure of solutions on finite intervals are
under consideration. In our books [84, 86], we study a class of discrete-time optimal
control problems which describe many models of economic dynamics except for
the model of forest management. This happens because some assumptions posed in
[84, 86], which are true for many models of economic dynamics, do not hold for
the model of forest management. By this reason, the forest management problem
is not a particular case of general models of economic dynamics and is studied
separately in the literature. In this book, we study the forest management problem
using the approach introduced and employed in our research [80, 81, 83]. Namely,
we analyze a class of optimal control problems which contains, as a particular case,
the forest management problem. For this class of problems, we show the existence
of optimal solutions over infinite horizon and study the structure of approximate
solutions on finite intervals and their turnpike properties, the stability of the turnpike
phenomenon, and the structure of approximate solutions on finite intervals in the
regions close to the endpoints.
In Chap. 1, we provide some preliminary knowledge on turnpike properties. The
forest management problem is discussed in Chap. 2, which also contains existence
results for infinite horizon problems. In Chap. 3, we establish the turnpike properties
of approximate solutions. Chapter 4 contains generic turnpike results. We consider
a class of optimal control problems which is identified with a complete metric space
of objective functions and show the existence of a Gδ everywhere dense subset of
the metric space, which is a countable intersection of open everywhere dense sets,
such that the turnpike property holds for any of its element. Chapter 5 is devoted to
v
vi Preface
the study of the structure of approximate solutions on finite intervals in the regions
close to the endpoints. In Chap. 6, we again consider the forest management problem
and show that the results of Chaps. 3 and 5 are true for it.
Rishon LeZion, Israel Alexander J. Zaslavski

October 30, 2018
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Convex Discrete-Time Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 The Turnpike Phenomenon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Infinite Horizon Optimal Control Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1 The Forest Management Problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Infinite Horizon Problems Without Discounting . . . . . . . . . . . . . . . . . . . . . 16
2.3 Auxiliary Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.4 Proofs of Theorems 2.6 and 2.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.5 Proof of Theorem 2.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.6 Proof of Theorem 2.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.7 Infinite Horizon Problems with Discounting . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.8 An Auxiliary Result for Theorem 2.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.9 Proof of Theorem 2.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.10 An Application to the Forest Management Problem . . . . . . . . . . . . . . . . . 30
3 Turnpike Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1 Preliminaries and Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3 Proof of Theorem 3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.4 Proof of Theorem 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.5 Proof of Theorem 3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.6 Proof of Theorem 3.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.7 Proof of Theorem 3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.8 Stability Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.9 Agreeable Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
4 Generic Turnpike Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Equivalence of the Turnpike Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.3 Generic Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
vii
viii Contents
5 Structure of Solutions in the Regions Close to the Endpoints . . . . . . . . . . 73

5.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.2 Lagrange Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.4 Proof of Theorem 5.13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.5 The First Class of Bolza Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.7 Proof of Theorem 5.15 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.8 The Second Class of Bolza Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.9 Auxiliary Results for Theorem 5.17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.10 Proof of Theorem 5.17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6 Applications to the Forest Management Problem . . . . . . . . . . . . . . . . . . . . . . . 109
6.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.3 Turnpike Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
6.4 Generic Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Chapter 1
Introduction
The study of optimal control problems and variational problems defined on infinite
intervals and on sufficiently large intervals has been a rapidly growing area of
research [9, 10, 16, 18, 24, 37, 43, 48, 57, 59, 66, 76, 84–86] which has various
applications in engineering [1, 44, 87], in models of economic growth [2, 5, 14, 20,
23, 28, 38–40, 47, 52–55, 60, 61, 63, 65, 67, 68, 70, 77, 80, 81, 83], in the game theory
[29, 32, 42, 74, 82], in infinite discrete models of solid-state physics related to dis-
locations in one-dimensional crystals [6, 71], and in the theory of thermodynamical
equilibrium for materials [19, 45, 50, 51]. Discrete-time problems were considered
in [7, 8, 13, 21, 26, 30, 33, 72, 73, 78, 79] while continuous-time problems were
studied in [3, 4, 11, 12, 15, 17, 22, 25, 27, 31, 41, 46, 49, 56, 58, 62, 69, 75].
In this chapter we discuss turnpike properties for a class of simple convex
dynamic optimization problems.
1.1 Convex Discrete-Time Problems
Let R n be the n-dimensional Euclidean space with the inner product ·, · which
induces the norm
n 1/2

|x| = xi2 , x = (x1 , . . . , xn ) ∈ R n .
i=1
Let v : R n × R n → R 1 be bounded from below function. We consider the

minimization problem
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 1

A. J. Zaslavski, Optimal Control Problems Arising in Forest Management,
SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1_1
2 1 Introduction
−1
T
v(xi , xi+1 ) → min, (P0 )
i=0
such that {xi }Ti=0 ⊂ R n and x0 = z, xT = y,
where T is a natural number and the points y, z ∈ R n .

The interest in discrete-time optimal problems of type (P0 ) stems from the study
of various optimization problems which can be reduced to it, e.g., continuous-time
control systems which are represented by ordinary differential equations whose cost
integrand contains a discounting factor [43], tracking problems in engineering [1,
44, 87], the study of Frenkel-Kontorova model [6, 71], and the analysis of a long
slender bar of a polymeric material under tension in [19, 45, 50, 51]. Optimization
problems of the type (P0 ) were considered in [72, 73].
In this section we suppose that the function v : R n × R n → R 1 is strictly convex
and differentiable and satisfies the growth condition
v(y, z)/(|y| + |z|) → ∞ as |y| + |z| → ∞. (1.1)
We intend to study the behavior of solutions of the problem (P0 ) when the points
y, z and the real number T vary and T is sufficiently large. Namely, we are interested
to study a turnpike property of solutions of (P0 ) which is independent of the length
of the interval T , for all sufficiently large intervals. To have this property means,
roughly speaking, that solutions of the optimal control problems are determined
mainly by the objective function v and are essentially independent of T , y, and z.
Turnpike properties are well known in mathematical economics. The term was first
coined by Samuelson in 1948 (see [67]) where he showed that an efficient expanding
economy would spend most of the time in the vicinity of a balanced equilibrium path
(also called von Neumann path). This property was further investigated for optimal
trajectories of models of economic dynamics (see, for example, [47, 53, 65] and the
references mentioned there). Many turnpike results are collected in [76, 84, 86].
In order to meet our goal we consider the auxiliary optimization problem
v(x, x) → min, x ∈ R n . (P1 )
It follows from the strict convexity of v and (1.1) that the problem (P1 ) has a unique
solution x̄. Let
∇v(x̄, x̄) = (l1 , l2 ), (1.2)
where l1 , l2 ∈ R n . Since x̄ is a solution of (P1 ) it follows from (1.2) that for each
h ∈ Rn,
l1 , h + l2 , h = (l1 , l2 ), (h, h)

= lim t −1 [v(x̄ + th, x̄ + th) − v(x̄, x̄)] ≥ 0.
t→0+
1.1 Convex Discrete-Time Problems 3
Thus
l1 + l2 , h ≥ 0 for all h ∈ R n ,
l2 = −l1 and
∇v(x̄, x̄) = (l1 , −l1 ), (1.3)
For each (y, z) ∈ R n × R n set
L(y, z) = v(y, z) − v(x̄, x̄) − ∇v(x̄, x̄), (y − x̄, z − x̄)

= v(y, z) − v(x̄, x̄) − l1 , y − z. (1.4)
It is not difficult to verify that the function L : R n × R n → R 1 is differentiable and

strictly convex. It follows from (1.1) and (1.4) that
L(y, z)/(|y| + |z|) → ∞ as |y| + |z| → ∞. (1.5)
Since the functions v and L are both strictly convex it follows from (1.4) that
L(y, z) ≥ 0 for all (y, z) ∈ R n × R n (1.6)
and
L(y, z) = 0 if and only if y = x̄, z = x̄. (1.7)
We claim that the function L : R n × R n → R 1 has the following property:

(C) If a sequence {(yi , zi )}∞
i=1 ⊂ R × R satisfies the equality
n n
lim L(yi , zi ) = 0,
i→∞
then
lim (yi , zi ) = (x̄, x̄).

i→∞
Assume that a sequence {(yi , zi )}∞i=1 ⊂ R × R satisfies limi→∞ L(yi , zi ) = 0.

n n
In view of (1.5) the sequence {(yi , zi )}∞

i=1 is bounded. Let (y, z) be its limit point.
Then it is easy to see that the equality
L(y, z) = lim L(yi , zi ) = 0

i→∞
holds and by (1.7) (y, z) = (x̄, x̄). This implies that (x̄, x̄) = limi→∞ (yi , zi ).
Thus the property (C) holds, as claimed.
4 1 Introduction
Consider an auxiliary minimization problem

−1
T
L(xi , xi+1 ) → min, (P2 )
i=0
such that {xi }Ti=0 ⊂ R n and x0 = z, xT = y,
where T is a natural number and the points y, z ∈ R n .

It follows from (1.4) that for any integer T ≥ 1 and any sequence {xi }Ti=0 ⊂ R n ,
we have
−1
T −1
T −1
T
L(xi , xi+1 ) = v(xi , xi+1 ) − T v(x̄, x̄) − l1 , xi − xi+1
i=0 i=0 i=0
−1
T
= v(xi , xi+1 ) − T v(x̄, x̄) − l1 , x0 − xT . (1.8)
i=0
Relation (1.8) implies that the problems (P0 ) and (P2 ) are equivalent. Namely,
{xi }Ti=0 ⊂ R n is a solution of the problem (P0 ) if and only if it is a solution of
the problem (P2 ).
Let T be a natural number and Δ ≥ 0. A sequence {xi }Ti=0 ⊂ R n is called (Δ)-
optimal if for any sequence {xi
}Ti=0 ⊂ R n satisfying xi = xi
, i = 0, T the inequality
−1
T −1
T
v(xi , xi+1 ) ≤ v(xi
, xi+1

)+Δ
i=0 i=0
holds. Clearly, if a sequence {xi }Ti=0 ⊂ R n is (0)-optimal, then it is a solution of the

problems (P0 ) and (P2 ) with z = x0 and y = xT .
We prove the following existence result.
Proposition 1.1 Let T > 1 be an integer and y, z ∈ R n . Then the problem (P0 )
has a solution.
Proof It is sufficient to show that the problem (P2 ) has a solution. Consider a
sequence {xi
}Ti=0 ⊂ R n such that x0
= z, xT
= y. Set
−1
T
M1 = L(xi
, xi+1

)
i=0
and
T −1

M2 = inf L(xi , xi+1 ) : {xi }Ti=0 ⊂ R , x0 = z, xT = y .
n
(1.9)
i=0
Clearly,
0 ≤ M2 ≤ M1 .
We may assume without loss of generality that
M2 < M1 . (1.10)
(k)
There exists a sequence {xi }Ti=0 ⊂ R n , k = 1, 2, . . . such that for any natural
number k,
(k) (k)
x0 = z, xT = y (1.11)
and
−1
T
(k) (k)
lim L(xi , xi+1 ) = M2 . (1.12)
k→∞
i=0
In view of (1.10), (1.11), and (1.12) we may assume that
−1
T
(k) (k)
L(xi , xi+1 ) < M1 for all integers k ≥ 1. (1.13)
i=0
By (1.13) and (1.5) there is M3 > 0 such that
|xi(k) | ≤ M3 for all i = 0, . . . , T and all integers k ≥ 1. (1.14)
In view of (1.14), extracting subsequences, using diagonalization process and re-

indexing, if necessary, we may assume without loss of generality that for each i ∈
{0, . . . , T } there exists
(k)

xi = lim xi . (1.15)
k→∞
By (1.15) and (1.11),
x0 = z,
xT = y. (1.16)
It follows from (1.15) and (1.12) that
−1
T
xi ,
L( xi+1 ) = M2 .
i=0
6 1 Introduction
Together with (1.16) and (1.9) this implies that { xi }Ti=0 is a solution of the
problem (P2 ). This completes the proof of Proposition 1.1.
Denote by Card (A) the cardinality of a set A.
The following result establishes a turnpike property for approximate solutions of
the problem (P0 ).
Proposition 1.2 Let M1 , M2 , be positive numbers. Then there exists a natural
number k0 such that for each integer T > 1 and each (M1 )-optimal sequence
{xi }Ti=0 ⊂ R n satisfying
|x0 | ≤ M2 , |xT | ≤ M2 , (1.17)
the following inequality holds:
Card({i ∈ {0, . . . , T − 1} : |xi − x̄| + |xi+1 − x̄| > }) ≤ k0 .
Proof By condition (C) there is δ > 0 such that for each (y, z) ∈ R n ×R n satisfying
L(y, z) ≤ δ, (1.18)
we have
|y − x̄| + |z − x̄| ≤ . (1.19)
Set
M3 = sup{L(y, z) : y, z ∈ R n and |y| + |z| ≤ |x̄| + M2 }. (1.20)
Choose a natural number
k0 > δ −1 (M1 + 2M3 ). (1.21)
Assume that an integer T > 1 and that an (M1 )-optimal sequence {xi }Ti=0 ⊂ R n
satisfies (1.17). Set
y0 = x0 , yT = xT , yi = x̄, i = 1, . . . , T − 1. (1.22)
Since the sequence {xi }Ti=0 is (M1 )-optimal it follows from (1.22) that
−1
T −1
T
v(xi , xi+1 ) ≤ v(yi , yi+1 ) + M1 .
i=0 i=0
Together with (1.7), (1.8), and (1.22) this implies that
−1
T −1
T
L(xi , xi+1 ) ≤ L(yi , yi+1 ) + M1 = L(x0 , x̄) + L(x̄, xT ) + M1 .
i=0 i=0
Combined with (1.17) and (1.20) this implies that
−1
T
L(xi , xi+1 ) ≤ M1 + 2M3 .
i=0
It follows from the choice of δ (see (1.18) and (1.19)), (1.21), and the inequality
above that
Card({i ∈ {0, . . . , T − 1} : |xi − x̄| + |xi+1 − x̄| > })
≤ Card({i ∈ {0, . . . , T − 1} : L(xi , xi+1 ) > δ})
−1
T
≤ δ −1 L(xi , xi+1 ) ≤ δ −1 (M1 + 2M3 ) ≤ k0 .
i=0
Proposition 1.2 is proved.

Proposition 1.2 implies the following turnpike result for exact solutions of the
problem (P0 ).
Proposition 1.3 Let M, be positive numbers. Then there exists a natural number
k0 such that for each integer T > 1, each y, z ∈ R n satisfying |y|, |z| ≤ M, and
each optimal solution {xi }Ti=0 ⊂ R n of the problem (P0 ) the following inequality
holds:
Card({i ∈ {0, . . . , T − 1} : |xi − x̄| + |xi+1 − x̄| > }) ≤ k0 .
It is easy now to see that the optimal solution {xi }Ti=0 of the problem (P0 ) spends
most of the time in an -neighborhood of x̄. By Proposition 1.3 the number of all
integers i ∈ {0, . . . , T − 1}, such that xi does not belong to this -neighborhood,
does not exceed the constant k0 which depends only on M, and does not depend
on T . Following the tradition, the point x̄ is called the turnpike. Moreover we can
show that the set
{i ∈ {0 . . . , T } : |xi − x̄| > }
is contained in the union of two intervals [0, k1 ]∪[T −k1 , T ], where k1 is a constant
depending only on M, .
8 1 Introduction
1.2 The Turnpike Phenomenon
In the previous section we proved the turnpike result for rather simple class of
discrete-time problems. The problems of this class are unconstrained and their
objective functions are convex and differentiable. It should be mentioned that,
using the methods of convex analysis [64], this result can be extended to the class
of objective functions which are merely convex. In the turnpike theory and its
applications to the mathematical economics our goal is to establish the turnpike
property for constrained optimal control problems. In particular, in this book and in
[84, 86] we study the structure of approximate solutions of an autonomous discrete-
time control system with a compact metric space of states X. This control system
is described by a nonempty closed set Ω ⊂ X × X which determines a class of
admissible trajectories (programs) and a bounded upper semicontinuous function
v : Ω → R 1 which determines an optimality criterion. We study the problem
−1
T
T −1
v(xi , xi+1 ) → max, {(xi , xi+1 )}i=0 ⊂ Ω, x0 = z, xT = y, (P )
i=0
where T ≥ 1 is an integer and the points y, z ∈ X. Clearly, these constrained

problems are more difficult and less understood than their unconstrained prototypes.
They are also more realistic from the point of view of mathematical economics.
In the turnpike theory the objective function v possesses the turnpike property
(TP) if there exists a point x̄ ∈ X (a turnpike) such that the following condition
holds:
For each positive number there exists an integer L ≥ 1 such that for each
integer T ≥ 2L and each solution {xi }Ti=0 ⊂ X of the problem (P ) the inequality
ρ(xi , x̄) ≤ is true for all i = L, . . . , T − L.
It should be mentioned that the constant L depends neither on T nor on y, z.
The turnpike phenomenon has the following interpretation. If one wishes to reach
a point A from a point B by a car in an optimal way, then one should turn to a
turnpike, spend most of the time on it, and then leave the turnpike to reach the
required point.
It should be mentioned that in general a turnpike is not necessarily a singleton
[76]. Nevertheless problems of the type (P ) for which the turnpike is a singleton
are of great importance because of the following reasons: there are many models of
economic growth for which a turnpike is a singleton; if a turnpike is a singleton, then
approximate solutions of (P ) have very simple structure and this is very important
for applications; if a turnpike is a singleton, then it can be easily calculated as a
solution of the problem v(x, x) → max, (x, x) ∈ Ω.
The turnpike property is very important for applications. Suppose that our
objective function v has the turnpike property and we know a finite number of
“approximate” solutions of the problem (P ). Then we know the turnpike x̄, or at
least its approximation, and the constant L (see the definition of (TP)) which is an
estimate for the time period required to reach the turnpike. This information can
1.2 The Turnpike Phenomenon 9
be useful if we need to find an “approximate” solution of the problem (P ) with a

new time interval [m1 , m2 ] and the new values z, y ∈ X at the end points m1 and
m2 . Namely instead of solving this new problem on the “large” interval [m1 , m2 ]
we can find an “approximate” solution of the problem (P ) on the “small” interval
[m1 , m1 + L] with the values z, x̄ at the end points and an “approximate” solution
of the problem (P ) on the “small” interval [m2 − L, m2 ] with the values x̄, y at
the end points. Then the concatenation of the first solution, the constant sequence
xi = x̄, i = m1 + L, . . . , m2 − L and the second solution is an “approximate”
solution of the problem (P ) on the interval [m1 , m2 ] with the values z, y at the end
points. Sometimes as an “approximate” solution of the problem (P ) we can choose
any admissible sequence {xi }m 2
i=m1 satisfying
xm1 = z, xm2 = y and xi = x̄ for all i = m1 + L, . . . , m2 − L.
In this book we study a class of optimal control problems arising in the forest
management. The forest management problem is an important and interesting topic
in mathematical economics studies by many researchers. We consider the discrete
time model for the optimal management of a forest of total area S occupied by k
species I = {1, . . . , k} with maturity ages of n1 , . . . , nk years respectively. This
model and its versions were studied in [20, 38–40, 54, 55, 61, 68, 80, 81, 83]. Here
the optimal harvesting of a mixed forest composed of multiple species is studied,
each one having a different maturity age, where only mature trees can be harvested.
As usual, for this problem the existence of optimal solutions over infinite horizon
and the structure of solutions on finite intervals are under consideration. In our books
[84, 86] we study a class of discrete-time optimal control problems which describe
many models of economic dynamics except of the model of the forest management.
This happens because some assumptions posed in [84, 86], which are true for many
models of economic dynamics, do not hold for the model of the forest management.
Namely, in [84, 86] we studied optimal control problems of type (P ) for which the
turnpike property holds with the turnpike x̄ such that (x̄, x̄) is an interior point of the
set Ω ⊂ X × X. This assumption, which plays an important role in [84, 86], holds
for many models of economic dynamics except of the forest management problem
considered in this book.
By this reason, the forest management problem is not a particular case of general
models of economic dynamics and is studied separately in the literature. In this
book we study the forest management problem using the approach introduced and
employed in our research [80, 81, 83]. Namely, we analyze a class of optimal control
problems which contains, as a particular case, the forest management problem. For
this class of problems we show the existence of optimal solutions over infinite
horizon, study the structure of approximate solutions on finite intervals and their
turnpike properties, the stability of the turnpike phenomenon, and the structure of
approximate solutions on finite intervals in the regions close to the end points.
Chapter 2
Infinite Horizon Optimal Control
Problems
In this chapter we present the forest management problem and study some properties
of its solutions. This problem is a particular case of a general optimal control
problem which is also introduced in this chapter. The corresponding infinite horizon
problems without discounting as well as with discounting are considered. The
results of the chapter were obtained from [80, 81].
2.1 The Forest Management Problem
We consider a discrete time model for the optimal management of a forest of total
area S occupied by k species I = {1, . . . , k} with maturity ages of n1 , . . . , nk years
respectively. This model and its versions were studied in [20, 38–40, 54, 55, 61,
68, 80, 81, 83]. Here the optimal harvesting of a mixed forest composed of multiple
species is studied, each one having a different maturity age, where only mature trees
can be harvested.
j
For each period t = 0, 1, . . . we denote xi (t) ≥ 0 the area covered by trees of
species i that are j years old with j = 1, . . . , ni and x̄i (t) ≥ 0 the area occupied
by over-mature trees (older than ni ). We must decide how much land vi (t) ≥ 0 to
harvest and how to reallocate this land to new seedlings.
Assuming that only mature trees can be harvested we must have
vi (t) ≤ x̄i (t) + xini (t), (2.1)
and then the area not harvested in that period will comprise the over-mature trees at
the next step, namely
x̄i (t + 1) = x̄i (t) + xini (t) − vi (t). (2.2)

12 2 Infinite Horizon Optimal Control Problems
The fact that immature trees cannot be harvested is represented by

j +1 j
xi (t + 1) = xi (t), j = 1, . . . , ni − 1. (2.3)

The total harvested area i∈I vi (t) is allocated to new seedlings which is
expressed by the equation

xi1 (t + 1) = vi (t). (2.4)
i∈I i∈I
In the sequel we use the notation
xini +1 = x̄i , i ∈ I. (2.5)
A representation of the forest in terms of the age distribution at time t is provided

by the state x(t) = (x1 (t), . . . , xk (t)), where
xi (t) = (xi1 (t), . . . , xini (t), xini +1 (t)), i ∈ I
describes the areas occupied in year t by trees of species i with ages 1, 2, . . . , ni

and over ni . The first and last components of each vector xi (t) are controlled by the
sowing and harvesting policies. Note that we do not control x(0) which corresponds
to the initial state reflecting the age class composition of the forest at time t = 0.
Let R+ = {x = (x1 , . . . , xm ) ∈ NR : xi ≥ 0, i = 1, . . . , m} and let N =
m m
(n
i∈I i + 1). Every vector x ∈ R is represented as x = (x1 , . . . , xk ), where
ni ni +1 +1
xi = (xi , . . . , xi , xi
1 )∈R n i for all integers i = 1, . . . , k.
Denote by Δ the set of all x ∈ R+ N such that
⎡ ⎤
n i +1
⎣ xi ⎦ = S.
j
(2.6)
i∈I j =1
Now we give a formal description of the model.

A sequence {x(t)}∞ t=0 ⊂ Δ is called a program if for all integers t ≥ 0 and all
i ∈ I Eqs. (2.1)–(2.4) hold (see (2.5)) with some v(t) = (v1 (t), . . . , vk (t)) ∈ R+ k.
T2
Let the integers T1 , T2 satisfy 0 ≤ T1 < T2 . A sequence {x(t)}t=T1 ⊂ Δ is called
a program if Eqs. (2.1)–(2.4) hold for all i ∈ I and for all integers t = T1 , . . . , T2 −1
(see (2.5)) with some v(t) = (v1 (t), . . . , vk (t)) ∈ R+
k.
An alternative equivalent definition of a program can be given with the help of

the transition possibility. Put

j +1 j
Ω = (x, y) ∈ Δ × Δ : yi = xi for all i ∈ I and all j ∈ {1, . . . , ni } \ {ni }

and for all i ∈ I, xini +1 + xini − yini +1 ≥ 0 . (2.7)
2.1 The Forest Management Problem 13
Clearly, if (x, y) ∈ Ω, then

n(i)+1
yi1 = (xi + xini − yin(i)+1 ). (2.8)
i∈I i∈I
It is easy to see that a sequence {x(t)}∞

t=0 ⊂ Δ is a program if and only if (x(t), x(t +
1)) ∈ Ω for all integers t ≥ 0.
Let the integers T1 , T2 satisfy 0 ≤ T1 < T2 . It is easy to see that a sequence
{x(t)}Tt=T 2
1
⊂ Δ is a program if and only if (x(t), x(t + 1)) ∈ Ω for all t =
T1 , . . . , T2 − 1.
For each (x, y) ∈ Ω set
V (x, y) = (v1 (x, y), . . . , vk (x, y)),
where for i = 1, . . . , k,
vi (x, y) = xini +1 + xini − yini +1 .
Define

k
Δ0 = v ∈ k
R+ : vi ≤ S .
i=1
In this book we assume that a benefit at moment t = 0, 1, . . . is represented by

an upper semicontinuous function wt : Δ0 → R 1 and at a moment t = 0, 1, . . . ,
wt (V (x, y)) is the benefit obtained today if the forest today is x and the forest
tomorrow is y, where (x, y) ∈ Ω.
Note that usually in the literature it is assumed that for t = 0, 1, . . . ,

k
wt (V (x, y)) = W (i) (vi (x, y)), (x, y) ∈ Ω
i=1
where W i : [0, ∞) → R 1 , i = 1, . . . , k are strictly concave, smooth, and increasing

functions.
Clearly, Δ is a compact set in R N , Ω is a closed subset of Δ × Δ, and wt ◦ V :
Ω → R 1 is an upper semicontinuous function for all integers t ≥ 0. Set
n̄ = max{ni : i ∈ I }. (2.9)
Our model has an important property established by the following result. Its proof
is given in [80].
+n̄+1
Proposition 2.1 Let x, y ∈ Δ. Then there exists a program {x(t)}N
t=0 such that
x(0) = x and x(N + n̄ + 1) = y.
Proof Put x(0) = x. For all integers t = 0, . . . , N − 1 define
j +1 j
xi (t + 1) = xi (t), i ∈ I, j ∈ {1, . . . , ni } \ {ni },
xi1 (t + 1) = 0, i ∈ I,
xini +1 (t + 1) = xini +1 (t) + xini (t), i ∈ I. (2.10)
It is easy to see that x(t) ∈ Δ for all t = 0, . . . , N, {x(t)}N

t=0 is a program and
j
xi (N ) = 0, i ∈ I, j = 1, . . . , ni ,

xini +1 (N ) = S. (2.11)
i∈I
For each s = 1, . . . , n̄ put
Is = {i ∈ I : ni = s}. (2.12)
(Note that for some integers s we can have Is = ∅.)

We assume that sum over empty set is zero.
Define x(N + 1) ∈ Δ as follows. Set
xi1 (N + 1) = yini +1 , i ∈ In̄ , (2.13)
xi1 (N + 1) = 0, i ∈ I \ In̄ .
For i ∈ I and all integers j satisfying 1 < j ≤ ni set
j
xi (N + 1) = 0. (2.14)
Clearly, there exist
ui ∈ [0, xini +1 (N )], i ∈ I (2.15)
such that

ui = xi1 (N + 1). (2.16)
i∈I i∈In̄
Put
xini +1 (N + 1) = xini +1 (N ) − ui , i ∈ I. (2.17)

2.1 The Forest Management Problem 15
By (2.13)–(2.17), x(N + 1) ∈ Δ and
(x(N ), x(N + 1)) ∈ Ω.
Assume that q is an integer, 1 ≤ q < n̄, and we have defined a program

N +q
{x(t)}t=0 such that the following properties hold:
(P1) If an integer
i ∈ ∪{Is : an integer s satisfies 1 ≤ s ≤ n̄ − q},
j
then xi (N + q) = 0 for all integers j satisfying 1 ≤ j ≤ ni ;
(P2) If an integer s satisfies n̄ ≥ s > n̄ − q and i ∈ Is , then
p n +1+p−(q+s−n̄)
xi (N + q) = yi i , p = 1, . . . , q + s − n̄, (2.18)
p
xi (N +q) = 0 for all integers p satisfying q+s−n̄ < p ≤ ni . (2.19)
(Note that for q = 1 our assumptions hold.)

Define x(N +q +1) ∈ Δ as follows. Let i ∈ I . If i ∈ Is , where 1 ≤ s ≤ n̄−q −1,
then set
j
xi (N + q + 1) = 0 for all integers j satisfying 1 ≤ j ≤ ni . (2.20)
If i ∈ In̄−q , then set
xi1 (N + q + 1) = yini +1 , xi (N + q + 1) = 0
p
(2.21)
for all integers p satisfying 1 < p ≤ ni .

If i ∈ Is , where n̄ ≥ s > n̄ − q, then set (see (2.18))
p+1 p n +1+p−(q+s−n̄)
xi (N + q + 1) = xi (N + q) = yi i , p = 1, . . . , q + s − n̄,
(2.22)
n +1−(q+s−n̄)
xi1 (N + q + 1) = yi i , (2.23)
p
xi (N + q + 1) = 0 for all integers p satisfying q + 1 + s − n̄ < p ≤ ni . (2.24)
It is not difficult to see that

n +1
xi1 (N + q + 1) ≤ xi i (N + q). (2.25)
i∈I i∈I
Therefore there exists

xini +1 (N + q + 1) ∈ 0, xini +1 (N + q) , i ∈ I, (2.26)
such that

[xini +1 (N + q) − xini +1 (N + q + 1)] = xi1 (N + q + 1). (2.27)
i∈I i∈I
Clearly,
x(N + q + 1) ∈ Δ, (x(N + q), x(N + q + 1)) ∈ Ω
and the assumption made for q holds also for q + 1. Thus by induction we have
+n̄
constructed a program {x(t)}N
t=0 such that (P1) and (P2) hold for all q = 1, . . . , n̄.
Consider the state x(N + n̄). Let i ∈ Is where 1 ≤ s ≤ n̄. By (P1) and (P2),
p p+1
xi (N + n̄) = yi , p = 1, . . . , ni . (2.28)
Set
x(N + n̄ + 1) = y. (2.29)
By (2.28) and (2.29),
(x(N + n̄), x(N + n̄ + 1)) ∈ Ω.
2.2 Infinite Horizon Problems Without Discounting
Let (K, ρ) be a compact metric space and Ω be a nonempty closed subset of K ×K.
A sequence {xt }∞ t=0 ⊂ K is called an (Ω)-program (or a program if the set Ω is
understood) if (xt , xt+1 ) ∈ Ω for all integers t ≥ 0.
Let the integers T1 , T2 satisfy T1 < T2 . A sequence {xt }Tt=T
2
1
⊂ K is called an
(Ω)-program (or a program if the set Ω is understood) if (xt , xt+1 ) ∈ Ω for all
integers t satisfying T1 ≤ t < T2 .
For each integer t ≥ 0, let ut : Ω → R 1 be a bounded upper semicontinuous
function.
For each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 and each y, z ∈ K we
consider the optimization problems
2.2 Infinite Horizon Problems Without Discounting 17
2 −1
T
ui (xi , xi+1 ) → max,
i=T1
{xi }Ti=T
2
1
⊂ K, xT1 = y, xT2 = z
and
2 −1
T
ui (xi , xi+1 ) → max,
i=T1
{xi }Ti=T
2
1
⊂ K, xT1 = y.
The results of this section were obtained from [81].

For each integer t ≥ 0 set
ut = sup{|ut (z)| : z ∈ Ω} (2.30)
and assume that
sup{ut : t = 0, 1, . . . } < ∞. (2.31)
In the sequel we assume that supremum of empty set is −∞.

For each y ∈ K and each pair of integers T1 , T2 satisfying T1 < T2 set
⎧ ⎫
⎨T
2 −1 ⎬
U (y, T1 , T2 ) = sup ut (xt , xtt+1 ) : {xt }Tt=T
2
is a program and xT1 = y .
⎩ 1 ⎭
t=T1
(2.32)
Let y, ỹ ∈ K and integers T1 , T2 satisfy T1 < T2 . Set
⎧
⎨T
2 −1
U (y, ỹ, T1 , T2 ) = sup ut (xt , xtt+1 ) :
⎩
t=T1
⎫
⎬
{xt }Tt=T
2
is a program and xT1 = y, xT2 = ỹ . (2.33)
1 ⎭
Let the integers T1 , T2 satisfy T1 < T2 . Set

⎧ ⎫
⎨T
2 −1 ⎬
(T1 , T2 ) = sup
U ut (xt , xtt+1 ) : {xt }Tt=T
2
is a program . (2.34)
⎩ 1 ⎭
t=T1
Upper semicontinuity of the functions ut , t = 0, 1, . . . implies the following

three results.
Proposition 2.2 For each z ∈ K and each pair of integers T1 < T2 such that
U (z, T1 , T2 ) is finite there exists a program {xt }Tt=T
2
1
such that xT1 = z and
2 −1
T
ut (xt , xt+1 ) = U (z, T1 , T2 ).
t=T1
Proposition 2.3 Let y0 , ỹ0 ∈ K, integers T1 , T2 satisfy T1 < T2 and let
U (y0 , ỹ0 , T1 , T2 )
be finite. Then there exists a program {xt }Tt=T

2
1
such that
xT1 = y0 , xT 2 = ỹ0 ,
2 −1
T
ut (xt , xt+1 ) = U (y0 , ỹ0 , T1 , T2 ).
t=T1
(T1 , T2 ) be finite.
Proposition 2.4 Let the integers T1 , T2 satisfy T1 < T2 and let U
T2
Then there exists a program {xt }t=T1 such that
2 −1
T
(T1 , T2 ).
ut (xt , xt+1 ) = U
t=T1
In this section we suppose that the following assumption holds.
(A) There exists a natural number L̄ such that for each y, z ∈ K there is a program
{xt }L̄
t=0 such that x0 = y and xL̄ = z.
Note that in [72, 73, 75] the case where L̄ = 1 was studied. In view of
Proposition 2.1, assumption (A) holds for the forest management problem.
Proposition 2.5 Let the integers T1 , T2 satisfy T1 ≤ T2 − L̄ and let y, z ∈ K. Then

U (y, z, T1 , T2 ) is finite.
Proof If T2 − T1 = L̄, then by Assumption (A), U (y, z, T1 , T2 ) is finite.
2 −L̄
Assume that T2 − T1 > L̄. By (A) there exists a program {xt }Tt=T 1
such that
xT1 = y and a program {xt }Tt=T
2
such that xT2 = z. This completes the proof of
2 −L̄
Proposition 2.5.
In Sect. 2.4 we prove the following two results.
2.2 Infinite Horizon Problems Without Discounting 19
Theorem 2.6 There exists M > 0 such that for each x0 ∈ K there exists a program
{x̄t }∞
t=0 such that x̄0 = x0 and that for each pair of integers T1 , T2 ≥ 0 satisfying
T1 < T2 ,

T2 −1

ut (x̄t , x̄t+1 ) − U (T1 , T2 ) ≤ M.

t=T1
Theorem 2.7 Assume that {xt }∞

t=0 is a program and there exists M0 > 0 such that
for each integer T > 0,
−1
T
ut (xt , xt+1 ) ≥ U (x(0), x(T ), 0, T ) − M0 .
t=0
Then there exists M1 > 0 such that for each pair of integers T1 ≥ 0, T2 > T1 ,

T2 −1

(T1 , T2 ) ≤ M1 .
ut (xt , xt+1 ) − U

t=T1
The program {x̄t }∞

t=0 whose existence is guaranteed by Theorem 2.6 in infinite
horizon optimal control is considered as an (approximately) optimal program [76,
84, 86].
Proposition 2.8 Let x0 ∈ K and let a program {x̄t }∞ t=0 and M > 0 be as
guaranteed by Theorem 2.6. Assume that {xt }∞
t=0 is a program. Then either the
sequence
T −1 −1
∞
T
ut (xt , xt+1 ) − ut (x̄t , x̄t+1 )
t=0 t=0 T =1
is bounded or
−1
T −1
T
ut (xt , xt+1 ) − ut (x̄t , x̄t+1 ) → −∞ as T → ∞. (2.35)
t=0 t=0
Proof Assume that the sequence

T −1 −1
∞
T
ut (xt , xt+1 ) − ut (x̄t , x̄t+1 )
t=0 t=0 T =1
is not bounded. Then by Theorem 2.6,

T −1 −1

T
lim inf ut (xt , xt+1 ) − ut (x̄t , x̄t+1 ) = −∞.
T →∞
t=0 t=0
Let Q > 0. Then there exists an integer T0 > 0 such that
0 −1
T 0 −1
T
ut (xt , xt+1 ) − ut (x̄t , x̄t+1 ) < −Q − M. (2.36)
t=0 t=0
By (2.36), the choice of {x̄t }∞

t=0 and Theorem 2.6 for each integer T > T0 ,
−1
T −1
T 0 −1
T
ut (xt , xt+1 ) − ut (x̄t , x̄t+1 ) = ut (xt , xt+1 )
t=0 t=0 t=0
0 −1
T −1
T −1
T
− ut (x̄t , x̄t+1 ) + ut (xt , xt+1 ) − ut (x̄t , x̄t+1 )
t=0 t=T0 t=T0
−1
T
(T0 , T ) −
< −Q − M + U ut (x̄t , x̄t+1 ) ≤ −Q.
t=T0
Since Q is any positive number we conclude that (2.35) is true. Proposition 2.8 is
proved.
Now assume that ut = u0 , t = 0, 1, . . . . The following result will be proved in
Sect. 2.5.
Theorem 2.9 Let M > 0 be as guaranteed by Theorem 2.6. There exists μ =
(0, p)/p and
limp→∞ U
(0, p) − μ| ≤ 2M/p for all natural numbers p.

|p−1 U
The following result will be proved in Sect. 2.6.

Theorem 2.10 Let K be a compact subset of a normed space (Z, · ), the set
Ω be convex, ut = u0 , t = 0, 1, . . . , the function u0 be concave, and let μ be as
guaranteed by Theorem 2.9. Then
μ = sup{u0 (x, x) : x ∈ K such that (x, x) ∈ Ω}.
In view of Proposition 2.1, the results stated in this section are applied to the
forest management problem which is discussed in Sect. 2.1. These results allow us to
use new optimality criterions for the forest management problem when the functions
2.3 Auxiliary Results 21
ut are not concave. In particular, the program {x̄t }∞

t=0 whose existence is guaranteed
by Theorem 2.6 can be considered as an optimal solution of the corresponding forest
management problem.
2.3 Auxiliary Results
Set
M = 2(2L̄ + 4) sup{ut : t = 0, 1, . . . }. (2.37)
Lemma 2.11 Let the integers T1 , T2 ≥ 0 satisfy T2 ≥ T1 + L̄, y, z ∈ K, and

M1 ≥ 0 and let a program {xt }Tt=T
2
1
satisfy
x0 = y, xT = z, (2.38)
2 −1
T
ut (xt , xt+1 ) ≥ U (y, z, T1 , T2 ) − M1 . (2.39)
t=T1
Then for each pair of integers τ1 , τ2 ≥ 0 satisfying
T1 ≤ τ1 < τ2 ≤ T2
2 −1
τ
(τ1 , τ2 ) − M1 − M.
ut (xt , xt+1 ) ≥ U (2.40)
t=τ1
Proof Let the integers τ1 , τ2 ≥ 0 satisfy
T1 ≤ τ1 < τ2 ≤ T2 .
By Proposition 2.4 and assumption (A), there is a program {xt

}τt=τ
2
1
such that
2 −1
τ
ut (xt
, xt+1

(τ1 , τ2 ).
)=U (2.41)
t=τ1
If τ2 − τ1 ≤ 2L̄ + 2, then in view of (2.37) inequality (2.40) holds.

Assume that
τ2 − τ1 > 2L̄ + 2. (2.42)

1 +L̄
By assumption (A) there exist programs {yt }τt=τ , {yt
}τt=τ
2
such that
1 2 −L̄
yτ
= xτ
, yτ
2 = xτ2 , yτ1 = xτ1 , yτ1 +L̄ = xτ
. (2.43)
2 −L̄ 2 −L̄ 1 +L̄
Set
zt = xt , t = T1 , . . . , τ1 , zt = yt , t = τ1 + 1, . . . , τ1 + L̄,
zt = xt
, t = τ1 + L̄ + 1, . . . , τ2 − L̄,
zt = yt
, t = τ2 − L̄ + 1, . . . , τ2 ,
zt = xt for all integers t satisfying τ2 < t ≤ T2 . (2.44)
Clearly, {zt }Tt=T

2
1
is a program. By (2.37), (2.39), (2.41), (2.43), and (2.44),
2 −1
T 2 −1
T
−M1 ≤ ut (xt , xt+1 ) − ut (zt , zt+1 )
t=T1 t=T1
2 −1
τ 2 −1
τ
= ut (xt , xt+1 ) − ut (zt , zt+1 )
t=τ1 t=τ1
2 −1
τ 2 −1
τ
≤ ut (xt , xt+1 ) − ut (xt
, xt+1

)
t=τ1 t=τ1
+ 2 sup{ut : t = 0, 1, . . . }2(L̄ + 1)
2 −1
τ
≤ (τ1 , τ2 ) + M
ut (xt , xt+1 ) − U
t=τ1
and
2 −1
τ
(τ1 , τ2 ) − M1 − M.
ut (xt , xt+1 ) ≥ U
t=τ1
Lemma 2.11 is proved.

Lemma 2.11 and (2.37) imply the following auxiliary result.
2.4 Proofs of Theorems 2.6 and 2.7 23
Lemma 2.12 Let the integers T1 , T2 ≥ 0 satisfy T2 > T1 , y ∈ K, M1 ≥ 0 and let a

program {xt }Tt=T
2
1
satisfy
x0 = y,
2 −1
T
ut (xt , xt+1 ) ≥ U (y, T1 , T2 ) − M1 .
t=T1
Then for each pair of integers τ1 , τ2 ≥ 0 satisfying
T1 ≤ τ1 < τ2 ≤ T2 ,
2 −1
τ
(τ1 , τ2 ) − M1 − M.
ut (xt , xt+1 ) ≥ U
t=τ1
2.4 Proofs of Theorems 2.6 and 2.7
Proof of Theorem 2.6 Let x0 ∈ K. By Proposition 2.2, for each natural number k
there exists a program {xt(k) }kt=0 such that
(k)

k−1
(k) (k)
x0 = x0 , ut (xt , xt+1 ) = U (x0 , 0, k). (2.45)
t=0
By (2.45) and Lemma 2.12 the following property holds:

(P3) For each integer k ≥ 0 and each pair of integers τ1 , τ2 ≥ 0 satisfying
0 ≤ τ1 < τ2 ≤ k,
2 −1
τ
ut (xt(k) , xt+1
(k) (T1 , T2 ) − M.
)≥U
t=τ1
Clearly, there exists a strictly increasing sequence of natural numbers {kj }∞

j =1
such that for each integer t ≥ 0 there exists
(kj )
x̄t = lim xt . (2.46)
j →∞
In view of (2.45) and (2.46),
x̄0 = x0 . (2.47)
It follows from (P3), (2.46), and upper semicontinuity of the functions ut , t =

0, 1, . . . that for each pair of integers τ1 , τ2 ≥ 0 satisfying τ1 < τ2 ,

τ
2 −1
ut (x̄t , x̄t+1 ) − U (T1 , T2 ) ≤ M.

t=τ1
Theorem 2.6 is proved.

Proof of Theorem 2.7 By Lemma 2.11, for all integers τ1 , τ2 ≥ 0 satisfying 0 ≤
τ1 < τ2 ,
2 −1
τ
(τ1 , τ2 ) ≥ −M0 − M.
ut (xt , xt+1 ) − U
t=τ1
This completes the proof of Theorem 2.7.
2.5 Proof of Theorem 2.9
Set u = u0 .
Let x0 ∈ K and let {x̄t }∞
t=0 be as guaranteed by Theorem 2.6. Then for each pair
of integers T1 , T2 ≥ 0 satisfying T1 < T2 ,

T2 −1

u(x̄t , x̄t+1 ) − U (T1 , T2 ) ≤ M.
(2.48)

t=T1
Choose Δ > 0 such that
Δ > u0 . (2.49)
Let p be a natural number. We show that for all sufficiently large natural numbers
T,

−1
T
−1
p U (0, p) − T −1 u(x̄t , x̄t+1 ) ≤ 2M/p. (2.50)

t=0
Assume that T ≥ p is a natural number. Then there exist integers q, s such that
q ≥ 1, 0 ≤ s < p, T = pq + s. (2.51)
2.5 Proof of Theorem 2.9 25
It follows from (2.51) that

−1
T
T −1 (0, p)
u(x̄t , x̄t+1 ) − p−1 U
t=0
⎛

pq−1
−1 ⎝
=T u(x̄t , x̄t+1 )
t=0
⎞

(0, p)
+ {u(x̄t , x̄t+1 ) : t is an integer such that pq ≤ t ≤ T −1}⎠ −p −1 U

= T −1 {u(x̄t , x̄t+1 ) : t is an integer such that pq ≤ t ≤ T − 1}

q−1 (i+1)p−1
+ (T −1
pq)(pq) −1 (0, p)
u(x̄t , x̄t+1 ) − p−1 U
t=0 t=ip
⎡ ⎛ ⎞ ⎤

q−1
(i+1)p−1
= (T −1 pq)(pq)−1 ⎣ ⎝ (0, p)⎠ + q U
u(x̄t , x̄t+1 ) − U (0, p)⎦
i=0 t=ip
− p−1 U(0, p)

+ T −1 u(x̄t , x̄t+1 ) : t is an integer such that pq ≤ t ≤ T − 1 .
(2.52)
By (2.48), (2.49), (2.51), and (2.52),

T −1
−1 −1
T u(x̄t , x̄t+1 ) − p U (0, p)

t=0
(0, p)||q/T − 1/p|

≤ T −1 pΔ + (pq)−1 qM + |U
(0, p)|s(pT )−1 → M/p as T → ∞.
≤ T −1 pΔ + M/p + |U (2.53)
Since p is any natural number we conclude that

−1
T
∞
−1
T u(x̄t , x̄t+1 )
t=0 T =1
is a Cauchy sequence. Clearly, there exists

−1
T
lim T −1 u(x̄t , x̄t+1 ),
T →∞
t=0
and that in view of (2.53) for each natural number p,
−1
T
(0, p) − lim T −1
|p−1 U u(x̄t , x̄t+1 )| ≤ 2M/p. (2.54)
T →∞
t=0
Since (2.54) is true for any natural number p we obtain that
−1
T
lim T −1 (0, p)/p.
u(x̄t , x̄t+1 ) = lim U (2.55)
T →∞ p→∞
t=0
Set
(0, p)/p.
μ = lim U (2.56)
p→∞
By (2.54)–(2.56), for all natural numbers p,
(0, p) − μ| ≤ 2M/p.
|p−1 U
Put u = u0 . By Proposition 2.4 and assumption (A), for each integer T > 0 there is
a program {xt(T ) }Tt=0 such that
−1
T
(T )
u(xt
(T ) (0, T ).
, xt+1 ) = U (2.57)
t=0
Set
−1
T −1
T
x (T ) = T −1 , y (T ) = T −1
(T ) (T )
xt xt+1 . (2.58)
t=0 t=0
Clearly,
(0, T ),
(x (T ) , y (T ) ) ∈ Ω, T u(x (T ) , y (T ) ) ≥ U
(0, T ),
u(x (T ) , y (T ) ) ≥ T −1 U
x (T ) − y (T ) ≤ 2T −1 sup{z : z ∈ K}.
2.7 Infinite Horizon Problems with Discounting 27
There is a strictly increasing sequence of natural numbers {Tk }∞

k=1 such that there
exist
lim x (Tk ) = x̄, lim y (Tk ) = ȳ.

k→∞ k→∞
Clearly,
x̄ = ȳ, (x̄, x̄) ∈ Ω,
(0, T ) = μ.
u(x̄, x̄) ≥ lim T −1 U
T →∞
On the other hand, it is easy to see that
u(z, z) ≤ μ
for all z ∈ K such that (z, z) ∈ Ω. Theorem 2.10 is proved.
2.7 Infinite Horizon Problems with Discounting
We consider the optimal control problem of Sect. 2.2 and continue to use the
notation, definitions, and assumptions introduced there.
In this section we suppose that the following equality holds:
lim sup{|ut (z)| : z ∈ Ω} = 0. (2.59)

t→∞
By (A) and Proposition 2.2, for each y ∈ K and each natural number T there is
(y,T ) T
a program {xt }t=0 such that
(y,T )
x0 = y, (2.60)
−1
T
(y,T ) (y,T )
ut (xt , xt+1 ) = U (y, 0, T ). (2.61)
t=0
In Sect. 2.9 we prove the following result.

Theorem 2.13 For any y ∈ K there exists a program {xt }∞
(y) (y)
t=0 such that x0 = y
and the following property holds:
For each > 0 there exists a natural number τ such that for each y ∈ K and
each integer T ≥ τ ,
T −1

(y) (y)
ut (xt , xt+1 ) − U (y, 0, T ) ≤ .

t=0
The next corollary easily follows from Theorem 2.13.

Corollary 2.14 Let y ∈ K. Then for any program {xt }∞
t=0 satisfying x0 = y,
−1
T
(y) (y)
lim sup ut (xt , xt+1 ) − ut (xt , xt+1 ) ≤ 0.
T →∞ t=0
Note that the program {xt }∞t=0 which exists by Corollary 2.14 is called in the
literature as an overtaking optimal program [76, 84, 86].
Example 2.15 Let w : Ω → [0, ∞) be a bounded upper semicontinuous function,
{ρt }∞
t=0 ⊂ (0, 1) satisfy
lim ρt = 0, (2.62)
t→∞
and let ut = ρt w, t = 0, 1, . . . . Then (2.59) holds. In the literature it is also

considered an optimality criterion with∞ρt = α t , t = 0, 1, . . . where α ∈ (0, 1).
∞
In this case for any program {xt }t=0 , t=0 α t w(xt , xt+1 ) < ∞. This convergence
does not hold in the general case with {ρt }∞
t=0 ⊂ (0, 1) satisfying (2.62). Therefore
in the general case the existence problem of an overtaking optimal program is more
difficult and less understood.
2.8 An Auxiliary Result for Theorem 2.13
Recall that for any integer t ≥ 0,
ut = sup{|ut (z)| : z ∈ Ω}. (2.63)
Lemma 2.16 Let > 0. Then there exists a natural number τ such that for each
y ∈ K and each pair of integers T1 ≥ τ and T2 ≥ T1 + L̄,
1 −1
T
(y,T ) (y,T )
ut xt 2 , xt+1 2 ≥ U (y, 0, T1 ) − .
t=0
Proof By (2.59) and (2.63), there exists a natural number τ such that
ut ≤ (4L̄)−1 for all integers t ≥ τ. (2.64)

2.8 An Auxiliary Result for Theorem 2.13 29
Assume that y ∈ K and that integers
T1 ≥ τ, T2 ≥ T1 + L̄. (2.65)
1 +L̄
In view of (A), there exists a program {xt }Tt=T 1
such that
(y,T1 ) (y,T2 )
xT1 = xT1 , xT1 +L̄ = xT . (2.66)
1 +L̄
Set
(y,T1 )
xt = xt , t = 0, . . . , T1 − 1, (2.67)
(y,T2 )
xt = xt for all integers t satisfying T1 + L̄ < t ≤ T2 .
Clearly, {xt }Tt=0

2
is a program and
x0 = y. (2.68)
By (2.60), (2.61), (2.63)–(2.66), and (2.68),
2 −1
T 2 −1
T
(y,T2 ) (y,T )
0≤ ut (xt , xt+1 2 ) − ut (xt , xt+1 )
t=0 t=0
+L̄−1
T1 +L̄−1
T1
(y,T2 ) (y,T )
= ut (xt , xt+1 2 ) − ut (xt , xt+1 )
t=0 t=0
1 −1
T 1 −1
T +L̄−1
T1
(y,T ) (y,T ) (y,T ) (y,T )
≤ ut (xt 2 , xt+1 2 ) − ut (xt 1 , xt+1 1 ) + 2 ||ut ||
t=0 t=0 t=T1
1 −1
T
, xt+1 2 ) − U (y, 0, T1 ) + 2L̄((4L̄)−1 )
(y,T2 ) (y,T )
≤ ut (xt
t=0
and
1 −1
T
(y,T2 ) (y,T )
ut (xt , xt+1 2 ) ≥ U (y, 0, T1 ) − .
t=0

Let y ∈ K. Using the diagonalization process and the compactness of K we obtain

a strictly increasing sequence of natural numbers {Tk }∞
k=1 such that for any integer
t ≥ 0 there exists
(y) (y,Tk )
xt = lim xt . (2.69)
k→∞
Clearly, {xt }∞
y
t=0 is a program for all y ∈ K.
Let > 0 and let a natural number τ be as guaranteed by Lemma 2.16. Assume
that an integer T ≥ τ and y ∈ K. Then for all sufficiently large natural numbers k,
−1
T
(y,Tk ) (y,T )
ut (xt , xt+1 k ) ≥ U (y, 0, T ) − .
t=0
By the inequality above, (2.69) and upper semicontinuity of the functions ut , t =

0, 1, . . . ,
−1
T
(y) (y)
ut (xt , xt+1 ) ≥ U (y, 0, T ) − .
t=0
2.10 An Application to the Forest Management Problem
We consider the forest management problem discussed in Sect. 2.1 using the
notation, definitions, and assumptions introduced there.
Recall that for each (x, y) ∈ Ω,
V (x, y) = (v1 (x, y), . . . , vk (x, y)),
where for i = 1, . . . , k,
vi (x, y) = xini +1 + xini − yini +1 ,
and that

k
Δ0 = u ∈ k
R+ : ui ≤ S .
i=1
2.10 An Application to the Forest Management Problem 31
In this section we assume that a benefit at moment t = 0, 1, . . . is represented

by an upper semicontinuous function wt : Δ0 → R 1 and at a moment t = 0, 1, . . . ,
tomorrow is y, where (x, y) ∈ Ω. In view of Proposition 2.1, (A) holds.
Assume that
sup{|wt (z)| : z ∈ Δ0 , t = 0, 1, . . . } < ∞.
It is easy now to see that all the results of Sect. 2.2 hold for our model with ut =
wt ◦ V , t = 0, 1, . . . .
If
sup{|wt (z)| : z ∈ Δ0 } → 0 as t → ∞,
then all the results of Sect. 2.7 hold for our model with ut = wt ◦ V , t = 0, 1, . . . .
Chapter 3
Turnpike Properties
In this chapter we consider a discrete-time optimal control problem. The forest

management problem is its particular case. We establish turnpike results for
approximate solutions and the stability of the turnpike phenomenon. We prove the
existence of solutions of the corresponding infinite horizon problems and show the
equivalence of different optimality criterions for these problems. Most results of this
chapter were obtained from [83].
3.1 Preliminaries and Main Results
We continued to study the class of optimal control problems introduced in Sect. 2.2
using the same notation, definitions, and assumptions.
Let (K, ρ) be a compact metric space and let Ω be a nonempty closed subset of
K × K.
A sequence {xt }∞ t=0 ⊂ K is called an (Ω)-program (or a program if Ω is
understood) if (xt , xt+1 ) ∈ Ω for all integers t ≥ 0.
2
1
⊂ K is called an
(Ω)-program (or a program if Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t
satisfying T1 ≤ t < T2 .
For each integer t ≥ 0, let ut : Ω → R 1 be a bounded upper semicontinuous
function.
For each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 and each y, z ∈ K we
consider the optimization problems
2 −1
T
ui (xi , xi+1 ) → max, (P1 )
i=T1

34 3 Turnpike Properties
{xi }Ti=T
2
1
⊂ K is a program, xT1 = y, xT2 = z,
2 −1
T
ui (xi , xi+1 ) → max, (P2 )
i=T1
{xi }Ti=T
2
1
⊂ K is a program, xT1 = y
and
2 −1
T
ui (xi , xi+1 ) → max, (P3 )
i=T1
{xi }Ti=T
2
1
⊂ K is a program.
The interest in discrete-time optimal problems of these types stems from the
study of various optimization problems which can be reduced to it, e.g., continuous-
time control systems which are represented by ordinary differential equations
whose cost integrand contains a discounting factor [16], tracking problems in
engineering [44, 87], the study of Frenkel-Kontorova model related to dislocations
in one-dimensional crystals [6, 71], the analysis of a long slender bar of a
polymeric material under tension in [19, 45, 50, 51], and models of economic
growth [84, 86]. See also [72, 73] where these problems were studied with
Ω = K × K.
For each integer t ≥ 0 set
ut = sup{|ut (z)| : z ∈ Ω} (3.1)
and assume that
sup{ut : t = 0, 1, . . . } < ∞. (3.2)
In the sequel we assume that supremum over empty set is −∞.

For every point y ∈ K and every pair of integers T1 , T2 satisfying T1 < T2 set
⎧ ⎫
⎨T
2 −1 ⎬
U (y, T1 , T2 ) = sup ut (xt , xtt+1 ) : {xt }Tt=T
2
is a program and xT1 = y .
⎩ 1 ⎭
t=T1
(3.3)
Let a point y, ỹ ∈ K be given and let the integers T1 , T2 satisfy T1 < T2 . Set
3.1 Preliminaries and Main Results 35
⎧
⎨T
2 −1
U (y, ỹ, T1 , T2 ) = sup ut (xt , xtt+1 ) :
⎩
t=T1
⎫
⎬
{xt }Tt=T
2
is a program and xT1 = y, xT2 = ỹ . (3.4)
1 ⎭
Let the integers T1 , T2 satisfy T1 < T2 . Define

⎧ ⎫
⎨T2 −1 ⎬
U(T1 , T2 ) = sup ut (xt , xtt+1 ) : {xt }Tt=T
2
is a program . (3.5)
⎩ 1 ⎭
t=T1
In this chapter we suppose that the following assumption, introduced in Sect. 2.2,
holds.
(A) There exists a natural number L̄ such that for every pair of points y, z ∈ K
there exists a program {xt }L̄
t=0 which satisfies x0 = y and xL̄ = z.
This assumption holds for our forest management problems. Note that all the
results stated in Sect. 2.2 (Theorems 2.6, 2.7, 2.9, and 2.10 and Propositions 2.5 and
2.8) are valid.
We suppose that ut = u0 for all integers t ≥ 0. Set
u = u0 and u = sup{|u(z)| : z ∈ Ω}. (3.6)
Let us consider the constant μ defined by Theorem 2.9 and suppose that
μ = μ(u) = sup{u(x, x) : x ∈ K and (x, x) ∈ Ω}. (3.7)
By upper semicontinuity of u, there exists a point x̄ ∈ K such that
(x̄, x̄) ∈ Ω and u(x̄, x̄) = μ. (3.8)
Denote by Card(A) the cardinality of a set A.

A program {xt }∞ t=0 is called (u, Ω)-good ((u)-good if the set Ω is understood or
good if the pair (u, Ω) is understood) if the sequence
T −1 ∞

{u(xt , xt+1 ) − μ
t=0 T =1
is bounded.
A program {xt }∞t=0 is called (u, Ω)-bad ((u)-bad if the set Ω is understood or
T −1
bad if the pair (u, Ω) is understood) if t=0 (u(xt , xt+1 ) − μ) → −∞ as T → ∞.
Theorem 2.9, Proposition 2.8, and Theorem 2.6 imply the following result.
Proposition 3.1 Let {xt }∞
t=0 be a program. Then it is either good or bad. For every
point x0 ∈ K there exists a good program {xt }∞
t=0 .
We study the structure of approximate solutions of problems (P1 )-(P3 ) and

suppose that the following assumptions hold.
(B1) For every good program {xt }∞t=0 , limt→∞ xt = x̄.
(B2) The function u is continuous at the point (x̄, x̄) with respect to Ω.
(B3) There exists a natural number L∗ such that for every positive number there
exists a positive number δ such that for every pair of points y1 , y2 ∈ K which
satisfies
ρ(y1 , x̄), ρ(x̄, y2 ) ≤ δ
there exists a program {xt }L

t=0 such that x0 = y1 , xL∗ = y2 and ρ(xt , x̄) ≤
∗
, t = 0, . . . , L∗ .
We will prove the following results which describe the structure of approximate
solutions of problems (P1 )–(P3 ).
Theorem 3.2 Let be a positive number. Then there exist an integer τ > 0 and a
positive number δ such that for every natural number T > 2τ and every program
{xt }Tt=0 which satisfy
−1
T
u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ
t=0
there exist integers τ1 ∈ [0, τ ], τ2 ∈ [T − τ, T ] such that
ρ(xt , x̄) ≤ , t = τ1 , . . . , τ2 .
Moreover τ1 = 0, if ρ(x0 , x̄) ≤ δ and τ2 = T if ρ(x, x̄) ≤ δ.

Theorem 3.2 is proved in Sect. 3.3.
Theorem 3.3 Let , M be positive numbers. Then there exist integers τ, M1 > 0
such that for every natural number T > τ and every program {xt }Tt=0 which satisfies
−1
T
u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − M
t=0
the inequality
Card({t ∈ {0, . . . , T } : ρ(xt , x̄) > }) < M1
holds.
Theorem 3.4 Let z ∈ K. Then there exists a program {xt }∞
t=0 such that x0 = z and
that for each program {yt }∞
t=0 satisfying y0 = z,
3.1 Preliminaries and Main Results 37
−1
T
lim sup [u(yt , yt+1 ) − u(xt , xt+1 )] ≤ 0. (3.9)
T →∞ t=0

A program {xt }∞t=0 is called (u, Ω)-overtaking optimal ((u)-overtaking optimal
if the set Ω is understood or overtaking optimal if the pair (u, Ω) is understood) if
−1
T
lim sup [u(yt , yt+1 ) − u(xt , xt+1 )] ≤ 0
T →∞ t=0
for all programs {yt }∞

t=0 satisfying x0 = y0 [76, 84, 86].
In other words Theorem 3.4 establishes the existence of an overtaking optimal
program for any initial state.
We also use another optimality criterion which was introduced and applied in
[6, 84, 86].
A program {xt }∞t=0 is (u, Ω)-locally optimal ((u)-locally optimal if the set Ω is
understood or locally optimal if the pair (u, Ω) is understood) if for any integer
T >0
T−1
u(xt , xt+1 ) = U (x0 , xT , 0, T ).
t=0
The following result will be proved in Sect. 3.6.

Theorem 3.5 A program {xt }∞
t=0 is locally optimal if and only if it is overtaking
optimal.
The next result is a generalization of Theorem 3.2 which will be proved in
Sect. 3.7.
Theorem 3.6 Let be a positive number. Then there exist an integer τ > 0 and a
positive number δ such that for every natural number T > 2τ and every program
{xt }Tt=0 satisfying
−1
S+τ
u(xt , xt+1 ) ≥ U (x(S), x(S + τ ), S, S + τ ) − δ
t=S
for all integers S ∈ {0, . . . , T − τ } there exist integers τ1 , τ2 such that
τ1 ∈ [0, τ ], τ2 ∈ [T − τ, T ],
ρ(xt , x̄) ≤ , t = τ1 , . . . τ2 .
Moreover, τ1 = 0 if ρ(x0 , x̄) ≤ δ and τ2 = T if ρ(xT , x̄) ≤ δ.

Proposition 3.7 Let x, y ∈ K and an integer T > 2L̄. Then
U (x, y, 0, T ) ≥ T μ − 4(L̄ + 1)u.
Proof In view of (A) and (3.8), there exists a program {xt }Tt=0 such that
x0 = x, xt = x̄, t = L̄, . . . , T − L̄, xT = y.
Combined with (3.6)–(3.8) this implies that

−1
T
U (x, y, 0, T ) ≥ u(xt , xt+1 ) ≥ T μ − 4(L̄ + 1)u.
t=0
This completes the proof of Proposition 3.7.

Lemma 3.8 Let , M be positive numbers. Then there exists an integer T > 2L̄
such that for every program {xt }Tt=0 which satisfies
−1
T
u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − M,
t=0
there exists an integer j ∈ {1, . . . , T } for which ρ(xj , x̄) ≤ .

Proof Assume the contrary. Then there exist a strictly increasing sequence of
natural numbers {Tk }∞
(k) Tk
k=1 and a sequence of programs {xt }t=0 , k = 1, 2, . . . such
that for every natural number k,
k −1
T
(k) (k) (k) (k)
u xt , xt+1 ≥ U x0 , xTk , 0, Tk − M, (3.10)
t=0

(k)
ρ xj , x̄ > , j = 1, . . . , Tk . (3.11)
In view of (3.10), for every natural number k and every pair of integers S1 , S2 which
satisfy
0 ≤ S1 < S2 ≤ Tk
we have
2 −1
S
(k) (k) (k) (k)
u(xt , xt+1 ) ≥ U (xS1 , xS2 , 0, S2 − S1 ) − M. (3.12)
t=S1
Extracting a subsequence and re-indexing if necessary we may assume that for every
nonnegative integer t ≥ 0 there is
xt = lim xt(k) . (3.13)

k→∞
It follows from Proposition 3.7 and (3.12) that the following property holds:
(P1) For every natural number k and every pair of integers S1 , S2 which satisfy
0 ≤ S1 and S1 + 2L̄ < S2 ≤ Tk , we have
2 −1
S
(k) (k)
u(xt , xt+1 ) ≥ (S2 − S1 )μ − 4(L̄ + 1)u − M.
t=S1
Property (P1) and (3.13) imply that for every pair of integers S1 , S2 ≥ 0 which
satisfy S2 − S1 > 2L̄ we have
2 −1
S
!
u(xt , xt+1 ) ≥ (S2 − S1 )μ − 4 L̄ + 1 u − M. (3.14)
t=S1
Proposition 3.1, (3.14), and property (B1) imply that the program {xt }∞
t=0 is good
and that
lim ρ(xt , x̄) = 0.
t→∞
Thus there exists a natural number S0 such that
ρ(xt , x̄) < /4 for all integers t ≥ S0 . (3.15)
In view of (3.13), there exists a natural number k such that Tk > S0 + 4 and that
(k)
ρ(xS0 , xS0 ) ≤ /4.
(k)
Combined with (3.15) this implies that ρ(xS0 , x̄) ≤ /2. This contradicts (3.11).
The contradiction we have reached proves Lemma 3.8.
Lemma 3.9 Let be a positive number. Then there exits a positive number δ such
that for every natural number T > 2L∗ and every pair of points y, z ∈ K which
satisfies
ρ(y, x̄), ρ(z, x̄) ≤ δ, (3.16)
the inequality
U (y, z, 0, T ) ≥ T μ −
holds.
Proof Since the function u is continuous at the point (x̄, x̄) (see assumption (B2))
there exists a number 0 ∈ (0, ) such that for every point (y, z) ∈ Ω which satisfies
ρ(y, x̄), ρ(z, x̄) ≤ 0
the inequality
|u(y, z) − u(x̄, x̄)| ≤ (4L∗ )−1 (3.17)
holds. In view of assumption (B3), there exists a number δ ∈ (0, 0 ) such that for
every pair of points y, z ∈ K which satisfies
ρ(y, x̄), ρ(z, x̄) ≤ δ
there exists a program {ξt }L∗

t=0 for which
ξ0 = y, ξL∗ = z, ρ(ξt , x̄) ≤ 0 , t = 0, . . . , L∗ . (3.18)
Assume that a natural number T > 2L∗ and that a pair of points y, z ∈ K
satisfies (3.16). It follows from (3.16) and the choice of the number δ (see (3.18))
that there exists a program {xt }Tt=0 such that
x0 = y, ρ(xt , x̄) ≤ 0 , t = 0, . . . , L∗ ,
xt = x̄, t = L∗ , . . . , T − L∗ ,
xT = z, ρ(xt , x̄) ≤ 0 , t = T − L∗ , . . . , T . (3.19)
In view of (3.19), (3.8), and the choice of 0 (see (3.17)),
−1
T ∗ −1
L
U (y, z, 0, T ) ≥ u(xt , xt+1 ) = T μ + [u(xt , xt+1 ) − u(x̄, x̄)]
t=0 t=0
−1
T
+ [u(xt , xt+1 ) − u(x̄, x̄)] ≥ T μ
t=T −L∗
− 2L∗ (4L∗ )−1 ≥ T μ − /2.
This completes the proof of Lemma 3.9.

Lemma 3.10 Let be a positive number. Then there exists a number δ ∈ (0, ) such
that for every natural number T > 2L∗ and every program {xt }Tt=0 which satisfies
−1
T
ρ(x0 , x̄), ρ(xT , x̄) ≤ δ, u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ,
t=0
the inequality
ρ(xt , x̄) ≤ , t = 0, 1, . . . , T
holds.
Proof Let k be a natural number. In view of assumption (B2), there exists a number
k ∈ (0, 2−k ) (3.20)
such that for every point (y, z) ∈ Ω which satisfies
ρ(y, x̄), ρ(z, x̄) ≤ k
the inequality
|u(y, z) − u(x̄, x̄)| ≤ 2−k (3.21)
holds.
Lemma 3.9 implies that there exists a number δk,1 ∈ (0, k ) such that the
following property holds:
(P2) For every natural number T > 2L∗ and every pair of points y, z ∈ K which
satisfies
ρ(y, x̄), ρ(z, x̄) ≤ δk,1 ,
we have
U (y, z, 0, T ) ≥ T μ − 2−k .
In view of assumption (B3), there exists a number δk,2 ∈ (0, k ) such that the
(P3) For every pair of points y, z ∈ K which satisfy
ρ(y, x̄), ρ(z, x̄) ≤ δk,2 ,
there exists a program {xt }L∗

t=0 which satisfies
x0 = y, xL∗ = z
and
ρ(xt , x̄) ≤ k , t = 0, . . . , L∗ .
Choose a positive number
δk < min{δk,1 δk,2 }. (3.22)
We may assume without loss of generality that the sequence {δi }∞ i=1 is monotone
decreasing.
Assume that the assertion of the lemma does not hold. Then for every integer
k ≥ 1 there exist an integer Tk > 2L∗ and a program {xt(k) }Tt=0
k
which satisfies
(k) (k)
ρ(x0 , x̄), ρ(xTk , x̄) ≤ δk , (3.23)
k −1
T
(k) (k) (k) (k)
u(xt , xt+1 ) ≥ U (x0 , xTk , 0, Tk ) − δk , (3.24)
t=0
(k)
max{ρ(xt , x̄) : t = 0, . . . , Tk } > . (3.25)
Set

k
S0 = 0, Sk = (Ti + L∗ ) (3.26)
i=1
for all natural numbers k. By induction define a program {xt∗ }∞

t=0 . Set
xt∗ = xt , t = 0, 1, . . . , T1 .
(1)
(3.27)
It follows from (3.22), (3.23), and property (P3) that there exists a program
1 +L∗
{xt∗ }Tt=T 1
such that
xT∗1 +L∗ = x0 , ρ(xt∗ , x̄) ≤ 1 , t = T1 , . . . , T1 + L∗ .

(2)
(3.28)
It is clear that {xt∗ }St=0

1
is a program.
Assume that k ≥ 1 is an integer and that we have defined a program {xt∗ }St=0
k
such
that
xS∗k = x0(k+1) (3.29)
and that for each integer i ∈ {0, . . . , k − 1}
xS∗i +t = xt
(i+1)
, t = 0, . . . , Ti+1 ,
ρ(xS∗i +Ti+1 +t , x̄) ≤ i+1 , t = 0, . . . , L∗ . (3.30)
(It is clear that this assumption holds for k = 1.)

Set
xS∗k +t = xt
(k+1)
, t = 1, . . . , Tk+1 . (3.31)
In view of (3.31), (3.23), property (P3), (3.26), and (3.22), there exists a program
{xt∗ }t=S
Sk+1
k +Tk+1
such that
xS∗k +1 = x0 , ρ(xt∗ , x̄) ≤ k+1 , t = Sk + Tk+1 , . . . , Sk+1 .

(k+2)
(3.32)
It is not difficult to see that {xt∗ }t=0

S
k+1
is a program and the assumption we made for k
also holds for k + 1. Thus the program {xt∗ }∞ t=0 was defined by induction such that
(3.30) holds for all integers i ≥ 0.
For every nonnegative integer k it follows from (3.30), (3.26), and (3.24), the
choice of k+1 (see (3.21)), property (P2), (3.23), (3.20), and (3.22) that
Sk+1 −1 Tk+1 −1
" #
∗ ∗
u(xt , xt+1 ) − u(x̄, x̄) = u(xt(k+1) , xt+1
(k+1)
) − u(x̄, x̄)
t=Sk t=0
Sk+1 −1
" ∗ ∗ #
+ u(xt , xt+1 ) − u(x̄, x̄)
t=Sk +Tk+1

(k+1) (k+1)
≥ U x0 , xTk+1 , 0, Tk+1 − δk+1
− Tk+1 μ − 2−k+1 L∗ ≥ −2−k+1 (L∗ + 2).
This implies that for every nonnegative integer k we have
k +1
S
k+1
" ∗ ∗ #
u(xt , xt+1 ) − μ ≥ −(L∗ + 2) 2−i+1
t=0 i=0
and by Proposition 3.1 and assumption (B1), the program {xt∗ }∞

t=0 is good and
satisfies
lim x ∗ = x̄.
t→∞
Combined with (3.30) this contradicts (3.25). The contradiction we have reached
proves Lemma 3.10.
We may assume that < 1/2. Lemma 3.10 implies that there exists a number
δ ∈ (0, ) such that the following property holds:
(P4) For every natural number T > 2L∗ and every program {xt }Tt=0 satisfying
ρ(x0 , x̄), ρ(xT , x̄) ≤ δ
and
−1
T
u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ
t=0
the inequality ρ(xt , x̄) ≤ is true for all t = 0, . . . , T .
Lemma 3.8 implies that there exists an integer T∗ > 2L̄ such that the following
property holds:
(P5) For every program {xt }Tt=0

∗
which satisfies
∗ −1
T
u(xt , xt+1 ) ≥ U (x0 , xT∗ , 0, T∗ ) − 4,
t=0
there exists an integer j ∈ {1, . . . , T∗ } for which ρ(xj , x̄) ≤ δ.
Set
τ = T∗ + L ∗ . (3.33)
Assume that a natural number T > 2τ and that a program {xt }Tt=0 satisfies
−1
T
u(xt , xt+1 ) > U (x0 , xT , 0, T ) − δ. (3.34)
t=0
In view of (3.34),
∗ −1
T
u(xt , xt+1 ) ≥ U (x0 , xT∗ , 0, T∗ ) − δ, (3.35)
t=0
−1
T
u(xt , xt+1 ) ≥ U (xT −T∗ , xT , 0, T∗ ) − δ. (3.36)
t=T −T∗
Property (P5), (3.35), and (3.36) imply that there exist integers τ1 ∈ [0, T∗ ], τ2 ∈
[T − T∗ , T ] such that
ρ(xτ1 , x̄) ≤ δ, ρ(xτ2 , x̄) ≤ δ. (3.37)
It is clear that if ρ(x0 , x̄) ≤ δ, then we may set τ1 = 0, and if ρ(xT , x̄) ≤ δ, then
we may set τ2 = T .
In view of (3.34),
2 −1
τ
u(xt , xt+1 ) ≥ U (xτ1 , xτ2 , τ1 , τ2 ) − δ.
t=τ1
Combined with (3.37), property (P4), (3.33), and the inequality T > 2τ this implies
that
ρ(xt , x̄) ≤ , t = τ1 , . . . , τ2 .
Theorem 3.2 implies that there exist a positive number δ and an integer τ0 > 0 such
that the following property holds:
(P6) For every natural number T > 2τ0 and every program {xt }Tt=0 which satisfies
−1
T
u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ,
t=0
there exist integers τ1 ∈ [0, τ0 ], τ2 ∈ [T − τ0 , T ] such that
ρ(xt , x̄) ≤ , t = τ1 , . . . , τ2 .
Choose an integer τ > 0 satisfying
τ > 8τ0 (δ −1 M + 1). (3.38)
Set M1 = τ .
Assume that an integer T > τ and that a program {xt }Tt=0 satisfies
−1
T
u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − M. (3.39)
t=0
Set T0 = 0. By induction we define a strictly increasing sequence of natural numbers

Ti , i = 0, 1, . . . as follows.
Assume that T0 , . . . , Tk have been defined where an integer k ≥ 0. If Tk =

T , then the construction of the sequence is completed. Assume that Tk < T . If
T ≤ Tk + 2τ0 , then set Tk+1 = T and the construction is completed. Assume that
Tk < T − 2τ0 . If
−1
T
u(xt , xt+1 ) ≥ U (xTk , xT , 0, T − Tk ) − δ,
t=Tk
then set Tk+1 = T and the construction is completed. Assume that

−1
T
u(xt , xt+1 ) < U (xTk , xT , 0, T − Tk ) − δ.
t=Tk
Then there exists an integer Tk+1 > Tk such that Tk+1 ≤ T and that
Tk+1 −1

u(xt , xt+1 ) < U (xTk , xTk +1 , 0, Tk+1 − Tk ) − δ, (3.40)
t=Tk
Tk+1 −2

u(xt , xt+1 ) ≥ U (xTk , xTk+1 −1 , 0, Tk+1 − Tk − 1) − δ. (3.41)
t=Tk
Therefore by induction we have constructed a finite sequence Ti , i = 0, . . . , q where

q is a natural number and Tq = T is the last element of the sequence. It follows from
q
(3.39) and the construction of {Tk }k=0 (see (3.40), (3.41)) that
−1
T
M ≥ U (x0 , xT , 0, T ) − u(xt , xt+1 )
t=0
⎧
⎨ Tk+1 −1

≥ U (xTk , xTk+1 , 0, Tk+1 − Tk ) − u(xt , xt+1 ) :
⎩
t=Tk
⎫
⎬
k is an integer such that 0 ≤ k < q − 1 ≥ δ|q − 1|,
⎭
|q − 1| ≤ δ −1 M and q ≤ δ −1 M + 1. (3.42)
It follows from the construction of Ti , i = 0, . . . , q that if an integer k satisfies

0 ≤ k ≤ q − 1 and Tk+1 − Tk − 1 > 2τ0 , then (3.41) is valid. Combined with
property (P6) this implies that
ρ(xt , x̄) ≤ , t = Tk + τ0 , . . . , Tk+1 − 1 − τ0 .

Together with (3.42) and (3.38) this implies that
Card({t ∈ {0, . . . , T } : ρ(xt , x̄) > })
≤ Card({0, . . . , T } \ ∪({[Ti + τ0 , . . . , Ti+1 − 1 − τ0 ] : i is an integer such that
0 ≤ i ≤ q − 1, Ti+1 − Ti − 1 > 2τ0 }))
≤ 2qτ0 + 4qτ0 < 8qτ0 ≤ 8τ0 (δ −1 M + 1) < τ = M1 .
Proposition 2.2 implies that for every natural number T there exists a program
(T )
{xt }Tt=0 which satisfies
−1
T
x0(T ) = z, u xt(T ) , xt+1
(T )
= U (z, 0, T ). (3.43)
t=0
It follows from Proposition 3.7 and (3.43) that for every integer T > 0 and every
pair of integers S1 , S2 satisfying 0 ≤ S1 < S2 ≤ T and S2 − S1 > 2L̄, we have
2 −1
S
(T ) (T )
u xt , xt+1 ≥ (S2 − S1 )μ − 4(L̄ + 1)(u). (3.44)
t=S1
Theorem 3.2 and (3.43) imply that the following property holds:
(P7) For every positive number there exists an integer τ > 0 such that for every
natural number T > 2τ , we have

(T )
ρ xt , x̄ ≤ , t = τ , . . . , T − τ .
It is clear that there exists a strictly increasing sequence of natural numbers {Tj }∞
j =1
such that for all integers t ≥ 0 there exists
(Tj )
xt = lim xt . (3.45)
j →∞
Evidently, {xt }∞
t=0 is a program satisfying
x0 = z. (3.46)
In view of (3.44) and (3.45), for all pairs of integers S1 , S2 satisfying S1 < S2 and
S2 − S1 > 2L̄ the inequality
2 −1
S
!
u(xt , xt+1 ) ≥ (S2 − S1 )μ − 4 L̄ + 1 (u) (3.47)
t=S1
is valid. Proposition 3.1, assumption (B1), and (3.47) imply that {xt }∞
t=0 is a good
program and that
lim xt = x̄. (3.48)
t→∞
We claim that the program {xt }∞ ∞

t=0 satisfies (3.9) for every program {yt }t=0 such that
y0 = z.
Assume the contrary. Then there exists a program {yt }∞ t=0 such that
−1
T
y0 = z, γ := lim sup [u(yt , yt+1 ) − u(xt , xt+1 )] > 0. (3.49)
T →∞ t=0
In view of (3.49), Proposition 3.1, (3.47), and assumption (B1) {yt }∞

=0 is a good
program and
lim yt = x̄. (3.50)

t→∞
Assumption (B2) implies that there exists a number ∈ (0, 1) such that for every
point (y1 , y2 ) ∈ Ω which satisfy
ρ(y1 , x̄), ρ(y2 , x̄) ≤
the inequality
|u(y1 , y2 ) − u(x̄, x̄)| ≤ γ (16L∗ )−1
holds.
Assumption (B3) implies that there exists a number δ ∈ (0, ) such that the
(P8) For every pair of points y1 , y2 ∈ K which satisfies ρ(yi , x̄) ≤ δ, i = 1, 2
there exists a program {zt }L∗
t=0 such that
z0 = y1 , zL∗ = y2 , ρ(yt , x̄) ≤ , t = 0, . . . , L∗ .
It follows from (3.48) and (3.50) that there exists an integer S0 > 0 such that
ρ(xt , x̄), ρ(yt , x̄) ≤ δ/4 for all integers t ≥ S0 . (3.51)

In view of (3.48), there exists an integer S1 > S0 + 4L∗ + 4 for which
1 −1
S
[u(yt , yt+1 ) − u(xt , xt+1 )] > γ /2. (3.52)
t=0
It follows from (3.45) and upper semicontinuity of the function u that there exists
an integer number k > 0 such that
Tk > S1 + 4, (3.53)

(T )
ρ xt k , xt ≤ δ/4, t = 0, . . . , S1 + 4, (3.54)

u xt(Tk ) , xt+1
(Tk )
≤ u(xt , xt+1 ) + γ (16(S1 + 4))−1 , t = 0, . . . , S1 + 4. (3.55)
Relations (3.51), (3.53), and (3.54) imply that

(T ) (T )
ρ(yS1 −L∗ , x̄) ≤ δ/4, ρ xS1 k , x̄ ≤ ρ xS1 k , xS1 + ρ(xS1 , x̄) ≤ δ/4 + δ/4.
(3.56)
By (3.56) and property (P8), there exists a program {x̄t }Tt=0
k
such that
x̄t = yt , t ∈ {0, . . . , S1 − L∗ }, x̄t = xt(Tk ) , t = S1 , . . . , Tk , (3.57)
ρ(x̄t , x̄) ≤ , t = S1 − L∗ , . . . , S1 .
It follows from (3.57), (3.49), (3.43), (3.55), the choice of , (3.51), and (3.52) that
k −1
T k −1
T
(Tk ) (T )
0≤ u(xt , xt+1k ) − u(x̄t , x̄t+1 )
t=0 t=0
1 −1
S 1 −1
S 1 −1
S
u(xt , xt+1 ) + γ (16)−1
(T ) (T )
= u(xt k , xt+1k ) − u(x̄t , x̄t+1 ) ≤
t=0 t=0 t=0
S1 −L
∗ −1
− u(yt , yt+1 ) − (u(x̄, x̄) − γ (16L∗ )−1 )(2L∗ )
t=0
1 −1
S
≤ γ (16)−1 + γ /8 + u(xt , xt+1 )
t=0
1 −1
S
− u(yt , yt+1 ) + γ /8 ≤ γ /4 + γ 16−1 − γ /2 < 0.
t=0
The contradiction we have reached completes the proof of Theorem 3.4.

It is clear that if {xt }∞

t=0 is an overtaking optimal program, then it is locally optimal.
Assume that the program {xt }∞ t=0 is locally optimal. Proposition 3.7 implies that the
program {xt }∞ t=0 is good. By assumption (B1),
lim xt = x̄. (3.58)

t→∞
Theorem 3.4 implies that there exists an overtaking optimal program {yt }∞
t=0
satisfying
y0 = x0 . (3.59)
It is clear that the program {yt }∞

t=0 is good. In view of assumption (B1),
lim yt = x̄. (3.60)

t→∞
We show that the program {xt }∞

t=0 is overtaking optimal. Assume the contrary. Then
T −1

γ := lim sup u(yt , yt+1 ) − u(xt , xt+1 ) > 0. (3.61)
T →∞ t=0
Assumption (B2) implies that there exists a number ∈ (0, γ /2) such that for every
point (z1 , z2 ) ∈ Ω which satisfies
ρ(z1 , x̄), ρ(z2 , x̄) ≤ ,
the inequality
|u(z1 , z2 ) − u(x̄, x̄)| ≤ γ (16L∗ )−1 (3.62)
holds.
In view of assumption (B3), there exists a number δ ∈ (0, ) such that the
(P9) For every pair of points z1 , z2 ∈ K which satisfies ρ(zi , x̄) ≤ δ, i = 1, 2

t=0 such that
ξ0 = z1 , ξL∗ = z2 , ρ(ξt , x̄) ≤ , t = 0, . . . , L∗ .
It follows from (3.58) and (3.60) that there exists an integer S0 > 4L∗ + 4
such that
ρ(xt , x̄), ρ(yt , x̄) ≤ δ for all integers t ≥ S0 − L∗ − 2. (3.63)

In view of (3.61), there exists a natural number S1 > S0 + 2 + 2L∗ such that
1 −1
S
(u(yt , yt+1 ) − u(xt , xt+1 )) > γ /2. (3.64)
t=0
Property (P9) and (3.63) imply that there exists a program {x̃t }St=0
1
such that
x̃t = yt , t = 0, . . . , S1 − L∗ , ρ(xt , x̄) ≤ , t = S1 − L∗ , . . . , S1 , (3.65)
x̃S1 = xS1 .
It follows from local optimality of {xt }∞

t=0 , (3.65), (3.59), (3.64), (3.63), and the
choice of (see (3.62)) that
1 −1
S 1 −1
S
0≤ u(xt , xt+1 ) − u(x̃t , x̃t+1 )
t=0 t=0
1 −1
S 1 −1
S
= u(xt , xt+1 ) − u(yt , yt+1 )
t=0 t=0
1 −1
S 1 −1
S
+ u(yt , yt+1 ) − u(x̃t , x̃t+1 )
t=S1 −L∗ t=S1 −L∗
≤ −γ /2 + γ 16−1 L−1
∗ (2L∗ + 2) < 0,
a contradiction. The contraction we have reached proves Theorem 3.5.
In view of Theorem 3.2, there exist an integer τ0 > 0 and a positive number δ such
(P10) For every natural number T > 2τ0 and every program {xt }Tt=0 which
satisfies
−1
T
u(xt , xt+1 ) ≥ U (x0 , xT , 0, T ) − δ,
t=0
there exist integers τ1 ∈ [0, τ0 ], τ2 ∈ [T − τ0 , T ] such that
ρ(xt , x̄) ≤ , t = τ1 , . . . , τ2 ,
τ1 = 0 if ρ(x0 , x̄) ≤ δ and τ2 = T if ρ(xT , x̄) ≤ δ.
Set
τ = 3τ0 + 1. (3.66)
Assume that an integer T > 2τ and that a program {xt }Tt=0 for every integer S ∈
{0, . . . , T − τ } satisfies
−1
S+τ
u(xt , xt+1 ) ≥ U (x(S), x(S + τ ), S, S + τ ) − δ. (3.67)
t=S
Property (P10), (3.66), and (3.67) imply that
ρ(xt , x̄) ≤ , t = τ0 , . . . , τ − τ0 ,
ρ(xt , x̄) ≤ , t = T − τ + τ0 , . . . , T − τ0 ,
if ρ(x0 , x̄) ≤ δ, then ρ(xt , x̄) ≤ , t = 0, . . . , τ − τ0 and if ρ(xT , x̄) ≤ δ, then
ρ(xt , x̄) ≤ , t = T − τ + τ0 , . . . , T .
In order to complete the proof it is sufficient to show that
ρ(xt , x̄) ≤ , t = τ − τ0 , . . . , T − τ0 .
Assume that an integer
S ∈ [τ − τ0 , T − τ0 ]. (3.68)
It follows from (3.66) and (3.68) that there exist integers S1 , S2 ∈ [0, T ] such that
S1 + τ0 ≤ S ≤ S2 − τ0 , S2 − S1 = τ. (3.69)
In view of (3.69), (3.67), and property (P10),
ρ(xS , x̄) ≤ .

3.8 Stability Results 53
3.8 Stability Results
For every function φ : Ω → R 1 set
φ = sup{|φ(z)| z ∈ Ω}.
Let T1 , T2 be integers such that T1 < T2 and ut : Ω → R 1 , t = T1 , . . . T2 − 1 be

bounded functions. For every pair of points z0 , z1 ∈ K define
⎧
⎨T
2 −1
T2 −1
U {ut }t=T , z 0 , z 1 = sup ut (xt , xt+1 ) : {xt }Tt=T
2
is a program
1 ⎩ 1
t=T1
⎫
⎬
such that xT1 = z0 , xT2 = z1
⎭
and
⎧
⎨T
2 −1
T2 −1
U {ut }t=T , z 0 = sup ut (xt , xt+1 ) : {xt }Tt=T
2
is a program
1 ⎩ 1
t=T1
⎫
⎬
such that xT1 = z0 .
⎭
(Here we assume that supremum over an empty set is −∞.) It is easy to see that the
following result holds.
Proposition 3.11 Let y0 , ỹ0 ∈ K, T1 < T2 be integers, ut , t = T1 , . . . , T2 − 1 be
T2 −1
bounded and upper semicontinuous functions and let U ({ut }t=T 1
, y0 , ỹ0 ) be finite.
Then there exists a program {xt }Tt=T
2
1
such that
2 −1
T
T2 −1
ut (xt , xt+1 ) = U {ut }t=T 1
, y0 , ỹ0 , xT1 = y0 , xT2 = ỹ0 .
t=T1
Theorem 3.12 Let be a positive number. Then there exist an integer τ > 0 and
a positive number γ such that for each natural number T > 2τ , each sequence
of bounded functions ut : Ω → R 1 , t = 0, . . . , T − 1 satisfying ut − u ≤ γ ,
t = 0, 1, . . . , T − 1 and each program {xt }Tt=0 which satisfies
−1
S+τ
S+τ −1
ut (xt , xt+1 ) ≥ U {ut }t=S xS , xS+τ − γ
t=S
for every integer S ∈ {0, . . . , T − τ } there exist integers τ1 , τ2 such that
τ1 ∈ [0, τ ], τ2 ∈ [T − τ, T ],
ρ(xt , x̄) ≤ , t = τ1 , . . . τ2 .
Moreover, τ1 = 0 if ρ(x0 , x̄) ≤ γ and τ2 = T if ρ(xT , x̄) ≤ γ .

Proof Theorem 3.12 follows easily from Theorem 3.6. Namely let a natural number
τ and δ > 0 be as guaranteed by Theorem 3.6. Put
γ = δ(4(τ + 1))−1 .
Now it is not difficult to see that the assertion of Theorem 3.12 holds.
Theorem 3.12 implies the following result.
Theorem 3.13 Let > 0. Then there exist a natural number τ , λ > 1, and γ > 0
such that for each integer T > 2τ , each sequence of bounded functions ut : Ω →
R 1 , t = 0, . . . , T − 1 satisfying ut − u ≤ γ , t = 0, 1, . . . , T − 1, each sequence
T −1
{αi }i=0 ⊂ (0, 1] such that for each i, j ∈ {0, . . . , T − 1} satisfying |i − j | ≤ τ the
inequality αi αj−1 ≤ λ holds and each program {xt }Tt=0 which satisfies
−1
S+τ
S+τ −1
αt ut (xt , xt+1 ) ≥ U {αt ut }t=S , xS , xS+τ − γ αS
t=S
for each integer S ∈ {0, . . . , T − τ } there exist integers τ1 , τ2 such that
τ1 ∈ [0, τ ], τ2 ∈ [T − τ, T ], (3.70)
ρ(xt , x̄) ≤ , t = τ1 , . . . τ2 .
Moreover,
τ1 = 0 if ρ(x0 , x̄) ≤ γ , (3.71)
and
τ2 = T if ρ(xT , x̄) ≤ γ . (3.72)

Theorem 3.14 Let > 0. Then there exist a natural number τ , γ > 0, and λ > 1
such that for each integer T > 2τ , each sequence of bounded upper semicontinuous
functions ut : Ω → R 1 , t = 0, . . . , T −1 satisfying ut −u ≤ γ , t = 0, 1, . . . , T −
T −1
1, each sequence {αi }i=0 ⊂ (0, 1] such that for each i, j ∈ {0, . . . , T −1} satisfying
|i −j | ≤ τ the inequality αi αj−1 ≤ λ holds and each program {xt }Tt=0 which satisfies
−1
T
T −1
αt ut (xt , xt+1 ) = U {αt ut }t=0 , x0 , xT
t=0
there exist integers τ1 , τ2 such that (3.70)–(3.72) hold.

Theorems 3.12–3.14 were obtained from [83]. The next turnpike result is new.
Theorem 3.15 Let , M be positive numbers. Then there exist a natural number
Q and γ > 0 such that for each integer T > Q, each finite sequence of bounded
functions ut : Ω → R 1 , t = 0, . . . , T −1 satisfying ut −u ≤ γ t = 0, 1, . . . , T −
1, and each program {xt }Tt=0 which satisfies
−1
T
T −1
ut (xt , xt+1 ) ≥ U ({ut }t=0 , x0 , xT ) − M
t=0
Card({t ∈ {0, . . . , T } : ρ(xt , x̄) > }) ≤ Q.
Proof Theorem 3.12 implies that there exist an integer τ ≥ 1 and a positive number
γ such that the following property holds:
(i) for each integer S > 2τ , each finite sequence of bounded functions ut : Ω →
R 1 , t = 0, . . . , S − 1 satisfying
ut − u ≤ γ , t = 0, . . . , S − 1
and each program {xt }St=0 which satisfies

S−1
ut (xt , xt+1 ) ≥ U ({ut }S−1
t=0 , x0 , xS ) − γ
t=0
we have
ρ(xt , x̄) ≤ , t = τ, . . . , S − τ.
Choose a natural number
Q > 9τ (γ −1 M + 1). (3.73)

Assume that an integer T > Q, a finite sequence of bounded functions ut : Ω →

R 1 , t = 0, . . . , T − 1 satisfies
ut − u ≤ γ , t = 0, . . . , T − 1 (3.74)
and that a program {xt }Tt=0 satisfies
−1
T
S−1
ut (xt , xt+1 ) ≥ U {ut }t=0 , x0 , xT − M. (3.75)
t=0
Set T0 = 0. By induction we define a strictly increasing sequence of integers T0 <

· · · < Tk ≤ T .
Assume that k ≥ 0 is an integer and T0 , . . . , Tk have been defined. If Tk = T ,
then the construction of the sequence is completed. Assume that Tk < T . In T ≤
Tk + 2τ , then we set Tk+1 = T and the construction is completed.
Assume that
Tk < T − 2τ.
If
−1
T
T −1
ut (xt , xt+1 ) ≥ U {ut }t=Tk
, xTk , xT − γ,
t=Tk
then set Tk+1 = T and the construction is completed.

Assume that
−1
T
T −1
ut (xt , xt+1 ) < U {ut }t=Tk
, xTk , xT − γ .
t=Tk
Then there exists an integer Tk+1 ≤ T such that
Tk+1 ≥ Tk + 2,
Tk+1 −1
Tk+1 −1
ut (xt , xt+1 ) < U {ut }t=Tk
, xTk , xTk+1 −γ (3.76)
t=Tk
and
Tk+1 −2
Tk+1 −2
ut (xt , xt+1 ) ≥ U {ut }t=Tk
, xTk , xTk+1 −1 − γ . (3.77)
t=Tk
Thus by induction we have constructed a finite sequence of integers
T 0 = 0 < · · · , Tq = T ,
q
where q > 0 is an integer. It follows from (3.75) and the construction of {Ti }i=0
(see (3.76) and (3.77)) that
−1
T
T −1
M ≥ U ({ut }t=0 , x0 , xT ) − ut (xt , xt+1 )
t=0
Tk+1 −1
T −1

≥ {U ({ut }t=T
k+1
k
, xTk , xTk+1 ) − ut (xt , xt+1 ) :
t=Tk
k is an integer satisfying 0 ≤ k < q − 1} ≥ γ (q − 1)
and
q ≤ γ −1 M + 1.
It follows from property (i), (3.74), and the construction of Ti , i = 0, . . . , q that if

an integer k satisfies
0 ≤ k ≤ q − 1, Tk+1 − Tk > 2τ + 1,
then (3.77) holds and
ρ(xt , x̄) ≤ , t = Tk + τ, . . . , Tk+1 − τ − 1.
Combined with (3.73) and the inequality
q ≤ γ −1 M + 1,
this implies that
Card({t ∈ {0, . . . , T } : ρ(xt , x̄) > })
≤ Card({t ∈ {0, . . . , T } \ ∪({Ti + τ, . . . , Ti+1 − τ − 1} :
i is an integer satisfying 0 ≤ i ≤ q − 1, Ti+1 − Ti > 2τ + 1)})
≤ 3qτ + 6qτ ≤ 9τ (γ −1 M + 1) < Q.

3.9 Agreeable Programs
A program {xt∗ }∞t=0 is called agreeable if for any natural number T0 and any > 0
there exists an integer T > T0 such that for any integer T ≥ T there exists a
program {xt }Tt=0 which satisfies
xt = xt∗ , t = 0, . . . , T0
and
−1
T
u(xt , xt+1 ) ≥ U (x0∗ , 0, T ) − .
t=0
The notion of agreeable programs is well known in the economic literature [34–
36]. In this section we employ a strong version of it.
A program {xt∗ }∞t=0 is called strongly agreeable if for any natural number T0 and
any > 0 there exist an integer T > T0 and δ > 0 such that for each integer
T ≥ T and each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T
satisfying
ut − u ≤ δ, t = 0 . . . , T − 1
there exists a program {xt }Tt=0 which satisfies
xt = xt∗ , t = 0, . . . , T0
and
−1
T
T −1 ∗
ut (xt , xt+1 ) ≥ U {ut }t=0 , x0 − .
t=0
Theorem 3.16 Let {xt }∞

t=0 be a program. Then the following properties are equiv-
alent:
(a) the program {xt }∞
t=0 is locally optimal;
(b) the program {xt }∞
t=0 is agreeable;
(c) the program {xt }∞
t=0 is strongly agreeable.
Proof Clearly, property (c) implies property (b) and property (b) implies property
(a). We show that (a) implies (c).
Assume that the program {xt }∞ t=0 is locally optimal. We show that the program
{xt }∞
t=0 is strongly agreeable.
Let ∈ (0, 1) and T0 be a natural number. By (B2) there is a positive number
δ1 < /2 such that the following property holds:
3.9 Agreeable Programs 59
(i) for each (x, y) ∈ Ω satisfying
ρ(x, x̄) ≤ δ1 , ρ(y, x̄) ≤ δ1
we have
|u(x, y) − u(x̄, x̄)| ≤ (/8)(L∗ + 1)−1 .
By (B3) there exists δ2 ∈ (0, δ1 ) such that the following property holds:
(ii) for each z1 , z2 ∈ K satisfying
ρ(zi , x̄) ≤ δ2 , i = 1, 2

t=0 such that
ρ(ξt , x̄) ≤ δ1 , t = 0, . . . , L∗ ,
ξ0 = z1 , ξL∗ = z2 .
By Theorem 3.12, there exist a natural number L1 and δ3 ∈ (0, δ2 ) such that
the following property holds:
(iii) for each integer T > 2L1 , each finite sequence of bounded functions ut : Ω →
R 1 , t = 0, . . . , T − 1 satisfying
ut − u ≤ δ3 , t = 0 . . . , T − 1
and each program {zt }Tt=0 which satisfies
−1
T
T −1
ut (zt , zt+1 ) ≥ U ({ut }t=0 , z0 , zT ) − δ3
t=0
we have
ρ(zt , x̄) ≤ δ2 , t = L1 , . . . , T − L1 .
Properties (a) and (iii) imply that
ρ(xt , x̄) ≤ δ2 for all integers t ≥ L1 . (3.78)
Choose an integer
T > T0 + 2L∗ + 2L1 + 8 (3.79)

and set
δ = δ3 (2T + 2)−1 /16. (3.80)
Assume that an integer T ≥ T , a finite sequence of bounded functions ut : Ω →

R 1 , t = 0, . . . , T − 1 satisfies
ut − u ≤ δ, t = 0 . . . , T − 1. (3.81)
There exists a program {yt }Tt=0 such that
y0 = x0 , (3.82)
−1
T
−1
ut (yt , yt+1 ) ≥ U {ut }Tt=0 , x0 − δ. (3.83)
t=0
It follows from property (iii) and (3.79)–(3.83) that
ρ(yt , x̄) ≤ δ2 , t = L1 , . . . , T − L1 . (3.84)
Property (ii), (3.78), (3.79), and (3.84) imply that there exists a program {zt }Tt=0

such that
zt = yt , t = 0, . . . , T − L∗ − L1 , (3.85)
zT −L1 = xT −L1 , (3.86)
ρ(zt , x̄) ≤ δ1 , t = T − L1 − L∗ , . . . , T − L1 , (3.87)
zt = xt , t = T − L1 , . . . , T . (3.88)
By properties (a) and (i) and (3.81), (3.85)–(3.87),
T −L
1 −1 T −L
1 −1
u(xt , xt+1 ) ≥ u(zt , zt+1 )
t=0 t=0
T −L
∗ −L1 −1
≥ u(yt , yt+1 ) + L∗ (u(x̄, x̄)
t=0
− (/8)(L∗ + 1)−1 )
T −L
∗ −L1 −1
≥ u(yt , yt+1 ) + L∗ u(x̄, x̄) − /8. (3.89)
t=0
3.9 Agreeable Programs 61
Property (i), (3.78), (3.79), and (3.89) imply that

T −L
∗ −L1 −1
u(xt , xt+1 )
t=0
T −L
∗ −L1 −1
≥ u(yt , yt+1 ) + L∗ u(x̄, x̄) − /8
t=0
T −L
1 −1
− u(xt , xt+1 )
t=T −L∗ −L1 −1
T −L
∗ −L1 −1
≥ u(yt , yt+1 ) + L∗ u(x̄, x̄) − /8 − L1 u(x̄, x̄) − /8
t=0
T −L
∗ −L1 −1
= u(yt , yt+1 ) − /4. (3.90)
t=0
It follows from (3.80), (3.81), and (3.90) that
T −L
∗ −L1 −1 T −L
∗ −L1 −1
ut (xt , xt+1 ) ≥ ut (yt , yt+1 ) − /4 − /8. (3.91)
t=0 t=0
Property (ii), (3.78), (3.79), and (3.84) imply that there exists a program {ξt }Tt=0 such
that
ξt = xt , t = 0, . . . , T − L∗ − L1 , (3.92)
ρ(ξt , x̄) ≤ δ1 , t = T − L1 − L∗ + 1, . . . , T − L1 , (3.93)
ξt = yt , t = T − L1 , . . . , T . (3.94)
By property (i), (3.78), (3.79), (3.84), and (3.91)–(3.94),

−1
T −1
T
ut (ξt , ξt+1 ) − ut (yt , yt+1 )
t=0 t=0
T −L
∗ −L1 −1 T −L
∗ −L1 −1
≥ ut (xt , xt+1 ) − ut (yt , yt+1 )
t=0 t=0
T −L
1 −1 T −L
1 −1
+ ut (ξt , ξt+1 ) − ut (yt , yt+1 )
t=T −L∗ −L1 t=T −L∗ −L1
≥ −3/8 − 2L∗ (/8)(L∗ + 1)−1 ≥ −5/8.

Combined with (3.80) and (3.83) this implies that
−1
T −1
T
ut (ξt , ξt+1 ) ≥ ut (yt , yt+1 ) − 5/8
t=0 t=0
T −1
≥ U ({ut }t=0 , x0 ) − 5/8 − δ
T −1
≥ U ({ut }t=0 , x0 ) − .
Thus (c) holds. Theorem 3.16 is proved.

Chapter 4
Generic Turnpike Properties
In this chapter we consider a class of discrete-time optimal control problems

identified with a complete metric space of objective functions. Using the Baire
category approach, we show that for most problems the results of Chap. 3 are true.
In particular, a typical (generic) problem possesses the turnpike property.
4.1 Preliminaries
We continue to study the class of optimal control problems introduced in Sect. 2.2
using the same notation, definitions, and assumptions.
K × K.
We recall that a sequence {xt }∞ t=0 ⊂ K is called an (Ω)-program (or a program
if Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t ≥ 0.
2
1
⊂ K is called an
(Ω)-program (or a program if Ω is understood) if (xt , xt+1 ) ∈ Ω for all integers t
satisfying T1 ≤ t < T2 .
Let u : Ω → R 1 be a bounded upper semicontinuous function. Set
u = sup{|u(z)| : z ∈ Ω}.
Recall that the supremum over empty set is −∞.

For each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 and each y, ỹ ∈ K define
⎧ ⎫
⎨T
2 −1 ⎬
U (u, y, T1 , T2 ) = sup u(xt , xtt+1 ) : {xt }Tt=T
2
is a program and xT1 = y ,
⎩ 1 ⎭
t=T1

64 4 Generic Turnpike Properties
⎧
⎨T
2 −1
U (u, y, ỹ, T1 , T2 ) = sup u(xt , xtt+1 ) :
⎩
t=T1
⎫
⎬
{xt }Tt=T
2
is a program and xT1 = y, xT2 = ỹ ,
1 ⎭
⎧ ⎫
⎨T
2 −1 ⎬
(u, T1 , T2 ) = sup
U u(xt , xtt+1 ) : {xt }Tt=T
2
is a program .
⎩ 1 ⎭
t=T1
holds.
(A) There exists a natural number L̄ such that for each y, z ∈ K there is a program
{xt }L̄
This assumption holds for the important class of forest management problems.
Note that in Chaps. 2 and 3 the function u was fixed and for each pair of integers
T1 , T2 satisfying 0 ≤ T1 < T2 and each y, ỹ ∈ K we used the notation
U (y, ỹ, T1 , T2 ) = U (u, y, ỹ, T1 , T2 ),
U (y, T1 , T2 ) = U (u, y, T1 , T2 ),
(T1 , T2 ) = U (u, T1 , T2 ).
U
Note that all the results stated in Sect. 2.2 (Theorems 2.6, 2.7, 2.9, and 2.10 and
Propositions 2.5 and 2.8) hold in our case with ut = u, t = 0, 1, . . . .
The results of this chapter are new.
4.2 Equivalence of the Turnpike Properties
We use the constant μ defined by Theorem 2.9 and suppose that
μ = sup{u(x, x) : x ∈ K and (x, x) ∈ Ω}. (4.1)
By the upper semicontinuity of u, there exists x̄ ∈ K such that
(x̄, x̄) ∈ Ω and u(x̄, x̄) = μ. (4.2)
Recall that a program {xt }∞

t=0 is called (u)-good (or good if the function u is
understood) if the sequence
4.2 Equivalence of the Turnpike Properties 65
T −1 ∞

{u(xt , xt+1 ) − μ
t=0 T =1
is bounded.
A program {xt }∞ t=0 is called (u)-bad (or bad if the function u is understood) if
T −1
t=0 (u(xt , xt+1 ) − μ) → −∞ as T → ∞.
By Proposition 3.1, any program is either good or bad and for any x0 ∈ K there
is a good program {xt }∞ t=0 .
We suppose that assumptions (B2) and (B3) introduced in Sect. 3.1 hold.
(B2) u is continuous at (x̄, x̄) with respect to Ω.

(B3) There exists a natural number L∗ such that for each > 0 there exists δ > 0
such that for each y1 , y2 ∈ K satisfying ρ(y1 , x̄), ρ(x̄, y2 ) ≤ δ there is
a program {xt }Lt=0 such that x0 = y1 , xL∗ = y2 and ρ(xt , x̄) ≤ , t =
∗
0, . . . , L∗ .
In Chap. 3 we also use the following assumption.
(B1) For each good program {xt }∞

t=0 , limt→∞ xt = x̄.
Property (B1) is known in the literature and is called as the asymptotic turnpike
property. Clearly, if (B1) holds, then all the results of Chap. 3 are true. In particular,
Theorem 3.2, a turnpike result, is true. The next result shows that if the turnpike
property established by Theorem 3.2 holds, then (B1) holds too.
Theorem 4.1 Assume that for each > 0 there exist a natural number τ and δ > 0
such that the following property holds:
(i) for each integer T > 2τ and each program {zt }Tt=0 satisfying
−1
T
u(zt , zt+1 ) ≥ U (z0 , zT , 0, T ) − δ
t=0
the inequality
ρ(zt , x̄) ≤
holds for all integers t = τ, . . . , T − τ.

Then for each good program {xt }∞
t=0 ,
lim xt = x̄.
t→∞
Proof Assume that {xt }∞

t=0 is a good program. Then there exists M > 0 such that
for each natural number T ,
T −1

u(xt , xt+1 ) − T μ ≤ M. (4.3)

t=0
Let > 0 and δ > 0 and a natural number τ be such that property (i) holds. We
show that there exists an integer Tδ ≥ 0 such that for each integer T > Tδ ,
−1
T
u(xt , xt+1 ) ≥ U (xTδ , xT , Tδ , T ) − δ. (4.4)
t=Tδ
Assume the contrary. Then there exists a strictly increasing sequence of integers
{Ti }∞
i=0 such that T0 = 0 and for each integer i ≥ 1,
Ti+1 −1

u(xt , xt+1 ) < U (xTi , xTi+1 , Ti , Ti+1 ) − δ. (4.5)
t=Ti
By (4.5), there exists a program {yt }∞

t=0 such that for each integer i ≥ 0,
yTi = xTi ,
Ti+1 −1 Ti+1 −1

u(xt , xt+1 ) < u(yt , yt+1 ) − δ.
t=Ti t=Ti
In view of the relation above and (4.3) for each natural number q,
Tq −1

u(yt , yt+1 ) − Tq μ
t=0
Tq −1 Tq −1 Tq −1

= u(yt , yt+1 ) − u(xt , xt+1 ) + u(xt , xt+1 ) − Tq μ
t=0 t=0 t=0
≥ qδ − M → ∞ as q → ∞.
This contradicts Proposition 3.1. The contradiction we have reached proves that
there exists an integer Tδ ≥ 0 such that the following property holds:
(ii) for each integer T > Tδ , (4.4) is true.
4.3 Generic Results 67
Properties (i) and (ii), (4.9), and the choice of δ, τ imply that for each integer
T > Tδ + 2τ , we have
ρ(xt , x̄) ≤ , t = τ + Tδ , . . . , T − τ.
This implies that
ρ(xt , x̄) ≤ for all integers t ≥ τ + Tδ .
4.3 Generic Results
We use the notation, definitions, and assumptions introduced in Sect. 4.1. Denote by
C(Ω) the set of all continuous functions on Ω. Clearly, (C(Ω), · ) is a Banach
space. For each u1 , u2 ∈ C(Ω) set
d(u1 , u2 ) = u1 − u2 .
Let v ∈ C(Ω). By Theorem 2.9 there exist
(v, 0, p)p−1
μ(v) = lim U (4.6)
p→∞
and Mv > 0 such that

(v, 0, p)p−1 ≤ Mv /p
μ(v) − U (4.7)
for all natural numbers p.

Assume that there exists z ∈ K such that
(z, z) ∈ Ω.
Denote by M the set of all v ∈ C(Ω) such that
μ(v) = sup{v(x, x) : x ∈ K and (x, x) ∈ Ω}. (4.8)
It is not difficult to see that M is a closed subset of C(Ω). We consider the complete
metric space (M, d). Suppose that the following assumption holds.
such that for each y1 , y2 ∈ K satisfying ρ(y1 , y2 ) ≤ δ there is a program
{xt }L∗
t=0 such that
x0 = y1 , xL∗ = y2
and
ρ(xt , x̄) ≤ , t = 0, . . . , L∗ .
Clearly, (B4) is a strong version of (B3).

Let v ∈ M. Recall that a program {xt }∞ t=0 is called (v)-good if the sequence
T −1 ∞

{v(xt , xt+1 ) − μ(v)
t=0 T =1
is bounded.
Theorem 4.2 There exists a set F ⊂ M which is a countable intersection of open
everywhere dense subsets of M such that for each v ∈ F the following properties
hold:
1. there exists a unique point xv ∈ K such that
μ(v) = v(xv , xv ).
2. for each (v)-good program {xt }∞

t=0 ,
lim ρ(xt , x̄) = 0.

t→∞
It is clear that if u ∈ F, then all the results of Chap. 3 hold for u.

Proof of Theorem 4.2 Let v ∈ M and r > 0. There exists xv ∈ K such that
(xv , xv ) ∈ Ω, μ(v) = v(xv , xv ). (4.9)
For (x, y) ∈ Ω define
vr (x, y) = v(x, y) − rρ(x, xv ). (4.10)
It is not difficult to see that vr ∈ M and
μ(vr ) = μ(v) = v(xv , xv ). (4.11)
Clearly, the set
{vr : v ∈ M, r ∈ (0, 1]}
is an everywhere dense subset of M.

Lemma 4.3 Let v ∈ M and r ∈ (0, 1]. Assume that {xt }∞
t=0 is a (vr )-good
program. Then
lim ρ(xt , xv ) = 0.
t→∞
Proof There exists M > 0 such that

T −1

vr (xt , xt+1 ) − T v(xv , xv ) ≤ M for all natural numbers T. (4.12)

t=0
Proposition 3.1, (4.10), and (4.12) imply that for all natural numbers T ,
−1
T −1
T
−M ≤ v(xt , xt+1 ) − T v(xv , xv ) − r ρ(xt , xv ). (4.13)
t=0 t=0
In view of (4.13), the program {xt }∞

t=0 is (v)-good and
−1
T
lim ρ(xt , xv ) < ∞.
T →∞
t=0
This implies that
lim ρ(xt , xv ).
t→∞

In view of Lemma 4.3, all the results of Chap. 3 are true for u = vr .
Completion of the Proof of Theorem 4.2 Let v ∈ M, r ∈ (0, 1] and n be a natural
number. Theorem 3.12 implies that there exist an open neighborhood U (v, r, n) of
vr in the metric space (M, d), a natural number τ (v, r, n), and γ (v, r, n) > 0 such
(i) for each integer T > 2τ (v, r, n), each ut ∈ U (v, r, n), t = 0, . . . , T − 1 and
each program {xt }Tt=0 which satisfies
−1
T
T −1
ut (xt , xt+1 ) ≥ U {ut }t=0 x0 , xT − γ (v, r, n)
t=0
we have
ρ(xt , xv ) ≤ 1/n, t = τ (v, r, n), . . . , T − τ (v, r, n).
Define
F = ∩∞
p=1 ∪ {U (v, r, n) : v ∈ M, r ∈ (0, 1], n ≥ p is an integer}. (4.14)
Clearly, F is a countable intersection of open everywhere dense sets in (M, d).

Let
w ∈ F, ∈ (0, 1). (4.15)
Choose a natural number p such that
p > 4 −1 . (4.16)
It follows from (4.14) and (4.15) that there exist v ∈ M, r ∈ (0, 1], and an integer
n ≥ p such that
w ∈ U (v, r, n). (4.17)
Assume that
ξ1 , ξ2 ∈ K,
w(ξi , ξi ) = μ(w), i = 1, 2. (4.18)
It is not difficult to see that for each i = 1, 2 and each integer T > 0,
U (w, ξi , ξi , 0, T ) = T w(ξi , ξi ).
Combined with property (i) this implies that
ρ(xv , ξi ) ≤ n−1 , i = 1, 2. (4.19)
By (4.16) and (4.19),
ρ(ξ1 , ξ2 ) ≤ 2/n ≤ 2/p < .
Since is an arbitrary positive number we conclude that ξ1 = ξ2 and
{ξ ∈ K : (ξ, ξ ) ∈ Ω, w(ξ, , ξ ) = μ(w)} = {ξ1 },
a singleton.
Assume that an integer T > 2τ (v, r, n) and that a program {xt }Tt=0 satisfies
−1
T
w(xt , xt+1 ) ≥ U (w, x0 , xT , 0, T ) − γ (v, r, n).
t=0
Together with property (i), (4.16), (4.17), and (4.19) this implies that for all integers
t = τ (v, r, n), . . . , T − τ (v, r, n),
ρ(xt , xv ) ≤ 1/n
and
ρ(xt , ξ1 ) ≤ ρ(xt , xv ) + ρ(xv , ξ1 ) ≤ 2/n ≤ 2/p < .
Combined with Theorem 4.1 this implies that every (w)-good program converges
to ξ1 . Theorem 4.2 is proved.
Chapter 5
Structure of Solutions in the Regions
Close to the Endpoints
In this chapter we continue to study the structure of approximate solutions of

the autonomous nonconcave discrete-time optimal control system with a compact
metric space of states. This control system is described by a bounded upper
semicontinuous objective function which determines an optimality criterion. In the
turnpike theory, it is known that approximate solutions are determined mainly by
the objective function, and are essentially independent of the choice of time intervals
and data, except in regions close to the endpoints of the time intervals. In this chapter
our main goal is to analyze the structure of approximate solutions in regions close
to the endpoints of the time intervals.
5.1 Preliminaries
We continued to study the class of optimal control problems studied in Chap. 3 (see
Sects. 3.1 and 3.8) using the same notation, definitions, and assumptions.
K × K.
Let u : Ω → R 1 be a bounded upper semicontinuous function.
holds.
(A) There exists a natural number L̄ such that for each y, z ∈ K there is a (Ω)-
program {xt }L̄
We also assume that there is x̄ ∈ K such that
(x̄, x̄) ∈ Ω and u(x̄, x̄) = μ(u) = μ. (5.1)

74 5 Structure of Solutions in the Regions Close to the Endpoints
We also assume that the following assumptions (B1)–(B3) introduced in Sect. 3.1
hold.
(B1) For each (u, Ω)-good program {xt }∞ t=0 , limt→∞ xt = x̄.
(B2) u is continuous at (x̄, x̄) with respect to Ω.
such that for each y1 , y2 ∈ K satisfying ρ(y1 , x̄), ρ(x̄, y2 ) ≤ δ there is an
(Ω)-program {xt }L t=0 such that x0 = y1 , xL∗ = y2 and ρ(xt , x̄) ≤ , t =
∗
0, . . . , L∗ .
We study the structure of approximate solutions of problems (P1 )–(P3 ) (see
Sect. 3.1).
We define a function π u (x), x ∈ K which plays an important role in our study.
Let x ∈ K. Set
−1
T
π (x) = sup lim sup
u
(u(xt , xt+1 ) − v(x̄, x̄)) :
T →∞ t=0

{xt }∞
t=0 is an (Ω) − program such that x0 = x . (5.2)
In view of Theorems 2.6 and 2.9, there exists Mu > 0 such that
− Mu ≤ π u (x) ≤ Mu for all x ∈ K. (5.3)
Let x ∈ K. By (5.3),
−1
T
π u (x) = sup lim sup (u(xt , xt+1 ) − u(x̄, x̄)) :
T →∞ t=0

{xt }∞
t=0 is an (u, Ω) − good program such that x0 = x . (5.4)
Denote by P(u, x) the set of all (u, Ω)-overtaking optimal programs {xt }∞ t=0
satisfying x0 = x. By Theorem 3.4, the set P(u, x) is nonempty. Definition (5.2)
implies the following result.
Proposition 5.1 Let T ≥ 1 be an integer and {xt }Tt=0 be a (Ω)-program. Then for
each integer t = 0, . . . , T − 1,
π u (xt ) ≥ u(xt , xt+1 ) − u(x̄, x̄) + π u (xt+1 ).
The next result follows from the definition of (u, Ω)-overtaking optimal pro-
grams.
5.1 Preliminaries 75

t=0 be a (u, Ω)-overtaking optimal program. Then
−1
T
π u (x0 ) = lim sup (u(xt , xt+1 ) − u(x̄, x̄)).
T →∞ t=0
Corollary 5.3 Let {xt }∞

t=0 be a (u, Ω)-overtaking optimal program. Then for any
integer t ≥ 0,
π u (xt ) = u(xt , xt+1 ) − u(x̄, x̄) + π u (xt+1 ). (5.5)
Set
sup(π u ) = sup{π u (z) : z ∈ K}. (5.6)
Proposition 5.4 π u (x̄) = 0.

Proof Set xt = x̄ for all integers t ≥ 0. By Theorem 3.5 and (5.1), the program
{xt }∞
t=0 is a (u, Ω)-overtaking optimal. In view of Proposition 5.2, π (x̄) = 0.
u

Proposition 5.5 The function π u is continuous at x̄.
Proof Let > 0. In view of (B2), there exists 1 ∈ (0, ) such that
|u(x, y) − u(x̄, x̄)| ≤ (4L∗ )−1
for each (x, y) ∈ Ω satisfying ρ(x, x̄), ρ(y, x̄) ≤ 1 . (5.7)
By (B3), there exists δ ∈ (0, 1 ) such that the following property holds:
(i) for each z1 , z2 ∈ K satisfying
ρ(zi , x̄) ≤ δ, i = 1, 2
there exists an (Ω)-program {xt }L∗

t=0 such that
x0 = z1 , xL∗ = z2
and
ρ(xt , x̄) ≤ 1 , t = 0, . . . , L∗ .
Assume that
z1 , z2 ∈ K, ρ(zi , x̄) ≤ δ, i = 1, 2. (5.8)

Property (i) and (5.8) imply that there exists an (Ω)-program {xt }L∗
t=0 such that
x0 = z1 , xL∗ = z2 , (5.9)
and
ρ(xt , x̄) ≤ 1 , t = 0, . . . , L∗ . (5.10)
By (5.7) and (5.10), for all t = 0, . . . , L∗ − 1,
|u(xt , xt+1 ) − u(x̄, x̄)| ≤ (4L∗ )−1 .
It follows from the relation above, (5.2), and (5.9) that
π u (z1 ) = π u (x0 )
∗ −1
L
≥ (u(xt , xt+1 ) − u(x̄, x̄)) + π u (xL∗ )
t=0
≥ −4−1 (/L∗ )L∗ + π u (z2 ) = π u (z2 ) − .

Proposition 5.6 Assume that x0 ∈ K and {xt }∞
t=0 ∈ P(u, x0 ). Then
−1
T
π (x0 ) = lim
u
(u(xt , xt+1 ) − u(x̄, x̄)).
T →∞
t=0
Proof It follows from (B1), Propositions 5.4, 5.5, and Corollary 5.3 that
π u (x0 ) = lim (π u (x0 ) − π u (xT ))

T →∞
−1
T
= lim (u(xt , xt+1 ) − u(x̄, x̄)).
T →∞
t=0

Proposition 5.7 The function π u : K → R 1 is upper semicontinuous.
Proof Assume that {x (i) }∞
i=1 ⊂ K, x ∈ K and
lim x (i) = x. (5.11)

i→∞
We show that

π u (x) ≥ lim sup π u x (i) .
i→∞
We may assume without loss of generality that

lim sup π u x (i) = lim π u x (i) . (5.12)
i→∞ i→∞
By Theorem 3.4 and Proposition 5.6, for each integer i ≥ 1, there exists a (u, Ω)-
overtaking optimal program {xt }∞
(i)
t=0 such that
(i)
x0 = x (i) , (5.13)
−1
T
(i) (i)
π u x (i) = lim u(xt , xt+1 ) − u(x̄, x̄) . (5.14)
T →∞
t=0
Extracting a subsequence and re-indexing, if necessary, we may assume without loss

of generality that for each integer t ≥ 0 there exists
(i)
xt = lim xt . (5.15)
i→∞
In view of (5.15), {xt }∞

t=0 is an (Ω)-program.
Let > 0. By Propositions 5.4 and 5.5, there exists δ ∈ (0, ) such that for each
x ∈ K satisfying ρ(x, x̄) ≤ δ,
|π u (x)| ≤ /2. (5.16)
Since {xt(i) }∞
t=0 , i = 1, 2, . . . are (u, Ω)-overtaking optimal programs it follows
from Theorem 3.2 that there exists a natural number L1 such that for all sufficiently
large natural numbers i,

(i)
ρ xt , x̄ ≤ δ for all integers t ≥ L1 . (5.17)
Let T ≥ L1 be an integer. By Corollary 5.3, (5.16), and (5.17), for all integers i ≥ 1,
−1
T
(i) (i)
(u(xt , xt+1 ) − u(x̄, x̄))
t=0
(i) (i) (i)
= π u (x0 ) − π u (xT ) ≥ π u (x0 ) − /2.
In view of the relation above and the upper semicontinuity of u, for all integers
T ≥ L1 ,
−1
T
(u(xt , xt+1 ) − u(x̄, x̄))
t=0
−1
T
≥ lim sup u(xt(i) , xt+1
(i)
) − u(x̄, x̄)
i→∞ t=0

≥ lim sup π u x0(i) − /2.
i→∞
By (5.2), (5.11), (5.13), (5.15), and the relation above,
−1
T
π u (x) ≥ lim sup (u(xt , xt+1 ) − u(x̄, x̄)) ≥ lim sup π u x (i) − /2.
T →∞ t=0 i→∞
Since is any positive number we conclude that

π u (x) ≥ lim sup π u x (i) .
i→∞

t=0 be an (Ω)-program such that for all integers t ≥ 0,
u(xt , xt+1 ) − u(x̄, x̄) = π u (xt ) − π u (xt+1 ). (5.18)
Then {xt }∞
t=0 is a (u, Ω)-overtaking optimal program.
Proof Proposition 5.1 and (5.18) imply that {xt }∞
t=0 is a (u, Ω)-locally optimal
program. By Theorem 3.5, {xt }∞
t=0 is a (u, Ω)-overtaking optimal program. Propo-
sition 5.8 is proved.
5.2 Lagrange Problems
In order to study the structure of solutions of the problems in the regions close to
the end points we introduce the following notation and definitions.
Set
Ω̄ = {(x, y) ∈ K × K : (y, x) ∈ Ω}. (5.19)
Clearly, Ω̄ is a nonempty closed subset of K × K.

5.2 Lagrange Problems 79
For each function w : Ω → R 1 define w̄ : Ω̄ → R 1 by
w̄(x, y) = w(y, x), (x, y) ∈ Ω̄. (5.20)
Let 0 ≤ T1 < T2 be integers and let {xt }Tt=T

2
1
be an (Ω)-program. Define
{x̄t }Tt=T
2
1
⊂ K by
x̄t = xT2 −t+T1 , t = T1 , . . . , T2 . (5.21)
Clearly, {x̄t }Tt=T

2
1
is an (Ω̄)-program.
Assume that ut : Ω → R 1 , t = T1 , . . . , T2 − 1. It is easy to see that
2 −1
T
ūT2 −t+T1 −1 (x̄t , x̄t+1 )
t=T1
2 −1
T
= uT2 −t+T1 −1 (xT2 −t+T1 −1 , xT2 −t+T1 )
t=T1
2 −1
T
= ut (xt , xt+1 ). (5.22)
t=T1
The next result easily follows from (5.22).

Proposition 5.9 Let T ≥ 1 be an integer, M ≥ 0, ut : Ω → R 1 , t = 0, . . . , T
(i)
and {xt }Tt=0 , i = 1, 2 are (Ω)-programs. Then
−1
T T
−1
(1) (1) (2) (2)
ut xt , xt+1 ≥ ut xt , xt+1 − M
t=0 t=0
if and only if
−1
T T
−1
(1) (1) (2) (2)
ūT −t−1 x̄t , x̄t+1 ≥ ūT −t−1 xt , xt+1 − M.
t=0 t=0
Let T1 , T2 be integers such that T1 < T2 and ut : Ω → R 1 , t = T1 , . . . T2 − 1 be

bounded functions. For each z0 , z1 ∈ K set
⎧ ⎫
⎨T 2 −1 ⎬
T2 −1 T2
U {ut }t=T1 , Ω = sup ut (xt , xt+1 ) : {xt }t=T1 is an (Ω)-program ,
⎩ ⎭
t=T1
(5.23)
⎧
⎨T
2 −1
T2 −1
U {ut }t=T , Ω, z0 = sup ut (xt , xt+1 ) : {xt }Tt=T
2
is an (Ω)-program
1 ⎩ 1
t=T1
⎫
⎬
such that xT1 = z0 , (5.24)
⎭
⎧
⎨T
2 −1
{ut }T2 −1 , Ω, z1 = sup
U ut (xt , xt+1 ) : {xt }Tt=T
2
is an (Ω)-program
t=T1 ⎩ 1
t=T1
⎫
⎬
⎭
⎧
⎨T
2 −1
T2 −1
U {ut }t=T , Ω, z 0 , z 1 = sup ut (xt , xt+1 ) : {xt }Tt=T
2
is an (Ω)-program
1 ⎩ 1
t=T1
⎫
⎬
such that xT1 = z0 , xT2 = z1 . (5.26)
⎭
Let T1 , T2 be integers such that T1 < T2 and wt : Ω̄ → R 1 , t = T1 , . . . T2 − 1

be bounded functions. For each z0 , z1 ∈ K set
⎧
⎨T2 −1
T2 −1
U {wt }t=T1 , Ω̄, z0 = sup wt (xt , xt+1 ) : {xt }Tt=T
2
is an (Ω̄)-program
⎩ 1
t=T1
⎫
⎬
⎭
⎧
⎨T
2 −1
T2 −1
U {wt }t=T1 , Ω̄, z0 , z1 = sup wt (xt , xt+1 ) : {xt }Tt=T
2
⎩ 1
t=T1
⎫
⎬
such that xT1 = z0 , xT2 = z1 , (5.28)
⎭
5.2 Lagrange Problems 81
⎧
⎨T
2 −1
{wt }T2 −1 , Ω̄, z1 = sup
U wt (xt , xt+1 ) : {xt }Tt=T
2
t=T1 ⎩ 1
t=T1
$
⎧ ⎫
⎨T
2 −1 ⎬
T2 −1 T2
U {wt }t=T , Ω̄ = sup w (x , x
t t t+1 ) : {x }
t t=T1 is an ( Ω̄)-program .
1 ⎩ ⎭
t=T1
(5.30)
Proposition 5.9 implies the following result.
Proposition 5.10 Let T ≥ 1 be an integer, M ≥ 0, ut : Ω → R 1 , t = 0, . . . T − 1
be bounded functions and {xt }Tt=0 be an (Ω)-program. Then {x̄t }Tt=0 is an (Ω̄)-
program and the following assertions
hold:
T −1 T −1
if t=0 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω − M, then
−1
T
T −1
ūT −t−1 (x̄t , x̄t+1 ) ≥ U {ūT −t−1 }t=0 , Ω̄ − M;
t=0
T −1
T −1
if t=0 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, x0 , xT − M, then
−1
T
−1
ūT −t−1 (x̄t , x̄t+1 ) ≥ U {ūT −t−1 }Tt=0 , Ω̄, x̄0 , x̄T − M;
t=0
T −1
T −1
if t=0 ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, x0 − M, then
−1
T
{ūT −t−1 }T −1 , Ω̄, x̄T − M;
ūT −t−1 (x̄t , x̄t+1 ) ≥ U t=0
t=0
T −1
if {ut }T −1 , Ω, xT − M, then
ut (xt , xt+1 ) ≥ U
t=0 t=0
−1
T
T −1
ūT −t−1 (x̄t , x̄t+1 ) ≥ U {ūT −t−1 }t=0 , Ω̄, x̄0 − M.
t=0
Note that the pair (ū, Ω̄) satisfies assumptions (A), (B2), and (B3), and that
μ(ū) = μ(u) = μ.
It follows from Theorems 3.2 and 4.1 and Proposition 5.10 that (B1) also holds for
the pair (ū, Ω̄). Therefore all the results presented above for the pair (u, Ω) are also
true for the pair (ū, Ω̄).
We prove the following results which describe the structure of approximate
solutions of our optimal control problems in the regions close to the endpoints.
Theorem 5.11 Let τ0 ≥ 1 be an integer and > 0. Then there exist δ > 0 and an
integer T0 ≥ τ0 such that for each integer T ≥ T0 , each finite sequence of bounded
functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfying
ut − u ≤ δ, t = 0 . . . , T − 1
and each (Ω)-program {xt }Tt=0 which satisfies
−1
T
T −1
ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, x0 − δ
t=0
there exists a (ū, Ω̄)-overtaking optimal program {xt∗ }∞

t=0
π ū (x0∗ ) = sup(π ū ),
ρ(xT −t , xt∗ ) ≤ , t = 0, . . . , τ0 .
ut − u ≤ δ, t = 0 . . . , T − 1
−1
T
T −1
ut (xt , xt+1 ) ≥ U {ut }t=0 ,Ω − δ
t=0
there exist a (u, Ω)-overtaking optimal program {yt }∞

(1)
t=0 and a (ū, Ω̄)-overtaking
optimal program {yt }∞
(2)
t=0 such that

(1) (2)
π u y0 = sup(π u ), π ū y0 = sup(π ū )
and that for all integers t = 0, . . . , τ0 ,

(2) (1)
ρ xT −t , yt ≤ , ρ xt , yt ≤ .
ut − u ≤ δ, t = 0 . . . , T − 1
−1
T
{ut }T −1 , Ω, xT − δ
ut (xt , xt+1 ) ≥ U t=0
t=0
there exists a (u, Ω)-overtaking optimal program {xt∗ }∞

t=0 such that
π u (x0∗ ) = sup(π u ),
ρ(xt , xt∗ ) ≤ , t = 0, . . . , τ0 .
Theorem 5.13 is proved in Sect. 5.4 while Sect. 5.3 contains an auxiliary result.
Theorem 5.11 follows from Theorem 5.13 applied with (ū, Ω̄) and Proposition 5.10.
Theorem 5.12 easily follows from Theorems 5.11 and 5.13.
Lemma 5.14 Let T0 ≥ 1 be an integer and ∈ (0, 1). Then there exists δ ∈ (0, )
such that for each (Ω)-program {xt }Tt=0
0
which satisfies
π u (x0 ) ≥ sup(π u ) − δ,
0 −1
T
(u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xT ) ≥ −δ,
t=0
there exists a (u, Ω)-overtaking optimal program {zt }∞

t=0 such that
π u (z0 ) = sup(π u ),
ρ(zt , xt ) ≤ , t = 0, . . . , T0 . (5.31)
Proof Assume that the lemma does not hold. Then there exist a sequence {δk }∞
k=1 ⊂
(k) T0
(0, 1) and a sequence of (Ω)-programs {xt }t=0 , k = 1, 2, . . . such that
lim δk = 0 (5.32)
k→∞
and that for each integer k ≥ 1 and each (u, Ω)-overtaking optimal program {zt }∞
t=0
satisfying (5.31),

(k)
π u x0 ≥ sup(π u ) − δk , (5.33)
0 −1
T
(k) (k) (k) (k)
u xt , xt+1 − u(x̄, x̄) − π u x0 + π u xT0 ≥ −δk (5.34)
t=0
we have

(k)
max ρ zt , xt : t = 0, . . . , T0 > . (5.35)

of generality that for each integer t ∈ [0, T0 ] there exists
(k)
xt = lim xt . (5.36)
k→∞
Proposition 5.7, (5.32), and (5.33) imply that

(k)
sup(π u ) ≥ π u (x0 ) ≥ lim sup π u x0 ≥ sup(π u ). (5.37)
k→∞
By upper semicontinuity of u, Proposition 5.7, (5.32)–(5.34), (5.36), and (5.37),
0 −1
T
(u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xT0 )
t=0
⎛ ⎞
0 −1
T
≥ lim sup ⎝ (u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xT0 )⎠
(k) (k) (k) (k)
k→∞ t=0
≥ lim sup(−δk ) = 0.
k→∞
Thus
0 −1
T
(u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xT0 ) ≥ 0.
t=0
Together with Proposition 5.1 this implies that for all integers t = 0, . . . , T0 − 1,

Theorem 3.4 implies that there is a (u, Ω)-overtaking optimal program {x̃t }∞
t=0
satisfying
x̃0 = xT0 . (5.39)
For all integers t > T0 set
xt = x̃t−T0 . (5.40)
It is clear that {xt }∞ ∞

t=0 is an (Ω)-program. Since {x̃t }t=0 is a (u, Ω)-overtaking
optimal program, Corollary 5.3 implies that (5.38) holds for all integers t ≥ 0. It
follows from (5.38) and Proposition 5.8 that {xt }∞
t=0 is a (u, Ω)-overtaking optimal
program. In view of (5.37),
π u (x0 ) = sup(π u ).
(k)
By (5.36), for all sufficiently large natural numbers k, ρ(xt , xt ) ≤ /4, t =
0, . . . , T0 . This contradicts (5.35). The contradiction we have reached proves
Lemma 5.14.
We may assume that < 1. By Lemma 5.14, there exists δ1 ∈ (0, /4) such that the
(i) for each (Ω)-program {yt }τt=0
0
which satisfies
π u (y0 ) ≥ sup(π u ) − δ1 ,
0 −1
τ
(u(yt , yt+1 ) − u(x̄, x̄)) − π u (y0 ) + π u (yτ0 ) ≥ −δ1
t=0
there exists a (u, Ω)-overtaking program {zt }∞

t=0 such that
π u (z0 ) = sup(π u ),
ρ(zt , yt ) ≤ , t = 0, . . . , τ0 .
By Propositions 5.4 and 5.5 and (B2), there exists δ2 ∈ (0, δ1 ) such that the
following properties hold:
(ii) for each z ∈ K satisfying ρ(z, x̄) ≤ 2δ2 , we have
|π u (z)| = |π u (z) − π u (x̄)| ≤ δ1 /8;

(iii) for each (x, y) ∈ Ω satisfying
ρ(x, x̄) ≤ 2δ2 , ρ(y, x̄) ≤ 2δ2
we have
|u(x, y) − u(x̄, x̄)| ≤ (δ1 /8)(L∗ + 1)−1 .
By (B3), there exists δ3 ∈ (0, δ2 ) such that the following property holds:
(iv) for each x, y ∈ K satisfying
ρ(x, x̄) ≤ δ3 , ρ(y, x̄) ≤ δ3
there exists an (Ω)-program {ξt }L∗

t=0 such that
ξ0 = x, ξL∗ = y
and
ρ(ξt , x̄) ≤ δ2 , t = 0, . . . , L∗ .
By Theorem 3.12, there exist an integer L0 ≥ 1 and a number δ4 > 0 such

(v) for each integer T > 2L0 , each finite sequence of bounded functions ut : Ω →
R 1 , t = 0, . . . T − 1 satisfying
ut − u ≤ δ4 , t = 0 . . . , T − 1
and each (Ω)-program {zt }Tt=0 which satisfies
−1
T
{ut }T −1 , Ω, zT − δ4
ut (zt , zt+1 ) ≥ U t=0
t=0
we have
ρ(zt , x̄) ≤ δ3 , t = L0 , . . . , T − L0 .
By Theorem 3.4, there exists a (u, Ω)-overtaking optimal program {zt }∞

t=0 such
that
π u (z0 ) = sup(π u ). (5.41)
(B1) implies that there exists a natural number τ1 such that
ρ(zt , x̄) ≤ δ3 for all integers t ≥ τ1 . (5.42)

Choose a positive number δ and an integer T0 such that
δ < (16(L0 + L∗ + τ1 + τ0 + 6))−1 min{δ1 , δ2 , δ3 , δ4 }, (5.43)
T0 > 2L∗ + 2L0 + 2τ0 + 2τ1 + 4. (5.44)
Assume that an integer T ≥ T0 , a finite sequence of bounded functions ut : Ω →

R 1 , t = 0, . . . T − 1 satisfies
ut − u ≤ δ, t = 0 . . . , T − 1 (5.45)
and {xt }Tt=0 is an (Ω)-program which satisfies
−1
T
T −1
ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, xT − δ. (5.46)
t=0
By (5.43)–(5.46) and the property (v),
ρ(xt , x̄) ≤ δ3 , t = L0 , . . . , T − L0 . (5.47)
Property (iv), (5.42), (5.44), and (5.47) imply that there exist an (Ω)-program
{x̃t }Tt=0 such that
x̃t = zt , t = 0, . . . , L0 + τ0 + τ1 + 3, (5.48)
ρ(x̃t , x̄) ≤ δ2 , t = L0 + τ0 + τ1 + 4, . . . , L0 + L∗ + τ0 + τ1 + 3, (5.49)
x̃t = xt , t = L0 + L∗ + τ0 + τ1 + 3, . . . , T . (5.50)
−1
T
δ≥ (ut (x̃t , x̃t+1 ) − ut (xt , xt+1 ))
t=0
L0 +L∗
+τ0 +τ1 +2
= (ut (x̃t , x̃t+1 ) − ut (xt , xt+1 ))
t=0
L0 +L∗
+τ0 +τ1 +2
≥ (u(x̃t , x̃t+1 ) − u(xt , xt+1 ))
t=0
− 2δ(L0 + L∗ + τ0 + τ1 + 3). (5.51)

Properties (ii) and (iii), (5.42), (5.48), and (5.49) imply that
|u(x̃t , x̃t+1 ) − u(x̄, x̄)| ≤ (δ1 /8)(L∗ + 1)−1 , (5.52)
t = L0 + τ0 + τ1 + 3, . . . , L0 + L∗ + τ0 + τ1 + 2.
Corollary 5.3, property (ii), (5.42), (5.43), (5.44), (5.47), (5.48), (5.51), and (5.52)
imply that
L0 +L∗
+τ0 +τ1 +2
δ≥ u(zt , zt+1 ) + L∗ u(x̄, x̄) − L∗ (δ1 /8)(L∗ + 1)−1
t=0
L0 +L∗
+τ0 +τ1 +2
− u(xt , xt+1 ) − 2δ(L0 + L∗ + τ0 + τ1 + 3)
t=0
= π u (z0 ) − π u (zL0 +τ0 +τ1 +3 )
+ (L0 + L∗ + τ0 + τ1 + 3)u(x̄, x̄) − δ1 /4

⎛ ⎞
L0 +L
∗ +τ1 +2
−⎝ (u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xL0 +L∗ +τ1 +3 )⎠
t=0
− (L0 + L∗ + τ1 + 3)u(x̄, x̄) − π u (x0 ) + π u (xL0 +L∗ +τ1 +3 )
≥ sup(π u ) − δ1 /8 − δ1 /4 − π u (x0 ) − δ1 /8
⎛ ⎞
L0 +L
∗ +τ1 +2
−⎝ (u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xL0 +L∗ +τ1 +3 )⎠ .
t=0
(5.53)
It follows from Proposition 5.1, (5.43), and (5.53) that
0 −1
τ
π u (x0 ) + (u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xτ0 )
t=0
≥ sup(π u ) − δ1 ,
π u (x0 ) ≥ sup(π u ) − δ1 , (5.54)
0 −1
τ
(u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xτ0 ) ≥ −δ1 . (5.55)
t=0
5.5 The First Class of Bolza Problems 89
By (5.54), (5.55), and the property (i), there exists a (u, Ω)-overtaking optimal
program {yt }∞
t=0 such that
π u (y0 ) = sup(π u ),
ρ(yt , xt ) ≤ , t = 0, . . . , τ0 .
5.5 The First Class of Bolza Problems
For each nonempty set Y and each function h : Y → R 1 put
sup(h) = sup{h(y) : y ∈ Y }.
Denote by M(K) the set of all bounded functions h : K → R 1 . For each h ∈

M(K) set
h = sup{|h(x)| : x ∈ K}.
Clearly, (M(K), · ) is a Banach space. For each h1 , h2 ∈ M(K) set
dK (h1 , h2 ) = h1 − h2 .
For each x ∈ K, each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 , each finite

sequence of bounded functions ut : Ω → R 1 , t = T1 , . . . , T2 − 1, and each h ∈
M(K) we consider the problem
2 −1
T
T −1
ut (xt , xt+1 ) + h(xT2 ) → max, {(xt , xt+1 )}t=0 ⊂ Ω, xT1 = x (PB,1 )
t=T1
and set
⎧
⎨T
2 −1
T2 −1
U h, {ut }t=T , Ω, x = sup ut (xt , xt+1 ) + h(xT2 ) :
1 ⎩
t=T1
⎫
⎬
{xt }Tt=T
2
is an (Ω) − program and xT1 = x .
1 ⎭
For each x ∈ X, each pair of integers T1 , T2 satisfying 0 ≤ T1 < T2 , each bounded

function u : Ω → R 1 , and each h ∈ M(K) set
T2 −1
U (h, u, T1 , T2 , Ω, x) = U (h, {ut }t=T 1
, Ω, x) where ut = u, t = T1 , . . . , T2 − 1.
In Sect. 5.7 we prove the following result which describes the structure of
approximate solutions of the problems of the type (PB,1 ) in the regions close to
the right endpoints.
Theorem 5.15 Let g ∈ M(K) be upper semicontinuous function, τ0 ≥ 1 be an
integer, and > 0. Then there exist δ > 0 and an integer T0 ≥ τ0 such that for each
integer T ≥ T0 , each h ∈ M(K) satisfying
h − g ≤ δ,
each finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . , T −1 satisfying
ut − u ≤ δ, t = 0 . . . , T − 1
τ +T
0 −1
τ +T0 −1
ut (xt , xt+1 ) ≥ U {ut }t=τ , Ω, xτ , xτ +T0 − δ
t=τ
for each τ ∈ {0, . . . , T − T0 },
−1
T
T −1
h(xT ) + ut (xt , xt+1 ) ≥ U h, {ut }t=T −T0 , Ω, xT −T0 − δ
t=T −T0
there exists a (ū, Ω̄)-overtaking optimal program {xt∗ }∞

t=0 such that
(π ū + g)(x0∗ ) = sup(π ū + g),
ρ(xT −t , xt∗ ) ≤ , t = 0, . . . , τ0 .
Let g ∈ M(K) be upper semicontinuous. Then π ū + g : K → R 1 is an upper

semicontinuous, bounded function. Therefore there exists x ∈ K such that
(π ū + g)(x) = sup(π ū + g).
Lemma 5.16 Let g ∈ M(K) be an upper semicontinuous function, T0 ≥ 1 be an

integer, and ∈ (0, 1). Then there exists δ ∈ (0, ) such that for each (Ω)-program
{xt }Tt=0
0
which satisfies
(π u + g)(x0 ) ≥ sup(π u + g) − δ,
0 −1
T
(u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xT0 ) ≥ −δ
t=0
there exists a (u, Ω)-overtaking optimal program {zt }∞

t=0 such that
(π u + g)(z0 ) = sup(π u + g), (5.56)
ρ(zt , xt ) ≤ , t = 0, . . . , T0 .
Proof Assume that the lemma does not hold. Then there exist a sequence of real
numbers {δk }∞ (k) T0
k=1 ⊂ (0, 1] and a sequence of (Ω)-programs {xt }t=0 , k = 1, 2, . . .
such that
lim δk = 0 (5.57)
k→∞
and that for each natural number k and each (u, Ω)-overtaking optimal program
{zt }∞
t=0 satisfying (5.56),

(k)
(π u + g) x0 ≥ sup(π u + g) − δk , (5.58)
0 −1
T
u xt(k) , xt+1
(k)
− u(x̄, x̄) − π u x0(k) + π u xT(k)
0
≥ −δk , (5.59)
t=0
we have

max ρ(zt , xt(k) ) : t = 0, . . . , T0 > . (5.60)

of generality that for each integer t ∈ [0, T0 ] there exists
(k)
xt = lim xt . (5.61)
k→∞
By (5.61) and upper semicontinuity of g and π u ,
(k)
π u (x0 ) ≥ lim sup π u (x0 ),
k→∞
(k)
g(x0 ) ≥ lim sup g(x0 ). (5.62)
k→∞

sup(π u + g) ≥ (π u + g)(x0 )
≥ lim sup(π u + g)(x0(k) ) ≥ sup(π u + g),

k→∞
(k)
sup(π u + g) = (π u + g)(x0 ) = lim (π u + g)(x0 ). (5.63)
k→∞
Relations (5.62) and (5.63) imply that

π u (x0 ) = lim π u (x0(k) ),
k→∞
(k)
g(x0 ) = lim g(x0 ). (5.64)
k→∞
It follows from upper semicontinuity of u and π u , (5.57), (5.59), (5.61), and (5.64)
that
0 −1
T
t=0
⎛ ⎞
0 −1
T
≥ lim sup ⎝ (u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xT0 )⎠
(k) (k) (k) (k)
k→∞ t=0
k→∞
Combined with Proposition 5.1 this implies that for all integers t = 0, . . . , T0 − 1,
It follows from Theorem 3.4 that there exists a (u, Ω)-overtaking optimal program
{x̃t }∞
t=0 satisfying
x̃0 = xT0 . (5.66)
For all integers t > T0 set
xt = x̃t−T0 . (5.67)
Evidently, {xt }∞
t=0 is an (Ω)-program. It follows from Corollary 5.3 that (5.65) holds
for all integers t ≥ 0. Proposition 5.8 and (5.65) imply that {xt }∞ t=0 is a (u, Ω)-
overtaking optimal program satisfying (5.63). By (5.61), for all sufficiently large
(k)
natural numbers k, ρ(xt , xt ) ≤ /4, t = 0, . . . , T0 . This contradicts (5.60). The
contradiction we have reached proves Lemma 5.16.
By Lemma 5.16 applied to the pair (ū, Ω̄) there exists a real number
δ1 ∈ (0, /2)

(i) for each (Ω̄)-program {yt }τt=0
0
which satisfies
(π ū + g)(y0 ) ≥ sup(π ū + g) − δ1 ,
0 −1
τ
(ū(yt , yt+1 ) − ū(x̄, x̄)) − π ū (y0 ) + π ū (yτ0 ) ≥ −δ1
t=0
there exists an (ū, Ω̄)-overtaking optimal program {zt }∞

t=0 such that
(π ū + g)(z0 ) = sup(π ū + g),
ρ(zt , yt ) ≤ , t = 0, . . . , τ0 .
Propositions 5.4 and 5.5 applied to the pair (ū, Ω̄) and assumption (B2) imply
that there exists a real number δ2 ∈ (0, δ1 ) such that the following properties
hold:
(ii) for each point z ∈ K satisfying ρ(z, x̄) ≤ 2δ2 ,
|π ū (z)| = |π ū (z) − π ū (x̄)| ≤ δ1 /16;
(iii) for each (x, y) ∈ Ω satisfying
ρ(x, x̄) ≤ 2δ2 , ρ(y, x̄) ≤ 2δ2 ,
|u(x, y) − u(x̄, x̄)| ≤ (δ1 /8)(L∗ + 1)−1 .
(iv) for each x, y ∈ K satisfying
ρ(x, x̄) ≤ δ3 , ρ(y, x̄) ≤ δ3

t=0 such that
ξ0 = x, ξL∗ = y
and
ρ(ξt , x̄) ≤ δ2 , t = 0, . . . , L∗ .
By Theorem 3.12, there exist an integer L1 ≥ 1 and a number δ4 ∈ (0, δ3 )

(v) for each integer T > 2L1 , each finite sequence of bounded functions ut : Ω →
R 1 , t = 0, . . . T − 1 satisfying
ut − u ≤ δ4 , t = 0 . . . , T − 1
and each (Ω)-program {zt }Tt=0 which satisfies
τ +L
1 −1
τ +L1 −1
ut (zt , zt+1 ) ≥ U {ut }t=τ , Ω, zτ , zτ +L1 − δ4
t=τ
for each integer τ ∈ [0, T − L1 ], we have
ρ(zt , x̄) ≤ δ3 , t = L1 , . . . , T − L1 .
By Theorem 3.4, there exists a (ū, Ω̄)-overtaking optimal program {zt }∞

t=0 such
that
(π ū + g)(z0 ) = sup(π ū + g). (5.68)
(B1) implies that there exists a natural number τ1 such that
ρ(zt , x̄) ≤ δ3 for all integers t ≥ τ1 . (5.69)
Choose a positive number δ and an integer T0 such that
δ < (16(L1 + L∗ + τ1 + τ0 + 8))−1 min{δ1 , δ2 , δ3 , δ4 }, (5.70)
T0 > 2L∗ + 2L1 + 2τ0 + 2τ1 + 8. (5.71)
Assume that an integer T ≥ T0 , h ∈ M(K) satisfies
h − g ≤ δ, (5.72)
a finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfies
ut − u ≤ δ, t = 0 . . . , T − 1 (5.73)
and {xt }Tt=0 is an (Ω)-program which satisfies
τ +T
0 −1
τ +T0 −1
ut (xt , xt+1 ) ≥ U {ut }t=τ , Ω, xτ , xτ +T0 − δ (5.74)
t=τ
for each integer τ ∈ [0, T − T0 ],

−1
T
T −1
h(xT ) + ut (xt , xt+1 ) ≥ U h, {ut }t=T −T0 , Ω, xT −T0 − δ. (5.75)
t=T −T0
By (5.70), (5.71), (5.73), (5.74), and the property (v),
ρ(xt , x̄) ≤ δ3 , t = L1 , . . . , T − L1 . (5.76)
Property (iv), (5.69), (5.71), and (5.76) imply that there exists an (Ω)-program
{x̃t }Tt=0 such that
x̃t = xt , t = 0, . . . , T − L1 − L∗ − τ0 − τ1 − 4, (5.77)
x̃t = zT −t , t = T − L1 − τ0 − τ1 − 4, . . . , T , (5.78)
ρ(x̃t , x̄) ≤ δ2 , t = T − L1 − L∗ − τ0 − τ1 − 4, . . . , T − L1 − τ0 − τ1 − 4.
(5.79)
By property (iii), (5.71)–(5.73), (5.75), and (5.77),

−1
T
T −1
δ ≥ U h, {ut }t=T −T0 , Ω, xT −T0 − h(xT ) − ut (xt , xt+1 )
t=T −T0
−1
T −1
T
≥ h(x̃T ) + ut (x̃t , x̃t+1 ) − h(xT ) − ut (xt , xt+1 )
t=T −T0 t=T −T0
−1
T
= h(x̃T ) − h(xT ) + (ut (x̃t , x̃t+1 ) − ut (xt , xt+1 ))
t=T −L1 −L∗ −τ0 −τ1 −4
−1
T
≥ g(x̃T ) − g(xT ) + u(x̃t , x̃t+1 )
t=T −L1 −L∗ −τ0 −τ1 −4
−1
T
− u(xt , xt+1 ) − 2δ(L1 + L∗ + τ0 + τ1 + 5)
t=T −L1 −L∗ −τ0 −τ1 −4
≥ g(x̃T ) − g(xT )
−1
T
+ u(x̃t , x̃t+1 ) + L∗ u(x̄, x̄) − L∗ (δ1 /8)(L∗ + 1)−1
t=T −L1 −τ0 −τ1 −4
−1
T
− u(xt , xt+1 ) − 2δ(L1 + L∗ + τ0 + τ1 + 5).
t=T −L1 −L∗ −τ0 −τ1 −4
Combined with (5.22), (5.70), and (5.78) this implies that
−1
T
g(xT ) + u(xt , xt+1 )
t=T −L1 −L∗ −τ0 −τ1 −4
−1
T
≥ g(x̃T ) + u(x̃t , x̃t+1 ) + L∗ u(x̄, x̄) − δ1 /8 − δ1 /8
t=T −L1 −τ0 −τ1 −4
L1 +τ
0 +τ1 +2
= g(z0 ) + ū(zt , zt+1 ) + u(x̃T −L1 −τ0 −τ1 −4 , x̃T −L1 −τ0 −τ1 −3 )
t=0
+ L∗ u(x̄, x̄) − δ1 /4. (5.80)
Property (iii), (5.69), and (5.78) imply that
|u(x̃T −L1 −τ0 −τ1 −4 , x̃T −L1 −τ0 −τ1 −3 ) − u(x̄, x̄)| ≤ δ1 /8. (5.81)
In view of (5.80) and (5.81),
−1
T
g(xT ) + u(xt , xt+1 )
t=T −L1 −L∗ −τ0 −τ1 −4
L1 +τ
0 +τ1 +2
≥ g(z0 ) + ū(zt , zt+1 ) + (L∗ + 1)u(x̄, x̄) − 3δ1 /8. (5.82)
t=0
Set
yt = xT −t , t = 0, . . . , T . (5.83)
By (5.22), (5.82), and (5.83),
L1 +L∗
+τ0 +τ1 +3
g(y0 ) + ū(yt , yt+1 )
t=0
−1
T
= g(xT ) + u(xt , xt+1 )
T −L1 −L∗ −τ0 −τ1 −4
L1 +τ
0 +τ1 +2
≥ g(z0 ) + ū(zt , zt+1 ) + (L∗ + 1)u(x̄, x̄) − 3δ1 /8. (5.84)
t=0
By (5.68), (5.84), Proposition 5.1, and Corollary 5.3,
(π ū + g)(y0 ) − sup(π ū + g)
0 −1
τ
+ (ū(yt , yt+1 ) − ū(x̄, x̄)) − π ū (y0 ) + π ū (yτ0 )
t=0
≥ (π ū + g)(y0 ) − (π ū + g)(z0 )
L1 +L∗
+τ0 +τ1 +3
+ (ū(yt , yt+1 ) − u(x̄, x̄)) − π ū (y0 ) + π ū (yL1 +L∗ +τ0 +τ1 +4 )
t=0
L1 +τ
0 +τ1 +2
≥ π ū (y0 ) − π ū (z0 ) + (ū(zt , zt+1 ) − ū(x̄, x̄))
t=0
− 3δ1 /8 − π ū (y0 ) + π ū (yL1 +L∗ +τ0 +τ1 +4 )

= −π ū (zL1 +τ0 +τ1 +3 ) − π ū (yL1 +L∗ +τ0 +τ1 +4 ) − 3δ1 /8. (5.85)
Property (ii), (5.69), (5.71), (5.76), and (5.83) imply that
|π ū (zL1 +τ0 +τ1 +3 )| ≤ δ1 /8, (5.86)
|π ū (yL1 +L∗ +τ0 +τ1 +4 )| ≤ δ1 /8. (5.87)
It follows from (5.85)–(5.87) that
(π ū + g)(y0 ) − sup(π ū + g)
0 −1
τ
+ (ū(yt , yt+1 ) − ū(x̄, x̄))
t=0
− π ū (y0 ) + π ū (yτ0 ) ≥ −δ1 .
Together with Proposition 5.1 this implies that
(π ū + g)(y0 ) − sup(π ū + g) ≥ −δ1 ,
0 −1
τ
(ū(yt , yt+1 ) − ū(x̄, x̄))
t=0
−π ū (y0 ) + π ū (yτ0 ) ≥ −δ1 .

It follows from the relations above, (5.83) and the property (i) that there exists an
(ū, Ω̄)-overtaking optimal program {ξt }∞
t=0 such that
(π ū + g)(ξ0 ) = sup(π ū + g),
ρ(ξt , xT −t ) = ρ(ξt , yt ) ≤ , t = 0, . . . , τ0 . Theorem 5.15 is proved.
5.8 The Second Class of Bolza Problems
Denote by M(K × K) the collection of all bounded functions h : K × K → R 1 .

For every function h ∈ M(K × K) put
h = sup{|h(x, y)| : x, y ∈ K}.
It is not difficult to see that (M(K × K), · ) is a Banach space.

For every pair of nonnegative integers T1 < T2 , every finite sequence of bounded
functions ut : Ω → R 1 , t = T1 , . . . , T2 − 1, and every function h ∈ M(K × K)
we consider the problem
2 −1
T
T2 −1
ut (xt , xt+1 ) + h(xT1 , xT2 ) → max, {(xt , xt+1 )}t=T 1
⊂Ω
t=T1
and define
⎧
⎨T
2 −1
T2 −1
U h, {ut }t=T1 , Ω = sup ut (xt , xt+1 ) + h(xT1 , xT2 ) :
⎩
t=T1
⎫
⎬
{xt }Tt=T
2
is an (Ω) − program .
1 ⎭
For every pair of nonnegative integers T1 < T2 , every bounded function v : Ω →

R 1 , and every function h ∈ M(K × K) define
T2 −1
U (h, v, Ω, T1 , T2 ) = U (h, {vt }t=T 1
, Ω) where vt = v, t = T1 , . . . , T2 − 1.
Let g ∈ M(K × K) be an upper semicontinuous function. Then the function
ψg (ξ, η) = π u (ξ ) + π ū (η) + g(ξ, η), (ξ, η) ∈ Ω (5.88)
is upper semicontinuous bounded function which has a point of minimum.

5.9 Auxiliary Results for Theorem 5.17 99
Our next result describes the structure of approximate solutions in the regions
close to the endpoints.
Theorem 5.17 Suppose that g ∈ M(K × K) is an upper semicontinuous function,
τ0 is a natural number, and ∈ (0, 1). Then there exist a positive number δ and a
natural number T0 ≥ τ0 such that for every natural number T ≥ T0 , every function
h ∈ M(K × K) satisfying h − g ≤ δ, every finite sequence of bounded functions
ut : Ω → R 1 , t = 0, . . . , T − 1 which satisfies
ut − u ≤ δ, t = 0 . . . , T − 1,
and every (Ω)-program {xt }Tt=0 which satisfies
−1
T
T −1
h(x0 , xT ) + ut (xt , xt+1 ) ≥ U h, {ut }t=0 , Ω − δ,
t=0
there exist a (u, Ω)-overtaking optimal program {xt∗ }∞

t=0 and a (ū, Ω̄)-overtaking
optimal program {x̄t∗ }∞
t=0 such that
π u (x0∗ ) + π ū (x̄0∗ ) + g(x0∗ , x̄0∗ )
≥ π u (ξ ) + π ū (η) + g(ξ, η) for all ξ, η ∈ K
and that for all integers t = 0, . . . , τ0 , ρ(xt , xt∗ ) ≤ and ρ(xT −t , x̄t∗ ) ≤ .
5.9 Auxiliary Results for Theorem 5.17
Lemma 5.18 Suppose that g ∈ M(K × K) is an upper semicontinuous function,

T0 is a natural number, and ∈ (0, 1). Then there exists δ ∈ (0, ) such that for
every (Ω)-program {xt }Tt=0
0
and every (Ω̄)-program {yt }Tt=0
0
which satisfy
ψg (x0 , y0 ) + δ ≥ sup(ψg ),
0 −1
T
(u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xT0 ) ≥ −δ,
t=0
0 −1
T
(ū(yt , yt+1 ) − u(x̄, x̄)) − π ū (y0 ) + π ū (yT0 ) ≥ −δ,
t=0
there exist a (u, Ω)-overtaking optimal program {xt∗ }∞

t=0 and an (ū, Ω̄)-overtaking
optimal program {yt∗ }∞
t=0 such that
ψg (x0∗ , y0∗ ) = sup(ψg )
and that for all t = 0, . . . , T0 ,
ρ(xt , xt∗ ) ≤ , ρ(yt , yt∗ ) ≤ .
Proof Assume that the lemma does not hold. Then there exist a sequence {δk }∞ k=1 ⊂
(k) T0
(0, 1], a sequence of (Ω)-programs {xt }t=0 , k = 1, 2, . . . and a sequence of (Ω̄)-
programs {yt }Tt=0
(k) 0
, k = 1, 2, . . . such that
lim δk = 0, (5.89)
k→∞
for every integer k ≥ 1,

ψg x0(k) , y0(k) + δk ≥ sup(ψg ), (5.90)
0 −1
T
(k) (k) (k) (k)
u(xt , xt+1 ) − u(x̄, x̄) − π u x0 + π u xT0 ≥ −δk , (5.91)
t=0
0 −1
T
(k) (k) (k) (k)
ū(yt , yt+1 ) − u(x̄, x̄) − π ū y0 + π ū yT0 ≥ −δk (5.92)
t=0
and that the following property holds:

(i) for each (u, Ω)-overtaking optimal program {zt }∞
t=0 and each (ū, Ω̄)-overtaking
optimal program {ξt }∞
t=0 satisfying
ψg (z0 , ξ0 ) = sup(ψg ),
we have

(k) (k)
max ρ zt , xt + ρ ξt , yt : t = 0, . . . , T0 > .
Extracting a subsequence and re-indexing, if necessary, we may assume without

loss of generality that for every integer t ∈ [0, T0 ] there exist
xt = lim xt(k) , yt = lim yt(k) . (5.93)

k→∞ k→∞
5.9 Auxiliary Results for Theorem 5.17 101
It follows from (5.88)–(5.90) and (5.93) and upper semicontinuity of the

functions g, π u , and π ū that
π u (x0 ) + π ū (y0 ) + g(x0 , y0 ) ≥ π u (ξ ) + π ū (η) + g(ξ, η) for all ξ, η ∈ K,

(5.94)
(k)
π u (x0 ) ≥ lim sup π u x0 , (5.95)
k→∞

π ū (y0 ) ≥ lim sup π ū y0(k) , (5.96)
k→∞

(k) (k)
g(x0 , y0 ) ≥ lim sup g x0 , y0 . (5.97)
k→∞
π u (x0 ) + π ū (y0 ) + g(x0 , y0 )

(k) (k) (k) (k)
= lim π u x0 + π ū y0 + g x0 , y0 . (5.98)
k→∞
By (5.95)–(5.98),

(k)
π u (x0 ) = lim π u x0 , (5.99)
k→∞

(k)
π ū (y0 ) = lim π ū y0 , (5.100)
k→∞

(k) (k)
g(x0 , y0 ) = lim g x0 , y0 . (5.101)
k→∞
By upper semicontinuity of the functions u, π u , and π ū , (5.80), (5.91)–(5.93),

(5.99), and (5.100), we have
0 −1
T
t=0
⎛ ⎞
0 −1
T
≥ lim sup ⎝ u(xt(k) , xt+1
(k)
− u(x̄, x̄)) − π u x0(k) + πu xT(k)
0
⎠
k→∞ t=0
≥ lim sup(−δk ) = 0,
k→∞
0 −1
T
(ū(yt , yt+1 ) − u(x̄, x̄)) − π ū (y0 ) + π ū (yT0 )
t=0
⎛ ⎞
0 −1
T
≥ lim sup ⎝ − u(x̄, x̄) − π ū y0 + π ū yT0 ⎠
(k) (k) (k) (k)
u yt , yt+1
k→∞ t=0
k→∞
Together with Proposition 5.1 this implies that for all integers t = 0, . . . , T0 − 1,
we have
u(xt , xt+1 ) − u(x̄, x̄) = π u (xt ) − π u (xt+1 ), (5.102)
ū(yt , yt+1 ) − u(x̄, x̄) = π ū (yt ) − π ū (yt+1 ). (5.103)
It follows from Theorem 3.4 that there exist a (u, Ω)-overtaking optimal
program {x̃t }∞
t=0 satisfying
x̃0 = xT0 (5.104)
and a (ū, Ω̄)-overtaking optimal program {ỹt }∞

t=0 satisfying
ỹ0 = yT0 . (5.105)
For all integers t > T0 put
xt = x̃t−T0 , yt = ỹt−T0 . (5.106)
Clearly, {xt }∞ ∞
t=0 is an (Ω)-program and {yt }t=0 is an (Ω̄)-program. Corollary 5.3
implies that (5.102) and (5.103) hold for all integers t ≥ 0. It follows from
(5.102), (5.103), and Proposition 5.8 that {xt }∞t=0 is a (u, Ω)-overtaking optimal
∞
program and {yt }t=0 is a (ū, Ω̄)-overtaking optimal program satisfying (5.98).
In view of (5.93), for all sufficiently large natural numbers k,
(k) (k)
ρ(xt , xt ) ≤ /4, ρ(yt , yt ) ≤ /4, t = 0, . . . , T0 .
Together with (5.98) this contradicts property (i). The contradiction we have
reached proves Lemma 5.18.
Lemma 5.18 implies that there exists a number δ1 ∈ (0, /2) such that the following
property holds:
(i) for every (Ω)-program {ξt }τt=0

0
and every (Ω̄)-program {ηt }τt=0
0
which satisfy
π u (ξ0 )+π ū (η0 )+g(ξ0 , η0 )+2δ1 ≥ π u (ξ )+π ū (η)+g(ξ, η) for all ξ, η ∈ K,
0 −1
τ
(u(ξt , ξt+1 ) − u(x̄, x̄)) − π u (ξ0 ) + π u (ξτ0 ) ≥ −2δ1 ,
t=0
0 −1
τ
(ū(ηt , ηt+1 ) − u(x̄, x̄)) − π ū (η0 ) + π ū (ητ0 ) ≥ −2δ1
t=0
there exists a (u, Ω)-overtaking optimal program {xt∗ }∞

t=0 and a (ū, Ω̄)-
overtaking optimal program {yt∗ }∞
t=0 such that
ψg (x0∗ , y0∗ ) = sup(ψg )
and that for all t = 0, . . . , τ0 ,
ρ(ξt , xt∗ ) ≤ , ρ(ηt , yt∗ ) ≤ .
Propositions 5.4, 5.5, and assumption (B2) imply that there exists a number
δ2 ∈ (0, δ1 ) such that the following properties hold:
(ii) for every point z ∈ X satisfying ρ(z, x̄) ≤ 2δ2 , we have
|π u (z)| = |π u (z) − π u (x̄)| ≤ δ1 /16,
|π ū (z)| = |π ū (z) − π ū (x̄)| ≤ δ1 /16;
(iii) for every (x, y) ∈ Ω satisfying
ρ(x, x̄) ≤ 2δ2 , ρ(y, x̄) ≤ 2δ2 ,
we have
|u(x, y) − u(x̄, x̄)| ≤ (δ1 /16)(L∗ + 1)−1
(iv) for each z1 , z2 ∈ K satisfying
ρ(zi , x̄) ≤ δ3 , i = 1, 2

t=0 such that
ξ0 = z1 , ξL∗ = z2
and
ρ(ξt , x̄) ≤ δ2 , t = 0, . . . , L∗ .
Theorem 3.12 implies that there exist a natural number L1 and a positive
number δ4 ∈ (0, δ3 ) such that the following property holds:
(v) for every natural number T > 2L1 , every finite sequence of bounded functions
ut : Ω → R 1 , t = 0, . . . T − 1 satisfying
ut − u ≤ δ3 , t = 0 . . . , T − 1
and every (Ω)-program {xt }Tt=0 which satisfies
−1
T
T −1
ut (xt , xt+1 ) ≥ U {ut }t=0 , Ω, x0 , xT − δ4
t=0
the inequality ρ(xt , x̄) ≤ δ3 holds for all t = L1 , . . . , T − L1 .

By Theorem 3.4, there exists a (u, Ω)-overtaking optimal program {ξt }∞
t=0 and a
∞
(ū, Ω̄)-overtaking optimal program {ηt }t=0 such that
ψg (ξ0 , η0 ) = sup(ψg ). (5.107)
In view of property (v),
ρ(ξt , x̄) ≤ δ3 for all integers t ≥ L1 , (5.108)
ρ(ηt , x̄) ≤ δ3 for all integers t ≥ L1 . (5.109)
Choose a positive number δ and an integer T0 ≥ 1 such that
δ < (64(2L1 + L∗ + τ0 + 8))−1 min{δ1 , δ2 , δ3 , δ4 }, (5.110)
T0 > 4L∗ + 8L1 + 4τ0 + 16. (5.111)
Assume that an integer T ≥ T0 , a function h ∈ M(K × K) satisfies
h − g ≤ δ, (5.112)
a finite sequence of bounded functions ut : Ω → R 1 , t = 0, . . . T − 1 satisfies
ut − u ≤ δ, t = 0 . . . , T − 1 (5.113)
and that {xt }Tt=0 is an (Ω)-program which satisfies
−1
T
T −1
h(x0 , xT ) + ut (xt , xt+1 ) ≥ U h, {ut }t=0 , Ω − δ. (5.114)
t=0
In view of (5.110), (5.111), (5.113), (5.114), and property (v),
ρ(xt , x̄) ≤ δ3 , t = L1 , . . . , T − L1 . (5.115)
Property (iv), (5.108), (5.109), (5.111), and (5.115) imply that there exists an (Ω)-
program {x̃t }Tt=0 such that
x̃t = ξt , t = 0, . . . , 2L1 + τ0 + 3, (5.116)
ρ(x̃t , x̄) ≤ δ2 , t = 2L1 + τ0 + 3, . . . , 2L1 + L∗ + τ0 + 3, (5.117)
x̃t = xt , t = 2L1 + τ0 + 3 + L∗ , . . . , T − 2L1 − τ0 − 3 − L∗ , (5.118)
ρ(x̃t , x̄) ≤ δ2 , t = T − 2L1 − τ0 − 3 − L∗ , . . . , T − 2L1 − τ0 − 3, (5.119)
x̃t = ηT −t , t = T − 2L1 − τ0 − 3, . . . , T . (5.120)
It follows from (5.110), (5.112)–(5.114), (5.116)–(5.120), and property (iii) that
−1
T
T −1
δ ≥ U (h, {ut }t=0 , Ω) − h(x0 , xT ) − ut (xt , xt+1 )
t=0
−1
T −1
T
≥ h(x̃0 , x̃T ) + ut (x̃t , x̃t+1 ) − h(x0 , xT ) − ut (xt , xt+1 )
t=0 t=0
2L1 +τ
0 +2+L∗
= h(ξ0 , η0 ) − h(x0 , xT ) + (ut (x̃t , x̃t+1 ) − ut (xt , xt+1 ))
t=0
−1
T
+ (ut (x̃t , x̃t+1 ) − ut (xt , xt+1 ))
t=T −2L1 −τ0 −3−L∗
≥ g(ξ0 , η0 ) − g(x0 , xT )
2L1 +τ
0 +2+L∗
+ (u(x̃t , x̃t+1 ) − u(xt , xt+1 ))
t=0
−1
T
+ (u(x̃t , x̃t+1 ) − u(xt , xt+1 )) − 2δ(1 + 2L1 + τ0 + 4 + L∗ )
t=T −2L1 −τ0 −3−L∗
≥ g(ξ0 , η0 ) − g(x0 , xT )
+τ0 +2
2L1
+ u(ξt , ξt+1 ) + L∗ (u(x̄, x̄) − (δ1 /16)(L∗ + 1)−1 )
t=0
2L1 +τ
0 +2+L∗ −1
T
− ut (xt , xt+1 ) + u(ηT −t , ηT −t−1 )
t=0 t=T −2L1 −τ0 −3
+ L∗ (u(x̄, x̄) − (δ1 /16)(L∗ + 1)−1 )

−1
T
− ut (xt , xt+1 ) − δ1 /16
t=T −2L1 −τ0 −3−L∗
≥ g(ξ0 , η0 ) − g(x0 , xT )
+τ0 +2
2L1 +τ0 +2
2L1
+ u(ξt , ξt+1 ) + ū(ηt , ηt+1 ) + 2L∗ u(x̄, x̄)
t=0 t=0
2L1 +τ
0 +2+L∗ −1
T
− u(xt , xt+1 ) − u(xt , xt+1 ) − 3δ1 /16.
t=0 t=T −2L1 −τ0 −3−L∗
(5.121)
Define
yt = xT −t , t = 0, . . . , T . (5.122)
2L1 +τ
0 +L∗ +2 2L1 +τ
0 +L∗ +2
g(x0 , y0 ) + u(xt , xt+1 ) + ū(yt , yt+1 )
t=0 t=0
+τ0 +2
2L1 +τ0 +2
2L1
≥ g(ξ0 , η0 ) + u(ξt , ξt+1 ) + ū(ηt , ηt+1 )
t=0 t=0
+ 2L∗ u(x̄, x̄) − 3δ1 /16 − δ1 /16. (5.123)
It follows from (5.108), (5.109), (5.111), (5.112), (5.115), (5.123), Proposition 5.1,
Corollary 5.3, and properties (ii) and (iii) that
g(x0 , y0 ) − g(ξ0 , η0 )
0 −1
τ
+ (u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xτ0 )
t=0
0 −1
τ
+ (ū(yt , yt+1 ) − u(x̄, x̄)) − π ū (y0 ) + π ū (yτ0 )
t=0
≥ g(x0 , y0 ) − g(ξ0 , η0 )
2L1 +τ
0 +2+L∗
+ (u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (x2L1 +τ0 +L∗ +3 )
t=0
2L1 +τ
0 +L∗ +2
+ (ū(yt , yt+1 ) − ū(x̄, x̄)) − π ū (y0 ) + π ū (y2L1 +τ0 +3+L∗ )
t=0
+τ0 +2
2L1
≥ (u(ξt , ξt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (x2L1 +τ0 +3+L∗ )
t=0
+τ0 +2
2L1
+ (ū(ηt , ηt+1 ) − ū(x̄, x̄)) − π ū (y0 ) + π ū (y2L1 +τ0 +3+L∗ ) − δ1 /4
t=0
≥ π u (ξ0 ) − π u (ξ2L1 +τ0 +3 ) − π u (x0 ) + π u (x2L1 +τ0 +3+L∗ )

+ π ū (η0 ) − π ū (η2L1 +τ0 +3 )
− π ū (y0 ) − π ū (y2L1 +τ0 +3+L∗ ) − δ1 /4
≥ π u (ξ0 ) − π u (x0 ) + π ū (η0 ) − π ū (y0 ) − δ1 /4 − δ1 /4. (5.124)
In view of (5.124), we have
g(x0 , y0 ) + π u (x0 ) + π ū (y0 ) − (g(ξ0 , η0 ) + π u (ξ0 ) + π ū (η0 ))

0 −1
τ
+ (u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xτ0 )
t=0
0 −1
τ
+ (ū(yt , yt+1 ) − u(x̄, x̄)) − π ū (y0 ) + π ū (yτ0 ) ≥ −δ1 . (5.125)
t=0
By (5.107), (5.125), and Proposition 5.1,
g(x0 , y0 ) + π u (x0 ) + π ū (y0 ) ≥ sup(ψg ) − δ1 ,
0 −1
τ
(u(xt , xt+1 ) − u(x̄, x̄)) − π u (x0 ) + π u (xτ0 ) ≥ −δ1 ,
t=0
0 −1
τ
(ū(yt , yt+1 ) − u(x̄, x̄)) − π ū (y0 ) + π ū (yτ0 ) ≥ −δ1 .
t=0
It follows from the relations above and property (i) that there exists a (u, Ω)-
overtaking optimal program {xt∗ }∞
t=0 and a (ū, Ω̄)-overtaking optimal program
{yt∗ }∞
t=0 such that
ψg (x0∗ , y0∗ ) = sup(ψg )
and for all t = 0, . . . , τ0 , ρ(xt , xt∗ ) ≤ , ρ(xT −t , yt∗ ) = ρ(yt , yt∗ ) ≤ .

Chapter 6
Applications to the Forest Management
Problem
In this chapter we continue the discussion of the forest management problem and
show that for this problem the results of Chaps. 3 and 5 hold.
6.1 Preliminaries
We consider a discrete time model for the optimal management of a forest of total
area S occupied by k species I = {1, . . . , k} with maturity ages of n1 , . . . , nk years
respectively. This model was introduced in Sect. 2.1. For the reader’s convenience
we recall here the notation and definitions.
j
For each period t = 0, 1, . . . we denote xi (t) ≥ 0 the area covered by trees of
species i that are j years old with j = 1, . . . , ni and x̄i (t) ≥ 0 the area occupied by
over-mature trees (older than ni ).
Assuming that only mature trees can be harvested we must have
ui (t) ≤ x̄i (t) + xini (t), (6.1)
and then the area not harvested in that period will comprise the over-mature trees at
the next step, namely
x̄i (t + 1) = x̄i (t) + xini (t) − ui (t). (6.2)
The fact that immature trees cannot be harvested is represented by
j +1 j
xi (t + 1) = xi (t), j = 1, . . . , ni − 1. (6.3)
110 6 Applications to the Forest Management Problem

The total harvested area i∈I ui (t) is allocated to new seedlings which is
expressed by the equation

xi1 (t + 1) = ui (t). (6.4)
i∈I i∈I
In the sequel we use the notation
xini +1 = x̄i , i ∈ I. (6.5)
A representation of the forest in terms of the age distribution at time t is provided

by the state
x(t) = (x1 (t), . . . , xk (t)),
where xi (t) = (xi1 (t), . . . , xini (t), xini +1 (t)) describes the areas occupied in year t
by trees of species i with ages 1, 2, . . . , ni and over ni . The first and last components
of each vector xi (t) are controlled by the sowing and harvesting policies.
Let R+m = {x = (x , . . . , x ) ∈ R m : x ≥ 0, i = 1, . . . , m}.
1 m i
Let N = i∈I (ni +1). Every vector x ∈ R N is represented as x = (x1 , . . . , xk ),
where xi = (xi1 , . . . , xini , xini +1 ) ∈ R ni +1 for all integers i = 1, . . . , k.
Denote by Δ the set of all x ∈ R+ N such that
⎡ ⎤
i +1
n
⎣ xi ⎦ = S.
j
(6.6)
i∈I j =1
A sequence {x(t)}∞ t=0 ⊂ Δ is called a program if for all integers t ≥ 0 and all
i ∈ I (6.1)–(6.4) hold (see (6.5)) with some u(t) = (u1 (t), . . . , uk (t)) ∈ R+
k.
T2
Let the integers T1 , T2 satisfy 0 ≤ T1 < T2 . A sequence {x(t)}t=T1 ⊂ Δ is called
a program if (6.1)–(6.4) hold for all i ∈ I and for all integers t = T1 , . . . , T2 − 1
(see (6.5)) with some u(t) = (u1 (t), . . . , uk (t)) ∈ R+
k.
An alternative equivalent definition of a program is given with the help of the

transition possibility. Define

j +1 j
Ω = (x, y) ∈ Δ × Δ : yi = xi for all i ∈ I and all j ∈ {1, . . . , ni } \ {ni }

and for all i ∈ I, xini +1 + xini − yini +1 ≥ 0 . (6.7)
Evidently, if (x, y) ∈ Ω, then

n(i)+1
yi1 = xi + xini − yin(i)+1 . (6.8)
i∈I i∈I
It is not difficult to see that a sequence {x(t)}∞

t=0 ⊂ Δ is a program if and only if
(x(t), x(t + 1)) ∈ Ω for all integers t ≥ 0.
Let the integers T1 , T2 satisfy 0 ≤ T1 < T2 . It is not difficult to see that a
sequence {x(t)}Tt=T 2
1
⊂ Δ is a program if and only if (x(t), x(t + 1)) ∈ Ω for all
t = T1 , . . . , T2 − 1.
For each (x, y) ∈ Ω set
V (x, y) = (v1 (x, y), . . . , vk (x, y)),
where for i = 1, . . . , k,
vi (x, y) = xini +1 + xini − yini +1 .
Set

k
Δ0 = x ∈ k
R+ : xi ≤ S .
i=1
In this chapter we assume that a benefit at moment t = 0, 1, . . . is represented

by an upper semicontinuous function wt : Δ0 → R 1 and at a moment t = 0, 1, . . . ,
tomorrow is y, where (x, y) ∈ Ω.
We suppose that wt = w for all integers t ≥ 0.
Evidently, Δ is a compact set in R N , Ω is a closed subset of Δ × Δ and w ◦ V :
Ω → R 1 is an upper semicontinuous function. Put
n̄ = max{ni : i ∈ I }. (6.9)
This model is a particular case of the general optimal control system studied in
Chaps. 2, 3, and 5. Proposition 2.1 implies that assumption (A) holds with
L̄ = N + n̄ + 1.
All the results of Chap. 2 hold for the particular case considered here with u = w◦V .
We use the notation and definitions introduced and used in Chaps. 2 and 3.
Let y, z ∈ Δ and let the integers T1 , T2 satisfy T1 < T2 . Set
⎧
⎨T
2 −1
U (y, T1 , T2 ) = sup w(V (xt , xtt+1 )) :
⎩
t=T1
⎫
⎬
{xt }Tt=T
2
is a program and xT1 = y ,
1 ⎭
⎧
⎨T
2 −1
U (y, ỹ, T1 , T2 ) = sup w(V (xt , xtt+1 )) :
⎩
t=T1
⎫
⎬
{xt }Tt=T
2
is a program and xT1 = y, xT2 = ỹ ,
1 ⎭
⎧ ⎫
⎨T
2 −1 ⎬
(T1 , T2 ) = sup
U w(V (xt , xtt+1 )) : {xt }Tt=T
2
is a program .
⎩ 1 ⎭
t=T1
Let
μ := μ(w ◦ V )
be defined by Theorem 2.9.

For any x = (x1 , . . . , xm ) ∈ R m set
x = max{|xi | : i = 1, . . . , m}.
Lemma 6.1 Assume that
w(x) ≤ w(y) (6.10)
for each
x = (x1 , . . . , xk ), y = (y1 , . . . , yk ) ∈ Δ0
satisfying
xi ≤ yi , i = 1, . . . , k
and that
μ(w ◦ V ) = sup{(w ◦ V )(x, x) : x ∈ Δ, (x.x) ∈ Ω}. (6.11)
Then there exists
x1 , . . . ,
x = ( xk ) ∈ Δ (6.12)
such that

xi = xini +1 , i = 1, . . . , k,
xi1 . . . ,
x,
( x ) ∈ Ω, (6.13)
x,
w(V ( x )) ≥ w(V (x, x)) for all x ∈ Δ such that (x, x) ∈ Ω, (6.14)
and that for each i = 1, . . . , k,
xini +1 = 0.
j
xi =
xi1 , j = 1, . . . , ni ,
Proof In view of (6.11), there exists

x = (
x1 , . . . ,
xk ) ∈ Δ satisfying (6.12)–(6.14).
By (6.12), (6.13), and the definition of Ω, for each i = 1, . . . , k,
j

xi =
xi1 , j = 1, . . . , ni .
If
xini +1 = 0, i = 1, . . . , k,

then the assertion of the lemma holds.

Assume that

k
d := xini +1 > 0.
(6.15)
i=1
Then
x,
V ( x ) = (
x11 , . . . ,
xk1 )
by definition. Define y = (y1 , . . . , yk ), for i = 1, . . . , k by

⎛ ⎞−1

k
+d⎝ np ⎠ , j = 1, . . . , ni , yini +1 = 0.
j j
yi =
xi (6.16)
p=1
It is not difficult to see that

y ∈ Δ, (y, y) ∈ Ω, V (y, y) = y11 , . . . , yk1 . (6.17)
w(V (y, y)) ≥ w(V (

x,
x )) ≥ w(V (x, x))
for all x ∈ Δ satisfying (x, x) ∈ Ω. Lemma 6.1 is proved.

Let

x = (
x1 , . . . ,
xk ) ∈ Δ, (
x,
x ) ∈ Ω, (6.18)
xi = (
xini +1 ), i = 1, . . . , k,
xi1 . . . ,
for i = 1, . . . , k,
xini +1 = 0,
j

xi =
xi1 , j = 1, . . . , ni , (6.19)
I˜ = {i ∈ I : xi1 > 0}. (6.20)
Proposition 6.2 Let
xi1 : i ∈ I˜}).
∈ (0, min{ (6.21)
Then for each x, y ∈ Δ satisfying
x − x̄, y − x̄ ≤ (2N)−1 ,

+n̄+1
there is a program {x(t)}N
t=0 such that
x(0) = x, x(N + n̄ + 1) = y,
x(t) −
x ≤ , t = 0, . . . , N + n̄ + 1. (6.22)
Proof We may assume without loss of generality that

I˜ = {1, . . . , k0 },
where k0 ∈ [1, k] is an integer.

Let
x, y ∈ Δ, x − x ≤ (2N )−1 .
x , y − (6.23)
+n̄+1
We construct a program {x(t)}N t=0 which satisfies (6.22). Set x(0) = x.
Assume that t ∈ {0, . . . , N + n̄} is an integer and that a program {x(p)}tp=0 was
defined. In order to define x(t + 1) we need to choose v(t) ∈ R+ k and
(x11 (t + 1), . . . , xk1 (t + 1)) ∈ R+

k
such that for all integers i = 1, . . . , k we have

k
k
vi (t) ≤ xini (t) + xini +1 (t), xi1 (t + 1) = vi (t) (6.24)
i=1 i=1
and then the state x(t + 1) is defined as follows for all integers i = 1, . . . , k:
j j −1
xi (t + 1) = xi (t), j is an integer satisfying 2 ≤ j ≤ ni , (6.25)
xini +1 (t + 1) = xini (t) + xini +1 (t) − vi (t). (6.26)
Thus the construction of the program is done as follows: if the states are defined till
moment t ≤ N + n̄, then we choose V (t) ∈ R+ k and (x 1 (t +1), . . . , x 1 (t +1)) ∈ R k
1 k +
such that (6.24) holds and define x(t + 1) by (6.25) and (6.26).
For each t = 0, . . . , N − 1 set
xi1 − (2N)−1 , i = 1, . . . , k0 ,
xi1 (t + 1) = vi (t) =
xi1 (t + 1) = vi (t) = 0 for all integers i satisfying k0 < i ≤ k (6.27)
j
and define xi (t + 1), i = 1, . . . , k, j = 2, . . . , ni + 1 by (6.25) and (6.26).
In order to show that {x(t)}N t=0 is a program it is sufficient to show that the first
part of (6.24) is true for all integers i = 1, . . . , k0 .
It follows from (6.23), (6.27), and (6.25) that for all integers t = 0, . . . , N, all
integers i = 1, . . . , k0 , and all integers j = 1, . . . , ni we have
xi1 − (2N)−1 ≤ xi (t) ≤

xi1 + (2N )−1 .
j
(6.28)
Equations (6.27) and (6.28) imply (6.24) for all integers i = 1, . . . , k0 . Then
{x(t)}N
t=0 is a program. In view of (6.23), (6.27), and (6.28), for all integers
t = 0, . . . , N ,

k
k0
0≤ xini +1 ≤ S − xi1 + N(2N )−1 ≤ 2−1 .
ni (6.29)
i=1 i=1
By (6.19), (6.28), and (6.29), we have
x ≤ 2−1 , t = 0, . . . , N.
x(t) − (6.30)
Let us consider the state x(N). In view of (6.25) and (6.27),
xi1 − (2N)−1 , j = 1, . . . , n(i),

j
xi (N ) =
j
i = 1, . . . , k0 , xi (N ) = 0, j = 1, . . . , ni , (6.31)
where an integer i satisfies k0 < i ≤ k.

We continue to construct the program.
If at the moment t the state x(t) has already been defined, then we define V (t) ∈
k and (x 1 (t + 1), . . . , x 1 (t + 1)) ∈ R k such that (6.24) holds and then define
R+ 1 k +
x(t + 1) by (6.25) and (6.26).
For every integer s = 1, . . . , n̄ set
Is = {i ∈ {1, . . . , k} : ni = s}. (6.32)
(Note that for some integers s we can have Is = ∅.)

For all integers i = 1, . . . , k and all integers t = N, . . . , N + n̄ − 1 we define
(1)
xi (t + 1) ≥ 0, show that there exists a vector V (t) ∈ R+ k which satisfies (6.24)
j
and then define xi (t + 1), i = 1, . . . , k where an integer j satisfies 2 ≤ j ≤ ni + 1.
We begin with the definition of xi1 (t + 1), i = 1, . . . , k, t = N, . . . , N + n̄ − 1.
Let i ∈ {1, . . . , k}. If i ∈ I1 (ni = 1), then we set

xi1 (t + N ) = max xi1 − (2N)−1 , 0 for all integers t satisfying 1 ≤ t < n̄,
(6.33)
xi (n̄ + N) = yi + yi .
1 1 2
If i ∈ Is (ni = s) with s > 1, then we set

xi1 − (2N)−1 , 0 for all integers t
xi1 (N + t) = max
satisfying 1 ≤ t < n̄ + 1 − ni , (6.34)
xi1 (n̄ + 1 − s + N) = yini +1 + yi1 , (6.35)
xi1 (N + t) = yin̄+2−t for all integers t satisfying n̄ + 1 − s < t ≤ n̄. (6.36)
j
We have defined xi1 (t + N), i ∈ I, t = 1, . . . , n̄. Then we define xi (t + 1) by
(6.25) for all t = N, . . . , N + n̄ − 1, all integers j satisfying 2 ≤ j ≤ ni , and all
j
i ∈ I . Thus we have defined xi (t + N), t = 1, . . . , n̄, i ∈ I, j = 1, . . . , n(i).
Set

ni
j

φ(t) = xi (N + t) − xini (N + t)
i∈I j =1 i∈I

+ xi1 (N + t + 1), t = 0, . . . , n̄ − 1. (6.37)
i∈I
In view of (6.37), (6.35), and (6.33), for all integers t = 0, . . . , n̄ − 1, we have

ni
j
φ(t) = xi (N + t + 1). (6.38)
i∈I j =1
By (6.33)–(6.38), (6.31), and (6.23),
φ(t) ≤ φ(t + 1) for all integers t ∈ [0, n̄ − 1]. (6.39)
It follows from (6.38) and (6.33)–(6.36) that

ni i +1
n
j j
φ(n̄ − 1) = xi (N + n̄) = yi = S.
i∈I j =1 i∈I j =1
Hence
φ(t) ≤ S, t = 0, . . . , n̄ − 1. (6.40)
+n̄
In order to complete the construction of the program {x(t)}N t=0 we need to
n +1
determine V (t) ∈ R+ k t = N, . . . , N + n̄ − 1 and x i (t + 1) ≥ 0, t =
i
N, . . . , N + n̄ − 1] such that the second part of (6.24) is valid for all integers
t = N, . . . , N + n̄ − 1 and
x(t) ∈ Δ, t = N + 1, . . . , N + n̄. (6.41)
We complete this construction by induction. Let t = 0. In view of (6.37) and (6.40),

xi1 (N + 1) ≤ xini (N ) + xini +1 (N ) . (6.42)
i∈I i∈I
Relation (6.42) implies that there exists V (N) ∈ R+ k such that

xi1 (N + 1) = vi (N ), vi (N ) ≤ xini (N ) + xini +1 (N ), i ∈ I. (6.43)
i∈I i∈I
Define xini +1 (N + 1), i ∈ I by (6.20) with t = N.

Assume that an integer τ satisfies 1 ≤ τ < n̄, we have defined V (t) ∈ R+ k,
t = N, . . . , N + τ − 1, xini +1 (t), i ∈ I , t = N + 1, . . . , N + τ , and x(t) ∈ Δ,

t = N + 1 . . . , N + τ such that (6.24) is valid for all integers t = N, . . . , N + τ − 1.
(Note that for τ = 1 this assumption holds.)
By (6.37) and (6.40),
n +1
xi1 (N + τ + 1) ≤ xi i (N + τ ) + xini (N + τ ) .
i∈I i∈I
This implies that there is V (N + τ ) ∈ R+k such that

xi1 (N + τ + 1) = Vi (N + τ ),
i∈I i∈I
Vi (N + τ ) ≤ xini +1 (N + τ ) + xini (N + τ ), i ∈ I.
Define xini +1 (N + τ + 1) by (6.26) with t = N + τ . Clearly, the assumption made

for τ also holds for τ + 1. Thus by induction we defined xini +1 (t + 1) ≥ 0, t =
N, . . . , N + n̄ − 1, V (t) ∈ R+ k , t = N, . . . , N + n̄ − 1 such that (6.31) holds and
(6.24) holds for all t = N, . . . , N + n̄ − 1.

+n̄
Thus we have constructed a program {x(t)}N t=0 . It follows from (6.33)–(6.36)
that for all integers i ∈ I ,
xini (N + n̄) = yini +1 + yi1 ,
j j +1
xi (N + n̄) = yi for all integers j satisfying 1 ≤ j < n(i),
xi1 (N + n̄) = yi2 .
Evidently,

ni i +1
n
j j
xi = yi = S.
i∈I j =1 i∈I j =1
+n̄+1
Set x(N + n̄ + 1) = y. It is clear that (x(N + n̄), y) ∈ Ω. Hence {x(t)}N
t=0 is a
program such that x(0) = x, x(N + n̄ + 1) = y. In view of (6.33)–(6.36),
x(t) − x̄ ≤ , t = 0, . . . , N + n̄ + 1.
This completes the proof of Proposition 6.2.
6.3 Turnpike Results
In the sequel the notation x ≥ y, x > y, x >> y where x, y ∈ R q have their

usual meaning. We assume that w : Δ0 → R 1 is a continuous and a strictly concave
function such that
w(x) ≤ w(y) for all x, y ∈ Δ0 satisfying x ≤ y. (6.44)
Note that w(V (x, y)) is the benefit obtained today if the forest today is x and the
forest tomorrow is y where (x, y) ∈ Ω.
In the sequel we assume that supremum over empty set is −∞.
Theorem 6.3 There exists M > 0 and the limit
(0, p)/p
μ := μ(w ◦ V ) := lim U (6.45)
p→∞
6.3 Turnpike Results 119
such that
(0, p) − μ| ≤ M/p for all natural numbers p.

|p−1 U
Clearly Ω is a convex set and w ◦V is a concave function. Let μ be as guaranteed

by Theorem 6.3. Then Theorem 2.10 implies the following result.
Theorem 6.4
μ = sup{w(V (x, x)) : x ∈ Δ and (x, x) ∈ Ω}.
Lemma 6.1 implies that there exists

x = (
x1 , . . . ,
xk ) ∈ Δ
such that
x,
( x ) ∈ Ω,
for i = 1, . . . , k,

xi =
xini +1 ,
xi1 . . . ,
xini +1 = 0,
j

xi =
xi1 , j = 1, . . . , ni ,
μ = w(V (
x,
x )) ≥ w(V (x, x)) for all x ∈ Δ satisfying (x, x) ∈ Ω.
Proposition 6.2 implies that (B3) holds. It is clear that (B2) holds too.
Theorem 6.5 Assume that a program {x(t)}∞
t=0 is good. Then
lim x(t) =
x.
t→∞
Proof First we show that
lim V (x(t), x(t + 1)) = V (

x,
x ). (6.46)
t→∞
For all nonnegative integers t set
z(t) = 2−1 (x(t) +

x ). (6.47)
Evidently, {z(t)}∞
t=0 is a program and
V (z(t), z(t + 1)) = 2−1 V (x(t), x(t + 1)) + 2−1 V (

x,
x ). (6.48)
It follows from (6.48) and the concavity of the function w that {z(t)}∞
t=0 is a good
program and the sequence
T −1 ∞

w(V (z(t), z(t + 1))) − T μ is bounded. (6.49)
t=0 T =1
Assume that (6.46) does not hold. There exist a positive number and a strictly
increasing sequence of natural numbers {tj }∞
j =1 such that
V (
x,
x ) − V (x(tj ), x(tj + 1)) ≥ . (6.50)
Since the function w is continuous and strictly concave there exists a positive
number δ such that for every pair of points y1 , y2 ∈ Δ0 which satisfies y1 − y2 ≥
, we have

w 2−1 (y1 + y2 ) − 2−1 w(y1 ) − 2−1 w(y2 ) ≥ δ. (6.51)
Since the program {xt }∞

t=0 is good there exists a positive number M0 such that for
every integer T > 0, we have
T −1

(w(V (x(t), x(t + 1))) − μ) ≤ M0 . (6.52)

t=0
In view of (6.48), the strict concavity of the function w, (6.50) and the choice of the
number δ (see (6.51)), (6.52), for every integer k > 0,
k +1
t
w(V (z(t), z(t + 1))) − tk μ − 2μ
t=0
k +1
t
= w(2−1 V (x(t), x(t + 1)) + 2−1 V (
x,
x )) − tk μ − 2μ
t=0
k +1
t
= [w(2−1 V (x(t), x(t + 1))
t=0
−1
+2 V ( x )) − 2−1 w(V (x(t), x(t + 1))) − 2−1 w(V (
x, x,
x ))]
k +1
t
− tk μ − 2μ + 2−1 w(V (x(t), x(t + 1))) + 2−1 (tk + 2)w(V (
x,
x ))
t=0
k +1
t
≥ δk + 2−1 [w(V (x(t), x(t + 1))) − μ]
t=0
≥ δk − M0 → ∞ as k → ∞,
a contradiction. The contradiction we have reached proves that (6.46) is valid.

6.3 Turnpike Results 121
Set
= V (
V x, x11 ,
x ) = ( x21 , . . . ,
xk1 ) (6.53)
and for all integers t ≥ 0 set
V (t) = V (x(t), x(t + 1)), (6.54)
where
V (t) = (v1 (t), . . . , vk (t)). (6.55)
Let t ≥ 0 be an integer. In view of (6.2), (6.5), (6.26), and (6.54), for all integers
i = 1, . . . , k,
xini (t) = xini +1 (t + 1) − xini +1 (t) + vi (t),
xini +1 (t + 1) = xini +1 (t) + xini (t) − vi (t), i = 1, . . . , k (6.56)
and
j +1 j
xi (t + 1) = xi (t), j = 1, . . . , ni − 1 (6.57)
for all integers i = 1, . . . , k if ni ≥ 2. It follows from (6.56) and (6.57) that for all
integers T ≥ 1 and for all integers i = 1, . . . , k,
T +n
i −1 T +n
i −1
vi (t) = xini +1 (t) + xini (t) − xini +1 (t + 1)
t=T t=T
T +n
i −1
= xini +1 (T ) − xini +1 (T + ni ) + xini (t)
t=T

ni
= xini +1 (T ) − xini +1 (T + ni ) +
j
xi (T ). (6.58)
j =1
By (6.58)
⎡ ⎤ ⎡ ⎤
T +n i −1 n +1 ni
⎣ vi (t)⎦ =
n 1
xi i (T ) − xi + (T + ni ) + ⎣ xi (T )⎦
j
i∈I t=T i∈I i∈I j =1

⎡ ⎤
i +1
n
⎣ xi (T )⎦ − xini +1 (T + ni )
j
=
i∈I j =1 i∈I

=S− xini +1 (T + ni ). (6.59)
i∈I
By (6.46), (6.53)–(6.55), and (6.59), we have

⎡ ⎤
T +n
i −1
ni
xi1 = lim ⎣ vi (t)⎦ = S − lim xini +1 (T + ni ). (6.60)
T →∞ T →∞
i∈I i∈I t=T i∈I
By (6.53), (6.60), the choice of

x , we have
lim x ni +1 (T + ni ) = 0, i = 1, . . . , k. (6.61)
T →∞ i
It follows from (6.56) and (6.61) that for all integers i = 1, . . . , k,
lim (xini (t) − vi (t)) = 0. (6.62)

t→∞
In view of (6.46), (6.59), and (6.62), for all integers i = 1, . . . , k,
lim x ni (t) = lim vi (t) = Vi (

x,
x) =
xi1 . (6.63)
t→∞ i t→∞
By (6.63), for all integers i = 1, . . . , k and every integer j which satisfies 1 ≤ j ≤

ni ,
j
lim x (t) = lim xini (t + ni − j ) =
xi1 .
t→∞ i t→∞
Combined with (6.61) this implies that
lim x(t) =
x.
t→∞

Thus (B1) holds for w ◦ V and all the results of Chaps. 3 and 5 are true in our
particular case.
6.4 Generic Results
Denote by C(Δ0 ) the set of all continuous functions on the set

k
Δ0 = {v ∈ R+
k
: vi ≤ S},
i=1
where S > 0. For each f ∈ C(Δ0 ) set
f = sup{|f (x)| : x ∈ Δ0 }.
We continue to study the forest management problem which is a particular case of

the optimal control problem considered in Chaps. 3 and 5 with u = w ◦ V , where
w ∈ C(Δ0 ). Recall that for each (x, y) ∈ Ω,
V (x, y) = (v1 (x, y), . . . , vk (x, y)), (6.64)
where for i = 1, . . . , k,
vi (x, y) = xini +1 + xini − yini +1 . (6.65)
Denote by Cm (Δ0 ) the set of all w ∈ C(Δ0 ) such that w : Δ0 → [0, ∞),
w(x) ≤ w(y) for all x, y ∈ Δ0 satisfying x ≤ y,
and that
μ(w ◦ V ) = sup{w(V (x, x)) : x ∈ Δ and (x, x) ∈ Ω}. (6.66)
It is not difficult to see that Cm (Δ0 ) is a closed subset of C(Δ0 ). The space Cm (Δ0 )
is equipped with the metric dC (f1 , f2 ) = f1 − f2 , f1 , f2 ∈ Cm (Δ0 ).
Let w ∈ Cm (Δ0 ). Lemma 6.1 implies that there exists

x = (
x1 , . . . ,
xk ) ∈ Δ
such that
x,
( x ) ∈ Ω, (6.67)
x,
w(V ( x )) ≥ w(V (x, x)) for all x ∈ Δ satisfying (x, x) ∈ Ω, (6.68)
for i = 1, . . . , k,
xi = (
xini +1 ),
xi1 . . . ,
xini +1 = 0.
j
xi =
xi1 , j = 1, . . . , ni , (6.69)
Assume that
μ(w ◦ V ) = w(V (
x,
x )) > 0. (6.70)
Let γ ∈ (0, k −1 ) and set
I0 = {i ∈ {1, . . . , k} :
xi1 > 0}. (6.71)
For each i ∈ I0 define a function φi : [0, ∞) → [0, ∞) by
φi (x) = w(V (
x,
x )) for all x ∈ [
xi1 , ∞), (6.72)
xi1 )−1 w(V (

φi (x) = x( x,
x )) for all x ∈ [0,
xi1 ]. (6.73)
For every x = (x1 , . . . , xk ) ∈ Δ0 define

wγ (x) = w(x) + γ φi (xi ). (6.74)
i∈I0
Clearly, w is a continuous increasing function.

Lemma 6.6 wγ ∈ Cm (Δ0 ) and for each (wγ ◦ V )-good program {x(t)}∞
t=0 ,
lim x(t) =
x.
t→∞
Proof Theorem 2.9 and (6.70) imply that there exists M0 > 0 such that for each
(Ω)-program {x(t)}∞
t=0 and each integer T ≥ 1,
−1
T
w(V (x(t), x(t + 1))) ≤ T w(V (
x,
x )) + M0 . (6.75)
t=0
Let {x(t)}∞
t=0 be a program. By (6.72)–(6.75), for each integer T > 0,
−1
T
wγ (V (x(t), x(t + 1)))
t=0
−1
T −1
T
= w(V (x(t), x(t + 1))) + γ φi (vi (x(t), x(t + 1)))
t=0 t=0 i∈I0

≤ T w(V ( x )) + M0 + γ T
x, (w ◦ V )(
x,
x)
i∈I0
= M0 + T w(V (
x,
x ))(1 + γ Card(I0 )) = M0 + (wγ ◦ V )(
x,
x )T .
This implies that
μ(wγ ◦ V ) = (wγ ◦ V )(

x,
x ), (6.76)
and that wγ ∈ Cm (Δ0 ).

Let {x(t)}∞
t=0 be a (wγ ◦ V )-good program. We show that
lim x(t) =
x.
t→∞
In view of (6.76), there exists M1 > 0 such that for each integer T ≥ 1,
T −1

wγ (V (x(t), x(t + 1))) − T wγ (V ( x )) ≤ M1 .
x, (6.77)

t=0
It follows from (6.72)–(6.75) and (6.77) that for all integers T ≥ 1,
− M1 + T w(V (
x,
x ))(1 + γ Card(I0 ))
= −M1 + T (wγ ◦ V )(
x,
x)
−1
T
≤ wγ (V (x(t), x(t + 1)))
t=0
−1
T −1
T
= w(V (x(t), x(t + 1))) + γ φi (vi (x(t), x(t + 1)))
t=0 t=0 i∈I0
−1
T
≤ T (w ◦ V )( x ) + M0 + γ
x, φi (vi (x(t), x(t + 1))). (6.78)
t=0 i∈I0
By (6.78), for each integer T > 0,

−1
T
M0 + M1 ≥ γ T Card(I0 )(w ◦ V )(
x,
x) − γ φi (vi (x(t), x(t + 1)))
t=0 i∈I0
−1
T
=γ ((w ◦ V )(
x,
x ) − φi (vi (x(t), x(t + 1)))). (6.79)
t=0 i∈I0
Let i ∈ I0 . It follows from (6.72), (6.73), and (6.79) that for every integer T > 0,
−1
T
γ −1 (M0 + M1 ) ≥ ((w ◦ V )(
x,
x ) − φi (vi (x(t), x(t + 1)))). (6.80)
t=0
Let > 0. In view of (6.80),
Card({t ∈ {0, 1, . . . } : (w ◦ V )( x ) − φi (vi (x(t), x(t + 1))) ≥ }

x,
≤ (γ )−1 (M0 + M1 )
and there exists an integer t > 0 such that for all integers t ≥ t ,
(w ◦ V )(
x,
x ) − φi (vi (x(t), x(t + 1))) < .
Since an arbitrary positive number it follows from (6.72) and (6.73) that
lim inf vi ((x(t), x(t + 1)) ≥

xi1 , i ∈ I0 . (6.81)
t→∞
For every nonnegative integer t ≥ 0, set
V (t) = (v1 (t), . . . , vk (t)) = V (x(t), x(t + 1)). (6.82)
Let t ≥ 0 be an integer. By (6.65) and (6.82) for all i = 1, . . . , k,
xini (t) = xini +1 (t + 1) − xini +1 (t) + vi (t), (6.83)
xini +1 (t + 1) = xini +1 (t) + xini (t) − vi (t), i = 1, . . . , k (6.84)
j +1 j
xi (t + 1) = xi (t), j = 1, . . . , ni − 1 (6.85)
for i = 1, . . . , k if ni ≥ 2. By (6.84) and (6.85) and all integers T ≥ 1 for all

i = 1, . . . , k,
T +n
i −1 T +n
i −1
vi (t) = xini +1 (t) + xini (t) − xini +1 (t + 1)
t=T t=T
T +n
i −1
= xini +1 (T ) − xini +1 (T + ni ) + xini (t)
t=T

ni
= xini +1 (T ) − xini +1 (T + ni ) +
j
xi (T ). (6.86)
j =1
By (6.86), for every integer T ≥ 1,

⎡ ⎤ ⎡ ⎤

k T +n
i −1 k
k
ni
⎣ +1 +1
vi (t)⎦ = ⎣ xi (T )⎦
n n j
xi i (T ) − xi i (T + ni ) +
i=1 t=T i=1 i=1 j =1
⎡ ⎤

k i +1
n
k
⎣ xi (T )⎦ − xini +1 (T + ni )
j
=
i=1 j =1 i=1

k
=S− xini +1 (T + ni ). (6.87)
i=1
In view of (6.67), (6.69), (6.71), (6.81), (6.82), and (6.87),

k
S= ni
xi1
i=1
⎡ ⎤

k T +n
i −1
≤ ⎣ lim inf vi (x(t), x(t + 1))⎦
T →∞
i=1 t=T
⎡ ⎤

k T +n
i −1
≤ lim inf ⎣ vi (t)⎦
T →∞
i=1 t=T

k
= lim inf S − xini +1 (T + ni )
T →∞
i=1

k
= S − lim sup xini +1 (T + ni ). (6.88)
T →∞ i=1
It follows from (6.88) that
lim x ni +1 (T + ni ) = 0, i = 1, . . . , k. (6.89)
T →∞ i
By (6.83) and (6.89) for all i = 1, . . . , k
lim (xini (t) − vi (t)) = 0. (6.90)

t→∞
By (6.87) and (6.89),

k T +n
i −1
lim vi (t) = S. (6.91)
T →∞
i=1 t=T
In view of (6.71) and (6.81), for every i = 1, . . . , k,
lim inf vi (t) ≥

xi1 , (6.92)
t→∞
By (6.88) and (6.90)–(6.92),
lim vi (t) =
xi1 , i = 1, . . . , k,
t→∞
lim x ni (t) =
xi1 , i = 1, . . . , k,
t→∞ i
j
lim x =
xi1 , i = 1, . . . , k, j = 1, . . . , ni .
t→∞ i

Theorem 6.7 There exists a set F ⊂ Cm (Δ0 ) which is a countable intersection of

open everywhere dense subsets of Cm (Δ0 ) such that for each w ∈ F there exists
x w = (x1w , . . . , xkw ) ∈ Δ
such that
(x w , x w ) ∈ Ω,
for i = 1, . . . , k,
xiw = (xiw,1 . . . , xiw,ni +1 ),
= xiw,1 , j = 1, . . . , ni , xiw,ni +1 = 0
w,j
xi
and that for each (w ◦ V )-good program {x(t)}∞

t=0 ,
lim x(t) = x w .
t→∞
Proof Let w ∈ Cm (Δ0 ) be such that
μ(w ◦ V ) > 0,
γ ∈ (0, k −1 ) and p ≥ 1 be an integer. By Lemmas 6.1 and 6.6 and Theorem 3.12,
there exist an open neighborhood U (w, γ , p) of wγ in Cm (Δ0 ), δ(w, γ , p) > 0, a
natural number τ (w, γ , p) and
x w = (x1w , . . . , xkw ) ∈ Δ
such that the following properties hold:

(a)
(x w , x w ) ∈ Ω,
for i = 1, . . . , k,
xiw = (xiw,1 . . . , xiw,ni +1 ),
= xiw,1 , j = 1, . . . , ni , xiw,ni +1 = 0,
w,j
xi
μ(w ◦ V ) = w(V (x w , x w ));
(b) for each (wγ ◦ V )-good program {x(t)}∞

t=0 ,
lim x(t) = x w ;
t→∞
(c) for each integer T > 2τ (w, γ , p), each ut ∈ U (w, γ , p), t = 0, . . . , T − 1 and
each program {x(t)}Tt=0 which satisfies
−1
T
T −1
(ut ◦ V )(xt , xt+1 ) ≥ U {ut ◦ V }t=0 , x0 , xT − δ(w, γ , p)
t=0
we have
x(t) − x w ≤ 1/n, t = τ (w, γ , p), . . . , T − τ (w, γ , p).
Define
F = ∩∞
q=1 ∪ {U (w, γ , p) :
w ∈ Cm (Δ0 ), μ(w ◦ V ) > 0, γ ∈ (0, 1/k), p ≥ q is an integer}. (6.93)
Clearly, F is a countable intersection of open everywhere dense sets in Cm (Δ0 ).

Let
u ∈ F, ∈ (0, 1). (6.94)
Choose a natural number q such that
q > 4 −1 . (6.95)
It follows from (6.93) and (6.94) that there exist w ∈ Cm (Δ0 ), γ ∈ (0, k −1 ), and an
integer p ≥ q such that
μ(w ◦ V ) > 0,
u ∈ U (w, γ , p). (6.96)
Assume that
ξ1 , ξ2 ∈ Δ,
(ξi , ξi ) ∈ Ω, i = 1, 2,
(u ◦ V )(ξi , ξi ) = μ(u ◦ V ), i = 1, 2.
It is not difficult to see that for each i = 1, 2 and each integer T > 0,
U (u ◦ V , ξi , ξi , 0, T ) = T (u ◦ V )(ξi , ξi ).
Combined with property (c) and (6.96) this implies that
x u − ξi ≤ p−1 ≤ q −1 < /4, i = 1, 2, (6.97)
ξ1 − ξ2 < .
Since is an arbitrary positive number we conclude that ξ1 = ξ2 and
{ξ ∈ Δ : (ξ, ξ ) ∈ Ω, u(V (ξ, ξ )) = μ(u ◦ V )}
is a singleton.
Denote its unique element by x u . In view of (6.97),
x u − x w ≤ p−1 < . (6.98)
Since is an arbitrary positive number it follows from property (a) that
x u = (x1u , . . . , xku ) ∈ Δ,
(x u , x u ) ∈ Ω,
for i = 1, . . . , k,
xiu = (xiu,1 . . . , xiu,ni +1 ),
= xiu,1 , j = 1, . . . , ni , xiu,ni +1 = 0.
u,j
xi
Assume that an integer T > 2τ (w, γ , p) and that a program {xt }Tt=0 satisfies
−1
T
(u ◦ V )(xt , xt+1 ) ≥ U (u ◦ V , x0 , xT , 0, T ) − δ(w, γ , p).
t=0
Together with property (c), (6.96), and (6.98) this implies that for all integers t =
τ (w, γ , p), . . . , T − τ (w, γ , p),
x(t) − x w ≤ 1/p
and
x(t) − x u ≤ 2/p < .
Combined with Theorem 4.1 this implies that every (u◦V )-good program converges
to x u . Theorem 6.7 is proved.
It is clear that for every w ∈ F (B1), (B2), and (B3) hold for w ◦ V and all the
results of Chaps. 3 and 5 hold.
References
1. Anderson BDO, Moore JB (1971) Linear optimal control. Prentice-Hall, Englewood Cliffs, NJ.
2. Arkin VI, Evstigneev IV (1987) Stochastic models of control and economic dynamics.
Academic Press, London.
3. Aseev SM, Kryazhimskiy AV (2004) The Pontryagin Maximum principle and transversality
conditions for a class of optimal control problems with infinite time horizons, SIAM J. Control
Optim.: 43, 1094–1119.
4. Aseev SM, Veliov VM (2012) Maximum principle for infinite-horizon optimal control
problems with dominating discount, Dynamics Contin. Discrete Impuls. Syst., Series B: 19,
43–63.
5. Atsumi H (1965) Neoclassical growth and the efficient program of capital accumulation,
Review of Economic Studies: 32, 127–136.
6. Aubry S, Le Daeron PY (1983) The discrete Frenkel-Kontorova model and its extensions I,
Physica D: 8, 381–422.
7. Bachir M, Blot J (2015) Infinite dimensional infinite-horizon Pontryagin principles for discrete-
time problems, Set-Valued and Variational Analysis: 23, 43–54.
8. Bachir M, Blot J (2017) Infinite dimensional multipliers and Pontryagin principles for discrete-
time problems, Pure and Applied Functional Analysis: 2, 411–426.
9. Baumeister J, Leitao A, Silva GN (2007) On the value function for nonautonomous optimal
control problem with infinite horizon, Syst. Control Lett.: 56, 188–196.
10. Blot J (2009) Infinite-horizon Pontryagin principles without invertibility, J. Nonlinear Convex
Anal.: 10, 177–189.
11. Blot J, Cartigny P (2000) Optimality in infinite-horizon variational problems under sign
conditions, J. Optim. Theory Appl.: 106, 411–419.
12. Blot J, Hayek N (2000) Sufficient conditions for infinite-horizon calculus of variations
problems, ESAIM Control Optim. Calc. Var.: 5, 279–292.
13. Blot J, Hayek N (2014) Infinite-horizon optimal control in the discrete-time framework.
SpringerBriefs in Optimization, New York.
14. Brock WA (1970) On existence of weakly maximal programmes in a multi-sector economy,
Review of Economic Studies: 37, 275–280.
15. Carlson DA (1990) The existence of catching-up optimal solutions for a class of infinite horizon
optimal control problems with time delay, SIAM Journal on Control and Optimization: 28,
402–422.
16. Carlson DA, Haurie A, Leizarowitz A (1991) Infinite horizon optimal control. Berlin, Springer-
Verlag.
SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1
132 References
17. Carlson DA, Jabrane A, Haurie A (1987) Existence of overtaking solutions to infinite
dimensional control problems on unbounded time intervals, SIAM Journal on Control and
Optimizaton: 25, 517–1541.
18. Cartigny P, Michel P (2003) On a sufficient transversality condition for infinite horizon optimal
control problems, Automatica J. IFAC: 39, 1007–1010.
19. Coleman BD, Marcus M, Mizel VJ (1992) On the thermodynamics of periodic phases, Arch.
Rational Mech. Anal.: 117, 321–347.
20. Cominetti R, Piazza A (2009) Asymptotic convergence of optimal harvesting policies for a
multiple species forest, Mathematics of Operations Research: 34, 576–593.
21. Damm T, Grune L, Stieler M, Worthmann K (2014) An exponential turnpike theorem for
dissipative discrete time optimal control problems, SIAM Journal on Control and Optimization:
52, 1935–1957.
22. De Oliveira VA, Silva GN (2009) Optimality conditions for infinite horizon control problems
with state constraints, Nonlinear Analysis: 71, 1788–1795.
23. Evstigneev IV, Flam SD (1998) Rapid growth paths in multivalued dynamical systems
generated by homogeneous convex stochastic operators, Set-Valued Anal.: 6, 61–81.
24. Gaitsgory V, Grune L, Thatcher N (2015) Stabilization with discounted optimal control,
Systems and Control Letters: 82, 91–98.
25. Gaitsgory V, Mammadov M, Manic L (2017) On stability under perturbations of long-run
average optimal control problems, Pure and Applied Functional Analysis: 2, 461–476.
26. Gaitsgory V, Parkinson, A, Shvartsman, I (2017) Linear programming formulations of
deterministic infinite horizon optimal control problems in discrete time, Discrete Contin. Dyn.
Syst. Ser. B: 22, 3821–3838.
27. Gaitsgory V, Rossomakhine S, Thatcher N (2012) Approximate solution of the HJB inequality
related to the infinite horizon optimal control problem with discounting, Dynamics Contin.
Discrete Impuls. Syst., Series B: 19, 65–92.
28. Gale D (1967) On optimal development in a multi-sector economy, Rev. Econ. Stud.: 34, 1–18.
29. Glizer VY, Kelis O (2017) Upper value of a singular infinite horizon zero-sum linear-quadratic
differential game, Pure and Applied Functional Analysis: 2, 511–534.
30. Grune L, Guglielmi R (2018) Turnpike properties and strict dissipativity for discrete time linear
quadratic optimal control problems, SIAM J. Control Optim.: 56, 1282–1302.
31. Gugat M, Trelat E, Zuazua, E (2016) Optimal Neumann control for the 1D wave equation:
finite horizon, infinite horizon, boundary tracking terms and the turnpike property, Systems
Control Lett. 90, 61–70.
32. Guo X, Hernandez-Lerma O (2005) Zero-sum continuous-time Markov games with unbounded
transition and discounted payoff rates, Bernoulli: 11, 1009–1029.
33. Hayek (2011) Infinite horizon multiobjective optimal control problems in the discrete time
case, Optimization: 60, 509–529.
34. Hammond PJ (1974) Consistent planning and intertemporal welfare economics. University of
Cambridge, Cambridge.
35. Hammond PJ (1975) Agreeable plans with many capital goods, Rev. Econ. Stud.: 42, 1–14.
36. Hammond PJ, Mirrlees JA (1973) Agreeable plans, Models of economic growth. Wiley, New
York, 283–299.
37. Jasso-Fuentes H, Hernandez-Lerma O (2008) Characterizations of overtaking optimality for
controlled diffusion processes, Appl. Math. Optim.: 57, 349–369.
38. Jerry M, Rapaport A, Cartigny P (2010) Can protected areas potentially enlarge viability
domains for harvesting management? Nonlinear Anal. Real World Appl.: 11 720–734.
39. Khan MA, Piazza A (2011) The economics of forestry and a set-valued turnpike of the classical
type, Nonlinear Anal.: 74, 171–181.
40. Khan MA, Piazza A (2012) On the Mitra-Wan forestry model: a unified analysis, J. Econom.
Theory: 147, 230–260.
41. Khlopin DV (2017) On Lipschitz continuity of value functions for infinite horizon problem,
Pure and Applied Functional Analysis: 2, 535–552.
References 133
42. Kolokoltsov V, and Yang W (2012) The turnpike theorems for Markov games, Dynamic Games
and Applications: 2, 294–312.
43. Leizarowitz A (1985) Infinite horizon autonomous systems with unbounded cost, Appl. Math.
and Opt.: 13, 19–43.
44. Leizarowitz A (1986) Tracking nonperiodic trajectories with the overtaking criterion, Appl.
Math. and Opt.: 14, 155–171.
45. Leizarowitz A, Mizel VJ (1989) One dimensional infinite horizon variational problems arising
in continuum mechanics, Arch. Rational Mech. Anal.:106, 161–194.
46. Lykina V, Pickenhain S, Wagner M (2008) Different interpretations of the improper integral
objective in an infinite horizon control problem, J. Math. Anal. Appl.: 340, 498–510.
47. Makarov VL, Rubinov AM (1977) Mathematical theory of economic dynamics and equilibria.
Springer-Verlag, New York.
48. Malinowska AB, Martins N, Torres DFM (2011) Transversality conditions for infinite horizon
variational problems on time scales, Optimization Lett.: 5, 41–53.
49. Mammadov M (2014) Turnpike theorem for an infinite horizon optimal control problem with
time delay, SIAM Journal on Control and Optimization: 52, 420–438.
50. Marcus M, Zaslavski AJ (1999) On a class of second order variational problems with
constraints, Israel J. Math.: 111, 1–28.
51. Marcus M, Zaslavski AJ (1999) The structure of extremals of a class of second order variational
problems, Ann. Inst. H. Poincaré, Anal. non linéaire: 16, 593–629.
52. Maruyama T (1981) Optimal economic growth with infinite planning time horizon, Proc.
Japan Acad. Ser. A Math. Sci.: 57, 469–472.
53. McKenzie LW (1976) Turnpike theory, Econometrica: 44, 841–866.
54. Mitra T, Wan HW (1985) Some theoretical results on the economics of forestry, Review of
Economics Studies: 52, 263–282.
55. Mitra T, Wan HW (1986) On the Faustmann solution to the forest management problem,
Journal of Economic Theory: 40, 229–249.
56. Mordukhovich BS (1990) Minimax design for a class of distributed parameter systems,
Automat. Remote Control: 50, 1333–1340.
57. Mordukhovich BS (2011) Optimal control and feedback design of state-constrained parabolic
systems in uncertainly conditions, Appl. Analysis: 90, 1075–1109.
58. Mordukhovich BS, Shvartsman I (2004). Optimization and feedback control of constrained
parabolic systems under uncertain perturbations, Optimal control, stabilization and nonsmooth
analysis. Lecture Notes Control Inform. Sci., Springer, 121–132.
59. Moser J (1986) Minimal solutions of variational problems on a torus, Ann. Inst. H. Poincaré,
Anal. non linéaire: 3, 229–272.
60. Ocana Anaya E, Cartigny P, Loisel P (2009) Singular infinite horizon calculus of variations.
Applications to fisheries management, J. Nonlinear Convex Anal.: 10, 157–176.
61. Piazza A (2009) The optimal harvesting problem with a land market: a characterization of the
asymptotic convergence, Economic Theory: 40, 113–138.
62. Pickenhain S, Lykina V, Wagner M (2008) On the lower semicontinuity of functionals involv-
ing Lebesgue or improper Riemann integrals in infinite horizon optimal control problems,
Control Cybernet.: 37, 451–468.
63. Rapaport A, Sraidi S, Terreaux JP (2003) Optimality of greedy and sustainable policies in the
management of renewable resources, Optimal Control Application and Methods: 24, 23–44.
64. Rockafellar RT (1970) Convex analysis, Princeton University Press, Princeton, NJ.
65. Rubinov AM (1984) Economic dynamics, J. Soviet Math.: 26, 1975–2012.
66. Sagara N (2018) Recursive variational problems in nonreflexive Banach spaces with an infinite
horizon: an existence result, Discrete Contin. Dyn. Syst. Ser. S: 11, 1219–1232.
67. Samuelson PA (1965) A catenary turnpike theorem involving consumption and the golden rule,
Amer. Econom. Rev.: 55, 486–496.
68. Samuelson PA (1976) Economics of forestry in an evolving society, Economic Inquiry: XIV,
466–492.
134 References
69. Trelat E, Zhang C, Zuazua E (2018) Optimal shape design for 2D heat equations in large time,
Pure and Applied Functional Analysis: 3, 255–269.
70. von Weizsacker CC (1965) Existence of optimal programs of accumulation for an infinite
horizon, Rev. Econ. Studies: 32, 85–104.
71. Zaslavski AJ (1987) Ground states in Frenkel-Kontorova model, Math. USSR Izvestiya: 29,
323–354.
72. Zaslavski AJ (1995) Optimal programs on infinite horizon 1, SIAM J. Control Optim.: 33,
1643–1660.
73. Zaslavski AJ (1995) Optimal programs on infinite horizon 2, SIAM J. Control and Optim.: 33,
1661–1686.
74. Zaslavski AJ (1999) Turnpike property for dynamic discrete time zero-sum games, Abstract
and Applied Analysis: 4, 21–48.
75. Zaslavski AJ (2005) The turnpike property of discrete-time control problems arising in
economic dynamics, Discrete and Continuous Dynamical Systems, B: 5, 2005, 861–880.
76. Zaslavski AJ (2006) Turnpike properties in the calculus of variations and optimal control.
Springer, New York.
77. Zaslavski AJ (2007) Turnpike results for a discrete-time optimal control systems arising in
economic dynamics, Nonlinear Anal.: 67, 2024–2049.
78. Zaslavski AJ (2009) Two turnpike results for a discrete-time optimal control systems,
Nonlinear Anal.: 71, 902–909.
79. Zaslavski AJ (2010) Stability of a turnpike phenomenon for a discrete-time optimal control
systems, Journal of Optimization theory and Applications: 145, 597–612.
80. Zaslavski AJ (2010) On a class of infinite horizon optimal control problems, Communications
in Mathematical Analysis: 9, 1–11.
81. Zaslavski AJ (2010) Good programs for a class of infinite horizon optimal control problems,
Communications on Applied Nonlinear Analysis: 17, 1–14.
82. Zaslavski AJ (2011) The existence and structure of approximate solutions of dynamic discrete
time zero-sum games, Journal of Nonlinear and Convex Analysis: 12, 49–68.
83. Zaslavski AJ (2011) Turnpike properties of solutions for a class of optimal control problems
with applications to a forest management problem, Dynamics of Continuous, Discrete and
Impulsive Systems, Ser. B Appl. Algorithms: 18, 399–434.
84. Zaslavski AJ (2014) Turnpike phenomenon and infinite horizon optimal control. Springer
Optimization and Its Applications, New York.
85. Zaslavski AJ (2016) Structure of solutions of optimal control problems on large intervals: a
survey of recent results. Pure and Applied Functional Analysis: 1, 123–158.
86. Zaslavski AJ (2017) Discrete-Time optimal control and games on large intervals. Springer
Optimization and Its Applications, Springer, 2017.
87. Zaslavski AJ, Leizarowitz A (1997) Optimal solutions of linear control systems with nonperi-
odic integrands, Math. Op. Res.: 22, 726–746.
Index
A E
Agreeable program, 58–62 Euclidean norm, 1
Approximate solution, 6, 8, 9, 33, 35, 36, 73, Euclidean space, 1
74, 82, 90, 99
Asymptotic turnpike property, 65
Autonomous discrete-time control system, 8 F
Forest management problem, 9, 11–16, 18, 20,
21, 30–31, 33, 35, 64, 109–130
B
Bad program, 35, 65 G
Baire category approach, 63 Generic turnpike property, 2, 6, 8, 9, 63–71
Banach space, 67, 89, 98 Good program, 35, 36, 65, 66, 68, 71, 74, 124,
Bolza problem, 89–90, 98–99 128, 130
C I
Cardinality of a set, 6, 35 Increasing function, 124
Compact metric space, 8, 16, 33, 63, 73 Infinite horizon, 9, 11–31, 33
Compact set, 13, 111 Infinite horizon optimal control problem,
Complete metric space, 63, 67 11–31
Concave function, 13, 20, 118, 119 Inner product, 1
Constrained problem, 8 Interior point, 9
Continuous-time problem, 1
Control system, 2, 8, 34, 73, 111
Convex discrete-time problems, 1–7 L
Convex problems, 1 Lagrange problem, 78–83
Convex set, 119 Locally optimal program, 37, 50, 58, 78
D N
Differentiable function, 2, 3, 8 Neumann path, 2
Discrete-time problem, 1–8 Normed space, 20
SpringerBriefs in Optimization, https://doi.org/10.1007/978-3-030-23587-1
136 Index
O S
Objective function, 2, 8, 63, 73 Smooth function, 13
Optimal control problem, 1, 2, 8, 9, 11–31, 33, Strictly convex function, 2, 3, 13
63, 73, 82, 123 Strongly agreeable program, 58
Optimality criterion, 8, 20, 28, 33, 37, 73
Overtaking optimal program, 28, 50, 74, T
75, 77, 78, 82–86, 90–94, 98–100, Turnpike, 7–9, 73
102–104, 108 Turnpike phenomenon, 8–9, 33
Turnpike property, 1, 2, 6, 8, 9, 33–71
Turnpike result, 2, 7, 8, 33, 55, 65, 118–122
P
Program, 8, 12–19, 21–23, 26–30, 33–45, U
47–56, 58–61, 63–71, 73–87, 89–95, Unconstrained problem, 8
98–100, 102–105, 108, 110–112, 114, Upper semicontinuous function, 8, 13, 28, 31,
115, 117–120, 124, 128–130 53, 63, 73, 90, 98, 99, 111

Optimal Control Problems Arising in Forest Management: Alexander J. Zaslavski

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Optimal Control Problems Arising in Forest Management: Alexander J. Zaslavski

Uploaded by

Copyright:

Available Formats

SPRINGER BRIEFS IN OPTIMIZATION

SpringerBriefs in Optimization showcases algorithmic and theoretical tech-

More information about this series at http://www.springer.com/series/8918

Optimal Control Problems

ISSN 2190-8354 ISSN 2191-575X (electronic)

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019

Rishon LeZion, Israel Alexander J. Zaslavski

5 Structure of Solutions in the Regions Close to the Endpoints . . . . . . . . . . 73

1.1 Convex Discrete-Time Problems

Let v : R n × R n → R 1 be bounded from below function. We consider the

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 1

such that {xi }Ti=0 ⊂ R n and x0 = z, xT = y,

where T is a natural number and the points y, z ∈ R n .

v(y, z)/(|y| + |z|) → ∞ as |y| + |z| → ∞. (1.1)

v(x, x) → min, x ∈ R n . (P1 )

∇v(x̄, x̄) = (l1 , l2 ), (1.2)

l1 , h + l2 , h = (l1 , l2 ), (h, h)

l1 + l2 , h ≥ 0 for all h ∈ R n ,

∇v(x̄, x̄) = (l1 , −l1 ), (1.3)

For each (y, z) ∈ R n × R n set

L(y, z) = v(y, z) − v(x̄, x̄) − ∇v(x̄, x̄), (y − x̄, z − x̄)

It is not difficult to verify that the function L : R n × R n → R 1 is differentiable and

L(y, z)/(|y| + |z|) → ∞ as |y| + |z| → ∞. (1.5)

L(y, z) ≥ 0 for all (y, z) ∈ R n × R n (1.6)

L(y, z) = 0 if and only if y = x̄, z = x̄. (1.7)

We claim that the function L : R n × R n → R 1 has the following property:

lim (yi , zi ) = (x̄, x̄).

Assume that a sequence {(yi , zi )}∞i=1 ⊂ R × R satisfies limi→∞ L(yi , zi ) = 0.

In view of (1.5) the sequence {(yi , zi )}∞

L(y, z) = lim L(yi , zi ) = 0

Consider an auxiliary minimization problem

such that {xi }Ti=0 ⊂ R n and x0 = z, xT = y,

where T is a natural number and the points y, z ∈ R n .

holds. Clearly, if a sequence {xi }Ti=0 ⊂ R n is (0)-optimal, then it is a solution of the

We may assume without loss of generality that

In view of (1.10), (1.11), and (1.12) we may assume that

By (1.13) and (1.5) there is M3 > 0 such that

|xi(k) | ≤ M3 for all i = 0, . . . , T and all integers k ≥ 1. (1.14)

In view of (1.14), extracting subsequences, using diagonalization process and re-

By (1.15) and (1.11),

It follows from (1.15) and (1.12) that

|x0 | ≤ M2 , |xT | ≤ M2 , (1.17)

the following inequality holds:

Card({i ∈ {0, . . . , T − 1} : |xi − x̄| + |xi+1 − x̄| > }) ≤ k0 .

|y − x̄| + |z − x̄| ≤ . (1.19)

M3 = sup{L(y, z) : y, z ∈ R n and |y| + |z| ≤ |x̄| + M2 }. (1.20)

Choose a natural number

k0 > δ −1 (M1 + 2M3 ). (1.21)

Together with (1.7), (1.8), and (1.22) this implies that

Combined with (1.17) and (1.20) this implies that

Card({i ∈ {0, . . . , T − 1} : |xi − x̄| + |xi+1 − x̄| > })

≤ Card({i ∈ {0, . . . , T − 1} : L(xi , xi+1 ) > δ})

Proposition 1.2 is proved.

Card({i ∈ {0, . . . , T − 1} : |xi − x̄| + |xi+1 − x̄| > }) ≤ k0 .

{i ∈ {0 . . . , T } : |xi − x̄| > }

1.2 The Turnpike Phenomenon

where T ≥ 1 is an integer and the points y, z ∈ X. Clearly, these constrained

be useful if we need to find an “approximate” solution of the problem (P ) with a

xm1 = z, xm2 = y and xi = x̄ for all i = m1 + L, . . . , m2 − L.

2.1 The Forest Management Problem

vi (t) ≤ x̄i (t) + xini (t), (2.1)

x̄i (t + 1) = x̄i (t) + xini (t) − vi (t). (2.2)

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2019 11

ut = sup{|ut (z)| : z ∈ Ω} (2.30)

sup{ut : t = 0, 1, . . . } < ∞. (2.31)

(0, p) − μ| ≤ 2M/p for all natural numbers p.

M = 2(2L̄ + 4) sup{ut : t = 0, 1, . . . }. (2.37)