(Nisio) Stochastic Control Theory (2015)

Probability Theory and Stochastic Modelling 72
Makiko Nisio
Stochastic
Control
Theory
Dynamic Programming Principle
Second Edition
Probability Theory and Stochastic Modelling
Volume 72
Editors-in-Chief
Søren Asmussen, Aarhus, Denmark
Peter W. Glynn, Stanford, CA, USA
Thomas Kurtz, Madison, WI, USA
Yves Le Jan, Paris, France
Advisory Board
Joe Gani, Canberra, Australia
Martin Hairer, Coventry, UK
Peter Jagers, Gothenburg, Sweden
Ioannis Karatzas, New York, NY, USA
Frank P. Kelly, Cambridge, UK
Andreas E. Kyprianou, Bath, UK
Bernt Øksendal, Oslo, Norway
George Papanicolaou, Stanford, CA, USA
Etienne Pardoux, Marseille, France
Edwin Perkins, Vancouver, Canada
Halil Mete Soner, Zurich, Switzerland
The Stochastic Modelling and Probability Theory series is a merger and con-
tinuation of Springer’s two well established series Stochastic Modelling and
Applied Probability and Probability and Its Applications series. It publishes
research monographs that make a significant contribution to probability theory or
an applications domain in which advanced probability methods are fundamental.
Books in this series are expected to follow rigorous mathematical standards, while
also displaying the expository quality necessary to make them useful and accessible
to advanced students as well as researchers. The series covers all aspects of modern
probability theory including
• Gaussian processes
• Markov processes
• Random Fields, point processes and random sets
• Random matrices
• Statistical mechanics and random media
• Stochastic analysis
as well as applications that include (but are not restricted to):
• Branching processes and other models of population growth

• Communications and processing networks
• Computational methods in probability and stochastic processes, including
simulation
• Genetics and other stochastic models in biology and the life sciences
• Information theory, signal processing, and image synthesis
• Mathematical economics and finance
• Statistical methods (e.g. empirical processes, MCMC)
• Statistics for stochastic processes
• Stochastic control
• Stochastic models in operations research and stochastic optimization
• Stochastic models in the physical sciences
More information about this series at http://www.springer.com/series/13205

Makiko Nisio
Stochastic Control Theory

Dynamic Programming Principle
123
Makiko Nisio (emeritus)
Kobe University
Kobe, Japan
Osaka Electro–Communication University
Osaka, Japan
First edition published in the series ISI Lecture Notes, No 9, by MacMillan India Limited publishers,
Delhi, c Makiko Nisio, 1981
ISSN 2199-3130 ISSN 2199-3149 (electronic)

ISBN 978-4-431-55122-5 ISBN 978-4-431-55123-2 (eBook)
DOI 10.1007/978-4-431-55123-2
Springer Tokyo Heidelberg New York Dordrecht London
Library of Congress Control Number: 2014953914
Mathematics Subject Classification: 93E20, 60H15
© Springer Japan 2015

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)

Preface
The purpose of this book is to provide an introduction to stochastic controls theory,

via the method of dynamic programming. The dynamic programming principle,
originated by R. Bellman in 1950s, is known as the two stage optimization
procedure. When we control the behavior of a stochastic dynamical system in order
to optimize some payoff or cost function, which depends on the control inputs to
the system, the dynamic programming principle gives a powerful tool to analyze
problems. Exploiting the dependence of the value function (optimal payoff) on
its terminal cost function, we will construct a nonlinear semigroup which allows
one to formulate the dynamic programming principle and whose generator provides
the Hamilton–Jacobi–Bellman equation. Here we are mainly concerned with finite
time horizon stochastic controls. We also apply the semigroup approach to control-
stopping problems and stochastic differential games, and provide with examples
from the area of financial market models.
This book is organized as follows. Chapters 1–4 deal with completely observable
finite-dimensional controlled diffusions. Chapters 5 and 6 are concerned with
Hilbert space valued stochastic processes, related to partially observable control
problems.
Chapter 1 is a review of stochastic analysis and stochastic differential equations
with random coefficients for later uses. Chapter 2 deals with control problems with
finite-time horizon. By a time-discretization method we construct a semigroup,
associated with the value function, whose generator provides the Hamilton–Jacobi–
Bellman equation. When the value function is smooth, it becomes a classical
solution of the Hamilton–Jacobi–Bellman equation. However, it satisfies the equa-
tion in viscosity sense even if it is not smooth. Chapter 3 is concerned with viscosity
solutions of nonlinear parabolic equation, including Hamilton–Jacobi–Bellman
equations of stochastic controls and also stochastic optimal control-stopping prob-
lems. Chapter 4 presents zero sum, two-player, time-homogeneous, stochastic
differential games and the Isaacs equations. We consider stochastic differential
games by using progressive strategies. Then we construct semigroups associated
with the upper and lower values, by using a semidiscretization method. These
semigroups lead to the formulation of the dynamic programming principle and
v
vi Preface
to the upper and lower Isaacs equations. The link between stochastic control and
differential game is given via the risk sensitive control. Chapter 5 is a review on
stochastic evolution equations on Hilbert spaces, in particular stochastic parabolic
equations with colored Wiener noises. Basic definitions and results and Itô’s
formula are presented. Chapter 6 is concerned with control problems for Zakai
equations. We again construct semigroups associated with the value functions.
The dynamic programming principle and viscosity solutions of Hamilton–Jacobi–
Bellman equations on Hilbert spaces are treated by using results obtained in the
previous chapters. We show the connection between controlled Zakai equations and
control of partially observable diffusions.
Kobe, Japan Makiko Nisio

Acknowledgement
This book was planned as a new edition of Stochastic Control Theory, ISI
Lecture Notes 9 (1981) following F. Delbaen’s recommendation. I would like to
acknowledge his recommendation together with valuable advice during preparation
of the manuscript.
The author is greatly indebted to W. H. Fleming, who read carefully the
manuscript and offered many valuable comments and suggestions, especially for
Chap. 4, which led to a much improved version. Many thoughtful helps and
encouragements had been given by experts and friends, particularly F. Asakura,
Y. Fujita, H. Nagai, and T. Uratani who helped to improve the book at various stages.
H. Morimoto assisted in writing the manuscript by carefully reading it and making
valuable comments, especially on mathematical economics.
vii
Contents
1 Stochastic Differential Equations . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1

1.1 Review of Stochastic Processes . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
1.1.1 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
1.1.2 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5
1.1.3 Itô Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10
1.1.4 Itô’s Formula .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13
1.2 Stochastic Differential Equations . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 15
1.2.1 Lipschitz Continuous SDEs with Random Coefficients . . . . . . 15
1.2.2 Girsanov Transformations .. . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 18
1.2.3 SDEs with Deterministic Borel Coefficients . . . . . . . . . . . . . . . . . . 22
1.2.4 Backward Stochastic Differential Equations . . . . . . . . . . . . . . . . . . 24
1.3 Asset Pricing Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 26
1.3.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 26
1.3.2 Backward SDE for the Selling Price . . . . . . .. . . . . . . . . . . . . . . . . . . . 27
1.3.3 Parabolic Equation Associated with (1.84) .. . . . . . . . . . . . . . . . . . . 29
2 Optimal Control for Diffusion Processes . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 31
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 31
2.1.1 Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 32
2.1.2 Value Functions: Basic Properties . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 34
2.2 Dynamic Programming Principle (DPP) . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 40
2.2.1 Discrete-Time Dynamic Programming Principle . . . . . . . . . . . . . 41
2.2.2 Approximation Theorem . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 46
2.2.3 Dynamic Programming Principle . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 47
2.2.4 Brownian Adapted Controls . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 50
2.2.5 Characterization of the Semigroup (V t ; ™ t) . . . . . . . . . . . . . . . 53
2.3 Verification Theorems and Optimal Controls .. . . . .. . . . . . . . . . . . . . . . . . . . 58
2.3.1 Verification Theorems . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 58
2.3.2 Examples of Optimal Control .. . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 62
2.4 Optimal Investment Models . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 68
2.4.1 Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 68
ix
x Contents
2.4.2 Investment Problems for Power Utility Function . . . . . . . . . . . . . 71

2.4.3 Optimal Investment Strategy . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 73
3 Viscosity Solutions for HJB Equations . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 79
3.1 Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 79
3.1.1 Definition of Viscosity Solution Based on
Parabolic Differentials .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 80
3.1.2 Equivalent Definitions .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 82
3.1.3 Viscosity Solutions via Semigroups . . . . . . .. . . . . . . . . . . . . . . . . . . . 86
3.2 Uniqueness of Viscosity Solutions . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 88
3.2.1 Crandall–Ishii Lemma .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 88
3.2.2 Structural Condition . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 93
3.2.3 Comparison Principle.. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 95
3.3 HJB Equations for Control-Stopping Problems . . .. . . . . . . . . . . . . . . . . . . . 100
3.3.1 Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 100
3.3.2 DPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 102
3.3.3 Semigroups Associated with DPP . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 109
3.3.4 American Option Price . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 112
4 Stochastic Differential Games . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 117
4.1 Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 117
4.1.1 Admissible Controls and Strategies .. . . . . . .. . . . . . . . . . . . . . . . . . . . 117
4.1.2 Formulation of Stochastic Differential Games . . . . . . . . . . . . . . . . 120
4.2 DPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 125
4.2.1 D-Lower and D-Upper Value Functions . .. . . . . . . . . . . . . . . . . . . . 125
4.2.2 DPP for Lower- and Upper Value Functions . . . . . . . . . . . . . . . . . . 131
4.3 Isaacs Equations.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 134
4.3.1 Semigroups Related to the DPP . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 135
4.3.2 Viscosity Solutions of the Isaacs Equations . . . . . . . . . . . . . . . . . . . 138
4.4 Risk Sensitive Stochastic Controls and Differential Games . . . . . . . . . . 141
4.4.1 Logarithmic Transformation . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 141
4.4.2 Small Noise Limit . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 144
4.4.3 Note on Control with Infinite Time Horizon . . . . . . . . . . . . . . . . . . 148
5 Stochastic Parabolic Equations. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 153
5.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 153
5.1.1 H-Random Variables . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 154
5.1.2 Continuous Martingales . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 156
5.1.3 Correlation Operators.. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 159
5.2 Stochastic Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 160
5.2.1 Definitions and Basic Properties . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 160
5.2.2 Martingale Inequalities . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 164
5.3 Stochastic Parabolic Equations with Colored Wiener Noises .. . . . . . . . 167
5.3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 168
5.3.2 Linear Stochastic Parabolic Equations .. . . .. . . . . . . . . . . . . . . . . . . . 171
5.3.3 Regularities of Solutions . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 174
Contents xi
5.3.4 Semilinear Stochastic Parabolic Equations with

Lipschitz Nonlinearity .. . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 177
5.4 Itô’s Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 183
5.4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 184
5.4.2 Itô’s Formula for H-Valued Semimartingales . . . . . . . . . . . . . . . . . 186
5.4.3 Itô’s Formula for Linear Stochastic Parabolic Equations . . . . . 190
5.5 Zakai Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 197
5.5.1 Partially Observable Controlled Diffusion . . . . . . . . . . . . . . . . . . . . 197
5.5.2 Zakai Equation for a Conditional Gaussian Process . . . . . . . . . . 203
6 Optimal Controls for Zakai Equations . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 209
6.1 Controlled Zakai Equations.. . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 209
6.1.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 210
6.1.2 Basic Properties of Solutions.. . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 211
6.2 Formulations of Control Problems . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 217
6.2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 218
6.2.2 Properties of the Terminal Cost . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 220
6.2.3 Running Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 222
6.2.4 Value Functions .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 224
6.3 Dynamic Programming Principle (DPP) . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 225
6.3.1 Discrete-Time DPP . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 225
6.3.2 Continuous-Time DPP. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 228
6.3.3 Characterization of .Vt / . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 231
6.3.4 Derivatives of Vt F(¥) .. . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 232
6.4 Viscosity Solutions of HJB Equations . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 236
6.4.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 236
6.4.2 Existence of Viscosity Solutions .. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 237
6.4.3 Uniqueness of the Viscosity Solution .. . . . .. . . . . . . . . . . . . . . . . . . . 241
References .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 245
Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 249
Notations
Let a and b be real numbers.

a _ b D maxfa; bg; a ^ b D minfa; bg
aC D maxfa; 0g; a D minfa; 0g
ıi;j D Kronecker symbol
Rd denotes d -dimensional Euclidean space
x i denotes
Pdthe i -th coordinate of x 2 Rd
x y D i D1 x y , jxj2 D x x
i i
RdC D fx 2 Rd I x i > 0; i D 1; : : : ; d g
Sr D fx 2 Rd I jxj rg
Rd ˝ Rm denotes the set of d m matrices
Aij D .i; j / entry of A 2 Rd ˝ Rm
A> D transpose of A
tr A D trace of A 2 Rd ˝ Rd
jAj2 D tr.A> A/ D tr.AA> / for A 2 Rd ˝ Rm
Id D d -dimensional unit matrix
S d D the set of symmetric d d matrices
SC d
D fA 2 S d I A is nonnegative definite}
SCC D fA 2 S d I A is positive definite}
d
B.U / = the -field generated by all open subsets of U

Lp .˝; GI Rd / D the set of G-measurable d -dimensional random variables with
E j jp < 1; for p 1
Lp .Œ0; T ˝; .Ft /I Rd / D the set of .Ft /-progressively measurable
RT
d -dimensional processes with 0 E j X.t/ jp dt < 1; for p 1
D L1 .Œ0; T ˝; .Ft /I /
./ D the -field generated by
Let ˙ be a metric space.
C.˙/ D the set of real valued continuous functions defined on ˙
Cb .˙/ D f 2 C.˙/I is boundedg
Cbu .˙/ D f 2 Cb .˙/I is uniformly continuousg
xiii
xiv Notations
Cp .˙/ D f 2 C.˙/I is polynomial growingg, when ˙ is a Banach space.

CK1 .Rd / D f l 2 C.Rd /I has compact support and continuous derivatives of
any orderg
C 12 ..0; T / Rd / D f 2 C..0; T / Rd /I @t ; @i ; @ij 2 C..0; T / Rd /;
i; j D 1; : : : ; d g,
2
where @t D @t@ ; @i D @x@ i ; @ij D @x@i @xj
C 12 .Œ0; T / Rd / D f 2 C.Œ0; T / Rd / \ C 12 ..0; T / Rd /I
@t ; @i ; @ij can be extended to continuous functions on
C.Œ0; T / Rd /; i; j D 1; : : : ; d g
C 12 ..0; T Rd / and C 12 .Œ0; T Rd / are defined similarly
D the indicator function of the set
@x D gradient vector of
@xx D matrix of second order partial derivatives of D .@ij /i;j D1;:::;d
= D restricted to a set
H0 D L2 .Rd / with the usual inner product .; / and norm k k

I D identity mapping
D Laplacian operator
Hp D f 2 H0 I generalized derivatives of up to order p belong to H0 g; p D
1; 2; : : : p
k kHp D k .I / 2 k
1
H1 D fBorel functions I .I / 2 2 H0 g
1
jjjjjj Dk kH1 ; k k Dk kH1 Dk .I / 2 k
h; i D duality product between 2 H1 and 2 H1
Let H and Y be Hilbert spaces.
L2 .HI Y/ D Hilbert space of Hilbert–Schmidt operators from H into Y
1
k ˚ kQ Dk ˚Q 2 kL2 .HIY/ , where Q is a symmetric and nonnegative definite
operator on H
M2c .Œ0; T ; .Ft /I H/ D set of continuous and square integrable H-valued

.Ft /-martingales on Œ0; T
hM i D quadratic variation process of M
hM; N i D quadratic variation process corresponding to M and N
D p F .z/ D p-th Fréchet derivative of F at z
C 12 .Œ0; T H/ D fF 2 C.Œ0; T H/I @t F; DF and D 2 F are continuous on
Œ0; T H}
A ) B D if A then B
Abbreviations
RHS D right-hand side

LHS D left-hand side
USC D upper semicontinuous
LSC D lower semicontinuous
ONB D orthonormal basis
w.r.t. D with respect to
a.e. D almost everywhere
xv
Chapter 1
Stochastic Differential Equations
Abstract The purpose of this chapter is to overview elements of the theory

of stochastic differential equations, based on Wiener processes, for use in the
subsequent chapters. This theory was founded by K. Itô in 1942 (Itô, Zenkoku Shijo
Sugaku Danwakai 244:1352–1400, 1942; Itô, On stochastic differential equations.
Memoirs of the American Mathematical Society, vol 4. AMS, New York City,
1951). His aim was to construct Markov processes, governed by Kolmogorov’s
differential equations via Wiener processes, and to analyze their sample paths.
After that, stochastic differential equations have been used to describe dynamical
processes in random environments of various fields. Here we consider stochastic dif-
ferential equations with random coefficients, because we aim at studying stochastic
control problems.
The chapter is organized as follows. Section 1.1 is preliminaries. The basic
definitions and results on stochastic processes are collected for later use. Stochas-
tic differential equations and stochastic analysis will be introduced in Sect. 1.2.
Section 1.3 deals with asset pricing problems as an application of previous results.
1.1 Review of Stochastic Processes
This section collects basic definitions and results on stochastic processes. Proofs of
the results are mainly referred to standard and easily accessible books.
1.1.1 Random Variables
1. Measurable maps
Let ˝ be a non-empty set and F a -field of ˝. We call the pair (˝; F ) a
measurable space. Let (˝; F ) and .˝ 0 ; F 0 / be measurable spaces and X W ˝ 7!
˝ 0 be a given map. For a -field G . F /, X is called G=F 0 -measurable (or G-
measurable in short), if X 1 .B/ 2 G for any B 2 F 0 . Sometimes X is called
measurable, when G D F .
© Springer Japan 2015 1

M. Nisio, Stochastic Control Theory, Probability Theory
and Stochastic Modelling 72, DOI 10.1007/978-4-431-55123-2_1
2 1 Stochastic Differential Equations
2. Random variables
Let .˝; F ; P / be a complete probability space, namely .˝; F / is a measurable
space, P a probability measure on F , and N WD fA 2 F I P .A/ D 0g satisfies the
condition
A 2 N; B A H) B 2 N : (1.1)
(1) Let .U; d / be a Polish space, i.e., U is a separable complete metric space with
a metric d , and let B.U / be the -field generated by the open subsets of U .
X W ˝ 7! U is called a U -valued random variable if X is F =B.U /-measurable.
We identify two U -valued random variables X and Y , whenever there is
N 2 N , such that
X.!/ D Y .!/; 8! … N: (1.2)
When (1.2) holds, we write
X DY P -a.s. (almost surely): (1.3)
(2) Any U -valued random variable X induces a probability measure PX on B.U /

by
PX .B/ D P .X 1 .B//; 8B 2 B.U /:
PX is called the probability distribution (or law) of X .

(3) When U D Rd , we call X a d -dimensional random variable.
3. Expectations and conditional expectations
(1) Let X be a real random variable. The expectation (or mean) of X is defined by
Z Z 1
EX D X.!/ dP .!/ D x dPX .x/
˝ 1
R
provided jxj dPX .x/ < 1.
For a d -dimensional random variable X D .X 1 ; : : : ; X d /, we put EX D
.EX 1 ; : : : ; EX d /.
(2) Conditional expectation
Let G be a sub -field of F . For a real integrable random variable X , we put
Z
EŒX I B D X dP D EŒB X ; B 2 G:
B
Since EŒX I is a signed measure on G, satisfying
EŒX I B D 0; if P .B/ D 0;
1.1 Review of Stochastic Processes 3
the Radon–Nikodým theorem yields a unique (up to indentification) G-

measurable integrable function , such that
Z
EŒX I B D dP D EŒI B; 8B 2 G:
B
Definition 1.1. is called the conditional expectation of X given G, and is denoted

by E.X jG/.
It holds that, if G1 G2 . F /, then
E.E.X jG2 /jG1 / D E.X jG1 / P -a.s. (1.4)
For 2 F , E. jG/ is called the conditional probability of given G, and denoted
by P . jG/.
Definition 1.2. Let X be a d -dimensional random variable and G be a sub--field
of F . We say X and G are mutually independent, if
P .X 2 AjG/ D P .X 2 A/ P -a.s.; 8A 2 B.Rd /; (1.5)
i.e.,
P ..X 2 A/ \ B/ D P .X 2 A/P .B/; 8A 2 B.Rd /; 8B 2 G:
We say that X and Y are mutually independent if X and .Y / are mutually
independent.
4. Regular conditional probability
Since P . jG/ is defined uniquely up to sets of P -measure zero (possibly
depending on ), P .jG/.!/ may not be a probability on F when ! is fixed. We
next introduce a regular conditional probability.
Definition 1.3. p W F ˝ 7! Œ0; 1 is called a regular conditional probability given
G, if
(a) p.; !/ is a probability measure on .˝; F /, 8! 2 ˝,
(b) p.A; / is G-measurable, 8A 2 FR,
(c) 8A 2 F , 8B 2 G, P .A \ B/ D B p.A; !/ dP .!/.
Concerning the existence of regular conditional probability we have;
Theorem 1.1. Suppose that ˝ is a Polish space and F D B.˝/. Then there exists
one and only one regular conditional probability given G, i.e., if both of p and pO are
regular conditional probabilities, then
p.A; / D p.A;
O /; 8A 2 F ; P -a.s.
(see [SV79], p. 13, [IW81], p. 14 for details).

5. Convergence theorems
Let Xn ; n D 1; 2; : : : and X be d -dimensional random variables.
Definition 1.4. (a) Xn converges to X in probability .Xn ! X in prob), if
limn!1 P .jXn X j > "/ D 0, 8" > 0;
(b) Xn converges to X almost surely .Xn ! X P -a.s./, if limn!1 jXn X j D 0
P -a.s.;
(c) Xn converges to X in p-th mean .Xn ! X in Lp /, if limn!1 EjXn X jp D 0;
(d) Xn converges to X in law .Xn ! X in law), if limn!1 Ef .Xn / D Ef .X /,
8f 2 Cb .Rd /.
For the law convergence, X and Xn ; n D 1; 2; : : : may be defined on different
probability spaces.
Proposition 1.1. The following relations hold:
(i) Xn ! X P -a.s. ) Xn ! X in prob ) Xn ! X in law.
(ii) If Xn ! X in prob, then there is a subsequence which converges P -a.s.
Now we state the basic theorems of the passage to the limit under the sign of
conditional expectation.
Let Xn be an integrable real random variable, n D 1; 2; : : : , and G be a sub--
field of F .
Monotone Convergence Theorem
Suppose that Xn 0 and is increasing to X. 1/ P -a.s. Then
E.Xn jG/ % E.X jG/. 1/ P -a.s.
Fatou’s Lemma
Let Xn 0; n D 1; 2; : : : . Then
E.lim inf Xn jG/ lim inf E.Xn jG/ P -a.s.

n!1 n!1
Dominated Convergence Theorem
Suppose that jXn j Y , with integrable Y; n D 1; 2; : : : ; and Xn ! X P -a.s. Then
E.jXn X jjG/ ! 0 P -a.s.
When Y D C D const, we call this the bounded convergence theorem.

Convergence Theorem
Let Xn ! X P -a.s. Suppose that Xn ; n D 1; 2; : : : are uniformly integrable given

G, that is, as c ! 1
sup E.jXn j.jXn j > c/jG/ ! 0 in prob: (1.6)

n
Then E.jXn X jjG/ ! 0 P -a.s.

Proposition 1.2. If there is p > 1 such that supn E.jXn jp jG/ < 1, P -a.s., then
(1.6) holds.
(see [LS01], Theorems 1.1–1.4).
1.1.2 Stochastic Processes
1. Basic definitions
(1) Let .˝; F ; P / be a complete probability space and T a positive constant.
A family of sub--fields .Ft / WD .Ft I t 2 Œ0; T / is called a filtration,
if Fs Ft F for 0 s < t, and F0 contains all T P -null sets.
If Ft is right continuous, namely Ft D Ft C WD s>t Fs ; 8t 2 Œ0; T /, then
the filtration.Ft / is said to satisfy the usual condition.
If .Ft / is a filtration, then .Ft C / satisfies the usual condition.
Definition 1.5. We call a quadruple .˝; F ; .Ft /; P / a filtered probability
space when .Ft / satisfies the usual condition.
(2) A stochastic process X D .X.t/I t 2 Œ0; T / is a family of (d -dimensional)
random variables, defined on .˝; F ; P /. We identify two stochastic processes
X and Y , whenever there is a P -null set N such that
X.t; !/ D Y .t; !/; 8t; for ! … N:
We call Y is a modification of X , if
P .X.t/ D Y .t// D 1; 8t:
For fixed !, the function t 7! X.t; !/ on Œ0; T is called a sample path.

If there is a P -null set N such that X.t; !/ is continuous (or right
continuous) in t for ! … N , then X is called a continuous (or right continuous)
process.
Put FtX D completion of the -field spanned by .X.s/I s t/.
(3) Let us introduce two kinds of measurability associated with the filtration .Ft /.
Definition 1.6. Suppose X W Œ0; T ˝ 7! Rd is B.Œ0; T / F =B.Rd /-

measurable.
(a) X is called .Ft /-adapted, if X.t/ is Ft -measurable for any t 2 Œ0; T .
(b) X is called .Ft /-progressively measurable, if the restriction of X to Œ0; t is
B.Œ0; t/ Ft =B.Rd /-measurable, for any t 2 Œ0; T .
From the definition it is clear that if X is .Ft /-progressively measurable, then
it is .Ft /-adapted. When X is continuous, the converse is true. We note that,
when X is .Ft /-adapted, it has an .Ft /-progressively measurable modification
(see [My66], p. 18).
2. Stopping times
Let .˝; F ; .Ft /; P / be a filtered probability space.
Definition 1.7. W ˝ 7! Œ0; T is called an .Ft /-stopping time, if
f!I .!/ tg 2 Ft ; 8t 2 Œ0; T :
For any stopping time ; F is defined by
F D fA 2 F I A \ . t/ 2 Ft ; 8t 2 Œ0; T g: (1.7)
F is a -field, which intuitively consists of information up to .

Example 1.1. Let X be a continuous .Ft /-adapted d -dimensional process. For an
open (or closed) set A Rd , the hitting time of A, defined as,
(
infft < T I X.t; !/ 2 Ag
.!/ WD
T; iff g D empty
is an .Ft /-stopping time.

For .Ft /-stopping times and , the following facts hold:
(i) F \ F D F ^ ;
(ii) If X is .Ft /-progressively measurable, then X. / is F -measurable;
(iii) Let be a random variable with Ejj < 1. Then
E.E.jF /jF / D E.jF ^ / P -a.s.
For details, refer to [LS01], Section 1.3, [KS91], Section 1.1.2 and [IW81],
1.5.
3. Continuous martingales
(1) Let X be a continuous .Ft /-adapted real process.
Definition 1.8. X is called a continuous .Ft /-martingale (supermartingale or

submartingale), if EjX.t/j < 1; 8t, and
martingale property: E.X.t/jFs / D X.s/ P -a.s. 8s t
(E.X.t/jFs / X.s/, or E.X.t/jFs / X.s/ P -a.s. 8s t, respectively).

For an .Ft /-martingale X (super- or submartingale), EX.t/ is constant
(decreasing or increasing).
(2) Let us recall some basic properties of continuous martiangales, for later use.
(i) Let X be a continuous .Ft /-martingale.
Suppose that f W R1 7! R1 is convex and Ejf .X.t//j < 1; 8t. Then
.f .X.t//; t 2 Œ0; T / is an .Ft /-submartingale.
For instance, we can take f .x/ D jxjp .p 1/.
(ii) Maximal inequality
Let X be a non-negative continuous .Ft /-submartingale. Then
h i p
E sup X.t/p EŒX.T /p ; p > 1; (1.8)
0t T p1
holds (see [LS01], Theorem 3.2, for the proof).

(iii) Optional sampling theorem
Let and be .Ft /-stopping times and X be a continuous .Ft /-martingale.
Then
E.X. /jF / D X. ^ / P -a.s.
(iv) Optional stopping theorem

Let be an .Ft /-stopping time and X an .Ft /-martingale. The stopped
process X ./ is defined by X .t/ D X. ^ t/. Then X ./ is an .Ft ^ /-
martingale. X ./ is also an .Ft /-martingale.
4. Square integrable martingales
Let X be a continuous .Ft /-martingale with X.0/ D 0.
Suppose that EjX.t/j2 < 1; 8t. Then .X.t/2 I t 0/ is a continuous .Ft /-
submartingale.
Hence, by the Doob–Meyer decomposition theorem, there exists one and only
one a continuous, integrable, and increasing .Ft /-adapted process .A.t/I t 0/
with A.0/ D 0, such that .X.t/2 A.t/I t 0/ is a continuous .Ft /-martingale.
A.t/ is denoted by hX i.t/ and is called the quadratic variational process of X .
hX i.t/ is obtained in the following way. Let 0 D t0 < t1N < < tNN D t be
a partition of Œ0; t such that
lim max jtiNC1 tiN j D 0:

N !1 i
Then, as N ! 1.
X
jX.tiNC1 / X.tiN /j2 ! hX i.t/ in prob (1.9)
i
(see [IW81], p. 36 and p. 69 for the proof).

Let X and Y be square integrable continuous .Ft /-martingales with X.0/ D
Y .0/ D 0. Then there exists one and only one continuous and integrable .Ft /-
adapted process hX; Y i./ with hX; Y i.0/ D 0 and bounded variation on Œ0; T ,
P -a.s., and such that .X Y /./ hX; Y i./ is an .Ft /-martingale. hX; Y i is called the
quadratic variational process corresponding to X and Y .
Indeed, hX; Y i.t/ WD 14 .hX C Y i.t/ hX Y i.t// has the required properties.
Considering (1.9), we formally denote
.dX.t//2 D d hX i.t/; dX.t/d Y .t/ D d hX; Y i.t/:
Then
jd hX; Y i.t/j2 d hX i.t/ d hY i.t/:
We have the following maximal inequality, for the quadratic variational process.
Burkholder–Davis–Gundy Inequality
Let X be a square integrable continuous .Ft /-martingale, with X.0/ D 0. Then, for
p 2 .0; 1/ there exist positive constants cp and Cp , such that
h i h i
cp E sup jX.s/j2p EŒhX i.t/p Cp E sup jX.s/j2p : (1.10)
0st 0st
In detail, c1 D 14 ; C1 D 1,
22p p 2pC1 p

for p > 1; cp D ; Cp D .4p/p ;
.2p 1/2p1
16 p
for p 2 .0; 1/; cp D ; Cp D p 2p
p
(see [IW81], p. 110 for the proof).

5. Local martingales
Let X be a continuous .Ft /-adapted real process. X is called a continuous .Ft /-
local martingale, if there exists a nondecreasing sequence of .Ft /-stopping times
. n ; n D 1; 2; : : : / such that P .limn!1 n D T / D 1 and the stopped process X n
is a continuous .Ft /-martingale for each n.
Let Qn be the exit time of X from Œn; n. Then n WD n ^ Qn is increasing to

T as n ! 1, and X n is a continuous square integrable .Ft /-martingale. Since
hX n i.t/ D hX nC1 i.t/ for t < n , we can define hX i.t/ by
hX i.t/ D hX n i.t/; 8t < n :
hX i WD .hX i.t/; t 2 Œ0; T / does not depend on the choice of n , and is called the
quadratic variational process of the local martingale X .
By Mc .Œ0; T ; .Ft // and Mcloc .Œ0; T ; .Ft //, we denote the set of continuous .Ft /-
martingales and continuous .Ft /-local martingales, respectively.
We put
M2c .Œ0; T ; .Ft // D fX 2 Mc .Œ0; T ; .Ft //I EjX.t/j2 < 1; 8t 2 Œ0; T g:
6. .Ft /-Wiener process

Let W D .W 1 ; : : : ; W d / be a continuous .Ft /-adapted d -dimensional process.
W is called a d -dimensional .Ft /-Wiener process, if W .0/ D 0 and W .t/ W .s/
is N.0; .t s/Id /-distributed and independent of Fs , for any s t, that is
Z
d jyj2
P .W .t/W .s/ 2 AjFs / D .2 .t s// 2 exp dy; 8A 2 B.Rd /:
A 2.t s/
The following properties hold:
(i) W i 2 M2c .Œ0; T ; .Ft //, i D 1; : : : ; d .
(ii) hW i ; W j i.t/ D ıij t, where ıij D Kronecker symbol.
(iii) Let X 2 .X 1 ; : : : ; X d / be a continuous .Ft /-adapted process with X.0/ D 0.
If X satisfies (i) and (ii), then X is an .Ft /-Wiener process.
(iv) FtW D FtWC , and F0W D trivial.
(v) Let us consider the time interval Œ0; 1/, instead of Œ0; T . Let be a bounded
(Ft /-stopping time and put FQ t D Ft C . Then .W .t C / W . /I t 0/ is also
an .FQ t /-Wiener process on Œ0; 1/, independent of F
(see [IW81], p. 74, [LS01], p. 192 for the proof).
7. Continuous Markov processes
(1) Let p./ be a transition probability function, namely p.; xI t; / is a probability
measure on B.Rd /, for any 0 t and x 2 Rd , and p.; I t; A/ is a Borel
1; x 2 A;
function on Rd , satisfying p.; xI ; A/ D
0; x … A:
Moreover, the Chapman–Kolmogorov equation
Z
p.; xI t; A/ D p.; xI s; dy/p.s; yI t; A/; 8A 2 B.Rd /
Rd
holds, for any s t and x 2 Rd .

(2) Definition 1.9. By a continuous Markov process with the transition probability
function p./, we mean a continuous .Ft /-adapted process X , with
(Markov property) P .X.t/ 2 AjFs / D p.s; X.s/I t; A/ P -a.s.;

8 0 s t and A 2 B.Rd /:
Namely, if the condition “X.s/ D x” is given then the probability distribution

of the future of the process .X.t/I t > s/ does not depend on its past .X./I <
s/. We sometimes denote p.s; xI t; A/ by Psx .X.t/ 2 A/, where the subscript
.s; x/ indicates the initial condition “X.s/ D x.”
Let us define a family of linear operator T t on B.Rd / (D bounded Borel
functions on Rd ), associated with the transition probability function p./, i.e.,
Z
T t .x/ D .y/p.; xI t; dy/ D E x .X.t// for t:
Rd
T t is called the transition operator of the Markov process. .

Noticing that T t 2 B.Rd / and Tt t D I, we see
Ts .Tst /.x/ D T t .x/ for s t;
by the Chapman–Kolmogorov equation.

(3) Backward evolution operator A
For a smooth function ˚, we set
1
A˚.; x/ D lim .E x ˚. C h; X. C h// ˚.; x//;
h!0 h
whenever the RHS exists.

The operator A is called the backward evolution operator. By definition, the
generator of T; C acts by G D lims!0 1s .T; Cs /. Hence
@˚
A˚.; x/ D G ˚.; /.x/ C .; x/:
@
When X is given by a stochastic differential equation, we can compute A by
using Itô’s formula (see (2.178)).
1.1.3 Itô Integrals

R
We next define the Itô integral ˚.s/ d W . Let W be a 1-dimensional .Ft /-Wiener
process. Since the paths of Wiener process are not differentiable, the integral cannot
be defined in the ordinary way.
Let L0 ..Ft // denote the set of .Ft /-progressively measurable real process and
8 n Z T o
ˆ
ˆ
< L .Œ0; T ˝; .Ft // D ˚ 2 L ..Ft //I E
2 0
j˚.s/j2 ds < 1 ;
Z T
0
ˆ 2 n o (1.11)
:̂ L.Ft / .Œ0; T / D ˚ 2 L ..Ft //I P
0
j˚.s/j2 ds < 1 D 1 :
0
RT
Setting k˚k2T D EŒ 0 j˚.s/j2 ds; L2 .Œ0; T ˝; .Ft // becomes a Hilbert space,
with the norm k kT . Rt
Let us define I.˚/.t; !/ D 0 ˚.s; !/ d W .s; !/ in three steps.
Step 1. ˚ is a simple process, say

f0 .!/; for s D 0;
˚.s; !/ D
fi .!/; for s 2 .ti ; ti C1 ; i D 0; : : : ; n:
where 0 D t0 < t1 < < tn < tnC1 D T , and fi is bounded and

Fti -measurable .i D 0; : : : ; n/. Define
X
n
I.˚/.t; !/ D fi .!/.W .ti C1 ^ t; !/ W .ti ^ t; !//
i D0
X
j1
D fi .!/.W .tiC1 ; !/W .ti ; !//Cfj .!/.W .t; !/W .tj ; !//;
iD0
for t 2 Œtj ; tj C1 :
Thus, I.˚/ 2 M2c .Œ0; T ; .Ft // and satisfies
EŒI.˚/.t/ D 0; EŒI.˚/.T /2 D k˚k2T ;
and
Z t
hI.˚/i.t/ D ˚.s/2 ds:
0
Step 2. For ˚ 2 L2 .Œ0; T ˝I .Ft //, there is a sequence of simple processes

.˚n ; n D 1; 2; : : : /, such that limn!1 k˚ ˚n kT D 0. Hence, as n; m ! 1
EŒjI.˚n /.T / I.˚m /.T /j2 D EŒjI.˚n ˚m /.T /j2 ! 0: (1.12)
From the Burkholder–Davis–Gundy inequality, it follows that there exists the

limit, limn!1 I.˚n /, in M2c .Œ0; T ; .Ft // and is unique (up to equivalence of
stochastic processes). Since this limit does not depend on the choice of ˚n , we
can define I.˚/ by
Z t
I.˚/.t/ D lim I.˚n /.t/ DW ˚.s/ d W .s/:
n!1 0
Consequently, we have
Z t
I.˚/ 2 M2c .Œ0; T ; .Ft // with hI.˚/i.t/ D ˚.s/2 ds: (1.13)
0
For any two .Ft /-stopping times and , with 0 , P -a.s., we put
Z t
˚.s/ d W .s/ D I.˚/. / I.˚/./: (1.14)


R
R / D 0 P -a.s.;
(i) E.R ˚.s/ d W .s/jF R
(ii) E. ˚.s/ d W .s/ .s/ d W .s/jF / D E. ˚.s/ .s/ dsjF / P -a.s.;
O !/ D ˚.s; !/.s / WD ˚.s; !/Œ0; .s/. Then
(iii) Put ˚.s;
Z t ^ Z t
˚.s/ d W .s/ D O
˚.s/ d W .s/; 8 t 2 Œ0; T :
0 0
Step 3. For ˚ 2 L2.Ft / .Œ0; T /, we define

Rt
infft < T I 0 ˚.s; !/ ng;
n .!/ D (1.15)
T; iff g D empty:
Then . n ; n D 1; 2; : : : / is a sequence of nondecreasing .Ft /-stopping times

satisfying P .limn n D T / D 1. Set ˚n .t; !/ D ˚.t; !/.t n /. Then ˚n 2
L2 .Œ0I T ˝; .Ft // and (iii) yields
Z t^ m
Z t
˚n .s/ d W .s/ D ˚.s/.s m /.s n / d W .s/
0 0
Z t
D ˚m .s/ d W .s/; for m < n:
0
By this consistency, the following is well-defined:

Z t Z t
˚.s/ d W .s/ WD ˚m .s/ d W .s/ for t m: (1.16)
0 0
R m ^
Since 0 ˚.s/ d W .s/ 2 M2c .Œ0; T ; .Ft //; m D 1; 2; : : : , the Itô integral
Rt
0 ˚.s/ d W .s/ is defined as a continuous .Ft /-local martingale (see [IW81],
pp. 45–22, [KS98], pp. 129–141, [LS01], pp. 92–112, for details).
Replacing real one-dimensional processes by m-dimensional ones, we define
L2 .Œ0; T ˝; .Ft /I Rm / and L2.Ft / .Œ0; T I Rm / in the same way as in (1.11).
R i process. For ˚ D .˚1 ; : : : ; ˚m / 2
i i i
Let W be an m-dimensional .Ft /-Wiener
L .Œ0; T ˝; .Ft /I R /, we define ˚ .s/ d W .s/ by
2 m
Z t m Z
X t
˚ i .s/ d W .s/ D ˚ji .s/ d W j .s/:
0 j D1 0
Definition 1.10. X is called a d -dimensional Itô process, if X admits the represen-

tation
Z t Z t
X.t/ D X0 C b.s/ ds C ˚.s/ d W .s/; 8t; P -a.s.; (1.17)
0 0
or the equivalent componentwise form

Z t m Z
X t
X .t/ D
i
X0i C b .s/ ds C
i
˚ji .s/ d W j .s/; i D 1; : : : ; d;
0 j D1 0
where
X0 is an F0 - measurable d -dimensional random variable,
b./ 2 L1 .Œ0; T ˝; .Ft /I Rd /,
R˚./ 2 L .Œ0; T ˝I R ˝ R /,
2 d m
b.s/ ds stands for the pathwise Lebesgue integral.

Sometimes we use for Itô process X the differential representation
dX.t/ D b.t/ dt C ˚.t/ d W .t/:
1.1.4 Itô’s Formula
Itô’s formula is one of the most important tools in stochastic calculus.

Let X be a d -dimensional Itô process, given by (1.17), and c./ be a bounded
.Ft /-progressively measurable real process.
Itô’s Formula
LetR F be a function of C 12 .Œ0; T Rd / and be an .Ft /-stopping time. Then

t
e 0 c.h/ dh F .t; X.t// is also a real Itô process and the following formula holds:
R t^
e 0 c.s/ ds F .t ^ ; X.t ^ // F .0; X0 /
Z t^ Rs n
D e 0 c.h/ dh @t F .s; X.s// C b.s/ @x F .s; X.s//
0
1 o
C tr a.s/@xx F .s; X.s// c.s/F .s; X.s// ds
2
Z t^ Rs
C e 0 c.h/ dh @x F .s; X.s//˚.s/ d W .s/; 8t; (1.18)
0
where a.t/ D ˚.s/˚.s/> , @x F D gradient of F , @xx F D Hessian of F

(see [IW81], pp. 66–73, [LS01], pp. 123–128 for the proof).
Roughly speaking, the formal expansion
Rt Rt
d.e 0 c ds
F .t; X.t/// D e 0 c ds
dF .t; X.t// c.t/F .t; X.t//
and the relation
@F X @F 1 X @2 F
d d
dF .t; X.t// D dt C dX i C dX i dX j C o.dt/
@t i D1
@xi 2 j D1
@xi @xj
lead to Itô’s formula, by using d W i .t/ d W j .t/ D ıij dt.

We finally state Itô’s formula with generalized derivatives.
Itô–Krylov Formula ([Kr09], 2.10, Theorem 1)
Suppose that
(a) ˚./ and b./ are bounded,
(b) a.t; !/ is uniformly positive definite, i.e., there is a positive constant 0 such
that
y > a.t; !/y 0 jyj2 ; 8y 2 Rd ; 8t!:
Let D be a bounded open set of Rd . D denotes the exit time of X from D.

Then, for any F 2 W 12 .Œ0; T D/ and any .Ft /-stopping time Q , (1.18)
holds when we replace with D ^ Q and take generalized derivatives, @F ; @F and
@t @xi
@2 F
@xi @xj
.i; j D 1; : : : ; d /.
Example 1.2 (Exponential martingales). Let us define q.t/ by
Z t Z
1 t
q.t/ D exp ˚.s/ d W .s/ j˚.s/j2 ds ; t 2 Œ0; T ;
0 2 0
where ˚ 2 L2 .Œ0; T ˝; .Ft /I Rm /.

1.2 Stochastic Differential Equations 15
Then q./ is positive, continuous, and .Ft /-adapted. Moreover,

(i) q./ is a continuous .Ft /-supermartingale with q.0/ D 1.
(ii) q./ is a continuous .Ft /-martingale if and only if Eq.T / D 1.
When q./ is a martingale, q./ is called an exponential .Ft /-martingale.
1.2 Stochastic Differential Equations
In this section, we will study stochastic differential equations (SDEs in short)

related to stochastic control problems. Hence we are interested in two kinds of
SDEs, those with random coefficients and those with deterministic Borel measurable
coefficients. In Sect. 1.2.1 we consider Lipschitz continuous SDEs with random
coefficient and prove the existence of a unique solution by using the successive
approximation method. Section 1.2.2 deals with the Girsanov transformations
of probability measures via exponential martingales, which provide a tool for
studying SDEs with Borel measurable drift terms. We also recall the Bayes formula
associated with Girsanov transformation. In Sect. 1.2.3 we overview the weak
solutions of SDEs with deterministic Borel coefficients [Kr09]. Section 1.2.4 deals
briefly with backward SDEs introduced by Pardoux and Peng [PP90].
1.2.1 Lipschitz Continuous SDEs with Random Coefficients
Let
b W Œ0; T Rd ˝ 7! Rd
and
˛ W Œ0; T Rd ˝ 7! Rd ˝ Rm
be measurable and, for any x 2 Rd ; b.; x; / and ˛.; x; / be .Ft /-progressively

measurable.
We assume the following two conditions:
.a1 / Equi-Lipschitz continuity
9l > 0, such that
jb.t; x; !/b.t; y; !/jCj˛.t; x; !/˛.t; y; !/j ljx yj; 8x; y; 8.t; !/:
.a2 / Linear growth

9K > 0, such that
jb.t; x; !/j C j˛.t; x; !/j K.l C jxj/; 8x; 8.t; !/:

Let 2 Œ0; T /. Consider the d -dimensional SDE
dX.t/ D b.t; X.t/; !/ dt C ˛.t; X.t/; !/ d W .t/; t > ; (1.19)
with the initial condition
X./ D X .2 Lp .˝; F I Rd / with p 1/: (1.20)
Although we customarily omit the probability parameter !, we put it in

coefficients b and ˛ in order to stress the randomness of the coefficients.
Definition 1.11. X D .X.t/I t 2 Œ; T / is called a solution of (1.19)–(1.20), if X
is a continuous and .Ft /-adapted d -dimensional process satisfying
Z t Z t
X.t/ D X C b.s; X.s/; !/ ds C ˛.s; X.s/; !/ d W .s/; 8t 2 Œ; T ; P -a.s.;

(1.21)
or, equivalently, the componentwise equation
Z t m Z
X t
X i .t/ D Xi C b i .s; X.s/; !/ ds C ˛ji .s; X.s/; !/ d W j .s/;
j D1
8t 2 Œ; T ; i D 1; : : : ; d; P -a.s.
We say that the solution X is unique if
O
P .X.t/ D X.t/; 8t 2 Œ; T / D 1 (1.22)
holds for any solution X and XO .

When we want to stress the initial condition, we denote X by X;X : b./ and
˛./ are called the drift coefficient and the diffusion coefficient, respectively.
By using the successive approximation method, we have
Theorem 1.2. Under conditions .a1 / and .a2 /, problem (1.19)–(1.20) has a unique
solution X . Moreover, the following properties hold:
(i) There is a constant Kp , which depends only on p and K, such that
h i
E sup jX.s/j2p Kp .1 C EjX j2p /; (1.23)
sT
h i
E sup jX.s/ X.t1 /j2p Kp .1 C EjX j2p /.t2 t1 /p
t1 st2
for any t1 < t2 < T : (1.24)

(ii) There is a constant KO p , which depends only on p; K and l, such that

h i
E sup jX.s/ XO .s/j2p KO p EjX XO j2p ; (1.25)
sT
where XO is the solution of (1.19) with the initial condition X./

O D XO , and
h i
E sup jX.s/ X .s/j2p KO p .1 C EjX j2p /. /p ; (1.26)
sT
where X is the solution of (1.19) with the initial condition X . / D X and

.
Example 1.3 (Linear SDE). Let us consider the SDE
dX.t/ D.B.t; !/X.t/ C b0 .t; !// dt

X
m
C .Aj .t; !/X.t/ C ˛j .t; !// d W j .t/; (1.27)
j D1
where B; Aj W Œ0; T ˝ 7! Rd ˝ Rd , b0 ; ˛j W Œ0; T ˝ 7! Rd , j D 1; : : : ; m

are bounded and .Ft /-progressively measurable, and W D .W 1 ; : : : ; W m / is an m-
dimensional .Ft /-Wiener process. Since .a1 / and .a2 / hold, SDE (1.27), together
with the initial condition X./ D x, admits a unique solution.
We will treat two cases.
(1) Linear Gaussian model
Suppose that B./; b0 ./ and ˛j ./ are deterministic and Aj ./ D 0 .j D
1; : : : ; m/. Consider the SDE

dX.t/ D .B.t/X.t/ C b0 .t// dt C ˛.t/ d W .t/; t 2 Œ0; T ;
(1.28)
X./ D x:
In order to seek a formula for X , we consider the following matrix-valued

differential equation:
8
ˆ
< dZ .t/ D B.t/Z.t/; t 2 .0; T ;
dt (1.29)
:̂ Z.0/ D I :
d
P Rt Rt
The solution is given by Z.t/ D 1 1
. 0 B.s/ ds/n D exp. 0 B.s/ ds/ and
Rt
nD0 nŠ
its inverse matrix Z 1 .t/ is exp. 0 B.s/ ds/.
Consequently, for the solution X of (1.28), we have

Z t Z t
1 1
X.t/ D Z.t/Z ./xCZ.t/ Z .s/b0 .s/ dsCZ.t/ Z 1 .s/˛.s/ d W .s/:

(1.30)
(2) 1-dimensional linear SDE
Let b 2 L1 .Œ0; T ˝; .Ft /I R1 / and ˛ 2 L2 .Œ0; T ˝; .Ft /I R1 ˝ Rm / be
given. Consider the linear SDE

dX.t/ D X.t/b.t; !/ dt C X.t/˛.t; !/ d W .t; !/; t 2 .0; T ;
(1.31)
X.0/ D x0 .> 0/:
Although neither .a1 / nor .a2 / hold, Itô’s formula yields

Z t Z th
1 i
X.t/ D x0 exp ˛.s; !/ d W .s/C b.s; !/ j˛.s; !/j2 ds (1.32)
0 0 2
is a unique solution (refer to [YZ99], Sections 3 and 4 in Chapter 6 for details
on (1.27)).
1.2.2 Girsanov Transformations
Let .˝; F ; .Ft /; P / be a filtered probability space. Here we are concerned with
transformations of probability distribution. So we write down the probability P
together with .Ft /.
Consider an exponential .Ft ; P /-martingale
Z t Z
1 t
q.t/ D exp ˚.s/ d W .s/ j˚.s/j2 ds ; t 2 Œ0; T ; (1.33)
0 2 0
with ˚ 2 L2 .Œ0; T ; .Ft /I Rm /.

Since q.t/ > 0 P -a.s. and Eq.T / D 1, we can define a new probability PO on
FT through the Radon–Nykodým derivative q.T /, namely
PO .^/ D EŒq.T /^ ; 8^ 2 FT : (1.34)
We denote PO by q ı P and call the transformation, P 7! q ı P Girsanov

transformation.
From the martingale property of q./, we deduce the consistency property
PO . / D EŒq.t/ for 2 Ft .
The following proposition deals with the conditional expectation.
Proposition 1.3 (Bayes formula). Let Z be FT -measurable positive random
variable with EZ D 1. Define a new probability Q by
Q. / D EŒZ ; 8 2 FT : (1.35)
Let be Q-integrable and Ft -measurable. Then, for s < t,
E Q .jFs / D E.E.ZjFt /jFs /=E.ZjFs / Q-a.s.; (1.36)
where E Q denotes the expectation w.r.t. Q.

In particular, for PO D q ı P ,
O
E P .jFs / D E.q.t/jFs /=q.s/ PO -a.s. (1.37)
for any Ft -measurable random variable with E PO jj < 1.

Proof. Put Zt D E.ZjFt / > 0. For 2 Fs .s < t/, we have
E Q Œ D EŒZ
D EŒE.ZjFt /
h E.Z jF / i
t s
D E Zs
Zs
h E.Zt jFs / i
D E E.ZjFs /
Zs
h E.Z jF / i E.Zt jFs /
t s
DE Z since is Fs -measurable
Zs Zs
h E.Z jF / i
t s
D EQ ; (1.38)
Zs
which yields (1.36). t
u
Now we state a basic theorem.
Theorem 1.3 (Girsanov’s Theorem). Let W be an m-dimensional .Ft ; P /-
Wiener process and ˚ 2 L2 .Œ0; T ˝; .Ft /I Rm /. Suppose that q.t/ given by
(1.33) is an exponential .Ft ; P /-martingale. Then WO .t/ D .WO 1 .t/; : : : ; WO m .t//,
given by
Z t
O
W .t/ D W .t/
i i
˚ i .s/ ds; i D 1; ; m (1.39)
0
is a .Ft ; q ı P /-Wiener process.

For the proof, we show that WO i .t/; WO i .t/2 t; WO i .t/WO j .t/ .i ¤ j /; i D
1; : : : ; m are all .Ft ; q ı P /-martingales, by using the Itô and Bayes formulas (see
[LS01], 6.3 for details).
We will give useful two criterions, under that q./ becomes a martingale.
Proposition 1.4 (Special case of Novikov’s criterion). If

1 Z T
E exp j˚.s/j2 ds < 1; (1.40)
2 0
then q./ is a martingale.

(see [IW81], p. 142, [LS01], p. 229 for details).
Proposition 1.5. Let X be a continuous .Ft /-adapted process. Suppose that X and
the .Ft ; P /-Wiener process W satisfy
Z t Z t
X.t/ D X0 C b.s; X.s/; !/ ds C ˛.s; X.s/; !/ d W .s/; (1.41)
0 0
where EjX0 j2 < 1 and b.; x; / and ˛.; x; / are .Ft /-progressively measurable
processes satisfying condition .a2 /. Assume that ˛ is bounded.
Let W Œ0; T / Rd ˝ 7! Rm be a measurable map such that, for any
x 2 R ; .; x; / is .Ft /-progressively measurable. Further assume that satisfies
d
.a2 /. Define M.t/ by

Z t
1
Z t
M.t/ D exp .s; X.s/; !/ d W .s/ j .s; X.s/; !/j2 ds ; t 2 Œ0; T :
0 2 0
(1.42)
Then, M./ is an exponential .Ft ; P /-martingale.
Proof. Since we can apply the same arguments as in [Be92], p. 77, to the random
coefficients case, we will sketch the proof. First we claim that there is a constant c0
such that
EŒM.t/jX.t/j2 c0 ; 8t 2 Œ0; T : (1.43)

.t /
Indeed, for " > 0, we compute 1C".t /
.D 1
"
1
".1C".t //
/, where .t/ D
M.t/jX.t/j2 . 0/.
Since Itô’s formula yields
.t/ 1
d D M.t/fjX.t/j2 C 2X.t/˛g d W .t/
1 C ".t/ .1 C ".t//2
1
C M.t/f2X.t/.b C ˛ / C tr.˛˛ > /g dt
.1 C ".t//2
"
d hi.t/ (1.44)
.1 C ".t//3
and
.t/
2nd term in the RHS c1 C M.t/ (1.45)
1 C ".t/
with a constant c1 independent of " and M.t/ a supermartingale, we have

Z
.t ^ n / t
.s ^ n /
E EjX0 j2 C c1 E ds C c1 T (1.46)
1 C ".t ^ n / 0 1 C ".s ^ n /
where n is the exit time of M./ from Œ0; n.

Thus, Gronwall’s inequality and the bounded convergence theorem show that
there is a constant c0 independent of ", such that
.t/
E c0 ; 8t: (1.47)
1 C ".t/
Now letting " # 0 leads to (1.43), by the monotone convergence

theorem.
M.T /
Next, using the same arguments, we evaluate E 1C"M.T / . By Itô’s formula,
Z Z
M.t/ 1 t
dM.s/ t
d hM i.s/
D C " :
1 C "M.t/ 1C" 0 .1 C "M.s//2 0 .1 C "M.s//3
Now take the expectation of both sites and apply (1.43) to the 3rd term of RHS.
Letting here " ! 0 yields EM.t/ D 1, from which the proposition follows thanks
to Example 1.2. t
u
Example 1.4 (Transformation of the drift term). Let W be a d -dimensional
.Ft ; P /-Wiener process and b W Œ0; T Rd 7! Rd a Borel function satisfying
jb.t; x/j K.1 C jxj/; 8t; x: (1.48)
We consider the SDE;
dX.t/ D b.t; X.t// dt C d W .t/; t 2 .; T ; (1.49)
X./ D X .2 L2 .˝; F ; P //:
We hardly expect a solution adapted to .W; X /. However, we can construct a

Wiener process WO and a continuous .Ft /-adapted process on a probability space
.˝; F ; .Ft /; PO /, so that (1.49) holds for .; WO /.
Indeed, let us consider the auxiliary SDE

(
d .t/ D d W .t/; t 2 .; T /;
(1.50)
./ D X :
Here, .t/ D X C .W .t/ W .//.

Define ˚ 2 L2 .Œ0; T ˝; .Ft /I Rd / and q./ by
(
b.s; .s//; s;
˚.s/ D (1.51)
0; s<
and
Z t Z
1 t
q.t/ D exp ˚.s/ d W .s/ j˚.s/j2 ds
0 2 0
8 Z t Z
< 1 t
exp b.s; .s// d W .s/ jb.s; .s//j2 ds ; t;
D 2 (1.52)
:
1; t < ;
respectively. Then q./ is a .Ft ; P /-martingale by Proposition 1.5. Thus Girsanov’s

theorem shows that
Z t
WO .t/ D W .t/ ˚.s/ ds (1.53)
0
is a .Ft ; PO /-Wiener process, where PO D q ı P . By (1.50) and (1.53), .; WO / is a

weak solution of (1.49) (see Definition 1.12).
From (1.49) it follows that
.WQ .s/ WQ ./I s 2 Œ; T / ..s/I s 2 Œ; T /: (1.54)
1.2.3 SDEs with Deterministic Borel Coefficients
In stochastic control problems, we encounter SDEs with Borel measurable coeffi-

cients. Here we will overview such equations.
Let b W Œ0; 1/Rd 7! Rd , and ˛ W Œ0; 1/Rd 7! Rd ˝Rm be Borel measurable.
Let us consider the following SDE, called of Markovian type:
dX.t/ D b.t; X.t// dt C ˛.t; X.t// d W .t/; t > ; (1.55)

X./ D X .2 L2 .˝; F I Rd //: (1.56)
Firstly we assume .a1 / and .a2 /, i.e., we have an SDE with Lipschitz continuous
coefficients. By Theorem 1.2, the unique solution X;X .t/ depends only on .; X /
and .W .s/ W ./; s 2 Œ; t/ (this solution is called a strong solution). Put
p.; xI t; A/ D P .X x .t/ 2 A/; A 2 B.Rd /; < t: (1.57)
Since X x .t/ D Xs;X;x .s/ .t/; s t; p.; xI t; A/ satisfies the Chapman–

Kolmogorov equation;
p.; xI t; A/ D E x ŒA .X.t//

D E x ŒE.A .Xs;X.s/ .t//jFs /
D E x Œp.s; X.s/I t; A/
Z
D p.; xI s; dy/p.s; yI t; A/: (1.58)
Rd
Namely, the solution is a Markov process with the transition probability function
p./.
For the transition operator T t .x/ D E x .X.t//, Itô’s formula gives the
expression for the backward evolution operator A. For ˚ 2 C 12 .Œ0; 1/ Rd /,
A˚.; x/ D lim .E x ˚. C s; X. C s// ˚.; x//=s

s!0
D @ ˚.; x/ C b.; x/ @x ˚.; x/

1
C tr.a.; x/@xx ˚.; x//; (1.59)
2
where a D ˛˛ > .
When we drop the continuity condition .a1 / and .a2 /, we can hardly expect the
existence of strong solutions. This leads us to the definition of the notion of weak
solution.
Definition 1.12 (Weak solution). Suppose that XQ and WQ are given on a filtered
Q FQ ; .FQ t /; PQ /. .XQ ; WQ / is called a weak solution of (1.55)–
probability space .˝;
(1.56), if
Q
(a) X./ is a d -dimensional random variable, having the same probability distri-
bution as X ,
(b) WQ is an m-dimensional .FQ t ; PQ /-Wiener process,
(c) XQ is a d -dimensional Q
R t continuous .FtR/-adapted process,
Q Q
(d) X .t/ D X./ C b.s; X .s// ds C ˛.s; XQ .s// d WQ .s/, t, PQ -a.s.
Q t
Sometimes XQ itself or the probability distribution of XQ are called the weak

solution of (1.55)–(1.56).
By uniqueness of the weak solution we mean uniqueness in law of XQ , namely,
if .XQ ; WQ / and .X ; W / are weak solutions, then the probability distributions of XQ
and X coincide.
We have already considered a weak solution in Example 1.4, as an application of
the Girsanov transformation.
Let us state an existence and uniqueness results for weak solutions.
Proposition 1.6 ([Kr09], 2–6, Theorem 1). Suppose that d D m and the coef-
ficients ˛./ and b./ are bounded Borel functions. Further assume that ˛./ is
symmetric and uniformly parabolic, i.e., there exists 0 > 0, such that
y > ˛.t; x/y 0 jyj2 ; 8y 2 Rd ; 8t; x: (1.60)
Then, the SDE (1.55)–(1.56) admits a weak solution provided EjX j4 < 1.
Proposition 1.7 ([SV79], Theorem 7.2.1). Let b W Œ0; 1/ Rd 7! Rd and ˛ W
Œ0; 1/ Rd 7! Rd ˝ Rm be bounded and Borel measurable. Suppose that ˛.t; x/ is
uniformly continuous w.r.t. x, uniformly in t and ˛˛ > is uniformly parabolic. Then
the SDE (1.55)–(1.56) admits the unique weak solution.
1.2.4 Backward Stochastic Differential Equations
In this subsection, we will overview the terminal value problems for SDEs, called
backward SDE, introduced by Pardoux and Peng [PP90].
Let W be a d -dimensional Wiener process defined on .˝; F ; .FtW /; P /. Let h W
Œ0; T R1 Rd ˝ 7! R1 be measurable and satisfy the conditions
.a1 / h.; y; z; / 2 L2 .Œ0; T ˝; .FtW /I R1 /; 8y; z
and
.a2 / 9l > 0 such that 8t; !,
jh.t; y1 ; z1 ; !/ h.t; y2 ; z2 ; !/j l.jy1 y2 j C jz1 z2 j/

8y1 ; y2 2 R1 ; z1 ; z2 2 Rd : (1.61)
Consider the real SDE
d Y .t/ D h.t; Y .t/; Z.t/; !/ dt C Z.t/ d W .t/; t 2 Œ0; T / (1.62)
with the lateral boundary condition
Y .T / D .2 L2 .˝; FTW I R1 //: (1.63)
The SDE (1.62)–(1.63) is called a backward SDE on Œ0; T .

Let Y and Z be .FtW /-progressively measurable real and d -dimensional pro-

cesses respectively.
Definition 1.13. .Y; Z/ is called a solution of (1.62)–(1.63), if
(a) Y is continuous, RT
(b) EŒsup0t T jY .t/j2 C 0 jZ.t/j2 dt < 1
RT RT
(c) Y .t/ D t h.s; Y .s/; Z.s/; !/ ds t Z.s/ d W .s/; 8t 2 Œ0; T ; P -a.s.
We identify two solution .Y1 ; Z1 / and .Y2 ; Z2 / whenever
Y1 .t/ D Y2 .t/; 8t 2 Œ0; T ;

Z1 .t/ D Z2 .t/ for almost all t 2 Œ0; T ; P -a.s. (1.64)
Now we have;
Theorem 1.4. Under the conditions .a1 / and .a2 / , problem (1.62)–(1.63) admits
a unique solution.
(see [ElKPQ97] and [YZ99], Chapter 7 for details).
Example 1.5 (Backward SDE associated with a quasi-linear parabolic equation).
Let F W Œ0; T Rd R1 Rd 7! R1 be a continuous function satisfying
jF .t; x; y1 ; z1 / F .t; x; y2 ; z2 /j l.jy1 y2 j C jz1 z2 j/; 8t; x; (1.65)
and
jF .t; x; 0; 0/j K.1 C jxjp /; 8t; (1.66)
with positive constants l, K and p.

Let be the diffusion process given by the SDE;
(
d .t/ D b.t; .t// dt C ˛.t; .t// d W .t/; t 2 .0; T /;
(1.67)
.0/ D c .2 Rd /
where coefficients b and ˛ are deterministic and satisfy .a1 / and .a2 / with d D m.
Now consider a backward SDE
(
d Y .t/ D F .t; .t/; Y .t/; Z.t// dt C Z.t/ d W .t/; t 2 Œ0; T /;
(1.68)
Y .T / D g..T //;
with a continuous function g satisfying a linear growth condition.

The following quasi-linear parabolic equation is related to (1.68):
@t v.t; x/ C Gt v.t; x/ F .t; x; v.t; x/; ˛.t; x/> @x v.t; x/> / D 0 on Œ0; T / Rd ;
(1.69)
v.T; x/ D g.x/; x 2 Rd ; (1.70)
where Gt is the generator of transition semigroup of .

We have
Proposition 1.8. If the Cauchy problem (1.69)–(1.70) admits a classical solution
v./, then
(
Y .t/ D v.t; .t//;
(1.71)
Z.t/ D ˛.t; .t//@x v.t; .t//> ;
RT
whenever EŒsupt jY .t/j2 C 0 jZ.t/j2 dt < 1.
Indeed, applying Itô’s formula to v.t; .t// and using (1.69), we obtain (1.71).
1.3 Asset Pricing Problems
Let us consider a contract to sell a contingent claim with payoff G and maturity T ,
at time 0. The problem is to price this contingent claims. Suppose that the agent
invests his own money in a bond and stocks whose prices evolve according to SDEs
and does not want to run any risk of losing money. The lowest price satisfying this
condition is called the selling price (refer [ElKQ95, ElKPQ97]).
In Sect. 1.3.1, we will formulate the problem and, by using the dynamics of
bond and stocks, we study the selling price and portfolio strategies in Sects. 1.3.2
and 1.3.3.
1.3.1 Formulation
Let us consider a financial market consisting of one bond (riskless asset) and d
stocks (risky assets). We suppose that the price of the bond is given by
S 0 .t/ D e rt ; t 0; (1.72)
where r is a positive constant, and the price of i -th stock evolves according to the
SDE
1.3 Asset Pricing Problems 27
8
< dS i .t/ D S i .t/.i .t; S.t// dt C X i .t; S.t// d W j .t//;
d
ˆ
j t > 0;
(1.73)
j D1
:̂
S i .0/ D s i > 0 .i D 1; : : : ; d /
where W .t/ D .W 1 .t/; : : : ; W d .t// is a d -dimensional Wiener process, defined on

.˝; F ; .FtW /; P /, ./ D .ji .//i;j D1;:::;d and ./ D .1 ./; : : : ; d .// are called
the volatility and the mean return rate, respectively.
Let X.t/ denote the agent’s wealth at time t. The P
agent invests the number of
d i .t /S i .t /
X.t /
shares i .t/ in the i -th stock at t and 0 .t/ WD i D1
S 0 .t /
in the bond. Now
we admit .t/ 2 R .i D 1; : : : ; d / where .t/ < 0 means selling i -th stock and
i 1 i
0
.t/ < 0 means borrowing money. Hence X evolves according to the SDE
dX.t/ D .X.t/ .t/ S.t//rdt C .t/ dS.t/; t > 0; (1.74)
X.0/ D x .> 0/: (1.75)
Let us consider the contract between the agent and a quest at time 0, such that the
agent pays G .2 L2 .˝; FtW I Œ0; 1/// to the quest at the maturity T . The problem
is what is the price of G at 0. .t/I Œ0; T ˝ 7! Rm is called a portfolio strategy,
./
when ./ S./ 2 L2 .Œ0; T ˝; .FtW /I R1 /. By Xx , we denote the solution of
(1.74)–(1.75). Since the agent does not want to run any risk of losing money, he
chooses a portfolio strategy ./ such that
Xx ./ .T / G; P -a.s. (1.76)
Thus the problem is to seek x G , defined by
x G D inffx > 0; 9 ./ such that (1.76) holdsg: (1.77)
x G is called the selling price of G at t D 0.
1.3.2 Backward SDE for the Selling Price
We are going to study the selling price by using a backward SDE. We assume that
the following conditions are satisfied:
(a) W Œ0; T Rd 7! Rd ˝ Rd , W Œ0; T Rd 7! Rd are bounded and
uniformly Lipschitz continuous w.r.t. x,
(b) .t; x/ is invertible and .t; x/1 is bounded.
Then (1.73) has the unique strong solutions S.t/ D .S 1 .t/; : : : ; S d .t// and
Z t
S .t/ D s exp
i i
i .; S.// d W ./
0
Z t
1
C i .; S.// j i .; S.//j2 ds > 0: (1.78)
0 2
Further (a) and (1.78) yield

h i
E sup jS.s/j2p < 1; for any p 1;
0sT
and (b) yields
FtW D FtS ; 8t 2 Œ0; T : (1.79)
Assume that G is given by
G D g.S.T //; (1.80)
with a linearly growing continuous function g on RdC .

When ./ is applied, the wealth evolves according to (1.74), namely
dX.t/ D .rX.t/ C .t/ ..t; S.t// r1d // dt

C .t/> .t; S.t// d W .t/ (1.81)
with a given initial value, where i .t/ D i .t/S i .t/, i D 1; : : : ; d and 1d D

.1; : : : ; 1/ 2 Rd .
./
Since Xx .T / is strictly increasing in x, we will seek an initial value xO such
O ./
that XxO .T / D g.S.T // with some O ./. The SDE (1.81) leads to the following
backward SDE
dX.t/ D .rX.t/ C .t/ ..t; S.t// r1d //dt

C .t/> .t; S.t// d W .t/; t 2 Œ0; T / (1.82)
X.T / D g.S.T //: (1.83)
Noticing that the backward SDE

1.3 Asset Pricing Problems 29
8
ˆ
ˆ d Y .t/ D .rY .t/ C Z.t/> .t; S.t//1 ..t; S.t// r1d //dt
<
C Z.t/> d W .t/; t 2 Œ0; T /; (1.84)
ˆ
:̂
Y .T / D g.S.T //
has the unique solution .Y; Z/ by Theorem 1.4, and comparing (1.82) with (1.84),
we conclude that x G D Y .0/ and
X
d
..t; S.t//1 /i Z j .t/=S i .t/;
j
O i .t/ D i D 1; : : : ; d; (1.85)
j D1
provides a claimed portfolio strategy.

We note that Y .t/ gives the selling price at time t.
1.3.3 Parabolic Equation Associated with (1.84)
In order to get an explicit formula for x G and O ./, we study the Cauchy problem
for a parabolic equation related to (1.84).
Using Proposition 1.8 and S i .t/ > 0; i D 1; : : : ; d , we have
8
ˆ
ˆ 1 X ij
d X d
ˆ
ˆ C i j
C x i @i v rv D 0;
ˆ
< @t v.t; x/ a .t; x/x x @ij v r
2 i;j D1 i D1
(1.86)
ˆ
ˆ .t; x/ 2 Œ0; T / RdC ;
ˆ
ˆ
:̂
v.T; x/ D g.x/; x 2 RdC
where a D > .
If (1.86) admits a classical solution, then we have
(
v.0; S.0// D x G ;
(1.87)
@i v.t; S.t// D O i .t/; i D 1; : : : ; d:
Since V .t/ WD v.t; S.t// satisfies (1.82)–(1.83), V .t/ provides the selling price at
time t. Accordingly V ./ is called the selling price process. Applying Itô’s formula
to v./, we have
v.t; x/ D Etx Œe r.T t / g..T //;
where .t/ D . 1 .t/; : : : ; d .t// is the diffusion process given by
d i .t/ D i .t/.rdt C i .t; .t// d W .t//: (1.88)

Example 1.6 (Constant coefficient markets). Let ./ and ./ be constants. Then
(1.86) reduces to
8
< @ v C 1 X aij x j x j @ v C r X x i @ v rv D 0;
d d
ˆ
t ij i on Œ0; T / RdC ;
2 ij D1 i D1
:̂
v.T; x/ D g.x/; x 2 RdC :
(1.89)
Referring to (1.88), we have
!
X
d 1
.t/ D ./ exp
i i
ji .W i .t/ W .// C r j i j2 .t /
j
(1.90)
j D1
2
and
Z
r.T t /
v.t; x/ D e g..T t; x; z//pd .T t; z/ d z; (1.91)
Rd

where pd .s; z/ D .2 s/ 2 exp jzj
d 2
2s
and D . 1 ; : : : ; d / with i .s; x; y/ D
x i exp.y i C .r 12 j i j2 /s/, i D 1; : : : ; d . In particular, for d D 1,
Z p
1
2
v.t; x/ D r.T t / g x exp T t z C r .T t/ p1 .1; z/ d z
1 2
(1.92)
holds. Suppose
g.x/ D .x K/ _ 0; (1.93)
with a positive constant K.

Then we can calculate (1.92) explicitly and obtain
Z 1 p
1
v.t; x/ D x exp T t z 2 .T t/ Ke r.T t / p1 .1; z/ d z
2
Z C Z
Dx p1 .1; z/ d z Ke r.T t / p1 .1; z/ d z; (1.94)
1 1
where
n x 1 o p
˙ D log C r ˙ 2 .T t/ = T t :
K 2
Equation (1.94) is the famous Black–Scholes formula [BS73].
Chapter 2
Optimal Control for Diffusion Processes
Abstract This chapter deals with completely observable stochastic control

problems for diffusion processes, described by SDEs. The decision maker chooses
an optimal decision among all possible ones to achieve the goal. Namely, for a
control process, its response evolves according to a (controlled) SDE and the payoff
on a finite time interval is given. The controller wants to minimize (or maximize)
the payoff by choosing an appropriate control process from among all possible
ones. Here we consider three types of control processes:
1. .Ft /-progressively measurable processes.
2. Brownian-adapted processes.
3. Feedback controls.
In order to analyze the problems, we mainly use the dynamic programming principle
(DPP) for the value function.
The reminder of this chapter is organized as follows. Section 2.1 presents
the formulation of control problems and basic properties of value functions, as
preliminaries for later sections. Section 2.2 focuses on DPP. Although DPP is known
as a two stage optimization method, we will formulate DPP by using a semigroup
and characterize the value function via the semigroup. In Sect. 2.3, we deal with
verification theorems, which give recipes for finding optimal Markovian policies.
Section 2.4 considers a class of Merton-type optimal investment models, as an
application of previous results.
2.1 Introduction
This section is devoted to formulating the time horizon stochastic control and
analyze basic notions. We introduce control processes, payoffs and value functions
in Sect. 2.1.1, and investigate their properties in Sect. 2.1.2.
Before we formulate the stochastic control problem, we give a typical example,
called the linear quadratic control.
Example 2.1 (Linear quadratic (LQ) control). Consider a d -dimensional stochastic
system with an external random force ./,
dX.t/ D .b.t/X.t/ C .t// dt C d W .t/;

32 2 Optimal Control for Diffusion Processes
where W is a d -dimensional .Ft /-Wiener process, b W Œ0; T 7! Rd ˝ Rd and

W Œ0; T ˝ 7! Rd are .Ft /-progressively measurable.
Let SC d
denote the set of d d non-negative definite matrices. Suppose that
M; N W Œ0; T 7! SC d
are given. The problem is to choose ./ so that the payoff:
hZ T i
J.; xI .// D E x .X.t/> M.t/X.t/ C .t/> N.t/.t// dt C jX.T /j2

is minimized and calculate the value function;
v.; x/ D inf J.; xI .//:

./
In particular, when d D 1; M D N D b D 0 and j./j 1 are assumed, the

choice O .t/ WD sgnX.t/ is optimal (see Examples 2.4 and 2.5).
2.1.1 Formulations
We are going to formulate the finite time horizon stochastic control problem.
Let .˝; F ; .Ft /; P / be a filtered probability space and W an m-dimensional
.Ft /-Wiener process. We call .˝; F ; .Ft /; P; W / a reference probability system.
Let T > 0 be given. A -compact and convex subset of Rq is called a control
region, where q is a positive integer.
Definition 2.1. The -valued .Ft /-progressively measurable process ..t/,
t 2 Œ0; T / is called a control process, if ./ 2 L1 .Œ0; T ˝; .Ft /I /, namely,
there is a compact set ./ . /, such that .t; !/ 2 ./ , for almost all .t; !/.
The 6-tuple A D .˝; F ; .Ft /; P; W; .// is called an admissible control.
We denote by A the set of all admissible controls.
Let b W Œ0; T Rd 7! Rd and ˛ W Œ0; T Rd 7! Rd ˝ Rm be given
and satisfy the following condition:
.b1 /
jb.t1 ; x1 ; 1 / b.t2 ; x2 ; 2 /j C j˛.t1 ; x1 ; 1 / ˛.t2 ; x2 ; 2 /j

ljx1 x2 j C m.jt
N 1 t2 j C j1 2 j/ (2.1)
and
jb.t; 0; /j C j˛.t; 0; /j K; 8.t; / 2 Œ0; T ; (2.2)
N is a bounded modulus function,

where l and K are positive constants and m./
N MN .
say m./
2.1 Introduction 33
When A.2 A/ is applied, the stochastic system evolves according to the SDE
(more precisely, controlled SDE)
dX.t/ D b.t; X.t/; .t// dt C ˛.t; X.t/; .t// d W .t/; .0 / < t T;
(2.3)
X./ D x.2 Rd /: (2.4)
By Theorem 1.2, there exists a unique solution of SDE (2.3)–(2.4), denoted by

XAx . This solution is sometimes called the response for A. XAx .t/ is clearly
..s/; W .s 0 / W ./; s; s 0 2 Œ; t/-measurable.
We omit the indices , x and A, when no confusion occurs.
Let us introduce three functions f; and with the conditions .b2 /–.b5 /.
.b2 / f W Œ0; T Rd 7! R1 is continuous and f .t; x; / is continuous w.r.t.
x, uniformly in .t; /,
.b3 / W Rd 7! R1 is continuous,
.b4 / jf .t; x; /j C j.x/j k.1 O C jxj2 /; 8t; x; with a constant k,
O
.b5 / W Œ0; T R 7! Œ0; c0 , satisfies .b2 /.
d
f and are called the running cost and the terminal cost, respectively, and is
the discount rate.
For the response X D XAx , the cost function on time interval Œs; t . Œ; T / is
given by
C.t; s; AI /
Z t Z
D expf .h; X.h/; .h// dhgf .; X./; .// d
s s
Z t
C expf .h; X.h/; .h// dhg.X.t//: (2.5)
s
When the response X stops at time t, we define the payoff (or criterion) by
J.t; ; x; AI / D E x C.t; ; AI /; (2.6)
where .; x/ refer to the initial condition of X . Clearly J.t; ; x; AI / depends on

the joint probability of ..s/; W .s 0 / W ./I s; s 0 2 Œ; t/, but not on A itself.
We want to minimize (or maximize) the payoff, by choosing an appropriate
admissible control.
Definition 2.2. v./, defined by
v.t; ; xI / D inf J.t; ; x; AI / .or sup J.t; ; x; AI // (2.7)

A2A A2A
is called the value function. If A 2 A gives the infimum (or supremum) of the RHS,
A is called an optimal control.
Thus we are concerned with the characterization of the value function and an
optimal control.
Let us introduce different classes of admissible controls: Brownian adapted
controls and feedback controls.
1. Let A D .˝; F ; .Ft /; P; W; .// be an admissible control. A is called
Brownian adapted, if ./ is .FtW /-progressively measurable. AW denotes
the set of all Brownian adapted controls. For a fixed reference probability
system .˝; F ; .Ft /; P; W /, W denotes the set of all .FtW /-progressively
measurable control processes. Since the payoff is calculated in terms of the joint
probability distribution of .W ./; .//, for any given a reference probability
system .˝; F ; .Ft /; P; W /, it holds that
inf J.t; ; x; AI / D inf J.t; ; x; AI /

A2AW ./2 W
2. We control a system by using the data on the system, in the customary manners.
A -valued Borel function O ./, defined on Œ0; T Rd , is called a Markovian
policy, if it is bounded. When we apply O ./, the system evolves according to the
SDE
dX.t/ D b.t; X.t/; .t;

O X.t/// dt C ˛.t; X.t/; O .t; X.t/// d W .t/: (2.8)
We can hardly expect that a strong solution will exist for (2.8), but a weak
solution does exist under mild conditions (see Sect. 1.3.3). Hence there exist
W and X , on an appropriate filtered probability space .˝ ; F ; .Ft /; P /,
such that (2.8) holds. Putting .s/ D O .s; X .s//, we have A D
.˝ ; F ; .Ft /; P ; W ; .// 2 A, but not in AW . We call A an admissible
control associated with the Markovian policy O ./.
2.1.2 Value Functions: Basic Properties
Recalling Theorem 1.2, we first list some basic properties of responses.

Proposition 2.1. Let p 1 be given. Then there exists a constant Kp .> 0/, such
that the following estimates hold:
(i) For any .; x/ 2 Œ0; T Rd ,
h i
E x sup jX A .t/j2p Kp .1 C jxj2p /; 8A 2 A: (2.9)
t T
(ii) For any .; x/ 2 Œ0; T Rd and t1 < t2 T ,

2.1 Introduction 35
h i
E x sup jX A .s/ X A .t1 /j2p Kp .1 C jxj2p /.t2 t1 /p ; 8A 2 A:
t1 st2
(2.10)
(iii) For any x; y 2 Rd and 2 Œ0; T ,
h i
E sup jXAx .s/ Xy
A
.s/j2 K1 jx yj2 ; 8A 2 A: (2.11)
sT
(iv) For 0 1 < 2 T and x 2 Rd ,

h i
E sup jXA1 x .s/ XA2 x .s/j2 K1 .1 C jxj2 /.2 1 /; 8A 2 A:
2 sT
(2.12)
Regarding the dependence on the control process, we have;
Proposition 2.2. There exists a constant K0 such that, for any controls A D
.˝; F ; .Ft /; P; W; .// and AO D .˝; F ; .Ft /; P; W; O .//,
h i Z t
O
E x sup jX A .s/ X A .s/j2 K0 N
EŒm.j.s/ O .s/j/2 ds
st
n E1.s/ O .s/12 t o
K0 a2 .t / C MN 2 ;
.mN 1 .a//2
8a > 0; 80 t T; x 2 Rd
(2.13)
where MN is a bound of m./,

N and 1./ O ./1 t D k./ O ./kL2 Œ;t . We put
1 1 D 1 1T .
Proof. Set
O O
Z.t/ D X A .t/ X A .t/; 4b.t/ D b.t; X A .t/; .t// b.t; X A .t/; O .t//;
and similarly for ˛./. Then

Z t ˇZ s ˇ
ˇ ˇ
sup jZ.s/j j4b.s/j ds C sup ˇ 4˛.h/ d W .h/ˇ
st st
holds by (2.3). Therefore, the Burkholder–Davis–Gundy inequality yields

hZ t Z t i
.t/ C .s/ ds C N
EŒm.j.s/ O .s/j/2 ds (2.14)

with a constant C > 0, where

h i
.t/ D E sup jZ.s/j2 :
st
Now Gronwall’s inequality and (2.14) lead to the left inequality in (2.13).
Now using the estimation
N
E m.jj/ 2
a2 C EŒm.jj/
N 2
N 1 .a/
I jj > m
a2 C MN 2 P .jj > m
N 1 .a//
Ejj2
a2 C MN 2 ;
N 1 .a//2
.m
we obtain the right inequality in (2.13). This completes the proof. t

u
Hence, when A; An ; n D 1; 2; : : : have the same reference probability system,
we obtain
Corollary 2.1. The response depends on its control process continuously in
L2 .Œ0; T ˝/. Further, if
lim E1n ./ ./12 D 0;

n!1
then we can choose a subsequence n0 such that, for any .; x/ 2 Œ0; T Rd ,
A 0
lim sup jX xn .s/ XAx .s/j D 0 P -a.s.
n0 !1 sT
and
h i
lim E x
0
sup jX An0 .s/ X A .s/j2 D 0:
n !1 sT
Next we discuss continuity properties of the payoff and the value function,
replying on Propositions 2.1 and 2.2.
Theorem 2.1. Suppose .b1 /–.b5 / hold. Then J.t; ; x; AI / and v.t; ; xI / have
the following properties:
(i) There is a constant KN such that, for any 0 < t T and x 2 Rd ,
N C jxj2 /;
jJ.t; ; x; AI /j K.1 8A 2 A;
N C jxj2 /:
jv.t; ; xI /j K.1 (2.15)
2.1 Introduction 37
(ii) Continuous dependence on initial conditions

Let R and ".> 0/ be given. Then there is a constant ı"R > 0 such that, for any
x1 ; x2 2 SR and 0 1 < 2 < t T ,
jJ.t; 1 ; x1 ; AI / J.t; 2 ; x2 ; AI /j < "; 8A 2 A; (2.16)
whenever jx1 x2 j C j1 2 j < ı"R , and
jv.t; 1 ; x1 I / v.t; 2 ; x2 I /j < "
whenever jx1 x2 j C jt1 t2 j < ı"R .

(iii) Continuous dependence on the terminal time
Let R and ".> 0/ be given. Then there is a constant "R > 0 such that, for
any x 2 SR and 0 t1 < t2 T ,
jJ.t1 ; ; x; AI / J.t2 ; ; x; AI /j < "; 8A 2 A (2.17)
and
jv.t1 ; ; xI / v.t2 ; ; xI /j < ";
whenever jt1 t2 j < "R .
Proof. We assume D 0, because the proof in the case of general is similar.

(i) is immediate from .b4 / and (2.9).
(ii) Fix and A and put X.t; x/ D XAx .t/ for simplicity. For given " > 0 and
R > 0, set
1
"R D .K1 .1 C .R C 1/2 /"1 / 2 ;
with K1 of (2.9). Then, for any x 2 SRC1 ,

h i
2
P sup jX.t; x/j > "R E sup jX.t; x/j2 "R < ": (2.18)
t T t T
Next choose a small ı0 D ı0 ."; R/ > 0 such that, for any x; y 2 SR ,
j.x/ .y/j C sup jf .t; x; / f .t; y; /j < ";

;t
whenever jx yj < ı0 ; (2.19)
which is possible thanks to .b2 / and .b3 /.

Let us fix y 2 SR arbitrarily. Then

P sup jX.t; x/ X.t; y/j > ı0
t T
h i
<E sup jX.t; x/ X.t; y/j2 ı02 < K1 jx yj2 ı02 (2.20)
t T
follows from (2.11).

Now using (2.18) and (2.20), we introduce two sets ˝x and ˝Q x by
n o
˝x D ! 2 ˝I sup jX.t; xI !/j "R (2.21)
t T
for x with jx yj 1, and

n o
˝Q x D ! 2 ˝I sup jX.t; xI !/ X.t; yI !/j < ı0 : (2.22)
t T
Then (2.18) and (2.20) imply
P .˝x / > 1 "; P .˝y / > 1 "; (2.23)
and
P .˝Q x / > 1 " (2.24)
whenever jx yj2 < "ı02 K11 .

Let us estimate the quantity
hZ t i
I1 .x; y/ WD E jf .s; X.s; x/; .s// f .s; X.s; y/; .s//j ds :

We have
I1 .x; y/
hZ t i
E jf .s; X.s; x/; .s// f .s; X.s; y/; .s//j dsI ˝x \ ˝y \ ˝Q x

hZ t i
O
C KE .2 C jX.s; x/j2 C jX.s; y/j2 / dsI ˝xc [ ˝yc [ ˝Q xc (2.25)

thanks to .b4 /. By (2.9) and (2.22)–(2.25), we can find a constant c1 > 0,

independent of ; t; A; ", and R, such that
p
I1 .x; y/ < c1 ".1 C R2 / (2.26)
q
whenever x; y 2 SR and jx yj < "
K 1 ı0 .
2.1 Introduction 39
Indeed, (2.19) and (2.22) imply that
1st term in the RHS of (2.25) < "T: (2.27)
For the 2nd term, (2.9), (2.23), and (2.24) yield

hZ t i
E jX.s; x/j2 dsI ˝xc [ ˝yc [ ˝Q xc

Z t 12 p
E. jX.s; x/j2 ds/2 3"

p p
TK2 .1 C jxj2 / 3" TK2 .1 C .R C 1/2 / 3" (2.28)
q
whenever x; y 2 SR and jx yj < "
K 1 ı0 .
Since for X.s; y/ one has the same estimate as (2.28), (2.27) and (2.28) together
with (2.25) establish (2.26).
Applying the same arguments to the terminal cost and using (2.26), we can find
ı1 WD ı1 ."; R/ > 0, such that
jJ.t; ; x; AI / J.t; ; y; AI /j < ";

8A 2 A; 0 < t T; (2.29)
whenever x; y 2 SR and jx yj < ı1 .

Finally, we consider the initial time. Let 1 < 2 < t and x 2 SR . Set X.s; / D
XAx .s/, and with the same "R and ı0 , define ˝i and ˝Q by
n o
˝i D ! 2 ˝I sup jX.s; i /j < "R ; i D 1; 2;
i sT
and
n o
˝Q D ! 2 ˝I sup jX.s; 1 / X.s; 2 /j < ı0 :
2 sT
By arguing as in the previous estimations, we can take ı2 D ı2 ."; R/ > 0, so that
jJ.t; 1 ; x; AI / J.t; 2 ; x; AI /j < ";

8t 2 ; 8x 2 SR ; 8A 2 A; (2.30)
whenever 2 1 < ı2 .
Now (ii) the proof of follows from (2.29) and (2.30).
Since (iii) the proof of mimics of (ii), the result is established. t
u
Theorem 2.1 asserts that, if the terminal cost function has quadratic growth,
then the value function also has quadratic growth. Put
n .x/ o
CQ D 2 C.Rd /I Q .x/ WD 2 C b .R d
/ : (2.31)
1 C jxj2
We introduce the norm k kCQ by
k kCQ D k Q kC.Rd / (2.32)
and the order by
' ” '.x/ .x/; 8x 2 Rd : (2.33)
Then CQ becomes a Banach lattice.

Recalling .b3 / and .b4 /, we define a two-parameter operator V t I CQ 7! CQ .0
t T / by
V t .x/ D v.t; ; xI /: (2.34)
Proposition 2.3. V t has the following properties:

(i) V is the identity map on CQ .
(ii) Monotonicity property: ) V t V t .
(iii) There is a constant KQ > 0, such that
Q
kV t V t kCQ Kk kCQ ; 8 t; 8; 2 CQ ; (2.35)
and
Q
kV t 0kCQ K; 8 t: (2.36)
Proof. (i) and (ii) are clear from the definition of payoff.
(iii) From the inequality j.x/ .x/j k kCQ .1 C jxj2 / it follows that

jJ.t; ; x; AI / J.t; ; x; AI /j k kCQ E x 1 C sup jX.t/j2 :
t T
Now (2.9) yields (iii). t

u
2.2 Dynamic Programming Principle (DPP)
The dynamic programming principle (DPP), introduced by R. Bellman [Be52,

Be57], gives a powerful tool for stochastic control problems and is known as a
two-stages optimization method. Here we will formulate DPP as a two parameter
2.2 Dynamic Programming Principle (DPP) 41
semigroup on a suitable Banach space, constructed by using time discretization

in Sects. 2.1.1–2.2.3 [N76, N81]. We characterize the semigroup as the envelope
of Markovian transition semigroups and show that its generator is related to HJB
equation in Sect. 2.2.5. We have two kinds of control processes on a fixed reference
probability system (˝; F ; .Ft /, P , W ): one is an .Ft /-progressively measurable
control process and the other is an .FtW /-progressively measurable one. But we
note that these two kind controls give the same value function (see Sect. 2.2.4).
In this section, we always assume the conditions .b1 /–.b5 /.
2.2.1 Discrete-Time Dynamic Programming Principle
Let D; 0 D t0 < t1 < < tp < tpC1 D T be a division of Œ0; T . We put

D D .t1 ; : : : ; tp /, PD D the set of division points of D.D ft1 ; : : : ; tp g/ and jDj D
Q we say that D D,
maxi D0;:::;p jti ti C1 j. For divisions D and D, Q if PD P Q .
D
D
Let A denote the set of all admissible controls .˝; F ; .Ft /; P; W; .// with
.t; !/ D .ti ; !/ for t 2 Œti ; ti C1 /; i D 0; : : : ; p: (2.37)
./ is called a switching control at D, or a D-admissible control. When we restrict

A to AD , the value function vD is given by
vD .t; ; xI / D inf J.t; ; x; AI /: (2.38)

A2AD
Since vD clearly satisfies (i)–(iii) in Theorem 2.1, VDt defined by
VDt .x/ D vD .t; ; xI / (2.39)
is a mapping from CQ into CQ .

We will show the following subsidiary theorem;
Theorem 2.2 (Discrete-time DPP). Let be convex and compact. Then for l <
m < n,
VtlDtm .VtmDtn / D VtlDtn ; (2.40)
that is,
inf J.tm ; tl ; x; AI vD .tn ; tm ; I // D vD .tn ; tl ; xI /: (2.41)

A2AD
Before we embark upon the proof, let us consider constant controls. Set A D
.˝; F ; .Ft /; P; W; /, where 2 stands for a constant control process: .t; !/ D
A
, for all t and !. Then its response Xy is the strong solution of (2.3), measurable
w.r.t. .W ./ W ./; y; /, and the payoff J.t; ; y; A I / depends only on
t; ; y; and . Denote the payoff by J.t; ; y; I /. We will seek an optimal
.2 / by using the measurable selection theorem (see [SV79], Lemma 12.1.7
and Theorem 12.1.10).
Since J.t; ; y; I / is continuous in .y; / by (2.11) and (2.13), the compact-
ness of implies that, for y 2 Rd , the set
y D f 2 I J.t; ; y; I / D inf J.t; ; y; I /g (2.42)

2
is non-empty and compact. Hence there is a minimum selector (Borel function)

t .I /I Rd 7! , such that
t .yI / 2 y ; 8y 2 Rd (2.43)
(see [FR75], Lemma B in Appendix).

Consequently
J.t; ; y; t .yI /I / D inf J.t; ; y; I / DW t .y/: (2.44)

2
Further, by (2.9) and .b4 /, there is a constant c > 0, such that
jf .s; x; /j C j t .x/j c.1 C jxj2 /;

8x 2 Rd ; 8 s t; 8 2 : (2.45)
Let us prove Theorem 2.2 by using the minimum selector.

Proof. For the proof, we assume D 0, because, for the general , the proof is
entirely similar. Putting
Y
j
tk tkC1 D ti ti C1 ti C1 ti C2 tj tj C1 ; (2.46)
kDi
we will show the inequality,
Y
j
VtD
i tj C1
.x/ tk tkC1 .x/ (2.47)
kDi
and its opposite,
Y
j
VtiDtj C1 .x/ tk tkC1 .x/: (2.48)
kDi
Put X A ./ D XtAi x ./. Since X A .tj / and .tj / are Ftj -measurable and W . C tj /
W .tj / is independent of Ftj ,
E.C.tj C1 ; tj ; AI /jFtj / D J.tj C1 ; tj ; X A .tj /; .tj /I /

tj tj C1 .X A .tj // P -a.s. (2.49)
holds for any A 2 AD . Hence, we have
Y
j
J.tj C1 ; ti ; x; AI / J.tj ; ti ; x; AI tj tj C1 / tk tkC1 .x/: (2.50)
kDi
Taking the infimum of LHS over A 2 AD , we obtain (2.47).

For (2.48), we construct a control process ./ for which the RHS of (2.47) is
attained. This is done by using the minimum selector t ./ of (2.43). On a reference
probability system .˝; F ; .Ft /; P; W /, we first define .ti / by
Y
j
.ti / D ti ti C1 xI tk tkC1 ; (2.51)
kDi C1
and, consider the SDE

(
dX.t/ D b.t; X.t/; .ti // dt C ˛.t; X.t/; .ti // d W .t/; t 2 .ti ; ti C1 ;
X.ti / D x:
(2.52)
Then we define .ti C1 / by
Y
j

.ti C1 / D ti C1 ti C2 X.ti C1 /I tk tkC1 : (2.53)
kDi C2
Repeating this procedure, we obtain an .FtW /-progressively measurable ./, such

that A WD .˝; F ; .Ft /; P; W; .// is in AD and
Y
j
J.tj C1 ; ti ; x; A I / D tk tkC1 .x/; (2.54)
kDi
which yields (2.48).

Now from (2.47) and (2.48) it follows that
m1
Y
VtlDtn .x/ D tk tk1 .VtmDtn /.x/
kDl
D VtlDtn .VtmDtn /.x/: (2.55)
This completes the proof. t

u
Put W D L1 .Œ0; T ˝; .FtW /I / and W;D D f./ 2 W ; switching
at Dg. Then referring to the proof above and to the minimum selector, we have
Remark 2.1. For ; t 2 PD [ f0; T g and .˝; F ; .Ft /; P; W /, there is ./ 2
W;D , such that
VDt .x/ D J.t; ; x; A I /;
where A D .˝; F ; .Ft /; P; W; .//.

Next we study the discrete-time DP property.
Proposition 2.4. Let D D .t1 ; : : : ; tp / be given.
(i) Let ./ be an optimal control process in W;D , given by the minimum selector.
Then, for any .FtW /-stopping time taking values in PD [ f0; T g,
hZ Z s
D
V0T .x/ DE0x expf .; X ./; .// dgf .s; X .s/; .s// ds
0 0
Z i
C expf .; X ./; .// dgV DT .X . // ; (2.56)
0
where X is the response for ./, with X .0/ D x,

(ii) Let ./ 2 W;D and X its response with X.0/ D x. Then, for any .FtW /-
stopping time taking values in PD [ f0; T g,
hZ Z s
D
V0T .x/ E0x expf .; X./; .// dgf .s; X.s/; .s// ds
0 0
Z i
C expf .; X./; .// dgV DT .X. // : (2.57)
0
Proof. P . D 0/ D 1 or 0, because . D 0/ 2 F0W . When P . D 0/ D 1, (2.56)

and (2.57) are trivial. Hence we may assume in the proof that the set of values of
is ft1 ; : : : ; tp ; T g.
We again assume D 0.
(i) We have
C.T; 0; ./I /
X
pC1 Z tk
D . D tk / f .s; X .s/; .s// ds C C.T; tk ; ./I / ; (2.58)
kD1 0
where tpC1 D T .
From the definition of ./, it follows that
E.C.T; tp ; ./I /jFtWp / D tp T .X .tp //;
E.C.T; tp1 ; ./I /jFtWp1 /

Z tp
DE f .s; X .s/; .s// ds C E.C.T; tp ; ./I /jFtWp / jFtWp1
tp1
D E.C.tp ; tp1 ; ./I tp T /jFtWp1 /
D tp1 ;tp .tp T /.X .tp1 //

D VtD
p1 T
.X .tp1 //
and
E.C.T; tk ; ./I /jFtWk / D VtkDT .X .tk //: (2.59)
In view of (2.58) and (2.59), we obtain

D
V0T .x/ D E0x C.T; 0; ./I /
X
pC1 h nZ tk oi
D E0x . Dtk / f .s; X .s/; .s// ds CE.C.T; tk ; ./I /jFtWp /
kD1 0
hZ i
D E0x f .s; X .s/; .s// ds C V DT .X . // :
0
(ii) Let us define Q .t/ by
X
pC1
Q .t/ D . D tk /..t/.t < tk / C .t/.t tk //:
kD1
Then Q ./ 2 W;D

. Further (2.59) yields
D
V0T .x/ J.T; 0; x; Q ./I /
X
pC1 h Z tk i
D E0x . D tk / f .s; X.s/; .s// ds C E.C.T; tk ; ./I /jFtWk /
kD1 0
D RHS of (2.57);
which concludes the proof of (ii). t

u
2.2.2 Approximation Theorem
Before we prove the dynamic programming principle, we establish the following

approximation result.
Theorem 2.3. Let be convex and compact and Dn D .tn;1 ; : : : ; tn;j.n/ /,
n D 1; 2; : : : .
Suppose that
Dn DnC1 ; n D 1; 2; : : : and lim jDn j D 0: (2.60)

n!1
Let A D .˝; F ; .Ft /; P; W; .// .2 A/ be given. Then there exists An D

.˝; F ; .Ft /; P; W; n .// of ADn such that
lim E1n./ ./12 D 0 (2.61)

n!1
and
lim jn .s/ .s/j D 0 a:e: on Œ0; T ˝: (2.62)

n!1
Proof. Since is convex and compact, we can apply a routine.

First fix 0 2 arbitrarily and set
.s/ D 0 for s < 0:
With this convention, we define Nl ./ by

Z t
Nl .t/ D 2 l
.s/ ds; t 0:
t 2l
Then Nl ./ is a continuous control process, satisfying
lim E1Nl ./ ./12 D 0: (2.63)

l!1
Second, we define Nl;m ./ by
Nl;m .t/ D Nl .tm;j / for t 2 Œtm;j ; tm;j C1 /; j D 0; 1 : : : ; j.m/ 1: (2.64)
Then Nl;m ./ provides a switching control at Dm , and, as m ! 1, Nl;m ./ approaches
Nl ./ uniformly on Œ0; T , P -a.s.
Third, we can choose .ln ; mn / so that mn n and
EŒ1ln ;mn ./ ./12 < 2n : (2.65)
Finally we define p ./ as follows:
1 .t/ D D m1 1 .t/ D 0 ; 8t 2 Œ0; T ;

mn .t/ D D mnC1 1 .t/ D ln ;mn .t/; 8t 2 Œ0; T ; n D 1; 2; : : : :
Then p ./; p D mn ; : : : ; mnC1 1 become switching controls at Dmn , and hence at

Dp , by (2.60). By (2.65), p ./ satisfies (2.61).
Appealing to the Borel–Cantelli Lemma, we can choose a subsequence pj ./,
so that (2.62) holds. This completes the proof. t
u
Regarding the responses X A and X An , (2.13) and (2.61) lead to
h i
lim E x sup jX An .t/ X A .t/j2 D 0; 8.; x/: (2.66)
n!1 t T
Set
[
As D AD D set of all switching controls.
D
Using (2.66), we get

Corollary 2.2. Let is convex and compact. Then
V t .x/ D inf J.t; ; x; AI /

A2As
D lim inf J.t; ; x; AI /; 8 2 CQ (2.67)

n!1 A2ADn
whenever Dn ; n D 1; 2; : : : satisfy (2.60).
2.2.3 Dynamic Programming Principle
Now we are ready to prove DPP

Theorem 2.4. Let be convex and -compact. We assume .b1 /–.b5 / in Sect. 2.1.1.
Then, for any < 1 < 2 T ,
V 2 D V 1 .V1 2 / for 2 CQ : (2.68)

Proof. We divide the proof into two steps.

Step 1. Let be convex and compact. Take a sequence Dn ; n D 1; 2; : : : such
that ; 1 ; 2 2 PD1 [ f0; T g and (2.60) holds.
Then Theorem 2.2 shows that
VD2n D VD1n VD n

1 2
: (2.69)
Since VD1 n2 V1 2 , the monotonicity property of VD1n leads to
VD1n .VD n
1 2
/ VD1n .V1 2 /: (2.70)
Hence from (2.69) and (2.70), it follows that
VD2n VD1n .V1 2 / (2.71)
Letting n ! 1, and using Corollary 2.2 we get
V 2 V 1 .V1 2 / (2.72)
D
Next we shall show the converse of inequality (2.72). Since V 2nCm .x/ D
D D D
V 1nCm .V1 nCm
2
/ V 1nCm .VD m
1 2
/.x/, we obtain, letting n ! 1,
V 2 .x/ V 1 .VD m
1 2
/.x/ .m D 1; 2; : : : /
J.1 ; ; x; AI VD m
1 2
/; 8A 2 A: (2.73)
Using (2.15) and (2.9), we have
E x jVD1 m2 .X A .1 //j2 c1 E x .1 C jX A .1 /j2 /2
c2 .1 C jxj4 /; (2.74)
with constants c1 and c2 independent of A and Dm .

Since Corollary 2.2 implies
lim VD m
1 2
.X.1 // D V1 2 .X.1 // P -a.s.; (2.75)
m!1
(2.74) and the convergence theorem lead to
lim J.1 ; ; x; AI VD m

1 2
/ D J.1 ; ; x; AI V1 2 /: (2.76)
m!1
Therefore the inequality

V 2 .x/ J.1 ; ; x; AI V1 2 /; 8A 2 A (2.77)
follows from (2.73) and (2.76). Taking the infimum of the RHS of (2.77) over
A 2 A, we obtain the converse of inequality (2.72).
This completes the proof of (2.68) when is convex and compact.
Step 2. Let be convex and -compact.
We can take a sequence of convex and compact sets n ; n D 1; 2; such that
[
n nC1 ; n D 1; 2; : : : and n D : (2.78)
n
We denote by An the set of all admissible controls with control region n . Then
[
An AnC1 and An D A; (2.79)
n
by Definition 2.1. Observing that (2.79) implies
J.t; ; x; AI / lim Vnt .x/; 8A 2 A; (2.80)

n!1
where
Vnt .x/ D infn J.t; ; x; AI /; (2.81)

A2A
and taking the infimum of (2.80) w.r.t. A over A, we have
V t .x/ D lim Vnt .x/; 8 2 t; x 2 Rd ; (2.82)

n!1
because V t .x/ Vnt .x/, 8n.

Since Vnt satisfies DPP,
V 2 .x/ D lim Vn2 .x/

n!1
D lim Vn1 .Vn1 2 /.x/

n!1
lim Vn1 .V1 2 /.x/ .by Vn1 2 V1 2 /

n!1
D V 1 .V1 2 /.x/: (2.83)
For the converse inequality, we have
2 .x/ D V 1 .V1 2 /.x/

VnCm nCm nCm
VnCm m
1 .V1 2 /.x/; m; n D 1; 2; : : : : (2.84)
Letting n ! 1, we get
V 2 .x/ V 1 .Vm1 2 /.x/; m D 1; 2; : : :

J.1 ; ; x; AI Vm1 2 /; 8A 2 A; m D 1; 2; : : : : (2.85)
Again (2.74) and the convergence theorem yield
lim J.1 ; ; x; AI Vm1 2 / D J.1 ; ; x; AI V1 2 /: (2.86)

m!1
Thus, the converse of inequality (2.83) follows from (2.85) and (2.86).
This completes the proof of Theorem 2.4. t
u
2.2.4 Brownian Adapted Controls
In this section, we will comment on the value function in the case where one restricts
the control processes to the class of Brownian adapted ones.
We denote the set of all admissible controls, A D .˝; F ; .Ft /; P; W; .// where
./ is .FtW /-progressively measurable, by AW . Put
VWt .x/ D inf J.t; ; x; AI /: (2.87)

A2AW
Then we have:
Proposition 2.5.
VWt D V t ; 8 < t: (2.88)
Proof. Since we can easily see that (2.82) is also valid for VWt , we may assume
for the proof that is convex and compact. Now the proposition follows from
Remark 2.1 and Corollary 2.2. t
u
For A 2 AW ; J.t; ; x; AI / can be calculated by means of the joint probability
distribution .W; .//. So, we fix a reference probability system .˝; F ; .Ft /; P; W /
and identify A with its control process ./.
For a stopping time , we again denote the payoff by J./:
hZ Z s
J. ; ; x; ./I / DE x expf .; X./; .// dgf .s; X.s/; .s// ds

Z i
C expf .; X./; .// dg .X. // ; (2.89)

where X D X ./ .
Proposition 2.6 (DP Property).

Let .˝; F ; .Ft /; P; W / and .; x/ 2 Œ0; T / Rd be given.
(i) For a Œ; T -valued .FtW /-stopping time and ./ 2 W
,
VT .x/ J. ; ; x; ./I V T /: (2.90)
(ii) For " > 0, there is " ./ 2 W

, such that
V t .x/ C " J. ; ; x; " ./I V T / (2.91)
for any Œ; T -valued .FtW /-stopping time .

Proof. Although the DP property was given in [FS06], Theorem 7.1, we will sketch
the proof by using Proposition 2.4. Assume that D 0; D 0, and is convex and
compact.
Put n D 2n Œ1C2n ^T . For ./ 2 W , we are mainly interested in estimating
J. n ; 0; x; ./I V n T / J. ; 0; x; ./I V T /. We have
Z n
In WD E0x jf .s; X.s/; .s//j ds
p n Z t 12
T 2 2 E0x K.1 C jX.s/j4 / ds .use .b4 //
0
n
2 2 c1 .1 C jxj2 / (2.92)
with a constant c1 independent of n and ./.

Since Vt T .y/ is continuous w.r.t. .t; y/,

jV nT .X. n//j KN 1 C sup jX.t/j2 (2.93)
0t T
by (2.15), and the RHS of (2.93) is integrable, the dominated convergence theorem
yields
lim E0x jV nT .X. n // V T .X. //j D 0: (2.94)

n!1
Let us fix n and take a switching controls, l ./ 2 W ; l D 1; 2; : : : ,

approaching ./, so that
h i
lim E0x sup jX l ./ .t/ X.t/j2 D 0 (2.95)
l!1 t
and
lim sup jX l ./ .t/ X.t/j D 0 P -a.s. (2.96)

l!1 t
We take a sequence of divisions Dl ; l D 1; 2; : : : ; such that (2.60) holds and Dl

contains all values of n and switching times of l ./. Then Corollary 2.2 yields
DlCm
lim V nT
.X l ./ . n // DV nT .X l ./ . n // P -a.s. and in L1 .˝/: (2.97)
m!1
On the other hand, Proposition 2.4 (ii) implies

D DlCm
V0T lCm .x/ J. n ; 0; x; l ./I V nT
/: (2.98)
Letting m ! 1 and using (2.97), we obtain

hZ n i
V0T .x/ E0x f .s; X l ./ .s/; l .s// ds C V nT .X l ./ . n // : (2.99)
0
Now taking l ! 1 and n ! 1 one obtains (i).

(ii) Let " > 0 be given. By Corollary 2.2 and Remark 2.1, we can choose a
switching control " ./ 2 W , such that
V0T .x/ C " > J.T; 0; x; " ./I /: (2.100)
Suppose that is a finitely-many valued .F W /-stopping time, say .tk ; k D

1; : : : ; p/. Since " .t/ is a functional of .W ./; tk / and .W .s/ W .tk /; s 2
Œtk ; t/, under P .jFtWk /; " ./ becomes a control process by freezing .W ./;
tk /, and similarly for its response X " .t/. Hence
Z T
E f .s; X " .s/; " .s// ds C .X " .T //jFtWk Vtk T .X " .tk // P -a.s.:
tk
(2.101)
Considering
X
p nZ tk
J.T; 0; x; ./I / D E0x
"
. D tk / f .s; X " .s/; " .s// ds
kD1 0
Z T o
C f .s; X " .s/; " .s// ds C .X " .T // (2.102)
tk
and applying (2.101) to (2.102), we have
J.T; 0; x; " ./I /

X
p nZ tk o
E0x . D tk / f .s; X " .s/; " .s// ds C Vtk T .X " .T //
kD1 0
D J. ; 0; x; " ./I V T /: (2.103)
Since any stopping time can be approached by finitely-many valued ones, (2.100)
and (2.103) complete the proof. t
u
Example 2.2 (DPP with hitting time). Assume f ./ D 0 and ./ D 0. Let O be
an open set. For ./ 2 W ; X ./ and ./ denote its response and the hitting
time of O by X ./ respectively. Then DP property yields that the value function
v.; x/ D VT .x/ satisfies
v.; x/ D inf E x v. ./

; X ./ . ./
// for 2 Œ0; T /; x … O: (2.104)
./2 W
2.2.5 Characterization of the Semigroup .Vt ; t/
Referring to Theorems 2.1 and 2.4 we summarize the basic properties of the
semigroup .V t ; t/; ; t 2 Œ0; T .
Proposition 2.7. V t W CQ ! CQ satisfies the followings:
(i) V D identity, 8 2 Œ0; T ,
(ii) Semigroup property
V t D Vs Vst for s t ;
(iii) Monotonicity
H) V t V t ;
(iv) 9kQ > 0 such that 8; 2 CQ and t,

(
Q
kV t V t kCQ kk kCQ ;
(2.105)
Q
kV t 0kCQ k;
(v) Continuity w.r.t. the time parameters

O we have
Let R > 0 be given. For any tO and ,
8
ˆ
< tlim sup jV t .x/ V tO.x/j D 0; 8 < tO;
!Ot jxjR
O (2.106)
:̂ lim sup jV t .x/ VO t .x/j D 0; 8t > :
!O jxjR
Next we will characterize .V t ; t/ from the point of view of semigroups.

For a constant control .t/ D ; 8t 2 Œ0; T , its response is a diffusion governed
by the SDE
dX.t/ D b.t; X.t/; / dt C ˛.t; X.t/; / d W .t/:

Let us define H t W CQ 7! CQ . t/, by

H t .x/ D J.t; ; x; I /; 8x 2 Rd : (2.107)

Then .H t I t/ is a semigroup satisfying properties (i)–(v) of Proposition 2.7
and
8 2 CQ :

V t H t ; (2.108)
Theorem 2.5. Let U t I CQ 7! CQ ; t, be a semigroup with the monotonicity

property. If
8 2 CQ ;

U t H t ; 8 2 ; 8 t; (2.109)
then
U t V t ; 8 2 CQ ; 8 t: (2.110)
In other words, .V t ; t/ is the maximal element in the set of monotone

semigroups satisfying (2.109).

.V t ; t/ is called the envelope of f.H t ; t/I 2 g.
The proof is easy, using (2.47), (2.48), and Corollary 2.2.
Finally, using the properties of the value function, we compute the generator of
.V t ; t/. Put
DDf 2 CQ \ Cp2 .Rd /I @x and @xx satisfy the polynomial growth conditiong:
Then
1
G D tr.a.; x; /@xx / C b.; x; / @x
2
is the generator of X , with domain D, and

G .x/ .; x; / .x/ C f .; x; /

is the generator of .H t ; t/. Put

G .x/ WD inf .G .x/ .; x; / .x/ C f .; x; //:
2
Proposition 2.8. For any R0 > 0 and 2 D,

ˇ 1 ˇ
ˇ ˇ
lim sup ˇ .V t .x/ .x// G .x/ˇ D 0: (2.111)
t ! jxjR t
0
Straightforward computations together with Proposition 2.1 give the following

lemma.
Lemma 2.1. Suppose that the Borel function h defined on Œ0; T Rd is
continuous w.r.t. .t; x/ uniformly on and satisfies
jh.t; x; /j K.1 C jxj2p /; 8t; x; ; (2.112)
with constants K > 0 and p 1. Then, for any positive " and R0 , there is ı"R0 > 0
such that for any ./ 2 W ,
h i
sup E x sup jh.s; X ./ .s/; .s// h.; x; .s//j < " (2.113)
jxjR0 st
whenever jt j < ı"R0 .

Proof (Proposition 2.8). Put
1
h.t; x; / D tr.a.t; x; /@xx .x// C b.t; x; / @x .x/
2
.t; x; / .x/ C f .t; x; /; 2 D:
Since h./ satisfies the conditions of Lemma 2.1,

ˇ hZ t i
ˇ
J1 WDˇ inf E x h.s; X ./ .s/; .s// ds
W
./2
hZ t iˇ
ˇ
inf E x h.; x; .s// ds ˇ
./2 W
< ".t /; 8jxj R0 (2.114)

On the other hand, (2.112) and .b5 / yield
hZ t Rs i
J2 WD E x je .;X ./ ./;.// d
1jjh.s; X ./ .s/; .s//j ds

Z t
< c3 E x .s /.1 C jX ./ .s/j2p / ds

c3 h i
.t /2 E x sup .1 C jX ./ .s/j2p /
2 st
c4 .t /2 .1 C jxj2p /; 8./ 2 W ; (2.115)
with constant c3 and c4 independent of ./; t; , and x.

Thus, Itô’s formula together with (2.114) and (2.115) yield

ˇ h Z t iˇ
ˇ ˇ
ˇV t .x/ .x/ inf E x h.; x; .s// ds ˇ
./2 W
2p
J1 C J2 < .t /.c4 .1 C R0 /.t / C "/; 8jxj R0 ; (2.116)

Using the inequalities
Z t
inf E x h.; x; .s// ds .t / inf h.; x; /
./2 W 2
Z t
D inf h.; x; / ds
2
Z t
inf E x h.; x; .s// ds;
./2 W
we have
Z t
inf E x h.; x; .s// ds D .t / inf h.; x; /: (2.117)
./2 W 2
Inserting (2.117) into (2.114) and referring to Proposition 2.5, we complete the
proof. u
t
Proposition 2.8 asserts that if the value function is smooth, then it satisfies the
Cauchy problem for the (nonlinear) parabolic equation;
@V
.t; x/ C H.t; x; @xx V .t; x/; @x V .t; x/; V .t; x// D 0; 8.t; x/ 2 Œ0; T / Rd
@t
(2.118)
V .T; x/ D .x/; x 2 Rd ; (2.119)
where H W Œ0; T Rd S d Rd R1 7! R1 , is given by
H.t; x; A; P; v/
1
D sup tr.a.t; x; /A/ b.t; x; / p C .t; x; /v f .t; x; / : (2.120)
2 2
The parabolic equation (2.118) is called Hamilton–Jacobi–Bellman equation (HJB

equation in short), or the dynamic programming equation.
Viscosity solutions of HJB equations will be considered in Chap. 3.
Example 2.3 (Time-homogeneous case). Suppose that all the coefficients ˛; b;

and f are independent of the time variable. Here we use Brownian adapted control
processes. In this case, we have
t D 0.t / : (2.121)
Indeed, for ./ 2 W , its response is the solution of the SDE

(
dX.t/ D b.X.t/; .t// dt C ˛.X.t/; .t// d W .t/; t > ;
(2.122)
X./ D x:
If . C / is adapted to W ./ WD W . C / W ./, then O .t/ WD .t C / can be

regarded as an element of W and the solution XO of the SDE

(
O
d X.t/ D b.XO .t/; O .t// dt C ˛.XO .t/; O .t// d W .t/; t > 0;
(2.123)
XO .0/ D x;
is nothing but X. C /. Consequently,
V t .x/ inf J.t; ; x; ./I /

./2 W
D V0.t / .x/: (2.124)
However, we have seen in (2.101) that for any ./ 2 W ,

E x C.t; ; ./I / D E x E.C.t; ; ./I /jFW /
inf J.t; ; x; ./I /: (2.125)

./2 W
Now (2.124) and (2.125) yield (2.121).

Putting Vt WD V0t , we have the semigroup .Vt ; t 2 Œ0; T / with the generator G
given by
1
G .x/ D lim .Vt .x/ .x//
t !0 t
1
D inf tr.a.x; /@xx / C b.x; / @x .x; / C f .x; /
2 2
(2.126)
for 2 D.
2.3 Verification Theorems and Optimal Controls
In this section we show how to construct optimal controls or optimal Markovian

policies using the HJB equations. In the previous section, we have shown, using DPP
and Itô’s formula, that under appropriate regularity assumption the value function
satisfies the HJB equation. However, it is rather difficult to verify the regularity of
value function. But, if a classical solution of HJB equation exists, then we can take
an optimal Markovian policy by using the minimum selector and obtain the value
function. This assertion is called a verification theorem.
In Sect. 2.3.1, we state verification theorems, which are useful in seeking optimal
controls in practical problems. In Sect. 2.3.2, we give three simple examples.
2.3.1 Verification Theorems
Here we assume the following conditions .b1 /0 and .b2 /0 weaker than .b1 /–.b5 /:
.b1 /0
b W Œ0; T Rd 7! Rd ; ˛ W Œ0; T Rd 7! Rd ˝ Rm
are uniformly continuous and satisfy
jb.t1 ; x1 ; 1 / b.t2 ; x2 ; 2 /j C j˛.t1 ; x1 ; 1 / ˛.t2 ; x2 ; 2 /j

ljx1 x2 j C m.jt1 t2 j C j1 2 j/; (2.127)
with a constant l and a modulus function m./, and
jb.t; x; /j C j˛.t; x; /j K.1 C jxj C j j/; (2.128)
with a constant K.
.b2 /0
W Œ0; T Rd 7! Œ0; c0 ; f W Œ0; T Rd 7! R1 ; W Rd 7! R1
are continuous and
O C jxj2 C j j2 /;
jf .t; x; /j C j.x/j K.1 (2.129)
O
with a constant K.
Here we admit a slightly bigger class of control processes. Specifically let
.˝; F ; .Ft /; P; W / be a reference probability system. We take . .t/; t 2 Œ0; T / to
be a -valued .Ft /-progressively measurable process, satisfying
2.3 Verification Theorems and Optimal Controls 59
hZ T i
E j .t/jn ds < 1 for n D 1; 2; : : : : (2.130)
0
This condition is sometimes convenient for establishing the existence of optimal

controls (see Example 2.4).
For a control A D .˝; F ; .Ft /; P; W; .//, its response evolves according to
the SDE
dX.t/ D b.t; X.t/; .t// dt C ˛.t; X.t/; .t// d W .t/; t 2 Œ; T ; (2.131)
X./ D x: (2.132)
We clearly have a unique solution of (2.3)–(2.4), satisfying

h i Z t
E x sup jX.s/j 2p
cp 1 C jxj C E
2p
j .s/j2p ds ; 8; t; x; A ;
st
(2.133)
with a constant cp .
By (2.131) and .b2 /0 , we can define the payoff J.T; ; x; A I / by (2.6) and the
value function v ./ by
v .; x/ D inf J.T; ; x; A I /; (2.134)

A
respectively.
Remark 2.2.
v .; x/ D v.; x/:
Proof. That v .; x/ v.; x/ is clear. For the opposite inequality, we fix A D
.˝; F ; .Ft /; P; W; .// arbitrarily and claim that
J.T; ; x; A I / v.; x/: (2.135)
Put N .t/ D .t/.j .t/j N / and AN D .˝; F ; .Ft /; P; N .//. Then AN 2 A

and
hZ T i hZ T i
lim E j .s/N .s/jn ds D lim E j .s/jn .j .s/j > N / ds D 0:
N !1 0 N !1 0
(2.136)
X and XN denote the responses for ./ and N ./, respectively.
Let N .t/ WD E x Œsup st jX.s/XN .s/j2 . From (2.127) and the Burkholder–
Davis–Gundy inequality, we easily deduce that
Z t
N .t/ cE fN .s/ C .1 C j .s/j2 /.j .s/j N /g ds ; (2.137)

with a constant c independent of ; x; t; N , and ./.

Hence, (2.136) and (2.137) yield
h i
lim E x sup jX.s/ XN .s/j2 D 0:
N !1 sT
Choosing a subsequence Nj so that
lim sup jX.s/ XNj .s/j D 0 P -a.s.

Nj !1 sT
and using (2.133), we conclude that
lim J.T; ; x; ANj I / D J.T; ; x; A I /;

Nj !1
which in turn yields (2.135). t

u
Let us consider the HJB equation;
@V
.t; x/ C H.t; x; @xx V .t; x/; @x V .t; x/; V .t; x// D 0; 8.t; x/ 2 Œ0; T / Rd
@t
(2.1180)
v.T; x/ D .x/; x 2 Rd : (2.1190)
The following result holds true.

Theorem 2.6 (([FS06], p. 159) Verification theorem).
Let ! 2 C 12 .Œ0; T / Rd / \ C.Œ0; T Rd / be a solution of (2.1180)–(2.1190),
satisfying
j!.t; x/j K1 .1 C jxj2 /; 8t; x; (2.138)
with a constant K1 > 0.

Then
(i) For any A D .˝; F ; .Ft /; P; W; .//,
!.t; x/ J.T; t; x; A I /; 8t; x: (2.139)

(ii) Let .t; x/ be given. Suppose that there exists AO D .˝;
O FO ; .FO t /; PO ; WO ; O .//
such that
min.Gs !.s; XO .s// .s; XO .s/; /!.s; XO .s// C f .s; XO .s/; //

2
DGsO .s/ !.s; XO .s// .s; XO .s/; O .s//!.s; XO .s// C f .s; XO .s/; O .s//;
8s 2 Œt; T ; PO -a.s.; (2.140)
where XO is the response for AO with XO .t/ D x.

Then
!.t; x/ D J.T; t; x; AO I /: (2.141)
(iii) AO is optimal for .t; x/, i.e.,
v .t; x/ D !.t; x/ D J.T; t; x; AO I /: (2.142)
For the case of optimal Markovian policies, we deduce from Theorem 2.6 and
Proposition 1.6 the following
Corollary 2.3. In addition to .b1 /0 and .b2 /0 , we assume that
(a) is convex and compact,
(b) d D m,
(c) b./ is bounded,
(d) ˛./ is bounded, symmetric and uniformly parabolic.
Suppose that the HJB equation (2.1180)–(2.1190) has a classical solution, satisfying
the growth condition (2.138). Then the maximum selector of H./ provides an
optimal Markovian policy.
Regarding the classical solution of HJB equation, we recall the following result.
Theorem 2.7 ([FS06], p. 163). Let D 0 and assume;
(a) is compact and d D m,
(b) ˛./ is uniformly parabolic and independent of . Moreover, ˛./ 2 C 12 .Œ0; T
Rd / and ˛./; ˛./1 , and @x ˛./ are bounded,
Q x/ C ˛.t; x/.t; x; /, where b./
(c) b.t; x; / D b.t; Q 2 C 12 .Œ0; T Rd / and
Q is bounded, and ./ and @x ./ are in Cb .Œ0; T Rd /,
@x b./
(d) f ./ and @x f ./ are in Cp .Œ0; T Rd /,
(e) ./ 2 C 3 .Rd / \ Cp1 .Rd /.
Then the HJB equation (2.1180)–(2.1190) has a unique classical solution.
2.3.2 Examples of Optimal Control
We are concerned with explicit formulations of optimal controls for three simple
models. The first two examples are related to Gaussian diffusions effected by linear
control processes. The third one is a stochastic control with state constraint.
Example 2.4 (Linear Gaussian quadratic regulator). Let D Rq and suppose
that matrix-valued continuous functions of suitable sizes A; B; ; M; N and D are
given on Œ0; T . For an admissible control A D .˝; F ; .Ft /; P; W; .//, we have
the d -dimensional SDE
dX.t/ D .A.t/X.t/ C B.t/ .t// dt C .t/ d W .t/; t 2 Œ; T ; (2.143)
X./ D x.2 Rd / (2.144)
and the payoff
J.T; ; x; A /
hZ T i
DE x .X.s/> M.s/X.s/ C .s/> N.s/.s// ds C X.T /> DX.T / :

(2.145)
We assume that for any t 2 Œ0; T ,

(a) D; M.t/ are non-negative definite symmetric d d matrices;
(b) N.t/ is a positive definite symmetric q q matrix.
Since the corresponding HJB equation reads
@! 1
0D .t; x/ C tr.a.t/@xx !.t; x// C .A.t/x/ @x !.t; x/
@t 2
C x > M.t/x C inf ..B.t/ / @x !.t; x/ C > N.t/ /;
2
t 2 Œ0; T /; x 2 Rd ; (2.146)
!.T; x/ D x > Dx; (2.147)
we are looking for the solution of (2.146)–(2.147).

Observing that the expression whose inf is taken in (2.146) is
ˇ1 ˇ2 1
ˇ 1 1 ˇ 1
ˇ N.t/ 2 B.t/> @x !.t; x/ C N.t/ 2 ˇ jN.t/ 2 B.t/> @x !.t; x/j2 (2.148)
2 4
and assuming that !./ is quadratic, we have

Z T
>
!.t; x/ D x P .t/x C tr.a.s/P .s// ds (2.149)
t
where P ./ is the solution of the following Riccati equation:
dP
0D .t/ C M.t/ C A.t/> P .t/ C P .t/A.t/
dt
P .t/B.t/N.t/1 B.t/> P .t/; t 2 Œ0; T / (2.150)
with
P .T / D D: (2.151)
Hence, from (2.148) and (2.149) it follows that an optimal Markovian policy is given
by N.t/1 B.t/> P .t/x.
More details for LQ problems are given in [YZ99], Chapter 6.
Example 2.5 (1-dimensional bang-bang control). Let D Œ1; 1. Suppose that
g 2 Cp1 .R1 / is even, g.0/ D 0 and, g 0 .x/ 0, on Œ0; 1/.
Let us consider the following time-homogeneous simple model. For A D
.˝; F ; .Ft /; P; W; .//, its response X A evolves according to the SDE
dX.t/ D .t/ dt C d W .t/; t 2 .0; T (2.152)
X.0/ D x .2 R1 /; (2.153)
and the payoff j./ is given by
j.t; x; A/ D Ex g.X A .t//: (2.154)
The (formal) HJB equation for the value function v.t; x/ D infA2A j.t; x; A/ reads
8
ˆ
ˆ @v .t; x/ D 1 @xx v.t; x/ C inf . @x v.t; x//
ˆ
ˆ
< @t 2 j j1
1 (2.155)
ˆ
ˆ D @xx v.t; x/ j@x v.t; x/j; t > 0; x 2 R1 ;
ˆ 2
:̂
v.0; x/ D g.x/; x 2 R1 :
Now we look for the explicit solution of (2.155).

Proposition 2.9 (Communicated by F. Asakura).

Z 1 jxj2 y
1 t
v.t; x/ D p e 2 g.y/ exp C jxj jyj dy
2 t 1 2t
Z 1 Z
1 3 jxj C jyj C s 2
1
p g.y/e jxjjyj s 2 .jxjCjyj C s/ exp ds dy
2 2 1 t 2s
Z 1
C e 2jyj g.y/ dy (2.156)
1
and
8
< 1; x > 0;
sgn.@x v.t; x// D sgn x D 0; x D 0; (2.157)
:
1; x < 0:
Outline of proof. Consider the auxiliary equation
@!
.t; x/ D @xx !.t; x/; t > 0; x>0 (2.158)
@t
with the boundary and initial conditions
8
ˆ 1
< @x !.t; 0/ C !.t; 0/ D 0; t > 0;
2 (2.159)
x
:̂ !.0; x/ D e 2 g
x
; x > 0:
2
We solve (2.158)–(2.159) using the Laplace transformation. Putting u.t; x/ D

2xt
!.t; x/e 4 , we have
@x u.t; x/ > 0 for x > 0 (2.160)
and
x
lim u.t; x/ D g : (2.161)
t !0 2
Further, v.t; x/ WD u.2t; 2jxj/; t > 0; x 2 R1 is given by expression (2.156) and

satisfies (2.155). From (2.156), it follows that
sgn.@x v.t; x// D sgn x: (2.162)

u
By Theorem 2.6, v./ is the value function and the minimum selector O .t; x/ D
sgn.@x v.t; x// D sgn x gives an optimal Markovian policy. Its response X
evolves according to SDE
dX.t/ D sgn X.t/ C d W .t/: (2.163)
Since (2.163) admits a unique strong solution, X is the diffusion with generator
1 d2 2jxj
2 dx 2 .sgn x/ dx and e
d
gives the density of the corresponding invariant
probability measure. From (2.156) it follows that for any R > 0, there is a constant
CR > 0 such that
ˇ Z 1 ˇ
ˇ ˇ t
sup ˇv.t; x/ e 2jyj g.y/ dy ˇ CR e 2 ; 8t > 1: (2.164)
jxjR 1
For related topics consult [Be75, BSW80] and [IW81], Chap. 6.

Finally we will consider stochastic control with state constraint. We assume that
the coefficients ˛; b, and f are independent of the time variable, and satisfy .b1 /0
and .b2 /0 .
Let O be a bounded and open set of Rd with smooth boundary @O. For
an admissible control A D .˝; F ; .Ft /; P; W; .//, the response X A evolves
according to the SDE
(
dX.t/ D b.x.t/; .t// dt C ˛.X.t/; .t// d W .t/; t > 0;
(2.165)
X.0/ D x:
By A
we denote the exit time of X A from O, that is
(
inffs > 0I X A .s/ … Og;
A
D (2.166)
1; if f g is empty:
For a given continuous function g on O and a positive constant , we define the

payoff J./ by
hZ A i
e s f .X A .s/; .s// ds C e g.X A .
A
J.x; A/ D Ex A
// ; x 2 OI
0
(2.167)
here, when A D 1; e g.X A . A // stands for 0.
A
When ./ is constant 2 , its response X A is the diffusion with the generator

G given by
1
G u D tr.a.x; /@xx u/ C b.x; / @x u (2.168)
2
and J.; A/ satisfies the elliptic PDE with boundary value g,

8
< G J.x; A/ J.x; A/ C f .x; / D 0; 8x 2 O;
(2.169)
: lim J.y; A/ D g.x/; x 2 @O;
y!x
provided J.; A/ is smooth up to boundary. v.x/ WD infA2A J.x; A/ is called the

value function.
Let us consider the following Dirichlet problem for the HJB equation:
(
inf .G u.x/ u.x/ C f .x; // D 0; x 2 O;
2 (2.170)
u.x/ D g.x/; x 2 @O:
Then, the following result holds true.

Theorem 2.8 (Verification theorem). Let u./ be a classical solution of (2.170).
Then, it is valid that
(i) u.x/ J.x; A/; 8x 2 O; 8A 2 A,
(ii) Let x0 2 O be given. If there exists A D .˝ ; F ; .Ft /; P ; W ; .// and
its response X with X .0/ D x0 , such that
0 D min.G u.X .t// u.X .t// C f .X .t/; //

2
.t /
DG u.X .t// u.X .t// C f .X .t/; .t//;
A
8t < P -a.s.; (2.171)
then
u.x0 / D J.x0 ; A /: (2.172)
From (i) and (2.172), it follows that u.x0 / D v.x0 / and A gives an optimal
control for the initial state x0 .
Regarding the classical solution of (2.170), we can refer to [E83], [E98], [Mo10],
Sect. 5.3.
Example 2.6 (Production planning problem). Consider the production planning for
one commodity, in the presence of random demand, fluctuating according to the
SDE
dZ.t/ D b dt d W; (2.173)
where the positive constant b represents the expected demand rate and and W
denote a positive constant and a real Wiener process. The firm adjusts its production
N Since the running

rate ./ and the finished products are stored in a buffer of size K.
cost f ./ and the terminal cost g./ are needed, the firm’s objective is to minimize
its expected total cost by choosing a suitable ./.
Let us formulate the problem in precise terms. Let K be the production capacity
and put D Œ0; K. Then, for A D .˝; F ; .Ft /; P; W; .//, the inventory level
X.t/ evolves according to the SDE
(
dX.t/ D .t/ dt dZ.t/ D ..t/ b/ dt C d W .t/; t > 0;
X.0/ D x:
X.t/ > 0 (resp. < 0) means a surplus (resp. backlog) of product. Since the inventory
state cannot exceed the buffer size K and the firm fixes the backlog size K.< 0/,
because a new policy is needed for a large amount of backlog, we give the payoff
by
hZ A i
J.x; A/ D Ex e s f .X A .s/; .s// ds C e A g.X A . A
// ; (2.174)
0
where A = exit time of X A from .K; K/ WD O.

We suppose that f ./ is given by
f .x; / D p.x _ 0/ C 2 ; (2.175)
with a positive constant p.

The corresponding HJB equation reads
1 2 00
u .x/ bu0 .x/ u.x/ C inf . u0 .x/ C 2 / C p.x _ 0/ D 0; 8x 2 .K; K/;
2 2
(2.176)
with the boundary conditions
u.K/ D g.K/; u.K/ D g.K/:
Since there exists a unique classical solution of (2.176), the verification theorem
asserts that the solution is equal to the value function and an optimal Markovian
policy is given by the minimum selector;
8
ˆ
ˆ 0 on fx 2 OI u0 .x/ 0g;
ˆ
< 0
O .x/ D u .x/ on fx 2 OI u0 .x/ 2 .2K; 0/g;
ˆ
ˆ 2
:̂
K on fx 2 OI u0 .x/ 2Kg:
Refer to [Mo10], Chapter 6 for related topics.

2.4 Optimal Investment Models
This section is devoted to finite time horizon optimal investment problems. We

consider a market with one bond and m. 1/ risky assets, where d factor processes,
X 1 ; : : : ; X d , determine the performance of the market. This model, called a factor
model, was introduced by Merton [Me71] (refer to [Me73, HP81, Na03]).
Suppose that the factor process X D .X 1 ; : : : ; X d / evolves according to the SDE
dX D b.t; X.t// dt C d W .t/: (2.177)
The prices of the bond and of the i -th asset are given by
(
dS 0 .t/ D S 0 .t/r.t; X.t// dt; t > 0;
(2.178)
S 0 .0/ D s 0 > 0;
and by the SDE

8
d Cm
< dS i .t/ D S i .t/ g i .t; X.t// dt C X i .t; X.t// d W j ;
ˆ
t > 0;
j
(2.179)
j D1
:̂
S i .0/ D s i > 0; i D 1; : : : ; m;
respectively, where W D .W 1 ; : : : ; W d / is the Wiener process of (2.177).

Let us consider an agent who invests at any t 2 .0; T / a proportion Pm
i
.t/ of
his/her wealth in the i -th risky asset .i D 1; : : : ; m/ and .t/ D 1 i Dl i .t/ in
0
the bond. The agent wants to maximize the expected utility from the terminal wealth
by choosing a suitable . i ./; i D 1; : : : ; m/.
As an application of our previous results, we study the problem by using a DPP
argument. We note that the martingale approach is also powerful (refer to [HP81]
and [HP83]).
2.4.1 Formulations
Let
W Œ0; T Rd 7! Rm ˝ Rd Cm ;
b W Œ0; T Rd 7! Rd ;
g W Œ0; T Rd 7! Rm ;
r W Œ0; T Rd 7! Œ0; K0
be continuous and Lipschitz continuous in the variable x 2 Rd , uniformly on Œ0; T .

We assume that
2.4 Optimal Investment Models 69
(a) a.t; x/ WD .t; x/.t; x/> is uniformly parabolic, i.e.,
y > a.t; x/y 0 jyj2 ; 8y 2 Rm ; 8t; x;
for a positive constant 0 ,

(b) .t; x/> a.t; x/1 .t; x/ is bounded,
(c) g.t; x/> a.t; x/1 g.t; x/ and .t; x/> a.t; x/1 g.t; x/ satisfy linear growth
condition in x uniformly on Œ0; T .
Let WQ D .W 1 ; : : : ; W d ; W d C1 ; : : : ; W d Cm / be a .d C m/-dimensional Wiener
process, defined on .˝; F ; P /, and put W D .W 1 ; : : : ; W d / (D first d -component
of WQ ).
Hence, the strong solution of SDE (2.177) is a diffusion process with the
generator Gt :
1
Gt D C b.t; x/ @x ; (2.180)
2
where is the d -dimensional Laplacian operator.

Further,
FtW D FtX ; 8t 2 Œ0; T ;
if X.0/ Dconstant.
Since S 0 and S i evolve according to (2.178) and (2.179), respectively, we have
Z t
S 0 .t/ D s 0 exp r.s; X.s// ds (2.181)
0
and
nZ t 1 Z t o
S i .t/ D s i exp g.s; X.s//i j.s; X.s//i j2 ds C .s; X.s//i d WQ .s/ :
0 2 0
(2.182)
g.t; X.t// and .t; X.t// are called the mean return process and the volatility
process, respectively.
We assume
g.t; x/i > r.t; x/; 8t; x; i D 1; : : : ; m: (2.183)
By using the data of X and S i ; i D 1; : : : ; m, the agent invests at any time

t 2 .0; T / a proportion
Pm
i
.t/ of its wealth in the i -th asset, i D 1; : : : ; m, and
0
.t/ D 1 i D1 .t/ in the bond. . i .t/ > 1/ and . i .t/ < 0/ stand for
i
borrowing money and selling, respectively. We call ./ D . 1 ./; : : : ; m .// an

investment strategy or an admissible strategy, if
Q
./> .; X.// 2 L1 .Œ0; T ˝; .FtW /I R1 ˝ Rd Cm /: (2.184)
Let A denote the set of all admissible strategies.

For ./ 2 A , the agent’s wealth Z ./ evolves according to the SDE
dZ ./ .t/
Z ./ .t/
X
m
D i
.t/.g.t; x.t//i dt C .t; X.t//i d WQ .t// C 0
.t/r.t; X.t// dt
i D1
D.r.t; X.t// C .t/.g.t; X.t// r.t; X.t//1m // dt C .t/> .t; X.t// d WQ .t/
(2.185)
by (2.178) and (2.179). The quantity
.t; x/ WD g.t; x/ r.t; x/1m (2.186)
is the excess rate of return from risky assets, where 1m D .1; 1; : : : ; 1/ 2 Rm .

From (2.185), we deduce that
nZ t
Z ./
.t/ Dz exp .r.s; X.s// C .s/ .s; X.s/// ds
0
Z t Z t o
1
C .s/ .s; X.s// d WQ .s/
>
j .s/> .s; X.s//j2 ds ;
0 2 0
(2.187)
where z D the initial wealth > 0.

U 2 C 2 ..0; 1// is called a utility function, if U 0 > 0 (increasing) and U 00 < 0
(concave).
We are interested in HARA (hyperbolic absolute risk aversion) utility functions
xı
power utility function: U.x/ D ; x>0 with ı < 1; ¤ 0; (2.188)
ı
and
logarithmic utility function: U.x/ D log x; x > 0: (2.189)
The agent’s objective is to maximize the expected utility from the terminal wealth,
EŒU.Z ./ .T //, by choosing an appropriate investment strategy.
2.4.2 Investment Problems for Power Utility Function
By (2.187) and (2.188), we have
zı Z t
U.Z ./
.t// D M ./
.t/ exp ı .s; X.t/; .s// ds (2.190)
ı 0
where
Z t Z
1 t
M ./
.t/ D exp ı .s/> .s; X.s// d WQ .s/ jı .s/> .s; X.s//j2 ds
0 2 0
(2.191)
and
1ı
.s; x; .s// D j .s/> .s; x/j2 C .s/ .s; x/ C r.s; x/: (2.192)
2
M is an exponential .FtWQ /-martingale, because of the boundedness of ./> ./.

./
For the computation of EU.Z ./ .t//, we apply Girsanov’s transformation;
Q
P ./
DM ./
.T / ı P on FTW : (2.193)
./
The following (i)–(iii) are valid with respect to P
Q
(i) WQ ./
given by (2.194) is an .FtW /-Wiener process, i.e.,
Z t
WQ ./
.t/ D WQ .t/ ı .s; X.s//> .s/ ds; t 2 Œ0; T : (2.194)
0
(ii) The factor process X is described by SDE.

dX.t/ D b.t; X.t// C ı N .t; X.t//> .t/ dt C d W ./
.t/; (2.195)
where W ./ is the first d -component of WQ ./

and N ./ is the m d -matrix
consisting of the first d columns of ./.
(iii) Since ./ is independent of z,
h1 Z t i
Ezx U.Z ./
.t// D zı Ex ./ exp ı .s; X.s/; .s// ds ; (2.196)
ı 0
./ ./
where E denotes the expectation w.r.t. P .
Referring to (2.195), we consider the following control problem. Let D

Rm and fix a reference probability system .˝; F ; .Ft /; P; B/Q with a .d C m/-
dimensional Wiener process B. Q For a control process ./ 2 BQ , its response Y ./
evolves according to the SDE

d Y .s/ D .b.s; Y .s// C ı N .s; Y .s//> .s/ ds C dB.s/; s 2 .t; T ; (2.197)
Y .t/ D x .2 Rd /; (2.198)
where B is the first d -dimensional component of B.Q We define the payoff j./ and
the value function u./ by
h1 Z T i
j.t; x; .// D Etx exp ı .s; Y .s/; .s// ds (2.199)
ı t
and
u.t; x/ D sup j.t; x; .//; (2.200)

./
respectively. Now let us apply the logarithmic transformation,
1
v.t; x/ D log.ıu.t; x//: (2.201)
ı
Calculating formally, we obtain
@v 1
0D C . v C ıj@x vj2 C 2b.t; x/ @x v C 2r.t; x//
@t 2
n 1ı o
C sup j > N .t; x/j2 C > ..t; x/ C ı .t;
N x/@x / : (2.202)
2Rm 2
Since the supremum is attained for D ˘O , where
1
˘O .t; x/ D fa.t; x/1 ..t; x/ C ı N .t; x/@x .t; x//g; (2.203)
1ı
we obtain
@v 1
0D C v C H.t; x; @x v.t; x// on Œ0; T / Rd ; (2.204)
@t 2
v.T; x/ D 0; x 2 Rd ; (2.205)
where
ın 2 ı o
H.t; x; p/ D jpj C p.N > a1 N /.t; x/p >
2 1ı
ı
C b.t; x/ p C f.> a1 N /.t; x/p > C p.N > a1 /.t; x/g
2.1 ı/
1
C r.t; x/ C .> a1 /.t; x/: (2.206)
2.1 ı/
2.4.3 Optimal Investment Strategy
For the existence of optimal strategy, we need to take a broader class than A . Put
Q
A2 D f ./I ./> .; X.// 2 L2 .Œ0; T ˝; .FtW /I R1 ˝ Rm g: (2.207)
By the condition (a),

Q
./ 2 L2 .Œ0; T ˝; .FtW /I Rm / if ./ 2 A2 (2.208)
and
Q
./ 2 L1 .Œ0; T ˝; .FtW /I Rm / if ./ 2 A : (2.209)
First we prepare Lemma, by using (2.208).

Lemma 2.2 (Approximation).
(i) For ./ 2 A2 , there is a sequence n ./; n D 1; 2; : : :, in A , such that
lim Z n ./
.T / D Z ./
.T / P -a.s. (2.210)
n!1
(ii)
sup Ezx ŒU.Z ./

.T // D sup Ezx ŒU.Z ./
.T //: (2.211)
A2 A
Proof. We divide the proof for (i) into three steps.

Step 1. For ./ 2 A2 , we can take a bounded l ./, such that
E1 l ./ ./12 < 2l .l D 1; 2; : : : /: (2.212)
On the other hand, for " > 0, there is a positive constant k D k" , such that

Px sup jX.s/j k < ": (2.213)
0sT
Put
l" .t/ D l .t/. k > t/; (2.214)
where

infft < T I jX.t/j kg;
k D
T if f g D empty:
Then l:" ./ is in A .

Step 2. We evaluate the stochastic integral term
Z T
WD .s/> .s; X.s// d WQ .s/: (2.215)
0
Replacing ./ by l ./ and l" ./, we define l and l" , respectively. Compute
Px .jl" j > "/ D Px .jl" j > "I k D T / C Px .jl" j > "I k < T /:
(2.216)
Since l .s/ D l" .s/, for all s.< T /, for k D T , one has that,
1st term in RHS D Px .jl j > "I k D T /

ˇZ T ˇ
ˇ ˇ
Px ˇ . l .s/ .s//> .s; X.s// d WQ .s/ˇ > "
0
1
2 EŒ1 l ./ ./12 c12 .1 C k 2 /; (2.217)
"
because j.s; x/j c1 .1 C jxj/.

Combining (2.212), (2.213), (2.216), and (2.217), we can take a large integer l"
such that
Px .jl" j > "/ < 2" whenever l > l" : (2.218)
Step 3. By (2.212), (2.214), and (2.218), we take a sequence n ./; n D 1; 2; : : :

in A , such that
lim 1 n ./ ./1 D 0 P -a.s. (2.219)

n!1
and
Z T Z T
lim > Q
n .s/ .s; X.s// d W .s/ D .s/> .s; X.s// d WQ .s/ P -a.s.
n!1 0 0
(2.220)
From (2.187), (2.219), and (2.220), we obtain (2.210).

(ii) (2.211) is an easy consequence of (2.210) together and Fatou’s lemma.
u
Now we are going to seek an optimal strategy.
Theorem 2.9. Assume (a)–(c). Suppose that (2.204)–(2.205) has a solution vQ 2
C 12 .Œ0; T Rd /, satisfying
j@x vQ .t; x/j K.1 C jxj/; 8t; x; (2.221)
with a positive constant K. Then
sup Ezx ŒU.Z ./

.T // D zı uQ .0; x/; (2.222)
A2
where
1
uQ .t; x/ D exp.ı vQ .t; x//: (2.223)
ı
Further, ˘O ./ given by (2.203) provides an optimal strategy.

Proof. We divide the proof into two steps.
Step 1. We show that O .t/ WD ˘O .t; X.t// is in A2 and M O ./ is an exponential
.FtWQ /-martingale.
Indeed, since ˘O ./ is the maximum selector of (2.202), (b), (c), and (2.221) show
that ˘O .t; x/> .t; x/ is linearly growing w.r.t. x uniformly in t. Hence, O ./ is in
A2 . Thus, Proposition 1.5 concludes step 1.
Step 2. We have
Ezx ŒU.Z O ./ .T // sup Ezx ŒU.Z ./

.T //: (2.224)
A
Indeed, for ./ 2 A , M ./

is an exponential .FtWQ /-martingale. Put
Z t
./
.t/ D uQ .t; X.t// exp ı .s; X.s/; .s// ds : (2.225)
0
Then Itô’s formula and (2.201) yield

h1 Z T i
Ezx ŒU.Z ./
.T // D zı Ex ./ exp ı.s; X.s/; .s// ds
ı 0
1
D zı Ex ./ Œ ./
.T / by uQ .T; / D : (2.226)
ı
Since (2.202) and (2.203) imply
Ex ./ Œ ./
.T / Ex ./ Œ ./
.0/ D uQ .0; x/ (2.227)
and
ExO ./ Œ O ./ .T / D ExO ./ Œ O ./ .0/ D uQ .0; x/; (2.228)
we have (2.224).
Now, using Lemma 2.2 we conclude that
Ezx ŒU.Z O ./ .T // D sup Ezx ŒU.Z ./

.T //;
A2
which completes the proof. t

u
Refer to [Na03, FSh00, FSh02] for details and related topics.
Example 2.7 (Linear Gaussian model). Suppose that
b.t; x/ D b0 C b1 x; .t; x/ D ;
g.t; x/ D g0 C g1 x; r.t; x/ D r;
where b0 ; b1 ; ; g0 ; g1 and r are constant and a WD > is a regular matrix. Hence

the d -dimensional factor process X and the asset prices S i ; i D 1; : : : ; m, evolve
according to the SDEs
(
dX.t/ D .b0 C b1 X.t// dt C d W .t/; t 2 .0; T ;
(2.229)
X.0/ D x .2 Rd /
and
(
dS i .t/ D S i .t/..g0i C g1i X.t// dt C i d WQ .t//; t 2 .0; T ;
(2.230)
S i .0/ D s i > 0 .i D l; : : : ; m/;
respectively.
The bond price S 0 .t/ is given by
S 0 .t/ D s 0 e rt : (2.231)
Following Theorem 2.9, we will seek an optimal investment strategy. In order

to solve the semilinear parabolic equation (2.204)–(2.205) for the Gauss model, we
assume the quadratic from of v./:
1 >
v.t; x/ D x Q.t/x C R.t/ x C S.t/; (2.232)
2
with d d symmetric matrix Q.t/; R.t/ 2 Rd and S.t/ 2 R1 . Then straightforward

computations lead to the following equations:
8
ˆ
ˆ dQ
ˆ
ˆ 0D .t/ C Q.t/K0 Q.t/
ˆ
< dt
1 (2.233)
ˆ
ˆ C K1> Q.t/ C Q.t/K1 C g > a1 g1 ; t 2 Œ0; T /;
ˆ
ˆ 2.1 ı/ 1
:̂
Q.T / D 0;
where
ı ı
K0 D ıId C N > a1 N ; K 1 D b1 C N > a1 g1 I
1ı 1ı
8
< 0 D dR .t/ C .K1 C K0 Q.t//> R.t/ C Q.t/K2 C K3 ; t 2 Œ0; T /;
dt
:
R.T / D 0;
(2.234)
where
ı 1
K 2 D b0 C N > a1 .g0 r1m /; K3 D g > a1 .g0 r1m /I
1ı 1ı 1
and
8
ˆ
ˆ dS 1 1
ˆ0 D
ˆ .t/ C trQ.t/ C R.t/> K0 R.t/
ˆ
< dt 2 2
1
ˆ
ˆ C K2 R.t/ C r C .g0 r1m /> a1 .g0 r1m /; t 2 Œ0; T /;
ˆ
ˆ 1 ı
:̂
S.T / D 0:
(2.235)
If the Riccati equation (2.233) has a solution, then we can solve (2.234) and (2.235),
and v./ given by (2.232) becomes a solution of (2.204)–(2.205) with linearly
growing @x v./.
Hence

O .t/ WD a1 g0 r1m C g1 X.t/ C ı @ N x v.t; X.t// (2.236)
gives an optimal investment strategy (refer to [KN02, W71, Wo68] for Riccati
equations).
Example 2.8 (Uniformly elliptic volatility model). We consider a random volatility
model presented in [Ph02].
Let d D m D 1. Suppose that 2 .1; 1/ and " > 0 are given. We assume the
following forms:
p p p
b.t; x/ D b0 C b1 x; .t; x/ D . c 2 x 2 C "; 1 2 c 2 x 2 C "/;
g.t; x/ D g0 C g1 x; r.t; x/ D r > 0
with constants b0 ; b1 ; c; g0 ; g1 and r.

is called the correlation between asset and the factor process. The factor
process X and the asset price S evolve according to the SDEs (2.237) and (2.238)
respectively,
dX.t/ D .b0 C b1 X.t// dt C d W .t/; (2.237)
and
p
dS.t/ DS.t/ .g0 C g1 X.t// dt C c 2 X.t/2 C " d W .t/
p p
C 1 2 c 2 X.t/2 C " dB.t/ ; (2.238)
where W and B are mutually independent real Wiener processes.

Let v./ be the logarithmic transformation of the value function. Then,
8
ˆ
ˆ @v 1 @2 v ı 2 ˇˇ @v ˇˇ2
ˆ
ˆ 0 D C C 1 C ˇ ˇ
ˆ
ˆ @t 2 @x 2 2 1 ı @x
ˆ
ˆ
ˆ
ˆ
< ı g0 r C g1 x @v
C b0 C b1 x C p
1ı c 2 x 2 C " @x (2.239)
ˆ
ˆ
ˆ
ˆ 1 jg0 r C g1 xj2
ˆ
ˆ C C r; t 2 Œ0; T /; x 2 R1 ;
ˆ
ˆ 2.1 ı/ c2 x2 C "
ˆ
:̂
v.T; x/ D 0; x 2 R1
holds by (2.204). Since [Ph02], Theorem 4.1 provides a solution v./ 2 C 12 .Œ0; T /
R1 / \ C.Œ0; T R1 / with @x
@v
satisfying the linear growth condition, we conclude
that
1 g r C g X.t/ @v
0 1
O .t/ D p p C ı .t; X.t//
c 2 X.t/2 C " c 2 X.t/2 C " @x
is an optimal investment strategy.

Chapter 3
Viscosity Solutions for HJB Equations
Abstract The theory of viscosity solutions was originated by M.G. Crandall and
P.L. Lions in the early 80s for the Hamilton–Jacobi equations and later P.L.
Lions developed it for the HJB equations (Lions, J Commun PDE 8:1101–1134,
1983; Acta Math 16:243–278, 1988; Viscosity solutions of fully nonlinear second-
order equations and optimal stochastic control in infinite dimensions. Part II.
Optimal control of Zakai equation. In: Da Prato, Tubaro (eds) Stochastic partial
differential equations and applications II. Lecture notes in mathematics, vol 1390.
Springer, Berlin/Heidelberg, 1989, pp 147–170, 1989; J Funct Anal 86:1–18, 1989).
In Chap. 2, we have seen the relation between the value function and the HJB
equations. If the value function is smooth, then it provides the classical solution
of the HJB equations. Unfortunately, when the diffusion coefficient is degenerate,
smoothness does not necessarily hold, even for a simple case, and the HJB equations
may in general have no classical equation, either. However, the theory of viscosity
solutions gives a powerful tools for studing stochastic control problems. Regarding
the viscosity solutions for the HJB equations, we claim only continuity for a
solution, not necessarily differentiability. Thus, it has been shown that under mild
conditions the value function is the unique viscosity solution of the HJB equation.
We will revisit this fact in terms of semigroups in Sect. 3.1.3.
This chapter is organized as follows. In Sects. 3.1 and 3.2, we recall some basic
results on viscosity solutions for (nonlinear) parabolic equations for later use. In
Sect. 3.3 we consider stochastic optimal control-stopping problems in a framework
similar to that of finite time horizon controls.
3.1 Formulations
Let F W Œ0; T Rd S d Rd R1 7! R1 be continuous and satisfy the following

two conditions .d1 / and .d2 /;
.d1 / Monotonicity
F .t; x; A; p; z/ F .t; x; A; p; zO/ if z zO:

.d2 / Ellipticity
O p; z/
F .t; x; A; p; z/ F .t; x; A; if A AO 2 SC
d
.i.e., AO A/:

80 3 Viscosity Solutions for HJB Equations
Here we are concerned with the Cauchy problem for the (non-linear) parabolic
equation
@t U.t; x/ C F .t; x; @xx U.t; x/; @x U.t; x/; U.t; x// D 0; .t; x/ 2 Œ0; T / Rd ;
(3.1)
U.T; x/ D .x/; x 2 Rd ; (3.2)

where 2 C.Rd /.
We first give the definition of viscosity solutions based on parabolic differentials
in Sects. 3.1.1. In 3.1.2 we provide some convenient equivalent definitions. Then we
deal with the HJB equations by using the semigroups related to DPP.
3.1.1 Definition of Viscosity Solution Based on Parabolic

Differentials
Definition 3.1. (a) Parabolic differentials

(i) For W 2 US C.Œ0; T Rd / and .t; x/ 2 Œ0; T / Rd , we set
n
P 2C W .t; x/ D .q; p; A/ 2 R1 Rd S d ;
W .s; y/W .t; x/q.s t/p .y x/ 12 .y x/> A.y x/ o

lim sup 0 :
.s;y/!.t;x/ js tjCjyxj2
s¤0
(3.3)
P 2C W .t; x/ is called the parabolic superdifferential of W ./ at .t; x/.
(ii) Similarly, for V 2 LS C.Œ0; T Rd / and .t; x/ 2 Œ0; T /Rd , the parabolic
subdifferential of V ./ at .t; x/ is given by
n
P 2 V .t; x/ D .q; p; A/ 2 R1 Rd S d I
V .s; y/V .t; x/q.s t/p .y x/ 12 .y x/> A.y x/ o

lim inf 0 :
.s;y/!.t;x/ js tjCjy xj2
s¤0
(3.4)
(b) Closure of parabolic differentials
(i) For W 2 US C.Œ0; T Rd /, we put
PN 2C W .t; x/ D f.q; p; A/ 2 R1 Rd S d I
9.tn ; xn / 2 .0; T / Rd and .qn ; pn ; An / 2 P 2C W .tn ; xn /
such that lim .tn ; xn ; W .tn ; xn /; qn ; pn ; An / D .t; x; W .t; x/; q; p; A/g:
n!1
(3.5)
3.1 Formulations 81
(ii) For V 2 LS C.Œ0; T Rd /; PN 2 V .t; x/ is defined in the same fashion

as (3.5) with P 2C replaced by P 2 .
The following properties are clear:
(
P 2 W .t; x/ D P 2C .W /.t; x/;
(3.6)
PN 2 W .t; x/ D PN 2C .W /.t; x/;
for f 2 C 12 ..0; T / Rd / \ C.Œ0; T Rd / and .t; x/ 2 .0; T / Rd ,
.q; p; A/ 2 P 2˙ W .t; x/
” .q C @t f .t; x/; p C @x f .t; x/; A C @xx f .t; x// 2 P 2˙ .W C f /.t; x/:
(3.7)
Definition 3.2. (i) A function U 2 US C.Œ0; T Rd / is called a viscosity

subsolution of the problem (equations) (3.1)–(3.2), if
U.T; x/ '.x/; 8x 2 Rd (3.8)
and, for .t; x/ 2 Œ0; T / Rd ,
q C F .t; x; A; p; U.t; x// 0 (3.9)
whenever .q; p; A/ 2 P 2C U.t; x/.

(ii) A function V 2 LS C.Œ0; T Rd / is called a viscosity supersolution of the
problem (equations) (3.1)–(3.2), if
V .T; x/ '.x/; 8x 2 Rd (3.80 )
and, for .t; x/ 2 Œ0; T / Rd ,
q C F .t; x; A; p; V .t; x// 0 (3.90 )
whenever .q; p; A/ 2 P 2 U.t; x/.

(iii) U./ is called a viscosity solution of Eqs. (3.1) and (3.2), if it is both a viscosity
subsolution and a supersolution.
Convention. Let U 2 US C.Œ0; T Rd /. If U.t0 ; x0 / D 1, then we do not define
P 2C U.t0 ; x0 / and we suppose that (3.9) is valid at .t0 ; x0 /.
We make the same convention for (3.90 ).
From the definitions of PN 2C U.t; x/ and PN 2 U.t; x/, we can easily deduce
Proposition 3.1 (Equivalent conditions).

(i) U 2 US C.Œ0; T Rd / is a viscosity subsolution, if and only if (3.8) and (3.9)
holds for PN 2C .
(ii) V 2 LS C.Œ0; T Rd / is a viscosity supersolution, if and only if (3.80 )
and (3.90) holds for PN 2 .
Regarding the relation between the notions of classical solutions and viscosity
solutions, the following proposition says that they are consistent.
Proposition 3.2. For U 2 C 12 .Œ0; T / Rd / \ C.Œ0; T Rd /; U./ is a classical
solution of (3.1)–(3.2), if and only if it is a viscosity solution of (3.1)–(3.2).
Proof. For U 2 C 12 .Œ0; T / Rd /, it holds that
P 2C U.t; x/ 3 .q; p; A/
8
ˆ
ˆ q D @t U.t; x/ C c; where c D 0 for t ¤ 0, c 0 for t D 0;
<
” p D @x U.t; x/;
ˆ
:̂
A D @xx U.t; x/ C B; with B 2 SC d
:
Suppose that U./ is a classical solution. Then the ellipticity condition .d2 /
implies
q C F .t; x; A; p; U.t; x//

@t U.t; x/ c C F .t; x; @xx U.t; x/; @x U.t; x/; U.t; x//
D c 0:
Hence, U./ is a viscosity subsolution. Similarly U./ is a viscosity supersolution.

So, U./ is a viscosity solution.
Conversely, suppose that U./ is a viscosity solution. Then
@t U.t; x/ C F .t; x; @xx U.t; x/; @x U.t; x/; U.t; x// D 0;
because .@t U.t; x/; @x U.t; x/; @xx U.t; x// 2 P 2C U.t; x/ \ P 2 U.t; x/.
u
3.1.2 Equivalent Definitions
We will introduce an equivalent definition of viscosity solution, which is suitable

for stochastic control problems. Firstly we recall;
3.1 Formulations 83
Proposition 3.3 (See [FS06], p. 211). Let W 2 C.Œ0; T Rd / and .t; x/ 2

Œ0; T / Rd be given. The following two statements are equivalent:
(a) .q; p; A/ 2 P 2C W .t; x/;
(b) There exists u 2 C 12 .Œ0; T / Rd / \ C.Œ0; T Rd / such that
(
u.t; x/ D W .t; x/; @t u.t; x/ D q;
(3.10)
@x u.t; x/ D p; @xx u.t; x/ D A;
and W ./ u./ attains its global maximum at .t; x/.

“Global maximum” in (b) can be replaced by “global strict maximum”. Indeed,
put uQ .s; y/ D u.s; y/ C .s t/2 C .y x/4 . Then uQ ./ satisfies (3.10) and W ./ uQ ./
attains its global strict maximum at .t; x/.
The following remark is useful.
Remark 3.1. Suppose that W ./ satisfies the polynomial growth condition
jW .s; y/j K.1 C jyj2p / on Œ0; T Rd (3.11)
with K > 0 and a non-negative integer p. Let u./ be the function figuring in
Proposition 3.3 (b). Then for any 0 < l < m < 1, we can take uQ 2 C 12 .Œ0; T /
Rd / \ C.Œ0; T Rd /, such that
uQ ./ D u./ on Bl WD f.s; y/ 2 Œ0; T Rd I js tj2 C jy xj4 l 2 g; (3.12)

uQ .s; y/ D K.1 C jyj2p / on Œ0; T Rd Bm ; (3.13)
and
uQ ./ W ./ on Œ0; T Rd : (3.14)
Moreover, uQ ./ satisfies (3.10) and W ./ uQ ./ attains its global maximum at .t; x/.
Proof. Let , W Œ0; 1/ 7! Œ0; 1 be smooth functions, decreasing and increasing,
respectively, and satisfying
8 h i
< 1; 2 0; l C m ;
./ D 2 (3.15)
:
0; 2 Œm; 1/;
and
8
ˆ
< 0; 2 Œ0; l;
./ D hl C m (3.16)
:̂ 1; 2 ;1 :
2
We define uQ ./ by
uQ .s; y/ D.js tj2 C jy xj4 /u.s; y/
C .js tj2 C jy xj4 /K.1 C jyj2p /: (3.17)
Then uQ 2 C 12 .Œ0; T / Rd / \ C.Œ0; T Rd / satisfies all claimed conditions. t

u
We note that
uQ .s; y/ D K.1 C jyj2p / for jyj jxj C 1: (3.18)
The following theorem is an immediate consequence of Proposition 3.3 and

Remark 3.1.
Theorem 3.1. Let U 2 C.Œ0; T Rd / be given.
(i) Suppose U.T; x/ .x/; 8x 2 Rd .
Then the following statements are equivalent:
(a) U./ is a viscosity subsolution of (3.1)–(3.2).
(b) Let u 2 C 12 .Œ0; T / Rd / \ C.Œ0; T Rd / be given.
Suppose that U./ u./ attains its global maximum at .t ; x/ 2 Œ0; T /
Rd . Then
@t u.t; x/ C F .t; x; @xx u.t; x/; @x u.t; x/; U.t; x// 0: (3.19)
(ii) Suppose U.T; x/ .x/; 8x 2 Rd .

Then the following statements are equivalent:
(a) U./ is a viscosity supersolution of (3.1)–(3.2).
(b) Let u 2 C 12 .Œ0; T / Rd / \ C.Œ0; T Rd / be given.
Suppose that U./ u./ attains its global minimum at .t; x/ 2 Œ0; T /
Rd . Then
@t u.t; x/ C F .t; x; @xx u.t; x/; @x u.t; x/; U.t; x// 0: (3.20)
(iii) If U./ satisfies the polynomial growth condition (3.11), then we may assume
that u./ in .b/ satisfies
u.s; y/ D k.1 C jyj2p / for jyj R; s 2 Œ0; T (3.21)
for some large R, where k D K in .i/ and k D K in .ii/.

Set
˙2p .k/ Dfu 2 C 12 .Œ0; T / Rd / \ C.Œ0; T Rd /I

9R > 0; such that u.s; y/ D k.1 C jyj2p / on Œ0; T SRc g (3.22)
3.1 Formulations 85
and
[
˙2p D ˙2p .k/: (3.23)
k>0
Using Theorem 3.1, we study the HJB equation.

Theorem 3.2. Under conditions .b1 /–.b5 /, the value function v.T; t; xI /
.D Vt T .x// is a viscosity solution of the HJB equation

0 D @t U.t; x/ C H.t; x; @xx U.t; x/; @x U.t; x/; U.t; x//; .t; x/ 2 Œ0; T / Rd ;
U.T; x/ D .x/; x 2 Rd :
(3.24)
Proof. Put U.t; x/ D Vt T .x/. Then Theorem 2.1 (ii) and (2.15) yield
U 2 C.Œ0; T Rd / N C jxj2 /:
with jU.t; x/j K.1 (3.25)
N
Firstly we will show that U./ is a viscosity subsolution of (3.24). Let u 2 ˙2 .K/.
Suppose that .tN; x/
N 2 Œ0; T / Rd is a global maximizer of U./ u./ and
U.tN; x/
N D u.tN; x/.
N Fix a arbitrary reference probability system .˝; F ; .Ft /; P; W /.
Then Proposition 2.5 and DPP yield
u.tN; x/
N D U.tN; x/
N
hZ t Rs Rt i
D inf EtNxN e tN d
f .s; X.s/; .s// ds C e tN d
U.t; X.t//
./2 W tN
hZ t Rs Rt i
inf EtNxN e tN d
f .s; X.s/; .s// ds C e tN d
u.t; X.t// ;
./2 W tN
(3.26)
where for simplicity we put X D X ./ .

Since u./ is smooth, (3.26) and Itô’s formula yield
hZ t Rs
0 inf EtNxN e tN d
f .s; X.s/; .s// C @t u.s; X.s//
W
./2 tN
i
C G.s/
s u.s; X.s// .s; X.s/; .s//u.s; X.s// ds : (3.27)
The same arguments as in Proposition 2.8 show that
0 @t u.tN; x/
N C GtNu.tN; x/;
N (3.28)
i.e., U./ is a viscosity subsolution of (3.24).

Since we can prove the supersolution part in the same fashion, this completes the
proof. t
u
3.1.3 Viscosity Solutions via Semigroups
Let us revisit Theorem 3.2, according to [LN83]. Set
˙Q 2 Df 2 C 2 .Rd /I 9 positive constants K and R;

such that j .x/j K.1 C jxj2 / for jxj Rg: (3.29)
Theorem 3.3. Let .T t ; 0 t T / be a 2-parameter semigroup on CQ

(see (2.31)) with (i)–(v) of Proposition 2.7. Suppose that, for 2 ˙Q 2 and ' 2 CQ ,
1
lim fT Cs . C s'/.x/ .x/g D G .x/ C '.x/;
s!0 s
8.; x/ 2 Œ0; T / Rd : (3.30)
Then U.t; x/ WD Tt T .x/ is a viscosity solution of (3.24).

Proof. U./ is continuous on Œ0; T Rd by (v).
Since (iv) yields
Q
kU.t; /kCQ kTt T Tt T 0kCQ C kTt T 0kCQ K.kkCQ C 1/; (3.31)
we have
Q
jU.t; x/j K.kkCQ C 1/.1 C jxj /; 8t; x:
2
(3.32)
Now let us show that U./ is a viscosity subsolution of (3.24). Let u./ 2 ˙2 .
Suppose that U./ u./ attains its global maximum at .tN; x/
N 2 Œ0; T / Rd and
N N D u.tN; x/.
U.T ; x/ N Then
0 D U.tN; x/
N u.tN; x/ N u.tN; x/
N D TtNT .x/ N
D TtN tNCs U.tN C s; /.x/
N u.tN; x/
N
TtN tNCs u.tN C s; /.x/
N u.tN; x/
N .by U u 0/: (3.33)
Since u./ 2 ˙2 implies that there is R > 0, such that
e.s; x/ WD u.tN C s; x/ .u.tN; x/ C s@t u.tN; x// D 0; 8jxj R;
we have
1
lim e.s; x/ D 0 (3.34)
s!0 s
uniformly on Rd .
3.1 Formulations 87
On the other hand, (iv) yields
Q
kTtN tNCs u.tN C s; / TtN tNCs .u.tN; / C s@t u.tN; //kCQ Kke.s; /kCQ : (3.35)
From (3.34) and (3.35), it follows that
1
lim jTtN tNCs u.tN C s; /.x/ TtN tNCs .u.tN; / C s@t u.tN; //.x/j D 0 (3.36)
s!0 s
uniformly on any bounded set.

Since u.tN; / 2 ˙Q 2 and @t u.tN; / 2 CQ , (3.33) and (3.36) together with (3.30) yield
that U./ is a viscosity subsolution.
Since we can prove that U./ is a viscosity supersolution in the same fashion,
U./ is a viscosity solution. t
u
Next we will prove that the semigroup .V t ; 0 t T / satisfies (3.30),
which gives Theorem 3.2 in a different way.
Proposition 3.4. For 2 ˙Q 2 and ' 2 CQ ,
1
lim fV Cs . C s'/.x/ .x/g
s!0 s
D G .x/ C '.x/; 8.; x/ 2 Œ0; T / Rd : (3.37)
Proof. Fix a reference probability system .˝; F ; .Ft /; P; W /. Then
V Cs . C s'/.x/ .x/ s'.x/

D inf fJ. C s; ; x; ./I / .x/
./2 W
R Cs
C sE x Œe d
'.X ./ . C s// '.x/g: (3.38)

R Cs
lim E x Œe d
'.X ./ . C s// '.x/ D 0; (3.39)
s!0
uniformly in ./ 2 W .
Further, Proposition 2.8 yields
1
lim .V Cs .x/ .x// D G .x/; 8x 2 Rd : (3.40)
s!0 s
By combining (3.39) and (3.40) with (3.38), we complete the proof. t

u
3.2 Uniqueness of Viscosity Solutions
This section deals mainly with the comparison principle between a viscosity
subsolution U./ and a viscosity supersolution V ./ of Eqs. (3.1)–(3.2).
The comparison principle asserts that
U.T; / V .T; / H) U./ V ./:
Hence the uniqueness of the viscosity solution is immediate.

First we state the Crandall–Ishii Lemma, which provides a key tool for the
proof of the comparison principle. Then, under the structural condition on F , we
will establish the comparison principle, when both U./ and V ./ are bounded and
continuous (Theorem 3.5).
3.2.1 Crandall–Ishii Lemma
Let us recall Ishii’s Lemma first, which states the maximum principle for elliptic
differential equations. This result provides a fundamental tool for the theory of
viscosity solutions.
Definition 3.3. 1. Let u 2 US C.RN /. J 2C u.x/ is defined as the superdifferential
of u./ at x,
n
J 2C u.x/ D .p; A/ 2 RN S N I
u.y/ u.x/ p .y x/ 12 .y x/> A.y x/ o

lim sup 0 :
y!x jy xj2
(3.41)
Its closure JN 2C u.x/ is defined as
JN 2C u.x/ Df.p; A/ 2 RN S N I
9xn and .pn ; An / 2 J 2C u.xn / such that
lim .xn ; u.xn /; pn ; An / D .x; u.x/; p; A/g: (3.42)
n!1
2. Let v 2 LS C.RN /. The subdifferential of v./ at x; J 2 v.x/, and its closure

JN 2 v.x/ are given by
J 2 v.x/ D J 2C .v/.x/
3.2 Uniqueness of Viscosity Solutions 89
and
JN 2 v.x/ D JN 2C .v/.x/;
respectively.
Ishii’s Lemma
Let ui 2 US C.RN /; i D 1; 2, and 2 C 2 .RN RN /. Suppose that u1 .x1 / C

u2 .x2 / .x1 ; x2 / attains its local maximum at .xN 1 ; xN 2 /. Then, for any " 2 .0; 1/,
there exist Xi 2 S N ; i D 1; 2, such that
.@xi .xN 1 ; xN 2 /; Xi / 2 JN 2C ui .xN i /; i D 1; 2; (3.43)
and
1 X1 0
C kAk I2N A C "A2 ; (3.44)
" 0 X2
where
@x1 x1 .xN 1 ; xN 2 / @x1 x2 .xN 1 ; xN 2 /

AD and kAk D sup y > Ay: (3.45)
@x2 x1 .xN 1 ; xN 2 / @x2 x2 .xN 1 ; xN 2 / jyjD1
See [CIL92], Appendix, [Ko04], p. 71, [Mo10], Theorem 4.4.6 for a proof.
For parabolic differential equations, Crandall and Ishii showed a similar fact, by
using Ishii’s lemma.
Crandall-Ishii Lemma [CI90]
Let ui 2 US C.Œ0; T Rd /; i D 1; 2, and 2 C 12 .Œ0; T Rd Rd / .

Suppose that for any M > 0 there is a constant c.M / such that, for .t; x/ 2
Œ0; T / Rd ,
.q; p; A/ 2 P 2C ui .t; x/
and jxj C jui .t; x/j C jpj C jAj M H) q c.M /: (3.46)
Then, for any " 2 .0; 1/ and any local maximizer .tN; xN 1 ; xN 2 /.2 .0; T / Rd Rd / of
u1 .t; x1 / C u2 .t; x2 / .t; x1 ; x2 /, there exist qi 2 R1 and Xi 2 S d ; i D 1; 2, for
which
.qi ; @xi .tN; xN 1 ; xN 2 /; Xi / 2 PN 2C ui .tN; xN i /; i D 1; 2; (3.47)

q1 C q2 D @t .tN; xN 1 ; xN 2 /; (3.48)
1 X1 0
C kAk I2d A C "A2 ; (3.49)
" 0 X2
@x1 x1 .tN; xN 1 ; xN 2 / @x1 x2 .tN; xN 1 ; xN 2 /
where A D :
@x2 x1 .tN; xN 1 ; xN 2 / @x2 x2 .tN; xN 1 ; xN 2 /
We can see that condition (3.46) is satisfied when ui .i D 1; 2/ is a viscosity

subsolution of (3.1)–(3.2).
Since we need the Crandall–Ishii lemma for ui 2 C.Œ0; T Rd / with the linear
growth condition
jui .t; x/j K0 .1 C jxj/; 8.t; x/ 2 Œ0; T Rd ; (3.50)
we will show the following theorem, following [CI90].

Theorem 3.4. Let ui 2 C.Œ0; T Rd / satisfying (3.50) be given. Suppose that
ui ; i D 1; 2, satisfy the condition (3.46). Put
c
.t; x1 ; x2 / D jx1 x2 j2 C ˇ.T t/ C .jx1 j2 C jx2 j2 /; (3.51)
2 2
with positive constants ; ˇ and c.

Then for any " > 0 and any global maximizer .tN; xN 1 ; xN 2 / 2 .0; T / Rd Rd
of u1 .t; x1 / C u2 .t; x2 / .t; x1 ; x2 /, there exists qi 2 R1 and Xi 2 S d ; i D 1; 2,
satisfying

.q1 ; .xN 1 xN 2 / C c xN 1 ; X1 / 2 PN 2C u1 .tN; xN 1 /;
(3.470)
.q2 ; .xN 1 xN 2 / C c xN 2 ; X2 / 2 PN 2C u2 .tN; xN 2 /;
q1 C q2 D ˇ; (3.480)
1 X1 0
C kAk I2d A C "A2 ; (3.490)
" 0 X2
. C c/Id Id
where A D :
Id . C c/Id
Outline of Proof. We divide the proof into three steps.

Step 1. Reductions
Put
ui .t; xi / D 1 for t … Œ0; T ; i D 1; 2: (3.52)

Set
c ˇ
uQ i .t; xi / D ui .t; xi / jxi j2 .T t/ (3.53)
2 2
and

.xi ; x2 / D jx1 x2 j2 : (3.54)
2
Define !i 2 US C.R1 Rd / by
!i .t; xi / DQui .t C tN; xi C xN i / uQ i .tN; xN i /

r
@xi .xN 1 ; xN 2 / xi .jxi j4 C jtj4 /; (3.55)
2
with r > 0.
Then (3.50) and (3.55) imply
!i .t; xi / < 1 for 8t and jxi j N0 ; i D 1; 2; (3.56)
with some N0 D N0 .c/.

Further, .0; 0; 0/ is the unique maximizer of !1 .t; x1 /C!2 .t; x2 / .x1 ; x2 / and
8
ˆ
< !1 .0; 0/ D !2 .0; 0/;
(3.57)
:̂ !1 .t; x1 / C !2 .t; x2 / < jx1 x2 j2 for .t; x1 ; x2 / … .0; 0; 0/:
2
Step 2. Associated Ishii’s Lemma

For ı > 0, we define ' and ˚ by
1
'.t1 ; x1 ; t2 ; x2 / D jx1 x2 j2 C jt1 t2 j2 (3.58)
2 2ı
and
˚.t1 ; x1 ; t2 ; x2 / D !1 .t1 ; x1 / C !2 .t2 ; x2 / '.t1 ; x1 ; t2 ; x2 /; (3.59)
respectively. Then ˚ attains its global maximum at some point z.ı/ D .t1 .ı/; x1 .ı/;
t2 .ı/; x2 .ı//.
Since ˚.0; 0; 0; 0/ D 0, (3.52) and (3.56) yield
ti .ı/ 2 ŒtN; T tN; xi .ı/ 2 SN0 ; 8ı > 0: (3.60)

By Ishii’s Lemma, we have Xi WD Xi .ı; "/ 2 S 1Cd such that
..@ti '.z.ı//; @xi '.z.ı///; Xi / 2 JN 2C !i .ti .ı/; xi .ı// (3.61)
and (3.44) hold.

Hence (3.3) indicates that Zi WD Zi .ı; "/ (D minor matrix obtained by deleting
the first row and the first column of Xi ) gives
.@ti '.z.ı//; @xi '.z.ı//; Zi / 2 PN 2C !i .ti .ı/; xi .ı// (3.62)
and
1 Z1 0
C 2 I2d O
.1 C 2"/A; (3.63)
" 0 Z2
Id Id
with AO D .
Id Id
Step 3. Limit of z.ı/ as ı ! 0
We can easily see that !i ./ also satisfies (3.46), with a different c.M /. Hence,
from (3.60) and (3.46) it follows that there is a positive constant c0 such that, for
any ı > 0,
8
ˆ 1
< @t1 '.z.ı// D .t1 .ı/ t2 .ı// < c0 ;
ı (3.64)
:̂ @t '.z.ı// D 1 .t1 .ı/ t2 .ı// < c0 ;
2
ı
that is
1
jt1 .ı/ t2 .ı/j < c0 ; 8ı > 0: (3.65)
ı
Next, taking into account that ˚.z.ı// ˚.0; 0; 0; 0/ D 0, we have
!1 .t1 .ı/; x1 .ı// C !2 .t2 .ı/; x2 .ı//

1
jx1 .ı/ x2 .ı/j2 C jt1 .ı/ t2 .ı/j2 : (3.66)
2 2ı
On the other hand, (3.60) implies that
KN WD sup !i .ti .ı/; xi .ı// < 1: (3.67)

ı>0
Combining (3.60), (3.65), and (3.63) together, we can choose ıj ; j D 1:2; : : :

tending to 0, so that
lim t1 .ıj / D lim t2 .ıj / DW .2 ŒtN; T tN/; (3.68)

j !1 j !1
lim xi .ıj / DW j .2 SN0 /; i D 1; 2; (3.69)

j !1
1
lim .t1 .ıj / t2 .ıj // DW q .jqj c0 /; (3.70)
j !1 ıj
and
lim Zi .ıj ; "/ DW Zi ."/; i D 1; 2; (3.71)

j !1
where Zi ."/; i D 1:2, satisfy (3.63).

Finally we show that
D 0; i D 0; i D 1; 2: (3.72)
Indeed, (3.68) and (3.69) together with (3.66) yield

!1 .; 1 / C !2 .; 2 / j1 2 j2 : (3.73)
2
However, (3.73) contradicts (3.57) if .; i ; 2 / ¤ .0; 0; 0/. Thus (3.72) is valid.
Collecting the above results, we see that
(
.q; 0; Z1 ."// 2 PN 2C !1 .0; 0/;
(3.74)
.q; 0; Z2 ."// 2 PN 2C !2 .0; 0/:
Coming back to ui ./, we conclude the proof of the theorem. t

u
3.2.2 Structural Condition
Let us introduce the structural condition on F related to (3.63).
Structural Condition
If A; B 2 S d and > 1 satisfy
Id 0 A 0 Id Id
3 3 ; (3.75)
0 Id 0 B Id Id
then there is a modulus function mF , such that
F .t; y; B; .x y/; z/ F .t; x; A; .x y/; z/

mF .jx yj2 C jx yj C jzjjx yj/; 8t; x; y; z: (3.76)
We give two examples.

Example 3.1 (HJB equation). Let be a parameter set. Suppose
˛ W Œ0; T Rd 7! Rd ˝ Rm ;
W Œ0; T Rd 7! Œ0; c0 ;
f W Œ0; T Rd 7! R1 (3.77)
are Lipschitz continuous w.r.t. x 2 Rd , uniformly on Œ0; T . Put a D ˛˛ > .

Then F given by
F .t; x; A; p; z/
1
D sup tr.a.t; x; /A/ b.t; x; / p C .t; x; /z f .t; x; / (3.78)
2 2
satisfies the structural condition.

Example 3.2 (Isaacs equation). Replacing of Example 3.1 by I II , and
giving ˛; b; ; f and a in the same way, we define F by
F .t; x; A; p; z/
n 1
D inf sup tr.a.t; x; I ; II /A/
I 2 I II 2 II 2
o
b.t; x; I ; II / p C .t; x; I ; II /z f .t; x; I ; II / : (3.79)
Then F satisfies the structural condition.

Indeed, we denote the k-th column vectors of ˛.t; x; / and ˛.t; y; / by k

and k , respectively ( stands for . I ; II / in Example 3.42). Then the Lipschitz
condition ensures that there is a constant c1 , such that
X
m

jk k j2 c1 jx yj2 ; 8: (3.80)
kD1
Observing that
tr.a.t; x; /A/ tr.a.t; y; /B/

X
m
> >
D .k Ak k Bk /
kD1
X
m >
k Id Id k
3 .by (3.75)/
k Id Id k
kD1
X
m

D 3 jk k j2 3c1 jx yj2 .by (3.80)/;
kD1
we conclude that the structural condition is satisfied.
3.2.3 Comparison Principle
In this subsection we compare a viscosity subsolution and a viscosity supersolution

of (3.1)–(3.2). Since we are mainly concerned with the HJB equation and the Isaacs
equation, we assume, besides .d1 / and .d2 /, that the following condition is satisfied.
.d3 / F .t; x; A; p; z/ is Lipschitz continuous in p; A and z, uniformly in .t; x/,
say
O p;
jF .t; x; A; p; z/ F .t; x; A; O zO/j
O C jp pj
l0 fjA Aj O C jz zOjg; 8t; x; (3.81)
with a constant l0 > 0.

Theorem 3.5 (Comparison Principle). Assume .d1 /–.d3 / and the structural con-
dition. Let U and V 2 Cb .Œ0; T Rd / be a viscosity subsolution and supersolution
of (3.1)–(3.2), respectively. Then
U.t; x/ V .t; x/; 8.t; x/ 2 Œ0; T Rd : (3.82)
Proof. We divide the proof into three steps. Suppose
jU.t; x/j C jV .t; x/j K; 8.t; x; y/ 2 Œ0; T Rd Rd : (3.83)
Step 1. Preparation
Let us suppose the contrary, namely there is .; z/ 2 .0; T / Rd , such that
44 WD U.; z/ V .; z/ > 0: (3.84)


For > 0, we put 0 D 1 and

U .t; x/ WD U.t; x/ ; .t; x/ 2 Œ0; T Rd : (3.85)
t
Then U 2 US C.Œ0; T Rd / and, for .t; x/ 2 .0; T / Rd ,

.q; p; A/ 2 P 2C U.t; x/ ” q C 2 ; p; A 2 P 2C U .t; x/:
t

Thus, putting q D q C t2
, condition .d1 / yields
q C F .t; x; A; p; U .t; x//

q C F .t; x; A; p; U.t; x// < 0: (3.86)
t2
Hence U is a viscosity subsolution of (3.1)–(3.2).
Next we define UQ and VQ by
UQ .t; x/ D U .t; x/ ".1 C jxj2 / (3.87)
and
VQ .t; x/ D V .t; x/ C ".1 C jxj2 /; (3.88)
respectively, where and " are positive constants satisfying

2 .0; 4/ and 2".1 C jzj2 / 2 .0; 4/: (3.89)
^1
Then from (3.84) and (3.89) it follows that
UQ .; z/ VQ .; z/ > 24 > 0: (3.90)
For ˇ 2 .0; 4
T
/ and > 1, we put

'.t; x; y/ D jx yj2 C ˇ.T t/ . 0/ (3.91)
2
and
˚.t; x; y/ D UQ .t; x/ VQ .t; y/ '.t; x; y/: (3.92)
Then there exists a global maximizer of ˚, say .tNˇ" ; xN ˇ" ; yNˇ" /, because (3.83)
leads to
lim UQ .t; x/ VQ .t; y/ D 1

jxjCjyj!1
uniformly in t.
Step 2. Limit of the maximizer .tNˇ" ; xN ˇ" ; yNˇ" / as ! 1

We omit the subscripts ; ˇ; " and , if no confusion occurs.
First we consider xN and y.
N Noting that
UQ .tN; x/
N VQ .tN; y/
N ˚.tN; x;
N y/
N ˚.; z; z/ 4 (3.93)
and
UQ .t; x/ VQ .t; y/ K ".jxj2 C jyj2 /; (3.94)
we have
K
jxj
N 2 C jyj
N2 DW C 2 : (3.95)
"
On the other hand, since

0 ˚.tN; x; N ˚.tN; x;
N y/ N D VQ .tN; y/
N x/ N C VQ .tN; x/
N jxN yj
N 2; (3.96)
2
(3.83) and (3.95) yield

N 2 V .tN; y/
jxN yj N C V .tN; x/
N C ".jxj
N 2 C jyj
N 2 / 3K: (3.97)
2
Next we consider tN. Since
lim UQ .t; x/ VQ .t; y/ D 1 uniformly on SC SC ;

t !0
there is t0 D t0 ."; / > 0 such that
tNˇ" t0 ; 8: (3.98)
Let us take t1 D t1 ."; / 2 . T2 ; T /, such that, for t > t1

jU.t; x/ .x/j C jV .t; y/ .y/j < ; 8x; y 2 SC : (3.99)
2T
Then for t > t1 ,

N y/
˚.t; x; N < .x/
N .y/
N C < .x/
N .y/:
N (3.100)
2T t
Since ./ is uniformly continuous on SC , (3.97) provides a large 0 ."/, such that
j.x/
N .y/j
N < for > 0 ."/: (3.101)
Hence (3.100), (3.101), and (3.93) yield
tNˇ" t1 for > 0 ."/: (3.102)
Finally, we consider the limit of the maximizer. Fix ˇ; ", and . By (3.95), (3.97),
(3.98), and (3.102), we can take n ; n D 1; 2 : : : , tending to 1, so that
lim xN n ˇ" D lim yNn ˇ" 2 SC (3.103)

n!1 n!1
and
lim tNn ˇ" DW tNˇ" 2 Œt0 ; t1 : (3.104)

n!1
By using (3.96), (3.103) and (3.104), we have

n
jxN n ˇ" yNn ˇ" j2 VQ .tNn ˇ" ; xN n ˇ" / VQ .tNn ˇ" ; yNn ˇ" / ! 0
2
as n ! 1: (3.105)
Step 3. Contradiction to the hypotheses (3.84)

Let .t; x/ 2 .0; T / Rd . We note the following facts:
.q; p; A/ 2 PN 2C UQ .t; x/
” .q; p C 2"x; A C 2"Id / 2 PN 2C U .t; x/; (3.106)
jF .t; x; A; p; U .t; x// F .t; x; A C 2"Id ; p C 2"x; U .t; x//j

p
2l0 ". d C jxj/ .by (3.81)/; (3.107)
and
F .t; x; A; p; UQ .t; x// F .t; x; A; p; U .t; x// .by .d1 //: (3.108)
Since U ./ is a viscosity subsolution, (3.106)–(3.108) yield

p
q C F .t; x; A; p; UQ .t; x// 2l0 ". d C jxj/: (3.109)
O p;
Similarly, for .q; O 2 PN 2 VQ .t; y/,
O A/
p
O p;
qO C F .t; y; A; O VQ .t; y// 2l0 ". d C jyj/: (3.110)
Now we apply Theorem 3.4 to ˚.t; x; y/. For tN D tNˇ" ; xN D xN ˇ" , and yN D
yNˇ" , we take q; qO 2 R1 and A; AO 2 S d such that
N A/ 2 PN 2C UQ .tN; x/;
.q; .xN y/; N
O .xN y/;
.q; N A/O 2 PN 2 VQ .tN; y/;
N
q qO D ˇ;
Id 0 A 0 Id Id
3 3 :
0 Id 0 AO Id Id
Referring to (3.109) and (3.110), we have

p
q C F .tN; x; N UQ .tN; x//
N A; .xN y/; N 2l0 ". d C jxj/
N (3.111)
and
p
qO C F .tN; y; O .xN y/;
N A; N VQ .tN; y/
N 2l0 ". d C jyj//:
N (3.112)
Subtracting (3.111) from (3.112) we get
F .tN; y; O .xN y/;

N A; N VQ .tN; y//
N F .tN; x; N UQ .tN; x//
N A; .xN y/; N
p
2l0 ".2 d C jxj N C jyj/
N C ˇ: (3.113)
On the other hand, the structural condition implies that
LHS of (3.113)
F .tN; y; O .xN y/;

N A; N VQ .tN; y//
N F .tN; x; N VQ .tN; y//
N A; .xN y/; N
(by (3.93) and .d1 /)
mF .jxN yj N C jVQ .tN; y/jj
N 2 C jxN yj N xN yj/:
N (3.114)
Thus, we obtain, letting ! 1 in (3.113),

p
0 4l0 ". d C C / C ˇ; (3.115)
q
thanks to (3.113), (3.114), and (3.103). Since C D K
", letting " ! 0 yields
“0 ˇ”. Since ˇ > 0, this completes the proof. t
u
The following uniqueness result is immediate from Theorem 3.5.
Theorem 3.6. For the HJB equation (3.24), we assume that ˛; b; and f are
bounded, continuous, and Lipschitz continuous in x, uniformly in .t; /. Then,
for 2 Cb .Rd /, the value function gives the unique bounded viscosity solution
of (3.24).
3.3 HJB Equations for Control-Stopping Problems
In Sect. 3.3, we control not only the dynamics of stochastic system, but also the
terminal time of its evolution. Accordingly, here an admissible system consists a
pair of admissible control A D .˝; F ; .Ft /; P; W; .// and an .Ft /-stopping time
. By A, we denote the set of all admissible systems. We will deal with this problem
in the same framework as the control problem.
Firstly we formulate control-stopping problems in Sect. 3.3.1. In Sects. 3.3.2
and 3.3.3, we study the DPP and the viscosity solutions for the HJB equations via
semigroup arguments respectively. Section 3.3.4 deals with the American option
price problem as an example.
3.3.1 Formulations
We assume that all coefficients ˛; b; and f are time independent and that
conditions .b1 /–.b5 / in Sect. 2.1 are satisfied. .A; / is called an admissible system
if A D .˝; F ; .Ft /; P; W; .// is an admissible control and is a Œ0; T -valued
.Ft /-stopping time. A denotes the set of all admissible systems.
For .A; / 2 A, we have the following SDE (3.116) for the response X A and the
payoff j./ in the control-stopping problem:
dX.t/ D b.X.t/; .t// dt C ˛.X.t/; .t// d W .t/; t 2 .0; T ; (3.116)
and
j.t; x; .A; /I / WD J.t ^ ; 0; x; AI / D E0x C.t ^ ; AI /; (3.117)
where the cost functional C./ is given by
C.s; AI /
Z s Z
D expf .X A .h/; .h// dhgf .X A ./; .// d
0 0
Z s
C expf .X A .h/; .h// dhg.X A.s//:
0
The value function is defined by
v.t; xI / D inf j.t; x; .A; /I /: (3.118)

.A; /2A
Thus, we aim to analyze the value function and find an optimal admissible system.
For simplicity, we assume that T is an integer.
3.3 HJB Equations for Control-Stopping Problems 101
Before we study the control-stopping problems, we shall summarize the proper-

ties of v./. Let 2 CQ and "; R > 0 be given. Then there are positive constants
Q ı"R and "R , such that
K;
Q C jxj2 /;
sup jv.t; xI /j K.1 (3.119)
0t T
sup jv.t; xI / v.t; yI /j < " (3.120)

0t T
for jx yj < ı"R and x; y 2 SR , and
jv.t; xI / v.; xI /j < " (3.121)
for jt j < "R and x 2 SR .

From (3.119)–(3.121), it is follows that
v.t; I / 2 CQ ; 8t 2 Œ0; T : (3.122)
Proposition 3.5 (Dependence on the terminal cost function). Suppose that

; n 2 CQ satisfy kkCQ ; kn kCQ K .n D 1; 2; : : : / and n converges to
uniformly on any bounded set. Then, for any R > 0,
lim sup jv.t; xI n / v.t; xI /j D 0: (3.123)

n!1 0t T;jxjR
From now on, we put Ex D E0x , for simplicity.

Proof. We have
jv.t; xI n / v.t; xI /j sup Ex jn .X A .t ^ // .X A .t ^ //j

.A /2A
h i
sup Ex sup jn .X A .s// .X A .s//j : (3.124)
A2A 0sT
By the assumptions, we can choose N."; R/ so that
sup jn .y/ .y/j < " for n > N."; R/:
y2SR
Hence
h i
Ex sup jn .X A .s// .X A .s//j
0sT
h i
" C 2KEx 1 C sup jX A .s/j2 I sup jX A .s/j R
0sT 0sT
s q
2 Ex Œsup0sT jX A .s/j2
" C c1 Ex 1 C sup jX A .s/j2
0sT R
1 C jxj3
"c2 (by (2.9)) (3.125)
R
with constants c1 and c2 independent
p
of A.
1C R
Taking R so that “c2 R < "”, we conclude that, for n > N."; R/,
1
sup jv.t; xI n / v.t; xI /j < 2" for jxj < R 6 :
0t T

u
3.3.2 DPP
Let us consider discrete-time DPP firstly. By AN , we denote the set of .A; / 2 A

whose control process is switching at fk2N I k D 1; 2; : : : ; 2N T 1g and with
stopping time taking values in fk2N I k D 0; 1; : : : ; 2N T g. Put
vN .t; xI / D inf j.t; x; .A; /I / for 2 CQ : (3.126)

.A /2AN
We sometimes write ../; / instead of .A; /, when there is no danger of

confusion.
Proposition 3.6. Let be convex and compact. Then
vN .t C ; xI / D inf j.t; x; .A; /I vN .; I //

.A; /2AN
D vN .t; xI vN .s; I // for t D 2N i; D 2Nj; t C T:

(3.127)
Before we start the proof, we prepare two lemmas. First of all, we introduce the
mapping jN W CQ ! CQ , according to [S08], (2.6), [N81], 2.4.1 [N78]. Put 4 D 2N
and fix a reference probability system .˝; F ; .Ft /; P; W /.
Let us define N and jN W CQ ! CQ by
N .x/ D inf Ex C.4; I / (3.128)

2
and
jN .x/ D .x/ ^ N .x/: (3.129)
That is to say, jN is the expected optimal value at one step.

Put jN0 D identity and jNkC1 D jN .jNk /; k D 0; 1; : : : ; 2N T 1.

Lemma 3.1.
jNkC1 D ^ N .jNk /; k D 0; 1; 2; : (3.130)
Proof. We use induction on k. For k D 0, (3.130) is the definition of jN . Suppose

that (3.130) holds for k. Then
jNkC2 D jN .jNkC1 / D jNkC1 ^ N .jNkC1 /

D ^ N .jNk / ^ N .jNkC1 / (3.131)
by the hypothesis for k.

Observing that jNkC1 jNk and N is monotone, we obtain (3.131) for k C 1.
u
In a way similar to that described in Sect. 2.2.4, we have
Lemma 3.2. Let 2 CQ , an integer k, and x 2 Rd be given. Then there exist .FtW /-
progressively measurable switching control process ./ and an .FtW /-stopping
time such that . ./; / 2 AN and
jNk .x/ D j.k2N ; x; . ./;

/I / D J.k2N ^
; x; ./I /: (3.132)
Proof. We divide the proof into three steps. Fix a reference probability system
.˝; F ; .Ft /; P; W /. By SW we denote the set of all .FtW /-stopping times with
values in Œ0; T .
Step 1. Construction of ./ on the reference probability system
For 2 CQ , J.4; y; I / is continuous w.r.t. .y; / 2 Rd . Since is
compact, there exists a minimum selector O W Rd ! , such that
J.4; y; O .yI /I / D inf J.4; y; I / D N .y/: (3.133)

2
Define ./ by
.t/ D O .xI jNk1 /; 8t 2 Œ0; 4/: (3.134)
Denoting by X its response, we have

(
dX .t/ D b.X .t/; .0// dt C ˛.X .t/; .0// d W .t/; t 2 .0; 4;
X .0/ D x:
(3.135)
Putting .t/ D O .X .4/I jNk2 /, for t 2 Œ4; 24/, we have its response X on
Œ0; 24.
Repeating this procedure successively, we obtain a switching control ./ 2

and its response X on the time interval Œ0; k4.
W
From (3.133), it follows that
J.4; x; ./; jNk1 / D N .jNk1 /.x/: (3.136)

Step 2. Construction of 2 SW
Put
D.l/ D fz 2 Rd I .z/ D jNl .z/g; l D 0; 1; : : : ; k: (3.137)
Then D.0/ D Rd and the closed set D.l/ is decreasing. Thinking of D.k l/ as
the stop region at time l4, we define by

.!/ D minfi 4I X .i 4; !/ 2 D.k i /g
D minfi 4I .X .i 4; !// D jNki .X .i 4; !//g: (3.138)

Since jN0 D , is an .FtW /-stopping time taking values in f0; 4; : : : ; k4g.
Step 3. . ./;
/2 W
SW satisfies (3.132). Note that the triviality of F0W
yields

Px . D 0/ D 1 or 0: (3.139)

For x 2 D.k/, we have Px . D 0/ D 1 and
jNk .x/ D .x/ D j.k4; x; . ./;

/I /: (3.140)
For x … D.k/, it holds that

Px . 4/ D 1: (3.141)
Let us compute the RHS of (3.132):

J WD Ex ŒC. ; ./I /
X
k
D Ex ŒC.l4; ./I /I
D l4
lD1
X
k
DW Jl : (3.142)
lD1

Observing that . D k4/ D . .k 1/4/c 2 F.k1/4
W
and
Z k4 Z s
Ex expf ./ dgf .X .s/; O .X ..k 1/4/I // ds
.k1/4 .k1/4
Z k4
C expf ./ dg.X .k4//jF.k1/4
W
.k1/4

DN .X ..k 1/4//; (3.143)
where ./ denotes .X ./; O .X ..k 1/4/I // for simplicity, we have
Jk D Ex ŒC..k 1/4; ./I N /I

D k4: (3.144)
Therefore,
Jk C Jk1 D Ex ŒC..k 1/4; ./I jN /I

.k 1/4: (3.145)
By using the same arguments successively, we obtain
J D Jk C Jk1 C C J1
D Ex ŒC.4; ./; jNk1 /I
4
D N .jNk1 /.x/ D jNk .x/; (3.146)

because x … D.k/ and Px . 4/ D 1.
Now (3.146) and (3.140) complete the proof of the lemma. t
u
Now we are in the position to prove Proposition 3.6.
Proof. First we will show that, for .A; / 2 AN ,
jNk .x/ j.k4; x; .A; /I /; k D 1; 2; : : : : (3.147)
We use induction on k. It holds that
j.4; x; .A; /I / D Ex .C.4; .0/I /I 4 C .x/Px . D 0/: (3.148)
Since . 4/ D . D 0/c 2 F0 and .0/ is F0 -measurable,
1st term of RHS of (3.148) D Ex ŒE.C.4; .0/I /jF0/I 4

Ex ŒN .x/I 4 D N .x/Px . 4/:
(3.149)
Substituting (3.149) into (3.148), we obtain (3.147) for k D 1.

Suppose that, for any 2 CQ , (3.147) is valid for k. Noting that
j..k C 1/4; x; .A; /I /

D Ex ŒC..k C 1/4; AI /I .k C 1/4 C Ex ŒC. ; AI /I k4 (3.150)
and recalling that . .k C 1/4/ D . k4/c 2 Fk4 and .k4/ is Fk4 -

measurable, we have
1st term of RHS of (3.150) Ex ŒC.k4; AI N /I .k C 1/4

Ex ŒC.k4; AI jN /I .k C 1/4: (3.151)
From (3.150) and (3.151) it follows that
j..k C 1/4; x; .A; /I / j.k4; x; .A; /I jN /

jNk .jN /.x/ (by the induction hypothesis)
D jNkC1 .x/; (3.152)
which yields (3.147) for k C 1.

Taking the infimum of the RHS of (3.147) over AN , we have
jNk .x/ vN .k4; xI /: (3.153)
Since (3.132) yields the opposite inequality, we obtain
jNk .x/ D vN .k4; xI / (3.154)
which in turn yields (3.127). t

u
Remark. By referring to Lemma 3.2, we can assert that there is a Brownian adapted
pair . ./; /, that is optimal in AN .
Next we need the following approximation result.
Lemma 3.3. Let be convex and compact. Then for binary rational t
v.t; x W / D lim vN .t; x W /; 8x 2 Rd : (3.155)

N !1
Proof. By the definitions of vN ./ and v./, vN ./ is decreasing, as N ! 1, and,

for a binary rational t,
v.t; xI / lim vN .t; xI /: (3.156)

N !1
For the converse inequality, it is enough to prove that
j.t; x; .A; /I / lim vN .t; xI /; 8.A; / 2 A: (3.157)

N !1
Let a reference probability system .˝; F ; .Ft /; P; W / be given. For an .Ft /-

progressively measurable control process ./, we can take an approximate switch-
ing control N ./ .N D 1; 2; : : :/ by Theorem 2.3, such that
N .t/ D N ..Œ2N t2N / ^ T / (3.158)
and
N ./ ! ./ (3.159)
in L2 .Œ0; T ˝/ and a.e. in Œ0; T P -a.s.

Moreover, its response XN converge to X.D response for .//,
sup jXN .t/ X.t/j ! 0 (3.160)

0t T
in L2 .˝/ and P -a.s.

Let be an .Ft /-stopping time with values in Œ0; T . Put N .!/ D T ^ Œ1 C
2N .!/2N . Then N & P -a.s. and .N ./; N / 2 AN .
By noticing that C.t ^ N ; N ./I / converges to C.t ^ ; ./I / P -a.s., the
convergence theorem together with .b4 / and (2.9) shows that
j.t; x; ../; /I / D lim j.t; x; .N ./; N /I /

N !1
lim vN .t; xI /: (3.161)

N !1
This completes the proof of the lemma. t

u
From the Remark after the proof of Lemma 3.2 it follows that
vN .t; xI / D inf j.t; x; ../; /I /

AN \. W SW /
inf j.t; x; ../; /I /: (3.162)

W SW
Consequently, Lemma 3.3 together with (3.162) yields
v.t; xI / inf j.t; x; ../; /I /:

W SW
Since the opposite inequality is clear, we have
v.t; xI / D inf j.t; x; ../; /I / (3.163)

W SW
for a binary rational t.

Finally we prove DPP for the value function v./.
Theorem 3.7. Let be convex and -compact. Suppose .b1 /–.b5 / holds. Then for
2 CQ ,
v.t C s; xI / D v.t; xI v.s; I // (3.164)
for 0 s, t s C t T , x 2 Rd .
Proof. We divide the proof into three steps.
Step 1. Let be convex and compact. Let t and s be binary, say, t D i 2p and
s D j 2p .
Since vN .; yI / is decreasing to v.; yI / as N ! 1, Proposition 3.6 and
the monotonicity property of vN ./ yield
vN .t C s; xI / D vN .t; xI vN .s; I //

vN .t; xI v.s; I // for N p; (3.165)
and, as N ! 1,
v.t C s; xI / v.t; sI v.s; I //: (3.166)
Hence we need to establish the opposite inequality of (3.166). Observing
vN Cm .t C s; xI / vN Cm .t; xI vm .s; I // (3.167)
for N p, m D 1; 2; : : : ; and, letting N ! 1, we have
v.t C s; xI / v.t; xI vm .s; I //

j.t; x; ../; /I vm .s; I //; 8../; / 2 W
SW ;
(3.168)
by (3.163). Again, using that
lim C.t ^ ; ./I vm .s; I // D C.t ^ ; ./I v.s; // P -a.s. (3.169)
m!1
and

jC.t ^ ; ./I vm .s; I /j c1 1 C sup jX.t/j2
0t T
with a constant c1 , independent of m, we obtain
lim j.t; x; ../; /I vm .s; I // D j.t; x; ../; /; v.s; I //; (3.170)
m!1
by the dominated convergence theorem. Taking the infimum over ../; / 2

W SW yields the opposite of inequality (3.166), by (3.163).
Step 2. Let t and s be given. Suppose that binary tm and sn approach t and
s, respectively. Setting n D v.sn ; I / in Proposition 3.5 and using (3.119)
and (3.121), we get
lim sup jv.tm ; xI v.sn ; I // v.tm ; xI v.s; I //j D 0: (3.171)

n!1 jxjR
Now
v.tm ; xI v.s; I // D v.tm C s; xI / (3.172)
follows from Step 1. Letting m ! 1, we obtain (3.164).

Step 3. is convex and - compact. Take a sequence of convex and compact sets
n ; n D 1; 2; : : : ; so that
[
n nC1 ; n D 1; 2; : : : and n D : (3.173)
n
S
Let AN denote the set of all .A; / with control region N . Then A D N AN
by Definition 2.1. Set
vN .t; xI / WD inf j.t; x; .A; /I /:

.A; /2AN
Since vN .t; xI / is decreasing as N ! 1, and
j.t; x; .A; /I / lim vN .t; xI / v.t; xI /; 8.A; / 2 A; (3.174)

N !1
by taking the infimum of LHS of (3.174) over A we have
v.t; xI / D lim vN .t; xI /; 8t; x: (3.175)

N !1
From DPP for vN and (3.175) we obtain (3.164) by using the same arguments as
in Step 2 of the proof of Theorem 2.4.
u
3.3.3 Semigroups Associated with DPP
Let us define a mapping Vt W CQ 7! CQ , by
Vt .x/ D v.t; xI /: (3.176)
Then by DPP .Vt I t 2 Œ0; T / is a one-parameter semigroup on CQ .

The following properties are easily verified.

Proposition 3.7.
(i) Semigroup V0 D identity, Vt Cs D Vt Vs D Vs Vt ;
(ii) Monotonicity
H) Vt Vt ; 8t 2 Œ0; T I
(iii) Lipschitz condition

9K > 0 such that
kVt Vt kCQ K k kCQ ; 8t 2 Œ0; T ; 8; 2 CQ I (3.177)
(iv) Continuity in the time parameter
lim sup jVt .x/ .x/j D 0; 8R > 0I (3.178)

t !0 jxjR
(v)

Vt and Vt Ht ; 8t 2 Œ0; T ; 8 2 ;
where Ht is the semigroup on CQ given by

Ht .x/ D Ex C.t; I /;
(vi) Maximality
Suppose that .Ut I t 2 Œ0; T / is a semigroup on CQ , satisfying (i)–(v). Then
Ut Vt ; 8t 2 Œ0; T ; 8 2 CQ :
For the generator of Vt , we have

Proposition 3.8. Put
O
G.x/ D 0 ^ inf h .x/; (3.179)
2

where h is the generator of Ht . Then for any R > 0,
ˇ1 ˇ
ˇ O ˇ
lim sup ˇ .Vt .x/ .x// G.x/ ˇD0 (3.180)
t !0 jxjR t
for any 2 CQ \ Cp2 .Rd /.

Proof. We use the same arguments as in Proposition 2.8. Fix a reference probability
system .˝; F ; .Ft /; P; W /. For simplicity, we put
I../; / D j.t; x; ../; /I / .x/ for ../; / 2 W

SW :
For 2 Cp2 .Rd /, Itô’s formula implies

Z t^ Z s
I../; / Ex expf .X./; .// dg 0 ^ h.s/ .X.s// ds
0 0
Z t
Ex 0 ^ h.s/.X.s// ds: (3.181)
0
On the other hand, Lemma 2.1 asserts that, for " and R > 0, there is t0 D t0 ."; R/
such that
sup Ex jh.s/ .X.s// h.s/.x/j < "; 8./ 2 W (3.182)

jxjR
whenever s < t0 .
Plugging (3.182) into the RHS of (3.181), we have
I../; / .0 ^ inf h .x//t "t

2
O
D .G.x/ "/t for x 2 SR ; t < t0 : (3.183)
Taking the infimum of the LHS of (3.183) over ../; / 2 W SW and dividing
both sides by t, we obtain
1 O
.Vt .x/ .x// G.x/ > "; 8x 2 SR ; t < t0 : (3.184)
t
For the converse inequality, we notice that
Vt .x/ .x/ ^ inf j.t; x; .; t/I /: (3.185)

2
Hence, for x 2 SR and t < t0 ,
Vt .x/ .x/
Z t Z s
0 ^ inf Ex expf .X./; / dgh .X.s// ds
2 0 0
hZ t Z s i
0 ^ inf Ex expf .X./; / dgh .x/ ds C "t (by (3.182))
2 0 0

0 ^ inf h .x/t C "t C sup jh .x/jt 2 c0 (by .b5 /): (3.186)
2 2
Now (3.184) and (3.186) yield (3.180). t

u
The HJB equation associated with the control-stopping problem reads and is
considered
@t v.t; x/ C 0 _ H.x; @xx v.t; x/; @x v.t; x/; v.t; x// D 0;

.t; x/ 2 .0; T Rd ; (3.187)
v.0; x/ D .x/; x 2 Rd ; (3.188)
where
1
H.x; Q; p; z/ D sup tr.a.x; /Q/ b.x; / p C .x; /z f .x; / :
2 2
Proposition 3.9. Suppose that the coefficients ˛; b; , and f are bounded, continu-
ous, and Lipschitz continuous w.r.t. x uniformly on . Then the value function is the
unique bounded continuous viscosity solution of (3.187)–(3.188), for 2 Cb .Rd /.
Proof mimics Theorem 3.3, because V t WD Vt is a 2 parameter semigroup
satisfying (3.30).
3.3.4 American Option Price
Let us apply previous results to the HJB equation arising from the problem of
American option price. Following [Ma00], we consider a time-homogeneous one-
dimensional model.
The asset price ./ evolves according to the linear SDE
d .t/ D .t/ dt C .t/ d W .t/; (3.189)
where and are positive constants and W is a real Wiener process. Let T >
0 denote the maturity time. For a Lipschitz continuous function g, the American
option gives an asset holder the right to get the amount of money g.. // with the
discount rate at his/her chosen stopping time . T /. Thus the asset holder wants
to maximize its expectation by choosing a suitable stopping time.
First we formulate the problem and recall its HJB equation. Put
WC
S D f I is an.Ft /-stopping time valued in Œ0; T g (3.190)
and
u.; x/ D sup E x Œe . /

g.. //; 2 Œ0; T ; x > 0: (3.191)
2S
u.; x/ is called the price of American option for g at . We assume that
g W .0; 1/ 7! Œ0; 1/ is Lipschitz continuous.

Put
uO .t; x/ D sup E0x Œe g.. // (3.192)

2St0
where
St0 D f 2 S0 I 2 Œ0; t P -a.s.g
is the set of .FtW /-stopping times taking values in Œ0; t.

Since the coefficients are independent of the time variable, we have
u.; x/ D uO .T ; x/: (3.193)
Hence, u./ is continuous on Œ0; T .0; 1/ and (3.187) yields that, when u./ is
smooth,
2 @2 u @u
0 D @t u C 0 ^ x 2 2 x u on Œ0; T / .0; 1/ (3.194)
2 @x @x
u.T; x/ D g.x/; x 2 .0; 1/: (3.195)
Next we revisit the problem, by changing the space variable x to y D log x. Put
f .y/ D g.e y /; .t/ D log .t/: (3.196)
Then ./ satisfies

2
d.t/ D dt C d W .t/: (3.197)
2
Namely, ./ is a Gaussian diffusion with generator
2 @2 2 @
GD C :
2 @x 2 2 @x
Since
E x Œe . /
g.. // D E log x Œe . /
f .. // (3.198)
holds, putting
v.; y/ D sup Ey Œe . /

f .. //; (3.199)
2S
we have
v.; y/ D u.; e y /; y 2 R1 : (3.200)
By (3.197) and (3.199), the HJB equation for v./ is
0 D @t v C 0 ^ .Gv.t; / C v/ on Œ0; T / R1 (3.201)
with
v.T; y/ D g.e y /; y 2 R1 :
Now Proposition 3.9 yields

Proposition 3.10. When g is bounded and Lipschitz continuous, v./ is bounded
and continuous and is the unique viscosity solution of (3.201).
Finally we note the variational inequality for v./, according to [JLL90]. When
g is convex, v.t; / is also convex. Referring to [JLL90], Theorem 3.6 and Corol-
lary 3.7, we have
Proposition 3.11. Suppose that g is convex and Lipschitz continuous. Then v./
@v
has a bounded continuous derivative @x and bounded generalized derivatives @v
@t
2
@ v
and @x 2 . Moreover,
@t v C Gv.t; / v 0 a.e. on Œ0; T / R1 (3.202)
and
.@t v C Gv.t; / v/.f v/ D 0 a.e. on Œ0; T / R1 : (3.203)
Let .; y/ be given. ./ denotes the solution of (3.197) with ./ D y. Noticing
that v f and v.T; / D f ./, and using (3.202) and (3.203), we can take an optimal
stopping time in S such that

D inffs 2 Œ; T I v.s; .s// D f ..s//g: (3.204)
Indeed, the Itô–Krylov formula together with (3.202) and (3.203) yields
^
Ey Œe . R /
v.
^ R ; .

^ R / D v.; y/; (3.205)
where R denotes the exit time of ./ from ŒR; R. Since v./ is bounded, the
bounded convergence theorem yields
/
Ey Œe . v.
; .
// D v.; y/;
which establishes the assertion.

Regarding the martingale approach to American option price problem, the reader
is referred to [KS98], Appendix D. [CPY09] and [St11] treat other topics on optimal
stopping problems.
Chapter 4
Stochastic Differential Games
Abstract In this chapter, we will deal with zero-sum two-player time-homogeneous

stochastic differential games and viscosity solutions of the Isaacs equations arising
from such games, via the dynamic programming principle.
In Sect. 4.1, we are concerned with basic concepts and definitions and we
introduce stochastic differential games, referring to (Controlled MarkovProcesses
and viscosity solutions, 2nd edn. Springer, New York 2006), XI. Then, using a semi-
discretization argument, we study the DPP for lower- and upper-value functions in
Sect. 4.2. In Sect. 4.3, we will consider the Isaacs equations, via semigroups related
to DPP. In Sect. 4.4, we consider a link between stochastic controls and differential
games via risk sensitive controls.
4.1 Formulations
In this section we introduce some basic concepts for stochastic differential games
and formulate problems. When players I and II choose Wiener-adapted control
processes y WD y./ and z WD z./ respectively, the response X evolves according to
SDE and the cost functional is given by the usual form:
Z t
C.t; ; x; y; zI / D f .X.s/; y.s/; z.s// ds C .X.t//

for y; z with X./ D x: Its expectation is called the payoff. Now player I wants to
maximize the payoff and player II wants to minimize it.
In Sect. 4.1.1, we define control processes and strategies, based on the concepts
in [FS06], XI. 4, and study properties of responses. In Sect. 4.1.2 we formulate
two kinds of stochastic differential games and investigate lower- and upper value
functions.
4.1.1 Admissible Controls and Strategies
Let W be an m-dimensional Wiener process, defined on .˝; F ; P /. Let Y and Z be

convex and compact subsets of Rq1 and Rq2 , respectively. Put

118 4 Stochastic Differential Games
8
< YW 1
0 Œ; t D L .Œ; t; .Ft /I Y/; Y0 D YW
W
0 Œ0; T ;
(4.1)
: ZW Œ; t D L1 .Œ; t; .F W /I Z/; Z0 D ZW
0 t 0 Œ0; T :
Let 2 .0; T / be given. Since y 2 Y0 is given by (4.2), with a progressively

measurable map ˚ W Œ0; T C.Œ0; T W Rm / C.Œ0; I Rm / 7! Y,
y.t; W / D ˚.t; WC ; W / P -a.s. (4.2)
under P .jFW /, we freeze W in (4.2) and .y.t/; t 2 Œ; T / can be regarded as an
WC
element of L1 .Œ0; T ; .Ft /I Y/.
Denote by YSC the set of all switching processes of Y0 , namely y.t/ D y.ti /; t 2
Œti ; ti C1 /; i D 0; : : : ; p with some 0 D t0 < t1 < < tp < tpC1 D T . For Y Y0
and t > 0; Yt denotes the set of restriction of y 2 Y to Œ0; t.
Next we introduce admissible controls, by referring to [FS06], X1.4. Let Y be a
subset of Y0 satisfying
(a) Y YSC ;
(b) For y1 ; y2 2 Y and 2 .0; T /, the concatenation y given by
y.t/ D y1 .t/Œ0; / .t/ C y2 .t/Œ;T .t/
is in Y;
WC
(c) Under P .jFW /; .y.t/; t 2 Œ; T / can be regarded as an element of YT

(D
C
the set given by YT with W replaced by W ).
Clearly YSC and Y0 satisfy (a)–(c) and
YSC Y Y0 : (4.3)
Replacing Y by Z, we define Z; Zt , and ZSC in the same way.

Definition 4.1. y 2 Y (resp. z 2 Z) is called an admissible control for player I
(resp. II).
Definition 4.2. An admissible strategy for player I is a map W Z 7! Y with the
property that if P .z.s/ D zQ .s// D 1; 8s 2 Œ0; t, then P ..z/.s/ D .Qz/.s// D
1; 8s 2 Œ0; t, for any t 2 Œ0; T .
The set of these is denoted by Y . For player II, an admissible strategy is defined
in the similar way and Z denotes the corresponding set of strategies. When Y D Y0
and Z D Z0 , the strategy is called Elliott–Kalton strategy and the sets of such
strategies is denoted by YEK and ZEK .
Now let us apply the time discretization to controls and strategies. Let D D
.t1 ; : : : ; tN / be a division of Œ0; T ; 0 D t0 < t1 < < tN < tN C1 D T , and put
jDj D maxi .ti C1 ti /.
4.1 Formulations 119
Definition 4.3. y 2 Y (resp. z 2 Z) is called D-admissible for player I (resp. II), if

y.t/ D y.tj / (resp z.t/ D z.tj // on Œtj ; tj C1 /, j D 0; 1; : : : ; N , P -a.s. YD (resp.
ZD / denotes the set of all D-admissible controls for player I (resp. II).
Definition 4.4. 2 Y is called D-admissible for player I, if W Z 7! YD satisfies
.z/.s/ D .z/.0/; s < t1 is a constant .2 Y/ independent of z.2 Z/ (4.4)
and
P .z.s/ D zQ .s// D 1 for s < tk

H) P ..z/.tk / D .Qz/.tk // D 1; k D 1; : : : ; N: (4.5)
We note that (4.5) and YD yield that
P .z.s/ D zQ .s// D 1; 8s < tk

H) P ..z/.s/ D .Qz/.s// D 1 for s < tkC1 : (4.50 )
Y D denotes the set of all D-admissible strategies. Z D is given in the similar

way for player II.
When players I and II choose y 2 Y and z 2 Z, respectively, the response
X D X yz evolves according to the d -dimensional SDE
dX.t/ D b.X.t/; y.t/; z.t// dt C˛.X.t/; y.t/; z.t// d W .t/; t 2 .0; T ; (4.6)
X.0/ D x .2 Rd /: (4.7)
We always assume that the coefficients ˛; b, and f .D running cost) are bounded
and Lipschitz continuous, say
(d)
j˛.x; y; z/j C jb.x; y; z/j C jf .x; y; z/j K0
and
j˛.x; y; z/ ˛.x;
Q y;
Q zQ/j C jb.x; y; z/ b.x;
Q y;
Q zQ/j C jf .x; y; z/ f .x;
Q y;
Q zQ/j
l0 .jx xj
Q C jy yj
Q C jz zQj/:
Then the following result is immediate.

Proposition 4.1. The SDE (4.6)–(4.7) admits a unique .FtW /-adapted solution.
Moreover, the following estimates hold:
h i
Ex sup jX.s/j2p Kp .1 C jxj2p /; p 1; (4.8)
0sT
h i
Ex sup jX.s/ X./j2p Kp jt jp ; p 1; (4.9)
st
h i
E sup jXx1 .s/ Xx2 .s/j2 K1 jx1 x2 j2 ; (4.10)
0sT
where xi denotes the initial state of the solution, and

h i
Ex sup jX yz .s/ X yQzQ .s/j2
0st
Z t
K1 E.jy.s/ yQ .s/j2 C jz.s/ zQ .s/j2 / ds; (4.11)
0
where the constants K1 and Kp depend only on l0 and K0 .
4.1.2 Formulation of Stochastic Differential Games
We are concerned with two kinds of stochastic differential games, lower games and
upper games. In the lower game, the player I chooses an admissible control y first
and y is known when the player II chooses an element of Z. Hence player II chooses
an admissible strategy from Z . When y 2 Y and 2 Z are chosen, the response
X y evolves according to (4.6) with z D .y/. Thus, when game stops at t, the
payoff is given by
hZ t i
J.t; x; y; I / D Ex f .X.s/; y.s/; .y/.s// ds C .X.t// (4.12)
0
with 2 Cbu .Rd /.

The .Y; Z /-lower value V is defined by
V .t; xI / D inf sup J.t; x; y; I /: (4.13)

2Z y2Y
Hence the minimizing player II has information advantage. When Y D Y0 and

Z D Z0 , V is called the Elliott–Kalton lower value and is denoted by VEK

. We also
write ZEK and YEK , instead of Z and Y .
The upper game is defined in the similar way and the .Z; Y /-upper value V C is
given by
V C .t; xI / D sup inf J.t; x; ; zI /: (4.14)

2Y z2Z
C
The Elliott–Kalton upper value. is denoted by VEK . However, it seems to be unfair
that only the stronger player chooses a strategy. Now following [FH11] we reduce
the class of strategies to a smaller one which eliminates the information advantage.
We will treat mainly the lower game for a while.
Definition 4.5. 2 ZEK is called a strictly progressively measurable strategy, if for
each 2 YEK the equations
y D .z/; z D .y/ (4.15)
have a solution, namely and have fixed points. Zs denotes the set of strictly
progressively measurable strategies.
Let 2 Zs and 2 YEK be given. Then, it is immediate that
J.t; x; yO ; I / D J.t; x; ; zO I /; 8.t; x/ (4.16)
where .Oy; zO / is a solution of (4.15). Hence we have

C
inf sup J.t; x; y; I / VEK .t; x; /; (4.17)
2Zs y2Y0
which says that Zs eliminates the advantage of player II.

Indeed, for " > 0, there is " 2 Zs , such that
LHS of (4.17) C " sup J.t; x; y; " I /: (4.18)

y2Y0
For any 2 YEK ,
RHS of (4.18) J.t; x; yO ; " I / D J.t; x; ; zO I /

inf J.t; x; ; zI /; (4.19)
z2Z0
where .Oy; zO / is a solution of (4.15) for .; " /.

Taking the supremum over YEK yields (4.17).
Example 4.1. Z D Zs .
Indeed, for 2 Z D and 2 YEK , we construct a solution of (4.15) in the
following way. Let D D .t1 ; : : : ; tn /. Fix z 2 Z arbitrarily. Regarding z as the
constant control process, we define yk 2 Y0 and zk 2 ZD by the following recursive

procedure: put y0 D .z/; z0 D .y0 /, and
yk D .zk1 /; zk D .yk /; k D 1; 2; : : : : (4.20)
Then, we can show that
yj C1 D yj on Œ0; tj /; zj C1 D zj on Œ0; tj C1 /; j D 1; 2; : : : ; n: (4.21)
Since (4.4) yields that, for s 2 Œ0; t1 /, .y/.s/ D .y/.0/ D constant independent of
y.DW zQ /, we see that
zk .s/ D zk .0/ D zQ for s 2 Œ0; t1 /; k D 1; 2; : : :

and
yk .s/ D .zk1 /.s/ D .Qz/.s/; s 2 Œ0; t1 /; k D 1; 2; : : :
holds by Definition 4.2. Hence, zk .t1 / D .yk /.t1 /.k D 1; 2; : : :/ does not depend
on k, and (4.21) holds for j D 1.
If (4.21) holds for j , then the same arguments lead to
yj C2 .s/ D .zj C1 /.s/ D .zj /.s/ D yj C1 .s/ for s < tj C1 (4.22)
and
zj C2 .s/ D zj C1 .s/ on Œ0; tj C1 /: (4.23)
Since zj C2 and zj C1 are in ZD , we have
zj C2 D zj C1 on Œ0; tj C2 / (4.24)
and
yj C2 D yj C1 on Œ0; tj C1 /; (4.25)
which yield (4.21) for j C 1.

Finally, we have znC1 .tn / D zn .tn /. Hence,
znC1 D zn on Œ0; T and ynC1 D .zn / D .znC1 /
show that .ynC1 ; znC1 / is a solution.

Example 4.2 (Constant strategy). Let z0 2 Z0 be given. Define z0 2 ZEK by
z0 .y/ D z0 for any y 2 Y0 . Then z0 2 Zs and the solution of (4.15) for .; z0 / is
unique.
Indeed, for 2 YEK , we put y0 D .z0 /. Then z0 .y0 / D z0 yields that .y0 ; z0 /
is a solution. If .Oy; zO / is a solution, then z0 .Oy/ D zO shows that zO D z0 . Hence
yO D .z0 / D y0 .
Regarding the solution of (4.15) and related topics in deterministic differential
games, [FKSh10], Sect. 4 presents interesting results.
The case when the equality holds in (4.17), i.e.,
inf sup J.t; x; y; I / D sup inf J.t; x; ; zI / (4.26)

2Zs y2Y0 2YEK z2Z0
suggests a saddle point property for the upper game.

By (4.17) and (4.26), we recall the Definition and Theorem from [FH11], Sect. 3.
Definition. The saddle point property for the upper game is said to hold, if there
exists a real valued function V .t; x/ such that for each " > 0 and .t; x/ 2 Œ0; T Rd
there exist " 2 Zs and " 2 YEK such that
(a) V .t; x/ " infz2Z0 J.t; x; " ; zI /,
(b) supy2Y0 J.t; x; y; " I / V .t; x/ C ".
C
Theorem. The saddle point property holds with V D VEK D v, where v is
the unique bounded, uniformly continuous viscosity solution to the upper Isaacs
equation with the initial state .
Finally, we revisit the saddle point property (4.26) in terms of strategies of the
two players. For 2 Zs and 2 YEK , denote
Y.; / D fOyI .Oy; .Oy// is a solutiong; (4.27)

Z.; / D fOzI ..Oz/; zO / is a solutiong; (4.270)
J .t; x; ; I / D sup J.t; x; y; .y/I /; (4.28)
y2Y.;/
J .t; x; ; I / D inf J.t; x; .z/; zI /; (4.280)

z2Z.;/
J.t; x; ; I / D fJ.t; x; yO ; zO I /I .Oy; zO / is a solutiong: (4.29)
If the solution is unique, we simply write
J.t; x; ; I / D J.t; x; yO ; zO I /: (4.30)
From (4.27) and Example 4.2, it follows that
sup J.t; x; y; I / D sup J.t; x; y ; I /

y2Y0 y2Y0
sup J .t; x; ; I / sup J.t; x; y; I /:

2YEK y2Y0
Consequently,
sup J.t; x; y; I / D sup J .t; x; ; I /:

y2Y0 2YEK
Taking the infimum over Zs leads to
LHS of (4.26) D inf sup J .t; x; ; I /: (4.31)

2Zs 2YEK
Let 2 YEK be given. By using similar arguments, we have
inf J.t; x; ; zI / D inf J .t; x; ; I /;

z2Z0 2Zs
and so
RHS of (4.26) D sup inf J .t; x; ; I /: (4.32)

2YEK 2Zs
Hence (4.26) implies that
inf sup J .t; x; ; I / D sup inf J .t; x; ; I /:

2Zs 2YEK 2YEK 2Zs
Before we end Sect. 4.1, let us list some basic properties of payoff, by using
Proposition 4.1.1.
Proposition 4.2. Let 2 Cbu .Rd /.
(i) Bound
jJ.t; x; y; zI /j K0 t C kkC ; (4.33)
(ii) Continuous dependence on t

For " > 0, there is " > 0, such that
jJ.t1 ; x; y; zI / J.t2 ; x; y; zI /j < "; 8x; y; z; (4.34)
whenever jt1 t2 j < " .

(iii) Continuous dependence on the initial state
For " > 0, there is ı"; > 0, such that
jJ.t; x1 ; y; zI / J.t; x2 ; y; zI /j < "; 8t; y; z; (4.35)
whenever jx1 x2 j < ı" .

4.2 DPP 125
(iv) Continuous dependence on admissible controls

For " > 0, there is " > 0, such that
jJ.t; x; y1 ; z1 I / J.t; x; y2 ; z2 I /j < "; 8t; x; (4.36)

RT
whenever EŒ 0 .jy1 .t/ y2 .t/j2 C jz1 .t/ z2 .t/j2 / dt < " ,
(v) Monotonicity
H) J.t; x; y; zI / J.t; x; y; zI /; 8t; x; y; z; (4.37)
(iv) Contractiveness
jJ.t; x; y; zI / J.t; x; y; z; /j k kC ; 8t; x; y; z:
4.2 DPP
We have already emphasized the importance of DPP for stochastic control problems
in Chap. 2. Here we will consider DPP (sometimes we say minimax principle) for
stochastic differential games. In this section, we deal with stochastic games where
the player with information advantageous uses Elliott–Kalton strategies and the
other player uses switching controls. Hence, it is dealing with an unfair game. By
using semi-discretization arguments, introduced in [N88] and [FSo89], we firstly
consider D-lower and D-upper value functions in Sect. 4.2.1. These cases admit the
discrete time DPP. Taking finer and finer divisions of Œ0; T , we obtain the DPP for
the limit games in Sect. 4.2.2 (see Theorem 4.2).
4.2.1 D -Lower and D -Upper Value Functions
For D D .t1 ; : : : ; tn /, we define the D-lower value function vD by replacing .Y; Z /

by .YD ; ZEK / in (4.13), namely
vD .t; xI / D inf sup J.t; x; y; I /: (4.38)

2ZEK y2YD
Similarly, D-upper value function uD is given by
uD .t; xI / D sup inf J.t; x; ; zI /: (4.380)

D
2YEK z2Z
Here we are concerned with DPP for vD and uD . From now on, we mainly consider
vD , because uD can be treated by the same arguments.
Recalling Proposition 4.2, we easily obtain;

Proposition 4.3. vD .t; I / 2 Cbu .Rd /.
(i) jvD .t; xI /j K0 t C kkC .
(ii) With the same constant " > 0, as in (4.34),
jvD .t1 ; xI / vD .t2 ; xI /j < "; 8x; (4.39)
whenever .t1 t2 / < " .

(iii) With the same constant ı" > 0, as in (4.35),
jvD .t; x1 I / vD .t; x2 I /j < "; 8t; (4.40)
whenever jx1 x2 j < ı" .

(iv) Monotonicity
H) vD .t; I / vD .t; I /; 8t:
(v) Contractiveness
kvD .t; I / vD .t; I /kC k kC ; 8t:
Corollary 4.1. vD . I / has D-independent bound and uniform continuity on

Œ0; T Rd . Moreover, the sequence fvDn . I /; n D 1; 2; : : : g has a subsequence
which converges uniformly on any bounded subset of Œ0; T Rd .
Considering y 2 Y as a constant control in Y0 , we introduce two notations;
8
ˆ
< I.t; x; yI / D z2Z
inf J.t; x; y; zI /;
0
(4.41)
:̂ v.t; xI / D sup I.t; x; yI /:
y2Y
Let us define an operator .t/ W Cbu .Rd / 7! Cbu .Rd / by
.t/.x/ D v.t; xI /: (4.42)
We aim to prove the following form which leads to DPP.

Theorem 4.1. Let 2 .tp ; tpC1 be given. Then
vD .t; xI / D .t1 /.t2 t1 / . tp /.x/: (4.43)
Note. Let k < p and 2 Œtk ; tkC1 / be given. By D we denote the division 0 <
tkC1 < tkC2 < < tn < tnC1 D T on Œ0; T .
4.2 DPP 127
We define vD in the same way as vD . Then Theorem 4.1 leads to the DPP
vD . ; xI / D vD .tk ; xI vDtk . tk ; I //: (4.44)
Proof. We divide the proof into five steps.

Step 1. Construct y"t .x/ 2 Y, which yields a nearly optimal control process:
v.t; xI / J.t; x; y"t .x/; zI / C 2"; 8z 2 Z0 : (4.45)
Since I.t; x; yI / is uniformly continuous in .x; y/ 2 Rd Y and Y is compact,

there is a maximum selector yO t W Rd 7! Y, such that
v.t; xI / D I.t; x; yO t .x/I / J.t; x; yO t .x/; zI /; 8z 2 Z0 ; (4.46)
where yO t .x/ 2 Y is regarded as a constant control process in Y0 . On the other

hand, Proposition 4.2 says that for a division of Rd .j ; j D 1; 2; : : : / with
diameter of j < ı" ; j D 1; 2; : : : one has
jJ.t; x1 ; y; zI / J.t; x2 ; y; zI /j

_ jI.t; x1 ; yI / I.t; x2 ; yI /j _ jv.t; x1 I / v.t; x2 I /j < "; 8t; y; z; y
(4.47)
whenever x1 ; x2 2 j .j D 1; 2; : : : /.
We fix xj 2 j arbitrarily and put
1
X
y"t .x/ D yO t .xj /j .x/: (4.48)
j D1
Then (4.46) and (4.47) yield (4.45).

Step 2. Using y"t , we define " 2 Y D as
X
p1
" .z/.t/ D Œti ;ti C1 / .t/y";ti C1 ti ;i C1 .X.ti // C Œtp ;T .t/y"; tp ; .X.tp //
i D0
(4.49)
where
pC1 D ; p D . tp /pC1 ; : : : ;
l D .tlC1 tl /lC1 ; : : : ; 0 D .t1 /1 ; (4.50)
and X is the response for ." ; z/ constructed by the following stepwise procedure.
Let x be the initial state. Fix z 2 Z0 arbitrarily and define " by
" .z/.s/ D y"t1 1 .x/; s 2 Œ0; t1 /: (4.51)

.X.s/; s t1 / denotes the response for ." ; z/ with X.0/ D x. Next we define
." .z/.s/; s 2 Œt1 ; t2 // by
" .z/.s/ D " .z/.t1 / D y";t2 t1 ;2 .X.t1 //: (4.52)
Thus, we have " .z/ on Œ0; t2 / and its response .X.s/; s 2 Œ0; t2 /. Repeating this
procedure, we obtain " .z/ on Œ0; tp / and its response .X.s/; s 2 Œ0; tp /. Finally,
we put
" .z/.s/ D " .z/.tp / D y"; tp ; .X.tp //; s 2 Œtp ; T : (4.53)
Then " 2 Y D satisfies (4.49).

Step 3. One has
J. ; x; " ; zI / .t1 / .tp tp1 /. tp /.x/ 2"p: (4.54)
The reason is following. Under P .jFtWp /; .z.s/; s 2 Œtp ; T / can be regarded as

WtC
an element of Z0 p and " .z/.tp / as constant .2 Y/ P -a.s.
Hence (4.45) yields
Z
E f .X.s/; " .z/.tp /; z.s// ds C .X. //jFtWp
tp
D E.C. ; tp ; X.tp /; y"; W

tp ; .X.tp //; zI /jFtp /
v. tp ; X.tp /I / 2"; P -a.s.

D . tp /.X.tp // 2"; P -a.s. (4.55)
Further, similar estimates yield
E.C. ; tp1 ; X.tp1 /; " ; zI /jFtWp1 /
.tp tp1 /. tp /.X.tp1 // 4" P -a.s. (4.56)
Repeating the arguments, we obtain (4.54).

Now taking the infimum over Z0 , in (4.56), we conclude
inf J. ; x; " ; zI / .t1 / . tp /.x/ 2"p: (4.57)

z2Z0
Observing " 2 Y D and letting " ! 0, we have
sup inf J. ; x; ; zI / .t1 / . tp /.x/: (4.58)

2Y D z2Z0
4.2 DPP 129
Next we will show that
vD . ; xI / .t1 / . tp /.x/: (4.59)
Step 4. Construct a nearly optimal " 2 ZEK by using arguments similar to those
in Steps 1 and 2.
Let " > 0 be given. By (4.36), we can take d" > 0, such that
jJ.t; x; y1 ; zI /J.t; x; y2 ; zI /jCjI.t; x; y1 I /I.t; x; y2 I /j < "; 8t; x; z;

(4.60)
whenever jy1 y2 j < d" .
Let Y" D .Y1 ; : : : ; Yl / be a division of Y with the diameter of Yj < d" ; j D
1; 2; : : : ; l. We fix yj 2 Yj and xi 2 i .i D 1; 2; : : : ; j D 1; : : : ; l/ arbitrarily
and take zt ij 2 Z0 so that
J.t; xi ; yj ; zt ij I / I.t; xi ; yj I / C ": (4.61)
Emphasizing the dependence on the Wiener path w, we put
1 X
X l
zt .y; x/.s; w/ D i .x/Yj .y/z"t ij .s; w/: (4.62)
i D1 j D1
Then the inequality
J.t; x; y; z"t .y; x/I / I.t; x; yI / C 5"; 8x 2 Rd ; y 2 Y; (4.63)
follows from (4.60), (4.61) and (4.47).

Now we construct " by using z"t . Let y 2 Y0 be given. Since y.0/ is constant,
say y0 , we define " by
" .y/.s; w/ D z"t1 1 .y0 ; x/.s; w/ for s 2 Œ0; t1 : (4.64)
.X.s/; s 2 Œ0; t1 / denotes the response for .y; " / with X.0/ D x. Then we put
" .y/.s; w/ D z";t2 t1 ;2 .y.t1 ; w/; X.t1 ; w//.s t1 ; wC

t1 / for s 2 .t1 ; t2 :
(4.65)
Hence we obtain " .y/ on Œ0; t2 and its response X . Repeating this procedure,
we construct " .y/ on Œ0; tp . Now put
" .y/.s; w/ D z"; tp ; .y.tp ; w/; X.tp ; w//.s tp ; wC

tp / for s 2 .tp ; T :
(4.66)
Then " is in ZEK .
Step 5. We will show that
J. ; x; y; " I / .t1 / . tp /.x/ C 5"p for y 2 YD : (4.67)
Using the same arguments as in Step 3, we deduce from (4.63) that

Z
E f .X.s/; y.tp /; " .y/.s// ds C .X. // jFtWp
tp
I. tp ; X.tp /; y.tp /I / C 5"

. tp /.X.tp // C 5": (4.68)
Again repeating the same arguments, we obtain (4.67).

Since y 2 YD is arbitrary, (4.67) yields
sup J. ; x; y; " I / .t1 / . tp /.x/ C 5"p: (4.69)

y2YD
Observing that " 2 ZEK and letting " ! 0 in (4.69) we conclude (4.59).
Finally we show the opposite of inequality (4.59) by using (4.57). Since "
of (4.57) is in Y D , Example 4.1 says that, for any 2 ZEK , there exist yO 2 YD and
zO 2 Z0 such that
" .Oz/ D yO ; .Oy/ D zO :
Therefore,
J.t; x; " ; zO I / D J.t; x; y;

O I /: (4.70)
LHS of (4.57) J. ; x; " ; zO I /

D J. ; x; yO ; I /
sup J. ; x; y; I / .by yO 2 YD /: (4.71)
y2YD
Since 2 ZEK is arbitrary, (4.71) yields
LHS of (4.57) vD . ; xI /; (4.72)
which concludes the opposite inequality in (4.59). This completes the proof of
theorem. u
t
4.2 DPP 131
4.2.2 DPP for Lower- and Upper Value Functions
Let Dn ; n D 1; 2; : : : be a sequence of division of Œ0; T . Suppose Dn DnC1 ; n D

1; 2; : : : . Then vDn .t; xI / is increasing uniformly on any bounded set of Œ0; T
Rd , by Corollary 4.1. Put
v.Dn / .t; xI / D lim vDn .t; xI /: (4.73)

n!1
The following result holds true.

Proposition 4.4. Suppose that Dn DnC1 ; n D 1; 2; : : : and limn! jDn j D 0.
Then v.Dn / does not depend on the sequence Dn ; n D 1; 2; : : : .
Definition 4.6. Put v.t; xI / D v.Dn / .t; xI /. The function v./ is called the lower
value function.
First let us prove the following Lemma.
Lemma 4.1. Let D D .t1 ; : : : ; tn / and DQ D .t1 ; : : : ; tp ; ; tpC1 ; : : : ; tn /. For t 2
.tj ; tj C1 ; j p C 1, we put
˚.x/ WD ˚t .x/ D .tpC2 tpC1 / .t tj /.x/:
Then
Q
jvD .t; xI / vD .t; xI /j < 2"; 8x; if tpC1 tp < "˚ ; (4.74)
where "˚ is the constant given by (4.34) for ˚.

Proof of Lemma. Theorem 4.1 implies that
vD .t; xI / D .t1 / .tpC1 tp /˚.x/ (4.75)
and
Q
vD .t; xI / D .t1 / .tp tp1 /. tp /.tpC1 /˚.x/: (4.76)
Put
v .t; x/ D .t1 / .tp tp1 /˚.x/: (4.77)
From (4.39) and (4.43) it follows that for 2 Œ0; s,
j./.s /˚.x/ ˚.x/j

sup sup jJ.s; x; y; zI ˚/ ˚.x/j < "; 8x (4.78)
z2Z0 y2Y0
whenever s < "˚ .

We notice that, if tpC1 tp < "˚ , then
jvD .t; xI / v .t; x/j k.tpC1 tp /˚ ˚kC < "
and
Q
jvD .t; xI / v .t; x/j k. tp /.tpC1 /˚ ˚kC < ";
which yields (4.74). t

u
Proof of Proposition 4.4. Let D [ DQ be the division with PD[DQ D PD [ PDQ .
Suppose that .Dn ; n D 1; 2; : : : / and .DQ n ; n D 1; 2; : : : / satisfy the condition of the
proposition. Put
Q
v.t; x/ D v.Dn / .t; xI /; vQ .t; x/ D v.Dn / .t; xI /;
and Dnk D Dn [ DQ k .
Then for all n; k D 1; 2; : : : ,
vDnk .t; xI / vDn .t; xI /; (4.79)

Q
vDnk .t; xI / vDk .t; xI /: (4.80)
On the other hand, by Lemma 4.1,
lim vDnk .t; xI / D lim vDn .t; xI / D v.t; x/; k D 1; 2; : : : (4.81)

n!1 n!1
and
lim vDnk .t; xI / D vQ .t; x/; n D 1; 2; : : : : (4.82)

k!1
Now letting k ! 1 in (4.79) and then letting n ! 1, we obtain vQ ./ v./

by (4.82). Since in the same way we have v./ vQ ./, we conclude that v./ D vQ ./.
t
u
We are ready to prove DPP.
Theorem 4.2. For 0 t, s; t C s T ,
v.t C s; xI / D v.t; xI v.s; I //; 8x: (4.83)

Step 1. We prove
v.t C s; xI / v.t; xI v.s; I //: (4.84)

4.2 DPP 133
Suppose that Dn DnC1 ; n D 1; 2; : : : ; limn!1 jDn j D 0 and t 2 PD1 .

Then (4.44) yields
vDn .t C s; xI / D vDn .t; xI vDnt .s; I //: (4.85)
Since .Dnt ; n D 1; 2; : : : / satisfies the condition of Proposition 4.4, by replacing

T with T t; vDnt is increasing to v, as n ! 1. Hence the monotonicity provides
vDn .t C s; xI / vDn .t; xI v.s; I // v.t; xI v.s; I //; n D 1; 2; : : : :

(4.86)
Letting n ! 1, we obtain (4.84).
Step 2. For the opposite inequality, we first need
Lemma 4.2. Suppose that l 2 Cbu .Rd /; l D 1; 2; : : : satisfy K0 WD
supl k l kC < 1 and l converges to 1 .2 Cbu .Rd // uniformly on any
bounded set. Then
lim jvDn .t; xI l/ vDn .t; xI 1 /j D0 (4.87)

l!1
uniformly in n.
Proof of Lemma. Let X be the response for y 2 Y0 and z 2 Z0 . Then (4.8)
implies
Ex Œj l .X.t/.jX.t/j > /j K0 Px .jX.t/j > / < " (4.88)

12
K0 K1 .1Cjxj2 /
whenever > " WD " .
Let " 2 C.R I Œ0; 1/ satisfy
d
(
1; for jyj " ;
" .y/ D (4.89)
0; for jyj " C 1:
Then we have
jJ.t; x; y; zI l/ J.t; x; y; zI l " /j Ex j l .X.t// . l " /.X.t//j
2K0 Px .jX.t/j > " / < 2K0 ";

(4.90)
for l D 1; 2; : : : ; 1. Since l " converges to 1 " uniformly,
sup jJ.t; x; y; zI l " / J.t; x; y; zI 1 " /j k l " 1 " kC ! 0

y;z
(4.91)
as l ! 1. Combining (4.90) and (4.91), we obtain
jvDn .t; xI l/ vDn .t; xI 1 /j 4K0 " C k l " 1 " kC ; n D 1; 2; : : : :

(4.92)
Now letting l ! 1, and then letting " ! 0, we get (4.87). t
u
Step 3. Opposite of inequality (4.84)
From (4.44), we deduce that, for n and l,
v.t C s; xI / vDnCl .t C s; xI / vDn .t; xI vDlt .s; I //: (4.93)
Putting l D vDlt .s; I / and 1 D v.s; I /, we have
vDn .t; xI l/ ! vDn .t; xI 1/ as l ! 1; (4.94)
by Lemma 4.2. Thus, (4.93) and (4.94) give the opposite inequality.
This completes the proof of the theorem. t
u
To conclude Sect. 4.2, we introduce the upper value function. Under the same
conditions as in Proposition 4.4, the Dn -upper value function uDn .I / is decreasing
uniformly on any bounded set of Œ0; T Rd , as n ! 1. Its limit u.I /, called the
upper value function, is independent of Dn ; n D 1; 2; : : : and the DPP holds:
u.t C s; xI / D u.t; xI u.s; I //; 8x; (4.95)
for 0 t, s t C s T .
Using (4.380), (4.58), and Theorem 4.1, we have
uD .t; xI / sup inf J.t; x; ; zI / vD .t; xI /: (4.96)

2Y D z2Z0
Hence,
u.t; xI / v.t; xI /: (4.97)
When the two value functions coincide, we use the name value function.
4.3 Isaacs Equations
In this section, we study the dynamics of lower- and upper value functions, via the
semigroup formulation of DPP. Referring to Sect. 2.2, we introduce semigroups on
Cbu .Rd / related to DPP for lower (resp. upper) value function in Sect. 4.3.1. Their
generators lead to a (nonlinear) parabolic equation, called the lower (resp. upper)
Isaacs equation. In Sect. 4.3.2, we prove that the lower value function is the unique
viscosity solution of the lower Isaacs equation, by using the same arguments as in
Sect. 3.1.3. Similar results are valid for the upper value function.
4.3 Isaacs Equations 135
4.3.1 Semigroups Related to the DPP
Let us define Vt I Cbu .Rd / 7! Cbu .Rd /, by
Vt .x/ D v.t; xI /: (4.98)
Then, the following properties are clear.

Proposition 4.5.
(i) Semigroup V0 D identity map,
Vt Cs D Vt .Vs / D Vs .Vt /; 0 t; s t C s T:
(ii) Monotonicity
H) Vt Vt ; 8t:
(iii) Contractiveness
kVt Vt kC k kC ; 8t:
(iv) Continuity
lim kVt C V kC lim kVt kC D 0; 8:

t !0 t !0
Fix y 2 Y arbitrarily. From the definition of I.t; x; yI / (see (4.41)) the maps
y
It I Cbu .Rd / 7! Cbu .Rd / given by
y
It .x/ D I.t; x; yI /; t 2 Œ0; T ; (4.99)
form a semigroup, which formulates the DPP for the value function of stochastic
control (cf. Sect. 2.2). Hence, putting
1
G.y; z/ .x/ D tr.a.x; y; z/@xx .x// C b.x; y; z/ @x .x/ C f .x; y; z/
2
(4.100)
for 2 Cb2 .Rd /, we have
Proposition 4.6. For any y 2 Y,
(i) Semigroup
y
I0 Didentity map,
y y y
It Cs D It .Iys / D Iys .It /; 0 t; s t C s T:
(ii) Monotonicity
y y
H) It It ; 8t:
(iii) Contractiveness
y y
kIt It kC k kC ; 8t:
(iv) Continuity
y y y
lim kIt C I kC lim kIt kC D 0; 8;
t !0 t !0
(v) Generator
For 2 Cb2 .Rd /,
1 y
lim .It .x/ .x// D inf G.y; z/ .x/:
t !0 t z2Z
By using the same arguments as in Sect. 2.2.5, the following proposition is also
immediate.
Proposition 4.7. Let Q denote the set of all continuous, monotone, and contractive
semigroup .Qt / WD .Qt I t 2 Œ0; T / satisfying
y
Qt It ; 8 2 Cbu .Rd / (4.101)
for any t and y. Then .Vt / is the minimal element of Q, that is .Vt / 2 Q and for any
.Qt / 2 Q,
Qt Vt ; 8 2 Cbu .Rd /; 8t: (4.102)
Next we compute the generator G of .Vt /.

Theorem 4.3. For 2 Cb2 .Rd /
1
G .x/ D lim .Vt .x/ .x// D sup inf G.y; z/ .x/ .WD G .x//: (4.103)
t !0 t y2Y z2Z

Step 1. Let y 2 Y0 ; z 2 Z0 be given. X denotes the corresponding response. For
R; " > 0, the uniform continuity of all coefficients and (4.9) provide a constant
" R > 0, independent of y and z, such that
Ex jG.y.s/; z.s// .X.s// G.y.s/; z.s// .x/j < " (4.104)
for any x 2 SR , whenever s < " R .

Hence Itô’s formula yields that for t < " R ,

ˇ Z t ˇ
ˇ ˇ
ˇJ.t; x; y; I / .x/ Ex G.y.s/; .y/.s// .x/ ds ˇ < "t;
0
8x 2 SR ; y 2 Y0 ; 2 ZEK : (4.105)
Step 2. For x 2 SR , we compute
c.t/ D c.t; x/ WD vD .t; xI / .x/ for t < " R :
Let D D .t1 ; : : : ; tn / and t 2 .tp ; tpC1 . For simplicity, we put t D tpC1 .

X
p Z tj C1
c.t/ inf sup E G.y.tj /; .y/.s// .x/ ds C "t: (4.106)
2ZEK y2YD tj
j D0
Since G.y; z/ .x/ is continuous in .y; z/ and Z is compact, there is a minimum

selector O W Y 7! Z, such that
O
G.y; .y// D inf G.y; z/ .x/: (4.107)
z2Z
Define 2 ZEK by
O
.y/.s/ D .y.tj // for s 2 Œtj ; tj C1 /; j D 0; 1; : : : ; n: (4.108)
Then (4.107) and (4.108) yield
RHS of (4.106)
X
p Z tj C1
sup E G.y.tj /; .y/.s// .x/ ds C "t (4.109)
y2YD j D0 tj
and
Z tj C1
G.y.tj /; .y/.s// .x/ ds D inf G.y.tj /; z/ .x/.tj C1 tj /
tj z2Z
G .x/.tj C1 tj /: (4.110)
Thus (4.106), (4.109), and (4.110) imply that
vD .t; xI / .x/ t.G .x/ C "/: (4.111)

Step 3. Conclusion
Suppose that Dn ; n D 1; 2; : : : satisfy the conditions of Proposition 4.4. Then
vDn .I / is increasing to v.I / uniformly on any bounded subset of Œ0; T Rd ,
and so (4.111) yields
1
lim sup .v.t; xI / .x// G .x/: (4.112)
t !0 t
For the converse inequality of (4.112), we recall

y
Vt .x/ It .x/; 8y; t; x: (4.113)
Now it follows from Proposition 4.6 (v) that
1
lim inf .Vt .x/ .x// G .x/: (4.114)
t !0 t
Inequalities (4.112) and (4.114) complete the proof. t

u
4.3.2 Viscosity Solutions of the Isaacs Equations
In the case when the lower value function v.I / is smooth, Theorem 4.3 says that
v.I / satisfies
@t v.t; x/ sup inf G.y; z/v.t; x/ D 0; t 2 .0; T ; x 2 Rd ; (4.115)

y2Y z2Z
v.0; x/ D .x/; x 2 Rd : (4.116)
Now we intend to prove that the lower value function satisfies (4.115)–(4.116) in
the viscosity sense. Define the Hamiltonian H W Rd S d Rd 7! R1 by
1
H.x; A; p/ D inf sup tr.a.x; y; z/A/ b.x; y; z/ p f .x; y; z/ ; (4.117)
y2Y z2Z 2
where a D ˛˛ > . Then (4.115) can be recast as
@t v.t; x/ C H.x; @xx v.t; x/; @x v.t; x// D 0: (4.118)
This parabolic equation (4.118) is called the lower Isaacs equation.

First we will show
Theorem 4.4. Let .St ; t 2 Œ0; T / be a continuous semigroup on Cbu .Rd / with
monotone and contraction properties. Suppose that for any 2 Cb2 .Rd / and 2
Cbu .Rd /,
1
lim .St . C t/ .x// D G .x/ C .x/; 8x: (4.119)
t !0
t
Then, for g 2 Cbu .Rd /; .St g; t 2 Œ0; T / is the unique viscosity solution of the
lower Isaacs equation (4.118) with the initial condition v.0; x/ D g.x/.
Proof. We can mimic the proof of Theorem 3.3. Put U.t; x/ D St g.x/. First let us
show that
U 2 Cbu .Œ0; T Rd /: (4.120)
Indeed, for " > 0, there is " > 0 such that
kU.t; / U.; /kC D kS St g S gkC

kSt g gkC < " for jt j < ": (4.121)
Taking the 12 " -net ftj ; j D 1; : : : ; pg of Œ0; T and noticing that U.tj ; / is in
Cbu .Rd /, we can choose a constant ı" > 0, so that
jU.tj ; x/ U.tj ; x/j

Q < "; j D 1; : : : ; p (4.122)
whenever jx xj Q < ı" . Thus, if jx xj

Q < ı" and jt j < 1
2 ", we take tj , such
that jt tj j; j tj j < " , and obtain
jU.t; x/ U.; x/j

Q
jU.t; x/ U.tj ; x/j C jU.tj ; x/ U.tj ; x/j
Q C jU.tj ; x/
Q U.; x/j
Q < 3";
which in turn yields (4.120).

Next we will show that U./ is a viscosity subsolution. Let 2 C 12 ..0; T
R / \ C.Œ0; T Rd / be a test function. Since U./ is bounded, we may assume that
d
is constant outside some compact set. Thus, we have
.t C ; x/ D .t; x/ C @t .t; x/ C o./ (4.123)
where o./ is small uniformly in x.

Suppose that U.t; x/ .t; x/ attains its global maximum at .tO; x/
O 2 .0; T / Rd
O O
O D .t ; x/.
and U.t ; x/ O Then, by the monotonicity of S ,
0 D U.tO; x/
O .tO; x/
O
D S U.tO ; /.x/
O .tO; x/
O
S .tO ; /.x/
O .tO; x/:
O (4.124)
On the other hand, the contractiveness of S together with (4.123) imply that
kS .tO ; / S ..tO; / @t .tO; //kC

k.tO ; / ..tO; / @t .tO; //kC ! 0 as ! 0: (4.125)
Thus, from (4.124) and (4.125) we deduce that
0 S ..tO; / @t .tO; //.x/

O .tO; x/
O C o./: (4.126)
Dividing both sides by and letting ! 0, we have, by (4.119)
O @t .tO; x/
0 G.tO; x/ O
and so U./ is a viscosity subsolution.

Since similar arguments show that U./ is a viscosity supersolution, the unique-
ness theorem (refer to Theorem 3.5 and Example 3.2) completes the proof. t
u
Next we will show that .Vt / satisfies (4.119), which in turn implies,
Theorem 4.5. The lower value function is the unique bounded viscosity solution of
the lower Isaacs equation.
Proof. For 2 Cb2 .Rd / and 2 Cbu .Rd /, Itô’s formula, (4.38) and (4.75) lead to
v.t; xI C t/ .x/ t.G .x/ C .x//

hZ t
D lim inf sup Ex f .X.s/; y.s/; .y/.s// ds
n!1 2ZEK y2YDn 0
i
C . .X.t// .x// tG .x/ C t..X.t// .x// (4.127)
where X WD X y . Observing that, by (4.9),
tEx Œj.X.t/ .x/j D o.t/ uniformly in y and ; (4.128)
we deduce (4.119) from (4.103) and (4.127). This completes the proof. t
u
Finally, we consider the upper value function, in the same way. Then the
following two results are easy to verify.
Proposition 4.8. For 2 Cbu .Rd /, the upper value function u.t; xI / is the unique
bounded viscosity solution of the upper Isaacs equation
8
< @t u.t; x/ inf sup G.y; z/u.t; x/ D 0; t 2 .0; T ; x 2 Rd ;
z2Z y2Y (4.129)
: u.0; x/ D .x/; x 2 Rd :
4.4 Risk Sensitive Stochastic Controls and Differential Games 141
Proposition 4.9. Assume the Isaacs condition

1
inf sup tr.a.x; y; z/A/ C b.x; y; z/ p C f .x; y; z/
z2Z y2Y 2
1
D sup inf tr.a.x; y; z/A/ C b.x; y; z/ p C f .x; y; z/ (4.130)
y2Y z2Z 2
for x 2 Rd ; A 2 S d ; p 2 Rd . Then,
v.t; xI / D u.t; xI /: (4.131)
4.4 Risk Sensitive Stochastic Controls and Differential

Games
This section is concerned with a link between stochastic controls and differential
games, via risk sensitive stochastic controls. Here we want to minimize the
exponential-of-integral risk sensitive criterion
1 Z T
E exp f .X ./ .t/; .t// dt ;
" 0
where the response X ./ evolves according to an SDE with small noise
(see (4.132)). Applying the logarithmic transformation to the value function, we
obtain the upper Isaacs equation, the corresponding stochastic differential games
of which are considered in Sect. 4.4.1. Section 4.4.2 deals with differential games
obtained as the limit of these stochastic differential games when " tends to 0. In
Sect. 4.4.3, we mention a control problem with infinite time horizon.
4.4.1 Logarithmic Transformation
Let W be an m-dimensional Wiener process on .˝; F ; P /. Let be convex and

compact and put W D L1 .Œ0; T ˝; .FtW /I /.
We assume the following condition: .a1 / and .a2 /;
.a1 /
˛ W Rd 7! Rd ˝ Rd ;
b W Rd 7! Rd ;
f W Rd 7! R1
1
are in Cbu .Rd / and Cbu
2
.Rd / uniformly on , say
kzkC 1 C sup kz.; /kC 2 K0 for z D ˛; b; f:

.a2 / Uniform positive definiteness

There exists 0 > 0, such that
y > a.x; /y 0 jy 2 j; 8y 2 Rd ; 8.x; /;
where a D ˛˛ > .
Let " 2 .0; 1/ be given. For a control process ./ 2 W
, its response evolves
according to the SDE
p
dX.t/ D b.X.t/; .t// dt C "˛.X.t/; .t// d W .t/ (4.132)
and the payoff is given by the exponential-of-integral risk sensitive criterion

1 Z t
J .t; xI .// D Ex exp
"
f .X.s/; .s// ds : (4.133)
" 0
Since a controller wants to minimize the payoff, the value function is defined by
v" .t; x/ D inf J " .t; xI .//: (4.134)

2 W
Regarding the regularity of value function, we recall;

Proposition 4.10 (cf. [FS06], IV Th. 4.2).
v" 2 Cb12 .Œ0; T Rd / (4.135)
and the Feynman–Kac formula yields the equation

" 1
@t v" inf tr.a.x; /@xx v" / C b.x; / @x v" C f .x; /v" D 0 (4.136)
2 2 "
on .0; T Rd , with the initial condition
v" .0; x/ D 1 on Rd : (4.137)
Next we explain how v" is related to the upper value function of stochastic
differential game, by performing the logarithmic transformation of v" .
Since the boundedness of f implies v" .t; x/ > 0, we can put
V " .t; x/ D " log v" .t; x/: (4.138)
Then (4.135) implies
V " 2 Cb12 .Œ0; T Rd /: (4.139)
Further, a direct computation together with the equality
1 > 1
p ap D sup p > ˛y jyj2 (4.140)
2 y2Rd 2
yields the upper Isaacs equation
@t V " inf sup G" .y; /V " D 0 on .0; T Rd ; (4.141)

2 y2Rd
with
V " .0; x/ D 0 on Rd ; (4.142)
where
"
G" .y; / .x/ D tr.a.x; /@xx .x//
2
1
C @x .x/ .b.x; / C ˛.x; /y/ C f .x; / jyj2 : (4.143)
2
Since, by (4.139), j@x V " .t; x/j M with some constant M D M" , the
function @x V " .t; x/> ˛.x; r/y 12 jyj2 attains its maximum at yO t;x; 2 SMK0 .
Consequently (4.141) is recast as
@t V " inf sup G" .y; /V " D 0; on .0; T Rd ; (4.141N )

2 jyjN
for N MK0 .
Proposition 4.11. Let U 2 Cb .Œ0; T Rd / be a viscosity solution of (4.141N )–
(4.142) for some large N . Then U coincides with the upper value function of the
stochastic differential game and U D V " holds.
Proof. Since (4.141N ) is the upper Isaacs equation, U is equal to the upper value
function of an stochastic differential game between player I and the controller
(player II) with control regions SN and , respectively. Now the proposition follows
from the uniqueness of the viscosity solution. t
u
4.4.2 Small Noise Limit
Let us consider the upper Isaacs equation when " D 0,
@t V inf sup G0 .y; /V D 0 on .0; T Rd ; (4.144)

2 y2Rd
with
V .0; x/ D 0 on Rd : (4.145)
Now we recall the following;

Proposition 4.12 ([FS06], XI Th 7.1). Equations (4.144)–(4.145) admits a unique
viscosity solution. This solution is bounded and Lipschitz continuous.
Proof. We sketch the proof in four steps.
Step 1. Differential games
Let us consider the auxiliary equation
@t V inf sup G0 .y; /V D 0 on .0; T Rd : (4.144R )

2 jyjR
S
Put YR D L1 .Œ0; T I SR /; Y0 D R>0 YR and D L1 .Œ0; T I /. For y 2
YR and ./ 2 , the response is given by the solution of the differential equation
dX
.t/ D b.X.t/; .t// C ˛.X.t/; .t//y.t/; t 2 .0; T ; (4.146)
dt
with initial condition
X.0/ D x .2 Rd /:
The payoff and the value function are respectively

Z t
1
J.t; x; y; .// D f .X.s/; .s// jy.s/j2 ds (4.147)
0 2
and
vR .t; x/ D sup inf J.t; x; ..//; .//; (4.148)

2YREK ./2
where YREK is the set of strategies from onto YR .

Step 2. There is a Lipschitz constant M1 , independent of t; x and R, such that
jvR .t; x/ vR .t; x/j

Q M1 jx xj;
Q 8t; R: (4.149)
Indeed, the estimates
J.t; xI 0; .// kf kC t; 8./ (4.150)
and
Z t
1
J.t; xI y; .// kf kC t jy.s/j2 ds; (4.151)
2 0
imply that the maximizer yO of J.t; x; ; .// satisfies

Z t
jOy.s/j2 ds 4kf kC T DW K1 : (4.152)
0
Hence it suffices to consider y() satisfying (4.154).

Let XQ denote the solution of (4.146) with XQ .0/ D x.Q Then .a1 / yields
p
jX.t/ XQ .t/j2 jx xjQ 2 exp 2.K0 .T C TK1 //; (4.153)
from which (4.149) follows.

Step 3. Noticing that
jvR .t; x/ vR .tO; x/j kf kC jt tQj; (4.154)
we conclude that vR is bounded and Lipschitz continuous, and is the unique

viscosity solution of (4.144R )–(4.145) (cf. [FS06], II Th 9.1).
Step 4. Let R > M1 k˛kC . Then
vR .t; x/ D lim vR .t; x/ DW V 0 .t; x/: (4.155)

R!1
Indeed, (4.149) shows that the sub- and super differential of vR .t; / are in SM1 .
Hence for jpj M1 and R > M1 k˛kC it holds that
1 1
sup p > ˛.x; /y jyj2 D sup p > ˛.x; /y jyj2 ; (4.156)
y2Rd 2 jyjR 2
which in turn yields (4.155).

Thus, vR ./ and V 0 ./ satisfy (4.144)–(4.145), whenever R > M1 k˛kC . This
concludes the proof of the proposition. t
u
Now the stability of viscosity solution yields;
Theorem 4.6 ([FS06] XI, Th 7.2). Under conditions .a1 / and .a2 /, V " ./ con-
verges to V 0 ./ uniformly on any compact set of Œ0; T Rd .
Example 4.3. Investment model with small noise
We shall consider a small noise limit of the investment model given in
Sect. 2.4.
1. Formulation
Let B and W be mutually independent d - and m-dimensional Wiener processes.
S 0 and S i .i D l; : : : ; m/ denote the price processes of bond and i -th asset,
respectively, and evolve according to the SDE
(
dS 0 .t/ D rS 0 .t/ dt; t 2 .0; T ;
(4.157)
S 0 .0/ D s 0 .> 0/;
and
8 r
ˆ " X i
m
< dS i .t/ D S i .t/ g i .X.t// dt C k .X.t// d W k .t/ ; t 2 .0; T ;
1C"
kD1
:̂
S i .0/ D s i > 0;
(4.158)
where X is the d -dimensional factor process, described by the SDE
p
dX.t/ D b.X.t// dt C " dB.t/: (4.159)
We assume that b W Rd 7! Rd ; W Rd 7! Rm ˝ Rm and g W Rd 7! Rm are

in Cbu1
.Rd / and > is uniformly positive definite. Further, r and " are positive
1
constants. We use the negative power utility U.z/ D "z " .
Using data of X and S i ; i D 1; : : : ; m, an agent invests atPt 2 .0; T / a proportion
i
.t/ of his/her wealth in the i -th asset and 0 .t/ D 1 m i D1
i
.t/ in the bond.
./ D . 1 ./; : : : ; m .// 2 L1 .Œ0; T ; .FtBW /I Rm / is called a control process,
where we admit selling . i .t/ < 0/ and borrowing . i .t/ > 1/. Accordingly, the
wealth process Z ./ is given by
nZ t
Z ./
.t/ D exp .r C .s/ .X.s// ds
0
Z t Z t o
> 2
C .s/ .X.s// d W .s/ j .s/> .X.s//j2 ds
0 2 0
(4.160)
q
where WD 1C" "
; i .x/ D g i .x/r; i D 1; : : : ; m, and we suppose Z ./ .0/ D 1
(see (2.187)).
The agent wants to maximize the expected utility from the terminal wealth,
EŒU.Z ./ .T //, by choosing an appropriate control process ./, where
./
U.Z .t//
n 1 Z t 2 Z
t o
> >
D " exp rC j j2 ds dW : (4.161)
" 0 2 " 0
2. Value function and its HJB equation

Since ./ is bounded,
Z t 2
Z t
> >
M ./
.t/ WD exp d W .s/ j j2 ds
" 0 2"2 0
R Under the new probability P

is an exponential martingale. ./
WD M ./ .T / ı P ,
O
B and W .WD W .t/ C " 0 t >
ds/ are mutually independent Wiener processes.
Hence, the value function u" and its HJB equation are as follows (cf. (2.199) and
(2.202)):
u" .t; x/ D sup Ex ŒU.Z ./

.T //
./
h n 1 Z t 1 oi
>
D sup Ex ./
" exp rC j j2 ds (4.162)
./ " 0 2
and
@u" " " r
0D C u C b.x/ @x u" u"
@t 2 "
n 1 1 o
C sup .x/ j > .x/j2 u" ; t 2 .0; T ; x 2 Rd ;
2Rm " 2
(4.163)
with u" .0; x/ D ".

Noting that u" < 0, > is uniformly positive definite and that coefficients are
bounded, we see that
1 1
>
the 5th term of RHS D sup .x/ j .x/j2 u" ;
j jc 2 "
with a positive constant c, independent of t; x and ". Thus, (4.163) admits a classical
solution with bounded @x u" , where the bound depends on ".
3. Logarithmic transformation and small noise limit
Take the logarithmic transformation, v" D " log. 1" u" /. Then
1
>
rt v" .t; x/ t sup r C .x/ j .x/j2 : (4.164)
;x 2
By the boundedness of @x u" , there is a constant d" > 0, such that

1 1
inf jyj2 y @x v" .t; x/ D inf jyj2 y @x v" .t; x/ :
y 2 jyjd" 2
Hence, v" is the unique viscosity solution of the Isaacs equation

8
ˆ @v" " "
ˆ
ˆ 0 D C v C b.x/ @x v" C r
ˆ
ˆ @t 2
ˆ
ˆ 1
< 1 >
C inf jyj2 y @x v" C sup .x/ j .x/j2 ;
jyjd" 2 2 (4.165)
ˆ
ˆ
j jc
ˆ
ˆ t 2 .0; T ; x 2 R d
;
ˆ
ˆ
:̂ "
v .0; x/ D 0; x 2 Rd :
On the other hand, the Isaacs equation associated with the differential game,
8
ˆ @v
ˆ
ˆ 0 D C b.x/ @x v C r
ˆ
ˆ @t
ˆ
ˆ 1
< 1 >
C inf jyj2 y @x v C sup .x/ j .x/j2 ;
y 2 j jc 2 (4.166)
ˆ
ˆ
ˆ
ˆ t 2 .0; T ; x 2 R d
;
ˆ
ˆ
:̂
v.0; x/ D 0; x 2 Rd
admits a unique viscosity solution, which is bounded and Lipschitz continuous.

Consequently, v" converges to v uniformly on any compact subset of Œ0; T Rd
(cf. Theorem 4.6).
4.4.3 Note on Control with Infinite Time Horizon
Finally we sketch a model problem with infinite time horizon, given in [FMcE95],
Sect. 7.
For a control process ./ 2 W , its response X 2 X ./ evolves according to
the SDE
r
"1
dX.t/ D b.X.t/; .t// dt C d W .t/; (4.167)
2
with a positive constant . The payoff is given by the long-run expected rate
1 1 Z T
J ..// D " lim inf log Ex exp
"
f .X ./ .s/; .s// ds ; (4.168)
T !1 T " 0
where " 2 .0; 1/ is called the noise intensity and the control region is convex and
compact.
Now we assume that the following conditions are satisfied:

.a1 /0
f 0; f 2 Cb1 .Rd / and b.; / 2 C 1 .Rd /
with supx j@x b.x; /j < 1.

.a2 /0
.x y/ .b.x; / b.y; // c0 jx yj2 ; 8x; y;
with a positive constant c0 .

Hence the payoff J " is independent of the initial state x. The problem is to
minimize J " over W .
Following [FMcE95], we will outline a link between ergodic controls and ergodic
Isaacs equations.
1. By (4.167)–(4.168), we have the following ergodic dynamic programming
equation
" 1
"
D W " .x/ C j@x W " .x/j2
42 42
C min.b.x; / @x W " .x/ C f .x; //; x 2 Rd : (4.169)
2
2. Under .a1 /0 and .a2 /0 , (4.169) has a solution . "

2 R1 ; W " 2 C 2 .Rd //,
satisfying
j@x W " .x/j B; 8x; 8" 2 .0; 1/; (4.170)
for some positive constant B.

Further, a minimum selector O" W Rd 7! , of .b.x; / @x W " .x/ C f .x; //
provides an optimal Markovian policy, and
"
D inf J " ..// D J " .O .X O .// (4.171)
./2 W
holds.
3. Using (4.140), Eq. (4.169) is recast of an ergodic Isaacs equation:
"
"
D W " .x/ C max .y @x W " .x/ 2 jyj2 /
42 y2Rd
C min.b.x; / @x W " .x/ C f .x; //: (4.172)

2
4. In order to obtain a stochastic differential game associated with (4.172), we con-

sider the following discounted cost stochastic differential game. For ./ 2 W
and y 2 L1 .Œ0; 1/; .FtW /I Rd /, the response D ./;y is given by the SDE
r
"1
d .t/ D .b..t/; .t// C y.t// dt C d W .t/; (4.173)
2
while the payoff with the discount rate is given by

Z t
J" .t; x; ./; y/ D Ex e s .f ..s/; .s// 2 jy.s/j2 / ds: (4.174)
0
Let us take the upper value function, v WD v" , in much the same way as in
Sect. 4.1.2. Then
"
@t v C v v
42
C min.b.x; / @x v C f .x; //
2
C max .y @t v 2 jyj2 / D 0 on .0; 1/ Rd ; (4.175)

y2Rd
v.0; x/ D 0; x 2 Rd : (4.176)
Equation (4.175)–(4.176) has a classical solution. By using various a priori

estimates, we see that
W" .x/ WD lim v" .t; x/ exists in C 2 .Rd /

t !1
and W" satisfies the stationary form of (4.175).

5. For a suitable sequence n ! 0, the constant " WD limn!1 n W"n .x/
exists and for a fixed x0 , the difference W"n .x/ W"n .x0 / converges to some
W " .2 C 2 .Rd // uniformly on any compact set. Hence . " ; W " / satisfies (4.172)
in the sense of viscosity. Observing that W " 2 C 2 .Rd /, we see that . " ; W " / is
the solution.
This fact shows that (4.172) is associated with the ergodic stochastic differen-
tial game.
6. Since " 2 Œ0; kf k1 and j@x W " j B, we can take "n ! 0, so that
"n
! 0 and W "n ! W 0 uniformly on any compact set. Therefore, . 0 ; W 0 /
satisfies (4.172) with " D 0, in the sense of viscosity, i.e., the small noise limit is
related to the ergodic differential game.
7. Let us consider the lower value function u" for the stochastic differential game
given by (4.173)–(4.174). Then u" satisfies (4.175)–(4.176), by changing v to u" .
8. Combining 5, 6, and 7, we see that the ergodic differential game related to the
small noise limit has the value.
For various applications of ergodic type stochastic controls, we can refer to
[Br06, BP99, FSh99, FSh00, FSh02, HNSh10] and [HS10].
Chapter 5
Stochastic Parabolic Equations
Abstract This chapter is devoted to stochastic evolution equations in Hilbert

spaces, in particular stochastic parabolic type equations of the form
d u.t; x; !/ D L.t; !/u.t; x; !/ dt C M.t; !/u.t; x; !/ d WQ .t/;
where L and M are second-order elliptic and first-order differential operators and
WQ is a colored Wiener process (see Example 5.1).
These equations are generalization of finite-dimensional SDEs and appear in the
study of random phenomena in natural sciences and the unnormalized conditional
probability of finite-dimensional diffusion processes (see Sect. 5.5), related to
filtering equations derived in Fujisaki et al. (Osaka J Math 9:19–40, 1972) and
Kushner (J Differ Equ 3:179–190, 1967).
In Sect. 5.1 we collect basic definitions and results for Hilbert space-valued pro-
cesses; in particular, for continuous martingales, quadratic variations and correlation
operators are treated. Stochastic integrals are introduced in Sect. 5.2. Section 5.3
is devoted to the study of stochastic parabolic equations from the viewpoint of
Hilbert space-valued SDEs, following Rozovskii (Stochastic evolution systems.
Kluwer Academic, Dordrecht/Boston, 1990). By using the results presented, we
also consider a semilinear stochastic parabolic equation with Lipschitz nonlinearity
in Sect. 5.3.4. Section 5.4 deals with Itô’s formula and in Sect. 5.5 Zakai equations
related to filtering problems are given.
5.1 Preliminaries
This section gives preliminaries for SDEs in Hilbert spaces. Here Hilbert space
always means separable real one. We first collect basic definitions and results for
stochastic processes taking values in Hilbert spaces. Proofs can be founded in the
accessible books (cf. [DaPZ92, M88, R90]).
For a Hilbert space H; k k WD k kH and . ; / WD . ; /H denote its norm and
inner product, respectively. H is equipped with a usual Borel field B.H/, generated
by the open sets. H denotes the conjugate space, namely the set of all continuous
linear functional on H. h 2 H can be identified with h 2 H, such that

154 5 Stochastic Parabolic Equations
h .z/ D .h; z/; 8z 2 H: (5.1)
B.H/ is nothing but the smallest -field which makes h measurable for any
h 2 H .
5.1.1 H-Random Variables
Let ˝ be a Polish space and .˝; F ; P / be a complete probability space, where F

denotes the completion of B.˝/ w.r.t. P .
Definition 5.1. X W ˝ 7! H is called an H-random variable, if X is F =B.H/-
measurable. X D XQ means P .X D XQ / D 1.
Let .ei ; i D 1; 2; : : : / be an ONB (orthonormal basis) of H. ˘N denotes
projection onto the linear space spanned by .e1 ; : : : ; eN / and ˘N? D 1 ˘N , where
1 is the identity map.
For an H-random variable X; X i WD .X; ei / is a real random variable and
X
n
lim X X i ei D 0 P -a.e. (5.2)
n!1
i D1
PX defined by PX .A/ D P .X 1 .A//; A 2 B.H/, is called the distribution (or law)

of X .
Expectation of X
When kX k is integrable, we write X 2 L1 .˝; RF I H/ and the expectation of X ,

denoted by EX , is defined as the integral EX D ˝ X.!/ dP .!/, as follows:
P
1. Let X be simple, say X D niD1 X i ei , with X i 2 L1 .˝; F I R1 /. Then set
X
n
EX D EX i ei : (5.3)
i D1
Hence, EX is an element of H satisfying
.EX; h/ D E.X; h/; 8h 2 H: (5.4)
Taking h D EX , we see that
kEX k EkX k: (5.5)

5.1 Preliminaries 155
2. For X 2 L1 .˝; F I H/, (5.5) implies that .EŒ˘n X ; n D 1; 2; : : : / is a Cauchy

sequence in H. Now define
EX D lim EŒ˘n X : (5.6)

n!1
Then (5.4) characterizes EX . Moreover EX does not depend on the choice of

the basis .ei ; i D 1; 2; : : : /.
Proposition 5.1. Properties of EX
(i) EŒaX C bY D aEX C bEY; 8a; b 2 R1 ;
(ii) .EX; h/ D E.X; h/; 8h 2 H;
(iii) Inequality
kEX kp EkX kp ; 8p 1I (5.7)
(iv) Convergence theorem

If kXn X k converges to 0 P -a.s. as n ! 1, and EŒsupn kXn k < 1, then
lim kEXn EX k D 0I (5.8)

n!1
(v) Let f W H 7! R1 be B.H/=B.R1/-measurable. Then f .X / is a real random

variable. If f .X / is integrable, then
Z
Ef .X / D f .h/ dPX .h/; (5.9)
H
where the integral in the RHS is the Bochner integral.
Conditional Expectation
Let X 2 L1 .˝; F I H/ and let a -subfield G F be given.

Definition 5.2. A G=B.H/-measurable H-random variable Y is called the condi-
tional expectation of X given G, if for any h 2 H
.Y; h/ D E..X; h/jG/ P -a.s.: (5.10)
Y exists and is unique P -a.s., and is denoted by E.X jG/.

Regarding the regular conditional probability, we refer to Theorem 1.1. Since F
is the completion of B.˝/ by P , the regular conditional probability given G exists
and is unique.
The following result is immediate.
Proposition 5.2. Properties of the conditional expectations
kE.X jG/k E.kX kjG/ P -a.s.; (5.11)

kE.X jG/k2 E.kX k2 jG/ P -a.s.; (5.12)
Q D E.E.X jG/jG/
E.X jG/ Q P -a.s. (5.13)
for GQ G F .
5.1.2 Continuous Martingales
Let .˝; F ; .Ft /; P / be a filtered probability space.

Definition 5.3. Let X I .T0 ; T / ˝ 7! H be given. X is called
1. An H-process, if it is B..T0 ; T // F =B.H/-measurable;
2. .Ft /-progressively measurable, if X=.T0 ;t ˝ is B..T0 ; t/ Ft =B.H/-measurable
for any t 2 .T0 ; T /;
3. .Ft /-adapted, if X.t/ is Ft =B.H/-measurable for any t 2 .T0 ; T /;
4. Continuous, if X has continuous paths, that is there exist X.T0 / and X.T / and
lim kX.s/ X.t/k D 0; 8t 2 ŒT0 ; T ; P -a.s.

s!t
If X is .Ft /-progressively measurable, then it is .Ft /-adapted. If X is continuous,

then the converse is true.
C.Œ0; T ˝; .Ft /I H/ denotes the set of all continuous and .Ft /-adapted H-
processes on Œ0; T .
Definition 5.4. M 2 C.Œ0; T ˝; .Ft /I H/ is called
1. An H-valued continuous .Ft /-martingale (more precisely, an .Ft ; P /-
martingale), if
(a) M.0/ D 0 P -a.s.;
(b) EkM.t/k < 1; 8t;
(c) Martingale property
For any s t,
E.M.t/jFs / D M.s/ P -a.s.I
equivalently, E..M.t/; h/jFs / D .M.s/; h/ P -a.s., for any h 2 H;

2. An H-valued continuous local .Ft /-martingale, if there is a sequence of .Ft /-
stopping times, n ; n D 1; 2; : : : , such that n is increasing to T , P -a.s., as n !
1, and M. ^ n / is a continuous .Ft /-martingale;
3. An H-valued continuous .Ft /-martingale is called square integrable, if

EkM.t/k2 < 1; 8t.
Similar notions are introduced for local martingales.
Further, we introduce the following notations:
Mc .Œ0; T ; .Ft /I H/ D set of H-valued continuous .Ft /-martingales,
M2c .Œ0; T ; .Ft /I H/ D set of H-valued square integrable continuous .Ft /-
martingales.
For local martingales, we use similar notations: Mcloc and M2cloc .
For M 2 M2c .Œ0; T ; .Ft /I H/, M i .t/ WD .M.t/; ei / is in M2c .Œ0; T ; .Ft /I R1 /
and
X
n
kM.t/k2 D lim M i .t/2 ; (5.14)
n!1
i D1
uniformly in t, P -a.s.
Hence kM./k2 is a continuous .Ft /-submartingale. Now the Doob–Meyer
decomposition asserts that there exists a unique continuous increasing .Ft /-adapted
process hM i./, such that
kM./k2 hM i./ 2 M c .Œ0; T ; .Ft /I R1 /: (5.15)
Definition 5.5. A continuous increasing .Ft /-adapted process N./, satisfying

kM./k2 N./ 2 M c ..Ft /I R1 /, is called the quadratic variational process of M
and is denoted by hM i./.
The following theorem justifies the following formal notation:
kdM.t/k2 D d hM i.t/:
Theorem 5.1.
X
n
hM i.t/ D lim hM i i.t/; uniformly in t; P -a.s.; (5.16)
n!1
i D1
EkM.t/ M./k2 D EŒhM i.t/ hM i./; (5.17)

ˇXln ˇ
ˇ 2 ˇ
lim E ˇ M.tjnC1 / M.tjn / .hM i.t/ hM i.//ˇ D 0; (5.18)
n!1
j Dkn
where Dn D .tjn ; j D 1; 2; : : : ; / is a division of Œ0; T satisfying Dn DnC1 and

maxj jtjnC1 tjn j < 2n , and ln D minflI t < tln g and kn D maxfkI tkn g.
The following statements are also useful.
Theorem 5.2. Let M 2 Mc .Œ0; T ; .Ft /I H/.

(i) kM./k is a real .Ft /-submartingale.
(ii) If EkM.T /kp < 1, for p > 1, then
p p
EŒ sup kM.t/kp EkM.T /kp :
0t T p1
(refer to [DaPZ92], p. 78).

For M; N 2 M2c .Œ0; T ; .Ft /I H/, we define hM; N i./ by
1
hM; N i.t/ D .hM C N i.t/ hM N i.t//; t 2 Œ0; T : (5.19)
4
Then hM; N i./ is a continuous .Ft /-adapted process which has bounded variation
and one has
.M./; N.// hM; N i./ 2 Mc .Œ0; T ; .Ft /I R1 /: (5.20)
Conversely suppose that ./ is continuous, bounded variation and .Ft /-adapted,
and moreover .M./; N.// ./ 2 Mc .Œ0; T ; .Ft /I R1 /.
Then
.t/ D hM; N i.t/; 8t 2 Œ0; T ; P -a.s.;
because a continuous martingale with bounded variation is 0 process.

Definition 5.6. hM; N i./ is called the quadratic variational process corresponding
to M and N .
The following corollary is immediate from Theorem 5.1.
Corollary 5.1.
hM; N i.t/ D hN; M i.t/; 8t; P -a.s.; (5.21)

X
hM; N i.t/ D hM i ; N i i.t/; 8t; P -a.s.; (5.22)
i
ˇ
ˇ
lim E ˇhM; N i.t/ hM; N i./
n!1
X ˇ
ˇ
.M.tjnC1 / M.tjn /; N.tjnC1 / N.tjn //ˇ D 0; (5.23)
j
where ftjn g is the same as in (5.18)
jhM; N i.t/ hM; N i./j2

.hM i.t/ hM i.//.hN i.t/ hN i.//; 8 < t; P -a.s. (5.24)
5.1.3 Correlation Operators
By (5.16) and (5.24) hM i ; M j i.dt/ is absolutely continuous w.r.t. hM i.dt/. Hence

there is a unique .Ft /-progressively measurable process q ij such that
Z t
hM ; M i.t/ D
i j
q ij .s/ d hM i.s/; 8t; P -a.s. (5.25)
0
Let us define qM .t; !/ 2 L.HI H/ by

X
qM .t; !/ei D q ij .t; !/ej ; i D 1; 2; : : : (5.26)
j
i.e., for any h; g 2 H,

Z t
hM ; M i.t/ D
h g
.qM .s/h; g/ d hM i.s/; 8t; P -a.s. (5.27)
0
where M h .t/ D .M.t/; h/ and M g .t/ D .M.t/; g/. Hence, qM .t/ becomes a
symmetric, nonnegative nuclear (trace class) operator on H with tr qM .t/ D 1.
Definition 5.7. qM is called the correlation operator of M (refer to [R90],
Ch. 2.1.13 for details). Generally, in our context a symmetric nonnegative nuclear
operator on H is called a covariance operator.
Example 5.1 (Colored Wiener process). M 2 M2c .Œ0; T ; .Ft /I H/ is called a
colored .Ft /-Wiener process with the covariance operator Q, if the following two
conditions are satisfied.
(a) hM i.t/ D t tr Q,
(b) qM .t/ D .tr Q/1 Q.
M can be expressed in terms of real Wiener processes in the following way.
Since we can take the ONB .eQi ; i D 1; 2; : : : / so that
QeQi D i eQi ; i D 1; 2; : : : (5.28)

P
with 1 2 0, and i i D tr Q.2 .0; 1//, we put M i .t/ D .M.t/; eQi /.
Then by (5.27) and (5.28)
hM i ; M j i.t/ D ti ıij : (5.29)
Thus, ˇ i .t/ D p1 M i .t/; i D 1; : : : ; l are mutually independent .Ft /-Wiener

i
processes, if l > 0. Consequently, if i > 0; i D 1; 2; : : : , then
1 p
X
M.t/ D i ˇ i .t/eQi in L2 .˝; C.Œ0; T //: (5.30)
i D1
If l > 0 and lC1 D 0, then
X
l p
M.t/ D i ˇ i .t/eQi :
i Dl
WQ denotes the colored Wiener process with covariance operator Q.
5.2 Stochastic Integrals
This section deals with Hilbert space valued stochastic integrals. We are mainly
concerned with stochastic integrals driven by colored Wiener processes. First we
treat a broad class. For an H-valued martingale M and a progressively measurable
integrand ˚ whose values R are Hilbert–Schmidt operators from H into Y, we define
the stochastic integral, ˚.s/ dM.s/, and list some of its basic properties in
Sect. 5.2.1, following [R90]. Section 5.2.2 is devoted to the Burkholder–Davis–
Gundy inequality for H-valued martingales.
5.2.1 Definitions and Basic Properties
First we define stochastic integral in the case H D R1 . Let 2 M2c .Œ0; T ; .Ft /I R1 /
be given. By L2 .hi P; .Ft /I Y/, we denote the set of all .Ft /-progressively
measurable Y-process , satisfying
Z T
E k.s/k2Y d hi.s/ < 1: (5.31)
0
Let .yi ; i D 1; 2; : : : / be an ONB of Y and 2 L2 .hi P; .Ft /I Y/. Put

n Z
X T
In .t/ D ..s/; yi /Y d .s/yi ; n D 1; 2; : : : : (5.32)
i D1 0
Then there is 2 M2c .Œ0; T ; .Ft /I Y/ such that
lim EŒsup k.t/ In .t/k2Y D 0; (5.33)

n!1 t
by the Burkholder–Davis–Gundy inequality

Rt for real martingales.
The stochastic integral I .t/ WD 0 .s/ d .s/ is defined by
I .t/ D .t/:
5.2 Stochastic Integrals 161
It is clear that
Z t
hI i.t/ D k.s/k2Y d hi.s/: (5.34)
0
Next we suppose that M 2 M2c .Œ0; T ; .Ft /I H/ is given. L2 .HI Y/ denotes the
Hilbert space of Hilbert–Schmidt operators, equipped with the norm
X 12
kkL2 .HIY/ D kei k2Y : (5.35)
i
By L2 .hM i P; .Ft /I L2 .HI Y// we denote the set of L2 .HI Y/-valued processes ˚,
such that .˚./h; y/Y is .Ft /-progressively measurable for any .h; y/ 2 H Y, and
Z T
E k˚.s/k2L2 .HIY/ d hM i.s/ < 1: (5.36)
0
Rt
We define the stochastic integral I˚ .t/ WD 0 ˚.s/ dM.s/ as an element of
M2c .Œ0; T ; .Ft /I Y/, following [R90], Ch. 2.2.2. We still omit the subscript H
R t . /H and kikH . For M .t/ D .M.t/; ei /,2cthe stochastic integral Ji .t/ WD
i
in
0 ˚.s/ei dM .s/ is given as an element of M .Œ0; T ; .Ft /I Y/ by (5.33). Define
˘np ˚ 2 L2 .hM i P; .Ft /I L2 .HI Y// by
(
˚.s/ei ; i D n; n C 1; : : : ; p;
˘np ˚.s/ei D
0; otherwise:
Pp
Since k i Dn Ji .t/k2Y is a real submartingale, we have
h X
p
2i h X
p
2i
E sup Ji .t/ 4E Ji .T /
0t T Y Y
i Dn i Dn
Z T 1
D 4E k˘np ˚.s/qM2 .s/k2L2 .HIY/ d hM i.s/
0
Z T
4E k˘np ˚.s/k2L2 .HIY/ d hM i.s/ ! 0 as n; p ! 1:
0
Now we define the stochastic integral.

Definition 5.8.
Z t n Z
X t
I˚ .t/ WD ˚.s/ dM.s/ D lim ˚.s/ei dM i .s/ in L2 .˝I C.Œ0; T I Y//:
0 n!1 0
i D1
(5.37)
The following properties are easy to verify.

Proposition 5.3. For I ; I˚ 2 M2c .Œ0; T ; .Ft /I Y/,
(i) Ia˚ Cb D aI˚ C bI ; P -a.s.; 8a; b 2 R1 ,
Rt RT
(ii) 0 ˚.s/ dM.s/ D 0 Œ0;t .s/˚.s/ dM.s/; 8t; P -a.s.,
Rt 1
(iii) hI˚ i.t/ D 0 k˚.s/qM2 .s/k2L2 .HIY/ d hM i.s/; P -a.s.,
Rt
(iv) hI˚ ; I i.t/ D 0 .˚.s/qM .s/; .s//L2 .HIY/ d hM i.s/; P -a.s.,
RT
(v) EŒsup0t T kI˚ .t/k2Y 4E 0 k˚.s/k2L2 .HIY/ d hM i.s/.
I˚ does not depend on the choice of ONB of H and Y.
Finally, we introduce a weaker condition for ˚:
Z T
k˚.s/k2L2 .HIY/ d hM i.s/ < 1 P -a.s.: (5.38)
0
We will define the stochastic integral of ˚ as a local martingale, in the same way as
in Step 3 of Sect. 1.1.3. Namely, put
8 n Z t o
<
inf t < T I k˚.s/k2L2 .HIY/ d hM i.s/ N ;
N D (5.39)
: 0
T; if f g D empty
and
˚N .s/ D Œ0; N .s/˚.s/: (5.40)
Then there is I˚ .t/, such that
I˚ .t/ D I˚N .t/ on . N t/; (5.41)

Rt
which implies that I˚ 2 M2c loc .Œ0; T ; .Ft /I Y/. We define 0 ˚.s/ dM.s/ as the
integral I˚ .t/ appearing in (5.41).
Example 5.2 (Stochastic integral driven by a colored Wiener process). Let WQ be
an .Ft /-Wiener process with covariance operator Q, say
Xp
WQ .t/ D i ˇ i .t/ei :
i
For ˚ 2 L2 .Œ0; T ˝; .Ft /I L2 .HI Y//, we introduce the stochastic integral I˚ by

Z t Xp Z t
I˚ .t/ D ˚.s/ d WQ .s/ D i ˚.s/ei dˇ i .s/ (5.42)
0 i 0
as an element of M2c .Œ0; T ; .Ft /I Y/.

X Z t
hI˚ i.t/ D i k˚.s/ei k2Y ds
i 0
Z t
D .˚.s/Q; ˚.s//L2 .HIY/ ds; 8t; P -a.s.; (5.43)
0
Z t
hI˚ ; I i.t/ D .˚.s/Q; .s//L2 .HIY/ ds; 8t; P -a.s.: (5.44)
0
Example 5.3 (Stochastic integral for H-processes). Let 2 L2 .hM i P; .Ft /I H/

be given. Under H D H , we can regard as an element R t of L .hM i
2
P; .Ft /I L .HI R // and define the stochastic integral I .t/ WD 0 ..s/; dM.s//
2 1
by (5.37), namely
Z t XZ t
..s/; dM.s// D ..s/; ei / dM i .s/: (5.45)
0 i 0
Thus, I is in M2c .Œ0; T ; .Ft /I R1 / and satisfies

Z t
hI ; I i.t/ D ..s/qM .s/; .s// d hM i.s/ (5.46)
0
and
h ˇZ t ˇ2 i Z T
ˇ ˇ
E sup ˇ ..s/; dM.s//ˇ 4E k.s/k2 d hM i.s/: (5.47)
0t T 0 0
Rt
Example 5.4 (Martingale part of kM.t/k2 ). Put M # .t/ D 0 .M.s/; dM.s//. Then
M # is in Mc .Œ0; T ; .Ft /I R1 / and 2M # .t/ gives the martingale part of kM.t/k2 ,
namely
kM.t/k2 D 2M #.t/ C hM i.t/: (5.48)
Indeed, for the exit time N of kM.t/k from Œ0; N ; M i .t/ WD .M.t/; ei / satisfies
Z t^ N
M i .t ^ N/
2
D2 M i .s/ dM i .s/ C hM i i.t ^ N /; 8t; P -a.s. (5.49)
0
Summing w.r.t. i leads to
kM.t ^ N /k
2
D 2M # .t ^ N/ C hM i.t ^ N /: (5.50)
Hence M # is in Mcloc .Œ0; T ; .Ft /I R1 /.

Further, (5.50) yields

h ˇZ ˇi 1 h i
ˇ ˇ
E sup ˇ .M.s/; dM.s//ˇ E sup kM.s/k2 C hM i.T /
0 N 0 2 sT
5
EhM i.T /: (5.51)
2
Thus the monotone convergence theorem for (5.50) implies that EjM #.t/j <
1 and (5.48).
Note that if M # 2 M2c .Œ0; T ; .Ft /I R1 /, then
Z t
hM #i.t/ D .M.s/qM .s/; M.s// d hM i.s/: (5.52)
0
Therefore,
d hM # i.t/ kM.t/k2 d hM i.s/: (5.53)
5.2.2 Martingale Inequalities
We have already stated a martingale inequality in Theorem 5.2. Now we are going
to prove the Burkholder–Davis–Gundy inequality for H-valued martingales.
Theorem 5.3. Let M 2 M2c .Œ0; T ; .Ft /I H/ and put
M .t/ D sup kM.s/k: (5.54)

0st
Then we have
1
EŒM .t/2 EhM i.t/ EŒM .t/2 ; (5.55)
4
cp EŒM .t/2p EhM i.t/p Cp EŒM .t/2p (5.56)
where
8 2p 2p p
ˆ
ˆ
< cp D p.2p 1/ ; Cp D .4p/p ; if p > 1;
2p 1
p (5.57)
ˆ
:̂ cp D p ; Cp D p 2p ; if p 2 .0; 1/:
16
Proof. Since (5.55) is clear from Theorem 5.2, we will prove (5.56), by using
arguments similar to those in the real martingale cases.
Let N be the exit time of kM.t/k from Œ0; N . By considering M.t ^ N / instead
of M.t/, we assume that M is bounded. We divide the proof into four steps.
Step 1. Compute cp for p > 1
Let " > 0 be given. Put X.t/ D kM.t/k2 . Applying Itô’s formula to (5.48) and
using (5.52), we obtain
d.X.t/ C "/p Dp.X.t/ C "/p1 .2dM # .t/ C d hM i.t//

C 2p.p 1/.X.t/ C "/p2 .M.t/qM .t/; M.t// d hM i.t/
2p.X.t/C"/p1 dM # .t/Cp.2p1/.X.t/C"/p1 d hM i.t/
(5.58)
since X.t/ 0. Letting " ! 0 leads to
EX.t/p p.2p 1/EŒM .t/2p2 hM i.t/

p1 1
p.2p 1/.EM .t/2p / p .EhM i.t/p / p ; (5.59)
which in conjunction with Theorem 5.2 (ii) yields the value of cp .

Step 2. Compute Cp for p > 1
Put
Z t
p1
N .t/ D
i
hM i.s/ 2 dM i .s/ (5.60)
0
and
X Z t p1
N.t/ D N .t/ei D
i
hM i.s/ 2 dM.s/: (5.61)
i 0
Then N 2 M2C .Œ0; T ; .Ft /I H/ and

Z t
1
hN i.t/ D hM i.s/p1 d hM i.s/ D hM i.t/p : (5.62)
0 p

Z t
p1 p1
N.t/ D hM i.t/ 2 M.t/ M.s/ d hM i 2 .s/ (5.63)
0
and
kN.t/k2 4M .t/2 hM i.t/p1 : (5.64)

Thus
1 p1
EhN i.t/ D EkN.t/k2 4.EM .t/2p / p .EhM i.t/p / p (5.65)
gives Cp by using (5.62).

Step 3. Compute cp for p 2 .0; 1/
Noting that
Z t
1p 1p
N.t/hM i.t/ 2 D M.t/ C N.s/ d hM i 2 .s/ (5.66)
0
by (5.61), we get
1p
kM.t/k 2N .t/hM i.t/ 2 ; (5.67)
where N .t/ D supst kN.s/k.

Since the RHS of (5.67) is increasing in t,
1p
M .t/ 2N .t/hM i.t/ 2 : (5.68)
Applying Hölder’s inequality, we have

1
EM .t/2p 4p .EN .t/2 / p .EhM i.t/p /1p .p 2 .0; 1// (5.69)
and now using Theorem 5.2 (ii) for N.t/, we obtain cp .

Step 4. Compute Cp for p 2 .0; 1/
Since, for " > 0,
hM i.t/p D hM i.t/p ." C M .t//2p.1p/ ." C M .t//2p.1p/ ; (5.70)
Hölder inequality yields
EhM i.t/p .EhM i.t/." C M .t//2.1p/ /p .E." C M .t//2p /1p : (5.71)
Let us put
Z t
N i .t/ D ." C M .s//p1 dM i .s/: (5.72)
0
Then, since p 1 < 0, it holds
hN i i.t/ ." C M .t//2.p1/ hM i i.t/ (5.73)

5.3 Stochastic Parabolic Equations with Colored Wiener Noises 167

X
N.t/ WD N i .t/ei
i
Z t

D M.t/." C M .t// p1
C .1 p/ M.s/." C M .s//p2 dM .s/
0
(5.74)
and
1
kN.t/k ." C M .t//p : (5.75)
p
Now (5.71), (5.73), and (5.75) yield
EhM i.t/p p 2p E." C M .t/2p /: (5.76)
Letting " ! 0, we obtain Cp .

This completes the proof.
For example, we have
p p p
EM .t/ 32E hM i.t/ 6E hM i.t/: (5.77)
5.3 Stochastic Parabolic Equations with Colored Wiener

Noises
This section is mainly concerned with the stochastic parabolic equation
nX
d X
d
d u.t; x; !/ D @i aij .t; x; !/@j u.t; x; !/ C b i .t; x; !/u.t; x; !/
i D1 j D1
o
C .t; x; !/u.t; x; !/ C z.t; x; !/ dt
X p nXd
j
C k gk .t; x; !/@j u.t; x; !/
k j D1
o
C „k .t; x; !/u.t; x; !/ C fk .t; x; !/ d W k .t/: (5.78)
We will consider (5.78) from the viewpoint of Hilbert space-valued SDE, following
[P79], [P93], [R90], Chs. 3 and 4. In Sect. 5.3.1, we introduce concepts, notations,
and basic results needed for later sections. We study the Cauchy problem for (5.78)
and properties of solutions in Sects. 5.3.2 and 5.3.3. By changing z.t; x; !/ to
z.t; x; u.t; x/; @x u.t; x/; !/, we will consider a semilinear stochastic parabolic
equation with Wiener noise in Sect. 5.3.4, based on results obtained in Sect. 5.3.3.
5.3.1 Preliminaries
In this subsection, we list some notations and basic results we need later.
Let X and H be Hilbert spaces. By X Õ H, we mean that the embedding; X 7! H
is continuous and dense. We use the identification H D H (D conjugate space of
Q is called normal, if X Õ H D H Õ X
H). The triple .X; H; X/ Q and there is a
constant N such that
j.y; x/H j N kykXQ kxkX ; 8x 2 X; y 2 H: (5.79)
Q be normal and x 2 X and y 2 X

Let .X; H; X/ Q be given. Since there is a sequence
yn 2 H .n D 1; 2; : : : / such that limn!1 kyn ykXQ D 0, we can define the dual
product between y and x by
hy; xi.X;X/
Q D lim .yn ; x/H : (5.80)
n!1
The RHS of (5.80) does not depend on the choice of .yn /, and
jhy; xi.X;X/
Q j N kykXQ kxkX ; Q
8y 2 X; x 2 X: (5.81)
Let WQ be an H-valued Wiener process with covariance operator Q. Let .ek ; k D

1; 2; : : :/ be an ONB of H with
Qek D k ek ; k D 1; 2; : : : : (5.82)
Put
X
L2Q .HI H/ D f 2 L.HI H/I k k2Q WD k k ek k2H < 1; g (5.83)
k
1
i.e., 2 L2Q .HI H/ means that Q 2 is a Hilbert-Schmidt operator.
Q and G.t; !/ 2 L.XI L2 .HI H//, consider the SDE
For given A.t; !/ 2 L.XI X/ Q
d u.t; !/ D.A.t; !/u.t; !/ C Z.t; !// dt

C .G.t; !/u.t; !/ C .t; !// d WQ .t; !/; t 2 .T0 ; T (5.84)
u.T0 ; !/ D u0 .!/: (5.85)
Assume the following conditions are satisfied:

Q and
.A0 / For any 2 X, A.t/ and G.t/ are .Ft /-progressively measurable X-
L2Q .HI H/-valued processes, respectively,
.A1 / There is a constant K, such that
kA.t/k2XQ C kG.t/k2Q Kkk2X ; 8 2 X and 8t; P -a.s.:
.A2 / Coerciveness
There are constants N > 0 and N 2 R1 , such that
N
CkG.t/k2Q kk Q Ckk
N 8 2 X; 8t;
2
2hA.t/; i.X;X/
Q X H; P -a.s.:
Q
.A3 / Z 2 L2 .ŒT0 ; T ˝; .Ft /I X/.
.A4 / 2 L .ŒT0 ; T ˝; .Ft /I L2Q .HI H//.
2
.A5 / u0 2 L2 .˝; FT0 ; H/.

Q we define the solution as follows;
Considering (5.84) on (X; H; X),
Definition 5.9. The X-valued (Ft )-progressively measurable process u./ is called
a solution of (5.84)–(5.85) if
u 2 L2 .ŒT0 ; T I X/ \ C.ŒT0 ; T I H/ P -a.s.
and, for any from some countable dense subset D of X,
.u.t/; /H D.u0 ; /H
Z t
C hA.s/u.s/ C Z.s/; i.X;X/
Q ds
T0
Z t
C ..G.s/u.s/ C .s// d WQ .s/; /H ; 8t; P -a.s. (5.86)
T0
We identify two solutions u1 ./ and u2 ./, if
P .ku1 .t/ u2 .t/kH D 0; 8t 2 ŒT0 ; T / D 1:
Remark 5.1. Noticing that

Z T Z T
kG.s/u.s/k2Q ds K ku.s/k2X ds < 1; P -a.s.
T0 T0
Rt
we confirm that M.t/ WD T0 G.s/u.s/ d WQ .s/; t 2 ŒT0 ; T is a continuous local
martingale with
Z t
hM i.t/ D kG.s/u.s/k2Q ds:
T0
Theorem 5.4 (Theorem 3.1.4 in [R90], p. 90). Under conditions .A0 /–.A5 /, the
SDE (5.84)–(5.85) has the unique solution u./, such that
u 2 L2 .ŒT0 ; T ˝; .Ft /I X/ \ L2 .˝I C.ŒT0 ; T I H//
and, for any 2 X, (5.86) holds.

Further, (i) and (ii) below are valid.
(i) Energy equality
For any (Ft )-stopping time taking values in ŒT0 ; T ,
ku.t ^ /k2H D ku0 k2H C

Z t^
2hA.s/u.s/ C Z.s/; u.s/i.X;X/
Q C kG.s/u.s/ C ˚.s/k2Q ds
T0
Z t^
C2 .G.s/u.s/ C .s// d WQ .s/; u.s/ ;
T0 H
8t 2 ŒT0 ; T ; P -a.s.; (5.87)
(ii) Estimate
h Z T i
E sup ku.t/k2H C ku.t/k2X dt
T0 t T T0
h Z T i
NE ku0 k2H C .kZ.s/k2XQ C k .s/k2Q / ds ; (5.88)
T0
N and .
where N is a constant depending only on K; , N
From now on, we are mainly concerned with L2 .Rd / as H and introduce the
following notations:
H0 WD L2 .Rd / with the usual inner product . ; / and the norm k k.
Hp WD f 2 H0 ; the generalized derivatives of order p belong to H0 g,
endowed with the inner product
p p
.; /Hp D ..1 / 2 ; .1 /2 / (5.89)
p
and the norm kkHp D k.1 / 2 k, where 1 is the identity map and is the
d -dimensional Laplacian operator.
For example, the triple (H2 ; H1 ; H0 ) is normal and is used later in Proposi-
tion 5.4.
1
H1 WD fI is a Borel function such that .1 / 2 2 H0 g;
endowed with the inner product and the norm such that
1 1
.; /H1 D ..1 / 2 ; .1 / 2 / (5.90)
and
1
kkH1 D k.1 / 2 k
D sup.j.; /jI 2 H1 with k kH1 D 1/: (5.91)
The triple (H1 ; H0 ; H1 ) is also normal. For simplicity, we put
jjj jjj D k kH1 ; k k D k kH1 ; I IDk kH2 ;

h ; i D h ; i.H1 ;H1 / ; .. ; // D . ; /H1 :
Then it is easy to see that
jhy; xij kyk jjjxjjj for y 2 H1 ; x 2 H1 ;

hy; xi D .y; x/ for y 2 H0 ; x 2 H1 : (5.92)
(Refer to [A03] for Hp ).
5.3.2 Linear Stochastic Parabolic Equations
Let us come back to (5.78). Here we will study equation (5.78) in the Hilbert space
framework. But we firstly introduce the notion of generalized solution from the
viewpoint of parabolic equations (cf. [R90], p. 130).
Let
u0 2 L2 .˝; FT0 I H0 /: (5.93)
The .Ft /-progressively measurable H1 -process u./ is called a generalized solution

of (5.78) with u.T0 / D u0 , if u 2 L2 .ŒT0 ; T I H1 / \ C.ŒT0 ; T I H0 /, P -a.s., and for
any 2 CK1 .Rd /,
.u.t/; / D .u0 ; /
d Z t X
X d
aij .s; /@j u.s/ C b i .s; /u.s/; @i ds
i D1 T0 j D1
Z t
C ...s; /u.s/; / C hz.s; /; i/ ds
T0
Xp Z t X
d
j
C k gk .s; /@j u.s/C„k .s; /u.s/Cfk .s; /; d W k .s/;
k T0 jD1
8t 2 ŒT0 ; T ; P -a.s.; (5.94)

P
where 1 2 0 and D k k 2 .0; 1/. Two generalized solutions u
and uQ are identified, if
P .ku.t/ uQ .t/k D 0; 8t 2 ŒT0 ; T / D 1: (5.95)
We assume the following three conditions;

.B1 / z D aij ; b i ; gki ; „k ; ; i; j D 1; : : : ; d; k D 1; 2; : : : satisfy
(a) B.Œ0; T Rd / F =B.R1/-measurable.
(b) For any x 2 Rd ; z.; x/ is.Ft /-progressively measurable.
(c) Boundedness
There is a constant K, such that
sup jz.t; x/j K; P -a.s.: (5.96)

t;x
.B2 / Superparabolicity
X
j
^.t; x/ WD 2aij .t; x/ k gki .t; x/gk .t; x/
ij D1;:::;d
k
is symmetric and uniformly positive definite:
y > ^ .t; x/y 0 .y/2 ; 8y 2 Rd ; 8t; x; P -a.s.; (5.97)
with a positive constant 0 .

.B3 / z and fk .k D 1; 2; : : : / satisfy
z 2 L2 .Œ0; T ˝; .Ft /I H1 /;

X Z T
fk 2 L2 .Œ0; T ˝; .Ft /I H0 / with k E kfk .s/k2 ds < 1:
k 0
We will consider (5.78) on (H1 ; H0 ; H1 ). Let ei ; i D 1; 2; : : : be an ONB in

H0 , and put
Xp
WQ .t/ D k W k .t/ek :
k
Next let us define A.t; !/ 2 L.H1 I H1 /; Gk .t; !/ 2 L.H1 I H0 /,

G.t; !/ 2 L.H1 I L2Q .H0 I H0 // and 2 L2 .Œ0; T ˝; .Ft /I L2Q .H0 I H0 // by
X
d X
d
A.t; !/ D @i aij .t; x; !/@j C b i .t; x; !/ C .t; x; !/; (5.98)
i D1 i D1
X
d
Gk .t; !/ D gki .t; x; !/@j C „k .t; x; !/; (5.99)
j D1
G.t; !/ek D Gk .t; !/; (5.100)
and
.t; !/ek D fk .t; x; !/ (5.101)
for 2 H1 .
Thus, (5.78) can be recast as the following SDE on .H1 ; H0 ; H1 /:
d u.t/ D .A.t/u.t/ C z.t// dt C .G.t/u.t/ C .t// d WQ : (5.102)
Since CK1 .Rd / is dense in H1 and since (5.96) and (B2 ) imply (A1 ) and (A2 ),
Theorem 5.4 leads to the following
Theorem 5.5. Under conditions (B1 )–(B3 ), Eq. (5.78) has a unique generalized
solution u./ 2 L2 .ŒT0 ; T ˝; .Ft /I H1 /\L2 .˝I C.ŒT0 ; T I H0 // with u.T0 / D u0 .
Moreover, (i) and (ii) hold:
(i) Energy equality
For any (Ft )-stopping time taking values in ŒT0 ; T ,
ku.t ^ /k2 Dku0 k2

Z t^
C .2hA.s/u.s/ C z.s/; u.s/i C kG.s/u.s/ C .s/k2Q / ds
T0
Z t^
C2 ..G.s/u.s/ C .s// d WQ .s/; u.s//;
T0
8t 2 ŒT0 ; T ; P -a.s.:
(ii) Estimate
h Z T i
E sup ku.s/k2 C jjju.s/jjj2 ds
T0 S T T0
h Z T i
NE ku0 k2 C .kz.s/k2 C k .s/k2Q / ds ; (5.103)
T0
where N is a constant depending only on K and 0 .

5.3.3 Regularities of Solutions
Let us introduce the following two conditions (B4 ) and (B5 );

(B4 ) There is a constant K1 , such that
8
< sup kz.t; /kC 2 K1 ; P -a.s. for z D aij ; bi ;
t
(5.104)
: sup kz.t; /kC 1 K1 ; P -a.s. for z D gki ; „k :
t
(B5 ) (a) z 2 L2 .Œ0; T ˝; .Ft /I H0 /,

(b) fk 2 L2 .Œ0; T ˝; .Ft /I H1 /; k D 1; 2; : : :
P RT
with k E 0 jjjfk .s/jjj2 ds < 1.
Let k ; k D 1; 2; : : : be an ONB in H1 and define G and by (5.100) and (5.101)
with k instead of ek . Then (5.104) provides a constant K2 , such that for any 2
H2 ,
kA.t/k2 C jjjG.t/jjj2Q K2 II2 ; 8t; P -a.s. (5.105)
Further (B2 ) and (B4 ) yield the coerciveness on (H2 ; H1 ; H0 ), namely there is a
constant 0 .2 R1 / depending only on K1 ; 0 , and , such that for any 2 H2 ,
2hA.t/; i.H0 H2 / C jjjG.t/jjj2Q

0
II C 0 jjjjjj2 ; 8t; P -a.s. (5.106)
2
Indeed, when 2 H3 , standard computations lead to following relations:
h@i .aij @j /; i.H0 H2 / D ..@i .aij @j /; //

D .@i .aij @j /; .1 //
D .a @j ; .1 /@i /;
ij
ˇˇˇX ˇˇˇ2 X X j
ˇˇˇ ˇˇˇ
ˇˇˇ gki @i ˇˇˇ D gki @i ; .1 / gk @j :
i i j
For any " > 0, there is a constant K" , independent of , such that
.aij @j ; @l l @i / D .aij @j .@l /; @i .@l // .@l aij @j ; @l @i /

1st term C "II2 C K" jjjjjj2 ;
j j
.gki @i ; @l l .gk @j // .gki @i .@l /; gk @j .@l // C "II2 C K" jjjjjj2 :
We apply the same calculations to other terms in the LHS of (5.106). Then letting
" small, depending only on 0 and K1 , and putting all estimates together, we
conclude (5.106) for 2 H3 .
For 2 H2 , we can take n 2 H3 , so that limn!1 I n I D 0. Hence, (B4 )
and (5.105) yield
jhA.t/ n; n i.H0 H2 / hA.t/; i.H0 H2 / j C jjjG.t/. n /jjj2Q ! 0; 8t; P -a.s.
This completes the proof of (5.106).

Proposition 5.4. Suppose u0 2 L2 .˝; FT0 I H1 /. Under conditions (B1 ), (B2 ), (B4 )
and (B5 ), the unique generalized solution u./ of (5.78) with u.T0 / D u0 is in
L2 .ŒT0 ; T ˝; .Ft /I H2 / \ L2 .˝I C.ŒT0 ; T I H1 //.
Moreover, one has the estimate
h Z T i
2
E sup jjju.t/jjj C Iu.t/I2 dt
T0 t T T0
h Z T i
2
NE jjju0 jjj C .kz.s/k2 C jjj .s/jjj2Q / ds (5.107)
T0
with a constant N depending only on K1 , 0 , and .

Proof. We use the arguments similar to those in the proof of Theorem 5.5.
Let k ; k D 1; 2; : : : be ONB in H1 and put
Xp
WQ .t/ D W k .t/k :
k
We consider on .H2 ; H1 ; H0 / the SDE
d v.t/ D .A.t/v.t/ C z.t// dt C .G.t/v.t/ C .t// d WQ .t/; t 2 .T0 ; T ;

(5.108)
v.T0 / D u0 : (5.109)
Then Theorem 5.4 yields the unique solution of problem (5.108)–(5.109) in the
space L2 .ŒT0 ; T ˝; .Ft /I H2 / \ L2 .˝I C.ŒT0 ; T I H1 //. Hence, for any 2 H2 ,
..v.t/; // D..u0 ; //
Z t
C hA.s/v.s/ C z.s/; i.H0 H2 / ds
T0
Xp Z t
C k ..Gk .t/v.t/ C fk .t/; // d W k .t/; t 2 .T0 ; T :
k T0
(5.110)
For 2 Ck1 .Rd /, we take D .1 /1 . Then by (5.110),

Z t
.v.t/; / D .u0 ; / C hA.s/v.s/ C z.s/; i ds
T0
Xp Z t
C k .Gk .s/v.s/ C fk .s/; / d W k .s/:
k T0
Consequently, v./ coincides with the unique generalized solution of (5.78) with
u.T0 / D u0 .
For the dependence on the time parameter, we have;
Proposition 5.5. Suppose z D 0 and fk D 0; k D 1; 2; : : : . Let u./ be the unique
generalized solution of (5.78) with u.T0 / D u0 .
(i) Let u0 2 L2 .˝; FT0 I H1 / and ; t 2 ŒT0 ; T . Under .B1 /, .B2 / and .B4 /,
h i
E sup ku.s/ u./k2 N1 jt jEjjju0 jjj2 ; (5.111)
st
with a constant N1 depending only on 0 , , and K1 .

(ii) Let u0 2 L2 .˝; FT0 I H2 / and ; t 2 ŒT0 ; T . Besides .B1 / and .B2 /, we assume
a stronger condition
.B6 / There is a constant K3 , such that
sup kz.t; /kC 3 K3 ; P -a.s. for z D aij ; b i ;

t
sup kz.t; /kC 2 K3 ; P -a.s. for z D gki ; „k :

t
Then
h i
E sup jjju.s/ u./jjj2 N2 .t /EIu0 I2 ; (5.112)
st
with a constant N2 depending only on 0 , , and K3 .

Proof. (i) From Proposition 5.4, it follows that u./ is in L2 .˝; F I H1 / and
v.s/ WD u.s/ u./; s 2 Œ; T , satisfies on (H1 ; H0 ; H1 ) the SDE
8
ˆ
ˆ d v.s/ D .A.s/v.s/ C A.s/u.// ds
<
C .G.s/v.s/ C G.s/u.// d W .s/; s 2 .; T ; (5.113)
ˆ
:̂
v./ D 0:
Further, z./ D A./u./ and ./ D G./u./ are in L2 .Œ; T ˝; .Ft /I H1 / and
L2 .Œ; T ˝; .Ft /I L2Q .H0 I H0 //, respectively. (B1 ) yields
kz.s/k2 C k k2Q K0 jjju./jjj2 ; 8s 2 Œ; T ; P -a.s.; (5.114)
with a constant K0 depending only on K of (5.96). Now (5.103) and (5.114)

yield (5.111).
Since (ii) mimics (i), this completes the proof of Proposition 5.5.
5.3.4 Semilinear Stochastic Parabolic Equations with Lipschitz

Nonlinearity
Let us change z.t; x; !/ of (5.78) to z.t; x; u; v; !/ with a Lipschitz condition in the

variables u and v. Precisely, we assume
.B7 / z W Œ0; T Rd R1 Rd ˝ 7! R1 is a measurable map, satisfying
(a) z.; x; u; v/ is (Ft )-progressively measurable for any x; u; v.
(b) Set z0 .t; x; !/ D z.t; x; 0; 0; !/ and z1 .t; x; u; v; !/ D z.t; x; u; v; !/
z0 .t; x; !/. There is a positive constant l0 , such that
jz1 .t; x; u1 ; v1 / z1 .t; x; u2 ; v2 /j2

l0 .ju1 u2 j2 C jv1 v2 j2 /; 8t; x; P -a.s.: (5.115)
(c) z0 is in L2 .Œ; T ˝; .Ft /I H0 /.

Now we consider the semilinear stochastic parabolic equation
d u.t; x/ D.A.t/u.t; x/ C z.t; x; u.t; x/; @x u.t; x/// dt

Xp
C k .Gk .t/u.t; x/ C fk .t; x// d W k .t/; (5.116)
k
where A.t/ and Gk .t/ are given by (5.98) and (5.99), respectively.
The definition of generalized solution for (5.116) is given in the same way
as (5.94), by replacing hz.s; /; i with hz.s; ; u.s/; @x u.s//; i.
Let us introduce the SDE on .H1 ; H0 ; H1 / associated with (5.116), whose
solution provides the generalized solution of (5.116) (see (5.118)).
Using (b) and (c), we can define Z W Œ0; T H1 ˝ 7! H0 by
Z.t; h; !/.x/ D z.t; x; h.x/; @x h.x/; !/

D z1 .t; x; h.x/; @x h.x/; !/ C z0 .t; x; !/ for h 2 H1 : (5.117)
Hence, for 2 L2 .Œ0; T ˝; .Ft /I H1 /, .t/ WD Z.t; .t// is defined as an element
of L2 .Œ0; T ˝; .Ft /I H0 / by (b) and (c), and
Z T Z T
E kZ.t; 1 .t// Z.t; 2 .t//k dt l0 E 2
jjj1 .t/ 2 .t/jjj2 dt
0 0
for 1 ; 2 2 L2 .Œ0; T ˝; .Ft /I H1 /.

This subsection is devoted to studying the similinear SDE on .H2 ; H1 ; H0 /
d u.t/ D .A.t/u.t/ C Z.t; u.t/// dt

C .G.t/u.t/ C .t// d WQ .t/; t 2 ŒT0 ; T ; (5.118)
that is
nX
d X
d
d u.t/ D @i aij .t; x/@j u.t/ C b i .t; x/u.t/
i D1 j D1
o
C .t; x/u.t/ C z.t; x; u.t; x/; @x u.t; x// dt
X p nXd
j
o
C k gk .t; x/@j u.t/ C „k .t; x/u.t/ C fk .t; x/ d W k .t/;
k i D1
u.T0 / D u0 .2 L2 .˝; FT0 I H1 //: (5.119)
Now we aim to prove;

Proposition 5.6. Under (B1 ), (B2 ), (B4 ), (B7 ) and (B5 )(b), problem (5.71)–(5.72)
has a unique solution u./ in L2 .ŒT0 ; T ˝; .Ft /I H2 / \ L2 .˝I C.ŒT0 ; T I H1 //.
Moreover, the following estimate holds
h Z T i
E sup jjju.t/jjj2 C Iu.s/I2 ds
T0 t T T0
Z T
NQ EŒjjju.0/jjj2 C E .kz0 .t/k2 C jjj .t/jjj2Q / dt ; (5.120)
T0
where the constant NQ depends only on 0 , K1 and l0 .

Proof. We divide the proof into four steps.
Step 1. Approximate Solutions We construct a solution by using the successive
approximation. Since the method is standard, we will only sketch the proof.
Put u0 .t/ D u0 and define u1 ./ as the solution of the SDE

8
ˆ
ˆ d u1 .t/ D .A.t/u1 .t/ C Z.t; u0 .t// dt
<
C .G.t/u1 .t/ C .t// d WQ .t/; t 2 .T0 ; T ; (5.121)
ˆ
:̂
u1 .T0 / D u0 :
Since Z.t; u0 .t// is an element of L2 .ŒT0 ; T ˝; .Ft /I H0 / with

Z T hZ T i
E kZ.t; u0 .t//k dt 2E
2
kz0 .t/k2 dt C l0 jjju0 jjj2 .T T0 / ;
0 T0
(5.122)
(5.121) admitsthe unique solution u1 ./ in L2 .ŒT0 ; T ˝; .Ft /I H2 / \
L2 .˝I C.ŒT0 ; T I H1 /, by Proposition 5.4.Now suppose un1 ./ 2 L2 .ŒT0 ; T
˝; .Ft /I H2 /\L2 .˝I C.ŒT0 ; T I H1 // is the solution of the approximate problem
of index n 1. We define un ./ as the solution of the SDE
8
ˆ
ˆ d un .t/ D .A.t/un .t/ C Z.t; un1 .t/// dt
<
C .G.t/un .t/ C .t// d WQ .t/; t 2 .T0 ; T ; (5.123)
ˆ
:̂
un .T0 / D u0 :
Again observing that

Z T
E kZ.t; un1 .t//k2 dt
T0
hZ T i
2E .kz0 .t/k2 C l0 jjjun1 .t/jjj2 / dt < 1; (5.124)
T0
we obtain the unique solution of (5.123) in L2 .ŒT0 ; T ˝; .Ft /I H2 / \

L2 .˝I C.ŒT0 ; T I H1 //, satisfying
h Z t i
n .t/ WDE sup jjjun .s/jjj2 C Iun .s/I2 ds
T0 st T0
h Z t i
NE jjju0 jjj2 C .kZ.s; un1 .s//k2 C jjj .s/jjj2Q / ds
T0
Z t
NEjjju0 jjj2 C NE .2kz0 .s/k2 C jjj .s/jjj2Q / ds
T0
Z t
C 2N l0 n1 .s/ ds; (5.125)
T0
with a constant N depending only on K1 , 0 , and .

Applying Gronwall’s inequality, we obtain
X
n1
.C0 .t T0 //j 1
n .t/ NEjjju0 jjj2
j D1
.j 1/Š
Z X
n1
t
.C0 .t s//j 1
CN EŒ2kz0 .s/k2 C jjj .s/jjj2Q ds
T0 j D1
.j 1/Š
.C0 .t T0 //n
C Ejjju0 jjj2 ; (5.126)
nŠ
where C0 D 2N l0 .
Step 2. Convergence of un ./
Put !1 D u1 and !nC1 D unC1 un ; n D 1; 2; : : :. Then !nC1 satisfies
8
ˆ
ˆ d!nC1 .t/ D .A.t/!nC1 .t/ C Z.t; un .t// Z.t; un1 .t/// dt
<
C G.t/!nC1 .t/ d WQ .t/; t 2 .T0 ; T ; (5.127)
ˆ
:̂
!nC1 .T0 / D 0:
Since (5.107) and (5.115) yield

h Z t i
nC1 .t/ WD E sup jjj!nC1 .s/jjj2 C I!nC1 .s/I2 ds
T0 st T0
Z t
N l0 n .s/ ds; (5.128)
T0
we have
.C0 .t T0 //nC1
nC1 .t/ sup 1 .s/: (5.129)
.n C 1/Š T0 st
Therefore,
hX Z T X i
E sup jjj!n .s/jjj C I!n .s/I ds < 1: (5.130)
n T0 sT T0 n
P Pn
Since supT0 sT n jjj! P n .s/jjj < 1, P -a.s., we deduce that un ./ D kD1 !k ./
converges to u./ WD k !k ./ in C.ŒT0 ; T I H1 /, P -a.s.
On the other hand, un ./ converges to some uQ ./ in L1 .ŒT0 ; T I H2 /, P -a.s.,
by (5.130). Therefore, un ./ converges to uQ ./ in L1 .ŒT0 ; T I H1 /, P -a.s., and so
u.t/ D uQ .t/, a.e. t 2 ŒT0 ; T , P -a.s. By using (5.126) and the Fatou’s lemma, we
conclude that
u 2 L2 .ŒT0 ; T ˝; .Ft /I H2 / \ L2 .˝I C.ŒT0 ; T I H1 //: (5.131)

Step 3. u./ satisfies (5.118)

For 2 H2 , we have the following estimates:
h ˇZ t ˇi
ˇ ˇ
E sup ˇ hA.s/.un .s/ u.s//; i.H0 ;H2 / ds ˇ
T0 t T T0
Z T
KI IE Iun .s/ u.s/I ds; (5.132)
T0
h ˇZ t ˇi
ˇ ˇ
E sup ˇ hZ.s; un .s// Z.s; u.s//; /i.H0 ;H2 / ds ˇ
T0 t T T0
p Z T
l0 I IE jjjun .s/ u.s/jjj ds: (5.133)
T0
For the stochastic integral term, the Burkholder–Davis–Gundy inequality yields

h ˇZ t ˇi
ˇ ˇ
E sup ˇ ..G.s/.un .s/ u.s// d WQ ; //ˇ
T0 t T T0
Z T 12
6E ..G.s/.un .s/ u.s//; //2Q ds
T0
s Z
T
6Kjjj jjj E Iun .s/ u.s/I2 ds: (5.134)
T0
Combining (5.132)–(5.134) and letting n ! 1, we obtain (5.118).

Step 4. Uniqueness and (5.120)
Let v./ be a solution and put !.t/ D u.t/ v.t/. Since !./ satisfies
d!.t/ D .A.t/!.t/CZ.t; u.t//Z.t; v.t///dtCG.t/!.t/d WQ .t/; t 2 .T0 ; T

(5.135)
!.T0 / D 0; (5.136)
(5.107) shows that .t/ WD EŒsupT0 st jjj!.s/jjj2 satisfies

Z t Z t
.t/ NE kZ.s; u.s// Z.s; v.s//k2 N l0 .s/ds: (5.137)
T0 T0
Now Gronwall’s inequality yields
.t/ D 0; t 2 ŒT0 ; T ;
which establishes the uniqueness of solution.

Regarding (5.120) for v./, we put

h Z t i
K.t/ D E sup jjjv.s/jjj2 C Iv.s/I2 ds :
T0 st T0
Again (5.107) and .B7 / show that

Z t
2
K.t/ NEŒjjju0 jjj C .kZ.s; v.s/k2 C jjj jjj2Q / ds
T0
Z t Z t
c1 EŒjjju0 jjj2 C 2 2
k.z0 .s/k C jjj jjjQ /ds C K.s/ ds ; (5.138)
T0 T0
with a constant c1 independent of t and v./. Now Gronwall’s inequality

yields (5.120).
The proof is completed.
If 1 D 2 D D m D 1 and mC1 D 0, then WQ is an mdimensional
Wiener process and we write k km instead of k kQ .
Example 5.5 (HJB equation with noise). Let us consider a time-homogeneous HJB
equation affected by a Wiener noise;
nX
d X
d
d u.t; x/ D @i aij .x/@j u.t; x/
i D1 j D1
X
d o
C sup b i .x; /@i u.t; x/ C .x; /u.t; x/ C z.x; / dt
2 i D1
m X
X d
C gki .x/@i u.t; x/ C „k .x/u.t; x/ C fk .x/ d W k .t/;
kD1 i D1
t 2 .0; T ; x 2 Rd ; (5.139)
u.0; x/ D .x/; x 2 Rd ; (5.140)
where 2 H1 .
We assume that
(a) is convex and compact,
(b) All coefficients are time-homogeneous and bounded continuous non-random
functions,
(c) aij 2 Cb2 .Rd /; gki ; „k 2 Cb1 .Rd /; ij D 1; : : : ; d; k D 1; : : : ; m,
5.4 Itô’s Formula 183
(d) Superellipticity
X
m
j
2aij .x/ gki .x/gk .x/
ij D1;:::;d
kD1
is uniformly positive definite,

(e) fk 2 H1 ; k D 1; : : : ; m, and there exists zO 2 H0 , such that jz.x; /j
zO.x/; 8x; .
Put
X
d
z.x; u; v/ D sup b i .x; /vi C .x; /u C z.x; / :
2 i D1
Then the SDE on .H2 ; H1 ; H0 /, associated with (5.139)–(5.140) has a unique

solution, which provides the unique generalized solution of (5.139)–(5.140).
Stochastic HJB equations with lateral condition can be treated via backward SDE
arguments. Refer to [Pe92, BM07] for details.
5.4 Itô’s Formula
This section is devoted to Itô’s formula for H-valued semimartingale, in particular

to the solution of linear stochastic parabolic equation (5.102). First we consider a
continuous semimartingale X given by
Z t
X.t/ D X0 C .s/ ds C M.t/; (5.141)
0
where 2 L2 .Œ0; T ˝; .Ft /I H/, X0 2 L2 .˝; F0 I H/, M 2 M2c .Œ0; T ; .Ft /I H/.
In Sect. 5.4.2, we will show that, for F 2 C 12 .Œ0; T H/ satisfying local
boundedness condition (F0 ), Itô’s formula holds:
F .t ^ ; X.t ^ // D F .0; X0 /
Z t^ n o
C @s F .s; X.s// C ..s/; DF .s; X.s///H ds
0
Z t^
C .DF .s; X.s//; dM.s//H
0
Z t^
1
C tr.D 2 F .s; X.s//qM .s// d hM i.s/; 8t; P -a.s.
2 0
(5.142)
for any (Ft )-stopping time taking values in Œ0; T . In the differential form,
dF .t; X.t// Df@t F .t; X.t// C ..t/; DF .t; X.t///H g dt

1
C .DF .t; X.t//; dM.t//H C tr.D 2 F .t; X.t//qM .t// d hM i.t/:
2
In Sect. 5.4.3, we study Itô’s formula for the solution of (5.102). In this case,
we impose stronger conditions on F , to deal with the unbounded operator A.t/
(see .F1 / in Sect. 5.4.3). When F .t; / D kk2 , Itô’s formula leads to the energy
equality. We also estimate the negative norm of the solution by using Itô’s formula.
5.4.1 Preliminaries
This subsection lists basic definitions and results that will be needed in the sequel.
We state linear- and bilinear maps, Hilbert–Schmidt operators, nuclear operators
and first and second derivatives.
Proofs are given in accessible books, cf. [La83, Y80].
1. Linear- and bilinear maps
Let X; Y, and Z be Hilbert spaces. L.YI Z/ denotes the Banach space of
continuous linear mappings from Y into Z, endowed with the norm k kL.YIZ/ given
by
k˚kL.YIZ/ D sup k˚ykZ :

kykY D1
A map W X Y 7! Z is said to be bilinear, if .x; / 2 L.YI Z/ for each fixed

x 2 X, and .; y/ 2 L.XI Z/ for each fixed y 2 Y. L.X; YI Z/ denotes the set of
all bilinear maps from X Y into Z, with the norm
k kL.X;YIZ/ D sup k .x; y/kZ :

kxkX ;kykY D1
2 L.X; YI Z/ can be identified with O 2 L.XI L.YI Z// by
.x; y/ D O .x/y: (5.143)
This identification gives an isometric isomorphism (norm preserving linear map)

between L.X; YI Z/ and L.XI L.YI Z//.
Example 5.6. Let Z D R1 and X D Y. Then
L.X; XI R1 / D L.XI X/; under the identification X D X ; (5.144)
i.e., bilinear map XX 7! R1 can be regarded as a continuous linear transformation

on X.
2. Hilbert–Schmidt operators, nuclear operators

Let ei ; i D 1; 2; : : : beP
an ONB of H. 2 L.HI H/ is called a Hilbert–Schmidt
operator, if k k2L2 .H/ WD i k ei k2H < 1. The set of Hilbert–Schmidt operators,
denoted by L2 .H/, becomes a Hilbert space with the norm k kL2 .H/ . The norm does
not depend on the choice of ONB.
˚ 2 L.HI H/ is called a nuclear operator, or trace class, if there exist N and
j ; Q j 2 L2 .H/; j D 1; : : : ; N , such that
X
N
˚D
j
Qj .
j D adjoint of i /: (5.145)
j D1
˚ is called nonnegative, if .˚h; h/ 0, 8h 2 H. The trace of ˚, tr.˚/, is given by
X X
N
tr.˚/ WD .˚ei ; ei / D . Qj ; j /L2 .H/ : (5.146)
i j D1
For a nuclear operator ˚, the following properties are valid:

(i) tr.˚/ does not depend on the choice of ONB.
(ii) ˚ is a compact operator.
(iii) There exists an ONB consisting of eigenvectors for ˚.
(iv) For A 2 L.HI H/, A˚ and ˚A are also nuclear and
tr.A˚/ D tr.˚A/: (5.147)
(v) Suppose that An 2 L.HI H/; n D 1; 2; : : : converge weakly to A 2 L.HI H/.

Then
lim tr.An ˚/ D tr.A˚/ (5.148)

n!1
(refer to [La83], VII).

3. First- and second derivatives
F W H 7! Z is said to be differentiable at x 2 H if there exists an operator

DF .x/ 2 L.HI Z/, such that
kF .x C h/ F .x/ DF .x/hkZ
lim D 0: (5.149)
khkH !0 khkH
Formally,
F .x C h/ F .x/ D DF .x/h C o.h/:

DF .x/ of (5.149) is unique and is called the first derivative of F at x. When F

is differentiable at any point and the map x 7! DF .x/ is continuous, we say that
F 2 C 1 .HI Z/.
Mean Value Theorem ([La83], p. 107)
Let F 2 C 1 .H/ WD C 1 .HI R1 /. Under the identification H D H,

Z 1
F .x C h/ F .x/ D .DF .x C th/; h/H dt:
0
Suppose that F 2 C 1 .HI Z/ and DF is differentiable at x. Then we can define

D F .x/ 2 L.HI L.HI Z// .D L.H; HI Z// in the same way as (5.149), namely
2
kDF .x C h/ DF .x/ D 2 F .x/hkL.HIZ/

lim D0 (5.150)
khkH !0 khkH
(The numerator D supkgkH D1 kDF .x C h/g DF .x/g D 2 F .x/.h; g/kZ ). When

D 2 F is continuous, we write F 2 C 2 .HI Z/. For F 2 C 2 .HI Z/, D 2 F .x/ is
symmetric:
Q D D 2 F .x/.h;
D 2 F .x/.h; h/ Q h/; 8h; hQ 2 H
(see [La83], p. 110).
Taylor Formula
Let F 2 C 2 .H/ WD C 2 .HI R1 /. Then

Z 1
F .x C h/ F .x/ D .DF .x/; h/H C .1 t/D 2 F .x C th/.h; h/ dt (5.151)
0
(see [La83], V for details).
5.4.2 Itô’s Formula for H-Valued Semimartingales
We denote by C 12 .Œ0; T H/ (or Cu12 .Œ0; T H/) the set of F 2 C.Œ0; T H/ such
that @t F , DF , and D 2 F are continuous (or uniformly continuous) on Œ0; T H.
For F 2 C 12 .Œ0; T H/, we assume the local boundedness condition;
.F0 / jF ./j, j@t F ./j, kDF ./kH and kD 2 F ./kL.HIH/ are bounded on any bounded
subset of Œ0; T H.
Theorem 5.6. Assume .F0 /. For X given by (5.141), Itô’s formula (5.142) holds.
Proof. Put
˝Q D f! 2 ˝I X.; !/ 2 C.Œ0; T I H/g (5.152)
Q
and, for ! 2 ˝,
H! D closed convex hull of fX.t; !/I t 2 Œ0; T g: (5.153)
Then P .˝/Q D 1 and H! is compact in H.

For the proof, we use a time discretization procedure, as in the finite-dimensional
cases. So we only sketch the proof, assuming that D T and the following
boundedness is satisfied:
jF .t; /j C j@t F .t; /j C kDF .t; /kH C kD 2 F .t; /kL.H;H/ K;
8.t; / 2 Œ0; T H (5.154)
and kX0 kH K, P -a.s. During the proof, we omit the suffix H in k kH and . ; /H ,
for simplicity.
From Taylor’s formula (5.151) it follows that
F .t; X.t// F .; X.//

D F .t; X.t// F .; X.t// C F .; X.t// F .; X.//
Z t
D @s F .s; X.t// ds C .DF .; X.//; 4.; t//

Z 1
C .1 /D 2 F .; X./ C 4.; t//.4.; t/; 4.; t// d; (5.155)
0
where 4.; t/ D X.t/ X./.

We divide the proof into four steps.
Step 1. Let DN D .tj ; j D 1; : : : ; p/ be a division of Œ0; T , satisfying
DN DN C1 ; jDN j < 2N ; N D 1; 2; : : : :
Put
N .s/ D tj ; NN .s/ D tj C1 on s 2 Œtj ; tj C1 /; j D 0; 1; : : : ; p: (5.156)
From the continuity of @s F and the bounded convergence theorem, it is clear that
Z T
lim j@s F .s; X.NN .s/// @s F .s; X.s//j ds D 0 (5.157)
N !1 0
in L1 .˝/ and P -a.s. Similarly, observing that

Z t Z t
DF .; X.//; .s/ ds D .DF .; X.//; .s// ds; (5.158)

we have
Z T
lim E j.DF .N .s/; X.N .s///DF .s; X.s//; .s//j ds D 0: (5.159)
N !1 0
For the stochastic integral term, the Burkholder–Davis–Gundy inequality yields

h ˇZ t ˇi
ˇ ˇ
lim E sup ˇ .DF .N .s/; X.N .s/// DF .s; X.s//; dM.s//ˇ D 0:
N !1 0t T 0
(5.160)
Step 2. We divide the 3rd term in the RHS of (5.155) into four parts, J1 ; J2 ; J3 ,
and J4 . Let ! 2 ˝Q and " > 0 be given. Put
J1 .; t; ; !/ D D 2 F .; X.; !/ C .X.t; !/ X.; !/// D 2 F .; X.; !//:
Since H! is compact and D 2 F is continuous, there is ı";! > 0, such that
sup kJ1 .; t; ; !/kL.HIH/ < " if jt j < ı";! : (5.161)

2Œ0;1
Step 3. Put
Z t
4M .; t/ D M.t/ M./ and I.; t/ D .s/ ds:

We will compute the following terms:
J2 .; t/ D D 2 F .; X.//.I.; t/; I.; t//;

J3 .; t/ D D 2 F .; X.//.I.; t/; 4M .; t//;
J4 .; t/ D D 2 F .; X.//.4M .; t/; 4M .; t//: (5.162)
From the Schwarz inequality and (5.17), it follows that

ˇ X ˇ Z t 12
ˇ ˇ N 1
Eˇ J3 .tj ; tj C1 /ˇ K2 2 E k.s/k2 ds .EhM i.t// 2 (5.163)
tj C1 t 0
and
ˇ X ˇ Z t
ˇ ˇ N
Eˇ J2 .tj ; tj C1 /ˇ K2 E k.s/k2 ds: (5.164)
tj C1 t 0
Step 4. For J4 , we have that

ˇX ˇ
ˇ ˇ
Eˇ D 2 F .tj ; X.tj //.˘n? 4M .tj ; tj C1 /; 4M .tj ; tj C1 //ˇ
tj C1t
1 1
K.Eh˘n? M i.t// 2 .EhM i.t// 2 ! 0;
as n ! 1; uniformly w.r.t. the divisions DN : (5.165)
On the other hand, (5.22) and (5.24) yield

X
D 2 F .tj ; X.tj //.˘n 4M .tj ; tj C1 /; ˘n 4M .tj ; tj C1 //
tj C1 t
X
n X
D D 2 F .tj ; X.tj //.ek ; ep /.4M .tj ; tj C1 /; ek /.4M .tj ; tj C1 /; ep /
k;pD1 tj C1 t
n Z
X t
! D 2 F .s; X.s//.ek ; ep /q k;p .s/ d hM i.s/; P -a.s. as N ! 1:
k;pD1 0
(5.166)
Now (5.165), (5.166), and (5.162) imply that
X Z t
lim J4 .tj ; tj C1 / D tr.D 2 F .s; X.s//qM .s// d hM i.s/ P -a.s.
N !1 0
tj C1 t
(5.167)
Putting (5.157), (5.159)–(5.161), (5.163), (5.164), and (5.167) together, we
complete the proof.
Example 5.7 (Energy equality). For F ./ D kk2H , Itô’s formula gives the energy
equality;
d kX.t/k2H D 2.X.t/; .t//H dt C 2.X.t/; dM.t//H C d hM i.t/: (5.168)
Example 5.8 (Stochastic integral driven by WQ on H0 ).

Suppose that
Z t
M.t/ D .s/ d WQ .s/;
0
with
Xp
WQ .s/ D k ˇ k .s/ek
k
and 2 L2 .Œ0; T ˝; .Ft /I L2Q .H0 I H0 //. Putting k .t/ D .t/ek ;

.2 L2 .Œ0; T ˝; .Ft /I H0 //, we have
n
dF .t; X.t// D @t F .t; X.t// C .DF .t; X.t//; .t//
1X o
C k D 2 F .t; X.t//. k .t/; k .t// dt
2
k
Xp
C k .DF .t; X.t//; k .t// dˇ
k
.t/: (5.169)
k
5.4.3 Itô’s Formula for Linear Stochastic Parabolic Equations
In this subsection, we always assume the conditions (B1 ), .B2 /, .B4 /, and (B5 ) are
satisfied. Since the solution u./ of (5.102) with the initial state u0 2 L2 .˝; FT0 I H1 /
belongs to L2 .ŒT0 ; T ˝; .Ft /I H2 /\L2 .˝I C.ŒT0 ; T I H1 //, Itô’s formula is valid
for F satisfying condition .F0 /, by using
Z t
.s/ WD A.s/u.s/ C z.s/ and M.t/ WD .G.s/u.s/ C .s// d WQ .s/:
T0
However, if u0 is an H0 -random variable, then may not be an H0 -process and

the previous arguments do not work. So, we impose the following stronger condition
.F1 /, besides .F0 /.
.F1 / DF .t; / maps H1 into H1 , and there exists a modulus mDF such that
8
ˆ
ˆ jjjDF .t1 ; 1 / DF .t2 ; 2 /jjj mDF .jt1 t2 j C jjj1 2 jjj/
ˆ
< for 1 ; 2 2 H1 ;
(5.170)
ˆ
ˆ kDF .t1 ; 1 / DF .t2 ; 2 /k mDF .jt1 t2 j C k1 2 k/
:̂
for 1 ; 2 2 H0 :
Remark 5.2. The following facts are obvious: under .F0 / and .F1 /,
(i) There is a constant KDF , such that
(
jjjDF .t; /jjj KDF .1 C jjjjjj/; 8.t; / 2 Œ0; T H1 ;
(5.171)
kDF .t; /k KDF .1 C kk/; 8.t; / 2 Œ0; T H0 :
(ii) Let and n ; n D 1; 2; : : : be in L2 .Œ0; T ˝I H1 /. If

Z T
lim E jjjn .t/ .t/jjj2 dt D 0;
n!1 0
then
Z T
lim E jjjDF .t; n .t// DF .t; .t//jjj2 dt D 0:
n!1 0
Example 5.9. By F 2 Cu12 .Œ0; T H1 / we mean that there is FQ 2 Cu12

.Œ0; T H0 /, such that
F .t; / D FQ .t; B0 /; 2 H1 ;

1
where B0 D .1 / 2 with the d -dimensional Laplacian . The restriction of
F .2 Cu12 .Œ0; T H1 // to H0 satisfies .F0 / and .F1 /.
Indeed, DF .t; / D B0 D FO .t; B0 / is in H1 . Since FQ ; D FQ , and D 2 FQ are
Q
uniformly continuous, there is a modulus m./, such that
jFQ .t1 ; 1 / FQ .t2 ; 2 /j C kD FQ .t1 ; 1 / D FQ .t2 ; 2 /k

C kD 2 FQ .t1 ; 1 / D 2 FQ .t2 ; 2 /kL.H0 IH0 / m.jt
Q 1 t2 j C k1 2 k/:
Hence, it holds that
jjjDF .t1 ; 1 / DF .t2 ; 2 /jjj D kD FQ .t1 ; B0 1 / D FQ .t2 ; B0 2 /k

Q 1 t2 j C k1 2 k /:
m.jt
Since D 2 F .t; / D B0 D 2 FQ .t; B0 /B0 , we conclude that (F0 ) and .F1 / hold.
Theorem 5.7. Assume that conditions .B1 /; .B2 /; .B4 / and .B5 / are satisfied.
Suppose that F 2 C 12 .ŒT0 ; T H0 / satisfies .F0 / and .F1 /. Then Itô’s formula
holds for the solution u./ of problem (5.102)–(5.93).
dF .t; u.t//
n
D @t F .t; u.t// C hA.t/u.t/ C z.t/; DF .t; u.t//i
1 1 1
o
C tr ..G.t/u.t/ C .t//Q 2 / D 2 F .t; u.t//.G.t/u.t/ C .t//Q 2 / dt
2
C ..G.t/u.t/ C .t// d WQ .t/; DF .t; u.t/// (5.172)
Proof. By taking the exit time of u./ from a ball of H0 , we may assume that the
boundedness condition (5.154) holds for H D H0 . We divide the proof into two
cases.
Case 1. ku0 k C0 (D constant), P -a.s.
Step 1. We approximate u0 by un0 2 L2 .˝; FT0 I H1 / so that
Ekun0 u0 k2 < 2n and kun0 k C0 P -a.s. (5.173)
By un ./, we denote the solution of (5.102) with the initial state un0 . Then un 2
L2 .ŒT0 ; T ˝; .Ft /I H2 / \ L2 .˝I C.ŒT0 ; T I H1 // and
h Z T i
E sup kun .t/ u.t/k C 2
jjjun .t/ u.t/jjj2 dt
t T0
NEkun0 u0 k ; 2
n D 1; 2; : : : (5.174)
with a constant N independent of n. Further, we may assume that

Z T
lim sup kun .t/ u.t/k C
2
jjjun .t/ u.t/jjj2 dt D 0 P -a.s. (5.175)
n!1 t T0
and
lim jjjun .t/ u.t/jjj2 D 0 a.e. on ŒT0 ; T ˝ (5.176)

n!1
by taking a subsequence n0 , if necessary.

Since Itô’s formula (5.172) is valid for un ./, we intend to show the conver-
gence of each term in the RHS as n ! 1.
From (5.175) it is immediate that
lim sup jF .t; un .t// F .t; u.t//j D 0 (5.177)

n!1 t
and
Z T
lim j@s F .s; un .s// @s F .s; u.s//j ds D 0; P -a.s. and in L1 .˝/:
n!1 T
0
(5.178)
Step 2. For DF terms, we have
Z T
E jhA.s/.un .s/ u.s//; DF .s; un .s//ij ds
T0
Z T
KKDF E jjjun .s/ u.s/jjj.1 C jjjun .s/jjj/ ds
T0
Z T 12 Z T 12
2
KKDF E jjjun .s/ u.s/jjj ds E .1 C jjjun .s/jjj/2 ds
T0 T0
! 0 as n ! 1; (5.179)
thanks to (5.174) and (5.103). Similarly, for any " > 0, we have
Z T
E jhA.s/u.s/; DF .s; un .s// DF .s; u.s//ij ds
T0
Z T
KE jjju.s/jjjmDF .jjjun .s/ u.s/jjj/ ds
T0
Z T
KE jjju.s/jjj." C C" jjjun .s/ u.s/jjj/ ds; (5.180)
T0
with some constant C" depending on mDF ./.

Hence (5.179) and (5.180) together with (5.175) yield
h ˇZ t
ˇ
E supˇ hA.s/un .s/ C z.s/; DF .s; un .s//i ds
t T0
Z t ˇi
ˇ
hA.s/u.s/ C z.s/; DF .s; u.s//i ds ˇ ! 0 as n ! 1: (5.181)
T0
Step 3. For the stochastic integral terms, (5.154) and the dominated convergence
theorem yield
hZ T
E .Gk .s/.un .s/ u.s//; DF .s; un .s///2 ds
T0
Z T i
C .Gk .s/u.s/; DF .s; un .s// DF .s; u.s///2 ds ! 0 as n ! 1;
T0
(5.182)
whence, by the Burkholder–Davis–Gundy inequality,

h ˇZ t
ˇ
E supˇ ..G.s/un .s/ C .s// d WQ .s/; DF .s; un .s///
t T0
Z t ˇ2 i
ˇ
..G.s/u.s/ C .s// d WQ .s/; DF .s; u.s///ˇ ! 0 as n ! 1:
T0
(5.183)
Step 4. For the second differential terms, we consider
D 2 F .s; un .s//.Gk .s/un .s/; Gk .s/un .s//

D 2 F .s; u.s//.Gk .s/u.s/; Gk .s/u.s//
D D 2 F .s; un .s//.Gk .s/.un .s/ u.s//; Gk .s/.un .s/ C u.s///
C .D 2 F .s; un .s// D 2 F .s; u.s///.Gk .s/u.s/; Gk .s/u.s//
WD Ik n .s/ C Jk n .s/: (5.184)
Using the same arguments, we have
X hZ T i
k E jIk n .s/j C jJk n .s/j ds ! 0 as n ! 1; (5.185)
k T0
because by (5.154) D 2 F 2 Cb .ŒT0 ; T H0 I L.H0 I H0 //.

Thus, putting the above results together, we obtain Itô’s formula for u./, since
Itô’s formula holds for un ./.
Case 2. Eku0 k2 < 1.
Let Cn ; n D 1; 2; : : : be an increasing sequence of positive numbers with
limn!1 Cn D 1. Put un0 D Œ0;Cn .ku0 k/u0 and let un ./ and u./ denote the
solutions of (5.100) with the initial states un0 and u0 respectively. Referring
to (5.174)–(5.176), we may assume that, as n ! 1
Z T
sup kun .t/ u.t/k C 2
jjjun .t/ u.t/jjj2 dt ! 0 P -a.s. and in L1 .˝/:
T0 t T T0
(5.186)
Since Itô’s formula (5.172) holds for un ./ and we can prove the convergence of
each term of (5.172) for un ./ as n ! 1, by using same arguments as in Case 1,
we obtain Itô’s formula for u./.
Example 5.10 (Estimation of the H1 -norm).
We will evaluate Eku.t/k2 , by using Itô’s formula. Let u./ be the solution
of (5.102) with the initial state u0 2 L2 .˝; FT0 I H0 /. Put F ./ D kk2 D
.; .1 /1 /. Since F satisfies .F0 / and .F1 /, Itô’s formula gives the dynamics
of ku.t/k2 :
d ku.t/k2 D2hA.t/u.t/ C z.t/; .1 /1 u.t/i dt

1
C k.1 / 2 .G.t/u.t/ C .t//k2Q dt
C 2..G.t/u.t/ C .t// d WQ .t/; .1 /1 u.t//: (5.187)
From (5.187), we deduce

Proposition 5.7. Under .B1 /, .B2 /, .B4 / and .B5 /, one has
(i) Estimate
h Z T i
E sup ku.s/k2 C ku.s/k2 ds
T0 sT T0
h Z T i
NE ku0 k C
2
.kz.s/k2H2 C k .s/k2Q / ds ; (5.188)
T0
(ii) Let u./ and uQ ./ be the solutions of (5.100) with initial states u0 and uQ 0
respectively. Then
h Z T i
E sup ku.s/Qu.s/k2 C ku.s/Qu.s/k2 ds NEku0 Qu0 k2 (5.189)
T0 sT T0
N N and K1 of (5.104).
where the constant N depends only on ;
1
Proof. Put A0 D 1 and B0 D .1 / 2 for simplicity. Referring to [GŚ00],
Lemma 3.3, we have for 2 H3
2hA.s/A0 ; i C kB0 G.s/A0 k2Q I I2 C jjj jjj2 ; 8s; (5.190)
with constants > 0 and 2 R1 .

First we assume u0 2 L2 .˝; FT0 I H1 /. Let u./ 2 L2 .˝I C.ŒT0 ; T I H1 // be a
unique solution of (5.100) with initial state u0 . Consequently, for v.t/ WD B02 u.t/,
2hA.s/A0 v.s/ C z.s/; v.s/i C kG.s/A0 v.s/ C .s/k2Q

2hA.s/A0 v.s/; v.s/i C kG.s/A0 v.s/k2Q
1
C 2hz.s/; v.s/i C "kG.s/A0 v.s/k2Q C 1 C k .s/k2Q (5.191)
"
for any " > 0. Since
kG.s/A0 v.s/k2Q c1 kA0 v.s/k2 D c1 Iv.s/I2 (5.192)
with a constant c1 independent of s, relations (5.187), (5.191), and (5.192) yield
jjjv.t/jjj2 ku0 k2

Z tn o
2
Iv.s/I2 C jjju.s/jjj
Q C c2 kz.s/k2H2 C k .s/k2Q ds
T0 2
C stochastic integral; (5.193)
with Q D C 12 and c2 D maxf 12 ; .1 C 2c1 / 4 g. Taking the expectation and using

Gronwall’s inequality, we evaluate Ejjjv.t/jjj2 and obtain
h Z t i
2
E jjjv.t/jjj C Iv.s/I2 ds
T0
h Z t i
c3 E ku0 k C
2
kz.s/k2H2 C k .s/k2Q ds ; (5.194)
T0
with a constant c3 .
Put
Z t
M.t/ D ..G.s/A0 v.s/ C .s// d WQ .s/; v.s//:
T0
Then M.t/ is a continuous real martingale with

Z t
hM i.t/ D .G.s/A0 v.s/ C .s/; v.s//2Q ds:
T0
Since the Burkholder–Davis–Gundy inequality yields

h i p
E sup jM.s/j 6E hM i.t/
T0 st
Z t
1
E supjjjv.s/jjj2 C 18E kG.s/A0 v.s/ C .s/k2Q ds;
2 st T0
(5.195)
(5.193)–(5.195) yield (i).

For u0 2 L2 .˝; FT0 I H0 /, we take un;0 2 L2 .˝; FT0 I H1 /; n D 1; 2; : : : such
that limn!1 EŒkun;0 u0 k2 D 0. Let un;0 ./ and u./ be solutions of (5.100) with
initial states un;0 and u0 , respectively. Then,
EŒ sup kun .t/ u.t/k2 NEŒkun;0 u0 k2

T0 t T
yields (i) for u./.

(ii) is immediate from (i). Thus, we have completed the proof.
Example 5.11 (Estimate of EŒsup st ku.s/ u./k2 when z D 0 and D 0).
Let ; t 2 ŒT0 ; T / be given. By using Proposition 5.7, we have
h i
E sup ku.s/ u./k2 c0 .t /Eku./k2 c0 N.t /Eku0 k2
st
(5.196)
with a constant c0 , independent of t and , and with N the constant in (5.103).

Indeed, the right inequality is clear by (5.103). So, let us prove the left inequality.
Take un 2 L2 .˝; F I H1 / such that
lim Eku./ un k2 D 0: (5.197)

n!1
5.5 Zakai Equations 197
Put vn .s/ D u.s/ un , s 2 Œ; T . Then vn ./ satisfies

(
d vn .s/ D .A.s/vn .s/CA.s/un / ds C.G.s/vn .s/CG.s/un / d WQ .s/; s 2 .; T ;
vn ./ D u./ un .2 L2 .˝; F I H0 //:
Further, there is a constant c1 , depending only on K1 of .B4 /, such that
kA.s/un k2H2 C kG.s/un k2Q c1 kun k2 ; 8s; P -a.s. (5.198)
Now from (5.188), it follows that

h i
E sup kvn .s/k2 c0 EŒku./ un k2 C .t /kun k2 (5.199)
st
with a constant c0 independent of t; and n. Since
u.s/ u./ vn .s/ D un u./; 8s ; (5.200)
(5.196) follows, from (5.197), (5.199), and (5.200).
5.5 Zakai Equations
This section is devoted to certain stochastic parabolic equations, called the Zakai
equations, related to filtering problems. Let X be a diffusion described by an
SDE. X cannot be measured directly. Only a partial measurement of X can be
obtained by means of another process Y affected by an observation noise. Hence,
we need to estimate X.t/, by using the information of Y up to time t. This is the
filtering problem (cf. [BC09]). Hence the problem is how to compute the conditional
probability of X.t/, given FtY , by using results obtained in Sects. 5.3 and 5.4.
In Sect. 5.5.1 we are concerned with partially observable controlled diffusions
and in Sect. 5.5.2 we study an explicit solution of the Zakai equation for conditional
Gaussion process.
5.5.1 Partially Observable Controlled Diffusion
Let X be a controlled diffusion, governed by the SDE
dX.t/ D ˛.t; X.t/; .t// dB.t/ C .t; X.t/; .t// d W .t/ C b.t; X.t/; .t// dt;
where B and W are mutually independent Wiener processes and ./ is a control
process. As mentioned above we suppose that X cannot be measured directly. Only
a partial measurement of X can be obtained by means of another process Y , given
by
Z t
Y .t/ D h.s; X.s/; .s// ds C W .t/:
0
Y is called the observation process and W is called the observation noise.

Accordingly, FtY stands for the information field up to time t. In this model we
assume that the control process ./ is .FtY /-progressively measurable.
The problem is how to determine the conditional probability of X , given the
information of Y up to t.
Let us formulate the problem precisely. Let B and W be d0 - and m-dimensional
.Ft /-Wiener processes defined on .˝; F ; .Ft /; P /. Suppose that the control region
is convex and compact subset of Rq . Suppose further that
˛ W Œ0; T Rd 7! Rd ˝ Rd0 ;
W Œ0; T Rd 7! Rd ˝ Rm ;
and
h W Œ0; T Rd 7! Rm
are continuous and satisfy the following conditions:

.d1 / z D ˛, and h are Lipschitz continuous w.r.t. x uniformly on Œ0; T , say
jz.t; x; / z.t; x;
Q /j ljx xj;
Q 8t; (5.201)
and
jz.t; x; /j K.1 C jxj/; 8t; x; I (5.202)
.d2 / b satisfies (5.202) and is locally Lipschitz continuous w.r.t. x uniformly on

Œ0; T , say, for any R > 0,
jb.t; x; / b.t; x;
Q /j lR jx xj
Q for jxj; jxj
Q R; 8t; I
.d3 / ˛ and are bounded.

Definition 5.10. W Œ0; T C.Œ0; T I Rm / 7! is called a policy, if it is non-
anticipative, namely =Œ0;t C.Œ0;T IRm / is B.Œ0; t/ B.C Œ0; tI Rm /-measurable for
any t 2 Œ0; T .
For a given policy , we consider the following SDEs:
System Equation
8
ˆ
ˆ dX.t/ D ˛.t; X.t/; .t; Y // dB.t/ C .t; X.t/; .t; Y // d W .t/
<
C b.t; X.t/; .t; Y // dt; t 2 .0; T ; (5.203)
ˆ
:̂
X.0/ D X0 .2 L2 .˝; F0 I Rd //;
and
Observation Equation
(
d Y .t/ D h.t; X.t/; .t; Y // dt C d W .t/; t 2 .0; T ;
(5.204)
Y .0/ D 0:
Now we seek a solution .X; Y / and then compute P .X.t/ 2 jFtY /. So the next
step is to introduce the dynamics of the conditional probability density process,
according to [R90], V.
Let ˇ and ˇQ be d0 - and m-dimensional Wiener processes, defined on .˝; F ; P /,
and 0 be a random variable with the same probability distribution as X0 . We assume
Q and 0 are mutually independent. Set
that ˇ; ˇ,
;ˇ; Q̌
Ft D Ft 0 D right continuous and complete -field
Q
generated by f0 ; ˇ.s/; ˇ.s/; s tg:
Hence ˇ and ˇQ are mutually independent (Ft )-Wiener processes and 0 is in

L2 .˝; F0 I Rd /.
Let us consider the auxiliary SDEs
8
Q
ˆ d .t/ D ˛.t; .t/; .t; // dˇ.t/ C .t; .t/; .t; // d ˇ.t/
ˆ
<
Q .t/; .t; // dt; t 2 .0; T ;
C b.t; (5.205)
ˆ
:̂
.0/ D 0 ;
where
Q x; / D b.t; x; / .t; x; /h.t; x; /;

b.t;
and
(
Q
d.t/ D d ˇ.t/; t 2 .0; T ;
(5.206)
.0/ D 0:
By conditions .d1 /–.d3 /, the auxiliary SDEs (5.205)–(5.206) admit a unique strong
solution .; /.
Applying Girsanov’s transformation, we obtain
Proposition 5.8. The SDE (5.203)–(5.204) has a weak solution. Moreover, the
probability distribution of a weak solution is unique.
Proof. Put
Z t Z
1 t
M.t/ D exp Q
h.s; .s/; .s; // d ˇ jh.s; .s/; .s; //j2 ds :
0 2 0
(5.207)
Then by Proposition 1.5 M is an exponentialRmartingale and PQ WD M.T / ı P gives
a probability on FT . It is easy that PQ . / D M.t/ dP for 2 Ft . Put
Z t
Q
WQ .t/ D ˇ.t/ Q ds;
h.s; .s/; .s; ˇ// t 2 Œ0; T : (5.208)
0
Then Girsanov’s theorem shows that ˇ and WQ are mutually independent .Ft /-
Wiener processes under PQ . Since .; / satisfies (5.203)–(5.204) with ˇ and WQ
on .˝; F ; .Ft /; PQ /; .; / gives a weak solution of (5.203)–(5.204). Moreover, the
distribution of the weak solution is unique, because (5.205)–(5.206) has a unique
strong solution.
Q ..t//jFt /, where EQ D the expectation w.r.t. PQ . Since
Next we study E.F

E.M.t/jFt / 2 .0; 1/ P -a.s., we can use Bayes formula (Proposition 1.3)

Q ..t//jFt / D E.M.t/F ..t//jF
E.F
t /
; P -a.s.; (5.209)
E.M.t/jFt /
which is called the Kallianpur–Stiebel formula. We compute the RHS of (5.209).

Since .t/ is .0 ; ˇ.s/; .s/; s t/-measurable and ˇ, , and 0 are mutually

independent, we can compute E.M.t/F ..t//jFt /, by taking the expectation w.r.t.
.0 ; ˇ/, freezing as if it were deterministic.

Definition 5.11. E.M.t/F ..t//jFt / is called the unnormalized conditional

expectation of F ..t// given Ft . An .Ft /-progressively measurable H0 -process qQ
is called a density-valued process, if
Q x/ 0;
q.t; 8.t; x/; P -a.s. (5.210)
and for any F 2 CK .Rd /,

Z

E.M.t/F ..t///jFt / D Q x/ dx;
F .x/q.t; 8t; P -a.s. (5.211)
Rd
Proposition 5.9.
(i) Besides .d1 /–.d3 /, we assume that ˛ and are in C 1 .Rd / w.r.t. x. Suppose
that 0 has the probability density p0 2 H0 . If the density-valued process qQ is in
L2 .Œ0; T I H1 / \ C.Œ0; T I H0 /, P -a.s., then qQ is a generalized solution of the
following stochastic linear parabolic equation (5.212)–(5.213).
(ii) If ˛˛ > is uniformly positive definite, then the generalized solution is unique.
X
m
Q D L.t;/ .t/ q.t/ .t/ q.t/
.t;/
d q.t/ Q dt C lk Q dk .t/; t 2 .0; T ;
kD1
(5.212)
Q
q.0/ D p0 ; (5.213)
where
1
L .t/.x/ D tr..˛˛ > C > /.t; x; /@xx .x// C b.t; x; / @x .x/;
2
(5.214)

X
d
lk .t/.x/ D ki .t; x; /@i .x/ C hk .t; x; /.x/: (5.215)
i D1
Proof. (i) Let f 2 Cb .Rd / and F 2 CK1 .Rd /. Since

E.M.t/f ..t/; .t; //jFT / D E.M.t/f ..t/; .t; //jFt /
D .f .; .t; //; q.t//;
Q 8t; P -a.s.;
Itô’s formula for M.t/F ..t// gives

Z t

Q
.F; q.t// .F; p0 / D E M.s/L.s;/ .s/F ..s//jFT ds
0
m Z
X t
.s;/
C E M.s/lk .s/F ..s//jFT dk .s/
kD1 0
Z t m Z t
X
.s;/
D L.s;/
Q
.s/F; q.s/ ds C Q
lk .s/F; q.s/ dk .s/:
0 kD1 0
(5.216)
This yields (i).

Since the superparabolicity condition ensures the uniqueness of the generalized

solution [R90], Ch. 4, Th. 4.1.1, we obtain (ii).
Regarding the relation between qQ and (5.212), one can find details in [R90],
Ch. 5.5.3.
Example 5.12 (Filtering of factor process). We consider a factor market model
(refer to Sect. 2.4). Let X be a d -dimensional factor process determining the
performance of market. Suppose that X evolves according to the SDE;
dX.t/ D b.X.t// dt C ˛.X.t// dB.t/; t > 0;
and X.0/ has probability density p0 2 H0 . The price of the k-th asset is given by
(
dS k .t/ D S k .t/.hk .X.t// dt C d W k .t//; t > 0;
(5.217)
S k .0/ D s0k > 0; k D 1; : : : ; m;
where B and W D .W 1 ; : : : ; W m / are mutually independent d - and m-dimensional

Wiener processes.
We assume that ˛ji ; b i 2 Cb3 .Rd /; hk 2 Cb2 .Rd /, and a.x/ D ˛.x/˛.x/> is
uniformly positive definite.
.S 1 ;:::;S m /
We seek the unnormalized conditional probability of X.t/ given Ft .
From (5.217) we deduce that
Y k .t/ WD log S k .t/

Z t
1
D log s0 C
k
hk .X.s// ds C W k .t/
0 2
.S 1 ;:::;S m / .Y 1 ;:::;Y m /
and Ft D Ft . Hence Proposition 5.9 yields the following stochastic
linear parabolic equation for its density-valued process q.t/:
8
ˆ X d X d X d
ˆ
ˆ D ij
i
ij
ˆ
ˆ dq.t/ @i a .x/@ j q.t/ b .x/ @j a .x/ q.t/ dt
ˆ
< i D1 j D1 j D1
Xm
ˆ 1
ˆ
ˆ C h .x/
k k
q.t/ d W .t/; t > 0;
ˆ
ˆ 2
:̂ kD1
q.0/ D p0 :
Hence (5.209) and (5.211) lead to

R
.S 1 ;:::;S m / RdR F .x/q.t; x/dx
E.F .X.t//jFt /D :
Rd q.t; x/dx
Now we will give the definition of the (controlled) Zakai equation.

Let W be an m-dimensional Wiener process and .t/ be a W -adapted control
process. The following stochastic linear parabolic equation is called the (controlled)
Zakai equation:
X
m
.t /
d u.t; x/ D A.t / .t/u.t; x/ dt C Gk .t/u.t; x/ d W k .t/; (5.218)
kD1
where
X
d X
d
A .t/.x/ D @i aij .t; x; /@j .x/ C b i .t; x; /.x/ C .t; x; /.x/;
i D1 j D1

X
d
j
Gk .t/.x/ D gk .t; x; /@j .x/ C „k .t; x; /.x/:
j D1
Hence (5.212) is the Zakai equation, because .t/ is a Wiener process on .˝; F ; P /.

Let us return to (5.209). For the normalization factor, N.t/ WD E.M.t/jFt /,
observing that
X
m

dN.t/ D E M.t/hk .t; .t/; .t; //jFt dk .t/;
kD1
we can easily deduce

Remark 5.3. N./ is an .Ft /-exponential martingale under P , satisfying
Z t Z t
N.t/ D exp Q d.s/ 1
h.s/ Q
jh.s/j 2
ds ; (5.219)
0 2 0
where
Q D E.h.t;
h.t/ Q
.t/; .t; //jFt /:
Q
because (5.209) yields dN.t/ D N.t/h.t/d.t/.
5.5.2 Zakai Equation for a Conditional Gaussian Process
Let us consider the following system-observation SDEs:

d-Dimensional System Equation
8
ˆ
ˆ dX.t/ D .r0 .t; Y / C r1 .t; Y /X.t// dt
<
C ˛.t; Y / dB C .t; Y / d W; t 2 .0; T ; (5.220)
ˆ
:̂
X.0/ D X0 .N.0; v0 /-distributed random variable/:
m-Dimensional Observation Equation
(
d Y .t/ D h.t; Y /X.t/ dt C d W .t/; t 2 .0; T ;
(5.221)
Y .0/ D 0:
where is a positive constant.

Starting from (5.212)–(5.213), formal computations yield the Zakai equation
8
ˆ Xm
ˆ
< dq.t/ D LY .t/ q.t/ dt C lkY .t/ q.t/ d Y k .t/; t 2 .0; T ;

kD1
(5.222)
ˆ
:̂ q.0/ D ..2 /d det v0 / 12 exp 1 x > v1 x ;
0
2
where
8
ˆ 1 > >
ˆ
< L .t/.x/ D 2 trf.˛˛ .t; Y / C .t; Y //@xx .x/g
Y
ˆ
ˆ C .r0 .t; Y / C r1 .t; Y /x/ @x .x/;

ˆ
:̂ Y
lk .t/.x/ D 1 fk .t; Y / @x .x/ C hk .t; Y /x.x/g; k D 1; : : : ; m:
(5.223)
Here we will give an explicit formula for density valued processes (see (5.235))
and show that it gives the unique generalized solution of (5.222).
Let B and W be mutually independent d0 - and m-dimensional Wiener processes.
Suppose that
˛ W Œ0; T C.Œ0; T I Rm / 7! Rd ˝ Rd0 ;

W Œ0; T C.Œ0; T I Rm / 7! Rd ˝ Rm ;
r0 W Œ0; T C.Œ0; T I Rm / 7! Rd ;
r1 W Œ0; T C.Œ0; T I Rm / 7! Rd ˝ Rd ;
h W Œ0; T C.Œ0; T I Rm / 7! Rm ˝ Rd
are bounded, continuous, and non-anticipative. We assume the superparabolicity

condition: ˛˛ > C > .1 2 / is uniformly positive definite.
Referring to [LS01], Ch. 11, 12, 13, we study equations (5.220)–(5.221).
1. Weak solution
Let ˇ and ˇQ be d0 - and m-dimensional Wiener processes and 0 be a Gaussian

random variable with N.0; v0 /-distribution, defined on .˝; F ; P /. We assume that
Q and 0 are mutually independent. Put
ˇ, ˇ,
ˇ Q̌
Ft D Ft 0 :
We consider the following auxiliary SDEs

8
ˆ
ˆ d .t/ D fr0 .t; / C .r1 .t; / 1 .t; /h.t; //.t/g dt
<
Q
C ˛.t; / dˇ.t/ C .t; / d ˇ.t/; t 2 .0; T ; (5.224)
ˆ
:̂
.0/ D 0
and
(
Q
d.t/ D d ˇ.t/; t 2 .0; T ;
(5.225)
.0/ D 0:
Since the system (5.224)–(5.225) has the unique strong solution .; / and .0 ; ˇ; /
are mutually independent, (5.224) shows that the conditional probability of

..ti /; i D 1; : : : ; n/ given Ft , is Gaussian for any t and 0 t1 < t2 < < tn t.
Note. This property is called the conditional Gaussian property.
Now we define a new probability PQ by
PQ D M.T / ı P on FT ; (5.226)
where M./ is the exponential martingale given by

Z t 2 Z t
M.t/ D exp 1 Q
h.s; /.s/ d ˇ.s/ jh.s; /.s/j2 ds : (5.227)
0 2 0
From Girsanov’s theorem it follows that

Z t
Q Q
W .t/ D ˇ.t/ Q 1
h.s; /.s/ ds; t 2 Œ0; T ; (5.228)
0
is an m-dimensional Wiener process independent of .ˇ; 0 / under PQ . Thus .; /

satisfies (5.220)–(5.221) with ˇ and WQ on .˝; F ; .Ft /; PQ /.
Hence (5.220)–(5.221) has one and only one weak solution.
Q
2. Conditional probability P..t/jF

t /
According to [LS01], Th. 12.6, the conditional Gaussian property is also valid
under PQ .
We see that PQ ..t/ 2 jFt / is Gaussian distribution with mean m.t/ D

Q
E..t/jF
Q >
t / and variance V .t/ D E...t/ m.t//..t/ m.t// jFt /. Since .; /
satisfies (5.220)–(5.221) with ˇ and WQ on .˝; F ; .Ft /; PQ /, filtering theory asserts
that
n Z t o
WN .t/ WD 1 .t/ Q
h.s; /E..s/jF
s / ds (5.229)
0

is an .Ft /-Wiener process and .V .t/; m.t// formally satisfies the following equa-
tions:
Riccati Equation
8
ˆ
ˆ dV
ˆ
ˆ .t/ D r1 .t; /V .t/ C V .t/r1 .t; />
ˆ
ˆ dt
ˆ
ˆ
ˆ
ˆ C .˛.t; /˛.t; /> C .t; /.t; /> /
ˆ
ˆ
<
ˆ
ˆ 2 ..t; / C V .t/h.t; /> /..t; / C V .t/h.t; /> /> ;
ˆ
ˆ
ˆ
ˆ
ˆ
ˆ t 2 .0; T ;
ˆ
ˆ
ˆ
:̂
V .0/ D v0 :
(5.230)
Filtering Equation
8
ˆ
ˆ d m.t/ D .r0 .t; / C r1 .t; /m.t// dt
<
C ..t; / C V .t/h.t; /> / d WN .t/; t 2 .0; T ; (5.231)
ˆ
:̂
m.0/ D 0:
d
When (5.230) admits a unique global solution V ./ in C.Œ0; T I SCC / for any
path of , (5.231) yields that for F 2 Cb .R /,
d
Z
Q ..t//jFt / D
E.F F .x/g.t; x/ ds; (5.232)
Rd

where g is an (Ft )-progressively measurable process given by the Gaussian density
1
1
g.t; x/ D ..2 /d det V .t// 2 exp .x m.t//V 1 .t/.x m.t//> : (5.233)
2
3. Density-valued process q./
From the Bayes formula and (5.232) it follows that

Z

F .x/q.t; x/ dx D E.M.t/F ..t//jFt /
Rd
Q ..t//jFt /E.M.t/jFt /
D E.F

P -a.s. (5.234)
Thus (5.232) and Remark 5.3 yield

Z t Z
2 t
q.t; x/ D g.t; x/ exp 1 h.s; /m.s/ d jh.s; /m.s/j2 ds ;
0 2 0
8t; P -a.s. (5.235)
Since V .t/ is positive definite and continuous in t, and g./ is in C.Œ0; T I H1 /,

P -a.s., we obtain
q./ 2 L2 .Œ0; T I H1 / \ C.Œ0; T I H0 / P -a.s. (5.236)
Now Proposition 5.9 shows that q./ is the unique generalized solution of (5.222).
Refer to [Be92], 4 and 6 for the solution of Zakai equations.
Chapter 6
Optimal Controls for Zakai Equations
Abstract This chapter is an application of previous one. In Sect. 5.5, we have

introduced the (controlled) Zakai equation, which is a stochastic linear parabolic
equation with a Brownian adapted control process. By using the results in Chap. 5,
we will study control problems for Zakai equations related to partially observable
diffusions. The control problem for partially observable diffusions turns out
to be a completely observable control problem on a Hilbert space, by using
the unnormalized conditional probability density given by the Zakai equation
(cf. Bensoussan A, Stochastic control of partially observable systems. Cambridge
University Press, Cambridge/New York, 1992; Lions, J Commun PDE 8:1101–
1134, 1983, I, II; Gozzi and Świech, J Funct Analy 172:466–510, 2000). Section 6.1
is devoted to the analysis of controlled Zakai equations. In Sect. 6.2, we formulate
control problems for a system governed by Zakai equations, in the same way as in
Chap. 2. WhenR t a control process ./ is chosen, the cost on a time internal [T0 ; t]
is given by T0 r.u./ .s/; .s// ds C F .u./ .t//, where u./ ./ is the response of
./. By taking a suitable control process, we want to minimize (or maximize)
the expectation of the cost. In Sect. 6.3 we formulate the DPP via the semigroup
constructed from the value function, whose generator is related to the HJB equation
on a Hilbert space. The viscosity solution of HJB equation is introduced following
Gozzi and Świech (J Funct Analy 172:466–510, 2000) in Sect. 6.4. Example 6.1
makes explicitly the connection between controlled Zakai equations and control of
partially observable diffusions.
6.1 Controlled Zakai Equations
We assume that the control region . Rq / is convex and compact. Let W be

an m-dimensional Wiener process, defined on (˝; F ; P ). Since we use W -adapted
control processes, we put
Ft D FtW and D W
:
Here we consider time-homogeneous controlled Zakai equations in our convenient

assumptions; .B10 / and .B20 /. By using results in Chap. 5, we investigate properties of
solutions. In Sect. 6.1.1 we recall basic results, obtained in Chap. 5, and in Sect. 6.1.2
we study properties of solutions needed in the sequel.

210 6 Optimal Controls for Zakai Equations
6.1.1 Preliminaries
For ./ 2 , we consider the time-homogeneous controlled Zakai equation
nX
d X
d
d u.t; x/ D @i aij .x; .t//@j u.t; x/ C b i .x; .t//u.t; x/
i D1 j D1
o
C .x; .t//u.t; x/ dt
m X
X d
j
C gk .x; .t//@j u.t; x/ C „k .x; .t//u.t; x/ d W k .t/:
kD1 j D1
(6.1)
We assume that the following conditions are satisfied:

.B10 /
j j
z WD aij ; bi ; @i aij ; @i b i ; ; gk ; @j gk ; and „k ;
.i; j D 1; : : : ; d; k D 1; : : : ; m/
are Lipschitz continuous, say
jz.x1 ; 1 / z.x2 ; 2 /j2 l.jx1 x2 j2 C j1 2 j2 /: (6.2)
Further, there is a constant K > 0, such that

(
kz.; /kC 3 K; 8 2 for z D aij ; b i ;
(6.3)
kz.; /kC 2 K; 8 2 for z D gki ; „k :
(B20 ) Superellipticity (stronger than (B2 ))

There is a positive constant 0 such that

y > 2a.x; / 3g.x; /g.x; /> y 0 jyj2 ; 8y 2 Rd ; 8.x; / 2 Rd :
We need (B20 ) in order to estimate Eku.t/k4 (see Proposition 6.2).

In this chapter, we always assume (B10 ) and (B20 ). Hence (B4 ) holds, i.e.,
z.t; x; !/ WD z.x; .t; !//, for z D aij ; b i ; ; gki ; „k , satisfies (5.104) with the
constant K1 independent of ./. We call the solution of (6.1) the response for ./
and denote it by u./ ./.
Using Theorem 5.5 and Proposition 5.4 we obtain
6.1 Controlled Zakai Equations 211
Theorem 6.1. Let u0 be in L2 .˝; FT0 I H0 /. Then

(i) The controlled Zakai equation (6.1) with the initial condition u.T0 / D u0
./
admits the unique solution u./ WD uT0 u0 ./ in L2 .ŒT0 ; T ˝; .Ft /I H1 / \
2
L .˝I C.ŒT0 ; T I H0 // and obeys the estimate
h Z T i
ET0 u0 sup ku./ .t/k2 C jjju./.t/jjj2 dt N0 Eku0 k2 ; (6.4)
T0 t T T0
with a constant N0 independent of ./, where the subscript .T0 ; u0 / refers to

the initial condition.
./
(ii) If u0 is in L2 .˝; FT0 I H1 /, then uT0 u0 ./ is in L2 .ŒT0 ; T ˝; .Ft /I H2 / \
L2 .˝I C.ŒT0 ; T I H1 // and satisfies
h Z T i
2
ET0 u0 sup jjju ./
.t/jjj C Iu./ .t/I2 dt N0 Ejjju0 jjj2 ; (6.5)
T0 t T T0
with a constant N0 independent of ./.

Sometimes we omit the subscript .T0 ; u0 / for simplicity. We recall the notation
1./ Q ./1T0 T WD k./ Q ./kL2 ŒT0 ;T
and omit the subscript T0 ; T , when there is no danger of confusion.
6.1.2 Basic Properties of Solutions
In this subsection, we will show that the solution depends continuously on the initial
state, the time parameter and the control process. Here we deal with Eq. (6.1) in the
Hilbert space framework, by putting
X
d X
d
A./ .t/.x/ D @i aij .x; .t//@j .x/Cb i .x; .t//.x/ C.x; .t//.x/;
i D1 jD1
./
X
d
j
Gk .t/.x/ D gk .x; .t//@j .x/ C „k .x; .t//.x/;
j D1
and

G ./ .t/ D .G1 ./.t/; : : : ; Gm
./
.t//:
(B10 ) and (B20 ) ensure that the following coercivity condition is satisfied.
There are two constants, N > 0 and N 2 R1 , such that, for any ./,
N
2hA./.t/; iC3kG ./ .t/k2m jjjjjj2
C kk
N 2
; 8t; P -a.s.; 8 2 H1
(6.6)
and
2hA./.t/; i.H0 H2 / C 3jjjG ./ .t/jjj2m

N
II2
C jjjjjj
N 2
; 8t; P -a.s.; 8 2 H2 (6.7)
1. Dependence on the initial state

Let u./ ./ and uQ ./ ./ be the responses for ./ with initial condition u0 and
uQ0 , respectively. Since (6.1) is a linear stochastic parabolic equation, u ./ is the 0
process, if the initial state u0 D 0, and Theorem 5.4 yields
Proposition 6.1.
h Z T i
E sup ku./ .t/ uQ ./ .t/k2 C jjju./ .t/ uQ ./.t/jjj2 dt N1 Eku0 uQ 0 k2
T0 t T T0
(6.8)
and
h Z T i
E sup jjju./ .t/ uQ ./ .t/jjj2 C Iu./ .t/ uQ ./.t/I2 dt N1 Ejjju0 uQ 0 jjj2
T0 t T T0
(6.9)
N ,
with a constant N1 depending only on ; N and K of (6.3).
2. Estimation of the 4th moment

Proposition 6.2.
h i
ET0 u0 sup ku./ .t/k4 N2 Eku0 k4 ; (6.10)
T0 t T
h i
ET0 u0 sup jjju./ .t/jjj4 N2 Ejjju0 jjj4 (6.11)
T0 t T
N ,
with a constant N2 depending only on ; N and K.
Proof. For simplicity of notation, we omit ./ and .T0 ; u0 /. ci ; i D 1; 2; : : : will
denote constants independent of t; ./ and u0 .
From the energy equality
d ku.t/k2 D .2hA.t/u.t/; u.t/i C kG.t/u.t/k2m / dt C 2.G.t/u.t/ d W .t/; u.t//
it follows that
d ku.t/k4 D f4ku.t/k2 hA.t/u.t/; u.t/i C 2ku.t/k2 kG.t/u.t/k2m

C 4j.G.t/u.t/; u.t//m j2 g dt C 4ku.t/k2 .G.t/u.t/; u.t//m d W .t/
N
.2ku.t/k2
jjju.t/jjj2 C 2ku.t/k
N 4
/ dt
C stochastic integral; (6.12)
by (6.6). Let be the exit time of ku.t/k from [0; n]. Then (6.12) yields
n
Z t^ n
Eku.t ^ n /k4 Eku0 k4 2E N
.ku.s/k 2
jjju.s/jjj2 C ku.s/k
N 4
/ ds:
T0
(6.13)
Therefore,
N T0 /
Eku.t ^ n /k
4
e 2.t Eku0 k4 (6.14)
and
Z n
E N 1 e 2T
ku.s/k2 jjju.s/jjj2 ds .2/ N
Eku0 k4 : (6.15)
T0
Next we compute the stochastic integral term. By the Burkholder–Davis–Gundy

inequality, we have
h ˇZ X
m ˇi
ˇ ˇ
E sup ˇ ku.s/k2 .Gk .s/u.s/; u.s// d W k .s/ˇ
<t ^ n T0 kD1
Z t^ n 12
6E ku.s/k6 kG.s/u.s/k2m ds
T0
Z t^ n 12
c1 E ku.s/k6 jjju.s/jjj2 ds
T0
h Z t^ n 12 i
c1 E sup ku.s/k2 ku.s/k2 jjju.s/jjj2 ds
T0 st ^ n T0
1 hZ t^ n i
J.t ^ n/ C c2 E ku.s/k2 jjju.s/jjj2 ds ; (6.16)
8 T0
where J./ D EŒsupT0 <s< ku.s/k4 .

Combining (6.12), (6.15), and (6.16) we obtain
Z t
1
J.t ^ n/ 2E
N ku.s ^ n /k
4
ds C J.t ^ n/ C c3 Eku0 k4
T0 2
1
J.t ^ n/ C c4 Eku0 k4 (by (6.14)). (6.17)
2
Letting n ! 1, we conclude (6.10) by the monotone convergence theorem.
Finally, we prove (6.11) in the same way. Suppose that Ejjju0 jjj4 < 1. Then u./
has continuous H1 -paths. Let n be the exit time of jjju.t/jjj from Œ0; n, and put
2
.t/ D jjju.t ^ n /jjj : (6.18)
The energy equality (cf. Theorem 5.4) gives the dynamics of .t/ and we obtain
d 2 .t/ D 2.t/.2hA.t/u.t/; u.t/i.H0 H2 / C jjjG.s/u.s/jjj2m / dt

X
m
C4 ..Gk .t/u.t/; u.t///2 dt
kD1
X
m
C 4.t/ ..Gk .t/u.t/; u.t/// d W k .t/
kD1
2.t/.2hA.t/u.t/; u.t/i.H0 H2 / C 3jjjG.s/u.s/jjj2m / dt

C stochastic integral: (6.19)

N T0 /
EŒ.t/2 e 2.t Ejjju0 jjj4 (6.20)
and
Z n
E N 1 e 2T
.s/Iu.s/I2 ds .2/ N
Ejjju0 jjj4 : (6.21)
T0
We estimate the stochastic integral term in the same way as (6.16), namely
h ˇZ X
m ˇi
ˇ ˇ
E sup ˇ .s/ ..Gk .s/u.s/; u.s/// d W k .s/ˇ c5 Ejjju0 jjj4 : (6.22)
T0 t ^ n T0 kD1
Combining (6.19), (6.21), and (6.22) we conclude (6.11).

3. Dependence on the time parameter
Proposition 6.3.
N N and K, such that, for any ; t
(i) There is a constant N3 , depending only on ;
and ./,
h i
ET0 u0 sup ku./ .s/ u./ ./k2 N3 jt jEjjju0 jjj2 ; (6.23)
st
h i
ET0 u0 sup ku./ .s/ u./ ./k2 N3 jt jEku0 k2 : (6.24)
st
(ii) Let u0 D .2 H0 /. For " > 0, there is ."; / > 0, such that for any t1 ; t2
and ./,
h i
ET0 sup ku./ .t/ u./ ./k2 < " (6.25)
;t 2Œt1 ;t2
whenever t2 t1 < ."; /.

(iii) Continuity at the initial time
h i
lim ET0 sup ku./ .s/ k2 D 0; uniformly in ./; (6.26)
#T0 T0 <s<
h i
lim ET0 sup ku./ .s/ k2 D 0; uniformly in ./: (6.27)
T0 " T0 <s
Proof. (i) is immediate from (5.111) and (5.196).

(ii) Suppose 2 H1 . By the inequality
sup ku.t/ u./k2 2 sup ku.t/ u.t1 /k2 C 2 sup ku./ u.t1 /k2 ;
;t 2Œt1 ;t2 t 2Œt1 ;t2 2Œt1 ;t2
N ."; / WD ".4N3jjj jjj2 /1 (6.28)
satisfies (6.25) thanks to (6.23).

For 2 H0 , we fix Q 2 H1 , so that
Q 2< "
k k : (6.29)
12N1
Since for the response uQ .t/ with initial state Q (6.8) yields
h i "
E sup ku.t/ uQ .t/k2 < ; (6.30)
T0 t T 12
we obtain
h i
E sup ku.t/ u./k2
;t 2Œt1 ;t2
h i h i
6E sup ku.t/ uQ .t/k2 C 3E sup kQu.t/ uQ ./k2
T0 t T ;t 2Œt1 ;t2
"
<" if t2 t1 < N ; Q .DW ."; //:
6
This gives (ii).
Since (iii) is clear by (ii), we have completed the proof.
4. Dependence on control processes

Proposition 6.4.
N ;
(i) There is a constant N4 , depending only on ; N K, and l, such that, for 2
H1 ,
h Z T i
Q Q
ET0 sup ku ./
.t/ u ./
.t/k C 2
jjju./ .t/ u./ .t/jjj2 dt
T0 t T T0
1
N4 jjj jjj2 .EŒ1./ Q ./14 / : 2 (6.31)
(ii) For 2 H0 and " > 0, there is ı."; / > 0, such that
h Z T i
Q ./
ET0 Q sup ku ./
.t/u .t/k C
2
jjju./ .t/uQ ./ .t/jjj2 dt < " (6.32)
T0 t T T0
Q 2<
for any Q 2 H0 with k k "
48N1 whenever EŒ1./ Q ./14 < ı."; /.
Q
Proof. For simplicity, we put u D u./ ; uQ D u./ and v D u uQ . Then we have
d v.t/ D A./ v.t/ dt C G ./ .t/v.t/ d W .t/ C Z.t/ dt C .t/ d W .t/; t 2 .T0 ; T
(6.33)
with
v.T0 / D 0;
where
Z.t/ D .A./ .t/ AQ ./ .t//Qu.t/
and
.t/ D .G ./ .t/ G Q ./ .t//Qu.t/:
Since (B10 ) provides a constant c, depending only on l, such that
kZ.t/k2 cj.t/ Q .t/j2 jjjQu.t/jjj2 ; 8t; P -a.s. (6.34)
and
k .t/k2m cj.t/ Q .t/j2 jjjQu.t/jjj2 ; 8t; P -a.s. (6.35)
by using the estimate

6.2 Formulations of Control Problems 217
Z T h i
E j.t/ Q .t/j2 jjjQu.t/jjj2 dt E sup jjjQu.t/jjj2 1./ Q ./12
T0 T0 t T
h i 12 1
E sup jjjQu.t/jjj4 .E1./ O ./14 / 2
T0 t T
and (6.33)–(6.35), we can deduce (i) from (6.11).

(ii) For 2 H0 and " > 0, we fix # 2 H1 such that
"
k # k2 < : (6.36)
48N1
Hence,
" "
kQ # k2 < Q 2<
for k k : (6.37)
12N1 48N1
Stressing the dependence on the initial state of response, we write
./ Q
./
uT Q .t/ uT Q .t/
0 0
./ ./ ./ Q

./ Q
./ Q
./
D .uT Q .t/ uT # .t// C .uT # .t/ uT # .t// C .uT # .t/ uT Q .t//: (6.38)
0 0 0 0 0 0
Taking into account (6.8) and (6.31), we obtain

" 1
LHS of (6.32) < C N4 jjj # jjj2 .E1./ Q ./14 / 2 :
2
Thus, putting ı."; / WD "2 .2N4 jjj # jjj2 /2 we complete the proof.
6.2 Formulations of Control Problems
In this section, we formulate optimization problems for stochastic systems described

by Zakai equations, namely, we are concerned with optimization problems similar
to those in Chap. 2, for Hilbert space-valued stochastic systems. When a control
process ./ is chosen, the cost on the time internal ŒT0 ; t is given by
Z t
r.u./.s/; .s// ds C F .u./ .t//;
T0
where the integral part and F .u./ .t// denote a running cost and a terminal cost,
respectively. We want to minimize (or maximize) the expectation of the cost, by
taking a suitable ./. This optimization is a generalization of optimal control
problem for partially observable diffusions, see Example 6.1. For the terminal cost,
we introduce a Banach space C # (see (6.40)). The basic properties of the terminal
costs and running costs are studied in Sects. 6.2.2 and 6.2.3, respectively. Putting
these results together, we state properties of the value function in Sect. 6.2.4.
6.2.1 Preliminaries
Firstly we define a Banach space C # . Put
F ./
F # ./ D for 2 H0 ; (6.39)
1 C kk
and define C # by
C # D fF 2 C.H0 /I F # 2 Cbu .H0 /g: (6.40)
Then C # becomes a Banach space, equipped with norm k k# given by
kF k# D sup jF # ./j for F 2 C # : (6.41)

2H0
We write
F1# F2# if and only if F1# ./ F2# ./; 8 2 H0 ; (6.42)
or, equivalently F1 ./ F2 ./; 8 2 H0 .

For the set C # , the following facts are clear. If F is uniformly continuous, then
F 2 C # . If F 2 C # , then F is (at most) linearly growing and uniformly continuous
on any bounded set.
From now on, we always take terminal costs from C # .
Proposition 6.5. Let F 2 C # be given. We define F ./ WD F .I T0 ; t; .// by
F ./ D ET0 F .u./ .t//; 2 H0 : (6.43)
Then F is in C # . More precisely,
jF ./j p
1 _ N0 kF k# ; (6.44)
1 C kk
with the constant N0 of (6.4), and, for " > 0, there is ı1 ."I F / > 0 independent of
t; T0 and ./, such that
ˇ F ./ Q ˇˇ
F ./
ˇ
ˇ ˇ<" (6.45)
1 C kk 1 C kk Q
Q < ı1 ."; F /.
whenever k k
Proof. ci denotes a constant depending only on ; N ;
N K, and l. By (6.8), F is
./
continuous. For simplicity, we put u.t/ D uT0 .t/. Equation (6.44) follows from the
inequality
jF .u.t//j 1 C ku.t/k
kF k# : (6.46)
1 C kk 1 C kk
./
Putting uQ .t/ D uT Q .t/ and v.t/ D u.t/ uQ .t/, let us show (6.45). We have
0
ˇ 1 C ku.t/k 1 C kQu.t/k ˇˇ
ˇ #
ˇF .u.t// F # .Qu.t// ˇ
1 C kk Q
1 C kk
1 C ku.t/k ˇ 1 C ku.t/k 1 C kQu.t/k ˇ
ˇ ˇ
mF # .kv.t/k/ C kF k# ˇ ˇ
1 C kk 1 C kk Q
1 C kk
WD J1 C kF k# J2 : (6.47)
Since the modulus m./ satisfies that, for any " > 0, there is C" > 0, such that
m.a/ " C C" a; a 0, we have
EŒmF # .kv.t/k/2 2."2 C C"2 Ekv.t/k2 /

Q 2 /;
2."2 C C"2 N1 k k (6.48)
by (6.8). Thus, (6.4) and (6.48) yield
Q 2 /:
.EJ1 /2 c1 ."2 C C"2 k k (6.49)
For EJ2 , we observe that
kv.t/k Q
.1 C kQu.t/k/k k
J2 C : (6.50)
1 C kk Q
.1 C kk/.1 C kk/
Hence again (6.4) and (6.8) yield
Q
EJ2 c2 k k: (6.51)
which in conjunction with (6.47) and (6.49) gives (6.45).

6.2.2 Properties of the Terminal Cost
Let us study how the terminal cost function, .t; T0 ; ; ./I F / WD ET0 F .u./.t//,
depends on T0 ; , and ./.

Proposition 6.6. For positive constants " and n, there is ı2 ."; nI F / > 0, such that
h ˇ ˇi
ˇ ./ ./ ˇ
E sup ˇF .uT0 1 .t// F .uT0 2 .t//ˇ < "; 8T0 and ./; (6.52)
T0 t T
whenever ki k < n .i D 1; 2/ and k1 2 k < ı2 ."; nI F /.

The proof is easy, using F . / D F # . /.1 C k k/.
2. Dependence on time
Proposition 6.7. For 2 H0 and " > 0, there is ı3 ."; I F / > 0, such that, for
t1 ; t2 2 ŒT0 ; T ,
h i
ET0 sup jF .u./.s2 // F .u./.s1 //j < "; 8./ 2 ; (6.53)
s1 ;s2 2Œt1 ;t2
whenever t2 t1 < ı3 ."; I F /.

Proof. By changing " to 2" , we may assume s1 D t1 for the proof. We omit the
superscript ./, for simplicity, and put D sups2Œt1 ;t2 ku.s/ u.t1 /k. Observing
that

jF .u.s//j F .u.t1 //j mF # ./ 1 C sup ku.t/k C kF k# ; (6.54)
T0 t T
we have
h i
ET0 sup jF .u.s// F .u.t1 //j
t1 st2
ET0 ŒRHS of (6.54)

p p
2." C C" E 2 /.1 C N0 kk/ C kF k# E: (6.55)
Now, applying Proposition 6.3 (ii) to E 2 , we complete the proof.
3. Dependence on ./
Proposition 6.8.
(i) For positive constants " and n, there is ı4 ."; nI F / > 0, such that
h i
Q
ET0 sup jF .u./.t// F .u./ .t//j < " (6.56)
T0 t T
for 2 H1 with jjj jjj n, whenever E1./ Q ./14 < ı4 ."; nI F /.

(ii) For positive constants " and n and 2 H0 with kk n, there are two positive
constants ı5 ."; nI F / and ı6 ."; nI F /, such that
h ˇ ˇi
ˇ ./ Q
./ ˇ
E sup ˇF .uT0 .t// F .uT Q .t//ˇ < " (6.57)
0
T0 t T
Q < ı5 ."; nI F / and E1./ Q ./14 < ı6 ."; nI F /.

whenever k k
Proof. (i) By using the estimate similar to (6.54) and (6.55) for WD
./ Q
./
supt kuT0 .t/ uT0 .t/k we can derive (6.56) from (6.31).
(ii) Put ı D 12 .1 ^ ı2 ."; n C 1I F //. Then Proposition 6.6 leads to
h ˇ ˇi
ˇ ./ ./ ˇ
E sup ˇF .uT0 .t// F .uT Q .t//ˇ < " (6.58)
0
T0 t T
Q < ı.
for any ./ 2 , whenever k k
Next we fix 2 H1 , so that k k < ı and k k < n. Then
kQ k < ı2 ."; n C 1I F /: (6.59)
Taking into account the inequality
./ Q
sup jF .uT0 .t// F .uT Q .t//j
t 0
./ ./
sup jF .uT0 .t// F .uT0 .t//j
t
./ Q
./
C sup jF .uT0 .t// F .uT0 .t//j
t
Q
./ Q
./
C sup jF .uT0 .t// F .uT Q .t//j
t 0
and using (6.52), (6.59), and (6.56), we obtain (ii), with
1 "
ı5 ."; nI F / D 1 ^ ı2 ; n C 1I F
2 3
and
"
ı6 ."; I F / D ı4 ; jjj jjjI F :
3
6.2.3 Running Cost
Suppose r W H0 7! R1 is Lipschitz continuous, say
jr.1 ; 1 / r.2 ; 2 /j lr .k1 2 k C j1 2 j/: (6.60)
Put
r0 D sup jr.0; /j: (6.61)

2
When a control process ./ 2 is taken, the running cost R on ŒT0 ; t is given by
Z t
./
R.t; T0 ; ; .// D r.uT0 .s/; .s// ds: (6.62)
T0
Let us list several properties of the running cost function, by using results
obtained in Sects. 5.4 and 6.1. By Ki , we denote a constant depending only on
N ;
; N K; l; lr and r0 .
1. Bound
h i Z T
E sup jR.t; T0 ; ; .//j r0 T C lr ET0 ku./ .s/k ds
T0 t <T T0
K1 .1 C kk /: (6.63)

h i
E sup jR.t; T0 ; 1 ; .// R.t; T0 ; 2 ; .//j K2 k1 2 k : (6.64)
T0 t T
3. Dependence on time
Z t p
ET0 jr.u./ .s/; .s//j ds K3 .1 C kk / t : (6.65)

4. Dependence on control processes

h i
E sup jR.t; T0 ; ; .// R.t; T0 ; ; Q .//j
T0 t T
p 1
diam K4 .1 C kk/.EŒ1./ Q ./12 / 4 : (6.66)
Q
Proof. For simplicity, we put u.t/ D u./ .t/; uQ .t/ D u./ .t/ and v.t/ D u.t/ uQ .t/.
Then v satisfies
8
ˆ
ˆ d v.t/ D .A./ .t/v.t/ C Z.t// dt
<
C .G ./ .t/v.t/ C .t// d W .t/; t 2 .T0 ; T ;
ˆ
:̂
v.T0 / D 0;
where
(
Z.t/ D .A./ .t/ AQ ./ .t//Qu.t/;
.t/ D .G ./ .t/ G Q ./ .t//Qu.t/:
From (5.188), we deduce that

Z T hZ T i
E kv.t/k2 dt K5 E .kZ.s/k2H2 C k .s/k2m / dt : (6.67)
T0 T0
Since (B10 ) yields
kZ.t/k2H2 K6 j.t/ Q .t/j2 kQu.t/k2

K6 .diam /j.t/ Q .t/jkQu.t/k2
and
k .t/k2m K6 .diam /j.t/ Q .t/jkQu.t/k2 ;
inserting these estimates into (6.67), we obtain

Z T Z T 12
1
E kv.t/k dt K7 .diam /.E1./ Q ./1 / E
2 2 2 kQu.s/k4 ds
T0 T0
1
K8 .diam /.E1./ Q ./1 / .1 C kk2 /:
2 2
Hence, noticing that

Z T
sup jR.t; T0 ; ; .// R.t; T0 ; ; Q .//j lr .kv.t/k C j.t/ .t/j/
Q dt;
t T0
we obtain (6.66).
ER.t;T0 ;;.//
5. R ./ WD
#
1Ckk
is bounded and Lipschitz continuous uniformly in
.t; T0 ; .//
Indeed, (6.63) and (6.64) imply
jR# ./j K1 (6.68)

and
jR# .1 / R# .2 /j

EjR.t; T0 ; 2 ; .//j
EjR.t; T0 ; 1 ; .// R.t; T0 ; 2 ; .//j C k1 2 k
1 C k2 k
.K1 C K2 /k1 2 k: (6.69)
6.2.4 Value Functions
To emphasize the dependence on F , we write F in the cost functional and the

payoff:
Z t
./ ./
C.t; T0 ; ; ./I F / D r.uT0 .s/; .s// ds C F .uT0 .t//
T0
and
J.t; T0 ; ; ./I F / D EC.t; T0 ; ; ./I F /:
From (6.57) and (6.66), we deduce that, for " > 0 and 2 H0 , there is a positive
constant ıC ."; I F / such that
h i
E sup jC.t; T0 ; ; ./I F / C.t; T0 ; ; Q ./I F /j < " (6.70)
T0 t T
whenever E1./ Q ./12 < ıC ."; I F /. Now we want to minimize the payoff by
choosing a suitable control process from .
Definition 6.1. The minimum payoff
V .t; T0 ; I F / WD inf J.t; T0 ; ; ./I F / (6.71)

./2
is called the value function.

From (6.44) and (6.63), one immediately obtain the following remark.
Remark 6.1. There are constants K1 and K2 , such that
jJ.t; T0 ; ; ./I F /j K1 .1 C kk / C K2 kF k# .1 C kk/; 8t; T0 ; ./:

(6.72)
Equation (6.72) provides a bound of the value function.
From the results in Sects. 6.2.2 and 6.2.3, the following properties of payoff and
the value function are immediate.
Proposition 6.9. Let F 2 C # be given.
(i) V .t; T0 ; I F / 2 C # .
(ii) V .t; T0 ; I F / D V .t T0 ; 0; I F /.
(iii) Dependence on the initial state
There are modulus functions, mn .I F /; n D 1; 2; : : : , such that
8
ˆ
ˆ jJ.t; T0 ; 1 ; ./I F / J.t; T0 ; 2 ; ./I F /j mn .k1 2 kI F /;
ˆ
ˆ
ˆ
< 8t; T0 ; ./;
ˆ
ˆ jV .t; T0 ; 1 I F / V .t; T0 ; 2 I F /j mn .k1 2 kI F /;
ˆ
ˆ
:̂
8t; T0 ;
(6.73)
whenever k1 k; k2 k n,
(iv) Dependence on time
For 2 H0 , there is a modulus function m.I ; F /, such that
8
ˆ
ˆ jJ.t1 ; T0 ; ; ./I F / J.t2 ; T0 ; ; ./I F /j m.jt1 t2 jI ; F /;
<
8./ 2 ;
ˆ
:̂
jV .t1 ; T0 ; I F / V .t2 ; T0 ; I F /j < m.jt1 t2 jI ; F /:
(6.74)
6.3 Dynamic Programming Principle (DPP)
As we said in Chap. 2, the DPP is a useful tool for analyzing control problems for
finite-dimensional processes governed by SDEs. In this section, we focus on DPP
for Hilbert space-valued processes governed by Zakai equations. We will formulate
DPP via the semigroup constructed by means of the value function, in the same way
as in Sect. 2.2. First, we show the DPP for the discrete-time case in Sect. 6.3.1. Then
we deal with the continuous-time DPP in Sect. 6.3.2, by using a time discretization
technique. We study properties of the semigroup and its generator in Sects. 6.3.3
and 6.3.4, respectively.
6.3.1 Discrete-Time DPP
Let D D .t1 ; : : : ; tp / be a division of Œ0; T . Put

D
D f./ 2 I .t/ D .ti / for t 2 Œti ; ti C1 /; i D 0; : : : ; pg:
./ 2 D is called a switching control process for D. Using switching control

processes in D , we define the value function V D by
V D .t; T0 ; I F / D inf J.t; T0 ; ; ./I F /: (6.75)

./2 D
The following proposition is easy.

Proposition 6.10. Let F be in C # .
(i) V D .t; T0 ; I F / 2 C #
p
kV D .t; T0 ; I F /k# 1 _ N0 ..r0 C lr /T C kF k# /; (6.76)
with the constant N0 of (6.4).

By (6.45) and (6.69), there is a modulus m.I F /, such that
ˇ V D .t; T ; I F / V D .t; T ; I F / ˇ
ˇ 0 1 0 2 ˇ
ˇ ˇ m.k1 2 kI F / (6.77)
1 C k1 k 1 C k2 k
for any t; T0 and D,

(ii) There is a sequence of moduli mn .I F /; n D 1; 2; : : : , such that
jV D .t; T0 ; 1 I F / V D .t; T0 ; 2 I F /j mn .k1 2 kI F / (6.78)
for any t; T0 , and D, whenever k1 k; k2 k n,

(iii) For 2 H0 , there is a modulus mt .I ; F /; such that
jV D .t1 ; T0 ; I F / V D .t2 ; T0 ; I F /j mt .jt1 t2 jI ; F / (6.79)
for any T0 and D.

For < t, we can define a monotone operator VDt I C # ! C # by
VDt F ./ D V D .t; ; I F /; 2 H0 : (6.80)
Now we have;
Remark 6.2.
Q
D DQ H) VDt F ./ VDt F ./; 8 2 H0
and
F FQ H) VDt F VDt FQ :
Regarding 2 as a constant control process, we introduce an auxiliary

monotone operator t I C # ! C # by
t F ./ D inf J.t; ; ; I F / D inf J.t ; 0; ; I F /: (6.81)

2 2
For any ; t and F; J.t; ; I F / is continuous on H0 . Since is compact,
WD f 2 I t F ./ D J.t; ; ; I F /g (6.82)
is non-empty and compact. Hence there is a minimum selector O tF (Borel mapping;
H0 7! ) such that O tF ./ 2 ; 8 2 H0 . Thus
t F ./ D J.t; ; ; O tF ./I F / (6.83)
holds.
Theorem 6.2. For 0 i j p,
Y
j
VtiD;tj C1 F ./ D ti ti C1 tj tj C1 F ./ DW tk tkC1 F ./; 8 2 H0 : (6.84)
kDi
Outline of Proof. Since we can apply the same arguments as in Theorem 2.2, we
only sketch the proof.
Let ./ 2 D be given. Since (W .s/ W .tj /; s tj ) is independent from Ftj ,
(u .t/I t 2 Œtj ; tj C1 /) is the response for the constant control .tj / with initial
./
state u./ .tj /, under the conditional probability P .jFtj /. Hence, by freezing .tj /
and u./ .tj / we obtain
E.C.tj C1 ; ti ; ; ./I F /jFtj / R.tj ; ti ; ; .// C tj tj C1 F .u./.tj //

D C.tj ; ti ; ; ./I tj tj C1 F /: (6.85)
Repeating the same arguments, we conclude that
Y
j
J.tj C1 ; ti ; ; ./I F / tk tkC1 F ./: (6.86)
kDi
D
Taking the infimum over we obtain
Y
j
VtiDtj C1 F ./ tk tkC1 F ./: (6.87)
kDi
Next we construct an optimal control process O ./ in D , by using the minimum

selector. From (6.83) it follows that
tj tj C1 F .u./.tj // D J.tj C1 ; tj ; u./ .tj /; Oj I F /; (6.88)
where Oj WD Otj tj C1 F .u./ .tj //.

Qj
Put Fm D kDm tk tkC1 F; m D i; : : : ; j . On Œti ; ti C1 /, we use the constant
control Oi WD Oti ti C1 Fi C1 ./ and denote its response with uO .ti / D , by uO .t/. Next
we define Oi C1 by
Oi C1 D Oti C1 ti C2 Fi C2 .Ou.ti C1 // on Œti C1 ; ti C2 /
and glue to Oi , namely
O .s/ D Œti ;ti C1 / .s/Oi C Œti C1 ;ti C2 / .s/Oi C1 ; s 2 Œti ; ti C2 /:
uO .t/ denotes its response with uO .ti / D . Repeating this procedure, we construct
O ./ 2 D and its response uO ./. Then we have
Y
j
J.tj C1 ; ti ; ; O ./I F / D tk tkC1 F ./: (6.89)
kDi
Since (6.89) yields the opposite inequality for (6.87), we have completed the
proof.
Corollary 6.1. Discrete-time DPP
Let D D .t1 ; : : : ; tp / be a division of Œ0; T . For ti tl tj ,
VtiDtj F ./ D VtiDtl .VtlDtj F /./; 8 2 H0 : (6.90)
6.3.2 Continuous-Time DPP
We will prove the continuous-time DPP by using time discretization approach

(cf. [BN90]). Let us define V t I C # ! C # by
V t F ./ D V .t; ; I F /; 2 H0 : (6.91)
We need to establish the semigroup property of fV t I tg, which formulates the
DPP.
Theorem 6.3. For 0 1 2 3 T ,
V1 3 F ./ D V1 2 .V2 3 F /./; 8 2 H0 : (6.92)

Step 1. Approximation
Let ; t be given. Let Dn ; n D 1; 2 : : : be a sequence of divisions satisfying
; t 2 D1 ; Dn DnC1 ; n D 1; 2 : : : ; and lim jDn j D 0: (6.93)

n!1
Then
lim VDt n F ./ D V t F ./; 8 2 H0 : (6.94)

n!1
Indeed, by (6.93), VDt n F ./ is decreasing as n ! 1, and
V t F ./ lim VDt n F ./: (6.95)

n!1
For the opposite inequality, we fix ./ 2 arbitrarily and take n0 ./ 2 Dn0 ,
so that
8
< lim E1n0 ./ ./12 D 0;
n0 !1
(6.96)
: lim n0 .t/ D .t/; a.e. on Œ0; T ˝:
0 n !1
Thus, (6.70) and (6.96) yield
lim J.t; ; ; n0 ./I F / D J.t; ; ; ./I F /: (6.97)

n0 !1
Observing that
D
J.t; ; ; n0 ./I F / V t n F ./ lim VDt n F ./;
0
n!1
we have
J.t; ; ; ./; F / lim VDt n F ./; 8./ 2 ;

n!1
from which the opposite of inequality (6.95) follows.

Step 2. Let us show that
V1 3 F ./ V1 2 .V2 3 F /./: (6.98)

Let Dn ; n D 1; 2; : : : be divisions satisfying (6.93) with 1 ; 2 ; 3 2 D1 instead

of ; t 2 D1 . Then
VD n
1 3
F ./ D VD n
1 2
.VD n
2 3
F /./
VD n
1 2
.V2 3 F /./ (6.99)
follows from Corollary 6.1 and Remark 6.2. Letting n ! 1 in (6.99), we

obtain (6.98).
Step 3. Opposite of inequality (6.98)
For any n and m, we have
D D
V1 nCm
3
F ./ V1 nCm
2
.VD m
2 3
F /./
J.2 ; 1 ; ; ./I VD2 m3 F /; 8./ 2 : (6.100)
Since (6.94) and Remark 6.1 yield
lim VD m
2 3
F .u./ .2 // D V2 3 F .u./ .2 // P -a.s.
m!1
and
jVD m
2 3
F .u./ .2 //j c.1 C kF k# /.1 C ku./ .2 /k/ P -a.s.
with a constant c independent of 2 , 3 , Dm , and ./, the dominated convergence

theorem shows that
lim J.2 ; 1 ; ; ./I VD2 m3 F / D J.2 ; 1 ; ; ./I V2 3 F /: (6.101)

m!1
Thus, letting m ! 1 in (6.100) and using (6.101), we have
V1 3 F ./ J.2 ; 1 ; ; ./I V23 F /; 8./ 2 (6.102)
from which the opposite of inequality (6.98) follows.

This completes the proof of the theorem.
Since V t F D V0t F thanks to the time homogeneity of the coefficients, we
denote V0t by Vt , when there is no danger of confusion. Using this notation, we
have a one-parameter semigroup (Vt ; t 2 Œ0; T ) on C # .
Remark 6.3. DPP
Vt1 Ct2 F ./ D Vt1 .Vt2 F /./: (6.103)

Collecting previous results, we can easily deduce that .Vt / WD .Vt ; t 2 Œ0; T / is
a monotone semigroup on C # , satisfying
(i) 0 -contractiveness
kVt F Vt FQ k# e 0 t kF FQ k# (6.104)
p
where 0 D 1 C N1 with the constant N1 of (6.8).
(ii) t-continuity
For any 2 Œ0; T /,
lim V Ct F ./ D V F ./; 8 2 H0 : (6.105)

t !0
6.3.3 Characterization of .Vt /

We will characterize (Vt ) via payoffs. For 2 , we define Jt F ./ by

Jt F ./ D J.t; 0; ; I F / .D J.t C ; ; ; I F //; 2 H0 : (6.106)

Since Jt F is nothing but the value function for D f g, .Jt ; t 2 Œ0; T / is a
monotone semigroup on C # , satisfying (6.104) and (6.105). Further, for any t and
,

Vt F Jt F; 8F 2 C # : (6.107)
Put ˙ D the set of all 0 -contractive and t-continuous monotone semigroup

.St / WD .St ; t 2 Œ0; T / on C # , satisfying

St F Jt F; 8F 2 C # ; 8t 2 Œ0; T ; 8 2 :
Now we can state

Theorem 6.4. .Vt / is in ˙. Moreover for any .St / 2 ˙,
St F Vt F; 8F 2 C # ; 8t 2 Œ0; T ; (6.108)

that is, .Vt / is the maximal element of ˙, called the envelope of ..Jt /; 2 /.
Proof. Let t be given. Take a sequence of divisions, Dn ; n D 1; 2; : : : ; so that,
t 2 D1 ; Dn DnC1 ; n D 1; 2; : : : ; lim jDn j D 0:

n!1
Since, for any F 2 C # and any s,

s F ./ D inf Js F ./ Ss F ./ (6.109)
2

the monotonicity of Jt yields

tj 1 tj .tj t F /./ inf Jtj tj 1 .St tj F /./
2
Stj tj 1 .St tj F /./ D St tj 1 F ./: (6.110)
Repeating this procedure and using (6.84), we have
V0tDn F ./ St F ./; n D 1; 2; : : : : (6.111)
Thus, taking into account (6.94) and letting n ! 1 in (6.111), we obtain

(6.108).
6.3.4 Derivatives of Vt F./
Let us determine the generator of .Vt /. Put
1X 2
m

L F ./ D hA ; DF ./i C hD F ./Gk ; Gk i; 2 : (6.112)
2
kD1
Theorem 6.5. For F 2 Cbu

2
.H1 /, we have
1
lim .Vt F ./ F .// D inf .L F ./ C r.; // for 2 H1 : (6.113)
t !0
t 2
Proof. Suppose that
jF ./j C jjjDF ./jjj C kD 2 F ./kL.H1 IH1 / K0
and
jF ./ F . /j C jjjDF ./ DF . /jjj C kD 2 F ./ D 2 F . /kL.H1 IH1 /

m0 .k k /:
Let 2 H1 be given. First we fix ./ and evaluate
./
I.s/ WD L.s/ F .u0 .s// L.s/ F ./: (6.114)
We are going to show that, for " > 0, there is ."; / > 0, independent of ./, such
that
h i
E sup jI.s/j < " for t < ."; /: (6.115)
0st
./
We divide the proof of (6.115) into four steps. For simplicity, we put u.t/ D u0 .t/
and ci denotes a constant independent of t; and ./.
Step 1. There is 1 ."; / > 0, independent of ./, such that
h i
E sup jhA./.u.s/ /; DF .u.s//ij < "; for t < 1 ."; /: (6.116)
0st
Indeed, the expression inside [ ] is
K sup jjju.s/ jjj sup jjjDF .u.s//jjj

0st 0st
KK0 sup jjju.s/ jjj; (6.117)

0st
where K is the constant of (B10 ).

On the other hand, for "Q > 0, we can take Q 2 H2 so that jjj jjj
Q < "Q. Thus,
./
using the response uQ .t/ WD uQ 0Q .t/ we have
E sup jjju.s/ jjj

0st
Q C jjjQ jjj
E sup jjju.s/ uQ .s/jjj C E sup jjjQu.s/ jjj
0sT 0st
p p p
Q
. N1 C 1/Q" C N2 t II; (6.118)
by (6.9) and (6.109). Now (6.117) and (6.118) yield (6.116).

h i
E sup jhA./ .s/; DF .u.s// DF ./ij < "; (6.119)
0st
whenever t < 2 ."; /.

Indeed, by (5.196), for any "Q > 0,

LHS of (6.119) KjjjjjjEm0 sup ku.s/ k
0st
h i
Kjjjjjj "Q C C"QE sup ku.s/ k
0st
p p
Kjjjjjj.Q" C C"Q c0 N t kk/:
This yields (6.119).


h i
./ ./
E sup jhD 2 F .u.s//Gk .s/u.s/; Gk .s/.u.s/ /ij < ";
0st
k D 1; 2; : : : ; m; (6.120)

Indeed, we have
LHS of (6.120)
h i
./ .s/
E sup jjjD 2 F .u.s//Gk .s/u.s/jjj kGk .s/.u.s/ /k
0st
h i
K0 K 2 E sup ku.s/k ku.s/ k
0st
12 12
K0 K 2 E sup ku.s/k2 E sup ku.s/ k2
0st 0st
p
c1 kk jjjjjj t (by (5.111));
with a constant c1 independent of k and ./. This yields (6.120).

h i
./ ./
E sup jh.D 2 F .u.s// D 2 F .//Gk .s/; Gk .s/ij < ";
0st
k D 1; 2; : : : ; m; (6.121)

Indeed, for any "Q > 0,
./
sup kD 2 F .u.s// D 2 F ./kL.H1 IH1 / kGk k2
0st
sup m0 .ku.s/ k /Kkk2

0st

K "Q C C"Q sup ku.s/ k kk2 : (6.122)
0st
Taking the expectation of (6.122) and using (5.196), we obtain (6.121).

Collecting the above results in Steps 1–4, we have (6.115).
Next we consider the running cost:
h i
E sup jr.u.s/; .s// r.; .s//j lr E sup ku.s/ k
0st 0st
p
c3 tjjjjjj (by (5.111)); (6.123)
with a constant c3 , independent of ./.

Thus, (6.115) and (6.123) yield that for any " > 0, there is ."; / > 0, such
that
h i
E sup jL.s/ F .u.s// C r.u.s/; .s// L.s/ F ./ r.; .s//j < "; 8./
0st
(6.124)
whenever t < ."; /.
Finally, we note that
Z t
inf E .L.s/ F ./ C r.; .s/// ds
./2 0
Z t
E inf .L F ./ C r.; // ds
0 2
D t inf .L F ./ C r.; // (because inside ( ) is non random)

2
D inf t.L F ./ C r.; //

2
Z t
inf E .L.s/ F ./ C r.; .s/// ds (6.125)
./2 0
because 2 becomes a constant control process. Therefore, we have

Z t
inf E .L.s/ F ./ C r.; .s/// ds D t inf .L F ./ C r.; //:
./2 0 2
(6.126)
Now we are ready to compute the LHS of (6.113). From Itô’s formula of
Theorem 5.7 it follows that
Vt F ./ F ./
hZ t i
D inf E r.u.s/; .s// ds C F .u.t// F ./
./2 0
hZ t i
D inf E .r.u.s/; r.s// C L.s/ F .u.s/// ds
./2 0
n hZ t i
D inf E .r.u.s/; .s// C L.s/ F .u.s// r.; .s// L.s/ F .// ds
./2 0
Z t o
CE .r.; .s// C L.s/ F .// ds :
0
Hence (6.124) and (6.126) lead to (6.113).

Now Theorem 6.5 shows that U.t; / WD VT t F ./ has the backward dynamics
@t U.t; / C inf .L U.t; / C r.; // D 0:

2
This equation is called HJB equation.
6.4 Viscosity Solutions of HJB Equations
This section is devoted to the study of the HJB equations introduced at the end of
Sect. 6.3. Let us consider HJB equation on H0
@t U.t; /
1X 2
m

C inf hA ; DU.t; /i C hD U.t; /Gk ; Gk i C r.; / D 0;
2 2
kD1
2 H0 ; t 2 .0; T / (6.127)
U.T; / D F ./; 2 H0 : (6.128)
When the value function is smooth, the HJB equation provides the backward
evolution equation by Theorem 6.5. However we can hardly expect that the value
function will be regular. Here we will consider viscosity solution of the HJB
equation according to [L89] and [GŚ00].
In Sect. 6.4.1, we give the definition of the viscosity solution. Since we treat
the equation on H0 , the relation between the value function and the viscosity
solution has already been stated in [GŚ00]. Thus we will sketch the proof of the
assertion that the value function becomes a viscosity solution in Sect. 6.4.2 and
the uniqueness theorem in Sect. 6.4.3. Applying these results, we revisit control
problems for partially observable diffusions in Example 6.1.
6.4.1 Definitions
Put
n o
D ı./ 2 C 1 .0; T /I lim ı.t/ D lim ı.t/ D 1 and ı D inf ı.t/ > 0 :
t !0 t !T t
(6.129)
Elements of Cbu12
..0; T / H1 / is called test functions. For a test function ./ and
2 , we put
6.4 Viscosity Solutions of HJB Equations 237
1
L .t; / D hA ; D.t; /i C hD 2 .t; /G ; G im : (6.130)
2
P
where h ; im D m kD1 h k ; k i .
Definition 6.2. Let v./ 2 C.Œ0; T H0 /.

(a) v./ is called a viscosity subsolution of (6.127)–(6.128), if for every test function
./ and for every ı./ 2 , whenever v . C 12 ı.t/kk2 / attains a global
maximum at .tO; / O 2 .0; T / H0 , then O 2 H1 , and the subsolution inequality
0 O
O C ı .t / kk
0 @t .tO; / O 2
2
n o
C inf L .tO; /O C 1 ı.tO/L kk
O 2 C r.;
O / (6.131)
2 2
holds and
v.T; / F ./; 8 2 H0 : (6.132)
(b) v./ is called a viscosity supersolution of (6.127)–(6.128), if for every test

function ./ and for every ı./ 2 , whenever v . 12 ı.t/kk2 / attains
O 2 .0; T / H0 , then O 2 H1 and the supersolution
a global minimum at .tO; /
inequality
0 O
O ı .t / kk
0 @t .tO; / O 2
2
n o
O 1 ı.tO/L kk
C inf L .tO; / O 2 C r.;
O / (6.133)
2 2
holds and
v.T; / F ./; 8 2 H0 : (6.134)
(c) v./ is called a viscosity solution, if it is both a viscosity subsolution and a

viscosity supersolution.
6.4.2 Existence of Viscosity Solutions
Referring to [GŚ00], we prove

Theorem 6.6. Let F 2 C # and put U.t; / D VT t F ./. Then U is a viscosity
solution of (6.127)–(6.128).
Proof. The proof is divided into three steps.

O 2 .0; T / H0 is a maximizer of U . C 1 ı.t/kk2 /.
Step 1. Suppose that .tO; / 2
Then O 2 H1 .
Indeed, we have
j.t; /j C j@t .t; /jCjjjD.t; /jjj C kD 2 .t; /kL.H1 IH1 / K

on .0; T / H0 (6.135)
because ./ 2 Cbu

12
..0; T / H1 /.
By u ./ we denote the response for ./ with the initial condition; u./ .tO/ D
./
O
:
Since the semigroup property VtOt Vt T D VtOT .D VT Ot / yields
hZ t i
0 D inf E O ;
r.u./ .s/; .s// ds C U.t; u./.t// U.tO; / (6.136)
./2 tO
it follows that for any given ./,

Z t
E O
r.u./.s/; .s// ds EŒU.t; u./.t// U.tO; /: (6.137)
tO
Fix 2 arbitrarily and put
O
I.t/ WD EŒU.t; u .t// U.tO; /:
O is a global maximizer of U . C 1 ı.t/kk2 /, we have

Noticing that .tO; / 2
Z tn
I.t/ E @s .s; u .s// C hA u .s/; D.s; u .s//i
tO
1
C hD 2 .s; u .s//G u .s/; G u .s/im
2
1 0
C ı .s/ku .s/k2 C ı.s/hA u .s/; u .s/i
2
1 o
C ı.s/kG u .s/k2m ds (6.138)
2
by Itô’s formula (see Sect. 5.4).
Next we calculate the expression inside f g in the RHS of (6.138). Take an
integer n0 , so that
h 1i
jı.s/ ı.tO/j C jı 0 .s/ ı 0 .tO/j 1 for s 2 tO; tO C : (6.139)
n0
By ci we denote positive constants, independent of s, O and . From (6.3), (6.6),

and (6.135) it follows that
inside f g c1 .1 C jjju .s/jjj C ku .s/k2 /

N jjju .s/jjj2 C .jı.
ı N tO/j C 1/ku .s/k2
N
c2 .1 C ku .s/k2 / ı jjju .s/jjj2
2
N
c2 1 C sup ku .s/k2 ı jjju .s/jjj2 : (6.140)
tOsT 2
Thus, for t 2 ŒtO; tO C 1

n0
,
Z t
O O
O
I.t/ c3 .t t /.1 C kk / ı E
2
jjju .s/jjj2 ds: (6.141)
2 tO
Noting that
Z t
E jr.u .s/; /j ds .t tO/c4 1 C E sup ku .s/k/
tO tOsT
O
c5 .t tO/.1 C kk/; (6.142)
and combining (6.137), (6.141), and (6.142), we obtain

h Z t i
1 O 2 / DW a:
E jjju .s/jjj2 ds c6 .1 C kk (6.143)
t tO tO
Put t D tO C 1
n
.n D n0 ; n0 C 1; : : : /. Then there is tn 2 ŒtO; tO C n1 , such that
EŒjjju .tn /jjj2 a .n D n0 ; n0 C 1; : : : /; (6.144)
by (6.143). Hence, along an appropriate subsequence n0 , we have
u .tn0 / ! uQ weakly in L2 .˝I H1 / (6.145)
for some uQ 2 L2 .˝I H1 /. Therefore u .tn0 / ! uQ weakly in L2 .˝I H0 /. But
O 2 D 0;
lim EŒku .tn0 / k (6.146)
n0 !1
because u 2 L2 .˝I C.ŒtO; T I H0 //. Consequently uQ D O P -a.s., which show

that O 2 H1 .
Step 2. Suppose that .tO; / O 2 .0; T / H0 is a global minimizer of U .

1
ı.t/kk 2
/. Then O 2 H1 .

2
Indeed, we take n1 -optimal control process n ./ at tn WD tO C n1 .n D n0 ; n0 C
1; : : : /, namely
hZ tn i
0 EtOO O < 1:
r.un ./ .s/; n .s// ds C U.tn ; un ./ .tn // U.tO; /
tO n
(6.147)
O is a global minimizer of U . 1 ı.t/kk2 /, referring to (6.143)
Since .tO; / 2
and omitting the subscript tO; O for simplicity, we have
Z tn
n O 2 / DW a:
Ejjjun ./ .s/jjj2 ds c7 .1 C kk (6.148)
tO
R1
Since the LHS D 0 Ejjjun ./ .tO C n /jjj2 d, there is n 2 ŒtO; tO C n1 , such that
Ejjjun ./ .n /jjj2 a .n D n0 ; n0 C 1; : : : /:
Thus, along an appropriate subsequence n0 ,
un ./ .n0 / ! uQ weakly in L2 .˝I H1 / (6.149)
for some uQ 2 L2 .˝I H1 /. On the other hand, (6.26) shows that
O 2 D 0:
lim EŒkun0 ./ .n0 / k (6.150)
n0 !1
Now, (6.149) and (6.150) imply that uQ D O P -a.s., i.e., O 2 H1 .

Step 3. Subsolution inequality
By Step 1, we suppose that .tO; /O 2 .0; T / H1 is a global maximizer of U
. C 12 ı.t/kk2 /. Then Itô’s formula together with (6.137) yields
hZ t
0 inf E r.u./ .s/; .s// ds
./2 tO
i
O C 1 .ı.t/ku./ .t/k2 ı.tO/kk
C .t; u./ .t// .tO; / O 2/
2
Z t
ı 0 .s/ ./
D inf E r.u./ .s/; .s// C @t .s; u./.s// C ku .s/k2
./2 tO 2
1
C L.s/ .s; u./ .s// C ı.s/L.s/ ku./ .s/k2 ds: (6.151)
2
We claim that, for " > 0, there is O

1 ."; / > 0, independent of ./, such that
h i
E O 2 < ";
sup jjju./.s/ jjj 8./; (6.152)
tOst
whenever t tO < O
1 ."; /.
Indeed, for "Q > 0, we take 2 H2 , so that jjjO jjj2 < ".
Q Let v./ denote the
./ O
response for ./, with v .t / D . Then, the inequality
O jjju./ .s/ v./ .s/jjj C jjjv./ .s/

jjju./ .s/ jjj jjj C jjj O
jjj
and relations (5.107) and (5.112) yield

h i
O 2 c6 "Q C c7 I I2 .t tO/:
E sup jjju./ .s/ jjj (6.153)
tOst
from which (6.152) follows.

By using (6.152), we can take O
2 ."; / > 0, independent of ./, so that
h i
E O 2 j < ";
sup ı.s/jL.s/ ku./ .s/k2 L.s/ kk 8./ (6.154)
tOst
O
whenever t tO < 2 ."; /.
We treat L .s; u .s// and other terms of (6.151) in the same way as the
.s/ ./
proof of Theorem 6.5. Then (6.151) shows that the subsolution inequality holds.
Since we can establish the supersolution inequality by the same arguments, this
completes the proof.
6.4.3 Uniqueness of the Viscosity Solution
To show the uniqueness of the viscosity solution, we state the comparison theorem
[GŚ00], Th. 6.1. Denote by C the set of all U 2 C.Œ0; T H0 / such that
(a)
jU.t; /j
lim sup D 0;
kk!1 t kk2
(b) for any bounded set of H0 , U=Œ0;T is in Cu .Œ0; T H1 /.
Comparison Theorem
Let U and V be in C . If U and V are a viscosity subsolution and a viscosity

supersolution of (6.127)–(6.128), respectively, then U V on Œ0; T H0 .
From this theorem, we deduce;

Theorem 6.7. Let F be in Cu .H1 /. Under (B10 ) and (B20 ), U.t; / D
VT t F ./.D V0 T t F .// is the unique viscosity solution of (6.127)–(6.128)
in C .
Proof. It is enough to show that U is in C . Since U satisfies (a) by (6.72), we only
need to prove (b).
Let ./ be given. Let u./ and uQ ./ denote the response for ./ with u.t/ D
Q respectively. We first consider the terminal cost. For " > 0, we have
and uQ .t/ D ,
EjF .u.T // F .Qu.T //j EmF .ku.T / uQ .T /k /

" C C" Eku.T / uQ .T /k
p
" C C" N k kQ (by (5.189)): (6.155)
For the running cost, (6.64) leads to

Z T Z T
E jr.u.s/; .s// r.Qu.s/; .s//j ds lr E ku.s/ uQ .s/k ds
t t
p
lr Q :
N T k k (6.156)
Thus, (6.155) and (6.156) show that U.t; / 2 Cu .H1 / uniformly on Œ0; T ,
namely there is a modulus m./, such that
Q m.k k
jU.t; / U.t; /j Q / 8t: (6.157)
Next we consider time continuity. Let u./ be the response for ./, with u.0/ D
. Since U.t; / D V0 T t F ./, we see that, for " > 0,
EjF .u.t1 // F .u.t2 //j EmF .ku.t1 / u.t2 /k /

p p
" C C" N3 kk .t1 t2 / (6.158)
by (6.24). Now (6.67) and (6.158) yield
sup jJ.t1 ; 0; I ./I F / J.t2 ; 0; I ./I F /j

./2
p
< " C c7 .1 C C" /kk jt1 t2 j; (6.159)
with a constant c7 independent of .

Now (6.158) and (6.159) conclude (b).
Example 6.1. Control of partially observable diffusion processes

Let us consider a controlled diffusion X and its observation process Y , according
to 5.5.1. For a policy (see Definition 5.5.1), X and Y evolve according to the SDEs
8
ˆ
ˆ dX.t/ D ˛.X.t/; .t; Y // dB.t/ C .X.t/; .t; Y // d W .t/
<
C b.X.t/; .t; Y // dt; t 2 .T0 ; T ; (6.160)
ˆ
:̂
X.T0 / D X0 ;
and
(
d Y .t/ D h.X.t/; .t; Y // dt C d W .t/; t 2 .T0 ; T ;
(6.161)
Y .T0 / D 0;
where B, W , and X0 are mutually independent and B and W are d0 - and m-

dimensional Wiener processes on .˝; Q FQ ; PQ /. The problem is to minimize the
payoff;
hZ T i
J.T0 ; X0 I / WD ET0 X0 l.X.t/; .t; Y // dt C f .X.T // ; (6.162)
T0
by choosing an appropriate policy . We denote the set of all policies by .

Recalling Proposition 5.9, we will formulate the problem as a control problem
with full observation by using the unnormalized conditional probability density,
given by the Zakai equation. Let ˇ be an m-dimensional Wiener process, defined
on .˝; F ; P /. Suppose that X0 has a probability density function 2 H0 . For
./ 2 ˇ , we consider the Zakai equation
8
ˆ X
m
< dq.t/ D A./ q.t/ dt C ./
Gk q.t/ dˇ k .t/; t 2 .T0 ; T ;
(6.163)
:̂ kD1
q.T0 / D ;
where
X
d hX
d X
d i
A D @i K ij .x; /@j b i .x; / @j K ij .x; / ;
i D1 i D1 j D1
with K D 12 .˛ ˛ > C > /, and
X
d X
d

Gk D ki .x; /@i C h .x; /
k
@i ki .x; / ; k D 1; : : : ; m:
i D1 i D1
We assume the smoothness condition .B10 / and uniform positive definiteness

condition for ˛ ˛ > :
y > ˛ ˛ > .x; /y 0 jyj2 ; 8y 2 Rd ; 8x; :
Further, we assume that l.; / 2 H0 with sup kl.; /k < 1 and f 2 H1 .

Since .t; ˇ/ is in ˇ and, for ./ 2 ˇ , we can take 2 so that .t/ D
.t; ˇ/ almost everywhere on Œ0; T ˝, Proposition 5.9 asserts
hZ T i
J.T0 ; X0 I / D ET0 .l.; .t//; q ./ .t// dt C .f; q ./ .T // ; (6.164)
T0
where q ./ ./ is the solution of (6.163), and J.T0 ; X0 I / depends on T0 , and .
Further, the value function V .T0 ; / WD inf2 J.T0 ; X0 I / is nothing but the
infimum of the RHS of (6.164) over ./ 2 ˇ . Since the terminal cost F . / WD
.f; / is in C.H1 /, Theorem 6.7 says that the value function V ./ is the unique
viscosity solution of the HJB equation
n 1X 2
m o

@t V .t; / C inf hA ; DV .t; /i C hD V .t; /Gk ; Gk i C .l.; /; /
2 2
kD1
D 0; t 2 .0; T /; 2 H1 ;
V .T; / D .f; /:

References
[A03] R.A. Adams, Sobolev Spaces, 2nd edn. (Academic, Amsterdam/Boston, 2003)
[BC09] A. Bain, D. Crisan, Fundamentals of Stochastic Filtering (Springer, New York/
London, 2009)
[Be52] R. Bellman, On the theory of dynamic programming. Proc. Nat. Sci. U.S.A. 38,
716–719 (1952)
[Be57] R. Bellman, Dynamic Programming (Princeton University Press, Princeton, 1957)
[Be75] V.E. Benes, Composition and invariance methods for solvinig some stochastic
control problems. Adv. Appl. Prob. 7, 299–329 (1975)
[BSW80] V.E. Venes, L.A. Shepp, H.S. Witsenhaussen, Some solvable stochastic control
problems. Stochastics 4, 39–83 (1980)
[Be92] A. Bensoussan, Stochastic Control of Partially Observable Systems (Cambridge
University Press, Cambridge/New York, 1992)
[BN90] A. Bensoussan, M. Nisio, Nonlinear semigroup arising in the control of diffusions
with partial observation. Stoch. Stoch. Rep. 30, 1–45 (1990)
[BP99] T.R. Bieleckii, S.R. Pliska, Risk sensitive dynamic asset management. Appl. Math.
Optim. 39, 337–366 (1999)
[BS73] F. Black, M. Scholes, The pricing of options and corporate liabilities. J. Polit. Econ.
81, 637–659 (1973)
[BM07] R. Buckdahn, J. Ma, Pathwise stochastic control problems and stochastic HJB
equations. SIAM J. Control Optim. 45, 2224–2256 (2007)
[Br06] S. Brendle, Portfolio selection under incomplete information. Stoch. Proc. Appl.
116, 701–723 (2006)
[CPY09] M.H. Chsng, T. Pang, J. Yong, Optimal stopping problem for stochastic differential
equations with random cofficients. SIAM J. Control Optim. 48, 941–971 (2009)
[CI90] M.G. Crandall, H. Ishii, The maximum principle for semicontinuous functions.
Differ. Integral Equ. 3, 1001–1014 (1990)
[CIL92] M.G. Crandall, H. Ishii, P.L. Lions, A user’s guide to viscosity solutions. Bull. AMS
NS 27, 1–67 (1992)
[DaPZ92] G. Da Prato, J. Zabczyk, Stochastic Equations in Infinite Dimenmensions
(Cambridge University Press, Cambridge, 1992)
[ElKPQ97] N. El Karoui, S. Peng, M.C. Quenez, Backward stochastic differential equations in
finance. Math. Finance 7, 1–71 (1997)
[ElKQ95] N. El Karoui, M.C. Quenez, Dynamic programming and pricing of contingent
claims in an incomplete market. SIAM. J. Control Optim. 33, 29–66 (1950)

and Stochastic Modelling 72, DOI 10.1007/978-4-431-55123-2
246 References
[E83] L.C. Evans, Classical solutions of Hamilton-Jacobi-Bellman equation for uniformly

elliptic operators. Trans. AMS 275, 245–255 (1983)
[E98] L.C. Evans, Partial Differential Equations. GSM19 (AMS, Providence, 1998)
[FH11] W.H. Fleming, D. Hernandez-Hernandez, On the value of stochastic differential
games. Commun. Stoch. Anal. 5, 341–351 (2011)
[FKSh10] W.H. Fleming, H. Kaise, S.J. Sheu, Max-plus stochastic control and risk sensitivity.
Appl. Math. Optim 62, 81–144 (2010)
[FMcE95] W.H. Fleming, W.H. McEneaney, Risk sensitive control on infinite time horizon.
SIAM J. Control Optim. 33, 1881–1915 (1995)
[FR75] W.H. Fleming, R.W. Rishel, Deterministic and Stochastic Optimal Control
(Springer, Berlin/New York, 1975)
[FSh99] W.H. Fleming, S.J. Sheu, Optimal long term growth rate of expected utility of
wealth. Ann. Appl. Prob. 9, 871–903 (1999)
[FSh00] W.H. Fleming, S.J. Sheu, Risk sensitive control and an optimal investment model.
Math. Finance 10, 197–213 (2000)
[FSh02] W.H. Fleming, S.J. Sheu, Risk sensitive control and an optimal investment model
II. Ann. Appl. Prob. 12, 730–767 (2002)
[FS06] W.H. Fleming, H.M. Soner, Controlled MarkovProcesses and Viscosity Solutions,
2nd edn. (Springer, New York 2006)
[FSo89] W.H. Fleming, P.E. Souganidis, On the existence of value function of two-plsyer,
zero-sum stochastic differential games. Indiana Math. J. 38, 293–314 (1989)
[FKK72] M. Fujisaki, G. Kallianpur, H. Kunita, Stochastic differential equations for the
nonlinear filtering problem. Osaka J. Math. 9, 19–40 (1972)
[GŚ00] F. Gozzi, A. Świech, Hamilton-Jacobi-bellman equations for the optimal control of
the Duncan-Mortensen-Zakai equation. J. Funct. Analy. 172, 466–510 (2000)
[HP81] J.M. Harrison, S.R. Pliska, Martingales and stochastic integrals in the theory of
continuous ytading. Stoch. Proc. Appl. 11, 215–260 (1981)
[HP83] J.M. Harrison, S.R. Pliska, Stochastic calculus model of continuous trading;
complete markets. Stoch. Proc. Appl. 15, 313–316 (1983)
[HNSh10] H. Hata, H. Nagai, S.I. Sheu, Asymptotics of the probability minimizing a down-
side risk. Ann. Appl. Prob. 20, 52–89 (2010)
[HS10] H. Hata, J. Sekine, Explicit solution to a certain nonELQG risk-sensitive stochastic
control problem. Appl. Math. Optim. 62, 341–380 (2010)
[IW81] N. Ikeda, S. Watanabe, Stochastic Differential Equations and Diffusion Processes
(North Holland, Amsterdam/New York, 1981)
[Is92] H. Ishii, Viscosity solutions for a class of Hamilton-Jacobi equations in Hilbert
spaces. J. Funct. Anal. 105, 301–341 (1992)
[I42] K. Itô, Differential equations determining Markov processes. Zenkoku Shijo Sug-
aku Danwakai 244, 1352–1400 (1942). (In Japanese)
[I51] K. Itô, On Stochastic Differential Equations. Memoirs of the American Mathemat-
ical Society, vol. 4 (AMS, New York City, 1951)
[JLL90] P. Jaillet, D. Lamberton, B. Lapeyer, Variational inequalities and the pricing of
American options. Acta Appl. Math. 21, 263–289 (1990)
[KS91] I. Karatzas, S.E. Sheve, Brownian Motion and Stochastic Calculus, 2nd edn.
(Springer, New York, 1991)
[KS98] I. Karatzas, S.E. Sheve, Methods of Mathematical Finance (Springer, New York,
1998)
[Ko04] S. Koike, A Biginner’s Guide to the Theory of Viscisity Solutions. MSJ Memoirs,
vol. 13 (JMS, Tokyo, 2004)
[KN02] K. Kuroda, H. Nagai, Risk sensitive portfolio optimization and infinite time horizon.
Stoch. Stoch. Rep. 73, 309–331 (2002)
[Kr09] N.V. Krylov, Controlled Diffusion Processes, 2nd edn. (Springer, Berlin, 2009)
[Ku67] H.J. Kushner, Dynamical equations for optimal nonlinear filtering. J. Differ. Equ. 3,
179–190 (1967)
References 247
[La83] S. Lang, Real Analysis, 2nd edn. (Addison-Wesley, New York, 1983)
[L83] P.L. Lions, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman
equations I. J. Commun. PDE. 8, 1101–1134 (1983)
[L88] P.L. Lions, Viscosity solutions of fully nonlinear second-order equations and
optimal stochastic control in infinite dimensions. Part I. The case of bounded
stochastic evolution. Acta Math. 161, 243–278 (1988)
[L89] P.L. Lions, Viscosity solutions of fully nonlinear second-order equations and
optimal stochastic control in infinite dimensions. Part II. Optimal control of Zakai
equation, in Stochastic Partial Differential Equations and Applications II, ed.
by G. Da Prato, L. Tubaro. Lecture Notes in Mathematics, vol. 1390 (Springer,
Berlin/Heidelberg, 1989), pp. 147–170. Part III. Uniqueness of viscosity solutions
for general second order equations. J. Funct. Anal. 86, 1–18 (1989)
[LN83] P.L. Lions, M. Nisio, A uniqueness result for the semigroup associated with HJB
equations. Proc. Jpn. Acad. 58, 273–276 (1983)
[LS01] R.S. Liptser, A.N. Shiryayev, Statistics of Random Processes I, II, 2nd edn.
(Springer, New York/Berlin, 2001)
[Ma00] C. Martini, American option prices as unique viscosity solutions to degenerate HJB
equations, Rapport de rech, INRIA, 2000
[Me71] R.C. Merton, Optimal consumption and portfolio rules in continuous time. J. Econ.
Theory 3, 373–413 (1971)
[Me73] R.C. Merton, Theory of rational option pricing. Bell J. Econ. Manag. Sci. 4,
141–183 (1973)
[M88] M. Metivier, Stochastic Partial Differential Equations in Infinite Dimensional
Spaces (Scuola Superiore, Pisa, 1988)
[My66] P.A. Meyer, Probability and Potentials (Blaisdell, Waltham, 1966)
[Mo10] H. Morimoto, Stochastic Control and Mathematical Modeling, Applications in
Economics (Cambridge University Press, Cambridge/New York, 2010)
[Na03] H. Nagai, Optimal strategies for risk sensitive portfolio optimization problems for
general factor models. SIAM J. Control Optim. 41, 1779–1800 (2003)
[N76] M. Nisio, On a nonlinear semigroup attached to stochastic optimal control. Pull.
RIMS 12, 513–537 (1976)
[N78] M. Nisio, On nonlinear semigroups for Markov processes associated with optimal
stopping. Appl. Math. Optim. 4, 143–169 (1978)
[N81] M. Nisio, Lecture on Stochastic Control Theory. ISI Lecture Notes, vol. 9 (McMil-
lan India, Delhi 1981)
[N88] M. Nisio, Stochastic differential games and viscosity solutions of Isaacs equations.
Nagoya Math. J. 110, 163–184 (1988)
[P79] E. Pardoux, Stochastic partial differential equations and filtering of diffusion
processes. Stochastics 3, 127–167 (1979)
[P93] E. Pardoux, Stochastic partial differential equations, a review. Bull. Sc. Math. 117,
29–47 (1993)
[PP90] E. Pardoux, S. Peng, Adapted solution of backward stoshastic equation. Syst.
Control Lett. 14, 55–61 (1990)
[Pe92] S. Peng, Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim.
30, 284–304 (1992)
[Ph02] H. Pham, Smooth solutions to optimal investment models with stochastic volatilities
and portfolio constraints. Appl. Math. Optim. 46, 55–78 (2002)
[R90] B.L. Rozovskii, Stochastic Evolution Systems (Kluwer Academic,
Dordrecht/Boston, 1990)
[S08] A.N. Shiryayev, Optimal Stopping Rules, 2nd edn. (Springer, Berlin/Heidelberg
2008)
[St11] L. Stettner, Penalty method for finite horizon stopping problems. SIAM J. Control
Optim. 49, 1078–1099 (2011)
248 References
[SV79] D.V. Strook, S.R.S. Varadhan, Multidimensional Diffusion Processes (Springer,

Berlin/New York, 1979)
[W71] J.C. Willems, Least squares stationary optimal control and the algebraic Ricatti
equation. IEEE. Trans. Auto. Control 16, 621–635 (1971)
[Wo68] W.M. Wonham, on a matrix Ricatti equation of stochastic control. SIAM J. Control
Optim. 6, 681–697 (1968)
[Y80] K. Yosida, Functional Analysis, 6th edn. (Springer, Berlin/New York, 1980)
[YZ99] J. Yong, X.Y. Zhou, Stochastic Controls, Hamiltonian Systems and HJB Equations
(Springer, New York, 1999)
Index
A D
adapted, 6 density-valued process, 200, 201
admissible control, 32, 118 differential games, 144
admissible strategy, 118 discrete-time DPP, 41, 102
admissible system, 100 discrete-time DP property, 44
almost surely, 4 DP property, 51
American option price, 112 DPP, 47, 107, 132, 228
approximation, 46, 106
E
B Elliott–Kalton strategy, 118
backward evolution operator, 10 Elliott–Kalton upper value, 121
backward stochastic differential equations, 24 Elliott-Kalton lower value, 120
Bayes formula, 18 energy equality, 170
Black–Scholes formula, 30 expectation, 2, 154
Brownian adapted control, 34, 50 exponential martingale, 14
Burkholder–Davis–Gundy inequality, 8, 164
F
C factor processes, 68
classical solution of HJB, 61 filtered probability space, 5
colored Wiener process, 159
Comparison Principle, 95
conditional expectation, 2, 155
conditional probability, 3 G
continuous martingale, 6, 156 generalized solution, 171
control process, 32 generator, 54, 110, 136, 232
control-stopping problem, 100 Girsanov transformation, 18
controlled Zakai equation, 211 Girsanov’s theorem, 19
convergence, 4
correlation operator, 159
cost function, 33 H
covariance operator, 159 HJB equation, 56, 60, 94, 112, 236
Crandall–Ishii Lemma, 89 H-process, 156
criterion, 33 H-random variable, 154

and Stochastic Modelling 72, DOI 10.1007/978-4-431-55123-2
250 Index
I SDEs with deterministic Borel coefficients, 22

in p-th mean, 4 selling price, 26
in law, 4 semigroup, 53, 109, 135, 230
in probability, 4 semilinear stochastic parabolic equations, 177
independent, 3 small noise, 144
Isaacs equation, 94, 148 solution, 16, 25, 169
Itô–Krylov formula, 14 square integrable, 157
Itô integrals, 10 square integrable martingale, 7
Itô’s formula, 13, 187, 191 stochastic differential equations, 15
stochastic differential games, 117, 120
L stochastic integral, 161
linear Gaussian quadratic regulator, 62 stochastic parabolic equation, 167
Lipschitz continuous SDEs with random stochastic process, 5
coefficients, 15 stopping time, 6
local martingale, 8, 156 strictly progressively measurable strategy, 121
logarithmic utility function, 70 Structural condition, 93
lower Isaacs equation, 138 superellipticity, 210
lower value, 120 superparabolicity, 172
lower value function, 131, 140 switching control, 41
system-observation SDEs, 203
M
Markov process, 10 T
Markovian policy, 34 Taylor formula, 186
mean, 2 time-homogeneous, 57
mean return process, 69 transition operator, 10
mean value theorem, 186 transition probability, 9
O U
optimal control, 33 unique viscosity solution, 242
optimal investment problems, 68 uniqueness, 99
unnormalized conditional expectation, 200
P upper Isaacs equation, 140
parabolic differentials, 80 upper value, 121
parabolic equation, 80 upper value function, 134
payoff, 33 utility function, 70
portfolio strategies, 26
power utility function, 70
probability space, 2 V
progressively measurable, 6 value function, 33, 134, 224
verification theorem, 60
viscosity solution, 81, 112, 139, 140, 237
Q viscosity solution of the HJB equation, 85, 236
quadratic variational process, 7, 157, 158 viscosity subsolution, 81, 84, 237
viscosity supersolution, 81, 84, 237
R volatility process, 69
random variable, 2
reference probability system, 32
regular conditional probability, 3 W
response, 33 weak solution, 23
risk sensitive stochastic controls, 141 Wiener process, 9
S Z
saddle point property, 123 Zakai equation, 203, 204

(Nisio) Stochastic Control Theory (2015)

Uploaded by

Copyright:

Available Formats

You might also like

(Nisio) Stochastic Control Theory (2015)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

(Nisio) Stochastic Control Theory (2015)

Uploaded by

Copyright:

Available Formats

Probability Theory and Stochastic Modelling 72

• Branching processes and other models of population growth

More information about this series at http://www.springer.com/series/13205

Stochastic Control Theory

ISSN 2199-3130 ISSN 2199-3149 (electronic)

Mathematics Subject Classification: 93E20, 60H15

© Springer Japan 2015

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

The purpose of this book is to provide an introduction to stochastic controls theory,

Kobe, Japan Makiko Nisio

1 Stochastic Differential Equations . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1

2.4.2 Investment Problems for Power Utility Function . . . . . . . . . . . . . 71

5.3.4 Semilinear Stochastic Parabolic Equations with

Let a and b be real numbers.

B.U / = the -field generated by all open subsets of U

Cp .˙/ D f 2 C.˙/I  is polynomial growingg, when ˙ is a Banach space.

H0 D L2 .Rd / with the usual inner product .; / and norm k k

M2c .Œ0; T ; .Ft /I H/ D set of continuous and square integrable H-valued

RHS D right-hand side

Abstract The purpose of this chapter is to overview elements of the theory

1.1 Review of Stochastic Processes

1.1.1 Random Variables

© Springer Japan 2015 1

X.!/ D Y .!/; 8! … N: (1.2)

When (1.2) holds, we write

X DY P -a.s. (almost surely): (1.3)

(2) Any U -valued random variable X induces a probability measure PX on B.U /

PX .B/ D P .X 1 .B//; 8B 2 B.U /:

PX is called the probability distribution (or law) of X .

Since EŒX I  is a signed measure on G, satisfying

the Radon–Nikodým theorem yields a unique (up to indentification) G-

Definition 1.1. is called the conditional expectation of X given G, and is denoted

E.E.X jG2 /jG1 / D E.X jG1 / P -a.s. (1.4)

P .X 2 AjG/ D P .X 2 A/ P -a.s.; 8A 2 B.Rd /; (1.5)

P ..X 2 A/ \ B/ D P .X 2 A/P .B/; 8A 2 B.Rd /; 8B 2 G:

(see [SV79], p. 13, [IW81], p. 14 for details).

Monotone Convergence Theorem

Suppose that Xn  0 and is increasing to X. 1/ P -a.s. Then

E.Xn jG/ % E.X jG/. 1/ P -a.s.

E.lim inf Xn jG/ lim inf E.Xn jG/ P -a.s.

Dominated Convergence Theorem

Suppose that jXn j Y , with integrable Y; n D 1; 2; : : : ; and Xn ! X P -a.s. Then

E.jXn X jjG/ ! 0 P -a.s.

When Y D C D const, we call this the bounded convergence theorem.

Let Xn ! X P -a.s. Suppose that Xn ; n D 1; 2; : : : are uniformly integrable given

sup E.jXn j.jXn j > c/jG/ ! 0 in prob: (1.6)

Then E.jXn X jjG/ ! 0 P -a.s.

1.1.2 Stochastic Processes

X.t; !/ D Y .t; !/; 8t; for ! … N:

P .X.t/ D Y .t// D 1; 8t:

For fixed !, the function t 7! X.t; !/ on Œ0; T  is called a sample path.

Definition 1.6. Suppose X W Œ0; T   ˝ 7! Rd is B.Œ0; T /  F =B.Rd /-

f!I .!/ tg 2 Ft ; 8t 2 Œ0; T :

For any stopping time ; F is defined by

F D fA 2 F I A \ . t/ 2 Ft ; 8t 2 Œ0; T g: (1.7)

F is a -field, which intuitively consists of information up to .

is an .Ft /-stopping time.

Cp .˙/ D f 2 C.˙/I is polynomial growingg, when ˙ is a Banach space.

M2c .Œ0; T ; .Ft /I H/ D set of continuous and square integrable H-valued

Since EŒX I is a signed measure on G, satisfying

Suppose that Xn 0 and is increasing to X. 1/ P -a.s. Then

sup E.jXn j.jXn j > c/jG/ ! 0 in prob: (1.6)

For fixed !, the function t 7! X.t; !/ on Œ0; T is called a sample path.

Definition 1.6. Suppose X W Œ0; T ˝ 7! Rd is B.Œ0; T / F =B.Rd /-

f!I .!/ tg 2 Ft ; 8t 2 Œ0; T :

F D fA 2 F I A \ . t/ 2 Ft ; 8t 2 Œ0; T g: (1.7)

(E.X.t/jFs / X.s/, or E.X.t/jFs / X.s/ P -a.s. 8s t, respectively).

Let Qn be the exit time of X from Œn; n. Then n WD n ^ Qn is increasing to

Ts .Tst /.x/ D T t .x/ for s t;

Thus, I.˚/ 2 M2c .Œ0; T ; .Ft // and satisfies

EŒI.˚/.t/ D 0; EŒI.˚/.T /2 D k˚k2T ;

Step 2. For ˚ 2 L2 .Œ0; T ˝I .Ft //, there is a sequence of simple processes

EŒjI.˚n /.T / I.˚m /.T /j2 D EŒjI.˚n ˚m /.T /j2 ! 0: (1.12)

Step 3. For ˚ 2 L2.Ft / .Œ0; T /, we define

LetR F be a function of C 12 .Œ0; T Rd / and be an .Ft /-stopping time. Then

y > a.t; !/y 0 jyj2 ; 8y 2 Rd ; 8t!:

where ˚ 2 L2 .Œ0; T ˝; .Ft /I Rm /.

X./ D X .2 Lp .˝; F I Rd / with p 1/: (1.20)

where B; Aj W Œ0; T ˝ 7! Rd ˝ Rd , b0 ; ˛j W Œ0; T ˝ 7! Rd , j D 1; : : : ; m