Modeling Driver, Driver Support,
and Cooperative Systems with
Dynamic Optimal Control
Serge Hoogendoorn, Raymond Hoogendoorn, Meng Wang,
and Winnie Daamen
Previous work proposed an optimal control framework for modeling
driver behavior. Drivers were assumed to minimize the predicted sub-
jective effort of their control actions, taking into account the anticipated
actions of other drivers. The framework was generic. Several assumptions
and simplifications had to be made; this factor hampered the applicability
of the framework. One of these assumptions was that the behavior of
other vehicles in the flow was stationary during the prediction horizon.
Furthermore, the resulting model was computationally complex. A new
approach based on the generic optimal control framework is proposed
for modeling and computing driving behavior. The model can deal with
the dynamics of the vehicles to which a driver reacts. At the same time,
the computational complexity is small and does not increase exponentially
with the complexity of the prediction model or with the size of the control
vector. The mathematical solution approach is presented and illustrated
with several examples. Face validity of the model is shown, and an appli-
cation of the theory in the field of automated vehicle guidance is dis-
cussed. In particular for these applications, the proposed optimization
approach allows for the computation of cooperative driving strategies
that minimize a generic range of objective functions. The improvements
in performance made by cooperation are substantial, as illustrated by
several examples.
Mathematical models of longitudinal driving tasks (e.g., speed choice
and car following) and lateral driving tasks (e.g., lane changing,
merging, and overtaking) describe and predict traffic flow operations.
This paper focuses on the mathematical modeling of car following by
use of an optimal control framework.
Theory and Models
Car Following
Existing car-following models can be divided into three broad classes
(1): safe-distance models (2, 3), stimulus–response models (4, 5),
and psycho-spacing models (6, 7). Safe-distance models assume
Department of Transport and Planning, Faculty of Civil Engineering and Geo­
sciences, Delft University of Technology, Stevinweg 1, P.O. Box 5048, 2600 GA Delft,
Netherlands. Corresponding author: S. Hoogendoorn, s.p.hoogendoorn@tudelft.nl.
Transportation Research Record: Journal of the Transportation Research Board,
No. 2316, Transportation Research Board of the National Academies, Washington,
D.C., 2012, pp. 20–30.
DOI: 10.3141/2316-03
20
that drivers choose their headway to avoid collisions, and stimulus–
response models describe the reaction of drivers as a function of, for
example, relative speed.
Safe-distance and stimulus–response models, however, are rather
mechanistic; the only human element in most of these models is a
finite reaction time. In the real world, a driver cannot adequately
observe a stimulus lower than given values (perceptual thresholds),
cannot precisely evaluate a situation to determine the required response
because of observation errors, and cannot precisely manipulate the gas
and brake pedals. Furthermore, because of the need to distribute
his attention among different tasks, a driver will not be permanently
occupied with the car-following task. These considerations have
inspired a different class of car-following models, the psycho-spacing
models. Michaels (6) and Leutzbach and Wiedemann (7) provided the
basis for the first psycho-spacing model by using theories borrowed
from perceptual psychology. In these models, car-following behavior
is described on a relative speed–headway distance plane. Since their
introduction, interesting new approaches have been proposed (8–11).
Lane Changing
The first theoretical framework on lane changing was developed by
Gipps, who proposed a structure that leads to the decision of whether
a driver will change lanes (12). According to the framework, this
decision depends on, for example, the location of permanent obstruc-
tions, the presence of transit lanes, and the distance from the driver’s
intended turn.
More recently, Kesting et al. proposed a theory in which lane
changes depend on the possibility for a driver to improve accelera-
tion (13). Their generic approach presented by utility functions for
the different lanes also incorporates a politeness factor. This factor
represents the effect a lane change would have on the acceleration
of other drivers.
Choudhury presented a decision model for drivers with latent plans
(not observable plans related to the chosen target or tactic of the driver)
(14). In this framework, separate models are given for freeway lane
selection, freeway merging, and lane selection on urban arterial
roads. The latent plan for a merging vehicle consists of three phases
(normal state, courtesy merging state, and forced merging state),
connected through a decision tree. A vehicle starts in the normal state
and can transit to other states under certain conditions. Each state has
its own gap acceptance characteristics. In the normal state, the exist-
ing gaps are considered and evaluated if they are acceptable, where
acceptable is defined as larger than the critical gap. The critical
Hoogendoorn, Hoogendoorn, Wang, and Daamen
gaps are assumed to follow lognormal distributions, the mean gap
a function of explanatory variables.
Toward a Joint Modeling Approach
Hoogendoorn and Hoogendoorn proposed a different approach, in
which car following and lane changing can be considered jointly in
a single decision framework (15). That work built on the work of
Hoogendoorn and Bovy (16) and resembled the work of Toledo
et al. (17). In comparison with the latter work, in which discrete
choice theory was used to compare short-term plans, the framework
proposed in the present paper describes car following and lane
changing as an optimal control task. The main theoretical assump-
tion is that drivers minimize the predicted subjective effort of their
control actions, taking into account the anticipated actions of other
drivers. With a control action decided on, the control will be applied
at (possibly nonequidistant) discrete time instants tk, similar to the
approach of Wiedemann (18). The framework provides a mathe-
matical formalization of the theory put forward by Stanton (19) and
Minderhoud (20).
The work presented in this paper extends earlier work (15). Some
of the major assumptions that were needed to derive car-following
laws are removed (i.e., the assumption that during the prediction
period, a driver assumes that the acceleration of all other drivers is
constant and equal to zero). Furthermore, to calculate optimal control
laws, the earlier approach required solving an n + 1 dimensional
partial differential equation, where n is equal to the number of ele-
ments in the vector describing the state of the considered driver
(usually two times the number of vehicles to which the driver reacts).
This leads to a computationally complex (and for large states, even
unfeasible) problem.
Both issues are resolved in this paper. The resulting mathematical
problem and solution algorithm can be solved efficiently and for more
complex, dynamic situations. That is, the main contribution of the
paper is not in proposing yet another car-following paradigm, but
rather in showing how the previously proposed approach can be
generalized and numerically solved in a computationally efficient
way. The generality and computational efficiency of the approach
is such that the framework can be used for human drivers and for
joint optimization of the cooperative car-following behavior in
a platoon.
First, the main theoretical assumptions used to derive the model are
described. Then the optimal control formulation for the car-following
task is formulated, and several applications of the approach are given.
Finally, application of the framework to driving support systems
(autonomous and cooperative) is exemplified.
Driving and Effort Minimization
Several authors have proposed a utility (or cost) optimization model to
conceptually model execution of the driving task (19, 20). Examples
of use of optimal control theory to model the driving tasks are also
available (21). In particular, this approach has been adopted to accu-
rately describe driving behavior (22, 23). The driving process, its
parameters, and the resulting control decisions will differ between
drivers because of differences in control objectives, preferences,
vehicle characteristics, and so forth. Differences in driving style are
reflected by preferences for desired driving speed, desired comfortable
deceleration levels, and minimum gap distances (e.g., some drivers
21
give priority to safe driving, whereas others prefer driving at a high
speed, accepting smaller headways and an increased risk).
Driving Objectives
It is assumed that the objective of drivers is a subset of the following:
1. To maximize safety and minimize risks,
2. To minimize lane changing maneuvers,
3. To maximize travel efficiency (restricting deviations from the
desired driving speed),
4. To maximize smoothness and comfort, and
5. To minimize stress, inconvenience, fuel consumption, accel-
erations, and decelerations.
The significance of each of these objectives will differ among indi-
viduals, who are limited by the capabilities of their vehicles, such
as maximum speed and braking.
An operational model for driving behavior that is based on these
conceptual notions is derived in this paper. Effectively, this implies
that the control objectives are formalized into a cost function J (pre-
dicted disutility). It is assumed that a driver tries to minimize this
disutility.
Behavioral Assumptions
Underlying the mathematical model is a behavioral theory that
describes how drivers are assumed to act and to react to other drivers
on the road (referred to as the opponents). These assumptions pertain
to the way drivers scan their environment and act on their observa-
tions, given their control objectives. The following is an overview
of the main behavioral assumptions:
1. Drivers minimize the generalized predicted costs of their control
actions. These generalized costs reflect the different (and subjective)
control objectives (e.g., risk, travel time, stress, smoothness).
2. Drivers reconsider their operational control decisions at
(possibly nonequidistant) discrete time instants tk based on
– Observations of the prevailing traffic conditions and
– Predictions of the impact of their control actions, possibly
including the expected reactions of other drivers (Item 4).
3. Drivers anticipate the behavior of other drivers by predict-
ing their driving behavior. However, drivers have limited prediction
possibilities. This is reflected by discounting the costs over both time
and space.
4. Drivers will minimize the predicted cost stemming from
– Not driving at the desired state, described by, for example, the
free speed, the desired distance, and the desired lane;
– Driving too close to other drivers;
– Accelerating and braking; and
– Changing lanes (lane switching costs).
5. Observing, decision making, and maneuvering take time and
are prone to errors.
6. Control objectives may change over time (adaptation effects).
All these behavioral assumptions are supported by empirical
evidence. However, it is beyond the scope of this paper to provide
a complete overview of all literature in favor of these assumptions.
The focus here is on the assumption of minimizing effort and how
this can be formalized to yield a microscopic driver behavior model.
22 Transportation Research Record 2316

Driving as Optimal Control Cycle where

It is assumed that decisions to adapt speed or to change lanes are t= time instant,
made at discrete time instants tk with k = 1, 2, . . . , K. Assume that vi(t) = speed of vehicle vi at t,
at each of these instants tk, driver i has information on the positions vi−1(t) = speed of vehicle vi−1 at t,
ui(t) = acceleration at t of driver i,
and speeds of all cars influencing his behavior. This information stems
ui−1 = acceleration of leader, and
from (erroneous) observations made by the driver, used in conjunction
vj, rj = speed and position of driver j = i − 1, i.
with the experience of the driver, mental model, and so forth. From this
conjunction an estimate of the current state of the system is formed. It is assumed that from the perspective of driver i, his state is fully
The estimate of the state of the system at tk is used as the starting described by the distance headway si and the relative speed Δvi.
conditions for the (conditional) predictions of the evolution of the
system.
It is assumed that the driver has a mental model that allows Car-Following Cost Formalization
him to make such a prediction, given the potential control decision
(longitudinal acceleration or changing lanes). He will then choose For the car-following task, the cost J of car following is described,
the control decision that minimizes the predicted effort (or cost or given that the acceleration ui = {ui(t)| t ≥ tk} is applied, with the
disutility) and apply this control. This decision is reconsidered when following function:
new information regarding the state of the system becomes available tk + T

(i.e., at tk+1). J ( t k , x ( t k ) ui ) = ∫ e − ηt L ( t , x ( t ) , u ( t )) dt
t = tk

Car Following as Deterministic
Optimal Control
The behavioral assumptions discussed in the previous section are
used to derive a mathematical modeling framework. First, a generic
car-following model is derived. The car-following task is formulated
as a deterministic optimal control problem to show the mathematical
model and discuss how the model can be solved with Pontryagin’s
minimum principle (24).
Mental Model for State Predictions
A driver is assumed to use an internal mental model for determin-
ing appropriate control decisions. Experience and knowledge have
given the driver skills for performing the driving task, therefore
the assumption of subjective optimal behavior appears justified.
Optimization, however, includes the limitations of the driver for
observation, information processing, internal state estimation, and
processing times and reaction times. Moreover, the process includes
the driving strategy.
For the car-following task, the following kinematic equations,
describing the dynamics of the distance si (t) of driver i at instant t,
are used:
+ e − η( tk +T )φ ( t k + T , x ( t k + T )) (4)
where
u = acceleration;
T = terminal time (i.e., end of planning period in optimization
process);
x = state of system from perspective of driver i, which in this
case equals distance si and relative speed Δvi;
ui = control (acceleration and braking);
L = running costs describing cost per time unit incurred during
infinitesimal period [t, t + dt);
η > 0 = discount factor (25), which describes that future costs
have less weight than current costs; and
ϕ = terminal cost or cost at end of control horizon.
The cost function J(tk, x(tk)) describes the expected cost given
the current state of the system x(tk) (as perceived by the driver), the
control actions of the driver, and the evolution of the system, start-
ing from time tk onward. At time tk+1, the decision will be reassessed
with the most current observations or estimates of the system state
(at time tk+1). Compared, for instance, with the work of Burnham et al.
(21), here nonquadratic running cost specifications are allowed, which
permits inclusion of more-realistic car-following behavior.
The rolling horizon optimal control problem is described by the
following mathematical program:

d u[*tk ,tk + T ) = arg min J ( t k , x ( t k ) ui ) (5)

si ( t ) = ∆vi ( t ) = vi −1 ( t ) − vi ( t ) (1)
dt
subject to
and
d
x = f (t, x, u) with x ( t k ) = x k (6)
d dt
∆vi ( t ) = ui −1 ( t ) − ui ( t ) (2)
dt with x = (si, Δvi) and f determined by Equations 1 and 2; that is,
subject to the initial conditions  vi −1 ( t ) − vi ( t )
f = 
si ( t k ) = ri −1 ( t k ) − ri ( t k ) ui −1 ( t ) − ui ( t )

and For notational simplicity, in the remainder of the paper tk = 0

without loss of generality. Furthermore, the index i is dropped from
∆vi ( t k ) = vi −1 ( t k ) − vi ( t k ) (3) the notation.
Hoogendoorn, Hoogendoorn, Wang, and Daamen 23

Solving Optimal Control Problem 4. Solve the costate dynamic equation

with Pontryagin’s Minimum Principle
d (i ) ∂H ∂φ
A well-known approach for solving the optimal control problem −
dt
λ =
∂x
( ( ) )
t, x (i ) , u* t, x (i ), Λ(i −1) , x (i ) λ (i ) ( T ) =
∂x
( )
T , x (i ) ( T )
is the minimum principle of Pontryagin (24). The approach entails (12)
defining the Hamiltonian H as follows:
backward in time.
H ( t , x , u, λ ) = L ( t , x , u ) + λ i f ( t , x , u ) (7) 5. Smoothly update the effective costates Λ(i):

Λ(i ) = (1 − α ) Λ(i −1) + αλ (i ) (13)

with L = e−ηtL and where λ = λ (t) denotes the so-called costates or
shadow costs of the state x. These costs reflect the relative extra cost
in making a small change δx on the state x(t).
The Hamiltonian is used to derive the following necessary condition
for the optimal control u*(t) (the stationarity condition):
H ( t , x , u*, λ ) ≤ H ( t , x , u, λ ) ∀u (8)
6. If ∈ = ||Λ(i) − λ(i)|| < ∈max, then stop; otherwise set i: = i + 1 and
go to Step 3.
The key is in choosing α such that the scheme will converge and
will converge as fast as possible. Although important, a thorough
investigation into good choices for α is beyond the scope of this paper.

In nearly all cases, this requirement will allow the optimal control
u* to be expressed as a function of the state x, the costate λ, and the Examples
time t.
Furthermore, for the costates λ, the costate dynamics can be This section illustrates the workings of the approach with simple
determined as car-following examples. This section does not develop better car-
following models per se; the objective is to show some of the key
d ∂H ∂L ∂f features of the approach.
− λ= = +λi (9)
dt ∂x ∂x ∂x
Model Verification
subject to the terminal conditions at t = T (i.e., at the end of the
control horizon), For all examples, the model has been (loosely) verified to ensure that
it acts according to expectations. This was also achieved by comparing
∂φ the performance J of the controller to a simple car-following law
λ (T ) = ( T , x ( T )) (10) u = c1 (x2 − s*) + c2x2. For all tested values of c1 and c2, it turned
∂x
out that the proposed control law yielded a substantially better per-
formance than the simple feedback control law, indicating that the
with φ ( T , x ( T )) = e − ηT φ ( T , x ( T )) .
optimization procedure functions well and optimal car-following
The remaining problem is to solve this set of ordinary differential laws are correctly computed.
equations. The main issue is that although for the state x the initial
conditions x(0) = x0 are known, for the costate λ, the terminal condi-
tions λ(T) = λT at t = T are known. This makes the resulting set of Basic Car-Following Behavior
differential equations difficult to solve.
The state of the system is defined as before, that is, x = (s, Δv). The
state dynamics are given by the simple kinematic equations shown
Iterative Numerical Solution Algorithm before. It is assumed that the considered driver predicts the dynamics
of State and Costate Equations of the system by making some assumption on speed profile of the
leader during the prediction horizon. This prediction is not neces-
A new solution approach is used to solve the preceding problem. sarily accurate compared with the realized profile; it is the expected
This approach is based on iteratively solving the state dynamic profile on which the considered driver (the follower) will anticipate
equation forward in time and subsequently solving the costate and base his response. However, for illustration, it is assumed that the
equation backward in time. The following program summarizes the follower has perfect knowledge for the considered planning horizon.
algorithm: The running cost is defined as follows:

1. Choose 0 < α < 1, set the iteration number i = 1, and set the 1 β β
L = u 2 + 1 ( x1 − s * ) + 2 x 22
2
(14)
stopping criterion ∈max. 2 2 2
2. Set Λ(0)(t) = 0 for 0 ≤ t ≤ T.
3. Solve the state dynamic equation where u is the acceleration of the follower; η = 0, that is, cost dis-
counting is not considered for now. For simplicity, ϕ = 0 was chosen
d (i )
dt
( ( ))
x = f t, x (i ), u* t, x (i ), Λ(i −1) , x (i ) ( t k ) = x k (11) as the terminal cost.
The running cost specification shows that there are three cost
components, between which there will be some trade-off: (a) the
forward in time. driver aims to minimize acceleration, (b) the driver aims to maintain
24 Transportation Research Record 2316

a distance headway close to some desired distance headway s*, and This completes the specification of the model. Next, the model is
(c) the driver aims to minimize the relative speed. The parameters tested in a specific test case scenario. In this scenario, it is assumed that
β1 and β2 describe, respectively, the cost weights of the cost of not a driver is following a vehicle at a distance of 100 m and with a relative
driving at the desired distance and the cost of having a nonzero speed speed of 0 m/s2. The desired distance headway is assumed to be 75 m.
difference with the leader. As will be shown, these weights strongly The follower expects its leader to decelerate with an acceleration of
determine the behavior of the model. −3 m/s2 between 1 and 3 s and to accelerate again between 5 and 7 s
For this simple running cost function, the stationarity condition with an acceleration of 2 m/s2. The prediction horizon is 10 s.
(Equation 8) can be used to find the following expression for the First considered is the resulting behavior for β1 = 0.1 and β2 = 1.
optimal control: Figure 1 shows the resulting behavior of the follower. The evolution
of the state x and the costates λ is shown in Figure 1, a and b, and
u* = λ 2 (15) the resulting speeds of the leader and the follower, and the applied
accelerations, are shown respectively in c and d.
In other words, the optimal control law, minimizing the objective In this case, the driver makes a trade-off between driving at the
function, equals the marginal cost of State 2 (the relative speed with desired distance s* and maintaining a relative speed around zero while
respect to the leading vehicle). also not allowing for accelerations that are too large. The resulting
It is now easily shown that the following costate dynamics are behavior is very smooth. The computation scheme converged quickly,
obtained: in just a few iterations (about 400 with a computation time of less
than 0.25 s).
Figure 2 shows the same experiment but now with β2 = 100. In this
d ∂H
λ1 = − = − β1 i ( x1 − s * ) (16) case, the importance of the relative speed is much greater than the
dt ∂x1 other two cost components. As a result, the driver is mainly trying to
minimize the differences in speed relative to the leading vehicle at
and the expense of not maintaining the desired distance headway or not
applying high accelerations.
d ∂H For the final example shown in Figure 3, β1 = 1 and β2 = 1, meaning
λ2 = − = − β 2 i x 2 − λ1 (17)
dt ∂x 2 that the desired distance headway is now much more important. In this
case, the driver mainly cares about maintaining the desired distance
Since ϕ = 0, one gets λ1(T) = λ2(T) = 0. headway s* and cares far less about the other cost components.

(a) (b)

(c) (d)

FIGURE 1   Example solution of Optimal Control Problem 1 with  1 = 0.1 and  2 = 1.
 (a) (b)

(c) (d)

FIGURE 2   Example solution of Optimal Control Problem 1 with  1 = 0.1 and  2 = 100.

(a) (b)

(c) (d)

FIGURE 3   Example solution of Optimal Control Problem 1 with  1 = 1 and  2 = 1.

26 Transportation Research Record 2316

Alternative Car-Following Model
Another example uses the desired speed v0 of the driver instead of
the speed difference with the predecessor. The following running
cost specification is used:
Figure 4 shows the predicted response of the driver if β1 = 0.1 and
β2 = 1. The driver decides to reduce his speed, thereby incurring the
cost of not driving at the desired speed and applying acceleration,
to increase the distance headway. The driver does this in such a way
that overall average costs are minimized.

1 β β
L = u 2 + 1 ( x1 − s * ) + 2 ( v 0 − vl + x 2 )
2 2
(18)
2 2 2 Toward Realistic Car-Following Model

Here v0 − vl + x2 = v0 − vf, where vf is the speed of the follower and Example specifications have been shown for the running costs that
vl is the speed of the leader. lead to different car-following behaviors. The goal was not to come
Again, u* = λ2. For the costate dynamics one gets up with a validated car-following law, but the results show that the
generated car-following behavior is face valid. The results shown
here show the intended car-following behavior of a driver for the con-
d ∂H
λ1 = − = − β1 i ( x1 − s * ) (19) sidered prediction horizon [tk,tk + T). Because the driver will reassess
dt ∂x1 the control decisions before the end of the prediction horizon T, based
on current situations and observations, the control signal most likely
and will not be fully executed. This means that the feedback loop that is
considered within the modeling approach ensures that, for instance,
d ∂H
λ2 = − = − β2 i (v 0
− vl + x 2 ) − λ 2 (20) inaccurate predictions, which are likely because the follower has
dt ∂x 2 limited prediction capabilities, will not be as harmful as in a situation
in which only feedforward control is applied.
since ϕ = 0, λ1(T) = λ2(T) = 0. Accurate modeling of the driving task requires careful specification
The example considers a driver who is initially following at a of the state x, the state dynamics f, and the running cost L. Furthermore,
distance of 50 m, whereas his desired distance is 75 m. The desired calibration and validation requires careful tuning of the model param-
speed of the driver is equal to the initial speed of the driver and of eters (e.g., weights in the running cost). The framework presented in
the leader, namely, 30 m/s. The prediction horizon T is again 10 s. the earlier work and generalized here to include the time component

(a) (b)

(c) (d)

FIGURE 4   Example solution of Optimal Control Problem 2 with  1 = 0.1 and  2 = 1.
Hoogendoorn, Hoogendoorn, Wang, and Daamen
is, however, flexible and generic (15). This allows relatively straight-
forward inclusion of factors deemed relevant in modeling of driving
behavior, such as consideration of multiple leaders and indifference
bounds.
Generalization of the framework to multiple lanes was discussed
in the earlier work (15); the same approach can be applied here, with-
out modification. The generalization entails computing the expected
optimal cost for driving on different lanes, given the predicted traf-
fic situation on these lanes. A lane change is performed when the
expected cost of the target lane is higher than the expected cost on the
current lane, with consideration of some nonnegative lane switching
cost (15).
Application to Advanced Driver
Assistance Systems and Cooperative
System Design
This paper proposes a new approach for modeling car-following
behavior that is based on behavioral assumptions supported by
literature. The idea is that drivers aim to minimize the generalized
predicted cost of their control actions. The cost reflects different
control objectives, as illustrated by the examples in the previous
section.
Compared with earlier work by the authors, this paper proposes
a solution approach that allows expected dynamic changes to be
included. That is, if a driver expects a certain dynamic speed profile of
the leading vehicle, or if he expects that the leader will change lanes or
leave the freeway by using an off-ramp, the driver will include these
predictions in choosing a car-following (and lane changing) strategy.
Because human drivers are generally assumed to be good at antici-
pating traffic conditions but are known to be poor responders, this
generalization is expected to substantially improve modeling of car
following.
The proposed approach, however, has other possible applications.
The proposed framework, the optimal control formulation, and the
proposed solution algorithm can be applied to the design of automated
vehicle guidance, advanced driver assistance systems, and driver
support systems, both for autonomous systems (such as adaptive
cruise control) and for cooperative systems.
Application to Adaptive Cruise Control Design
Wang et al. used optimization to design an adaptive cruise control
system, called the Eco-ACC system, that regulates longitudinal
vehicle behavior to minimize fuel consumption or carbon dioxide
emissions while maximizing travel efficiency, safety, and driving
comfort (22).
The algorithm they presented is computationally complex, and
the approach is unable to easily capture the dynamics of the leading
vehicle in the approach. This holds in particular when the approach
will be extended to control multiple vehicles in the flow jointly.
The solution approach discussed here can be applied straightaway
to solving these issues, yielding an algorithm that not only can include
the expected dynamics of the leader vehicle (e.g., known to the fol-
lowing vehicle through vehicle–vehicle communication) but that also
is sufficiently efficient computationally to be used online. The flex-
ibility in choosing the control objective allows the designer to specify
multiple objectives (efficiency, safety, comport, livability, etc.) and
the trade-off between these objectives.
27
Application to Design of Cooperative Systems
The application of the approach is not restricted to the design of auton-
omous controllers. The flexibility in the state and state dynamics
definition allows definition of a joint control problem, in which mul-
tiple vehicles in the flow are simultaneously controlled, such that
some joint objective is optimized.
In an example platoon of three vehicles, the leader vehicle is a
human driver whose behavior is exogenous. The two following
vehicles are automated vehicles that can exchange information and in
doing so cooperate to optimize some joint objective function J.
The framework presented earlier for modeling car-following
behavior can be used to describe such a system mathematically.
It is assumed that one overall objective J needs to be optimized
and that the two following vehicles will cooperate. The resulting
challenge is thus to find which control law u* will yield this optimal
behavior.
The state x of the system is defined by the distance headways
and the relative speeds, in line with the preceding examples: x =
(s1, s2, Δv1, Δv2). Here, sj is the distance between vehicle j and j + 1,
and Δvj is the relative speed of vehicle j with respect to vehicle j + 1.
The state dynamics are easily derived and equal to
 x3 
 
d  x4 
 = f (t, x, u)
x= (221)
dt
 al − u1 
 
 u1 − u2 
where u1 and u2 are the accelerations of the first and second followers
and where al is the acceleration of the leader.
For the cost specification, a specification similar to the first example
presented in this paper is used:
1
2
β
L = ( u12 + u22 ) + 1
2
(( x − s * ) + ( x − s * ) ) + β2 ( x
1
2
2
2 2 2
3 + x 42 ) (22)
This running cost specification shows that the aim is to minimize
the joint cost of acceleration, not driving at the desired distance
headway, and having a nonzero relative speed. The weights β1 and
β2 again reflect the relative importance of the different factors. It is
also assumed that ϕ = 0.
These specifications can be used to define the Hamiltonian H
and use it to derive the optimal control and the costate dynamics.
For optimal control, the stationarity conditions ∂H/∂u = 0 are used
to find
u1* = λ 3 − λ 4 (23)
and
u*2 = λ 4 (24)
Equation 23 shows that Follower 1 determines his control action
based on his own marginal cost of the relative speeds, as well as on the
marginal cost of Follower 2. Not being able to improve the situation
for Follower 1, Follower 2 considers only his own marginal cost of the
relative speed. The optimal control law of the first follower captures
the cooperative nature of the controller.
28 Transportation Research Record 2316

Derivation of the costate equations is straightforward and yields Figure 5 shows the results. As expected, the behavior of Follower 1
the following equations: is the same as in the previous example. Follower 2 reacts on the
behavior of Follower 1 and optimizes the situation accordingly.
d ∂H In the cooperative case, the situation is quite different. In the non-
λj = − = − β1 i ( x j − s * ) for j = 1, 2 (25)
dt ∂x j cooperative case, Follower 1 initially increases speed substantially
to decrease the distance headway from 100 m as quickly as possible;
and this decrease is now substantially less. The reason to do this is to
ensure that Follower 2 can maintain a distance headway, which is
d ∂H closer to the desired distance headway of 75 m, and the incurred
λj = − = − β2 i x j − λ j−2 for j = 3, 4 (26)
dt ∂x j costs are thus (relatively) small. Figure 6 shows these results.
Computation of the value of the objective function shows the
This completes the model specification for the cooperative system difference. In the cooperative case, the total cost is equal to 284.3.
formulation. The main difference is in the specification of the optimal In this case, Follower 1 incurs a cost of 143.9, and Follower 2 incurs
control law for the first follower, which takes into account the cost a slightly higher cost of 140.3. In the noncooperative case, the total
incurred by the second follower. cost equals 294.3, where Follower 1 incurs a cost of 133.9, while
The functioning of this controller can be shown with a simple Follower 2 incurs a cost of 160.4. The cooperative case yields a
example. Again, β1 = 0.1 and β2 = 1. The acceleration behavior of substantial improvement, in particular for Follower 2.
the leader is equal to the behavior in the first example. Initially, both More substantial improvements are achieved when the conditions
followers have a distance headway of 100 m and relative speeds of for Follower 2 are worse. Suppose that the initial distance headway of
1 m/s. The desired distance headway equals 75 m. Follower 2 is 150 m, and all other conditions are the same. Applying
In the noncooperative case, Follower 1 does not consider the noncooperative control provides a total cost of 1687.3, and Follower 1
incurred cost of Follower 2, and Follower 2 is convinced to optimize still incurs a cost of 133.9, while Follower 2 now incurs a cost of
his performance on the basis of the behavior of Follower 1. 1553.4. In the cooperative case, the total cost can be reduced to 1486.6;

(a) (b)

(c) (d)

FIGURE 5   Example solution of optimal control problem with  1 = 0.1 and  2 = 1 for noncooperative system.
Hoogendoorn, Hoogendoorn, Wang, and Daamen
(a)
(c)
FIGURE 6   Example solution of optimal control problem with  1 = 0.1 and  2 = 1 for cooperative system.
the first follower now incurs a cost of 261.1, while Follower 2 has a
substantially reduced cost of 1225.6.
As the example shows, the cooperative approach can be applied
successfully to improve the total conditions for the platoon. Because
the approach is generic, many types of objective functions can be used,
including livability and sustainability (22).
The approach yields results sufficiently quickly that it can be applied
in a real-time setting. Compared with the situation in which the control
is one-dimensional (only the behavior of a single follower is consid-
ered), the computational performance of the situation in which the
control is two-dimensional is only slightly worse. The approach thus
appears to be applicable for simultaneously controlling multiple
vehicles in a traffic stream in a cooperative way.
Conclusions and Future Work
This paper proposed a new approach to modeling driving behav-
ior based on the generic optimal control framework. The model
is dynamic and the computational complexity is relatively small,
compared with previous approaches. Several application examples
verified the approach and illustrated its function. These applications
showed that the resulting car-following models are face valid. Various
extensions of the modeling approach were also discussed.
29
(b)
(d)
An important application of the optimal control framework is in
automated guided vehicles or driving support systems, both autono-
mous and cooperative. The proposed algorithm is used to compute
the control signals, and then optimal control laws can be derived to
optimize a range of objective functions. The algorithm is sufficiently
efficient to be applied in real time, allowing for applications in real-life
controllers. This holds for the simultaneous and cooperative control of
multiple vehicles in the flow, which is a promising application for the
approach, as shown by the examples in the paper.
This paper focused on the generic framework, rather than on
application of the framework to specific situations. Future research
will apply the framework to designing realistic car-following strat-
egies, calibrated and validated with empirical data. This research
entails specifying the required state, the state dynamics, and, espe-
cially, the running cost function. Also, the approach for modeling
lane changes needs further specification.
Along with modeling of car following, application of the frame-
work to optimal cooperative car following will be investigated in
the future. The examples shown here are promising and justify further
research. In particular, the resulting computational complexity appears
to be only slightly worse in the case of joint control of multiple
vehicles, compared with isolated control. Detailed analysis of this
computational complexity, however, was beyond the scope of this
paper but will be made in the future.
30
Acknowledgment
The research presented in this article is part of the research program
Traffic and Travel Behavior in Case of Exceptional Events, sponsored
by the Dutch Foundation of Scientific Research.
References
1. Brackstone, M., and M. McDonald. Car-Following: A Historical Review.
Transportation Research Part F, Vol. 2, No. 4, 1999, pp. 181–196.
2. Pipes, L. A. An Operational Analysis of Traffic Dynamics. Journal of
Applied Physics, Vol. 24, No. 1, 1953, pp. 274–287.
3. Forbes, T. W., H. J. Zagorski, E. L. Holshouser, and W. A. Deterline.
Measurement of Driver Reactions to Tunnel Conditions. Highway
Research Board Proceedings, Vol. 37, 1958, pp. 345–357.
4. Chandler, R. E., R. Herman, and E. W. Montroll. Traffic Dynamics:
Studies in Car Following. Operations Research, Vol. 6, No. 2, 1958,
pp. 165–184.
5. Helly, W. Simulation of Bottlenecks in Single Lane Traffic Flow. Proc.,
Symposium on Theory of Traffic Flow, 1959, pp. 207–238.
6. Michaels, R. M. Perceptual Factors in Car Following. Proc., 2nd Sym-
posium on Theory of Road Traffic Flow, 1963, pp. 44–59.
7. Leutzbach, W., and R. Wiedemann. Development and Applications of
Traffic Simulation Models at the Karlsruhe Institute für Verkehrswesen.
Traffic Engineering and Control, Vol. 27, No. 5, 1986, pp. 270–278.
8. Wagner, P. Empirical Description of Car-Following. Traffic and Granular
Flow 2003, Springer, Berlin, 2003.
9. Treiber, M., A. Kesting, and D, Helbing. Delays, Inaccuracies and
Anticipation in Microscopic Traffic Models. Physica A, Vol. 360,
No. 1, 2006, pp. 71–88.
10. Hamdar, S. H., and H. S. Mahmassani. From Existing Accident-Free
Car-Following Models to Colliding Vehicles: Exploration and Assess-
ment. In Transportation Research Record: Journal of the Transporta-
tion Research Board, No. 2088, Transportation Research Board of the
National Academies, Washington, D.C., 2008, pp. 45–56.
11. Kerner, B. S. Introduction to Modern Traffic Flow Theory. Springer,
Berlin, 2009.
12. Gipps, P. G. A Model for the Structure of Lane-Changing Decisions.
Transportation Research Part B, Vol. 20, No. 5, 1986, pp. 403–414.
13. Kesting, A., M. Treiber, and D. Helbing. General Lane-Changing Model
MOBIL for Car-Following Models. In Transportation Research Record:
Transportation Research Record 2316
Journal of the Transportation Research Board, No. 1999, Transportation
Research Board of the National Academies, Washington, D.C., 2007,
pp. 86–94.
14. Choudhury, C. F. Modeling Driving Decisions with Latent Plans. PhD
thesis, Massachusetts Institute of Technology, Cambridge, 2008.
15. Hoogendoorn, S., and R. G. Hoogendoorn. Driving Behavior Theory
and Modeling by Stochastic Optimal Control. Presented at 90th Annual
Meeting of the Transportation Research Board, Washington, D.C., 2011.
16. Hoogendoorn, S. P., and P. H. L. Bovy. Generic Driving Behavior
Modeling by Differential Game Theory. Proc., Traffic and Granular
Flow 2007, Springer, Berlin, 2009.
17. Toledo, T., H. N. Koutsopoulos, and M. Ben-Akiva. Integrated Driving
Behavior Modeling. Transportation Research Part C, Vol. 15, No. 2,
2007, pp. 96–112.
18. Hoogendoorn, S., R. G. Hoogendoorn, and W. Daamen. Wiedemann
Revisited: New Trajectory Filtering Technique and Its Implications for
Car-Following Modeling. In Transportation Research Record: Jour-
nal of the Transportation Research Board, No. 2260, Transporta­tion
Research Board of the National Academies, Washington, D.C., 2011,
pp. 152–162.
19. Stanton, N. Back to the Future: Analysing Driver Performance with
Potential IVHS. Proc., 2nd Annual World Congress on Intelligent Trans-
portation Systems, Yokohama, Japan, 1995.
20. Minderhoud, M. Supported Driving: Impacts on Motorway Traffic Flow.
PhD thesis. Delft University of Technology, Netherlands, 1999.
21. Burnham, G., S. Jinborm, and G. Bekey. Identification of Human Driving
Models in Car Following. IEEE Transactions on Automatic Control,
Vol. 19, No. 6, 1974, pp. 911–915.
22. Wang, M., W. Daamen, S. Hoogendoorn, and B. van Arem. Investigat-
ing Potential Impacts of an Optimization-Based Ecological Adaptive
Cruise Control System on Traffic and Environment. Presented at 91st
Annual Meeting of the Transportation Research Board, Washington,
D.C., 2012.
23. Hamdar, S. H., M. Treiber, H. S. Mahmassani, and A. Kesting. Model-
ing Driving Behavior as a Sequential Risk-Taking Task. In Transporta-
tion Research Record: Journal of the Transportation Research Board,
No. 2088, Transportation Research Board of the National Academies,
Washington, D.C., 2008, pp. 208–217.
24. Fleming, W. H., and H. M. Soner. Controlled Markov Processes and
Viscosity Solutions (Applications of Mathematics). Springer, Berlin, 1993.
25. Hoogendoorn, S. P., and P. H. L. Bovy. Normative Pedestrian Behav-
iour Theory and Modelling. Proc., 15th International Symposium on
Transportation and Traffic Theory, Adelaide, Australia, 2002.
The Traffic Flow Theory and Characteristics Committee peer-reviewed this paper.