Model-based Experimental Design for

Biological Systems Modelling

Hong Yue
Department of Electronic and Electrical Engineering
University of Strathclyde, Glasgow, UK

Outline

Introduction

Briefs on experimental design based on FIM

Robust experimental design

Measurement set selection for NF-kB signal pathway

Conclusions

Introduction to experimental design

Experimental data can be gathered for various reasons:

To get a better understanding of the system under study

To discriminate between several possible model structures

To estimate parameters of a model

To validate models

Purpose of experimental design

Design experiments in such a way that the parameters are

estimated from the resulting experimental data with the
best possible statistical quality

A central (and sometimes hidden) methodology in every occasion where

unknown quantities must be estimated and the choice of a estimation
method is open (Pronzato, Automatica, 2008)

Introduction to experimental design

Background

Well developed methodology in statistics

Early efforts in experimental design can be

traced back to early 20th century

The work of Fisher (1925) formed the basis

Initially for steady-state models from linear

regression models to (later) nonlinear
models with analytical solutions.

Later (mostly in recent two decades) to

dynamic models described by ODEs, DAEs.

Preece, D.A., Fisher and experimental design: a review. Biometrics (1990)

Introduction on experimental design

In control theory, construction of persistently exciting inputs

for dynamical systems (Goodwin & Payne, 1977; Zarrop, 1979;
Ljung, 1987; Walter & Pronzato, 1997; Gevers, 2005;
Hjalmarsson, 2005)

Relative little work in life sciences

Bridge the gap between experimental and theoretical scientists?

Theoreticians appraise experimental feasibility and efforts

Experimenters know which kind of experimental information

is required or helpful to establish a mathematical model

Modelling & experimental design

Model Identification
Data: D
Structure Identification
Model parameter (feature)
selection
Experimental Design
Sequential or robust
process

Global
sensitivity

Local
sensitivity

Parameters:

Parameter Estimation
Local, global,
Semi-parametric

Overall model-building strategy

Experimental design

Hypothesis

Prior
knowledge

Design

Experimental degrees of freedom

Manipulations

Excitation signals applied to the system to produce

information-rich data (time-dependent stimuli profile)

Measurements

What to measure? (measurement set selection)

Where to measure? (position)

When to measure? (sampling schedule)

Main aspects in collecting data for biological system modelling

Sampling time

Pattern of stimulation

Observables

Least squares (LS) parameter estimation

System model
f : nonlinear state transition

X f ( X, , , t ), X(t0 ) X0

function

y g ( X, , t )

g : measurement function

: zero-mean, Gaussian additive

noise vector

- model parameters to be
estimated

: design parameter vector

LS Parameter estimator
N

arg min y (tl ) y ( , tl ) Q 1 y (tl ) y ( , tl )

l 1

Q - measurement error covariance matrix

N number of sampling points

Information content
Covariance matrix (uncertainties caused by process and
measurement noises)

E ( )( )T

E{}
: expectation operator

Parameter uncertainty
The expectation E{} and covariance matrix can be used
to measure the estimation accuracy
The bias is ignored in general, i.e. E{} E{}
The parameter uncertainty can be described by a cost
function () -- basis for experimental design

Information content
Fisher information matrix (FIM)
N

FIM S(tl )T Q 1S(tl )

S X - sensitivity matrix

l 1

measurement noises are additive and white

The inverse of FIM provides a lower bound for parameter

estimation error covariance matrix

FIM 1

(1)

(Cramer-Rao theory)

under conditions
measurement noise is additive and white
model is linear in its parameters
the estimation is unbiased

Most experimental designs are based on the use of (1) with either
or FIM.

Design criteria based on FIM

Several scalar functions of FIM have been defined as a measure
of the quality of the estimated parameters.

A-optimal

min tr FIM 1

Minimise sum of variances or errors of parameter estimates

D-optimal

max det FIM

Minimise the overall volume of the confidence

region

E-optimal

max min FIM

Minimise the largest confidence ellipsoid

axis (estimation error)

Geometric interpretation of the design criteria

This produces a vector
optimization problem and
common matrix (alphabet)
scalarization measures
A-optimality: max tr(FIM) /
min tr(S) (eigenvalue sum)

D-optimality: max det(FIM)

(eigenvalue product)
E-optimality: max min(FIM)
(largest eigenvalue)

Optimal experimental design (OED)

Optimal experimental design

arg max FIM ,

: experimental parameter vector

: OED criterion

Non-optimal experiment (red curve)

Optimal experiment (blue curve)
Parameter estimation based on
optimal experiment has:

narrower objective function

profile
smaller confidence interval

Maximise data information = reduce covariance

(uncertainty) in parameter estimation

An example of experimental design

Compartment model used in pharmacokinetics
A drug is injected in blood , which moves from the central compartment C to
the peripheral compartment P, where the quantities of drugs are denoted as xc
and xp (DArgenio, 1981)

u(t)

xC(t)

xP(t)

ODE model

xC ( K EL K CP ) xC K PC xP u (t )
xP K CP xC K PC xP
y (t ) xC / V (t )

y(t)

Unknown parameters

KCP , K PC , K EL ,V

(measurement)

V: volume of the central

compartment
: identical, independent distributed
Gaussian noise N(0,2)

An example of experimental design

The input profile u(t) consists of a 1 min loading infusion of 75mg/min
followed by a continuous maintenance infusion of 1.45mg/min.

Assume 8 samplings are allowed for measurement between [1, 720]

min.
Conventional design
t = (5, 10, 30, 60, 120, 180, 360, 720)
D-optimal design
t* = (1, 1, 10, 10, 74, 74, 720, 720)

An example of experimental design

For each design, 400
experiments were performed
and the lines denote the
least squares parameter
estimate histograms
KEL=0.0242 min-1

0.2g/ml

Approximate marginal densities for the LS estimator K EL

The optimal design significantly reduces the uncertainty in the

identified system

Robust experimental design

The quality of conventional OED depends on the accuracy of

model parameters and local sensitivity analysis
Robust experimental design addresses model uncertainties
during the design process

Existing techniques
Sequential design
Iterative process between experimental design and parameter
estimation
Expected value approach/Bayesian approach
Minimisation of the expected value (over the prior parameter
distribution) of a local optimality criterion

Maximin design
Optimises the worst possible performance for any parameter
values in the uncertainty region

Robust experimental design

Generalized robust -optimal design problem

max (FIM(, ))

: robust design approach

s.t.

c: positive real number

* c
p

Maximin robust experimental design problem

max min FIM(, )

Expected (Bayesian) robust experimental design problem

max FIM(, )

or

max FIM(, ) P()d

E{}
: expectation operator
P() : probability density
function of

Maximin robust experimental design

Maximin robust experimental design is computationally

expensive
max min FIM(, )

An alternative maximin formulation (Flaherty et al. 2006)

max min FIM(* , )

: bounded uncertainty in FIM

The optimization problem is further relaxed into a

standard semi-definite program (SDP) problem
- Computationally efficient

Uncertainty in FIM
Parametric uncertainty

FIM uncertainty

Re-write FIM
N

FIM S(tl ) Q S(tl ) S(tl )T W S(tl )

T

l 1
N

l 1

ns

iSTi (tl )Si (tl )

l 1 i 1

A local Taylor expansion of FIM around the nominal * is

applied to obtain (He, Brown, Yue, Yeung, 2008)
FIM (* ) FIM (* )

FIM

FIM (* )
ns

FIM( ) S (t , )S (t , )
N

l 1

i 1

Bayesian robust experimental design

Represent the parametric uncertainty with a prior distribution P()

Integrate the uncertainty effects into the design

max FIM(, ) P()d

Bayesian information matrix (BIM) (Merle & Mentre, 1995)

FIM(, ) FIM(, ) P()d

Advantages:

Bayesian experimental design is to explore the admissible parameter

space as described with prior distribution rather than to find a single
solution with nominal values (optimal design) or worst-case values
(maximin desgin).

The Bayesian approach balances the result between an optimal design

and a maximin design

Robust measurement set selection

Purpose: prioritize the contribution of different states to parameter estimation

x1 , , xny

,
1 , , ns
1

ns

i 1

1, i 0, i
weights for states

Fisher information matrix

N

ns

FIM , , t iSTi tl , Si tl ,
l 1 i 1

Si the ith row of the sensitivity matrix S

Robust measurement set selection

(1) E-optimal design (Flaherty et al, 2006)
1
ns

T
max min iSi Si

i 1

ns

s.t. i 1, i 0, i
i 1

max

ns

s.t. iSTi Si I nI
i 1
ns

i 1

1, i 0, i

Relax the E-optimal design to a semi-definite program (SDP)

Robust measurement set selection

(2) Maximin Robust E-optimal design
Uncertainty bound

blk diag(1 ,

, ns ),

Robust E-optimal design (maximin method)

max

max
min

s.t. (S t S t )
N

ns

l 1 i 1

1, 0
i

ns

s.t. S (t )S (t ) n J
T

l 1 i 1

nI

ns

i 1

ns

nI

ns

1, 0
i 1

SDP solver: SeDuMi (http://sedumi.ie.lehigh.edu/) (Matlab)

Brown et al. , Int. J. Bioinformatics Res. Appl. (2008)

nI

Robust measurement set selection

(3) Bayesian Robust E-optimal design

max FIM(, ) P()d

With a uniform distribution in P(), Bayesian E-optimal design for

measurement set selection can be written as
max
v

ns

s.t. 1 S (t , )S (t , ) vI
K
K

r 1 l 1 i 1

nI

ns

1, 0
i 1

K is the number of sampling points for within its admissible range

Example: IB-NF-B signal transduction pathway

Graphical Connection Map for IkB-NF-kB Model

Ihekwaba et al. (2004)

Robust measurement set selection

At each uncertainty level
larger weight value

larger influence of
the state

optimal design

robust design

uniform design

Comparison of different designs

Model: simplified IkBa-NF-kB signal pathway

Parameters to be estimated: , , , ,

Experimental design: measurement set selection

12

13

16

18

Top four selected states as measurement variables

methods

top states
(30% uncertainty)

top states
(60% uncertainty)

E-optimal

5 8 7 1

5 8 7 1

Bayesian E-optimal

5 8 7 1

5 8 7 10

Maximin E-optimal

5 8 7 1

8 1 5 9

He, Brown, Yue, Int. J. Robust & Nonlinear Contr. (2010)

Comparison of different designs

95% confidence intervals
Uncertainty: 30%

Uncertainty: 60%

optimal (black), Bayesian (red) and maximin (blue) design

He, Brown, Yue, Int. J. Robust & Nonlinear Contr. (2010)

Conclusions

Model-based experimental design is to maximise the

information gathered for quantitative model identification

Standard optimal experimental design can be inadequate

due to the local validity of sensitivity analysis
Robust design strategies are proposed to incorporate
uncertainties in the model parameters into design processes
The uncertainty formulation is crucial in robust
experimental design

Further readings on experimental design

Atkinson, A. C. and , A. N. Donev, Optimum Experimental Design. Oxford University Press, Oxford, 1992.
Bruwer, M.-J. and J.F. MacGregor, Robust multi-variable identification: optimal experimental design with
constraints, J. Proc. Contr., 2006, 16: 581-600.
Chaloner, K. and I. Verdinelli, Bayesian experimental design: a review. Statist. Sci., 1995, 10: 273-304.
Dette, H., V.B. Melas, A. Pepelyshev and N. Strigul, Robust and efficient design of experiments for the
Monod model, J. Theor. Biol., 2005, 234: 537-550.
Dette, H., I.M. Lopez, I.M.O. Rodriguez and A. Pepelyshev, Maximin efficient design of experiment for
exponential regression models, J. Statistical Planning and Inference, 2006, 136: 4397 - 4418.
Faller, D., U. Klingmuller and J. Timmer, Simulation methods for optimal experimental design in system
biology, Simulation, 2003, 79: 717-725.
Flaherty, P., M.I. Jordan and A.P. Arkin. Robust design of biological experiments, In: Proc. Neural
Information Processing Symp., 2005 .
Franceschini, G. and S. Macchietto, Model-based design of experiments for parameter precision: State of the
art, Chem. Eng. Sci., 2008, 63: 4846-4872.
Goodwin, G. C., Identification: experimental design. Pergamon Press, Oxford, 1987.
He, F., M. Brown and H. Yue, Maximin and Bayesian robust experimental design for measurement set
selection in modelling biochemical regulatory systems. Int. J. Robust and Nonlinear Contr., 2010, 24:
1059-1078.
Pronzato, L., Optimal experimental design and some related control problems, Automatica, 2008, 44: 303-

325.
Rojas, C.R., J. S.Welsh, G.C. Goodwin and A. Feuer, Robust optimal experiment design for system
identification, Automatica, 2007, 43: 993 - 1008.
Kreutz, C. and J. Timmer, Systems biology: experimental design, FEBS J., 2009, 276: 923-942.