Validation UK

Model Parameterization and
Validation
Prof. Cesar de Prada
Dpt. Systems Engineering and Automatic Control
University of Valladolid
prada@autom.uva.es
Outline
 Introduction
 Parameterization / Calibration
 Sensitivities
 Identifiability
 Dynamic optimization
 Model validation
Cesar de Prada ISA-UVA 2

Modelling / Simulation
Tools
Range of validity Hypothesis

Knowledge
Model aims,
required Modelling
precision,…
Experimentation
Parameterization
Formulation for a
Simulation
specific calculation Coding language
Is it adequately All stages must

solved? Experimentation be validated

Modelling Methodology
Process knowledge
There are Aims, Model kind
specific
methods and Hypothesis
tools to facilitate
the Model formulation
implementation
of these steps Parameter estimation
Model Evaluation
time Explotation
Modelling
Two steps:
d x(t)
= f ( x ( t ), u ( t ), p, t ) • Model structure building
dt
y( t ) = g ( x , u ( t ), t ) • Estimating the value of the
model parameters
Generally, a choice among different possible model

structures must be made, in agreement with the selected
hypothesis
At the validation stage, several test can be proposed to select
the best option
Parameterization
d x(t)
= f ( x ( t ), u ( t ), p, t )
dt
y( t ) = g ( x , u ( t ), t )
Some model parameters can be obtained from bibliography,

documentation, etc. but there are always others that must be
estimated from experimental data.

Example: Chemical reactor
TT
A
A→B Coolant
Reactor
d cA − E RT
V = Fc Ai − Fc A − Vke cA AT
dt
Product
d cB −E
V = − Fc B + Vke RT
cA
dt
dT − E RT
Vρc e = Fρc e Ti − Fρc e T + Vke c A ∆H − UA ( T − Tr )
dt
d Tr
Vr ρ r c er = Fr ρ r c er Tri − Fr ρ r c er Tr + UA ( T − Tr )
dt
Parameterization (calibration)
TT
A
u
Coolant
Reactor
AT
t Product
y y
pa u
Model
pb Given a set of inputs, the
model response depends
p
on the value of the
parameters p

If N measurements yp(t) of the
u process output have been collected,
it is possible to define the error
function:
J = ∑ e( t ) = ∑ [y(u , p, t ) − y p ( t )]
1 N 2 1 N 2
N t =1 N t =1
y(p) yp(t)
u y
Model
y(u,p,t)
p
t
Other formulations can be
Cesar de Prada ISA-UVA made also for J 9
min J = min ∑ e( t ) = min ∑ [y(u , p, t ) − y p ( t )]

1 N 2 1 N 2
p p N t =1 p N t =1
v
u yp
Process
e(t)
Optimization
Model
y(p)
p
J normally is a non-linear function of the parameters p, and

the problem must be solved numerically using DO methods.
Dynamic model parameterization
∑ [y(u, p, t ) − y p ( t )]
N
min J = min
2
p
p t =1
x ( t ) = f ( x ( t ), u ( t ), p) y(t) = g(x(t), u(t), p)

p≤p≤p
Dynamic optimization with constraints problem, DO
Besides the explicit model parameters, unknown initial states ,

disturbances or non measured inputs can also be included in the
parameter estimation problem.

Parameterization
min J = min ∑ e( t ) = min ∑ [y(u , p, t ) − y p ( t )]

1 N 2 1 N 2
p p N p N
t =1 t =1
Generally, there are

y
Process several measured
process outputs:
min ∑ γ1 [y1 (p, t ) − y p1 ( t )] + γ 2 [y 2 (p, t ) − y p 2 ( t )] + γ 3 [y 2 (p, t ) − y p3 ( t )]

1 N 2 2 2
p N
t =1
Attention should be paid to avoid mixing variables with different

units and orders of magnitude. Normalization is required. Weights
reflect relative importance
Which parameters should be
estimated?
u
It may happen that the model
response to a given set of inputs
for two values p1 and p2 of a
certain parameter does not
change in a significant way.
y A parameter should be included

p1
in the optimization only if the
model output presents a sensible
p2 sensitivity to changes in the
parameter.

Output sensitivities
Cost function sensitivity
J = ∑ [y(p, u , t ) − y p ( t )]
N
2
t =1
∂y i ( t ) ∂J
Sij ( t ) =
∂p j ∂p j
Sensitivity of cost function J
Sensitivity of the model output i with respect to parameter j in a
with respect to parameter j in a given experiment.
given experiment. Notice that it
Summarize the effect of the
is a time function.
parameter change over the
Cesar de Prada ISA-UVA whole experiment. 14
Output sensitivities
∂y i ( t )
Sij ( t ) = It is difficult to compare output
∂p j sensitivities due to the different
units in which they are expressed.
It is better to use relative
p j ∂y i ( t ) sensitivities
s ij ( t ) =
y i ∂p j
 s11 s12  s1d  The norm of column j of the output
s s 22  s 2d  sensitivity matrix provides a measure
 21
 of the importance of parameter pj in
      the value of the model outputs.
s  s md 
 m1 s m 2
Computing output sensitivities
They can be obtained using finite differences approximation or

integrating the extended system.
One option is to use model simulations with small perturbations
on each parameter involved
∂y i ( t ) y i (p + ∆p, t ) − y i (p, t )
Sij ( t ) = ≈
∂p j ∆p
The value of the sensitivities obtained depends on the point p

considered and the experiment that was performed as u(t) is involved
in the simulations.

Extended system
∂x ( t )
s( t ) =
J = ∑ [y(p, u , t ) − y p ( t )]
N
sensitivities
2
∂p
t =1
∂J ∂y
= 2∑ [y(p, u , t ) − y p ( t )]
N
∂p t =1 ∂p
∂x d ∂x ∂f ∂x ∂f
= = + Integrating this equation, besides
∂p dt ∂p ∂x ∂p ∂p the ones of the model, it is possible
to obtain the time evolution of the
∂y ∂g ∂x ∂g sensitivities ∂x/∂p and, hence, the
= +
∂p ∂x ∂p ∂p output sensitivities.

Identifiability
We say a model is identifiable if it is possible to obtain the
value of its parameters provided that sufficient number of
process measurements is available.
In practice, identifiability means that a certain model output
corresponds to a certain value of the parameter set. But it
may happens that the same effect on the output can be
attained either modifying one parameter or another. In this
case, there is a certain co-linearity in that parameter set
which makes very difficult the identification.
Identifiability is a structural property of the model, but the
identification of particular parameter can depend also on the
experimental data.
Identifiability examples
In a chemical reactor, without temperature measurements,

parameters k and E, cannot be estimated independently
In the model, it is possible to identify the

dc B F −E
= − c B + ke RT c A ratio F/V, but not F and V independently.
dt V
With steady state data, V cannot be
identified.
µ ms In the Monod model, using data

limited to small values of s, only
K+s the ratio µm/K can be identified.
With data containing only large
s
values of s, only µm can be
identified well.
Identifiability
With the output sensibility matrix, it is possible to see if a

certain degree of co-linearity exists, by inspecting range of its
sub-blocks (by rows).
 s11 s12  s1d  sij

s s 22  s 2d 
 21 
      sik
s  s md 
 m1 s m 2 t
Co-linearity makes parameter identification more difficult

Re-parameterization
Sometimes it is only possible to identify certain parameter

combinations, or new variables can be defined in the model to obtain
a model structure easier to identify.
If either of these alternatives has been implemented, it is necessary to
remember that:
 Statistical characteristics of new variables are different to the ones
of the data set.
 Confident regions of the new parameters are different to the ones
of the original parameter.

Experiments
 In order the model to capture the process dynamics, the
experimental data used in the identification must contain
information about that dynamics.
 Hence, the experiments should cover different operating
conditions, exciting the diffferent modes of operation of the
process.
 Historical records tend to be useless as they have poor
dynamic information, as the operators or the control system try
to maintain the process as stable as possible.

Experiments
u2
TT
u1 Tr Reactant
Ti T
Reactor
Coolant
AT
Product
If the process has several inputs, changes applied to themshould
be uncorrelated, sothat the optimization algorithm can
distinguish the effects of every input on every output.
Experiments
Notice that, if correlated signals, such as u1=αu2, are applied
simultaneously, there are many combinations of models M1
and M2 that correspond to the same couples of input output
values
y ( t ) = M 1u 1 ( t ) + M 2 u 2 ( t ) = ( α M 1 + M 2 ) u 2 ( t )
Additionally, experiments should cover adequate ranges of

amplitude and frequency in order to excite the different
process dynamics and modes of operation.
Recorded data should have a signal to noise ratio larger than 3
and parametric sensitivities should also present sensible
values.
Experiments
u2
TT
u1 Tr Reactant
Ti T
Reactor
Coolant
AT
Product
Change one input while maintaining constant the others, and
repeat the experiment agin with the remainder ones, it is not a
good policy: it increases the time required for data collection and
provides data in special operating conditions.
Solving parameterization problems
min J = ∑ [y(p, u , t ) − y p ( t )]
N
2
p t =1

p≤p≤p
Selection of cost function J
Numerical solution
•Sequential approach
•Simultaneous approach
Initialization of states
DO: Sequential approach
min J = ∑ [y(p, u , t ) − y p ( t )]
N
2
p t =1

p≤p≤p
parameters
NLP Optimizer
p*
p J
Simulation from 1 to N u(t), y(t)

In order to compute J(p)

Dynamic Optimization DO
Starting from a certain operation point, and performing a

single change in the process MV, bring the process to the
maximum production point respecting in the transient a set
of constraints.
Ti, ci, q
TT AT Row product A
Fr, Tri T, x
Reactor A→B
Coolant
Product: A & B
Dynamic Optimization
q cB
q, Fr
A→B
t t
T max qc B
q , Fr
Dynamic model equations

... ≤ q ≤ ...
t ... ≤ Fr ≤ ...
Tmin ≤ T ( t ) ≤ Tmax
DO in EcosimPro
EXPERIMENT
COMPONENT L1 ≤ z1 ≤ U1
Constraints in the L2 ≤ z2 ≤ U 2
PROCESS MODEL decision variables z

dx ( t )
= f ( x ( t ), u ( t ), z) Decision variables : z1, z2, ...
dt
y( t ) = g ( x ( t ),u ( t ), z) OPTIMIZATION
ALGORITHM
CONSTRAINTS
NLP
g 1 ( x ( t ), u ( t )) ≤ 0 Value of the cost function J and
constraints g
g 2 ( x ( t ), u ( t )) ≤ 0

COST FUNCTION
z*
J (z )
The optimization algorithm obtains the
values of the cost function J and
Dynanic simulator constraints g when need them by calling
the dynamic simulation module
Component / Variables
Component name
COMPONENT reactor_ab_dynamic_max
DATA
Decision Variables
REAL ....
DECLS
REAL q = 2.832 “Volumetric Inflow [m3/h]“
REAL Fr “Coolant flow [m3/h]"
REAL ....
-- Indice a minimizar
REAL J_cost Cost Function
-- Vector de las restricciones no lineales
REAL val_func[3]
…….. Constraints
Constraints / Cost function
CONTINUOUS
... ecuaciones del modelo Dynamic model
-- balance de masa de A
V*cA' = q*(cA0 - cA)- V*k*cA
-- balance de energia en el reactor
V*rho*Cp*T' = q*rho*Cp*(Te - T) - V*k*cA*DH - Q
………
-- calculo de la funcion de costo a minimizar Cost function
J_costo = - q*cB
-- calcular las restricciones no lineales (expresiones 0 >= g(x))

val_func[1] = 0.75 - x
val_func[2] = LiminfT - minabs(T, Tmin)
val_func[3] = maxabs(T, Tmax) - LimsupT
END COMPONENT Max(T(t)) < LimsupT

Experiment / Variables
EXPERIMENT exp1 ON reactor_ab_dynamic_max.open_loop
DECLS
REAL val_objetivo -- valor de la funcion objetivo J
REAL param_estim[2] -- vector de variables a optimizar
REAL L_est_inf[2] -- limite inferior de las variables de decision
REAL L_est_sup[2] -- limite superior de las variables de decision
INIT -- set initial values for variables
BOUNDS -- set expressions for boundary variables: v = f(t,...)
Te = 21.11
Tre = 21.11
cA0 = 14.46
-- q = 2.832 -- La fija el optimizador en cada iteracion junto a Fr
Experiment / Optimization
BODY
q = 55 Fr = 3.4 -- Inicializaciones
param_estim[1] = q -- Formar vector de variables de decisión

param_estim[2] = Fr
L_est_inf[1] = 0.5 -- Limites de las variables de decision (q)

L_est_sup[1] = 5
L_est_inf[2] = 5 -- Limites de Fr Decision variables range
L_est_sup[2] = 90
-- Llamar a la rutina de optimización de NAG

optim_Nag (objetivo, n_x, n_res, param_estim, L_est_inf, L_est_sup)
-- Recoger los resultados
q = param_estim[1] Fr = param_estim[2]
Calling the optimizer
END EXPERIMENT
Model call
FUNCTION NO_TYPE objetivo (IN INTEGER n_est, IN REAL var_est[], IN REAL
func_res[])
BODY Reset
RESET()
q = var_est[1]
Fr = var_est[2] Selecting the values of
the decision variables
TIME=0
TSTOP = 10
Model execution
CINT=10
INTEG()
Passing values of const function
func_res[1] = J and constraints
func_res[2] = val_func[1]
END FUNCTION
Results in EcosimPro (EcoMonitor)

Parameterization / Methodology
 Experiments (data recorded for calibration and validation)
 Data analysis and filtering
 Choice of parameters to be identified
– Structural identifiability
– Sensitivities computation
 Optional model re-parameterization to avoid co-linearities
 Initial estimates and possible ranges of the parameters
 Select the cost function
 Estimate the parameters by optimization
 Validate the model. Estimate residuals and confident bands
for the parameter.

Example: Heated tank
Ti T temperature
q m tank mass
Te H liquid level
V voltage
m q Inflow
V R T h F Outflow
a valve opening
H entalphy
F ce specific heat
a
A tank cross section
Hypothesis: ρ density
R resistence
T lumped temperature
Te external temperature
ρ constant density
Ti inflow temperature
ce constant specific heat
Dynamic model
State space non-linear model
Ti
q
dh 1
= (q − F) Te
dt A
dT q V2 U m
= (Ti − T ) + − amb (T - Te )
dt Ah AhRρc e Ahρc e V R T h
F = ak h
F
CV: T temperature MV: V voltage a
h level a valve opening
Disturbances: q Inflow, Ti Input temperature
States: h , T
Other variables: F outflow Cesar de Prada ISA-UVA 39
Model calibration
dh 1 One experiment was

= (q − F)
dt A performed where the inflow
q and voltage V to the
dT q V2 U
= (Ti − T ) + − amb (T - Te ) resistance were changed over
dt Ah AhRρc e Ahρc e time and the values of the
F = ak h liquid level and temperature
in the tank were recorded
Disturbances in Te were not

Unknown parameters to be estimated recorded in the data set
-- k friction factor
-- Uamb Heat transfer coefficient to ambient
-- A tank cross section
-- R electrical resistance

Calibration

Example in EcosimPro:
Van der Vusse Reactor
Highly non-linear
reactor, difficult to
control
A→B→C
2A → D
Parameters:
Volume: 10 l
Refrigerant mass: 5 Kg
Reactor Van der Vusse

Model validation
 Validating a model consists of implementing several

test, so that, if the model responds adequately to
them, a certain trust on its soundness for the aims it
was designed for is obtained.
 There is no “prove” of model validity, but a certain
degree of confidence in the model based on results of
the tests.
 Model validity can be lost because of a single
negative result in a test.

Validation
 The set of tests with positive and negative results

allow to fix the range of validity of the model
 Different situations must be treated with different
validation methods:
– The process exists and a model must be built reproducing
its behaviour
– The process does not exists yet and the purpose of the
model is to design it or predict future behaviour and
optimize it.

Validation
 Set of tests of different nature

 Validation should be made at all model building
levels:
– Hypothesis
– Formulation
– Software coding
– Numerical methods (verification)
– Application

Validation levels
Does the model use

Validity range Hypothesis fit the hypothesis?
Data recording / Does the model
Reconciliation Modelling reflect the key
process aspects?
How much
sensitive is Which is the
model to Parameterization quality of the
parameter data?
changes?
Software coding Is the model well
Is the code coded?
robust, fast,…?
Do results match
Does it provide Experimentation the reality?
sensible results?
Validation methods
 Qualitative model responses

 Experimental data fit
 Statistical tests
 Sensitivity analysis
 Prediction capabilities
 Parameter distortion
 Coherence between different models
…
Qualitative model responses
Evaluation of the model responses to standard inputs
u y
Model
Evaluation of the model

responses with domain
experts.

Example of the model response to a step change in the cooling
jacket input temperature to be analysed with domain experts

 Turing Test: Is it possible to distinguish

between the model and process responses to
the same input if they are presented in the
same format?
 Examine the time evolution of variables such
as flows through different components

Use of experimental data
Compare the process and

v
model responses to the yp
u
Process
same inputs with data sets
different to the ones used in
the model parameterization Model(p)
y
1 N
∑ (y ( t ) − y p ( t ) )2 y
N t =1
model
yp
Errors due to: experiment
t
 Initial conditions
 Disturbances
 Model Cesar de Prada ISA-UVA 52
Error indexes
Numerical measure of the matching between model

responses y(t) and experimental data yp(t)
Can be used to select the best model among several
candidates to represent a process
1 N
∑ (y ( t ) − y p ( t ) )2
model yp
N t =1
experiment

Error indexes
Most of the indexes combine Akaike Information

sum of errors and number of Criterion AIC
parameters in the model
2d N
(1 + )∑ ( y( t ) − y p ( t )) 2
Final Prediction Error FPE N t =1
1 1+ d N N
( )∑ ( y( t ) − y p ( t )) 2  N 
N 1 − d N t =1  ∑ ( y( t ) − y p ( t )) 2 
N log t =1  + 2d
Estimates of the prediction error  N 
variance with the new data  
 
N number of data
D number of parameters in the model
Error indexes
Bayesian Information Rissamen’s Minimal

Criterion BIC Description lenght
 2 
N
2d
(1 + log N)∑ ( y( t ) − y m ( t )) 2
N
 ∑ ( y( t ) − y m ( t ))  N t =1
N log t =1  + d log( N)
 N  Gives more weight to model
  complexity
 
Consistent: Probability of selecting an
incorrect model tends to zero with N

Error indexes
F Test: Model j (with more parameters) is significantly

better than model i, with a confidence level α
 N   N
 
 ∑ ( y( t ) − y p ( t )) 2
− ∑ ( y( t ) − y p ( t )) 
2  /(d j − d i )

  t =1  
  mod i  t =1  mod j 
N 2
∑ ( y( t ) − y p ( t ))  /( N − d j )
 t =1  mod j
This statistical must be compared with the value provided by

the F(dj-di, N-dj) distribution with a confidence level α
Data fitting errors
 Sources of data fitting errors are:

– Measurements noise and disturbances. A characterization
can be made measuring output variables while maintainig
constant the process inputs.
– Unknown initial conditions in the model. Its influence
disappear after the initial transient.
– Structural or parametric errors in the model. If the model
were perfect, the residuals should have similar statistical
characteristics as the output noise and disturbances.

Statistical test
v
u yp
Process
e
Model y
If the model is adequate, residuals should not show a

“systematic” structure, but be the result of the stochastic noise
and disturbances acting on the process.
Residuals e(t) = y(t) - yp(t)

Statistical test
 Visual residual inspection
e e
t t
 Number of residual sign changes (ncs)

 Do they fit an AR model?
N
ncs − This statistic should follow a N(0,1) distribution if
Test: 2 there is no bias in the number of sign changes
N/2 Cesar de Prada ISA-UVA 59
Statistical test
No correlation between input and residuals: With a good
model, residuals represent the effect of noises and
disturbances and they should be independent of a particular
experiment given by u
Rue(k) Re(k)
2σ
2σ
k k
Open loop, closed loop Residuals whiteness

Parameter variance
Parameter confidence regions estimated in the model

calibration provide a measure of its quality and the model
goodness.
 v11 v12 
V =  v 21 v 22 
 
   
Main diagonal elements of the covariance matrix obtained in the

calibration step, give a measure of the confidence range of each
parameter, while the off diagonal terms measure parameter independence,
as ideally, they should be uncorrelated.
Parameter variance
v
u yp
Process
e
Model y
In general terms, the variance of

the estimated parameters
{ } d Φ v (w ) increases with the number of

var G (e jwt ˆ
, θ) ≅ parameters d in the model, and
N Φ u (w )
decreases with the ratio
signal/noise and the number of
data N used in the calibration
Sensitivity to parameters
How much J changes per unit change in a parameter p ?

How much a given output changes per unit change in a
parameter p?
min J = ∑ [y(p, u , t ) − y p ( t )]
N
2
p t =1

p≤p≤p
They can be obtained from the sensitivities computed

from integration of the extended system

Prediction capability
ŷ( t + j)
y(t+j)
u(t+j)
t
Future values can be predicted and compared with actual

data. If the model is linear, explicit prediction formulas for
DMC, GPC, etc. can be used.
Parameter distortion
d x(t)
= f ( x ( t ), u ( t ), p, t )
dt
y( t ) = g ( x , u ( t ), t )
Calibration provides the value of the parameter set p*

(constant) giving the best fit to the experimental data
e* ( t ) = y ( p * , t ) − y p ( t )

Parameter distortion
Butterfield 1986
Assuming now that the set p could evolve over time,
Which would be the time evolution of model parameters p
such that the model responses y would correspond exactly with
the process ones for the same inputs?
Admissible distortions of
p parameters with physical
meaning provide a measure
of model credibility
t
min y( t , p* + ∆p( t )) = y p ( t ) t = 1,..., n

∆p

Validation UK

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Validation UK

Uploaded by

Copyright:

Available Formats

Model Parameterization and

Cesar de Prada ISA-UVA 2

Range of validity Hypothesis

Is it adequately All stages must

Cesar de Prada ISA-UVA 3

Generally, a choice among different possible model

Some model parameters can be obtained from bibliography,

Cesar de Prada ISA-UVA 6

Cesar de Prada ISA-UVA 8

min J = min ∑ e( t ) = min ∑ [y(u , p, t ) − y p ( t )]

J normally is a non-linear function of the parameters p, and

x ( t ) = f ( x ( t ), u ( t ), p) y(t) = g(x(t), u(t), p)

Dynamic optimization with constraints problem, DO

Besides the explicit model parameters, unknown initial states ,

Cesar de Prada ISA-UVA 11

min J = min ∑ e( t ) = min ∑ [y(u , p, t ) − y p ( t )]

Generally, there are

min ∑ γ1 [y1 (p, t ) − y p1 ( t )] + γ 2 [y 2 (p, t ) − y p 2 ( t )] + γ 3 [y 2 (p, t ) − y p3 ( t )]

Attention should be paid to avoid mixing variables with different

y A parameter should be included

Cesar de Prada ISA-UVA 13

x ( t ) = f ( x ( t ), u ( t ), p) y(t) = g(x(t), u(t), p)

They can be obtained using finite differences approximation or

The value of the sensitivities obtained depends on the point p

Cesar de Prada ISA-UVA 16

x ( t ) = f ( x ( t ), u ( t ), p) y(t) = g(x(t), u(t), p)

Cesar de Prada ISA-UVA 17

In a chemical reactor, without temperature measurements,

In the model, it is possible to identify the

µ ms In the Monod model, using data

With the output sensibility matrix, it is possible to see if a

 s11 s12  s1d  sij

Co-linearity makes parameter identification more difficult

Cesar de Prada ISA-UVA 20

Sometimes it is only possible to identify certain parameter

Cesar de Prada ISA-UVA 21

Cesar de Prada ISA-UVA 22

Additionally, experiments should cover adequate ranges of

x ( t ) = f ( x ( t ), u ( t ), p) y(t) = g(x(t), u(t), p)

x ( t ) = f ( x ( t ), u ( t ), p) y(t) = g(x(t), u(t), p)

Simulation from 1 to N u(t), y(t)

Cesar de Prada ISA-UVA 27

Starting from a certain operation point, and performing a

Dynamic model equations

-- calcular las restricciones no lineales (expresiones 0 >= g(x))

END COMPONENT Max(T(t)) < LimsupT

Cesar de Prada ISA-UVA 32

EXPERIMENT exp1 ON reactor_ab_dynamic_max.open_loop

INIT -- set initial values for variables

BOUNDS -- set expressions for boundary variables: v = f(t,...)

param_estim[1] = q -- Formar vector de variables de decisión

L_est_inf[1] = 0.5 -- Limites de las variables de decision (q)

-- Llamar a la rutina de optimización de NAG

Cesar de Prada ISA-UVA 36

Cesar de Prada ISA-UVA 37

dh 1 One experiment was

Disturbances in Te were not

Cesar de Prada ISA-UVA 40

Cesar de Prada ISA-UVA 41

Cesar de Prada ISA-UVA 43

 Validating a model consists of implementing several

Cesar de Prada ISA-UVA 44

 The set of tests with positive and negative results