Grey box model

In mathematics, statistics, and computational modelling, a grey box model[1][2][3][4] combines a partial
theoretical structure with data to complete the model. The theoretical structure may vary from information
on the smoothness of results, to models that need only parameter values from data or existing literature.[5]
Thus, almost all models are grey box models as opposed to black box where no model form is assumed or
white box models that are purely theoretical. Some models assume a special form such as a linear
regression[6][7] or neural network.[8][9] These have special analysis methods. In particular linear regression
techniques[10] are much more efficient than most non-linear techniques.[11][12] The model can be
deterministic or stochastic (i.e. containing random components) depending on its planned use.

Model form
The general case is a non-linear model with a partial theoretical structure and some unknown parts derived
from data. Models with unlike theoretical structures need to be evaluated individually,[1][13][14] possibly
using simulated annealing or genetic algorithms.

Within a particular model structure, parameters[14][15] or variable parameter relations[5][16] may need to be
found. For a particular structure it is arbitrarily assumed that the data consists of sets of feed vectors f,
product vectors p, and operating condition vectors c.[5] Typically c will contain values extracted from f, as
well as other values. In many cases a model can be converted to a function of the form:[5][17][18]

m(f,p,q)

where the vector function m gives the errors between the data p, and the model predictions. The vector q
gives some variable parameters that are the model's unknown parts.

The parameters q vary with the operating conditions c in a manner to be determined.[5][17] This relation can
be specified as q = Ac where A is a matrix of unknown coefficients, and c as in linear regression[6][7]
includes a constant term and possibly transformed values of the original operating conditions to obtain non-
linear relations[19][20] between the original operating conditions and q. It is then a matter of selecting which
terms in A are non-zero and assigning their values. The model completion becomes an optimization
problem to determine the non-zero values in A that minimizes the error terms m(f,p,Ac) over the
data.[1][16][21][22][23]

Model completion
Once a selection of non-zero values is made, the remaining coefficients in A can be determined by
minimizing m(f,p,Ac) over the data with respect to the nonzero values in A, typically by non-linear least
squares. Selection of the nonzero terms can be done by optimization methods such as simulated annealing
and evolutionary algorithms. Also the non-linear least squares can provide accuracy estimates[11][15] for the
elements of A that can be used to determine if they are significantly different from zero, thus providing a
method of term selection.[24][25]

It is sometimes possible to calculate values of q for each data set, directly or by non-linear least squares.
Then the more efficient linear regression can be used to predict q using c thus selecting the non-zero values
in A and estimating their values. Once the non-zero values are located non-linear least squares can be used
on the original model m(f,p,Ac) to refine these values .[16][21][22]

A third method is model inversion,[5][17][18] which converts the non-linear m(f,p,Ac) into an approximate
linear form in the elements of A, that can be examined using efficient term selection[24][25] and evaluation
of the linear regression.[10] For the simple case of a single q value (q = aTc) and an estimate q* of q.
Putting dq = aTc − q* gives

m(f,p,aTc) = m(f,p,q* + dq) ≈ m(f,p.q*) + dq m’(f,p,q*) = m(f,p.q*) + (aTc − q*) m’(f,p,q*)

so that aT is now in a linear position with all other terms known, and thus can be analyzed by linear
regression techniques. For more than one parameter the method extends in a direct manner.[5][18][17] After
checking that the model has been improved this process can be repeated until convergence. This approach
has the advantages that it does not need the parameters q to be able to be determined from an individual
data set and the linear regression is on the original error terms[5]

Model validation
Where sufficient data is available, division of the data into a separate model construction set and one or two
evaluation sets is recommended. This can be repeated using multiple selections of the construction set and
the resulting models averaged or used to evaluate prediction differences.

A statistical test such as chi-squared on the residuals is not particularly useful.[26] The chi squared test
requires known standard deviations which are seldom available, and failed tests give no indication of how
to improve the model.[11] There are a range of methods to compare both nested and non nested models.
These include comparison of model predictions with repeated data.

An attempt to predict the residuals m(, ) with the operating conditions c using linear regression will show if
the residuals can be predicted.[21][22] Residuals that cannot be predicted offer little prospect of improving
the model using the current operating conditions.[5] Terms that do predict the residuals are prospective
terms to incorporate into the model to improve its performance.[21]

The model inversion technique above can be used as a method of determining whether a model can be
improved. In this case selection of nonzero terms is not so important and linear prediction can be done
using the significant eigenvectors of the regression matrix. The values in A determined in this manner need
to be substituted into the nonlinear model to assess improvements in the model errors. The absence of a
significant improvement indicates the available data is not able to improve the current model form using the
defined parameters.[5] Extra parameters can be inserted into the model to make this test more
comprehensive.

See also
Mathematics
portal

Computer experiment
Computer simulation
Design of experiments
Grey box testing
Mathematical model
Nonlinear system identification
 Parameter estimation
Research design
Scientific modelling
Simulation
Statistical model
System dynamics
System identification
System realization
Systems theory

References
1. Bohlin, Torsten P. (7 September 2006). Practical Grey-box Process Identification: Theory
and Applications (https://books.google.com/books?id=SZtEAAAAQBAJ). Springer Science
& Business Media. ISBN 978-1-84628-403-8.
2. "Grey-box model estimation" (http://www.mathworks.com.au/help/ident/grey-box-model-esti
mation.html). Mathworks 2. 2012.
3. Kroll, Andreas (2000). Grey-box models: Concepts and application. In: New Frontiers in
Computational Intelligence and its Applications, vol.57 of Frontiers in artificial intelligence
and applications, pp. 42-51. IOS Press, Amsterdam.
4. Sohlberg, B., and Jacobsen, E.W., 2008. Grey box modelling - branches and experiences (ht
tps://www.sciencedirect.com/science/article/pii/S1474667016408025), Proc. 17th World
Congress, Int. Federation of Automatic Control, Seoul. pp 11415-11420
5. Whiten, B., 2013. Model completion and validation using inversion of grey box models (htt
p://journal.austms.org.au/ojs/index.php/ANZIAMJ/article/view/6125/1682), ANZIAM J.,54
(CTAC 2012) pp C187–C199.
6. Draper, Norman R.; Smith, Harry (25 August 2014). Applied Regression Analysis (https://boo
ks.google.com/books?id=uSReBAAAQBAJ). John Wiley & Sons. pp. 657–. ISBN 978-1-
118-62568-2.
7. Weisberg, Sanford (25 November 2013). Applied Linear Regression (https://books.google.c
om/books?id=7pg3AgAAQBAJ). Wiley. ISBN 978-1-118-59485-8.
8. Heaton, J., 2012. Introduction to the math of neural networks, Heaton Research Inc.
(Chesterfield, MO), ISBN 978-1475190878
9. Stergiou, C.; Siganos, D. (2013). "Neural networks" (https://web.archive.org/web/200912161
10504/http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html). Archived from
the original (http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html) on 2009-
12-16. Retrieved 2013-07-03.
10. Lawson, Charles L.; J. Hanson, Richard (1 December 1995). Solving Least Squares
Problems (https://books.google.com/books?id=AEwDbHp50FgC). SIAM. ISBN 978-0-
89871-356-5.
11. Press, W.H.; Teukolsky, S.A.; Vetterling, W.T.; Flannery, B.P. (2007). Numerical Recipes
(3rd ed.). Cambridge University Press. ISBN 978-0-521-88068-8.
12. Gelman, Andrew; Carlin, John B.; Stern, Hal S.; Dunson, David B.; Vehtari, Aki; Rubin,
Donald B. (1 November 2013). Bayesian Data Analysis, Third Edition (https://books.google.c
om/books?id=ZXL6AQAAQBAJ). CRC Press. ISBN 978-1-4398-4095-5.
13. Mathworks, 2013. Supported grey box models (http://www.mathworks.com.au/help/ident/ug/s
upported-grey-box-models.html)
14. Hauth, J. (2008), Grey Box Modelling for Nonlinear Systems (https://kluedo.ub.uni-kl.de/files/
2045/diss.pdf) (PDF) (dissertation, Kaiserslautern University of Technology).
15. Nash, J.C. and Walker-Smith, M. 1987. Nonlinear parameter estimation, Marcel Dekker, Inc.
(New York).
16. Whiten, W.J., 1971. Model building techniques applied to mineral treatment processes,
Symp. on Automatic Control Systems in Mineral Processing Plants, (Australas. Inst. Min.
Metall., S. Queensland Branch, Brisbane), 129-148.
17. Whiten, W.J., 1994. Determination of parameter relations within non-linear models, SIGNUM
Newsletter, 29(3–4,) 2–5. 10.1145/192527.192535.
18. Whiten, B., 2014. Determining the form of ordinary differential equations using model
inversion (http://journal.austms.org.au/ojs/index.php/ANZIAMJ/article/view/7767), ANZIAM J.
55 (EMAC2013) pp.C329–C347.
19. Polynomial
20. Spline (mathematics)
21. Kojovic, T., and Whiten W. J., 1994. Evaluation of the quality of simulation models,
Innovations in mineral processing, (Lauretian University, Sudbury) pp 437–446.
ISBN 088667025X
22. Kojovic, T., 1989. The development and application of Model - an automated model builder
for mineral processing, PhD thesis, The University of Queensland.
23. Xiao, J., 1998. Extensions of model building techniques and their applications in mineral
processing, PhD thesis, The University of Queensland.
24. Linhart, H.; Zucchini, W. (1986). Model selection (https://books.google.com/books?id=fzDm3
4Z8ctUC). Wiley. ISBN 978-0-471-83722-0.
25. Miller, Alan (15 April 2002). Subset Selection in Regression (https://books.google.com/book
s?id=V-vLBQAAQBAJ). CRC Press. ISBN 978-1-4200-3593-3.
26. Deming, William Edwards (2000). Out of the Crisis p272 (https://books.google.com/books?id
=LA15eDlOPgoC&pg=PA272). MIT Press. ISBN 978-0-262-54115-2.

Retrieved from "https://en.wikipedia.org/w/index.php?title=Grey_box_model&oldid=1017274915"