Chapter 8

Hybrid Models and Multi-model Data Fusion

Abstract The need for increased accuracy and precision in data-driven models has
motivated the researchers to develop innovative models. Hybrid models and multi-
model ensemble estimations are applied to increase accuracy and precision of
single models. To get an idea about how different models could be combined in a
way to increase each other’s abilities, the chapter begins with a summary on
the characteristics of the models presented in the previous chapters of the book.
The models are compared based on different criteria to give the readers ideas on
how to take advantages of the models’ strengths and avoid their weakness through
the hybrid models and multi-model data fusion approach. The chapter continues
with the examples of hybrid models and general techniques of multi-model data
fusion. The approach of multi-model data fusion contains an important process of
individual model generation which is going to be discussed in the last section of the
chapter.

Keywords Hybrid models • Multi-model data fusion • Stacking • Individual model

generation

8.1 Introduction

The need for increased accuracy and precision in data-driven models has motivated
the researchers to develop innovative models. Actually, hybrid models and multi-
model ensemble estimations are applied to meet these specifications and to inte-
grate strengths of single models. In other words, the main aim of developing hybrid
models is to integrate the advantages of two or more models in a way to improve the
capability of single ones. The hybrid model is usually a combination of models in
series. Conventionally, one model among the others is considered as the main
model, and the others play the role of preprocessing or postprocessing techniques.
Data fusion is an emerging area of research that covers a broad range of application
areas. The principal objective of data fusion is to provide a solution that either is

S. Araghinejad, Data-Driven Modeling: Using MATLAB® in Water Resources 253

and Environmental Engineering, Water Science and Technology Library 67,
DOI 10.1007/978-94-007-7506-0_8, © Springer Science+Business Media Dordrecht 2014
254 8 Hybrid Models and Multi-model Data Fusion

a
Input Model 1 Final
Output 1 Model 2 Output 2 Model 3
output

b
Model 1 Output 1

Input Model Final

Model 2 Output 2
Fusion output

Model 3 Output 3

Fig. 8.1 Schematic algorithms of hybrid models (a) and multi-model data fusion (b) approaches

more accurate or offers additional inferences more than those that could be obtained
through the use of individual data. Data fusion provides new modeling opportuni-
ties in the water resource and environmental fields. Model fusion might benefit
from the ability to combine information derived from multiple sources, such as
the individual outputs from different simulation models. Data fusion researches
are divided into two broad groups. The first takes the view that data fusion is the
amalgamation of raw information to produce an output, while the second makes use
of a more generalized view of data fusion in which both raw and processed
information can be fused into useful outputs including higher-level decision.
Recently, researchers such as See and Abrahart (2001), Abrahart and See (2002),
and Shu and Burn (2004) have used model fusion approaches in hydrological
engineering. See and Abrahart (2001) used data fusion approach for continuous
river-level forecasting where data fusion was the amalgamation of information
from multiple sensors and different data sources. Abrahart and See (2002) evalu-
ated six data fusion strategies and found that data fusion by an artificial neural
network (ANN) model provided the best solution. Shu and Burn (2004) applied
artificial neural network ensembles in pooled flood frequency analysis for estimat-
ing the index flood and the 10-year flood quintiles. The data fusion method was used
to combine individual ANN models in order to enhance the final estimation.
Figure 8.1 demonstrates the general structure of hybrid models as well as the
model fusion approach. As shown in the figure, different individual models in the
process of hybrid models contribute in different levels of the modeling process,
while, in a multi-model data fusion approach, all individual models might have a
chance to contribute in the final output.
The contents of this chapter are structured as presented in Fig. 8.2. As it is
demonstrated in the figure, the chapter begins with a summary on the characteristics
of the models presented in this book. The models are compared in that section based
on different criteria to give the readers ideas on how to take advantages of the
models’ strengths and avoid their weakness through the mentioned models.
The chapter continues with the examples of hybrid models and general techniques
8.2 Characteristics of the Models 255

Fig. 8.2 Structure of this chapter

of data fusion. The approach of multi-model data fusion contains an important

process of individual model generation which is going to be discussed in the last
section of the chapter.

8.2 Characteristics of the Models

There is no general stepwise algorithm for developing a hybrid model. The basic
ideas of hybrid models are usually originated by the experience and knowledge
of the modeler. Of course, the review of the well-known existing hybrid models
could also be very helpful for a modeler to develop his/her own hybrid model.
To understand better the idea of these models and getting ideas on how the
combined models could improve the capability of single ones, it may be useful to
review the characteristics of the most famous data-driven models.
To get an idea about how different models could be combined in a way to
increase each other’s abilities, a summary of the applicable fields where different
models could be applied is shown in Table 8.1. The applicable fields are considered
as event modeling (like the application of linear regression and MLP network as
256 8 Hybrid Models and Multi-model Data Fusion

Table 8.1 Different fields of applications for data-driven models

Models Event modeling Time series modeling Spatial analysis Clustering/classification
ANFIS √ √
ARIMA √
ARMA √
ARMAX √
FCM √
FIS √ √
FR √ √
GRNN √ √ √
IDNN √ √
K-NN √ √ √ √
MAR √
MLP √ √ √
MLR √ √
NLR √ √
PNN √
RBE √ √
RNN √
SVM √ √
TDRNN √ √

discussed in Chaps. 3 and 5), time series modeling (as discussed in Chap. 4), spatial
analysis, and the field of clustering/classification. Furthermore, the selected
strength and weakness of models are reviewed in Table 8.2.
For simplification, the following expression reviews the abbreviation of the
models, which have been reviewed in the previous:
ANFIS Adaptive neuro-fuzzy inference system
ARIMA Autoregressive integrated moving average
ARMA Autoregressive moving average model
ARMAX Autoregressive moving average with eXogenous input
FCM Fuzzy C-means
FIS Fuzzy inference system
FR Fuzzy regression
GRNN Generalized regression neural network
IDNN Input delay neural network
K-NN K-nearest neighbor regression
MAR Multivariate autoregressive model
MLP Multilayer perceptron
MLR Multiple linear regression model
NLR Nonlinear regression model
PNN Probabilistic neural network
RBE Radial basis estimator
RNN Recurrent neural network
SVM Support vector machine
TDRNN Time-delay neural network
8.2 Characteristics of the Models 257

Table 8.2 Strengths and weaknesses of different models

Models Strengths Weaknesses
ANFIS Benefits from the physical concept Needs adequate data to represent different
of a system by if-then rules patterns and rules observed through the
life of a system
ARIMA Can be used for a time series Cannot be used for time series data generation
with a slight trend
ARMA Can model the temporal Cannot be used for a multivariable problem
correlation between data
ARMAX Considers an extra variable Can use only one variable as external variable
except the information of modeling
within a time series
FCM Is one of the best clustering methods Is only applicable to the field of clustering
FIS Can use descriptive data in Does not use a systematic way for calibration
the modeling process
FR Provides outputs by the possibility Is not suitable for nonlinear problems
logic (fuzzy numbers), explicitly
GRNN Offers a straightforward Not much flexible for function approximation
nonparametric regression as it contains only one parameter to be
calibrated
IDNN Can model time series by neural Needs more complicated calibration
network methods
K-NN Could be used for both linear Cannot be used for extrapolation
and nonlinear problems
Does not need any specific distribu-
Needs parameter calibration before being
tion to be applied in a case applied
MAR Can model time series in a multivar-
Becomes too complex in case of higher
iate manner order of the model
MLP Acts flexible in fitting on the data
Does not have a straight forward calibration
algorithm
MLR Provides outputs in a probabilistic Increases estimation error significantly in
manner, explicitly case of increased nonlinearity in the
structure of the problem
NLR Suitable for nonlinear problems Needs real data to have the form of a
well-known mathematical function
PNN One of the best classification models Is applicable to the limited field of classification
RBE Can be used as a spatial model Is not much flexible for function approximation
as it contains only one parameter to be
calibrated
RNN Considers recurrent structures in the Has calibration problems
process of modeling
SVM More robust than neural networks in Some limitations in higher-order problems
some cases
TDRNN Considers both temporal and recur- Contains too many parameters to be calibrated
rent structures in the process comparing to the other networks
of modeling