Multiresolution radial basis model for

nonlinear time series prediction

Cite as: AIP Conference Proceedings 1755, 120002 (2016); https://doi.org/10.1063/1.4958542
Published Online: 21 July 2016

Rukun Santoso, Subanar, Dedi Rosadi, and Suhartono

ARTICLES YOU MAY BE INTERESTED IN

Influence of parameter on the performance of a standing-wave thermoacoustic prime mover

AIP Conference Proceedings 1755, 110004 (2016); https://doi.org/10.1063/1.4958538

Heart sound feature extraction and classification using autoregressive power spectral density
(AR-PSD) and statistics features
AIP Conference Proceedings 1755, 090007 (2016); https://doi.org/10.1063/1.4958525

Partial purification and biochemical characterization of extracellular lipase from Azospirillum

sp. JG3 bacteria
AIP Conference Proceedings 1755, 080003 (2016); https://doi.org/10.1063/1.4958511

AIP Conference Proceedings 1755, 120002 (2016); https://doi.org/10.1063/1.4958542 1755, 120002

© 2016 Author(s).
 Multiresolution Radial Basis Model for Nonlinear Time
Series Prediction
Rukun Santoso1,2, a), Subanar2,b), Dedi Rosadi2,c), and Suhartono3, d)
1
Statistics Department of Universitas Diponegoro, Semarang, Indonesia
2
Mathematics Department of Universitas Gadjah Mada, Yogyakarta, Indonesia
3
Statistics Department of Institut Teknologi Sepuluh Nopember, Surabaya, Indonesia
a)
Corresponding author: rukunsantoso@undip.ac.id
b)
subanar@ugm.ac.id
c)
dedirosadi@ugm.ac.id
d)
suhartono@statistika.its,ac,id

Abstract. The multiresolution radial basis autoregressive model for nonlinear time series prediction is proposed in this
paper. This is a development form of the multiresolution autoregressive model. This constitutes an alternative method for
nonlinear time series prediction, especially for threshold model. In the beginning, time series is decomposed by wavelet
transform to produce smooth part and detail part. The both are representing the main pattern and noise pattern of time
series, respectively. Some radial basis functions are performed to refine the decomposition result into homogenous
clusters. This refining result becomes an input series for a neural network structure. The model is applied to make one
step prediction ahead of a simulation data. The normality test for residual is accepted, and the statistic test shows that the
model coefficients are significant.

INTRODUCTION
Time series modeling is one of the important tools in economic data analysis. It grows up continuously after the
historical book “Time Series Analysis: Forecasting and Control” by Box and Jenkins [1]. This book discussed AR
model, MA model and the development of AR and MA as well as ARMA, ARIMA, and SARMA models. The
models are included in the linear class model. In the fact, sometimes be found out that the linear class model is not
supporting the time series data properties, namely, when the data has nonlinear properties and the variance changes
over time (heteroscedastic). There are some proposed models to cover non-linearity and heteroscedastic problem.
The threshold autoregressive (TAR) model can be implemented when the time series has several values of the mean
process. The model assumes that the process jumps in some alternative functions which refer to the threshold
domain value [2].
Engel proposed the autoregressive conditional heteroscedasticity (ARCH) model which can be implemented
when the variance of error process changes over time [3]. The model consists of the mean model and variance
model. The ARCH model was developed by Bollerslev [4] called as GARCH (Generalized ARCH) model. To find
out non-linearity existence becomes an important step before decides to use nonlinear model. The test with neural
network approximation was proposed by Teräsvirta et al. [5] and the neural network approximation with random
weights test was proposed by Lee et al. [6].
Sometimes, the non-parametric class models are chosen over the parametric class for some reason. The recently
class of non-parametric model is including neural network model, wavelet model, fuzzy model and mixture (hybrid)
of these models. Haykin [7] was introducing wide discussion of neural network model. The basic of wavelet neuro
model can be sought in Murtagh et al. [8]. The study of the wavelet-fuzzy model can be sought in Popoola [9].

Advances of Science and Technology for Society

AIP Conf. Proc. 1755, 120002-1–120002-7; doi: 10.1063/1.4958542
Published by AIP Publishing. 978-0-7354-1413-6/$30.00

120002-1
Usually, the logical sense of non-parametric method is simpler but involving a high quantity of numerical
computation. Fortunately, there are some computer software and hardware to fulfill this necessity.
The development form of the multiresolution autoregressive (MAR) model is proposed in this paper. The model
constitutes an alternative solution for time series consists of jumping pattern. Usually, threshold model is chosen to
handle the problem. The new model uses wavelet, radial basis function and Neural network structure is introduced.
The role of the wavelet is decomposing an original function into the main pattern (smooth) and noise pattern
(fluctuating). The radial basis function overtakes the role of threshold function. Meanwhile, the neural network
structure gives a chance to improve the parameter estimation. The basic model refers to wavelet neuro model
proposed by Murtagh et al. [8].
The wavelet coefficients selected from each level become to inputs for radial basis nodes. The model is called as
a multiresolution radial basis autoregressive (MRBAR) model. It has been proved that wavelet has the capability to
reconstruct an obscure function caused by random noise [10]. The radial basis function with univariate inputs has
been used for approximating nonlinear function in the neural network structure [7]. The MRBAR model will
combine the role of the wavelet, radial basis and-and neural network simultaneously in one structure. R software
[11] supported with wavelets package [12] is used for computations needs.

REVIEW OF NONLINEAR TIME SERIES MODEL

The early mathematical model of time series proposed by Box and Jenkins [1] was called as Autoregressive
Moving Average (ARMA) model. The best result will be reached when the process fulfills the model assumption i.e.
stationary condition. In common, differentiation process is used for obtaining stationary condition. Equation (1)
shows the general form of ARMA
௣ ௤
ܻ௧ ൌ ߤ ൅ σ௜ୀଵ J௜ ܻ௧ି௜ ൅ σ௝ୀଵ T௝ H ൅ H௧ (1)
௧ି௜

where ߝ௧ is a normal random variable with zero mean and constant variance.
It’s found out that many of processes can’t be modeled satisfactorily by linear model class. Recently, an effort to
discover new nonlinear model becomes an interesting work of researchers. The autoregressive conditional
heteroscedastic (ARCH) model was proposed by Engel [3]. The residuals of ARMA model are independent, have
zero mean and constant variance. Otherwise, the residuals of ARCH model (called as innovation) are nonlinear
independent, have zero mean and the variance change over time. The development form of ARCH model was
proposed by Bollerslev [2]. He has developed the ARCH model into GARCH (Generalized ARCH) model. The
heteroscedastic properties can be discovered by Lagrange multiplier (LM) test proposed by Breusch and Pagan [13]
and developed by Lee et al. [6].
The Eq. (2) shows the other form of the nonlinear model.
୮
୲ ൌ Ɋ ൅ σ୧ୀଵ Z୧ )୧ ሺ୲ି୧ ሻ ൅ H୲ (2)
In this case, the dependencies of variables in a nonlinear sense become the center of interest. There are some kinds
of nonlinear function which can be supplied for Ȱ as well as logistic function, an exponential function, high
polynomial, radial basis function, wavelet, and many others. Haykin [7] has made a wide review about the use of
radial basis function in the nonlinear model, and special notes can be found in Orr [14, 15]. This paper discusses the
development of Eq. (2) when the argument of Ȱ is substituted by wavelet transformation form.

MULTIRESOLUTION AUTOREGRESSIVE MODEL

There is a small wave function able to build orthonormal basis for ଶ ሺሻ space. This function is called as mother
wavelet or be abbreviated by wavelet. The good basic discussion about wavelet can be found in Daubechies [16].
Every wavelet has a unique couple called as father wavelet or scaling function. In usual, ߰ and ߶ symbols are used
for wavelet and father wavelet. This couple of wavelets build wavelet family through translation and dilation
operation.
The mother and father wavelets work together to construct a multiresolution space through a wavelet
transformation process. As the consequence, every function ݂ଶ ሺሻ can be manifested by linear combination of

120002-2
dilation-translation form of wavelets. This expression at resolution level of ‫ܬ‬, where ‫ ܬ‬is fixed integer, can be
described by Eq. (3).
݂ ሺ‫ݐ‬ሻ ൌ σ௞ఢ௓ ‫ݏ‬௃ǡ௞ ߶௃ǡ௞ ሺ‫ݐ‬ሻ ൅ σ௝ஸ௃ σ௞ఢ௓ ݀௝ǡ௞ ߰௝ǡ௞ ሺ‫ݐ‬ሻ (3)
ൌ ܵ ሺ‫ݐ‬ሻ  ൅ ‫ܦ‬ሺ‫ݐ‬ሻ
The main pattern of the function is consisted in the smooth part ܵሺ‫ݐ‬ሻ and the fluctuation rhythm is consisted in
the detail part ‫ܦ‬ሺ‫ݐ‬ሻ. In this case the domain function is time ܶሺ‫ܶݐ‬ሻ and the range function is real number
ܴሺ݂ሺ‫ݐ‬ሻܴሻ. Some wide discussion about basic wavelet application in statistics can be found in Ogden [10].
When the function values are discretely observed with unit increment, then the result constitute a set of time
series data. The expression of Eq. (3) still valid for time series data refers to discrete wavelet transform. The
comprehensive study about discrete wavelet transform and its variation types can be explored in Percival and
Walden [17].
Murtagh et al. [8] proposed a model familiar called as a multiresolution autoregressive model (MAR). The
similar research also can be found in Renaud et al. [18]. This procedure can be used for one step ahead time series
prediction. General form of prediction model is described in Eq. (4), and special model for ‫ ܬ‬ൌ Ͷ and ȁ‫݆ܣ‬ȁ ൌ ʹ for
all ݆ is expressed in Eq. (5)
ห஺ೕ ห ห஺಻శభ ห
ܻ෠௧ାଵ ൌ σ௃௝ୀଵ σ௞ୀଵ ܽො௝ǡ௞ ݀௝ǡ௧ିଶೕ ሺ௞ିଵሻ ൅ σ௞ୀଵ ܽො௃ାଵǡ௞ ܿ௃ǡ௧ିଶ಻ ሺ௞ିଵሻ (4)
෠
ܻ௧ାଵ ൌ ܽොଵǡଵ ݀ଵǡ௧ ൅ ܽොଵǡଶ ݀ଵǡ௧ିଶ ൅ ܽොଶǡଵ ݀ଶǡ௧ ൅ ܽොଶǡଶ݀ଶǡ௧ିସ ൅
ܽොଷǡǡଵ ݀ଷǡ௧ ൅ ܽොଷǡଶ ݀ଷǡ௧ି଼ ൅ ܽොସǡଵ ݀ସǡ௧ ൅ ܽොସǡଶ ݀ସǡ௧ିଵ଺ ൅
ܽොହǡǡଵܿସǡ௧ ൅ ܽොହǡଶܿସǡ௧ିଵ଺ (5)

The parameters estimation of Eq. (5) can be carried out by the least square method or maximum likelihood method.
Although MAR model can capture nonlinear properties in some cases, sometimes the normality test of residuals is
rejected especially for clustered data. It will decrease the level of parameters significancies test.

FIGURE 1. Plot of Time Series Simulation Data

The time series data consist of three clusters was generated by computer assistance. The stats package of R
software was used for generating simulation data [11]. The series represent to the white noise process with mean
ߤ ൌ Ͳ and variance ߪ ଶ ൌ ʹͷ in the first 50 points, with mean ߤ ൌ ʹͲ and variance ߪ ଶ ൌ ͳ in the next 50 points, and
with meanߤ ൌ ͳͲ and variance ߪ ଶ ൌ Ͷ in the last 50 points. The white noise process is chosen by the reason that it
is usually more difficult to be approximated by MAR model [19], and will be tried to be approximated by MRBAR
model.
The plot of simulation time series data is shown in Fig. 1. Table 1 shows the statistics summary of MAR model
estimation for this data a. The Kolmogorov-Smirnov test indicates that the residuals normality assumption is
rejected at the 0.05 probability level. This summary also shows that only 4 predictor variables are significant at this
level.

120002-3
 TABLE 1. The Statistics Summary of MAR Model for Simulation Data
Variable ෝ ࢏ǡ࢐
ࢻ Std. Error t Value Prob.
݀ଵǡ௧ିଶ -0.06158 0.08030 -0.767 0.444598
݀ଵǡ௧ 1.03690 0.08047 12.886 0.000000
݀ଶǡ௧ିସ -0.02659 0.17719 -0.150 0.880939
݀ଶǡ௧ 0.93253 0.21058 4.428 0.000000
݀ଷǡ௧ି଼ 0.04716 0.24008 0.196 0.844588
݀ଷǡ௧ 0.81038 0.22727 3.566 0.000518
݀ସǡ௧ିଵ଺ 0.21248 0.18323 1.160 0.248448
݀ସǡ௧ 0.23464 0.18432 1.273 0.205420
ܿସǡ௧ିଵ଺ 0.15452 0.14707 1.051 0.295489
ܿସǡ௧ 0.31170 0.15507 2.010 0.046612
Residual standard error 3.814 on 123 degrees of freedom
Multiple R-squared 0.9295, Adjusted R-squared 0.9238
F-statistic 162.3 on 10 and 123 DF. , p-value < 2.2 e-16
Kolmogorov-Smirnov tests for residuals normality
D = 0.1459 p-value = 0.006927

MULTIRESOLUTION RADIAL BASIS AUTOREGRESSIVE MODEL

The development of MAR model is proposed in this paper. The model is called as multiresolution radial basis
autoregressive model abbreviated by MRBAR. This name is referred to as the radial basis function addition to the
MAR model. There are three popular kinds of radial basis type i.e. Gaussian type(G), Multiquadrics type (M) and
Multiquadrics Inverse (MI) type [7]. The architecture of MRBAR model can be shown in Fig. 2. The input variables
of MRBAR model is similar to MAR model described in Eq. (5). There are two layers consisted in the model. The
first layer represents a nonlinear process performed by radial basis function. The number of radial basis node in
every input line is equivalent to the number of clusters occurred. The mean and variance clusters are estimated to the
inputs membership. The second layer represents a linear process performed by linear summation function. The
mathematical form of MRBAR estimation model can be shown in Eq. (6). The parameters estimation can be
calculated by the least square method or maximum likelihood method.
෡୲ାଵ ൌ σସ୨ୀଵ σଶ୩ୀଵ σ௤ೕǡೖ ƒො ୨ǡ୩ǡ୪ Ȱ௝ǡ௞ǡ௟ ቀ݀୨ǡ୲ିଶౠ ሺ୩ିଵሻቁ ൅ σଶ୩ୀଵ σ௤ఱǡೖ ƒොହǡ୩ǡ୪ Ȱହǡ௞ǡ௟ ൫…ହǡ୲ିଶర ሺ୩ିଵሻ൯
 (6)
௟ୀଵ ௟ୀଵ

Table 2 shows the statistics summary of MRBAR model approximation for simulation data. The combination of
node numbers and radial basis type is run ladder for every transformation level. There has not a standard regulation
yet to choose this combination. Figure 3 shows a plot of simulation data (black), MAR prediction (green) and
MRBAR prediction (red). The F-statistics of MAR model and MRBAR model indicate that the both model good
enough to fit the data. But the MRBAR model has excessive result refer to residual normality.

120002-4
 FIGURE 2. The Architecture of MRBAR Model

TABLE 2. Statistics Summary of MRBAR Model for Simulation Data

Variable Node RBF Type ෝ ࢏ǡ࢐
ࢻ Std. Error t Value Prob
݀ଵǡ௧ 1 M 3.770 0.5835 6.474 0.0000
ܿସǡ௧ିଵ଺ 1 G 13.4989 1.0608 12.726 0.0000
2 5.9926 1.0959 5.468 0.0000
3 3.0158 1.3165 2.291 0.0236
ܿସǡ௧ 1 G 2.8379 1.3116 2.164 0.0324
2 9.8058 1.4811 6.651 0.0000
3 -14.6080 1.3663 -10.692 0.0000
Residual standard error 4.006 on 126 degrees of freedom
Multiple R-squared 0.9204, Adjusted R-squared 0.9159
F-statistic 208 0n 7 and 126 DF. , p-value < 2.2 e-16
Kolmogorov-Smirnov tests for residuals normality
D = 0.1096 p-value = 0.08184

120002-5
 FIGURE 3. Plot of Simulation Data and Its Predictions

CONCLUSION
The choice of the initial mean cluster in k-means procedure plays the role to the goodness of final model.
Computer package may become the provisional choice as be done in this paper. Some combination of nodes and
radial basis type are a run ladder to capture the best result. However, the expertise may be necessary to be sharpened
to choose the finest value. The type of radial basis function also needs to be matched to the data properties. A perfect
method for all data condition is nothing. Although this simulation data can be approximated well by MRBAR
model, its excitement needs to be compared to another model. Finally, the advancement of MRBAR model still
opens to be observed.

ACKNOWLEDGMENT
This research was supported by Kementerian Riset, Teknologi dan Pendidikan Tinggi, Republic of Indonesia.

REFERENCES
1. G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control, (Holden-Day, San Francisco,
1976).
2. H. Tong, Nonlinear Time Series: A Dynamic System Approach, (Clarendon Press, Oxford, 1990).
3. R F Engel, J. Econometrica 50, 987–1008, (1982).
4. T. S. Breusch and A. R. Pagan, Econometrica 47, 1287–1294 (1979)
5. Timo Teräsvirta, C. F. Lin, and C. W. J. Granger, J. of Time Series Analysis 14, 209–220, (1993).
6. T. H. Lee, H. White, and C. W. J. Granger, J. of Econometrics 56, 269–290 (1993).
7. S. Haykin, Neural Networks: a Comprehensive Foundation, (Prentice Hall, 1999), pp.256-315.
8. Murtagh, J-L Starck, and O Renaud, Decision Support System 37, 475–490, (2004).
9. A. O. Popoola, “Fuzzy wavelet method for time series analysis”, Ph.D. thesis, Surrey University 2007.
10. R. T. Ogden, Essential Wavelets for Statistical Applications and Data Analysis, (Birkhauser, Berlin, 1997).
11. R Core Team, “R a language and Environment for Statistical Computing”, User Guide, (http://www.R-
project.org/, 2014).
12. E. Aldrich, Wavelets: “A Package of Functions for Computing Wavelet Filters, Wavelet Transforms, and
Multiresolution Analyses”, User Guide, (http://CRAN.Rproject.org/package=wavelets, 2013)
13. T. S. Breusch and A. R. Pagan, Econometrica 47, 1287–1294 (1979)

120002-6
14. Mark J. L. Orr, Introduction to Radial Basis Function Networks, (Centre for Cognitive Science University of
Edinburgh, 1996)
15. Mark J. L. Orr, Recent Advances in Radial Basis Function Networks, (Centre for Cognitive Science, University
of Edinburgh, 1999).
16. I. Daubechies, Ten Lectures on Wavelets, (SIAM, Philadelphia, 1992).
17. D. B. Percival and A. T. Walden, Wavelet Methods for Time Series Analysis”, CU Press, Cambridge, 2000.
18. O. Renaud, J. L. Starck, and F. Murtagh, Int. J. of Wavelets Multiresolution and Information Processing,
1(2):217–232, (2003).
19. Santoso Rukun, Subanar, D. Rosadi, and Suhartono, J. Phys.: Conf. Ser, 423, 012010 (2013).

120002-7