Content

 Customized Optimization Criteria

 Forecasting
 Real-time signal extraction
 Forecasting

A Practical Example: ESI

A Simulated Example: one- vs multi-step
NN3-Competition: Customized Criteria
 The Economic
Sentiment Indicator
ESI (EFN Report)
Comments
 Forecasting-model:
 Integrated process
 Forecasting intervals spread out rapidly
 Point-forecasts do not converge to the mean
 Series is bounded: integration=misspecification
 The 40%-Interval (and a fortiori higher confidence
intervals) contains both trend directions
 Impossible to infer the occurrence of a turning-point
 Forecasts are uninformative
An Artificial Example
(Dynamics Close to
Business Surveys)
Model-Misspecification
Multi-Step Ahead Forecasting
Artificial Time Series (close to KOF-
Economic Barometer)
X t  Ct  St  I t  3* cos( / 20 * t )  cos( / 6 * t )   t
Forecasting-Model and Diagnostics
(1  B)(1  B12 ) X t  (1  0.36 B )(1  0.99 B12 ) t
   1.16
Problems
 In applications TRAMO and/or X-12-ARIMA
often identify airline-models
 Misspecification cannot be detected
 One-step ahead forecasts are good
 σ=1.16 (true innovations are N(0,1) )
 What about multi-step ahead performances?
Multi-step ahead Forecasts
0 months after TP1 of cycle
Multi-step ahead Forecasts
6 months after TP1 of cycle
Multi-step ahead Forecasts
1 year after TP1 of cycle
Multi-step ahead Forecasts 20 months
after TP1 and 0 months after TP2
Multi-step ahead Forecasts
3 months after TP 2
Multi-step ahead Forecasts
6 months after TP 2
Comments
 One-step ahead forecasts are good
 σ=1.16
 Poor multi-step ahead performance
 Huge delays
Multi-step ahead 95% Interval-
Forecasts: 6 months after TP2
Multi-step ahead 50% Interval-
Forecasts: 6 months after TP2
Comments
 Forecast intervals spread out much too
rapidly
 It is impossible to assert the occurrence of
TP’s
 even 50%-intervals are completely uninformative
 NN3
Customized Criterion
 Receive updates:

 www.neural-forecasting-competition.com
Objectives
Forecast a set of 111 economic time series as accurately as possible, using
methods from computational intelligence and a consistent methodology. We
hope to evaluate progress in modelling neural networks for forecasting & to
disseminate knowledge on “best practices”. The competition is conducted for
academic purposes and supported by a grant from SAS & the International
Institute of Forecasters (IIF).
Methods
The prediction competition is open to all methods of computational intelligence,
incl. feed-forward and recurrent neural networks, fuzzy predictors, evolutionary &
genetic algorithms, decision & regression tress, support vector regression, hybrid
approaches etc. used in financial forecasting, statistical prediction, time series
analysis
 Competitors
 Theta-model (winner of M3)
 Forecast-Pro (best commercial package M3)
 Autobox (ARIMA-based high-performer)
 X-12-ARIMA
 Latest neural networks designs
 …
Rank on CONFERENC
SM E
AP PRESEN
E Participant SMAPE TATION DESCRIPTION

- Stat. Contender - Wildi 14,84%

- Stat. Benchmark - Theta Method (Nikolopoulos) 14,89% description missing

1 Illies, Jäger, Kosuchinas, Rincon, Sakenas, Vaskevcius 15,18%

- Stat. Benchmark - ForecastPro (Stellwagen) 15,44%

presentation
- CI Benchmark - Theta AI (Nikolopoulos) 15,66% missing
description missing

- Stat. Benchmark - Autobox (Reilly) 15,95%

2 Adeodato, Vasconcelos, Arnaud, Chunha, Monteiro 16,17%

presentation
3 Flores, Anaya, Ramirez, Morales 16,31% missing

presentation
4 Chen, Yao 16,55% missing

5 D'yakonov 16,57%

6 Kamel, Atiya, Gayar, El-Shishiny 16,92%

7 Abou-Nasr 17,54%

8 Theodosiou, Swamy 17,55%

 not disclosed
21,05
29 C49 by
% author

21,48
Stat. Benchmark - X12 ARIMA (McElroy)
%

not disclosed
24,03
30 C35 by
% author
Summary
 Well-designed (customized) optimization
criterion performs best
 Prototypical package
 NN3-series were the first series passed through
the code
 No experience, limited time
 2 weeks for code implementation and
computations
 We expect substantial `fine-tuning’-potential
A Methodological Essay for the ESI
 Forecast the series by focusing on TP’s
 Traditional ARIMA-based approaches perform
worst in TP’s
 Real-time signal extraction
 Relevant customized criteria
 Real-time TP-filter
Real-Time Signalextraction
Research Purpose, Signal and Real-
Time Problem
MBA
DFA
Example: Economic Sentiment
Indicator
Research Purpose,
Signal and Real-
Time Problem
Research Purpose
 Anticipate TP’s (GDP,ESI,…)
 Data: selected KOF-Business-surveys
 Questionnaire - and hence data - are informative
about growth-rate (not level)
 Data is contaminated by noise and
seasonal components
 Signal:
 Eliminate noise and seasonal components
Business-cycle frequencies (cutoff 1.5 y.)

 Research purpose:
anticipate TP’s of GDP
 Signal:
 Trend = output of the
transfer function on the
right hand side.
 Signal definition matches
research purpose
Symmetric MA- and Real-Time Filters

 60
Tt   k X t k TN  60  
k 60
k X N  60  k
k 

 k   k
60 1
TˆN    k X N  k   k Xˆ N  k
k 0 k 60
MBA’s
Traditional Approach
(TRAMO/SEATS, X-12-ARIMA)
 Identify a time series model for the DGP
 Forecast the future
 One- and multi-step forecasts
 Apply the symmetric filter to the extended
time series
Problem 1:
Model-Misspecification
•Business survey data: bounded time series
Series nmb. Models d Series nmb. Models d
1 (210)(011) 2 19 (100)(011) 1
2 (011)(011) 2 20 (011)(011) 2
3 (011)(011) 2 21 (011)(011) 2
4 (011)(000) 1 22 (011)(000) 1
5 (210)(100) 1 23 (100)(000) 0
6 (110)(000) 1 24 (010)(001) 1
7 (100)(011) 1 25 (011)(011) 2
8 (011)(011) 2 26 (013)(001) 1
9 (011)(011) 2 27 (011)(011) 2
10 (121)(011) 3 28 (110)(000) 1
11 (110)(011) 2 29 (011)(011) 2
12 (011)(000) 1 30 (012)(000) 1
13 (300)(011) 1 31 (011)(011) 2
14 (011)(001) 1 32 (011)(000) 1
15 (011)(011) 2 33 (010)(011) 2
16 (110)(000) 1 34 (010)(011) 2
17 (011)(011) 2 35 (011)(011) 2
18 (011)(011) 2 36 (112)(000) 1
Problem 2:
multi-step forecasting performance
 Models perform well with respect to short-
term (one-step) forecasting
 Poor multi-step ahead performance
 Performance particularly poor in TP’s
 Mean-reversion is not accounted for by
misspecified integrated processes
 TP’s cannot be accounted for explicitly by
traditional MBA’s
 DFA
Customized Criteria
 DFA: new `Customized’ Optimization
Criteria
 Level criterion: N /2
min ˆ  | (k )  
( ) |2 I ( )
k NX k
 TP criterion : k 1

 2 [T / 2]
 T  k 0 W ( ) 2
|  ( )  ˆ ( ) |2 I ( )
A
k k k NX k
min ˆ 
 2  [T / 2] 2W ( ) 2 A( ) Aˆ ( )(1  cos( ˆ ( )))I ( )
 T k 0 k k k k NX k

 Book, chapters 3-5

DFA TP-criterion
 Speed and reliability of the real-time filter can be
accounted for explicitly
 λ: speed
 W(ω): reliability (smoothness)
 Improved performance specifically in TP‘s
 User preferences (risk-aversion) can be
accounted for explicitly
 Both criteria match exactly the structure of the
relevant estimation problems
 Efficiency
 ESI
DFA versus Dainties
 90.000000000
95.000000000
100.000000000
105.000000000
110.000000000
115.000000000
Apr 96
Jul 96

Okt 96

Jan 97
1996:2001

Apr 97
Jul 97

Okt 97

Jan 98

Apr 98
Jul 98

Okt 98

Jan 99
Apr 99

Jul 99

Okt 99

Jan 00
Apr 00

Jul 00
DFA is both fast and reliable

Okt 00

Jan 01
Apr 01

Jul 01
DFA
Dainties
Towards the current boundary
Recall ESI-Forecasts (EFN Report)
106.000000000

104.000000000

102.000000000

100.000000000
DFA
Dainties
98.000000000

96.000000000

94.000000000

92.000000000
Summary
 Very effective detection of TP’s
 Very fast (delay zero)
 Very reliable (no false alarms between
consecutive TP’s)
 TP-filter can neatly improve over traditional
ARIMA-based forecasts
Amplitude and time delay (DFA)

7.00E+00 2.50E+00

6.00E+00
2.00E+00
5.00E+00

4.00E+00
1.50E+00
Delay
3.00E+00
Amplitude
1.00E+00
2.00E+00

1.00E+00
5.00E-01
0.00E+00
2π/6

3π/6

4π/6

5π/6
π/6

π
0

-1.00E+00 0.00E+00
Response by the EC

 The approach of the European Commission to adjusting the

business and consumer surveys (BCS) has always been
seasonal adjustment, not trend/cycle extraction, i.e.
smoothing.

 The irregular component carries information on respondents‘

perception of economically relevant special events such as
strikes, elections or strong exchange rate or commodity price
movements, we believe that retaining the component is
important in interpreting the data.

 To my knowledge, the seasonal adjustment approach is also

the dominant, if not exclusive, approach used by the national
survey institutes…
Response by the EC

 Of course, the real-time trend/cycle extraction approach

of your DFA filter is very relevant in its own right for the
identification of the cycle and its turning points.
Therefore, your work is very interesting and stimulating.

 The calculation and publication of such smooth

indicators representing the trend/cycle component is
not among our priorities for the nearer future, we would
still be interested in learning more about the possible
advantageous or complementary features of your DFA
approach with respect to our traditional approach of
processing the BCS data.
Non-Response by contributors to
EFN-Report
 The relevant forecasting issues ere
addressed several times to all contributors
 IGIER, CEPII, IWH, EUI, AQR, IFL, DAE,
 Coordinator of report
 Book, NN3, articles
 No response!
 Typical monopolistic behavior
 Lack incentives and/or time to scrutinize
methods and/or to improve performances
 Available Material
 Theory:
• latest book “Real-Time Signal Extraction:
Beyond Maximum Likelihood Principles”
• `Open source’ pdf-vintages: www.idp.ch
 On-going project
 Software:
• R-package signalextraction
• CRAN-depository: cran.r-project.org
 Questions/remarks: marc.wildi@zhaw.ch