Lecture 1: Introduction

We focus on techniques for analyzing time series data, and forecasting.

- A time series is a time stamped sequence of observations on a variable.

What is this class about?

- offers a survey of main statistical methodologies for analysis of business time series data
- provides the tools necessary to extract information required for specific tasks such as forecasting, quantifying prediction
uncertainty.
- Emphasis will be given to business applications of predictive analytics methods using modern software tools.
- Emphasis on state-of-the-art forecasting methods, recent research in this area

References
- Forecasting: Methods and Applications, 3rd Edition, 1998 by Makridakis, Wheelwright, and Hyndman, Wiley, ISBN
0-471-53233-9. (MWH)
- Business Forecasting, 9th edition, 2008 by Hanke and Wichern, ISBN 0-13-500933-2.
- Forecasting practice and Process for Demand Management, 2006 by Levenbach H. and Cleary, J. P., ISBN0-534-26268-6,
Thomson.
- Applied Econometric Time Series, by Walter Enders, 2nd Edition, Wiley 2004.
- Introduction to Time Series and Forecasting, 2nd edition, 2002 by Brockwell, P. J., and Davis, R. A. Springer-Verlag: New York,
ISBN 0-387-95351-5.

Notes
- Blue: first introduced, red/ italic: emphasized, *:technical but not tested

Data Sources
Australian Data US Data

Data Libraries Yahoo Finance Database

OZDasl - Australian Data and Story Library Data and Story Library —CMU
ANU Social Science Data Archives Bureau of Economic Analysis
University of Sydney Library Databases Collection Original Data Economagic - times series data
Sources FRED - Federal Reserve Economic Data
Reserve Bank of Australia White House - Economic Statistics Brieng Room
Australian Bureau of Statistics NBER - National Bureau of Economic Research
Penn World Tables ? Australia NBER - Marriage and Divorce Data
Time Series Data Library (Hyndman)
Datastream International ICensus - Statistical Abstracts Census Data
ICPSR - Interuniversity Consortium for Social and Political
Research
Panel Survey of Income Dynamics (PSID)
Bureau of Labor Statistics
Survey of Income and Program Participation (SIPP)
National Center for Health Statistics
Statistics in Sport (American Statistical Association)

Types of problems: regression, classification

Types of data
- time series data
- cross-sectional data (same time point, different features)
Types of forecasting
Qualitative Delphi method
(judgmental) - Assumption: forecasts from a group is more accurate than those from individuals;
forecasting - Stages: forming panel, setting tasks, initial expert views, feedback to experts, aggregating
expert views for forecasting
1. Panel of experts BUT never meet as a group.
2. Use questionnaire to obtain forecasts from all participants
3. Summarize and re-distribute results to all participants PLUS appropriate new questions
4. Summarize again, refining forecasts and conditions, and developing new questions.
5. Repeat Step 4 if necessary. Distribute the final results to all participants.

Expert opinion/ judgment (subjective): lack historical data, used in conjunction with data-based
forecasting. Subjectively extending previous patterns into future

Quantitative Based on historical data, use formal econometric or statistical forecasting methods.
(data based) Project previous patterns into the future using a statistical model.
forecasting Focus:
Time series modeling forecasts.
- A class of forecasting techniques based on time series data analysis: AR, ARMA, Recurrent
neural networks, etc.
- Project previous patterns into future using a formal statistical model
- Only concerned with forecasting, not reasons why the variable changes
Regression modeling forecasts.
- A class of forecasting techniques based on regression modeling
- Use a formal statistical regression model
- Can assess other quantities related to target changes and scenario forecasting

Combination: judgmentally adjusted statistical forecasting

The process of forecasting

1. Formulation of the business problem
- Understand what to solve and what to forecast (can it be forecasted?), forecast usage.
- Principles: Use experts' knowledge/previous studies to examine if forecasting is considered possible.
Use theory to guide the search for possible explanatory factors.
Communicate with all involved in data collection, decision making, etc.
2. Gathering information
- What info is needed and available? Where to find it? customer/ transaction database
- Comparing the cost and benefits in data collection investment
3. Preliminary data analysis
- Convert raw data into the required forms (numerical, categorical, tabular)
- Cleaning data, checking typos and unusual observations
- Visualize the data to get some insights
4. Choose and test models.
- Appropriate models: Occam’s Razor principle (no over complex model)
Combine models and predictions
5. Using and evaluating a forecasting model (forecast, assess forecasts, and implement decision making).
- Put the forecasts into real use in order to realize some return on investment

Elements of a Good Forecast

Timely + Reliable + Accurate + Meaningful + Written + Easy to use

Forecasting techniques require good data, proper understanding and validation.

Forecaster’s creativity, business knowledge, common sense and human judgment are essential for their success.
Lecture 2: Time series components, naive forecasting methods and prediction accuracy
(Reading: Chapter 2 & 3)
Data type

Data Graphing
- Visualize features: trend, correlation, relationship
- Give insights into the data
- Data type determines the graphing technique: time series and scatter plot
- Good graphs convey both patterns and randomness (message + noise) data analyst removes noise.

Python plotting
- Import matplotlib.pyplot as plt
- 1. Prepare data” loading data from file, processing and computing
2. Define a drawing window: size, subplots (default setting: plt.plot() )
3. Use the main plotting functions plot and/ or scatter etc.

Time Series Components:

Trend Reflects the long-run growth or decline in the time series. Trend may depend on the length of the
observed time series

Cycle Slow rises and falls that are not in a regular repeating pattern, no fixed period!

Seasonal variations Rises and falls that are in a regular repeating pattern, on a seasonal basis such as months of the
year or days of the week. (fixed period M is unchanging!)
Fixed seasonal period/ frequency is denoted by M

Irregular fluctuations Follow no visualizable pattern, need statistical models to capture, assumed unexplainable
Might be ’unusual’ events: earthquakes, accidents, hurricanes, wars, strikes
OR just random variation i.e. noise!
(after the seasonal component is removed, see the fluctuations)

Notation
- A time series of length T 𝑇 = {𝑌1, 𝑌2, 𝑌3,..., 𝑌𝑇}, 𝑜𝑟 𝑌1:𝑇 observations

- The forecast of 𝑌𝑇+ℎ based on data 𝑌1:𝑇 is 𝑌 𝑇+ℎ|𝑇

, h is horizon, h-step-ahead forecast.

Prediction methods:
1. Naїve forecasting method: Most Recent Value
2. Seasonal Naїve Method: Most Recent Season’s Value
3. Drift Method ( )
- A variant of the na ̈ıve method, allowing the changes in the forecasts over time
- The amount of change over time, called drift, is the average change seen in the historical data.

- This is equivalent to drawing a line between the first and last observation, and use that line to forecast for times after
T.
Prediction Errors and Measures
- All measures can be used simultaneously. Report the ones that can be best understood.
- If there are 1 or 2 large undesirable errors, use MSE. IF not then use MAD or MAPE
- MAD, MSE, RMSE and MAPE are suitable for numerical data.
- For categorical or direction forecasting, use percentage agreement (disagreement)

Mean Absolute Deviation (MAD) average distance between actual and forecast, i.e. average
forecast error.
- same units as Y
- Does not heavily penalize a small number of large errors

Mean Squared Error (MSE) - Harder to interpret

- Heavily penalizes large errors
- RMSE has same units as Y

Root Mean Squared Error

(RMSE)

Mean Absolute Percentage Error - measures percentage error

(MAPE)

Evaluating forecast accuracy

(how well a model performs on new data)
- Training and test sets: fit well not necessarily forecast well, we can always add parameters to fit perfectly.
Occam’s Razor: The simpler the better, overfitting can fail to identify the systematic pattern in the data.
- The size of the test set should ideally be at least as large as the maximum forecast horizon required.
Time series references often call the training set the “in-sample data” and the test set the “out-of-sample data”
- Cross-validation: model selection method.
1. For Cross-Sectional Data: leave one out as test set, and repeat T times (observation volume)
Compute the forecast accuracy measures based on the errors obtained

2. Forward Chaining
There are T total observations, and we wish to use at least K previous observations to build the model
(suppose T=10, K=3)
Use Y1, Y2, Y3 to build the model to predict Y4, producing e4
Use Y1, Y2, Y3, Y4 to build the model to predict Y5, producing e5
Use Y1, Y2, Y3, Y4, Y5 to build the model to predict Y6, producing e6
Use Y1, Y2, Y3, Y4, Y5, Y6, Y7, Y8, Y9 to build the model to predict Y10, producing e10
Assess the overall errors e4, e5, ..., e10 for the modeling method.

3. For time series (one-step-ahead forecast)

Given time series Y1:T , use at least k observations for constructing forecasts, then select the best model.

4. For time series (h-step-ahead forecast)

Given time series Y1:T , use at least k observations for constructing forecasts.
Lecture 3: Time Series Decomposition
(Reading: Online textbook Ch. 6; BOK section 6.3 and Ch. 7)
(how to extract the trend and seasonal components)

Time Series Decomposition

Decompose a time series into 4 components: 𝑦𝑡 = 𝑓 (𝑇𝑡 , 𝐶𝑡 , 𝑆𝑡 , 𝑅𝑡 )
- The goal: to understand and interpret the time series and improve forecast accuracy.
- We combine the cycle component into trend: 𝑦𝑡 = 𝑓 (𝑇𝑡 , 𝑆𝑡 , 𝑅𝑡 )

Additive decomposition: 𝑦𝑡 = 𝑇𝑡 + 𝑆𝑡 + 𝑅𝑡
- Appropriate when the variation around the trend or the seasonal pattern doesn’t change over time.
trend trend variance multi

Multiplicative decomposition: 𝑦𝑡 = 𝑇𝑡 × 𝑆𝑡 × 𝑅𝑡
- Appropriate when the variation changes over time
- Convert into a additive model with log transformation: 𝑙𝑜𝑔 𝑦𝑡 = 𝑙𝑜𝑔 𝑇𝑡 + 𝑙𝑜𝑔 𝑆𝑡 + 𝑙𝑜𝑔 𝑅𝑡 and plot the log series
Log-transformation can make reserves more constant??? Need review

Seasonal adjustment: 𝑦𝑡 = 𝑦𝑡 − 𝑆𝑡 or 𝑦𝑡 = 𝑦𝑡 / 𝑆𝑡
- Estimates (identifies) and removes the influence of predictable seasonal patterns to reveal the change in time series value
(overall trend?)
- Obtain the seasonally adjusted data when seasonal components are removed. Leads to more accurate forecast

Moving Average for smoothing

Moving average is often used to smooth out noise and reveal the trend components. (is seasonal patterns noise?)
- MA is symmetrical centered with same n on either side if Yt when k is an odd number
MA-k (odd): Smoothing with a moving average of order k, given the time series {𝑌1, 𝑌2, 𝑌3,..., 𝑌𝑇}
Example: Smoothing a time series using MA-5, 𝑇 = {𝑌1, 𝑌2, 𝑌3, 𝑌4, 𝑌5, 𝑌6, 𝑌7, 𝑌8, 𝑌9, 𝑌10}
The smoothed time series is 𝑇 = {−, −, 𝑇3, 𝑇4, 𝑇5, 𝑇6, 𝑇7, 𝑇8, −, −}
𝑌1+𝑌2+𝑌3+𝑌4+𝑌5 𝑌6+𝑌7+𝑌8+𝑌9+𝑌10
𝑇3 = 5
,..., 𝑇8 = 5

𝑇𝑡−0.5+𝑇𝑡+0.5
- MA-k (even) example:4, 𝑇𝑡 = 2
, see t need to start from 3
𝑌𝑡−2+𝑌𝑡−1+ 𝑌𝑡 +𝑌𝑡+1 𝑌𝑡−1+𝑌𝑡+ 𝑌𝑡+1 +𝑌𝑡+2
𝑇𝑡−0.5 = 4
, 𝑇𝑡+0.5 = 4

- Selecting k: when k increases, we lose more data on both sides, larger k, soother line.

MA is a special case of Weighted Moving Averages (WMA)

- WMA-k, k=2m+1 odd 𝑇𝑡 = 𝑤𝑡 −𝑚𝑌𝑡 −𝑚 + 𝑤𝑡−𝑚+1𝑌𝑡−𝑚+1 ... + 𝑤𝑡 +𝑚𝑌𝑡 +𝑚, where Σ𝑤𝑖 = 1
- A centered MA-4 is a WMA-5
M is the seasonal frequency

Steps Multiplicative Additive

Model assumption 𝑦𝑡 = 𝑇𝑡 × 𝑆𝑡 × 𝑅𝑡 𝑦𝑡 = 𝑇𝑡 + 𝑆𝑡 + 𝑅𝑡

Trend component Data smoothing. E.g. MA-k Data smoothing. E.g. MA-k
estimation

Detrend 𝑌𝑇/n
𝑇𝑡 = 𝑆 𝑇 × 𝑅𝑡 n =𝑆 + 𝑅
𝑌𝑇 − 𝑇 𝑡 𝑇 𝑡
What's left: seasonal pattern and noise.

Detrended seasonal avg Seasonal average for all years 𝑠1, ... , 𝑠𝑀 Seasonal average for all years 𝑠1, ... , 𝑠𝑀
(March: avg of all the detrended March values)

Math part Normalize M values to make sure they add up to M Normalize M values to make sure they add up to 0
- Normalizing constant 𝑐 = 𝑀/(𝑠1 + ... + 𝑠𝑀) - Mean µ =(𝑠1 + ... + 𝑠𝑀)/M
- Seasonal index 𝑆𝑚 = 𝑐 × 𝑠𝑚, 𝑚 = 1, 2,..., 𝑀 - Seasonal index 𝑆𝑚 = 𝑠𝑚 − µ, 𝑚 = 1, 2,..., 𝑀

Seasonal component n
𝑆𝑡 = {𝑆1, 𝑆2,..., 𝑆12, 𝑆1, 𝑆2,..., 𝑆12} n
𝑆𝑡 = {𝑆1, 𝑆2,..., 𝑆12, 𝑆1, 𝑆2,..., 𝑆12}
estimation
(For monthly data, only 12 different values. )

Model the seasonally 𝑌𝑡 𝑌𝑇 − 𝑆𝑡 = 𝑑𝑡

Remove the seasonal component, 𝑇𝑡 × 𝑅𝑡 =
adjusted (trend) 𝑆𝑡

Model this trend, fit the model and find the

parameters.

Do the forecast

Forecast the next 2 month? h=1,2

The X11 Method: a more sophisticated time series decomposition method

- Integrated into software packages such as SAS, R and EViews.
- It seasonally adjusted data and estimates the components of a time series.
- For the X11 method, there must be at least three years of observations in the input data sets.
- Import and call R functions in Python using r2py.

Python: from loading the data then process the raw data: converting the text to datetime data type
 notcorrelated couci 0
Week4 TimeSeriesRegression o conditionalmeanEceilxito
ReviewLinear forcrosssectionaldata
Regression Homoscedasticity varies o
SimpleLinearRegressionModel agood point
starting normaldistributionassumptionnotrequiredbutit
isusefulforpredictioninterval statisticali nference
linetruepopulationmodel x botsix
true
terror makesobservationscatters random
errorassumptions
model
observation
somite notlieonthe
true line
p opulation

findBoandI withleastsquareestimation
line
regression estimationmodel
Epix minicab Eet Il BoExit Eisai e ex x
y fitted

y Iii ai te
residual observed
ei xi a
anestimateofrandomerror
Bo T Bix
ameasureofmodelaccuracy residual ei ti ti observedfitted

residualproperties
te
o and new randomerror ei y'tYi trueline
onthe
error
forecast y fi the
cyton trueline

Goodness
of fit howwellthelinearregressiontiesthedata
21 75 Recon
ofdetermination171 55 ziyi
wetlicient
er
bythemodel
theproportionofvariationexplained
Ra sawhenpredictwell R o poorlypredict
isnotreliable more p added
redictor Rat explain toowellovertie
problem

pi i hit a a
Adjusted whennos nigneradinstear ispreferred
adjustedrise equal

inissamplesize Kispredictor d ifferentlywitha

number explain

withregression
Forecasting assumethe dist
errorsearenorm
regression

95 Geraint Yeast's Ntotalnumberofobservation

pwidetwnenxisfarfrom
aosermore
mean
sample
c ertainty Sx xvaluesstd.dev.tv Ecxi
sestandarderrorofregression Vizziniat
exampleor car
python emission

Nonlinearfunctionalform mostcommon naturallogarithm

logtransformation LinLin Yibotbixitei unit ay Binnie

9108 Innit botsmix e ox 1 ox Bil
Linlog Yi botBilnexiste ox 1 ay Baloounit
login incyitbotbixiteiox lunit oy bi.io

iii
yi Botbixiitbaxai BakitEi
ThecoefficientsBmeasurethemarginalexecsonthepredictorsCarothersconstant

prediction Botoxtaxa taxi

LinearRegressionModelfortimeseriesdata
simplelinearmodel ye botbixetet.tl it ix xe i it istimeseriesdata
Cge predictorxmightbeunknow scenariobasedforecasting predictxwith rate
growth butuncertaintyisnotincluded
Genuineforecastforx CAREHate.MGEnH

2typesofforecastdependson seminar

et
wnatisassameatobernan Expostforecast useinfobeyondtheforecastmoment
notgenuineforecast butusefulinbehaviorstudy
LinearRegressionModelfortimeseriesdata
simplelinearmodel ye botsixet et t t T Ntxe t t it istimeseriesdata
Pge predictorxmightbeunknow growthrate
scenariobasedforecasting predictxwith butuncertaintyisnotincluded
Genuineforecastforx

2typesofforecastdependson Exanteforecast onlyuseintoavailablewhenforecast genuineforecast

whatisassumedtobeknown exposttore
notgenuineforecast gg
tiaaii
butusefulinbehaviorstudy

Useful Predictorsusingsuitablepredictorstocapturetrend seasonalcomponentandoutliers

trading alter
days may sales
lineartrend TeBotBit Ee distributed
FeBotBit Bat'tEt lags l agged
trend quadratictrend advertising
long enter
lasting
t t xm adformmonthspreviously
trend
piecewiselinear laspecialsplineregression
it E II a bendsattimes
trend

mmmm
eg aseasonaltimeserieswithahorizontaltrend TeBoto.ttBad BedsBudy
ta whentis atimeofseasonone da da da 0 A Bo
t2 a BoBe thewetticientsofotherseasonsmeasurethedifference inbetweenthefirstseason
owhent 7 A BotBs 8 As Bot By
modelwithLinear
Regression trend quarterlydummyvariable
ytfttbzdutbsds.ttBodyt tee diet incertainquarter ordin o
outliers usedummyvariablestoremoveoutlierseffect Isinobservation oinelsewhere
t to interventiontime
foreltectslastonlyforaperiod spikevariable xp ao otherwise
instlike foroutliers
dummy

stepvariable xe t to interventiontime
L
permanenteltece
alevelshift o t to

selectingimportantpredictors
as
gg
adjusted
spigggy tqmamiiIedigssIib inelatian errors
forecast

crossvalidation prefer value

smallest

c caraikesInformationcriterion ate
minAec best N
nog E
observation
K
tack
predictor
andthevarianceofresiduals
intercept
numier

BICBayesianInformationcriterion Bec vlog sf actslogin

subsetregression oneoferor
Best It canbeused butgusted atendtoselecttoomanyvariableslesssuitableforforecasting
Residual diagnostics

plotforresidualsagainstpredictors
scatter inthemodel ifitshowsapattern nonlineari modify needed
notinthemodelifitshowsapattern addinthispredictors
7
residualagainst fittedvalues patternobserved heteroscedasticityintheerror

inresiduals commonintimeseriesdata assumptionid

Autocorrelation

ADF toseeifthere'sanyremaininginformationnotinvolvedinthemodel Aconsistanttrend highvalue

Autocorrelation Function
indicateseriescorrelation
Histogram
for it
residuals check normallydistributed
 Week5 ForecastingwithExponentialSmoothing
Exponentialsmoothingforecast weightedaverage
ofpreviousobservation theweightdecayexponentiallygofurtherintopast
simpleexponentialsmoothingSES Mitlexponentiallyweightedmovingaverage
Ew doesn'tconsidertrend
NaiveMethod Inlit Yt equaltothelatestobservationweightof 1
anAverageMethod
Over Inlit IEYe equalweighttoanpreviousobservations
SES Somethinginbetween Faint ay acta ye t actasyet tactatly 10441
ay ta ayetact ayest tail atay
NYT ta Flat Agin

t
ID ay totally
ctmost

1
weighted'Éterage
most obs
recent forecasts
recent

Thecomponentform one thelevel thesmoothvaluesoftheseries timet.at H he

ie
Le Lei t nee

Initialvaluesettings Lt ay ta Ley Li ay ta a to La ay t cta hi at a Day 1 a Lo

i iii iii my
Losetting avgofinitialobsyayaYs or simplyy
parameter algorithm
minimizingsse ÉYe Leit
a setting expert's knowledge
parameter algorithm
É Yeayei taLes

canbeusedforsmoothingaforecasting forseasonallyadjusted detrendeddata

nocleartrendorseasonalpatterns
Statisticalmodel produceforecastintervals slightlydifferent
domultiplestepforecast
Measurementequation Lt howthehiddenLeevolvesovertime
ma ayeto a Let osasI
Transitionequation Yet L tEet with EetN N o 04 E areindependent eachother
of
thenextlevelisthe level
current c orrupted noise
Yetiis a function
ofahiddenLt 4th
form
Lt aye ta h i Let aeye hey EYE gettin 9
Forecastequations forecast
theaffriage
overallpossibleuncertainty
Attn ECyanine Le Lei taee theuncertaintyoftheforecast
Teti Ellete Ine Varyetly o Var LetEerilyre
Elle ECE Iya VarLt Varlettilyre
Lt to
o to
theforecastisalways Ltaftertimet p
Fun ElLetae it tenlyin Le VarYtthly t o tch1 x
whenhincreases variancegetslarger uncertainty t
TrendcorrectedexponentialsmoothingICES ConsiderTrend

Le aye craft aye ta alla bed one

berth 4 1 oerellaweight
btferioustrend

yet tb and
Level trend

StatisticalModel Yeti Letbeteet Ettinn o o

Lt att l a kettbet
be rice Lei i r bet
chooseaandrbyminimizing Ssecair
Ight Let be it
allcomponentscanbeexpressedintermsoferror Ennio o
Errorcorrectionform
bt the cult ctr be
Le aye it a centbet t beta be trite ca bei
Lettbe italy ca bei L't beita.ee beta
Lex be a.ee betr Lettbettree Lei bei
betraee

Forecastingequations
yet Elyelyne Varlyttilyc var Letbetetti yet
Eketbetettilyc p
Letbe var yeenyet 02 Ha II Chir
Tun Lethbe o ha InchDin2 crash