Time series

• Sequence of measurements taken at equally spaced points in time

• Very common in fields like applied sciences, engineering and economy
• Today’s technology leads to very frequent and real-time TS
• Applications
– Governments → tax receipts and spending
– World Bank and IMF → inflation, economic activity
– Energy companies → reserves, production, demand, prices
– Transport → number of passengers
– Retail stores → sales

Descriptive
- Time series analysis
• Trends
• Seasonal patterns
• Relationship to external factors
Predictive
- Forecast future values
• Single TS forecasting
• Multivariate TS forecasting
– Cross-correlation TS’s

Time series components (Descriptive)

Four components
- Level
- Average value of the series
- Trend,
- Change in TS from one period to next
- Seasonality
- Short-term cyclical behavior
- Noise
- Random variation from measurement error or other non accounted for causes

Time series forecasting (Predictive)

• Time is the main predictor
• Model-driven
– Regression-based forecasting
– Analyst needs to specify model
– Assumptions about data structure
– Good for global patterns
• Data-driven
– Smoothing-based methods
– Method learns patterns from data
– Good for local patterns
Data partition & performance evaluation
• Random partition creates 2 TS’s with holes
• Data partition for TS
– Training set: earlier periods
– Validation set: later periods
• Evaluation metrics
– Same as for numerical prediction (e.g. MAE, MAPE, RMSE)

Naive forecast
• Naive forecast = last value of TS (green line)
• Seasonal Naive forecast = last similar value (orange line)
• Can be surprisingly difficult to outperform
• Always compare forecast to this baseline

Forecasting new values

• Before forecasting recombine train and validation TS’s and re-run model
• Why?
– Validation is most recent and has valuable information
– More data improves accuracy
– Otherwise needs to forecast even further into future
Modelling trend
- Linear
- Yt = β0 + β1t + ε
- Exponential
- Yt = Ceβ1t +ε
- lnYt = β0 + β1t + ε
- Polynomial
- Yt = β0 + β1t + β2t 2 + ε

Modeling seasonality
• Values higher/lower depending on season
• Examples: day-of-week, monthly, quarterly
• Integrating in regression model
– Create categorical variable for season
– Make dummies from these variables
– Each dummy is an extra predictor
• Types
– Additive: fixed amount more/less
– Multiplicative: higher/lower by percentage
Monthly additive
- Yt = β0 + C(2)δ(2) + C(3)δ(3) + ... + C(12)δ(12) + ε
Modeling trend & seasonality
- Quadratic trend & monthly additive seasonality
- Yt = β0 + β1t + β2t 2 + C(2)δ(2) + C(3)δ(3) + ... + C(12)δ(12) + ε

Autocorrelation

• Relationship between the series and itself

– Neighboring values tend to be correlated
• Can help in improving forecast
• Computation of autocorrelation
– Correlation TS and its lagged version
• Behaviors
– Strong lag-k and its multiples (2k, 3k,
etc), e.g. annual seasonality (k=12)
– Stickiness: positive lag-1 autocorrelation,
e.g. linear trend
– Swings: negative lag-1 autocorrelation,
high values followed by low values
AutoRegressive (AR) models
• Similar to linear regression, but predictors are past values
• AR model order 2 – Yt = β0 + β1Yt−1 + β2Yt−2 + ε
• Autocorrelation integrated into model
– Need to remove trend and seasonality
• Second-level forecasting model
1. Generate Ft+k (k-step ahead) forecast using forecast model
2. Generate Et+k forecast of residuals using AR model
3. Improved forecast: Ft+k ∗ = Ft+k + Et+k
• AutoRegressive Integrated Moving Average (ARIMA) models
– Very flexible forecasting method
AR(1) model on residuals
- Et = β0 + β1Et−1 + ε
Can we forecast the future?
Random walk series
• Can the past be used to predict the future beyond Naive forecast?
• Test whether series is a random walk
Yt = β0 + Yt−1 + εt
Yt − Yt−1 = β0 + εt
– Right side is random walk term
– Random walk is a special case of AR(1) with slope 1
• Performing test
1. Fit AR(1) to series
2. Test hypothesis H0 : β1 = 1
3. If null-hypothesis is rejected TS is not a random walk, we can attempt to forecast
series

Smoothing methods

What are they?

• Average over multiple periods to reduce noise
• They are data-driven and able to adapt to
changes
• Regression relies on underlying theoretical
model
• Useful when TS has patterns that change over
time

Moving average
- Average values across a window of consecutive periods
• Centered moving average (a)
– MAt = 1 /w (Yt−(w−1)/2 + ... + Yt−1 + Yt + Yt+1 + ... + Yt+(w−1)/2)
– Good for visualizing trends
– Can remove seasonality and noise
• Trailing moving average (b)
– Ft+k = 1/ w (Yt + Yt−1 + ... + Yt−w+1)
– Useful for forecasting
– Not good for TS with trend and/or
seasonality
Choosing window
• Balance between over/under-smoothing
• Centered
– Wider exposes global trends
– Narrower exposes local trends
• Trailing
– Relevance of past values
– How fast does TS change?
– Empirical search, but be careful with overfitting

Simple exponential smoothing

• Weighted average of all past values
– Weights decrease exponentially into past
– Recent information has higher influence
– But past information is not ignored
• Exponential smoothing forecaster
– Ft+1 = αYt + α(1 − α)Yt−1 + α(1 − α) 2Yt−2 + ...
– Ft+1 = Ft + αEt
– α : {0, 1}
– Method is active learner
– Ft+k = Ft+1 • Not fit for TS with trend and/or seasonality
• Can be applied to TS of residuals

Choosing α
• Determines rate of learning
– Close to 1: fast learning (recent values have more influence)
– Close to 0: slow learning (past has more influence)
• α = 0.1 − 0.2 leads to good results in general
• Empirical search can be done, but attention to overfitting
Advanced exponential smoothing Trend
• Can capture trend and seasonality
• Trend shape is not assumed to be global and can change over time
• Future forecasts are no longer identical
• α&β : {0, 1}
• Lt : weighted average of actual period and level in previous adjusted by trend
• Tt : weighted average of trend in previous period and recent change of level

Advanced exponential smoothing

Trend & seasonality
• Holt-Winters exponential smoothing
• Condition: t > M (full cycle of seasons)
• Lt : similar interpretation from before, but uses seasonally adjusted value
• Tt : same interpretation as before
• St : weighted average of index in previous cycle and current value adjusted
• Delete trend term for TS with only seasonality
• Multiplicative seasonal relation, additive version also exists