EC403: Time Series Analysis

Topic: Introduction, Solution to Difference Equations

& Concept of Stationarity

Reetika Garg
Department of Economics
Delhi School of Economics

Winter Semester, 2023-24

 Example of a Time Series

US Real and Potential GDP, Consumption and Investment

Decomposition of Time Series Data: Various Components
DGP behind above series

• In Time Series Analysis we observe data at adjacent points in time. And so values tend to
be correlated, which is why values at time t depend on its values in t-1, t-2,....
• Autoregressive time series models can be thought of as difference equation with a
stochastic term
• Difference Equations used to solve for Equation of Motion
• Irregular Component above is AR(1)
How do we decode the irregular random component
In general given historical data can we make predictions about the irregular component?
If Yes then
- If patterns in the level of the series use ARMA type models
-If patterns in the second order moments, use ARCH / GARCH type models
-If patterns of a series is indicated by patterns in another series, Multivariate
If Not, then the random process is white noise.
If given any amount of history the best prediction is the average of the series, then the
randomness is unpredictable.
A sequence{εt} is white noise sequence if each value in the sequence has a zero mean,
constant variance and uncorrelated with all the other realisations.
White Noise
Let {εt} be a sequence of white noise process so that

1. E(εt) = 0
2. V(εt) = E(ε2t ) = σ2ε
3. E(εt εt-j) = 0 for j≠ 0 i.e. εt’s are uncorrelated across time.

If we replace the condition 3 above with a stronger condition that εt’s are independent across
time then we have a process that is independent white noise

4. εt, εt-j independent for j≠ 0

If in addition we have εt ∼N(0, σ2ε ) then the process is Gaussian white noise

5. εt ∼N(0, σ2ε )
Stochastic Process Vs. Time Series
• STOCHASTIC PROCESS/DGP is the general process, underlying the data that we see. It
is the distribution of all possible outcomes that can occur over time
• TIME SERIES is the data that we see. It is one draw from the more general stochastic
process
• We do not have the luxury of obtaining an ensemble (i.e., multiple time-series data of the
same process over the same time period). Typically, we observe only one set of realizations
for any particular series.
• So there's a probability distribution at each point in time; pdf of Yt at time t, pdf of Yt+1 at
time t+1, and so on.
• Does this pdf change overtime ? ➔ Stationary versus Non-stationary
Stationarity
Definition of stationarity?

• Strict: Probability distribution of the series does not change over time.
• Weak stationarity / Covariance stationary

- Mean and variance of the underlying process is constant

- Autocovariance depends only on the time lag and not on time origin
Standard Methods In The Presence Of Non-Stationary Data - I
ARMA MODEL
• A sequence of a stochastic variable{xt} formed by using white noise process

is called moving average of order q or MA(q). {xt} is not white noise.

• This is combined with autoregressive components to get ARMA models

• Inaccurate Predictions in case of non-stationary series

• Resort to ARIMA in case of non-stationary series.
Standard Methods In The Presence Of Non-Stationary Data - II
CLASSICAL LINEAR REGRESSION MODEL

• If time series data is used to obtain estimates under CLRM, then one of the implicit
underlying assumptions of CLRM is stationarity of variables.
• If we regress one non-stationary variable on another, sometimes, it may result in problem of
autocorrelation, which renders OLS estimates inefficient. In such cases, Phillips (1986)
have shown that t-statistics do not have limiting Student’s t-distribution, and so cannot be
used for hypothesis testing. Forecasts based on such regression equations are sub-optimal.
• Granger and Newbold (1974) simulated two series 𝑌𝑡 and 𝑋𝑡 both as nonstationary random
walk series such that 𝑌𝑡 and 𝑋𝑡 are independent of each other and then regressed 𝑌𝑡 on 𝑋𝑡 .
• The regression output “looked good” with high R square even though Y and X are
independent of each other; the Durbin Watson test statistic however had a low value.
Standard Methods In The Presence Of Non-Stationary Data – II
(ctd…)
• Phillips (1986) demonstrated that in cases when 𝑌𝑡 and 𝑋𝑡 are both nonstationary random
walk series such that 𝑌𝑡 and 𝑋𝑡 are cointegrated and not independent of each other, even in
those cases the t-statistics and F-statistics do not have limiting Student’s t distribution and
F-distribution, respectively.
• If the mean and variances of non-stationary / unit root variable change overtime, all the
computed statistics in a regression model, which uses these means and variances, are also
time dependent and fail to converge to their true values as the sample size increases
• Conventional tests of hypothesis will be biased towards rejecting the null hypothesis of no
relationship between dependent and independent variables. This is misleading especially if
null hypothesis is true. This is because t-statistics and F-statistics diverge as t tends to
infinity
• low d i.e. strong autocorrelation and R square > d is a signal of spurious regression
Is an AR (p) process Stationary ?
- Each AR(p) process can be characterised by difference equations
- Each Difference equation has a characteristic equation, which gives at most ‘p’
characteristic roots.
- Characteristic roots define the stability properties of the time path of the variable
- For different values of characteristic roots, check whether stationarity holds?
- When absolute value of all characteristic roots are less than 1, then for a large sample,
stationarity holds.
- When absolute value of one of the characteristic roots is equal to 1 we have a unit root
process
- When absolute value of more than one characteristic roots is equal to 1 we have multiple
unit root process
- If any characteristic roots is greater than 1, the weights on past terms increase with t, the
past is more important than the present. Here the series is explosive and rapidly diverges
towards +∞ or -∞. This is of course counter-intuitive; so we safely assume that in time
series models of real life data all roots are either less than or equal to 1.
When is an AR(1) process stationary?
For an AR(1) model: 𝑦𝑡 − 𝑎1 𝑦𝑡−1 = 0

We have a Characteristic Equation: 𝛼 − 𝑎1 = 0

This equation has one Characteristic Root: 𝛼 = 𝑎1

We can also have an Inverse Characteristic Equation: 1 − 𝑎1 𝐿 = 0

Inverse characteristic equation gives us inverse Characteristic Roots 𝐿 = 1/𝛼

1
=𝐿
𝑎1

If |α| <1 or |L| >1 then AR(1) process is stationary

If |α| =1 or |L| =1 then AR(1) process has a unit root and is non-stationary

We rule out explosive processes when |α| >1 or |L| <1 and avoid discussing such cases.
When is an AR(2) process stationary?
Hint:
For an AR(2) model:

𝑦𝑡 − 𝑎1 𝑦𝑡−1 − 𝑎2 𝑦𝑡−2 = 0

We have a Characteristic Equation: 𝛼 2 − 𝑎1 𝛼 − 𝑎2 = 0

This equation has two Characteristic Roots: 𝛼1 and 𝛼2

Stationary vs Non-stationary: Some Examples
Note: Sometimes when we see that a series has a trend it may give the illusion of a
nonstationary series, when in fact it may be a stationary series with a trend.
Deterministic Trend and Trend Stationary
• Consider ∆𝑦𝑡 = 𝑦𝑡 − 𝑦𝑡−1 = 𝑎0
• Solution: 𝑦𝑡 = 𝑦0 + 𝑎0 𝑡
• Add a stochastic error term, say 𝜀𝑡
𝑦𝑡 = 𝑦0 + 𝑎0 𝑡 + 𝜀𝑡
where 𝑎0 𝑡 is the deterministic trend component
𝜀𝑡 is stationary (white noise error or stochastic error term which has ARMA specification)
• 𝑦𝑡 differs from the deterministic trend by an amount εt
• since the deviation is stationary { 𝑦𝑡 } will exhibit temporary departures from the
deterministic trend.
• long term forecasts will converge to deterministic trend
• such a model is also called trend stationary model as we can obtain a stationary series after
removing the deterministic trend.
Deterministic Trend and Trend Stationary (ctd…)

𝐸 (𝑦𝑡 ) = 𝑦0 + 𝑎0 𝑡
𝑉 (𝑦𝑡 ) = 𝜎 2
𝐶𝑜𝑣(𝑦𝑡 , 𝑦𝑡−𝑠 ) = 𝐸[𝑦𝑡 − 𝑦0 + 𝑎0 𝑡][𝑦𝑡−𝑠 − 𝑦0 + 𝑎0 (𝑡 − 𝑠)]
= 𝐸 [𝜀𝑡 𝜀𝑡−𝑠 ] = 0
• This series is stationary around a deterministic trend
• If we remove trend from the time path of 𝑦𝑡 , the series would be stationary.
Stochastic Trend and Difference Stationary
• Consider ∆𝑦𝑡 = 𝑦𝑡 − 𝑦𝑡−1 = 𝑎0 + 𝜀𝑡
• Solution: 𝑦𝑡 = 𝑦0 + 𝑎0 𝑡 + ∑𝑡𝑖=1 𝜀𝑖

where 𝑎0 𝑡 is the deterministic trend component and the rest is stochastic intercept term

• εi represents a shift in the intercept in each period.

• Since the coefficients on {𝜀𝑖 }is 1, the effect of the error in each period is permanent.
• After a moment’s reflection, the form of the solution is quite intuitive. In every period t, the
value of 𝑦𝑡 changes by 𝑎0 + 𝜀𝑡 units. After t periods, there are t such changes; hence, the total
change is t a0 plus the t values of the {𝜀𝑖 }sequence. Notice that the solution contains
summation of all disturbances from 𝜀1 through 𝜀t. Thus, when a1 = 1, each disturbance has a
permanent non-decaying effect on the value of 𝑦𝑡
Stochastic Trend and Difference Stationary (ctd…)

• Such a {yt} sequence is said to have stochastic trend, since each εi imparts a permanent,
although random impact, and so it seems that the sample path seems to have the appearance
of a trend
• The sequence { yt} will exhibit permanent departures with no tendency to revert to mean

𝐸 (𝑦𝑡 ) = 𝑦0 + 𝑎0 𝑡

𝑉 (𝑦𝑡 ) = 𝑡𝜎 2

𝐶𝑜𝑣(𝑦𝑡 , 𝑦𝑡−𝑠 ) = 𝐸[𝑦𝑡 − 𝑦0 + 𝑎0 𝑡][𝑦𝑡−𝑠 − 𝑦0 + 𝑎0 (𝑡 − 𝑠)]

𝑡 𝑡−𝑠

= 𝐸 [∑ 𝜀𝑖 ∑ 𝜀𝑖 ] = (𝑡−𝑠)𝜎 2
𝑖=1 𝑖=1
Stochastic Trend and Difference Stationary (ctd…)

• From such a model which has stochastic trend, we can difference the data to obtain a
stationary series, which is why {yt} will be Difference Stationary series.

𝐸 (∆𝑦𝑡 ) = 𝑎0

𝑉 (∆𝑦𝑡 ) = 𝜎 2

𝐶𝑜𝑣(𝑦𝑡 , 𝑦𝑡−𝑠 ) = 𝐸[∆𝑦𝑡 − 𝑦0 ][∆𝑦𝑡−𝑠 − 𝑦0 ] = 0

Shocks to a stationary time series are necessarily temporary; over time, the effects of the shocks
will dissipate, and the series will revert to its long-run mean level. On the other hand, a series
containing a stochastic trend will not revert to a long-run level.
Random Walk Model
For an AR(1) model: 𝑦𝑡 = 𝑎1 𝑦𝑡−1 + 𝜀𝑡 , εt is stationary white noise error term

when 𝑎1 = 1 , we get a random walk model

𝑦𝑡 = 𝑦𝑡−1 + 𝜀𝑡

Check for stationarity in this model.

Given the initial condition, y0, the general solution of the difference equation
represented by the random walk model is
𝑡

𝑦𝑡 = 𝑦0 + ∑ 𝜀𝑖
𝑖=1
 𝐸 (𝑦𝑡 ) = 𝑦0

𝑉 (𝑦𝑡 ) = 𝑡𝜎 2

𝐶𝑜𝑣(𝑦𝑡 , 𝑦𝑡−𝑠 ) = 𝐸[𝑦𝑡 − 𝑦0 ][𝑦𝑡−𝑠 − 𝑦0 ]

𝑡 𝑡−𝑠

= 𝐸 [∑ 𝜀𝑖 ∑ 𝜀𝑖 ] = (𝑡−𝑠)𝜎 2
𝑖=1 𝑖=1

• Mean is constant but variance is not constant.

• The covariance between observations at time t and t-s not only depend on s but also on t.
• Since variance and covariance is time dependent, random walk model is non stationary
Random Walk Model with a drift
Adding a constant term gives a Random Walk model with drift

𝑦𝑡 = 𝑎0 + 𝑦𝑡−1 + 𝜀𝑡

The general solution to the above difference equation is given by

𝑡

𝑦𝑡 = 𝑦0 + 𝑎0 𝑡 + ∑ 𝜀𝑖
𝑖=1

Check for stationarity. (already seen earlier)

Visual similarity of Non-stationary and Trend stationary models necessitate the
deployment of formal testing procedures called the Unit Root Tests
Does strict stationarity imply weak stationarity?

Does weak stationarity imply strong stationarity?

Bibliography

Walter Enders (2015) Applied Econometric Time Series 4th Edition John Wiley &
Sons, Inc., Chapter 1 Section 1,2, 3 and 6, Chapter 2 Section 3

C.W.J. Granger, P. Newbold,(1974) Spurious regressions in econometrics, Journal of

Econometrics, Volume 2, Issue 2, Pages 111-120,

P.C.B. Phillips, (1986) Understanding spurious regressions in econometrics,

Journal of Econometrics, Volume 33, Issue 3, Pages 311-340,