Time Series Regression

The Nature of TS Data

• An obvious characteristic of TS data is temporal ordering
• The past can affect the future, but not vice versa
• Economic time series satisfy the intuitive requirements for being outcomes
of random variables ==> we don’t know what the IHSG index will be at the
close of the next trading day (similar like cross section data)
• A sequence of random variables indexed by time is called a stochastic
process or a time series process
• The actual value of time series observation called realization
• The distinction between the stochastic process and its realization is akin to
the distinction between population and sample in cross sectional data
• The sample size for a time series data set is the number of time periods
over which we observe the variable of interest
TS Regression Model
Static Models
• Suppose two variable in time series data, 𝑦 and 𝑧, where 𝑦! and 𝑧!
are date contemporaneously
• The model set up: 𝑦! = 𝛽" + 𝛽# 𝑧! + 𝑢! , 𝑡 = 1,2, … , 𝑛
• A change in 𝑧 in time 𝑡 is believed to have an immediate efffect on 𝑦:
△ 𝑦! = 𝛽# △ 𝑧! , when △ 𝑢! = 0
Finite Distributed Lag Models (FDL)
• We allow one or more variables to affect 𝑦 with a lag
• The model set up: 𝑦! = 𝛽" + 𝛽# 𝑧! + 𝛽$ 𝑧!%# + 𝛽& 𝑧!%$ + 𝑢! ,
• Which is a FDL of order two
• Lag distribution summarize the dynamic effect that a temporary
increase in 𝑧 has on 𝑦
• A primary purpose for estimating a distributed lag model is to test
whether 𝑧 has a lagged effect on 𝑦.
Finite Sample Properties of OLS under
Classical Assumption
• Unbiasedness of OLS
1. Linear in parameters
2. No perpect collinearity
3. Zero conditional mean
• The variance of the OLS estimator and Gaus Markov Theorem
4. Homoskedasticity
5. No Serial Correlation
• Under assumption 1-5, the OLS estimators are the Best Linear
Unbiased Estimators (BLUE) conditional on X
• Inference under the Classical Linear Model Assumption
6. Normality
Trend
• Many economic time series have a common tendency of growing over
time è Some series contain a time trend
• In many cases, two time series processes appear to be correlated only
because they are both trending over time for reasons related to other
unobserved factors.
• Model regression with trend : 𝑦! = 𝛼" + 𝛼# 𝑡 + 𝑒! , 𝑡 = 1,2, … , 𝑛
• Where 𝑒! is an independent and identically distributed (i.i.d)
sequence with E(𝑒! )=0 and Var(𝑒! )=𝜎'$
• Interpreting 𝛼# : holding other factors fixed, 𝛼# measures the change
in 𝑦! from one period to the next due to the passage of time.
Using Trending Variable in Regression
• Spurious regression è one of example: the phenomenon of finding a
relationship between two or more trending variables simply because each
is growing over time
• Adding time trend eliminates the spurious regression problem.
• Consider a model where two observed factors, 𝑥!" and 𝑥!# , affect 𝑦. In addition,
there are unobserved factors that are systematically growing or shrinking over time:
• Model set up: 𝑦! = 𝛽$ + 𝛽" 𝑥!" + 𝛽# 𝑥!# + 𝛽% 𝑡 + 𝑢!
• Omitting 𝑡 in regression will generally yield biased estimators of 𝛽" and 𝛽# . This is
especially true if 𝑥!" and 𝑥!# are themselves trending è Then, they can be highly
correlated with 𝑡.
• Also, adding time trend can make a key explanatory variable more
significant
Stationarity
• Stationarity process has played an important role in the analysis of
time series
• A stationarity time series process is one whose probability
distributions are stable over time in the following sense: if we take
any collection of random variables in the sequence and then shift that
sequence ahead h time periods, the joint probability distribution
must remain unchanged
• A formal definiton:
• The stocastic process 𝑥! : 𝑡 = 1,2, . . is stationary if for every collection of
time indices 1 ≤ 𝑡" < 𝑡# < … < 𝑡$ , the joint distribution of (𝑥!", 𝑥!#, …,
𝑥!$ ) is the same as the joint distribution of (𝑥!"%& , 𝑥!#%& , …, 𝑥!$%& ) for all
integers h ≥ 1
Covariance Stationarity
• A stocastic process 𝑥! : 𝑡 = 1,2, . . with a finite second moment
𝐸 𝑥!" < ∞ is covariance stationarity if (i) 𝐸 𝑥! is constant, (ii) Var 𝑥! is
constant, (iii) for any t, h ≥ 1, Cov(𝑥! , 𝑥!#$ ) depends only on h and not on t
• Rata-rata: 𝐸 𝑥! = 𝜇
• Varian: 𝑣𝑎𝑟 𝑥! = 𝐸(𝑥! − 𝜇)# = 𝛿 #
• Kovarian: 𝛾& = 𝐸[(𝑥! − 𝜇! )(𝑥!'& − 𝜇! )
• In other word, Covariance stationarity focuses only on the first two
moments of stocastic process: the mean and variance of the process are
constant across time, and the covariance between 𝑥! and 𝑥!#$ depends
only on the distance between the two terms, h, and not on the location of
the initial time period, t.
Look this

If time series is stationary, its mean, variance, and autocovariance (at various lags) remain the same no matter at
what point we measure them; that is, they are time invariant

Panel A: stationarity time series

Panel B: mean is different across time
Panel C: variance is different across time
Panel D: covariance is different across time
When TS data Non-Stationary
• We can study the behavior of data only for the time period under
consideration è it is not possible to generalize it to other time
periods
• Some consequences:
• Pada data yang tidak stasioner, pengaruh guncangan (shock) yang terjadi pada
periode t akan tetap pada periode t+1, t+2, dan seterusnya
• Jika digunakan untuk forecasting, data yang tidak stasioner akan
menghasilkan hasil forecast yang go off to infinity (tidak masuk akal dan tidak
valid)
• Menghasilkan regresi lancung (spurious), ditandai dengan nilai R-squared
yang tinggi (lebih dari 0,9). Hasil regresi tampak valid, tapi tidak bermakna
secara ekonomi
• Nilai t-statistik dan f-statistik tidak valid
Example of Non-Stationary Model
• Random walk without drift
• Suppose 𝑢! is a white noise error term with mean 0 and variance 𝜎 #. Then
the series 𝑌! is said to be a random walk if:
𝑌! = 𝑌!'" + 𝑢!
• The equation shows, the value of Y at time t is equal its value at time (t-1)
plus a random shock.
• If 𝑌! is nonstationary, then its first difference is stationary:
𝑌! − 𝑌!'" = Δ 𝑌! = 𝑢!
• Random walk with drift
• Adding the drift paramater as follow:
𝑌! = 𝛿 + 𝑌!'" +𝑢!
• 𝑌! drifts upward or downward depending on 𝛿 being positive or negative
Unit Root Stochastic Process
• Example Random Walk Model (RWM): 𝑌! = 𝜌 𝑌!%# + 𝑢! , where 1 ≤
𝜌 ≥1
• If 𝜌=1, the model becomes a RWM without drift è we face what is
known as the unit root problem, that is a situation of nonstationarity
• If 𝜌 <1, that is if the absolute value of 𝜌 is less than one, then it can
be shown that the series 𝑌! is stationary.
Trend Stasionary (TS) and Difference
Stationary (DS)
• RWM without drift is a difference
stationary process
• RWM with drift and stocastic trend
is difference stationary process
• RWM with drift and deterministic
trend is trend stationary è
removing the deterministic trend.
• Deterministic trend: if the trend in a
time series is a deterministic function
of time, such as time, time squared
etc è 𝑌! = 𝛽" + 𝛽# 𝑡 + 𝑌!(" +𝑢!
• Stocastic trend: if the trend in a time
series is not predictable ==> 𝑌! =
𝛽" + 𝑌!(" +𝑢!
Integrated Stocastic Process
• RWM stationary in first difference called integrated of order 1,
denoted as I(1). If a time series has to be difference twice to make it
stationary, we call integrated of order 2
• In general, if time series has to be differenced d times to make it
stationary, that time series is said to be integrated of order d.
• Most economic time series are generaly I(1)
Test of Stationary
• Graphical analysis è plot the value of variable during observation
period
• The correlogram test
• Check: nonstationary if the coefficient start at a very high value and decline
very slowly toward zero as the lag lengthens.
• The choice of lag length: a rule of thumb is to compute ACF up to one-third to
one-quarter the length of the time series.
Unit Root Test
• For example the RWM: 𝑌! = 𝜌 𝑌!%& + 𝑢! , where 1 ≤ 𝜌 ≥ 1
• If 𝜌=1, then there is unit root
• However, we can not estimate by OLS and test the hypothesis by the usual
t statistict because that test is severely biased in the case of a unit root. So,
we substract 𝑌!%& from both side:
𝑌! − 𝑌!%& = 𝜌 𝑌!%& − 𝑌!%& + 𝑢!
∆𝑌! = (𝜌 − 1) 𝑌!%& + 𝑢!
Then alternatively, we can write:
∆𝑌! = 𝛿 𝑌!%& + 𝑢!
We estimate this model, then test the null hypothesis that 𝛿=0, and the
alternative hypothesis 𝛿 <0. If 𝛿=0, then 𝜌=1, that we have a unit root è
the time series is nonstationary
The Dickey Fuller
• Dickey and Fuller have shown that under the null hypothesis that 𝛿=0, the
estimated t value of 𝑌!%& follow the 𝜏 (tau) statistic (because t value of the
estimated coefficient does not follow the t distribution
• In the literature, the tau statistic or test is known as the Dickey Fuller (DF)
test
• The DF test is one-sided because the alternative hypothesis is that 𝛿 < 0 or
𝜌 < 1.
• If the computed (negative) tau value is smaller than the critical value, we
reject the null hypothesis, then time series is stationary. Otherwise, we do
not reject the null hypothesis.
• The model set up:
∆𝑌! = 𝛿 𝑌!%& + 𝑢!
∆𝑌! = 𝛽& + 𝛿 𝑌!%& + 𝑢!
∆𝑌! = 𝛽& + 𝛽" 𝑡 + 𝛿 𝑌!%& + 𝑢!
The Augmented Dickey Fuller
• This test is conducted by “augmenting” the preceding three equations
by adding the lagged values of the dependent variable ∆𝑌! .
• The model set up:
∆𝑌! = 𝛽# + 𝛽$ 𝑡 + 𝛿 𝑌!%# + ∑*
()# 𝛼( ∆𝑌!%( + 𝑢!
• The number of lagged difference term to include is often determined
empirically, the idea behind to include enough term so that the error
term is serially uncorrelated.
• We can obtain an unbiased estimate of 𝛿.
The Phillilp Perron Test
• An important assumption of DF test is that the error term 𝑢! are
independently and identically distributed. ADF adding the lagged
difference term of regressand to take care of possible serial
correlation
• PP test use nonparamateric statistical methods to take care of the
serial correlation in the error term without adding lagged difference
term
Cointegration
• Motivation è the regression of a nonstationary time series on another
nonstationary time series may produce a spurious regression
• If two time series are I(1), they contain stochastic trend, it is quite possible
that the two series share the same common trend so that the regression of
one on the other will not be necessarily spurious
• The linear combination of two time series cancels out the stocastic trends
• If we take consumption and income as two I(1) variables, savings defined as
(income – consumption) is I(0).
• In this case, we say that two variables are cointegrated
• In economic, two variables will be cointegrated if they have a long term, or
equilibrium, relationship between them
Testing Cointegration
• Engle-Granger (EG) or Augmented Engle-Granger (AEG) Test
• Model set up: 𝑌! = 𝛽 𝑋! + 𝑢!
• If 𝑌! and 𝑋! are both I(1), then we can estimate the residual from this
equation. Then, check the stationary of residual.
• It has similiar procedure with the DF and ADF test, but the critical
significance values are not quite appropriate. EG have calculated
these values separetely