PSTAT 174/274

LECTURE NOTES 3
NOTES - 3

1 / 24
Autoregressive Processes

(Yt − µ) = φ1 (Yt−1 − µ) . . . + φp (Yt−1 − µ) + at

or
Πp (B)(Yt − µ) = at
where
Πp (B) = 1 − φ1 B − φ2 B 2 − . . . − φp B p

2 / 24
The process is always invertible since at can be written as an infinite sum
of Yt and earlier values.
Remember: Random shock model is always stationary ! Now, the above
model can be written as:
1
(Yt − µ) = at
Πp (B)

For it to be stationary, the roots of Πp (B) = 0 must lie outside the unit
circle.
The AR process is useful in describing situations in which the present value
of a time series depends on its preceding values plus a random shock.

3 / 24
AR(1) Process

Assume µ = 0
Yt = φ1 Yt−1 + at
(1 − φ1 B)Yt = at ⇒ Yt = (1 + φ1 B + φ21 B 2 + . . .)at
Yt = at + φ1 at−1 + φ21 at−2 . . .

a) Always invertible
b) For stationarity, roots of Π(B) = (1 − φ1 B) = 0 should lie outside
unit circle. So, roots of m − φ1 = 0 should lie within the unit circle.
So, m = φ1 and |φ1 | < 1
Here, m = 1/B

4 / 24
Now assuming stationarity,
E(Yt ) = 0
and
σ2
V (Yt ) = γ0 = (1 + φ21 + φ41 + . . .)σ 2 =
1 − φ21
and

γ1 = E(Yt Yt−1 )
= E[φ1 Yt−1 Yt−1 ] + E[at Yt−1 ]
= φ1 γ0 + 0
σ 2 φ1
=
1 − φ21

σ 2 φ21
γ2 = E(Yt Yt−2 ) = φ1 γ1 =
1 − φ21

5 / 24
so,
σ 2 φk1
γk =
1 − φ21
= φ1 γk−1 = φk1 γ0
and
ρk = φk1
If 0 < φ1 < 1, all ACFs are positive.
If −1 < φ1 < 0, ACFs alternate signs.
(
ρ1 = φ1 k=1
φkk =
0 |k| > 1
So PACF cuts off after lag 1.

6 / 24
AR(2) Process

Assume µ = 0
Yt = φ1 Yt−1 + φ2 Yt−2 + at
(1 − φ1 B − φ2 B 2 )Yt = at ⇒ Yt = (1 − φ1 B − φ2 B 2 )−1 at

a) Always invertible
b) For stationarity, roots of Π(B) = (1 − φ1 B − φ2 B 2 ) = 0 should lie
outside unit circle. So, roots of m2 − φ1 m − φ2 = 0 should lie within
the unit circle.

7 / 24
So,  
 φ ± pφ2 + 4φ 
 1 1 2
(m1 , m2 ) =  <1
 2 
Hence,
m1 m2 = φ2 ⇒ |m1 m2 | = |φ2 | < 1
and
m1 + m2 = φ1 ⇒ |m1 + m2 | = |φ1 | < 2

8 / 24
Now,
γ0 = φ1 γ1 + φ2 γ2 + σ 2
and,
γ1 = φ1 γ0 + φ2 γ1
φ1 γ0
⇒ γ1 =
1 − φ2
In general,
γk = φ1 γk−1 + φ2 γk−2 , k>0
ACF: Known as Yule Walker equations where,

ρk = φ1 ρk−1 + φ2 ρk−2

9 / 24
PACF: 
φ1
ρ1 =

 1−φ2
ρ2 −ρ21
φkk = = φ2
 1−ρ21

0 k>2
PACF cuts off after lag 2

10 / 24
In general for an AR(p) process,

γk = φ1 γk−1 + . . . + φp γk−p , k>0

ACF: Yule Walker equations

ρk = φ1 ρk−1 + . . . + φp ρk−p , k>0

PACF cuts off after lag p !

11 / 24
Moving Average Process

Also a generalization of the Random shock form. The process Yt

depends on all the errors at .

Yt = µ + at − θ1 at−1 − θ2 at−2 − . . . − θq at−q

So,
Yt = µ + Θq (B)at
where,
Θq (B) = 1 − θ1 B − θ2 B 2 − . . . − θq B q

12 / 24
a) MA processes are always stationary
b) They are invertible if the roots of the equation Θq (B) = 0 lie outside
the unit circle.
c) Useful to describe events producing an immediate effect that lasts for
a short period of time.

13 / 24
MA(1) Process

Assume µ = 0
Yt = at − θ1 at−1
E(Yt ) = 0, V (Yt ) = (1 + θ12 )σ 2
γk = E(Yt Yt−k )
= E[(at − θ1 at−1 )(at−k − θ1 at−k−1 )]

2 2
σ (1 + θ1 ) k = 0

= −θ1 σ 2 k = ±1

0 |k| > 1


14 / 24
 (
θ1
− 1+θ 2 |k| = 1
ρk = 1
0 |k| > 1
ACF cuts off after lag 1 and in general, 2|ρk | < 1
Try finding the PACF !
MA(1) process is always stationary.
It is invertible if the roots of Θ(B) = (1 − θ1 ) = 0 lie outside the unit
circle or the roots of m − θ1 = 0 lie within the unit cicle. Thus,

m − θ1 = 0 ⇒ m = θ1

So,
|θ1 | < 1

15 / 24
MA(2) Process

Assume µ = 0
Yt = at − θ1 at−1 − θ2 at−2
= (1 − θ1 B − θ2 B 2 )at
Now,
E(Yt ) = 0 V (Yt ) = γ0 = (1 + θ12 + θ22 )σ 2

16 / 24
Autocovariance function:



σ 2 (1 + θ12 + θ22 ) k=0
−θ σ 2 (1 − θ )

|k| = 1
1 2
γk = 2


−θ2 σ |k| = 2

0 |k| > 2

17 / 24
ACF: 
−θ1 (1−θ2 )
 1+θ12 +θ22

 |k| = 1
−θ2
ρk = |k| = 2
 1+θ12 +θ22

0 |k| > 2

• ACF cuts off after lag 2

• PACF tails off exponentially
• always stationary
• invertible if the roots of 1 − θ1 B − θ2 B 2 lie outside the unit circle OR
if the roots of m2 − θ1 m − θ2 = 0 lie inside the unit circle.

18 / 24
In general for a MA(q) process,

E(Yt ) = 0 V (Yt ) = γ0 = (1 + θ12 + . . . + θq2 )σ 2


2 2 2
σ (1 + θ1 + . . . + θq )
 k=0
2
γk = σ (−θk + θk θ1 + . . . + θq−k θq ) |k| = 1, 2, . . . , q

0 |k| > q

( (−θk +θk θ1 +...+θq−k θq )
(1+θ12 +...+θq2
|k| = 1, 2, . . . , q
ρk =
0 |k| > q
ACF cuts off after lag q !

19 / 24
Autoregressive Moving Average Process ARMA(p,q)

If we assume that the series is partly AR(p) and MA(q), we have an

ARMA(p,q) process as:

Yt = φ1 Yt−1 + . . . + φp Yt−p + at − θ1 at−1 − . . . − θq at−q

So,
Πp (B)Yt = Θq (B)at

20 / 24
• for invertibility, the roots of Θq (B) should lie outside the unit circle.
• for stationarity, the roots of Πp (B) should lie outside the unit circle.
• Autocovariance function:

γk = φ1 γk−1 + . . . + φp γk−p , k > q + 1

• ACF:
ρk = φ1 ρk−1 + . . . + φp ρk−p , k > q + 1
• Both ACF and PACF tail off after lag max(p,q)

21 / 24
ARMA(1,1) Process

Yt = φ1 Yt−1 + at − θ1 at−1
(1 − φ1 B)Yt = (1 − θ1 B)at

• stationary if |φ1 | < 1

• invertible if |θ1 | < 1

22 / 24
 E(Yt ) = 0
1 + θ12 + 2φ1 θ1
V (Yt ) = γ0 =
1 − φ21
γ 1 = φ 1 γ 0 − θ1 σ 2
ACF:
γk = φ1 γk−1 = φk1 γ1 , k > 1
ρk = φ1 ρk−1 = φk1 ρ1 , k > 1
Both ACF and PACF tail off after lag 1 !

23 / 24
Causality

Yt is causal if it can be represented as a function of current or past at

values. MA(q) is always causal:

Yt = at − θ1 at−1 − . . . − θq at−q

AR(1) is causal for |φ1 | < 1 as it has a MA(∞) representation

24 / 24