21/22

THE UNIVERSITY OF HONG KONG

DEPARTMENT OF STATISTICS AND ACTUARIAL SCIENCE
STAT3907 Linear Models and Forecasting (2021-2022)

EXAMPLE CLASS 8

1 Time Series and Stationarity

1. Time series data: all observed time series data (z1 , z2 , ..., zT ) is a realization of T
random variables (Z1 , Z2 , ..., ZT ).

2. Time series: a time series is a sequence of random variables, which are ordered by
time, and denoted by {Zt , t ∈ Z}.

3. Auto-covariance function (ACVF):

Cov(Zt , Zs ) = E[(Zt − µt )(Zs − µs )]

= E(Zt Zs ) − µt µs

A useful formula
" m n
# m X
n
X X X
Cov ci Y i , dj Zj = ci dj Cov(Yi , Zj ).
i=1 j=1 i=1 j=1

4. Auto-correlation function (ACF):

Cov(Zt , Zs ) γ(t, s)
ρ(t, s) = Corr(Zt , Zs ) = p =p
V ar(Zt )V ar(Zs ) γ(t, t)γ(s, s)

5. Strict stationarity: a time series {Zt } is said to be strictly stationary if the joint
distribution of Zt1 , Zt2 , . . . , Ztn is the same as that of Zt1 −k , Zt2 −k , . . . , Ztn −k for
all choices of natural number n, all choices of time points t1 , t2 , . . . , tn and all
choices of time lag k.

6. Weak stationarity: A time series {Zt } is said to be weakly (second-order, or

covariance) stationary if

(a) the mean function µt is constant over time, and

(b) γ(t, t − k) = γ(0, k) for all times t and lags k.

7. White noise: A stationary process is said to be a white noise process if it is a

sequence of i.i.d random variables {at } with zero mean and variance σa2 > 0. Such
a white noise is usually abbreviated as W N (0, σa2 ).

1
2 Examples
1. Suppose that {Zt , t ∈ Z} is given by

Zt = at − 0.5at−1 , t ∈ Z, (1)

where the a’s are assumed to be a sequence of i.i.d. random variables with zero
mean and variance σa2 .

(a) The mean function of {Zt };

(b) The variance function of {Zt };
(c) The auto-covariance function (ACVF);
(d) The auto-correlation function (ACF).

2. Revisit the moving average example, Zt = at − 0.5at−1 in Question 1, where the

at ’s are assumed to be a sequence of i.i.d. random variables with zero mean and
variance σa2 . Is it weakly stationary?

3. Signal plus noise. Let {Xt } be a time series in which we are interested. However,
because the measurement process itself is subject to error in some circumstances,
we actually observe Zt = Xt + at . We assume that {Xt } and {at } are independent
processes and that {at } is white noise. We call {Xt } the signal and {at } the
measurement noise or error.
If {Xt } is stationary with autocorrelation function ρk , show that {Zt } is also sta-
tionary with
ρk
corr(Zt , Zt−k ) = 2
, k ≥ 1,
1 + σa2 /σX

where σa2 = var(at ) and σX

2
= var(Xt ). We call σX 2
/σa2 the signal-to-noise ratio,
or SNR. Show that the larger the value of the SNR, the closer the autocorrelation
function of the observed series Zt is to that of the desired signal {Xt }.

4. Is the following time series {Xt } weakly stationary? Specify the reasons for your
answers. Xt = A cos(tλ)+B sin(tλ), where A and B are given uncorrelated random
variables with mean zero and variance σ 2 and λ is a given number.