Lesson 5: The Autocovariance Function of a

stochastic process

Umberto Triacca

Dipartimento di Ingegneria e Scienze dell’Informazione e Matematica

Università dell’Aquila,
umberto.triacca@ec.univaq.it

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Mean and Autocovariance Functions of a stochastic
process

A discrete stochastic process {xt ; t ∈ Z} is a family of random

variables indexed by a parameter t (usually the time).
Thus the moments of the random variables in a stochastic process
are function of the parameter t.
We will consider two moments functions:
1 The mean function;
2 The autocovariance function.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Mean Function of a stochastic process

First, we consider the mean function.

Definition. Let {xt ; t ∈ Z} be a stochastic process such that
Var(xt ) < ∞ ∀t ∈ Z. The function

µx : Z → R

defined by
µx (t) = E (xt )
is called Mean Function of the stochastic process {xt ; t ∈ Z}.

The mean function describes the expectation of the random

variables in the process.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Mean Function of a Random Walk with Drift

Let
{xt ; t = 0, 1, 2, ...}
be a stochastic processs where x0 = δ and xt = λ + xt−1 + ut for
t = 1,2,..., with ut ∼ WN(0, σu2 ).

This process is called random walk with drift. The constant λ is

called the drift.
The mean function of this process is

µx (t) = δ + λt

which is linear function with intercept δ and slope λ.

Why?

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a stochastic process
Definition. Let {xt ; t ∈ Z} be a stochastic process such that
Var(xt ) < ∞ ∀t ∈ Z. The function

γx : Z × Z → R

defined by
γx (t1 , t2 ) = cov(xt1 , xt2 )
is called Autocovariance Function of the stochastic process
{xt ; t ∈ Z}.

The autocovariance function describes the strength of the linear

relationship between the random variables xt1 and xt2 .
It is clear that autocovariance function evaluated in (t,t) gives the
variance, because
h i
γx (t, t) = E (xt − µt )2 = var(xt )

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a Random Walk

Let
{xt ; t = 0, 1, 2, ...}
be a random walk processs with initial condition x0 = 0, and where
xt = xt−1 + ut for t = 1,2,..., with ut ∼ WN(0, σu2 ).

We have
γx (t1 , t2 ) = min {t1 , t2 } σu2

Why?

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a stationary stochastic
process

Consider a weakly stationary stochastic process {xt ; t ∈ Z}.

We have that
γx (t + k, t) = cov(xt+k , xt ) = cov(xk , x0 ) = γx (k, 0) ∀t, k ∈ Z.
We observe that γx (t + k, t) does not depend on t. It depends
only on the time difference k, therefore is convenient to redefine
the autocovariance function of a weakly stationary process as the
function of one variable.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a weakly stationary
process

Definition. The function

γx : Z → R

defined by
γx (k) = Cov(xt , xt−k )
is called autocovariance function of the weakly stationary
stochastic process {xt ; t ∈ Z}.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a weakly stationary
process

Example. Consider a stochastic process {xt ; t ∈ Z} defined by

xt = ut + θut−1
with ut ∼ WN(0, σu2 ).

It is possible to show that this process is weakly stationary.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a weakly stationary
process

The autocovariance function of this process is given by

γx (k) = Cov(xt , xt−k )

 1 + θ2 σu2


for k = 0
= θσu2 for k = 1
0 for k = 2, 3, ...


We note that autocovariance function of this process ‘cuts off’

after lag 1.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a weakly stationary
process

Some basic properties of the autocovariance function are:

1 γx (0) ≥ 0
2 |γx (k)| ≤ γx (0) ∀k
3 γx (k) = γx (−k) ∀k

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Functions of a weakly stationary
process

γx (0) ≥ 0 The first is simply the statement that Var(xt ) ≥ 0

|γx (k)| ≤ γx (0) ∀k The second is an immediate consequence

of the fact that the correlation between xt and xt−k is less
than or equal to 1 in absolute value

γx (k) = γx (−k) ∀k The third follows straight from the

definition

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a stationary process

Another important property of γx (.) is that it is non-negative

definite, that is
X n Xn
αi γx (i − j)αj ≥ 0
i=1 j=1

for all positive integers n and vectors α = (α1 , ..., αn )0 ∈ Rn .

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

Parametric Functions of a covariance-stationary process

In fact, we have
n X
n n
!
X X
αi γx (i − j)αj = Var αi xi
i=1 j=1 i=1

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a stationary process

In the class of stationary, zero mean, Gaussian processes

there is a one-to-one correspondence between the family of
the finite dimensional distributions and autocovariance
function

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function of a stationary process

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

The Autocovariance Function

Due to this one-to-one correspondence the statistical properties of

a stationary, zero mean, Gaussian process are completely
determined by its autocovariance function.

Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process

Autocovariance function

Stationary, zero mean, Gaussian process

'$

DGP γx (k)
&%
7










 ? 
x1 , ..., xT
 
Umberto Triacca Lesson 5: The Autocovariance Function of a stochastic process