5.

INTRODUCTION TO BOX-JENKINS MODELS

5.1 Introduction
Probability models for time series: They are collectively known as
stochastic processes.
Stochastic processes are families of random variables that are functional
of time.
In practice, many time series are clearly non-stationary, and
so the stationary models cannot be applied directly.
In Box-Jenkins modeling, the general approach is to difference an observed
time series until it appears to come from a stationary process.

Monday, May 27, 2024 Time Series Analysis 1

Therefore, the Box-Jenkins approach is one of the most widely used
methodologies for the analysis of time series data that involves
 identifying an approximate ARIMA process,
 fitting it to the data and then using the fitted model for forecasting.
It is popular because of its generality; it can handle any series, stationary or
not,
with or without seasonal elements, and it has well-documented computer
programs.
5.2 The Concept of Stationary
Broadly speaking a time series is said to be stationary
 if there is no systematic change in mean (no trend),
 if there is no systematic change in variance and if strictly periodic variations
have been removed.
Monday, May 27, 2024 Time Series Analysis 2
5.3. ARIMA Models
Autoregressive models:Suppose that {  t} is a purely
random process with mean zero and variance  . Then a
2
e

process {yt} is said to be an autoregressive process of order

p, AR(p), if

Yt =  +  Yt-1+  Yt-2+…+  Yt-p+  t.

1 2 p

Monday, May 27, 2024 Time Series Analysis 3

In AR model, the current value of the process is expressed as a finite linear
aggregates of previous values of the process and a shock  t,

is a parameter that determines the “level” of the process.

Moving Average models:Suppose that {  t} is a purely

random process with mean zero and variance  . Then a 2
e

process {yt} is said to be a moving average process of order

q, MA(q), if

Yt =  +  t -  1  t-1 -  2  t-2 - …-  q  t-q

Monday, May 27, 2024 Time Series Analysis 4

 Autoregressive Moving Average , ARMA (p,q)
 are processes that are formed as combination of autoregressive
and moving average processes
 In combination take once for repeated terms

ARMA(p,q) has the form:

Yt =  +  Yt-1+  yt-2+…+  Yt-p-  t-1 -  t-2 - …-  t-q+  t.

1 2 p 1 2 q

Monday, May 27, 2024 Time Series Analysis 5

Autoregressive Integrated Moving Average ,ARIMA (p, q, d):

 Many models are capable of representing a wide range of non-stationary time

series.
 Such non stationary series that can be reduced to a stationary series with a
 degree of differencing (d)
 p autoregressive and
 q moving average term.

 Therefore, differencing is a procedure of converting a time series of length n

into another time series of length n-d.

Monday, May 27, 2024 Time Series Analysis 6

Operators
1. The backward shift operator, B, is defined such that
BYt = Yt-1. In general, this implies that

BjYt= Yt-j.

B2Yt= B(BYt) = BYt-1=Yt-2

2. The backward difference operator,,
 defined as Yt = Yt- Yt-1
But BYt =Yt-1

Yt – Yt-1 = Yt- BYt = (1-B)Yt, and therefore ,  = 1-B.

2Yt = (1-B)2Yt = (1-2B+B2)Yt

Monday, May 27, 2024 Time Series Analysis 7

 5.4 Methodological tools for model identification
1. Auto-covariance

Suppose two random variables X and Y have

means E(X) =  X and E(Y)=  Y respectively. Then
covariance of X and Y is
Cov(X, Y) = E[(X-  X )(Y-  Y )].

Monday, May 27, 2024 Time Series Analysis 8

With reference to time series, the covariance
between Yt and another observation Yt+k is called
auto-covariance at lag k, denoted by  k , and is
given by
 k = cov(Yt, Yt+k)
= E[(Yt-E(Yt))( Yt+k-E(Yt+k))]

Monday, May 27, 2024 Time Series Analysis 9

2. Autocorrelation:

 Autocorrelation coefficient measures the relationship, or

correlation, between a set of observations and a lagged set of
observations in a time series

k
cov(Yt , Yt  k )
k = = .
var(Yt ) var(Yt  k )  0

Monday, May 27, 2024 Time Series Analysis 10

 The sample autocorrelation coefficient at lag k (denoted by rk)
nk

 (Y  Y )(Y
t t k Y )
rk  t 1
n
, where Yt = the data from the stationary

 (Y  Y )
t 1
t
2 time series;
Yt+k = the data k time periods ahead

 A graph displaying the sample autocorrelation coefficient, r k,

versus the lag k is called the sample autocorrelation function
(ACF) or a correlogram.

Monday, May 27, 2024 Time Series Analysis 11

 This graph is useful both in determining whether or not a series
is stationary and in identifying a tentative ARIMA model.

Properties of Autocorrelation Coefficient

• - 1 ≤ r k≤ 1

• The ACF is an even function of the lag k (i.e. rk = r-k.).

This follows from the result
rk = cov(Yt, Yt+k) = cov(Yt-k, Yt)
• r is unit-less and for a random time series r is approximately
k k

normally distributed with mean zero and variance 1/n,

where n is number of observations.

Monday, May 27, 2024 Time Series Analysis 12

Partial autocorrelation coefficients

 is the measure of the relationship between two variables when

the effect of other variables has been removed or held constant.

The partial autocorrelation coefficient is denoted

by 𝜙kk.

The plot of partial autocorrelation coefficient (𝜙kk)

against the lag k gives the Partial Autocorrelation
Function (PACF)

Monday, May 27, 2024 Time Series Analysis 13

Note:𝜙00 =1, 𝜙11 =  1

Computing 𝜙kk.

Consider the following autocorrelation ‘matrix’ for a

stationary time series with length k is given by

Monday, May 27, 2024 Time Series Analysis 14

 𝜌𝑘 measures the
correlation
between Yt and
Yt+k, then

∗
where ℙ𝑘 is ℙ𝑘 with the last column replaced by
𝜌1
𝜌2
൦ … ൪ and ℙ∗𝑘 andℙ𝑘 are a k by k square matrice
𝜌𝑘
Monday, May 27, 2024 Time Series Analysis 15
 Note that, by definition, 𝜙11 = 𝜌1 =1 and the
sample partial autocorrelation coefficient is
denoted by 𝜙෠kk.
For example, if we compute sample partial autocorrelation, for k = 2.

Monday, May 27, 2024 Time Series Analysis 16

 ෠22for the following
Example: Compute r1, r2 and 𝜙
time series.
t 1 2 3 4 5 6 7 8 9 10
Yt 47 64 23 71 38 64 55 41 59 48

Solution: sum =510, mean = 51

T 1 2 3 4 5 6 7 8 9 10
Yt 47 64 23 71 38 64 55 41 59 48
Yt - -4 13 -28 20 -13 13 4 -10 8 -3
(Yt - )( Yt+1 - ) -52 -364 -560 -260 -169 52 -40 -80 -24
(Yt - )( Yt+2 - ) 112 260 364 260 -52 -130 32 30

Monday, May 27, 2024 Time Series Analysis 17

 σ 9𝑡=1(Y t − 𝑌ത)( Yt+1− 𝑌ത)
r1 = σ 10 ത
= -1497/1896 = -0.790
𝑡=1 (Yt− 𝑌 )
2

σ 8𝑡=1 (Yt− 𝑌ത)( Yt+2− 𝑌ത)

r2 = σ 10 ത 2
= 876/1896 = 0.462 and
𝑡=1 (Yt− 𝑌 )

1 𝑟1
ቤ ቤ 𝑟
𝑟 1𝑟 2 2−𝑟 2 0.462−ሺ−0.790ሻ2
𝜙෠22 = 1 𝑟 = 1
= = -0.431.
1 1−𝑟12 1−(−0.790)2
ቤ ቤ
𝑟1 1

Monday, May 27, 2024 Time Series Analysis 18

 5.5 Stages of Box-Jenkins methodology
The basis of Box-Jenkins approach to modeling time series consists of three phases:

 Model selection/Identification
 Parameter estimation
 Model checking/Diagnostics

A. Model identification
 The data must also be edited to deal with:
 Extreme
 Missing values
 Other distortions

Monday, May 27, 2024 Time Series Analysis 19

 Data Editing is done through:
 the use of functions as log or inverse transformation to achieve
stabilization (Stationary) and
 differencing to avoid obvious patterns such as trend and seasonality.

 Once stationarity have been addressed, we need to decide how many

autoregressive (p) and moving average (q) parameters are necessary to
yield an effective model of the process.

 The major tools for doing this are autocorrelation function (ACF), and
partial autocorrelation function (PACF).

Monday, May 27, 2024 Time Series Analysis 20

B. Parameter estimation:
 Once a tentative model has been identified, the estimates for the
constant and the coefficients of the equation must be obtained.
 It is a way that finding the values of the model coefficient
(like 𝜙1 ,𝜙2 , 𝜙3 , …,𝜙𝑝 ,𝜃1 ,𝜃2 ,𝜃3 , .., 𝜃𝑞 ).
 There are several methods such as methods of moments, maximum
likelihood, and least squares that can be employed to estimate the parameters
in the tentatively identified model. But most ARIMA models are nonlinear
models and require the use of a nonlinear model fitting procedure called quasi-
Newton method. The user may have the choice of estimation method and can
accordingly choose the most appropriate method based on the problem
specifications. This is usually automatically performed by sophisticated software
packages such as Minitab, JMP, and SAS.
Monday, May 27, 2024 Time Series Analysis 21
C. Model checking/Diagnostics

 After a tentative model has been fit to the data, we must

examine its adequacy and, if necessary, suggest potential
improvements.
This is done through residual analysis. = t ˆ(Y  Y )
t t

 If the specified model is adequate and hence the appropriate

orders p and q are identified, it should transform the observations
to a white noise process. Thus the residuals  t should behave like
white noise. Or purely random residuals.
 white noise: if {X t } is a sequence of uncorrelated random
variables, each with zero mean and variance.
Monday, May 27, 2024 Time Series Analysis 22
Monday, May 27, 2024 Time Series Analysis 23
 6. MODEL IDENTIFICATION AND ESTIMATION
Introduction
The Box-Jenkins approach consists of extracting the predictable movements
(pattern) from the observed data through a series of iterations.
One first tries to identify a possible model from a general class of linear models.
The chosen model is then checked against the historical data to see if it accurately
describes the underlying process that generates the series.
 If the specified model is not satisfactory, the process is repeated by using
another model designed until a satisfactory model is found.

Box-Jenkins models can only describe or represent stationary series

or series that have been made stationary by differencing.
The models fall into one of the three following categories:
1. Autoregressive (AR),
2. Moving average (MA) and
3. Mixed process (ARMA).
If differencing is applied together with AR and MA, they are referred to as
Autoregressive Integrated Moving Average (ARIMA), with the ‘I’ indicating
"integrated" and referencing the differencing procedure.
Monday, May 27, 2024 Time Series Analysis 24
 .

A. Autoregressive models
 In autoregressive models, the current value of the process Yt is a
linear function of past stationary observations Yt-1, Yt-2, Yt-3,…. and
the current shock εt.
where {εt} denotes a purely random process with zero mean and
Variance 𝜎𝜀2 .
AR(p) model:
Yt    1Yt 1   2Yt  2  ........... p Yt  p   t ,
 The number of past stationary observations used in an
autoregressive model is known as the order, p.

Monday, May 27, 2024 Time Series Analysis 25

  Let Yt - μ = Zt
 Yt-1 - μ = Zt-1 be deviation of values from μ.
etc.
Then the process is rewritten as:
Z t  1 Z t 1   2 Z t  2  ........... p Z t  p   t
.
Notice that the above model can be written in terms of Zt by using the
backward shift operator B.
BZt =Zt-1 B2Zt = Zt-2 …,BpZt =Zt-p

Monday, May 27, 2024 Time Series Analysis 26

 The AR(p) model may be written in the form:

Z t  (1 B   2 B 2  .........   p B p ) Z t   t .
 t   ( B) Z t
 Where:
 (B)= 1  1 B   2 B  .........   p B
2 p

Monday, May 27, 2024 Time Series Analysis 27