Time Series Forecasting

Time-series Forecasting
Introduction
A time series is a sequence of observations recorded over a certain period of time.
A simple example of time series is how we come across different temperature
changes day by day or in a month.
There are different methods to forecast the values.
while Forecasting time series values, 3 important terms need to be taken care of
and the main task of time series forecasting is to forecast these three terms.
1) Seasonality
Seasonality is a simple term that means while predicting a time series data there
are some months in a particular domain where the output value is at a peak as
compared to other months. for example if you observe the data of tours and travels
companies of past 3 years then you can see that in November and December the
distribution will be very high due to holiday season and festival season. So while
forecasting time series data we need to capture this seasonality.
2) Trend
The trend is also one of the important factors which describe that there is certainly
increasing or decreasing trend time series, which actually means the value of
organization or sales over a period of time and seasonality is increasing or
decreasing.
3) Unexpected Events
Unexpected events mean some dynamic changes occur in an organization, or in
the market which cannot be captured. for example a current pandemic we are
suffering from, and if you observe the Sensex or nifty chart there is a huge
decrease in stock price which is an unexpected event that occurs in the
surrounding.
Methods and algorithms are using which we can capture seasonality and trend But
the unexpected event occurs dynamically so capturing this becomes very difficult.
Rolling Statistics and Stationarity in Time-series
A stationary time series is a data that has a constant mean and constant variance.
If I take a mean of T1 and T2 and compare it with the mean of T4 and T5 then is
it the same, and if different, how much difference is there? So, constant mean
means this difference should be less, and the same with variance.
If the time series is not stationary, we have to make it stationary and then proceed
with modelling. Rolling statistics is help us in making time series stationary. so
basically rolling statistics calculates moving average. To calculate the moving
average we need to define the window size which is basically how much past
values to be considered.
For example, if we take the window as 2 then to calculate a moving average in the
above example then, at point T1 it will be blank, at point T2 it will be the mean of
T1 and T2, at point T3 mean of T3 and T2, and so on. And after calculating all
moving averages if you plot the line above actual values and calculated moving
averages then you can see that the plot will be smooth.
This is one method of making time series stationary, there are other methods also
which we are going to study as Exponential smoothing.
Additive and Multiplicative Time series
In the real world, we meet with different kinds of time series data. For this, we
must know the concepts of Exponential smoothing and for this first, we need to
study types of time series data as additive and multiplicative. As we studied there
are 3 components we need to capture as Trend(T), seasonality(S), and
Irregularity(I).
Additive time series is a combination(addition) of trend, seasonality, and
Irregularity while multiplicative time series is the multiplication of these three
terms.
Components of Time Series
Time series analysis provides a body of techniques to better understand a dataset.
Perhaps the most useful of these is the decomposition of a time series into 4
constituent parts:
1. Level. The baseline value for the series if it were a straight line.
2. Trend. The optional and often linear increasing or decreasing behavior of
the series over time.
3. Seasonality. The optional repeating patterns or cycles of behavior over time.
4. Noise. The optional variability in the observations that cannot be explained
by the model.
Time Series Models:
There are two types of time series models:
1. Univariate time series: Time series with a one time-dependent variable

and a single independent variable.
2. Multivariate time series: Time series with one time-dependent variable
and more than one independent variable.
Types of forecasting methods
Model Use
Decompositional Deconstruction of time series
Smooth-based Removal of anomalies for clear patterns
Moving-Average Tracking a single type of data
Exponential
Smooth-based model + exponential window function
Smoothing
TOP 10 ALGORITHMS
1. Autoregressive (AR)
2. Autoregressive Integrated Moving Average (ARIMA)
3. Seasonal Autoregressive Integrated Moving Average (SARIMA)
4. Exponential Smoothing (ES)
5. XGBoost
6. Prophet
7. LSTM (Deep Learning)
8. DeepAR
9. N-BEATS
Moving Averages
Moving averages alone aren’t that useful for forecasting. Instead, they are mainly
used for analysis. For example, moving averages help stock investors in technical
analysis by smoothing out the volatility of constantly changing prices. This way,
short-term fluctuations are reduced, and investors can get a more generalized
picture. Math-wise, simple moving averages are dead simple to implement:
Where t represents the time period and s the size of a sliding window.
Let’s take a look at an example. x will represent a sample time series without the
time information, and we’ll calculate moving averages for sliding window sizes 2
and 3.
Here’s the MA(2) calculation:
As you can see, the first data point is copied, as there’s no data point before it to
calculate the average. Pandas won’t copy the value by default, but will instead
return NaN. You can specify min_periods=1 inside the rolling() functions to avoid
this behavior.
The calculations are dead simple, once again. The first value is copied, but the
second value is calculated like MA(2), since there isn’t enough data for a complete
calculation. You can use the rolling() function in Pandas to calculate simple
moving averages. The only parameter you should care for right now is window, as
it specifies the size of a sliding window. Read the previous section once again if
you don’t know what it represents.
Calling rolling() by itself won’t do anything. You still need an aggregate function.
Since these are moving averages, sticking .mean() to the end of a call will do.
Looking at the above visualization uncovers a couple of problems with simple
moving averages:
• Lag — moving average time series always lags from the original one.
Look at the peaks to verify that claim.
• Noise — too small sliding window size won’t remove all noise from the
original data.
• Averaging issue — averaged data will never capture the low and high
points of the original series due to, well, averaging.
• Weighting — identical weights are assigned to all data points. This can
be an issue as frequently the most recent values have more impact on the
future.
Exponential Moving Average

Similarly to the Weighted Moving Average, the Exponential Moving Average
(EMA) assigns a greater weight to the most recent price observations. While it
assigns lesser weight to past data, it is based on a recursive formula that includes
in its calculation all the past data in our price series.
The EMA at time t is calculated as the current price multiplied by a smoothing

factor alpha (a positive number less than 1) plus the EMA at time 𝑡−1 multiplied
by 1 minus alpha. It is basically a value between the previous EMA and the current
price:
The smoothing factor 𝛼 ( alpha ) is defined as:

where 𝑛 is the number of days in our span. Therefore, a 10-day EMA will have a
smoothing factor:
Pandas includes a method to compute the EMA moving average of any time
series: .ewm(). Will this method respond to our needs and compute an average that
matches our definition?
Let's test it:

ema10 = data['Price'].ewm(span=10).mean()ema10.head(10)
Which gives:
Exponential Smoothing Method

The drawback of Simple Moving Average Method is that it gives equal weight to
all the observations. Intuitively the most recent observation should be given more
weight that the earlier observations. The Exponential Smoothing
Method removes this limitation by assigning differential weights to the past
observations. Here, the weights assigned to past data decline exponentially
with the most recent observations assigned higher weights. The rate at which
the weights decreases is controlled by a hyperparameter also called the
‘smoothing constant’ .
Another important advantage of exponential smoothing method over simple

averaging method is that the smoothing constant can be easily optimized using
‘Solver’ to minimize the mean squared errors.
Exponential Smoothing Methods are of three types :
1. Single Exponential Smoothing

2. Double Exponential Smoothing and
3. Triple Exponential Smoothing or Holt Winters Method
Let’s cover each one of them in detail.
Single Exponential Smoothing :

This method addresses only the level component of the time series. It uses a hyper
parameter alpha which is called a smoothing constant, the value of which lies
between 0 and 1. Since only one smoothing constant is used, it is called Single
Exponential Smoothing.
Here the forecast at time t is given as Ft = α * y(t-1) + (1-α)*F(t-1)
Double Exponential Smoothing :

This addresses both the level(l) and trend (b) component of the time series.
Thus, two smoothing constants are used i.e. alpha for the level component and
beta for the trend component. The equations are as follows:
l(t) = α * y(t) + (1-α)*(l(t-1)+b(t-1)) — — — — — Level l

b(t) = β * (l(t) - l(t-1)) + (1-β)* b(t-1) — — — — Trend b
y(t+1) = l(t) + b (t) — — — — Forecast
Models with low values of beta assume that the trend changes very slowly over
time while models with larger value of beta assume that the trend is changing
very rapidly.
Single Exponential Smoothing method assumes the time series to be relatively

stable with no trend and seasonality. However, in this case, the data exhibits strong
trend and seasonality. Thus, we cannot use single exponential method of
forecasting. The double exponential smoothing method takes account of the
average or level component and trend component. It does not account for
seasonality. Thus, we cannot use double exponential smoothing as well.
We will thus use Triple Exponential Smoothing, also known as Holt Winter
Model. It takes into account level, trend and seasonal components.
Triple Exponential Smoothing / Holt Winter’s Method :

In this method, we apply smoothing to seasonal component in addition to level and
trend components. The smoothing is applied across seasons. That is, the fourth
component of one season is smoothed against the fourth component of the previous
season, fourth component of two seasons ago and so on. Thus, there will be four
equations — one each for level, trend, seasonality and the final equation with
all the individual components.
The equations vary given the model is additive or multiplicative. Honestly, one
needs to put in significant effort to understand the math behind these equations. In
case you want to go into more depth on the intuition behind the equations
The four equations of the additive Holt Winter’s Method is given as :
Here s is the season length i.e. the number of data points after which a new season
begins.
Autoregression Model
Before you learn what is autoregression, let’s recall what is a regression model.
A regression model is a statistical technique to estimate the relationship

between a dependent variable (y) and an independent variable (X). Thus, while
working with a regression model, you deal with two variables.
For example, you have the stock prices of Bank of America (referred to as
BAC) and the stock prices of J.P. Morgan (referred to as JPM).
• Now you want to predict the stock price of JPM based on the stock price
of BAC.
• Here, the stock price of JPM would become the dependent variable, y
and the stock price of BAC would be the independent variable X.
Assuming that there is a linear relationship between X and y, the
•
regression model estimating the relationship between x and Y is of the
form
y=mX+cy=mX+c
where m is the slope of the equation and c is the constant.
Now, let’s suppose you only have one series of data, say, the stock price of
JPM. Instead of using a second time series (the stock price of BAC), the current
value of the stock price of JPM will be estimated based on the past stock price
of JPM. Let the observation at any point in time, t, be denoted as yt. You can
estimate the relationship between the value at any point t, yt, and the value at
any point (t-1), yt-1using the regression model as below.
AR(1)=yt=Φ1yt−1+cAR(1)=yt=𝚽1yt−1+c
where 𝚽1is the parameter of the model and c is the constant. This is the
autoregression model of order 1. The term autoregression means regression of
a variable against its own past values.
Like the linear regression model, the autoregression model assumes that there
is a linear relationship between yt and yt-1. This is called the autocorrelation.
You will be learning more about this later.
Let us have a look at other orders of the autoregression model.
Autoregression Models of Order p

In the above example, you saw how the value of yt is estimated using one past
value (lag term), yt-1. You can use more than one past term to predict the value
at time t. Let’s see the form of the model when two lag values are used in the
model.
AR(2)=yt=Φ1yt−1+Φ2yt−2+cAR(2)=yt=𝚽1yt−1+𝚽2yt−2+c
where 𝚽1 and 𝚽2 are the parameters of the model and c is the constant. This is
the autoregression model of order 2. Here yt is assumed to have a correlation
with its past two values and is predicted as a linear combination of past 2 values.
This model can be generalised to order p as below.
AR(p)=yt=Φ1yt−1+Φ2yt−2+...+Φpyt−p+cAR(p)=yt=𝚽1yt−1+𝚽2yt−2+...+𝚽py
t−p+c
where 𝚽p's ; (p = 1, 2, ..., p) are the parameters of the model and c is the
constant. This is the autoregression model of order p. Here yt is assumed to
have a correlation with its past p values and is predicted as a linear combination
of past p values.
There are a number of ways available to compute the parameters 𝚽p's.
Autocorrelation & Partial Autocorrelation

As you have already seen, an autoregression model predicts the current value
based on past values. That means that the model assumes that the past values
of the time series are affecting its current value. This is called the
autocorrelation.
In other words, autocorrelation is nothing but a correlation coefficient. But the
correlation here is not measured between two variables. It is measured between
a time series with its own lagged values over successive time intervals.
As the name suggests, autocorrelation means correlation with itself.

Sometimes, autocorrelation is also referred to as serial correlation or lagged
correlation.
Just like the correlation coefficient, autocorrelation also measures the degree
of relationship between a variable’s current value and its past values. The value
of autocorrelation lies between -1 and +1.
• -1 means a perfect negative autocorrelation,

• +1 means a perfect positive autocorrelation, and
• 0 means no autocorrelation.
As autocorrelation measures the relationship between the current value and the
past value of the data, it is very useful in measuring the randomness of data.
This randomness in the data can be detected using the autocorrelation
function (ACF) plot.
Let us understand what is partial autocorrelation.
Partial autocorrelation is the conditional correlation between the variable and

it’s lagged value. That means, the partial autocorrelation between the current
value of the time series, yt and it’s lagged value yt-h would be the conditional
correlation between yt and yt-h, conditional on all the lag terms in between t and
t-h, i.e. yt-1, yt-2, ..., yt-h+1. This mean, unlike the autocorrelation values, the
partial autocorrelation values controls the other lag orders and ignore their
effect.
A partial autocorrelation function (PACF) plot is used to identify the order
of the autoregression model.
What is autocovariance?
First, you need to understand what covariance and correlation are. Remember
that covariance is applied to 2 assets. The autocovariance is the same as the
covariance.
The only difference is that the autocovariance is applied to the same asset, i.e.,
you compute the covariance of the asset price return X with the same asset price
return X, but from a previous period.
How’s that possible? Simple, check it out:
Where X and Y can be the returns of asset X and Y, respectively.
Now, the autocovariance function can be defined as:
Where:
γs: Autocovariance at lag “s”.γs: Autocovariance at lag “s”.

rt: Asset price returns at time “t”.rt: Asset price returns at time “t”.
rt−s: Asset price returns at time “t−s”.rt−s: Asset price returns at time “t−s”.
What is autocorrelation?
In simple terms, autocorrelation is the same as the correlation function! To be

specific, autocorrelation, as the autocovariance, is applied to the same asset.
Check the difference between correlation and autocorrelation (also called serial
correlation) below:
Where:
Cov(X,Y: Covariance between X and Y.Cov(X,Y: Covariance

between X and Y.
SDX: Standard Deviation of variable X.SDX: Standard Deviation of
variable X.
SDY: Standard Deviation of variable Y.SDY: Standard Deviation of
variable Y.
Now let’s check the autocorrelation:
Where:
ρs: Autocorrelation at lag “s”.ρs: Autocorrelation at lag “s”.

rt: Asset price returns at time “t”.rt: Asset price returns at time “t”.
rt−s: Asset price returns at time “t−s”.rt−s: Asset price returns at time “t−s”.
Var(rt): Variance of returns.Var(rt): Variance of returns.
You might ask us:
Why variance and not the multiplication of the standard deviation of the returns
at the different lags?
Well, you must remember that an ARMA model is applied to stationary time
series. This topic belongs to time series analysis. Consequently, it is assumed
that the price returns, if stationary, have the same variance for any lag, i.e.:
What are the autocovariance and autocorrelation at lag zero?
Interesting question and simple to answer! Let’s see first for the former:
Since we know, from the above algebraic calculation, that the covariance of the
same variable is its variance, we have the following:
Consequently, the autocorrelation function for any asset price return at lag 0 is
always 1.
Calculation of the autocovariance with an example

You might have been thinking up to now:
Why are the autocovariance and autocorrelation defined with an “s”
subscript?
Great question!
Let us explain: Actually, the autocovariance formula defined above is a
function which allows the calculation of the autocovariance for different lags.
The same for the autocorrelation function.
Confused? Don’t worry! We got you covered!

Let’s see an example to make the concept clear to your thoughts! We are going
to make an example of how to calculate the autocovariance of the Microsoft
price returns at lag 1. We are going to use the autocovariance function shown
above.
Imagine we have the following returns for Microsoft prices:
Day Day Day Day Day Day Day Day Day Day
1 2 3 4 5 6 7 8 9 10
5% 1% -2% 3% -4% 6% 2% -1% -3% 4%
Let’s suppose we want to compute the autocovariance at lag 1. You will need
the returns up to day 10, and the 1-period lagged returns up to day 9.
Thus, you have the following data structure for returns on days 10 and 9:
Da
Variab Da Da Da Da Da Da Da Da Da y
le y1 y2 y3 y4 y5 y6 y7 y8 y9 10
- - - -
5% 1% 2% 3% 4% 6% 2% 1% 3% 4%
rtrt
- - - -
rt−1rt− 5% 1% 2% 3% 4% 6% 2% 1% 3%
1
Do you get to see the difference between the 2 variables?

The second one is the first lag of Xt.
Now, since the 2 variables have different dimensions (the first one has 10
observations, while the second one has 9), we are going to use data from day 2
onwards.
Consequently, our data is as follows:

Day Day Day Day Day Day Day Day Day
Variable 2 3 4 5 6 7 8 9 10
1% -2% 3% -4% 6% 2% -1% -3% 4%

rtrt
5% 1% -2% 3% -4% 6% 2% -1% -3%
rt−1rt−1
The covariance between these 2 variables will be the autocovariance of the
returns at lag 1.
You can do this, right?

Check an example we give in our previous article.
Before you get ready to use a pencil and a piece of paper, let us tell you
something important.
Remember the autocovariance formula:
If you paid attention to details, you could see that the average return is the same
for both returns, in our case, for returns up to day 10 and up to day 9. As we
explained before, autocovariance and autocorrelation functions are applied
only to stationary time series.
Consequently, not only the variance but also the mean is a unique value for the
whole span. That’s why the mean is the same for any lag of the price returns.
The mean of the Microsoft price returns is 1.1%. Let’s follow the procedure to
compute the autocovariance
The autocovariance is just the sum of the last column values divided by (N-1,
equal to 8), which results in -0.046%.
Calculation of the autocorrelation with an example

Let’s follow the same exercise and compute the autocorrelation of the
Microsoft price returns up to day 10 at lag 1. The autocorrelation is the
autocovariance divided by the variance. We give you the exact hint you need:
The variance of Microsoft price returns up to day 10 is 0.121%.
Let’s follow the algebraic formulas and use the numbers to compute the
autocorrelation:
What Is partial autocorrelation?

Since our mission is to prepare you to model an ARMA process, we need to
explain to you this function. For example, the autocorrelation function at lag 5
might have correlations with its previous lags’ autocorrelations. The partial
autocorrelation function gives the autocorrelation at lag 5, but without the
relationship of the shorter lags’ autocorrelations.
Putting it in another way, the autocorrelation of Microsoft price returns at lag
5 is about the autocorrelation between returns at time t and at the time (t-5).
But this autocorrelation is also influenced by the correlations from lag 1 to lag
4.
The partial autocorrelation is a function that allows having the autocorrelation

of returns t and (t-5) removing the indirect relationship that returns from lag 1
to 4 have on it.
Autocorrelation Function (ACF) Plot & Partial Autocorrelation Function

(PACF) Plot
An autocorrelation function plot is the plot of the autocorrelation for the
different lagged values. r1 measures the correlation between the variable and its
first lagged value, i.e. yt and yt-1. Similarly, r2 measures the correlation between
the variable and its second lagged value, i.e. yt and yt-2. And so on.
An ACF plot will plot the values of r0, r1, r2, ... corresponding to the respective
lag orders 0, 1, 2, … These values are plotted with a confidence band that helps
us identify whether a value is statistically significant or not. The same has been
illustrated while visualising the ACF plot in Python.
Note that r0 is the correlation between the variable with itself, and hence will
always be equal to 1.
The ACF plot is a good indicator of the randomness of the data. For non-
random data, the value of the autocorrelation for at least one lagged term would
be statistically significant (significantly non-zero). However, this is not the
only measure of randomness. Zero autocorrelation for all the lagged terms does
not necessarily mean random data and vice versa.
Just like an autocorrelation function plot, a partial autocorrelation function plot

is the plot of the partial autocorrelation for different lagged terms. They are
also plotted with a confidence band that helps us identify the significant lagged
terms, which then becomes the order of the autoregression model.You can plot
the ACF and PACF plots using the plot_acf and plot_pacf methods from the
statsmodels library respectively.
From the above plot, you can see that the value of autocorrelation at lag 0 is 1
(as it is the correlation of the variable with itself). The blue region that you see
is the confidence band and autocorrelation up to lag 20 lies outside this blue
region.
This means that values up to lag 20 are statistically significant, that is, they
affect the current price. Also, the autocorrelation is gradually approaching zero
as the lag term increases. This means that farther we go, lesser is the correlation.
Form the above plot you can see that lag 1, 2, 3, 4, etc are outside the confidence
band (blue region) and hence are statistically significant.
Finally, you will learn to estimate the parameters of the autoregression models.
But before that, let’s see if there are any challenges.
Limitations of Autoregression Models

A very important point to note that an autoregression model makes an
assumption that the underlying data comes from a stationary process. A
stationary time series is a time series whose statistical properties like mean and
variance are independent of the point in time where it is observed. You can read
more about Stationarity here.
As most of the time series in real life are non-stationary, the use of AR models
cannot be used on them without transforming the data.
Introduction to ARIMA Models

So what exactly is an ARIMA model?
ARIMA, short for ‘Auto Regressive Integrated Moving Average’ is actually a
class of models that ‘explains’ a given time series based on its own past values,
that is, its own lags and the lagged forecast errors, so that equation can be used
to forecast future values. Know more about parameters of ARIMA and its
limitations, in this free video tutorial
Any ‘non-seasonal’ time series that exhibits patterns and is not a random white
noise can be modeled with ARIMA models.
An ARIMA model is characterized by 3 terms: p, d, q where,

p is the order of the AR term
q is the order of the MA term
d is the number of differencing required to make the time series stationary
If a time series, has seasonal patterns, then you need to add seasonal terms and
it becomes SARIMA, short for ‘Seasonal ARIMA’. More on that once we finish
ARIMA.
So, what does the ‘order of AR term’ even mean? Before we go there, let’s first
look at the ‘d’ term. The first step to build an ARIMA model is to make the
time series stationary.
Because, term ‘Auto Regressive’ in ARIMA means it is a linear regression
model that uses its own lags as predictors. Linear regression models, as you
know, work best when the predictors are not correlated and are independent of
each other.
So how to make a series stationary?
The most common approach is to difference it. That is, subtract the previous
value from the current value. Sometimes, depending on the complexity of the
series, more than one differencing may be needed.
The value of d, therefore, is the minimum number of differencing needed to

make the series stationary. And if the time series is already stationary, then d =
0.
Next, what are the ‘p’ and ‘q’ terms?
‘p’ is the order of the ‘Auto Regressive’ (AR) term. It refers to the number of
lags of Y to be used as predictors. And ‘q’ is the order of the ‘Moving Average’
(MA) term. It refers to the number of lagged forecast errors that should go into
the ARIMA Model.
AR and MA models
So what are AR and MA models? what is the actual mathematical formula for
the AR and MA models?
A pure Auto Regressive (AR only) model is one where Yt depends only on its
own lags. That is, Yt is a function of the ‘lags of Yt’.
where, $Y{t-1}$ is the lag1 of the series, $\beta1$ is the coefficient of lag1 that
the model estimates and $\alpha$ is the intercept term, also estimated by the
model.
Likewise a pure Moving Average (MA only) model is one where Yt depends
only on the lagged forecast errors.
where the error terms are the errors of the autoregressive models of the
respective lags. The errors Et and E(t-1) are the errors from the following
equations :
[Error: The beta coefficients in the second equation above is incorrect. ]

That was AR and MA models respectively.
So what does the equation of an ARIMA model look like?

An ARIMA model is one where the time series was differenced at least once to
make it stationary and you combine the AR and the MA terms. So the equation
becomes:
ARIMA model in words:
Predicted Yt = Constant + Linear combination Lags of Y (upto p lags) + Linear

Combination of Lagged forecast errors (upto q lags)
The objective, therefore, is to identify the values of p, d and q. But how?
Let’s start with finding the ‘d’.
5. How to find the order of differencing (d) in ARIMA model

The purpose of differencing it to make the time series stationary.
But you need to be careful to not over-difference the series. Because, an over
differenced series may still be stationary, which in turn will affect the model
parameters.
So how to determine the right order of differencing?
The right order of differencing is the minimum differencing required to get a

near-stationary series which roams around a defined mean and the ACF plot
reaches to zero fairly quick.
If the autocorrelations are positive for many number of lags (10 or more), then
the series needs further differencing. On the other hand, if the lag 1
autocorrelation itself is too negative, then the series is probably over-
differenced.
In the event, you can’t really decide between two orders of differencing, then
go with the order that gives the least standard deviation in the differenced
series.
Let’s see how to do it with an example.
First, I am going to check if the series is stationary using the Augmented
Dickey Fuller test (adfuller()), from the statsmodels package.
Why?
Because, you need differencing only if the series is non-stationary. Else, no

differencing is needed, that is, d=0.
The null hypothesis of the ADF test is that the time series is non-stationary. So,
if the p-value of the test is less than the significance level (0.05) then you reject
the null hypothesis and infer that the time series is indeed stationary.
So, in our case, if P Value > 0.05 we go ahead with finding the order of
differencing.
Since P-value is greater than the significance level, let’s difference the series
and see how the autocorrelation plot looks like.
Order of Differencing
For the above series, the time series reaches stationarity with two orders of
differencing. But on looking at the autocorrelation plot for the 2nd differencing
the lag goes into the far negative zone fairly quick, which indicates, the series
might have been over differenced.
So, I am going to tentatively fix the order of differencing as 1 even though the
series is not perfectly stationary (weak stationarity).
Find the order of the AR term (p)
The next step is to identify if the model needs any AR terms. You can find out
the required number of AR terms by inspecting the Partial Autocorrelation
(PACF) plot.
But what is PACF?
Partial autocorrelation can be imagined as the correlation between the series

and its lag, after excluding the contributions from the intermediate lags. So,
PACF sort of conveys the pure correlation between a lag and the series. That
way, you will know if that lag is needed in the AR term or not.
So what is the formula for PACF mathematically?
Partial autocorrelation of lag (k) of a series is the coefficient of that lag in the
autoregression equation of Y.
That is, suppose, if Y_t is the current series and Y_t-1 is the lag 1 of Y, then
the partial autocorrelation of lag 3 (Y_t-3) is the coefficient $\alpha_3$ of Y_t-
3 in the above equation.
Good. Now, how to find the number of AR terms?
Any autocorrelation in a stationarized series can be rectified by adding enough

AR terms. So, we initially take the order of AR term to be equal to as many
lags that crosses the significance limit in the PACF plot.
Order of AR Term
You can observe that the PACF lag 1 is quite significant since is well above the
significance line. Lag 2 turns out to be significant as well, slightly managing to
cross the significance limit (blue region). But I am going to be conservative and
tentatively fix the p as 1.
How to find the order of the MA term (q)

Just like how we looked at the PACF plot for the number of AR terms, you can
look at the ACF plot for the number of MA terms. An MA term is technically,
the error of the lagged forecast.
The ACF tells how many MA terms are required to remove any autocorrelation
in the stationarized series.
Let’s see the autocorrelation plot of the differenced series.
Order of MA Term
Couple of lags are well above the significance line. So, let’s tentatively fix q as
2. When in doubt, go with the simpler model that sufficiently explains the Y.
How to handle if a time series is slightly under or over differenced

It may so happen that your series is slightly under differenced, that differencing
it one more time makes it slightly over-differenced.
How to handle this case?
If your series is slightly under differenced, adding one or more additional AR

terms usually makes it up. Likewise, if it is slightly over-differenced, try adding
an additional MA term.
How to build the ARIMA Model

Now that you’ve determined the values of p, d and q, you have everything
needed to fit the ARIMA model. Let’s use the ARIMA() implementation
in statsmodels package. (** You can also check out the free video lesson
on forecasting restaurant visitors with ARIMA and then check how to test and
improve the model)
from statsmodels.tsa.arima_model import ARIMA
# 1,1,2 ARIMA Model
model = ARIMA(df.value, order=(1,1,2))
model_fit = model.fit(disp=0)
print(model_fit.summary())
ARIMA Model Results
=========================================================
=====================
Dep. Variable: D.value No. Observations: 99

Model: ARIMA(1, 1, 2) Log Likelihood -253.790
Method: css-mle S.D. of innovations 3.119
Date: Wed, 06 Feb 2019 AIC 517.579
Time: 23:32:56 BIC 530.555
Sample: 1 HQIC 522.829
=========================================================
========================
coef std err z P>|z| [0.025 0.975]
---------------------------------------------------------------------------------
const 1.1202 1.290 0.868 0.387 -1.409 3.649
ar.L1.D.value 0.6351 0.257 2.469 0.015 0.131 1.139
ma.L1.D.value 0.5287 0.355 1.489 0.140 -0.167 1.224
ma.L2.D.value -0.0010 0.321 -0.003 0.998 -0.631 0.629
Roots
=========================================================
====================
Real Imaginary Modulus Frequency
-----------------------------------------------------------------------------
AR.1 1.5746 +0.0000j 1.5746 0.0000
MA.1 -1.8850 +0.0000j 1.8850 0.5000
MA.2 545.3515 +0.0000j 545.3515 0.0000

-----------------------------------------------------------------------------
The model summary reveals a lot of information. The table in the middle is the
coefficients table where the values under ‘coef’ are the weights of the
respective terms.
Notice here the coefficient of the MA2 term is close to zero and the P-Value in
‘P>|z|’ column is highly insignificant. It should ideally be less than 0.05 for the
respective X to be significant.
So, let’s rebuild the model without the MA2 term.

# 1,1,1 ARIMA Model
model = ARIMA(df.value, order=(1,1,1))
model_fit = model.fit(disp=0)
print(model_fit.summary())
ARIMA Model Results
=========================================================
=====================
Dep. Variable: D.value No. Observations: 99
Date: Sat, 09 Feb 2019 AIC 515.579
Time: 12:16:06 BIC 525.960
=========================================================
========================
coef std err z P>|z| [0.025 0.975]
---------------------------------------------------------------------------------
const 1.1205 1.286 0.871 0.386 -1.400 3.641
ar.L1.D.value 0.6344 0.087 7.317 0.000 0.464 0.804
ma.L1.D.value 0.5297 0.089 5.932 0.000 0.355 0.705
Roots
=========================================================
====================
-----------------------------------------------------------------------------
AR.1 1.5764 +0.0000j 1.5764 0.0000
MA.1 -1.8879 +0.0000j 1.8879 0.5000
-----------------------------------------------------------------------------
The model AIC has reduced, which is good. The P Values of the AR1 and MA1
terms have improved and are highly significant (<< 0.05).
Let’s plot the residuals to ensure there are no patterns (that is, look for constant
mean and variance).
# Plot residual errors
residuals = pd.DataFrame(model_fit.resid)
fig, ax = plt.subplots(1,2)
residuals.plot(title="Residuals", ax=ax[0])
residuals.plot(kind='kde', title='Density', ax=ax[1])

plt.show()
Residuals Density
The residual errors seem fine with near zero mean and uniform variance. Let’s
plot the actuals against the fitted values using plot_predict().
# Actual vs Fitted
model_fit.plot_predict(dynamic=False)
plt.show()
Actual vs Fitted
When you set dynamic=False the in-sample lagged values are used for
prediction.
That is, the model gets trained up until the previous value to make the next
prediction. This can make the fitted forecast and actuals look artificially good.
So, we seem to have a decent ARIMA model. But is that the best?
Can’t say that at this point because we haven’t actually forecasted into the
future and compared the forecast with the actual performance.
So, the real validation you need now is the Out-of-Time cross-validation.
10. How to do find the optimal ARIMA model manually using Out-of-Time Cross
validation
In Out-of-Time cross-validation, you take few steps back in time and forecast
into the future to as many steps you took back. Then you compare the forecast
against the actuals.
To do out-of-time cross-validation, you need to create the training and testing

dataset by splitting the time series into 2 contiguous parts in approximately
75:25 ratio or a reasonable proportion based on time frequency of series.
Why am I not sampling the training data randomly you ask?
That’s because the order sequence of the time series should be intact in order
to use it for forecasting.
Forecast vs Actuals
From the chart, the ARIMA(1,1,1) model seems to give a directionally correct
forecast. And the actual observed values lie within the 95% confidence band.
That seems fine.
But each of the predicted forecasts is consistently below the actuals. That
means, by adding a small constant to our forecast, the accuracy will certainly
improve. So, there is definitely scope for improvement.
So, what I am going to do is to increase the order of differencing to two, that is

set d=2 and iteratively increase p to up to 5 and then q up to 5 to see which
model gives least AIC and also look for a chart that gives closer actuals and
forecasts.
While doing this, I keep an eye on the P values of the AR and MA terms in the
model summary. They should be as close to zero, ideally, less than 0.05.
# Build Model
model = ARIMA(train, order=(3, 2, 1))
fitted = model.fit(disp=-1)
print(fitted.summary())
# Forecast
fc, se, conf = fitted.forecast(15, alpha=0.05) # 95% conf
# Make as pandas series
fc_series = pd.Series(fc, index=test.index)
lower_series = pd.Series(conf[:, 0], index=test.index)
upper_series = pd.Series(conf[:, 1], index=test.index)
# Plot
plt.figure(figsize=(12,5), dpi=100)
plt.plot(train, label='training')
plt.plot(test, label='actual')
plt.plot(fc_series, label='forecast')
plt.fill_between(lower_series.index, lower_series, upper_series,
color='k', alpha=.15)
plt.title('Forecast vs Actuals')
plt.legend(loc='upper left', fontsize=8)
plt.show()
ARIMA Model Results
=========================================================
=====================
Dep. Variable: D2.value No. Observations: 83
Date: Sat, 09 Feb 2019 AIC 440.497
Time: 12:49:01 BIC 455.010
=========================================================
=========================
coef std err z P>|z| [0.025 0.975]
----------------------------------------------------------------------------------
const 0.0483 0.084 0.577 0.565 -0.116 0.212
ar.L1.D2.value 1.1386 0.109 10.399 0.000 0.924 1.353
ar.L2.D2.value -0.5923 0.155 -3.827 0.000 -0.896 -0.289
ar.L3.D2.value 0.3079 0.111 2.778 0.007 0.091 0.525
ma.L1.D2.value -1.0000 0.035 -28.799 0.000 -1.068 -0.932
Roots
=========================================================
====================
-----------------------------------------------------------------------------
AR.1 1.1557 -0.0000j 1.1557 -0.0000
AR.2 0.3839 -1.6318j 1.6763 -0.2132
AR.3 0.3839 +1.6318j 1.6763 0.2132
MA.1 1.0000 +0.0000j 1.0000 0.0000
-----------------------------------------------------------------------------
Revised Forecast vs Actuals
The AIC has reduced to 440 from 515. Good. The P-values of the X terms are
less the < 0.05, which is great.
So overall it’s much better.
Ideally, you should go back multiple points in time, like, go back 1, 2, 3 and 4
quarters and see how your forecasts are performing at various points in the year.
Here’s a great practice exercise: Try to go back 27, 30, 33, 36 data points and
see how the forcasts performs. The forecast performance can be judged using
various accuracy metrics discussed next.
Accuracy Metrics for Time Series Forecast

The commonly used accuracy metrics to judge forecasts are:
1. Mean Absolute Percentage Error (MAPE)

2. Mean Error (ME)
3. Mean Absolute Error (MAE)
4. Mean Percentage Error (MPE)
5. Root Mean Squared Error (RMSE)
6. Lag 1 Autocorrelation of Error (ACF1)
7. Correlation between the Actual and the Forecast (corr)
8. Min-Max Error (minmax)
Typically, if you are comparing forecasts of two different series, the MAPE,
Correlation and Min-Max Error can be used.
Why not use the other metrics?
Because only the above three are percentage errors that vary between 0 and 1.
That way, you can judge how good is the forecast irrespective of the scale of
the series.
The other error metrics are quantities. That implies, an RMSE of 100 for a
series whose mean is in 1000’s is better than an RMSE of 5 for series in 10’s.
So, you can’t really use them to compare the forecasts of two different scaled
time series.
# Accuracy metrics
def forecast_accuracy(forecast, actual):
mape = np.mean(np.abs(forecast - actual)/np.abs(actual)) # MAPE
me = np.mean(forecast - actual) # ME
mae = np.mean(np.abs(forecast - actual)) # MAE
mpe = np.mean((forecast - actual)/actual) # MPE
rmse = np.mean((forecast - actual)**2)**.5 # RMSE
corr = np.corrcoef(forecast, actual)[0,1] # corr
mins = np.amin(np.hstack([forecast[:,None],
actual[:,None]]), axis=1)
maxs = np.amax(np.hstack([forecast[:,None],
actual[:,None]]), axis=1)
minmax = 1 - np.mean(mins/maxs) # minmax
acf1 = acf(fc-test)[1] # ACF1
return({'mape':mape, 'me':me, 'mae': mae,

'mpe': mpe, 'rmse':rmse, 'acf1':acf1,
'corr':corr, 'minmax':minmax})
forecast_accuracy(fc, test.values)
#> {'mape': 0.02250131357314834,
#> 'me': 3.230783108990054,
#> 'mae': 4.548322194530069,
#> 'mpe': 0.016421001932706705,
#> 'rmse': 6.373238534601827,
#> 'acf1': 0.5105506325288692,
#> 'corr': 0.9674576513924394,
#> 'minmax': 0.02163154777672227}
Around 2.2% MAPE implies the model is about 97.8% accurate in predicting
the next 15 observations.
Now you know how to build an ARIMA model manually.
But in industrial situations, you will be given a lot of time series to be

forecasted and the forecasting exercise be repeated regularly.
So we need a way to automate the best model selection process.

How to do Auto Arima Forecast in Python
Like R’s popular auto.arima() function, the pmdarima package
provides auto_arima() with similar functionality.
auto_arima() uses a stepwise approach to search multiple combinations of p,d,q
parameters and chooses the best model that has the least AIC.
from statsmodels.tsa.arima_model import ARIMA
import pmdarima as pm
df =
pd.read_csv('https://raw.githubusercontent.com/selva86/datasets/master/wwwusa
ge.csv', names=['value'], header=0)
model = pm.auto_arima(df.value, start_p=1, start_q=1,
test='adf', # use adftest to find optimal 'd'
max_p=3, max_q=3, # maximum p and q
m=1, # frequency of series
d=None, # let model determine 'd'
seasonal=False, # No Seasonality
start_P=0,
D=0,
trace=True,
error_action='ignore',
suppress_warnings=True,
stepwise=True)
print(model.summary())
#> Fit ARIMA: order=(1, 2, 1); AIC=525.586, BIC=535.926, Fit time=0.060

seconds

seconds

seconds

seconds

seconds

seconds

seconds

seconds

seconds
#> Total fit time: 0.865 seconds
#> ARIMA Model Results
#>
=========================================================
=====================
#> Dep. Variable: D2.y No. Observations: 98

#> Model: ARIMA(3, 2, 1) Log Likelihood -250.276
#> Method: css-mle S.D. of innovations 3.069
#> Date: Sat, 09 Feb 2019 AIC 512.552
#> Time: 12:57:22 BIC 528.062
#> Sample: 2 HQIC 518.825
#>
#>
=========================================================
=====================
#> coef std err z P>|z| [0.025 0.975]
#> ------------------------------------------------------------------------------
#> const 0.0234 0.058 0.404 0.687 -0.090 0.137
#> ar.L1.D2.y 1.1586 0.097 11.965 0.000 0.969 1.348
#> ar.L2.D2.y -0.6640 0.136 -4.890 0.000 -0.930 -0.398
#> ar.L3.D2.y 0.3453 0.096 3.588 0.001 0.157 0.534
#> ma.L1.D2.y -1.0000 0.028 -36.302 0.000 -1.054 -0.946
#> Roots
#>
=========================================================
====================
#> Real Imaginary Modulus Frequency
#> -----------------------------------------------------------------------------
#> AR.1 1.1703 -0.0000j 1.1703 -0.0000

#> AR.2 0.3763 -1.5274j 1.5731 -0.2116
#> AR.3 0.3763 +1.5274j 1.5731 0.2116
#> MA.1 1.0000 +0.0000j 1.0000 0.0000
#> -----------------------------------------------------------------------------
How to interpret the residual plots in ARIMA model

Let’s review the residual plots using stepwise_fit.
model.plot_diagnostics(figsize=(7,5))
plt.show()
Residuals Chart
So how to interpret the plot diagnostics?
Top left: The residual errors seem to fluctuate around a mean of zero and have
a uniform variance.
Top Right: The density plot suggest normal distribution with mean zero.
Bottom left: All the dots should fall perfectly in line with the red line. Any
significant deviations would imply the distribution is skewed.
Bottom Right: The Correlogram, aka, ACF plot shows the residual errors are
not autocorrelated. Any autocorrelation would imply that there is some pattern
in the residual errors which are not explained in the model. So you will need to
look for more X’s (predictors) to the model.
Overall, it seems to be a good fit. Let’s forecast.
# Forecast
n_periods = 24
fc, confint = model.predict(n_periods=n_periods, return_conf_int=True)
index_of_fc = np.arange(len(df.value), len(df.value)+n_periods)
# make series for plotting purpose
fc_series = pd.Series(fc, index=index_of_fc)
lower_series = pd.Series(confint[:, 0], index=index_of_fc)
upper_series = pd.Series(confint[:, 1], index=index_of_fc)
# Plot
plt.plot(df.value)
plt.plot(fc_series, color='darkgreen')
plt.fill_between(lower_series.index,
lower_series,
upper_series,
plt.title("Final Forecast of WWW Usage")
plt.show()
How to automatically build SARIMA model in python

The problem with plain ARIMA model is it does not support seasonality.
If your time series has defined seasonality, then, go for SARIMA which uses
seasonal differencing.
Seasonal differencing is similar to regular differencing, but, instead of

subtracting consecutive terms, you subtract the value from previous season.
So, the model will be represented as SARIMA(p,d,q)x(P,D,Q), where, P, D and

Q are SAR, order of seasonal differencing and SMA terms respectively
and 'x' is the frequency of the time series.
If your model has well defined seasonal patterns, then enforce D=1 for a given
frequency ‘x’.
Here’s some practical advice on building SARIMA model:
As a general rule, set the model parameters such that D never exceeds one. And
the total differencing ‘d + D’ never exceeds 2. Try to keep only either SAR or
SMA terms if your model has seasonal components.
Let’s build an SARIMA model on 'a10' – the drug sales dataset.

# Import
data =
pd.read_csv('https://raw.githubusercontent.com/selva86/datasets/master/a10.csv',
parse_dates=['date'], index_col='date')
# Plot
fig, axes = plt.subplots(2, 1, figsize=(10,5), dpi=100, sharex=True)
# Usual Differencing
axes[0].plot(data[:], label='Original Series')
axes[0].plot(data[:].diff(1), label='Usual Differencing')
axes[0].set_title('Usual Differencing')
axes[0].legend(loc='upper left', fontsize=10)
# Seasinal Dei
axes[1].plot(data[:], label='Original Series')
axes[1].plot(data[:].diff(12), label='Seasonal Differencing', color='green')
axes[1].set_title('Seasonal Differencing')
plt.legend(loc='upper left', fontsize=10)
plt.suptitle('a10 - Drug Sales', fontsize=16)
plt.show()
Seasonal Differencing
As you can clearly see, the seasonal spikes is intact after applying usual
differencing (lag 1). Whereas, it is rectified after seasonal differencing.
Let’s build the SARIMA model using pmdarima‘s auto_arima(). To do that,

you need to set seasonal=True, set the frequency m=12 for month wise series
and enforce D=1.
# !pip3 install pyramid-arima
# Seasonal - fit stepwise auto-ARIMA
smodel = pm.auto_arima(data, start_p=1, start_q=1,
test='adf',
max_p=3, max_q=3, m=12,
start_P=0, seasonal=True,
d=None, D=1, trace=True,
stepwise=True)
smodel.summary()
Fit ARIMA: order=(1, 0, 1) seasonal_order=(0, 1, 1, 12); AIC=534.818,

BIC=551.105, Fit time=1.742 seconds




(...TRUNCATED...)



Fit ARIMA: order=(3, 0, 0) seasonal_order=(1, 1, 2, 12); AIC=nan, BIC=nan, Fit

time=nan seconds
Total fit time: 31.613 seconds

The model has estimated the AIC and the P values of the coefficients look
significant. Let’s look at the residual diagnostics plot.
The best model SARIMAX(3, 0, 0)x(0, 1, 1, 12) has an AIC of 528.6 and the P
Values are significant.
Let’s forecast for the next 24 months.
# Forecast
n_periods = 24
fitted, confint = smodel.predict(n_periods=n_periods, return_conf_int=True)
index_of_fc = pd.date_range(data.index[-1], periods = n_periods, freq='MS')
fitted_series = pd.Series(fitted, index=index_of_fc)

# Plot
plt.plot(data)
plt.plot(fitted_series, color='darkgreen')
lower_series,
upper_series,
plt.title("SARIMA - Final Forecast of a10 - Drug Sales")
plt.show()
There you have a nice forecast that captures the expected seasonal demand
pattern.
How to build SARIMAX Model with exogenous variable
The SARIMA model we built is good. I would stop here typically.
But for the sake of completeness, let’s try and force an external predictor, also
called, ‘exogenous variable’ into the model. This model is called the
SARIMAX model.
The only requirement to use an exogenous variable is you need to know the
value of the variable during the forecast period as well.
For the sake of demonstration, I am going to use the seasonal index from
the classical seasonal decomposition on the latest 36 months of data.
Why the seasonal index? Isn’t SARIMA already modeling the seasonality, you
ask?
You are correct.
But also, I want to see how the model looks if we force the recent seasonality
pattern into the training and forecast.
Secondly, this is a good variable for demo purpose. So you can use this as a
template and plug in any of your variables into the code. The seasonal index is
a good exogenous variable because it repeats every frequency cycle, 12 months
in this case.
So, you will always know what values the seasonal index will hold for the
future forecasts.
# Import Data
data =
pd.read_csv('https://raw.githubusercontent.com/selva86/datasets/master/a10.csv',
parse_dates=['date'], index_col='date')
Let’s compute the seasonal index so that it can be forced as a (exogenous)
predictor to the SARIMAX model.
# Compute Seasonal Index
from statsmodels.tsa.seasonal import seasonal_decompose
from dateutil.parser import parse
# multiplicative seasonal component
result_mul = seasonal_decompose(data['value'][-36:], # 3 years
model='multiplicative',
extrapolate_trend='freq')
seasonal_index = result_mul.seasonal[-12:].to_frame()
seasonal_index['month'] = pd.to_datetime(seasonal_index.index).month
# merge with the base data
data['month'] = data.index.month
df = pd.merge(data, seasonal_index, how='left', on='month')
df.columns = ['value', 'month', 'seasonal_index']
df.index = data.index # reassign the index.
The exogenous variable (seasonal index) is ready. Let’s build the SARIMAX
model.
# SARIMAX Model
sxmodel = pm.auto_arima(df[['value']], exogenous=df[['seasonal_index']],
start_p=1, start_q=1,
test='adf',
max_p=3, max_q=3, m=12,
start_P=0, seasonal=True,
d=None, D=1, trace=True,
stepwise=True)
sxmodel.summary()





(...Truncated...)


Fit ARIMA: order=(3, 0, 0) seasonal_order=(1, 1, 2, 12); AIC=nan, BIC=nan, Fit

time=nan seconds
Total fit time: 30.781 seconds
So, we have the model with the exogenous term. But the coefficient is very
small for x1, so the contribution from that variable will be negligible. Let’s
forecast it anyway.
We have effectively forced the latest seasonal effect of the latest 3 years into
the model instead of the entire history.
Alright let’s forecast into the next 24 months. For this, you need the value of
the seasonal index for the next 24 months.
# Forecast
n_periods = 24
fitted, confint = sxmodel.predict(n_periods=n_periods,
exogenous=np.tile(seasonal_index.value, 2).reshape(-1,1),
return_conf_int=True)
index_of_fc = pd.date_range(data.index[-1], periods = n_periods, freq='MS')
fitted_series = pd.Series(fitted, index=index_of_fc)
# Plot
plt.plot(data['value'])
plt.plot(fitted_series, color='darkgreen')
lower_series,
upper_series,
plt.title("SARIMAX Forecast of a10 - Drug Sales")
plt.show()

Time Series Forecasting

Uploaded by

Copyright:

Available Formats

You might also like

Time Series Forecasting

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Time Series Forecasting

Uploaded by

Copyright:

Available Formats

Time-series Forecasting

There are two types of time series models:

1. Univariate time series: Time series with a one time-dependent variable

Types of forecasting methods

Decompositional Deconstruction of time series

Smooth-based Removal of anomalies for clear patterns

Moving-Average Tracking a single type of data

Here’s the MA(2) calculation:

Exponential Moving Average

The EMA at time t is calculated as the current price multiplied by a smoothing

The smoothing factor 𝛼 ( alpha ) is defined as:

Let's test it:

Exponential Smoothing Method

Another important advantage of exponential smoothing method over simple

1. Single Exponential Smoothing

Let’s cover each one of them in detail.

Single Exponential Smoothing :

Here the forecast at time t is given as Ft = α * y(t-1) + (1-α)*F(t-1)

Double Exponential Smoothing :

l(t) = α * y(t) + (1-α)*(l(t-1)+b(t-1)) — — — — — Level l

Single Exponential Smoothing method assumes the time series to be relatively

Triple Exponential Smoothing / Holt Winter’s Method :

The four equations of the additive Holt Winter’s Method is given as :

A regression model is a statistical technique to estimate the relationship

Autoregression Models of Order p

Autocorrelation & Partial Autocorrelation

As the name suggests, autocorrelation means correlation with itself.

• -1 means a perfect negative autocorrelation,

Let us understand what is partial autocorrelation.

Partial autocorrelation is the conditional correlation between the variable and

How’s that possible? Simple, check it out:

Where X and Y can be the returns of asset X and Y, respectively.

Now, the autocovariance function can be defined as:

γs: Autocovariance at lag “s”.γs: Autocovariance at lag “s”.

In simple terms, autocorrelation is the same as the correlation function! To be

Cov(X,Y: Covariance between X and Y.Cov(X,Y: Covariance

ρs: Autocorrelation at lag “s”.ρs: Autocorrelation at lag “s”.

Calculation of the autocovariance with an example

Confused? Don’t worry! We got you covered!

Imagine we have the following returns for Microsoft prices:

5% 1% -2% 3% -4% 6% 2% -1% -3% 4%

Do you get to see the difference between the 2 variables?

Consequently, our data is as follows:

1% -2% 3% -4% 6% 2% -1% -3% 4%

You can do this, right?

Remember the autocovariance formula:

Calculation of the autocorrelation with an example

What Is partial autocorrelation?

The partial autocorrelation is a function that allows having the autocorrelation

Autocorrelation Function (ACF) Plot & Partial Autocorrelation Function

Just like an autocorrelation function plot, a partial autocorrelation function plot

Limitations of Autoregression Models

Introduction to ARIMA Models

An ARIMA model is characterized by 3 terms: p, d, q where,

So how to make a series stationary?

The value of d, therefore, is the minimum number of differencing needed to

[Error: The beta coefficients in the second equation above is incorrect. ]