Application of Markov-Chain in

Forecasting Crude Oil Price

Contents
1. Introduction .................................................................................................................... 1

2. Motivation ....................................................................................................................... 1

3. Method............................................................................................................................. 2

4. Analysis and Results ...................................................................................................... 4

4-1. Two-Step Transition Matrix ........................................................................................ 4

4-2. Long-Run Probabilities ................................................................................................ 4

4-3. Recurrence Time ........................................................................................................... 5

4-4. First Passage .................................................................................................................. 5

4-5. Expected value of crude oil price................................................................................. 5

5. Conclusion ....................................................................................................................... 6
 1. Introduction

In this project, we will study the application of Discrete Markov Chain in forecasting crude
oil price. In particular, the main focus of this project is considering the fluctuation of crude oil
price in order to design a stochastic model to predict the price.
There are a number of methods that can be used to predict the price. One of these methods is
Discrete Markov Chain. We believe that it can help to build a good model to forecast. Our
method uses price information through the history to predict crude oil price.
140

120

100
Price in Dollar ($)

80

60

40

20

0
1861
1866
1871
1876
1881
1886
1891
1896
1901
1906
1911
1916
1921
1926
1931
1936
1941
1946
1951
1956
1961
1966
1971
1976
1981
1986
1991
1996
2001
2006
2011
Years

Figure 1. Crude oil price from 1861-2013

The fluctuation of crude oil price from 1861 to 2013 has been shown in figure 1.

First, we describe why this problem is important for us, then we will describe and illustrate
our methodology with case study.

2. Motivation

Nowadays, energy supply and demand, and consequently energy price are unpredictable.
Therefore, predicting energy price is one of the most important issues in the world, and
projecting price of all kinds of energy source is a very critical process. The focus of this project
is on forecasting crude oil price. In order to describe its importance, this is enough to know that
the cost of imported oil to US which is around one-third of its use; is $192 billion annually (US
Department of Energy, n.d.). Falling or rising in oil prices means decreasing or increasing in

1
people income, it will affect their culture, welfare, country’s GNP, and so on. So, accurate
forecasting of crude oil price is very significant.

3. Method

We assumed that crude oil price is a kind of discrete time stochastic process, and we will use
Discrete Time Markov Chain1 to build a model. One of the key properties of markov chain is
being memory less. In other words, the next state is dependent only on the current state.

𝑃(𝑋 = 𝑗|𝑋 = 𝑖, 𝑋 =𝑖 , … , 𝑋 = 𝑖 ) = 𝑃(𝑋 = 𝑗|𝑋 = 𝑖)

A transition matrix is used to describe the transitions of a DTMC. Historical crude oil price
data will help to make the probabilities of the transition matrix. P ij matrix is as follows:

Table 1. Transition Matrix

To 0 1 2 3 4 5 6 7 8 9 10 11
From

0-10 10-20 20-30 30-40 40-50 50-60 60-70 70-80 80-90 90-100 100-110 110-120

1 10-20 0.01515 0.8484 0.121212 0 0 0.015152 0 0 0 0 0 0

2 20-30 0 0.2352 0.5 0.205882 0 0.058824 0 0 0 0 0 0

3 30-40 0 0 0.470588 0.411765 0.117647 0 0 0 0 0 0 0

4 40-50 0 0 0 0.25 0 0.25 0.5 0 0 0 0 0

5 50-60 0 0 0.111111 0.111111 0.222222 0.333333 0 0 0 0 0.111111 0.111111

6 60-70 0 0 0 0 0 0.166667 0.333333 0.333333 0.166667 0 0 0

7 70-80 0 0 0 0.25 0 0 0.25 0 0.5 0 0 0

8 80-90 0 0 0 0 0 0 0 0.333333 0 0 0.333333 0.333333

9 90-100 0 0 0 0 0 0.5 0 0.5 0 0 0 0

10 100-110 0 0 0 0 0 0 0.333333 0 0 0.333333 0.333333 0

11 110-120 0 0 0 0 0 0 0 0 0 0.333333 0.333333 0.333333

Crude Oil Price data from 1861 to 2013 is driven from Chartsbin2 website. We defined the
states as 12 price ranges which started from $0 to $120, and each state has an equal range of $10.
We counted the number of events that we go range i to j in historical data, and divided this
number to the total number of events that we leave range i. as we assumed that collected data
from 1861 to 2013 is enough to predict the future data, this number give us the probability of

1
Discrete Time Markov Chain (DTMC)
2
http://chartsbin.com/view/oau

2
going from state i to j, Pij. For instance, p23 = means that we left state 2 to 3, 7 times, and 34

times we left state 2.

P23 shows the probability of going from state 2 to state 3. In particular, if the price is
between $20 and $30 in a year, the probability that the price will be between 30 to 40 dollars in
the next year, is 0.205882.

Each price range as a state and the probabilities of leaving state 1 in the next step has been
shown in figure 1.

0 4
$0-$10 $40-$50
0.015

0.015
0.848

1
$10-$20

0.121

2 3
$20-$30 $30-$40

Figure 2. States of Markov Chain

3
 4. Analysis and Results

We believe that this Markov chain is ergodic. We calculated P 256, and saw that all rows are
identical to five decimal digits. Therefore, it is assumed that it remains the same.

In this section, we are going to discuss some applications of this study.

4-1. Two-Step Transition Matrix

The following table shows 𝑃 . As an example, 𝑃 is equal to 0.763599 means that the
probability that we go from sate 10-20 dollars to state 10-20 dollars in two years.

Table 2. Two-step transition matrix

To 0 1 2 3 4 5 6 7 8 9 10 11
From

0-10 10-20 20-30 30-40 40-50 50-60 60-70 70-80 80-90 90-100 100-110 110-120
0
0-10 0.015152 0.848485 0.121212 0 0 0.015152 0 0 0 0 0 0
1
10-20 0.012856 0.763599 0.165136 0.026639 0.003367 0.025036 0 0 0 0 0.001684 0.001684
2
20-30 0.003565 0.317291 0.381942 0.194252 0.037293 0.052585 0 0 0 0 0.006536 0.006536
3
30-40 0 0.110727 0.429066 0.295848 0.048443 0.057093 0.058824 0 0 0 0 0
4
40-50 0 0 0.145425 0.130719 0.084967 0.166667 0.166667 0.166667 0.083333 0 0.027778 0.027778
5
50-60 0 0.026144 0.14488 0.16122 0.087146 0.173203 0.148148 0 0 0.074074 0.111111 0.074074
6
60-70 0 0 0.018519 0.101852 0.037037 0.111111 0.194444 0.166667 0.222222 0 0.074074 0.074074
7
70-80 0 0 0.117647 0.102941 0.029412 0.041667 0.083333 0.25 0.041667 0 0.166667 0.166667
8
80-90 0 0 0 0.083333 0 0 0.194444 0 0.166667 0.222222 0.222222 0.111111
9
90-100 0 0 0.055556 0.180556 0.111111 0.166667 0.125 0 0.25 0 0.055556 0.055556
10
100-110 0 0 0 0 0 0.222222 0.222222 0.277778 0.055556 0.111111 0.111111 0
11
110-120 0 0 0 0 0 0.166667 0.111111 0.166667 0 0.222222 0.222222 0.111111

4-2. Long-Run Probabilities

This is the probability of event that we will visit state i after infinite number of steps. In order
to calculate these values, we used 𝜋 = 𝜋𝑝 and ∑ 𝜋 = 1. The following table shows the results.
As it can be seen, 𝜋 = 0.037929 means the probability of visiting state 11 in long run
iterations.

Table 3. Long run Probabilities

State i 1 2 3 4 5 6
𝜋 0.005547 0.366097 0.21217 0.115269 0.028492 0.067188
State i 7 8 9 10 11 12
𝜋 0.054636 0.03814 0.028176 0.021072 0.037929 0.025286

4
 4-3. Recurrence Time
Now, we are going to calculate the expected number of steps that is required to return to one
specific state. For instance, we are in state 2 (20-30 dollars), and we want to know the expected
number of steps to return to state 2.

1 1
𝜇 = = = 2.731516
𝜋 . 366097

It means that the expected number of steps for returning to state 2 is 2.731516.

4-4. First Passage

( )
The probability of first time going from sate i to j after n steps is 𝑓 . For instance, we
calculated

( ) ( ) ( ) ( )
𝑓, = 𝑝 , −𝑓, 𝑝 , = 0.074074 − (0.111111 ∗ 0.333333) = 0.037037

Which means that probability of visiting sate 11 from state 5 for the first time after 2 steps is
equal to 0.037037.

4-5. Expected value of crude oil price

In this part, we are going to calculate the expected value of crude oil price in next year when
we have the adjusted crude oil price in the current year. Using the total expected law, the
expected value of crude oil price in the following year have calculated in table 4.

𝐸(𝑋 ) = 𝑃(𝐵 ) ∗ 𝐸(𝑋|𝐵 )

For instance, if the crude oil price is between $40 to $50, we will expect the crude oil price 55
dollar in the next year.

5
 Table 4. Expected crude oil price in the next year

crude oil price in Expected crude oil price in the

current year next year

0-10 $ 15
10-20 $ 16.66666667
20-30 $ 26.47058824
30-40 $ 31.47058824
40-50 $ 55
50-60 $ 59.44444444
60-70 $ 70
70-80 $ 67.5
80-90 $ 98.33333333
90-100 $ 65
100-110 $ 88.33333333
110-120 $ 105

Figure 3. Expected value of crude oil price

5. Conclusion

In this project, crude oil as one of the most important factors on worldwide economy
predicted using Markov chain. It is illustrated that calculated transition matrix remains the same
in higher power which means that the Markov chain is ergodic.

6
 Two-step transition matrix, long-run probabilities, recurrence time, first passage, and
expected value of crude oil price were discussed in the analysis part. As an example, we will
expect that the crude oil price in 2016 will be $55 as it was $48.67 in 2015.
In the future, other prediction methods can be used, and the results can be compared by the results of
this project.

7
 6. Appendix

Year Money of the Day Value of 2013 Year Money of the Day Value of 2013
1861 0.49 12.65 1937 1.18 19.14
1862 1.05 24.4 1938 1.13 18.67
1863 3.15 59.36 1939 1.02 17.1
1864 8.06 119.56 1940 1.02 16.93
1865 6.59 99.88 1941 1.14 18.03
1866 3.74 59.26 1942 1.19 17
1867 2.41 40.01 1943 1.2 16.16
1868 3.63 63.27 1944 1.21 16.01
1869 3.64 63.45 1945 1.05 13.58
1870 3.86 70.82 1946 1.12 13.35
1871 4.34 84.05 1947 1.9 19.8
1872 3.64 70.5 1948 1.99 19.24
1873 1.83 35.44 1949 1.78 17.39
1874 1.17 23.99 1950 1.71 16.54
1875 1.35 28.52 1951 1.71 15.32
1876 2.56 55.78 1952 1.71 15
1877 2.42 52.73 1953 1.93 16.79
1878 1.19 28.61 1954 1.93 16.71
1879 0.86 21.41 1955 1.93 16.78
1880 0.95 22.84 1956 1.93 16.54
1881 0.86 20.68 1957 1.9 15.71
1882 0.78 18.75 1958 2.08 16.75
1883 1 24.9 1959 2.08 16.61
1884 0.84 21.69 1960 1.9 14.93
1885 0.88 22.72 1961 1.8 14.01
1886 0.71 18.33 1962 1.8 13.85
1887 0.67 17.3 1963 1.8 13.69
1888 0.88 22.72 1964 1.8 13.5
1889 0.94 24.27 1965 1.8 13.28
1890 0.87 22.47 1966 1.8 12.92
1891 0.67 17.3 1967 1.8 12.55
1892 0.56 14.46 1968 1.8 12.05
1893 0.64 16.53 1969 1.8 11.43
1894 0.84 22.53 1970 1.8 10.79
1895 1.36 37.93 1971 2.24 12.87
1896 1.18 32.91 1972 2.48 13.81
1897 0.79 22.03 1973 3.29 17.25
1898 0.91 25.38 1974 11.58 54.74

8
1899 1.29 35.98 1975 11.53 49.93
1900 1.19 33.19 1976 12.8 52.39
1901 0.96 26.77 1977 13.92 53.47
1902 0.8 21.45 1978 14.02 50.09
1903 0.94 24.27 1979 31.61 101.43
1904 0.86 22.21 1980 36.83 104.12
1905 0.62 16.01 1981 35.93 92.08
1906 0.73 18.85 1982 32.97 79.59
1907 0.72 17.93 1983 29.55 69.12
1908 0.72 18.59 1984 28.78 64.53
1909 0.7 18.08 1985 27.56 59.67
1910 0.61 15.19 1986 14.43 30.67
1911 0.61 15.19 1987 18.44 37.8
1912 0.74 17.79 1988 14.92 29.39
1913 0.95 22.3 1989 18.23 34.24
1914 0.81 18.76 1990 23.73 42.29
1915 0.64 14.68 1991 20 34.21
1916 1.1 23.45 1992 19.32 32.08
1917 1.56 28.32 1993 16.97 27.36
1918 1.98 30.6 1994 15.82 24.86
1919 2.01 27.06 1995 17.02 26.01
1920 3.07 35.68 1996 20.67 30.69
1921 1.73 22.51 1997 19.09 27.71
1922 1.61 22.36 1998 12.72 18.17
1923 1.34 18.28 1999 17.97 25.13
1924 1.43 19.47 2000 28.5 38.55
1925 1.68 22.31 2001 24.44 32.15
1926 1.88 24.73 2002 25.02 32.4
1927 1.3 17.43 2003 28.83 36.5
1928 1.17 15.9 2004 38.27 47.19
1929 1.27 17.26 2005 54.52 65.03
1930 1.19 16.59 2006 65.14 75.28
1931 0.65 9.94 2007 72.39 81.33
1932 0.87 14.83 2008 97.26 105.23
1933 0.67 12.04 2009 61.67 66.97
1934 1 17.39 2010 79.5 84.93
1935 0.97 16.45 2011 111.26 115.22
1936 1.09 18.32 2012 111.67 113.31
2013 108.66 108.66