Resources Policy 74 (2021) 102244

Contents lists available at ScienceDirect

Resources Policy
journal homepage: www.elsevier.com/locate/resourpol

Forecasting crude oil real prices with averaging time-varying VAR models
Krzysztof Drachal
Faculty of Economic Sciences, University of Warsaw, Poland

A R T I C L E I N F O A B S T R A C T

Keywords: The aim of this research is to discuss the ability to forecast real crude oil price by the use of Time-Varying Vector
Bayesian models Autoregression (TVP-VAR) models. In particular, model averaging and model selection schemes over several
Dynamic model averaging TVP-VAR models are performed. These methods address the problem of variable uncertainty. Indeed, several
Oil prices
previous studies indicate that explanatory variables for crude oil prices can be different in different periods of
Time-varying parameters
Vector AutoRegression
time. Further, the strength of the relationship between crude oil price and its determinants can vary in time. The
applied model combination schemes are an extension of Dynamic Model Averaging, which has already been
found viable. Moreover, geopolitical risk is included in each model as an endogenous variable, and the model
combination scheme is constructed in a way to tackle the joint forecasting ability (with respect to crude oil real
price and geopolitical risk). It is found that, indeed, the Vector Autoregression approach results in more accurate
forecasts than single equation a Time-Varying Regression or the standard Dynamic Model Averaging. Also, the
model combination scheme of several Vector Autoregression models outperforms a single Vector Autoregression
model approach. However, forecast accuracy is tested with some novel tools, such as Giacomini and Rossi
fluctuation test and Murphy diagrams, which are able to capture time-varying predictive ability significances and
several scoring functions.

1. Introduction and Anderson, 2002).

This paper is focused on forecasting by one-month ahead the crude
oil real prices within a newly proposed Bayesian-based model combi­
nation scheme (Koop and Korobilis, 2013; Raftery et al., 2010). The
applied modelling scheme combines several features that are desirable
for a crude oil price forecasting model. Indeed, several problems emerge
in modelling oil prices. In particular, the applied scheme includes model
and variable uncertainty. In other words, instead of performing some
harsh preselection of variables, before the model construction, it is the
modelling scheme itself, which “estimates” the usefulness of a given
variable in improving forecast accuracy.
Indeed, various researches show that the set of important oil price
drivers vary in time. As a result, different models perform in different
ways at different times and affect forecast accuracy (Aastveit and
Bjornland, 2015; Stefanski, 2014). In other words, the suitable model
itself can vary in time. Moreover, it is well documented that there is no
“one” or “best” oil price model which has superior forecast ability (de
Albuquerquemello et al., 2018; Yang et al., 2002). One of the usual
solutions to this problem is to consider model averaging scheme. In
other words, the forecast is produced as a weighted average from several
components, which come from numerous individual models (Burnhan
Secondly, the relationship between crude oil price and its drivers can
be time dependent, which from the econometrical point of view requires
the use of time-varying coefficients. This corresponds to the situation
that the impact of a given variable on crude oil price can be different in
different times. Indeed, the crude oil market can be highly volatile,
which additionally supports the use of time-varying coefficients models
(Wang et al., 2017).
The third important feature of the applied modelling scheme is that
the estimations are performed in a dynamic, on-line way. Instead of
calibrating a model on the fixed, in-sample testing period and using the
obtained parameters to forecast the future, it seems better to continu­
ously expand the in-sample period as new data can be obtained from the
markets. More technically, forecast for time t should be based on the
data available in time t-1, and forecast for time t+1 should be based on
the updated data available in time t, which requires re-estimation of
model’s coefficients with any new data that becomes available (Jacquier
and Polson, 2011). This corresponds to the real-life situation of policy­
makers, investors, market players, etc.
These problems usually result in using Bayesian-based methodology
in econometric forecasting. This is because conventional methods (like,
for example, Ordinary Least Squares) cannot be applied if time-series are

E-mail address: kdrachal@wne.uw.edu.pl.

https://doi.org/10.1016/j.resourpol.2021.102244
Received 21 September 2020; Received in revised form 11 July 2021; Accepted 12 July 2021
Available online 20 July 2021
0301-4207/© 2021 Elsevier Ltd. All rights reserved.
K. Drachal
relatively short compared to the number of explanatory variables in the
model. In case of model uncertainty, one of the conventional approaches
is to estimate several models and then, based on some criterion, select
the most preferable model to formulate a final forecast. Another
approach – more preferable – is to ascribe weights to each of the
different models and construct a weighted average forecast from them.
Indeed, model combination schemes have been found to be very bene­
ficial in forecasting (Zhang et al., 2019; Baumeister and Kilian, 2015;
Burnhan and Anderson, 2002).
One such approach, Dynamic Model Averaging (DMA), was pro­
posed by Raftery et al. (2010). This scheme leads to better forecast ac­
curacy, for instance (Naser, 2016), in crude oil prices. However, for the
oil market, this scheme has not yet been applied jointly with Vector
Autoregression models (VAR). On the other hand large VAR models
seem to outperform small VAR models (Koop, 2013a; Banbura et al.,
2010).
Therefore, this paper expands DMA methodology use into the VAR
modelling approach. Indeed, Bayesian and conventional VAR models
have already yielded interesting oil price forecasting (Roubard and
Arouri, 2018; Gupta and Wohar, 2017), and were found to be beneficial
in the context of analysing bidirectional relationships between oil prices
and policy uncertainty (Bekiros et al., 2015). As a result, this paper
implements a forecasting scheme that includes policy uncertainty
impact in forecast accuracy of crude oil price in a specific way. All in all,
it seems natural, then, to discern if mixing a VAR approach with an
already effective model combination scheme with desirable properties
can lead to more accurate forecasts.
In particular, this paper deals with applying DMA forecast combi­
nation scheme to VAR-type modelling. Such an econometric method is
used to forecast one-month ahead crude oil prices. Besides, the stress is
put on the relationship with policy uncertainty, as emphasized by, for
instance, Bekiros et al. (2015). The applied model combination scheme
joins all the important modelling features that were already mentioned
in the previous paragraphs. Together with the features mentioned
below, this paper fills certain literature gap and presents some novel
econometric tool applied to forecasting crude oil prices.
Additional problems addressed in this paper are connected with
detecting important oil price drivers in time-varying contexts. Indeed,
the proposed modelling scheme leads to certain quantitative measures
with this context. In diagnosing forecast accuracies recently proposed,
new methods are applied. First, the Giacomini and Rossi fluctuation test
is applied (Giacomini and Rossi, 2010). The commonly used
Diebold-Mariano test leads to conclusions based on the whole
out-of-sample period. However, the superiority of certain forecasts,
compared to the benchmarks, can be concluded only from behaviour
within particular time periods. The fluctuation test verifies whether the
potential “domination” of one forecast is present all the time.
Secondly, the quantitative measuring of forecast accuracy requires
fixing scoring functions (leading to, for example, Root Mean Squared
Error – RMSE, Mean Absolute Error – MAE, etc.). Different scoring
functions can result in different ordering of obtained forecasts, therefore
Ehm et al. (2016) suggested using the so-called “Murphy diagrams”
because they are capable of providing concrete measures that indicate if
one forecast is superior to another under several scoring functions. This
methodology is also used in this paper.
This paper is organised as follows: First a literature review is pre­
sented in order to demonstrate the need to use a DMA combination
scheme together with the VAR approach. It also provides arguments for
the pre-selection of potentially important oil price drivers. Next, the
Data section provides a description of the time-series that was used and
the transformations made. In the Methodology section, a brief descrip­
tion of the modelling scheme is presented. Finally, in the Result section,
the outcomes are presented and discussed. The brief summary of out­
comes is presented in the last section, Conclusions.
2
Resources Policy 74 (2021) 102244
2. Literature review
VAR modelling is one of various methods used in forecasting crude
oil price. For a comprehensive review of several other techniques, the
paper by Bashiri Behmiri and Manso (2013) is a good reference. More
updated reviews can be found in the paper by Degiannakis et al. (2018).
Although VAR models are used mostly to study the impulse-response
issues between several variables, there is also evidence that their fore­
casting ability is superior to other selected methods (Alquist et al., 2013;
Kilian, 2009). This conclusion holds not only for crude oil prices, but
energy commodities prices in general (Guo et al., 2016; Nick and
Thoenes, 2014).
First of all, it should be noticed that traditionally much attention was
paid to fundamental factors such as supply and demand forces in fore­
casting oil prices. This approach dominated in the 1980s, but later re­
searchers noticed the influence of exchange rates and stock markets. At
the beginning of the new millennia, much attention was given to the
financialization of commodity markets, speculative forces, and more
complex links with financial markets (Drachal, 2016). Fan and Xu
(2011) analysed structural changes in the oil market even more closely,
focusing on four-year or even smaller periods between 2000 and 2009.
They examined weekly international oil prices and concluded that the
main drivers of oil price and their way of influence changes in time. They
strongly suggested that structural changes must be taken into account,
and the utility of analysis of the full sample periods usually is small.
Improvement in forecasting can be achieved if certain periods of the oil
market are identified and analysed separately. Both papers (Drachal,
2016; Fan and Xu, 2011) contain substantial literature reviews over
various potential oil price drivers.
Of course, these considerations on changing drivers of oil prices
migrated into VAR modelling. For example, Fueki et al. (2018) studied a
Structural VAR model. They concluded that supply and demand shocks
can explain only around 30% of oil price fluctuations. However, they
noticed that shocks on expectations and those due to financial factors are
able to explain more than 40% of oil price fluctuations. Because there
are several important drivers of oil prices and the relationship between
oil markets and other parts of economies is strong, several studies within
a VAR framework have been performed. For example, Cologni and
Manera (2008) used a structural cointegrated VAR model to study the
effects of oil price shocks on output and prices, as well as the reaction of
monetary variables to external shocks in G-7 countries. They found a
significant relationship between oil prices and inflation rates, and that
inflation rate shocks result in increasing interest rates. They also found
the existence of an instantaneous, temporary effect of oil price in­
novations on prices. Gupta and Wohar (2017) considered the Qualitative
Vector Autoregressive model with oil prices, stock prices, and economic
activity. They analysed a period of more than 150 years and confirmed
that VAR-type models lead to more accurate forecasts than, for example,
a random walk model. Their conclusions are applicable mostly to me­
dium and long term forecasting horizon. Byrne et al. (2019) considered
time-varying VAR models to investigate the relationship between oil
prices, traditional fundamentals, and expectations. They found that the
impact of oil fundamentals and expectations shocks on oil prices is
time-varying. In particular, they found that oil prices respond differently
to business leaders’, consumers’ and markets’ prospects expectations.
The most important was business leaders’ expectations. Bekiros et al.
(2015) observed that oil price forecasts derived from a time-varying
VAR model, containing both oil prices and economic policy uncer­
tainty measures, are more accurate than those from univariate models.
They also stressed another aspect of a time-varying parameters
approach, i.e., ability to capture the non-linearities in the relationship
between variables in a model. Chen et al. (2019) investigated causality
between oil price shocks, global economic policy uncertainty, and
China’s industrial economic growth. They have found Granger-type
causalities, even robust taking into account different oil price and pol­
icy uncertainty proxies. Baumeister and Peersman (2013) used VAR
K. Drachal Resources Policy 74 (2021) 102244

Table 1 models, they found that the Bayesian approach led to the most accurate
Time-series used in VAR models. forecasts. These conclusions are accurate for short term forecasting
Abbreviation Description Source horizon. However, higher accuracy was detected both by Mean Squared
Prediction Error and Directional Accuracy measure.
WTI WTI spot oil price in USD per barrel EIA (2019)
GPR Global Economic Policy Uncertainty Index Caldara and However, mixing Bayesian methods with large VAR models can lead
Iacoviello (2019) to important computational obstacles. Fortunately, in recent years there
PROD_WORLD World production of crude oil including EIA (2019) have been numerous improvements in this direction. For example, Koop
lease condensate in thousand barrels per and Korobilis (2019) developed methods for estimating and forecasting
day
CONS_US U.S. product supplied of crude oil and EIA (2019)
large Bayesian panel VARs with time-varying parameters and stochastic
petroleum products in thousand barrels volatility. Korobilis (2013) considered large VARs and proposed a
INV U.S. ending stocks excluding Strategic EIA (2019) method for variable selection using the Gibbs sampler. His method also
Petroleum Reserve of crude oil and estimated time-varying parameters in VAR models. As a practical
petroleum products in thousand barrels
example, he showed that this method leads to more accurate forecasts
CPI Consumer Price Index for all urban FRED (2019)
consumers: All items in U.S. city average, for the selected macroeconomic time-series form UK. Naser (2016)
Index 1982–1984 = 100, seasonally considered several VAR-based models, as well as Dynamic Model
adjusted Averaging (DMA) and Dynamic Model Selection (DMS), in crude oil
IP U.S. Industrial Production Index, Index FRED (2019) price forecasting. She also identified several important oil price drivers
2012 = 100, seasonally adjusted
MSCI_WORLD MSCI WORLD Index of stock prices in USD MSCI (2019)
in these models, such as inflation, supply and demand forces, etc. She
TB3MS U.S. 3-month treasury bill secondary FRED (2019) found that DMA models generate significantly more accurate forecasts
market rate in % of oil prices. Kapetanios et al. (2019) proposed a non-parametric esti­
VXO Implied volatility of S&P 100 in % CBOE (2019) mation for large VARs with time-varying parameters. Their method is
FX Trade weighted U.S. dollar index, major FRED (2019)
computationally efficient and suitable for Factor Augmented VARs.
currencies, goods, March 1973 = 100, not
seasonally adjusted They applied it to forecasting selected macroeconomic variables, and the
KEI Index of global real economic activity Kilian (2019) obtained forecasts were more accurate than those from benchmark
models. Interestingly, they also linked it slightly with the oil market, i.e.,
they studied the effects of oil price innovations on sectoral U.S. indus­
models with drifting coefficients and stochastic volatility to analyse the trial output.
effects of oil supply shocks on the U.S. economy in a time-varying This paper aims to join the model combination scheme in VAR
context. Time-varying effects were found. modelling of crude oil price based on an extension of the DMA method
As it can be seen, a large number of potentially important oil price (Koop and Korobilis, 2013), as is described in the Methodology section.
drivers are considered in large VAR models (Kaya, 2016). Indeed, recent As this method joins Bayesian approach, time-varying parameters and
econometric methods that are able to deal with large numbers of po­ uncertainty on variable selection, as the above literature review states, it
tential predictors are gaining increased traction in the research (Chin should be an interesting method for oil price forecasting. Additionally,
and Li, 2019). In other words, instead of a certain “harsh” pre-selection this method can construct time-varying component models’ weights
of independent variables for a model or model structure itself, it has according to the joint forecasting performance with respect to more than
become popular to consider a relatively large number of models (and one variable. In other words, the considerations of, for example, Bekiros
explanatory variables) in order to construct a combined forecast out of et al. (2015) on economic policy uncertainty and oil price can be
the forecasts from component individual models. This trend can be seen expanded and fit into the proposed modelling scheme.
not only in the case of oil price studies or even energy market research,
but in financial forecasting generally (Chin and Li, 2019; Amisano and 3. Data
Geweke, 2017; Wang et al., 2016).
Another aspect of forecast modelling is to switch from the practice of Monthly data beginning in June 1986 and ending in June 2019 were
narrowing to one particular model towards a weighted averaging fore­ analysed. The time horizon of the data was a compromise between the
cast derived from component separate models. Baumeister and Kilian inclusion of as many interesting indices as possible and the availability
(2015) studied forecast combinations over the naïve method, based on of the data. The initial pre-selection of variables was done on the basis of
futures contracts, and those from VAR models for oil price. Moreover, the above Literature review. It also is consistent with, for example, the
they considered this in time-varying framework. As a result, they ob­ research of Beckers and Beidas Strom (2015). As a result, each
tained improvement in forecast accuracy. Similar advantages of forecast time-series consisted of 397 observations, which allowed for reasonable
combinations for oil price forecasting were confirmed by Manescu and estimations. Table 1 presents the time-series used in the research.
Van Robays (2014). Indeed, with VAR models it was observed even Following Koop and Korobilis (2013), Banbura et al. (2010), and
earlier by Clark and McCracken (2010) that a forecast combination Stock and Watson (2008), these variables were transformed to be
approach can be beneficial in the presence of uncertain model in­ approximately stationary. In particular, all except TB3MS, VXO, GPR,
stabilities. Their approach was expanded by Jore et al. (2010), who and KEI were taken in log-differences. KEI is assumed to be stationary by
observed that if the weights ascribed to component models change as a its construction (Kilian, 2009). Also, before the transformation, WTI was
result of particular economic situations, then the obtained forecasts are deflated by the CPI in order to obtain the real price of crude oil.
more accurate. These studies support the use of time-varying weights in The variables, which were not log-differenced, were divided by 100
averaging forecasts from VAR models. Andersson and Karlsson (2008) in order to keep all time-series in a similar magnitude. Indeed, following
focused on forecast combinations for VAR models and allowed for Koop and Korobilis (2013), after these transformations, all obtained
different combinations of endogenous variables. They studied the time-series were standardized based on the mean and standard deviation
impact of several factors on U.S. GDP. Their approach was also Bayesian, derived from first 100 observations. This was done in order to avoid
and they were able to obtain more accurate forecasts than those from the forward-looking bias, whereas more tight “normalization” of magni­
benchmark models. Baumeister and Kilian (2012) constructed a model tudes was empirically found to improve DMA-based scheme forecast
based on fundamental factors such as oil production, inventories, and accuracy (Drachal, 2018a). Indeed, certain pre-simulations for the
global real economic activity, and were able to generate more accurate models used in this paper also showed that standardization results in
forecasts than those by ARIMA and the naïve method, as well as fore­ smaller forecast errors.
casts based on future contracts. Interestingly, out of various VAR-based The descriptive statistics for the whole sample are reported in

3
K. Drachal Resources Policy 74 (2021) 102244

Table 2
Descriptive statistics.
mean standard deviation min max skew kurtosis

WTIrowhead 0.0077 1.0162 -3.8374 4.7040 -0.2210 1.9708

GPRrowhead 0.5254 1.5904 -0.9961 12.3160 3.1654 15.2616
PROD_WORLDrowhead 0.0182 0.6700 -4.6307 2.8990 -1.2477 10.3301
CONS_USrowhead -0.0057 0.9715 -2.5130 2.6610 -0.0229 -0.3941
INVrowhead 0.0194 0.8903 -3.7799 2.2280 -0.5617 1.2106
CPIrowhead -0.5938 1.5020 -12.5297 6.2860 -1.3920 12.2307
IProwhead -0.1505 1.2427 -9.4014 3.6030 -1.6093 9.2182
MSCI_WORLDrowhead -0.0424 0.9502 -4.7366 2.2320 -0.8914 2.3896
TB3MSrowhead -1.3211 1.3679 -3.0553 1.8420 0.2133 -1.1527
VXOrowhead 0.0842 1.1150 -1.5507 5.6490 1.7035 4.5428
FXrowhead 0.1111 0.9484 -2.6184 3.8210 0.0094 0.4193
KEIrowhead 0.4879 2.9856 -7.7918 9.9600 0.8515 0.9943

Table 3
Stationarity tests.
ADF stat. ADF p-val. PP stat. PP p-val. KPSS stat. KPSS p-val.

WTIrowhead -7.9799 0.0100 -269.3704 0.0100 0.0565 0.1000

GPRrowhead -4.0706 0.0100 -87.1406 0.0100 1.2664 0.0100
PROD_WORLDrowhead -9.6588 0.0100 -383.7078 0.0100 0.0190 0.1000
CONS_USrowhead -12.7025 0.0100 -569.5815 0.0100 0.0248 0.1000
INVrowhead -9.5808 0.0100 -314.7628 0.0100 0.0944 0.1000
CPIrowhead -7.3381 0.0100 -216.1026 0.0100 1.3564 0.0100
IProwhead -5.2285 0.0100 -417.3793 0.0100 0.4658 0.0494
MSCI_WORLDrowhead -6.7989 0.0100 -368.6159 0.0100 0.0756 0.1000
TB3MSrowhead -3.6188 0.0313 -7.4032 0.6960 5.5970 0.0100
VXOrowhead -3.3925 0.0554 -62.4017 0.0100 0.4785 0.0465
FXrowhead -7.3301 0.0100 -239.5633 0.0100 0.1457 0.1000
KEIrowhead -2.5139 0.3602 -20.7778 0.0595 0.7492 0.0100

Table 2. It can be seen, for example, that standard deviations of all

with errors having the multivariate normal distribution, i.e., εt ∼ N(0,
′

variables have a similar magnitude. This is quite an important feature

Σ t ), where yt denotes the vector of endogenous variables, xt – the vector
′

for the models used in this research (Drachal, 2018a). Table 3 reports
stationarity tests for the whole sample, i.e., augmented Dickey-Fuller
test (ADF), Phillips-Perron test (PP), and
Kwiatkowski-Phillips-Schmidt-Shin test (KPSS). Assuming a 5% signifi­
cance level, the time-series can be assumed (at least approximately)
stationary. It should be noticed that KEI is stationary by the construc­
tion. The discrepancy is the result of trimming this time-series to the
period analysed in this research. Fig. 1 presents all time-series after the
performed transformations. This confirms that the final forms of vari­
ables are suitable for inserting into the econometric models. In partic­
ular, there seems to be lack of important seasonality patterns, and the
values seem to oscillate consistently around mean values with deviations
being stable in time.
4. Methodology
All computations were done in R (R Core Team, 2017). The original
DMA method is described in detail by Rafter et al. (2010), and is
extended for VAR-type models in this study according to the idea pro­
posed and explored by Koop (2014), Koop (2013b), and Koop and
Korobilis (2013), and following the paper by Adam and Plasil (2014).
Indeed, the R script underlying the paper by Adam and Plasil (2014) was
modified for the purpose of this paper. Here, only the basic sketch of the
method is presented. For fuller details the reader can review the original
papers.
4.1. Time-varying parameters VAR models
The VAR model can be described by the following matrix-vector
equation:
p q
∑ ∑
yt = ct + Ai,t yt− i +
′
Bi,t xt− j + εt
i=1 j=1
of exogenous variables, ct – the vector of constant terms, and Ai,t and Bj,t
are the matrices of parameters, and p and q denote the number of lags. It
is stressed that herein matrices Ai,t and Bj,t are time-varying.
Herein, the endogenous variables were WTI, GPR, PROD_WORLD,
CONS_US, INV, CPI and IP, which is in line with, for example, Beckers
and Beidas Strom (2015) and Bekiros et al. (2015). For some
pre-simulations, models with MSCI_WORLD were tested, but no signif­
icant gain in forecast accuracy was obtained. Therefore, MSCI_WORLD,
TB3MS, VXO, FX and KEI were taken as exogenous variables (Beckers
and Beidas Strom, 2015). Models with 1 lag, i.e., p = 1 and q = 1, were
estimated. During pre-simulations, models up to 12 lags were tested, but
no significant gain in forecast accuracy was obtained. Moreover, a large
number of lags can lead to over-parametrization and computational
issues.
The key point of this paper is that instead of estimating one particular
VAR model – as there is uncertainty which variables should be really
inserted into the model – one can consider numerous models. In
particular, the set of endogenous variables is desired to always contain
WTI and GPR (Bekiros et al., 2015), but for the other variables there
exists 25 choices (to include or to not include in the model 5 rest vari­
ables). Similarly, for exogenous variables 25 choices exist. As a result, 25
. 5
2 = 1024 time-varying VAR models can be constructed.
Following Koop and Korobilis (2013) after suitable transformation,
Equation (1) can be written as
yt = Zt βt + εt
βt+1 = βt + ut
with εt ∼ N(0,Σ t ),ut ∼ N(0,Qt ), where εt and ut are i.i.d., and εt and us for
all s and t are independent. In order to perform the estimation for each of
(1) these 1024 time-varying VAR models, the initial values of β0 , Σ 0 andQ0
have to be set. β0 was set to 0. Σ 0 was taken with respect to covariances

4
K. Drachal Resources Policy 74 (2021) 102244

of the variables; and Q0 as the identity matrix, which seems reasonable

for the standardized data (Adam and Plasil, 2014; Koop and Korobilis,
2013). It should be also noted that, from the numerical point of view,
such choices are good if forecasted time-series are non-stationary.
Although formally the Kalman filter in such a case requires some
slight modification (Koopman, 1997), in practice, if the forgetting pro­
cedure is applied (mentioned later herein), it numerically also captures
non-stationarity (Andersson and Karlsson, 2008).
Another approach to setting priors for Q0 can be, for example, as
Koop and Korobilis (2013) used the Minnesota prior covariance matrix,
which shrinks on lags. As here models with only 1 lag were estimated,
that would not make a significant change. Moreover, pre-simulations
with different shrinkage parameters did not indicate significance fore­
cast accuracy improvement if the Minnesota prior is used.
The Kalman filter is used for updating βt . Whereas, due to compu­
tational issues, the forgetting procedure with the forgetting factor λ ε
(0,1] is used for updating Qt . This factor controls the allowed variation
of coefficients βt . It corresponds to the efficient window size of (1 – λ)-1.
For example, for λ = 0.99, for the data used herein, this corresponds to
approximately 8 years.
Σ t is updated with the Exponentially Weighted Moving Average
(EWMA) filter, i.e., Σ t = κ Σ t + (1- κ) εt εTt with κ ε (0,1]. Following Koop
and Korobilis (2013), as the monthly data are used, κ = 0.97 was taken.

4.2. Averaging time-varying parameters VAR models

Fig. 1. Time-series used directly in the estimated models.

By the above method single time-varying VAR models can be esti­

mated. The key point is to ascribe time-varying weights to each of these
models. For each J = 1024 models (i = 1, …, J) weights are recursively
computed, following the original idea of Raftery et al. (2010) as
( )α/
πt|t− 1,i = πt− 1|t− 1,i + c ∑J ( π )α
t− 1|t− 1,j + c
j=1

( ⃒ t− 1 )/∑ ( ⃒ t− 1 )
πt|t,i = πt|t− 1,i fi yt ⃒Y J
π t|t− yt ⃒Y
1,j fj
j=1

⃒
where the second forgetting factor α ε (0,1] is used, and fi (yt ⃒Yt− 1 ) de­
notes the predictive likelihood of i-th model based on the multivariate
normal density function. A small constant c = 0.001/J was taken due to
possible numerical rounding to 0, following Raftery et al. (2010).
However, as the single VAR models contain different sets of endogenous
⃒
variables, the fi (yt ⃒Y t− 1 ) measures cannot be compared across different
models.
In order to obtain a meaningful measurement common to all single
VAR models, this measure should be taken with respect only to the
common variables, included in all single models, i.e., to WTI and GPR
(Koop and Korobilis, 2013). In other words, the weights are updated
according to how single VAR models jointly forecast both the WTI and
GPR variable. Initially, π0|0,i = 1/J were taken for all i = 1, …, J (non-­
informative priors).
The forgetting parameter α corresponds to the fact that observations t
periods ago receive αt as much weight as observation in the last period.
For the data used in this paper, it means that, for example, data from
one-year back receive 89% weight, if α = 0.99 is taken. As it is not clear
which α and λ would be “the best” choices, as well as whether the out­
comes would be robust to different forgetting factors, all combinations
of α, λ = {1, 0.99, 0.95, 0.90} were tested.
If α = λ, the DMA scheme recovers Bayesian Model Averaging (BMA),
as explained by Raftery et al. (2010). They also suggested using just
some small value, like α = 0.99 = λ. However, some studies found that
manipulating with forgetting factors can result in more accurate fore­
casts. On the other hand, it is expected to find more differences between
α = 1 = λ and α = 0.99 = λ, i.e., values close to 1 rather than between α
= 0.95 = λ and α = 0.90 = λ. Moreover, too small of forgetting factors
can correspond to too high volatility in estimations and “catching the
noise, rather than the signal” (Drachal, 2018a). Therefore, this

5
K. Drachal Resources Policy 74 (2021) 102244

Table 4 However, Barbieri and Berger (2004) observed that the model with
Forecast accuracy measures for the estimated models. the highest posterior probability might not be optimal. Instead, they
RMSE MAE MAPE MASE proposed selecting the model which would contain exactly those vari­
ables for which relative variable importance is greater than or equal to
DMA-VAR (α = 1, λ = 1)rowhead 2.0571 1.4853 6.2445 0.9515
DMS-VAR (α = 1, λ = 1)rowhead 2.0734 1.5185 6.3837 0.9728 0.5. The relative variable importance of a given variable is defined as the
MPM-VAR (α = 1, λ = 1)rowhead 2.0504 1.4964 6.2952 0.9587 sum of the weights πt|t− 1,i of those single models, which contain the given
DMA-VAR (α = 1, λ = 0.99)rowhead 2.0647 1.4874 6.2556 0.9529 variable. This is called Median Probability Model, denoted herein as
DMS-VAR (α = 1, λ = 0.99)rowhead 2.0798 1.5191 6.3779 0.9732 MPM-VAR. It should be noted that in general MPM might not exist.
MPM-VAR (α = 1, λ = 0.99)rowhead 2.0552 1.5101 6.3346 0.9675
DMA-VAR (α = 1, λ = 0.95)rowhead 2.1780 1.5513 6.4477 0.9939
However, it exists if all combinations of variables are considered; this is
DMS-VAR (α = 1, λ = 0.95)rowhead 2.1877 1.5576 6.4721 0.9979 in the case studied in this paper. As a result, herein 3 different
MPM-VAR (α = 1, λ = 0.95)rowhead 2.1884 1.5614 6.4957 1.0003 time-varying model combination schemes were applied to VAR models.
DMA-VAR (α = 1, λ = 0.90)rowhead 2.2369 1.5979 6.6390 1.0237
DMS-VAR (α = 1, λ = 0.90)rowhead 2.2378 1.5985 6.6393 1.0241
MPM-VAR (α = 1, λ = 0.90)rowhead 2.2383 1.5995 6.6475 1.0248 4.4. Benchmark models
DMA-VAR (α = 0.99, λ = 1)rowhead 2.0413 1.4738 6.1870 0.9442
DMS-VAR (α = 0.99, λ = 1)rowhead 2.0479 1.4809 6.1920 0.9488
The estimated models were compared for forecast accuracy to certain
MPM-VAR (α = 0.99, λ = 1)rowhead 2.0517 1.4806 6.1826 0.9485
DMA-VAR (α = 0.99, λ = 0.99)rowhead 2.0676 1.4797 6.1897 0.9480 benchmark models. The comparison was done on the basis of how the
DMS-VAR (α = 0.99, λ = 0.99)rowhead 2.0392 1.4788 6.1762 0.9474 models can forecast the levels of crude oil real price one period ahead. In
MPM-VAR (α = 0.99, λ = 0.99)rowhead 2.0539 1.4804 6.1816 0.9484 particular the naïve (no-change) method was used (NAÏVE). This
DMA-VAR (α = 0.99, λ = 0.95)rowhead 2.0745 1.4887 6.2012 0.9537
method simply assumes that the forecast is given by ŷt = yt− 1 . Another
DMS-VAR (α = 0.99, λ = 0.95)rowhead 2.1474 1.5328 6.3973 0.9820
MPM-VAR (α = 0.99, λ = 0.95)rowhead 2.1455 1.5271 6.3780 0.9783 benchmark model is the recursively computed auto ARIMA model of
DMA-VAR (α = 0.99, λ = 0.90)rowhead 2.2340 1.5937 6.6167 1.0210 Hyndman and Khandakar (2008).
DMS-VAR (α = 0.99, λ = 0.90)rowhead 2.2383 1.5999 6.6505 1.0250 Additionally, single time-varying VAR model consisting of all
MPM-VAR (α = 0.99, λ = 0.90)rowhead 2.2384 1.6003 6.6541 1.0253
possible variables was also estimated (TVP-VAR) over the corresponding
DMA-VAR (α = 0.95, λ = 1)rowhead 2.0166 1.4556 6.1343 0.9325
DMS-VAR (α = 0.95, λ = 1)rowhead 1.9205 1.4035 5.8901 0.8992
grid of forgetting factor λ. For λ = 1 it is called BVAR. The time-varying
MPM-VAR (α = 0.95, λ = 1)rowhead 1.9303 1.4176 5.9206 0.9082 estimations are done exactly as presented above except they discard the
DMA-VAR (α = 0.95, λ = 0.99)rowhead 2.0625 1.4650 6.1480 0.9386 averaging or selection part, as only one VAR model is then estimated.
DMS-VAR (α = 0.95, λ = 0.99)rowhead 1.9589 1.4171 5.8960 0.9079 Moreover, over the corresponding grid for forgetting factors, the
MPM-VAR (α = 0.95, λ = 0.99)rowhead 2.0097 1.4320 5.9510 0.9174
standard DMA, DMS, and MPM models were estimated in order to check
DMA-VAR (α = 0.95, λ = 0.95)rowhead 2.0725 1.4684 6.1021 0.9407
DMS-VAR (α = 0.95, λ = 0.95)rowhead 1.9808 1.4270 5.8726 0.9143 if switching to VAR-type modelling improves forecast accuracy. Analo­
MPM-VAR (α = 0.95, λ = 0.95)rowhead 1.9766 1.4338 5.9366 0.9186 gously, time-varying linear regressions were estimated (TVP). Analo­
DMA-VAR (α = 0.95, λ = 0.90)rowhead 2.1367 1.5206 6.2947 0.9742 gously, they are simply a DMA scheme reduced to one single model, i.e.,
DMS-VAR (α = 0.95, λ = 0.90)rowhead 2.1847 1.5486 6.3911 0.9921 containing all possible variables in one linear regression equation.
MPM-VAR (α = 0.95, λ = 0.90)rowhead 2.2141 1.5829 6.5693 1.0141
DMA-VAR (α = 0.9, λ = 1)rowhead 2.0100 1.4508 6.1210 0.9295
Analogously, BMA-VAR, BMS-VAR, BMPM-VAR denote the time-
DMS-VAR (α = 0.9, λ = 1)rowhead 1.8665 1.3576 5.7233 0.8698 varying VAR models with α = 1 = λ and averaging, selection of the
MPM-VAR (α = 0.9, λ = 1)rowhead 1.8784 1.3707 5.7625 0.8782 model with the highest posterior probability, and MPM-VAR models
DMA-VAR (α = 0.9, λ = 0.99)rowhead 2.0605 1.4626 6.1411 0.9370 respectively.
DMS-VAR (α = 0.9, λ = 0.99)rowhead 1.9084 1.3635 5.7137 0.8735
MPM-VAR (α = 0.9, λ = 0.99)rowhead 1.9121 1.3693 5.7146 0.8773
DMA-VAR (α = 0.9, λ = 0.95)rowhead 2.0860 1.4728 6.1228 0.9436 4.5. Forecast comparison methods
DMS-VAR (α = 0.9, λ = 0.95)rowhead 1.9023 1.3596 5.6353 0.8710
MPM-VAR (α = 0.9, λ = 0.95)rowhead 1.8999 1.3593 5.6712 0.8709
DMA-VAR (α = 0.9, λ = 0.90)rowhead 2.1404 1.5255 6.3273 0.9773 The forecasts obtained from the models were tested to determine
DMS-VAR (α = 0.9, λ = 0.90)rowhead 1.9933 1.4587 6.0409 0.9345 whether one out of two competing forecast is more accurate with the
MPM-VAR (α = 0.9, λ = 0.90)rowhead 2.1496 1.5328 6.3407 0.9820 Diebold-Mariano test (Diebold and Mariano, 1995). However, because
ARIMArowhead 2.1569 1.5880 6.6974 1.0174 of the numerous forecasts from different models, a Model Confidence Set
NAIVErowhead 2.2269 1.5609 6.5325 1.0000
B-VAR (TVP-VAR with λ = 1)rowhead 2.1029 1.4947 6.3031 0.9576
procedure (MSC) was also performed, as proposed by Hansen et al.
TVP-VAR (λ = 0.99)rowhead 2.2047 1.5218 6.4053 0.9750 (2011). This was done with the help of “MCS” R package (Bernardi and
TVP-VAR (λ = 0.95)rowhead 2.4957 1.7105 7.1072 1.0959 Catania, 2018). Also, the multivariate version of the Diebold-Mariano
TVP-VAR (λ = 0.90)rowhead 2.8278 2.0073 8.3942 1.2860 test procedure was performed (Drachal, 2018b; Mariano and Preve,
2012). For both procedures squared errors (SE) and absolute scaled er­
rors (ASE) loss functions were taken.
motivated the construction of a grid for forgetting factors.
ASE loss function is consistent with the Diebold-Mariano test
Another approach would be to consider time-varying λ, following the
(Franses, 2016). Moreover, MASE (Mean Absolute Scaled Error) derived
idea of Park et al. (1991), as was done by Koop and Korobilis (2013).
from ASE function is a better forecast accuracy measure than commonly
However, pre-simulations with different design parameters for this
used RMSE (Root Mean Squared Error) derived from SE loss function
scheme (Park et al., 1991) did not indicate significant forecast accuracy
(Hyndman and Koehler, 2006). In particular, it is scale invariant. It has
improvements.
better properties when dealing with values close to 0. Additionally, it
Finally, the forecast ŷt is obtained as the weighted average forecast,
J
penalizes positive and negative errors equally, as well as large and small
∑
i.e., ŷt = t,i , where y
πt|t− 1,i ŷ ̂ t,i is the forecast from the single i-th time- values. It also has an easy interpretation: if greater than 1, it indicates
i=1
that the naïve method performs better in a sense of forecast accuracy. On
varying VAR model. This method is called DMA-VAR.
the other hand, measures like Mean Absolute Percentage Error (MAPE)
are not consistent with the Diebold-Mariano test, except that Mean
4.3. Model selection instead of model averaging Absolute Error (MAE) was also reported.
In addition to the Diebold-Mariano test, the fluctuation test of Gia­
Another approach would be to select the ŷ t,i corresponding to the comini and Rossi (2010) was performed. This test is based on the
highest πt|t− 1,i over all i = {1, …, J} for each t. This is called DMS-VAR, Diebold-Mariano test, but it investigates the stability of relative local
from Dynamic Model Selection. forecasting performance. In other words, the Diebold-Mariano test

6
K. Drachal Resources Policy 74 (2021) 102244

Table 5
The Diebold-Mariano test (p-values).
NAIVE (SE) NAIVE (ASE) ARIMA (SE) ARIMA (ASE) B-VAR (SE) B-VAR (ASE)

DMA-VAR (α = 1, λ = 1)rowhead 0.0009 0.0119 0.0562 0.0035 0.1293 0.3459

DMS-VAR (α = 1, λ = 1)rowhead 0.0050 0.1124 0.0775 0.0325 0.2835 0.7914
MPM-VAR (α = 1, λ = 1)rowhead 0.0019 0.0387 0.0332 0.0075 0.1546 0.5234
DMA-VAR (α = 1, λ = 0.99)rowhead 0.0038 0.0306 0.0575 0.0050 0.2495 0.4012
DMS-VAR (α = 1, λ = 0.99)rowhead 0.0163 0.1634 0.0941 0.0338 0.3643 0.7484
MPM-VAR (α = 1, λ = 0.99)rowhead 0.0092 0.1208 0.0384 0.0220 0.2462 0.6690
DMA-VAR (α = 1, λ = 0.95)rowhead 0.2950 0.4299 0.6595 0.1772 0.7820 0.8678
DMS-VAR (α = 1, λ = 0.95)rowhead 0.3319 0.4757 0.7260 0.2236 0.8112 0.8906
MPM-VAR (α = 1, λ = 0.95)rowhead 0.3348 0.5040 0.7306 0.2541 0.8133 0.9042
DMA-VAR (α = 1, λ = 0.90)rowhead 0.5353 0.7139 0.8797 0.5803 0.8793 0.9489
DMS-VAR (α = 1, λ = 0.90)rowhead 0.5384 0.7171 0.8820 0.5849 0.8807 0.9497
MPM-VAR (α = 1, λ = 0.90)rowhead 0.5400 0.7223 0.8834 0.5930 0.8815 0.9514
DMA-VAR (α = 0.99, λ = 1)rowhead 0.0007 0.0061 0.0418 0.0021 0.0527 0.1553
DMS-VAR (α = 0.99, λ = 1)rowhead 0.0015 0.0192 0.0637 0.0061 0.0997 0.2841
MPM-VAR (α = 0.99, λ = 1)rowhead 0.0018 0.0196 0.0699 0.0055 0.1401 0.2870
DMA-VAR (α = 0.99, λ = 0.99)rowhead 0.0035 0.0213 0.0942 0.0055 0.2585 0.2788
DMS-VAR (α = 0.99, λ = 0.99)rowhead 0.0027 0.0354 0.0389 0.0062 0.1269 0.3004
MPM-VAR (α = 0.99, λ = 0.99)rowhead 0.0049 0.0385 0.0704 0.0076 0.2072 0.3202
DMA-VAR (α = 0.99, λ = 0.95)rowhead 0.0620 0.1113 0.0740 0.0123 0.3807 0.4544
DMS-VAR (α = 0.99, λ = 0.95)rowhead 0.2020 0.3148 0.4393 0.1244 0.6721 0.7573
MPM-VAR (α = 0.99, λ = 0.95)rowhead 0.1958 0.2809 0.4275 0.1032 0.6671 0.7254
DMA-VAR (α = 0.99, λ = 0.90)rowhead 0.5249 0.6913 0.8710 0.5460 0.8742 0.9415
DMS-VAR (α = 0.99, λ = 0.90)rowhead 0.5401 0.7241 0.8835 0.5958 0.8816 0.9520
MPM-VAR (α = 0.99, λ = 0.90)rowhead 0.5405 0.7263 0.8838 0.5992 0.8818 0.9526
DMA-VAR (α = 0.95, λ = 1)rowhead 0.0008 0.0014 0.0151 0.0003 0.0234 0.0298
DMS-VAR (α = 0.95, λ = 1)rowhead 0.0005 0.0001 0.0020 0.0000 0.0020 0.0004
MPM-VAR (α = 0.95, λ = 1)rowhead 0.0011 0.0004 0.0036 0.0000 0.0054 0.0026
DMA-VAR (α = 0.95, λ = 0.99)rowhead 0.0023 0.0072 0.0710 0.0015 0.2359 0.1148
DMS-VAR (α = 0.95, λ = 0.99)rowhead 0.0005 0.0009 0.0010 0.0000 0.0164 0.0104
MPM-VAR (α = 0.95, λ = 0.99)rowhead 0.0029 0.0025 0.0204 0.0001 0.1175 0.0353
DMA-VAR (α = 0.95, λ = 0.95)rowhead 0.0625 0.0717 0.0992 0.0070 0.3737 0.3156
DMS-VAR (α = 0.95, λ = 0.95)rowhead 0.0200 0.0191 0.0195 0.0011 0.1101 0.1158
MPM-VAR (α = 0.95, λ = 0.95)rowhead 0.0179 0.0245 0.0172 0.0018 0.1008 0.1392
DMA-VAR (α = 0.95, λ = 0.90)rowhead 0.2166 0.2729 0.3854 0.0852 0.6210 0.6627
DMS-VAR (α = 0.95, λ = 0.90)rowhead 0.3522 0.4250 0.6512 0.2243 0.7625 0.7996
MPM-VAR (α = 0.95, λ = 0.90)rowhead 0.4544 0.6317 0.7935 0.4595 0.8333 0.9148
DMA-VAR (α = 0.9, λ = 1)rowhead 0.0009 0.0009 0.0124 0.0002 0.0206 0.0181
DMS-VAR (α = 0.9, λ = 1)rowhead 0.0001 0.0000 0.0003 0.0000 0.0003 0.0000
MPM-VAR (α = 0.9, λ = 1)rowhead 0.0002 0.0000 0.0005 0.0000 0.0006 0.0000
DMA-VAR (α = 0.90, λ = 0.99)rowhead 0.0018 0.0053 0.0638 0.0011 0.2224 0.0923
DMS-VAR (α = 0.9, λ = 0.99)rowhead 0.0000 0.0000 0.0001 0.0000 0.0006 0.0000
MPM-VAR (α = 0.90, λ = 0.99)rowhead 0.0000 0.0000 0.0002 0.0000 0.0007 0.0001
DMA-VAR (α = 0.90, λ = 0.95)rowhead 0.0789 0.0816 0.1472 0.0095 0.4301 0.3440
DMS-VAR (α = 0.90, λ = 0.95)rowhead 0.0036 0.0008 0.0014 0.0000 0.0227 0.0083
MPM-VAR (α = 0.90, λ = 0.95)rowhead 0.0035 0.0008 0.0013 0.0000 0.0221 0.0074
DMA-VAR (α = 0.90, λ = 0.90)rowhead 0.2244 0.3018 0.4070 0.1091 0.6358 0.6911
DMS-VAR (α = 0.9, λ = 0.90)rowhead 0.0396 0.0666 0.0351 0.0074 0.1703 0.2884
MPM-VAR (α = 0.9, λ = 0.90)rowhead 0.2421 0.3321 0.4568 0.1409 0.6607 0.7263

compares two forecasts’ on average global performance, whereas the 5. Results

fluctuation test of Giacomini and Rossi focuses on the entire time path of
the models’ relative performance. These computations were done in the
R package by Jordan and Kruger (2019).
Finally, Murphy diagrams proposed by Ehm et al. (2016) were
sketched. The motivation behind this relatively new tool is that when
comparing forecasts with accuracy measures that are aimed to be the
estimates of the mean, the comparison to only MSE is not enough. This is
because SE loss function is not the only loss function whose minimising
is obtained through the mean. Moreover, different scoring functions can
lead to different forecasts rankings. The consistent with mean functions
can differ by certain parameters.
Therefore a certain measure can be derived from considering several
values of this parameter and plotting them as a function of this param­
eter for the given two competing forecasts. The aim is to check if one
forecast dominates over the second for all values of this parameter. As a
result, a more general tool than a particular selected forecast accuracy
measure can be obtained, which can also be easily interpreted graphi­
cally. These computations were done with the script provided by
Hyndman (2015).
Table 4 presents forecast accuracy measures for the estimated
models. For all forecast accuracy diagnostics, the first 100 observations
were treated as in-sample period, which was removed from these com­
putations. Only the remaining observations were taken as an out-of-
sample set. This table reports Root Mean Squared Error (RMSE), Mean
Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and
Mean Absolute Scaled Error (MASE). It can be seen that DMS-VAR model
with α = 0.90 and λ = 1 minimised both RMSE and MASE, which was
determined to be the most important measure, as previously mentioned.
As a result, it can be seen that, first, it was the model selection scheme
that lead to the most accurate forecasts – not model averaging scheme.
Secondly, λ = 1 means that the time-varying property of coefficients in
VAR linear equations is not as beneficial as time-varying weighting of
models, i.e., switching between multiple component single VAR models.
It can be seen that almost 78% of estimated VAR models with a
model combination scheme generated smaller RMSE than ARIMA
method, and therefore also NAÏVE method. As for the data considered in
this paper, the ARIMA method generated more accurate forecasts than
the NAÏVE method. In the case of MASE measures, it was almost 88% of

7
K. Drachal Resources Policy 74 (2021) 102244

Table 6 Table 7
The Diebold-Mariano test (p-values) for DMS-VAR with α = 0.95 and λ = 1. MCS and MDM test procedures outcomes.
SE ASE x – the model was excluded MCS MSC MDM MDM
(SE) (ASE) (SE) (ASE)
BMA-VARrowhead 0.0006 0.0000
BMS-VARrowhead 0.0008 0.0000 p-value 0.1794 0.1659 1.0000 1.0000
BMPM-VARrowhead 0.0025 0.0000 DMA-VAR (α = 1, λ = 1)
TVP (λ = 1)rowhead 0.0000 0.0000 DMS-VAR (α = 1, λ = 1) x X
TVP (λ = 0.99)rowhead 0.0007 0.0000 MPM-VAR (α = 1, λ = 1)
DMArowhead 0.0000 0.0000 DMA-VAR (α = 1, λ = 0.99) x
DMSrowhead 0.0001 0.0000 DMS-VAR (α = 1, λ = 0.99) x
MPM-VAR (α = 1, λ = 0.99) x
DMA-VAR (α = 1, λ = 0.95) x
DMS-VAR (α = 1, λ = 0.95) x
MPM-VAR (α = 1, λ = 0.95) x
DMA-VAR (α = 1, λ = 0.90)
DMS-VAR (α = 1, λ = 0.90) x
MPM-VAR (α = 1, λ = 0.90) x
DMA-VAR (α = 0.99, λ = 1)
DMS-VAR (α = 0.99, λ = 1)
MPM-VAR (α = 0.99, λ = 1)
DMA-VAR (α = 0.99, λ = x
0.99)
DMS-VAR (α = 0.99, λ =
0.99)
MPM-VAR (α = 0.99, λ =
0.99)
DMA-VAR (α = 0.99, λ = x
0.95)
Fig. 2. Number of variables in DMS-VAR (α = 0.90, λ = 1). DMS-VAR (α = 0.99, λ = x
0.95)
MPM-VAR (α = 0.99, λ = x
0.95)
DMA-VAR (α = 0.99, λ = x x
0.90)
DMS-VAR (α = 0.99, λ = x x
0.90)
MPM-VAR (α = 0.99, λ = x x
0.90)
DMA-VAR (α = 0.95, λ = 1)
DMS-VAR (α = 0.95, λ = 1)
MPM-VAR (α = 0.95, λ = 1)
DMA-VAR (α = 0.95, λ = x
0.99)
DMS-VAR (α = 0.95, λ =
0.99)
MPM-VAR (α = 0.95, λ =
0.99)
DMA-VAR (α = 0.95, λ = x
0.95)
Fig. 3. Inclusion of variables in the model with the highest posterior proba­
DMS-VAR (α = 0.95, λ =
bility for DMSVAR (α = 0.90, λ = 1) scheme. 0.95)
MPM-VAR (α = 0.95, λ =
models. Also, amongst TVP-VAR models, λ = 1 minimised errors. 0.95)
DMA-VAR (α = 0.95, λ = x
Interestingly, 67% of VAR models with model combination schemes
0.90)
generated lower RMSE than TVP-VAR model with λ = 1. In the case of DMS-VAR (α = 0.95, λ = x x
MASE measures, it was 58%. 0.90)
The general conclusion is that if λ is fixed, then lowering α results in MPM-VAR (α = 0.95, λ = x x
smaller RMSE. On the other hand, for the fixed value of α, increasing λ 0.90)
DMA-VAR (α = 0.9, λ = 1)
leads to smaller RMSE. This confirms the observation that a model DMS-VAR (α = 0.9, λ = 1)
combination with a time-varying manner (i.e., weights) is more bene­ MPM-VAR (α = 0.9, λ = 1)
ficial to forecast accuracy than time-variation of coefficients. As DMA-VAR (α = 0.90, λ =
mentioned before, much lower values of forgetting factors were not 0.99)
DMS-VAR (α = 0.9, λ =
tested, as they would lead to too much volatility and over-fitting,
0.99)
making the results not reasonable (Karny, 2006). The obtained results MPM-VAR (α = 0.90, λ =
are in line with Baumeister and Kilian (2014), who concluded that 0.99)
TVP-VAR models are not able to provide better forecasts compared to DMA-VAR (α = 0.90, λ = x x
the established VAR-based forecasts, but forecast averaging is able to 0.95)
DMS-VAR (α = 0.90, λ =
improve the VAR-based forecasts, except that model selection schemes 0.95)
usually resulted in smaller errors than model averaging schemes. MPM-VAR (α = 0.90, λ = x
Interestingly, in the case of the model selection scheme, DMS was usu­ 0.95)
ally preferred over MPM. DMA-VAR (α = 0.90, λ = x x x
0.90)
Because original DMA, DMS, and MPM models for oil prices were
examined in detail elsewhere (Drachal 2016, 2018a; Naser, 2016), the
(continued on next page)
outcomes of these methods are not reported here in full. It is noted,

8
K. Drachal Resources Policy 74 (2021) 102244

Table 7 (continued ) though, that in 67% of cases VAR-type model combination schemes
x – the model was excluded MCS MSC MDM MDM generated lower RMSE than the DMA/DMS/MPM/TVP scheme mini­
(SE) (ASE) (SE) (ASE) mising RMSE out of all DMA/DMS/MPM/TVP models. This means that
DMS-VAR (α = 0.9, λ =
switching from a single equation modelling scheme to VAR is beneficial
0.90) for forecast accuracy. For MASE measures, it was 60% of models. In
MPM-VAR (α = 0.9, λ = x DMA/DMS/MPM/TVP models, DMS with α = 0.90 and λ = 1 minimised
0.90) RMSE (2.0906), and DMA with the same combination of forgetting
factors minimised MASE (0.9628). In TVP models, the one with λ = 1

Fig. 4. Test statistics from the Giacomini and Rossi test for various combination of forgetting factors for TVP-VAR models. The dotted line depicts critical value for
5% significance level.

9
K. Drachal
Fig. 5. Test statistics from the Giacomini and Rossi test for the selected DMS-VAR model and the selected standard DMS model. The dotted line depicts critical value
for 5% significance level.
minimised both RMSE (2.1184) and MASE (0.9809).
Table 5 reports p-values from the Diebold-Mariano test both with SE
and ASE loss functions. The null hypothesis is that forecasts from both
tested models have the same accuracy. The alternative hypothesis is that
the VAR model forecast is more accurate than the forecast from the
benchmark model. As the benchmark models, NAÏVE, ARIMA, and B-
VAR models were taken. If not stated otherwise, 5% significance level
was assumed. It can be seen that 60% of VAR-type models generated
statistically significantly more accurate forecasts than the NAÏVE
method if SE loss function is considered, or 52% of models if ASE loss
function is considered. However, if the ARIMA method is taken as the
benchmark, then it is only 40% of models for SE loss function, and 67%
of models for ASE loss function. Unfortunately, only 23% of models
generated statistically significantly more accurate forecasts than the B-
VAR model, i.e., Bayesian VAR model with no averaging, all variables,
and estimations with forgetting factor λ = 1 and the Kalman filter
updating, if SE loss function was considered. For ASE loss function it was
25% of models. For clarification, B-VAR is the same as TVP-VAR (λ = 1)
under the notation used in this paper. On the other hand, the DMS-VAR
with α = 0.9 and λ = 1 passed the Diebold-Mariano test in all versions
with very small p-values. Additionally, both DMA-VAR and MPM-VAR
schemes with the same forgetting passed this test. Nevertheless, DMS
and MPM schemes, i.e., selection schemes, generated lower p-values
than DMA, i.e., model averaging schemes. This suggests that suitable
manipulation of forgetting factors can lead to better forecast accuracy.
Additionally, Table 6 reports the p-values for the Diebold-Mariano
test for the selected models. It was checked to determine if forecasts
from the selected DMS-VAR model with α = 0.90 and λ = 1 are statis­
tically significantly more accurate than some additional benchmark
models. In particular, BMA-VAR (i.e., DMA-VAR with α = 1 = λ), BMS-
VAR (i.e., DMS-VAR with α = 1 = λ), BMPM-VAR (i.e., MPM-VAR with α
= 1 = λ), TVP models with λ = 1 and with λ = 0.99, and standard DMA
and DMS models were taken. In the case of DMA and DMS, the ones
minimising RMSE out of the previously mentioned grid of forgetting
factors were taken. For both schemes it happened to be the combination
of α = 0.90 and λ = 1. It can be seen that at 5% significance level, for
both loss functions – SE and ASE – the selected DMS-VAR with α = 0.95
and λ = 1 generated statistically significantly more accurate forecasts.
This means that the selected model combination scheme indeed im­
proves forecast accuracy over considering just one Bayesian VAR model
(with no consideration of several single component models to be com­
bined, i.e., averaged or selected). Secondly, VAR-type modelling is
preferred over standard DMA/DMS schemes (i.e., considering single
linear equations). It is also preferred over the Bayesian-called averaging,
i.e., with α = 1 = λ. Therefore, forgetting the time-varying weights led to
more accurate forecasts.
10
Resources Policy 74 (2021) 102244
Because, the selected DMS-VAR model with α = 0.90 and λ = 1
generated statistically significantly more accurate forecasts, it is worth
seeing the outcomes from this model. Fig. 2 depicts the number of
variables (both endogenous and exogenous, except the constant term) in
the model which obtained in a given time the highest posterior proba­
bility, i.e., was selected by the DMS scheme. It can be seen that quite
parsimonious models are preferred, i.e., those with mostly between 2
and 6 variables. The model with all variables was never selected.
However, during market turbulence, the model combination scheme
increases the number of variables, i.e., the VAR model with more vari­
ables was selected. This clearly presents the “learning” ability of the
described model combination scheme.
Fig. 3 presents which variables were included in the model with the
highest posterior probability in DMS-VAR with α = 0.90 and λ = 1. For
example, it can be seen that shortly after 2005, IP was not included, nor
was KEI. This means that no indicator of economic activity was included
in that period. However, at least one of these indicators of economic
activity was included most of the time. For quite a long period after
2010, VXO measuring market stress was not included. For interest rates
at least one (TB3MS or CPI) was included. However, between 2010 and
2015 inflation was not included. Before 2000 MSCI_WORLD, i.e., vari­
able corresponding to stock prices, was not included. Quite often in­
ventories level (INV) was included. Similarly, supply forces
(PROD_WORLD) were included. On the other hand, demand forces
(CONS_US) were included much more rarely.
Table 7 presents outcomes from Model Confidence Set (MCS) test
and multivariate Diebold-Mariano test procedure (MDM) for both SE
and ASE loss functions. In particular, this table reports the models which
were excluded by the test. In other words, the remaining models
generate forecasts with the same accuracy according to the given tests. It
can be seen that MCS test is more restrictive, i.e., it excluded more
models than MDM procedure. Similarly, SE loss function trims the set of
equally accurate forecasts more than ASE loss function. According to
many excluded models, MCS indicated that the combination of suitable
forgetting factors is essential and impacts forecast accuracy. According
to MDM procedure, which excluded few models, there is not much dif­
ference in forecast accuracy if different combinations of forgetting fac­
tors are chosen. Table 7 also reports p-values for the corresponding test
procedures.
Fig. 4 depicts outcomes from the Giacomini and Rossi test for several
DMA-VAR, DMS-VAR, and MPM-VAR models with different forgetting
factors. The bolded thick line corresponds to the selected DMS-VAR
model with α = 0.90 and λ = 1. It can be seen that at 5% significance
level, the forecasts from TVP-VAR models (but even then when only
some specific combination of forgetting factors is used) are statistically
significantly more accurate than the considered benchmark models only
K. Drachal Resources Policy 74 (2021) 102244

Fig. 6. Murphy diagrams for the selected DMS-VAR model with α = 0.90 and λ = 1 and the selected benchmark models.

11
K. Drachal
during the period between 2009 and 2012. This is an interesting result.
Although the standard Diebold-Mariano test indicated that selected
forecasts are statistically significantly more accurate than the ones from
the benchmark models, more detailed checks with the Giacomini and
Rossi test indicated that this happened only within the particular period.
Generally, for commodities prices it is hard to generate more accurate
forecasts than even the no-change (NAÏVE) method (Drachal, 2016).
However, researchers from time to time find models that on some
out-of-sample set are able to generate more accurate forecasts. Despite
this fact, when the new method is used in practice (for example, as an
investing strategy), it does not generate extraordinary gains over a
longer time. One of possible explanation of this paradox is that, as found
herein, such extraordinary forecasting ability is only temporary, and the
next research question could be how to identify periods when the given
forecasting method should be used.
Fig. 5 depicts a similar comparison as Fig. 4, but only for the selected
DMS-VAR model with α = 0.90 and λ = 1, and the selected standard DMS
model is taken as the benchmark model. The results are similar to those
from Fig. 4. It is quite an interesting result that improved forecast ac­
curacy was found during the period of the global financial crisis. Chin
and Li (2019) also found that their Bayesian VAR-based model combi­
nation scheme significantly improved the forecast accuracy only during
specific periods.
Finally, Murphy diagrams are depicted in Fig. 6. The parameter θ
corresponds to different scoring functions (Ehm et al., 2016). The blue
line corresponds to the selected DMS-VAR model and the red line to the
selected benchmark model. Unfortunately, the line corresponding to the
selected DMS-VAR model does not lie below the red line for all θ,
although it does in many cases. This means that it cannot be said that
under all scoring functions the forecasts from the selected DMS-VAR
model are more accurate than those from the benchmark model. The
right diagrams depict the differences, and the shaded region the 95%
confidence interval. In some cases the differences are positive, and in
some they are negative, which cannot lead to the formulation of a
general conclusion. The DMS-VAR forecast is preferable when the dif­
ferences from the right diagrams are positive.
6. Conclusions
Generally, the benefits of model combination schemes for VAR-type
models were found. In particular, it was found that time-varying co­
efficients in VAR models for the crude oil real prices are not as beneficial
as the implementation of the model combination scheme. Generally,
under uncertainty, whose predictors should be included in the VAR
model, it was found that the model selection scheme performs better
than the model averaging. It was also found that a time-varying model
selection scheme is desired. In other words, during different periods of
time, the combination of important oil price drivers varies in time.
On the other hand, the estimated model combination schemes were
able to generate statistically more significant forecasts than the bench­
mark models, like the no-change (NAÏVE) forecast or recursively esti­
mated ARIMA model. However, this conclusion applies when the
forecast performance is considered for the overall tested period in gen­
eral. More detailed examination with the fluctuation test indicated that
only in the selected periods is it possible to generate significantly more
accurate forecasts. Further, using the relatively new tool of Murphy
diagrams, it was found that the conclusions can be highly dependent on
the choice of the scoring function.
It was possible to find different oil price drivers in different periods of
time. The proposed models have good explanatory properties; however,
their practical applications, in for example, investment strategies, need
more attention. By using relatively new econometric tools like the
fluctuation test or Murphy diagrams, some time-varying patterns in
forecast accuracies were found. This, up to the best of our knowledge,
has not yet been examined extensively. From the outcomes of this
research, some clues can be formulated why so many forecasting models
12
Resources Policy 74 (2021) 102244
can claim to have good out-of-sample properties, but still fail in practical
forecasting applications, and why forecasting commodities prices is still
a very hard topic.
Nonetheless, it was found that using VAR-type models in forecasting
can lead to more accurate predictions than using single equation models.
This paper also filled a literature gap in studying TVP-VAR models under
uncertainty about variables by implementing model averaging and
model selection schemes to forecast the real price of crude oil.
Author statement
Krzysztof Drachal: Conceptualization; Data curation; Formal anal­
ysis; Funding acquisition; Investigation; Methodology; Project admin­
istration; Resources; Software; Supervision; Validation; Visualization;
Writing – original draft; Writing – review & editing.
Acknowledgment
Research funded by the Polish National Science Centre grant under
the contract number DEC-2015/19/N/HS4/00205.
References
de Albuquerquemello, V.P., de Medeiros, R.K., da Nobrega Besarria, C., Maia, S.F., 2018.
Forecasting crude oil price: does exist an optimal econometric model? Energy 155,
578–591.
Andersson, M., Karlsson, S., 2008. Bayesian forecast combination for VAR models. In:
Chib, S., Griffiths, W., Koop, G., Terrell, D. (Eds.), Bayesian Econometrics. Emerald
Group Publishing Limited, Bingley, pp. 501–524.
Aastveit, K.A., Bjornland, H.C., 2015. What drives oil prices? Emerging versus developed
economies. J. Appl. Econom. 30, 1013–1028.
Adam, T., Plasil, M., 2014. The Impact of Financial Variables on Czech Macroeconomic
Developments: an Empirical Investigation, vol. 11. Czech National Bank Working
Paper Series.
Alquist, R., Kilian, L., Vigfusson, R.J., 2013. Forecasting the real oil price. In: Elliot, G.,
Timmermann, A. (Eds.), The Handbook of Economic Forecasting. Elsevier,
Amsterdam, pp. 427–507.
Amisano, G., Geweke, J., 2017. Prediction using several macroeconomic models. Rev.
Econ. Stat. 99, 912–925.
Banbura, M., Giannone, D., Reichlin, L., 2010. Large Bayesian vector auto regressions.
J. Appl. Econom. 25, 71–92.
Barbieri, M.M., Berger, J.O., 2004. Optimal predictive model selection. Ann. Stat. 32,
870–897.
Bashiri Behmiri, N., Manso, J.R.P., 2013. Crude oil price forecasting techniques: a
comprehensive review of literature. CAIA Alternat. Invest/ Analy. Rev. 2, 30–48.
Baumeister, C., Kilian, L., 2012. Real-time forecasts of the real price of oil. J. Bus. Econ.
Stat. 30, 326–336.
Baumeister, C., Kilian, L., 2014. What central bankers need to know about forecasting oil
prices. Int. Econ. Rev. 55, 869–889.
Baumeister, C., Kilian, L., 2015. Forecasting the real price of oil in a changing world: a
forecast combination approach. J. Bus. Econ. Stat. 33, 338–351.
Baumeister, C., Peersman, G., 2013. Time-varying effects of oil supply shocks on the US
economy. Am. Econ. J. Macroecon. 5, 1–28.
Beckers, B., Beidas Strom, S., 2015. Forecasting the nominal Brent oil price with VARs -
one model fits all? IMF Work. Pap. 251.
Bekiros, S.D., Gupta, R., Paccagnini, A., 2015. Oil price forecastability and economic
uncertainity. Econ. Lett. 132, 125–128.
Bernardi, M., Catania, L., 2018. The model confidence set package for R. Int. J. Comput.
Econ. Econom. 8, 144–158.
Burnhan, K.P., Anderson, D.R., 2002. Model Selection and Multimodel Inference.
Springer, New York.
Byrne, J.P., Lorusso, M., Xu, B., 2019. Oil prices, fundamentals and expectations. Energy
Econ. 79, 59–75.
Caldara, D., Iacoviello, M., 2019. Measuring geopolitical risk. Working Paper of Board of
Governors of the Federal Reserve Board, p. 12.
CBOE, 2019. Cboe Global Markets. http://www.cboe.com.
Chen, J., Jin, F., Ouyang, G., Ouyang, J., Wen, F., 2019. Oil price shocks, economic
policy uncertainty and industrial economic growth in China. PloS One 14. https://
doi.org/10.1371/journal.pone.0215397. ID e0215397.
Chin, K.-H., Li, X., 2019. Bayesian forecast combination in VAR-DSGE models.
J. Macroecon. 59, 278–298.
Clark, T.E., McCracken, M.W., 2010. Averaging forecasts from VARs with uncertain
instabilities. J. Appl. Econom. 25, 5–29.
Cologni, A., Manera, M., 2008. Oil prices, inflation and interest rates in a structural
cointegrated VAR model for the G-7 countries. Energy Econ. 30, 856–888.
Degiannakis, S., Filis, G., Arora, V., 2018. Oil prices and stock markets: a review of the
theory and empirical evidence. Energy J. 39 https://doi.org/10.5547/
01956574.39.5.sdeg. ID 3178.
K. Drachal
Diebold, F.X., Mariano, R.S., 1995. Comparing predictive accuracy. J. Bus. Econ. Stat. 13,
253–263.
Drachal, K., 2016. Forecasting spot oil price in a dynamic model averaging framework -
have the determinants changed over time? Energy Econ. 60, 35–46.
Drachal, K., 2018a. Determining time-varying drivers of spot oil price in a Dynamic
Model Averaging framework. Energies 11. https://doi.org/10.3390/en11051207. ID
2107.
Drachal, K., 2018b. multDM: Multivariate Version of the Diebold-Mariano Test. https://c
ran.r-project.org/package=multDM.
Ehm, W., Gneiting, T., Jordan, A., Kruger, F., 2016. Of quantiles and expectiles:
consistent scoring functions, Choquet representations, and forecast rankings. J. Roy.
Stat. Soc. B 78, 505–562.
EIA, 2019. U.S. Energy information administration. https://www.eia.gov/petroleum.
Fan, Y., Xu, J.-H., 2011. What has driven oil prices since 2000? A structural change
perspective. Energy Econ. 33, 1082–1094.
Franses, P.H., 2016. A note on the mean absolute scaled error. Int. J. Forecast. 32, 20–22.
FRED, 2019. Federal Reserve bank of St. Louis. https://fred.stlouisfed.org.
Fueki, T., Higashi, H., Higashio, N., Nakajima, J., Ohyama, S., Tamanyu, Y., 2018.
Identifying oil price shocks and their consequences: the role of expectations in the
crude oil market. BIS Work. Pap. 725.
Giacomini, R., Rossi, B., 2010. Forecast comparisons in unstable environments. J. Appl.
Econom. 25, 595–620.
Guo, X., Shi, J., Ren, D., 2016. Coal price forecasting and structural analysis in China.
Discrete Dynam. Nat. Soc. https://doi.org/10.1155/2016/1256168, 2016, ID
1256168.
Gupta, R., Wohar, M., 2017. Forecasting oil and stock returns with a Qual VAR using
over 150 years off data. Energy Econ. 62, 181–186.
Hansen, P.R., Lunde, A., Nason, J.M., 2011. The model confidence set. Econometrica 79,
453–497.
Hyndman, R., 2015. Murphy diagrams in R. https://robjhyndman.com/Rfiles/murphy.R.
Hyndman, R.J., Khandakar, Y., 2008. Automatic time series forecasting: The forecast
package for R. J. Stat. Software 26, 1–22.
Hyndman, R.J., Koehler, A.B., 2006. Another look at measures of forecast accuracy. Int.
J. Forecast. 22, 679–688.
Jacquier, E., Polson, N., 2011. Bayesian methods in finance. In: Geweke, J., Koop, G., van
Dijk, H. (Eds.), The Oxford Handbook of Bayesian Econometrics. Oxford University
Press, Oxford, pp. 439–512.
Jordan, A., Krueger, F., 2019. Murphydiagram: Murphy diagrams for forecast
comparisons. https://CRAN.R-project.org/package=murphydiagram.
Jore, A.S., Mitchell, J., Vahey, S.P., 2010. Combining forecast densities from VARs with
uncertain instabilities. J. Appl. Econom. 25, 621–634.
Kapetanios, G., Marcellino, M., Venditti, F., 2019. Large time-varying parameter VARs: a
non-parametric approach. J. Appl. Econom. 34, 1027–1049.
ed, Karny, M., 2006. Optimized Bayesian Dynamic Advising. Theory and Algorithms.
Springer, Berlin.
Kaya, H., 2016. Forecasting the price of crude oil with multiple predictors. Siyasal
Bilgiler Fakultesi Dergisi 1, 133–151.
Kilian, L., 2009. Not all oil price shocks are alike: Disentangling demand and supply
shocks in the crude oil market. Am. Econ. Rev. 99, 1053–1069.
13
Resources Policy 74 (2021) 102244
Kilian, L., 2019. Updated version of the index of global real economic activity in
industrial commodity markets. https://sites.google.com/site/lkilian2019/research/
datasets.
Koop, G., 2013a. Forecasting with medium and large Bayesian VARs. J. Appl. Econom.
28, 177–203.
Koop, G., 2013b. Using VARs and TVP-VARs with many macroeconomic variables.
Central Eur. J. Econ. Modell. Econometr. 4, 143–167.
Koop, G., 2014. Forecasting with dimension switching VARs. Int. J. Forecast., Elsevier
30, 280–290.
Koop, G., Korobilis, D., 2013. Large time-varying parameter VARs. J. Econom. 177,
185–198.
Koop, G., Korobilis, D., 2019. Forecasting with high-dimensional panel VARs. Oxf. Bull.
Econ. Stat. 81, 937–959.
Koopman, S., 1997. Exact initial Kalman filtering and smoothing for nonstationary time
series models. J. Am. Stat. Assoc. 92, 1630–1638.
Korobilis, D., 2013. VAR forecasting using Bayesian variable selection. J. Appl. Econom.
28, 204–230.
Manescu, B., Van Robays, C.I., 2014. Forecasting the Brent Oil Price: Addressing Time-
Variation in Forecast Performance. ECB Working Paper, p. 1735.
Mariano, R.S., Preve, D., 2012. Statistical tests for multiple forecast comparison.
J. Econom. 169, 123–130.
MSCI, 2019. End of day index data search. https://www.msci.com/end-of-day-data-sea
rch.
Naser, H., 2016. Estimating and forecasting the real prices of crude oil: a data rich model
using a dynamic model averaging (DMA) approach. Energy Econ. 56, 75–87.
Nick, S., Thoenes, S., 2014. What drives natural gas prices? - a structural VAR approach.
Energy Econ. 45, 517–527.
Park, D., Jun, B., Kim, J., 1991. Fast tracking RLS algorithm using novel variable
forgetting factor with unity zone. Electron. Lett. 27, 2150–2151.
R Core Team, 2017. R: A Language and Environment for Statistical Computing. R
Foundation for Statistical Computing, Vienna. https://www.R-project.org.
Raftery, A.E., Karny, M., Ettler, P., 2010. Online prediction under model uncertainty via
Dynamic Model Averaging: Application to a cold rolling mill. Technometrics 52,
52–66.
Roubaud, D., Arouri, M., 2018. Oil prices, exchange rates and stock markets under
uncertainty and regime-switching. Finance Res. Lett. 27, 28–33.
Stefanski, R., 2014. Structural transformation and the oil price. Rev. Econ. Dynam. 17,
484–504.
Stock, J., Watson, M., 2008. Forecasting in dynamic factor models subject to structural
instability. In: Castle, J., Shephard, N. (Eds.), The Methodology and Practice of
Econometrics, A Festschrift in Honour of Professor David F. Hendry. Oxford
University Press, Oxford, pp. 173–205.
Wang, Y., Liu, L., Wu, C., 2017. Forecasting the real prices of crude oil using forecast
combinations over time-varying parameter models. Energy Econ. 66, 337–348.
Wang, Y., Ma, F., Wei, Y., Wu, C., 2016. Forecasting realized volatility in a changing
world: a dynamic model averaging approach. J. Bank. Finance 64, 136–149.
Yang, C.W., Hwang, M.J., Huang, B.N., 2002. An analysis of factors affecting price
volatility of the US oil market. Energy Econ. 24, 107–119.
Zhang, Y., Wei, Y., Zhang, Y., Jin, D., 2019. Forecasting oil price volatility: forecast
combination versus shrinkage method. Energy Econ. 80, 423–433.