Judgmental Forecasting and Adjustments

ECON3039: Topics in Econometrics

Dr Muhammad Shafiullah

School of Economics
University of Nottingham Malaysia

1 August 2021
Relevant book chapter

Chapters 10, “Forecasting: Methods and Applications”, 3rd Edition,

Spyros Makridakis, Steven C. Wheelwright and Rob J. Hyndman,
John Wiley & Sons, Inc.

MS (UNM) Judgmental Forecast 1 August 2021 2 / 27

List of Topics

1 The accuracy of judgmental forecasts.

2 The nature of judgmental biases and limitations.
3 Combining statistical and judgmental forecasts.

MS (UNM) Judgmental Forecast 1 August 2021 3 / 27

Judgmental Forecasting

So far, we have discussed statistical forecasting methods that allow us

to extrapolate established patterns and/or existing relationships in
order to predict their continuation, assuming that such
patterns/relationships will not change during the forecasting phase.
At the same time, because changes can and do occur, these must be
detected as early as possible to avoid large, usually costly, forecasting
errors.

MS (UNM) Judgmental Forecast 1 August 2021 4 / 27

Judgmental Forecasting (cont’d)

Source: Australian Bureau of Statistics (2021).

MS (UNM) Judgmental Forecast 1 August 2021 5 / 27

Judgmental Forecasting (cont’d)

When changes are detected, or if we can know when they are about
to occur, human judgment is the only viable alternative for predicting
both their extent and their implications on forecasting.
Human judgment is also needed to incorporate inside information and
knowledge, as well as managers’ experience, about the future.

MS (UNM) Judgmental Forecast 1 August 2021 6 / 27

The accuracy of judgmental forecasts
The accuracy of forecasts in financial and other markets

The accuracy of judgmental forecasts is, on average, inferior to

statistical ones.
This is because our judgment is often characterized by considerable
biases and limitations.
A large number of forecasts are made virtually every day on the stock,
bond, interest, foreign exchange, and futures markets.
As the purpose of buying and selling in these markets is to maximize
profits, it is straightforward to evaluate the accuracy of these market
forecasts.

MS (UNM) Judgmental Forecast 1 August 2021 7 / 27

The accuracy of judgmental forecasts (cont’d)

Source: World Economic Forum (2016).

https://www.weforum.org/agenda/2016/10/10-predictions-for-the-future-that-got-it-wildly-wrong/

MS (UNM) Judgmental Forecast 1 August 2021 8 / 27

The accuracy of judgmental forecasts (cont’d)
The accuracy of forecasts in financial and other markets (cont’d)

The advice of newsletters for investments

In a comprehensive study of 237 investment strategies recommended by
newsletters over a period of 13 years (1980 to 1992), Graham and Harvey
(1995) conclude the following about the accuracy of these strategies:
Only 22.8% of newsletters have average returns higher than a passive
portfolio of equity and cash with the same volatility.
Consistent with mutual funds studies, we find that poor performance
is far more persistent than good performance.

MS (UNM) Judgmental Forecast 1 August 2021 9 / 27

The accuracy of judgmental forecasts (cont’d)
The accuracy of forecasts in financial and other markets (cont’d)

The advice of newsletters for investments (cont’d)

We find no evidence that the investment letters as a group have any
knowledge over and above the common level of predictability.
The bottom line is that very few newsletters can “beat” the S&P
500. In addition, few can beat the market forecasts derived from a
statistical representation of publicly available information.

MS (UNM) Judgmental Forecast 1 August 2021 10 / 27

The accuracy of judgmental forecasts (cont’d)
The accuracy of forecasts in financial and other markets (cont’d)

Professionally managed investment funds

Professional managers of investment funds make actual ones by investing
many billions in the stock, bond, and other markets.
Their performance is not satisfactory.
Figure 10-1 (next slide) shows the percentage of professional
managers who, over a period of three, five, and ten years, have
beaten the S&P 500 index, or the benchmark master bond index.

MS (UNM) Judgmental Forecast 1 August 2021 11 / 27

The accuracy of judgmental forecasts (cont’d)
The accuracy of forecasts in financial and other markets (cont’d)

Source: Makridakis et al. (1998), Chapter 10, p.486.

MS (UNM) Judgmental Forecast 1 August 2021 12 / 27

The accuracy of judgmental forecasts (cont’d)
The accuracy of forecasts in financial and other markets (cont’d)

Professionally managed investment funds (cont’d)

This percentage is always below 50%, indicating that these managers
have not done better than someone selecting stocks and bonds
randomly—that is, if the managers had used the random walk model:

ŷT +h = yt

The random walk approach uses the most recent actual value as the
forecast for the next period.

MS (UNM) Judgmental Forecast 1 August 2021 13 / 27

The accuracy of judgmental forecasts (cont’d)
The accuracy of forecasts in financial and other markets (cont’d)

Professionally managed investment funds (cont’d)

The same conclusion can be drawn from Figure 10-2 (next slide),
which shows the average return of professionally managed funds
versus those of the S&P index.
There was even an instance when a (house) cat was as good as
professional managers in picking financial instruments.

MS (UNM) Judgmental Forecast 1 August 2021 14 / 27

The accuracy of judgmental forecasts (cont’d)
The accuracy of forecasts in financial and other markets (cont’d)

Source: Makridakis et al. (1998), Chapter 10, p.486.

MS (UNM) Judgmental Forecast 1 August 2021 15 / 27

The accuracy of judgmental forecasts (cont’d)
Non-investment type forecasts

Salespeople forecasts
“Salesmen” forecasts were once very popular, since salespeople, being
close to customers, are presumably in a position to know about
forthcoming changes in the marketplace.
Empirical evidence (Walker and McClelland, 1991; Winklhofer et al.,
1996) has shown, however, that salespeople’s forecasts are notoriously
inaccurate.

MS (UNM) Judgmental Forecast 1 August 2021 16 / 27

The accuracy of judgmental forecasts (cont’d)
Non-investment type forecasts (cont’d)

Management forecasts
Managers, unlike salespeople, have a more global picture of the firm,
its market(s), and the economy.
However, they are often overoptimistic about the firm’s future or the
products they are responsible for; managers rarely forecast decreasing
sales or predict that products will fail, for instance.
Managers are also not the appropriate forecasters to assess
competitive threats or the impact of new technologies that might
make their products obsolete.

MS (UNM) Judgmental Forecast 1 August 2021 17 / 27

The accuracy of judgmental forecasts (cont’d)
Non-investment type forecasts (cont’d)

Eurotunnel forecasts
The actual cost of building the Eurotunnel was more than twice the
original estimate while its intended date of opening was missed by
almost two years.
This is the usual pattern for similar forecasts of big projects, whose
costs and completion times are seriously underestimated while
potential revenues are exaggerated.

MS (UNM) Judgmental Forecast 1 August 2021 18 / 27

The accuracy of judgmental forecasts (cont’d)
Non-investment type forecasts (cont’d)

“Expert” forecasts
There is significant empirical evidence comparing “expert” forecasts
with those of statistical models.
In nearly all cases (Dawes, 1988; Hogarth and Makridakis, 1981;
Kahneman et al., 1982; Meehl, 1954) where the data can be
quantified, the predictions of the models are superior to those of the
expert.

Are there judgmental forecasts that relate to Bangladesh? Is it possible to

forecast them using econometric/statistical methods? How accurate are
these forecasts?

MS (UNM) Judgmental Forecast 1 August 2021 19 / 27

The nature of judgmental biases and limitations
Judgmental biases in forecasting

Judgmental biases do not imply stupidity, however; their presence is

clearly discernible among highly intelligent people. Rather, they result
from the way the mind operates and reflect its attempts to reconcile
conflicting objectives.
Differences in judgmental forecasting are caused by personal
considerations, springing from optimism or wishful thinking, and have
little or nothing to do with an objective assessment of future
conditions.
Inconsistency, a human bias with serious negative consequences, refers
to changing our minds (or decisions) when there is no need to do so.

MS (UNM) Judgmental Forecast 1 August 2021 20 / 27

The nature of judgmental biases and limitations
Judgmental biases in forecasting (cont’d)

Source: Makridakis et al. (1998), Chapter 10, p.495.

MS (UNM) Judgmental Forecast 1 August 2021 21 / 27

The nature of judgmental biases and limitations
Judgmental biases in forecasting (cont’d)

How will the forecasts change if people are told that the data in Figure
10-5 represent a new—or, alternatively, a mature or old—product?
The answers will vary widely, which demonstrates how often we ignore
concrete data and instead forecast using stereotypes—the sales of a
new product must increase, for example, while those of an old one
must decrease.
The differences in the forecasts for new, mature, and old products (in
particular for 2003) are enormous, highlighting the need to exclude
judgmental biases while forecasting.

MS (UNM) Judgmental Forecast 1 August 2021 22 / 27

The nature of judgmental biases and limitations (cont’d)
Dealing with judgmental biases

Inconsistency can be avoided by formalizing the decision-making

process (today this is called building expert systems).
Decision rules cannot be used indefinitely. The environment changes,
as does competition; new objectives might be set and so on. Thus,
the effectiveness of decision rules must be monitored constantly to
make sure they are still appropriate.
That means learning must be introduced into the expert system;
otherwise, we run the risk of applying obsolete rules.

MS (UNM) Judgmental Forecast 1 August 2021 23 / 27

The nature of judgmental biases and limitations (cont’d)
Conventional wisdom

Another type of judgmental bias that can threaten decision-making

effectiveness is unfounded beliefs or conventional wisdom. We believe,
for instance, that the more information we have, the more accurate
our decisions will be.
Empirical evidence does not support such a belief, however. Instead,
more information merely seems to increase our confidence that we are
right without necessarily improving the accuracy of our decisions
(Oskamp, 1965).
Another example of conventional wisdom we are willing to accept is
that we can discriminate between useful and irrelevant information.
Empirical research indicates that this is rarely the case.

MS (UNM) Judgmental Forecast 1 August 2021 24 / 27

Combining statistical and judgmental forecasts

The big challenge in arriving at accurate forecasts is to utilize the

best aspects of statistical predictions while exploiting the value of
knowledge and judgmental information, as well as capitalizing on the
experience of top and other managers.
Many firms exploit the advantages of both judgmental and statistical
forecasts while avoiding their drawbacks.

MS (UNM) Judgmental Forecast 1 August 2021 25 / 27

Combining statistical and judgmental forecasts (cont’d)

A typical example is the annual budget meeting firms hold to decide

how much sales will grow in the following year.
Good practice often involves exploiting the objectivity advantage of
statistical methods, which can best identify and extrapolate
established patterns, and drawing upon inside information and the
experience of top management about forthcoming changes and their
implications.

MS (UNM) Judgmental Forecast 1 August 2021 26 / 27

Conclusion

Judgmental forecasts are indeed indispensable, for they present the

only alternative to predicting systematic changes from established
patterns and/or existing relationships (Lawrence et al., 2006).
At the same time, we must be careful to avoid the biases and other
limitations that characterize our judgment while reducing their
negative consequences on forecasting.
The challenge for firms is to exploit both the advantages of statistical
predictions (including their low cost) and the unique ability of human
judgment to deal with systematic changes in patterns/relationships
which statistical methods cannot predict as they can only extrapolate
the continuation of such patterns/relationships.
Significant progress has been made in the last few decades with
regard to accuracy (Lawrence et al., 2006).

MS (UNM) Judgmental Forecast 1 August 2021 27 / 27