Forecasting Demand – Judgmental Methods

Anton J. Kleywegt, Ph.D.

School of Industrial and Systems Engineering
Review of Previous Lessons

• Careful thought is required when modeling

demand
• Customer behavior often is not rational
• Customer behavior exhibits predictable
patterns even when it is not rational
Learning Outcomes
• Become familiar with some characteristics of the most
widely used method of forecasting: judgmental
methods
Forecasting Methods
• Classification of forecasting methods according to Armstrong, J.S. Long-range
Forecasting, Second Edition, 1985
Judgmental Forecasting Methods

• As the name indicates, judgmental forecasting methods are driven mostly by human
judgment, typically the judgment of experts
• By far the most widely used forecasting approach in practice
• Examples
• Sales managers forecasting future sales or demand
• Medical doctors diagnosing causes of medical problems
• Medical doctors predicting prognoses for patients
• Pundits forecasting outcomes of elections
• Brokers recommending investments
• College admissions staff forecasting academic success
Judgmental Forecasting Methods

• “Expert” opinion often produces poor forecasts

• Quality of forecasts tends to increase and then
decrease as the level of expertise increases
• See the following articles:
• Armstrong, J.S., “How Expert Are the Experts?”,
Inc, pp.15-16, 1981
• Armstrong, J.S., “The Seer-Sucker Theory: The
Value of Experts in Forecasting”, Technology
Review, pp.16-24, 1980
Judgmental Forecasting Methods

• Proposed explanations for poor forecasting

performance of experts
• Experts think they know better, and do not learn
as well from data as more humble forecasters
• Experts are overconfident, overestimate their
knowledge and forecasting skills, and
underestimate their forecasting errors
Judgmental Forecasting Methods
• Overconfidence are positively correlated with various
factors:
• The higher the publically perceived level of
expertise, the greater the tendency to be
overconfident
• Most managers and companies perform
poorly after their praises have been sung
in the popular media
• Past success greatly increases overconfidence
• If a person is successful, it is usually
ascribed to some inherent talent, and if
a person is unsuccessful, it is usually
ascribed to external influences or bad
luck
Judgmental Forecasting Methods
• Overconfidence are positively correlated with various
factors (continued):
• Expectations of skill is positively correlated with
overconfidence
• The higher a manager’s level in the
organization, the more the tendency to
be overconfident (note that either may
be the cause of the other)
• Gender: men have a greater tendency to be
overconfident than women
• Do you think that you are a better than
average driver?
Judgmental Forecasting Methods

• Organizations continue to use experts for their

forecasting even if they know that experts do not
produce better quality forecasts
• Why?
• The manager knows that the forecast is always
wrong
Judgmental Forecasting Methods

• Judgmental forecasting methods perform poorly

compared with data-driven forecasting methods
• Even very simple, crude regression models produce
better forecasts than experts
• See the following articles:
• Dana, J. and Dawes, R.M., "The Superiority of
Simple Alternatives to Regression for Social
Science Predictions", Journal of Educational and
Behavioral Statistics, pp.317-331, 2004
• Dawes, R.M., "The Robust Beauty of Improper
Linear Models in Decision Making", American
Psychologist, pp.571-582, 1979
Judgmental Forecasting Methods
• Judgmental forecasting methods perform poorly compared with data-driven
forecasting methods
• See the following articles (continued):
• Kahneman, D. and Tversky, A., "On the Psychology of Prediction“, Psychological
Review, pp.237-251, 1973
• Lovallo, D. and Kahneman, D., "Delusions of Success", Harvard Business Review,
pp.56-63, 2003
• Overconfidence inhibits use of the “outside view”
• Example of “outside view”:
• Most entrepreneurs who start a new business forecast that it will be
successful with high probability, but fewer than 20% of startups
survive more than a few years (forbes.com)
Judgmental Forecasting Methods

• Judgmental forecasting methods perform poorly

compared with data-driven forecasting methods
• See the following articles (continued):
• Meehl, P.E., "Causes and Effects of My Disturbing
Little Book", Journal of Personality Assessment,
pp.370-375, 1986
• Sarbin, T.R., "A Contribution to the Study of
Actuarial and Individual Methods of Prediction",
American Journal of Sociology, pp.593-602, 1943
• Sarbin, T.R., "Prediction and Clinical Inference:
Forty Years Later", Journal of Personality
Assessment, pp.362-369, 1986
Summary
• Judgmental forecasting methods are driven mostly by
the opinions of experts
• By far the most widely used forecasting approach in
practice
• Judgmental forecasting methods almost always
perform poorly compared with data-driven forecasting
methods
• It is important not to be overconfident, and to take
into account the outside view