Professional Documents
Culture Documents
Predicting Demand Forecastability For Inventory Planning: C S S C & O I
Predicting Demand Forecastability For Inventory Planning: C S S C & O I
In a recent study, a research team led by Professor Arthur Hill in the Supply Chain and Operations
Department at the University of Minnesota’s Carlson School of Management determined that previously
developed measures of forecastability are not effective. They developed the new “Q-Metric” that
reliably measures the economic advantage of using a forecasting system over a simple reorder point
(pull) system. The Q-Metric is defined as the ratio of the standard deviation of demand and the
standard deviation of forecast error for double exponential smoothing (a simple forecasting method).
When the Q-Metric is greater than one, the variability of the demand is higher than that of the forecast
error for a simple forecasting model, and the potential benefit of the forecasting model is high. For
example, a Q-Metric of 110% means that the inventory carrying cost for a reorder point (pull) system
will likely be 10% more than for a forecasting-based system. Conversely, when the Q-Metric is less than
one, the variability of the demand is less than that of the forecast error and the potential benefit of the
forecasting model is low. For example, a Q-Metric of 90% means that the inventory carrying cost for a
reorder point (pull) system will be likely be 10% less than for a forecasting-based system.
Copyright © 2014 Professor Arthur V. Hill, Carlson School of Management, University of Minnesota
The researchers analyzed the Q-Metric on a database of historical demand with thousands of items from
a Fortune 100 firm. They de-seasonalized the demand for each item and divided it into an estimation
sample (the earlier half of the time series demand data) and a holdout sample (the latter half of the
demand data). Three major surprises emerged from this analysis.
Surprise 1. The first major surprise was that the Q-Metric for most items was less than one, which
means that most items were not forecastable. This was quite interesting given that the firm was
spending millions of dollars to create statistical forecasts and then even more money in having analysts
review and tamper with these forecasts.
Surprise 2. The second major surprise was that highly touted time series metrics (e.g., CV) and forecast
accuracy metrics (e.g., MAPE) were not good predictors of the Q-Metric in the holdout sample. (Note:
Many time series metrics were considered including CV, skewness, kurtosis, t-statistic for trend, first-
order sample autocorrelation, runs test statistic, Durbin Watson statistic, and approximate entropy.
Many forecast accuracy metrics were also considered including MAPE, mean absolute scaled error
(MASE), Theil’s U, percent variation explained, noise-to-signal ratio, and demand-forecast correlation.)
Surprise 3. The third major surprise was that the Q-Metric in the holdout sample could be accurately
predicted with just three variables measured in the estimation sample. The research team validated this
model for predicting the Q-Metric on four additional datasets with a wide variety of products including
batteries, hand tools, tires, and Department of Defense repair parts.
Stop believing the conventional wisdom that time series metrics such as the coefficient of variation
(CV) measure the forecastability of the demand history for an item.
Stop using flawed forecast accuracy metrics such as the mean absolute percent error (MAPE). The
MASE is better and the Q-Metric is even better.
Use the new Q-Metric to measure forecastability. The Q-Metric is directly related to the economics
of a forecasting system for inventory planning and is a good measure of the economic value of a
forecasting system vis-à-vis a reorder point (pull) system.
Use the predicted Q-Metric to triage items. For items that are not forecastable (i.e., Q < 1), stop
wasting resources developing sophisticated forecasting systems. Instead, use a reorder point (pull)
system and focus on reducing leadtimes, stabilizing demand, and improving supply chain visibility.
Investing time and money in developing forecasts for unforecastable items adds noise and nervousness
to the system and adds no value to anyone except the forecasting software vendor.
Reference: Hill, A.V., W. Zhang, and G.F. Burch (2014). “Forecasting the Forecastability Quotient for Inventory
Management,” under second review by the International Journal of Forecasting.
Contact: Arthur V. Hill, Associate Dean for MBA Programs and John and Nancy Lindahl Professor, Carlson School of
Management, University of Minnesota, Supply Chain and Operations Department, 321 19th Avenue South,
Minneapolis, MN 55455-0413. Voice 612-624-4015. Email ahill@umn.edu.
Copyright © 2014 Professor Arthur V. Hill, Carlson School of Management, University of Minnesota Page 2