Who Needs a Good

Forecast Anyways?
AGIFORS RM Study Group

Ross Winegar, PhD

Tom Gorin PhD
PROS Data Science

Helsinki, Finland
June 6-7, 2023

Copyright © by PROS.
Published by AGIFORS with permission.
Abstract

Who spends all day worried about forecasts? We’re all here to make
money. Maybe that bad forecast is good? What would happen if
forecasts were perfect? What does it even mean to have a perfect
forecast? Since users intentionally hurt forecast accuracy all the time
for ulterior motives, could I help them be more effective in that? Or
should I just ditch the forecast altogether?

In this presentation we will briefly review a quick history of airline

forecasting and explore what makes a good forecast and how to
evaluate the forecast efficacy.

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

Bios

ROSS WINEGAR TOM GORIN

Ph.D. in Economics from Purdue
University
Ph.D. in Transportation, MIT

10 years at PROS in the Science and 9 years at PROS

Research department developing data
Continental Airlines, BCG, CVS Health
science models for airline revenue
management.

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

A few questions to get us started…

Do you worry about forecast Which of these outcomes would

accuracy? you select?
10
Do you spend time thinking about it? 9
8
How do you measure accuracy? 7
6
What does it even tell you?
5
4
How do you use your internal forecasts (RM only,
3
other uses for your RM forecasts)?
2

What is the primary use for your forecasts? 1

0
Actual demand Case1 Case 2 Case 3

Q - Bottom class M - Class 3 B - Class 2 Y - Top class

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary
What do you look for in accurate forecasts?

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

What do you miss in forecasting?

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

Please say whether your overall view of your forecaster is positive or very positive

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

Ask ChatGPT: Summarize the history of forecasting in airline
revenue management in one slide
Title: History of Forecasting in Airline Revenue Management
• Early Approaches (1960s-1970s): Basic statistical methods and historical data used for aggregate demand forecasting.
• Yield Management Systems (1980s-1990s): Advanced statistical techniques employed (moving averages, exponential smoothing,
linear regression) to optimize pricing decisions.
• Introduction of Network Revenue Management (1990s-2000s): Focus on managing inventory and pricing across multiple flights
and routes; time-series analysis, ARIMA, and econometric models used.
• Demand-Driven Forecasting (2000s-2010s): Real-time data, predictive analytics, and machine learning algorithms applied;
inclusion of social media sentiment, competitor pricing, and economic indicators.
• Big Data and Advanced Analytics (2010s-Present): Leveraging big data, AI-driven algorithms, and ensemble models; optimizing
pricing, inventory control, and revenue management decisions.
• Continuous Innovation: Ongoing exploration of real-time data, customer behavior analysis, and dynamic pricing strategies.

Conclusion: From basic methods to sophisticated analytics and AI, forecasting in airline revenue management has evolved, enabling
data-driven decisions and revenue maximization.

Note: Please note that the slide is a condensed summary and may require further elaboration during a presentation or discussion.

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

History of Forecasting in Airline Revenue Management

Introduction of
Yield Big Data and
Early Network Demand-Driven
Management Advanced Continuous
Approaches Revenue Forecasting
Systems Analytics Innovation
1960s-1970s Management 2000s-2010s
1980s-1990s 2010s-Present
1990s-2000s

Conclusion: From basic methods to sophisticated analytics and AI, forecasting in airline
revenue management has evolved, enabling data-driven decisions and revenue maximization.

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

What’s a good forecast

The definition of a good forecast is one that makes money.

Could a better forecast make more money?

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

Forecasts are Wrong and have always been Wrong

• Number 1 rule of forecasting: Forecasts are always wrong.

• Not often does an airline accept 2.4856 passengers in Y class.

• What’s the revenue impact compared to an integer forecast of 2 or 3?

• Lots of focus on metrics. Suppose there were 2 actual bookings.

• Forecast 2.2 → RMSE/MAE = 0.2 and MAPE = 10%
• Forecast 2.4 → RMSE/MAE = 0.4 and MAPE = 20%

• The forecast of 2.4 is 100% worse than 2.2 as measured by

RMSE/MAE/MAPE

• The world has ended…

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

What’s the Revenue Impact?
Remaining Capacity: 20 seats 4 seats 3 seats 2 seats

Class Fare Forecast

Bid Price = $0 Bid Price = $800 Bid Price = $800 Bid Price = $1000
Y $1000 2
LCA = H LCA = B LCA = B LCA = Y
B $800 4
H $400 6

Class Fare Forecast

Bid Price = $0 Bid Price = $800 Bid Price = $800 Bid Price = $1000
Y $1000 2.4 LCA = H LCA = B LCA = B LCA = Y
B $800 4
H $400 6

Class Fare Forecast Bid Price = $0 Bid Price = $800 Bid Price = $1000 Bid Price = $1000
Y $1000 3 LCA = H LCA = B LCA = Y LCA = Y
B $800 4
H $400 6
Bid price vector only changes when 3 units of remaining capacity
Changing forecasts to fractional values has no impact
©2023 PROS, Inc. All rights reserved. Confidential and Proprietary
 Research or Practice

90% of RM research is optimization while 90% of RM practice is scrutinizing forecasts

Is the Model Right? Is Every Possible Plug’n Play Python Fun

Variable in Here?

• Forecasting yieldable bookings when • Customer demand goes beyond historical • Fundamental human desire to test
customers buy-down across fare classes bookings and prices/availability. every model on the internet
• Customer choice across itineraries and • Data processing can make or break a model
offer sets

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

What’s important here?

Hit the big HSE like Thanksgiving and Christmas

Bookings can jump up to 25% month-to-month from seasonality

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

What if forecasts were perfect (with some variability)?

Remember this example?

Perfect 20% over in 20% under in
forecast all classes all classes
10
Do you actually want a “perfect”
forecast?
9
8
Is “intelligent aggressiveness” a good
7
thing?
6
5 How do your RM teams manage
4 flights? Do they influence the
3 forecasts, and if so, how, and why?
2
1
What’s the “right” outcome to look for?
0
Actual demand Case1 Case 2 Case 3

Q - Bottom class M - Class 3 B - Class 2 Y - Top class

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary
 Why would someone tank their forecast accuracy?

• Inducing demand
• The RM system is a means to an end
• How can I affect the ends (bid price)
without going straight to it.

• Knowing better than the system

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

 Unintended Consequences

Unintentional Damage the

exploration optimization

Buying Improved Windshield

observations unconstraining Spiral down wiper
the naiive for closed controls
way classes

Training
Random analysts in a
experiment bad way

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

 Forecasts will never be perfect. Deal with it.

Ditch the Forecast AI Points Where to Go Focus on Pricing

Product features and price paid per remaining NLP intakes analyst questions and evaluates Focus more on the pricing elasticity
capacity directly estimate opportunity cost of not forecasts for diagnostics problem and less on bid price capacity
selling seat costs

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

Have a Nice Day

©2023 PROS, Inc. All rights reserved. Confidential and Proprietary

pros.com