Multi-disciplinary Approach to Procurement

Optimization: A U.S. Defense Case Study

October 19, 2021

Presenter Biography

Adam James is a Subject Matter Expert at Technomics, Inc.,

where he serves as a thought leader in data analytics and data
science. In this role, Adam provides his expertise to develop
modern, innovative solutions to address new – and old – defense
acquisition problems. In addition to numerous U.S. organizations,
Adam has supported several ship acquisition programs for both
the Royal Canadian Navy and the Canadian Coast Guard.
Adam earned a M.S. in Statistics from Virginia Tech in 2012. He
also has a B.S. from Virginia Tech with a double major in
Mathematics and Statistics. He received the International Cost
Estimating and Analysis Association’s (ICEAA) 2016 Technical
Achievement Award. Adam also has been active in contributing to
the community knowledge base, winning best paper in both the
Adam H. James
“Analysis & Modeling, Machine Learning” and “Methods, Data
Subject Matter Expert, Data
Analytics Collection & Management” categories at ICEAA in 2019.1 2

1 James, Adam, Jeff Cherwonik, and Brandon Bryant. Don’t Just Use Your Data… Exploit It. ICEEA Professional Development & Training
Workshop, Tampa, FL, May 14-17, 2019. Unpublished Conference Paper. Online.
2 Bazzy, Rich, Adam James, and Jeff Cherwonik. A New Approach When Cost/Capability Trades Matter Most. ICEEA Professional

Development & Training Workshop, Tampa, FL, May 14-17, 2019. Unpublished Conference Paper. Online. 2

1
 “All models are wrong, but some are useful”
– George E.P. Box

Defense Procurement

Analyze potential solutions; develop, test, and mature the

Development technology; complete and demonstrate the final product

Procure the developed product to achieve an operational

Production capability to satisfy the user and mission needs

Maintain the operational capability and assure that the user

Sustainment and mission needs continue to be met

2
 Defense Procurement

Development

Procure the developed product to achieve an operational

Production capability to satisfy the user and mission needs

Sustainment

The Problem

 We have a large widget that is replacing an existing widget for the U.S. military
 Tens of thousands of units will be purchased for multiple customer organizations
 The widget is highly configurable – there are multiple distinct versions, several
colors, and a hundred different add-ons that define any one unit
 Widgets are shipped and configured at a dozen locations each year, both inside the
U.S. and internationally
 Lead time to order widgets is nearly two years from the need-by date
 The government requirements shift inside of this two-year window
 The widget experiences engineering change orders inside of this two-year window
 Widgets are constantly rolling off the production line

The right widget must be procured at the right time and shipped to the right location within a narrow time window – of course, with
the right configuration options

3
 Simplistically…

Supply Deman
d
7

…but with constraints

Supply Deman
d
8

4
 …and constant changes

Supply Deman
d
9

Problem Specifics
Supply, Demand, and Constraints

10

10

5
 Understanding Demand

 Customer organizations are structured according to their:

 Mission
 Personnel
 Mission Equipment List
 Major equipment (e.g., planes, helicopters, trucks, boats, ships)
 Electronics (e.g., radios, radars, communications)
 Weapons (e.g., firearms, ammunition, mortars, launchers)
 Hospitality (e.g., mobile kitchens)
 Etc…
 An “Official Order” specifies which organizations will modernize their widget from the
old version to the new one, and in what year
 Widgets are provided to a customer according to the schedule specified if and only if
the widget is able to support their mission

11

11

It’s Not that Easy

 The Order states which customers but does not specify an accurate widget quantity
 The equipment lists for a given widget cannot be taken at face value
 Certain missions are misalignments for the widget due to a lack of support (e.g., an
insufficient power supply)
 Equipment list experiences “out of cycle updates” after the time which the program has to
lock in their production contract
 Customer on-the-ground realities are different from what the equipment list says they
should have, resulting in challenges when fielding the widgets
 Includes items that are not obviously tied to a widget but that the widget is still expected to
be compatible with (e.g., widget A and widget B are integrated together, but show up on
the list as separate items)

There is no “order form” from the customers to the program that succinctly expresses what to buy

12

12

6
 Compiling the Demand Data
Equipment List Customer
(Customer 1) Equipment Fielding
Provided
Compatibility Schedule
Equipment List Information
(Customer 2)

Equipment List
(Customer 3)

Equipment List
(Customer 4)  Hundreds of equipment lists are combined and
Equipment List
~ 500,000 blended with other information to determine a
(Customer 5) records profile of several thousand widgets
Equipment List  This profile includes when and where they are
(Customer 6) needed, what color they should be, what
version they should be, and what configuration
Equipment List
(Customer 7)
options they should have

Equipment List
 This process is not straightforward – there are
(Customer …) hundreds of lines of R code to prep the data with
Profile of over complex logic to determine which equipment
Equipment List
3,000 widgets profiles define a valid widget
(Customer n)

13

13

Understanding Supply

 Government purchases widgets from the manufacturer (i.e., awards a contract) well
in advance of the demand being finalized
 Government can modify the contract to make configuration changes at the factory
 Optional extras that cannot be integrated at the factory must instead be shipped to
and installed at the customer location
 The manufacturer must have a clear instruction of what to ship, where, and when

Widgets not yet started can be If changes are made too

changed, but require contract late, extras must be
mods, which take time and money shipped to the customer

not yet started in progress completed shipped

Harder
Easier
More Expensive
Less Expensive
Widgets in progress or completed can Resource
have extras integrated at the Constrained
manufacturer, post-production

14

14

7
 Programmatic Constraints

 Schedule – Widgets must not be late

 Customers must receive widgets on time
 Slightly late is okay, because the manufacturer can tweak production to make it work

 Add-ons – Often it is fine to give a customer a freebie

 Some add-ons are simply not allowed to go certain places
 Other add-ons have high visibility and best not be wasted

 Administration – It takes time and money to make changes

 Changes should be minimized, even if it results in tradeoffs elsewhere
 Management does not like making multiple changes to the same widgets
 Contracts can only be changed so often, and at certain times

15

15

Finding the Optimal Solution

The Assignment Problem

16

16

8
 The Assignment Problem Example (1)

 I am having a party and need to complete three tasks: Prepare Food, Cut the Lawn,
and Clean the House
 Three people are available to help, and each can complete each task in a different
amount of time
 Each person is only willing to complete one task

Prepare Cut Lawn Clean House

Food

Adam 5 3 3

James 10 3 5

Mike 7 2 5

Who should do what to complete the work in the least amount of total time?

17

17

The Assignment Problem Example (2)

Thinking of this as a linear optimization…

 Define a weighting matrix with each element being either 0 or 1
 Minimize the total time (the sum of each element multiplied by its weight)

Prepare Cut Lawn Clean House Prepare Cut Lawn Clean House
Food Food

Adam 5 3 3 Adam 1 0 0

James 10 3 5 James 0 0 1

Mike 7 2 5 Mike 0 1 0

𝑿 minimize 𝑐 𝑥 𝑪

5·1 3·0 ⋯ 5·0 𝟏𝟐

18

18

9
 The Assignment Problem Example (3)

Thinking of this another way…

 Rearrange the columns so that each person’s task lies along the diagonal
 Minimize the total time (the sum of the diagonal elements, or the trace)

Prepare Cut Lawn Clean House Prepare Clean House Cut Lawn
Food Food

Adam 5 3 3 Adam 5 3 3

James 10 3 5 James 10 5 3

Mike 7 2 5 Mike 7 5 2

𝑿 minimize 𝑥

5 5 2 𝟏𝟐

19

19

The Assignment Problem Example (4)

Thinking of this in yet another way as a bipartite graph…

5
Adam Food Adam 5 Food
3
3
10

James Lawn James Lawn

3
5
7 5
2
2
Mike Clean Mike Clean
5

5 5 2 𝟏𝟐

20

20

10
 Applying Additional Information

 Suppose Adam can prepare the food very quickly – but is a bad cook
 Likewise, Mike’s house cleaning is slightly less thorough than James’
Prepare Cut Lawn Clean House Prepare Cut Lawn Clean House
Food Food

Adam 5 3 3 Adam 100 0 0

𝑿𝑡𝑖𝑚𝑒 James 10 3 5 James 0 0 0 𝑿𝑝𝑒𝑛𝑎𝑙𝑡𝑦

Mike 7 2 5 Mike 0 0 1

Prepare Cut Lawn Clean House

Food

Adam 105 3 3

𝑿𝑡𝑖𝑚𝑒 𝑿𝑝𝑒𝑛𝑎𝑙𝑡𝑦 James 10 3 5 7 3 3 𝟏𝟑

Mike 7 2 6

21

21

Linear Sum Assignment Problem

 This particular problem is known as a Linear Sum Assignment Problem (LSAP)

 In our example we have an unbalanced problem (the number of widgets that form
the supply is greater than the number of widgets that we need to assign)

 Can be solved using the Hungarian method in polynomial time

 Runs efficiently in 𝑂 𝑛 time
 Not trivial to implement by hand, often implemented as a graph
 Fortunately, convenient implementations already exist

 Utilized the function solve_LSAP() from the package clue in R

Hornik K (2021). clue: Cluster ensembles. R package version 0.3-59, https://CRAN.R-project.org/package=clue.

Hornik K (2005). “A CLUE for CLUster Ensembles.” Journal of Statistical Software, *14*(12). doi: 10.18637/jss.v014.i12 (URL:
https://doi.org/10.18637/jss.v014.i12)

22

22

11
 Optimizing Supply and Demand
Customer Needs
𝑗 1, … , 𝑚

Widgets available for

Assignment “cost” for widget 𝑖 to
meet need 𝑗
𝑖 1, … , 𝑛

𝑐𝑜𝑠𝑡 𝑐𝑜𝑠𝑡 𝑐𝑜𝑠𝑡 𝑐𝑜𝑠𝑡 𝑐𝑜𝑠𝑡

The “cost” is both literal dollars and intrinsic penalties to meet the demands of
reality
23

23

Our Specific Assignment Problem

Customer Needs
𝑗 1, … , 𝑚

0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
Widgets available for
Assignment 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0

𝑖 1, … , 𝑛 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Minimize the total cost by assigning a widget to a need

Preserve schedule (as a top priority)
Minimize changes to contracts and shipping plans
24

24

12
 The Model

 A Linear Sum Assignment Problem (LSAP) is set up as a giant matrix in R

 Over 3,000 rows make up the demand
 Over 15,000 columns make up the supply
 Each cell indicates the “cost” (including penalties) for a widget to meet a given need

 Assign each “need” a widget so that the total cost is minimized

 Constrained such that changes to the contracts are minimal
 Constrained such that every customer gets their full request, as close to schedule as
possible

25

25

Automation

 Each step in the process is automated

 Data management to optimization model to output reports
 Enables same day turnaround of analysis updates to reflect new inputs and changes to
the source data

 Everything is a variable or a function

 Model can be changed and updated rapidly
 Unanticipated updates can be made without substantial rework

 Powerful automation can very quickly evaluate a large amount of information to

support decisions such as the most effective way to replace a broken widget, or
how to react to ever-changing requirements from the government such as widget
demand or fielding site locations

26

26

13
 The Output

 Output from this model serves as the direct inputs to the attachments and exhibits
put on contract for the manufacturer to use when producing and shipping widgets

 Output from this model also serves as instruction to the fielding sites for what
widgets should be configured and integrated with what equipment

 Changes to configurations and other systems engineering issues can be viewed not
just through the technical lens, but from the all-important cost perspective critical to
most decision making

27

27

Questions

28

28

14