›  Conclusions Paper

How Analytics Improves Decision Making at the

Department of Defense
Finding new ways to add value and insights to big data
Contents
Introduction....................................................................... 1

Big Data: The DOD’s Big Challenge ........................... 1

Vehicles that Tell You When They
Need Repair........................................................................... 1

Getting Soldiers What They Need,

When They Need It.............................................................. 2

Improving Transparency in Strategic Sourcing ............. 2

Taking Analytics to a New Level: Learning

from the Commercial Sector ........................................ 3

Impediments to Using All the Data.............................. 4

Training Leaders to Be ‘Complete’ Analysts............... 4

Final Thoughts.................................................................. 5

About the Contributors.................................................. 5

Learn More........................................................................ 5

Appendix........................................................................... 6

Based on the panel discussion Data Versus Gut: Using Analytics to Improve Decision Making
held at the 2013 Annual Meeting of the Association of the United States Army

Featuring
Retired Army Lt. Gen. Mitchell Stevenson, former Deputy Chief of Staff, Logistics, G4
Karen Terrell, Vice President of SAS Federal
Mark Cramer, President of Institute for Defense & Business (Moderator)
Vijay Khatri, Associate Professor at Kelley School of Business, Indiana University Bloomington

Introduction
The military prides itself in its decision making. And while the basic
tenets of that decision making have remained the same since
WWII, one thing has changed: the amount of data the military has
to work with. Today, the military receives a flood of data through
vehicle diagnostic systems, surveillance systems, unmanned aerial
systems and more.
The military’s enterprise resource planning (ERP) systems produce
even more data. While ERP systems do a good job of streamlining
business processes, the ever-increasing volume of data they kick
out is overwhelming.
How do you make use of all this data? How do you turn it into
useful information? Analytics – the techniques used for analyzing
large data sets to develop insights for decision making – offers
some solutions. But how do you capture data in a format that
makes it useful for analytics? These are some of the challenges
the military is looking to address.
Big Data: The DOD’s Big
Challenge
According to retired Army Lt. Gen. Mitchell Stevenson, data
collected by the Department of Defense (DOD) has lots of poten-
tial. But one of the challenges is the vast amount of data available.
“We don’t know exactly what data we want, so we’re collecting it all
– just reams and reams of data,” he said.
“We don’t know exactly what data we
want, so we’re collecting it all – just
reams and reams of data.”
– Retired Army Lt. Gen. Mitchell Stevenson,
former US Army Deputy Chief of Staff,
Logistics, G4
What he is referring to is big data – data sets so large and complex
they can’t be processed by traditional applications. “What’s
changed for us military guys is now we have tools to analyze that
data,” he said. Many of those tools are produced by SAS.
Stevenson points out three areas where the Army is leading the
way in analytics: vehicle maintenance and tracking, supply chain
management and logistics.
1
Vehicles that Tell You When They
Need Repair
A program called condition-based maintenance (CBM) is
changing the way the Army tracks and maintains its ground
vehicles. In the past, maintenance and repairs were done on a
schedule – whether the equipment needed it or not – or when
something broke.
CBM relies on sensors located throughout a vehicle to collect data
on everything from engine diagnostics to fuel consumption and
tire pressure. By running this data through an algorithm, CBM can
tell a soldier when something on his equipment is failing. As a
result, vehicles can be repaired before an expensive catastrophic
failure occurs. Stevenson likens CBM to aspects of General Motor’s
OnStar program in the commercial world.
Embedded technology, such as that used in CBM, has other
applications, too. One example: pulling usage information from
a vehicle. The Army is testing such a program now.
“The way it is now, when you finish operating your truck or tank,
you have to manually fill out a form and present it to a clerk who
works in the motor pool of your organization,” explained
Stevenson. “The clerk then reads the handwritten information and
enters it into an online maintenance tracking system. As a result,
our data isn’t as accurate as we’d like, so we don’t have good infor-
mation about usage by fleet.”
Stevenson says vehicle prognostics and sensors eventually will
take the soldier out of the data-gathering loop, eliminating oppor-
tunities for human error. “Eventually, we’re going to have data we
can rely on, fed off the odometer, through the data bus, back to us,
all untouched by human hands,” he said. “We’ll have much, much
better fleet management capability.”
“Eventually, we’re going to have
data that we can rely on, fed off the
odometer, through the data bus, back
to us, all untouched by human hands.
We’ll have much, much better fleet
management capability.”
– Retired Army Lt. Gen. Mitchell Stevenson,
former US Army Deputy Chief of Staff,
Logistics, G4
 2
Right now, the Army is testing its vehicle usage program on 500
trucks. Funding the expansion of the project is a challenge, says
Stevenson. But the Army is trying to reduce its number of tactical
trucks from 250,000 to 170,000, which would significantly reduce
costs. Eventually new vehicle acquisition programs, such as the
Joint Light Tactical Vehicle Project, will ensure vehicles come
prebuilt with these data collection devices already on them.
The Army also has started to embed sensor technology on all of its
helicopters. Stevenson thinks in another couple of years the Army
will have 100 percent of its helicopters outfitted with mission-
based sensors.
Getting Soldiers What They Need,
When They Need It
The US Army has a long history of using analytics to inform
decision making. As early as 1995, a program called Velocity
Management (VM) used analytics to reshape Army logistics.
Prior to VM, the Army kept mass stockpiles of supplies – spare
parts, fuel tanks, extra vehicles and ammunition – on hand in case
it needed anything. But a mass-based system is not a quick and
reliable one. In early 1990, it took a month (on average) for a
military mechanic to receive an ordered part if it was not available
at his installation.
Mass stockpiling also requires a lot of manpower to manage. In his
book Moving Mountains: Lessons in Leadership and Logistics from
the Gulf War, retired General William Gus Pagonis admits at one
point during Operation Desert Storm, 28,000 of the 41,000
shipping containers that arrived in the desert had to be opened to
determine their contents because nobody knew what was in them.
As part of VM, RF tags and scanners are now used to inventory
what is in containers. VM reduced the Army’s footprint and
resulted in a leaner, faster and more reliable logistics system.
“We collected all our data and subjected it to SAS® tools and
algorithms. Velocity Management has been a huge success,” said
Stevenson. “We cut our shipping time in half, in many cases better
than that, particularly in our conflicts in Iraq and Afghanistan.”
Improving Transparency in Strategic Sourcing
When it comes to cutting supply costs, big data offers big
opportunities. According to a July 2013 McKinsey Global Institute
report, by making the supply chain more transparent, big data
has the potential to reduce federal procurement costs by up to
$140 billion annually by 2020.1
Stevenson points out one instance where analytics brought clearer
insight in the Army’s supply chain. “We began looking at what we
were spending money on and with whom. In one case, we thought
we were buying tires from a single source, but found within the
Defense Logistics Agency, we were writing contracts with 300
different contractors for tires. We had no idea we were doing this,
but analytics helped. Analytics points to areas where we can
improve, do better,” he said.
“In one case, we thought we were buying
tires from a single source, but found
within the Defense Logistics Agency, we
were writing contracts with 300 different
contractors for tires. We had no idea we
were doing this, but analytics helped.
Analytics points to areas where we can
improve, do better.”
– Retired Army Lt. Gen. Mitchell Stevenson,
former US Army Deputy Chief of Staff,
Logistics, G4
SAS is also helping the DOD identify high-risk procurements like
counterfeit products and fraud. Using SAS Analytics, contracting
officials can rapidly identify vendors barred from doing business
with the government – and avoid mistakenly issuing contracts to
them. Analytics can also detect if a supplier’s street address is
associated with prior fraud elsewhere. Or it can tell if a name or
address is linked with contracts for duplicate services. By identi-
fying procurement risks early, the DOD potentially can save
millions of dollars each year.
1
Game Changers: Five Opportunities for US Growth and Renewal. McKinsey Global
Institute Report. mckinsey.com/insights/americas/us_game_changers. Page 83.

“SAS is working on a project with the DOD, and very quickly
applying analytics to high-volume data to determine whether or
not a procurement is risky and needs further investigation,”
explained Karen Terrell, Vice President of SAS Federal.
“SAS is working on a project with the
DOD, and very quickly applying
analytics to high-volume data to deter-
mine whether or not a procurement is
risky and needs further investigation.”
– Karen Terrell, Vice President of SAS Federal
Taking Analytics to a
New Level: Learning from
the Commercial Sector
The military was a first mover in analytics, but the commercial
sector has since embraced the technology – taking it to inspiring
new heights that have captured the military’s attention. Many
businesses now embed analytics directly into their operational
processes, explained Terrell. The results are lightning-fast
answers, tremendous cost savings, and significantly better
protection against fraud. For example:
• A major department store chain uses analytics to speed and
considerably slash the cost of routine pricing and discounting
activities.
• A global leader in precision marketing uses analytics to generate
relevant point-of-sale coupons for its customers, drastically boost-
ing coupon redemption rates.
• Banks use analytics to spot unusual behavior and detect potential
credit card fraud in real time.
3
Pricing and Discounting
A major department store chain wanted to sell
off as much seasonal merchandise as possible
at the right price. In the past, to figure out what
products to discount for the week, the retailer
ran a batch process that took up to 30 hours.
With SAS Analytics, the retailer now runs that
same process in less than two hours, repricing
270 million SKUs at 850 stores each week. The
software analyzes previous sales to forecast
how long an item will be on the shelf. As a
result, the store saves millions of dollars
annually in markdowns.
Targeted Advertising
Few companies handle more data than this
global leader in precision marketing. When a
consumer checks out with a loyalty card at one
of 50,000 grocery or drug stores in the US, UK
or Japan, insights derived from this company’s
database trigger point-of-sale coupons. Using
SAS solutions to look at historical purchase
behavior, this marketing giant increased
coupon redemption rates for its customers
from 1 percent to 15 percent.
Pinpointing Fraud
Banks use predictive analytics to flag potential
card fraud faster and more accurately. Three of
the four biggest banks in the US rely on fraud
management solutions from SAS. When a bank
customer scans a debit or credit card at the point
of sale, card information is sent to a processing
facility in North Carolina. There, the system looks
at the card user’s previous buying behavior and
patterns to determine – in real time – whether to
approve the purchase.
4

Impediments to Using
All the Data
Advocates of using analytics in the military say it has big
potential, especially on the front line. But the issue is how
to make use of all the data that is collected. A September
2013 report by the Royal United Services Institute (RUSI)
cites a US Army Intelligence officer (unnamed) who said as
much as 95 percent of battlefield video data is never
viewed by analysts, let alone assessed.2
Stevenson says the number is probably right. “It’s sort of
like the Army’s current approach to CBM – just collect it all,
maybe we’ll find some gems,” he said. The challenge with
this approach is data has to flow through some pretty
restricted pipes in the military. “It’s not like in the US where
you have good bandwidth and good networks to move
big chunks of data around. We’re talking about the battle-
field – sending it up and back through a satellite downlink,”
he said.
Another challenge of using big data, Stevenson points
out, is insufficient training. Military leaders need a better
all-around understanding of how data is collected, how it’s
stored and how they can use analytics to draw insights for
decision making.
“We’ve done some work with the RAND Arroyo Corporation
in the past. (Arroyo is the Army’s federally funded R&D
center for analysis.) They have SAS tools, and in fact, they
taught us how to use them. But the bigger challenge is
getting our leaders to understand the essential elements of
big data and analytics,” he said.
Stevenson feels analytics is something that should be taught
at every level of leadership. “The Army needs to strike a
balance between teaching these subjects and getting
soldiers out to the field. Everybody’s got bright ideas for
things we should train in Army schools, but analytics is
certainly one that has to be included,” he said.
Institute for Defense & Business:
Training Leaders in Analytics
Recognizing how crucial analytics is to the
military, the Institute for Defense & Business (IDB)
offers courses relevant to military officers as well
as to civilians working in government agencies
and the private sector. The IDB’s executive
education courses are geared to help logisticians
identify the importance of analytics and how vital
appropriate data collection is to their results.
Training Leaders to Be
‘Complete’ Analysts
According to Khatri, who teaches an analytics program at Indiana
University Bloomington, the military’s struggle with big data is not
unique. He says many companies collect huge amounts of data
– but not all of the data they collect is in a format they can use.
That data might be useful for day-to-day business, but not for
running analyses.
“Even if your data is useful from an analytics perspective, it might
not be of adequate quality. You might have missing or incorrect
values. You might have inaccurate or redundant data. You might
have a lot of things wrong with the data that prevent you from
using it to produce insights,” he said.
“Even if your data is useful from an
analytics perspective ... You might
have a lot of things wrong with the
data that prevent you from using it to
produce insights.”
– Vijay Khatri, Associate Professor at Kelley
School of Business, Indiana University
Bloomington

2
Big Data for Defence and Security. Royal United Services Institute.
rusi.org/downloads/assets/RUSI_BIGDATA_Report_2013.pdf. Page 2

Khatri suggests proper training to overcome some of these
barriers. According to him, leaders need to become “complete
analysts.” By that, he means leaders who understand:
• The conceptual aspects of data and how it is stored. How do you
profile data proactively? How do you gauge the quality of the
data? How do you understand the meaning of the data elements,
so you can integrate data from different sources?
• How to use data to produce insights. “You need to know how to
import the data and how to apply the analytic tools to produce
meaningful insights. You also need to understand how to interpret
the results,” he said.
• How to apply those insights to decision making. “It has to make
an impact in a business or military scenario. That impact could be
an increase in revenue. It could be to decrease cost. It could be a
better way to manage risk,” he added.
Final Thoughts
With future budget cuts looming, analytics offers new ways for
the military to save money and operate more efficiently. While
the military is applying analytics to vehicle maintenance and
logistics, the commercial sector has demonstrated what is
possible in terms of an even broader application of the tech-
nology. Additional, targeted training promises to give military
leaders a greater understanding of analytics – so they can
benefit from its full potential.
About the Contributors
Retired Army Lt. Gen. Mitchell Stevenson
US Army Deputy Chief of Staff, Logistics, G4
Stevenson served for more than 37 years in the Army. His last
active duty assignment was Deputy Chief of Staff, Logistics,
Headquarters, Department of the Army. During that time, he
was the principal logistics advisor to the Army’s Chief of Staff
and responsible for sustainment of the Army to the Assistant
Secretary for Acquisition, Logistics and Technology. Stevenson
is an IDB Executive Fellow Emeritus from the Class of 2012-13.
He is currently a Senior Account Manager at Leidos Logistics
Solutions and widely recognized for his experience and
expertise in logistics.
5
Karen Terrell
Vice President and General Manager of SAS Federal
At SAS, Terrell leads a $100 million business unit that applies SAS
analytics tools to deliver tailored solutions to various business and
mission issues the federal government faces. Her areas of focus
include supply chain management, logistics, fraud, waste and
improper payments.
Mark Cramer
President of Institute for Defense & Business and
Executive Director, LOGTECH
Cramer has a 35-year career as an attorney, businessman, senior
government official and chief executive of nonprofit organizations.
He helped found the Institute for Defense & Business in 1997 and
has served as the organization’s president since 2007.
Vijay Khatri
Associate Professor at Kelley School of Business,
Indiana University Bloomington
Vijay Khatri has a PhD in management information systems from
the University of Arizona. Prior to his graduate work, he worked for
Infosys Technologies and IBM Consulting. His research interests
center on data management and technology adoption. His
writings on data management and business analytics are widely
published in internationally recognized journals. Khatri is also the
co-director of the Institute for Business Analytics at the Kelley
School of Business.
Learn More
Read about the Indiana University Kelley School of
Business Analytics Certificate Program:
kelley.iu.edu/EDP/Certificate/BACP/page20960.html
Read about educational programs at the Institute for Defense &
Business, such as executive education courses that help
logisticians identify the importance of analytics and how proper
data collection can affect results:
idb.org/programs
Download this Harvard Business Review white paper from SAS:
The Promise and Challenge of Big Data: sas.com/en_us/white-
papers/promise-challenge-bigdata-106222.html
 6
Appendix
Big data raises many complex issues for the military. Here are
some questions Stevenson addressed to delve deeper into
some of the approaches being used today.
Question: Analytics relies on data fidelity; ERP systems
rely on users to input data. Yet users don’t always have
the proper discipline to input data correctly. How can
we improve data fidelity so it’s useful for analytics?
Answer: Clearly, this is an area we must pay attention to in an
ERP environment. One way to help the user is to build checks
into the data input GUI and flag the user when he or she enters
data incorrectly; for example, entering letters when numbers are
expected. Business intelligence software can tell us when data
input is outside the norm. (We do this today at US Army Logistics
Support Activity, with Army maintenance management system
and usage data from the field). But, at the end of the day, the old
adage – garbage in, garbage out – holds true. We can have the
best property accountability management information system
(MIS) in the world, but if the property book officer is entering
incorrect data, and our checks don’t catch it, we end up with an
inaccurate picture. For this reason, we will always need to do
things like joint change-of-command inventories with both the
outgoing and incoming accountable officers to ensure that
what’s in our MIS matches ground truth.
Question: What is the importance of logistics support
analysis (LSA) and how can we use it to drive down
product lifecycle costs?
Answer: LSA is a disciplined methodology for figuring out
operational support requirements during the acquisition life-
cycle process. If done right, an LSA tells us where – at what
echelon – to perform maintenance actions. During Operation
Iraqi Freedom and Operation Enduring Freedom, we needed to
get end items to the armed forces in Iraq and Afghanistan as
quickly as possible. To do it, we sometimes had to bypass the
LSA process. Instead, we had the OEM handle all the mainte-
nance requirements. This forced us to put expensive contractor
field technicians (CFTs) in the battle space, where they often
performed simple maintenance tasks that our uniformed
mechanics had the training to do. Or, worse, when an item broke
in the deployed area of operations, we simply replaced the item
and evacuated the broken one back to the manufacturer. These
strategies worked, but they were expensive. Also, had the nature
of the battlefield been different, CFTs may not have been willing
to deploy there.
Question: What types of analytics tools do soldiers on
the front line require?
Answer: They need user-friendly, intuitive business intelligence
tools that are as easy to use as Microsoft Office products.
Soldiers should be able to use these without any training at all.
Several commercial products like this are available – and that is
what the Army intends to deliver.
To contact your local SAS office, please visit: sas.com/offices

SAS and all other SAS Institute Inc. product or service names are registered trademarks or
trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration.
Other brand and product names are trademarks of their respective companies. Copyright ©
2014, SAS Institute Inc. All rights reserved. 107024_S114175.0314