THE JOURNAL OF AACE® INTERNATIONAL -

COST
TM
THE AUTHORITY FOR TOTAL COST MANAGEMENT

November/December 2012

ENGINEERING www.aacei.org

INTEGRATED
COST-SCHEDULE
RISK ANALYSIS
ESTIMATE ACCURACY:
DEALING WITH REALITY
QUANTIFYING ESTIMATE ACCURACY
AND PRECISION FOR THE
PROCESS INDUSTRIES:
A REVIEW OF INDUSTRY DATA
COMMON ERRORS IN
DEALING WITH PROJECT RISK
 CONTENTS

COST ENGINEERING

TECHNICAL ARTICLES
        5    Integrated Cost-Schedule Risk Analysis
Dr. David T. Hulett and Michael R. Nosbisch, CCC PSP

      17    Estimate Accuracy: Dealing With Reality

John K. Hollmann, PE CCE CEP

      28    Quantifying Estimate Accuracy and

Precision for the Process Industries:
A Review of Industry Data
Alexander Ogilvie, Robert A. Brown Jr., Fredrick P. Biery and Paul Barshop

      39    Common Errors in Dealing With Project Risk

Joseph A. Lukas, PE CCE

ALSO FEATURED
                   2        AACE International Board of Directors                 45        2012-2013 Nominating Committee
                   2        Cost Engineering Journal Information                  46        The AACE International Online Store
                 16        2012 Statement of Ownership 49        Professional Services Directory
38        2013 Constitution and Bylaws 49        Index to Advertisers
Amendments                  52        Calendar of Events

THE AACE INTERNATIONAL ONLINE BUTTON  - This edition content online associated with that article or feature. Direct
of the Cost Engineering journal has access to additional ma- your browser to www.aacei.org and look for the online but-
terial on the AACE International website, www.aacei.org. ton to access additional resources. If you are already reading
Anytime you see the symbol at center, there is additional the electronic version, just click the online button directly.

COST ENGINEERING  NOVEMBER/DECEMBER 2012 1


AACE INTERNATIONAL
BOARD OF DIRECTORS
PRESIDENT
Marlene Hyde, CCE EVP
303.940.3200 / president@aacei.org
PRESIDENT-ELECT
John J. Ciccarelli, PE CCE PSP
303.940.3200 / preselect@aacei.org
PAST PRESIDENT
Michael R. Nosbisch, CCC PSP
562.733.2472 / pastpres@aacei.org
VICE PRESIDENT-ADMINISTRATION
Martin Darley, CCC FRICS
713.372.2426 / vpadmin@aacei.org
VICE PRESIDENT-FINANCE
John C. Livengood, CFCC PSP
202.669.1360 /vpfinance@aacei.org
VICE PRESIDENT-TEC
George Whyte, CCC CEP EVP
301.957.7434 /vptec@aacei.org
VICE PRESIDENT-REGIONS
Julie Owen, CCC PSP
213.922.7313 / vpregions@aacei.org
DIRECTOR-REGION 1
Matthew Nicholas, PSP
416.498.1303 / dirregion1@aacei.org
DIRECTOR-REGION 2
Maria Cristina Baltazar, PSP
410.276.0076 / dirregion2@aacei.org
DIRECTOR-REGION 3
James H. Carson, CCC CEP
770.444.9799 / dirregion3@aacei.org
DIRECTOR-REGION 4
Logan Anjaneyulu, CCC CEP PSP
949.608.7371/ dirregion4@aacei.org
DIRECTOR-REGION 5
Earl J. Seabrook, III, CCC
713.372.0521 / dirregion5@aacei.org
DIRECTOR-REGION 6
Nicholas Kellar, CCC EVP PSP
dirregion6@aacei.org
DIRECTOR-REGION 7
Gulam Farooq Quadir, CCE
+966.50.3860575 /
dirregion7@aacei.org
DIRECTOR-REGION 8
Keith Webb
+61.8.93485045 / dirregion8@aacei.org
DIRECTOR-REGION 9
Jim Perry
+33.14.7664729 / dirregion9@aacei.org
EXECUTIVE DIRECTOR
Dennis G. Stork
304.296.8444 / dstork@aacei.org
2 COST ENGINEERING NOVEMBER/DECEMBER 2012
CONTENTS
COST ENGINEERING Established 1958
Vol. 54, No.6/November/December 2012
Managing Editor Marvin Gelhausen
mgelhausen@aacei.org
Art Director Noah Kinderknecht
nkinderknecht@aacei.org
Graphic Designer Cevin Thornbrugh
cthornbrugh@aacei.org
Advertising Sales Garth Leech
304.296.8444   fax - 304.291.5728
gleech@aacei.org
HEADQUARTERS
1265 Suncrest Towne Centre Dr
Morgantown, WV 26505-1876
800.858.COST    fax - 304.291.5728 
AACE® International - The Authority for Total Cost Management
TM
OUR VISION - To be the recognized technical authority in cost and schedule management
for programs, projects, products, assets, and services.
OUR MISSION - The members of AACE® enable organizations around the world to achieve
thier investment expectations by managing and controlling projects, programs, and port-
folios; we create value by advancing technical knowledge and professional development.
Cost Engineering (ISSN: 0274-9696/12) is published bi-monthly by AACE International, Inc, 1265 Suncrest Towne Centre Dr,
Morgantown, WV 26505 USA. Periodicals postage paid at Morgantown, WV, and at additional mailing office. POSTMASTER:
Send address changes to AACE International; 1265 Suncrest Towne Centre Dr, Morgantown, WV 26505-1876 USA. Customer
#7012359 (APC), Publications Mail Agreement No 40624074, Return undeliverable Canadian addresses to PO Box 503, RPO
West Beaver Creek, Richmond Hill, ON L4B 4R6. Single  copies:  US$9  members/  US$14  nonmembers  (both  +  shipping),
excluding special inserts available to AACE members only. Subscription rates: United States, US$72/year; all other countries,
US$91/year. Overseas airmail delivery is available at US$99. Subscriptions are accepted on an annual-year basis only. Copyright
© 2012 by AACE International, Inc. All rights reserved. This publication or any part thereof may not be reproduced in any form
without written permission from the publisher. AACE assumes no responsibility for statements and opinions advanced by the
contributors to its publications. Views expressed by them or the editor do not necessarily represent the official position of Cost
Engineering, its staff, or AACE International, Inc. Printed in York, PA, USA. Cost Engineering is a refereed journal. All technical
articles are subject to review by a minimum of three experts in the field. To submit a manuscript for peer review, see author
guidelines  at  www.aacei.org  and  submit  a  200  word  or  less  abstract  to  editor@aacei.org..  PHOTOCOPY PERMISSION:
Authorization to photocopy articles herein for internal or personal use, or the internal or personal use of specific clients, is
granted by AACE International, Inc., provided that the base fee of US$4.00 is paid directly to Copyright Clearance Center, 222
Rosewood  Drive,  Danvers,  MA  01923  USA.  Telephone:  978.750.8400.  For  those  organizations  that  have  been  granted  a
photocopy  license  by  CCC,  a  separate  system  of  payment  has  been  arranged.  The  fee  code  for  users  of  the  transactional
reporting service is ISSN-0274-9696/02 US$4.00. Payment should be sent directly to CCC. Copying for other than personal or
internal reference use without the express permission of AACE is prohibited. E-mail requests for photocopy permission on bulk
orders  may  be  sent  to  editor@aacei.org.  ADVERTISING COPY: Contact  Garth  Leech,  1265  Suncrest  Towne  Centre  Dr.,
Morgantown,  WV  26505-1876.  Telephone:  304.296.8444.  E-mail:  gleech@aacei.com  for  rates.  Advertisers  and  advertising
agencies assume liability for all content (including text, representation, and illustrations) of advertisements printed and also
assume responsibility for any claims arising therefrom made against the publisher. The publisher reserves the right to reject
any advertising that is not considered in keeping with the publication’s mission and standards. The publisher reserves the right
to  place  the  word  advertisement with  copy  which,  in  the  publisher’s  opinion,  resembles  editorial  matter.  All  advertising
accepted  for  publication  in  Cost Engineering is  limited  to  subjects  that  directly  relate  to  the  cost  management  profession.
Current rate card available on request. COST ENGINEERING DEADLINES: Submissions for Cost Engineering must be received at
least 8 weeks in advance of the issue date. Send to: Editor, 1265 Suncrest Towne Centre Dr, Morgantown, WV 26505-1876 USA.
Deadlines do not apply to technical papers.
Policy Concerning Published Columns, Features, and Articles
Viewpoints expressed in columns, features, and articles published in Cost Engineering journal are solely those
of the authors and do not represent an official position of AACE International. AACE International is not en-
dorsing or sponsoring the author’s work. All content is presented solely for informational purposes. Columns,
features, and articles not designated as Technical Articles are not subject to the peer-review process.
 Bechtel is among the most respected engineering, procurement, and construction
companies in the world. We stand apart for our ability to get the job done right —
no matter how big, how complex, or how remote the project.

Opportunities in Project Controls include:

Project Estimators
Planners & Schedulers
Trend Engineers
Cost Engineers
Project Controls Managers
Discover more at bechtel.com/careers
CIVIL INFRASTRUCTURE
Bechtel is an Affirmative Action/Equal Opportunity Employer (AA/EOE).
COMMUNICATIONS

MINING & METALS

OIL, GAS & CHEMICAL

POWER

The ambition
to make history.
..........................................................................................................
Risk Analysis doesn’t have to be
complicated.
Acumen Risk: Risk ReInvented
Proactively reduce risk exposure with the first easy-to-use and truly
integrated cost/schedule risk analysis and risk register software tool.

Powerful analytics, consistently accurate results.

Coming September 18, 2012. Find out more:

www.projectacumen.com/risk
 TECHNICAL ARTICLE

Integrated Cost-Schedule
Risk Analysis
Dr. David T. Hulett and Michael R. Nosbisch, CCC PSP

Abstract: This article is based on AACE International Recommended Practice 57R- consistent results for project schedule
09 that defines the integrated analysis of schedule and cost risk to estimate the and cost in each iteration.
appropriate level of cost and schedule contingency reserve on projects. The main
contribution of this RP is to include the impact of schedule risk on cost risk and Context
the need to hold cost contingency reserves. Additional benefits include the ap- Corporate culture is important. The
plication of the risk drivers method of using risks to drive the Monte Carlo simu- organization should be “risk-aware.”
lation of a resource-loaded CPM schedule. By using the risks, one can prioritize Management sets this tone, must want to
the risks to cost, some of which are commonly thought of as risks to schedule, so know the truth about the risks to the
that risk mitigation may be conducted in an effective way. The methods presented project, and must view the risk analysis as
in the RP are based on integrating the cost estimate with the project schedule by an important input to project success.
resource-loading and costing the schedule’s activities. Using Monte Carlo tech-
Management must be seen to use the risk
niques one can simulate both time and cost simultaneously, permitting calculation
of the impacts of schedule risk on cost risk. This article was presented as analysis results as they make key project
RISK.1043 at the 2012 AACE International Annual Meeting in San Antonio, Texas. decisions. Without this condition the
It was ranked as the number three paper out of 99 presentations given and rated analysis will fail, no matter how much
at the Annual Meeting. sophisticated software and training the
staff has had, because the risk data will
Key Words: Contingency, cost, estimates, mitigation, Monte Carlo, projects, risk, not be high-quality.
and schedule Traditionally, schedule risk has not
been a major factor in assessments of

T
his article describes an
improvement in cost risk
analysis over the traditional
methods that address cost risk
without explicit reference, or any
reference at all to the project schedule
and its risk. It is now known how to
represent the role that schedule risk has
in driving project cost, because the longer
some resources, such as engineering or
construction work, the more they cost.
One can also identify the risks that cause
overall cost (and schedule) objectives to
be placed in jeopardy, so one can use the
results to conduct a risk mitigation
exercise to improve the project’s
prospects for success.
The platform of this cost risk analysis
is a resource-loaded project schedule.
One may use a summary schedule or a
detailed project schedule. The budget is
assigned to the activities using summary
resources to insert the entire budget into
the schedule at the activity level.
Monte Carlo simulation is the most
commonly used approach to analyzing
the impact of multiple risks on the overall
project schedule or cost risk. Simulating a
resource-loaded project schedule derives
cost risk. More recently cost risk analyses
have included attempts to represent
uncertainty in time, but usually these
analyses occurred outside of the
framework of the project schedule.
Only recently have the tools been
available to include a full analysis of the
impact of schedule uncertainty on the
uncertainty in cost. The Monte Carlo tools
first calculated labor cost proportional to
the duration of activities. This was not a
complete assessment of cost risk because
it ignored other cost-type risks that are
not related to schedule such as risks
affecting the time dependent resources’

COST ENGINEERING NOVEMBER/DECEMBER 2012 5


Table 1 — Example Project Cost by Activity
burn rate per day and the uncertainty in
time-independent equipment or
material cost.
The integrated cost-schedule risk
analysis has several inputs, uses
specialized Monte Carlo simulation
tools, and produces several valuable
outputs.
Inputs
The Cost Estimate
The cost estimate is a basic input to
the risk analysis. Since the risk analysis
calculates the probability of achieving
the cost estimate and allows the
organization to calculate the cost
contingency reserve, the cost elements
used as inputs need to be stated without
contingency embedded in them. A good
rule is to make the cost estimate, for
each project element, the unbiased
“most likely” estimate. Some estimators
are uncomfortable about stripping the
contingency amounts from the estimate,
but the Monte Carlo simulation will re-
estimate the contingency reserve that is
appropriate for:
• the risks to the specific project’s
cost plan; and,
• the desired level of certainty of the
project management and other
stakeholders.
The values of cost that will be
assigned to the activities in the schedule
are based on the resources that will be
used to accomplish and manage the
work and the daily rate of those
resources. The cost of a project
component may involve several assigned
6 COST ENGINEERING NOVEMBER/DECEMBER 2012
resources, some of which are time-
dependent and others are time-
independent:
• Time dependent resources cost
more the longer they are employed,
e.g., construction, detailed
engineering, heavy-lift barges and
drilling rigs, equipment, project
management team or procurement.
These resources may cost more or
less even if the activity duration is
fixed since the burn rate per day
may be variable.
• Time independent resources such as
procured equipment and raw
materials, even subcontracts
(particularly before they are
awarded) may cost more or less
than the engineering estimate, but
not because they take longer to
produce and deliver.
In this article, the authors use a
simple project as an example. It is a
construction project estimated to cost
$624 million over a 28-month period.
The cost estimate is shown in table 1.
The CPM Schedule
The platform for the integrated cost-
schedule risk analysis is a cost-loaded
CPM schedule. To incorporate the
schedule risk into the cost risk, the
schedule has to be taken into account
directly and transparently.
For an integrated cost-schedule risk
analysis (and for schedule risk analysis) a
summary schedule needs to be:
• integrated;
• include representation of all the
work;
• has activities properly linked with
logic; and,
• includes enough detail to highlight
the main project milestones to be
used.
Experience shows that schedules of
300 – 1,000 activities can be used in a
risk analysis, even of projects as large as
$10 billion.
A detailed schedule may be used
but it has several limitations:
• It is usually too difficult to identify
and correct a detailed schedule with
many activities and logical
relationships for best practices.
• Applying resources to activities is
more difficult for a detailed
schedule than for a summary
schedule, even if summary
resources are used. And,
• Simulation of the detailed schedule
with risks attached is often time
consuming.
The first task in the risk analysis of
cost and schedule is to debug the
schedule. The schedule needs to follow
CPM scheduling recommended practices
because it needs to calculate the
milestone dates and critical paths
correctly during Monte Carlo simulation.
It should be noted that the scheduling
requirements for schedule risk analysis
and for high quality CPM schedules are
the same.
The scheduling principles that are
particularly important to the success of a
Monte Carlo simulation include:
• All work needed to complete the
project must be represented in the
schedule, because: (1) one does not
know whether the critical path or
risk critical path will be a priority,
and (2) for integration of cost and
schedule risk, one needs to be able
to assign all the project cost to
appropriate activities.
• The logic should not contain any
open ends, called “danglers.” This
means that: (1) each activity, except
the first activity, needs a
predecessor to drive its start date,
and (2) each activity, except the final
 Figure 1 — Example Construction Project Schedule
delivery, needs a relationship from
its finish date to a successor.
• The schedule should not rely on
constraints to force activities to start
or finish by certain dates. It should
use logic for this purpose. Date
constraints can turn a CPM network
into a calendar.
• Lags and leads are appropriate only
in limited circumstances and are
generally to be avoided in project
scheduling. And,
• The schedule used should be the
statused current schedule.
These points are consistent with
those found in GAO Cost Estimating and
Assessment Guide [1].
It is good scheduling practice to
review the total float values to make
sure they are reasonable. Large float
values may indicate incomplete logic;
and perhaps, the need to introduce and
logically link additional activities.
Table 2 — Resources for Example Construction Project
Simple Construction Case Study
A simple schedule example of a 28-
month construction project is shown in
figure 1. This figure and several others
shown in this article are screen shots
from Primavera Risk Analysis, formerly
Pertmaster Risk Expert, now owned by
Oracle.
Resources Loaded into the CPM
Schedule
Loading resources into the CPM
schedule for the purpose of integrated
cost-schedule risk analysis can be
accomplished using summary resources.
Summary resources are not sufficiently
detailed to perform resource leveling.
Their purpose is to get the entire budget
into the project schedule. Simple
categories of resources that can be given
budgeted values and placed on the
activities they work on are needed.
Resources used on the simple
construction project are shown in table
2.
In addition, the resources need to
be tagged as “labor-type”—time-
COST ENGINEERING NOVEMBER/DECEMBER 2012
dependent resources; or “material-
type”—time-independent resources, as
mentioned above. An alternative
method of applying resources to the
schedule, when the cost estimate is not
specified down to the detail of the
schedule activities, is to apply resources
to hammocks that span the activities
that get the resources.
Resources
Resources are applied to the
schedule activities, Sometimes, in doing
this, the cost estimate and schedule have
evolved largely independently of one
another and the cost estimates are not
consistent with the activity durations. It
is important that if the estimate and
schedule are initially developed
independently of one another, that they
are reconciled prior to holding the risk
assessment, or else the implied “daily
rates” of cost will be wrong.
The costs that result from placing
the resources on the example project
schedule are shown in table 3.
THE RISK DRIVER METHOD
Risk Data Inputs for the Risk Driver
Method
Applying first principles requires
that the risk to the project cost and
schedule is clearly and directly driven by
identified and quantified risks. In this
approach, the risks from the risk register
drive the simulation.
The risk driver method differs from
older, more traditional approaches—
wherein the activity durations and costs
are given a 3-point estimate which
results from the influence of potentially
7
 Table 3 — Cost Example Construction Project Showing Resources

several risks, which cannot be
individually distinguished and kept track
of. Also, since some risks will affect
several activities, one cannot capture
the entire influence of a risk using
traditional 3-point estimates of impact
on specific activities.
Using the risk driver method, the
risks that are chosen for the analysis are
generally those that are assessed to be
“high” and perhaps “moderate” risks
from the risk register. Risks are usually
strategic risks rather than detailed,
technical risks.
While the risk data are collected in
interviews with project SMEs, new risks
emerge and are analyzed. There may be
perhaps 20-40 risks, even in the analysis
of very large and complex projects. Risks
to project schedule and cost include:
• risk events that may or may not
happen; and,
• uncertainties that will happen but
with uncertain impact.
Once the risks are identified from
the risk register, certain risks data are
collected:
• Probability of occurring with some
measurable impact on activity
durations and/or costs. In any
iteration during the Monte Carlo
simulation, a risk will occur or not
depending on its probability.
• The risk also has an impact on
project activities if it does occur.
This impact is specified as a range of
possible impacts, stated in multiples
of the activity’s estimated duration
and cost. If the risk does occur, the
durations and costs of the activities
in the schedule that the risk is
assigned to will be multiplied by the
multiplicative impact factor that is
chosen from the impact range for
that iteration.
• The risks are then assigned to the
activities and resources they affect.
Collection of risk data relies on the
processes of the risk identification and
risk prioritization. It is important during

Table 4 — Example Risks and Their Parameters For The Case Stude

8 COST ENGINEERING NOVEMBER/DECEMBER 2012

risk data collection to be alert to
possible biases that crop up during
workshops. Some people want to
influence the results, while others
genuinely do not understand the
concepts or have some cognitive bias
that has to be overcome. One approach
is to conduct risk interviews with
individuals, or small groups, for which
there is a promise of confidentiality to
the participants. This is so they can talk
honestly and openly without fear that
Table 5 — Assigning Risks to Activities
management will be displeased with
them.
The degree of correlation between
the activity durations has long been
viewed as being important for
understanding and estimating correctly
the project cost risk analysis. Correlation
arises if one risk affects two (or more)
activities’ durations, or if a risk affects
the cost of two time-independent
resources. If a risk occurs, the degree to
which their durations are longer and
COST ENGINEERING NOVEMBER/DECEMBER 2012
shorter together is called correlation. As
shown below, the risk driver method
causes correlation between activity
durations, so one no longer has to
estimate (guess) at the correlation
coefficient between each pair of
activities.
Probabilistic branching, or existence
risk, requires another type of risk data—
the probability that an activity and its
cost will exist on this project. Some risks
may cause activities to occur only if the
9

Figure 2 — Histogram with Cumulative Distribution (S-Curve)
for the Project Completion Date
risk occurs. Some risk events such as
failure of a test or a commissioning
activity, if they occur, may require new
activities such as finding the root cause
of the failure, determining the recovery
plan, executing the recovery plan, and
retesting the article. These activities will
all take time and increase project cost.
They can be inserted in the schedule as
probabilistic branches, or existence
activities, with time and cost
implications if they occur.
Simulation Using the Risk Drivers
Method
In the simple example used in this
article, the risks’ impacts are specified as
ranges of multiplicative factors that are
then applied to the duration or cost of
Table 6 — Summary Schedule Risk Analysis Results for the Example Construction Project
10 COST ENGINEERING NOVEMBER/DECEMBER 2012
the activities to which the risk is
assigned.
The risks operate on the cost and
schedule as follows:
• A risk has a probability of occurring
on the project. If that probability is
100%, then the risk occurs in every
iteration. If the probability is less
than 100%, it will occur in that
percentage of iterations.
• The risks’ impacts are specified by 3-
point estimates of multiplicative
factors, so a schedule risk will
multiply the scheduled duration of
the activity to which it is assigned.
The 3-point estimate, for instance,
(low 90%, most likely 105% and high
120%), is converted to a triangular
distribution. For any iteration, the
software selects an impact
multiplicative factor at random from
the distribution. If the risk occurs
during that iteration, the
multiplicative factor selected
multiplies the duration of all the
activities to which the risk is
assigned.
• The cost risk factor is applied
differently depending on whether
the resource is labor-type or
equipment-type.
o For a labor-type resource, the
cost risk factor varies the daily
burn rate, representing more or
fewer resources applied, higher
or lower cost of those resources
per day. For these resources,
their total cost is also affected
by the uncertainty in the
duration, but they may cost
more or less even if their
durations are as scheduled.
o For equipment-type resources,
the cost risk factor varies the
total cost. For these resources,
the cost may be uncertain but it
is not affected by time.
Simulation Tools
Monte Carlo simulation is the most
commonly applied method for
conducting quantitative risk analysis. A
Monte Carlo simulation calculates the
possible project cost and schedule
values that may result from individual
risks and translates them into project-
level cost and schedule histograms or
distributions from which statistical
statements can be made.
Since one does not know whether
any risk will occur on any specific project

Figure 3 — Histogram with Cumulative Distribution (S-Curve)
for the Project Cost
or what its impact will be, one cannot
tell when a project will finish or how
much it will cost. One can only tell
probabilistically when the project might
finish and how much it might cost.
Suppose the simulation contains
3,000 iterations—separate runs using
randomly-selected risk data—and
creates 3,000 pseudo-projects. Each of
the 3,000 projects could be the sample
project in this article, since it is based on
Table 7 — Summary Cost Risk Analysis Results for the Example Construction Project
Table 8 — Cost, Schedule and Interaction Effects
a different combination of risks applied
to this project schedule and cost. These
different combinations of input data
generally compute different completion
dates and project costs. The Monte Carlo
simulation provides probability
distributions of cost and schedule from
which one can make probabilistic
statements about this project.
COST ENGINEERING NOVEMBER/DECEMBER 2012
Risk Data Used for the Construction
Case Study
The following is a sample case study,
but the risks are similar to those found
on real projects.
Suppose there is a project with the
activities shown in figure 1, and
resources/costs as shown in table 1, and
assigned to the activities as shown in
table 3. Also, suppose one has identified
risks through a workshop or interviews
and have elicited the probability and
time/cost impacts as shown in table 4.
After the risks are listed and their
parameters quantified, they need to be
assigned to the activities and their
resources. For this case study the risks
are assigned according to table 5.
Results From the Construction Case
Study Simulation
The schedule risk results from a
Monte Carlo simulation are shown in the
histogram for the case study in figure 2.
It shows that the deterministic date of
29 April 2013, is about 4% likely to be
achieved following the current plan and
without further risk mitigation actions.
Next, suppose that the project
stakeholders have agreed that their
acceptable level of confidence is at the
80th percentile. At that point, it is 80%
likely that the current project plan with
all of its risks will finish on that date or
earlier (and, if it is applied to cost, with
11
 that cost or less). At the P-80, the project
finishes on 26 November 2013, or
earlier, and needs about a 7-month
contingency reserve of time. These
results are shown in figure 2 and in table
6.
The cost risk results, including the
impact on cost of schedule risk, indicate
the need for a contingency reserve of
cost of about $169 million, or 27% at the
80th percentile (P-80). At that level there
is an 80 percent probability that the
project will cost $793 million or less,
given the risks and following the current
plan. These results are shown in figure 3
and table 7.
One can find out whether cost-type
risks or schedule-type risks are more
important in determining the cost Figure 4 — Cost and Time Results From the Simulation
contingency to, say the P-80 point. The
source of the cost contingency can be Correlation Between Cost and Schedule somewhat higher than is common in
discovered by eliminating all schedule The time-cost scatter diagram, these analyses.
risks, so as to compute the marginal shown in figure 4, is diffuse because
impact of cost risks, then repeating the there are some time-independent cost Probabilistic Branches or Project-
process by eliminating the cost risks and risks that affect the burn rate of labor- Busting Risks
computing the impact of schedule risks type resources and total cost of procured Some risks will add activities to the
on contingency. The results are shown in items. The cross-hairs shown on the project schedule if they occur, and hence
table 8. diagram cross at the deterministic point will add time and cost. Most often a
Table 8 shows that if only cost risks of 29 April 2013 and $624.2 million. The project plan assumes that the project
were present (the schedule is static) the sparse collection of points in the lower- goes well and that there are no major
cost contingency at the P-80 could be left quadrant indicate that there is only a problems. It is also common that
$78 million, whereas if only schedule 1% chance that this project will satisfy something goes wrong leading to a
risks were included (no cost risk on burn both cost and schedule targets without mandatory change in plans as the
rate or on procurement/materials), the contingency reserve. There is also a 95% project tries to recover from a
contingency needed at the P-80 is $103. chance that this project, following this discontinuous event. An example of this
These results depend on the case study plan, will overrun both cost and time problem might be the failure of the
assumptions, but in many examples of objectives. project at commissioning or final testing.
integrated cost and schedule risk There is clearly a positive slope These activities might be:
conducted on projects, the majority of running through the cloud or “football
the risk to cost arises from uncertainty in (US version) chart,” showing the strong • Determine the root cause of the
the schedule as it does in the example in impact on cost of schedule risks. The failure;
this article. correlation between time and cost is • Decide what to do;
77% in this case study, which is • Implement the action; and,

Figure 5 — Activities Added to Provide for a Risk That Commissioning May Not Complete Successfully

12 COST ENGINEERING NOVEMBER/DECEMBER 2012


Figure 6 — Schedule Impact of Probabilistic Branch on
Commissioning
• Re-test and, hopefully, pass the test
this time.
One common characteristic of these
activities is that they are almost never
found in the initial project schedule,
which assumes success. However, in risk
analysis, the possibility of test failure or
some other discontinuous uncertain
event, must be modeled using existence
risks or probabilistic branching.
Suppose that the commissioning
activity might uncover a problem that
takes time to fix. Simple changes can be
made in the project schedule to
accommodate this potentially project-
busting occurrence. One cannot use the
risk drivers that are assigned to existing
Table 9 — Schedule and Cost Impact of a 40% Probable Commissioning Risk
activities, since when one introduces
them they are given a duration of zero
(0) days. Their uncertainty is represented
by traditional 3-point estimates, but
their risk source is known. An
implementation of probabilistic
branching is shown in figure 5. The
probability that commissioning will not
complete successfully the first time is
estimated as 40%.
The schedule results for adding a
probabilistic branch are shown in figure
6. Notice that the schedule is slightly bi-
modal, with 60% of the results in the
left-hand part of the distribution and
40% in the right-hand part. There is a bit
of a “shoulder” in the cumulative
distribution at 40%, that follows the
COST ENGINEERING NOVEMBER/DECEMBER 2012
specification that the commissioning will
fail 40% of the time.
The cost of the project goes up at
the P-80, since resources are placed on
the activities in the probabilistic branch.
The impact on cost and schedule of a
40% probable problem during
commissioning, (with the parameters
shown here), are shown in table 9.
Prioritized Risks to Schedule and Cost
If the risk results for the overall
schedule and cost are not “acceptable”
to the customer, the analyst can
prioritize the risks for the project
manager who will want to mitigate the
highest-priority risks.
• For schedule risk, one needs to
identify the most important risk by
taking each risk out entirely (make
the probability = 0%) one-at-a-time
and re-run the simulation to
determine the P-80 date, allowing
one to identify the risk that has the
greatest marginal impact on the P-
80 date. Then, keeping the most
important risk out, one explores the
remaining risks to see which of
those is next-most-important, and
so forth. And,
• For cost risk, this is done by taking
each risk out of the project one at a
time, computing the impact to the
P-80 cost compared to the all-in
results, and finding the risk that has
the largest impact on the P-80 cost.
It is logical to identify the schedule
risks that have cost risk implications
as described above, but the list of
the risks in order of priority may
differ for time and for cost.
13
 Table 10 — Highest Priority Risks to Project Schedule at the P-80 Level of Confidence
Tables 10 and 11 show which risks
are the most important for schedule and
for cost.
Risk Mitigation Using Prioritized Risks
Using the prioritized risks in table 8,
one can recommend risk mitigation. The
first thing to recognize is the inaccuracy
of the estimates, which is viewed as
moderate at risk impact multipliers of
95%, 105% and 115%. However, this risk
is 100 % likely to occur, since estimating
error is with one until project financial
completion, and it is assigned to each
activity in the project, hence its
importance. The next item to be
concerned about is the probability of
problems during commissioning, which
is also the highest schedule risk. The next
largest item would be the unavailability
of key engineering staff. Down the list at
position five is the inaccuracy of the
schedule.
In fact, in the simple example the
authors created for this article, only the
top risk to project cost is a pure cost risk.
The other important risks are mostly
schedule risks (some with cost risk
components, see table 4) that increase
cost if their activities are longer than
assumed in the schedule.
14 COST ENGINEERING NOVEMBER/DECEMBER 2012
These schedule risks may be missed
or underestimated if the cost risk
analysis does not explicitly handle the
relationship of time and cost risk, as is
shown in the approach described in this
article. It is common to find that
schedule risks are important in driving
cost risk. It reinforces the benefits of
integration of cost and schedule.
Conclusion
Integrating cost and schedule risk
into one analysis, based on the project
schedule loaded with costed resources
from the cost estimate provides both:
• more accurate cost estimates than if
the schedule risk were ignored or
incorporated only partially; and,
• illustrates the importance of
schedule risk to cost risk when the
durations of activities using labor-
type (time-dependent) resources
are risky.
Many activities such as detailed
engineering, construction or software
development are mainly conducted by
people who need to be paid even if their
work takes longer than scheduled. Level-
of-effort resources, such as the project
management team or QA/QC, are
extreme examples of time-dependent
resources, since if the project duration
exceeds its planned duration the cost of
these resources will increase over their
budgeted amount.
The integrated cost-schedule risk
analysis is based on:
• A high quality CPM schedule.
• A contingency-free estimate of
project costs that is loaded on the
activities of the schedule using
resources distinguishing them by
their time-dependent and time-
independent nature.
• Good-quality risk data that is usually
collected in risk interviews of the
project team, management and
others knowledgeable in the risk of
the project. The risks from the risk
register are used as the basis of the
risk data in the risk driver method.
The Risk Driver Method is based in
the fundamental principle that
identifiable risks drive overall cost
and schedule risk and that one can
model this process. And,
• A Monte Carlo simulation software
program that can simulate schedule
 Table 11 — Highest Priority Risks to Project Cost at the P-80 Level of Confidence

risk, burn-rate risk and time- REFERENCE ABOUT THE AUTHORS

independent resource risk. 1. GAO Cost Estimating and
Assessment Guide, US Government
The results include the standard Accountability Office, March 2009
histograms and cumulative distributions (GAO-09-3SP), pgs. 218-224.
of possible cost and time results for the Dr. David T. Hulett is
project. However, by simulating both RECOMMENDED READING with Hulett &
cost and time simultaneously, one can 1 AACE International Recommended Associates, LLC. He
collect the cost-time pairs of results and Practice No. 57R-09 Integrated Cost can be contacted by
hence show the scatter diagram and Schedule Risk Analysis Using sending e-mail to:
(“football chart”) that indicates the joint Monte Carlo Simulation of a CPM david.hulett@projectrisk.com
probability of finishing on time and on Model, AACE International,
budget. Also, one can derive the Morgantown, WV, 2011.
probabilistic cash flow for comparison 2 Hulett, D.T., Practical Schedule Risk Michael R. Nosbisch,
with the time-phased project budget. Analysis, Gower Publishing Limited, CCC PSP, and Past
The risks to schedule completion Farnham Surrey England. (2009): pg. President of AACE
and to cost can be prioritized, say at the 218. International is with
P-80 level of confidence, to help focus 3 Hulett, D.G., Integrated Cost- Project Time & Cost.
the risk mitigation efforts. If the cost and Schedule Risk Analysis, Gower He can be contacted
schedule estimates including Publishing Limited, Farnham Surrey by sending e-mail to:
contingency reserves are not acceptable England. (2011): pg. 211. mike.nosbisch@ptcinc.com
to the project stakeholders, the project
team should conduct risk mitigation
workshops and studies, deciding which
risk mitigation actions to take, and re-run
the Monte Carlo simulation to determine
the possible improvements to the
project’s objectives. ◆

COST ENGINEERING NOVEMBER/DECEMBER 2012 15

 Now:What
“If schedules needed analysis,
is the difference between
Primavera would have included it.”
reviewing your P6 schedule and
reviewing your P6 XER file?

Everything

See more at
http://ScheduleAnalyzer.com

% Interfaces Directly with Database

% Hundreds of Reports & Analyses
% Advanced Longest Path Analysis
% Built-in Report & Editing Facility
% Drill-down from Dashboard to P6

16 COST ENGINEERING NOVEMBER/DECEMBER 2012

 TECHNICAL ARTICLE

Estimate Accuracy:
Dealing With Reality
John K. Hollmann, PE CCE CEP

Abstract: This article reviews over 50 years of empirical cost estimate accuracy known to be a “disaster” when systemic
research and compares this reality to common but unrealistic management ex- risks are present [27].
pectations. The empirically-based accuracy research of John Hackney, Edward One researcher said this behavior
Merrow, Bent Flyvbjerg, and others on large projects in the process industries is verges on “criminal” [19]. Cost disasters
summarized. The article then highlights risk analysis methods documented in re- and criminality are economic in nature,
cent AACE Recommended Practices that yield outputs based upon and compara- but deadly serious to owners, investors
ble to empirical reality. Tragically, many cost engineers are facilitating and tax-payers; we must ultimately take
management’s collective and sometimes willful biases regarding accuracy by using responsibility for our role in their
flawed, unreliable risk analysis methods; those who use empirically valid practices economic well-being. To help improve on
face the fate of Cassandra. The article is intended as a fundamental reference on the situation, this article surveys the
the topic of accuracy, as well as a call for our profession to use reliable practices
research facts (reality), exposes flawed
and speak the truth to management. Readers will gain an understanding of esti-
mate accuracy reality, the risks that drive it, management’s biases about it, and practices. and highlights better practices.
methods that analyze risks and address the biases in a way that results in more The article summarizes data from
realistic accuracy forecasts, better contingency estimates and more profitable in- well referenced studies by others;
vestments. This article was first presented as RISK.1027 at the 2012 AACE Inter- however, the data confirms the author’s
national Annual Meeting in San Antonio, Texas. experience. The author’s data and
observations are added, as well as
Key Words: Cost estimate accuracy, recommended practices, and risk analysis observations by others. While fact and
opinion are mixed, it is hoped that

A
readers will draw the same conclusions as
ccuracy is a measure of how a and how well risk analysis methods
the article and work to improve the
cost estimate will differ from forecast it is generally poor.
situation.
the final actual outcome. It is Investment decision makers seem
also a measure of cost particularly unaware of our research and
Studies of Overall Estimate Accuracy
uncertainty or risk (these terms are recommended practices. Sometimes they
How accurate have cost estimates
essentially synonymous in Total Cost are aware, but seem to ignore them.
been for owners? To answer this,
Management (TCM). Worse, many cost engineers facilitate
references providing empirical data on
Empirical estimate accuracy data has management ignorance by standardizing
estimate accuracy and cost uncertainty
been researched for over 50 years [30]. In their wishful thinking (i.e., tunneling or
were sought. This article focuses on
addition, reasonably reliable practices for neglecting sources of uncertainty) as
engineering and construction projects in
quantifying project cost uncertainty have exemplified by bias towards 10%
the process (e.g., oil, gas, chemicals,
been recommended by AACE and others. contingency and +10/-10% range [42].
mining, metals, utilities, etc.) and
However, the level of industry Poor investment decisions may result
infrastructure (often associated with
understanding of the reality of accuracy from using risk analysis methods that are
process plant projects) industries. These

COST ENGINEERING  NOVEMBER/DECEMBER 2012 17

 are generally characterized by
complexity, unique work scopes, design
change and sometimes new technology.
The chosen references represent
academic, research, consulting and
industry practitioner sources. Empirical
research on defense, aerospace and IT
projects was found but excluded; their
experience is analogous but more
extreme [13,16, and 20].
Estimate accuracy and cost
uncertainty data from 12 empirical
studies are summarized in table 1. These
include over 1,000 projects with samples
ranging from about 20 to 250 projects
each. The projects were typically large
enough to affect enterprise success (i.e.,
typically one million US dollars up to
megaprojects). The costs studied are the
costs to the owners. Study purposes
varied; however, the typical the
questions were: “what is the accuracy of
our estimates and why?” in reaction to a
perceived preponderance of cost
overruns.
In summary, the approximate range
of ranges for accuracy or uncertainty
around the reference amounts are as
follows:
• P10: -32% to +8%
(average about -9%)
• P50 or mean: 0% to +88%
(average about 21%)
• P90: +34% to 190%
(average about 70%)
The “accuracy” shown is the
percentage variation of the final actual
cost from the reference estimate. The
reference estimate was usually the basis
for an actual or defacto investment
decision by the owner. Estimate names
are industry specific; for example,
“feasibility” is the funding estimate for
mining projects, but not for projects in
other industries. The reference
estimates usually include contingency;
therefore, the accuracies are
understated in respect to base estimates
without contingency. From the author’s
experience, the contingency applied at
sanction is usually between 5 and 15
percent.
The statistics provided ranged from
mean and standard deviations alone to
distribution charts or tables or values at
18 COST ENGINEERING NOVEMBER/DECEMBER 2012
various confidence levels. For
comparison, the accuracies in table 1 are
summarized at approximate
p10/p50/p90 confidence levels where
the “p” value indicates the percentage
that underran. If a mean was provided
(μ), it is shown as such. If p-values were
not provided, they were approximated
from the mean and standard deviation
assuming a normal distribution; i.e., p90
equals the mean plus 1.28 times
standard deviation (the ~ symbol
indicates an approximation.) This
approximation underestimates the high
range when actual/estimate accuracy
data is skewed to the high side (i.e.,
actual data is not normally distributed).
The project samples were not
scientifically random, but were not
selected specifically because their
estimates were inaccurate; the authors
generally considered the projects in their
samples to be reasonably
representative. Studies done in reaction
to overruns may be biased toward that
experience; however, the number of
studies and the variety of industries,
regions and project types covered
indicate that cost overruns are prevalent
for large process industry projects.
The quality of the datasets varied,
but in general the authors lament the
poor state of historical project records.
For many studies, the only reliable data
was the cost at the time of project
funding approval (i.e., sanction or
investment decision) and the cost at
completion. However, some studies
were corrected for major scope changes
and escalation, which many practitioners
would not expect an estimate to cover.
The key observation is that in no
case was the nominal p90 value ever less
than +34% of the funding estimate (i.e.,
about +40 to +50% of the base estimate).
Also, the average mean or median
overrun is about 21%. This is the best
picture we have of reality for large
process industry projects with all their
imperfections and risks (unfortunately,
causal data is lacking).
Arguably, the most notable studies
are by John Hackney and Edward
Merrow because these are the
foundation for process industry phase-
gate project systems [22, 35]. However,
the studies by Dr. Bent Flyvbjerg are
perhaps best known in the popular press
[19]. Dr. Flyvbjerg has made the
following statements regarding industry
estimating practices: “We conclude that
the cost estimates used in public
debates, media coverage, and decision
making for transportation infrastructure
development are highly, systematically,
and significantly deceptive.” “(those)
who value honest numbers should not
trust the cost estimates presented by
infrastructure promoters and
forecasters.” He adds, “institutional
checks and balances—including
financial, professional, or even criminal
penalties for consistent or foreseeable
estimation errors—should be developed
to ensure the production of less
deceptive cost estimates [19].”
Merrow disagrees with Flyvbjerg in
the following: “There is widely held
belief that large public sector projects
tend to overrun because the estimates
are deliberately low-balled. Our (IPA’s)
analysis of large private sector projects
suggests that no Machiavellian
explanation is required. Large projects
have a dismal track record because we
have not adjusted our practices to fit the
difficulty that the projects present [33].”
Regardless of motives and causes,
large process and infrastructure projects
(and defense, aerospace and IT) are
frequently overrunning our funding
estimates and by very large margins. The
search found no research that showed
otherwise. Further, as “forecasters” (as
we are referred to by Flyvbjerg) we are
failing to reliably predict the proper
point of funding including contingency,
but the range of project cost uncertainty.
Studies of Estimate Accuracy Progression
Versus Level of Scope Definition
It is generally agreed that the less
well defined the project scope is, the
wider the estimate accuracy range will
be. This is a premise of phase-gate
project systems. Table 2 summarizes
accuracy studies from among this
article’s sample that also addressed
accuracy and uncertainty at various
levels of scope definition, approximated
to AACE classifications (Class 5 to 1).
Complicating the comparison, each
study has different data attributes and
uses different scope definition rating
schemes (i.e., AACE classification ratings
 Table 1 — Empirical Estimate Accuracy Studies (Typically From the Funding Estimate)
were not used). However, the author’s
experience is that process industry
funding decisions are being made based
on scope definition somewhat better
than AACE Class 4, but worse than Class 3.
Research indicates that the design
development necessary to thoroughly
mitigate definitional risk includes issued-
for-design, signed-off process and overrun; there is a huge potential for
instrumentation diagrams (P&IDs) for all overruns if the scope is more poorly
process and utility units; the author rarely defined than Class 3. Note that the
sees this level of definition at the time of Hackney and RAND models based on this
funds authorization [4]. data are available in working Excel tools
The key observation from table 2 is found at the AACE website
that even projects funded on better scope (www.aacei.org [11]).
definition (AACE Class 3) tend to be
COST ENGINEERING NOVEMBER/DECEMBER 2012 19
 What Our Estimates Say and What
Owners Want Are the Same (i.e.,
Wishful Thinking)
The next question is, “are the
overrunning projects within the cost
range of our risk analyses?”
Unfortunately, they usually are not. The
author has reviewed many industry risk
analyses by owner companies and their
EPC contractors and their p90 forecast is
rarely great than 30% over the base
estimate excluding contingency. Table 3
provides an indicative sample of risk
analysis outcomes.
The first project risk analysis shown
in table 3 had a p90 value of +45%;
however, the risks on this project were
extreme and while the team captured
some of them, the range was overrun by
the next phase estimate. P50 values are
often as little as 5% even for highly risky
projects. The author’s experience is that
despite extensive risk registers and
brainstorming sessions, most risk
quantification is dominated by an
estimator’s bias in which the team
consciously or unconsciously perceives
uncertainty in terms of estimate and
takeoff assumptions and math (i.e.,
“estimator’s risks”).
A high (p90) range of about +30%
reflects the perceived worst case
Table 2 — Empirical Estimate Accuracy Studies (Progression by Level of Scope Definition)
20 COST ENGINEERING NOVEMBER/DECEMBER 2012
uncertainty around quantities, rates,
pricing, and productivity; while
unrealistically assuming the scope is
fixed, the execution strategy and plan is
never changed, no risk events occur and
if they do, risk responses are always
effective. The result is a range that
seems to be what the owner wants to
hear.
So the next question is, “what does
the owner want to hear?” Table 4
provides an indicative sample from
different industry segments of owner
accuracy range expectations as stated in
their phase-gate project scope
development processes.
The table compares the owner
targets to the range-of-ranges in the
AACE Recommended Practice 18R-97
[4]. Is it coincidence that the owner p90
targets in table 4 are about the same as
the p90 values estimated in table 3?
By quoting specific accuracy range
targets in their processes, owners
display a dangerous misunderstanding
of risk and estimating. Once a project
plan reaches the target level of scope
definition (e.g., Class 3), the residual risk
and its potential impact is a project
scope attribute and no estimator can
appreciably improve the accuracy range
by doing a “better estimate.”
For a project with substantial risks
(most large projects), the company
accuracy ranges in table 4 have no
relevance. Unfortunately, targets tend to
pre-determine risk analysis outcomes;
i.e., they drive the risk analysis
outcomes seen in table 3 (owners get
what they ask for). Targets are prima
facie evidence of risk ignorance
(tunneling) driven by the inflated
expectations that phase-gate processes
alone will manage risks.
Further, the communication of
targets by many owners is statistically
meaningless. First, many misquote AACE
Recommended Practices by stating that
the targets are “per AACE” when no
AACE document includes specific targets
[4]. Also, few state the confidence
interval represented or the reference
value that the range is around (the base
or the funded amount?).
To wrap up the target/as-estimated
versus actual accuracy discussion, figure
1 shows the averages of table 1 (Reality)
and table 3 (As Estimated) as log-normal
curves with p10/50/90 values
comparable to the table 1 and 3
averages for those confidence levels.
Several of the studies and the
author’s experience suggest that the
lognormal distribution of
actual/estimate data is representative
[14,35]. Teams are assuming a p10/p90
accuracy range around the base
estimate of about -10/+30% with a p50
value of about 10%, while the reality is
closer to -20/+120% with a p50 of about
20%. Arguably, a 20% or even 30%
overrun will not render most projects
unprofitable; however, a 120% overrun
at p90 would.
Challenging the Data : Nowhere to Hide
The following are likely challenges
to this article’s findings along with the
author’s responses:
1. The actual data includes the impact
of major scope changes and
escalation.
Major scope change and escalation
are by definition excluded from
contingency [3]. In table 1, 7 of the 12
studies corrected for price changes over
time and 3 corrected for major scope
changes (i.e., changes to basic project
premises such as plant location, product
specifications or capacity).
Table 3 — Reported Accuracy Ranges from Owner and EPC Contractor Risk Analyses
Studies show wide accuracy ranges
with or without correction. In the
author’s benchmarking experience,
major owner scope change (as opposed
to design changes which owner costs
must cover) is uncommon (this may be
less true for public projects).
2. We cannot forecast volatility and/or
black swan events (i.e., unknown
unknowns).
While any one black swan event is
improbable, the probability of any black
swan or an equivalent confluence or
compounding of lesser risks occurring
during the extended duration of large
projects is likely. The accuracy findings
appear to hold for all time periods and
regions, in both hot and cold economies.
For example, Ernst & Young found
that mining projects estimated during
hot markets (when market risks were
known) were still overrunning during the
post-2008 recession; “Of the companies
that reported project overruns publically
(between Oct 2010 and March 2011),
COST ENGINEERING NOVEMBER/DECEMBER 2012
the average overrun was about 71% of
the original project cost estimate”[37].
Another mining article referenced a
series of studies which indicate that
overruns have been the norm in every
time period since 1965 [38]:
• “A study of 18 mining projects,
covering the period 1965 to 1981,
showed an average cost overrun of
33 percent, compared to feasibility
study estimates.
• A study of 60 mining projects,
covering the period from 1980 to
2001, showed average cost overruns
of 22 percent, with almost half of
the projects reporting overruns of
more than 20 percent.
• A review of 16 mining projects,
carried out in the 1990s, showed an
average cost overrun of 25 percent”.
Historical experience alone is
enough to quantify the probability and
impact “unknown-unknowns” as a class.
We may not know the risk’s name, but
we know about what it will cost (i.e.,
Table 1.)
21
 Table 4 — Owner Phase-Gate Target Accuracy Ranges Vs. AACE Classes and Empirical Studies

Figure 1 — As Estimated (and Target) Accuracy Vs. Empirical Accuracy at Funding

3. Some systemic risks are difficult to
measure and/or politically sensitive.
The tools for rating scope
development, as well as competency
and project system discipline (e.g., weak
change management) are well
established [4, 21, 22, 35, and 45]. While
including, “incompetent management,”
in a risk register is problematic, it is
necessary to identify and quantify such
risks. The risk analyst must have
sufficient independence to do so.
4. Estimating “all” project cost risk is
not part of the job (not in my work
scope).
If one declared in the Basis of
Estimate reports that, “most significant
risks were excluded” and/or “past

22 COST ENGINEERING NOVEMBER/DECEMBER 2012

experience with similar projects was
ignored,’ this challenge might have some
validity. However, in the author’s
experience, such statements (or
confessions) are rarely made.
Unfortunately, breaking risk down
(e.g., operational, project, strategic,
enterprise, contextual, global,
background, etc. [42]) and disseminating
responsibility for its analysis and
quantification is a potential recipe for
forecasting failure. Risks interact and
often compound and cannot readily be
parsed for quantification like elements in
a work breakdown.
In summary, it is the author’s
experience that these “challenges” are
usually just reasons for our failings; they
do not excuse them. We know better
and the data is clear; with empirical
insight added to other methods, risk is
always quantifiable albeit imperfectly.
Flawed Practices and Lost Credibility
Flawed practices such as a bias
toward estimator’s risk, misguided
targets, tunneling and parsing risk
quantification have been mentioned.
The 1990s also brought reengineering
and downsizing to the industry with the
loss of empirical data and analytical
skills. Concurrently, Monte Carlo
simulation (MCS) for spreadsheets was
introduced which made risk analysis
seem simple and doable regardless of
skill level.
Unfortunately, MCS was applied in
“line-item ranging” (as opposed to range
Figure 2 — Actual Cost/Appropriation Estimate For Small Projects
estimating) in which the team assigns
cost ranges to line-items in their
estimate (i.e., contributing to estimator’s
bias) based on brainstorming, and then
runs the MCS, usually without
considering line-item dependency [24].
The risks listed in the register (which
tend to exclude systemic risks) are not
explicitly included in these models. This
is the method that research has shown
to be a “disaster” for projects with
systemic risks [27]. “Line-item ranging”
(or activity duration ranging for
schedule) fails in part because of faulty
application (i.e., no dependencies), but
also because brainstorming is unable to
elicit the impacts of systemic risks on
individual estimate line items or
activities, and finally, the impact of risk
register events are difficult to ascribe to
individual estimate line items in
aggregate. This method is not an AACE
Recommended Practice.
It is easy to conclude from the
research and observations that our risk
analyses and contingency estimates are
not credible for large process industry
projects. Decision analysis expert John
Schulyer defines a credible analysis as,
“one that gets used [44].” The following
statements by industry executives
indicate that our analyses are not useful
(self-criticism by owner executives is
understandably more difficult to find):
• Schlumberger CEO Andrew Gould
stated: “...while not wishing to
embarrass any of my customers, I
would add that many greenfield
COST ENGINEERING NOVEMBER/DECEMBER 2012
(upstream oil) projects suffer
significant cost overruns. Indeed, as
a general rule, 30 percent of such
projects experience budget
overruns of 50 percent [41].”
• Financier Jasper Bertisen of
Resource Capital Funds (RCF) had
this observation: “the vast majority
of mining projects have been
coming in way over budget for the
past couple of decades. As a result,
RCF now automatically factors in an
average cost overrun of 25% when it
considers the cost of mining projects
[28].”
The prevailing use of flawed
analyses has damaged our collective
credibility. This will be difficult to remedy
because poor practices have become
institutionalized. For example, in the
mining industry, the author commonly
finds companies funding projects at a
p80 level of confidence.
This has evolved because (as
indicated by prior quotation) managers
intuitively understand that the p50
values we provide in our estimates are
too low (i.e., often <10% contingency on
even the riskiest projects) and they
believe that the p80 level of about 15 to
20% contingency is more realistic.
However, it is “more realistic” because in
fact this forecast p80 is the p50 of the
“reality” that we fail to predict!
Cost engineers who do use realistic
risk quantification practices are treated
like Cassandra; management will not
believe the truth after being fed
23
 Figure 3 — Balancing Of Over And Underruns When All Project Sizes Are Studied Together
unreality for decades. The real p80 or
p90 is likely to be unprofitable; as shown
in studies, the least p90 capital cost
growth is >40 to 50%. If management
faced this reality, no project would ever
be authorized without stellar scope
definition and optimization, top-notch
planning, team building, risk
management, and all of the other best
practices we know of. Isn’t that the
point? Why are we facilitating anything
less? Why do we let them assume that
poor practices are a safe bet when they
are courting disaster!
The lesson from the empirical
history (table 1) and the practice history
(table 2) is that we need to address the
entire scope of risks (project-specific,
systemic, and escalation) and the
empirical “reality” of uncertainty on
large process industry projects. Research
by others points in the same direction
[16, 17, 18, 19, 22, 27, 32, 33, 42]. AACE
is currently developing a Decision and
Risk Management Professional (DRMP)
Certification that will focus on risk
identification and quantification
competencies, including AACE
Recommended Practices that document
reliable methods.
24 COST ENGINEERING NOVEMBER/DECEMBER 2012
The Project Size Dichotomy
There is less empirical research of
small project estimate accuracy because
these projects are individually less of a
threat to overall profitability and
shareholder’s perceptions. However, we
know that the realities of small and large
projects differ; small projects are biased
to overestimating and underruns. As
stated by one researcher, “when a
project team sets a soft (cost) target,
about half of the unneeded funds are
usually spent…about 70% of small
projects underrun” [29].
This research also indicated that in
small project systems, overruns tend to
be punished. To avoid punishment (in
less disciplined cultures) teams avoid
overruns by including “fat” (i.e., above-
the-line contingency) in the base
estimate because high visibility
contingency is often poorly received by
management for any project size. This
can bias a company’s perception of risk
and partly explain their misguided
targets.
Few researchers study small projects
because not only is record keeping lax,
but underruns are rewarded and are not
seen as a problem despite being
associated with wasteful capital
spending. Figure 2 shows a combination of small project target-
representative distribution of
actual/estimate values observed by the
author for small project portfolios; often,
no projects overrun by more than 10%.
In this “cresting wave” pattern, most
projects spend all their funds, while
some return all or some of the excess; for
this outcome, management and/or
teams are rewarded. The more that
funds are wasted, the sharper the peak
between 0.9 and 1.1. Perversely, the
more “accurate” the outcome, the less
desirable (though best rewarded) it is;
underruns and tight accuracy often
indicate overfunding and wasted capital
rather than excellent project control
discipline
The small vs. large project overrun
dichotomy can induce a kind of corporate
schizophrenia. Many owner companies
have “major project” organizations that
are separate from small or plant-based
organizations. A newly formed major
project group will often inherit the small
project system trait of risk-ignorance
(expectation of underruns.) They do not
appreciate that EPC contractors for major
projects prepare base estimates with less
fat because reviews expose fat and there
is sometimes a bias to keep estimates
low to see the project get funded. The
reinforced tunneling and risk-free base
estimates is a recipe for overruns on
large projects.
Looking at an entire company
project portfolio, the combined
distribution of accuracy data for small
and large projects can look serenely
“normal.” Benchmarking data observed
by the author indicates that in a
population of all project sizes, the
P10/P90 range is about +/-20% around
the funding estimate, as illustrated in
figure 3. Given just one distribution,
management will be unaware that there
are two conflicting realities.
The Measurement Dilemma
Unfortunately, accuracy (i.e.,
actual/estimated cost) is often misused
as a measure of estimate “quality” (as in
“a high quality estimate is an accurate
estimate”) or estimating performance.
This is inappropriate because, as
discussed, the only way for an estimator
to deliver a targeted accuracy for a given
scope is to over-estimate the cost; risk
and project performance are not in the
estimator’s control.
Faced with overruns, estimators and
the team tend to hide behind the excuses
discussed previously. Accuracy should be
used to measure the performance of the
risk management process (not the
estimating process) in conjunction with
project historical data including causal
information so we can improve our risk
identification, analysis and
quantification, and treatment.
Tight accuracy may indicate wasted
capital funds; accuracy measures must
always be accompanied by measures of
project control process discipline and
project cost competitiveness (lower
absolute costs) or cost bias.
AACE Recommended Practices (RPs)
There is an AACE RP that guides the
selection and development of risk
quantification and contingency
estimating methods [2]. This RP provides
“principles” that any method should
align with including;
• start with identifying risk drivers;
• link risk drivers and cost/schedule
outcomes; and
• employ empiricism.
Note that the previously discussed,
“line-item ranging” method, is not
explicitly in accordance with any of
above principles.
Risks differ in how they impact
project costs and therefore methods
vary in how the risks are quantified. To
cover the whole scope of risks, AACE has
defined a risk breakdown [24] in respect
to quantification methods that includes:
• Project-Specific Risk: risk affecting
the specific project and plan;
• Systemic Risk: artifacts or inherent
attributes of the system, enterprise
or strategy; and,
• Escalation Risk: driven by
economics (which regionally may
involve politics).
Analogies for these risks suggested
by others include: operational (project),
strategic (enterprise), and contextual
(global) risks respectively [42]. Methods
that address these risk types can be
integrated to generate a “universal” cost
risk profile to support decision making.
AACE also recommends that cost and
schedule risk analysis be integrated.
For each risk type, there are AACE
RPs for risk analysis methods that apply
as follows (“how-to” descriptions for
these methods are covered in the
references):
• Project-Specific Risk:
o 41R-08: Risk Analysis and
Contingency Determination Using
Range Estimating [12].
o 44R-08: Risk Analysis and
Contingency Determination Using
Expected Value [9].
o 57R-09: Integrated Cost and
Schedule Risk Analysis Using Monte
Carlo Simulation of a CPM Model
[8].
o 65R-11: Integrated Cost and
Schedule Risk Analysis and
Contingency Determination Using
Expected Value [7].
• Systemic Risk:
o 42R-08: Risk Analysis and
Contingency Determination Using
Parametric Estimating [10].
o 43R-08: Risk Analysis and
Contingency Determination Using
COST ENGINEERING NOVEMBER/DECEMBER 2012
Parametric Estimating – Example
Models as Applied for the Process
Industries [11].
• Escalation Risk:
o 58R-10: Escalation Principles and
Methods Using Indices [6].
o 68R-11: Escalation Estimating Using
Indices and Monte Carlo Simulation
[5].
Regardless of the risk analysis
methods used, the findings of this article
suggest that, at a minimum, you always
test your p90 outcomes (the “high”
scenario given to the business
organization to test the robustness of
their decision) against the empirical
reality. If no other historical data is
available, this article provides actual
examples to consider. If your p90 is 25%
or less over the base estimate, ask why
NO study ever showed less than about
40% for p90; what risks are you missing?
what impacts have you underestimated?
Finally, and most important, ask “how
can we improve project practices and
scope in consideration of the risk
reality?”
Conclusion
As a student of cost engineering and
the editor/lead author of AACE’s Total
Cost Management Framework process
[26], I am dismayed by the extreme
disconnect between our practices and
the long-known reality as shown in
figure 1. There is an ongoing failure to
effectively address the reality of project
cost uncertainty and there is a lack of
good historical data with causal
information. This has led to a credibility
crisis. It also raises an ethical question (if
not a criminal one per Flyvbjerg); what
does it mean if we understand reality but
continue to use failed methods known to
be contrary to experience to the
potential detriment of our employers
and clients? The AACE Canons of Ethics
(item 2g) states: “When, as a result of
their studies, members believe a
project(s) might not be successful…they
should so advise their employer or
client” [1]. In respect to large process
industry projects, readers of this article
can consider themselves so advised.
There are of course practitioners
who do address the entire scope of cost
25
 risks (in AACE terms; systemic, project-
specific and escalation), capture data
and consider the empirical record [16,
17, 18, 21, 42, and 43]. However, in the
author’s experience, the application of
robust practices is uncommon. At a
minimum, teams should at least test
their worst case analysis outcomes
against the empirical reality. They should
study this article’s references and their
own enterprise’s historical experience
(watching out for the small versus large
project behavioral dichotomy). And, they
should then seek to improve their
practices to improve on past outcomes.
The author does not agree with Dr.
Flyvbjerg’s approach to using empiricism
(i.e., “reference class forecasting” [19])
which implies that biases are so
intractable that we are doomed to
repeat the past.
The article also points out that
companies should not use accuracy as a
cost estimating quality measure; it is a
risk management and project control
process quality measure. Tight accuracy
is often an indicator of wasted capital;
measures of project control process
discipline and project cost
competitiveness must accompany
accuracy measures.
In summary, this article references
and summarizes over 50 years of
empirical cost estimate accuracy
research on large projects in the process
industries. It shows how this reality
compares (or does not compare) to what
we say and do. Recommended risk
analysis methods have been highlighted.
Failed methods are exposed. It is hoped
that the facts, observations and opinions
brought together here will serve as a
valuable reference on the topic of cost
accuracy and uncertainty so that we can
better speak the truth among ourselves
and with management. The path to
more realistic uncertainty forecasts,
better contingency estimates and more
profitable investments is clear and
documented by AACE International. ◆
REFERENCES
1. AACE International, Canon of Ethics.
AACE International, Morgantown,
WV, 2012.
2. AACE International, “Contingency
Estimating – General Principles,”
26 COST ENGINEERING NOVEMBER/DECEMBER 2012
Recommended Practice 40R-08,
AACE International, Morgantown,
WV, (latest revision).
3. AACE International, “Cost
Engineering Terminology,”
Recommended Practice 10S-90,
AACE International, Morgantown,
WV, (latest revision).
4. AACE International, “Cost Estimate
Classification System – As Applied in
Engineering, Procurement, and
Construction for the Process
Industries,” Recommended Practice
18R-97, AACE International,
Morgantown, WV, (latest revision).
5. AACE International, “Escalation
Estimating Using Indices and Monte
Carlo Simulation,” Recommended
Practice (draft) 68R-11, AACE
International, Morgantown, WV,
(latest revision).
6. AACE International, “Escalation
Principles and Methods Using
Indices,” Recommended Practice
58R-10, AACE International,
Morgantown, WV, (latest revision).
7. AACE International, “Integrated Cost
and Schedule Risk Analysis and
Contingency Determination Using
Expected Value,” Recommended
Practice (draft) 65R-11, AACE
International, Morgantown, WV,
(latest revision).
8. AACE International, “Integrated Cost
and Schedule Risk Analysis Using
Monte Carlo Simulation of a CPM
Model,” Recommended Practice
57R-09, AACE International,
Morgantown, WV, (latest revision).
9. AACE International, “Risk Analysis
and Contingency Determination
Using Expected Value,”
Recommended Practice 44R-08,
(latest revision).
10. AACE International, “Risk Analysis
and Contingency Determination
Using Parametric Estimating,”
Recommended Practice 42R-08,
AACE International, Morgantown,
WV, (latest revision).
11. AACE International, “Risk Analysis
and Contingency Determination
Using Parametric Estimating –
Example Models as Applied for the
Process Industries,” Recommended
Practice 43R-08, AACE
International, Morgantown, WV,
(latest revision).
12. AACE International, “Risk Analysis
and Contingency Determination
Using Range Estimating,”
Recommended Practice 41R-08,
AACE International, Morgantown,
WV, (latest revision).
13. Arena, Mark, Robert S. Leonard,
Sheila E. Murray, and Obaid
Younossi, “Historical Cost Growth of
Completed Weapon System
Programs” TR-343, The RAND
Corporation, 2006.
14. Bertisen, Jasper and Graham A.
Davis, 'Bias and Error in Mine Project
Capital Cost Estimation,' The
Engineering Economist, 53: 2, 118
— 139, 2008.
15. Biery, Frederick, “Improving Mineral
Project Performance,” Canadian
Institute of Mining, Metallurgy and
Petroleum Conference, May 2009.
16. Butts, Glenn and Kent Linton, “The
Joint Confidence Level Paradox; A
History of Denial,” 2009 NASA Cost
Estimating Symposium, NASA, April
2009.
17. Curran, Kevin M., “Value-Based Risk
Management (VBRM),” Cost
Engineering, AACE International,
AACE International, Morgantown,
WV, February 2006.
18. Curran, Kevin M and Michael
Curran, “Handling the Truth in Risk
Management,” 2010 AACE
International Transactions, AACE
International, Morgantown, WV,
2010.
19. Flyvbjerg, Bent, Mette Skamris Holm
and Søren Buhl, “Underestimating
Costs in Public Works Projects; Error
or Lie?” APA Journal, 68:3, 279-295,
2002.
20. Flyvbjerg, Bent and Alexander
Budzier, “Why Your IT Project May
Be Riskier Than You Think,” Harvard
Business Review, September 2011.
21. Gough, Martin and Peter Maidment,
“The Stability Model Method of Risk
Management and Early Prediction
of Project Performance,” The Revay
Report; Revay and Associated
Limited, February 2006.
22. Hackney, John W. (Kenneth K.
Humphreys, Editor), Control &
Management of Capital Projects,
Chapter 18, AACE International,
Morgantown, WV, 2002.
23. Harbuck, Robert H, “Are Accurate
Estimates Achievable During the
Planning of Transportation
Projects?,” 2007 AACE International
Transactions, AACE International,
Morgantown, WV, 2007.
24. Hollmann, John K., “The Monte-
Carlo Challenge: A Better
Approach,” 2007 AACE
International Transactions, AACE
International, Morgantown, WV,
(latest revision).
25. Hollmann, John K., “Recommended
Practices for Risk Analysis and Cost
Contingency Estimating,” 2009
AACE International Transactions,
AACE International, Morgantown,
WV, 2009.
26. Hollmann, John K (editor), Total
Cost Management Framework,
AACE International, Morgantown,
WV, 2006.
27. Juntima, Gob and Scott E.
Burroughs, “Exploring Techniques
for Contingency Setting,” 2004 AACE
International Transactions, AACE
International, Morgantown, WV,
2004.
28. Kosich, Dorothy, “Vast Majority of
Mining Projects Experience Steeper
Cost Over-runs,” MineWeb, 30 Nov
2011.
29. Kulkarni, Phyllis, “Stop Punishing the
Overruns,” InSites, IPA, Inc., Sept 7,
2011.
30. Linder, Stefan, “Fifty Years of
Research on Accuracy of Capital
Expenditure Project Estimates: A
Review of the Findings and Their
Validity,” Center for Research in
Controllership and Management,
WHU - Otto Beisheim Graduate
School of Management, Vallendar,
Germany. April 2005.
31. Lundberg, Mattias, Anchalee
Jenpanitsub and Roger Pyddoke,
“Cost Overruns in Swedish Transport
Projects,” CTS working paper
2011:11, Centre for Transport
Studies, Stockholm Sweden, 2011.
32. Merrow, Edward W., Industrial
Megaprojects, John Wiley & Sons,
Inc., New York, NY, 2011.
33. Merrow, Edward, W., “Why Large
Projects Fail More Often;
Megaproject Failures:
Understanding the Effects of Size,”
Presentation to the Joint Meeting of
the AACE National Capital Section
and American Society of Mechanical
Engineers Section, April 20, 2011.
34. Merrow, Edward W. and Brett R
Schroeder, “Understanding the
Costs and Schedule of Hydroelectric
Projects,” 1991 AACE International
Transactions, AACE International,
Morgantown, WV, 1991.
35. Merrow, Edward W., Kenneth E.
Phillips and Christopher W. Myers,
“Understanding Cost Growth and
Performance Shortfalls in Pioneer
Process Plants,” R-2569-DOE, p48,
The RAND Corporation, 1981.
36. Merrow, Edward W., Lorraine
McDonnell and R. Yilmaz Arguden,
“Understanding the Outcomes of
Megaprojects,” R-3560-PSSP, The
RAND Corporation, 1988.
37. Mitchell, Paul, “Achieving Major
Capital Project Effectiveness and
Corporate Performance,” Effective
Capital Project Execution: Mining
and Metals, EYGH Limited, 2011.
38. Noort, D.J., and C. Adams, “Effective
Mining Project Management
Systems” International Mine
Management Conference,
Australasian Institute of Mining and
Metallurgy, October 2006.
39. Oil & Gas Journal Online Research
Center, US Pipeline Study 2009;
Actual vs. Estimate (database for
fee), Penwell Corporation, Houston
TX, 2009.
40. Rath, Michael, Robert Spittle, Paul
Osman and Nicholas Tawa. “Major
Water Capital Projects: Critical
COST ENGINEERING NOVEMBER/DECEMBER 2012
Factors For Improving Outcomes,”
Deloitte Touche Tohmatsu, March
2010.
41. Reddall, Braden (reporting), “Cost
Overruns Common on Oil Projects –
Schlumberger,” Thomson Reuters,
Energy and Oil News, 2 June, 2011.
42. Rolstadås, Asbjørn, Per Willy
Hetland, George Farage Jergeas and
Richard E. Westney, Risk Navigation
Strategies for Major Capital
Projects: Beyond the Myth of
Predictability, Springer- Verlag
London Limited, 2011.
43. Schroeder, Brett and Jan A. Jackson,
“Why Traditional Risk Management
Fails in the Oil and Gas Sector:
Empirical Front-Line Evidence and
Effective Solutions,” 2007 AACE
International Transactions, AACE
International, Morgantown, WV,
2007.
44. Schulyer, John R., “Investment
Decision Making,” Chapter 14 of The
Engineer’s Cost Handbook, Editor
Richard E. Westney, Marcel Dekker,
Inc., New York NY, 1997.
45. Stephenson, H. Lance, “Cost
Engineering Maturity Model
(CEMM),” 2011 AACE International
Transactions, AACE International,
Morgantown, WV, 2011.
46. Thomas, Steve, “Project
Development Costs-Estimates Vs.
Reality,” Pincock Perspectives,
November 2000.
ABOUT THE AUTHOR
John K. Hollmann is
with
Validation Estimating,
LLC. He can be
contacted by sending
e-mail to:
jhollmann@validest.com
27
 TECHNICAL ARTICLE

Quantifying Estimate Accuracy and

Precision for the Process Industries:
A Review of Industry Data
Alexander Ogilvie, Robert A. Brown Jr., Fredrick P. Biery
and Paul Barshop

Abstract: This article quantifies industry metrics for estimate accuracy based on comprehensive set of industry projects.
a sample of 462 capital projects executed by the process industries over the last Despite these limitations, these studies
20 years, comparing estimated costs at the various stages of project definition to were valuable as they provided a set of
actual costs, normalized for a number of factors. The authors then contrasted the industry norms that quantify estimate
results from the industry survey with industry expectations, as presented in AACE accuracy based on actual project data.
International Recommended Practice 18R-97. The results of our survey indicate In the absence of more robust
there is a much greater level of variability in estimate accuracy as compared to industry data, the cost engineering
industry expectations. Further analysis demonstrates that projects which have community has defaulted to a range of
completed a definitive project scope at authorization benefit from a reduction in sources to quantify typical estimate
estimate variability as compared to industry. This article was first presented as accuracy, including in-house data,
RISK.1100 at the 2012 AACE International Annual Meeting in San Antonio, Texas.
historical experience, or industry
standards and Recommended Practices
Key Words: Capital projects, cost, estimate accuracy, process industries and scope
(RP) [1]. The objective of this article is to
enhance industry’s understanding of

I
n virtually all applications and
scenarios, a cost estimate’s value is
related to its capacity to predict the
final cost of a planned scope of
work. The importance of an accurate
estimate confronts the cost estimator
from all directions—from:
• the business/project sponsor, who is
relying on the estimate to quantify
the expected return on the capital
invested;
• to the finance group, who is relying
on the estimate to manage quarterly
cash flow; and,
• to the project controls group, who is
using the estimate as the basis for
project cost and schedule control
during execution.
All of these stakeholders have estimate accuracy by providing
different uses for the estimate, but share
benchmarks of estimate accuracy for a
a need in that they are relying on the range of estimate classifications. The
estimate to have a reasonable degree of authors calculate these metrics from a
accuracy. large sample of industry projects
The cost engineering community has executed by the oil and gas, chemical
devoted a substantial amount of time to process, and mining and minerals
consider practices and processes to industries over the last 20 years. Given
improve estimate accuracy [5,16,17]. In the importance of achieving a reasonable
contrast, only a limited number of articles
level of accuracy and precision for cost
and papers have also quantified the estimates, the goal of this article is to
estimate accuracy ranges using actual provide an independent measure of
industry data from completed projects estimate accuracy and compare these
[3,12,13]. In these instances, the data results against current industry
presented were only for a subset of expectations.
industry (i.e., mining and minerals) or a
specific type of project (i.e., megaprojects Review and Definitions
or new technology projects), and The authors first considered the
therefore were limited in an ability to definition of estimate accuracy as this
quantify estimate accuracy for a concept has been such an important part

28 COST ENGINEERING NOVEMBER/DECEMBER 2012

of the cost estimating vernacular that target, precision is defined as the many of the models developed by the
the concept of an accurate estimate has absence of variation. Cost estimates financial industry to quantify risk as
evolved into a variety of potential with a wide confidence interval (e.g., see related to mortgage-backed securities
definitions. In this section, the authors the ±50 percent estimate) are focused on improving precision, but
will review and define estimate accuracy considered to be imprecise, while were entirely inaccurate with respect to
and estimate precision, both of which estimates with a small confidence the range of potential outcomes.
are closely related. interval are considered to be precise. In considering cost estimates and
The authors will also use these Figure 2 presents the outcome confidence intervals, both accuracy and
concepts to introduce AACE matrix between accuracy and precision. precision should be considered within
International Recommended Practice As based on figure 2, precision without the same context. Of course, the
18R-97 (and 17R-97), which serves as a accuracy is a potentially dangerous objective of cost estimating (as with
basis of comparison for our analysis phenomenon as it fools the users into an most related professions) is to achieve
[1,2]. Following a review of the unwarranted sense of confidence. For both accuracy and precision with respect
definitions, the authors will consider the example, one could argue (within to all cost estimates.
standard stage-gated capital project another article, paper, or forum) that
development work process for the
process industries and apply the AACE
International estimate classification to
this process.
Accuracy measures the closeness an
activity is able to achieve to meet a
desired value or target. For example, a
field goal kicker in football is considered
accurate if his field goal attempts are
consistently through the field goal posts.
AACE Technical Board Chair Larry
Dysert defines estimate accuracy as, “an
indication of the degree to which the
final cost outcome of a project may vary
from the single point value used as the
estimated cost of the project [6].” While
estimate accuracy is measured based on
a single point estimate, the accuracy of
an estimate is best represented as a
confidence interval ranging around the
single point estimate.
As presented in figure 1, the Figure 1 — Estimate Accuracy Presented as a Probability Density Function
concept of estimate accuracy can be
depicted using a standard probability
density function that quantifies the
potential minimum and maximum cost
for a cost estimate. Based on the
probability density curve as presented in
figure 1, it has become an industry norm
to present the estimate accuracy range
as a plus or minus (±) confidence interval
around the point estimate (e.g., a ±10
percent estimate translates that the
estimate will be accurate within 10
percent of the total point estimate;
conversely, a ±50 percent estimate will
be accurate within 50 percent of the
total point estimate).

In addition to estimate accuracy, it is

important to consider the concept of
estimate precision. While accuracy
measures one’s ability to achieve a
Figure 2 — Outcome Matrix for Accuracy and Precision

COST ENGINEERING NOVEMBER/DECEMBER 2012 29

 Within the cost estimating
community, the concept of a ±10 or ±50
percent estimate has evolved to describe
potentially all aspects of an estimate,
including the estimating methods,
estimating tools, estimate accuracy,
estimate precision, and inputs used for
the estimate.
Review of AACE International
Recommended Practice 18R-97
AACE International Recommended
Practice 18R-97 was developed to
integrate all aspects of estimate
accuracy within a common framework
and set of guidelines In the absence of
robust historical data, this RP has
evolved into the de facto industry
standard for estimate accuracy. Given
that this RP will serve as a comparison
against the results of this study, a brief
summary of this RP is provided. RP 18R-
97 provides a set of guidelines for
estimate classes considering projects
managed through a standard capital
project development /definition process
(commonly known as the front-end
loading [FEL] process). Figure 3
tabularizes the characteristics of the five
estimate classes developed in this RP.
The five estimate classes presented
provide guidelines for how estimates
progress from high-level approximations
to more detailed estimates as a project
progresses through the FEL process.
The major guidelines for this RP are
summarized as follows:
Figure 3 — Summary of AACE International Estimate Classification System
30 COST ENGINEERING NOVEMBER/DECEMBER 2012
• The expected precision of an
estimate improves through the
classification system, with Class 1
estimates exhibiting much less
expected variability than Class 5 and
Class 4 estimates.
• The estimate classes show
approximately the same degree of
accuracy with the average point
estimate for each class approaching,
on average, the actual project cost.
• The estimating methodology
becomes more detailed and
complex as estimates progress
through FEL.
• The degree of effort required to
prepare an estimate becomes
exponentially greater as estimates
progress through FEL.
• The data/project inputs required (as
measured by the percent of
definition/engineering complete) to
develop an estimate become
greater and more detailed for lower
estimate classes.
The last point is perhaps the most
valuable as it formalizes the linkage
between the class of estimate and
quality and completeness of data,
information, and deliverables that serve
as inputs into the estimate. This concept
(i.e., inputs driving outputs) cannot be
underscored as it maintains that the key
driver for estimate accuracy and
precision resides in the completeness
and accuracy of the information which
serves as the basis for that estimate.
In addition to the above points, the
RP provides the expected accuracy range
for each estimate classification. Table 1
summarizes the expected accuracy
range guidelines. The RP has reached
consensus that estimates improve with
respect to both accuracy and precision
as a project progresses through
development. However, the RP is unable
to reach consensus regarding the
relative degree of precision for each
estimate class.
As presented in table 1, each
estimate class contains a range of both
minimum and maximum confidence
intervals. These ranges are insightful as
they provide some indication of the wide
range of accuracy expectations that
currently reside within Industry. Given
the variability of inputs for Class 4 and 5
estimates, the Industry expected ranges
show the least consensus in estimate
ranges as compared to the other
estimate classes.
Project Life Cycle for the Process
Industries
Given the high capital requirements
for the oil and gas and associated
process industries, most owner
companies have adopted a formal,
stage-gated process to develop business
ideas into working capital assets and
plants [4,9,10,11]. While the details of
these processes vary by company, they
adhere to a stage-gated process of
identify-select-define-execute-operate
to transform a business opportunity to a
working plant [4,11].
In this model, the capital project is
developed through a series of phases
with specific deliverables required to
finish a specific phase. Over time, the
stage-gated process has become an
industry standard for the development
of capital projects and is used by most
owners and contractors in the process
industries. Past research has indicated
that the stage-gated work process
improves overall capital performance
[10,18]. In addition, the principles of a
Table 1 — Expected Estimate Accuracy Ranges for AACE International Estimate Classification
(adopted from 18R-97)
Figure 4 — Stage Gated Process for Project Development
stage-gated process serve as a basis for
the AACE International Total Cost
Management Framework [7].
Figure 4 depicts a generic stage-
gated work process for capital project
development. In most cases, the system
has five gates, with three (generically
termed as appraise, select, and define)
composing project definition (or pre-
authorization), or the FEL stage.
It is this portion of the project in
which the cost estimator is generally the
most involved as each stage gate will
require an estimate of the planned
scope so the gate review can be
completed (e.g., a business lead cannot
COST ENGINEERING NOVEMBER/DECEMBER 2012
assess the financial gains of a capital
investment without some estimate of
the capital cost required to execute the
project).
Most company project systems will
require an estimate of a particular
quality to meet the FEL pass-gate
requirements. A review of these project
systems shows that these classifications
generally follow the AACE International
categories with some variation in the
specific practices and related accuracy
classifications required by each company
(or even each business unit).
For this reason, the cost estimates
captured for each FEL stage may not, in
31
 Figure 5 — Matching The AACE International Estimate Classification With Typical FEL
Phases

some cases, directly match the AACE
International estimate classifications. As
such, one must not think of these data as
corresponding directly to particular
AACE International classifications, but
rather the range of possible AACE
International classification requirements
enforced by this set of companies at a
particular gate. However, the authors
argue, that as an industry sample, they
do provide a good approximation of the
AACE International estimate classes
(which are approximations themselves).
In addition, a review of industry
project systems have many similarities to
the AACE International RP 18R-97. For
example, as projects move closer to full-
funds authorization, the accuracy of the
estimate relative to the expected
outcome should improve, as well as the
requirements for greater definition and
knowledge regarding the project scope
and execution plan. Based on this
review of these industry systems, figure
5 merges the AACE International
Estimate Classification system with the
Stage-Gated Work process based on this
review of company standards and
norms.
In general, estimates at the end of
appraise (or FEL 1) correspond to AACE
International Class 5, estimates at the
end of select (FEL 2) correspond to AACE
International Class 4, and estimates at
the end of define (FEL 3) correspond to
AACE International Class 3.
sample of projects executed by the
process industries over a 20-year period
from 1988 to 2008. Cost data for the
project sample was collected as part of
an overall industry benchmarking effort
as directed by Independent Project
Analysis (IPA) and managed through IPA’s
project evaluation system benchmarking
process (PES® is a registered trademark
of IPA).
For the project data collection,
interviews were conducted by IPA
project analysts with the project teams
with over 2,000 variables captured to
reflect each project’s technical scope,
characteristics, use of best practices, and
cost/schedule outcomes. The industry
dataset employed for this study included
462 completed large capital projects for
which estimated costs were collected at
the various FEL phases and actual costs
at project completion.
Figure 6 presents summary
characteristics for this industry sample.
The industry projects were executed by
62 different owner companies and
include a range of project scopes, project
types, and locations. The industry
sample consisted primarily of large
capital projects with a mean cost of $64
million (with a range from $10 million to
$3.3 billion). The projects were executed
primarily in North America, but also in
Europe, Asia, and the rest of the world
(ROW)—primarily the Middle East and
South America.
For each project, cost estimate data
were collected at four distinct points: at
the end of each FEL gate (estimated
costs), as well as at project completion
(actual costs). Cost deviation metrics
were calculated based on equation 1.
Project costs were normalized to a
constant currency and time period to

Database Description and Metrics

Calculation
The authors quantified the estimate
accuracy ranges for industry from a Figure 6 — Industry Database of Projects by Project Type and Industry Sector

32 COST ENGINEERING NOVEMBER/DECEMBER 2012

remove any bias driven by general price
escalation and currency fluctuation. The
costs collected included all project
management, engineering, materials,
and labor costs expended on the project
from the beginning of project definition
through the completion of construction
(as defined as mechanical completion).
For the estimates, included were
both the base estimate as well as any
estimated contingency included in the
estimate. Estimated escalation was
excluded from this analysis as the
authors normalized both estimated and
actual costs to a constant dollar basis.
In terms of scope and project
changes, the authors included all costs
associated with normal scope
development and project changes,
assuming there were no changes to the
underlying business case and overall
project scope. Therefore, costs
associated with major scope changes
(e.g., increase in design capacity for a
unit) were excluded from the estimates
to maintain an “apples to apples”
comparison with the actual costs ,while
costs associated with normal scope
development (e.g. increase equipment
spacing to improve unit maintenance)
was included in this analysis.
The currencies were normalized
using published exchange rates based on
the estimate month [15]. For actual
costs, currencies were normalized based
on an average exchange rate during the
project execution schedule. Project
costs were also adjusted to remove the
effects of force majeure and other
uncontrollable external effects (e.g.,
labor strikes, natural disasters, etc.).
greater than 0.00 indicate an overrun
from the estimated costs, while values
less than 0.00 indicate an underrun of
actual costs versus the estimated costs.
Figure 7 presents the distribution of
cost deviations for the FEL 1 cost
estimates (or Class 5 estimates). At FEL
1, the objective is to develop an order of
magnitude/conceptual estimate to
determine the reasonableness of a
project’s business case. These estimates
generally serve as inputs into financial
models (e.g., to calculate net present
value or return on capital) to assist
business in allocating capital to the most
profitable business opportunities.
As shown in figure 7, the accuracy of
industry estimates at FEL 1 are extremely
variable with the cost deviation ranging
from −50 percent to over 200 percent at
the extremes. Industry will tend to
overrun these estimates with an mean
overrun of 71 percent and median
overrun of 38 percent. The standard
deviation of 118 percent indicates the
wide range of variability for industry FEL
1 estimates.
Equation 1
The probability distribution for FEL 1
estimates indicates a log-normal (or right
skew) distribution with a majority of FEL
1 estimates overrunning against the final
costs. The author’s note that the
industry sample size for FEL 1 estimates
is smaller than the other estimates
primarily because of the difficulty in
obtaining reliable early estimates from
Industry.
Figure 8 presents the distribution of
cost deviations for a set of estimates
published at FEL 2, as compared to the
final costs. This industry sample includes
462 projects executed from 62 owner
companies.
As discussed in the background
section, there exists among companies
some variability with regard to the
estimating requirements at the end of
FEL 2. However, in most cases,
companies will require an estimate that
meets the AACE International Class 4
requirements (with perhaps a few
exceptions of requiring a Class 3
estimate).
For Industry, the mean cost
deviation for FEL 2 estimates was 15

Study Results
This section presents the cost
deviation statistics for each FEL
phase/AACE International Estimate Class
based on the industry project sample.
Figures 7-9 present the cost deviation
distribution for estimates at the end of
FEL 1, FEL 2, and FEL 3, respectively.
Each figure presents the overall
distribution of cost deviation, as well as
the summary descriptive statistics (i.e.,
mean, median, standard deviation).
Based on the cost deviation metrics
as calculated using equation 1, a value of
0.00 means that the estimated costs
matches the actual cost exactly. Values Figure 7 — Cost Deviation Results for FEL 1 Estimates Compared to Actual Costs

COST ENGINEERING NOVEMBER/DECEMBER 2012 33

 percent with a range of −61 to 224 18R-97 as summarized in table 2. This systems tend to systematically
percent. The median cost deviation is 5 comparison, presented in figure 10, underestimate projects during the initial
percent, indicating that Industry will highlights a number of disconnects with stages of the project definition process.
typically overrun the FEL 2 estimate. The respect to industry expectations and The mean cost deviation of 71 percent
standard deviation of 41 percent actual industry data. for FEL 1 estimates marks a substantial
indicates a relatively large degree of Perhaps the most striking difference departure from industry expectations.
variability for FEL 2 estimates. is the industry’s lack of precision for FEL From a cost estimator’s perspective,
Assuming a normal distribution, 1 estimates, as compared to the industry this degree of systematic
approximately 67 percent of industry FEL expectation. As discussed in the underestimating is not surprising given
2 estimates would fall between −26 and previous sections, it was industry’s that most projects (at that point in the
56 percent of the actual costs. As with expectation that while early estimates FEL process) have only established the
the FEL 1 estimates, the distribution for show a large degree of variability and most rudimentary project scope. From a
FEL 2 estimates indicates a left skew with imprecision, the overall accuracy (as business planning perspective, the
a greater percentage of FEL 2 estimates measured by the average cost deviation) results are troubling as they indicate that
overrunning rather than underrunning remains relatively constant through the the early estimates, which serve as input
the actual cost. estimate classes, with all industry into any business case and metrics (NPV,
Figure 9 shows the distribution of estimates (regardless of class) having a IRR, ROC), are systematically much lower
cost deviations from a set of estimates mean cost deviation at or near 0 percent. than the actual cost required for a typical
published at FEL 3, or full-funds The authors results show this project. From this perspective, these
authorization, against actuals. For most assumption to be incorrect as project early estimates may be insufficient to
industry project systems, the FEL 3
estimate will correspond to a Class 3
estimate; however, in some cases these
estimates may be closer to a Class 2
estimate.
As presented in figure 9, the mean
cost deviation for FEL 3 estimates is 8
percent with an overall range of −40 to
95 percent. The median value of 1
percent shows that the probability of an
overrun versus an underrun is virtually
identical with a 50 percent probability of
both. The overall variability is reduced
from the earlier FEL 1 and FEL 2
estimates, with the standard deviation of
30 percent. However, the authors note
that this degree of variability is much
greater than the ±10 percent confidence
level generally attributed to an Figure 8 — Cost Deviation Results for FEL 2 Estimates Compared to Actual Costs
authorization grade estimate. The
distribution of industry cost growth
shows a slight right skew, however, this is
much less pronounced that the FEL 1 and
FEL 2 estimates.

Comparison With AACE International

Estimate Classes
The results presented in figures 7 to
9 indicate that, as based on historical
performance from completed industry
projects, the actual cost deviation for
industry shows a greater degree of
variability and inaccuracy than current
expectations. As further analysis, the
authors have compared the industry
distributions presented in figures 7 to 9
against the expected accuracy ranges
taken from the AACE International RP
Figure 9 — Cost Deviation Results for FEL 3 Estimates Compared to Actual Costs

34 COST ENGINEERING NOVEMBER/DECEMBER 2012

serve as a robust input into any capital
allocation process.
A potential contributing factor to
this finding is that during the business
planning process, early estimates that
may be near the actual cost (or even
conservative/higher than the industry
cost) are at a disadvantage because the
high capital costs tend to “kill” these
projects because they may not meet a
financial hurdle to move into later FEL
stages.
In their review of estimating bias for
the mining industry, Bertisen and Davis
also consider this hypothesis noting that
for the mining industry, the scarcity of
project financing drives project sponsors
to enhance the project economics
through lower capital estimates.[3]
Further to this analysis, the authors
employed the Shapiro Wilk test for
statistical normality to test whether the
FEL 1 industry estimates, on average, will
overrun. As expected, the sample of
industry FEL 1 estimates failed this test
for statistical normality.
Figure 10 — Industry Estimate Accuracy as Compared to the AACE International RP 18R-97 Ranges
As with the FEL 1 estimates, the
industry sample of FEL 2 estimates
exhibited a lack of precision with these
estimates being, on average, 14 percent
above actual costs. While the overall
variability for FEL 2 estimates show an
improvement as compared to the FEL 1
estimates (the standard deviation for
these estimates is 41 percent), this
degree of variability is greater than the
AACE International Class 2 expectations.
The results for the FEL 2 estimates
are somewhat troubling, as it is in this
stage that most project systems will
require a capital project to have finalized
the project scope and location and
completed early engineering and design
sufficient to complete a process flow
diagram and heat and material balances.
Despite the requirements for closing
scope, Industry tends to systematically
underestimate the project costs.
For FEL 3 (or authorization grade)
estimates, both the precision and
accuracy of estimates improve from the
previous classes. Unlike the FEL 1 and
FEL 2 estimates, the FEL 3 estimates
COST ENGINEERING NOVEMBER/DECEMBER 2012
meet the AACE International Class 3
expectation of precision with a median
cost deviation of 1 percent. The mean
cost deviation of 8 percent indicates
some degree of right skew in the
distribution, however this skew is much
less pronounced than in the previous
estimate classes.
However, while the precision
improves, the variability of the estimates
(as measured by the standard deviation
of 30 percent) is much greater than
industry expectations. As discussed in
the review of estimate accuracy, the
majority of industry project systems will
require a “plus or minus 10 percent
estimate” for the authorization gate.
Given that any project which
exceeds the 10 percent threshold will
usually require a supplemental
authorization, one could argue that this
a ±10 percent threshold represents
either the minimum or maximum range
for the actual costs or a range in which
the majority of capital projects, under
normal circumstances, will be
completed.
35
 A review of industry data will show
that approximately 66 percent of
industry projects will be completed with
a cost deviation ranging from -22 to 38
percent. The probability is approximately
50 percent that any given estimate will
meet the ±10 percent threshold.
It is argued that the ±10 percent
estimate for authorization is an
estimating concept that has evolved
from its intended purpose. This concept
has developed multiple meanings that
may, in fact, conflict with each other.
For example, it is likely while this
definition may be used by the cost
estimator to simply describe the overall
quality of an estimate, it is considered by
the business or project sponsor as an
absolute range to predict the final costs.
If one accepts the latter definition, then
one must conclude that Industry is not
achieving its objectives with respect to
estimate accuracy. Another potential
conclusion is that the cost estimating
community, as a whole, is doing an
inadequate job in communicating the
potential range of actual costs for any
given project when they describe an
estimate in these terms.
In either case, these results point to
an overall lack of robustness for
Industry’s track record in quantifying risk
(generally through contingency) for
estimates. It is noted that the cost
estimating community, using the most
advanced risk quantification and
contingency setting techniques, will set
expectations to their clients (e.g., project
teams, business sponsors) that the final
estimate will be within ±10 of the final
estimate. It is also noted that this
expectation is only met 50 percent of the
time.
Unlike the FEL 1 and FEL 2
estimates, which are systematically
underestimated, it is noted that the FEL
3 estimates, on average, have a cost
deviation near 0 percent. However,
these estimates deviate from the AACE
International RP as they show a larger
degree of variability than expected with
respect to both underruns and overruns,
which indicates Industry is as likely to
underestimate risk as overestimate risk.
36 COST ENGINEERING NOVEMBER/DECEMBER 2012
Figure 11 — Cost Deviation Results For FEL 3 Estimates With Finalized Project
Definition Compared to Actual Costs
Discussion: The Importance of the Level
of Definition
As presented in figure 10, the results
from our industry survey show a
disconnect between industry’s
expectation of estimate accuracy and
precision as compared with actual
project data. Given the degree of
difference between the industry
expectations and historical data, a
number of contributing factors to this
difference are proposed. Potential
factors are summarized as follows:
• There could be systemic issues for a
particular company or industry that
contributes to a pattern of
overrunning or underrunning that
combine to produce a broader
distribution of variability.
• There are execution issues that
occur on projects that one is not
controlling, such as poor
management of resources, design
changes, poor labor productivity,
poor planning, and/or a host of
other possibilities that drive the
larger than expected cost variations.
• The quantification of risk (through
the contingency setting exercise) is
not robust enough to properly
identify and quantify risks to the
project [8]. And,
• The cost estimates are being
developed with deficient inputs,
such as incomplete project design or
underdeveloped project execution
plans.
These points (as well as many other
potential contributing factors) likely
contribute to the overall variability
experienced by Industry. However, the
authors will focus on the last point given
that IPA, as part of it project
benchmarking, routinely collects the use
of particular engineering and design
practices employed during project
definition.
IPA has developed a model to
measure the completeness of a project’s
scope and key deliverables at
authorization. This model, which is
termed the Front-End Loading (FEL)
Index is a quantitative measure of how
well defined a capital project is at
authorization. It is noted that AACE
International RP 18R-97 references the
“level of project definition” that
corresponds to each estimate class.
These data are presented in figure 3 and
are expressed as the percentage of
project definition complete.
IPA’s FEL Index is a more robust
measure of the “level of project
definition” as it measures the status of
key deliverables related to the project
scope (e.g., process and instrumentation
diagrams, plot plans). From a cost
estimating perspective, the FEL Index is
useful as it provides a measure of the
completeness of the project inputs used
to develop the estimate. Using the FEL
index, one is able to segregate industry
projects which have substantial gaps and
uncertainties in their design, versus
those projects which have completed all
key deliverables to a best practical level
of project definition.
Figure 11 presents the distribution
of cost deviations for FEL 3 estimates
from a subset of industry projects that
had completed all necessary design
engineering deliverables to support the
authorization estimate. As presented in
figure 11, the cost deviation for this
sample of 253 projects shows
significantly less variability (i.e., greater
amount of precision) than the overall
industry FEL 3 distribution.
The mean cost deviation for FEL 3
estimates that have achieved an
appropriate level of engineering
definition is 2 percent with a range of
−35 to 70 percent. The standard
deviation of 21 percent indicates that
approximately 66 percent of this
industry subset will be completed with
finals costs within ±20 percent of the
estimated cost.
Using a standard t-test for statistical
difference, the authors found statistically
significant differences for both the mean
and standard deviation of both project
sets (Pr < 0.0001 and 2*Pr < 0.0001,
respectively) between the well-defined
project group and projects group that
had gaps in engineering
The results in figure 11 clearly
demonstrate that to improve estimate
accuracy and precision, an organization
must first focus on project inputs and
completing those key deliverables which
are required to produce an accurate
estimate. This conclusion is not new to
the industry and has been a major
finding in a range of studies focusing on
the outcomes of capital projects.
[5,12,13,14] Given these results, one
must conclude that Industry should first
and foremost focus on project
fundamentals to improve overall
estimate accuracy.
Conclusion
The authors’ data demonstrate that
the AACE International estimate
classifications express a set of ranges
that roughly correspond with the degree
of confidence in a given point estimate.
However, the current level of variability
expected by industry (as quantified in
the AACE International RP 18R-97) does
not represent the overall variability
experienced by industry as based on a Capital Cost Estimation. The
review of historical project data. Engineering Economist: 53, (April-
Given this finding, one concludes June 2008): pp 118-139.
that the estimating community, as it is 4. Chuong, Yuteck. Project Estimate
currently presented to the sponsors and Reviews for Oil and Gas Projects.
gatekeepers of projects, is setting the 2007 AACE International
expectation of estimate accuracy that is Transactions. Morgantown, WV
not sustainable when compared to 2007.
historical performance. In most cases, 5. Dysert, Larry. Scope Development
the project stakeholders have an Problems in Estimating. 1997 AACE
unfounded sense of confidence that an International Transactions.
estimate will accurately predict final cost Morgantown, WV: AACE
within its published ranges. International, 1997.
This disconnect between 6. Dysert, Larry. Is “Estimate Accuracy”
expectations and reality is difficult for an Oxymoron. 2006 AACE
both the cost estimating community and International Transactions.
the project management community as Morgantown, WV: AACE
a whole. Would a decision board feel International 2006.
comfortable authorizing a project with a 7. Hollman, John, editor. 2006. Total
Class 3 estimate if it knew that there was Cost Management Framework: A
a less than 50 percent chance that it Process for Applying Skills and
would actually come within its published Knowledge of Cost Engineering.
range? The authors suspect that this is AACE International. Morgantown,
not the case. WV.
However, the authors note that one 8. Hollman, John. Estimate Accuracy:
tool that is available to the estimator and Dealing with Reality. 2012 AACE
the gatekeeper is the knowledge that International Transactions.
those chances greatly improve if the Morgantown, WV 2012.
project has completed all key design 9. Jambhekar, Vijay and Stephen D.
deliverables at authorization. This means Weeks. Estimate Review and
moving forward without any open scope, Assurance – Owners Challenges.
having the required deliverables to 2007 AACE International
support quantity take-offs, and having a Transactions. Morgantown: WV
functionally integrated team that is 2007.
aligned on that scope. Even then, 10. Lavingia, Nick. Improve Profitability
opportunities still clearly exist to Through Effective Project
improve estimate accuracy to the point Management and TCM. 2003 AACE
at which industry is actually capable of International Transactions.
achieving the AACE International Morgantown: WV 2003.
classification standards in practice. ◆ 11. Lavingia, Nick. How to Create a
World Class Project Management
Organization. 2006 AACE
REFERENCES International Transactions.
1. AACE International. Cost Estimate Morgantown: WV 2006.
Classification System -- As Applied in 12. Merrow, E., Kenneth S. Phillips, and
Engineering, Procurement, and Christopher W. Myers.
Construction for the Process Understanding Cost Growth and
Industries. Recommended Practice Performance Shortfalls in Pioneer
18R-97. AACE International. Process Plants. RAND/R-2569-DOE,
Morgantown, WV, (latest revision). Santa Monica, CA. The RAND
2. AACE International. Cost Estimate Corporation, 1981
Classification System – Cost 13. Merrow, E. Understanding the
Estimating and Budgeting. Outcomes of Megaprojects: A
Recommended Practice 17R-97. Quantitative Analysis of Very Large
AACE International, Morgantown, Civilian Projects. RAND/R-3560-
WV September 2003. PSSP, Santa Monica, CA, The RAND
3. Bertisen, J. and Graham A. Davis. Corporation, 1988.
Bias and Error in Mine Project
COST ENGINEERING NOVEMBER/DECEMBER 2012 37
 14. Merrow, Edward W. Industrial ABOUT THE AUTHORS
Megaprojects. John Wiley & Sons,
Inc. New York, NY 2011.
15. Oanda Currency Converter,
http://www.oanda.com/currency/c
onverter/. Alexander Ogilvie is Fredrick P. Biery is
16. Smith, Mark A. and Richard L. with Independent with Independent
Tucker. An Assessment of the Project Analysis. He Project Analysis. He
Potential Problems Occurring in the can be contacted by can be contacted by
Engineering Phase of an Industrial sending an e-mail to: sending an e-mail to:
Project: A Report to Texaco, Inc. aogilvie@ipaglobal.com fbiery@ipaglobal.com
Austin, TX: The University of Texas,
1983.
17. The Construction Industry Institute.
Pre-Project Planning Handbook. CII
Special Publication 39-2. 1995. Robert A. Brown, Jr., is
18. Whiteside, James D. and Tyler with Independent Paul Barshop is with
Humes. Front-End Engineering and Project Analysis. He Independent Project
Design: Influence Over a Project’s can be contacted by Analysis. He can be
Outcomes. 2010 AACE International sending an e-mail to: contacted by sending
Transactions. Morgantown, WV: rbrown@ipaglobal.com an e-mail to:
2010. pbarshop@ipaglobal.com

2013 CONSTITUTION AND BYLAWS AMENDMENTS EXPLAINED

The Governance Task Force created a PowerPoint presen-
tation to explain its proposed amendments to the AACE Con-
stitution and Bylaws. The PowerPoint presentation may be
viewed at http://www.aacei.org/mbr/gtf_recommendations.ppt
An online webinar is set for 8 a.m. Eastern Time on Nov.
13 at https://www2.gotomeeting.com/register/251679458.
Registration is free but does require advance sign up.
AACE International Board of Directors is recommending
that members approve provisions to provide more direct rep-
resentation in the Association’s decision-making process by its
three associate boards. In addition, the Governance Task Force
recommends the creation of an Executive Committee to han-
dle routine administrative and executive functions requiring
Board of Directors attention. If approved by the membership
in an election in 2013, the net effect will be to expand the AACE
International Board of Directors by three members in 2014-15.
The recommended amendments include a provision for
direct representation by the three associate boards, Technical,
Education and Certification on the Board of Directors. Cur-
rently, those entities are represented by a single officer, Vice
President-TEC (Technical, Education and Certification).
As a result of the direct representation on the Board of Di-
rectors, the current position of Vice President-TEC would be
phased out.
Each associate board’s representative to the Board of Di-
rectors would be nominated by each associate board’s respec-
tive nominating committee and approved by two-thirds of the
respective associate board. The new positions would be enti-
tled Vice President-Technical Board, Vice President-Education
Board and Vice-President-Certification Board. All would serve
as ex-officio members of their respective associate boards and
these positions could be filled by the current associate chair if
desired by the associate board.
The Second Recommended Change in the Board of Direc-
tors structure would be splitting the current Vice President-Re-
gions into two positions, Vice President-North American
Regions and Vice President-International Regions. Consistent
with the direction of the current strategic direction to pursue
international growth, this structural change will assure that
there will always be international representation on the Board
of Directors while also assuring adequate attention is provided
for the sections in North America. The two vice presidents for
regions will be elected on alternate years for continuity pur-
poses. Regional representatives to the Board of Directors will
remain unaffected.
A Third Recommended Change is the creation of an Exec-
utive Committee comprised of the officers of the Association.
The Executive Committee will recommend actions for consid-
eration of the entire Board. Any actions recommended will re-
quire a subsequent two-thirds approval by the entire Board of
Directors by electronic ballot. The entire Board of Directors
focus on strategic and policy issues when we are deliberating
as a Board of Directors. “Our task force firmly believes that
these recommended amendments are in the best interests of
the Association in achieving our mission and vision. At the
same time we want to assure that all members have an oppor-
tunity to ask questions and express their opinions prior to vot-
ing on the recommendations from February 1-March 15,
2013,” Task Force Chair, Steve Revay stated.

38 COST ENGINEERING NOVEMBER/DECEMBER 2012

 TECHNICAL ARTICLE

Common

in Dealing With Project Risk

Joseph A. Lukas, PE CCE

Abstract: Unfortunately, many people using risk management do not fully under- that should be followed when dealing
stand basic risk concepts and therefore use incorrect techniques in preparing and with risks. Here are the processes that are
implementing risk management plans. The author has reviewed and critiqued to be done as part of risk management
client risk management process and procedures, along with risk management [3]:
plans for projects, and the same mistakes reoccur on a regular basis. Based on
these reviews, this article will present the top ten errors people make in dealing • Prepare  a  Risk  Management  Plan—
with project risks and how these mistakes greatly reduce the value of risk man- This should be completed early in the
agement. The author has also done numerous projects where risk management project planning phase. First decide if
was successfully used, and this article will also discuss effective risk techniques risk management is needed. On
that should be used on all projects. If you think you know how to deal with project some small, simple projects it may
risks, read this article to see if you are making any of these common errors. You
not be value adding. But for most
may be surprised! This article was first presented as RISK.944 at the 2012 AACE
International Annual Meeting in San Antonio, Texas. projects risk management is needed.
The risk management plan should
Key Words: Contingency, projects, risk management, and schedule document the techniques, tools and
responsibilities for risk identification,
risk analysis, risk response planning

W
hen the word risk is used
in casual conversation it
invariably has a negative
connotation: risk is bad
and should be avoided whenever
possible. The dictionary reinforces this
negative connotation by defining risk as
the possibility of loss or injury, or
someone or something that creates or
suggests a hazard [1].
However, for project management
risk has a less threatening and more
realistic definition as, “any uncertain
event or condition that, if it occurs, has a
positive or negative effect on a project’s
objectives” [2]. In this expanded scope of
understanding, risk is associated with
uncertainty—the possibility that things
will not go as planned because of
uncertain risk events, which can have
good or bad impacts on the project.
Risk management includes the
processes of identifying potential project
risks, analyzing risks, planning responses
to the most important project risks, and
also monitoring and controlling risks over
the life of the project. The goal of risk
management is to maximize the effects of •
potential opportunities, and minimize or
eliminate the threats to the project
objectives.
Risk Management Process
Before exploring the common errors
made in dealing with risks, let’s review
the suggested process steps for managing
project risks. This will ensure a common
understanding of the sequential actions
and risk monitoring and control.
• Identify the Project Risks—The most
commonly used techniques are
brainstorming and checklists. As part
of risk identification also identify any
triggers for each risk, which are early
warning indicators that the risk event
may occur.
Analyze  the  Project  Risks—The
simplest technique is assigning a
value for the probability and impact
of each risk and calculating the risk
factor. The risks are then prioritized
and the highest scores are the
threats and opportunities that
should be actively managed as part
of risk response planning. On larger
projects advanced risk analysis using

COST ENGINEERING  NOVEMBER/DECEMBER 2012 39

 quantitative techniques, such as
Monte Carlo analysis, may be
appropriate.
• Plan  Responses  for  Project  Risks—
The project team should decide on
actions to be taken to deal with the
threats and opportunities. The
responses should include the actions
to be taken to deal with the risk
event before it may happen, along
with contingency plans that will be
implemented if the risk event
happens. And,
• Monitor  Control  Risks—Managing
risks is an ongoing process over the
entire life of the project. This
includes identifying and dealing with
any new risks.
Common  Errors  Made  in  Managing
Project Risks
The top ten errors project teams
make with risk management will now be
listed. As noted, this is based on
consulting work with numerous clients.
The list covers the mistakes in sequential
order when doing risk management on a
project, and is not intended to be a
prioritized list of errors.
Here are the top ten risk errors made •
on projects: • Competitive market conditions for a
• Not considering opportunities;
• Confusing risk causes, events, and
impacts;
• Using checklists and not looking for
other possible risk events;
• Underestimating impacts;
• Not using 100% probability during
project planning;
• Not considering sensitivity with risks;
• Calling risk response planning
mitigation;
• Not considering contingency plans
along with response plans;
• Not making team members
responsible for specific risk events;
and,
• Not making risk management an on-
going process.
The rest of this article will cover each projects? That happens when specific
risk mistake in more detail, along with plans are not developed to make the
effective risk techniques that should be opportunities reality.
used on all projects.
40 COST ENGINEERING NOVEMBER/DECEMBER 2012
Error #1: Not Considering Opportunities
Most project teams view risk as bad •
and to be avoided whenever possible,
and therefore ignore the possibility that •
something beneficial might happen as a
result of a risk event. However, successful
project teams need to deal with all
project risks, and using the simple •
definition below will help you focus on
both good and bad risk events:
PROJECT RISKS = OPPORTUNITIES +
THREATS
Consequently there are two kinds of
risks to consider on projects—threats and
opportunities. Threats are easier to
understand because when they occur
there is a negative impact on the project
objectives. In contrast, a favorable or
advantageous impact on project
objectives when a risk event occurs is an
opportunity. Dealing with major threats
obviously needs to be done on projects,
but don’t forget to consider and take
advantage of opportunities. Many people
have a difficult time thinking of project
opportunities, so here are some
examples from recent projects:
Special pricing offered by a supplier.
specific service.
• Sudden availability of a key resource
for a short time period because of
another project being postponed.
• Availability of some needed
equipment (such as servers or
desktop computers) from another
company that is downsizing their
operations. And,
• Availability of a government •
investment tax credit for work done
before a specified date.
Most project teams do develop plans
to deal with threats, but unfortunately
very few plan for opportunities. That is a
mistake because planning will let you take
advantage of these opportunities if they
do occur. How many times have you
heard about ‘lost opportunities’ on
Listed next are the four risk response
strategies for opportunities [4]:
Exploit – take steps to make sure the
opportunity happens.
Share – shares the opportunity with
another group that is better
positioned to take advantage of it for
the benefit of the project.
Enhance – take steps to increase the
probability and/or the positive
impact of the opportunity (note that
increasing the probability and/or
impact values increases the risk
factor value). And,
• Accept – take no steps to actively
pursue the opportunity, but take
advantage of the opportunity if it
happens.
Let’s look at an example for each of
these response strategies, and how the
identification and planning for a specific
opportunity might look:
• Exploit—Your resource plan includes
adding Susan, a senior electrical
engineer, starting in January, when
her work on another project will
finish. It’s now early November, and
you find out the other project has
been cancelled. Looking at your
schedule, you realize changing the
sequencing of some project work will
allow you to use Susan now and save
three weeks in your project
schedule. You contact the resource
manager and request Susan be
immediately reassigned to your
project. Reworking your schedule
and immediately requesting Susan is
exploiting the opportunity created by
the cancellation of the other project.
Share—During the planning phase
for your laboratory renovation
project, you discover a new flexible
piping technology that can reduce
service piping installation time for
the lab benches by 30% with minimal
cost impact. You preliminary plan is
based on using an internal
maintenance group for the piping
work. However, they don’t have the
knowledge and experience in
working with the flexible piping
technology. Since the lab bench
piping is on the critical path and the
client sponsor wants to improve the
end date, you decide to outsource
this work to a contractor with
 Figure 1 — Effective Risk Identification Tool
expertise in flexible piping systems,
and release the internal
maintenance group for work on
other projects. This is a share since
you are taking advantage of the
opportunity (using flexible piping) to
shorten the schedule, and the
contractor is making a profit on the
work.
• Enhance—Your company has been
selected as the painting contractor
for a bridge renovation project. The
project specification calls for a paint
with a 15-year life. However, your
company has developed a new epoxy
paint that will last 30 years or more,
and it only costs 30% more than the
standard paint (but has a higher
profit margin since it is a proprietary
product of your company). You notify
the project manager, Rita, of this
opportunity. She likes the added
value, but the state Department of
Transportation (DOT) Director will
have to approve the change. Rita
suggests a meeting with the DOT
project engineer, Jim, to discuss the
benefits and enlist his support in
getting the change approved. Rita is
confident Jim will support the scope
change and help obtain the
necessary approval within the DOT.
This is an opportunity for the project
since the better paint will last
substantially longer, plus an
opportunity for the painting
contractor to earn some extra profit.
The action of meeting with Jim and
getting his support increases the
probability of the scope change
being approved.
• Accept—The testing team leader
informs you there is a chance the
integration testing on your software
development project will not take as
long as the plan shows, but the
probability is low and best case it
might save up to one week for the
one year project. You decide to
record the opportunity in the risk
register, but take no other actions at
this time since the impact to the
project is minimal.
Error  #2:  Confusing  Risk  Causes,  Events
and Impacts
The first step taken by a team in
implementing the risk management plan
is risk identification, which is determining
which risks may affect the project and
understanding the characteristics of each
specific risk event. Common tools used by
many project teams for risk identification
are checklists and brainstorming along
with interviews of key project
stakeholders.
However, a common error during risk
identification is not differentiating
between risk causes, risk events, and
impacts. This failure can dilute the risk
COST ENGINEERING  NOVEMBER/DECEMBER 2012
identification process by not providing a
focus on actual risk events and this is
important because risk events have one
unique feature, and that is uncertainty!
• A risk  cause is a definite event or
circumstance that exists. A cause is a
fact and has no uncertainty. For
example, multiple job opportunities
for top notch programmers, resulting
from a shortage of computer
programmers in the region, would be
a risk cause.
• Risk  events are simply things that
may or may not happen. An example
of a risk event would be losing one or
more key software programmers to
another software company who
offers them more money.
•      Risk impacts are the results that will
occur if the risk event happens. For
the example above, if a key software
programmer leaves for a higher
paying job, it probably will have an
impact on the project cost and
schedule.
An effective risk identification
technique is expressing each project risk
as a sentence. Your sentence should
include the cause, risk event, and impact
as shown below [5]:
41
 Because of <Cause>, <Risk Event> Could To initially develop the checklist for There are four possible
Occur, Resulting In <Impact> group of similar projects, get some consequences for any risk event:
experienced project personnel together
Note that a risk event may have more and brainstorm the risk events they have • cost;
than one cause, and more than one encountered on projects. The checklist • schedule;
impact. This technique can be used to should be considered a dynamic • functionality; and
fully describe each identified project risk, document, and as previously unidentified • quality.
and is an effective method to help keep risk events occur on other projects, these
the project team focused on the actual risks should be added to the checklist. Each needs to be considered when
risk events. The lesson learned sessions done at deciding the risk impact value. However,
project completion is also a good place to it’s also important to prioritize these
One tool that can be useful during identify new risk events that should be impacts based on what’s most important
brainstorming sessions is the use of post- added to the risk checklist. for the project. In some cases it may be a
it notes as shown in figure 1. A separate When using checklists, it can be specific end date that must be met
sheet of chart paper is used to list risk helpful to segregate the risks into because of a government regulation, or
causes. The post-it note format has the categories. The exact grouping will maybe a fixed budget that cannot be
risk event description in the center, and depend on the needs of the organization. exceeded. If the most important driver
the cause numbers listed on the left. The One set of categories frequently used is for a project is meeting a completion
four possible impacts are listed on the by project phase: initiation, planning, date, then a risk event that will delay the
right side of the post-it note and the design, build, test, and deployment. For completion date should be assigned a
applicable impacts are circled. Risk the civil engineering company doing road bigger impact value compared to a risk
categories can be added at the top, and construction, a grouping that could be event that just impacts cost.
the post-it note is also used for risk used is design, government/regulatory, Shown in table 1 is the impact table
analysis, with values assigned for contractor, suppliers and weather. for a specific software development
probability and impact, and the Some teams have used the project [6]. For this project the budget
calculated risk factor. This format has categories of cost, schedule, quality and was fixed and was the most important
been successfully used on numerous functionality to organize their risks. This project driver—the budget could not be
projects. isn't a good idea, however, since these exceeded. Functionality and quality were
four categories are also the risk impacts, secondary drivers, and if the budget was
Error #3: Using Checklists and Not and many risk events will have more than in jeopardy, functionality reductions
Looking for Other Possible Risks one impact. What that means is that if could be considered. Schedule was least
Many organizations do similar you try using this set of categories you important, since the company had
projects over and over. Examples include won't be able to easily place risk events existing software in use and could
a civil engineering company specializing with multiple impacts into a single continue using the software if the
in road construction, or a computer category. completion date slipped.
programming company that only does As discussed above, a risk checklist is Let’s consider a sample risk event
data migration projects. Most of the risk not inherently a bad thing, but you must that has a cost impact score of 8, a
events will be the same from project to make sure the identification of any schedule impact value of 0, and impact
project, and in these situations the use of additional risks is done in conjunction scores of 4 for functionality and quality.
a risk checklist is effective. with use of the checklist. Use The obvious question is what impact
The problem with this approach is brainstorming to find new risks. One value would the project team use for this
complacency—making the false technique is to award a simple 'prize' risk event? Is it the average of the four
assumption that the only possible risk such as a car wash or movie ticket to the impact scores, which would be 4? The
events are included in the checklist. The team member who identifies the most answer is no! The correct method is using
checklist can certainly be a good place to new risk events not already on the the highest value for the impact score. In
start, provided it is not the only thing checklist. this case that value is 8, which is the cost
done to identify risks. Your team also impact score.
needs to ‘scan the horizon’ to identify any Error #4: Underestimating Impacts Be careful when discussing
other possible risk events not on the One area that some project teams prioritization of project drivers with
checklist that may impact the project. struggle with is underestimating the clients. The answer you’ll frequently here
For example, the civil engineering impacts of risk events. This may be is that everything is equally important!
company may have an excellent risk because of the optimistic nature of That’s an easy answer but most often not
checklist, but the company is awarded a project teams—especially early in the really true. If a client has a project with a
new project and the road will cross an old project. When the impacts are fixed end date that MUST be met, cost is
abandoned landfill. This will undoubtedly understated the risk factor calculations probably not as much a consideration. It
create some unique risk events for this become skewed, the prioritization may still be important—but not as much
new project which are not currently on becomes flawed, and the entire risk as schedule. A perfect example of this is
the checklist. management process is threatened. the year 2000 projects (Y2K). There was a

42 COST ENGINEERING NOVEMBER/DECEMBER 2012

definite end date of December 31, 1999, plan for the requirements document considered. Monte Carlo is a risk
and for clients that was the project driver. deliverable, listing the specific people at simulation technique, and can help better
each plant site needed for requirements understand the project risk events. With
Error #5: Not Using 100% Probability definition, and the dates of the planned risk simulation, when discussing the
During Project Planning meetings to define requirements. The probability and impact of a risk event,
At the beginning of this article a risk actions also included a contingency plan instead of a single value being identified
event was defined as having uncertainty, where if a person missed a meeting, the there is a most likely value, along with an
so it may be confusing now stating that a project sponsor was immediately notified optimistic and pessimistic value. Risk
risk event can have 100% probability and the sponsor contacted the simulation software can show the
during project planning. 100% probability appropriate plant manager to resolve the frequency of possible outcomes and a
means no uncertainty—it’s a fact. This is problem. By the time the project planning cumulative cost and/or schedule with
not a conundrum because before phase was completed, the risk probability overrun probabilities.
completion of project planning you will of having missed requirements dropped
take actions to deal with some of the from 100% down to around 30%. Error #7: Calling Risk Response Planning
project risks, and these actions should A final caution—don’t overuse the ‘Mitigation’
reduce the risk event probability below 100% probability approach. It should only A common mistake made by many
100%. Project execution cannot begin be used where the risk really will happen project organizations is calling risk
with risk events that have a probability of unless actions are taken and the impacts response planning ‘mitigation.” That is
100% because they are facts and the are major. Otherwise, you run the risk of incorrect, since mitigation is just one risk
project plan must be prepared to deal losing credibility. response technique and is specifically
with these facts. used for dealing with risks that are
However, giving a risk event a Error #6: Not Considering Sensitivity threats. Another term often used to
probability of 100% during project with Risk Analysis describe risk response planning is ‘risk
planning will obviously call lots of The simplest technique for management plan,’ and that’s also
attention to that risk event provided it qualitative risk analysis is assigning a incorrect since that is part of the project
also has a high impact—and that will value for the probability and impact of plan and describes what the team will do
either result in actions taken to reduce each risk and calculating the risk factor. to manage risks on the project. One final
the probability and/or impact to a lower The risks are then prioritized and the term that should not be used to describe
value, or changing the project plan to highest scores are the threats and risk response planning is ‘risk
eliminate the risk, or constructing the opportunities that should be actively assessment,’ which more correctly means
project plan to deal with the 100% reality managed as part of risk response identifying and analyzing risks—so the
of the risk event. planning. This simplified approach recommendation is to avoid use of this
An actual project example is a ignores the sensitivity around the term.
customized software development probability and impact values. In reality, The product of risk management is
project for use at company these are not fixed values, but are a range called the risk register and it includes the
manufacturing locations around the of values. list of identified risk, the risk analysis
world. Business was good and the On smaller projects qualitative risk results and the risk response plans
manufacturing areas were extremely analysis is usually ‘good enough.’ including contingency plans. The correct
busy. The project team correctly realized However, on larger projects advanced risk term to use for describing what steps will
a major risk event would be having analysis using quantitative techniques, be taken to deal with risks is, ‘risk
incomplete project requirements such as Monte Carlo analysis, should be response plan.’
resulting from an inability to get the
manufacturing representatives to take
time to meet and adequately explain
their requirements.
Incomplete requirements would
probably result in a solution design that
didn’t meet all of the manufacturing
plant needs. The project team gave the
risk of incomplete requirements a
probability of 100% and informed the
project sponsor the project would fail
unless this risk was dealt with during
project planning. A risk response plan
was developed and implemented for this
risk.
The actions taken included a
comprehensive schedule and resource Table 1 — Project Impact Table for a Sample Software Project

COST ENGINEERING NOVEMBER/DECEMBER 2012 43

 Table 2 — Risk Register Example
When risk mistake #1 was discussed,
the commonly used risk responses for
opportunities were described. Listed next
are the common risk responses for
threats [4]:
• Reduce (Also called Mitigation)—
This response seeks to reduce the
probability and/or impact of a
negative risk event. An example of
reduce is offering a performance
bonus to a key project resource to
mitigate the potential of the person
leaving the company before project
completion. The performance bonus
reduces the probability of the person
leaving.
• Eliminate (Also called Avoidance)—
This response looks to eliminate the
threat by taking actions to cause the
risk to go away completely. This is
usually done by eliminating the
causes. When you avoid, the risk is
gone! There is no need to do any
contingency planning, because the
risk no longer exists. An example of
avoidance is a developer looking to
build a new store decides not to
build in a certain town because of a
hostile town board which probably
will hold up the zoning approval.
• Transfer—This response shifts some
or all of the threat to a third party.
The risk doesn’t go away, but the
ownership and impact of the risk
belongs to somebody else. An
example of transference is using a
firm fixed price contract with a
vendor for the data migration work
44 COST ENGINEERING NOVEMBER/DECEMBER 2012
on a software project. This provides
the project team with a known cost,
and the threat of a cost overrun
belongs to the vendor (along with
the opportunity to complete the
work for less money and make more
profit).
• Accept—This response is deciding
not to do anything about the risk.
This may be because no other
actions are feasible, or the risk factor
isn’t significant. The response can be
active acceptance, which means
developing a contingency plan if the
risk event occurs. An example is
building a new store in upstate New
York in January. It will snow—so
active acceptance is the best
approach. You can’t control the
weather!
In summary, mitigation is just one
risk response technique, and it is
specifically used for threats. Do not call
risk response planning ‘mitigation’—the
correct term to use for describing what
steps will be taken to deal with risks is
‘risk response plan.’
Error #8: Not Considering Contingency
Plans Along With Response Plans
As mentioned with mistake #7, the
correct term to use for describing what
steps will be taken to deal with risks is
‘risk response plan,’ but don’t forget that
should also include a contingency plan!
To clarify the difference, the risk
responses are actions taken before the
risk event occurs to make the risk event
more likely to happen for opportunities
and less likely to happen for threats.
Contingency plans are more specifically
the actions that will be taken if and when
the risk event occurs. Risk response
planning should deal with both!
Let’s revisit the common risk
responses for threats covered in mistake
#7. The mitigation example was losing a
key project resource because of a person
leaving the company. The contingency
plan for this example would be
identifying a back-up resource (either
internal or a contractor) that could step in
and handle the work if the person does
leave the company. For the acceptance
example of building a store in upstate
New York in winter weather, contingency
planning could be adding some extra days
in the schedule as an allowance for non-
working days because of snow, and/or
including money in the budget for plastic
sheeting and space heaters so work can
continue even if snow occurs.
One final comment is that not all
risks need a contingency plan! The
example used for transference was using
a firm fixed price contract with a vendor
for the data migration work on a software
project. The risk impact now belongs to
the vendor so the project team doesn’t
need to worry about a cost overrun. The
example used for avoidance was a
developer looking to build a new store
deciding not to build in a certain town
because of a hostile town board. Since
the decision was made not to build in the
town with a hostile town board, the risk
no longer exists. In both cases a For each risk event, the responsible Second Edition, Simon & Schuster,
contingency plan is not pertinent. team member should provide a brief Inc., New York, (1979): pg. 1565.
status report on the risk response actions 2. A Guide to the Project Management
Error #9: Not Making Team Members taken and/or planned, along with Body of Knowledge, Fourth Edition,
Responsible for Specific Risks changes to the probability and/or impact Project Management Institute,
One observation is that some project values based on actions taken. The Newtown Square, PA, (2008): pg.
managers are not effective at delegating project team should also discuss if any 438.
work, and end up taking responsibility risk triggers have occurred, and whether 3. Lukas, Joseph A., It Works! Risk
where other team members could be any new potential risk events have Management on an IS Project,
used. Part of risk response planning is surfaced since the last meeting. Proceedings of the PMI Seminars
assigning ownership of each risk to an The key point is that risk and Symposium, Project
individual. management is an on-going process over Management Institute, Newtown
Assign each risk to a specific the entire life of the project! Watch for Square, PA, (2002): pg. 2.
individual on the project team, and the certain phrases that appear in people’s 4. Brady, David C., January 12, 2012,
project manager should minimize the conversation, such as “there is a chance Risk Treatment AACE® International
number of risks assigned to her/himself. that …” or “maybe this would cause Recommended Practice No. 63R-11
The assigned person should implement problems …” When you hear words like Public Review Draft, (Jan. 12, 2012):
the risk response plans, monitor the risk that, ask for clarification, and it may pgs 6-8.
and report on the risk status. Assign each become apparent that this is a new risk 5. Hillson, David. Project Risks,
risk to the team member who has the event and should be added to the risk Identifying Causes, Risks and Effects,
particular skills related to the risk. register. PM Network, (Sept. 2000): pgs. 48-
For example, a risk concerning the 51.
requirements elicitation work on a Conclusion 6. Lukas, Joseph A. Risk Management in
software project should be assigned to a Many people using risk management the Real World: A Look at an IS
business analyst, while a metallurgy risk use incorrect techniques in preparing and Project, Inside Project Management,
on a construction project should be implementing risk management plans. Element Journals, Rochester, NY.
handled by an engineer. This article has covered the top ten errors Volume 1, Number 6, (2001): pg. 13.
people make in dealing with project risks
Error #10: Not Making Risk Management and how these mistakes greatly reduce ABOUT THE AUTHOR
an On-going Process the value of risk management. Check how
Unfortunately, some project teams you are doing risk management on your
think of risk management as a task that is projects and see if you are making any of
started and completed as part of project these common errors. You may be
planning. The risk register is completed surprised! ◆ Joseph A. Lukas is with
and then filed away and forgotten. This is PMCentersUSA. he
the mistake—a risk management review REFERENCES can be contacted by
should be part of every project team 1. McKechnie, Jean (Editorial Staff sending e-mail to:
meeting! Shown in table 2 is a sample Supervisor), Webster’s New joe.lukas@pmcentersusa.com
risk register, which is a simple worksheet. Twentieth Century Dictionary,

2012-2013 NOMINATING AND AWARDS COMMITTEES

Past President Michael Nosbisch will serve as chair of the
2012-13 Nominating Committee, which will include Ginette B.
Basak P.Eng FAACE; Philips Tharakan Mulackal CCE EVP,
Michael B. Pritchett CCE CEP, and Stephen P. Warhoe PE CCE
CFCC FAACE. The Nominating Committee will also serve as the
2012-13 Awards Committee.
The responsibilities of the Nominating Committee are out-
lined in the Association Bylaws as follows:
Section 2—The Nominating Committee shall be responsible for: (a) se-
lecting one or more nominees for each office, (b) obtaining the consent of
each nominee to serve if elected, and (c) securing biographical data of each
nominee. The entire slate of candidates, complete with biographical data, shall
be sent by the Nominating Committee to the Vice President-Administration
no later than October 31st of each year.
Section 3—The Nominating Committee’s slate of candidates, complete
with biographical data, shall be made available to the membership along with
information regarding nominations by petition no later than November 15th
of each year.
Section 4—Other nominations for the office of Director or the office of
an Officer of the Association may be made by petition signed by at least twenty
(20) Members in good standing. The petitioners shall be responsible for (a)
obtaining in writing the agreement of the nominee to serve if elected, (b) se-
curing the biographical data of the nominee, (c) submitting the petitions, the
agreement, and the biographical data to be received by the Vice President-
Administration no later than December 15th of each year. Each candidate’s
name and biographical data shall be made available to the membership no
later than December 31st of each year.
To be eligible to vote, a member’s membership dues must be paid
through Dec. 31, 2012. Members also are restricted to voting for regional di-
rector candidates within the region where the member’s primary section affil-
iation is located, according to the AACE records as of Dec. 31, 2012. Electronic
online voting will open on February 1 and concludes at 4 p.m. Eastern US Time
Zone on March 15.

COST ENGINEERING NOVEMBER/DECEMBER 2012 45

AACE INTERNATIONAL ONLINE STORE
more online at www.aacei.org
Skills and Knowledge of Cost Cost Engineering
Engineering, 5th Edition, Revised The international journal
Scott J. Amos, Editor, 2007 of cost estimation,
This updated and expanded guide for cost/schedule control,
fundamentals is an excellent choice for anyone project management,
interested in a concise reference to all aspects of and total cost
the profession. The new 5th edition includes management.
twenty-seven chapters on estimating, Subscriptions are
manufacturing and operating costs, scheduling, accepted on an annual
planning progress and cost control, and much basis. An automatic
more. This is a very useful book for those studying benefit of AACE
for the certification exam. 450 pages International
1595-02zip - Download - US$50.00 member/US$80 nonmember membership, also
available to nonmembers.
Paper version available through our Amazon.com link
5060-07 - US$72.00 (US)
- US$171.00 (other countries)
CCC/CCE Certification Study Guide, 
Please add US$80.00 for airmail
3rd Edition - US$61 electronic subscription
Michael B. Pritchett, CCE, Editor, 2006
The AACE International CCC/CCE Certification Cost Engineers’ Notebook          
Study Guide provides an all-encompassing This CD-ROM is an important reference for any
reference text to prepare for the exam. The project or cost professional. It includes data and
CCC/CCE Certification Study Guide provides procedures related to basic skills and knowledge
background information on how to become that all cost engineers should possess, extensive
certified; gives those studying for the certification material on capital and operating cost estimation,
exam a single reference text that includes theory, and papers in four subject areas: cost control,
worked problems with answers, references, and a planning and scheduling, project management, and
full discussion of key topics; allows students to economic analysis and business planning.
maximize their study time; and provides a concise 4060-28zip - Download
overview of the fundamentals of cost and project management. - US$65.00 member/US$105.00 nonmember
1825-36zip - Download - US$50.00 member/US$80 nonmember
Paper version available through our Amazon.com link AACE International Recommended Practices
Cost Engineering Terminology; Cost Estimate
Classification System; Estimate Preparation Costs in
PSP Certification Study Guide, 1st Edition the Process Industries; Project Code of Accounts;
Peter W. Griesmyer, Editor, 2008 Required Skills and Knowledge of a Cost Engineer;
This study guide is intended to assist you in your Roles and Duties of a Planning and Scheduling
study and review of the overall topics as one step Engineer; Profitability Methods; plus many more.
toward successful Planning and Scheduling Profes- 4060-05zip - Download
sional certification. The outline provides a listing of - US$70.00 member/US$110.00 nonmember
the terms you should know & topics for which you
should have a good understanding of how to apply The Total Cost Management Framework
the concepts to solve problems. Each chapter also John K. Hollmann, PE CCE, Editor, 2006
contains sample exercises, which test your knowl- 4060-20zip - Download
edge of that chapter's concepts. Additional sample
questions are provided in an appendix. - US$50.00 member/US$70.00 nonmember
Paper Version available through Amazon.com
1820-38zip - Download - US$50.00 member/US$80.00 nonmember
Paper version available through our Amazon.com link 2012 AACE International Transactions 
EVP Certification Study Guide,  5220-12 zip -
2nd Edition Download
Ken Cressman, CCC EVP and Gary C. Humphreys, - US$75.00
Editors, 2009 member
This study guide is intended to assist you in your - US$80.00
study and review of the overall topics as one step
toward successful Earned Value Professional certi- nonmember
fication. The outline provides a listing of the terms
you should know & topics for which you should For CD-ROM
have a good understanding of how to apply the con- version please
cepts to solve problems. Each chapter also contains contact AACE
sample exercises, which test your knowledge of that International
chapter's concepts. Headquarters
1820-40zip - Download - US$50.00 member/US$80.00 nonmember
Paper Version available through our Amazon.com link
The
The AACE
AACE International
International Professional
Professional Practice
Practice Guides
Guides (PPGs)
(PPGs)
(PPGs) are a series of reference s that
consists of selected Cost Engin eering  Dennis M. Thompson, Editor, 2007
articles, AACE Inter national Transaction
papers, and other previously published
documents to which AACE has rights.
Price per PPG:  PPG#8: Contingency, 2nd Ed.
Download Member Price US$50.00
Download Non-Member Price US$70.00
Price for the PPG Package  Covers: Basis of Estimates; Labor Costs; Overhead and
includes all 21 PPGs:
Download Member Price US$874.00
Download Non-Member Price
US$1223.00
PPG#1: Contracts and Claims, 4th Ed.
James G. Zack Jr., Editor, 2008
Covers: Contract Administration; Management of
Construction Schedules; Schedule Control; Schedule
Float Ownership; Cost Control; Management of
Change; Cost Impacts; Productivity Impacts; Manage-
ment and Analysis of Delay; Concurrent Delay Issues;
Pricing of Delay; and more.
PPG#2: Risk, 2nd Ed.
Keith D. Brienzo, PE, Editor, 2007
Covers: Dictionary; Capital Investments; Cash Flow;
Competitive Bidding; Contingency Analysis; Contracts;
Cost Engineering; Currency Rates; Decision Trees; Eco-
nomic Analysis; Escalation; Human Factors; Manufac-
turing; Research & Development; Safety & Health;
Schedule; Technological Risk; and Value Engineering.
PPG#3: Cost Engineering in Aerospace and Avi-
ation                   Cost Control; Schedule Control; Quality Control; Doc-
Sarwar A. Samad, Editor, 1998
Covers: Aerospace and Aviation.
PPG#4: Planning and Scheduling, 2nd Ed.
Trevor X. Crawford, CCC, Editor, 2006
Covers: Planning; Schedule Development; Schedule
Management/Control; and Classics.
PPG#5: Earned Value, 2nd Ed.
Robert A. Marshall, Editor, 2007
Covers: Why Use Earned Value?; Basics of Earned
Value; Cost/Schedule Control System Criteria; Actual
Physical Percent Complete; Productivity and Earned
More AACE Publications at the Online Store - www.aacei.org
Value; Earned Value Reporting; Applications of Earned
Value Project Management; and more.
PPG#6: Construction Cost Estimating, 2nd Ed.
Dr. Douglas D. Gransberg, PE CCE, And Carla Lopez
del Puerto, CCC, Editors, 2006
Covers: Recommended Practices; Estimating Theory;
Conceptual, Parametric, and Range Estimating; Esti-
mating Factors and Indices; Estimating Material Costs
and Quantity Surveying; Estimating Labor Costs; Esti-
mating Equipment Costs; Subcontracting Costs; Esti-
mating Overhead and Indirect Costs; Profit,
Contingencies, and Mark-Ups; Estimating Interna-
tional Construction Costs; and more.
PPG#7: Cost Engineering in the Utility  Estimating Salvage/Residual Value; Estimating Sustain-
Industries, 2nd Ed.
Covers: Auditing; Cost Estimating; Cost Modeling;
Cost/Schedule Control; Generation Power Plant; Nat-
ural Gas Industry; Nuclear Power Plant; Other Energy
Related Topics; Planning and Scheduling; Project
Management; Utility Rates; and Utility Property Valu-
ation.
Kul B. Uppal, PE, Editor, 2005
Covers: General Topics On Contingency; Cost Estimat-
ing and Contingency; Risk Analysis and Contingency;
and Other Related Topics.
PPG #10: Project Delivery Methods, 2nd Ed.  Profit; Soft Costs; Bid/Estimate Reconciliation; and
Dr. Douglas D. Gransberg, PE CCE, Tammy L. Mc-
Cuen, and Keith Molenaar, Editors, 2008
Covers: Design-Bid-Build (DBB) – DBB Estimating, DBB
Scheduling, DBB Project Management; Construction
Management (CM) – CM Estimating, CM Scheduling,
CM Project Management; Design-Build (DB) – DB Es-
timating, DB Scheduling, DB Project Management; In-
ternational Project Delivery; Constructability; and
Partnering.
PPG #11: Environmental Remediation &  Using Value Engineering; Evaluating Competing Sustain-
Decommissioning, 2nd Ed.
Richard A. Selg, CCE, Editor, 2009
Covers: Environmental Remediation Planning and
Scheduling Methodology; Cost Estimating, Project Con-
trols, Cost Modeling, and Reporting; Contingency Man-
agement, Risk Analysis, and Environmental Regulations;
Benchmarking and Lessons Learned; Economics of En-
vironmental and Waste Management; Cost-Effective
Waste Minimization and Pollution Prevention; Design,
Construction Practices, and Other Related Topics.
PPG #12: Construction Project Controls
Dr. Douglas D. Gransberg, PE CCE, and James E.
Koch, Editors, 2002
Covers: Introduction to Construction project Controls;
ument Control; Computer Applications; and Interna-
tional Project Controls
PPG #13: Parametric and Conceptual  General Schedule Analysis Articles
Estimating, 2nd Ed.
Douglas W. Leo, CCC, Larry R. Dysert, CCC, and
Bruce Elliott, CCC, Editor, 2004
Covers: Parametric/Conceptual Estimating; Classifica-
tion; Methodology; Capacity Factoring; Process and
Non-Process Industries; and Systems
PPG #14: Portfolio and Program Management,
2nd Ed. 
Randy R. Rapp, PE CCE, Editor, 2007
Covers: Enterprise Management: General Imperatives and
Concerns; Asset Requirements Elicitation and Analysis;
Asset Planning and Investment Decision-Making; Asset Per-
formance Assessment and Change Management; and Pro-
gram Management.
PPG #15: Life-Cycle Cost Analysis
Dr. Douglas D. Gransberg, PE CCE, and Carla Lopez
del Puerto, CCC, Editors, 2007
Covers: Life-Cycle Cost Theory; Life-Cycle Cost Methods,
Determining Discount Rate; Estimating Capital Cost of
Design and Construction; Estimating Operating Costs;
ability; Life-Cycle Cost Risk Analysis; Life -Cycle Cost
Case Studies; Life-Cycle Cost Analysis in the Interna-
tional Context
PPG #16: Cost Engineering in the 
Global Environment
Kul B. Uppal, PE, Editor, 2007
Covers: General Topics on International Projects; Appli-
cable AACE International Recommended Practices; Cost
Estimating Methodology; Risk and Contingency; and
Miscellaneous Topics
PPG #17: Public Sector Estimating
Joseph L. Macaluso, CCC, Editor, 2007
Change Orders
PPG #18: Green Building
Joseph L. Macaluso, CCC, Editor, 2008
Covers: Recognition of Affects and Economic Costs on
the Environment; Formulating Ways of Addressing
Green Building Strategies and Associated Economic
Costs; Specific Green Building Strategies and Project
Costs; Budgeting and Justifying the Cost of Sustainable
Practices; Evaluating Competing Sustainable Strategies:
able Strategies: Other Techniques
PPG #19: Leadership and Management 
of People
John J. Hannon, CEP, Editor, 2008
Covers: Leadership; Teams; Leadership Roles; Motiva-
tion; and Ethics.
PPG #20: Forensic Schedule Analysis
James G. Zack, Jr., CFCC, Editor, 2008
Covers: Recommended Practice No. 29R-03 Forensic
Schedule Analysis; Synopsis of Recommended Practice;
Basics of Schedule Delay Analysis; MIP-Observational
Static Gross; MIP-Observational Static Periodic; MIP-Ob-
servational Dynamic Contemporaneous As-Is; MIP-Ob-
servational Dynamic Contemporaneous Split;
MIP-Observational Dynamic Modified or Recreated; MIP-
Modeled Additive Single Base; MIP-Modeled Additive
Multiple Base; MIP-Modeled Subtractive Single Simula-
tion; Non-CPM Schedule Delay Analysis Techniques;
PPG#21: Cost Engineering in the Process 
Industries
Kul B. Uppal, PE CEP, Editor, 2009
Covers: General Topics on Process Industries; Cost Es-
timating Methodology; Project Management; Inter-
national Projects; Scheduling; Construction Activities;
Risk Management; Project Controls; and Applicable
AACE International Recommended Practices.
 PROFESSIONAL SERVICES DIRECTORY

All AACE International Books are

"Print-on-Demand" Products
AACE has reformatted each of its print products (i.e., the
Skills and Knowledge of Cost Engineering; CCC/CCE Certifi-
cation Study Guide; EVP Study Guide; PSP Study Guide; and
the TCM Framework) to be produced by a subsidiary of
Amazon.com. Each of these AACE publications became
available online from Amazon.com in 2012. We maintain a
link from the AACE website, www.aacei.org, to Amazon.com
and will recommend additional products as well. ◆

INDEX TO ADVERTISERS ADVERTISE

IN THE
Acumen, page 4 COST ENGINEERING
YOUR JOURNAL
ARES Corporation, back cover
VISIBILITY
Bechtel Corporation, page 3
REACH the entire AACE International mem-
▲

•
D.R. McNatty and Associates, this page bership every month by placing an ad in the
Cost Engineering journal.
EcoSys, inside front cover
PLACE your products/services in front of
▲

•
Faithful+Gould, page 48 over 50,000 users each month with a banner
Infinitrac, this page ad at our website, www.aacei.org.
EXHIBIT at the AACE International Annual
▲

•
Management Technologies, this page
Meetings, and contact AACE International
Ron Winter Consulting, page 16 members face to face.

Sage Software, inside back cover CONTACT

Skire, Inc., page 16 Garth Leech
phone 1.304.296.8444 fax 1.304.291.5728
For additional information about the listed advertisers or about e-mail gleech@aacei.org
advertising with us, please phone Garth Leech, 1.304.296.8444
x122, or e-mail him at gleech@aacei.org
OR GO ONLINE AT www.aacei.org

COST ENGINEERING  NOVEMBER/DECEMBER 2012 49

 Contrary to rumor, there is no certified League of TCM Heroes. Individuals who have earned
certifications such as the Certified Cost Consultant (CCC)/Certified Cost Engineer (CCE), Certified
Estimating Professional (CEP), Certified Forensic Claims Consultant (CFCC), Earned Value Profes-
sional (EVP), Planning & Scheduling Professional (PSP) and Certified Cost Technician (CCT) are:
• Not able to leap tall buildings in a single bound • Not more powerful than a locomotive
• Not faster than a speeding bullet • Not in possession of x-ray vision
But the AACE International Salary Survey did show that certificate holders earn an average
of $5,000 per year more than non-certification holders with the same age and experience!
If you want to be in the same salary “league” then visit
www.aacei.org for more information!
 ARTICLE REPRINTS AND PERMISSIONS

COST ENGINEERING Vol. 54, No.6/November/December 2012

Members of AACE International have access to free downloads of selected articles
that are published with an AACE International reference number. These articles are
available at the online Virtual Library at www.aacei.org. Electronic files of each
month’s technical articles are posted and members can download an Adobe Acrobat
(PDF) version of any of the technical articles for free. You can search for articles
using the reference numbers listed in the Cost Engineering journal. Non-members
can subscribe to the AACE Virtual Library at an annual cost of US $100.00. AACE In-
ternational no longer offers reprints of individual articles.

Pages 5-15
Integrated Cost-Schedule Risk Analysis
Dr. David T. Hulett and Michael R. Nosbisch, CCC PSP
This article was presented as RISK.1043 at the 2012 AACE International Annual Meet-
ing in San Antonio, Texas.
Article Reference Number - 29188

Pages 17-27
Estimate Accuracy: Dealing With Reality
John K. Hollmann, PE CCE CEP
This article was first presented as RISK.1027 at the 2012 AACE International Annual Meet-
ing in San Antonio, Texas.
Article Reference Number - 29189

TO ORDER Pages 28-38

Contact: AACE International Publications Sales Quantifying Estimate Accuracy and Precision for the
at pubsales@aacei.org Process Industries: A Review of Industry Data
Alexander Ogilvie, Robert A. Brown Jr., Fredrick P. Biery
Photocopying Prices: and Paul Barshop
For permission to photocopy individual articles This article was first presented as RISK.1100 at the 2012 AACE International Annual
for personal use, or to request permission for Meeting in San Antonio, Texas.
bulk photocopying, please contact the Copy- Article Reference Number - 29190
right Clearance Center at 978.750.8400, and
pay the required photocopying fees. For any Pages 39-45
other use or reprint requests, please e-mail: Common Errors in Dealing With Project Risk
editor@aacei.org. Joseph A. Lukas, PE CCE
This article was first presented as RISK.944 at the 2012 AACE International Annual
Contact Us Meeting in San Antonio, Texas.
AACE International Article Reference Number - 29191
1265 Suncrest Towne Centre Dr
Morgantown, WV 26505-1876
USA
Phone: 304.296.8444
Fax: 304.291.5728

For Information Concerning

Other Reuse Requests
If you are seeking permission to quote or
translate into another language any material
from any issue of the Cost Engineering journal,
please contact our Managing Editor, Marvin
Gelhausen at mgelhausen@aacei.org

COST ENGINEERING  NOVEMBER/DECEMBER 2012 51

 CALENDAR OF EVENTS
NOVEMBER 2012
14-15 AACE International’s
International Total Cost Manage-
ment Conference,
AACE International
Hyatt Regency Dubai and Galleria
Dubai, United Arab Emirates
Contact: phone 1-800-858-COST
fax (304) 291-5728
info@aacei.org
www.aacei.org
24 2012 ICEC Region 2
(Europe/Near East) Meeting,
The International Cost Engineering
Council (ICEC)
FAST/AICE Premises
Milan, Italy
Contact:  www.costengineering.eu
www.icoste.org/category/meetings
28 - Dec 2 The First Australasia
and South East Asia Structural
Engineering and Construction
Conference ASEA-SEC-1),
The ISEC Society
Curtin University
Perth, Australia
Contact: www.isec-society.org
DECEMBER 2012
3-7 National BIM Conference,
AEC Science & Technology, LLC
Washington D.C. Convention Center
Washington D.C.
Contact: phone 1.508.790.4751
1.800.996.3863
support@aecst.org
52 COST ENGINEERING NOVEMBER/DECEMBER 2012
4 The Federal Design &
Construction Outlook Conference,
AEC Science & Technology, LLC
Washington D.C. Convention Center
Washington D.C.
Contact: phone 1.508.790.4751
1.800.996.3863
support@aecst.org
APRIL 2013
17-19 Cost Engineering Event 2013
- Total Cost Management; A World
of Opportunities, Cost Engineering
Consultancy
Ara Hotel
Zwijndrecht, Netherlands  Contact: phone 1-800-858-COST
Contact: +31 (0)78 620 0910
BDames@costengineering.eu
JUNE 2013
18-23 ISEC-7,
The ISEC Society
Campus Center of the University of
Hawaii at Manoa
Honolulu, HI
Contact:  1265 Suncrest Towne Centre Dr,
www.isec-society.org/ISEC_07/
27-30 AACE International
Education Seminars,
AACE International
Marriott Wardman Park
Washington, D.C.
Contact: phone 1-800-858-COST
fax (304) 291-5728
info@aacei.org
www.aacei.org
30-July 3 AACE International
Education Seminars,
AACE International
Marriott Wardman Park
Washington, D.C.
fax (304) 291-5728
info@aacei.org
www.aacei.org
Please submit items for future
calendar listings at least 60
days in advance of desired
publication.
AACE International, 
Morgantown, WV 26505-1876
USA   
phone: 304-296-8444        
fax: 304-291-5728       
e-mail:  editor@aacei.org
website: www.aacei.org
JOB SITE,
MEET INSIGHT.
Learn about Sage solutions for estimating and construction.
» Visit our website at SageCRE.com/bidready to get the facts.
Make it easier for your estimators to bid on and win more projects with our business-aware, people-
connected applications for specialty and general contractors. Over 40,000 construction and real
estate firms, including over half of the ENR 400, rely on Sage solutions to manage more than 400,000
jobs and 5.7 million subcontracts every year. Join them and gain the insight to get job-ready today.
© 2012 Sage Software, Inc. and its aɉliated entities. All rights reserved.

Sage 50 • Sage 100 Contractor • Sage 300 Construction and Real Estate • Sage Construction Anywhere • Sage Estimating

Enabling Sage contractors of any size to achieve their ambitions.