CONSTRUCTION INDUSTRY INSTITUTE

LEADING INDICATORS
DURING PROJECT EXECUTION

Research Summary 220-1

 Construction Industry Institute

3M ABB Lummus Global

Abbott ALSTOM Power
Air Products and Chemicals AMEC
Alcoa AZCO
Amgen Inc. Aker Kværner
Anheuser-Busch Companies BE&K
Aramco Services Company Baker Concrete Construction
BP America Bechtel Group
Biogen Idec Black & Veatch
CITGO Petroleum Corporation Burns & McDonnell
Cargill CB&I
Chevron CCC Group
Codelco-Chile CDI Engineering Solutions
ConocoPhillips CH2M HILL
Dofasco CSA Group
The Dow Chemical Company Day & Zimmermann International
DuPont Dick Corporation
Eastman Chemical Company Dresser-Rand Company
ExxonMobil Corporation Emerson Process Management
Genentech Fluor Corporation
General Motors Corporation Fru-Con Construction Corporation
GlaxoSmithKline Grinaker-LTA
Intel Corporation Harper Industries
International Paper Hatch
Kraft Foods Hilti Corporation
Eli Lilly and Company Hyundai Engineering & Construction
Marathon Oil Corporation JMJ Associates
Merck Jacobs
NASA Kellogg Brown & Root
NOVA Chemicals Corporation Kiewit Construction Group
Naval Facilities Engineering Command J. Ray McDermott
Ontario Power Generation M. A. Mortenson Company
Petroleo Brasileiro S/A - Petrobras Mustang Engineering, L.P.
Praxair R. J. Mycka
The Procter & Gamble Company Parsons
Rohm and Haas Company Perot Systems Corporation
Sasol Technology Primavera Systems
Shell Oil Company S&B Engineers and Constructors
Smithsonian Institution The Shaw Group
Solutia Technip
Southern Company Victaulic Company
Sunoco Walbridge Aldinger Company
Tennessee Valley Authority Washington Group International
U.S. Army Corps of Engineers WorleyParsons Limited
U.S. Bureau of Reclamation Yates Construction
U.S. Department of Commerce/NIST/ Zachry Construction Corporation
Building and Fire Research Laboratory Zurich
U.S. Department of Energy
U.S. Department of Health & Human
Services
U.S. Department of State
U.S. General Services Administration
U.S. Steel
Weyerhaeuser Company
Leading Indicators during Project Execution

Prepared by
Construction Industry Institute
Leading Indicators to Project Outcome Research Team

Research Summary 220-1

September 2006
© 2006 Construction Industry Institute™.

The University of Texas at Austin.

CII members may reproduce and distribute this work internally in any medium at no cost
to internal recipients. CII members are permitted to revise and adapt this work for their
internal use provided an informational copy is furnished to CII.

Available to non-members by purchase; however, no copies may be made or distributed

and no modifications made without prior written permission from CII. Contact CII at
http://construction-institute.org/catalog.htm to purchase copies. Volume discounts may
be available.

All CII members, current students, and faculty at a college or university are eligible to
purchase CII products at member prices. Faculty and students at a college or university
may reproduce and distribute this work without modification for educational use.

Printed in the United States of America.

 Contents

Chapter Page

Executive Summary v

1. Introduction 1

2. Methodology 3

3. Findings 12

4. Application/Implementation 16

5. Conclusions and Recommendations 22

References 23
 Executive Summary

Over the years, the Construction Industry Institute (CII) and others
have conducted significant research to analyze and measure risks during
project development, resulting in several risk analysis and prediction
tools. Perhaps the best known of these is the CII Project Definition Rating
Index (PDRI), which is used by owners and contractors during front-end
planning. Once the PDRI exercise is completed, most tracking during
project execution is accomplished with traditional measurements. What
is needed is a method, once the front-end planning and risk analysis
have ended, of identifying indicators that most likely will lead to project
success.

To address this problem, CII established the Leading Indicators to

Project Outcome Research Team. Its objective was to develop a tool
focused on nontraditional measures that can help identify project issues
and mitigation strategies earlier in project execution. The research team
examined the PDRI and elected to exclude it due to its focus on front-end
planning. The research team also examined the potential of one versus
two tools because of differing leading indicator priorities between owners
and contractors. Ultimately, the research team concluded that different
owner and contractor priorities would not influence measurements using
a single tool.

The goal of this research was to identify leading indicators, both

quantifiable and non-quantifiable, during project execution and to
develop a tool that could be used as a predictive evaluator of future
project risk. The tool attempts to fill the void between the PDRI and
traditional project control methods. It is intended as a complement to
these methods and can be used by project teams, project managers,
engineering managers, or anyone who has the need to evaluate a project
during execution. This publication summarizes the research effort that
led to the development of this new tool.

v
 1

Introduction

Objectives
The research project had the following major objectives:
1. Define a “leading indicator” and formalize the scope of this
project.
2. Develop a list of possible measures and means for evaluation.
3. Develop a prioritized list of measures and a weighting system
based on the statistical correlation of these indicators to project
health and phase.
4. Incorporate the measures and weighting system into a tool that
may be used to evaluate projects.
5. Develop a management process for implementing periodic
project reviews based on composite and individual scoring
systems.

Another consideration for the research team was how to address

traditional measures. Most project measurements shift to these traditional
methods (cost variance, schedule variance, critical path tracking,
procurement and expediting documents, cost reports, performance
ratios, and earned value) once a project has been authorized. Although
effective at measuring problems, they require the production of hard
data and tend to report problems after the fact — making mitigation
more difficult. Essentially, traditional tracking has three drawbacks: (1)
the requirement of hard data; (2) the cost of collecting data; and (3) the
processing time to report results.

A tool that does not rely on hard data and allows projects to identify
issues earlier is a valuable supplement to traditional tools. The research
team developed this new tool, called the Project Health Indicator or PHI,
which has broader application because management can utilize it at all
levels of the organization to assess project “health.”

1
 The research team also wanted the tool to give some guidance regarding
project characteristics that lead to project risks to enable a starting point
for mitigation strategies. The method chosen was to relate the leading
indicators to identified project practices. Therefore, in addition to
identifying risks to project outcomes, the PHI tool shows the potential
project practices that could be utilized to improve the project risks.

2
 2

Methodology

What is a leading indicator?

The first focus of the research effort was to determine what constitutes
a leading indicator. The research team defined leading indicators as:

Fundamental project characteristics and/or events that reflect or

predict project health. Revealed in a timely manner, these indicators
allow for proactive management to influence project outcomes.

The definition contains several key characteristics that were used in

evaluating the leading indicators and traditional measures that were
reviewed by the research team. First and foremost, the indicator must
predict potential risks to future outcomes. The evaluation of the indicator
must correlate future impacts to project outcomes. It cannot simply be
a measure of what has already happened. Second, the indicator must
have a strong correlation to project outcomes. These outcomes define a
project’s success or failure.

What are project outcomes?

The research team established the following definitions of five key
project outcomes:
1. Cost: Cost performance is viewed in terms of overall actual
final cost versus the established project budget. Secondary cost
outcomes can include cost/cash flow deviation (compliance
with spending plans), cost efficiency (how efficiently an asset is
design and constructed versus similar facilities in industry), and
consumption of contingency or reserves.
2. Schedule: Schedule performance is viewed in terms of overall
actual final duration versus planned project duration. Secondary
schedule performance can include outage duration performance
and overall engineering and construction cycle time (for certain
fast-track projects).

3
 3. Quality/Operability: these are outcomes that are based upon a
facility being capable of operating per its intended function and
that the quality of the facility and construction craftsmanship
matches the intended asset life.
4. Safety: Safety as an outcome is a combination of construction
safety during the course of the project and overall safety
considerations of the new facility that will enable it to operate
safely over its life cycle. Construction safety involves accidents
to personnel in the battery limits of the construction zone and
is generally viewed in terms of recordable cases or Days Away
or Restricted Time (DART) cases. Facility safety, a more long-
term outcome, is based upon the facility having the equipment,
protections, and/or warning/safety devices, safe job procedures,
and energy control procedures required for the facility to operate
and be maintained in a safe manner.
5. Stakeholder satisfaction: Stakeholder satisfaction is the overall
pride, contentment, and/or happiness that the stakeholders have
with the outcome of the project. For contractors, it is somewhat a
measure of the potential for future, repeat business.

Obviously, not every project and organization weighs the above

outcomes equally. No attempt was made by the research team to establish
a priority of one outcome versus another on projects. Instead, each
leading indicator was individually correlated to each project outcome
and jointly correlated to an overall score. This feature will allow the PHI
tool to be customized by assigning personalized scaling of each outcome
based on the culture of individual organizations and the priority assigned
to the outcomes for a specific project.

Leading Indicator Generation

The indicators were first generated by research team members and
later were expanded, filtered, and weighted through several iterations
of surveys that were submitted to the industry at large. A total of over
180 leading indicators were considered. Eventually, 43 were selected
and are currently used in the PHI tool. Each potential leading indicator
was reviewed by the research team and further tested by CII member
organizations to arrive at the most significant indicators.

4
 The original 180 potential leading indicators (LIs) were evaluated using
three criteria:
1. The problem the LI addressed
2. The project outcome the LI had an impact on
3. Whether the LI could be measured using a tangible or intangible
measurement means.

This initial screening effort reduced the number of LIs from 180 to
126. These were then ranked using a survey that assessed the potential
negative impact of each LI on the five project outcomes. This survey
was completed internally by 15 research team members. The results
were analyzed statistically based on a LI’s total negative impact on each
project outcome. Each LI was ranked from highest to lowest impact.
Based on the rankings and further evaluation, the number of LIs was
reduced to about 90.

A second screening effort, required to further reduce the number of

LIs, used a broader cross-section of the industry. The negative impact
of each LI on the five project outcomes was evaluated using a survey
completed by representatives from research team organizations. A total
of 63 individuals ranked the 90 LIs. Statistical analysis was used to assess
the rankings and reduce the LIs to about 65.

A third survey was conducted to reduce the LIs to 43. This survey
included all CII membership. The results are discussed later.

This process of identifying and reducing the number of LIs was by far
the most significant effort in the research project and each measure was
carefully scrutinized, worded, and reworded in order to ensure that the
indicator represented a leading — not lagging — measure. The overall
goal was to ensure the tool had adequate measures to forecast results,
but to keep measures limited to avoid making the tool too cumbersome.

Traditional Measures
One dilemma was the problem of traditional measures versus this
new approach to identifying leading indicators. Obviously, owners and
contractors both spend significant effort and resources on traditional

5
methods of project control. Although not always effective, these methods
have led to improved project outcomes. Rather than simply ignore them,
the research team opted to include and scrub them along with the other
leading indicators. In the end, however, most traditional measures did
not meet the agreed-upon definition of a leading indicator and were
dropped from the refined list.

For documentation purposes, the following list of traditional measures

and metrics was considered:

• Ratio of Engineering/Total Project Cost

• Labor/Material/Equipment vs. Total Installation Cost (TIC)
• Percent of Engineering Complete at Authorization
• Type of Estimate (detailed, factored, analogous)
• Mobilization Date
• Team Schedule Development (wall schedule, interactive
planning session)
• Change Management Planning/Training (plan)
• Physical Percent Complete vs. Percent Plan
• Cash Flow (spending actual vs. plan)
• Number of Change Orders (pending/approved)
• Safety Statistics (drug tests)
• Quality and Turnaround of Requests for Information
• Permitting
• Risk Issues (major issues)
• Commissioning Planning
• Quality Control (factory and field testing results, percent of
rework)
• Estimated vs. Actual Cost (cost variances)
• Construction Management/TIC

6
 • TIC/Process Equipment
• Percent of Engineering Complete at Start of Construction
• Productivity and Forecasting for Engineering/Construction
• Source of Estimate (owner, contractor, third party)
• Number of Holds on Construction Drawings (quality of
authorized for construction documents)
• Cost Reports (committed cost/spending)
• Milestone Variances
• Cost of Hourly Rate vs. Plan (engineering and construction)
• Contingency Used (start of construction, reporting periods)
• Procurement Status (expediting)
• Submittal Status Report (vendor drawings)
• Overtime Percent
• Accounts Receivable
• Craft Turnover/Availability
• Earned Value = Percent Spend/Percent Budget
• Estimated Quantities vs. Actual Bulks

Project Definition Rating Index (PDRI) Disclaimer

Another issue that needed to be resolved was how to treat projects
that are experiencing problems resulting from poor or insufficient pre-
authorization development. When examining the PDRI for inclusion,
the team concluded that every question in the PDRI could easily be
turned into a leading indicator for the execution tool. However, project
team members would have already documented this within the PDRI.
Therefore, the research team opted to establish a criterion that the
PHI tool would assume that the pre-authorization development work
was complete and performed in sufficient detail to address the risks
associated with items raised by the PDRI. The new leading indicator tool
would thus begin from the completion point of the PDRI and assumes

7
that the project has been sufficiently developed for execution. It should
be noted that if this assumption is incorrect, the PDRI questions should
be a primary source of potential execution risk.

It is the completion of the tasks associated with project development,

not the authorization itself, that defines when a project moves from
facility planning, basic scope development, and execution planning into
detailed engineering and execution. For cases where the PHI tool is
being used on projects that did not fully develop the front-end planning
process, the tool will indicate areas of risk but the correlation to outcomes
and the practices that need to be addressed will be second-tier effects.
For example, if the basic facility scope or process flow diagrams are
incomplete, then gaps will exist in detailed engineering. Since detailed
engineering is a project execution task, it is this gap that will show up in
the PHI tool.

Project Practice Correlations

In addition to correlating to outcomes, the research team grouped each
of the LIs into categories centered on specific project practices. These
practices are loosely based on the CII areas of knowledge utilized during
project execution, and each measure points to potential CII sources for
reference. The project practice list includes:

Alignment: These are practices associated with the overall

alignment of the project team with respect to project goals
and objectives. The make-up of project teams can change
considerably from the front-end planning phase to the
execution phase. The owner project team generally changes
from business planning personnel to those responsible for
implementation. New contractors and suppliers also are usually
added at this time. Both owner and contractor teams are
generally expanded to address the increasing volume of work.
How these new team members and contractors understand and
are aligned to common goals plays a key role to project success.

8
Change Management: CII and others have accumulated large
amounts of research regarding the effects of late project scope
changes and high volumes of rework to poor project outcomes.
How the project team makes decisions on, controls, tracks, and
implements changes on a project can have a significant effect
on project outcomes.
Constructability: Constructability (construction input to design)
generally involves construction-related methodology and
planning. The ability to plan and execute the construction of a
facility efficiently is a major driver behind project success.
Contracting: Contracting is based on matching contract types
to project risks. It is not an endorsement of any one particular
contract type. There is no weighting of the PHI tool that values
turnkey versus lump sum versus design-build versus cost
reimbursable. It is purely a measure of whether the project team
is seeing potential issues between the contracts in place and the
scope that needs to be executed.
Quality Management: Quality management includes items such
as quality of engineering, construction quality and rework,
equipment inspections and testing, and facility start-up.
Safety Practices: This is a measure of whether or not the project
team is fully engaged in the practices that drive project safety
(see CII Target Zero practices).
Project Control: Project control involves the tools and techniques
used to track, evaluate, and improve schedule and cost
performance. In terms of this tool, it is not simply the use of
a project schedule and cost reporting. It is a measure of how
accurate the schedule is; how effective the schedule is in
tracking work and identifying gaps; whether the cost reporting
is utilized in future decision making; and whether the team is
effectively using the information as a planning tool. Too often,
schedules and cost reports become deliverables themselves
instead of tools to be used in planning the work.

9
 Team Building: People implement projects. The core
competencies of the people that constitute the project team and
how the people that make up the project team play a key role
in the success of any project. Good project teams overcome
gaps in scope, risk events, design issues, and project changes
in a proactive way to minimize the negative effects on project
outcomes. Poor teams do not.

Identifying the project practices was an attempt to identify to the user

of the PHI tool where opportunities for improvement exist. The practices
identify potential sources of risks to project outcomes to guide the user
toward a potential mitigation strategy.

Leading Indicator Surveys/Results

With the leading indicators established by the research team, a
survey was generated for distribution to the CII membership for input.
Requirements were that the indicators be leading, that they not be a
repeat of items contained within the PDRI, and that they be consolidated
in order to be more macro in nature. A total of 65 leading indicators
were included. Each CII member organization was asked to submit the
survey based upon the early execution stage of an actual project.

Each LI was individually scored based on its potential negative impact

on each of the five project outcomes. Eighty-four questionnaires were
received and analyzed statistically by the research team. The LIs were
ranked across all the questionnaires based on an aggregate score (total
negative impact). Based on the strength of the ranking of LIs, the research
team reduced the number of LIs to 43.

Some discussions were held regarding whether the tool should be the
same for owners and contractors or whether separate tools would be
more appropriate. Although the individual scoring between owners and
contractors varied somewhat, statistical checks were made to ensure that
the differences between owner and contractor scores would not bias
the results. Thus, the overall ranking of the leading indicators was not
substantially affected. Therefore, to allow the PHI tool to be used in a
team environment, the research team decided to develop only one tool.

10
 The research team also decided to normalize the scoring for each
outcome and practice to 1000 points. This was done to remove the
scoring from leading indicators that were answered as “Not Applicable”
from the scoring system. A “Not Applicable” answer in the tool removes
the scoring for that LI from the calculation and the remaining scores
are then normalized around a maximum of 1000 points. In this way,
answering “Not Applicable” has a different effect from answering
“None.” An answer of “None” keeps the impact in the calculation. Thus,
the tool does not give credit for an LI that is not applicable to a particular
project and recalibrates itself to the leading indicators that can have an
impact on project risk.

11
 3

Findings

Leading Indicator Selection/Descriptions

Based on the consolidated CII member survey results, the research team
decided to include the top 25 overall leading indicators (highest ranked
impact on all five outcomes). This added several LIs to the total because
of differences in ranking between owners and contractors. In addition,
so that all five-project outcomes are equal on a relative scale, the top 10
leading indicators for each outcome were included. This also added a
few more LIs as some of the top ten ranked LIs by project outcome were
not included in the top 25 from either the owner or contractor results.
Thus, the overall total number of leading indicators was determined to
be 43. Some adjustments were made to the aggregate score of each LI
based upon its standard deviation. The concept behind this adjustment
was that consistent answers have more impact on outcomes than variant
answers. The adjusted score determines the final ranking of each LI.
These 43 LIs are the leading indicators that comprise the PHI tool.

The next step in generating the tool was to establish a detailed

description and scoring guideline for each of the leading indicators.
The research team developed a one-page definition describing each
leading indicator and its relationship to project outcomes. It was decided
that the tool should be based on potential negative project impacts.
Therefore, the one-page definition also included general measurement
criteria on potential impact levels. The problem levels in the tool are:
serious, major, moderate, minor, and none. The measurement criteria
guidelines for each leading indicator are included in the research team’s
implementation guideline (CII Implementation Resource 220-2) as well
as embedded as comments in the PHI tool.

12
Weighting
During the testing of the tool, an issue arose regarding how the weighting
magnified differences between LIs. As noted earlier, aggregated scores of
each LI for outcomes became the basis of calculating the impact of LIs on
outcomes. The base or normal weight for the LIs in the tool was derived
by dividing the aggregated score of a LI by the standard deviation of the
survey answers for that LI. This determined the maximum points assigned
to each LI. Thereafter, the team assumed a linear relationship for the
problem levels in the tool and, therefore, the model simply assigned a
multiplier of serious (maximum points) = 100%, major = 75%, moderate
= 50%, minor = 25%, and none = 0% to the weighted scores to quantify
the risk impact of the LI.

Two effects were observed. First, the risk scoring for the outcomes
tended to be similar. Since only high impact LIs were included in the
tool, the range in the LI weights across all outcomes was not that great.
This caused some concern since the results did not always highlight the
outcomes that may be at a higher risk of poorer performance. Second, it was
noted that the normal weighting (dividing by a standard deviation) often
did not create substantial differences between outcomes. Thus, alternate
weighting methods were explored that would produce differences in the
risk assessments. That is, dividing by the standard deviation might not
be significant enough to differentiate between outcome scores. Based
on these two issues, the research team concluded that it needed to test
different scoring weights.

In an effort to arrive at a statistical supportable scoring weight, a final

round of surveying was initiated. The tool was distributed along with a
questionnaire regarding project outcomes. The tool was scored based
on where a project was at the start of construction, and the outcomes
of the project were documented. Overall, 14 completed projects were
evaluated as part of this final validation effort.

In the evaluation of the project data, a version of the PHI tool was
created that allowed users to choose one of five different weighting
options (created by modifying the standard deviation). The users were

13
asked to select the weighting option for the tool that best reflected the
health of the project being evaluated. In the tool, the higher weighted
options tend to emphasize the LIs that have consistent answers (small
standard deviations) and de-emphasize the LIs that have inconsistent
answers (large standard deviations). Statistical correlations were evaluated
between each of different weight options and the project outcomes.
Overall, dividing by the fifth power of the standard deviation had a
higher correlation and was therefore selected as the scoring method of
choice for the tool. The scatter plots of the project scoring using the
selected weighting method versus the cost and schedule outcomes are
shown in Figure 1.

The drawback, however, in using the fifth power of the standard

deviation is that it diminishes the impact of certain LIs that have large
standard deviations, regardless of the total score from the survey. The
research team decided to allow users to choose one of the two options
(normal or amplified weight) in the PHI tool because the health scores
generated by two different weights can aid in interpreting the outputs.
This weighting approach was adopted only for outcomes because project
practice groups are solely representing potential problem areas that
cannot be influenced by LIs in the other groups, and thus, the normal
weight was applied for generating project practice group health scores.

14
 Cost Correlation
Series 1 Linear Regression
1000

900
Project Cost Health Score

800

700

600

500
R2 = 0.6798
400

300
80 90 100 110 120 130
Performance (% - Actual / Budget)

Schedule Correlation
Series 1 Linear Regression
1000

900
Project Schedule Health Score

800

700

600

500

400
R2 = 0.5976
300
80 100 120 140 160
Performance (% - Actual / Budget)

Figure 1. Project Health Score Correlations to Outcomes

15
 4

Application/Implementation

Use of the Project Health Indicator Tool

The PHI tool that has been developed is an Excel® spreadsheet-based
tool. Figure 2 is an example of some of the leading indicators contained
within the tool.

Figure 2. Examples of Leading Indicator Statements

The user simply reads each LI statement and selects the potential
problem impact that a particular indicator may have on the project.
Generally speaking, it requires less than 30 minutes for someone familiar
with the tool to read and answer the questions. The scoring criteria is
embedded in the tool to assist the user with quick guidance should
questions arise concerning, for example, the definition of “serious”
versus “major” as shown in Figure 2 for LI 3.

16
 During development, several options emerged as ways to fill out the
tool. Many organizations had the project manager fill it out. Some had an
engineering manager with oversight responsibility fill it out while some
did it as a group exercise.

Instead of establishing a fixed methodology, the research team

decided any of the above methods could be used. It is up to the end
user to utilize the tool as deemed beneficial by their organization or
project-specific circumstances. The comments that emerged from the
group filling in the form version were that: (1) it took significantly longer
(each question was discussed); (2) some of the questions are pointed
regarding the competencies and capabilities of team members, which
can cause tension in a group setting; and (3) the group tended to refer
to the reference material for guidance on scoring. The teams did find
additional value in the discussion surrounding the assignment of a score.
This may be a benefit gained indirectly from the tool and in someway
may be even more valuable.

Tool versus Traditional Measures

The PHI tool that has been developed is a supplement and is not
intended for use as a replacement for traditional measures. Under no
circumstances should a project team or organization abandon efforts
regarding traditional project controls in favor of just utilizing the PHI
tool. The tool is intended to fill the gap between what is quantifiable and
what tends to be subjective in evaluating project status.

Interpreting the Outputs

The PHI tool contains two output sections. The first output is the
score that reflects the risk related to project outcomes or a measure of
project health. As discussed previously, two sets of health scores can be
generated. The user should start with the normal weight. The user can
then assess the impact of the amplified weighting on project outcomes.
Examples of the normal and amplified weights for the project outcome
indicators are shown in Figures 3 and 4 (next page).

17
Figure 3. Example Project Health Outcome Gauges
with Normal Weight

Figure 4. Example Project Health Outcome Gauges

with Amplified Weight

18
 Again, these scores are normalized to 1000 possible points, with a
lower score being interpreted as higher risk to that outcome. It is important
to note that the scores are a measure of potential risk of not achieving
desired project outcomes. This view is different than that provided by
traditional project controls. The PHI tool is predicting risks based on the
evaluation of non-quantitative information. It is an evaluation regarding
the likelihood of future events effecting outcomes. Also, in addition to
the five project outcomes, a composite overall health score is included.

The second output of the tool is the correlation to project practices.

This output was included so that the user can have an idea as to the
practice area that may be indicating risks to outcomes. It helps identify the
areas that should be improved to lower the risk of undesirable outcomes.
An example of the project practice output is depicted in Figure 5. The
project practice outcomes are based on the normal weight only and not
influenced by the amplified weight.

Figure 5. Project Practice Gauges

19
 Similar to the project outcome risk scores, the practice correlation
scores are normalized around 1000 possible points, with lower scores
being considered not desirable. Unlike the project outcome scores that
are a measure of risk, the practice correlation scores are an indicator
of deficiencies or gaps in the project practices that are leading to the
outcome risks.

Customizing the Tool

The default setting of the color ranges for the outcome and practice
gauges are based upon the correlation graphs depicted in Figure 1. A
score below 700 is considered high risk (red) because the regression line
below 700 indicates 10 percent overrun in cost and schedule. Although
the health scores when projects meet the planned cost and schedule were
around 800, the research team decided to divide the rest of the range
into halves. Therefore, a score between 850 and 1000 is considered low
risk (green), and a score between 700 and 850 is considered moderate
risk (yellow).

However, not all organizations view risks equally. Also, individual

projects may have different business drivers that allow for certain risks
on specific outcomes. For this reason, the ability to calibrate the gauges
has been included in the tool. Figure 6 illustrates the input required to
change the gauge width on the outcome dial gauges.

The above only adjusts coloring of the gauges on the output screens. It
has no effect on the scores. It is provided purely as a means to adjust what
levels of risk are considered acceptable to an individual organization.
Some examples for reasons to adjust the gauges could include sensitivity
to safety issues or schedule compression. Moving the colors of the gauge
has no effect on the risk assessment of the project by the tool and the
correlations of the data to project outcomes.

20
Figure 6. Input Form for Calibrating the Outcome Gauges

21
 5

Conclusions and Recommendations

This research set out to identify leading indicators for conducting

project reviews during the execution of a project and to create a tool
capable of measuring the health of a project. The PHI tool examines 43
leading indicators that can be assessed by a project team, project auditor,
or manager to evaluate the risks to project outcomes and to suggest
areas for improvement. The PHI tool is used after front-end planning,
that is, from the beginning of detailed engineering through construction.
The scoring of the PHI tool has been evaluated using actual projects
and demonstrates a high correlation between the scoring and project
outcomes.

The PHI tool is intended as a complement to traditional project controls

and as an early warning system to identify problems earlier than by
traditional methods. The early identification of risks and issues can allow
a project team to address the issues proactively, thus making it easier to
affect project outcomes. The tool also can be used in benchmarking and
lessons learned processes to identify systemic problems that may exist in
an organization’s project execution process system.

22
 References

Choi, J., Anderson, S., and Kim, T., “Forecasting Potential Project Risks
through Leading Indicators to Project Outcome,” Research Report
220-11, a report to the Construction Industry Institute, The University
of Texas at Austin, in press.

Hawley, R., and Hawley, D., Excel Hacks: 100 Industrial-Strength Tips
and Tools, O’Reilly Media, Cambridge, MA., 2004.

Project Heath Indicator Tool: Assessing Project Health during Project

Execution, Implementation Resource 220-2, Construction Industry
Institute, The University of Texas at Austin, in press.

23
Notes

25
Leading Indicators to Project Outcome Research Team
* Stuart D. Anderson, Texas A&M University
Robert H. Briest, ALSTOM Power
David Campbell, Air Products
* Jiwon Choi, Texas A&M University
William Chumchal, Mustang Engineering
* Paul Ennis, U.S. Steel, Co-Chair
Ian D. Etzkin, CDI Engineering
Gilles L. Gelinas, DuPont
* Kevin J. Gierc, Dick Corporation, Chair
Stephen E. Goodman, J. Ray McDermott
Robert MacLaren, Ontario Power
Michael G. Pappas, Eastman Chemical Co.
Steven D. Rench, ConocoPhillips
Mike Sheridan, International Paper
Brian E. Sichter, CB&I

Past Members
Michael Bronkowski, Amgen
S. J. Kim, Texas A&M University
Minesh Kinkhabwala, Aker Kværner
Don G. Miller, Gilbane

* Principal authors

Editor: Rusty Haggard

The Construction Industry Institute®
The University of Texas at Austin
3925 W. Braker Lane (R4500)
Austin, Texas 78759-5316
(512) 232-3000
FAX (512) 499-8101

Not printed with state funds

 ®

Construction Industry Institute®

The University of Texas at Austin