EST.2833 Maturity Assessment For Engineering Deliverables

2018 AACE® INTERNATIONAL TECHNICAL PAPER
EST.2833
Maturity Assessment for

Engineering Deliverables
H. Lance Stephenson, CCP FAACE; and
Peter R. Bredehoeft, Jr. CEP FAACE
Abstract – Over the last 60 years, AACE International has provided numerous recommended
practices that have advanced the skills and knowledge of cost engineering. In an effort to
provide further enhancement to the practice of cost engineering, the authors have developed a
maturity assessment tool based on AACE International’s Recommended Practice No. 18R-97,
Cost Estimate Classification System - As Applied in Engineering, Procurement, and Construction
for the Process Industries.
The Maturity Assessment for Engineering Deliverables (MAED) tool, as introduced in this paper,
is a maturity methodology that further defines the “Estimate Input Checklist and Maturity
Matrix” as provided in the Recommended Practice No. 18R-97 Cost Estimate Classification
System (CECS).
The AACE International RP No. 18R-97 Cost Estimate Classification System, complimented with
the Maturity Assessment for Engineering Deliverables, can be used as a quantitative approach
in determining the appropriate level of maturity required for improved estimate accuracy and
confidence, and project delivery success. This article will provide readers with:
• A refined understanding of the maturity for each of the deliverables identified in the
Estimate Input Checklist and Maturity Matrix.
• A quantitative assessment for the completeness and usability of the engineering
deliverables.
EST.2833.1
Copyright © AACE® International.
This paper may not be reproduced or republished without expressed written consent from AACE® International
Table of Contents
Table of Contents ............................................................................................................................ 2

Introduction .................................................................................................................................... 3
AACE International RP 18R-97 ........................................................................................................ 4
Estimate Classifications............................................................................................................... 5
Estimate Input Checklist and Maturity Matrix............................................................................ 7
Engineering Maturity ...................................................................................................................... 8
Estimating Accuracy .................................................................................................................... 9
Stage-Gating, FEL and the Cost Estimate .................................................................................. 10
General Project Data and Engineering Deliverables ................................................................. 12
Influence and Key Deliverables ................................................................................................. 14
Descriptors of Maturity............................................................................................................. 15
Weightings ................................................................................................................................ 16
The Refined “Estimate Input Checklist and Maturity Matrix” .................................................. 16
Assessment Tool ........................................................................................................................... 18
Maturity Levels and Scoring...................................................................................................... 18
Assessment Tool Input Form .................................................................................................... 19
Reporting................................................................................................................................... 20
Corrective Actions ..................................................................................................................... 22
Approvals .................................................................................................................................. 23
Conclusion ..................................................................................................................................... 23
References .................................................................................................................................... 24
EST.2833.2
Introduction
The purpose of the development of the Maturity Assessment for Engineering Deliverables tool
further supports the use of AACE International’s Cost Estimate Classification System
Recommended Practice (RP). This assessment compliments the RP in that it provides
organizations and cost engineering practitioners with a refined approach to assessing the
maturity of project definition, and therefore, overall completeness and usability of the
engineered deliverables. The maturity assessment would introduce improved estimating
techniques, reduce design, quantity and productivity allowances and contingencies based on an
understanding of the completeness of engineering deliverables.
The Maturity Assessment for Engineering Deliverables is considered a quantitative assessment

that identifies any deficiencies and perceived errors or omissions in the design due to the lack
of complete engineering information. The maturity assessment process will provide an
evaluation or grading system, reporting system and dashboard for conveying the maturity of
the engineering documents. Finally, the Maturity Assessment for Engineering Deliverables
provides a structured and repeatable approach that allows organizations the ability to improve
project delivery, cost effectiveness and predictability which will ultimately result in cost
competitiveness.
Who it speaks to…
The approach presented in this paper would specifically be used by any organization or
individual who utilizes AACE International’s RP 18R-97 Cost Estimate Classification System.
Similar to the application of RP 18R-97, this paper provides guidelines for applying a refined
assessment of engineering deliverables for engineering, procurement, and construction (EPC)
work for the process industries.
What is different about this maturity assessment tool as compared to others in the industry…
When conducting the research for this paper, it became apparent that there were not many
models available today that provided the user with a defined solution for improving
engineering deliverables in order to ensure greater project definition and estimate accuracy.
Listed below are three major differences that separate the Maturity Assessment for
Engineering Deliverables tool from other maturity models.
1. Other assessment tools do not define a threshold to promote a required overall level of
acceptance; rather these assessment tools only provide an overall score just to provide
a score.
The Maturity Assessment for Engineering Deliverables identifies the overall score
required to ensure that engineered product meets the desired need. Regardless of the
score, if certain deliverables do not meet the required level of maturity, the project is
considered a “no-go” until corrective actions and resolutions have been completed.
EST.2833.3
2. Other engineering assessment tools do not define the importance of key deliverables in
relation to estimate accuracy.
The Maturity Assessment for Engineering Deliverables provides focus on key deliverables
and the necessary requirements for achieving a higher level of maturity as defined by the
Estimate Input Checklist and Maturity Matrix. These key deliverables influence the
overall maturity assessment in order to ensure greater project definition and estimate
accuracy. The key deliverables also support the go / no-go decision making process.
3. Finally, most maturity models do not demonstrate a driving relationship between the
key deliverables and the other engineering documents.
The key engineered deliverables need to drive the desired behavior and results for
improved project definition. However, for a complete, effective and efficient end-to-end
process, secondary deliverables must be completed as well. The Maturity Assessment for
Engineering Deliverables provides this requirement.
The intent of this paper and subsequent benefits…
The intent of this paper is to provide the reader with a richer understanding of the required
quality and usability of engineering deliverables and technical documents in order to ensure
greater project definition and estimate accuracy. While equally important, the authors have
excluded discussions about developing the overall formulation of the estimates, including such
activities as costing and pricing, etc. However, the maturity assessment process can be applied
to any industry, based upon that industries typical design deliverables.
Benefits for completing such an assessment are numerous. The following points provide the
audience with the few key benefits of complete the Maturity Assessment of Engineering
Deliverables:
• A structurally sound approach to assessing the quality and completeness of engineering

deliverables.
• A validation process for ensuring estimate accuracy for improved cost predictability.
• A platform for improved engineering and estimating effectiveness and efficiency.
AACE International RP 18R-97
AACE International’s Recommended Practice (RP) 18R-97 Cost Estimate Classification System
“provides guidelines for applying the general principles of estimate classification to project cost
estimates” and “maps the phases and stages of project cost estimating together with a generic
maturity and quality matrix” [1, p. 24].
EST.2833.4
Therefore, the overall purpose of this recommended practice is to outline the relationship of
specific design input data and design deliverables to the estimate accuracy and methodology
used to produce the cost estimate. An implied confidence level can be inferred by the
completeness of project data and design deliverables, coupled with the quality of the
information shown. In essence, the estimate confidence level or estimate accuracy range is
therefore dependent by the reliability of the scope information available at the time of the
estimate, in addition to other variables.
Estimate Classifications
Cost Estimating is considered one of the most important aspects of the project delivery model.
The cost estimate provides organizations and project teams with the opportunity to ensure the
financial success of the organization. Owner organizations and contractors use cost estimates
to support the go / no-go decision-making process, as well as the financial analysis of return-on-
investment. An owner organization will use the cost estimate to assess the improvement or
sustainability to operational production, while a contractor will use the cost estimate to weigh
the financial risks of executing the project. At the end of the day, the cost estimate is used to
understand financial risk and profitability, and whether the investment is required or not.
As indicated by AACE international’s Total Cost Management (TCM) Framework, “cost

estimating is the predictive process used to quantify, cost, and price the resources required by
the scope of an investment option, activity, or project” [2, p. 24]. In an effort to support the
predictive process, and “given the goals of reducing uncertainty in the estimating process and
improving communication of estimate results, it is desirable to establish standard estimate
classifications for the enterprise” [2, p. 24]. The standard estimate classifications provide an
understanding of the required information needed to ensure that the appropriate estimate
provides the desired usability, and therefore, allowing for effective and improved decision
making.
From the table below (Table 1), the readers can understand the estimate classes as defined in
the recommended practice. The Cost Estimate Classification Matrix represented in the
recommended practice identifies the primary and secondary characteristics for each of the
classifications.
EST.2833.5
Table 1–Cost Estimate Classification Matrix for Process Industries [3, p. 24]
As identified in the table, the reader can also see that the maturity level of project definition
deliverables defines a range of maturity (in percentage) for each of the specific estimate
classes. Further to this, the table identifies the “expected accuracy range” for each estimate
class as well. The Class 5 estimate has the greatest range in expected accuracy as the level of
project definition is least mature. Subsequently, the Class 1 estimate has the smallest range in
expected accuracy as the level of project definition is the most mature. In most circumstances,
the owner or client prepares the Class 5, 4, 3 estimates, while the contractor prepares the class
2 and 1 estimates. The following graph (Figure 1) illustrates the expected accuracy range based
on estimate classes.
Figure 1–Example of the Variability in Accuracy Ranges for a Process Industry Estimate
[4, p. 24]
EST.2833.6
From Figure 1, the reader can visually see the range of maturity level of project definition for
each estimate. As an example, the graph provides and understanding of how the class 3
estimate overlaps with both the class 4 estimate and the class 2 estimate. The graph also
demonstrates the correlation of the expected accuracy range as the maturity level of project
definition increases.
Estimate Input Checklist and Maturity Matrix
To further support the requirements to meet the appropriate levels of maturity of the project
deliverables, RP-18R-97 identifies the required estimate input information and the completion
status for each specific estimate class. The “Estimate Input Checklist and Maturity Matrix”,
Table 2, divides the checklist into two specific categories of deliverables, which include general
project data, and engineering deliverables. The following table represents the deliverables by
category, estimate class and desired state of completeness.
ESTIMATE CLASSIFICATION
Class 5 Class 4 Class 3 Class 2 Class 1
MATURITY LEVEL OF PROJECT
0% to 2% 1% to 15% 10% to 40% 30% to 75% 65% to 100%
DEFINITION DELIVERABLES
General Project Data:
Project Scope Description Preliminary Preliminary Defined Defined Defined
Plant Production / Facility Capacity Preliminary Preliminary Defined Defined Defined
Plant Location Preliminary Preliminary Defined Defined Defined
Soils & Hydrology Not Required Preliminary Defined Defined Defined
Integrated Project Plan Not Required Preliminary Defined Defined Defined
Project Master Schedule Not Required Preliminary Defined Defined Defined
Escalation Strategy Not Required Preliminary Defined Defined Defined
Work Breakdown Structure Not Required Preliminary Defined Defined Defined
Project Code of Accounts Not Required Preliminary Defined Defined Defined
Contracting Strategy Not Required Preliminary Defined Defined Defined
Engineering Deliverables:
Block Flow Diagrams S/P P/C C C C
Plot Plans NR S/P C C C
Process Flow Diagrams (PFDs) NR P/C C C C
Utility Flow Diagrams (UFDs) NR S/P C C C
Piping & Instrument Diagrams (P&IDs) NR S/P C C C
Heat & Material Balances (HMBs) NR P/C C C C
Process Equipment List NR S/P C C C
Utility Equipment List NR S/P C C C
Electrical One-Line Diagrams NR S/P C C C
Design Specifications & Datasheets NR S/P C C C
General Equipment Arrangement
NR S C C C
Drawings
Spare Parts Listing NR NR P P C
Mechanical Discipline Drawings NR NR S/P P/C C
Electrical Discipline Drawings NR NR S/P P/C C
Instrumentation/Control System Drawings NR NR S/P P/C C
Civil/Structural/Discipline Drawings NR NR S/P P/C C
Table 2–Cost Estimate Classification Matrix for Process Industries [5, p. 24]
EST.2833.7
The following descriptors are provided in order to enhance the understanding of the
completion status for each of the deliverables based on the estimate class. This information can
also be found in the current recommended practice.
• Not Required: May not be required for all estimates of the specified class, but specific
project estimates may require at least preliminary development.
• Preliminary: Project definition has begun and progressed to at least an intermediate
level of completion. Review and approvals for its current status has occurred.
• Defined: Project definition is advanced and reviews have been conducted. Development
may be near completion with the exception of final approvals.
Engineering Deliverables:
• Not Required (NR): Deliverable may not be required for all estimates of the specified
class, but specific project estimates may require at least preliminary development.
• Started (S): Work on the deliverable has begun. Development is typically limited to
sketches, rough outlines, or similar levels of early completion.
• Preliminary (P): Work on the deliverable is advanced. Interim, cross‐functional reviews
have usually been conducted. Development may be near completion except for final
reviews and approvals.
• Complete (C): The deliverable has been reviewed and approved as appropriate.” [6, p.
24]
AACE International’s Recommended Practice (RP) 18R-97 Cost Estimate Classification System
provides the necessary foundation for establishing the general principles in using the estimate
classification system. The following section further refines this well established process.
Engineering Maturity
Webster’s dictionary defines maturity as "the quality or state of being mature; especially: full
development". To support this endeavor, our efforts were to ensure that we have achieved full
development based on the final or desired state. By establishing engineering maturity, we
therefore need to move the engineering deliverables from “preliminary” to a “defined” state
(our desired advanced state).
Understanding the maturity and state of completeness of the engineering deliverables, as well
as the overall project definition, can be difficult to achieve. There are many things to consider.
Temperature is one such example. Temperature is measured with a thermometer, historically
calibrated in various temperature scales and units of measurement. However, to some, this is
an inaccurate measure of temperature. To support this claim, AccuWeather identities weather
based on two numbers, the actual temperature based on units of measurement, and the
EST.2833.8
perceived temperature based on real-feel. The real-feel uses multiple factors including the
temperature, humidity, cloud cover, sun intensity, and wind to explain how hot it feels outside.
Like temperature, the maturity of project definition and the accuracy of the estimate are based
on many factors. To assist in defining engineering maturity, the following section bridges the
understanding of estimate accuracy, stage-gating and engineering deliverables.
Estimating Accuracy
The ranges identified for both the maturity level of the project definition deliverables and
“expected accuracy range” can have considerable thresholds in which each estimate class are
governed. For the purposes of the RP and this paper, estimate accuracy correlates with the
completion (in percentage) of the project definition deliverables.
An estimate itself is considered a prediction or forecast of the resources required to achieve or

obtain an agreed upon scope” [7, p. 24]. Based on this understanding, accuracy is needed to
recognize that there would then be variability of an estimate based on the scope of work,
application of technology, and complexity of the project, to name a few. It is this variability that
will determine the accuracy of the estimate.
Simply put, accuracy is regarded as the condition or quality of being true, correct, or exact;
freedom from error or defect.
Therefore, estimate accuracy is an indication of the degree to which the final cost outcome of a
project may vary from the single point value used as the estimated cost for the project. [8, p.
24].
As presented in Table 1 - Cost Estimate Classification Matrix for Process Industries (pg. 6), the
expected accuracy range is represented as a + / - percentage of low and high values. Therefore,
“estimate accuracy is traditionally represented as a +/- percentage range around the point
estimate; with a stated confidence level that the actual cost outcome will fall within this range”
[8, p. 24].
The following graph illustrates the relationship of the point estimate with the minimum
estimated costs and the maximum estimated costs, i.e. the range of costs in relation to the
point cost estimate.
EST.2833.9
Figure 2–Accuracy Range
Based on this representation of costs in relation to accuracy, the variability of the cost of the
project is heavily dependent on the amount of project information received, as well as the level
of maturity in which is developed.
Stage-Gating, FEL and the Cost Estimate
To further compliment the maturity level of project definition, industry practitioners have
introduced the stage-gate arrow as an opportunity to “shepherd” the project deliverables and
estimate through specific stages and gates. As the result of defining specific stages and gates,
project stages provide the opportunity to further develop and refine the estimate based on
predetermined requirements and deliverables (inputs). This opportunity allows the end user to
focus on specific requirements based on need and timing.
For instance, an earlier stage estimate may be required to support the decision to pursue the
development and execution of a project. This early stage estimate would be developed at a
higher level, specifically for screening and feasibility studies. For other stages, an estimate may
be required for budget authorizations, to control a project, or for estimate validation. Each type
of end usage is based on the stage and predetermined requirements and deliverables, which
ultimately provides the understanding as to the predictive nature in which the estimate should
deliver.
In an effort to calibrate project definition and engineering deliverables with the appropriate
level of estimates, AACE has aligned the cost estimate classification matrix in accordance with
the process industry stage–gating arrow. While all estimate classes cascade into a more refined
understanding of the cost of the scope of work, some may consider the class 3 estimate and the
class 1 estimate as the most important of the classes. The reason behind this thinking is that
most owner organizations appropriate, in other words ask for funding, at FEL 3 (the class 3).
Subsequently, the class 1 estimates that are completed during the “Design & Construct” stage
provide an understanding of “production costs” based on installation layouts and bulk
quantities.
EST.2833.10
The following graphic illustrates the typical process industry stage-gate arrow. The stage-gate
arrow identifies a series of front end loading (FEL) stages in which specific products or
deliverables are to be completed. The class of estimate to be completed is also identified. For
example, the class 3 estimate is an output of stage FEL 3.
Figure 3–Typical Process Industry Stage-Gating Arrow [9, p. 24]
The stage-gate process is an excellent tool to assist in the assessment of technical and
commercial viability of the project. To advance to the next stage, the project team must
demonstrate that specific deliverables and information is completed. The project team will
submit its proposal to a gate-keeper who then assesses the project deliverables for maturity
and completeness. The approval supports the effective and efficient go / no-go decision-making
process, which allows organizations to ensure that the right projects are being executed.
In general, most of the organizations within the process industry approve projects in the
manner described above. Depending on an organizations appetite for risk, an organization may
choose to approve the project after FEL 2 is completed, or once the detailed engineering is
completed during the execution phase. Of course, the more engineering that is completed,
there is an increase in the maturity level of the project definition, and therefore, an increase in
understanding cost certainty.
Through the last 20 years, organizations have completed studies that in fact support the need
for introducing maturity requirements for the execution of a project. IPA (Independent Project
Analysis) is an organization that benchmarks “owner” companies in the efficiencies of the
delivery of their respective projects. This information provides positive and negative findings of
the delivery of these projects, which in turn assist in determining best practices and
improvement studies. One such study identified that if an organization properly guides their
respective projects through the stage-gating process and completes the right deliverables, to
their required level of maturity, the overall cost of the project and the cycle time decrease. The
following graphs illustrate their findings in relation to FEL maturity.
EST.2833.11
Figure 4–FEL [10, p. 24]
Similar to IPA’s findings, it’s the expectations of the authors that an organization would improve
their delivery of their projects if the maturity of the project deliverables were increased.
General Project Data and Engineering Deliverables
To understand the maturity of the project, one must understand the engineering deliverables
and their required state of completeness. As identified in the Estimating Maturity Matrix,
project deliverables can be divided into general project (deliverables) data and engineering
deliverables. The following diagram represents the perceived formula for achieving project
definition maturity. Both general project data and engineering deliverables are equally
important for establishing maturity and possible success.
Figure 5–Deliverable Framework
EST.2833.12
One aspect that the authors identified is that there is an opportunity to expand on certain
deliverables within the Estimate Maturity Matrix in order to enhance the maturity. With this
said, the authors introduced the following expanded list that compliments the integrated
project plan, Table 3.
Integrated Project Plan
• Engineering / Design Plan
• Partnership / Funding Strategy
• Project Controls Plan
• Fabrication / Modularization Plan
• Project / Construction Strategy
• Environmental / Health & Safety Design
Review
• Materials Management Plan
• Logistics Plan
• Operating Plan
Table 3–Integrated Project Plan Deliverables
The added information that is presented above provides for the disclosure of required data that
would provide a more specific understanding of the integrated plan. This would also support
the development of the estimate, by eliminating unknowns and generalities as well as expedite
the problem solving and decision-making process.
In addition to expanding the integrated plan, the authors felt it necessary to provide a detailed
list of the discipline drawings, as represented in the following table, Table 4. This list may not be
all inclusive to the reader’s specific project(s), and therefore, it can be expanded on or deleted
as the user sees fit.
Mechanical Discipline Electrical Discipline Instrumentation / Control Civil / Structural Discipline

Drawings Drawings System Discipline Drawings Drawings
• Thermal Protection • Electrical Classification • Process Control • Site Grading Plans
Requirements Drawings Philosophy Document • Site Survey Plans
• Composite Piping • Electrical Equipment • Control Council • Piling Drawings
Isometric Drawings (Major) Layouts Drawings • Foundation Drawings
• Material Specifications • Electrical Heat Tracing • Programming / System • Slab On Grade Drawings
• Vendor Drawings Requirements Documents • Rebar Schedules
• Plumbing / Utility • Utility Load Sheet • Input / Output (I/O) • Demolition Drawings
Drawings • Wire, Conduit, Cable Count • Building Drawings
• Heating / Ventilation Tray & Cable • Termination Drawings • Structural Support
Drawings Schedules • Loop Drawings Drawings
• Termination Drawings • Structural Steel Drawings
• Vendor Drawings • Structural Connection
Drawings
• Structural Miscellaneous
Drawings
Table 4–Cost Estimate Classification Matrix for Process Industries
The discipline drawings as described in the above table are usually prepared in the detailed
engineering phase of the execution stage. These drawings provide three specific applications
which include; 1) supporting the development of class 2 and 1 estimates, 2) identifying the
interfaces between one discipline to another, and 3) allowing for the fabrication, installation
and building of the facility.
EST.2833.13
Influence and Key Deliverables
One aspect in defining maturity is to determine the influence and affect that each of the project
deliverables has in reference to the other project deliverables. To support the effort of
identifying the “project influencers”, AACE has also defined key engineering deliverables. These
key deliverables can be described as being more qualitative in nature, in which they support the
business plan, capacity and facility expectations (i.e. block flow diagrams, P&ID’s, etc.). These
key deliverables provide the content for designing the quantitative aspect of the engineered
deliverables (i.e. composite piping isometric drawings, civil foundation drawings, etc.).
As indicated earlier, the key deliverables provide an influence to the overall maturity
assessment and ensures greater project definition and estimate accuracy. This influence drives
the go / no-go decision making process by ensuring that each key deliverable is considered
complete prior to approval or advancing the project. If the key deliverable is not complete as
defined, then the overall maturity assessment is considered incomplete and is “failed” as a
result. This supports the importance of the key deliverables and how they drive the desired
results and completeness of engineering deliverables. It must also be understood that if these
key deliverables are incomplete, then the other secondary deliverables will also be incomplete.
The following table, Table 5, represents the key engineering deliverables as defined by AACE, as
well as a description for each deliverable and specific estimate class in which the deliverable is
to be completed. Furthermore, the authors determined that plot plans, heat & material balance
sheets, and electrical one lines should be represented in this list.
AACE Key Engineering

Description
Class Deliverables
Class 5 Block Flow The Block Flow Diagram provides an understanding of the general material flows throughout the entire
Diagrams plant, plant area, operational zones, or processes.
Class 4 Process Flow The PFD is considered a more refined deliverable of the Block Flow Diagrams. The PFDs provide an
Diagrams (PFDs) understanding of the general flow of the plant "product" processes based on major equipment
requirements and piping interconnects. It indicates the interfaces between the equipment of the entire
process while providing an understanding of the state of refinement for each of the desired products.
Class 3 Piping & The P&IDs are considered a more refined deliverable of the PFDs. The P&ID's are detailed drawings /
Instrument diagrams that identify the piping and instrument and control devices in relation to defined vessels and
Diagrams equipment. The P&IDs usually define piping sizes, system/piping names, directional flow, values,
(P&IDs) controls, etc.
Class 3 Plot Plans The Plot Plan is an engineered "plan" drawing that provides an understanding of equipment layouts and
piping / utility runs in relation to buildings, roads and other facilities that will be represented on project
site. The plot plan may identify any obstructions, underground or above ground features, etc. The plot
plan drawing is of "top-down" orientation (looking at the project site from a bird's eye view).
Class 3 Heat & Material Heat Material Balances are calculations that identify operating temperatures and pressures for each and
Balances (HMBs) every process "piping" stream. The HMBs are usually developed in concert with the PFD's. The HMBs
also provides an understanding of inlet and outlet streams.
Class 3 Electrical One- Electrical One-line diagrams (single-line diagrams) provide an understanding of the power flow
Line Diagrams requirements needed to energize systems, etc. The Electrical One-line schematically defines circuit
breakers, transformers, bus bars, etc. and the path of power flow between elements of the system.
Class 2 Design Specifications & Datasheets are considered technical documentation that provides an understanding of
Specifications & product characteristics, tolerances, dimensions, maintenance requirements, etc. These may also be
Datasheets considered vendor drawings and specifications. Specifications and data sheets are available for
mechanical (vessels, rotating equipment), electrical and instrumentation equipment. The specifications
and datasheets may identify power supply demands as well.
Table 5–Key Engineering Deliverables
EST.2833.14
The key engineering deliverables provide the necessary information to further the design of the
facility. These key engineering deliverables allows the project team to progressively develop the
engineered products, reaching its most mature states during the FEL 3 stage and the Detailed
Engineering phase of the Execution stage. Specifically, once FEL 3 is complete, the expected
output is the class 3 estimate, supported by all required engineered drawings as “Issued for
Design” (IFD). Subsequently, once the detailed engineering phase of the execution stage is
complete, the expected output is the class 1 estimate, supported by all required engineered
drawings as “Issued for Construction” (IFC). In some circumstances, the project team may
release partial design output to support the bidding and award process. Some project
practitioners may indicate that this information is considered class 2 quality.
The following diagram, Figure 6, illustrates the influence that the key deliverables have on the
overall cost of the project. Without ensuring the correctness and completeness of these
deliverables, the project is in jeopardy of not meeting its required performance metrics, in
terms of both the organization (ROI, IRR, NPV) and the project (CPI, SPI, Rework, Safety-TRIF).
Figure 6 – The Influence Curve [11, p. 24]
Descriptors of Maturity
Now that the expanded general project data and engineering deliverables are understood, the
next step is to define the maturity requirements for each of these deliverables. In assessing the
maturity of the engineering deliverable, three levels of maturity have been defined. These
maturity levels will support the scoring or ranking of the quality of the engineering deliverables.
The following is the proposed maturity level and scoring system.
EST.2833.15
Maturity Descriptors of Maturity

Level
1 • The deliverable has not been designed on an end-to-end basis. Project team members use legacy design
information primarily for perspective. [PURPOSE]
• The deliverable's inputs, outputs, suppliers, and customers have not been identified. [CONTEXT]
• The documentation of the deliverable is primarily functional but does not or is limited in identifying the
interconnections among the disciplines deliverables involved in executing the project. [DOCUMENTATION]
2 • The technical process has been designed to fit with other discipline designs in order to optimize the project's
technical solution. [PURPOSE]
• The deliverable owner (discipline) and the owners of the deliverables with which there are interfaces have
established mutual design expectations. [CONTEXT]
• The deliverable's documentation describes the interfaces with, and expectations of, other discipline
deliverable. [DOCUMENTATION]
3 • The deliverable has been designed to fit with customer and supplier deliverables in order to optimize the
"inter-dependent" technical solution. [PURPOSE]
• The deliverable owner and the owners of customer and supplier deliverables with which there are interfaces
have established mutual design expectations. [CONTEXT]
• An electronic representation of the deliverable supports interfaces with, and expectations of, other
customer and supplier deliverables and allows analysis of design changes and process reconfigurations.
[DOCUMENTATION]
Table 6–Maturity Level Descriptors
The maturity levels are ascending in completeness and refinement of the engineering
deliverables. The ascending maturity allows the project team to manage project risks based on
completeness of the deliverables. Level 1 is considered the least mature, while level 3 is
considered the most mature. These expected maturity levels for each deliverable is identified
on the “refined” estimate input checklist and maturity matrix. These new maturity levels
replace the existing desired state of completeness, i.e. not required (NR), started (S), etc.
Weightings
The intent of the Maturity Assessment for Engineering Deliverables tool is to also define the
maturity of the engineering deliverables and its end-to-end processes. Consequently,
consideration on weightings was warranted based on the influence in which the key
deliverables provided. Therefore, as represented on the refined estimate input checklist and
maturity matrix, each deliverable has an assigned weighting / score.
The Refined “Estimate Input Checklist and Maturity Matrix”
Based on all the specific considerations in defining maturity, and the need for a deeper
understanding of the state of completeness of the engineering deliverables, the authors have
developed a “refined” estimate input checklist and maturity matrix. This table, as represented
below, provides the user with a complete understanding of the expanded list of deliverables as
well as identifies which deliverables are original to RP 18R-97 and what has been added. The
table (Table 7) also defines the desired state (level) of maturity for each deliverable for each
estimate class. Finally, the table also defines the assigned weighting / score.
EST.2833.16
Original Wtg ESTIMATE CLASSIFICATION

18R-97 or Class 5 Class 4 Class 3 Class 2 Class 1
MATURITY LEVEL OF PROJECT DEFINITION Or Score 10% to 30% to 65% to
Added 0% to 2% 1% to 15%
DELIVERABLES 40% 75% 100%
Project Scope Description 18R-97 30 2 3 3 3 3
Plant Production / Facility Capacity 18R-97 30 2 3 3 3 3
Plant Location 18R-97 15 2 3 3 3 3
Soils & Hydrology 18R-97 15 1 2 3 3 3
Project Master Schedule 18R-97 15 1 2 3 3 3
Escalation Strategy 18R-97 15 1 2 3 3 3
Work Breakdown Structure 18R-97 15 1 2 3 3
Project Code of Accounts 18R-97 15 1 2 3 3
Contracting Strategy 18R-97 15 1 2 3 3 3
Integrated Project Plan 18R-97
Engineering / Design Plan Added 30 1 2 3 3 3
Partnership / Funding Strategy Added 15 2 3 3 3 3
Project Controls Plan Added 15 1 2 3 3 3
Fabrication / Modularization Plan Added 15 1 2 3 3
Project / Construction Strategy Added 15 1 2 3 3 3
Environmental / Health & Safety Design Added 15
1 2 3 3
Review
Materials Management Plan Added 15 1 2 3 3
Logistics Plan Added 15 1 2 3 3
Operating Plan Added 15 1 2 3 3 3
Engineering Deliverables: 18R-97
Block Flow Diagrams [1] 18R-97 45 2 3 3 3 3
Plot Plans [1]1 18R-97 45 1 2 3 3 3
Process Flow Diagrams (PFDs) [1] 18R-97 45 2 3 3 3 3
Utility Flow Diagrams (UFDs) 18R-97 30 1 2 3 3 3
Piping & Instrument Diagrams (P&IDs) [1] 18R-97 45 2 3 3 3
Heat & Material Balances (HMBs) [1] 18R-97 30 2 3 3 3
Process Equipment List 18R-97 15 2 3 3 3 3
Utility Equipment List 18R-97 15 2 3 3 3
Electrical One-Line Diagrams [1] 18R-97 30 2 3 3 3
Design Specifications & Datasheets [1] 18R-97 30 1 2 3 3
General Equipment Arrangement Drawings 18R-97 15 2 3 3 3
Spare Parts Listing 18R-97 15 2 3 3
Mechanical Discipline Drawings 18R-97
Thermal Protection Requirements Added 15 3 3
Composite Piping Isometric Drawings Added 15 3 3
Material Specifications Added 15 2 3 3
Vendor Drawings Added 15 3 3
Plumbing / Utility Drawings Added 15 3 3
Heating / Ventilation Drawings Added 15 3 3
Electrical Discipline Drawings 18R-97 3 3
Electrical Classification Drawings Added 15 2 3 3 3
Electrical Equipment (Major) Layouts Added 15 2 3 3 3
Electrical Heat Tracing Requirements Added 15 2 3 3 3
Utility Load Sheet Added 15 2 3 3 3
Wire, Conduit, Cable Tray & Cable Added 15
1 2 3 3
Schedules
Termination Drawings Added 15 2 3 3
Vendor Drawings Added 15 3 3
Instrumentation / Control System Discipline 18R-97
Drawings
Process Control Philosophy Document Added 15 2 3 3 3
EST.2833.17
Control Council Drawings Added 15 2 3 3

Programming / System Documents Added 15 2 3 3
Input / Output (I/O) Count Added 15 2 3 3 3
Termination Drawings Added 15 2 3 3
Loop Drawing Added 15 2 3 3
Civil / Structural Discipline Drawings 18R-97
Site Grading Plans Added 15 2 3 3
Site Survey Plans Added 15 1 2 3 3
Piling Drawings Added 15 1 2 3 3
Foundation Drawings Added 15 1 2 3 3
Slab On Grade Drawings Added 15 1 2 3 3
Rebar Schedules Added 15 2 3 3
Demolition Drawings Added 15 1 2 3 3
Building Drawings Added 15 1 2 3 3
Structural Support Drawings Added 15 2 3 3
Structural Steel Drawings Added 15 2 3 3
Structural Connection Drawings Added 15 2 3 3
Structural Miscellaneous Drawings Added 15 2 3 3
[1] Identified as Key Deliverables
Table 7–Cost Estimate Classification Matrix for Process Industries
Assessment Tool
The maturity assessment tool provides an objective methodology for assessing the quality and
completeness of the engineering deliverables, which relates to the estimate classification. It is
expected that the maturity assessment will change for each estimate classification. Also, to
support the estimate validation, the assessment tool and report should be included as part of
the estimating deliverables.
As indicated earlier, the assessment tool allows for customization to any industry, organization,
or company needs by modifying the engineering processes and deliverables. The assessment
tool can be used within the engineering deliverables as shown in 18R-97 Cost Estimate
Classification System or expanded to include a more refined list of engineering deliverables as
presented by the authors, which corresponds to a greater level maturity or completeness of the
design. The maturity level assessment can be converted into a mathematical scoring system for
quantitative analysis. Furthermore, the maturity assessment tool provides an input sheet and
scoring sheet, dashboard reporting system, and a corrective action plan. With this said, the
authors recommend that a threshold is established to ensure consistency in the required
maturity level.
Maturity Levels and Scoring
Each of the 61 deliverables within the maturity matrix, Table 7, has three specific elements for
determining maturity. Also provided are three numerically designated levels of maturity for
each of the elements. These levels of maturity provide the basis for assessing the overall
maturity of the deliverables. As indicated earlier, a rating of three (3) indicates the highest level
achieved, and subsequently, a rating of one (1) indicates the lowest level. In laymen terms, level
EST.2833.18
one suggests that the work is “just getting started” and level three can be considered “best in
class”. The Maturity Levels table also provides a narrative “statement” for each maturity level
for each element to assist in the understanding of the maturity level.
The elements of maturity are classified as follows:
• Purpose
• Context
• Documentation
The following table, Table 8, identifies the maturity levels and their respective maturity level
statement for each element.
Element Maturity Levels

1 2 3
The deliverable has not been The technical process has been The deliverable has been
designed on an end-to-end designed to fit with other designed to fit with customer and
basis. Project team members discipline designs to optimize the supplier deliverables to optimize
Purpose
use legacy design information project's technical solution, and is the "inter-dependent" technical
primarily for perspective. reviewed by the owner of the solution, and is approved by the
asset. owner of the asset.
The deliverable's inputs, The deliverable owner (discipline) The deliverable owner and the
outputs, suppliers, and and the owners of the owners of customer and supplier
Context customers have not been deliverables with which there are deliverables with which there are
identified. interfaces have established interfaces have established
mutual design expectations. mutual design expectations.
The documentation of the The deliverable's documentation An electronic representation of
deliverable is primarily describes the interfaces with, and the deliverable supports
functional but does not or is expectations of, other discipline interfaces with, and expectations
Documentation limited in identifying the deliverable. of, other customer and supplier
interconnections among the deliverables and allows analysis
disciplines deliverables involved of design changes and process
in executing the project. reconfigurations.
Table 8–Elements Table
The user of the assessment tool would “score” the quality of the elements of the engineering
deliverables based on the statements provided.
Assessment Tool Input Form
The assessment tool input form is based on RP 18R-97 Cost Estimate Classification System, but
expanded to include the complete list of engineering deliverables as defined by the authors.
The input form provides a weighted scoring system based on the key project drivers and
subsequent secondary engineering deliverables for an overall score. The maturity score
provides a methodology for objectively scoring the maturity of the engineering deliverables
whereby the user inputs a maturity level score for the purpose, context and documentation.
Based on the weighted scoring system, the input form is configured to provide a score for each
deliverable, a disposition flag based upon the scoring, and a column for key assessment
comments. The following is an example of the engineering deliverables maturity assessment
input form, Figure 7.
EST.2833.19
Figure 7–Evaluation Form
Reporting
The next part of the assessment tool is the Dashboard Report. The Dashboard Report provides
an understanding of the aggregated summary of the scoring based on each category. This
report includes the total points, points available and score to indicate and identify if issues are
prevalent. The report also includes a disposition flag and a spider diagram of the scoring points
compared to the available points. The spider diagram visually shows areas for improvement
and further evaluation, with a need for corrective action if required.
The following is an example of the Dashboard Report for the engineering deliverables maturity
assessment. The overall maturity assessment score is calculated from the input sheet scoring.
As discussed earlier, the scoring system is weighted based on the Key Project Drivers that affect
the project deliverables. Each deliverable score is summarized based on category, where the
value is reflected against the available points (based on estimate class). From there, the total
score is determined and validated against the expected range and related scale.
EST.2833.20
Figure 8–Dashboard Report
Another aspect that the Dashboard Report provides is further understanding of the state of
deliverables based on purpose, context and documentation. This allows the team to recognize
where certain elements may need improvement. For instance, an organization may indicate
that purpose and context are more influential on the outcome of the deliverables than
documentation, and therefore may introduce measures that the team would not progress
beyond the next stage if both purpose and context did not achieve a ranking of excellent (the
range between 95-100). A company can adjust the range of scoring or grading system in order
to meet the company standards by weighting the priority of the engineering deliverable
documents.
Figure 9–Dashboard Report
The Dashboard Report also identifies the acceptable range in which the overall assessment
must meet in order to qualify as a pass. If specific conditions are met, then the estimate should
receive a ranking of “excellent”. The ranges identified in figure 10 represent the author’s
approach for acceptability, or if more effort is required in order to attain a specific range. If an
assessment identifies that the engineering is incomplete, and if no corrective actions have been
completed to enhance the deliverables, then the estimate should not be considered for use for
its intended purposes. For instance, if the engineered deliverables are required to be mature to
EST.2833.21
develop a class 3 estimate, and if the deliverables fall short in meeting the level of maturity as
desired, then the estimate would be considered “class 4” quality.
Figure 10–Disposition Flags & Ranges
Based on the evaluation criteria, each assessment should strive to meet the threshold of 95-
100, or the rating of excellent. Please remember, if any key deliverable is not completed as
defined, then the overall maturity assessment is considered incomplete and is “failed” as a
result, regardless of an overall score of excellent.
Corrective Actions
Finally, the Maturity Assessment tool includes a corrective action plan, Figure 11, so that any
adjustments to the quality and completeness of the Engineering documents can be
documented and managed. The form allows for the project deliverable to be stated as well as
identifying the issue or inadequacy of the document. The corrective action and responsible
person should also be identified as well as the date of completion.
Figure 11–Corrective Action Plan
EST.2833.22
Approvals
One final consideration is that all assessments and engineering deliverables should be reviewed
and approved by all governing authorities and representatives as deemed necessary. For
instance, the authors recommend that all key deliverables be approved by the asset owner. As
an example, it is considered a best practice to have operations personnel approve the P&ID
deliverable in order to ensure design intent is met. An authority matrix identifying the
stakeholder’s involvement and required approvals should be established to assist the project
teams in ensuring that the highest level of maturity is attained.
Conclusion
In closing, the intent of this paper was to provide the reader with a richer understanding of the
required quality and usability of engineering deliverables in order to ensure greater project
definition and estimate accuracy. This paper also defined the parameters of assessing maturity,
which provides organizations and cost engineering practitioners with a refined approach to
assessing the maturity of project definition, and therefore, overall completeness and usability
of the engineered deliverables. Companies can customize the Maturity Assessment of
Engineering Deliverables tool for company standards based upon the weighting and the scoring
system. The Maturity Assessment of Engineering Deliverables provide an example expands list
of typical engineering deliverables.
Finally, the maturity assessment would introduce improved estimating techniques, and
hopefully reduce the need for additional design, quantity and productivity allowances and
contingencies, while increasing accuracy and cost predictability.
Benefits for completing such an assessment are numerous. The following points provide the
audience with the few key benefits of complete the Maturity Assessment of Engineering
Deliverables:
• A structurally sound approach to assessing the quality and completeness of engineering
deliverables.
• A validation process for ensuring estimate accuracy for improved cost predictability.
• A platform for improved engineering and estimating effectiveness and efficiency.
In conclusion, the organization can now identify the key requirements and deliverables for
establishing a leading position in engineering maturity, and subsequent execution of the
project.
EST.2833.23
References
1. AACE® International, 2016, AACE International Recommended Practice No. 18R-97, Cost
Estimate Classification System – As Applied in Engineering, Procurement, and Construction
for the Process Industry, Version March 1, 2016, Page 3, AACE® International.
2. H. Lance Stephenson (Editor), 2015, Total Cost Management Framework, An Integrated
Approach to Portfolio, Program, and Project Management, Chapter 7 – Process Map for
Cost Estimating and Budgeting, Second edition, Page(s) 175, AACE® International.
for the Process Industry, Table 1, Version March 1, 2016, Page 3, AACE® International.
for the Process Industry, Figure 1, Version March 1, 2016, Page 5, AACE® International.
for the Process Industry, Table 3, Version March 1, 2016, Page 13, AACE® International.
for the Process Industry, Version March 1, 2016, Page 12, AACE® International.
7. AACE® International, 2017, AACE International Recommended Practice No. 10S-90, Cost
Engineering Terminology, Version October 31, 2017, Page 48, AACE® International.
8. Mr. Larry R. Dysert CCC., Is “Estimate Accuracy” an Oxymoron?, EST.01, Pg.1, 2006 AACE
International Transactions, 50th AACE International Annual Conference, Las Vegas Nevada,
2006.
9. Edward W. Merrow, 2011, Industrial Mega Projects; Concepts, strategies, and Practices for
Success, Chapter 2 – Data and Methods, 1st Edition, Page(s) 24, John Wiley & Sons, Inc.,
Hoboken, New Jersey.
10. Edward W. Merrow, 2011, Industrial Mega Projects; Concepts, strategies, and Practices for
Success, Chapter 2 – Data and Methods, 1st Edition, Page(s) 35, John Wiley & Sons, Inc.,
Hoboken, New Jersey.
11. H. Lance Stephenson (Editor), 2015, Total Cost Management Framework, An Integrated
Approach to Portfolio, Program, and Project Management, Figure 11.5-1, The Influence
Curve, Chapter 11 – Enabling Processes, Second edition, Page(s) 175, AACE® International.
H. Lance Stephenson, CCP FAACE

AECOM
h.lance.stephenson@gmail.com
Peter R. Bredehoeft, Jr. CEP FAACE

Jacobs
pete.bredehoeft@jacobs.com
EST.2833.24

EST.2833 Maturity Assessment For Engineering Deliverables

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

EST.2833 Maturity Assessment For Engineering Deliverables

Uploaded by

Copyright:

Available Formats

2018 AACE® INTERNATIONAL TECHNICAL PAPER

Maturity Assessment for

Table of Contents ............................................................................................................................ 2

The Maturity Assessment for Engineering Deliverables is considered a quantitative assessment

Who it speaks to…

The intent of this paper and subsequent benefits…

• A structurally sound approach to assessing the quality and completeness of engineering

AACE International RP 18R-97

As indicated by AACE international’s Total Cost Management (TCM) Framework, “cost

Estimate Input Checklist and Maturity Matrix

General Project Data:

An estimate itself is considered a prediction or forecast of the resources required to achieve or

Figure 2–Accuracy Range

Stage-Gating, FEL and the Cost Estimate

Figure 3–Typical Process Industry Stage-Gating Arrow [9, p. 24]

Figure 4–FEL [10, p. 24]

General Project Data and Engineering Deliverables

Figure 5–Deliverable Framework

Mechanical Discipline Electrical Discipline Instrumentation / Control Civil / Structural Discipline

Influence and Key Deliverables

AACE Key Engineering

Figure 6 – The Influence Curve [11, p. 24]

Maturity Descriptors of Maturity

The Refined “Estimate Input Checklist and Maturity Matrix”

Original Wtg ESTIMATE CLASSIFICATION

Control Council Drawings Added 15 2 3 3

Maturity Levels and Scoring

The elements of maturity are classified as follows:

Element Maturity Levels

Assessment Tool Input Form

Figure 7–Evaluation Form

Figure 8–Dashboard Report

Figure 9–Dashboard Report

Figure 10–Disposition Flags & Ranges

Figure 11–Corrective Action Plan

H. Lance Stephenson, CCP FAACE

Peter R. Bredehoeft, Jr. CEP FAACE

You might also like