Project Quality Reviews &

Audits

© 1
Sam
 The 5 Phases of Project
Management

FEASIBILITY DESIGN, DEVELOPMENT CLOSEOUT OPERATIONS

PLANNING

Finance Organization Monitoring & Control

Evaluation Estimation Project Aberrations
Planning Changes & Claims
Quality & Reviews, & Audits This Lecture

2
 Contents of this Lecture

Quality Performance – What are the issues?

Project Reviews

Project Audits – What are they?

3
 Quality Performance – What are the
Issues?
Who cares? Also, is QP influenced by construction method?
Interrelationships between Quality and the other performance
metrics (Time, Cost)
Quality Control
Quality Assurance
Quality Management
Total Quality Management (TQM)
Note: In recent years – increasing focus on Quality, particularly on
federal contracts
4
Project Quality – Who
Cares?
• Owner
– Resident Project Representative
– Architect’s superintendent
• Manufacturers
– Prefabrication factory inspector
• Utility companies
– Site inspectors
• Project Developer/Contractor
– Field manager
– Quality manager
– Foremen
• Other Stakeholders
– The general public (or its reps) e.g., Local building department (code compliance)
– Safety and health inspectors (U.S. OSHA)
– Insurance company inspectors
– Financial institution inspectors
5
 How is Quality Related to Construction
Method?
Method 1 - Product may be produced
elsewhere and then brought to site, …

Prefab factory
2

Source: Matthew Norman 6

 … or Method 2 - product may
be produced at the Site

Source: brian moriarty 7

So, … Is Product Quality Related to Construction
Method?

Many ways of classifying construction methods.

One way is whether Project’s physical component are…
- created on site (concrete cast in-situ) OR
- created elsewhere and brought to site (precast concrete)
• Pre-fabricated components: generally higher quality
– Tighter tolerances
– Manufactured under tightly-controlled conditions
– More rigorous quality control mechanisms
– Shortcoming: Longer delay if identify problems!

• Site-created components: generally lower quality

– Vulnerable to Weather effects, relaxed adherence to standards, etc.
8
Quality Inspections of Externally-produced
Components

• Inspection may occur at the source (factory) or after they are

delivered at the site.
• Examples of external sites
– Precast concrete plants
– Steel plate fabrication plants
– Concrete production plants
– Welded steel plants
– Large, specialized pieces of equipment
• Quality checks at external fabrication site OR at project site can
be carried out by Owner (or his rep), or the Contractor
• May also monitor product quality DURING transport of the
product

9
Quality Control vs. Quality Assurance: What’s the
Difference?

10
Quality Control vs. Quality Assurance: What’s the
Difference?

Analogy: You Driving on a Freeway

11
 Quality Control vs. Quality Assurance: What’s the
Difference?

Analogy: You Driving on a

Freeway

Driving Quality Assurance

Before Driving: adjust seats, mirrors, temp, etc.
During Driving: These are
Occasional glances at:
- speed gage to ensure speed is not excessive
preventive
- side mirror before changing lane measures
- road sides to read road signs, etc.
No distractive activities (reading, eating, etc.) 12
 Quality Control vs. Quality Assurance: What’s the
Difference?

Analogy: You Driving on a

Freeway

Driving Quality Assurance

Before Driving: adjust seats, mirrors, temp, etc. Driving Quality Control
During Driving: Swerving to avoid deer crossing the highway
Occasional glances at: Steering to right if car is straying into left lane
- speed gage to ensure speed is not excessive Braking to avoid hitting slowed car in front
- side mirror before changing lane
- road sides to read road signs, etc.
No distractive activities (reading, eating, etc.)
During and After Driving: Generate/report data to prove These are
that necessary precautions were taken and that your
driving did not violate any rules
corrective 13
Quality Control vs. Quality Assurance: What’s the
Difference?

• Quality Assurance- “Do it right the first time". Preventive Quality checks.

• Quality Control: Fix it when ever it goes or is going wrong

•
But note! In recent years, QA is defined to include QC
Newer Definition of QA:
The acts of (i) establishing that product is being delivered according to all specifications,
(2) providing evidence needed to establish confidence that all quality-related activities are
being performed effectively
- Includes that part of project monitoring that covers project quality.
- Covers all activities from design, development, production, installation, servicing and
documentation
- Includes regulation of the quality of raw materials, assemblies, products and
components; services
14
Quality Control vs. Quality Assurance: What’s the
Difference?

Important
Exact definitions of QA-QC may vary from…
industry to industry,
agency to agency,
country to country.
15
 The Relationships:
Quality Control, Quality Assurance, Project
Monitoring

Project
Monito
ring Monitoring of other
project Quality
performance Assuranc
metrics
e
(time,
money, Monitorin
g of
etc.) project Qualit
quality y
Contro
l

16
 More on Quality Assurance (QA)

• Involves regular but random testing of materials and

workmanship (time-based or work-based intervals)
• Prevent, identify, and correct quality-related problems
• During the production process, QA instructors mostly
provide guidance and leadership to the production
people rather than criticizing their work

17
 More on Quality Control (QC)

• QC mostly carried out by Contractor, but …

the Owners rep. may draw Contractor’s attention to carry out QC at any
time. Owner also can have his independent testing.
• When the QC Division and Production Division belong to same organization,
these parties often have adversarial relationships
• The production people often tend to cover and hide their mistakes by nature.
• During/after every project activity, Contractor provides Owner with
documentation that the product complies with specifications

18
 What is Quality Management?

• Initiated and orchestrated by senior managers

• Involves all parts of the organization (main C, sub-C, suppliers, etc.)
• Through a systematic, comprehensive and well-documented QA process
• Rationale: Improved QA serves to enhance all other performance metrics of
the project (time, cost, etc.) in long run
• Objective: A “zero-defects” product
• Technique: Preventive as well as corrective -- Ensure quality NOT ONLY by
rejecting failures but ALSO by investigating possible risks and threats to quality
at root sources

19
 What is Total Quality Management
(TQM)?

• Not just an operational strategy for PM – A PM

philosophy
• Aimed at continuous improvement of the organization
and personal growth of its individual members
• Ensures that the environment surrounding the Product
is viewed in the broadest sense including:
- Quality of Life
- Well-being and satisfaction of all people Involved
- Long-lasting relationships with customers and suppliers etc.

20
 Linkages b/n Quality and Key Performance
Tradeoffs Metrics

Acceleration  $(Overtime, shift Need for rework imposes

work, Rework, higher-end equipment,
better crews etc.) $ high expenses
High quality needs can lead
to costly miscalculations
on labor time
Slow progress  $
Delayed occupation,
Trying to save $
Higher interest on
Can lead to substitution,
const. loan
lower quality
Loss of tenants
Less $  Low progress workmanship
Opportunity cost
Resource reduction
Selection of poor quality workers
Default of contractor/subs

Time Quality level impacts speed of work, Quality

Level of rework

Quality problems may result from

overtime, shift work, new hires
21
 Project
Reviews
• Going back to examine the product from the project.
- Has product (or part thereof) been produced?
- When was it done? punctual?
- Was it done right?
- With the needed resources? Within the given time?

• Reviews enable us to learn from mistakes as well as successes.

• Models used in PM should also be reviewed for their efficacies
– Example CPM, WBS/OBS/CBS, fishbone, etc.

• Project reviews may be carried out by external parties (e.g. consultants)

• However, transience of project activities/teams can impair the

accumulation of institutional knowledge and thus can impede
Project Learning
Source: F. Pena-Mora 2003 22
 The Need for Project Reviews

• Why do we review projects?

– Bridging Gaps
– Validation of Work Done
– Quality Assurance
– Learning

23
 More on the Need for Project Reviews

• Why construction projects in particular need

formal reviews:
– Design and Planning Phase Generating 75 % of the Problems
Encountered in the Construction Site  Need for
Understanding and Coordinating the Contract Documents and
Technical Specifications

– Current era: construction very fast paced  Errors More Likely

Frequent critical reviews are needed more and more to keep up
with the pace.
– Rising requirements for high quality and corporate effectiveness

24
 Other Benefits of Project Reviews

• Other effects:
- Knowledge Transfer
- Team Building
• These are positive side-effects rather than pursued goals
• Team Building is a much desired side effect particularly for
projects that are highly segmented (such as construction
projects)

25
Project Review – How to do
it?
Specific Approaches (Tools) for reviewing projects

– Peer Reviews

– Project Meetings and Walkthroughs

– Inspection

– Any combo of the above

26
 Project Meetings – A Useful Tool for Project
Review

• Purposes and Types of Project Meetings

– Assessing what has been done, what has

happened?
• Reviews – Discussion
• May also include inspections – Visual validation

– Assessing why it has happened: Audits

• How have resources been used?

27
 How to Conduct a Successful Project Meeting

• Schedule your meetings well ahead of time to increase likelihood of

participation
• Meetings should have well-defined purpose and agenda
• Keep project meetings very impersonal!
• Goal is to focus on problems rather than people
– Discuss process rather than ad-hoc decisions
– Keep focus on needs of project
• Avoid holding meetings that are perceived as “witch hunting” or challenging
some project party or team member
• Recriminating meetings …
– … do not make progress
– … can lead to ongoing rifts among personnel
– … can lead to defensiveness and emotions that can overwhelm rational analysis

28
Some Categories of Project
Reviews

• Level of the Review: Technical vs. managerial

• What is the orientation? Work output-oriented vs.

process-oriented

• Status of the Reviewers: Internal review vs. peer

review

• Tool for the Review: Meetings, Inspections, etc.

29
Project Reviews at Technical Level

• Focus on
– Technical problems and issues
– Interdependencies b/n design and construction methods
• Typical technical review program at project start-up
– System requirements review
– System design review
– Preliminary design review
• Typically, project reviews become more technical over time

30
Project Reviews at Management
Level
• Focus on overall practices and process
– Communication Channels
– Information Coordination
– Teamwork Effectiveness
– Client Relationships
– Supervision Efficiency
– Reliability
– Contract Management
– Learning Programs
– Overall and general (not specific) levels attained for the performance
measures
• Cost
• Quality
• Safety
• Performance
31
Technical vs. Managerial Reviews

FEASIBILITY DESIGN, DEVELOPMENT CLOSEOUT OPERATIONS

PLANNING

1 2 3 4

Strategic System Detailed Development

Planning Design Design

Managerial Managerial Managerial Managerial

Peňa-Mora ,et. al., 2003

Technical Technical Technical Technical 32
 Output-Oriented vs. Process-Oriented
Reviews

• Work output-oriented reviews

– Seek to identify issues with completed work
– Primarily focused on shorter-term issues

• Process-oriented reviews
– Seek to identify problems with processes
– Tend to be focused on longer-term
– Can recourse to higher-order reviews

• Reviews could be comprised of both

33
 Project Reviews by Peers

• An alternative to Project Review by Internal personnel

• Informal review conducted among specialists in the same field,
focused on a particular project aspect
• Focus is on work rather than project team
• Benefits:
– Early Discovery of Mistakes and Reduction of Rework
– Avoidance of Similar Pitfalls in the Future
– Enhancement of Team Spirit
– Promotion of Learning
• Documentation Limited to Memos and Duration not Exceeding
30 to 60 Minutes

34
 Project Review by
Inspection
• Inspection viewed as one form of formal technical reviews
• A formal review: technical or managerial personnel analyses of
the quality of
– an original piece of work product
– the process itself
• Inspection in construction: substantial completion, final
completion, …
• Importance of documentation and formalism to foster feedback
and diffusion of learned lessons

35
 Project Review – What form does it
take?

Major Forms of Overall Project Reviews:

– Value Engineering Review

– Constructability Walkthroughs

– Substantial Completion Inspection

– Final completion inspection

Each of these forms may involve one of

more review tools previously discussed
36
 Forms of Major Project Reviews I –Value
Engineering

• Value engineering is the process of:

– Identifying and quantifying the performance of various systems in
the design of a facility;
– Evaluating the costs and benefits of alternative solutions that
achieve similar or better performance for the same or lower costs

• Reviews are necessary for two reasons:

– Need of extensive communication and collaboration
– Uncertainty in the construction industry

37
 Design Essentials
Forms of • Dimensions of Elements (e.g., beams, columns, structural steel etc.)
• Distances between elements (e.g., spans, story heights etc.)
Overall • Changes in dimensions and distances between floors

Project • Repetition and modularity of dimensions and distances

• Constructability of details (e.g., distances between re-bars)
Reviews • Formwork considerations (e.g., availability, size and wasted material)

Constructabilit • Variations in soil and foundation conditions

II - • Landscaping
y • Structural and architectural details (e.g., steel connections)
• Alternative solutions of similar performance but better economies
Walkthroughs • Maintainability of the structure
Construction Methods
• Aggressive environment & weather condition effects
• Loads & deflections during construction
• Electrical and hydraulic conduit locations
• Large equipment accessibility and topography
• Availability of utilities
• Existing infrastructre and societal requirements during constrution
• Compatibility of the location with the construction strategy
• Simultaneous co-location of construction teams (and interference)
• Skills availability
Design Deliverables
• Lack of references
• Lack of details
• Conflicting details
• Ambiguous or conflicting specifications
• Poor quality construction drawings
• Conflicting construction and shop drawings
Administrative
• Standardization of contracts
• Contract language
• Incomplete or changing specifications and requirements
• Availability of qualified construction inspectors or site engineers
• Communication of the design rationale
Communication of critical components and issues
38
Forms of Overall Project Reviews III – Substantial
Completion Inspection

• Is a formal inspection that follows the contractor’s request and

a subsequent number of walkthroughs towards the end of the
works.
• The project manager, the owner, the engineer and the
architect inspect the (almost) completed project, and decide
whether it is suitable for “occupancy for its intended purpose.”
• The SCI is a formal, contractually prescribed review, with
important legal significance.
• Often serves as trigger for release of retainage
• Sometimes followed by final completion inspection

39
 Summary
Approaches, Functions, and Forms for Project
Review

• Approaches for Project Reviews: In form, formality and

functionality, these range from informal peer reviews to
contractually prescribed inspections with legal significance.
• Key functions of Project Review: (1) Where has project reached
and how/when, (2) quality assurance, and (3) knowledge transfer
and learning.
• Forms of Project Review established by the construction industry:
Value engineering reviews, Constructability walkthroughs and
Substantial completion inspections.

40
 Project Audits – What are they?

• Definition: A thorough examination of the management of a

project (methodology and procedures, records, properties, budgets
and expenditures, and degree of completion. (Meredith and Mantel, 2006)

• Difference:
Project Review: What problems are there? When did they occur? How severe?
Project Audit: Detailed form of Project Review
Mostly, Why did the problems occur? More formal and more documented.
• Audits are the most formal and comprehensive form of project review.

• Goal here is to understand why project is in its current state

• Project audits may be for the entire project or only for parts thereof
• Project audits present the best opportunity for formal learning.
41
 Contents of a Project Audit

• Current status of the overall project: Why work actually

completed does or does not match the planned level of
completion?

• Future status: Likelihood of any changes in schedules, cash

flows, etc. Why?

• Status of critical activities: Why the current and predicted status

of tasks could make or break the project

• A critical examination of project management issues at all

management levels

42
Contents of a Project Audit
(cont’d)

Risk assessment and analysis: What is the potential for project

failure or monetary loss?

Lessons learned from (or to) other similar projects

Assumptions, limitations and quality of data used in the audit

Comments and conclusions by the author of the audit

43
 Who Needs the Audit?

• The project manager, who seeks unbiased and comprehensive

information from groups or individuals within the project
organization.
• The organization, which seeks to identify the errors made, track
their causes, and learn not to repeat them.
• The client, who can relate the value of project development to
their own actions and decisions.
• Any external stakeholders or sponsors of the project; financial
institutions, government agencies, consumer groups,
environmental or religious organizations and social groups.
44
 Financial, Management and Project Audits
Meredith J.& Mantel S., 2000
FINANCIAL AUDIT MANAGEMENT AUDIT PROJECT AUDIT
Compares the current
Compares the current
organizational
status of the business to Compares the actual
breakdown structure and
STATUS the accepted standard status of the project to
culture, to the
(ex. Industry, the planned one.
competition and
competitors)
standards.
The future status of the
The company’s project including
The company’s state
management culture deviations from
PREDICTIONS related to economical
related to future market schedule, cost, quality,
growth
needs. natural and socio-
political environment.
Benchmarking, Strengths- Financial terms,
Weaknesses- schedule progress,
MEASUREMENT Finance-accounting Opportunities-Threats resource utilization,
CRITERIA methods/terms (SWOT) analysis, market quality –goals
analysis, statistical established by the
information contract.
No standard record-
Standard finance- No standard system, uses
keeping system, uses the
accounting format records available in the
RECORD KEEPING one established by the
established by legal company or from
SYSTEM contract, company, or
criteria and professional analyses, produces
general applied
standards. report.
practices.

Many, depending on the

Few or none depending Usually covers
results of the research
RECOMMENDATIONS on the management management issues and
and the objectives of the
system risk mitigation.
audit.

Management consulting
Sometimes, lack of
firms usually undertake
Professionally and legally technical expertise
QUALIFICATIONS such audits, with
qualified auditors auditors, lack of funds
experienced and
and time for audit.
qualified personnel. 45
 Focus on Project Auditing –
Scopes
General Audit (overview) Detailed (administrative) Audit Technical Audit

A focused and detailed

Includes the general
General overview investigation into
overview, focuses and
prescribed in technical issues that
What elaborates on
contractual present learning
administrative issues in
arrangements. opportunities or
greater depth.
problems

After Milestones, on Usually early in project

When After Milestones demand, according to development, or in post-
need. mortem evaluations

Internal auditing Technically qualified

Internal Auditing
Who department, external personnel, external
Department
consultants experts.
So that they provide
Under schedule and cost In context of the project
How technical knowledge for
constrains ecosystem
the future.
Peňa-Mora ,et. al., 2003
46
Causal Relationships examined in
a Project Audit

What is affecting What?

47
 Causal Relationships examined in
a Project Audit

• Causal Relationships and the Dynamic Interaction between …

– Schedule – Cost – Quality
– Staff Experience & Skill – Quality – Productivity
– Project Requirements – Organization Culture
– Stakeholders – Value Perception – Value Delivery
– Schedule, Cost and Quality – Strategic and Tactical Decisions
– Product-Specific Requirements – Work Breakdown Structure, Organizational
Breakdown Structure, Product Development Model
– Information Flow & Record-keeping – Work Breakdown Structure, Organizational
Breakdown Structure
– Monitoring Configuration – Corrective Actions – Time Delays– Ultimate Effects
– Resource Planning – Cost – Schedule – Performance
– Hiring Policy – Cost – Quality – Performance
– Financial Strategy – Explicit and Implicit Constraints Imposed – Strategic and
Tactical Decisions

48
 Project Audit Essentials

It is essential that a project audit …

• Succeeds in serving as a Learning Tool that consists of
Two Separate Processes:
– Generation of the Lesson
– Adoption of the Lessons by the Organization or Project Team

• Consists of some key ingredients such as:

– An effective Audit Team
– Unfettered Access to Project Records
– Communication With Project Personnel and Others
– Ability of Audit Team to investigate Exogenous Causes of Project
problems
– The Audit Should be Truthful and Honest
– Auditing Performance of Project Personnel
– Audit Report Distribution
49
 The Need for Communication With
Personnel during an Audit

• Free flow of information from project personnel to

audit team
• Two reasons for communication problems:
– Unavailability of certain personnel
– Distrust towards the audit team
• Personnel mental and psychological training to
audits (pre-announced scheduling can help!)
• Compromise between friendliness and
professionalism of the audit team

50
 Auditing the Performance of Project
Personnel

• Evaluation of Personnel Performance

Productivity, creativity, commitment, quality , response to unplanned
situations , leadership, teamwork, adoptability to the project and
organizational culture and relationships with other personnel.

• Evaluation Often Viewed As criticism and threat to the Profession

• Solution:
– Focus on Situations, Not Individuals
– Personnel Performance Taking Into Account the Organizational
Structure and the Culture of the Organization

51
 Audit Report
Distribution

• Selective report distribution

• Importance of defining a distribution list for the
report early in the life-cycle of the audit
• Comprehensibility of the report to all the
addressees
• Implementation of information technology in
report distribution

52
Phases of a Typical Project Audit

Initiation
Planning
Execution
Reporting and Release

53
Phase 1 : Initiation
• Setting of the Pursued Goals
• Formal Determination of Audit Scope: length, formality,
the recipient list and the parameters of the project
• Tasks Involved:
– Determination of the goals of the audit
– Expert Team Formation.
– Establishing the Purpose and the Scope of the Audit.
– Notification of the Project Team

54
Phase 2: Planning

• Three Major Work Elements:

– Determine the cost, time and technical constraints that do or
will govern the auditing effort
– Determine the means and methods to be used in the audit
– Agreeing on a performance baseline

• This Step Includes:

– building the questionnaires , the checklists and the data
gathering forms
– planning an interview schedule

55
 Phase 3: Audit Execution

• Investigation Starts With Data Collection

• Investigation Follows With Data Analysis and Investigation,
Focusing On:
– Assessment of the Project Organization, Management, Methods and
Controls
– Statement of both Current and Former Status
– Preliminary Statement of Forecasted Project Status
– Working Quality Assessment
– Delivered Quality Assessment
– Lessons Learned – Action Plan

• Audit Execution Followed by Self-Scrutiny Review

56
 Phase 4 : Report Preparation and
Release

• Audit Report should conform to the audit requirements and

needs
• Report should be released according to the distribution plan
• Importance of planning the logistics of report preparation and
release
• The policies for conducting and distributing the report:
- balancing political correctness
- ease of assimilation of the lessons
- truthfulness
- brevity.

57
 Phase 5: Finishing up the Project
Audit
• Three Essential Steps:
– Audit Database and Document Filing by audit service
management division
– Post-Audit Consulting: the last chance for the project team to
discuss with and learn from the auditors
– Audit Program Evaluation. The effectiveness, quality,
sophistication and depth of the audit are examined.
- the problems the audit team encountered during their job are
also listed

58
 Project Audit
Report

• A formal document presenting all the work done and the

conclusions reached

• Table of Contents for a Typical Audit Report:

– Introduction: Project and Audit Scope, Objectives and Circumstances
– Current Project Status: entire project ecosystem
– Future Projection
– Recommendations about changes in technical approach, budget and
schedule for the remaining tasks
– Risk Assessment and Analysis:
– Limitations and Assumptions of the Audit : listing of Time, depth, lost
records, negative attitudes, focus in specific directions, assumptions and
poor inputs
59
 Project Audits -
Summary
• Use Project Audits to reveal the interactions between elements in the
project ecosystem and their effects on the project.
• Also to investigate the project (or a part of it). In doing so, they must be
honest and objective. Only then will they be useful.
• Plan audits carefully: Establish a baseline for comparison, but be open to
in-depth investigation if necessary. If the project is long or complex, plan
the Audit as a continuous learning processes.
• Make audits worthwhile: Introduce the auditing procedure as an integral
part of the product development process. Teach people to appreciate it
and learn from it. It works for them, not against them.

60
Questions?

61