ECM 523: CONSTRUCTION

PROJECT MANAGEMENT
Chapter 7.0 :
Project Evaluation and
Auditing
Presented by: Suman Mishra
MSc. in Civil Engineering
University of Calgary,
Canada
 Project
• Evaluation
Evaluation is an objective and systematic judgmental
process for determining relevance, efficiency,
effectiveness and impact of project performance.
• It is an assessment of project during implementation.
• Evaluation is done to improve project
implementation and to improve future project
planning and decision making.
Project evaluation is a systematic and objective
assessment of an ongoing or completed project. The
aim is to determine the relevance and level of
achievement of project objectives, effectiveness,
efficiency, impact and sustainability.
It focuses on the Scope, Quality, Schedule, Budget,
Resources and the Risk of Construction Project.
 Evaluation
• What? Evaluation is an assessment that refers to assessment of
implementation and results of completed or on-going project / program / policy.
design,
• How? Evaluation should be systematic and with objectives. It should consider
relevance, fulfillment of objectives, developmental efficiency, effectiveness, impact
and sustainability.
• Why? Evaluation should provide credible and useful information to enable the
incorporation of lessons learned into the decision-making process.

Evaluation comprises three things:

Whether we are doing the right
things Whether we are doing it right
Whether there are better ways of
doing it
 Need and
Importance
• The objectives of project evaluation are :
• To verify whether the project implementation progress is as planned.
• To take corrective measures for deviations in performance.
• To ascertain that actual costs are within the budgets.
• To ensure that quality standards are being attained.
• To identify unexpected problem areas and manage them.
• To bring about overall improvement in project performance to
achieve
project objectives.
 Evaluation and purpose
• Determine relevancy of the project and decide go and no-go with the
• project
See efficiency and effectiveness of the project and improve
implementation
project
• Observe project performance and correct deficiencies
• Observe impact of the project on beneficiaries
• Improve future project planning
• Lessons for future projects for better planning - a learning
strategy
• Enhance decision making for future project
• An effective tool for project management
• Track project progress and take corrective action for deviations
• An objective look on the planned operations
 Project Evaluation
Myth
Myth: a widely held but false belief or idea.
Myth # 1: Evaluations only take place in the middle or at the end of a project
• The truth is that an evaluation can take place at any point before, during or after
the project cycle.
• Formative evaluations take place in the lead up to the project, as well as during
the project in order to improve the project design as it is being implemented
(continual improvement).
• While summative evaluations occurs at the end of a program cycle and provides
an overall description of program effectiveness.
• In recent years real time evaluation has emerged.

Just as the name suggests, Real time evaluation (RTE) is an approach with 'the primary objective is to provide
feedback in a participatory way in real time (i.e. during the evaluation fieldwork) to those executing and
managing the programme'
Myth #2: The types of evaluation are limited
The truth is that there are different types of available depending
evaluation on
context and the available information and resources.
• needs assessment
• Formative evaluationdetermines who needs the program, how great the need
is,
and what might work to meet the need
• evaluability assessment determines whether an evaluation is feasible and how
stakeholders can help shape its usefulness
• structured conceptualization helps stakeholders define the program
or technology, the target population, and the possible outcomes
• implementation evaluation monitors the precision/accuracy of the program or
technology delivery
• process evaluation investigates the process of delivering the program
or technology, including alternative delivery procedures
• Summative evaluation
• outcome evaluations investigate whether the program or technology
caused demonstrable effects on specifically defined target outcomes
• impact evaluation is broader and assesses the overall or net effects -- intended
or unintended -- of the program or technology as a whole
• cost-effectiveness and cost-benefit analysis address questions of efficiency by
standardizing outcomes in terms of their dollar / rupees costs and values
• secondary analysis re-examines existing data to address new questions or use
methods not previously employed
• meta-analysis integrates the outcome estimates from multiple studies to arrive
at an overall or summary judgment on an evaluation question
Myth # 3: Only the success stories from an evaluation have value
• Oftentimes the motivation behind an evaluation is a desire to show that a project
was a success.
• This is to prove that public/private funds or donor moneys were well spent and
targeted people / communities were well served. However, most evaluations show
mixed results and the 'brilliant failures' of a project are just as useful as the
successes.
• One example comes from a health, education and water program in Mali. The
evaluation findings revealed that failure to establish terms of engagement led to
divisions and miscommunication affecting the program's implementation.
• As a result of this finding, the different stakeholders developed a Memorandum of
Understanding.
• The moral of the story is that a 'failure' that is unearthed by an evaluation can
(and should be) be used for learning and program improvement.
6 Ways of Measuring Project
Success
Measuring the success of a project once it’s brought to completion is a
valuable practice. It provides a learning opportunity for future
undertakings, and, the opportunity to assess the true effectiveness of the
project.
1.Scope
2.Schedule
3.Budget
4.Team satisfaction
5.Customer satisfaction
6.Quality
 Project
Success
“As Project Manager, juggling all of the
balls is important, but keeping your eye
on the right ball is the key to delivering
truly successful projects” –
Robert Goatham
• Tier 1 – The project was a success if it
delivers all or most of what it said it
would (the scope), regardless of
schedule or budget performance
• Tier 2 – The project was a success if it
delivers what it said it would,
schedul onand/or within the
e agreed
• budget
Tier 3 – The project was a success if it
delivers what it said it would,
on
schedule, within the agreed
budget and to the expected quality
standards
• Tier 4 – The project was a success it
delivers on all agreed project
objectives, be they scope, schedule,
budget, quality or outcomes based
(i.e. goals to be achieved or strategic
positions to be attained)
• Tier 5 – The project was a success if
the product produced by the project
creates significant net value for the
organization after the project is
completed.
Project Success
criteria
Factors for Project Success and Failure
?
Since projects are temporary in nature, the success of the project should be
measured in terms of completing the project within the constraints of scope, time,
cost, quality, resources, and risk as approved between the client and contractor
(executing and construction bodies)

The project manager is responsible and accountable for setting realistic and
achievable boundaries for the project and to accomplish the project within the
approved baselines. The success of the project will depend on how well project
time, cost, scope and quality meet the requirements established at the beginning
of the project.
Success or Failure Factors in Relation to the Initial Project
Definition
• project scope (the extent of work required) is not clearly stated and
understood;
• the technical requirements are vague;
• estimates of cost, timescale or benefits are too optimistic;
• risk assessment is incomplete or flawed;
• the intended project strategy is inappropriate;
• insufficient regard is paid to cash flows and the provision of funds;
• the interests and concerns of stakeholders are not taken into account;
Success or Failure Factors in Relation to the Initial Project
Definition
• undue regard is paid to the motivation and behavior of the people who will
execute the project;
• particularly in management change projects, insufficient thought is given to how
all the managers and workpeople affected by the project will adapt to the
changes expected of them;
• approval to proceed with the project is given for political, personal or intuitive
reasons without due consideration of the business plan.
 Types of evaluation based on time
Ex-ante evaluation - is performed before programme implementation and its
objective is to assess whether the planned intervention is accurate with regard to
needs as well as coherent with reference to planned aims and how they will be
implemented.
On-going evaluation - has a supplementary character for all above-mentioned
evaluation types and can be conducted independently. The on-going evaluation is
carried out throughout the period of implementation of an intervention; however it
can not be taken for monitoring, as it consists of the in-depth assessment of the
chosen problems that have appeared during other evaluations.
Mid-term evaluation is the evaluation performed towards the middle of the
implementation of an intervention. This evaluation critically considers the first
outputs and results, which enable assessing the quality of programme
implementation.
 Types of evaluation based on time
Thematic evaluations focus on the analysis of a selected part of the policy, and
this
analysis is of the cross-sectional and/or comparative character.
Ex-post evaluation is the evaluation of an intervention after it has been
completed. It should be carried out not later than three years after the end of
the implementation period. The ex-post evaluation aims at examining long-lasting
effects of a programme and their sustainability.
Evaluator and
Purpose
Role of Evaluator
• To track project progress and take corrective actions.
• To check the efficiency and effectiveness of the project and improve
project implementation.
• To suggest improvement for future planning.

Purpose
• To observe project performance
• To observe impact of project on beneficiaries
• Lessons for future project for better planning- a learning strategy
• Enhance decision making for future project
Method of
Evaluation
Data collection methods
The most popular data collection methods for the evaluation needs
are:
document analysis,
individual interviews,
questionnaires,
focus groups,
observation and
group techniques
 Project
Audit
-A formal review of any aspect of a project
- An audit is systematic, independent, documented
assessment using standards and set criteria.

Project auditing can be defined the

as
process of detailed inspection of the
management of a project, its
methodology, its techniques, its
procedures, its documents, its properties,
its budgets, its expenses and its level of
completion.
A project audit is a key step in the process
of closing a project. This audit evaluates
the total project processes and outcomes.
 Purpose and
Goals
• The main goal of an audit is to
-inspect and evaluate the current state of project realization,
-find out to what extent it complied with defined criteria for project
success
and
-identify opportunities for improving the project realization
management.
and
• This inspection can be performed after the project completion or after the end
one
of project realization stage, or it can be performed during the
project realization.
Benefit of Project
Audits
Project audits can help identify when a project is about to go off-course.
In addition, a project audit can provide the following benefits:
• Improve project performance.
• Increase customer and stakeholder satisfaction.
• Save costs.
• Control scope and avoid scope creep.
• Provide early problem diagnostics.
• Clarify performance/cost/schedule relationships.
• Identify future opportunities for improvement.
• Evaluate performance of the project team.
• Inform client of project status/prospects.
• Reconfirm feasibility of/commitment to project.
 Need of Project Audit
• Audits can be triggered when:
• Situations come about which do not give a clear picture of the happenings on
a project.
• A doubt arises on whether the project deliverables are being achieved.
• Technical gap seem to be apparent.
• The progress on the project is vastly in variance with the project plan.
• There are doubts about the organization of the project and its ability to
produce results.
• A need for technical changes is felt, to see whether they conform to
best
practices in the industry.
• Needs to pinpoint defects and suggest the best methods to move forward to
proper implementation.
• Projects that have a number of milestones and deliverables are best audited
at regular intervals.
• Existing documents and data would have to be studied.
• Key team members would have to be interviewed to understand the
processes which they have followed for execution of the project.
• When audit completed reports would have to be prepared which would give
a summary, the background that initiated, and risks identified.
• It must also lay down clearly the effects of the risks perceived and their
likely
outcome on the project in case they are ignored.
Components of Audit Planning
Process
1. Research the Audit Area

2. Maintain Open Communications Throughout the Planning

Process
3. Conduct Process Walk-Throughs
4. Map Risks to the Organization, Process, or Function
5. Obtain Data Prior to Fieldwork
Types of
Audit
Basically two types of auditing
• External auditing is the verification process involving the independent
appraisal of financial accounts and statements.

• Internal audit is performed by the internal auditing staff of the

organization itself to ensure that its assets are properly safeguarded
and its financial records reliably kept.
 Types of Project Audit
• Financial – Financial audits typically involve a focus on financial controls as they
relate to reporting. These audits focus on accounting controls present in the
general ledger or sub-ledger systems. Financial statement auditing is the focus
of
our external auditors. Internal Audit will complement the work they
based
performon an agreed plan.
• Operational – Operational audits focus on the review and assessment of a
business process. The activities of the business process may result in a direct or
indirect financial impact to the organization such as the collection of student
tuitions or patient account balances. Internal Audit primarily focuses on
operational audits but can extend the scope to include accounting procedures
that can impact financial reporting.
• Compliance – Compliance audits review the level of compliance with internal
policies or external regulatory requirements.
 Types of Project Audit
• Investigate Audit- This is an audit that takes place as a result of a report of
unusual suspicious activity on the part of an individual or department. It usually
focus on specific part of work of a department or individual.
• Follow-up Audit- These are audits conducted approximately six months after an
internal or external audit report has been issued. They are designed to evaluate
the corrective actions that has been taken on the audit issues reported to the
original report.
• Integrated Audits – Integrated audits look at controls that address financial,
operational, compliance and information systems risks. These audits are typically
centered on a business cycle or a specific part of a cycle or process.
 Purpose of audits will
be:
• Determining
• What is going right and why?
• What is going wrong and why?
• Identifying
• Forces and factors that have prevented or may prevent achieving cost,
schedule, technical performance goal
• Evaluating
• Efficacy of existing project management strategy including organizational
support, policies, procedures, practices, techniques, guidelines, action
plans, funding patterns, human and other resources utilization
• Providing opportunity
• To exchange ideas, information, problem, solutions and strategies
 Key components of project
• Engineering
audit
• Manufacturing
• Finance and accounts
• Contracts
• Purchasing
• Marketing
• Human resources
• Organization and
management
• Quality
• Reliability
• Tests
• Logistics
• Construction etc.
 Depth of Audit
Time and money limit the depth of an audit.
Audits are distracting to those working on the
project. A poor audit result will lower morale on the
project.
Planning & Procurement Implementation Impact of
Design / Purchase Project
•Need analysis & Demand •Reduction in operation cost
analysis •Timely completion of or improvement in operation.
•Technical Specification •Compliance with project •Increase market share
preparation and Financial Manual •Provision of liquidity •Improvement in Key
authorization •Independent damage in case of Performance Indicator (KPI)s
•Quality Assurance evaluation delay in project •Increase customer loyalty
and Experience of of received completion. or decrease customer churn
Supplier required in proposal. •Acceptance Test •Increase Revenue of the
the project Report company.
 Major task of Project Audit
• Identifying the need for evaluation and selecting the topics/themes to be
evaluated.
• To continue project implementation as planned or to re orient or in the
worst case to stop the project.
• Ensuring the production of a high quality evaluation report and of
the dissemination of evaluation findings and recommendations.
• Concerning the questions, whether or not, in future similar projects
should be initiated.
Life-cycle of Project Audit
• Project Audit Initiation
• Project Baseline Definition
• Establishing an Audit Database
• Preliminary Analysis of the
Project
• Audit Report Preparation
• Project Audit Termination
 Auditor’s responsibility

• General
• The auditor must maintain independence in mental attitude in all
matters related to the audit.
• The auditor must have adequate technical training & proficiency to perform
the audit.
• The auditor must exercise due professional care during the performance of
the audit and the preparation of the report.
 Auditor’s responsibility

• Standards of Field Work

• The auditor must adequately plan the work and must properly supervise any
assistants.
• The auditor must obtain a sufficient understanding of the entity and its
environment, including its internal control, to assess the risk of material,
misstatement of the financial statements whether due to error or fraud, and
to design the nature, timing, and extent of further audit procedures.
• The auditor must obtain sufficient appropriate audit evidence by performing
audit procedures to afford a reasonable basis for an opinion regarding the
financial statements under audit.
• The auditor must assign qualified personal to the engagement.
• Standards of Reporting
• The auditor must state in the auditor's report whether the financial
statements are presented in accordance with generally accepted accounting
principles.
• The auditor must identify in the auditor's report those circumstances in which
such principles have not been consistently observed in the current period in
relation to the preceding period.
• When the auditor determines that informative disclosures are not reasonably
adequate, the auditor must so state in the auditor's report.
• The auditor must either express an opinion regarding the financial
statements, taken as a whole, or state that an opinion cannot be expressed,
in the auditor's report.
• In all cases where an auditor's name is associated with financial statements,
the auditor should clearly indicate the character of the auditor's work, if any,
and the degree of responsibility the auditor is taking, in the auditor's report.
Step of Project
Audit
• Critically review project documents
• Interview project team and stakeholders – to gain insights
project perceptions and affairs
• Participate in some project activities – to gain an appreciation of project
opportunities and problem
• Prepare and submit audit report
 Technical
A systematic Auditof a company’s
analysis Research & Development activities
conducted by an outside expert can be called a technical audit. The audit
examines all current R & D activities, to make an honest appraisal of their
potential and worthiness. After an audit, priorities on all projects should be clear
to all involved.

Technical Audit (TA) is an audit performed by an auditor, engineer or

subject-matter expert evaluates deficiencies or areas of improvement in a
process, system or proposal. Technical audit covers the technical aspects of the
project implemented in the organization.

In general, technical audit means regular monitoring and examination (continuous

audit) of construction works adherence to the sustainability, quality improvement,
cost effectiveness, schedule effectiveness.
 Technical
Audit
• It is a tool to monitor the overall (work/activities) performance of the project.
• It help to maintain the construction work adherence to the appropriate quality,
sustainability of construction work, on schedule, in time, correction of
technical deficiencies and defects if any.
• It target to achieve the economy, efficiency, transparency, accountability, zero
tolerance on corruption and increment in value for money.
In Nepal, we don’t have long history on technical audit. However, in 2056 B.S
amendment on financial administration regulation (FAR) introduced the technical
audit and its management section at NPC (National Procurement Commission).

World bank had supported the ministry of physical planning and works for capacity
enhancement related activities and technical audit works from the public entity.

To avoid the conflict of interest in technical audit within the same public entity, it
has been transferred to the National vigilance Centre (NVC) by Shrawan 1st 2060
B.S. however, in 2063 B.S as per PPA this responsibility has been assigned to
(Public Procurement Monitoring Office) PPMO but PPMO is only responsible for
Procurement related issues rather than technical audit of construction works. So
then after the responsibility has been transferred to NVC.
Objectives of Technical
auditing
• The technical operations are being performed as per requirement.
• Sound framework of control is in place to sufficiently mitigate the potential
risk.
• The procured technical equipment is technically suitable for the purpose.
• Authority and responsibility for operating activities are assigned properly.
• Information system is adequate to provide assurance of operating
activities
being performed properly.
 Scope of Technical
• Audit
Cost effectiveness
• Timeliness
• Quality assurance
• Efficiency and
effectiveness
• Outcome
• Value for money
Stages of technical
audit
In general, technical audit shall be carried out it four different stages of the
project
• Design phase
• Procurement phase
• Construction phase
• Operation phase
Design
Phase
In this phase, technical audit regarding the effectiveness and relevancy of
conceptual study, feasibility study, work plan, data collection, site selection,
cost estimates etc. are examined.

Procurement Phase
In this phase, technical audit regarding procurement plan, procurement
strategies, selection of procurement method, procurement process, use of PPA,
PPR, PPMO guidelines, preparation of bidding documents, bid or proposal
evaluation, contract agreement, effectiveness in implementation of agreement,
timeliness, quality, transparency, competitive process, accountability etc. are
examined.
Construction
Phase
In this phase, technical audit regarding the use of technical norms, quality
standards, drawings and designs, cost estimates, work schedule, human resources
of contractor, material, fund mobilization, workmanship in terms of effectiveness
in cost , quality, time etc. are examined.

Operation Phase
In this phase, technical audit regarding sustainability, relevancy of design,
environmental study, effectiveness of cost and benefit analysis, effectiveness of
outcomes, operation cost, effectiveness of maintenance cost, participation for
people etc. are examined
Process of TA and Criteria to be an Technical
Auditor
• In Nepal, NVC usually schedules and defines the programmes related to TA.

• The project implemented by GON and Donor Agencies are taken for TA in sample basis which
shall be programmed by NVC.
• NVC mobilizes the third party consultant for the TA work.
• NVC prepares the standing list of third party consultant.
• NVC sets the criteria for Technical Auditor as:
• Bachelor in engineering + 10 years experience in relevant field
• Registered in Nepal engineering Council

• NVC manage the training programme for short listed TA

• NVC open for the proposal (technical and financial) submission from short listed TA and select
Lowest evaluated substantially responsible proponent.
• NVC prepare the TOR and undergoes the agreement with the selected consultant
TA shall examine the following during technical
auditing
• Effectiveness of cost estimates, timeliness, quality and result including following details
• Relevancy of design
• Possibility in getting proposed result
• Effectiveness and sustainability of project
• Relevancy of Conceptual study, feasibility study, procurement of services, works,
goods,
quality standards
• Budgetary provisions and work plan
• Goal setting and objective setting
• Work schedule (start of activities and end of project)
• Progress of work schedule and use of work schedule
• Quality related issues and its handling
• Budgetary provision for operation and maintenance
TA shall follow following Methodologies for above mentioned
TOR

• Questionnaire and Document

reviewing
• Examination
• Reporting
• Immediate communication
 Financial Audit
• A financial statement audit is conducted to provide an opinion whether the
financial statements (the information being verified) are stated in accordance with
specified criteria.
• The primary purpose for financial audits is to give regulators, investors, directors,
and managers reasonable assurance that financial statements are accurate and
complete.
• In providing an opinion whether financial statements are fairly stated in
accordance with accounting standards, the auditor gathers evidence to determine
whether the statements contain material errors or other misstatements.
• The audit opinion is intended to provide reasonable assurance, but not absolute
assurance, that the financial statements are presented fairly, in all material
respects, and/or give a true and fair view in accordance with the financial
reporting framework.
Environmental
Audit
• Environmental auditing is essentially an environmental management tool for
measuring the effects of certain activities on the environment against set criteria
or standards.
• Depending on the types of standards and the focus of the audit, there are
different types of environmental audit. Organizations of all kinds now recognize
the importance of environmental matters and accept that their environmental
performance will be scrutinized by a wide range of interested parties.
Environmental auditing is used to
• investigate
• understand
• Identify
These are used to help improve existing human activities, with the aim of reducing
the adverse effects of these activities on the environment.
Environmental
Audit
• Environmental auditing should not be confused
assessment (EIA). Both environmental auditing
with environmental impact
and EIA are environmental
management tools, and both share some terminology, for example, 'impact',
'effect', and 'significant', but there are some important differences between the
two.
• Environmental impact assessment is an anticipatory tool, that is, it takes place
before an action is carried out (ex ante). EIA therefore attempts to predict the
impact on the environment of a future action, and to provide this information to
those who make the decision on whether the project should be authorized. EIA is
also a legally mandated tool for many projects in most countries.
• Environmental auditing is carried out when a development is already in place,
and is used to check on existing practices, assessing the environmental effects of
current activities (ex post). Environmental auditing therefore provides a 'snap-
shot' of looking at what is happening at that point in time in an organization.
Social
Audit
A social audit is a way of measuring, understanding, reporting and ultimately
improving an organization's social and ethical performance. A social audit helps
to narrow gaps between vision/goal and reality, between efficiency and
effectiveness.

Social auditing creates an impact upon governance. It values the voice of

stakeholders, including marginalized/poor groups whose voices are rarely heard.
Social auditing is taken up for the purpose of enhancing local governance, particularly
for strengthening accountability and transparency in local bodies.

The key difference between development and social audit is that a social audit
focuses on the neglected issue of social impacts, while a development audit has a
broader focus including environment and economic issues, such as the efficiency of a
project or program.
Social
Audit
A social audit looks at factors such as company’s record of charitable giving, volunteer
activity, energy use, transparency, work environment and worker pay and benefits to
evaluate what kind of social impact a company is having in the location where it
operates.

Social audit is based on the principle that democratic local governance should be
carried out, as far as possible, with the consent and understanding of all concerned.
It is thus the process not an event.
Objectives of Social
Audit
• Assessing the physical and financial gaps between needs and
resources available for local development.
• Creating awareness among beneficiaries and providers of local social
and
productive services.
• Increasing efficacy and effectiveness of local development programmes.
• Scrutiny of various policy decisions, keeping in view stakeholder interests
and
priorities, particularly of rural poor.
•Estimation of the opportunity cost for stakeholders of not getting
timely access to public services.
Advantages of social
audit
• Encourages local democracy.
• Encourages community participation.
• Benefits disadvantaged groups.
• Promotes collective decision making and sharing
responsibilities.
• Develops human resources and social capital
DIFFERENCE BETWEEN EVALUATION AND
AUDIT
• Evaluation is a part of the Management Cycle(planning
implementation evaluation)
• Audit is independent from Management Cycle since it comes into
scene after the completion of Management Cycle.
• Evaluation talks about doing the right thing while audit talks about how
the things are done.
• Evaluation involves doing it right while audit involves managing it right.
• Evaluation is about sustainability while audit is about efficiency.
• Evaluation approach follows good practices while audit works
against norms.
Similarities

Though audit and evaluation differ in the skills and methods used and
practiced, there are also some basic similarities between the two. Both
methods analyze the data objectively and within defined standards and
protocols. Both are conducted by professionals and fall under committees.
These are mutually supportive and organizationally close. The results of
both should be posted and easily accessible.
THANK
YOU