CCAF-FCVI Fellow

2009/2010

A Risk-based Approach to
Performance Auditing
Strategic Paper by
Levina Rusk Kishimba

TANZANIA

CANADA

Table of Contents
Page
Acknowledgements

Executive Summary

vii

Introduction

Background

Objective

A Risk-based Approach to Performance Auditing

Context
Problems to be addressed
Rationale for using one-pass planning in the office
Challenges ahead
Description of the Strategy
Phase 1Create awareness and share knowledge
Phase 2Train performance auditors
Phase 3Develop OPP approach
Phase 4Learn by doing
Proposed Approach
Phase 1Briefing sessions with management and staff
Phase 2In-house training
Phase 3Workshops
Phase 4Pilot project for the selected entity
Evaluation of the Strategy and Next Steps

2
2
2
2
3
4
4
4
4
5
5
5
5
6
6
6

Conclusion

Appendix AProject Implementation Schedule

Appendix BSteps to Preparing a One-Pass Plan

11

Bibliography

14

iii

Acknowledgements
Praise is to the Lord God who always guides my every step and keeps me in the centre
of his will.
This strategy paper is the result of a nine-month International Fellowship Program
I attended in Canada from August 2009 to May 2010. The fellowship was awarded by
the Canadian Comprehensive Auditing Foundation (CCAF) with funding provided by
the Canadian International Development Agency (CIDA).
Special thanks and appreciation to CCAF; CIDA; the Auditor General of Canada,
Ms. Sheila Fraser; and the staff of the Office of the Auditor General of Canada
(OAG Canada) for organizing and running the program.
I would also like to express my gratitude to OAG Canadas Canada Revenue Agency
Performance Audit Team (Group 7, Team 8), the people who ran the Offices training
program, and International Relations staff for the support they gave me in developing
my strategy paper and gaining a lot of knowledge in performance auditing.
My participation in this fellowship program would not have been possible without the
vision of the Controller and Auditor General of the United Republic of Tanzania,
Mr. Ludovick Utouh, in improving the knowledge and skills of his staff.
I sincerely thank my mentor, Mr. Ronald Bergin, Principal of Strategic Planning at
OAG Canada, for sharing his time and insight throughout the development of my
strategy paper.
I am very grateful to my darling husband, Timothy, who was always there to support
and encourage me during my stay in Canada.
Many thanks also to all 20092010 CCAF International Fellows for their friendship.
It was nice spending time with them.
Finally, I wish to extend my deep and lasting appreciation to all who, in one way or
another, helped me throughout the fellowship program.
Levina Rusk Kishimba
May 2010

Executive Summary
This strategy paper proposes how the National Audit Office of Tanzania (NAOT) will
adapt and implement the Office of the Auditor General of Canadas one-pass planning
approach to selecting performance audits. One-pass planning (OPP) is a risk-based
audit planning approach which focuses on how well an entity is managing its major risks
rather than on areas of suspected weakness.
Preparing one-pass plans for performance audits will provide assurance to Parliament
and other stakeholders that NAOT is following a systematic and independent, risk-based
and objective approach to selecting the areas for audit. This will demonstrate that it is
efficiently using the available resources. In addition, this approach will ensure that any
entity deemed to be significant has a current, multi-year plan based on a high level of
risk analysis.
This approach will be implemented incrementally. It will begin with a briefing for
management and staff, followed by training for an initial core of staff on preparing and
applying an OPP approach to audit selection. Current audit structures will be adopted to
suit the OPP process, and then a pilot survey will be carried out. The new approach will
be monitored, evaluated, and adjusted throughout the implementation process.

vii

Introduction
Background
1.
In the past, long-term entity planning in the Office of the Auditor General of
Canada (OAG Canada) focussed on determining the value-for-money audit priorities.
Planning was carried out from the perspective of making a difference for Canadians by
identifying areas that likely needed improvement. As a result of a project entitled
Advancing Audit Practices, approved in June 2001, OAG Canada implemented a more
systematic, integrated, and risk-based approach to long-range entity planning, referred
to as one-pass planning (OPP).
2.
The term one-pass signifies that the knowledge of both performance and
financial auditors is brought together in one analysis at the beginning of the process to
help document the auditors understanding of the risks that a particular entity is facing
and how well it manages those risks. Near the end of the process, the auditors again
consider a combined analysis to determine to what extent they will use financial audits
and performance audits to address the issues they wish to audit.
3.
The National Audit Office of Tanzania (NAOT) has not yet established a strategic
planning process to ensure that only relevant matters involving significant risks are
selected for performance audits. Resources allocated for performance audits are always
limited, which makes selecting entities or areas to be audited a key issue. Putting into
place a process which allows the preparation of one-pass plans for the audit entities
would enable the Office to focus its limited resources on areas of greatest risk.

Objective
4.
This strategy paper proposes the integration of a risk-based approach to
selecting potential performance audits in the office of the Controller and Auditor
General (CAG) of the United Republic of Tanzania, namely, one-pass planning (OPP).
OPP will assist with the selection of audits based on significance and business risks (i.e.
risks to the achievement of the entity's objectives). It will use resources more effectively
and reduce the time and cost of conducting an audit. This will therefore help NAOT
improve efficiency in the planning phase of its performance audits.
5.
The main objective of this paper is to provide a strategy which will enable NAOT
to implement an OPP approach as a tool for selecting performance audit topics and to
integrate the preparation of one-pass plans into the strategic planning process of the
office by July 2012.

A Risk-based Approach to Performance Auditing

Context
Problems to be addressed
6.
The National Audit Office of Tanzania (NAOT) does not have a formal way to
select its performance audits. The audits are selected based on complaints from the
public and problems identified by the media. There are various risks associated with this
approach, including:

choosing topics that are sensational but not fundamental to a government

departments mandate;

choosing topics that are sometimes hard to audit, for example, due to a lack of
background, difficulty determining scope, or risk of missing important areas; and

risking the Offices credibility by picking irrelevant topics for political reasons (not
being independent of the political process of the day).

7.
Other problems include limited resources allocated for performance audits and a
lack of cooperation from auditees.
Rationale for using one-pass planning in the office
8.
A risk-based approach offers what may be the best way for NAOT to
demonstrate independence and objectivity in addressing the problem of deciding what
areas, entities, or themes should be chosen for performance audit.
9.
As used by the Office of the Auditor General of Canada (OAG Canada), the onepass planning (OPP) approach emphasizes that planning at the entity level should
address all mandate areas of OAG Canada simultaneously. It applies to audit planning
for programs and functional areas within and across federal departments and agencies.
By focusing on areas of greatest risk to entities, OAG Canada is moving toward an
assurance-based perspective. This reinforces its goal of addressing the areas of
greatest significance to entities and the expectation that its audits can result in a positive
opinion about the management systems and practices it examines. Applied to NAOT,
this approach could also improve its relationships with auditees.
10.
Further, since adopting OPP, OAG Canada is in a better position to assure
Parliament that audit efforts have been devoted to areas of highest importance, and that
OAG Canada is achieving an appropriate balance of effort with the limited resources
available for the audit. It is in a better position to respond when questions are asked
about why it did or did not audit certain areas. Similarly, adopting an OPP approach for
the audit entities would enable NAOT to focus its limited resources only on areas of
greatest risks.

Challenges ahead
11.
Performance Audit is a small division of NAOT, with only 11 auditors. Success in
implementing the approach will involve some challenges that can be overcome in
different ways by the implementation committee. The committee will include
performance auditors, the selected financial auditors, Development Plans Unit officials,
the Controller and Auditor General and the Assistant Auditor General, Value for Money.
Its worth noting that the success of the whole process is highly dependent on the
support and involvement of NAOT management. The following sections describe
expected challenges.
12.
Performance auditors are not familiar with risk-based approaches to
performance auditing. Identifying and assessing risks are activities central to OPP.
None of the performance auditors at NAOT has technical expertise, experience, or
training on using risk-based approaches to selecting matters for performance auditing.
Making such a fundamental change will be a challenge, and therefore some
performance auditors may be reluctant to adopt this new approach. This challenge can
be addressed by explaining the benefits of using OPP and training the auditors on how
to prepare and use the approach.
13.
Ministries, departments, and agencies (MDAs) as well as local government
authorities do not identify and assess risks that would prevent them from delivering their
programs. OAG Canada uses the auditees own identification of risks as a check in its
OPP approach. In Tanzania, MDAs do not identify and assess their risks. To address
this challenge, the NAOT performance auditors, with the assistance of MDA officials, will
identify and assess the entities risks.
14.
Relations between auditees and NAOT performance auditors are strained
and currently not conducive to good collaboration. Sharing information and a
collaborative spirit are very important to the success of a one-pass plan. Lack of
cooperation has been identified as an issue and is being addressed by another Fellow
from NAOT. Both projects are running at the same time, which could have represented
either another challenge or an advantage. In this case, the Fellow who will be
implementing his strategy paper Developing an Auditees Guide to the Performance
Audit Process, will be helpful because his paper will help inform the selected pilot
project entity about what NAOT will be doing before the pilot project begins.
15.
The NAOT does not have expertise yet to train its performance auditors to
conduct OPP. The key players at OAG Canada who use OPP are the principals and
directors of audit teams, the Strategic Planning and Professional Practices group, and
the consultants. To address NAOTs lack of expertise, I will transfer the knowledge I
have gained from OAG Canada effectively to all performance auditors. Further, an OPP
Team will conduct a pilot study as a training project. In addition, consultations and
advice will be applied whenever required.

16.
OPP takes time and resources. OAG Canada uses between 500 and 600 hours
to prepare a one-pass plan for a single department. In addition, it has a total of
103 performance auditors. The NAOT performance auditors represent a tiny portion of
the office operations (11 performance auditors out of around 600 employees), which
could prompt some there to wonder why the auditors should get the money required to
put the OPP approach into place. Also, OPP does not produce visible results in the short
term, a condition which usually makes it harder to get funds and resources in light of
competing priorities.
17.
Nevertheless, NAOT has accepted the implementation of the risk-based
approach (OPP) in selecting its performance audit topics, and so funding will not be a
fundamental issue. Since the office has a limited number of performance auditors, the
approach will be adapted to suit our local conditions.

Description of the Strategy

18.
The one-pass planning (OPP) approach will be implemented in four phases,
each including a number of supporting activities.
Phase 1Create awareness and share knowledge
1. Submit the strategy paper to the Controller and Auditor General (CAG) and the
Assistant Auditor General, Value for Money (AAG-VFM) for review and approval.
2. Prepare agenda for the meetings to be conducted.
3. Select financial auditors for the implementation committee.
4. Hold briefings with management, all performance auditors, selected financial
auditors, and Development Plans Unit officials, about the approach and the cost
involved (human resources, financial, and time).
Phase 2Train performance auditors
1. Prepare and review presentations and other material.
2. Deliver training to performance auditors on preparing an entitys one-pass plan.
Phase 3Develop OPP approach
1. Conduct workshops with the CAG, the AAG-VFM, and all performance auditors
to develop the OPP approach for the Office.
2. Form the team of at least three performance auditors (OPP Team) to prepare a
one-pass plan for a small entity to be determined.
3. Develop the approach (the one-pass plan format to be used in the Office).
4. Submit the prepared approach to the CAG for approval.

Phase 4Learn by doing

1. Plan a pilot one-pass plan for the selected entity.
2. Prepare and conduct the one-pass plan for the entity.
3. Write the one-pass plan report.
4. Present the one-pass plan report to NAOT management.
5. Finalize the one-pass plan report and submit it to the CAG for approval.
6. Introduce results of the one-pass plan to NAOT management and the
implementation committee.
7. Present the approved one-pass plan report to the management of the entity.
8. Prepare for the audit (select the risks to base it on).
9. Undertake a full performance audit for the selected entity based on the selected
risks.
10. Integrate the OPP approach into the Offices strategic planning process.

Proposed Approach
19.
Since its inception in 2005, the NAOT Value for Money Audit Division has been
working hard to build capacity and achieve intended objectives through workable
budgets and action plans. To ensure that this strategy is smoothly implemented, the
OPP Team will frequently consult with NAOT management and hold a number of
meetings with them during the project planning stage.
Phase 1Briefing sessions with management and staff
20.
I will hold a briefing with the CAG and AAG-VFM to discuss the strategy and its
objectives. Then, they will review the implementation schedule and provide their
feedback and approval. The AAG-VFM will select financial auditors to be involved during
the briefing session and on the implementation committee. I will then review OPP
discussion papers and other related material obtained from OAG Canada before
preparing an agenda for the coming briefing session. The session will include
management and other staff and focus on OPP, its related concepts, and the rationale
for adopting and implementing OPP as a risk-based approach to selecting our audits.
Phase 2In-house training
21.
I will refer to the knowledge Ive gained and reading material Ive obtained from
OAG Canada when preparing presentations on how to conduct OPP. I will review the
presentations before starting the in-house training with the auditors.

Phase 3Workshops
22.
All performance auditors will be involved in developing the OPP approach to be
used in the office. The CAG and the AAG-VFM will also participate in developing the
approach. Then, we will hold workshops to develop a simplified approach which will suit
our local conditions. We will review and discuss the Integrated Risk Management
process and OAG Canadas OPP process to determine what will apply to developing a
one-pass plan format for the office.
Phase 4Pilot project for the selected entity
23.
The pilot one-pass plan will be used as a training project for learning how to
conduct them. Further, it will be helpful in determining whether the developed approach
(the adapted OPP process) works, and hence in identifying what adjustments need to be
made before integrating it into the Offices strategic planning process and putting it into
practice across NAOT.
24.
The performance auditors selected to develop the Offices OPP approach (OPP
Team) will conduct a pilot one-pass plan using the agreed upon format. Once the pilot
project is complete, the team will hold a lessons learned session with management and
other staff to identify what worked well and what could be improved. The team will then
write an OPP report and forward it to both NAOT management and the surveyed entity.
Based on the results of the report, NAOT management and performance auditors can
decide on a list of possible audits to be conducted for that particular entity over the next
three years, as well as the timing for the audits.
25.
Through this exercise, the implementation committee will become familiar with
the approach. The approach will then be introduced to other entities (at all levels of the
government) to reinforce the understanding of the government-wide audit issues and the
role of all players.

Evaluation of the Strategy and Next Steps

26.

The successful implementation of this strategy will be measured by

the completion of a one-pass plan for the selected entity as a pilot project by
July 2011;

the integration of OPP into the strategic planning process of the office by
July 2012; and

the number of performance audits that will successfully be completed on time

and within budget as well as the quality of performance audit products of the
office, effective July 2012.

27.
This success will also depend on the support of other stakeholders inside and
outside NAOT. Therefore, the Office must ensure that all of its stakeholders are aware of
this new approach to its performance audit work.

28.
Further, efforts will be made to ensure that all activities in each phase of the
strategy are completed within two years (from July 2010 to July 2012).
29.
The newly developed approach will then be monitored, evaluated, and adjusted
on a periodic basis to ensure continuous learning and improve performance audit
reporting.

Conclusion
30.
The National Audit Office of the United Republic of Tanzania faces a complex
challenge in selecting its performance audit topics. This makes the Office inefficient
during the planning phase of an audit. The huge number of audit entities and the limited
resources available for performance audit work makes the task of choosing priority areas
to audit very critical. The introduction of one-pass planning as a risk-based approach to
selecting matters for audit will enable the office to make optimal use of its resources and
provide assurance to the parliamentarians, central and local governments, and the
public that the office is fulfilling its responsibilities.

Appendix AProject Implementation Schedule

Activity

Time frame

Key player(s)

Phase 1Create awareness and share knowledge

1. Submit the strategy paper
to the CAG and AAG-VFM for
review and approval

June 2010

Vacation

July 2010

Levina R. Kishimba

2. Prepare agenda for

meetings

August 2010

Levina R. Kishimba

3. Select financial auditors for

the implementation
committee

September 2010

The AAG-VFM

4. Hold briefings with

management, all
performance auditors,
selected financial auditors,
and Development Plans Unit
officials

September 2010

Levina R. Kishimba

(3rd and 4th week)

(1st week)
Levina R. Kishimba

(2nd, 3rd and 4th week)

Phase 2Train performance auditors

1. Prepare and review
presentations and other
material

October 2010

Levina R. Kishimba

2. Deliver training to
performance auditors on
preparing an entitys onepass plan

October 2010

Levina R. Kishimba

Phase 3Develop OPP approach

1. Conduct workshops to
develop the OPP approach
for the office

November 2010

2. Identify the entity and OPP

team

November 2010

3. Develop the one-pass plan

December 2010

4. Submit the prepared

approach to the CAG for
approval

(1st to 3rd week)

The CAG, the AAG-VFM, and

all performance auditors
The CAG and the AAG-VFM

(4th week)

(1st and 2nd week)

All performance auditors, the

CAG and the AAG-VFM

December 2010

Levina R. Kishimba

(3rd week)

Activity

Time frame

Key player(s)

1. Plan a pilot one-pass plan

for the selected entity

January to April 2011

OPP Team

2. Prepare and conduct the

one-pass plan for the entity

January to April 2011

OPP Team

3. Write the one-pass plan

report

May 2011

OPP Team

4. Present the OPP report to

NAOT management

June 2011

Levina R. Kishimba

5. Finalize the one-pass plan

report and submit it to the
CAG for approval

June 2011

6. Introduce results of onepass plan to NAOT

management and
implementation committee

June 2011

7. Present the approved onepass plan report to the

management of the entity

June 2011

8. Prepare for the audit

(select the risks to base it on)

June 2011

9. Undertake a full
performance audit for the
selected entity based on the
selected risks

July 2011 to June 2012

OPP Team and other

performance auditors

10. Integrate the OPP

approach into strategic
planning process of the office

July 2012

OPP Team

Phase 4Learn by doing

10

(1st week)
Levina R. Kishimba

(1st week)
Levina R. Kishimba

(2nd week)

Levina R. Kishimba

(3rd week)

(4th week)

OPP Team and other

performance auditors

Appendix BSteps to Preparing a One-Pass Plan

The OAG Canada audit teams consistently follow a series of steps in analysing business
risks that are critical to the entitys success and reporting the results of the analysis to
both the executive committee and the entity.

1. Conduct interviews and review documents

This step involves interviewing key officials of the entity and the officials who are
charged with its stewardship responsibilities to identify its objectives. Further, the audit
team interviews OAG Canada personnel with experience auditing the entity. The audit
team then reviews all key documents needed to assist in analysing the risks facing the
entity.

2. Document knowledge of the entity

The second step in the OPP process involves documenting knowledge of the entity. The
audit team records a description of the entitys objectives, responsibilities, and expected
results. The team also prepares a summary of the following:

interviews completed,

the entitys enabling legislation,

its other key mandates and financial authorities,

its mission and objectives,

its strategies to achieve objectives, and

its plans for business and operational structure.

3. Prepare the entity risk profile

In conducting the risk analysis, the audit team uses a generic government risk model
to help identify the most significant internal and external risk factors believed to exist in
government. In using the model, attempts are made to capture the most common factors
liable to result in hazards or risks that will prevent the entity from achieving its mandate
along with risks to good governance and operations. Having identified the significant
internal and external risks, the audit team prepares an entity risk profile that includes:

a brief description of the risk and its impact on the entitys mandate, governance,
and operations;

a brief description of how internal and external risks to the entity affect the entity;

11

an assessment of the impact of the risk, as well as its likelihood (categorized as

High, Medium, or Low); and

a description of any previous audit work conducted by OAG Canada to address

the risk identified.

4. Prepare the control profile

The audit team prepares an entity control profile to document the assessment and
consideration of the key controls within the entity. This profile helps to determine
whether the controls are adequate for addressing the risks identified in the entity risk
profile and for supporting the conclusion on the overall control environment. The entity
control profile has two main components:

Summary assessment of key controls: This provides an overview of the key

controls that make up the overall control environment within the entity. Each
control is assessed to determine its presence in a given area and its importance
in responding to the risk identified in the entity risk profile. The audit team must
also document the strengths and weaknesses of each of the controls.

Consideration and assessment of key controls: A description of the key

features of each control objective is documented, along with the strengths and
weaknesses of the control objective and any factors that may mitigate the
weaknesses identified.

5. Identify potential products and assign priorities

Potential audit products for the next three to five years are identified and assigned
priority using the following key procedures:

12

Identify possible products: The audit products identified will depend on the
focus of the Auditor General and will usually address multiple risks facing the
entity. Care is exercised to limit the products to the risks that were given higher
priority.

Respond to entity risks: The proposed product is aligned to the risks that were
identified.

Assess relative priority: This procedure is undertaken after considering

whether the product supports the Auditor Generals focus areas and whether
there has been previous audit coverage or other independent reviews of the
subject matter.

Document risks to the office: These are the risks of delivering (auditability) or
not delivering (credibility) the proposed product.

6. Report to the Executive Committee

The final step in the OPP process is to prepare and present a report to the Executive
Committee of OAG Canada. The discussions during the presentation are kept at a high
level, with attention only to key risks facing the entity that have not been mitigated. The
report is checked by a quality reviewer, who provides feedback on its completeness. It is
then presented to the Executive Committee.

13

Bibliography
Bergin, Ron. Strategic Planning in the Office of the Auditor General of Canada,
Presentation to the Canadian Comprehensive Audit Foundation Fellows, October 2009.
Hopwood, Tom and Wiltshire, Collin. OAG Risk Strategies for the Next Decade.
Discussion paper on use of risk concepts in OAG planning, auditing, and management,
September 2001.
Office of the Auditor General of Canada, Strategic Planning and Professional Practices.
One-Pass Planning Guidance to the Entity Team, March 2002.
_____. Update on our Strategic Plan Challenges, PowerPoint presentation,
October 2003. Linked to the following document:
http://notes.oag-bvg.gc.ca/intranet/intranet_menus.nsf/html/e_index.htm
_____. Performance Audit Manual, June 2004.
_____. Guidance on Preparing One-Pass Plans, September 2004. Available from:
http://notes.oag-bvg.gc.ca/intranet/intranet_menus.nsf/html/e_index.htm
_____. One Pass Plan for the Canada Revenue Agency, 2007.
_____. One Pass Plan for the Government of Nunavut, 2007.
Treasury Board of Canada Secretariat. Integrated Risk Management Framework, 2001.
Available from: http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12254
_____. Integrated Risk Management Framework: A Report on Implementation Progress,
March 2003. Available from: http://www.tbs-sct.gc.ca/rm-gr/irmf-cgir/2003-03rprt01_e.asp

14