ND 682 Lamp - Anindya Atma Zulatsari - 20240515115514

Gedung Prof. Dr. Mr.
Prajudi Atmosudirjo, Komplek Fakultas Ilmu Sosial dan Politik

UNIVERSITAS INDONESIA Kampus UI Depok 16424, Indonesia
T.62.21.78849087 F.78849050
FAKULTAS ILMU ADMINISTRASI Kampus Salemba, Gedung Mardjono Reksodiputro
Jl. Salemba Raya No. 4 Jakarta 10430, Indonesia
T.62.21.3913606-7 F.3921088
Eadm@ui.ac.id | www.fia.ui.ac.id
Nomor : S-830/UN2.F16.D5/PDP.04.04/2024 20 Maret 2024

Sifat : Biasa
Perihal : Permohonan Izin Pengumpulan Data
Lampiran : -
Yth. Kepala Biro Humas dan Kerja Sama Internasional

Badan Pemeriksa Keuangan Republik Indonesia
di Jalan Gatot Subroto Kav. 31, Bendungan Hilir,
Tanah Abang, Kota Jakarta Pusat, Daerah Khusus
Ibukota Jakarta 10210
Sebagai salah satu syarat untuk menyelesaikan studi di Program Studi Ilmu Administrasi
Fakultas Ilmu Administrasi Universitas Indonesia, mahasiswa diwajibkan untuk melakukan
penulisan Tesis. Terkait dengan hal tersebut, bersama ini disampaikan permohonan
mahasiswa atas:
Nama : Anindya Atma Zulatsari

Nomor Pokok Mahasiswa : 2306308750
Program Studi : Ilmu Administrasi
No Handphone : 085843839807
Email : anindya.atma@ui.ac.id
: Assessing The Influence of Government Oversight on Audit
Judul Impact of Badan Pemeriksa Keuangan Republik Indonesia (BPK
RI)
untuk melakukan pengumpulan data sekunder berupa:

1. Pengisian kuesioner penelitian terkait audit eksternal pemerintah oleh pemeriksa/auditor
Badan Pemeriksa Keuangan
Demikian surat ini disampaikan, besar harapan kami agar Bapak/Ibu berkenan
menindaklanjuti permohonan ini dengan baik.
Atas perhatian Bapak/Ibu, kami ucapkan terima kasih.
Manajer Pendidikan dan Kemahasiswaan
Dr. Achmad Lutfi, S.Sos., M.Si

NIP 197603042008121001
Gedung Prof. Dr. Mr. Prajudi Atmosudirjo, Komplek Fakultas Ilmu Sosial dan Politik
UNIVERSITAS INDONESIA Kampus UI Depok 16424, Indonesia
T.62.21.78849087 F.78849050
FAKULTAS ILMU ADMINISTRASI Kampus Salemba, Gedung Mardjono Reksodiputro
Jl. Salemba Raya No. 4 Jakarta 10430, Indonesia
T.62.21.3913606-7 F.3921088
Eadm@ui.ac.id | www.fia.ui.ac.id
NOTA DINAS
Nomor : ND-830/UN2.F16.D5/PDP.04.04/2024
Yth. : Kepala Biro Humas dan Kerja Sama Internasional

Dari : Manajer Pendidikan dan Kemahasiswaan
Perihal : Permohonan Izin Pengumpulan Data
Sebagai salah satu syarat untuk menyelesaikan studi di Program magister Fakultas Ilmu
Administrasi Universitas Indonesia, mahasiswa diwajibkan untuk melakukan penulisan tesis.
Terkait dengan hal tersebut, Bersama ini disampaikan permohonan mahasiswa atas:
Nama : Anindya Atma Zulatsari

Nomor Pokok Mahasiswa : 2306308750
Program Studi : Ilmu Administrasi
HP : 085843839807
Email : anindya.atma@ui.ac.id
: Assessing The Influence of Government Oversight on Audit
Judul Impact of Badan Pemeriksa Keuangan Republik Indonesia (BPK
RI)
Untuk melakukan pengumpulan data sekunder berupa:

1. Pengisian kuesioner penelitian terkait audit eksternal pemerintah oleh pemeriksa/auditor
Badan Pemeriksa Keuangan
Demikian surat ini disampaikan, besar harapan kami agar Bapak/Ibu berkenan
menindaklanjuti permohonan ini dengan baik.
Atas perhatiannya, kami ucapkan terima kasih.
20 Maret 2024
Manajer Pendidikan dan Kemahasiswaan
Dr. Achmad Lutfi, S.Sos., M.Si

NIP197603042008121001
UNIVERSITY OF INDONESIA
Assessing The Influence of Government Oversight on Audit Impact of

Badan Pemeriksa Keuangan Republik Indonesia (BPK RI)
THESIS
Submitted as one of the requirements for

Master Degree in Administrative Science
ANINDYA ATMA ZULATSARI

2306308750
FACULTY OF ADMINISTRATIVE SCIENCE

ADMINISTRATIVE SCIENCE DOUBLE DEGREE PROGRAM
PUBLIC POLICY AND ADMINISTRATION
DEPOK
2024
TABLE OF CONTENTS
TABLE OF CONTENTS ................................................................................................ i

LIST OF TABLES .......................................................................................................... ii
LIST OF FIGURES ....................................................................................................... iii
CHAPTER I .................................................................................................................... 1
1.1 Background ............................................................................................................. 1
1.2 Problem Formulation .............................................................................................. 7
1.3 Research Purposes And Significances .................................................................... 8
1.4 Writing Structure ..................................................................................................... 9
CHAPTER II ................................................................................................................ 10
2.1 Literature Review .................................................................................................. 10
2.1.1 Previous Research Regarding Impact of Supreme Audit Institution .................... 10
2.1.2 Novelty of the Research ........................................................................................ 12
2.2 Theoretical Framework ......................................................................................... 16
2.2.1. Evaluation ............................................................................................................. 17
2.2.2. Performance Measurement.................................................................................... 27
2.2.3. Government Oversight .......................................................................................... 41
2.2.4. Audit Impact.......................................................................................................... 48
2.3 Research Model ..................................................................................................... 54
2.3.1. Conceptual Framework ......................................................................................... 54
2.4 Hypothesis Development ...................................................................................... 57
2.5 Concept Operationalization ................................................................................... 60
CHAPTER III ............................................................................................................... 62
3.1 Research Approach................................................................................................ 62
3.2 Data Collection Technique .................................................................................... 63
3.3 Population and Sampling ...................................................................................... 63
3.4 Data Analysis Technique ....................................................................................... 66
3.4.1 Descriptive Analysis .............................................................................................. 66
3.4.2 Validity and Reliability Test .................................................................................. 66
3.4.3 Classic Assumption Test ........................................................................................ 67
3.4.4 Hypothesis and Regression Analysis ..................................................................... 68
3.5 Research Limitation .............................................................................................. 70
REFERENCES ............................................................................................................. 71
i
LIST OF TABLES
Table 2.1 Matrix of Previous Research on Impact of Supreme Audit Institution .............. 10
Table 2.2 Comparison of Evaluation and Performance MeasurementError! Bookmark not
defined.23
Table 3.1 List of Respondents for Confirmation................................................................ 66
ii
LIST OF FIGURES
Figure 1.1 Strength of SAI Oversight ............................................................................2

Figure 1.2 Relative Strength of Oversight Ecosystem in ASOSAI Regional Groups ...3
Figure 2.1 General Model of Self-Auditing and Control ............................................27
Figure 2.2 Elements of Accountability ........................................................................42
Figure 2.3 Accountability Organization Maturity Model ............................................45
Figure 2.4 Audit Impact Model ...................................................................................55
Figure 2.5 Conceptual Framework ..............................................................................56
iii
CHAPTER I
INTRODUCTION
1.1 Background
Prudent management of public fund and services derives from the realization
that government resources are fundamentally established to support nation’s
objectives and the welfare of its people. The paradigm shifts to New Public
Management (NPM) in the 1980s has brought various reform in the public sector and
newly found focus on improvement in the quality of public services (Ferdousi & Qiu,
2013). As the quality of public service delivery is a palpable manifestation of the
efficacy of governance (Ketelaar et al., 2007), the public holds bigger demands for
better governance, especially the management of public fund. Public sector reforms
occur globally, partly in pursuit of better public service (Curristine & Flynn, 2013;
Boyne, 2003), improved resource management (Hemici & Kontogeorga, 2020), as
well as a response to the public's desire for a government that is more transparent and
accountable (Cordery & Hay, 2022; Aziz et al., 2015).
Due to the limitations of resources, the public expects their government to
utilize public resources efficiently, achieve effective results, and consistently aim to
reduce the cost of public services (Andrianto et al. 2021; Khotami, 2017). In the
current era of transparency and accountability, addressing this issue means ensuring
effective management of budgets and allocation of public funds. Budgets need to be
aligned with the objectives that the organization expects to accomplish (Hepworth,
2023) while still maintaining the capacity to generate desirable results (Campos,
2001). In this sense, Supreme Audit Institution (SAI) as an external auditing body
maintains a central position as it has the authority and ability to perform high-quality
and impactful assessments of government bodies (Iosefo, 2022). The expectations
placed upon SAIs have extended beyond basic audit functions of financial and
compliance auditing towards an evaluation of the effectiveness and value-for-money
(VfM) of government programs (van Zyl et al., 2009). SAI need to quickly adapt to
these demands and seek better strategies to strengthen its role while fulfilling its duty.
Independent external oversight conducted by SAI ensures the financial
statements of governments, and government bodies, examining compliance with
1
2
applicable laws and regulations (FAI, 2021). On the other hand, the concept of
“public value” (Moore, 1995; Denhadrt & Denhardt, 2003) also resonates with SAI’s
main goal to bring benefits to the public through audit work as mandated by the
INTOSAI Principle-12 “making a difference to the lives of citizens” in the Lima
Declaration (IDI, 2021b). Despite the optimistic view, an international survey
conducted by Open Budget Survey (OBS) across INTOSAI regions in 2020 further
demonstrates wide disparity in strengths and weaknesses throughout audit and
oversight systems. High-quality audit work provides a means of solution, but it can
only do so much without the auditee’s initiative to follow-up on the
recommendations. In general, the least robust elements of the oversight ecosystem
exhibit a consistent pattern, specifically in executive response, citizen participation,
and independent follow-up as shown by Figure 1.1 and Figure 1.2.
Figure 1. 1 Strength of SAI Oversight

Source: INTOSAI Development Initiative working paper “All Hands on Deck” (2020)
Following this ongoing concern, INTOSAI Development Initiatives (IDI) and

International Budget Partnership (IBP) conducted a joint initiative to uncover issues
regarding SAI audit impact around the world. International Budget Partnership (IBP)
and INTOSAI Development Initiative (IDI) are both internationally recognized
organizations that conducted surveys globally in which the result, method, and
indicators are legitimate to be used as standard measures for assessing the impact of
audits on government oversight worldwide.
University of Indonesia
3
Conclusively, the overall global overview on this matter is noted to be

relatively weak for years and shows a significantly uneven level of strength in the
oversight system (2020, p. 14). INTOSAI recognizes the complex challenges faced
by SAIs around the world (IDI, 2021a), but it has not been able to establish a
standardized framework that could assist SAIs in audit impact measures and
optimization.
This left SAIs to determine audit impact measurement and conduct an
independent evaluation on their own, essentially requiring SAIs to address and
analyze challenges unique to their country and decide the best approach. In response,
INTOSAI took a step further and launched the Facilitating Audit Impact (FAI) to
identify challenges and propose alternatives in the issue regarding strengthening
audit impact. This task force ultimately addresses the need for more transparent
conduct of public fund management as well as collaborative efforts between SAI and
the government (FAI, 2021). These reports cover SAIs around the world, including
Indonesia, one of the South Asia countries and part of the ASOSAI region where the
score of the oversight system has been recorded low in several aspects. For instance,
the component of audit impact measured in Asia region SAI or Asian Organization
of Supreme Audit Institutions (ASOSAI) scores highly in the institutional framework
but poorly in public participation and executives’ response as illustrated in Figure
1.2. Consequently, this calls for an in-depth evaluation of audit impact from a more
holistic point of view to identify room for improvement.
Figure 1. 2 Relative Strength of Oversight Ecosystem in ASOSAI Regional Groups

Source: INTOSAI Development Initiative working paper “All Hands on Deck” (2020)
In Indonesia, this process is carried out in accordance with the state finances
package of laws in 2003 and 2004 consisting of the Law of State Finance, the Law
of State Treasury, and the Law of State Financial Management and Accountability.
4
Oversight of public expenditure management is carried out by Indonesia Audit Board

or Badan Pemeriksa Keuangan (BPK). As the government's external auditor and SAI
of Indonesia, BPK is the entity responsible for conducting audits on various
government agencies, programs, and activities. BPK as part of ASOSAI has actively
taken part in these initiatives and consolidating ideas through discussions and sharing
insights in knowledge transfer forums. During the 51st Governing Board Meeting in
2017, BPK held a seminar with the diverse ASOSAI representatives' experiences
regarding SAI PMF in the Asia region. In 2023, BPK also hosted the ASOSAI
Capacity Development Administrative (CDA) forum, where it facilitated dialogues
among the ASOSAI community concerning the strategic approach to increase SAI's
capability to meet public expectations.
President Joko Widodo in his opening speech of the 17th Assembly Meeting
of the International Organization of Supreme Audit Institutions (INTOSAI) Working
Group on Environmental Auditing (WGEA) also emphasizes the crucial role of BPK
as the Indonesian SAI to promote participation, transparency, accountability, and
engagement of the stakeholders within the government (Secretary Cabinet, 2016). As
an independent external auditing body, BPK serves the mandate to examine
government institutions established and regulated specifically in Article 23E, Article
23F, and Article 23G of the Constitution of the Republic of Indonesia. BPK shares
the same responsibility with the government to accelerate the realization of the
nation’s mid- and long-term goals. To affirm this commitment, BPK also reflected its
vision in the Handbook of Strategic Planning of 2020-2024 to be a trusted auditing
organization that plays a significant role in achieving high-quality management of
public funds and improving state finance governance to sustain the country’s
objectives (BPK, 2020).
As part of the oversight mechanism for public fund management, the
Indonesian government enacted three sets of financial laws in 2003-2004. According
to the Indonesian Law Number 17 Year 2003 On State Finances, governments in
Indonesia must provide financial statements and submit them to the Regional House
of Representatives for auditing by the Indonesian Supreme Audit Board (BPK).
Furthermore, Law Number 15 Year 2004 stated explicitly that officials are obliged to
provide answers, explanations, and or execute solutions to address audit
5
recommendations suggested by BPK in its audit report, leading to the increased

significance of BPK's monitoring on government financial management and overall
performance. BPK plays a vital role in enhancing the efficiency, responsibility, and
openness of the Indonesian government by conducting financial, performance, and
non-compliance audits. Reports generated from these audit works guide BPK in
providing audit opinions as a means to enhance the transparency and accountability
of the respective government institutions (Pramono & Wibisono, 2022). However,
Wibowo (2019) noted that the current public sector audit has not been sufficient in
improving the effectiveness of government programs and the necessity to reach
beyond financial or compliance aspects toward the ability to offer solutions,
alternatives, preventive measures, better internal control, and other valuable
information for decision-making purposes.
Over the years, BPK has developed a monitoring system to measure the extent
of audit impact based on follow-up by the government and/or relevant officials. An
audit recommendation is a suggestion from the auditor based on the results of a
thorough audit work addressed to a government body to whom lies the authority to
deliver improvement responsibly within their organization (Budiman, 2021). BPK
actively oversees the implementation of the audit recommendations through Tindak
Lanjut Rekomendasi Hasil Pemeriksaan (TLRHP), a compilation of detailed records
consisting of audit work, findings, as well as follow-up status updates. This progress
is periodically recorded and published every 6 (six) months within a semi-annual
Audit Summary namely Ikhtisar Hasil Pemeriksaan Semester (IHPS).
In the latest issue of the 2022 Second Semester Audit Summary, the report
showcases an overview of the implementation of audit recommendations from 2005-
2022 in which BPK has filed 669,268 recommendations which generated Rp312,44
trillion in value. However, only 77% or 517,602 of the recommendations equalling
Rp153,84 trillion in value have been resolved while the rest of 23% or 151.666
recommendations totalling Rp158,6 trillion remained unsolved (BPK, 2022). Such
major progress in recent years is acknowledged but it is important also to note that
quite a substantial number of BPK findings are repetitive as they gradually tend to
address recurring problems every year (Alkam & Bastian, 2019; Octaviani, 2018;
Asis & Supriyadi, 2017; DPD RI, 2023;).
6
This phenomenon indicates weakness in government control and how BPK

audits have brought low level of impact (Alkam & Bastian, 2019, p. 2). Congruent
with Morin's (2001) assertion that relying solely on monitoring the adoption of
recommendations to determine the success of an audit is an oversimplified method.
If recommendations failed to address the root cause of problem at hand, then
recommendations may inflict only minimal changes to the system (Budiman &
Amyar, 2021) while the same issues persist. This drew an understanding of how
positive follow-up progress does not necessarily surmise a successful audit impact.
In regard to this issue, audit institutions generally employ diverse methods to
collect information which encompasses both quantitative and qualitative approaches
to analyze its impact (Lonsdale, 2000; Oţetea, Tiţa, & Ungureanu, 2015). However,
there has not been any standardized guideline to measure impact until INTOSAI’s
global survey and of 2020 entitled “All Hands-on Deck”, where it developed key
indicators of powerful audit impact and good government oversight. It then further
utilized to determine the capability and performance of SAI globally.
Their proposed focus on the oversight system and the role of SAI combined
are assumed to be the foundation of robust control capable of orchestrating greater
audit impact and effective government oversight. Notwithstanding, BPK as SAI of
Indonesia and a member of INTOSAI must adhere to this framework despite the
over-generalized global view on the relation of government oversight and audit
impact. Other challenges to also consider as SAI from a young developing country
are issues related to limited resources, traditional bureaucracy, political instability,
and cultural barriers that could hamper progress in optimizing their effort to
implement and fulfil the indicators required. In which, most of those challenges have
been minimally addressed in INTOSAI strategic plan (FAI, 2020; IDI, 2020).
Evaluation of these impact measures is needed to translate the normative
concept and jargon into practical objectives and quantifiable outputs of BPK as an
external auditing organization. to generate impact in the government oversight
system. A deeper understanding of the components of audit impact measures is
needed to improve BPK capacity for government oversight, helping decision-makers
identify and prioritize issues, plan strategies, mitigate risks, and build programs that
can translate to real changes. Hence, examining elements of audit impact which
7
employed and acknowledged as standard measurement globally is beneficial and

intriguing, as the knowledge gained can contribute to the development of research in
government auditing and enhance practical applications in the role of SAI within the
oversight system of Indonesia.
Ultimately, this research aims to assess the government oversight assessment
introduced by INTOSAI and IBP in relevance to the audit impact of Badan Pemeriksa
Keuangan as Indonesia’s Supreme Audit Institution. The research will first present
characteristics of effective government oversight as developed by INTOSAI and
identify indicators significant to the concept. From then on, the research will explore
the influence of government oversight on audit impact using literature related to
organization, evaluation, and performance measurement.
The relation of each indicator will be explored using the audit impact
framework developed by Johnsen et al., (2018) and Raudla et al, (2015). Further
analysis will utilize the aspect of performance measurement explored by Christensen
et al. (2007) and the evaluation approach by Stufflebeam & Coryn (2014) to explain
the multidimensional components of understanding audit contribution in the
government oversight system. Quantitative data will be collected from a survey
through snowball sampling to gather information on each indicator concerning
government oversight and audit impact of BPK. The data will then processed using
the multivariate regression analysis to examine how each of the indicators presented
contribute to government oversight and its overall influence on audit impact.
1.2 Problem Formulation
This research aims to investigate core elements of government oversight audit
impact and evaluate how these measures are able to contribute to the improvement
of audit impact BPK RI. Further research and evaluation of oversight mechanisms
are necessary to enhance efficiency and address limitations. Insights on prominent
indicators of audit impact will positively aid BPK in building constructive
assessments and strategies for high-quality audits, promoting accountability, and
achieving more reliable public fund management while also considering
circumstances surrounding the bureaucratic culture, state of managerial, and
organizational capacity, public and stakeholders’ expectations, and other challenges.
Thus, the research problems are composed as follows:
8
a. What are the driving factors of government oversight?

b. How does the government oversight contribute to SAI’s audit impact in
Indonesia?
1.3 Research Purposes and Significances

Based on the background and research questions, the aims of this research are
as follows:
a. To determine the driving factors of government oversight.
b. To analyze government oversight contributes to SAI’s audit impact in Indonesia.
Additionally, this research is expected to deliver significance both in an
academic and pragmatic sense below:
a. Academic significance
The current research on audit and audit impact mostly grounded on the
context of developed countries and there are limited studies on the development
of performance audits in developing countries. Considering the limited scientific
literature available for audit impact, especially in Indonesia, the Southeast Asia
region, and developing countries, this body of work could also serve as a point
of reference for future research, particularly for scholars, academics, and
practitioner specializing in audit, evaluation, organization performance, and
administrative sciences.
b. Practical significance
This research intends to provide a better understanding of elements and
indicators used as the basis of measurement of audit impact thus contributing to
add value and insights within the topic. The results of this research provide
helpful information for both the BPK and other government institutions on what
elements and indicators need to be looked upon in enhancing audit impact. Thus,
helping policymakers to understand and develop strategies necessary to
strengthen the impact of audit work and its contribution to government oversight
in Indonesia.
9
1.4 Writing Structure

This thesis is comprised of five chapters, each divided into several sub-
chapters containing the following discussion.
CHAPTER I INTRODUCTION
This chapter describes the background, problem

formulation, objectives, and significance of the research.
At the end of this chapter, there are descriptions of each
chapter's main points of discussion.
CHAPTER II LITERATURE REVIEW AND THEORETICAL

FRAMEWORK
This chapter discusses the relevant theories used as the

basis for research. Besides that, it will also describe the
results of previous research, research models, hypothesis
development, and concept operationalization.
CHAPTER III RESEARCH METHOD
This chapter contains research approaches, types of

research, data collection techniques, population and
sample, data analysis techniques, and research limitations.
CHAPTER IV DISCUSSION OF RESEARCH RESULTS
This chapter discusses the general description of the

research object, analysis of research variables, and
discussion of research results.
CHAPTER V CONCLUSIONS AND RECOMMENDATIONS
This chapter contains conclusions from research results that

answer the research questions and recommendations or
suggestions for various parties.
CHAPTER II
THEORETICAL REVIEW
2.1 Literature Review

This chapter provides a summary of prior research findings and explores the
relevance of performance audit and public accountability in the mapping process
variables presented as the focus of the study, namely, government oversight and audit
impact. The literature review functions as a thorough exploration of prior research
and scholarly publications on the topic addressed in this academic thesis. It sets the
background and theoretical foundation for the research while emphasizing the
justification and unique contribution into the study of public administration and
governance. Lies the groundwork on which this research constructs, leveraging
existing knowledge while seeking to expand it through new empirical inquiries. The
goal is to offer a comprehensive understanding of the existing knowledge, highlight
areas where the literature lacks coverage, establish the researcher's position on the
subject, and underscore the importance of the research presented in this study.
2.1.1 Previous Research Regarding Impact of Supreme Audit Institution
There has yet a study specifically addresses government oversight in
the context of public sector auditing. Considering the scarce research
assessing the relation between government oversight and audit impact, the
limited sources are focused on the available examination on the topic of audit
impact within scope of government external audit (which mostly centred on
performance audit) conducted by Supreme Audit Institution around the world
deemed relevant to this research as summarized in Table 2.1.
Table 2.1 Matrix of Previous Research on Impact of Supreme Audit Institution
Title Author Aim Year Variables Method Result
Supreme Audit Åge To measure 2019 SAI Legitimacy, Type of Multivariate Performance
Institutions in A Johnsen, performance audit performance audit, regression, audits have
High-Impact Kristin impact, possible and Administrative level, Actors descriptive positive impacts
Context: A Reichborn- common determinants furthering own interest, Auditee’s statistics, on public
Comparative Kjennerud, in the audit process existing plans, Audit rigidity, and the administration, as
Analysis of Thomas that influence the Audit process strain, Audit bivariate perceived by civil
Performance Carrington, impact across Nordic process communication, Audit correlation servants who have
Audit in Four Kim countries report quality, Media attention, coefficients experienced
Nordic Countries Klarskov Usefulness, Holding to account, performance
Jeppesen, Change, Improvement audits
Külli Taro,
Jarmo
Vakkur
10
11
Performance Kristin To study the impact of 2017 Performance auditing, public Descriptive Performance audit
Audit As A Reichborn- performance audits on administration, Perception of statistics reports can lead to
Contribution to Kjennerud, public administration performance, audit usefulness, positive
Change and Signy Irene and changes in resistance to audit reports substantial
Improvement in Vabo governance systems of changes for
Public organization auditees
Administration
The Impact of Yousef Ali To analyse Saudi 2017 Contribution of performance Multivariate SAI contributes
Performance Alwardat Supreme Audit audits, Relevance of auditors’ regression, positively to the
Audit on Public & Institution (SSAI) recommendations, Preventive descriptive management of
Administrations Abdullatif influence on effect on administrators, statistics public
in Saudi Arabia: Mohamed administrators in Performance audits influence on organizations
An Exploratory Basheikh improve management administrators’ management
Study of public resources. practices, organisations’ relations
with stakeholders.
Are Performance Lourdes To study the impact of 2016 Performance audit, audit Exploratory Not all SAI
Audits Useful? A Torres, Ana performance audits recommendation implementation, analysis, observed
Comparison of Yetano, and carried out by EU audit impact Cluster successfully
EU Practices Vicente Pina Supreme and Regional analysis, achieve impact
Audit Institutions Multidimen from their audit
thorough analysis of sional work.
the audit scaling,
recommendations
implementation by
auditee
Performance Kristin To analyze whether 2015 Auditee’s tendency to change, the Multivariate SAI’s
Audits and Reichborn- performance audits extent of changes and systems regression, performance
Supreme Audit Kjennerud, result in changes in improvement, leader held into descriptive audits lead to
Institutions’ Åge Johnsen public administration account, executives' responses, statistics, changes in
Impact on Public Auditees' agreement with the and Norwegian public
Administration: SAI’s assessments Number of bivariate administration
The Case of the audit reports correlation
Office of the coefficients
Auditor General
in Norway
The Impact of Ringa To analyze the impacts 2015 Auditees’ perception of the Bivariate Performance
Performance Raudla, of auditors, audit process, quality of correlation audits are useful,
Audit on Public Külli Taro, performance audit on audit report, legislative analysis though the role of
Sector Cherlin Agu, public sector involvement, media engagement legislature and
Organizations: James W. organizations media coverage of
The Case of Douglas audit a significant
Estonia factor for changes
The Performance Alexandra To demonstrate how 2015 Changes made by the Systemic Administration of
Impact of the Otetea, SAI report their government, financial savings, Literature state funds and
Supreme Audit Cristina performance and impact on Parliament, media, Review assets operates
Institutions on Maria Tia assess the financial independent impact assessment under a system
National Budgets. (Batusaru), impact of public audit where prioritizing
Great Britain and Mihai on national budgets in the efficiency of
Romania Case - Aristotel Great Britain and their utilization is
Comparative Ungureanu Romania not a primary
Study consideration
Evaluation of Agus To study the 2015 Stakeholder’s perception on Critical Indonesian SAI
performance Bambang implementation of performance auditing, Auditors Systemic need to expand
auditing in Irawan performance auditing perception on performance Approach, auditing focus
Indonesia: A and determine its auditing, accountability Evaluation from efficiency,
critical systemic impact on research efficacy, and
approach to accountability economical (3E)
addressing public in Indonesia, explore aspects to
accountability stakeholders’ and environmental
auditor’s view on SAI and sustainability
performance auditing aspects
Auditors General’s Danielle To analyse the impact 2014 Contribution of performance Multivariate Auditor general
Impact on Morin of performance audit audit, Preventive effect, Influence regression, exert influence on
on management practices, variance the organizations
12
Administrations: A in Auditor’s General Usefulness of auditor’s report, audited entities to

Pan-Canadian from 2001-2011 Concrete actions taken following a moderate extent
Study (2001- the performance audit,
2011) Consequences for the
organization.
Measuring The Danielle To examines the 2004 Relevance of recommendations, Descriptive Value-for-money
Impact of Value For Morin impact VfM have had Perceived utility of audit reports, statistic audits generally
Money Audits on the management of Actions taken by audited had a positive
(VfM): A Model organizations in the Managers, Changes occurred in influence on the
For Surveying government of Quebec the audited organization’s,, management of
Audited Managers Organizational and personal audited
consequences of the audits. organizations
2.1.2 Novelty of the Research

In the field of public administration, the question of utilization of social
science research and how the result of examination is genuinely put into
practice remains a complex issue to address (Weiss, 1979). A shared interest
in ascertaining whether social science research designed to influence policy
is prevalent between academic community and government as practitioners,
not excluding the subject of evaluation, performance management, oversight,
and audit.
However, as Hay & Cordery (2018) noted, examination on public
sector auditing has been severely limited. The ongoing discussion among
scholars and the academic community about audits revolves around whether
audits in practice enhance the efficiency and effectiveness of the public sector
or if they are simply ritualistic verifications (Lægreid, 2014; Lapsley, 2009;
Power, 1997, 2003). Differing views exist on the effectiveness of oversight
mechanisms on audit and its impact, while there is little agreement over its
nature and mechanisms. Some studies suggest that oversight may not always
lead to improved audit quality (Blann et al., 2022). Meanwhile, research on
the influence and value of audits on government has been carried out (Johnsen
et al., 2018; Hay & Cordery, 2018; Alwardat & Basheikh, 2017; Reichborn-
Kjennerud & Johnsen, 2015; Raudla et al., 2015; Irawan, 2015). A common
perception is that impacts are frequently presumed, anticipated, or pledged,
but rarely adequately substantiated by systematic investigations (Christensen
et al., 2007). Meanwhile, claims on audits influence on the effectiveness,
efficiency, and accountability of the public sector is varied due to the barely
sufficient empirical proof to support these assertions (Bovens et al. 2008).
13
In both theoretical understanding and practical application of audit

impact, the challenge lies in the fact that claims of a positive influence on
public administration and society as a result of audits are subject to debate
and the factors influencing them are predominantly unidentified (Johnsen et
al., 2018). For Supreme Audit Institutions (SAI), assessing the impact of their
activities poses a challenge, both in qualitative and quantitative terms. When
SAI fails to maintain legitimacy and support for its work, it may lose the
authority to oversee and facilitate reform and improvement in government.
Thus, SAI is consistently encouraged to develop new ways to demonstrate its
impact and ongoing relevance to the public (Cordery and Hay, 2018).
However, even at a smaller scale, measuring the impact of recommendations
from SAI and quantifying the value generated by SAI in public sector entities
proves to be a difficult task (Ototea, Batusaru, Ungureanu, 2015).
As the question arises about SAIs and their contribution to
governance, few evidence-based knowledge is available to back up these
claims (Reichborn-Kjennerud & Johnsen, 2015). This condition is
exacerbated seeing how previous studies on the impact, especially of
performance audits are mostly grounded on the perceptions of those audited
(Morin, 2014). The current discourse of audit impact today mainly focuses on
performance audit and mostly has been done in Western and European
countries (Reichborn-Kjennerud & Vabo, 2017; Raudla et al., 2016; Torres et
al., 2016; Van Loocke & Put, 2011).
Research on performance audits in Norway (Johnsen et al., 2018;
Reichborn-Kjennerud & Johnsen, 2015) has identified that SAI audit reports
are perceived as valuable when the auditee’s feedback is considered during
the process as well as the auditee’s perception of the quality of the audit report
and the legitimacy of SAI. Those elements consequently lead to concrete
improvement in public administration as high-quality reports that give
attention to policy areas, demonstrate tangible improvements in the systems
post-audit, and encourage best practices, are perceived as valuable to the
auditee (Reichborn-Kjennerud, 2013).
14
Meanwhile, research on Value-for-Money (VfM) audit in Canada by

Morin (2005) portrays the audited manager response on audit report, the study
found that parliamentarians exerted significant influence in compelling
government managers to adhere to auditors' recommendations, with the
presence of both parliamentarians and press media coverage leading to more
effective action than if auditors depend solely on their limited scope of
authority.
On the other hand, further study on of Auditors-General (AG) impact
in Canada (Morin, 2014) shows AG works demonstrate a moderate impact,
as the figures indicate that auditors do influence the audited organizations,
although their attempts to exert influence often fail, due to the indirect powers
granted to AG by their institutional status, which are influenced by both the
political authorities and the capacity of parliamentarians to control the
executive branch, and one possible reason for this limited influence is that the
administrations' respect for SAIs obscures the true value perceived by
auditees in the work of auditors.
A comparison study by Torres et al. (2016) examines impact of
performance audits in European Union (EU) countries has varied result
whereas countries experiencing limited impact of performance audits with
issues attributed to the infrequent adoption of the recommendations and
follow-up procedures, the absence of substantial personnel, and the lack of
any adverse effects resulting from not implementing their recommendations.
The parameter of impact utilized in Torres et al., (2016) is mainly based on
the extent of recommendation follow-up and action taken by auditee.
However, Morin (2001) advises that evaluating the success of an audit based
solely on monitoring recommendations is an oversimplified approach, as
recommendations that do not target the root cause of the problem may result
in minimal changes to the system (Budiman & Amyar, 2021).
In Indonesia context, Irawan (2015) explores implementation of
performance auditing conducted by Badan Pemeriksa Keuangan (BPK) while
determine its impact on accountability through exploratory and evaluation
approach. The study shows that auditing brings improvement in public
15
accountability, however, the Parliament little attention on audit

recommendation and follow-up reports remains a concern fostering the need
for better collaborative mechanism among BPK, Parliament, the audited
organisations, media and citizen for better improvement.
While these studies contribute to understanding elements of audit
impact, they mostly developed variables through exploratory research and not
derived nor aligned with INTOSAI standards. Although some of the variables
are closely related and overlaps with indicators used in INTOSAI and OBP,
they are partially incorporated and most of other factors are not attributed.
This is unsurprising considering the research was done years before both of
those internationally acclaimed organizations put forward their indicators and
assessments in 2020.
As Power (2000) argues that auditing as an agent of change in
organization has not been assessed thoroughly. It remains unclear whether
measures introduced for auditing does have an effect on forcing changes
within the organization. Hemici & Kontogeorga (2020) also noted several
challenges in building indicators for SAI as data in the public sector are
mostly qualitative, difficulty in measuring long-term impacts, and limited
dimensions of measurement available in the model. Indicators should be
highly perceptive, dynamic, and adaptable to changes in public expectation,
while maintaining the accuracy and ability to define the issue they aim to
measure should be evaluated, modified or refreshed as necessary.
Upon a comparative study of performance management
arrangements for civil servants and the linkages between individual
performance and public sector effectiveness conducted by The Organisation
for Economic Co-operation and Development (OECD), Ketelaar et al. (2007)
highlights the importance of setting a clear objective in order to establish
appropriate measurement of performance. On top of that, SAI also need to
address the ongoing concern of delivering benefits to the people by staying
relevant to public needs and expectations. It is a prime example of how the
role of auditing is dynamic in the sense that it’s constantly changing to
accommodate increasing stakeholder expectations.
16
While an increasing number of social scientists are expressing

apprehension about ensuring the applicability of their research for
stakeholders and government, the government in return also exhibiting
publicized distress regarding the efficacy of the government-funded social
science research. Therefore, study related to auditing, accountability, and
performance measures especially in identifying indicators of SAI’s impact
especially in Indonesia is much needed. An in-depth evaluation of audit
impact measures will help BPK as Indonesian SAI building a holistic
approach to measure the value of their work and identify room for
improvement. Evaluation of these measures lays the groundwork for the
development of BPK as an external auditing organization to generate more
impact in the government oversight system.
2.2 Theoretical Framework
The economic downturn and heightened global competition for the past
decades have stimulated governments to embrace public administration reforms in
many countries (Nicoll, 2017; Curristine & Flynn, 2013). These developments have
given rise to a greater need for evaluation and auditing, as advocated by the principles
of New Public Management (NPM). The NPM paradigm in the field of public
administration advocates the production of vast library of discourse, including the
body of knowledge of evaluation and performance assessment. NPM have prompted
accountability and transparency to extend beyond financial and legal facets
(Andrianto et al., 2021). NPM also shifted stakeholders' attention towards matters
related to public resource management and the efficiency and effectiveness of public
sector performance (Leeuw, 1996)). Waves of performance trends began in 1980s,
marked by a state of financial limitation and a dedication to the reorganization and
financial improvement of public sector establishments resulted in a greater emphasis
on evaluating performance within the public sector in line with the development of
administrative reform (Power, 2000).
In response, countries are allocating more resources to the measurement and
evaluation of performance in the public sector. Pressures on governments have led to
reforms in both governance and public management, as they sought to meet these
demands (Aucoin & Heintzman, 2000). Report by OECD (2005) on reform trends in
17
public sector noted a variety of transformation within government, such as

reorganizing government structure, changing service provision, decentralizing
government responsibilities, implementing market-based competition, more efficient
services for citizens. Administrative reform in state management is accomplished by
adopting principles and techniques of private enterprises where evaluations resulted
in the development policies with the goal of improving the effectiveness and
efficiency attached (van Thiel, 2002). Over the years, concerns on government’s
performance have gradually become the norm and has brought more attention for
administrators in public sector organizations to exercise evidence-based policies and
evaluation.
2.2.1. Evaluation
Evaluation possesses the ability to influence and improve public
services (Anderson, 2021). An evaluation culture, according to Mackay
(2006) is the potential to enhance the effectiveness of a government around
the world starting to adopt a results-centric approach by establishing the
governmental monitoring and evaluation (M&E) frameworks. In public
sector domain, evaluation serves an integral function of improvement.
Evaluation generates information regarding the value or usefulness of policy
outcomes and their impact on performances (Dunn, 2015; Perrin 2006), thus
providing insights to organizations on the impact of their work along with the
elements of intervention necessary to lead organization towards favorable
results (Perrin, 2006).
Similarly, van der Meer (1999) believes that evaluation by provides
relevant information to decision maker, thus having the capacity to stimulate
adjustments in policy development, administration, management, procedures,
implementation strategies while promoting the organizational learning.
Evaluation also presents collaborative opportunity for different levels of
stakeholders to work together and assist them in developing the mechanisms
necessary to generate change within organization (Azzam & Whyte, 2018).
Perrin (2006) further argues that evaluation encompassed the question of
“what” works, “why”, and “how” as it assesses the continued relevance and
suitability of organization’s strategies and programs, obtaining information
18
on both intended and unintended consequences, identify adequate objectives

and indicators used for monitoring.
More importantly, evaluation help determining causal relation
between the initiated programs with the observed outcomes. Thus, answering
whether organization changes can be attributed to an intervention and if an
intervention is responsible for said changes. Evaluation studies have
developed ways to address the main issue of causality (Nielsen and Ejler,
2008) by providing representation of cause-and-effect in program, project, or
initiatives using the open system and logic model (Davies, 1999). However,
it is important to note that the relationship of causality cannot be treated as a
given pathway or as a phenomenon with 100% chance of occurrence.
The theoretical and practical aspects of evaluation have expanded into
the discourse of utilization and impact on agency, decision-makers, and other
stakeholders through various approaches. However, it's rather unrealistic and
a general misconception to think that the application of evaluation tools such
as logical modelling or open system approach alone would be able to
completely resolve the methodological difficulties of outcome attribution,
especially in the typical performance management context where numerous
factors and circumstances should be taken into account.
Meanwhile, as the discourse of assessment and the utilization of
evaluation methods expands, discussion on categories and functions of
evaluation also continue to develop. An approach by Vedung (2004) extends
these categories into six types of evaluation, namely: conceptual use,
instrumental use, process use, tactical use, ritual use, and legitimization use.
a. Conceptual use involves learning that changes in state of mind, eventually
leads to awareness on the relevance, objectives, and evaluation.
b. Instrumental use relates to the application of evaluation findings directly
to decision-making and policy within organization.
c. Process use focuses on implementation of evaluation, placing importance
on how it carried out as part of organizational process rather than results.
19
d. Tactical use implies function of evaluation to seek assurance that control

is maintained within organization, targets are being met, and programs are
responsibly managed.
e. Ritual use relates to evaluations performed without a clear purpose.
f. Legitimization use refers to the use of evaluation to justify and defend
existing views, interests, or policies, rather than providing solutions of
seeking alternatives to unresolved questions.
Anderson (2021) further reorganize the categories of evaluation into:
1) Instrumental evaluation where utilization of findings aids decision-makers
in modifying the object of evaluation; 2) the utilization of the process, or the
utilization that arises when modifications emerge from involvement and
acquisition during the assessment procedure, as opposed to the discoveries;
3) evaluation as an idea or enlightenment where it is utilized to inform public
policies and their outcomes; and 4) as conceptual use of evaluation where the
result of evaluation helps build the foundation of shared understanding on
problems, solutions, and the effects of public policy among various actors.
In examining the effect and utilization of evaluation surrounding
public administration, one can’t divert from addressing the issue of changes
in the structure or conduct of government, procedures, strategies, to policies
and its implementation. These various theoretical orientations and
methodologies have dedicated significant focus towards the essence of
evaluation impact, its influences, the responsibilities of the evaluator as a
catalyst for change, mentor, facilitator, consultant or advocate, and the
utilization of evaluation (Coryn, 2006). However, Bastoe (2006) noted that
many governments encounter restricted, unreceptive, and unresponsive
situation where the flow of the evaluation findings into decision-making
process does not automatically occur.
Studies into evaluation uncovers several factors attributed to the
utilization, one of them conducted van der Meer (1999) on report from Dutch
Court of Audit over procurement project of Royal Dutch Navy, in which the
paper concludes that evaluation has hardly significant impact other than
emphasizing debates and options already exist in the organization. Despite
20
demands on prompt information of progress and outcomes, evaluation reports

are also often perceived as being lengthy and untimely by public managers
Nielsen & Ejler (2008). These conditions further evoke question whether
evaluation has the capability to generate as big of an impact as it claimed.
In an attempt mitigate risks mentioned above, evaluation studies have
developed ways to address the main issue of causality (Nielsen and Ejler,
2008) by providing representation of cause-and-effect in program, project, or
initiatives using the open system and logic model (Davies, 1999). Indeed,
relationship of causality cannot be treated as a given pathway or as a
phenomenon with 100% chance of occurrence regardless of circumstances.
However, it's unrealistic and rather dangerous misconception to think that the
application of logical modelling or open system approach alone would be able
to completely resolve the methodological difficulties of outcome attribution,
especially in the typical performance management implementation context.
One of the renowned concepts to address this issue is the utilization-
focused evaluation (UFE) developed by Pattons (2000), where he emphasizes
the importance of developing useful and relevant evaluation to its users and
stakeholders. UFE emerges as a response to the constraints encountered in
conventional evaluation methods, which frequently prioritize the evaluation
of program efficacy rather than promoting improvement in the organization.
Several key concepts and principles that guide the evaluation in UFE are:
a. Stakeholder Involvement highlights the significance of stakeholder
participation throughout the evaluation process. It includes their
involvement in the planning, design, implementation, interpretation, and
dissemination of evaluation results.
b. Relevance and Usefulness emphasizes creating evaluations that are
beneficial for stakeholders. Evaluations should cater to the specific needs
of stakeholders and address their queries and worries. The primary focus
is to utilize evaluation results for informed decision-making about
program enhancement, rather than merely assessing program
performance.
21
c. Contextual Consideration acknowledges the vital role of context in

evaluation. Evaluations need to the unique environment of the
organization being evaluated considering the wider social, cultural,
values, beliefs, expectations and political aspects surrounding the
organization.
d. Continuous Improvement promotes constant improvement in evaluation
as it should be flexible and adaptable following the changing
circumstances, new data, and emerging priorities.
e. Credibility and Rigor of UFE represents concern on the crucial aspect
of professional evaluation process. Evaluations must be grounded in
sound research principles and methods and adhere to established
methods to ensure credible and reliable findings.
f. Capacity Building: UFE underlines the importance of developing
evaluation capacity among stakeholders. Evaluations should aim to
enhance user capacity and foster improvement within organization.
This approach has invoked compliance in users as by meeting their
need, thus evokes the feeling of ownership on the evaluation itself as study
on the use of evaluation (Patton, 2008) found that evaluation findings and
recommendations are more likely to implemented by the intended users if
they understand and was engaged in the process of evaluation. The role of the
evaluator is not to independently make decisions without considering its
users, but rather to enable and support decision-making processes of officials
who will utilize the evaluation's findings. Therefore, instead of being as a
distant examiner, Patton (2003) believes that evaluator should act as a
facilitator collaborating with its users and stakeholders in designing
evaluation model with careful consideration of how it will affect the
organization to ensure the findings are relevant, plausible, and actionable.
Similarly, Perrin (2006) stated that an outcome-oriented approach will
be more relevant and useful when there is enough flexibility to develop a
meaningful approach for each of the program’s unique context and when
staffs are actively involved in the development. This ensures evaluation’s
relevance and involvement of users and stakeholders to gain further insight
22
and optimize their organization based on data provided by the evaluation.

However, in lieu of its flexibility and context-sensitive approach, UFE does
not specifically endorsed any particular evaluation content, model, method,
theory, rather it advocates the process of selecting the most appropriate
framework for the situation.
Another perspective by Davies (1999) proposed a complementary
view between evaluation and performance measurement as evaluation tools
are useful in identifying appropriate indicators for performance measurement,
meanwhile the performance data can then be used for development of
evaluation and said indicators in the organization. This complementary
relationship investigated and distinguished by Rist (2006) into three
following relation types, as follows:
a. Sequential complementarity refers to the reciprocal relationship between
performance monitoring and evaluation studies, where the former can
prompt inquiries for the latter, while evaluation studies will generate
knowledge on organization which require continuous monitoring and
performance monitoring.
b. Information complementarity, where both monitoring and evaluation can
utilize the same data sources, thus avoiding the need for parallel streams
of information and supporting the interdependence sequential
relationship mentioned before.
c. Organizational complementarity emphasizes the importance of sharing
monitoring and evaluation information within an organization, often
achieved through a single administrative unit rather than multiple
discrete units.
The fourth category namely methodological complementarity was
introduced by Nielsen and Ejler (2008) describing evaluation and
performance measurement as knowledge production sharing similar
processes and methodology. Methodological complementarity suggests that
evaluative knowledge can be applied at every stage of the policy cycle.
Nielsen and Ejler (2008) draws upon the comprehensive guide of European
Commission’s resources for evaluating socioeconomic development to
23
further illustrate the comparison between evaluation and performance

measurement, where the phases of evaluation studies is attributed into 5
(five) categories, including: structuring and planning, obtaining data,
analyzing data, evaluative judgment, and utilization as shown in Table 2.2
below.
Table 2.2 Comparison of Evaluation and Performance Measurement
Performance
No Phase Item Evaluation
Measurement
Purpose Negotiated upfront Evolves over time
Scope Issue specific Broader focus
Budget Separate budget item Integral to programme
Structuring Frequency Episodic Ongoing
1. and Timing During or after programme All through programme
planning Units of Customized quantitative Quantitative indicators
measurement and qualitative indicators reproduced over time
Input, output, outcome,
Type Input, output, outcome
impact
Data production One time Routinized processes
Obtaining Desk research, interviews,
2. Surveys, information
data Tools surveys, information
systems
systems
Data triangulation of Balanced on few or some
Means
multiple sources sources
Contribution analyses, time
Contribution analyses, time
Tools series, regression analyses,
3. Analysing series
experimental designs
Attribution of the outcomes
Attribution to the programme is often a Attribution is assumed
key aim
Benchmarking,cost Predominantly
effectiveness, efficiency, benchmarking, cost-
Tools
multi criteria analyses, effectiveness, cost-
expert panel, etc efficiency
Evaluative
4. Performance standard Descriptive or prescriptive Descriptive
judgement
External or internal Internal programme
Assessor
programme evaluators managers
Tabular reporting with short
Format Evaluation report
explanatory text
Organizational
Low to medium High
learning
Budgeting cycle
Occasionally Mostly
Users
Tactical decision- All levels of programme
5. Utilization Few
making application organization
Strategic decision-
Low High, real-time data
making application
Organizational
High Low to medium
learning
Evaluation and performance measures address the question of how

successful a program, project, initiative, or to an extent an organization.
Along the development of these two studies, scholars shares agreement on the
complementary roles of evaluation and performance management (Davies,
24
1999; McDavid and Hawthorn, 2006; Julnes, 2006; Mayne, 2007; Rist, 2006),
the potential benefits of combining knowledge produced by the two draws
further realized (ZapicoGoñi and Mayne, 1997). Understanding on how
evaluation and performance measurement are mutually supportive elements
emerged as the presence of one is contingent upon the efficient
implementation of the other. De Lancer Junes (2006) argues that the results
of performance measurement systems are deemed insufficient in guiding
government resource allocation decisions, and therefore, comprehensive
results can be better achieved when program evaluators and performance
measurement practitioners work together.
Similarly, Nielsen and Ejler (2008) also found that despite
methodological differences of evaluation and performance measurements,
both scores interconnecting relationship when applied simultaneously
whereas evaluation offers fundamental insights and key methods in
enhancing performance measurement. This differing yet interdependent
relation between evaluation and performance when utilized strategically,
sustains their integrated role in all phases of policy cycle of planning,
implementation, evaluation, and feedback on performance management.
Not only that evaluation can be applied to all stages of policy cycle.
the various tools of evaluation (ranging from reviews to theory of change)
may be used as integral to performance management to enrich its execution
(Nielsen and Ejler, 2008). This relation can be explored using United Nations
Development Agency’s managerial model of performance management with
four distinguishing features (UNDP, 2001), such as; 1) Clear identification of
strategic goals providing a focus for action, 2) Communicating specific
results and outcome expected in designing strategic goals and aligning
programmes and resources accordingly, 3) Continuous monitoring and
assessment of performance data as lessons learnt need to be adapted in future
planning, and 4) Fostering improvement in accountability through ongoing
feedback from performance data. Based on key components above, Nielsen
and Ejler (2008) further analysed the utilization of evaluation into the phase
of performance management are as follows:
25
a. Setting strategic goals

In order to render evaluation an effective instrument for enhancing
public interventions, it is imperative that the generation of evaluative
knowledge be intricately interconnected with the mission and strategic
objectives of the organization. Performance management centres around
strategic goals, serving as the focal point to guide actions and procedures.
This infers that the evaluation layout susceptible to the necessity of the
decision-makers and the objectives that had been set in the organization.
b. Specifying expected results and strategic alignment
Evaluation serves as a powerful tool in identifying appropriate
measures for input, output, outcomes, as well as their derivatives and
ultimately ensures that specific objectives can be measured and evaluated.
Therefore, a significant challenge for evaluation is to shift focus to the
early stages of policymaking and actively contribute to the policy design
phase by providing an established framework.
c. Ongoing monitoring and assessment
Performance monitoring and assessment should assist
stakeholders, and public managers on policy making, however, they often
find evaluation to be time consuming. As the data generated from the
lengthy process also have less relevance and minimal impact on public
service. If the integration of evaluation in the policy cycle is to be
achieved, one of the main difficulties that needs to be addressed is the
establishment of a monitoring and evaluation system that retains
meaningful, accurate, and relevant data in a timely manner remains a vital
for the utilization of evaluation in performance management.
d. Improved accountability
A prerequisite for enhancing accountability is that management
effectively communicates the criteria and standards by which the various
agents are expected to perform. As previously mentioned, it is crucial to
analyze performance data that is fed back into the program organization in
order to identify the contextual evidence of successful or unsuccessful
performance.
26
Nevertheless, key differences exist between evaluation and

performance measurement specifically in their approach on impact and
attribution. Nielsen and Ejler (2008) remarked that performance
measurements seldom include impact indicators and instead often divert it to
outcome or result-based measures basically aiming for improvement.
Meanwhile, evaluation studies are designed to prove the existence of a causal
relationship between an intervention and its intended or unintended effect,
providing the evidence to answer the attribution question. Therefore,
determining impact and attribution remain the main concern in evaluation
studies (McDavid and Hawthorn, 2006; De Lancer Junes, 2006; Rist, 2006)
while assessment of impact is considered as vital indicators of the
effectiveness and sustainability of the program for evaluators.
The logic of evaluation has shifted to that of auditing (Power, 1997)
in which the desire for accountability endorsed by NPM as the transition of
evaluative cultures from a social scientific perspective towards a managerial
knowledge basis. Meanwhile, the widely adopted administrative reforms of
NPM are meant to shift away stakeholders’ focus from processes towards the
examination of activities, products, services, and societal impact.
Organization would then need to identify the focus of the evaluation from the
very beginning, whether it will give attention to process and treat it as equally
important as results (Davies, 1999) or to focus less on factors related to the
ex-ante (before the event) and more on issues on the ex-post (after the event)
(Sendjaja et al., 2015).
In the context of auditing, Power (2000) noted one of the factors that
urged the role of evaluation, monitoring, and auditing in public sector is the
development of quality assurance. Quality assurance methodologies, emerged
from industrial manufacturing field, through auditing serves a dual purpose
in the structure as the dedicated process of monitoring, reporting, and the
validating the organization’s system performance in achieving objectives as
shown in Figure 2.1 below.
27
Figure 2.1 General Model of Self-Auditing and Control

Source: Power (2020)
On the other hand, evaluators also must avoid opting for the easiest
way to check the mere administration aspect and law compliance of the
organization instead of studying the effect of policy, program, or regulation
on society. Evaluation in the public sector and government, in particular, need
to move beyond procedural concerns to a culture that value immediate and
long-term results. Ahlenius (2000) also noted that in an evaluation scenario,
evaluator has the option to question objectives as a part of the study while in
an audit objective are not open to scrutiny. This limitation may happen due to
objectives being set by political entities, along with the demand of external
transparency on audit which also relates to the benefit of political decision-
makers, the media, and the public. Ultimately, evaluation discipline maintains
position as the integral component of organizational improvement and
management of public intervention highly relevant to stakeholders and policy
makers.
2.2.2. Performance Measurement
As the pressure to adopt NPM arises, practices in private sector
valuing efficiency, accountability, productivity, and professionalism slowly
translated into the need for achieving the same values in public service.
Professional public service can only be attained when its performance is
assessed by professional standard (Aucoin & Heintzman, 2000). While NPM
urges productivity and reformed in public sector, it has gained attention
towards performance or often referred as result-based management (Perrin,
2006), a framework assists managers in comprehending the conversion of
intentions and objectives into actions and outcomes, shifting attention from
input and output (Bastoe, 2006).
28
Governments face growing demands to deliver and prove results, and

the significance of focusing on outcomes for efficient public management is
widely acknowledged. Performance measurement systems are portrayed as
notable reactions to current public expectations for government transparency
regarding social expenditure and social initiatives (Greene, 1999). However,
implementing a result or outcome-based approach has proven to be
unexpectedly challenging (Perrin, 2006).
There has been a shift towards a stronger focus on outcomes, but the
measurement and management of expected outcomes have not followed suit
(Parker et al., 2019). To serve the fundamental aspect of effective governance,
government officials and stakeholders in the public sectors are expected to
deliver results (Perrin, 2006) and display continuous efforts in improving
their performance on the management of public funds (Davies, 1999). The
necessity to measure output, outcomes, and evaluation activities remains an
important step in improving government’s performance (van Thiel, 2002).
The concept of performance, to an extent, is associated either with
how well the actions or the outcomes derived from it (Hemici & Kontogeorga,
2020). According to Wholey (2007), performance is a subjective, intangible,
and socially constructed concept that is not easily quantified or assessed needs
to be captured within the process of inputs, outputs, and outcomes of the
program actual level compared with the projected level set by key
stakeholders. Thus, management of performance, interpreted broadly, refers
to the systematic procedure employing various evaluative methods and
measurements through which data related to organization performance is
gathered and utilized to leverage improvement (Davies, 1999).
Encompassed by the evaluation process of organization, performance
incorporates monitoring on objective indicators on a regular basis (Poister,
2003) while measuring and documenting their progress toward achieving
predetermined goals (Curristine & Flynn, 2013). The main components of
performance measures as highlighted by De Lancer Junes (2006) constitutes
of 5 (five) elements, including:
29
a. Inputs, which denote the resources employed in producing outputs, such

as costs, staff, time, materials, and other resources.
b. Outputs, refer to the final products or services delivered, with the
expectation that it will ultimately lead to desired outcomes.
c. Outcomes, which are the consequences of outputs, are often more intricate
to assess and can be divided into two phases. Intermediate outcomes, for
instance, are outcomes that are projected to result in a desired end but are
not an end in themselves. Following the previously mentioned example,
an intermediate outcome would be teachers actively enhancing the
curriculum utilized in their classrooms subsequent to completing the
curriculum improvement workshop. Lastly, end outcomes signify the
ultimate result sought, such as the number of students attaining mastery
in a subject.
d. Efficiency measures, also known as unit-cost ratios, indicate the
proportion of output to input or outcome to input.
e. Explanatory information providing additional context to help readers in
interpreting and understanding the data.
However, despite the progressive work surrounding performance
measurement and result-based framework, there exist challenges in
formulating a coherent sequence of intentions and purposes. This has been
observed from a study of twelve countries by the World Bank and
International Business Machines (IBM) Centre for the Business of
Government on the shift from an output to an outcome-oriented public
management concludes that an outcome approach is essential and generally
acknowledged, however the implementation of a results-focused strategy has
proven to be unexpectedly challenging. Perrin (2006) found that the diverse
and unique political settings, cultural environment, social context, historical
background, and other relevant factors pose a challenge in formulating a
universally applicable model.
Similarly, Lægreid (2005) also believes that measuring performance
retains a degree of political complexity, and the pursuit to find a singular
optimal measure of performance might as well be a futile attempt. It becomes
30
evident that a standardized solution is almost not feasible in which there is no

“one-size fits all” and a tailored approach is indispensable for each specific
country. As Perrin (2006) further noted that implementing an outcome focus
in the public sector signifies a significant change in thinking, acting, and
managing, shifting from process-oriented tasks to prioritizing benefits. He
further explores the significance of connecting outcomes to organization’s
strategy and argues a number of benefits from outcome focused approach as
follows:
a. Shifting towards an outcome approach is crucial and valuable, as it is
fundamental for the efficient management of government and the
provision of public services, supported by substantial evidence from
global experiences.
b. An outcome approach set by government fundamentally address issues
deemed significant by citizens essential such as poverty reduction, job
opportunities, social welfare, healthcare, education, or democracy.
However, in a democratic state, policies made by leaders, stakeholders,
and top managers are not isolated from political will and circumstances.
Thus, the success of an outcome-oriented approach and the focus on
meaningful benefits cannot be attained without alignment with
government's political priorities.
c. A focus on outcomes has the potential to create a framework for a
comprehensive government approach that coordinates various policy and
program areas, all of which are expected to contribute to outcome that
generally benefit citizen.
d. Political leadership employed an emphasis on results as a strategy to
showcase their efforts in meeting the requirements and apprehensions of
their populace. In this manner, it can serve as a valuable instrument to
portray trust of political establishments.
Ultimately, executing major reforms focusing on outcome, results, and
improvement in performance within organizations serves minimal benefits
when the data produced it is not utilized properly. Therefore, Perrin (1998)
observed key areas to ensure effective utilization of performance
31
measurement. First, adequate resources are imperative facilitate the

development of sufficient performance indicators as sufficient amount of time
must be allocated to the process of developing, testing, refining, revising, and
updating measures. However, in practical scenarios these may not be
attainable as political situation within the organization tend to impose
performance measures in a top-down system approach, where accountability
for results is demanded within unrealistically short time frames, disregarding
the need for adequate resources.
Additionally, active involvement of stakeholders in the formulation
and implementation of measures is crucial. Perrin (1998) argues that
performance indicators often perceive programs as an independent activity,
overlooking the characteristic of interconnectedness in bureaucratic
government institution where policy is rarely operated in isolation.
Ultimately, without widespread support throughout the system, especially at
top and intermediate-management levels, there is a risk of an outcome focus
being reduced to a mere administrative exercise rather than a true shift in
management. To resolve this issue, Perrin (2006) noted the necessary the top-
down and bottom-up synergy in an outcome approach, as strong support from
higher authorities is crucial to establish legitimacy and priority for outcome-
oriented actions.
However, challenges presented in applying outcome approach also
require regular monitoring, review, evaluation, and revision from government
to identify issues and improve the approach as necessary (Perrin, 2006). The
ability to document events is crucial for achieving desired outcomes, and
without accurate information, it is impossible to focus on results, therefore
monitoring progress, evaluating broader aspects, and tracking indicators are
necessary. Along with pre-testing indicators, utilizing reviewed and updated
indicators, employing independent data collection methods, engaging
stakeholders for validation, obtaining independent peer review differentiating
between political goals and practical results, are some of available methods
that can be done to ensure identification of adequate indicators or
performance.
32
Similarly, Davies (1999) also believes the significance of designing

performance measures based on its utilization by introducing the initial steps
of performance management, which covers:
a. Formulating definition of performance to be used in the evaluation,
including specific outputs, outcomes, effects, impacts expected.
b. Setting up performance targets in measurable unit (i.e. service standards,
quality check and assurance, projects executed).
c. Determining indicators as standard of intended results (performance
measurement, performance monitoring, impact evaluation, etc)
d. Measures account for the achieved results in relation to the resources
utilized (performance report, value-for-money audit, etc.)
e. Generate strategic decision based on information of performance
evaluation gathered (resource allocation, evidence-based budgeting,
organization trajectory, etc)
Another perspective by Ketelaar et al. (2007) sees that enhancing the
efficacy of public administration, to a large extent, is closely associated with
public servants’ active contribution towards their organizations' objectives.
Thus, public servant’s individual performance needs to be taken into
consideration when establishing sufficient evaluation indicator. On the other
hand, Christensen et al., (2007, p. 155) argues that in the conventional
bureaucratic organizational structure characterized by rigid hierarchy and
standardized procedures, a civil servant duty is to adhere correct protocols.
Thus, government’s opportunity to perform is limited due many
circumstances, political interest, leadership, resources, personnel’s
competence, are few factors to consider. Therefore, performance
measurement also needs to provide a clear distinction between an
organization's ability to identify the issue and its capacity to actually take
action to resolve it.
While evaluation supplies the structure necessary for the development
of sound performance monitoring indicators, the ongoing collection of
performance data can be employed for evaluation objectives. As Poister
(2003) further remarked that when performance measures are executed
33
properly, it will be an indispensable instrument for managers to track

progress, ensure accountability, guide them in reaching desired goals,
improve efficiency, evaluate alternatives, showcase their achievement to
stakeholders, and supply them with necessary information for decision
making. Performance measures has the potential to influence and re-orient
the administrative culture within a result-oriented organization.
However, there exist limitations on the use of performance indicators
as Perrin (1998) distinguished them in eight key points, which are:
a. Varying interpretations of terms and concepts
Misconception on terms and concepts introduced by performance
measures indicators lead to a lack of comparability across different sites
and staff, making it difficult to measure program performance accurately.
Inconsistent recording and reporting of measures may result in
misleading aggregated statistics and meaningless outcomes Bottom-up
approaches that allow staff to understand the purpose of the information
collection will result in more accurate, consistent and meaningful
information than the more common top-down approach.
b. Goal displacement
It occurs when the focus of performance measures shifts towards
prioritizing indicators as the sole objective. This leads to a misguided
emphasis on activities that are not aligned with the actual objectives, thus
promoting practices such as cherry-picking and other methods of
achieving numerical targets without actually improvement.
Consequently, the true direction of programs is often distorted as goal
displacement diverts attention from the intended purpose of the program
rather than aligning with it, especially when consequences of the
program's performance are tied to its ability to meet numerical targets.
c. Inconsistent recording and reporting of measures
Public service domain is complex by nature and cannot be easily reduced
to a numerical figure. On the other hand, performance measures attempt
to simplify evaluation of this multidimensional concept into a number of
indicators. However, indicators can be irrelevant, even if they are
34
accurate. When a limited number of indicators are used to represent the

overall results of a complex and intangible social program such as child
welfare, economic development, or health promotion. Using incorrect
measures and indicators can risk to misleading statistics and meaningless
results rather than informative reality and accurate portrayal of result.
d. Prioritizing cost savings or cost shifting
Arguably, performance measures may confuse efficiency and cost-saving
initiatives with cost shifting, where problems are transferred elsewhere
rather than addressing them.
e. Misleading aggregate indicators concealing critical disparities
Performance indicators often hide gap and differences among
subdivisions and departments. Due to performance being measured in
numbers, totalled, and counted as average, thus drawing conclusion
based on an aggregate. This conclusion may depict an optimistic trend in
overall performance but missing smaller issues exist in the subgroups.
f. Limitations of objective-based approaches
It is challenging to measure adequately set objectives and the actual
fairness of evaluating results achieved. As programs with ambitious
objectives may be unfairly penalized, while mediocre programs are easily
reach their goals. Objective-based evaluation does not account for
unintended effects and consequences, which can be significant. While
objectives and plans are fixed, while the environment and program
activities change constantly, especially in public sector. Thus, when a
program adapts to situation at hand, its objectives and targets might
become obsolete. Ironically, a practical and effective initiative may risk
being penalized by performance measures for responding to changes
accordingly.
g. Usefulness to decision-making and resource allocation processes
A common example of incorrect use of performance indicators is
assuming causality relation, implying that the identified outcomes are
directly caused by program activities. Performance indicators lack
actionable implications unless additional methods are employed to
35
investigate the causes for outcomes and the potential for future effects.
Relying solely on performance indicators to make decisions regarding
program future can lead to inappropriate actions.
h. Less focus on outcome
Measurement based solely on quantitative data may not always result in
empowerment, improvement, and innovation. On contrary, it bears the
risk of decreased focus on outcomes, critical reflection, and the
exploration of new approaches, ultimately leading to a defensive mindset
and a reluctance to acknowledge the need for improvement. Furthermore,
inappropriate use of measures for accountability can lead to cynicism,
resistance, and manipulation of data to portray a positive image.
Another important aspect of performance measurement is its
usefulness for the organization. Aucoin and Heintzman (2000) believes that
performance measurement should not be limited on assurance and
compliance as assessment tools need to go beyond checking adherence on
regulations. Ammons (2012) argues noted that for performance data to be
used for decision-making, government officials should first collect
meaningful measures. In support of this view, a study by Rivenbark et al.
(2019) on local government in Italia found that government officials are more
inclined to engage in utilization of performance data when input, output, and
outcome measures, are reliable.
Poister (2003) further noted that the true value of performance
measures indicators is tested in their ability to consistently support progress
and bring improvement to the organization. Hence, to be deemed useful,
measurement system needs to be tailored to meet the requirements of specific
management process it aims to assist. Similarly, Davies (1999) presented
several issues need to be anticipated in regard of performance measures and
the inherent risks it possesses which can defeat the purpose of enhancing
productivity, such as:
a. Setting up inadequate indicators that fail to measure the intended variable
of performance. Indicators should base on proper consideration and not
chosen in haste or under-pressure of certain interest.
36
b. Assuming direct causality of performance measure to intended outcomes.

Though performance evaluation provides insights necessary on
improvements needs, claims on relationship among the indicators
measured, the executed day-to-day operation, performance and/or quality
of the program, and the targeted goals need to be supported by valid and
sufficient evidence.
c. Misuse of information resulted from performance evaluation by senior
management who use the data by tailoring them to shift the organization
towards their personal interest and away from the initial objectives.
d. Performance measures overrides organization’s original purpose as it
become the priority and end-goal instead of a helpful device. Creates
pressure on managers to work only in accordance with the indicators
established and not more. Resulting in an environment of passive
working culture with no sense of urgency to generate more impact or
productive initiatives. Consequently, hampering organizations’s ability to
achieve bigger outcome and does not translate on improvement or better
achievements.
Perrin (1998) proposed key strategies in effective utilization of
performance measures. First, there need to be an awareness on performance
measures as a planning and monitoring mechanism to identify relevant
performance issues. Performance measures are invaluable in directing the
attention of management towards areas where thorough performance
evaluation is needed, aiding stakeholders on effective allocation of resources.
Secondly, it is important to also recognize limitations of measurements as
very few reliable evaluation methods depend on a single source of
information thus denoting necessary balance between quantitative data with
its qualitative (social, cultural, political aspects) counterparts to consult each
other to get the systemic and holistic view of the organization. Lastly, users
and stakeholders need to strategic in the use of performance measures,
understand its characteristics and features, and assigns on the most
appropriate indicators considering the circumstances surrounding it.
37
Indeed, measuring effects, outcome, and implications in public

organizations possess a challenging task (Christensen et al., 2007; Poister,
2003). Improving performance entails the identification of specific outcomes
in which normative goals are translated into measurable outputs and
quantifiable results (Christensen et al., 2007). However, public service
domain is complex by nature and cannot be easily reduced to a numerical
figure as measurements may overlook the intricate nature of social
phenomena with multiple interrelated elements that hardly effectively
simplified into a limited set of quantitative indicators (Perrin, 1998).
As noted by Nielsen and Ejler (2008) that measurement plays a central
role in performance management as measurement data from performance
reports are delivered timely for decision-making, on the other hand,
performance monitoring supports program operations by supplying real-time
data for responsive organization’s strategies. Secondly, evaluative judgment
derived from performance measures offers insights applicable for day-to-day
management while also enabling the identification of necessary for
improvements in various aspects of management.
Over the years, performance measurement has garnered significant
attention as a contemporary novel approach to guarantee a focus on
production of responsible outcomes and accountability, ultimately serves as a
device to evaluate government programs and public expenditures through a
system of oversight of (Perrin, 1998). Ultimately, performance measures can
benefit its users as a constituent of a more extensive and all-encompassing
assessment approach. Nonetheless, measuring impact within organization has
proven to be a deceptively difficult task, especially in the public sector.
In the auditing context, Supreme Audit Institutions (SAIs) utilizes
diverse mechanisms to assess their capability and efficiency. Ranging from
documentation of operations and progress to the International. As the auditing
body at the global level, the SAIs are entities with such diverse and distinct
attributes that no universally recognized framework or unequivocal
benchmarks of distinction have been established to holistically evaluate them
(DFID, 2005). If a supreme audit institution is to assess its operation
38
objectively, it must identify the affirmative aspects, discrepancies, and

restraints it experiences before formulating a program to enhance its
capabilities. Kontogeorga (2013) noted a number of factors constraining
SAI's capabilities including: insufficient resources, inadequate expertise
among personnel, absence of autonomy, and, within the domestic sphere,
underdeveloped management of the public finance system (e.g. inadequate
bookkeeping systems, issues on financial statements, limited or lack of
internal control, and unfamiliarity with accounting and audit standards, etc.).
Standards of Supreme Audit Institutions (ISSAI) framework
(Kontogeorga, 2013). More initiatives are being done within SAI community
indicated by the establishment of the SAI Performance Measurement
Framework (PMF), a global framework developed by the INTOSAI Working
Group on the Value and Benefits of SAIs (WGVBS) and first introduced in
2016 at the XXII INTOSAI Congress in Abu Dhabi (IDI, 2019).
Fundamentally, SAI PMF is an evidence-based assessment to assist
SAI in self-evaluating its performance against the International Standards of
Supreme Audit Institutions (ISSAIs), INTOSAI Framework of Professional
Pronouncements (IFPP) and other recognized international best practices in
external audit (IDI, 2019). SAI PMF was developed to help SAI in evaluating
their system and designing strategic plan on the institutional, organizational,
and professional performance. It is the outcome of a collective effort to
establish a performance assessment framework that can be implemented by
all SAI, regardless of their background and other differences by evaluating
SAI in both audit and non-audit functions domains including its institutional
independence and autonomy, operational and financial management, human
resources, audit methodologies and procedure, all in a holistic manner
(Vosawale-Katuba, 2022).
According to the INTOSAI Capacity Building Committee (2022), key
interconnected features that need to be identified in the SAI PMF report are:
a. Independence and Legal Framework
This aspect encompasses the legal mandate and independence of the SAI
relative to other government stakeholders. The Lima Declaration and the
39
Mexico Declaration have emphasized the importance of independent

SAIs, both in financial and organizational domains. SAI should be free
from interference that could hamper and weaken its audit work. This
indicator is placed first as the result may significantly affect other
indicators. The lack of organizational independence may constrain other
areas such as audit quality, recruitment practices and communication
with stakeholders.
b. Internal Governance and Ethics
To maintain credibility in doing oversight works on other government
bodies, SAI should lead by example and serve as an exemplary institution
for its auditees. INTOSAI principle requires SAI to adhere to high
standards of integrity and ethics for all staff levels. SAI should first set
an example by encouraging transparency, accountability, and good
governance practised consistently within their organization.
c. Audit Quality and Reporting
This indicator assesses the output quality of SAI related to its core
business. SAI holds the mandate to perform financial, performance, and
compliance audits on the public sector. Ensuring sufficient quality of its
work, methodology, and procedures will highly determine its ability to
produce valuable findings and recommendations in its audit reports.
d. Financial Management, Assets and Support Services
Commitment to accountability means SAI should also be able to
demonstrate with a reasonable level of confidence that responsible
management practices are taking place in the use of its resources. Good
organizational management and robust internal control can be applied in
the area of finance, asset, and support service from all the staff in day-to-
day operations.
e. Human Resources and Training
The role of human resource management is crucial in ensuring the
delivery of high-quality audit work. As endorsed by The International
Standards of Supreme Audit Institutions (ISSAI) 100 and 140, SAIs must
devise policies and strategies on human resources to ensure the
40
organization is equipped with adequate personnel with skills, abilities,

and competence. For SAI to successfully execute its work, processes
from recruiting individuals possessing appropriate qualifications,
providing opportunities for staff development, and maintaining
professional competence through training should not be overlooked.
f. Communication and Stakeholder Management
Effective communication is key to promoting SAI’s contribution and
relevance to its auditee. An important aspect to consider in this process
of communication and stakeholder management is the selection of an
appropriate style, language, and format that effectively engages
stakeholders.
Fundamentally, SAI framework is a means to improve SAI's ability to
deliver value in the public sector. However, the persistent challenge in
building a reliable measurement model lies in determining robust, reliable,
meaningful, and significant indicators (Hemici & Kontogeorga, 2020). An
ideal assessment model needs to encapsulate this mission and translate the
INTOSAI Principle-12 into executable indicators for SAI. Although SAI
PMF departs from the noble idea of strengthening SAI's ability to deliver
value, the framework lacks a vital point in its holistic claim. Despite drawing
its indicators from INTOSAI’s principles, best practices, and other
international standards from numerous sources, a closer look at the model
shows a narrow perspective on SAI performance as a form of internal
assessment rather than a comprehensive evaluation as it presents a limited
discussion on external engagement.
However, as an external auditing body with an oversight mandate SAI
also plays important role in validating information, data, measures, indicators,
and evaluation done by other government’s institution as its auditee. In order
to execute efficient supervision, audit bodies must possess the necessary
capacity and expertise, acknowledging the distinct nature of performance
assessment and outcome-based report in comparison to financial information
and treat them, a skill set that many auditors may not have been trained in
(Perrin, 2006).
41
Governments are adopting performance measurement systems to be

implemented in many facets of managements and incorporated into the
process auditing mandates. However, developing effective measurement
systems demand deliberate effort and commitment from the organization. As
a public institution, it is also important for SAI to not divert the focus needed
to maintain its significant role in ensuring accountability through its audit
work. INTOSAI acknowledges this condition and has mentioned the
necessity to involve various stakeholders in SAI evaluation (IDI, 2022; 2012),
indicating room for improvement concerning organization assessment. Thus,
the authority that SAI holds, entails more responsibility and initiative to not
only resolve its internal issue of capacity building but to also take into account
the various points of view of stakeholders, feedback from its auditee and the
interest of the greater public.
2.2.3. Government Oversight
In public sector perspective, accountability serves as a mechanism
to ensure appropriate conduct and adherence to laws and regulations by
executive agencies and officials (Aucoin & Heintzman, 2000). To address the
issue of accountability in the lens of public institution, this research will
further discuss the matter in the scope of public accountability.
Management of budgeting, allocation, and expenditure of public
fund by government entities sheds light on the importance of oversight for
holding these agencies accountable. All governments face growing demands
to achieve and showcase results, highlighting the significance of focusing on
outcomes for efficient public administration (Perrin, 2006).
Meanwhile, the dialectics of accountability proposed by Aucoin &
Heintzman (2000) built upon dimensions of policy, organizational and
management with three main purposes. The first is to prevent the abuse and
misuse of public authority. The second, as instrument of assurance on proper
use of public resources and adherence to the law and public service values.
The third, as stimuli to foster continuous improvement in governance and
public management. In management and public governance, accountability
42
represents the guiding principle for public authority being held to account
(Bovens, 2007; Mulgan, 2000; Aucoin & Heintzman, 2000).
Accountability, as Bovens (2006) succinctly defined, is the
responsibility to explain and justify action. Elements covered within the
concept of accountability reflected in four interdependent questions
illustrated in Figure 2.2. The first, concerning the question of to whom is the
account to be presented. Secondly, the question of who should be held to
account. Thirdly, the topic or program that presented to held accountable.
Lastly, the rationale behind urgency and willingness of being accountable.
Figure 2.2 Elements of Accountability

Source: Bovens (2007)
On characteristics of accountability, Mulgan (2000) further

emphasizes external scrutiny, interaction, authority as key features and its
application as a mechanism of control in which the core sense of
accountability is based on the general purpose of constraining power and
imposing control over government bodies. A solid mechanism of external and
internal control prevents misuse of authority and endorsed accountability
(Paudel & Gupta, 2019). However, the diverse authority within government
requires equally powerful constraints of strong legislative oversight along
with comprehensive system of judicial and administrative review alongside
them (Aucoin & Heintzman, 2000).
Government is tasked with complex issues of making progressive
policies while remaining accountable to external scrutiny (Naidoo, 2015;
Mulgan, 2000). To navigate this responsibility, control and oversight serve an
integral role with auditing as one of the safeguards for better management of
public fund. Thus, institutions who oversee accountability are established to
control potential misconduct and restrain government power. Consequently,
43
embracing public accountability means adapting these principles consistently

to overcome the challenge in building a strong system of control, compliance,
assurance, and improvement into the oversight system for responsible state
management.
The discourse on oversight thus further evolved to encompass
accountability frameworks involving various stakeholders, moving beyond
traditional government-centric governance models (Stone & Ostrower, 2007).
Oversight can no longer be seen as a single factor variable (Rockman in Selin
and Moore, 2003), but rather on the collective efforts of multiple
interconnected of parties and stakeholders (FAI, 2021). Government
oversight refers to the role of monitoring and regulating the use of public
funds by government agencies, departments, and organizations (Schiavo‐
Campo, 2019). It involves governing bodies scrutinizing policies to ensure
their effective implementation (Ríos et al., 2014) and assure that officials
comply with regulations and fulfill their duties (Carlson et al., 2015).
Essentially, oversight is a series of conducts to collect information
on executives through the process of evaluating, monitoring, and overseeing
implementation of policy made by government agencies and public officials
(Selin & Moore, 2023). Independent public fund management oversight
bodies such as audit institutions, ombudsmen, and watchdog agencies, all
taken a crucial role in government oversight. These entities operate
independently of the government and are tasked with monitoring and
investigating government activities, promoting transparency, and protecting
the rights of citizens. They conduct audits, investigations, and reviews to
identify waste, fraud, abuse, and inefficiency in government operations.
As Selin & Moore (2023) emphasizes the complexity of oversight
processes, it highlights audit as one of the acclaimed oversight techniques.
Independent auditing body like SAI plays an important role in the system to
provide an additional layer of accountability, enhancing public trust and
confidence in government institutions. To implement policy
recommendations offered by SAI, there need to be collaborative movement
within all levels of government institutions. This indicates the intricate
44
relationship between government oversight and audit effectiveness, with

SAIs playing a unique role in advancing public value beyond traditional
oversight functions. It takes a critical role in promoting good governance,
accountability, and transparency across various sectors. Their independence,
mandate, and authority enable them to conduct thorough and objective audits,
providing valuable insights and recommendations for improving
organizational performance and compliance.
Meanwhile, legislative oversight is one of the primary forms of
government oversight, conducted by the legislative branch of government,
such as parliaments or congresses. Legislatures review and evaluate
government actions, policies, and expenditures through hearings,
investigations, and the enactment of laws. This oversight ensures that
government agencies operate within their authorized powers, allocate
resources efficiently, and achieve their objectives effectively. Legislators also
have the authority to demand information, subpoena witnesses, and hold
hearings to hold government officials accountable.
Though SAI operates independently, it exercises control over public
administration on behalf of legislature body or parliaments as one of its
stakeholders. SAI represents citizens and parliaments in overseeing
government operations through its audit reports (Bringselius, 2015;
Reichborn-Kjennerud, Carrington, Gonzales, & Klarskov, 2015) in turn,
parliament also contributed by strengthening legitimacy and support for SAI
(Cordery and Hay, 2018).
On the other hand, executive oversight involves internal controls and
monitoring mechanisms implemented by the executive branch of
government, led by the head of state or government. This includes executive
agencies, departments, and offices responsible for implementing and
enforcing laws and policies. Executive oversight ensures that government
agencies comply with laws and regulations, adhere to established procedures,
and reach desired outcomes. It often involves performance evaluations,
audits, inspections, and internal investigations to identify and address
deficiencies and improve organizational effectiveness.
45
Lastly, judicial oversight refers to the role of the judiciary in

interpreting laws, adjudicating disputes, and reviewing government actions
for compliance with the Constitution and legal standards. Courts have the
authority to review the constitutionality and legality of government actions,
regulations, and decisions through judicial review. Judicial oversight serves
as a check on the power of the other branches of government, ensuring that
they do not exceed their constitutional authority or violate individual rights.
Considering this interconnected value chain, the oversight system
has many actors and stakeholders taking part in its operation. In Indonesia,
the authority to oversee the management of public funds in government
bodies is conducted by BPK as an external audit institution independent from
executive, legislative and judicial elements (Kurniawati & Pratama, 2021).
BPK RI has shifted from focusing on internal capacity and infrastructure to a
more outward orientation aiming to have a greater impact on its strategic
environment, congruent with the Accountability Organization Maturity
Model (AOMM), as depicted by figure 2.3 below.
Figure 2.3 Accountability Organization Maturity Model

Source: Suartama, Nurdin, & Susilo (2015)
46
Therefore, SAI contribution towards government oversight and

accountability will be improved only when: (1) it properly recognizes the
respective responsibilities of government and public servants; (2) it
acknowledges the limits of performance auditing; (3) its audits and reviews
take into account the constraints imposed on public servants in the
management of programs and resources; and, (4) it assesses performance
from the perspective of public management as governance.
The use of audits is a complex process that also involves information
sources other than the audit and existing insights and bias, such as political-
legitimizing use and tactical use. Political-legitimizing use means that
knowledge is an asset in a political debate (Feldman & March, 1981), while
tactical use means that knowledge is used to influence a decision-making
process. Therefore, this use of the information may provide new knowledge
that someone regards as useful; for example, the opposition for holding a
minister to account, and for the government in arguing for policy changes and
systemic improvements. However, this goal can only be realized when SAI's
robust audit works meet an equally responsive government as impactful audit
depends on the implementation of recommendations. Though audit office
serves the primary function as the institutions of accountability, it shares
responsibility with legislature, judiciary, civil society, media, and the general
public. Without such mechanism of collective control and oversight, attempts
in imposing accountability to government from these institutions risk failing
their purpose (Mulgan, 2000).
Over the past decades, SAIs around the world have grown more
concerned about identifying appropriate measures for effective government
oversight until INTOSAI finally introduces six key components for
optimizing audit impact in its 2020 working paper entitled “All Hands on
Deck”. Indicators for measuring the impact of audits can provide insight into
the effectiveness, efficiency, and overall value of the audit process. These six
components are said to be integral elements of the oversight ecosystem, each
of them explained (IDI, 2021a) as follows:
47
a. Institutional mandate of SAI guarantees that public auditors possess the

autonomy, authority, independence, and necessary resources to satisfy
their duties. SAI needs to be independent of the audited entity to enable
them to deliver their mandate without interference and fear of
repercussions. A well-defined legal basis is a critical prerequisite for the
effective functioning of SAIs as it will provide sufficient operational
powers to establish its role as the external auditor of the government.
b. Quality of audit refers to the production of relevant high quality audit
reports. For public sector auditors, the required competency to sustain
such quality need to continuously improved with the increased
complexity of government operation. There is a recognition for SAIs to
strengthen their human resources capacity to meet these changes,
especially in this era of rapid progressive technology.
c. Legislative oversight is indicated by the existence of a dedicated
legislative body committee that engages with SAI and makes use of their
audit report to oversee the government. This factor should not be
overlooked as SAIs’ ability to create impact also depends on their
abilities to interact with other stakeholders. INTOSAI (IDI, 2020) reports
regular communication with the legislature is reported by 75% of all
SAIs, but only 47% report that they involve the legislature regularly in
follow-up discussions about the extent of implementation of
recommendations.
d. Executive response
Affirmative behavior from government officials during and after
completing the audit process shows how the audit report can give
accurate and reliable solutions to the existing problem. This positive
response should then be translated action or decision made by the audited
official and/or other competent parties to carry out audit
recommendations (Budiman, 2021).
However, according to the responses to the INTOSAI Global Survey
(IDI, 2020), the biggest obstacle to delivering audit impact globally is the
executive response, reported by 60% of the SAIs. The commitment and
48
implementation of audit recommendations by executives remains one of

the weakest link in the audit and oversight ecosystem.
e. Independent follow-up
An independent system of monitoring and follow-up is inevitable to
ensure audit recommendations are effectively implemented. The audit
recommendation is suggestions from the auditors in light of the findings
of an audit, intending to prompt the authorized individual or entities to
undertake rectifying measures to address issues in the management of
public funds. It is generally perceived that the extent to which the
suggestions from the audit findings are implemented will demonstrate
the progress made by an organization in overseeing government
performance (Budiman, 2021). This monitoring is typically conducted by
SAI as an independent oversight body.
f. Public participation in the audit and oversight process refers to the
involvement of public civil society organizations, the media, and citizens
– to engage, influence, and hold scrutiny over the auditing process.
Globally, SAIs are still closed, inward-looking organisations that could
demonstrate their own accountability better by being more transparent,
undergoing external audits of their financial statements, and publishing
performance results.
These elements constitute the assessment standard in measuring the
collective work of both SAI and the government in optimizing audit findings
and oversight systems. The indicators demonstrate what needs to be
considered when dealing with government oversight and SAI's contribution
of impact to society.
2.2.4. Audit Impact
Organizations need to evaluate the relevance and effectiveness of
their strategies and identify areas where improvements are needed. An audit
should be able to help assess whether an organization's initiatives are aligned
with its objectives, assist proper implementation, and ensure programs and
activities effectively contribute to reaching the desired outcomes.
Subsequently, as governments face growing demands to enhance both
49
productivity and efficiency (Aucoin & Heintzman, 2000), auditing give. An

audit institution plays a crucial role in enhancing public confidence in the
governance system by assisting and improving the process of public
administration through their independent examination and reporting, thereby
providing assurance of responsible financial management and effective
government performance to the public (Morin, 2001).
The need of organisational control, greater demands for
accountability and transparency have all triggered a systemic increase of
auditing (Power, 2000). Audit in the public sector serves as a means of
ensuring government accountability in relation to administration of state
finances (Akbar, Aswar & Lastiningsih, 2020; Suzuki, 2004). Auditing sheds
light on government financial management dealing with the urgency for
public scrutiny (Naido, 2019), as citizen needs to be informed on the
performance of government institutions and auditing functions need to adapt
accordingly to these demands (Irawan, 2015).
Audit serves the primary objective as a tool that enables
administrative accountability and promotes improvement (Reichborn-
Kjennerud & Johnsen, 2015). Auditing body also holds a significant role in
endorsing transparency, efficiency, and effectiveness of public management
(Akyel & Kose, 2013; Bovens, 2007) by evaluating the economic, efficient,
effective, and transparency aspects of public administration (Pollitt et al.,
1999). Auditing provides an independent and objective assessment of the
financial processes, controls, and transactions within government institutions.
Government auditing or more often referred as public auditing aims
to ensure public accountability (Suzuki, 2004). In a constantly evolving
expectation of society, SAI is essentially designed to secure a desirable level
of impact in improving the management of public finances, promoting trust,
transparency, and accountability (Nagy, 2012). SAI is required to provide
assurance by reporting whether government have exercised accountable
conduct and adherence to law in their management of taxpayers’ funds
(Friedberg & Lutrin, 2005). Accordingly, audit carried out by the Supreme
Audit Institution (SAI) aims to promote public accountability of public funds
50
(Cordery & Hay, 2022; OECD, 2011) in which the responsibility for
managing public business and using public resources will be held accountable
(Brétéché & Swarbrick, 2017). Most importantly, SAI offers analysis on the
correlation between performance management and evaluation, as well as
decision-making across all government levels (OECD, 2016, p. 46).
SAI conducts the external examination of government organizations
and serves a crucial component within the official framework of financial
responsibility in many nations (DFID, 2005). SAI carries out the external
audit of public sector bodies and are one of the key links in the formal system
of financial accountability, assessing performance and improving the public
accounting system (Suzuki, 204). SAI have a pivotal role in this context as
the sole entities capable of providing an impartial judgment regarding public
sector management and the degree to which the bureaucracy functions in
accordance with predetermined principles (INTOSAI, 2001) by serving
several functions. Primarily, their audit reports suggest measures for
enhancing administration and assist government in maximizing the utilization
of public funds. Secondly, they ensure efficiency in that objectives are met
with the minimum feasible cost and that the accounts are prepared in a clear
manner. Additionally, the disclosure of audit findings provides assurance for
citizens and accountability for actions of the government and its
representatives.
The International Organization of Supreme Audit Institutions
(INTOSAI) International Development Initiative joint report pointed out that
external audit is an integral component of Public Financial Management
(PFM) in which audit serves as the core, auditors as the key actors (IDI, 2020).
This strategic role earns SAI the ability to probe into government programs
and activities and ensure the exploitation of public resources will be held
accountable by the law (Brétéché & Swarbrick, 2017). Thus, improving trust
and legitimacy is essential for SAI to maintain the strategic position of
ensuring a responsible allocation of public resources (Ototea, Batusaru,
Ungureanu, 2015).
51
External Auditing office like SAI holds authority allows them to

demand compliance from auditees to dutifully implement changes following
findings outlined in their reports. Therefore, to maintain its authority would
require SAI to demonstrate its benefits, the issue of increasing value of SAI
audit work has become a natural concern (Lonsdale, 2000). As SAIs need to
deliver wider, constitutional accountability (Noussi, 2012; Pallot, 2003) the
importance for SAI to report their outputs, outcomes, impact and analysis the
public value they provide is prevalent (Cordery and Hay, 2018; Raudla et al.,
2015). In this sense, audit impact measure is a means to showcase SAI’s value
and the extent to which its work has an impact on society.
The topic of generating more impact for SAI has been a persistent
discourse from the early days of auditing through the era of public
administration reforms. As Frost (2021) believes that to create substantial
impact in the public sector, audit culture needs to be reformed and improved.
Audit impact as defined by INTOSAI (FAI, 2021) is SAI's positive
contribution to society, by making a difference in the lives of citizens through
accountability, transparency and integrity of government. Impact itself can be
described as the variety of direct or indirect, intentional or unintentional
effects of evaluation (van Loocke & Put, 2011, p. 176). Audit impact should
be able to bring change as it remains a crucial component in addressing
systemic improvement, practical changes, and substantial impact on
government bodies and their programs. Thus, measuring audit impact refers
to evaluating the extent of audit influence within an organization, covering
aspects of effectiveness, efficiency, and success of audits against the
organization's objectives.
A robust mechanism for monitoring to follow-up the implementation
of audit recommendations is exercised by many SAI, including BPK. In
perspective that without an effective follow-up process, the fundamental
purpose of performance auditing, which is to drive improvements in the
audited entities, cannot be achieved. Irawan (2015) noted the follow-up
mechanism in the context of performance auditing in the Indonesian public
52
sector involves multiple stakeholders, including the Indonesian Supreme

Audit Institution or BPK as auditor, the audited entity, and the parliament.
However, progressive changes require a clear vision of what it takes
to create an impact. Though SAI need to have an impact-based approach
enabling implementation of preventive measures and corrective actions,
INTOSAI (2012) has designed 20 frameworks related to SAI assessment and
Public Fund Management, but these frameworks are considered inadequate
which prompted the development of SAI Performance Management
Framework (SAI PMF) and Government Oversight measures. Such
endeavors indicating the challenges faced by SAIs in creating a universally
applicable measure for audit impact.
Cordery & Hay (2018) argues that finding an unbiased measure to
assess the impact of SAI on public sector is an undeniably challenging task.
Torres et al. (2018) noted that audits aim to generate actions and the capacity
to produce these actions is assessed by examining their impact.
Unsurprisingly, audit impact assessment can not only rely on the ability to
count the number of implemented or unimplemented recommendations as the
benchmark of success but also need to dive into the dimensions of public
service value of improvement in policy, organizational, and government
performance where audit impact should be able to bring changes even in the
absence of a capacity to translate them into quantitative measures. Wilkins
(1995) suggested measures accommodating auditee rating or other relevant
third-party examiners to assess audit offices. Thus, there are several key
dimensions to consider when measuring the impact of audits, including:
improvements in internal controls, compliance with regulations, operational
efficiency, and successful execution of strategic goals.
First and foremost, audit impact measurements pay attention to the
enhancement of control within the organization. Audits help identify
weaknesses and risks in the internal control system, thereby enabling
necessary improvements to enhance the security, accuracy, and reliability of
business processes. By measuring the extent to which audits successfully
53
enhance internal controls as organizations able to assess the positive impact

of audits on their day-to-day operation and management.
Furthermore, compliance with regulations is a critical aspect of audit
impact measurements. Audits help ensure that organizations comply with all
applicable laws, internal policies, and industry standards. By measuring the
level of compliance obtained through audits, organizations can assess the risk
of compliance and the effectiveness of their control systems in ensuring
compliance.
Operational efficiency is also a focus in audit impact measurements.
Audits help identify opportunities to improve business process efficiency,
reduce costs, and optimize resource utilization. By measuring the impact of
audits on operational efficiency, organizations can assess the success of audits
in helping obtain higher efficiency and productivity (Andrianto et al., 2021).
Additionally, audit impact measurements encompass the evaluation of the
organization's strategic goals. Impact can be demonstrated through changes
in the practices of the auditee, typically measured by the implementation of
recommendations, or by influencing the political debate (Torres et al., 2018).
This will provide valuable information in understanding the effect of audit,
identifying areas for improvement, and the extent of recommendations
considered useful to the auditee.
In measuring the impact of audits, it is also important to consider
both qualitative and quantitative aspects. In addition to numerical data such
as cost reductions or efficiency improvements, it is important to take into
account factors such as stakeholder’s opinion and auditee perceptions on the
impact and benefit of audit reports to the intended outcomes and objectives
already set by government. Key stakeholders’ engagement and feedback from
auditee reflects the quality of SAI audit work; however, research by Cordery
& Hay (2018) reveals that not all SAIs prioritize this aspect, as most SAIs
focus on the number of audit reports produced rather than linking measures
of outputs to desired outcomes.
54
Finally, to ensure sustainability and continuous improvement in

enhancing audit impact, it is important for organizations to regularly evaluate
and update their metrics and measurement processes. This allows
organizations to continually improve the effectiveness of their audits and
ensure that audit works remain relevant to the organization's objectives and
needs over time. Therefore, audit impact measurements are not only a tool for
evaluating past audit performance but also provide a foundation for
improving audit practices and delivering sustainable value to the
organization. While this factor remained an essential variable in audit work,
the literature on audit impact evaluation thus far has not fully considered the
organizational structure, political, social cultural, and many other elements
that must be addressed in determining the level of audit impact. Thus, audit
impact measurements will be able to provide an extensive view of the
contribution of audits to overall organizational performance and success.
SAI utilize different methods including reports on their programs,
evaluations based on the ISSAI framework, and indicators developed at
national or international levels (Kontogeorga, 2013). However, evaluation,
measures and its overall processes related to the practitioner’s perspective of
audit are generally accepted and rarely challenged when it should have been
subject to critical examination. A comparative study by Cordery & Hay
(2018) also highlights how SAI reports on improvement of public sector
prioritize critiques on efficiency and effectiveness rather than the overall
government policy. This arises questions on how the plethora of techniques,
frameworks, and measurements developed within audit community has
become uncontested arbiter standards. Therefore, evaluating impact,
especially in the public sector domain needs to be exercised with caution.
2.3 Research Model
2.3.1. Conceptual Framework
INTOSAI’s government oversight measurement indicators
incorporate the six dimensions to conceptualize impacts on various levels.
The evaluation model can also be used during implementation to identify
which indicators must be monitored. This can help design more realistic
55
goals, clarify accountabilities, and establish a common understanding of the

strategies to reach the goals. The next step is to identify possible important
determinants and how effective government oversight contributes to the audit
impact of BPK.
The dimensions of audit impact evaluation are synthesized by
Johnsen et al., (2018) into four dimensions characterizing impacts across
various levels of usefulness, change, improvement, and accountability as
shown in Figure 2.4. Usefulness as perceived on the individual level.
Accountability is the concern of the organization. Change and improvement
relate to programs possibly affecting control systems and/or policies.
Figure 2.4 Audit Impact Model

Source: Johnsen et al. (2018)
The first dimension of usefulness assesses the capacity for conceptual

impact. It investigates the utility of the audit by confirming to the auditee to
what extent they believe the audit to be beneficial for their organization. The
perception of the report's usefulness does not automatically imply tangible
modifications within the audited organization. However, if the public officials
perceive the report as valuable, it has the potential to influence their decision.
56
The second dimension evaluates audit as an accountability device,

indicating whether members of parliament utilize the audit to hold the
government accountable for its management of public funds. This is done by
posing inquiry about the degree they believe audit reports can hold the heads
of departments or agencies responsible for their actions. The third dimension
of change delved into the state of alteration that occurred as a consequence of
the audit. This variable quantifies the perceived modifications auditee
implements in response to the evaluations and recommendations outlined in
the audit report. The fourth dimension investigates the impact of audits as a
device to generate improvement within the organization. In the examination
of this aspect, one needs to also consider the potential bias on the term
improvement because what one group conceives as an improvement another
group might conceive differently (Boyne, 2003). Opinions may even differ
internally within the organization.
Ultimately, this variable is expected to be able to indicate a link
between the audit report as an output and improvement as an outcome and the
pinnacle objective of an audit. The measurement of the dependent variable
referred to previous research is modified by incorporating INTOSAI’s
government oversight indicators as shown in Figure 2.5 below.
Institutional
Mandate
Quality of Usefulness
Audit Work
Accountability
Legislative
Oversight Government Audit
Oversight Impact
Change
Executive
Response
Improvement
Independent
Follow-up
Public
Participation
Figure 2.5 Conceptual Framework
57
The independent variable is government oversight with dimensions

consisting of institutional mandate, quality of audit work, legislative
oversight, executive response, independent follow-up, and public
participation. as the independent variable, it represents the factor that
influences or drives the evaluation process. The dependent variable is audit
impact which refers to the assessment or analysis of the impact of audit
activities conducted by the SAI of Indonesia. This evaluation includes the
four audit impact dimensions of usefulness, accountability, change, and
improvement developed by Johnsen at al. (2018) and Raudla et al. (2015).
2.4 Hypothesis Development
Previous studies have shown factors that influence government oversight
audit impact in the public sector, but their results still vary. Based on the theoretical
discussions, review of the theories, and previous research on factors that are expected
to influence the audit quality; the formulation of hypotheses concerning the audit
impact measurement is as follows:
a. Institutional mandate of SAI
The possibility of the auditor not reporting material misstatement depends on the
auditor's independence. When auditors cannot maintain their independence, it
will directly have a negative impact and reduce audit quality (Deis & Giroux,
1992). Research results by Aswar (2020) conclude that independence has a
significant effect on audit quality. Thus, the hypothesis is formulated as follows:
(H1): The institutional mandate of SAI has a significant direct effect on
Government Oversight.
b. Quality of audit work
Audit quality can limit managerial opportunistic attitudes and can detect the
possibility of audit failure by the auditor (De las Heras, Canibano & Moreira,
2012). SAI needs to pay attention to the auditor's expertise and continuously
develop them through training and practical experience from the audit process.
Consequently, auditor’s competence, skills, experience, expertise, and integrity
are considered related to audit quality (Kertarajasa et al., 2019; Mansouri,
Pirayesh & Salehi, 2009). Thus, the hypothesis is formulated as follows:
58
(H2): Quality of audit work has a significant direct effect on government

oversight.
c. Legislative oversight
Studies show that legislative oversight positively influences budget transparency
(Ríos et al., 2014) and the legislative body has the authority to investigate
potential wrongdoing by the executive branch (Bussing & Pomirchy, 2022).
Mkasiwa (2019) also highlights that legislative budgetary oversight fosters
public accountability and positively impacts budget transparency. In order to
ensure accountability, it is crucial for Parliament to possess the necessary
technical capabilities to effectively manage information (Reichborn-Kjennerud
& Johnsen, 2015). As government body, SAI is also given the mandate of
auditing by parliament, ultimately making SAI responsible for supporting
Parliamentary oversight committees by providing comprehensive reports and
enhancing their capacity for scrutiny (Santiso, 2015). Thus, the hypothesis is
formulated as follows:
(H3): Legislative oversight has a significant direct effect on Government
oversight.
d. Executive response
Conflicting evidence suggests that the response from government officials'
perception may not always align with the goals of audit and oversight. Lin et al.
(2013) suggest that the independence of government officials can be contentious
as they may prioritize governmental interests over audit quality. Thus, the
hypothesis is formulated as follows:
(H4): Executive response has a significant direct effect on Government
oversight.
e. Independent follow-up
Challenges in establishing fully independent monitoring systems within
government oversight frameworks. Nau et al. (2019) emphasize the significance
of coordination structures for governance and oversight over implementation
and monitoring, indicating that true independence in monitoring mechanisms is
seldom attained. Thus, the hypothesis is formulated as follows:
59
(H5): Independent follow-up has a significant direct effect on government

oversight.
f. Public participation
Research by Sabet (2020) emphasizes that forward-looking audits, which aim to
hold officials accountable for performance, require independent accountability
mechanisms, systematic follow-up methodologies, public dissemination, and
pressure from the media and civil society. Thus, the hypothesis is formulated as
follows:
(H6): Public participation has a significant direct effect on Government
oversight.
g. Government oversight
Studies suggest that government oversight, particularly in the form of audits, can
significantly improve the efficiency of government spending, ensure adherence
to rules and regulations, and detect unethical practices (Mihret, James, & Mula,
2010; Bertrand et al., 2007). However, Lapsley (2009) suggests that the adoption
of NPM principles, often associated with government oversight, may not always
lead to the desired transformation of public sectors. Thus, to further investigate
the significance of government oversight towards audit impact, the hypothesis is
formulated as follows:
(H7): Government oversight has a significant direct effect on audit impact
60
2.5 Concept Operationalization

Concept Variable Dimension Definition Indicator Item Respondent Scale
Evaluation Audit Impact Usefulness Audit provide new 1.1. SAI report provides 1.1.1. Does BPK audit report help uncover issues in important policy areas? Auditee Likert
of Audit knowledge to the auditee better insight for 1.1.2. Does BPK audit report give solution on dealing with issues within the organization? scale
Impact (Johnsen et al., 2018) organization 1.1.3. Does the link between audit criteria, facts and assessments in the performance audit 1-6
Measures (Raudla et al., 2015) report was clear?
Change Actions or modifications 1.2. SAI findings are 1.2.1 Does the recommendation offered by BPK realistic to implement? Auditee Likert
made by the audited entity in incorporated within 1.2.2 Does the auditee implement BPK recommendations? scale
response to findings and organization 1.2.3 Does more controls as consequence of the audit recommendations? 1-6
recommendations of audit
report (Johnsen et al., 2018)
(Raudla et al., 2015)
Accountability Responsibility of individuals 1.3 SAI audit promotes 1.3.1 Does the auditee system control score improve in every audit year? Auditee Likert
or organizations to justify transparency and 1.3.2 Does SAI reports and findings helps detect fraud, corruption, or non-compliance of law scale
their actions, decisions, and prevents fraud and regulation? 1-6
use of resources. (Johnsen et
al., 2018)
Improvement The positive changes within 1.4 SAI audits are 1.4.1 Does BPK findings and recommendation help organization perform better? Auditee Likert
the organization(Johnsen et contributing to 1.4.2 Does the auditee have repeated issues in every audit year? scale
al., 2018) improvements in the 1-6
audited entities
Government Institutional Independence and sufficient 2.1. SAI is an independent 2.1.1. Does BPK is an independent organization body protected by law? Auditor Likert
Oversight Mandate resources for SAI (IDI, 2020) organization and 2.1.2. Does BPK has the authority to manage its resources? scale
manages its resources 2.1.3. Does BPK has the liberty to choose what to audit and select methods following 1-6
professional standards? (ISSAI 3000, 3200);
Quality of Assurance of high-quality 2.2 Auditors have good 2.2.1. Does BPK auditor have standardized training? Auditor Likert
Audit Work audit held by scientific/good knowledge and 2.2.2. Does BPK auditor undergo training regularly? scale
standard (IDI, 2020) expertise 1-6
2.3 SAI has standardized 2.3.1. Does BPK has a standardized audit procedure?
method and 2.3.2 Is there QA/QC mechanism to maintain those standards?
procedure for all its 2.3.3 Are auditees informed on audit criteria and the way they were used in the audit process?
audit work
2.4 Audit report gives a 2.4.1 Are audit criteria, evidence, and analysis in the audit report clear?
sufficient 2.4.2 Are audit recommendations considered practical and economically viable to execute?
explanation of its
findings
Legislative Legislature active 2.5 The legislature body of 2.5.1 Does the legislative body involved in BPK work? Auditor Likert
Oversight involvement in SAI works the Public Account 2.5.2 Is there a designated time for BPK to deliver its report to the legislative body? scale
and reports (IDI, 2020) Committee collaborates 2.5.3 Does BPK conduct regular communication with the Public Account Committee of the 1-6
with SAI legislative body?
Executive Government’s response and 2.6. Positive reactions 2.6.1 Does auditee’s officials agree with BPK's recommendation? Auditor Likert
Response adherence to change following towards audit/auditor from 2.6.2 Does auditee’s officials support the audit team during the auditing process? scale
audit report (IDI, 2020) officials 1-6
61
Independent Active monitoring of 2.7. Independent tracking 2.7.1 Does SAI have a tracking and monitoring system for audit follow-up? Auditor Likert
Follow-up implementation audit and monitoring system of 2.7.2 Is the system updated regularly? scale
recommendations (IDI, 2020) audit follow-up 2.7.3 Are entities informed on the system? 1-6
Public Public engagement in 2.8. Accessible 2.8.1 Is there civil society participation in audit? Auditor Likert
Participation auditing process (IDI, 2020) participatory channel for 2.8.2 Is there citizen participation in the audit process? scale
citizen and civil society 1-6
2.9. Media coverage of 2.9.1 Does media supply information on issues that are used for audit plan/audit risk?
audit report/findings 2.9.2 Does the media able to scrutinize the audit done by BPK?
62
CHAPTER III
RESEARCH METHOD
3.1 Research Approach

According to Creswell & Creswell (2018), a research approach comprises three
elements, which are: (1) Paradigms, (2) Research design, and (3) Research Method. This
particular study research departs from a positivist perspective with a quantitative approach,
measuring objective facts, and focusing on variables using statistical analysis. A key
characteristic of the positivist paradigm is the formulation of causal hypotheses from an
existing theory followed by quantitative data collection and analysis to support or reject
the hypotheses. However, a theory in the context of scientific research is a framework of
concepts that have been tested but may not always be directly measurable (Gregor, 2006).
Thus, their validity and applicability are assessed through empirical testing and
measurement validation processes.
The aims to prove theory through deductive strategies need to be translated in the
form of operationalizing concepts. When conducting measurement validity in theories,
Fawcet (2021) reminds the importance of ensuring the definition of the theory concept
measured by the empirical indicator aligns with the conceptual model guiding the research.
Therefore, the operational concepts matrix is essential to deduce the theory into variables,
dimensions, and indicators so that it can be measured and used as the basis for making
questionnaires.
Consequently, this research employs a causal point of view which investigates
evidence of cause-and-effect relationships which is particularly beneficial in investigations
that seek to validate hypotheses and determine patterns of independent and dependant
variables. Whereas in this study, government oversight is identified as the independent
variable, and audit impact is reserved as the dependent variable. The hypothesis to be tested
is the influence of government oversight on audit impact.
This research falls into the category of pure research that may not have a direct
practical contribution but remains useful for advancing knowledge on social phenomena.
This research utilizes a rapid and cost-effective method of cross-sectional study to provide
a snapshot of a population at a specific moment, enabling the researcher to observe
phenomena and produce results efficiently (Sedgwick, 2014). Based on the time of data
collection, analysis will be drawn from data collection at a single point in time (Wang &
Cheng, 2020) namely at the time the research is conducted from April to May 2024.
63
3.2 Data Collection Technique

The research utilized primary data for statistical analysis. Primary data is the data
obtained directly from the respondent Sekaran & Bougie (2016). The data collection
method adopted is a structured survey using a questionnaire filled out directly by the
respondent. The items of questions presented are an elaboration of each indicator derived
from the variables measured in the hypothesis. There are two types of questionnaires
chosen for this study. The first one is close-ended questions to collect information about
respondents’ background profiles, such as age, gender, education, etc. The second type is
a fix-response alternative type where answer choices have been provided and respondents
are asked to select them. The questionnaire will utilize a Likert scale of 1 to 6 as alternative
answers. where 1. strongly disagree (STS), 2. disagree (TS), 3. somewhat disagree (ATS),
4. somewhat agree (AS), 5. agree (S), 6. strongly agree (S). (S), 6. Strongly agree (SS). To
avoid confusion, translation losses, and bias in the response, the questionnaire uses Bahasa
Indonesia because the target respondents spoke Bahasa Indonesia.
There will be different questionnaires conducted for each of the variables as it is
intended to be distributed to different groups of respondents. The first group of respondents
for government oversight will be targeted towards auditors of BPK as they are suitable for
indicators and question items proposed in measuring the degree of government oversight
done by BPK from the internal perspective of the organization. The second group of
respondents will be targeted toward auditees or government officials who have
experienced audits done by BPK to measure audit impact as perceived by external entities
or subject of audit.
3.3 Population and Sampling
a. Population
Population in the context of research refers to a group or collection of all
individuals, objects, or events that have certain characteristics and are the focus of a
study. It includes all elements relevant to the research questions and objectives of the
research being carried out. This research considers individual respondents as the unit of
analysis. In connection with the two target groups for the questionnaire, in this study
there are also two categories of population and sample which are explained as follows:
1. The population of the government supervision questionnaire is BPK RI skilled
auditors who have carried out all types of audits (financial, performance and
compliance) and are involved in monitoring recommendations for follow-up audits.
The population of the government supervision questionnaire is 8,400 employees.
64
2. The population of the audit impact questionnaire is government officials who are
experienced with all types of audits carried out by BPK RI and have been involved
in monitoring audit follow-up recommendations. The population of the
examination impact questionnaire is unknown.
b. Sampling
A sample is a part of a population that has certain characteristics or conditions to
be studied. Samples can also be defined as members of a population selected using
certain procedures so that they are expected to represent the population. In quantitative
research, the sample is a very crucial issue that can determine the validity of research
results. In this research, researchers used a probability sampling technique, namely a
sampling technique that provides equal opportunities for all members of the population
to be selected as sample members. The research used a stratified random sampling
technique. The stratified random sampling technique is a sample determination
technique that is used when the population has members or elements that are not
homogeneous and proportionally stratified. Therefore, in this sampling the formula is
used:
1. The government supervision questionnaire sample is a skilled BPK RI auditor who
has carried out all types of audits (financial, performance and compliance) and is
involved in monitoring recommendations for follow-up audits using the solvin
formula:
N
n=
1 + N(e)2
Information:
n = Number of samples searched
N = Number of population
e = Tolerated margin of error
The population of the government supervision questionnaire is 8,400
employees. so by using the Slovin formula with an error rate of 10% as follows:
N
Sample: n = 1+N(e)2
8.400
n=
1 + 8.400(10%)2
8.400
n=
1 + 8.400(0,1)2
65
8.400
n=
1 + 84
8.400
n=
85
n = 98,8 round to 99
From the Slovin formula above, the number of samples used in this study was
99 people. To avoid errors in data collection, the number of samples was rounded up
to 99 samples (respondents)
2. The audit impact questionnaire samples are government officials who are
experienced with all types of audits carried out by BPK RI and have been involved
in monitoring audit follow-up recommendations. To determine the research sample
size from the population, researchers used the Lemeshow formula, which uses a
standard error rate of 10%. Researchers use the Lemeshow formula because the
population size is not yet known with certainty.
Lemeshow Formula:
Z2 .p.(1−p) (1,96)2 .0,5.(1−0,5) 0,9604
N= = = = 96,04
d2 (0,10)2 0,01
Information:
n = Size/Number of Samples required
Z = Z score at 95% confidence or (1.96)
p = Maximum estimate 0.5
d = Alpha (0.10) or sampling error used is 10%
From the results of calculations using the lemeshow formula above, a good
number of samples can be obtained, namely 96.04 which can be rounded up to 96
samples. Therefore, the number of samples for this research was 96 respondents.
Considering the homogeneous background and general knowledge of targeted
sampling towards the questions asked in the questionnaire, this study chose
snowballing sampling using a Google Forms web-based questionnaire because it is
efficient in collecting large data in a short time.
After the results of the statistical analysis were obtained, interviews were
conducted with respondents who were considered relevant and credible to confirm the
results. The information obtained is only confirmatory and does not influence the
results of the statistical analysis that has been obtained. Confirmation is carried out to
obtain an explanation that supports the results of statistical analysis.
66
Therefore, researchers chose sources who were considered to have knowledge

and experience related to government supervision and audit impact assessment. The
selection is based on BPK RI Regulation Number 1 of 2019 concerning Organization
and Implementation Work Procedures. The respondents selected as sources of
confirmation are presented as follows, but it should be noted that adjustments may be
made depending on the required information related to the results of the statistical
analysis:
Table 3.1 List of Respondents for Confirmation
Name Position Roles
Flora Anita Diassari Inspektorat Pemeriksaan Internal Conduct inspection and obtain quality
M.M., Ak., CSFA, dan Mutu Kelembagaan assurance on institutional performance within
CFrA the BPK
Yuan Candra Djaisin Evaluasi dan Pelaporan Compiler of IHPS and Audit Evaluation Report,
S.E., M.M., Ak., CPA., Pemeriksaan Keuangan Negara compiles monitoring data on follow-up of audit
CSFA Direktorat Utama Perencanaan, results including state/regional losses, and
Evaluasi, dan Kebijakan drafts opinion materials based on the audit
Pemeriksaan Keuangan Negara report.
3.4 Data Analysis Technique

Data Analysis Technique was carried out using the following statistical methods:
3.4.1 Descriptive Analysis
Descriptive statistics is used to describe the data collected without any
intention to conclude or generalize the data (Black, 2010). The descriptive statistics
is useful in presenting background information such as frequencies, means, and
standard deviations (Creswell & Creswell, 2018). The data can be presented in
tables, graphs, pie charts, and pictograms as well as calculations of median, decile,
percentile, and percentage (Sugiyono, 2016).
3.4.2 Validity and Reliability Test
Valid and reliable instruments are necessary for quantitative research to
produce valid and reliable data as well (Creswell & Creswell, 2018; Sugiyono,
2016). Therefore, a pre-test is needed to ensure the reliability and validity of the
statements made in the questionnaire. Pre-test is conducted collecting data from a
questionnaire distributed to non-sample respondents to make sure the questionnaire
is well structured, and the instructions are clear and easy to follow.
The pre-tests will be done twice to accommodate the two targeted
populations of government and audit impact respectively. There will be 25 item
questions for the government oversight variable questionnaire and 10 item questions
for the audit impact variable questionnaire along with 5 additional questions
regarding the background of the respondent.
67
The pre-test results obtained were then tabulated to be tested for validity
and reliability. An instrument is considered valid when it can measure things that
should be measured (Black, 2010). Thus, the validity test was conducted to test
whether the questionnaire as the instrument used in this research has the capacity to
accurately measure the indicators given. Meanwhile, an instrument is reliable when
it retains consistent answers from respondents, demonstrating the same result when
measuring the same object over time (Black, 2010). The reliability test is carried out
to test the consistency of the questionnaire in measuring the same object out using
IBM SPSS Statistic software version 25.
i. Validity is a tool used to measure whether a questionnaire is valid or not. A
questionnaire is said to be valid if the questions in the questionnaire are able to
reveal something that the questionnaire will measure (Ghazali). To carry out a
validity test, look at the Item-Total Statistics table. This value is compared with
the calculated r value. If the rcount value > rtable value then it is said to be valid, and
vice versa.
ii. Reliability is the consistency or stability of the scores of a research instrument on
the same individual, and given at different times (Yusuf, 2014). Wrightstone wrote
that reliability is an estimate of the degree of consistency or stability between
repeated measurements and the first 61 measurements using the same instrument.
So, an instrument is said to be reliable if the instrument is tested on the same
subjects repeatedly but the results remain the same or tend to be the same. The
technique used for this reliability test is the Cronbach's Alpha technique, where a
questionnaire is considered reliable if Cronbach's Alpha is > 0.60.
3.4.3 Classic Assumption Test
a. Normality Test
The assumption of normality is the assumption that the residuals (εi) are normally
distributed, so that the normality test is carried out on the residuals of the
regression model. To identify violations of the normality assumption, the
Kolmogorov-Smirnov test, Anderson-Darling test, Shapiro-Wilk test, and Jarque-
Bera test can be used. The cause of not fulfilling the normality assumption is
because there are outliers in the data and it is possible that the data is not normally
distributed or has other distributions, such as exponential, gamma, etc. (Sarwono,
2018). Researchers used the Kolmogorov-Smirnov test to determine violations of
the normality assumption. Kolmogorov-Smirnov rule, namely:
68
1. If the Kolmogorov-Smirnov probability > 0.05 then the residual is normally

distributed
2. If the Kolmogorov-Smirnov probability < 0.05 then the residual is not
normally distributed
b. Multicollinearity
Test Multicollinearity is a condition where there is a perfect or definite linear
relationship between some or all of the independent variables of the multiple
regression model. In a broad sense, this means that there is a high correlation
between the independent variables. In econometric models, symptoms of
multicollinearity often appear because variables in the economic field are in fact
interconnected. To see whether there are symptoms of multicollinearity, you can
use the correlation value between variables. (Sarwono, 2018). Multicollinearity
Detection:
1. Test the correlation value between independent variables, if the correlation
value is the same then there is multicollinearity.
2. Test the VIF value, if each independent variable has a centered VIF value > 10
then there is multicollinearity.
c. Heteroscedasticity Test
The heteroscedasticity test aims to test the regression model whether there is an
inequality of variance between one residual and another observation (Ghazali,
2013). If the variance of the residuals from observations is constant, it is called
homoscedasticity and if it is different, it is called heteroscedasticity. A good
regression model is a model where heteroscedasticity or homoscedasticity does
not occur. To detect heteroscedasticity, the Glesjer test, White test, Breusch Pagan
test, Harvey test, Park test, and others can be carried out. Researchers use
scatterplot image patterns to detect heteroscedasticity. Rule for detecting
heteroscedasticity in scatterplot image patterns is that if the points are spread
randomly and are spread both above and below the number 0 on the Y axis, then
hetroscedasticity does not occur.
3.4.4 Hypothesis and Regression Analysis
a. Multiple Linear Regression Analysis
Multiple linear regression analysis is a tool for predicting the value of the
influence of two or more independent variables on the dependent variable with
the aim of proving whether or not there is a functional relationship or normal
69
relationship between two or more independent variables and one dependent

variable. Thus the multiple linear regression formula is as follows:
Y = β0 + β1X1+ β2 X2 + β3 X3 + β4 X4 + β5 X5 + β6 X6 + β7 X7 + e
Information :
Y = Government supervision
β0 = Constant
β1-β2 = Regression Coefficient
X1 = SAI institutional mandate
X2 = Quality of audit work
X3 =Legislative supervision
X4 = Executive response
X5 = Independent follow-up
X6 = Community participation
X7 = Audit impact
e = Error
b. Coefficient of determination test (R2)
The coefficient of determination is used to determine the extent to which the
suitability or quality of the regression line formed represents the group of
observational data. The coefficient of determination describes the portion of the
total variation that can be explained by the model. The greater the value (closer
to 1), the better the accuracy. The nature of determining the coefficient is as
follows:
Value 0 ≤ R2 ≤ 1
1. R2 = 0, meaning that the regression model formed is not appropriate for
predicting the value of the dependent variable, this is because there is no
relationship between the independent variable and the dependent variable.
2. R2 = 1, meaning that the regression model formed can predict the value of the
dependent variable perfectly.
c. Partial Significance Test (t)
Partial testing is used to test whether the independent variable has a significant
influence on the dependent variable or not. The number of individual tests is 65
the same as the number of independent variables (Sarwono, 2018).
The hypothesis of this individual testing is as follows:
H0 : βJ ≠ 0, j = 1,2,3,....,p
70
To test this hypothesis, statistics are used with the following decision-making
criteria:
1. If the significant value (P value) <0.05 then H0 is rejected and Ha is accepted,
which means there is an influence between the independent variable and the
dependent variable.
2. If the significant value (P value) is > 0.05 then H0 is accepted and Ha is
rejected, which means there is no influence between the independent variable
and the dependent variable.
3. Simultaneous Significance Test (f)
The Simultaneous Significance test can also be called a simultaneous test.
Regression parameters or coefficients are tested simultaneously using ANOVA
(Analysis of Variance), to find out whether the resulting regression model is
suitable or whether the independent variables simultaneously have an influence on
the dependent variable (Sarwono, 2018). The assumption of normality is the
assumption that the residuals (εi) are normally.
3.5 Research Limitation
Limitations in conducting the research are divided into methodological
limitations and contextual limitations. Methodological limitations occurred with the Likert
scale. As Heo et al. (2022) have pointed out, respondents may have differing assumptions
on what constitutes the quality asked and fail to choose the appropriate gap between one
scale and another, thus resulting in the tendency of respondents to avoid extreme scales.
This issue is mitigated by conducting interviews with respondents and the results of the
interviews were used to compare and confirm the results of the questionnaires. The
contextual limitations are the variables and locus of this research. The results and argument
assessed will be based on the answers from the sample group. The locus of research in
Indonesia specifically BPK as the Supreme Audit Institution of Indonesia limits the extent
to which the researcher can analyse and draw conclusions on the subject.
References
Ahlenius, Inga-Britt. (2000). Performance audits, evaluations and supreme audit

institutions. International Journal of Government Auditing, 27(1), 1-2.
https://doi.org/10.1111/1468-0408.00123
Akbar, F. G., Aswar, K., & Lastiningsih, N. (2020). Determinants of audit quality in
Indonesia supreme audit institution: a conceptual study. Information Management
and Business Review, 12(2), 38-41.
Akyel, Recai &Kose, H. Omer (2013), Auditing and governance: importance of citizen
participation and the role of supreme audit institutions to enhance democratic
governance. Journal of Yasar University 8(32), 5495-5514.
Alkam, R., & Bastian, I. (2019). Analysis of repeated findings in the audit of public funding
of the regional representative council (DPD) of the political parties in the province
of yogyakarta special region. Accounting and Business Information System
Journal, 1-18. doi:https://doi.org/10.22146/abis.v7i1.58826.
Alwardat, Y. A. (2010). External auditors and clients: An investigation of perceptions of
Value for Money (VfM) audit practices in the UK public sector (Doctoral thesis,
Harrow Business School). University of Westminster, London, England.
Alwardat, Y. A., & Basheikh, A. M. (2017). The impact of performance audit on public
administrations in Saudi Arabia: an exploratory study. International Journal of
Business and Management, 12(12), 144-157. doi:10.5539/ijbm.v12n12p144.
Ammons D. N. (2012). Municipal benchmarks: Assessing local performance and
establishing community standards (3rd ed.). New York, NY: Routledge.
Anderson, L. (2021). Understanding evaluation's influence on stakeholders' attitudes and
actions at a social services organization (Doctoral dissertation). University of
Minnesota, USA.
Andrianto, N., Sudjali, I. P., & Karunia, R. L. (2021). Assessing the development of
performance audit methodology in the supreme audit institution: The case of
Indonesia. Jurnal Tata Kelola dan Akuntabilitas Keuangan Negara, 7(1), 19-37.
Asis, L. D. (2017). Analisis tindak lanjut rekomendasi dan temuan audit berulang badan
pemeriksa keuangan atas laporan keuangan pemerintah kabupaten wakatobi (studi
kasus pada pemerintah kabupaten wakatobi). Yogyakarta: Universitas Gadjah
Mada.
Aswar, K., Akbar, F. G., & Lastiningsih, N. (2020). Determinants of audit quality in
Indonesia supreme audit institution: A conceptual study. Information Management
and Business Review, 12(2(I)), 38-41.
https://doi.org/10.22610/imbr.v12i2(i).3075.
Aucoin, P., & Heintzman, R. (2000). The dialectics of accountability for performance in
public management reform. International Review of Administrative Sciences,
66(1), 45-55. https://doi.org/10.1177/0020852300661005.
Aziz, M. A. A., Rahman, H. A., Alam, M. M., & Said, J. (2015). Enhancement of the
accountability of public sectors through integrity system, internal control system
and leadership practices: A review study. Procedia Economics and Finance, 28,
163–169. doi:10.1016/s2212-5671(15)01096-5.
Azuma, N. (2005). The role of the supreme audit institutions in new public management
(NPM): The trend of continental countries. Government Auditing Review, 12, 69-
84.
Azzam, T., & Whyte, C. E. (2018). Evaluative feedback delivery and the factors that affect
success. Evaluation and Program Planning, 67, 148–159.
https://doi.org/10.1016/j.evalprogplan.2017.11.005.
Bastoe, P. O. (2006) Implementing Results-Based Management, in R. C. Rist and N. Stame
(eds) From Studies to Streams: Managing Evaluative Systems, pp. 97–110.
London: Transaction Publishers.
Bertrand, M., Djankov, S., Hanna, R., & Mullainathan, S. (2007). Obtaining a driver's
license in India: An experimental approach to studying corruption. The Quarterly
Journal of Economics, 122(4), 1639-1676.
https://doi.org/10.1162/qjec.2007.122.4.1639.
Black, K. (2010). Business statistics for contemporary decision making (6th ed.). John
Wiley & Sons, Inc.
Blann, J., Kleppe, T., & Shipman, J. (2022). Did the PCAOB's 2009 office expansion
improve audit quality?. Contemporary Accounting Research, 40(1), 89-119.
https://doi.org/10.1111/1911-3846.12824.
Blalock, A. B. (1999). Evaluation research and the performance management movement:
From estrangement to useful integration?. Evaluation 5(2): 117–49.
Bovens, M. (2007). Analysing and assessing accountability: A conceptual framework.
European Law Journal, 13, 447-468. https://doi.org/10.1111/j.1468-
0386.2007.00378.x.
Boyne, G. A. (2003). Sources of public service improvement: A critical review and
research agenda. Journal of public administration research and theory, 13(3), 367-
394.
BPK. (2020, July 21). Perbaikan dan peningkatan SPI dilakukan agar temuan berulang
tidak terjadi BPK RI: Jakarta. https://www.bpk.go.id/news/perbaikan-dan-
peningkatan-spi-dilakukan-agar-temuan-berulang-tidak-terjadi.
BPK. (2020). Rencana strategis 2020-2024. Jakarta, Indonesia: Badan Pemeriksa
Keuangan.
BPK. (2022a). Ikhtisar hasil pemeriksaan semester II. Jakarta, Indonesia: Badan Pemeriksa
Keuangan.
BPK. (2022b, July 13). BPK Temukan Permasalahan Berulang Dalam LHP Atas LK
Kementerian PUPR Dan Kementerian Pertanian. BPK RI: Jakarta.
https://www.bpk.go.id/news/bpk-temukan-permasalahan-berulang-dalam-lhp-
atas-lk-kementerian-pupr-dan-kementerian-pertanian.
BPK. (2023, July 10). Temuan signifikan dan berulang, anggota bpk soroti
pertanggungjawaban perjalanan dinas K/L. BPK RI: Jakarta.
https://www.bpk.go.id/news/temuan-signifikan-dan-berulang-anggota-bpk-soroti-
pertanggungjawaban-perjalanan-dinas-kl.
Brétéché, B., & Swarbrick, A. (2017, June 13). Developing effective working relationships
between supreme audit institutions and parliament. Support for Improvement in
Governance and Management (SIGMA), 1-228.
doi:https://doi.org/10.1787/d56ab899-en.
Budiman, M. A., & Amyar, F. (2021). The effect of audit opinions, implementation of audit
recommendations, and findings of state losses on corruption levels within
ministries and institutions in the Republic of Indonesia. Jurnal Tata Kelola dan
Akuntabilitas Keuangan Negara, 7(1), 113-129.
https://doi.org/10.28986/jtaken.v7i1.471.
Bussing, A. and Pomirchy, M. (2022). Congressional oversight and electoral
accountability. Journal of Theoretical Politics, 34(1), 35-58.
https://doi.org/10.1177/09516298211061516.
Campos, E. J. (2001, October). Public expenditure management (PEM). Asian
Development Bank: https://www.adb.org/publications/what-public-expenditure-
management-pem.
Carlson, V. P., Chilton, M. J., Corso, L., & Beitsch, L. M. (2015). Defining the functions
of public health governance. American Journal of Public Health, 105(S2), S159-
S166. https://doi.org/10.2105/ajph.2014.302198.
Christensen, L. (2022). Internal audit: A case study of impact and quality of an internal
control audit. International Journal of Auditing, 26(3), 339–353.
https://doi.org/10.1111/ijau.12280.
Christensen, T., Lægreid, P., Roness, P. G., & Røvik, K. A. (2007). Organization theory
and the public sector: Instrument, culture and myth. London, England: Routledge.
Cordery, C. J., & Hay, D. C. (2022). Public sector audit in uncertain times. Financial
accountability & management, 38(3), 426-446.
https://doi.org/10.1111/faam.12299.
Cordery, C. J., & Hay, D. (2018). Supreme audit institutions and public value:
Demonstrating relevance. Financial Accountability & Management, 35(2), 128-
142. https://doi.org/10.1111/faam.12185.
Coryn, C. L. (2006). A conceptual framework for making evaluation support meaningful,
useful and valuable. Evaluation Journal of Australasia, 6(1), 45-51.
https://doi.org/10.1177/1035719X0600600107
Creswell, J. W., & Creswell, J. D. (2018). Research design: qualitative, quantitative, and
mixed methods pproaches (5th ed.). Sage Publications.
Curristine, T., & Flynn, S. (2013). In search of results: Strengthening public sector
performance. Public financial management and its emerging architecture, 225-258.
International Monetary Fund (IMF). Washington: USA.
https://doi.org/10.5089/9781475531091.071
Davies, I. C. (1999). Evaluation and performance management in
government. Evaluation, 5(2), 150-159. Sage Publications.
Deis Jr, D. R., & Giroux, G. A. (1992). Determinants of audit quality in the public
sector. Accounting review, 462-479.
De las Heras, E., Cañibano, L., & Moreira, J. A. (2012). The impact of the spanish financial
act (44| 2002) on audit quality. Spanish Journal of Finance and Accounting/Revista
Española de Financiación y Contabilidad, 41(156), 521-546.
De Lancer Julnes, P. (2006). Performance Measurement: An Effective Tool for
Government Accountability? The Debate Goes On. Evaluation, 12(2), 219-235.
https://remote-lib.ui.ac.id:2075/10.1177/1356389006066973
Denhardt J. V. and R. B. Denhardt. (2003). The new public service; serving, not steering.
New York: M.E. Shape.
Department for International Development (DFID). (2005, July). How to note: Working
with supreme audit institutions. Part of the Policy Division Info series. Ref. No:
PD Info 079.
Dunn, W. N. (2015). Public policy analysis: An integrated approach. Routledge.
Edler, J., Karaulova, M., & Barker, K. (2022). Understanding conceptual impact of
scientific knowledge on policy: The role of policymaking
conditions. Minerva, 60(2), 209-233.
FAI. (2021, November 16). Facilitating audit impact strategy 2021-2023. INTOSAI
Development Initiative.
Fawcett, J. (2021). Empirical indicators: Conceptual and theoretical origins. Aquichan,
21(4), 1-12. https://doi.org/10.5294/aqui.2021.21.4.4
Ferdousi, F. and Qiu, L. (2013). New public management in Bangladesh: Policy and reality.
iBusiness, 05(03), 150-153. https://doi.org/10.4236/ib.2013.53b032.
Friedberg, A. and C. Lutrin (2005), State Audits in the United States, 1996-2000. Journal
of Public Budgeting, Accounting & Financial Management, Vol. 1 7, No. 1, pp. 1-
32.
Frost, N. (2021). The myth of measurement: inspection, audit, targets and the public sector.
The Myth of Measurement, 1-100.
Ghozali, Imam. (2013). Aplikasi Analisis Multivariate dengan Program IBM SPSS 21
Update PLS Regresi. Semarang: Badan Penerbit Universitas Diponegoro.
Gregor, S. (2006). The nature of theory in information systems. MIS Quarterly, 30(3), 611.
https://doi.org/10.2307/25148742.
Greene, J. C. (1999). The Inequality of Performance Measurements. Evaluation, 5(2), 160-
172. https://remote-lib.ui.ac.id:2075/10.1177/13563899922208904.
Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, Marko. (2017). A primer on partial
least squares structural equation modeling (PLS-SEM) (2nd ed.). Sage
Publications, Inc.
Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to use and how to
report the results of PLS-SEM. European business review, 31(1), 2-24.
Hay, D., & Cordery, C. (2018). The value of public sector audit: Literature and history.
Journal of Accounting Literature, 40, 1-15.
doi:https://doi.org/10.1016/j.acclit.2017.11.001
Hemici, F., & Kontogeorga, G. (2020). Performance measurement in public auditing and
challenges for supreme audit institutions (SAIs). Journal of Turkish Court of
Accounts/Sayistay Dergisi, 32(117).
Heo, M., Kim, N., & Faith, M. (2015). Statistical power as a function of cronbach alpha of
instrument questionnaire items. BMC Medical Research Methodology, 15.
https://doi.org/10.1186/s12874-015-0070-6.
Hepworth, N. (2023). Public financial management and internal control the importance
of managerial capability for successful reform in developing and transition
economies. Cham: Palgrave Macmillan. doi:https://doi.org/10.1007/978-3-031-
35066-5.
Huber, G. P. (1991). Organizational learning: The contributing processes and the
literatures. Organization Science, 2, 88–115. https://doi.org/10.1287/orsc.2.1.88
IBP. (2021). Open budget survey 2021. International budget partnership. Retrieved from
International Budget Partnership.
IDI. (2022, October). Supreme Audit Institutions Performance Measurement Framework.
INTOSAI Development Initiative.
IDI. (2012, May 20). Mapping of Tools for Assessing Performance of Supreme Audit
Institutions. INTOSAI Development Initiative.
IDI. (2021a, September 15). Global SAI stocktaking report 2020. INTOSAI Development
Initiative.
IDI. (2021b, December 21). Reimagining SAI audit impact. INTOSAI Development
Initiative
IDI. (2020). All hands on deck: harnessing accountability. INTOSAI Development
Initiative.
IDI. (2019, July 8). About SAI PMF. INTOSAI Development Initiative.
IDI. (2020). Mapping of Tools for Assessing Performance of Supreme Audit Institutions.
INTOSAI Development Initiative.
Igibayeva, Z. (2020, March). The impact of internal public audit on the use of public
resources. Advances in Economics, Business and Management Research. In
International Scientific Conference Far East Con (ISCFEC 2020) (pp. 3337-3341).
Atlantis Press. https://doi.org/10.2991/aebmr.k.200312.482
INTOSAI. (2013, October 25). INTOSAI-P 12: the value and benefits of supreme audit
institutions – making a difference to the lives of citizens. SAI Independence
Research Center.
INTOSAI (2001). Independence of SAIs Project, Final Task Force Report.
Iosefo, S. P. (2022, February 23). Designing audits for impact. Pacific Association of of
Supreme Audit Institutions: https://www.pasai.org/blog/2022/2/23/designing-
audits-for-impact.
Irawan, A. B., and McIntyre-Mills, J. (2016) Application of critical systems thinking to
performance auditing practice at the indonesian supreme audit institution: issues
and challenges. Syst. Res, 33: 24–44. doi: 10.1002/sres.2325.
Irawan, A. B. (2015). Evaluation of performance auditing in Indonesia: a critical systemic
approach to addressing public accountability. Adelaide, Australia: Flinders
University.
Iskandar, T. M., Lasa, Y. M., & Hassan, N. S. A. (2014). Financial management
performance of public sector: quality of internal auditor. International Journal of
Accounting, Auditing and Performance Evaluation, 10(3), 229.
doi:10.1504/ijaape.2014.064233.
Johnsen Å, Reichborn-Kjennerud K, Carrington T, Jeppesen KK, Taro K, Vakkuri J. (2018)
Supreme audit institutions in a high-impact context: A comparative analysis of
performance audit in four Nordic countries. Financial Acc & Man. 2019; 35: 158–
181. https://doi.org/10.1111/faam.12188.
Kertarajasa, A. Y., Marwa, T., & Wahyudi, T. (2019). The effect of competence, the effect
of competence, experience, independence, due professional care, and auditor
integrity on audit quality with auditor ethics as moderating variable. Journal of
Accounting Finance and Auditing Studies (JAFAS), 5(1), 80-99.
https://doi.org/10.32602/jafas.2019.4.
Ketelaar, A., N. Manning and E. Turkisch (2007), "Performance-based Arrangements for
Senior Civil Servants OECD and other Country Experiences", OECD Working
Papers on Public Governance, No. 5, OECD Publishing, Paris,
https://doi.org/10.1787/160726630750.
Khotami, M. (2017, November). The concept of accountability in good governance.
In International Conference on Democracy, Accountability and Governance
(ICODAG 2017) (pp. 30-33). Atlantis Press.
Kock, N., & Hadaya, P. (2018). Minimum sample size estimation in PLS‐SEM: The
inverse square root and gamma‐exponential methods. Information systems
journal, 28(1), 227-261.
Kontogeorga, Georgia. (2013). Adapting business practices to the SAIs environment:
Towards a new performance measurement framework. Journal Cour des Comptes
Européenne, May, No 5. 2-4.
Kurniawati, A. D., & Pratama, Y. M. (2021). The role of government auditing in controlling
the level of corruption in Indonesia. Jurnal Akuntansi dan Auditing Indonesia,
25(1). https://doi.org/10.20885/jaai.vol25.iss1.art6.
Lægreid, P. (2014). Accountability and the new public management. In R. E. M. Bovens,
M. Bovens, R.E. Goodin, & T. Schillemans (Eds.), The Oxford handbook of public
accountability (pp. 324–338). Oxford: Oxford University Press.
Lapsley, I. (2009). New public management: The cruellest invention of the human spirit?
Abacus, 45, 1–21. https://doi.org/ 10.1111/j.1467-6281.2009.00275.x.
Leeuw, F.L. (1996). Performance auditing, new public management and performance
improvement: Questions and answers. Accounting, Auditing & Accountability
Journal, Vol. 9 No. 2, pp. 92-102.
Lin, T., Hutchinson, M., & Percy, M. (2013). Earnings management and the role of the
audit committee: an investigation of the influence of cross-listing and government
officials on the audit committee. Journal of Management and Governance, 19(1),
197-227. https://doi.org/10.1007/s10997-013-9284-3.
Lonsdale, J. (2000). Developments in value-for-money audit methods: Impacts and
implications. International Review of Administrative Sciences, 66(1), 73-89.
https://doi.org/10.1177/0020852300661007.
Mackay, Keith. 2006. Institutionalization of Monitoring and Evaluation Systems to
Improve Public Sector Management. ECD Working Paper Series: No. 15. World
Bank, Washington, DC. http://hdl.handle.net/10986/36708 License: CC BY 3.0
IGO
Mansouri, A., Pirayesh, R., & Salehi, M. (2009). Audit competence and audit quality: Case
in emerging economy. International Journal of Business and Management, 4(2),
17-25.
Mayne, J. (2007). Challenges and Lessons in Implementing Results-Based Management.
Evaluation, 13(1), 87–109. doi:10.1177/1356389007073683.
McDavid, J. and L. R. L. Hawthorn (2006) Program Evaluation and Performance
Measurement: An Introduction to Practice. New York and Thousand Oaks, CA:
Sage publication.
Mihret, D. G., James, K., & Mula, J. M. (2010). Antecedents and organisational
performance implications of internal audit effectiveness: Some propositions and
research agenda. Pacific Accounting Review, 22(3), 224-252.
10.1108/01140581011091684
Mkasiwa, T. A. (2019). Reforms and budgetary oversight roles in Tanzania. Journal of
Public Budgeting, Accounting and Financial Management, 32(1), 1-25.
https://doi.org/10.1108/jpbafm-12-2018-0155
Moore, M. H. (1995). Defining public value. In Creating public value: Harvard University
Press.
Morin, D. (2014). Auditors general’s impact on administrations: A pan-Canadian study
(2001–2011). Managerial Auditing Journal, 29, 395-426. doi:10.1108/maj-10-
2013-0948.
Morin, D. (2008). Auditors general’s universe revisited: An exploratory study of the
influence they exert on public administration through their value for money audits.
Managerial Auditing Journal, Vol. 23 No. 7, pp. 697-720.
Morin, D. (2001). Influence of value for money audit on public administrations: Looking
beyond appearances. Financial Accountability & Management, Vol. 17, No. 2, pp.
99-117.
Mulgan, R. (2000). ‘Accountability’: an ever‐expanding concept?. Public
administration, 78(3), 555-573.
Munda, G. (2024). Social Multi-Criteria Evaluation for a Sustainable Economy. Berlin:
Springer. https://doi.org/10.1007-978-3-540-73703-2.
Munda, G. (2004). Social multi-criteria evaluation: Methodological foundations and
operational consequences. European Journal of Operational Research, 158(3),
662–677. https://doi.org/10.1016/s0377-2217(03)00369-2.
Nagy, Sandor. (2012). The role of Supreme Audit Institutions. Munich Personal RePEc
Archive (MPRA), 270-282.
Nau, T., Lee, K. C., Smith, B. J., Bellew, W., Reece, L., Gelius, P., Rutter, H & Bauman,
A. (2019). Toward whole-of-system action to promote physical activity: a cross-
sectoral analysis of physical activity policy in Australia. Journal of Physical
Activity and Health, 16(11), 1029-1038. https://doi.org/10.1123/jpah.2019-0122.
Nicoll, P. (2017). Creating good and clean government: performance audit in Indonesia.
Australia: Amazon.
Nielsen, S. B., & Ejler, N. (2008). Improving performance? Exploring the
complementarities between evaluation and performance
management. Evaluation, 14(2), 171-192.
https://doi.org/10.1177/1356389007087538.
Octaviani, M. (2018). Studi eksploratif terhadap temuan pemeriksaan BPK RI atas sistem
pengendalian internal kabupaten/kota di provinsi sumatera barat. Padang:
Universitas Negeri Padang.
Organisation for Economic Cooperation and Development (OECD). (2016). Supreme
audit institutions and good governance: Oversight, insight and foresight. OECD
Publishing. https://doi.org/10.1787/9789264263871-en
OECD. (2011). Good practices in supporting supreme audit institutions. Organisation for
Economic Cooperation and Development.
Secretary Cabinet. (2016, Oktober 25). Committed to achieving SDGs, President Jokowi
reminds the importance of local solutions. Office of Assistant to Deputy Cabinet
for State Documents & Translation. Jakarta, Indonesia.
Oţetea, Alexandra., Batusaru, Cristina Maria Tiţa., Ungureanu, Mihai Aristotel. (2015).
The performance impact of the supreme audit institutions on national budgets, great
britain and romania case – comparative study. Procedia. Economics and
Finance, 27, 621–628. https://doi.org/10.1016/s2212-5671(15)01042-4.
Pamungkas, B., Avrian, C., & Ibtida, R. (2019). Factors influencing audit findings of the
Indonesian district governments’ financial statements. Cogent Business &
Management, 6(1), 1673102. https://doi.org/10.1080/23311975.2019.1673102.
Parker, L. D., Jacobs, K., & Schmitz, J. (2019). New public management and the rise of
public sector performance audit: Evidence from the Australian case. The rise of
public sector performance audit. Accounting, Auditing & Accountability
Journal, 32(1), 280-306. doi:https://doi.org/10.1108/AAAJ-06-2017-2964.
Paudel, R. C., & Gupta, A. K. (2019). Determinants of accountability in the bureaucracy:
the case of Nepal. Modern Economy, 10(9), 2085-2109.
Patton, M. Q. (2008). Utilization-focused evaluation. Sage publications.
Patton, M.Q. (2003). Utilization-focused evaluation. International Handbook of
Educational Evaluation. International Handbooks of Education, vol 9. Dordrecht:
Springer. https://doi.org/10.1007/978-94-010-0309-4_15.
Patton, M.Q. (2000). Utilization-focused evaluation. Evaluation Models: Evaluation in
Education and Human Services, vol 49. Springer, Dordrecht.
https://doi.org/10.1007/0-306-47559-6_23.
Perrin, B. (1998). Effective Use and Misuse of Performance Measurement. American
Journal of Evaluation 19(3): 367–79. https://doi.org/10.1016/S1098-
2140(99)80218-5.
Perrin, B. (2006) Moving from outputs to outcomes: practical advice from governments
around the world. Arlington: IBM Center for the Business of Government.
Poister, T. H. (2003). Measuring performance in public and nonprofit organizations. Wiley
& Sons.
Pollitt, C., Girre, X., Lonsdale, J., Mul, R., Summa, H., & Waerness, M. (1999).
Performance or compliance? Performance audit and public management in five
countries. Oxford: Oxford University Press.
Power, M. (1997). The audit society: Rituals of verification. Oxford: Oxford University
Press.
Power, M. (2000). The audit society—Second thoughts. International Journal of Auditing,
4(1), 111-119. https://doi.org/10.1111/1099-1123.00306.
Power, M. (2003). Evaluating the audit explosion. Law and Policy, 25, 185−202.
https://doi.org/10.1111/j.1467-9930.2003.00147.x.
Pramono, A. J., & Wibisono, W. (2022). The Role Of Sai In Strengthening National
Institutions Towards Sdgs Realization: Case Study Of Bpk. Journal of Positive
School Psychology, 6(9), 616-627.
Raudla, R., Taro, K., Agu, C., & Douglas, J. W. (2015). The impact of performance audit
on public sector organizations: The case of Estonia. Public Organization Review,
16, 217–233. https://doi.org/10.1007/s11115-015-0308-0.
Reichborn-Kjennerud, K., Vabo, SI. (2017). Performance audit as a contribution to change
and improvement in public administration. Evaluation: International Journal of
Theory, Research and Practice. 2017;23(1):6-23
http://doi.org/10.1177/1356389016683871.
Reichborn-Kjennerud, K., Johnsen, Åge. (2015). Performance audits and supreme audit
institutions’ impact on public administration: the case of the office of the auditor
general in norway. Administration & Society, Administration & Society
50(10):1422–1446. https://doi.org/ 10.1177/0095399715623315.
Reichborn-Kjennerud, K., Carrington, T., Gonzales, B., & Klarskov, K. (2015). Supreme
audit institutions’ role in fighting corruption: a comparative study between SAIs
with different institutional and public administrative structures and cultures. Paper
presented at the IRSPM workshop Birmingham, Great Britain.
Ríos, A., Bastida, F., & Benito, B. (2014). Budget transparency and legislative budgetary
oversight. The American Review of Public Administration, 46(5), 546-568.
https://doi.org/10.1177/0275074014565020.
Rist, R. C. (2006) ‘The “E” in Monitoring and Evaluation: Using Evaluative Knowledge
to Support a Results-Based Management System’, in R. C. Rist and N. Stame (eds)
From Studies to Streams: Managing Evaluative Systems, pp. 3–22. London:
Transaction Publishers.
Rivenbark, W. C., Fasiello, R., & Adamo, S. (2019). Exploring Performance Management
in Italian Local Government: The Necessity of Outcome Measures and Citizen
Participation. The American Review of Public Administration, 49(5), 545-553.
Sabet, D. M. (2020). Auditing as a tool of government accountability? exploring divergent
causal mechanisms through three honduran cases. Public Administration and
Development, 40(4), 209-219. https://doi.org/10.1002/pad.1895.
Santiso, C. (2015), Why budget accountability fails? The elusive links between
Parliaments and audit agencies in the oversight of the budget. Brazilian Journal of
Political Economy, Vol. 35, No. 3, pp. 601-621.
Sedgwick, P. (2014). Cross sectional studies: advantages and disadvantages. BMJ,
348(mar26 2), g2276-g2276. https://doi.org/10.1136/bmj.g2276.
Sekaran, U., Bougie, R. (2016). Research methods for business: A skill-building
approach (7th ed.). West Sussex: John Wiley & Sons Ltd.
Selin, J. L., & Moore, G. (2023). Keeping tabs on the executive. Presidential Studies
Quarterly.
Sendjaja, D. W., Rismanto, G. Y., & Andrianto, N. (2015). Analisis kebijakan publik dalam
pemeriksaan kinerja. Jurnal Tata Kelola dan Akuntabilitas Keuangan Negara, 1, 67-
81.
Sonnichsen, R. C. (2000). High impact internal evaluation. Sage Publications.
Suchman, M. C. (1995). Managing legitimacy: strategic and institutional approaches. The
Academy of Management Review, 20(3), 571–610.
https://doi.org/10.2307/258788.
Stone, M. E., Ostrower, F. (2007). Acting in the public interest? another look at research
on nonprofit governance. Nonprofit and Voluntary Sector Quarterly, 36(3), 416-
438. https://doi.org/10.1177/0899764006296049.
Stufflebeam, D. L., Coryn, C. L. (2014). Evaluation theory, models, and applications (Vol.
50). John Wiley & Sons.
Suartama, I. K., Nurdin, S., & Susilo, F. S. (2015). Enhancing BPK RI’S Audit Design for
People’s Welfare: A Practical Approach. Jurnal tata kelola & akuntabilitas
keuangan negara, 1(2), 173-190.
Sugiyono. (2016). Metode penelitian kuantitatif, kualitatif, dan R&D. Alfabeta.
Suzuki, Y. (2004). Basic structure of government auditing by a supreme audit
institution. Government Auditing Review, 11(3), 39-53.
Thakkar, J. J. (2020). Structural equation modelling application for research and practice
(with AMOS and R) (Vol. 285). Springer Nature Singapore Pte Ltd.
https://doi.org/https://doi.org/10.1007/978-981-15-3793-6.
Torres, L., Yetano, A., & Pina, V. (2019). Are performance audits useful? A comparison of
European Union (EU) practices. Administration & Society, 51 (3) , 431-462.
https://doi.org/10.1177/00953997166585.
van Loocke, E., Put, V. (2011). The impact of performance audits: a review of the existing
evidence. Performance auditing: contributing to accountability in democratic
government, 175-208. https://doi.org/10.4337/9780857931801.00016.
van Thiel, Leeuw. (2002). The performance paradox in the public sector. Public
Performance & Management Review. Vol. 25 No. 3, pp. 267-281. Sage
Publications.
van Zyl, A., Ramkumar, V., & De Renzio, P. (2009). Responding to the Challenges of
Supreme Audit Institutions: Can legislatures and civil society help?. U4 Issue,
2009(1).
Vedung, E. (2004). Public Policy and Program Evaluation. London: Transaction
Publishers.
Vosawale-Katuba, Meresimani. (2022, January 24). The impact of SAI PMF reports -
modernising SAI legislation in the pacific. The Pacific Association of Supreme
Audit Institutions.
Wang, X., Cheng, Z. (2020). Cross-sectional studies: strengths, weaknesses, and
recommendations. Chest, 158(1), S65-S71.
https://doi.org/10.1016/j.chest.2020.03.012.
Weiss, C. H. (1979). The many meanings of research utilization. Public Administration
Review, 39, 426–431. https://doi.org/10.2307/3109916.
Wholey, J. S. (2007). Monitoring performance in the public sector: Future directions from
international experience. Transaction Publishers.
Wibowo, A. (2019). Promoting development program effectiveness: Expected role of
publik sector audit in state finance management. Jurnal Tata Kelola dan
Akuntabilitas Keuangan Negara (JTAKEN), 103-123.
doi:https://doi.org/10.28986/jtaken.v5i2.358.
Wilkins, P. (1995). Performing Auditors? Assessing and reporting the performance of
national audit offices - a three country comparison. Australian Journal of Public
Administration, Vol. 54, No. 4, pp. 421430. https://doi.org/10.1111/j.1467-
8500.1995.tb01154.x
Zapico-Goñi, E. and J. Mayne. 1997. Performance monitoring: Implications for the future,
in E. Zapico-Goñi and J. Mayne (eds) Monitoring Performance in the Public Sector.
London: Transaction Publishers.
Annex 1
Research Instrument
1. Questionnaire assessing variable Government Oversight aimed for BPK auditors

(internal evaluation) Link Google Form: https://forms.gle/TSs5iwSsxBNPqVAk7
2. Questionnaire assessing variable Audit Impact aimed for auditee or government
officials having been audited by BPK (external evaluation) Link Google Form:
https://forms.gle/BUSbkVMqYeroCycGA

ND 682 Lamp - Anindya Atma Zulatsari - 20240515115514

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ND 682 Lamp - Anindya Atma Zulatsari - 20240515115514

Uploaded by

Copyright:

Available Formats

Gedung Prof. Dr. Mr.

Prajudi Atmosudirjo, Komplek Fakultas Ilmu Sosial dan Politik

Nomor : S-830/UN2.F16.D5/PDP.04.04/2024 20 Maret 2024

Yth. Kepala Biro Humas dan Kerja Sama Internasional

Nama : Anindya Atma Zulatsari

untuk melakukan pengumpulan data sekunder berupa:

Atas perhatian Bapak/Ibu, kami ucapkan terima kasih.

Manajer Pendidikan dan Kemahasiswaan

Dr. Achmad Lutfi, S.Sos., M.Si

Yth. : Kepala Biro Humas dan Kerja Sama Internasional

Nama : Anindya Atma Zulatsari

Untuk melakukan pengumpulan data sekunder berupa:

Atas perhatiannya, kami ucapkan terima kasih.

Dr. Achmad Lutfi, S.Sos., M.Si

Assessing The Influence of Government Oversight on Audit Impact of

Submitted as one of the requirements for

ANINDYA ATMA ZULATSARI

FACULTY OF ADMINISTRATIVE SCIENCE

TABLE OF CONTENTS ................................................................................................ i

Figure 1.1 Strength of SAI Oversight ............................................................................2

Figure 1. 1 Strength of SAI Oversight

Following this ongoing concern, INTOSAI Development Initiatives (IDI) and

Conclusively, the overall global overview on this matter is noted to be

Figure 1. 2 Relative Strength of Oversight Ecosystem in ASOSAI Regional Groups

Oversight of public expenditure management is carried out by Indonesia Audit Board

recommendations suggested by BPK in its audit report, leading to the increased

This phenomenon indicates weakness in government control and how BPK

employed and acknowledged as standard measurement globally is beneficial and

a. What are the driving factors of government oversight?

1.3 Research Purposes and Significances

1.4 Writing Structure

This chapter describes the background, problem

CHAPTER II LITERATURE REVIEW AND THEORETICAL

This chapter discusses the relevant theories used as the

CHAPTER III RESEARCH METHOD

This chapter contains research approaches, types of

CHAPTER IV DISCUSSION OF RESEARCH RESULTS

This chapter discusses the general description of the

CHAPTER V CONCLUSIONS AND RECOMMENDATIONS

This chapter contains conclusions from research results that

2.1 Literature Review

Administrations: A in Auditor’s General Usefulness of auditor’s report, audited entities to

2.1.2 Novelty of the Research

In both theoretical understanding and practical application of audit

Meanwhile, research on Value-for-Money (VfM) audit in Canada by

accountability, however, the Parliament little attention on audit

While an increasing number of social scientists are expressing

public sector noted a variety of transformation within government, such as

on both intended and unintended consequences, identify adequate objectives

d. Tactical use implies function of evaluation to seek assurance that control

demands on prompt information of progress and outcomes, evaluation reports

c. Contextual Consideration acknowledges the vital role of context in

and optimize their organization based on data provided by the evaluation.

further illustrate the comparison between evaluation and performance

Evaluation and performance measures address the question of how

a. Setting strategic goals

Nevertheless, key differences exist between evaluation and

Figure 2.1 General Model of Self-Auditing and Control

Governments face growing demands to deliver and prove results, and

a. Inputs, which denote the resources employed in producing outputs, such

evident that a standardized solution is almost not feasible in which there is no

measurement. First, adequate resources are imperative facilitate the

Similarly, Davies (1999) also believes the significance of designing