Professional Documents
Culture Documents
The TQM Journal: Emerald Article: External and Internal Quality Audits in Higher Education
The TQM Journal: Emerald Article: External and Internal Quality Audits in Higher Education
Article information:
To cite this document: Juha Kettunen, (2012),"External and internal quality audits in higher education", The TQM Journal, Vol. 24
Iss: 6 pp. 518 - 528
Permanent link to this document:
http://dx.doi.org/10.1108/17542731211270089
Downloaded on: 24-09-2012
References: This document contains references to 31 other documents
To copy this document: permissions@emeraldinsight.com
Access to this document was granted through an Emerald subscription provided by TURKU UNIVERSITY OF APPLIED SCIENCES
For Authors:
If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service.
Information about how to choose which publication to write for and submission guidelines are available for all. Please visit
www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
With over forty years' experience, Emerald Group Publishing is a leading independent publisher of global research with impact in
business, society, public policy and education. In total, Emerald publishes over 275 journals and more than 130 book series, as
well as an extensive range of online products and services. Emerald is both COUNTER 3 and TRANSFER compliant. The organization is
a partner of the Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive
preservation.
*Related content and download information correct at time of download.
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1754-2731.htm
TQM
24,6
518
Juha Kettunen
Received 29 May 2011
Revised 21 July 2011
Accepted 29 September
2011
Abstract
Purpose The purpose of this paper is to analyse the maintenance of the process-based quality
assurance system in a higher education institution.
Design/methodology/approach The paper introduces the process management as the essential
element of quality assurance in higher education and discusses the external quality audit of the quality
assurance agency, extends the study to the quality management between the external audits and
presents the procedure of internal quality audits. Finally, the results of the study are discussed
and summarised. Action research methodology was adopted in this study. The paper shows that
the process-based quality assurance system makes the organisation responsive, agile and enables the
achievement of strategic objectives.
Findings The audit group must first evaluate the necessary improvements in the process. If no
improvements are found, the quality deviations must be reported. The audit helps the institution take
corrective actions to amend the process descriptions or maintain the processes.
Originality/value The paper shows that the necessary processes of a higher education institution
can be systematically described and audited.
Keywords Finland, Higher education, Process management, Auditing, Quality assurance
Paper type Case study
1. Introduction
The recent literature on quality assurance in higher education is focused on the
planning and auditing of quality assurance systems, but the literature describing the
maintenance of the quality assurance system after obtaining the desired certificate is
meagre (Costes et al., 2008). Water (2000) defines maintenance as the control of the
quality management which can be seen as the control system itself. Autonomous
higher education institutions (HEIs) have an obligation to plan and implement their
quality assurance systems. They typically include dissimilar descriptions of processes.
The maintenance of the processes is an essential activity that can be carried out in
a systematic manner.
External quality assurance in the European higher education systems has
developed tremendously in recent years. The main purpose of the quality assurance
agencies is to support the development of the quality of HEIs. The quality assurance
agencies have formally been recognised by public authorities in the European higher
education area as agencies with responsibilities for external quality assurance
(Costes et al., 2008). These agencies regularly perform external quality assurance as
a core function.
Each HEI is responsible for the design and implementation of its own quality
assurance system. Certified systems, such as the ISO-norms, speak of external and
internal auditing (ISO 19011, 2002; Hernandez, 2010). Because HEIs typically have
no standardized quality assurance systems, they rather have value-added auditing
(Dereli et al., 2007) to give assistance to the institution and the quality assurance
agency. The auditing can be seen as part of the maintenance of the system. In an ideal
case, the maintenance system is designed at the moment of implementation of the
quality assurance system or when the maintenance of a particular part has been
started, but the maintenance should be designed not later than after the external audit
based on the recommendations of the audit group.
The purpose of this study is to analyse the internal and external auditing of
processes to maintain the quality assurance system of an HEI. Process management is
integral to the quality assurance system of the organisation ( Jeston and Nelis, 2006,
2008). The internal process auditing requires management, personnel and the target
unit of the institution to continuously improve the processes or amend education,
research and development and support services when corrective actions are needed.
This study emphasises that HEIs should adopt procedures to maintain and develop
their quality culture between the external evaluations.
The purpose of institutional quality assurance is to ensure that the defined
objectives can be achieved (Kettunen, 2008, 2011b). Quality, therefore, is the verified
achievement of objectives. The quality assurance system produces information about
how the institution and the extent to which its administrative units have succeeded
in their activities. The information is used to improve the activities and ensure that
the processes are reasonable and can be controlled. The quality assurance system
communicates the outcomes and activities of the institution to the management,
personnel, students and external stakeholders. The system enhances public confidence
in the quality of education and other activities.
The quality audit establishes the qualitative objectives set by the HEI for its own
activities, and evaluates the procedures and processes that the HEI uses to maintain
the quality of its education and other activities. Its purpose is to evaluate whether the
quality assurance of the HEI works as intended, whether the quality assurance system
produces useful and relevant information for the improvement of its operations and
whether it results in effective improvement measures (Finnish Higher Education
Evaluation Council, 2008).
The study is organised as follows. Section 2 introduces the process management as
the essential element of quality assurance in higher education. Section 3 discusses the
external quality audit of the quality assurance agency. Section 4 extends the study
to the quality management between the external audits and presents the procedure
of internal quality audits. Finally, the results of the study are discussed and
summarised in the concluding section.
2. Process management in the quality assurance
The European Association for Quality Assurance (ENQA) in higher education
disseminates information, experiences and good practices in the field of quality
assurance in higher education to European quality assurance agencies, public
authorities and HEIs. ENQA promotes European co-operation in the field of quality
assurance in higher education in order to develop and share good practice in
quality assurance and to foster the European dimension of quality assurance.
The guidelines, requirements and membership criteria of ENQA have been described
in the Standards and Guidelines for Quality Assurance in the European higher
education area (ENQA, 2009).
The FINHEEC the primary quality assurance agency in charge of evaluation of
Finlands universities and universities of applied sciences. According to the
Universities Act and the Universities of Applied Sciences Act, the institutions are
responsible for the evaluation of their education, research and other activities and
the impact of those activities. The institutions participate in the external evaluation
Quality audits in
higher education
519
TQM
24,6
520
of their operation and quality assurance system on a regular basis and publish the
findings of those evaluations.
The institutional audits of all the Finnish higher educational institutions are
undertaken by FINHEEC in six-year cycles. The institutional audits evaluate the
quality assurance system of the institutions. The evaluation of centres of excellence
undertaken on a regular basis is the other task, whose aim is to develop the quality of
processes, evaluate the outcomes achieved and provide feedback for participating
institutions (Kettunen, 2011a). The Finnish institutional quality audit does not evaluate
the real outcomes of the HEIs. FINHEEC also undertakes thematic evaluations such as
the national implementation of the Bologna Process.
Action research methodology was adopted in this study, because it provides
solutions of immediate relevance and generalised knowledge. According to Rowley
(2003), action research generates insights that enhance the understanding, analysis and
critical evaluation of theory. Action research encourages the use of concepts and
models from theory as a lens through which sense can be developed about the
organisation. Action research also helps researchers demonstrate analytical abilities in
linking theory and practice.
External and internal audits are the maintenance of the quality assurance
system. The results and recommendations of external audits affect the design and
implementation of internal audits, which are typically more thorough than the external
audits. The external audits can result in the recommendation that the internal audits
should be performed more frequently or be more attentive to certain shortcomings.
When one process is improved, other processes may require scrutiny. This scrutinising
can be seen as a maintenance activity. The internal auditing can be carried out on
a continuous or periodical basis. If the result of an auditing violates the defined
processes, maintenance has to be started.
The processes are maintained and continuously improved following Demings
(1986) Plan-Do-Check-Act sequence. At the first stage, the objectives for the
activities are planned and defined and the processes are inspected. The plans are
then implemented according to the process descriptions. The third stage is the
evaluation of the achievement of objectives and the conformance to processes.
If non-conformance is detected, it is reported along with the plan for corrective
actions. Based on the result of the evaluation the processes are improved and
approval is given to proceed with successive activities. Once the processes have been
amended, all affected personnel are notified of the changes. It is clear that without
an integrated system of process descriptions there will be no successful continuous
improvement.
Continuous improvement usually focuses on the improvement of existing processes,
without inquiring into completely alternative ways of performing the tasks, for
instance, integrating processes or eliminating unnecessary activities. Business
process re-engineering is typically cross-functional and changes are typically larger in
business process re-engineering. Usually continuous improvement provides
incremental innovations but re-engineering is related to the strategic objectives and
other goals of the organisation and produces radical innovations (Pereira and
Aspinwall, 1997).
The process approach of quality assurance means that the processes, their mutual
interaction and management are inter-related. Integration is a way to make a whole,
or to unify disparate activities or parts. Integration is more than the mere
physical compatibility of equipment and components. An integrated system works in
Quality audits in
higher education
521
TQM
24,6
522
site visits to faculties and thematic interviews with personnel administration, language
education and the support services of branches. The third day was spent continuing
the thematic interviews with communication and marketing activities. The quality
audits usually also include a random target of an interesting topic or tours of the
campus and facilities. Finally the audit group interviewed the top-management a
second time and gave the preliminary feedback of the audit results. In case of nonconformity the management should immediately take corrections, corrective actions or
preventive actions (ISO 9001, 2000).
The closing seminar was arranged for the management and personnel of the TUAS
in February 2010. The audit group presented the recommendations, which were
discussed at the seminar. The auditing procedure and the recommendations of the
audit group were published by FINHEEC in the audit report (Hintsanen et al., 2010).
The TUAS clearly passed the audit, which is not necessarily obvious, because many
institutions have had re-audits. Some of the recommendations of the audit group were
implemented at once but others were included in the 2011 or later action plans.
Table I describes the audit targets and the conclusions made by the audit groups
about the strengths, good practices and weaknesses of the quality assurance systems
of Finnish HEIs in 2005-2008. Audit targets two o four and six are directly related to
the internal processes of the institution. It can be seen that 52 per cent of the
weaknesses are directly linked with the quality assurance of processes, but indirectly
the processes have larger importance in quality assurance. Therefore the institutions
Audit targets
Table I.
Strengths, good practices
and weaknesses of
the quality assurance
systems of Finnish HEIs
in 2005-2008
Strengths
Good practice
Weaknesses
19
10
23
13
9
18
7
19
6
10
5
7
9
5
1
8
10
15
10
13
18
22
2
22
103
0
7
84
5
24
159
should pay more attention to process management. When institutions and their quality
assurance systems mature over time the auditing should be focused more on the
processes and objectives moving away from compliance auditing but at the same time
making sure that all regulatory and statutory requirements are met (Pollit et al., 2002).
One of the main problems is the insufficient comprehensiveness of the quality
assurance procedures. The quality assurance does not cover all the detailed processes
of the institution. Moitus (2009) points out that attention of audit groups is focused
on specific institution-specific targets, which may lead to weaknesses in other targets.
The Finnish evidence indicates that 29 per cent of the re-audits are related to the
quality assurance as a whole and 35 per cent are directly related to internal processes
and the continuous improvement. The other main reasons that an HEI would fail
the quality audit are the inadequate documentation and information generated by the
quality assurance system.
4. Internal quality audits
The quality assurance system of the TUAS has identified and developed processes
across the organisation. The processes are described in flow charts, process cards and
detailed documents that the institution places in its intranet (Chase, 1998; Lari, 2002;
Chin et al., 2004; Haraburda, 1999; Hussain et al., 2009). The sequence and interaction
among the processes are determined. The institution has also determined the
necessary criteria and methods of assuring that both the operation and control of these
processes are adequate. It has also ensured the auditing and continuous improvement
of these processes. The personnel of the institution are expected to act in accordance
with the established processes.
The external quality audit of the TUAS recommended the systematic auditing of
internal processes. The internal quality audits support the usability of process
descriptions and development of processes (Hintsanen et al., 2010). The internal audits
show potential, because they produce information about the functioning of the
quality assurance system and the necessary improvements of processes. The internal
quality audits are targeted only to the processes and are therefore only part of the
quality assurance system which includes many other procedures (Kettunen, 2010).
The internal audit is a procedure to evaluate the described internal processes of the
institution according to the annual action plan based on the general two- to three-year
plan. The audit group prepares the auditing documents and comprises the members of
management, personnel and students who have passed the training for internal
auditing. The Rectors decision includes the purpose of auditing, the main topics, the
stipulations where processes are based, the timetable, chairman and auditors.
Traditional compliance-based audits are shifting towards performance-based
audits. A cross-functional audit group is essential for performance-based internal
quality audits. It is a team of dedicated people with different functional expertise
working towards a common goal. The success of the cross-functional audit groups
depends upon the support of top-management and the competence of the group how to
audit against the process descriptions. Empirical results by Milena and Rusjan (2010)
show that internal audits contribute to the achievement of business goals and that
internal audits have positive effects on business performance.
The internal audit can suggest ways to improve the processes to increase efficiency.
It is necessary to evaluate how critical operations function to achieve the objectives
and meet the customer needs. Auditing also identifies the procedures that do not
work as planned and documented in the process descriptions. Consequently, the
Quality audits in
higher education
523
TQM
24,6
524
internal auditing has two main parts. The auditing identifies the achievement of
objectives and the improvement of processes to avoid operational risks as part of the
quality assurance system.
One of the ethical principles of the quality audit is its objectivity, which means that
the observations and conclusion are based on oral or written evidence. The auditors
are disinterested observers. Another principle is independence, which means that
the auditors do not audit any process for which they are responsible and that they have
no affiliation with the target unit. The auditors must have sufficient and adequate
qualification and experience. The planning and procedure of auditing are systematic
and consequential. The audit uses proven methods to ensure the reliability and validity
of data. The auditing pays attention to all essential matters.
Figure 1 describes the essential elements of the internal quality audit at the TUAS.
The quality manager takes the main responsibility of the quality audits and prepares
the action plan of the quality auditing for the period of two to three years and
presents the plan for the Rector and management team. The Rector of the institution
supervises the preparation of the auditing and approves the action plan of auditing
based on the proposal made by the quality manager.
The audit visits start with the opening session, where the chairman of the audit
group describes the purpose of auditing, procedure and timetable for the target unit.
The opening session is followed by the site visits. The audit group plans the themes
and questions of the site visits. The visits comprise individual or group interviews
depending on the nature of the process. The audit group does not collect only oral
evidence, but also written documentation about the functioning of the processes.
It is necessary to focus first on process management in auditing and look for
improvements in the process. If improvements are not found, the audit group should
Quality audit of processes
Management
group
Contributes
the action plan
of auditing
Coordinates
the improved
process
Rector
Quality
manager
Target
unit
Process
owner
Figure 1.
The procedure of internal
quality audit
Audit group
Supervises
the action plan
of auditing
Approves
the action plan
of auditing
Prepares
the action plan
of auditing
Plans
the auditing
Approves
the improved
process
Prepares
the improved
process
Approves
the audit
report
Prepares
the audit
report
Summary
meeting of audit
Audit visits
Improves
the process
Yes
Yes
Improved
process
No
Quality
deviation
No
look for the quality deviations. There is no reason to look at the quality deviations if
the process is flawed. The audit group makes two important decisions in this order:
(1)
(2)
The results of the audit are prepared for the summary meeting of audit and presented to
the management and personnel of the target unit. The results include good practice,
suggested improvements in the process and quality deviations, which are presented,
discussed and specified in the meeting. The purpose of the collaborative summary
meeting is to effect improvements in the processes, action and disseminate good practice.
If the process is improved, the quality manager prepares and submits the improved
process description for the management team, which coordinates the process
descriptions of the institution. The audit group and quality manager prepare a written
audit report. The members of the audit group have different themes during the site
visits. The audit group and the target unit write feedback from the auditing to the
audit report. The feedback to the audit group is used to evaluate and improve the
procedure of the internal quality audit. Finally, the Rector approves the improved
process and the audit report.
5. Conclusions
HEIs have an obligation to plan and implement their quality assurance systems, but
the autonomous institutions typically have no specifications as regards the structure
and contents of the system. That allows HEIs to develop the systems and procedures
that are consistent with their own strategic and operational objectives and
requirements. The autonomy allows the use of external support to audit the quality
assurance systems and to select the internal procedures of quality assurance.
This study found the evidence of the process management as a mechanism to
describe, maintain and improve the quality assurance system of an HEI. The systematic
procedure of external audits and internal process audits to maintain processes promote
the continuous improvement of the institution. If the processes are not known, they
cannot be managed in a systematic way. The process descriptions evolve through
collaborative planning, implementation, auditing and continuous improvement.
According to the evidence, the weaknesses found in the external audits of the
quality assurance systems are mainly related to process management. The process
management has not yet attained a strong position in higher education even though it
is widely used by private companies. The most prominent weaknesses of the quality
assurance can be found in education, but there are challenges also in the outreach and
Quality audits in
higher education
525
TQM
24,6
526
Quality audits in
higher education
527
TQM
24,6
528