A Guide for the Academic Audit Auditor

2012-2013 Cohort

Tennessee Board of Regents

 A Guide for the Academic Auditor 2012-13...

CONTENTS

INTRODUCTION …...................................................................................................................................3

OVERVIEW OF THE ACADEMIC AUDITOR PROCESS …..……………………………....................3

PART 1: FORMATION OF THE ACADEMIC AUDITOR TEAM …………………………...................4

PART 2: PREPARATIONS PRIOR TO THE SITE VISIT .........................................................................5

PART 3: THE SITE VISIT EXPERIENCE ……………………….……….………...................................6

PART 4: POST SITE VISIT RESPONSIBILITIES ....................................................................................7

PART 5: SAMPLES AND SCENARIOS ………….…………………………….…….........................….8

APPENDIX A: SUGGESTED ACADEMIC AUDIT SCHEDULE …..…..................................................9

APPENDIX B: ACADEMIC AUDIT ON-SITE EVALUATION CHECKLIST……...........................…10

APPENDIX C: ACADEMIC AUDIT VISITING TEAM REPORT/RECORD OF

COMMENDATIONS, AFFIRMATIONS, AND RECOMMENDATIONS ................................11

APPENDIX D: ACADEMIC AUDITORS REPORT GUIDELINES……………….…............................12

APPENDIX E: ACADEMIC AUDITOR TEAM LEADER RESPONSIBILITIES ..........................……14

APPENDIX F: ACADEMIC AUDIT SUMMARY SHEET FOR PERFORMANCE

FUNDING: UNDERGRADUATE PROGRAMS ..........................................................................17

APPENDIX G: ACADEMIC AUDIT SUMMARY SHEET FOR PERFORMANCE

FUNDING: GRADUATE PROGRAMS ........................................................................................20

APPENDIX H: FOLLOW-UP OF PREVIOUS ACADEMIC AUDIT .......................................................25

REFERENCES …..………………………………….…………………………………...........................…27

Page 2
 A Guide for the Academic Auditor 2012-13...

INTRODUCTION
Which way to quality assurance in higher education? (Through) a new, mutually reinforcing
system of institution-based quality assessments of teaching and learning and a coordinated
regional system of external academic audits (Dill, 1996).
The commitment to quality assurance in higher education through the continuous improvement of
teaching and learning processes lies at the heart of the Academic Audit initiative in the Tennessee Board
of Regents (TBR) system. The 2012-2013 academic year marks the ninth cycle of the Academic Audit
initiative in the TBR. Since its inception during the pilot year of 2004 – 2005, over 300 academic
programs in TBR institutions have undergone the Academic Audit process. In support of the Academic
Audit process, more than 400 faculty members and academic leaders from all 19 TBR campuses and
other institutions have served as members of Academic Auditor Teams.
The importance of the auditor in the Academic Audit process cannot be overstated. Reasons cited by the
director of the Academic Audit Unit in New Zealand for the acceptance of the academic audit process
throughout its university system include:
 Thoroughness of the auditor training;
 Commitment and professionalism of the auditors leading to a rigorous but fair audit process;
 The tone of the reports, with commendations as well as recommendations (Meade).
Auditor training has been a foundational element of the Academic Audit process from its inception, and
the TBR is committed to conducting a thorough course of preparation for academic auditor team
members and team leaders.
The Academic Audit, like more traditional program review process, is an academic auditor process
including a self-study and a site visit by peers from outside the institution. The Academic Audit process
emphasizes self-reflection and self-improvement rather than compliance with predetermined standards.
The purpose of an academic audit is to encourage departments or programs to evaluate their “education
quality processes” – the key faculty activities required to produce, assure, and regularly improve the
quality of teaching and learning. An academic audit asks how faculty approach educational decision-
making and how they organize their work, using the resources available to them and working collegially
to provide a quality education in the best interests of the discipline and student learning.
OVERVIEW OF THE ACADEMIC AUDITOR TEAM PROCESS
The TBR Academic Audit program issues a handbook, updated annually, that provides the following
description of the academic auditor team review process:

 Academic auditors are volunteers (primarily faculty) who receive training on education quality
processes and academic audit methodology.
 Academic Auditor Teams (2-4 members) most likely but not exclusively come from other TBR
institutions.
 All departments are given the opportunity to nominate peers for service on their Academic
auditor Teams.
 Because the academic auditors will be focusing on quality processes, they do not have to come
from the academic discipline of the department being audited although TBR strives to have
faculty from the discipline or a closely related discipline on the team.

Page 3
 A Guide for the Academic Auditor 2012-13...

 Academic auditor team visits are typically one day per department.
 Academic auditors meet with campus and departmental leadership, faculty, and students.
 Academic auditors ask questions similar to the self-study questions.
 Academic Auditors write a report:
o highlighting examples of exemplary practice,
o noting areas for improvement, and
o evaluating a department’s approach to educational quality practices.

The purpose of this training document is to further explore and explain these processes and functions.
PART 1: FORMATION OF THE ACADEMIC AUDITOR TEAM
Academic Auditor Team selection process
Each Academic Auditor Team is assembled by the TBR Academic Audit coordinator. The Team
consists of a team leader and usually two team members for a total of three members. For some small
programs, a two person team is appointed; for a few very large programs or a discipline that issues
degrees at more than one level, a four person team may be formed. The TBR Academic Audit
Coordinator will assign the team leader (chair) for each academic auditor team. Up to two team
members may be nominated by the program undergoing review. These nominees should be from other
institutions and not necessarily from the same discipline as the program undergoing review. Also, the
nominees need NOT have previously served as members of an academic audit site review team.
Training is provided to all members of Academic auditor Teams. If nominations are not made or if a
nominee is unable to serve, the TBR Academic Audit Coordinator will identify team members from the
pool of auditors.
Discipline-specific or not?
Every effort is made to identify one auditor from the discipline or from a closely related discipline for
each program undergoing the academic auditor team review process. As the focus of the academic audit
is on the processes - both in place and planned - to continuously improve the quality of teaching and
learning, interdisciplinary participation is of great value. Educators share the common challenge of
reaching, teaching and assessing students as they aspire to attain their educational goals. A precept of the
academic audit process is that such a challenge is not discipline specific and that we can all learn from
our colleagues who demonstrate the very best practices of teaching and learning in higher education:
“…auditors need not be expert in any particular academic discipline. Disciplinary knowledge is not a
prerequisite for conversations about education quality processes. Indeed, too much disciplinary
knowledge tempts auditors to advocate their own preconceived views about how things should be done.”
(Massy, 2003)
Characteristics of academic auditor team members
Academic auditor team members should be recognized as dedicated and talented faculty members and
academic leaders. They may be subject matter experts whose discernment is highly valued, or they may
be learning leaders who are prized for their insights into quality teaching and learning processes.
Regional/distance considerations should not be the most important criterion for selecting team members.
Partnerships and previous associations are acceptable as long as objectivity is not compromised. Also
desirable are the abilities to:
 Work cooperatively with team members,

Page 4
 A Guide for the Academic Auditor 2012-13...

 Communicate effectively in speaking and writing,

 Apply a broad perspective in an open-minded fashion, and
 Be willing and able to make a commitment of time and effort to this important process.
Commitments required of academic auditor team members
Service as an Academic auditor team member involves these commitments:
 Thorough familiarization with the Academic Audit concept and process,
 Participation in collaborative work sessions with other academic auditor team members before,
during, and after the site visit,
 Preparation of questions prior to the site visit, responses on the day of the site visit, and segments
of a final written report after the site visit, and
 Careful discernment of the strengths and opportunities for improvement in quality academic
processes by the program undergoing the academic audit.
Conflicts of interest
Every attempt is made to avoid a conflict of interest between the academic auditor and the program/
institution undergoing the academic audit process. As there are extensive interactions between all
professionals within the higher education system and especially within the TBR system, it is not
expected that no prior interactions exist between an academic auditor and the program or institution.
However, if there is any concern that excessive bias (either favorable or unfavorable) exists by the
academic auditor towards the program or institution or by the program /institution towards the academic
auditor, that academic auditor would be excused from service on that particular academic auditor team
and replaced.
PART 2: PREPARATIONS PRIOR TO THE SITE VISIT
Success of overall academic auditor team experience will be determined in large part to both the level of
preparation by the individual academic auditor team members as well as the level of collegiality
between the team members. The following steps are encouraged to foster high performance by the
academic auditor team:
1. All team members should become familiar or reacquainted with the academic audit process.
They should review the Academic Audit Handbook to gain an understanding of the purpose
of the Academic Audit process, the nature of the focal areas, the emphasis on “processes”,
and the role of the academic auditor team. Those who have served on an academic auditor
team previously and/or been a part of a self-study process should reflect judiciously on those
experiences.
2. Once the academic auditor team is formed, team members should become acquainted
through email exchange, teleconference, or, if possible, a face-to-face meeting. An auxiliary
benefit to the academic audit process is the opportunity to build new relationships with
colleagues and gain from one another’s experiences and insights.
3. When the self-study document written by the program to be reviewed is received (in early
February), each team member should read the document from beginning to end without
analysis or notation. The goal is to acquire an overview of the scope and nature of the

Page 5
 A Guide for the Academic Auditor 2012-13...

program, its self-study procedure, and the processes it identified that are either in place or are
planned to assure continuous quality improvement in teaching and learning.
4. Team members should also review the appendices for further perspective on the program.
The self-study is limited by design to 20 pages for undergraduate programs. Additional
information may be included in an appendix as well as via web page links.
5. All team members should clearly understand if the program being reviewed is using the
academic audit process as its means to meet the program review requirement by THEC for
Performance Funding credit. If so, team members should review the appropriate Performance
Funding Summary Sheet (undergraduate or graduate) to prepare for discussions on those
criteria with fellow team members.
6. During a second reading of the program’s self-study, each team member should make notes
and draft questions that could be used at the site visit.
7. Communication between team members is essential. Organization of these communications
will come from the team leader and may be in the form of teleconferences, email exchanges,
web postings, etc. Each team member will likely be assigned specific focal areas on which to
concentrate.
8. The academic auditor team should develop a proposed site visit schedule which the team
leader will finalize with the academic audit campus coordinator at the program’s campus.
9. The team will collectively prepare questions for each set of conversations that will be held on
the day of the site visit. These questions should address areas that may have been unclear, not
fully explained or not discussed at all in the self-study document.
10. Team members must each make arrangements with their home institution for travel and any
missed teaching assignments or other responsibilities for the time period spent on the site
visit experience.
PART 3: THE SITE VISIT EXPERIENCE
“The site visit by the audit team sets in motion the academic auditor process…The role of the audit team
is to set the tone for a collegial, supportive dialogue” (Massy, 2007). The site visit experience is a very
full immersion in the academic auditor process beginning with the gathering of the academic auditor
team at an area hotel the evening prior to the site visit date and ending the following afternoon with a
report to the program. The success of the site visit is predicated upon preparation both by the academic
auditor team and the program itself. Both parties agree upon a schedule well in advance. A sample
schedule is included in Appendix A, but the actual schedule will be customized according to the specific
nature of the program undergoing the site visit. Key elements of a successful site visit include the
following actions by the academic auditor team and its leader:
1. On the evening before and/or the morning of the site visit day, the team should meet together
off campus in order to cement collegiality; clarify each individual’s responsibilities; and
share any lingering concerns regarding the self-study, appendices, or auxiliary materials
referenced in the self-study (for example, postings on a program’s website). Team members
should review the site visit schedule [see Appendix A for a sample schedule].
2. From arrival on campus on the morning of the site visit date and throughout the day, the team
should make every effort to adhere to the schedule.

Page 6
 A Guide for the Academic Auditor 2012-13...

3. During all conversations (meetings) with faculty, students, and/or other stakeholders, each
team member should maintain the role of auditor meaning “listener.” Questions should be
succinct without lengthy framing or anecdotal embellishment.
4. An effective way to validate responses is by repeating a question to other participants and/or
at another session. For example, if faculty members are asked this question: “How do you
determine if your web-based assessments measure student learning?”, then during the session
with students, a few students could be asked, “How effectively do the web-based assessments
measure what you are supposed to have learned?”
5. Each team member should take good notes. These will be invaluable at two pivotal junctures:
a) that afternoon when the team prepares the lists of Commendations, Affirmations, and
Recommendations; and b) when each team member writes a part of the academic auditor
team report.
6. The host program may provide additional information on site that the team members should
also review.
7. After all meetings are over, the team needs to complete the Academic Audit Onsite
Evaluation Checklist [see Appendix B] and come to consensus regarding Commendations,
Affirmations, and Recommendations [see Appendix C]. If there is a need to ask for
information or to hold an additional conversation, this is permissible.
8. When the Performance Funding Summary Sheet is required for the program, the team must
make decisions regarding the criteria and complete the Summary Sheet. The Summary Sheet
asks for the name, title, institution, and signature of each team member, which should be
issued on the form prior to departure [see Appendix D].
9. At the end of the day, the team collectively makes a presentation of about 15 minutes in
length to the program faculty, administrators, and guests at which the Commendations,
Affirmations, and Recommendations are given.
10. The team leader provides the original, signed copy of the Academic Audit Onsite Evaluation
Checklist and, when necessary, the Performance Funding Summary Sheet to the program’s
representative. [Note: the team leader and members keep copies one of which is subsequently
included in the team’s report sent to the TBR.]
11. Before departing, team members should complete the travel forms and provide receipts for
costs incurred related to the site visit to the host institution as team members will be
reimbursed by the host institution.
12. Team members should depart with a plan and timetable for completing the written response
by the academic auditor team. In several cases, academic auditor teams are able to draft a
significant part of the final written report during the early afternoon work session prior to the
oral presentation to the program.
PART 4: POST-SITE VISIT RESPONSIBILITIES
While the campus site visit marks the zenith of activity for the academic auditor team, several very
important tasks remain. These require the cooperation of all team members.
1. First and foremost, the final written report of the Academic auditor Team needs to be
completed in a timely manner but no later than May 17, 2013. While the team leader usually

Page 7
 A Guide for the Academic Auditor 2012-13...

drafts the final document, team members will be expected to submit write-ups of assigned
sections.
2. It may also be necessary for team members to participate in a follow-up conference call to
address particular points that need clarification and consensus. The team leader may ask team
members to review the final draft of the report for accuracy and completeness.
3. When the report is finalized, the team leader sends it to the Academic Audit Coordinator at
the Tennessee Board of Regents along with a copy of the Summary Sheet when relevant. The
TBR Coordinator will forward the final report to the host campus.

PART 5: ACADEMIC AUDITOR TEAM MEMBER TRAINING: SAMPLES AND SCENARIOS

This overview is intended to provide academic auditor team members (auditors) with a conceptual and
pragmatic foundation from which they may engage the academic auditor process as individuals and as
team members. The auditor training program will also include an interactive training session. At this
session, auditors will have the opportunity to review segments of sample self-study reports; collaborate
with peers to evaluate elements of a sample self-study; and formulate possible questions for a site visit
conversation with faculty, students, and other stakeholders. A role-playing exercise will also be included
in this part of the auditor training experience.

That is what learning is. You suddenly understand something you've understood all your life, but in a
new way.
Doris Lessing

Page 8
 A Guide for the Academic Auditor 2012-13...

Appendix A: Suggested Academic Audit Schedule

Academic Program, Department and Institution
Date of Visit
Session Time/Attendees Location
Breakfast and Team Meeting at Hotel 7:30am – 8:15am Hotel
Audit Team

Opening Session & Introductions 8:30am – 9:00am ABC Bldg. Room 1

Academic auditor Team – overview
Faculty and Administrators

Small Group Meeting #1 9:00am – 10:00am ABC Bldg. Room 2

Academic Auditor Team conversation with
Faculty Members

Small Group Meeting #2 10:15am – 11:00am ABC Bldg. Room 2

Academic Auditor Team conversation with
Students

Small Group Meeting #3 11:15am – 12:15pm ABC Bldg. Room 2

Academic Auditor Team conversation with
Faculty Members/other Stakeholders

Working Lunch 12:15pm – 1:30pm ABC Bldg, Room 2

Academic Auditor Team only

Flexible Meeting and Work Time 1:30pm – 3:30pm ABC Bldg. Room 2
Academic Auditor Team
-Possible time for materials review, tours, or additional meetings with faculty or students
-Time for group to discuss observations and work on reports

Exit Session 3:30pm – 4:00pm ABC Bldg, Room 1

Academic Auditor Team, Faculty and Administrators

-Brief report of initial commendations, affirmations, and recommendations

-NOTE: If the Performance Funding Summary Sheet is required, a copy of that sheet must be left
with the campus contact prior to departure.

Logistics Information
Campus Contact for Academic Audit– Name, Position, Phone (office, cell), E-mail
Hotel Accommodations – Hotel Name, Address, Phone

Page 9
 A Guide for the Academic Auditor 2012-13...

Appendix B: Academic Audit Onsite Evaluation Checklist

Institution: _______________________________________________________________
Program: ________________________________________________________________
CIP Code: ________________________________________________________________
Degree Level:  Certificate  Associate  Baccalaureate  Master’s  Doctoral
Instructions for Audit Chairs and Teams
Part I: Academic Audit Visiting Team Report -- Record of Commendations, Affirmations, and
Recommendations
This form must be completed by each audit review team prior to concluding the visit. The original will be
forwarded to TBR but a copy must be left with the department prior to departure. All observations included on
this form should be represented as commendations, affirmations, or recommendations. Please be concise in your
descriptions as you will have opportunity to expand upon your justification for each item in your written report
due to TBR by May 17, 2013.
Part II: Academic Audit Summary Sheet (only for use if program is being reviewed for Performance
Funding purposes)
This form is only to be completed if the program review is serving as the performance funding review.
Using the Academic Audit Summary Sheet complete the 25 elements on the evaluation results checklist (marking
“met” or “not met’). This exercise must be completed and signed by the team prior to the Exit Session. The
original will be forwarded to TBR but a copy must be left with the department prior to departure.
Part III: Narrative Evaluation and Written Report
The Audit Chair and Team will use their evaluations indicated on the Audit Visiting Team Report and
Academic Audit Summary Sheet (if used for Performance Funding purposes) as the basis of a written report.
Summarized findings from the self-study report and on-site visit will represent a narrative report of the team’s
conclusions and the final responsibility of the visiting team. The template for completing this report (limited to
10 pages) is attached. This report is due to TBR on May 17, 2013.
The Audit Evaluation will become part of the record of the academic program review and will be shared with
the academic department/unit, the college, and the central administration, as well as the Tennessee Higher
Education Commission. Each department/campus will be provided opportunity to respond and comment on the
written report.
Audit Chair’s name, title, and institution: ______________________________________
Audit Chair’s signature: ___________________________ Date_____________________
Names, titles, institutions, and signatures of other Audit Team members:
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________
_________________________________________________________________________

Page 10
 A Guide for the Academic Auditor 2012-13...

Appendix C: Academic Audit Visiting Team Report

Record of Commendations, Affirmations, and Recommendations
This form must be completed by each audit review team prior to concluding the visit. All
observations included on this form should be represented as commendations, affirmations, or
recommendations. Please be concise in your descriptions as you will have opportunity to expand upon
your justification for each item in your written report due to TBR by May 17, 2013.
This document should serve as the outline of information to be disclosed during the exit session with
the department. The original signed copy is to be forwarded to TBR with one copy left with the campus
audit contact or department chairperson prior to leaving campus.

******************************************************************

Total Number of Commendations ___

Commendation #1 –
Commendation #2 –
Commendation #3 –
Commendation #4 –
Total Number of Affirmations ___
Affirmation #1 –
Affirmation #2 –
Affirmation #3 –
Affirmation #4 –
******************************************************************
Total Number of Recommendations ___
Recommendation #1 –
Recommendation #2 –
Recommendation #3 –
Recommendation #4 –

Page 11
 A Guide for the Academic Auditor 2012-13...

Appendix D: Audit Report Guidelines

Updated 6/06
Template for Academic Audit Team’s Narrative Report

General Report Outline [NOTE: The report must be limited to ten (10) pages.]

I. Introduction -- Briefly describe the unit/program evaluated, the date of the Audit, the
protocol observed by the auditors, and other relevant information.

II. Overall Performance - What is the team’s overall summary conclusion about the state
of the unit/program?

III. Performance in the Focal Areas – How does the unit/program’s work in each focal area
measure up against the quality and evidentiary principles?

A. Learning Objectives
B. Curriculum and Co-Curriculum
C. Teaching and Learning Methods
D. Student Learning Assessment
E. Quality Assurance

IV. Conclusions. Briefly indicate the team’s conclusions regarding the following by
providing each in bulleted or numbered sequence:

A. Commendations – What processes, practices, initiatives, and commitments are

particularly commendable and merit recognition?
B. Affirmations – What processes, practices, or plans warrant the team’s affirmation and
encouragement? The key idea of Affirmations is that the unit being audited identified
the need for improvement before the Audit Team.
C. Recommendations – What are some areas for improvement identified by the team on
the basis of the unit/program’s self-study and site visit?

Page 12
 A Guide for the Academic Auditor 2012-13...

Key Themes and Tips for Writing Audit Reports

It is important for respondents to observe the following suggestions when developing the Audit
Report.1

1. The purpose of the report is to summarize findings from the self-study report and the visit.
2. Keep the tone of the report developmental – focus on how the department can improve on its
own. Address what it will work on as a result of the review.
3. The report is the synthesis of the teaching and research focal areas pulled together by the
team leader.
4. Provide feedback on all five themes in focal areas in summary form and give overview
perceptions.
5. Reports are written by the audit team and assignments are made at the visit, and members
provide bullet-type comments.
6. Use the debriefing time to gather ideas from members and create an overview of the report.
7. At the visit, agree on a report format and how long each section should be. Find some
consensus from the group and develop a strategy for the report (e.g. bullet comments only).
8. At the visit there is an oral debriefing with the department/unit before the team arrives. The
team decides on overall points. Use these themes to provide the context for the report and
the executive summary.
9. Ask for evidence as you gather impressions. Don’t build summary points around comments
from one faculty member or one small group of students – look for confirming evidence.
10. Keep it simple and tell them what they did well as well as what processes they may want to
address to facilitate continuous improvement.

NOTE: Write the report so that it is helpful for the department, but also write it as if others will read
it (i.e., be sensitive to language and tone of comments).

1
Key themes taken from materials provided by Dr. Bill Massy during work with Tennessee Board of Regents in the 2004-05
Pilot Phase of Academic Audit.

Page 13
 A Guide for the Academic Auditor 2012-13...

Appendix E: Academic Auditor Team Leader Responsibilities

1) Pre-Site Visit
Status
a) With the Campus Coordinator of the host site…
i) Decide upon hotel selection and related details
(restaurants nearby, directions to campus, parking on
campus, etc.)
ii) Share contact information including multiple contact
numbers
iii) Set up the specific schedule for the site visit day

(1) Establish parameters for each session – with whom,

how many, how long, etc.
(2) Make sure needs are addressed including computer
access, printer access, visits to a particular
program-related facility, coffee, etc.
b) With the Team Members…
i) Make initial contact with Academic auditor Team
members via email or telephone
(1) Make sure everyone has the program’s Self Study
and Appendices (if provided by the program)
(2) Confirm contact information including multiple
phone numbers including cell number
(3) Set up a time for a teleconference to discuss each
team member’s primary responsibilities. [NOTE: A
typical arrangement is for one team member to
focus on Learning Objectives and Curriculum/Co-
curriculum; a second takes Teaching & Learning
and Student Learning Assessment; and the team
leader takes Quality Assurance and also
responsibilities for the Introduction and Overall
Performance sections.] Each team member takes
the lead in her or his assigned areas for the
following:
(a) Close reading of those sections of the Self
Study
(b) Formulating questions for the onsite visits from
those sections
(c) Writing the parts of the final Auditor Report for
those sections
ii) Collect suggested questions for the site visit (via email)
iii) Share a composite of these questions with team
iv) Conduct one or more teleconferences to discuss
questions, concerns, strengths, etc. of the program with

Page 14
 A Guide for the Academic Auditor 2012-13...

team members to finalize the common set of questions

to be asked at the site visit
v) Prepare draft of Onsite Evaluation Checklist and
Summary Sheet (if required for Performance Funding)
to bring on laptop and/or travel-drive
vi) Share information with team members about practical
preparations for the site visit [NOTE: Each team
member makes her/his own travel request, hotel
reservation, and transportation arrangements]
2) Site Visit
a) Try to meet as a team either the evening before the site
visit – perhaps over dinner – or at breakfast – often at the
host hotel – to get to know one another a little better, to
review the site visit schedule, and to confirm roles.
b) Once on campus, guide the conversation sessions
especially in terms of managing time and moving through
the questions
c) After the morning sessions, lead the process for
formulating Commendations, Affirmations, and
Recommendations [NOTE: It is ideal if a team member
volunteers to serve as scribe, perhaps bringing a laptop.
Access to computer and printer should be provided at the
site.]
d) Set up a schedule for each team member submitting her/his
section of the final auditors report to the Team Leader
e) Be sure that the required forms are completed by the team:
i) Checklist
ii) Academic Audit Summary Sheet (for Performance
Funding programs only)
f) Give the final report at the end of the site visit day – all
team members can participate in this if the team so
chooses.
g) Give copies of the report to the program representatives,
but retain the original, signed copies for submission to the
TBR.
h) Remind team to submit mileage, hotel receipt, and any
other expense directly related to the Site Visit to the host
institution’s Campus Coordinator.

Page 15
 A Guide for the Academic Auditor 2012-13...

3) Post Site Visit Status

a) Writing the Auditors Report
i) Collect the sections written by other team members
ii) Assemble these parts into the final report following the
report guidelines contained in the Guide
iii) Add the Commendations, Affirmations, and
Recommendations to the final report
b) Submitting the Auditors Report
i) Send the report itself as a PDF or Word file to
randy.schulte@tbr.edu
ii) Mail these original, signed documents to the address
below on or before May 17, 2013
(1) Checklist (including Commendations, Affirmations
and Recommendations)
(2) Summary Sheet for Performance Funding (only if
applicable)

.
For additional information contact:

Randy Schulte, Assistant Vice Chancellor for Academic Affairs

Tennessee Board of Regents
1415 Murfreesboro Road – Suite 350
Nashville, TN 37217 – 2833
randy.schulte@tbr.edu
Office: 615-365-1505
Cell: 423-424-6400
TBR Academic Affairs: 615-366-4482

Page 16
 A Guide for the Academic Auditor 2012-13...

Appendix F: Academic Audit Summary Sheet

2010-15 Performance Funding Cycle

Appendix H: Academic Audit
Undergraduate Programs
Institution: _____________________________________________________________________________
Program Title(s): ________________________________________________________________________
CIP Code(s): ____________________________________________________________________________
Academic Audit Status: ______ First Academic Audit _____ Second Academic Audit

Instructions for Academic Audit Team:

In accordance with the 2010-15 Performance Funding guidelines of the Tennessee Higher Education Commission
(THEC), each non-accreditable undergraduate program undergoes either an academic audit or external academic
auditor according to a pre-approved review cycle. If the program under review contains embedded Technical
Certificates, the names of each certificate should be included on the “Program Title” line above. The review of
embedded certificates must be included as part of the review of the program in which they are embedded. Embedded
certificates do not require a separate Summary Sheet.
The criteria used to evaluate a program appear in the following "Academic Audit Summary Sheet.” The Summary
Sheet lists 26 criteria grouped into eight categories. THEC will use the criteria in categories 1-6 to assess Performance
Funding Standard 1C when the Academic Audit process is used for programs undergoing the Academic Audit process
for the first time. For programs undergoing the Academic Audit for the second time, criteria 7 (follow-up) will also be
used to assess Standard 1C. The criteria in the eighth category, Support, may be used by the institution and submitted
as part of the Performance Funding report. If the Academic Audit process did not include information about criteria
8.1 - 8.3, they should be marked N/A. These criteria will not be included in the THEC Performance Funding point
calculation.
These criteria have been selected based on the Academic Audit Focal Areas to be consistent with the spirit and process
of the Academic Audit. The program faculty has provided a self-study document that includes information for each
criterion within the Focal Areas. Supporting documents will be available as specified in the self study. As the
Academic Audit Team Leader, you should assess this and other evidence observed during the site visit to determine
whether the process has met each criterion within a category. A checkmark should be placed in the appropriate box to
indicate whether you believe that a program has “met” or “not met” each criterion in the table.
The Academic Audit Summary Sheet will be sent to the appropriate campus official for inclusion in the Annual
Performance Funding Report. When combined with the self study and the written report prepared by the visiting team,
the Summary Sheet will facilitate institutional development of a program action plan to ensure continuous quality
improvement.
Your judgment of the criteria will be used in allocating state funds for the community college or university's budget.

Name, Title, and Institutional Affiliation of Academic Audit Team Leader (s):
______________________________________ ____________________________________
Name Name

_____________________________________ ____________________________________
Title Title

______________________________________ ____________________________________
Institution Institution

______________________________________ ____________________________________
Signature Date Signature Date

Page 17
 A Guide for the Academic Auditor 2012-13...

Academic Audit Summary Sheet

Institution: ____________________________________________________________________
Program Title: _______________________________________________________________
Degree Level: __________________________________________________________________ Evaluation
Results
CIP Code:______________________________________________________________________
Academic Audit Status: ______ First Academic Audit _____ Second Academic Audit

Not
1. LEARNING OBJECTIVES Met
Met
The faculty completed a thorough and candid analysis of their process for
1.1 developing learning objectives for the program, considering measurability, clarity
and what students need to know.
1.2 The faculty documented or proposed a process for developing learning objectives
that are based on realistic and appropriate evidence.
1.3 The faculty documented or proposed specific plans to take best practices and
appropriate benchmarks into account in the analysis of learning objectives.
Not
2. CURRICULUM AND CO-CURRICULUM Met
Met

2.1 The faculty completed a thorough and candid analysis of the extent to which they
collaborate effectively on the design of curriculum and planned improvements.
The faculty documented or proposed a plan for analyzing the content and
2.2
sequencing of courses in terms of achieving program learning objectives.
The faculty documented or proposed a plan for the ongoing review of curriculum
2.3 and co-curriculum based on appropriate evidence including comparison with best
practices where appropriate.
Not
Met
3. TEACHING AND LEARNING PROCESSES Met
The faculty completed a thorough and candid analysis of their process for guiding
3.1
and improving teaching and learning throughout the program.
The faculty documented or proposed a plan that promotes the effective use of
3.2 instructional methods and materials for achieving student mastery of learning
objectives.
3.3 The faculty analyzed the extent to which there is true, ongoing collaboration in the
design and delivery of the teaching and learning processes of the program.
Not
4. STUDENT LEARNING ASSESSMENT Met
Met
The faculty documented or proposed indicators of student learning success that are
4.1
keyed to the learning objectives of the program.
4.2 The faculty documented or proposed assessments of student learning that are
grounded in best practices and appropriate comparisons.
4.3 The faculty documented or proposed a plan for using student learning assessments
that leads to continuous improvements in the program.
The faculty documented or proposed a continuous improvement plan that
4.4
incorporates multiple measures to assess student learning and program
effectiveness.

Page 18
 A Guide for the Academic Auditor 2012-13...

Not
5. QUALITY ASSURANCE Met
Met
There is an evident commitment to making continuous quality improvements in
5.1
the program a top priority.
The faculty documented or proposed a continuous improvement plan that
5.2 incorporates multiple measures to assess student learning and program
effectiveness.
Not
6. OVERALL ASSESSMENT Met
Met

6.1 The Academic Audit process was faculty driven.

The Academic Audit process (self-study and visit) included descriptions of the
6.2
program’s quality processes including all five focal areas.
The process resulted in a candid description of weaknesses in program processes
6.3
and suggestions for improvements.
Overall, the visiting team affirms the openness and thoroughness of the program
6.4
faculty in completing the academic audit of this program.
The Academic Audit process included involvement of and inputs from stakeholder
6.5
groups identified by the program’s faculty.

Not
7. FOLLOW-UP OF PREVIOUS AUDIT * Met
Met

7.1 An action plan was developed as a result of the previous Academic Audit.
There is documented evidence that recommendations made by the Academic
7.2 Audit Team have been considered and, when feasible and appropriate,
implemented and tracked.
There is documented evidence that the program has implemented and tracked the
7.3 progress of and use of results from improvement initiatives cited by the faculty its
self study.

8. SUPPORT (Note: The Support category is NOT included in the Performance Funding Not
calculation. If the Academic Audit process did not address these criteria, they should be marked Met
Met
“NA.”)

The program regularly evaluates its library, equipment and facilities, encouraging
8.1
necessary improvements within the context of overall college resources.
8.2 The program's operating budget is consistent with the needs of the program.
The program has a history of enrollment and graduation rates sufficient to sustain
8.3
high quality and cost-effectiveness.

* Criterion only included in the performance funding calculation for programs undergoing the Academic Audit
during the 2010-2015 cycle that also used the Academic Audit in the 2005-10 cycle. Note: please be sure that
the “Second Academic Audit” is checked on page 1.

Page 19
 A Guide for the Academic Auditor 2012-13...

Appendix G: Academic Audit Summary Sheet: Graduate Programs

2010-15 Performance Funding Cycle

Appendix I: Academic Audit
Graduate Programs
Institution: _____________________________________________________________________________
Program Title(s): ________________________________________________________________________
Degree Level(s): _________________________________________________________________________
CIP Code(s): ____________________________________________________________________________
Academic Audit Status: ______ First Academic Audit _____ Second Academic Audit

Instructions for Academic Audit Team:

In accordance with the 2010-15 Performance Funding guidelines of the Tennessee Higher Education Commission
(THEC), each non-accreditable graduate program undergoes either an academic audit or external academic
auditor according to a pre-approved review cycle.

The criteria used to evaluate a program appear in the following "Academic Audit Summary Sheet.” The Summary
Sheet lists 55 items grouped into 14 categories. THEC will use the criteria in categories 1 – 13 to assess
Performance Funding Standard 1C when the Academic Audit process is used for programs undergoing the
Academic Audit process for the first time. For programs undergoing the Academic Audit for the second time,
criteria in category 14 will also be used to assess Standard 1C.

These criteria have been selected based on the Academic Audit Focal Areas to be consistent with the spirit and
process of the Academic Audit. The program faculty has provided a self-study document that includes
information for each criterion within the Focal Areas. Supporting documents will be available as specified in the
self study. As the Academic Audit Team Leader, you should assess this and other evidence observed during the
site visit to determine whether the process has met each criterion within a category. A checkmark should be
placed in the appropriate box to indicate whether you believe that a program has “met” or “not met” each criterion
in the table. If a particular criterion is inappropriate or not applicable to the program, the criterion should be
marked “NA”.

The Academic Audit Summary Sheet will be sent to the appropriate campus official for inclusion in the Annual
Performance Funding Report. When combined with the self-study and the written report prepared by the visiting
team, the Summary Sheet will facilitate institutional development of a program action plan to ensure continuous
quality improvement.

Name, Title, and Institutional Affiliation of Academic Audit Team Leader(s):

_________________________________ ____________________________________
Name Name

_________________________________ ____________________________________
Title Title

___________________________________ ____________________________________
Institution Institution

___________________________________ ____________________________________
Signature Date Signature Date

Page 20
 A Guide for the Academic Auditor 2012-13...

Academic Audit Summary Sheet

Institution: _____________________________________________________________________
Program Title(s): _______________________________________________________________
Degree Level(s):_________________________________________________________________ Evaluation
CIP Code(s):____________________________________________________________________ Results

Academic Audit Status: ______ First Academic Audit _____ Second Academic Audit
Not
1. LEARNING OBJECTIVES Met
Met
The faculty completed a thorough and candid analysis of their process for
1.1 developing learning objectives for the program, considering measurability, clarity
and what students need to know.
1.2 The faculty documented or proposed a process for developing learning objectives
that are based on realistic and appropriate evidence.
The faculty documented or proposed specific plans to take best practices and
1.3
appropriate benchmarks into account in the analysis of learning objectives.
The faculty clearly communicates program objectives to current and potential
1.4 students, employers or other stakeholders.
Not
2. CURRICULUM AND CO-CURRICULUM Met
Met
The faculty completed a thorough and candid analysis of the extent to which they
2.1 collaborate effectively on the design of curriculum and planned improvements
which will reflect attained competencies in the outcome data.
The faculty documented or proposed a plan for analyzing the content, format and
sequencing of courses in terms of achieving program learning objectives with
2.2
appropriate breadth and depth for the degree offered which allows for attainment of
the degree in a timely manner.
The faculty documented or proposed a plan for determining the soundness of and
rationale for curriculum and co-curriculum based on appropriate evidence,
2.3
including comparison with best practices where appropriate, and communicate
these views to the student body.
Not
3. TEACHING AND LEARNING PROCESSES Met
Met
The faculty examined the extent to which there is focus on and periodic, systematic
3.1
review of the actual process of teaching and learning throughout the program.
The faculty documented or proposed a plan that promotes the effective use of
3.2 instructional methods and materials for achieving student mastery of learning
objectives.
The faculty analyzed the extent to which there is true, ongoing collaboration in the
design and delivery of the teaching and learning processes of the program with
3.3
reliance on best practices and resources beyond the confines of the program or
department.
3.4 There is a critical mass of faculty and students to promote a scholarly community
and assure an appropriate group of peers.
Faculty/graduate student ratio, average course load, average thesis/dissertation load
3.5 per faculty and distribution across department, and teaching evaluations evidence
support of graduate teaching and learning processes.

Page 21
 A Guide for the Academic Auditor 2012-13...

The faculty documented or proposed a plan to inform students of course offerings

3.6 and the provision of professional development activities and relevant courses to
supplement departmental offerings in a timely fashion.
The faculty documented or proposed a plan to ensure that all students are
3.7 adequately oriented, advised, mentored and socialized within the discipline and the
larger graduate community.
Not
4. STUDENT LEARNING ASSESSMENT Met
Met
The faculty documented or proposed indicators of student learning success that are
4.1
keyed to the learning objectives of the program.
4.2 The faculty documented or proposed assessments of student learning that are
grounded in best practices and appropriate comparisons.

4.3 The faculty documented or proposed a periodically and systematically reviewed

plan for using student learning assessments that leads to continuous improvements
in the program.
The faculty documented or proposed a continuous improvement plan that
4.4
incorporates multiple measures to assess student learning and program
effectiveness.
Not
5. QUALITY ASSURANCE Met
Met
There is an evident commitment to making continuous quality improvements in
the program (e.g., student advisement and mentoring, use of best practices,
5.1
recognition of faculty performance, regular and systematic evaluation of student
performance) a top priority.
The faculty documented or proposed ways to ensure that quality assurance will be
5.2
a systematic and regular process for program improvement.
The coursework offers sufficient breadth and depth appropriate for the degree
5.3
offered.
The faculty are documented to hold terminal degrees in the discipline in which
5.4 they teach and have experience sufficient to serve as effective mentors for
graduate students.
Data on current students and follow-up data on graduating students including
5.5
placement data are regularly and systematically collected.
Not
6. RESEARCH OUTCOMES Met
Met

The faculty documented or proposed a plan to ensure that there is a commitment to

6.1
matching or exceeding comparable institutions in research activities.
The faculty documented or proposed a plan to assure sufficient depth and breadth
6.2 in faculty research expertise to enable competitiveness in the external funding
arena while allowing for collaboration when desired.
Not
7. RESEARCH ENVIRONMENT Met
Met
The faculty documented or proposed a plan to ensure a commitment to
7.1
communicate the program’s research environment, research values and priorities.
The faculty candidly and thoroughly examined the extent to which the department
7.2 describes itself accurately and completely to current and prospective students and
other “publics”.

Page 22
 A Guide for the Academic Auditor 2012-13...

The department takes active steps to support both junior and senior faculty in
7.3
remaining vital in their respective research areas.
The faculty documented or proposed a plan to ensure that departmental processes,
7.4 policies and procedures positively influence faculty research activities and
program competitiveness.
The faculty documented or proposed a plan that engages graduate students in
7.5
inquiry and research in collaboration with faculty.
Not
8. SYNERGY WITH EDUCATION Met
Met

The faculty documented or proposed a plan that honestly evaluates departmental

resource demands in light of departmental research and scholarship’s contribution
8.1
to its educational programs and the mission of the department, college and
university.
There is a commitment to activities designed to keep the faculty and students
8.2 informed on contemporary issues related to research (e.g., lecture series,
responsible conduct of research workshops, professional development activities).
The program demonstrates best practices in integrating the science with the
8.3
practice of the discipline.
The faculty candidly and thoroughly evaluated the extent to which they
8.4
incorporate research into the educational programs in support of best practices.
The program demonstrates best practices in addressing workload demands of
8.5
theses and dissertations supervision.
Not
9. SPONSORED PROGRAMS Met
Met
The faculty documented and proposed a plan to strive for sponsored research
9.1 funding at comparable levels with other comparable departments within the
institution and across peer institutions.
The faculty documented or proposed a plan to assure that faculty are consistently
9.2 informed of external funding opportunities as well as the availability of assistance
in areas such as proposal writing and project management.
Not
10. QUALITY AND PRODUCTIVITY INDICATORS Met
Met
There are appropriate indices endorsed by the program faculty as means to gauge
10.1
faculty quality and productivity.
The faculty documented or proposed ways to ensure that quality research and
10.2 productivity will be systematically and regularly examined across the faculty
lifespan.
The culminating experience required by the program both in terms of
comprehensives examination and/or research allows the student to demonstrate the
10.3 breadth, depth and integration of the disciplinary coursework and experiences with
the demonstration of communication skills and the ability to apply knowledge
independently.
Not
11. CONTRIBUTIONS TO PROGRAM, DEPARTMENTAL AND UNIVERSITY GOALS Met
Met
There is a process in place which is communicated to other levels of the institution
that evaluates the sufficiency of resources in place to meet the teaching
11.1
responsibilities while actively engaging in research with graduate students and
undergraduates.
11.2 The faculty documented and proposed a plan to encourage and support research

Page 23
 A Guide for the Academic Auditor 2012-13...

outcomes congruent with the department’s purpose and the university mission.
The faculty clearly state and embrace appropriate admission standards, completion
11.3 standards and graduation rates which are readily available to prospective and
current students.
Not
12. OVERALL ASSESSMENT Met
Met
12.1 The Academic Audit process was faculty driven.
The Academic Audit process (self-study and visit) included descriptions of the
12.2
program’s quality processes.
The process resulted in a candid description of weaknesses in program processes
12.3
and suggestions for improvements.
Overall, the visiting team affirms the openness and thoroughness of the program
12.4
faculty in completing the academic audit of this program.
Not
13. SUPPORT Met
Met
The program regularly evaluates its library, equipment and facilities, encouraging
13.1
necessary improvements within the context of overall college resources.
13.2 The program's operating budget is consistent with the needs of the program.
The operating budget is sufficient to attract quality students and provides adequate
13.3
support without substantially delaying progress toward the degree.
The program has a history of enrollment and graduation rates sufficient to sustain
13.4
high quality and cost-effectiveness.
The operating budget is sufficient to allow faculty regular opportunities for
professional development including travel and presentation of research findings,
13.5
participation in professional organizations, workshops and other learning
activities.
Not
14. FOLLOW-UP OF PREVIOUS ACADEMIC AUDIT Met
Met
14.1
An action plan was developed as a result of the previous Academic Audit.
There is documented evidence that Recommendations made by the Academic
14.2 Auditor Team have been considered and, when feasible and appropriate,
implemented and tracked.
There is documented evidence that the program has implemented and tracked the
14.3 progress of and use of results from Improvement Initiatives cited by the faculty in
its self study.

Page 24
 A Guide for the Academic Auditor 2012-13...

Appendix H: Follow-up of Previous Academic Audit

During the 2010 – 2015 Strategic Planning and Performance Funding cycles, many academic
programs will undergo the Academic Audit process for the second time. The expectation of a program
undergoing the Academic Audit process for the second time is that it has acted upon the Initiatives for
Improvements that it set for itself after its first Academic Audit Self Study. Also, that it has responded to
the Recommendations made by the Academic Auditor Team that reviewed the program in its first
Academic Audit experience. Following is the text from the Academic Audit Handbook that provides
guidance to programs undergoing the Academic Audit process a second time regarding reporting its
follow-up processes and activities:

6. Follow-up of Previous Academic Audit [Only for programs undergoing the academic audit process for
the second and subsequent times. Not to exceed 5 pages.]

This section of the self-study reflects upon two outcomes of the previous academic audit process: 1)
Initiatives for Improvement that were generated by the program itself; and 2) Recommendations by the
Academic Auditor Team. The expectation is that the program will have acted in a number of ways to
advance improvement in these areas. Thus, a brief discussion of the process that the program followed to
develop and enact its action plan is appropriate here.

It is expected that the Initiatives for Improvement that were generated by the program itself have
been implemented and that progress on that initiative has been tracked. Further, it is expected that the
results of those implementations have been considered and used to further improvement in the program.
If the program has a detailed summary report for each initiative or a series of annual reports for each
initiative, these do not need to be reiterated in the self-study report. Rather, those reports may be
included as appendices. However, a brief narrative describing the process followed by the program to
initiate and follow through with its initiatives would be appropriate in the self-study as would be a
summary of the effectiveness of the initiatives.

Regarding Auditor Team Recommendations, the self-study report should address how each
recommendation was considered by the faculty. If a recommendation was not pursued, then a brief
explanation why it was not pursued should be included. If it was pursued as written or pursued in a
modified manner, a similar narrative about its inception, follow-through, and effectiveness would be
appropriate in the self-study. As with Initiatives for Improvement, if the program maintained detailed
annual, periodic or summary reports regarding the implementation of a Recommendation, then these
reports could be provided in the Appendix.

Page 25
 A Guide for the Academic Auditor 2012-13...

The Academic Auditor Team is responsible for reviewing and assessing the processes and activities
of the program regarding its Initiatives for Improvement as well as its responses to the initial Academic
Auditor Team’s Recommendations. As noted above, “It is expected that the Initiatives for Improvement
that were generated by the program itself have been implemented and that progress on those initiatives
have been tracked. Further, it is expected that the results of those implementations have been considered
and used to further improvement in the program.” Auditor Team members should carefully review the
self study report and appendices for evidence that these expectations Initiatives for Improvement have
been addressed.

For Recommendations made by the initial Academic Auditor Team, the program may have acted
upon these. If the program has implemented a Recommendation, that implementation should be
documented in a fashion similar to the documentation of an Initiative for Improvement. If a
Recommendation has not been acted upon, it is expected that, “a brief explanation why it was not
pursued should be included.”

For programs undergoing the Academic Audit for the second time and are using the Academic
Audit to meet Performance Funding requirements, the Academic Auditor Team must complete Section 7
of the Undergraduate Summary Sheet for undergraduate programs or Section 14 of the Graduate
Summary Sheet for graduate programs. The three criteria in these sections provide guidance for the
overall review of the program’s follow-up of its previous Academic Audit.

Page 26
 A Guide for the Academic Auditor 2012-13...

REFERENCES

Dill, D. (2000). Designing academic audit: lessons learned in Europe and Asia. Quality in Higher
Education, 6, 187.
Dill, D., Massey, W., Williams, P. & Cook, C. (1996, September-October). Accreditation and academic
quality assurance: can we get there from here? Change, September-October, 16.
Jennings, J. (2002). Academic audit manual for use by the New Zealand universities academic audit
unit. Wellington, New Zealand: Wet Bock Education House.
Massy, W. (2003, June 20). Auditing higher education to improve quality, The Chronicle of Higher
Education, B16.
Massy, W., Graham, S. & Short, P. (2007). Academic Quality Work: A Handbook for Improvement.
Bolton, MA: Ankar Publishing Company.
Meade, P. & Woodhouse, D. (2000), Evaluating the effectiveness of the New Zealand academic audit
unit: Review and outcomes, Quality in Higher Education, 6 (1).
Tennessee Board of Regents. (2007). Educational Quality Improvement: A Handbook for the Academic
Audit. Nashville, TN: Author.

Page 27