Jennifer K. Clayton, Ph.D.

Old Dominion University CREATE Conference October 2009

 Need

to ensure parity to face-to-face programs (Martinez, Liu, Watson, & Bichelmeyer, 2006) focus on individual courses rather than programs (Rovai, 2003) leadership programs notoriously engage in program evaluation only during accreditation requirements (Glasman et al., 2002)

Historically

Educational

 Quantity Quality Status

of learning

of learning

of learning

Relative

cost of learning

(Keegan, 1996)

 Three

main theories of evaluation

Use

Methods

Valuing

 Management-oriented

model that stresses the collection of relevant information that is analyzed and presented in useful manner between evaluation of an educational system and an instructional program

Distinction

Focus

on how the information will be used in decision-making

Mid-Atlantic research state university Masters program in Educational Leadership Program in transition to fully distance learning model through closed circuit televised classes and web based courses using Adobe Connect
Year Enrollment (Matriculated Students) 56 62 60 67

2005 2006 2007 2008

 Prepares Most

students for positions as PK-12 administrators are teachers or PK-12 employees

Sample

included those students who had taken both televised and web-based courses
seeks to be competitive in terms of both quality of instruction and quality of students

Program

 UCLA

Evaluation developed in 1970 by Marvin Alkin of the Center for the Study of Evaluation

Includes Needs assessment Program planning Implementation evaluation Program improvement Program certification

Implementation Evaluation Degree to which the program has actually followed the plan Program Improvement Examines outcomes of the program Examines current status of the program as it functions to meet the needs of those it serves

Program Certification Looks for applicability to other programs

 What

is the acceptance rate for students to the program?

Examination of data for number of applicants and

number of acceptances as maintained by department

What

is the geographic makeup of students enrolled in the program?

Examination of data for student home address as

maintained by department

 What

information does the department need to provide to graduate students in their handbook to increase successful navigation of the distance-learning delivery methods?
Online survey to students Student interviews Faculty interviews

 What

mode of course delivery do students prefer?

Online student survey question Follow up questions in student interviews

What

is the graduation rate for students from the program?

Percentage based upon number of students enrolled

and number of degrees conferred in each term derived from university data system

Faculty have worked to address behavioral issues at televised sites or have developed alternatives
A web-based resource has been established to provide information to students within the department including writing, licensure, handbooks, technology assistance More resources have been implemented including a web based course orientation required for all new students and a communication structure with computing technical services

Model used only in part Challenging to fit this model as decisions had already been made for the program

Seeking to use new and evolving distance learning models for comparative research with face-to-face programs
Desire by faculty to find a model that encompasses more about student characteristics for successful distance learning

 Main

focus of distance learning is often accessibility (Muller, 2008) evaluations will need to examine more fully if the programs have expanded accessibility and if these students are well-served and achieve completion of program

Future

the worth of an evaluation model should be gauged by the extent to which it leads to improving educational programs in addition to making summative decisions as to their general worth.
(Klein, Fenstermacher, & Alkin, 1971)

Jennifer K. Clayton, Ph.D. 757-683-3333 jclayton@odu.edu