Evaluation Model

Evaluation Model
• Ralph Tyler - Curriculum should be organised
around objectives therefore provide the
criteria for evaluation.
• Daniel L. Stufflebeam - CIPP
• Michael Scriven – Goal free evaluation
• Donald Kirkpatrick (1994) – Four levels of
Evaluation
• Malcolm Baldridge – Model of Quality
 Ralf Tyler
• Tyler contended that the curriculum should be
organised around objectives and that these
objectives should serve as a basis for planning
instruction. Objectives would therefore
provide the criteria for evaluation.
 Daniel L. Stufflebeam
• The CIPP model proposed by Stufflebeam was also a
reaction to Tyler's approach which concentrated on the
final outcomes of a learning program. This model
requires evaluators to study the effectiveness and
feasibility of the initial policy decisions that gave rise to
the programme as well as the programme's operation.
- Context of the programme
- Input into the programme
- Process within the programme
- Product of the programme
 Michael Scriven
• Goal-free evaluation is the title of Scriven's
model.
- The evaluator would be external to the
programme and would record what seemed to
be the main effects of the programme.
- The importance of this model is that it
acknowledges the significance of the
unplanned results of learning.
 Donald Kirkpatrick (1994)
Kirkpatrick's Hierarchy of Evaluation comprises 4
levels.
• At level 1 is the reaction of the participants.
• At level 2 is the actual learning that has occured.
• At level 3 is the transference of learning from
the learning situation to real life settings.
• At level 4 is the impact of the programme on the
wider community.
 Malcolm Baldridge
• The Baldrige management quality model
provides a proven approach to performance
management that promotes understanding
and improving the value delivered to
whomever your organization serves, overall
organizational effectiveness and capabilities,
and organizational and personal learning.
Model Penilaian “Quasi-Legal” : Adversary Model

• Pro and Con / Penilaian Positif dan Negatif & Hakim

dan juri / Defendan
• Diperkenalkan Rice (1915), Guba (1965), Wolf (1973,
1975, 1990)
• “Adversary evaluation is distinguished by the use of
wide array of data; the hearing of testimony; and,
most importantly; an adversarial approach, meaning
that the two sides present positive and negative
judgments, respectly, about the program evaluated”
(Wolf, R.L 1975 dipetik dari Gall et. al. 2003:566)
 Adversary Model

• 4 tahap / peringkat
– Mengenalpasti isu dalam sesuatu program secara umum dengan
membuat tinjauan terhadap “stakeholders”(Generating a broad range
of issues concerning the program by surveying various stakeholders)
– Memilih isu mengikut keutamaan (ranking) oleh sukarelawan
– Dua pasukan penilaian dibentuk untuk mengmukakan fakta masing-
masing
– Pra-pembentangan dan pembentangan kes. Ia dinilai juri untuk
membuat keputusan.
• Digunakan untuk menilai “minimum competency testing program” yang
dilaksanakan di National Institute of Education, A.S
(Gall et. al 2003)
Contoh Kajian Adversary Model
Tajuk:
Learning Through Advocacy: An Experimental Evaluation o
f an Adversary Instructional Model
Autor: Marilyn Kourilsky
Sumber: Journal of Economic Education; Spring 72, Vol. 3
Issue 2. Heldref Publications
Kourilsky , Marilyn. 1972. Learning Through Advocacy: An Experimental Evaluation
of an Adversary Instructional Model. Journal of Economic Education; Spring 72, Vol.
3 Issue 2: 86-94. (atas talian) http://web29.epnet.com/citation.ap? (29 Ogos 2005)
 Model Penilaian “Quasi-Legal” : Judicial Model

• Diperkenalkan Wolf (1973, 1975)

• The judicial evaluation model simulates the use of legal
procedures for the purpose of promoting broad
understanding of a program, clarifying the subtle and
complex nature of the educational issues it raises, and
producing recommendations and policy guidelines that
lead to institutional growth and / or improved practice
(Wolf, R.L 1990 dipetik dari Gall et. al. 2003:567)

• Ia tidak melibatkan perbahasan antara dua pasukan

penilai.
• 1977 Penilaian “team teaching” di Hawai oleh
Northwest Regional Education Laboratory.
 Judicial Model
• Proses penilaian “judicial”
– Public presentation of the data is made. Following the
format of hearing in a court law.
– A panel comprised policy makers, citizens, and other
interested stakeholders is convened to hear the evidence.
– Case presenters call witnesses who present their views in
order to make case elative to a given issue
– Direct or cross examination on witnesses by two case
presenter
– Opening and closing arguments are presented
– Finally, the panel deliberates and makes its
recommendation
(Gall et. al 2003)
 Model “Illuminative“
• Diperkenalkan oleh Parlett dan Hamilton 1976

“illuminative evaluation takes account of the wider

contexts in which educational programs
function. Its primary concern is with description
and interpretation rather than measurement
and prediction. Its stand unambiguously within
the alternative anthropological paradigm.
(Parlett dan Hamilton 1976, dipetik dari Clarke dan Dawson 1999: 55)
 Model Expertise-Based (Flexner)
• Diperkenalkan mulai tahun 1800. di Amerika Syarikat. Flexner
1910 membuat penilaian sekolah perubatan di Amerika
Syarikat dan Canada.
• “Time and time again it has bee shown that an unfettered lay
mind is ... best suited to undertake a general survey… The
expert has his place, to be sure; but if I were asked to suggest
the most promising way to study legal education, I should
seek a layman, not a professor of law; or for the sound way to
investigate teacher training, the last person I should think of
employing would be a professor of education”
(Flexner 1960 dipetik dari Fitzpatrick et. al. 2004:115)
 Model Expertise-Based
• Menurut Fitzpatrick et al. 2004 terdapat 4
jenis penilaian;

– Formal Review System

– Informal Review System
– Ad hoc panel review
– Ad hoc individual review
 Formal Review System
• Dilaksanakan secara berstruktur oleh organisasi untuk melaksanakan secara
berkala.
– North Central Association (NCA)
– National Council for the Accreditation of Teacher Edcation (NCATE)
• Merekabentuk piawaian dan instrumen untuk digunakan dalam semakan /
kajian semula
• Perlaksanaan secara berkala yang tetap.
• Menggabungkan pendapat pakar untuk melihat “overall judgments of
value”
• Memberi kesan status kepada kajian semula bergantung kepada “outcome”.
• Dikendalikan badan akreditasi yang dilantik.
 Informal Review System
• Site-visit teams mengendalikan penilaian
• Mereka dilantik oleh pembiaya
• Membangunkan prosedur penilaian tersendiri tanpa
panduan.
• Penilaian dalaman
– Contoh: Badan penyeliaan pelajar dilantik yang
terdiri dari pakar dalam bidang berkenaan dilantik.
 Ad hoc panel review
• “One-shot” evaluations
• Funding Agency Review Panels @ Peer-review
panels to review competitive proposals
• Blue-Ribbon Panels ( National Commission on
Excellence in Education )
– Dilantik disebabkan kepakaran dan pengalaman
– Particular situation
– Documenting their observation
– Making recommendation for action
 Ad hoc individual review
• Pakar dalam bidang tertentu.
• Perkara yang tidak dilihat oleh “Site-visit teams”
• Cth: menilai buku teks, program latihan, job-
placement test dan perancangan program.
• Tidak perlu datang ke “site”
Contoh Kajian Model Expertise-Based
Tajuk:
Medical education in China’s leading medical schools
Autor: M. ROY SCHWARZ, ANDRZEJ WOJTCZAK2 &
TONGFU ZHOU
Sumber: Medical Teacher, Vol. 26, No. 3, 2004, pp. 215–
222
Schwarz,M.R, Wojtczak, Andzej & Zou,Tongfu 2004. Medical education in
China’s leading medical.Medical Teacher, Vol. 26, No. 3, 2004, pp. 215–
222 (atas talian) ttp://web29.epnet.com/citation.asp? (29 Ogos 2005)
Contoh Laporan Model Expertise-Based

Tajuk: How We Rank Schools

Autor: Morse, Robert J., Flanigan, Samuel, & Yerkie,
Meadow
Sumber: U.S. News & World Report, 00415537, 4/12/2004,
Vol. 136, Issue 12

Morse, R.J, Flanigan, Samuel & Yerkeie, Meadow.2004. How We Rank SchoolsU.S.
News & World Report, 00415537, 4/12/2004, Vol. 136, Issue 12. 62 (atas talian)
ttp://web1.epnet.com/citation.asp? (29 Ogos 2005)
Model Connoisseurship- Criticism
• Diperkenalkan oleh Elliot Eisner 1975, 1999,
2002
• Penilaian pakar terutama dalam bidang seni.
• Kaedah fenomenologi .
• Meneliti apa berlaku seperti antropologi.
• Mementingkan ciri kualitatif, humanistik dan
non-saintifik
• Laporan seperti karya seni atau sastera.
 Model Connoisseurship- Criticism
• Connoisseurship is a term that traditionally has been
associated with phenomenon such as wine testing
and art appreciation rather than the supposedly
scientific and technical enterprise of formal
evaluation. Similarly the notion of criticism
traditionally has been associated with the
assessment of literacy or artistic works and not with
the assessment of education or the sorts of special
programs.
2005.Encyclopedia of Evaluation:76
 Model Connoisseurship- Criticism
• Terbahagi kepada tiga komponen dalam
pendidikan:
– Deskriptif
– Iterpretif
– Penilaian
2005.Encyclopedia of Evaluation
 Komponen Deskriptif
• Proses menilai kualiti program pendidikan dan
maknanya. Seperti apresiasi karya seni
– Contoh: kualiti wain = warna, rasa, rasa lepas minum, rupa
dan lain-lain.
• Literary language & metaphor to capture aesthetic
(feeling) dimensions of phenomenon being evaluated.
• To be able to vicariously experience the phenomenon
that is being evaluated at visceral, not just an
intelectual, level.
2005.Encyclopedia of Evaluation
 Komponen Deskriptif
• Tidak menekankan kepada yang lebih suka atau tidak
suka tapi kesedaran terhadap kualiti dan hubungan
antara perkara tersebut.
– Penilaian orang awam tidak diperlukan.
• Presentational not representational
• Representational = nonpoetic language and numbers
• Presentational = Meaning inherent in the symbol
itself
2005.Encyclopedia of Evaluation
 Komponen Interpretif
• Social sciences theory was to be employed in the
interpretive component of the educational criticism
process and the written products produced by that
process
• Sense-making process is the essence of the interpretive
component of educational criticism: the expectation is
that educational connoisseurs will use social science
constructs and theories eclectically to aid them and their
readers in interpreting the phenomenon that have been
observed and described.
2005.Encyclopedia of Evaluation
 Komponen penilaian
• “Education is normative enterprise and, consequently,
evaluation models that merely describe or interpret
what is happening are inadequate”
• “Evaluators in a normative field such as education need
to asses the worth of what they describe and interpret”
Eisner dipetik dari Encyclopedia of Evaluation 2005

• Kepakaran connoisseurs penting untuk membuat

penilaian.
• Penilaian dibuat dalam aspek yang luas berdasarkan
kepakaran penilai.
2005.Encyclopedia of Evaluation
 Contoh kajian
• Contoh Kajian McCutcheon (1978) penilaian
terhadap pelajar gred 4 di Amerika.
– Persoalan yang terbuka;
– Apa yang berlaku di sini?
– Adakah ia bernilai?
– Apakah dilaksanakan secara betul?
– Apakah kanak-kanak di sini dapat mempelajari
sesuatu?
(Gall et. al 2003)
 Performance Evaluation / Assessment
• Performance assessment affords us, in principle, an
opportunity to develop ways of revealing the
distinctive features of individual students. It affords
us an opportunity to secure information about
learning that can help improve the quality of both
curriculum and teaching. In short, it affords us an
opportunity to use evaluation formatively and to
treat assessment as an educational medium.
Eisner, Elliot W. 1999.
 Bentuk laporan Expertised-Based
• “Myriad sounds, smells, and sights greet the
newcomer to Mr. Clements’ room. The sequel of a
guinea pig and the scrabbling of rats in their wire
cages mingle with the voices of children as they
discuss their private lives and school work…………
The smells of guinea pigs, rats, clean wood
shavings, school disinfectant, and a freshly peeled
orange intermix. Randy is eating raisins. could
you share with me”.
(McCutcheon 1978 dipetik dari Gall et.al 2003:588 & 589)
 Bentuk laporan Expertised-Based
• “In this lesson, then, children had the opportunity to
learn many things-things about the solar system,
construction, visual problem-solving, self-control, and
social interaction. We might wonder, thought, wether
responsibility for decision-making, planning socially,
and self-control are worthwhile lessons…..don’t
children learn these responsibilities anyway-in the
home, the community, and school without so much
emphasis being placed upon them?”
(McCutcheon 1978 dipetik dari Gall et.al 2003:588 & 589)
 Bentuk laporan Expertised-Based
• “When school is seen as integral part of children’s
lives, children may be more likely to apply school
learning and to consider doing schoollike things
at home. A more unified life may make schooling
seem more relevant. The less formal setting of
this classroom and Mr. Clemente’s acknowledging
the existence of children’s personal interest may
work toward end”
(McCutcheon 1978 dipetik dari Gall et.al 2003:588 & 589)
 Contoh Kajian Expertised-Based

• Tajuk: School
effectiveness research: Criticisms and recom
mendations

• Autor: Robert Coe

• Sumber: Oxford Review of Education; Dec. 98,
ol.24 Issue 4, 421-438
Coe, Robert, 1998. School effectiveness research: Criticisms and Recommendations
Oxford Review of Education; Dec. 98, ol.24 Issue 4, 421-438 (atas talian)
http://web29.epnet.com/citation.asp? (29 Ogos 2005)
 Model Countenance
Diperkenalkan oleh Robert Stake 1967
• Menekankan 2 perkara iaitu “countenances”
iaitu “description” dan “judgment”
“Focus on potrayal and processing the
judgments of participants, was so alter
dramatically the thinking of evaluators in the
next decade”
Fitzpatrick et al. 2004
 Proses penilaian Model Countenance

• provides background, justification and description

of the program rationale (including its need)
• List intended antecedents (input, resources,
existing conditions), transactions (activities,
processes) and outcomes
• Record observed antecedents, transactions and
outcomes (including observations of unintended
features of each)
• Explicitly states the standards (criteria, expectation
performance of comparable programs) for judging
program antecedents, transaction and outcomes
Stake 1967 dipetik dari Fitzpatrick et al. 2004: 135
Proses penilaian Model Countenance

• Record judgment made about the antecedents

conditions, transaction and outcomes.
• The evaluator analyzes information in the description
matrix y looking at the congruence between intents
and observation, and by looking at dependencies
(congruence) of outcomes on transactions and
antecedent, and of transactions on antecedents.
• Judgments are made by applying standards to
descriptive data.
Stake 1967 dipetik dari Fitzpatrick et al. 2004: 135
 Kerangka Konseptual Model
DESCRIPTION MATRIX JUDGMENT MATRIX

Intents Observations Standards Judgments

Congruence
Antecedents
Contingencies

Contingencies
Rationale

Congruence Transactions

Congruence Outcomes

Stake 1967 dipetik dari Fitzpatrick et al. 2004: 135

 Contoh Kajian Model Countenance

• Tajuk:
Stake’s Countenance Model: Evaluating an En
viromental Education Professional Developm
ent Course
• Autor: Wood, Barbara Bonsall
• Sumber: Journal of Environmental Education,
Winter 2001, Vol. 32 Issue 2, 18 -27
Wood, Barbara Bonsall, 2001. Stake’s Countenance Model: Evaluating an Enviromental Education
Professional Development Course. Journal of Environmental Education, Winter 2001, Vol. 32 Issue 2, 18 -
27(atas talian) http://web29.epnet.com/citation.asp? (29 Ogos 2005)
 Model Responsif
• Diperkenalkan oleh Robert Stake 1972, 1975, 1978, 1980)
• Beliau mendapati terdapat kelemahan dalam Model Countenance
• Memberi fokus kepada masalah oleh stakeholders dan semua
partisipan.
“An educational evaluation if it orients more directly to program activities
than program to program intents; responds to audience requirements
for information; and if the different value perspectives present are
referred to in reporting the success and failure of the program”
• Model ini diperkembangkan oleh Guba & Lincoln untuk mencipta
“Fourth Generation Model
Stake 1975 dipetik dari Fitzpatrick et al. 2004
 Kerangka Konseptual/Proses
Talks with Clients, Program
Staff, Adiences

Assemble Formal Reports, If Identify Program Scope

Any

Winnow, Format for Overview Program Activities

Audience Use

Validate, Confirm, Attempt to Discover Purposes, Concerns

Disconfirm

Thematize: Prepare Portrayals, Case

Studies Conceptualize Issues,
Problems

Observed Designated, Antecedents,

Transactions and Outcomes Identify Data Needs, Re
Issues

Select Observes, Judges, Intruments

if Any

Stake 1975b dipetik dari Fitzpatrick et al. 2004:138

Kerangka Konseptual / Proses
Fasa 1. Menggalakkan dan mentadbir penilaian
Stakeholders dikenalpasti – pentadbir sekolah, guru, murid, keluarga,
pegawai pendidikan daerah dan pihak bertanggungjawab

Fasa 2. Mengenalpasti isu utama

Temuduga dilaksanakan terhadap stakeholders untuk mendapatkan
isu utama.
Contoh: pelaksanaan sistem berpusat atau tidak berpusat.

Fasa 3. Mengumpul maklumat penting

Penilai memungut pelbagai data mengenai isu yang dipilih
Metod – pemerhatian secara naturalistik, temuduga, soal selidik dan
ujian standard

Fasa 4. Membuat laporan keputusan secara berkesan dan

mengemukakan cadangan (recommendation)

(Guba dan Lincoln 1981 dipetik dari Gall et.al 2003)

 Contoh Kajian Model Responsive
• Tajuk:
Application of a responsive evaluation approach in
medical education
• Autor: Vernon Curran, Jeanette Christopher,
Francine Lemire, Alice Collins2 & Brendan
Barrett
• Sumber: Medical Education; March 2003, Vol.
37 Issue 3, 256 -266
Curran et. al. 2003. Application of a responsive evaluation approach in medical education , Medical Education;
March 2003, Vol. 37 Issue 3, 256 -266
(atas talian) http://web1.epnet.com/citation.asp?tb (29 Ogos 2005)
 Kekuatan
• Membantu audien memahami program jika penilai memberi tumpuan
kepada kaedah yang natural di mana audien lebih memahami dan boleh
berkomunikasi mengenai sesuatu perkara
• Ilmu didapati dari pengalaman.(tacit knowledge). Memberi bimbingan
terhadap kefahaman manusia dan menambah pengalaman.
• “Naturalistic generalizations”. Dibina dengan melihat kesamaan objek
atau isu di dalam atau di luar kontek melalui pengalaman.
Mengukukuhkan kefahaman dan persepsi audien terhadap sesuatu
program.
• Dengan melihat kepada ‘single’objek, membandingkan pengalaman
tersebut dengan objek lain. Berdasarkan pengalaman sebenar.
 Model Naturalistik
• Diperkenalkan oleh Elspeth Huxely 1982
• “aims at naturalistic generalization (based on
the experience of the audience); is directed
more at non-technical audiences like
volunteers or the general public; uses ordinary
language and everyday categories of events;
and is based more on informal than formal
logic”
Huxely 1982 dipetik dari Fitzpatrick et al. 2004
 Contoh Kajian Menggunakan 3 Model

• Tajuk:
Workplace Literacy: Evaluation of Three Model Progr
ams
• Autor: Askov, Eunice N.
• Sumber: Adult Basic Education; Summer 2000,
Vol. 10 Issue 2, p100-107
 Proses Penilaian
• Descriptive information about the object of the
evaluation and its context
• Information responsive to concerns (documenting
concerns, seeking causes and consequences, and
identifying possible actions)
• Information responsive to issues (clarifying issues,
identifying potential courses of action to resolve them)
• Information responsive to values (clarifying issues, finding
out about their source and degree of conviction)
• Information about standards to be used in the evaluation
(identifying criteria, expectations, and needs)
Huxely 1982 dipetik dari Fitzpatrick et al. 2004