ECPS 581 CURRICULUM EVALUATION

Winter T2 2014
Sandra Mathison sandra.mathison@ubc.ca

This course will examine issues, practices, and approaches for curriculum evaluation.
Fundamentally, curriculum evaluation focuses on two questions: Do planned courses,
programs, activities, and learning opportunities as developed and organized lead to desired
outcomes? How can curriculum be improved? ‘Curriculum’ has many meanings, but this course
will focus on applying the general logic of evaluation to a number of common conceptions of
curriculum: statements of educational goals and intentions (like professional standards, BC
Ministry IRPs); educational resources (like textbooks, software, online resources, simulations);
pedagogical/andragogical strategies (like cooperative groups, field trips, practica, dialogic
methods); and programmatic interventions (like reading programs, drug education, inquiry
based science) at both organizational and individual classroom levels. The course will cover
curriculum evaluation from an individual, pedagogical perspective AND from a large scale,
system-wide curriculum perspective.

Course Readings:

All course readings will be available to you through the UBC Library, usually online, or on my
blog, E-Valuation http://blogs.ubc.ca/evaluation/ The UBC Education Library has a curriculum
collection that may be a useful source of curriculum examples for completing the course
requirements. Readings are listed on the syllabus, but others may be added as we progress
through the course.

Course Requirements:

! Indicates a date on the course schedule when an assignment is due.

1) Students are expected to attend all classes, complete assigned reading, and participate in
class discussion and activities. Throughout the term we will focus on curriculum evaluation case
examples, which will involve group work in almost every class meeting. It will be necessary for
you to review the material for each of the case examples, in addition to doing the assigned
reading for each class. (10% of grade)

2) Review of prior knowledge to inform curriculum evaluation (20%)

Understanding what is already known in the curricular area facilitates planning and doing an
evaluation of a particular curriculum. Understanding various points of view, generally accepted
wisdom, and strategies used in evaluation is helpful. Using at least 5 sources, write a short
review of what is generally known about a curriculum you might be interested in evaluating.
Include what research and evaluation suggests works and doesn’t work, advantages and
disadvantages, any advice from prior evaluations. Your review should conclude with a bulleted
list of the key things you have learned that will be useful in future evaluations of this sort of
curriculum.
MAX 5 pages, excluding references, use APA style. http://wiki.ubc.ca/images/6/6f/Apastyle.pdf
Due 2/4
3) Evaluating curriculum content (20%)

Identify a curriculum that interests you—this should be a real example and one you can access.
Your purpose is to evaluate the content of the curriculum. To do so you will need to choose
criteria, indicating what the source is. If the curriculum is broad you might want to focus on a
subset of goals, intentions, topics. The following questions can be starting points for
consideration:
• Where does the content come from (e.g., research materials, references)? Are these
reputable sources (e.g., professional standards; research evidence)?
• Are activities represented accurately in pictures, graphs, and written text?
• Are data, information, and sources of information up to date and accurately interpreted?
• Are accurate and appropriate terminology used (e.g., “physical education class” versus
“gym class”)?
• Are information, examples, scenarios, etc., relevant to students’ lives?
Due 3/18

4) Evaluating learning outcomes (20%)

Choose a curriculum; include a brief description (no more than a paragraph).

! Identify 3 learning outcomes (these can be cognitive, affective, behavioral) for this
curriculum.
! Identify or develop 2 strategies for assessing each of the learning outcomes you have
identified, i.e., 6 strategies. Include an explanation of why you think these are good
strategies for evaluating learning outcomes. These strategies should include a mix of
already extant measures, adapted measures, and ones you develop. Provide references
were appropriate.
Note: this assignment is probably best presented in a table or figure format, rather than as an
essay.
Due 4/8

5) Curriculum evaluation project (30%)

The BC Ministry of Education is currently revising the K-12 curriculum.

https://curriculum.gov.bc.ca/curriculum
Any new or substantially revised curriculum should have an evaluation plan. Using the
framework for curriculum evaluation developed in this course, prepare a plan for evaluating the
new BC curriculum that you would propose to the Ministry. You can provide a holistic plan
covering all curricular areas OR choose a specific subject area (social studies, science,
mathematics or language arts).

Write the plan to the Ministry. It should communicate clearly why it is important to have an
evaluation plan and a detailed blueprint for conducting the curriculum evaluation.
Due 4/15

NOTE ON LATE WORK: It is my discretion whether to accept late assignments. If you anticipate
you will not have an assignment completed on time you must speak with me in advance.

2
Course Schedule:

1/7 introduction

1/14 basic concepts in evaluation

logic of evaluation
stakeholders

Read:
Mathison, Evaluation
Greene, Stakeholders & Stakeholder involvement
Scriven, Logic of Evaluation
Fournier, Working Logic of Evaluation

Greene, Scriven & Fournier readings can be found in Encyclopedia of Evaluation, which is available online
in the UBC Library

1/21 basic concepts in curriculum

what is curriculum?
politics of curriculum and evaluation
who decides & how

Read:
Teitlebaum, Curriculum
Pingel, UNESCO Guidebook on Textbook Research and Textbook Revision (main body
of paper)
Hamilton, Making sense of curriculum evaluation: Continuities and discontinuities in an
educational idea

CASE EXAMPLE: Be a Creator

1/28 components of curriculum evaluation

content
process
principles of learning/pedagogy
teaching & learning environment
activities, experiences
learning outcomes

identifying stakeholders
using prior knowledge about similar evaluands

Read:
Clements, Curriculum research: Toward a framework for research-based curricula

CASE EXAMPLE: HIV / STI and Sexuality Education Curriculum Evaluation Tool

2/4! focusing curriculum evaluation

questions
change

3
 claims/concerns/issues
success/failure

credible evidence for demonstrating quality

data sources
data collection strategies

Read:
Mathison, Reflections on Evaluating Innovative Curriculum Projects
Russ-Eft, Evaluation Questions in the Encyclopedia of Evaluation
Donaldson, In Search of a Blueprint for an Evidence-Based Global Society

CASE EXAMPLE: UCSMP Transition Mathematics Evaluation Study

2/11 evaluation design issues

needs assessment
focusing on particulars
appropriate comparisons
relationship to accreditation/institutional reviews
designs
naturalistic designs objectives focused
action research comparison focused
experimental designs
internal & external evaluation
evaluation clearinghouses

Read:
Mathison, What’s the difference between evaluation and research?
Mathison, Internal evaluation, historically speaking
Schug, Curriculum Evaluation: Using National League for Nursing Accrediting
Commission Standards and Criteria

CASE EXAMPLE: UCSMP Transition Mathematics Evaluation Study

CASEL Guide to Effective Social and Emotional Learning Programs
Graphite.org

2/18 NO CLASS ~ READING WEEK

2/25 Evaluating content

accuracy
age/developmental appropriateness
coherence, sequencing, alignment, relevance
usability (readability, organization, vocabulary)
acceptability (values, biases, balance, diversity, legality)
affordability
Relevant comparisons
discipline based
professional curriculum standards
professional practice standards
learner centered

4
 interdisciplinary
research literature

Read:
Pingel, UNESCO Guidebook for Textbook Research and Textbook Revision (Annex A)
AAAS Project 2061 Textbook Content Analysis
http://www.project2061.org/publications/textbook/hsbio/report/about.htm (this is a link to
the biology textbook analysis and while you need not read everything here, explore the
examples enough to get a solid sense of the AAAS evaluation process)

CASE EXAMPLE: Stern, L. & Roseman, J. (2004). Can middle-school science textbooks
help students learn important ideas? Journal of Research in Science Teaching, 41(6),
538 - 568

3/4 Evaluating content

bias

Read:
Guidelines for Identifying Bias in Curriculum Materials, available at
http://www.safeschoolscoalition.org/guidelinesonbias-print.pdf
Bias Evaluation Instrument, Nova Scotia Department of Education

CASE EXAMPLE: Tuscon Unified School District, Mexican American Studies curriculum

3/11 Analyzing principles of learning/pedagogy/andragogy

how learning occurs
role of teacher and learner
motivation
assessment

Read:
Smith, Andragogy
Koulaidis & Tsatsaroni, Pedagogical analysis of science textbooks: How can we
proceed?
Shulman, Signature pedagogies in the professions
Cogill, Pedagogy and Models of Teacher Knowledge
This website will give you a quick review of theories of learning
http://www.learning-theories.com/

CASE EXAMPLE: Reading Curricula

3/18 ! Evaluating the learning environment

feasibility within the context
professional development
administrative support
access for all students

Read:

5
 Burgstahler, Universal design of instruction
http://www.washington.edu/doit/Brochures/Academics/instruction.html
Watch this very short video: http://nzcurriculum.tki.org.nz/Curriculum-stories/Media-
gallery/Effective-pedagogy/Universal-design-for-learning
School resources/administrative context http://www.horizon-
research.com/horizonresearchwp/wp-content/uploads/2013/04/math_program.pdf
Mathison, An Evaluation Model for Teacher Inservice Programs

3/25 Evaluating process

instructional strategies
activities
interactions

Read:
Asking Students about Teaching
http://www.metproject.org/downloads/Asking_Students_Summary_Doc.pdf
Asking Teachers about Curriculum http://www.horizon-
research.com/horizonresearchwp/wp-content/uploads/2013/04/interview.pdf
Classroom Observation Protocol http://www.horizon-
research.com/horizonresearchwp/wp-content/uploads/2013/04/cop1.pdf

4/1 Evaluating learning outcomes

declarative knowledge, procedural knowledge, performative
knowledge/skill, critical analysis, attitudes/habits of mind, uses of
information/technology
direct & indirect techniques

Read:
Earl, L. (2003) Assessment of learning, for learning and as learning. Using Classroom
Assessment to Enhance Student Learning. Thousand Oaks: Corwin Press.
Principles of Good Practice for Assessment of Student Learning available at
http://www.academicprograms.calpoly.edu/pdfs/assess/nine_principles_good_practice.p
df
Assessment available at http://eduscapes.com/instruction/8.htm#b

CASE EXAMPLE: D.A.R.E. (Drug Abuse Resistance Education)

4/8! Evaluating learning outcomes (continued)

6
Readings

Clements, D. H. (2007). Curriculum research: Toward a framework for research-based curricula.

Journal for Research in Mathematics Education, 38(1), 35-70. Available at
http://gse.buffalo.edu/fas/clements/files/clements_crf.pdf

Codgill, J. Pedagogy and models of teacher knowledge. http://juliecogill.com/Chapter_2.pdf

Donaldson, S. (2009). In search of a blueprint for an evidence-based global society, in What

counts as credible evidence in applied research and evaluation? Thousand Oaks, CA:
Sage. http://www.cgu.edu/PDFFiles/sbos/Donaldson_Credible_Evidence_1.pdf

Hamilton, D. (1977). Making sense of curriculum evaluation: Continuities and discontinuities in

an educational idea. Review of Research in Education, 5(1), 318 - 347.

Koulaidis, V. & Tsatsaroni, A. (1996). Pedagogical analysis of science textbooks: How can we
proceed? Research in Science Education, 26(1), pp. 55-71.

Mathison, S. (2011). Internal evaluation, historically speaking. In B. B. Volkov & M. E. Baron

(Eds.), Internal evaluation in the 21st century. New Directions for Evaluation, 132, 13–23.

Mathison, S. (2007) What is the difference between evaluation and research? And why do we
care? In N. L. Smith & P. Brandon (Eds.). Fundamental issues in evaluation. New York:
Guilford Publishers.

Mathison, S. (1996). Evaluation as a democratizing force in schools. International Journal of

Social Education, 11(1).

Mathison, S. (1994). Evaluation. In A. C. Purves (Ed.). Encyclopedia of English studies

language arts. Champaign, IL: NCTE & Scholastic Inc.

Mathison, S. (1992). An evaluation model for in-service teacher education. Evaluation and
Program Planning, 15(3), 255-261.

Pingel, F. (2010). UNESCO Guidebook on Textbook Research and Textbook Revision. Paris:
UNESCO. Available at http://unesdoc.unesco.org/images/0011/001171/117188e.pdf

Schug, V. (2012). Curriculum evaluation: Using national league for nursing accrediting
commission standards and criteria. Nursing Education Perspectives, 33(5), 302-305.

Shulman, L. (2005). Signature pedagogies in the professions. Daedalus, 134(3), pp. 52-59.

Smith, M. K. (2010) ‘Andragogy’, the encyclopedia of informal education.

http://infed.org/mobi/andragogy-what-is-it-and-does-it-help-thinking-about-adult-learning/

7
 Grading Guidelines

A level - Good to Excellent Work

A+ (90-100%) A very high level of quality throughout every aspect of the work. It shows the
individual (or group) has gone well beyond what has been provided and has extended the usual
ways of thinking and/or performing. Outstanding comprehension of subject matter and use of
existing literature and research. Consistently integrates critical and creative perspectives in
relation to the subject material. The work shows a very high degree of engagement with the
topic.

A (85-89%) Generally a high quality throughout the work. No problems of any significance, and
evidence of attention given to each and every detail. Very good comprehension of subject and
use of existing literature and research. For the most part, integrates critical and creative
perspectives in relation to the subject material.
Shows a high degree of engagement with the topic.

A- (80-84%) Generally a good quality throughout the work. A few problems of minor
significance. Good comprehension of subject matter and use of existing literature and research.
Work demonstrates an ability to integrate critical and creative perspectives on most occasions.
The work demonstrates a reasonable degree of engagement with the topic.

B level - Adequate Work

B+ (76-79%) Some aspects of good quality to the work. Some problems of minor significance.
There are examples of integrating critical and creative perspectives in relation to the subject
material. A degree of engagement with the topic.

B (72-75%) Adequate quality. A number of problems of some significance. Difficulty evident in

the comprehension of the subject material and use of existing literature and research. Only a
few examples of integrating critical and creative perspectives in relation to the subject material.
Some engagement with the topic.

B- (68-71%) Barely adequate work at the graduate level.

C & D level - Seriously Flawed Work

C (55-67%) Serious flaws in understanding of the subject material. Minimal integration of critical
and creative perspectives in relation to the subject material. Inadequate engagement with the
topic. Inadequate work at the graduate level.

D level
D (50-54%)

F level - Failing Work

F (0-49%)
______________________________________________________________________

NOTE: For UBC’s Faculty of Graduate Studies (FOGS), a final mark below 68% for
Doctoral students and below 60% for Masters students is the equivalent of a Failing mark.
_____________________________________________________________________

8
Plagiarism
Plagiarism, whether intentional or unintentional, is a form of cheating that can lead to a failing
grade for the course and to suspension from the University. As defined within UBC policies
(http://www.vpacademic.ubc.ca/integrity/policies.htm), and as outlined in the UBC Calendar,
plagiarism is a serious “form of academic misconduct in which an individual submits or presents
the work of another person as his or her own”. As a form of intellectual theft, plagiarism involves
taking the words, ideas or research of another without properly acknowledging the original
author. Students need to become familiar with the many different forms that plagiarism can take,
including accidental and intentional plagiarism. For more information see
http://www.arts.ubc.ca/Plagiarism_Avoided.373.0.html OR
www.library.ubc.ca/home/plagiarism/for-students.doc OR
http://www.indiana.edu/~wts/pamphlets/plagiarism.shtml)

Please take care to acknowledge your sources, including the Internet, using APA Style
(American Psychological Association).

Non-sexist Language
Please incorporate and use non-sexist language [also called gender inclusive language] in your
oral and written language. This language positions women and men equally, it does not exclude
one gender or the other, nor does it demean the status of one gender or another. It does not
stereotype genders [assuming all childcare workers are female and all police officers are male],
nor does it use false generics [using mankind instead of human kind, or using man-made
instead of hand crafted]. In addition, this language requires an attention to gender balance in
personal pronouns, for example, use "he and she" rather than "he" or balance gendered
examples in a paper, referring to both male and female examples. You may also recast subjects
into the plural form, e.g., when a student raises his hand Š when students raise their hands.

Person First Language

Please incorporate and use person first language in your oral and written language. Disabilities
and differences are not persons and they do not define persons, so do not replace person-
nouns with disability-nouns. Avoid using: the aphasic, the schizophrenic, stutterers, the hearing
impaired. Also avoid using: cleft palate children, the hearing impaired client, the dyslexic lawyer,
the developmentally disable adult. Instead, emphasize the person, not the disability, by putting
the person-noun first: the lawyer who has dyslexia, persons who stutters, the children described
as language impaired, the teacher with a hearing impairment.

Students with Disabilities

We strive to include all students, including those with special learning needs in this
course. Please let us know (or have the UBC Disability Resource Center let us know) if you
have a disability documented with the UBC Disability Resource Centre and/or if you need any
special accommodations in the curriculum, instruction, or assessment of this course to enable
you to fully participate. We adhere to UBC Policy 73: Academic Accommodations for Students
with Disabilities. This information is located at: http://www.students.ubc.ca/access/drc.cfm. We
will respect the confidentiality of the information you share and work with you so your learning
needs are met.