http://www.sii.soe.umich.

edu/
http://sitemaker.umich.edu/lm
t


1
Developing Measures of
Mathematical Knowledge for Teaching
Geoffrey Phelps, Heather Hill,
Deborah Loewenberg Ball, Hyman Bass

Learning Mathematics for Teaching
Study of Instructional Improvement
Consortium for Policy Research in Education
University of Michigan

MSP Regional Conference
Boston, MA
March 30-31, 2006


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


2
Overview of todays session
1. LMT/SII Measures Development

2. Some Sample Results

3. LMT/SII Measures and Dissemination


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


3
Subtract:
What is Content Knowledge for Teaching?
An Example From Subtraction
3002
783
-


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


4
Analyzing Student Errors
3002
783
-
2781
3002 - 783 = 4832


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


5
Analyzing Unusual Student Solutions
3002
783
-
299
2219
12
3 0 0 2
7 8 3
-
3-7-8-1
2 2 1 9
LMT/SII Measures Development


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


7
Why Would We Want to Measure
Teachers Content Knowledge for Teaching?
To understand what constitutes mathematical
knowledge for teaching
To understand the role of teachers content
knowledge in students performance
To study and compare outcomes of professional
development and teacher education
To inform design of teachers opportunities to
learn content knowledge


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


8
Measuring Teachers Mathematics
Knowledge: Background and History
Research on teacher behavior
Early research on student achievement
Proxy measures for teacher knowledge
Tests of basic skills
1985 on: the missing paradigm pedagogical
content knowledge or PCK
1990s: interview studies of teachers
mathematics knowledge (MSU -- NCRTE)


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


9
Study of Instructional Improvement
Study of three Comprehensive School Reforms;
teacher knowledge a key variable
Instrument development goals:
Develop measures of content knowledge teachers
use in teaching
K-6 content for elementary school teachers
Not just what they teach - specialized knowledge
Develop measures that discriminate among teachers
(not criterion referenced)
Non-ideological
But we faced significant problems.



http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


10
Problems As We Began This Work
No way to measure teachers content
knowledge for teaching on a large scale
Small number of items, many written by Ball,
Post, others appeared on every instrument
Nothing known about the statistical qualities of
those items (difficulty, reliability)
Studies relied on single items, yet single items
unlikely valid or reliable measures of teacher
knowledge



http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


11
Early Decisions and Activity
Survey-based measure of CKT-M
3000 teachers participating in SII
Multiple choice
Specified domain map

5 people + 5 lbs cheese + 5 weeks = 150
items (May 2001)
Large-scale piloting, summer 2001


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


12
Early Decisions and Activity
Types of knowledge

M
a
t
h
e
m
a
t
i
c
a
l

c
o
n
t
e
n
t

Content knowledge Knowledge of
content
and students
Number
Operations
Patterns, functions,
and algebra



http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


13
Early Analyses and Validity Checks
Results from piloting
We can measure teachers CKT-M
Reliabilities of .70-.90
Factor analysis shows distinct types of
knowledge
Knowledge of content and students (KCS)
separate from CK
Specialized content knowledge (SCK) vs.
common content knowledge (CCK)
Hill, H.C., Schilling, S.G., & Ball, D.L. (2004) Developing measures of teachers
mathematics knowledge for teaching. Elementary School Journal 105, 11-30.


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


14
Reliabilities (1PL-IRT): Elementary
Knowledge of content Knowledge of content
and students
Number and
operations (K-6)

.72-.81

.58-.67
Patterns, functions,
and algebra (K-6)
.70-.85
Geometry (3-8)
.85-.86


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


15
Reliabilities (1PL-IRT):
Middle School
Knowledge of content Knowledge of content
and students
Number and
operations (5-9)

.74-.75
Patterns, functions,
and algebra (5-9)
.86-.89
Geometry (3-8)
.84-.86


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


16
Content Knowledge :
Number and Operations
Common knowledge
Number halfway between 1.1 and 1.11

Specialized knowledge
Representing mathematical ideas and operations
Providing explanations for mathematical ideas and
procedures
Appraising unusual student methods, claims, or
solutions



http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


17
Representing Number Concepts
Mrs. Johnson thinks it is important to vary the whole when she
teaches fractions. For example, she might use five dollars to be the
whole, or ten students, or a single rectangle. On one particular day,
she uses as the whole a picture of two pizzas. What fraction of the
two pizzas is she illustrating below? (Mark ONE answer.)
a) 5/4
b) 5/3
c) 5/8
d) 1/4


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


18
Providing Mathematical Explanations:
Divisibility Rules
Ms. Harris was working with her class on divisibility rules. She told her
class that a number is divisible by 4 if and only if the last two digits of the
number are divisible by 4. One of her students asked her why the rule for
4 worked. She asked the other students if they could come up with a
reason, and several possible reasons were proposed. Which of the
following statements comes closest to explaining the reason for the
divisibility rule for 4? (Mark ONE answer.)

a) Four is an even number, and odd numbers are not divisible by even
numbers.

b) The number 100 is divisible by 4 (and also 1000, 10,000, etc.).

c) Every other even number is divisible by 4, for example, 24 and 28 but
not 26.

d) It only works when the sum of the last two digits is an even number.



http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


19





Student A Student B Student C
x
3
2
5
5 x
3
2
5
5 x
3
2
5
5
+
1
7
2
5
5
+
1
7
7
0
5
0 1
2
5
5
0
875
+
1
6
0
0
0
0
875
875
Which of these students is using a method that
could be used to multiply any two whole numbers?

Appraising Unusual Student Solutions


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


20
Common vs. Specialized CK
Appears in exploratory factor analyses on
2/7 forms; confirmatory on 3/7
Individuals can be strong in common but
not specialized; vice versa
Support from cognitive interviews of
mathematicians
Suggests there is professional knowledge
for teaching


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


21
Ongoing Work
Item and measures development
Middle school national probability study
Develop new measurement modules for data
analysis and for probability
Validation efforts
Videotape study
Cognitive tracing studies
Content validity checks

Some Sample Results


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


23
An Example:
Establishing a Relationship to Student
Growth



http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


24
Links to Study of Instructional Improvement
Student Achievement Analysis
SII CKT-M measure 38 items
SII: .89 IRT reliability
Model: Student Terra Nova gains predicted by:
Student descriptors (family SES, absence rate)
Teacher characteristics (math methods/content,
content knowledge)
Teacher content knowledge significant
Small effect (LT 1/10 standard deviation)
But student SES is also on same order of
magnitude
Hill, H.C., Rowan, B., & Ball, D.L. (2005) Effects of teachers'
mathematical knowledge for teaching on student achievement. American
Educational Research Journal 42, 371-406.


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


25
A Second Example:
Evaluating Teacher Professional
Development



http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


26
Tracking Teacher Growth
Items piloted in Californias Mathematics
Professional Development Institutes
(MPDI)
Instructors: Mathematicians and mathematics
educators
40-120 hours of professional development
Focus is squarely on mathematics content
Summer 2001
Pre/post assessment format (parallel forms)
Hill, H. C. & Ball, D. L. (2004) Learning mathematics for teaching: Results
from Californias Mathematics Professional Development Institutes. Journal
of Research in Mathematics Education 35, 330-351.



http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


27
MPDI Teacher Growth (Year 1)
For all institutes for
which we have data,
teachers gained .48
logits, or roughly
standard deviation
Translates to 2-3 item
increase on
assessment
Considered
substantial gain
0
0.2
0.4
0.6
0.8
1
1.2
All institutes
Pre-test
Post-test


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


28
Results from Sample Institutes
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
MPDI I MPDI II MPDI III MPDI IV MPDI V
Pre
Post


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


29
MPDI Evaluation: Other Findings
Length of institute predicts teacher gains
120-hour institutes most effective, on average
But some 40-hour institutes very effective
Focus on mathematical analysis, proof, and
communication leads to higher gains
Many questions remain
Effects of content (e.g., mathematics vs. student
thinking)
Treatment of content: common vs. specialized
Effects of teacher motivation
Long term learning from colleagues, curriculum,
practice

LMT/SII Measures and Dissemination


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


31
Current Item Pool
Equated forms for elementary school:
Number & operations / Content knowledge
(K-6)
Number & operations/ Knowledge of content
and students (K-6)
Patterns Functions & Algebra/ Content
knowledge (K-6)
Geometry (3-8)




http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


32
Current Item Pool
Equated forms for middle school:
Number & operations / Content knowledge
(5-9)
Patterns Functions & Algebra/ Content
Knowledge (5-9)
Geometry (3-8)


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


33
Item Workshops and
Dissemination
Interested users attend a one-day workshop in
Ann Arbor
We cover
History of item development
Analytic methods and validation studies
How to use technical materials
Users get
Access to measures
Support materials


http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t


34
Dates and Contact Information
Learning Mathematics for Teaching
http://sitemaker.umich.edu/lmt

Dates for LMT Workshops
May 19, 2006
August 10, 2006
Brenda Ely (elyb@umich.edu)

Geoffrey Phelps
gphelps@umich.edu
734-615-6076