DATA COLLECTION AT

SCHOOLS: WHAT TO Ben

KEEP TRACK OF, Kenigsberg
Feb. 16, 2017

WHAT WORKS, AND

WHAT DOESNT
 THE CLIENT

Oakland Unifi ed School District specifi cally, the

Offi ce of Continuous School Improvement, which aims
to foster a culture of continuous improvement for
the districts 118 schools.
What that means: School leaders and staff will use
data that the district collects to ensure that the
academic results at each school increase from year to
year, while other indicators of school performance,
like absences and suspensions, decrease.
(In accordance with Merriam-Webster and the Chicago
Manual of Style, I will treat data as a singular noun
for purposes of this presentation. Thank you for your
concern.)
 THE PROBLEM(S)

Bardach problem defi nition: To enhance the quality of

Oaklands schools and success for those schools
students, the Offi ce of Continuous School Improvement
wants to collect analyze data, but it needs to know
more about how other school districts across the
nation use and share data effi ciently.
How I see the problem (in two parts):
1. Defi ning what constitutes improvement, and
fi guring out which types of data are most relevant for
getting there. (Theres a diff erence between
perception of improvement and actual improvement.)
2. Coming up with a plan for data management, based
on what works at other schools.
How my client sees the problem: just Part 2.
 WHAT OUSD ALREADY DOES
W eek 2
D r a f t 2 .0 ( 6 / 1 9 / 1 6 )
H o w t o SI ma mp l pe ml e e Cn yt c lae Mo fo nI nt qh ul yi r yC I G C y cD leev e lo p a c y c le P LA N :
O a k l a n d U n if ie d S c h o o l D i s t r i c t
W eek 1 D e f in e t h e f o c a l p r a c t ic e (T e a c h e r s w ill )
S e t a S M A R T e g o a l(s ) f o r s t u d e n t le a r n in g
D e t e r m in e c o m m o n a s s e s s m e n t s / e v id e n c e t o b e
A N A LY Z E D a t a
c o lle c t e d d u r in g t h e c y c le a n d a t t h e e n d o f t h e c y c le
I d e n t if y t r e n d s in t h e d a t a (s t r e n g t h s a n d c h a lle n g e s ) I d e n t if y a d d it io n a l L e a r n in g n e e d e d t o im p r o v e t h e
R e f l e c t o n t h e im p a c t o f a d u lt p r a c t ic e o n s t u d e n t f o c a l p r a c t ic e .
r e s u lt s (b a s e d o n e v id e n c e o f t e a c h e r p r a c t ic e )
B a s e d o n d a t a a n a ly s is , d e t e r m in e t h e m o s t h ig h -
le v e r a g e a r e a o f f o c u s f o r t h e n e x t c y c le ( s c h o o l- w id e )
W eek 3
E a c h le a r n in g t e a m e s t a b lis h e s a p r o b le m o f p r a c t ic e
r e la t e d t o t h e a r e a o f f o c u s . IM P LE M E N T : E n g a g e in n e w le a r n in g t h a t w ill s u p p o r t
im p le m e n t a t io n o f t h e f o c a l p r a c t ic e , f o r e x a m p le

O n - lin e p r o f e s s io n a l le a r n in g ( K D S ), e . g . a n a ly s is o f
v id e o le s s o n ( s )
W eek 6 P D p r o v id e d b y t e a c h e r , c o a c h , s p e c ia lis t , e t c .
S h a r e d - r e a d in g a n d d is c u s s io n
T e a m s Sh a r e Le a r n in g

I n q u ir y t e a m s s h a r e t h e ir le a r n in g o n t h e ir
p r o b le m o f p r a c t ic e b y p r e s e n t in g t h e ir
f in d in g s t o t h e ir c o lle a g u e s ( i. e . a c r o s s
g r a d e s o r d e p a r t m e n t ). L e a r n in g c o u ld b e
W eeks 4 & 5
s h a r e d v ia :
E V A LU A T E im p a c t :
V id e o e x c e r p t o f le s s o n
P o w e r p o in t p r e s e n t a t io n R e v ie w e v id e n c e o f s t u d e n t le a r n in g ( f o c a l s t u d e n t
G a lle r y w a lk / le a r n in g b o a r d s w o r k o r c o m m o n f o r m a t iv e a s s e s s m e n t )
A n a ly z e r e s u lt s , id e n t if y in g p a t t e r n s a c r o s s c la s s e s
R e f le c t o n in it ia l im p lic a t io n o f p r a c t ic e w h a t is
p r o d u c in g o u r s t r o n g e s t r e s u lt s ? T h e s e s t e p s h o u ld
T h is s a m p le is d e s ig n e d t o g u id e s c h o o ls in o r g a n iz in g p r o f e s s io n a l le a r n in g (e .g . in c lu d e r e f le c t io n o n p e e r le s s o n o b s e r v a t io n s
Source:
W e d n e s d a y P D ) a r o u n d t h e s t e p s o f a c y c le o f in q u ir y . C y c le s c a n v a r y in le n g t h a n d e a c h
OUSD Continuouss c h o o l s h o u ld s e le c t a le a r n in g d e s ig n t h a t f it s t h e ir n e e d s . R e f e r t o C o lla b o r a t iv e S c h o o l A D J U ST p r a c t ic e :
Improvement I m p r o v e m e n t f o r e s s e n t ia l e le m e n t s w h e n a d o p t in g a c y c le . T e a c h e r s m u s t b e o r g a n iz e d
D e t e r m in e h o w t e a m m e m b e r s w ill r e f in e t h e ir
Guide & Toolkit, in c o lla b o r a t iv e t e a m s ( e .g . g r a d e le v e l, c o n t e n t , e t c .) b e f o r e b e g in n in g t h is c y c le . C y c le s
p r a c t ic e t o m o v e s t u d e n t s t o t h e g o a l
201617 edition s h o u ld b e c o n n e c t e d t o t h e s c h o o ls p r io r it ie s ( B ig R o c k s ).
 KEY PERFORMANCE INDICATORS

Source:
OUSD
Continuous
Improvement
Guide & Toolkit,
201617 edition
 SCHOOL PERFORMANCE
FRAMEWORK
Release version date: 10/27/2016

Chabot Elementary Result Summary

Elementary School Grades TK-5 106ES
Academic
Status 5.48
School Demographics 2015-16 N1 K-5 Growth 5.10 Color Score
567 students ELL: 1.4% AA: 11.1% ME: 13.9%
Culture/Climate
Status 4.75 Blue 5+
SWD: 7.1% A: 7.8% NA: 1.1% Growth 3.37 Green 4
LI: 14.1% F: 0.5% PI: 0.5%
Overall Tier 4.80 Yellow 3 School Performance
L: 7.2% W: 69.7%
See reverse for abbreviation definitions & notes
Orange
Red
2
1
Framework
Academic Status 30%
5.48 Total
Index
All Students
LPRG (lowest performing
racial/ethnic group)
ELL(English Language Learners) SWD (students with disabilities) LI (low income)
Total

Academic Growth 30%

5.10 Score Result N
Index Points
Level Earned
Group Result N
Index Points
Level Earned
Result N
Index Points
Level Earned
Result N
Index Points
Level Earned
Result N
Index Points
Level Earned
Weight

Academic Domain
0.250 0.100 0.000 0.000 0.125 0.475
SBACELA
(Smarter Balanced English/Language
status 5.69 77% 263 5 out of
0.250
AA 33% 21 4 out of
0.125
1 out of
0.000
15 out of
0.000
51% 39 5 out of
0.125
out of
0.500
0.250 0.025 0.000 0.000 0.125 0.400
Arts) status
growth 4.75 0pp 271 5 out of
0.250
AA -2pp 31 1 out of
0.125
2 out of
0.000
18 out of
0.000
-13pp 45 5 out of
0.125
out of
0.500
0.250 0.125 0.000 0.000 0.125 0.500

SBACMath
status 6.00 78% 265
5 out of
0.250
AA 45% 22
5 out of
0.125
1 out of
0.000
18 out of
0.000
57% 40
5 out of
0.125
out of
0.500
0.250 0.125 0.000 0.000 0.125 0.500
5 5 5
(Smarter Balanced Mathematics) status
growth 6.00 -2pp 270 out of AA +2pp 30 out of 2 out of 18 out of -11pp 45 out of out of
0.250 0.125 0.000 0.000 0.125 0.500
0.250 0.050 0.000 0.050 0.05 0.400
SRI
status 4.75 85% 370
5 out of
0.250
AA 67% 36
3 out of
0.083
4 out of
0.000
53% 34
3 out of
0.083
61% 51
3 out of
0.083
out of
0.500
(Scholastic Reading Inventory) 0.250 0.083 0.000 0.033 0.017 0.383
98.9%students tested
status
growth 4.54 +1pp 366 5 out of
0.250
AA +10pp 42 5 out of
0.083
3 out of
0.000
0pp 34 2 out of
0.083
-11pp 60 1 out of
0.083
out of
0.500
N representsthe count of studentscontributing to the result in the most recent year of data for status, or the prior year for growth. Elementary School Network 1

SPFGuidebook2016-2017

SPFColorsandTiers
Just as we use color on a stoplight, our SPF uses different colorsto show quicklyhow schools
performed for each of the measuresincluded. In addition to the three colors of a stoplight
(Green, Yellow, Red), the SPF usesfive colorsto allow for more differentiation between
performance levels. Each color correspondsto a different tierof performance. Orangeis
added to provide more separation between schools and performance in the lower Tiers. Blue
isadded to identify disnguished school performance, helpingto bringattention to Bright Spots
and opportunitiesfor colloration around effective practices.

Color Tier
Blue =5.0-6.0
Sources: Green =4.0-4.9
ousddata.org and OUSD School Performance Yellow =3.0-3.9
Orange =2.0-2.9
Framework Guide, 201617 edition Red =0.0-1.9

IndicatorTier Scores
 WHAT ELSE IS DONE

OUSDs Offi ce of Research & Development produces

weekly engagement reports, which include a
current rundown of absence and suspension rates,
but also focus, newsletter-style, on a diff erent data
issue every week.
The Continuous Improvement Guide comes with
guiding questions of the month, such as How are
college readiness indicators informing your plan for
second semester? to make sure that principals and
other administrators are on track.
Many of the suggestions come straight from Data
Analysis for Continuous School Improvement (Victoria
L. Bernhardt, 2004)
 SO REALLY, WHAT ELSE NEEDS TO
BE DONE?
OUSD has already collected the data it thinks is relevant.
The offi ce of Continuous School Improvement wants
anecdotal research on how other school districts do the
same thing particularly when it comes to data
management.
Im asked to become a Studs Terkel for school data
analysts.

Conundrum: How to identify which school districts work?

National Center for Education Statistics is good on
demographics, but not on improvement.
 WHERE SHOULD I LOOK?

Every school district wants to improve continuously. How do

I know which districts are actually doing that?
OUSD has its own suggestions, which include districts in
Long Beach and Fresno, both of which have long-serving
superintendents. (But whats the causal relationship there?)
According to The New York Times (May 11, 2015), the
Menomonee Falls, Wisc., district uses data to measure
everything from grades to custodial cleanings.
Montgomery County, Md., has a data portal that is the envy
of every district.
Organizations like the Carnegie Foundation, which aims to
bring improvement science to education, and Schoolzilla,
which provides data templates to districts, may have
suggestions.
 THE PROBLEMS (REVISITED)

How to confi rm that what I learn in interviews is valid.

How to translate the never-ending stream of
buzzwords and fl owcharts cherished by school
administrators into concrete suggestions.

The most
meaningless graphic
ever created.
John Oliver