Professional Documents
Culture Documents
When Johnny Takes The Test Fields Leslie Hoge 2005 10pg Edu
When Johnny Takes The Test Fields Leslie Hoge 2005 10pg Edu
1
In response to requests from some schools . NAEP has prepared a parental
The National Assessment of
information letter (shown below) which explains the assessment and its importance . If your school
Educational Progress (NAEP) requires parental notification, you may use or modify the letter as you see fit.
Each year, certain children are selected
to participate in the National Assessment
of Educational Progress (NAEP) . It is a
PARENT INFORMATION LETTER
national test mandated by Congress ; how-
ever, participation by states according to
federal law is voluntary . Many states are January, 1992
considering using the NAEP as their state
assessment, supplanting norm-referenced Dear Parent :
tests and locally or state-developed tests .3 Approximately 225 schools per stare including (school) have been selected for
participation in the 1992 National Assessment of Educational Progress, Trial State Assessment .
The NAEP test is called "The Nation's The National Assessment is a continuing survey of the knowledge, skills, and attitudes of
young Americans and the subjects they are taught in school . This assessment will focus on
Report Card ." It assesses subjects such as mathematics at the eighth-grade level and on reading and mathematics at the fourth-grade
reading, writing, mathematics, and other level.
subjects included in the National Educa- A sample of about 30 students was randomly selected to participate in each assessment
session in our school . The parents of each selected student are receiving this letter to inform
tion Goals, eight new mandates for educa- them about the study .
tion reform associated with AMERICA The assessment takes about 1 3/4 hours of a student's time and is confidential . To
ensure confidentiality, National Assessment results are not reported to, or about, individual
2000/GOALS 2000 . The NAEP is conducted by students, schools, or districts. Students' names are not recorded on any of the assessment
materials taken from the school or reported in any way . Participation in the program will am
the Commissioner of Education Statistics affect your child's grades or progress in school .
under the auspices of a National Assess- The National Assessment of Educational Progress is an important activity to keep us . as
ment Governing Board . The National parents and citizens, informed about what American youth are learning. We wanted to inform
you of the survey and let you know that the participation of every selected child is greatly
Center for Education Statistics (NCES), appreciated .
Exhibit
1
2
Seen- I s«tien I
SECTION 1
This section has 20 qumtiom . Mark your answers in your booklet . Fill in 7. Does your family get a newspaper 11 . How much television do you usually
only one oval for each question. regularly? watch each day.'
0 Yes 0 None
1 . Which best describes your 3 . Were you born in one of the SO states of
the United States. in the District of O No 0 1 hour or less
O white lnot Hispanic) Columbia, or in one of the United States
temtonesf © I don't know . © 2 hours
CD Black not Hispanic!
0 Yes O 3 hours
CID Hispanic 1-Hispanic" means someone 8. Is there an encyclopedia in your home?
who is Mexican. Mexican American. O No O a hours
. Puerto Rican, Cuban . or from
Chicano O Yes
some other Spanish or Hispanic back . O : hours
ground .) 4. How often do the people in your home tD No
speak a language other than English! m 6 hours or mate »Y
iD Asian or Pacific Islander r'Asian a iD I don't know n®ma
paging Islander' means someone who IM Never
is Chinese . Japanese . Koran . Filipino, 12 . How much time do you usually spena an
Vietnamese . Asian American, or from m Sometimes 9. Are there more than 2S books in your homework each day+
some other Asian a Pacific Island back . home!
pound .) tD Always C I don't usually have homework
O Yes assigned.
m American Indian or Alaskan Native
-Amer can Indian or Alaskan S. How far in school did your mother go? O No m I have homework but I don't
Native" means someone who u from usually do it .
one of the American Indian tribes, or O She did not finish high school. m I don't know . lama.
one of the original people of Alaska .) T ' i hour or less
® She graduated from high school .
CDOther (whatf) Write your answer on 10. Does your family get any magazines CD I hour
the line. tent tD She had some education after high regularly!
school . CD More than I hour
O Yes
2. If you are Hispanic. what is your Hispanic C She graduated from college .
background! (D No 13 . About how many ;aces a day do
O I don't know. you have to read in school and for
O I am not Hispanic. (D I don't know fsarsaa homework!
iD Meaicano. Mexican American, a 6. How fat in school didd your father go? O More than 20
Chicano' .
O He did not finish high school. T 16.20
I (D Puerto Rican
CD He graduated from high school . X 11-13
O Cuban
CD He had some education after high Q 6-10
tD Other Spanish or Hispanic school .
background fri•uu 0 i or fewer
m He graduated fromcollege .
tD I don't know .
Note that in the Script . instructions to you. the Assessment Administrator. are in bald.
capital letters. Do not read these to the students .
om .
READ-'
• Whether there are certain kinds of television When the study is over, your answers will be combined with
information from other students all over the country . The results
programs the child is not allowed to watch . will help government leaders, school administrators, and teachers to
determine what students are learning . Because the study will have
• How many books are in the home . an impact on schools and students all over the country, we hope that
you will do the best that you can .
• How often the child sings, dances, plays music DISTRIBUTE Now I would like to give you the things you will be using. Before I
or puts on a show for adults in the family. MATERIALS hand out your materials, please clear your desks . As I call your
name, please come forward to get a booklet and pencil . After you
• How often the child sees adults in the family get your booklet, turn it face down on your desk and do not open it
until I tell you to . If you need another pencil after we begin, please
writing, using mathematics or reading . l raise your hand .
READ STUDENTS' NAMES FROM ADMINISTRATION
The Trouble With Confidentiality SCHEDULE. DISTRIBUTE BOOKLETS AND PENCILS.
RECORD A".1" OR "A' IN PRESENT OR ABSENT COLUMN OF
By now some alarm bells should be going off. This THE ADMINISTRATION SCHEDULE. DOUBLE-CHECK THAT
THE STUDENT'S NAME ON THE ADMINISTRATION
is a congressionally-mandated test . This test is given SCHEDULE MATCHES THE NAME ON THE POST-IT FOR
by a federal agency . The federal government gets EACH STUDENT . REMOVE POST-IT BEFORE GIVING
MATERIALS TO STUDENT .
the results of these tests . And we can assume that CODING THE Please turn your booklet over . Code your grade, birth date, sex, and
the government isn't asking Johnny if he talks to you BOOKLET COVER zip code in the box at the top of the booklet . Write '4' in the box
labeled 'Grade . Then fill in the oval next to the number in the grid
about school just to take up space in the booklet . below the box . In the box labeled 'Birthday", write the month and
year you were born and fill in the correct ovals . Next, write 'M' for
But wait, you say, the letter about the test that male or 'P for female in the box labeled 'Sex' and fill in the correct
came home with Johnny said it was "confidential" . oval. In the box labeled zip code, write the zip code of your home
address. Be sure to fill in the ovals completely .
Well, yes, it did say "confidential ." A quick read
WRITE THE SCHOOL'S NAEP ID NUMBER ON THE
would lead the average reader to believe that no BLACKBOARD. Next, write this number (REPEAT NUMBER) in
child's information is linked to him personally in any the boxes labeled 'School number' and fill in the corresponding
ovals below the boxes .
way. This, we assert, is the exact illusion NAEP
intends to create . BOOKLET Now turn your book over and open it to the Directions on the first
DIRECTIONS page. Read them to yourself as I read them out loud.
Look at the letter again . It does not say anony-
mous . It says confidential . There is a significant SHOW DIRECTIONS ON THE NEXT PACE OF THIS SCRIPT.
READ ALL THE DIRECTIONS, EXAMPLES, AND ANSWERS
difference between confidential and anonymous . VERBATIM AND SLOWLY.
DO NOT ADMIT ANY MORE STUDENTS TO THE SESSION.
Anonymous means just that - no way to personally
link the individual to the information . Confidential Exhibit 4
means that personal identity is linked to the infor-
mation, much like the information kept by your between reporting data and collecting data . NAEP is not saying
doctor . It is confidential because only authorized persons have they are not collecting data on individual children ; they are
access to it . In this case, the government thinks you should trust saying that they do not report the data with individual identity
them to have access to personal information and to act honor- attached . Technically, in terms of the wording of the letter, they
ably with it. are not lying. They rely on the hope that the parent will not
Note also in the parent information letter the statement : "To know the true meaning of the words chosen .
ensure confidentiality, National Assessment results are not
reported to, or about, individual students, school, or districts ."
The key word here is "reported ." There is a vast difference
4
PIAEI' IV92 INIAI :i1AIL ASSESSMEIJI ADMINISTRATION SCIIEDIII .E 1' 1 . .1 . 2 of the Administra-
`' wbw. .•LrJ.J.,I . •
1-0) . 1"I IWO
L .1 ... 6/9!
- SCIwW /I Q I . I
is 1 / 14 I .I ••~
• ^^wr• ~1~T/' . / INIyM .J
tors Manual . Notice
.. :
Ru •/ERWcxy Cod
ScIrW N..,.: ~"(icrat EA_C^f&_THRr~ „q ._:__S/ Ed 3 the paragraph in
bold letters remind-
1 ...0• . : iq/) 1/ y
W • WMI.-1INq.nk
S • SI
... .61 NIq ..k
N
A .
.A
.
/-IM
1
• Nlq.wk Nw11 .S. . O.y. S..p1 . 1m, Ab . . .1
I'M," /D // F
name must match
n
ey •/oors~
ministration Sched-
17
S • 67A~/3
2
ule .
w/r?Y PI,? r, Now the mystery
begins to unfold as
Exhibit 5a
we get a further
glimpse into the in-
How Johnny Gets Bar-Coded ternal workings of the test. Follow along .
A third item of interest on the parent letter is the sentence Please refer to Exhibit 5A, which appears on page 57 of the
"Students' names are not recorded on any of the assessment Training Example . You will see a sample Administration Sched-
materials taken from the school ." It is true that your child's name ule . This is the list of children chosen by the government to take
is not recorded on the test booklet . But, your child's name is the NAEP test . This list is sent to the school and each school is
recorded elsewhere . responsible for assuring that the children on the list take the
Here is the clue: remember the "script" referred to earlier? NAEP test . Note that for each name on each line there is a unique
The script is simply a set of instructions for taking the test and number seven columns to the right . We draw your eye to
it is to be read to all children taking the NAEP . The Administrators student 02, Casey Bailey . Her booklet number in the column
Manual is very emphatic that this be done to the letter . Witness headed "Booklet or Exc. Student Quex . No." is 14-67210.3."
page 51 of the Administrators Manual, where administrators are Also, notice that Casey is checked "present" in the eighth
given their instructions . They are told to read the script column, just to the right of the booklet number in Exhibit 5A .
This will be important later.
WORD FOR WORD (but with expression) to ensure Now, look at Exhibit 5B (from p . 60) . You will see a copy of
that all sessions are administered uniformly . The only the cover of a NAEP test booklet . What number do you see under
liberty you may take with the script is the very first line the bar code? 14-67210-3. Yes, it is the same number. The test
which reads 'Hello. My name is ' . . . The remainder of booklet number matches the number next to Casey Bailey's
the script must be read verbatim, without additions or name on the Administration Schedule!
deletions . (Exhibit 3)
The administrator is also warned : "Note that in the script, Tracking Johnny to the Data Bank
instructions to you, the Assessment Administrator, are in bold, Refer back to Exhibit 1, the sample letter to parents, that
capital letters . Do not read these to the students ." All one has states that "Students names are not recorded on any of the
to do is read the bold capital letters to know why! assessment materials taken from the school . . . ." You now know
Please refer to Exhibit 4 . This is the actual script the test that this statement is not true . Your child's name does not
administrator must read to the students from pages READ 1 and appear on the test booklet itself. The "post it" is used for that!
5
• a) a) 0 0
Administration Schedule ." To doubly con- xCD oo 00000
firm that this is true, on page 60 we read • X0 •0 00000
® CD 0 •0 00
"Students' names must not be written on kVV o
THE NATION'S m
the cover or inside of any assessment book- REPORT
XC
"Individually identifiable" data, then, is a bureaucratic defi- Schedule with your child's name and test number on it if it isn't
nition that means that the data is able to be identified . To 1 personally identifiable? NAEP circumvents federal laws with its
confirm this, we find a definition in the LACES Field Restricted Use "post it" note maneuverings and semantic gymnastics . This is
Data Procedures Manual (3/31/93, B-2) for test data : key to understanding the next thing that happens to Johnny's
test.
Individually Identifiable Information - Any item,
collection, or grouping of information pertaining to an What Happens to Johnny's Test Data?
individual and maintained by the National Center of
When Johnny's name, number and test are shipped back,
Education Statistics or one of its contractors, including,
who will have access to this data? The answer may shock you .
but not limited to the individual's education, financial
Here is a list of 29 research organizations, academic institutions,
transactions, medical history and criminal or employment
companies, and contractors which have access to restricted use
history, and containing the name, or an identifying num-
NAEP data bases that contain individually identifiable informa-
ber, symbol, or other identifying particular assigned to the
individual, such as a fingerprint, voiceprint, or photo- tion :
graph .
AERA-Appalachia Education
Did your elected officials realize that the language contained Arizona Dept . Ed, Res & Dev
in the Federal Privacy Act (Sec . 552a of Title 5, U .S. Code) Boston Col - Center Study Testing
would end up meaning just the opposite? What is the "post it" Bureau of Census . Popula. Div.
if not a personal identifier? And, what is the Administration CTB/Macmillan/McGraw-Hill
Dept. of Agriculture, Eco. Res . Ser.
Pack all materials from the original session according to the instructions ED Office of Research
in Task 4. Enclose in this shipment : the used and unused (Le ., those not
needed for the make-up session) booklets, remaining calculators, rulers Economic Policy Institute
and geometric shapes, the Roster of Questionnaires (or a copy if all
questionnaires have not been returned), all questionnaires, the ow of
Educational Testing Service
the Administration Schedule, and the spy of the Packing List. LMP Associates, Inc.
Ship the box to NAEP within one day after the original session . MPR Associates, Inc .
a If the original session was attended by a NAEP representative, this Montana State Attorney General
person will keep the materials needed for the make-up session (except NCREL-North Central Reg. Ed. Lab.
for the Administration Schedule with the students' names and Roster of
Questionnaires if not all have been returned) and will ship the materials Penn State Univ ., Voc. and Ind. ED
RAND, Inc.
from the original session .
Westat
Mala It". at N* WwowsaaN ." ..1-
R.aO .q Pm.e. 0,
Iaw RK-0N
s"I_ 11. Yn.easneaus --
01
Packing Diagram American Institute of Research
M.m.Neala . Saw.,
Pac. .q LM
Sr.a . 6. . Ceaa-.q
National Computer Systems 5
G.o ac S...O.S G,eMa I.s Ma. asses
R, .W Ca- aw
So,oW C1.a.anes.es ad rap . R,cew-
Pokws 0."_.e ISCPo1
E.oNdd SWO . ..OeeMOn.wn
fMw Once Johnny's name, number and test
data end up in the national data bank, the
-1-Os.naw.es
T
Ras.N W Oo.vo. .w.es
Asses~ NWMCal0. L .OR a erk. ScMak
P--0 kbm4IO, Lwl-
Pd W Apoe.O..e.. C-11(10)
OEP S...- ..ans M ..Scw.a0
0 a-lea a S. .n parent has no control over what happens
to that information next . Parents should
Pmt, .o.4
Ream Pmlaq. Pad La0es
Su 511_ Mt S.oo .q F ...bw P-
C-a0eaa
5.14 .9 I.-
0-
u-, a-- know that there are two ways that Johnny's
To No Q009 Sq.
6-d'. Sins 6 PLEASE RETURN ALL UNUSED MATERIALS test data can be accessed by sources outside
of NCES . A contractor (or subcontractor)
can access it, or an organization can apply
Exhibit 6
for a license to access it . Information may
7
practices . . . as they relate to the students selected for the ered ill-suited to their job .'
assessment . . . ." I We must remind the reader that student test results under
Here is how teachers are tracked . A Roster of Questionnaires various education reform measures and proposals are the con-
is filled out which links the teacher to the Teacher Question- troversial new "outcomes" of Outcome-Based Education (OBE)
naire in the same manner that the test booklet and the Admini- that measure a child's feelings, attitudes, behaviors, values and
stration Schedule link the children . See Exhibit 7 from page 31 beliefs . Will teachers (and schools) be penalized if their students
of the Training Example . do not score the proper "attitude" on the NAEP .'
Note the column next to the teacher's name . It is numbered
"teacher 01, 02" and so on . Now look in the second to the last Pulling Johnny Out
column (Teacher #) of the Student Administration Schedule So what can parents do? We propose the removal of children
from participation in the NAEP test. However, there
ADMINISTRATION CODES
is a hitch to this .
Whenever scientific study is done, the researcher
The seven Administration Codes are defined below. The most frequently used
attempts to control, or to take into account all the
code is '1', meaning that the student was in the session full time . Jf the student was absent,
variables . The NAEP is a longitudinal study, which
enter the letter'A'in the box, code the last ova( and code the reason for his/her absence in the means that in order to be longitudinal, it must meet
ABS box using the codes on the next page, consulting school records as necessary. at least two criteria - individual identity is involved,
and the participants are re-assessed at a later date .
In session full-time. Student at least attempted to complete the booklet .
Most booklets will receive this code .
Students who take the NAEP in grade four will be
In session full-time, but no response in booklet .
the very same students who take it in grade eight, and
2-
so on. The nature of longitudinal studies is to re-ex-
In session part-time . Student left and did or did not return . Specify the
3-
reason on booklet cover (e.g.. student refusal after starting booklet) . Also. amine, or re-test the same subject in order to measure
note on booklet cover the section number when student left and the section
number when the student returned, if s/he did return .
changes in individual progress . That is why personal
identity must be attached to the participants taking
Session Incomplete . Specify the reason. Use this code when the session was
interrupted and no student was able to complete the booklet. Reasons may the test.
include fire drills, equipment failures, etc
However, on the NAEP, one of the variables taken
s- Student Ineligible. Use this code when you are told or discover after the into account is the student who for one reason or
session begins that the student is not eligible for the assessment . Explain fully
on the booklet cover how you determined the student was ineligible. another does not take, or does not complete the test .
6- Other, specify. Use this code for any situation which is not covered by the
The cover of the booklets provides spaces to enter
other codes, such as the discovery of a page missing from a booklet . Explain codes for some of these "variables ." On Exhibit 5B
the situation fully on the booklet cover .
note the column on the test booklet cover marked
A- Absent. Use ABS codes in next column to explain why student was absent
from session . (See explanation of ABS codes on next page .)
"AD" to the right center of the page . That is the code
to indicate the' outcome of the session ." These codes
Exhibit 8 include :
• In session full-time . Student at least attempted to
(Exhibit 5A) . Here we find "01, 02" and so on . This means the complete the booklet .
teacher's answers to the Teacher Questionnaire can be linked • Session incomplete. Specify the reason . Use this code
to the results of the child's test . when the session was interrupted and no student was able
This questionnaire probes into the teaching methods of each to complete the booklet . Reasons may include fire drills,
teacher . It would seem logical that since NAEP is collecting data equipment failures, etc . (pg . 65)
on how teachers teach, and then correlating it to what students Parents should be alerted to Exhibit 5B and the meaning of
know, that the government would have ample explanation as the last box in the column, just to the right of "AD:" an "ABS ."
to what methods of teaching produce the best result . The In Exhibit 8 we learn this stands for "Absent . Use ABS codes
current debate over whole language vs . the phonics method of in next column to explain why student was absent from session ."
teaching reading is one example . [From page 65 of the Administrators Matutal, emphasis ours .]
Numerous education reform plans and documents connect
The administrator is to indicate why each child is absent .
student assessment test peformance directly to teacher incen- The possible "whys" include "parent refusal . . . to allow student
tives, as well as school rewards and penalties . Teachers could to participate in the assessment," Read carefully through the list
lose out on promotions and schools could be shut down if of Absent Codes on page 66 of the Administrators Manual in
student test results aren't up to par . We have just demonstrated Exhibit 9 .
how NAEP student results are connected with teacher perform- It is apparent that NAEP has all bases covered . If you refuse
ance . In the near future teachers who cannot produce the to let your child take the test . NAEP knows, and if your child
desired student outcomes on an assessment test will be consid-
9
refuses, NAEP knows . No matter what you do, once your child tribution of $3 .50 . Documentation from this article available for
is selected, NAEP knows. a contribution of $20 .00. Order from Melanie Fields, 6$ Poca-
Many parents have pulled their children out of public and hontas Rd ., Front Royal, VA 22630 .
even private schools to avoid this type of testing altogether . We A generic reprint of this article, including exhibits, with
endorse this course of action . However, we caution those who references to any publication removed, may be obtained by
privately educate their children to use discernment regarding sending a self-addressed-stam ed envelope and $1 .00_to,
anystandardized test . There is ample evidence that in the future Iowa Research Group, Inc ., PO Box 449, Ravenna, OH 44266
all children may be compelled to take tests such as the NAEP . Multiple reprints are available for $1 .00 per copy, plus S3 .00 for
For those parents who must keep their children in the postage and handling .
classroom, there is a way . This way would require that parents For supplementary documentation and resource material,
make choices that categorize their children under the "Tempo- order "Womb To Tomb" and/or "T alking Papers" by nits
rary absence" code by ensuring that their children have un- Hoge .
scheduled absences for both the original testing period and the
make-up session. This article was first published in The Christian Con-
Naturally, NAEP wants as many of the selected students as science, a magazine _ about_ family
.-education and- cu-Itural
.
possible to be tested . And they have provided for this by issues .
requiring a make up session : "If four or more students were
absent (not counting student and parent refusals), you must
schedule a make-up session . . . ." (pg. 61) . "Four" appears to be ABSENT CODES
the magic number for norming the test .
It is possible for the NAEP to adjust its test forming if only 1- Temporary absence (less than 2 weeks). Student is not in school temporarily
a few parents/students refuse to take the test . However, it is due to illness, disability, or excused absence .
not as easy to adjust test forming to temporary, unscheduled, 2- Long-term absence (more than 2 weeks) . Student has been absent from
unpredictable absences .A great number of temporary absences school 2 weeks or more due to an illness or disability.
might cause some chaos in the testing process, including the 3- Chronic truant . Student only attends school occasionally, if ever .
"make-up" sessions . Not only will this protect many children 4- Suspended or expelled from school.
from intrusive testing, it will foul up NAEP's collection and 5- Parent refusal . Parent officially notified school that he/she refuses to allow
norming of the data . The more the merrier! student to participate in the assessment .
6- Student refusal. Student refused to participate in the assessment before
Conclusion being given an assessment booklet .
Federal Law authorizing the NAEP (P.L . 100-297) states : "It 7- In school, did not attend session . Use if student was known to be in school on
day of assessment but, for example, was not released by teacher .
is not the purpose of this Act to authorize the collection or 8- In school . but not notified of assessment because of disruptive behavior .
reporting of information on student attitudes or beliefs or on
9- Other. Use this code for any absence not covered by other codes . Specify the
other matters that are not germane to (relevant to) the reason completely on booklet cover (e.g., student came to room after session
acquisition and analysis of information about academic started) .
disk, text file format, (scanned graphics of exhibits) for a con- another publicationn or electronically . Photocopied reproductirnu may be made of
this document in its entirety . Mass distribution( is encouraged! i'
10