Asmtrdtestspecshs 2011-12

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 45

OREGON STATEWIDE ASSESSMENT

Reading/
Literature
TEST
SPECIFICATIONS
and BLUEPRINTS

2011-2012

HIGH SCHOOL
It is the policy of the State Board of Education and a priority of the Oregon Department of Education that there will be no discrimination or harassment
on the grounds of race, color, sex, marital status, religion, national origin, age or handicap in any educational programs, activities, or employment.
Persons having questions about equal opportunity and nondiscrimination should contact the State Superintendent of Public Instruction at the Oregon
Department of Education.

Developed by the Office of Assessment and Information Services


Oregon Department of Education
255 Capitol Street NE
Salem, Oregon 97310-0203
(503) 947-5600

Susan Castillo
Rachel Aazzerah
State Superintendent of Public Instruction
Science and Social Sciences Assessment Specialist
Doug Kosty
James Leigh
Assistant Superintendent
Mathematics Assessment Specialist
Steve Slater Dianna Carrizales-Engelmann
Manager, Scoring, Psychometrics and Validity Director, Monitoring, Systems, and Outcomes

Kathleen Vanderwall Bradley J. Lenhardt


Manager, Test Design and Administration Monitoring and Assessment Specialist
Holly Carter Sheila Somerville
Assessment Operations and Policy Analyst Electronic Publishing Specialist
Michelle McCoy Kathy Busby
ELPA and Assessment Implementation Specialist Project Manager
Ken Hermens
Language Arts Assessment Specialist

All or any part of this document may be photocopied for educational purposes without permission from the
Oregon Department of Education and distributed for the cost of reproduction.
TABLE of CONTENTS
Introduction ....................................................................................................................................................................................................... 1
Background ...................................................................................................................................................................................................... 1
Content Standards Maps ................................................................................................................................................................................... 3
Item Specifications ........................................................................................................................................................................................... 10
Test Blueprints ................................................................................................................................................................................................. 14
Weighting Chart ............................................................................................................................................................................................... 16
Cognitive Demand and Item Distribution by Difficulty ................................................................................................................................... 18
Achievement Level Descriptors ....................................................................................................................................................................... 19
Local Performance Assessments ...................................................................................................................................................................... 24

Appendices
A. Oregon Achievement Standards Summary ................................................................................................................................................ A-1
B. Cognitive Demand and Item Distribution Goals ........................................................................................................................................ B-1
C. Item Development Process ......................................................................................................................................................................... C-1
D. Life of an Item ............................................................................................................................................................................................ D-1
E. Scoring Guide ............................................................................................................................................................................................. E-1
Reading/Literature, High School

Reading and Literature Test Specifications


Introduction to the Test Specifications and Blueprints Background
The primary purpose of Oregon’s Test Specifications and The purposes of the Oregon Statewide Assessment Program are
Blueprints is to provide the consistency necessary for the (1) to provide information on individual student achievement on
development and administration of the Oregon Assessment of performance standards set by the State Board of Education at
Knowledge and Skills (OAKS). OAKS provide critical data for grade and benchmark levels; (2) to provide information for
Oregon’s accountability system which meets Peer Review federal NCLB requirements and for policy decisions by the
Requirements of No Child Left Behind. All students in grades 3 legislature, the governor, the State Board of Education, and local
through 8 are required to take the reading and mathematics school districts; (3) to support instructional program
assessments. All students in grades 5 and 8 are required to take improvement efforts; and (4) to inform the public about student
the science assessment, and those in grades 4 and 7 take the achievement in Oregon schools.
writing assessment. In high school, reading, writing,
mathematics, and science are required assessments. The Oregon Statewide Assessment is different from national,
norm-referenced tests used in many districts and states. The
OAKS is also one way for students to demonstrate proficiency Oregon Statewide Assessment is a criterion-referenced
in the Essential Skills of reading, writing, and mathematics, assessment based on the Oregon Content Standards. As a result,
which will be necessary for earning a high school diploma the types of scores produced from the Oregon Statewide
beginning with graduating seniors in 2011-2012 who must show Assessment are somewhat different from those produced by
proficiency in the Essential Skill of Reading. In addition, national, norm-referenced tests.
English Language Proficiency Assessment (ELPA) is required
for non-English speaking students until they acquire enough Oregon educators contribute to the test development and
skills in English to exit the program. Social Sciences is an alignment process by serving on advisory committees called
optional assessment. Content and Assessment Panels. Stakeholders in these
committees are involved in each phase of the development of
Test specifications provide guidelines for item writers, who are these specifications to assure that they accurately and clearly
typically Oregon teachers, on what content may be tested and explain the overall design of the test and describe the specific
how items must be written. These specifications lead to test content that might appear on the test to measure the knowledge
blueprints that outline test design and the number of questions to and skills described in the content standards.
be tested in each score reporting category (SRC). The Test
Specifications and Blueprints document is an important Oregon’s knowledge and skills test questions use a multiple choice
resource, not only for item writers and reviewers, but for format, with each item having a single correct answer and three
educators administering OAKS and the general public who are incorrect answers. A computer scans the “fill-in-the-bubble” answer
interested in understanding the content and format of test items. sheets or electronically collects scores. The results are scored
against the answer key to produce a raw score. The raw score is

Reading/Literature Test Specifications 1 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

converted to a scale score called a Rasch unit or RIT score. documents must be destroyed/shredded immediately following
Students receive a scale score based on the number of questions the testing session.
answered correctly compared to the total number of questions on
the form—taking into account the difficulty of the questions. The OAKS online test delivery system for the 2011-2012 school
Students are not penalized for guessing. year allows students with visual impairments to have three
opportunities to take an adaptive reading assessment in Braille.
The content of these specifications reflects the skill Beginning with the 2011-2012 school year, tests will no longer
expectations outlined in the Content Standards adopted by the be available in a paper/pencil format.
State Board of Education for implementation during the 2005-
2006 school year. These standards were developed, in part, to Having the tests fully adaptive allows for more precision in
correlate to the skills assessed on the reading portion of the measurement and less frustration for students. A practice, or
National Assessment of Educational Progress. sample, test is available online for students who may need
practice using a scrollbar to view the full length of passages and
Electronic Administration the accompanying questions, particularly those students in the
On the reading and literature knowledge and skills OAKS tests, early grades.
there are multiple opportunities to participate in the fully-
adaptive testing. In this format, the accuracy of the student Grade 3 Spanish Reading Assessment
responses to items associated with one passage determines the During the 2009-2010 school year, Oregon field tested a Spanish
next block of items and passage the student will see. Students translation of the grade 3 reading and literature assessment. To
can preview test questions prior to reading the associated produce this test, a targeted sampling of the English passages and
passage by using a scrollbar. Students can also highlight items in the test pool was translated into Spanish based on
sections of the passage they feel may be useful for answering appropriateness of the materials taking into account cultural and
questions after reading the passage. translatability considerations. This Spanish assessment was
approved by a federal peer review panel for AYP and other
Students who require a print copy of test passages and items accountability purposes for 2010-2011 and subsequent years.
may submit a request to the test administrator during the testing
session to have either or both of these printed. These secure

Score Reporting Categories: Detailed Descriptions


The following pages contain a more detailed examination of the test content for reading and literature. The first column lists the standards
assessed for that particular score reporting category. The second column gives a more detailed explanation of how the standard would be
assessed. Finally, the third column gives some sample stems of items that would be very similar to the type of questions asked on a test of that
particular content.

Reading/Literature Test Specifications 2 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

VOCABULARY Score Reporting Category 1


Common Curriculum Goal: Increase word knowledge through systematic vocabulary development; determine the meaning of new
words by applying knowledge of word origins, word relationships, and context clues; verify the meaning of new words; and use those new
words accurately across the subject areas.

High School Standards Explanation Sample Stems/Questions


Determine meanings of words Students use context clues to choose What does the word emulation mean as used in the first
using contextual and structural the correct meaning for identified paragraph of this editorial?
clues. words on the state assessment. The
context clues may be presented in a The best definition of the word camaraderie as it is used in the
directly-stated phrase, or they may passage is
appear in sentences before or after the
target word.

Identify and use the literal and Students use context clues to interpret The narrator says, “Teaching the way I do is hard, for I touch
figurative meanings of words the meaning of phrases using the raw nerves of faith, the beginnings of things.” This is
and phrases. unfamiliar words or to interpret another way of saying that
phrases that have figurative
expressions. Steinbeck called the turkeys a “reservoir for Thanksgiving.”
As used here, the word reservoir means

Distinguish between the Students identify what an author hopes The phrase “Arkansas had been injured and kept back by
denotative and connotative to convey by using a particular word generations of exaggerations” is used in the first paragraph in
meanings of words, and in a specific context rather than a word this passage. The word injured is used here because it has a
interpret the connotative power with a similar meaning. Students may connotation of
of words. also analyze why the author uses
italics or quotation marks, or some In the second paragraph, the author puts the word “act” in
other device to change a word’s quotation marks because
connotative meaning. Near the end of the article, the author uses the words
“contentious,” “interminable,” and “recurrent” to describe the
rebuilding process. He does this in order to create a sense of

Reading/Literature Test Specifications 3 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

READ TO PERFORM A TASK Score Reporting Category 2


Common Curriculum Goal: Find, understand, and use specific information in a variety of texts across the subject areas to perform a task.

High School Standards Explanation Sample Stems/Questions


Synthesize information found in Students synthesize information One expert says, “The more television you watch,
various parts of charts, tables, presented in a variety of formats, the lower your reading ability.” How well is his
diagrams, glossaries, or related grade- including charts, graphs, and diagrams, conclusion supported by these charts?
level text to reach supported to gain information and reach logical
conclusions. conclusions. Students may be asked to If you wanted a snack and you were only
recognize the relationship between ideas concerned about low total fat, which would you
presented in a text passage and those choose?
found in a graphic representation, e.g., a If you were 15 years old and got a job at a local
chart or graph.
business, which of the following schedules could
you legally work?

Analyze the structure and format of Students interpret information found in Both options use a table to list the costs of various
job and consumer-related materials, texts associated with jobs (applications, passes. Why is there a difference in the way they
including the graphics and headers, position requirements, brochures are laid out?
and explain how the features support detailing labor regulations) and
the intended purpose. consumer-related materials (labels, What is the purpose for including both the Key
directions, warranties, disclaimers), to Features chart and the Performance Rating chart?
gain information and reach logical The author uses a series of subheadings and
conclusions. Students also analyze arrows in this document. The main purpose of
structural and format features to evaluate
these is to
the materials.
The author uses shading in certain parts of this
chart to

Reading/Literature Test Specifications 4 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

DEMONSTRATE GENERAL UNDERSTANDING Score Reporting Category 3


Common Curriculum Goals: Demonstrate general understanding of grade-level informational text across the subject areas.
Demonstrate general understanding of grade-level literary text.

High School Standards Explanation Sample Stems/Questions


Identify and/or summarize sequence Students show the ability to Which of these events preceded the others?
Informational Text

of events, main ideas, facts, comprehend informational text,


Which statement from the article provides the
supporting details, and opinions in including the sequence of an article’s
informational and practical selections. events. Students recognize sentences best summary of its main idea?
that identify the main idea and recall According to this article, hypertension is most
evidence used as support. closely related to a person’s
According to the author, why did many
American fliers of 1918 feel cheated?

Identify and/or summarize sequence Students show the ability to follow What do the subjects discussed in the second,
of events, main ideas, and supporting the chronology of a story’s plot, third, and fourth stanzas all have in common?
details in literary selections. placing the events in order if asked,
as well as recall important supporting When is the first time Grace regrets her decision
details in grade-level text. to withdraw from school?
Literary Text

If placed in a chronological sequence, which of


the following events would have occurred first?
Why does the foreman want to be rid of the
“pests”?
Which statement from the poem most accurately
reflects the poet’s main idea?
Which event happened closest in time to when
this scene begins?

Reading/Literature Test Specifications 5 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

DEVELOP AN INTERPRETATION Score Reporting Category 4


Common Curriculum Goals: Develop an interpretation of grade-level informational text across the subject areas.
Develop an interpretation of grade-level literary text.

High School Standards Explanation Sample Stems/Questions


Predict probable future outcomes Students use information explicitly What do you think is the most likely reason that Marge might
supported by the text, including stated to predict likely future events, ever leave her place at the junction?
foreshadowing clues. or situations. Students identify What do the last two sentences suggest about the author’s
specific clues that foreshadow later conclusions about the future?
information or future outcomes.

Infer an author’s unstated meaning Students use explicit (facts, events) The information in this article supports which conclusion about
and draw conclusions about an and implicit (images, patterns or a child’s visit to the dentist?
author’s stated meaning based on symbols) information in the text to The information provided in this article supports all of the
facts, events, images, patterns or reach supported conclusions about the following conclusions except
Informational Text

symbols found in text. author’s meaning.

Make reasoned assertions about an Students find evidence within the text Based on the information the author presents in this selection, he
author’s arguments by using to support conclusions about the would most likely argue that
elements of the text to defend and author’s perspective or major The author’s main objection to the concept of raising the speed
clarify interpretations. arguments. limit appears to be

Analyze implicit relationships, Students examine different types of By comparing Muir and Crusoe, Stafford wants readers to know
such as cause-and-effect, relationships established by the author that
sequence-time relationships, within a text to reach logical and The authors suggest that approval voting would cut down on
comparisons, classifications, and supported interpretations and negative campaigning. They feel this way because
generalizations. conclusions.

Infer the main idea when it is not Students distinguish among statements What does the author most want readers to learn from this
explicitly stated, and support with giving the main idea of a selection and article?
evidence from the text. those that present supporting details or Which choice communicates this article’s main idea?
inaccurate interpretations of the text.

Reading/Literature Test Specifications 6 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

DEVELOP AN INTERPRETATION (cont.) Score Reporting Category 4


Common Curriculum Goals: Develop an interpretation of grade-level informational text across the subject areas.
Develop an interpretation of grade-level literary text.

High School Standards Explanation Sample Stems/Questions


Predict probable future outcomes Students use clues included in the What does the last line in the story suggest about the old man’s
supported by the text. text to predict events or actions future?
that are likely to occur later in the Although this passage doesn’t actually say, what do you think is most
story. likely to happen in the future?

Analyze interactions between Students examine the interactions The interactions between the two characters in this excerpt would
characters in a literary text (e.g., between characters within a text to indicate that their relationship is
internal and external conflicts, determine the motivations behind What prevents Hartshaw from revealing to his father the secret that
Literary Text

motivations, relationships, their actions, what their could resolve their conflict?
influences) and how these relationship might be, and how Why does Kate hesitate to dance with her son, even though it is clear
interactions affect the plot. their interaction affects the plot. that he wishes to?

Identify themes in literary works, Students interpret a story to What central message or theme is being expressed in this poem (play,
and provide support for identify its prevalent theme or story)?
interpretations from the text. themes. Which line (sentence, event) in the text provides information crucial to
understanding the overall theme?

Infer the main idea when it is not Students summarize the main or Although the selection doesn’t really say it, the author probably wants
explicitly stated, and support with most important ideas being readers to realize that
evidence from the text. expressed in a literary selection. Which of the following most accurately reflects a main idea of this
selection?

Identify and analyze unstated Students infer character Why does the narrator of this poem steal the desktop?
reasons for actions or beliefs motivations when they are not Why does the main character disobey orders and allow the rabbit to
based on explicitly stated directly stated escape at the end of this selection?
information.
The author describes himself as “a/realist poet” because

Reading/Literature Test Specifications 7 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

EXAMINE CONTENT AND STRUCTURE: Score Reporting Category 5


INFORMATIONAL TEXT

Common Curriculum Goals: Examine content and structure of grade-level informational text across the subject areas.

High School Standards Explanation Sample Stems/Questions


Draw conclusions about the Students identify the author’s purpose for writing The author probably wrote this article so readers would
author’s purpose based on a piece of text and the inclusion or placement of Why does the author include information about the Pavia, Italy tower
evidence in the text. some of the information.

Differentiate among reasoning Students analyze text to examine the kind, The author supports some of his arguments using facts and others
based on fact versus reasoning quality, and quantity of reasoning used by an through opinions. Which statement is an unsupported opinion?
based on opinions, emotional author to substantiate arguments. Students may be The author’s bias comes through most clearly by examining
appeals, or other persuasive asked to find instances where emotional appeals
The story about the poacher that concludes this passage is intended to
techniques. or other persuasion is present.

Evaluate if and how the author Students examine text in order to evaluate the Which of the following does the author use to support her arguments?
uses authoritative sources to quality of source material an author uses to Testimony from experts is provided in support of which argument?
establish credibility for provide support for arguments and suggested
Which of the quotations below has credible support for the claim it
arguments, proposed actions, actions and policies.
makes?
or policies.

Compare and contrast Students read two “companion” or similarly- Which is true about the way both authors support their positions?
information on the same topic themed pieces and analyze the two texts to find All of the following qualities are shared by these two passages EXCEPT
after reading several passages similarities and differences.
Which of the following statements would be supported by the author
or articles.
of the first passage but rejected by the author of the second?
Although they deal with the same subject, the two passages differ in
that they

Evaluate the logic, unity, and Students evaluate how effectively an author has The claim and response structure of this text is effective because it
consistency of text. structured a sample of informational text. What does the author use as a unifying device in this passage?
In terms of the flow of information, what detail seems out of place?

Reading/Literature Test Specifications 8 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

EXAMINE CONTENT AND STRUCTURE: Score Reporting Category 6


LITERARY TEXT
Common Curriculum Goals: Examine content and structure of grade-level literary text.
High School Standards Explanation Sample Stems/Questions
Identify various literary devices, Students will identify the use of Which statement is true about the author’s use of fire as a symbol?
including figurative language, literary devices within a work, as This passage derives much of its humor from the use of
imagery, allegory, and symbolism; well as evaluate what impact they
Which of the following devices is used most extensively to appeal to the
evaluate the significance of the have on that selection.
reader’s senses in this passage?
devices; and explain their appeal.

Interpret and evaluate the impact of Students evaluate the author’s use Irony depends on the contrast between what exists and what would seem
subtleties, contradictions, and ironies of plot elements, twists, and appropriate. Which contrast in the story is ironic?
in a text. ironies within a literary text. The author uses irony to inject humor into the play through the line

Explain how voice and the choice of a Students analyze and evaluate the How might this passage have been different if told from Gatsby’s point of
narrator affect characterization and the point of view used by an author to view rather than Nick’s?
tone, plot, and credibility of a text. narrate a story The likely reason Fitzgerald chose Nick to narrate this instead of Daisy is

Analyze an author’s development of Students analyze how the author What does the author gain by beginning this selection in the present and
time and sequence, including the use manipulates the time sequence for using flashback to fill in the past?
of complex literary devices, such as effect within a literary passage. Which of the following is most likely included to foreshadow future events?
foreshadowing or flashbacks.

Evaluate the impact of word choice Students analyze the impact the The author’s word choice in this selection is designed to create a tone of
and figurative language on tone, author’s use of words has on a The main effect of the similes used in the first paragraph is to set a mood of
mood, and theme. story’s presentation.

Identify and describe the function of Students analyze the presence The dialogue serves all of the following purposes in this scene except
dialogue, soliloquies, asides, character and effect of structural If performed in a traditional way, the asides written into Eugene’s
foils, and stage directions in dramatic elements used by an author monologue would be delivered
literature. within a scene from a play.
The stage directions noted in this scene are primarily included to

Reading/Literature Test Specifications 9 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

Item Specifications rarely and only when it offers substantial advantages


for the item construction.
Oregon Assessment of Knowledge and Skills (OAKS) is a Items should be free of absolute wording, such as
multiple choice statewide assessment scored by the state. It is a “always” and “never,” and have qualifying words
required assessment that provides the base for the (e.g., LEAST, MOST, EXCEPT) printed in small caps for
accountability system. It also measures proficiency in the emphasis.
Essential Skills and the reading assessment is one way to Masculine pronouns should NOT be used to refer to
determine a student’s eligibility for a high school diploma or both sexes. Plural forms should be used whenever
modified diploma beginning with the graduating class of 2012. possible to avoid gender-specific pronouns (e.g.,
instead of “The student will make changes so that he
Criteria for All OAKS Test Questions ….,” use “The students will make changes so that
Test items must they….”).
be appropriate for students in terms of grade-level Test items aligned to standards may contain
difficulty, cognitive complexity, reading level, interest extraneous information.
and experience. Each Score Reporting Category will have items with
be free of age, gender, ethnic, religious, socioeconomic, or a range of difficulty and complexity levels.
disability stereotypes or bias. Each test item will measure only one Score Reporting
provide clear and complete instructions to students. Category
Item Style and Format Criteria
Criteria for Reading and Literature Passages
Test items will be in the form of questions or
sentences that require completion. Various types of reading selections will be included in the test.
Selections will reflect that reading serves three basic purposes:
Each item will have four answer choices. Students
will be told in the test directions to choose the best o Reading for literary experience (Literary selections,
answer from among the choices. including fiction, drama, poetry);
Answer choices will be stacked, whenever possible,
o Reading to gain information (Informative
and arranged from shortest to longest or alternating
selections, including articles, biographies,
lengths (short, long, short, long).
autobiographies); and
Neither “None of the above” nor “All of the above”
will be used as one of the four answer choices. o Reading to perform a task (Practical selections,
Test items may be worded in the negative (“Which of including instructions, reference materials).
these is NOT …”), but this structure will be used

Reading/Literature Test Specifications 10 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

Each selection will be free of age, gender, geographic, In order to provide an appropriate testing experience for as
ethnic, socioeconomic, physical disability or religious many students as possible, passages and items reflecting a
stereotypes. (For this purpose, passages have been broad spectrum of reading abilities will be part of the
reviewed by a sensitivity panel that applies specific criteria passage pool. This means that some passages will be
to each passage and item to assure that it is appropriate for appreciably higher or lower in difficulty than their targeted
state testing.) Many selections have been chosen grade level in terms of readability.
specifically because they show diverse peoples, cultures
and time periods. Excerpted selections will contain enough substance to
“stand alone” and to support the test items which are based
on them. In most cases, this will mean that selections have
Materials that focus on religious themes or deal with
a beginning, middle and end.
controversial subject matter (e.g., death, scenes of extreme
violence, drug and alcohol abuse, criminal activities, or the Each selection will be preceded by a title and an
occult) will not be used. introduction that provides the source of the selection or
some context to promote student comprehension. These
As much as is possible, each grade level will have some introductions are designed to stimulate interest in reading
selections by Oregon and/or Northwest authors. the selection. The introduction may provide the answer to
one or more questions.
Selections will vary in length from a few paragraphs to
several pages and will be appropriate in length, reading Illustrations that are integral to the passage (i.e., they show
level, experience and interest for the grade level tested. The incidents from the story or details important to the
following general guidelines regarding length are used for comprehension of informational text) are included in many
passage selection: reading selections. Illustrations that relate to the content of
the passages, but are not integral to them, appear within the
Grade 3, 300 words or less (150-250 on average)
text of most selections.
Grade 4, 400 words or less (250-350 on average)
Graphics for selections that are testing Reading to Perform
Grade 5, 500 words or less (350-450 on average) a Task will appear on the same screen or page as the
Grade 6, 600 words or less (500-600 on average) questions that relate to them.
Grade 7, 700 words or less (600-700 on average) Type size will be appropriate for the grade level.
Grade 8, 800 words or less (700-800 on average) Selections will be similar in format to excerpts from
High School, 1,000 words or less (800-900 on average) content textbooks, literature or practical reading tasks.

Reading/Literature Test Specifications 11 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

Criteria for Reading and Literature Test Items the state assessment, students are asked to focus primarily on
context clues. Passages providing context clues include well-
Each reading selection will be followed by at least 3 and no known, high-frequency words that explain the target word’s
more than ten knowledge and skills test items. meaning. The clues may be presented in a directly-stated
phrase, in sentences prior to or after the target word’s use, or
Each test item will measure only one Score Reporting may be found through a careful reading of the entire text. At
Category (see the following Definitions of Score Reporting varying grade levels, students may also be asked to use context
Categories for the list). clues to determine the meanings of words with multiple
meanings, or phrases such as idioms and figurative
Each test item will be followed by a set of four answer expressions.
choices. “None of the above” will not be used as an answer
choice. Read to Perform a Task (SRC 2)
When reading to perform a task, students use skimming and
Test items will be stated in the clearest manner possible. scanning techniques to search for information in what is termed
“practical” text. Depending on the grade, practical text may
Words which qualify the choice in test items (LEAST, MOST, include charts, schedules, directions, recipes, forms, maps,
FIRST, BEST, EXCEPT, PROBABLY) will be printed in small graphs, or job and consumer-related materials. The reader’s
capitals when it is likely that students may otherwise purpose is to look for information in order to do something. At
overlook the qualifier. grade 8 and high school level, questions will ask students to
synthesize information and reach logical conclusions, not
When testing Vocabulary, words will be underlined in the simply to understand the selection’s content.
reading selection and in the quote of the sentence or portion
of the sentence when it appears in the test question. If the Demonstrate General Understanding (SRC 3)
tested word is repeated in the test question, it will be Students show a general understanding by accurately
italicized. responding to questions that cover material that is explicitly
stated in the text. After reading informational text, students
Definitions of the Score Reporting Categories (SRCs) might be asked to identify an article’s topic statement, recall
the correct sequence of events, or identify important details that
were stated in the reading passage. Similarly, after reading
Vocabulary (SRC 1)
literary text, students might be asked questions about the
In this skill area, students use appropriate strategies to
sequence of events in the plot, or to identify details or events
determine the meaning of unknown words. For the items on
that were critical to the development of the plot.

Reading/Literature Test Specifications 12 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

Develop an Interpretation (SRC 4) Examine Content and Structure: Literary Text


To develop an interpretation, students must look beyond what (SRC 6)
is explicitly stated in a selection and show a more complete Examining content and structure requires students to critically
understanding of what was read. For informational text, analyze and evaluate text. Students stand apart from the text,
questions include drawing inferences about the author’s consider it objectively, and evaluate its quality and
meaning, making predictions about forthcoming information in effectiveness. For literary text, students evaluate the use of
the text or events that are likely to occur in the future, and literary elements and devices and the impact and purpose of
drawing conclusions about reasons for actions when those their use within a selection. Questions may ask students to
reasons are not explicitly stated. For literary text, students examine selections to determine their mood or tone, and how
make predictions about events likely to happen later in the that mood or tone is achieved. While questions about literary
story, interpret the story to uncover its themes, and draw forms are, for the most part, no longer part of what is tested
conclusions about traits present in the character and (see exception at grade 5), literary genre questions may be
motivations for their actions. asked at specific grades (poetry at grade 6 and drama at the
high school level). At the upper grades, students may be asked
Examine Content and Structure: Informational Text to compare the treatment of themes and make connections
(SRC 5) between two literary selections. This reporting category is not
Examining content and structure requires students to critically assessed at grades 3 and 4.
analyze and evaluate text. Students stand apart from the text,
consider it objectively, and evaluate its quality and
effectiveness. For informational text, questions ask students to .
consider the author’s purpose and style. Depending on the
grade level, students may be asked questions about instances
where the author has relied upon facts or opinions, which
arguments or statements have support, if the passage has
evidence of bias, and what structural elements are present in
the work. At the upper grades, students may be asked to
compare information and make connections across parts of a
text or between texts. This reporting category is not assessed at
grade 3.

Reading/Literature Test Specifications 13 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

Test Blueprint
2. Alignment of items to the standards is further verified
Introduction during a review by members of a Content and
The blueprints used to construct Knowledge and Skills Tests Assessment Panel, who ensure items not only match the
for Reading/Literature prescribe the: standards, but also verify overall quality and
appropriateness. Reviewers either accept items as a
Score Reporting Categories (SRC) included on each strong match to the targeted standards, edit items to
test, achieve a strong match, or reject items which do not
the cognitive demand and difficulty level of items as strongly match the standards.
distributed on a test form,
the number and percentages of test items from each The Appendix to this document includes additional evidence
SRC included on each test, and describing procedures ensuring alignment during item
the total number and percentages of operational and development, including description of Item Development and
field test items included for each test. the Life of an Item.

Teachers and other educators have historically played a vital Content Coverage
role in the development of these specifications and blueprints Prior to item writing activities, item databases are reviewed to
by serving on Content and Assessment Panels and other review determine the extent that the available items represent the
groups. These groups have advised the Department as content emphasis and content in the standards. If any content standards
standards have been developed, and have helped establish are underrepresented in the item pool, they are identified and
priorities on which standards to assess and the weighting of the targeted specifically for additional item development. This
strands within each content area assessment. assures that every year the proportion of items in the item pools
is comparable in both emphasis and content to the content
standards. The table on page 13 describes the emphasis given
Alignment of Test Items to Grade Level Standards
to each content strand; this emphasis is reflected in both the
Test items are carefully aligned to content standards at the item pools and administered tests (OAKS online and Braille
appropriate grade level through a rigorous process at two paper and pencil).
points in the test item development process:
For electronic administration, all tests and the item pools from
1. At item development workshops, item writers are which they are constructed follow the weighting of each
provided with adopted content standards to which they content strand as reflected in the table on page 13. Items within
must write test items; during a peer review process, this a stand, or SRC, are selected to provide a range of difficulty so
alignment is verified by another grade level item that the progressive nature of the test is maintained as students
developer. of varied ability levels are presented with items most

Reading/Literature Test Specifications 14 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

appropriate to their ability from that pool. Although a student Correct key placement is controlled on Braille paper/pencil
may not see an item addressing every one of the assessable tests to restrict a repeat key to 3 or less in sequence. Key
standards in a single test event, the item pool contains multiple placement cannot be controlled for online-progressive
items for each content standard at a variety of difficulty levels. assessments, so to ensure more random correct keys, item
writers are instructed to rotate the correct key for their items
In addition, the adaptive algorithm specifically considers during item authoring.
alignment criteria when drawing test items. As a result, we
accomplish a dual purpose of creating a test form that is Items for reading passages are grouped so that they are
appropriately developed for each student and it meets the comparable in difficulty. RIT values for an item set generally
criteria set forth for alignment (e.g., balance of representation, will not vary by more than 10 points from the least difficult
depth of knowledge). item in an item set to the most difficult item in that set. Rare
exceptions to this occur when the only available items to
For Braille paper and pencil tests, the item set will include ensure proper weighting of SRCs have a difficulty level outside
items aligned with as many of that grade level’s standards as the passage’s difficulty range.
possible. Across years, Braille paper and pencil test forms will
be designed to sample items from all adopted standards at that Passage Types and Lengths
grade level.
Online tests are set up to deliver a set proportion of passage
In order to report subscores, or strand scores, no less than six types during each testing event: for longer forms, four literary
items will be used. pieces, four informational pieces, and two practical pieces
(directions, tables, charts, graphs, etc.) will appear.
Additional Test Design Criteria
Passage lengths/word counts fall within a certain range at each
Each item assesses only one SRC at one grade. grade level. This is done to make sure that students are required
to read a comparable amount of material no matter what
Online test opportunities provide a range and breadth of passages are delivered to them during a progressive testing
items within each SRC and sub-SRC. Test pools attempt to experience.
provide a minimum of one item at each difficulty level for each
piece of testable content. Test pools range in size from 350 to The range for each grade level is listed earlier in this document.
500 or more items.

Reading/Literature Test Specifications 15 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

Weighting of Score Reporting Categories


The chart below shows the score reporting categories for each of the grade levels and the percentage of questions on a given test
administration that would be assessing that category. For example, at grade 3, 28% of the items on a test assess Vocabulary, which
equals about 14 items on a 50-item test. NA indicates that skills for a particular SRC are not assessed at that grade level.

Reading/Literature Test Specifications 16 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

Reading/Literature Test Blueprint—High School

Score Reporting Number of % of questions OAKS Online


Testable
Categories KS items assessed per Pool size
content
on OAKS test
codes
online
Vocabulary 1.1.HS-1.3.HS 9-12 20% 100
Read to
2.1.HS-2.2.HS 6-8 12% 70
Perform a Task
Demonstrate 3.1.HS
General 7-10 16% 120
3.2.HS
Understanding
Develop an 4.1.HS-4.5.HS
9-12 20% 120
Interpretation 4.6.HS-4.10.HS

Ex. Content
and Structure: 5.1.HS-5.5.HS 7-10 16% 75
Inform. Text
Ex. Content
and Structure: 6.1.HS-6.6.HS 7-10 16% 50
Literary Text

Operational
Item Total 50 -

Field Test Item


Total (Inserted 12 -
in January)
Total items on
Test 62 100%

Reading/Literature Test Specifications 17 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School
Cognitive Demand and Item Distribution by Difficulty

The reading/literature test pools and Braille paper/pencil tests


are designed so that each test includes items that encompass a
range of depth of knowledge and level of difficulty. The
difficulty distribution goals for test pools and Braille Difficulty Criteria for High School Assessment
paper/pencil tests are outlined in the chart opposite. The three
depth of knowledge levels used to qualify Oregon’s multiple
choice test items in reading/literature are:
High School Target Item Pool
Recall/Literal Comprehension: Item requires student to Difficulty
recall a fact, information, or procedure from the passage.
Reading/Literature Distribution
Interpretation/Application: Item requires a student to use a Difficulty
skill or concept, or to infer information not directly stated
215-227 33%
in the passage.
Analysis/Evaluation: Item requires a student to use 228-239 33%
analytical skills to support a conclusion about the author’s
purpose or craft. 240-252 33%
Cognitive Demand
Online progressive tests provide students with questions at the
beginning of the test at or about the mean RIT level and as the Recall/Literal Comprehension 25%
student responds, makes adjustments by selecting appropriate
items for each student based upon his or her correct and Interpretation/Application 40%
incorrect responses.
Analysis/Evaluation 35%
Student scores on each test will vary due to performance and
the set of unique test items issued to the student. Generally, RIT Range 215-252
students will earn scores between the maximum high and
minimum low range. The following are the possible high and Mean RIT 233
low RIT student scores for high school tests, within one or two
points, based on a given year’s item pool.

High RIT 300


Low RIT 175

Reading/Literature Test Specifications 18 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

Achievement Level Descriptors


Achievement level descriptors describe what students know and can do based on their performance on statewide knowledge and skills
tests in the various content areas. These may be used by educators to target instruction and inform parents and students of the
expectations for students to be considered proficient at a particular grade level.

The Achievement Level Descriptors are based on a sampling of a larger set of testable content outlined in the Oregon Content
Standards. Results for individual students are only one indicator of student ability as measured at the time of testing. These statements
give a general description of what most students know and can do within a particular band of achievement and are presented in the
order of the way they are reported rather than by importance or test emphasis.

Students who score at or within a particular level of achievement possess the bulk of the abilities described at that level and generally
have mastered the skills described in the preceding achievement levels.

Achievement Level Descriptors for each subject area were developed by groups of parents, educators, and business people who
worked with state officials to establish the minimum scores required for Exceeds, Meets, Nearly Meets and Does Not Yet Meet.

Reading/Literature Test Specifications 19 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

High School Achievement Level Descriptors


Levels Does Not Yet Meet Nearly Meets Meets Exceeds
Policy Definitions Students do not demonstrate Students demonstrate partial Students demonstrate Students demonstrate
(Apply to all grades and all mastery of grade-level mastery of grade-level mastery of the grade-level mastery of grade-level
subjects) knowledge and skills required knowledge and skills required knowledge and skills knowledge and skills
for proficiency. for proficiency. required for proficiency. exceeding the requirement
for proficiency.
Reading Achievement Level Students do not demonstrate Students demonstrate a Students demonstrate Students demonstrate a
Definitions mastery of grade level partial mastery of mastery of strong mastery of
(Apply to all grades) Reading/Literature Reading/Literature knowledge Reading/Literature Reading/Literature
knowledge and skills. They and skills by comprehending knowledge and skills by knowledge and skills by
have a limited the literal meaning of grade accurately comprehending thoroughly comprehending
comprehension of grade level level text. They are able to grade level text. They complex and challenging
text and cannot make make obvious interpretations have the skills to interpret text. They are able to make
meaningful interpretations or but often lack analysis skills. and analyze text. thoughtful interpretations
an analysis of the text. and evaluations.

Reading Achievement Level Are unable to determine Primarily use immediate Accurately use Effectively use subtle,
Descriptors the meaning of unfamiliar preceding context to contextual and abstract contextual and
vocabulary or phrases due determine the meaning of structural clues to structural clues to
to a weak understanding of unfamiliar vocabulary, determine the determine vocabulary
VOCABULARY contextual and structural struggling when the context meanings of unfamiliar meaning,
clues. involves distant clues. vocabulary Accurately analyze the
Make only a literal May not consistently literal and figurative
interpretation of figurative interpret the figurative meanings of words and
expressions. meanings of words and phrases,
May recognize denotative phrases. Effectively interpret the
meanings of words, but not Demonstrate the ability connotative power of
connotative. to distinguish between words.
the denotative and
connotative meanings
of words.

Reading/Literature Test Specifications 20 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

High School Achievement Level Descriptors

Levels Does Not Yet Meet Nearly Meets Meets Exceeds


Reading Achievement Level Misuse or ignore aspects of Find information isolated in Find information in Given complex technical or
Descriptors practical text (headings, one area in practical text, practical text and often practical text, synthesize
bold print, numbering) to and are sometimes able to synthesize information information found in a
decipher text when synthesize information across diagrams, charts, variety of formats to reach
READ TO PERFORM A TASK reading to perform a task. across diagrams, charts, and and tables to reach supported conclusions
tables to reach logical logical conclusions when reading to perform
conclusions when reading when reading to a task.
to perform a task. perform a task. Analyze how the structure
At times overlook how Recognize how the and format of job and
features of practical text, structure and format of consumer-related
including graphics and job and consumer- materials support their
headers, support their related materials purpose.
intended purpose. support their purpose.

Are unable to identify the Struggle to differentiate Accurately Demonstrate general


Reading Achievement Level main ideas of main ideas from demonstrate general understanding of complex
Descriptors informational text when developmental details in understanding of texts by recognizing and
asked to demonstrate informational text when grade-level text by comparing main ideas that
general understanding. asked to demonstrate determining main ideas are explicitly stated or
Frequently confuse the general understanding. explicitly stated in implied in informational
DEMONSTRATE GENERAL main idea with supporting May identify plot events in informational text and text and interpret details
UNDERSTANDING details in informative and literary text, but often the details supporting and references supporting
literary texts. struggle to determine the these ideas. these ideas.
A lack of comprehension sequence of ideas and Recall important Interpret important
prevents following the details in more complex supporting details in supporting details in
sequence of informational pieces. literary selections but literary selections.
or literary text. may be inaccurate in Understand sequence and
more complex texts. structure in a variety of
Correctly identify the textual forms.
correct sequence of
events in a story’s plot.

Reading/Literature Test Specifications 21 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

High School Achievement Level Descriptors

Levels Does Not Yet Meet Nearly Meets Meets Exceeds


Rarely make accurate May make obvious Develop an Develop an interpretation
Reading Achievement Level predictions about predictions about future interpretation by by making predictions
Descriptors upcoming events when events, but at times struggle making predictions about future outcomes
asked to develop an to make accurate about future outcomes based on clues in a
interpretation. predictions based on subtle based on clues in the passage.
Are unable to comprehend clues when asked to passage. Analyze implicit
DEVELOP AN information directly stated develop an interpretation. Identify facts about relationships.
INTERPRETATION in a text, preventing Sometimes struggle to grasp literary characters. Interpret facts, events, and
comprehension of inferred implied meaning, focusing Becoming proficient images to draw
ideas. more on literal statements when analyzing conclusions about the
Show few insights into in informational text. character interactions author’s unstated
aspects of characterization, Identify facts about literary and reasons for actions meaning.
character motivation, or a characters, but may be less and beliefs. Analyze interactions
story’s theme. proficient when analyzing Use evidence in text to between characters to
character interactions and determine themes. determine motivations and
reasons for character reasons for actions and
actions and beliefs. beliefs.
Struggle to interpret more Reach conclusions about
complex themes, intentions, the development of
and character motivation. complex or subtle themes
or messages.
Identify textual references
that support
interpretations.

Reading/Literature Test Specifications 22 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

High School Achievement Level Descriptors

Levels Does Not Yet Meet Nearly Meets Meets Exceeds


Attempt to examine the Attempt to examine the Accurately examine the Thoroughly examine the
Reading Achievement Level content and structure of content and structure of content and structure content and structure of
Descriptors informational text and informational and literary of informational and informational text to
literary text but are text but show mixed results. literary text, and can determine the author’s
generally unsuccessful. May sometimes identify a recognize the primary underlying purpose,
May be able to identify an main purpose, but struggle characteristics of each. evaluate the amount and
EXAMINE CONTENT AND author’s broad purpose, to recognize textual support Consistently identify an kind of evidence given for
STRUCTURE / but there is little or no or to make connections author’s main purpose support,
INFROMATIONAL TEXT analysis of support or between articles on similar and recognize instances Analyze passages on the
structure. subjects. of textual support. same topic, and evaluate
EXAMINE CONTENT AND Are unfamiliar with literary May sometimes recognize Make connections the logic and unity of text.
STRUCTURE / LITERARY TEXT devices, figurative literary devices and between articles on Evaluate the use of literary
language, and conventions figurative language and similar subjects. devices and figurative
of drama, which prevents begin to analyze their Recognize the use of language, and make
an analysis of their impact on a selection’s literary devices and insightful observations
purpose or use. effectiveness. figurative language, and about their impact on a
Limited familiarity with the make accurate selection’s effectiveness.
conventions of dramatic observations about Demonstrate a thorough
literature. their impact on a awareness of the
selection’s conventions of dramatic
effectiveness. literature
Demonstrate a
familiarity with the
conventions of
dramatic literature.

Reading/Literature Test Specifications 23 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Reading/Literature, High School

LOCAL ASSESSMENTS REQUIRED BY OAR 581-22-0615


ASSESSMENT OF ESSENTIAL SKILLS
Local Performance Assessments
School districts and public charter schools that offer instruction with the graduating class of 2012, high school students must
at grades 3 through 8 or high school must administer annual demonstrate proficiency in the Essential Skills of Reading,
local performance assessments for students in grades 3 through Writing, Speaking, and Mathematics. Students may
8 and at least once in high school for the skill areas of writing, demonstrate proficiency in these Essential Skills using any of
speaking, mathematics problem solving, and scientific inquiry. the assessment options approved by the State Board of
The purpose of the local performance assessment requirement Education.
is to ensure that students in grades 3 through high school are As of May 2009, the Oregon Assessment of Knowledge and
afforded opportunities to learn and to receive feedback Skills (OAKS) is one of the approved assessment options for
regarding their progress toward meeting specific state the Essential Skills of Reading, Writing, and Mathematics.
standards throughout their years in public schools. Another approved option for the Essential Skills of Writing,
A local performance assessment is a standardized measure Speaking, and Mathematics is the completion of work samples
(e.g., activity, exercise, problem, or work sample scored using scored locally using an official state scoring guide. Appendix A
an official state scoring guide), embedded in the school – Work Samples and State Scoring Guides of the 2009-10 Test
district’s or public charter school’s curriculum that evaluates Administration Manual provides guidance for those school
the application of students’ knowledge and skills. Local districts and public charter schools choosing to use a work
performance assessments must be designed to closely align sample to satisfy this requirement.
with state standards and to promote independent, individual The Assessment of Essential Skills Review Panel (AESRP),
student work. which consists of experts from school districts and post-
School districts and public charter schools may either use a secondary education institutions, reviews and recommends
work sample scored using an official state scoring guide or a additions or changes to the list of approved assessment
comparable measure adopted by the school district or public options. The AESRP bases its recommendations on evidence
charter school to satisfy the local performance assessment provided by the school districts, research organizations, and
requirement. Appendix A – Work Samples and State Scoring other experts that the proposed assessment option accurately
Guides of the 2009-10 Test Administration Manual provides measures the Essential Skill. The State Board of Education
guidance for those school districts and public charter schools then makes the determination whether to adopt the AESRP’s
choosing to use a work sample to satisfy this requirement. recommendations. ODE anticipates that the State Board of
Education will approve additional assessment options based
Assessment of Proficiency in the Essential Skills on recommendations from the AERSP in the coming
As part of the new graduation requirements, high school months. In addition, the AESRP is developing a set of
students must demonstrate proficiency in a set of Essential criteria for approval by the State Board of Education that
Skills, which are defined as process skills that cross academic school districts and public charter schools may use in
disciplines and are embedded in the content standards. Starting developing local assessment options.

Reading/Literature Test Specifications 24 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendices
The Appendices of this document includes additional documents that deal with test construction, design,
and Reading/Literature assessment.

Included in this section are:

Appendix A: Oregon Achievement Standards Summary

Appendix B: Cognitive Demand and Item Distribution Goals

Appendix C: Item Development Process

Appendix D: Life of an Item

Appendix E: Reading Scoring Guide


(Official scoring guide to be used for scoring Local Reading Performance Assessments for the purpose of
demonstrating proficiency in the Essential Skill of Reading in lieu of meeting the standard on the OAKS
Reading and Literature Assessment)
Appendix A Reading/Literature, High School

ACHIEVEMENT STANDARDS
2011-12 Achievement Standards Summary
The charts below show the achievement standards (requirements to meet and exceed) for Oregon’s Assessments of Knowledge and
Skills (OAKS) by content area and grade or benchmark level. All students are required to take reading/literature and mathematics
assessments in grades 3-8 and 11; in writing in grades 4, 7, and 11; and science in grades 5, 8, and 11. Assessments in social
sciences are optional; however, they may be required by some districts or schools. For detailed assessment information, refer to the
2011-12 Test Administration Manual (www.ode.state.or.us/go/TAM). It provides timelines, options, and procedures that ensure both
test reliability and validity from classroom to classroom, teacher to teacher, school to school, and district to district.

Grade 3 MEET EXCEED Grade 6 MEET EXCEED

Reading/Literature 211 224 Reading/Literature 226 237


Mathematics 212 219 Mathematics 227 237
Writing, Speaking, Science, Writing, Speaking, Science,
No state test No state test
Social Sciences Social Sciences

Grade 4 MEET EXCEED Grade 7 MEET EXCEED

Reading/Literature 216 226 Reading/Literature 229 241


Writing Writing
• Composite Score • 32 to 39* (out of 48) • 40 to 48 (out of 48) • Composite Score • 40 to 49* (out of 60) • 50 to 60 (out of 60)
• Minimum score in each trait • 3 (out of 6) • 4 (out of 6) • Minimum score in each trait • 3 (out of 6) • 4 (out of 6)
• Conventions score • Not doubled • Not doubled • Conventions score • Doubled • Doubled
Voice and Word Choice are not included in the achievement standard. Voice and Word Choice are not included in the achievement standard.
*A composite score of 28 to 31 points nearly meets the standard. Scores in *A composite score of 35 to 39 points nearly meets the standard. Scores in
this range indicate that the writing is close to meeting the standard and that this range indicate that the writing is close to meeting the standard and that
local performance assessments could be used to provide a more local performance assessments could be used to provide a more
comprehensive view of student proficiency in writing. comprehensive view of student proficiency in writing.
Mathematics 219 227 Mathematics 232 242
Speaking, Science, and Speaking, Science, and
No state test No state test
Social Sciences Social Sciences

Grade 5 MEET EXCEED Grade 8 MEET EXCEED

Reading/Literature 221 230 Reading/Literature 232 242


Mathematics 225 234 Mathematics 234 245
Science 226 239 Science 235 247
Social Sciences # 215 225 Social Sciences # 231 241
# Optional state test; may be required by districts or schools. # Optional state test; may be required by districts or schools.

Writing, Speaking No state test Writing, Speaking No state test

Reading Test Specifications A-1 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix A Reading/Literature, High School

ACHIEVEMENT STANDARDS
Achievement Standards Oregon Assessment of Knowledge and Skills (OAKS) is one
High School
for Oregon Statewide Assessments1 option to provide evidence of proficiency in Essential Skills.
Subject Area Meets Exceeds Notes Essential Skill
Content of the 2011-2012 OAKS Reading/Literature Read and
Reading/Literature 236 247 Assessment is based on the Grade Level Content comprehend a variety
Standards adopted in 2002-2003. of text
*A composite score of 35 to 39 points nearly meets the
standard. Scores in this range indicate that the writing
Writing
is close to meeting the standard and that local
• Composite Score • 40 to 49 (out of 60) • 50 to 60
performance assessments could be used to provide a Write clearly and
• Minimum score allowed in • 3 (out of 6) • 4 (out of 6)
more comprehensive view of student proficiency in accurately.
any trait
writing.
• Conventions score • Doubled • Doubled
• Score on Voice and Word Choice traits are not
included in the achievement standard.
Content of the 2011-12 OAKS Mathematics test is Apply mathematics
Mathematics 236 251 based on the Content Standards adopted in 2009 in a variety of
for high school and 2007 for grades K-8. settings
Content of the 2011-12 OAKS Science test is
Science 240 252 based on the Content Standards adopted in 2009.
Optional State Assessment; content of the 2011-
Social Sciences 239 249 12 OAKS Social Sciences Assessment is based
on the Content Standards adopted in 2001.

Achievement Standards for Demonstrating Proficiency in


Essential Skills for High School Diploma2
Essential Skill OAKS Assessment Required Scores Other Options
Reading (Class of 2012 & beyond) Reading/Literature 236 Meets Other approved standardized test;
247 Exceeds Work samples

Writing (Class of 2013 & beyond) Writing Work samples


40 Meets
Performance
50 Exceeds
Assessment
Apply Mathematics (Class of 2014 Mathematics 236 Meets Other approved standardized test;
& beyond) 251 Exceeds Work samples

1
In future years, Achievement Standards may change for the purposes of accountability and earning a high school diploma.
2 th
For purposes of demonstrating mastery of Essential Skills, students must meet the achievement standards in effect during their 8
th th
grade year. However, students may use achievement standards adopted in their 9 through 12 grade years that are equal to or
th
lower than the achievement standards approved as of March 1 of the students’ 8 grade year. In addition, students may demonstrate
th th
proficiency in the Essential Skills using additional assessment options adopted in their 9 through 12 grade years.

Reading Test Specifications A-2 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix A Reading/Literature, High School

ACHIEVEMENT STANDARDS
A Look at Work Samples as Required Local Performance Assessments
(Grades 3 – 8 and High School)
Local Performance assessments evaluate the application of students’ knowledge and skills. OAR 581-022-0615 Assessment of
Essential Skills requires students to complete one or more local performance assessments for each assessed skill area per year in
grades 3-8 and at least once in high school. The table below outlines the achievement standards for work samples scored with an
official state scoring guide and used as a local performance assessment. For detailed assessment information refer to the 2011-12
Test Administration Manual at www.ode.state.or.us/go/TAM. It provides work sample guidelines, options, and procedures that help
ensure both work sample reliability and validity from classroom to classroom, teacher to teacher, school to school, and district to
district.

Achievement Standard
for Purpose of Local
Skill Area
Performance
(Official State Grade Notes about Work Samples
Assessment
Scoring Guide)
Meets Exceeds
(out of 6) (out of 6)
Grade 3 students are not held to a standard in Sentence
Grade 3 3 4
Fluency.
Writing Voice and Word Choice may be scored but are not
Grades 4-8 and High
4 5 required traits. Exemplars reflect expectations at each
School
grade level.
Grade 3 3 4 Grade 3 students are not held to a standard in Language.
Speaking Grades 4-8 and High
4 5 Exemplars reflect expectations at each grade level.
School
Mathematics Grades 3-8 and High
4 5 Exemplars reflect expectations at each grade level.
Problem Solving1 School
Separate Official scoring guides exist for each grade/band
Grades 3-8 and High
Scientific Inquiry2 4 5 (Grade 3, Benchmark 2 (Grades 4-5), Benchmark 3
School
(Grades 6-8), and High School).

Related Web Links:


Official State Scoring Guides: www.ode.state.or.us/search/page/?id=32
Exemplars of scored work samples are currently found on subject-specific assessment pages linked from:
www.ode.state.or.us/search/page/?id=1307

1
Revised mathematics problem scoring guide was adopted by the State Board of Education (May 19, 2011) for use beginning with the
2011-2012 school year.
2
Revised scientific inquiry scoring guides and newly-developed engineering design scoring guides were adopted by the State Board
of Education (May 19, 2011) for use beginning with the 2011-2012 school year.

Reading Test Specifications A-3 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix A Reading/Literature, High School

ACHIEVEMENT STANDARDS
Using Work Samples to Assess Essential Skills for the Oregon Diploma
Essential Skills graduation requirements are determined based on when a student is first enrolled in grade 9, which is referred to as the
cohort year. These requirements are applied to students earning either the regular or modified diploma. Students who entered grade 9
in the 2008-2009 school year (most of whom will graduate in 2012) are required to demonstrate proficiency in the Essential Skill of
Reading. The remaining implementation timeline is described in the table below.

Work samples are one assessment option that high school students may use to demonstrate they are proficient in the Essential Skills.
Regarding demonstration of proficiency in the Essential Skills, districts must:
provide students with instruction in and multiple assessment opportunities to demonstrate proficiency in the Essential Skills for
the purpose of earning a high school or modified diploma.
allow students to use assessment options adopted in a student’s 9th through 12th grade years.
allow students to use achievement standards adopted in their 9th through 12th grade years that are equal to or lower than the
achievement standards approved as of March 1 of the students’ 8th grade year.

At the high school level, students may use work samples to fulfill both the local performance assessment and the Essential Skills
requirements.

The table below describes the achievement standard for work samples scored for the purpose of demonstrating proficiency in the
Essential Skills with regard to conferring a high school diploma.

Achievement Standard for


Number and Types of Work First Purpose of Conferring High
Essential Skill Scoring Guide
Samples Implementation School Diploma
(Cut Scores)
Read and comprehend 2 total work samples: Official Reading Students who Total score of 12 (6-point scale)
a variety of text at least one must be informative Scoring Guide entered grade 9 across 3 traits with no trait lower
the second may be informative in 2008-2009 than a 3; score of 5 or 6 on all
or literary traits to exceed.
Write clearly and 3 total work samples: Official Writing Students who Score of 4 (6-point scale) to
accurately one expository Scoring Guide entered grade 9 meet in each of the 4 required
in 2009-2010 traits; score of 5 or 6 to exceed.
one persuasive
one narrative sample
Apply mathematics in 2 total work samples: Official Students who Score of 4 (6-point scale) to
a variety of settings One each from two of these: Mathematics entered grade 9 meet in each required trait; score
algebra Problem in 2010-2011 of 5 or 6 to exceed.
Solving Scoring
geometry Guide
statistics

Reading Test Specifications A-4 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix B Reading/Literature, High School

Cognitive Demand and Item Distribution Goals


Oregon recognizes the importance of cognitive complexity as part of test specification. To that end, we implemented a strategy to
overtly incorporate a test design process that includes the three dimensions of content difficulty, often called depth of knowledge.
 The first step in the process occurred when we content panels were convened and asked for their determination as to what
appropriate level of cognitive complexity should be assigned to each of the content standards.
 Secondly, we analyzed the gap between the proportion of cognitive complexity available in our current item pools against the
content panels’ recommendations.
 The third step involved engaging item writers to write items to fill in the critical gaps. These items are reviewed and field
tested through our standard processes.
Cognitive complexity is controlled on reading and literature assessments by the close relationship between the strands of reading and
the three levels of cognitive complexity.. The three levels of cognitive complexity addressed in Reading/Literature and the strands or
SRC levels they generally relate to are:
Recall/Literal Comprehension: Item requires a student to recall a fact, information, or procedure from the passage.
(SRC’s 2 and 3)
Interpretation/Application: Item requires a student to use a skill or concept, or to infer information not directly stated in the
passage. (SRC’s 1 and 4)
Analysis/Evaluation: Item requires a student to use analytical skills to support a conclusion about the author’s purpose
or craft. (SRC’s 5 and 6)
Depth of Knowledge/Difficulty Distribution: Reading/Literature
Distribution in Distribution in Distribution in Distribution in
Grade 3 Test Pool Grade 4 Test Pool Grade 5 Test Pool Grade 6 Test Pool
Difficulty Difficulty Difficulty Difficulty
185-196 33% 192-203 33% 197-207 33% 201-212 33%
197-209 33% 204-216 33% 208-222 33% 213-225 33%
210-225 33% 217-232 33% 223-237 33% 226-241 33%
Cognitive Complexity Cognitive Complexity Cognitive Complexity Cognitive Complexity
Recall/Literal Recall/Literal Recall/Literal Recall/Literal
Comprehension 40% Comprehension 35% Comprehension 35% Comprehension 30%
Interpretation/ Interpretation/ Interpretation/ Interpretation/
Application 50% Application 50% Application 50% Application 40%
Analysis/ Analysis/ Analysis/ Analysis/
Evaluation 10% Evaluation 15% Evaluation 15% Evaluation 30%
RIT Range 185-225 RIT Range 192-232 RIT Range 197-237 RIT Range 201-241
Mean RIT 203 Mean RIT 210 Mean RIT 215 Mean RIT 219

Reading/Literature Test Specifications B-1 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix B Reading/Literature, High School

Cognitive Complexity/Difficulty Distribution: Reading/Literature

Distribution in Distribution in Distribution in


Grade 7 Test Pool Grade 8 Test Pool High Sch Test Pool
Difficulty Difficulty Difficulty
205-216 33% 207-219 33% 215-227 33%
217-229 33% 220-232 33% 228-239 33%
230-245 33% 233-247 33% 240-252 33%
Cognitive Complexity Cognitive Complexity Cognitive Complexity
Recall/Literal Recall/Literal Recall/Literal
Comprehension 28% Comprehension 28% Comprehension 25%
Interpretation/ Interpretation/ Interpretation/
Application 40% Application 40% Application 40%
Analysis/ Analysis/ Analysis/
Evaluation 32% Evaluation 32% Evaluation 35%
RIT Range 205-245 RIT Range 207-247 RIT Range 215-255
Mean RIT 223 Mean RIT 225 Mean RIT 233

Reading/Literature Test Specifications B-2 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix C Reading/Literature, High School

APPENDIX C: ITEM DEVELOPMENT PROCESS


Oregon’s item development process is consistent with industry following classification, developed from Bloom’s (1956)
practice and takes approximately two years, including writing, educational taxonomy:1
reviewing, and field-testing new items. Just as the development
of Oregon’s content and performance standards is an open, • Recall: Recall, label, or locate information; define or
consensus-driven process, the development of test items and describe facts or processes.
prompts to measure those constructs is grounded in a similar
philosophy. • Skill/Concept (Basic Application): Use information
or conceptual knowledge, often requiring two or more
Item Writing steps; summarize, classify, or explain information or
processes; make predictions or generalizations; solve
For the Knowledge and Skills (multiple-choice) tests and the problems.
Writing Performance Assessment, most item writing takes
place during item writing workshops, in which Oregon teachers • Strategic thinking: Analyze, critique, compare or
across the five main content areas write and review items. contrast; create new information; or organize
presented information.
Item writers are typically Oregon teachers who have received
training in item construction, are familiar with test • Extended thinking: Make connections and extensions
specifications, and have demonstrated skill in writing items (exclusively assessed in the Writing Performance
that pass content and sensitivity panel review. Item writers Assessment and local performance assessments).
receive professional development compensation for their time
and travel expenses. Among other security precautions, ODE During the item writing workshop, writers draft items,
requires item writers to sign confidentiality forms assuring that document rationale of distracters, and conduct peer reviews of
they will work with the items in a secure manner. each other’s items. Examples of items are provided, and
facilitators provide process guidance and additional review.
All items are written to measure specific subdomains of the Writers and reviewers evaluate the strength and clarity of the
content standards at a variety of specified levels of cognitive match between the drafted item and the standard it measures.
complexity. Cognitive complexity is represented by the All issues are worked out or solved multiple times by multiple

1
Bloom, B. S. (ed.), Engelhart, M. D., Furst, E. J., Hill, W. H., &
Krathwohl, D. R. (1956). Taxonomy of educational objectives: Handbook I:
Cognitive domain. New York: David McKay.

Reading/Literature Test Specifications C-1 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix C Reading/Literature, High School

reviewers who verify that distracters are plausible, that answers Following item writing workshops, items are entered into the
are correct, and that each item has only a single correct answer. Item Tracking System (ITS). Oregon’s original graphics are
initially entered into the ODE’s Comprehensive Item
Figure 1. Management System (CIMS) and then transferred to ITS.
Sample Oregon Item Writing Form Within ITS and CIMS, each item is given a unique item
identification number to facilitate the monitoring and tracking
of changes to and usage of the item throughout the review
process and each item’s history. ITS provides authorized users
with access to each item’s alignment and attributes, field-test
results and use, response rationales, and previous versions.

Although item writing workshops may still occur annually,


ODE has recently moved toward distributed item writing in
which consistently strong item writers author additional items
throughout the year. Items still go through the review process
previously described. Item writers are trained on the use of
secure item entry using ITS, and graphic drafts are scanned by
the item writers and securely transmitted to ODE.

Committee/Panel Review

ODE convenes a series of advisory groups to advise ODE both


on assessment-related policy and on item development. ODE
seeks to ensure that membership on these advisory groups
reflects the demographics of Oregon’s student population. Each
advisory group has approximately 15–35 members who serve
three-year terms with one-third of the members rotating out
each year and being replaced by new representatives. The
following table describes the structure of these groups.

Reading/Literature Test Specifications C-2 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix C Reading/Literature, High School

Structure of ODE Assessment-Related Advisory Groups


Number of
Committee/Panel Meeting Frequency Who Nominates Members?
Members
Assessment Policy Advisory Committee School districts, COSA, OSBA, OEA, ESDs, and
15–20 2-3 times a year
OPTA
Sensitivity Panel School districts, OEA, ESDs (application
15–20 4–6 times a year
process)
English/Language Arts Content and School districts, OEA, ESDs, and self-nominate
35 4-6 times a year
Assessment Panel (application process)
Mathematics Content and Assessment Panel School districts, OEA, ESDs, and self-nominate
35 4 - 6 times a year
(application process)
Science Content and School districts, OEA, ESDs, and self-nominate
Assessment Panel 35 4- 6 times a year
(application process)
Social Sciences Content and Assessment School districts, OEA, ESDs, and self-nominate
25 1 - 2 times a year
Panel (application process)
English Language Proficiency Content and School districts, OEA, ESDs, and self-nominate
35 1 – 2 times a year
Assessment Panel (application process)

Note. Oregon’s Accommodations and Modifications Review Panel is not described here.
Source: http://www.ode.state.or.us/teachlearn/testing/dev/panels/structurecapanels.doc

Panel members commit up to 6 school days of service with an ESDs who are knowledgeable about assessment-related issues.
additional 3 or 4 days during the summer. However, panels will The purpose of the Committee is to advise ODE on both the
be convened remotely rather than in person as secure procedural and policy implications of Oregon’s assessment
technology improvements allow distributed work. Although system, as well as the feasibility of proposed improvements to
committee members on district contracts are not compensated Oregon’s assessment system. Committee members provide
for their service, they do receive travel reimbursement for input regarding the various elements of the state assessment
committee travel of more than 70 miles, and substitute teachers system such as educational technology, electronic reporting,
are provided for service during the school year. When operational assessment issues, and test administration.
classroom teacher members work for ODE during non-contract In addition to seeking advice on assessment-related policy,
time, they are compensated at an hourly wage as temporary
ODE requires that all items generated for use on Oregon
employees statewide assessments must pass a series of rigorous reviews
The Assessment Policy Advisory Committee consists of before they can be used in field and operational tests. All items
representatives from Oregon school districts, schools, and go through both a content and a sensitivity review as part of the

Reading/Literature Test Specifications C-3 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix C Reading/Literature, High School

item development process; only those items that measure the the panels appraise the technical quality of items, looking for
grade-level expectations and meet both overall quality and items that are free from such flaws as (a) inappropriate
sensitivity criteria are carried forward to the field-test stage. readability level, (b) ambiguity, (c) incorrectly keyed answers
and distracters, (d) unclear instructions, and (e) factual
ODE Content and Assessment Panels exist for each of the
inaccuracy. The panels for each content area use the following
content areas for which statewide tests are given:
review process:
English/Language Arts (this panel reviews Writing and
Reading/Literature assessment items), Mathematics, Science, 1. Three content panel members review each item
Social Sciences, and English Language Proficiency. independently and complete an Item Review Form
(IRF) (figure 1) using a pre-assigned reviewer ID.
Most members of these panels are classroom teachers, with
some representation from higher education, district curriculum 2. Then, the three content panel members review the item
and assessment personnel, and related businesses. Criteria for collectively, and item reviewers make a
panel selection include the following: recommendation for each item on the IRF to either (a)
accept the item as written, (b) accept the item with
• Knowledge of Oregon’s content standards and
revisions, or (c) reject the item (sometimes an alternate
expertise in the subject area and its eligible content
question is offered that entails a simple revision).
• Teaching experience at the grade level or benchmark
to which the individual will be assigned 3. When all three reviewers agree that an item should be
• Geographical location to ensure that all regions of accepted or rejected, no further discussion is needed. If
Oregon are represented one or more of the reviewers feel that an item should be
• Gender and ethnic diversity to ensure that the panel revised, then they attempt to reach a consensus and
represents the diversity of Oregon’s student population produce a “master copy” of their recommendation. The
same is true if one or two of the reviewers reject an
Current item writers are not allowed to serve on item review item that another reviewer finds acceptable with or
committees. However, in some cases, content and assessment without revisions.
panel experts may be utilized as item writing facilitators.
4. In most cases, recommendations are followed and
Items are accepted, rejected, or modified by the Content and revisions are made, or items are eliminated. The ODE
Assessment Panel to make sure they represent the constructs assessment specialist can override the recommendation,
embodied in grade-specific content standards and test but this occurs rarely and only for compelling reasons.
specifications. In addition to judgments of content relevance,

Reading/Literature Test Specifications C-4 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix C Reading/Literature, High School

Figure 2. The content panels perform specific checks on items to confirm


Sample Oregon Content and Assessment Panel Item that:
Review Form
• the SRC and subcategory match.
• the key is correct.
• alternate valid interpretations making the distracters
correct do not exist.
• the item is grade-level appropriate in content and reading
levels.
• the item is of overall high quality (wording and
grammar, graphic quality, curricular importance, etc).
• the identified level of difficulty (i.e., easy, medium,
hard) is correct.
• Reading/Literature passages are appropriate in content
and reading levels. Science and Social Sciences stimuli
align to appropriate content and reading skills.
• the level of cognitive complexity (i.e., recall,
skill/concept or strategic thinking) is appropriate to the
item and correctly identified.

Following review by the content panel, and according to panel


feedback, ODE assessment specialists edit and revise items in ITS
in preparation for review by the Sensitivity Panel.

All items that pass review by the content specialist are next
presented to the sensitivity panel. The sensitivity panel reviews
convenes day-long meetings, four to six times a year. The panel
reviews items from all grade levels and content areas for bias,
controversial content, and overly emotional issues.

Reading/Literature Test Specifications C-5 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix C Reading/Literature, High School

In general, the sensitivity panel ensures that items: • the item is of overall high quality (wording and grammar,
graphic quality, curricular importance, etc).
• present racial, ethnic, and cultural groups in a positive
light. Following the expert review in most cases, recommendations are
• do not contain controversial, offensive, or potentially followed and revisions are made, or items are eliminated. The ODE
upsetting content. assessment specialist can override the recommendation, but this
• avoid content familiar only to specific groups of occurs rarely and only for compelling reasons.
students because of race or ethnicity, class, or
geographic location. FIELD TESTING
• aid in the elimination of stereotypes. Once the items have been reviewed by the content and assessment
• avoid words or phrases that have multiple meanings. panel, the sensitivity panel, and an expert reviewer, all Mathematics,
Reading/Literature, Science, and Social Sciences test items are field
Following the sensitivity panels and according to panel tested. Field test items identified by the ODE assessment specialists
feedback, ODE assessment specialists edit and revise items in are embedded in the operational tests by content area. As students
the ITS system. take the operational tests, they also respond to approximately 5-8
field test items embedded in the test.
EXPERT REVIEW
ODE then receives data files of the student responses, which ODE
Next, ODE assessment specialists submit the new items for
analyzes to determine whether the field test items are behaving as
review by experts that have experience in the roles of item expected. The ODE assessment specialists eliminate those items
writer and content and assessment panel member. Expert which the data analysis indicate performed weakly. ODE assessment
reviewers add an additional quality control check for the online staff calibrate the difficulty level for those items that performed
assessments. Experts have received extensive professional successfully in preparation for using the item operationally.
development in ITS to review items in a web-preview format
providing the exact rendering provided in the online
assessments. Experts review each item and confirm that:

• the key is correct.


• alternate valid interpretations making the distracters
correct do not exist.
• the item is grade-level appropriate in content and
reading levels.

Reading/Literature Test Specifications C-6 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix C Reading/Literature, High School

TRANSLATION OF ITEMS TO SPANISH translation based upon grade level and cultural relevance. A variety
of resources are used for selecting the proper translated words
Concurrent with the field testing of items in English, all including: dictionaries from Mexico, South America and Spain
Mathematics, Science, and Social Sciences test items are (e.g. Diccionario Hispanoamericano de Dudas, Diccionario de
translated into Spanish. All required grade-level and benchmark- Matemáticas), and ODE’s list of translated terms for Science at
level statewide tests for Mathematics and Science are offered in http://www.ode.state.or.us/search/page/?id=517
English-Spanish tests. English-Spanish tests are also available for and for Mathematics at
Social Sciences. Stacked English-Spanish items are used on http://www.ode.state.or.us/search/page/?id=500 .
electronic tests. Side-by-side English-Spanish and English-
Russian Paper/Pencil assessments are available in Mathematics ADDITIONAL EXPERT REVIEW OF ITEMS
and Science.
On an annual basis, ODE assessment specialists review items from
Following translation by ODE’s translation vendor, the translated the field test pool for inclusion within the operational test. This
items are reviewed by ODE’s Spanish- and Russian-speaking level of review acts as an additional quality control for the online
experts to ensure that each item accurately conveys the intent of assessments. In addition, whenever ODE transitions to a different
the English text. While the procedure described below specifically test delivery system, ODE submits all of its Reading/Literature,
addresses Spanish translation, ODE follows a similar procedure Mathematics, Science, and Social Sciences items for an additional
for translation of Paper/Pencil items into Russian. level of expert review to ensure that all items appear consistently
from year to year when presented to students.
The following linguistic guidelines are used by ODE’s translation
vendor and Spanish-speaking experts: ITEM USE AND RELEASE
• Students are expected to have subject knowledge and use Approximately every three years, ODE releases one sample test for
proper terminology/vocabulary for that subject. In other
each content area and grade-level and benchmark-level comprised
words, what is expected from English-speaking students is
of items used on previous test forms. These items are no longer
also expected from Spanish-speaking students.
secure and are taken out of the pool of eligible test items.
• ODE uses formal Spanish (usted, not tú) for test items and
includes proper verb conjugation. Released items are provided in the form of practice tests. Practice
• ODE strives to use Global Spanish language that will be tests for Reading/Literature, Mathematics, Social Sciences, and
interpreted and understood by all Spanish speakers from Science are available on ODE’s Website at
anywhere in the world. Global Spanish language includes http://www.ode.state.or.us/search/page/?id=1222 .
words used worldwide by most Spanish speakers.
Sample Writing prompts are also available at
After the ODE Spanish reviewers complete a review of the http://www.ode.state.or.us/teachlearn/subjects/elarts/writing/asses
newly translated items, extensive research is conducted by a sment/usingsampleprompts.pdf
small group of reviewers on any word that has not met group
consensus. Every attempt is made to choose the most correct

Reading/Literature Test Specifications C-7 Oregon Department of Education


and Test Blueprints Office of Assessment and Information Services
Appendix D

The complete two-year Lifecycle of a Knowledge and Skills Item


Mathematics, Reading/Literature, Science, Social Sciences
PAGE 1

1 Phase 1›Item Writing 2 Phase 2›Item Review 3 Phase 3›Field Testing

SITES SORT
A. Assessment Specialist sorts and
FIELD TEST
A. Assessment staff schedules and organizes items for review. A. Assessment Specialist identifies
coordinates item writing activities, items to be field tested.
and recruits Oregon teachers to

REVIEW
construct items to be entered into
an item database

EMBED
B. Subject Specific Content and
Assessment Panels, consisting of

WRITING
Oregon teachers, review test items
with respect to content validity and B. Field test items are embedded
grade appropriateness. in an operational test.
B. Item Writing: Teachers receive
professional development

EDIT
training on item development,
including a focus on standards

TEST
alignment and item content and
format. Items are written explicitly C. Assessment Specialist edits
to measure Oregon academic and revises items according to
content standards. content panel feedback. C. Students complete operational
tests with embedded field test items.

REVIEW REVIEW
PROCESS
D. Sensitivity Panel reviews items in
C. Teachers review items written by two-day meetings, generally held
their peers. four times a year.
D. Data files of student responses

ENTRY EDIT
are submitted to ODE for analysis.

D. After items are written, assess- E. Assessment Specialist edits and


ment staff enter items into a revises items according to Sensitivity
database. Panel feedback.

Bank of POTENTIAL items Bank of REVIEWED items Bank of FIELD items

NEXT PHASE NEXT PHASE


D-1
Appendix D

The complete two-year Lifecycle of a Knowledge and Skills Item


Mathematics, Reading/Literature, Science, Social Sciences
PAGE 2

4 Phase 4›Data 5 Phase 5›Test 6 Phase 6›Data


Analysis of Construction Analysis of Operational

SELECT
Field Test Items Test Items

ANALYZE A. Assessment Specialist selects


items for operational testing.
PRESENTED
A. Tests are sent to contractor for
A. Assessment staff generates
psychometric data to determine if print distribution or delivery online.

RANGE
the item “behaves” as expected.

B. Assessment Specialist balances


SCORES
REVIEW
items across Score Reporting
Categories (SRCs) (such as B. Students complete the opera-
Geometry in Mathematics or tional test and receive instant
Vocabulary in Reading/Literature) scores when using online
B. Assessment Specialist reviews
and range of difficulty according to delivery.
data to determine which items
should be “dropped” because of test specifications.

TEST
weak performance.

CONSTRUCT C. Assessment staff analyze item

CALIBRATE
C. Assessment staff construct statistics to verify the item
tests, online test pools,and performs as expected
finalize Administration Manual.
C. Assessment staff calibrate the
difficulty of field test items that
PROCESS
REVIEW
meet the successful criteria.

D. Assessment staff analyze item


D. Assessment staff and statistics to make sure items are
expert reviewers proofread test not biased against a particular sub-
items and stimuli for errors. group (e.g., students with
disabilities, ethnic groups, or gender).

FINAL TARGET
E. Item performance tables which
E. Final Operational Tests and pools
Bank of CALIBRATED items describe how well each item
are prepared.
performs are used to review items
and pools of items to identify any
NEXT PHASE NEXT PHASE additional items to be dropped.
D-2
Oregon Department of Education

255 Capitol St NE, Salem, Oregon 97310 (503) 947-5600

You might also like