STANDARDIZED

TESTING
Group 3

1. Sadira Elin Kinasih (2520007)

2. Nur Rahmah (2520008)
3. Hanna Fairussania (2520012)
4. Ma’rifatul Azkiyah (2520014)
TABLE OF CONTENTS
01 The Meaning of
Standardization
02 Plus and Minus of
Standardized Test

03 Developing a
04 Standardized
Standardized Test Language
Proficiency Testing
05 Four Standardized
Language
Proficiency Tests
 01.
The Meaning of
Standardization
 What is Standardization Test?
A standardized test presupposes certain standard objectives, or criteria, that
are held constant across one form of the test to another. The criteria in
large-scale standardized tests are designed to apply to a broad band of
competencies that are usually not exclusive to one particular curriculum. A
good standardized test is the product of a thorough process of empirical
research and development. It dictates standard procedures for
administration and scoring. And finally, it is typical of a norm-referenced
test, the goal of which is to place test-takers on a continuum across a range
of scores and to differentiate test-takers by their relative ranking. For
example: TOEFL, IELTS, or College Entrance Exam.
02.
Plus Minus of
Standardized Test
 + Plus
1. A ready-made previously validated product that frees the teacher
from having to spend hours creating a test.
2. Administration to large groups can be accomplished within
reasonable time limits.
3. In the case of multiple-choice formats, scoring procedures are
streamlined (for either scannable computerized scoring or hand-
scoring with a hole-punched grid) for fast turnaround time.
4. There is often an air of face validity to such authoritative-looking
instruments.
 - Minus
1. Using an overall proficiency test as an achievement test simply
because of the convenience of the standardization.
2. The potential misunderstanding of the difference between direct and
indirect testing.
03.
Developing
Standardized Test
 How are Standardized Test Developed?

In the steps outlined below, three different standardized tests will be

used to exemplify the process of standardized test design:

01 The Test of English as a Foreign Language (TOEFL), Educational Testing

Service (ETS).

02 The English as a Second Language Placement Test (ESLPT), San Francisco

State University (SFSU).

03 The Graduate Essay Test (GET), SFSU.

01 Determine the purpose and objectives of
the test

Most standardized tests are expected to provide high practicality in

administration and scoring without unduly compromising validity. It is
therefore important for its purpose and objectives to be stated specifically.

A. The purpose of the TOEFL is “to evaluate the English proficiency of

people whose native language is not English” (TOEFL Test and Score
Manual, 2001, p. 9). More specifically, the TOEFL is designed to help
institutions of higher learning make “valid decisions concerning English
language proficiency in terms of [their] own requirements” (p. 9).
B. The ESLPT, referred to in Chapter 3, is designed to place already admitted
students at San Francisco State University in an appropriate course in academic
writing, with the secondary goal of placing students into courses in oral
production and grammar-editing.

C. The GET. Another test designed at SFSU, is given to prospective graduate

students-both native and non-native speakers-in all disciplines to determine
whether their writing ability is sufficient to permit them to enter graduate-level
courses in their programs
02 Design test specifications
Before specs can be addressed, a comprehensive program of research must identify
a set of constructs underlying the test it self

a) Construct validation for the TOEFL is carried out by the TOEFL staff at ETS under
the guidance of a Policy Council that works with a Committee of Examiners that is
composed of appointed external university faculty, linguists, and assessment
specialists.
Because of TOEFL is a proficiency test, the first step in the developmental process is
to define the construct of language proficiency.
(Bachman & Palmer, 1996) prefer the term ability to proficiency and thus speak of
language ability as the overarching concept.
TOEFL Specifications

- Listening Section
The listening section measures the examinee’s ability to understand english. Conversational features of
the language are stressed, and the skills tested include vocabulary and idiomatic expression as well as
special grammatical constructions that are frequently used in spoken english.

- Structure section
The Structure section measures an the examinee’s ability to recognize language that is appropriate for
standard written english.

- Reading Section
The Reading Section measures the ability to read and understand short passages similar in topic and
style to academic texts.

- Writing Section
The Writing Section measures the ability to write in english, including the ability to generate, organize, and
development ideas, to support those ideas with examples or evidence, and to compose a response to one
assigned topic in standard written english.
b) The designing of the test specs for the ESLPT was a somewhat simple task
because the purpose is placement and the construct validation of the test consisted of
an examination of the content of the ESL courses.

c) Specifications for the GET are the skills of writing grammatically and rhetorically
acceptable prose on a topic of some interest, with clearly produced organization of
ideas and logical development. The GET is a direct test of writing ability in which test-
takers must, in a two hour time period, write an essay on a given topic.
03 Design, select, and arrange test tasks/
items

The specs act much like a blueprint in determining the number and types of items to
be created.

a) TOEFL test design specifies that each item be coded for content and statistical
characteristics. Content coding ensures that each examinee will receive test questions
that assess a variety of skills (reading, comprehending the main idea, or
understanding inferences) and cover a variety of subject matter without unduly biasing
the content toward a subset of test-takers.
Consider the following sample of a reading selection and ten items based on it, from a
practice TOEFL (Philips, 2001, pp. 4233-424)

Items target the assessment of comprehension of the main idea (item 11), stated
detail ( 17,19), unstated details (12,15,18), implied details (14,20) and vocabulary in
context (13,16)
b) The selection of items in the ESLPT entailed two entirely different processes. In the
two subsections of the test that elicit writing performance (summary of reading;
response to reading)

c) The GET prompts are designed by a faculty committee of examiners who are
specialist in the field of university academic writing. The assumption is made that the
topics are universally appealing and the intended product of an essay that requires an
organized logical argument and conclusion.
04 Make appropriate evaluations of different
kinds of items
Performing them may not be practical, especially if the classroom-based test is a one-
time test. But for a standardized multiple-choice test that is designed to be marketed
commercially, or administered a number of times, and administered in different form,
these indices are a must.

Item facility (IF) - % of people who give the right answer

Item Discrimination (IDis) – indicates the extent to which success on an item
corresponds to success on the whole test
Distractor Analysis – finding out the % of people who get the item right in the try-out
group
04 Make appropriate evaluations of different
kinds of items
There are different form of evaluation for other types of response format.

Practically
clarity of directions, timing of the test, ease of administration, time required to score
responses

Reliability
is the degree to which an assessment tool produces stable and consistent results.

Facility
unclear directions, complex language, obscure topics, fuzzy data, culturally biased
information.
05 Specify scoring procedures and reporting
formats

A systematic assembly of test items in pre-selected arrangements

and sequences, all of which are validated to conform to an expected
difficulty level, should yield a test that can be scored accurately and
reported back to test-takers and institutions efficiently.
06 Perform ongoing construct validation
studies
No standardized instrument is expected to be used repeatedly without a rigorous
program of in going construct validation. Any standardized test, once developed, must
be accompanied by systematic corroboration of its effectiveness and steps towards its
improvement.

The process of ongoing validation will no doubt continue as new forms of the editing
section are created and as new prompts and reading passages are created for the
writing section. Such a validation process should also include consistent checks on
placement accuracy and on face validity.
 04.
Standardized Language
Proficiency Testing
04. Standardized Language Proficiency Testing

Tests of language proficiency presoppose a comprehensive definition of

the specific competencies that comprise the overall language ability.
The specifications for the TOEFL provided an illustration of an operational
definition of ability for assessment purposes. This is not the only way to
conceptualize the concept.

Swain (1990) offered a multidimensional view of proficiency assessment

by referring to three linguistic traits :
“Grammar, Discourse and Sociolinguistic”
that can be assessed by means of oral, multiple choice and written
responses.
 Another definition and conceptualization of proficiency is suggested by the ACTFL
association. ACTFL takes a holistic and more unitary view of proficiency in
describing four levels :

1.Superior
2. Advance
3. Intermediate
4. Novice.

Within each level, descriptions of Listening, Speaking, Reading, and Writing are
provided as guidelines for assessment.
 The ACTFL Guidelines describe the superior level of speaking as follows : ACTFL
Speaking guidelines, summary, superior level.Superior level speakers are
characterized by the ability to :
- Participate fully and effectively in conversations in Formal and Informal settings
on topics related to practical needs and areas of professional and or scholarly
interest.
- Provide a structured argument to explaun and defend opinions and develop
effective hypotheses within extended discourse.
- Discuss topics concretely and abstractly.
- Deal with a linguistically unfamiliar situation.
- Maintain a high degree of linguistic accuracy.
- Satisfy the linguistic demands of professional and or scholarly life.
 05.
FOUR STANDARDIZED
LANGUAGE PROFICIENCY TESTS
 Three standardized oral production tests :
1. TSE ( Test of Spoken English)
2. OPI ( The Oral Proficiency Inventory)
3. TWE ( Test of Written English)

Four commercially produced standardized tests of English Language Proficiency

are described briefly in this section :
1. TOEFL (Test of English as a Foreign Language)
2. MELAB (Michigan English Language Assessment Battery)
3. IELTS (International English Language Testing System)
4. TOEIC (Test of English for International Communication)
 When you turn to that appendix, use the following questions to help you evaluate
these four tests and their subsections :
1. What item types are included ?
2. How practical and reliable does each subsection of each test appear to be ?
3. Do the item types and tasks appropriately represent a conceptualization of
language proficiency ( ability ) ? That is , can you evaluate their construct validity ?
4. Do the tasks achieve face validity ?
5. Are the tasks authentic ?
6. Is there some washback potential in the tasks
THANK
YOU
Do you have any questions ?