Professional Documents
Culture Documents
Issues in Testing Practices Test Bias Test Fairness Test Accomodations VNB
Issues in Testing Practices Test Bias Test Fairness Test Accomodations VNB
Issues in Testing Practices Test Bias Test Fairness Test Accomodations VNB
2
TEST BIAS
• Explain concept of test bias
• Discuss several categories of test bias
• Discuss several types of bias TEST ACCOMMODATIONS
• Explain the effects of bias
• Identify the bias • Explain the concept of test accommodations
• Explain types of cheating (Prior to test, During the test) • State the types of student's disabilities eligible for
• Discuss stealing test questions (piracy) test accommodations
• Discuss categories of test accommodations
• List out the examples of test accommodations (in
wide areas)
TEST FAIRNESS • Explain helpful and harmful aspects of test
• Explain the concepts of test fairness accommodations
• Discuss the fairness framework
5
Bias is defined as the presence of some characteristic
of an item that results in differential performance for
two individuals of the same ability but from different
ethnic, sex, cultural or religious groups.
Jorgensen (2005) 6
7
• it's refer to the degree to which a test fails to capture important aspects of
the construct.
• Too few test questions results in failure to insufficient sample the learning
content in the achievement domain desired. Consequently, the reliability of
the examination suffers.
Example:
• In the case mathematics test, it might over reliance on
reading comprehension skills that english language learner
may be lacking.
9
Refers to whether a test
exactly measures what it
was designed to measure.
For Example:
On an intelligence test, for
example, students who are
learning English will likely
encounter words they 10
It occurs when the content of a test is comparatively more difficult for one
group of students than for others.
It can occur when members of a student from a subgroup, such as various
minority groups, when:
they have not been given the same opportunity to learn the material being tested
scoring is unfair to a group
questions are worded in ways that are unfamiliar to certain students because of
linguistic or cultural differences.
11
Gender Bias
Cultural Bias
Regional Bias
Ethnic/Racial Bias
Language Bias
Socio Economic Status (SES) Bias
Special Needs Bias
12
Gender bias exists
• prejudice for or against one gender, especially in
a way considered unfair.
• it's occurs because of personal values,
perceptions and outdated, traditional views
about men adn women.
13
1 . Axman (1990)
• A test is culturally biased when it makes
assumptions about what is “common
knowledge.1”
• Examples: cultural norms and conventions,
literary knowledge, conventions of
language2,etc.
1. McGinley, S. (2002)
2. Del Rosario B. (1998)
14
nA test has regional bias when
people are penalized because of the
use of regional names or dialects.
n Examples: sundrop, pop, bubbler,
pies for pizzas
(pizza pie) in NY, etc.
15
Ethnic Bias or Racial Bias occurs
- when members of a group are portrayed in a
stereotypical manner.
- when members of such groups obtain different scores
than members of another group–with the same skills and
knowledge1.
Popham (2006)
16
n Socio-economic bias occurs in tests when students
are penalized based on their socio-economic status
(SES)1:
¨ The isolated, rural environment, the restrictive poverty of
many families, and cultural ties with a tribe or other
group deny many students
important knowledge of the
outside world2.
1. Popham (2006)
2. Brescia & Fortune (1988)
17
n Language bias in tests occurs when
second language learners (L2) are
penalized because of their lack of
knowledge of the English Language.
Popham (2006)
19
• Emotional effects
• Students struggle with testing and its results, and they may
question their own capabilities to learn.
• They lose confidence and the ability to take risks and become
embarrassed and feel the need to “save face”.
21
• Content Bias
-fair for all examinees
• Language Bias
-Is the item free of group specific language, vocabulary, or
reference pronouns
• Item Structure and Format Bias
-Are there any flaws in the items to which members of DSI
(designated subgroup of interest) are differentially sensitive?
• Stereotyping of Minorities
-Avoid material that is controversial, inflammatory, demeaning
or offensive to members of DSI.
22
• Is the test item fair for all examinees and
free of annoying stereotypes?
• Are both sex groups recognized fairly?
• Will all examinees have equal opportunity
to respond?
23
24
q Purchasing test items, item pools, or tests from web sites.
q Using a chat room.
q Bulletin board.
q Discussion forum to solicit.
q Collaborating with individuals who take the test during an
earlier administration and then provide test items to individuals
who take the test at a later time.
25
Strategies using various forms of soft technology to interact with others
who may also be taking the test or who are outside the testing environment.
26
§ Item resellers to help other examinees to
get high scores.
§ Buyer’s (i.e., the examinee) only objective
is to memorize the specific content and
response options for a set of items.
§ Put the items on the Internet for resale to
future test takers.
§ Tell their friends or classmates about the
test.
27
1. What are the categories of test bias?
28
29
This presentation will....
• explain the concepts of test fairness
• discuss the fairness framework
30
CONCEPT OF TEST FAIRNESS
• Test Fairness of a test relates to fair use.
• The analytical methods that can help bring about fair tests and testing practice
have not been clearly articulated.
• Willingham and Cole (1997) and Willingham (1999) view test fairness as an
important aspect of validity, arguing that fairness should be viewed as
“comparability in assessment; more specifically, comparable validity for all
individuals and groups
• Fairness in testing is defined many ways (AERA, APA, & NCME, 1999) but at its
core is the idea of comparable validity. A fair test is one that yields comparably
valid inferences from person to person and group to group.
CONCEPT OF TEST FAIRNESS (cont…)
“[A testing] principle that every test taker should be assessed in an equitable way.”
(AERA, et al., 1999, p. 175)
THE CODE OF FAIR TESTING PRACTICES IN
EDUCATION (1988)
• The Code of Fair Testing Practices in Education (1988) presents standards for
educational:
1) test developers (thinker, researchers, raters, writers)
Test developers are trained to examine all aspects of a test for its fairness
2) test users (population of peoples)
Test users are peoples and agencies that select test for its fairness
• The Code (1988) presents standards for educational test developers and users
in four areas:
• developing and selecting tests
• interpreting scores
• striving for fairness and
https://www.apa.org/science/programs/testing/fair-testing.pdf
• informing test takers
THE FAIRNESS FRAMEWORK (Kunnan, 2004)
1) VALIDITY
2) ABSENCE OF BIAS
3) ACCESS
4) ADMINISTRATION
5) SOCIAL CONSEQUENCES
THE FAIRNESS FRAMEWORK (Kunnan, 2004)
1) VALIDITY:
Content representativeness • (described as content validity) refers to the adequacy with which the
or coverage evidence: test items, tasks, topics and language dialect represent the test domain.
• refers to content that may cause unfair penalisation because of a test taker’s group membership (such as
Unfair penalisation that based on gender, race and ethnicity, religion, age, native language, national origin).
based on test taker’s
background:
• differing performances and resulting outcomes by test takers from different group memberships. Such
group differences are defined by salient test-taker characteristics such as gender, race and ethnicity,
Disparate impact
and standard religion, age, native language, national origin
setting:
THE FAIRNESS FRAMEWORK (Kunnan, 2004)
3) ACCESS
Educational access Geographical access
refers to whether or not a test is accessible to refers to whether a test site is accessible
test takers in terms of opportunity to learn the in terms of distance to test takers.
content and to become familiar with the types of
task and cognitive demands.
Physical conditions
• Refers to appropriate conditions for test administration, such as optimum light and
temperature levels and facilities considered relevant for administering tests.
Uniformity or consistency
• refers to uniformity in test administration exactly as required so that there is
uniformity and consistency across test sites and in equivalent forms, and that test
manuals or instructions specify such requirements.
THE FAIRNESS FRAMEWORK (Kunnan, 2004)
5) SOCIAL CONSEQUENCES
Wash back:
• refers to the effect of a test on instructional practices, such as teaching,
materials, learning, test-taking strategies, etc.
Remedies:
• refers to remedies offered to test takers to reverse the detrimental
consequences of a test, such as re-scoring and re-evaluation of test responses,
and legal remedies for high-stakes tests.
Recap....
• What is test fairness?
• List out 5 components of test fairness framework by
Kunnan (2004)
TEST
ACCOMMODATIONS
42
This presentation will....
üexplain the concept of test accommodations
üstate the types of student's disabilities eligible for test
accommodations
üdiscuss categories of test accommodations
ülist out the examples of test accommodations (in wide areas)
üexplain helpful and harmful aspects of test accommodations
43
Concepts of Test Accomodations
44
Concepts (continued…)
§ The accommodations may apply to both
standardized tests administered to large
populations of students, including high-stakes tests
used to make important decisions about students,
and to the assessments that teachers use to
evaluate what students have learned in a particular
course.
§ The general goal of providing testing
accommodations is to create a level playing field for
students whose disabilities or language abilities that
may adversely affect their ability to show on a test
what they have learned. 45
Types of Students Disabilities Eligible for
Test Accommodations
§ Blindness/Vision Impairment
§ AD/HD (formerly known as A.D.D./A.D.H.D.)
§ Learning Disabilities
§ Deafness/Hearing Impairment
§ Certain Medical Conditions
§ Certain Physical Disabilities
§ Certain Psychiatric Conditions
46
Categories of Testing Accommodations
Presentation Response
Time Location
47
Presentation Accomodations
48
Response Accomodations
49
Time Accomodations
50
Location Accomodations
51
Examples of Testing Accommodations
(in wide areas)
§Braille or large-print exam booklets
§Screen reading technology
§Scribes to transfer answers to Scantron bubble sheets
or record dictated notes and essays
§Extended time
§Wheelchair-accessible testing stations
§Distraction-free rooms
52
Helpful Aspects of Test Accommodations
§ When carefully applied, accommodations enable teachers to get a
more accurate assessment of student's knowledge and skills.
§ Accommodations can help students feel more comfortable with the
testing process, knowing that his/her disability will not prevent
him/her from succeeding.
§ Accommodations can potentially help students receive better grades
and possibly more academic recognition.
§ Accommodations may result in slightly higher scores on college
entrance exams
53
Harmful Aspects of Test Accommodations
§ Testing accommodations may be misused and misapplied.
§ Students may not get extra help they need because they appears to
have higher skill levels than they really have.
§ In the worst cases, teachers may inadvertently focus less on students,
believing that the accommodations are all the support they needs.
§ When accommodations are over-used, students with disabilities have
inflated scores.
54
Recap....
1) List out 4 categories of testing accommodations.
56
MedSI Test
57
STPM 2018
58
UPSR 2014
59
PT3
60
PBS
61
Recap
• Form into a group of 5 members. Each group can think of
one example of test item that has one or more type(s) of
bias. Put the examples on padlet.
Link :
https://padlet.com/n_bukhar/SGDE4013_A181_TestBiasFai
rnessAccommodations
TEST BIAS
• Explain concept of test bias
• Discuss several categories of test bias
• Discuss several types of bias TEST ACCOMMODATIONS
• Explain the effects of bias
• Identify the bias • Explain the concept of test accommodations
• Explain types of cheating (Prior to test, During the test) • State the types of student's disabilities eligible for
• Discuss stealing test questions (piracy) test accommodations
• Discuss categories of test accommodations
• List out the examples of test accommodations (in
wide areas)
TEST FAIRNESS • Explain helpful and harmful aspects of test
• Explain the concepts of test fairness accommodations
• Discuss the fairness framework