Validity

Arman Argynbayev
MA in TEFL
Suleyman Demirel University
arman.argynbayev@sdu.edu.kz
 Validity
“a test is said to be valid if it measures accurately what it is
intended to measure” (Hughes, 2003, p. 26)

“the extent to which inferences made from assessment

results are appropriate, meaningful, and useful in terms of
the purpose of the assessment” (Gronlund, 1988, p. 226)
 Content Validity
- The content matches the instructional objectives;
- Performance of the appropriate behaviour;
- Representative samples of language skills;
- Proper samples of relevant structures;
- Refers to quality of testing;
- Measures knowledge;
 Content Validity
Examples:
- The final exam which only includes content covered during the last
six weeks.
- The test measures only 2 topics out of 15 in the midterm exam.
- The candidates at the beginner level must paraphrase the
sentences.
- General knowledge questions are asked in the reading
comprehension passage.
- Assigning ‘Summary and response’ essay for the writing task.
 Content Validity (keys)
Examples:
- The final exam which only includes content covered during the last six
weeks. (low validity)
- The test measures only 2 topics out of 15 in the midterm exam. (very low
content validity)
- The candidates at the beginner level must paraphrase the sentences. (low
content validity)
- General knowledge questions are asked in the reading comprehension
passage. (low content validity)
- Assigning ‘Summary and response’ essay for the writing task. (high content
validity if students learned it in class)
 Criterion-related validity
There are two kinds of criterion-related validity:
1. Concurrent Validity
2. Predictive Validity
 Criterion-related validity
Concurrent validity
- correlates with other tests;
- measures the test against benchmark tests.

Predictive validity:
- predicts candidates’ future performance, skills.
 Concurrent validity
Examples:
- The students score similarly in the placement test designed by
you and online Oxford level test.
- The results of the students in the progress test, designed by
you, are similar to the results of the previous progress test
designed by your colleague or by state experts.
 Concurrent validity (keys)
Examples:
- The students score similarly in the placement test designed by
you and online Oxford level test. (high concurrent validity)
- The results of the students in the progress test, designed by
you, are similar to the results of the previous progress test
designed by your colleague or by state experts. (high
concurrent validity)
 Predictive validity
Examples:
- The test designed by you showed that Student X cannot study
at a university with English as a medium of instruction. The
student applies for this university but is not accepted.
- Student X always scores higher than 90% in your English tests
at school. However, he hardly gets 50% in English tests at
university.
 Predictive validity (keys)
Examples:
- The test designed by you showed that Student X cannot study
at a university with English as a medium of instruction. The
student applies for this university but is not accepted. (high
predictive validity)
- Student X always scores higher than 90% in your English tests
at school. However, he hardly gets 50% in English tests at
university. (low predictive validity)
 Construct validity
- measures the ability which it claims to measure.

Does a language assessment accurately measure

language ability?
 Construct validity
Examples:
- Odd one out in the pronunciation test
- Fill in the gaps in the writing test
- Asking questions in the reading part without paraphrasing
them from the text
- Administering a listening test in a classroom with terrible
acoustics
 Construct validity (keys)
Examples:
- Odd one out in the pronunciation test (low construct validity)
- Fill in the gaps in the writing test (low construct validity)
- Asking questions in the reading part without paraphrasing
them from the text (low construct validity)
- Administering a listening test in a classroom with terrible
acoustics (low construct validity)
 Face validity
- Refers to how good people think the test is
- A test should look valid
- Face validity is subjective
 Face validity
Examples:
- A proficiency test might lack face validity, if it doesn’t have
speaking and listening part.
- Some people believe that IELTS is neither valid nor reliable.
 The use of validity
• Ensure content validity;
• Validate high-stake exams empirically (practically)
against some criterion;
• Use direct testing whenever possible;
• Tests with no information on its validity should be
treated with caution.
 Activity
Assess ЕНТ (KZ), TOEFL (US), FCE (UK) and one achievement
test on any course that you have taken so far in terms of
various kinds of validity that have been presented in this
slide-show. What evidence is there that the test is valid /
invalid?
 References
• Hughes, A. (2003). Testing for language teachers. Ernst Klett
Sprachen.
• Douglas, B. H. (2004). Language assessment principles and
classroom practice. NY: Pearson Education.