Psychometric Reliability →

Reliability Coefficient
SOURCES OF ERROR VARIANCE
Item Sampling or Content Sampling
The Concept of Reliability
➢
→
Error content sampled
the way the content is sampled
𝑿=𝑻+𝑬
Where
→

Variance (𝜎 2 ) Test Administration

𝝈𝟐 = 𝝈𝟐𝒕𝒉 + 𝝈𝟐𝒆 → Testtaker Variables

Where 𝝈 𝟐

𝝈𝟐𝒕𝒉 𝝈𝟐𝒆

Reliability → Examiner-Related Variables

→  

Test Scoring and Interpretation
→

→ Scorer (or Rater)

Measurement Error

Sampling Error
Random Error

→
Methodological Error

Systematic Error

→
 Reliability Estimates Parallel Forms

TEST-RETEST RELIABILITY ESTIMATES

Test-Retest Method or Time Sampling →

→
Test-Retest Reliability
→ Parallel Forms Reliability

→
Alternate Forms

→
→

→ Alternate Forms Reliability

Coefficient of Stability

➢
➢
➢

→ Carryover Effects ➢

→ Practice Effects ➢

INTERNAL CONSISTENCY ESTIMATE OF RELIABILITY

Internal Consistency Estimate of Reliability or Estimate of

PARALLEL-FORMS & ALTERNATE-FORMS RELIABILITY
Inter-Item Consistency
ESTIMATES

Coefficient of Equivalence

Inter-Item Consistency

→
➢
equivalent forms reliability
➢ 𝟐𝒓
𝒄𝒐𝒓𝒓𝒆𝒄𝒕𝒆𝒅 𝒓 =
𝟏+𝒓

→ Homogeneity Where 𝒄𝒐𝒓𝒓𝒆𝒄𝒕𝒆𝒅 𝒓 =

𝒓=

→
𝒓 =. 𝟕𝟖
𝟐(. 𝟕𝟖) 𝟏. 𝟓𝟔
→ Heterogeneity 𝒄𝒐𝒓𝒓𝒆𝒄𝒕𝒆𝒅 𝒓 = = =. 𝟖𝟕𝟔
𝟏+. 𝟕𝟖 𝟏. 𝟕𝟖
➢
→

➢ →
→
𝜶
Cronbach’s Coefficient Alpha (𝜶)

Split-Half Method
→

→
→ →
→

→ →
→

Odd-Even System
→

𝟐 [𝝈𝟐𝒙 − (𝝈𝟐𝒚𝟏 𝝈𝟐𝒚𝟐 )]

𝜶=
𝝈𝟐𝒙
Where 𝜶 =
𝝈𝟐𝒙 =
𝝈𝟐𝒚𝟏 𝝈𝟐𝒚𝟐 =

➢  
Kuder–Richardson Formula 20 (KR-20)
➢

Spearman-Brown Formula

➢
➢ ➢

𝒌 ∑ 𝒑𝒒
𝒓𝑲𝑹𝟐𝟎 = ( ) (𝟏 − ) ➢
𝒌−𝟏 𝝈𝟐
Where 𝒓𝑲𝑹𝟐𝟎 ;𝒌
𝟐
;𝝈 ;
➢
𝒑 ;𝒒
∑ 𝒑𝒒
𝑝𝑞

➢
𝟏
(𝒓𝟏𝟏 + 𝒓𝟐𝟐 ) − 𝒓𝟏𝟐
𝒓=𝟐
𝟏 − 𝒓𝟏𝟐
Coefficient Alpha Where 𝒓𝟏𝟏 = 𝒓𝟐𝟐 =
𝒓𝟐𝟐 =

𝒌 ∑ 𝝈𝟐𝒊
𝒓𝜶 = ( ) (𝟏 − )
𝒌−𝟏 𝝈𝟐
𝟏
(. 𝟗𝟎+. 𝟕𝟎) − . 𝟕𝟎 . 𝟏𝟎
Where 𝒓𝜶 𝒌 𝒓=𝟐 = = . 𝟑𝟑
𝟏− . 𝟕𝟎 . 𝟑𝟎
𝝈𝟐𝒊 ∑ 𝝈𝟐𝒊
𝟐
𝝈
MEASURES OF INTER-SCORER RELIABILITY

➢ Inter-Scorer Reliability

→
Coefficient of Inter-Scorer Reliability

➢ 
➢
➢
→

➢ Factor Analysis Using & Interpreting a Coefficient

of Reliability
→
THE PURPOSE OF THE RELIABILITY COEFFICIENT
➢
→
Type
Average Proportional Distance (APD) Uses
Number of Testing Sessions
Source of Error Variance
Type Static Characteristic

Uses →

Number of Testing Sessions

Source of Error Variance The range of test scores is or is not restricted.

Type Restriction of Range/Variance

Uses

Number of Testing Sessions Inflation of Range/Variance

Source of Error Variance

Type
The test is a speed or a power test.
Uses
Power Test

Number of Testing Sessions

Speed Test
Source of Error Variance

THE NATURE OF THE TEST

➢ →

The test items are homogeneous or heterogeneous in →

nature.
Homogenous Test Items
The test is or is not criterion-referenced.
→ Criterion-Referenced Test

Heterogenous Test Items →

The characteristic, ability, or trait being measured is

→
presumed to be dynamic or static.
Dynamic Characteristic
→

→
 THE TRUE SCORE MODEL OF MEASUREMENT AND →
ALTERNATIVES TO IT 𝝈𝒎𝒆𝒂𝒔
→
Classical Test Theory (CTT)

→ true score model of

measurement –

→
𝑿 = 𝑻 + 𝑬 𝑜𝑟 𝑿 − 𝑻 = 𝑬
Where 𝑿= 𝑻= 𝑬=

→ →

→
→
Domain Sampling Model

→
→

→
 →

STANDARD ERROR OF MEASUREMENT

Standard Error of Measurement (SEM or 𝝈𝒎𝒆𝒂𝒔 )

→

Item Response Theory

→ standard error of a score
→ 

→ latent-trait theory
→

→ →

𝝈𝒎𝒆𝒂𝒔 = 𝝈√𝟏 − 𝒓𝒙𝒙

Where 𝝈𝒎𝒆𝒂𝒔 = 𝝈=
→ 𝒓𝒙𝒙 =

Given that
→

→
𝝈𝒎𝒆𝒂𝒔 = 𝟏𝟎√𝟏−. 𝟖𝟒 = 𝟒

→
→

→ Difficulty

→ Discrimination

→
Reliability & Individual Scores
 →
𝝈𝒅𝒊𝒇𝒇 = √𝝈𝟐𝒎𝒆𝒂𝒔𝟏 + 𝝈𝟐𝒎𝒆𝒂𝒔𝟐

Where 𝝈𝒅𝒊𝒇𝒇 =
→ 𝝈𝟐𝒎𝒆𝒂𝒔𝟏 =
𝝈𝟐𝒎𝒆𝒂𝒔𝟐 =

→ confidence →
interval

𝝈𝒅𝒊𝒇𝒇 = 𝝈√𝟐 − 𝒓𝟏 − 𝒓𝟐
𝝈𝒅𝒊𝒇𝒇 =
75 ± 1.96𝜎𝑚𝑒𝑎𝑠 = 75 ± 1.96(2.37) = 75 ± 4.645 ≈ 5 𝒓𝟏 =
𝒓𝟐 = 𝝈=
→

STANDARD ERROR OF THE DIFFERENCE BETWEEN TWO

SCORES →
Standard Error of the Difference (SED)

→
→

𝝈𝒅𝒊𝒇𝒇 = 𝟏𝟒√𝟐 − . 𝟗𝟐 − . 𝟗𝟐 = 𝟏𝟒√. 𝟏𝟔 = 𝟓. 𝟔

→
→