Criterion-referenced vs.

norm-referenced tests
To understand what happened, we need to understand the difference
between criterion-referenced tests and norm-referenced tests.

The first thing to understand is that even an assessment expert couldn’t

tell the difference between a criterion-referenced test and a norm-
referenced test just by looking at them. The difference is actually in the
scores—and some tests can provide both criterion-referenced
results and norm-referenced results!

How to interpret criterion-referenced tests

Criterion-referenced tests compare a person’s knowledge or skills

against a predetermined standard, learning goal, performance level, or
other criterion. With criterion-referenced tests, each person’s
performance is compared directly to the standard, without considering
how other students perform on the test. Criterion-referenced tests often
use “cut scores” to place students into categories such as “basic,”
“proficient,” and “advanced.”

Example of criterion-referenced measures

If you’ve ever been to a carnival or amusement park, think about the

signs that read “You must be this tall to ride this ride!” with an arrow
pointing to a specific line on a height chart. The line indicated by the
arrow functions as the criterion; the ride operator compares each
person’s height against it before allowing them to get on the ride.

Note that it doesn’t matter how many other people are in line or how tall
or short they are; whether or not you’re allowed to get on the ride is
determined solely by your height. Even if you’re the tallest person in line,
if the top of your head doesn’t reach the line on the height chart, you
can’t ride.

Criterion-referenced assessments work similarly: An individual’s score,

and how that score is categorized, is not affected by the performance of
other students.

In the charts below, you can see the student’s score and performance
category (“below proficient”) do not change, regardless of whether they
are a top-performing student, in the middle, or a low-performing student.
This means knowing a student’s score for a criterion-referenced test will
only tell you how that specific student performed in relation to the
criterion, but not whether they performed below-average, above-
average, or average when compared to their peers.

How to interpret norm-referenced tests

Norm-referenced measures compare a person’s knowledge or skills to

the knowledge or skills of the norm group. The composition of the norm
group depends on the assessment.

For student assessments, the norm group is often a nationally

representative sample of several thousand students in the same grade
(and sometimes, at the same point in the school year). Norm groups may
also be further narrowed by age, English Language Learner (ELL) status,
socioeconomic level, race/ethnicity, or many other characteristics.

Example of norm-referenced measures

One norm-referenced measure that many families are familiar with is the
baby weight growth charts in the pediatrician’s office, which show
which percentile a child’s weight falls in:

 A child in the 50th percentile has an average weight

 A child in the 75th percentile weighs more than 75% of the babies in the norm group and the
same as or less than the heaviest 25% of babies in the norm group
 A child in the 25th percentile weighs more than 25% of the babies in the norm group and the
same as or less than 75% of them.
It’s important to note that these norm-referenced measures do not say
whether a baby’s birth weight is “healthy” or “unhealthy,” only how it
compares with the norm group.

For example, a baby who weighed 2,600 grams at birth would be in the
7th percentile, weighing the same as or less than 93% of the babies in
the norm group. However, despite the very low percentile, 2,600 grams
is classified as a normal or healthy weight for babies born in the United
States—a birth weight of 2,500 grams is the cut-off, or criterion, for a
child to be considered low weight or at risk. (For the curious, 2,600
grams is about 5 pounds and 12 ounces.)

Thus, knowing a baby’s percentile rank for weight can tell you how they
compare with their peers, but not if the baby’s weight is “healthy” or
“unhealthy.”

Norm-referenced assessments work similarly: An individual student’s

percentile rank describes their performance in comparison to the
performance of students in the norm group, but does not indicate
whether or not they met or exceed a specific standard or criterion.

In the charts below, you can see that, while the student’s score doesn’t
change, their percentile rank does change depending on how well the
students in the norm group performed. When the individual is a top-
performing student, they have a high percentile rank; when they are a
low-performing student, they have a low percentile rank.

What we can’t tell from these charts is whether or not the student
should be categorized as proficient or below proficient.
This means knowing a student’s percentile rank on a norm-referenced
test will tell you how well that specific student performed compared to
the performance of the norm group, but will not tell you whether the
student met, exceeded, or fell short of proficiency or any other criterion.

The difference between norm-referenced scores and criterion-

referenced scores
Some assessments provide both criterion-referenced and norm-
referenced results, which can often be a source of confusion.

For example, you might have a student who has a high percentile rank,
but doesn’t meet the criterion for proficiency. Is that student doing well,
because they are outperforming their peers, or are they doing poorly,
because they haven’t achieved proficiency?

The opposite is also possible. A student could have a very low percentile
rank, but still meet the criterion for proficiency. Is this student doing
poorly, because they aren’t performing as well as their peers, or are they
doing well, because they’ve achieved proficiency?
However, these are fairly extreme and rather unlikely cases. Perhaps
more common is a “typical” or “average” student who does not achieve
proficiency because the majority of students are not achieving
proficiency.

In fact, this is the pattern we see with 2017 National Assessment of Educational
Progress (NAEP) scores, where the “typical” fourth-grade student (50th
percentile) has a score of 226 and the “average” fourth-grade student
(average of all student scores) has a score of 222, but proficiency
requires a score of 238 or higher.

In all of these cases, educators must use …

  Professional judgement
 Knowledge of the student
 Familiarity with standards and expectations
 Understanding of available resources, and
 Subject-area expertise

Norm-referenced assessment refers to an assessment that ranks students on a “bell curve”

to determine the highest and lowest performing students. This method is used to understand how students'
scores compare to a predefined population with similar experience.
Examples of norm-referenced tests include the SAT, IQ tests, and tests that are graded on a curve. Anytime a
test offers a percentile rank, it is a norm-referenced test. If you score at the 80th percentile, that means that
you scored better than 80% of people in your group.

What is the difference between norm-referenced and criterion-referenced?

The difference between norm-referenced and criterion-referenced tests is in the scoring. A norm-referenced
test compares the test-taker's score to a representative group, or norming group, and reports where the
tester falls in relationship to other testers. The criterion-referenced test, on the other hand, compares a
tester's score to an objective standard or criteria.

What is an example of a norm-referenced test?

Norm-referenced tests are standardized tests characterized by scoring that compares the performance of the
test-taker to a norming group (a group with similar characteristics such as age or grade level). Examples of
norm-referenced tests are the SAT and ACT and most IQ tests.

What is the purpose of norm-referenced tests?

The purpose of norm-referenced tests is to rank individuals in relation to others of a similar representative
group. Norm-referenced tests are used for many purposes such as college entrance (the SAT and ACT) and IQ
tests. A norm-referenced test assumes that the majority of test-takers will score in the average range, with a
few people testing above and below average. The test reflects an individual's performance compared to that
of others.

What is a norm-referenced test? A norm-referenced test is a type of standardized test (that is, a test that is
identical for every test-taker). After the items on a norm-referenced test are scored, the scores are compared
to those of a comparison group, or norming group. Because the test-taker is compared to other people, the
results can be considered subjective.
 A norm-referenced test compares the performance of the tester to that of
other testers with similar characteristics or demographics.

To develop a norm-referenced test, the test developers select a statistically relevant group of individuals and
administer the test items. The scores of this norming group are used to create the scoring system for that test.
The composition of the norming group depends on the test, but factors considered usually include age or
grade level, and may also be narrowed down by other demographic information. In addition, some tests are
normed for more than one group. If so, a test administrator gives the test, then chooses the correct scoring
system based on the subject's qualifications (i.e., the administrator uses a different scoring chart for a seven-
year-old than for a twelve-year-old).

The norm-referenced test definition indicates that the results are reported as a percentage or a percentile
ranking. The purpose of this number is to tell the test-taker what percentage of the norming group scored
above and below them. Many test developers use a bell curve to organize their data. This means that they
expect the largest percentage of test-takers to score in the middle range of the test, with smaller numbers of
test-takers performing below average and above average.

The following table provides a summary of the key features of criterion and norm referenced assessment approaches: 2,13

Criterion-referenced assessments Norm-referenced assessment

 specify criteria or standards (eg. essential elements
of a task),
 judgements about performance can be made against
set, pre-specified criteria and standards,
 focus is on mastery with the achievement of a
criterion representing a minimum, optimum or
essential standard,
 recorded via rating scale or set of scoring rubrics,
and
 examples include clinical skill competency tools.
 do not utilise criteria,
 assessment is competitive,
 involves making judgements about an individual's
achievement by ranking and comparing their performance
with others on the same assessment, and
 examples include examinations.
Criterion-referenced assessment means that teacher judgements about how a student
does in an assessment task are based on standards and criteria that are pre-determined
and made available to students at the time the assignment is set.

Standards are a specified and definite level of achievement that may be attained.
Criteria means the characteristics by which the quality of something may be judged.

Criterion-referenced assessment improves transparency and consistency for students and

supports the following University principles of assessment:
 Assessment design is coherent and supports learning progression within courses
and across programmes
 Assessment tasks are demonstrably aligned with course-level learning outcomes,
and programme and University-level Graduate Profiles
 Assessment is reliable and valid, and is carried out in a manner that is inclusive and
equitable
 Assessment practices are consistent and transparent, and assessment details are
available to students in a timely manner
Criterion-referenced assessment is made clear to students by the use of carefully
designed rubrics.
Norm-Referenced vs. Criterion-Referenced Testing
PSYCHOMETRICS

The two terms Norm-Referenced and Criterion-Referenced are commonly used to describe
tests, exams, and assessments. They are often some of the first concepts learned
when studying assessment and psychometrics.
Norm-referenced means that we are referencing how your score compares to other
people. Criterion-referenced means that we are referencing how your score
compares to a criterion such as a cutscore or a body of knowledge.

Do we say a test is “Norm-Referenced” vs. “Criterion-

Referenced”?

Actually, that’s a slight misuse.

The terms Norm-Referenced and Criterion-

Referenced refer to score interpretations. Most
tests can actually be interpreted in both ways,
though they are usually designed and
validated for only one of the other.

Hence the shorthand usage of saying “this is a

norm-referenced test” even though it just means that it is the primarily intended
interpretation.

Examples of Norm-Referenced vs. Criterion-Referenced

Suppose you received a score of 90% on a Math exam in school. This could be
interpreted in both ways. If the cutscore was 80%, you clearly passed; that is the
criterion-referenced interpretation. If the average score was 75%, then you
performed at the top of the class; this is the norm-referenced interpretation. Same
test, both interpretations are possible. And in this case, valid interpretations.

What if the average score was 95%? Well, that changes your norm-referenced
interpretation (you are now below average) but the criterion-referenced
interpretation does not change.

Now consider a certification exam. This is an example of a test that is specifically

designed to be criterion-referenced. It is supposed to measure that you have the
knowledge and skills to practice in your profession. It doesn’t matter whether all
candidates pass or only a few candidates pass; the cutscore is the cutscore.

However, you could interpret your score by looking at your percentile rank compared
to other examinees; it just doesn’t impact the cutscore

On the other hand, we have an IQ test. There is no criterion-referenced cutscore of

whether you are “smart” or “passed.” Instead, the scores are located on the standard
normal curve (mean=100, SD=15), and all interpretations are norm-referenced.
Namely, where do you stand compared to others? The scales of the T score and z-
score are norm-referenced, as are Percentiles. So are many tests in the world, like
the SAT with a mean of 500 and SD of 100.

Is this impacted by item response theory (IRT)?

If you have looked at item response theory (IRT), you know that it scores examinees
on what is effectively the standard normal curve (though this is shifted if Rasch). But,
IRT-scored exams can still be criterion-referenced. It can still be designed to measure
a specific body of knowledge and have a cutscore that is fixed and stable over time.
Even computerized adaptive testing can be used like this. An example is the NCLEX
exam for nurses in the United States. It is an adaptive test, but the cutscore is -0.18
(NCLEX-PN on Rasch scale) and it is most definitely criterion-referenced.
Building and validating an exam
The process of developing a high-quality assessment is surprisingly difficult and time-
consuming. The greater the stakes, volume, and incentives for stakeholders, the more
effort that goes into developing and validating. ASC’s expert consultants can help you
navigate these rough waters.