Review

For reprint orders, please contact reprints@expert-reviews.com

Rating scales and

Rasch measurement
Expert Rev. Pharmacoeconomics Outcomes Res. 11(5), 571–585 (2011)

David Andrich
Graduate School of Education,
The University of Western Australia,
M428, 35 Stirling Highway, Crawley,
Western Australia, 6009, Australia
Tel.: +61 864 881 085
Fax: +61 864 881 052
david.andrich@uwa.edu.au
Assessments with ratings in ordered categories have become ubiquitous in health, biological
and social sciences. Ratings are used when a measuring instrument of the kind found in the
natural sciences is not available to assess some property in terms of degree – for example, greater
or smaller, better or worse, or stronger or weaker. The handling of ratings has ranged from the
very elementary to the highly sophisticated. In an elementary form, and assumed in classical
test theory, the ratings are scored with successive integers and treated as measurements; in a
sophisticated form, and used in modern test theory, the ratings are characterized by probabilistic
response models with parameters for persons and the rating categories. Within modern test
theory, two paradigms, similar in many details but incompatible on crucial points, have emerged.
For the purposes of this article, these are termed the statistical modeling and experimental
measurement paradigms. Rather than reviewing a compendium of available methods and models
for analyzing ratings in detail, the article focuses on the incompatible differences between these
two paradigms, with implications for choice of model and inferences. It shows that the differences
have implications for different roles for substantive researchers and psychometricians in designing
instruments with rating scales. To illustrate these differences, an example is provided.

Keywords : graded response model • item response theory • ordered category formats • Rasch measurement
• Rasch models • rating scales

Introduction complement of many assumptions behind

This article is not an examination of a compen-
dium of technical issues for analyzing rating
scales using Rasch measurement  [1,2] . Rather,
it shows first that the Rasch measurement of
rating scales sits within a statistical paradigm
that is different from the standard one within
which item analysis sits, and second, how this
difference leads to a major consequence in
the construction and analysis of rating scales.
To illustrate the points, a concrete example
is provided.
The term paradigm is used explicitly in
the sense of Kuhn  [3] to be a complement
of many reinforcing assumptions taken for
granted in a field. Paradigms govern the ques-
tions researched and how they are answered.
Within the paradigm, substantial, normal,
puzzle­- solving research advancing the field
is carried out. However, periodically, a para-
digm’s assumptions are challenged. When
the assumptions of the original paradigm are
replaced, a scientific revolution occurs. Some
of the assumptions of the different paradigms
are mutually incompatible. An example is the
Copernican revolution, which replaced the
the belief that Earth was the center of the
universe [4] .
The article clarifies the confusion that can
arise in the comparisons of different methods
of analyzing rating scales, in which it may
appear that the differences are primarily tech-
nical. It will be seen that the differences are
much deeper, and that they are at the level
of paradigms. To see this, it is first neces-
sary to examine the principles underpinning
rating scales.
Measurement & rating scales
In the prototype of measurement, an instru-
ment maps some property of an entity that
can be considered greater or smaller, stronger
or weaker, better or worse, and so on, onto a
linear continuum [5] . The linear continuum has
a more or less arbitrary origin and a specified
unit. Although they have been referred to as
constructs and traits, following Reise  [6] , such
properties are referred to here as variables.
In measurement, the linear continuum is
partitioned into equal, contiguous intervals
by thresholds that are all equally fine (same

www.expert-reviews.com 10.1586/ERP.11.59 © 2011 Expert Reviews Ltd ISSN 1473-7167 571

 Review Andrich
discrimination) and, relative to the size of the property being
measured, fine enough that their width can be ignored. Then
the measurement of the variable is the count of the number of
intervals, the units, from the chosen origin to where the property
of the object is mapped on the continuum. A prototype of a con-
tinuum partitioned into equal intervals, the very familiar ruler
partitioned into cm and mm, is shown in Figure 1. In anticipation
of the analogy with a rating scale, the ruler has a rating scale
with five ordered categories superimposed upon it.
Measurement is a sophisticated concept and process.
Constructing an instrument in the natural sciences requires
a theoretical understanding of the variable being measured.
For example, to construct a thermometer, it was necessary to
understand the effects of heat on the expansion properties of
selected materials [7] . After an instrument has been established,
it can be used to relate its measurements to other factors. For
example, in some circumstances, such as the presence of a fever,
the health statuses of individuals can be assessed by measuring
their temperatures.
In the natural sciences, instruments are not expected to func-
tion outside their designated operating ranges. In addition, it is
understood that all measurement contains at least, and ideally
only, random error. Depending on the size of this error, replicated
measurements may be made and then in a well-defined way, their
mean is a more precise estimate of the size of the property. The
aforementioned characteristic of a measuring instrument, a linear
continuum partitioned into units, is not negotiable in the natural
sciences and is independent of any dataset.
Rating scales as instruments in health & social sciences
Rating scales, which seem to have been first formalized by
Likert [8] , are used in the health and social sciences in the assess-
ment of variables when no measuring instrument of the kind
found in the natural sciences is available. Moreover, as they are
used more and more to assess the effects of interventions, whether
these be educational, physical or drug interventions, their impor-
tance is constantly growing [9] . Therefore, it may be argued that
no less care and knowledge should be applied to constructing
rating scales than are used to construct measuring instruments
in the natural sciences.
Figure 2 shows a specific rating scale, the Ashworth Scale, con-
cerned with the clinical assessment of the degrees of muscle tone
that is used illustratively in this article. It immediately shows the
implied partitioning of a continuum [9] .
Ruler in mm
1cm 2 3 4
Five ordered
0 1 2 3 4 categories
Figure 1. A partitioned continuum in the prototype of
measurement with five ordered categories.
572
It may appear redundant to stress that the ordering requires
that successive categories from left (rigid) to right (normal)
reflect successively more muscle tone and less spasticity, and that
whether or not this is the case is central to any subsequent inter­
pretations. (Ashworth’s original scale was labeled 0 = normal tone
to 4 = rigid. For the purposes of this article, the numbering has
been reversed, making a higher score correspond to a better level
of muscle tone). This is an a priori requirement that is independ-
ent of any empirical ratings. If empirical ratings do not meet this
requirement, then it seems that it is a problem with the ratings.
However, we do stress this requirement because how the empiri-
cal ordering is handled is a distinctive difference between the two
paradigms and is the focus of this article. To anticipate this differ-
ence, in one paradigm the empirical ordering is taken for granted
and models are found that fit the ratings; in the other paradigm,
the ordering is treated as an a priori requirement that needs to be
verified empirically. However, although the ordering appears not to
be negotiable, we note in anticipation of the application of Rasch
measurement that, unlike the case of physical science measurement,
the distances between the thresholds do not have to be equal.
Most instruments are composed of more than one rating scale
and the ratings of an individual are combined into a single value.
The distinct rating scales are generally referred to as items. Having
more than one item is a form of replication that is analogous to
making replicated measurements in the natural sciences. However,
unlike some examples in natural science where the same object
may be measured repeatedly with the same instrument, the same
item (same wording) cannot be administered repeatedly to the
same person. Therefore, different items assessing the same vari-
able are used. Having more than one item potentially increases
both the precision and the validity of an assessment. The example
provided in this article has multiple items to assess each person on
the Ashworth scale of muscle tone.
Theories & paradigms in the analysis of ratings
Three theories for the analysis of ratings are available: first, clas-
sical test theory (CTT); second, item response theory (IRT); and
third, Rasch measurement theory (RMT). The histories of these
are summarized respectively by Traub [10] , Bock [11] and Wright [12] .
The latter two theories, IRT and RMT, are often placed together,
and for the purposes of this article will be referred to as modern test
theory. (The term modern test theory is not used in a reference to
an absolute period in the development of test theory, but to contrast
it to classical test theory. The basics of modern test theory were
established by Thurstone in the 1920s [5]). There is no space here
to consider all three theories in relation to paradigms, and there-
fore some elements of CTT will be referred to only where relevant
in reviewing the features of IRT and RMT. Cano et al. provide a
review of all three theories in relation to an area of assessment in
health outcomes [13] .
Two paradigms for the applications of modern test theory have
emerged. For the purpose of this article, the two paradigms are
termed the statistical modeling paradigm and the experimental
measurement paradigm, respectively. The article shows that IRT
sits within the former paradigm and RMT within the latter. The
Expert Rev. Pharmacoeconomics Outcomes Res. 11(5), (2011)
 Rating scales & Rasch measurement Review

next section briefly reviews the common

features of ratings from the perspectives of Limb rigid Increased tone
(minimal (restricting Increased tone
these theories, before returning to examine movement) movement) (easily flexed) Catch Normal tone
their differences.
0 1 2 3 4
Analysis of ratings δ1 δ2 δ3 δ4 Thresholds
Classical test theory  [10] is historically the
oldest theory for the analyses of ratings, but Figure 2. The implied rating scale continuum partitioned by four thresholds into
is currently being augmented, and in many five ordered categories. 
cases superseded, by modern test theory. In
CTT, the elementary scoring of successive ratings by successive category x, which is a function of para­meters aix,bix of category x
integers is employed. Then the sum across items is taken as the of item i and the location bn of person n on the continuum, giving
summary score for a person. This score is taken to be composed Equation 1:
of a true score and an error. The characterization of each person
by a single score implies a single variable, and evidence is used to Pr " Xni = x; bn, aix, bix , = f " x; bn, aix, bix , (1)
confirm that the responses to the items do form replications with
respect to the assessment of the same variable, rather than that dif- Figure 3 shows an example of the probabilities of responses in five
ferent items may assess different variables. This does not preclude successive rating categories, x = 0, 1, 2, 3 and 4, as a function of
the idea that multiple factors may contribute to any person’s loca- the location b on the continuum and of the characteristics of the
tion on a variable. This is analogous to the many factors, including categories of an item. These probabilities are generally referred to
nutrition, genes and age, which contribute to any person’s weight, as category characteristic curves (CCCs) and appear in both IRT
which is a single variable [6] . and RMT.
Rather than assigning successive integers to categories and
treating them as measurements, in modern test theory (both IRT Two paradigms in modern test theory
and RMT), three modifications are made. First, a probabilistic The two paradigms of modern test theory for the analysis of
model for an observed rating is specified as a function of the ratings have many aspects in common and more than one aspect
location of the person and categories on a continuum; second, that is different. The terms statistical modeling and experimental
as noted earlier, categories are not presumed to be the same size; measurement are chosen for the paradigms because their dif-
third, the number of categories is explicitly finite. ferences can be characterized by the following feature: in the
Formally, let Xni be the random variable of the observed response former, the model is chosen to fit the observed ratings; in the
xni ! "0, 1, 2,...mi , of person n to item i where the successive inte- latter, the ratings are analyzed according to a model to check
gers are simply assigned descriptively to the successive categories, whether they meet a priori specifications. The statistical mod-
as shown in Figures 1 & 2, and do not imply scores. Furthermore, let eling paradigm, within which IRT sits, is the presently dominant
Pr " Xni = x , be the probability that person n responds to item i in paradigm [14] . The experimental measurement paradigm, within

1.0

0 4

2
1
Probability

3
0.5

0.0
-5 -4 -3 -2 -1 0 1 2 3 4 5
Location

Figure 3. Category characteristic curves for five ordered categories.

www.expert-reviews.com 573
 Review Andrich

Table 1. Models of modern test theory used in the analysis of ratings.

Class Description
Class 1 Divide by total models [19]
The Rasch model expressed in terms of thresholds dix and bix = x is mi
Pr " Xni = x , = exp (zix (bn))/ / exp (zik (bn))
an integer scoring function for successive categories† k=0
zix (bn) = aix + bix bn
x
aix = - / dix; bix = xi; di0 / 0
k=0

The generalized partial credit model with different discriminations mi

Pr " Xni = x , = exp (zix (bn)) / / exp (zik (bn))
ai among items k=0
zix (bn) = aix + bix bn
x
aix = - / ai dix; bix = aix x; di0 / 0
k=0

The nominal response model with no specifications of the mi

Pr " Xni = x , = exp (zix (bn))/ / exp (zik (bn))
parameters aix and bix k=0
zix (bn) = aix + bix bn

Class 2 Difference models [19]

The graded response model with common discrimination at
Pr " Xni = x , = r*nix - r*ni(x + 1)
thresholds among items
r*nxi = Pr " Xni = x , + Pr " Xni = x + 1 , ... + Pr " Xni = m ,; r*n0i = 1
r*nx = (exp (bn - d*x ))/(1 + exp (bn - d*x ))

The graded response model with different discriminations ai

Pr " Xni = x , = r*nix - r*ni(x + 1)
among items
r*nxi = Pr " Xni = x , + Pr " Xni = x + 1 , ... + Pr " Xni = m ,; r*n0i = 1
r*nx = (exp ai (bn - d*ix))/(1 + exp ai (bn - d*ix))
†
The only model within this table with sufficient statistics for the person and item parameters.

which RMT sits, is the alternative paradigm for the analysis of
rating scales.
Owing to the complication that the Rasch model is used in both
IRT and RMT, and in anticipation of the explication of the models,
theories and paradigms, Table 1 summarizes the class of models used
in modern test theory. All models specialize for dichotomous rat-
ings. For the purposes of this exposition, the algebraic Rasch model
in row 1 of Table 1 is distinguished from RMT.
is simply taken for granted in discussions on the application of
models in IRT. For example:
“Normally, an assumption is made when fitting an IRT model to
a set of data” [16] ;
and
“First, there is the difficulty of finding a model that fits the
available data and estimating model parameters” [16] .

The statistical modeling paradigm More explicitly:

The statistical modeling paradigm seems to be traceable to a
major figure in statistics, Karl Pearson. McKenzie writes: “one
aspect of Pearson’s approach – the construction of models to fit
the data – has if anything gained importance since his day…” [15] .
This perspective is consistent with Pearson’s view of the
application of statistics [15] :
“Using statistics, the biologist could (apparently) measure
without theorizing, summarise facts without going beyond them,
describe without explaining.”
Item response theory sits within this paradigm. That statistical
fit is the basis of choosing a model, within a broad class of models,
“…when the criterion indicates nonrandomness, an examina-
tion of residuals may suggest how the model should be modified to
improve fit” [17] ;
and
“If the proportion of misfitted items is large, the reasonable solu-
tion is to discard the Rasch model and try a model that includes
discrimination … in a systematic manner” [18] .
The range of models that can be chosen for analyzing ratings
within this paradigm is summarized in Table 1. Only the model
in the first row of the table is a Rasch model.

574 Expert Rev. Pharmacoeconomics Outcomes Res. 11(5), (2011)


There are two classes of models in Table 1. Thissen and Steinberg
referred to the first class as divide by total models and the second
class as difference models [19] . In the former, an expression charac-
terizes the probability of a response in each category, and to ensure
that the probabilities sum to one, each expression is divided by their
sum; in the second, a cumulative probability of successive categories
is specified, and then the probability of a response in any category
is the difference between these successive, cumulative probabilities.
In Table 1, the Rasch model [2,20–22] , the generalized partial credit
model [23] and the nominal response model [24] belong to the first
class. The nominal response model is the most general and the
Rasch model the most specialized, as it has the fewest parameters
to be estimated.
The Rasch model in Table 1 can be applied within IRT accord-
ing to the statistical modeling paradigm, and as quoted above,
if the model does not fit the data, then one of the other models
with more parameters that fits the data better is sought. There
is a further possible parametric specialization of the Rasch
model in which all items have the same relative thresholds,
(dix - di(x - 1) = dx - d(x - 1) 6i) , which is referred to as a rating scale
parameterization. However, because the structure and inferences
are identical at the level of an item, this is considered the same
Rasch model but with the same, rather than different, threshold
values specified across items.
The graded response model [25] belongs to the second class and
cannot be specialized to the Rasch model in the case of ratings
in more than two categories.
The experimental measurement paradigm
The experimental measurement paradigm is traceable to another
major figure in statistics, the geneticist Ronald Fisher. In refer-
ence to Fisher’s book, Statistical Methods for Research Workers [26] ,
McKenzie also writes [15] :
“But, almost more importantly, the book incorporated an
effectively new concept of the statistician’s role, and therefore a new
function for statistical theory. The message was that the statistician
should get involved in the practical business of experimentation.
This clearly pre-supposed the diffusion of the type of occupational
role that Fisher … occupied. It was not enough that the scientist
should hand their results to the statistician for ana­lysis: experi-
ments (especially large-scale applied experiments that were difficult
to ‘control’) had to be designed by those with statistical expertise.” 
Rasch  [1] studied with Fisher, and Fisher influenced Rasch at
two levels. The first, and more explicit, was Fisher’s formaliza-
tion of sufficient statistics; the second, less explicit but no less
significant, was the way Rasch, as a statistician, approached data:
it is characterized by the above quote from McKenzie. As both
influences have implications for the construction and analysis of
rating scales, both are reviewed briefly.
Sufficient statistics
Fisher’s formulation of sufficiency  [27] was decisive in Rasch’s
formulation of his measurement theory. Permitting the separation
www.expert-reviews.com
Rating scales & Rasch measurement Review
of person and item parameters, sufficiency is the defining
characteristic of Rasch models [1] :
“The realization of the concept of sufficiency, I think, is a sub-
stantial contribution to the theory of knowledge and the high mark
of what Fisher did … His formalization of sufficiency nails down
the … conditions that a model must fulfill in order to yield an
objective basis for inference.” 
From the observation of the separation of parameters as a result
of sufficiency, Rasch was able to derive the models with sufficiency
from the following requirements of the invariance of comparisons
within a specified frame reference, which includes defined classes
of individuals and classes of stimuli (items), and the conditions
for engagement between members of each class:
“The comparison between two stimuli should be independ-
ent of which particular individuals were instrumental for the
comparison;…
Symmetrically, a comparison between two individuals should be
independent of which particular stimuli within the class considered
were instrumental for comparison…” [2] .
Sufficiency implies that there is a statistic for a parameter such
that, given this statistic, the distribution of responses is inde-
pendent of the parameter. A sequence of mathematical deriva-
tions [2,20–22] led to the Rasch model of Table 1. In this model, the
successive ratings are scored with successive integers commencing
with zero. Furthermore, it follows that across a set of items, the
total score of a person is a sufficient statistic for the person’s loca-
tion parameter. Although this integer scoring and summing to
characterize a person is exactly as in CTT, it is stressed that this
follows from the requirement of invariant comparisons realized
through sufficiency, and is not simply asserted as in CTT.
Then, given this statistic, a probability distribution of responses
to the items can be written, which is independent of the person
parameter and dependent only on the threshold parameters of the
items. Specifically, from the Rasch model equation in Table 1, the
distribution of the pair of responses (xni,xnj ) to items and of person,
given the person’s total score rn = xni + xnj, is given by Equation 2 :
xni xnj
Pr "(xni, xnj) rn , = =exp (- / dik - / d jk) G/cij (2)
k=1 k=1
where
xni xnj
cij = / exp (- / dik - / d jk)
(xni, xnj) r k=1 k=1
and where / is the sum of all possible pairs of scores that
(xni, xnj) r
sum to r.
From a generalization of Equation 2 across any number of persons
and more than two items, estimates of the item thresholds can be
made. However, because the equation is independent of the person
parameters b, it means no assumptions need be made regarding the
distribution of persons. For practical and theoretical reasons, the
575
 Review Andrich
thresholds need not only to be ordered, but in the process of scale
construction, also be relatively well aligned to the person locations.
However, even though the successive categories are scored with suc-
cessive integers beginning with zero, the successive thresholds do
not need to be exactly equally spaced. Furthermore, the estimates
of the thresholds reflect the structure of the ratings and not the dis-
tribution of the persons. This feature is powerful in diagnosing any
potential problems with rating formats. The non-Rasch models in
Table 1 do not have sufficient statistics and, in the conduct of analyses
of responses, require assumptions for the distribution of persons.
Thurstone [5] and Guttman [28] preceded Rasch in arguing for
invariance in measurement data. However, Rasch went beyond
both by expressing the requirement in the form of a probabilistic,
mathematical model. Consequently, mathematical derivations
that have experimental implications can be made. We will see such
an implication for the analysis of rating scales that is present in
the experimental measurement paradigm but not in the statistical
modeling paradigm.
The model to fit the data or the other way around?
Rasch analyzed data from a study of progress in reading by stu-
dents in Denmark, which led him to the role of sufficiency in
the separation of parameters and to the implications of invariant
comparisons within a frame of reference [1] . He then formulated
the model for dichotomous responses that had the property of
sufficiency. He applied it to data from two existing tests. One was
Raven’s Progressive Matrices, a non-verbal intelligence test, and
the other was a Danish military intelligence test [29] . According
to the fit statistics that Rasch employed, the former test fitted the
model; the latter did not [1,14] .
The question arises as to which was seen to fail in the second
example: the data from the intelligence test or the model character-
ized by sufficiency? Rasch realized that specifying a priori criteria
1.0
0 4
Probability
1
0.5
0.0
-5 -4 -3 -2 -1
δ1 δ2
Location
Figure 4. The latent dichotomous responses at the threshold of the Rasch model.
Dashed lines represent the latent dichotomous Rasch model characteristic curves at the four thresholds.
576
to which responses should conform was not consistent with the
standard statistical modeling paradigm (emphasis in original) [1] :
“It is tempting, therefore, in the case with deviations of one sort
or other to ask whether it is the model or the test that has gone
wrong. In one sense this of course turns the question upside down,
but in another sense the question is meaningful. For one thing, it
is not easy to believe that several cases of accordance between model
and observations should be isolated occurrences. Furthermore
the application of the model must have something to do with the
construction of the test; at least, if a pair of tests showed results
in accordance with our theory, this relationship could easily be
destroyed by adding alien items to the tests. Anyhow, it may be
worthwhile to know of conditions for the applicability of such
relatively simple principles for evaluating test results.” 
The contrast between Rasch’s position, which specifies an
a priori criterion, and one where the fit of data to the model
is the criterion is starkly shown by the following quote from
Lord regarding IRT models for dichotomous responses [30] :
“The reader may ask for some a priori justification of (equations)
(2-1) or (2-2). No convincing a priori justification exists (how-
ever, see chapter 3). The model must be justified on the basis of the
results obtained, not on a priori grounds.” 
Thus, only the Rasch model of row one of Table 1 inherently sits
within the experimental measurement paradigm.
An example
The issue of choice of model and implications for the empirical
operation of the putative ordering of the categories goes beyond
the normal tests of fit. As dicussed later, even if ratings fit the
2
3
0 1 2 3 4 5
δ3 δ4
Expert Rev. Pharmacoeconomics Outcomes Res. 11(5), (2011)

Rasch model according to standard statistical tests of fit, it is pos-
sible to use the model to diagnose whether or not the empirical
ordering of the categories is as required. By contrast, within an
IRT and statistical modeling paradigm, the empirical ordering is
not questioned. The analysis of the example in this section makes
concrete this important, incompatible difference. The example
also shows that the difference is not only with respect to the analy-
ses of ratings, but also with the roles of substantive researchers and
psychometricians in constructing and improving scales.
The example concerns ratings of muscle tone according to the
clinician-rated Ashworth spasticity scale shown in Figure 2 [31] . The
ratings were generated by a multicenter randomized controlled
clinical trial of cannabinoid treatment of multiple sclerosis [32] . The
study recruited 661 people with multiple sclerosis from 33 clini-
cal sites. Ashworth scores were generated by therapists at each site
who were blinded as to whether the people were in the control or
treatment groups. For the purpose of this article, analyses of ratings
of 656 people with responses to all items from only their first visit
are reported. All analyses were conducted using RUMM2030 [33] .
Muscle tone was graded for multiple muscles (ten each side: six
upper limb, four lower limb). The data in the example are from
eight ratings (items) of the parts of the lower limbs (hip adduc-
tion, knee extension, knee flexion and foot plantar flexion) for
each side. The total score was taken as a summary of the muscle
tone of the lower limbs for each person.
The following Rasch model analysis shows one set of results
that are first interpreted within the experimental measurement
paradigm. This interpretation is then contrasted with an IRT
interpretation from within the statistical modeling paradigm.
The thresholds of the Rasch model
As shown in Figure 2 , the thresholds that define categories are rel-
evant in a rating scale. However, in order to reflect that they are
1.0
Threshold 1 2
Probability
0.5
0.0
-5 -4 -3 -2 -1
δ1 δ2
Location
Figure 5. Expected threshold characteristic curves for assessments at successive thresholds by independent therapists.
www.expert-reviews.com
Rating scales & Rasch measurement Review
not explicit in IRT, they are not shown in Figure 3. By contrast, to
reflect the fact that they are explicit in RMT, the thresholds dix
are shown in the CCCs of Figure 4. These thresholds are from the
Rasch model equation of the first row in Table 1. Although suc-
cessive thresholds need not be equidistant, each is located at the
point where the probability of a response in two adjacent catego-
ries is equal. We observe that in Figure 4, if a person is located in a
category (i.e., between the two thresholds that define a category),
then the response in that category has the greatest probability.
Furthermore, the Rasch model implies a latent dichotomous
response at each threshold  [21,34,35] . Figure 4 also shows, in dashed
lines, the latent dichotomous Rasch model characteristic curves
at the four thresholds. These curves are necessarily parallel in the
Rasch model.
Other dichotomous models could be considered at the thresh-
olds, but only the dichotomous Rasch model of Table 1 permits
estimation of the thresholds independently of the distribution of
the persons.
This independence of the distribution of persons is entirely
compatible with the requirement of the thresholds of a measuring
instrument. For example, it would be untenable in the natural
sciences that the locations of a measuring instrument’s thresholds
were dependent on the distribution of the entities being measured.
Of course, we stress that for estimates of the thresholds to be
independent of the distribution of persons in the Rasch model,
the ratings must be consistent with the model to a level accept-
able for the purpose. However, as is shown later, this consistency,
generally referred to as fit, is not enough.
The ratings of the example did indeed fit the Rasch model
according to the general criterion that the expected values of the
responses for each item, given the threshold and person estimates,
were not statistically different from their expected values. In
addition, the distribution of the persons was well aligned, and
3 4
0 1 2 3 4 5
δ3 δ4
577
 Review Andrich

1.0

Threshold 1 2 3 4
Probability

0.5

0.0
-5 -4 -3 -2 -1 0 1 2 3 4 5
δ1 δ2 δ3 δ4
Location

Figure 6. Threshold characteristic curves that might appear if a therapist assessing at threshold 4 was assessing at
threshold 3 in error.

in particular, a substantial number of persons were located near • A second therapist decides independently whether or not mus-
the thresholds that are now examined closely in the following cle tone meets the second threshold standard of increased tone
section. but easily flexed;
• A third therapist decides independently whether or not muscle
Requirements in an inferred experimental design
tone meets the third threshold standard of catch;
Central to the study of ratings is that in the Rasch model, the latent
dichotomous responses at the thresholds are equivalent in theory • Finally, a fourth therapist decides independently whether or not
to observed dichotomous responses at the same thresholds, which muscle tone meets the fourth threshold standard of normal tone.
are experimentally independent. In the case of the aforementioned
Ashworth spasticity rating scale, they are equivalent to a design in
In each case, the descision is made irrespective of the standards
which four different therapists decide independently whether orat the other thresholds [34,35] . In such a design, it is possible that a
not muscle tone meets the standards at each of the four thresholds.
therapist assessing at a higher threshold (e.g., normal tone) decides
Making this design concrete for a particular part of the limb:
that a part of a limb meets this higher standard, while the therapist
• One therapist decides whether or not muscle tone meets the assessing at the lower threshold (catch) decides that the same part
of the limb does not meet the lower standard. Although inconsist-
first threshold standard of increased tone restricting movement;
ent, from the independence of the responses,
Table 2. Estimates of thresholds in the Rasch model. such pairs of responses are inherent to the
Item Item Thresholds design that can, as a result, provide evidence
|i (. 4df)
2

number as to whether or not categories are working

p-value Mean 1 2 3 4 as intended.
1 LhpAd 0.168 0.041 -2.958 0.083 1.650 1.389 However, because the successive thresh-
2 RhpAd 0.583 0.050 -2.958 0.170 1.603 1.386 olds imply successively better muscle tone,
it would be required that the success rate
3 LknEx 0.233 0.093 -2.654 0.055 1.787 1.185
at the successive thresholds is decreasing
4 RknEx 0.495 0.102 -2.665 -0.147 1.681 1.537 and that the inconsistent pairs of responses
5 LknFx 0.417 -0.329 -2.748 -0.699 1.282 0.850 exemplified above are in the minority. This
6 RknFx 0.314 -0.331 -2.697 -0.850 1.129 1.097 decreasing success rate characterizes the cat-
egory order, namely, that the same part of
7 LftPl 0.388 0.175 -2.211 0.187 1.606 1.119
the limb for the same person should have a
8 RftPl 0.456 0.198 -2.221 0.234 1.605 1.175 smaller probability of being rated as meeting
Total c2 (≈ 32 df) 0.340 the standard of threshold dx than meeting
the standard at the lower threshold dx-1. For
^ 3 2 d^ 4 in all items.
Note: d
example, in the Ashworth spasticity scale,

578 Expert Rev. Pharmacoeconomics Outcomes Res. 11(5), (2011)

 Rating scales & Rasch measurement Review

because normal tone is intended to be a 4.0

more exacting standard than catch, if a per-
son’s part of a limb has some probability of
3.0
meeting the standard of catch at threshold 3,

Expected value
then the same person’s same part of the limb
should have a smaller probability of meeting 2.0
the standard of normal tone at threshold 4.
Notice again that this ordering is a require-
ment of the decisions specified independ- 1.0
ently of any observed (dichotomous) deci-
sions, and that the responses themselves may
0.0
provide evidence to the contrary.
-3 -2 -1 0 1 2 3
Figure 5 shows just such an expected order- CI 1 2 3 4 5
ing and characteristic curves at the thresh- Location
olds for the above design according to the Figure 7. Item characteristic curve and observed means (plotted points) in five
dichotomous Rasch model. As in Figure 4, it class intervals for item 5 (left knee flexion).
is not expected that the distances between CI: Class interval.
the successive thresholds are equal; never-
theless, it is expected that they are ordered more or less as in Figure 5. problem. To understand the source of the problem, it would be
Critically, as emphasized earlier, actual ratings from the above necessary to consult the therapists making the ratings.
design may not show the required threshold order. For example, To summarize so far:
if the therapist assessing at threshold d4, normal tone, had a con- • The increasing order of the thresholds is a requirement of ratings
sistently much lower standard than intended (such as a standard independent of any observed ratings;
closer to that of threshold d3, catch), then the success rates at the
two thresholds would appear similar, making estimates d^3 and d^4 • Whether observed ratings meet this requirement can be tested
also similar. Figure 6 shows the results that are likely to appear in empirically using the Rasch model;
such a case. Indeed, from the experimental design described, the • If they do not meet this requirement, then it is necessary to
estimates may even be reversed in their order, that is d^ 3 2 d^ 4 . In understand substantively and empirically what has gone wrong
this case, inconsistent responses among the therapists would not with the empirical ratings.
be in the minority.
Evidence that threshold 4 was close to threshold 3 would indi- It is stressed again that close or reversed threshold estimates are
cate the presence of a rating problem, but not the source of the possible even if the judgments fit the dichotomous Rasch model

1.0

0
1
Probability

0.5 2

0.0
-4 -3 -2 -1 0 1 2 3 4

δ1 δ2 δ4 δ3
Location

Figure 8. Category characteristic curves and latent threshold response curves for item 5.
Dashed lines represent the latent dichotomous Rasch model characteristic curves at the four thresholds.

www.expert-reviews.com 579
 Review Andrich
1.0
1 2
Probability
0.5
0.0
-4 -3 -2 -1
δ1 δ2
Location
Figure 9. Latent threshold response curves for item 5.
according to statistical tests of fit. Fit would demonstrate that
the estimates of the thresholds are invariant with respect to dif-
ferent degrees of muscle tone, but fit would not preclude, in the
previously described experimental design, that a pair of threshold
estimates are very close or reversed relative to the required order.
Furthermore, if in an experimental design as previously
described, the thresholds were close or not in the correct order, it
would be unwise to proceed to a single rating scale as shown in
Figure 2 . It would be unwise because there could be no confidence
that a rating of 4 really represents a better muscle tone than a rat-
ing of 3. In that case, choosing treatments to improve tone and
studies of improvement as a result of treatment would have prob-
lems of interpretation in the region where two adjacent thresholds
are very close. It is emphasized that evidence of the problem of the
ordering of the thresholds is obtained from an experimental design
and that the solution to the problem requires experimentation, not
statistical modeling.
Because a Rasch model analysis of ratings [21,34,35] is equivalent
to the design in which therapists make independent decisions at
the different thresholds and the responses are analyzed according
to the dichotomous Rasch model, two important properties follow:
• From a Rasch model ana­lysis of the ratings, pairs of adjacent
thresholds can be close or even reversed;
• If estimates of a pair of adjacent thresholds are close or reversed,
there is a problem with the operation of the ratings defined by
the thresholds.
Threshold estimates in the example
With the aforementioned background, an illustrative ana­lysis of
ratings from the Ashworth spasticity scale using the Rasch model
580
3 4
0 1 2 3 4
δ4 δ3
of Table 1 is now reported. We first note that when means of five
class intervals are compared with their expected values for each
item to give an approximate c2 statistic, no item showed misfit,
nor did the general test of fit with all item statistics pooled. These
statistics and threshold estimates are shown in Table 2 .
The item characteristic curve of item 5 (left knee flexion),
together with the observed means of five class intervals, is shown
illustratively in Figure 7. The observed means are clearly close to
their expected values. However, as indicated earlier, this statisti-
cal fit, which exploits the threshold estimates from the data to
obtain the expected values, is essentially immaterial in studying
the relative estimates of thresholds.
Regarding threshold estimates, Figure 8 shows the CCCs for the
same item 5. The figure also shows, in dashed lines, the latent,
inferred dichotomous response curves at the thresholds. It is evident
that the estimates of thresholds d3 and d4 are not only similar, but
that they are reversed: d^ 3 2 d^ 4 . In this case, and unlike in Figure 4,
the rating of 3 (catch) never has the greatest probability. Thus a
rating of 3 is not defined on the continuum. In addition, as is evi-
dent in Table 2, the reversal d^ 3 2 d^ 4 is present in all items. Standard
errors, which are available, are not provided for simplicity reasons
and are not required to infer a problem with the reversal d^ 3 2 d^ 4 .
To emphasize the inference of their equivalence to indepen-
dent dichotomous responses at the thresholds, Figure 9 shows the
threshold curves from Figure 8 again, but with the CCCs removed.
Interpretation from the experimental measurement
paradigm of the Rasch model analysis
The most important observation, from the experimental measure-
ment paradigm, is the closeness (and reversal) of estimates d^ 3 and
d^ 4 . Paraphrasing Rasch as mentioned earlier, the ratings, not the
Expert Rev. Pharmacoeconomics Outcomes Res. 11(5), (2011)
 Rating scales & Rasch measurement Review

1.0

π*4
0.8
π*3
Probability

0.6

π*2
0.4

0.2

π*1
0.0
-5 -4 -3 -2 -1 0 1 2 3 4 5
δ1* δ2* δ3* δ4*

Location

Figure 10. The cumulative probabilities px* and thresholds dx* in the graded response model for item 5.

model, would be interpreted as having gone wrong. The fact that of thresholds 3 and 4 are d^ 3 = 1.543 and d^ 4 = 1.217 , and the
there is something wrong with the ratings is confirmed by the number of persons located between 1.0 logits and 2.0 logits is
evidence that all items show exactly the same pattern d^ 3 1 d^ 4 . In 199 – that is, 30%.
particular, the evidence challenges the very understanding of the A comment on the significance of the difference between thresh-
assumed difference between catch and normal tone. olds may be relevant. First, if statistical significance tests are con-
Furthermore, the solution to the problem needs to be substan- ducted, they would be one-tailed tests that di(x + 1) 2 dix 6x $ 1
tive, empirical and experimental. The analysis cannot reveal the and if any di(x + 1) < d^ ix , the hypothesis of the required order is
source of the problem, only the location of the problem [14] . As a automatically rejected. Thus generally, it would be required that
first step, feedback needs to be sought from the clinicians carrying adjacent thresholds are significantly different and in the correct
out the ratings; in particular, whether they experienced difficulty order. However, two features of such significance testing need to
distinguishing between catch and normal tone. The evidence sug- be appreciated. First, that statistical significance of such tests is
gests that they did. Then a new definition of the categories, which substantially a function of the sample size and therefore it can
attempts to distinguish better between them, might be trialed. be contrived. Second, there is no particular distance between
Alternatively, perhaps the levels catch and normal cannot be distin- thresholds in the experimental design, which the Rasch model
guished either theoretically or empirically, and the two categories represents, that can be specified. It would be as if, in measure-
need to be amalgamated into one category, given a new name and ment, we considered that every ruler should have its thresholds
again trialed. Thus, to help construct an improved rating format, exactly 1 cm apart. Clearly, this would be absurd, and different
the psychometrician who has analyzed the ratings needs to engage rulers used for different purposes must have different distances
with the clinicians who have constructed and used the ratings. between their thresholds.
Of course, other factors might contrib-
ute to the closeness of thresholds, such as Table 3. Estimates of the Rasch model and graded response model
few persons in the region of these thresh- thresholds for item 5.
olds, rendering the estimates very unstable. Model Mean Threshold estimates Note
However, having few people in a region does
1 2 3 4
not necessarily lead to reversed thresholds.
In any case, in this example, there is a sub- Rasch model d^ x 0.37 -2.99 -0.48 1.14 0.85 ^
d3 2 d^ 4
stantial number of people in the region of
Graded response model -0.35 -3.05 -0.58 0.76 1.46 * *
the reversed thresholds. Specifically, the
^d*x d^ 3 1 d^ 4
averages across the items of the estimates

www.expert-reviews.com 581
 Review Andrich
Thus, an excessive focus on statistical significance plays into the
statistical modeling paradigm. In general, the thresholds should
be correctly and clearly ordered and show a consistent structure.
In the example of this paper, one pair of thresholds was slightly
disordered and this was sufficient to conclude that there was a
problem. However, even if those two thresholds were ordered and
statistically significantly different, but only just ordered when com-
pared with the other thresholds, as illustrated in Figure 6, it would be
evidence that the category between the thresholds was not work-
ing as well as the others. In the case that all items showed exactly
the same pattern, even if they were just significantly ordered, this
would help in concluding that there is a problem in the format of
all items in this example.
In many cases, as in the example in this article, multiple items
have the same response format. However, in many cases, it may
also be more consistent with the nature of the items to have dif-
ferent numbers of categories for different items, with the number
of categories present natural to the content of the item. Different
items with different numbers of categories can be relevant in per-
formance assessments that have specific protocols for the different
criteria. The general features of the assessments, governed by the
experimental setup and expectations, should govern a decision that
ordering of the categories is not working as intended. Each case
must be justified in its own right, not on some general statistical
rule that is apparently independent of any case.
Interpretation from a CTT analysis
The interpretation from a CTT perspective can be very brief – the
empirical ordering of the categories simply does not arise in CTT.
Studies of the empirical ordering of categories has been carried out
independently [36] , but textbooks on CTT do not refer to this issue.
Interpretation from an IRT analysis from the statistical
modeling paradigm
The divide by total models
As indicated earlier, the Rasch model is used in IRT theory as
part of the statistical modeling paradigm. In the example in this
article, it would be reported that the model fits the data and
there would be no concern for the reversal of the thresholds. As a
consequence, interpretations regarding locations of persons, for
example before and after some intervention, would be reported
confidently. It might also be noted that simply collapsing two cat-
egories, where the thresholds are reversed, in order to fit the Rasch
model without further experimentation, is not consistent with the
experimental measurement paradigm. The fact that the data fit
the Rasch model is central to the argument of the article, but it is
only a necessary condition. Post hoc adjustments of responses to fit
the Rasch model are excellent exploratory tools and may be neces-
sary in some cases before making other relevant interpretations,
but they need to be backed up by relevant experimental evidence.
In the nominal response model of Table 2 , there are no con-
straints on the estimates of aix,bix and with its analysis of the
ratings, the parameters would not be interpreted in terms of
thresholds. They would simply be taken as descriptive parameters
in the model.
582
Difference models
Instead of a distinct latent response at each threshold as in the
Rasch model and shown in Figure 4, in the graded response model
only one response process across the continuum is assumed, with
this one process partitioned at thresholds post hoc. This is illus-
trated in Figure 10 for item 5. As a result of this structure, the
thresholds in the graded response model, which are different
from the thresholds in the Rasch model, are necessarily in order.
Table 3 shows estimates of thresholds for item 5 of the example for
the Rasch and graded response models, showing that although
* *
d^ 3 2 d^ 4 from the Rasch model, d^ 3 1 d^ 4 from the graded response
model. Since the latter ordering is an integral property of the
model, irrespective of the empirical properties of the ratings, it
means that evidence from the graded response thresholds can-
not tell whether or not the intended ordering of the categories is
present in the ratings.
Category characteristic curves can be drawn from the graded
response model. In the example, and despite the thresholds d^*x
being ordered, these curves will appear as in Figure 8 , with catch
never having the greatest probability. However, because the model
accounts for the ratings, no questions follow from such a figure.
Samejima  [25] , and many other papers which apply the graded
response model, show curves as in Figure 8 , and no comment is
made that there are ratings that never have the greatest proba­
bility. If a comment is made, it does not follow from the logic of
the graded response model, and it would be an acknowledgement
of the implications that follow only from an understanding of the
response structure of the Rasch model.
Contrasting responsibilities for action
Clearly, the RMT analysis in the experimental measurement
paradigm identifies a problem with the rating scale format (a
problem not identified in an IRT analysis in the statistical mod-
eling paradigm). Therefore, the following question arises: who
deals with the problem? One answer might be that it is the cli-
nician/substantive researcher who uses the scales and not the
statistician/psycho­metrician. Of course, the clinician/substan-
tive researcher and the psychometrician/statistician may be the
same person, as was often the case with Fisher. A second, more
tenable, answer is that the psychometrician of necessity has to
work with the clinician. To paraphrase McKenzie’s [15] contrast-
ing of Pearson’s and Fisher’s approaches, it is not enough for a
psychometrician to find a model that fits the data and to describe
results without explaining them.
Further implications of differences in paradigms
Although they are important, the implications of the two dif-
ferent paradigms go beyond the analyses and inferences from
ratings. Observing what might be seen as a clash of the two
paradigms described and illustrated in this article, Cook argued
that the models were ‘just tools’ and asked for less controversy
in the use of these tools  [37] . However, as demonstrated earlier,
the way in which the tools are used can be so different that
different questions about the data are answered and different
outcomes supported.
Expert Rev. Pharmacoeconomics Outcomes Res. 11(5), (2011)
 Rating scales & Rasch measurement Review

This difference in paradigms manifests itself in papers and
research proposals that are submitted and the reviews they
engender. If research from one paradigm is reviewed from the
perspective of the different paradigm, then a paper or a research
proposal is very likely to be rejected, and rejected with some pas-
sion and hostility, the feature observed by Cook. Controversy
seems inherent when a paradigm is challenged  [3] . However, if
it is understood that there is a paradigm difference, then this
passion and hostility towards excellent research within a para-
digm may be overcome. It may be overcome by researchers stat-
ing explicitly the paradigm within which they are working and
reviewers respecting the paradigm. In some cases, reviewers may
declare that, because of the paradigm within which the paper
or research proposal is written, they are not in a position to
review a paper.
This of course requires an articulation of the different para-
digms, a task inherently difficult because their respective assump-
tions are deeply embedded and taken for granted. An aim of
this article is to help bring to the surface the presence of two
measure­ment paradigms in modern test theory, and to demon-
strate not only their basic difference, but also the implications of
this difference in the case of rating scale construction.
In summary, for the purpose of the construction and validation
of rating scales that may be used at different levels from clinical
trials to individual assessments, the article makes the case for the
application of the Rasch model within the experimental measure-
ment paradigm, and by implication, makes the case against the
application of the graded response model within the dominant
statistical modeling paradigm.
Expert commentary
In the analysis of rating scales in the social sciences, it is gener-
ally considered that models of modern test theory that ostensibly
transform ratings into measurements are only tools. Furthermore,
it is generally considered that the criterion for choosing the model
as only a tool, from a broad class of models, is the one that
best accounts for the data. This criterion sits within what is
termed a statistical modeling paradigm. However, an alternative
case for choosing a model is that it provides criteria that ratings
should meet if they are to provide interpretable measurements.
The Rasch model, which arises from the criterion of invariance
of comparisons of persons and ratings within a specified frame
of reference, is such a model. This criterion sits within what is
termed an experimental measurement paradigm. While hav-
ing features in common, this article illustrates how a mutually
exclusive question arises from the two paradigms and the appli-
cation of the respective models. Specifically, it shows that in the
experimental measurement paradigm, ordering of the categories
of the rating scales is tested empirically, and if ordering is not
met, further experimentation is required before inferences from
the ratings are made. It also shows that in the statistical mod-
eling paradigm, inferences from the ratings are made with their
empirical ordering taken for granted, whether justified or not.
Therefore, with the importance of rating scales, it is considered
that models are not merely tools, but that the choice of model
governs the very understanding of the requirement of ratings and
the experimentation that goes into constructing them.
Five-year view
In the next 5 years, the use of rating scales for the assessment
of health outcomes will increase and therefore become more
important. It is suggested that there will be an increasing rec-
ognition of the paradigm difference between the application
of RMT and IRT and the potential advantages of the former
in which criteria that rating scales must meet are specified in
advance. With this recognition will come a stronger relationship
between psychometricians and clinicians in the construction and
verification of rating scales, with the consequent better under-
standing of psychometrics by clinicians, and of clinical issues
by psychometricians.
Acknowledgements
John Zajicek from the Peninsula College of Medicine and Dentistry
(Plymouth, UK) permitted use of the data in the empirical example. Stefan
Cano and two anonymous reviewers provided valuable comments for earlier
drafts of the article.

Key issues
• Rating scales in the measurement of health outcomes for variables where no instruments of the kind found in the natural sciences can
be constructed are used extensively and their use will increase.
• Modern test theory, encompassing item response theory (IRT) and Rasch measurement theory (RMT), will increasingly supersede
classical test theory in the analysis of rating scales and in the reporting of outcome measures.
• Deep differences between a statistical modeling paradigm and an experimental measurement paradigm that underpin IRT and RMT in
the analysis of rating scales have emerged: in the former, the criterion for the choice of a model is determined by its fit to the observed
ratings; in the latter, the criterion of the invariance of comparisons that rating scales should meet is specified a priori in terms of a
response model. This model is a Rasch model.
• A specific critical issue in rating scales is the ordering of the categories: in IRT, it is taken for granted; in RMT, it is treated as a
hypothesis with the implication that if the ordering is not working as intended, then the response categories need to be studied and
improved experimentally. Furthermore, if it is found that the empirical ordering is not as required, then the clinician and
psychometrician must work together to create ratings that are substantively valid and empirically ordered.
• Paradigm differences usually result in controversy and hostility towards papers submitted for publication and proposals submitted for
funding. The controversy and hostility may be overcome if authors and researchers make explicit the paradigm within which they are
working and reviewers respect the paradigms.

www.expert-reviews.com 583
 Review Andrich
Financial & competing interests disclosure
The author has no relevant affiliations or financial involvement with any
organization or entity with a financial interest in or financial conflict with
the subject matter or materials discussed in the manuscript. This includes
References
Papers of special note have been highlighted as:
• of interest
•• of considerable interest
1 Rasch G. Probabilistic Models for Some
Intelligence and Attainment Tests. The
University of Chicago Press, IL, USA
(1980).
• A relatively easy introduction to the
principles of Rasch models.
2 Rasch G. On general laws and the meaning
of measurement in psychology. In:
Proceedings of the Fourth Berkeley Symposium
on Mathematical Statistics and Probability
(IV). Neyman J (Ed.). University of
California Press, CA, USA, 321–334
(1961).
• Makes the case for invariance of
comparisons in measurement.
3 Kuhn TS. The Structure of Scientific
Revolutions (2nd Edition). The University
of Chicago Press, IL, USA (1970).
• Introduces the concept of research
paradigms with mutually
exclusive assumptions.
4 Kuhn TS. The Copernican Revolution.
Harvard University Press, MA, USA
(1957).
5 Thurstone LL. Attitudes can be measured.
Am. J. Sociol. 33, 529–554 (1928).
6 Reise SP. The impact of
multidimensionality on unidimensional
item response theory model parameters.
Presented at: International Conference on
Outcome Measurement. Bethesda, MD,
USA, 1–3 September 2010.
7 Choppin BHL. Lessons for psychometrics
from thermometry. Eval. Educ. 9, 9–12
(1985).
8 Likert R. A technique for the measurement
of attitudes. Arch. Psychol. 140, 5–55 (1932).
9 Hobart JC, Cano SJ, Zajicek JP, Thompson
AJ. Rating scales as outcome measures for
clinical trials in neurology: problems,
solutions, and recommendations. Lancet
Neurol. 6, 1094–1105 (1970).
•• Shows the increasing significance and use
of rating scales in medicine.
10 Traub RE. Classical test theory in
historical perspective. EM:IP 16(4), 8–14
(1997).
584
employment, consultancies, honoraria, stock ownership or options, expert
testimony, grants or patents received or pending, or royalties.
No writing assistance was utilized in the production of this
manuscript.
11 Bock RD. A brief history of item response
theory. EM:IP 16(4), 21–33 (1997).
12 Wright BD. A history of social science
measurement. EM:IP 16(4), 33–45
(1997).
•• Places the Rasch model within the
history of measurement in the
social sciences.
13 Cano S, Klassen AF, Scott A, Thoma A,
Feeny D, Pusic A. Health outcome and
economic measurement in breast cancer
surgery: challenges and opportunities.
Expert Rev Pharmacoeconomics Outcomes
Res. 10(5), 583–594 (2010).
14 Andrich D. Controversy and the Rasch
model: a characteristic of incompatible
paradigms? Med. Care 42, 7–16 (2004).
•• An introduction to the characteristics of
different measurement paradigms.
15 McKenzie DA. Statistics in Britain,
1865–1930. Edinburgh University Press,
Edinburgh, Scotland, UK (1981).
16 Hambleton RK. Emergence of item
response modeling in instrument
development and data analysis. Med. Care
38(9 Suppl.), II60–II65 (2000).
17 Bock RD, Jones LV. The Measurement and
Prediction of Judgement and Choice.
Holden Day, CA, USA (1968).
18 Divgi DR. Does the Rasch model really
work for multiple choice items? Not if you
look closely. J. Educ. Measure. 23, 283–298
(1986).
19 Thissen D, Steinberg L. A taxonomy of
item response models. Psychometrika 51,
567–577 (1986).
20 Andersen EB. Sufficient statistics and latent
trait models. Psychometrika 42, 69–81
(1977).
21 Andrich D, A rating formulation for
ordered response categories. Psychometrika
43(4), 561–574 (1978).
22 Wright BD, Masters GN. Rating Scale
Analysis: Rasch Measurement. MESA Press,
IL, USA (1982).
23 Muraki E. A generalized partial credit
model: application of an EM algorithm.
Appl. Psychol. Measure. 16(2), 159–176
(1992).
24 Bock RD. Estimating item parameters
and latent proficiency when the responses
Expert Rev. Pharmacoeconomics Outcomes Res. 11(5), (2011)
are scored in two or more nominal
categories. Psychometrika 37, 29–51
(1972).
25 Samejima F. Estimation of latent ability
using a response pattern of graded scores.
Psychometric Monographs 34(17), 100–114
(1969).
26 Fisher RA. Statistical Methods for Research
Workers. Oliver and Boyd, Edinburgh,
Scotland, UK (1925).
27 Fisher RA. Two new properties of
mathematical likelihood. Proc. Royal Soc. A
144, 285–307 (1934).
28 Guttman L. The principal components of
scalable attitudes. In: Mathematical
Thinking in the Social Sciences. Lazarsfeld PF
(Ed.). The Free Press, IL, USA, 216–257
(1954).
29 Raven JC. Matrix tests. Mental Health 1,
10–18 (1940).
30 Lord, FM. Applications of Item Response
Theory to Practical Testing Problems.
Lawrence Erbaum Associates, NJ, USA
(1980).
31 Ashworth B. Preliminary trial of
carisoprodol in multiple sclerosis.
Practitioner 192, 540–542 (1964).
32 Zajicek J, Fox P, Sanders H et al.
Cannabinoids for treatment of spasticity
and other symptoms related to multiple
sclerosis (CAMS study): multicentre
randomised placebo-controlled trial.
Lancet 362(8), 1517–1526 (2003).
33 Andrich D, Sheridan, BS, Luo G.
RUMM2030: a windows program for the
analysis of data according to Rasch
unidimensional models for measurement.
RUMM Laboratory, Perth, Australia
(2010).
34 Andrich D. Educational measurement:
Rasch models. In: International
Encyclopedia of Education (3rd Edition).
Baker E, Peterson P,McGaw B (Eds).
Elsevier, Amsterdam, The Netherlands
(2010).
35 Andrich D. Understanding the response
structure and process in the polytomous
Rasch model. In: Handbook of Polytomous
Item Response Theory Models: Developments
and Applications. Nering M, Ostini R
(Eds). Lawrence Erlbaum Associates, NJ,
USA, 123–152 (2010).
 Rating scales & Rasch measurement Review

• Provides the mathematics for the
characteristics of the Rasch model
exploited in the article.
36 Dubois B, Burns JA. An analysis of the
meaning of the question mark response
category in attitude scales. Educ. Psychol.
Measure. 35, 869–884 (1975).
37 Cook KF. Bewildered, befuddled,
be-tooled; a blue collar psychometrician’s
defense of measurement models as tools:
just tools. Presented at: International
Conference on Outcomes Measurement.
Bethesda, MD, USA, 1–3 September
2010.

www.expert-reviews.com 585