IRT

Henning

Advantages

- Sample free item calibration

In classical theory P value is sample dependent, in IRT an item difficulty
scale, i.e., independent of ability differences of sample is made.
-Test free person measurement
In classical theory, a test should be administered to everybody. Test A to
person A and test B to person B would not be comparable.
In IRT it is possible to compare abilities of two persons using different tests
by referring to small bank of common items or common persons.
- multiple Reliability estimate
A test may be reliable towards the mean rather than either and this means that
ability estimates varies accuracy according to position along the scoring
continuum.

Item response theory

1) comparing the three approaches
CT and G are compatible, G is more powerful
2) disadvantage of CT
a) individual’s ability and test performance cannot be determined
b) in determining person ability the only information is item facility.
Major theorem:
1) an individual’s ability (expected performance) is the function of both the level of
item difficulty and the level of individual’s ability.
2) The unidimentionality assumption
IRT assumes that the items in a trait measure a single or unidimensional
ability. Or trait and that the items form a unidimensional scale of
measurement.
3) items are locally independent

there are some other assumptions that are stated by item characteristics curve
(ICC). ICC explains the relationship between an individuals ability (probability of
getting an time right)
The type of information
1) the degree ot which an item discriminates among individuals of different
abilities (the discrimination parameters (a) )
2) the level of difficulty of the item ( the difficulty parameter (b) )
 3) the probability that an individual of low ability can answer an item correctly
(the chance of guessing, parameter (c) )

Three parameter model

This model assumes that the relationship between the level of ability and the
probability of correct response is non-linear, and that it is a function of all three
parameters.
Ability scale
Chance for guessing for all items is .20. This parameter defines the lower bound
(asym?) never meets the horizontal line
The probability is parameter (a). Then the parameter (b), the level of ability at
which the probability of a correct response is half way between the chance
parameter and one in this example it is .60.

Item
1difficulty parameter = - 2.0 easiest
2 difficulty parameter = 00 just about right
3 difficulty parameter = + 2 the most difficult one

a person of low ability (2 standard deviation below the mean and in ability level)
has 60% chance ot get item # 1 correct while people with 1 standard deviation
below the mean or above are certain that they will get the item correct.
Item # 3 only those whose ability level is 2 standard deviations above the mean
ability have a probability greater than chance of answering the item correctly.
Those items also vary in terms of discrimination parameter (a) which is
proportional to the slope of the ICC at the point of the difficulty parameters. The
steeper the slope, the greater the discrimination will be. Thus item #2 which has
the smoothest steep has the lowest discrimination.
That is, there is very little change in the probability of correct response as function
of differences in ability.
In two parameter model it is assumed that individuals of low ability will have
virtually no chance of a correct response so chance parameter is zero.
In one parameter model (Rash) the discrimination of all the items is assumed to be
equal and it is also assumed that there is no guessing.
Θ is the ability level
First a set of observed responses are collected from a relatively large sample.
Then IRT model is selected. Then through item ? a particular model is determined.
If the assumptions are met:
1) Assuming a large number of items that measure the same trait, an individual's
ability estimate is independent of the particular set of items.
2) Assuming a large population of test takers , the item parameter estimates are
independent of the particular sample
3) Precision of measurement.

Item information function:

 Limitations on CTS and GT (which are sample dependent) in IRT, reliability is
given by item information function. IIF refers to the amount of information a given
item provides for estimating an individual’s level of ability, and is a function of both
the slope of the ICC and the amount of variation at each ability level.
The steeper the slope and the smaller the variation at each ability level, the greater the
information. A difficult item will provide little information at low ability level and
vice-versa.
Item #1 has a very high information function, which is at its maximum at –2.0 on the
ability level.
Item #3 is like #1 at high level.
Item #2 gives little information at ability level.
Test information function
Test information function (TIF) is the sum of the item information functions, each of
which contributes independently to the total, and is a measure of how much
information a test provides at different ability levels.
TIF is the IRT analog of CTS theory reliability.