contribution of the revised
History of Psychological Assessment
• Chinese – civil service testing to
determine suitable people to work in a
government.
o Determine work evaluations
and promotions decisions.
PEOPLE IN PSYCHOLOGY (PSYCHOLOGICAL
ASSESSMENT)
• Charles Darwin – higher forms of life
evolved partially because of differences
among individual forms of life within a
species. (“the origin of species” 1859)
• Francis Galton – some people
possessed characteristics that made
them more fit than others. (“Hereditary
Genius” 1869)
• James McKeen Cattell – coined the
term Mental Test.
• Wilhelm Wundt – set up a laboratory at
the University of Leipzig in
o Founded the science of
psychology.
o Following in the tradition of
Weber and Fechner.
• Alfred Binet (with Theodore Simon) –
developed the 1st Major General
Intelligence Test.
TESTS
Binet-Simon Scale
• 1st version (Binet-Simon Scale, 1905).
o 30 items of increasing difficulty
and designed to identify
intellectually subnormal
individuals.
o Standardization sample = 50
children who had been given
the test under standard
conditions – same instructions
and format.
o Mental age concept – one of
the most important
1908 Binet-Simon Scale.
Standford-Binet Intelligence Scale
• Lewis Madison Terman of Stanford
University – revised the Binet test for
use in United States.
- The only American version of the
Binet test that flourished.
- Standardization sample = 1000
people
- Original items were revised, many
new items were added.
Army Alpha & Army Beta
• Developed by WW1, Robert Yerkes
(head of committee of distinguished
psychologist)
- Two structured group tests of
human abilities.
- Most fair enough test (culturally)
- Alpha = required reading abilities
- Beta = measures illiterate adults’
intelligence.
Woodworth Personal Data Sheet
- 1st structured personality test.
- Developed during the WW1
- Published in final form just after the
war.
- Assumed that a test response can
be taken at Face Value.
Rorschach Inkblot Test (1932)
• 1st published by Herman Rorschach of
Switzerland, 1921.
• David Levy (US) introduced it.
- 1st doctoral dissertation in US.
- Was not published until 1932, when
Sam Beck (Levy’s student) decided
to investigate the properties of the
Rorschach test scientifically.
• Highly controversial projective test that
provides an ambiguous stimulus and
 asks the subject to explain what the
inkblot might be.
Thematic Apperception Test (TAT)
• Henry Murray & Christian Morgan,
1935.
- Requires the subject to make up a
story about the ambiguous scene.
- Purported to measure human
needs and thus to ascertain
individual differences in motivation.
Minnesota Multiphasic Personality Inventory
(MMPI)
- New era for structured personality
tests.
- Test items were originally
developed by selecting questions
that have been endorsed by people
diagnosed with different mental
health conditions.
- Test that made no assumptions
about the meaning of a test
response. Such meaning was to
determined by empirical research.
California Psychological Inventory (CPI)
- Structured personality test
developed according to the same
principles as the MMPI.
16 Personality Factor Questionnaire (16PF)
- A structured personality test based
on the statistical procedure of factor
analysis.
CONCEPTS
• Representative sample
- Comprises individuals similar to
those for whom the test is to be
used.
- Must reflect all segments of the
population in proportion to their
actual numbers.
• Mental age
- Measurement of a child’s
performance on the test relative to
other children of that particular age
group.
• Factor Analysis
- Is a method of finding the minimum
number of dimensions
(characteristics, attributes) called
factors, to account for a large
number of variables.
• Face Value (Face Validity)
- When an assessment or test
appears to do what it claims to do.
NORMS AND BASIC STATISTICS FOR TESTING
• Descriptive statistic – methods used to
provide a concise description of a
collection of quantitative information.
- Mean, Standard Deviations etc.
• Inferential statistics - methods used to
make inferences from observations of a
small group of people known to a larger
group of individuals known as a
population.
- Practical applications or real-life
applications of the results.
Properties of Scale
• Magnitude
- Property of “moreness”.
- Scale has magnitude if (for example)
attribute represents more, less, or
equal amounts of the given quantity
than does another instance.
- Ex. Ranges (1-10 none at all, 11-20
mild)
- Values can be rank ordered.
• Equal intervals
- Difference between two points at
any place on the scale has the same
meaning as the difference between
two other points that differ by the
same number of scale unit.
 - Scale units are equal. Frequency Distributions
- Scale has equal interval when the
- Displays scores on a variable or a measure
relationship between the measured
to reflect how frequently each value was
units and some outcome can be
obtained.
described by a straight line or a
- Simple frequency distribution – indicates
linear equation in form Y = a+bX.
that individual scores have been used and
• Absolute 0
the data have not been grouped.
- Is obtained when nothing of the
- Group frequency distribution – test-score
property being measured exist.
intervals (class intervals), replace the
- For example, if you are measuring
actual test scores.
heart rate and observe that your
➢ ex. 95-99; f=5
patient has a rate of 0 and has died,
then you would conclude that there Graphs
is no heart rate at all.
1. Histogram
Types of Scales 2. Bar graph
3. Frequency polygon
• Nominal Scales
- not really a scale. Measure of Central Tendencies
- Only purpose is to name objects.
1. Arithmetic Mean
- Does not have any of the 3
- Is equal to the sum of the
properties.
observations (test scores) divided
• Ordinal Scale
by the number of observations.
- Has magnitude
- Appropriate for: INTERVAL & RATIO
- IQ tests are in ordinal scale.
data. (When normal distribution is
• Interval scale
assumed to be observed.)
- Has magnitude and equal intervals.
- For frequency distribution:
- Common example: temperature in
degrees Fahrenheit.
- 35 degrees Fahrenheit is warmer
than 32 degrees Fahrenheit.
- Difference between 90*F and 80*F
is equal to a similar difference of 10
degrees at any point on the scale. 2. Median – in ascending or descending
• Ratio order; middle.
- Has all 3 properties. - Appropriate for: ORDINAL,
- Ex. Kelvin Scale which is based on INTERAL, & RATIO data.
the absolute zero. - When relatively few scores fall
under the HIGH or LOW end of the
Describing Data distribution.
- A distribution may be defined as a set of 3. Mode – frequently occurring.
test scores arrayed for recording or study. - AVERAGE – adjacent scores occur
- Raw scores – straightforward, unmodified equally often and more often than
accounting of performance that is usually others.
numerical.
 - BIMODAL DISTRIBUTION – 2
modes.
- Appropriate for: NOMINAL DATA.
- Useful in analyses of qualitative or
verbal nature.
Measurement Scales in Psychology
- Ordinal level of measurement is
most frequently used in psychology.
- Kelinger, “Intelligence, aptitude, and
personality test scores are basically
and strictly speaking, ordinal.”
Measure of Variability
• Variability – how scores in a distribution
are scattered or dispersed.
• Two or more distributions of test scores
can have the same mean even though
differences in the dispersion of scores
around the mean can be wide.
The range
- Equal to the difference between the
highest and lowest scores.
- Simplest measure but potential use is
limited.
The average deviation
➢ One extreme score can radically alter
the value of the range.
➢ Resulting description of variations may
be understated or overstated.
- Provides a quick but gross description of
the spread of scores.
The interquartile and semi-quartile ranges
- Distributions of test scores can be
The standard deviation
divided into four parts such that
25% of the test scores occur in each
quarter.
- Equal to the difference between Q3
and Q1. Like the media, it is an
ORDINAL STATISTICS.
- Q1, Q2 (median), Q3.
- SEMI-INTERQUARTILE RANGE; equal
to the interquartile range divided by
2.
- Quartile: specific point, Quarters:
interval.
- An individual score may, for
example, fall AT THE THIRD
QUARTILE or IN THE THIRD
QUARTER (but not “in” the third
quartile or “at the third quarter)
- Symmetrical distribution; Q1 and
Q3 have equal distance from the
median.
➢ Skewness; is the lack of
symmetry.
- Absolute value of the deviation
score, ignoring the positive or
negative sign and treating all
deviation scores as positive.
- All deviation score are then
summed and divided by the total
number of scores.
- A measure of variability equal to the
square root of the average squared
deviations about the mean.
- Square root of the variance Commented [AD1]: First: find the mean
➢ Variance – equal to the arithmetic 2nd: minus each score to the mean (deviation)
3rd: square every deviation
mean of the squares of the Variance: add all the squared deviation then divided by it to
difference between the score in a the total number of score.
distribution.
SD: squared the variance
 ∑(𝑥−𝜇)2
➢ Formula: 𝜎 2 =
𝑁
▪ When squared = SD.
Skewness
- Symmetry is absent.
- Positive skew; most of the scores fall under
the low end.
➢ May indicate test was too difficult.
- Negative skew; most of the scores fall
under the high end.
➢ May indicate test was too easy.
Kurtosis
- Steepness of a distribution in its center.
➢ Platykurtic – relatively flat.
➢ Leptokurtic – relatively peaked
➢ Mesokurtic – somewhere in the
middle.
The Normal Curve
- Development:
➢ 18th century: Abraham DeMoivre
➢ Marquis de Laplace
➢ 19th century: Karl Friedrich Gauss
➢ “Laplace-Gaussian Curve”
➢ Karl Pearson “Normal Curve” to be
more inclusive to those who
contributed.
- Theoretically, the normal curve is a bell-
shapes, smooth, mathematically defined
curve that is highest at its center. (Ideal
curve.)
- From the center it tapers on both sides
approaching the X-axis asymptotically (it
approaches, but never touches, the axis.)
- In theory, the distribution of the normal
curve ranges from negative infinity to
positive infinity.
The area under the normal curve
- 50% of the scores occur above the mean
and 50% of the scores occur below the
mean.
- Approximately 34% of all scores occur
between the mean and 1 SD below and/or
above the mean.
- Approximately 68% of all scores occur
between the mean and ±1 SD.
- Approximately 95% of all scores occur
between the mean and ±2 SD.
- Approximately 99% of all scores occur
between the mean and ±3 SD.
- Has two tails
- General rule: the larger the sample size
and the wider the range of abilities
measured by a particular test, the more the
graph of the test scores will approximate
the normal curve.
- In terms of mental ability as
operationalized by tests of intelligence,
performance that is approximately 2 SD
from the mean (i.e., IQ of 70-75 or lower or
IQ of 125-130 or higher) is one key element
in identification.
➢ IQ tests: mean= 100, SD=15
➢ Success at life’s tasks, or its absence,
also plays a defining role, but the
primary classifying feature of both
 gifted and retarded groups is intellectual
deviance. These individuals are out of
sync with more average people, simply
by their difference from what is
expected for their age and
circumstances.
Describing Distributions
• Mean – average score.
• Standard deviation – approximation of
the average deviation around the mean.
- Square root of the variance.
- Square root of the average squared
deviation around the mean.
- Tho not average deviation, it gives a
useful approximation of how much
a typical score is above or below the
average score.
• Variance - average squared deviation
around the mean
Standard Scores
• Z-score – transforms data into
standardized units that are easier to
interpret.
- Standard deviation = 1, Mean = 0.
- Difference between a score and the
mean, divided by the SD.
- Score: equal to the mean, Z-score is
0.
o Greater than the mean, Z-
score is positive.
o Less than the mean, Z-score
is negative.
- equal to the difference between a
particular raw score and the mean
divided by the standard deviation.
• McCall’s T (T-Score)
- Exactly the same as standard scores
(Z-score)
- Standard deviation = 10, Mean =
50.
- transforming Z-score into T-Score
- advantage: none of the scores is
negative.
• Stanine (standard nine) – converts any
set of scores into a transformed scale,
which ranges from 1 to 9.
- Developed in the US Air Force
during WWII.
- Mean = 5, SD = approximately 2.
- Five steps to go from raw scores to
stanines:
➢ Find the mean of the raw
scores.
➢ Find the SD of the raw
scores.
➢ Change the raw scores to Z-
scores.
➢ Change the Z-scores to
percentiles. (use “areas of
standard normal
distribution”)
➢ Use the table below to
convert percentiles into
stanines.
 - Usually used in achievement tests ➢ Determine how many cases
(elementary and secondary are in the group.
schools). ➢ Divide the number of cases
- 5th stanine = performance in the below the score of interest
average range from ¼ SD below the (step 1) by the total number
mean to ¼ SD above the mean, and of cases in the group (step
captures the middle 20% of the 2).
scores in a normal distribution. ➢ Multiply the result of step 3
- 4th and 6th stanines are also ½ SD by 100.
wide and capture the 17% of cases • Percentiles
below and above (respectively) the - Are the specific scores or points
5th stanine. within a distribution.
- Total frequency for a set of
observations divided into
hundredths.
- Indicates the particular score, below
which a defined percentage of score
falls.
Percentages Percentiles Stanines
of Cases
4 1-4 1 Top 4 percent
7 5-11 2
Standardized measures of rank 12 12-23 3
17 24-40 4
• Quartiles – points that divide the
20 41-60 5
frequency distribution into equal 4ths.
17 61-77 6
- 1st quartile = 25th percentile.
12 78-89 7
- 2nd quartile = median/ 50th 7 90-96 8
percentile. 4 97-100 9 Bottom 4
- 3rd quartile = 75th percentile. percent
• Deciles – use points that mark 10% Standard Score for SAT and GRE
rather than 25% intervals.
- Top decile (D9) = point below - Scholastic Aptitude Test (SAT)
which 90% of the cases fall. - Graduate Record Examination (GRE)
- D8 = 80th percentile ➢ Mean: 500, SD: 100
• Percentile ranks Deviation IQ
- Replace simple ranks when we want
to adjust for the number of scores - Mean: 100, SD: 15
in a group. - Typical mean and SD for IQ tests
- To calculate: results in approximately 95% (SD
➢ Determine how many cases ±2) of deviation IQs ranging from 70
fall below the score of to 130.
interest. Commented [AD2]: When you standardized your normally
Linear transformation distributed data most probability your raw scores’s mean and
deviations will parallel the standard scores. (direct numerical
relationship) This is now called as Linear transformation.
- Standard score obtained by linear
transformation is one that retains a direct
numerical relationship to the original raw
score.
Nonlinear transformation
- May be required when the data under
consideration are not normally distributed
yet comparisons with normal distribution
need to be made.
Normalized standard scores
- Method used when test developers have a
skewed distribution regardless of having
large samples and aims to have a normal
distribution.
- Involves “stretching” the skewed curve into
the shape of normal curve and creating a
corresponding scale of standard scores.
- Formula: Y=aX+b.
Concepts (2)
• Norm-referenced test
- Compares each person with a norm.
These tests do not compare
students with one another; they
compare each student’s
performance with a criterion or
expected level of performance.
• Criterion-referenced test
- Describes the specific types of skills,
tasks, or knowledge that the test
taker can demonstrate such as
mathematical skills.
- Criterion-referenced testing
Commented [AD3]: When you standardized your not
movement emphasizes the
normally distributed data, you will have a nonlinear
diagnostic use of tests – that is, transformation. Wherein your raw scores’s mean and
using them to identify problems deviation does not parallel with your standardized scores.
that can be remedied. There is no direct numerical relationship.
CORRELATION AND REGRESSION
Correlation coefficient – mathematical index
that describes the direction and magnitude of a
relationship.
• Positive correlation
- High scores on Y are associated with
high scores on X, and low scores on
Y correspond to low scores on X.
- When Y increases/decreases, X also
increases/decreases.
• Negative correlation
- Higher scores on Y are associated
with lower scores on X, and lower
scores on Y are associated with
higher scores on X.
- When Y increases, X decreases.
- When Y decreases, X increases.
• No correlation
- Variables are no related.
Regression – related technique (to correlation).
Used to make predictions about scores on one
variable from knowledge of scores on another
variable.
- Regression line – where the
predictions are obtained from.
 ➢ Defined as the bestfitting
straight line through a set of
points in a scatter diagram.
➢ It is found by using the
principle of least squares,
which minimized the
squared deviation around
the regression line.
▪ Mean – point of
least squares for
any single variable.
This means that the
sum of the squared
deviations around
the mean will be
less than it is
around any value
other than the
mean.
Pearson product moment correlation
coefficient
- Ratio used to determine the degree
of variation in one variable that can
be estimated from knowledge
about variation in the other
variable.
Terms and Issues in the Use of Correlation
- The correlation coefficient can take
on any value from -1.0 to 1.0.
- Commonly used; most often we
want to find the correlation
between two continuous variables.
(height, weight, intelligence.)
- PARAMETRIC MEASURE (PEARSON
R)
• Spearman’s rho
- Use for finding the association
between two sets of ranks.
- Easy to calculate and is often used
when the individual in a sample can
be ranked on two variables but their
actual scores are not known or have
a normal distribution.
- Popular method for correlating
unvalidated survey, instruments or
Likert-type survey responses.
- NON-PARAMETRIC MEASURE.
• Biserial correlation
- Relationship between a continuous
variable and an artificial Commented [AD5]: Variables that can are typically
dichotomous variable. obtained by measuring. Variables that can assume any value
within a special range. Ex. Ratio and interval variables.
- For example: passing or failing the
(Weight & Temperature)
bar examination (artificial
dichotomous variable) and grade
point average in law school
(continuous variable)
• Point biserial correlation
- Continuous variable and true
dichotomous variable (gender).
- For example: relationship between
gender and GPA.
• Phi coefficient
- Both variables are dichotomous
variables and at least one is “true”.
- For example: relationship between
passing or failing the bar
examination and gender. Commented [AD4]: Main difference between Spearman’s
• Tetrachoric correlation rho and Pearson r
- Both dichotomous variables are Spearman’s rho
artificial. Non-parametric measure.
Evaluates monotonic relationship (one V decreases as the
other V increases or one V increases as the other V
decreases.) [it can be positive (linear correlation) or
• Residual negative correlation]
Evaluates the strength and direction of the correlation.
- The difference between the
-1 (perfect negative correlation), 1 (perfect positive
predicted and observed values. correlation)
- Y-Y’ (R= actual y value – predicted y
Pearson r
value)
Parametric measure
➢ Regression equation gives a Assess linear correlation.
predicted value of Y for Assess the strength and direction of the correlation.
1 (positive correlation), -1 (negative linear correlation), 0
each value of X.
(no correlation)
 ➢ In addition to these
predicted values, there are
observed values of Y.
- Interpretation: Having a negative
residual means that the predicted
value is too high, similarly if you
have a positive residual, it means
that the predicted value was too
low.
• Standard error of estimate
- Measure of the accuracy of
prediction.
- Prediction is most accurate when
the standard of error of estimate is
relatively small.
- As it become larger, the prediction
becomes less accurate.
• Coefficient of determination
- Correlation coefficient squared.
- This value tells us the proportion of
the total variation in scores on Y
that we know as a function of
information about X.
- For example, correlation between
SAT score and performance of 1st
year college is .40, then the
coefficient of determination is .16.
➢ .40² = .16.
➢ This means that we can
explain 16% of the
variation in 1st year college
performance by knowing
SAT scores.
• Coefficient of alienation
- Measure of non-association
between two variables.
- Computation: subtracting
coefficient of determination to 1.
√1 − 𝑟
2 dependent and independent
- This is the proportion of common
variance not shared between the
variables, the unexplained variance
between the variables.
- A high coefficient of alienation
indicates that the two variables
share very little variance in
common. (non-association)
• Shrinkage
- Amount of decrease observed
when a regression equation is
created for one population and then
applied to another.
- one problem with regression
analysis is that it takes advantage of
chance relationships within a
particular sample of subjects.
➢ There is a tendency to
overestimate the
relationship, particularly if
the sample of subjects is
small.
• Cross validation
- Best way to ensure that proper
references are being made.
➢ Use the regression equation
to predict performance in a
group of subjects other
than to which the equation
was applied.
➢ Then a standard error of
estimate can be obtained
for the relationship
between the values
predicted by the equation
and the values actually
observed.
• Multivariate analysis
- Broad term that refers to the
analysis of multiple variables
simultaneously.
- Multiple variables, can include both
variables.
- Considers the relationship among
combinations of three or more
variables.
 - For example: the prediction of
success in the 1st year of college
from the linear combination of SAT
verbal and math scores.
• Discriminant analysis (multivariate
method)
- When the task is to find the linear
combination of variables that
provides a maximum discrimination
between categories.
- For example: attempts to determine
whether set of measures predicts
success or failure on a particular
performance evaluation.
• Multiple regression analysis
(Multivariate method)
- Specifically deals with multiple
independent variables influencing
a single dependent variable. It
quantifies the extent to which each
independent variable contributes to
the variability in the dependent
variable.
Discriminant analysis and multiple
Standard error of the difference
regression analysis find linear
combinations of variables that
maximize the prediction of some
criterion.
Factor analysis is used to study the
interrelationships among a set of
variables without reference to a
criterion.
Dependent Variable: Also known as the
"criterion" or "outcome" variable.
Independent Variable: Also known as
the "predictor" or "explanatory"
variable.
Standard error of measurement/ standard
error of score
- Measure of the precision of an
observed test score.
- Estimate the extent to which an
observed test score deviates from a
true score.
- An INDEX of the extent to which
one individual’s scores vary over Commented [AD6]: For example:
tests presumed to be parallel. Test battery includes: personality and intelligence test. (IV;
predictor)
- Formula:
DV: successful and non-successful.
➢ SD divided by 1 minus the
reliability coefficient. If there is a small difference between the scores of successful
group and non-successful group in intelligence test, then
- SEM functions like a SD = predict
intelligence would have a discriminant evidence. This is
what would happen if an individual because the IQ of the non-successful and successful group
took additional equivalent test does not significantly differs with each other, thus IQ does
not predict if an individual will become successful or not in
➢ 68% = +- 1 SD
their job.
➢ 95% = +- 2 SD
➢ 99% = +-3 SD
➢ If SD is constant = the
smaller the SEM,
correlation coefficient
increases; more reliable.
- useful in establishing what is called
a confidence interval: a range or
band of test scores that is likely to
contain the true score.
- aid a test user in determining how
large a difference should be before
it is considered statistically
significant.
- Address this question
➢ How did this individual’s
performance on test 1
compare with his or her
performance on test 2?
➢ How did this individual’s
performance on test 1
compare with someone
else’s performance on test
1?
➢ How did this individual’s
performance on test 1
compare with someone
else’s performance on test
2?
 - Formula:
➢ Square root of the sum of
SEM of first test and SEM
of second test.
➢ Or
➢ square root of the 2 minus
reliability coefficient of 1st
test minus the reliability
coefficient of 2nd test.
Reliability
• Reliability coefficient
- the ratio of the variance of the true
scores on a test to the variance of
the observed scores.
• Test reliability
- Usually estimated in one of three
ways.
1. Test-retest method
➢ Consistency: when
administered on different
occasions.
2. Parallel forms method
➢ We evaluate the test across
different forms of the test.
3. Internal consistency method
➢ We examine how people
perform on similar subsets
of items selected from the
same form of the measure.
• Test-retest reliability
- Estimates are used to evaluate the
error associated with administering
a test at two different times.
- This type of analysis is of value only
when we measure traits or
characteristics that do not change
over time.
- Test that measures some constantly
changing characteristics are not
appropriate for test-retest
evaluation (Rorschach Inkblot Test).
• Carryover effect
- Occurs when the 1st session
influences scores from the 2nd
session.
- When there are carryover effects,
the test-retest correlation usually
overestimates the true reliability.
• Practice effects
- Type of carryover effects.
- Some skills improve with practice.
• Parallel forms reliability
- Compares two equivalent forms of a
test that measure the same
attribute.
- Use different items but have the
same level of difficulty.
• Split-half reliability
- Estimates the internal consistency.
- Test is given and divided into halves
that are scored separately.
- Results of one half is compared with
the results of the other.
- If the items get progressively more
difficult, then you might be better
advised to use the odd-even
system.
Test scores gain reliability as the number of
items increases.
An estimate of reliability based on two half-tests
would be deflated because each half would be
less reliable than the whole test. Commented [AD7]: Reliability of the WHOLE TEST is much
higher than the two halves, since whole test has more items.
The correlation between the two halves of the
test would be a reasonable estimate of the
reliability of half test.
• Spearman-Brown (Correction) Formula
- To correct for half-length test.
- Allows you to estimate what the
correlation between the 2 halves
would have been if each half had
been the length of the whole test.
 - Estimate: Correlation between 2
halves = whole test.
- Using it is not always advisable.
• Cronbach’s coefficient alpha (a) [1951]
- When the two halves of a test have
unequal variances.
- This general reliability coefficient
provides the lowest estimate of
reliability that one can expect.
- Estimates the internal consistency.
➢ Items are not scored as 0 or
1 (right or wrong).
➢ More general method off
finding reliability estimate
through internal
consistency.
➢ N = number of items.
➢ S = sum of the total scores
for all items.
➢ Si = the sum of the item
scores for the each item.
- Measure of agreement between 2
judges who each rate a set of
Commented [AD8]: used when two halves of the test
have unequal variances and on tests containing non-
dichotomous items, unequal variances
Commented [AD9]: For the researchers to know if a
certain instrument is reliable enough or not. It is important
to know how much an instrument is NOT reliable for the
objects using NOMINAL SCALES. researchers to do various methods to increase the reliability.
- Best method for assessing the level Much better to know how UNRELIABLE THE INSTRUMENT IS
of agreement among several rather than OVERESTIMATE ITS RELIABILITY.
observers.
- Values:
➢ 1 (perfect agreement)
➢ -1 (less agreement that can
be expected on the basis of
of chance alone.)
➢ A value greater than .75
generally indicates
“excellent” agreement.
➢ A value between .40 and
.75 indicates “fair to good”
(satisfactory) agreement.
➢ A value less than .40
indicates “poor”
agreement.

Sources of measurement error and methods of

reliability assessment

Commented [AD10]: Add:

• Kuder-Richardson 20 (KR20 / KR-20)
- Estimating the internal consistency.
- Calculating the reliability of a test
when Items are dichotomous.
- Scored 0 or 1 (usually for right or
wrong) – not Likert’s scale.
• Kappa Statistic (intro; J.Cohen, 1960).
It has been suggested that reliability estimates
in the range of .70 and .80 are good enough for
most purposes in basic research. For a test used
KR-20: used for inter-item consistency of
dichotomous items (intelligence tests, personality tests
with yes or no options, multiple choice), unequal
variances, dichotomous scored
KR-21: if all the items have the same degree of
difficulty (speed tests), equal variances, dichotomous
Scored
to make a decision that affects some person’s
future, evaluators should attempt to find a test
with a reliability greater than .95 (medical
researchers)
• Spearman-Brown prophecy formula
- can estimate how many items will
have to be added in order to bring a
test to an acceptable level of reliability.
To ensure that the items measure the same
things; (methods to do to deal with low
reliability)
• Factor Analysis
- Tests are most reliable if they are
unidimensional.
➢ One factor should account
for considerably more of
the variance than any other
factor. Items that do not
load on this factor might be
best omitted.
• Discriminability Analysis
- Examine the correlation between
each item and the total score for
the test.
- Correlation between the score on a
single item and the total score is
low, the item is probably measure
something else.
➢ Too easy or hard that
people do not differ in their
response.
➢ It drags down the estimate
of reliability and should be
excluded.
• Correction for attenuation
- Estimating what the correlation
between tests would have been if
there had been no measurement
error.
Inter-item reliability (Internal consistency)
- Error: Item Sampling Homogeneity
- used when tests are administered
once
- consistency among items within the
test
- measures the internal consistency
of the test which is the degree to
Commented [AD11]: when it measures a single
which each item measures the
underlying construct or factor. Factors should be explaining a
same construct single factor. Example: OCEAN (big 5) explains the personality
- measurement for unstable traits or is CORRELATED TO PERSONALITY.
- if all items measure the same
construct, then it has a good
internal consistency
- useful in assessing Homogeneity
- Homogeneity: if a test contains
items that measure a single trait
(unifactorial)
- Heterogeneity: degree to which a
test measures different factors
(more than one factor/trait)
- more homogenous = higher inter-
item consistency
Validity
- The agreement between a test score or
measure and the quality it is believed to
measure.
- Sometimes defined as the answer to the
question, “does the test measure what it is
supposed to measure?”
• Face validity
- Not recognized as a legitimate category
because it is not technically a form of
validity.
- Mere appearance that a measure has
validity.
- If the items seem to be reasonably
related to the perceived purpose of the test.
- Not a validity at all because it does not
offer evidence to support conclusions drawn
from test scores.
• Content-related evidence validity
- Consider the adequacy of
representation of the conceptual
domain the test is designed to cover.
- Consider how the content is related or to
what extent does the content represent
the variable that the test is designed to
measure.
- Validity evidence that is something not
separate from other types; boundaries
between content and other types of
evidence for validity is not clearly
defined.
- Unique features: Logical rather than
statistical, other than Face validity.
• Construct underrepresentation
- Describes the failure to capture
important components of a construct.
• Construct-irrelevant variance
- Occurs when scores are influenced by
factors irrelevant to the construct.
• Criterion validity evidence
- Tells us just how well a test corresponds
with a particular criterion.
- Criterion is the standard against which the
test is compared.
• Predictive validity evidence
- Form of criterion validity evidence that has
a forecasting function of tests.
• Concurrent related evidence for validity
- Another type of evidence for validity.
- Comes from assessments of the
simultaneous relationship between the test
and the criterion.
The Strong-Campbell Interest Inventory (SCII)
uses as criteria patterns of interest among
people who are satisfied with their careers
(Campbell, 1977). Then the patterns of interest
for people taking the tests before they have
chosen an occupation are matched to patterns
of interest among people who are happy in
various occupations.
- It has a concurrent validity.
- If the pattern of interest of those people
who have not yet chosen an occupation did
not match the pattern of interest of the
people who are satisfied with their career,
then we can assume why they have not yet
chosen an occupation. It could be a
problem with pattern interest that is why
they have not yet decided on what career
to pursue.
Validity coefficient
- The relationship between a test and a
criterion.
- This coefficient tells the extent to which the Commented [AD12]: For example, if a test of
test is valid for making statements about mathematical knowledge included algebra but not geometry,
the validity of the test would be threatened by construct
the criterion.
underrepresentation.
- In practice, one rarely sees a validity
coefficient larger than .60, and validity Commented [AD13]: For example, a test of intelligence
might be influenced by reading comprehension, test anxiety,
coefficients in the range of .30 to .40 are or illness.
commonly considered high.
Commented [AD14]: For example, the SAT serves as
• Cross validation (study) predictive validity evidence as a college admissions test if it
- Good validity study accurately forecasts how well high-school students will do in
- Assesses how well the test actually forecasts their college studies. The SAT, including its quantitative and
verbal subtests, is the predictor variable, and the college
performance for an independent group of grade point average (GPA) is the criterion. The purpose of the
subjects. test is to predict the likelihood of succeeding on the
- The initial validity study assesses the criterion—that is, achieving a high GPA in college
relationship between the test and the Commented [AD15]: Example: between a learning
criterion, whereas the cross validation study disability test and school performance.
checks how well this relationship holds for measures and criterion measures are taken at the same time
an independent group of subjects. because the test is designed to explain why the person is
now having difficulty in school.
- The larger the sample size in the initial evidence for what the test does not
study, the better the likelihood that the measure.
relationship will cross validate.
• Generalizability Writing and Evaluating Test Items
- Evidence that the findings obtained in one Item Formats
situation can be generalized or applied to
• Dichotomous format
other situation.
- Offers two alternatives for each item.
• Construct - Ex. true-false examination.
- Defined as something built by mental • Polytomous format (polychotomous)
synthesis. - More than two alternatives Commented [AD16]: the process of combining ideas into
- As a construct, intelligence does not exist a congruous object of thought.
- Ex. multiple choice examination.
as a separate thing we can touch or feel, so
- Incorrect choices – distractors.
it cannot be used as an objective criterion. - Correction for guessing:
• Construct validity evidence Commented [AD17]: Here you define the construct, since
- Established through a series of activities in there is no existing universal define for it, then develops an
instrument to measure the validity of the definition you have
which a researcher simultaneously defines given to the construct.
some construct and develops the
instrumentation to measure it.
- This process is required when “no criterion
or universe of content is accepted as
• Likert format
entirely adequate to define the quality to be
- Respondent are required to indicate the
measured.”
degree of agreement with a particular
• Convergent evidence validity attitudinal question.
- When a measure correlates well with other ➢ Popular format for attitude and
tests believed to measure the same personality scales.
construct.
• Category format
- This sort of evidence shows that measures
- Similar to the Likert format but that uses an
of the same construct, converge, or narrow
even greater number of choice.
in, on the same thing.
- 1-10
• Discriminant evidence (Divergent
• Visual analogue scale
validation)
- Uniqueness from other tests/studies.
- If a health index measures the same thing
that self-ratings of health, symptoms, and
chronic medical conditions measures, then
why do we need it in addition to all these
other measures?
- The answer is that the index taps
something other than the tests used in the
convergent evidence studies.
- To demonstrate discriminant evidence for
validity, a test should have low correlations
with measures of unrelated constructs, or
Wechsler adult intelligence scale- IV (WAIS IV)
- widely used intelligence test designed to
assess cognitive abilities in adults aged 16 to
90+. It measures various cognitive domains,
including verbal comprehension, perceptual
reasoning, working memory, and processing
speed.
Use: The WAIS-IV is used for a variety of
purposes, including clinical assessment and
diagnosis of intellectual disabilities,
neuropsychological evaluation, educational
planning, vocational assessment, and research.
(ADAPTIVE TEST)
Wechsler intelligence scale for children - IV
(WISC-IV)
- The WISC-IV is a widely used intelligence test
designed to assess cognitive abilities in children
aged 6 to 16 years old. It measures various
cognitive domains, including verbal
comprehension, perceptual reasoning, working
memory, and processing speed.
Use: The WISC-IV is used for a variety of
purposes, including educational assessment,
clinical evaluation of intellectual abilities,
identification of learning disabilities or
developmental disorders, and research.
SB-5 vs WISC-IV
1. both test published in 2003, individual
administration, 1 hour, full scale IQ composite
score based on 10 subtests.
2. WISC-IV, have 5 supplemental tests (30mins
administration), SB-5 have none.
3. SB5, have short forms (with 2 subtest), WISC-
IV have none.
4. both test contain child-friendly materials
5. both have optional available softwate for
scoring and reporting writing
6. norming sample for testtakers ages 6-16 was
2,200 for both tests
7. WISC-IV included parent education, sb5 did
not.
8. sb5 included socioeconomic status, wisc-iv
did not.
9. both fans of the CHC model of intelligence
wechsler preschool and primary scale of
intelligence-iii (WPPSI-III)
- Age Range: The WPPSI-III is designed to assess
the intelligence and cognitive abilities of
children aged 2 years 6 months to 7 years 3
months.
Purpose: The primary purpose of the WPPSI-III
is to assess cognitive functioning in young
children, including their intellectual abilities,
strengths, and weaknesses.
Use: The WPPSI-III is used for a variety of
purposes, including educational assessment,
clinical evaluation of intellectual abilities,
identification of learning disabilities or
developmental disorders, and research.
Parent and Teacher Rating Scales: In addition to
the direct assessment of the child's cognitive
abilities, the WPPSI-III includes parent and
teacher rating scales to gather information
about the child's behavior, social skills, and
adaptive functioning in everyday settings.

Adaptive Behavior Assessment: The WPPSI-III

includes an optional measure of adaptive
behavior, which assesses the child's ability to
function independently in daily activities such as
self-care, communication, and social interaction.
This provides a more comprehensive
understanding of the child's overall functioning

wechsler abbreviated scale of intelligence

- short-form intelligence test designed for

individuals aged 6 to 89 years old.

Purpose: The WASI is designed to provide a

quick and reliable estimate of an individual's
intellectual abilities. It is often used in clinical
and research settings when time constraints or
other factors make administration of a full-scale
intelligence test impractical.

Administration: The WASI can be administered

individually or in a group setting by trained
professionals, such as psychologists or
educational specialists. Administration typically
takes 30 to 45 minutes to complete.