11/17/2020

Basic Assumptions About

Assessment
1. Psychological traits and states exist
2. Psychological traits and states can be quantified
Chapter 4 and measured
3. Test related behavior predicts non-test related
Of Tests and Testing behavior
4. Tests and other measurement techniques have
strengths and weaknesses

McGraw-Hill/Irwin © 2013 McGraw-Hill Companies. All Rights Reserved. McGraw-Hill/Irwin © 2013 McGraw-Hill Companies. All Rights Reserved.

Basic Assumptions About Basic Assumptions About

Psychological Testing
Assessment
1. Psychological Traits and States
Exist
5. Various sources error are part of the assessment
process
6. Testing and assessment can be conducted in a • A trait has been defined as “any
fair and unbiased manner distinguishable, relatively stable
7. Testing and assessment benefit society variables in which one individual
varies from another” (Guilford, 1959, p.
6).

• States also distinguish one person

from another but are relatively less
McGraw-Hill/Irwin © 2013 McGraw-Hill Companies. All Rights Reserved. enduring (Chaplin et al., 1988). 4-4

Basic Assumptions About Basic Assumptions About

Psychological Testing Psychological Testing
1. Psychological Traits and States Exist
2. Traits and States Can Be Quantified and
Measured
• Psychological traits exist as constructs - an
informed, scientific concept developed to describe • Different test developers may define and measure
or explain behavior. constructs in different ways.

• Once a construct is defined, test developers turn to

item content and item weighting.

4-5 4-6

1
 11/17/2020

Basic Assumptions About Basic Assumptions About

Psychological Testing Psychological Testing
2. Traits and States Can Be Quantified and Measured
3. Test-Related Behavior Predicts Non-Test-Related
A scoring system and a way to interpret results need to behavior
be devised. • Responses on tests are thought to predict real-
– aggressive behavior world behavior.
• first define the construct according to the purpose • The obtained sample of behavior is expected to
• then operationally define test items predict future behavior.
– Physical harm to others, pushing, hitting, – true-false questions on personality test predict
kicking in play ground mental disorders.

4-7 4-8

Basic Assumptions About Basic Assumptions About

Psychological Testing Psychological Testing

4. Tests Have Strengths and Weaknesses 5. Various Sources of Error are Part of Assessment

• Competent test users understand and appreciate Error refers to factors other than what a test
the limitations of the tests they use. attempts to measure

• They know how those limitations might be

compensated for by data from other sources. Error variance - the component of a test score
attributable to sources other than the trait or
ability measured.

4-9 4-10

Basic Assumptions About Basic Assumptions About

Psychological Testing Psychological Testing

5. Various Sources of Error are Part of Assessment
• Both the assessee, assessor and the instrument are
sources of error variance
6. Testing and Assessment can be Conducted in a Fair
Manner
• major test publishers strive to develop instruments
that are fair

• should be used in strict accordance with guidelines in

the test manual

4-11 4-12

2
 11/17/2020

Basic Assumptions About Basic Assumptions About

Psychological Testing Psychological Testing

6. Testing and Assessment can be Conducted in a 7. Testing and Assessment Benefit Society
Fair Manner
• Problems arise • Considering the many areas of our lives that they
– if the test is used with people for whom it was not benefit, there is a great need for tests
intended. – Diagnosing educational difficulties
– Diagnosing neuropsychological impairments
– Hiring based on merit

4-13 4-14

What is a “Good Test?” What’s a “Good Test?”

Criteria for a good test include:
• Reliability: The consistency of the measuring tool.
The precision with which the test measures and the
extent to which error is present in measurements.
Other Considerations
• Administration, scoring, interpretation should be
straightforward for trained examiners.

• Validity: The test measures what it purports (intends) • A good test is a useful test that will ultimately
to measure. benefit individual test-takers or society at large.

4-15 4-16

Norms Norms
• Norms are the test performance data of a
• Guide smooth and peaceful interactions by defining
particular group of test-takers that are designed
predictable behavior in different situations.
for use as a reference when evaluating or
interpreting individual test scores.
• Norms tend to be institutionalized and
internalized.
• A normative sample is the reference group to
which test-takers are compared.

4-17 4-18

3
 11/17/2020

Norms Sampling to Develop Norms

• The meaning of an individual test score is
understood relative to other scores on the same
Standardization - The process of administering a test
test.
to a representative sample of testtakers for the
purpose of establishing norms.
• Norm-referenced testing and assessment is
evaluating an individual testtaker’s score and
Sampling - Test developers select a population, for
comparing it to scores of a group of testtakers.
which the test is intended, that has at least one
common, observable characteristic.

4-19 4-20

Sampling to Develop Norms Stratified Random Sampling

• The most common strata used in stratified random
Stratified sampling - Sampling that includes sampling are age, gender, socioeconomic status, religion,
different subgroups, or strata, from the population. nationality

Stratified-random sampling- Every member of the
population has an equal opportunity of being
included in a sample.
• Best used with a sample population easily broken into
distinct subgroups.
• Samples are then randomly taken from each subgroup
based on the ratio of the subgroup’s size to the total data
population.

4-21 4-22

Strata Size Calculation Strata Size Calculation

• In general the size of the sample in each stratum is taken in • The first step is to calculate the percentage in each group
proportion to the size of the stratum. based on the total number of staff (180).

• Suppose that in a company there are the following staff: • male, full-time = 90 ÷ 180 = 50 %
• male, full-time : 90 • male, part-time = 18 ÷ 180 = 10 %
• male, part-time : 18 • female, full-time = 9 ÷ 180 = 5%
• female, full-time : 9 • female, part-time = 63 ÷ 180 = 35%
• female, part-time : 63
• Total : 180 This tells us that of our sample of 40,
• 50 % of 40 is 20 (male, full-time)
take a sample of 40 staff, stratified according to the • 10 % of 40 is 4 (male, part-time)
above categories. • 5 % of 40 is 2 (female, full- time)
• 35% of 40 is 14 (female, part-time)

4-23 4-24

4
 11/17/2020

Stratified Random Sampling

Sampling to Develop Norms
• For example, assume a total data population of 1000, broken
into four subgroups with data populations as follows:
» A: 450 Purposive sample - Arbitrarily selecting a sample that
» B: 250 is believed to be representative of the population.
» C: 200 – The manufacturer test a new product in a city to make
» D: 100 assumptions about how that product would sell nationally

• To perform stratified random sampling with 200 pieces of Incidental/convenience sample - A sample that is
the data, convenient or available for use which may not be
• 45% of the sample must come from A, 90
representative of the population.
• 25% must come from B, 50
• 20% must come from C 40
• 10% must come from D. 20
• Total 200
4-25 4-26

Sampling to Develop Norms

Incidental/convenience sample
• Generalization of findings from convenience
samples must be made with caution.
– student volunteers as subjects for the research
– subjects that are selected from a clinic
– choosing five people from a class
– choosing the first five names from the list of patients
Sampling to Develop Norms
Developing Norms
• Administer the test with standard set of instructions
• Recommend a setting for test administration
• Collect and analyze data
• Summarize data using descriptive statistics
• Provide a detailed information to support
recommended interpretations of the results

4-27 4-28

Types of Norms Types of Norms

Percentile - the percentage of people whose score on a • Percentiles are a popular method for organizing test-
test or measure falls below a particular raw score. related data because they are easily calculated.

• The ranking is out of a hundred and the percentile
reports where the person would fall if ranked against
100 others.
– 30th percentile they are as good as 30 out of 100
– 70th percentile they are better than 70 out of 100
• The problem is that real differences between raw
scores may be minimized near the ends of the
distribution and exaggerated in the middle of the
distribution.

4-29 4-30

5
 11/17/2020

Types of Norms BMI = weight (kg) / height (m)²

Age norms- average performance of different

samples of test-takers who were at various ages when
the test was administered.

Grade norms- the average test performance of

testtakers in a given school grade.

4-31 4-32

Types of Norms

National norms- derived from a normative sample

that was nationally representative of the population
at the time the norming study was conducted.

National anchor norms- an equivalency table for

scores on two different tests that allows for a basis
of comparison.

4-33 4-34

Types of Norms Types of Norms

Local norms- provide normative information with
Subgroup norms- a normative sample can be
respect to the local population’s performance on
segmented by any of the criteria initially used in
some test.
selecting subjects for the sample.
– developing your own company norms for selection
– regional norms might find as most useful by a
– developing your own school norms for a kind of test
community school board
– handedness norm might be most useful to study brain
lateralization

4-35 4-36

6
 11/17/2020

Norm-Referenced versus Criterion-

Fixed Reference Group Scoring Systems Referenced Interpretation
The distribution of scores obtained on the test from
• Norm referenced tests involve comparing individuals
one group of testtakers is used as the basis for the to the normative group.
calculation of test scores for future administrations
of the test.
• With criterion referenced tests testtakers are evaluated
–The SAT employs this method.
as to whether they meet a set standard (e.g. a driving
–2 million testtakers who completed the SAT in 1990
exam).
began to be used as fixed reference group in 1995.
–If the test is used in 2008 was somewhat easier than
previous tests, the scores of the testtakers should be
calibrated downward.

4-37 4-38

Norm-Referenced versus Criterion-

Referenced Interpretation
Exercise
• These terms are not mutually exclusive.
For licensing of psychologists would you
• In a sense, all testing is ultimately normative, even if
advocate to use criterion or norm-referenced the scores are seemingly criterion-referenced as pass-
assessment? Why? fail.

4-39 4-40

Culture and Inference

• In selecting a test for use, test users should
research the test’s available norms to check how
appropriate they are for use with the targeted
testtaker population.

• It is important to conduct culturally informed

assessment which takes into consideration the
culture and era of the test-taker.

4-41