VALIDITY, RELIABILITY & ETHICAL CONSIDERATIONS

Objectives
By the end of the lesson, you will be able to

 Distinguish between reliability and validity

 List the major types of evidence used to support the valid

interpretation of test scores.

 Distinguish random and systematic errors of measurements

 Be aware of ethical considerations in conducting research.

Reliability vs Validity
Accuracy of a measure

49 kg

60 kg
 7.00 AM
9am
Consistency of a
55 kg measure

10am

60 kg
11am

53 kg
 7.00 AM
9am
Consistency of a
60 kg measure

10am

60 kg
11am

60 kg
 7.00 AM
9am
Consistency of a
49 kg measure

10am

49 kg
11am

49 kg
Reliability Validity

How accurately a method measures what it is intended

to measure

How consistently a method measures something

 Reliability Validity

The extent to which the results

The extent to which the results
What does it can be reproduced when the
really measure what they are
tell you? research is repeated under the
supposed to measure.
same conditions.

By checking the consistency of By checking how well the results

How is it results across time, across correspond to established
assessed? different observers, and across theories and other measures of
parts of the test itself. the same concept.

A reliable measurement is not A valid measurement is

How do they always valid: the results might generally reliable: if a test
relate? be reproducible, but they’re not produces accurate results, they
necessarily correct. should be reproducible.
Reliability vs validity

“A measuring instrument can be reliable without being

valid, but it cannot be valid unless it is first reliable”
(Ary et al, 2010, p. 239)
 7.00 AM
9am
Consistency of a
55 kg measure

10am

60 kg
11am

53 kg
 7.00 AM
9am
Consistency of a
60 kg measure

10am

60 kg
11am

60 kg
 7.00 AM
9am
Consistency of a
49 kg measure

10am

49 kg
11am

49 kg
Types of validity
Construct validity → overall ability to measure the construct
(phenomenon) of interest

Internal validity → accuracy/fairness

External validity → generalizable to wider population (beyond

your study)
 Example:
RQ: “Do boys have more motivation to learn online than girls?”

1. How will we know that our results are due to differences in

motivation rather than something else?

2. Are our results valid only for our sample or are they
generalizable beyond our sample?
Internal validity vs external validity
Internal validity:

 the degree to which your interpretations of your data can

be sustained by your data (Ashton, 2014, p. 54)

 the inferences about whether the changes observed in a

dependent variable are, in fact, caused by the
independent variable(s) in a particular research study rather
than by some extraneous factors (Ary et al, 2010, p. 272)
External validity:

 the degree to which the results, inferences and

conclusions of your study generalise to other situations or
populations (Ashton, 2014, p.55)

 the extent to which the findings of a study can be

generalized to other subjects, settings, and treatments
(Ary et al, 2010, p. 292)
Validation
 Validation = The process of gathering evidence to support (or fail to
support) a particular interpretation of test scores

1. Content-related evidence

2. Criterion-related evidence

3. Construct-related evidence
Content-related evidence

The degree to which the sample

Test Construct
of items, tasks, or questions on a
test are representative of some
defined universe or domain of
content.
 Criterion-related evidence

The extent to which test scores are Test Criteria

systematically related to one or more
outcome criteria.
1. Concurrent validity evidence
2. Predictive validity evidence
 Construct-related evidence

Psychological
The extent to which the test Test construct
scores reflect the theory behind
the psychological construct
being measured
Ary et al, 2010, p. 235)
 Ethical considerations
Key ethical considerations
 respect for persons;
 minimization of harm to participants, researchers, institutions and groups;
 informed and voluntary consent;
 respect for privacy and confidentiality;
 the avoidance of unnecessary deception;
 avoidance of conflict of interest;
 social and cultural sensitivity to the age, gender, culture, religion, social
class of the participants;
 justice.