Glossary
A-B design A single-subject design in which baseline
measurements are repeatedly made until stability is
presumably established, treatment is introduced, and
an appropriate number of measurements are made
during treatment.
A-B-A design A single-subject design in which base- line
measurements are repeatedly made until stability is
presumably established, treatment is introduced, an
appropriate number of measurements are made, and the
treatment phase is followed by a second baseline phase.
A-B-A-B design A single-subject design in which base-
line measurements are repeatedly made until stability
is presumably established, treatment is introduced, an
appropriate number of measurements are made, and
the treatment phase is followed by a second baseline
phase, which is followed by a second treatment phase.
abstract A summary of a study that describes the purpose,
procedures, results and implications.
accessible population The population from which the
researcher can realistically select subjects. Also called
available population. Compare target population.
accidental sampling See convenience sampling.
achievement test An instrument that measures an indi-
vidual’s current proficiency in given areas of knowl-
edge or skill.
action research Any systematic inquiry conducted by
teachers, principals, school counselors, or other stakehold-
ers in the teaching–learning environment to gather infor-
mation about the ways in which their particular schools
operate, the teachers teach, and the students learn.
additive design Any of the variations of the single-
subject A-B design that involve the addition of another
phase or phases in which the experimental treatment
is supplemented with another treatment.
affective characteristic A mental characteristic related
to emotion, such as attitude, interest, and value.
affective test An assessment designed to measure men- tal
characteristics related to emotion.
alternating treatments design A variation of multi-ple-
baseline design that involves the relatively rapid
alternation of treatments for a single participant. Also
called multiple schedule design, multi-element base-
line design, multi-element manipulation design, or
simultaneous treatment design.
alternative assessment See performance assessment.
analysis of covariance A statistical method of equat- ing
groups on one or more variables and for increas- ing the
power of a statistical test; adjusts scores on a
dependent variable for initial differences on some other
variable (e.g., pretest performance or IQ) related to
performance on the dependent variable.
analysis of narrative In narrative research, a processin
which the researcher collects stories as data and
analyzes common themes to produce a description that
applies to all the stories captured in the narratives.
Compare narrative analysis.
analysis of variance An inferential statistics technique
used to test for significant differences among the means
of three or more groups.
anonymity State of being unknown; study participants are
anonymous when their identities are hidden from the
researcher.
applied research Research conducted for the purpose
of applying, or testing, a theory to determine its useful-
ness in solving practical problems.
aptitude test A measure of potential used to predict how
well an individual is likely to perform in a future
situation.
artificial categories Categories that are operationally
defined by a researcher.
assessment General term for the process of collecting,
synthesizing, and interpreting information; also, the
instrument used for such purposes. A test is a type of
assessment.
assumption Any important fact presumed to be true butnot
actually verified; assumptions should be described in
the procedure section of a research plan or report.
attenuation The reduction in correlation coefficients that
tends to occur if the measures being correlated have low
reliability.
attitude scale A measurement instrument used to
determine what a respondent believes, perceives, or
feels about self, others, activities, institutions, or situ-
ations.
attrition See mortality.
authentic assessment See performance assessment.
available population See accessible population.
673
674 GLOSSARY
baseline measures Multiple measures of pretest per-
formance conducted in single-subject research designs
to control for threats to validity.
basic research Research conducted for the purpose of
developing or refining a theory.
bias Distortion of research data that renders the data
suspect or invalid; may occur due to characteristics of
the researcher, the respondent, or the research design
itself.
Buckley Amendment See Family Educational Rights
and Privacy Act of 1974.
case study research The in-depth investigation of one
unit (e.g., individual, group, institution, organization,
program, or document).
category A classification of ideas and concepts in quali-
tative data analysis.
causal–comparative research Research that attempts
to determine the cause, or reason, for existing differ-
ences in the behavior or status of groups of individuals.
Also called ex post facto research.
census survey Descriptive research that involves
acquiring data from every member of a population.
changing criterion design A variation of the A-B-A
design in which the baseline phase is followed by suc-
cessive treatment phases, each of which has a more
stringent criterion for acceptable or improved behavior.
chi square A nonparametric test of significance appro-
priate when the data are in the form of frequency
counts; compares proportions observed in a study with
proportions expected by chance.
clinical replication The development and application of
a treatment package, composed of two or more
interventions that have been found to be effective indi-
vidually, designed for persons with complex behavior
disorders.
closed-ended item See structured item.
cluster Any location that contains an intact group with
similar characteristics (e.g., population members).
cluster sampling Sampling in which intact groups, not
individuals, are randomly selected.
coding The process of categorically marking or ref-
erencing units of text (e.g., words, sentences, para-
graphs, and quotations) with codes and labels as a way
to indicate patterns and meaning in qualitative data.
cognitive characteristic A mental characteristic relatedto
intellect, such as mathematics achievement, literacy,
reasoning, or problem solving.
cognitive test An assessment designed to measure
intellectual processes.
common variance The variation in one variable that is
attributable to its tendency to vary with another vari-
able. Also called shared variance.
compensatory rivalry See John Henry effect.
concurrent validity The degree to which the scores on
a test are related to the scores on a similar test admin-
istered in the same time frame or to some other valid
measure available at the same time; a form of criterion-
related validity.
confidentiality Right to have information about one- self
kept private; researchers protect confidentiality when
they know the identities of study participants but do not
disclose that information.
consequential validity The extent to which an instru-
ment creates harmful effects for the user.
construct An abstraction that cannot be observeddirectly;
a concept invented to explain behavior.
construct validity The degree to which a test measures an
intended hypothetical construct or nonobservable trait
that explains behavior.
content validity The degree to which a test measures an
intended content area; it is determined by expert
judgment and requires both item validity and sampling
validity.
control Efforts on the part of a researcher to remove
the influence of any variable other than the indepen-
dent variable that may affect performance on a depen-
dent variable.
control group A group of participants in a research study
who either receive a different treatment than the
experimental group or are treated as usual.
control variable A nonmanipulated variable, usually
a physical or mental characteristic of the subjects
(e.g., IQ).
convenience sampling The process of including who-
ever happens to be available in a sample (e.g., volun-
teers). Also called accidental sampling or haphazard
sampling.
correlation A quantitative measure of the degree of
correspondence between two or more variables.
correlation coefficient A decimal number between
−1.00 and +1.00 that indicates the degree to which
two variables are related.
correlational research Research that involves collect-
ing data to determine whether, and to what degree,
a relation exists between two or more quantifiable
variables.
counterbalanced design A quasi-experimental design in
which all groups receive all treatments, each group
receives the treatments in a different order, the num- ber
of groups equals the number of treatments, and all
groups are tested after each treatment.
credibility A term used in qualitative research to indicate
that the topic was accurately identified and described.

criterion variable In a prediction study or analysis of
concurrent or predictive validity, the variable that is
predicted.
criterion-referenced scoring A scoring approach in
which an individual’s performance on an assessment is
compared to a predetermined, external standard.
criterion-related validity Validity that is determinedby
relating performance on a test to performance on
another criterion (e.g., a second test or measure);
includes concurrent and predictive validity.
critical action research A type of action research in
which the goal is liberating individuals through
knowledge gathering. Also called emancipatory action
research.
critical ethnography A highly politicized form of eth-
nography written by a researcher to advocate against
inequalities and domination of particular groups that
exist in society.
Cronbach’s alpha (a) The general formula for estimat-
ing internal consistency based on a determination of
how all items on a test relate to all other items and to the
total test. The Kuder-Richardson 20 (KR-20) is a special
case of the Cronbach’s alpha general formula.
cross-sectional survey A survey in which data are col-
lected from selected individuals in a single time period.
Compare longitudinal survey.
cross-validation Validation of a prediction equation with
at least one group other than the group on whichit was
developed; results in the removal from the equa-tion of
variables no longer found to be related to the criterion
measure.
culture The set of attitudes, values, concepts, beliefs and
practices shared by members of a group; a central
construct in ethnographic research.
curvilinear relation A relation in which an increase in
one variable is associated with a corresponding
increase in another variable to a point at which a
further increase in the first variable results in a corre-
sponding decrease in the other variable (or vice versa);
represented graphically as a curve.
data (sing. datum) Pieces of information.
data analysis An attempt by a researcher to summa- rize
data, collected for a study, in a dependable and accurate
manner. In qualitative research, data analysis usually
involves coding and finding patterns or themes in
narrative data. Compare data interpretation.
data interpretation An attempt by a researcher to find
meaning in the data collected for a study. Compare
data analysis.
data saturation A point in qualitative research at whichso
much data are collected that it is very unlikely that
additional data will add new information.
GLOSSARY 675
database A sortable, analyzable collection of units of
information maintained on a computer.
deductive hypothesis
A hypothesis derived from the- ory that provides evidence
to support, expand, or contradict the theory.
deductive reasoning Reasoning that involves develop-
ing specific predictions based on general principles,
observations, or experiences.
dependent variable The change or difference in a
behavior or characteristic that occurs as a result of the
independent or grouping variable. Also called effect,
outcome, or posttest.
descriptive research Research that determines and
describes the way things are; involves collecting numer-
ical data to test hypotheses or answer questions about
the current subject of study. Also called survey research.
descriptive statistics Data analysis techniques that enable
a researcher to describe many pieces of data
meaningfully with numerical indices.
descriptive validity The degree to which qualitative
research is factually accurate.
design A general strategy or plan for conducting a
research study; indicates the basic structure and goals of
the study.
diagnostic test A type of achievement test that yields
scores for multiple areas of achievement to facilitate
identification of a student’s weak and strong areas.
dialectic action research spiral A four-step process
for conducting action research, including identifying
an area of focus, data collection, data analysis and
interpretation, and action planning.
differential selection of participants Selection of sub-
jects who have differences at the start of a study that
may at least partially account for differences found on
a posttest; a threat to internal validity.
direct replication Replication of a study by the same
investigator, with the same subjects or with different
subjects, in a specific setting.
directional hypothesis A research hypothesis that states
the expected direction of the relation or differ- ence
between variables.
ecological validity See external validity.
educational research The formal, systematic applica-
tion of the scientific method to the study of educational
problems.
environmental variable A variable in the settingof
a study (e.g., learning materials) that may cause
unwanted differences between groups.
equivalence The degree to which two similar forms of
a test produce similar scores from a single group of
676 GLOSSARY

test takers. Also called equivalent-forms reliability or

alternate-forms reliability.

equivalent-forms reliability See equivalence.
ethnographic case study A form of ethnography that
focuses on describing the activities of a specific group
and the shared patterns of behavior it develops over
time.
ethnographic research The study of the cultural pat-
terns and perspectives of participants in their natural
setting; a form of qualitative research. Also called eth-
nography.
ethnography Ethnographic research; also, the narrative
produced to summarize the results of such research.
ethnomethodolgy A qualitative approach to studying
how participants make sense of everyday activities and
act in a social way.
evaluation research The systematic process of collect-
ing and analyzing data on the quality, effectiveness,
merit, or value of programs, products, or practices for
the purpose of making decisions about those pro- grams,
products, or practices.
evaluative validity The degree to which a qualitative
researcher is able to present data objectively, without
being evaluative or judgmental.
experimental group A group of participants in a research
study who typically receive the new treatment under
investigation.
experimental research Research in which at least one
independent variable is manipulated, other relevant
variables are controlled, and the effect on one or more
dependent variables is observed.
experimental variable See independent variable.
experimenter bias effect A situation in which a
researcher’s expectations of study results contribute to
producing the outcome.
experimenter effects Threats to the external validityof
an experiment caused by the researcher’s uninten-
tional or intentional influences on participants or on
study procedures.
external observation See nonparticipant observation.
external validity The degree to which results are gen-
eralizable or applicable to groups and environments
outside the experimental setting. Also called ecological
validity.
F ratio A computation used in analysis of variance to
determine whether variances among sample means are
significantly different.
face validity The degree to which a test appears to
measure what it claims to measure.
factorial analysis of variance A statistical technique that
allows the researcher to determine the effects of
independent or grouping variables and control vari-
ables on the dependent variable both separately and in
combination. It is the appropriate statistical analysis if
GLOSSARY 677
a study is based on a factorial design and investigates
two or more independent or grouping variables and
the interactions between them; yields a separate F ratio
for each variable and one for each interaction.
factorial design Any experimental design that involves
two or more independent or grouping variables, at least
one of which is manipulated, to study the effects of the
variables individually and in interaction with each other.
Family Educational Rights and Privacy Act of 1974
Federal law that protects the privacy of student educa-
tional records. Also called Buckley Amendment.
field notes Qualitative research material gathered,
recorded, and compiled, usually on-site, during the
course of a study.
field-oriented research See qualitative research.
fieldwork Qualitative data collection; involves spend-
ing considerable time in the setting under study,
immersing oneself in this setting, and collecting asmuch
relevant information as possible as unobtrusively as
possible.
formative evaluation Evaluation whose function is to
form and improve a program or product under devel-
opment so that weaknesses that can be remedied dur-
ing implementation.
generalizability The applicability of research findings to
settings and contexts different from the one in which
they were obtained.
grading on the curve See norm-referenced scoring.
grounded theory A qualitative approach in which the
researcher focuses on how an individually derived
theory about a phenomenon is grounded in the data
in a particular setting.
haphazard sampling See convenience sampling.
Hawthorne effect A type of reactive arrangement
resulting from the subjects’ knowledge that they are
involved in an experiment or their feelings that they
are in some way receiving special attention.
historical research A qualitative approach in which the
researcher focuses on collecting and evaluating data to
understand and interpret past events.
history Any event occurring during a study that is not
part of the experimental treatment but may affect
performance on the dependent variable; a threat to
internal validity.
hypothesis An explanation for the occurrence of cer- tain
behaviors, phenomena, or events; a prediction of
research findings.
independent variable A behavior or characteristic under
the control of the researcher and believed toinfluence
some other behavior or characteristic. Also called
experimental variable, manipulated variable, cause, or
treatment.
678 GLOSSARY
inductive hypothesis A generalization based on obser-
vation.
inductive reasoning Reasoning that involves develop-
ing generalizations based on observations of a limited
number of related events or experiences.
inferential statistics Data analysis techniques for
determining how likely it is that results obtained from
a sample or samples are the same results that would
have been obtained for the entire population.
instrument In educational research, a test or other tool
used to collect data.
instrumentation Unreliability in measuring instru-
ments that may result in invalid assessment of partici-
pants’ performance.
interaction A situation in which different values of the
independent or grouping variable are differentially
effective depending on the level of a second (e.g.,
control) variable.
interjudge reliability The consistency of two or more
independent scorers, raters, or observers.
internal validity The degree to which observed differ-
ences on the dependent variable are a direct result of
manipulation of the independent variable, not some
other variable.
interpretive research Collective, generic term for qual-
itative research approaches.
interpretive validity The degree to which a qualitative
researcher attributes the appropriate meaning to the
behavior or words of the participants in the study and
therefore captures the participants’ perspective.
interval scale A measurement scale that classifies and
ranks subjects, is based on predetermined equal inter-
vals, but does not have a true zero point.
intervening variable A variable (e.g., anxiety) that
intervenes between or alters the relation between an
independent variable and a dependent variable but that
cannot be directly observed or controlled.
interview An oral, in-person question-and-answer ses-
sion between a researcher and an individual respon-
dent; a purposeful interaction in which one person is
trying to obtain information from the other.
intrajudge reliability The consistency of one individu-
al’s scoring, rating, or observing over time.
item validity The degree to which test items are rel- evant
to the measurement of the intended content area.
John Henry effect The phenomenon in which mem- bers
of a control group who feel threatened or chal- lenged
by being in competition with an experimental group
outdo themselves and perform way beyond what
would normally be expected. Also called com-
pensatory rivalry.
judgment sampling See purposive sampling.
Kuder-Richardson 20 (KR-20) See Cronbach’s alpha.
Likert scale An instrument on which individuals respond
to a series of statements by indicating whether they
strongly agree (SA), agree (A), are undecided (U),
disagree (D), or strongly disagree (SD) with each
statement.
limitation An aspect of a study that the researcher knows
may negatively affect the results or generaliz- ability of
the results but over which the researcher has no control.
linear relation A relation in which an increase (or
decrease) in one variable is associated with a corre-
sponding increase (or decrease) in another variable;
represented graphically as a straight line.
longitudinal survey A survey in which data are col-
lected at two or more times to measure changes or
growth over time. Compare cross-sectional survey.
matching A technique for equating sample groups on
one or more variables, resulting in each member of one
group having a direct counterpart in another group.
maturation Physical, intellectual, and emotional changes
that naturally occur within subjects over a period of
time; poses a threat to internal validity because changes
may affect subjects’ performance ona measure of the
dependent variable.
mean The arithmetic average of a set of scores.
measurement The process of quantifying or scoring
performance on an assessment instrument.
measurement scale A system for organizing data so
that data may be inspected, analyzed, and interpreted.
measures of central tendency Indices that represent
the typical or average score for a group of scores.
measures of variability Indices that indicate how spread
out the scores are in a distribution.
median The midpoint in a distribution; 50% of the scores
are above the median and 50% are below.
meta-analysis A statistical approach to summarizing the
results of many quantitative studies that address
basically the same problem.
mixed methods research designs Research designs that
include both quantitative and qualitative data in a
single study.
mode The score that is attained by more subjects in a
group than any other score.
mortality A reduction in the number of research partic-
ipants that occurs over time as individuals drop out of
a study; poses a threat to internal validity because sub-
jects who drop out of a study may share a characteristic
and their absence may therefore have a significant effect
on the results of the study. Also called attrition.
multi-element baseline design See alternating treat-
ments design.

multi-element manipulation design See alternating
treatments design.
multiple comparisons Procedures used following
application of analysis of variance to determine which
means are significantly different from which other
means. Also called post hoc comparisons.
multiple prediction equation See multiple regression
equation.
multiple regression equation A prediction equationusing
two or more variables that individually predict a
criterion to make a more accurate prediction. Also
called multiple prediction equation.
multiple schedule design See alternating treatments
design.
multiple time-series design A variation of the time-
series design that involves the addition of a control
group to the basic design.
multiple-baseline design A single-subject design in
which baseline data are collected on several behaviors
for one subject, one behavior for several subjects, or one
behavior and one subject in several settings. Treat-ment
is applied systematically over a period of time to each
behavior (or each subject or setting) one at a time until
all behaviors (or subjects or settings) are under
treatment.
multiple-treatment interference Phenomenon that
occurs when carryover effects from an earlier treat-
ment make it difficult to assess the effectiveness of a
later treatment; a threat to external validity.
narrative analysis In narrative research, the develop-
ment of a narrative or story that focuses on particular
knowledge about how or why an outcome occurred.
Compare analysis of narrative.
narrative research The study of how different humans
experience the world around them; involves a meth-
odology that allows people to tell the stories of their
“storied lives.”
National Research Act of 1974 Act that led to the
establishment of federal regulations governing the pro-
tection of human subjects in research; mandates that
activities involving human participants be reviewedand
approved by an authorized group before execu- tion of
the research.
naturalistic inquiry See qualitative research.
naturalistic research See qualitative research.
negatively skewed distribution A distribution with
more extreme scores at the lower end than at the
upper, or higher, end.
nominal scale A measurement scale that classifies per-
sons or objects into two or more categories. A person
can be in only one category, and members of a cat-
egory have a common set of characteristics. Variables
GLOSSARY 679
measured on a nominal scale are called nominal or
categorical variables.
nondirectional hypothesis A research hypothesis that
states simply that a relation or difference exists between
variables.
nonequivalent control group design A quasi-
experimental design involving at least two groups, both
of which are pretested; one group receives the experi-
mental treatment, and both groups are posttested.
nonparametric test A test of significance appropriate
when the data are measured on an ordinal or nominal
scale, when a parametric assumption has been greatly
violated, or when the nature of the distribution is not
known.
nonparticipant observation Observation in which the
observer is not directly involved in the situation being
observed; that is, the observer does not intentionally
interact with or affect the object of the observation. Also
called external observation.
nonprobability sampling The process of selecting a
sample using a technique that does not permit the
researcher to specify the probability, or chance, that
each member of a population will be selected for the
sample. Also called nonrandom sampling.
nonrandom sampling See nonprobability sampling.
norm-referenced scoring A scoring approach in whichan
individual’s performance on an assessment is com-
pared to the performance of others. Also called grading
on the curve.
novelty effect The increased interest, motivation, or
participation participants develop simply because they
are doing something different; a type of reactive
arrangement.
null hypothesis A hypothesis stating that there is no
relation (or difference) between variables and that any
relation/difference found will be due to chance; i.e.,
the result of sampling error.
observer bias The phenomenon whereby an observer
does not observe objectively and accurately, thus pro-
ducing invalid observations.
observer effect The phenomenon whereby persons being
observed behave atypically simply because they are
being observed, thus producing invalid observations.
one-group pretest–posttest design A pre-experi-
mental design involving one group that is pretested,
exposed to a treatment, and posttested.
one-shot case study A pre-experimental design involv-
ing one group that is exposed to a treatment and then
posttested.
ordinal scale A measurement scale that classifies per-
sons or objects and ranks them in terms of the degree
to which they possess a characteristic of interest.
680 GLOSSARY
organismic variable A characteristic of a subject or
organism (e.g., sex) that cannot be directly controlled
but can be controlled for.
parameter A numerical index describing the behavior of
a population.
parametric test A test of significance appropriate when
the data represent an interval or ratio scale of measure-
ment and other assumptions have been met.
participant effects See reactive arrangements.
participant observation Observation in which the
observer becomes a part of and a participant in the
situation being observed.
participant variable A variable on which participants in
different groups in a study may differ (e.g., intel-
ligence).
Pearson r A measure of correlation appropriate when both
variables are expressed as continuous (i.e.,ratio or
interval) data; it takes into account every score and
produces a coefficient between -1.00 and
+1.00.
percentile rank A measure of relative position indicat-
ing the percentage of scores that fall at or below a
given score.
performance assessment A type of assessment that
emphasizes a respondent’s performance of a process
or creation of a product. Also called authentic assess-
ment or alternative assessment.
phenomenology A qualitative approach in which the
researcher focuses on capturing the experience of an
activity or concept from participants’ perspectives.
pilot study A small-scale trial of a study conductedbefore
the full-scale study to identify problems with the
research plan.
placebo effect Any beneficial effect caused by a per-
son’s expectations about a treatment rather than the
treatment itself.
population General term for the larger group from
which a sample is selected or the group to which
the researcher would like to generalize the results of
the study. Compare target population and accessible
population.
positively skewed distribution A distribution in which
there are more extreme scores at the upper, or higher,
end than at the lower end.
post hoc comparisons See multiple comparisons.
posttest-only control group design A true experimen-tal
design involving at least two randomly formed groups;
one group receives a new or unusual treat- ment, and
both groups are posttested.
power The ability of a significance test to identify a
true research finding (i.e., there’s really a difference,
and the statistical test shows a significant difference),
allowing the experimenter to reject a null hypothesis
that is false.
practical action research A type of action researchthat
emphasizes a how-to approach and has a less
philosophical bent than critical action research.
prediction study An attempt to determine which ofa
number of variables are most highly related to acriterion
variable, a complex variable to be predicted.
predictive validity The degree to which a test is able to
predict how well an individual will do in a future
situation; a form of criterion-related validity.
predictor In a prediction study or analysis of concur- rent
or predictive validity, the variable on which the
prediction is based.
pretest–posttest control group design A true experi-
mental design that involves at least two randomly
formed groups; both groups are pretested, one group
receives a new or unusual treatment, and both groups
are posttested.
pretest sensitization See testing.
pretest-treatment interaction Phenomenon that occurs
when subjects respond or react differently to a treatment
because they have been pretested; a threatto external
validity.
primary source Firsthand information, such as the tes-
timony of an eyewitness, an original document, a relic,
or a description of a study written by the person who
conducted it.
probability sampling The process of selecting a sample
using a sampling technique that permits the researcher
to specify the probability, or chance, that each member
of a defined population will be selected for the sample.
problem statement See topic statement.
projective test An instrument that includes ambigu- ous
stimuli. The test taker’s responses to the stimuliare
interpreted as projections of his or her feelings or
personality.
prospective causal–comparative research A variationof
the basic approach to causal–comparative research;
involves starting with the causes and investigating
effects.
purposive sampling The process of selecting a sample
that is believed to be representative of a given popula-
tion. Also called judgment sampling.
qualitative research The collection, analysis, and
interpretation of comprehensive narrative and visual
data to gain insights into a particular phenomenon of
interest. Sometimes called naturalistic research, natu-
ralistic inquiry, or field-oriented research.

qualitative sampling The process of selecting a small
number of individuals for a study in such a way that
the selected individuals can help the researcher under-
stand the phenomenon under investigation.
quantitative research The collection of numerical data
to explain, predict and/or control phenomena of interest.
quartile deviation One half of the difference between the
upper quartile (the 75th percentile) and the lower
quartile (the 25th percentile) in a distribution.
questionnaire A written collection of self-report ques-
tions to be answered by a selected group of research
participants.
quota sampling The process of selecting a sample based
on required, exact numbers (i.e., quotas) of per- sons of
varying characteristics.
random sampling The process of selecting individuals
for a sample on a completely chance basis.
range The difference between the highest and lowest
score in a distribution.
rating scale A measurement instrument used to deter-
mine a respondent’s attitude toward self, others, activi-
ties, institutions, or situations.
ratio scale A measurement scale that classifies subjects,
ranks them, is based on predetermined equal intervals,
and has a true zero point.
raw score The numerical calculation of the number
or point value of items answered correctly on an
assessment.
reactive arrangements Threats to the external validity of
a study associated with the way in which a study is
conducted and the feelings and attitudes of the sub- jects
involved. Also called participant effects.
realist ethnography A form of ethnography written with
an objective style and using common categories (e.g.,
“family life”) for cultural description, analysis, and
interpretation.
reasoning The process of using logical thought to reach
a conclusion.
refereed journal A journal in which articles arereviewed
by a panel of experts in the field and are thus seen as
more scholarly and trustworthy than articles from
nonrefereed or popular journals.
relationship study An attempt to gain insight into the
variables, or factors, that are related to a complex vari-
able, such as academic achievement, motivation, orself-
concept.
reliability The degree to which a test (or qualitative
research data) consistently measures whatever it mea-
sures.
replication A repetition of a study using different sub-
jects or retesting its hypothesis.
GLOSSARY 681
research The formal, systematic application of the sci-
entific method to the study of problems.
research and development (R&D) An extensive pro-
cess of researching consumer needs and then devel-
oping products specifically designed to fulfill those
needs; R&D efforts in education focus on creating
effective products for use in schools.
research hypothesis A statement of the expected rela-
tion or difference between two variables.
research plan A detailed description of a proposed study
designed to investigate a given problem.
response set The tendency of an assessed individual to
respond in a particular way to a variety of instruments,
such as when a respondent repeatedly answers as he
or she believes the researcher desires even when such
answers do not reflect the respondent’s true feelings;
also, the tendency of an observer to rate the majority of
observees the same regardless of the observees’ behavior.
restorying A process, unique to narrative research, in
which a researcher gathers stories, analyzes them for
key elements, and then synthesizes them into a coher-
ent story with a chronological sequence.
retrospective causal–comparative research The basic
approach to causal–comparative research; involves
starting with effects and investigating causes.
review of related literature The systematic identifica-
tion, location, and analysis of documents containing
information related to a research problem; also, the
written component of a research plan or report that
discusses the reviewed documents.
sample A number of individuals, items, or events selected
from a population for a study, preferably in such a way
that they represent the larger group from which they
were selected.
sample survey Research in which information about a
population is inferred based on the responses of a
sample selected from that population.
sampling The process of selecting a number of indi-
viduals (i.e., a sample) from a population, preferably in
such a way that the selected individuals represent the
larger group from which they were selected.
sampling bias Systematic sampling error; two major
sources of sampling bias are samples including only
volunteers and sampling based on available groups.
sampling error Expected, chance variation in vari- ables
that occurs when a sample is selected from a population.
sampling validity The degree to which a test samples the
total content area of interest.
Scheffé test A conservative multiple comparison tech-
nique appropriate for making any and all pairwise
comparisons involving a set of means.
682 GLOSSARY
scientific method An orderly process that entails rec-
ognition and definition of a problem, formulation of
hypotheses, collection of data, and statement of con-
clusions regarding confirmation or disconfirmation of
the hypotheses.
secondary source Secondhand information, such as a
brief description of a study written by someone other
than the person who conducted it.
selection–maturation interaction Phenomenon that
occurs when already-formed groups are included in
a study and one group profits more (or less) from
treatment or has an initial advantage (or disadvantage)
because of maturation factors; a threat to internal valid-
ity. Selection may also interact with factors such as
history and testing.
selection–treatment interaction Phenomenon that
occurs when nonrepresentative groups are included
in a study and the results of the study apply only to
the groups involved and are not representative of the
treatment effect in the population; a threat to external
validity.
self-referenced scoring approach A scoring approach in
which an individual’s repeated performances on a
single assessment are compared over time.
semantic differential scale An instrument that requiresan
individual to indicate his or her attitude about atopic
(e.g., property taxes) by selecting a position on a
continuum that ranges from one bipolar adjective (e.g.,
fair) to another (e.g., unfair).
shared variance See common variance.
shrinkage The tendency of a prediction equation to
become less accurate when used with a group other
than the one on which the equation was originally
developed.
simple analysis of variance (ANOVA) A parametric test
of significance used to test for a difference betweentwo
or more means at a selected probability level. Also
called one-way analysis of variance.
simple random sampling The process of selecting a
sample in such a way that all individuals in the defined
population have an equal and independent chance of
selection for the sample.
simultaneous replication Replication that involves a
number of subjects with the same problem, at the same
location, at the same time.
simultaneous treatment design See alternating treat-
ments design.
single-case experimental designs See single-subject
experimental designs.
single-subject experimental designs Designs applied
when the sample size is one; used to study the behav-
ior change that an individual or group exhibits as a
result of some intervention or treatment. Also called
single-case experimental designs.
single-variable design Any experimental design that
involves only one independent variable, which is
manipulated.
single-variable rule An important principle of single-
subject research that only one variable should be
manipulated at a time.
skewed distribution A nonsymmetrical distribution in
which there are more extreme scores at one end of the
distribution than the other.
Solomon four-group design A true experimental design
that involves random assignment of subjectsto one of
four groups; two groups are pretested, and two are not;
one of the pretested groups and one ofthe unpretested
groups receive the experimental treat- ment; and all
four groups are tested again.
Spearman rho A measure of correlation appropriate
when the data for at least one variable is expressed as
rank or ordinal data; it produces a coefficient between
-1.00 and +1.00.
specificity of variables Refers to the fact that a given
study is conducted with a specific kind of subject,using
specific measuring instruments, at a specific time, and
under a specific set of circumstances. These factors
affect the generalizability of the results.
split-half reliability A measure of internal consistency
that involves dividing a test into two equivalent halves
and correlating the scores on the two halves.
stability The degree to which scores on a test are consis-
tent, or stable, over time. Also called test–retest reliability.
standard deviation A measure of variability that is stable
and takes into account every score in a distri- bution.
Calculated as the square root of the variance, or
amount of spread among test scores, it is the most
frequently used statistical index of variability.
standard error of measurement An estimate of how
often one can expect errors of a given size in an indi-
vidual’s test score.
standard score A derived score that expresses how far
a given raw score is from some reference point, typi-
cally the mean, in terms of standard deviation units.
standardized test A test that is administered, scored, and
interpreted in the same way no matter where or when it
is given.
static-group comparison A pre-experimental design that
involves at least two nonrandomly formed groups; one
receives a new or unusual (experimental) treat- ment,
the other receives a traditional (control) treat- ment, and
both groups are posttested.
statistic A numerical index describing the behavior ofa
sample or samples.

statistical regression The tendency of subjects who score
highest on a pretest to score lower on a posttest and of
subjects who score lowest on a pretest to score higher
on a posttest; a threat to internal validity.
statistical significance The conclusion that results are
unlikely to have occurred by chance—that is, that the
observed relation or difference is probably a real one.
statistics A set of procedures for describing, synthesiz-
ing, analyzing, and interpreting quantitative data.
stratified sampling A purposive process of selecting a
sample: The population is subdivided into subgroups,
and participants are strategically selected from each
subgroup.
stratum (pl. strata) A subgroup derived from a sample;
a variable that can be divided into groups.
structured interview An interview that includes a
specified set of questions to be asked.
structured item An item on a questionnaire that requires
the respondent to choose from among response options
(e.g., by circling a letter, checkinga list, or numbering
preferences). Also called closed- ended item.
summative evaluation Evaluation whose function is to
summarize the overall quality or worth of a program or
product at its completion.
systematic replication Replication that involves differ-
ent investigators, behaviors, or settings.
systematic sampling Sampling in which individuals are
selected from a list by taking every Kth name,where K
equals the number of individuals on the list (i.e.,
population size) divided by the number of sub- jects
desired for the sample.
T score A standard score derived from a z score by
multiplying the z score by 10 and adding 50. Alsocalled
Z score.
t test An inferential statistics technique used to deter-
mine whether the means of two groups are signifi-
cantly different at a given probability level. See also t
test for independent samples and t test for noninde-
pendent samples.
t test for independent samples A parametric test of
significance used to determine whether, at a selected
probability level, the means of two independent sam-
ples are significantly different.
t test for nonindependent samples A parametric test
of significance used to determine whether, at a
selected probability level, the means of two matched,
or nonindependent, samples are significantly different
or whether the means for one sample at two different
times are significantly different.
GLOSSARY 683
table of random digits See table of random numbers.
table of random numbers A list of multidigit num- bers,
arranged in a table, that have been randomlygenerated
by a computer to have no defined patterns or
regularities; used in sampling. Also called table of
random digits.
target population The population to which the researcher
would ideally like to generalize study results. Compare
accessible population.
test A formal, systematic, usually paper-and-pencil pro-
cedure for gathering information about people’s cogni-
tive and affective characteristics.
test of significance A statistical test used to determine
whether or not there is a significant difference between
or among two or more means at a selected probability
level.
testing A threat to experimental validity in which
improved performance on a posttest is the result of
subjects having taken a pretest. Also called pretest
sensitization.
test–retest reliability See stability.
theoretical validity How well a qualitative research
report explains the phenomenon being studied in rela-
tion to a theory.
theory An organized body of concepts, generalizations,
and principles that can be subjected to investigation.
time-series design A quasi-experimental design involv-
ing one group that is repeatedly pretested, exposed to an
experimental treatment, and repeatedly posttested.
topic statement A statement in a research plan or report
that describes the variables of interest to the researcher,
the specific relation among those variables and, ideally,
important characteristics of the study par- ticipants.
Also called problem statement.
treatment diffusion A threat to external validity that
occurs when individuals from different treatment
groups in an experiment communicate with and learn
from each other.
triangulation The use of multiple methods, data col-
lection strategies, and data sources to get a more
complete picture of what is being studied and to cross-
check information.
true categories Categories into which persons or objects
naturally fall, independent of a research study.
trustworthiness Along with understanding, a feature
essential to the validity of qualitative research; is estab-
lished by addressing the credibility, transferability,
dependability, and confirmability of study findings.
Type I error The rejection by the researcher of a null
hypothesis that is actually true.
684 GLOSSARY
Type II error The failure of a researcher to reject a null
hypothesis that is really false.
unobtrusive measures Ways to collect data that do not
intrude on or require interaction with research par-
ticipants; examples include observation and collecting
data from written records.
unstructured interview An interview that consists of
questions prompted by the flow of the interview itself.
unstructured item An item on a questionnaire that
gives the respondent complete freedom of response.
validity The degree to which a test measures what itis
intended to measure; a test is valid for a particular
purpose for a particular group. In qualitative research,
validity refers to the degree to which qualitative data
accurately gauge what the researcher is trying to
measure.
variable A concept (e.g., intelligence, height, aptitude)
that can assume any one of a range of values.
variance The amount of spread among scores.
z score The most basic standard score; expresses how
far a score is from a mean in terms of standard devia-
tion units.
Z score See T score.