Learning English Through Research Methods Textbook

Learning English through Research Methods
Nicholas Lassi, PhD, LLM
This is an English learning textbook for the future scientist. Learning the fundamentals of how to
conduct research may be the most important thing you ever learn. It was for me. Understanding
the processes of how our medicine is tested before it is provided to patients, how psychological
experiments are designed, and how financial data is effectively analyzed and displayed is
foundational to our understanding of how the world functions. This textbook provides a thorough
education in research methods in a way that improves English abilities. If you are planning to
engage in experimentation or compete academically at an international level, it’s essential that
you understand the information in these pages and can complete the accompanying exercises.
With this text, you will improve your English, you will improve your ability to conduct research
and experimentation, and you will improve your ability to conduct research and experimentation
in English. The road to becoming an effective researcher is long and filled with struggle, but with
the right mindset, education, and materials the process becomes less stressful and more fulfilling.
By utilizing this text, you will develop a solid foundation from which to begin your own research
and experimentation. Go forth on your adventure with research and experimentation; a world of
possibilities will open for you.
Chapter 1: Empiricism (经验主义)

Chapter 2: Scientific Inquiry and the Scientific Method
Chapter 3: Exploratory (探索) Research, Descriptive (描述) Research, and Causal (因果)
Research
Chapter 4: Quantitative (定量) Research, Qualitative (定性) Research, and Mixed Methods
Research
Chapter 5: Independent Variables and Dependent Variables
Chapter 6: Between Subjects Designs and Within Subjects Designs
Chapter 7: Internal Validity, External Validity, and Reliability (可靠性)
Chapter 8: Sampling (采样) Methods
Chapter 9: True Experiments and Quasi (拟)-Experiments
Chapter 10: The Meta-Analysis
Chapter 11: Ethical (伦理) Concerns
Contents
1
Chapter 1: Empiricism (经验主义).............................................................................................................4
Exercise 1: Word Match..........................................................................................................................9
Exercise 2: Crossword Puzzle..................................................................................................................9
Exercise 3: Fill in the Blanks.................................................................................................................11
Chapter 2: Scientific Inquiry and the Scientific Method............................................................................12
Exercise 1: Word Match........................................................................................................................15
Exercise 2: Crossword Puzzle................................................................................................................16
Exercise 3: Word Search.......................................................................................................................17
Chapter 3: Exploratory (探索) Research, Descriptive (描述) Research, and Causal (因果) Research......19
Chapter 4: Quantitative (定量) Research, Qualitative (定性) Research, and Mixed Methods Research....27
Chapter 5: Independent Variables and Dependent Variables.....................................................................41
Chapter 6: Between Subjects Designs and Within Subjects Designs.........................................................51
Chapter 7: Internal Validity, External Validity, and Reliability (可靠性).................................................61
2
Chapter 8: Sampling (采样) Methods........................................................................................................77
Chapter 9: True Experiments and Quasi (拟)-Experiments.......................................................................89
Chapter 10: The Meta-Analysis.................................................................................................................98
Exercise 2: Crossword Puzzle..............................................................................................................100
Exercise 3: Fill in the Blanks...............................................................................................................101
Chapter 11: Ethical (伦理) Concerns.......................................................................................................102
Exercise 1: Word Match......................................................................................................................108
Exercise 2: Crossword Puzzle..............................................................................................................109
Exercise 3: Word Search.....................................................................................................................110
Exercise 4: Fill in the Blanks...............................................................................................................111
References...............................................................................................................................................112
Terminology............................................................................................................................................113
Crossword Puzzle Answers.....................................................................................................................160
Chapter 1: Empiricism.........................................................................................................................160
Chapter 2: Scientific Inquiry and the Scientific Method......................................................................161
Chapter 3: Exploratory Research, Descriptive Research, and Causal Research...................................162
Chapter 4: Quantitative Research, Qualitative Research, and Mixed Methods Research.....................163
Chapter 5: Independent Variables and Dependent Variables...............................................................164
Chapter 6: Between Subjects Designs and Within Subjects Designs...................................................165
Chapter 7: Internal Validity, External Validity, and Reliability...........................................................166
Chapter 8: Sampling Methods.............................................................................................................167
Chapter 9: True Experiments and Quasi Experiments.........................................................................168
Chapter 10: Meta-Analysis..................................................................................................................169
Chapter 11: Ethical Concerns..............................................................................................................170
Fill in the Blanks Answers:.....................................................................................................................171
3
Chapter 1: Empiricism.........................................................................................................................171
Chapter 2: Scientific Inquiry and the Scientific Method......................................................................171
Chapter 3: Exploratory, Descriptive, and Causal Research..................................................................172
Chapter 4: Quantitative Research, Qualitative Research, and Mixed Methods Research.....................173
Chapter 5: Independent Variables and Dependent Variables...............................................................173
Chapter 6: Between Subjects Designs and Within Subjects Designs...................................................174
Chapter 7: Internal Validity, External Validity, and Reliability...........................................................175
Chapter 8: Sampling Methods.............................................................................................................176
Chapter 9: True Experiments and Quasi Experiments.........................................................................177
Chapter 10: Meta-Analysis..................................................................................................................177
Chapter 11: Ethical Concerns..............................................................................................................178
A note on the underlined words: Underlined words are also used in the exercise sections, and
they are found in the “terminology” section at the end of the textbook. Underlined words are
important words; they should be understood and carefully considered.
4
Chapter 1: Empiricism (经验主义)
Empiricism ( 经验主义 ) is the cornerstone of much of science and experimental research. It
involves gathering information from the senses ( 感官 ) and from experience—this information is
the basis for research, experimentation, and testing.
With empirical ( 经验 ) research, phenomena ( 现象 ) are observed and investigated using
our five sensory experiences: sight, touch, sound, smell, and taste.
Evidence and data are gathered from these sensory processes. This data is usually
obtained systematically ( 系统 ) and procedurally ( 程序 ) within well-defined experimental
operations. Research generally, but empirical (经验) research particularly, may be best thought of
as, “a process through which questions are asked and answered systematically” (Dane, 1990; 18).
The data and information obtained through empirical processes is then used for hypothesis (假
设 ) testing and other kinds of experimentation. This hypothesis testing and other kinds
experimentation often involves statistical ( 统计 ) procedures that determine the relationships
among different variables (变量).
Empirical ( 经验 ) research means that scientists generate evidence that is derived from
sensory ( 感官 ) experiences, and then use this evidence in experimentation and in making
observations about the world.
The five senses: hear, see, taste, smell, and touch. (Wikimedia commons).
Empirical ( 经验 ) research involves theory in two main ways: theory ( 理论 ) testing and theory
building.
Theory (理论) Testing

Theory ( 理论 ) testing involves: true experiments, quasi ( 拟 ) -experiments, the use of secondary
data, and survey (调查) studies.
True Experiments
True experiments are causal ( 因果) studies (also known as “cause and effect” studies). The basic
question is this: does the independent variable, as it is manipulated ( 操纵 ) by the researcher
under controlled ( 控制 ) conditions ( 条件 ), cause a statistically significant ( 重大 ) change in the
dependent variable? Regarding the purpose of experiments, Singleton and Straits (1999) explain
5
that, “Experiments frequently offer the best approach for investigating the causes ( 原因 ) of
phenomena. In an experiment the researcher systematically ( 系统 ) manipulates some feature of
the environment and then observes whether a systematic change follows the behavior under
study” (7).
This statistically significant change, or systematic change with group comparison, can be
thought of in this way: it is believed that there is a difference between the groups (comparing the
treatment (治疗) group(s) and the control group) because of the manipulation of the independent
variable, but was the difference so large that chance or luck can be reasonably ruled out in
explaining the difference between the groups? In other words, if the difference between the
experimental groups is so great that it is very unlikely the difference is due to chance (luck), then
it can be reasonably claimed that the groups are statistically (统计) different from each other (the
treatment group is different from the control group) and, thus, there is greater evidence that the
treatment is effective.
Four Essential Parts Required to Have a True Experiment
True experiments should encompass (拥有) these four essential parts:

1. The researcher manipulates (操纵) the independent variable. This means that there needs
to be more than one level of the independent variable—e.g. a treatment group and a
control group. Stated differently, there should be more than one group (and the
researcher has the capacity to determine the number of groups and how the groups will
receive the different treatments) so that comparisons can be made among the groups.
2. There is a way to measure the dependent variable. The researcher should have a way to
measure the dependent variable to compare the experimental groups. If the researcher
cannot measure and compare the groups, there will be no way of knowing if there is a
significant difference between the groups (e.g., is the treatment group different from the
control group in the variable under study?).
3. The researcher has control over the experimental environment. These experimental
controls help minimize or eliminate other potential explanations for the experimental
results (i.e., the differences between the groups regarding their scores on the dependent
variable), as Singleton and Straits (1999) explain, “The use of control procedures that
rule out biases ( 偏差 ) and confounding explanations of the events being studied is the
principle way in which scientific inquiry differs from casual observation” (32). It’s
important to control the experimental environment so that everything is the same or
nearly the same among the different groups except for the different levels of the
treatment/stimulus that are provided to the groups.
4. The subjects are randomly ( 随机 ) assigned to the different experimental groups. This is
known as “random assignment.” Otherwise stated, each subject in a sample ( 样品 )
should have a known and equal chance of being assigned to the different experimental
groups.
To illustrate, let’s say a researcher wants to study the effects of caffeine ( 咖啡因 )
on math ability. The subjects are provided different amounts of caffeine, they study
some mathematical material, and their math scores are measured. There are three
experimental groups: 200mg of caffeine group, 100mg of caffeine group, and a water
group (the control group). A sample ( 样品 ) of 300 people is randomly selected from the
population ( 人群 ) to participate in the study. Now, this is where random assignment
6
plays a key role, the researcher then takes the sample of 300 and randomly assigns the
subjects to the different experimental groups. In this way, 100 people will be assigned to
the 200mg group, 100 people to the 100mg group, and 100 people to the water group.
Random assignment, acting in conjunction with an appropriate sample size, helps
to ensure that the experimental groups are considered statistically equal before the onset
of the treatments. If there is no random assignment, the experimenter will not know for
sure that the groups are statistically equal before the treatments are delivered to the
groups; without this knowledge, a true experiment cannot be claimed.
Quasi (拟) -Experiments
Quasi ( 拟 ) -experimentation represents non-causal studies that usually lack random ( 随机 )

assignment or some other fundamental mechanism ( 机制 ) within a true experiment (causal
study). Typically, a quasi-experiment will closely resemble a true-experiment, but it will lack
random assignment (randomly assigning subjects to the different experimental groups). Usually,
the experimenter will be unable to randomly assign subjects to the different experimental groups
for ethical (伦理) or impractical (不切实际) reasons and will have to settle with preexisting (原有)
groups as means for experimentation.
For instance, let’s say an experimenter has a new teaching method for children that
involves using smartphones in the classroom. The optimal experimentation method would be to
generate a sample ( 样品 ) and randomly assign students to either a smartphone classroom or to a
normal, non-smartphone, classroom (the control group), but for ethical reasons an experiment
like this may be impractical. It may be unethical because it’s commonly not good to break
children up from their friends and familiar environment to be assigned ( 分配 ) to random
classrooms for experimental purposes. It’s not good for them psychologically ( 心理 ), and
certainly not for some small-time experiment. The other option is to conduct a quasi-experiment
where the new teaching method and the control method are applied to preexisting ( 原有 )
classrooms of students. So, there are preexisting classrooms with students in a school, and the
experimenter provides preexisting classroom of students A with the smartphone teaching
method, and preexisting classroom of students B becomes the control group. Without random
assignment it’s difficult to be certain that the groups (the preexisting classrooms of students) are
statistically equal at the start of the experiment, thus, it is not a true-experiment.
The Use of Secondary Data
The use of secondary data involves using data that was collected by others and incorporated ( 合
并) into a new and unaffiliated (无关) study in some way.
Survey (调查) Studies
A survey ( 调查 ) (or questionnaire) is a method of information and data collection—usually

administered ( 措 ) by questioning subjects through interviews ( 采访 ) —conducted face-to-face
with participants, through email, through cellphone communications, etc. Questionnaires, Harris
(2013) writes, “Present participants with a highly structured set of stimuli in the form of specific
questions or statements (with which the participants indicate their level of agreement) about the
7
issue at hand. These questions can be open-ended, asking the participants to respond in their own
words, or can involve fixed alternatives, from which the respondent is asked to choose” (52). The
data and information derived from one subject or a group of subjects can then be compared ( 比
较) to other subjects.
Theory (理论) Building
Theory ( 理论 ) building involves: qualitative ( 定性 ) research, case-study methods, and

exploratory (探索) research.
Qualitative (定性) Research
Qualitative (定性) research uses words instead of numbers to describe and explain a phenomenon
( 现象 ). This research often involves investigations into the lived experiences of people: how
people interpret ( 解释 ) the world, find meaning in their lives, construe events or phenomenon in
their lives, communicate, operate within cultures, etc. Qualitative research routinely uses an
inductive (归纳的) approach to understand issues and phenomena.
Case-study Methods
Case-study methods are an in-depth analysis of one or a small number of cases ( 事例) to achieve
a better understanding of a subject, phenomenon ( 现象 ), or event. It is a rich examination of a
singular (单数) case, or a small number of cases, that is unique and interesting.
Case-study of life working on an archaeological excavation (Photo: David Hallam).
Exploratory (探索) Research
Exploratory ( 探索 ) research gathers preliminary ( 初步 ) information and data to generate

hypotheses (假设) or as preparation for a larger study.
Study Questions:
1) What is empirical evidence?

2) A true experiment should contain four essential parts, what are they?
8
3) What does it mean to “manipulate the independent variable”?
4) A quasi-experiment is like a true experiment, but it lacks what?
Exercise 1: Word Match
Match the term with the definition:
Data / Pieces of information that are often generated quantitatively.

Experience / A better understanding of something through previous involvement with it.
Experiments / Testing phenomena through scientific processes.
Evidence / Data/information that can increase or decrease the acceptance of an assertion.
Hypothesis / An educated explanation for an event or phenomena; an educated prediction of the
results of an experiment.
Investigation / An exploration or inquiry into a phenomena or event.
Observation / The collection and recording of data/information.
Qualitative / Descriptive explanations concerning how people interpret reality.
Quasi / Things are closely related in their fundamental design, though different in important
ways that distinguish them from one another.
Research / An investigative undertaking to better understand the nature of the world.
Senses / Physiological organs that generate information and data; sight, hearing, etc.
Statistics / Mathematical process that can make statements about the relationships among
variables.
Survey / Gathering information by questioning human subjects systematically or in a more open-
ended fashion.
Systematic / Process oriented; concise; sequential in operation.
Theory / A unified collection of seemingly correct propositions that explain a phenomenon.
Exercise 2: Crossword Puzzle
Using the terms from the “word match” exercise provided previously, complete the crossword
puzzle:
9
ACROSS
4 Data/information that can increase or decrease the acceptance of an assertion.
6 Physiological organs that generate information and data; sight, hearing, etc.
8 Things are closely related in their fundamental design, though different in important ways that
distinguish them from one another.
9 The collection and recording of data/information.
11 An educated explanation for an event or phenomena; an educated prediction of the results of
an experiment.
12 An exploration or inquiry into a phenomena or event.
DOWN
1 Mathematical process that can make statements about the relationships among variables.
2 Descriptive explanations concerning how people interpret reality.
3 An investigative undertaking to better understand the nature of the world.
5 A better understanding of something through previous involvement with it.
7 Testing phenomena through scientific processes.
10 Pieces of information; often generated quantitatively.
10
Exercise 3: Fill in the Blanks
Fill in the blanks with the correct terms. Each term should be used only once (one term for one
question). Some of the terms are interchangeable for some of the questions (more than one term
is acceptable for different questions). For example, one term may be acceptable for three
different question, so it’s up to you to choose the most appropriate question for the term. Discuss
with a partner which terms could be used interchangeably to complete the questions and why.
Terms: data, experience, experiment, evidence, hypothesis, qualitative, quasi, senses, statistical,
survey, and systematic.
1. Once all the DATA has been collected, statistical analysis are conducted to determine if
there are differences between the groups.
2. A person who has little EXPERIENCE with statistical software will need to consult
guidebooks to help them along.
3. A true EXPERIMENT should contain four elements: the random assignment of subjects to
different experimental groups, more than one level of the independent variable (more than
one experimental group), the capacity to control the research environment, and the ability to
measure the dependent variable.
4. Cause and effect statements about phenomena usually require empirical EVIDENCE
5. The HYPOTHESIS was correct. The researcher’s ability to predict and forecast experimental
results was quite impressive.
6. A mixed-methods approach combines both quantitative and QUALITATIVE methods. It’s
research that combines statistics with a word or dialog centric approach to understand a
phenomenon in a deeper way.
7. The researcher’s QUASI-experiment was valuable in displaying the effectiveness of the
therapy despite the lack of random assignment.
8. Empirical information is gathered from the SENSES.
9. STATISTICAL procedures are employed after the data is collected to determine if there are
differences among the groups, or, in other cases, to determine relationships among the
variables.
10. An online SURVEY is often an effective way to quickly and conveniently gather opinion-
based data. Though, external validity is often low because the responses may not fully
capture the opinions of the population as whole.
11. The researchers experimental design was SYSTEMATIC in its operational framework; the
experimental process was clearly defined, sequential, and structured logically.
11
Scientific exploration is largely conducted through the scientific method. The word “science” is
defined by Merriam-Webster (2018) as “The state of knowing: knowledge as distinguished from
ignorance or misunderstanding.” Myers and Hansen (1997) describe science as, “What we know,
such as the facts we learn in our psychology or chemistry courses. But science is also a process
—that is, an activity that includes the systematic ways in which we go about gathering data,
noting relationships, and offering explanations” (3). Because of the scientific method and
scientific inquiry (查询), the pool of knowledge, from which we all pull our understanding of the
world, expands and deepens.
The scientific method represents ( 代表 ) several principles instrumental ( 仪器的 ) to the
systematic ( 系统 ) inquiry of phenomena. Using the general principles of the scientific method,
empirically ( 经验 ) derived data and information can be collected and tested. It is through this
process that hypotheses are generated, and experiments are conducted.
Experiments are normally conducted after we observe the natural ( 自然 ) world and
develop hypotheses from these observations.
Researchers generate hypotheses, and then test these hypotheses through experimentation
or observation to determine if their predictions ( 预测 ) are correct or incorrect. After the
experimentation is conducted, explanations for the results are surmised ( 推测 ). These
explanations and the experimental results are expected to be validated ( 验证 ) through the
replication ( 复制 ) process and other future experimentation. Singleton and Strait (1999) explain
the relationship between predictions and explanations, “The process of prediction is logically
equivalent to the process of explanation; prediction is carried out before the event occurs and
explanation afterward. Any valid scientific explanation, therefore, is at once a prediction of the
future” (22).
An experiment needs to be falsifiable ( 证伪 ) or the results will likely not be considered
legitimate ( 合法 ) or fundamentally sound ( 扎实 ). The process of falsification means that
hypotheses are retested, and the results of these retests modify, validate ( 验证 ), or refute ( 反驳 )
the initial tests or initial explanations. Therefore, it’s important that experiments are
operationally defined (that the experimental processes undertaken by researchers is clearly
diagramed ( 可图示 )) in their write-ups, so that other researchers have the ability and tools
necessary to falsify experimental results.
Experimental results provide evidence for or against predictions drawn from hypotheses.
If a hypothesis acquires a great deal of corroborating ( 确凿 ) evidence, through rigorous
experimentation and testing, a theory may be proposed ( 提出 ). A theory ( 理论 ) is a set of
assumptions ( 假设 ) and concepts ( 概念 ) about the world that are substantiated ( 证实 ) by
scientific data. Merriam-Webster (2018) provides two helpful definitions of a theory, referring to
it as “a plausible or scientifically acceptable general principle or body of principles offered to
explain phenomena,” and they describe it as “a principle that has been formed as an attempt to
explain things that have already been substantiated by data…Because of the rigors of
experimentation and control, it is understood to be more likely to be true than a hypothesis is.”
12
When an interconnected web (互联网) of theories are robust (鲁棒), unimpeachable (无可
指责 ), and possess universal applicability, they can become a law, or a general principle of
science.
The scientific method should be considered general principles that offer guidelines for
experimental operations, rather than a concrete ( 具体 ) and rigid ( 硬性 ) processes that should be
adhered to without fail. It is not a rigid system that must always be followed; this is due to the
variation in experiments, the assortment of research areas, the different materials tested, etc. The
research process is also considered an always ongoing, cyclical ( 周期性 ) endeavor ( 努力 ), so to
promote strict starting and ending points of research would be limiting the overall nature of this
process. The essence of science, and the scientific method, is in generating questions, forming
predictions based on preliminary information, and then testing the predictions using empirical
evidence.
The scientific method as an ongoing process (Wikimedia Commons).
The Scientific Method: Steps in the General Research Process
1) Generating a basic research question: These questions are often derived from reviewing
academic literature, examining prior experiments, or by making empirical ( 经验 )
observations about the world. The research question can be very specific or broader in
scope (范围).
2) Formulating a hypothesis: A hypothesis is a proposition ( 主张 ), assumption, or
explanation for the occurrence of a phenomena ( 现象 ) or an event. The researcher takes
the research question, gathers as much information as possible on the phenomena or issue
at hand, and develops propositions or explanations to account for the phenomena. The
wording of the experimental hypothesis should be stated clearly, (usually/preferably)
simply, and concisely ( 简明 ), without vague abstractions ( 抽象 ) or obscurities ( 晦涩 ). If
there is no effect or no difference between variables ( 变量 ), then the hypothesis is
considered incorrect, or, conversely ( 相反 ), if there is an effect or a difference between
variables, there is greater evidence for the hypothesis being correct.
13
In hypothesis testing, the hypothesis commonly has two parts: the null ( 无效 )
hypothesis and the alternative hypothesis. The null hypothesis represents (代表) the belief
that the treatment group is no different from the control group (or that the treatment is not
effective, the levels of the independent variable are no different from each other, there is
no correlation ( 相关 ) among the variables, etc.)—in essence, there is no effect in that
specific experiment. The alternative hypothesis represents the belief that the treatment
group is significantly different from the control group (the treatment is effective, there is
a statistical ( 统计 ) difference somewhere among the experimental groups, there is a
correlation (相关) among the variables, etc.)—in essence, the treatment is effective in that
specific experiment. A hypothesis encompasses anticipating ( 期待 ) or forecasting ( 预测 )
the results of an experiment or study. Ultimately, it’s when researchers describe how they
expect an experiment to transpire (发生).
3) Testing the hypothesis: Testing represents (代表) the means to determine if the hypothesis
(and the predictions encompassed within) is accurate within an experimental procedure.
This accuracy is decided through experimentation and testing. Myers and Hansen (1997)
describe testing as, “The means for manipulating antecedent conditions ( 条件 ) and
measuring the resulting behavior…” (594). If the experiment reveals that the hypothesis
accurately reflects the real world or reality (within the confines of the experimentation
process), general acceptance of the hypothesis is strengthened; if it does not reflect the
real world, general acceptance is weakened.
4) Analyzing the experimental results: The researcher examines the results of the experiment
and ascertains ( 查明 ) the most appropriate direction to take with this information. If the
experiment, or the statistical data derived from the experiment, show that the treatment is
likely effective (e.g., the treatment group is statistically different from the control group),
greater evidence for the overall effectiveness of the treatment is garnered within the
academic community ( 社区 ). This data and the analytical procedures are then presented
to the academic community (this information should be presented regardless of whether
the results were statistically significant).
5) Additional processes: When an experiment has run its course, additional processes, such
as the replication of an experiment and the peer review process, are often undertaken.
Replication ( 复制 ) is the process of repeating ( 重复 ) an experiment to reduce the
chances that the results were the product of experimental error, luck, bias ( 偏差), etc., and
to bolster (支持) support for the conclusions.
The peer ( 同行 ) review process is a major component in the publication of
research. Once the experiment is complete and the paper is written, the researcher will
send the paper to an academic journal for peer review and potential publication. During
the peer review process, the research is assessed by a series of experts in the field to
determine if the experimental process is well-grounded, reasonable, and acceptable. If the
reviewers determine that the paper is acceptable, it may be published in the academic
journal. Some journals choose who will review and examine the research papers, other
journals will have the author volunteer the names of experts who possess the capacity ( 能
力) to examine the material in question (often in more specialized areas of academia).
Study Questions:
14
1) What is a law within science?
2) Why is the scientific method useful?
3) What does it mean to forecast or predict something?
4) What is the peer review process?
5) What is the relationship between the null hypothesis and the alternative hypothesis?
Alternative / The hypothesis that states that the new treatment appears to be effective.
Corroborate / Information that supports a claim.
Cyclical / Circular and round.
Empirical / Collecting information from the senses.
Falsifiable / Having the capacity to reproduce an experiment.
Forecast / An educated prediction regarding what will occur in the future.
Hypothesis / Explains the predicted results.
Inquiry / Investigate, search, and question.
Legitimate / Accepted within the scientific community; understood to be valid.
Null / The hypothesis that states that there is no effect or no difference between the levels of the
independent variable.
Observation / Collecting and recording information or data.
Peer / A fellow researcher operating within the same field.
Phenomena / Something that happens that is studied.
Prediction / When the researcher makes a statement regarding what they think will happen in the
future.
Principles / A guiding doctrine fundamental to the operation of some process.
Proposition / Proposal or hypothesis.
Publication / The introduction of the researcher’s study to the public through formal and peer-
reviewed processes.
Replication / Running the same experiment repeatedly to see if the same results are observed.
15
Significant / When something hits a point when it is considered worthy of notice.
Sound / Unburdened by problems, fault, or apprehension.
Systematic / When a proceeding or process is operational, detailed, and step-by-step.
Variation / Something that is changeable or something that does not take on a definite property.
puzzle:
ACROSS
6 Collecting information from the senses.
8 Collecting and recording information or data.
16
9 The hypothesis that states that there is no effect or no difference between the levels of the
10 Having the capacity to reproduce an experiment.
11 The hypothesis that states that the new treatment appears to be effective.
12 A fellow researcher operating within the same field.
DOWN
1 Investigate, search, and question.
2 Explains the predicted results.
3 An educated prediction regarding what will occur in the future.
4 Accepted within the scientific community; understood to be valid.
5 Information or statements that support a claim or other information.
7 Circular and round.
Exercise 3: Word Search
For each definition numbered below there is a corresponding term in the “word search” puzzle.
The terms are from the “word match” exercise provided previously. Locate that term in the word
search puzzle:
1. Something that happens that is studied.
17
2. When the researcher makes a statement regarding what they think will happen in the
future.
3. A guiding doctrine fundamental to the operation of some process.
4. Proposal or hypothesis.
5. The introduction of the researcher’s study to the public through formal and peer reviewed
processes.
6. Running the same experiment repeatedly to see if the same results are observed.
7. When something hits a point when it is considered worthy of notice.
8. Unburdened by problems, fault, or apprehension.
9. When a proceeding or process is operational, detailed, and step-by-step.
10. Something that is changeable or something that does not take on a definite property.
Terms: alternative, corroborate, cyclical, empirical, hypothesis, inquiry, legitimate, null, peer,
phenomena, principles, publication, replication, significant, and variation.
1. When something is not the default or normal option, it is considered the __________ option.
2. __________ your evidence with other evidence to support your claims about the world.
3. When something is moving in a circular manner, it can be said to have a __________ nature.
4. Scientists usually generate __________ evidence to test theories.
5. Prior to the experiment, the researcher made numerous predictions. Her main __________
was that the new medicine would be effective in reducing blood pressure.
6. The lead researcher’s __________ into the loss of several team members yielded no
information.
7. Using the scientific method is a __________ way to investigate and test phenomena.
8. The __________ hypothesis indicates that there is no difference between the new medicine
group and the control group.
9. A __________ can provide important feedback and suggestions concerning one’s research
plans.
10. The __________ that you wish to examine has already been thoroughly studied. In what way
will you be adding to our understanding of it?
18
11. The ethical __________ diagramed in the “conduct manual” were closely followed by the
research team to minimize any future criticisms.
12. After an academic paper passes the peer-review process it is ready for __________.
13. An experiment can be falsified through the __________ process.
14. Without statistically __________ results, one cannot claim that a new medicine is effective.
15. There was too much __________ in your research environment—strange people entering and
leaving, the random incorporation of new measurement instruments, changing room
temperatures, etc.
Chapter 3: Exploratory (探索) Research, Descriptive (描

述) Research, and Causal (因果) Research
There are three general types of research: Exploratory ( 探索 ) Research, Descriptive ( 描述 )
Research, and Causal ( 因果 ) Research. These forms of research may operate independently or
together with one another. Most research contains one or more of these three basic research
types.
Exploratory (探索) Research
Exploratory (探索) Research involves gathering preliminary (初步) information to generate ideas
for research, formulate ( 制定 ) experimental designs, define hypotheses, suggest hypotheses, and
things of this nature. It often involves both qualitative ( 定性 ) and quantitative ( 定量 )
research/exploration, wherein case studies, interviews ( 采访 ), surveys ( 调查 ), focus groups, or
pilot studies are conducted to gather preliminary information or data. Secondary research/data or
archival (档案) data may also be examined to develop new ideas. Exploratory research is usually
not generalizable to any larger population ( 人群 ), because it is an exploration into a topic to
determine if there is anything that can be tested or studied.
Pilot Studies and Focus Groups
Two illustrative forms of exploratory research worth mentioning are pilot studies and focus
groups: A pilot study is a limited experiment wherein treatments/stimuli are tested with a small
number of subjects, commonly within a controlled environment. It’s a good way to quickly
gauge (测量) whether the different treatments and controls are appropriate for a larger study, or if
they need to be adjusted prior to the actual experiment. A pilot study could be as simple as
providing a group of ten people a pretest, the treatment, and a posttest, and measuring to see if
there was a difference between the pretest and the posttest to determine if the treatment had
seemingly made a difference (or was effective). Obviously, this type of quick study with a small
sample size and no control group will not hold-up under academic scrutiny, but it’s a quick,
cheap, and easy way to get a sense of whether conducting a larger study would be worth it.
A focus group involves interviews (采访) that are conducted with small groups of people,
rather than in a one-to-one format, to gather information. Subjects are brought together, either as
strangers to each other or with a group affiliation ( 联系 ) (depending on the nature of the study),
19
and encouraged to interact, converse ( 交谈 ), express opinions and feelings, etc. as they are
introduced to questions and other material provided to them by researchers. From this group
interaction and conversation, information is provided to the researcher regarding opinions and
feelings about such topics as product designs, the wording used in survey questions, buying
patterns, the taste of sports drinks, etc. Gomm (2008) describes how information from focus
groups is generated, recorded, and presented, “[Focus groups] consist of a group discussion
focused on topics provided by the researcher and sometimes involve the use of stimulus material,
such as problem scenarios for participants to discuss, videos for them to comment on, or set
piece debates. The group discussion will probably be audio or video recorded and the write up is
most likely to be qualitative rather than quantitative, although there are exceptions” (226).
Descriptive (描述) Research
Descriptive ( 描述 ) research is a report of the characteristics or relationships within a sample ( 样

品 ), population ( 人群 ), or phenomena ( 现象 ) under study. It explains the properties ( 性能 ) and
makeup of the data under study. It is non-causal research that may involve correlational ( 相关 )
studies, statistical ( 统计 ) research, classification ( 分类 ), categorization, meta-analysis, averages
( 平均 ), frequencies ( 频率 ), dispersion ( 分布 ), graphs, etc. Ultimately, descriptive research
describes and presents data and information.
Descriptive research: A frequency histogram displaying the number of days respondents do homework.
Correlational (相关) Studies
A major form of descriptive research is the correlational ( 相关 ) study. Correlational ( 相关 )

studies look to determine the nature and strength of the relationship between variables ( 变量 )
(these variables could be such things as choice preferences, behaviors, phenomenon, events,
etc.). When a correlation ( 相关 ) exists among two variables, it could be said that changes in one
variable are related to changes in another variable. A positive relationship among two variables
is present if an increase in the amount of one variable is attended by an increase in the amount of
another variable. Similarly, a positive relationship is found if a decrease in the amount of one
variable is attended by a decrease in the amount of another variable. Conversely ( 相反 ), a
20
negative relationship among two variables is present if an increase in the amount of one variable
is attended by a decrease in the amount of another variable. The relationships among variables
are often displayed visually using what are known as scatterplots (散点图).
A scatterplot (散点图) of the relationship between family routines and average grades received in the 8th grade.
A scatterplot will visually show where a subject’s score or data point is located, as a
product of where both variables under analysis coincide ( 重合 ). Within these scatterplots,
regression lines are formed to best display the relationship among the variables. Regression lines
are mathematical representations of the relationship among two variables. The Pearson Product-
Moment Correlation Coefficient, or r, reveals the direction and strength of the relationship
among variables. This coefficient ( 系数 ), or r, is given within a range of -1.00 to +1.00 as a
measure of the strength and direction of the relationship. The closer that r is to +1.00 or -1.00,
the stronger the relationship.
A multiple regression analysis can be conducted when there are more than two variables
being examined. This multiple regression analysis, or R, works to predict the value of one
variable when provided the values of the other variables.
Causal (因果) Research
Causal (因果) research tests experimental hypotheses. It has the power to determine cause ( 原因)
and effect relationships (determine causality). It usually involves tightly controlled, laboratory-
based experimentation that produces statistically (统计) driven conclusions.
21
Laboratory-based experimentation: Barbara McClintock (1902-1992) in her laboratory. Department of
Genetics, Carnegie Institution, New York. This picture was publicized after she received the American
Association of University Women Achievement Award in 1947 for her work on cytogenetics (Creative
Commons).
An experimenter engaged in causal research should emphasize obtaining high internal

validity (high internal validity means that there is a great likelihood that the researcher is directly
measuring what the researcher wants to measure; put differently, the researcher is studying the
relationship between the independent variable(s) and the dependent variable(s), and outside
variables are not impacting the study and influencing the results). With causal research,
hypothesized causal factors are tested through true experimentation (experimentation that
possess random ( 随机 ) assignment, a controlled research environment, the manipulation ( 操纵 )
of an independent variable, and the measurement of a dependent variable).
Study Questions:
1) What is exploratory research?

2) What is descriptive research?
3) What is a pilot study?
4) What is a focus group?
5) What is a correlational study?
6) The relationships among variables in a correlational study are displayed visually using
what?
Average / The middle position in a data set.
22
Categorize / Arranging data, people, things, etc. into groups based upon some criteria.
Causal / When an independent variable produces a statistically significant change in the
dependent variable within a true experiment.
Characteristics / Specific features or distinguishing properties.
Classify / Grouping things together based upon some requirement(s).
Confounding / A variable that disrupts an experiment and produces erroneous results.
Control / Reducing variability and error within an experiment.
Correlation / A change in one variable is or is not associated with change in another variable.
Dispersion / The reflection of the spread of data in a data set.
Experimentation / Testing phenomena using the scientific method.
Formulate / To develop or construct something.
Frequency / Enumerating the occurrence of an event or phenomena.
Generate / Develop or bring into existence.
Interview / Obtaining information from a subject through a question/answer interaction.
Laboratory / The arena in which many controlled scientific experiments take place.
Meta-analysis / An analysis of several independent studies, which cover the same or similar
content, to gauge the overall effect of a medicine, therapy, or other stimulus.
Operation / When events or actions take place in a procedural and preplanned fashion.
Phenomena / Something that happens.
Preliminary / Doing something before the advent of the main action or main event.
Property / A characteristic or distinguishing feature of someone or something.
Qualitative / An in-depth study of a smaller number of subjects or phenomena that focuses more
on words rather than numbers.
Variable / Something that can be changed or manipulated.
puzzle:
23
ACROSS
6 To develop or construct something.
8 When an independent variable produces a statistically significant change in the dependent
variable within a true experiment.
10 Specific features or distinguishing properties.
11 A variable that disrupts an experiment and produces erroneous results.
DOWN
1 Testing phenomena using the scientific method.
2 Arranging data, people, things, etc. into groups based upon some criteria.
3 Reducing variability and error within an experiment.
4 A change in one variable is or is not associated with another variable.
5 The reflection of the spread of data in a data set.
7 The middle position in a data set.
9 Grouping things together based upon some requirement(s).
24
search puzzle:
1. Enumerating the occurrence of an event or phenomena.

2. Develop or bring into existence.
3. Obtaining information from a subject through a question/answer interaction.
4. The arena in which many controlled scientific experiments take place.
5. An analysis of several independent studies, which cover the same or similar content, to
gauge the overall effect of a medicine, therapy, or other issue.
6. When events or actions take place in a procedural and preplanned fashion.
7. Something that happens.
8. Doing something before the advent of the main action or main event.
25
9. A characteristic or distinguishing feature of someone or something.
10. An in-depth study of, usually, a small number of subjects or phenomena that focuses
more on words rather than numbers.
11. Something that can be changed or manipulated.
Terms: average, categorize, causal, characteristics, confounding, control, correlation, formulate,

interview, laboratory, meta-analysis, preliminary, qualitative, and variable.
1. The treatment groups __________ or mean was compared to the control groups mean at the
end of the study to determine the effectiveness of the treatment.
2. The researcher didn’t __________ subjects according to their age or any other dimension
involving time.
3. It is only through the implementation of true experiments that one can make __________
claims.
4. The __________ that makeup a population may influence how subjects respond to survey
questions.
5. A highly controlled research environment and well-defined research procedures will often
reduce the influence of __________ variables.
6. The __________ group is usually that which does not receive any treatment or any new
stimuli.
7. __________ does not equate to causation. Just because there is a strong relationship between
two variables does not mean that one is secure in the knowledge of the directionality of the
variables, nor is one secure that there wasn’t a third variable influencing the results.
8. The mathematician would regularly __________ difficult mathematical equations.
9. Ethnographers (民族志) will often repeatedly __________ subjects over the course of their
research to garner a deeper understanding of the subjects lived experiences.
10. The __________ was in pristine condition, with clean floors, disinfected countertops, and
spotless instruments.
11. A __________ is often effective in determining what the academic literature in its entirety
says about something.
12. The __________ tests showed that the medicine was effective, but further, and more rigorous
studies, were needed to confirm the findings.
26
13. The __________ information was gathered through face-to-face interviews, wherein subjects
were asked questions about their experiences.
14. The independent __________ is manipulated by the experimenter.
Chapter 4: Quantitative (定量) Research, Qualitative (定

性) Research, and Mixed Methods Research
Will you use experiments and statistics or words and interviews as your main research style?
Let’s look at these different research styles (quantitative ( 定量 ) versus qualitative( 定性 )) to see
what is most suitable for your interests.
Quantitative (定量) Research
Quantitative ( 定量 ) research typically uses numbers, data, information that can be measured and
counted, statistical (统计) procedures, and systematic (系统) operations. Within experimentation,
quantitative research involves hypothesis testing, theory ( 理论 ) testing, comparing
data/information, specific research questions, structured operations, controlled research
environments, dispassionate ( 冷静 ) and objective ( 客观性 ) researchers, precise and standardized
measurements, conclusive results, and generalizable results.
It is fundamental to most quantitative research that the researchers remain impartial and
objective as they engage in their work, so as not to bias ( 偏差 ) the results. In other words, it’s
vital that researchers detach emotionally from their research as much as possible to avoid altering
the results. Objective ( 客观性 ) researchers try to be dispassionate about the observed data and
results, they are concerned only with truth (rather than personal promotion or opinion), and they
frame the results, discussion, and write up of an experiment accurately and without a personal
agenda.
Qualitative (定性) Research
Qualitative ( 定性 ) research mainly uses words rather than numbers for its research procedure.
This often involves interviews (采访) and observations to better understand the nature of subjects
lived experiences, the meanings and values people attribute to their lives, how people interpret
and respond to events and phenomena, how people understand the world around themselves, and
so on. Qualitative research is often impressionistic and interpretive ( 解释性 ) with a high level of
researcher involvement (which makes it more subjective (主观) and less generalizable).
Qualitative ( 定性 ) research is often used during the exploration ( 探索 ) phase of a study,
prior to experimentation, where the researcher engages in information gathering, idea
formulation, and theory ( 理论 ) development. During the exploration phase, questions are asked,
27
surveys ( 调查 ) are given, observations are made, interviews are conducted, data is analyzed,
records are reviewed, etc., to better understand a phenomenon or event.
Qualitative processes often provide a method for an in-depth and holistic ( 整体 ) analysis
of smaller numbers of subjects. It usually uses an inductive ( 归纳的) approach to study behavior,
a phenomenon, a culture, etc. Creswell (2013) writes of the inductive nature of qualitative ( 定性)
research, “The procedures of qualitative research, or its methodology, are characterized as
inductive ( 归纳的 ), emerging, and shaped by the researcher’s experience in collecting and
analyzing the data. The logic that the qualitative researcher follows is inductive, from the ground
up, rather than handed down entirely from a theory or from the perspectives of the inquirer” (22).
Interviews and Field Studies
Two main avenues to obtain qualitative data are through interviews and by undertaking field
studies (interviews can be conducted both independently and in field studies):
Interviews (采访)
Interviews ( 采访 ) are conducted in three main ways: structured, unstructured, and semi-
structured.
An in-person interview environment (photo: David Davies)
1. Structured interviews are conducted systematically ( 系统 ), the questions are

predetermined, the sequence ( 顺序 ) of the questions are diagramed ( 可图示 ) and
generally unchangeable, and the entire interview process is standardized in a way that it
is expected to be the same for each participant.
2. Unstructured interviews, on the other hand, are much less formal ( 正式 ), much less rigid
( 硬性 ), and, as the name states, unstructured in their deployment. They allow the
researcher and the subject a large amount of leeway ( 余地 ) in which way to take the
interview, allowing the interview to drift into different and, perhaps, unexpected
directions. Under these conditions ( 条件 ), questions can be changed or altered during the
interview process depending upon the subject’s previous responses, or they can be
changed whenever the researcher determines it is of value for the study.
3. Semi-structured interviews are a combination of both structured and unstructured
interviewing, where there are areas of focus that the researcher is seeking to cover, yet, at
the same time, the researcher allows for a certain amount of conversational drift within a
limited range of exploration ( 探索 ). Stated differently, the researcher has a general path
28
predetermined in which to proceed during the interview, but at the same time the
researcher allows room for limited detours (走弯路).
Surveys (调查): Questioning Subjects for Information
Surveys ( 调查 ) (or questionnaires) involves questioning subjects for information, and they come
formatted in two major types: closed-ended questions and open-ended questions. With closed-
ended questions, the participants may only respond in a limited number of ways (e.g., a Likert
Scale or in multiple choice form). This technique allows the researcher to effectively code ( 编码)
responses numerically for purposes of comparison. Though, the downside of this method is that
it may railroad ( 草率 ) respondents into choosing answers they normally would not. Perhaps a
respondent wanted to write in a different response to a question apart from those that were
provided, but was unable, given the limited confines of the survey structure.
Open-ended questions allow participants to respond in any way they choose,
unconstrained by the researchers controlling hand. The downside here is that abstract ( 抽象 ),
vague, indiscernible, or uninterpretable responses may be difficult to systematize and code. In
addition, the coding ( 编码 ) and comparison process can become problematic due to the infusion
of experimenter subjectivity ( 主观 ) into the interpretation of participants responses. Lastly, it’s
often an arduous process to interpret, systematize, and code open-ended responses; it requires a
lot of time and energy.
Field Studies
A researcher is conducting a field study when information is accrued in a subjects natural setting.
Under these (typically non-systematic, unstructured, or semi-structured) conditions, the
researcher will be immersed in an environment and take field notes, probe ( 探测 ), hold
interviews, ask questions, and make observations to study and better understand a phenomena
( 现象 ), event, culture, people, etc. Continuously recording observations is common in field
research—this often entails detailed recordings of subject’s behaviors and interactions on
complex coding (编码) charts.
As mentioned, most of this type of research is conducted in the subject’s natural
environment or natural setting—this is not laboratory-based experimentation or research.
Creswell (2013) explains the naturalistic ( 自然主义 ) positioning of qualitative research, “They
[researchers] do not bring individuals into a lab (a contrived situation), nor do they typically send
out instruments for individuals to complete, such as in survey ( 调查 ) research. Instead,
qualitative researchers gather up-close information by actually talking directly to people and
seeing them behave and act within their context” (45).
Research strategies such as ethnographies ( 民族志 ), phenomenological ( 现象学 ) studies,
and the grounded theory approach (among similar methodological procedures) represent field
studies and will be detailed next.
Research in the Field: Ethnographies ( 民族志 ), Phenomenological ( 现象学 ) Studies, and the

Grounded Theory Approach
Three of the most important styles of qualitative research are the ethnography, the
phenomenological method, and the grounded theory approach:
29
Ethnography (民族志)
An ethnography ( 民族志 ) is a qualitative ( 定性 ) study (though, quantitative ( 定量 ) elements are

often included) of people as they exist and interact within cultures or social groups. Researchers
generally study cultures, groups, and the lives of subjects over extended periods of time to
develop a better understanding of how people engage the world, cultures create meaning and
function, beliefs are formed, phenomenon and events are understood, and so on. An
ethnographer will work to view the world through the perspective of the subjects—as the
ethnographer gathers information about the thinking, behavior, beliefs, language, symbolism,
motivations, decisions, etc. of the subjects as the subjects interact within a culture. Information is
usually obtained in the field (fieldwork ( 实习 ))—in the subject’s environment, holistically ( 整
体), using interviews, field notes, observations, and symbols as major data sources.
Ethnographic Museum of Kruja (photo: Collective Commons).
Creswell (2013) urges qualitative researchers to employ nontraditional methods to their

data gathering processes, stating, “I especially like to see unusual forms of qualitative data
collection, such as using photographs to elicit responses, sounds, visual materials, or digital text
messages” (53).
An ethnography is commonly a long-term endeavor, where researchers engage subjects
face-to-face for extended periods of time as they gather information and data. Ethnographies
encourage researchers to be aware of their own sociocultural positions, thoughts, beliefs, and
viewpoints, to better study and understand other people in other cultures. In closing, an
ethnography is a qualitative study of social and cultural groups, it does not involve
experimentation.
Phenomenological (现象学) Method
Under the phenomenological ( 现象学 ) method, researchers qualitatively ( 定性 ) detail their own
personal experiences or the experiences of others as they engage in some event, phenomena, or
exploration ( 探索 ). Researchers will immerse themselves in cultures, groups, networks, and the
lives of subjects over extended periods of time to study the subject’s responses to a specific
phenomenon or event. Researchers may live with subjects, engage in the same behaviors as the
subjects, and develop relationships and bonds with subjects to produce a greater understanding of
how groups operate, cultures generate meaning and function, thought processes are formed,
30
beliefs are maintained, phenomenon operate, etc. all within the context of a specific phenomenon
or event. Information will usually be collected through extensive interviews, personal journals,
observations, and field notes.
To illustrate, an organizational psychologist may want to study the social and
psychological responses of subjects within a company immediately before and after it goes
public on a stock exchange (a major event in the life of a company). So, the researcher stays with
the employees of a company for several months before and after this process to study their
behavior and social interactions to identify potential issues or to study whatever the researcher
wishes to pursue. Conversely, instead of studying a company about to go public, a researcher
may want to conduct a similar study of a company that is about to file for bankruptcy and be
dissolved.
Grounded Theory Approach
The grounded theory approach uses coding ( 编码 ) methods that convert qualitative ( 定性 )
information, generated through interactions with subjects, observations, questionnaires, etc., into
quantitative ( 定量 ) data. This coded quantitative data is useful for generating patterns in the
information obtained, thus, highlighting and better understanding the operational nature of
certain relationships, people, groups, and cultures. The researcher will take recorded dialog ( 对话
框 ), transcripts, questionnaire responses, or detailed notes, and will code ( 编码 ) the information
as it accords with predetermined guidelines. The material will then be categorized in a
prearranged matrix ( 矩阵 ), eventually producing a visual model that details this information.
Smith and Davis (2005) explain how this coded ( 编码 ) and categorized material, as it exists
within a complex matrix, elucidates ( 阐明 ) the “transactional system” among the participants
under study, “The final product of grounded theory is a “model of process and a transactional
system which essentially tells the story of the research” (Smith & Davis, 2001, p. 73). “Process”
refers to a linking of actions and interactions that result in some outcome. The transactional
system allows researchers to examine the various interactions that have been observed” (114).
Researcher and Subject Interaction in Qualitative Studies
With qualitative studies, the researcher may interact with the subjects (or participants) in varying
degrees depending upon the circumstances or the goals of the research. There are four main
kinds of researcher and participant involvement: complete observer, observer-as-participant,
participant-as-observer, and complete participant.
I. Complete observer: This is where the researcher observes the subjects but does not
engage with the subjects in any way (does not participant with the subjects), and,
usually, the subjects do not know that the researcher is observing them. The researcher
is acting covertly ( 秘密 ), sometimes even monitoring the participants without their
knowledge.
31
With the “complete observer” method, the researcher could be thought of as a spy, watching the subjects
without their knowledge (photo: Tony Fischer; Creative Commons).
This method may involve having the researcher on-site observing the subjects in some
hidden or discrete manner, or it may involve something like an audio or video analysis
conducted by the researcher.
Analyzing video is a popular method for the complete observer (photo: Charbel Akhras; Collective
Commons).
Naturally, this method comes with serious ethical (伦理) concerns, the most pressing of
which is a lack of participant consent (同意).
II. Observer-as-participant: This is where the subjects are aware that they are being
observed by the researcher, but the researcher does not participate with the subjects in
their behavior and events. In effect, the researcher is around the participants observing
their behavior, and the participants have been notified that this is occurring, but the
researcher does not join the subjects in their conduct in any way. The researcher usually
wants to fade into the background, become forgotten, and observe. This method often
entails that the researcher spends a considerable amount of time with the subjects for
the subjects to become comfortable with the researcher to the point that they let their
defenses down and behave as they would under normal circumstances.
III. Participant-as-observer: This is where the subjects are aware that they are being
observed by the researcher, as they have been notified that this is occurring, and the
researcher also participates with the subjects in their actions and activities. The
researcher participates with the subjects in their events, while observing and studying
their behavior.
IV. Complete participant: This is where the researcher participates with the subjects in their
behaviors and events, but the subjects are not aware that they are being observed and
32
studied by the researcher. The researcher is effectively “undercover,” ( 卧底 ) behaving
as a normal member of a group and deceiving ( 骗局) the subjects as to her true motives,
all while observing and documenting the subject’s behaviors, interactions, etc.
It’s important to note that due to the researcher’s covert ( 秘密) involvement and
participation, the researcher is naturally influencing the subjects or altering their
behavior in a way that may not have occurred under normal circumstances, so this issue
should be considered.
This type of research also comes with serious ethical ( 伦理 ) concerns. For
instance, the subjects do not know that they are being studied and, thus, are not
consenting (同意) to be studied—in other words, subject participation is involuntary. In
addition, in much of qualitative research, relationships are formed between the subjects
and the researcher for the researcher’s personal benefit and professional gain.
Therefore, subjects may feel that they were used rather than befriended under complete
participant circumstances. The negative psychological effects and shock of subjects
realizing that what they thought was a genuine friendship/relationship turning out to be
fraudulent (欺诈) are of concern within this type of research.
A major concern in all forms of participation, no matter the degree of subject awareness,
is the concept of going native. This is where researchers forget or lose track of their positions as
researchers and observers and become fully immersed in participating with the subjects. Under
these conditions ( 条件 ), researchers may be pulled into doing things that normally wouldn’t be
done, or they may, due to an extreme immersion in a group, lose objectivity ( 客观性 ) and side
with the group despite clear evidence that it’s a biased (偏差) or incorrect position.
Content Analysis
Qualitative ( 定性 ) studies also take the form of a procedural ( 程序 ) and systematic ( 系统 ) text
analysis known as a “content analysis.” A content analysis is a way to systematically categorize,
quantify, and interpret information that is qualitatively ( 定性 ) provided to researchers by
participants or through some other channel (such as newspapers, magazines, books, historical
texts, etc.). The subject’s responses to interviews, long-form questionnaires, or to other qualitive
research procedures, is coded, categorized, and analyzed in a way that is systematic, rigorous,
and unbiased. From this coding ( 编码 ) and categorization, response patterns are formed and
larger gestalts ( 完形 ) are generated. This method of data collection and categorization is
multifaceted ( 多方面 ), intricate, and time-consuming, where large quantities of information and
dialog ( 对话框 ) (often in transcript form) is condensed ( 浓缩 ) into a limited number of patterns
and themes for explanation.
For example, let’s say a researcher wants to conduct a content analysis to study the
emotional states of different members of a family (the mother, father, and the children), to
examine the different levels of happiness, sadness, depression, etc. among the family members.
Each subject undergoes a lengthy question and answer session each evening for several weeks
that is focused on the subject’s emotions for that day. The subject’s lengthy responses are
recorded and transcribed into print form. Once all the sessions are finished and the responses
have been put into print form, the researcher counts the number of times each family member
uses “happy” words, “sad” words, etc. in the interviews, and looks for patterns in the responses
for comparative purposes (comparing family members based upon the number of times they
33
used, for instance, sad words). One can see how similar processes can be applied to media
coverage, historical texts, etc.
Case Studies
A qualitative study may be a deep and thorough analyses of a singular or a limited number of
unique cases that warrant intense examination, this is known as a “case study.” A case study is
an in-depth analysis of a singular phenomena ( 现象 ) to better understand that phenomena, and,
potentially, produce ideas for experimentation. Myers and Hansen (1997) describe a case study,
within the realm of human cases, as, “A descriptive record of an individual’s experiences and/or
behaviors kept by an outside observer. Such a record may be produced by systematically
recording experiences and behaviors as they have occurred over time” (56).
A case study is a deep and thorough dive into a phenomenon or an event. Due to the
small sample (样品) sizes used in this method, having the capacity (能力) to generalize the results
to a larger population (人群) is often sacrificed to depth and scrupulousness.
A good example of a case study is the curious and unfortunate case of a man named
Phineas Gage. Phineas Gage was a likeable, competent, and hardworking 25-year-old railway
foreman ( 工头 ) blasting rock for a railway company in the year 1848. One day, while stamping
down into the ground a small explosive device that was to be used to clear a path for train tracks,
the stamping rod Gage was using to push the explosive into the ground unexpectedly struck the
explosive in a way that detonated it. The detonation shot the stamping rod vertical, up through
Gage’s jaw, and out through the top of his skull, leaving a hole the size of a golf ball where his
brain previously resided. Amazingly, Gage, after an initial period of struggle, survived the ordeal
with little outward problems. Despite losing a significant portion of the frontal lobe ( 叶 ) of his
brain, he seemed to have recovered well physically, but problems began to appear in his
personality and behavior.
The previously hard working, well adjusted, and thoughtful man had now lost interest in
work and a dedication to the task at hand, and instead he took to drink, profane ( 粗俗 ) language,
and other acts of impulse. His personality had dramatically changed. Harlow (1868) describes
this transformation, “His contractors, who regarded him as the most efficient and capable
foreman in their employ previous to his injury, considered the change in his mind so marked that
they could not give him his place again. The equilibrium or balance, so to speak, between his
intellectual faculties and animal propensities, seems to have been destroyed. He is fitful,
irreverent, indulging at times in the grossest profanity (which was not previously his custom),
manifesting but little difference for his fellows, impatient of restraint or advice when it conflicts
with his desires…In this regard his mind was radically changed, so decidedly that his friends and
acquaintances said he was “no longer Gage”” (13, 14). Gage was removed from his position with
the rail company. He went on to live twelve additional years after the incident, employed first by
a traveling carnival ( 嘉年华 ) where he displayed his physical abnormality to paying audiences,
and later as a stage coach driver in Chile.
This unfortunate event led to one of the first thorough studies of different areas of the
brain and their influence on personality, and it is one of the great case studies in the history of
scientific inquiry.
34
Phineas Gage holding the iron rod that shot up through his under-jaw area and out through the top of his skull,
altering his personality and changing his behavior (Wikipedia Commons).
Mixed-Methods Research
Mixed-methods research involves combining quantitative ( 定量 ) and qualitative ( 定性 ) research

methods to create a more all-encompassing understanding of a phenomena ( 现象 ). It entails
integrating ( 整合 ) more than one method, data source, testing strategy, or source of information
within a study. It can be considered a type of experimental triangulation ( 三角 ) or multimethod
operation, wherein a more holistic ( 整体 ) understanding of a phenomenon is generated from the
inclusion of two or more different information sources.
Study Questions:
1) What is the difference between qualitative research and quantitative research?

2) What does it mean when a researcher is being “objective” or “subjective”? How does this
relate to qualitative research?
3) Regarding surveys and questionnaires, what is the difference between close-ended
questions and open-ended questions? What are the different benefits and drawbacks of
close-ended and open-ended questions?
4) Field studies are conducted in what type of environment?
5) Within qualitative research, what is the difference between the complete observer and the
complete participant?
6) What is mixed-methods research?
7) Do you use qualitative research within your field/department? If yes, how is it used? If
no, why is it not used?
35
Conclusive / When there is a resolution or finality to an experiment.

Count / The process of determining the numerical amount of something.
Data / Pieces of information that are often numerical in identity.
Descriptive / A detailed account or interpretation of phenomena; often displayed visually.
Dispassionate / Seeking truth and facts, while being detached from subjective opinion.
Exploration / Seeking, searching, and questioning.
Generalize / The ability for the sample to speak for the population from which the sample was
drawn.
Holistic / All encompassing; incorporating many variables or experiences into an interpretation
or study of something.
Hypothesis / Proposed explanation for a phenomena or event.
Impressionistic / Someone’s personal, usually subjective, understanding or explanation of a
phenomena.
Integration / Merging different sources of information together to better explain a phenomenon.
Interpretive / A subjective or personal understanding of a phenomena.
Interview / A communication channel in which a subject will provide a researcher with
information.
Measure / Quantitatively record the amount of something.
Multimethod / Collecting data and information using multiple sources.
Nature / Empirically derived understanding of a phenomena; expressed constitution or makeup
of something.
Objective / Unbiased, dispassionate, and non-subjective.
Observation / Using scientific processes to record and collect information.
Paradigm / A guiding framework from which to operate.
Patterns / Predictable behaviors or results.
Precise / Accurate; without error.
36
Probing / Exploring, questioning, and looking for answers.
Procedure / Operational framework for conduct, usually in a step-by-step formation.
Qualitative / Studies that analyze words rather than numbers or statistics.
Reflexive / Considering a subject or phenomena while being aware of one’s own cultural bias
and subjective nature.
Standard / Formal; legitimate.
Statistics / A mathematical method used to distinguish relationships among variables.
Structure / A regulatory mechanism useful to increase the effective functioning of operations.
Subjective / An interpretation of reality founded on one’s personal understanding or one’s
experience.
Survey / An information gathering technique, usually involving questioning and interviewing
subjects.
Systematic / Regulated, procedural, and prearranged.
Triangulation / Analyzing a phenomenon from different quantitative and/or qualitative
perspectives.
Unstructured / Without formal guidelines or regulatory mechanisms.
puzzle:
37
ACROSS
1 Pieces of information that are often numerical in identity.
11 Merging different sources of information together to better explain a phenomenon.
12 Someone’s personal, usually subjective, understanding or explanation of a phenomena.
13 A subjective or personal understanding of a phenomena.
DOWN
1 A detailed account or interpretation of phenomena; often displayed visually.
2 Quantitatively record the amount of something.
3 Proposed explanation for a phenomena or event.
4 When there is a resolution or finality to an experiment or to the results of an experiment.
5 Seeking truth and facts, while being detached from subjective opinion.
6 A communication channel in which a subject will provide a researcher with information.
7 Seeking, searching, and questioning.
8 The ability for the sample to speak for the population from which the sample was drawn.
9 All encompassing; incorporating many variables or experiences into an interpretation or study
of something.
10 The process of determining the numerical amount of something.
38
search puzzle:
1. A communication channel in which a subject will provide a researcher with information.

2. Quantitatively record the amount of something.
3. Collecting data and information using multiple sources.
4. Empirically derived understanding of a phenomena; expressed constitution or makeup of
something.
5. Unbiased, dispassionate, and non-subjective.
6. Using scientific processes to record and collect information.
39
7. A guiding framework from which to operate.
8. Predictable behaviors or results.
9. Accurate; without error.
10. Exploring, questioning, and looking for answers.
11. Operational framework for conduct, usually in a step-by-step formation.
12. Studies that analyze words rather than numbers or statistics.
13. Considering a subject or phenomena while being aware of one’s own cultural bias and
subjective nature.
14. Formal; legitimate.
Terms: count, data, descriptive, dispassionate, generalize, holistic, hypothesis, integrate,

interviews, measure, paradigm, patterns, precise, qualitative, and unstructured.
1. The researcher would __________ the number of times the mice turned in the wrong
direction.
2. The __________ was run through statistical processes to determine if there were differences
between the groups.
3. The student’s thesis was a __________ account of his time spent living in the rainforest.
4. His position was derived from a __________ review of the literature and an analysis of the
relevant data, it was not subjective or motivated by personal concerns.
5. If the sample is unlike the population, it is doubtful that the researcher will be able to
__________ the results of the experiment to that population.
6. Chinese medicine often considers the __________ nature of the individual; it considers the
entirety of the individual in the healing process.
7. The researcher’s __________, that the new therapy would be more effective than the regular
therapy, was based on a preliminary test of the effectiveness of the therapy.
8. When developing surveys, a researcher may want to __________ old survey questions and
new survey questions to maximize data collection.
9. The __________ were a disorganized failure. The researcher regularly went off-topic, argued
with subjects, and randomly skipped necessary questions.
10. One of the main requirements of a true experiment is having the capacity to __________ the
dependent variable.
40
11. __________ shifts have been witnessed throughout the history of science; wherein new
theories are developed that usurp old theories—often after great struggle among the
advocates of the theoretical platforms.
12. Much of qualitative research involves finding recurring __________ in the open-ended
responses of subjects.
13. The researchers instrument was not __________; it failed to accurately record the correct
measurement.
14. __________ studies often consist of ethnographies, case studies, and focus groups.
15. Ethnographic and exploratory studies are often __________ in nature, wherein a researcher
will enter an environment and work to understand what is happening with no clearly defined
direction in mind.
Chapter 5: Independent Variables and Dependent

Variables
Independent variables and dependent variables work in conjunction with each other and
represent the two most fundamental elements of experimentation. Most experiments contain at
least one independent variable and one dependent variable.
The Independent Variable
The independent variable is the variable that the researcher changes or manipulates ( 操纵 ) to
determine how this change influences a dependent variable. The independent variable is the
cause (原因), input, experimental stimuli (刺激), treatment, or intervention. It is the cause ( 造成)
of the change in the subjects or the cause of the difference between the experimental groups, and
the dependent variable is the effect.
When writing up a research paper, the independent variable should be clearly, and
operationally, defined. It’s necessary to employ operational definitions so that those who read the
paper fully understand the makeup and nature of the variable under analysis, and how the
variable was used within the experiment (how it was manipulated, measured, and incorporated
( 合并 ) into the experimental process). Dane (1990) defines operational definitions as, “Concrete
representations of abstract theoretical concepts” (33).
Levels Within the Independent Variable
Within the independent variable there are usually two or more “levels.” These levels consist of
the different treatments or inputs, and one (or more, but almost always one) of these levels may
take the form of a control group or a placebo (安慰剂) group.
To illustrate, one independent variable (let’s say a new type of medicine) with three
levels in the independent variable may look like this: The independent variable is a new medicine
that the researcher wants to test. The three levels are as follows: Level 1) 1000mg of the new
41
medicine, Level 2) 500mg of the new medicine, and Level 3) No medicine (the control group).
Each level of the treatment is generally applied to a different group of subjects, and their
responses on a certain metric ( 度量 ) or variable (the dependent variable) would be measured.
Here there are three levels within the independent variable: two treatment groups and one control
group. The control group took no medicine at all, but often, rather than take nothing at all, the
control group will be provided a placebo and renamed the placebo group.
The Placebo (安慰剂) Group
A placebo (安慰剂) is a control (subjects are usually given sugar pills, colored water, etc., instead
of the actual treatment) that is meant to replicate ( 复制 ) the feeling or sensation of undergoing a
treatment (consuming medicine, undergoing a kind of therapy, etc.), but the placebo does not
actually possess the pharmacological ( 药理 ) or active qualities that constitute ( 构成 ) the
treatment that is given to the treatment group(s). The placebo is used in modern medicine to
remove the anticipatory ( 预期 ) effects, and other psychologically and physiologically ( 生理 )
manufactured effects (e.g. the psychological effects of interacting with doctors/researchers, being
handed medicine, unwrapping the wrapper of the medicine, putting the medicine in one’s mouth,
etc.), of simply taking a treatment from the actual pharmacological effects of the treatment (e.g.,
the biochemical reactions taking place between the treatment and the body because of the active
ingredients encompassed within). Once these two elements are separated, any anticipatory ( 预期)
effects associated with undergoing a treatment versus the actual effects of a treatment, a better
understanding of the true effects of the treatment itself can be obtained.
Factorial Designs
When an experiment has two or more independent variables it becomes a “factorial design.” The
independent variables are now referred to as “factors,” and they possess two types of effects:
main effects and interaction effects. The main effects are the effects occurring within one
independent variable itself on the dependent variable, just as in a one-independent variable
design. The interaction effect is the effect of the independent variables in combination as they
produce experimental results.
The Dependent Variable
The dependent variable is the variable that the researcher observes or measures after the
introduction of the independent variable. The dependent variable is the effect, output, outcome,
or result, predicated ( 断言 ) upon the influence of the independent variable. The dependent
variable “depends” on the independent variable; it is measured as it changes in response to the
Often, the results recorded in the dependent variable are compared (the different
experimental groups are compared to one another) to determine if significant differences among
the experimental groups are present. If there are differences in the dependent variable, and these
differences conform to the alternative hypothesis, then the researcher has obtained evidence that
the treatment is effective.
42
Like the independent variable, the dependent variable should be clearly and operationally
defined. As previously stated, it’s necessary to employ operational definitions so that readers
fully understand the makeup and nature of the variable under analysis, and how it will be used
within the experiment (how it will be measured, how it will be incorporated ( 合并 ) within the
experimental process, etc.).
Consider the previous example regarding the new medicine with three levels of the
independent variable (the 1000mg of the new medicine group, the 500mg of the new medicine
group, and the control group). Let’s say that the new medicine is designed to reduce headaches in
people who suffer from constant headaches. The independent variable (the medicine) is split into
three groups to the test its effects on headaches. The different groups receive the treatments
simultaneously over the course of several days, during which “headache time” is measured (the
dependent variable). When this is complete the average headache time for each group is
compiled and the groups are compared to determine if there is a difference in medicine
effectiveness among the groups (comparing the dependent scores to determine if there is a
significant difference between the experimental groups). If there is a significant difference in
“headache time” (the dependent variable) among the experimental groups, and this difference
conforms with the alternative hypothesis (that the medicine will reduce the headaches), then a
significant effect may be claimed.
Controlling Extraneous (无关) Variables within the Research Environment
Controlling extraneous ( 无关 ) variables involves holding everything within the experimental

environment constant ( 不变 ) or unchanged except for the independent variable. In effect, the
researcher diagrams (可图示) the different levels of the independent variable and applies them to
the subjects, while keeping everything else in the experimental environment the same. It involves
removing, or minimizing, confounding (混杂) variables from the experimental environment.
Experimentation is often seeking to determine relationships among variables ( 变量 ), and
all other unnecessary variables (not of interest in the experiment) that could potentially influence
these relationships should be controlled (removed or held constant ( 不变 )). By controlling or
removing extraneous ( 无关 ) variables the researcher can eliminate certain alternative
explanations for the results of the study. Myers and Hansen (1997) define extraneous variables
as, “factors that are not the main focus of the experiment. They are neither intentionally
manipulated independent variables nor dependent variables measured as indexes of the effect of
the independent variable. They can include differences among subjects, equipment failures,
inconsistent instructions—in short, anything that varies” (167).
By controlling extraneous ( 无关 ) variables, the researcher may increase reliability ( 可靠
性), increase internal validity, reduce error, and reduce spurious (杂散) results.
It’s important to note that a certain degree of error within experimentation is to be
expected (naturally, this degree is dependent upon the type of experiment conducted, the type of
science explored (e.g. chemistry versus psychology), etc.). Dane (1990) illustrates this idea,
“Whenever we observe something, we make errors; period, no exceptions, ever. The errors come
from selecting what to observe and interpreting what we observe, as well as from the act of
observation itself. We cannot avoid error entirely, but we can attempt to reduce error to a
minimum and be aware of error that we have not been able to eliminate” (28).
43
Confounding (混杂) Variables
Within experimentation, confounding variables are unwanted variables that may influence the
relationship between the independent variable and the dependent variable, and, thus, may change
the results of an experiment. If there are confounding variables influencing the levels of the
independent variable at different rates or in a systematic (nonrandom) way, the outcome of the
study may be in error. Does the independent variable (X) cause ( 造成) a change in the dependent
variable (Y)? Or is there a third variable, (Z), engaging the levels of the independent variable in a
way that alters the results. The researcher works to ensure that a third variable is not confounding
(混杂) the results and causing the changes that are produced.
Sources of Confounding ( 混杂 ): Measurement Instruments, Situational Characteristics, and

Personal Characteristics
Measurement Instruments
Measurement instruments may confound experimental results in the following ways: 1)

Researchers and research assistants may join an experiment with little understanding of the
measurement tools that are employed in the experiment, thus producing inexact results, 2)
measurement instruments may get worn down and corroded over time and become less exacting,
3) different versions of the measuring instruments may be used on different groups or at different
times in the experimentation process, which may produce greater variation and less precision
within experimentation, and 4) the researchers fail to provide a precise definitional account of
the measurement process before experimentation.
Employing knowledgeable people to operate the experimental instruments, ensuring a
consistent (一致性 ) recalibration ( 重新校准 ) of instruments over time, using one version or style
of measuring instrument consistently throughout the experimentation process, and producing
precise definitional accounts of the measurement process, can reduce confounding ( 混杂) effects
induced by measurement instruments. It’s never going to be perfect, there will likely be some
form of human and instrument error in the measurement process, but researchers should work to
minimize this as much as possible through scrupulous (一丝不苟) oversight.
Situational Characteristics
Holding the research environment static ( 静态的 ) or unchanged for the different experimental
treatments can often control for the confounding effects of different situational characteristics.
The experimental environment should remain nearly identical for all the different experimental
groups within the independent variable. Any significant environmental differences or situational
differences among the different groups may threaten the integrity ( 诚信) of the study and change
the results of the study.
For instance, let’s say one is testing a new treatment on mice and there are two groups of
mice for the study, each group located in a different room. Group one receives the treatment and
group two is the control group. The situational characteristics, such as the temperature, lighting,
size, smell, etc. of the rooms, should be as similar as possible, so they do not alter the behavior of
the mice.
44
In addition, there are other variables that one may not consider that may have the
potential to influence a study, such as the gender of the researchers involved. Believe it or not,
mice and most other mammals (including humans) tend to respond differently to researcher’s
dependent upon the researcher’s gender (it possibly has something to do with pheromones).
Thus, it’s often valuable to use only one gender of researcher for all experimental groups when
the researchers interact with the subjects (or there could be vacillating schedule where one
gender interacts with the subjects on one day and the other gender interacts with the subjects on
the next day).
Removing any unnecessary variability or potential variability within the research
environment is a valuable means to reduce the effects of different situational characteristics. It’s
usually best to remove anything from the research environment if it does not serve a purpose in
the study, as it can only act to cause variability in the results. Single and double-blind studies are
also important preventative measures to institute within certain forms of experimentation to
reduce the likelihood that the subjects and the researchers know which subjects were designated
to which experimental group.
Personal Characteristics
People possess individual differences that could influence the results of a study—from physical
differences to psychological and character differences. These differences could influence the
effectiveness of a treatment in different ways, possibly altering the results of a study or making it
more difficult to determine if a treatment is effective (greater personal variability may make it
more difficult for a researcher to find a difference between the groups if a difference does indeed
exist).
The randomization ( 随机 ) of subjects into the different experimental groups (random
assignment), together with large sample sizes, can usually control for the confounding effects of
subject differences. When subjects are randomly ( 随机 ) assigned to different experimental
groups, and there is a large enough sample size, it can generally be assumed that the
experimental groups are statistically equal prior to the introduction of the independent variable.
Certainly, there are going to be differences among individual people in the groups, but,
with a large enough sample size and random assignment, those differences will most likely equal
out and it can be assumed that the different groups are equivalent.
Another method that is used to minimize some individual differences, to reduce personal
variability and obtain a clearer picture of the effects of an independent variable under certain
conditions, is to limit the variability of the population ( 人群 ) under study. To illustrate, if a
researcher is studying the effects of a new teaching method with university students in mind, it
may be best to only conduct the research with university students, rather than with the entire
adult population or some other larger population. This is because the characteristics of this group
will likely be less variable than if a larger population (all students, all people over 18 years of
age, etc.) is used. Usually, the larger and more expansive the population from which the sample
is drawn, the greater the personal variability. So, if the population from which the sample is
drawn is narrowed down to fit the population that is intended use the independent variable, the
experiment may prove to have less variability and clearer results. The downside of this technique
is that the ability to generalize the results to the larger world is more limited (e.g. in the case of
the new teaching method with university students in mind, the researcher would likely be limited
45
to generalizing the results back to the university population from which the sample was drawn,
and not to the larger community because the sample was not taken from the larger community).
Study Questions:
1) How are independent variables and dependent variables related to each other?
2) What are “levels” within an independent variable?
3) What is a placebo?
4) What does it mean to “control extraneous variables”?
5) What are confounding variables?
6) How can the confounding effects of subject differences (personal characteristics) be
controlled?
Cause / Produces an “effect” on the dependent variable.

Confounder / An outside variable that may disrupt the results of an experiment.
Constant / Unchanged; stable; when the researcher wants to keep all variables unchanged except
for the independent variable.
Control / It is this group within the independent variable that remains unchanged.
Dependent / The researcher measures this variable after manipulating the independent variable.
Effect / Another name for the outcome, output, or result.
Error / The distance between the obtained score and the true score.
External / The environment that exists outside of the experiment.
Extraneous / Variables outside of an experiment that may negatively influence the experiment.
Input / Another name for the independent variable, treatment, or intervention.
Independent / The researcher changes or manipulates this variable.
Intervention / Another name for the independent variable, treatment, or input.
Manipulate / The researcher does this with the independent variable, creating different levels for
testing.
46
Measure / Quantitatively determine and record the amount, length, etc. of something.
Observe / To watch and record.
Outcome / Another name for the result, effect, or output.
Output / Another name for the result, effect, or outcome.
Placebo / A fake treatment that may induce subjects into believing that they are receiving a
proper treatment.
Randomization / Arranging subjects so that they are selected by chance.
Reliability / The researcher is repeatedly obtaining the same results time after time.
Spurious / An incorrect or tenuous interpretation of the relationships among variables.
Stimulus / Initiates a response or reaction; also known as the treatment or intervention.
Treatment / Also known as the stimulus or intervention; often associated with medicine or
therapy.
Unchanged / Steady, unvarying, and static.
puzzle:
47
ACROSS
4 The researcher changes or manipulates this variable.
7 Another name for the outcome, output, or result.
8 Unchanged; stable; when the researcher wants to keep all variables unchanged except for the
9 The environment that exists outside of the experiment.
10 The distance between the obtained score and the true score.
11 An outside variable that may disrupt the results of an experiment.
12 Produces an “effect” on the dependent variable.
DOWN
1 Another name for an independent variable, treatment, or input.
2 It is this group within the independent variable that remains unchanged.
3 Another name for an independent variable, treatment, or intervention.
5 Variables outside of an experiment that may negatively influence the experiment.
6 The researcher measures this variable after manipulating the independent variable.
48
search puzzle:
1. The researcher does this with the independent variable, creating different levels for
testing.
2. Quantitatively determine and record the amount, length, etc. of something.
3. To watch and record.
4. Another name for the result, effect, or output.
5. Another name for the result, effect, or outcome.
6. A fake treatment that may induce subjects into believing that they are receiving a proper
treatment.
7. Arranging subjects so that they can be selected by chance.
8. The researcher is repeatedly obtaining the same results time after time.
9. An incorrect or tenuous interpretation of the relationships among variables.
10. Initiates a response or reaction; also known as the treatment or intervention.
49
11. Also known as the stimulus or intervention; often associated with medicine or therapy.
12. Steady, unvarying, and static.
Terms: cause, confounding, constant, control, dependent, effect, error, external, intervention,
manipulate, measure, placebo, randomization, and reliability.
1. The independent variable is the __________, and the dependent variable is the effect.
2. When a serious __________ variable exists within an experiment, differences among groups
may not be attributable to the independent variable.
3. In a true experiment, it’s important that the experimenter hold all variables unrelated to the
independent variable __________ within the experimental environment.
4. The __________ group is often considered the baseline group because it is the normal
situation/condition.
5. The __________ variable is measured after the implementation of the independent variable.
6. True experiments can make “cause and __________” claims.
7. Measurement __________ may produce great variability in the results and make it difficult
to find differences between group means if differences do indeed exist.
8. __________ validity is usually high when a sample is large and adequately reflects the
population.
9. The __________ caused a significant change in the subject’s behavior, leading the researcher
to conclude that the therapy is likely effective.
10. We __________ the independent variable to create different groups (e.g., treatment and
control) for testing.
11. We __________ the dependent variable to determine if there are differences between the
groups.
12. The __________ group produced little change in the subject’s behavior, lending greater
evidence that it was the medicinal properties of the medicine, and not the simple act of taking
the medicine, that induced behavioral change.
13. The __________ process was successful. The pre-test determined that there were no
significant differences between the groups prior to the implementation of the intervention.
14. The __________ of the experiment was put into question when the researcher failed to find
similar results on subsequent tests.
50
Chapter 6: Between Subjects Designs and Within
Subjects Designs
Between subjects designs and within subjects designs represent the two most basic forms of
experimental design. Between subjects designs are the most commonly used and recognized type
of experimental design. Though, it’s important to understand both forms of design as within
subjects designs are not uncommon, and it’s useful to become familiar with its structure and
operation to better understand experimentation in general.
Between Subjects Designs
A between subjects design is an experimental design where there are two or more experimental
groups within an independent variable, and each group receives only one specific treatment or
the control (i.e., each subject is placed in one and only one experimental group and that group
receives only one kind of treatment). After the introduction of the independent variable, the
dependent variable is measured, and the different groups are compared to each other to examine
the effectiveness of the treatment. This research attempts to determine if there is a difference
between groups (usually a difference among the means of the different groups).
Stated differently, subjects are placed into different experimental groups (usually
randomly ( 随机 ) assigned to the different groups), each group receives one specific treatment or
the control (this is the control group or the placebo ( 安慰剂 ) group), and, after the dependent
variable is measured, the groups are compared.
Between subject’s experiments are often blinded (single-blind or double-blind), involve
pretests and posttests, are controlled, employ the random assignment of subjects, and incorporate
(合并) a control group (a standard control group or a placebo group).
Examples of the between subjects design include:
Randomized Pretest-Posttest Control Group Experimental Design:
R O1 X O2
R O3 O4
R = Random Assignment
O = Observation
X = Treatment
In this between subjects design the subjects are randomly assigned to different groups, the
treatment group(s) and the control group, and there is a pretest and a posttest. Random
51
assignment is used to create greater equality among the groups (so that one group is not
statistically different from another in a general way, on the dimension being studied, or both).
The sequence of the experiment is as follows: subjects are randomly assigned to different
groups, a pretest is provided to ensure equality among the groups, the treatment(s) are provided
to one or more groups while one group remains the control group (the group receiving nothing or
the placebo), the subjects are measured on the dimension of concern, and group differences are
determined through statistical processes using the results of the dependent variable.
A Note on Pretests
Pretests are generally used to help ensure that groups are statistically equal on the dimension
being studied by testing their level of equality before the introduction of the independent
variable.
The pretest has other functions as well. If subjects are dropping out of a study, either
systematically or randomly, the pretest can provide valuable information as to the makeup of the
subjects who drop out in relation to those who remain in the study. The pretest can also be
compared to the posttest to provide some idea as to the effectiveness of a treatment.
Though, it’s important to reiterate, a treatments effectiveness is not determined by
looking at differences between the pretest and the posttest in a between subjects design, instead,
a treatments effectiveness is determined by looking at differences between the groups regarding
the scores on the dependent variable only (in other words, looking at differences between the
groups on the posttest only). After the pretest is provided to the subjects, the treatment(s) are
provided, the effects of the treatment are then measured (which is the dependent
variable/posttest), and potential differences between the groups, as determined by scores on the
dependent variable, are assessed.
Randomized Posttest-only Control Group Experimental Design:
R X O1
R O2
O = Observation
X = Treatment
Under this between subjects design, there is no pretest, only the randomized assignment of
subjects into experimental groups. When random assignment is used and there is a large enough
sample ( 样品 ) size, the researcher can assume that the groups are statistically ( 统计 ) equivalent,
and, thus, there may be no pressing need for a pretest. A major purpose of a pretest is to ensure
that the groups are equivalent prior to the introduction of the independent variable, but, as
mentioned, this can be assumed with random assignment and a large sample size. There are
times when a pretest is unnecessary, impractical ( 不切实际 ), or detrimental to a study—it
requires more time and money to institute a pretest (this is of greater concern in large scale
studies), and, importantly, it may influence the results in the posttest phase either directly or in
52
combination with the treatment (The Solomon four-group design will explain how this problem
can be detected).
Solomon Four-Group Design
The Solomon four-group design is a type of between-subjects design that does two things: 1)
determine if the pretest influenced the results of a study (the posttest), and 2) test how the
independent variable influenced the dependent variable (essentially a traditional between
subjects design).
The researcher does not want the pretest to influence the posttest. The treatment is the
only variable that should influence the posttest. So, it is valuable to gauge whether, or to what
extent, the pretest may be impacting the results of a study. The Solomon four-group design looks
at the influence of the pretest on the posttest in two ways: 1) whether the pretest directly
influences the posttest, and 2) whether there is an interaction between the pretest and the
treatment that influences the posttest.
1) A direct influence of the pretest on the posttest:

The Solomon four-group design looks to determine whether the pretest directly influenced
the subject’s posttest results by comparing the results of a group that underwent a pretest
(this group does not receive a treatment) with a group that experienced the same
circumstances without a pretest (this group also does not receive a treatment). This design
functions like this:
Group1 R O O
Group2 R O
O = Observation
As shown, there are no treatments in this portion of the design. The experimenter is looking
to see if the pretest will influence the posttest directly. If the posttest of Group 1 is
significantly different from the posttest of Group 2, the pretest may be directly influencing
the results of the study. Let’s move to the second half of the design.
2) An interaction between the pretest and the treatment that influences the posttest:
The Solomon four-group design also looks to see if the pretest interacted with the treatment
to influence the results. It does this by comparing the group that experienced a pretest and the
treatment with a group that experienced the same experimentation without a pretest (this
group also received the treatment). The second half of this design functions like this:
Group1 R O X O
Group2 R X O
O = Observation
X = Treatment
53
The experimenter is looking to see if the interaction of the pretest and the treatment is
influencing the posttest. If the posttest in Group 1 is significantly different from the posttest
in Group 2, the pretest may be interacting with the treatment in a way that is influencing the
results of the study.
In the end, there are four groups in all. When the two halves are put together, the design
looks like this:
The Solomon Four-Group Design:
Group1 R O X O
Group2 R O O
Group3 R X O
Group4 R O
O = Observation
X = Treatment
The results of Group 1 are compared to the results of Group 3 to analyze the interaction
effects of the pretest and treatment on the posttest.
The results of Group 2 are compared to the results of Group 4 to analyze the direct effect of
the pretest on the posttest.
The Solomon four-group design also functions as a large between subjects design, wherein
the groups are compared to determine the effectiveness of a treatment.
Within Subjects Designs (also known as repeated (重复) measures designs)
A within subjects design is an experimental design wherein all subjects receive all levels of the
independent variable. Put differently, all the participants receive all the different treatments (and
any controls), and responses to these treatments are measured after the implementation ( 实施 ) of
each treatment. There is usually a sequence ( 顺序 ) of treatments for the subjects to undergo (the
sequence ( 顺序 ) can be deliberate or randomized), during which subjects are measured on a
specific metric ( 度量 ), and these measurements are compared with one another to look for
differences between measurements. To reiterate, subjects receive a sequence ( 顺序 ) of all the
different treatments and are measured after each treatment.
To illustrate, let’s say that a researcher has randomly selected one group of subjects
suffering from high blood-pressure, and she wants to conduct a within subject’s design to test a
new medicine (treatment 1 has 100mg of the new medicine; treatment 2 has 10mg of the new
medicine). The group could be given a pretest, then the group could be provided with treatment
1. The group would then be measured on the metric ( 度量 ) of study (blood-pressure). Perhaps
after some time has passed, the group would then be given treatment 2, and then the responses
would be measured. These measurements would then be compared with each other: the pretest
compared to the measurement after treatment 1; the pretest to the measurement after treatment 2;
54
and the measurement after treatment 1 to the measurement after treatment 2. This example looks
like this:
O1 X1 O2 X2 O3
O = Observation
X = Treatment
The measurements would be compared to determine if there is a significant change due to a

treatment (to see if the measurements are significantly different from one another): O1 compared
to O2; O2 compared to O3; O1 compared to O3.
The introduction of treatments could also be randomized ( 随机 ) among different groups or

among different subjects to identify if there are any carryover effects. The design might look
something like this:
Treatment Treatment
Group1 R 1 2 Sequence (顺序) of introduction
Group2 R 2 1
Which would translate into this visual depiction:
Group1 R O1 X1 O2 X2 O3
O = Observation
X = Treatment
Other Examples of Within Subjects Designs:
Treatment1 Treatment2 Treatment3

Subject1 1 2 3 Sequence (顺序)
Subject2 1 2 3
Subject3 1 2 3
Subject4 1 2 3
As you can see above, all subjects receive all treatments, and they receive them in the same
sequence.
55
Treatment1 Treatment2 Treatment3
Group1 1 2 3 Sequence (顺序)
Group2 2 3 1
Group3 3 1 2
Group4 1 3 2
Group5 2 1 3
Group6 3 2 1
As you can see above, all groups receive all treatments, and they receive them in a different
sequence to test for any carryover effects (carryover effects will be discussed soon).
The Interrupted Time-Series Design:
O1 O2 X1 O3 O4 X2 O5 O6 X3 O7 O8
O = Observation
X = Treatment
As indicated, there is only one group and a long duration of time (with numerous
observations/measurements) between treatments in this design. This is to more thoroughly
examine the impact of the treatments and to provide a greater delay ( 延迟 ) between treatments.
This way, researchers can better determine the aftereffects of the treatment and be better assured
that the participants have returned to their baseline ( 基线 )/normal state prior to the onset of the
next treatment (as means to reduce or eliminate any carryover effects).
Carryover Effects
A major concern involving within subjects designs is that the effects of previous treatments will
“carryover” into future treatments, effectively merging ( 合并) together to distort the results of an
experiment. In the design below, you can see how the effects of X1 could potentially carryover
into X2, thus distorting the measurement of 03.
O1 X1 O2 X2 O3
O = Observation
X = Treatment
The carryover effect can be answered in two ways: Counterbalancing and delaying the time
between treatments
56
Counterbalancing
In counterbalancing the treatment order is randomized ( 随机) for each subject or randomized for
a group of subjects (each group receives all treatments in a uniquely randomized fashion); or
there is a unique sequence for each subject or group of subjects that is distinct from other
subjects/groups. This randomization method has the potential to reveal where a carryover effect
may be occurring, if such an effect exists, so that it can be remedied ( 补救 ) through delaying the
time between treatments, switching to a between subjects design, or through other mechanisms.
When the sequence of treatments is randomized among the subjects or the groups
(counterbalancing) it will become possible to determine if carryover has occurred. By using the
design below, the carryover effect can potentially be detected by looking at differences between
O3 and O5. If the measurements of O3 and O5 are significantly different there may be carryover
in Group 1, and the same goes for the relationship between O2 and O6 (if the group scores are
significantly different there may be carryover in Group 2).
O = Observation
X = Treatment
Delaying (延迟) the Time Between Treatments
To reduce or eliminate any carryover effects, it’s often valuable to delay ( 延迟 ) the onset of
future treatments until the effects of the previous treatment have worn-off or have been removed.
Less complex organisms require less of a delay. Mice usually only require a day or so for a
treatment to wear-off, while for humans the treatment/therapy/memory exercise, etc. may require
a much longer time-frame (weeks, months, or it may never completely wear-off).
Practice Effects and Additional Concerns in Within Subjects Designs
Practice Effects
The more that subjects engage in an experimentation process, the more they interact with the
testing materials, and the more they become familiar with the people involved in an experiment,
the more they will become comfortable and competent with the materials, tests, and experimental
process, thus, potentially increasing their efficiency and potentially improving their scores. The
more one is comfortable with a test or a testing procedure, as one engages with it over a series of
tests, the greater the likelihood that one’s score will improve as one’s nervousness ( 紧张 ) and
fears are reduced, and the question structure and response structure are incorporated ( 合并 ) into
one’s memory. Over time, and with greater experience, one may naturally adapt and improve on
whatever task is set before oneself in an experimental situation. This may influence the results of
57
an experiment in that the researcher may believe that the subjects are improving due to the
treatment, but it may be a result of practice (or practice may play some role in the improvement).
Additional Concerns
There are other issues that may alter the effectiveness of this experimental method. The first
major concern is time. Within subjects designs commonly require a greater investment of time
from the subjects, which can dissuade ( 劝阻 ) subjects from participating or produce subject
dropout.
Secondly, within subjects designs, as a product of this greater investment in time and
energy, can induce greater subject fatigue, stress, and a feeling that they “just want to finish this
as fast as possible.” Naturally, this may produce different results compared to those more eager,
rested, and interested subjects (in, for instance, the often less demanding and less time
consuming between subjects designs; remember that in a between subjects design the participants
receive only one treatment, so the demand on their time and energy is likely less when compared
to a within subjects design).
Study Questions:
1) What is a between subjects design?

2) What is a within subjects design?
3) What is a Solomon four-group design?
4) When can it be assumed that the experimental groups are equal prior to giving them the
treatments (so that a pretest is likely unnecessary)?
5) What are carryover effects in the within subjects design?
6) What can be done to minimize carryover effects?
7) What are practice effects in the within subjects design?
Blind / Purposefully withholding information from subjects and experimenters for practical
reasons.
Carryover / The effects of one treatment merge into the effects of subsequent treatments,
distorting the results of a study.
Compare / Contrast; examine one thing in relation to another thing.
58
Counterbalancing / Randomizing the sequence of treatments provided to identify and combat
carryover effects.
Delay / Increase the time between the onset of one treatment and subsequent treatments.
Placebo / A fake treatment that may produce psychological or physiological effects on subjects
to gauge the impact of subjects taking any kind of treatment.
Posttest / A test or measurement conducted after the treatment is applied.
Pretest / A test or measurement conducted before the treatment is applied.
Randomized / Subjects have a known, and usually equal, chance of being selected for a sample.
Repeat / Recurring; something happens over and over again.
Sequence / Something takes place in a predetermined order.
Treatment / The intervention that may produce a change or reaction.
puzzle:
59
ACROSS
3 The effects of one treatment merge into the effects of subsequent treatments, distorting the
results of a study.
7 Something takes place in a predetermined order.
10 A test or measurement conducted after the treatment is applied.
11 Randomizing the sequence of treatments provided to identify and combat carryover effects.
12 Increase the time between the onset of one treatment and subsequent treatments.
DOWN
1 A test or measurement conducted before the treatment is applied.
2 Subjects have a known, and usually equal, chance of being selected for a sample.
4 The intervention that may produce a change or reaction.
5 A fake treatment that may produce psychological or physiological effects on subjects to gauge
the impact of subjects taking any kind of treatment.
6 Recurring; something happens over and over again.
8 Contrast; examine one thing in relation to another thing.
9 Purposefully withholding information from subjects and experimenters for practical reasons.
60
Terms: blind, carryover, compare, counterbalanced, delay, placebo, posttest, pretest, randomized,
and repeat.
1. It is often important that the experimenters are __________ as to which subjects receive the
actual treatment and which subjects receive the placebo. This is so the experimenters do not
unconsciously influence subjects, thus, altering the results.
2. When giving subjects multiple treatments in a sequence, the experimenter can delay the onset
of new treatments to reduce the __________ effect.
3. In between subject’s designs, the main objective is to __________ group means in the
dependent variable to determine if there is a significant difference among the groups.
4. Carryover effects can be located and __________ through a randomization of the sequence
of treatments.
5. When treatments are provided to subjects sequentially, it’s often valuable to __________ the
onset of new treatments to minimize the carryover effect among treatments.
6. Throughout history, a __________ has taken many forms—sugar pills, colored water, and
bread pills are a few examples.
7. One group of subjects were given both a pretest and a __________. Due to learning
processes, it was thought that subjects may have improved on their final test because of
taking the pretest. Due to the pretest, subjects may have become more comfortable with
testing, more accustomed to the questions, they may have been thinking of the questions after
the pretest, etc. This is a threat to the internal validity of an experiment known as “testing,”
and it’s another reason why a control group is necessary. When adding a control group, it can
be assumed that both groups are experiencing this “testing” effect and, thus, cancel it out,
leaving the treatment as (hopefully) the only difference between the groups.
8. A __________ is often used to determine if the experimental groups are statistically equal on
the variable under study prior to the implementation of the treatment. Though, this test is not
always necessary. If there is a large enough sample size, and if the researchers employ
random assignment (randomly assign a sample of subjects to the different experimental
groups), it can usually be assumed that the groups are statistically equal, and, thus, this type
of testing is unnecessary.
9. With a large enough sample size and a __________ process of subject assignment (to the
different experimental groups), researchers can assume that that experimental groups are
statistically equal, and, thus, a pretest is often unnecessary.
61
10. To garner high reliability, the researcher should have the capacity to __________ an
experiment and achieve the same results.
Chapter 7: Internal Validity, External Validity, and

Reliability (可靠性)
The accuracy, generalizability, and consistency of experiments are foundational to determine if
experiments are useful and legitimate. Internal validity corresponds to accuracy, external validity
to generalizability, and reliability to consistency. Threats to the accuracy, generalizability, and
consistency of an experiment should be recognized and minimized; this chapter is an explanation
for how this can be accomplished.
Internal Validity
The basic questions with internal validity involve the integrity ( 诚信 ) and accuracy of an
experiment, and are the following: are we measuring what we want to measure? Are we
measuring what we are claiming to measure? Was it the independent variable that produced the
change in the dependent variable or was it possibly something else that caused ( 造成 ) this
change? Internal validity is largely interested in the causal link ( 链接 ) between the independent
variable and the dependent variable.
When considering internal validity, it’s valuable to consider these questions: Is the
relationship between the independent variable and dependent variable causal ( 因果 )? Does the
cause ( 原因 ) come before the effect? Are there alternative explanations for the change in the
dependent variable? Have other external/extraneous ( 无关 ) variables been ruled out that may
have changed the dependent variable? Has the experiment been controlled to the extent that the
potential for bias (偏差) and error is limited?
If the researcher can show a strong relationship between the independent variable and the
dependent variable, that the cause came before the effect, that extraneous variables were ruled
out, and that the experiment was controlled to the extent that bias ( 偏差) and error were seriously
minimized, then it could be argued that the experiment possessed high internal validity. If one or
more of these elements are weak, then the internal validity is reduced.
Threats to Internal Validity
Threats to internal validity include: selection, attrition (mortality) ( 消耗/损耗 ), maturation ( 成

熟 ), history, instrumentation, testing, regression ( 恢复 ) (to the mean), confounding ( 混杂 )
variables, experimenter bias ( 偏差 ), participant bias ( 偏差 ), diffusion ( 扩散 ), resentful ( 不满 )
demoralization, and compensatory (补偿) rivalry.
i. Selection: Were the subjects randomly ( 随机 ) assigned to the different experimental
groups? If not, the experimental groups may be different in some important way prior to
62
the onset of the treatment. Thus, the results may not be a product of the treatment, but
instead may be due to preexisting (原有) differences among the experimental groups. The
knowledge that this is a possibility will likely lower internal validity.
ii. Attrition (Mortality) ( 消耗 / 损耗 ): Subjects dropping out of a study may influence the
results of a study. This attrition ( 消耗 / 损耗 ) generally needs to be nonrandom and
systematic ( 系统 ) for it to significantly threaten internal validity. When subjects drop out
of the groups in a random fashion, the groups can be expected to remain statistically
equivalent (if the sample sizes remain large enough). When subjects drop out of the
groups in a nonrandom or systematic fashion, the groups may not remain equivalent, and
internal and external validity may be seriously compromised to the extent that the study
needs to be discarded.
For instance, when there is a certain number of subjects who drop out of a
treatment group in a systematic way (i.e., subjects that possess some distinct
characteristic are dropping out disproportionally compared to subjects who do not
possess this characteristic), and, at the same time, the control group is undergoing random
dropouts, the results may be biased ( 偏差 ), as the two groups are no longer being
compared on equal terms. The control group has now been changed to a group that
reflects those with the capacity ( 能力 ) to remain in the study, those who are not
experiencing the side effects of the treatment, those who do not have to experience the
tedious ( 乏味 ) nature of the treatment process, or any other possible explanation for the
difference in the nature of subject dropout. In other words, a study that once possessed
randomly assigned subjects no longer maintains groups that are statistically similar due to
forms of nonrandom attrition. Although, if subjects randomly drop out of experimental
groups at an equal rate, the integrity ( 诚信 ) of the study may remain intact, since the
groups may remain statistically equal.
In addition, because of nonrandom attrition ( 消耗/损耗 ), the sample ( 样品 ) may
no longer reflect the population ( 人群 ) from which it was originally drawn. With a
significant and nonrandom or systematic loss of subjects, the sample may become altered
from its original state, thus, reducing external validity (generalizability). The sample, in
effect, does not represent ( 代表 ) the population because it has been altered through some
form of systematic attrition.
iii. Maturation ( 成熟 ): Normal human developmental processes may influence the subjects,
and, thus, the results of an experiment. A study that takes a long period of time to conduct
may be susceptible to concerns that normal biological changes over time are influencing
the results of the study. Although this type of concern is relatively uncommon, there are
other bodily or biological issues that are worth considering, Dane (1990) explains, “Most
experiments do not last long enough for maturation to occur in the everyday sense of the
word—people growing older—but maturation also includes such things as fatigue,
boredom, thirst, hunger, and frustration” (80).
To illustrate, let’s say that an experimenter is studying health and fitness and
wants to develop a way to increase the muscle mass of first year university students. The
experimenter gives a single group of students a new muscle building workout routine that
lasts for three months. It turns out that after three months of using this new muscle
building routine the students do increase their muscle size. Great news, right? Maybe not.
The problem is that the students may have naturally increased their muscle size anyway
through normal developmental processes. Muscle growth cannot be attributed to the
63
treatment unless there is a control group that doesn’t receive this treatment. With a
control group incorporated ( 合并 ) into the study, the natural muscle growth can be
assumed to be statistically equal (provided there is random assignment and a large
enough sample size) between both groups (the treatment group and the control group)
with the only difference being the introduction of the treatment.
iv. History: Events outside the immediate experimental environment may influence a study,
thus, biasing ( 偏差 ) the results and producing error. Don’t let the name of this threat
(“history”) mislead you; it doesn’t mean that historical events prior to the experiment will
alter the results of a study, it means that large events outside of the experimentation
environment during the study may influence the results. These outside events can be
more localized ( 本地化 ) (e.g., a tornado in the area or the local sports team wins a
championship) or they can be more global in nature (e.g., a major military attack takes
place in some part of the world).
As mentioned, these outside events typically should occur during the experiment,
not before the experiment. For instance, in studies of depression, people may become
more depressed when unexpected catastrophic ( 灾难 ) events occur globally during the
experimentation period, in turn changing the experimental results.
A control group can be used to minimize this threat. If both the treatment group
and the control group experience the same outside events during the experiment, the only
differences between the groups would be the introduction of the treatment—therefore, the
groups can be effectively compared on the dimension being studied.
v. Instrumentation: Many instrument related issues may influence the results of an
experiment, such as: which instruments are chosen to be used in experimentation,
instrument degradation ( 降解 ) or variability due to continued usage, the precision of
instruments, the quality or edition of the instrument, etc. After repeated ( 重复 ) use, an
instrument may become less precise and more variable in its results, thus, influencing a
study—perhaps the posttest measurements are more variable than the pretest
measurements because precision is reduced over time due to instrument degradation ( 降
解 ). Or consider a scenario ( 脚本 ) wherein the pretest was less precise compared to the
posttest because the experimenters became more experienced and effective by the time
the posttest measurements were taken. Surely this is problematic.
Thus, it’s important to have a control group included in an experiment so that the
groups (the treatment group(s) and the control group) will likely experience the same
instrumentation issues, the only difference among the groups being the introduction of
the treatment. When the different groups experience the same instrumentation issues, the
groups remain equivalent, the study remains internally consistent and valid, and any
differences between the groups regarding the dependent variable could theoretically be
attributed to the treatment (given that there are no other problematic issues with the
experiment).
vi. Testing: Subjects have the potential to change their behavior when they know they are
being tested. Some examples of how testing may influence subject behavior, which in
turn may influence the results of a study, are: subjects working harder than normal to
impress the researchers; subjects putting themselves in the best and most favorable light
possible when being interviewed or surveyed (in other words, not admitting personal
flaws or personal problems, which may be important omissions for an experiment);
subjects thinking that they are helping the researchers by telling the researchers what they
64
think the researchers want to hear so that the researchers find positive results (rather than
helping the researchers, this behavior obviously does much more harm to the
experiment).
A good way to minimize this issue is by having a control group. With a control
group, the groups (the treatment group(s) and the control group) will all know that they
are being tested, and they will change their behavior due to this knowledge at a relatively
equal rate (the effects of “testing” on all groups will, in effect, cancel each other out), the
only difference between the groups being the introduction of the treatment.
In summary, subjects often change their behavior when they know that they are
being watched by a researcher or when they are receiving attention from a researcher,
thus potentially altering the results of a study. Think about how people will often change
their behavior at work when a supervisor or manager is in the immediate vicinity. When
the manager is nearby, people may sit up straighter, begin working harder, feel a sense of
anxiety, etc. Substitute a researcher in for a manager and the same phenomena presents
itself in experimentation.
vii. Regression (恢复) (to the mean): This is the propensity for extreme scores to revert ( 恢复)
closer to the mean on subsequent ( 随后 ) tests. This typically involves extreme scores or
extreme groups of subjects. Those subjects that produce extreme scores or extreme
behaviors on pretests are naturally likely to gravitate closer to the mean on future tests,
thus, influencing the results of experimentation.
To illustrate, let’s say that researchers developed a treatment to help those who
suffer from high levels of depression. The researchers select a group of people who
scored very high on depression on a pretest, and then they administer the treatment to the
group for one month. After this month, the subjects are given a posttest to measure their
levels of depression. As it turns out, their depression scores seemed to go down. So, it
seems that the treatment was successful. The problem is that more extreme scores on
pretests will often revert ( 恢复 ) closer to the mean score on subsequent ( 随后 ) tests
anyway, without the help of a treatment. Perhaps many who scored high on depression on
the pretest did so because they had just broken-up with a partner, maybe a subject’s dog
died, maybe they were unusually depressed that day for whatever reason, etc. It could be
that the subject was susceptible to depression, and during this time they were
experiencing abnormally high levels of depression that will naturally revert ( 恢复 ) closer
to the mean in the future. The point is that the researchers wouldn’t know if the subjects
reverted closer to the mean (improved) because of the treatment or through normal
processes.
Therefore, it’s important to have a control group in this case. Both the treatment
group and the control group will naturally revert to the mean, but, if the treatment was
effective, the treatment group would be expected to revert faster, more precipitous, and
closer to the mean score.
viii. Confounding (混杂 ) variables: Confounding variables are unwanted variables ( 变量 ) that
may disrupt the relationship between the independent variable and dependent variable,
thus, distorting the results. These variables should be removed or held constant ( 不变 ) so
that all experimental groups experience the confounders equally—as a result, the only
difference among the groups would theoretically be the treatment. Confounding variables
are most severe when they influence the levels of the independent variable in a systematic
or nonrandom fashion. Confounding variables are less of an issue when they are
65
influencing subjects randomly, because the influence will likely equal itself out among
the different groups. Yet, if the confounding influence is systematic or nonrandom, one
experimental group may experience greater influence by the confounding variable when
compared to another group, thus, potentially biasing ( 偏差 ) the study, and altering the
results.
ix. Experimenter bias (偏差): Experimenters possess the capacity (能力) to bias (偏差), either
unconsciously ( 不知不觉 ) or consciously ( 自觉 ), the results of a study. For instance, the
experimenter might unconsciously ( 不知不觉 ) favor the treatment group over the control
group by being happier to see the treatment group (smiling at them more, joking with
them more, being more enthusiastic ( 热情 ) and upbeat with them, etc.), they may spend
more time with the treatment group, they may put greater emphasis on the positive
aspects and minimize the negative aspects of the treatment group, etc., all of which could
potentially influence subject responses, and, thus, influence the results of the study.
This bias ( 偏差 ) may be brought on by the experimenters’ desire for the
experiment to succeed, for the treatment to be effective, for the hypotheses to be correct,
etc. Also, experimenters often have a lot of time, money, and reputation ( 声誉 ) invested
in their research, and this may be driving their conscious ( 自觉 ) or unconscious ( 不知不
觉) conduct.
It’s often valuable to employ double-blind precautions in experimentation to
minimize this experimenter bias ( 偏差 ). A double-blind study is one where the
researchers are not told which group is receiving the treatment and which group is
receiving the control (i.e., the placebo ( 安慰剂 )). Usually, under these circumstances, the
control group is provided a placebo. The placebo, aside from not possessing any active
qualities or ingredients, is made to resemble the treatment in every way, and it is
randomly distributed to participants. The placebo, along with the treatment, is often
provided to the participants through a third party unaffiliated ( 无关 ) with the study. This
third party will then keep the information regarding who received the treatment and who
received the control from the experimenters until after the data is collected. Through this
process, it is unlikely that the experimenters will know who was given the actual
treatment and who was given the placebo, thus, reducing experimenter bias ( 偏差 ).
Gomm (2008) describes the power of this type of procedure, “The archetype of the
double-blinded experiment is the randomized, double-blinded, placebo-controlled trial
which is common in testing the effectiveness of pharmaceuticals. Here it is easy to hide
to which group subjects have been allocated, since a dummy pill—the placebo—can be
made to look, taste and smell exactly like the active drug being tested” (73).
x. Participant bias ( 偏差 ): The participants perception ( 感知 ) of an experiment may
influence the results. The participants may want the results of a study to go in a specific
direction, and they will change their responses or behavior to suit their predetermined
ideas. Participants may discern the studies hypothesis, and, seeking to “help” the
experimenters, alter their responses or behaviors to bolster ( 支持 ) the likelihood that the
hypothesis is correct. Conversely ( 相反 ), the subjects may discern the hypothesis, and,
perhaps out of a distaste for a researcher, authority, the experiment itself, etc., may
consciously ( 自觉 ) conspire ( 合谋 ) to produce results that refute the researcher’s
hypothesis.
This threat can be minimized by clearly stating to the participants their obligation
to be as truthful as possible and follow the study naturally. These instructions are to be
66
provided before the study begins. Also, it’s often important that the researcher effectively
conceals the study’s hypotheses and effectively blind the treatments (by conducting a
single-blind or double-blind study) from the participants. A single-blind study should be
employed when possible so that the participants do not know which group is receiving
the treatment and which group is receiving the control (placebo).
xi. Diffusion (扩散): When a group of subjects becomes aware that they were given a special
treatment or that they were given no treatment (a placebo), they may share this
information among the different groups, thus, altering the subject’s behavior and the
results of a study.
Typically, this issue arises when a group becomes aware that it is the control
group (placebo (安慰剂) group). If, through communication with the treatment group, the
placebo group discovers that they are likely the placebo group, they may change their
behavior in a way that fundamentally alters the study. The placebo group may engage in
forms of resentful ( 不满 ) demoralization and compensatory ( 补偿 ) rivalry (detailed
below). The placebo group, with knowledge that they are the placebo group, may work
harder than they normally would to makeup for some perceived ( 感知) deficit. Naturally,
this will alter the results of an experiment. Conversely ( 相反 ), the placebo group, again,
knowing that they are the placebo group, may become disenchanted at not being selected
for what they think is the more interesting treatment group, and stop working as hard as
they normally would and take a more halfhearted attitude toward the experiment.
Also, the treatment group, were they to determine that they are the treatment
group, may become excitable and energetic in a way that is out of the normal range of
conduct. This excitability may compel ( 迫使) them to work harder than normal or compel
( 迫使 ) them to respond in a more positive way than normal, thus, biasing ( 偏差 ) the
results.
Therefore, it’s important that the experimenters do their best to prevent any
communication or interaction between the different experimental groups, and, when
possible, conduct a single-blind experiment, lest diffusion ( 扩散 ) alter the results of a
study.
At the same time, issues present themselves when groups are kept away from each
other to any great degree to reduce this diffusion. The researcher wants to keep the
groups separated and isolated so that they do not communicate with each other, but the
greater the distance between groups in both space and time, the greater the likelihood that
the research environment will be different or unequal, and, regarding time, the greater the
likelihood that personal moods or personal energy may be different, all of which may
have an influence on the results.
Regarding issues of space, if the groups are separated into different rooms in a
building for research purposes, the research environment will likely change somewhat
(the lighting, the air temperature, the objects in the room, the welcoming nature of the
room, the smell, etc. may be different), thus possibly impacting the results. If subjects are
separated into different buildings, the research environment may become even more
different, potentially altering the results even more.
Regarding issues of time, if one group is scheduled to come to the laboratory on a
Friday afternoon and another is scheduled for a Monday morning, to take a more extreme
example, the behaviors of the subjects may be somewhat different due to people’s normal
responses to the day of the week and time of day, thus potentially impacting the results.
67
Therefore, it’s important that the researcher weigh the benefits of reducing diffusion
through group distancing, with the potential experimental pitfalls of separating groups
into different environments or into different time periods.
xii. Resentful ( 不满 ) demoralization: Through diffusion ( 扩散 ), the placebo group may
discover that it is likely the placebo group and become resentful ( 不满 ) for not being
chosen for the special treatment group (treatments, within human testing, are usually
hypothesized to provide some benefit or advantage for the participants). From this
resentment ( 不满 ), people may become demoralized, downcast, less interested in the
study, less interested in performing tasks, they may give up on the study, etc. Because of
knowledge acquired through diffusion ( 扩散 ), the results of the study may be invalid or
significantly compromised (危及).
xiii. Compensatory ( 补偿 ) rivalry: Through diffusion ( 扩散 ), the placebo group may discover
that it is likely the placebo group, and, instead of becoming resentful ( 不满 ), become
determined to do better than the treatment group despite a perceived ( 感知) disadvantage.
The placebo group may increase its determination, dedication, physical exertion ( 用力 ),
practice, etc. to overcome its perceived disadvantage, thus, distorting the results of the
study. Under these circumstances, the placebo group is no longer behaving naturally, or
as it would normally behave, resulting in a comparison between the treatment group and
an unnatural control group (usually the placebo group).
Internal Validity: The Validity of Measurement Instruments
The validity of measurement instruments includes: face validity, content validity, construct
validity, and criterion validity.
i. Face validity: Does the measurement instrument appear to measure what it is meant to
measure as determined by several qualified judges ( 法官 )? Experts in the field will
examine the measurement instrument, the survey questions, the experimental
mechanizations ( 机械化), or similar components, and determine if it looks effective from
a more detached and untested perspective.
ii. Content validity: Does the measurement instrument cover or speak to all the
issues/content it is purporting ( 看来 ) to measure? The researchers may claim that they
want to measure X with their new measurement instrument, so the questions then are:
does the new measurement device completely and thoroughly measure all of X? What is
missing? What should be removed?
iii. Construct validity: Do the measurements obtained correspond ( 对应 ) with what is
commonly theorized about that area of interest? Or do the measurement results differ
with what is commonly understood about that area of study? If the new measurement
does not align with standard theory ( 理论 ) in the field, the instrument may be measuring
something different than what it was intended, or the instrument may be ineffective.
iv. Criterion validity: There are two types of criterion validity: concurrent validity and
predictive validity.
1. Concurrent validity compares the results of a new measurement instrument to the
most respected/best performing/most trusted measurement instrument that is
commonly used for similar purposes. If the results diverge ( 分歧 ) significantly
68
between the two measurement instruments, the new measurement instrument may
be less effective, or it may be measuring something different than what is
intended. With concurrent validity, the researcher wants to see how the new
measurement device “holds-up” against other similar notable and well-regarded
measurement devices.
2. Predictive validity pertains to whether a new measurement instrument can predict
some future result, event, or phenomenon. The question here is this: can the new
measurement instrument predict what will happen in the future with any degree of
certainty? This can be determined by taking the predicted results of a new
measurement instrument and later comparing them to actual outcomes.
External Validity
The basic question concerning external validity is this: Can the results of the research be
generalized back to the initial population ( 人群 ) from which the sample ( 样品 ) was gathered?
Simply put, can the results be generalized or inferred (推断) back to a larger population?
The most important area of concern is having the ability to generalize the results of an
experiment to the initial population from which the sample ( 样品 ) was drawn. After determining
the results of the study, the researcher wants to be able to speak to the effectiveness of the
treatment (or lack thereof) in the population from which the sample was drawn. Bear in mind,
generalization is usually limited to the initial population from which the sample was taken, it
does not entail other populations or conditions ( 条件 ). Dane (1990) describes the limitations of
generalization, “Generalizing the results of an experiment well beyond the experimental situation
is logically impossible, for the major purpose of most experiments is to demonstrate that the
cause-effect relationship can occur, not that it always occurs” (102).
To achieve high external validity, the researcher needs to ensure that the sample ( 样品 )
resembles the population from which the sample was gathered. To do this random ( 随机 )
selection is employed. Random selection means that subjects or elements are randomly selected
from the population that the researcher wants to study, and each subject should have a known
and equal chance of being selected. When there is sampling (采样) bias (偏差), when the sample
(样品) does not reflect the population in some way, there will likely be lower external validity.
To illustrate, let’s say that a researcher wants to survey ( 调查 ) university students about
their opinions on the taste of the food at the new campus cafeteria. The researcher stands out on
the corner of the medical school building and asks every student that walks past to respond to a
survey concerning the new cafeteria food. This sample may not represent ( 代表 ) the population
of students at the university because the researcher is only collecting data from a limited area
within the university. Students near the medical school building will likely be overrepresented
and may possess a somewhat different viewpoint on average when compared to students walking
near other departments, or they may not possess a different viewpoint, but we can’t say for sure
either way. Thus, a more comprehensive ( 全面 ) sampling ( 采样 ) procedure would likely be
needed to generate greater representativeness between the sample and the population (namely,
that of random selection from a list of known subjects in the target population). Without a
comprehensive (全面) sampling (采样) procedure, the study would probably have lower external
validity.
69
Additional Threats to External Validity
Additional threats to external validity include: situational specifics and reactivity.
i. Situational specifics: Certain responses to stimuli, treatments, surveys, etc. by the

subjects may only arrive under specific situations or in specific environments. People
may behave differently in a laboratory compared to how they would behave in the real
world. People may respond to a survey ( 调查 ) in a certain way, and then, when away
from researchers, go on to behave in a way counter to their survey responses. This creates
a potential division between what happens with the sample in an artificial ( 假 )
environment, and what occurs in the population (人群) in the real world.
ii. Reactivity: This involves how subjects respond to a new stimulus or entity in their
environment, be a it a researcher or some other entity, and how this response may impact
the experimental results.
Reactivity comes in two main forms: The Hawthorn effect and the novelty effect.
1. The Hawthorn effect: This is a phenomenon wherein participants change their
behavior or their responses simply because they know that they are being studied.
The attention that subjects receive from researchers as a product of their
cooperation in a study may influence the subject’s behavior and responses. This
alters experimental results and it reduces a researcher’s ability to claim that the
behavior of the sample reflects the behavior of the population under real-world
conditions (条件).
This could be thought of in terms of enthusiasm ( 热情). Subjects will often
become enthusiastic about being studied or observed and will change their
behavior—participants may work harder to impress those involved, they may
change their responses to make themselves look better, they may change their
responses in a way that may “help” the experimenter obtain the results the
participants believe the experimenter is seeking to obtain, etc.
This enthusiasm ( 热情 ) may be particularly evident when one
experimental group is being watched or studied more than another group. The
group that is receiving greater attention may become enthusiastic and work
harder, become more stimulated, become more engaged, and so on when
compared to the lesser watched/studied group. Thus, enthusiasm may alter the
results in a way that seriously threatens the integrity (诚信) of an experiment.
2. The novelty ( 新颖 ) effect: The introduction of new stimuli ( 刺激 ) into an
environment may temporarily change the behavior of subjects, but that change of
behavior may fade as subjects become accustomed to it. It is in effect not the
substantive nature of a new stimuli that causes ( 造成 ) a change, but the simple
fact that the stimuli is new or novel ( 新颖 ). Subjects will later respond to the new
stimuli as if it was commonplace, and, because there are no substantive qualities
attached to the new stimuli that influence subjects, no behavioral differences will
be found.
So, a new stimulus could produce a significant change in subject behavior
or thought during the short experimentation period in which it is introduced, but
that change could fade as the novelty ( 新颖 ) wears-off and the new stimuli
becomes just another entity ( 实体 ). Thus, it’s often important to continually test
70
subjects after the introduction of a stimuli/treatment to ensure that the novelty
effect isn’t driving the results.
It’s valuable that the researcher also considers the effectiveness of the treatment in other
populations, environments, situations, and with other people. Rephrased, the researcher should
consider the effectiveness of the treatment outside of the initial research setting and the initial
population ( 人群 ) from which the sample was drawn. Questions like these may be contemplated
(考虑): does the treatment work in other places? Does it work in other laboratories? Does it work
in other populations? Does it work under other conditions ( 条件)? Does it work with people who
possess different characteristics?
Reliability (可靠性)
The basic questions concerning reliability ( 可靠性 ) involve experimental consistency ( 一致性 ):
Are the results of the research repeatable ( 重复 )? Are the same results consistently ( 一致性 )
being obtained under the same experimental conditions ( 条件 )? Are the results of the research
reproduceable? If the researcher is running the same experiment or same test and is consistently
getting different results, it can be said that the experiment has low reliability, and that there may
be something wrong with the experimental design, measurement methods, testing procedure,
operational definitions, etc. On the other hand, if the same experiment is being conducted
repeatedly, and the same results are consistently being obtained, it can be said that the
experiment has higher reliability (可靠性).
Test Measurement Reliability (可靠性)
Test measurement reliability ( 可靠性 ) uses the statistical ( 统计 ) method of correlation ( 相关 ) to

determine the reliability of a measurement instrument or measurement method. The greater the
correlation among the different tests or measurements, the greater the likelihood that the
measurement instrument or the measurement method is reliable or consistent ( 一致性 ). It is this
reliability or consistency that determines the amount of measurement error that can be assumed.
Singleton and Straits (1999) describe how measurement error and reliability are related, “If a
measurement yields the same result time after time, then it is free of random error; furthermore,
the greater the variation in repeated ( 重复 ) measurements, the greater the random error.
Reliability assessment is essentially a matter of checking for such consistency—either over time
(as when the same measurements are repeated) or over slightly different but equivalent measures
(as when more than one indicator or more than one observer/interviewer/recorder is used)” (117).
Test measurement reliability can be examined using these four methods: test-retest reliability ( 可
靠性), split-half reliability, interrater reliability, and item-total reliability.
i. Test-retest reliability ( 可靠性 ): This involves giving the same test or measurement to the
same subjects more than one time to determine the measurements consistency ( 一致性 ).
Once these measurements are taken, the results are correlated ( 相关 ) to determine the
tests/measurements reliability. When utilizing this method, the researcher should be able
71
to argue that it’s unlikely that the subject’s memory of the initial test/measurement will
influence subsequent (随后) results.
This is a test of reliability or consistency ( 一致性 ) over time. If the results from
the first test do not correlate with the results from the second test, it could be argued that
the test lacks precision, is too vague, or is ineffective in some other way.
ii. Split-half reliability: This is commonly used when subjects are surveyed ( 调查 ) or
provided questionnaires (otherwise known as a “multi-item measure”). This process
operates in the following way: the researcher provides subjects with a survey. When the
subjects are finished with the survey and the survey is collected, the researcher splits the
survey into two parts, effectively producing two surveys per subject. The two halves are
then correlated ( 相关 ) to determine measurement reliability. If the type of responses
provided in one half of the survey do not align with the type of responses in the other half
of the survey, the measurement instrument has low reliability and should be reconsidered.
This measure of reliability assesses the “agreement or equivalence among the constituent
parts or items of a multi-item measure” (Singleton & Straits, 1999; 118).
A major way to split surveys, or the questions encompassed within, is to divide
the questions by “even” or “odd” as they exist within their sequence ( 顺序 ), and then
correlate the evens with the odds.
iii. Interrater reliability: This involves using more than one researcher (or rater) to observe
the same behavior, event, or phenomenon. The researcher’s observations or judgments
are then correlated ( 相关 ) to determine the reliability or consistency ( 一致性 ) of their
observations or judgements. If the researchers or raters are closely aligned in their
interpretations of behaviors, events, or phenomenon, it’s likely that their observations
will have a higher level of correlation, and, thus, higher measurement reliability (or
higher interrater reliability). If the raters are not seeing the same thing, if they are in
seeming disagreement concerning what transpired, the correlation ( 相关 ) will be lower,
and measurement reliability will be lower as well.
Rather than have one researcher observe the behavior of a subject (for example,
recording the number of times a child engages in adversarial ( 敌对) behavior with another
child), and rely on the opinion and interpretation of one person, it may be better to have
three researchers observe a subject and correlate the assorted scores to determine
measurement reliability. This form of reliability is another type of equivalency measure
(in other words, are the researcher’s judgements equivalent with one another? To what
extent do the judgements diverge (分歧) from each other?).
iv. Item-total reliability: This involves comparing and correlating ( 相关 ) each specific item
on a measuring or evaluation method with the total score of the rest of the measuring or
evaluation method. The researcher is looking at the reliability or equivalency of singular
items on a measuring method as they are correlated with the total score of that measuring
method.
Study Questions:
1) What is internal validity?

2) What is external validity?
3) How is high external validity achieved?
72
4) What is attrition?
5) What is diffusion?
6) What is resentful demoralization?
7) What is compensatory rivalry?
8) What is the Hawthorn effect?
9) What is the novelty effect?
10) What is reliability?
Accuracy / The extent to which a measurement is without error.

Alternative / Another possible option, response, or result.
Attrition / When certain subjects fail to remain within a study.
Bias / A predisposition to thought or theory that may influence the results of a study.
Causal / When an independent variable produces a significant change in a dependent variable it
establishes added evidence that the relationship is…
Cause / The independent variable or intervention.
Consistent / Reliable; without waver, change, or variation.
Diffusion / The spread of information among experimental groups.
Effect / The results when the dependent variable is measured.
Estimate / Approximate; a level of uncertainty regarding a conclusion.
External / Outside of something; outside of the experimental context.
Generalize / To take the results of an experiment conducted with a sample and infer those results
back to the population from which the sample was drawn.
Internal / Inside of something; inside of the experimental context or experimental environment.
Link / Relationship or connection between variables.
Maturation / Inherent and organic developmental processes that naturally occur over time.
Mortality / When subjects exit a study prematurely.
Novelty / Something that is new.
73
Population / All the subjects or elements that a researcher wants to study.
Reactivity / Responses to a treatment, intervention, or stimulus.
Regression / The tendency for more extreme scores to become less extreme on subsequent
testing regardless of applied treatments.
Repeatable / Finding the same experimental results on subsequent trials.
Reproduceable / Obtaining consistent results upon repeated tests.
Sampling / Generating a group of subjects that are to represent a population for experimental
purposes.
Subpopulations / Distinct groups within a population.
puzzle:
74
ACROSS
1 To take the results of an experiment conducted with a sample and infer those results back to the
population from which the sample was drawn.
4 Outside of something; outside of the experimental context.
5 Approximate; a level of uncertainty regarding a conclusion.
7 The extent to which a measurement is without error.
9 Reliable; without waver, change, or variation.
11 The spread of information among experimental groups.
12 The results when the dependent variable is measured.
DOWN
2 Another possible option, response, or result.
3 When certain subjects fail to remain within a study.
6 The independent variable or intervention.
8 When an independent variable produces a significant change in a dependent variable it
establishes added evidence that the relationship is…
10 A predisposition to thought or theory that may influence the results of a study.
search puzzle:
75
1. Inside of something; inside of the experimental context or experimental environment.
2. Relationship or connection between variables.
3. Inherent and organic developmental processes that naturally occur over time.
4. When subjects exit a study prematurely.
5. Something that is new.
6. All the subjects or elements that a researcher wants to study.
7. Responses to a treatment, intervention, or stimulus.
8. The tendency for more extreme scores to become less extreme on subsequent testing
regardless of applied treatments.
9. Finding the same experimental results on subsequent trials.
10. Obtaining consistent results upon repeated tests.
11. Generating a group of subjects that are to represent a population for experimental purposes.
12. Distinct groups within a population.
76
Terms: accuracy, alternative, attrition, bias, causal, diffusion, effect, external, generalize,
internal, maturation, mortality, novelty, population, regressed, and sampling.
1. The __________ of the measurement instrument was negatively influenced by the

researcher’s inexperience.
2. The __________ hypothesis was that the new medicine was significantly more effective than
the previous medicine.
3. Subject __________ made generalizing the results to the population a challenge, because the
sample, as it was now altered due to subject drop-out, no longer resembled the population.
4. The __________ exhibited by the researcher was palpable. When the subjects responded
positively, he attributed it to the new therapy, but when they responded negatively, he always
attributed it to something else.
5. There was no __________ relationship between the two variables. Differences in the
independent variable did not produce differences in the dependent variable.
6. Understanding the __________ or spread of scores provides an indication of where an
individual subjects score is in relation to the group.
7. The cause and __________ relationship claimed by the experimenter was questioned due to
the existence of numerous potential confounding variables in the study.
8. Random selection, where subjects are randomly selected from a population, is a fundamental
component in the attainment of high __________ validity.
9. When the sample does not reflect the population, it makes it difficult for the researcher to
__________ the results of the study back to the population from which the sample was
drawn.
10. When the independent variable has a direct influence on the dependent variable, and other
potentially confounding variables have been ruled out, the researcher may claim high
__________ validity.
11. Concerns that subject __________ may have influenced the results arose when it was
revealed that the study lasted an additional ten months.
12. Subject __________ may create differences between groups that can reduce internal validity.
Subjects dropping out systematically in one group may create inequalities among groups.
13. The __________ of having a new teacher in the class initially caught the student’s attention
and the students studied diligently, but after this initial effect diminished, the students
returned to their normal study habits.
77
14. For research to have high external validity, the sample should meet size standards and reflect
the __________.
15. Extreme scores on the pre-test naturally __________ closer to the mean on the post-test, thus
distorting the experimenter’s results, and calling into question the effectiveness of the
treatment.
16. The __________ process was disorganized and chaotic. The researchers initially used
random sampling but grew tired and resorted to convenience sampling.
Chapter 8: Sampling (采样) Methods

Whole populations (人群) generally cannot be tested because of their large sizes, so a sample (样
品 ) is generated from a population. It is from the sample that the test or research is conducted.
The sample is meant to represent ( 代表 ) the population. The experiment is conducted on the
sample, and the results are inferred ( 推断 )/generalized back to the population from which the
sample was obtained. When a sample adequately represents (代表) a population, external validity
is increased (external validity is the ability to infer ( 推断 )/generalize from the sample to the
population from which the sample was drawn).
The Sampling (采样) Process:

1) Define the population (人群).
2) Determine the sampling frame. The sampling frame is a formal list or formal statement of
all possible subjects/elements that could be selected from the population for the sample.
The population is chosen, and then a sampling frame is diagramed ( 可图示 ) to express
who the potential candidates are for selection.
The sampling frame may not completely reflect the population in which the
researcher wishes to study, rather, it is a best estimate based upon the available
information. For instance, let’s say that a researcher wants to survey a community
regarding some issue. The researcher obtains a list of all the people who are over the age
of eighteen that live in that community to be the sampling frame, yet that list is simply a
snapshot in time of the available information about the community. Since the list was
formed, which could have been weeks earlier, people may have moved in or out of the
community, or perhaps there were people living in the community who didn’t make the
list for whatever reason (e.g., clerical error). Whatever the case, it’s unlikely that the
sampling frame will completely match the population for which it is meant to represent.
This mismatch between the population and the sampling frame is usually not much of a
problem; it is typically the best that can be achieved given the circumstances.
3) Determine the sampling method (probability or nonprobability; systematic or haphazard;
etc.).
4) Specify the sample size.
78
5) Specify the sampling operation (develop a detailed plan regarding how the sample will be
collected).
6) Carry out the sampling procedure.
Then an experiment, survey (调查), etc. is conducted with the sample, and the results are inferred
(推断) back to the population from which the sample was obtained.
Probability (可能性) Sampling and Nonprobability Sampling
There are two main types of sampling ( 采样): probability (可能性) sampling and non-probability
sampling.
Probability (可能性) Sampling
With probability ( 可能性 ) sampling, selection is randomized ( 随机 ). Every unit or subject has a
known, and usually equal, chance of being selected. Harris (2005) describes the ideal
relationship between the population, the sample, and probability sampling, so as to improve the
chances of adequate representation ( 代表 ) and to achieve higher external validity, “Researchers
should define the population to which they wish to generalize and then devise a sampling scheme
that will guarantee each member of the target population an equal chance of representation in the
study” (45). Probability sampling, if conducted effectively, increases the likelihood that the
sample represents (代表) the population, thus increasing external validity.
Types of probability ( 可能性 ) sampling ( 采样 ) include: simple-random sampling, systematic ( 系

统) sampling, stratified (分层) sampling, and cluster (簇) sampling.
i. Simple-random sampling: Subjects are selected randomly using mechanisms such as

random number generators, drawing lots, computer algorithms ( 算法), etc. Each subject is
expected to have a known and usually equal chance of being selected for the sample.
Simple-random sampling (image: Dan Kernler).
ii. Systematic ( 系统 ) sampling: Subjects are systematically selected through interval ( 间隔 )

arrangements. An interval ( 间隔) is preselected, and every “Nth” subject is selected from
79
a list of available subjects. To illustrate, say there is a population of 400 people, and 100
of them are to be selected to be in the sample. A list of the 400 people is obtained, and it
is determined that every 4th person is to be systematically selected to be in the sample.
The researcher would randomly select the first person out of the first 4 people on the list,
then the next 4th person would be systematically selected, then the next 4 th person, and so
on until 100 subjects were selected. So, let’s say the researcher first randomly selected
person number 2 (remember, the first subject is randomly selected from the first 4 people
(1-4) to begin the systematic selection), then subjects 6, 10, 14, 18… would be selected
from the sampling frame (the sampling frame is the list of available subjects).
The systematic sampling of every 3rd person (image: Dan Kernler).
iii. Stratified ( 分层 ) sampling: A population ( 人群 ) is broken into mutually exclusive parts

based on some criteria (标准) to better ensure the representativeness (代表) of that criteria
within a sample. Once the population is separated into distinct parts based on a specific
criterion ( 标准 ), each part undergoes its own random selection until that part is
adequately represented.
For example, let’s say a researcher wants to survey the undergraduate population
at a university. Usually, first and second year students respond to surveys at a higher rate
compared to third- and fourth-year students. So, if a phone survey was provided to the
whole undergraduate population without stratification ( 分层 ), the first- and second-year
students would likely be greatly overrepresented. If the population was stratified ( 分层 )
and a quota ( 配额) was created for each grade at 25% of the sample, each grade could be
randomly sampled until the quota was reached (to continue to randomly select subjects in
these underrepresented grades until the quota is filled). Through this stratified selection
process, better representation within a sample may be obtained in situations when it is
difficult to acquire the same level of participation from different groups of people.
80
Stratified sampling (image: Dan Kernler).
iv. Cluster ( 簇 ) sampling: Cluster ( 簇 ) sampling is used to break large populations down
through selection processes for sampling purposes. The cluster sampling process operates
like this: subjects are typically divided into naturally forming clusters ( 簇 ) by the
researcher. Clusters are randomly selected from among this population of clusters. From
within these newly selected clusters, new clusters are formed by the researcher based on a
new dimension, and these new clusters go through the same selection process. This
process continues until the subjects themselves are randomly selected from the last
remaining clusters. Myers and Hansen (1997) provide a crisp explanation of this
procedure, “Instead of sampling individuals from the whole population or from
subgroups, they sample entire clusters, or naturally occurring groups, that exist within the
population. As with all forms of probability ( 可能性) sampling, participants are randomly
selected, but whole groups of people are selected rather than individuals” (77).
Cluster (簇) sampling is used when there is a very large population with no secure
sampling frame from which to directly sample from (no clear list of the members in a
population is available because the population is large, fluid, and regularly changing).
Dane (1990) describes the appropriate time to utilize cluster sampling, “Sometime,
however, you don’t have a list of the sampling frame and cannot obtain one. You would
not, for example, be likely to generate a sampling frame list for all college students in the
United States. When sampling frame lists are unavailable, researchers use cluster
sampling—randomly selecting hierarchal groups from the sampling frame” (300).
For instance, let’s say a researcher wants to survey people in large cities about life
and the living conditions in large cities in the United States. The population here is
massive, millions of people, so how to manage this? Cluster sampling could be used. All
cities with populations over 500,000 could be considered a single cluster. Clusters (cities)
could be randomly selected from this initial population of clusters (cities). After having
randomly selected this group of clusters (cities), new clusters could be generated from
each of the selected clusters (cities) in the form of districts, and these new clusters
(districts) could undergo random selection. The same could then be done for the
81
neighborhood, street, and so on, down to the subject. Under these sampling conditions,
there are numerous hierarchical ( 分级 ) levels under which clusters are being formed and
sampled: first cities, then districts, then neighborhoods, then streets, and all the way down
to the individual subjects.
Cluster sampling (image: Dan Kernler).
Random (随机) Selection
With random selection, subjects have a known, and usually equal, chance of being selected for a
sample. Random samples ( 样品 ) can be produced using random number generators, drawing
numbers from a hat, using complicated statistical algorithms ( 算法 ), etc. Random selection
increases the chances that the sample represents ( 代表 ) the population, thus, increasing external
validity.
Sampling Bias (偏差)
Without random ( 随机 ) selection there is a much greater likelihood of sampling bias ( 偏差 ).

When parts, elements, or characteristics of a population are, for whatever reason, favored or
overrepresented in a sample compared to other parts of a population, the sample may be biased
(偏差) in some way. Because of this overrepresentation, there is a greater chance that the sample
does not accurately reflect the population. This is often the result of bias ( 偏差 ) in the sampling
process.
Even with random selection there may be some level of bias ( 偏差 ) involved in the
selection process. Singleton and Straits (1999) note the most likely forms of sampling bias:
coverage error and nonresponse bias. The authors state that coverage error is a result of
incomplete sampling frames (i.e., the sampling frame does not fully capture or represent ( 代表 )
the population), and nonresponse bias is the result of incomplete or distorted data collection (i.e.,
the sample is systematically distorted because of who is willing (or unwilling) or who is able (or
unable) to participate in the research). Singleton and Straits (1999) detail the causes ( 造成 ) of
nonresponse bias, “The problem of nonresponse bias arises when, through refusals to cooperate,
unreturned questionnaires, missing records, or some other means, the sample turns out to be a
82
fraction of the number of cases originally selected for observation. The crux of this problem is
that nonobservations tend to differ in systematic ways from observations” (170).
Non-Probability Sampling (采样)
With non-probability sampling ( 采样 ), selection is not randomized ( 随机 ). The probability ( 可能

性) that a subject is selected is indeterminable. Non-probability sampling can be haphazard (杂乱
无章 ) or methodical, but it is not random. Non-probability sampling is not necessarily a bad
thing, it is often the most appropriate method given the circumstances.
Types of non-probability sampling include: quota (配额) sampling, convenience (方便) sampling,
snowball sampling, voluntary (自愿) sampling, and purposeful sampling.
i. Quota ( 配额 ) sampling: Subjects are selected in a nonrandom fashion until some

specified quota ( 配额 ) is reached. A researcher may devise different quotas within a
sample for assorted purposes, but usually quotas are used to ensure sample
representativeness ( 代表 ). Quota ( 配额 ) sampling is very much like stratified ( 分层 )
sampling in that a population is separated into groups and quotas are placed on each
group to satisfy some form of representation requirement. The difference between quota
sampling and stratified ( 分层 ) sampling is that in quota sampling there is no random
selection of subjects. Instead, selection is conducted through alternative means
determined by the researcher (such as simply approaching someone who seems to fit the
criteria (标准) for group membership).
ii. Convenience (方便) sampling: Subjects are incorporated ( 合并) into a study in a way that
is convenient ( 方便 ) for the researcher. For example, a professor may survey his/her
class, a researcher may interview people directly outside of a department at a university,
student researchers may survey (调查) people on WeChat, etc.
iii. Snowball sampling: As a snowball accumulates mass as it rolls down a hill, with
snowball sampling, a sample of subjects for a study is accumulated ( 积累 ) as one subject
introduces the researcher to other people who are like the subject in a way that
fundamentally corresponds ( 对应 ) to the study. These new participants are then studied,
and they may introduce more people to the researcher, and so on, until the target sample
size is acquired.
This type of sampling usually involves smaller samples and people who are
difficult to find and study. It involves locating and sampling subjects who are often
isolated ( 孤立 ), vulnerable, or concealed; Atkinson and Flint (2001) explain, “[This]
technique offers real benefits for studies which seek to access difficult to reach or hidden
populations. These are often obscured from the view of social researchers and policy
makers who are keen to obtain evidence of the experiences of some of the more marginal
excluded groups” (para. 1).
To illustrate, let’s say that a researcher wants to study car thieves to understand
the car thieves’ motivations, how they’re indoctrinated ( 灌输 ) into this criminal lifestyle,
why they chose this type of lifestyle, etc., so that social programs can be developed to
prevent people from engaging in this behavior in the future. Car thieves do not want to be
found, obviously, and, thus, are quite difficult to study. Let’s say that through good
83
fortune a member of the research staff is able contact a car thief somehow (maybe
through some rehabilitation based social program, through a relative, or something of this
nature). The subject is interviewed and then asked to introduce some of his fellow car
thieves to the researcher so that they can be interviewed as well. The subject does this,
these new subjects are studied, and they introduce other car thieves to the researcher, and
this process continues, providing the researcher with the desired sample size.
iv. Voluntary ( 自愿 ) sampling: Subjects actively volunteer ( 自愿 ) to participate in a study.
It’s assumed within most experimentation that subjects are volunteering ( 自愿 ) to
participate (this is often asserted ( 断言 ) through consent ( 同意 ) forms or through verbal
consent ( 同意 )). Though, under these sampling conditions ( 条件 ), subjects are taking the
initiative and going out of their way to enter into a study. There is a difference between
one being approached by a researcher on the street to participate in a survey, and one
randomly seeing an advertisement on a noticeboard for a general psychology experiment
in the psychology department that pays 150 yuan an hour and joining the study. Both are
voluntary ( 自愿), but in the second condition ( 条件 ) the subjects are inserting themselves
into a study on their own accord and without direct prompting ( 促使 ), usually for some
expected reward (monetary or otherwise).
This type of incentive based voluntary sampling is often an effective way to
generate a group of subjects, but it has disadvantages. The subjects that constitute ( 构成 )
these voluntary samples may not represent ( 代表 ) the population: they may need money
more than the average person (this could be particularly damaging in investment or
financial studies); they may be more outgoing or extraverted ( 外倾 ) than the average
person (this could be particularly damaging in psychological studies focused on
individual characteristics); they may be more lonely than the average person and want
someone to talk to or want something to do (this could be particularly damaging in
studies of depression or in other psychological studies). Whatever the case may be, it’s
possible that these kinds of samples do not adequately represent a population.
Nevertheless, the main advantage of this type of sampling is that it generates a group of
people willing to participate in a study. Without this method, and the incentives for
participation usually encompassed within, a study may not have any participants.
v. Purposeful sampling: Subjects are selected because they are judged ( 判断 ) by the
researcher to represent ( 代表 ) a group or a population. Subjects may also be selected
because they possess some unique quality that is of interest to the researcher. This is a
way for researchers to select subjects for a study (usually qualitative studies) whom they
deem representative of a specific group or whom they believe are particularly valuable
for a study. For instance, a researcher may spot someone on the street or in a mall that
seems to have some characteristic or quality that the researcher wants to study; the
researcher will approach this person and ask them to participate in the study.
Study Questions:
1) What is a sampling frame?

2) What is simple-random sampling?
3) What is random selection?
84
4) What is sampling bias?
5) What is snowball sampling?
6) What are some examples of convenience sampling?
7) What is a major problem with voluntary sampling?
Bias / Viewing or analyzing something from a personal perspective; allowing one’s personal
opinions or ideas to influence one’s conduct.
Chance / Selection without bias or subjective influence.
Cluster / The grouping or bunching of subjects for sampling purposes.
Convenience / A reduction in difficulty or hassle.
Generalize / Infer; extrapolate; to take the results of an experiment conducted with a sample and
use that information to speak to the population from which the sample was drawn.
Generate / Make; produce; create.
Haphazard / Without complete predesign, precision, or structure.
Infer / Generalize; to take a singular finding derived from a sample and make observations about
a larger population.
Operation / Process to carry out some purpose; mode or design for action.
Population / The group of people or elements from which a sample is drawn.
Probability / The likelihood that someone or something is selected for a sample.
Quota / Selecting subjects, either randomly or haphazardly, who fit some predetermined
dimension.
Random / Selection by chance; selection without bias.
Represent / One thing or one person standing in the place for something or someone else, to
speak for or symbolize that something or someone; to hold a position that is a symbol for
something else or for others.
Sample / The group of subjects that are selected from a population who are then used within an
experiment or survey as representatives of the population.
Snowball / Subjects are accumulated into a sample by having subjects introduce people (people
whom the researcher is interested in studying) to the researcher.
85
Stratified / Dividing a population into groups based on some dimension to better ensure group
representativeness during the sampling process.
Survey / Collecting information from subjects through interviews and questionnaires.
Systematic / Operations that are conducted in a purposeful and orderly fashion.
Voluntary / Offered on one’s own volition; decided independently and without coercion.
puzzle:
ACROSS
5 Generalize; to take a singular finding derived from a sample and make observations about a
larger population.
7 Infer; extrapolate; to take the results of an experiment conducted with a sample and use that
information to speak to the population from which the sample was drawn.
86
9 The likelihood that someone or something is selected for a sample.
10 Viewing or analyzing something from a personal perspective; allowing one’s personal
opinions or ideas to influence one’s conduct.
11 Process to carry out some purpose; mode or design for action.
12 Selection without bias or subjective influence.
DOWN
1 The grouping or bunching of subjects for sampling purposes.
2 A reduction in difficulty or hassle.
3 The group of people or elements from which a sample is drawn.
4 Make; produce; create.
6 Without complete predesign, precision, or structure.
8 Selecting subjects, either randomly or haphazardly, who fit some predetermined dimension.
search puzzle:
87
1. Selection by chance; selection without bias.
2. One thing or one person standing in the place for something or someone else, to speak for or
symbolize that something or someone; to hold a position that is a symbol for something else
or for others.
3. The group of subjects that are selected from a population who are then used within an
experiment or survey as representatives of the population.
4. Subjects accumulated into a sample by having subjects introduce people (people whom the
researcher is interested in studying) to the researcher.
5. Dividing a population into groups based on some dimension to better ensure group
representativeness during the sampling process.
6. Collecting information from subjects through interviews and questionnaires.
7. Operations that are conducted in a purposeful and orderly fashion.
8. Offered on one’s own volition; decided independently and without coercion.
Terms: bias, chance, cluster, convenience, generalize, generate, population, probability, quota,
random, represent, snowball, survey, systematic, and voluntary.
1. The experimenter showed no __________ in his interpretation of the results; he looked at the
data and came to an objective and dispassionate conclusion.
2. We want to minimize the influence of __________ within our study. We can do this by
controlling the research environment, using more precise and accurate measurement
instruments, clearly defining the research process, etc.
3. __________ sampling is an effective way to break down and randomize the selection of
subjects within very large populations.
4. He gathered his sample based on __________. He approached people on the street and asked
them to take his survey.
5. When the sample reflects the population, we can appropriately __________ the results of the
study back to the population from which the sample was drawn.
88
6. Engaging in exploratory research is often a good way to __________ ideas for experimental
research.
7. The __________ is everyone or everything the researcher wants to study.
8. The __________ that the sample represents the population is greatly increased when random
selection is conducted.
9. The __________ of 120 subjects from the 50-60 age group was not reached, reducing that
age group’s representativeness within the study and lowering the study’s external validity.
10. __________ selection was used so that the sample adequately represented the population.
11. To claim that a study has high external validity, the sample needs to __________ the
population from which it was drawn.
12. __________ sampling is often an effective way to recruit new and difficult to find subjects
through accumulative processes.
13. A __________ was distributed to members of the class. This convenience sample was helpful
in obtaining a quick and cost-effective view of the opinions of the students.
14. His study lacked __________ rigor. The experimental process was not clearly defined, the
experiment was conducted in a haphazard fashion, and the experimental environment was not
sufficiently controlled.
15. Subject participation within an experiment needs to be __________.
Chapter 9: True Experiments and Quasi (拟)-Experiments

Both true experiments and quasi-experiments make predictions, test hypotheses, and make
statements about the world. It’s important to understand how these two experiments function and
the differences between the two experimental types.
True Experiments
A true experiment involves comparing experimental groups, after an independent variable has
been administered, with the possibility to imply causation ( 原因 ). Within this experimental
framework, a claim can be made that an independent variable has caused ( 原因 ) a change in a
dependent variable. When the independent variable is manipulated by the researcher (i.e., there is
more than one experimental group), subjects are randomly assigned to the different experimental
groups, extraneous variables are effectively controlled, and the dependent variable can be
measured, it can be claimed that a true experiment was conducted. This form of experimentation
examines cause (原因) and effect relationships and is to contain these four parts:
The Four Essential Elements Required to Have a True Experiment
1. The independent variable is manipulated ( 操纵 ). The experimenter can and does change
the quantity or quality of the independent variable that is provided to the different groups.
Stated differently, there are two or more experimental groups within the independent
89
variable that receive different treatments (e.g., a treatment group and a control group), all
of which is controlled by the experimenter.
2. Subjects are randomly (随机 ) assigned to the different levels of the independent variable.
A sample is collected from a population ( 人群 ), and the subjects within this sample are
randomly assigned to the different experimental groups. Harris (2005) explains how
participants are randomly assigned to the different experimental groups, “Random
assignment involves the use of some mechanical procedure or algorithm (based, e.g., on a
table of random numbers or the pseudo-random number generator built into a statistical
package) to determine to which level of a manipulated independent variable any given
participant is to be exposed” (47).
3. The researcher has control over extraneous ( 无关 ) variables within the experimental
environment. Otherwise stated, the researcher has a great degree of control over the
experimental setting. The research environment can be manipulated ( 操纵 ) and changed
so that all the different experimental groups receive an equal, or near equal, amount of
outside influence (or extra-treatment influence), the only difference being the different
treatments provided to the different experimental groups.
4. The dependent variable/outcome can be measured. If the dependent variable cannot be
measured, the experimental groups cannot be compared.
To reiterate, with true experiments, there is an independent variable, a dependent variable,

control over the research environment, and random assignment. Statistical ( 统计 ) analyses are
used to determine if there are differences among groups (e.g., if the treatment groups are
statistically different from the control group, and if the treatment groups are different from each
other).
Examples of true experiments:
Randomized Pretest-Posttest Control Group Experimental Design:
R O1 X O2
R O3 O4
O = Observation
X = Treatment
This design contains random assignment, at least two levels of the independent variable (the
treatment group(s) and the control group), control over extraneous or outside variables, and the
ability to measure the dependent variable.
Randomized Posttest-only Control Group Experimental Design:
R X O1
90
R O2
O = Observation
X = Treatment
treatment group(s) and the control group), control over extraneous or outside variables, and the
ability to measure the dependent variable. As you can observe, there is no pretest in this design,
only the randomized assignment of subjects into experimental groups. Keep in mind, pretests are
often unnecessary when there is random assignment and a large enough sample size (under these
conditions, it can be assumed that the groups are statistically equal).
Solomon Four-Group Design (the Solomon Four-Group Design is a True Experiment):
R Group1 O X O
R Group2 O O
R Group3 X O
R Group4 O
treatment group(s) and the control group(s)), control over extraneous or outside variables, and
the ability to measure the dependent variable.
The results of Group 1 are compared to the results of Group 3 to analyze the interaction effects
of the pretest and treatment on the posttest.
The results of Group 2 are compared to the results of Group 4 to analyze the direct effect of the
pretest on the posttest.
Apart from testing the influence of a pretest, the Solomon four-group design also functions as a
large true experiment, wherein the groups can be compared to determine the effectiveness of a
treatment.
True Experiments in Laboratories
True experiments are usually laboratory based, and, thus, easier to control and replicate ( 复制 ).
In the physical sciences, for example, exact or near exact replication ( 复制 ) is of paramount
importance. Gomm (2008) describes the constraints ( 约束 ) the experimenter should employ
when conducting causal ( 因果 ) or true experiments with human subjects (this applies to other
living organisms as well), “The experimenter has to:
 Create two or more comparison groups of people who are similar in all respects which
might be relevant to the experiment.
 Make sure that all the subjects are treated the same, except in the ways in which they are
supposed to be treated differently.
 Ensure that the same observations are made of each subject in the experiment in the same
way, at least before (pre-) and after (post-)” (61).
91
Due to the high level of experimental control that is provided in a laboratory, and the greater
experimental control in experimentation in general, the results that are produced may not be
representative ( 代表 ) of what occurs in the real world (particularly when dealing with humans).
In other words, the artificial ( 假) nature of a highly controlled laboratory or experimental process
may not be as applicable to real world situations as is predicted in laboratory-based experiments.
Quasi ( 拟 )-Experiments (also referred to as “natural experiments” because of a lack of

experimenter control)
Quasi ( 拟 )-experiments cannot make cause ( 原因 ) and effect claims because the experimental
design is missing one or more of the four fundamental aspects of a true experiment, usually
random assignment. Quasi-experiments are typically used when true experiments (and the
random assignment encompassed within) are not possible, often for ethical ( 伦理 ) reasons (e.g.,
it would be unethical to randomly assign people to a “cigarette smoker” group and a
“nonsmoker” group).
As mentioned, this type of experimentation arises when participants cannot be randomly
assigned to different groups—often the experiment looks just like a true experiment, but with
preexisting (原有) groups and no random assignment. In this case, nonequivalent groups serve as
the treatment and control groups, and either:
1. Treatments and controls are assigned to preexisting groups (no random assignment of
subjects to the groups). For example, a researcher wants to test the effects of a new
teaching method on students’ grades, but it is often considered unethical to separate
children from their friends and classmates so that they can be randomly assigned to
different classes/treatments. Therefore, the new teaching method is implemented ( 实施 )
in one preexisting classroom of students, while the old method of instruction is
implemented in another preexisting classroom. Here, the new teaching method would be
assigned to one of the preexisting groups (classrooms), and the control method would be
assigned to the other preexisting group. Or…
2. The subjects are in preexisting groups, and the treatments and controls are already located
within these preexisting groups (the treatments and controls are not assigned to the
preexisting groups because they already exist in the preexisting groups). For example, the
previously mentioned “smoker” or “nonsmoker” experiment—wherein both groups
preexist, as do the treatments and controls. Or consider a study that examines the
neurological ( 神经 ) health of a population of people living in a city that sells large
quantities of leaded ( 含铅 ) gasoline for use in automobiles versus a city that sells only
unleaded ( 无铅 ) gasoline. Both groups are preexisting in the form of leaded or unleaded
cities. This study is also known as a “ex post facto study,” where groups are compared
after the fact.
Examples of Quasi-Experimental Designs:
Nonequivalent Control Group Design:
92
O1 X O2
O3 O4
O = Observation
X = Treatment
R = Randomization
As shown above, subjects are not randomly assigned to the different groups, but all else is like a
true experiment.
Interrupted Time-Series Design:
O1 O2 O3 O4 X O5 O6 O7 O8
As you can tell, there is no random assignment. A series of observations/measurements are taken
before and after the treatment is provided to generate a baseline ( 基线 ) measurement. When a
predictable baseline measurement is established prior to the implementation of a treatment, any
changes in the subjects after the treatment is established will likely provide more legitimacy ( 合
法 ) for a study. The same idea holds true for the after-treatment observations. Wherein, if, after
the treatment wears-off, the scores return to a stable baseline in the after-treatment
measurements, the study potentially achieves greater legitimacy. So, through a series of
observations prior to the treatment, a predictable baseline ( 基线 ) is developed that can be
compared with the effects of the treatment. These same comparisons can also be made with the
observations undergone after the treatment is implemented. The results of this type of design are
usually displayed visually on a frequency graph. The results are always used to make an
argument, not a causal claim.
Quasi-Experiments and Internal Validity
With the absence of random assignment in quasi-experimentation, internal validity issues

involving selection become more pronounced ( 明显 ). This is because the researcher may have a
hard time determining whether the results were the product of the different stimuli ( 刺
激 )/treatments in which the experimental groups were exposed, or if the results were due to
preexisting (原有) differences among the groups prior to experimentation. Harris (2013) explains
the foundational threat to internal validity that exists in quasi ( 拟)-experimentation, “The number
one threat is the preexisting differences among the extant groups assigned to the different levels
of the IV(s)… thereby making it difficult to disentangle the effects of the manipulation from the
effects of the selection process” (54).
Longitudinal and Cross-Sectional Studies
Quasi ( 拟 )-studies are often employed to analyze the changes or effects of something over long
periods of time, such as in longitudinal studies. They also look at the changes or effects of
something at different phases in its process at a single point in time, such as in cross-sectional
studies. Myers and Hansen (1997) describe the two main ways that time can be employed within
the quasi-experimental framework, “Some quasi-experiments look for changes or differences
93
over time. Longitudinal studies follow the same group of subjects and take measurements at
different points in time. Cross-sectional studies select groups of subjects who are already at
different stages and compare them at a single point in time. A pretest/posttest design can be used
to assess whether the occurrence of an event alters behavior, but the design is low in internal
validity” (111).
There are two other important elements to consider with longitudinal and cross-sectional
studies. As covered, in longitudinal studies the subjects are studied over extended periods of
time. So, for example, a cohort ( 队列 ) could be measured every year as it transitions through
some multi-year program. Under these conditions ( 条件 ), the researcher can choose to measure
the same subjects year after year, or they can randomly select subjects from the cohort ( 队列 )
population each year for measurement. Regarding cross-sectional studies, the idea is that subjects
at different phases of some process or program are measured at a single point in time, but this
“single-point” isn’t expected to be at the same exact time to the minute for all subjects. For
practical reasons this is usually unfeasible. Instead, it is expected that different subjects or
different groups are measured within the shortest, yet most reasonable, time frame (days, weeks,
perhaps even months) to capture the subject’s measurements or scores as they exist in their
positions in some process.
ABA Designs
Experiments are generally deemed “quasi” ( 拟 ) when they are lacking in certain controls; e.g.
when there is no random assignment, when there is only one group (there is no control group, as
in single group or time-series designs), or when the experimental environment cannot be
controlled (as in many “natural experiments”). A prime example of this type of quasi research is
the “ABA design.”
ABA designs involve a single subject, or a small number of subjects, who undergo
treatments in a sequential ( 顺序 ) fashion to determine their effectiveness. A treatment phase is
compared to a baseline ( 基线 ) phase in a controlled or uncontrolled experimental environment.
The ABA arrangement is the simplest version of this technique; though, it will usually come in
more complex forms, such as ABABA. Here A is the baseline ( 基线 ) condition ( 条件 ) and B is
the treatment. The basic design operates like this: a baseline ( 基线) condition (条件) is observed,
a treatment phase is instituted for a time wherein a treatment is provided to the subject to reduce
some problem or promote some positive behavior. The treatment is then removed, and the
subject’s behavior is observed again without the treatment (the return of the baseline phase). This
cycle may continue for numerous rounds. The subject can be measured repeatedly ( 重复 )
throughout this process (often tens to hundreds of times), and the results are usually diagramed
( 可图示) visually (often using frequency graphs) to show any potential swings from baseline ( 基
线 ) to treatment and vice-versa. This design is like the Interrupted Time-Series Design as it uses
a single group and deploys treatments in a sequential fashion, but the ABA design usually studies
fewer subjects and generates many more observations.
For instance, let’s say a new treatment is being tested on a severely misbehaving child.
The severely misbehaving child’s normal behavior is observed for a period of time—this initial
time is the A condition ( 条件 ), otherwise known as the baseline ( 基线 ) condition. The child then
undergoes a treatment phase for a period of time where the behavior is corrected or changed in
some way by the experimenters—this is the B condition ( 条件 ), otherwise known as the
treatment condition. Then, the treatment is removed, the A condition (baseline/no treatment)
94
reinstated, and the child’s behavior is observed without the treatment. If the number of times the
child misbehaved during the A phase was high, and if it was lower during the B phase, and
higher again in the second A phase, there would likely be greater support for the effectiveness of
the treatment.
These ABA designs require a detailed analysis of the responses of a small number of
subjects. Unlike in true experimentation where subjects are often measured once or twice, the
pretest and the posttest or just the posttest, in these designs, subjects are usually measured
numerous times so that detailed information of any potential swings between the baseline ( 基线)
and the treatment phases can be provided.
Also, in true experiments, the data in the dependent variable is usually put through
rigorous statistical tests to determine if there are differences between the experimental groups,
but in ABA designs, because of the small number of subjects and lack of controls, statistical tests
are usually impractical ( 不切实际 ) and of little value. Instead of statistical tests, ABA designs
often use visual representations (frequency graphs) that detail differences between the baseline
phases and the treatment phases—thus, one can see the necessity of employing numerus
measurements/observations throughout the experimental process. ABA designs are often used to
better understand how unwanted behaviors or other unwanted things can be changed in both
human and animal subjects.
Quasi-experiments are most often used in the social sciences, in fields such as psychology,
sociology, and education.
Study Questions:
1) What is a true experiment?

2) What is the Solomon-four group design trying to test?
3) What is a quasi-experiment? Provide an example of a quasi-experiment.
4) What is the purpose of having many pretests and posttests in interrupted time-series
designs?
5) What is the difference between a longitudinal study and a cross-sectional study?
6) What is an ABA design? Provide an example of an ABA design.
Assignment / Coordinating subjects or elements into different groups.

Causation / When the independent variable produces a significant change in the dependent
variable, it provides greater evidence of this.
95
Extraneous / Variables that operate outside of the experimental design that may influence the
results.
Imply / Transmitting a message without directly stating or saying it.
Manipulate / Having the capacity to change or alter something to test its ability to influence the
dependent variable.
Nonequivalent / Unequal; experimental groups are different in some way prior to receiving the
treatment.
Outcome / The results of an experiment or study.
Pre-existing / Something that exists prior to the onset of something else.
Quasi / When one thing is nearly the same, though distinctly different, compared to another thing
in makeup or operation.
Random / Selection determined by chance.
Replicate / Reproducing the same experiment one or more times.
Representative / One person, group, or entity takes the place or speaks for another person, group,
or entity.
puzzle:
96
ACROSS
4 Selection determined by chance.
10 Unequal; experimental groups are different in some way prior to receiving the treatment.
11 Having the capacity to change or alter something to test its ability to influence the dependent
variable.
DOWN
1 When the independent variable produces a significant change in the dependent variable, it
provides greater evidence of this.
2 Transmitting a message without directly stating or saying it.
3 One person, group, or entity takes the place or speaks for another person, group, or entity.
5 Coordinating subjects or elements into different groups.
6 Reproducing the same experiment one or more times.
7 Variables that operate outside of the experimental design that may influence the results.
8 The results of an experiment or study.
9 When one thing is nearly the same, though distinctly different, compared to another thing in
makeup or operation.
97
Terms: assignment, causation, extraneous, manipulated, nonequivalent, quasi, random, replicate,

and representative.
1. Random __________, randomly designating subjects to different experimental groups, is a

fundamental requirement if one is planning to conduct a true experiment.
2. Experimental __________ cannot be claimed unless there is random assignment, a way to
measure the dependent variable, a controlled experimental environment, and the capacity to
manipulate the independent variable.
3. The researcher failed to control numerous __________ variables, causing the post-test
measurements to be variable to the point that no differences were found between the groups.
4. The dependent variable is not directly __________ by the researcher, it responds to the
independent variable and changes accordingly.
5. When experimental groups are __________, when there is no random assignment or when
there is random assignment with a small sample size, subjects can be given pre-tests to
determine equivalency on certain metrics.
6. __________-experimentation usually reflects true-experimentation in its general framework,
the difference being that the former lacks one or more fundamental elements of the latter.
7. Without __________ selection the sample may not reflect the population, external validity
may be reduced, and the experimental results may not be generalizable.
8. To claim that a study has high reliability, researchers should be able to __________ their
results on subsequent tests.
9. To claim that the effects of a study will likely be reflected in the population, the sample
should be ___________ of the population.
Chapter 10: The Meta-Analysis
98
A meta-analysis is a systematic ( 系统 ) collection and analysis of all the known research that
directly studies the effects of a specific independent variable on a specific dependent variable, to
determine the overall effect of the independent variable on the dependent variable. A meta-
analysis also takes the form of a systematic collection and analysis of all the known research that
directly studies the relationship between certain variables ( 变量 ), to determine the overall
strength of the relationship. Mainly though, a meta-analysis estimates the overall statistical ( 统
计 ) effect of a treatment across all the known research that directly relates to the treatment and a
dependent variable. Statistical methods are used to produce a clearer understanding of the effects
of a treatment, as numerous studies are combined and synthesized (合成) to provide evidence for
or against the treatment’s effectiveness.
The results of numerous corresponding ( 对应 ) studies are combined to paint a clearer
picture of: 1) the effects of a specific treatment on a dependent variable, or 2) the strength of the
relationship among specific variables.
A meta-analysis uses effect sizes to estimate the effectiveness of a treatment. Effect sizes
measure the magnitude (大小) of the difference between the experimental groups. The greater the
difference between the groups, the more likely it is that the treatment is effective (given that this
difference is in the direction of improvement rather than diminishment). When the effect sizes
are uniformly ( 均匀 ) similar among the different studies included in a meta-analysis, greater
evidence for or against the effectiveness a treatment may be obtained. If effect sizes are
relatively dissimilar, a meta-analysis may be able to forward evidence explaining why this is
potentially the case (such as experimental differences, subject differences, environment
differences, etc.).
Selecting Research for Inclusion in a Meta-Analysis
When conducting a meta-analysis, concise ( 简洁 ) and objective ( 客观性 ) criteria ( 标准 )

for the selection of the research studies to be included in an analysis should be employed. The
researcher should work to reduce the possibility that publication bias ( 偏差 ) (only published
research is included in the meta-analysis) or researcher bias (studies with positive results are
disproportionately selected for inclusion) has not infiltrated the selection process within the
meta-analysis. This bias can be reduced by carefully defining the criteria that is to be used when
selecting research for an analysis.
There will often be many studies (using slightly different variables, slightly different
treatments, slightly different environments, etc.) that could potentially be incorporated ( 合并 ) in
the meta-analysis, thus, selection guidelines and selection criteria ( 标准 ) are important in
determining the exact variables that are to be studied in the analysis. Harris (2005) explains how
meta-analytic data is retrieved, “The data are gleaned from published research reports and (where
they can be obtained) raw data sets (usually unpublished but provided by cooperative
colleagues). The meta-analyst usually strives for a complete census of all studies of a given
effect (i.e., of the relation between a particular conceptually defined independent and a particular
conceptually defined dependent variable)” (50).
The researcher should also determine if/how unpublished research is to be included in the
meta-analysis—as the meta-analysis may lose validity due to the file-drawer problem.
 The file-drawer problem: Research that is not published is often unknown to the
researcher conducting the meta-analysis. This unpublished research is not included into
the meta-analysis, potentially biasing ( 偏差 ) the results of the meta-analysis. Research
99
may not be published for many reasons, but it is often due to disinterest among
publications because the treatment was ineffective, there was no correlation among the
variables, there were no interesting findings, etc. If only widely known research or
published research is included in a meta-analysis, while unpublished results are excluded
(research that has been placed into the “file-drawer”), readers of the meta-analysis may
not be receiving a complete understanding of the treatment in question.
Study Questions:
1) What is a meta-analysis?
2) What is an effect size?
3) What is the file-drawer problem?
4) Should unpublished research be included in a meta-analysis?
Criteria / Guidelines for the operation, procedure, or for subject selection.

Dependent / The variable that is measured after the onset of the treatment.
Effect Size / The magnitude or the amount of difference between treatment groups.
Estimate / Approximating a score or figure.
Objective / Dispassionate; not allowing one’s personal opinions or motivations to influence one’s
interpretations or decision making.
Statistics / A form of mathematics used to identify relationships among variables.
Synthesize / When elements are combined or merged to present a more whole understanding of
something, increase the effectiveness of something, etc.
Treatment / Cause; independent variable; the variable that is manipulated.
100
puzzle:
ACROSS
3 The variable that is measured after the onset of the treatment.
5 Guidelines for the operation, procedure, or for subject selection.
6 A form of mathematics used to identify relationships among variables.
7 Dispassionate; not allowing one’s personal opinions or motivations to influence one’s
interpretations or decision making.
8 Cause; independent variable; the variable that is manipulated.
DOWN
1 When elements are combined or merged to present a more whole understanding of something,
increase the effectiveness of something, etc.
2 Approximating a score or figure.
4 The magnitude or the amount of difference between treatment groups.
101
Terms: criteria, dependent, effect size, estimates, statistics, and synthesize.
1. The selection __________ was so limiting that too few studies were available for the meta-
analysis.
2. The __________ variable was improperly measured. This is because the measurement
instrument was used over a long period of time and it became more variable and less precise.
3. The __________ of the experiment was low, indicating that the magnitude of the difference
between the treatment group and the control group was not large.
4. When researchers select subjects from a population, split subjects into groups, apply
treatments and controls, and compare the mean scores of the groups, the results will be
__________ of the true scores—experimental error will likely distance these approximations
from the true score in some way.
5. The __________ were conducted improperly; data was entered haphazardly, the columns
were incorrectly labeled, and the wrong tests were run through the software.
6. It is often valuable to __________ quantitative research and qualitative research into a
singular study to provide a more holistic and all-encompassing view of a phenomena.
Chapter 11: Ethical (伦理) Concerns

Ethics ( 伦理 ), when boiled down and simplified, represents ( 代表 ) “What ought to be done?
Ethics is the study of the right behavior” (Singleton & Straits, 1999; 513). The safety and
102
wellbeing of the subjects throughout the entirety of the research process (this includes any
potential aftereffects of experimentation once the study is completed) should be the
experimenters’ primary concern. Subjects are to be engaged ethically during the research
process, and their identities and personal information, when appropriate, need to be kept strictly
confidential ( 机密 ). Miller (2005) explains the foundational ethical principles that ought to be
maintained throughout the experimentation or research process:
“Ethical practices require that the participant:
 be allowed the opportunity to decline to participate in the research and to withdraw from
the research at any time without penalty;
 be informed of what will be asked of him or her as a function of participating in the
research;
 be protected from psychological and physical harm;
 can assume that whatever information he or she has provided the researcher will be
treated confidentially;
 has the opportunity to be debriefed in a manner which removes any misconceptions he or
she has about the research” (141).
Voluntary (自愿) Participation
Participants need to voluntarily ( 自愿 ) subject themselves to an experiment. Dane (1990)

describes the voluntary nature of experimentation, “Voluntary participation refers to the
participants’ rights to freely choose to subject themselves to the scrutiny inherent in research.
Anyone involved in research should be a willing participant” (39). Subjects should have the
capacity ( 能力 ) to give clear consent ( 同意 ) with full knowledge of the risks involved in the
experiment. If there is forced participation or coercion ( 强迫 ), the experimentation is unethical,
and the experimenters may be subject to severe reprimands ( 谴责). Lavrakas (2008) explains the
basic requirements of “voluntariness” (自愿):
“The essential qualities of “voluntariness” include the following:
 The subject has a choice to participate.
 The choice is made without coercion.
 The choice is made without undue influence.
 The subject has foreknowledge of risks or benefits.
For a choice to occur, subjects must be of age and have the mental capacity to make such
decisions. When this assumption cannot be upheld, this decision is left to the subjects’ parents or
legal guardians” (para. 2).
Full Discloser (披露)
Prospective ( 预期 ) participants should be made aware of any potential risks involved in a future
study, and they should have a good understanding of what will occur during the experimentation
(if there is predetermined and necessary deception ( 骗局 ) involved in a study, then different
considerations may be made). After the subject is given the information concerning the risks
involved and other aspects of the experimentation process (the amount of time that is required,
who or what the subject will be interacting with, etc.), the subject is then free to voluntarily ( 自
愿) join the study or refuse inclusion into the study.
103
Physical and Psychological Harm
There should be no physical or psychological harm incurred (招致) by the subjects. If there is the
possibility of physical or psychological harm to the subjects over the course of the
experimentation, then:
1) The subjects should have a clear understanding of the risks involved prior to
experimentation.
2) The harm should be minimized as much as possible both during and after
experimentation.
3) The subjects should be able to quit the experiment at any time without shame or a
negative response from the research staff.
4) The benefits to society and to science in knowledge gained should far exceed any
potential for harm.
Miller (2005) details some of the major risks or costs that could arise that need to be taken into
consideration when deciding if the potential research is ethical ( 伦理 ) and should be carried out,
“Some of the potential costs to the participant are social and physical discomfort, boredom,
anxiety, stress, loss of self-esteem, legal risks, economic risks, social risks, and other aversive
consequences” (136).
Risk/Benefit Analyses to Determine Research Acceptability
Institutions generally have review boards that are responsible for determining if potential
research meets the ethical ( 伦理 ) standards that are necessary for the research to proceed—
review boards such as the Institutional Review Board (IRB) in the United States. These review
boards conduct a risk/benefit analysis on proposed research submitted by researchers to
determine if the research is acceptable. This review is conducted to ensure the safety of the
prospective participants. If the risks, physical or psychological, experienced by the subjects are
far less than the potential scientific knowledge gained from the undertaking, there is a greater
likelihood that the review board will accept the proposal and allow the experiment to proceed.
Myers and Hansen (1997) summarize this risk/benefit analysis, “It is up to the IRB to determine
that any risks to the individual are outweighed by potential benefits or the importance of the
knowledge to be gained” (30).
Coercion (强迫)
People should not be forced or pushed to participate in experimentation through acts of coercion
( 强迫 ). The experimenters’ power or social position should in no way determine whether people
agree to participate in experimentation. One’s employment or status as a student ought not be
predicated (断言) upon one’s participation or nonparticipation in a study. A student should never
feel compelled ( 迫使 ) to participate in a study, feeling that he/she may incur ( 招致 ) the disfavor
of the faculty, incur (招致) the disfavor of the academic department, or receive a reduced grade.
Also, the monetary rewards or other rewards provided to subjects for their participation
should not be so great (beyond a reasonable reward for hourly or daily work) that the subjects
would be compelled ( 迫使 ) to participate, feeling that the reward for participation is too great to
pass-up no matter what the risk. Dane (1990) refers to this type of coercion ( 强迫 ) as giving
104
participants an “offer they can’t refuse” (39). He explains that coercive incentives are offered in
two ways, through the reduction of punishment and through positive rewards, writing,
“Sometimes coercion is rather clear, as when an administrator offers a prison inmate a “choice”
between participation and solitary confinement, or when a researcher offers several hundreds of
dollars to a prospective participant for a single day of participation” (39). Both types of coercion
are unethical and should be avoided.
Deception (骗局):
Deception involves misleading participants as to the true nature of a study. It’s imperative that
subjects fully understand the true nature of an experiment prior to agreeing to participate in it.
Though, there are times when deception ( 骗局 ) is necessary and part of the experimental design.
These deception-based ( 骗局 ) experimental designs are usually required to meet more strict
ethical criteria (标准) before they are implemented (实施) (namely, the no harm criteria). Once an
experiment is complete, subjects are usually debriefed ( 汇报 ) as to the true nature of the
experiment.
Sometimes researchers will work alongside what are referred to as confederates ( 帮凶 ) to help
trick or deceive subjects in some way during experimentation.
Confederate ( 帮凶 ): A confederate ( 帮凶 ) is a member of the research staff who works
with the experimenters to deceive ( 骗局 ) the participants. A confederate may pose as a
participate to manipulate (操纵) the behavior of the other, real, participants.
A good representation ( 代表 ) of experimental deception ( 骗局 ) is Stanley Milgram’s famous

obedience ( 服从 ) experiments. Milgram deceived subjects into believing that they were
“teaching” other subjects (or “learners”) how to correctly answer questions on a test using
electrical shocks as means of punishment/education. The other subjects (the learners) were
confederates ( 帮凶 ) working with Milgram with full knowledge of the real purpose of the study.
The real purpose of the study was to see if, and how far, the subjects (the teachers) would punish
the learners with gradated amounts of electrical shocks if given orders from an authority figure
(also known as the “experimenter”) within the controlled experimental environment. No one was
given electrical shocks or physically harmed during the experiment, though the subjects (the
teachers) did not know this information. The confederates ( 帮凶 ) (the learners) pretended to be
shocked and suffer harm when the subjects (the teachers) applied the electrical punishment.
105
A visual diagram (可图示) of the layout of Milgram’s obedience (服从) experiment. The experimenter or authority
figure is located at the top left of the image, the subject or “teacher” positioned on the right side, and the confederate
(帮凶) or “learner” stationed on the bottom obscured (遮蔽) by a wall so that only the learner’s sounds could pass
through (Wikipedia Commons).
Study this experiment on your own. Do you think that the deception ( 骗局 ) undertaken in this
experiment reached a point that was unethical from a risk/benefit perspective? Do you think that
the experiment should have been conducted or should it have been prevented from happening for
ethical reasons? Do you think that the subjects (the teachers) experienced significant
psychological harm?
106
The advertisement posted to attract participants to Stanley Milgram’s well-known obedience experiment (Wikipedia
Commons).
Debriefing (汇报)
Debriefing involves explaining the true nature of a study to the participants after the study has
been concluded. Debriefing ( 汇报 ) takes place after a study is over and after the subjects have
been deceived ( 骗局 ), mislead, or not provided with all the necessary information to have a
complete understanding of the study. For example, subjects will be told that they were either the
treatment group or the placebo (安慰剂) group.
This is how an experimental process that involves debriefing proceeds: a study begins,
the participants are deceived ( 骗局 ) or not provided with all the necessary information in some
way, the experimentation is concluded, the data is collected, and then the subjects are debriefed
(汇报) as to the true nature of the study.
Funnel (漏斗) Debriefing (汇报)
Funnel ( 漏斗 ) debriefing ( 汇报 ) is a common method used to debrief participants. When the

experiment has been concluded and it is time to debrief ( 汇报 ) the subjects as to the true nature
of the research, the researcher will often debrief the subjects in a way that maximizes the
extraction ( 提取 ) of useful information for the researcher to improve future research. This
information involves the degree to which the subjects were aware that they were being deceived
( 骗局 ) by the researcher. The researcher may ask the subject, “What did you think we were
studying?” or “What did you think we were measuring?” and if the subject responds that there
was some notion or feeling of deception, the researcher will work to ascertain how the subject
came to this conclusion so that in future studies this can be remedied ( 补救). Slowly, through this
question and answer process, the experimenter will inform the subject of the actual nature of the
study. As mentioned, funnel ( 漏斗 ) debriefing ( 汇报 ) is used to improve future research. If a
participant can articulate ( 表达 ) if, when, and how they were alerted to the true nature of an
experiment, then researchers can adjust future experiments to reduce the possibility of this
occurring again.
Anonymity (匿名) and Confidentiality (机密)
The participants identities are usually kept anonymous ( 匿名 ) or unknown to the public. The
experimenters are expected to construct an experiment in a way that makes it extremely difficult
to identify which subject produced which data set. Subjects are usually assigned numbers for
identification to hide their identities, and any means to trace ( 追踪 ) the subjects assigned
numbers to their identities are expected to be locked way in secure files and later destroyed. Any
breach (违反) of subject anonymity (匿名) is expected to result in severe ethical (伦理) violations
for the experimenter. The participants identities should not be included in the formal write-up
and published results. Usually, the participants’ personal information will be unknown to anyone
not directly interacting with the participants.
Informed Consent (同意)
107
A research participant is usually required to sign a consent ( 同意) form prior to experimentation.
A consent form is a formal document that makes it clear that the subject is aware of the risks
involved in the experiment, that the subject understands the general nature of the experiment, and
that the subject knows that it’s alright to drop out of the study at any time. Consent ( 同意 ) is
usually given in signature form, though there are times in less risky situations where a subject’s
participation is consent ( 同意) enough (e.g. survey ( 调查) responses), or verbal consent is all that
is required. Miller (2005) describes this contractual ( 合同 ) agreement between researcher and
participant, “Consent can be thought of as a contract in which the participant agrees to tolerate
experimental procedures that may include boredom, deception, and discomfort for the good of
science, while the researcher guarantees the safety and well-being of the participant. In all but
minimal-risk research, informed consent is a formal process whereby the relevant aspects of the
research are described along with the obligations and responsibilities of both the participant and
the researcher” (131).
Animal Research
When conducting research with animals, it’s important that the animals are treated responsibly
and are cared for in a humane ( 人道 ) way. It’s important to understand that animals feel pain as
well, and that researchers are not to inflict needless or unnecessary suffering upon the animal
subjects. A cost/benefit analysis should be conducted prior to experimentation to determine if the
physical or cognitive ( 认知 ) costs that the animals could potentially experience justify the
potential scientific value of the research.
A rat swimming the “Morris Water Maze”—a experimental maze designed to test memory and body orientation (方
向) (Wikipedia Commons).
Plagiarism (剽窃) and Fraud (欺诈)
Plagiarism is when one represents ( 代表 ) the research or work of others as if it is one’s own
research or work. This ranges from one failing to cite ( 引用 ) the data of others, to one copying
large portions of the work of others and taking credit for it without attributing (归因) the work to
the original author. Fraud is the knowing misrepresentation (失实陈述) of the results of research.
It is stating that some research or experimentation produced a certain kind of data or results when
in fact this was not the case. This is naturally a massive ethical ( 伦理 ) breach ( 违反 ) within
academia.
108
Study Questions:
1) What are ethics?

2) What is deception?
3) What does it mean to be debriefed about something?
4) What is funnel debriefing?
5) What does full discloser mean?
6) What is informed consent?
7) When is a subject’s participation consent enough?
8) What is fraud?
Anonymous / Nameless; keeping the participants names and other personal information
unknown to the public and other researchers whose knowledge of the information is unnecessary.
Attribute / Giving the author or creator recognition for their work; designation.
Breach / To transgress a previous agreement; to turn against an accepted contract or promise.
Cite / To give credit or acknowledgment to the work and research of others if it exists within
one’s own research.
Coercion / Pressuring someone into doing something in which they normally would not do.
Cognition / Pertaining to the mind and mental functioning.
Confidential / Keeping personal information private between researchers and participants.
Consent / Agreeing to join and participate in an experiment or research of some kind.
Contract / A legally binding document that expresses the intentions and expectations of people.
Debrief / Once the research is complete, the researchers will explain to the participants the real
reason for the research.
Deception / The employment of deceit to obtain a more real or natural reaction from the
participants.
Disclose / Revealing or releasing information.
Ethics / How people should behave and conduct themselves under certain circumstances.
109
Humane / Treating others with dignity and compassion.
Incentive / A stimulus that motivates people to engage in a behavior or action.
Incur / To experience or receive something.
Misrepresentation / The researcher tells participants that the reality of something is one way,
while knowing that this is information is untrue.
Voluntary / Offered on one’s own volition; deciding to participate in something independently
and without coercion.
puzzle:
110
ACROSS
4 A legally binding document that expresses the intentions and expectations of people.
8 To transgress a previous agreement; to turn against an accepted contract or promise.
9 Keeping personal information private between researchers and participants.
11 Pressuring someone into doing something in which they normally would not do.
12 Agreeing to join and participate in an experiment or research of some kind.
DOWN
1 Giving the author or creator recognition for their work; designation.
2 To give credit or acknowledgment to the work and research of others if it exists within one’s
own research.
3 Revealing or releasing information.
5 Pertaining to the mind and mental functioning.
6 Once the research is complete, the researchers will explain to the participants the real reason
for the research.
7 The employment of deceit to obtain a more real or natural reaction from the participants.
10 Nameless; keeping the participants names and other personal information unknown to the
public and other researchers whose knowledge of the information is unnecessary.
search puzzle:
111
1. How people should behave and conduct themselves under certain circumstances.
2. Treating others with dignity and compassion.
3. A stimulus that motivates people to engage in a behavior or action.
4. To experience or receive something.
5. The researcher tells participants that the reality of something is one way, while knowing
that this is information is untrue.
6. Offered on one’s own volition; deciding to participate in something independently and
without coercion.
112
Terms: attributing, breach, cite, coerced, contract, debrief, ethical, incentive, and voluntary.
1. __________ the correct authors for their work in group publications is vital in determining
who should receive recognition for academic publications.
2. A __________ of proper ethical behavior occurred when the subject’s anonymity was lost.
3. Remember to __________ all the material and ideas that belong to others that were included
in your research.
4. The instructor __________ his students into engaging in his experiment by explaining that if
they did not participate they would fail his class.
5. The lead researcher consulted with the legal department to help her draw up the __________
that was to be offered to the research assistants.
6. When possible, it’s important to __________ subjects so that they understand the true
reasons for their participation in the research and so that they can continue to follow the
results of the study if they have an interest.
7. The __________ behavior exhibited by the researcher was impeccable; she had the
participants sign a consent form, treated them with respect and dignity, and fully disclosed
the true nature of the research once the experiment was completed.
8. It would be unethical for researchers to provide too strong of an __________ to attract
participants, such as an overabundance of cash, because people may find it difficult to refuse
involvement in experimentation in the face of large rewards.
9. Subject participation within an experiment should be __________.
References
Atkinson, P. & Flint, J. (2001). Accessing hidden and hard-to-reach populations: Snowball
research strategies. Sociological Research Update, Issue 33.
Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five
approaches. LA: Sage.
Dane, F. C. (1990). Research methods. CA: Brooks/Cole Publishing Company.
Gomm, R. (2008). Social research methodology: A critical introduction. NY: Palgrave
Macmillan.
113
Harlow, J. M. (1868). Recovery from the passage of an iron bar through the head. Publications
of the Massachusetts Medical Society, 2, 327-47.
Harris, R. J. (2005). Traditional nomothetic approaches. In S. F. Davis (ed.), Handbook of
research methods in experimental psychology (pp. 41-65). MA: Blackwell Publishing.
Lavrakas, P. J. (2008). Voluntary participation. Encyclopedia of Survey Research Methods.
http://dx.doi.org/10.4135/9781412963947.n629
Miller, R. L. (2005). Ethical issues in psychological research with human participants. In S. F.
Davis (ed.), Handbook of research methods in experimental psychology (pp. 127-150).
MA: Blackwell Publishing.
Myers, A. & Hansen, C. (1997). Experimental psychology. CA: Brooks/Cole Publishing
Company.
Smith, R. A., & Davis, S. F. (2005). The changing face of research methods. In S. F. Davis (ed.),
Handbook of research methods in experimental psychology (pp. 106-126). MA:
Blackwell Publishing.
Singleton, R. A. & Straits, B. C. (1999). Approaches to social research. Oxford: Oxford
University Press.
“Science." Merriam-Webster.com. Merriam-Webster, n.d. Web. 22 Aug. 2018.
"Theory." Merriam-Webster.com. Merriam-Webster, n.d. Web. 22 Aug. 2018.
Terminology
Chapter 1: Empiricism
Data: Units of information, usually formed and recorded numerically.
Pieces of information; usually generated or counted quantitatively: Data
Once all the data has been collected, statistical analysis can be conducted to determine if there
are differences between the groups.
Much of the experimental data was lost when the computer’s hard-drive overheated and was
destroyed.
Experience: Practice. The enhanced perception of something, or the ability to do something
through practice. Previous engagement or involvement in an activity that provides greater insight
into these activities or events.
A better understanding of something through previous involvement with it: Experience
A person who has little experience with statistical software will need to consult guidebooks to
help them along.
The researchers experience was on full display when she effectively resolved a difficult issue.
114
Experiment: Scientific test. Provides evidence for or against a prediction drawn from a
hypothesis. Systematic procedure that often has “cause and effect” ramifications. The process of
determining the relationship between the researcher’s hypothesis and reality. It involves the
systematic testing of predictions within the real world to provide evidence for or against “cause
and effect” statements.
Testing phenomena through scientific processes: Experiment
The experiment was flawed in numerous ways: there was no control group, external variables
were not controlled, there was no random assignment, and the dependent variable was not
correctly measured.
A true experiment should contain four elements: the random assignment of subjects to different
experimental groups, more than one level of the independent variable (more than one
experimental group), the capacity to control the research environment, and the ability to measure
the dependent variable.
Evidence: Any empirical data, experimental data, or other information that supports or refutes a
hypothesis.
Data/information that can increase or decrease the acceptance of an assertion: Evidence
There was little evidence that the medicine was effective—the experiment yielded no conclusive
results.
Cause and effect statements about phenomena usually require empirical evidence.
Hypothesis: Proposition; educated prediction; assumption; speculation. Explanation for the
occurrence of a phenomena, behavior, event, etc.
An educated explanation for an event or phenomena; an educated prediction of the results of an
experiment: Hypothesis
The hypothesis was correct; the researcher’s ability to predict and forecast experimental results
was quite impressive.
It’s usually important that researcher’s clearly state the hypothesis prior to the onset of
experimentation. Stating the hypothesis after the experimental data becomes available can
change the statistical procedures that are used to draw conclusions, and it can change the
experimental procedures that are followed (e.g., using a one-tailed test or using a two-tailed test).
Investigation: Inquiry into a phenomenon. Seeking answers or information about how the world
operates.
An exploration or inquiry into a phenomena or event: Investigation
The investigation into the cause of the behavioral change yielded no convincing information.
The researcher’s investigation into the issue revealed that the illness was not caused by the
substance in question.
115
Observation: Obtaining information and data through sensory experiences. Collecting data
through scientific processes. Also, monitoring a person, social interaction, phenomena, event,
etc. and recording the relevant information.
The collection and recording of data/information: Observation
The observation of the subjects was halted after it was found that the subjects determined who
was in the placebo group and who was in the treatment group.
After careful observation, it was revealed that the mice would locate the cheese faster if under
the influence of medium levels of caffeine.
Qualitative: Interpretive; impressionistic. Interviews and observations. Uses words instead of
numbers to describe and explain a phenomenon. Often used to describe the lived experiences of
people.
Descriptive explanations concerning how people interpret reality: Qualitative
The qualitative study captured the way in which subjects understood and interpreted their social
interactions.
A mixed-methods approach combines both quantitative and qualitative methods. It’s research
that combines statistical research with dialog centric research to understand a phenomenon in a
deeper way.
Quasi: Something has nearly the same makeup or constitution as another thing. Nearly the same.
Almost identical.
Things are closely related in their fundamental design, though different in important ways that
distinguish them from one another: Quasi
The researcher’s quasi-experiment was valuable in displaying the effectiveness of the therapy
despite the lack of random assignment.
The research was deemed “quasi” when it became clear that the researcher would be unable to
adequately control the experimental environment.
Research: A systematic investigation, usually through the collection of data and information, to
increase our knowledge and understanding of the world.
An investigative undertaking to better understand the nature of the world: Research
The experimenter’s research never met the standards set by the academy.
Research combining both quantitative and qualitative investigative processes is known as a
“mixed-methods” approach.
Senses: Physiological responses to an environment that increase our understanding of the world.
The gathering of data and information through physiological organs. Our five senses: sight,
smell, touch, taste, and hearing.
116
Physiological organs that generate information and data; sight, hearing, touch, etc.: Senses
Empirical information is gathered from the senses.
We use our senses to gather information about the world. We then take this information and,
after analyzing it, use it to make claims about the world.
Statistics: A form of mathematics (data is analyzed and processed using mathematical formulas)
useful for making statements and conclusions about the relationships between variables. A form
of mathematics in which data is analyzed and exhibited.
Mathematical processes that make statements about the relationships among variables: Statistics
The researcher manipulated the statistics so that the results would match the hypothesis. This
unethical behavior is antithetical to the expected behavior of academy members.
Statistical procedures are employed after the data is collected to determine if there are
differences among the groups or to determine relationships among the variables.
Survey: Method of information gathering and data collection, usually administered by asking
subjects questions.
Gathering information by questioning subjects systematically or in a more open-ended fashion:
Survey
Many participants were unwilling to complete the survey due to the large number of questions it
encompassed.
An online survey is often an effective way to quickly and conveniently gather opinion-based
data. Though, external validity is often low because the responses may not fully capture the
opinions of a population.
Systematic: Methodical; orderly; procedural; a logical order of operation.
Process oriented; sequential in operation: Systematic
The researchers experimental design was systematic in its operational framework; the
The systematic collection of data is typically a major component of the experimental process;
without which greater variability in the results is likely.
Theory: A collection of propositions that are assumed to be true that explain a phenomenon.
A unified collection of propositions that explain a phenomenon: Theory
The researcher’s theory was never supported by empirical evidence.
A theory is usually accepted only after years of rigorous study, experimentation, and critical
review.
117
Alternative (Alternative Hypothesis): A different option. Choosing amongst two or more
things or potentialities. The hypothesis that represents a significant “effect” within the
experiment. There is a difference. There is a difference between the treatment group and the
control group. There is a difference between the levels of the independent variable.
The hypothesis that states that the new treatment appears to be effective: Alternative
When something is not the default or normal option, it may be considered the alternative option.
The alternative is not the standard position; it represents a new or different direction.
Corroborate: Supporting; supporting evidence; validating information or data.
Information or statements that support a claim: Corroborate
One should corroborate one’s evidence with other evidence to provide a stronger case for one’s
theory.
Corroborate your evidence with other evidence to support your claims about the world, without
which you leave yourself open for criticism.
Cyclical: Circular. Round and round in a circular manner. Series of events that follow the pattern
of a circle.
Circular and round: Cyclical
Seasons come and go in a cyclical manner.
When something is moving in a circular manner, it can be said to have a cyclical nature.
Empirical: Information or data gathered from the senses or from experience. Evidence derived
from sensory organs (sight, sound, touch, taste, and hearing).
Information that is collected from the senses: Empirical
Scientists usually generate empirical evidence to test theories.
Citing empirical data is usually the most legitimate way to make claims about the world.
Falsifiable: The ability to reproduce an experiment to affirm or refute the results. Legitimate
research needs to provide a way for others to test the research to determine if the results are true
or false. It’s important that a researcher explicitly detail the steps taken to produce an experiment
along with the results so that it is falsifiable.
Having the capacity to reproduce an experiment: Falsifiable
The steps taken to conduct research ought to be clearly defined so that the experiment is
falsifiable.
118
If an experiment is not falsifiable, the academic community will likely not respond favorably to
the experimental results.
Forecast: Predicting. Predicting future events or future behaviors.
An educated prediction regarding what will occur in the future: Forecast
She is excellent at predicting what will happen in the future; her forecasts are always correct.
The weather forecast failed to predict rain, causing the experiment to be delayed for three hours.
Hypothesis: Proposition; educated prediction; assumption; speculation. Explanation for the
occurrence of a phenomena, behavior, event, etc.
Explains the predicted results: Hypothesis
Prior to the experiment, the researcher made numerous predictions. Her main hypothesis was that
the new medicine would be effective in reducing blood pressure.
The experimenter’s hypothesis was that the empirical evidence would show that the alternative
hypothesis (that the new therapy was more effective than the old therapy) would be accepted,
while the null hypothesis (that the new therapy was no different from the old therapy in
effectiveness) would be rejected.
Inquiry: Investigation; exploring a phenomenon; study of the nature of our world.
Investigate, search, and question: Inquiry
The lead researcher’s inquiry into the loss of several team members yielded no information.
The researcher’s inquiry into the phenomena produced a great amount of new information and
new insights.
Legitimate: Valid; formally acceptable.
Accepted within the scientific community; understood to be valid: Legitimate
Using the scientific method is a legitimate way to investigate and test phenomena.
There was no legitimate reason to conclude that the researcher altered the data, thus, weakening
the accusations leveled against him.
Null: No difference; no effect. The hypothesis that represents no difference between the
treatment group and the control group. No difference between the levels of the independent
variable. The default position prior to the results or the concluding information.
The hypothesis that states that there is no effect or no difference between the levels of the
independent variable: Null
The null hypothesis represents the position that there is no difference between the new medicine
group and the placebo group.
119
The null hypothesis was rejected in favor of the alternative hypothesis because the two
experimental groups (the treatment group and control group) were statistically different from one
another.
through scientific processes.
Collecting and recording information or data: Observation
Three researchers engaged in the observation of the phenomena over numerous weeks, but they
found nothing extraordinary.
The observation of the subjects was stopped when saturation was achieved (the subjects provided
no new useful data).
Peer: Contemporary. An equal or near equal within a field, occupation, or other sector within
society.
A fellow researcher operating within the same field: Peer
The peer review process determined that the study’s research methodology was flawed.
A peer can provide important feedback and suggestions concerning one’s research plans.
Phenomena: An event, behavior, or occurrence that is observable.
Something that happens that is studied: Phenomena
The phenomena under study was difficult to measure due to its variability and high fluctuation
rate.
The phenomena that you wish to examine has already been thoroughly studied. In what way will
you be able to add to our understanding of it?
Prediction: Forecasting; hypothesizing. A statement about what will transpire in the future.
When a researcher makes a statement about what will happen in the future: Prediction
A hypothesis is a prediction based upon the best available evidence.
Even though the researcher’s prediction was based on previous empirical studies, the results
turned out not to conform with the researcher’s forecast.
Principle: Doctrine; guideline; information that provides behavioral or social guidelines.
A guiding doctrine fundamental to the operation of some process: Principle
The principles of the scientific method are foundational to our understanding of the world.
The ethical principles diagramed in the conduct manual were closely followed by the research
team to minimize any future criticisms.
Proposition: Hypothesis; assertion; explanation of a phenomena or event.
120
Proposal or hypothesis: Proposition
The experimenter’s proposition was shown to be incorrect through a statistical comparison of the
experimental groups.
An experimental proposition should be a product of the study of previous research or pre-
experimentation; a highly speculative assertation should be avoided.
Publication: A formal transmission of written information. Formally making public one’s
research by way of academic journals, books, etc.
The introduction of research to the public through formal and peer-reviewed processes:
Publication
After an academic paper passes the peer-review process it is ready for publication.
The publication of the study came after many rewrites and critical reviews.
Replication: Reproduce; repeatable; consistency of results; obtaining the same result again and
again.
Running the same experiment repeatedly to see if the same results are observed: Replication
An experiment is falsifiable through replication.
Experimental replication revealed no variation in results, thus, increasing experimental
reliability.
Significant: Meaningful; important; worthy of attention and recognition.
When something hits a point when it is considered worthy of notice: Significant
Without statistically significant results, one cannot claim that a new medicine is effective.
The statistically significant results were still not enough to convince many of the top scientists,
they required further testing.
Sound (within the context of general research methods): Valid; reasonable; formally acceptable.
Unburdened by problems, fault, or apprehension: Sound
The experiment is sound, there is little that can go wrong.
The experiment was unsound, the experimental environment was not controlled and the research
process not clearly detailed.
When a proceeding or process is operational, detailed, and step-by-step: Systematic
Your procedure is not systematic; there is too much uncertainty, randomness, and surprise
occurrences.
121
True experiments are usually systematic in operation. This is to ensure that the study can be
replicated and is, therefore, falsifiable.
Variation: Changeable; malleable; can be manipulated.
Something that is changeable or something that does not take on a definite property: Variation
There is too much variation in your research environment—strange people entering and leaving,
the random incorporation of new measurement instruments, changing room temperatures, etc.
There was great variation in the scores of the treatment group, leading the researcher to conclude
that the research environment was not properly controlled.
Chapter 3: Exploratory Research, Descriptive Research, and Causal Research

Average: Denotes a middle-point or middle position (e.g. mean, median, and mode) amongst a
group of data.
The middle position in a data set: Average
The treatment groups average or mean was compared to the control groups mean at the end of
the study to determine the effectiveness of the treatment.
When the average scores of the treatment and control groups are not significantly different, we
cannot stay that the treatment was effective.
Categorize: Classify. The grouping or sorting of people, things, etc.
Arranging data, people, things, etc. into groups based upon some criteria: Categorize
The researcher didn’t categorize subjects according to their age or any other dimension involving
time.
The experimenter didn’t categorize subjects according to time spent with the disease, so that the
effectiveness of the medicine could not be analyzed on this dimension.
Causal: Pertaining to a “cause and effect” relationship. A causal framework for the execution of
research. A causal link between variables is strengthened or weakened dependent upon scientific
results. Does the independent variable “cause” a change in the dependent variable?
When an independent variable produces a statistically significant change in a dependent variable:
Causal
To make an argument for a causal relationship between variables, one needs to ensure that there
are no outside variables influencing the results.
It is only through the implementation of true experiments that one can make causal claims.
Characteristics: Features. The elements that constitute something.
122
Specific features or distinguishing properties: Characteristics
The characteristics that makeup a population may influence how subjects respond to survey
questions.
There were certain characteristics that some of the mice possessed that made them more likely to
quickly locate the cheese.
Classify: Categorize. Group together or sort according to some criteria.
Grouping things together based upon some requirements: Classify
We can classify mice based on their age and the amount of active behavior in which they exhibit.
Before the implementation of the intervention, pre-tests were given to classify people on their
levels of depression.
Confounding: Any variable within an experiment that interferes with and disrupts the
relationship between the independent variable and the dependent variable, thus, distorting the
results. Confounding variables are to be controlled within the experiment or removed to
eliminate or minimize their influence.
A variable that disrupts an experiment and produces erroneous results: Confounding
A highly controlled research environment and well-defined research procedures will often reduce
the influence of confounding variables.
A confounding variable altered the experimental results, forcing the experimenter to restart the
experiment while controlling for this troublesome variable.
Control: Securing and managing the research environment so that confounding variables are
removed or minimized, thus, promoting a more direct relationship between the independent
variable and the dependent variable. All variables within an experimental design should be held
constant (or controlled) except for the independent variable.
Reducing variability and error within an experiment: Control
The control group is usually that which does not receive any new treatment or new stimuli.
The control group can be a placebo group, the old/regular medicine group, or the no medicine
group.
Correlation: The relationship among two variables. Does a change in one variable correspond
with a change in another variable?
A change in one variable is or is not associated with a change in another variable: Correlation
Despite the realization that there was a strong correlation between the two variables, it was not
clear which variable was influencing the other.
123
Correlation does not equate to causation. Just because there is a strong relationship between two
variables does not mean that one is secure in the knowledge of the directionality of the variables,
nor is one secure that there wasn’t a third variable influencing the results.
Dispersion: Distance between data points. The spread of data.
A reflection of the spread of data in a data set: Dispersion
There was a wide dispersion of subjects scores. This led the experimenter to tighten controls
within the experimental environment and increase the precision of the measurement instrument.
There was a minimal dispersion of scores; subject’s responses fell in nearly the same region on
the distribution.
Experimentation: Scientific test. Provides evidence for or against a prediction drawn from a
hypothesis. Systematic procedure that often has “cause and effect” ramifications. The process of
determining the relationship between the researcher’s hypothesis and reality. It involves the
systematic testing of predictions within the real world to provide evidence for or against “cause
and effect” statements.
Testing phenomena using the scientific method: Experimentation
The effectiveness of the medicine required further experimentation before it was determined safe
for the public.
The researcher’s experimentation process consisted of numerous testing phases under an
assortment of conditions.
Formulate: Develop; design; plan. The preparatory stages in the construction or production of
something.
To develop, plan, or construct something: Formulate
The researcher failed to formulate a research design that controlled several extraneous variables.
The mathematician would regularly formulate difficult mathematical equations.
Frequency: Occurrence. How often a phenomenon or an event occurs.
Enumerating the occurrence of an event or phenomena: Frequency
The frequency of the subject’s adversarial behavior was recorded over several weeks.
Despite the survey being distributed to thousands of people, the response frequency was low,
greatly limiting the sample size.
Generate: Produce; create.
Develop or bring into existence: Generate
The researcher failed to generate any new data after the second week of interviews.
124
It is often important that a researcher generate a null hypothesis and alternative hypothesis prior
to the onset of the experiment.
Interview: A formal transfer of information from a subject to a researcher. Meeting to exchange
information to expand our knowledge of the world.
Obtaining information from a subject through a question/answer interaction: Interview
When the experiment is finished, the experimenter will interview subjects to determine if they
knew, or thought they knew, that they were a member of the treatment group or the placebo
group.
Ethnographers will often repeatedly interview subjects over the course of their research to garner
a deeper understanding of the subjects lived experiences.
Laboratory: A controlled environment used for scientific testing, research, and experimentation.
The arena in which many controlled scientific experiments take place: Laboratory
The laboratory was disorganized and unregulated; chemicals were improperly labeled, and the
equipment was seldom cleaned or recalibrated.
The laboratory was in pristine condition, with clean floors, disinfected countertops, and spotless
instruments.
Meta-analysis: By collecting all the known research that studies the effectiveness of a treatment
on a dependent variable, and by combining the effect sizes of this research, it potentially
provides a better estimate of the true effectiveness of the treatment.
An analysis of several independent studies, which cover the same or similar content, to gauge the
overall effect of a medicine, therapy, or other treatment: Meta-analysis
A meta-analysis is often effective in determining what the academic literature as a whole says
about something.
It’s important that researchers clearly define the characteristics of the studies that are eligible to
be included in their meta-analysis, or they may be subject to criticisms of bias in the selection
process.
Operation: The working order of a process. A detailed explanation of how something functions.
When events or actions take place in a procedural and preplanned fashion: Operation
The research operation was broken down into five sequential parts: random selection, random
assignment, pre-test, intervention, and post-test.
The operation of the experiment should be clearly defined so that the results can be replicated,
and so that the work can be falsifiable.
Phenomena: Events, behaviors, or occurrences that are observable or measurable in some way.
125
Something that happens which warrants study or experimentation: Phenomena
The phenomena being studied was difficult to measure, so the experimenters gave participants
different tests which enabled them to analyze it from numerous angles.
The researchers were unable to study the new phenomena; because of the extreme cold, their
instruments were malfunctioning.
Preliminary: Prior to the main portion. Preparing for the main procedure or process.
Doing something before the advent of the main action or main event: Preliminary
The preliminary tests showed that the medicine was effective, but further, and more rigorous
The preliminary analysis of the research process determined that there were likely no potential
ethical concerns.
Properties: Attributes or traits belonging to someone or something.
Distinguishing features of someone or something: Properties
The medicinal properties of the new treatment were unknown.
The molecular properties of the chemical compound were unknown to the research team.
numbers to describe a phenomenon. Often used to describe the lived experiences that people
undergo.
An in-depth study of, usually, a small number of subjects or phenomena that focuses more on
words rather than numbers: Qualitative
A qualitative study of those who suffer from the disease proved to be an intimate depiction of the
intense personal struggles they encountered on a day-to-day basis.
The qualitative information was gathered through face-to-face interviews, wherein subjects were
asked questions about their experiences.
Variable: Something that changes or is changeable. E.g., the researcher varies the independent
variable, meaning that the nature or quantity of the independent variable is changed to examine
its effects on the dependent variable.
Something that can be changed or manipulated: Variable
The independent variable is the variable that is manipulated by the experimenter.
The dependent variable is the variable that is changed by the independent variable and measured
by the experimenter.
126
Chapter 4: Quantitative Research, Qualitative Research, and Mixed Methods Research
Conclusive: Deciding something based on the available evidence; a clear decision or
determination; finality.
When there is a resolution or finality to an experiment: Conclusive
The results were not conclusive. There were not enough subjects and the measurement
instruments produced too much variability.
The results were conclusive. The statistics showed that there was a significant difference
between the two therapies.
Count: Tallying. Determining the number of units or pieces of information in a data set.
The process of determining the numerical amount of something: Count
We are to count all the online transactions last year, not just those transactions covered through
the major online relators.
The researcher would count the number of times the mice turned in the wrong direction.
Data: Units of information; often formed and recorded numerically.
Pieces of information that are often numerical in identity: Data
Gathering data from one subject is not enough to generalize the results to the wider population.
The data was run through statistical processes to determine if there were differences between the
groups.
Descriptive: Describing or providing an account of something. Within research, this descriptive
account is often in the form of a graph that displays the information visually.
A detailed account or interpretation of phenomena; often displayed visually: Descriptive
The student’s thesis was a descriptive account of his time spent living in the rainforest.
The researcher failed to provide a descriptive account of the event. Instead, the researcher simply
noted that it occurred.
Dispassionate: Objective; fair and unbiased; lacking in emotion or compulsions that may
influence research.
Only seeking truth and facts; being detached from motivations unrelated to seeking truth:
Dispassionate
His position was derived from a dispassionate review of the literature and an analysis of the
127
The researchers did not engage in the experimentation process in a dispassionate way. Instead,
they regularly favored the subjects in the treatment group—spending more time with them and
behaving in a way that was disproportionately nice to them.
Exploration: Seeking knowledge or an understanding; searching for information or data;
generating/developing ideas and questions.
Seeking, searching, and questioning: Exploration
If you do not engage in exploration within your academic work, you may miss the opportunity to
generate new theory and new ideas.
The researcher’s exploration into the event yielded little new information.
Generalize: The results of a study can be inferred or extrapolated to the population from the
which the sample was drawn. Conclusions derived from scientific research that can speak to a
larger population.
The ability for a sample to speak for the population from which the sample was drawn:
Generalize
Random selection often enables the researcher to generalize the results back to the population
from which the sample was selected.
If the sample is unlike the population, it is unlikely that the researcher will be able to generalize
the results of the experiment to that population.
Holistic: Different parts are interconnected to form a whole. Considering the whole entity when
examining or analyzing specific parts of the whole.
All encompassing; incorporating different variables into a study of something: Holistic
Holistic medicine often considers the entirety of the human body or human experience when
working to heal patients.
Chinese medicine often considers the holistic nature of the individual; it considers the entirety of
the individual in the healing process.
The holistic nature of the study enabled the researcher to obtain a more complete understanding
of the impact of the therapy.
Hypothesis: Proposition; educated prediction; assumption. Explanation for the occurrence of a
phenomena, behavior, event, etc.
Proposed explanation for a phenomena or event: Hypothesis
The researcher’s hypothesis, that the new therapy would be more effective than the regular
The statistical results found that the null hypothesis should be rejected in favor of the alternative
one.
128
Impressionistic: Subjective interpretation of phenomena or events. Engagement with the world
that may not closely adhere to objectivity, rather, this engagement encompasses opinion,
intuition, etc.
Someone’s personal, usually subjective, understanding or explanation of phenomena:
Impressionistic
The researcher’s impressionistic account of the subject’s behavior was criticized for a lack of
objectivity.
When a researcher becomes impressionistic and subjective while conducting an experiment, the
experiments credibility may be called into question.
Integrate: Incorporate; combine. Merge or combine data/information sources into one coherent
whole to better understand a phenomenon.
Merge different sources of information together to better explain a phenomenon: Integrate
It is often valuable to integrate quantitative and qualitative information to create a more complete
representation of a study.
When developing surveys, a researcher may want to integrate old survey questions and new
survey questions to maximize data collection.
Interpretive: A subjective understanding of phenomena or events. A non-objectivist
interpretation or explanation of something.
A subjective or personal understanding of phenomena: Interpretive
The researcher’s interpretive style of analyzing people’s behavior left him open to criticism from
his detractors, but this style strengthened the support of those who appreciate the unique
viewpoint from which he sees the world.
Experimental researchers should generally minimize interpretive analysis of phenomena, as it
may distort the results and leave their work open to criticism.
Interview: A formal transfer of information from a subject to a researcher. Meeting to exchange
information to expand our knowledge of the world.
A communicatory channel in which a subject will provide a researcher with information:
Interview
The interviews were a disorganized failure. The researcher regularly went off-topic, argued with
subjects, and randomly skipped necessary questions.
During the interview process, subjects may be unwilling to provide personal information to the
researcher.
129
Measure: Determine or judge the effectiveness, quantity, structure, or proportion of something,
usually quantitatively. Record the quantity or amount of something. Register the amount of
change as indicated by the dependent variable.
Quantitatively record the amount of something: Measure
One of the main requirements of a true experiment is having the capacity to measure the
dependent variable.
Due to the use of an outdated instrument, the researcher failed to properly measure the dependent
variable.
Multimethod: Numerous sources used to provide data or information. Information/data is
gathered and presented using multiple techniques or processes.
Collecting data and information from multiple sources: Multimethod
A multimethod approach was used—which included both quantitative and qualitative analysis—
to capture a more complete understanding of the phenomenon.
When the nature of the dependent variable is more abstract, it may be beneficial to use a
multimethod approach to capture the effects of the independent variable.
Nature: The true composition or qualities of something; the empirically derived makeup of the
world.
Empirically derived understanding of phenomena; expressed constitution or makeup of
something: Nature
The nature of the study is such that a high-level of competency in the field is required to conduct
the experiment and analyze the data.
It is not in a cat’s nature to obey the orders of humans.
Objective: Unbiased; lacking in subjectivity. Seeing or determining the universal nature of
something. Viewing the world from a dispassionate position.
Unbiased, dispassionate, and non-subjective: Objective
The objective of the experiment is to determine if a high-fat diet is healthier than a low-fat diet.
In quantitative research, it is usually imperative that experimenters be as objective as possible in
their conduct and in their interpretations of the results.
through scientific processes. Also, monitoring a person, social interaction, phenomena, event,
etc. and recording relevant information.
Using scientific processes to record and collect information: Observation
130
Numerous observations were made throughout the course of the experiment. This was to
measure the subject’s behavior under different circumstances and at different times.
The researcher’s subjective observations were criticized for being biased.
Paradigm: A framework for theory, thought, operation, or process. Guidelines to follow when
engaging the world (in this case, in scientific pursuits).
A guiding framework from which to operate: Paradigm
Paradigm shifts have been witnessed throughout the history of science. Wherein new theories are
developed that usurp old theories—often after great struggle among the advocates of the
different theoretical platforms.
A researcher often operates within a specific scientific paradigm, which provides a framework
for scientific conduct, scientific theory, and scientific experimentation.
Patterns: Recurring data or information. Repetition of information or action that works to
explain a phenomenon.
Predictable behaviors or results: Patterns
Much of qualitative research involves finding recurring patterns in the open-ended responses of
subjects.
The behavioral patterns of the mice were recorded to better understand the effects of the
treatment.
Precise: Exacting; accurate.
Accurate; without error: Precise
The researchers instrument was not precise; it failed to accurately record the correct
measurement.
The precise nature of the experiment was not clearly conveyed in the abstract.
Probing: Seeking, searching, questioning, or examining something.
Exploring, questioning, and looking for answers: Probing
The researchers probing questions made the subject feel uncomfortable.
The probing interview questions unearthed a large amount of new information.
Procedure: Formal framework for operations; standardized mechanism to control conduct.
Operational framework for conduct, usually in a step-by-step formation: Procedure
The procedure was incorrect. The researcher gave the different treatment groups the wrong
quantities of the treatment.
The scrupulous and studious researcher followed the experimental procedure exactly.
131
numbers to describe a phenomenon. Often used to describe peoples lived experiences.
Studies that analyze words rather than numbers or statistics: Qualitative
Qualitative studies often consist of ethnographies, case studies, and focus groups.
A qualitative study is an in-depth analysis of a small number of subjects.
Reflexive: Reflection. Reflecting on a subject or phenomenon while being conscious of one’s
own cultural biases and the lenses from which one views the world.
Considering a subject or phenomena while being aware of one’s own cultural bias and subjective
nature: Reflexive
The researcher should try to be as reflexive as possible when studying other ethnic groups to
minimize how her own cultural viewpoints alter her interpretations of phenomena or events.
One who is not reflexive when conducting research on other ethnic groups, may distort the
research with one’s own cultural conventions or preexisting assumptions.
Standard: A formal and commonly used framework from which variables can be compared or
analyzed. A recognized and generally implemented method or process.
Formal and legitimate: Standard
The final exam was standard across the department. All students were required to pass the exam,
or they would not be able to graduate.
It is standard procedure for scientists to replicate their findings.
used to make statements and conclusions about the relationships between variables. A form of
mathematics in which data is analyzed and exhibited.
A mathematical method used to distinguish relationships among variables: Statistics
The researcher used statistics to determine if the groups in the experiment were significantly
different from one another.
Once the post-test measurements were completed, the statistics revealed that there were no
significant differences among the experimental groups.
Structure: A procedural framework. Limits diversion from the core of a study. Arrangement of
operations.
A regulatory mechanism useful to ensure the effective functioning of operations: Structure
The architectural structure of the laboratory was unsuitable for experiments that require large
equipment or a great amount of space.
132
The structure of the research proposal was flawed in numerous ways. There was no mention of
subject selection, controlling extraneous variables, or ethical concerns.
Subjective: Reality or information potentially altered by one’s internal constitution, personal
views, or personal interpretations. A study may be biased due to the researcher’s personality,
personal experiences, etc.
An interpretation of reality founded on one’s personal understanding or one’s experience:
Subjective
The researcher’s subjective nature often led him to interpretations that did not align with his
colleagues.
Ethnographic studies are often more subjective in nature, as the researcher has greater license to
choose where attention is directed and in how events and phenomena are interpreted.
Survey: Method of information gathering and data collection—usually administered by
questioning subjects.
An information gathering technique that usually involves questioning and interviewing subjects:
Survey
The researcher wanted to survey everyone in the small town, but many people were excluded
from the study because they did not have a telephone or a cell-phone.
The researcher wanted to survey all adults who lived in the county. Subjects were randomly
selected from a telephone directory and interviewed over the telephone.
Regulated, procedural, and predetermined: Systematic
The researcher developed a systematic plan to test the subjects; she collected samples every third
day for a year.
True experiments are systematic, clearly defined, orderly, and unwaveringly sequential.
Triangulation: Viewing, analyzing, or examining phenomena using various methods. Looking
at or engaging phenomena from various angles.
Analyzing a phenomenon from different quantitative and/or qualitative perspectives:
Triangulation
The triangulation used within the study was effective at capturing a more whole and all-
encompassing view of the phenomena.
The use of written surveys, spoken interviews, and a qualitative analysis of behavior in a singular
study is an example of the use of triangulation to study a phenomenon.
Unstructured: Lacking a distinct framework or strict guidelines; potentially providing greater
opportunities for exploration and discovery.
133
Without formal guidelines or regulatory mechanisms: Unstructured
The unstructured nature of the research enabled the researcher to ask new questions as they arose
during the interview process.
Ethnographic and exploratory studies are often unstructured in nature, wherein a researcher will
enter an environment and work to understand what is happening with no clearly defined
direction.

Cause: Input; treatment; independent variable; intervention; stimulus. Produces a change or
reaction in someone or something. Stimulates a change in the dependent variable (effect).
Produces an “effect” in the dependent variable: Cause
The independent variable is the cause, and the dependent variable is the effect.
Cause and effect claims can only be made when true experiments are conducted.
Confounding: Something unwanted, unexpected, or inappropriate that disrupts the relationship
between variables and alters the results of the experiment. An element within an experiment (not
the independent variable) that influences the results of an experiment.
A variable that interferes with the experiment and potentially changes the results of the study:
Confounding
It’s important that the researcher control all confounding variables, lest greater variability, error,
and incorrect conclusions are produced.
A confounding variable may distort the relationship between the independent variable and the
dependent variable, misleading the researcher as to the true relationships within a study.
Constant: Without change; without variation; static.
Unchanged; stable; when the researcher wants to keep all variables unchanged except for the
independent variable: Constant
The researcher failed to hold all the extraneous variables constant, thus, producing unwanted
variability in the results.
In a true experiment, it’s important that the experimenter hold all variables unrelated to the
independent variable constant within the experimental environment; the independent variable is
to be manipulated, and, thus, not held in stasis.
Control: Supervised; managed; manipulated; governed; kept constant; free of variation;
contained. Reduction in unwanted change or unpredictability.
It is this “group” within the independent variable that remains unchanged: Control
134
The control group was no different from the treatment group when the dependent variable was
measured, leading the researcher to conclude that the new medicine was ineffective.
The control group is often considered the baseline group because it is the normal
Controlled: Supervised; managed; manipulated; governed; kept constant; free of variation;
contained. Reduction in unwanted change and unpredictability.
Researchers often want to keep these variables constant within their experimental design:
Controlled
The experimental environment was sufficiently controlled; it is unlikely that unwanted variables
disrupted the experiment.
The experiment was highly controlled; extraneous variables influenced all levels of the
independent variable equally.
Dependent (Variable): The effect; what is measured; outcome; output; result. Varies depending
on the nature of the treatment/independent variable.
The researcher measures this variable after manipulating the independent variable: Dependent
The researcher was unable to measure the dependent variable efficiently because of defects in the
measurement instrument.
The dependent variable is measured after the implementation of the independent variable.
Effect: Output; outcome; result; dependent variable. What is measured after the introduction of
the stimulus. Influenced by the causal element.
Another name for the outcome, output, or result: Effect
The study found that the new medicine had no effect on the patient’s symptoms.
True experiments can make “cause and effect” claims about a phenomenon.
Error: Distance from the true score; miscalculated variance from the actual score.
The distance between the obtained score and the true score: Error
Measurement error may produce great variability in the results and make it more difficult to find
differences between group means if differences do indeed exist.
Experimental error may be the result of imprecise or uncalibrated measurement instruments.
External: Exterior; outside of something. The environment outside of the immediate
experimental environment.
The environment that exists outside of the experiment: External
A small sample size often means that an experiments external validity will be low.
135
External validity is usually high when the sample adequately reflects the population.
Extraneous (Variables): Unnecessary; needless. Outside variables needlessly influencing the
research process.
Variables that are unneeded and unwanted within an experiment: Extraneous
Extraneous variables usually need to be controlled by the experimenter or they may create a high
degree of variability in the results of an experiment.
Extraneous variables, if not properly controlled, have the potential to produce a great amount of
unwanted variability or unwanted error within an experiment.
Input: Stimulus; cause; independent variable; intervention; treatment. Produces a change or
reaction in someone or something. Stimulates a change in the dependent variable.
Another name for the independent variable, treatment, or intervention: Input
The input is expected to produce changes in the output (dependent variable).
The input comes prior to, and often determines, the output.
Independent (Variable): Stimulus; cause; intervention; input; treatment. Produces a change or
The researcher changes or manipulates this variable: Independent
We failed to reject the null hypothesis. The independent variable produced no significant change
in the dependent variable.
Before the experiment the researcher will determine the makeup of the different levels of the
Intervention: Stimulus; cause; independent variable; input; treatment. Produces a change or
Another name for an independent variable, treatment, or input: Intervention
The intervention failed to stimulate hunger in mice.
The intervention caused a significant change in the subject’s behavior, leading the researcher to
conclude that the therapy is likely effective.
Manipulate: Change something; vary or change the quantity or quality of something.
The researcher does this with the independent variable, creating different levels for testing:
Manipulate
We manipulate the independent variable to create different groups (treatment, control, placebo,
etc.) for testing.
136
Without double-blind experimentation, the researcher may unconsciously manipulate the
opinions and feelings of the subjects.
Measure: Determine or judge the effectiveness, quantity, structure, or proportion of something,
usually quantitatively. Record the quantity or amount of something. Register the amount of
change as indicated by the dependent variable.
Quantitatively determine and record the amount, length, etc. of something: Measure
We measure the dependent variable to determine if there are differences between the groups.
In qualitative studies, researchers do not measure dependent variables.
Observe: Obtaining information and data through sensory experiences. Collecting data through
scientific processes. Also, monitoring a person, social interaction, phenomena, event, etc. and
recording relevant information.
To watch and record: Observe
The researchers wanted to observe the behavior of the subjects for several months, recording and
categorizing as much as possible.
Because of exhaustion, the tired researcher failed to observe subject behavior that was integral to
the study.
Outcome: Result; effect; output; what is measured after the introduction of the input/stimulus.
Another name for the result, effect, or output: Outcome
The outcome of the experiment was not as the researcher had hypothesized, there was no
correlation between the two variables.
The outcome of the experiment was not as the researchers had expected. The new medicine
failed to improve the subject’s condition, and the researchers were forced to start anew.
Output: Result; effect; outcome; what is measured after the introduction of the input/stimulus.
Another name for the result, effect, or outcome: Output
The different inputs produced great variability in output measurements.
The output is the variable that is measured in response to the input.
Placebo: A control variable used to deceive participants into thinking that they are receiving a
treatment. Used to measure the psychological and physiological effects of simply thinking or
believing that a certain medicine/treatment will improve one’s condition.
A fake treatment that may induce subjects into responding in a certain way: Placebo
The placebo produced little change in the subject’s behavior, lending greater evidence that is was
the medicinal properties of the medicine, and not the simple act of taking the medicine, that
induced behavioral change.
137
The placebo group was given sugar pills that were made to look like the actual treatment.
Randomization: Elements or subjects selected by chance. Unbiased selection, sampling, or
assignment process.
Selection by chance: Randomization
The randomization process was successful. The pre-test determined that there were no significant
differences between the groups prior to the implementation of the intervention.
The randomization process produced three groups that were statistically equal on the variable
under study.
Reliability: Consistency; consistency of results. A research process is conducted repeatedly, and
the same results are obtained again and again.
The researcher is repeatedly obtaining the same results time after time: Reliability
The reliability of the experiment was put into question when the researcher failed to find similar
results on subsequent tests.
Reliability is normally increased when an experiment is replicated, and the results are the same.
Spurious: Incorrectly positing that there is a certain statistical relationship (or any other kind of
relationship) between variables when there is no relationship. Making incorrect or tenuous
relational determinations between variables.
An incorrect or tenuous interpretation of the relationships among variables: Spurious
The researcher’s interpretation of the results was spurious, there was, in contradiction to the
researcher’s position, no evidence of a relationship between the variables under study.
The academic community considered the results of the experiment spurious and criticized them
at every turn.
Stimulus: Causes a change, reaction, or response in someone or something.
Initiates a response or reaction; also known as the treatment or intervention: Stimulus
Once the mice receive the stimulus, their responses are to be recorded electronically every two
seconds.
The stimulus failed to produce a significant response among the subjects.
Treatment: Stimulus; cause; independent variable; input; intervention. Can produce a change or
reaction in someone or something. May stimulate a change in the dependent variable.
Also known as the stimulus or intervention; often associated with medicine or therapy:
Treatment
The patients are to receive a new dose of the treatment once a week for ten weeks. They are then
to be evaluated after this ten-week period.
138
A true experiment, in its most basic form, will have at least one treatment group and one control
group.
Unchanged: Constant; without variation.
Steady, unvarying, and static: Unchanged
Despite repeated objections due to ethical concerns, the researchers mind remained unchanged.
The experiment would proceed as planned.
Despite greater experimental controls and a larger sample size, the results of the experiment
remained unchanged.

Blind: Deprive of information. Purposefully conceal information from participants and, at times,
experimenters for practical and legitimate reasons. The experimenters and participants are not
made aware of which group receives the treatment and which group is the control group.
Purposefully withholding information from subjects and experimenters for practical reasons:
Blind
The subjects were blind as to who received the treatment and who received the placebo.
It is often important that the experimenters are blind as to which subjects receive the actual
treatment and which subjects receive the placebo. If this is not achieved, the experimenters may
Carryover: The effects of one treatment extend to, influence, and interact with the effects of
subsequent treatments when subjects are receiving treatments sequentially. Treatments merge
together to muddy and confound their individual effects.
The effects of one treatment merge into the effects of subsequent treatments, distorting the
results of a study: Carryover
When giving subjects multiple treatments in a sequence, the experimenter delayed the onset of
new treatments to reduce the carryover effect.
In within subjects designs, it’s important to be aware that sequential treatments distributed to
subjects have the capacity to carryover into each other and distort the results of an experiment.
Compare: Equate; juxtapose. Analyze one thing in relation to another, noting their similarities
and differences.
Contrast; examine one thing in relation to another thing: Compare
In means testing, the researcher will compare group means to determine if there is a difference
among the groups.
139
In between subjects designs, a main objective is to compare group means to determine if there is
a significant difference among the groups.
Counterbalancing: The rearrangement of a sequence of treatments to counteract the carryover
effects of a sequential experimental arrangement. Randomization of the sequence of treatments
to detect carryover effects. Typically used for within subjects designs.
Randomizing the sequence of treatments given to a single group to identify and combat
carryover effects: Counterbalancing
The counterbalancing measures revealed no carryover influences among the variables.
Carryover effects can be counterbalanced through a randomization of the sequence of treatments.
Delay: Hold-up; stop something for a period. Expand the time between the introduction of new
treatments to reduce the carryover effect.
Increase the time between the onset of one treatment and subsequent treatments: Delay
When treatments are provided to subjects sequentially, it’s often valuable to delay the onset of
new treatments to minimize the carryover effect among treatments.
In human memory experiments, where subjects are provided treatments sequentially, it is often
the case that no matter how long there is a delay before subsequent treatments, a carryover of
some magnitude will occur.
Placebo: A control variable used to deceive participants into thinking that they are receiving a
treatment. Used to measure the psychological and physiological effects of simply thinking or
believing that a medicine/treatment will improve one’s condition.
A fake treatment that may produce psychological or physiological effects on subjects to gauge
the impact of subjects taking any kind of treatment: Placebo
Due to the double-blind nature of the experiment, the experimenter did not know which group
was given the placebo and which group was given the treatment. This was implemented to
reduce preferential treatment or unconscious bias.
Throughout history, a placebo has taken many forms—sugar pills, colored water, and bread pills
are a few examples.
Posttest: A test or observation administered after the onset of the treatment or independent
variable. Measurement of the output.
A test or measurement conducted after the treatment is applied: Posttest
The posttest was implemented after the treatment to determine if there were differences between
the experimental groups.
A group of subjects was given both a pretest and a posttest. Due to learning processes, it was
thought that the subjects may have improved on their final test because they took the pretest.
140
Because of the pretest, subjects may have become more comfortable with testing, more
accustomed to the questions, they may have been thinking of the questions after the pretest, etc.
This is a threat to the internal validity of an experiment known as “testing,” and it’s another
reason why a control group is necessary. With a control group we can assume that both groups
are experiencing this “testing” effect and, thus, cancel it out, leaving the treatment as (hopefully)
the only difference between the groups.
Pretest: Baseline. A test or observation administered before the onset of the treatment or
independent variable. Measurement before the input.
A test or measurement conducted before the treatment is applied: Pretest
A pretest was provided prior to the implementation of the treatment to determine if there were
differences between the experimental groups on the metric of study.
A pretest is often used to determine if the experimental groups are statistically equal on the
variable under study. Though, this test is not always necessary. If there is a large enough sample
size, and if the researchers employ random assignment (randomly assigning a sample of subjects
to the different experimental groups), it can normally be assumed that the groups are statistically
equal, and, thus, this type of testing is unnecessary.
Randomized: Elements or subjects selected by chance. Unbiased selection, sampling, or
assignment process.
Making it so that subjects or elements have a known, and usually equal, chance of being selected
for a sample: Randomized
Subject assignment was randomized to increase the chances that the groups were equal prior to
the onset of the treatment.
With a large enough sample size and a randomized process of subject assignment (to the
Repeat: Something occurs again and again.
Recurring; something happens over and over again: Repeat
To garner high reliability, the researcher should have the capacity to repeat an experiment and
achieve the same results.
For an experiment to be considered falsifiable, a researcher needs to provide all the necessary
information for others to repeat the experiment.
Sequence: Succession. A series of events that occur in a predetermined order. Things or events
arranged in an orderly fashion to accomplish some function.
Something takes place in a predetermined order: Sequence
141
The sequence in which the treatments were introduced to the subjects were randomized to
determine if there were any carryover effects within the experimental design.
If subjects are provided numerous treatments in a sequence, the order of the treatments may
influence the results of the experiment.
Treatment: Stimulus; cause; independent variable; input; intervention. Produces a change or
The intervention that may produce a change or reaction: Treatment
The treatment proved to be effective. There was a statistical difference between the new
medicine group and the placebo group.
The statistical analysis determined that the treatment group was no different from the control
group, leading the experimenter to conclude that the compound was ineffective as a cure.
Chapter 7: Internal Validity, External Validity, and Reliability

Accuracy: The precision or truth of the data or results.
The extent to which a measurement is without error: Accuracy
The accuracy of the measurement instrument was precise, there was no error or unwanted
variability.
The accuracy of the measurement instrument was negatively influenced by the researcher’s
inexperience.
Alternative: A different option or potentiality. Not the standard option. Choosing amongst two
or more things.
Another possible option, response, or result: Alternative
The alternative hypothesis was that the new medicine was significantly more effective than the
previous medicine.
Alternative explanations for the experimental results were limited due to the highly controlled
research environment.
Attrition: When subjects leave or drop-out of a study.
When certain subjects fail to remain within a study: Attrition
Subject attrition made generalizing the results to the population a challenge, because the sample,
as it was now altered due to subject drop-out, no longer resembled the population.
Due to the great amount of time in which subjects were expected to invest in the study, the
attrition rate was high.
142
Bias: Subjective orientation or inclination toward something or someone. Thinking about
something or someone in a way that is predicated on preconceived notions.
A predisposition to thought or theory that may influence the results of a study: Bias
The bias exhibited by the researcher was palpable. When the subjects responded positively, he
attributed it to the new therapy, but when they responded negatively, he always attributed it to
something else.
The double-blind experimental process is often an effective way to minimize researcher bias.
Causal: Produces a significant change or reaction in someone or something. Stimulates a
significant change in the dependent variable (effect).
When an independent variable produces a significant change in a dependent variable it
establishes added evidence that the relationship is: Causal
There was no causal relationship between the two variables. Differences in the independent
variable did not produce differences in the dependent variable.
It is by using true experiments that a researcher can potentially make causal statements about the
world.
Cause: Input; treatment; independent variable; intervention; stimulus. Produces a change or
reaction in someone or something. Stimulates a change in the dependent variable (effect).
The independent variable or intervention: Cause
A lack of funding would cause the experiment to be discontinued.
The cause is the independent variable, and the effect is the dependent variable.
Consistent: Repeatable; reproduceable; consistency of results; reliable. An experimental process
is conducted over and over, and the researcher obtains the same results.
Reliable; without waver, change, or variation: Consistent
Subsequent follow-up studies were consistent with the results of the initial study, lending support
for the validity of the study.
Subject responses were not consistent with the alternative hypothesis, leading the researcher to
retain the null hypothesis.
Diffusion: Spread. The unwanted spread of information from one experimental group to another.
The spread of information among experimental groups: Diffusion
Understanding the diffusion, or the spread of the scores, provides an indication of where an
individual subject’s score is in relation to a group.
Diffusion represents the spread of experimental information among subjects.
143
Effect: Output; outcome; result. What is measured after the introduction of the
treatment/stimulus. Influenced by the causal element.
The results when the dependent variable is measured: Effect
The effect of the treatment led the researcher to champion the potential benefits of the medicine.
The cause and effect relationship claimed by the experimenter was questioned due to the
existence of numerous potentially confounding variables in the study.
Estimate: Ascertain to the best of one’s knowledge. Formulate an approximation based on
evidence.
Approximate; compiling evidence to conclude something, with a level of uncertainty: Estimate
The doctor could only estimate the effectiveness of the medicine—as patients often responded
differently to it.
Much of experimentation (means testing) relies on estimates of group differences, as mean
scores are seldom without a certain degree of error.
External: Exterior; outside of something. The outside of the immediate experimental
environment.
Outside of something; outside of the experimental context: External
If the sample reflects the population, one can potentially claim high external validity.
Random selection, where subjects are randomly selected from a population, is a fundamental
component in the attainment of high external validity.
which the sample was drawn. Conclusions derived from samples in scientific research can speak
to the larger population.
To take the results of an experiment conducted with a sample, and infer those results back to the
population from which the sample was drawn: Generalize
Researchers focus on sampling processes to better generalize their results to the populations from
which the samples were drawn.
When the sample does not reflect the population, it makes it difficult for the researcher to
generalize the results of the study back to the population from which the sample was drawn.
Internal: Inside; processes within a study.
Inside of something; inside of the experimental context or experimental processes: Internal
When the independent variable has a direct influence on the dependent variable, and other
potentially confounding variables have been ruled out, the researcher may claim high internal
validity.
144
Adding a control group to experimentation (means testing) is usually implemented to increase
internal validity.
Link: Connection. Relationship or association between variables.
Relationship or connection between variables: Link
The link between the independent variable and the dependent variable was challenged by the
researcher’s colleagues because of the large number of potential confounding variables.
The results from the meta-analysis determined that there was a weak link between the medicine
and a reduction in the disease.
Maturation: Natural human growth and developmental processes.
Inherent and organic developmental processes that naturally occur over time: Maturation
Concerns that subject maturation may have influenced the results arose when it was revealed that
the study lasted an additional ten months.
With no control group, and a long experimental time-frame, the natural development of the
subjects, or maturation, may become a realistic threat to internal validity.
Mortality: It commonly means death or rate of death, but within experimentation it means the
rate of attrition. When subjects leave or drop-out of a study.
When subjects exit a study prematurely: Mortality
Subject mortality may create differences between groups that can reduce internal validity.
Mortality has the potential to reduce external validity. If a sizable percentage of a sample is lost
due to systematic subject drop-out, the remaining sample may no longer resemble the original
sample, thus, no longer reflecting the population.
Novelty: New; previously unencountered. The administration of a completely new stimulus may
reduce the validity of the experimental results (e.g. the newness of the stimulus, not the stimulus’
effectiveness, may cause a change in the dependent variable).
Something that is new: Novelty
The novelty of a new stimulus may induce responses in subjects that may quickly fade after
acclimation.
The novelty of having a new teacher in the class initially caught the student’s attention and the
students studied diligently, but after this effect diminished the students returned to their normal
study habits.
Population: The group of people or things you want to study. The people or things the
researcher wants to generalize or infer back to with the results of the study.
145
All the subjects or elements that you want to study: Population
When a population is very large it is often beneficial to use cluster sampling.
For a study to have high external validity, the sample should meet size standards and reflect the
population.
Reactivity: Reactions or responses to a stimulus.
Responses to a treatment, intervention, or stimulus: Reactivity
Subject reactivity to the medicine was nonexistent due to the low dosages provided.
Subject reactivity to the medicine was so strong that the researchers were forced to halt the study
for fear that the subjects may be harmed.
Regression (to the mean): Extremely high and extremely low scores will often migrate closer to
the mean on successive tests.
The tendency for the more extreme scores to become less extreme on subsequent testing
regardless of applied treatments: Regression
Extreme scores on the pre-test naturally regressed closer to the mean on the post-test, thus,
distorting the experimenter’s results and calling into question the effectiveness of the treatment.
The extreme scores did not regress to the mean, leading the experimenter to conclude that the
subject’s behavior was stable over time.
Repeatable: Consistent; consistency of results; reliable; reproduceable. An experiment is
conducted over-and-over and the researcher is obtaining the same results.
Finding the same experimental results on subsequent trials: Repeatable
The results of one’s experiment should be repeatable, or the study will not possess robust
reliability.
The operational definitions and operational processes of an experiment should be clearly
conveyed so that the study is repeatable, and, thus, falsifiable.
Reproduceable: Repeatable; consistent; consistency of results; reliable. An experiment is
Obtaining consistent results upon repeated tests: Reproduceable
The study was reproduceable, the experimenters yielded the same results on repeated tests.
Were the experimental results reproduceable or were they not? If not, if the results were different
on subsequent tests, it is quite difficult for the study to withstand criticism from the academic
community as it lacks reliability.
Sampling: A collection of people or things that represent a population and is used for
information gathering and experimental purposes. Populations are usually too large to be used
146
for experimentation, thus, samples are drawn to represent populations; researchers then use these
samples for experimentation.
Generating a group of subjects that are to represent a population for experimental purposes:
Sampling
The sampling process was disorganized and chaotic. The researchers initially used random
sampling but grew tired and resorted to convenience sampling.
The sampling process lacked randomization, thus greatly reducing external validity.
Subpopulations: Dividing a population into groups according to some criteria.
Distinct groups within a population: Subpopulations
The researcher divided the population into subpopulations so that different groups could be
better represented during the sampling process.
The population was broken into subpopulations and sampled using stratified sampling processes.
This was conducted so that there was a greater representation of different subjects within the
sample.
Chapter 8: Sampling Methods

Bias: Subjective orientation or inclination toward something or someone. Thinking about
something or someone in a way that is predicated on preconceived notions.
Viewing or analyzing something from a personal or predetermined position; allowing one’s
personal opinions or ideas to influence one’s research: Bias
The experimenter showed no bias in his interpretation of the results. He looked at the data and
came to an objective and dispassionate conclusion.
Researchers may be particularly susceptible to experimental bias when they have invested a great
deal of time, resources, and personal reputation in their theories, experiments, etc.
Chance: Luck; randomly selected.
Selection without bias or subjective influence: Chance
We want to minimize the influence of chance within our study. We can do this by controlling the
research environment, using more precise and accurate measurement instruments, clearly
defining the research process, etc.
When the research environment is uncontrolled, the researchers leave the study open to chance
playing a greater role in the results.
Cluster: Bunches; collections of people or things. With large populations, it is often valuable to
first sample from clusters of subjects to draw a sample in a more manageable way.
147
The grouping or bunching of subjects for sampling purposes: Cluster
Cluster sampling is an effective way to break down and randomize the selection of subjects
within very large populations.
A large and unwieldy population can be broken into clusters so that sampling is more effectively
managed.
Convenience: Ease of use or operation. Comfort. Selecting subjects or groups in a way that is
more streamlined, less difficult, and less of a hassle.
A reduction in difficulty or hassle: Convenience
He gathered his sample based on convenience. He approached people on the street and asked
The researcher surveyed the people in her neighborhood for purposes of convenience.
which the sample was drawn. Conclusions derived from samples in scientific research may speak
to a larger population.
Infer; extrapolate; to take the results of an experiment conducted with a sample and use that
information to speak to the population from which the sample was drawn: Generalize
When the sample reflects the population, we can generalize the results of the study back to the
population from which the sample was drawn.
When dealing with human subjects, natural experiments (experiments taking place in natural
environments) may be easier to generalize back to the population when compared to laboratory-
based experimentation.
Generate: Produce; create.
Make; produce; create: Generate
Engaging in exploratory research is often a good way to generate ideas for experimental
research.
Perusing academic literature is a good way to generate research ideas.
Haphazard: Unstructured; unsystematic. Subjects are chosen at a whim, nonrandomly, or in a
way that is driven by an agenda.
Without complete or effective predesign, precision, or structure: Haphazard
Convenience sampling is often nonsystematic and haphazard, wherein subjects are selected
because they are easy to contact, the process is relatively quick, etc.
The students haphazard experimental design was rejected by his advisor. The design was poorly
constructed, the terms not clearly defined, and the research too costly and time consuming.
148
Infer: Generalize; extrapolate. Take results, conclusions, or data derived from experimentation
with a sample and use it to speak for (infer) the population from which the sample was drawn.
Generalize; to take a singular finding derived from a sample, and make observations about a
larger population: Infer
Random selection enables the researcher to take the results of a study and infer them to the
population from which the sample was obtained.
Making inferences from samples to populations is the reason why subjects are randomly selected
from populations.
Operation: Procedure; controlled process. A framework for action or behavior.
Process to carry out some purpose; mode or design for action: Operation
The operation was successful. The doctor effectively replaced the patient’s kidney with a healthy
kidney.
When an experimental operation is not clearly defined, subjects scores may exhibit greater
variability, and the study may become less reliable.
Population: The group of people or things the researcher wants to study. The people or things a
researcher wants to generalize or infer back to with the results of the study.
The group of people or elements from which the sample is drawn: Population
The population is everyone or everything the researcher wants to study.
The population should be clearly defined before the sampling process begins.
Probability: Likelihood. The likelihood or level of expectation that something will occur.
The likelihood that someone or something is selected for the sample: Probability
The probability that the sample represents the population is greatly increased when random
The p-value, or probability value, is the likelihood that the results were due to chance.
Quota: Required or expected number or portion. Selecting subjects based on predetermined
criteria; each predetermined criterion has a set number of subjects (or quota) that is to be attained
to better represent a population or a characteristic within a population.
Selecting subjects, either randomly or haphazardly, who fit some predetermined dimension:
Quota
When researchers meet their quota on a group, they no longer need to sample for that group and
can focus their sampling on other groups.
The quota of 120 subjects from the 50-60 age group was not reached, reducing that age group’s
representativeness within the study.
149
Random: Elements or subjects selected by chance. Unbiased selection, sampling, or assignment
process.
Selection by chance; selection without bias: Random
Utilizing random selection when selecting subjects from a population will usually increase
external validity (increase the ability for the sample to represent the population).
Random selection was used so that the sample adequately represented the population.
Random assignment is used so that the different experimental groups can be assumed to be
statistically equal before the independent variable is inserted into the experimental process.
Represent: Speak for someone or something. Take the place of someone or something. A
sample can be thought of as something that portrays, or represents, a population.
One thing or one person standing in the place for something or someone else, to speak for or
symbolize that something or someone: Represent
When the sample does not represent the population, the study likely has low external validity.
To claim that a study has high external validity, the sample should represent the population from
which it was drawn.
Sample: A collection of people or things that are expected to represent a population and are used
for information gathering and experimental purposes. Populations are usually too large to be
used for experimentation, thus, samples are drawn to represent populations. Researchers then use
these samples for experimentation.
The group of subjects that are selected from a population that are then used within an experiment
or survey as representatives of the population: Sample
A sample will have low external validity when subjects are not randomly selected from the
population.
Populations are often too large to be fully included into the actual experimentation. Therefore, a
sample is extracted from the population to represent the population in experimentation.
Snowball: The collection of something in a way that the collected element compounds on itself
in greater numbers over time.
People accumulated into a sample by having subjects introduce people (people whom the
researcher is interested in studying) to the researcher: Snowball
The experimenters snowball sampling method failed because the first and only subject did not
know any other people who could be introduced to the experimenter.
Snowball sampling is often an effective way to recruit new and difficult to find subjects through
accumulative processes.
150
Stratified: Dividing subjects from a population into groups based on some criteria to generate a
sample that better represents a population.
Dividing a population into groups based on some dimension to better ensure group
representativeness during the sampling process: Stratified
To achieve sample representativeness, it’s often valuable to implement stratified sampling so that
underrepresented groups within a sample have a better chance to be represented. This process
will break the population into different subpopulations, and these subpopulations will be sampled
based upon a specified quota.
Stratified sampling separates a population into different subpopulations, and these
subpopulations are sampled based upon a specified quota.
Survey: Method of information gathering and data collection—usually administered by
questioning subjects.
Collecting information from subjects through interviews and questionnaires: Survey
A survey was distributed to members of the class. This convenience sample was helpful in
obtaining a quick and cost-effective view of the opinions of the students.
To add an element of randomization to the sampling process and reduce experimental bias, a
survey was offered to every fifth person who walked past the researchers.
Operations that are conducted in a purposeful and orderly fashion: Systematic
His study lacked systematic rigor; the experimental process was not clearly defined, conducted in
a haphazard fashion, and not sufficiently controlled.
Without a systematic approach to the experimental process, details may be overlooked,
operations may become disorganized, and reliability may become diminished.
Voluntary: Freely chosen; behavior or action without coercion.
Offered on one’s own volition; decided independently and without coercion: Voluntary
Subject participation within an experiment needs to be voluntary.
A researcher needs to provide a good case for involuntary subject participation in a study, or the
researcher may be committing ethical violations.
Chapter 9: True Experiments and Quasi-Experiments

Assignment: Designation. Placing subjects or elements into specific experimental groups (e.g.
using random assignment to determine which subjects go into the treatment group or the control
group).
151
Coordinating subjects or elements into different groups: Assignment
Random assignment, randomly designating subjects to different experimental groups, is a
Without random assignment, wherein subjects have an equal chance of being selected for each
experimental group, one cannot claim that one is conducting a true experiment.
Causation: Something produces a significant change or reaction in someone or something.
Stimulating a significant change in a dependent variable (effect).
When the independent variable produces a significant change in the dependent variable, it
provides greater evidence of this: Causation
When seeking to claim experimental causation, we should conduct a true experiment.
Experimental causation cannot be claimed unless there is random assignment, a way to measure
the dependent variable, a controlled experimental environment, and the capacity to manipulate
the independent variable.
Extraneous (Variables): Unnecessary; needless. Outside variables needlessly influencing the
research process.
Unwelcome and unwanted variables that operate outside of the experimental design that may
influence the results: Extraneous
The researcher failed to control numerous extraneous variables, likely causing the post-test
Extraneous variables have the potential to alter the relationship between an independent variable
and a dependent variable, reduce internal validity, and change experimental results.
Implied: Conveying information without directly stating it.
Transmitting a message without directly stating or saying it: Implied
Without directly stating it, the researcher implied that the results may have been the product of
experimental mismanagement.
Using facial gestures, the researcher implied that the data may not have come from a reliable
source.
Manipulate: Change something (change the quantity or quality of something). Vary or control
the quantity or quality of something.
Having the capacity to change or alter something to test its ability to influence the dependent
variable: Manipulate
The treatment is manipulated to create different groups within the independent variable for
testing purposes.
152
The dependent variable is not directly manipulated by the researcher, it responds to the
Nonequivalent: Not equal; not the same; differing in some aspect.
Unequal; the experimental groups are different in some way prior to the experiment:
Nonequivalent
When experimental groups are nonequivalent, when there is no random assignment or when
there is random assignment with a small sample size, subjects can be given pre-tests to determine
equivalency on certain metrics.
Because the groups were nonequivalent—the groups were not statistically equal—a “cause and
effect” relationship could not be established.
Outcome: Result; effect; output; what is measured after the introduction of the input/stimulus.
The results of an experiment or study: Outcome
The outcome of the experiment was not clearly conveyed in the final write-up of the study,
producing a great deal of skepticism among the readers.
In much of experimentation, the outcome of the experiment cannot be determined until the
dependent variables are measured and the statistical tests are conducted.
Pre-existing: Something is present prior to the implementation of the treatment/independent
variable. Something exists before the onset of the stimulus/input.
Something that exists prior to the onset of something else: Pre-existing
Pre-existing differences among subjects can often be minimized through random assignment and
by using a large sample size.
Without the random assignment of subjects to the different experimental groups, any pre-existing
differences among the experimental groups may alter the results of the experiment.
Quasi: Something has nearly the same makeup or constitution as another thing. Nearly the same.
Almost identical.
When one thing is nearly the same, though distinctly different in some way, compared to another
thing in makeup or operation: Quasi
When researchers are unable to randomly assign subjects to groups, usually due to ethical
reasons, the experiment becomes a quasi-experiment.
Quasi-experimentation usually reflects true-experimentation in its general framework; the
difference being that the former lacks one or more fundamental elements of the latter.
Random: Elements or subjects selected by chance. Unbiased selection, sampling, or assignment
process.
153
Selection determined by chance: Random
When researchers are unable to assign subjects to groups in a random fashion, they cannot claim
that they are conducting a true experiment; instead, they are likely conducting a quasi-
experiment.
Without random selection, the sample may not reflect the population, external validity may be
reduced, and the experimental results may not be generalizable.
Replicate: Consistent; consistency of results; reliable; reproduceable. An experiment is
Reproducing the same experiment one or more times: Replicate
To claim that a study has high reliability, researchers ought to be able to replicate their results on
subsequent tests.
Falsifiability is lost when other researchers are unable to replicate an experiment.
Representative: Speak for someone or something. Take the place of someone or something. A
sample can be thought of as something that portrays, or represents, a population.
One person, group, or entity takes the place or speaks for another person, group, or entity:
Representative
To claim that the effects of a study will be reflected in a population, the sample needs to be
representative of the population.
When the sample is not representative of the population, the study will usually have low external
validity.
Chapter 10: Meta-Analysis

Criteria: Guidelines; rules. A model that explains what the researcher will do in a given
situation.
Rules for operations, procedures, or subject selection: Criteria
The selection criteria were so limiting that too few studies were available for the meta-analysis.
The criteria to become a member of the research team is rather strict. One should possess at least
a master’s degree and hold publications in the field.
Dependent (Variable): The effect; what is measured; outcome; output; result. Varies depending
on the nature of the treatment/independent variable.
The variable that is measured after the onset of the treatment: Dependent
The dependent variable was improperly measured. The measurement instrument was used over a
long period of time and it became more variable and less precise.
154
Without the capacity to properly measure the dependent variable, it cannot be claimed that the
study is a true-experiment.
Effect Size: The effect size is the magnitude of the difference between treatment groups (e.g. the
difference between the treatment group and the control group). The effect size can also explain
the strength of the relationship among variables.
The magnitude of the difference between treatment groups: Effect Size
The effect size of the experiment was low, indicating that the magnitude of the difference
When the effect size is large (in means testing), the magnitude of the difference between at least
some of the experimental groups is likely great.
Estimate: Approximate. Gauge the nature of a figure/score based on the available
information/data.
Approximating a score or figure: Estimate
When researchers select subjects from a population, split subjects into groups, apply treatments
and controls, and compare the mean scores of the groups, the results will be estimates of the true
scores—experimental error will likely distance these estimates from the true score in some way.
Many experimental results are estimates of the true scores, as there is often a certain amount of
experimental error inherent within testing.
Objective: Unbiased; lacking in subjectivity. Determining the universal nature or something.
Viewing the world from a dispassionate position.
Dispassionate; not allowing one’s personal opinions or motivations to influence one’s
interpretations or decision making: Objective
It’s imperative that researchers remain as objective as possible when involved in experimental
endeavors.
The objective of the study was to determine if there was a causal relationship between the
independent variable and the dependent variable.
useful for making statements and conclusions about the relationships between variables. A form
of mathematics in which data is analyzed and exhibited.
A form of mathematics used to identify relationships among variables: Statistics
The statistics were conducted improperly. The data was entered haphazardly, the columns were
incorrectly labeled, and the wrong tests were run through the software.
After the experimental data was collected, statistical tests were conducted to determine if there
were significant differences among the groups.
155
Synthesize: Combine. The merging of elements to produce something different.
When elements are combined or merged to present a better understanding of something, increase
the effectiveness of something, etc.: Synthesize
It is often valuable to synthesize quantitative research and qualitative research into a singular
study to provide a more holistic and all-encompassing view of a phenomenon.
A meta-analysis is often an effective way to synthesize many closely related studies to better
understand a phenomenon.
Treatment: Stimulus; cause; independent variable; input; intervention. Produces a change or
Cause; independent variable: Treatment
The new treatment was not effective. There was not a statistically significant difference between
the groups in the independent variable.
Half of the subjects were randomly assigned to the treatment group and half of the subjects were
randomly assigned to the control group. Those subjects placed into the intervention group
displayed marked improvement in their recovery time.
Chapter 11: Ethical Concerns
Anonymous: Without identifying information; nameless; being unable to connect the
participants with their personal information; nameless or unknown participants.
Nameless; keeping the participants names and other personal information unknown to the public
and other researchers whose knowledge of the information is unnecessary: Anonymous
When participants remain anonymous, other participants will be more likely to join experiments
in the future.
When a participant’s personal information is revealed to the public the participant is no longer
anonymous. From this, the participant may suffer negative psychological consequences.
Attribute: Assign; ascribe; designate; recognition; impute.
Giving the author or creator recognition for their work; designation: Attribute
The work of the author should be attributed to her, and not to someone else who uses her work
for personal benefit.
Attributing the correct authors for their work in group lead experimentations is vital in
determining who should receive recognition in academic publications.
Breach: Offend; turn against; mistreat; transgress; insult; infringe.
To transgress a previous agreement; to turn against an accepted contract or promise: Breach
156
The professor’s failure to attribute the research conducted to the graduate student was a breach of
trust between advisor and advisee.
A breach of proper ethical behavior occurred when the subject’s anonymity was lost.
Cite: Citation; credit; recognition; reference; refer; mention; acknowledgment; attribute. To
provide recognition or credit to the author or creator of a piece of intellectual property, idea,
work of literature, academic article, etc.
To give credit or acknowledgment of the work others if it exists within one’s own research: Cite
Remember to cite all the material and ideas that belong to others that were included in your
research.
When the researcher failed to cite the original author’s ideas within his work, the academic
community lost trust in his ethical conduct.
Coercion: Compulsion; pressure; force. To exert pressure onto someone to get them to
participate in research or experimentation of some kind.
Pressuring someone into doing something in which they normally would not do: Coercion
The instructor coerced his students into engaging in his experimentation by explaining that if
they did not participate they would fail his class.
The researcher coerced her subjects into continuing to participate in the study even though they
had objected to further testing. She explained to the subjects that if they failed to continue they
wouldn’t receive payment for their participation and that they wouldn’t be allowed to participate
in any other experiments.
Cognition: Mind; mental; thought; processing information, learning, thinking, and other
operations within the brain.
Pertaining to the mind and mental functioning: Cognition
His cognitive functioning was impaired by the new medication.
Lesions in the rat’s brain produced a significant minimization of the rat’s brain functioning and
cognition.
Confidential: Private; between the researcher and the subject; not to be revealed to the public or
others; in confidence.
Keeping personal information private between researchers and participants: Confidential
The researcher failed to keep the participants personal information confidential, causing
psychological harm to the participants.
From survey questions taken on the street to medical research in the laboratory, confidentiality is
important anytime people engage in research.
157
Consent: Acceptance; agreement; allowing something to happen; knowingly complying with
some operation or process.
Agreeing to join and participate in an experiment or research of some kind: Consent
The subjects did not consent to be a part of the research, thus making the researchers conduct
unethical.
The participants agreed to join the research by signing the researchers consent form.
Contract: Agreement; legally binding arrangement; declaration of intention; an expression of
future behavior or action.
A legally binding document that expresses the intentions and expectations of people: Contract
The contract did not stipulate where the researcher would store the participants personal data.
The lead researcher consulted with the legal department to help her draw up a contract that was
to be offered to the future research assistants.
Debrief: Explaining to the participants the actual reasons for the research and the actual research
process.
Explaining the true nature of the research to the subjects after the research is complete: Debrief
When their research is finished, researchers will often debrief participants by asking them
questions concerning their knowledge of the experiment.
When possible, it’s important to debrief subjects so that they understand the true reasons for their
participation in the research.
Deception: Deceit; misrepresentation; trick; ruse; the researcher hides certain aspects of the
research to produce a more natural response from participants.
The employment of deceit to obtain a more real or natural reaction from participants: Deception
A researcher will often use deception to obtain a more natural response from participants.
A researcher needs to be careful when using deception within research, because there is a
possibility of encountering ethical problems.
Disclose: Reveal; bring to light; divulge, make known; uncover; release information.
Revealing or releasing information: Disclose
When, after the research is finished, the experimenter fails to disclose the real purpose of the
research to the deceived participants, the participants may develop negative feelings toward the
research and toward the experimenter.
The researchers disclosed to the subjects the true nature of the research.
Ethics: Right or wrong; good or bad; morals; duty; honor; moral philosophy.
158
How people should behave and conduct themselves under certain circumstances: Ethics
The ethical behavior exhibited by the researcher was impeccable; she had the participants sign a
consent form, treated them with respect and dignity, and fully disclosed the true nature of the
research once the experiments were completed.
Ethical considerations should be central to experimentation, namely, the protection and well-
being of the subjects involved.
Humane: Compassionate; decent; benevolent; moral; uplifting; kind; good-natured.
Treating others with dignity and compassion: Humane
The most important focus of any researcher should be the humane treatment of the participants.
The default position of any research design should be to treat the participants in a humane
manner.
Incentive: Inducement; stimulus; encouragement; motivational element.
A stimulus that motivates people to engage in a behavior or action: Incentive
Researchers will often provide incentives, such as money, to attract people to participate in their
experiments.
It would be unethical for researchers to provide too strong of an incentive to attract participants,
such as an overabundance of cash, because people may find it difficult to refuse involvement in
experimentation in the face of large rewards.
Incur: Undergo; obtain; receive; experience.
To experience or receive something: Incur
The experimenters should ensure that the participants do not incur any unnecessary harm.
Subjects may incur a large amount of psychological harm from unethical research.
Misrepresentation: Deception; falsify; fake; to misstate or distort something to trick someone.
The researcher tells participants that the reality of something is one way, while knowing that this
is information is untrue: Misrepresentation
The researcher’s misrepresentation of the true nature of the experiment deceived the participants.
The experimenter misrepresented his research data, thus incurring a great reprimand from his
colleagues.
Voluntary: Freely chosen; behavior or action without coercion.
Offered on one’s own volition; deciding to participate in something independently and without
coercion: Voluntary
Subject participation within an experiment should be voluntary.
159
A researcher needs to provide a good case for involuntary subject participation in a study, or the
researcher may be committing ethical violations.
160
Crossword Puzzle Answers
161
162
Chapter 3: Exploratory Research, Descriptive Research, and Causal Research
163
164
165
166
167
168
Chapter 9: True Experiments and Quasi Experiments
169
170
171
Fill in the Blanks Answers:
1. Once all the data has been collected, statistical analysis are conducted to determine if there
are differences between the groups.
2. A person who has little experience with statistical software will need to consult guidebooks
to help them along.
3. A true experiment should contain four elements: the random assignment of subjects to
different experimental groups, more than one level of the independent variable (more than
one experimental group), the capacity to control the research environment, and the ability to
measure the dependent variable.
4. Cause and effect statements about phenomena usually require empirical evidence.
5. The hypothesis was correct. The researcher’s ability to predict and forecast experimental
results was quite impressive.
6. A mixed-methods approach combines both quantitative and qualitative methods. It’s research
that combines statistics with a word or dialog centric approach to understand a phenomenon
in a deeper way.
7. The researcher’s quasi-experiment was valuable in displaying the effectiveness of the therapy
despite the lack of random assignment.
8. Empirical information is gathered from the senses.
9. Statistical procedures are employed after the data is collected to determine if there are
differences among the groups, or, in other cases, to determine relationships among the
variables.
10. An online survey is often an effective way to quickly and conveniently gather opinion-based
data. Though, external validity is often low because the responses may not fully capture the
opinions of the population as a whole.
11. The researchers experimental design was systematic in its operational framework; the
1. When something is not the default or normal option, it is considered the alternative option.
2. Corroborate your evidence with other evidence to support your claims about the world.
3. When something is moving in a circular manner, it can be said to have a cyclical nature.
4. Scientists usually generate empirical evidence to test theories.
5. Prior to the experiment, the researcher made numerous predictions. Her main hypothesis was
that the new medicine would be effective in reducing blood pressure.
6. The lead researcher’s inquiry into the loss of several team members yielded no information.
7. Using the scientific method is a legitimate way to investigate and test phenomena.
172
8. The null hypothesis indicates that there is no difference between the new medicine group and
the control group.
9. A peer can provide important feedback and suggestions concerning one’s research plans.
10. The phenomena that you wish to examine has already been thoroughly studied. In what way
will you be adding to our understanding of it?
11. The ethical principles diagramed in the “conduct manual” were closely followed by the
research team to minimize any future criticisms.
12. After an academic paper passes the peer-review process it is ready for publication.
13. An experiment is falsified through the replication process.
14. Without statistically significant results, one cannot claim that a new medicine is effective.
15. There was too much variation in your research environment—strange people entering and
leaving, the random incorporation of new measurement instruments, changing room
temperatures, etc.
Chapter 3: Exploratory, Descriptive, and Causal Research
1. The treatment groups average or mean was compared to the control groups mean at the end
of the study to determine the effectiveness of the treatment.
2. The researcher didn’t categorize subjects according to their age or any other dimension
involving time.
3. It is only through the implementation of true experiments that one can make causal claims.
4. The characteristics that makeup a population may influence how subjects respond to survey
questions.
5. A highly controlled research environment and well-defined research procedures will often
reduce the influence of confounding variables.
6. The control group is usually that which does not receive any treatment or any new stimuli.
7. Correlation does not equate to causation. Just because there is a strong relationship between
two variables does not mean that one is secure in the knowledge of the directionality of the
variables, nor is one secure that there wasn’t a third variable influencing the results.
8. The mathematician would regularly formulate difficult mathematical equations.
9. Ethnographers will often repeatedly interview subjects over the course of their research to
garner a deeper understanding of the subjects lived experiences.
10. The laboratory was in pristine condition, with clean floors, disinfected countertops, and
spotless instruments.
11. A meta-analysis is often effective in determining what the academic literature in its entirety
says about something.
12. The preliminary tests showed that the medicine was effective, but further, and more rigorous
13. The qualitative information was gathered through face-to-face interviews, wherein subjects
were asked questions about their experiences.
14. The independent variable is manipulated by the experimenter.
173
1. The researcher would count the number of times the mice turned in the wrong direction.
2. The data was run through statistical processes to determine if there were differences between
the groups.
3. The student’s thesis was a descriptive account of his time spent living in the rainforest.
4. His position was derived from a dispassionate review of the literature and an analysis of the
5. If the sample is unlike the population, it is doubtful that the researcher will be able to
generalize the results of the experiment to that population.
6. Chinese medicine often considers the holistic nature of the individual; it considers the
entirety of the individual in the healing process.
7. The researcher’s hypothesis, that the new therapy would be more effective than the regular
8. When developing surveys, a researcher may want to integrate old survey questions and new
survey questions to maximize data collection.
9. The interviews were a disorganized failure. The researcher regularly went off-topic, argued
with subjects, and randomly skipped necessary questions.
10. One of the main requirements of a true experiment is having the capacity to measure the
dependent variable.
11. Paradigm shifts have been witnessed throughout the history of science; wherein new theories
are developed that usurp old theories—often after great struggle among the advocates of the
theoretical platforms.
12. Much of qualitative research involves finding recurring patterns in the open-ended responses
of subjects.
13. The researchers instrument was not precise; it failed to accurately record the correct
measurement.
14. Qualitative studies often consist of ethnographies, case studies, and focus groups.
15. Ethnographic and exploratory studies are often unstructured in nature, wherein a researcher
will enter an environment and work to understand what is happening with no clearly defined
direction.
1. The independent variable is the cause, and the dependent variable is the effect.
2. When a serious confounding variable exists within an experiment, differences among groups
may not be attributable to the independent variable.
3. In a true experiment, it’s important that the experimenter hold all variables unrelated to the
independent variable constant within the experimental environment.
174
4. The control group is often considered the baseline group because it is the normal
5. The dependent variable is measured after the implementation of the independent variable.
6. True experiments can make “cause and effect” claims about a phenomenon.
7. Measurement error may produce great variability in the results and make it difficult to find
differences between group means if differences do indeed exist.
8. External validity is usually high when a sample is large and adequately reflects the
population.
9. The intervention caused a significant change in the subject’s behavior, leading the researcher
to conclude that the therapy is likely effective.
10. We manipulate the independent variable to create different groups (e.g., treatment and
control) for testing.
11. We measure the dependent variable to determine if there are differences between the groups.
12. The placebo group produced little change in the subject’s behavior, lending greater evidence
that it was the medicinal properties of the medicine, and not the simple act of taking the
medicine, that induced behavioral change.
13. The randomization process was successful. The pre-test determined that there were no
significant differences between the groups prior to the implementation of the intervention.
14. The reliability of the experiment was put into question when the researcher failed to find
similar results on subsequent tests.
1. It is often important that the experimenters are blind as to which subjects receive the actual
treatment and which subjects receive the placebo. This is so the experimenters do not
2. When giving subjects multiple treatments in a sequence, the experimenter may delay the
onset of new treatments to reduce the carryover effect.
3. In between subjects designs, the main objective is to compare group means in the dependent
variable to determine if there is a significant difference among the groups.
4. Carryover effects can be located and counterbalanced through a randomization of the
sequence of treatments.
5. When treatments are provided to subjects sequentially, it’s often valuable to delay the onset
of new treatments to minimize the carryover effect.
6. Throughout history, a placebo has taken many forms—sugar pills, colored water, and bread
pills are a few examples.
7. One group of subjects were given both a pretest and a posttest. Due to learning processes, it
was thought that subjects may have improved on their final test as a result of taking the
pretest. Because of the pretest, subjects may have become more comfortable with testing,
more accustomed to the questions, they may have been thinking of the questions after the
pretest, etc. This is a threat to the internal validity of an experiment known as “testing,” and
it’s another reason why a control group is necessary. When adding a control group, it can be
175
assumed that both groups are experiencing this “testing” effect and, thus, cancel it out,
leaving the treatment as (hopefully) the only difference between the groups.
8. A pretest is often used to determine if the experimental groups are statistically equal on the
variable under study prior to the implementation of the treatment. Though, this test is not
always necessary. If there is a large enough sample size, and if the researchers employ
random assignment (randomly assigning a sample of subjects to the different experimental
groups), it can usually be assumed that the groups are statistically equal, and, thus, this type
of testing (pretest) is unnecessary.
9. With a large enough sample size and a randomized process of subject assignment (to the
10. To garner high reliability, the researcher should have the capacity to repeat the experiment
and achieve the same results.
1. The accuracy of the measurement instrument was negatively influenced by the researcher’s
inexperience.
2. The alternative hypothesis was that the new medicine was significantly more effective than
the previous medicine.
3. Subject attrition made generalizing the results to the population a challenge. This is because
the sample, as it was now altered due to subject drop-out, no longer resembled the
population.
4. The bias exhibited by the researcher was palpable. When the subjects responded positively,
he attributed it to the new therapy, but when they responded negatively, he always attributed
it to something else.
5. There was no causal relationship between the two variables. Differences in the independent
variable did not produce differences in the dependent variable.
6. Understanding the diffusion or spread of scores provides an indication of where an individual
subjects score is in relation to the group.
7. The cause and effect relationship claimed by the experimenter was questioned due to the
existence of numerous potential confounding variables in the study.
8. Random selection, where subjects are randomly selected from a population, is a fundamental
component in the attainment of high external validity.
9. When the sample does not reflect the population, it makes it difficult for the researcher to
generalize the results of the study back to the population from which the sample was drawn.
10. When the independent variable has a direct influence on the dependent variable, and other
potentially confounding variables have been ruled out, the researcher may claim high internal
validity.
11. Concerns that subject maturation may have influenced the results arose when it was revealed
that the study lasted an additional ten months.
176
12. Subject mortality may create differences between groups that can reduce internal validity.
13. The novelty of having a new teacher in the class initially caught the student’s attention and
the students studied diligently, but after this effect diminished, the students returned to their
normal study habits.
14. For a study to have high external validity, the sample should meet size standards and reflect
the population.
15. Extreme scores on the pre-test naturally regressed closer to the mean on the post-test, thus,
distorting the experimenter’s results and calling into question the effectiveness of the
treatment.
16. The sampling process was disorganized and chaotic. The researchers initially used random
sampling but grew tired and resorted to convenience sampling.
1. The experimenter showed no bias in his interpretation of the results; he looked at the data and
came to an objective and dispassionate conclusion.
2. We want to minimize the influence of chance within our study. We can do this by limiting
the variation in the results by controlling the research environment, using more precise and
accurate measurement instruments, clearly defining the research process, etc.
3. Cluster sampling is an effective way to break down and randomize the selection of subjects
within very large populations.
4. He gathered his sample based on convenience. He approached people on the street and asked
5. When the sample reflects the population, we can appropriately generalize the results of the
study back to the population from which the sample was drawn.
6. Engaging in exploratory research is often a good way to generate ideas for experimental
research.
7. The population is everyone or everything the researcher wants to study.
8. The probability that the sample represents the population is greatly increased when random
9. The quota of 120 subjects from the 50-60 age group was not reached, reducing that age
group’s representativeness within the study and lowering the study’s external validity.
10. Random selection was used so that the sample adequately represented the population.
11. To claim that a study has high external validity, the sample needs to represent the population
from which it was drawn.
12. Snowball sampling is often an effective way to recruit new and difficult to find subjects
through accumulative processes.
13. A survey was distributed to members of the class. This convenience sample was helpful in
obtaining a quick and cost-effective view of the opinions of the students.
177
14. His study lacked systematic rigor. The experimental process was not clearly defined,
experimental processes were conducted in a haphazard fashion, and the experimental
environment was not sufficiently controlled.
15. Subject participation within an experiment needs to be voluntary.
Chapter 9: True Experiments and Quasi Experiments
1. Random assignment, randomly designating subjects to different experimental groups, is a

2. Experimental causation cannot be claimed unless there is random assignment, a way to
measure the dependent variable, a controlled experimental environment, and the capacity to
manipulate the independent variable.
3. The researcher failed to control numerous extraneous variables, causing the post-test
4. The dependent variable is not directly manipulated by the researcher, it responds to the
5. When experimental groups are nonequivalent, when there is no random assignment or when
there is random assignment with a small sample size, subjects can be given pre-tests to
determine equivalency on certain metrics.
6. Quasi-experimentation usually reflects true-experimentation in its general framework, the
difference being that the former lacks one or more fundamental elements of the latter.
7. Without random selection, the sample may not reflect the population, external validity may
be reduced, and the experimental results may not be generalizable.
8. To claim that a study has high reliability, researchers should be able to replicate their results
on subsequent tests.
9. To claim that the effects of a study will likely be reflected in the population, the sample
needs to be representative of the population.
1. The selection criteria were so limiting that too few studies were available for the meta-
analysis.
2. The dependent variable was improperly measured. This is because the measurement
instrument was used over a long period of time and it became more variable and less precise.
3. The effect size of the experiment was low, indicating that the magnitude of the difference
4. When researchers select subjects from a population, split subjects into groups, apply
treatments and controls, and compare the mean scores of the groups, the results will be
estimates of the true scores—experimental error will likely distance these estimates from the
true score in some way.
178
5. The statistics were conducted improperly; data was entered haphazardly, the columns were
incorrectly labeled, and the wrong tests were run through the software.
6. It is often valuable to synthesize quantitative research and qualitative research into a singular
study to provide a more holistic and all-encompassing view of a phenomenon.
1. Attributing the correct authors for their work in group experiments is vital in determining
who should receive recognition for academic publications.
2. A breach of proper ethical behavior occurred when the subject’s anonymity was lost.
3. Remember to cite all the material and ideas that belong to others that were included in your
research.
4. The instructor coerced his students into engaging in his experiment by explaining that if they
did not participate they would fail his class.
5. The lead researcher consulted with the legal department to help her draw up a contract that
was to be offered to the research assistants.
6. When possible, it’s important to debrief subjects so that they understand the true reasons for
their participation in the research.
7. The ethical behavior exhibited by the researcher was impeccable. She had the participants
sign a consent form, treated them with respect and dignity, and fully disclosed the true nature
of the research once the experiments were completed.
8. It would be unethical for researchers to provide too strong of an incentive to attract
participants, such as an overabundance of cash, because people may find it difficult to refuse
involvement in experimentation in the face of large rewards.
9. Subject participation within an experiment needs to be voluntary.
179

Learning English Through Research Methods Textbook

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Learning English Through Research Methods Textbook

Uploaded by

Copyright:

Available Formats

Learning English through Research Methods

Nicholas Lassi, PhD, LLM

Chapter 1: Empiricism (经验主义)

Theory (理论) Testing

Four Essential Parts Required to Have a True Experiment

True experiments should encompass (拥有) these four essential parts:

Quasi (拟) -Experiments

Quasi ( 拟 ) -experimentation represents non-causal studies that usually lack random ( 随 机 )

The Use of Secondary Data

Survey (调查) Studies

A survey ( 调 查 ) (or questionnaire) is a method of information and data collection—usually

Theory (理论) Building

Theory ( 理 论 ) building involves: qualitative ( 定 性 ) research, case-study methods, and

Qualitative (定性) Research

Case-study of life working on an archaeological excavation (Photo: David Hallam).

Exploratory (探索) Research

Exploratory ( 探 索 ) research gathers preliminary ( 初 步 ) information and data to generate

1) What is empirical evidence?

Exercise 1: Word Match

Match the term with the definition:

Data / Pieces of information that are often generated quantitatively.

Exercise 2: Crossword Puzzle

The scientific method as an ongoing process (Wikimedia Commons).

The Scientific Method: Steps in the General Research Process

Exercise 1: Word Match

Match the term with the definition:

Exercise 2: Crossword Puzzle

Exercise 3: Word Search

1. Something that happens that is studied.

Exercise 4: Fill in the Blanks

Chapter 3: Exploratory (探索) Research, Descriptive (描

Exploratory (探索) Research

Pilot Studies and Focus Groups

Descriptive (描述) Research

Descriptive ( 描述 ) research is a report of the characteristics or relationships within a sample ( 样

Correlational (相关) Studies

A major form of descriptive research is the correlational ( 相 关 ) study. Correlational ( 相 关 )

Causal (因果) Research

An experimenter engaged in causal research should emphasize obtaining high internal

1) What is exploratory research?

Exercise 1: Word Match

Match the term with the definition:

Average / The middle position in a data set.

Exercise 2: Crossword Puzzle

1. Enumerating the occurrence of an event or phenomena.

Exercise 4: Fill in the Blanks

Terms: average, categorize, causal, characteristics, confounding, control, correlation, formulate,

Chapter 4: Quantitative (定量) Research, Qualitative (定

Quantitative (定量) Research

Qualitative (定性) Research

Interviews and Field Studies

An in-person interview environment (photo: David Davies)

1. Structured interviews are conducted systematically ( 系 统 ), the questions are

Surveys (调查): Questioning Subjects for Information

Research in the Field: Ethnographies ( 民 族 志 ), Phenomenological ( 现 象 学 ) Studies, and the

An ethnography ( 民族志 ) is a qualitative ( 定性 ) study (though, quantitative ( 定量 ) elements are

Ethnographic Museum of Kruja (photo: Collective Commons).

Creswell (2013) urges qualitative researchers to employ nontraditional methods to their

Phenomenological (现象学) Method

Grounded Theory Approach

Researcher and Subject Interaction in Qualitative Studies

Mixed-methods research involves combining quantitative ( 定量 ) and qualitative ( 定性 ) research

Quasi ( 拟 ) -experimentation represents non-causal studies that usually lack random ( 随机 )

A survey ( 调查 ) (or questionnaire) is a method of information and data collection—usually

Theory ( 理论 ) building involves: qualitative ( 定性 ) research, case-study methods, and

Exploratory ( 探索 ) research gathers preliminary ( 初步 ) information and data to generate

A major form of descriptive research is the correlational ( 相关 ) study. Correlational ( 相关 )

1. Structured interviews are conducted systematically ( 系统 ), the questions are

Research in the Field: Ethnographies ( 民族志 ), Phenomenological ( 现象学 ) Studies, and the

Controlling extraneous ( 无关 ) variables involves holding everything within the experimental

Sources of Confounding ( 混杂 ): Measurement Instruments, Situational Characteristics, and

The introduction of treatments could also be randomized ( 随机 ) among different groups or

Without random ( 随机 ) selection there is a much greater likelihood of sampling bias ( 偏差 ).