III.

RESEARCH
METHODS
(A. Experimental Design)

Mayeyean T. Taguibalos, RPm

College Instructor 1
WHAT IS A RESEARCH METHOD?
▪ Research methods should be selected based on their
ability to best address your research purpose and to help you
test your hypothesis or answer your research questions.

▪ The primary quantitative designs are experimental

research and survey research.

2
What is an Experimental Research?

is a type of research method in

which you manipulate one or Experimental design means
more independent variables and creating a set of procedures
measure their effect on one or to test a hypothesis.
more dependent variables.

3
EXPERIMENTS

▪ Experiments rely on hypothesis testing

(testing variable relationships). The basic use
of experiments is to test how introducing an
intervention (a variable) affects what
happens.

4
EXPERIMENTS
▪ In other words, you select research subjects, do
something to them, and observe the effect of what
you’ve done (Babbie, 2013).

▪ In order to be certain that you are measuring the effect

of the intervention (variable) you have introduced, you
need to control for all other factors.

5
 Manipulation

• Applying certain treatments (levels of independent variables) to the different

conditions

6
• The researcher’s attempts to
make the groups as similar as
possible on most variables other
than independent variable.

• Direct control e.g. focus on

certain characteristics by
exclusion CONTROL
• Age, intelligence, socioeconomic
class, previous experience, good
school system etc

7
RANDOMIZATION
In a randomized experimental design, objects or
individuals are randomly assigned (by
chance) to an experimental group. Using
randomization is the most reliable method of
creating homogeneous treatment groups, without
involving any potential biases or judgments.

8
VARIABLES IN EXPERIMENTAL
RESEARCH
❖ Independent Variable:
- Experimental Variable, Cause, or
Treatment
- The activity or characteristic the
researcher believes makes a difference.

9
VARIABLES IN EXPERIMENTAL
RESEARCH
❖ Dependent Variable:
- Criterion Variable, Effect, or Posttest
- Outcome of the study
- Difference in group(s) that occurs as a
result of the manipulation of the IV
- Only constraint: must represent a
measurable outcome
10
LEARNING CHECK!
Dr. Pi does a study with three groups
of fifteen students each. Each group
consumes a different amount of
caffeine (high, medium, and low
amounts). Dr. Pi then measures all the
students on how well they do on a
concentration test.
What is the independent variable in
this study? Dependent variable?
11
 Answer:

IV- different amount

of caffeine
DV- concentration

LEARNING CHECK!
12
LEARNING CHECK!
A researcher wants to figure out if
uncomfortable seats in a classroom will
increase the research participant's
attention to the lesson and decrease
their fidgeting.

13
 Answer:

IV- uncomfortable
seats
DV- attention to the
class and fidgeting

LEARNING CHECK!
14
15
 1 2 3 4 5 6
Select and Select subjects Select design. Execute Analyze data. Formulate
define problem. and procedures. conclusions.
[measurement]
instruments.

STEPS IN EXPERIMENTAL RESEARCH

16
FACTORS NEED TO CONTROL IN MEASURING
THE EFFECT OF THE INTERVENTION:

1) Settings- for experiments include natural

environments, labs, and the Internet.
▪ Field experiments occur in natural
environments.
▪ Research or scientific labs are the most
common setting for experiments.

17
 FACTORS
NEED TO Finally, experiments may be

CONTROL IN conducted on the Internet. This setting

is appropriate when you are studying
people’s beliefs and attitudes

MEASURING (Babbie, 2013).

THE EFFECT Your research purpose and

OF THE hypothesis will ultimately dictate the

setting of your experiment as well as
the kind of experiment you design.
INTERVENTI
ON:
18
FACTORS NEED TO CONTROL IN MEASURING THE
EFFECT OF THE INTERVENTION:

2) Experiments are used in explanatory

research and are based on causal logic (or
cause-and-effect logic).

19
THREE CONDITIONS NEEDED TO SUPPORT THE
PRESENCE OF A CAUSAL RELATIONSHIP:

The independent variable precedes the

dependent variable.

The independent variable must be related to the

dependent variable.

There must be no alternative explanation for the

dependent variable (no extraneous variable).
20
TWO GROUPS:

Experimental groups are those that receive the experimental intervention

(also called the experimental stimulus). Control groups do not receive the
intervention (in some cases, they may receive a placebo).

All experiments have at least one experimental group, but not all
experiments have control groups. Control groups are used to compare the
results of the experimental group (whose members received the intervention)
with a similar group whose members have not received the intervention.

21
PRETEST & POSTTEST
Experiments may involve one, two, or four
groups in total, depending on the type of
experiment. Some experiments involve
pretests and/or posttests in addition to the
experimental intervention.
A pretest determines a subject’s baseline
prior to introducing the experimental
intervention. A posttest is given after the
experimental intervention to assess the
impact of the intervention.
22
FOR EXAMPLE:
▪ Let’s say you want to study middle-aged people’s
biases against hip hop music because previous
research indicates that people carry assumptions
that hip hop music promotes violence. Research
indicates that some people who hold these
negative attitudes aren’t familiar with hip hop
music. You create the following hypothesis:

23
FOR EXAMPLE:
▪ People’s negative attitudes about hip hop music
will decrease with exposure to hip hop music. You
give research subjects a pretest, such as a
questionnaire, to determine their current attitudes
toward hip hop music. Then, for the experimental
intervention, you expose the subjects to 30
minutes of hip-hop music by various artists. A
posttest questionnaire determines if their attitudes
have changed after exposure to hip hop
24
THREE PRIMARY
CATEGORIES OF
EXPERIMENTS

25
THREE PRIMARY CATEGORIES
OF EXPERIMENTS:
1) Pre-experimental designs are
focused on studying a single group
that is given the experimental
intervention (experimental groups
only). Campbell and Stanley (1963)
identified three types of pre-
experiments.

26
THREE TYPES OF PRE-
EXPERIMENT DESIGNS
▪ In a one-shot case study, a
single group is given the
experimental intervention and
then observed to see if the
intervention causes any changes.
These are the weakest forms of
experiments.

27
THREE TYPES OF PRE-
EXPERIMENT DESIGNS
▪ With a one-group pretest–posttest
design, a single group is given a
pretest (to determine the subject’s
baseline), then given the
experimental intervention, and
then given a posttest. Pretest and
posttest scores are compared, and
any differences are attributed to
the experimental intervention.
28
THREE TYPES OF PRE-
EXPERIMENT DESIGNS
▪ Finally, the static-group comparison
involves two groups. First, a single
group is given the experimental
intervention. Then a comparison
group (a group that is like the
experimental group) is selected.
Then both groups are given a
posttest. The posttests of both
groups are compared, and any
differences are attributed to the
experimental intervention.

29
SUMMARY TABLE:

30
 A Quiz for this topic will be posted on our Google
Classroom.

Asynchronous Class: The continuation of the

discussion for this topic was uploaded on our
Google Folder.

ANNOUNCEMENT: Midterm Exam (100 items) (November 04,

2021,Thursday) (8:00-9:30 am)
Type of Exam: Multiple Choice, True or False,
Situational Analysis

Coverage : Introduction to Research in Psychology

– Research Methods (Experimental Research)

31
 Three primary categories of experiments:

2) True experimental designs (also called classical experiments) are based on randomization.
Research subjects are randomly assigned to experimental and control groups. Because both
randomization and control groups are used, true experiments are considered the strongest form of
experiments. Campbell and Stanley (1963) identified three types of true experiments.

32
 Three types of true experiments

▪ The pretest–posttest control-group design involves two groups, with each given a pretest;
only one group is given the experimental intervention, and then both groups are given a posttest.
The group that receives the experimental intervention is the experimental group, and the group
that receives only the pretest and posttest is the control group.

33
 Three types of true experiments

▪ In the Solomon four-group design, all groups receive the posttest, but the pretest and
experimental intervention combination differs for each group. The first group receives a pretest,
the experimental intervention, and a posttest. The second group receives the pretest and posttest.
The third group receives the experimental intervention and the posttest. The final group receives
only the posttest. This rigorous design controls for both the effect of the pretest and the effect of
the intervention on posttest scores.

34
 Three types of true experiments

▪ The posttest-only control-group design involves two groups. One is given the experimental
intervention and a posttest, and the second group is given only the posttest. The group that
receives the experimental intervention is the experimental group, and the group that receives only
the posttest is the control group. (See Table 4.2.

35
SUMMARY TABLE

36
 Three primary categories of experiments:

3) Quasi-experimental designs involve taking advantage of natural settings or groups, and thus
subjects are not randomly assigned.
For example, these designs are often used in education research when researchers have access to
specific educational institutions in which to conduct the study, in management research when
researchers have access to specific businesses, in health research when researchers have access to
specific medical institutions, and so on.

37
 Three primary categories of experiments:

Quasi-experimental designs may involve experimental groups only or experimental and control groups.
Campbell and Stanley (1963) noted that quasi-experiments are appropriate when “better designs are
not feasible” (p. 34). They identified 10 types of quasi-experiments but only four will be tackled.

38
 Four types of quasi-experimental designs

▪ The time-series experiment involves taking measures of a single group for a predetermined
period of time, then giving the group the experimental intervention, and then again taking
measures of the group for a predetermined period of time.

39
 Four types of quasi-experimental designs

▪ The multiple time-series experiment involves taking measures of two groups for a
predetermined period of time, then giving one group the experimental intervention, and then
again taking measures of both groups for a predetermined period of time. The group that receives
the experimental intervention is the experimental group and the group that does not is the control
group. T

40
 Four types of quasi-experimental designs

▪ The nonequivalent control-group design involves two groups. One group receives a pretest,
experimental treatment, and posttest. The second group receives only the pretest and posttest.
(See Table 4.3.)

41
▪ Whereas research subjects do not know if they have been placed in an experimental or a control
group, in double-blind experiments neither the subjects nor the researchers know which
subjects are in the experimental group and which are in the control group. This design eliminates
the possibility that researchers’ observations will be skewed based on their desire to see changes
in the experimental group.

42
 Four types of quasi-experimental designs

▪ In addition to the preceding types of experimental designs, there are also single-subject
designs (also known as N-of-1 designs). A single-subject design involves multiple observations
of one individual. First, multiple observations are recorded to determine the individual’s baseline;
then the experimental intervention is introduced, and additional observations are recorded
(Creswell, 2014). (See Table 4.4.)

43
44
 Hawthorne effect

▪ The Hawthorne effect (also referred to as testing effect) refers to how participation in a
research study may, on its own, impact subjects’ responses. For instance, in the hip hop music
example given earlier, if subjects are asked to complete a pretest prior to the experimental
intervention, they may believe, when they are completing the posttest, that the researchers want
to see a change in their attitudes, so their answers may reflect their desire to “do well” for the
researchers.

45
 ▪ refers to how accurately a method
measures what it is intended to measure.
If research has high validity, that
means it produces results that correspond
to real properties, characteristics, and
variations in the physical or social world.
WHAT IS High reliability is one indicator that a
measurement is valid.
VALIDITY?

46
THREATS TO VALIDITY
 Threats to validity

are factors/potential threats in an

experiment that could possibly affect or
alter the outcome of the study
 • Internal validity threat
• external validity threat

there are 2 types of threats to

validity
INTERNAL VALIDITY THREAT
are factors that threatens the
researcher's ability to draw correct
inference from the data about the
population in an experiment

e.g. experimental procedures, treatments, experiences of

the participants
EXTERNAL VALIDITY THREATS
happens when experimenters draw incorrect inference from the sample data to other persons,
other settings and past or future situations
these threats arise because of the characteristics of individuals selected for the sample, the
uniqueness of the setting, and the timing of the experiment. For example, threats to external
validity arise when the researcher generalizes beyond the groups in the experiment to other
racial or social groups not under study, to settings not examined, or to past or future situations.
 Other threats that might be mentioned in the method section are the threats to statistical
conclusion validity that arise when experimenters draw inaccurate inferences from the data because
of inadequate statistical power or the violation of statistical assumptions. Threats to construct
validity occur when investigators use inadequate definitions and measures of variables.
 PRACTICAL RESEARCH TIPS FOR PROPOSAL
WRITERS TO ADDRESS VALIDITY ISSUES AS
FOLLOWS:
Identify the potential threats to validity that may arise in your study. Separate section in a proposal may be
composed to advance this threat.
Define the exact type of threat and what potential issue it presents to your study
Discuss how you plan to address the threat in the design of your experiment.
Cite references to books that discuss the issue of threats to validity such as Cook and Campbell
(1979(; Shandish, Cook, Campbelll (2001): and Tuckman (1999)
THE PROCEDURE

Describe A proposal developer needs to describe in detail the procedure for

conducting the experiment. A reader should be able to understand
in the design being used, the observations, the treatment, and the
timeline of activities.

Discuss Discuss a step-by-step for the procedure in the experiment.

58
FOR EXAMPLE, BORG AND GALL (2006) OUTLINED STEPS TYPICALLY USED IN THE PROCEDURE
FOR A PRETEST-POSTTEST CONTROL GROUP DESIGN WITH MATCHING PARTIC IPANTS IN THE
EXPERIMENTAL AND CONTROL GROUPS:

Administer measures of the dependent variable or a variable

closely correlated with the dependent variable to the research
participants.

Assign participants to matched pairs on the basis of their scores on

the measures described in step 1

Randomly assign one member of each pair to the experimental

group and the other member to the control group

59
FOR EXAMPLE, BORG AND GALL (2006) OUTLINED STEPS TYPICALLY USED IN THE PROCEDURE
FOR A PRETEST-POSTTEST CONTROL GROUP DESIGN WITH MATCHING PARTIC IPANTS IN THE
EXPERIMENTAL AND CONTROL GROUPS

Expose the experimental group to the experimental treatment and

administer no treatment or an alternative treatment to the control
group

Administer measures of the dependent variables to the

experimental and control group

Compare the performance of the experimental and control groups

on the post-test(s) using tests of statistical significance

60
 DATA ANALYSIS
Tell the reader about the types of statistical analysis that will be used during the experiment.
Report the descriptive statistics calculated for observations and measures at the pretest or posttest
stage of experimental designs. This call for descriptive analysis is consistent with the recent APA
Publication Manual (APA, 2010). These statistics are means, standard deviations, and ranges.
 DATA ANALYSIS
Indicate the inferential statistical tests used to examine the hypotheses in the study. For
experimental designs with categorical information (groups) on the independent variable and
continuous information on the dependent variable, researchers use t tests or univariate analysis of
variance (ANOVA), analysis of covariance (ANCOVA), or multivariate analysis of variance
(MANOVA—multiple dependent measures).
 DATA ANALYSIS
(Several of these tests are mentioned in Table 8.3, which was presented earlier.) In factorial
designs, both interaction and main effects of ANOVA are used. When data on a pretest or posttest
show marked deviation from a normal distribution, use nonparametric statistical tests. Also,
indicate the practical significance by reporting effect sizes and confidence intervals.
 DATA ANALYSIS
For single-subject research designs, use line graphs for baseline and treatment
observations for abscissa (horizontal axis) units of time and the ordinate (vertical axis)
target behavior. Researchers plot each data point separately on the graph, and connect
the data points with lines (e.g., see Neuman & McCormick, 1995). Occasionally, tests of
statistical significance, such ast tests, are used to compare the pooled mean of the
baseline and the treatment phases, although such procedures may violate the
assumption of independent measures (Borg & Gall, 2006).
 INTERPRETING RESULTS
The final step in an experiment is to interpret the findings in light of the hypotheses or research
questions set forth in the beginning. In this interpretation, address whether the hypotheses or
questions were supported or whether they were refuted. Consider whether the treatment that was
implemented actually made a difference for the participants who experienced them.
 INTERPRETING RESULTS
Suggest why or why not the results were significant, drawing on past literature that you reviewed
(Chapter 2), the theory used in the study (Chapter 3), or persuasive logic that might explain the
results. Address whether the results might have occurred because of inadequate experimental
procedures, such as threats to internal validity, and indicate how the results might be generalized
to certain people, settings, and times. Finally, indicate the implications of the results for the
population studied or for future research.