Doctor of Philosophy (Management Science)

Designing Organizational Research

Lecture 07
Design: Addressing Internal Validity - I

Dr. Muhammad Zahid Iqbal

Associate Professor
Department of Management Sciences
COMSATS Institute of Information Technology, Islamabad 1
Research Design
Foundations

2
 Contents

• Causal Challenges

• Using Design to Address Causal Challenges

•Design Types

3
 Research Design Foundations

Causal Challenges

Internal validity is present when variation in an independent variable

is responsible for variation in a dependent variable.

Criteria of internal validity suggest two general problems for

concluding that an observed relationship means an independent
variable causes a dependent variable (XY).

•Causal Direction
•Specification: Uncontrolled Variables and the Danger of Bias
4
Research Design Foundations Cont.

…
Causal Direction

5
 Research Design Foundations Cont.

…
Specification: Uncontrolled Variables and the Danger of Bias

Misspecification occurs if variables that operate in the causal model

are not included in the model studied. These variables if uncontrolled
are called nuisance variables. They provide potential alternative
explanations for the relationship(s) of interest.

For Example

IV = Education Level and DV = Subsequent Employment Income

NV = Years of Work Experience and Type of Occupation 6

 Research Design Foundations Cont.

…
…

Consequences for causal understanding resulting from uncontrolled

(nuisance) variables depend on how they are related to the
independent and dependent variables studied. Several types are:

•Bias
•Spurious Relationships
•Suppressor Variables
•Noise
•Mediators
•Moderators 7
 Research Design Foundations Cont.

Bias

For example, a researcher believes consumer satisfaction depends on computer speed.

Greater speed is expected to lead to higher satisfaction. However, the researcher does not
account for computer memory, which also influences satisfaction. The larger the memory,
the greater the satisfaction. Furthermore, computer speed and memory are positively related.
Manufacturers make computers faster as they increase computer memory.
8
 Research Design Foundations Cont.

Spurious (false) Relationships

It occurs when an uncontrolled variable accounts for all the observed relationship between a
dependent and independent variable(s). Assume, as stated previously, that memory
influences satisfaction and is related to speed. However, now assume the improbable—that
there is no causal relation between speed and satisfaction. A study that investigates only
speed and satisfaction nevertheless will find the two related. They are related spuriously
because of their joint relationships with memory. 9
 Research Design Foundations Cont.

Suppressor Variables

Suppressor variables represent biasing variables that lead to an understatement of the true
causal relationship. Suppressor variables most commonly have a positive relationship with
the independent variable and no relationship or a small negative relationship with the
dependent variable. For example, memory would suppress rather than inflate the
relationship between speed and satisfaction if it is positively related to speed but has no
relationship with satisfaction. 10
 Research Design Foundations Cont.

Noise

A noise variable is related to the dependent variable but unrelated to the independent
variable(s). Consumer affability (sociability) illustrates such a possibility. More affable
consumers report higher satisfaction with their computers because they are more satisfied
generally than are less affable consumers. However, there is no reason to suppose that
affability is related to computer speed. 11
 Research Design Foundations Cont.

Mediators

Independent variables are ordinarily thought to influence dependent

variables directly: variability in X leads immediately to variability in
Y. Mediator or intervening variables come between an independent
and a dependent variable in a causal chain. If a mediator is present,
then some or all the influence of the independent variable operates
indirectly on the dependent variable through the mediator.

12
Research Design Foundations Cont.

…
…
…

13
 Research Design Foundations Cont.

Moderators

Moderator variables have values that are associated with different

relationships between an independent and a dependent variable.

In psychology, moderator variables are often also called interaction

variables. In sociology, moderators are sometimes called boundary
conditions).

The strength and/or direction of relationships between independent

and dependent variables depend on values of a moderator variable.

14
Research Design Foundations Cont.

…
… Failure to control for a
… moderator variable
misrepresents (biases) the
observed causal relationship
between weight and
satisfaction for both computer
types. The relationship will be
overstated for desktop
computer owners and
understated for laptop
computer owners.
15
 Research Design Foundations Cont.

Using Design to Address Causal Challenges

Research design may help meet causal challenges, especially when

coupled with strong conceptualization. Following are key
characteristics of research design:

•Sampling: Selecting Cases to Study

•Measurement Decisions
•Administering Measures to Cases

Let us learn how decisions about above mentioned design

characteristics can contribute to causal clarity. 16
 Research Design Foundations Cont.

…
Sampling: Selecting Cases to Study
Decisions about samples to study often emphasize generalization of
research results. Generally, probability sampling is contrasted with
convenience or judgment sampling. Former permits valid
statistical generalization; later does not.

Two sampling decisions that bear on the control of nuisance variables

are discussed here.

•Restriction of Range
•Comparison Groups
17
 Research Design Foundations Cont.

…
…
Restriction of Range

Cases may be selected so that variation in an otherwise uncontrolled

(nuisance) variable(s) is restricted or eliminated. Restriction of range
reduces or eliminates relationships between nuisance and other
variables.

For example, a researcher includes only consumers with computers

of one memory size. Eliminating the variation in computer memory
through sample selection eliminates its effect on the speed-
satisfaction relationship. 18
 Research Design Foundations Cont.

…
…
Comparison Groups

Some research designs compare dependent variable scores across

comparison groups—two or more groups of cases. The groups are
chosen to represent different levels of an independent variable.
However, efforts are made to ensure that the groups are similar on
potential nuisance variables.

19
 Research Design Foundations Cont.

…
Measurement Decisions

Following measurement decisions can help avoid biases of nuisance

variables:

•Control Over Independent Variables

•Measurement and Statistical Control

20
 Research Design Foundations Cont.

…
…
Control Over Independent Variables
The purpose of causally motivated research is to determine how
independent variables influence dependent variable responses. Thus,
values of the dependent variable are always a characteristic of the
cases studied; the cases control or determine values of the dependent
variable. Indeed, researchers must take care so that dependent
variables are not influenced by the procedures used to measure them.
Two alternative forms of measurement remove control of the
independent variable from the cases studied.
(Exogenous events and Levels of IVs) 21
 Research Design Foundations Cont.

First, this control may be exercised by some exogenous event. For

example, a researcher may study property insurance purchases (Y) in
two communities, one having recently experienced a hurricane and
the other not. In this instance, the exogenous determination of values
of the independent variable (hurricane or no hurricane) rules out the
possibility that Y  X.

22
 Research Design Foundations Cont.

Second, a researcher may exercise direct control over levels of an

independent variable. The leadership study is illustrative. In that
study, researchers established two levels of subordinate behavior
carried out by their research confederate; these levels represent the
operationalization of the independent variable. The research
participants who role-played supervisors had no control over the
confederate and hence no control over the levels of the independent
variable they experienced. Although any relationship between
subordinate behavior and leadership behavior still might be biased by
other uncontrolled variables, the design does rule out the possibility
that a relationship resulted because the dependent variable influenced
the independent variable.
23
 Research Design Foundations Cont.

…
…
Measurement and Statistical Control
Measurement combined with data analysis procedures also can
contribute to the control of nuisance, mediator, and moderator
variables. Specifically, when researchers measure variables that
require control, they can include them in statistical models designed
to deal with more than one causal variable.

24
 Research Design Foundations Cont.

…
Administering Measures to Cases

There are two general procedures to equate values of nuisance

variables across different levels of an independent variable.

•Matching
•Random Assignment

25
 Research Design Foundations Cont.

Matching

One procedure involves matching (equating) cases that receive

different levels of the independent variable on a nuisance variable(s).
For example, assume a researcher is interested in the relationship
between computer type (laptop or desktop) and satisfaction;
computer speed and memory are recognized to be nuisance variables.
The researcher sorts among a group of computer owners matching
each laptop owner with a desktop owner whose computer has the
same speed and memory. Subsequent satisfaction differences
between laptop and desktop owners cannot be due to differences in
these two nuisance variables, because each group has a similar mix
of computer speed and computer memory. 26
 Research Design Foundations Cont.

Random Assignment

Another way to equalize groups on nuisance variables is through

random assignment; cases are randomly assigned to different levels
of an independent variable(s).

This was the procedure used in the leadership study. Participants who
role-played leaders were randomly assigned to the two levels of
subordinate behavior carried out by the research confederate.
Random assignment of supervisor participants to subordinate
behavior is attractive because it should average out the effect of
personal characteristics that may serve as nuisance variables.
27
 Research Design Foundations Cont.

Design Types
The characteristics outlined call for decisions in all research designs.
Researchers must make decisions about cases to study, measures, and
administration of measures to cases. Figure on next slide shows these
decisions combine to describe four common types of research design.

•Experiments
•Quasi-Experiments
•Surveys
•Field Studies
28
Research Design Foundations Cont.

29
 Research Design Foundations Cont.

…
Experiments

Experiments are characterized by random assignment of cases to

levels of an independent variable(s). Researchers must be able to
control (manipulate) levels of the independent variable(s) in
experiments.

30
 Research Design Foundations Cont.

…
Quasi-Experiments

Quasi-experiments share several characteristics of experiments. They

also use an independent variable(s) that is not controlled by the cases
studied.

31
 Research Design Foundations Cont.

…
Surveys and Field Studies

Field studies and surveys differ from experiments and quasi-

experiments with regard to the independent variable. In both,
responses on the independent variable are provided by the cases
studied and not by the researcher.

32