Experimental

Research Design
Module IV continued
What is Experiment?
An experiment is used to infer a causality.
In an experiment, a researcher actively manipulates one or more causal
variables and measures their effects on the dependent variables of interest.
Since any changes in the dependent variable may be caused by a number of
other variables, the relationship between cause and effect often tends to be
probabilistic in nature.
It is virtually impossible to prove causality.

The sales manager of a soft drink bottling company sends some of his sales
personnel for a new sales training programme. Three months after they return from
the training programme, the sales in the territory where this sales force was working
increases by 20 per cent. The sales manager concludes that the training programme is
very effective and, therefore, the sales force from the other territories should also be
sent for the same. Is he correct?
Causality
Causality : one can infer a cause-and-effect relationship (which
is probabilitistic). We can’t prove causality.
Conditions:
Concomitant variation: the extent to which a cause, X and an
effect, Y vary together in the way predicted by the hypothesis
under consideration, e.g. sales highly depends on the service
provided within the store.
This means that there has to be a strong association between X
and Y. Moreover, both of them need to occur together.
However, a strong association between the two does not imply
causality. It could be due to the influence of other extraneous
factors which may be influencing both the variables or it may be
the of result of random variations.
Causality
Occurrence of variables: The causing event must occur either
before or simultaneously with the effect, it cannot occur
afterwards. Sales increase should occur after improvement of in-
store service.
There might be a two-way relation b/w the variables!!

Other Causal factors: Absence of other factors implies that the

variable taken into consideration should be the only possible
causal variable. Practically, it is very difficult to control;
The researcher cannot rule out the influence of other causal factors
such as the weather condition. However, it may be possible to
control some or more of the extraneous variables by the use of
experimental design.
Experiment: Contexts
The objective of an experiment is to measure the influence of the independent
variables on a dependent variable while keeping the effect of other extraneous
variables constant.
• Can a change in the package design of a product enhance its sales?
• Should a supermarket introduce a discount scheme on bulk purchase to increase
its sales?
• Will an increase in the shelf space allocated to a brand of a particular product
increase its sales?
• Will a reduction in the price of the menu items of a restaurant increase sales?
• What will be the impact of POP display of ‘Arrow’ shirts on their sales?
• Which of several promotional techniques is most effective in increasing the sales
of a product?
• What is the impact of increasing the proportion of female counter clerks from 30
to 60 per cent on the sales of the store?
• Does mentoring help in acclimatizing a person to the organizational culture?
• Does organizational climate impact the quality of working life of a company?
• What is the impact of change in home loan rates on the investor investment in
real estate?
Experimental Research Design
Experiment is a study in which the researcher manipulates the
independent variables by applying some treatments i.e.
conditions and measures the effect on dependent variables, while
controlling for the extraneous variables

Research Design Concepts

Dependent & Independent Variables/ Treatments (e.g. effect of a

price change on sales)
Extraneous variable & Confounded relation
Test Units
Experimental & control groups
Validity in Experiments
Internal Validity
External Validity

The researcher has two goals while conducting an experiment:

1. To draw valid conclusions about the effect of treatments
(independent variables) on the dependent variables.
2. To make generalizations about the results to a wider
population.
Here, the concern of the first goal lies with internal validity,
whereas the second one is concerned with the external validity.
Internal Validity
Internal validity tries to examine whether the observed effect
on a dependent variable is actually caused by the treatments
(independent variables)
For an experiment to be possessing internal validity, all the
other causal factors except the one whose influence is being
examined should be absent.
Internal validity is the basic minimum that must be present.
It is impossible to draw inferences about the causal
relationship between the independent and dependent variables
if the observed effects on test units are influenced by
extraneous variables.
Control of extraneous variables is a necessary condition for
inferring causality.
Without internal validity, the experiment gets confounded.
External Validity
External validity refers to the generalization of the results of
an experiment. The concern is whether the result of an
experiment can be generalized beyond the experimental
situations.
If it is possible to generalize the results, then to what
population, settings, times, independent variables and the
dependent variables can the results be projected.
It is desired to have an experiment that is valid both internally
and externally.
However, in reality, a researcher might have to make a trade-
off between one type of validity for another. To remove the
influence of an extraneous variable, a researcher may set up an
experiment with artificial setting, thereby increasing its internal
validity. However, in the process the external validity will be
reduced.
Factors affecting validity
External validity is affected by
The environment at the time of test may be different from
the environment of the real world where these results are to be
generalized. For example, a commercial advertisement may be
shown to a set of prospective customers and their reaction to
the advertisement may be very favourable. However, the same
advertisement appears while the respondents are watching TV
at another channel and the results may not be the same.
Population used for experimentation of the test may not be
similar to the population where the results of the experiments
are to be applied. Example: suppose the students of a college
are asked to perform a task (say, an assignment) that could be
manipulated to study the effects on their performance.
However, the findings of this study cannot be generalized to
the real world when the same task is assigned to the
employees of an organization.
Factors affecting validity
External validity is affected by
• Results obtained in a 5–6 week test may not hold in an
application of 12 months. Suppose a company wants to launch
ice cream in Delhi NCR. The results of the survey conducted
during the months of May and June may be extremely
favourable. These results would certainly not be applicable
during the winter months in December and January, thereby
raising questions on the generalizability of the results.
• Treatment at the time of the test may be different from the
treatment of the real world. This can happen when while testing
the effect of a treatment, it is administered in the form of a pill
and in reality it is given as a part of a cereal.
Example: before sending for training the employees are
informed about the consequence, e.g. they are chosen for future
leadership roles. But, if they are not told and sent for training.
Then the effectiveness of training cannot be generalized.
Factors Affecting Validity
Internal Validity is affected by extraneous variables
1. History
2. Maturation
3. Testing Effect
4. Instrumentation
5. Statistical regression
6. Selection bias
7. Test unit mortality
Some Common Extraneous Variables
History : External, but the events occur simultaneously with the experiment (e.g.
employees’ awareness on financials of a company – sessions; newspaper report may
influence this)
Maturation: Changes in the test units - people becoming older, more experienced, tired, or
uninterested.
Testing effect: possible effect on the experiment of taking a measurement on the dependent
variable before presentation of the treatment: main (prior obsvn or pretest affects latter
obsrvn or posttest: effect of adv. on attitude towards a brand, people may try to be
consistent or differ – threat to internal validity) and interactive (prior measurements
(pretest) affects the response : people’s sensitivity/ perception changes, they behave
systematically in a different manner, hence result may not be generalizable, threat to
external validity)
Instrumentation: change in the measurement tool (e.g. change in sales incorporate change
in price)
Selection bias: improper assignment of test units to treatments / test units to control and
experimental groups (effect of display on sales, display of same stuff should be there in
identical stores)
Mortality: Loss of test units during experiment
Statistical Regression: test units with extreme scores chosen for treatment. E.g. poor
performers chosen for training. Increase in sales might have happened for other factors.
But for regression effect, poor performers are becoming good performers – experiment is
confounded
How to Control Extraneous Variables
• random assignment of test units to experimental groups
Randomi as well as random assignment of treatment conditions to
experimental groups - because of random assignment,
zation extraneous factors will be operating in experimental
groups. For randomization to be effective, a large
sample size is required
• Comparison of test units on background variables before
assigning them to treatment conditions; match different groups
by confounding variables – eg, Suppose there are 120 people
Matching to be distributed in three groups. If there are 45 females, each
of the three groups is assigned 15 females. This way, the effect
of gender can be distributed among all three groups. But
matching all confounding vars may not be possible
Experiment
al Design • use of experiments to control specific extraneous variables
Control
Statistical
use of experiments to control specific extraneous variables,
Control ANCOVA
Two types of Environment
These are called laboratory environment and field
environment.
In a laboratory experiment, the researcher conducts the
experiment in an artificial environment constructed exclusively
for the experiment. E.g. the interest is in studying the
effectiveness of a TV commercial. If the test units are made to
see a test commercial in a theatre or in a room, the environment
would be of a laboratory experiment.
Field experiment is conducted in actual market conditions.
There is no attempt to change the real-life nature of the
environment. Showing of test commercial in an actual TV
telecast is an example of a field experiment.
Lab and Field Experiments
ADVANTAGES of LAB EXPERIMENTS
Laboratory experiments have higher internal validity as they provide the
researcher with maximum control over the maximum number of confounding
variables.
The effect of history can be minimized.
The results of a laboratory experiment could be repeated with almost similar
subjects and environments. Laboratory experiments are generally shorter in
duration, make use of smaller number of test units, easier to conduct and relatively
less expensive than field experiments.
DISADVANTAGES
Laboratory experiments lack in external validity i.e., it is not possible to
generalize the results of the experiment.
FIELD EXPERIMENTS
Experiments conducted in the field have lower internal validity. The ability to
generalize the results of the experiment is possible in case of a field experiment,
thereby leading to higher external validity.
Types of Experimental Designs
Experimental Designs
It’s a set of procedures that specify the test units, treatments, dependent
variables and the method to control extraneous variables
Statistical: a series of basic experiments that allows for statistical analysis and control of external
variables
Others: (Pre-experimental, true experimental and quasi-experimental )
Statistical research designs :
Completely randomized design
Randomized block design
Latin square design
Factorial design
Experimental Designs :
Pre-experimental: Designs that don’t control for extraneous factors
True-experimental: Designs where one can assign test units to EG randomly
and also assign treatments randomly to EGs
Quasi-experimental: Designs where true-experimental procedures can be
applied partly, but not fully
18
Pre Experimental Designs
Pre-experimental Designs: don’t use randomization to
control extran. Vars (pretest or CG not present)
 One-shot Case Study ( x O1)
 One group pre test-post test design ( O1 x O2)
 Static group comparison/design ( EG: x O1,
CG: O2)
e.g. training programme for those selected without any basis/
they have volunteered

Cannot control extraneous variables

Quasi-experimental Designs

It is used when it is not possible to assign test units randomly to

treatments. But, it helps in controlling extraneous variables.
Example: Time-series designs (Single and Multiple)

Controlling history is a problem in this design (many things can

change over time)
Time Series Design:
O1 O2 O3 O4 x O5 O6 O7 O8
Multiple Time Series Design:
You have two groups EG and CG
Quasi Experimental Designs
This design lacks complete control of scheduling of treatment and also lacks the
ability to randomize test units’ exposure to treatments.
Time series design involves a series of periodic measurements on the dependent
variable for a group of test unit.
The treatment X is then administered and a series of periodic measurements are
again taken to measure the effect of treatment.
O1 O2 O3 O4 X O5 O6 O7 O8
This is a quasi-experimental design since there is no randomization of treatment to
test units.
Because of the multiple observations in time series design, the effect of maturation,
main testing effect, instrumentation and statistical regression can be ruled out. If test
units are selected at random, selection bias can be reduced. Further, if a strong
measure like giving certain incentives to the respondents is introduced, mortality
effect can more or less be controlled.
The major drawback of this experiment is the inability of a researcher to control
the effect of history. The results of the experiment may be affected by an interactive
testing effect because multiple measurements are made on these test units.
Quasi Experimental Design
Multiple time series design: one more group called the ‘control
group’ is added to the time series design.
Experimental Group: O1 O2 O3 O4 X O9 O10 O11 O12
Control Group: O5 O6 O7 O8 O13 O14 O15 O16
Example: sales training
The treatment effect (sales training) is found by comparing the
average sales of the two groups before and after the training
programme.
The major drawback of this design is the possibility of the
interactive effect in the experimental group.
True Experimental Designs
After-only with Control (Post test only Control Group): we take both experimental
and control groups and the effect is measured as the difference between the
observation on control group and that of the experimental group after treatment
EG: R X O1
CG: R O2 effect: O1 – O2 (treatment effect + extraneous factors) –
(extraneous factors) = treatment effect
a large randomized sample is required. This design is widely used in marketing
research.
Before and After with Control (Pre test Post test Control Group): The effect is
measured as the difference between the change in observations before and after the
treatment for the two groups.
EG: R O1 X O2
CG: R O3 O4 effect: (O2 – O1) – (O4 – O3)
Difference in the post-test and pre-test measurements of EG and CG control group
is denoted by A and B respectively, then
A = O2 – O1 = Treatment + extraneous variables
B = O4 – O3 = Extraneous variables
This design controls for most extraneous variables
True Experimental Design
Solomon four group design: Two control and two experimental groups are
taken. The effect can be measured in many ways.
EG1: R O1 X O2
CG1: R O3 O4
EG2: R X O5
CG2: R O6
It controls interactive testing effect as well as extraneous variables

It involves huge time and cost, not frequently used

Solomon 4 Group Design: Analysis
Experimental Group 1:
O2 – O1 = Treatment effect + extraneous factors without interactive testing effect +
interactive testing effect
Control Group 1:
O4 – O3 = Extraneous factors without interactive testing effect

………………………………………………………………….
O5 – O6 = Treatment effect
O2 – O1 – [O5 – (O1 + O3)/2] = Interacting testing effect
This design helps not only in measuring the effect of treatment, but also in
obtaining magnitude of the interactive testing effect and extraneous factors.
this experimental design guarantees the maximum internal validity.
In businesses where establishing cause-and-effect relationship is very crucial for
survival, this design is useful.
 Exercise
1. An NGO wanted to test the effectiveness of an anti-dowry
commercial. Two random samples, each of 250 respondents, were
recruited in UP. One group was shown the commercial and then
attitude towards dowry were measured for respondents in both
groups.
 Identify the dep. and indep. Variables.
 What type of design is used?
 What are the extraneous factors here?
 If the respondents had been selected by convenience, what type of design
would result?
Dep: attitude towards dowry; indep var: anti-dowry campaign
Post only with control design – true experimental
Education, occupation, gender, age etc.
It would have been a pre-experimental design
 Chawla and Sondhi – chap 4, q2, q3 (pg 81)

2.
3.
Statistical Designs
What are they?
Statistical designs allow for statistical control and analysis of
external variables.
Advantages:
• The effect of more than one level of independent variable on
the dependent variable can be manipulated.
• The effect of more than one independent variable can be
examined.
• The effect of specific extraneous variable can be controlled.
Types
Completely Randomized Design (CRD) – studying the effect
of 1 independent variable on 1 dependent variable
Randomized Block Design (RBD) – design based on 1
specific extraneous variable
Latin Square Design (LSD) – design based on 2 specific
extraneous variables
Factorial Design – many extraneous variables
Completely randomized design (CRD)
It is used when the researcher has to evaluate the effect of a
single variable.
The effects of extraneous variables are controlled by the
randomization technique.
This involves randomly assigning test units to treatments. But
this is applicable only when the test units are homogeneous.
Example: sales depend on prices (low, medium, high)
Here the test units are the stores which are randomly assigned to
the three treatment levels.
The average sales for each price level is computed and examined
to see whether there is any significant difference in the sale at
various price levels.
main limitation that it does not take into account the effect of extraneous
Variables (e.g. store size, competitor’s prices)
CRD
Example 2: A company wants to know whether sales generated by
salesmen differs across different sections. They appoint newly recruited
trainees in increasing sales in different sections. It is assigning 9 newly
recruited management graduates from 3 different business schools to 3
different sections of its store and sales for 4 weeks are recorded and
they can be assigned on the basis of simple random sampling. Assume
that the management graduates are of same quality.

Clothings(S1) Lifestyle Products(S2) Household products(S3)

A3, B2, B3 C1, C2, A1 A2, B1, C3

CRD
Example 3: Study wants to find out whether maintaining higher
spreads from national average price affects sales

3 price spreads of a brand compared to a national brand are used

as treatment levels (Rs. 7, 12, 17) and sales is recorded for 18
stores
R O1 X1 O2
R O3 X3 O4
R O5 X5 O6
X1, X3, X5 are treatments
Stores are homogeneous
Analysis
Stores Prices Spread 7
Average Average
Sales wk1 Sales wk2 Sales wk3 Sales wk4 Sales wk5 Sales wk6 Sales wk7 Sales wk8 (Pre) (Post)
1 100 110 105 117 98 100 95 91 108 96
2 120 134 125 141 112 100 124 116 130 113
3 114 110 92 100 98 94 93 91 104 94
4 165 146 133 148 142 152 137 145 148 144
5 87 97 101 127 112 100 99 97 103 102
6 128 117 120 131 125 112 101 90 124 107
Prices Spread 12
7
8
9
10
11
12
Prices Spread 15
13
14
15
16
17
18

Difference between means test, categorywise and altogether

Randomized complete block design (RBD)
It is used when the researcher feels that there is one major
extraneous variable that will influence experimental results. So the
test units are blocked or grouped based on that extraneous variable.
Example 1: size of the store is the extraneous variable and becomes the blocking
factor.
Now the treatments are randomly assigned to the blocks in such a way that each
treatment appears in each block at least once.
If there are 3 stores each is small, medium and big categories, how will the
design look like?

  Store Size
Mediu
(treatment) Small m Large Small 3 S1 to S3
Low Price S1 M3 L2 Medium 3 M1 to M3
Medium Price S2 M1 L3 Large 3 L1 to L3
High Price S3 M2 L1
Latin Square Design (LSD)
It is used when the researcher has to control two extraneous variables having
significant effect on the dependent variable.
The design has to have same no. of rows, columns (i.e. no. of categories of 2
extraneous variables) and treatments
Example: If along with store size, there is another extraneous variable, say, type of
packaging that influences sales, then we may use LSD.
  Store Size
Larg
Packaging Small Medium e
the treatment should be assigned
pouch A1 B3 C2 randomly to cells in such a way that
each
tetra A2 B1 C3
bottle A3 B2 C1 treatment occurs once and only once
in
each row and in each column
Factorial Design
It is used to examine the effect of two or more independent variables (or treatments) even if they are
interactive.
Measures the effect of two or more independent variables at various levels.
Allows interaction between the variables.
An interaction is said to take place when the simultaneous effect of two or more variables is different
from the sum of their individual effects.
An individual may have a high preference for mangoes and may also like ice-cream, which does not
mean that he would like mango ice cream, leading to an interaction.
2n factorial design deals with n factors having 2 levels each
Example 1: The store may be interested to know whether the sales generated depends on price as well
as discounts (10% and 20%) offered; It will have 3*2 = 6 cells

Prices

Discounts Low P1 Medium P2 High P3

D1 (10%) sales

D2 (12%)
Factorial Design
It is used to examine the effect of two or more independent
variables (or treatments) even if they are interactive.
Example2: Suppose the company wants to study the effect of sales regions as
well as salesmen on quarterly sales revenues. We’ll use factorial design here.
Sales region has two levels: east and west
Salesmen have 4 levels: having no work-ex, work ex of <5 yrs, 5 to 10 years,
>10 years

Salesmen
Sales B1 B2 B3 B4
Region
A1 (East) sales
A2 (West)
Factorial Design
Advantages
• It is possible to measure the main
effects and interaction effect of two
or more independent variables at
various levels.
• It allows a saving of time and
effort because all observations are
employed to study the effects of
each factor.
• The conclusion reached using
factorial design has broader
applications as each factor is studied
with different combinations of other
factors.
Limitations
The number of combinations
(number of cells) increases
with increased number of
factors and levels.
However, a fractional factorial
design could be used if
interest is in studying only a
few of the interactions or main
effects.
Reliability
Consistency
Testability
Repeatability
Possible errors associated with research design

Random sampling error (a sample is an imperfect representation of the

population, hence the error = variation in popln mean and original sample
mean)
Nonsampling error related to problem definition, approach, scales,
questionnaire design, interview methods, data preparation and analysis
(response and nonresponse error)
Nonresponse error ( varn b/w true mean in the original sample and the mean
in net sample)
Response error (varn b/w true mean in the net sample and the mean obtained
finally)
Response error = Researcher error + Interviewer error + Respondent error
Errors - sources
https://www.qualtrics.com/blog/5-common-errors-in-the-resear
ch-process/

Population specification
Sampling/ Sampling frame
Sample selection

Non response error

Measurement error
Exercise 1
XYZ Pharmaceuticals is planning to manufacture a new hepatic drug A on a
mass scale, with the intention of outperforming two existing hepatic drugs B
and C. Before making the crucial decision, the company wishes to conduct an
experiment to study the new drug’s efficacy. As a preliminary step, the
research group conducting the study wants to know whether there are
differences in the effects of these three drugs on the patients irrespective of the
age and gender of the patient. 60 individuals were selected to participate in the
study.
What type of statistical research design should the research group employ
and why?
 Suppose the outcome of the above study is that there is no difference
between the effects of the three drugs. If it is given that people below 30 are
more sensitive to the three drugs than those above 30 then what type of
statistical design would you recommend and why? People below 30 are 24
and people above 30 are 36 in numbers
In addition to that, if it is believed that the sensitivity differs across gender,
which design will you choose? 50% of persons below 30 are male and 12
persons above 30 are male.
Exercise 2
You are asked to develop an experiment for studying the effect
that monetary compensation has on the response rates secured
from personal interview of certain people. This study will
involve 300 people who will be assigned to one of the following
conditions: (1) no compensation, (2) compensation of 250. A
number of sensitive issues will be explored concerning various
social problems and 300 people will be drawn from the adult
population.
Identify your variables, hypotheses and test units.
What is your research design?
Independent and Dependent Variable
Synonyms

Independent Dependent Variable

Variable (IV) (DV)
Predictor Criterion
Presumed cause Presumed effect
Stimulus Response
Predicted from… Predicted to….
Antecedent Consequence
Manipulated Measured outcome
Moderating Variables (MV)
The introduction of a four-day week (IV) will lead to
higher productivity (DV), especially among younger
workers (MV)
The switch to commission from a salary compensation
system (IV) will lead to increased sales (DV) per
worker, especially more experienced workers (MV).
The loss of mining jobs (IV) leads to acceptance of
higher-risk behaviors to earn a family-supporting
income (DV) – particularly among those with a limited
education (MV).
Extraneous Variables (EV) :cv and cfv

With new customers (EV-control), a switch to

commission from a salary compensation system (IV) will
lead to increased sales productivity (DV) per worker,
especially among younger workers (MV).

Among residents with less than a high school education

(EV-control), the loss of jobs (IV) leads to high-risk
behaviors (DV), especially due to the proximity of the
firing range (MV).
Intervening Variables (IVV)

The switch to a commission compensation system (IV)

will lead to higher sales (DV) by increasing overall
compensation (IVV).

A promotion campaign (IV) will increase savings

activity (DV), especially when free prizes are offered
(MV), but chiefly among smaller savers (EV-control).
The results come from enhancing the motivation to
save (IVV).
Relationships Among Variable Types
Relationships Among Variable Types
Relationships Among Variable Types
Discussion of Old QP

Factorial design as there are more than one independent

variables
Exercise…
…………………………contd
Ref
Chawla and Sondhi, Chapter 4