An Overview of Experimental Design

Rajdeep Chakraborti

A Classification of Qualitative Research Procedures

Qualitative Research Procedures

Direct (Non disguised)

Indirect (Disguised) Projective Techniques

Focus Groups

Depth Interviews

Association Techniques

Completion Techniques

Construction Techniques

Expressive Techniques

Causal Research methods Experimentation Definition of Terms used :

Independent variables are variables or alternatives that are manipulated and whose effects are measured and compared, e.g., price levels.
Test units are individuals, organizations, or other entities whose response to the independent variables or treatments is being examined, e.g., consumers or stores.

 Dependent variables are the variables which measure the effect of the independent variables on the test units, e.g., sales, profits, and market shares. Extraneous variables are all variables other than the independent variables that affect the response of the test units

Confounding variable: Correlated with both dependent and independent variable.

Experiments
An experiment is formed when a researcher manipulates one or more independent variables and measure their effect on one or more dependent variables while controlling the effect of other extraneous variables. ex: store sales and promotion.

Experimental Design
An experimental design is a set of procedures specifying
the test units and how these units are to be divided into homogeneous subsamples, what independent variables or treatments are to be manipulated, what dependent variables are to be measured, and how the extraneous variables are to be controlled.

Issues in Experimentation
Treatment of independent variable. Experimental Groups and Control Groups.

Selection and measurement of the dependent variable (Ex-brand image, brand awareness or product sales).
Control of extraneous variables.

Validity in Experimentation
Internal validity :whether the manipulation of the independent variables or treatments actually caused the effects on the dependent variables. Control of extraneous variables is a necessary condition for establishing internal validity. External validity :whether the cause-and-effect relationships found in the experiment can be generalized. To what populations, settings, times, independent variables and dependent variables can the results be projected?

Categories of extraneous variables

History: Specific events that are external to the
experiment but occur at the same as the experiment.

Ex-general economy has declined and you do not get the effect of promo on sales.
Longer the time interval between observations greater the chance that history will confound the effect of the experiment.

Categories of extraneous variables

Maturation: Similar to history, but when it
happens to the test units itself. Hence the changes are not caused due to the change of the independent variables, but the test units has changed.
People become older, more experienced, tired and bored.

For store, it could be a change in store layout, dcor etc.

Categories of extraneous variables

Testing effect: An effect of testing occurring when a prior observation affects a later observation. ex: Pre-treatment questionnaire making the respondent familiar with the brand which results consistent response of the test units.

Categories of extraneous variables

Mortality: The respondents are dropping out of the study. Dropouts should be distributed evenly in the sample, not only on high and low score achievers.

Experimental environment Laboratory Experiments : Contrived setting,

Artificial and controlled conditions, control on influence of extraneous variables (ex: history), high interval validity, low external validity, time and cost less, implementation easy.

Field experiments: Natural setting, no control on

external variables, low internal validity, high external validity, time and cost more, implementation not so easy.

A Classification of Experimental Designs

Experimental Designs

Pre-experimental Designs One-Shot Study One Group Pretest-Posttest Static Group

True Experimental Pretest-Posttest Control Group Posttest: Only Control Group Solomon FourGroup

Quasi Experimental Time Series

Statistical Designs Completely Randomised Randomised Blocks

Latin Square
Factorial Design

Notations used in Experimentation

X = Exposure of a group to an experimental treatment or independent variable O = Observation or measurement of the dependent variable on the test units O1, O2, O3 etc.) R = Random assignment of test units to experimental groups EG = experimental group which is assigned to experimental treatment CG = control group of test units involved in the experiment. This group is not exposed to experimental treatment Note that the notation used in experimental design assumes a time flow from left to right

Pre-Experimental Designs
One-shot design EG: X O1

Example: Impact of sales promotion on sales in select supermarkets Disadvantages: (i) The test units are not selected randomly (ii) Various extraneous variables may influence the dependent variable including history, maturation. Used for exploratory research rather than conclusive research

One-group pre-test-post-test design

EG: O1 X O2

TE = O2 O1
Example: Impact of training program on employee productivity.

Extraneous variables are uncontrolled.

Static Group Design

EG: CG: X O1 O2

TE = (O1- O2)

Two groups of test units are involved in the experiment the experimental group and the control group
Influence of various extraneous variables such as 1. Selection bias (Non random selection of test units) 2. Mortality effect ( Test units may drop out )

True Experimental Designs: Pretest-Posttest Control Group Design

EG: R CG:R 01 03 X 02 04

Test units are randomly assigned to either the experimental or the control group. A pretreatment measure is taken on each group. The treatment effect (TE) is measured as:(02 - 01) - (04 - 03). Selection bias is eliminated by randomization.

Posttest-Only Control Group Design

EG : R X 01 CG : R 02
Ex: impact of Pvt tuitions on performance.

The treatment effect is obtained by TE = 01 - 02 Except for pre-measurement, the implementation of this design is very similar to that of the pretestposttest control group design. No pre measurements hence testing effect absent

Solomon Four Group Design

Experimental Group 1: Control Group 1: Experimental Group 2: Control Group 2: R O 1 X O2 R O3 O4 R X O5 R O6

Note: Use this design when it is suspected that, in taking a test more than once, earlier tests have an effect on later tests, for example by learning or priming effects.
For a reliable result, several sets of four tests should be applied and the means used.

 Quasi-experimental designs result when the researcher is unable to achieve full manipulation of scheduling or allocation of treatments to test units but can still apply part of the apparatus of true experimentation

example: To perform an educational experiment, a class might be arbitrarily divided by alphabetical selection or by seating arrangement.
time series designs. 01 02 03 04 05 X 06 07 08 09 010

There is no randomization of test units to treatments. The timing of treatment presentation, as well as which test units are exposed to the treatment, may not be within the researcher's control.

 A statistical design is a series of basic experiments that allows for statistical control and analysis of external variables: completely randomized design randomized block design Latin square design factorial designs.

Completely Randomized Design

Evaluating the effect of a single variable Effect of extraneous variable controlled using the randomization technique EG1: R EG2: R EG3: R X1 X2 X3 O1 O2 O3

Effect on sales of coffee when product is kept in three levels of shelf height (knee level, waist level and eye level).

Randomized Block Design

A statistical design in which the test units are blocked on the basis of external variable to ensure that various experimental and control groups are matched closely on that variable. Used when one extraneous variable may influence experimental results. Store size while measuring effect of promo on sales.

Test units are blocked or grouped based on the extraneous variables.

Factorial Design
A statistical experimental design that is used to measure the effect of two or more independent variables in various levels and allow the interaction between the variables. Two levels of k independent variables.

Two effects can be determined-the main effect and the interaction.

Latin square design

Two blocking effects i.e two non interacting extraneous variables Example: Impact of different pricing schemes on sales will be influenced by store patronage and POP display Always a square design levels of all factors are same Interaction effect cannot be measured

Thank You!