Lecture 2

Basic Concepts and Steps in Experimental

Designs
Also use the definitions and terminologies
from Lecture1 for basic concepts

Prof. Bhaswat Chakraborty

1
Experimental Process & Design
• An experiment is a process or study that results in the collection of data.
• Usually, statistical experiments are conducted in situations in which researchers
can manipulate the conditions of the experiment and can control the factors that
are irrelevant to the research objectives.
• For example, a rental car company compares the tread wear of four brands of tires,
while also controlling for the type of car, speed, road surface, weather, and driver.
• Experimental design is the process of planning a study to meet specified
objectives.
• Planning an experiment properly is very important in order to ensure that the
right type of data and a sufficient sample size and power are available to answer
the research questions.

SAS White Paper

2
General model of a process or system
Montgomery, Douglas C. (1997). Design and Analysis of Experiments (4th
edition). New York: John Wiley & Sons.
3
Designing an Experiment
• Perform the following steps when designing an experiment:
• Define the problem and the questions to be addressed
• Define the population of interest
• Determine the need for sampling
• Define the experimental design
• Write Down the Research Problem and Questions
• Before data collection begins, identify specific questions that is planned to be
answered
• Identify the sources of variability in the experimental conditions
• One of the main goals of a designed experiment is to partition the effects of the
sources of variability into distinct components in order to examine specific questions
of interest
SAS White Paper
4
Objectives of an Experiment
• Determining which variables are most influential on the response y
• Determining where to set the influential x’s so that y is almost always near the
desired nominal valu
• Determining where to set the influential x’s so that variability in y is small
• Determining where to set the influential x’s so that the effects of the
uncontrollable variables z1, z2, . . . , zq are minimized

• Usually, an objective of the experimenter is to determine the influence that these

factors have on the output response of the system
• The general approach to planning and conducting the experiment is called the
strategy of experimentation
Montgomery, Douglas C. (1997). Design and Analysis of Experiments (4th
edition). New York: John Wiley & Sons.
5
An Analogy Example of Cricket
• Some of the factors that may influence the score of your team, are as
follows:
• The type of pitch used (dry, bare, some grass, red soiul)
• Type of ball used
• Balance of skills and fitness of players
• Food and Drins
• Day or day/night play
• Weather
• Sometimes the spectators abd environment
• …

6
Basic Statistical Concepts
• Each of the individual observations in any experiment differ
• This noise is usually called experimental error or simply error or statistical error
meaning that it arises from variation that is uncontrolled and generally
unavoidable
• The presence of error or noise implies that the response variable is a random
variable
• A random variable may be either discrete or continuous. If the set of all possible
values of the random variable is either finite or countably infinite, then the
random variable is discrete, whereas if the set of all possible values of the
random variable is an interval, then the random variable is continuous
• Often variability is described by graphs like a dot diagram or a histogram
• The diagram enables the experimenter to see quickly the general location or
central tendency of the observations and their spread or variability
Montgomery, Douglas C. (1997). Design and Analysis of Experiments (4th
edition). New York: John Wiley & Sons. 7
 A Dot Diagram

A Histogram

Montgomery, Douglas C. (1997). Design and Analysis of Experiments (4th

edition). New York: John Wiley & Sons. 8
Probability Distributions
• The probability structure of a random variable, say y, is described by its
probability distribution
• If y is discrete, we often call the probability distribution of y, say p(y), the
probability mass function of y
• If y is continuous, the probability distribution of y, say f(y), is often called the
probability density function for y
• Figure in the next slide illustrates hypothetical discrete and continuous
probability distributions
• Notice that a) in the discrete probability distribution, it is the height of the
function p(yj ) that represents probability
• b) whereas in the continuous pd, it is the area under the curve f(y) associated
with a given interval that represents probability

Montgomery, Douglas C. (1997). Design and Analysis of Experiments (4th

edition). New York: John Wiley & Sons. 9
Montgomery, Douglas C. (1997). Design and Analysis of Experiments (4th
edition). New York: John Wiley & Sons. 10
Parameter and Statistic: Mean, Median & Variance
• A parameter is a numerical descriptive measure of a population; μ is an example of a parameter
• A statistic is a numerical descriptive measure of a sample; X is an example of a statistic
• To each sample statistic there corresponds a population parameter. We use or , s2, s , p, etc. to
estimate μ, σ2, σ, P (or π), etc.

11
Population Parametric Assumptions
• Parametric and nonparametric are two broad classifications of statistical
procedures
• Parametric statistics assume about the shape of the distribution in the
underlying population
• assume a normal, lognormal, Weibull distribution
• Also about the form or parameters of the assumed distribution
• means and standard deviations
• Nonparametric statistics rely on no or few assumptions about the shape or
parameters of the population distribution from which the samples were drawn
• If the data deviate strongly from parametric assumptions, using the parametric
procedure could lead to incorrect conclusions

12
Parametric
&
Analogous
Non-
parametric
Procedures

13
Understanding Sample Size Determination
H0: μ0 – μA = 0 HA : μ 0 – μ A = δ
Critical Value

S.Error =σ(√2/N) S.Error =σ(√2/N)

Power = 1-β
β

α/2 α/2

0 δ
X0–XA
0+Z1-α/2σ√(2/N) δ–Z1-βσ√(2/N)

14
From the previous graph, we have

0+Z1-α/2σ√(2/N) = δ–Z1-βσ√(2/N)

Upon simplification,

2 σ2 [Z1-α/2 + Z1-β/2]2
N=
δ2

15
Planning Statistical Analysis:
Answer those Five Key Questions
1. What is the main purpose of the trial?

2. What is the principal measure of patient outcome?

3. How will the data be analysed to detect a treatment difference?

4. What type of results does one anticipate with standard

treatment?

5. How small a treatment difference is it important to detect and

with what degree of certainty?

Stuart Pocock in Clinical Trials, Wiley Int.

16
Sample Size for a t Test
Input variables you will need
α
The Type I error probability for a two sided test.
n
For independent t-tests n is the number of experimental subjects. For pair test n is the number of
pairs.
power
For independent tests power is probability of correctly rejecting the null hypothesis of equal
population means
δ
A difference in population means
σ
For independent tests σ is the within group standard deviation. For paired designs it is the
standard deviation of difference in the response of matched pairs.
m
For independent tests m is the ratio of control to experimental patients. m is not defined for
paired studies.
17
Design of 2-product, 2-period, crossover studies

Period I W Period II
A
Sequence 1 Test S Reference
H
Subjects Randomizaion O
U
Sequence 2 Reference T
Test

18
Design of 4-period, Replicate Studies

W W W
PI PII PIII PIV
A A A
S S S
Sequence 1 T R T R
H H H
O O O
Subjects Randomizaion
U U U

Sequence 2 R T T T R T T
1 2 3

19
QUESTIONS?

20