Chapter 14 Experimental Design Part B

Sample Size Sampling Error

Agenda
Describe and illustrate the impact of sample size on treatment effects

Sample Size
How many subjects to enroll? Need a sample size that offers sufficient power or precision Too few subjects introduces ethical concerns Considering
Variable, test statistic, hypotheses, withdrawals, protocol violations, and subjects lost to follow-up

Precision
Magnitude of the treatment effect
95% or greater confidence interval Are sample sizes the same across groups Assumptions or normality Uncertainty

Sample Size
Precision

Sampling Error
Sampling error occurs when using samples by which to make inferences about populations from which they are drawn
Random error
Pattern of errors that tend to cancel one another out so that the overall result still accurately reflects the true value Every sample design generates random error

Bias
Pattern of errors loaded in one direction or another that dont balance each other out True distortion

 If the noise is smaller, it is easier to detect a given signal Can achieve with a smaller s or larger n

Reducing Sampling Error

Reducing Sampling Error

Replication Three statistical design procedures aimed at minimizing effects of sampling error
Replication Apply each treatment to multiple experimental subjects Balance between the groups Blocking
Grouped by site yielding multiple repetitions in different setting

Repeat experiment multiple times, if possible

Blocking
An experimental design strategy to account for extraneous variation Blocks are strata (characteristics) Paired design is an example of blocking

Sample Size
Achieve a predetermined level of precision of an estimate of treatment effect Achieve a predetermined power of a test of the null hypothesis of no treatment effect
Goal

11

Sample Size
Aim for .8 power
Power

If H0 is false in 80% of experiments, the other 20% would fail to reject H0 even though it is false Power calculations assume we know
Often, we dont know, so use caution

For a given power and significance level, a larger sample size is needed when the within groups is large or if the minimum difference that we wish to detect is small

Sample Size
Lost Data

Plan in advance
Lost to follow-up Withdrawal Deaths of subjects/animals Protocol violations

13

Experimental Design
Determining Sample Size
Critical value

Experimental Design
Determining Sample Size
Find minimum n to get 80% power in a test or a desired level of precision (confidence interval)

Must make some assumptions about your data

Can use existing literature for effect size and standard deviation

Determining Sample Size

The formula to calculate minimum sample size depends on the type of test you are going to run Check text pages 381384 Proportions Log-odds ratio Difference between two proportions CI of the mean Difference between two means Binomial tests Contingency test Paired t-test Two sample t-test

Experimental Design
One sample or paired t-test

Sample Size 80% Power to reject H0

H0 : 1 = 2

vs. HA : 1 2
2

Minimum per group 16 where D = 1 2

Experimental Design
Sample Size Precision for 95% Confidence Interval of a mean

X (uncertainty)
Minimum per group 4 uncertainty
2

19

Experimental Design
Summary Reduce bias: Control group Randomization Blinding Reduce sampling error: Replication Balance Blocking

Questions

21