Experiment design

Chapter 4 Experimental design 4.1 Overview A designed experiment is a test or series of tests in which purposeful changes are made to the input variables of a process so that the resulted changes in the output variables (responses) may be observed and identified. Many of the earlier applications of experimental design were in the agriculture and biological sciences. It has found broad applications in many disciplines now. Consider the tension bond strength of cement mortar which is an important characteristic of ready-mixed cement. An engineer is interested in comparing the strength of a modified formulation in which polymer latex emulsions (referred to as a factor) have been added during the mixing to the strength of unmodified formulation. The engineer will collect observations on tested items selected these two formulations. The two different formulations are referred to as treatments. A two-sample t-test can be used to test whether the modified one on the average has larger tension bond strength. If we have more than two formulations to compare the statistical techniques ANOVA we discussed earlier may be used to test the difference in the bond strength of different formulations. Formally, this involves several steps in the experimental process. 1. Statement of the problem objectives 2. Factors choose the factors to be varied in the experiment, and the specific levels at which runs will be made. Need practical experience and theoretical understanding. 3. Selection of response variable provides useful information about the process under study. 4. Choice of experimental design sample size, replications, run order, blocking, randomization. Keep design and analysis as simple as possible. Should consider costs and other resources. 5. Performing the experiment 6. Data analysis appropriate statistical methods, results and conclusions should be objective rather than judgmental in nature. 7. Conclusions and recommendations practical conclusions about results and recommendations for action.

Experiment design

4.2 Factorial experiments Factorial experiments are employed to study simultaneously the effects of two or more factors, each covers a pre-determined number of levels. By a factorial design we mean that in each complete trial or replicate of the experiment all possible combinations of the levels of the factors are investigated. Thus if there are a levels of factor A and b levels of factor B, then each replicate contains all ab treatment combinations. We shall have a twoway or higher-way classification experiment. The effect of a factor is defined as the change in response produced by a change in the level of the factor. This is called a main effect because it refers to the primary factor under study. In some experiments, the difference in response between the levels of one factor is not the same at all levels of the other factors. When this occurs there is an interaction effect between the factors. Factorial designs are more efficient than one-factor-at-a-time experiments. A factorial design is necessary when interactions are present. 4.2.1 Two-factor factorial design Let these factors be A and B. Each observation will be denoted by xijt . Formally, we assume

xijt = + Ai + B j + ( AB)ij + ijt

i refers to factor A (a treatments) j refers to factor B (b treatments) t refers to the replications (n replicates) Ai refers to the main effect of A Bj refers to the main effect of B (AB)ij refers to the interaction effect of A and B ijt are independent N(0, 2)

Experiment design

The treatment sums will be

a factor A treatment sums = X i.. , (nA = nb observations each) b factor B treatment sums = X . j . , (nB = na observations each)

ab factor AB combined treatment sums = X ij . , (n observations each)

Total treatment = X ... , (N = nab observations) Treatment A SS = nb ( xi.. x... ) 2 =
i =1
b

2 X i2 X ... .. = SSA N i =1 n A a

Treatment B SS = na ( x. j . x... ) 2 =
j =1

n
j =1 a

X .2j .
B b

2 X ... = SSB N n 2 ijt

Total SS

( xijt x... )2 =
i =1 j =1 t =1

x
i =1 j =1 t =1

2 X ... N

When factors A and B interact the interaction variation is: 2 a b X2 a b X ... ij . 2 - SSA - SSB AB interaction SS = n ( xij . xi.. x. j . + x... ) = N i =1 j =1 n i =1 j =1
The total SS is decomposed as Total SS = Treatment A SS + Treatment B SS + AB Interaction SS + Error SS Error SS is obtained by subtraction. As before, Error SS =
2 2 2 ( xijt xi.. )2 ~ 2 n21 = 2 ab ( n 1) = N ab i =1 j =1 t =1 i =1 j =1 a b n a b

E(Error SS) = (N - ab)

Here we do not require H0. The error SS always has a 2 - distribution whether or not H0 is true. Under H0, Treatment A SS = nb ( xi.. x... ) 2 ~ 2 a 1 Treatment B SS = na ( x. j . x... ) 2 ~ 2 b 1
j =1 a

i =1 b

AB interaction SS = n ( xij . xi.. x. j . + x... ) 2 ~ 2 ( a 1)( b 1)

i =1 j =1

Experiment design

Mean squared SS

s 2 = Error SS / (N-ab) 2 sA = Factor A SS / (a-1) 2 sB = Factor B SS / (b-1) 2 s AB = Interaction SS / (a-1) (b-1)

Whether or not H0 is true E( s 2 ) = 2 , thus s 2 is unbiased for 2.
2 sA ~ Fa 1, N ab s2 2 sA ~ Fb 1, N ab s2 2 s AB ~ F( a 1)(b 1), N ab s2

4.2.2 ANOVA Table

Source of variation
a

Sum of squares
2 X i2 X ... .. N i =1 n A

Degree of freedom

Mean square

F-ratio
2 sA s2 2 sA s2 2 s AB s2

Treatments A Treatments B AB Interaction Random variation or Error Total variation

a-1
b-1
(a-1)(b-1) By subtraction = ab(n-1) = N- ab

2 A

j =1
a

X .2j . nB
b

2 X ... N

2 sB

i =1 j =1

2 X ij .

2 X ... - SSA - SSB N

2 s AB

s2

By subtraction

x
i =1 j =1 t =1

2 ijt

2 X ... N

N-1

Note: When the interaction effect is not significant or when there are no replications (i.e. n = 1), we may omit the row of interaction. In the former situation, the random SS will be the original random SS plus the AB interaction SS and the d.f. will accordingly be increased. In the latter, no interaction SS can be calculated due to no replications.
The randomized block design discussed earlier may be regarded as a two-factor (one factor is plot and the other is fertilizer) experiment without replication. The interaction effect cannot be separated from the error SS.

Experiment design

Example
A department store conducted an experiment to study the sales of a product for different package designs and different selling prices. There were three different package designs at two different selling prices. (This is equivalent to a randomized block design with two replications. Package is the block and price is the treatment.)

Package design 1 2 3

Selling price (Higher) 14 12 12 16 37 41

Selling price (Lower) 17 13 11 9 35 41

They want to test: 1. The hypothesis that there are no interactions for any combination of package design and a selling price. 2. The hypothesis that the mean population amount of sales is the same for three package designs. 3. The hypothesis that the mean population sales amount is the same for two selling prices. 4. Locate where the difference are if the hypothesis is significant. Totals: the values inside table are interaction totals X ij . the row and column totals are factor totals X i.. , X . j . the overall total is grand treatment total X ... Package Selling price design (Higher) 1 26 2 28 3 78 132 X i.. a = 2, b = 3, n =2, N = 12, Selling price (Lower) 30 20 76 126

X . j.
56 48 154 258

Total SS = 142 + 122 + 172 + 132 + 372 + 412 + 352 + 412 + 122 + 162 + 112 + 92 - 2582/12 = 1809 Package SS = 562/4 + 1542/4 + 482/4 - 2582/12 = 1742 Price SS = 1322/6 + 1262/6 + 482/4 - 2582/12 = 3 Interaction SS = 262/2 + 302/2 + 782/2 + 762/2 + 282/2 + 202/2 - 2582/12 - 1742 3 = 18

Experiment design

ANOVA Table
Source of variation Package Price Package x Price Error Total Sum of squares 1742 3 18 46 1809 Degree of freedom 2 1 2 6 11 Mean square 871 3 9 7.67 F-ratio 113.6 0.391 1.173 F-critical at 5% 5.143 5.987 5.143

Conclusion At 5% level, the difference in package design effect on the mean sales is significant, whereas the difference in price effect and interaction effect of price and package design are not significant.

Experiment design

Below is computer output from EXCEL Anova: Two-Factor With Replication

SUMMARY
Package design 1

Selling price (Higher) 2 26 13 2

Selling price (Lower) 2 30 15 8

Total 4 56 14 4.666667

Count Sum Average Variance

Package design 2

Count Sum Average Variance

Package design 3

2 78 39 8

2 76 38 18

4 154 38.5 9

Count Sum Average Variance

Total

2 28 14 8

2 20 10 2

4 48 12 8.666667

Count Sum Average Variance ANOVA Source of Variation Sample Columns Interaction Within Total

6 132 22 177.2

6 126 21 184

SS 1742 3 18 46 1809

df 2 1 2 6 11

MS 871 3 9 7.666667

F 113.6087 0.391304 1.173913

P-value 1.7E-05 0.554646 0.371307

F crit 5.143249 5.987374 5.143249

Experiment design

4.3 Example Three factor design

An experiment is conducted to determine the thrust forces in drilling at different speeds, feeds and in different materials. Five speeds are used, three feeds and two materials with two samples tested under each set of conditions. The order of the experiment is completely randomized and the levels of all factors are fixed. Data in coded units are obtained as follows: Speed 475 108 60 248 295 543 450 122 85 318 330 810 750

Material B

Feed .004 .008 .014

.004 .008 .014

100 122 110 332 330 640 500 196 170 386 365 810 725

220 108 85 276 310 612 500 136 130 333 330 779 670

715 66 50 248 275 612 610 108 75 472 350 893 890

870 80 60 276 310 696 610 136 75 499 390 1820 890

4.3.1 Three-factor Interaction totals, Speed totals

Speed 475 168 543 993 207 648 1560 4119 Row total 849 2900 5773 1233 3773 9037 23565

Material B

Feed .004 .008 .014 .004 .008 .014

Speed total

100 232 662 1140 366 751 1535 4686

220 193 586 1112 266 663 1449 4269

715 116 523 1222 183 822 1783 4649

870 140 586 1306 211 889 2710 5842

The above is similar to a three-factorial design without replications.

Experiment design

4.3.2 Two-factor Interaction totals and Factor totals Speed x Material

Material B V Speed total 100 2034 2652 4686 220 1891 2378 4269 Speed 475 1704 2415 4119 715 1861 2788 4649 870 2032 3810 5842 Material total 9522 14043 23565

Speed x Feed
Feed .004 .008 .014 Speed total 100 598 1413 2675 4686 220 459 1249 2561 4269 Speed 475 375 1191 2553 4119 715 299 1345 3005 4649 870 351 1475 4016 5842 Row total 2081 6673 14810 23565

Material x Feed
Material B V Feed total .004 849 1233 2081 .008 2900 3773 6673 Feed .014 5773 9037 14810 Material total 9522 14043 23565 = 5831591.25

Total SS = 1222 + 1102 + 1082 + + 18202 + 8902 - 235652 / 60

Speed SS = 46862/12 + 42692/12 + 41192/12 + 46492/12 + 58422/12 - 235652 / 60 = 152453.17 Material SS = 95222/30 + 140432/30 - 235652 / 60 Feed SS = 20822/20 + 66732/20 + 148102/20 - 235652 / 60 SxM SS = 20342/6 + 18912/6 + + 27882/6 + 38102/6 - 235652/ 60 - 152453.17 - 340657.35 SxF SS = 5982/4 + 4592/4 + + 30052/4 + 40162/4 - 235652 / 60 - 152453.17 - 4154833.9 MxF SS = 8492/10 + 29002/10 + + 37732/10 + 90372/10 - 235652 / 60 - 340657.35 - 4154833.9 = 340657.35 = 4154833.9

= 88111.56

= 255471.43

= 237506.7

Experiment design

SxMxF SS = 2322/2 + 1932/2 + + 17832/2 + 27102/2 - 235652 / 60 - 152453.17 - 340657.35 -4154833.9 - 88111.56 - 255471.43 - 237506.7

= 113084.64

Higher Interaction SS = sum of interaction total squares/replication number - Total correction SS All main factor SS and lower interaction SS

4.3.3 ANOVA Table

Source Speed Material Feed SxM SxF MxF SxMxF Error Total SS 152453.17 340657.35 4154833.9 88111.56 255471.43 237506.7 113084.64 489472.5 5831591.25 DF 4 1 2 4 8 2 8 30 59 MS 38113.3 340657.35 2077416.95 22027.89 31933.9 118753.35 14135.58 16315.75 F 2.336 20.879 127.326 1.350 1.957 7.278 0.866 Remark * at 0.001 * at 0.001

* at 0.01

Since the three factor interaction effect is not significant, we may assume no three factor interaction so that its SS is combined with Error SS to form a new Error SS with 38 degrees of freedom. ANOVA Table without three factor interaction Source Speed Material Feed SxM SxF MxF Error Total SS 152453.17 340657.35 4154833.9 88111.56 255471.43 237506.7 602557.14 5831591.25 DF 4 1 2 4 8 2 38 59 MS 38113.3 340657.35 2077416.95 22027.89 31933.9 118753.35 15856.77 F 2.404 21.483 131.011 1.389 2.0139 7.489

The conclusions are not affected. The latter is a more powerful test because it has larger d.f. for the Error SS.

10

Experiment design

4.3.4 Main effects

Look at the treatment means of Feed at different levels of the other factors. Main effect of Feed at different levels of Speed Speed Feed 100 220 475 .004 149.5 114.75 93.75 .008 353.25 312.25 297.75 .014 668.75 640.25 638.25

715 74.75 336.25 751.25

870 87.75 368.75 1004

Main effect of Feed at different levels of Material Feed Material .004 .008 .014 B 84.9 290 577.3 V 123.3 377.3 903.7

11

Experiment design

Interaction Plot of Feed at Levels of Speed

1050 950 850 750

S1 S2 S3 S4 S5

Response

650 550 450 350 250 150 50 F1 F2 F3

No interaction effect means lines have the same slope

Feed

Interaction Plot of Feed at Levels of Material

880 780 680

Response

580

B
480 380 280 180 80 F1 F2 F3

Significant interaction effects means lines have different slopes

Feed

12

Experiment design

Computer output from MINITAB General Linear Model: Thrust versus Speed, Feed, Material
Factor Speed Feed Material Type fixed fixed fixed Levels 5 3 2 Values 100, 220, 475, 715, 870 0.004, 0.008, 0.014 B, V

Analysis of Variance for Thrust, using Adjusted SS for Tests Source Speed Feed Material Speed*Feed Speed*Material Feed*Material Speed*Feed*Material Error Total DF 4 2 1 8 4 2 8 30 59 Seq SS 152453 4154834 340657 255471 88112 237507 113085 489473 5831591 Adj SS 152453 4154834 340657 255471 88112 237507 113085 489473 Adj MS 38113 2077417 340657 31934 22028 118753 14136 16316 F 2.34 127.33 20.88 1.96 1.35 7.28 0.87 P 0.078 0.000 0.000 0.088 0.275 0.003 0.555

S = 127.733

R-Sq = 91.61%

R-Sq(adj) = 83.49%

Unusual Observations for Thrust Obs 55 60 Thrust 1820.00 890.00 Fit 1355.00 1355.00 SE Fit 90.32 90.32 Residual 465.00 -465.00 St Resid 5.15 R -5.15 R

R denotes an observation with a large standardized residual.

13

Experiment design

Main Effect Plot

800 600 400 Speed Feed

Mean of Thrust

200 100 800 600 400 200 B V 220 475 Material 715 870 0.004 0.008 0.014

Interaction Plot
0.004 1000 0.008 0.014 S peed 100 220 475 715 870 1000 F eed 0.004 0.008 0.014

500

Speed

Feed

500

1000

0 M aterial B V M ater ial

500

0 100 220 475 715 870 B V

14

Experiment design

4.4 Latin Square

In randomized block design we attempt to eliminate a source of error by applying each of our treatments just once in each block. In a Latin square we try to eliminate two sorts of undesired variability from our treatment comparisons. These are called rows and columns. Again each treatment occurs just once within each row and just once within each column. A Latin square of side n or n x n Latin square is an arrangement of n letters, each repeated n times, in a square array of side n in such a manner that each letter appears exactly once in each row and in each column. Examples of Latin squares: 4x4 B C D A 5x5 C D D E E A A B B C 5x5 C E B D A

A B C D

D A B C

C D A B

A B C D E

B C D E A

E A B C D

A C E B D

B D A C E

D A C E B

E B D A C

Model
xijt = + Ri + C j + Tt + ijt , i, j, t = 1, 2, , n. Ri : ith row effect Cj : jth column effect Tt : tth treatment effect

Assumptions

R C
i

=0 =0 =0

T
t

ijt independently ~ N (0, 2 )

15

Experiment design

4.3.1 Sum of squares

The total sum of squares can be decomposed into components attributable to different effects.

( x
i =1 j =1

ijt

x... ) 2 = n ( xi.. x... ) 2 + n ( x. j . x... ) 2 + n ( x..t x... ) 2

i =1 j =1 t =1

( x
i =1 j =1

ijt

xi.. x. j . x..t + 2 x... ) 2

Total SS = Row SS + Column SS + Treatment SS + Error SS

4.3.2 Computational formulae

N = n2
Row total Column total Treatment total Grand total = X i.. = xijt
j =1 n n

= X . j . = xijt
i =1 n

= X ..t =

i or j =1 j n

xijt

= X ... = xijt
i =1 j =1

Total SS Row SS Column SS Treatment SS

2 X ... = xijt N i =1 j =1
n n

d.f. = N - 1 d.f. = n - 1 d.f. = n - 1 d.f. = n 1

X i2 .. i =1 n n X2 = . j. j =1 n
n

2 X ... N

2 X ... N

2 X ..2t X ... N t =1 n n

16

Experiment design

4.3.3 ANOVA Table

Source Treatments Rows Columns Error Total SS 2 X ..2t X ... N t =1 n 2 2 n X i.. X ... N i =1 n 2 2 n X X ... . j. N j =1 n By subtraction
n

DF n1 n1 n1 (n 1)( n 2) N1

MS st2
2 sR

2 sC

F st2 / s 2 to test H0: Tt = 0 2 sR / s 2 to test H0: Ri = 0 2 sC / s 2 to test H 0 : Cj = 0

s2

2 xijt i =1 j =1

2 X ... N

E( s 2 ) = 2 whether of not H0 are true. Thus s 2 is unbiased for 2.

Example
An experimenter is studying the effect of 5 different formulations of an explosive mixture in the manufacture of dynamite on the observed explosive force. Each formulation is mixed from a batch of raw materials that is only enough for five formulations to be tested. Furthermore, the formulations are prepared by several operators. The appropriate design for this problem consists of testing each formulation exactly once in each batch of raw material and for each formulation to be prepared exactly once by each of five operators. A Latin square design is resulted for dynamite formulation Operator 3 C=19 D=30 E=26 A=26 B=20 121

Batch 1 2 3 4 5 Column Total

1 A=24 B=17 C=18 D=26 E=22 107

2 B=20 C=24 D=38 E=31 A=30 143

4 D=24 E=27 A=27 B=23 C=29 130

5 E=24 A36 B21 C22 D31 134

Row total 111 134 130 128 132 635

A, B, C, D and E are the five formulations. Treatment totals are A = 143, B = 101, C = 112, D = 149, E = 130

17

Experiment design

Operator SS Batch SS Formulation SS Total SS

= 1072/5 + 1432/5 + 1212/5 + 1302/5 + 1342/5 - 6352/5 = 1112/5 + 1342/5 + 1302/5 + 1282/5 + 1322/5 - 6352/5 = 1432/5 + 1012/5 + 1122/5 + 1492/5 + 1302/5 - 6352/5 = 242 + 172 + + 312 - 6352/5

ANOVA Table
Source Formulations Batches Operators Error Total SS 330 68 150 128 676 DF 4 4 4 12 24 MS 82.50 17.00 37.50 10.67 F 7.73

Conclusion: Different formulations have different explosive forces. Though we may test the batches and operator effects these usually are not of prime interest.

18

Experiment design

Practical Exercise
Question 1. (Training.xls) The personnel manager of a large insurance company wished to evaluate the effectiveness of four different sales-training programs designed for new employees. A group of 32 recently hired college graduates were randomly assigned to four programs so that there were eight subjects in each program. At the end of the month-long training period a standard exam was administered to the 32 subjects; the scores are given below: Programs A 66 74 82 75 73 97 87 78 B 72 51 59 62 74 64 78 63 C 61 60 57 60 81 55 70 71 D 63 61 76 84 58 65 69 80

Determine whether there is evidence of a difference in the four sales-training programs at the 0.05 level of significance. Question 2. (Cholesterol.xls) The following are the cholesterol contents, in milligrams per package, which four laboratories obtained for 6-ounce packages of three very similar diet foods: Diet food A 3.4 3.0 3.3 3.5 Diet food B 2.6 2.7 3.0 3.1 Diet food C 2.8 3.1 3.4 3.7

Laboratory 1 2 3 4

Test whether the cholesterol contents differ among the 3 diet foods at the 0.05 level of significance.

19

Experiment design

Question 3. (Effluent.xls) Ozonization as a secondary treatment for effluent, following absorption by ferrous chloride, was studied for three reaction times and three pH levels. The study yielded the following results for effluent decline. Reaction time (min) 20 pH level 7.0 23 21 22 20 22 19 21 20 19 pH level 9.0 16 18 15 14 13 12 13 12 12 pH level 10.5 14 13 16 12 11 10 11 13 12

40

60

a) Test for significant interaction. b) Test for significant differences among reaction times. c) Test for significant differences among pH levels.

20