1

Selecting Effective Designs

Presented by: Larry Scott
Process Technologies
Northville, Michigan
248-347-1522
Welcome to:
Graphics by Nark Anderson @ Stat-Ease !nc.
DOE Evaluation:
Agenda
Intro to DOE
Overall Strategy of DOE
Design Evaluation: Focus on
Fraction of Design Space (FDS)
Full Factorial Designs
Fractional Designs
CCD, Box Benken, D-Optimal, MLC
Simplex Designs (Formulations)
2
Process Process
Controllable Factors (X) Controllable Factors (X)
Responses (Y) Responses (Y)
Uncontrollable Variables (Z) Uncontrollable Variables (Z)
What is DOE: What is DOE: 6 6- -sigma enthusiast Y = f (x sigma enthusiast Y = f (x
i i
) )
DOE is:
A series of tests,
in which purposeful changes
are made to input factors,
so that you may identify causes
for significant changes
in the output responses.
History
Fisher & Yates: DOE concepts 1920s
Plackett & Burman designs 1940s
1st Textbook: Box, Hunter & Hunter - 1960
Optimal Designs: a- , d- 1960s
Douglas Montgomery resurgence 1976
DOE software commercially available 1980s
Six Sigma drives interest in DOE 1990s
Minimum Run Designs; Whitcomb & Oley 1999
Fraction of Design Space (FDS) Introduced - 2007
3
Phase: Screening
Designs: resolution IV 2
k-p
Phase: Characterization
Designs: 2
k
, resolution V 2
k-p
Phase: Optimization
Designs : CCD, BB, etc.
Phase: Verification
Designs: resolution III 2
k-p
yes
Strategy of Experimentation
Factor effects
and interactions
Response
Surface
methods
Curvature?
Confirm?
Known
Factors
Unknown
Factors
Screening
Backup
Celebrate!
no
no
yes
Trivial
many
Vital few
Presentation Intent
Applying design evaluations
techniques that ensure effective design
properties for YOUR experiments.
Concept of Power: an under utilized
tool in DOE, until NOW! until NOW!
+
Input Requirements for Design
Signal estimates
Noise (variance) estimates
Resolution: Main Effects & Interactions
Resource Availability
# of Model Coefficients
Signal / Noise
The golf ball represents the signal: .
The grass represents the noise: .
/ = 1/2
p = 8.6 %
/ = 1
p = 19.5 %
/ = 2
p = 57.2 %
5
System Signal
is reflected in the magnitude of the output.
is a function of the range or spread
of the input variables.
X1: Temperature - Low = 50 deg C
High = 100 deg C
X2: Pressure - Low = 10 psi
High = 15 psi
Input Requirements for Design
Signal estimates
Noise (variance) estimates
Resolution: Main Effects & Interactions
Resource Availability
# of Model Coefficients
6
Resolution
The ability to independently estimate
model coefficients.
Full: A, B, C, AB, AC, BC, ABC
III: ME + 2FI A + BC
IV: ME + 3FI A + BCD
2FI + 2FI AB + CD
V: ME + 4FI A + BCDE
2FI + 3FI AB + CDE
Input Requirements for Design
Signal estimates
Noise (variance) estimates
Resolution: Main Effects & 2FI (AB)
Resource Availability $$$$$$
# of Model Coefficients
Y = b
0
+ b
1
X
1
+ b
2
X
2
+ b
12
X
1
X
2
+
7
Selecting # of Design Points
Given how many factors (k) you want to
study and the number of coefficients (p) in
the model you select, the design will be
built as follows:
Model: estimation of all coefficients.
Lack-of-Fit: test how well model
represents actual behavior.
Replicates: estimate pure error.
Why these inputs ?????
POWER POWER !!!!!
The ability to find a factor effect!
{1 - Beta) 100%
8
Power Depends On:
The size of the difference y being measured.
the larger the difference the higher the power.
The size of the experimental error :
the smaller the higher the power.
The risk chosen:
the larger the higher the power.
The number of replicates:
the more runs the higher the power.
Choose design appropriate to the problem:
more orthogonal & larger designs have more power.
Agenda
Intro to DOE
Overall Strategy of DOE
Power using Fraction of Design Space
Full Factorial Designs: traditional (%)
Fractional Designs: FDS
CCD, Box Benken, D-Optimal, MLC
Simplex Designs (Formulations): FDS
9
Good Response Surface Designs
1. Allows chosen polynomial to be estimated
well.
2. Sufficient information to test for lack of fit.
Have more unique design points than
coefficients in model.
Replicates to estimate pure error.
3. Remain insensitive to outliers, influential
values and bias from model
misspecification.
4. Be robust to errors in the factor levels.
Good Response Surface Designs
5. Permit blocking and sequential
experimentation.
6. Provide a check on variance
assumptions, e.g., test that residuals are
N(0,
2
).
7. Generate useful information throughout
the region of interest, i.e., provide a
constant distribution of variance across
design space.
8. Do not contain an excessively large
number of runs.
10
RSM vs OFAT OFAT
-2 -1 0 1 2
30
45
60
75
90
Factor A
R
e
s
p
o
n
s
e
-2 -1 0 1 2
60
65
70
75
80
85
90
Factor B
Response
R
e
s
p
o
n
s
e
65
73
80
88
95
Response
-4
-2
0
2
4
-4
-2
0
2
4
Factor A
Factor B
R
e
s
p
o
n
s
e
LCD Case Study
The objective is to find the best
color/typeface combination to maximize
readability on a LCD video display terminal.
Response: Time (seconds)
11
1. Identify opportunity and define objective.
Find best color/typeface combination to
maximize readability.
2. State objective in terms of measurable
responses.
Response = Time in seconds.
a. Define the change ( y) that is important to
detect for each response. = 1 second
b. Estimate experimental error ( ) for each
response. = 1 second.
c. Use the signal to noise ratio (/ / / / = 1.0)
to estimate power.
DOE Process: LCD Case Study
3. Enter the factor names, levels and units.
continue>>
Building the Design
LCD Case Study
12
4. Enter the response name, units, y (1), (1)
and a / of 1 is calculated.
continue>>
Building the Design
LCD Case Study
Evaluating Power: LCD Case Study
Reading Time: = 1 sec, = 1 sec, / / / / = 1.0
Want power of at least 80% for effects of
interest!
19.5%
13
Power Depends On:
The size of the difference y being measured.
the larger the difference the higher the power.
The size of the experimental error :
the smaller the higher the power.
The risk chosen:
the larger the higher the power.
The number of replicates:
the more runs the higher the power.
Choose design appropriate to the problem:
more orthogonal & larger designs have more power.
4. Build a 2
3
factorial with 2 replicates or 16
runs.
Replicate to Increase Power:
44.6%
1+
5. Build a 2
3
factorial with 5 replicates or 40
runs.
Replicate to Increase Power
S6.6%
However, As Collinearity Increases
Calculating power becomes a less
effective tool.. So..
We rely on a new tool..
Fraction of Design Space (FDS)
Plus:
StdErr: Prediction error for the design.
15
Stent Delivery System
A stent is a wire mesh tube used to prop open an
artery recently cleared using angioplasty. The stent
is collapsed to a small diameter over a balloon
catheter. It's then moved into the area of the
blockage.
When the balloon is inflated, the stent expands,
locks in place and forms a scaffold. This holds the
artery open. The stent stays in the artery
permanently, holding it open to improve blood flow
to the heart muscle.
Stent Delivery System
This case study is meant to illustrate typical DOE
use in research to develop an improved product; in
this case a stent delivery system. Typical factors
include:
Lengths and diameters of various components,
e.g. tip, balloon, catheter, etc.
Materials used for the components.
Assembly parameters, e.g. weld locations, how
the balloon is folded, etc.
Stent geometry, wall thickness, how it is
crimped on the balloon, etc.
16
Stent Delivery System
MR-5 Factorial Design
Twelve factors (11 numeric and 1 categoric)
were studied in a factorial design. After
analysis of the design:
A CCD design with seven factors was
run.
The seven factors and the region of interest are:
(The actual factor names and levels are proprietary.)
Stent Delivery System
MR-5 CCD Design
Factor
Type Low Level
()
High Level
(+)
A numeric 1 +1
B numeric 1 +1
C numeric 1 +1
D numeric 1 +1
E numeric 1 +1
F numeric 1 +1
G numeric 1 +1
17
Stent Delivery System
MR-5 CCD Design
Possible design choices:
2
7
CCD (143 builds and 152 runs)
2
7-1
CCD (79 builds and 88 runs)
MR-5 CCD (47 builds and 50 runs)
Small CCD (37 builds and 41 runs)
BB (57 builds and 62 runs)
Stent Delivery System
MR-5 CCD Design
Use Design-Expert to build a 7-factor MR-5 CCD
design.
18
MR-5 Designs
Provide Considerable Savings
466 1024 30 106 256 14
326 1024 25 92 256 13
232 512 21 80 256 12
212 512 20 68 128 11
192 512 19 56 128 10
172 512 18 46 128 9
154 256 17 38 64 8
138 256 16 30 64 7
122 256 15 22 32 6
MR5 2
k-p
k MR5 2
k-p
k
Check Size of Design
Confirm that the design has enough runs to
give the results needed.
Desired Precision
Process Sigma
Alpha risk
This tool bar allows the user
to estimate StdErr Mean for a
specific design based on
these inputs:
19
Check Size of Design
80% of the design space will be able to estimate the
response with the needed precision.
Fraction of Design Space
S
t
d
E
r
r
M
e
a
n
FDS Graph
0.00 0.25 0.50 0.75 1.00
0.000
0.475
0.950
1.425
1.899
0.62
Design-Expert Software
Min StdErr Mean: 0.302
Max StdErr Mean: 1.899
Cuboidal
radius = 1
Points = 10000
t(0.05/2,14) = 2.14479
Reference X= 0.80
Reference Y = 0.619
FDS Graph
Fraction of Design Space
S
t
d
E
r
r

M
e
a
n
0.00 0.25 0.50 0.75 1.00
0.000
0.475
0.950
1.425
1.899
Software Screenshot
Fraction of Design Space
S
t
d
E
r
r
M
e
a
n
20
Remember, As Collinearity Increases
Power calculations becomes a less
effective tool..
So, in Simplex designs we rely
extensively on ..
Fraction of Design Space (FDS)
calculations.
Simplex-Lattice Designs
X = 1 1
X = 1
3 X = 1
2
{3, 3} Simplex Lattice {3,3} Simplex Lattice
augmented & 4 reps
,
X = 1
1
X = 1
3 X = 1
2
21
Compare to Orthogonal Designs:
2
3
factorial with 3 level factorial
center point
Design Evaluation
Two Simplex-Lattice Designs
X = 1 1
X = 1
3 X = 1
2
{3, 3} Simplex Lattice {3,3} Simplex Lattice
augmented & 4 reps
,
X = 1
1
X = 1
3 X = 1
2
22
Design Matrix Evaluation for Mixture
Special Cubic Model
No aliases found
Degrees of Freedom for Evaluation
Model 6
Residuals 3
Lack 0f Fit 3
Pure Error 0
Corr Total 9
Simplex-Lattice
Without Augmentation or Replicates
Minimal
Poor
Design Matrix Evaluation for Mixture
Special Cubic Model
No aliases found
Degrees of Freedom for Evaluation
Model 6
Residuals 10
Lack 0f Fit 6
Pure Error 4
Corr Total 16
Simplex-Lattice
With Augmentation and 4 Replicates
Good
Better
23
Term 0.5 Std. Dev. 1 Std. Dev. 2 Std. Dev.
A 6.2 % 9.8 % 23.8 %
B 6.2 % 9.8 % 23.8 %
C 6.2 % 9.8 % 23.8 %
AB 6.2 % 10.0 % 24.5 %
AC 6.2 % 10.0 % 24.5 %
BC 6.2 % 10.0 % 24.5 %
ABC 6.0 % 9.2 % 21.6 %
Simplex-Lattice
Without Augmentation or Replicates
Power at 5 % alpha level to detect signalJnoise ratios of:
A 8.6 % 19.9 % 60.1 %
B 8.6 % 19.9 % 60.1 %
C 8.6 % 19.9 % 60.1 %
AB 7.6 % 15.6 % 46.7 %
AC 7.6 % 15.6 % 46.7 %
BC 7.6 % 15.6 % 46.7 %
ABC 7.8 % 16.5 % 49.6 %
Simplex-Lattice
With Augmentation and 4 Replicates
Power at 5 % alpha level to detect signalJnoise ratios of:
Term 0.5 Std. Dev. 1 Std. Dev. 2 Std. Dev.
2+
Fraction of Design Space
FDS:
Calculates the volume of the design space
having a standard error (StdErr) less than or
equal to a specified value.
The ratio of this volume to the total volume of
the design volume is the fraction of design
space.
Produces a single plot showing the cumulative
fraction of the design space on the x-axis
(from zero to one) versus the StdErr on the y-
axis.
Simplex-Lattice
Without Augmentation or Replicates
A: A
1.00
B: B
1.00
C: C
1.00
0.00 0.00
0.00
0.63 0.63
0.63
0.78
0.78
0.78
0.78
0.78 0.78
0.78
S
t
d
E
r
r
M
e
a
n
0.00 0.25 0.50 0.75 1.00
0.00
0.25
0.50
0.75
1.00
0.63
0.78
Fraction of Design Space
StdErr of Design
25
Simplex-Lattice
With Augmentation and 4 Replicates
A: A
1.00
B: B
1.00
C: C
1.00
0.00 0.00
0.00
StdErr of Design
0.47
0.47
0.47
0.57
0.57
0.57
0.57 0.57
0.57
0.57
2
2
2 2
Fraction of Design Space
S
t
d
E
r
r
M
e
a
n
0.00 0.25 0.50 0.75 1.00
0.00
0.25
0.50
0.75
1.00
0.47
0.57
StdErr of Design
Simplex-Lattice: 3D
Without With
A (1.000)
B (0.000)
C (1.000)
C (0.000)
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1


S
t
d
E
r
r

o
f

D
e
s
i
g
n


A (0.000)
B (1.000)
S
t
d
E
r
r
A (1.000)
B (0.000)
C (1.000)
C (0.000)
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1


S
t
d
E
r
r

o
f

D
e
s
i
g
n


A (0.000)
B (1.000)
S
t
d
E
r
r
26
Evaluating Mixture Designs
All mixture DOEs have collinearity due to the equality
constraint, ie. proportions.. X1 + X2 + X3 = 100%
As other constraints are added collinearity worsens.
The fall out is:
Power depend on orthogonality and therefore
looses value as collinearity increases.
Standard error plots and FDS are better design
evaluation tools in the presence of collinearity.
A: T EA-L S
28 . 0 0
B: Co ca mi d e
9 . 0 0
C: L au ra mi d e
9 . 0 0
1 .0 0 1 .0 0
20 . 0 0
Height
140
150
160
160
160
170
176
22
22
22
22
22
P re d i c t i 1 7 6 . 9 3
9 5 % L o 1 6 1 . 9 2
9 5 % H i 1 9 1 . 9 4
S E m e a 2 . 8 0 6 5 4
S E p re d 6 . 8 8 8
X 1 2 4 . 4 7
X 2 3 . 4 9
X 3 2 . 0 4
Simplex: Simple Constraints
27
90
50
70
30
10
9
0
5
0
7
0
3
0
1
0
9
0
5
0
7
0
3
0
1
0
X
1
X
2
X
3
1. 0.1 A
2. A 0.5
3
.


0
.
1

B
4
.


B

0
.
7
5. C 0.7
Simplex: Multi-component Constraints
90
50
70
30
10
9
0
5
0
7
0
3
0
1
0
9
0
5
0
7
0
3
0
1
0
X
1
X
2
X
3
1
2
3
4
5
8
6
7
Additional Extreme Vertices
28
DOE Software
N
e
w

i
n

2
0
0
7
1) Readily generate many design options
2) Quick, powerful data analysis
3) Generate useful response surface models
4) Design evaluation include:
Power: percentage, contour & FDS plots FDS plots
StdErr estimates for:
Mean optimization
Point prediction
Estimating differences
If you always do what you always did;
youll always get what you always got.
- wise but unknown philosopher
Old Habits Methods Die Hard
29
www.doetraining.com
Software: Design-Expert
from Stat-Ease, Inc.
Review this Presentation @...