What is Design of Experiment ?

This branch of applied statistics deals with planning, conducting, analyzing and interpreting controlled tests to
evaluate the factors that control the value of a parameter or group of parameters.
A strategically planned and executed experiment may provide a great deal of information about the effect on a
response variable due to one or more factors. Many experiments involve holding certain factors constant and altering
the levels of another variable. This OneFactorataTime (or OFAT) approach to process knowledge is, however,
inefficient when compared with changing factor levels simultaneously.
Many of the current statistical approaches to designed experiments originate from the work of R. A. Fisher in the early
part of the 20th century. Fisher demonstrated how taking the time to seriously consider the design and execution of
an experiment before trying it helped avoid frequently encountered problems in analysis. Key concepts in creating a
designed experiment include blocking, randomization and replication.
A wellperformed experiment may provide answers to questions such as:

What are the key factors in a process?

At what settings would the process deliver acceptable performance?

What are the key, main and interaction effects in the process?

What settings would bring about less variation in the output?

A repetitive approach to gaining knowledge is encouraged, typically involving these consecutive steps:

1.
2.
3.

A screening design which narrows the field of variables under assessment.

A full factorial design which studies the response of every combination of factors and factor levels, and
an attempt to zone in on a region of values where the process is close to optimization.
A response surface design to model the response.
Blocking: When randomizing a factor is impossible or too costly, blocking lets you restrict randomization by carrying
out all of the trials with one setting of the factor and then all the trials with the other setting.
Randomization: Refers to t he order in which the trials of an experiment are performed. A randomized sequence
helps eliminate effects of unknown or uncontrolled variables.
Replication: Repetition of a complete experimental treatment, including the setup.

Design of experiments (DOE) is a powerful tool that can be used in a variety of experimental situations. DOE allows
for multiple input factors to be manipulated determining their effect on a desired output (response). By manipulating
multiple inputs at the same time, DOE can identify important interactions that may be missed when experimenting
with one factor at a time. All possible combinations can be investigated (full factorial) or only a portion of the possible
combinations (fractional factorial). Fractional factorials will not be discussed here.
When to Use DOE
Use DOE when more than one input factor is suspected of influencing an output. For example, it may be desirable to
understand the effect of temperature and pressure on the strength of a glue bond.
DOE can also be used to confirm suspected input/output relationships and to develop a predictive equation suitable
for performing what-if analysis.
DOE Procedure
1.

Acquire a full understanding of the inputs and outputs being investigated. A process flow diagram or
process map can be helpful. Utilize subject matter experts as necessary.

2.

Determine the appropriate measure for the output. A variable measure is preferable. Attribute measures
(pass/fail) should be avoided. Ensure the measurement system is stable and repeatable.

3.

Create a design matrix for the factors being investigated. The design matrix will show all possible
combinations of high and low levels for each input factor. These high and low levels can be generically coded as
+1 and -1. For example, a 2 factor experiment will require 4 experimental runs:

Input A Level

Input B Level

Experiment #1

-1

-1

Experiment #2

-1

+1

Experiment #3

+1

-1

Experiment #4

+1

+1

4.
Note: The required number of experimental runs can be calculated using the formula 2 where n is the number of
factors.
n

5.

For each input, determine the extreme but realistic high and low levels you wish to investigate. In some
cases the extreme levels may be beyond what is currently in use. The extreme levels selected should be realistic,
not absurd. For example:

Temperature

Pressure

-1 Level

+1 Level

100 degrees

200 degrees

50 psi

100 psi

6.
7.

Enter the factors and levels for the experiment into the design matrix. Perform each experiment and record
the results. For example:

Temperature

Pressure

Strength

Experiment #1

100 degrees

50 psi

21 lbs

Experiment #2

100 degrees

100 psi

42 lbs

Experiment #3

200 degrees

50 psi

51 lbs

Experiment #4

200 degrees

100 psi

57 lbs

8.
9.

Calculate the effect of a factor by averaging the data collected at the low level and subtracting it from the
average of the data collected at the high level. For example:

Effect of Temperature on strength:

(51 + 57)/2 - (21 + 42)/2 = 22.5 lbs
Effect of Pressure on strength:
(42 + 57)/2 - (21 + 51)/2 = 13.5 lbs
10.

11.

The interaction between two factors can be calculated in the same fashion. First, the design matrix must
be amended to show the high and low levels of the interaction. The levels are calculated by multiplying the coded
levels for the input factors acting in the interaction. For example:

Input A Level

Input B Level

Interaction

Experiment #1

-1

-1

+1

Experiment #2

-1

+1

-1

Experiment #3

+1

-1

-1

Experiment #4

+1

+1

+1

Calculate the effect of the interaction as before. Effect of the interaction on strength: (21 + 57)/2 - (42 +
51)/2 = -7.5 lbs

12.

The experimental data can be plotted in a 3D Bar

Chart.
factor can be plotted in a Pareto

The effect of each

Chart.
The negative effect of
the interaction is most easily seen when the pressure is set to 50 psi and Temperature is set to 100 degrees.
Keeping the temperature at 200 degrees will avoid the negative effect of the interaction and help ensure a strong
glue bond.

Conduct and Analyze Your Own DOE

Experiment Design Process

DOE

An experiment conducted under controlled conditions, using factor-levels intended to

estimate specified factors and their interactions. A designed experiment may be contrasted
with the analysis of happenstance, historical or casual data.
A designed experiment presumes that the process being studied or the simulation being
performed is stable and that the important factors in the process have been recognized.
While an experiment may be performed on an unstable process, the results may not be
directly useful because of large experimental error. All the important factors must be
included; otherwise, the factors cannot completely describe the process

Design of Experiments (DOE) with JMP

Design of experiments, or DOE, is a practical and ubiquitous approach for exploring multifactor
opportunity spaces, and JMP offers world-class capabilities for design and analysis in a form you can
easily use.

Methodical experimentation has many applications for efficient and effective information gathering. To
reveal or model relationships between an input or factor and an output or response, the best approach is
to deliberately change the former and see whether the latter changes, too. Actively manipulating factors
according to a pre-specified design is the best way to gain useful, new understanding.
However, whenever there is more than one factor that is, in almost all real-world situations a design
that changes just one factor at a time is inefficient. To properly uncover how factors jointly affect the
response, you need to use design of experiments (DOE).
In addition to a complete library of tried and tested classical DOE designs, JMP also offers an innovative
custom design capability that tailors your design to answer specific questions without wasting precious
resources. Once the data has been collected, JMP streamlines the analysis and model building so you
can easily see the pattern of response, identify active factors and optimize responses.

Role of DOE in Process Improvement

DOE is a formal mathematical method for systematically planning and

conducting scientific studies that change experimental variables together in
order to determine their effect of a given response.

DOE makes controlled changes to input variables in order to gain maximum

amounts of information on cause and effect relationships with a minimum
sample size.

DOE is more efficient that a standard approach of changing one variable at a

time in order to observe the variables impact on a given response.

DOE generates information on the effect various factors have on a response

variable and in some cases may be able to determine optimal settings for
those factors.
DOE encourages brainstorming activities associated with discussing key
factors that may affect a given response and allows the experimenter to
identify the key factors for future studies.
DOE is readily supported by numerous statistical software packages available
on the market.

BASIC STEPS IN DOE

Four elements associated with DOE:

1. The design of the experiment,

2. The collection of the data,
3. The statistical analysis of the data, and

4. The conclusions reached and recommendations made as a result of

the experiment.

TERMINOLOGY

Replication repetition of a basic experiment without changing

any factor settings, allows the experimenter to estimate the
experimental error (noise) in the system used to determine
whether observed differences in the data are real or just
noise, allows the experimenter to obtain more statistical power
(ability to identify small effects)

Randomization a statistical tool used to minimize potential

uncontrollable biases in the experiment by randomly assigning
material, people, order that experimental trials are conducted, or
any other factor not under the control of the experimenter.
Results in averaging out the effects of the extraneous factors
that may be present in order to minimize the risk of these factors
affecting the experimental results.

Blocking technique used to increase the precision of an

experiment by breaking the experiment into homogeneous
segments (blocks) in order to control any potential block to block
variability (multiple lots of raw material, several shifts, several
machines, several inspectors). Any effects on the experimental
results as a result of the blocking factor will be identified and
minimized
Confounding - A concept that basically means that multiple
effects are tied together into one parent effect and cannot be
separated. For example,
1. Two people flipping two different coins would result in the
effect of the person and the effect of the coin to be confounded
2. As experiments get large, higher order interactions (discussed
later) are confounded with lower order interactions or main
effect.

Factors experimental factors or independent variables

(continuous or discrete) an investigator manipulates to capture
any changes in the output of the process. Other factors of
concern are those that are uncontrollable and those which are
controllable but held constant during the experimental runs.

Responses dependent variable measured to describe the output

of the process.
Treatment Combinations (run) experimental trial where all
factors are set at a specified level.

Fixed Effects Model - If the treatment levels are specifically

chosen by the experimenter, then conclusions reached will only
apply to those levels.

Random Effects Model If the treatment levels are randomly

chosen from a population of many possible treatment levels, then
conclusions reached can be extended to all treatment levels in
the population.

Design of Experiments
One methodical tool used in Six Sigma that is very effective in
validating the correlation between input variables and output
variables is the Design of Experiments or DOE. The name
itself states its definition designing an experiment so that
once conducted it gives reliable data and you wont have to
conduct it again due to an error in the process of

experimenting. It consists of several experimental processes

that are used to calculate undetermined measurements of
causes and the connections between these causes. For
further reading see our article on Planning your DoE.
Its done statistically throughout the execution of mandatory
changes that is logically managed by mathematically
methodical charts. The implementation of changes within the
organization results in unexpected and sometimes
unfavorable outcomes. Thus, further analysis is required
otherwise the organization will suffer.
DOE is a methodical approach that will investigate the current
process or system. Sequence of prearranged experiments is
designed to which intended changes are placed to the input
factors of the system or process. The outcome of these
intended changes on the pre-identified output is then
evaluated. The method uses not just one set of experimental

concepts but more depending on what the project owners

want to establish.
Most of the time it is the work of the Black Belt to choose
which project concept should apply to the analysis. The
difference of DOE from other Six Sigma tools is that the input
variables are manipulated and its the outputs that are being
measured and studied. Other Six Sigma tools measure both
input and output factors.
There are so many designs used to measure and determine
the impact of each input. Project owners may use the full
factorial scheme, or the fractional factorial model, or the
response surface design.
The full factorial scheme analyzes the impact of every input or
mixture of inputs to performance. Response surface and
fractional factorial on the other hand uses definite algorithms

to minimize the occurrence of iterations essential to achieve

favorable results.
DOE is mostly used on projects with insufficient data. In
the Analyze phase it is used to validate possible defects. This
is done by showing if there is change in performance and if
the supposed caused is changed. In the Improve phase, DOE
is used to confirm if the designed developments will give the
anticipated outcome. DOE is also capable of determining
which design concept can give the best result.
Since DOE is commonly used when there is inadequate data,
it maximizes the available information. Most methodical
techniques offer results that promote either a single change
done at one time or a combination of changes done at the
same time, both of which have their own limitations.
A single change done at one time is time consuming and may
not be cost-efficient. A combination of changes done at the

same time on the other hand may give more unfavorable

results rather than a favorable outcome. Since there are
several changes made, effects might be greater than
expected. Using DOE, the project will be ready for all potential
dependencies by recommending the required data that needs
to be evaluated.
DOE is also best used to answer questions such as: what
factor contributes most to the problem?, what is the most
excellent arrangement of aspect values to reduce dissimilarity
in response?, how far can the process work if there is noise
involved? Most questions seek for the effectiveness and
strength of the design.
There are several intentions on why DOE is used. It could
either be to compare alternatives, to determine significant
inputs that affect output, accomplishing the most favorable
output, minimizing variability, minimizing and maximizing

responses, developing the product or process robustness,

and harmonizing tradeoffs.
It is a must that you have basic understanding in statistics for
you to perform the experiment. Understanding the
components of an experimental concept is also a must. You
should be able to understand what are the input (factors),
setting (levels), and output (responses) that should be
measured.
The factors or also referred to as the input can be categorized
as either uncontrollable or controllable. The response of the
experiment may vary depending on their respective factors.
Important responses are calculated and evaluated to identify
the factors and their levels that will present the most excellent
overall response for the CTQ characteristics both should
have variables that are measurable and attributes that are
assessable.

Every experiment has stipulating guidelines: the factors

(input) to be evaluated, the levels (setting) of these factors,
and the structure or conditions of experimental trials. Two
things to keep in mind when doing experiments, they should
be very simple and cost-effective. Two more conditions also
should be considered: (1) measurement system should be
capable, and (2) process should be stable.
Deficiency on either condition will result in unreliable
response. Dissimilarity impacting the output must be restricted
to random error causes and not those that are caused by
specific actions. Difficulties and complexities are always
present when creating experiments. General rule is the
simpler the experiment is the better.
Here are some experimental difficulties that you might
encounter and should stay away from:

Error could either be a measurement error or error on

sources or inexplicable variation, these can greatly obscure
the response. Error is referred to inexplicable variation
either on the experiment trial or between experiment trials.
Experiments that are accurately designed can easily
determine and calculate the cause of errors.

Noise factors these are uncontrollable inputs that

stimulate variation even when under the usual operating
condition. Examples of this are raw materials, numerous
machineries, multiple shifts, and humidity. A strong feature
of DOE is that it is capable of determining the inputs and
levels that reduces the effects of unmanageable factors.

Correlation referred to as the relationship between two

factors, correlation and causation are often confused.
There may have instances where two different factors
maybe well correlated devoid of one causing another factor.
The root of the cause could be due to another third factor.
Data analysis that lacks operational knowledge could result
to false conclusion. It is best to involve people, who are
from the affected environment, in the brainstorming activity.

Interaction also called the combined effects. Factors

could result in non-linear effects that do not add value.
Here is an example of DOE flowchart:

Design of Experiments (DOE) Flowchart Example

DOEs advantage over other systematic techniques is that it

allows project owners to select which design has the utmost
impact on the process or product. It is also capable of solving
long-standing and intricate problems in segments. DOE also
is the best tool to identify which amongst the various factors of
a complex process gives the biggest impact.