The Catapult Project: An Innovative Approach for Learning Statistical Design of
Experiments
Siong Lin Ho, Wan Lyn Nge, Kian Hong Chua
Multidiscipline Engineering Division, School of Engineering,
Ngee Ann Polytechnic, Singapore 599489
Abstract- This article illustrates the use of a
catapult as a n innovative teaching tool for enhancing
the understanding of Design of Experiments (DOE) in
a classroom environment. This device can be adopted
to simulate the design of a real-life product using the
ZKfactorial design and 2‘ factorial design with centre
points. I t is a n effective method to introduce the
fundamental concepts of experimental design.
Experimental data collected from the catapult is
analyzed to determine the key factors that affect the
distance traveled by the projectile.
Keywords -.. Design of Experiments; Catapult;
Factorial design.
1. INTRODUCTION
Quality and productivity improvement are most
effective when they are an integral part of the product and
process development cycle. In particular, Design of
Experiments (DOE) methodology [I] adopted at the
earliest stage of the development cycle, where new
products are designed, existing product designs improved,
and manufacturing processes optimized, is often the key
to overall product success.
DOE is a powerful technique used for discovering and
identifying a set of process or design variables (input
factors) that can significantly affect the process responses
(outputs). It was developed in the early 1920s by Sir
Ronald Fisher. This approach to experimentation for
improving the yield of corps was a direct replacement of
traditional One-time-factor-at-a-time (OFAT) approach.
DOE as an efficient strategy and powerful technique for
characterizing and optimizing products or processes in the
industq has been widespread in the industry over the past
two decades.
On the other hand, Six Sigma as a systematic
framework for quality improvement and business
excellence has been gaining popularity and acceptance
worldwide only in recent years 121. With the increase in
awareness of the various Six Sigma tools and techniques
- both management and statistical, as well as their
roles in the “Define-Measure-Analyze-Improve-Contro~
(DMAIC) methodology, it becomes clear that the next
challenge is on educating how these tools and techniques
can he effectively learned and appropriately applied in
various contexts, i.e. service sectors, manufacturing
processes, ... etc.
1056
0-7803-8519-5/04/$20.00@ 2004 IEEE
Statistical quality engineering constitutes the
backbone of the “Improve” phase in DMAIC, where DOE
plays a significant role in identifying the “vital few”
(critical parameters) among the “trivial many” in a
product or process. How could experimental design
techniques he learned effectively by students or non-
statisticians (e.g. managers and engineers)?
In this paper, the authors would like to share their
experiences in using the Catapult as an effective teaching
aid for understanding the fundamental principles of DOE.
This approach not only enhances students’ learning, hut
also stimulates their interest in the module. A Catapult
project with four factors is illustrated using the 2‘
factorial design technique. The objective of the
experiment is to identify the critical variables (factors)
that will optimize performance (i.e. maximize the distance
traveled hy the projectile). Using Analysis of Variance
(ANOVA) and F-tests, significant factors are thus
determined. Furthermore, the optimal combination of
levels for the factors is also investigated.
11. METHOOOLOGY
A. Experimental Design Strategies
Experimental design strategies are useful statistical
methods for maximizing the knowledge gained from
analyzing experimental data. Via the DOE methodology
of experimentation and statistical analysis, the researchers
can gain a better understanding of the processes on hand
and will therefore be in a better position for predicting
future outcomes of the system, i.e. enhances the decision-
making process. Moreover, the identification of optimal
setting of input factors can assist indusmal practitioners to
improve the quality of products, reducing scrap as well as
manufacturing costs. On the other hand, it is also possible
to indirectly improve the effectiveness and efficiency of
the manufacturing processes from drawing a valid,
reliable and sound conclusion from the experiments.
Therefore, this methodology is widely used in industries
for the improvement in either the product or process.
B. Description of Catapult
The catapult is a device for launching a small
projectile, such as hall, towards a targeted impact site. In
Ngee Ann Polytechnic, there are two different designs of
catapults which are made from wood (traditional catapult
in Figure 1) and Perspex (an improved version, the
modem catapult developed by students recently, in Figure
2). Experiments were carried out for both designs. In this
paper, for illustration purpose, all the data analysis and
discussion that follows are based on the traditional
catapult shown in Figure 1.
In the catapult design of Figure I , there are altogether
six controllable factors that can affect the distance
traveled by the projectile different types of ball were
~
experimented in our project. The design factors are: (i)
Arm-tension position, (ii) Upright-tension position, (iii)
Ball-seat position, (iv) Types of ball material, (v)
Tumtable (vi) Elevator height. Experimental design
techniques can be employed to determine the optimal
combinatioii of the various levels of each factor in order
to maximize or minimize the projected distance.
C . Selections of Factors
An important goal of this catapult project is to
identify the key input factors that greatly affect the
projected distance of ball. In the “Planning” phase of
experimentation, we carried out a “Cause & Effect”
analysis through brainstorming. As shown in Figure 3, the
lshikawa fish bone diagram provides an organized
approach for collecting engineering knowledge, coupled
with common sense, of all possible causal factors that will
influence the response. It was found that the Type of ball
material, Tumtable, Upright-tension and Elevator height
factors are potentially important parameters that affect the
distance traveled by the ball. Hence, it was decided that
the design matrix would include investigation of these
four factors, while both the Arm-tension and Ball position
factors are set to their respective nominal levels.
D. Factorial Design Techniques
Factorial designs are widely adopted in industries to
investigate several factors and their joint effects on the
response under study [3, 41. The 2‘ Full Factorial Design
is the simplest factorial design; it is used to study k
number of factors, each at two levels. In each complete
trial of the experiment, all possible combinations of the
International Engineering Management Conference 2004
levels of the factors are investigated. The levels can be
qualitative or quantitative.
.The advantages of a 2‘Sactorial design are:
Useful in the early stages of experimental work,
especially when there are many factors to be
. investigated.
The interaction between factors can be examined
to avoid misleading conclusions.
To screen and identitj key important factors that
affect the response. This can lead to improved
productivity, yield, mean response, reduced cost
and wastage.
Further extension to the conventional 2‘ factorial
design may be necessaly if the assumption of linearity in
the factor effects is in doubt. To protect against curvature
and possible quadratic effects, center points are
incorporated into the standard design matrix, although the
2‘ factorial works quite well even when linearity
assumption holds only approximately. In our study, a
statistical test of curvature is conducted.
Ill. STATISTICAL
DATAANALYSIS
A. Analysis of Variance and Efecfs Esfimates
Table II summarizes the experimental results of a
standard 24 factorial design with three replicates.
Randomizing all the 48 runs is to safeguard the
experiments from systematic bias that might cause
excessive variations in the response. The Analysis of
Variance (ANOVA) can be used as a basis for analyzing
data from designed experiments. The estimate of main
and interaction effects can be obtained using the Yates’
algorithm by Yates IS]. This technique can be used to
determine whether the effects are dominant at a specified
significance level. The main effect of a primaly factor can
be interpreted as the overage change in response produced
by a change in the level of the factor. As for interaction
effect, it means that the difference in response between
the levels of one factor is not the same at all levels of the
other factors. The estimate of any main effect or
interaction can be expressed as: Effect = (Contrast) / n2“’.
The ANOVA for this experiment is obtained in Table
111. By relatively comparing the magnitudes of effects and
also testing the various FsrariItirswith Fcrit,,, ( Fo.os,,, 31 =
7.53 ), it is concluded that all main effects (Factor A, B, C
and D) and interaction effects AD, BC, BD, ABC and
ABD are statistically significant at a = 5%. Since the F-
Statistics of Turn Table and Upright Tension are the
highest as compared with others, we can conclude that
they are the most important factors that greatly affect the
projectile distance of ball.
1057
B. Analysis of Center Point Design
In our center point design , all the four factors are
set at their nominal level and a total of six replicates are
obtained (Refer to Table I). Moreover, t-test is used to
determine the presence of non-linearity of the significant
factors. The null and alternative hypotheses are defined
as:
Ho:pf - pc= 0 (absence of non-linearity)
H i : pc- pc t 0 (significant non-linearity)
where pf= Population offactor to be tested
pc = Population of center point
(1)
1Sz+6Sc’
Sp’ = (2)
32+6
where Sc’ = variance of center points data
From the analysis, we conclude that the degree of
curvature is not significant for factors A, B, C and D.
C. Normal Probability Plot and Optimal Settings
The normal probability plot (refer to Figure 4) is a
graph of cumulative distribution of residuals on the
normal probability paper, i.e. a graph paper with the
ordinate scaled so that the cumulative normal distribution
plots as straight line. Residuals from factorial experiments
play an important role in assessing model adequacy. We
need to c a n y out a test for normality assumption of the
data by examining the normal probability plot of
residuals, which are the difference between observed and
predicted values of response at each design point. From
Figure 4, the residuals lie approximately close to a
straight line, which implies that there is no severe non-
normality in the data.
From the interaction effect plots of AD, BC and BD
in Figure 5 , the optimal levels of the factors that
maximize the projectile distance of the ball are: Factor B
(Turn Table) at “Low” level, Factor C (Upright Tension)
at “High” level and Factor D (Elevator) at “Low” level.
Factor A (Ball Type) is relatively not as important a
variable as the others. However, it is still a significant
factor and choosing a ball material specified at the “Low”
level may seem appropriate. Further confirmation
experiments are mn to verify the optimal combination of
factor levels.
1058 Znternational Engineering Management Conference 2004
4.5
Fig. 4. Normal Probability of residual
Far validation, we experimented with the optimal
settings: Factors A (Ball Type), B (Turn Table) and D
(Elevator) set at “Low” level; whereas Factor C (Upright
Tension) is set to “High” level. From the results, it was
found that the average projectile distance is around 3.40111
and its variance is ahout +/- 7%. Thus we verify that the
recommended factor levels can yield optimal
performance, i.e. maximizing the response with minimal
variability.
D. Regression Model and Response Surface
An empirical model is fined to the data. This
regression model can he used to predict the “projectile
distance” at any point in the original experimental region.
The model can be formulated as:
Y = 1.50 - 0.175 Xa- 0.52 XB+ 0.49 XL -0.405 XD
0.095 X,J-0.085 X,Xc+ 0.155 XeXo-
0.065 X,XsXc+ 0.08 X A X B X ~
The response surfaces that show the relationship
between the “projectile distance” and the significant pairs
affactors are depicted in Figure 6 .
IV. CONCLUSION
The catapult is an innovative teaching tool for the
DOE class as it allows the students to gain a better
understanding of the theoretical knowledge of DOE in
classroom through conducting actual experiments. Using
this device, it is possible for us to simulate the design o f a
real-life product 01 the improvement of an indusmal
process. Furthermore, students have the opportunity to get
involved in the several stages of product prototype design
-- from planning the design strategy, brainstorming, cause
and effect analysis, to developing both sofl-skills and
technical skills while conducting experiments. Students
learned to develop empirical models for the catapult as
well as determining optimal settings of the variables that
maximize performance. These activities allow students to
apply the knowledge they learned in their course.
Futhermore, the catapult experiment makes the students'
learning process interesting and enjoyable. It is concluded
that the catapult device is a very useful teaching aid for
enhancing the understanding of experimental design
techniques such as the 2' factorial design and others. The
creative learning of DOE via the catapult project is
effective and more importantly, "fun"!
REFERENCES
[I] Box, G. E. P., Hunter, W. G., and Hunter, J. S.Slalisticsfor
Ezpwimelers, New York, John Wiley. 1978.
[Z] Goh, T. N. "A Strategic Assessment of Sin Sigma",
Quality orrd Relizbiliry Engineering Inrernotionol, Vol. 18,
pp. 403-410,2002.
131 Montgomery, D.C. Design ondAnnlysb ofExperimenrs (5'
ed.), New York, John Wiley, 2002.
[4] Gob, T. N."A Pragmatic Approach 10 Experimeofal Design in
Industry", Journol ofApplied Srorirtics, Vol. 28, No.3 & 4,
pp. 391-398.2001.
151 Yates. F. Derign a d Arwlysb of Focmrlol Experimenrs.,
Tech. Cumm. No. 35. Imperial Bureau of Sail Sciences,
London, 1937.

Fig.3 Fishbane diagram

I A+
E'tGl
A- E+
Et"
B

Fig. 5 Main Interaction Effects plots

1
... .. .. _. I

Fig. 6 Respame surfaces

International Engineering Management Conference 2004 1059

 TABLE I
DATA COLLECTION (CENTER PONT DESIGN)

A B C D I I1 111 IV V VI AVE
0 0 0 0 149 156 158 142 166 I44 153

TABLE II
DATA COLLECTION (DESIGN MATRIXOF 2' FACTORIAL)

U - - + + 2.25 2.28 2.52 2.35

1 4 + - + + 1.49 1.61 1.55 1.55
IS - + + + 1.24 1.33 1.47 1.35
1 6 + + + + 0.85 0.81 1.06 0.91

Factors BCD I 0.08 I 1.00 I 0.08 0.03 I 0.08

TABLE IV
VERIFICATION OF OPTIMAL SETTNGS

- - + . 3.48 3.42 3.4 3.43 3.6 3.42 3.46

1060