Aquaculture 437 (2015) 344–350

Contents lists available at ScienceDirect

Aquaculture
journal homepage: www.elsevier.com/locate/aqua-online

Short communication

Misuse of multiple comparison tests and underuse of contrast procedures

in aquaculture publications
Rodrigue Yossa a,⁎, Marc Verdegem b
a
Department of Biology, Gus R. Douglass Institute, WV State University, Institute, WV, USA
b
Aquaculture and Fisheries Group, Wageningen University, Wageningen, The Netherlands

a r t i c l e i n f o a b s t r a c t

Article history: In aquaculture research, independent variables are qualitative (with or without a structure), quantitative,
Received 19 July 2014 or factorial combinations. A qualitative independent variable is a variable that has unquantifiable, nominal
Received in revised form 3 November 2014 variants (levels), which represent different categories such as the fish gender. The structure in a qualitative
Accepted 15 December 2014
independent variable refers to the existence of a relation between its different variants, in a way that sug-
Available online 20 December 2014
gests that some variants can be grouped together and then compared to other groups of variants. A quanti-
Keywords:
tative independent variable is a variable with measurable variants that are expressed numerically and are
Aquaculture research fixed throughout the experiment, such as water temperatures. In a study with one independent variable,
Contrast procedures each variant of this variable represents a treatment. In a study with two or more independent variables,
Multiple comparison tests also called a factorial or multifactorial experiment, the treatments represent all the possible combinations
Structured treatments of the two or more independent variables. Following an analysis of variance (ANOVA) (or a multiple factor
Biostatistics ANOVA) showing that there is a significant difference among the three or more treatment means, a multiple
comparison test, an orthogonal contrast procedure, or a polynomial contrast procedure is applied to sepa-
rate or present the relationship among the treatment means, in accordance with the nature and structure of
the independent variable. The use of multiple comparison tests such as Least Significant Difference,
Duncan's Multiple Range, Tukey's Honest Significant Difference, Bonferroni and Scheffé's tests, is more rel-
evant when there is no structure in the qualitative independent variable; otherwise the use of the orthog-
onal contrast procedure, which allows the comparison of related treatment means or groups of means to
other treatment means, is more appropriate. The orthogonal contrast procedure is also appropriate for fac-
torial experiments. With quantitative independent variables, the use of polynomial procedure, which de-
tects the trend of the relationship or regression that exists between the independent and response
variables, is appropriate.
The present paper critically analyzed the statistical methods used in articles published in ten selected interna-
tional peer-reviewed aquaculture journals in the year 2013. This analysis showed that in none of the studies in
which the independent variable was qualitative with a structure, the data have been analyzed using orthogonal
contrast procedure. Also, the data of only 34% of the studies in which the independent variable was quantitative
have been analyzed using polynomial contrast (regression), whereas the data of only 13% of studies with a fac-
torial design have been analyzed using contrast procedure. More attention should be paid on publishing only
studies that used appropriate statistical procedures, which conform to the nature of the independent variables
of interest.
© 2014 Elsevier B.V. All rights reserved.

1. Introduction
Aquaculture is the fastest growing sector in agricultural production
across the world, and it currently contributes over 42% in global fish
production (FAO, 2014). The growth in aquaculture production during
the last four decades coincided with increasing global efforts in
⁎ Corresponding author at: Department of Biology, Gus R. Douglass Institute, West
Virginia State University, P.O. Box 1000, Institute, WV 25112, USA. Tel.: +1 304 766
4165x4165; fax: +1 304 7665244.
E-mail addresses: rodyossa@yahoo.fr, ryossa@wvstateu.edu (R. Yossa).
aquaculture research and development. For every finfish and shellfish
belonging to the 600 aquatic species currently raised in captivity world-
wide, aquaculture research essentially focuses on investigating the
range of water physicochemical parameters (temperature, pH, hard-
ness, ammonia, dissolved oxygen, etc.), nutrient requirement and sub-
stitution levels (macronutrients such as proteins, lipid and
carbohydrates, and micronutrients such as vitamins and minerals),
feeding (feeding rate, feeding frequency, etc.), use of therapeutants,
and production system and technology (light intensity levels, photope-
riod, animal density, etc.). In other words, in aquaculture like in other
sectors of agricultural sciences, most of the independent variables

http://dx.doi.org/10.1016/j.aquaculture.2014.12.023
0044-8486/© 2014 Elsevier B.V. All rights reserved.
 R. Yossa, M. Verdegem / Aquaculture 437 (2015) 344–350
(experimental variables) are qualitative (with a structure), quantitative,
or factorial combinations. The use of qualitative variable with a struc-
ture and quantitative experimental variables, alone or in combination,
allows making “planned comparisons” (Steel et al., 1997).
A qualitative independent variable is a variable that has unquantifi-
able, nominal variants (levels), which represent the different categories
included in this variable such as the presence or absence of a character
or the fish gender (male or female). The structure in a qualitative inde-
pendent variable refers to the existence of a relation between the differ-
ent variants of this variable, in a way that suggests that some variants
can be grouped together and then compared to other groups of variants
(Steel et al., 1997). For example, assuming that the objective of an aqua-
culture experiment is to determine the dietary protein optimizing fish
growth, there might be four protein types (treatments) tested: soybean
meal, corn gluten meal, fish meal, and beef blood meal. There is a struc-
ture among these protein types as the two plant proteins (soybean meal
and corn gluten meal) can be grouped and compared to a group includ-
ing the animal proteins (fish meal and beef blood meal). In addition,
soybean meal can be compared to corn gluten meal and fish meal can
be compared to beef blood meal. Another example is an aquaculture ex-
periment aiming to identify the plant protein that when substituting
10% of fish meal in a salmon diet results in the best growth. Four exper-
imental plant proteins (treatments) might be tested: corn gluten meal,
soybean meal, black eye pea meal and canola meal. There is no relevant
relationship among the experimental plant proteins that can be consid-
ered in order to classify them into definable treatment groups; therefore
there is no structure in the independent variable of this experiment.
A quantitative independent variable is a variable with levels that are
measurable quantities that are expressed numerically such as the water
temperatures, dietary protein levels or fish densities per tank. The levels
of a quantitative independent variable are maintained unchanged
throughout the experimental period. An example is an experiment in
which the independent variable is the water temperature, with five
levels, 20, 22, 26, 28, and 30 °C.
In a study with one independent variable (unifactorial experiment),
each variant of this variable represents a treatment, and the total num-
ber of treatments (k) equals the total number of variants of the indepen-
dent variable. In a study with two or more independent variables, also
called a factorial or multifactorial experiment, the total number of treat-
ments (k) represents all the possible combinations of the two or more
independent variables (Petersen, 1977; Steel et al., 1997). For instance,
a study with the independent variables (factors) water temperature
with two levels (20 °C and 30 °C) and dietary protein source with
three levels (soybean meal, gluten meal, and black eye pea meal) is a
2 × 3 factorial experiment, with 2 × 3 = 6 treatments (Table 1).
Several studies have pointed out the extensive misuse of pairwise
multiple comparison tests in agricultural sciences (Chew, 1976; Gates,
1991; Gill, 1973; Little, 1981; Madden et al., 1982; Petersen, 1977).
However, the examination of the use of statistics in aquaculture science
has received little attention. The present paper therefore critically ana-
lyzes the statistical methods used in manuscripts published in interna-
tional aquaculture journals, to draw the attention of researchers on
Table 1
Example of the different treatments obtained by combining the level of the two indepen-
dent variables (protein sources and water temperatures) in a factorial experiment.
Protein sources
Soybean meal Corn gluten meal Black eye pea meal
(SB) (CG) (BEP)
Water temperatures 20 SB20* CG20* BEP20*
(°C) 30 SB30* CG30* BEP30*
*SB20 refers to soybean meal fed to fish reared at 20 °C, SB30 refers to soybean meal fed to
fish reared at 30 °C, CG20 refers to corn gluten meal fed to fish reared at 20 °C, CG30 refers
to corn gluten meal fed to fish reared at 30 °C, BEP20 refers to black eye pea meal fed to fish
reared at 20 °C, and BEP30 refers to black eye pea meal fed to fish reared at 30 °C.
345
the importance of good statistical planning and proper data analysis.
Nevertheless prior to performing this critical analysis, it is worth
reviewing some basic and applied concepts in relation to the planning
and execution of data analysis (and interpretation).
2. Materials and methods
We start with a comparative description of multiple comparison
tests and contrast procedures, presenting the circumstances in which
it is more appropriate to use these statistics. Then we present the meth-
odological approach that was used to collect and evaluate statistical
methods applied in aquaculture experiments published in peer-
reviewed journals.
2.1. Multiple comparison tests versus contrast procedures
In an experiment with one independent variable that has more than
three levels, and assuming that the independence, normality and homo-
geneity of the residuals are satisfied, analysis of variance (ANOVA) is
used to test the null hypothesis: “there is no significant difference be-
tween the treatment means”. In an experiment with a combination of
two or more independent variables (factorial experiment) the null hy-
pothesis is tested using (multi)factorial ANOVA. When there is a signif-
icant difference among the treatment means, the null hypothesis is
rejected and a multiple comparison test, an orthogonal contrast proce-
dure or a polynomial contrast procedure is applied to separate treat-
ment means or to analyze the relationships that might exist among
the treatment means. The choice of one of these statistical procedures
depends on the nature and structure of independent variables (qualita-
tive with or without structure, quantitative, or factorial combinations).
Multiple comparison tests are statistical methods that are intended
to compare each factor level mean with every other factor level mean
assuming that there is no definable structure among the factor levels
(Gates, 1991; Montgomery, 1997; Petersen, 1977; Saville, 1990; Steel
et al., 1997). In this manner, a pairwise multiple comparison compares
factor level effects on the response variable. For instance, in the above
experiment with the four experimental plant proteins, the use of a mul-
tiple comparison test is relevant, as no definable structure exists among
the experimental plant proteins (treatments). Examples of multiple
comparison tests include Least Significant Difference (LSD), Duncan's
Multiple Range, Waller and Duncan's, Tukey's Honest Significant Differ-
ence (Tukey-HSD), Bonferroni and Scheffé's procedures. The merit of
each of these tests over the others has extensively been discussed
(Montgomery, 1997; Steel et al., 1997), and is beyond the scope of
this paper. However, with qualitative independent factors showing a
structure, statistical procedures exist which lead to more meaningful
conclusions on treatment effects; furthermore, pairwise multiple com-
parison tests should not be used on quantitative main factors and facto-
rial interaction terms as this leads to misinterpretation of research
results and flawed conclusions (Gates, 1991; Montgomery, 1997;
Olsen, 2003; Petersen, 1977; Rafter et al., 2002; Saville, 1990; Steel
et al., 1997).
In an experiment with a qualitative variable showing a structure, an
adequate statistical procedure for comparing treatment means is the or-
thogonal contrast procedure (Davis, 2010; Montgomery, 1997; Steel
et al., 1997). With this procedure related treatment means or groups
of treatment means are specifically compared to other treatment
means, for a total of comparisons equal to the degrees of freedom.
Each comparison is planned prior to the execution of the experiment
and can represent an objective in the research protocol. For instance,
in a fish nutrition experiment aiming at i) selecting the best protein
source for fish growth, ii) determining the best plant-based protein for
fish growth, and iii) determining the best animal-based protein source
for fish growth, the independent variable is the fish diet formulation
(qualitative variable). In this example, there is only one source of varia-
tion, the diet, and the four experimental treatments (k) that can be
346 R. Yossa, M. Verdegem / Aquaculture 437 (2015) 344–350
divided into two groups, one group composed of the two dietary plant-
based protein sources (soya bean and corn gluten) and the other group
composed of the two dietary animal-based protein sources (fish meal
and beef blood meal). There is thus a clear relationship between these
treatments, which means that the independent variable is structured.
Assuming that the ANOVA will show that there is a significant difference
among the treatment means, the following three (i.e. degree of free-
dom = k − 1 = 4 treatments – 1 = 3) contrasts can be planned prior
to the execution of the experiment:
i) Comparison of plant-based protein versus animal protein (ex-
perimental objective 1),
ii) Comparison of the two plant-based proteins (experimental ob-
jective 2),
iii) Comparison of the two animal proteins (experimental objec-
tive 3).
It is worth noting that the number of contrasts (3) is equal to the
number of comparisons you would make with a post-hoc test assuming
an experiment with four treatments (diets), resulting in 3 post-hoc
comparisons, each with 1 degree of freedom. However, the contrast pro-
cedure offers the possibility of comparing groups of treatments, which
increases the power of the contrasts over multiple comparisons; for in-
stance in this example a multiple comparison would not specifically
make the i)th comparison.
On the other hand, the most relevant method recommended to an-
alyze the means obtained from an experiment with a quantitative inde-
pendent variable is the polynomial contrast procedure (Davis, 2010;
Montgomery, 1997; Steel et al., 1997). Polynomial contrasts detect the
trend of the relationship or regression that exists between an indepen-
dent experimental variable and the response (dependent) variable. This
relationship might be linear (linear regression), quadratic (quadratic re-
gression or second order polynomial) or cubic (cubic regression or third
order polynomial). For instance in a dose response experiment aiming
at determining the minimum vitamin levels that leads to the highest
growth in fish, the graded vitamin A levels (k) might be 0, 0.01, 0.05,
0.1, 0.2, 0.5, 0.7 and 1 mg/kg diet. The optimum vitamin A level is esti-
mated by analyzing the regressions (linear, quadratic or cubic) between
the growth and the vitamin levels. Let us assume that the regression is
linear (Fig. 1), that is the curve function or regression equation is in
the form: growth = X (vitamin A level) + Y, or y = 142.57x + 135,
where “X” (142.57) is the slope and “Y” (135) is the value of the growth
when no dietary vitamin is added to the diet; then the researcher will
300
275
y = 142.57x + 135
250
r² = 0.91161
225
200
Fish growth (g)
175
150
125
100
75
50
25
0 0
0 0.2 0.4 0.6 0.8 1 1.2
Dietary vitamin A level (mg/ kg diet)
Fig. 1. Example of the use of the polynomial contrast (regression) to analyze the data of an
experiment in which there is a significant (p b 0.05) linear regression between fish growth
(dependent variable) and dietary vitamin A levels (independent quantitative variable).
conclude that fish growth increases with an increase in the dietary vita-
min A level.
Let us now assume that the regression is quadratic (Fig. 2), that is the
curve function or regression equation is in the form: growth = X (vita-
min A level)2 + Y (vitamin A level) + Z, or growth = −240.3 (vitamin
A level)2 + 294.2 (vitamin A level) + 125.1 where “X” (=−240.3), “Y”
(=294.2), and “Z” (=125.1) are the coefficients of the quadratic curve
function, with “Z” (=125.1) being the value of the growth when no di-
etary vitamin is added to the diet; then the next step is to estimate the
vitamin A level (vitamin A level-max) that leads to the maximum
growth of fish, that is the inflection point which represents the point
at which the curvature or concavity changes sign from minus to plus
or from plus to minus on a curve. This inflection point is obtained
through the equation in which the first derivate of the curve function
is equal to zero; in the current example d(growth) = 0 → 2X (vitamin
A level) + Y = 0 → vitamin A level-max = − Y / (2X) = − 294.2 /
(2 × −240.3) = 0.612 mg vitamin A.
In a situation where there is a significant cubic regression (Fig. 3A),
the use of straight broken-line regression procedure has been suggested
(Robbins et al., 1979, 2006). The straight broken-line regression proce-
dure divides the data in two parts and assigns a linear regression to each
part (Fig. 3B). The intersection point of the two linear regressions is the
optimal level of the quantitative factor (Fig. 3B). For more details on the
broken-line regression procedure, the readers are encouraged to read
Robbins et al. (2006).
Furthermore, the polynomial contrast procedure also provides the
coefficient of determination “r2”, which measures the adjustment of
the regression equation in regard to the points that illustrate the rela-
tionship between the independent and the dependent variable in a dia-
gram (Figs. 1, 2 & 3); for instance, a coefficient of determination of 0.91
(r2 = 0.91; Fig. 1) indicates that the current regression equation ex-
plains 91% of the variation observed in the dependent variable
(growth), as a result of changes in the independent variable (vitamin
A); this information cannot effectively be provided with the multiple
comparison procedures (Petersen, 1977).
In practice, the first step in applying the polynomial contrast proce-
dure is to check whether or not the cubic regression is significant. If the
cubic regression is significant, then the quadratic and linear regressions
are not considered; otherwise, the significance of the quadratic regres-
sion is examined. If the quadratic regression is significant, the linear re-
gression is not considered; otherwise, the significance of the linear
regression is tested.
Let us now assume an experiment aiming at studying the combined
effects of different dietary plant proteins and water temperatures on
300
250 y =-240.3x 2 + 294.2x + 125.16
r² = 0.89834
200
Fish growth (g)
150
Vitamin level-max = 0.61 mg/kg diet
100
50
0
0.2 0.4 0.6 0.8 1 1.2
Dietary vitamin A level (mg/ kg diet)
Fig. 2. Example of the use of the polynomial contrast (regression) to estimate the dietary
vitamin A level (0.061 mg/kg diet) that leads to the maximum fish growth, in an experi-
ment in which there is a significant (p b 0.05) quadratic relationship between fish growth
(dependent variable) and dietary vitamin A levels (independent quantitative variable).
 R. Yossa, M. Verdegem / Aquaculture 437 (2015) 344–350
A B
175
150
125
Fish growth (g)
100
75
y = 727.02x 3 - 1068.5x 2 + 339.14x + 126.49
r² = 0.81564
50
25
0 0
0 0.2 0.4 0.6 0.8
Dietary vitamin A level (mg/kg diet)
Fig. 3. Example of the use of the polynomial contrast (regression) to analyze the data of an experiment in which there is a significant (p b 0.05) cubic regression between fish growth
(dependent variable) and dietary vitamin A levels (independent quantitative variable) (A); the broken-line regression procedure estimated the vitamin A level that leads to the maximum
fish growth at 0.21 mg/kg diet (B).
fish growth, with three different dietary plant proteins (soybean meal,
corn gluten meal, and black eye pea meal) and two water temperature
levels (20 and 30 °C). The former represents a qualitative factor showing
no structure while the latter represents a quantitative factor. If the study
is conducted following a completely randomized factorial design, there
are six possible treatments (Table 1). In this experiment, the number of
planned contrasts equals the number of degrees of freedom, which is 6
treatments − 1 = 5. The ANOVA will verify whether or not the follow-
ing sources of variations significantly affect fish growth:
1) Interaction protein source × water temperature,
2) Main effect of water temperature,
3) Main effect of protein source.
To interpret ANOVA results, the first step is to check whether or not
the interaction protein source × water temperature is significant. If this
interaction is significant, only the interactive effect of the two factors is
considered and the two main effects are not considered; hence, the in-
terpretation of the data is that every dietary protein source behaves dif-
ferently at each water temperature. Hence, the effects of water
temperature and dietary protein source on fish growth cannot be sepa-
rated. In contrast, if the interaction is not significant, then the effects of
water temperature and dietary protein source are independent and can
be analyzed separately. If the effect of water temperature is significant,
then the water temperature leading to the highest growth is the best
rearing temperature. If the effect of protein source is significant, then
the following contrasts can be considered:
1) Soybean meal versus corn gluten meal,
2) Soybean meal versus black eye pea meal,
Furthermore, if neither the interaction effect nor the main effects are
significant, then it is concluded that the individual and combined effects
of these independent variables do not lead to any change in fish growth.
For details on the formulation, statistical programing and interpreta-
tion of orthogonal and polynomial contrasts in planned experiments in
agricultural sciences, readers are referred to the book edited by Steel
et al. (1997).
2.2. Data collection and analysis
We analyzed the statistical methods used in the articles published in
the last issues of the year 2013 of ten selected international aquaculture
journals, namely Journal of the World Aquaculture Society, Journal of
347
Intersection point= Optimum vitamin A level
175 = 0.25 mg/kg diet
150
125
Fish growth (g)
100
75
50
25
1 1.2 0 0.2 0.4 0.6 0.8 1 1.2
Dietary vitamin A level (mg/kg diet)
Applied Aquaculture, Journal of Aquaculture Research and Develop-
ment, Aquaculture, Aquaculture Nutrition, Aquaculture Research, Aqua-
culture International, Fish Physiology and Biochemistry, Aquacultural
Engineering, and North American Journal of Aquaculture. The selection
of these journals was based on the editorship and scientific experiences
of the authors in fundamental and applied aquaculture research. As
such, the authors considered these journals as the major academic
journals that are currently specialized in publishing general aquaculture
research results in English, with a worldwide audience and internation-
al acclaim. Additionally, these journals are published by the main pub-
lishers in agricultural research and technology, namely Springer,
Wiley-Blackwell, Elsevier, Taylor and Francis Group, and OMICS Pub-
lishing Group. For each of these journals, the total number of published
articles was provided, but attention focused on typical original aquacul-
ture research articles, which directly deal with scientific experiments in
aquatic animal and plant culture. In other words, studies conducted at
basic levels with no immediate application in aquatic animal or plant
farming such as papers dealing exclusively with aquaculture genomics,
fundamental fish biology on the one hand, and aquaculture socio-
economic and review articles on the other hand were not considered
for analysis in the current study.
The materials and methods section of each aquaculture article pub-
lished in any of the selected journals was first scrutinized in order to de-
termine the type of the independent experimental factor (qualitative,
quantitative or factorial combinations), the possible structure between
the qualitative independent experimental factors, the experimental de-
sign as well as the statistical procedures used to compare the means ob-
tained with the experimental treatments. Then, the results section was
examined in order to make sure that the procedures used to analyze the
treatment means, following ANOVA or multiple factor ANOVA, conform
to what was planned in the materials and methods section. Also, we
checked whether or not the presentation and interpretation of results
were in accordance with the appropriate statistical practice described
previously (Gates, 1991; Montgomery, 1997; Petersen, 1977; Rafter
et al., 2002; Steel et al., 1997).
3. Results and discussion
The present study intended neither to compare scientific aquacul-
ture journals nor to discredit the quality of aquaculture publications.
This study rather aimed at encouraging aquaculture scientists to use
 348
Table 2
Analysis of the statistical procedures used in articles with qualitative independent variables showing structures, quantitative independent variables and factorial combinations, published in selected peer reviewed aquaculture journals in the year
2013.

Academic journal [years: volume (issue), month] Total number of Total number of Total number of Total number of Total number of Total number of Total number of Total number of Publisher
published typical articles with a articles with a articles with a articles with a articles with a articles with a
research articles aquaculture qualitative quantitative factorial qualitative quantitative factorial combination
articles independent independent combination independent independent variable analyzed using
variable showing variable that has variable showing a that has more than 2 contrast procedures
a structure more than 2 levels structure analyzed levels analyzed using (% total number of
using orthogonal polynomial articles with a

R. Yossa, M. Verdegem / Aquaculture 437 (2015) 344–350

contrasts (% total regression (% total factorial
number of articles number of articles combination)
with structured with quantitative
qualitative factors) factors)

Journal of the World Aquaculture Society (2013: 11 6 2 5 0 0 (0%) 4 (80%) – Wiley

44 (6), Dec.)
Journal of Applied Aquaculture (2013: 25 (4), Dec.) 8 7 1 3 0 0 (0%) 0 (0%) – Taylor and Francis
Group
Journal of Aquaculture Research and Development 9 8 3 5 0 0 (0%) 0 (%) – OMICS Publishing
(2013: 4 (5), Sep.)a Group
Aquaculture (2013: 416–417 (–), Dec.) 55 38 2 17 7 0 (0%) 8b (47%) 2c (29%) Elsevier
Aquaculture Nutrition (2013: 19 (6), Dec.) 11 11 3 6 4 0 (0%) 2d (33%) 0 (0%) Wiley
Aquaculture Research (2013: 44 (12); Nov.) 13 11 1 5 0 0 (0%) 1 (20%) - Wiley
Aquaculture International (2013: 21 (6), Dec.) 18 14 5 7 1 0 (0%) 2e (29%) 0 (0%) Springer
Fish Physiology and Biochemistry (2013: 39 (6), Dec.) 26 15 1 6 4 0 (0%) 2f (33%) 0 (0%) Springer
Aquacultural Engineering (2013: 57 (–), Nov.) 18 17 0 1 4 – 1g (100%) 2 (50%) Elsevier
North American Journal of Aquaculture (2013: 75 15 15 1 3 3 0 (0%) 0 (0%) 0 (0%) Taylor and Francis
(4), –) Group

– = Missing information.
a
The issue 5 (Sept. 2013) was preferred to issue 6 (Dec. 2013) because the former issue contained a total of 9 manuscripts with 8 typical aquaculture articles while the latter contained a total of 6 articles with only 1 typical aquaculture article on
January 9, 2013.
b
In 5 papers, polynomial regression and a multiple comparison test were simultaneously performed on the data.
c
In 1 paper, contrast procedure and a multiple comparison test were simultaneously performed on the data.
d
In 2 papers, polynomial regression and a multiple comparison test were simultaneously performed on the data.
e
In 2 papers, contrast procedure and a multiple comparison test were simultaneously performed on the data.
f
In 2 papers, contrast procedure and a multiple comparison test were simultaneously performed on the data.
g
In 1 paper, contrast procedure and a multiple comparison test were simultaneously performed on the data.
 R. Yossa, M. Verdegem / Aquaculture 437 (2015) 344–350
the available statistical tools correctly. Doing so, more reliable and sci-
entifically sound conclusions will be drawn from experiments.
The analysis of the recent articles published in the year 2013 in ten
selected international aquaculture journals (Table 2) showed that in
none of the studies in which the independent variable was qualitative
showing a structure, the data have been analyzed following the contrast
procedure. On average, in 34% of the studies with a quantitative inde-
pendent variable, the data were analyzed using the polynomial contrast.
In factorial experiments, 13% of the studies used the contrast procedure
to analyze the data. In all the other cases, multiple comparison tests
were used.
Moreover, in 60% of the studies where the quantitative independent
variable was analyzed using the polynomial contrast (regression) pro-
cedure, a multiple comparison test was simultaneously performed
with the regression. In these studies authors based their conclusion ex-
clusively on the result of the regression analysis, probably because each
of the multiple comparison tests used did not provide a precise optimal
level of the quantitative experimental factor, but rather a relatively large
range. For instance, in the study aiming at estimating the dietary isoleu-
cine requirement of fingerling catla, Catla catla (Zehra and Khan, 2013),
fish were fed diets containing 0.5, 0.75, 1.0, 1.25, 1.5 and 1.75% isoleu-
cine in dry diet. Based on absolute weight gain, the multiple comparison
test that was used in this study, the Tukey's HSD test, estimated the di-
etary isoleucine requirement between 1.25 and 1.5% isoleucine in dry
diet (Zehra and Khan, 2013). The polynomial regression procedure esti-
mated the optimum dietary isoleucine requirement at 1.13% isoleucine
in dry diet for 95% maximum absolute weight gain or 1.35% isoleucine in
dry diet for 100% maximum absolute weight gain (Zehra and Khan,
2013). Another study aimed at estimating the dietary linolenic acid re-
quirement for juvenile Nile Tilapia Oreochromis niloticus (Chen et al.,
2013). Fish were fed diets containing 0.00, 0.10, 0.32, 0.63, 0.98, 1.56
and 2.04% linolenic acid in dry feed weight. The multiple comparison
test that was used in this study, the Tukey's HSD test, failed to estimate
the dietary linolenic acid requirement for weight gain (WG) because the
results were difficult to interpret; WG was similar among 0.00, 0.10,
0.32, 0.98, 1.56 and 2.04% of dry feed weight on the one hand, and
among 0.10, 0.32, 0.63, 0.98, 1.56 and 2.04% of dry feed weight on the
other hand (Chen et al., 2013). However, based on weight gain the
broken-line regression procedure estimated the optimum dietary
linolenic acid requirement for juvenile Nile Tilapia 0.45% of dry fed
weight (Chen et al., 2013).
The improper use of statistical methods to test the statistical hypoth-
esis is not endemic to agricultural sciences, including aquaculture. This
situation has also been extensively criticized in medical and health sci-
ences (Avram et al., 1985; Cruess, 1989; Emerson and Colditz, 1983;
Felson et al., 1984; MacArthur and Jackson, 1984; Olsen, 2003; White,
1979), as well as in plant production science (Chew, 1976; Gates,
1991; Little, 1981; Madden et al., 1982; Petersen, 1977). The results of
the present study suggest that aquaculture researchers, reviewers and
editors seem to be uncritical when it comes to the correct use of statis-
tical procedures. Aquaculture researchers should give close and
thoughtful consideration to the statistical planning of experiments, ac-
curate analysis of data, adequate interpretation of the results and their
presentation to the scientific community, government, extension ser-
vices, aquaculture industry and farmers. Several materials that can
help in performing these tasks have been published in the last decades
(Steel et al., 1997), with a recent book specifically dedicated to aquacul-
ture research (Bhujel, 2009).
4. Conclusions
An analysis of the statistical methods used in the articles published
in ten selected international peer-reviewed aquaculture journals in
the year 2013 showed that over 65% of the papers did not apply the cor-
rect statistical analyses, which conform to the experimental design and
nature of independent variables used. In the aquaculture research
349
community, there is ample of room for improvement in the use of statis-
tics, and contrast procedures should be considered in this regard. How-
ever, although orthogonal and polynomial contrasts are appropriate for
qualitative variable showing a structure and quantitative variable, re-
spectively, as well as for factorial combinations, care should still be
taken in interpreting the results and in drawing conclusions.
Aquaculture researchers, peer-reviewers and journal editors should
therefore pay attention on the use of correct statistical procedures
starting with defining the nature of the independent variables studied
and aiming to provide correct and meaningful conclusions. Aquaculture
researchers are encouraged to seek advice from statisticians both during
the planning of experiments and during data analysis and interpretation
upon completion of the experiment (Gates, 1991; Madden et al., 1982).
It would thus be a good common practice to always mention statistical
support received in the acknowledgements section of manuscripts. In
addition, reviewers and editors should check on the appropriate use of
statistical methods in manuscripts, and ask authors to reanalyze their
data when statistics were not used correctly.
Important tips to improve data analysis include:
- Clearly define the experimental treatments in order to identify the
possible structures among the treatments,
- Only perform a multiple comparison tests if there is no structure
among the treatments,
- Only use polynomial contrast procedure to analyze data from exper-
iments in which the independent factor is quantitative,
- Do not perform both a multiple comparison test and a polynomial
contrast on the same data,
- Do not hesitate to seek assistance from a statistician during experi-
ment design as well as data analysis and interpretation.
Acknowledgment
The authors would like to thank Dr. André-Marie Taptué for provid-
ing the literature used in this paper, and Dr. Roberte Gaelle Ngaha
Siankhe for her thoughtful comments on the initial version of this paper.
References
Avram, M.J., Shanks, C.A., Dykes, M.H., Ronai, A.K., Stiers, W.M., 1985. Statistical methods
in anesthesia articles: an evaluation of two American journals during two six-month
periods. Anesth. Analg. 64, 607–611.
Bhujel, R.C., 2009. Statistics for Aquaculture. Wiley-Blackwell, Iowa.
Chen, C., Sun, B., Li, X., Li, P., Guan, W., Bi, Y., Pan, Q., 2013. N−3 essential fatty acids in
Nile tilapia, Oreochromis niloticus: quantification of optimum requirement of dietary
linolenic acid in juvenile fish. Aquaculture 416–417, 99–104.
Chew, V., 1976. Comparing treatment means: a compendium. HortSci. 11, 348–356.
Cruess, D.F., 1989. Review of use of statistics in the American Journal of Tropical Medicine
and Hygiene for January–December 1988. Am. J. Trop. Med. Hyg. 41, 619–626.
Davis, M.J., 2010. Contrast coding in multiple regression analysis: strengths, weaknesses,
and utility of popular coding structures. J. Data. Sci. 8, 61–73.
Emerson, J.D., Colditz, G.A., 1983. Use of statistical analysis in the New England Journal of
Medicine. N. Engl. J. Med. 309, 709–713.
FAO, 2014. The State of World Fisheries and Aquaculture 2014. Food and Agriculture Or-
ganization of the United Nations, Rome.
Felson, D.T., Cupples, L.A., Meenan, R.F., 1984. Misuse of statistical methods in arthritis and
rheumatism: 1982 versus 1967–68. Arthritis Rheum. 27, 1018–1022.
Gates, C.E., 1991. A user's guide to misanalyzing planned experiments. HortSci. 26,
1262–1265.
Gill, J.L., 1973. Current status of multiple comparison of means in designed experiments.
J. Dairy Sci. 56, 973–977.
Little, T.M., 1981. Interpretation and presentation of results. HortSci. 16, 637–640.
MacArthur, R.D., Jackson, G.G., 1984. An evaluation of the use of statistical methodology in
the Journal of Infectious Diseases. J. Infect. Dis. 149, 349–354.
Madden, L.V., Knoke, J.K., Louie, R., 1982. Considerations for the use of multiple compari-
son procedures in phytopathological investigations. Phytopathology 72, 1015–1017.
Montgomery, D.C., 1997. Design and Analysis of Experiments. fourth ed. John Wiley &
Sons, New York.
Olsen, C.H., 2003. Review of the use of statistics in infection and immunity. Infect. Immun.
71, 6689–6692.
Petersen, R.G., 1977. The use and misuse of multiple comparison procedures. Agron. J. 69,
205–208.
350 R. Yossa, M. Verdegem / Aquaculture 437 (2015) 344–350

Rafter, J.A., Abell, M.L., Braselton, J.P., 2002. Multiple comparison methods for means.
SIAM Rev. 44 (2), 259–278.
Robbins, K.R., Norton, H.W., Baker, D.H., 1979. Estimation of nutrient requirements from
growth data. J. Nutr. 109, 1710–1714.
Robbins, K.R., Saxton, A.M., Southern, L.L., 2006. Estimation of nutrient requirements
using broken-line regression analysis. J. Anim. Sci. 84 (E. Suppl), E155–E165.
Saville, D.J., 1990. Multiple comparison procedures: the practical solution. Am. Stat. 44,
174–180.
Steel, R.G., Torrie, J.H., Dickey, D.A., 1997. Principles and Procedures of Statistics: A Bio-
metrical Approach. third ed. McGraw-Hill Series in Probability and Statistics, Boston,
MA.
White, S., 1979. Statistical errors in papers in the British Journal of Psychiatry. Br.
J. Psychiatry 135, 336–342.
Zehra, S., Khan, M.A., 2013. Dietary isoleucine requirement of fingerling catla, Catla catla
(Hamilton), based on growth, protein productive value, isoleucine retention efficien-
cy and carcass composition. Aquac. Int. 21, 1243–1259.