Ecological Modelling 157 (2002) 119
/129

www.elsevier.com/locate/ecolmodel

Continuum theory revisited: what shape are species responses

along ecological gradients?
Jari Oksanen a,*, Peter R. Minchin b
a
Department of Biology, University of Oulu, 90014 Oulu, Finland
b
Department of Biological Sciences, Louisiana State University, 202 Life Sciences Building, Baton Rouge, LA 70803, USA

Abstract

The shape of species’ responses along ecological gradients has important implications for both continuum theory and
community analysis. Most current theories and analytical models in community ecology assume that responses are
unimodal and symmetric. However, interactions between species and extreme environmental stress may cause skewed
or non-unimodal responses. To date, statistical tools for evaluating response shapes have been either inappropriate,
inefficient or biased. Using a data set on vascular plant distributions along an elevation gradient, we show that
Huisman
/Olff
/Fresco (HOF) models are an effective method for this purpose, allowing models of various forms
(skewed, symmetric, plateau, monotonic) to be tested for adequacy. HOF modeling was compared with alternative
methods for response fitting, including Gaussian responses as Generalized linear model (GLMs), Generalized Additive
Models (GAM) and Beta Functions with fixed or estimated endpoints. In our data set, skewed and plateau responses
are less common than symmetric ones. Less than half of the species have skewed or plateau responses that can not be
adequately modeled by Gaussian models. We show that Beta function models with fixed endpoints are biased,
confounding skewness and the location of the mode and should not be used to test response shapes. Beta models with
estimated endpoints are fairly consistent with other models. GAM’s cannot provide clear tests of skewness or kurtosis
of response curves, though we show that GAM’s, in general, confirmed the shapes chosen by HOF modeling. We
provide free software for fitting HOF models and encourage further applications to community data collected along
different types of ecological gradients.
# 2002 Elsevier Science B.V. All rights reserved.

Keywords: Gradient analysis; Beta response function; Gaussian response function

1. Introduction ecological gradients (Gauch and Whittaker, 1972;

The canonized view in vegetation ecology is that
species have symmetric, unimodal responses to
* Corresponding author. Tel.: /358-8-553-1526; fax: /358-
8-553-1061
E-mail addresses: jari.oksanen@oulu.fi (J. Oksanen),
minchin@lsu.edu (P.R. Minchin).
0304-3800/02/$ - see front matter # 2002 Elsevier Science B.V. All rights reserved.
PII: S 0 3 0 4 - 3 8 0 0 ( 0 2 ) 0 0 1 9 0 - 4
Whittaker, 1978; Begon et al., 1996). At the
extreme, this leads into an unrealistic species
packing model, where all species are assumed to
have Gaussian response functions with identical
width and height but with uniformly located
optima (Gauch and Whittaker, 1972; ter Braak,
1985). Austin (1999, 2002) and many earlier
articles, shows clearly that this view is based on
120 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129
very scanty evidence and its canonical status is
unfounded. Indeed, theory predicts that response
shapes should differ among gradient types (Austin
and Smith, 1989) or gradient locations (Austin and
Gaywood, 1994). In addition, species interactions
may change the response shape even if the funda-
mental response were symmetric (Austin et al.,
1990). Therefore, the analysis of response shape is
of great theoretical interest (Austin, 1999). More-
over, it has a practical interest for ecologists
because of its implications for ordination metho-
dology. Correspondence analysis (CA) and its
derivatives, have been justified using the Gaussian
model (ter Braak, 1985), though the validity of
justification is open to debate (Austin, 1999, 2002).
The main alternative to CA-based methods, non-
metric multidimensional scaling, appears to be
robust to variation in response shapes (Minchin,
1987a).
The symmetric Gaussian response function has
been the only ecological response model for which
the parameters can be estimated effectively (ter
Braak and Barendregt, 1986; Jongman et al.,
1987). Although the evidence for the Gaussian
model can be criticized strongly (Austin, 1976,
1980, 1999, 2002) all the methods proposed so far
to test the significance of skewness of the response
have been either inadequate, inefficient or biased:
visual judgment of skewness (Økland, 1986) is
subjective; third-degree polynomials (Austin et al.,
1990) have unrealistic shapes; Beta functions
(Austin et al., 1994) confound the location of the
maximum and the skewness if the response end-
points are fixed (Oksanen, 1997); and the shape of
smoothed generalized additive models (GAM)
(Hastie and Tibshirani, 1990; Yee and Mitchell,
1991; Bio et al., 1998) must be assessed subjec-
tively. However, Huisman et al. (1993) have
proposed a set of hierarchical models which
include a skewed response that can be statistically
tested against a symmetric response. They assume
that these models cannot be estimated using
maximum likelihood (Huisman et al., 1993), but
we found that this can be done and so it is possible
to use more realistic error distribution for the
observations than the originally implied normal
distribution and to test statistically whether a
response is significantly skewed, or whether sym-
metric or monotonic responses could be used.
In this paper, we study the suitability of
Huisman
/Olff
/Fresco models (HOF) for detect-
ing the shape of species responses along gradients.
We compare HOF models against symmetric
Gaussian responses, variable Beta responses and
non-parametric GAM.
2. Material and methods
2.1. Data set
We analyzed the response patterns of vascular
plants along an altitude gradient on the Mt. Field
Plateau, Tasmania, which was earlier used to study
the shapes of response surfaces and some other
properties of community patterns (Minchin, 1989).
Vegetation was sampled with equally spaced
sample plots along transects with random starting
locations. We stratified the data by soil drainage
so that interactions with this second gradient
would not distort response shapes. We studied
only the well-drained subset in this paper and
fitted response models for all species, which
occurred on 20 or more sites in this subset.
Coverage of sample plots was dense and even in
this subset, as is obvious in the graphics in this
article. This gave us a data set of 167 sites and 52
species with an altitude range 930
/1380 m. We
analyzed the data as presence/absence, or binomial
with denominator m /1.
2.2. Bell-shaped or Gaussian response model
The Gaussian model defines a symmetric, bell-
shaped response m along gradient x with three
interpretable parameters: location of the optimum
u, height of the response h and tolerance or width
of the response t (Fig. 1):
 
(x  u)2
m h exp  (1)
2t2
Instead of direct estimation of Gaussian para-
meters, it is customary to fit an equivalent poly-
nomial model (ter Braak and Looman, 1986):
 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129
Fig. 1. Gaussian response function fitted to Bauera rubioides .
Continuous lines show a polynomial model with logit link (Eq.
(2)) and shading its approximate 95% confidence band. Broken
lines show the strict Gaussian model fitted to the same data.
mexp(b0 b1 xb2 x2 ) (2)
This can be easily fitted as a generalized linear
model (GLM) with a logarithmic link function
(McCullagh and Nelder, 1989; Guisan et al., 2002)
and the maximum likelihood estimates of the
Gaussian parameters can be easily found from
the polynomial coefficients (ter Braak and Loo-
man, 1986; Oksanen et al., 1988, 2001). Quite
commonly, the polynomial form is simply called a
Gaussian response without explicit reference to the
original form (Eq. (1)) and it is even used with link
functions other than logarithmic. With other link
functions, the general shape of the response is
similar to strict Gaussian, but the curve still differs
somewhat from the Gaussian and the width
parameter (t) has a different interpretation for
the ecologist (Heegaard, 2002): it no longer gives
the gradient distance at which the expected abun-
dance of species drops to a certain fraction of
expected maximum h (Fig. 1). In this paper, we
model the probability of occurrences of species
using Binomial m /1 error and apply the logistic
link function, as is customary with binomial
GLM’s, instead of the strictly correct logarithmic
link.
2.3. Beta response model
Austin (1976; Austin et al., 1994) proposed
replacing the Gaussian response function with
the Beta response function (Fig. 2):
121
Fig. 2. Beta response models fitted to B. rubioides with
endpoints p1 and p2 fixed either 10 or 50 m beyond the extreme
presences and a model where the endpoints were estimated
simultaneously with other parameters (the estimates were p1 /
/907 and p2 /1239 m, observed range 945
/1220 m).
m k(xp1 )a (p2 x)g (3)
This has five parameters and so it can find other
response shapes than the simple Gaussian re-
sponse. The Beta function has been found useful
for simulation of gradient responses, because it
can assume a wide range of unimodal shapes
(Minchin, 1987b). In principle, it should be
possible to fit skewed, unimodal responses with
various degrees of kurtosis. However, it is cus-
tomary to fix parameters p1 and p2 to be the
endpoints of the range of occurrence of species.
Originally this was seen only as a fitting device,
since in this way Eq. (3) simplifies into a special
case of GLMs (Kay and Little, 1987):
log(m)log(k)a log(xp1 )g log(p2 x) (4)
with explanatory variables log(x/p1) and log(p2/
x) and logarithmic link function. However, Austin
and Nicholls (1997) regard endpoints, which
define the range, as important ecological para-
meters. This leaves us only three free parameters,
just like in the Gaussian response function (Oksa-
nen, 1997): parameter k is a scaling parameter
adjusting the response height to fit the observa-
tions and parameters a and g determine (1) the
location of the optimum; (2) skewness and (3)
kurtosis. Obviously, there is one feature too many
for two parameters and therefore, the model is not
as versatile as suggested. However, it is suggested
122 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129

that a and g are mainly parameters of response cases where either a or g is negative, or where the
skewness (Austin and Nicholls, 1997). If a /g, the optimum lies outside the studied range.
response is symmetric (and the optimum in the
middle of the range), implying the model:
2.4. Huisman
/Olff
/Fresco or HOF model
log(m) log(k)a[log(xp1 )log(p2 x)] (5)
Huisman et al. (1993) proposed a set of five
with a single explanatory variable log(x1/p1)/ hierarchic models for species response. The most
log(p2/x). Eqs. (4) and (5) define nested GLMs complex model defines a skewed response function
and can be used to test the null hypothesis of and can be written in slightly modified form:
symmetry, a /g .
The selection of endpoints has a great effect on 1 1
m M (6)
fitted response shapes (Fig. 2) and Oksanen (1997) 1  exp(a  bx) 1  exp(c  dx)
suggested that they should be estimated together
Here, m is the expected value, which is depen-
with other model parameters, but suspected this to
dent on the known values of the gradient (x ),
be difficult. However, we found that the simulta-
maximum possible value (M ) and four estimated
neous fitting of all five parameters is relatively
parameters of the function (a, b , c , d ). This model
simple, but has to be done with non-linear max-
imum likelihood regression instead of more stan- can be simplified into four other models by
aliasing some parameters or fixing them to con-
dard GLM.
stant values (Table 1). These simplified models
We fitted Beta responses in two alternative
define symmetric, plateau type or monotone
ways. Firstly, we followed Austin and Nicholls
responses (Fig. 3).
(1997) and fixed the endpoints before fitting the
Huisman et al. (1993) presented only a least-
response as a GLM (Eq. (4)). The Beta function is
squares method for fitting these models and
undefined beyond the endpoints p1 and p2 and
suggested that maximum likelihood estimation
therefore, we pushed these parameters somewhat
outwards from the most extreme occurrences. The cannot be used. However, the use of maximum
likelihood is fairly simple (e.g. Venables and
degree of pushing has a great influence in the
Ripley, 1999). The maximum likelihood estimates
response shapes (Fig. 2). We used 50 m, since it
can be found by maximizing a log-likelihood
produced better fitting curves that were more
function instead of minimizing the squared resi-
similar to other response models than did a smaller
duals. The maximum likelihood function used in
extension. Austin and Meyers (1996), Austin and
the HOF model is non-linear in parameters a, b , c ,
Nicholls (1997) similarly pushed the endpoints
d and so these must be estimated using iterative
outwards by a fixed number of zero-observations.
The second alternative was estimation of p1 and non-linear methods (e.g. Venables and Ripley,
1999).
p2 together with other parameters. In this case, the
parameters p1 and p2 are selected to give a good fit
Table 1
for the observations and they have nothing to do
Simpler HOF models can be derived from the most complex
with the expected range of species (see Fig. 2). In model V (Eq. (6)) by fixing some parameters to constant values
both cases, we used Eq. (4) with logistic link
function instead of the strict form (Eq. (3)), since Model Parameters
we had binary responses. V Skewed a b c d
Beta response function is unimodal only if a
/0 ¯ ¯
IV Symmetric a¯ b c¯ b
¯ ¯
and g
/0 and so it can define monotone or III Plateau a b c¯ 0
¯
inverted responses (no optimum, but a minimum). II Monotone a¯ b 0¯ 0
¯ ¯
I Flat a 0 0 0
In this article, we interpreted the response shape as ¯
monotone if it was strictly monotone within The estimated parameters are underlined and the fixed
observations. Thus monotone responses include parameters in ordinary print.
 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129
Fig. 3. Hierarchic set of HOF models (Table 1) fitted to B.
rubioides . Model IV was rejected and the estimation continued
through model III which was found better than model II.
For statistical testing, the maximized likelihood
is transformed into a related statistic known as
deviance (Dobson, 1983; McCullagh and Nelder,
1989) which follows asymptotically the x2 distri-
bution. We study only Binomial m /1 models and
so we can base our testing directly on the deviance,
since overdispersion can occur only in Binomial
models where denominator m
/1 (McCullagh and
Nelder, 1989).
The selection of the final model is based on
backward elimination, starting with the most
complex model (V). The critical level used was
P /0.05. As the models form a hierarchic series
(Table 1), they are simplified in a pre-determined
order (McCullagh and Nelder, 1989), V; IV; II and
I (Table 1). Model III has the same number of
estimated parameters as model IV (Table 1) and
therefore, it is omitted from this sequence. How-
ever, if model IV is rejected in favor of Model V,
the latter is compared against model III and model
simplification is continued normally. This intro-
duces a bias against model III, because it may be
about as good- or even better-choice than model
IV and therefore, we will report results for both
models.
2.5. Generalized additive models
All previous models had a strict, parametric
functional form: we forced them to fit the data.
GAM do not specify any strict parametric func-
tion, but instead use non-parametric smoothers
123
(Hastie and Tibshirani, 1990). Consequently, they
may be able to produce shapes that differ from any
parametric model we have studied so far (Fig. 4).
GAM’s have recently become popular in gradient
modeling in vegetation science (e.g. Yee and
Mitchell, 1991; Bio et al., 1998; Leathwick, 1998;
Heegaard and Hangelbroek, 1999; Ejrnæs, 2000;
Lehmann et al., 2002; Zaniewski et al., 2002).
It may be difficult to characterize the shape of
response. If a parametric shape is completely
within the confidence band of a fitted GAM, the
response could be of that parametric shape. In this
way, we use GAM to verify or falsify our
parametric models and in seeking possible para-
metric forms.
We use smoothing splines for the predictor. The
number of degrees of freedom of the smoother is
selected by generalized cross validation (Gu and
Wahba, 1991), with upper limit alternatively 3 or 6
degrees of freedom. This was only the upper limit
for degrees of freedom: the actual degrees of
freedom were found by cross-validation (Wood,
2000; Wood and Augustin, 2002) and were gen-
erally much lower. The use of generalized cross-
validation is an improvement over the older
practice of comparing only some integer alterna-
tives of degrees of freedom.
It may be necessary to trim out extreme sites
beyond the species occurrence in multi-gradient
models, because GAM fits the model only for the
main effects instead of local neighborhood and so
zeros outside the main range of the species reduce
the fitted values within the main range (Hastie and
Tibshirani, 1990). However, we achieved the same
Fig. 4. GAM fitted to B. rubioides with approximate 95%
confidence intervals.
124 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129

by stratifying the data for the secondary gradient 3. Results

(drainage) and so we could use an identical set of
sites for all models.
2.6. Software
Most statistical analyses were performed using
the R software (Ihaka and Gentleman, 1996),
which is ‘not unlike S-PLUS’ (Venables and Ripley,
1999). Most of the analyses are identical in S-PLUS,
but in some cases the algorithms or user com-
mands differ slightly. R is available for free down-
load for several platforms and operating systems
at http://cran.r-project.org/.
Gaussian and Beta response functions with fixed
endpoints were fitted as GLMs (function glm).
GAM are not included in base R, but they are in
the recommended library mgcv (function gam).
The algorithm used differs somewhat from S-PLUS
(Wood, 2000; Wood and Augustin, 2002).
Maximum likelihood fitting of non-linear HOF
models and Beta responses with estimated ‘end-
points’ can be performed with non-linear mini-
mization (function nlm). This is different from S-
PLUS that uses function ms. However, most HOF
models were initially fitted using a stand-alone
program HOF written by Jari Oksanen (JO).
The program HOF is available through web
pages of JO as compiled binaries for MS-DOS/
WINDOWS and LINUX. R functions for the analyses
and further discussion on analysis methods are
available at the same address.
According to HOF models, symmetric responses
were the most common response type (Table 2).
These symmetric responses could be adequately
fitted using the Gaussian model. About 21% (11 of
52) of responses were skewed which is substan-
tially less than the 33% reported by Minchin
(1989). In contrast, Beta functions with fixed
endpoints found 37% (19 out of 52) as significantly
skewed. However, this was caused by fixing the
endpoints before fitting the Beta model, so that
skewness and location of optima were confounded
(Oksanen, 1997) and Beta functions with freely
estimated ‘endpoints’ were more consistent with
HOF models (Table 2). Actually, most ‘skewed’
Beta responses with fixed endpoints were very
similar to strictly symmetric Gaussian and HOF
IV responses and ‘symmetric’ Beta responses fitted
the data badly and differed from all alternative
models (Fig. 5). Technically ‘skewed’ Beta re-
sponses (in the sense that a "/g ) with fixed end-
points were actually almost symmetric in most
cases: instead of producing an asymmetric peak,
different a and g were used for better fitting of
tails, or locating the peak in other places than at
the average of the endpoints.
All four response models produced very similar
responses for species, which were regarded as
symmetric, unimodal HOF model IV (Fig. 5).
Any of the alternative response models tested
seems to be adequate for such species, which are
the largest group in our data (Table 2). Similarly,

Table 2
The most parsimonous HOF models cross-tabulated against the most parsimonous Beta response models

HOF model Total Beta
/estimated endpoints Beta
/fixed endpoints

Skewed Symmetric Monotone Skewed Symmetric Monotone

V 11 9 1 1 7 3 1
IV 22 4 17 1 12 10
III 4 3 1 4
II 12 1 2 9 4 8
Total 49 14 23 12 19 21 9

The endpoints were fixed 50 m beyond the most extreme presences. One species had an inverted response (no optimum but a
minimum) in all models and two species with flat response (model I) were flat or inverted in all other models and these three are not
tabulated. The total number of species before these omissions was 52.
 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129
Fig. 5. Examples of species which were found to have symmetric bell-shaped responses by HOF models: HOF model IV, Gaussian
bell, ‘skewed’ Beta and GAM (df5/3) produced almost similar response, but ‘symmetric’ Beta differs sharply from all others.
species classified as HOF model II (monotone
response) were very similar in all models.
Species classified as skewed responses (HOF
model V) or plateau responses (HOF model III)
are more interesting. These responses cannot be
adequately fitted by Gaussian responses or Beta
responses with fixed endpoints and therefore, we
compare them against GAM responses and Beta
with estimated endpoints. Our comparison was
biased in favor of model IV, because we studied
model III only if model IV failed. In fact, model
III was an alternative parsimonious choice for nine
species out of 22 model IV species (Table 2) and
for five of these it was indeed a better choice (lower
deviance) than model IV. We used this rigorous
but discriminatory strategy because we think that
a plateau cannot be a global response model, but
valid only for a limited range on a gradient.
It seems that GAM models with df5/3 are
usually more symmetric than skewed HOF re-
sponses (Fig. 6). However, GAM models with
df 5/6 usually confirm the main peak of the
skewed HOF response, but deviate from it in
other parts, being usually less regular (Fig. 6).
Either the true response is less regular than
assumed in parametric modeling, or HOF has a
sharper peak than GAM is allowed to have. In
general, HOF V models are within confidence
limits of GAM’s, so that the response looks
credible (Fig. 6). Similarly, Beta responses with
estimated endpoints were fairly consistent with
HOF models (Table 2).
125
Plateau responses (HOF model III) are uncom-
mon and were found only for four species. All
these are within the confidence intervals of GAM
responses (Fig. 7). GAM responses were not as
regular as the simple parametric plateaux of HOF
model III. In some cases the GAM response
resembles the skewed HOF model V, which was
not retained in backwards selection.
4. Discussion
Symmetric, bell-shaped responses were the most
common type at Mt. Field and only about one
fifth of the responses were clearly skewed. About
42% of species could have been fitted using
Gaussian response models. At first sight, these
conclusions appear to be in contrast with Minchin
(1989) who wrote that ‘‘Although the majority of
response surfaces appear to be unimodal, only
45% are symmetric’’. Actually, despite important
differences between two studies (the original paper
examined mean cover responses, rather than
probabilities of occurrence and fitted two-dimen-
sional response surfaces to both altitude and soil
drainage), the observed proportion of symmetric
bell responses was similar in both analyses. How-
ever, in this paper, we found a larger number of
monotone responses (HOF models III, II and I).
Minchin (1989) used non-parametric smoothing
and had to interpret the shapes subjectively with-
out help of confidence intervals. Obviously his
126 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129

Fig. 6. GAM (df5/6) (solid line) with approximate 95% confidence intervals against GAM(df5/3) (dotted line) and HOF V responses
(broken line).

smoothing was not sufficient: in present parlance,
he used too many degrees of freedom in fitting.
Consequently, he emphasized the irregularity of
many responses (regarding 22% of species as
having multimodal surfaces), whereas our present
analyses did stronger smoothing and found most
responses to be fairly regular. Nevertheless, the
proportion of non-Gaussian responses is so large
that it is hardly possible to say that the Gaussian is
the universal and general response type. In parti-
cular, one third (11/33) of unimodal responses
(models V and IV) are skewed rather than sym-
metric.
Austin et al. (1994) have proposed strict rules
for selecting the species that can be studied for
their response shapes. For instance, there should

Fig. 7. HOF models III responses (broken line) and GAM (df5/3) (solid line) with approximate 95% confidence intervals.
 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129
be at least 100 sites with zero observations beyond
the extreme presence. Such rules may be applicable
to huge data sets at the subcontinental scale, but
they are hardly practical for ordinary vegetation
surveys. Moreover, even in subcontinental data
sets, the rule results in a biased selection of species,
leaving out those with modes closer to the gradient
extremes. Gradient limits may be natural, like the
Mt. Field peak at 1380 m in our case, or they may
be arbitrary, like the lower limit of 930 m in our
case. Austin’s rule does not distinguish between
these two cases. The main difference between our
approaches is that we are mainly interested in the
shape of the species response at the central area of
species occurrence, whereas Austin seems more
interested in species range and the shapes of the
tails. An essential feature of response shape is the
slope of decrease near the species optimum and
this can be studied even with truncated responses.
Contrary to Austin and Nicholls (1997), we think
that there is no general need to remove extreme
zeros, as most response models are able to predict
near-zero occurrence for these. The trimming of
zero tails is necessary only for Beta responses with
fixed endpoints, since the response is undefined
beyond these points (Austin et al., 1994; Austin
and Nicholls, 1997; Oksanen, 1997). Trimming of
zeros may be necessary in GAM with several
gradients, since GAM fit the response for each
dimension separately without regarding interac-
tions (Hastie and Tibshirani, 1990). For Gaussian
responses, HOF models or Beta models with freely
estimated endpoints, there is no need to remove
any extreme zeros, since these functions can fit
these adequately.
Beta response functions with fixed endpoints
failed badly in our analyses. After fixing the
response endpoints, or the species range (Austin
and Nicholls, 1997), the responses either fitted the
data very badly or else failed to detect the
appropriate shape because they preferred to find
the location of the optimum instead of skewness
(Oksanen, 1997). Indeed, Oksanen (1997) sug-
gested that the response endpoints should not be
fixed but estimated together with other para-
meters, so that response curve could try to find
both the skewness and the location of the optimum
127
independently and in our case this really improved
the applicability of Beta functions.
Fixing the endpoints at the location of the most
extreme presences ignores the fact that the data are
only a sample. A unimodal response with tapering
tails implies that the probability of occurrence near
the extremes will be very low. Even with a very
large data set, it is, therefore, unlikely that a
presence will be observed at the real limit of
distribution. Beta response models may be very
effective for simulating realistic response patterns
along ecological gradients (Minchin, 1987b), but
our results show that they are unsuitable for
analyzing response shapes, as also concluded by
Austin and Meyers (1996).
Any of the studied alternative models could be
used for symmetric, bell-shaped responses, even
the Beta response model with estimated ‘end-
points’. However, for other response types, Beta
functions and Gaussian response models provide
biased or inappropriate models. GAM’s are
usually very similar to the selected HOF models,
but smoothness and regularity of shape is depen-
dent on the number of degrees of freedom (Hastie
and Tibshirani, 1990). We used generalized cross-
validation for selecting the effective smoothness
(Gu and Wahba, 1991). Hastie and Tibshirani
(1990) found that this fails in several cases and
produces responses that are too irregular. There-
fore, we defined a fairly low maximum limit for
degrees of freedom. However, for detecting skew-
ness, a somewhat higher limit must be used,
although this comes at cost of an irregular,
perhaps biased pattern in other parts of the
response.
We analyzed a real data set and therefore, we
cannot know the true underlying response shapes
and so we do not know if HOF models actually
were correct. However, we compared HOF models
against GAM’s and symmetric and monotone
models against other alternatives and found con-
sistent patterns. In most cases, the final selected
HOF model was entirely within the approximate
confidence limits of a GAM. The most notable
differences were near the extremes of the gradient,
where the GAM often flipped up, whereas the
corresponding HOF model decreased asymptoti-
cally. However, the confidence bands in fitted
128 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129
GAM models increase in the margins as well, so
that obviously this is caused by uncertainty at the
margins and does not reflect real response shapes.
Another inconsistency was in skewed models when
compared against strongly smoothed GAM’s, but
the location and shape of the main peak became
consistent with less strong smoothing in GAM, at
cost of differences near the gradient margins.
GAM’s and HOF models were usually consis-
tent and either could be used in gradient analysis.
Recently, GAM’s have enjoyed considerable po-
pularity and are seen by some authors as an
optimal solution (Yee and Mitchell, 1991; Bio et
al., 1998; Leathwick, 1998; Heegaard and Hang-
elbroek, 1999; Ejrnæs, 2000). The strongest feature
of GAM’s is that they are indeed non-parametric
and can take any smooth shape, whereas even
HOF models are restricted to certain shapes. For
instance, multimodal responses may be detected
with GAM’s but are beyond the scope of HOF
models. However, the shape of a fitted GAM
response must be interpreted subjectively and it
may be difficult to say whether a response is really
skewed or significantly skewed, or could be just as
adequately described in other terms. Shape is, after
all, a parametric concept: if we say that a response
is skewed, the only difference compared with a
symmetric response should be the skewness.
GAM’s may be a good choice for general pre-
dictive modeling of species responses, but for
parametric questions of shape, parametric HOF
models appear to provide a better alternative.
Many more studies of response patterns along
ecological gradients are needed to provide a sound
observational basis for continuum theory. HOF
models provide an effective tool for studying the
shapes of species’ responses and we hope that the
availability of software for HOF modeling will
encourage the reexamination of existing data sets
and stimulate further investigations into the nature
of community patterns.
Acknowledgements
We wish to thank Dr Mike Austin and Dr Alain
Zuur for invaluable comments on the manuscript.
References
Austin, M.P., 1976. On non-linear species responses models in
ordination. Vegetatio 33, 33
/41.
Austin, M.P., 1980. Searching for a model for use in vegetation
analysis. Vegetatio 42, 11
/21.
Austin, M.P., 1999. On silent clash of paradigms: some
inconsistencies in community ecology. Oikos 86, 170
/178.
Austin, M.P., 2002. Spatial prediction of species distribution:
an interface between ecological theory and statistical
modelling. Ecol. Model. 157 (2
/3), 101
/118.
Austin, M.P., Smith, T.M., 1989. A new model for the
continuum concept. Vegetatio 83, 35
/47.
Austin, M.P., Gaywood, M.J., 1994. Current problems of
environmental gradients and species response curves in
relation to continuum theory. J. Veg. Sci. 5, 473
/482.
Austin, M.P., Meyers, J.A., 1996. Current approaches to
modelling the environmnetal niche of eucalypts: implica-
tions for management of forest biodiversity. Forest Ecol.
Manage. 85, 95
/106.
Austin, M.P., Nicholls, A.O., 1997. To fix or not to fix the
species limits, that is the ecological question: response to
Jari Oksanen. J. Veg. Sci. 8, 743
/748.
Austin, M.P., Nicholls, A.O., Margules, C.R., 1990. Measure-
ment of the realized qualitative niche: environmental niches
of five Eucalyptus species. Ecol. Monogr. 60, 161
/177.
Austin, M.P., Nicholls, A.O., Doherty, M.D., Meyers, J.A.,
1994. Determining species response functions to an environ-
mental gradient by means of a b-function. J. Veg. Sci. 5,
215
/228.
Begon, M., Harper, J.L., Townsend, C.R., 1996. Ecology:
Individuals, Populations and Communities, third ed.. Black-
well.
Bio, A.M.F., Alkemade, R., Barendregt, A., 1998. Determining
alternative models for vegetation response analysis: a non-
parametric approach. J. Veg. Sci. 9, 5
/16.
Dobson, A.J., 1983. An Introduction to Statistical Modelling.
Chapman and Hall.
Ejrnæs, R., 2000. Can we trust gradients extracted by detrended
correspondence analysis. J. Veg. Sci. 11, 565
/572.
Gauch, H.G., Jr, Whittaker, R.H., 1972. Coenocline simula-
tion. Ecology 53, 446
/451.
Gu, C., Wahba, G., 1991. Minimizing GCV/GML scores with
multiple smoothing parameters via the Newton method.
SIAM J. Sci. Stat. Comp. 12, 383
/398.
Guisan, A., Edwards, T.C., Hastie, T., 2002. Generalized linear
and generalized additive models in studies of species
distributions: setting the scene. Ecol. Model. 157 (2
/3),
89
/100.
Hastie, T.J., Tibshirani, R.J., 1990. Generalized Additive
Models. Chapman and Hall, London, p. 92.
Heegaard, E., 2002. The outer border and central border for
species environmental relationships estimated by non-para-
metric generalized additive models. Ecol. Model. 157 (2
/3),
131
/139.
 J. Oksanen, P.R. Minchin / Ecological Modelling 157 (2002) 119
/129
Heegaard, E., Hangelbroek, H.H., 1999. The distribution of
Ulota crispa in relation to both dispersal- and habitat-
related factors. Lindbergia 24, 65
/74.
Huisman, J., Olff, H., Fresco, L.F.M., 1993. A hierarchical set
of models for species response analysis. J. Veg. Sci. 4, 37
/
46.
Ihaka, R., Gentleman, R., 1996. R: a language for data analysis
and graphics. J. Comput. Graph. Stat. 5, 299
/314.
Jongman, R.H., ter Braak, C.J.F., van Tongeren, O.F.R., 1987.
Data analysis in community and landscape ecology, Pudoc,
Wageningen.
Kay, R., Little, S., 1987. Transformations of the explanatory
variables in the logistic regression model for binary data.
Biometrika 74, 495
/501.
Leathwick, J.R., 1998. Are New Zealand’s Nothofagus species
in equilibrium with their environment. J. Veg. Sci. 9, 719
/
732.
Lehmann, A., Overton, J.M., Leathwick, J.R., 2002. General-
ized regression analysis and apatial predictions. Ecol.
Model. 157 (2
/3), 187
/205.
McCullagh, P., Nelder, J.A., 1989. Generalized Linear Models.
Chapman and Hall, London, p. 125.
Minchin, P.R., 1987a. An evaluation of relative robustness of
techniques for ecological ordinations. Vegetatio 69, 89
/107.
Minchin, P.R., 1987b. Simulation of multidimensional commu-
nity patterns: towards a comprehensive model. Vegetatio 71,
145
/156.
Minchin, P.R., 1989. Montane vegetation of the Mt. Field
massif, Tasmania: a test of some hypotheses about proper-
ties of community patterns. Vegetatio 83, 97
/110.
Økland, R.H., 1986. Rescaling of ecological gradients. The
effect of scale on symmetry of species response curves.
Nordic J. Bot. 6, 671
/677.
Oksanen, J., 1997. Why the beta-function cannot be used to
estimate skewness of species responses. J. Veg. Sci. 8, 147
/
152.
129
Oksanen, J., Läärä, E., Huttunen, P., Meriläinen, J., 1988.
Estimation of pH optima and tolerances of diatoms in lake
sediments by the methods of weighted averaging, least-
squares and maximum likelihood, and their use for the
prediction of lake acidity. J. Paleolimnol. 1, 39
/49.
Oksanen, J., Läärä, E., Meriläinen, J., Warner, B.G., 2001.
Confidence intervals for the optimum in the Gaussian
response function. Ecology 82, 1191
/1197.
ter Braak, C.J.F., 1985. Correspondence analysis of incidence
and abundance data: properties in terms of a unimodal
response model. Biometrics 41, 859
/873.
ter Braak, C.J.F., Barendregt, L.G., 1986. Weighted averaging
of species indicator values: its efficiency in environmental
calibration. Math. Biosci. 78, 57
/72.
ter Braak, C.J.F., Looman, C.W.N., 1986. Weighted averaging,
logistic regression and the Gaussian response model.
Vegetatio 65, 3
/11.
Venables, W.M., Ripley, B.D., 1999. Modern Applied Statistics
with S-PLUS, third ed.. Springer, Heidelberg.
Whittaker, R.H., 1978. Direct gradient analysis. In: Whittaker,
R.H. (Ed.), Ordination of Plant Communities. Junk, The
Hague, pp. 7
/50.
Wood, S.N., 2000. Modelling and smoothing parameter
estimation with multiple quadratic penalties. J. Roy. Stat.
Soc. B62, 413
/428.
Wood, S.N., Augustin, N.H., 2002. Improving GAMs for
environmental modelling using GCV and penalized regres-
sion splines. Ecol. Mod., This Volume, in this issue.
Yee, T.W., Mitchell, N.D., 1991. Generalized additive models
in plant ecology. J. Veg. Sci. 2, 587
/602.
Zaniewski, A.E., Lehmann, A., Overton, J.M., 2002. Predicting
species distribution using presence-only data: A case study
of native New Zealand ferns. Ecol. Model. 157 (2
/3), 259
/
278.