Generalized Linear Mixed
Definition
Models
Introduction
Generalized linear mixed models (GLMMs) are an
extension of the class of generalized linear models
in which random effects are added to the linear pre-
dictor. This modification extends the broad class of
generalized linear models to accommodate correla-
tion via random effects, while retaining the ability to
model nonnormal distributions and allowing nonlin-
ear models of specific form. The class of GLMMs
includes the special cases of linear mixed models,
random coefficient models, random effects logistic
regression, and random effects Poisson regression,
to name a few.
The incorporation of random effects is a natu-
ral way to model or accommodate correlation in the
context of a nonlinear model for nonnormal data. It
generates a rich class of correlated data models that
would be difficult to specify directly. Readily avail-
able, flexible, multivariate distributions analogous
to the multivariate normal distribution do not exist
for most nonnormally distributed data.
Inferences for these models can be of the usual
variety, that is, modeling the effect of predictors on
the mean, in which case the random effects and cor-
relation are “nuisance” features of the model. In other
situations, however, both estimation and testing of the
variances of the random effects, as well as prediction
of the realized values of the random effects, may be
of interest (see Variance Components).
We will illustrate several of our points using as
an example the longitudinal study of physicians and
their patients described by Korff et al. [8]. This study
classified 44 primary care physicians in a large HMO
according to their practice styles in treating back pain
management (low, moderate, or high frequency of
prescription of pain medication and bed rest), and
followed an average of 24 patients per physician for
2 years (1 month, 1 year, and 2 year follow-ups) after
the index visit. Outcome variables included functional
measures (e.g. Did you experience moderate to severe
activity limitation?), patient satisfaction (e.g. “After
your visit with the doctor, did you fully understand
how to take care of your back problem?”), and cost.
Encyclopedia of Biostatistics, Online © 2005 John Wiley & Sons, Ltd.
This article is © 2005 John Wiley & Sons, Ltd.
This article was published in the Encyclopedia of Biostatistics in 2005 by John Wiley & Sons, Ltd.
DOI: 10.1002/0470011815.b2a10021
Generalized Linear Mixed Models: A
Generalized linear mixed models constitute a class of
models for describing the stochastic relationship of
an n-dimensional outcome vector Y to an (n × p)-
dimensional matrix of covariates X, with rows x
i .
The construction of generalized linear mixed mod-
els begins with the specification of a generalized
linear model conditional on a vector u of random
effects. That is, given a vector u (often with compo-
nents specific to a subject or cluster), the conditional
density of Yi is of the exponential family form
f (yi |u) = exp[{yi θi − b(θi )}φ + c(yi , φ)], where b
and c are functions of known form. In addition, one
assumes that E(Yi | u, x i ) = g −1 (x
i β + z
i u), where
zi is a specified vector of covariates, analogous to
x i . Given u, the model additionally assumes that the
responses Yi are independent. The function g links the
linear predictor to the expected value of the response.
The model further assumes that the random effects
u follow a distribution G, typically (but not neces-
sarily) multivariate normal with mean vector 0 and
covariance matrix Σ(γ ), where γ is a vector of
(co)variance parameters, for example, variances and
correlation coefficients.
Thus, the model assumes that the linear predic-
tor consists of two portions: the fixed effects portion
x
i β, and the random effects portion z
i u, for which a
distribution is assigned to u. Just as with linear mixed
models, the assumption of a distribution for the ran-
dom effects induces correlations among observations.
Finally, the assumptions underlying GLMMs specify
the multivariate distribution of Y = (Y1 , . . . , Yn )
, so
that one can base inference with these models on
likelihood methods.
The specification of covariate effects conditional
on random effects determines the interpretation of the
fixed effects parameters β. For example, considering
the activity limitation outcome in the back pain study,
GLMMs would measure how the risk of activity lim-
itation in a particular patient of a particular physician
changes over time, and how that change relates to
the practice style of the physician. Using GLMMs,
one can also directly relate changes in explanatory
variables within an individual subject to changes in
the expected value of the subject’s response.
To be more specific, we consider the simple and
common case in which the data are correlated in clus-
ters, where i = 1, . . . , m indexes the clusters (e.g.
2 Generalized Linear Mixed Models
patients) while j = 1, . . . , ni indexes units within
clusters (e.g. different time points) (see Cluster Sam-
pling). In a GLMM with a single fixed effect, the
parameter β measures the change in the conditional
expectation of Y corresponding to a unit increase in
the covariate within the ith cluster,
β = g[E(Yij |xij + 1, ui )] − g[E(Yij |xij , ui )], (1)
where ui is a cluster-specific vector of random
effects. This contrasts with marginal models, where
one specifies the marginal or population-averaged
(PA) distribution of the response of the j th unit in
the ith cluster, integrated over the distribution of u,
together with some working (hypothesized) covari-
ance structure for the ni responses in the ith cluster
to account for intraclass correlation. For the single
fixed effect example, marginal models measure the
change in the marginal expectation E(Yij |xij ), uncon-
ditional on the random effects, associated with change
in the covariate. Such covariate effects are exactly
those one would estimate with a single response per
subject; the cluster structure thus plays no role in the
interpretation of the model regression coefficients.
In addition to estimates of the effects of covariates
on the expected value of the response, GLMMs can
provide estimates of the dependence of responses
within clusters, such as subjects. Measures such as
the intraclass correlation coefficient, corr(Yij , Yij
),
depend on the random effects distribution G, along
with its parameters γ ; using estimates of γ one can
construct estimates of intraclass correlation.
As well as modeling within-cluster response de-
pendence and estimates of its magnitude, GLMMs
allow consideration of the individual random effects,
which themselves may be of interest. For example, in
the back pain study, we would include random effects
to describe both the physician and patient effects on
each of the outcomes. We might be interested in
obtaining predicted values for the random effects of
each physician to help indicate which physicians had
better outcomes and/or lower costs, after adjusting
for fixed effects of model covariates. The random
effects in a GLMM are best predicted by their
conditional expectations given the data, E(u|Y ).
However, this expectation is unknown since it
depends on the unknown parameters β and γ . Hence,
one typically estimates E(u|Y ) using estimates of
these parameters. The estimated values of E(u|Y ) are
shrinkage estimates and “borrow strength” across the
data set in order to improve estimates of individual
Encyclopedia of Biostatistics, Online © 2005 John Wiley & Sons, Ltd.
This article is © 2005 John Wiley & Sons, Ltd.
This article was published in the Encyclopedia of Biostatistics in 2005 by John Wiley & Sons, Ltd.
DOI: 10.1002/0470011815.b2a10021
random effects, especially when data incorporating a
particular random effect are sparse (see Shrinkage
Estimation).
Inference and Estimation
Maximum likelihood (ML), or variants such as
restricted maximum likelihood (REML), are stan-
dard methods of estimation for linear mixed mod-
els and generalized linear models (e.g. logistic
regression). Evaluation of the likelihood and hence
likelihood inference with GLMMs is computation-
ally difficult, however, because the random effects
on which the likelihood is conditioned must be inte-
grated out of the distribution prior to maximization
as a function of the fixed effects. Although sev-
eral useful computational methods currently exist, the
development of new methods for GLMMs continues
to be an active research area.
To illustrate the inherent complexity, consider a
general mixed logistic regression model for binary
data. The marginal likelihood takes the form
 



··· exp Yi (x
i β + z
i u)
i

× {1 + exp(x
i β + z
i u)}−1 dG(u), (2)
i
where G is the distribution function of the random
effects and the integration is of a dimension equal
to the dimension of u. For most choices of G, the
integral cannot be evaluated in closed form although,
for simple cases like random intercept and/or random
slope models, (1) reduces to a product of lower-
dimensional integrals amenable to numerical integra-
tion. Numerical integration becomes inaccurate for
three or more dimensions, but simulation-based meth-
ods such as Markov Chain Monte Carlo [3], in par-
ticular, Gibbs sampling, and Monte Carlo EM [13]
have proven useful in these settings. Such methods
can also handle complications such as crossed ran-
dom effects.
If ML estimation is feasible, then the usual infer-
ential methods are available. In particular,
• ML estimators are asymptotically normal, with
standard errors available from second deriva-
tives of the log likelihood.

• One can carry out hypothesis tests using like-
lihood ratio, score, or Wald procedures.
• One can calculate best-predicted values as ex-
pected values of the random effects conditional
on the data, substituting ML or REML esti-
mates for unknown parameters. Typically, one
cannot evaluate the conditional expected val-
ues in closed form, so these calculations involve
numerical integration.
• One can test whether variances of random
effects are zero using the likelihood ratio
statistic. As with linear mixed models, the
asymptotic null distribution involves a mixture
of chi-square distributions rather than the null
chi-square distribution usual in fixed effects
models [15].
For the case of random intercepts only, sev-
eral authors have proposed a semiparametric mixed
model approach that jointly estimates the regres-
sion parameters and the (nonparametric) mixing
distribution G. These methods are given in several
papers, including [1, 2, 9, 10, 11]. This approach
provides consistent estimation of the effects of all
covariates and of G under conditions of identifiabil-
ity [7].
Although GLMMs require specification of the ran-
dom effects distribution G, several studies of the
performance of logistic models with random inter-
cepts show that estimates of the fixed effects param-
eters β are robust to misspecification of G. For
example, using both approximations and simulations,
Neuhaus et al. [14] showed that incorrectly assum-
ing that G was Gaussian produced fixed effects
covariate effect estimates with very little bias (see
Unbiasedness). Heagerty and Kurland [5] corrob-
orated these findings but pointed out that more
severe model misspecifications, such as the fail-
ure to model interactions of covariates and random
effects, could yield biased estimates of covariate
effects.
Contrasting the Marginal and Conditional
Approaches
We return to the special case of correlated clusters and
the notation used previously to describe them. Many
investigators have considered alternative methods for
clustered data that focus on models for the marginal
Encyclopedia of Biostatistics, Online © 2005 John Wiley & Sons, Ltd.
This article is © 2005 John Wiley & Sons, Ltd.
This article was published in the Encyclopedia of Biostatistics in 2005 by John Wiley & Sons, Ltd.
DOI: 10.1002/0470011815.b2a10021
Generalized Linear Mixed Models 3
expectation of the response,
E(Yij | x ij )




= y · · · f (y|x ij , ui ) dG(ui ) dy, (3)
where x ij is the vector of covariate values associated
with Yij . Some approaches, such as Generalized
Estimating Equations (GEEs), do this without fully
specifying the functional form of the joint distribution
of the responses Yi1 , . . . , Yini within the ith cluster.
GLMM and marginal models are similar in that
both parameterize the mean and covariance matri-
ces of correlated groups of observations, and both
base inferences on marginal likelihoods or marginal
quasi-likelihoods of the observed data. However, the
implications of modeling the response distribution
conditional on random effects, as do GLMMs, rather
than averaged over random effects, as do marginal
models, are profound. With a nonlinear link function,
the impact of a conditional main effect in the linear
predictor varies, on the scale of E(Y ), with the values
of accompanying fixed and random effects. Hence,
in contrast with the linear case, when averaging
over the random effect distribution, the mean linear
predictor does not correspond (transform to) to the
marginal conditional expectation Eui [E(Yij |x ij , ui )].
Similarly, linear predictors based on within-group
covariate means do not transform to means of the
response within the corresponding groups. Hence, as
stated earlier, marginal approaches measure concep-
tually and numerically different covariate effects than
do GLMMs. In some cases, scientific interest and
inferential goals of a problem lead one clearly to the
marginal or conditional specification of a model; in
other cases, the appropriate direction is less clear.
Since the distinctions between marginal models and
GLMMs are commonly blurred in practice, it is useful
to contrast them further.
Marginal models are most helpful when corre-
lation among observations cannot be ignored, but
neither the nature of the clustering that generates
such correlations, nor the individual clusters them-
selves, are otherwise of particular scientific interest.
This characterization often applies when public health
impact is the focus of an investigation. From that
perspective, there may be little concern with either
intracluster factors or with predictions about the spe-
cific units, such as families observed cross-sectionally
or individuals observed over time, that generate the
4 Generalized Linear Mixed Models
measurement clusters. In the back pain study men-
tioned earlier, in which practice style was constant
within subjects over time, marginal modeling would
be natural to study a presumed homogeneous effect
of practice style across subjects and times, and any
population time trend. (However, it would not be use-
ful for quoting the odds of reduction over time in,
say, the risk of activity limitation for an individual
patient.)
In such circumstances, for example, where lon-
gitudinal observation is used for logistical reasons
or statistical efficiency, and correlations are nui-
sance parameters in analysis, then, there is techni-
cal advantage in bypassing a conditional model (see
Efficiency and Efficient Estimators). For example,
GLMMs that mistakenly assume random effects that
are homoscedastic can produce biased estimators [5,
6]. In contrast, estimates of marginal model parame-
ters obtained using GEE are consistent, even if the
association structure is misspecified.
In other circumstances, however, marginal models
may not measure covariate effects of primary scien-
tific interest, for example, in longitudinal studies in
which explanatory variables change over time within
a subject and interest is in how individuals respond
to such changes. In the back pain study, investigators
were interested specifically in assessing patterns of
change over time in individual subjects. In such sit-
uations, GLMMs are more ambitious than marginal
models in attempting to (i) parse, using explicit ran-
dom effects and predicted values for them, sources
of variation that produce correlated observations, and
to (ii) portray and predict, for instance, shapes of
individual longitudinal disease trajectories, vulnera-
bilities of individual litters to teratogenesis, breeding
values of individual bulls, and susceptibilities of indi-
vidual families to inheritable diseases. The price of
this ambition is paid in more stringent assumptions,
and in greater complexity of the model fitting process
and computations.
If one is willing to pay such a price, then GLMMs
may be used to make inferences about marginal
distributions, even when purely marginal methods
would be adequate. Marginal modeling, however,
without such assumptions does not allow conditional
inference, for example, about longitudinal trajectories
typical of individual subjects. More specifically,
• Variations of Simpson’s paradox and the
ecologic fallacy may apply at several levels.
Encyclopedia of Biostatistics, Online © 2005 John Wiley & Sons, Ltd.
This article is © 2005 John Wiley & Sons, Ltd.
This article was published in the Encyclopedia of Biostatistics in 2005 by John Wiley & Sons, Ltd.
DOI: 10.1002/0470011815.b2a10021
The marginal distribution may be of a different
form from any conditional distribution and,
indeed, the form of a conditional mixing
distribution required for common marginal
models may be quite unusual [17]. In extreme
cases, features in every conditional model
may be absent in the marginal model, for
example, the marginal effect averaged across
2 × 2 conditional tables might be opposite in
direction to those in each conditional table.
More commonly, however, population average
effects may simply understate the strengths of
effects on individuals [18].
• GEE for marginal models may not estimate the
variance–covariance structure efficiently and
does not allow, without further assumptions,
prediction of random effects. However, see [4,
16, 18] for developments in these directions.
For a more detailed critique of marginal modeling,
see [12].
Summary
During the past decade, GLMMs have become an
important statistical tool and now see heavy use for
modeling correlated, nonnormally distributed data.
Software for fitting GLMMs is starting to mature,
and much experience has contributed to better appre-
ciation of both the utility and pitfalls of the currently
available techniques. GLMMs are a natural modeling
approach for longitudinal data when changes within
subjects are of interest.
References
[1] Butler, S.M. & Louis, T.A. (1992). Random effects
models with non-parametric priors, Statistics in Medicine
11, 1981–2000. Disc:2017–2023.
[2] Follmann, D. & Lambert, D. (1989). Generalizing logis-
tic regression by nonparametric mixing, Journal of the
American Statistical Association 87, 295–300.
[3] Gilks, W.R., Richardson, S. & Spiegelhalter, D. (1996).
Markov Chain Monte Carlo in Practice. Chapman &
Hall, London.
[4] Heagerty, P. (1999). Marginally specified logistic-
normal models for longitudinal binary data, Biometrics
55, 688–698.
[5] Heagerty, P. & Kurland, B. (2001). Misspecified maxi-
mum likelihood estimates and generalised linear mixed
models, Biometrika 88, 973–986.

[6] Heagerty, P.J. & Zeger, S.L. (2000). Marginalized mul-
tilevel models and likelihood inference (with comments
and a rejoinder by the authors), Statistical Science 15(1),
1–26.
[7] Kiefer, J. & Wolfowitz, J. (1956). Consistency of
the maximum likelihood estimator in the presence of
infinitely many incidental parameters, Annals of Mathe-
matical Statistics 27, 887–906.
[8] Korff, M., Barlow, W., Cherkin, D. & Deyo, R. (1994).
Effects of practice style in managing back pain, Annals
of Internal Medicine 121, 187–195.
[9] Laird, N. (1978). Nonparametric maximum likelihood
estimation of a mixture distribution, Journal of the
American Statistical Association 73, 805–811.
[10] Lesperance, M. & Kalbfleisch, J. (1992). An algorithm
for computing the nonparametric MLE of a mixing dis-
tribution, Journal of the American Statistical Association
87, 120–126.
[11] Lindsay, B., Clogg, C. & Grego, J. (1991). Semiparamet-
ric estimation in the Rasch model and related exponential
response models, including a simple latent class model
for item analysis, Journal of the American Statistical
Association 86, 96–107.
[12] Lindsey, J.K. & Lambert, P. (1998). On the appropriate-
ness of marginal models for repeated measurements in
clinical trials, Statistics in Medicine 17, 447–469.
Encyclopedia of Biostatistics, Online © 2005 John Wiley & Sons, Ltd.
This article is © 2005 John Wiley & Sons, Ltd.
This article was published in the Encyclopedia of Biostatistics in 2005 by John Wiley & Sons, Ltd.
DOI: 10.1002/0470011815.b2a10021
Generalized Linear Mixed Models 5
[13] McCulloch, C.E. (1997). Maximum likelihood algo-
rithms for generalized linear mixed models, Journal of
the American Statistical Association 92, 162–170.
[14] Neuhaus, J.M., Hauck, W.W. & Kalbfleisch, J.D. (1992).
The effects of mixture distribution misspecification when
fitting mixed-effects logistic models, Biometrika 79,
755–762.
[15] Self, S. & Liang, K.-Y. (1987). Asymptotic properties of
maximum likelihood estimators and likelihood ratio tests
under nonstandard conditions, Journal of the American
Statistical Association 82, 605–610.
[16] Waclawiw, M.A. & Liang, K.-Y. (1993). Prediction of
random effects in the generalized linear model, Journal
of the American Statistical Association 88, 171–178.
[17] Wang, Z. & Louis, T. (2003). Matching conditional and
marginal shapes in binary random intercept models using
a bridge distribution function, Biometrika 90 765–775.
[18] Zeger, S.L., Liang, K.-Y. & Albert, P.S. (1988). Models
for longitudinal data: A generalized estimating equation
approach (Corr: V45 p347), Biometrics 44, 1049–1060.
CHARLES E. MCCULLOCH &
JOHN M. NEUHAUS