Professional Documents
Culture Documents
Ch13slides Generalized Linear Models
Ch13slides Generalized Linear Models
• What is a GLM?
• Logistic regression
• Poisson regression
Generalized Linear Models (GLMs)
What is a GLM?
In ordinary linear regression, we assume that the response is a linear
function of the regressors plus Gaussian noise:
y = β0 + β1 x1 + · · · + βk xk +
|{z} ∼ N (x0 β, σ 2 )
| {z }
linear form x0 β N (0,σ 2 ) noise
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 3/24
Generalized Linear Models (GLMs)
beta=(1,2)
5
4
3
β0 + β1 x b
y
2
y
1
0
−1
0.0 0.2 0.4 0.6 0.8 1.0
x x
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 4/24
Generalized Linear Models (GLMs)
That is,
g(µ) = β 0 x
or equivalently,
µ = g −1 (β 0 x)
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 5/24
Generalized Linear Models (GLMs)
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 6/24
Generalized Linear Models (GLMs)
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 7/24
Generalized Linear Models (GLMs)
Applications:
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 8/24
Generalized Linear Models (GLMs)
Logistic regression
Logistic regression is a GLM that combines the Bernoulli distribution (for
the response) and the logit link function (relating the mean response to
predictors):
µ
log = β0 x (y ∼ Bernoulli(p))
1−µ
p
log = β0 x (y ∼ Bernoulli(p))
1−p
p p
where p: probability of success, 1−p : odds, log( 1−p ): log-odds.
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 9/24
Generalized Linear Models (GLMs)
• s(0) = 0.5
0.6
as z goes from −∞ to +∞
0.0
−4 −2 0 2 4
z
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 10/24
Generalized Linear Models (GLMs)
1.0
each given x (sampled location),
0.8
µ = p = s(z), z = β0 x
0.6
mu
0.4
has the following interpretations:
0.2
• mean response
0.0
−4 −2 0 2 4
z
E(y | x, β) = s(z)
Population model:
• probability of success:
y | x, β ∼ Bernoulli(p = s(β 0 x))
P (y = 1 | x, β) = s(z)
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 11/24
Generalized Linear Models (GLMs)
beta=(−3,2)
1.0
0.8
0.6
y
0.4
0.2
0.0
0 1 2 3 4
x
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 12/24
Generalized Linear Models (GLMs)
1.0
fitting a logistic regression model is
0.8
equivalent to choosing the value of
0.6
β such that the mean response
y
0.4
0.2
µ = s(β 0 x)
0.0 0 1 2 3 4
x
and
1
pi =
1 + e−β0 xi
However, there is no closed-form solution, and the optimal β has to be
computed numerically.
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 14/24
Generalized Linear Models (GLMs)
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 15/24
Generalized Linear Models (GLMs)
The prediction at x0 is
1, if p̂0 > 0.5
ŷ0 =
0, if p̂0 < 0.5
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 16/24
Generalized Linear Models (GLMs)
R scripts
x = c(162, 165, 166, 170, 171, 168, 171, 175, 176, 182, 185)
y = c(0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1)
model ← glm(y∼x,family=binomial(link=’logit’))
p = model$fitted.values
# p = [0.0168, 0.0708, 0.1114, 0.4795, 0.6026, 0.2537, 0.6026, 0.9176,
0.9483, 0.9973, 0.9994]
fitted.prob ← predict(model,data.frame(x=c(168,170,173)),type=’response’)
# fitted.prob = [0.2537, 0.4795 0.8043 ]
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 17/24
Generalized Linear Models (GLMs)
p=1/(1+exp(−84.8331+0.4985 x))
1.0
0.8
0.6
p
0.4
0.2
0.0
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 18/24
Generalized Linear Models (GLMs)
y | x, β ∼ Bernoulli(p)
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 20/24
Generalized Linear Models (GLMs)
Poisson regression
Poisson regression is a GLM that combines the Poisson distribution (for the
response) and the log link function (relating mean response to predictors):
That is,
0
y | x, β ∼ Poisson(λ = eβ x )
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 21/24
Generalized Linear Models (GLMs)
beta=(1,−3)
80
sample
true model
fitted model
60
40
y
20
0
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 22/24
Generalized Linear Models (GLMs)
R code
poisson.model ← glm(y∼x,family=poisson(link=’log’))
poisson.model$coefficients
(Intercept) x
1.003291 -3.019297
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 23/24
Generalized Linear Models (GLMs)
Dr. Guangliang Chen | Mathematics & Statistics, San José State University 24/24