1

1.2 Outline of generalized linear models

(a) Model assumptions

Linear model:
( ) | |. ,
2 1 p
x x x x x Y E L = = =
.
Generalized linear model:
( ) ( ) | | , x Y E g g = = =
g is called the
link function.
In addition, the response Y has a distribution in the exponential family,
taking the form
( )
( ) | |
( )
( )

,
, ,
y c
a
b y
e y f
+
)
`

=
.

Intuitively, generalized linear model is the extension of the linear
model. As the distribution is normal and the link function is identity
function, the generalized linear model reduces to the linear model.

Example 1:

( )
2
, ~ N Y
. Then,
( ) ( )
( )
(
(
(
(

)
`

+
|
.
|

\
|

=
(

=
2
2
2
2
2
2
2
2
2 log
2
1
2
exp
2
1
exp
2
1
, ,

y
y
y y f
.
2
( ) ( )
( ) ( ) ( )
(

=
(

=
= = = = = =

2 log
2
1
2 log
2
1
,
,
2 2
, , ,
2
2
2
2
2 2
2 2
y y
y c
b a


Example 2:

( ) P Y ~
(Poisson). Then,
( )
( ) ( ) ! log log
!
, ,
y y
y
e
y
e
y f

= =

.
( ) ( ) ( ) ( ) ( ) ! log , , , 1 , 1 , log y y c e b a = = = = = = =

.

Example 3:

( )
m
p m B
Y
,
~
(Binomial distribution, in frequency), 1 0 Y . Then,
( ) ( )
( ) ( ) ( )

|
|
.
|

\
|
+
(

|
|
.
|

\
|

|
|
.
|

\
|

|
|
.
|

\
|
+ + =

|
|
.
|

\
|
=

my
m
m
p p
p
y
my
m
p my m p my
p p
my
m
y f
my m
my
log
1
1
1
log
1
log
exp
log 1 log log exp
1 , ,


3

( ) ( ) ( )
( )
(

|
|
.
|

\
|
=
+ =
|
|
.
|

\
|

= = = =
|
|
.
|

\
|

=
my
m
y c
e
p
b
m
a
m p
p
log ,
, 1 log
1
1
log ,
1
,
1
,
1
log

Example 4:

( ) v G Y , ~
(Gamma distribution). Then,
( )
( )
( ) | | ( )
( ) ( ) | | ( )

+
(

|
|
.
|

\
|
+
|
|
.
|

\
|
=
)
`

|
|
.
|

\
|
+
|
|
.
|

\
|
=
|
|
.
|

\
|
=

y v r vy v
v
y
y v r
vy
v
v
y
y
e
vy
v r
y f
vy v
log log log
1
1
log
1
exp
log log log exp
1 1
, ,

( ) ( ) ( )
( ) ( ) | | ( ) y r
y
y v r vy v y c
b
v
a
v
log
1
log log
1
) log( ) ( log log ,
, log
1
log ,
1
,
1
,
1

|
|
.
|

\
|

|
|
.
|

\
|
= =
=
|
|
.
|

\
|
= = = =
|
|
.
|

\
|
=

Example 5:

( )
2
, ~ IG Y
(Inverse Gaussian distribution). Then,
4
( )
( )
( )
( )
( )

+
|
|
.
|

\
|
=
(

+
=
(

=
3 2
2 2
2
3 2
2 2
2 2
2 2
2
3 2
2 log
2
1
2
1
1
2
1
exp
2 log
2
1
2
2
exp
2
exp
2
1
, ,
y
y
y
y
y
y y
y
y
y
y f

( ) ( )
( ) ( ) ( )
(

=
(

=
=

= = = =

=
3 3 2
2
2 2
2
2 log
1
2
1
2 log
1
2
1
,
, 2
1
, , ,
2
1

y
y
y
y
y c
b a

(b) Properties of generalized linear model

Important Result:
Let the response Y has a distribution in the exponential family,
taking the form
( )
( ) | |
( )
( )

,
, ,
y c
a
b y
e y f
+
)
`

=
.
Then,
z
( ) ( )
'
b Y E = =

z
( ) ( ) ( ) a b Y Var
' '
=


[derivation:]

Let
5
( ) ( ) | | function likelihood - log , , log , , = y f y l

Then,
( ) ( ) ( )
2
2
2
, , , ,
, 0
, ,
(

=
(

=
(

y l
E
y l
E
y l
E
.

Thus,
( ) ( )
( )

a
b y y l
'
, ,
=

( ) ( )
( )
( ) ( )
( )
( ) ( )

'
' '

0
, ,

b Y E
a
b Y E
a
b Y
E
Y l
E
=
=

=
(

=
(

Also,
( ) ( )
( )

a
b y l
' '
2
2
, ,
=

( ) ( )
( )
( )
( )
( )
( )
( )
( )
( )

2
2
2
2
'
2
' '
2
2


, , , ,

a
Y Var
a
Y E
a
b Y
E
Y l
E
a
b Y l
E
=

=
(

=
(

= =
(

( )
( )
( )
( )
( ) ( ) ( )

a b Y Var
a
Y Var
a
b
i ' '
2
' '
= =


Example 1 (continue):

( )
2
, ~ N Y
. Then,
6
( ) ( )
( ) ( ) ( ) ( ) ( )

,
2 2
, , ,
2 ' ' '
2 2
2 2



= = = = = =
= = = = = =
a b Y Var b Y E
b a

Example 2 (continue):

( ) P Y ~
(Poisson)..
( ) ( ) ( )
( ) ( ) ( ) ( ) ( )

= = = = = =
= = = = = =
e a b Y Var e b Y E
e b a
' ' '
,
, 1 , 1 , log
.

Example 3 (continue):

( )
m
p m B
Y
,
~
(Binomial distribution, in frequency),
1 0 Y
. Then,
( ) ( ) ( )
( ) ( )
( ) ( ) ( )
( ) p p
m
m e
e
e
e
m e
e
e
e
a b Y Var
p
e
e
b Y E
e
p
b
m
a
m p
p
=
(

|
|
.
|

\
|
+

+
=
(
(

|
|
.
|

\
|
+

+
= =
=
+
= = =
+ =
|
|
.
|

\
|

= = = =
|
|
.
|

\
|

=
1
1

1
1
1
1
1
1 1

,
1

1 log
1
1
log ,
1
,
1
,
1
log
2
' '
'

Example 4 (continue):

( ) v G Y , ~
(Gamma distribution). Then,
( ) ( ) ( )
( ) ( ) ( ) ( ) ( )
v v
a b Y Var b Y E
b
v
a
v
2
2
' ' '
1 1
,
1

, log
1
log ,
1
,
1
,
1

= = = =

= =
=
|
|
.
|

\
|
= = = =
|
|
.
|

\
|
=

7

Example 5 (continue):

( )
2
, ~ IG Y
(Inverse Gaussian distribution). Then,
( ) ( )
( ) ( )
( ) ( ) ( )
( )
( )
( )
2 3 2
2
3
2
2
3
' '
'
2 2
2
2
1
2
2
1
2
1

,
2
1

2
1
, , ,
2
1

= =
=

= =
=

= = = =

=
a b Y Var
b Y E
b a


(c) Link function

Canonical Link Function:
Let the response Y has a distribution in the exponential family,
taking the form
( )
( ) | |
( )
( )

,
, ,
y c
a
b y
e y f
+
)
`

=
.
with link function
( ) ( ) | | . x Y E g g = = =
As = , the
link function is called canonical link.

Commonly Used Link Functions:
The canonical links for the following distributions are
1. Normal distribution: ( ) = = g
2. Poisson distribution: ( ) ( ) log = = g
8
3. Binomial distribution:
( )
|
|
.
|

\
|

= =

1
log g

4. Gamma distribution:
( )


1
= = g

5. Inverse Gaussian distribution:
( )
2
2
1

= = g


Other Useful Link Functions:
1. Probit link: ( ) ( )
1
= = g
2. Complementary log-log link:
( ) ( ) | | = = 1 log log g
3. Power family of links:
( )
( )

= =
0 , log
0 ,




g