Gamma Distribution

Paul E. Johnson <pauljohn@ku.edu>

June 10, 2013

1 Mathematical Description
Gamma is a probability model for a continuous variable on [0, ). It can look like a ski
slope or it can look single-peaked.
It has 2 parameters, shape and scale. I hate to put in symbols for these because I always
forget which is which. So lets just use the words.
In some books, the scale is specified as 1/rate. Many models set scale=1, so rate=1, and
there is no difference.
If xi is Gamma distributed, the probability density function is:
1 (shape1) ( xi )
f (xi ) = x e scale
scaleshape (shape) i
If you read a different book, you might find it rearranged like so:

1
(shape1)
xi
 xi
f (xi ) = e( scale )
scale (shape) scale
Thats really just the same distribution.

2 Mathematical detail
What is that function ?
The function (s) is the Gamma function, formally defined as (s) = 0 ts1 et dt if
R

s > 0. The Gamma function arises in many statistical applications. The formula appears
to be complicated, but just remember: its just the factorial function extended to take on
values between the integers. Heres why
If you pick s as an integer, (s) is:

(s) = (s 1)! f or s = 1, 2, 3...

So, the value of (1) = 1. And (2) = 1, (3) = 2, (4) = 6, and so forth. And (20) is
some impossibly huge number.
I never really worry very much about the value of Gamma, but in case you do, heres a
graph of it:

1
 Figure 1: The Gamma Function
x v a l s<s e q ( 0 , 5 , by=0 . 0 5 )
gammaxvals < gamma( x v a l s )
p l o t ( x v a l s , gammaxvals , type = " l " , x l a b = " x " , y l a b = e x p r e s s i o n
(Gamma( x ) ) , main = " The Gamma Function " )

The Gamma Function

20
15
(x)

10
5
0

0 1 2 3 4 5

2
 Figure 2: Gamma Density Functions

1.0 Gamma Probability Densities

gamma probability density

0.8

shape=1, scale=1
0.6
0.4
0.2

shape=2, scale=1
0.0

0 2 4 6 8 10

xvals

3 Illustrations
The probability density function of a the Gamma distribution changes quite a bit when one
puts in different values of the parameters. If somebody knows some interesting parameter
settings, then a clear, beautiful illustration of the Gamma can be produced. Consider the
following code, which can be used to create the illustration of 2 possible Gamma density
functions in Figure 2.
x v a l s < s e q ( 0 , 1 0 , l e n g t h . o u t =1000)
gam1 < dgamma( x v a l s , shape = 1 , s c a l e = 1 )
gam2 < dgamma( x v a l s , shape =2, s c a l e= 1 )
matplot ( x v a l s , cbind ( gam1 , gam2 ) , type = " l " , y l a b = "gamma
p r o b a b i l i t y d e n s i t y " , ylim = c ( 0 , 1 ) , main = "Gamma
Probability Densities " )
t e x t ( . 4 , . 7 , " shape =1, s c a l e =1" , pos =4, c o l =1)
t e x t ( 3 , . 2 , " shape =2, s c a l e =1" , pos =4, c o l =2)
In many applications, such as the Negative Binomial regression model for oversdispersed
count data, Gamma density is used to insert observation-level heterogeneity. The scale
parameter is set equal to 1. The following code will produce example density functions for
a variety of shapes with the scale (and hence the rate) set at 1.
par ( m f c o l=c ( 4 , 2 ) )
for ( i in 1:8) {

3
 shape < c ( 0 . 5 , 1 , 2 , 3 , 5 , 1 0 , 1 5 , 3 0 )
s c a l e < 1
xrange<s e q ( from = 0 , t o = 5 0 , by = 0 . 1 )
m a i n l a b e l< p a s t e ( "Gamma( shape=" , shape [ i ] , " , s c a l e=" , s c a l e , "
, ) " , sep=" " )
xprob < dgamma( xrange , shape = shape [ i ] , s c a l e = s c a l e , l o g
= F)
p l o t ( xrange , xprob , type = " l " , main = m a i n l a b e l , x l a b = "
p o s s i b l e values of x " , ylab = " p r o b a b i l i t y of x " )
}
If you assume away the scale parameter by setting it equal to 1, then the probability
density formula for the Gamma( scale=1, shape) simplifies to:
1 (shape1) xi
f (xi ) = xi e xi > 0
(shape)

If shape=1, then this is an exponential distribution (because (1) = 1). But, of course,
the shape parameter can vary.
If the scale can vary, what is the impact on the probability density? We can explore by
making a grid of graphs with 3 different values for the scale parameter. Consider Figure 4,
which results from this code:
par ( m f c o l=c ( 4 , 3 ) )
for ( j in 1:3) {
for ( i in 1:4 ){
shape < c ( 0 . 5 , 1 , 2 , 1 0 )
s c a l e < c ( 0 . 5 , 1 , 3 )
xrange < s e q ( from = 0 , t o = s c a l e [ j ] 2 0 , by = 0 . 1 )
m a i n l a b e l< p a s t e ( " shape=" , shape [ i ] , " , s c a l e=" , s c a l e [ j ] , sep=" "
)
xprob < dgamma( xrange , shape = shape [ i ] , s c a l e = s c a l e [ j ] ,
l o g = FALSE)
p l o t ( xrange , xprob , type=" l " , main = m a i n l a b e l , x l a b = "
p o s s i b l e values of x " , ylab = " p r o b a b i l i t y of x " )
}
}
The horizontal range of all of the graphs in each column is kept fixed, but the ranges
used in the columns does differ. Hopefully, this gives you an idea why the scale parameter
is often set at some arbitrary value, such as 1. On the surface, at least, the scale appears to
be a nuisance, in the sense that its role can easily be understood and replicated later on.
If one sets scale=1, and creates some variable x, then it appears one re-scale it after the
fact.

4
 Figure 3: Gammas Densities with scale=1

Gamma(shape=0.5,scale=1,) Gamma(shape=5,scale=1,)

0.20
1.5
probability of x

probability of x
1.0

0.10
0.5

0.00
0.0

0 10 20 30 40 50 0 10 20 30 40 50

possible values of x possible values of x

Gamma(shape=1,scale=1,) Gamma(shape=10,scale=1,)

0.00 0.04 0.08 0.12

0.8
probability of x

probability of x
0.4
0.0

0 10 20 30 40 50 0 10 20 30 40 50

possible values of x possible values of x

Gamma(shape=2,scale=1,) Gamma(shape=15,scale=1,)
0.3
probability of x

probability of x

0.08
0.2

0.04
0.1

0.00
0.0

0 10 20 30 40 50 0 10 20 30 40 50

possible values of x possible values of x

Gamma(shape=3,scale=1,) Gamma(shape=30,scale=1,)
probability of x

probability of x
0.20

0.04
0.10
0.00

0.00

0 10 20 30 40 50 0 10 20 30 40 50

possible values of x possible values of x

5
 Figure 4: Gammas with Various Parameters

0.0 0.5 1.0 1.5 2.0 shape=0.5,scale=0.5 shape=0.5,scale=1 shape=0.5,scale=3

1.5

0.8
probability of x

probability of x

probability of x
1.0

0.4
0.5
0.0

0.0
0 2 4 6 8 10 0 5 10 15 20 0 10 30 50

possible values of x possible values of x possible values of x

shape=1,scale=0.5 shape=1,scale=1 shape=1,scale=3

0.0 0.5 1.0 1.5 2.0

0.00 0.10 0.20 0.30

0.8
probability of x

probability of x

probability of x
0.4
0.0

0 2 4 6 8 10 0 5 10 15 20 0 10 30 50

possible values of x possible values of x possible values of x

shape=2,scale=0.5 shape=2,scale=1 shape=2,scale=3

0.12
0.6

0.3
probability of x

probability of x

probability of x

0.08
0.4

0.2

0.04
0.2

0.1

0.00
0.0

0.0

0 2 4 6 8 10 0 5 10 15 20 0 10 30 50

possible values of x possible values of x possible values of x

shape=10,scale=0.5 shape=10,scale=1 shape=10,scale=3

0.00 0.04 0.08 0.12

0.04
probability of x

0.20

probability of x

probability of x

0.02
0.10
0.00

0.00

0 2 4 6 8 10 0 5 10 15 20 0 10 30 50

possible values of x possible values of x possible values of x

6
4 Expected Value, Variance, and the role of the pa-
rameters
The Gamma probability distribution has these interesting properties:

E(xi ) = shape scale

V ar(xi ) = shape scale2

q
Std.Dev(x) = shape scale