Special Probability

Distributions
Reza Abdolmaleki

Probability + Probability Honors

Lecture 13
In this chapter we shall study some of the probability distributions that figure most prominently in
statistical theory and applications. We shall also study their parameters, that is, the quantities that
are constants for particular distributions but that can take on different values for different members of
families of distributions of the same kind. The most common parameters are the lower moments,
mainly μ and , and there are essentially two ways in which they can be obtained: We can evaluate the
necessary sums directly or we can work with moment generating functions. Although it would seem
logical to use in each case whichever method is simplest, we shall sometimes use both. In some
instances this will be done because the results are needed later; in others it will merely serve to
provide the reader with experience in the application of the respective mathematical techniques.
The Discrete Uniform Distribution
If a random variable can take on k different values with equal probability, we say that it has a discrete
uniform distribution; symbolically, we have the following definition:

Definition. A random variable has a discrete uniform distribution and it is referred to as a discrete
uniform random variable if and only if its probability distribution is given by

where when

In the special case where , the discrete uniform distribution becomes

Example 1.
1.Rolling a Fair Six-Sided Die:
Each face of a fair six-sided die has an equal probability of . The discrete uniform distribution
applies to the possible outcomes (1, 2, 3, 4, 5, 6) with for each .
2.Selecting a Card from a Standard Deck:
In a standard deck of playing cards, each card has an equal probability of ​ of being selected. The
discrete uniform distribution applies to the 52 possible outcomes.
3.Choosing a Random Digit from 0 to 9:
If you randomly select a digit from to , each digit has an equal probability of ​. The discrete
uniform distribution applies to the set
4.Flipping a Fair Coin:
When flipping a fair coin, the outcomes (Heads or Tails) have an equal probability of ​. The
discrete uniform distribution applies to this binary set {Heads, Tails}.
5.Choosing a Random Day of the Week:
If you randomly select a day of the week, each day (Monday, Tuesday, ..., Sunday) has an equal
probability of ​. The discrete uniform distribution applies to the set of days in a week
The Bernoulli Distribution
If an experiment has two possible outcomes, “success” and “failure,” and their probabilities are,
respectively, and , then the number of successes, or , has a Bernoulli distribution; symbolically, we
have the following definition:
Definition. A random variable has a Bernoulli distribution and it is referred to as a Bernoulli random
variable if and only if its probability distribution is given by

Thus, and are combined into a single formula. Observe that we used the notation f(x; θ) to indicate
explicitly that the Bernoulli distribution has the one parameter θ.

Example 2. In connection with the Bernoulli distribution, a success may be getting heads with a
balanced coin, it may be catching pneumonia, it may be passing (or failing) an examination, and it
may be losing a race. This inconsistency is a carryover from the days when probability theory was
applied only to games of chance (and one player’s failure was the other’s success). Also for this
reason, we refer to an experiment to which the Bernoulli distribution applies as a Bernoulli trial, or
simply a trial, and to sequences of such experiments as repeated trials.
 The Binomial Distribution
Repeated trials play a very important role in probability and statistics, especially when the number of
trials is fixed, the parameter θ (the probability of a success) is the same for each trial, and the trials are
all independent. As we shall see, several random variables arise in connection with repeated trials. The
one we shall study here concerns the total number of successes; others will be given in the next lecture.

The theory that we shall discuss in this section has many applications; for instance, it applies if we want
to know the probability of getting heads in flips of a coin, the probability that of persons will recover
from a tropical disease, or the probability that of persons will respond to a mail-order solicitation.
However, this is the case only if each of the persons has the same chance of recovering from the
disease and their recoveries are independent (say, they are treated by different doctors in different
hospitals), and if the probability of getting a reply to the mail-order solicitation is the same for each of
the persons and there is independence (say, no two of them belong to the same household).
 To derive a formula for the probability of getting “x successes in n trials” under the stated
conditions, observe that the probability of getting x successes and failures in a specific order is .
There is one factor for each success, one factor for each failure, and the x factors and factors are all
multiplied together by virtue of the assumption of independence. Since this probability applies to any
sequence of trials in which there are x successes and n – x failures, we have only to count how many
sequences of this kind there are and then multiply by that number. Clearly, the number of ways in
which we can select the trials on which there is to be a success is , and it follows that the desired
probability for “successes in trials” is

Symbolically, we have the following definition:

Definition. A random variable X has a binomial distribution and it is referred to as a binomial
random variable if and only if its probability distribution is given by

Thus, the number of successes in trials is a random variable having a binomial distribution with the
parameters and The name “binomial distribution” derives from the fact that the values of for are the
successive terms of the binomial expansion of ;
this shows also that the sum of the probabilities equals 1, as it should.
Values of the binomial coefficients for n = 0, 1, ... , 20 and r = 0, 1, ... , 10 are given in the following
table
Example 3. Find the probability of getting five heads and seven tails in flips of a balanced coin.

Solution.
Substituting x = 5, n = 12, and into the formula for the binomial distribution, we get

and, looking up the value of in the table , we find that the result is 792 .

Example 4. Find the probability that of persons will recover from a tropical disease if we can
assume independence and the probability is that any one of them will recover from the disease.
Solution.
Substituting , , and into the formula for the binomial distribution, we get

and, looking up the value of in the table we find that the result is , or approximately
 If we tried to calculate the third probability asked for on the previous page, the one concerning the
responses to the mail-order solicitation, by substituting , and, say, , into the formula for the binomial
distribution, we would find that this requires a prohibitive amount of work. In actual practice,
binomial probabilities are rarely calculated directly, for they are tabulated extensively for various
values of θ and , and there exists an abundance of computer software yielding binomial probabilities
as well as the corresponding cumulative probabilities

upon simple commands.

The following table gives the values of to four decimal places for and
.
 To use this table when is greater than 0.50, we refer to the following identity:

Theorem 1.

The proof is obvious from the definition of binomial distribution.

For instance, to find we look up b(7; 18, 0.30) and get . Also, there are several ways in which binomial
probabilities can be approximated when is large; one of these will be mentioned in upcoming lectures. Let
us now find formulas for the mean and the variance of the binomial distribution.

Theorem 2. The mean and the variance of the binomial distribution are
Proof.

where we omitted the term corresponding to , which is , and canceled the x against the first factor of in the
denominator ofThen, factoring out the factor in and one factor , we get

and, letting and , this becomes

since the last summation is the sum of all the values of a binomial distribution with the parameters and ,
and hence equal to 1.
To find expressions for and , let us make use of the fact that and first evaluate Duplicating for all practical
purposes the steps used before, we thus get
 Letting and , this becomes

Therefore,

and, finally,

It should not have come as a surprise that the mean of the binomial distribution is given by the product . After all, if a
balanced coin is flipped times, we expect (in the sense of a mathematical expectation) heads and tails; similarly, if a
balanced die is rolled 240 times, we expect sixes, and if the probability is that a person shopping at a department store
will make a purchase, we would expect persons
shopping at the department store to make a purchase.
 The formula for the variance of the binomial distribution, being a measure of variation, has many
important applications; but, to emphasize its significance, let us consider the random variable , where
is a random variable having a binomial distribution with the parameters and . This random variable is
the proportion of successes in trials, and the following result is obvious :

Theorem 3. If has a binomial distribution with the parameters and and , then

and
Now, if we apply Chebyshev’s theorem with , we can assert that for any positive constant c the
probability is at least

that the proportion of successes in trials falls between and + c. Hence, when , the probability
approaches that the proportion of successes will differ from by less than any arbitrary constant .
This result is called a law of large numbers, and it should be observed that it applies to the
proportion of successes, not to their actual number. It is a fallacy to suppose that when
is large the number of successes must necessarily be close to

An easy illustration of this law of large numbers can be obtained through a computer simulation
of the repeated flipping of a balanced coin. This is shown in the following figure, where the ’s and
’s denote heads and tails:
 Reading across successive rows, we find that among the first five simulated flips there are heads,
among the first ten there are heads, among the first fifteen there are heads, among the first twenty
there are heads, among the first twenty-five there are heads, ..., and among all hundred there are 51
heads. The corresponding proportions, plotted in the following figure, are . Observe that the
proportion of heads fluctuates but comes closer and closer to 0.50, the probability of heads for each
flip of the coin.
Since the is easy to obtain, let us find moment-generating function of the binomial distribution
and use it to verify the results of Theorem 2 .

Theorem 4. The moment-generating function of the binomial distribution is given by

Proof is easy an is left to the reader as an exercise.

If we differentiate twice with respect to , we get

and, upon substituting , we get and Thus, and which agrees with the formulas given in Theorem 2.