BERNOULLI SCHEME

Let any stochastic experiment 𝐺 has the 2 results 𝐴 and 𝐴̅.

𝑝 = 𝑃(𝐴) 𝑎𝑛𝑑 𝑞 = 𝑃(𝐴̅), 𝑞 = 1 − 𝑝 , 0 < 𝑝 < 1, 0 < 𝑞 < 1

Let experiment 𝐺 repeat 𝑛 − 𝑡𝑖𝑚𝑒𝑠 under the same conditions and probability of the event
𝐴 doesn’t change from one experiment to another experiment. Such experiments are called
Bernoulli experiment or Bernoulli scheme. We interested in the following question:

What is the probability that in 𝑛 independent experiments event 𝐴 happen 𝑘 − 𝑡𝑖𝑚𝑒𝑠?

It is not difficult to see that

𝑃𝑛 (𝑘) = 𝐶𝑛𝑘 ∙ 𝑝𝑘 ∙ 𝑞 𝑛−𝑘 , 𝑘 = ̅̅̅̅̅

0, 𝑛, 𝑞 = 1 − 𝑝.

A is called success (positive result). In Bernoulli scheme also interested in the following
questions:

1) What is the probability that the number of success less than k?

𝑃𝑛 (0) + ⋯ + 𝑃𝑛 (𝑘 − 1)
2) What is the probability that the number of success more than k?

𝑃𝑛 (𝑘 + 1) + ⋯ + 𝑃𝑛 (𝑛)

3) What is the probability that the number of success at least k?

𝑃𝑛 (𝑘) + ⋯ + 𝑃𝑛 (𝑛)

4) What is the probability that the number of success at most k?

𝑃𝑛 (0) + ⋯ + 𝑃𝑛 (𝑘)

5) What is the probability that the number of success between 𝑘1 𝑎𝑛𝑑 𝑘2 inclusive
themselves?
𝑘2

𝑃𝑛 (𝑘1 ; 𝑘2 ) = 𝑃𝑛 (𝑘1 ) + ⋯ + 𝑃𝑛 (𝑘2 ) = ∑ 𝑃𝑛 (𝑘)

𝑘1

Example1: Let one symmetrical coin be tossed 5 times. Find the probability that tail appear 3
times.

Solution: It is clear that tossing of coin are independent trails and in this case tail is the success.
By the condition probability of the success is

1 1
𝑝= 𝑎𝑛𝑑 𝑞 = 1 − 𝑝 =
2 2
Hence,

1 3 1 2 5
𝑃5 (3) = 𝐶53 ∙( ) ∙( ) =
2 2 16
Example2: Find all probabilities 𝑃5 (0) … 𝑃5 (5) and after then draw the graph and create the
table.

Solution: It is clear that tossing of coin are independent trails and in this case tail is the success.
By the condition probability of the success is

1 1
𝑝= 𝑎𝑛𝑑 𝑞 = 1 − 𝑝 =
2 2
Hence,

1 0 1 5 1
𝑃5 (0) = 𝐶50 ∙ ( ) ∙ ( ) =
2 2 32
1 1 1 4 5
𝑃5 (1) = 𝐶51 ∙ ( ) ∙ ( ) =
2 2 32
1 2 1 3 5
𝑃5 (2) = 𝐶52 ∙ ( ) ∙ ( ) =
2 2 16
1 3 1 2 5
𝑃5 (3) = 𝐶53 ∙ ( ) ∙ ( ) =
2 2 16
1 4 1 1 5
𝑃5 (4) = 𝐶54 ∙ ( ) ∙ ( ) =
2 2 32
1 5 1 0 1
𝑃5 (5) = 𝐶55 ∙ ( ) ∙ ( ) =
2 2 32
Now let’s draw a graph.

And create the table

𝑘 0 1 2 3 4 5
𝑃𝑛 (𝑘) 1 5 5 5 5 1
32 32 16 16 32 32
Example3: Find the probability that tail appear at least 2 times.

Solution: According to 3) we write

5 5 5 1 13
𝑃5 (2) + 𝑃5 (3) + 𝑃5 (4) + 𝑃5 (5) = + + + = .
16 16 32 32 16
 Limit theorems for the Bernoulli scheme

From the formula

𝑃𝑛 (𝑘) = 𝐶𝑛𝑘 ∙ 𝑝𝑘 ∙ 𝑞 𝑛−𝑘 , 𝑘 = ̅̅̅̅̅

0, 𝑛, 𝑝 + 𝑞 = 1.

we see that for the large 𝑛, i.e., for the large values of the independent Bernoulli trails this
formula is not useful. Therefore we’ll introduce the asymptotic formulas for 𝑃𝑛 (𝑘). We’ll
consider 2 cases:

1. 𝑝 is not a small number

2. 𝑝 is small number closed to zero.

In the first case we’ll introduce 2 types of approximation formulas:

Moivre-Laplace local theorem

Moivre-Laplace integral theorem

Moivre-Laplace local theorem

Let the number of Bernoulli trails be large, i.e., 𝑛 → ∞ then

1
𝑃𝑛 (𝑘)~ ∙ 𝜑(𝑥)
√𝑛𝑝𝑞

𝑥2
1 𝑘−𝑛𝑝
where 𝜑(𝑥) = 𝑒 − 2 and 𝑥 = . Function 𝜑(𝑥) is called Gauss function and there are
√2𝜋 √𝑛𝑝𝑞
tables of the function 𝑥.

Moivre-Laplace integral theorem

Let in Bernoulli scheme 𝑛 → ∞. Then

𝑃𝑛 (𝑘1 , 𝑘2 )~𝛷(𝑥2 ) − 𝛷(𝑥1 )

where
𝑥

𝛷(𝑥) = ∫ 𝜑(𝑡)𝑑𝑡
0

is Laplace function. There are tables of function 𝛷(𝑥) in probability books and

𝑘𝑖 − 𝑛𝑝
𝑥𝑖 = , 𝑖 = 1,2, …
√𝑛𝑝𝑞

Note that Moivre-Laplace local and integral theorems give us better results in case 𝑛𝑝𝑞 > 10.
If 𝑝 is small number these theorems don’t give good approximations.

In this case we’ll introduce

Poisson theorem.

Let in Bernoulli scheme 𝑛 → ∞, 𝑝 → 0 such that 𝑛𝑝 → 𝜆 = 𝑐𝑜𝑛𝑠𝑡 > 0.

Then

𝜆𝑘 −𝜆
𝑃𝑛 (𝑘) → 𝑒 ; 𝜆 > 0.
𝑘!
Proof.

𝑛! 1
𝑃𝑛 (𝑘) = 𝐶𝑛𝑘 ∙ 𝑝𝑘 ∙ 𝑞 𝑛−𝑘 = ∙ 𝑘 (𝑝𝑛)𝑘 ∙ (1 − 𝑝)𝑛 (1 − 𝑝)−𝑘 =
𝑘! (𝑛 − 𝑘)! 𝑛
𝑛 −𝑘
1 𝑛 ∙ (𝑛 − 1) … (𝑛 − 𝑘 + 1) 𝑘
𝜆 1 𝜆 1
= ∙ ∙ (𝑝𝑛) ∙ (1 − + 𝑜 ( )) ∙ (1 − + 𝑜 ( ))
𝑘! 𝑛𝑘 𝑛 𝑛 𝑛 𝑛

Here,

𝑛 ∙ (𝑛 − 1) … (𝑛 − 𝑘 + 1)
lim =1
𝑛→∞ 𝑛𝑘
−𝑘
𝜆 1
lim (1 − + 𝑜 ( )) =1
𝑛→∞ 𝑛 𝑛

𝑛 𝜆
𝑛
𝜆 1 𝜆 1 𝜆
lim (1 − + 𝑜 ( )) = lim ((1 − + 𝑜 ( )) ) = 𝑒 −𝜆
𝑛→∞ 𝑛 𝑛 𝑛→∞ 𝑛 𝑛

lim (𝑛𝑝) = 𝜆
𝑛→∞

Taking into account these limits we get

𝜆𝑘 −𝜆
𝑃𝑛 (𝑘) → 𝑒
𝑘!
𝜆𝑘
Note that 𝜋𝑘 = 𝑒 −𝜆 , 𝑘 = 0,1,2, … creates the probability distribution. This distribution is
𝑘!
called Poisson distribution. It is not difficult to see that
∞

∑ 𝜋𝑘 = 1
𝑘=0

In practice the formula

𝜆𝑘 −𝜆
𝑃𝑛 (𝑘) ≈ 𝑒
𝑘!
is used. T

Note that from the Poisson theorem we’ll obtain approximate formula for the probability
𝑘2
𝜆𝑘 −𝜆
𝑃𝑛 (𝑘1 , 𝑘2 ) = ∑ 𝑒 .
𝑘!
𝑘=𝑘1

Law of large numbers for the Bernoulli scheme

Let in Bernoulli scheme 𝜇 = 𝜇(𝑛) be the number of success and p be probability of success in
each trail.

Bernoulli theorem

If the number of trails 𝑛 tends to ∞ then

𝜇
𝑃 {| − 𝑝| ≥ 𝜀} → 0 as n → ∞.
𝑛
This statement is called the law of large numbers. This statement has the following meanings.
𝜇
Here 𝑛 is relative frequency and p is probability of success in each trail. The probability of
difference between relative frequency and probability of success is decreasing as n → ∞.