WOLAITA SODO UNIVERSITY

College of Engineering
Department of Electrical and Computer Engineering
(Electronics and Communication Stream)
Academic Year : 2015 E.C (2023G.C)
Year/Sem : II/I
Sub. Probability and Random Process
Course Code: ECEG-2047

1
 Course Outlines
1. Introduction about Basic Concept of Probability Theory

1. Concept of Probability
2. Terminology associated with probability theory
Random Experiments, Sample space, Events, theorems on

conditional Probability, Permutations, Combinations…

3. Axioms of Probability
4. Bayes’s Theorem

2
.
 1.1 Concept of Probability
It’s defined as the ratio of the number of favorable outcomes to the total number of outcomes
of an event.
 The formula to calculate the probability of an event is as follows.
Probability(Event) = Favorable Outcomes/Total Outcomes = x/n

Let us check a simple application of probability to understand it better. Suppose we have to

predict about the happening of rain or not. The answer to this question is either "Yes" or
"No". There is a likelihood to rain or not rain. Here we can apply probability.

 Probability is used to predict the outcomes for the tossing of coins, rolling of dice, or
drawing a card from a pack of playing cards.

3
.
 Probability Formula
 It is the ratio of favorable outcomes to the total favorable outcomes.

 The probability formula can be expressed as,

the number of favorable outcomes of an event ' A'  n(A)

P(A) 
the total number of events occurring in a samplespace 'S' n(S)

where,

• P(A) is the probability of an event 'A'.

• n(A) is the number of favorable outcomes of an event 'A'.

• n(S) is the total number of events occurring in a sample space.

4
Cont…

EXAMPLE 1.1 If we toss a coin, the result of the experiment is that it will
either come up “tails,” symbolized by T (or 0), or “heads,” symbolized by H
(or 1), i.e., one of the elements of the set {H, T} (or {0, 1}).
n(A)
P(A) 
n(S)

EXAMPLE 1.2 If we toss a die, the result of the experiment is that it will
come up with one of the numbers in the set {1, 2, 3, 4, 5, 6}.

5
 1.2 Terminology of Probability Theory
The following terms in probability help in a better understanding of the concepts
of probability.

Experiment: A trial or an operation conducted to produce an outcome is called

an experiment. Or it is a procedure we perform (quite often hypothetical) that
produces some result.

Sample Space: All the possible outcomes of an experiment together constitute

a sample space. For example, the sample space of tossing a coin is head and tail.

The letter S is used to designate the sample space, which is the universal set of
outcomes of an experiment.
6
Cont…

EXAMPLE 1.1 If we toss a coin, the result of the experiment is that it will
either come up “tails,” symbolized by T (or 0), or “heads,” symbolized by H
(or 1), i.e., one of the elements of the set {H, T} (or {0, 1}).

S= {H, T}

EXAMPLE 1.2 If we toss a die, the result of the experiment is that it will
come up with one of the numbers in the set i.e.

S={1, 2, 3, 4, 5, 6}.

7
Cont…

Favorable Outcome: An event that has produced the desired result or expected event
is called a favorable outcome. For example, when we roll two dice, the
possible/favorable outcomes of getting the sum of numbers on the two dice as 4 are
(1,3), (2,2), and (3,1).

 Trial: A trial denotes doing a random experiment.

Random Experiment: An experiment that has a well-defined set of outcomes is

called a random experiment. For example, when we toss a coin, we know that we
would get ahead or tail, but we are not sure which one will appear.

Event: The total number of outcomes of a random experiment is called an event or a

certain set of outcomes of an experiment.
8
Cont…

Equally Likely Events: Events that have the same chances or probability of
occurring are called equally likely events. The outcome of one event is
independent of the other. For example, when we toss a coin, there are equal
chances of getting a head or a tail.

Exhaustive Events: When the set of all outcomes of an experiment is equal to

the sample space, we call it an exhaustive event.

Mutually Exclusive Events: Events that cannot happen simultaneously are

called mutually exclusive events. For example, the climate can be either hot or
cold. We cannot experience the same weather simultaneously.
9
 Different Probability Formulas
A) Probability formula with addition rule: Whenever an event is the union of two other
events, say A and B, then

P(A or B) = P(A) + P(B) - P(A∩B) and P(A � B) = P(A) + P(B) - P(A∩B)

B)Probability formula with the complementary rule: Whenever an event is the complement
of another event, specifically, if A is an event, then

P(A) + P(A′) = 1.
P(A')=1-P(A).
C)Probability formula with the conditional rule: When event A is already known to have
occurred and the probability of event B is desired, then P(B, given A) = P(A and B), P(A, given
B). It can be vice versa in the case of event B.

P(B∣A) = P(A∩B)/P(A)
10
 D) Probability formula with multiplication rule:

Whenever an event is the intersection of two other events, that is, events A and B need
to occur simultaneously. then,
P(A and B) = P(A)⋅P(B), for independent events

P(A and B) = P(A)⋅P(B∣A), for dependent events

Example 1: Find the probability of getting a number less than 5 when a dice is rolled by
using the probability formula.

Solution

To find Probability of getting a number less than 5, Given: Sample space =

{1,2,3,4,5,6}Getting a number less than 5 = {1,2,3,4}. Therefore, n(S) = 6 n(A) = 4
Using Probability Formula, P(A) = (n(A))/(n(s)), p(A) = 4/6 = 2/3
11
Cont…

Example 2: What is the probability of getting a sum of 9 when two dice are
thrown?

Solution:
There is a total of 36 possibilities when we throw two dice. To get the desired
outcome i.e., 9, we can have the following favorable
outcomes.(4,5),(5,4),(6,3)(3,6). There are 4 favorable outcomes.

Probability of an event

P(E) = (Number of favorable outcomes) ÷ (Total outcomes in a sample space)

Probability of getting number 9 = 4 ÷ 36 = 1/9

12
 Finding the Probability of an Event
In an experiment, the probability of an event is the possibility of that event occurring. The probability of any event is a value
between (and including) "0" and "1".
Events in Probability
 In probability theory, an event is a set of outcomes of an experiment or a subset of the sample space.
• If P(E) represents the probability of an event E, then, we have,
1. P(E) = 0 if and only if E is an impossible event.
2.P(E) = 1 if and only if E is a certain event.
3. 0 ≤ P(E) ≤ 1.
 Suppose, we are given two events, "A" and "B", then the probability of event A, P(A) > P(B) if and only if event "A" is more
likely to occur than the event "B". Sample space(S) is the set of all of the possible outcomes of an experiment and n(S)
represents the number of outcomes in the sample space.
P(E) = n(E)/n(S)
P(E’) = (n(S) - n(E))/n(S) = 1 - (n(E)/n(S))
E’ represents that the event will not occur.
Therefore, now we can also conclude that, P(E) + P(E’) = 1
13
 Examples on Probability

Example 1: (a)What is the probability of getting a sum of 10 when two dice are thrown?
(b)Probability of getting a doublet(Same number) = 6/36 = 1/6. (c) Probability of getting a
number 3 on at least one dice = 1/36 (d) Probability of getting a sum of 7 = 6/36 = 1/6

Solution: (a)
There are 36 possibilities when we throw two dice.

The desired outcome is 10. To get 10, we can have three favorable outcomes.
{(4,6),(6,4),(5,5)}
Probability of an event = number of favorable outcomes/ sample space
Probability of getting number 10 = 3/36 =1/12
14
Cont…

Example 2: In a bag, there are 6 blue balls and 8 yellow balls. One ball is selected
randomly from the bag. Find the probability of getting a blue ball.

Solution:

Let us assume the probability of drawing a blue ball to be P(B)

Number of favorable outcomes to get a blue ball = 6

Total number of balls in the bag = 14

P(B) = Number of favorable outcomes/Total number of outcomes = 6/14 = 3/7

15
 Cont…

 A set is a collection of objects and will be denoted by capital letters S, A, R…

A set A is a collection of objects from U, and these objects are called the
elements or points of the set A and will be denoted by lowercase letters, , a, b, x,
y,….

The universal set U is usually represented as the set of all points within a
rectangle or within a circle.

 We use the notation: to indicate that

“x is an element of A” or “x is not an element of A,” respectively.

16
Cont…

17
Cont…

18
Cont…

19
Cont…

20
 1.2 Permutations of n Distinct Objects
Consider sampling without replacement i.e. simply drawing objects from
an urn containing n distinct objects until the urn is empty. Thus, the number
of possible orderings (arrangements, permutations) of n distinct objects is
equal to the number of ordered n-tuples in sampling without replacement K=n
number of permutations of n objects = n(n-1) = n! (without replacement)

We refer to n! as n factorial.
We will see that n! appears in many of the combinatorial formulas. For large n,
Stirling’s formula is very useful:

21
 1.3 Combinations

Example 1.

22
 Difference Between Permutation and Combination

The major difference between the permutation and combination are given below:

Permutation Combination

Permutation means the selection of objects, The combination means the selection of objects,
where the order of selection matters in which the order of selection does not matter.

In other words, it is the selection of r objects

In other words, it is the arrangement of r
taken out of n objects irrespective of the object
objects taken out of n objects.
arrangement.

The formula for permutation is The formula for combination is

nPr = n! /(n-r)! nCr = n!/[r!(n-r)!]

23
 Cont…
Example 1. 1

24
 Cont…
Example 3: It is required to seat 5 men and 4 women in a row so that the women occupy the even places. How many
such arrangements are possible?
Solution: We are given that there are 5 men and 4 women.
i.e. there are 9 positions.
The even positions are: 2nd, 4th, 6th and the 8th places
These four places can be occupied by 4 women in P(4, 4) ways = 4!
= 4 . 3. 2. 1
= 24 ways
The remaining 5 positions can be occupied by 5 men in P(5, 5) = 5!
= 5.4.3.2.1
= 120 ways
Therefore, by the Fundamental Counting Principle,
Total number of ways of seating arrangements = 24 x 120
= 2880

25
 1.4 Axioms and Properties of Probability
 Axioms Probability
 Probability is a rule that assigns a number to each event A in the sample space, Ω.
 In short , the probability of any event A is given by

P( A)  n(A)
n()
where
n( A) - is the number of elements in the event A
n() - is the number of elements in the sample space 

26
Cont…
1. Probability is a non-negative number, 3. Probability of the union of two mutually
exclusive (disjoint) events is the sum of
i.e., P( A)  0 the probability of the events, i.e.,
2. Probability of the whole set is unity, i.e., If A ∩B  , then P( A ∪B)  P( A)  P(B)
 We can generalize axiom (3) for n
P()  1
pairwise mutually exclusive (disjoint)
events.
From axioms (1) and (2), we obtain
 If A 1 , A 2 , A 3 , …, A n is a sequence of n
pairwise mutually exclusive (disjoint)
events in the sample space Ω such that
0  P( A)  1

Ai ∩A j  , for i  j, then

P ∪
 n  n
Ai    P( Ai )
 i 1  i1
27
28
Cont…

 From the above Venn diagram, we can write the following relations.
i. A  ( A ∩ B ) ∪( A ∩ B ) ii. B  ( A ∩B ) ∪( A ∩B )
 P ( A)  P ( A ∩ B )  P ( A ∩ B )  P ( B )  P ( A ∩B )  P ( A ∩B )
 P ( A ∩ B )  P ( A)  P ( A ∩ B )  P ( A ∩B )  P ( B )  P ( A ∩B )

iii. A ∪ B  A ∪( A ∩ B )
iv. A ∪B  B ∪( A ∩B)
 P ( A ∪B )  P ( A )  P ( A ∩B )
 P( A ∪ B ) P ( A ) P ( A ∩B)

29
Cont…
Example-1:
A box contains 10 identical balls numbered 0, 1, 2,…,9. A single ball is selected from the
box at random. Consider the following events.
A: number of ball selected is odd
B: number of ball selected is multiple of 3
C: number of ball selected is less than 5
Find the following probabilities.
a. P( A) d. P( A ∩B)
b. P(B) e. P( A ∪B ∪C)
c. P(C)

30
Cont…
Solution:
 The sample space and the events are given by:
  {0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9} C  {0 , 1 , 2 , 3 , 4}
A  {1 , 3 , 5 , 7 , 9} A ∩ B  {3 , 9}
B  {3 , 6 , 9} A ∪ B ∪ C  {0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 9}
 The number of elements in the sample space and events are:
n()  10 n (C )  5
n( A)  5 n( A ∩ B )  2
n(B)  3 n( A ∪ B ∪C )  9

31
Cont…

 Thus, probabilities of the given events are given by:

a. P( A)  n( A)  5  1 d. P( A ∩B)  n( A ∩B)  2  1
n() 10 2 n() 10 5
b. P(B)  n(B)  3 e. P( A ∪B ∪C)  n( A ∪B ∪C)  9
n() 10 n() 10
c. P(C)  n(C)  5  1
n() 10 2

Example-2
Given P( A)  0.9, P(B)  0.8 and P( A ∩B)  0.75, find :
a. P( A ∪B) c. P( A ∩B) e. P( A ∪B)
b. P( A ∩B) d. P( A ∪B) f . P(B)
32
 1.5 Bayes's Theorem
 Bayes’ theorem describes the probability of occurrence of an event related to any
condition. It is also considered for the case of conditional probability.
 Bayes theorem is also known as the formula for the probability of “causes”.

if we have to calculate the probability of taking a blue ball from the second bag
out of three different bags of balls, where each bag contains three different colour balls viz. red,
blue, black. In this case, the probability of occurrence of an event is calculated depending on
other conditions is known as conditional probability.

33
 Cont…
Bayes Theorem Formula
If A and B are two events, then the formula for the Bayes theorem is given by:

Where P(A|B) is the probability of condition when event A is occurring while event B has already occurred.
Also, get the Bayes Theorem Calculator here.

Bayes Theorem Derivation

Bayes Theorem can be derived for events and random variables separately using the definition of conditional probability and
density.
From the definition of conditional probability, Bayes theorem can be derived for events as given below:
P(A|B) = P(A ⋂ B)/ P(B), where P(B) ≠ 0
P(B|A) = P(B ⋂ A)/ P(A), where P(A) ≠ 0
Here, the joint probability P(A ⋂ B) of both events A and B being true such that,
P(B ⋂ A) = P(A ⋂ B)
P(A ⋂ B) = P(A | B) P(B) = P(B | A) P(A)
P(A|B) = [P(B|A) P(A)]/ P(B), where P(B) ≠ 0
34
 Cont…
Examples and Solutions
Some illustrations will improve the understanding of the concept.
Example 1:
A bag I contains 4 white and 6 black balls while another Bag II contains 4 white and 3 black balls. One ball is drawn at
random from one of the bags, and it is found to be black. Find the probability that it was drawn from Bag I.
Solution:
Let E1 be the event of choosing bag I,
E2 the event of choosing bag II, and
A be the event of drawing a black ball.
Then,

Also, P(A|E1) = P(drawing a black ball from Bag I) = 6/10 = 3/5

P(A|E2) = P(drawing a black ball from Bag II) = 3/7
By using Bayes’ theorem, the probability of drawing a black ball from bag I out of two bags,

35
 Cont…

Example 2: A man is known to speak the truth 2 out of 3 times. He throws a die and reports that the number obtained is a
four. Find the probability that the number obtained is actually a four.
Solution:
Let A be the event that the man reports that number four is obtained.
Let E1 be the event that four is obtained and E2 be its complementary event.
Then, P(E1) = Probability that four occurs = 1/6.
P(E2) = Probability that four does not occur = 1- P(E1) = 1 – (1/6) = 5/6.
Also, P(A|E1)= Probability that man reports four and it is actually a four = 2/3
P(A|E2) = Probability that man reports four and it is not a four = 1/3.
By using Bayes’ theorem, probability that number obtained is actually a four, P(E1|A)

36
 Cont…
Bayes Theorem Applications
 One of the many applications of Bayes’ theorem is Bayesian inference, a particular approach
to statistical inference.
 Bayesian inference has found application in various activities, including medicine, science,
philosophy, engineering, sports, law, etc. For example, we can use Bayes’ theorem to define
the accuracy of medical test results by considering how likely any given person is to have a
disease and the test’s overall accuracy.
 Bayes’ theorem relies on consolidating prior probability distributions to generate posterior
probabilities. In Bayesian statistical inference, prior probability is the probability of an event
before new data is collected.

37
 Cont…
Practice Problems
Solve the following problems using Bayes Theorem.
1.A bag contains 5 red and 5 black balls. A ball is drawn at random, its colour is noted, and again the ball is returned to the
bag. Also, 2 additional balls of the colour drawn are put in the bag. After that, the ball is drawn at random from the bag. What
is the probability that the second ball drawn from the bag is red?
2.Of the students in the college, 60% of the students reside in the hostel and 40% of the students are day scholars. Previous
year results report that 30% of all students who stay in the hostel scored A Grade and 20% of day scholars scored A grade. At
the end of the year, one student is chosen at random and found that he/she has an A grade. What is the probability that the
student is a hosteler?
3.From the pack of 52 cards, one card is lost. From the remaining cards of a pack, two cards are drawn and both are found to
be diamond cards. What is the probability that the lost card is a diamond?
Frequently Asked Questions on Bayes Theorem
What is meant by Bayes theorem in probability?
In Probability, Bayes theorem is a mathematical formula, which is used to determine the conditional probability of the given
event. Conditional probability is defined as the likelihood that an event will occur, based on the occurrence of a previous
38
40
outcome.
 Cont…

How is Bayes theorem different from conditional probability?

As we know, Bayes theorem defines the probability of an event based on the prior knowledge of the conditions related to the
event. In case, if we know the conditional probability, we can easily find the reverse probabilities using the Bayes theorem.
When can we use Bayes theorem?
Bayes theorem is used to find the reverse probabilities if we know the conditional probability of an event.
What is the formula for Bayes theorem?
The formula for Bayes theorem is:
P(A|B)= [P(B|A). P(A)]/P(B)
Where P(A) and P(B) are the probabilities of events A and B.
P(A|B) is the probability of event A given B
P(B|A) is the probability of event B given A.
Where can we use Bayes theorem?
Bayes rule can be used in the condition while answering the probabilistic queries conditioned on the piece of evidence.

39
Cont…

40
QUIZE 1

1. A fair die is tossed twice. Find the probability of getting a 4, 5, or 6 on the first toss
and a 1, 2, 3, or 4 on the second toss.

41