Probability Theory and Random Processes- 19MAT205

Dr. Satyendra Singh Chauhan

Assistant Professor, Department of Mathematics,
ASE, Amrita Vishwa Vidyapeetham, Chennai- 601103
c satyendrasingh@ch.amrita.edu, sdg25792@gmail.com
Course Delivery Plan (CDP)

Course Syllabus
Unit- 1
Review of probability concepts - conditional probability - Bayes theorem
Random Variable and Distributions: Introduction to random variable – discrete and
continuous random variables and its distribution functions - mathematical
expectations – moment generating function and characteristic function - Binomial,
Poisson, Geometric, Uniform, Exponential, Normal distribution functions (moment
generating function, mean, variance and simple problems) – Chebyshev’s theorem.

Unit- 2
Random processes: General concepts and definitions - stationarity in random
processes - strict sense and wide sense stationary processes - autocorrelation and
properties - special processes – Poisson points, Poisson and Gaussian processes and
properties.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 2 / 31

Cont...

Unit- 3
Systems with stochastic inputs - power spectrum- spectrum estimation,
ergodicity –Markov process and Markov chain, transition probabilities,
Chapman Kolmogrov theorem, limiting distributions classification of states.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 3 / 31

Textbooks and Reference Books

Textbooks
1 Douglas C. Montgomery and George C. Runger, Applied Statistics and
Probability for Engineers, John Wiley and Sons Inc., 2005.
2 A. Papoulis, and Unnikrishna Pillai, Probability, Random Variables and
Stochastic Processes, Fourth Edition, McGraw Hill, 2002.

Reference Books
1 J. Ravichandran, Probability and Random Processes for Engineers, First
Edition, IK International, 2015.
2 Scott L. Miller, and Donald G. Childers, Probability and Random Processes,
Academic Press, 2012.
3 J.S. Milton and J.C. Arnold, Introduction to Probability and Statistics, 4th
Edition, McGraw-Hill Publication, 2007.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 4 / 31

Evaluation and Grading

Internal External Total

Components Weightage
Continuous Assessment + Mid Term Exam End Semester
50%
Regular Quizzes (3) 15 Marks (Online Test 20
100%
Regular Assignment (1) 5 Marks Marks +
50% Viva-Voce 30
Mid-Semester Exam 30 Marks Marks)
(Online 10
Marks +
Viva-Voce
20 Marks)

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 5 / 31

 Lecture- 1

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 6 / 31

Introduction

Scope and Objective

Probability theory deals with many real life problems, which either inherently
involve the chance phenomena or describing the behavior of the system
explicitly with statistical properties.

Applications in Engineering
Interpretation of the research in many engineering aspects depends on concept
of probability and statistics that familiarize with the computational aspects.
The course deals with basic properties of various distributions and other
related things.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 7 / 31

Mathematical Models

A model is a theoretical explanation of the phenomenon under study and, at

the outset, is usually expressed verbally. To use the model for predictive
purposes this verbal description must be translated into one or more
mathematical equations.

These equations can be used to determine the value of a specific variable in

the model based on the knowledge of the values assumed by othermodel
variables.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 8 / 31

Cont...

Deterministic Model
It allows us to determine an exact value for the variable of interest under
specified conditions.

Non-Deterministic Model
Also called probabilistic model or stochastic model.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 9 / 31

Cont...

Population
1 The overall group of objects about which conclusions are to be drawn is
called the population.
2 A population is the set of all measurements of interest to the investigator.

Sample
1 A subset or portion of population that is actually obtained and that is
used to draw conclusions about the population is called a sample.
2 A sample is a subset of measurements selected from the population of
interest.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 10 / 31

Cont..

Example
For body temperature experiment the sample is the set of body temperature
measurements for 1000 healthy people chosen by the experimenter. The
population is the body temperatures of all healthy people in the world.

In other words we try to describe or predict the behavior of the population on

the basis of information obtained from a representative sample from that
population. The objects on which measurements are taken are called elements
of the sample.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 11 / 31

Introduction to Probability Theory

Interpretation of Probabilities
1 Probabilities are numbers between 0 and 1, inclusive, that reflect the
chances of a physical event occuring.
2 Probability near 1 indicates that the event is extremely likely to occur.
They do not nean that the event will occur, only that the event is
considered to be a common occurrence.
3 Probability near 0 indicates that the event is not very likely to occur.
They do not mean that the event will fail to occur, only that the event is
considered to be rare event.
1
4 Probability 2 indicates that the event as likely to occur or not.
5 Since numbers between 0 to 1 can be expressed as percentages between
0 to 100, probabilities are often expressed as percentages.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 12 / 31

Assigning Probabilities
There are three methods widely used to assign the probabilities
Personal Approach
Always applicable, Personal opinion about anything based on judgment or
feeling. Accuracy depends on the amount of accurate information gathered
and accurate assess.

Example
An oil spill has occured. An environmental scientist asks, ”What is the
probability that this spill can be contained before it causes widespread
damage to nearby beaches?” Many factors come into play, among them the
type of spill, the amount of oil spilled, the wind and water conditions during
the clean-up operation, and the nearness of the beaches. These factors make
this spill unique. The scientist is called upon to make a value judgement, that
is, to assign a probability to the event based on informed personal opinian.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 13 / 31

Cont...
Relative Frequency Approach
It is used whenever the experiment can be repeated many times and result
observed.
If the occurrence of event is A, then
f number of times A occured
P[A] = =
n number of times experiment was run

Example of Relative Frequency Approach

An electrical engineer is studying the peak demand at a power plant. It is
observed that on 80 of the 100 days randomly selected for study from past
records, the peak demand occured between 6 and 7p.m. It is natural to assume
that the probability of this occuring on another day is atleast approximatly
80
100 = 0.80. This figure is not simply a personal opinion. It is a figure based
on repeated experimentation and observation. It is a relative frequency.
Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 14 / 31
Cont...

Classical Approach
Used only when it is reasonable to assume that the possible outcomes of the
experiment are equally likely.
If the occurrence of event is A, then
n(A) Number of ways A can occur
P[A] = =
n(S) Number of ways the exeripent can proceed

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 15 / 31

Cont...

Example
What is the probability that a child born to a couple heterozygous for eye color
(each with genes for both brown and blue eyes) will be brown-eyed? We note
that since the child receives one gene from each parent, the possibilities for
the child are (brown, blue), (blue, brown), (blue, blue) and (brown, brown),
where the first member of each pair represents the gene received from the
father. Since each parent is just as likely to contribute a gene for brown eyes
as for blue eyes, all four possibilities are equally likely. Since the gene for
brown eyes is dominant, three of the four possibilities lead to a brown-eyed
child. Henec, the probability that the child will be brown-eyed is 3/4 = 0.75.

Experiment
Any process that yields a result or an observation; A physical action or activity
that is observed and result noted. It may be either deterministic or random.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 16 / 31

 Lecture- 2

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 17 / 31

Deterministic and Random Experiment
Deterministic Experiment
An experiment whose outcome or result can be predicted with certainty is
called a deterministic experiment. For example, if the potential difference E
between the two ends of the conductor and the resistance R are known, the
current I flowing in the conductor is uniquely determined by Ohm’s law,
I = ER .

Random Experiment
All the possible outcomes of an experiment may be known in advance, the
outcome of a particular performance of the experiment cannot be predicted
owing to a number of unknown causes. Such an experiment is called a
random experiment. For rxample, whenever a fair 6- faced cubic dice is
rolled, it is well known that any of the 6 possible outcomes will occur but it
cannot be predicted what exactly the outcome will be, when the dice is rolled
at a point of time.
Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 18 / 31
Sample Space and Events

Sample Space
A sample space for an experiment is a set S with the property that each
physical outcome of the experiment corresponds to exactly one element of S.
An element of S is callled a smaple point.

Event
Any subset A of a sample space is called an event. The empty set φ is called
the impossible event and the subset S is called the certain event.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 19 / 31

Axioms of Probability
Mutually Exclusive
A set of events is said to be mutually exclusive if the occurence of any one of
them excludes the occurence of others. Two events A and B are mutually
exclusive if A occurs and B does not occur and vice-versa, i.e., A and B cannot
occur simultaneously means that P(A ∩ B) = 0.

Axioms
Let S be the sample space and A be an event associated with a random
experiment. Then the probability of the event A, denoted by P(A), is defined
as a real number satisfying the following axioms:
1 0 ≤ P(A) ≤ 1
2 P(S) = 1
3 If A and B are mutually exclusive events, P(A ∪ B) = P(A) + P(B)
4 If A1 , A2 , A3 , ..., An are a set of mutually exclusive events, then
P(A1 ∪ A2 ∪ A3 ... ∪ An ) = P(A1 ) + P(A2 ) + P(A3 ) + ... + P(An ).
Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 20 / 31
General Addition Rule

The probability that at least one of two events will occur when the events are
not necessarily mutually exclusive. If A and B are any two events, then

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 21 / 31

Theorems

Theorem- 1
The probability of the impossible event is zero, i.e., P(φ ) = 0.

Proof: The certain event S and the impossible event φ are mutually exclusive,
i.e., S ∩ φ = φ . Hence,

P(S ∪ φ ) = P(S) + P(φ ), (by axiom 3)

P(S) = P(S) + P(φ ), (since S ∪ φ = S)
P(φ ) = 0.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 22 / 31

Cont...

Theorem- 2
If Ā is the complementary event of A, then P(Ā) = 1 − P(A) ≤ 1.

Proof: A and Ā are mutually exclusive events, such that A ∪ Ā = S.

P(A ∪ Ā) = P(S)

= 1 (by axiom 2)
A Ā
P(A) + P(Ā) = 1 (by axiom 3)
∴ P(Ā) = 1 − P(A)
S
Since P(A) ≥ 0, it follows that P(Ā) ≤ 1.

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 23 / 31

Cont...

Theorem- 3
If A and B are any two events,
P(A ∪ B) = P(A) + P(B) − P(A ∩ B) ≤ P(A) + P(B).

Proof: A is the union of the mutually exclusive events AB̄ and AB. B is the
union of the mutually exclusive events ĀB and AB.

P(A) = P(AB̄) + P(AB) (axiom 3) S

and P(B) = P(ĀB) + P(AB) (axiom 3)
AB̄ AB ĀB
∴ P(A) + P(B) = P(AB̄) + P(AB)
+ P(ĀB) + P(AB)
A B
= P(A ∪ B) + P(A ∩ B)

The result follows. Clearly, P(A) + P(B) − P(A ∩ B) ≤ P(A) + P(B).

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 24 / 31
Cont...

Theorem- 4
If B ⊂ A, then P(B) ≤ P(A).

Proof: B and AB̄ are mutually exclusive events such that B ∪ AB̄ = A.
S
P(B ∪ AB̄) = P(A)
B
P(B) + P(AB̄) = P(A) (axiom 3)
P(B) ≤ P(A). AB̄
A

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 25 / 31

 The End

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 26 / 31

 Thank You

Dr. Satyendra (ASE, Chennai) PRP- 19MAT205 27 / 31