MA 6.101 Probability and Statistics: Assistant Professor, IIIT Hyderabad

MA 6.
101
Probability and Statistics
Tejas Bodas
Assistant Professor, IIIT Hyderabad
1 / 49
Logistics
2 / 49
Logistics
� Second half of the course by Prof. Pawan Kumar.
2 / 49
Logistics
� Feel free to contact me anytime at tejas.bodas@iiit.ac.in.
2 / 49
Logistics
� Office @ A5304.
2 / 49
Logistics
� Office @ A5304.
� TA list: Around 9 TAs, you will meet them during tutorials
2 / 49
Logistics
� Office @ A5304.
� Lectures: Monday and Thursday 10:00 to 11:25
2 / 49
Logistics
� Office @ A5304.
� Tutorial on Friday 14:00 to 15:30.
2 / 49
Logistics
� Office @ A5304.
� Tutorial on Friday 14:00 to 15:30.
� Phones in pocket, laptops in bag!
2 / 49
Resources
3 / 49
Resources
� Wont be following any one particular book.
3 / 49
Resources
� Lecture slides will have material from variety of sources.
3 / 49
Resources
� Some popular books

1. Intro. to Probability and Statistics for Engineers and Scientists
by Sheldon Ross (Elsevier)
2. Probability and Statistics by DeGroot and Schervish
(Addison-Wesley)
3. Introduction to probability by Bertsekas and Tsisiklis (Athena
Scientific)
4. A first course in probability by Sheldon Ross (Prentice Hall)
3 / 49
Resources
� Some popular books

1. Intro. to Probability and Statistics for Engineers and Scientists
by Sheldon Ross (Elsevier)
2. Probability and Statistics by DeGroot and Schervish
(Addison-Wesley)
3. Introduction to probability by Bertsekas and Tsisiklis (Athena
Scientific)
4. A first course in probability by Sheldon Ross (Prentice Hall)
� Some urls
1. https://www.statlect.com/
2. https://www.randomservices.org/
3. https://www.probabilitycourse.com/
3 / 49
Evaluation scheme
� Quiz 1 : 15%.
� Midsem exam: 30%.
� Quiz 2: 15%
� Endsem 40 %.
4 / 49
Course Outline
� Module 1 (4 Lectures)
Motivation & Probability basics
5 / 49
Course Outline
All about random variables!
5 / 49
Course Outline
Random processes
5 / 49
Course Outline
Random processes
� Module 4 (10 lectures)

Probability inequalities and Statistics
5 / 49
Where is probability & statistics useful?
6 / 49
� Machine learning
6 / 49
� Reinforcement learning
6 / 49
� Insurance and Finance
6 / 49
� Inventory control & (dynamic) pricing
6 / 49
� Forecasting (weather, demand)
6 / 49
� Analysis of Computer systems (routing, scheduling)
6 / 49
� Analysis of Computer systems (routing, scheduling)
� Biostatistics (clinical trials)
6 / 49
How is it used in all these fields?
7 / 49
� Draw inferences from the underlying data
7 / 49
� Theoretical tool to establish performance guarantees in such

systems.
7 / 49
� Theoretical tool to establish performance guarantees in such

systems.
� Establish proof of concept for robustness of algorithms under

randomness.
7 / 49
Laws
� There are laws of physics
� and there are laws of chemistry
8 / 49
Laws
� There are laws of physics
� and there are laws of chemistry
This course is about laws of uncertainty/randomness.
8 / 49
Philosophy of Probability
9 / 49
Randomness in one way is defined as our inability to have a

deterministic say about an outcome of some event.
9 / 49

For example in a coin toss, if you could mathematically capture all

possible influences on the coin exactly (force of tossing, metal
density, temperature, wind speed etc) you can predict the outcome
of each coin toss experiment with certainty.
9 / 49

For example in a coin toss, if you could mathematically capture all

possible influences on the coin exactly (force of tossing, metal
density, temperature, wind speed etc) you can predict the outcome
of each coin toss experiment with certainty.
Probability theory is all about finding regularity and patterns

is seemingly random experiments (experiments lacking a de-
terministic understanding of it) and expressing them mathe-
matically.
9 / 49
Random experiments and Sample space
10 / 49
� Random experiment : Experiment involving randomness
10 / 49
� Coin toss
10 / 49
� Coin toss
� Roll a dice
10 / 49
� Coin toss
� Roll a dice
� Pick a number at random from [0, 1].
10 / 49
� Coin toss
� Roll a dice
� Sample space Ω: set of all possible outcomes of the random

experiment. It could be a finite or infinite set.
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
� Ωd = {1, 2, . . . , 6}
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
� Ωd = {1, 2, . . . , 6}
� Ωu = [0, 1]
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
� Ωd = {1, 2, . . . , 6}
� Ωu = [0, 1]
� Ω2c
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
� Ωd = {1, 2, . . . , 6}
� Ωu = [0, 1]
� Ω2c = {H, T } × {H, T }
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
� Ωd = {1, 2, . . . , 6}
� Ωu = [0, 1]
� Ω2c = {H, T } × {H, T } = {(H, H), (H, T ), (T , H), (T , T )}
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
� Ωd = {1, 2, . . . , 6}
� Ωu = [0, 1]
� Ω2c = {H, T } × {H, T } = {(H, H), (H, T ), (T , H), (T , T )}
� Sample space Ω should satisfy the following.
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
� Ωd = {1, 2, . . . , 6}
� Ωu = [0, 1]
� Ω2c = {H, T } × {H, T } = {(H, H), (H, T ), (T , H), (T , T )}

� Mutually exclusive outcomes: When the random experiment is
performed, only one of the things in Ω can happen.
10 / 49
� Coin toss
� Roll a dice

� Ωc = {H, T }
� Ωd = {1, 2, . . . , 6}
� Ωu = [0, 1]
� Ω2c = {H, T } × {H, T } = {(H, H), (H, T ), (T , H), (T , T )}

� Mutually exclusive outcomes: When the random experiment is
performed, only one of the things in Ω can happen.
� Mutually exhaustive outcomes: Atleast one of the things in Ω
can happen.
10 / 49
Outcomes and Events
11 / 49
Outcomes and Events
� Element ω ∈ Ω is called a sample point or possible outcome.
11 / 49
Outcomes and Events
� We use the notation ω̄ to denote a realized outcome.
11 / 49
Outcomes and Events
� A subset A ⊆ Ω is called an event.
11 / 49
Outcomes and Events
� Examples of events
11 / 49
Outcomes and Events
� Events in the coin experiment: C1 = {T }.
11 / 49
Outcomes and Events
� Events in the dice experiment: D1 = 6, D2 = {1, 3, 5}
11 / 49
Outcomes and Events
� Events in U[0, 1] experiment: U1 = {0.5}, U2 = [.25, .75].
11 / 49
Outcomes and Events
� In this course, we are interested in probability of events.
11 / 49
Outcomes and Events

� Probability of event A is denoted by P(ω̄ ∈ A) or simply P(A).
11 / 49
Outcomes and Events

� It may not be possible to measure/assign probability for every
subset A (more later).
11 / 49
Outcomes and Events

� It may not be possible to measure/assign probability for every
subset A (more later).
� Any guesses for P(C1 ), P(D1 ), P(D2 ), P(U1 ) and P(U2 ) ?
11 / 49
Probability theory
12 / 49
Probability theory
{Random experiment, Sample space, Events} are the key in-

gredients in probability theory.
12 / 49
Probability theory

In probability theory, we are interested in measuring the

probability of subsets of Ω (events).
12 / 49
Probability theory

In probability theory, we are interested in measuring the

probability of subsets of Ω (events).
Probability measure P is a set function, i.e. it acts on sets

and measures the probability of such sets.
12 / 49
Set theory 101
Visualizing operations on events using Venn diagram!
13 / 49
Set theory 101
13 / 49
Set theory 101

� Compliments: Ac
13 / 49
Set theory 101

� Compliments: Ac
� ∅ denotes empty set.
13 / 49
Set theory 101

� Compliments: Ac
� ∅ denotes empty set.∅ ⊆ A for all A.
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Intersections: A ∩ B
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Difference: A\B
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Symmetric difference:
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Mutually exclusive or disjoint events A and B:
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Identity laws,
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Identity laws,Compliment laws,
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Identity laws,Compliment laws,Associative,
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Identity laws,Compliment laws,Associative, Commutative
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Identity laws,Compliment laws,Associative, Commutative &
Distributive laws,
13 / 49
Set theory 101

� Compliments: Ac
� Union: A ∪ B
� Identity laws,Compliment laws,Associative, Commutative &
Distributive laws, De’Morgans law.
13 / 49
Set theory 101–Cardinality & Countability
14 / 49
14 / 49
� Cardinality of A is denoted by |A|.
14 / 49
� Inclusion-exclusion principle |A ∪ B| = |A| + |B| − |A ∩ B|.
14 / 49
� Inclusion-exclusion principle for n sets ?
14 / 49
� Countable sets: Set A is said to be countable if
14 / 49
� Countable sets: Set A is said to be countable if it is either

finite or has 1-1 correspondence with natural numbers N.
14 / 49
� Countable sets: Set A is said to be countable if it is either

finite or has 1-1 correspondence with natural numbers N.
� Uncountable sets: These are sets which are not countable.
14 / 49
Set theory 101 – Monotone sequence of sets
15 / 49
� Increasing sequence A1 ⊆ A2 ⊆ A3 . . .
15 / 49
� Decreasing sequence A1 ⊇ A2 ⊇ A3 . . .
15 / 49
� Decreasing sequence A1 ⊇ A2 ⊇ A3 . . .
� Examples from U[0, 1]:

� In = [0, 1 − n1 ]
� Dn = [0, n1 ]
15 / 49
Set theory 101 – Cartesian product of sets
16 / 49
� Cartesian product of sets A and B is denoted by A × B.
16 / 49
� A × B is itself a set whose members are sets of the form (a, b)

where a ∈ A and b ∈ B.
16 / 49

� Suppose A = {1, 3, 5} and B = {2, 4, 6} what is A × B?
16 / 49

� What is [0, 1] × [0, 1]?
16 / 49

� What is [0, 1] × [0, 1]? unit square!
16 / 49
Set theory 101 – Powersets
17 / 49
Powerset of A is denoted by P(A) is a set whose members

are all possible subsets of A. (P and P are different!)
17 / 49

17 / 49

� What is P(Ωc ) ?
17 / 49

� What is P(Ωd ) ?
17 / 49

� What is P(Ωu ) ?
17 / 49

� What is the cardinality of P(Ωc ), P(Ωd ), P(Ωu ) ?
17 / 49

� What is the cardinality of P(Ωc ), P(Ωd ), P(Ωu ) ?
� For discrete sets Ω, often the power set is denoted by 2Ω .
17 / 49
functions and set functions
18 / 49
18 / 49
� What are functions?
18 / 49
� What are functions? Functions are rules or maps that map
elements from a domain D to elements in the range R.
18 / 49
� f : D → R.
18 / 49
� f : D → R.
� Example: f : R → R defined by f (x ) = x .
18 / 49
� f : D → R.
� Read more on injection, surjection, bijection!
18 / 49
� f : D → R.
� What are set functions?
18 / 49
� f : D → R.
� What are set functions? these are functions that act on sets
and hence domain D is a collection of sets.
18 / 49
� f : D → R.
� Example: length of closed segments on the real line.
18 / 49
� f : D → R.
� Example: length of closed segments on the real line.
� l : D → R+ where D = {[a, b] : a ≤ b, a, b ∈ R} and where

l([c, d]) = d − c.
18 / 49
Back to P
19 / 49
Back to P
19 / 49
Back to P
� Why this detour to set theory?
19 / 49
Back to P
� Recall that Probability measure P acts on sets and measures

the probability of such sets.
19 / 49
Back to P

� In set theory 101 we looked at operations on sets A and B

that gave new sets like A ∪ B, A\B, A × B, P(A).
19 / 49
Back to P


� So given P(A) and P(B), can we deduce P(A ∪ B) or P(A/B)?
19 / 49
Back to P


� So given P(A) and P(B), can we deduce P(A ∪ B) or P(A/B)?
� We want to understand how the probability measure P acts on

sets such as A ∪ B, A\B, A × B.
19 / 49

MA 6.101 Probability and Statistics: Assistant Professor, IIIT Hyderabad

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MA 6.101 Probability and Statistics: Assistant Professor, IIIT Hyderabad

Uploaded by

Copyright:

Available Formats

MA 6.

Assistant Professor, IIIT Hyderabad

� Second half of the course by Prof. Pawan Kumar.

� Second half of the course by Prof. Pawan Kumar.

� Feel free to contact me anytime at tejas.bodas@iiit.ac.in.

� Second half of the course by Prof. Pawan Kumar.

� Feel free to contact me anytime at tejas.bodas@iiit.ac.in.

� Second half of the course by Prof. Pawan Kumar.

� Feel free to contact me anytime at tejas.bodas@iiit.ac.in.

� TA list: Around 9 TAs, you will meet them during tutorials

� Second half of the course by Prof. Pawan Kumar.

� Feel free to contact me anytime at tejas.bodas@iiit.ac.in.

� TA list: Around 9 TAs, you will meet them during tutorials

� Lectures: Monday and Thursday 10:00 to 11:25

� Second half of the course by Prof. Pawan Kumar.

� Feel free to contact me anytime at tejas.bodas@iiit.ac.in.

� TA list: Around 9 TAs, you will meet them during tutorials

� Lectures: Monday and Thursday 10:00 to 11:25

� Tutorial on Friday 14:00 to 15:30.

� Second half of the course by Prof. Pawan Kumar.

� Feel free to contact me anytime at tejas.bodas@iiit.ac.in.

� TA list: Around 9 TAs, you will meet them during tutorials

� Lectures: Monday and Thursday 10:00 to 11:25

� Tutorial on Friday 14:00 to 15:30.

� Phones in pocket, laptops in bag!

� Lecture slides will have material from variety of sources.

� Lecture slides will have material from variety of sources.

� Some popular books

� Lecture slides will have material from variety of sources.

� Some popular books

� Midsem exam: 30%.

� Module 4 (10 lectures)

� Insurance and Finance

� Insurance and Finance

� Inventory control & (dynamic) pricing

� Insurance and Finance

� Inventory control & (dynamic) pricing

� Forecasting (weather, demand)

� Insurance and Finance

� Inventory control & (dynamic) pricing

� Forecasting (weather, demand)

� Analysis of Computer systems (routing, scheduling)

� Insurance and Finance

� Inventory control & (dynamic) pricing

� Forecasting (weather, demand)

� Analysis of Computer systems (routing, scheduling)

� Biostatistics (clinical trials)

� Draw inferences from the underlying data

� Draw inferences from the underlying data

� Theoretical tool to establish performance guarantees in such

� Draw inferences from the underlying data

� Theoretical tool to establish performance guarantees in such

� Establish proof of concept for robustness of algorithms under

� There are laws of physics

� and there are laws of chemistry

� There are laws of physics

� and there are laws of chemistry

This course is about laws of uncertainty/randomness.

Randomness in one way is deﬁned as our inability to have a

Randomness in one way is deﬁned as our inability to have a

For example in a coin toss, if you could mathematically capture all

Randomness in one way is deﬁned as our inability to have a