Probability Theory

Sample Spaces and Events

Sample Space – Set of all possible outcomes of a random experiment
Event – Subset of a sample space
Mutually Exclusive Events

Counting Techniques
Multiplication Rule: 𝑛1 𝑥 𝑛2 𝑥 … 𝑛𝑘 – total amount of ways to complete operation with k steps

Permutation – Order is important

Combination – Order does not matter

Axioms of Probability

• P(S) = 1 where S is the sample space

• 0 ≤ 𝑃(𝐸) ≤ 1 for any event E
• For two events 𝐸1 dan 𝐸2 with 𝐸1 ∩ 𝐸2 = ∅ → 𝑃(𝐸1 ∩ 𝐸2 ) = 𝑃(𝐸1 ) + 𝑃(𝐸2 )

Probability of Union
𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵)
For Mutually exclusive events, 𝑃(𝐸1 ∪ 𝐸2 ∪ … ∪ 𝐸𝑘 ) = 𝑃(𝐸1 ) + 𝑃(𝐸2 ) + ⋯ + 𝑃(𝐸𝑘 )

Conditional Probability
𝑃(𝐵∩𝐴)
Probability of event B given an event A → 𝑃(𝐵|𝐴) = 𝑃(𝐴)

Independence
If events are independent of each other:

• 𝑃(𝐴|𝐵) = 𝑃(𝐴)
• 𝑃(𝐵|𝐴) = 𝑃(𝐵)
• 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴)𝑃(𝐵)
• 𝑃(𝐸1 ∩ 𝐸2 ∩ … ∩ 𝐸𝑘 ) = 𝑃(𝐸1 ) 𝑥 𝑃(𝐸2 ) 𝑥 … 𝑥 𝑃(𝐸𝑛 )

Total Probability

Bayes Theorem
Random Variables and Probability Distribution
Random Variable – function which associates a real number with each element in the
sample space of a random element
Ex: ℎ𝑒𝑎𝑑𝑠 → +1, 𝑡𝑎𝑖𝑙𝑠 → −1
Discrete – Finite number of possibilities
Continuous – Infinite number of possibilities

Probability Distribution – mathematical function, graph, or table which displays

probabilities of different possible outcomes in an experiment
Discrete – Bar graph

• ∑𝑛𝑖=1 𝑓(𝑥𝑖 ) = 1
• 𝑃(𝑋 = 𝑥𝑖 ) = 𝑓(𝑥𝑖 )
Continuous – Integral
∞
• ∫−∞ 𝑓(𝑥)𝑑𝑥 = 1
𝑏
• 𝑃(𝑎 ≤ 𝑋 ≤ 𝑏) = ∫𝑎 𝑓(𝑥) 𝑑𝑥

Cumulative Distribution – considers probability of cases with values less than x

Discrete

Continuous

Joint Probability Distribution – Probability of two or more random variables happening

together (multivariate distribution)

Discrete
Continuous

Marginal Distribution – probability of various values in subset without reference to the

value of other variables
Probability of event X is achieved by summing/integrating 𝑓(𝑥, 𝑦) over the values of Y

Conditional Probability Distribution – probability distribution of Y when X is known to be a

particular value

And vice versa for conditional distribution of X

Statistical Independence
Mathematical Expectation
Expected Value – similar to sample mean, but takes probability distribution into account

X can be replaced with any function of x such as g(x)

Expected Value of Joint Probability Distribution

Variance and Covariance of Random Variables

Variance – variability of the distribution (how far are we from the mean value)

Covariance – measure of the nature of the association between 2 random variables

Correlation Coefficient (nih kalo pada inget belajar ini pas pengkom :))))

• Indicates strength of relationship between 2 random variables

Mean and Variances of Linear Combination and other Properties

Expectation of Linear Combination

Expectation of difference of two or more functions

Expectation of multiplication of two or more functions

Theorem of Variance Calculation

1.

2.

3.
Discrete Distribution

Discrete Uniform Distribution

Binomial Distribution
Binomial Random Variable – number X of successes in n trials (where probability of
success is constant)
Binomial Distribution – The corresponding probability distribution of BRV

Binomial Sums

where 𝑃(𝑋 < 𝑟)

Mean and Variance of Binomial Distribution

Poisson Distribution

• X as the number of outcomes which occur randomly in an interval or region is called

Poisson Experiments
Properties of Poisson Process
1. Probability of more than one event in a subinterval is negligible
2. Probability of one event in subinterval tends to 𝜆∆𝑡
3. Event in each subinterval is independent of other intervals
Poisson random variable – random variable X that equals number of events in Poisson
process

Where 𝜆 is the average number of outcomes per unit, time, distance, area, or volume

Poisson Distribution Mean

Sum of Two Poisson Variables

Continuous Probability Distribution
Probability Density Function (pdf) f(x)
Describes the probability distribution of a continuous random variable X, with
properties:
1. 𝑓(𝑥) ≥ 0
∞
2. ∫−∞ 𝑓(𝑥)𝑑𝑥 = 1
𝑏
3. 𝑃(𝑎 ≤ 𝑋 ≤ 𝑏) = ∫𝑎 𝑓(𝑥)𝑑𝑥 = 𝑎𝑟𝑒𝑎 𝑢𝑛𝑑𝑒𝑟 𝑓(𝑥) 𝑓𝑟𝑜𝑚 𝑎 𝑡𝑜 𝑏

Continuous Random Variable – random variable with an interval of real number for its
range
Cumulative Distributive Function
Mean and Variance of Continuous Random Variables

Expected Value of Function of a Continuous Random Variable

Normal Distribution

Properties:

• Total Area = 1
• Symmetric about 𝑥 = 𝜇
• Mode at 𝑥 = 𝜇
• Area under curve → 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑡ℎ𝑒 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙
Standard Normal Distribution

Where 𝜇 = 0 𝑎𝑛𝑑 𝜎 2 = 1

Normal Approximation to the Binomial

• Discrete Distribution → Normal Distribution