Lecture 3: Basic Concepts of Probability Theory II

Muhammad Ryan Sanjaya

Universitas Gadjah Mada

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 1/1

 1 Random Variables and Probability Distributions
Motivation
Discrete random variable
Continuous random variable

2 Moments
Expectation
Variance

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 2/1

Motivation

In an experiment, we may want to focus on specific measurable

characteristics or outcomes
I Example
These outcomes can be associated with a number by specifying a rule
of association
We are going to learn the concept of a random variable
The concept allows us to use a numerical function of the outcomes
I Simplifies the sample space

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 3/1

Random variable

Definition
A random variable X is a function defined on a sample space, S, that
associates a real number, X (ω) = x , with each outcome ω in S.

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 4/1

Example

Tossing two (fair) coins

I Define the random variable

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 5/1

Remarks on random variable

A random variable is denoted by a capital letter (X , Y , Z , . . .) while

the real value of the random variable is denoted with a lowercase
letter (x , y , z, . . .).
Note that probability does not play a role in the definition of random
variable
Nevertheless, each value of a random variable is associated with
events
We can then associate each event with probability measures

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 6/1

Two types of random variables

There are two types of random variables:

I Discrete
I Continuous

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 7/1

 1 Random Variables and Probability Distributions
Motivation
Discrete random variable
Continuous random variable

2 Moments
Expectation
Variance

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 8/1

Discrete random variable

Definition
A random variable X is said to be discrete if it can assume only a finite or
countably infinite number of distinct values.

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 9/1

Example

The number of passes in a football match

The number of visitors in Borobudur in any given day
The number of poor households in the economy
The number of vehicles in the road in a given hour
etc.

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 10 / 1

Probability mass function (pmf)

Tossing two coins

I Define the random variable X
Associate each value of the random variable xi with probabilities p(xi )
Write the value of the random variable and the corresponding
probabilities in tabular form

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 11 / 1

Probability mass function (pmf)

Definition
The discrete probability mass function of a discrete random variable X or
the probability function is:

p (xi ) = p (X = xi )

where i = 1, 2, 3, . . .

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 12 / 1

Cumulative distribution function (cdf)

Definition
The cumulative distribution function F of the random variable X or simply
the distribution function is defined by:

F (x ) = P (X ≤ x )
X
= p (y ) , −∞ < x < ∞
∀y ≤x

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 13 / 1

Remarks

The probability function p (x ) is non-negative

Furthermore,
∞
X
p (xi ) = 1
i=1

The probability function is defined for a set of discrete values

However, the distribution function F (x ) is defined for all real values x
of X .

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 14 / 1

Example

Tossing two coins

I Define the random variable X
Find the probability function p(xi ) = P(X = xi )
Find the distribution function F (x ) = P(X ≤ x )
Graph the distribution function

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 15 / 1

Example

The cdf of a discrete random variable X is:

x −1 0 2 5 6
F (x ) 0.1 0.15 0.4 0.8 1

I Find P (X = 2)
I Find P (X > 0)

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 16 / 1

 1 Random Variables and Probability Distributions
Motivation
Discrete random variable
Continuous random variable

2 Moments
Expectation
Variance

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 17 / 1

Continuous random variable

Definition
Let X be a random variable. Suppose that there exists a non-negative
real-valued function: f : R → [0, ∞) such that for any interval [a, b],
Z b
P (X ∈ [a, b]) = f (t) dt.
a

Then X is called a continuous random variable. The function f is called

the probability density function (pdf) of X .

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 18 / 1

Remarks

The pdf f must satisfy the following conditions:

I fR (x ) ≥ 0 for all values of x
∞
I
−∞
f (x ) dx = 1
For any real numbers a, b:
Z b
P (a ≤ X ≤ b) = f (x ) dx
a

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 19 / 1

Remarks

Furthermore:
I P (x = a) = 0
I P (a ≤ X ≤ b) = P (a < X ≤ b) = P (a ≤ X < b) = P (a < X < b)

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 20 / 1

Cumulative distribution function (cdf)

Definition
The cumulative distribution function is given by:
Z x
F (x ) = P (X ≤ x ) = f (t) dt.
−∞

0 ≤ F (x ) ≤ 1
limx →−∞ F (x ) = 0 and limx →∞ F (x ) = 1
F is a non-decreasing function, and right continuous.

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 21 / 1

Remarks

Recall that: Z b
P (a ≤ X ≤ b) = f (x ) dx
a
We can also obtain P (a ≤ X ≤ b) using the cdf F :

P (a ≤ X ≤ b) = F (b) − F (a)

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 22 / 1

Example

Let the function: (

cx 2 0<x <3
f (x ) =
0 otherwise
I Find the value of c so that f (x ) is a density function
I Find the distribution function F (x )
I Compute P (2 < X < 3)

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 23 / 1

 1 Random Variables and Probability Distributions
Motivation
Discrete random variable
Continuous random variable

2 Moments
Expectation
Variance

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 24 / 1

Motivation

We will now discuss an important concept in the probability theory:

expectation.
The expected value of a random variable is the balance point of
distribution of probability
In other words, the expected value is the average.

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 25 / 1

Expected value of a discrete r.v.

Definition
Let X be a discrete random variable with pmf p (x ). Then, the expected
value of X is: X
µ = E [X ] = xp (x )
∀x

provided that the sum exists.

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 26 / 1

Example

Let X be a discrete r.v. with the following probability function:

x −1 0 1 2 3 4 5
p (x ) 17 71 1
14
2
7
1
14
1
7
1
7

I Find µ

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 27 / 1

Expected value of a continuous r.v.

Definition
The expected value of a continuous random variable X with pdf f (x ) is:
Z ∞
µ = E [X ] = xf (x ) dx
−∞

provided that the integral exists.

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 28 / 1

Example

Let X be continuous r.v. with pdf f (x ) = kx 2 where 0 ≤ x ≤ 1.

I Find k
I Find E (X )

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 29 / 1

Extensions

Theorem
Let g (X ) be a function of X , then the expected value of g (X ) is:
(P
g (x ) p (x )
x if X is discrete
E [g (X )] = R∞
−∞ g (x ) f (x ) dx if X is continuous

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 30 / 1

Properties of expected value

Theorem
Let c be a constant and let g (X ) , g1 (X ) , . . . , gn (X ) be functions of a
random variable X such that E [g (X )] and E [gi (X )] for i = 1, 2, . . . , n
exist. Then:
E (c) = c
E [cg (X )] = cE [g (X )]
P P
E[ i gi (X )] = i E [gi (X )]

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 31 / 1

 1 Random Variables and Probability Distributions
Motivation
Discrete random variable
Continuous random variable

2 Moments
Expectation
Variance

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 32 / 1

The variance of a r.v. X

Definition
The variance of a random variable X is:

σ 2 = Var (X ) = E (X − µ)2 .

The variance is a measure of spread of a random variable around the

mean

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 33 / 1

The variance of a r.v. X

Using the properties of expected value, we can show that:

h i
Var (X ) = E X 2 − µ2
R∞
where E X 2 = 2 f (x )dx
 
−∞ x (why?)

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 34 / 1

Example

x2
Try find the variance if f (x ) = 3 and 0 ≤ x ≤ 1

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 35 / 1

Properties of variance

Theorem
Let b be a constant:
Var (aX + b) = a2 Var (X )
Var (X ) = E X 2 − µ2

Muhammad Ryan Sanjaya (Econ UGM) Statistics II: Lecture 3 36 / 1