Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Sampling and sampling distribution

September 12, 2017

STAT 101 Class 5 Slide 1

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Outline of Topics

1 Sampling

2 Sampling distribution of a mean

3 Sampling distribution of a proportion

STAT 101 Class 5 Slide 2

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Statistical Inference

Many economic and social decisions are based on figures from

the entire population, e.g.,
how many homeless people are there?
what is the household income?

A census – every unit in the population is studied – is the

gold standard but very costly

Statisticians use a representative portion of the population – a

sample (a “pseudo-population”) – to solve these problems

The method of using a sample to study a population is called

statistical inference

STAT 101 Class 5 Slide 3

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Population and sample

Population – The set of all units of interest

Finite – Population size N is enumerable
Infinite – N is not finite (note that infinite 6= “continuous” as
in the definition of random variables)

Sample – Any subset of a population. Sample size n can be

as small as one unit of the population

A finite population can be analysed as an infinite population if

(1) N is very big
(2) Nn < 0.05
(3) N is small but sampling is carried out with replacement

We assume an infinite population or a finite population with

(1), (2) or (3)
STAT 101 Class 5 Slide 4
Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Parameters and statistics

Every problem about a population can be characterised by

some summaries called parameters, e.g.,
the proportion of homeless people
the mean income

A statistic is the equivalence of a parameter calculated from

a sample, e.g.,
the proportion of homeless people in the sample
the sample mean income

Parameters are usually unknown whereas statistics are known

Inferential statistics uses a statistic to infer about a parameter

STAT 101 Class 5 Slide 5

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Common population quantities and sample counterparts

Parameter Statistic
Probability distribution Histogram
(Population) mean, µ (Sample) mean, X̄
(Population) variance, σ 2 (Sample) variance, s 2
(Population) standard deviation, σ (Sample) standard deviation, s
(Population) proportion, p (Sample) proportion, p̂

STAT 101 Class 5 Slide 6

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Simple random sample

A simple random sample (SRS) is chosen in such a way

that every member of the population has the same probability
of being selected

A SRS is a pseudo-population that mimics the true population

and hence, we can use the SRS statistic(s) to answer
questions about the unknown population parameter(s)

We assume members in our sample are independently drawn

from the population– each unit in the sample to contribute a
separate piece of information about the parameter of interest

There are other sampling schemes but we focus on SRS here

Hereafter, we refer a SRS of independent observations as a

“sample”
STAT 101 Class 5 Slide 7
Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Sampling from a population

PN
i=1 Xi
µ=
N
X1 , X2 , · · · , XN
Population
PN
i=1 (Xi − µ)2
σ2 =
N

Pn
i=1 Xi
X̄ =
n
X1 , X2 , · · · , Xn
Sample
Pn
i=1 (Xi − X̄ )2
s2 =
n
Intuition tells us X̄ is similar to µ and s2 is similar to σ2
STAT 101 Class 5 Slide 8
Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Sampling error
Example Sampling with replacement from a finite population
Population Sample∗
Units X1 , ..., X7 = 1, 2, 3, 4, 5, 6, 7 X1 , ..., X5 =3, 6, 5, 1, 6
Size N=7 n=5
X1 +...+XN 1+...+7 X1 +...+Xn 3+6+5+1+6
Mean µ= N = 7 =4 X̄ = n = 5 = 4.2

X̄ − µ = 4.2 − 4 ≡ × is called a sampling error

Every sample of size n is subject to sampling error because only a

subset of the population is used to infer about the whole

In practice, µ is unknown and hence × is also unknown and it cannot

be estimated
∗
X1 , ..., X5 are generic symbols for five units randomly selected with replacement from
the population; they are not necessarily the first five units in the population
STAT 101 Class 5 Slide 9
Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Sampling distribution

Every SRS is randomly drawn from the population, hence X̄ and its
sampling error X̄ − µ ≡ × are both random — we cannot make
definitive statements about anything random (c.f., class 1 slide 10)

We look for the probabilities of different values of X̄ , i.e., we want to

find its distribution

The distribution of X̄ is sometimes called a sampling distribution

The distribution of X̄ also tells us the distribution of its sampling error

× = X̄ − µ

since µ is just a constant even though it is unknown

The distribution of × tells us the likely values of ×

STAT 101 Class 5 Slide 10

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Sampling distribution (2)

Sample k

4, 5, 6, 1, 7 4.6 − µ = ×
Population
X̄ = 4.6
1, 2, 3, 4, 5, 6, 7

Sample 2
1, 4, 6, 2, 2
3−µ=×
X̄ = 3
Sample 1
3, 6, 5, 1, 6
4.2 − µ = ×
X̄ = 4.2

STAT 101 Class 5 Slide 11

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Finding sampling distribution – Method 1

Different SRSs give an empirical sampling
Distribution of X and x
distribution of X̄
0 to 500
500 to 1000
Few samples have X̄ near 1 or 7 — only 1000 to 1500
1500 to 2000
appear if SRS gives nearly all 1s or all 7s — a 2000 to 2500
rare event 2500 to 3000
3000 to 3500
3500 to 4000

Highest frequencies of X̄ near population

mean µ = 4 — very often X̄ is similar to µ
and sampling error × is small
µ
The distributions of X̄ and × are identical 0 1 2 3 5 6 7 8

except the values are translated X

This method is not feasible because: −4 −3 −2 −1 0 1 2 3 4

1. Drawing many SRSs is time consuming sampling error x
2. Usually the entire population, such as
1,2,3,4, 5,6,7, and hence µ are unknown

STAT 101 Class 5 Slide 12

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Find sampling distribution – Method 2

Possible sampling errors =

(Possible values of X̄ )

0 Sampling error
µ X̄

By assuming a sufficiently large sample of n observations, the Central Limit

Theorem (CLT, P.-S. Laplace, 1810, 1811) shows that using X̄ to estimate
µ of any population, the sampling distribution of X̄ (and its sampling error) is
approximately normal

X̄ ∼ Normal
|{z} | {z } (µ, var(X̄ )) and × = X̄ − µ ∼ Normal
| {z } (0, var(×))
| {z } | {z } | {z }
statistic sampling sampling sampling error sampling sampling
distribution variation distribution variation

We do not know where exactly is × among the red ×s. However, using p the
empirical rules, we can be 95% certain that × is no more than 0 ± 2 var(×)
STAT 101 Class 5 Slide 13
Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Sampling variation
Sample X̄ Sampling error ×
1 3, 6, 5, 1, 6 4.2 4.2 − µ
2 1, 4, 6, 2, 2 3 3 −µ
.. .. .. ..
. . . .
k 4, 5, 6, 1, 7 4.6 4.6 − µ

X1 +...+X5 X1 +...+X5
Any X1 , X2 , X3 , X4 , X5 5 5 −µ

1 Sampling variation, var(X̄ ), tells us if we wish to estimate µ using X̄ ,

different samples may give different estimates and different sampling errors ×
X1 +...+X5
2 X̄ = 5 so var(X̄ ) is due to var(X1 ),..., var(X5 )

3 X1 , X2 , ..., X5 are randomly drawn from the population, they must have the
same behaviour as any X randomly drawn from the population, i.e.,

var(X1 ) = var(X2 ) = ... = var(X5 ) = var(X )

STAT 101 Class 5 Slide 14

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Sampling variation (2)

 
X1 + ... + Xn
var(X̄ ) = var
n
1
= var(X1 + ... + Xn )
n2
1
= [var(X1 ) + ... + var(Xn )]
n2 | {z }
X1 ,...,Xn are independent
1
= n × var(X )
n2 | {z }
var(X1 )=...=var(Xn )≡var(X )
var(X )
=
n }
| {z
depends on var(X ) and n

var(sampling error ×) = var(X̄ − µ) = var(X̄ )

| {z }
µ is a constant

Sampling variation depends on


(1) var(X ), how different are the values of X in the population
(2) n, the sample size

STAT 101 Class 5 Slide 15

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Why sampling variation matters?

Sampling distribution

Sampling error

0 0

Small sampling variation

Large sampling variation
Our sampling error × is
Our sampling error × is among
among the ×s and so
the ×s and so may be large
unlikely to be large

STAT 101 Class 5 Slide 16

Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

What is a proportion?
Example We wish to estimate the proportion, p, of homeless people in
a population of N individuals. Let X indicate whether someone is
homeless: 
1 homeless
X =
0 not homeless

Suppose the value of X in the population are X1 = 1 (homeless),

X2 = 0 (not homeless), X3 = 0,...,XN = 1, which is a collection
of 1’s and 0’s
#10 s
p=
N
1 + 0 + 0 + ... + 1
=
N
X1 + X2 + X3 + ... + XN
= =µ
N
Hence a proportion is a special case of µ with only 1’s and 0’s
STAT 101 Class 5 Slide 17
Outline Sampling Sampling distribution of a mean Sampling distribution of a proportion

Sampling to estimate a proportion

Example (cont’d) We take a sample X1 , ..., Xn and estimate p ≡ µ using

X1 + ... + Xn
X̄ ≡ p̂ =
n


1 with probability p
X1 , ..., Xn are:
0 with probability 1 − p

var(X )
We use CLT for X̄ , i.e., X̄ ∼ N(µ, )
n }
| {z
var(X̄ )

p2
z}|{
var(X ) = E(X 2 ) − E(X )2 = (1)2 p + (0)2 (1 − p) − µ2
= p − p 2 = p(1 − p)

Hence CLT for p̂ is p̂ ∼ N(p, p(1−p)

n )
STAT 101 Class 5 Slide 18