Hypothesis Testing in Linear

Regression Analysis

Session 5
ECON314
 Introduction to Regression Inference

Last week we saw that for the PRF:

But we can estimate the SRF:

we estimated the a consumption function: C 2.7 0.467I
Now, we focus on answering the question:
Having estimated b1 and b2 for our sample, what can we say
about 1 and 2 , and about the model as a whole?
Thus we need to know how well our SRF estimates the true PRF.

InferenceDrawing conclusions & testing hypotheses about a

population, based on evidence collected in a sample
 How well does our SRF estimates the true PRF?

Goal: To gain insight into specific

parameters that exist for a
population

Problem: Because the population is

unobserved the specific
population parameters are unobserved.

Approach:
Draw a specific random sample from the population and observe sample
statistics for that sample.
Use those observed sample statistics as the best guess of the unobserved
population parameters.
Ask whether it is likely to observe the sample statistics actually observed
if the unobserved population parameter equals a hypothesized value.
 Desirable Properties of Simple Linear Regression Estimators
Two desirable properties of estimators are:
(1) Unbiased when the average value of all possible
estimates equals the true population value or

E ( 1 ) 1
E ( )

If 1 1 then the estimator is biased

(1) Efficient if two estimators are unbiased then one

estimator is more efficient than the other estimator if it
has a lower variance.
(the smaller the variance of , the more reliable is as an
estimator of i.e. the less changes from sample to sample
 Visual Depiction of
Efficiency (Best)

If both
estimators are
unbiased then
an estimator is
more efficient
than another
estimator if it
has a lower
variance.
A more efficient estimator is preferred to a less efficient
estimator because it does a better job estimating the true
population parameter (extreme values are less likely).
 Properties of the OLS estimator

Thus the Gauss-Markov theorem states that the OLS estimators of

the PRF (b1; b2 ) must be BLUE, Meaning the OLS estimator is:
B: best, i.e. an efficient estimator or have minimum variance among
all alternative estimators.
- i.e. var(b1) is less than the variance of any other unbiased estimator of 1, and
similarly for 2.

L: linear, i.e. a linear function of the random variable Y

U: unbiased, that is. its average or expected value, E(b1), is equal to
the true value 1, and E(b2) = 2
E: estimator.
 Understand the Simple Linear Regression Assumptions Required for OLS
to be the Best Linear Unbiased Estimator

Assumptions Required for OLS to be Unbiased

Assumption1: The model is linear in the parameters
Assumption2: The data are collected through independent, random
sampling
Assumption3: There must be sample variation in the independent
variable
Assumption4: The error term has zero mean
Assumption5: The error term is uncorrelated with the
independent variable and all functions of the independent
variable.
Additional Assumption Required for OLS to be BLUE
Assumption 6: The error term has constant variance.
Note that these assumptions are theoretical and typically cant be proven or
disproven.
 Estimate precision and test of
significance
1. Use of standard errors

2. Confidence intervals

3. Test of significance

a. Critical t-value method

b. p-value method

c. F- distribution method
STANDARD ERRORS
 STANDARD ERRORS

Consider again the basic two variable model:

Y = 1 + 2 X + u
Last week we saw that b1 and b2 (the sample estimators of 1 and 2)
are random, because their values change from sample to sample.
To know the precision or reliability of a particular estimate, we need
to know the sampling variability of the estimators (i.e. how a
sample mean deviates from the actual mean of a population).

In statistics, the precision of an estimate is measured by its

variance or standard error
 Points to note about Standard
Errors
Standard errors of the parameters indicate the reliability of the
estimated parameters:

The smaller the standard error of b, the more representative

the sample estimator (b) will be of the true population . i.e. the
less b changes from sample to sample.
Therefore we are fairly certain that our estimate is a reliable indicator of
the true parameter

The standard error of a statistic is also inversely proportional to

the sample size. In general, the larger the sample size the
smaller the standard error.
 Standard Errors
The variance of the estimators (b1 and b2) are computed as:
var( b )
X i
2

2 se(b1) var( b1 )
n ( X X )
1 2
i

2
var( b2 ) se(b2 ) var( b2 )
(X i X )2

The smaller the standard error of b, the more reliable

b is as an estimator of i.e. the less b changes from
sample to sample.
 The variance of the error terms/residual mean square (2)
is computed as:

= standard error of the estimate/regression,

n-k = degrees of freedom
Test hypotheses
 Test hypotheses

There are three different methods used to test hypotheses

1. Confidence Interval

2. Critical Value

3. P-value

All three of these methods yield the same conclusion.

 Classical Null and Alternative
Hypotheses

The researcher first states the hypotheses to be tested

Here, we distinguish between the null and the
alternative hypothesis:
Null hypothesis (H0): the outcome that the
researcher does not expect
Alternative hypothesis (H1): the outcome the
researcher does expect
Example:
H0: = 0 (the values you do not expect)
H1: 0 (the values you do expect)
 Type I and Type II Errors
Two types of errors are possible in hypothesis testing:
Type I: Rejecting a true null hypothesis
Type II: Not rejecting a false null hypothesis

Example: Suppose we have the following null and

alternative hypotheses:

H0: 0
H1: > 0

The true is not positive, But, in any one sample we observe an

estimate of that is positive, leading to the rejection of the null
hypothesis
 HYPOTHESIS TESTING

We write two mutually exclusive hypotheses. The goal is to decide

which of these two hypotheses to support.
The null hypothesis always contains an equality (= or or ).

Examples of pairs of hypotheses

1. For most cases, the first hypothesis that we want to test is whether
X and Y are related at all. This is written as:
H0 : = 0 (there is no linear relationship between X and Y )
H1 : 0 (there is a linear relationship between X and Y )

2. Other examples of pairs of hypotheses that we could test about the

relationship are: H 0: 0
H1: < 0 (a negative relationship, eg. inferior good)
 Hypothesis Testing

Pairs of hypotheses
3. or
H0 : 0
H1 : > 0 (a positive relationship, eg. normal good)

4. We can also write hypotheses about specific values:

H0: 0.5
H1: < 0.5
Hypothesis tests can be conducted using various probability
distributions, depending on the hypothesis being tested.
The most common distributions we use are: t, F and chi-squared
 Hypothesis Testing
Note about interpreting results:

Begin with the assumption that the null hypothesis is true

The alternative hypothesis Is the opposite of the null hypothesis. It is
generally the hypothesis that the researcher is trying to prove
We interpret the results of a hypothesis test with reference to the null
hypothesis. We can either:
reject the null hypothesis; or
do not reject the null hypothesis.
When we cannot reject H0, we never say that we accept the null
hypothesis. An alternative H0 may also be compatible with the data.
In statistics, when we reject the null hypothesis, we say that our finding
is statistically significant
When we do not reject the null hypothesis, we say that our finding is
statistically insignificant
CONFIDENCE INTERVALS
 CONFIDENCE INTERVALS
Point Estimation
An estimated regression coefficient (i.e. b1 or b2) is a point estimate,
taken from a sample, and is used to estimate a population parameter.
However, a point estimate is only as good as the sample it represents.
If other random samples are taken from the population, the point
estimates derived from those samples are likely to vary. Therefore a
point estimate will vary from sample to sample.

Interval Estimation
Because of variation in sample statistics, it is more desirable to specify
the interval into which the coefficient is likely to fall.
Estimating a population parameter with a confidence interval is often
preferable to using a point estimate because it provides additional
information about the variability of the estimate
 CONFIDENCE INTERVALS

Lower Upper
confidence confidence
limit limit

Point Estimate
Width of
confidence interval

An interval estimate provides more information about a population

characteristic than does a point estimate

Common levels of confidence intervals used by analysts are 90%,

95%, 98%, and 99%. (the range of values that will include the true
coefficient value with 90%, 95%, 98%, and 99% probability)
 Confidence Intervals
A 95% confidence interval states the range of values that will
include the true coefficient value with 95% probability:

P[(bk tcse(bk )) k (bk + tcse(bk))] = 0.95

Lower limit of Upper limit of
Confidence level
confidence interval confidence interval

Or bk tc se(bk )
Where:
o bk is the estimated coefficient/point estimate.
o se(bk) is its estimated standard error (also written as ).
o tc is a critical value from the t-distribution, using the area for two-tails and
df = n k.
 Example 1: Constructing a
Confidence Interval
Construct a 95% confidence interval for the effect of the price of
chicken on quantity consumed:
cc = 73.08 1.95pc + 1.15pb + 0.1pcy n = 40
(11.12) (0.538) (0.288) (0.040)

tc = 2.021 (See next slide)

df = n-k (40-4)
Using our sample data for the slope parameter:
bpc tc se(bpc ) = 1.95 (2.021)(0.538) = 3.037
bpc + tc se(bpc ) = 1.95 + (2.021)(0.538) = 0.863
Therefore, the 95% confidence interval for pc is
3.037 pc 0.863 with 95% confidence.
 Example: Constructing a Confidence
Interval
The confidence interval indicates the range of values within which
the true population parameter is likely to lie.
From the above example, the interpretation of the CI is: Given the
confidence level of 95%, in the long run, in 95 out of 100 cases,
intervals like (-3.037 ; -0.863) will contain the true pc

3.037 0.863
 Using a confidence interval to test a
hypothesis
Confidence intervals can thus be used to test hypotheses about the
value of individual coefficients:
Example
Suppose we postulate that:
H0: pc = 0
H1: pc 0 CI 3.037 pc 0.863
We must reject the hypothesis that the true value of the parameter is zero since
zero lies outside of the 95% CI. Reject H0 if k lies
in this region
If the hypothesised value of k
lies within the CIs, then the If the hypothesised value
Reject H0 if k lies hypothesised value is plausible, of k lies outside the CI,
in this region
and we cannot reject it (at the then we must reject the
chosen confidence level). hypothesis that this is the
true value of the
Do not reject H0 if k lies in this region parameter.

3.037 0.863 0
CRITICAL VALUES
 TESTS OF SIGNIFICANCE
t-distribution
Tests of significance can also be conducted using various probability
distributions. We will use the t-distribution as an example.
It involves determining a range of values of t for which one must
reject the null hypothesis. This is known as the rejection region.

A set of regression results is used to calculate a test statistic:

If the test statistic falls into the rejection region, then reject H0
(and vice versa).
 t-distribution

There are three types of possible tests of significance, which are

determined by the nature of the alternative hypothesis being tested:
1. A two-tailed test : This occurs when the alternative hypothesis is
of the form
H1: * i.e. reject H0 if the calculated t
lies to the left of tc, or lies to the
right of tc
 t-distribution
2. A left-tailed test : This occurs when the alternative hypothesis is of
the form
H1: < *

Rejection region in left tail

of distribution only

negative
3. A right-tailed test : This occurs when the alternative hypothesis is
of the form
H1: > *

Rejection region in right

tail of distribution only

positive
 TESTING HYPOTHESES ABOUT INDIVIDUAL
COEFFICIENTS
1. Start by writing H0 and H1
2. Then use the chosen method to decide which of the two
hypotheses to support.
1. The critical value method
a. Decide on the significance level
b. Look up a critical value, tC, (or use the rule-of-thumb).
c. Calculate the test statistic (or use the one given by Stata)
d. Compare the test statistic to the critical value and make a decision:

If tcalculated is more extreme than tcritical (i.e. lies in the rejection

regions of the distribution), reject H0.
If tcalculated is less extreme than tcritical (i.e. lies in the non-rejection /
center region of the distribution), do not reject H0.
 Example a.
Use the regression results to test these hypotheses at 1% sig level:

cc = 73.08 1.95pc n = 40
(11.12) (0.538)
1. Our hypothesis is that there is no relationship between the price
of chicken and the consumption of chicken
H0: pc = 0 H1: pc 0 (two-tailed test )
.()
=
= = -3.62
.

tc (at 1% with df = n k = 40 2 = 38) are 2.704. Therefore:

tcal is more extreme than tcrit (lies in the rejection region on the left
hand side) so reject the null hypothesis.
Price does affect consumption, at 1% significance level.
 Example a.
H0: pc = 0 H1: pc 0 (two-tailed test )
= -3.62 < tc 2.704

99%

0.5% 0.5%

-2.704 2.70
4
t is more extreme than tc (lies in the rejection region on the left hand
side) so reject the null hypothesis.
Price does affect consumption, at 1% significance level.
 Example b. cc = 73.08 + 1.15pb n = 40
(11.12) (0.288)

b) a one unit increase in the price of beef increases the

consumption of chicken by more than one unit.
H0: pb 1
H1: pb > 1 (right-tailed test )
.()
=
= = 0.52
.

Critical value of tc (at 1% for a one-tail test, with df = 38) is 2.423

Calculated t does not fall into the rejection region, therefore do
not reject the null hypothesis.
A one unit increase in the price of beef does not increase the
consumption of chicken by more than one unit.
 Example b.

H0: pd 1
H1: pd > 1 (right-tailed test )
= 0.52 < = 2.423 99% Rejection
region

2.423 tc

Calculated t does not fall into the rejection region, therefore do not
reject the null hypothesis.
A one unit increase in the price of beef does not increase the
consumption of chicken by more than one unit.
 TESTS OF SIGNIFICANCE
t-statistic

If a regression coefficient is different from zero in a two-sided test, the

corresponding variable is said to be "statistically significant

"statistically significant at 10 % level

"statistically significant at 5 % level

"statistically significant at 1 % level

 Example C.

Example: Campus crime and enrollment

An interesting hypothesis is whether crime increases by
one percent if enrollment is increased by one percent

Estimate is different from

one but is this difference
statistically significant?

The hypothesis is
rejected at the 5%
level
P-VALUES
TESTING HYPOTHESES ABOUT INDIVIDUAL
COEFFICIENTS
2. The p-value method
Selection of the level of significance in hypothesis tests is arbitrary:
the choice of level is driven by convention.
1%, 5% and 10% are the levels most commonly used.
An alternative is to find the p-value (probability value):
this is the lowest probability (or level of significance) at which we can reject
H0
The smaller the p-value, the stronger is the evidence against the null
hypothesis.
We reject H0 at all levels of significance above the p-value but fail to
reject it at all levels below p:
Do not
Reject H0
reject H0

0 p Probability values 1
Example: The p-value on the I coefficient is 0.020.
This means that the lowest probability level at which you can reject the null
hypothesis (that there is no relationship between consumers incomes and
the consumption of chicken) is 2%.
Thus we reject H0 at all levels above 2% (such as 5%) but fail to reject it
at all levels below 2% (such as 1%).