Professional Documents
Culture Documents
Logit and Probit: Models With Discrete Dependent Variables
Logit and Probit: Models With Discrete Dependent Variables
1.2
1
0.8
0.6
Y, P(Y)
0.4
0.2
0
-100 -50 -0.2 0 50 100
-0.4
X
Note that a linear regression line through the actual data cuts through the data
at the point of greatest concentration on each end.
The residuals from this regression line will only be close to the regression line if
the X variable is also Bernoulli distributed. This means that measures of fit or
hypothesis tests involving the squared errors will be silly. The regression line
will seldom lie near the data.
Linear Probability Model
1.2
1
0.8
0.6
Y, P(Y)
0.4
0.2
0
-100 -50 -0.2 0 50 100
-0.4
X
Relatedly, this feature also means that the residuals from the linear model will
be dichotomous and heteroskedastic, rather than normal, raising questions
about hypothesis tests.
This means that the residuals from the linear probability model will be
heteroskedastic and have a dichotomous character.
Note that the residuals change systematically with the values of X. This implies
what it termed endogeneity. They are also not distributed normally.
We could "fix" this problem by estimating the linear probability model using
weighted least squares.
However, the problem with this model runs deeper. We must be able to
interpret results from this model as expected values of probabilities. However,
the graph below suggests further problems.
Observe that some of the probabilities lie above 1 and below zero. This is not
consistent with the rules of probability. We could truncate the model at 0 and 1
to "fix" this problem.
Suppose we have prospective buyers with income around 10k per year. If we
change their income by 1k, how much does the probability that they will buy a
home change? Suppose we have prospective buyers with income around 30k.
If we change their income by 1k, how much does the probability that they will
own a home change? Suppose we have prospective buyers with income around
80k. If we change their income by 1k, how much does the probability that they
will own a home change?
In practice, there are many situations where the probability of a yes outcome
follows an S shaped distribution, rather than the linear distribution alleged by
the linear probability model.
Non-Linear Probability Models
To begin, assume the appropriate statistical experiment. The statistical
experiment is draws from a Bernoulli distribution. The probability model
from the Bernoulli distribution is given:
f ( y | p ) p yi (1 p )1 yi
The issue then becomes how to specify the probability that y=1. We noted
above that this probability often follows an S shaped distribution. In other
words, the probability that y=1 remains small until some threshold is
crossed, at which point it switches rapidly to remain large after the
threshold. This suggests a cumulative density function.
Two different cumulative density functions are commonly used in this
situation: the cumulative standard normal distribution (probit) and the
cumulative logistic distribution (logit).
t Z2
1
P(Y 1)
2
e 2
dt ( z )
z 1 2 X ki ... k X ki
Logit- The cumulative logistic function for logit is grounded in the concept of
an odds ratio.
P
ez
1 P
P (1 P )e z e z Pe z
P Pe z e z
P (1 e z ) e z
ez
P
1 ez
Probit/Logit
1.2
0.8
P(Y)
Probit
0.6
Logit
0.4
0.2
0
-10 0 10
z
Choosing Between Logit/Probit- In the dichotomous case, there is no basis in
statistical theory for preferring one over the other. In most applications it
makes no difference which one uses.
If we have a small sample the two distributions can differ significantly in their
results, but they are quite similar in large samples.
Various R2 measures have been devised for Logit and Probit. However, none is
a measure of the closeness of observations to an expected value as in regression
analysis. All are ad hoc.
Hypothesis testing
t or z test- We can test the significance of the individual coefficients simply
using the point estimates and standard errors (square roots of the diagonal
elements of the asymptotic covariance matrix of estimates). Form a z or t
test by taking
ˆ k k 0
t N k
s ˆ
k
Confidence Intervals
Interpretation
Interpreting Dichotomous Logit and Probit
The signs are meaningful, but the magnitudes may not be, particularly when
the variables are in different metrics.
Above all, note that you cannot interpret the coefficients directly in terms of
units of change in y for a unit change in x, as in regression analysis.
There are various approaches to imparting substantive meaning into logit and
probit results, including:
Probability Calculations
Graphical methods
First differences
First Partial derivatives.