Regression Discontinuity Designs

Kosuke Imai

Harvard University

S TAT 186/G OV 2002 C AUSAL I NFERENCE

Fall 2019

Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 1 / 16

Observational Studies
In many cases, we cannot randomize the treatment assignment
ethical constraints
logistical constraints
But, some important questions demand empirical evidence even
though we cannot conduct randomized experiments!
Designing observational studies find a setting where credible
causal inference is possible
Key = Knowledge of treatment assignment mechanism
Regression discontinuiety design (RD Design):
1 Sharp RD Design: treatment assignment is based on a
deterministic rule
2 Fuzzy RD Design: encouragement to receive treatment is based on
a deterministic rule
Originates from a study of the effect of scholarships on students’
career plans (Thistlethwaite and Campbell. 1960. J. of Educ. Psychol)
Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 2 / 16
Regression Discontinuity Design
Idea: Find an arbitrary cutpoint c which determines the treatment
assignment such that Ti = 1{Xi ≥ c}
828 QUARTERLY JOURNAL OF ECONOMICS
Close elections as RD design (Lee et al. 2004. Q. J. Econ):

FIGURE I
Total Effect of Initial Win on Future ADA Scores: ␥
This figure plots ADA scores after the election at time t ⫹ 1 against the
Democrat vote share, time
Kosuke Imai (Harvard) t. EachDiscontinuity
Regression circle is theDesigns
average ADA score within 0.01
Stat186/Gov2002 Fall 2019 3 / 16
Identification
Estimand:
E(Yi (1) − Yi (0) | Xi = c)
Assumption: E(Yi (t) | Xi = x) is continuous in x for t = 0, 1
deterministic rather than stochastic treatment assignment
violation of the overlap assumption: 0 < Pr(Ti | Xi = x) < 1 for all x
RD design is all about extrapolation

Regression modeling:

E(Yi (1) | Xi = c) = lim E(Yi (1) | Xi = x) = lim E(Yi | Xi = x)

x↓c x↓c
E(Yi (0) | Xi = c) = lim E(Yi (0) | Xi = x) = lim E(Yi | Xi = x)
x↑c x↑c

Advantage: internal validity

Disadvantage: external validity
Make sure nothing else is going on at Xi = c
Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 4 / 16
Analysis Methods under the RD Design
Simple linear regression within a window
How should we choose a window in a principled manner?
How should we relax the functional form assumption?
higher-order polynomial regression not robust to outliers

Local linear regression (same h for both sides): better behavior at

the boundary than other nonparametric regressions
n
Xi − c
X  
2
(α̂+ , β̂+ ) = argmin 1{Xi > c}{Yi − α − (Xi − c)β} · K
α,β h
i=1
n
Xi − c
X  
2
(α̂− , β̂− ) = argmin 1{Xi < c}{Yi − α − (Xi − c)β} · K
α,β h
i=1

Weighted regression with a kernel function of one’s choice:

uniform kernel: K (u) = 21 1{|u| < 1}
triangular kernel: K (u) = (1 − |u|)1{|u| < 1}

Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 5 / 16

Optimal Bandwidth (Imbens and Kalyanaraman. 2012. Rev. Econ. Stud.)
Choose the bandwidth by minimizing the MSE:
MSE = E[{(α̂+ − α̂− ) − (α+ − α− )}2 | X]
= E{(α̂+ − α+ )2 | X} + E{(α̂− − α− )2 | X}
−2 · E(α̂+ − α+ | X) · E(α̂− − α− | X)
= (Bias+ − Bias− )2 + Variance+ + Variance−

Bias and variance of local linear regression estimator at the

boundary:
Bias = E(m̂(0) | X) − m(0), Variance = V(m̂(0) | X)
where m(x) = E(Yi | Xi = x), m̂(x) = α̂(x), and
n
Xi − x
X  
2
(α̂(x), β̂(x)) = argmin (Yi − α − β(Xi − x)) · K
α,β h
i=1

Refinements, e.g., bias correction (Calonico et al. 2014. Econometrica)

Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 6 / 16
The “As-if Random” Assumption
RD design does NOT require the local randomization or “as-if
random” assumption within a window:
{Yi (1), Yi (0)}⊥
⊥Ti | c0 ≤ Xi ≤ c1
PL19CH20-Imai ARI 16 April 2016 8:58
The “as-if random” assumption implies zero slope of regression
lines difference-in-means within the window
The assumption may be violated regardless of the window size
a Democratic experience advantage b Share of total spending by Democratic candidate
0.8 0.8
Proportion candidates with advantage

0.7 0.7

Proportion of total spending

0.6 0.6

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2
r personal use only.
.annualreviews.org

0.1 0.1
–0.05 0.00 0.05 –0.05 0.00 0.05
Democratic margin Democratic margin

Figure 1
Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 7 / 16
Close Elections Controversy (de la Cuesta and Imai. 2016. Annu. Rev.
Annu. Rev. Polit. Sci. 2016.19:375-396. DownloadedAnnu.
fromRev. Polit. Sci. 2016.19:375-396. Downloaded from www.annualreviews.org
www.annualreviews.org
Political Sci) Access provided by Princeton University Library onAccess provided
05/17/16. by Princeton
For personal University Library on 05/17/16. For personal use only.
use only.

a Difference-in-means in window a Difference-in-means

b Linear regressionininwindow
window b Linear
c Localregression in window
linear regression

Sorting? CQ rating
CQ rating
Dem. donation %
CQ rating
CQ rating
Dem. spending %
Dem. win t − 1
Dem. spend
Dem. donat
Dem. donation % Dem. win t − 1 Dem. donation %
Dem. spending % Dem. experience adv. Dem. # prev.
1 Pre-election
Dem. spending % Dem. experience adv. Dem. # prev. terms

388
% Urban Dem. inc. in race CQ
% Urban Dem. inc. in race CQ rating
Dem. # prev. terms Dem. donation % %U
behavior or
Dem. # prev. terms
Dem. share t − 1
de la Cuesta Dem. donation %
Dem. share t − 1
Dem. spending %
Dem. spending %
% Urban
Dem. share t − 1
Dem. share
Dem. margin t − 1 Dem. share t − 1 Dem. experienc
Dem. margin t − 1 Dem. share t − 1 Dem. experience adv.
characteristics
Dem. experience adv.
Dem. experience adv.
Dem. margin t − 1
Dem. margin t − 1
Dem. win t − 1
Dem. win
Dem. win t − 1 % Govt. worker Dem. gov
·

Dem. win t − 1 % Govt. worker Dem. governor

of candidates,
Imai

Dem. inc. in race Dem. # prev. terms Ope

Dem. inc. in race Dem. # prev. terms Open seat
Dem. sec. of state Dem. sec. of state %
Dem. sec. of state Dem. sec. of state % Black
e.g., resource
Open seat
Open seat
% Foreign born
% Urban
% Urban
Dem. margin t − 1
% Foreign born
Dem. margin
Dem.-held ope
% Foreign born % Foreign born Dem.-held open seat
advantages
Dem. governor
Dem. governor
Dem.-held open seat
% Black
% Black
% Foreign born
Dem.-held open seat
% Foreign
Dem. inc. in
Dem.-held open seat Dem.-held open seat Dem. inc. in race
steep slope
Rep.-held open seat
% Black
Rep.-held open seat
Open seat
% Black
% Voter turnout
Open seat
Dem. sec. of state
% Voter turnout
Rep.-held open seat
Dem. sec. of
Rep.-held ope
% Voter turnout Rep.-held open seat % Voter tu
% Voter turnout Rep.-held open seat % Voter turnout
2 Post-election
% Govt. worker
% Govt. worker
Dem. pres. margin
Dem. pres. margin
Dem. pres. margin
Dem. pres. m
Dem. pres. margin Dem. governor Rep. inc. in
Dem. pres. margin Dem. governor Rep. inc. in race
advantages in
Rep. inc. in race
Rep. inc. in race
Rep. inc. in race
Rep. experience adv.
Rep. inc. in race
Inc.'s D1 nominate
Rep. experience adv.
Inc.'s D1 nom
Partisan
Rep. experience adv. Rep. experience adv. Partisan swing
vote tallying,
Partisan swing
Partisan swing
Rep. # prev. terms
Inc.'s D1 nominate
Rep. # prev. terms
% Govt. worker
Partisan swing
% Govt. w
Rep. experienc
Inc.'s D1 nominate Partisan swing Rep. experience adv.
e.g., election
Rep. # prev. terms
Rep. # prev. terms
Inc.'s D1 nominate
Inc.'s D1 nominate
Rep. # prev. terms
Rep. # prev.

–1.5 –1.0 –0.5 0.0 0.5 1.0 1.5 –1.5 –1.0 –0.5 0.0 0.5 1.0 1.5
fraud Estimated
sorting
–1.5 –1.0 –0.5
discontinuity
0.0 0.5 1.0 1.5 –1.5 –1.0 –0.5
Estimated
Estimated
0.0 0.5 1.0 1.5
discontinuity
discontinuity
−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5
Estimated discontinuity
Estimated discontinuity
(standard deviation units (standard deviation
(standard unitsunits
deviation (standard deviation
(standard unitsunits
deviation
for nonbinary measures) for nonbinary measures)
for nonbinary measures) for nonbinary measures)
for nonbinary measures)

Figure 2 Figure 2

Comparison of estimated
Kosuke Imai discontinuities Comparison
(Harvard) in pretreatment of estimated
covariates discontinuities
across three
Regression in pretreatment
methods. Solid
Discontinuity and dashedcovariates
Designs across
lines in each three
panel methods.
represent
Stat186/Gov2002 Solid
95% and
Falldashed
confidence lines innot
intervals,
2019 8each
/ 16pane
 ARTICLE IN PRESS
Density Test of Sorting
J. McCrary (McCrary. 2008.
/ Journal of Econometrics J. Econom.)
142 (2008) 698–714

150
1.60
120 1.40
Frequency Count

Density Estimate
1.20
90 1.00
0.80
60
0.60

30 0.40
0.20
0 0.00
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
Democratic Margin

Democratic
1 vote share
Create relative to with
histogram cutoff:apopular elections
selected bintosize
the House of Representatives, 1900–1990.
2Fit local linear regression to bin midpoints to smooth the histogram
3Estimate the difference in the logged histogram height at the
estimates threshold
Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 9 / 16
Placebo Test
838 QUARTERLY JOURNAL OF ECONOMICS

FIGURE V
Specification Test: Similarity of Historical Voting Patterns between Bare
Democrat and Republican Districts
The panel plots one time lagged ADA scores against the Democrat vote share.
Time t and t ⫺ 1 refer to congressional sessions. Each point is the average lagged
What is a good placebo?
ADA score within intervals of 0.01 in Democrat vote share. The continuous line is
from a fourth-order polynomial in vote share fitted separately for points above and
1 expected not to have any effect
below the 50 percent threshold. The dotted line is the 95 percent confidence
interval.
2 closely related to outcome of interest
Lagged outcome future cannot affect past
V.C. Sensitivity to Alternative Measures of Voting Records
Interpretation: failure
Our results to are
so far reject
basedthe
on anull 6= the
particular nullindex,
voting is correct
the
ADA score. In this section we investigate whether our results
Kosuke Imai (Harvard)
generalize Regression
to other voting Discontinuity
scores. We find Designs Stat186/Gov2002
that the findings do not Fall 2019 10 / 16
Fuzzy RD Design (Hahn et al. 2001. Econometrica)
Sharp regression discontinuity design: Ti = 1{Xi ≥ c}
What happens if we have noncompliance?
Forcing variable as an instrument: Zi = 1{Xi ≥ c}
Potential outcomes: Ti (z) and Yi (z, t)
Assumptions
1 Monotonicity: Ti (1) ≥ Ti (0)
2 Exclusion restriction: Yi (0, t) = Yi (1, t)
3 E(Ti (z) | Xi = x) and E(Yi (z, Ti (z)) | Xi = x) are continuous in x
Estimand:

E(Yi (1, Ti (1)) − Yi (0, Ti (0)) | Complier , Xi = c)

Estimator:
limx↓c E(Yi | Xi = x) − limx↑c E(Yi | Xi = x)
limx↓c E(Ti | Xi = x) − limx↑c E(Ti | Xi = x)
Disadvantage: external validity
Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 11 / 16
Class Size Effect (Angrist and Lavy. 1999. Q. J. Econ)
Effect of class-size on student test scores
Maimonides’ Rule: Maximum class size = 40
z
f (z) =  z−1 
40 +1
40
35
30
Class Size

25
20
15

Maimonides Rule
Actual class size
10

0 50 100 150 200

Enrollment Count

Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 12 / 16

Empirical Analysis
Yi : class average verbal test score
Ti : classsize
Window size: w
Construction of forcing variable:

 40 − Zi if 40 − w/2 ≤ Zi ≤ 40 + w/2

Xi = 80 − Zi if 80 − w/2 ≤ Zi ≤ 80 + w/2
 .. ..
. .


Linear models (cluster standard errors by schools):

Ti = δ1 + α1 × I{Xi ≥ 0} + β1 Xi + γ1 Xi × I{Xi ≥ 0} + 1i

Yi = δ2 + α2 × I{Xi ≥ 0} + β2 Xi + γ2 Xi × I{Xi ≥ 0} + 2i

where α̂1 = −7.90 (s.e. = 1.90) and α̂2 = −0.056 (s.e. = 2.08)
Two-stage least squares estimate: est. = 0.007 (s.e. = 0.261)
Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 13 / 16
Interrupted Time Series Design

Time as the forcing variable

Possibility of multiple events at the same time
Must model time trend: seasonality, etc.
Effect of the “stand your ground” bill in Florida on homicide

(Humphreys, Gasparrini, and Wiebe. 2017. JAMA)

Use of other states difference-in-differences designs

Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 14 / 16

 to reduce the incidence of compound treatments as much as possible. To do that, we
he boundaries of four different administrative units: U.S. congressional districts, state

Geographical RD Design (Keele and Titiunik. 2015. Political Anal.)

ricts, state house districts, and school districts. We found that for many parts of the
ket boundary, the boundaries of at least one of these units overlapped perfectly with the
ket boundary. In other words, in various boundary segments, not only did the media
ange at the boundary, but so did the school and/or the legislative districts. This is not
rprising since, as we discussed above, the media market boundary in this area (and in
RD in two dimensions
e United States) follows county boundaries.
rlap between media and county boundaries means that we cannot escape the problem of
don’t use distance as forcing variable
treatments entirely, but we can minimize it by restricting our analysis to those segments
Example: media markets in Princeton, New Jersey

144 Luke J. Keele and Rocı́o Titiunik

ect to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1093/pan/mpu014

Frankling Township School District

Montgomery Township School District

New York-Philadelphia Media Market Boundary

South Brunswick School District

Treated Area of Analysis

Hopewell Valley School District
Control Area of Analysis
Princeton School District

Cranbury Township School District

West Windsor-Plainsboro School District

Lawrence Township School District

East Windsor School District

Robbinsville Township School District

Milestone Township School Distric
New York Media Market
Philadelphia Media Market

Upper Freehold School District

undary between Philadelphia and New York City media markets. The dashed Fig.line
5 represents
Detail of the boundary between Philadelphia and New York City media markets. Area marked with
ry between the Philadelphia, PA, media market (located southwest of the boundary)
gray hash and
linesthe
indicates the West Windsor-Plainsboro school district, which straddles the media market
Kosuke
City, NY, media marketImai (Harvard)
(located Regression
northeast of the boundary), which divides the Discontinuity
boundary. state of analysisDesigns
Empirical Stat186/Gov2002
is confined to the West Windsor-Plainsboro schoolFall 2019
district only, where 15 / 16
legis-
Summary

Observational studies treatment assignment is not random

“Design” observational studies for credible causal inference
Sharp regression discontinuity (RD) designs:
deterministic (rather than stochastic) treatment assignment rule
continuity assumption no sorting
does not require “as-if random” assumption
limited external validity extrapolation required for generalization
incumbency effects controversy (Eggers et al. 2015. Am. J. Political Sci)
Importance of placebo tests
Fuzzy RD designs: noncompliance
Other RD designs: interrupted time series, geographical boundary
Suggested readings:
A NGRIST AND P ISCHKE, Chapter 6
I MBENS AND L EMIEUX. (2008). J. of Econom

Kosuke Imai (Harvard) Regression Discontinuity Designs Stat186/Gov2002 Fall 2019 16 / 16