Professional Documents
Culture Documents
Abadie Et Al. (2009), Synthetic Control Methods For Comparative Case Studies - Estimating The Effect of California's Tobacco
Abadie Et Al. (2009), Synthetic Control Methods For Comparative Case Studies - Estimating The Effect of California's Tobacco
Abadie Et Al. (2009), Synthetic Control Methods For Comparative Case Studies - Estimating The Effect of California's Tobacco
To cite this article: Alberto Abadie, Alexis Diamond & Jens Hainmueller (2010): Synthetic Control Methods for
Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program, Journal of the American
Statistical Association, 105:490, 493-505
This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to
anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses
should be independently verified with primary sources. The publisher shall not be liable for any loss, actions,
claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.
Synthetic Control Methods for Comparative Case
Studies: Estimating the Effect of California’s
Tobacco Control Program
Alberto A BADIE, Alexis D IAMOND, and Jens H AINMUELLER
Building on an idea in Abadie and Gardeazabal (2003), this article investigates the application of synthetic control methods to comparative
case studies. We discuss the advantages of these methods and apply them to study the effects of Proposition 99, a large-scale tobacco
control program that California implemented in 1988. We demonstrate that, following Proposition 99, tobacco consumption fell markedly
in California relative to a comparable synthetic control region. We estimate that by the year 2000 annual per-capita cigarette sales in
California were about 26 packs lower than what they would have been in the absence of Proposition 99. Using new inferential methods
proposed in this article, we demonstrate the significance of our estimates. Given that many policy interventions and events of interest in
social sciences take place at an aggregate level (countries, regions, cities, etc.) and affect a small number of aggregate units, the potential
applicability of synthetic control methods to comparative case studies is very large, especially in situations where traditional regression
methods are not appropriate.
KEY WORDS: Observational studies; Proposition 99; Tobacco control legislation; Treatment effects.
Downloaded by [Lakehead University] at 03:09 19 June 2013
493
494 Journal of the American Statistical Association, June 2010
and Gardeazabal (2003). Data-driven procedures reduce discre- comparative case studies of aggregate events. In Section 3 we
tion in the choice of the comparison control units, forcing re- apply synthetic control methods to estimate the effect of Cali-
searchers to demonstrate the affinities between the affected and fornia’s Proposition 99. Section 4 concludes. Appendix A lists
unaffected units using observed quantifiable characteristics. In the data sources for the application in Section 3. Appendix B
practice, it is often difficult to find a single unexposed unit that contains technical details.
approximates the most relevant characteristics of the unit(s) ex-
posed to the event of interest. The idea behind the synthetic 2. SYNTHETIC CONTROL METHODS FOR
control approach is that a combination of units often provides a COMPARATIVE CASE STUDIES
better comparison for the unit exposed to the intervention than 2.1 Comparative Case Studies
any single unit alone. For example, in their study of the eco-
nomic impact of terrorism in the Basque Country, Abadie and Case studies focus on particular occurrences of events or in-
Gardeazabal (2003) use a combination of two Spanish regions terventions of interest. Often, the motivation behind case stud-
to approximate the economic growth that the Basque Coun- ies is to detect the effects of an event or policy intervention on
try would have experienced in the absence of terrorism. Card some outcome of interest by focusing on a particular instance
(1990) implicitly uses a combination of cities in the southern in which the magnitude of the event or intervention is large rel-
United States to approximate the evolution that the Miami la- ative to other determinants of the outcome, or in which iden-
bor market would have experienced in the absence of the Mariel tification of the effects of interest is facilitated by some other
Boatlift. characteristic of the intervention. In comparative case studies,
Relative to traditional regression methods, transparency and researchers compare one or more units exposed to the event or
Downloaded by [Lakehead University] at 03:09 19 June 2013
safeguard against extrapolation are two attractive features of intervention of interest to one or more unexposed units. There-
the synthetic control method. Because a synthetic control is fore, comparative case studies are only feasible when some
a weighted average of the available control units, the syn- units are exposed and others are not (or when their levels of
thetic control method makes explicit: (1) the relative contribu- exposure differ notably).
tion of each control unit to the counterfactual of interest; and To simplify the exposition, we proceed as if only one unit or
(2) the similarities (or lack thereof) between the unit affected region is subject to the intervention of interest (otherwise, we
by the event or intervention of interest and the synthetic con- could first aggregate the data from the regions exposed to the
trol, in terms of preintervention outcomes and other predictors intervention). In addition, we adopt the terms “region” or “unit”
of postintervention outcomes. Because the weights can be re- and “intervention” or “treatment,” which can be substituted for
stricted to be positive and sum to one, the synthetic control “country,” “state,” “city,” etc. and “event,” “shock,” “law,” etc.,
method provides a safeguard against extrapolation. respectively for specific applications.
In addition, because the choice of a synthetic control does not
2.2 A Motivating Model
require access to postintervention outcomes, the synthetic con-
trol method allows researchers to decide on study design with- The following simple model provides a rationale for the
out knowing how those decisions will affect the conclusions of use of synthetic control methods in comparative case study re-
their studies. Rubin (2001) and others have advocated that the search. Suppose that we observe J + 1 regions. Without loss of
ability to make decisions on research design while remaining generality, suppose also that only the first region is exposed to
blind to how each particular decision affects the conclusions of the intervention of interest, so that we have J remaining regions
the study is an important device for promoting research honesty as potential controls. Borrowing from the statistical matching
in observational studies. literature, we refer to the set of potential controls as the “donor
We describe a simple model that justifies the synthetic con- pool.” Also without loss of generality and to simplify notation,
trol approach. The model extends the traditional linear panel assume that the first region is uninterruptedly exposed to the
data (difference-in-differences) framework, allowing that the intervention of interest after some initial intervention period.
effects of unobserved variables on the outcome vary with time. Let YitN be the outcome that would be observed for region
In addition, we propose a new method to perform inferential ex- i at time t in the absence of the intervention, for units i =
ercises in comparative case studies that produce potentially in- 1, . . . , J + 1, and time periods t = 1, . . . , T. Let T0 be the num-
formative inference regardless of the number of available com- ber of preintervention periods, with 1 ≤ T0 < T. Let YitI be the
parison units and the level of aggregation of the data. outcome that would be observed for unit i at time t if unit i
We apply the synthetic control method to study the effects of is exposed to the intervention in periods T0 + 1 to T. We as-
California’s Proposition 99, a large-scale tobacco control pro- sume that the intervention has no effect on the outcome be-
gram implemented in California in 1988. We demonstrate that fore the implementation period, so for t ∈ {1, . . . , T0 } and all
following the passage of Proposition 99 tobacco consumption i ∈ {1, . . . , N}, we have that YitI = YitN . In practice, interventions
fell markedly in California relative to a comparable synthetic may have an impact prior to their implementation (e.g., via an-
control region. We estimate that by the year 2000 annual per ticipation effects). In those cases, T0 could be redefined to be
capita cigarette sales in California were about 26 packs lower the first period in which the outcome may possibly react to the
than what they would have been in the absence of Proposi- intervention. Implicit in our notation is the usual assumption of
tion 99. Using new inferential methods proposed in this article, no interference between units (see Rosenbaum 2007 for a de-
we demonstrate the significance of our estimates. tailed discussion of the assumption of no interference between
The rest of the article is organized as follows. Section 2 de- units). That is, we assume that outcomes of the untreated units
scribes the main ideas behind the synthetic control approach to are not affected by the intervention implemented in the treated
Abadie, Diamond, and Hainmueller: Synthetic Control Methods for Comparative Case Studies 495
unit. In Section 3 we discuss this assumption in the context of zero if the number of preintervention periods is large relative to
our empirical investigation. the scale of the transitory shocks. This suggests using
Let αit = YitI − YitN be the effect of the intervention for unit i at
time t, and let Dit be an indicator that takes value one if unit i is
J+1
α1t = Y1t −
w∗j Yjt
exposed to the intervention at time t, and value zero otherwise.
j=2
The observed outcome for unit i at time t is
for t ∈ {T0 + 1, . . . , T} as an estimator of α1t .
Yit = YitN + αit Dit . Equation (2) can hold exactly only if (Y11 , . . . , Y1T0 , Z1 ) be-
Because only the first region (region “one”) is exposed to the longs to the convex hull of {(Y21 , . . . , Y2T0 , Z2 ), . . . , (YJ+11 ,
intervention and only after period T0 (with 1 ≤ T0 < T), we . . . , YJ+1T0 , ZJ+1 )}. In practice, it is often the case that no set
have that of weights exists such that Equation (2) holds exactly in the
data. Then, the synthetic control region is selected so that Equa-
1 if i = 1 and t > T0 ,
Dit = tion (2) holds approximately. In some cases, it may not even be
0 otherwise. possible to obtain a weighted combination of untreated units
We aim to estimate (α1T0 +1 , . . . , α1T ). For t > T0 , such that Equation (3) holds approximately. This would be the
case if (Y11 , . . . , Y1T0 , Z1 ) falls far from the convex hull of
α1t = Y1t
I
− Y1t
N
= Y1t − Y1t
N
. {(Y21 , . . . , Y2T0 , Z2 ), . . . , (YJ+11 , . . . , YJ+1T0 , ZJ+1 )}. Notice,
Because Y1t I is observed, to estimate α we just need to esti- however, that the magnitude of such discrepancy can be cal-
1t
culated for each particular application. So for each particular
Downloaded by [Lakehead University] at 03:09 19 June 2013
j=2 j=2
w∗j Zj = Z1 and w∗j μj = μ1 , (4)
j=2 j=2
T0
In Appendix B, we prove that if t=1 λt λt is nonsingular, then,
would provide an unbiased estimator of Y1t N . Choosing a syn-
−1 thetic control in this manner is, of course, not feasible because
J+1
J+1
T0
T0
N
Y1t − w∗j Yjt = w∗j λt λn λn λs (εjs − ε1s ) μ1 , . . . , μJ+1 are not observed. However, under fairly standard
j=2 j=2 s=1 n=1 conditions (see Appendix B), the factor model in Equation (1)
implies that a synthetic control can fit Z1 and a long set of prein-
J+1
tervention outcomes, Y11 , . . . , Y1T0 , only as long as it fits Z1
− w∗j (εjt − ε1t ). (3)
and μ1 , so Equation (4) holds approximately.
j=2
Synthetic controls can provide useful estimates in more gen-
In Appendix B we prove also that, under standard conditions, eral contexts than the factor model considered thus far. Con-
the mean of the right-hand side of Equation (3) will be close to sider, for example, the following autoregressive model with
496 Journal of the American Statistical Association, June 2010
and w2 + · · · + wJ+1 = 1. Each value of W represents a also be data-driven. One possibility is to choose V such that the
weighted average of the available control regions and, there- resulting synthetic control region approximates the trajectory
fore, a synthetic control. Notice that, although we define our of the outcome variable of the affected region in the preinter-
synthetic controls as convex combinations of unexposed units, vention periods. Following Abadie and Gardeazabal (2003), in
negative weights or weights larger than one can be used at the the empirical section of this article we choose V among posi-
cost of allowing extrapolation. tive definite and diagonal matrices such that the mean squared
The outcome variable of interest is observed for T periods, prediction error of the outcome variable is minimized for the
t = 1, . . . , T, for the region affected by the intervention, Y1t , preintervention periods (see Abadie and Gardeazabal 2003, ap-
and the unaffected regions, Yjt , where j = 2, . . . , J + 1. Let pendix B, for details). Alternatively, if the number of avail-
the (T0 × 1) vector K = (k1 , . . . , kT0 ) define a linear combi- able preintervention periods in the sample is large enough, re-
T0 searchers may divide them into an initial training period and a
nation of preintervention outcomes: ȲiK = s=1 ks Yis . For ex-
subsequent validation period. Given a V, W∗ (V) can be com-
ample, if k1 = k2 = · · · = kT0 −1 = 0 and kT0 = 1, then Ȳ K = puted using data from the training period. Then, the matrix V
YiT0 , the value of the outcome variable in the period immedi- can be chosen to minimize the mean squared prediction error
ately prior to the intervention. If k1 = k2 = · · · = kT0 = 1/T0 , produced by the weights W∗ (V) during the validation period.
T0
then ȲiK = T0−1 s=1 Yis , the simple average of the outcome
variable for the preintervention periods. Consider M of such 2.4 Inference
linear combinations defined by the vectors K1 , . . . , KM . Let The standard errors commonly reported in regression-based
X1 = (Z1 , Ȳ1K1 , . . . , Ȳ1KM ) be a (k × 1) vector of preinterven- comparative case studies measure uncertainty about aggregate
tion characteristics for the exposed region, with k = r + M. data. For example, Card (1990) uses data from the U.S. Cur-
Similarly, X0 is a (k × J) matrix that contains the same vari- rent Population Survey to estimate native employment rates in
ables for the unaffected regions. That is, the jth column of X0 Miami and a set of comparison cities around the time of the
is (Zj , ȲjK1 , . . . , ȲjKM ) . The vector W∗ is chosen to minimize Mariel Boatlift. Card and Krueger (1994) use data on a sample
some distance, X1 − X0 W, between X1 and X0 W, subject of fast food restaurants in New Jersey and Pennsylvania to esti-
to w2 ≥ 0, . . . , wJ+1 ≥ 0, w2 + · · · + wJ+1 = 1. One obvious mate the average number of employees in fast food restaurants
KT in these two states around the time when the minimum wage
choice for ȲiK1 , . . . , ȲiKM is ȲiK1 = Yi1 , . . . , Ȳi 0 = YiT0 , that is, was increased in New Jersey. The standard errors reported in
the values of the outcome variable for all the available prein- these studies reflect only the unavailability of aggregate data
tervention periods. In practice, however, the computation of the on employment (for native workers in Miami and other cities,
weights w∗2 , . . . , w∗J+1 can be simplified by considering only a and in fast food restaurants in New Jersey and Pennsylvania,
few linear combination of preintervention outcomes and check- respectively). This mode of inference would logically produce
ing whether Equation (2) holds approximately for the resulting zero standard errors if aggregate data were used for estimation.
weights. However, perfect knowledge of aggregate data does not elim-
To measure the discrepancy√between X1 and X0 W, we inate all uncertainty about the parameters of interest. That is,
will employ X1 − X0 WV = (X1 − X0 W) V(X1 − X0 W), even if aggregate data are used for estimation, in most cases
where V is some (k × k) symmetric and positive semidefinite researchers would not believe that there is no remaining uncer-
matrix, although other choices are also possible. If the relation- tainty about the value of the parameters of interest. The reason
ship between the outcome variable and the explanatory vari- is that not all uncertainty about the value of the estimated pa-
ables in X1 and X0 is highly nonlinear and the support of the ex- rameters comes from lack of knowledge of aggregate data. In
planatory variables is large, interpolation biases may be severe. comparative case studies, an additional source of uncertainty
(For instance, an equally weighted combination of a 65%-White derives from ignorance about the ability of the control group to
Abadie, Diamond, and Hainmueller: Synthetic Control Methods for Comparative Case Studies 497
reproduce the counterfactual of how the treated unit would have of no intervention effect the estimated effect of the interven-
evolved in the absence of the treatment. This type of uncertainty tion is not expected to be abnormal relative to the distribution
is present regardless of whether aggregate data are used for es- of the placebo effects. In this sense, our inferential procedure
timation or not. The use of individual micro data, as opposed to is related to that of DiNardo and Pischke (1997) and Auld and
aggregate data, only increases the total amount of uncertainty if Grootendorst (2004). DiNardo and Pischke (1997) compare the
the outcome of interest is an aggregate. wage differential associated with computer skills (as reflected
Large sample inferential techniques are not well suited to in the on-the-job computer use) to the wage differentials associ-
comparative case studies when the number of units in the com- ated with the use of other tools (pencils, telephones, calculators)
parison group is small. In this article, we propose exact inferen- that do not proxy for skills that are scarce in the job market.
tial techniques, akin to permutation tests, to perform inference Similarly, to assess the validity of the rational addiction model,
in comparative case studies. The methods proposed here can be Auld and Grootendorst (2004) compare the result of a test of
used whether data are individual (micro) or aggregate (macro), rational addiction for cigarette consumption to the results of the
and do not require a large number of comparison units in the same test applied to substances that are not considered addictive
donor pool. (milk, eggs, oranges, apples).
The inferential techniques proposed in this article are related For cases in which the number of available comparison re-
to Abadie and Gardeazabal (2003). In their study of the eco- gions is very small, one can use the longitudinal dimension of
nomic effects of terrorism, Abadie and Gardeazabal (2003) use the data to produce placebo studies, as in Bertrand, Duflo, and
a synthetic control region to estimate the economic growth that Mullainathan (2004) where the dates of the placebo interven-
the Basque Country would have experienced in the absence of tions are set at random. Heckman and Hotz (1989) provide an
Downloaded by [Lakehead University] at 03:09 19 June 2013
for tobacco-related research. As Glantz and Balbach (2000) put Previous studies have investigated the impact of Proposi-
it, “[t]hese programs dwarfed anything that any other state or tion 99 on smoking prevalence using a variety of methods. Bres-
the federal government had ever done on tobacco.” low and Johnson (1993), Glantz (1993), and Pierce et al. (1998)
Proposition 99 triggered a wave of local clean-air ordinances show that cigarette consumption in California after the passage
in California. Before Proposition 99 no city or town in Califor- of Proposition 99 in 1988 was lower than the average national
nia required restaurants to be 100% smoke-free. From 1989 to trend and lower than the linearly extrapolated preprogram trend
2000 approximately 140 such laws were passed (Siegel 2002). in California. Hu, Sung, and Keeler (1995) use time-series re-
By 1993 local ordinances prohibiting smoking in the workplace gression to disaggregate the effects of Proposition 99’s tax hike
protected nearly two-thirds of the workers in California (Glantz and media campaign on per capita cigarette sales.
and Balbach 2000). In 1994 the State of California passed addi- A related literature has studied the effect of smoking bans on
tional legislation that banned smoking in enclosed workplaces. smoking prevalence. Woodruff et al. (1993) show that smok-
By 1996 more than 90% of California workers were covered ing prevalence in California in 1990 was lower among work-
by a smoke-free workplace policy (Siegel 2002). Nonsmokers’ ers affected by workplace smoking restrictions than among un-
rights advocates view the wave of local ordinances passed un- affected workers. More generally, Evans, Farrelly, and Mont-
der the impetus of Proposition 99 as an important step in the gomery (1999), Farrelly, Evans, and Sfekas (1999), and Longo
effort to undercut the then-existing social support network for et al. (2001) have provided evidence on the effectiveness of
tobacco use in California (Glantz and Balbach 2000). workplace smoking bans.
The tobacco industry responded to Proposition 99 and the In a study closely related to the analysis in this section, Ficht-
spread of clean-air ordinances by increasing its political ac- enberg and Glantz (2000) use least-squares regression to predict
Downloaded by [Lakehead University] at 03:09 19 June 2013
tivity in California at both the state and local levels. Tobacco smoking rates in California as a function of the smoking rate
lobby groups spent 10 times as much money in California in for the rest of the United States. The regressions in Fichten-
1991–1992 as they had spent in 1985–1986 (Begay, Traynor, berg and Glantz (2000) estimate the effect of Proposition 99 as
and Glantz 1993). In addition, after the passage of Proposi- a time trend in per capita cigarette consumption starting after
tion 99, tobacco companies increased promotional expenditures the implementation of Proposition 99 in 1989. Fichtenberg and
in California (Siegel 2002). Glantz (2000) allow also for a change in this trend after 1992,
In 1991 California passed Assembly Bill 99, a new piece of when the anti-tobacco media campaign was first temporally
legislation implementing Proposition 99. Contrary to the orig- eliminated and then reestablished but with reduced funds. Us-
inal mandate of Proposition 99, Assembly Bill 99 diverted a ing this regression specification, Fichtenberg and Glantz (2000)
significant fraction of Proposition 99 tobacco tax revenues into estimate that during the period 1989–1992 Proposition 99 ac-
medical services with little or no connection to tobacco (Glantz celerated the rate of decline of per capita cigarette consumption
and Balbach 2000). In 1992 Governor Pete Wilson halted the in California by 2.72 packs per year. Due to program cut-backs
media campaign, which provoked a lawsuit by the American after 1992, Fichtenberg and Glantz (2000) estimate that dur-
Lung Association (ALA). The ALA won the suit and the cam- ing the period 1993–1997 Proposition 99 accelerated the rate
paign was back by the end of 1992, although with a reduced of decline of per capita cigarette consumption in California by
budget (Siegel 2002). only 0.67 packs per year.
Even so, Proposition 99 was widely perceived to have suc-
3.2 Data and Sample
cessfully cut smoking in California. From the passage of Propo-
sition 99 through 1999 adult smoking prevalence fell in Cal- We use annual state-level panel data for the period 1970–
ifornia by more than 30%, youth smoking levels dropped to 2000. Proposition 99 was passed in November 1988 and went
the lowest in the country, and per capita cigarette consumption into effect in January 1989, giving us 19 years of preinter-
more than halved (California Department of Health Services vention data. Our sample period begins in 1970 because it is
2006). Prior to 1988 per capita cigarette consumption in Cali- the first year for which data on cigarette sales are available
fornia trailed the national average by 22.5 packs; ten years later for all our control states. It ends in 2000 because at about
per capita consumption was 40.4 packs lower than the national this time anti-tobacco measures were implemented across many
average (Siegel 2002). states, invalidating them as potential control units. Moreover, a
Following early reports of California’s success with Propo- decade-long period after the passage of Proposition 99 seems
sition 99, other states adopted similar policies. In 1993 Massa- like a reasonable limit on the span of plausible prediction of the
chusetts raised taxes on cigarettes from 26 to 51 cents per pack effect of this intervention.
to fund a Health Protection Fund for smoking prevention and Recall that the synthetic California is constructed as a
cessation programs. Similar laws passed in Arizona in 1994, weighted average of potential control states, with weights cho-
with a 50-cent tax increase, and Oregon in 1996, where the tax sen so that the resulting synthetic California best reproduces
on cigarettes rose from 38 to 68 cents per pack (Siegel 2002). the values of a set of predictors of cigarette consumption in
In November 1998 the tobacco companies signed a $206 bil- California before the passage of Proposition 99. Because the
lion Master Settlement Agreement that led the industry to im- synthetic California is meant to reproduce the smoking rates
pose an immediate 45-cent increase in cigarette prices nation- that would have been observed for California in the absence
wide (Capehart 2001). As of April 20, 2009, 30 states, the Dis- of Proposition 99, we discard from the donor pool states that
trict of Columbia, and 792 municipalities across the country had adopted some other large-scale tobacco control program during
laws in effect requiring 100% smoke-free workplaces, bars, or our sample period. Four states (Massachusetts, Arizona, Ore-
restaurants (ANRF 2009). gon, and Florida) introduced formal statewide tobacco control
Abadie, Diamond, and Hainmueller: Synthetic Control Methods for Comparative Case Studies 499
per capita does not have substantial power predicting the per thetic California.
capita cigarette consumption in California before the passage
of Proposition 99. This explains the discrepancy between Cali-
that, in contrast to per capita sales in other U.S. states (shown
fornia and its synthetic version in terms of log GDP per capita.
in Figure 1), per capita sales in the synthetic California very
Table 2 displays the weights of each control state in the syn-
thetic California. The weights reported in Table 2 indicate that closely track the trajectory of this variable in California for the
smoking trends in California prior to the passage of Proposi- entire pre-Proposition 99 period. Combined with the high de-
tion 99 is best reproduced by a combination of Colorado, Con- gree of balance on all smoking predictors (Table 1), this sug-
necticut, Montana, Nevada, and Utah. All other states in the gests that the synthetic California provides a sensible approxi-
donor pool are assigned zero W-weights. mation to the number of cigarette packs per capita that would
Figure 2 displays per capita cigarette sales for California and have been sold in California in 1989–2000 in the absence of
its synthetic counterpart during the period 1970–2000. Notice Proposition 99.
Our estimate of the effect of Proposition 99 on cigarette con-
sumption in California is the difference between per capita ciga-
Table 2. State weights in the synthetic California
rette sales in California and in its synthetic version after the pas-
State Weight State Weight sage of Proposition 99. Immediately after the law’s passage, the
two lines begin to diverge noticeably. While cigarette consump-
Alabama 0 Montana 0.199
tion in the synthetic California continued on its moderate down-
Alaska – Nebraska 0
Arizona – Nevada 0.234 ward trend, the real California experienced a sharp decline. The
Arkansas 0 New Hampshire 0 discrepancy between the two lines suggests a large negative ef-
Colorado 0.164 New Jersey – fect of Proposition 99 on per capita cigarette sales. Figure 3
Connecticut 0.069 New Mexico 0 plots the yearly estimates of the impacts of Proposition 99, that
Delaware 0 New York – is, the yearly gaps in per capita cigarette consumption between
District of Columbia – North Carolina 0 California and its synthetic counterpart. Figure 3 suggests that
Florida – North Dakota 0 Proposition 99 had a large effect on per capita cigarette sales,
Georgia 0 Ohio 0 and that this effect increased in time. The magnitude of the es-
Hawaii – Oklahoma 0
timated impact of Proposition 99 in Figure 3 is substantial. Our
Idaho 0 Oregon –
Illinois 0 Pennsylvania 0
results suggest that for the entire 1989–2000 period cigarette
Indiana 0 Rhode Island 0 consumption was reduced by an average of almost 20 packs per
Iowa 0 South Carolina 0 capita, a decline of approximately 25%.
Kansas 0 South Dakota 0 In order to assess the robustness of our results, we included
Kentucky 0 Tennessee 0 additional predictors of smoking prevalence among the vari-
Louisiana 0 Texas 0 ables used to construct the synthetic control. Our results stayed
Maine 0 Utah 0.334 virtually unaffected regardless of which and how many predic-
Maryland – Vermont 0 tor variables we included. The list of predictors used for robust-
Massachusetts – Virginia 0 ness checks included state-level measures of unemployment,
Michigan – Washington –
income inequality, poverty, welfare transfers, crime rates, drug
Minnesota 0 West Virginia 0
Mississippi 0 Wisconsin 0
related arrest rates, cigarette taxes, population density, and nu-
Missouri 0 Wyoming 0 merous variables to capture the demographic, racial, and social
structure of states.
Abadie, Diamond, and Hainmueller: Synthetic Control Methods for Comparative Case Studies 501
thetic California.
scale tobacco control program during the sample period of our
study. If the placebo studies create gaps of magnitude similar to
the one estimated for California, then our interpretation is that
Our analysis produces estimates of the effect of Proposi- our analysis does not provide significant evidence of a nega-
tion 99 that are considerably larger than those obtained by tive effect of Proposition 99 on cigarette sales in California. If,
Fichtenberg and Glantz (2000) using linear regression meth- on the other hand, the placebo studies show that the gap esti-
ods. In particular, Fichtenberg and Glantz (2000) estimate that mated for California is unusually large relative to the gaps for
by 1997 Proposition 99 had reduced per capita cigarette sales in the states that did not implement large-scale tobacco control
California by about 14 packs per year. Our estimates increase program, then our interpretation is that our analysis provides
this figure substantially, to 24 packs per year. Part of this dif- significant evidence of a negative effect of Proposition 99 on
ference is likely to be explained by the fact that Fichtenberg cigarette sales in California.
and Glantz (2000) use per capita cigarette sales in the rest of To assess the significance of our estimates, we conduct a
the United States to reproduce how this variable would have series of placebo studies by iteratively applying the synthetic
evolved in California in the absence of Proposition 99. As ex- control method used to estimate the effect of Proposition 99 in
plained above, after the enactment of Proposition 99 in Califor- California to every other state in the donor pool. In each iter-
nia, other states, like Massachusetts and Florida passed similar ation we reassign in our data the tobacco control intervention
tobacco control legislation. While we eliminate these states as to one of the 38 control states, shifting California to the donor
potential controls, Fichtenberg and Glantz (2000) do not do so, pool. That is, we proceed as if one of the states in the donor
which is likely to attenuate their estimates. pool would have passed a large-scale tobacco control program
There are several ways in which the assumption of no in- in 1988, instead of California. We then compute the estimated
terference between units of Section 2 could be violated in the effect associated with each placebo run. This iterative proce-
context of our analysis of the effects of Proposition 99. In our dure provides us with a distribution of estimated gaps for the
judgment, these potential violations do not appear to be se- states where no intervention took place.
vere, and in some cases would likely attenuate the estimated Figure 4 displays the results for the placebo test. The gray
effect of Proposition 99. Perhaps the most important concern lines represent the gap associated with each of the 38 runs of
in this regard is that the increase in anti-tobacco sentiment the test. That is, the gray lines show the difference in per capita
created in California by Proposition 99 could have spread to cigarette sales between each state in the donor pool and its re-
other states, contaminating the donor pool. Another concern is spective synthetic version. The superimposed black line denotes
that in response to Proposition 99 the tobacco industry could the gap estimated for California. As the figure makes apparent,
have diverted funds from planned advertising campaigns in the estimated gap for California during the 1989–2000 period
other states to California. In both cases, interference would is unusually large relative to the distribution of the gaps for the
likely cause lower levels of smoking in the control states, ar- states in the donor pool.
tificially reducing the magnitude of our estimate of the effect As Figure 4 indicates, the synthetic method provides an
of Proposition 99. On the other hand, it is possible that the rise excellent fit for per capita cigarette sales in California prior
in tobacco taxes implemented under Proposition 99 increased to the passage of Proposition 99. The preintervention mean
cigarette smuggling or cross-border purchases from nearby ju- squared prediction error (MSPE) in California (the average of
risdictions. However, Lovenheim (2008) and DeCicca, Kenkel, the squared discrepancies between per capita cigarette sales in
and Liu (2008) provide evidence that large distances to lower California and in its synthetic counterpart during the period
tobacco price jurisdictions keep the level of cross-border ciga- 1970–1988) is about 3. The pre-Proposition 99 median MSPE
rette purchases low in California. There is much less informa- among the 38 states in the donor pool is about 6, also quite
tion about organized smuggling, although it has been argued small, indicating that the synthetic control method is able to
502 Journal of the American Statistical Association, June 2010
Figure 4. Per-capita cigarette sales gaps in California and placebo Figure 5. Per-capita cigarette sales gaps in California and placebo
Downloaded by [Lakehead University] at 03:09 19 June 2013
gaps in all 38 control states. gaps in 34 control states (discards states with pre-Proposition 99
MSPE twenty times higher than California’s).
4. CONCLUSION
Comparative case study research has broad potential in the
social sciences. However, the empirical implementation of
comparative case studies is plagued by inferential challenges
and ambiguity about the choice of valid control groups. In this
paper, we advocate the use of data-driven procedures to select
synthetic comparison units in comparative case studies. We dis-
cuss the conditions under which synthetic control estimators are
appropriate. In addition, we propose a method to produce infer-
ence in comparative cases studies that incorporates uncertainty
about the validity of the control unit. Moreover, we provide
software to implement the estimators proposed in this article
(available at the authors’ webpages).
We demonstrate the applicability of the synthetic control
Figure 7. Per-capita cigarette sales gaps in California and placebo
Downloaded by [Lakehead University] at 03:09 19 June 2013
YitN = δt + θ t Zi + λt μi + εit ,
∗
As a result, and R3t = J+1
j=2 wj (εjt − ε1t ). Consider the case of t > T0 . Then, R2t
and R3t have mean zero. Notice that,
J+1
J+1
N−
Y1t wj YjtN = θ t Z1 − wj Zj T0 T −1
J+1 0
j=2 j=2 R1t = wj∗ λt
λn λn λs εjs .
j=2 s=1 n=1
J+1
J+1
+ λt μ1 − wj μj + wj (ε1t − εjt ). T0
Because t=1 λt λt is symmetric and positive definite, so is its inverse.
j=2 j=2
Then, applying the Cauchy–Schwarz Inequality, we obtain
We assume that the terms εit are independent across units and in T −1 2
0
time. The analysis can, however, be extended to more general settings. λt λn λn λs
Notice that even with εit independent across units and in time, the n=1
unobserved residual uit = λt μi + εit may be correlated across units T −1 T −1
and in time because the presence of the term λt μi . Assume also that the 0 0
≤ λt
λn λn
λt λs
λn λn λs
terms εit are mean-independent of {Zi , μi }J+1 P
i=1 . Let Yi be the T0 × 1
n=1 n=1
vector with tth element equal to Yit . Similarly, let εP i be the (T0 × 2
P
1) vector with tth element equal to εit . Finally, let θ and λP be the λ̄ F 2
≤ .
(T0 × r) matrix and (T0 × F) matrix with tth rows equal to θ t and λt , T0 ξ
respectively. We obtain,
Let
T −1
Downloaded by [Lakehead University] at 03:09 19 June 2013
J+1
J+1 T0
0
YP1 − w Y
j j
P = θP Z −
1 w Z
j j ε̄jL = λt λn λn λs εjs
j=2 j=2 s=1 n=1
J+1
J+1 for j = 2, . . . , J + 1.
+ λP μ1 − wj μj + wj (ε P P
1 − ε j ). Assume that, for some even p, the pth moments of |εjt | exist for
j=2 j=2 j = 2, . . . , J + 1 and t = 1, . . . , T0 . Using Hölder’s Inequality:
Let ξ(M) be the smallest eigenvalue of J+1 1/p J+1 1/p
J+1
w∗j |ε̄jL | ≤ w∗j |ε̄jL |p ≤ |ε̄jL |p .
T0
1
λt λt .
j=2 j=2 j=2
M
t=T0 −M+1 Therefore, applying again Hölder’s Inequality:
J+1 1/p
Assume that ξ(M) is bounded away from zero: ξ(M) ≥ ξ > 0, for each
J+1
positive integer, M. Assume also that there exists a constant, λ̄, such E w∗j |ε̄jL | ≤ E |ε̄jL |p .
that |λtf | ≤ λ̄ for all t = 1, . . . , T, f = 1, . . . , F. Therefore, because j=2 j=2
{uit , vit }T0 +1≤t≤T0 +n have mean zero conditional on FT0 , the bias of Fichtenberg, C., and Glantz, S. (2000), “Association of the California Tobacco
the synthetic control estimator is equal to zero if Equation (6) holds. Control Program With Declines in Cigarette Consumption and Mortality
From Heart Disease,” New England Journal of Medicine, 343 (24), 1772–
[Received December 2008. Revised June 2009.] 1777. [498,501]
Glantz, S. (1993), “Changes in Cigarette Consumption, Prices, and Tobacco
REFERENCES Industry Revenues Associated With California’s Proposition 99,” Tobacco
Control, 2, 311–314. [498]
Abadie, A., and Gardeazabal, J. (2003), “The Economic Costs of Conflict: Glantz, S., and Balbach, E. (2000), Tobacco War: Inside the California Battles,
A Case Study of the Basque Country,” American Economic Review, 93 (1), Berkeley: University of California Press. Available at http:// ark.cdlib.org/
112–132. [493,494,496,497,501] ark:/13030/ ft167nb0vq/ . [498]
Angrist, J. D., and Krueger, A. B. (1999), “Empirical Strategies in Labor Eco- Heckman, J. J., and Hotz, V. J. (1989), “Choosing Among Alternatives Nonex-
nomics,” in Handbook of Labor Economics, Vol. 3, eds. A. Ashenfelter and perimental Methods for Estimating the Impact of Social Programs,” Journal
D. Card, Amsterdam: Elsevier. [497] of the American Statistical Association, 84, 862–874. [497]
ANRF (2009), “Municipalities With 100% Smokefree Laws,” available at Hu, T., Sung, H., and Keeler, T. (1995), “Reducing Cigarette Consumption in
http:// www.no-smoke.org/ pdf/ 100ordlisttabs.pdf . [498] California: Tobacco Taxes vs. an Anti-Smoking Media Campaign,” Ameri-
Athey, S., and Imbens, G. W. (2006), “Identification and Inference in Nonlinear can Journal of Public Health, 85 (9), 1218–1222. [498]
Difference-in-Differences Models,” Econometrica, 74 (2), 431–497. [497] Ibragimov, R., and Sharakhmetov, S. (2002), “The Exact Constant in the Rosen-
Auld, M. C., and Grootendorst, P. (2004), “An Empirical Analysis of Milk Ad- thal Inequality for Random Variables With Mean Zero,” Theory of Proba-
diction,” Journal of Health Economics, 23, 1117–1133. [497] bility and Its Applications, 46, 127–131. [504]
Begay, M., Traynor, M., and Glantz, S. (1993), “The Tobacco Industry, State King, G., and Zheng, L. (2006), “The Dangers of Extreme Counterfactuals,”
Politics, and Tobacco Education in California,” American Journal of Public Political Analysis, 14 (2), 131–159. [500]
Health, 83 (9), 1214–1221. [498] Kleine, R. (1993), “The Declining Role of Interstate Cigarette Smuggling in the
Bertrand, M., Duflo, E., and Mullainathan, S. (2004), “How Much Should United States,” Tobacco Control, 2, 38–40. [501]
We Trust Differences-in-Differences Estimates?” Quarterly Journal of Eco- Longo, D., Johnson, J., Krusea, R., Brownson, R., and Hewett, J. (2001),
nomics, 119 (1), 249–275. [497,501] “A Prospective Investigation of the Impact of Smoking Bans on Tobacco
Downloaded by [Lakehead University] at 03:09 19 June 2013
Breslow, L., and Johnson, M. (1993), “California’s Proposition 99 on Tobacco, Cessation and Relapse,” Tobacco Control, 10, 267–272. [498]
and Its Impact,” Annual Review of Public Health, 14, 585–604. [498] Lovenheim, M. F. (2008), “How Far to the Border?: The Extent and Impact of
California Department of Health Services (2006), “Fast Facts,” PS 11, available Cross-Border Casual Cigarette Smuggling,” National Tax Journal, 61 (1),
at http:// www.dhs.ca.gov/ opa/ FactSheets/ PDF/ ps11.pdf . [498] 7–33. [501]
Capehart, T. (2001), “Trends in the Cigarette Industry After the Master Settle- Orzechowski and Walker (2005), The Tax Burden on Tobacco. Historical Com-
ment Agreement,” Electronic Outlook Report TBS-250-01, USDA. [498] pilation, Vol. 40, Arlington, VA: Orzechowski & Walker. [499,503]
Card, D. (1990), “The Impact of the Mariel Boatlift on the Miami Labor Mar- Pierce, J., Gilpin, E., Emery, S., Farkas, A., Zhu, S., Choi, W., Berry, C.,
ket,” Industrial and Labor Relations Review, 44, 245–257. [493,494,496] Distefan, J., White, M., Soroko, S., and Navarro, A. (1998), Tobacco Con-
Card, D., and Krueger, A. B. (1994), “Minimum Wages and Employment: trol in California: Who’s Winning the War? An Evaluation of the Tobacco
A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania,” Control Program, 1989–1996, La Jolla, CA: University of California at San
American Economic Review, 84, 772–793. [493,496] Diego, Chapter 2. [498]
Cochran, W. G. (1937), “A Catalogue of Uniformity Trial Data,” Supplement to Rosenbaum, P. R. (2002a), Observational Studies (2nd ed.), New York:
the Journal of the Royal Statistical Society, 4 (2), 233–253. [497] Springer. [497]
Conley, T. G., and Taber, C. R. (2008), “Inference With Difference in Dif- (2002b), “Covariance Adjustment in Randomized Experiments and
ferences With a Small Number of Policy Changes,” Mimeo, University of Observational Studies,” Statistical Science, 17 (3), 286–327. [497]
Chicago. [497] (2007), “Interference Between Units in Randomized Experiments,”
DeCicca, P., Kenkel, D., and Liu, F. (2008), “Excise Tax Avoidance: The Case Journal of the American Statistical Association, 102 (477), 191–200. [494]
of State Cigarette Taxes,” Mimeo, McMaster University, Canada. [501] Rubin, D. B. (2001), “Using Propensity Scores to Help Design Observational
Dinan, J., and Heckelman, J. (2005), “The Anti-Tobacco Movement in the Pro- Studies: Application to the Tobacco Litigation,” Health Services and Out-
gressive Era: A Case Study of Direct Democracy in Oregon,” Explorations comes Research Methodology, 1, 169–188. [494]
in Economic History, 42 (4), 529–546. [497] Siegel, M. (2002), “The Effectiveness of State-Level Tobacco Control Interven-
DiNardo, J. E., and Pischke, J. S. (1997), “The Returns to Computer Use Revis- tions: A Review of Program Implementation and Behavioral Outcomes,”
ited: Have Pencils Changed the Wage Structure Too?” Quarterly Journal of Annual Review of Public Health, 23, 45–71. [497,498]
Economics, 112, 291–303. [497] Tate, C. (1999), Cigarette Wars: The Triumph of the Little White Slaver, New
Donald, S. G., and Lang, K. (2007), “Inference With Difference in Differences York: Oxford University Press. [497]
and Other Panel Data,” Review of Economics and Statistics, 89 (2), 221– Woodruff, T., Rosbrook, B., Pierce, J., and Glantz, S. (1993), “Lower Levels
233. [497] of Cigarette Consumption Found in Smoke-Free Workplaces in California,”
Evans, W., Farrelly, M., and Montgomery, E. (1999), “Do Workplace Smok- Archives of Internal Medicine, 153, 1485–1493. [498]
ing Bans Reduce Smoking?” American Economic Review, 89 (4), 728–747. Wooldridge, J. M. (2003), “Cluster-Sample Methods in Applied Econometrics,”
[498] American Economic Review, 93 (2), 133–138. [497]
Farrelly, M., Evans, W., and Sfekas, A. (1999), “The Impact of Workplace Wooldridge, J. M., and Imbens, G. W. (2008), “Recent Developments in the
Smoking Bans: Results From a National Survey,” Tobacco Control, 8, 272– Econometrics of Program Evaluation,” Working Paper W14251, NBER.
227. [498] [497]