Republic of Benin

University of Abomey-Calavi

(UAC)

Faculty of Agronomic Sciences

(FAS)

MASTER STATISTICS, ORIENTATION BIOSTATISTIC

DIFFERENCE IN DIFFERENCES (DD)

Group 6

Presented by : Lecturer :
Boris BEHINGAN Dr. Ir. Epiphane SODJINOU
Auric DJENONTIN Agricultural Economist, Biostatistician

Elisé TOHO
July 2016
Outline
Introduction.................................................................................................................................3
1- Principles.............................................................................................................................3
2- Limitations of the Difference-in-Differences Method.........................................................4
3- Application..........................................................................................................................5
Conclusion..................................................................................................................................6
References...................................................................................................................................7

2
Introduction
There are three impact evaluation methods: randomized assignment, randomized promotion,

and regression discontinuity design (RDD). These methods offer credible estimates of the
counterfactual with relatively few assumptions and conditions. Two types of methods
difference-in-differences (DD) and matching methods offer the evaluator an additional set of
tools that can be applied in situations in which the program assignment rules are less clear or
in which none of the three methods previously described is feasible. The present report will
focus on DD.

1- Principles
The difference-in-differences method compares the changes in outcomes over time between a
population that is enrolled in a program (the treatment group) and a population that is not (the
comparison group).

It also combines the two counterfeit counterfactuals (before-and-after comparisons and

comparisons between those who choose to enroll and those who choose not to enroll) to
produce a better estimate of the counterfactual. In our roads case, the DD method might
compare the change in employment before and after the program is implemented for
individuals living in areas affected by the road construction program to changes in
employment in areas where the roads program was not implemented.

For example, a road construction program. One of the program’s objectives is to improve
access to labor markets, with one of the outcome indicators being employment.

Tableau 1 : DD computation

After Before Difference

Treatment or enrolled B A B-A
Comparison or not enrolled D C D-C

Difference B-D A-C DD = (B – A) – (D - C)

After Before Difference
Treatment or enrolled 0.74 0.60 0.14
Comparison or not enrolled 0.81 0.78 0.03
Difference -0.07 -0.18 DD = 0.14 – 0.03 = 0.11

3
As the table 1 shows the DD equal to 0.11 and it is shown by the figure 1.

Figure 1 : DD method using graph

2- Limitations of the Difference-in-Differences Method

 DD attributes to the intervention any differences in trends between the treatment and
comparison groups that occur from the time intervention begins.
 If any other factors are present that affect the difference in trends between the two
groups, the estimation will be invalid or biased.
 For example we are trying to estimate the impact on rice production of subsidizing
fertilizer and are doing this by measuring the rice production of subsidized (treatment)
farmers and unsubsidized (comparison) farmers before and after the distribution of the
subsidies.
 If in year 1 the subsidized farmers are affected by drought, whereas the unsubsidized
farmers are not, then the difference-in-differences estimate will produce an invalid
estimate of the impact of subsidizing fertilizer.
 In general, any factor that affects only the treatment group, and does so at the same
time that the group receives the treatment, has the potential to invalidate or bias the
estimate of the impact of the program.
 Difference-in-differences assumes that no such factor is present.

4
3- Application
There is no particular package to perfom DD analysis. But the following steps explain the
way to do it. In this command y is the dependent for example the outcome.

 Now we need to create a variable to indicate the time when the treatment started. Let’s
assume that treatment started in 1994. In this case, years before 1994 will have a value
of 0 and after 1994 a 1.
 Create a variable to identify the group exposed to the treatment. Let’s assumed it will
be 1 for treated (=1) and 0 for not treated (=0).
 Create an interaction between time and treated. We will call this interaction ‘did’.

Estimating the DID estimator

didreg = lm(y ~ treated + time + did, data = mydata)

summary(didreg)

 With the summary we can get the result.

Call:
lm(formula = y ~ treated + time + did, data = mydata)
Residuals:
Min 1Q Median 3Q Max
-9.768e+09 -1.623e+09 1.167e+08 1.393e+09 6.807e+09
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 3.581e+08 7.382e+08 0.485 0.6292
treated 1.776e+09 1.128e+09 1.575 0.1200
time 2.289e+09 9.530e+08 2.402 0.0191 *
did -2.520e+09 1.456e+09 -1.731 0.0882 .
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 2.953e+09 on 66 degrees of freedom
Multiple R-squared: 0.08273, Adjusted R-squared: 0.04104
F-statistic: 1.984 on 3 and 66 DF, p-value: 0.1249
5
  We can realize that the impact of the program on the outcome is -2.520e+09. The
coefficient for ‘did’ is the differences-in-differences estimator. The effect is
significant at 10% with the treatment having a negative effect or impact

Conclusion
Despite its limitations, DD method remains one of the most frequently used impact evaluation
methodologies, and many examples appear in the literature.

6
References

Gertler J. P., Martinez S., Premand P., Rawlings B. L., Vermeersch C. M. J., 2007.
Impact
Evaluation in Practice. p 94-105.
Torres-Reyna O., 2015. Difference in differences using R. Princeton University, 5p