Estimating the Effect of School Closures on

Societal Transmission of Covid-19:

Evidence from lower secondary school closures
in Sweden

Sebastian Shaqiri Johansson

Ph.D. Candidate

BI Norwegian Business School

Department of Economics

10th October 2022

Contents
Introduction

▶ School closures (SC) was a very common strategy to

mitigate virus transmission
➢ Over 80% of students worldwide had distance teaching
during Covid-19 (Castex et al., 2020)
▶ Purpose of school closures: Reduce virus transmission
▶ Not only positive effect of school closures:
➢ Worsened school outcomes (Andrew et al., 2020; Engzell et
al., 2021)
➢ Negative impact on physical and mental well-being (Takaku
and Yoyaman 2021; Yamamura and Tsustsui 2021)
➢ Negative effect on mothers labor force participation (Collins
et al., 2021)
➢ Increased economic and social inequality (Andrew et al.,
2020; Engzell et al., 2021; Hanushek and Woessmann 2020)
Research Question

▶ Did closing lower secondary schools have any effect on the

municipal transmission of Covid-19?
Overview of the Literature
▶ Studies evaluating the effect of SC are often divided into
two subgroups:
1. Simulation studies: Use epidemiological models (variations
of SIR-models) to try and predict potential effects
2. Empirical studies: Tries to identify the effect on school
closures on societal transmission
▶ Various problems arise in both these categories:
➢ Simulation studies provides little evidence of any causal
effect
➢ Many empirical papers lack reliable identification strategy:
1. Do not have sufficient control variables
2. Do not deal with endogeneity of school closures (reverse
causality)
▶ Empirical evidence suggests highly heterogenous results
when evaluating the effect of school closures (or
re-openings)
➢ Vlachos et al., (2021) show a small, but significant, effect on
societal transmission when comparing infection rates
between parents, teachers, and their partners in lower and
upper secondary schools in Sweden
Motivation

▶ This paper extends the result in Vlachos et al., (2021) by

looking at the direct effect on societal transmission and not
extrapolating from within municipality family differences.
▶ There exists huge gaps in the literature that this paper
contributes to:
➢ Empirical evidence: Increase the knowledge on the effect of
SC
➢ Metholodogical: Provide a framework on how to deal with
endogeneity, confounding and collinearity with other
Non-pharmaceutical interventions (NPIs)
➢ Policy relevant:Evaluating the effect of SC is important, not
only for guidelinse for future pandemics, but also when
contrasting the other potential negative (or positive) effects
that also comes with SC
Data
▶ I have data on when lower secondary schools (högstadiet)
closed in Sweden from week 45 of 2020 to week 24 of 2021
for 40 municipalities
▶ The outcome variable is the number of new Covid-19 cases
on municipality level
▶ School-level data is first aggregated to the municipal level,
and then merged with municipal covariates that affect both
the number of Covid-19 cases and the likelihood of closing
schools
▶ The treatment variable is derived from the intensity of
school closures (ISC)
➢ ISC is derived by taking the average weekly intensity per
municipality
➢ If municipal ISC is equal to 3 all grades in that municipality are home
that week; if ISC is equal to 2, then one grade is school and two at home;
if ISC is equal to 1, then two grades are in school and one at home; and
zero if no grade is home

➢ Treatment is equal to 1 when ISC is equal to 3, otherwise

zero
➢ This is the baseline model
➢ By defining treatment from ISC, I can vary the treatment level and
compare different scenarios of depending on how many grades were home
Descriptive statistics
Empirical Approach

▶ Question: How do we identify causal effect?

➢ Answer: Utilize variation of school closures across
municipalities – some closed, some did not
➢ Problem: Can we really compare municipalities – are they
similar enough?
➢ Solution: Synthetic control method (Abadie and
Gardeazabal 2003; Abadie et al., 2010)
Synthetic Control Method (with some math)

▶ Let’s first present the general framework by Abadie et al.,

(2010) in terms of potential outcomes, and then, for
purposes of this presentation, talk about this method in
relation to the standard DiD framework.
▶ Suppose that there is only one municipality that closed all
their schools, and denote their number of Covid-19 cases by
Y1tI , and the outcome for the rest as YjtN (j = 2, ..., n), then
the treatment effect is the difference between the outcomes:

τb1t = Y1tI − Yb1tN (1)

where
➢ τb1t is estimated treatment effect
➢ Y1tI is the observed outcome at time t
➢ Y1N is the estimated counterfactual outcome
How is counterfactual estimated?
▶ Our main problem is how to estimate the counterfactual
scenario
➢ In simple Difference-in-Difference framework, the
counterfactual is estimated using the control group by
”direct comparison” before and after treatment.
➢ The problem here is that we cannot compare municipalities
this way, parallel trend assumption not likely to hold =⇒
Biased estimates
➢ Synthetic control let us deal with this problem of parallel
trends
➢ Easily explained, we SCM allows us to construct an
artificial control group from the untreated units that
matches outcomes before treatment
➢ In other words, SCM makes use of some fancy math that
constructs a weighted average of the control groups such
that Y1tI = YbjtN , before treatment, so any dispersion after
treatment should be attributed to treatment itself
➢ If this process works as it should, then the SCM estimates
will be unbiased
SCM in my paper

▶ The treated group is the municipality(ies) that closed all

their schools (ISC=3)
▶ The control group is the municipalities that did not close
their schools (ISC=0)
▶ The estimated counterfactual is the weighted average of the
control group
n
X
YbjtN = ωj YjN .
j=2
Illustration of when SCM is useful

Before SCM After SCM

# Covid cases Treatment # Covid cases Treatment

Week Week
Generalizing SCM

▶ Framework presented above is just when one municipality

closed their schools — I have more than one treated unit.
▶ Fairly intuitive generalization: Suppose we have
g ∈ {1, ..., G} treated units and, j ∈ {G + 1, ..., J} number
of control units, the treatment effect is just the average of
all. That is, the average treatment effect is
G
1 X
τ̄ = (τ̄T0 +1 + · · · τ̄T ) = τg,T0 +1 + · · · + τbg,T ).
(b (2)
G
g=1

▶ This for staggered (over time) treatment and heterogenous

(over municipality) treatment effect.
Generalized SCM vs TWFE

▶ Why not use Two-way Fixed Effects (TWFE)?

➢ Simple answer: Staggered treatment in standard TWFE
includes already treated units in periods after some of the
units has been treated =⇒ parallel trend assumption will
fail
➢ One could use methods from Callaway and Sant’Anna
(2020) or Sun and Abraham (2020), although we still
cannot use these because parallel trend is not likely to hold
anyways
➢ Synthetic DiD could work (see Arkhangelsky et. al., (2021))
▶ These problems are easily solved in SCM
Another win for SCM: Solving the endogeneity problem

▶ Endogeneity of SC stems from the fact that schools tend to

close if the transmission is high, while we are interested in
the effect of SC on transmission (reverse causality)
▶ SCM solves this: If the synthetic control group is valid, the
number of weekly Covid-19 cases for the average treated
unit and synthetic control will be identical at the treatment
date
➢ There is no variation in the number of cases between
treated and control, the only variation is school closures
which solve the problem!
Results
Results cont.

▶ Synthetic control group is valid since the average control

unit and the control group are virtually identical before
treatment
▶ Effect of school closures are seen by the deviation between
the lines after treatment (between 15-25 weekly new cases
during the examined period)
Inference

▶ The estimated difference between treated and untreated is

not sufficient to claim statistical significance.
▶ To test if the result is significant, Abadie et al., 2010
suggest using placebo tests.
➢ Placebo tests = assign treatment onto untreated units
➢ If the untreated units experienced a ”treatment effect” that
did not occur, our estimated treatment effect is not
statistically significant.
➢ The estimated effect is statistically significant (see paper for
details)
Robustness

▶ Robustness is assessed by:

➢ Time placebos: Artificially shift treatment date backward
(weeks 3 through 7).
➢ Restrict donor pool: Change the available control
municipalities
▶ Our result is robust to these tests (see paper for more info).
School Closure Heterogeneity
▶ By varying the intensity of SC we can assign treatment
based on different scenarios:
➢ Baseline: Treatment = if all grades have distance teaching
➢ Medium: Treatment = if two grades have distance teaching
➢ Low: Treatment = if one grade have distance teaching

Table: Heterogeneous effects in intensity of school closures

Dependent variable
Basline Medium Low
Time 0.930∗∗∗ 0.997∗∗∗ 1.009∗∗∗
(0.229) (0.243) (0.247)

Treatment 1.036 0.981 1.67

(3.966) (4.657) (7.857)

Time × Treatment -21.417∗∗∗ -13.332 -18.347

(5.926) (12.248) (16.252)
Observations 1302 18022 18022
Note: Standard errors in parentheses.*, ** and *** denotes significance level at 10, 5, and
1 percent, respectively. The dependent variable is new weekly Covid-19 cases. The baseline
model estimates the treatment effect when school intensity is equal to three. The medium
model and low model estimate the treatment effect when school intensity is equal to two and
one, respectively. The control variables are the seen in ??.
Concluding remarks

▶ This paper provides two central empirical results:

1. Closing lower secondary schools in Sweden did reduce
municipal transmission with 15-25 weekly new cases
2. However, only having all students at home simultaneously
yielded enough contagion management to reduce
transmission
▶ This paper also provides methodological guidance:
1. SCM estimates are robust even in small samples
2. SCM deals well with the endogeneity of SC