The effects of access to health

insurance: Evidence from a regression

discontinuity design in Peru
Bernal, Carpio & Klein (2017)

A presentation by Georgina Gonzales & Ralph Manginsay

on Regression Discontinuity Design
for DE292/Econ298
21 March 2024
 Background
Objective evaluate the effects of access to health insurance on health care utilization, out-of-
pocket expenditures, and potentially on health outcomes among low-income populations
Context Lima Province of Peru, 2011
Intervention provision of health insurance coverage through Seguro Integral de Salud (SIS) -
comprehensive health care services, straightforward enrollment, mixed financing
Participants 4,161 individuals that
• belong to a household in which no member is formally employed
• are without other forms of health insurance, and
• whose Index of Household Welfare (IFH) scores are near the eligibility threshold for SIS
– i.e., just above (control group) and just below (treatment group) the cutoff point set
for SIS coverage
o Specifically, IFH index is below or equal to a value of 55, provided that both water
and electricity expenditures do not exceed 20 and 25 Soles, respectively
Evaluation sharp regression discontinuity design
method
 Theory of Change and Indicators
The study uses cross-sectional data from the ENAHO household survey for the year 2011, representative at the level of each of the 24 departments
in Peru. It provides component information needed to recompute the IFH index and on health care utilization, its financing and out-of-pocket
expenditures. Data are collected using face-to-face interviews.

Assumption: Individuals cannot manipulate their IFH index score (eligibility criterion). Study assumes eligible individuals are covered.

Access to SIS as an Mechanisms of change Intermediate outcomes Long-term impacts

intervention

•Provision of SIS coverage to
low-income individuals and
households in Peru – Index
of Household Welfare (IFH
Index)
•Increased health care
utilization – probability of
visits to doctors, receipt of
medicines, medical
analysis, hospitalization
and surgery
•Increase or reduction of
out-of-pocket expenditures
– amount spent by type of
health care
service, likelihood of
incurring catastrophic
health
expenditures, expenditure
patterns (e.g., across
income groups)
•Changes in behavior and
health outcomes
•Higher demand for •Health equity
healthcare services from •Broader economic benefits
increased awareness of (e.g., productivity, reduced
health needs and long-term health care costs
knowledge about available due to prevention and early
health care options treatment)
•Supply-side response to
address limitations amid
growing demand
•Financial protection and
stability, increased
disposable income from
reduced out-of-pocket
expenditure
 IE Method – Sharp Regression Discontinuity Design
Key Features Description
Running variable • Index of Household Welfare (IFH) – composite score based on household
characteristics related to socio-economic status
Cutoff point • Eligibility threshold – individuals with IFH scores below the threshold are
Identification of
Selection of deemed eligible for SIS, whereas those above the threshold are ineligible
the running
localized sample • Threshold scores vary across different regions or clusters within Peru*
variable and cutoff
Identification • Assumption that individuals just above and below the cutoff are comparable
strategy in all respects except for their eligibility for SIS
• Attribute discontinuities in the outcome variables at the threshold to the
effect of SIS coverage
Treatment group • Households or individuals with IFH scores slightly below the eligibility
threshold for SIS, thus qualifying for health insurance coverage under the
Estimation of Fitting local program
treatment effects regression models • Expected to experience the potential benefits of SIS, such as reduced out-of-
pocket health care expenditures and increased access to health care services
Control group • Households or individuals with IFH scores slightly above the eligibility
threshold, making them ineligible for SIS coverage
• Serves as a comparison to estimate what outcomes would have occurred in
the absence of SIS coverage
Addressing Interpretation and Outcome • Health care utilization (e.g., doctor visits, hospitalizations, preventive care)
potential policy variables • Out-of-pocket health care expenditures
limitations implications • Health outcomes (e.g., incidence of illnesses, chronic conditions)*
Analytical • Local linear regression models to estimate the average treatment effect of SIS
approach at the cutoff
• Robustness checks, sensitivity analyses to validate the results and ensure
reliability of the estimates
 Addendum – Linear Regression Model

• For data at each side of the cut-off

• Local linear regression – ordinary least squares which involves
calculating the best-fitting line that minimizes the squared residuals for
each group
o x – running variable
o y – outcome variable
• Results
o intercepts – difference just below and above the threshold reveals the treatment
effect
o slopes - while does not directly measure treatment effect, could suggest that
the treatment not only affects the level of outcome but also alters how the
outcome variable responds to changes in the running variable
 Addendum – Robustness Checks & Sensitivity Analyses
Potential Limitation
Manipulation of the running
variable
Linearity assumption
Comparability of groups
Discontinuities at non-
threshold points
Influence of other programs
Awareness and utilization of
coverage
External validity
Impact on
objectively measured health
Test
Tested for potential manipulation of the IFH index to
qualify for SIS using the McCrary test
Compared results from a localized sample near the
eligibility threshold to those from the full sample
Checked for jumps in covariates (e.g., gender, age,
household income) at the eligibility threshold to
confirm the similarity of individuals on either side
Investigated the presence of discontinuities at points
other than the eligibility threshold to assess the
assumption of a single discontinuity
Examined the potential influence of other programs
like Juntos and various food aid programs on the
study's findings
Considered the possibility that not all eligible
individuals are aware of their coverage, potentially
underestimating the actual effects of SIS
- -
- -
Finding
Found no evidence of discontinuity at the eligibility
threshold, supporting the validity of the RDD
assumptions
Observed that the effects remained qualitatively and
quantitatively similar, indicating that the findings are
robust to sample size variations
Found no evidence of discontinuities, suggesting that the
treatment and control groups are comparable
Conducted separate RDD analyses using hypothetical
thresholds and found no significant effects, reinforcing
the premise of a single relevant discontinuity
Concluded that these programs likely did not affect the
study's results due to their different targeting
mechanisms and the geographic focus of the study
The effects estimated may be interpreted as intent-to-
treat effects or lower bounds, indicating that the true
impact of SIS could be even greater for those aware of
their coverage
 Findings – Increase in Utilization & Expenditure
Utilization Expenditure

+9 ppt visiting a doctor +US$102 average annual total

+15 ppt receiving medicines +55% medicines
+5 ppt undergoing medical analysis +41% hospital care or surgery
+8 ppt hospitalization or surgery +12ppt likelihood of catastrophic expenditures

Overall, the study suggests that while SIS coverage in Peru has led to increased health care utilization, indicating improved access
to health services, it has also resulted in higher out-of-pocket expenditures for beneficiaries.
 Cost, Effectiveness and Policy Implications
Increased access Revealing
Monetary cost of Willingness to pay
to healthcare preference for Hypothetically
enrolling is zero, for services that
leads to increased medical care, better health
so that eligibility = may even be in
awareness about individuals spend outcomes
coverage short supply
health problems out-of-pocket

• Study does not directly comment on the cost-effectiveness of the SIS program but does find the program potentially beneficial in
achieving certain desirable health/health system outcomes
• Initial cost vs. subsequent costs
• Paradoxical outcome highlights the complexities of implementing public health insurance in settings with significant supply -side
limitations and suggests a need for further policy adjustments
o Increased health care utilization necessitates expansion in the range and depth of services covered by SIS
o Addressing supply-side limitations is crucial in addressing gap in financial protection (increased out-of-pocket expenditure
and catastrophic expenditure) and ensuring better health outcomes
o There may also be a need to review and adjust copayment structures, coverage limits, and exclusion lists to reduce
the financial burden on beneficiaries
o Increasing awareness about SIS eligibility, benefits, and access, as well as simplifying enrollment, can further maximize
program impact