Statewide Asthma Learning

Collaborative Participation and

Asthma-Related Emergency
Department Use
Valerie S. Harder, PhD, MHS,a,b Judith S. Shaw, EdD, MPH, RN,a Charles E. McCulloch, PhD,c Lindsay Kill, MS,a
Keith J. Robinson, MD,a Michelle T. Shepard, MD, PhD,a Michael D. Cabana, MD, MPH,d,e Naomi S. Bardach, MD, MASd

BACKGROUND: Quality improvement (QI) efforts can improve guideline-recommended asthma abstract
care processes in the pediatric office setting. We sought to assess whether practice
participation in an asthma QI collaborative was associated with decreased asthma-related
emergency department (ED) visits.
METHODS: A statewide network of practices participated in a pediatric asthma QI collaborative
from 2015 to 2016. We evaluated asthma-related ED visit rates per 100 child-years for
children ages 3 to 21 years with asthma, using the state’s all-payer claims database. We used
a difference-in-differences approach, with mixed-effects negative binomial regression models
to control for practice and patient covariates. Our main analysis measured the outcome before
(2014) and after (2017) the QI collaborative at fully participating and control practices.
Additional analyses assessed (1) associations during the intervention period (2016) and (2)
associations including practices partially participating in QI collaborative activities.
In the postintervention year (2017), participating practices’ (n = 20) asthma-related
RESULTS:
ED visit rate decreased by 5.8 per 100 child-years, compared to an increase of 1.8 per
100 child-years for control practices (n = 15; difference in differences = 27.3; P = .002).
Within the intervention year (2016), we found no statistically significant differences in
asthma-related ED visit rates compared to controls (difference in differences = 24.3; P = .17).
The analysis including partially participating practices yielded similar results and inferences
to our main analysis.
CONCLUSIONS: Participationin an asthma-focused QI collaborative was associated with decreased
asthma-related ED visit rates. For those considering implementing this type of QI
collaborative, our findings indicate that it takes time to see measurable improvements in ED
visit rates. Further study is warranted regarding QI elements contributing to success for
partial participants.

WHAT’S KNOWN ON THIS SUBJECT: Participation in an asthma quality

Departments of aPediatrics and bPsychiatry, The Robert Larner, M.D. College of Medicine, University of Vermont,
Burlington, Vermont; cDepartments of Epidemiology and Biostatistics and dPediatrics, School of Medicine,
University of California San Francisco, San Francisco, California; and eAlbert Einstein College of Medicine and
Children’s Hospital at Montefiore, New York, New York
Dr Harder designed the study, conducted statistical analyses, interpreted results, and drafted the
improvement learning collaborative in primary care is associated with improved
processes of care in the office setting for children with asthma.
WHAT THIS STUDY ADDS: Participation in an asthma quality improvement learning
collaborative in primary care is associated with a substantial decrease in asthma-
related emergency department visits .1 year after the end of the collaborative.

manuscript; Ms Kill extracted and cleaned data for analysis, helped identify relevant literature for
citing, and drafted sections of the methods; Dr McCulloch supported the statistical analyses and To cite: Harder VS, Shaw JS, McCulloch CE, et al.
guided the display and interpretation of results; Drs Shepard and Robinson were part of the quality Statewide Asthma Learning Collaborative Participation
improvement implementation team and drafted sections of the introduction; Drs Shaw and Cabana and Asthma-Related Emergency Department Use.
helped interpret findings and contributed to the discussion; (Continued) Pediatrics. 2020;146(6):e20200213

Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020

PEDIATRICS Volume 146, number 6, December 2020:e20200213 ARTICLE
Asthma leads to substantial health
care use for children, accounting for
.10 million office visits and 1.6
million visits to emergency
departments (EDs) annually.1
Between 2001 and 2010, the rate of
asthma-related ED visits remained
high and unchanged.2 In 2010, ED
visits for asthma cost Medicaid
.$272 million.3 National data
indicated that children accounted for
an increasing percentage of all
asthma-related ED visits (from 36%
in 2011 to 40% in 2016).4
Improving care in pediatric primary
care may help improve asthma
management and decrease
exacerbations, thereby decreasing ED
visits. The National Asthma Education
and Prevention Program, sponsored
by the National Heart, Lung, and
Blood Institute (NHLBI), includes
evidence-based recommendations to
improve practitioner and patient
management of asthma.5
Unfortunately, significant variability
in primary care adherence to NHLBI
guidelines persists.6–8
Quality improvement (QI) in primary
care provides mechanisms to address
variation in guideline adherence and
therefore potentially decrease ED use.
Studies of asthma-focused QI in
primary care have been focused on
process measures like adherence to
NHLBI guidelines: tracking asthma
scores by using a validated tool,9
increasing use of asthma action
plans,9,10 increasing prescription of
inhaled corticosteroids,11,12 and
increasing use of spirometry.13 Few
studies of primary care, asthma-
focused QI in which researchers
examine ED use exist, and the results
of these studies are mixed. In 1 study,
researchers reported providing
education to primary care practices
and implementing clinical decision-
making tools led to a decrease in ED
visits and hospitalizations for
pediatric asthma patients.6 This study
was limited to 6 practices in 1 city,
did not use a learning collaborative
approach, and did not have a control
Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020
2 HARDER et al
group. In a more recent multistate
study, researchers using a modified QI
collaborative model reported
improvements to clinical asthma
management measures by primary
care practitioners but no significant
change in ED or urgent care use.14
This study did not report statistical
tests on ED run chart data and did not
have a control group.
The Vermont Child Health
Improvement Program (VCHIP), the
longest-running improvement
partnership nationally,15 leads
statewide QI collaboratives based on
the Model for Improvement16 for
pediatric-serving primary care
practices.17,18 A recent VCHIP QI
collaborative, focused on asthma care
and management in primary care,
revealed significant improvement in
multiple NHLBI guideline-
recommended processes (eg,
increased controller medication
prescription);19 however, it was not
investigated whether this also led to
reduced asthma-related ED use. This
presents an opportunity for a longer
follow-up study of a primary care
asthma-focused QI collaborative
examining asthma-related ED use as
the outcome. Our overall objective is
to test whether participation in an
asthma QI collaborative is associated
with a decrease in asthma-related ED
use over time, compared to controls.
METHODS
Study Setting
In 2012, VCHIP created Child Health
Advances Measured in Practice
(CHAMP), a voluntary network of
pediatric and family medicine
primary care practices working
collaboratively on QI. Practices in the
CHAMP network care for ∼60% of
children in Vermont. Each year, in
collaboration with these practices and
state partners, VCHIP selects a focus
area for the CHAMP QI learning
collaborative. After successful QI
collaboratives on improving
immunization rates17 and addressing
adolescent depression,18 from mid-
September 2015 through mid-May
2016, CHAMP practices worked
collaboratively to improve asthma
management.19 During the CHAMP
asthma learning collaborative,
VCHIP’s approach followed the Model
for Improvement,16 a QI methodology
designed to advance patient care and
outcomes through setting an aim,
measuring processes, and identifying
areas of improvement through
iterative Plan-Do-Study-Act
cycles.20,21 Full details of the asthma
QI collaborative, clinical measures,
and office systems strategies are
published,19 and a brief summary of
key elements is included below in the
QI collaborative section.
Data Sources
Administrative claims spanning
2014–2017 from Vermont’s all-payer
claims database (Vermont Health
Care Uniform Reporting and
Evaluation System) served as our
primary source of data. Medicaid,
Medicare, and most commercial
insurers in Vermont submitted
medical and pharmacy claims to this
database for this period. Vermont’s
all-payer claims database excluded
the uninsured and included an
estimated 90% of state residents
receiving health care in 2014 and
2015. This percentage decreased to
∼75% in 2016 and 2017 because of
a Supreme Court ruling,22 which
resulted in some commercial self-pay
insurance plans opting out of
submitting data. VCHIP collected data
on practice characteristics (Table 1)
from state administrative data and
national registries and compiled
these data annually as part of VCHIP’s
attribution process.23
QI Collaborative
In brief, practitioners and practice
staff in the CHAMP network
interested in participating in the QI
collaborative attended an all-day in-
person learning session in mid-
September 2015 covering topics on
office systems strategies (eg, asthma
TABLE 1 Characteristics of the Participating and Control Practices
Participating Practices
(n = 20), n (%)
Practice size
,500 patients
500–1499 patients
.1500 patients
Practice specialty
Pediatrics
Family medicine
Mixed practitioners
Organization structure
Hospital owned
Independently owned
HSA
One metropolitan HSA
All other HSAs
FQHC or RHC
FQHC or RHC
Neither FQHC nor RHC
Mixed practitioners include pediatricians and family medicine physicians. Unadjusted analyses were done by using
Fisher’s exact tests at the practice level.
assessment, control, and management
and patient education). Practices
received evidence-based resources,
such as NHLBI guidelines and
validated tools. Those that decided to
participate in the QI collaborative
subsequently attended monthly
phone calls, including didactics on
asthma topics (eg, the use of
validated asthma control tests,
implementing and documenting
asthma action plans, and coding and
billing) and knowledge sharing on
successes and challenges. During the
collaborative, practitioners worked to
set practice goals, implement
changes, and measure improvements
with their practice teams.
Participating practitioners submitted
monthly convenience samples of
patients to monitor progress on
clinical asthma measures and
received monthly feedback reports
with summaries of their performance
on the clinical measures, overall
collaborative performance, and QI
coaching notes. The main findings
from this study revealed that
practitioners exceeded their 20%
improvement goals, aligning with
NHLBI guidelines for the assessment
of asthma with a validated tool,
planned asthma visits, assessment of
tobacco exposure, provision of
Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020
PEDIATRICS Volume 146, number 6, December 2020
Control Practices
(n = 15), n (%)
.22
2 (10) 3 (20)
8 (40) 9 (60)
10 (50) 3 (20)
.87
11 (55) 7 (47)
8 (40) 7 (47)
1 (5) 1 (7)
.04
12 (60) 3 (20)
8 (40) 12 (80)
.49
8 (40) 4 (27)
12 (60) 11 (73)
.27
4 (20) 7 (47)
16 (80) 8 (53)
asthma education, and instruction on
asthma device use.19
Practice Participation
The VCHIP approach to QI is
inclusive, encouraging primary care
practices interested in the annual QI
topic to send practitioners and staff
to the all-day learning session,
without requiring a commitment to
participate in the subsequent QI
collaborative. This inclusive approach
led to the definition of 3 categories of
practices for this study on the basis of
level of participation: full
participants, partial participants, and
nonparticipants. Fully participating
practices met 3 criteria: (1) attended
the in-person learning session, (2)
submitted medical record review data
for 6 of 7 months, and (3) attended at
least 1 all-practice conference call. Of
46 primary care practices in the
CHAMP network in 2015, 20 fully
participated in the asthma QI
collaborative, including 2 practices
with practitioners working across
both sites and, later, merging. We
included 15 nonparticipating CHAMP
network practices as controls, and we
designated the remaining 11
practices as partial participants
because they attended the in-person
learning session but did not attend
monthly phone calls and submitted
P ,6 months of QI data. For the partial
participants, we did not collect
information on the extent to which
they did or did not implement asthma
QI at their practices. Therefore, we
excluded partial participants from
our main analyses but included them
in additional analyses
described below.
Patient Population
Patients ages 3 to 21 years, who met
criteria for identifiable asthma
(Supplemental Table 5)24,25 and who
were attributed to primary care
practices (see below), were included.
In 2014, the data set included 9883
patients 3 to 21 years old with
identifiable asthma, of whom we
attributed 4589 patients to CHAMP
network practices: 2376 at
participating, 1282 at control, and
931 at partially participating
practices. In 2017, the data set
included 8677 patients 3 to 21 years
old with identifiable asthma, of whom
we attributed 4186 patients to
CHAMP network practices: 2257 at
participating, 899 at control, and
1030 at partially participating
practices.
Attribution of Children to Practices
Attributing each child to a single
practice each year allowed us to
measure improvement at the
practice level over time. There were
2 steps in our attribution of children
to primary care practice, by using
a hierarchical approach based on
previous work by Christensen et al.26
First, we selected a single primary
care practitioner for each child out
of all practitioners listed on claims
within a year. Second, we linked the
child to a practice on the basis of
the chosen primary care practitioner
from step 1. We describe the full
details of our attribution of children
to practices using Vermont’s all-payer
claims database in a previous
publication.23
3
Outcome of visits with an asthma diagnosis in participating and control practices
The outcome measure was the ED the second position reflects measure (Table 2).
visit rate (the number of visits per development Delphi panel
100 child-years) for children ages 3 recommendations and exploratory Statistical Analyses
analyses that found that claims with
to 21 years with identifiable We used Fisher’s exact and x2
asthma.24,25 The measure definition is a second diagnosis of asthma often
analyses, respectively, to compare
from the Pediatric Quality Measures had a primary diagnosis with
practice and patient characteristics
Program (PQMP), created under the a related symptom (eg, fever or
between participating and control
Children’s Health Insurance Program wheezing) or a known asthma trigger
practices.
Reauthorization Act, funded through (eg, upper respiratory tract infection,
the Centers for Medicare and pneumonia, or influenza). To assess the relationship between
Medicaid Services, and overseen by asthma QI collaborative participation
the Agency for Healthcare Research and asthma-related ED use, we used
Covariates a difference-in-differences approach.
and Quality. The PQMP was
established to address gaps in For comparing participating and This approach compared the asthma-
assessing quality in pediatric care and control practices and for inclusion in related ED visit rate per 100 child-
led to the development of numerous the difference-in-differences analyses, years before (2014) and after (2017)
pediatric quality measures,27 we used the following practice the QI collaborative, at participating
including the one used in this study. characteristics (Table 1): size (,500, versus control practices. The choice
500–1499, or .1500 patients), of 2017 reflects the assumption that
The denominator and numerator specialty (pediatrics, family medicine, improvements will continue past the
definitions followed the asthma- or mixed, including pediatricians and end of the QI collaborative. We used
related ED visit rate measure family medicine physicians), a mixed-effects negative binomial
specifications developed28 and organization structure (hospital multivariable regression model, with
refined through the PQMP.24,25,27,29 owned versus not), hospital service random intercepts for patient and
Eligibility was assessed for each area (HSA) location (located in the practice to account for clustering
month of data. The criteria for the largest metropolitan HSA versus not), within patient and practice. To
measure were (1) having $3 months and federally qualified health center estimate the difference-in-differences
of consecutive enrollment in the same (FQHC) or rural health center (RHC) effect, models included variables for
insurance plan (the measurement versus neither. We also included participation, year, and the
month and the 2 months before) and covariates for patient characteristics: interaction term between
(2) evidence of claims for identifiable age categories (3–5, 6–11, 12–17, or participation and year. The P value for
asthma (see Supplemental Table 5 for 18–21 years), sex (male, female, or the interaction term was used to test
definition) during a look-back period, unknown), and insurance (Medicaid whether the change in ED use was
including the measurement month, all versus not), for comparing different between participating and
previous months in the measurement
year, and the year before the
TABLE 2 Characteristics of Patients at Participating and Control Practices Before (2014) and After
measurement year (see Supplemental
(2017) the Asthma QI Learning Collaborative
Fig 2). The total number of eligible
2014 2017
child-months in each year was
summed and divided by 1200 to Participating Control P Participating Control P
calculate the denominator in units of n (%) n (%) n (%) n (%)
100 child-years for the measurement Total study sample 2376 (100) 1282 (100) — 2257 (100) 899 (100) —
year. The numerator was a count of Age categories, y .047 ,.001
all ED visits or hospitalizations with 3–5 453 (19) 206 (16) 430 (19) 160 (18)
6–11 935 (39) 498 (39) 868 (38) 326 (36)
a first or second diagnosis of asthma
12–17 839 (35) 478 (37) 854 (38) 327 (36)
in eligible patients for the 18–21 149 (6) 100 (8) 105 (5) 86 (10)
measurement month. The inclusion of Sex .26 .32
hospitalizations is in response to Male 1193 (50) 644 (50) 1125 (50) 442 (49)
measure development work,30 which Female 1046 (44) 580 (45) 1009 (45) 419 (47)
Unknown 137 (6) 58 (5) 123 (5) 38 (4)
revealed that the measure was more
Insurance .001 ,.001
accurate when including Non-Medicaid 897 (38) 412 (32) 627 (28) 183 (20)
hospitalizations because claims are Medicaid 1479 (62) 870 (68) 1630 (72) 716 (80)
often not submitted for ED care that P values are from bivariate x2 comparing patient-level percentages within demographic categories across participant
leads to hospitalization. The inclusion and control practices within each year. —, not applicable.

Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020

4 HARDER et al
control practices. Practice (Table 1)
and patient (Table 2) covariates were
included in models as potential
confounders. Adjusted ED rates at
each time point were obtained by
using the postestimation margins
command in Stata version 15 (Stata
Corp, College Station, TX).31 This is
used to calculate a marginal (or
average) rate by averaging predicted
values derived for each observation,
assuming it was in each of the
intervention groups at each time
point (but by using its covariate
values for all the other variables to
generate the predicted values). We
tested for a difference in participating
versus nonparticipating
preintervention slopes from monthly
ED visit rate data (January 2014 to
September 2015) to ensure there
were no preintervention trend
differences. We ran the same model
over time in months, including an
interaction between participation and
time, and we found no difference (P =
.684; results not shown), thus
supporting the notion that trends in
the control practices serve as
a reasonable counterfactual for what
would have occurred to the
participating practices in the absence
of the QI learning collaborative.
Additional Analyses
We conducted two additional
analyses to test the robustness of our
findings and better understand the
potential effects of the QI
collaborative. We hypothesized that
the effects of the asthma
improvements may take some time to
translate into changes in the distal
use outcome. Thus, we repeated our
analysis using an earlier time point
(2016) for the postintervention
period to examine whether
immediate effects were apparent,
although the QI collaborative was still
active during the beginning of 2016.
In addition, we hypothesized that the
effect of the QI may be different in
partially participating practices, so we
repeated our analysis, including
Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020
PEDIATRICS Volume 146, number 6, December 2020
children with asthma from those
partial participating practices.
All analyses were conducted in Stata
version 15.1.32 The study was
approved by the institutional review
board (Committee on Human
Research in the Medical Sciences 17-
0232).
RESULTS
The 20 participating practices did not
have significantly different
characteristics compared to the 15
control practices, except there were
a larger number owned by hospitals
(Table 1). Participating practices had
slightly younger children and fewer
Medicaid patients in both 2014 and
2017 (Table 2), compared to control
practices.
In our main analysis, the asthma-
related ED visit rate per 100 child-
years for participating practices
decreased from 15.3 in 2014 to 9.5 in
2017, for an overall decrease of 25.8
(nearly 40%). Over the same period,
the asthma-related ED visit rate per
100 child-years for control practices
increased from 17.0 to 18.8, for an
overall increase of 11.8. These
findings led to a statistically
significant adjusted difference in
differences of 27.6 visits per
100 child-years associated with
participation in the collaborative
(Table 3; P = .002). Figure 1 reveals
the ED visit rates over time,
comparing participants to controls in
2014 and 2017. The dashed lines
TABLE 3 Comparison of Participating and Control Practice Mean Asthma-Related ED Visit Rates From
Baseline and Postcollaborative Years (Main Difference-in-Differences Analysis)
Main Analysis Asthma-Related ED Visits per 100 Child-Years
Control 2014
Participating 2014
Control 2017
Participating 2017
Difference in differences
Marginal rates are adjusted for patient (age, sex, and insurance) and practice (specialty, organizational structure, size,
geographic region, and FQHC or RHC) characteristics and, also, for patients clustering within practices; SEs are estimated
by using the d method; Coefficients for relative rates are in Supplemental Table 6. —, not applicable.
between our comparison years are
data excluded from our analyses,
revealing variation in the ED visit
rates during that time. For point
estimates from the full negative
binomial model, including
confounders, see Supplemental
Table 6.
Results from additional analyses
comparing the change in ED visit rate
(1) from 2014 to 2016 and (2)
including partial participants are in
Table 4. Comparing the slight
decrease of 20.8 ED visits per
100 child-years for participating
practices to the increase of 13.5 ED
visits per 100 child-years for control
practices, the overall adjusted
difference in differences of 24.3 visits
per 100 child-years was not
statistically significant (P = .17). The
analysis, including the partial with
fully participating practices, yielded
similar results and inferences as our
main analysis, with an adjusted
difference in differences of 27.3 visits
per 100 child-years (P = .003).
DISCUSSION
Our study of a statewide asthma QI
collaborative was focused on a use
measure, asthma-related ED visit rate,
building on our previous findings that
the QI collaborative led to
improvements in care processes.19
We found that participation in the QI
collaborative was associated with
a substantial decrease of nearly 40%
in asthma-related ED visit rates
P 95% Confidence
Interval
Lower Upper
16.97 — 11.99 21.95
15.29 — 11.62 18.96
18.80 — 12.89 24.71
9.50 — 7.07 11.93
27.62 .002 213.45 21.78
5
 controls. This is a potential benefit
not only to public and commercial
payers but also to children and
parents, by avoiding missed school
and work and, potentially, achieving
an improved quality of life.33

We found that there was a time lag to

the association between participation
in the QI collaborative and improved
asthma-related ED rates. Although
there was an improvement by 2016, it
was not statistically significant. The
improvement by 2017 was nearly
twice that by 2016 and represented
close to a 40% difference from the
control group rate in 2014. Although
the confidence intervals for these
effect estimates overlap, there are
several possible explanations for this
FIGURE 1
Quarterly number of ED visits per 100 child-years for participants versus controls from 2014 to 2017, lag worth mentioning. Over the
highlighting the QI collaborative (mid-September 2015 to mid-May 2016). Solid lines indicate the data course of a collaborative, process
analyzed and presented in the main analysis comparing the calendar year 2017 vs 2014. measures, like asthma control, take
time to improve, and seeing
.1 year after the end of the Our findings indicate that successful improvement on downstream clinical
collaborative. The large number of QI collaboratives, with documented outcomes, like ED visits, may take
nonparticipating practices as improvements in NHLBI guideline additional time. Also, asthma
a control group strengthens the rigor adherent process improvements, are exacerbations are seasonal, with ED
and validity of our findings by using associated with a meaningful visits occurring often in winter
a difference-in-differences approach. decrease in ED visits compared to months. The collaborative ended
early in 2016, so calendar year data
from 2016 included 3 winter months
TABLE 4 Comparison of Participating and Control Practices Change in Mean ED Asthma Visit Rate when the QI collaborative was still
per 100 Child-Years (Additional Difference-in-Differences Analyses) for (1) Comparison of underway, capturing the higher rate
Baseline (2014) to 2016 (Includes the Last 3 Months of the Learning Collaborative) and (2) of asthma ED visits, before
Including Partial With Full Participants Compared to Control Practices participants had finished
Additional Analyses Asthma-Related ED Visits per 100 Child- P 95% improvements. This may explain why
Years Confidence the effect of the improvements was
Interval
not apparent until the next year.
Lower Upper Finally, in previous asthma clinical
2014–2016 trials revealing that primary care
Control 2014 17.93 — 13.83 22.04 interventions can deliver changes in
Participating 2014 14.11 — 11.57 16.65 ED outcomes in a shorter 1-year time
Control 2016 21.46 — 15.94 26.98
Participating 2016 13.33 — 10.96 15.70 frame,34 researchers specifically
Difference in differences 24.31 .17 210.49 1.88 recruited patients at “high risk” for
Including partial participants asthma exacerbations (eg, enrolled
(2014–2017) only patients with a recent asthma-
Control 2014 16.43 — 11.01 21.86
related ED visit). This focus on higher
Participating and partial 2014 15.90 — 12.07 19.72
Control 2017 18.15 — 11.79 24.52 risk patients can lead to more rapid
Participating and partial 2017 10.34 — 7.84 12.84 changes in ED use. In our study, we
Difference in differences 27.27 .003 212.97 21.58 included all children with asthma
Marginal rates are adjusted for patient (age, sex, and insurance) and practice (specialty, organizational structure, size, (and did not use severity as eligibility
geographic region, and FQHC or RHC status). Because all participants are included in the marginal effect calculation, the criteria), which may also explain the
control estimates for the same year across models are not the same (eg, control 2014 estimates across models).
Analyses are also adjusted for patients clustering within practices; SEs are estimated by using the d method. —, not
longer observation period before
applicable. detecting changes. This additional

Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020

6 HARDER et al
analysis may be useful to payers (or
primary care practices participating
in value-based accountable care
organizations), to forecast potential
decreased ED use through a primary
care asthma QI collaborative.
We found that including patients
from partially participating practices
did not change our main results or
inferences. One potential implication
for those implementing a QI
collaborative is that partial
participation can still be associated
with improvements in ED use. We
would add the caveat that the
partially participating practices in
this study previously engaged in
VCHIP QI collaboratives and,
therefore, were not naïve to QI
implementation. Full participation
may be more important in a less
experienced group of practices. In
a future study, researchers using an
implementation science approach
(tracking types and extent of
participation, previous QI experience,
and asthma-related process and
outcome measures) may more
accurately pinpoint which
components of the collaborative
implemented in which practice
settings most directly affect
improvement in asthma health
outcomes.
There were several limitations to our
current study. Although we
demonstrate difference in differences
in ED visit rates from 2014 to 2017,
during the intervening years, there is
variation in the ED visit rates
between the participating and control
practices. It is possible that
unmeasured confounders, such as
other concurrent QI or public health
efforts that differed across practices,
or other unmeasured differences
Dr Bardach supported the analytic design, helped interpret findings, and contributed to all sections of the manuscript draft; and all authors reviewed and revised
the manuscript, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.
DOI: https://doi.org/10.1542/peds.2020-0213
Accepted for publication Sep 3, 2020
Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020
PEDIATRICS Volume 146, number 6, December 2020
between participating and control
practices, such as practice culture and
management structure, contributed
to the association between
participation and decreased ED use
and may limit the difference-in-
difference approach. However, we
accounted for secular trends and
state-level efforts in our analysis.
Also, the Supreme Court ruling22
resulting in a loss of some self-
insured patients could have biased
our results, if there was differential
loss between participating and
control practices. This may be why
the percent of non-Medicaid patients
decreased in both groups over time
but the difference in the proportion of
non-Medicaid patients between
participating and control practices
was stable over time. This study was
also limited to children attributed
annually to practices in a QI network,
limiting the generalizability of our
study findings to practices with
previous exposure to QI. Finally,
children were reattributed to
practices each year, and we did not
limit our analyses to only children
attributed to the same practice over
time. This attribution method could
have biased our results to find
a positive effect, if children with more
severe asthma differentially moved
away from participating practices to
control practices over time, but
there are no data to suggest this
differential movement of patients
with asthma.
CONCLUSIONS
Participation in a 9-month QI
collaborative to improve primary care
systems and supports for children
with asthma led to a substantial
decrease in asthma-related ED visits,
with improvements continuing over
time, reaching statistical significance
a year after completion of the QI
collaborative. Our study suggests that
implementation of an asthma-related
QI collaborative may be associated
with reductions in asthma-related ED
use, although the return on
investments may take .1 year to
realize after completion. Our network
practices that participated partially in
the QI collaborative activities still
showed improvement in asthma
outcomes, implying that practices
benefited from partial participation.
Future studies may better delineate
which collaborative elements result in
greatest benefit.
ACKNOWLEDGMENTS
We thank Robert Thombley, BS, and
Victoria Hart, PhD, for their statistical
programming to measure ED visit
rates. The analyses, conclusions, and
recommendations from these data are
solely those of the authors and are
not necessarily those of the Green
Mountain Care Board.
ABBREVIATIONS
CHAMP: Child Health Advances
Measured in Practice
ED: emergency department
FQHC: federally qualified health
center
HSA: hospital service area
NHLBI: National Heart, Lung, and
Blood Institute
PQMP: Pediatric Quality Measures
Program
QI: quality improvement
RHC: rural health center
VCHIP: Vermont Child Health Im-
provement Program
7
Address correspondence to Valerie S. Harder, PhD, MHS, Department of Pediatrics and Vermont Child Health Improvement Program, The Robert Larner, M.D. College
of Medicine, University of Vermont, 1 S Prospect St, Burlington, VT 05401. E-mail: vharder@uvm.edu
PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).
Copyright © 2020 by the American Academy of Pediatrics
FINANCIAL DISCLOSURE: Dr Cabana is a member of the United States Preventive Services Task Force. This article does not necessarily reflect the views of the
United States Preventive Services Task Force; the other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by the Agency for Healthcare Research and Quality (U18HS025297).
POTENTIAL CONFLICT OF INTEREST: Dr Cabana is a member of the United States Preventive Services Task Force. This article does not necessarily reflect the views of
the United States Preventive Services Task Force; the other authors have indicated they have no potential conflicts of interest to disclose.

REFERENCES
1. Centers for Disease Control and
Prevention. 2016 National Health
Interview Survey (NHIS) data. Available
at: https://www.cdc.gov/asthma/nhis/
2016/data.htm. Accessed January 14,
2020
2. Nath JB, Hsia RY. Children’s emergency
department use for asthma, 2001–2010.
Acad Pediatr. 2015;15(2):225–230
3. Pearson WS, Goates SA, Harrykissoon
SD, Miller SA. State-based Medicaid
costs for pediatric asthma emergency
department visits. Prev Chronic Dis.
2014;11:E108
4. Agency for Healthcare Research and
Quality. HCUPnet, Healthcare Cost
and Utilization Project. Rockville, MD:
Agency for Healthcare Research and
Quality; 2019. Available at: https://
hcupnet.ahrq.gov/. Accessed December
10, 2019
5. National Heart, Lung, and Blood
Institute. Expert Panel Report 3:
Guidelines for the Diagnosis and
Management of Asthma. Bethesda, MD:
National Heart, Lung, and Blood
Institute, National Institutes of Health,
US Department of Health and Human
Services; 2007
6. Cloutier MM, Hall CB, Wakefield DB,
Bailit H. Use of asthma guidelines by
primary care providers to reduce
hospitalizations and emergency
department visits in poor, minority,
urban children. J Pediatr. 2005;146(5):
591–597
7. Shapiro A, Gracy D, Quinones W,
Applebaum J, Sarmiento A. Putting
guidelines into practice: improving
documentation of pediatric asthma
management using a decision-making
tool. Arch Pediatr Adolesc Med. 2011;
165(5):412–418
8. Yawn BP, Yawn RA. Measuring asthma
quality in primary care: can we develop
better measures? Respir Med. 2006;
100(1):26–33
9. Vernacchio L, Francis ME, Epstein DM,
et al. Effectiveness of an asthma quality
improvement program designed for
maintenance of certification. Pediatrics.
2014;134(1). Available at: www.
pediatrics.org/cgi/content/full/134/1/
e242
10. Mangione-Smith R, Schonlau M, Chan
KS, et al. Measuring the effectiveness of
a collaborative for quality improvement
in pediatric asthma care: does
implementing the chronic care model
improve processes and outcomes of
care? Ambul Pediatr. 2005;5(2):75–82
11. Hogan AH, Rastogi D, Rinke ML. A quality
improvement intervention to improve
inpatient pediatric asthma controller
accuracy. Hosp Pediatr. 2018;8(3):
127–134
12. Homer CJ, Forbes P, Horvitz L, Peterson
LE, Wypij D, Heinrich P. Impact of
a quality improvement program on
care and outcomes for children with
asthma. Arch Pediatr Adolesc Med.
2005;159(5):464–469
13. Mangione-Smith R, Zhou C, Corwin MJ,
Taylor JA, Rice F, Stout JW. Effectiveness
of the Spirometry 360 quality
improvement program for improving
asthma care: a cluster randomized
trial. Acad Pediatr. 2017;17(8):855–862
14. Dolins JC, Powell J, Wise E, et al.
Improving asthma care by building
statewide quality improvement
infrastructure. Pediatrics. 2017;140(2):
e20161612
15. Shaw JS, Norlin C, Gillespie RJ,
Weissman M, McGrath J. The national
improvement partnership network:
state-based partnerships that improve
primary care quality. Acad Pediatr.
2013;13(suppl 6):S84–S94
16. Institute for Healthcare Improvement.
The Breakthrough Series: IHI’s
Collaborative Model for Achieving
Breakthrough Improvement. Boston,
MA: Institute for Healthcare
Improvement; 2003
17. Harder VS, Barry SE, Ahrens B, Davis
WS, Shaw JS. Quality improvement to
immunization coverage in primary care
measured in medical record and
population-based registry data. Acad
Pediatr. 2018;18(4):437–444
18. Harder VS, Barry SE, French S, Consigli
AB, Frankowski BL. Improving
adolescent depression screening in
pediatric primary care. Acad Pediatr.
2019;19(8):925–933
19. Weinberger SJ, Cowan KJ, Robinson KJ,
et al. A primary care learning
collaborative to improve office systems
and clinical management of pediatric
asthma [published online ahead of
print December 14, 2019]. J Asthma.
doi:10.1080/02770903.2019.1702199
20. Langley GL, Moen RD, Nolan KM, Nolan
TW, Norman CL, Provost LP. The
Improvement Guide: A Practical
Approach to Enhancing Organizational
Performance, 2nd ed. San Francisco,
CA: Jossey-Bass; 2009
21. Courtlandt CD, Noonan L, Feld LG. Model
for improvement - part 1: a framework
for health care quality. Pediatr Clin
North Am. 2009;56(4):757–778
22. . Gobeille v Liberty Mutual Insurance
Company, 136 US 936 (2016)
23. Wasserman RC, Varni SE, Hollander MC,
Harder VS. Change in site of children’s
primary care: a longitudinal population-

Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020

8 HARDER et al
 based analysis. Ann Fam Med. 2019; Program. Acad Pediatr. 2014;14(suppl 31. Norton EC, Miller MM, Kleinman LC.
17(5):390–395 5):S19–S26 Computing adjusted risk ratios and risk
24. Bardach NS, Neel C, Kleinman LC, et al. differences in Stata. Stata J. 2013;13(3):
28. Collaboration for the Advancement of
Depression, anxiety, and emergency 492–509
Pediatric Quality Measures. Collaboration
department use for asthma. Pediatrics. for the Advancement of Pediatric Quality 32. . StataCorp LLC. Stata Statistical
2019;144(4):e20190856 Measures. Available at: www.capquam. Software: Release 15 [computer
25. Implement for Child Health. Documents. org. Accessed January 14, 2020 program]. College Station, TX:
Available at: https://chipper.ucsf.edu/ StataCorp LLC; 2017
29. Agency for Healthcare Research and
studies/implement/documents. Quality. About the Pediatric Quality 33. Luskin AT, Chipps BE, Rasouliyan L,
Accessed January 14, 2020 Measures Program. Available at: www. Miller DP, Haselkorn T, Dorenbaum A.
26. Christensen AL, Zickafoose JS, Natzke B, ahrq.gov/pqmp/about/index.html. Impact of asthma exacerbations and
McMorrow S, Ireys HT. Associations Accessed January 14, 2020 asthma triggers on asthma-related
between practice-reported medical quality of life in patients with severe
30. Egorova N, Gesten F, Anarella J.
homeness and health care utilization or difficult-to-treat asthma. J Allergy
Measuring undesirable utilization
among publicly insured children. Acad Clin Immunol Pract. 2014;2(5):
outcomes when assessing the quality of
Pediatr. 2015;15(3):267–274 544–552–2
care for children with asthma: findings
27. Mistry KB, Chesley F, LLanos K, and considerations. In: Proceedings 34. Cabana MD, Slish KK, Evans D, et al.
Dougherty D. Advancing children’s from the AcademyHealth Annual Impact of physician asthma care
health care and outcomes through Research Meeting; June 8–10, 2014; San education on patient outcomes.
the Pediatric Quality Measures Diego, CA Pediatrics. 2006;117(6):2149–2157

Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020

PEDIATRICS Volume 146, number 6, December 2020 9
 Statewide Asthma Learning Collaborative Participation and Asthma-Related
Emergency Department Use
Valerie S. Harder, Judith S. Shaw, Charles E. McCulloch, Lindsay Kill, Keith J.
Robinson, Michelle T. Shepard, Michael D. Cabana and Naomi S. Bardach
Pediatrics 2020;146;
DOI: 10.1542/peds.2020-0213 originally published online November 23, 2020;

Updated Information & including high resolution figures, can be found at:
Services http://pediatrics.aappublications.org/content/146/6/e20200213
References This article cites 22 articles, 5 of which you can access for free at:
http://pediatrics.aappublications.org/content/146/6/e20200213#BIBL
Subspecialty Collections This article, along with others on similar topics, appears in the
following collection(s):
Administration/Practice Management
http://www.aappublications.org/cgi/collection/administration:practic
e_management_sub
Practice-Based Learning & Development
http://www.aappublications.org/cgi/collection/practice-based_learnin
g_-_development_sub
Pulmonology
http://www.aappublications.org/cgi/collection/pulmonology_sub
Asthma
http://www.aappublications.org/cgi/collection/asthma_subtopic
Permissions & Licensing Information about reproducing this article in parts (figures, tables) or
in its entirety can be found online at:
http://www.aappublications.org/site/misc/Permissions.xhtml
Reprints Information about ordering reprints can be found online:
http://www.aappublications.org/site/misc/reprints.xhtml

Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020

Statewide Asthma Learning Collaborative Participation and Asthma-Related
Emergency Department Use
Valerie S. Harder, Judith S. Shaw, Charles E. McCulloch, Lindsay Kill, Keith J.
Robinson, Michelle T. Shepard, Michael D. Cabana and Naomi S. Bardach
Pediatrics 2020;146;
DOI: 10.1542/peds.2020-0213 originally published online November 23, 2020;

The online version of this article, along with updated information and services, is
located on the World Wide Web at:
http://pediatrics.aappublications.org/content/146/6/e20200213

Data Supplement at:

http://pediatrics.aappublications.org/content/suppl/2020/11/18/peds.2020-0213.DCSupplemental

Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it
has been published continuously since 1948. Pediatrics is owned, published, and trademarked by
the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, 60143. Copyright © 2020
by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.

Downloaded from www.aappublications.org/news at Indonesia:AAP Sponsored on December 4, 2020