765977

research-article2018
AJCXXX10.1177/1062860618765977American Journal of Medical QualityHerrin et al

Article
American Journal of Medical Quality

HCAHPS Scores and Community

2018, Vol. 33(5) 461­–471
© The Author(s) 2018
Article reuse guidelines:
Factors sagepub.com/journals-permissions
DOI: 10.1177/1062860618765977
https://doi.org/10.1177/1062860618765977
ajmq.sagepub.com

Jeph Herrin, PhD1,2, Kathleen G. Mockaitis, MPH2,

and Stephen Hines, PhD3

Abstract
The objective was to examine associations between patient experience, as measured by Hospital Consumer Assessment
of Healthcare Providers and Systems (HCAHPS) scores, and the sociodemographic, cultural, and access-to-care
factors of the surrounding community. Using an empty model, this study found that 27% of the variation in HCAHPS
scores was at the level of the county in which the hospital was located. The county factors examined explained about
half of this county-level variation. Among community factors most strongly associated with increased HCAHPS scores
were higher numbers of primary care physicians per capita, fewer specialists per capita, and smaller percentages of
African Americans and Hispanics in the county. A number of community factors beyond the hospitalization experience
may influence HCAHPS scores. As HCAHPS gains importance as a measure of quality of care and is linked to higher
payments over time, it will be essential to understand the possible effects of community factors.

Keywords
HCAHPS, quality, community, Medicare/Medicaid

In use since 2006, the Hospital Consumer Assessment of
Healthcare Providers and Systems (HCAHPS) is a stan-
dardized survey instrument and data-collection method-
ology for measuring patients’ experiences at US
hospitals.1,2 The survey asks about multiple aspects of
hospital experience: communication with doctors, com-
munication with nurses, responsiveness of hospital staff,
pain management, communication about medicines, dis-
charge information, hospital cleanliness, noise level,
overall rating of hospital, willingness to recommend hos-
pital, and transition to post-hospital care. Results of these
measures are available on the Hospital Compare website
(www.hospitalcompare.hhs.gov). The goals of HCAHPS
are 3-fold: to support consumer choice, to incentivize
hospitals to improve care quality, and to enhance trans-
parency, leading to increased accountability.1
Hospitals are financially motivated to achieve good
HCAHPS scores. The Affordable Care Act of 2010 enacted
the Hospital Value-Based Purchasing (HVBP) program, in
which the Centers for Medicare & Medicaid Services
(CMS) partially links payments to Acute Care Inpatient
Prospective Payment System hospitals based on the
HCAHPS measures, excluding the scores for willingness
to recommend the hospital and transition to post-hospital
care3; 25% of the total performance score is a composite of
these HCAHPS measures.4 There are approximately 3000
hospitals participating in the HVBP program, out of
approximately 4770 general acute care hospitals.5,6 To
fund the incentives, CMS reduces the amount of its regular
payments to participating hospitals by a small percentage.
This reduction was 1.75% in 2016 and is 2% in 2017 and
2018.4,7 This incentive has been associated with modest or
no improvements8,9; however, hospitals also can be finan-
cially motivated to achieve good HCAHPS scores if they
are in competitive markets and believe they could poten-
tially lose market share if patients choose alternative hospi-
tals because of other hospitals’ higher HCAHPS scores.
Yet, there is evidence that factors beyond the direct
hospitalization experience influence HCAHPS scores.
Prior research has found that differences in HCAHPS
scores are explained in part by patient characteristics. For
example, sicker patients tend to rate their hospital experi-
ences lower than do patients with better health. Previous
research also has demonstrated that patient-reported
health care experiences differ by patients’ race/ethnicity,
language, education level, age, and proxy status.10-12 The
1
Yale University, New Haven, CT
2
Health Research & Educational Trust/American Hospital Association,
Chicago, IL
3
IMPAQ International, Columbia MD
Corresponding Author:
Kathleen G. Mockaitis, MPH, Health Research & Educational Trust/
American Hospital Association, 155 N Wacker Drive, Suite 400,
Chicago, IL 60606.
Email: kmockaitis@aha.org
462 American Journal of Medical Quality 33(5)
impact of these “social risk factors” on health outcomes
has been explicitly addressed in a report from the National
Academies of Science, Engineering and Medicine
(NASEM), which endorses the need to account for these
factors in payment programs.13 Extending the idea from
patient-specific factors, NASEM defined “community
context” as the characteristics of residential environments
that include the built/physical environment, social sup-
ports and cohesion, safety, and health behavior norms,
and identified this community context as an important
mediator of health outcomes in value-based programs.
However, little has been done to examine the impact of
these factors on the patient experience. McFarland et al
found that several hospital and county demographic fac-
tors were associated with HCAHPS scores.14 At the same
time, there is evidence that variation in other measures of
hospital quality can be explained in part by the character-
istics of the community the hospital serves.15-19 The study
team hypothesized that HCAHPS scores also would be
explained in part by community factors other than demo-
graphic features, and so to build on McFarland et al’s
work, the team included a range of community factors
reflecting cultural variations and differences in access to
care in addition to county demographic factors. As in
their prior work, the study team used the county (or
county equivalent) as the unit of community because,
while fairly crude, there are a wide range of data available
at the county (or county equivalent) level.
The conceptual model was that 3 types of community
factors had the potential to influence the HCAHPS scores
of the hospitals within the community. The first was
sociodemographic factors, including education, income,
and employment levels; this would be consistent with
McFarland et al, which reflects that patients of differing
sociodemographic status have differing health care expe-
riences or perceive their experiences differently. The sec-
ond type of factor was cultural, by which is meant factors
that may represent expectations and behavioral norms
regarding hospital utilization and experience; these
include urban status, population density, racial composi-
tion, and census region (as a proxy for potential geo-
graphic variation in cultural standards). Although this
domain has some overlap with sociodemographic factors,
the study team hypothesized that the effect of sociodemo-
graphic factors, as well as other unmeasured factors, may
differ by geography; this domain is meant to capture that
distinction. The third type of community factor that might
influence patient HCAHPS score would be one that
reflected level of access to primary care, as primary care
experience also may shape expectations regarding the
hospital experience; for example, a patient with a regular
primary care physician (PCP) who is available during the
inpatient stay may be more likely to have a better inpa-
tient experience. In this study, the team first assessed the
degree of variation in HCAHPS that was unexplained at
the county level by considering these 3 types of commu-
nity factors separately; among those factors that explained
the most variation at the county level, the team assessed
their independent effects overall.
Methods
Overview
This observational study linked publicly available
HCAHPS scores with hospital and county data from a
number of data sources. Hospital characteristics, includ-
ing location, were taken from the 2012 American Hospital
Association (AHA) Annual Survey.20 Community factors
were taken from the Area Health Resource File (AHRF)
for 2001 to 2013. Hierarchical linear models (HLMs)
were used to partition variance in HCAHPS summary
scores into county and hospital components, and to assess
the association between hospital and county factors and
those scores. Because hospital structural factors, such as
bed size, often correlate with county factors, such as rural
location, the primary analysis ignored such hospital char-
acteristics; these were included in a secondary analysis to
determine whether any primary findings would be
affected. Put differently, the study team used county fac-
tors to adjust the scores of all hospitals in a given county
and analyzed the impact of hospital-specific factors on
the adjusted scores.
Sample and Outcome
Third-party organizations administer HCAHPS surveys to
patients after discharge. Hospitals are requested to survey
at least 300 patients per year, and the study team included
all hospitals that had HCAHPS scores for the reporting
period of the fourth quarter of 2014. Starting in April
2015, CMS reports the HCAHPS results as a single 5-Star
rating; this rating system uses a method that reflects both
the intrinsic properties of the survey and wide-ranging
expert and stakeholder input.21 To be consistent with this
5-Star rating, the study team adopted a modified version
for the primary outcome. The CMS 5-Star Quality Rating
is based on a summary score that combines the HCAHPS
items using weightings of item categories and categories
of patient ratings for each item.22 The team constructed a
summary HCAHPS score using CMS weights shown in
Table 1 applied to the reported percentages of patient rat-
ings for each item. To aid interpretation, the team then
standardized this score to a mean of 0 and a standard devi-
ation of 10. The study team recognizes that there is mea-
surement error associated with this score; however,
because CMS reports only the underlying items’ response
rates categorically (number of respondents as <100,
Herrin et al 463
Table 1.  Scoring System Used for HCAHPS Summary.
Weight
HCAHPS composite measures
  1.  Communication with nurses
  2.  Communication with doctors
  3.  Responsiveness of hospital staff
  4.  Pain management
  5.  Communication about medicines
  6.  Discharge information
  7.  Care transition
HCAHPS individual items
  8.  Cleanliness of hospital environment
  9.  Quietness of hospital environment
HCAHPS global items
10.  Overall hospital rating
11.  Recommendation of the hospital
Abbreviation: HCAHPS, Hospital Consumer Assessment of
Healthcare Providers and Systems.
100-299, 300+), and more than 10% of hospitals have no
reported response rate, the team ignored measurement
uncertainty associated with response rate in the analysis.
Note that these scores are not adjusted for any case-mix
factors.
Community
The study team used the geographic regions defined by
the Federal Information Processing Standard (FIPS) as the
unit of analysis. The FIPS codes correspond to counties or
county equivalents (parish or borough) in most states or to
standard geographic areas in states without such boundar-
ies (eg, some New England states). They will be referred
to as “counties” or FIPS areas interchangeably.
Data Sources and Variables
Hospitals were linked using Medicare Provider Numbers
to the 2012 AHA Annual Survey file; from this file, the
study team obtained ownership type (public, private not-
for-profit, and private for-profit), teaching status (no
teaching program, residency program, and Council of
Teaching Hospitals [COTH] designation), bed size (clas-
sified as 1-25, 26-100, 101-200, 201-300 and ≥301),
community hospital status, and the FIPS codes for where
the hospital was located. The FIPS codes were used to
link hospitals to data from the 2001 to 2013 AHRF, from
which the team obtained for each FIPS area the census
region from the US Census (New England, Mid-Atlantic,
South Atlantic, East North Central, East South Central,
West North Central, West South Central, Mountain,
Pacific), population (2010 census), number of PCPs
(2011; this included both general practice and family
medicine physicians), and number of specialists (2011;
this included all physician specialists). From these, the
study team constructed variables representing numbers of
Medicare beneficiaries, PCPs per capita (×100 000), spe-
3 cialists per capita (×100 000), and the ratio of PCPs to
3 specialists. Because some areas had no PCPs or no spe-
3 cialists, 1 was added to each before calculating the ratio.
3 Also from the AHRF, the team obtained percent of the
3 population in poverty in 2010, percent African American,
3 percent Hispanic, median income, percent with no insur-
3 ance, percent high school and college graduates, and 4
county classifications made by the Department of
1
Agriculture: retirement community, low employment,
1
persistent poverty classification, and low education.
Finally, to classify each community according to urban
1
1
status, the team used the 2013 National Center for Health
Statistics (NCHS) Urban-Rural Classification Scheme
for Counties, developed by the NCHS and the Centers for
Disease Control and Prevention.23 The categories are
large central metro, large fringe metro, medium metro,
small metro, micropolitan, and rural.
Statistical Analysis
The study team summarized the HCAHPS score for all
hospitals, as well as by hospital ownership, bed size,
teaching status, community-hospital status, and census
region. For each continuous county measure, the team
constructed categories representing quintiles for use in
analysis.
To assess the amount of variation in HCAHPS score
that could be attributed to hospital and county character-
istics, respectively, the study team estimated an HLM
with HCAHPS score as the dependent variable and a ran-
dom intercept that varied over counties. HLMs allowed
the team to partition the total variance in HCAHPS scores
into components at the hospital and county levels, respec-
tively.15,24,25 Such models also can be used to assess the
relationship between hospital and county factors and
HCAHPS scores while accounting for any within-county
correlation of scores; the method of Bryk and Raudenbush
was used to calculate a version of R2 for each level of the
model (county and hospital); although it cannot be inter-
preted in the conventional sense of proportion of variance
explained, it does provide a comparative metric across
models.23
To identify the most important county factors both
within each domain and overall, the study team used a
sequential approach that has been employed previ-
ously.15,26 Because of collinearity between many charac-
teristics that were not conceptually related (eg, population
density and poverty rate), the team used an approach that
was both mechanical and conceptual to reduce the num-
ber of factors. After estimating the empty model, the team
464 American Journal of Medical Quality 33(5)

estimated a series of bivariate models, with each county- Table 2.  Characteristics of Hospitals Included in the
level factor included as the only independent variable. Analysis, and Bivariate Relationships With HCAHPS Scores.
For each model, the study team calculated the P value for Hospitals Standardized
the independent variable, as well as the variance explained
at the hospital and county levels, respectively.23 Those   n (%) HCAHPS Score
variables with a county-level R2 of at least 5% were N 4021 (100.0) 0
retained and grouped into county demographic factors, Beds (category)
county access-to-care factors, and county structural fac-   ≤25 738 (18.4) 8.3
tors. Within each of these groups, the study team included  26-100 1088 (27.1) 2.1
all retained factors in a multivariable HLM with a random  101-200 896 (22.3) −3.2
intercept across counties; all county variables with Wald  201-300 531 (13.2) −4.3
P < .05 in one of these domain models were retained for   ≥301 768 (19.1) −4.3
a final multivariable HLM. Ownership
Because hospital characteristics and community fac-  Government 776 (19.3) 1.3
tors are often correlated (eg, smaller bed size correlates  Nonprofit 2474 (61.5) 0.4
with rural location), the primary analysis did not include  Profit 771 (19.2) −2.6
hospital characteristics. However, as a secondary analysis Teaching status
to assess whether any community effects were attenuated  None 2891 (71.9) 1
by inclusion of hospital characteristics, the team included  Residency 877 (21.8) −2.3
hospital ownership, bed size, teaching status, and com-  COTH 253 (6.3) −3.7
munity hospital status in another model. For interpreta- Community hospital
tion, the adjusted rates from each model are reported.  No 30 (0.7) −13
All analyses were performed using Stata version 14  Yes 3991 (99.3) 0.1
Region
(StataCorp LLC, College Station, Texas) statistical software.
  New England 173 (4.3) 1.1
 Mid-Atlantic 382 (9.5) −5.7
Results   South Atlantic 662 (16.5) 2.5
  East North Central 506 (12.6) 5.5
A total of 4065 hospitals had HCAHPS results reported   East South Central 606 (15.1) −3.5
on Hospital Compare. Of these, 4021 could be linked to   West North Central 339 (8.4) 0.8
the AHA Annual Survey file (Table 2). These hospitals   West South Central 583 (14.5) 2.2
were in 2117 counties, with a median [interquartile range]  Mountain 318 (7.9) 0.5
of 2 [1, 6] hospitals per county.  Pacific 452 (11.2) −4.4
Of the total variance in the HCAHPS summary score
across hospitals explained by the 5 hospital characteris- Abbreviations: COTH, Council of Teaching Hospitals; HCAHPS,
Hospital Consumer Assessment of Healthcare Providers and Systems.
tics variables, 27.4% was at the county level and 72.6%
was at the hospital level. HCAHPS summary scores dif-
fered significantly according to all of the hospital charac- African American, and percent Hispanic explaining
teristics in Table 2, with smaller hospitals, public between 19.8% and 29.4% of the variation in HCAHPS
hospitals, nonteaching hospitals, and community hospi- scores across counties.
tals having higher HCAHPS summary scores. Multivariable models for the access-to-care and cul-
Examination of county factors and their bivariate rela- tural domains, including those factors explaining more
tionship with HCAHPS scores found that all of them than 10% of the variation across counties, found similar
explained significant amounts of variation (P < .01 for results, with all being significant (P < .01; Table 4).
all; Table 3). The final multivariable model included all of the fac-
Among the 3 domains, sociodemographic factors tors from Table 4. In this model, shown in Table 5, only 4
explained the least percent variance at the county and county-level factors remained independently significant:
hospital levels. In contrast, access-to-care measures (in PCPs per capita, specialists per capita, percent African
particular, PCPs per capita, specialists per capita, and American, and percent Hispanic. Only PCPs per capita
their ratio) explained up to 30.7% of the variation in had a positive association, with counties in the highest
HCAHPS scores across counties (negative percent quintile of this metric having adjusted HCAHPS summary
explained at the county level indicates that, while explain- scores of 4.46, or nearly half a standard deviation, above
ing some hospital-level variation, the effect at the county those in the lowest quintile. Together, the county factors
level was to widen differences across counties). Cultural explained 15.1% of the total hospital-level variance in
factors were similar, with density, urban status, percent HCAHPS summary scores; the county-level R2 equaled
Herrin et al 465

Table 3.  Bivariate Relationships Between County Factors and HCAHPS Scoresa.

R2

Factor Value n (%) Mean (SD) County Hospital P

Sociodemographic  
% College 4.1% −0.4% .001
  Q1 [5.7-13.7] 424 (20.0) 1.4 (9.5)  
  Q2 [13.8-16.7] 427 (20.2) 2.1 (9.0)  
  Q3 [16.8-20.5] 426 (20.1) 2.1 (9.3)  
  Q4 [20.6-27.4] 417 (19.7) 0.9 (8.7)  
  Q5 [27.5-71.2] 422 (19.9) −0.1 (8.3)  
Median income 2.3% 0.5% <.001
  Q1 [$22K-$37K] 424 (20.0) −0.4 (10.1)  
  Q2 [$37K-$42K] 423 (20.0) 1.1 (8.8)  
  Q3 [$42K-$46K] 423 (20.0) 2.4 (7.9)  
  Q4 [$46K-$53K] 424 (20.0) 2.5 (8.7)  
  Q5 [$53K-$120K] 422 (19.9) 0.8 (9.0)  
Low education 1.4% 0.2% <.001
  No 1766 (83.4) 1.6 (8.8)  
  Yes 351 (16.6) −0.2 (9.6)  
Low employment 4.4% 0.3% <.001
  No 1864 (88.0) 1.7 (8.5)  
  Yes 253 (12.0) −1.9 (11.7)  
High poverty area 1.9% 0.8% <.001
  No 1640 (77.5) 2.0 (8.6)  
  Yes 476 (22.5) −1.2 (9.9)  
Persistent poverty 2.7% 0.4% <.001
  No 1914 (90.4) 1.7 (8.7)  
  Yes 202 (9.5) −2.3 (11.1)  
   
Cultural  
Population density 25.8% 0.4% <.001
  Q1 [4-221] 424 (20.0) 4.6 (10.8)  
  Q2 [223-490] 425 (20.1) 3.6 (8.2)  
  Q3 [491-943] 421 (19.9) 1.4 (8.1)  
  Q4 [945-2578] 423 (20.0) 0.3 (7.3)  
  Q5 [2579-690 000] 423 (20.0) −3.4 (8.0)  
NCHS code 29.4% −0.9% <.001
  Large central metro 68 (3.2) −7.8 (10.9)  
  Large fringe metro 292 (13.8) −0.9 (8.6)  
  Medium metro 278 (13.1) −0.0 (7.7)  
  Small metro 262 (12.4) 0.4 (7.8)  
  Micropolitan 541 (25.6) 0.8 (7.6)  
  Noncore 676 (31.9) 4.4 (9.7)  
Retirement area 1.0% 0.0% .005
  No 1831 (86.5) 1.5 (9.0)  
  Yes 286 (13.5) 0.1 (8.8)  
African Americans (%) 22.2% 0.6% <.001
  Q1 [0.0-0.5] 470 (22.2) 5.2 (9.3)  
  Q2 [0.6-1.5] 380 (17.9) 2.6 (8.0)  
  Q3 [1.6-4.7] 425 (20.1) 1.1 (8.1)  
  Q4 [4.8-15.0] 418 (19.7) −1.3 (8.5)  
  Q5 [15.1-83.4] 423 (20.0) −1.5 (9.0)  

(continued)
466 American Journal of Medical Quality 33(5)

Table 3. (continued)
R2

Factor Value n (%) Mean (SD) County Hospital P

Hispanics (%) 19.8% −1.4% <.001

  Q1 [0.1-1.6] 464 (21.9) 3.3 (8.7)  
  Q2 [1.7-2.8] 395 (18.7) 2.8 (8.6)  
  Q3 [2.9-5.2] 423 (20.0) 0.9 (8.3)  
  Q4 [5.3-11.5] 413 (19.5) 0.5 (8.2)  
  Q5 [11.6-95.7] 421 (19.9) −1.2 (10.2)  
   
Access to care  
% <65 No Insurance 4.2% 0.5% <.001
  Q1 [2.9-12.2] 431 (20.4) 3.3 (8.2)  
  Q2 [12.3-15.2] 422 (19.9) 2.3 (8.3)  
  Q3 [15.3-18.2] 433 (20.5) 0.9 (8.9)  
  Q4 [18.3-21.1] 410 (19.4) 0.6 (9.3)  
  Q5 [21.2-38.6] 420 (19.8) −0.7 (9.7)  
PCPs per 100 000 19.1% −0.3% <.001
  Q1 [0-15.3] 424 (20.0) −1.0 (10.7)  
  Q2 [15.3-22.1] 423 (20.0) 0.1 (8.6)  
  Q3 [22.1-29.3] 423 (20.0) 0.5 (8.3)  
  Q4 [29.3-40.3] 423 (20.0) 2.7 (7.3)  
  Q5 [40.4-254.2] 423 (20.0) 4.2 (8.8)  
Specialists per 100 000 22.1% 0.8% <.001
  Q1 [0-9.6] 424 (20.0) 5.6 (10.0)  
  Q2 [9.6-21.1] 423 (20.0) 3.2 (9.1)  
  Q3 [21.1-38.3] 423 (20.0) 0.5 (7.8)  
  Q4 [38.3-67.7] 423 (20.0) −1.1 (7.2)  
  Q5 [67.7-1321.4] 423 (20.0) −1.9 (8.4)  
PCPs/specs 30.7% 1.2% <.001
  Q1 [3.3-35.5] 424 (20.0) −3.1 (8.6)  
  Q2 [35.6-61.5] 428 (20.2) −1.3 (7.4)  
  Q3 [61.8-100.0] 449 (21.2) 1.5 (7.8)  
  Q4 [102.3-200.0] 407 (19.2) 3.9 (9.0)  
  Q5 [207.4-2000.0] 408 (19.3) 5.8 (9.2)  
ED visits per 100 000 −0.8% 0.6% .006
  Q1 [0-29K] 424 (20.0) 3.1 (10.3)  
  Q2 [29K-41K] 423 (20.0) 1.8 (8.6)  
  Q3 [41K-51K] 423 (20.0) 1.0 (9.1)  
  Q4 [51K-65K] 423 (20.0) 0.9 (8.0)  
  Q5 [65K-77K] 423 (20.0) −0.4 (8.5)  
OP visits per 100 000 3.5% 0.2% <.001
  Q1 [0-75K] 424 (20.0) 0.5 (10.2)  
  Q2 [76K-13K] 423 (20.0) 0.2 (8.5)  
  Q3 [13K-21K] 423 (20.0) 1.0 (8.7)  
  Q4 [21K-34K] 423 (20.0) 2.4 (8.8)  
  Q5 [34K-43K] 423 (20.0) 2.2 (8.6)  

Abbreviations: ED, emergency department; HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; NCHS, National
Center for Health Statistics; OP, outpatient; PCP, primary care provider; Q, quarter.
a
Some Ns do not add to total because of missing data.
Herrin et al 467

Table 4.  Results of Multivariable Models for Each Domain.

Access to Care Coeff (SE) P Wald P Cultural Coeff (SE) P Wald P

Intercept 1.2 (1.0) .236 Intercept 4.0 (1.1) <.001  
PCPs/100 000 <.001 Density <.001
  Q1 [0.0-15.34] Ref   Q1 [4-221] Ref  
  Q2 [15.35-22.11] 2.1 (0.6) <.001   Q2 [223-490] 0.3 (0.6) .621  
  Q3 [22.12-29.28] 3.0 (0.6) <.001   Q3 [491-943] −1.4 (0.7) .043  
  Q4 [29.29-40.33] 4.9 (0.7) <.001   Q4 [945-2578] −2.0 (0.8) .007  
  Q5 [40.36-254.24] 6.0 (0.8) <.001   Q5 [2579-690 000] −3.7 (0.8) <.001  
Specialists/100 000 <.001 NCHS code <.001
  Q1 [0.00-9.58] Ref   Large central metro Ref  
  Q2 [9.58-21.15] −2.4 (0.7) .001   Large fringe metro 0.9 (0.8) .24  
  Q3 [21.18-38.32] −4.5 (0.8) <.001   Medium metro 2.4 (0.8) .002  
  Q4 [38.33-67.68] −6.2 (1.0) <.001   Small metro 1.8 (0.9) .047  
  Q5 [67.75-1321.40] −6.2 (1.2) <.001  Micropolitan 0.5 (0.9) .562  
PCPs/spec .006  Noncore 2.2 (1.0) .024  
  Q1 [3.34-35.53] Ref % African American <.001
  Q2 [35.56-61.54] 1.0 (0.6) .089   Q1 [0.0-0.5] Ref  
  Q3 [61.76-100.00] 2.2 (0.8) .005   Q2 [0.6-1.5] −0.9 (0.6) .147  
  Q4 [102.33-200.00] 2.0 (1.0) .056   Q3 [1.6-4.7] −1.6 (0.6) .009  
  Q5 [207.41-2000.00] 0.7 (1.3) .59   Q4 [4.8-15.0] −2.9 (0.6) <.001  
    Q5 [15.1-83.4] −4.1 (0.6) <.001  
  % Hispanic <.001
    Q1 [0.1-1.6] Ref  
    Q2 [1.7-2.8] 0.6 (0.6) .295  
    Q3 [2.9-5.2] −0.1 (0.6) .907  
    Q4 [5.3-11.5] −0.4 (0.6) .509  
    Q5 [11.6-95.7] −2.6 (0.6) <.001  
Unexplained variance  
 County 15.0 15.1  
 Hospital 67.4 67.7  
 Total 82.4 82.8  
R2 county 41.4% 41.0%  
R2 hospital 0.9% 0.4%  

Abbreviations: NCHS, National Center for Health Statistics; PCP, primary care provider; Q, quarter; SE, standard error.

51.9%. The addition of the hospital factors more than dou-
bled the variance explained to 32.9%, and increased the
county R2 to 76.9%, while somewhat attenuating the
effects of the county factors (Table 5). The largest effect in
the final model was that of hospital bed size.
Discussion
Findings
This study examined the relationship between hospital
and county characteristics and publicly reported HCAHPS
scores. Using a summary HCAHPS metric, this study
found that although the strongest associations were for
hospital factors, especially bed size and community-hos-
pital status, more than a quarter of the variation in scores
was at the county level. Both higher numbers of PCPs per
capita and lower numbers of specialists per capita, mea-
sured at the county level, were strongly associated with
higher HCAHPS summary scores for hospitals in those
counties. At the same time, increasing percentages of
both African Americans and Hispanics were associated
with decreasing HCAHPS summary scores for hospitals
in those counties.
Notably, this study found that of the 3 domains of influ-
ence, sociodemographic factors had the least relationship
to HCAHPS scores; none of the factors in this domain had
county-level R2 values greater than 5%. Hospitals in coun-
ties with the highest percentage of college graduates had
scores that differed by only 0.03 standard deviations from
hospitals in counties with the lowest percentage of college
graduates. Areas of persistent poverty, high poverty, low
employment, and low education all had lower HCAHPS
summary scores than other areas, but effects were modest.
468 American Journal of Medical Quality 33(5)

Table 5.  Final Multivariable Models of County Factors, Without and With Hospital Factorsa.

Coeff (SE) P Wald P Coeff (SE) P Wald P

Intercept 2.3 (1.4) .115 −12.0 (2.1) <.001  
PCPs/100 000 <.001 <.001
  Q1 [0.0-15.34] Ref Ref  
  Q2 [15.35-22.11] 2.1 (0.5) <.001 1.9 (0.5) <.001  
  Q3 [22.12-29.28] 2.8 (0.6) <.001 2.4 (0.5) <.001  
  Q4 [29.29-40.33] 4.1 (0.7) <.001 3.5 (0.6) <.001  
  Q5 [40.36-254.24] 4.5 (0.8) <.001 3.4 (0.7) <.001  
Specialists/100 000 <.001 .04
  Q1 [0.00-9.58] Ref Ref  
  Q2 [9.58-21.15] −1.8 (0.7) .009 −0.8 (0.6) .17  
  Q3 [21.18-38.32] −3.7 (0.9) <.001 −0.3 (0.8) .66  
  Q4 [38.33-67.68] −4.5 (1.0) <.001 0.6 (0.9) .545  
  Q5 [67.75-1321.40] −3.4 (1.2) .007 1.5 (1.1) .18  
PCPs/spec .11 .11
  Q1 [3.34-35.53] Ref Ref  
  Q2 [35.56-61.54] 0.0 (0.6) .941 −0.2 (0.5) .708  
  Q3 [61.76-100.00] 1.3 (0.8) .105 0.8 (0.7) .234  
  Q4 [102.33-200.00] 1.0 (1.0) .325 0.5 (0.9) .556  
  Q5 [207.41-2000.00] 0.2 (1.3) .897 −0.5 (1.1) .673  
Density .06 .22
  Q1 [4-221] Ref Ref  
  Q2 [223-490] 0.7 (0.6) .279 1.1 (0.6) .049  
  Q3 [491-943] −0.4 (0.7) .602 0.9 (0.7) .16  
  Q4 [945-2578] −0.4 (0.8) .632 1.2 (0.7) .121  
  Q5 [2579-690000] −1.7 (0.9) .062 0.5 (0.9) .593  
NCHS code .06 <.001
  Large central metro Ref Ref  
  Large fringe metro 1.1 (0.8) .14 1.0 (0.6) .118  
  Medium metro 2.1 (0.8) .007 2.1 (0.6) .001  
  Small metro 1.8 (0.9) .048 1.6 (0.8) .039  
 Micropolitan 0.9 (0.9) .312 0.3 (0.8) .659  
 Noncore 1.5 (1.0) .124 0.5 (0.9) .579  
% Black <.001 .03
  Q1 [0.0-0.5] Ref Ref  
  Q2 [0.6-1.5] −0.8 (0.6) .166 −0.5 (0.5) .375  
  Q3 [1.6-4.7] −1.0 (0.6) .101 −0.3 (0.5) .558  
  Q4 [4.8-15.0] −1.9 (0.6) .003 −1.0 (0.6) .082  
  Q5 [15.1-83.4] −3.1 (0.6) <.001 −1.8 (0.6) .005  
% Hispanic <.001 <.001
  Q1 [0.1-1.6] Ref  
  Q2 [1.7-2.8] 0.3 (0.6) .546 0.55 (0.50) .273  
  Q3 [2.9-5.2] −0.2 (0.6) .691 0.01 (0.50) .98  
  Q4 [5.3-11.5] −0.6 (0.6) .253 0.35 (0.52) .497  
  Q5 [11.6-95.7] −2.5 (0.6) <.001 −1.62 (0.57) .005  
Beds (category) <.001
  ≤25  
 26-100 −5.8 (0.4) <.001  
 101-200 −10.4 (0.5) <.001  
 201-300 −11.2 (0.5) <.001  
  ≥301 −11.4 (0.6) <.001  
Ownership <.001
 Government Ref  
 Nonprofit 0.9 (0.4) .009  
(continued)
Herrin et al 469

Table 5. (continued)

Coeff (SE) P Wald P Coeff (SE) P Wald P

 Profit −2.6 (0.4) <.001  
Teaching Status .049
 None Ref  
 Residency −0.5 (0.3) .194  
 COTH 0.9 (0.6) .13  
Community hospital <.001
 No Ref  
 Yes 15.9 (1.5) <.001  
Region <.001
  New England Ref  
 Mid-Atlantic −2.5 (0.9) .004  
  South Atlantic 2.0 (0.8) .016  
  East North Central 2.8 (0.8) .001  
  East South Central −0.9 (0.9) .272  
  West North Central 1.6 (0.9) .07  
  West South Central 2.8 (0.9) .001  
 Mountain 0.7 (1.0) .492  
 Pacific −2.6 (0.9) .005  
Unexplained variance  
 County 12.3 5.9  
 Hospital 67.2 57.0  
 Total 79.5 62.9  
R2 county 51.9% 76.9%  
R2 hospital 1.3% 16.3%  

Abbreviations: COTH, Council of Teaching Hospitals; HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; NCHS,
National Center for Health Statistics; PCP, primary care provider; Q, quarter.
a
Coefficients are in standardized HCAHPS score units.

In contrast, in the final multivariable model (Table 5),
counties that had larger numbers of PCPs and fewer spe-
cialists per capita had hospitals with substantially higher
HCAHPS summary scores; this is consistent with previ-
ous findings that these access-to-care factors were strongly
associated with lower hospital readmission rates15; per-
haps patients who must be readmitted shortly after dis-
charge give lower HCAHPS ratings, regardless of the true
experience. This also would be consistent with the bivari-
ate finding (Table 3) that areas with more outpatient visits
per capita and fewer emergency department visits per
capita also had hospitals with higher HCAHPS summary
scores. These findings suggest that future work might look
at the relationship of HCAHPS scores with the primary
care CAHPS scores in the same community.27
Findings regarding the factors in the cultural domain
are more difficult to interpret. The finding that hospitals
in areas with higher proportions of African American
and/or Hispanic residents had lower HCAHPS scores was
consistent with McFarland et al, who found higher
HCAHPS scores for hospitals in areas with higher per-
centages of whites. This could be a consequence of issues
such as patients from these minority groups having a
different objective experience; African American and
Hispanic patients having different expectations; cultural
or communication barriers between patients and staff if
staff are comprised of fewer minorities; or hospitals in
those areas being qualitatively different from hospitals in
other areas. There is also evidence that minorities have
different communication experiences; Zhu et al examined
patient-level data and found a relationship between
minority racial status and specific components of
HCAHPS related to communication.11
Limitations
There are several limitations to this study. First, as with
any observational study, associations identified cannot
be interpreted as causal relationships. Furthermore, the
number and response rate of administered surveys all
vary across hospitals in ways that are unknown. Ignoring
variation in the number of surveys included may produce
artificially small standard errors; however, the study
team has focused on findings that have both meaningful
effect sizes and a very small probability of type II error.
The unknown response rate is a more critical limitation
470 American Journal of Medical Quality 33(5)
in that it is likely that the individuals who respond to
surveys differ systematically from those who do not with
regard to a number of community factors, particularly
sociodemographic and cultural factors. However, if com-
munity factors influence differential response rates,
these findings still can inform the perspective of the hos-
pital that must report the HCAHPS scores, while indicat-
ing an area for future research. A related limitation is that
the study team must assume that 2 hospitals in the same
county are influenced similarly by the same community
factors, whereas in dense, urban areas especially, it is
likely that 2 nearby hospitals may serve very different
populations; addressing this limitation would require
residence-level data on respondents, which is currently
not available at the national level. Conversely, by restrict-
ing community to a single county, the team also has
excluded, for some hospitals, a substantial area they
serve; however, expanding the definition of community
to some larger catchment area also risks measuring com-
munity factors that represent only a small fraction of the
patients admitted to the hospital. A related limitation is
that residents of rural or suburban areas may seek care
locally for less critical concerns but travel to remote ter-
tiary or quaternary hospitals for more serious conditions,
biasing the patient experience at rural hospitals toward
that of patients with less severe conditions. An additional
limitation is that, because this study is exploratory, the
study team may have omitted from consideration county
factors that could have strong associations with HCAHPS
scores; although this does limit the breadth of the con-
clusions (in that there may be other, more critical com-
munity factors at work), it does not bias the current
findings. Finally, a limitation that is important to keep in
mind is that HCAHPS scores themselves are self-
reported measures and, as such, they may be biased by
factors not considered here.
Implications
In summary, these findings provide evidence that com-
munity factors have the potential to influence HCAHPS
scores in ways that are beyond the hospitalization experi-
ence. Some of these factors may be amenable to hospital
influence; for example, cultural differences can be
addressed with better communication and health literacy
promotion, and hospitals increasingly have ownership
influence over some PCPs. As HCAHPS scores become
more integrated into the CMS payment system and the
overall Hospital Star Quality Ratings system,28 it will be
ever more important for policy makers to understand the
influence of community factors. Though hospitals can be
rewarded for improvement as well as achievement under
the payment program,3 those hospitals serving communi-
ties that possess characteristics linked to lower HCAHPS
scores still could be disadvantaged if these differences
can be attributed to factors outside of their control.
Conclusion
This study found that a number of community factors
beyond the hospitalization experience potentially influ-
ence HCAHPS scores. As HCAHPS gain importance as a
measure of patient experience and are linked to higher pay-
ments over time, it will be important to know and better
understand the potential effects of community factors.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The authors received no financial support for the research,
authorship, and/or publication of this article.
References
1. Centers for Medicare & Medicaid Services. The
HCAHPS survey—frequently asked questions. http://
www.cms.gov/Medicare/Quality-Initiatives-Patient-
Assessment-Instruments/HospitalQualityInits/Downloads/
HCAHPSFAQ092713.pdf. Accessed November 20, 2017.
2. Centers for Medicare & Medicaid Services. HCAHPS fact
sheet. http://www.hcahpsonline.org/files/HCAHPS%20
Fact%20Sheet,%20revised1,%203-31-09.pdf. Published
2009. Accessed November 20, 2017.
3. Centers for Medicare & Medicaid Services. Hospital
value-based purchasing. https://www.cms.gov/Outreach-
and-Education/Medicare-Learning-Network-MLN/
MLNProducts/downloads/Hospital_VBPurchasing_Fact_
Sheet_ICN907664.pdf. Accessed November 22, 2017.
4. Centers for Medicare & Medicaid Services. CMS Hospital
Value-based Purchasing Program results for fiscal year 2018.
https://www.cms.gov/Newsroom/MediaReleaseDatabase/
Fact-sheets/2017-Fact-Sheet-items/2017-11-03.html.
Accessed February 9, 2018.
5. US Government Accountability Office. Hospital Value-
based Purchasing Program: CMS should take steps to
ensure lower quality hospitals do not qualify for bonuses.
https://www.gao.gov/assets/690/685586.pdf. Published
June 2017. Accessed February 9, 2018.
6. American Hospital Association. AHA Guide. 2013 ed.
Chicago, IL: Health Forum, LLC; 2013.
7. Centers for Medicare & Medicaid Services. Frequently asked
questions—hospital value-based purchasing program.
http://www.cms.gov/Medicare/Quality-Initiatives-Patient-
Assessment-Instruments/hospital-value-based-purchasing/
Downloads/FY-2013-Program-Frequently-Asked-Questions-
about-Hospital-VBP-3-9-12.pdf. Accessed October 20, 2015.
8. Papanicolas I, Figueroa JF, Orav EJ, Jha AK. Patient
hospital experience improved modestly, but no evidence
Herrin et al 471
Medicare incentives promoted meaningful gains. Health
Aff (Millwood). 2017;36:133-140.
9. US General Accounting Office Report. Hospital Value-
Based Purchasing. https://www.gao.gov/products/GAO-
16-9. Accessed January 11, 2018.
10. Elliott MN, Lehrman WG, Goldstein E, Hambarsoomian
K, Beckett M, Giordano LA. Do hospitals rank differently
on HCAHPS for different patient subgroups? Med Care
Res Rev. 2010;67:56-73.
11. Zhu J, Weingart SN, Ritter GA, Tompkins CP, Garnick
DW. Racial/ethnic disparities in patient experience with
communication in hospitals: real differences or measure-
ment errors? Med Care. 2015;53:446-454.
12. Elliott MN, Lehrman WG, Beckett MK, Goldstein E,
Hambarsoomian K, Giordano LA. Gender differences in
patients’ perceptions of inpatient care. Health Serv Res.
2012;47:1482-1501.
13. National Academies of Science, Engineering, and
Medicine. Accounting for Social Risk Factors in Medicare
Payment: Identifying Social Risk Factors. Washington,
DC: National Academies Press; 2016.
14. McFarland DC, Ornstein KA, Holcombe RF. Demographic
factors and hospital size predict patient satisfaction vari-
ance-implications for hospital value-based purchasing. J
Hosp Med. 2015;10:503-509.
15. Herrin J, St. Andre J, Kenward K, Joshi MS, Audet AM,
Hines S. Community factors and hospital readmission
rates. Health Serv Res. 2015:50:20-39.
16. Arbaje AI, Wolff JL, Yu Q, Powe NR, Anderson GF,
Boult C. Post-discharge environmental and socioeco-
nomic factors and the likelihood of early hospital readmis-
sion among community-dwelling Medicare beneficiaries.
Gerontologist. 2008;48:495-504.
17. Foraker RE, Rose KM, Suchindran CM, Chang PP, McNeill
AM, Rosamond WD. Socioeconomic status, Medicaid cov-
erage, clinical comorbidity, and rehospitalization or death
after an incident heart failure hospitalization: atherosclerosis
risk in communities cohort (1987 to 2004). Circ Heart Fail.
2011;4:308-316.
18. Hawkins NM, Jhund PS, McMurray JJ, Capewell S. Heart
failure and socioeconomic status: accumulating evidence
of inequality. Eur J Heart Fail. 2012;14:138-146.
19. Joynt KE, Orav EJ, Jha AK. Thirty-day readmission rates
for Medicare beneficiaries by race and site of care. JAMA.
2011;305:675-681.
20. American Hospital Association. AHA Guide. 2012 ed.
Chicago, IL: Health Forum, LLC; 2012.
21. Centers for Medicare & Medicaid Services. HCAHPS star
ratings. http://www.hcahpsonline.org/StarRatings.aspx.
Accessed November 22, 2017.
22. Centers for Medicare & Medicaid Services. Technical
Notes for HCAHPS star ratings. http://www.hcahpsonline.
org/globalassets/hcahps/star-ratings/tech-notes/2017-10_
star-ratings_tech-notes.pdf. Published 2015. Accessed
January 2, 2018.
23. Ingram DD, Franco SJ. 2013 NCHS urban–rural classification
scheme for counties. Vital Health Stat 2. 2014;(166):1-73.
24. Bryk AS, Raudenbush SW. Hierarchical Linear Models in
Social and Behavioral Research: Applications and Data
Analysis Methods. 1st ed. Newbury Park, CA: Sage; 1992.
25. Haymart MR, Banerjee M, Stewart AK, Koenig RJ,
Birkmeyer JD, Griggs JJ. Use of radioactive iodine for thy-
roid cancer. JAMA. 2011;306:721-728.
26. Bradley EH, Curry LA, Spatz ES, et al. Hospital strategies
for reducing risk-standardized mortality rates in acute myo-
cardial infarction. Ann Intern Med. 2012;156:618-626.
27. Agency for Healthcare Research and Quality. CAHPS©:
Assessing Health Care Quality from the Patient’s Perspective.
https://www.ahrq.gov/cahps/about-cahps/cahps-program/
cahps_brief.html. Accessed January 11, 2018.
28. QualityNet. Hospital star ratings overview. https://www.
qualitynet.org/dcs/ContentServer?c=Page&pagename=Q
netPublic%2FPage%2FQnetTier2&cid=1228775183434.
Accessed October 20, 2015.