Journal Pre-proof

The Patient Acceptable Symptom State for the Harris Hip Score Following Total Hip
Arthroplasty Validated thresholds at 3 months, 1, 3, 5, and 7 years follow-up

Vincent Galea, BA, Isabella Florissi, BA, Pakdee Rojanasopondist, BA, James W.
Connelly, BA, Lina Holm Ingelsrud, PhD, Charles Bragdon, PhD, Henrik Malchau,
MD, PhD, Anders Troelsen, MD, PhD
PII: S0883-5403(19)30768-5
DOI: https://doi.org/10.1016/j.arth.2019.08.037
Reference: YARTH 57476

To appear in: The Journal of Arthroplasty

Received Date: 1 June 2019

Revised Date: 7 August 2019
Accepted Date: 14 August 2019

Please cite this article as: Galea V, Florissi I, Rojanasopondist P, Connelly JW, Ingelsrud LH, Bragdon
C, Malchau H, Troelsen A, The Patient Acceptable Symptom State for the Harris Hip Score Following
Total Hip Arthroplasty Validated thresholds at 3 months, 1, 3, 5, and 7 years follow-up, The Journal of
Arthroplasty (2019), doi: https://doi.org/10.1016/j.arth.2019.08.037.

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition
of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of
record. This version will undergo additional copyediting, typesetting and review before it is published
in its final form, but we are providing this version to give early visibility of the article. Please note that,
during the production process, errors may be discovered which could affect the content, and all legal
disclaimers that apply to the journal pertain.

© 2019 Elsevier Inc. All rights reserved.

The Patient Acceptable Symptom State for the
Harris Hip Score Following Total Hip Arthroplasty
Validated thresholds at 3 months, 1, 3, 5, and 7 years follow-up

Vincent Galea BAa, Isabella Florissi BAa, Pakdee Rojanasopondist BAa, James W. Connelly
BAa, Lina Holm Ingelsrud PhDb, Charles Bragdon PhDa,c, Henrik Malchau MD, PhDa,c, Anders
Troelsen MD, PhDb

a
Harris Orthopaedic Laboratory, Massachusetts General Hospital, 55 Fruit St. GRJ 1231, Boston, MA 02114
b
Copenhagen University Hospital, Department of Orthopedic Surgery, Kettegård Alle 30, 2650 Hvidovre, Denmark
c
Harvard Medical School, Department of Orthopaedic Surgery, 25 Shattuck St., Boston, MA 02115

Corresponding Author:
Vincent Galea, BA
Harris Orthopaedic Laboratory
Massachusetts General Hospital
55 Fruit St. GRJ 1231, Boston, MA 02114
Email: vgalea@mgh.harvard.edu
Tel: (917) 683-3185
 1

1 The Patient Acceptable Symptom State for the

2 Harris Hip Score Following Total Hip Arthroplasty
3 Validated thresholds at 3 months, 1, 3, 5, and 7 years follow-up
4

5 Abstract

6 Background

7 The Patient Acceptable Symptom State (PASS) represents the value on a patient-reported outcome

8 measure (PROM) scale beyond which patients consider themselves well or in a satisfactory state. PASS

9 values for the Harris Hip Score (HHS) after total hip arthroplasty (THA) are currently lacking. The aim of

10 this study was to define and validate the PASS threshold for the HHS at 3 months, 1, 3, 5, and 7 years

11 after THA.

12 Methods

13 A total of 976 patients from 14 centers in 7 countries were enrolled into a prospective study. Patients

14 completed the HHS and a 21-point numerical rating scale (NRS) for satisfaction at each follow-up

15 interval. PASS thresholds for the HHS were calculated at each follow-up interval using the anchor-based,

16 80% specificity method. Satisfaction from the NRS served as the anchor. A bootstrapping method was

17 used to internally validate the primary PASS thresholds. External validation was also conducted on

18 patients sourced from the institutional registry of a tertiary academic medical center.

19 Results

20 Based on receiver operator characteristics analysis, the HHS was an excellent predictor of satisfaction at

21 each time point (area under the curve (AUC)>0.8; p<0.001). PASS thresholds for the HHS were 76 points

22 at 3-months, 89 points at 1-year, 93 points at 3-years, 94 points at 5-years, and 93 points at 7-years. When

23 applied to the bootstrapped internal validation cohorts as well as the external validation cohorts, all PASS

24 thresholds showed acceptable or excellent ability to predict satisfaction (AUC = 0.73-80; p<0.001).
 2

25 Conclusion

26 The current study is the first to present validated PASS thresholds for the HHS at 3 months, 1, 3, 5, and 7

27 years following THA. These findings will serve as a useful reference for future THA outcome studies and

28 as benchmarks for surgeons in their assessment of their patients’ clinical success.

29 Keywords: Total Hip Arthroplasty; Patient-Reported Outcome Measures; Patient Acceptable Symptom

30 State; Harris Hip Score; Satisfaction

31
 3

32 Introduction
33 Patient-reported outcome measures (PROMs) have been widely adopted in the field of orthopedics in

34 recent years [1]. PROMs are able to capture a patient’s everyday experience as well as the effects of a

35 treatment on a patient’s activities of daily living, directly from the patient’s perspective. Furthermore,

36 PROMs offer a standardized way to measure important, subjective health status information that cannot

37 be detected by objective or surgeon-reported outcome measures [2,3]. In the evaluation of total hip

38 arthroplasty (THA), PROMs are especially valuable metrics due to the relative infrequency of other

39 traditional endpoints such as revision and infection [4].

40 PROMs, however, pose some difficulty in their analysis and interpretation. An important potential pitfall

41 is the erroneous attribution of clinical significance to a finding of statistically significant differences

42 between PROMs [5]. The patient acceptable symptom state (PASS) is a tool that attempts to mitigate this

43 issue, and is defined as the point on a PROM scale that represents the value beyond which patients

44 consider themselves well or in a satisfactory state [6,7]. It is distinct from the minimum clinically

45 important difference, another commonly utilized PROM interpretation tool, in that it is used to evaluate a

46 cross-sectional treatment outcome as opposed to the change in score before and after treatment [8–10].

47 The PASS has most commonly been applied to the field of rheumatology [10], but its use in arthroplasty

48 is increasing; a number of arthroplasty studies have derived PASS values [11–14] or have used PASS

49 achievement as their primary outcome [15–17].

50 Although there are several different methods that may be used to calculate the PASS, one of the most

51 common methods is the 80% specificity technique using Receiver Operating Curve (ROC) analysis,

52 which implies that 80% of patients who report being dissatisfied with their intervention will fail to

53 achieve a score higher than the PASS [18,19]. By dichotomizing a PROM score based on patient-reported

54 satisfaction, the PASS ensures that subjective patient perspectives are incorporated into the determination

55 of a successful outcome. The PASS threshold may be used as: (1) a clear and clinically-relevant treatment
 4

56 benchmark for providers, (2) a patient-centered outcome for comparative effectiveness research, (3) as a

57 tool that helps physicians contextualize their patient’s post-operative symptom state.

58 The Harris Hip Score (HHS), originally developed by Dr. William Harris in 1969, is one of the oldest and

59 most commonly used PROMs in orthopedic research and clinical care [20,21]. The HHS has been shown

60 to have good construct validity and to be responsive to surgical interventions such as THA [22]. While the

61 original HHS includes both patient- and surgeon-administered portions, an exclusively patient-

62 administered version of the HHS, also known as the modified HHS (mHHS), has since been developed.

63 The mHHS has been shown to have excellent concordance with the original HHS [23,24].

64 Despite the ubiquity of the HHS in THA research, PASS values for the HHS after THA are currently

65 lacking. Only one study has proposed a PASS threshold of 74 points for the HHS after surgical treatment

66 for femoroacetabular impingement (FAI) [25]. While this threshold has been used by studies considering

67 the outcomes of THA, this is not appropriate as a PASS threshold for a given PROM is context-specific.

68 The aim of this study was to define PASS thresholds for the HHS at 3 months, 1, 3, 5, and 7 years after

69 THA using data from a prospective, multicenter study of a modern, well-performing implant system. The

70 secondary aim was to perform an internal validation as well as an external validation of these thresholds

71 using data from an institutional arthroplasty registry.

72
 5

73 Methods

74 Patients and Data Collection

75 Patients were sourced from a prospective, international, clinical, longitudinal follow-up study that was

76 established in 2007 for the primary purpose of evaluating the outcomes of patients treated with THA

77 using highly-crosslinked, vitamin-E infused polyethylene liners (VEPE). A total of 976 patients from 17

78 centers in 8 countries were enrolled. Study sites were in the United States (6 centers, 398 patients),

79 Denmark (3 centers, 229 patients), Sweden (2 centers, 135 patients), Spain (1 center, 58 patients),

80 Norway (1 center, 51 patients), Mexico (1 center, 50 patients), the Netherlands (1 center, 25 patients), and

81 the United Kingdom (2 centers, 30 patients). The participating institutions in this study represent a diverse

82 group of medical centers – some are private practices, others are public hospitals, and others are academic

83 centers. Given this wide range of practice settings, this study was well-suited to generate generalizable

84 results. All centers were staffed by fellowship trained arthroplasty surgeons who were interested in

85 conducting research and in participating in a multicenter investigation of VEPE, sponsored by

86 ZimmerBiomet (Warsaw, IN).

87 To be enrolled in this multicenter study, patients had to be diagnosed with primary hip osteoarthritis, be

88 within 20 to 75 years of age, and demonstrate the potential ability to return for follow-up. Patients with

89 previous infection, osteoporosis, metabolic disorders that may impair bone formation, or any other major

90 medical complication that could limit their ability to return for follow-up for ten years were excluded

91 from the study.

92 Patients received either a porous titanium coated (Regenerex®, ZimmerBiomet) or a plasma sprayed

93 (Ringloc®, ZimmerBiomet) cementless acetabular shell paired with either a highly-crosslinked (130kGy)

94 VEPE (E1®, ZimmerBiomet) or a moderately (50kGy) crosslinked and mechanically annealed

95 polyethylene (ArComXL®, ZimmerBiomet) liner. All patients also received either a 32mm or 36mm

96 femoral head and a cementless Biomet femoral stem of the surgeon’s choice. Radiostereometric analysis,
 6

97 clinical, and registry studies have shown that these implants have comparable wear properties, fixation,

98 PROMs, and survivorship to other contemporary, widely-used implants [26,27].

99 Baseline and follow-up data were anonymized by each center and uploaded to an Academic Contract

100 Research Organization (ACRO) via a secure, web-based portal for independent analysis. The study

101 protocol was approved by the institutional review board (IRB) at each respective study site and at the

102 ACRO, and all patients signed informed consent prior to study start.

103 PROMs

104 Patients were followed with plain radiographs and a battery of PROMs preoperatively and at several

105 postoperative follow-up intervals (3 months, 1, 3, 5, and 7 years). PROMs included the mHHS and a

106 numerical rating scale (NRS) for satisfaction. PROMs were administered on paper in the local language

107 of each study site. Translation was done using the forward-and-back technique, followed by multiple

108 independent translators until consensus was reached [28].

109 Satisfaction from the NRS served as the anchor: “How satisfied are you with the result of your most

110 recent hip treatment?” Possible responses range from 0.0 (very satisfied) to 2.5 (satisfied) to 5.0

111 (moderately satisfied) to 7.5 (doubtful) to 10.0 (dissatisfied), with 0.5-point increments. Based on the

112 labels of the administered NRS, the distribution of patient responses, and previous literature [29], patients

113 reporting satisfaction at or below 2.5 were considered satisfied and all others were considered dissatisfied.

114 To assess the reliability of this definition of satisfaction, alternate satisfaction cut-offs of 1.5 and 3.5 were

115 also considered as a sensitivity analyses (see Appendix).

116 Statistical Analysis

117 Assessment of Satisfaction Anchor Validity

118 To investigate the validity of the satisfaction anchor, a Spearman’s rank correlation test was performed

119 between the HHS and the anchor at each follow-up interval. Receiver operating characteristic (ROC)
 7

120 analysis was also used to assess the HHS’s ability to discriminate between satisfied and dissatisfied

121 patients [30].

122 Derivation of PASS Thresholds

123 PASS thresholds for the HHS were calculated at the 3-month, 1-, 3-, 5-, and 7-year intervals using three

124 different methods. The primary PASS derivation method was the anchor-based, 80% specificity method,

125 which has been shown to be the most reliable PASS derivation method [19,31]. In this method, the PASS

126 is deemed the point on the HHS below which 80% of dissatisfied patients are correctly identified. 95%

127 confidence intervals for each PASS were generated using 1000 non-parametric bootstrapped samples

128 [32].

129 The remaining two statistical methods were performed as sensitivity analyses. The first method used for

130 sensitivity analysis was the Youden method [33–35]. In this method, the point on the ROC curve

131 corresponding to the highest combination of sensitivity and specificity is used as the PASS. The 75th

132 percentile approach method was the second sensitivity analysis [36]. This method indicates that 75% of

133 patients classified as reporting satisfaction with their THA outcome report a score equal to or larger than

134 the PASS.

135 Internal Validation

136 A bootstrapping method was used to internally validate the primary PASS thresholds. Bootstrapping is a

137 statistical method by which new samples are generated from an original data set through random

138 sampling with replacement. By resampling the original data set, the sampling distribution of the statistics

139 being measured can be estimated [37]. As such, a bootstrapped sample of equal size to the original data

140 set was created by drawing samples with replacement from the original data set [38]. A bootstrapped

141 sample was generated for each study interval. Each PASS threshold was then applied to its respective

142 bootstrap sample and evaluated on its ability to predict satisfaction based on area under the curve (AUC),

143 sensitivity, and specificity using ROC analysis.

 8

144 External Validation

145 External validation was also conducted on patients sourced from the institutional registry of a tertiary

146 academic medical center to assess whether the PASS values derived apply to a more diverse cohort of

147 patients treated by a variety of providers with different implants. This registry is an associate member of

148 the International Society of Arthroplasty Registries due to its high data quality and capture rate. It is also

149 a Level IV registry, which captures PROM data for patients undergoing THA [39]. The mHHS and the

150 same NRS Satisfaction questionnaire as described above are collected in this registry. The mHHS was

151 administered since the registry’s conception in 1969, but the satisfaction NRS was administered

152 beginning in 2007. Therefore, patients undergoing primary THA between January 1, 2007 and April 1,

153 2016 at the institution were considered for inclusion in the external validation analysis. During this time

154 4544 primary THAs were captured in the registry. Patients with PROMs at 3 months, 1, 3, 5, and 7 years

155 were analyzed. A total of 1197 patients were available for external validation at 3-months, 753 at 1-year,

156 557 at 3-years, 375 at 5-years, and 479 at 7-years. A Little’s MCAR test was performed to assess whether

157 the missing data for this external validation cohort was missing completely at random [40]. PASS

158 thresholds for the HHS were applied to the registry sample and evaluated on their ability to predict

159 satisfaction based on the same ROC analysis conducted for the internal validation.

160
 9

161 Results

162 Study Cohort and Follow-up

163 A total of 910 (94%) patients in the multicenter follow-up study completed 3-month follow-up, 800

164 (86%) completed the 1-year, 742 (82%) completed the 3-year, 701 (81%) completed the 5-year, and 627

165 (75%) completed the 7-year (Figure 1). Demographic data for the cohorts at each follow-up interval is

166 presented in Table 1.

167 At 3-months 783 (86%) patients were satisfied with their THA, at 1-year 696 (87%) were satisfied, at 3-

168 years 646 (87%) were satisfied, at 5-years 596 (85%) were satisfied, and at 7-years 539 (86%) were

169 satisfied (Figure 2). Median HHS scores were: 74 (interquartile range (IQR): 58-87) at 3-months, 95

170 (IQR: 86-100) at 1-year, 96 (IQR: 88-100) at 3-years, 96 (IQR: 87-100) at 5-years, and 95 (IQR: 86-100)

171 at 7-years (Figure 3).

172 Correlation between the HHS and the Satisfaction Anchor

173 The Spearman’s correlation coefficients between the HHS and the hip-specific NRS satisfaction question

174 were higher than 0.40 (p<0.001) at all time points (Table 2). When considering the dichotomized NRS

175 satisfaction (NRS satisfaction ≤ 2.5), HHS scores were significantly lower in the dissatisfied anchor group

176 at all time points (all p<0.001) (Figure 4). The HHS was either an acceptable or excellent predictor of

177 satisfaction (NRS satisfaction ≤ 2.5) as determined by ROC analysis at all follow-up time points (Table

178 2). The same was true when the alternative 1.5-point and 3.5-point thresholds were used to dichotomize

179 the NRS satisfaction. However, the 2.5 threshold yielded the highest AUC at each timepoint, which

180 indicates that NRS satisfaction dichotomization at 2.5 provides the binary satisfaction measure most

181 closely associated with the HHS (Table 2, Appendix).

182 PASS Thresholds

183 Based on the primary PASS derivation method, PASS thresholds for the HHS were found to be 76 (95%

184 Confidence Interval (CI): 71-82) points at 3-months, 89 (95% CI: 86-94) points at 1-year, 93 (95% CI:
 10

185 90-97) points at 3-years, 94 (95% CI: 91-97) points at 5-years, and 93 (95% CI: 89-95) points at 7-years.

186 There was minimal variation in PASS threshold when comparing the three derivation methods (Table 3).

187 The PASS was significantly lower at the 3-month follow-up when compared to the subsequent follow-ups

188 (p<0.001). The confidence intervals for the HHS PASS overlap across 1-7-year study visits, indicating no

189 statistically significant change in PASS over this period of time, although a modest 4-point increase in

190 PASS value was noted between 1 and 3 years (Figure 5).

191 PASS Validation

192 Demographic data for the external validation cohort is presented in Table 4, which also shows that the

193 missing data for this cohort was found to be missing completely at random for all follow-up intervals

194 (p>0.179).

195 When applied to the bootstrapped internal validation cohorts as well as the external validation cohorts, all

196 PASS thresholds showed acceptable or excellent ability to predict satisfaction (AUC=0.73-80; p<0.001).

197 Each threshold maintained comparable specificity and sensitivity between the validation and original

198 cohorts (Tables 5 and 6).

199
 11

200 Discussion
201 The present analysis provides a novel method of PROM assessment for one of the oldest and most-widely

202 utilized PROMs in the field of arthroplasty, the HHS. The primary aim of the study was to present PASS

203 thresholds for the HHS at various follow-up intervals after THA. We found the thresholds to be 76 points

204 at 3-months, 89 points at 1-year, 93 points at 3-years, 94 points at 5-years, and 93 points at 7-years. These

205 thresholds proved valid when applied to internal and external validation samples.

206 The PASS can provide valuable insight into the interpretation of PROMs in both clinical and research

207 settings. In the clinic, physicians may use the PASS to help patients better understand their reported

208 PROM scores and, in turn, empower them during the preoperative decision-making process as well as

209 contextualize their reported pain and function during postoperative recovery. In research, PASS

210 thresholds not only allow for the easy identification of patients who consider their status satisfactory, but

211 also help researchers avoid the common pitfall of overemphasizing the importance of statistically

212 significant but not clinically relevant differences in PROMs.

213 However, the limitation of the PASS concept should be noted. First, there may be a range of adjacent cut

214 points in a PROM scale that have similar abilities to predict satisfaction. There are therefore a range of

215 plausible PASS values. This concept is clearly demonstrated by the 95% confidence intervals for each

216 PASS calculated in the study. Second, PASS calculations are tied to the definition of satisfaction used.

217 More stringent definitions of satisfaction will lead to higher PASS thresholds, and vice versa. This is

218 reflected by the differences in PASS thresholds generated between our primary definition of satisfaction

219 and our sensitivity analysis definition of satisfaction (see Appendix). The authors maintain that the

220 primary values presented provide a balanced definition of the HHS PASS, however, providers and

221 researchers may choose to utilize the more or less stringent PASS values presented in the Appendix

222 depending on their goals.

 12

223 The PASS values established by the present study indicate that patients’ appraisal of hip-related pain and

224 function is subject to change following THA, especially within the first year. At 3-months, most patients

225 are satisfied with a HHS score greater than 76 points. This is likely due to appropriate expectation

226 management by the providers during the rehabilitation period. However, by 1 year, and certainly between

227 3 and 7 years, patients expect, and are not likely to be satisfied by, scores lower than those of age-

228 matched population norms of approximately 91 points [41]. This finding is comparable with that of

229 another study, which determined the PASS for the Hip Dysfunction and Osteoarthritis Outcome Score

230 pain and function to be 91 and 88 points (out of 100), respectively, after THA [42].

231 It should be noted that there are patients who have a HHS well below the PASS but report being

232 extremely satisfied, as well as vice versa. These instances of HHS and satisfaction divergence highlight

233 the complex, multifactorial nature of patient satisfaction – different patients are satisfied by different

234 levels of pain and function. It also highlights both the importance of having the PASS as a benchmark and

235 outcome that establishes the symptom-state that THA patients are generally likely to associate with

236 success as well as the fact that PASS users should not use the PASS (or any PROM) alone to determine

237 the success of any individual case. While the PASS can provide context for the symptom state of an

238 individual patient in a standardized way, success for any given individual should be determined by a

239 holistic review of goals, expectations, symptom-state, and objective outcomes such as radiography.

240 Furthermore, future research would be well-served to investigate whether certain sub-cohorts may benefit

241 from having specific PASS values. One previous study did show that Oxford Hip Score (OHS) PASS

242 thresholds do vary based on preoperative OHS score [43]. However, it is possible that PASS values may

243 also vary by other baseline characteristics such as age, sex, amount of comorbidities, etc.. While the

244 present analysis provides general PASS thresholds for the HHS, which are most useful when evaluating

245 and comparing performance across cohorts, more specific thresholds derived by future studies may

246 elucidate would be more accurate in evaluating scores of individuals.

 13

247 Prior studies assessing the HHS following THA have either analyzed it as a continuous variable, utilized

248 the PASS value of 74 points derived from a cohort of patients treated for FAI [25], or used the

249 categorizations proposed by Dr. Harris in his original paper from 1969 [20]. Each of these approaches is

250 problematic. First, the analysis of the HHS as a continuous measure is confounded by the high ceiling

251 effect of this PROM [44]. The ceiling effect indicates that patients scoring 100 points likely have better

252 hip pain and function than the PROM can measure; in other words, the HHS has trouble distinguishing

253 subtle differences between exceptional scores. This ceiling effect results in averages being lower than

254 they should be when the HHS is considered as a continuous variable. Dichotomizing the HHS at either 74

255 points as proposed by the FAI study or at 80 points, which is the category proposed by Dr. Harris as being

256 “excellent”, underestimates the pain and function levels that patients consider to be acceptable following

257 THA. The PASS values proposed by the present study are higher than the previously utilized cut-points,

258 yet they are lower than the HHS ceiling of 100 points. This indicates that while the HHS is not suitable to

259 distinguish between various levels of exceptional scores, it may be used to determine whether a patient

260 deems the outcome of THA to be acceptable or not.

261 This study had some limitations that should be considered when interpreting the results. First, our anchor

262 question was a non-dichotomous NRS, requiring the use of a cutoff to define the satisfied and non-

263 satisfied anchoring groups. Different cutoff values on the NRS Satisfaction anchor could lead to different

264 PASS thresholds. This is highlighted by our sensitivity analyses, in which satisfaction cutoffs of 1.5 and

265 3.5, resulted in slightly higher and lower PASS thresholds, respectively (Appendix Figure 1). Moreover,

266 though the NRS Satisfaction used for this study had not undergone previous formal validation, it has been

267 used by a previous study to derive the PASS in an arthroplasty setting [42]. As such, the authors have

268 deemed it appropriate to also apply the NRS as a satisfaction anchor in this analysis. In addition, the study

269 cohort considered only patients with a single THA system, which may limit the generalizability of the

270 findings. Finally, the multicenter study, which provided the data for the present analysis, had inclusion

271 criteria that may have caused our patient population to be more likely to experience good outcomes;
 14

272 namely, patients with previous infection, osteoporosis, metabolic disorders that may impair bone

273 formation, or any other major medical complication that could limit their ability to return for follow-up

274 for ten years, were excluded.

275 Conclusions

276 The current study is the first to present validated PASS thresholds for the HHS at 3-months, 1-, 3-, 5-, and

277 7-years following THA. The HHS PASS thresholds, representing the HHS values beyond which the

278 average THA patient is likely to be satisfied, were found to be as follows: 76 points at 3 months, 89 at 1

279 year, 93 at 3 years, 94 at 5 years, and 93 at 7 years. We found that PASS thresholds are time-dependent,

280 especially in early follow-up period; starting at 1-year, patients expect and are not likely to be satisfied by

281 HHS values less than age-matched population norms. As our study considered a large cohort of patients

282 from a multicenter study spanning 8 countries, our results are generalizable to a broad range of patient

283 demographics, cultures, and hospital settings. These findings will serve as a useful reference for future

284 THA outcome studies and help elucidate which outcomes patients consider satisfactory. Additionally, our

285 cutoff values can serve as benchmarks for surgeons to assess and inform patients on their recovery

286 progress.

287
 15

288 References
289 [1] Rolfson O, Wissig S, van Maasakkers L, Stowell C, Ackerman I, Ayers D, et al. Defining an

290 International Standard Set of Outcome Measures for Patients With Hip or Knee Osteoarthritis:

291 Consensus of the International Consortium for Health Outcomes Measurement Hip and Knee

292 Osteoarthritis Working Group. Arthritis Care Res 2016;68:1631–9. doi:10.1002/acr.22868.

293 [2] Deshpande PR, Sudeepthi BL, Rajan S, Abdul Nazir C. Patient-reported outcomes: A new era in

294 clinical research. Perspect Clin Res 2011;2:137–44. doi:10.4103/2229-3485.86879.

295 [3] Chin R, Lee BY. Principles and Practice of Clinical Trial Medicine. 2008. doi:10.1016/B978-0-12-

296 373695-6.00022-3.

297 [4] American Academy of Orthopaedic Surgeons. AAOS Registry Program Patient-Reported

298 Outcome Measures Guide. 2018.

299 [5] McGlothlin AE, Lewis RJ. Minimal clinically important difference: Defining what really matters

300 to patients. JAMA - J Am Med Assoc 2014. doi:10.1001/jama.2014.13128.

301 [6] Sayers A, Wylde V, Lenguerrand E, Gooberman-Hill R, Dawson J, Beard D, et al. A unified

302 multi-level model approach to assessing patient responsiveness including; Return to normal,

303 minimally important differences and minimal clinically important improvement for patient

304 reported outcome measures. BMJ Open 2017;7. doi:10.1136/bmjopen-2016-014041.

305 [7] Tubach F, Ravaud P, Baron G, Falissard B, Logeart I, Bellamy N, et al. Evaluation of clinically

306 relevant states in patient reported outcomes in knee and hip osteoarthritis: The patient acceptable

307 symptom state. Ann Rheum Dis 2005. doi:10.1136/ard.2004.023028.

308 [8] Maltenfort M, Díaz-Ledezma C. Statistics In Brief: Minimum Clinically Important Difference—

309 Availability of Reliable Estimates. Clin Orthop Relat Res 2017. doi:10.1007/s11999-016-5204-6.

310 [9] Kvien TK, Heiberg T, Hagen KB. Minimal clinically important improvement/difference

311 (MCII/MCID) and patient acceptable symptom state (PASS): What do these concepts mean? Ann

312 Rheum Dis 2007;66:40–1. doi:10.1136/ard.2007.079798.

 16

313 [10] Dougados M. It’s good to feel better but it’s better to feel good. J Rheumatol 2005.

314 [11] Naal FD, Impellizzeri FM, Lenze U, Wellauer V, von Eisenhart-Rothe R, Leunig M. Clinical

315 improvement and satisfaction after total joint replacement: a prospective 12-month evaluation on

316 the patients’ perspective. Qual Life Res 2015. doi:10.1007/s11136-015-1042-3.

317 [12] Paulsen A, Roos EM, Pedersen AB, Overgaard S. Minimal clinically important improvement

318 (MCII) and patient-acceptable symptom state (PASS) in total hip arthroplasty (THA) patients 1

319 year postoperatively. Acta Orthop 2014;85:39–48. doi:10.3109/17453674.2013.867782.

320 [13] Anderson MR, Baumhauer JF, DiGiovanni BF, Flemister S, Ketz JP, Oh I, et al. Determining

321 Success or Failure After Foot and Ankle Surgery Using Patient Acceptable Symptom State

322 (PASS) and Patient Reported Outcome Information System (PROMIS). Foot Ankle Int 2018.

323 doi:10.1177/1071100718769666.

324 [14] Connelly JW, Galea VP, Rojanasopondist P, Matuszak SJ, Ingelsrud LH, Nielsen CS, et al. Patient

325 Acceptable Symptom State at 1 and 3 Years After Total Knee Arthroplasty. J Bone Jt Surg

326 2019;101:995–1003. doi:10.2106/JBJS.18.00233.

327 [15] Scholes C, Ebrahimi M, Ektas N, Ireland J. Efficacy of a Second-Generation Rotating Bearing

328 Tibial Platform in Total Knee Arthroplasty: A Prospective Observational Cohort Study with

329 Registry Analysis. J Knee Surg 2019. doi:10.1055/s-0039-1678679.

330 [16] Judge A, Arden NK, Cooper C, Kassim Javaid M, Carr AJ, Field RE, et al. Predictors of outcomes

331 of total knee replacement surgery. Rheumatology 2012;51:1804–13.

332 doi:10.1093/rheumatology/kes075.

333 [17] Vogl M, Wilkesmann R, Lausmann C, Hunger M, Plötz W. The impact of preoperative patient

334 characteristics on health states after total hip replacement and related satisfaction thresholds: a

335 cohort study. Health Qual Life Outcomes 2014;12:108. doi:10.1186/s12955-014-0108-1.

336 [18] Aletaha D, F. Perception of Improvement in Patients With Rheumatoid Arthritis Varies With

337 Disease Activity Levels at Baseline 2013;6:313–20. doi:10.1021/nn300902w.Release.

338 [19] Kvamme MK, Kristiansen IS, Lie E, Kvien TK. Identification of cutpoints for acceptable health
 17

339 status and important improvement in patient-reported outcomes, in rheumatoid arthritis, psoriatic

340 arthritis, and ankylosing spondylitis. J Rheumatol 2010;37:26–31. doi:10.3899/jrheum.090449.

341 [20] Harris WH. Traumatic arthritis of the hip after dislocation and acetabular fractures: treatment by

342 mold arthroplasty. J Bone Jt Surg 1969;51-A:737–55.

343 [21] Lovelock TM, Broughton NS, Williams CM. The Popularity of Outcome Measures for Hip and

344 Knee Arthroplasties. J Arthroplasty 2018. doi:10.1016/j.arth.2017.08.024.

345 [22] Söderman P, Malchau H. Is the Harris Hip Score System Useful to Study the Outcome of Total

346 Hip Replacement? Clin Orthop Relat Res 2001. doi:10.1097/00003086-200103000-00022.

347 [23] Shervin N, Dorrwachter JM, Bragdon CR, Malchau H. Comparison of Physician and Patient

348 Administered Harris Hip Score. J Arthroplast 2009;24:e75. doi:10.1016/j.arth.2008.11.089.

349 [24] Mahomed NN, Arndt DC, McGrory BJ, Harris WH. The Harris hip score: Comparison of patient

350 self-report with surgeon assessment. J Arthroplasty 2001. doi:10.1054/arth.2001.23716.

351 [25] Chahal J, Van Thiel GS, Mather RC, Lee S, Song SH, Davis AM, et al. The Patient Acceptable

352 Symptomatic State for the Modified Harris Hip Score and Hip Outcome Score among Patients

353 Undergoing Surgical Treatment for Femoroacetabular Impingement. Am J Sports Med 2015.

354 doi:10.1177/0363546515587739.

355 [26] Galea VP, Connelly JW, Shareghi B, Kärrholm J, Sköldenberg O, Salemyr M, et al. Evaluation of

356 in vivo wear of vitamin E-diffused highly crosslinked polyethylene at five years. Bone Joint J

357 2018;100-B:1592–9. doi:10.1302/0301-620X.100B12.BJJ-2018-0371.R1.

358 [27] Nebergall AK, Rolfson O, Rubash HE, Malchau H, Troelsen A, Greene ME. Stable Fixation of a

359 Cementless, Proximally Coated, Double Wedged, Double Tapered Femoral Stem in Total Hip

360 Arthroplasty: A 5-Year Radiostereometric Analysis. J Arthroplasty 2016.

361 doi:10.1016/j.arth.2015.11.036.

362 [28] Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life

363 measures: Literature review and proposed guidelines. J Clin Epidemiol 1993. doi:10.1016/0895-

364 4356(93)90142-N.
 18

365 [29] Connelly JW, Galea VP, Rojanasopondist P, Matuszak SJ, Ingelsrud LH, Nielsen CS, et al. Patient

366 Acceptable Symptom State at One and Three Years After Total Knee Arthroplasty: Thresholds for

367 the Knee Injury and Osteoarthritis Outcomes Score (KOOS). J Bone Jt Surg Am 2018.

368 [30] Hosmer DW, Lemeshow S. Applied Logistic Regression Second Edition. 2000.

369 doi:10.1002/0471722146.

370 [31] Aletaha D, Funovits J, Ward MM, Smolen JS, Kvien TK. Perception of improvement in patients

371 with rheumatoid arthritis varies with disease activity levels at baseline. Arthritis Care Res 2009.

372 doi:10.1002/art.24282.

373 [32] Campbell MK. Bootstrapping: estimating confidence intervals for cost-effectiveness ratios. QJM

374 1999. doi:10.1093/qjmed/92.3.177.

375 [33] Youden WJ. Index for rating diagnostic tests. Cancer 1950;3:32–5. doi:10.1002/1097-

376 0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3.

377 [34] Heiberg TT, Kvien TK, Mowinckel P, Aletaha D, Smolen JS, Hagen KB. Identification of disease

378 activity and health status cut-off points for the symptom state acceptable to patients with

379 rheumatoid arthritis. Ann Rheum Dis 2008. doi:10.1136/ard.2007.077503.

380 [35] Maksymowych WP, Richardson R, Mallon C, Van Der Heijde D, Boonen A. Evaluation and

381 validation of the Patient Acceptable Symptom State (PASS) in patients with ankylosing

382 spondylitis. Arthritis Care Res 2007. doi:10.1002/art.22469.

383 [36] Tubach F, Ravaud P, Baron G, Falissard B, Logeart I, Bellamy N, et al. Evaluation of clinically

384 relevant changes in patient reported outcomes in knee and hip osteoarthritis: The minimal

385 clinically important improvement. Ann Rheum Dis 2005. doi:10.1136/ard.2004.022905.

386 [37] Boos DD, Brownie C. Bootstrap Methods for Testing Homogeneity of Variances. Technometrics

387 1989;31:69–82. doi:10.2307/1270366.

388 [38] Steyerberg EW, Harrell FE, Borsboom GJJM, Eijkemans MJC, Vergouwe Y, Habbema JDF.

389 Internal validation of predictive models: Efficiency of some procedures for logistic regression

390 analysis. J Clin Epidemiol 2001. doi:10.1016/S0895-4356(01)00341-9.

 19

391 [39] Barr CJ, Barbalace RJ, Wessinger SJ, Bragdon CR, Kwon YM, Malchau H. Validation of a

392 Hospital-Based Joint Registry. Quantification of Errors and Maximizing Utility. J Arthroplasty

393 2012;27:1766–71. doi:10.1016/j.arth.2012.04.028.

394 [40] Little RJA. A test of missing completely at random for multivariate data with missing values. J

395 Am Stat Assoc 1988. doi:10.1080/01621459.1988.10478722.

396 [41] Lieberman JR, Hawker G, Wright JG. Hip function in patients &gt;55 years old. J Arthroplasty

397 2001;16:901–4. doi:10.1054/arth.2001.26593.

398 [42] Paulsen A, Roos EM, Pedersen AB, Overgaard S. Minimal clinically important improvement

399 (MCII) and patient-acceptable symptom state (PASS) in total hip arthroplasty (THA) patients 1

400 year postoperatively. Acta Orthop 2014. doi:10.3109/17453674.2013.867782.

401 [43] GJ H, AR M, Wilkinson B, Mathew J, TB W, ID P, et al. The early radiological results of the

402 uncemented Oxford medial compartment knee replacement. J Bone Jt Surgery, Br Vol

403 2012;94:334–8.

404 [44] Wamper KE, Sierevelt IN, Poolman RW, Bhandari M, Haverkamp D. The Harris hip score: Do

405 ceiling effects limit its usefulness in orthopedics? A systematic review. Acta Orthop 2010.

406 doi:10.3109/17453674.2010.537808.

407

408
Source of Funding

We received no funding for this analysis. Data collection for the cohort of patients included in the
multicenter follow-up study has been supported with funding from ZimmerBiomet. Our institutional
registry is supported by funds from the Orthopaedics Department at Massachusetts General Hospital.
Table 1. Cohort demographics at the follow-up intervals.
Variable Postoperative 1-year 3-year 5-year 7-year p-value
N 910 800 742 701 627 -
Age (years)* 61.5 (9.0) 61.4 (8.9) 61.6 (8.6) 61.4 (8.5) 61.6 (8.8) 0.717
Female (vs
440 (48.4) 382 (47.8) 358 (48.2) 349 (49.8) 314 (50.1) 0.216
Male) Sex†
BMI* 28.6 (5.1) 28.4 (4.9) 28.4 (5.1) 28.1 (4.9) 28.0 (5.1) 0.599
* Values reported as mean (standard deviation). Comparisons by Student’s T-test.
† Values reported as N (%). Comparisons by Chi2 Test.
Table 2. Spearman’s correlation and receiver operator characteristics analyses of the patient-
reported Harris Hip Score and the satisfaction anchor question*.
Spearman’s Rank Correlation Receiver Operator Characteristic Analysis
Interval
Coefficient p-value AUC 95% CI p-value
3-month 0.49 <0.001 0.78 0.74-0.84 <0.001
1-year 0.50 <0.001 0.85 0.81-0.91 <0.001
3-year 0.51 <0.001 0.85 0.79-0.88 <0.001
5-year 0.52 <0.001 0.86 0.82-0.91 <0.001
7-year 0.50 <0.001 0.84 0.79-0.87 <0.001
* Satisfaction score from the numerical rating scale, dichotomized at 2.5
AUC: Area Under the Curve; CI: Confidence Interval
Table 3. Patient Acceptable Symptom State thresholds at each follow-up interval, derived using
the three statistical methods, and their respective sensitivity and specificity.
80% Specificity Method* Youden Method 75th Percentile Method
Interval Sensitivity, Sensitivity, Sensitivity,
Threshold Threshold Threshold
Specificity Specificity Specificity
3-month 76 0.63, 0.80 73 0.64, 0.77 72 0.68, 0.72
1-year 89 0.75, 0.80 88 0.80, 0.68 89 0.75, 0.80
3-year 93 0.70, 0.81 91 0.75, 0.76 91 0.75, 0.76
5-year 94 0.72, 0.80 94 0.72, 0.80 89 0.76, 0.72
7-year 93 0.70, 0.82 90 0.74, 0.72 89 0.77, 0.69
* Indicates primary PASS derivation method.
PASS: Patient Acceptable Symptom State.
Table 4. External validation cohort demographics at the follow-up intervals.
Variable Postoperative 1-year 3-year 5-year 7-year
N (%)* 1197 (26.3) 753 (16.6) 557 (14.0) 375 (12.5) 479 (26.4)
Little’s MCAR Test 0.198 0.695 0.649 0.179 0.984
Age (years)† 64.5 (9.9) 63.9 (9.9) 64.7 (10.9) 64.5 (10.1) 63.8 (11.0)
Female (vs Male) Sex†† 587 (49.0) 364 (48.3) 272 (48.8) 183 (48.8) 240 (50.1)
BMI† 28.9 (7.7) 28.9 (7.8) 28.6 (6.9) 28.5 (6.5) 28.8 (6.8)
* Percent given as percent of eligible as not all patients had the opportunity to complete follow-up
PROMs due to when they were operated (i.e. patients operated in 2015 would not have been able to
complete 7-year PROMs).
† Values reported as mean (standard deviation). Comparisons by Student’s T-test.
†† Values reported as N (%). Comparisons by Chi2 Test.
Table 5. Internal validation of the primary Patient Acceptable Symptom State threshold values
at each follow-up interval using bootstrapped validation cohorts.
Interval AUC 95% CI p-value Sensitivity Specificity
3-month 0.75 0.71-0.79 <0.001 0.70 0.80
1-year 0.80 0.75-0.86 <0.001 0.79 0.79
3-year 0.77 0.73-0.82 <0.001 0.77 0.80
5-year 0.76 0.72-0.80 <0.001 0.69 0.80
7-year 0.78 0.75-0.81 <0.001 0.70 0.79
AUC: Area Under the Curve; CI: Confidence Interval
Table 6. External validation of the primary Patient Acceptable Symptom State threshold values
at each follow-up interval using cohorts from an institutional arthroplasty registry.
Interval AUC 95% CI p-value Sensitivity Specificity
3-month 0.77 0.73-0.82 <0.001 0.72 0.82
1-year 0.79 0.76-0.85 <0.001 0.74 0.84
3-year 0.78 0.74-0.83 <0.001 0.72 0.87
5-year 0.73 0.71-0.77 <0.001 0.70 0.79
7-year 0.73 0.70-0.77 <0.001 0.69 0.85
AUC: Area Under the Curve; CI: Confidence Interval
Figure Legends
Figure 1. Flowchart of patient loss-to-follow-up throughout the study intervals.
Figure 2. Distribution of responses to the numerical rating scale for satisfaction at each study interval.
Patients who reported satisfaction scores ≤ 2.5 (red vertical line) were considered satisfied for anchor-
based PASS derivations.
Figure 3. For the whole cohort, median Harris Hip Score (line) and percent of patients reporting
satisfaction scores ≤ 2.5 (bars).
Figure 4. Box plots depicting differences in patient-reported Harris Hip Scores (mHHS) between the
satisfied and dissatisfied patients at each study interval.
Figure 5. Patient Acceptable Symptom State threshold values for the patient-reported Harris Hip Score at
each study interval. Values displayed were derived using the primary method (80% specificity method).
Error bars represent 95% confidence intervals, calculated using 100 non-parametric bootstrap
replications.
Appendix Figure 1. Patient Acceptable Symptom State threshold values for the patient-reported Harris
Hip Score at each study interval. Different bar series represent the different cut-offs used for the
satisfaction numerical rating scale used as the anchor. Values displayed were derived using the primary
method (80% specificity method).
Appendix
Table 1. Receiver operator characteristics analyses of the patient-
reported Harris Hip Score and the satisfaction anchor question*.
Interval AUC 95% CI p-value
3-month 0.75 0.70-0.79 <0.001
1-year 0.81 0.76-0.87 <0.001
3-year 0.82 0.77-0.89 <0.001
5-year 0.76 0.70-0.83 <0.001
7-year 0.79 0.71-0.86 <0.001
* Satisfaction score from the numerical rating scale, dichotomized at 3.5
AUC: Area Under the Curve; CI: Confidence Interval
Appendix
Table 2. Patient Acceptable Symptom State thresholds at each follow-up interval, derived using
the three statistical methods, and their respective sensitivity and specificity. In this table, the
satisfaction anchor used is the satisfaction score from the numerical rating scale, dichotomized at
3.5.
80% Specificity Method* Youden Method 75th Percentile Method
Interval Sensitivity, Sensitivity, Sensitivity,
Threshold Threshold Threshold
Specificity Specificity Specificity
3-month 74 0.61, 0.81 72 0.64, 0.77 62 0.75, 0.69
1-year 88 0.66, 0.81 84 0.77, 0.65 88 0.66, 0.81
3-year 91 0.66, 0.82 88 0.80, 0.75 89 0.76, 0.66
5-year 93 0.60, 0.81 88 0.75, 0.64 88 0.75, 0.64
7-year 91 0.68, 0.80 87 0.80, 0.68 88 0.74, 0.69
* Indicates primary PASS derivation method.
PASS: Patient Acceptable Symptom State.
Appendix
Table 3. Receiver operator characteristics analyses of the patient-
reported Harris Hip Score and the satisfaction anchor question*.
Interval AUC 95% CI p-value
3-month 0.74 0.69-0.78 <0.001
1-year 0.82 0.77-0.89 <0.001
3-year 0.80 0.75-0.84 <0.001
5-year 0.75 0.71-0.80 <0.001
7-year 0.77 0.71-0.82 <0.001
* Satisfaction score from the numerical rating scale, dichotomized at 1.5
AUC: Area Under the Curve; CI: Confidence Interval
Appendix
Table 4. Patient Acceptable Symptom State thresholds at each follow-up interval, derived using
the three statistical methods, and their respective sensitivity and specificity. In this table, the
satisfaction anchor used is the satisfaction score from the numerical rating scale, dichotomized at
1.5.
80% Specificity Method* Youden Method 75th Percentile Method
Interval Sensitivity, Sensitivity, Sensitivity,
Threshold Threshold Threshold
Specificity Specificity Specificity
3-month 79 0.58, 0.80 77 0.64, 0.76 75 0.75, 0.69
1-year 92 0.68, 0.80 91 0.75, 0.70 91 0.75, 0.70
3-year 94 0.66, 0.81 92 0.77, 0.74 92 0.77, 0.74
5-year 95 0.67, 0.80 95 0.67, 0.80 92 0.75, 0.64
7-year 95 0.67, 0.81 93 0.74, 0.70 92 0.78, 0.69
* Indicates primary PASS derivation method.
PASS: Patient Acceptable Symptom State.