See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/326714049

Development and evaluation of an obstetric quality-of-recovery score

(ObsQoR-11) after elective Caesarean delivery

Article  in  BJA British Journal of Anaesthesia · January 2019

DOI: 10.1016/j.bja.2018.06.011

CITATIONS READS

32 1,013

8 authors, including:

Sarah Ciechanowicz Emily J Robson

University College London Hospitals NHS Foundation Trust 4 PUBLICATIONS   41 CITATIONS   
16 PUBLICATIONS   259 CITATIONS   
SEE PROFILE
SEE PROFILE

Maria Chazapis Brendan Carvalho

University College London Hospitals NHS Foundation Trust Stanford University
10 PUBLICATIONS   140 CITATIONS    349 PUBLICATIONS   6,192 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Peri-operative morbidity View project

Obstetric Case Reports View project

All content following this page was uploaded by Pervez Sultan on 14 August 2018.

The user has requested enhancement of the downloaded file.

 British Journal of Anaesthesia, ▪ (▪): 1e10 (2018)

doi: 10.1016/j.bja.2018.06.011
Advance Access Publication Date: xxx
Clinical Investigation

CLINICAL INVESTIGATION

Development and evaluation of an obstetric quality-

of-recovery score (ObsQoR-11) after elective
caesarean delivery
S. Ciechanowicz1,*, T. Setty1, E. Robson1, C. Sathasivam2, M. Chazapis1,
J. Dick1, B. Carvalho3 and P. Sultan1
1
Department of Anaesthesia and Perioperative Medicine, University College London Hospital, London,
UK, 2Royal Free Hospital, London, UK and 3Stanford University School of Medicine, Stanford, CA, USA

*Corresponding author. E-mail: s.ciechanowicz@nhs.net

Abstract
Background: Whilst validated quality-of-recovery (QoR) tools exist for general surgery, there is no specific obstetric
equivalent. We aimed to develop and evaluate a modified QoR score after elective caesarean delivery.
Methods: Twenty-two obstetric specific items were selected following review and modification of the QoR-40 survey by
16 experts and interviews with 50 stakeholders. Item selection was based on relevance to caesarean delivery and
endorsement by >66% of stakeholders. Items were tested on women pre-delivery, at 24 h, and 25 h post-delivery. An 11-
item obstetric-specific QoR score (ObsQoR-11) was created based on correlation with a numerical rating scale (NRS) of
global health status (r>0.20) at all time points. Reliability, responsiveness, acceptability, and feasibility were tested.
Results: One hundred and fifty-two women responded to the 22-item questionnaire pre-delivery (complete in 146), 100 at
24 h, and 10 at 25 h. The ObsQoR-11 correlated with the global health status NRS (r¼0.53; 95% confidence interval:
0.43e0.62; P<0.0001) and discriminated good vs poor recovery (NRS score 70 vs <70 mm) at 24 h. There was a negative
correlation between the ObsQoR-11 score at 24 h and hospital length of stay (r¼e0.39; P¼0.003). ObsQoR-11 was reliable
(internal consistency: 0.85; split-half 0.76; testeretest intra-class correlation coefficient ri>0.6 in 82% of items) and
responsive (Cohen effect size: 1.36; standardised response mean: 0.85). A longer 22-item ObsQoR had high (97%)
completion rates and short (median: 2 min) completion times.
Conclusions: The ObsQoR-11 provides a valid, reliable, and responsive global assessment of recovery after elective
caesarean delivery.

Keywords: Caesarean delivery; recovery; postoperative

Editorial decision: 25 June 2018; Accepted: 25 June 2018

© 2018 British Journal of Anaesthesia. Published by Elsevier Ltd. All rights reserved.
For Permissions, please email: permissions@elsevier.com

1
 2 - Ciechanowicz et al.
Editor’s key points
 A patient-centred global measure of quality of recovery
can be used to evaluate new treatments in periopera-
tive practice.
 A surgery-specific, in this case elective caesarean de-
livery, quality-of-recovery measurement tool may offer
additional value.
 Here, the 11-item ObsQoR-11, derived from the 40-item
QoR-40, was found to be valid, reliable, and responsive
to changes in health status.
 The ObsQoR-11 is thus a suitable outcome measure in
caesarean delivery.
Recovery after caesarean delivery (CD) is a complex process,
dependent on patient, surgical and anaesthetic characteris-
tics, and is also influenced by the occurrence of postoperative
complications. Postoperative outcome measures have tradi-
tionally focused on mortality and morbidity, which, although
important, do not describe the patient experience or quality of
recovery (QoR) from surgery.
The QoR-401 and QoR-15 surveys2 were developed to eval-
uate patient-reported recovery outcome measures after sur-
gery. These scoring tools accurately evaluate postoperative
recovery by measuring key domains, including: pain, physical
comfort, physical independence, psychological support, and
emotional state. They were however, developed and validated
in non-obstetric inpatients and day-case surgery pop-
ulations.2,3 Consequently, they contain recovery aspects not
relevant to CD (mostly performed under neuraxial anaes-
thesia) and do not include key items, such as ability to care for
the newborn child, that are synonymous with good post-
operative recovery after CD. To date, no validated scoring tools
exist evaluating postoperative recovery after CD.4,5 Functional
recovery outcome measures after CD generally focus on pain
scores as the single most important dimension of post-
operative recovery.6 Functional scales measuring recovery
domains beyond pain, including: the QoR-40 survey, rating
scales of global health or quality of life, and rehabilitation
scores, fail to include multiple aspects relevant to recovery in
the early postpartum period.7e9
This study aimed to develop and evaluate an obstetric-
specific QoR score. Specifically, we aimed to adapt the QoR-
40 to measure patient-centred dimensions of functional re-
covery relevant to the early postpartum period after CD and
test reliability, responsiveness, clinical acceptability, and
feasibility of this devised scoring tool.
Methods
We conducted a prospective observational study of term
women undergoing elective CD between October 2016 and July
2017 at a UK university hospital. Local research office desig-
nated the study as service evaluation, and it was therefore
exempt from ethical approval requirement, consistent with a
previous study performed at our institution.3
At our institution, women are instructed to fast for 6 h
before surgery and are encouraged to drink until 2 h before
surgery. Our institutional analgesic regimen includes: intra-
thecal diamorphine 300 mg administered via either single-shot
spinal or combined spinaleepidural, regular paracetamol 1 g,
and ibuprofen 400 mg four times daily after operation, and, as
required, oral morphine 20 mg 4 hourly on the day of surgery
and dihydrocodeine 30 mg 6 hourly from Day 1 after CD for
breakthrough pain. Women are also routinely prescribed i.v.
ondansetron 4e8 mg and cyclizine 50 mg 8 hourly as required
after surgery. Women are encouraged to mobilise 6 h after
spinal anaesthesia, and a trial without urinary catheter is
attempted 8 h after neuraxial drug administration.
We elected to utilise the QoR-40 scoring tool as a template
from which the Obstetric Quality of Recovery-11 score
(ObsQoR-11) was developed. Formulation of the ObsQoR-11
scoring tool occurred in five stages (Table 1; Fig. 1): (i) removal
of non-obstetric and non-neuraxial relevant items, resulting in
19 remaining items; (ii) expert evaluation and addition of 12
obstetric- and neuraxial-related items; (iii) stakeholder eval-
uation and ranking of the 31 items to select those identified as
most important to recovery after elective CD, and those items
with >2/3 stakeholder support led to a 22-item scale; (iv) the
ObsQoR-22 questionnaire was then tested against a numerical
rating scale (NRS) of global health status on term women
scheduled for elective CD preoperatively, at 24 h, and a subset
a further 1 h (i.e. 25 h) after surgery; and (v) items with a sig-
nificant correlation (r>0.2) with an NRS of global health status
were retained, resulting in an 11-item ObsQoR-11 question-
naire. We conducted further post hoc factor analysis on
ObsQoR-22 items as outlined below.
Stages (i) and (ii) occurred through interviews of 16 experts:
four obstetric anaesthetists (C.S., M.C., J.D., and P.S.), three
obstetricians, three midwives, three recovery nurses, and
three post-CD women (Fig. 1). The resultant 31 items were then
evaluated by a new group of 50 stakeholders: 13 term pregnant
women, 4 partners, 13 anaesthetists, 11 midwives, and 9 ob-
stetricians. The stakeholders were asked to select items
important for evaluating recovery after elective CD. During
this phase, the 31 stakeholder items were ranked according to
popularity. We also sought feedback to evaluate whether any
modifications were required to improve clarity of items. The
stakeholders were encouraged to add to the list of recovery
items if necessary. Additional items were considered further
in item list development if added by >1 stakeholder. Twenty-
two of the 31 items were chosen by >2/3 stakeholders result-
ing in a 22-item ObsQoR questionnaire, which was evaluated
further. The >2/3 threshold was chosen to allow inclusion of
more than 20 items, as advised by expert opinion (M.C.).
For stage (iv), women were approached on the morning of
their scheduled CD and invited to complete the 22-item
ObsQoR questionnaire before and after their operation. The
inclusion criteria were: women undergoing elective CD, 37
weeks gestational age, and surgery performed under either
neuraxial anaesthesia or general anaesthesia. The exclusion
criteria were: age <18 yr, patient refusal, and inability to read
or understand written English. Recruitment days corre-
sponded to investigator availability over the study period.
Baseline patient characteristic and clinical data were
collected at the time of enrolment for descriptive purposes.
Upon recruitment and before CD, women were asked by
investigators to complete the 22-item ObsQoR questionnaire,
rating each item on an 11-point numerical Likert scale
(0¼strongly negative; 10¼strongly positive; see Supplementary
Fig. S1). Women were also asked to rate their baseline status
utilising a global health NRS, represented as a 100 mm line and
ruler, marked at each end with anchors ‘worst imaginable
health state’ to ‘best imaginable health state’, and with ‘sad’ or
‘happy’ stylised representations of faces. They were then
asked to repeat the 22-item ObsQoR questionnaire and overall
global health NRS at 24 h after CD. A subset of women were
Table 1 Summary of ObsQoR-11 item development. Deleted item represented as ‘X’, after a review by obstetric/anaesthetic experts or <2/3 agreement amongst stakeholders. Five stages of
development: (i) removal of non-obstetric and non-neuraxial relevant items from QoR-40 to produce QoR-19 (Column 3), (ii) addition of obstetric- and neuraxial-related items to produce
QoR-31 (Column 5), (iii) stakeholder evaluation of QoR-31 items, (iv) items selected by >2/3 stakeholders (ObsQoR-22), and (v) post hoc analysis of ObsQoR-22 items with a significant cor-
relation to global health status NRS of r>0.2 resulting in a final ObsQoR-11 questionnaire. QoR, quality of recovery. *Feedback from stakeholders resulted in nausea and vomiting becoming
one variable (‘nausea or vomiting’) rather than two separate items

Domains QoR-40 Items removed (X) Obstetric items Draft 31 items Items removed Draft 22-item scale ObsQoR-11 scale
after expert review added after expert
review

Physical comfort Able to breathe Able to breathe Able to breathe Able to breathe Able to breathe
Good sleep Good sleep Good sleep X
Enjoy food X
Itchiness Itchiness Itchiness Itchiness
Feel rested Feel rested Feel rested X
Nausea Nausea Nausea Nausea or Nausea or Nausea or
Vomiting Vomiting Vomiting vomiting* vomiting vomiting
Dry retching X
Dry lips/mouth Dry lips/mouth X
Restless X
Shaking/twitching X
Shivering Shivering Shivering Shivering Shivering Shivering
Too cold X
Dizzy Dizzy Dizzy Dizzy Dizzy Dizzy
Emotional state General well-being X
In control In control In control In control In control In control
Comfortable Comfortable Comfortable Comfortable Comfortable Comfortable
Bad dreams X
Anxious Anxious Anxious Anxious Anxious
Angry X
Depressed Depressed Depressed Depressed Depressed

Development and evaluation of ObsQoR-11

Alone X
Difficulty falling asleep X
Physical independence Normal speech X
and care of the Sensation normal Sensation normal Sensation normal Sensation normal
neonate Able to hold baby Able to hold baby Able to hold baby Able to hold baby Able to hold baby
Wash, brush teeth X
Mobilising Mobilising Mobilising Mobilising Mobilising
independently independently independently independently independently
Look after appearance Look after Look after X
appearance appearance
Write X
Personal hygiene Personal hygiene Personal hygiene Personal hygiene Personal hygiene
Return to work X
Pre-pregnancy state Pre-pregnancy state X
Able to change Able to change X
nappy nappy
Able to feed/nurse Able to feed/nurse Able to feed/nurse Able to feed/nurse Able to feed/nurse
baby baby baby baby baby
Able to pass urine Able to pass urine Able to pass urine Able to pass urine
Psychological support Communicate with Communicate with Communicate with Communicate with Communicate with

-
staff staff staff staff staff

3
Communicate with X
family

Continued
 4 - Ciechanowicz et al.

also asked to repeat the ObsQoR-22 questionnaire 30e60 min

later (at 25 h) to test the reliability.2 An 11-item (ObsQoR-11)
ObsQoR-11 scale

survey was finalised based on ObsQoR-22 item correlation

Moderate pain
Severe pain
(r>0.20) with the global health status NRS assessment and
confirmed with post hoc factor analysis (see web-Supplement).
We conducted factor analysis and found that 21 items
(including all 11 items of the ObsQoR-11) were identified from

11
the original 22 items selected in the stage (iii). There were four
Communicate with
Draft 22-item scale

underlying factors, and these four factors explained more than

93% of the variance.
Moderate pain
instructions

After the development of the ObsQoR-11 questionnaire, the

Perineal pain
Understand

Severe pain scoring tool was evaluated post hoc to determine the following:

Headache
partner

(i) Validity as a measure of accuracy

Although not specifically tested on an independent

22
sample of women, ObsQoR-11 was assessed by two

nauseaþvomiting
subtypes of construct validity: convergent and
Communicate with

8 items removed;
discriminant validity, and content validity, which are

combined; 31
Items removed

Moderate pain
instructions

outlined as follows:
Perineal pain
Understand

Severe pain

(a) Convergent and discriminant validity: ObsQoR-11

Headache
partner

e9¼22 scores were compared with 100 mm global health

status NRS assessment scores of 70 vs <70 mm at 24
h, respectively.
X
X

(b) Content validity: ObsQoR-11 scores were assessed for

Communicate with

correlation with the length of hospital stay (time from

admission to discharge from hospital) and age of
Moderate pain
Draft 31 items

instructions

Perineal pain

women. Given the well-established increase in periop-

Muscle pain
Understand

Severe pain

Headache

erative risk with advancing maternal age, we hypoth-

Backache
partner

esised that age could negatively correlate with QoR.

(ii) Reliability as a measure of consistency was assessed by
31

evaluating the following:

(a) Internal consistency of ObsQoR-11: measured using
Communicate with
added after expert

Cronbach’s alpha and inter-item correlation tests.

12 items added;
Obstetric items

(b) Split-half reliability of ObsQoR-11: assessed by evalu-

Perineal pain

19þ12¼31

ating the correlation between random split segments.

partner

(c) Testeretest reliability of ObsQoR-11: assessed in a

review

subset of women who were asked to repeat the ques-

tionnaire 30e60 min later (at 25 h post-CD) to assess
correlation to 24 h responses.
after expert review

(d) Floor and ceiling effects of ObsQoR-11: assessed by

21 items removed;
Items removed (X)

evaluating whether <15% respondents achieved

Moderate pain
instructions

highest (110) or lowest (0) possible scores.

Muscle pain

40e21¼19
Understand

Severe pain

(iii) Responsiveness, defined as the ability to detect a clinically

Headache

Backache

important change, was assessed in ObsQoR-11 by calcu-

lating the following:
X
X
X

X
X

(a) Cohen effect size: the average change of ObsQoR-11

scores (pre- vs 24 h post-CD) divided by the standard
Support from doctors

deviation (SD) at baseline.

Support from nurses
Support from family

(b) Standardised response mean: the change in ObsQoR-

11 scores (pre- vs 24 h post-CD) divided by the SD of
Moderate pain
instructions

the change in scores.10

Muscle pain
Understand

Sore mouth
Severe pain

Sore throat
Headache
Confused

Backache

(c) Mean ObsQoR-11 scores: compared pre- vs 24 h post-

QoR-40

CD.
(iv) Acceptability and feasibility were assessed by evaluating
40

the interim 22-item ObsQoR questionnaire:

(a) Recruitment rate.
Total number of items

(b) Successful completion rate.

Table 1 Continued

(c) Time taken to complete the questionnaire (measured

and recorded by the investigator).
Domains

Statistical analysis
Pain

The sample size for this study was guided by previous studies,
as power calculation is not reliable for correlation analysis.2
 Development and evaluation of ObsQoR-11 - 5

Fig 1. Flow diagram summarising the study methodology and ObsQoR-11 development. CD, caesarean delivery; ObsQoR, obstetric-specific
QoR; QoR, quality of recovery.

Data are presented as mean (SD), median [inter-quartile range ObsQoR version. No additional items were added at this stage,
(IQR)], number (%), and 95% confidence intervals (CIs). A c2 as no new items were suggested by more than one stakeholder.
analysis was used to compare stakeholder groups in the We recruited 152 women to the next stage of the study, all
formulation phase. Continuous data were tested for normality of whom filled out the 22-item version of the ObsQoR
using the ShapiroeWilk and KolmogoroveSmirnov normality
tests; all percentages were rounded up to the nearest integer.
Correlations between the 22-item ObsQoR questionnaire items Table 2 Patient, obstetric, and clinical characteristics of the
study group (n¼152 women). CD, caesarean delivery; IQR,
(which contain ObsQoR-11 items) and global health NRS scores
inter-quartile range; number (%); SD, standard deviation; VD,
were determined using Spearman rank correlation coefficients vaginal delivery
(r). Non-parametric data were compared using Wilcoxon
signed-rank test. Internal consistency was measured with Variable
Cronbach’s alpha.11 The testeretest reliability was measured
by intra-class correlation coefficient. Post hoc factor analysis Age (yr)
was performed on the 22-item ObsQoR. We selected factors Mean (SD) 34.6 (5.6)
Range 20e50
based on eigenvalue >1. Items with loading >0.4 were selected
Length of hospital stay (h)
for each factor. The internal consistency of each factor was Median (IQR) 36.5 (33e59.6)
assessed by Cronbach’s alpha. Statistical analysis was per- Range 26e204
formed using GraphPad Software (7.0; USA) and SAS Institute, Pre-existing medical conditions
USA. The null hypothesis was rejected if the two-tailed P-value Respiratory 8 (5)
was <0.05. Cardiovascular 7 (5)
Neurological 7 (5)
Endocrine 9 (6)
Haematological 4 (3)
Results Musculoskeletal 1 (1)
Psychiatric 4 (3)
The formulation of the ObsQoR-11 from QoR-40 items is sum- Others 4 (3)
marised in Table 1. Thirty-one recovery items were initially Indication for elective CD
included from five dimensions: pain, physical comfort, physical Previous CD 49 (46)
independence and care of the neonate, psychological support, Breech 18 (17)
Maternal request 18 (17)
and emotional state. The items subsequently removed after
Abnormal placentation 5 (5)
endorsement by <2/3 of stakeholders are outlined in Previous traumatic VD 5 (5)
Supplementary Table S1 and Supplementary Figure S2. The Fetal pathology 3 (3)
resultant 22-item ObsQoR questionnaire is shown in Twins 3 (3)
Supplementary Figure S1. Based on feedback from multiple Cardiovascular 2 (2)
patients that found it difficult to define and distinguish between Neurological 2 (2)
Age 1 (1)
‘nausea’ and ‘vomiting’, these items were combined to form a
single item ‘nausea or vomiting’, before inclusion in the 22-item
 6 - Ciechanowicz et al.

Fig 2. Final ObsQoR-11 questionnaire and global health NRS scoring. ObsQoR, obstetric-specific QoR; QoR, quality of recovery.
 Development and evaluation of ObsQoR-11 - 7

Table 3 Inter-item correlation matrix for final ObsQoR-11. NRS, numerical rating scale; ObsQoR, obstetric quality of recovery. ObsQoR-
11 items taken from shortlisted ObsQoR-22 items with correlation to global health NRS. 1¼moderate pain; 2¼severe pain; 3¼nausea or
vomiting; 4¼feeling dizzy; 5¼shivering; 6¼have been comfortable; 7¼able to mobilise independently; 8¼can hold my baby without
assistance; 9¼can feed/nurse baby without assistance; 10¼can look after personal hygiene/toilet; 11¼feeling in control

ObsQoR-11 item Global health Total ObsQoR-11 1 2 3 4 5 6 7 8 9 10 11

number NRS score

1 0.45 0.71 d
2 0.43 0.72 0.68 d
3 0.28 0.56 0.34 0.39 d
4 0.25 0.63 0.35 0.41 0.47 d
5 0.23 0.61 0.35 0.42 0.45 0.49 d
6 0.42 0.55 0.33 0.37 0.17 0.25 0.14 d
7 0.28 0.56 0.33 0.32 0.21 0.28 0.49 0.22 d
8 0.22 0.49 0.25 0.29 0.17 0.28 0.42 0.19 0.59 d
9 0.22 0.54 0.26 0.25 0.24 0.3 0.47 0.24 0.54 0.71 d
10 0.28 0.57 0.31 0.31 0.29 0.38 0.40 0.23 0.63 0.57 0.54 d
11 0.37 0.43 0.15 0.18 0.27 0.27 0.32 0.55 0.33 0.24 0.34 0.30 d

questionnaire prior to CD, 100 at 24 h and 10 at 25 h after
surgery; six questionnaires had incomplete or missing data,
and were therefore excluded from the analysis, leaving 146
complete pre-CD questionnaires. There were two incomplete
questionnaires filled at 24 h, leaving 98 complete question-
naires and no incomplete data at 25 h. The clinical charac-
teristics of women who filled the preoperative 22-item ObsQoR
questionnaire are presented in Table 2. The final 11 items
included in the ObsQoR-11 were determined by excluding
ObsQoR items from the combined pre-CD and 24 h post-CD
questionnaires that failed to achieve a Spearman correlation
of 0.2 to global health NRS scores at their corresponding time
point (Supplementary Table S2; Fig. 2).
A post hoc factor analysis of the 22-item ObsQoR identified
four underlying factors, which explained more than 93% of the
variance. A Scree plot of factor loading is provided in
Supplementary Figure S3. Item clustering in the four factors is
outlined as follows, with items included in ObsQoR-11 distin-
guished in italics:
(i) Factor 1: shivering, dizziness, difficulty breathing, nausea or
vomiting, headache, severe pain, feeling depressed, itchy,
and perineal pain
(ii) Factor 2: ability to communicate with staff, communicate
with family, understand instructions, hold baby, feed baby,
and feeling in control
(iii) Factor 3: personal hygiene, ability to pass urine, mobilise
independently, and full sensation to the body
(iv) Factor 4: feeling comfortable and moderate pain
The total combined (pre- and 24 h post-CD) ObsQoR-11
scores correlated with the combined pre- and 24 h post-CD
global health NRS scores [Spearman r¼0.53 (95% CI:
0.43e0.62); P<0.0001]. The preoperative ObsQoR-11 scores
correlated to the preoperative global health NRS scores [r¼0.46
(0.31e0.59); P<0.0001], and also to the postoperative global
health NRS scores (r¼0.25; P¼0.019). The post-CD ObsQoR-11
scores correlated to the 24 h post-CD global health NRS scores
[r¼0.44 (0.26e0.60); P<0.0001].
The ObsQoR-11 scores differed significantly for women
who had a good or poor postoperative recovery. The median
[IQR] ObsQoR-11 scores at 24 h were 100 [91.3-105] for good
recovery vs 87 [72-95] for poor recovery (P<0.0001). The median
[IQR] hospital stay for women in the study was 36.5 h [33e59.6],
with a range of 26e204 h (Table 2). ObsQoR-11 at 24 h had a
negative correlation with duration of hospital stay (r¼e0.39;
P¼0.003). There was a positive association between preopera-
tive ObsQoR-11 and maternal age (r¼0.22; P¼0.027), but not to
24 h post-ObsQoR-11 scores (r¼e0.13; P¼0.31).
The inter-item correlation matrix for the ObsQoR-11 is
presented in Table 3. The split-half reliability was 0.76. The
testeretest reliability of the ObsQoR-11 items was ri>0.6 in 82%
of items and >0.5 in the remaining items, suggesting adequate
repeatability and reliability. The percentage of women
achieving the highest possible ObsQoR-11 score was 6% (n¼8)
before operation and 0% (n¼0) at 24 h postoperatively; there-
fore, no floor or ceiling effects of the questionnaire were
demonstrated (Supplementary Fig. S4). The ObsQoR-11 scores
were negatively skewed (levels of skewness were e1.89 pre-
operative and e0.78 at 24 h postoperatively), indicating that
the majority of the ObsQoR-11 total scores were in the upper
half of the scale (>55).
Responsiveness to change for the ObsQoR-11 was excellent
(Cohen effect size: 1.36), indicating an important change from
pre- to post-CD. The standardised response mean was 0.85,
which also indicates a large effect (Table 4). There was a sig-
nificant difference between the pre- and 24 h postoperative
ObsQoR-11 scores (median difference: e14; P<0.0001), sup-
porting clinical utility (Table 4). Box plots of the total ObsQoR-
11 pre- and post-CD are presented in Supplementary Figure S5.
Although the clinical acceptability and feasibility of the
ObsQoR-11 were not formally assessed, there were no refusals
to complete the longer 22-item ObsQoR questionnaire with all
of those approached agreeing to complete the questionnaires
pre-CD and post-CD. One hundred (66%) women completed
both the pre- and post-CD 22-item ObsQoR questionnaires;
drop out was attributable to early discharge or absence at the
time of interviewer attendance. Eight questionnaires (six pre-
CD and two post-CD) were excluded from the analysis because
of incomplete data, therefore attaining a 97% inclusion rate
overall. The median time [IQR] taken to complete the 22-item
ObsQoR questionnaire (n¼93) was 2 [2e3] min, range: 1e10
min.
Discussion
This study developed and evaluated an obstetric-specific re-
covery scoring tool, the ObsQoR-11, for use in patients un-
dergoing elective CD. The ObsQoR-11 performed well in
 8 - Ciechanowicz et al.

Table 4 Change in recovery status of women before caesarean delivery (preoperative baseline) and again at 24 h after caesarean
delivery (postoperative). Mean (SD). CI, confidence interval; Cohen effect size, mean change in score divided by the baseline (preop-
erative) SD; QoR, quality of recovery; SD, standard deviation; standardised response mean, mean change in score divided by its SD.
Scored on an 11-point numerical Likert scale (0e10).

Final ObsQoR-11 item Preoperative Postoperative Mean change Change from Cohen effect Standardised
(95% CI) baseline (%) size response mean

1. Moderate pain 7.5 (2.8) 4.9 (2.1) 2.6 (1.9e3.3) 35 0.95 0.77
2. Severe pain 8.7 (2.7) 6.9 (3.0) 1.7 (0.9e2.5) 20 0.64 0.42
3. Nausea or vomiting 8.6 (2.7) 7.9 (2.4) 0.7 (0.1e1.3) 8 0.25 0.22
4. Feeling dizzy 8.7 (2.6) 7.6 (2.4) 1.4 (0.8e2.0) 16 0.53 0.45
5. Shivering 9.2 (2.4) 8.3 (2.5) 1.0 (0.4e1.7) 11 0.43 0.33
6. Have been 6.5 (2.8) 5.3 (5.7) 0.9 (0.3e1.6) 14 0.33 0.28
comfortable
7. Able to mobilise 9.5 (1.6) 7.6 (2.9) 1.8 (1.1e2.5) 18 1.15 0.53
independently
8. Can hold baby 9.8 (1.1) 8.4 (2.5) 1.3 (0.6e2.0) 13 1.14 0.37
without assistance
9. Can feed/nurse baby 9.5 (1.7) 7.9 (2.8) 1.5 (0.7e2.2) 16 0.89 0.40
without assistance
10. Can look after 9.8 (1.1) 8.2 (2.7) 1.6 (0.9e2.2) 16 1.43 0.46
personal hygiene/toilet
11. Feeling in control 8.9 (2.1) 8.2 (2.2) 0.5 (e0.2 to 1.1) 6 0.25 0.15
Total 96.7 (13.1) 80.6 (17.6) 17.9 (13.6e22.2) 19 1.36 0.85

measures of reliability, responsiveness, clinical acceptability,
and feasibility. There was moderate correlation of 24 h
ObsQoR-11 scores to 24 h global health NRS scores. However,
the ObsQoR-11 was not independently evaluated on a patient
sample, and therefore, future work is needed to validate
ObsQoR-11 and determine its generalisability. The post hoc
factor analysis confirmed that the selected 11 items contrib-
uted significantly to the four underlying factors. Factor 1 rep-
resents physical comfort and pain. Factor 2 covers both
physical independence and emotional state. Factor 3 repre-
sents physical independence, and Factor 4 is a supplement to
Factor 1.
Recovery after surgery is a multidimensional process, the
definition of which varies from clinician or patient perspec-
tive. Clinically, our focus has been to achieve the functions
necessary to allow for early discharge from hospital. However,
more recently, assessment tools of recovery have expanded
from single one-dimensional measures of physiological vari-
ables to more continuous holistic assessments, including
physical, nociceptive, emotive, cognitive, and functional out-
comes.12,13 Available recovery assessment tools are heteroge-
neous in content and timing of assessment, with a
combination of subjective and objective outcomes, and com-
posite and dichotomous scoring. The currently available re-
covery assessment tools are not extensively validated for use
in the obstetric population, and therefore, may be irrelevant
for the elective CD population.13
Recovery scoring tools have been shown to be sensitive in
detecting clinically important differences in recovery after
non-obstetric surgery.14 Perioperative interventions that
result in a change of 0.9 for the QoR score, 8.0 for QoR-15, or 6.3
for QoR-40 signify a clinically important improvement or
deterioration.14 Whilst the QoR-40 is a well-established
scoring tool,1 it takes approximately 10 min to complete. The
previous QoR scales and other scores described in the litera-
ture, including short-term quality of life,8 the surgical recovery
index,15 and recovery of bowel function scoring tools,16 were
not specifically designed to measure recovery in the
immediate postoperative period, and omit pertinent items
regarding recovery following caesarean delivery. NRS scores
for global health and pain scores, although quick to measure,
do not provide a holistic indication of patient-reported func-
tional recovery.6,7
To develop an obstetric-specific QoR score, we started with
a broad selection of recovery items, and then evaluated and
refined the tool in five stages. ObsQoR-11 contains no items
from the ‘psychological support’ domain, which could imply
that recovery factors relating to self-efficacy and functional
ability are more important for CD-related health than the
perceived support from staff or relatives during hospital-
isation. Post hoc assessment of reliability, responsiveness,
clinical acceptability, and feasibility of ObsQoR-11 appears to
be very good to excellent, which is encouraging for future
validation studies. ObsQoR-11 can discriminate between good
and poor postoperative recovery in patients undergoing elec-
tive CD, without floor and ceiling effects.17 A perfect final
ObsQoR-11 score is 110, which was not achieved by any
woman assessed at 24 h after surgery. A 24 h ObsQoR-11 score
of 100 is associated with a ‘good’ health status at 24 h (NRS
70), and therefore, a good recovery, whereas a median score
of 87 is associated with a ‘poorer’ health status (NRS <70) or
poor recovery outcome after elective CD in this patient cohort.2
The post hoc assessment of ObsQoR-11 convergent validity
revealed a moderate correlation to global health NRS score,
comparable with QoR-15 and the more extensive QoR-40, used
for general postoperative recovery evaluation.12 However, the
ObsQoR-11 correlation with global health NRS fell short of the
>0.6 value recommended for scoring tools.18 Yet, the upper
limit of the 95% CI does include 0.6, suggesting ObsQoR-11 may
have utility in predicting QoR as measured by the NRS of global
health status. The global health NRS score has not been psy-
chometrically evaluated and may overrate the general well-
being in obstetric patients. Therefore, it may not be the ideal
score or ‘gold standard’ to compare ObsQoR-11 with. We also
demonstrated that ObsQoR-11 negatively correlates with hos-
pital stay, further supporting the use of this tool in measuring a

clinically important outcome related to recovery. Interestingly,
ObsQoR-11 correlated positively with age before surgery, but
not after, which could indicate better psychological prepara-
tion for CD with age or prior experience of childbirth.
Discriminant validity was determined by comparing
women who had good or poor postoperative recovery, defined
by the global health NRS score.12 The internal consistency, as
measured using Cronbach’s alpha and split-half reliability,
was high. These values were within recommended limits
(0.7e0.9),18 and were comparable with those reported with
QoR-152 and QoR-40.1 The inter-item correlation also deter-
mined the internal consistency, with values indicating good
correlation of items within ObsQoR-11, each adding its own
dimension to the recovery status.
The reproducibility (testeretest reliability) of ObsQoR-11
was excellent, with 82% of items having a correlation of >0.6.
The short duration (30e60 min) between test and retest means
that any meaningful improvement in recovery is unlikely to
have occurred, but this was vulnerable to testeretest bias. The
responsiveness of ObsQoR-11 (assessed using Cohen effect
size and standardised response means to measure the relative
size of postoperative change in ObsQoR-11 score) was excel-
lent. ObsQoR-11 had a large Cohen effect size of 1.36 and a
large standardised response mean of 0.85, suggesting signifi-
cant differences in ObsQoR-11 score pre- and 24 h post-CD.
These values are greater than QoR-40 and are comparable
with QoR-15, supporting the ability of ObsQoR-11 to detect
clinically significant differences in recovery, potentially mak-
ing it useful in trials, quality improvement and clinical prac-
tice. Individual item effect sizes ranged from low to high, but
most items had moderate to strong responsiveness. The
acceptability and feasibility of ObsQoR-11 were very good. The
median [IQR] time taken to complete the ObsQoR-22 ques-
tionnaire was 2 [2e3] min, and we anticipate the shorter
ObsQoR-11 would be completed in significantly less time than
this. The clinical brevity of ObsQoR-11 in practice may also
make it less prone to bias from non-response.
There was only a weak correlation of pre-CD ObsQoR-11 to
24 h global health NRS scores. These results would not support
the use of pre-ObsQoR-11 scores as a predictor of post-
operative recovery in this setting. We, therefore, recommend
this scoring tool be used for evaluation, not prediction, of
postoperative recovery.
Our study had several limitations. The 66% response rate at
24 h was mainly attributable to women not being present at
the planned interview time rather than refusal to complete the
questionnaire. This makes our data vulnerable to selection
bias, as we may have missed more women with a good 24 h
recovery. It also highlights the difficulties in obtaining patient-
reported postoperative scores in obstetric clinical practice.
Only 10% of women repeated the postoperative test at 25 h,
which falls short of the 25% and 20% achieved by comparable
evaluation studies.2,3 All interviews and patient interactions
were conducted at a single university hospital in the UK that
serves a multicultural population. This study only developed
and evaluated ObsQoR-11. This instrument, therefore, still
warrants further evaluation and validation, and also needs to
be applied to other hospitals serving cohorts with different
patient characteristics to determine the generalisability, and
the application in the emergency CD and vaginal delivery
settings. We excluded women unable to read or understand
written English; therefore, validation of this scoring tool would
also be required in other languages relevant to local patient
characteristics. Pre-CD ObsQoR-11 scores were taken on the
Development and evaluation of ObsQoR-11 - 9
same day of surgery, in the morning, which may have influ-
enced the baseline ObsQoR-11 scores because of the presence
of anxiety and the effects of fasting and fatigue. We chose to
measure QoR scores at 24 h post-CD, as this is becoming a
desirable standard for discharge from hospitals with enhanced
recovery programmes, as recommended by the National
Institute for Health and Care Excellence.19,20 We did not test for
the reproducibility of ObsQoR-11 beyond 25 h, as has been
demonstrated for QoR-15 in day surgery.3 In the future, we aim
to validate ObsQoR-11 in the early (24e48 h), intermediate (1
week), and late (6e8 weeks) recovery periods post-CD.
In summary, we have developed an obstetric-specific re-
covery tool (ObsQoR-11) for use in patients undergoing CD, by
adapting the QoR-40 questionnaire. ObsQoR-11 performed
well in measures of reliability, responsiveness, clinical
acceptability, and feasibility. The questionnaire appears to be
an easy-to-use recovery outcome measure, which could be
utilised as a standardised patient-centred tool to evaluate re-
covery and well-being in the obstetric population. However,
future work is needed to validate ObsQoR-11, determine its
generalisability, and evaluate its role in unplanned CD and
after assisted vaginal delivery. Further testing for reproduc-
ibility beyond 24 h is desirable. In addition, exploring the
relationship between ObsQoR-11 and other markers of recov-
ery and postoperative morbidity would further validate this
instrument.
Authors’ contributions
Study design: S.C., P.S., C.S., M.C., J.D.
Data collection: S.C., P.S., T.S., E.R., C.S.
Data analysis: S.C., P.S., M.C., B.C.
Data interpretation: S.C., P.S., B.C.
Manuscript preparation: S.C., P.S., M.C., B.C.
Acknowledgements
The authors would like to thank N. Guo for invaluable contri-
butions in performing factor analysis of the data.
Declaration of interest
The authors declare that they have no conflicts of interest.
Appendix A. Supplementary data
Supplementary data related to this article can be found at
https://doi.org/10.1016/j.bja.2018.06.011.
References
1. Myles PS, Weitkamp B, Jones K, Melick J, Hensen S. Val-
idity and reliability of a postoperative quality of recovery
score: the QoR-40. Br J Anaesth 2000; 84: 11e5
2. Stark PA, Myles PS, Burke JA. Development and psycho-
metric evaluation of a postoperative quality of recovery
score: the QoR-15. Anesthesiology 2013; 118: 1332e40
3. Chazapis M, Walker EM, Rooms MA, Kamming D,
Moonesinghe SR. Measuring quality of recovery-15 after
day case surgery. Br J Anaesth 2016; 116: 241e8
4. Kealy MA, Small RE, Liamputtong P. Recovery after
caesarean birth: a qualitative study of women’s accounts
in Victoria, Australia. BMC Pregnancy Childbirth 2010; 10: 47
 10 - Ciechanowicz et al.
5. Keita H, Ducloy-Bouthors AS. Enhanced recovery after
caesarean delivery. Not just a postoperative rehabilita-
tion. Ann Fr Anesth Reanim 2013; 32: 130e3
6. Lui MW, Li TKT, Lui F, Ong CYT. A randomised, controlled
trial of rectus sheath bupivacaine and intrathecal bupi-
vacaine, without or with intrathecal morphine, vs. intra-
thecal bupivacaine and morphine after caesarean section.
Anaesthesia 2017; 72: 1225e9
7. Acikel A, Ozturk T, Goker A, Hayran GG, Keles GT. Com-
parison of patient satisfaction between general and spinal
anaesthesia in emergency Caesarean deliveries. Turk J
Anaesthesiol Reanim 2017; 45: 41e6
8. Fatusic Z, Hudic I, Sinanovic O, Kapidzic M, Hotic N,
Music A. Short-term postnatal quality of life in women
with previous Misgav Ladach caesarean section compared
to PfannenstieleDorffler caesarean section method.
J Matern Fetal Neonatal Med 2011; 24: 1138e42
9. Deniau B, Bouhadjari N, Faitot V, Mortazavi A, Kayem G,
Mandelbrot L, et al. Evaluation of a continuous improve-
ment programme of enhanced recovery after caesarean
delivery under neuraxial anaesthesia. Anaesth Crit Care
Pain Med 2016; 35: 395e9
10. Norman GR, Wyrwich KW, Patrick DL. The mathematical
relationship among different forms of responsiveness
coefficients. Qual Life Res 2007; 16: 815e22
11. LJ C. Coefficient alpha and the internal structure of tests.
Psychometrika 1951; 16: 297e334
12. Bowyer AJ, Royse CF. Postoperative recovery and out-
comesdwhat are we measuring and for whom? Anaes-
thesia 2016; 71: 72e7
View publication stats
13. Bowyer A, Jakobsson J, Ljungqvist O, Royse C. A review of
the scope and measurement of postoperative quality of
recovery. Anaesthesia 2014; 69: 1266e78
14. Myles PS, Myles DB, Galagher W, Chew C, MacDonald N,
Dennis A. Minimal clinically important difference for three
quality of recovery scales. Anesthesiology 2016; 125: 39e45
15. Talamini MA, Stanfield CL, Chang DC, Wu AW. The sur-
gical recovery index. Surg Endosc 2004; 18: 596e600
16. Pereira Gomes Morais E, Riera R, Porfirio GJ, Macedo CR,
Sarmento Vasconcelos V, de Souza Pedrosa A, et al.
Chewing gum for enhancing early recovery of bowel
function after caesarean section. Cochrane Database Syst
Rev 2016; 10, CD011562
17. Terwee CB, Bot SD, de Boer MR, van der Windt DA,
Knol DL, Dekker J, et al. Quality criteria were proposed for
measurement properties of health status questionnaires.
J Clin Epidemiol 2007; 60: 34e42
18. McDowell I. Measuring health: a guide to rating scales and
questionnaires. 3rd Edn. New York: Oxford University Press;
2006
19. Wrench IJ, Allison A, Galimberti A, Radley S, Wilson MJ.
Introduction of enhanced recovery for elective caesarean
section enabling next day discharge: a tertiary centre
experience. Int J Obstet Anesth 2015; 24: 124e30
20. National Institute for Health and Care Excellence.
Caesarean section November 2011. Accessed 12th July 2018.
Available from: https://www.nice.org.uk/guidance/cg132/
evidence/full-guideline-pdf-184810861
Handling editor: P.S. Myles