Open access
BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
To cite: Brown B, Galpin K,
Simes J, et al. Development
of clinically meaningful quality
indicators for contemporary lung
cancer care, and piloting and
evaluation in a retrospective
cohort; experiences of the
Embedding Research (and
Evidence) in Cancer Healthcare
(EnRICH) Program. BMJ Open
2024;14:e074399. doi:10.1136/
bmjopen-2023-074399
► Prepublication history
and additional supplemental
material for this paper are
available online. To view these
files, please visit the journal
online (https://doi.org/10.1136/​
bmjopen-2023-074399).
8 November 2023
Received 06 April 2023
Accepted 31 January 2024
© Author(s) (or their
employer(s)) 2024. Re-­use
permitted under CC BY-­NC. No
commercial re-­use. See rights
and permissions. Published by
BMJ.
For numbered affiliations see
end of article.
Correspondence to
Dr Bea Brown;
​bea.​brown@s​ ydney.​edu.a​ u
Development of clinically meaningful
quality indicators for contemporary
lung cancer care, and piloting and
evaluation in a retrospective cohort;
experiences of the Embedding Research
(and Evidence) in Cancer Healthcare
(EnRICH) Program
Bea Brown ‍ ‍,1 Kirsty Galpin,1 John Simes,1 Michael Boyer,2 Chris Brown,1
Venessa Chin,3,4 Jane Young,5 on behalf of the Clinical Advisory Group
ABSTRACT
Objectives Lung cancer continues to be the most
common cause of cancer-­related death and the leading
cause of morbidity and burden of disease across
Australia. There is an ongoing need to identify and reduce
unwarranted clinical variation that may contribute to
these poor outcomes for patients with lung cancer. An
Australian national strategy acknowledges clinical quality
outcome data as a critical component of a continuously
improving healthcare system but there is a need to ensure
clinical quality indicators adequately measure evidence-­
based contemporary care, including novel and emerging
treatments. This study aimed to develop a suite of lung
cancer-­specific, evidence-­based, clinically acceptable
quality indicators to measure quality of care and outcomes,
and an associated comparative feedback dashboard
to provide performance data to clinicians and hospital
administrators.
Design A multistage modified Delphi process
was undertaken with a Clinical Advisory Group of
multidisciplinary lung cancer specialists, with patient
representation, to update and prioritise potential
indicators of lung cancer care derived from a targeted
review of published literature and reports from national
and international lung cancer quality registries.
Quality indicators were piloted and evaluated with
multidisciplinary teams in a retrospective observational
cohort study using clinical audit data from the Embedding
Research (and Evidence) in Cancer Healthcare Program, a
prospective clinical cohort of over 2000 patients with lung
cancer diagnosed from May 2016 to October 2021.
Setting and participants Six tertiary specialist cancer
centres in metropolitan and regional New South Wales,
Australia.
Results From an initial 37 potential quality indicators, a
final set of 10 indicators spanning diagnostic, treatment,
quality of life and survival domains was agreed.
Conclusions These indicators build on and update
previously available measures of lung cancer care and
Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399
Original research
STRENGTHS AND LIMITATIONS OF THIS STUDY
⇒ A rigorous, multistep modified Delphi process was
undertaken to identify a set of clinically meaning-
ful quality indicators to measure contemporary lung
cancer care.
⇒ The Clinical Advisory Group included multidisci-
plinary lung cancer specialists from metropolitan
and regional areas, as well as patient representation.
⇒ A pragmatic, iterative process was undertaken to
ensure quality indicators met the key criteria of
being linked to evidence-­based care and readily
measurable.
⇒ While applicable to local contemporary lung cancer
care, indicators may not be directly transferable
across healthcare settings.
outcomes in use by national and international lung cancer
clinical quality registries which, to our knowledge, have not
been recently updated to reflect the changing lung cancer
treatment paradigm.
BACKGROUND
Globally, lung cancer is the second most
commonly diagnosed cancer, accounting for
11.4% of total cancer cases.1 It continues to
be the most common cause of cancer-­related
death and is the leading cause of morbidity
and burden of disease in New South Wales
(NSW), across Australia and elsewhere.2 The
outlook for patients with lung cancer is poor,
with a median survival time of only 5–20
months depending on clinical stage at diag-
nosis. Overall 5-­year relative survival is 22%3
increasing to 68% if diagnosed and treated
at an early stage.4 Concerningly, however,
1
Open access
overall 5-­year survival is less than 5% for patients diag-
nosed with advanced stage disease.
Optimal care pathways and national clinical practice
guidelines5 6 have identified evidence-­based approaches
to diagnosis, treatment and supportive care that can
improve survival and quality of life for people with lung
cancer, however, these recommendations are not always
followed. Thus, there is an ongoing need to identify and
reduce unwarranted clinical variation—defined as ‘care
that differs in ways that are not a direct and proportionate
response to available evidence or to the healthcare needs
and informed choices of patients’—which may contribute
to poor outcomes for patients with lung cancer.7–9
Clinical quality indicators are evidence-­ based stan-
dards of care that describe the performance that should
occur for specific health-­ related conditions, therefore,
allowing evaluation of whether or not care is consistent
with these proposed standards.10–12 Quality indicators can
be measures of the structure, processes and/or outcomes
of care. They can be disease specific, thus describing
the quality of care related to a specific diagnosis, such
that comparative performance enables identification of
unwarranted clinical variation. Further, quality indicators
act as a benchmark for quality improvement initiatives,
enabling quantification of implemented changes,11 13
and play a crucial role in setting appropriate and realistic
targets for better performance. Measurement and feed-
back of quality indicator performance data has demon-
strably resulted in clinical quality improvement in lung
cancer care and outcomes in several international juris-
dictions including Denmark, the Netherlands and the
UK.14–17
Embedding Research (and Evidence) in Cancer
Healthcare (EnRICH) is a programme of translational
research in lung cancer, established in 2016 to describe
the natural history and patterns of care for lung cancer
and identify current gaps in evidence and practice for
clinical quality improvement.18 The EnRICH Program
includes six specialist cancer centres across three
Local Health Districts in metropolitan and regional
NSW. To date, the EnRICH cohort includes over 2000
patients. Data collection is ongoing, and the dataset
includes comprehensive patient, diagnostic, treatment
and outcome data, with up to 5-­year follow-­up for the
earliest enrolled patients. The EnRICH dataset provides
a unique opportunity to leverage existing research infra-
structure and comprehensive research quality clinical
audit data to measure routine clinical practice and iden-
tify unwarranted clinical variation, with more granularity
than routinely collected data.
This paper describes the development of a set of
evidence-­based lung cancer clinical quality indicators to
measure contemporary lung cancer care, and an asso-
ciated interactive comparative feedback dashboard to
provide performance data to clinicians and hospital
administrators, funded by a Cancer Institute NSW Inno-
vations in Cancer Control Grant (2019/INN1068) under
the auspices of the EnRICH Program.
2 Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399
BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
METHODS
The quality indicator development, piloting and evalua-
tion process is outlined in figure 1 and detailed further
below.
Review of Australian clinical practice guidelines
In November 2016, Grade A and B recommendations from
the Cancer Council Australia Clinical practice guidelines for
the management of lung cancer,6 most recently updated in
2012 based on evidence predominantly published prior
to 2010, were reviewed by an expert panel of specialist
clinicians (surgery, medical oncology, radiation oncology,
palliative care, cancer nursing, respiratory medicine) to
assess their perceived continued relevance in the context
of emerging best practice and a rapidly changing lung
cancer treatment paradigm. Recommendations with less
than 80% agreement were excluded from further review.
Targeted literature review
A targeted review of English language published litera-
ture6 8 9 13 19–28 and publicly available clinical quality registry
and other quality agency reports15 29–45 was undertaken by
two authors (BB, KG) to identify potential national and
international evidence-­based lung cancer clinical quality
indicators.
The targeted literature review was conducted in
November 2016 and updated in July 2020 in PubMed
(Medline) using the search terms ‘quality indicators’ and
‘lung cancer’ (title/abstract), limited to English language
and published from 1 January 2000 to 30 June 2020.
Grey literature was retrieved using purposive searching
of websites of national cancer agencies and national and
international lung cancer quality registries and health-
care quality agencies from equivalent healthcare systems.
Snowballing techniques were employed to review refer-
ence lists of known lung cancer quality indicator litera-
ture and relevant search results.
Draft quality indicators
Based on the review of published literature and clinical
practice guidelines recommendations, an initial set of
draft quality indicators was compiled for review and ratifi-
cation through a multistep modified Delphi process.
Clinical Advisory Group
In October 2020 an investigator meeting took place
to review the scope of the draft quality indicators and
determine membership of the Clinical Advisory Group
to include representation of each clinical specialty and
ensure results from the Delphi process would be gener-
alisable.46 Thereafter, a multidisciplinary Clinical Advi-
sory Group was convened comprising 15 members from
a range of specialties (medical oncology n=4, surgery
n=1, pathology n=1, respiratory medicine n=1, pallia-
tive care n=1, lung cancer nurse specialist n=1, health
service executive n=1, biostatistics n=1, epidemiology,
quality improvement and implementation research n=3,
consumer/patient representative n=1) with representa-
tion from metropolitan and regional clinical sites in NSW.

Figure 1 Overview of quality indicator development process.
Modified Delphi process
Several iterative Delphi rounds took place to discuss the
initial set of draft clinical quality indicators as follows:
► Individual ranking (May 2021). The draft quality indi-
cator list was circulated to Clinical Advisory Group
members who were asked to rank their top 10 most
clinically meaningful quality indicators in order of
priority. Examples of performance data were provided
for the EnRICH cohort (calculated as proportions
with 95% CIs), alongside published Victorian Lung
Cancer Registry data.29
► Clinical Advisory Group consensus meeting (May
2021). Following individual rankings, a moderated
meeting was held online (due to COVID-­19 restric-
tions), during which collated individual member
rankings were presented. The Group discussed the
utility and feasibility of each ranked quality indicator
as a measure of local contemporary lung cancer care.
At the end of the meeting the Clinical Advisory Group
agreed a set of quality indicators to be piloted with
multidisciplinary teams (MDTs).
► Ongoing correspondence was undertaken with indi-
vidual Clinical Advisory Group members (May 2021 to
Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399
Open access
BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
October 2021) regarding quality indicator definitions
and inclusion/exclusion criteria.
► Final prioritisation exercise (October 2021). A final
online consensus meeting was held, which included
detailed discussion of definitions and patient inclu-
sion/exclusion criteria for each agreed quality
indicator.
Dashboard development
An interactive, html web-­hosted, comparative feedback
dashboard was developed in RStudio. The dashboard
content included a landing page with an overview of
comparative performance data for each quality indicator
stratified by cancer centre, and tabs detailing patient
characteristics and further stratification of indicators by
disease stage and treatment type, as applicable. Results
were reported in the form of bar charts, box plots,
Kaplan-­Meier curves and summary tables. See figure 2 for
an example of the feedback dashboard overview page.
Pilot with lung cancer MDTs
The quality indicators and feedback dashboard were
piloted in November and December 2021 with lung cancer
3
Open access
Figure 2 Example of feedback dashboard overview page. EnRICH, Embedding Research (and Evidence) in Cancer Healthcare;
MDT, multidisciplinary team; NSCLC, non-­small cell lung cancer.
MDTs using clinical data from the EnRICH dataset for a
cohort of 750 patients diagnosed between 8 September
2016 and 23 May 2019 at specialist cancer centres, repre-
senting both metropolitan and regional areas, namely:
Royal Prince Alfred Hospital; Chris O’Brien Lifehouse;
Concord Repatriation General Hospital; St Vincent’s
Hospital/The Kinghorn Cancer Centre; Orange Health
Service/Bathurst Base Hospital/Dubbo Base Hospital;
and Coffs Harbour Health Campus. Performance data
were circulated via the comparative feedback dashboard
(figure 2) to MDT meeting lists. Clinician feedback was
obtained through MDTs on the clarity and transparency
of the quality indicator definitions and inclusion/exclu-
sion criteria, and the feedback dashboard format and
usability. Changes were implemented as required.
Evaluation of final quality indicators and feedback dashboard
In July 2022, a second round of quality indicator perfor-
mance data was provided, via the updated dashboard,
from the EnRICH dataset for a cohort of 426 patients
diagnosed between 1 January and 28 October 2021 at
the same six cancer centres. These data provided the
focus for an evaluation of the acceptability and utility
of the clinical quality indicators, perceptions of the
accuracy of data provided in the comparative feedback
dashboard, proposed performance benchmarks and
priorities for future quality improvement interventions to
address observed clinical variation (full evaluation results
published elsewhere47).
4 Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399
BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
Patient and public involvement statement
Dr Diane McPhail, a qualified consumer/community
representative, has been a member of the Clinical Advi-
sory Group for this study since it’s conception in 2020,
and has been a member of the EnRICH Program Steering
Committee, which informs the overall design and conduct
of the Program and related sub-­studies, since 2018.
RESULTS
Review of Australian clinical practice guidelines
Of 90 Grade A and B clinical practice recommendations
in the Cancer Council Australia clinical practice guide-
lines for the management of lung cancer, there was less
than 80% agreement with 31 (34%) (online supplemental
file 1). Of these recommendations, 1 related to surgery
(or lack thereof in unselected patients with stage IIIA
non-­small cell lung cancer (NSCLC)), 19 to treatment-­
specific systemic therapy regimens and 11 to palliative
and supportive care interventions.
Targeted literature review
The PubMed (Medline) search resulted in 54 abstracts
and online searching and snowballing identified an addi-
tional 29 articles. After excluding duplicates and arti-
cles that were not directly related to the development
or implementation of lung cancer quality indicators, 32
full-­text publications6 8 9 13 19–28 and reports15 29–45 from
Australia, Belgium, Canada, Denmark, the Netherlands,
the UK (England and Scotland) and the USA were

reviewed, resulting in 90 potential lung cancer quality
indicators (online supplemental file 2). Indicators were
considered for inclusion for review if they were used by
three or more jurisdictions. Some indicators meeting this
criterion were excluded based on local healthcare service
organisation/resourcing, for example, coordination of
care/referral to a lung cancer nurse specialist as, despite
advocacy, there are currently only 12 of these roles in
the whole of Australia. Two-­year survival was retained as
a potential indicator, being used by both the Victorian29
and Danish Lung Cancer Registries38 to enable national
and international comparisons. A number of indicators
related to timeliness of diagnosis and treatment, based
on timeframes specified in the Cancer Australia Optimal
Care Pathway for patients with lung cancer,5 were also
included.
Modified Delphi process
Based on the expert panel and targeted literature reviews,
an initial set of 37 draft lung cancer quality indicators
(table 1) was compiled for review by the Clinical Advisory
Group through the modified Delphi process.
Scores from individual rankings were reverse coded
from 10 to 1 (indicators with a score of zero were
excluded) and summed to give a final score and rank for
each quality indicator, such that the highest scoring indi-
cator was ranked first, the second highest scoring indi-
cator ranked second and so on. Quality indicator sources
and collated rankings are detailed in table 1 and online
supplemental file 3.
Seven of the 37 quality indicators (19%) were not
ranked by any individual, resulting in a summed score
of zero. Individual rankings were variable and summed
scores for the remaining 30 indicators ranged from 29 to
1. Including those that scored evenly, 12 indicators were
ranked as highest priority, eight of which related to time-
liness of diagnosis.
Rankings were discussed at the moderated Clinical
Advisory Group meeting, initially focused on the top
12 quality indicators, which were included, excluded or
adapted based on discussion. Of the initial 12 indicators,
six were excluded based on group discussion with reasons
detailed in table 1.
Following exclusion of these six indicators, further
discussion identified the need for indicators to be consis-
tently recorded in electronic medical records, so as to
be measurable, and to cover the full diagnostic and care
pathway, including all disease stage and treatment types,
rather than being heavily focused on diagnosis or being
highly treatment specific, as well as including a measure
of quality of life. Based on randomised controlled trial
evidence demonstrating early integration of palliative
care within 8 weeks of diagnosis in metastatic NSCLC
improved quality of life and mood and reduced aggressive
end-­of-­life care,48 referral to palliative care was included as
a quality of life indicator. Although not ranked in the top
10 based on individual ranking scores, 1-­year survival was
Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399
Open access
BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
included as a collectively agreed key outcome measure,
consistent with other jurisdictions.28 29 31 38
In order for the indicators to be used and impactful,
the Clinical Advisory Group universally agreed that they
should be able to be presented visually on a single over-
view page, with further information available by click
through. Therefore, rather measuring every aspect of
care, a pragmatic decision was made to limit the indi-
cators to a brief set of targeted and focused indicators,
which were evidence based and meaningful to a broad
audience—not limited to clinicians but also including
nursing and allied health staff, healthcare administrators
and patients.
Following the final prioritisation exercise, consensus
was reached on definitions and inclusion/exclusion
criteria for a set of eight quality indicators spanning diag-
nostic, treatment, quality of life and survival domains that
were piloted with MDTs through the feedback dashboard.
These were by domain:
Diagnosis
► Proportion of patients diagnosed ≤28 days* of first
investigation of symptoms suspicious of lung cancer
(earliest of clinical or pathological diagnosis).
► Proportion of patients with a pathological diagnosis
≤28 days* of first investigation of symptoms suspicious
of lung cancer.
► Proportion of patients with stage IV, non-­ squamous
NSCLC with standard of care molecular testing.
► Proportion of patients with stage III lung cancer
reviewed by an MDT prior to curative treatment.
Treatment
► Proportion of patients with stage I–III lung cancer
initiating curative treatment ≤28 days* of diagnosis.
► Proportion of patients with stage IV lung cancer initi-
ating systemic treatment ≤28 days* of diagnosis.
Quality of life
► Proportion of patients with stage IV NSCLC referred
to palliative care ≤8 weeks* of diagnosis.
Survival
► One-­year survival in patients with lung cancer.
*Refer to table 2 for rationale and justification of time
thresholds.
Pilot with lung cancer MDTs
Quality indicators
While the included indicators were well accepted and
considered to be clinically meaningful measures of
quality of care and patient outcomes, MDT feedback
highlighted clinicians’ concerns about the potential for
postdiagnostic delays in primary or secondary care, prior
to tertiary referral, to impact the timeliness of treatment.
The Clinical Advisory Group, therefore, proposed the
addition of two extra treatment-­related quality indicators
related to time from first presentation at the treating clin-
ical site to commencement of curative or systemic thera-
pies as a measure of the quality of tertiary care:
► Proportion of patients with stage I–III lung cancer initi-
ating curative treatment ≤28 days of site presentation.
5
Open access

BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
Table 1 Initial quality indicators identified through targeted literature review, review of clinical practice guidelines and clinical
quality registry reports
Domain Quality indicator Source(s) Collated ranking Included, excluded or adapted
1 Diagnosis Proportion of patients 26 28 29 43 4th Included
diagnosed within 28 days from
first presentation*
2 Diagnosis Proportion of patients with a 29 8th (equal) Excluded after consensus
clinical diagnosis within 28 meeting—pathological diagnosis
days of first presentation* prioritised
3 Diagnosis Proportion of patients with a 29 2nd Included
pathological diagnosis within
28 days of first presentation*
4 Diagnosis Proportion of patients CAG NR Excluded
diagnosed within 28 days of
site presentation
5 Diagnosis Proportion of patients with a CAG NR Excluded
clinical diagnosis within 28
days from site presentation
6 Diagnosis Proportion of patients with a CAG 5th Excluded after consensus
pathological diagnosis within meeting—majority have
28 days from site presentation pathological diagnosis prior to
site presentation
7 Diagnosis Proportion with performance 24 26 29 31 35 NR Excluded
status assessment at diagnosis
8 Diagnosis Proportion with documented 24 29 31 35 36 6th (equal) Excluded after consensus
clinical stage prior to definitive meeting—high compliance rate,
treatment other indicators prioritised as
evidence-­based metrics for
improved patient outcomes
9 Diagnosis Proportion with anatomical 29 31 35 37 NR Excluded
pathology studies after curative
resection (surgical only)
10 Diagnosis Proportion reviewed by 15 28 29 31 37 6th (equal) Included and adapted†
multidisciplinary team prior to
curative treatment
11 Diagnosis Proportion of patients with 28 29 31 32 36 37 NR Excluded
PET/CT available prior to
potentially curative treatment
12 Diagnosis Proportion of active smokers 24 35 36 42 1st Excluded after consensus
with lung cancer who have had meeting—compulsory for
smoking cessation counselling surgical patients, not reliably
discussion documented documented for others
13 Diagnosis Proportion of patients who 15 24 31 32 35 NR Excluded
underwent brain imaging
14 Diagnosis Proportion of patients who 31 32 36 NR Excluded
underwent a pulmonary
function test
15 Diagnosis Proportion of patients with 32 37 NR Excluded
NSCLC with a pathological
diagnosis of stage IIIB/C or IV
who have analysis of predictive
markers undertaken (excluding
squamous cell and PS 4)
16 Diagnosis Proportion of patients with 6 15 24 31 32 10th (equal) Included and adapted†
NSCLC with molecular testing
Continued

6 Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399

 Open access

BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
Table 1 Continued
Domain Quality indicator Source(s) Collated ranking Included, excluded or adapted
17 Diagnosis Proportion of patients with 21 29 NR Excluded
NSCLC who have undergone a
surgical resection and clinical
stage agrees with pathological
stage
18 Treatment Proportion of patients initiating 29 33 NR Excluded—see ref 21
definitive treatment within 14
days from diagnosis date
19 Treatment Proportion of patients 29 NR Excluded
commencing curative treatment
within 14 days from clinical
diagnosis date
20 Treatment Proportion of patients 29 3rd Included and adapted†
commencing curative
treatment within 14 days from
pathological diagnosis date
21 Treatment Proportion of patients initiating 15 29–31 38 44 10th (equal) Excluded after consensus
definitive treatment within 42 meeting—time interval
days from first presentation from pathological diagnosis
prioritised
22 Treatment Proportion of patients 5 NR Excluded
commencing curative treatment
within 42 days from site
presentation
23 Treatment Proportion of patients with 29–31 36–38 44 NR Excluded
NSCLC who had a resection
(stages I–IIIA)
24 Treatment Proportion of patients with 6 15 24 35–37 45 NR Excluded (CAG increased to 6
NSCLC undergoing surgery nodes prior to ranking based on
who have adequate sampling newer evidence.)
of lymph nodes (5 or more
nodes) performed at time of
primary tumour resection
25 Treatment Proportion of patients with 29 31 35 NR Excluded
NSCLC with stage II or III
disease with surgical resection
who commenced adjuvant
chemotherapy within 6 weeks
26 Treatment 30-­day postoperative mortality 15 29 30 37 41 NR Excluded
27 Treatment Proportion of patients with 29 NR Excluded
NSCLC not undergoing surgery
who receive chemotherapy
28 Treatment Proportion of patients with 6 15 30 35 37 NR Excluded
NSCLC with inoperable
infiltrative stage III (N2,
3) and PS 0–1, receiving
curative intent platinum-­
based chemotherapy and
radiotherapy
29 Treatment Proportion of patients with 29 37 NR Excluded
NSCLC with stage IIIB/C or
IV and PS 0–1 not undergoing
surgery who receive platinum-­
based chemotherapy
Continued

Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399 7

Open access

BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
Table 1 Continued
Domain Quality indicator Source(s) Collated ranking Included, excluded or adapted
30 Treatment Proportion of patients with 6 31 37 NR Excluded
limited-­stage SCLC (I–IIIB,
T1–4, N0–3, M0, PS 0–1)
treated with curative intent
who receive both platinum-­
based chemotherapy and
radiotherapy at 40 Gy or
greater
31 Treatment Proportion of patients referred 27 37 40 8th (equal) Excluded after consensus
to/enrolled in clinical trials meeting—variability in
availability of trials between
metropolitan and regional areas
considered a confounder
32 QoL Proportion of patients with 6 15 27–29 10th (equal) Excluded after consensus
documented referral to meeting—inconsistently defined
supportive care services
33 QoL Proportion of advanced-­ 6 29 30 32 NR Included and adapted†
stage patients referred to
palliative care within 90 days of
diagnosis
34 QoL Proportion of patients who 29–31 37 NR Excluded
die within 30 days of active
treatment for lung cancer
35 QoL Proportion of patients receiving 29 NR Excluded
anticancer treatment
36 Survival 1-­year survival 28 29 31 38 NR Included after consensus
meeting as key outcome
measure
37 Survival 2-­year survival 29 38 NR Excluded
*Defined as first investigation of symptoms suspicious of lung cancer.
†See table 2 for definitions. Quality indicator definitions may vary between sources.
CAG, Clinical Advisory Group; NR, not ranked in top 10 after individual scores collated; NSCLC, non-­small cell lung cancer; PET, positron
emission tomography; PS, performance status; QoL, quality of life; SCLC, small cell lung cancer.

► Proportion of patients with stage IV lung cancer initi-
ating systemic treatment ≤28 days of site presentation.
Feedback dashboard
The format of the feedback dashboard was acceptable to
MDTs; it was considered clear, easy to navigate and gave
a satisfactory overview of all sites’ performance mapped
against the quality indicators. Based on feedback, minor
changes were made to figure labelling and further infor-
mation was included on inclusion/exclusion criteria for
each indicator.
Table 2 includes the final 10 lung cancer clinical quality
indicators, their rationale, and their definitions and inclu-
sion and exclusion criteria.
DISCUSSION
This paper describes the development of a set of clinical
quality indicators to measure contemporary lung cancer
care. Potential indicators were derived from a targeted
review of published literature and reports from national
and international lung cancer quality registries and
reviewed by an expert multidisciplinary Clinical Advi-
sory Group through a modified Delphi process to reach
consensus on a final set of 10 clinically meaningful indi-
cators that reflect the lung cancer patient population
and recent advancements in lung cancer diagnosis and
treatment. These 10 quality indicators span diagnostic,
treatment, quality of life and survival domains. Specific
indicators were selected based on the criteria of being:
(1) linked to evidence-­based care, with the recognition
that some, such as survival rates, are indirect measures
of clinical care; (2) reliably and accurately recorded in
medical records and, therefore, readily measured. An
associated feedback dashboard was developed to provide
local and comparative performance data to clinicians and
hospital administrators.
A 2021 review10 identified over 300 published lung
cancer quality indicators but only 73 of these were deter-
mined to meet the minimum criteria of being evidence
based (or developed by RAND-­modified Delphi process),

8 Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399

 Open access

BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
Table 2 Final set of prioritised quality indicators
ID Quality indicator Rationale Definition/inclusion/exclusion
1 Diagnosis Proportion of patients Inclusion: (1) All patients diagnosed
Date of ‘referral’ is frequently used but
diagnosed* ≤28 days of first with lung cancer with a first
inconsistently defined and recorded
investigation of symptoms presentation date. (2) All stages.
in electronic medical records. First
suspicious of lung cancer Exclusion: (1) Patients where
investigation of suspected lung cancer
diagnosis date was before first
(ie, date of imaging or biopsy) is more
presentation date. (2) No first
reliably recorded in electronic medical
records. 28-­day timeframe based presentation date available.
on Cancer Australia Optimal Care Denominator: All patients with a lung
cancer diagnosis (earliest clinical or
Pathway (OCP) for patients with lung
cancer5—first specialist appointment
pathological) and a first presentation
≤2 weeks of GP referral, diagnostic
date.
tests complete ≤2 weeks of first Numerator: Number of patients with a
specialist appointment. lung cancer diagnosis (earliest clinical
or pathological) within 28 days from
first presentation date.
2 Diagnosis Proportion of patients with a Pathological diagnosis is the gold Inclusion: (1) All patients with a
pathological diagnosis ≤28 standard to enable histological and pathological diagnosis of lung cancer.
days of first investigation of molecular typing and is consistent (2) All stages.
symptoms suspicious of lung with the Victorian Lung Cancer Exclusion: (1) Patients where the
cancer Registry quality indicator enabling pathological diagnosis date was
cross-­jurisdiction comparison. 28-­day before first presentation date. (2)
timeframe based on Cancer Australia Patients with missing pathological
OCP5—first specialist appointment ≤2 diagnosis date. (3) Patients with no
weeks of GP referral, diagnostic tests first presentation date available.
complete ≤2 weeks of first specialist Denominator: All patients with a
appointment. pathological diagnosis of lung cancer
and a first presentation date.
Numerator: Number of patients with a
pathological diagnosis within 28 days
of first presentation date.
3 Diagnosis Proportion of patients with Multiple guidelines recommend all Inclusion: All patients with stage III
patients be reviewed by an MDT. In
stage III lung cancer reviewed lung cancer.
by an MDT prior to curative the context of high patient volume at Exclusion: (1) All patients with stage
treatment (per documented specialist cancer centres, the Clinical I, II and IV lung cancer. (2) Patients
intent in electronic medicalAdvisory Group considered it was treated with palliative intent first-­line
record) not feasible or necessary to discuss therapy.
patients for whom there are clear Denominator: All patients with stage
evidence-­based treatment regimens. III lung cancer where treatment intent
The recommendation was rationalised is curative.
to include only stage III lung cancers Numerator: Number of stage III
due to their complexity and the patients who were reviewed by an
potential requirement for multimodal MDT prior to commencing curative
therapy. treatment.
4 Diagnosis Proportion of patients with Definition in line with Medicare Inclusion: All patients with stage IV
stage IV non-­squamous Benefits Schedule rebate for non-­squamous NSCLC.
NSCLC with standard of care molecular testing. Exclusion: (1) Stage I–III patients. (2)
molecular testing Proportion of all patients with Patients with SCLC. (3) Squamous cell
NSCLC with molecular testing is also carcinoma. (4) Patients with missing/
measured but not reported in the undefined lung cancer histological
feedback dashboard. type.
Denominator: All patients with a
stage IV lung cancer diagnosis.
Numerator: Number of patients with
a stage IV lung cancer diagnosis who
had molecular testing.
Continued

Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399 9

Open access
Table 2 Continued
ID Quality indicator
5 Treatment Proportion of patients with
stage I–III lung cancer
initiating curative treatment
≤28 days of diagnosis*
6 Treatment Proportion of patients with
stage I–III lung cancer
initiating curative treatment
≤28 days of site presentation
7 Treatment Proportion of patients with
stage IV lung cancer initiating
systemic treatment ≤28 days
of diagnosis*
8 Treatment Proportion of patients with
stage IV lung cancer initiating
systemic treatment ≤28 days
of site presentation
9 Quality of Proportion of patients with
life stage IV NSCLC referred to
palliative care ≤8 weeks of
diagnosis*
10
Rationale
Multiple quality indicators focus only
on surgery as curative treatment. The
Clinical Advisory Group considered
all treatments with curative intent,
including chemotherapy and
radiotherapy, be included in this
indicator to reflect changes in
evidence and best practice. 28-­day
timeframe proposed by CAG—
reduced from 42 days in Cancer
Australia OCP.5
Indicator added by Clinical Advisory
Group following piloting, based on
feedback from multidisciplinary
teams, as a measure of the timeliness
of tertiary care, excluding potential
postdiagnostic delays in primary
or secondary care prior to site
presentation.
Indicator added by Clinical Advisory
Group as a measure of quality of
care for advanced-­stage patients
who comprise approximately 45%
of lung cancer diagnoses, including
chemotherapy, targeted therapy,
immunotherapy and palliative
radiotherapy. 28-­day timeframe
reduced from 42 days in Cancer
Australia OCP5 by CAG based on
patient input.
Indicator added by Clinical Advisory
Group, following piloting, based
on feedback from multidisciplinary
teams, as a measure of the timeliness
of tertiary care, excluding potential
postdiagnostic delays in primary
or secondary care prior to site
presentation.
Timeframe based on published
evidence.48 Clinical Advisory Group
excluded patients with actionable
molecular mutations based on
availability of new targeted therapies
and related evidence demonstrating
fewer symptoms and longer survival.52
Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399
BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
Definition/inclusion/exclusion
Inclusion: All stage I–III patients.
Exclusion: (1) Stage IV patients. (2)
Patients treated with palliative intent
first-­line therapy.
Denominator: All patients with a
stage I–III lung cancer diagnosis.
Numerator: Number of patients with
stage I–III lung cancer diagnosis who
commenced curative treatment within
28 days of diagnosis date.
Inclusion: All stage I–III patients.
Exclusion: (1) Stage IV patients. (2)
Patients treated with palliative intent
first-­line therapy.
Denominator: All patients with a
stage I–III lung cancer diagnosis.
Numerator: Number of patients with
stage I–III lung cancer diagnosis who
commenced curative treatment within
28 days of site presentation date.
Inclusion: All patients with stage IV
lung cancer.
Exclusion: (1) Stage I–III patients. (2)
Stage IV patients treated with curative
intent first-­line therapy.
Denominator: All patients with a
stage IV lung cancer diagnosis.
Numerator: Number of patients with
stage IV lung cancer diagnosis who
commenced systemic treatment
within 28 days of diagnosis date.
Inclusion: All patients with stage IV
lung cancer.
Exclusion: (1) Stage I–III patients. (2)
Stage IV patients treated with curative
intent first-­line therapy.
Denominator: All patients with a
stage IV lung cancer diagnosis.
Numerator: Number of patients with
stage IV lung cancer diagnosis who
commenced systemic treatment within
28 days of site presentation date.
Inclusion: All patients with stage IV
NSCLC.
Exclusion: (1) Stage I/II/IIIA/B/C
patients. (2) Patients with SCLC. (3)
Patients with missing/undefined lung
cancer type.
Denominator: All patients with a
stage IV NSCLC diagnosis.
Numerator: Number of patients with
a stage IV NSCLC referred to palliative
care within 8 weeks of diagnosis.
Continued
 Open access

BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
Table 2 Continued
ID Quality indicator Rationale Definition/inclusion/exclusion
10 Survival 1-­year survival in patients Globally agreed quality of care Inclusion: All patients with lung
with lung cancer outcome. cancer.
Calculated from date of pathological Exclusion: Patients deceased before
diagnosis (or clinical diagnosis if no/ 1 year.
inconclusive pathology). Denominator: All patients with lung
Date of death based on monthly cancer.
linkage with death notifications in Numerator: Number of patients
the NSW Registry of Births, Deaths where patient status=alive 1 year after
& Marriages. Patient assumed alive if diagnosis.*
not deceased on the first day of the
reporting month.
*Based on pathological diagnosis date—date of sample collection for earliest conclusive cytology/biopsy/or resection OR if no pathology,
clinical diagnosis date—date of first imaging with outcome ‘probable lung cancer’.
CAG, Clinical Advisory Group; GP, general practitioner; MDT, multidisciplinary team; NSCLC, non-­small cell lung cancer; SCLC, small cell lung
cancer.

feasible or measurable (assessed by documented measure-
ment with the quality indicator), and demonstrably able
to detect variation in care. The EnRICH quality indicators
developed through this study meet all of these criteria.
The same review concluded that published indicators
disproportionately relate to preoperative and surgical
measures. Of note, the final set of EnRICH quality indi-
cators includes all evidence-­ based potentially curative
therapies for early-­stage patients, and systemic and palli-
ative therapies for advanced-­stage patients who comprise
a significant proportion of lung cancer diagnoses. While
there are additional quality indicators that could be
considered, such as use of specific systemic therapies,
increased specificity has implications for data collection
and scrutiny, requiring much effort to identify appro-
priate patient groups.
The Australian National Clinical Quality Registry and
Virtual Registry Strategy 2020–203049 acknowledges clin-
ical quality outcome data as a critical component of a
continuously improving healthcare system and identifies
a need to maximise the value of clinical quality outcome
datasets with the potential to be expanded nationally in
areas with the greatest burden of disease and cost to the
Australian health system and/or with greatest variation
in care and outcomes, such as lung cancer. Others have
echoed the need to develop a lung cancer clinical quality
registry to monitor the safety, quality and effectiveness of
lung cancer care in Australia and New Zealand, building
on the established Victorian Lung Cancer Registry,29 but
note that quality indicators need to be further developed
to consider the inclusion of novel/emerging or under-­
represented measures.50 The lung cancer quality indica-
tors developed through the EnRICH Program build on
and update previously available measures of lung cancer
care and outcomes in use by national29 and international
lung cancer clinical quality registries15 38 39 which, to our
knowledge, have not been recently updated to reflect the
changing lung cancer treatment paradigm. The changes
to existing quality indicators proposed by the Clinical
Advisory Group support the need for regular review to
ensure they continue to be clinically relevant and fit for
purpose.
Of note, throughout the modified Delphi process, clini-
cians tended to prioritise indicators related to their own
specialty and clinical context. A strength of this study is
the inclusion of a broad range of multidisciplinary lung
cancer specialists from metropolitan and regional NSW,
as well as a patient representative, in the Clinical Advisory
Group to ensure the final set of quality indicators is appli-
cable to the diverse patient populations, case mixes and
clinical services available across the State. It is anticipated
that the indicators should, therefore, be generalisable at a
national, and potentially international, level. Definitions
and explicit patient inclusion/exclusion criteria for each
quality indicator have been provided for transparency,
and to enable others to measure and report on perfor-
mance in a comparable way. Inclusion/exclusion criteria
additionally account for variation in risk level, whereby
recommended care is not applicable for some defined
groups. A further strength of the study is the develop-
ment of an associated dashboard to provide a feedback
loop for clinicians and hospital administrators to identify
clinical variation and to prompt examination of outliers
on poor performing quality indicators to prioritise areas
for service improvement. This dashboard is being used
to feed back ongoing comparative performance data to
MDTs (2022 data were circulated in July 2023 and 2023
data are anticipated to be circulated in July 2024) to iden-
tify areas for quality improvement. Two initial clinician-­
prioritised quality improvement interventions are: (1) a
process to ensure all stage III patients are discussed by an
MDT prior to commencing treatment; and (2) a referral
pathway to palliative care within 8 weeks for patients diag-
nosed with stage IV disease.47
A limitation of the quality indicators developed
through this study, and for clinical quality registries more
broadly, is the challenge arising from clinical audit data
collection from electronic medical records and hospital

Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399 11

 Open access
administrative datasets. In the absence of a universal elec-
tronic medical record system, with mandatory data points,
data entry is necessarily manual and, therefore, highly
resource intensive. Future efforts to establish a national
lung cancer quality registry will need to ensure electronic
medical records adequately and accurately record the data
required to calculate compliance with quality measures,
and benchmarks for recording relevant data items could
be included as discrete quality indicators. A further
limitation of this study is that non-­English publications
on international lung cancer quality improvement initia-
tives were not included in the targeted literature review.
However, many such initiatives, for example, the Belgian
Health Care Knowledge Centre,31 the Danish Lung
Cancer Registry38 and the Dutch Lung Cancer Audit,15
have published reports in English, and, therefore, were
included. Necessarily, the literature review also excluded
publications after the most recent search in July 2020.
An updated search conducted in November 2023 identi-
fied an additional 26 publications potentially eligible for
inclusion, suggesting growing interest in this space.
The acceptability and utility of the clinical quality indi-
cators, perceptions of the accuracy of data provided in the
comparative feedback dashboard, proposed performance
benchmarks and priorities for future quality improve-
ment interventions to address observed clinical variation
have been formally evaluated and the results are reported
elsewhere.47 Performance on the quality indicators strati-
fied by a number of patient, disease and hospital charac-
teristics, as well as the impact of COVID-­19, has also been
reported elsewhere.51
Future research will explore the utility of these lung
cancer quality indicators as a platform for the develop-
ment and evaluation of feasibility, acceptability, effec-
tiveness and cost-­effectiveness of randomised discovery
and intervention-­based trials, including: (1) local prior-
ities for reducing variation in survival and quality of life;
and (2) integration of novel diagnostic and therapeutic
technologies.
CONCLUSIONS
The number and types of quality indicators integrated
into clinical practice, and their measurability and ability
to detect variation, are an important consideration to
ensure they are clinically meaningful. Through a modi-
fied Delphi process, a set of 10 quality indicators, span-
ning diagnostic, treatment, quality of life and survival
domains, was prioritised. These will be regularly reviewed
to ensure they continue to measure emerging and novel
therapies and resultant changes to the treatment and
management of lung cancer.
Author affiliations
1
NHMRC Clinical Trials Centre, The University of Sydney, Camperdown, New South
Wales, Australia
2 2021. Canberra, 2021.
Chris O'Brien Lifehouse, Camperdown, New South Wales, Australia
3
Garvan Institute of Medical Research, Darlinghurst, New South Wales, Australia
4
St Vincent's Hospital, Darlinghurst, New South Wales, Australia
12
BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
5
School of Public Health, The University of Sydney, Camperdown, New South Wales,
Australia
Acknowledgements We acknowledge the support and guidance of the Clinical
Advisory Group and the EnRICH Steering Committee, and data collection undertaken
by the EnRICH Project Team.
Collaborators Clinical Advisory Group: Amy O’Donnell (Lung Cancer Nurse
Care Coordinator), Bea Brown (EnRICH Program Manager, Health Services/
Implementation Research), Brian McCaughan (surgeon), David Barnes (respiratory
physician), Davinia Seah (palliative care physician), Diane McPhail (consumer
representative), Jane Young (EnRICH T2/T3 Lead, Epidemiology/Health Services
Research), John Simes (EnRICH Program Chair, medical oncologist), Karen
Briscoe (medical oncologist), Kirsty Galpin (Health Services Research), Michael
Boyer (EnRICH Clinical Lead, medical oncologist), Ruth Jones (Director of Cancer
Services), Venessa Chin (EnRICH T1/T2 Lead, medical oncologist), Wendy Cooper
(pathologist).
Contributors BB contributed to study conceptualisation, methodology, literature
review and writing of the manuscript. KG contributed to literature review and
writing of the manuscript. JS, JY, MB and VC contributed to study conceptualisation
and methodology and reviewed the manuscript. CB provided biostatistics support,
developed the dashboard and reviewed the manuscript. BB is responsible for the
overall content as guarantor.
Funding This work was supported by the Cancer Institute New South Wales (grant
number: 2019/INN1068) and Sydney Local Health District.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the
design, or conduct, or reporting, or dissemination plans of this research. Refer to
the Methods section for further details.
Patient consent for publication Not applicable.
Ethics approval Ethical approval for this project was authorised by Sydney Local
Health District Lead Human Research Ethics Committee (RPA Zone) under protocol
number X16-­0447. Participants gave informed consent to participate in the study
before taking part.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request. Data are
not reported in this paper. Analysis code for quality indicator calculations available
on request, subject to necessary ethical approvals.
Supplemental material This content has been supplied by the author(s). It has
not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been
peer-­reviewed. Any opinions or recommendations discussed are solely those
of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and
responsibility arising from any reliance placed on the content. Where the content
includes any translated material, BMJ does not warrant the accuracy and reliability
of the translations (including but not limited to local regulations, clinical guidelines,
terminology, drug names and drug dosages), and is not responsible for any error
and/or omissions arising from translation and adaptation or otherwise.
Open access This is an open access article distributed in accordance with the
Creative Commons Attribution Non Commercial (CC BY-­NC 4.0) license, which
permits others to distribute, remix, adapt, build upon this work non-­commercially,
and license their derivative works on different terms, provided the original work is
properly cited, appropriate credit is given, any changes made indicated, and the use
is non-­commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
ORCID iD
Bea Brown http://orcid.org/0000-0003-2851-1809
REFERENCES
1 Sung H, Ferlay J, Siegel RL, et al. Global Cancer Statistics 2020:
GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36
Cancers in 185 Countries. CA Cancer J Clin 2021;71:209–49.
2 Australian Institute of Health and Welfare (AIHW). Cancer in Australia
3 Cancer Australia. Lung cancer in Australia statistics. 2023. Available:
https://www.canceraustralia.gov.au/cancer-types/lung-cancer/​
statistics
Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399

4 Cancer Australia. Relative survival by stage at diagnosis (lung
cancer). Sydney, 2019. Available: https://ncci.canceraustralia.gov.​
au/outcomes/relative-survival-rate/relative-survival-stage-diagnosis-​
lung-cancer
5 Cancer Council Victoria and Department of Health Victoria. Optimal
care pathway for people with lung cancer. Melbourne, 2021.
6 Cancer Council Australia Lung Cancer Prevention and Diagnosis
Guidelines Working Party. Clinical practice guidelines for the
prevention and diagnosis of lung cancer. Sydney, Available: https://​
wiki.cancer.org.au/australia/Guidelines:Lung_cancer/Prevention_​
and_diagnosis
7 Sutherland K, Levesque JF. Unwarranted clinical variation in health
care: definitions and proposal of an analytic framework. J Eval Clin
Pract 2020;26:687–96.
8 McGregor D, Rankin N, Butow P, et al. Closing evidence-­practice
gaps in lung cancer: results from multi-­methods priority setting in the
clinical context. Asia Pac J Clin Oncol 2017;13:28–36.
9 Rankin NM, McGregor D, Stone E, et al. Evidence-­practice
gaps in lung cancer: a scoping review. Eur J Cancer Care (Engl)
2018;27:e12588.
10 Chiew K-­L, Sundaresan P, Jalaludin B, et al. Quality indicators in lung
cancer: a review and analysis. BMJ Open Qual 2021;10:e001268.
11 Mainz J. Defining and classifying clinical indicators for quality
improvement. Int J Qual Health Care 2003;15:523–30.
12 Lilford RJ, Brown CA, Nicholl J. Use of process measures to monitor
the quality of clinical practice. BMJ 2007;335:648–50.
13 Stirling RG, Evans SM, McLaughlin P, et al. The Victorian Lung
Cancer Registry pilot: improving the quality of lung cancer care
through the use of a disease quality registry. Lung 2014;192:749–58.
14 Jakobsen E, Rasmussen TR, Green A. Mortality and survival of lung
cancer in Denmark: Results from the Danish Lung Cancer Group
2000–2012. Acta Oncologica 2016;55:2–9.
15 Ismail RK, Schramel F, van Dartel M, et al. The Dutch lung cancer
audit: nationwide quality of care evaluation of lung cancer patients.
Lung Cancer 2020;149:68–77.
16 Aveling E-­L, Martin G, Jiménez García S, et al. Reciprocal peer
review for quality improvement: an ethnographic case study
of the improving lung cancer outcomes project. BMJ Qual Saf
2012;21:1034–41.
17 Russell GK, Jimenez S, Martin L, et al. A multicentre randomised
controlled trial of reciprocal lung cancer peer review and supported
quality improvement: results from the improving lung cancer
outcomes project. Br J Cancer 2014;110:1936–42.
18 Simes RJ, Boyer M, Hogg P, et al. Embedding research (and
evidence) in cancer Healthcare (enrich) - program protocol. 2021.
Available: https://doi.org/10.25910/5n3z-z838
19 Cancer Institute NSW. NSW Cancer Plan. Focus: Lung cancer
Sydney: Cancer Institute NSW; 2016-­2020, Available: https://www.​
cancer.nsw.gov.au/About-cancer/Document-library/NSW-Cancer-​
Plan-2016-2021/Focus-Lung-cancer
20 Cameron JK, Aitken J, Reid A, et al. Geographic distribution of
malignant mesothelioma incidence and survival in Australia. Lung
Cancer 2022;167:17–24.
21 Jakobsen E, Green A, Oesterlind K, et al. Nationwide quality
improvement in lung cancer care: the role of the Danish Lung Cancer
Group and Registry. J Thorac Oncol 2013;8:1238–47.
22 Leighl NB, Rekhtman N, Biermann WA, et al. Molecular testing for
selection of patients with lung cancer for epidermal growth factor
receptor and anaplastic lymphoma kinase tyrosine kinase inhibitors:
American Society of Clinical Oncology endorsement of the College of
American Pathologists/International Association for the study of lung
cancer/association for molecular pathology guideline. J Clin Oncol
2014;32:3673–9.
23 National Institute for Health and Care Excellence (NICE). EGFR-­TK
Mutation testing in adults with locally advanced or metastatic non-­
small-­cell lung cancer. 2013. Available: https://www.nice.org.uk/​
guidance/dg9
24 Detterbeck FC, Postmus PE, Tanoue LT. The stage classification
of lung cancer: diagnosis and management of lung cancer, 3rd
ed: American College of Chest Physicians evidence-­based clinical
practice guidelines. Chest 2013;143(5 Suppl):e191S–e210S.
25 Kerr KM, Bubendorf L, Edelman MJ, et al. Second ESMO consensus
conference on lung cancer: pathology and molecular biomarkers for
non-­small-­cell lung cancer. Ann Oncol 2014;25:1681–90.
26 Mak KS, van Bommel ACM, Stowell C, et al. Defining a standard
set of patient-­centred outcomes for lung cancer. Eur Respir J
2016;48:852–60.
27 NHS England Lung Clinical Expert Group. The National optimal lung
cancer pathway (NOLCP) and implementation guide. 2017. Available:
Brown B, et al. BMJ Open 2024;14:e074399. doi:10.1136/bmjopen-2023-074399
Open access
BMJ Open: first published as 10.1136/bmjopen-2023-074399 on 14 February 2024. Downloaded from http://bmjopen.bmj.com/ on February 15, 2024 by guest. Protected by copyright.
https://rmpartners.nhs.uk/our-work/early-diagnosis/earlier-diagnosis-​
of-lung-cancer/national-optimal-lung-cancer-pathway-nolcp/#:~:​
text=The%20National%20Optimal%20Lung%20Cancer,treatment%​
20could%20be%20more%20effective
28 NHS Cancer Programme. Implementing A timed diagnositic lung
cancer pathway. A handbook for local health and care systems,
Available: https://www.england.nhs.uk/wp-content/uploads/2018/04/​
implementing-timed-lung-cancer-diagnostic-pathway.pdf
29 Stirling R R, Martin C, Brand M, et al. The Victorian Lung Cancer
Registry Annual Report. Report No.: No 6. Department of
Epidemiology and Preventive Medicine, 2020.
30 Matheson LM, Pitson G, Yap CH, et al. Measuring the quality of
cancer care in the Barwon South Western region, Victoria, Australia.
Int J Qual Health Care 2021;33:mzaa145.
31 Vrigens F, Verleye L, Gendt C, et al. Quality indicators for the
management of lung cancer. Report No.: KCE Reports 266. Brussels:
KCE) BHCKC, 2016.
32 National Institute for Health and Care Excellence (NICE). Lung
cancer: diagnosis and management. 2022.
33 International Consortium for Health Outcomes Measurement
(ICHOM). Patient-­centered outcome measures: lung cancer. Boston,
USA, 2023. Available: https://connect.ichom.org/patient-centered-​
outcome-measures/lung-cancer
34 Mazzone PJ, Vachani A, Chang A, et al. Quality indicators for the
evaluation of patients with lung cancer. Chest 2014;146:659–69.
35 Malafa MP, Corman MM, Shibata D, et al. The Florida Initiative for
Quality Cancer Care: a regional project to measure and improve
cancer care. Cancer Control 2009;16:318–27.
36 Cancer Care Ontario. Lung cancer. n.d. Available: https://www.​
cancercareontario.ca/en/types-of-cancer/lung
37 Scottish Cancer Taskforce National Cancer Quality Steering Group.
Lung cancer clinical quality performance indicators. 2021. Available:
https://www.healthcareimprovementscotland.org/our_work/cancer_​
care_improvement/cancer_qpis/quality_performance_indicators.​
aspx
38 Jakobsen E, Green A, Østerlind K, et al. Danish lung cancer registry.
Ugeskr Laeger 2012;174:2539.
39 Royal College of Physicians. National Lung Cancer Audit annual
report (for the audit period 607 2019 England, Wales and Guernsey
and 2020 England only). London, 2022.
40 Cancer Institute NSW. Cancer control in NSW: Statewide report
2020. Sdyney, 2020.
41 Dalton SO, Frederiksen BL, Jacobsen E, et al. Socioeconomic
position, stage of lung cancer and time between referral and
diagnosis in Denmark, 2001-­2008. Br J Cancer 2011;105:1042–8.
42 National Institute for Health and Care Excellence (NICE). Lung cancer
in adults. 2019. Available: https://www.nice.org.uk/guidance/qs17/​
chapter/Quality-statement-2-Stopping-smoking
43 National Institute for Health and Care Excellence (NICE). Suspected
cancer: recognition and referral. 2021. Available: https://www.nice.​
org.uk/guidance/ng12/chapter/Recommendations-organised-by-​
site-of-cancer#lung-and-pleural-cancers
44 NHS England. Manual for cancer services. Lung cancer measures.
London: Programme NPR, 2013.
45 Brand MC, McNeil J, et al, Stirling R. The Victorian lung cancer
Registry report. Contract No.: Report No 5. Monash University,
Department of Epidemiology and Preventative Medicine, 2018.
46 Nasa P, Jain R, Juneja D. Delphi methodology in healthcare
research: how to decide its appropriateness. World J Methodol
2021;11:116–29.
47 Brown B (Bea), Galpin K, Simes RJ, et al. Clinicians’ evaluation of
lung cancer clinical quality indicators and comparative performance
data in practice. European Journal of Cancer Care 2023;2023:1–11.
48 Temel JS, Greer JA, Muzikansky A, et al. Early palliative care for
patients with metastatic non-­small-­cell lung cancer. N Engl J Med
2010;363:733–42.
49 Department of Health. National clinical quality Registry and virtual
Registry strategy. Canberra,
50 Smith S, Brand M, Harden S, et al. Development of an Australia and
New Zealand Lung Cancer Clinical Quality Registry: a protocol paper.
BMJ Open 2022;12:e060907.
51 Brown B, Young J, Galpin K, et al. The impact of COVID-­19 on
lung cancer care in NSW: real-­world data from the EnRICH cohort.
Submitted
52 Shimamura SS, Shukuya T, Asao T, et al. Survival past five years with
advanced, EGFR-­mutated or ALK-­rearranged non-­small cell lung
cancer-­is there a “tail plateau” in the survival curve of these patients?
BMC Cancer 2022;22:323.
13