Original Research
Otolaryngology–
Assessing the Quality of the Management
of Tonsillitis among Australian Children:
A Population-Based Sample Survey
Peter Hibbert1,2, Jacqueline H. Stephens, PhD2,
Carl de Wet, MBChB3,4, Helena Williams, MBBS5,
Andrew Hallahan, MBBS6,7, Gavin R. Wheaton, MBBS8,
Chris Dalton, MBBS9, Hsuen P. Ting, MSc1, Gaston Arnolda, PhD1,
and Jeffrey Braithwaite, PhD1
Sponsorships or competing interests that may be relevant to content are dis-
closed at the end of this article.
Abstract
Objective. The aims of this study were twofold: (1) to design
and validate a set of clinical indicators of appropriate care
for tonsillitis and (2) to measure the level of tonsillitis care
that is in line with guideline recommendations in a sample of
Australian children.
Study Design. A set of tonsillitis care indicators was devel-
oped from available national and international guidelines and
validated in 4 stages. This research used the same design as
the CareTrack Kids study, which was described in detail
elsewhere.
Setting. Samples of patient records from general practices,
emergency departments, and hospital admissions were
assessed.
Subjects and Methods. Patient records of children aged 0 to
15 years were assessed for the presence of, and adherence
to, the indicators for care delivered in 2012 and 2013.
Results. Eleven indicators were developed. The records of
821 children (mean age, 5.0 years; SD, 4.0) with tonsillitis
were screened. The reviewers conducted 2354 eligible indi-
cator assessments across 1127 visits. Adherence to 6 indica-
tors could be assessed and ranged from 14.3% to 73.2%
(interquartile range 31.5% to 72.2%).
Conclusion. Our main findings are consistent with the inter-
national literature: the treatment of many children who
present with confirmed or suspected tonsillitis is inconsis-
tent with current guidelines. Future research should con-
sider how the indicators could be applied in a structured
and automated manner to increase the reliability and effi-
ciency of record reviews and help raise clinicians’ awareness
of appropriate tonsillitis management.
Head and Neck Surgery
1–8
Ó American Academy of
Otolaryngology–Head and Neck
Surgery Foundation 2018
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/0194599818796137
http://otojournal.org
Keywords
tonsillitis, patient safety, guideline adherence, health care
quality indicators, child health
Received May 8, 2018; revised July 18, 2018; accepted August 2, 2018.
T
onsillitis is the fifth-most common condition man-
aged by general practitioners (GPs) for children aged
0 to 14 years and has been estimated to represent
3.7% of all consultations.1 This equates to about 550,000
consultations per year in Australia.2 More than 74,000 chil-
dren aged \15 years have tonsillectomies or adenotonsillec-
tomies in the Australian public health system annually, with
a financial cost of approximately AUD$272 million.3,4
Most episodes of tonsillitis are viral in origin and self-
limiting and do not require antibiotics.5 Inappropriately pre-
scribing antibiotics for suspected or acute tonsillitis can con-
tribute to the increasing problem of antibiotic resistance and
may cause avoidable adverse drug events.6,7 However, ton-
sillitis of bacterial origin that is not treated appropriately
1
Centre for Healthcare Resilience and Implementation Science, Australian
Institute of Health Innovation, Macquarie University, Australia
2
School of Health Sciences, University of South Australia, Adelaide,
Australia
3
School of Medicine, Griffith University, Southport, Australia
4
Menzies Health Institute Queensland, Nathan, Australia
5
Southern Adelaide Local Health Network, Adelaide, Australia
6
Children’s Health Queensland, Lady Cilento Children’s Hospital, South
Brisbane, Australia
7
School of Medicine, University of Queensland, Herston, Australia
8
Department of Cardiology, Women’s and Children’s Hospital, North
Adelaide, Australia
9
Bupa ANZ, Sydney, Australia
Corresponding Author:
Peter Hibbert, Centre for Healthcare Resilience and Implementation
Science, Australian Institute of Health Innovation, Macquarie University,
Level 6, 75 Talavera Rd, Macquarie University, NSW 2109.
Email: peter.hibbert@mq.edu.au
2 Otolaryngology–Head and Neck Surgery
may have rare but serious complications, such as peritonsil-
lar abscesses, rheumatic fever, and glomerulonephritis.8,9
Antibiotic therapy is therefore recommended for certain
high-risk or vulnerable patients, including indigenous
patients, unwell patients, and/or patients with particularly
severe clinical features suggestive of streptococcal infec-
tion.10,11 Measures such as advice/counseling and analgesia
are also recommended as part of the overall management.
Clinical practice guidelines (CPGs) have been developed
for the management of tonsillitis and for the more general
issue of diagnosing and treating pharyngeal and respiratory
tract infections.12,13 In addition, several Australian cam-
paigns have been designed to help encourage judicious anti-
biotic use.14,15 Despite these initiatives and active
promotion of guidelines, the majority (89%) of children \5
years old with confirmed or suspected episodes of tonsillitis
are prescribed antibiotics, which is higher than that recom-
mended in the current Australian therapeutic guidelines,
while a minority (14%) receive the recommended counsel-
ing and advice.5
The CareTrack Kids (CTK) program was designed to
determine the quality of care that Australian children aged 0
to 15 years received for 17 common conditions over a 2-
year period from 2012 to 2013.16 The overall findings of
CTK were previously reported.16 This article reports the
findings relating to 1 of the 17 conditions: tonsillitis. The
aims of this study were twofold: (1) to design and validate a
set of clinical indicators that describes appropriate tonsillitis
care and (2) to measure the quality of tonsillitis care in a
sample of Australian children.16,17,18,19
Methods
Study Design
This study used the same design as the larger CTK program,
which was described in detail elsewhere.16,18,19 Essentially,
a set of tonsillitis care indicators were developed from avail-
able national and international guidelines and validated in 4
stages. Samples of patient records were then selected and
screened for the presence of indicators. Trained clinician
reviewers then determined whether clinical care complied
with the indicators.
Development of Indicators
We defined a clinical indicator as a measurable component
of a standard or guideline, with explicit criteria for inclu-
sion, exclusion, time frame, and practice setting.
Stage 1: Identify Candidate Tonsillitis Indicators from Existing
Guidelines. We searched for national and international CPGs rel-
evant to tonsillitis and pediatric patients.19 Recommendations
were extracted, and initial draft indicators were created from
these. Eight CPGs were found for tonsillitis—3 from Australia,
2 from United Kingdom, and 1 each from Europe, New
Zealand, and the United States20-27—from which 25 recommen-
dations were extracted. Recommendations were excluded from
extraction if they were guiding statements only, with no
recommended actions; if they used auxiliary verbs, such as
‘‘may,’’‘‘consider,’’ and ‘‘could,’’ to indicate the recommenda-
tion’s strength; if there was a low likelihood of information
being documented in the medical record; or if they were out of
scope for our purposes (eg, structure-level measures). After dele-
tion of such recommendations, 13 candidate indicators were cre-
ated from the recommendations.
Stage 2: Internal Review of Candidate Indicators. Candidate
indicators were reviewed over 3 rounds with a modified
Delphi technique by 3 clinicians of the CTK team, all
whom had .20 years’ experience. There was 1 GP and 2
hospital-based pediatricians, 1 of whom had a university
associate professor appointment. Reviews were undertaken
by email. Indicators were excluded if their acceptability,
feasibility, or impact was perceived as ‘‘low’’; if the con-
cept was covered in other indicators; or if clinicians
assigned them low ‘‘appropriateness’’ scores. Reviewers
were able to provide comments to explain their ratings. The
CTK team aggregated the scores and de-identified com-
ments at the end of each round, providing these back to the
reviewers during the next round, with changes made to the
indicators.
Stage 3: External Review of Preliminary Indicators. The prelimi-
nary indicators were reviewed and modified by 4 pediatri-
cians who were not members of the CTK team, using a
modified Delphi method and making the same assessments
as in the internal review. Reviewers were recruited via
advertisements in the publications of relevant medical col-
leges, professional associations, and local clinical networks
and through direct consultation with them. Three of the 4
pediatricians had joint university appointments, and all were
hospital based. Reviews were undertaken on a customized
wiki site. In stages 2 and 3, reviewers worked independently
to minimize groupthink.16 Seven final indicators were
included, which were formatted into 11 medical record
audit indicator questions; all indicator questions are shown
in Table 1, with additional details in Appendix 1 (available
online only).
Stage 4: Validation of the Final Indicators. Each clinician indi-
cated whether the final indicators were acceptable, feasible
to collect, and clinically important.19 For the internal and
external reviews, a modified RAND-UCLA method for
scoring indicators was used,28 which was shown to be reli-
able and to have content, construct, and predictive valid-
ity.29-33
Sampling Strategy and Sample Size
The planned sample size for this study was 400 patient
records. However, if consultations for tonsillitis were
detected during audit of medical records selected for the
other 16 conditions of the CTK study, those records were
also assessed for the presence of, and adherence to, the ton-
sillitis indicators.
The sampling strategy was described previously,16 and
additional details specific to tonsillitis are shown in
Hibbert et al 3

Table 1. Quality of Care and Indicator Adherence.

Health Care Proportion
Indicator ID: Description Setting Children, n Visits, n Adherent, % (95% CI)

TONS01: Children aged 3-14 y with a sore throat had GP 369 467 70.6 (54.1-84.0)
their temperature assessed. ED 121 143 99.6 (96.7-100.0)
Inpatient 42 43 100.0 (91.8-100.0)
Overall 500 653 72.2 (58.7-83.4)
TONS02: Children with a sore throat and with no other GP 91 99 36.7 (12.3-67.6)
symptoms or signs of tonsillitis were not prescribed ED 50 55 92.6 (82.2-97.9)
antibiotics. Inpatient 5 5 Insufficient data
Overall 144 159 40.9 (16.9-68.6)
TONS03: Parents of children with a sore throat were GP 558 742 28.5 (13.6-47.9)
instructed to provide fluids. ED 238 278 73.0 (63.0-81.6)
Inpatient 74 79 83.4 (61.4-95.8)
Overall 813 1099 31.5 (19.0-46.2)
TONS04: Children aged \4 y with a sore throat and GP 120 154 12.8 (8.0-19.2)
associated cough who did not require hospitalization ED 59 62 38.6 (26.3-52.0)
were not prescribed antibiotics. Overall 179 216 14.3 (9.9-19.7)
TONS05: Children aged 3-14 y assessed as high risk or GP 30 35 34.8 (15.2-59.1)
GABHS positive and allergic to penicillin were ED 9 11 Insufficient data
prescribed oral erythromycin. Inpatient 4 4 Insufficient data
Overall 42 50 34.7 (17.4-55.6)
TONS06: Children with recurrent acute sore throat with GP 53 83 72.1 (45.7-90.6)
episodes that were disabling and prevented normal ED 28 37 78.3 (55.8-92.8)
functioning were indicated for tonsillectomy. Inpatient 18 22 Insufficient data
Overall 89 142 73.2 (52.8, 88.3)
TONS07: Children who had a tonsillectomy and Inpatient 7 7 Insufficient data
adenoidectomy were not administered perioperative Overall 7 7 Insufficient data
antibiotics.
TONS08: Children who had a tonsillectomy and Inpatient 7 7 Insufficient data
adenoidectomy were given a stat dose of Overall 7 7 Insufficient data
dexamethasone.
TONS09: Children who had a tonsillectomy and Inpatient 7 7 Insufficient data
adenoidectomy were prescribed antiemetic medication Overall 7 7 Insufficient data
postsurgery.
TONS10: Children who had a tonsillectomy and Inpatient 7 7 Insufficient data
adenoidectomy were informed of the potential for pain Overall 7 7 Insufficient data
to increase for up to 6 d postsurgery.
TONS11: Parents/carers of children who had a Inpatient 7 7 Insufficient data
tonsillectomy and adenoidectomy were informed of the Overall 7 7 Insufficient data
risk of postoperative hemorrhage: primary (within 24 h)
and secondary (4-9 d) after surgery.

Abbreviations: ED, emergency department; GABHS, group A beta-hemolytic streptococci; GP, general practitioner.

Appendix 2 (available online only). We randomly sampled
patient records from general practices and health districts in
Queensland (Hospital and Health Services), New South
Wales (local health districts), and South Australia (local
health networks) for children aged 15 years receiving
care in 2012 and 2013. Records contained patient interac-
tions in 3 clinical settings: hospital admissions, emergency
department attendances, and GP consultations. Health care
providers electronically sent lists of medical record numbers
of patients whom they identified as having tonsillitis via
codes from the ICD-10-AM (International Classification
of Diseases, Tenth Revision, Australian Modification),
SNOMED, or practice software. The study team then ran-
domly selected from these. For the main CTK study, 34 of
37 (92%) eligible hospitals agreed to participate. For GPs,
the estimated recruitment rate was 24%.
Data Collection
Data were collected by 9 experienced pediatric nurses (sur-
veyors) who underwent 5 days of training and competency
4 Otolaryngology–Head and Neck Surgery

assessment on reviewing medical records with our protocol.
An existing electronic data collection tool was adapted for
conditions and indicators examined in this study. The data
tool filtered out indicators as ‘‘not applicable’’ based on the
participant’s age (eg, .3 years for TONS04) or setting (eg,
TONS07-TONS11 were restricted to inpatients). The sur-
veyors then assessed the remaining indicators for eligibility
based on inclusion criteria. If the indicator was deemed
ineligible, the surveyor recorded ‘‘not applicable’’; other-
wise, the surveyors assessed the adherence to each indicator
as ‘‘yes’’ (care provided was consistent with the indicator)
or ‘‘no’’ (inclusion criteria met but no documented perfor-
mance of the adherence action).
Data Analysis
Adherence was estimated as the proportion of eligible indi-
cators that were scored ‘‘yes.’’ Sampling weights were con-
structed as described in Appendix 4 of the main CTK
article.16 The weighted data were analyzed in SAS/STAT
9.4 (SAS Institute, Cary, North Carolina) with the
SURVEYFREQ procedure. Variance was estimated by
Taylor series linearization, and domain analysis was
applied. Exact 95% CIs were generated with the modified
Clopper-Pearson method, except when the point estimate
was 0% or 100%, where the unmodified Clopper-Pearson
method was used.34 Results were suppressed if there were
\25 eligible indicator assessments. The primary cluster-
sampling unit (health district) was specified as the clustering
unit. At the indicator level, state and health care setting
were specified as strata, but when indicator adherence was
examined by health care setting, only state was specified as
a stratum.
Table 2. Patient Ages from Reviewed Records.
Age, ya Children in the CTK Tonsillitis Study, n (%)
\5 465 (56.6)
5-9 225 (27.4)
10-15 131 (16.0)
Total 821 (100)
Abbreviation: CTK, CareTrack Kids.
a
Age was calculated at the time of the review or at the midpoint between
the first and last tonsillitis visits.
indicated or contraindicated). Five were applicable in all 3
care settings; 1 was applicable in GP and emergency depart-
ment settings only; and 5 (related to tonsillectomy and ade-
noidectomy) were for inpatient settings only.
Medical Records Reviewed
A total of 821 records of children with tonsillitis or sus-
pected tonsillitis who had 1 eligible assessments of indica-
tor adherence for tonsillitis were reviewed (Table 2). The
majority of children (57%) were \5 years of age, with
approximately equal numbers of males (50.5%) and
females. Each child was assessed for 1 to 10 tonsillitis visits
(median and mean = 1) and for 1 to 7 indicator assessments
per visit (median and mean = 2).
Of 13,695 possible tonsillitis indicator assessments, 7415
(54.1%) were filtered by age or health care setting restric-
tions, and a further 3926 (28.7%) were designated as not
applicable or otherwise ineligible by surveyors. The sur-
veyors conducted 2354 eligible indicator assessments
grouped into 1127 visits.

Ethical Considerations Quality of Care

The quality of care, as determined by the adherence to the
We received primary ethics approval from relevant bodies,
indicators, is shown in Table 1. Adherence is not reported
including hospital networks and the Royal Australian
for 5 of the 11 indicators, because they had \25 indicator
College of General Practitioners (HREC/14/SCHN/113;
assessments—the excluded indicators all related to admis-
HREC/14/QRCH/91; HREC/14/WCHN/68; NREEC 14-
sions for tonsillectomies. For the 6 indicators where adher-
008), and site-specific approvals from 34 sites. Australian
ence could be measured, adherence ranged from 14.3% for
Human Research Ethics Committees can waive require-
indicator TONS04 (‘‘Children aged \4 years with a sore
ments for patient consent for external access to medical
throat and associated cough who did not require hospitalisa-
records if the study entails minimal risk to health care pro-
tion were not prescribed antibiotics’’) to 73.2% for TONS06
viders and patients18,35,36; all relevant bodies provided this
(‘‘Children with recurrent acute sore throat with episodes
approval. Ethics approvals included reporting by health care
that were disabling and prevented normal functioning were
setting type for condition-level papers. Participants were
indicated for tonsillectomy’’). Adherence with TONS04
protected from litigation by gaining statutory immunity for
ranged from 12.8% for GPs to 38.6% for emergency depart-
CTK as a quality assurance activity, from the Federal
ments, while TONS02 (‘‘Children with a sore throat and
Minister for Health under part VC of the Health Insurance
with no other symptoms or signs of tonsillitis were not pre-
Act of 1973 (Commonwealth of Australia).
scribed antibiotics’’) ranged from 36.7% for GPs to 92.6%
Results for emergency departments. The interquartile range for
adherence to the 6 indicators reported was 31.5% to 72.2%.
Validated Indicators
We derived a final validated list of 11 indicators that Discussion
describe the appropriate management of tonsillitis (Table 1). We developed a set of validated indicators to measure the
Eight were related to underuse (actions that are recommended care received by patients with confirmed or suspected ton-
but not undertaken) and 3 to overuse (actions that are not sillitis. The main finding from applying the indicators to a
Hibbert et al
sample of patient records is that children with a sore throat
and with no other symptoms or signs of tonsillitis were pre-
scribed antibiotics during most of their visits (59%). This
number increased (86%) if patients were \4 years old, pre-
sented with a cough, but did not require hospital admission.
The temperatures of patients were recorded in approxi-
mately three-quarters of visits, and in approximately a third
of the records, clinicians documented that they advised par-
ents to regularly offer fluids to children. Indicator adherence
varied considerably among the different clinical settings.
GPs were less likely than emergency department clinicians
to record a temperature or parental advice and were also
more likely to prescribe antibiotics. Approximately one-
quarter of patients with serious recurrent episodes of tonsil-
litis were not referred for consideration for tonsillectomy.
The majority of studies about appropriate management of
tonsillitis focus on antibiotic prescribing. Guidelines suggest
that approximately 15% to 30% of children with a sore
throat will have group A streptococcal pharyngitis, in which
case antibiotics are indicated. With electronically linked
clinical and prescribing databases in general practice, 2
studies in the Netherlands37,38 and single studies in
Australia,39 Ireland,40 Norway,41 Spain,42 and Sweden43
found antibiotic prescribing rates for children with acute
tonsillitis ranging from 55% to 94%. CTK used different
methods, had a wider focus, and studied children in differ-
ent geographic settings, but the main finding is comparable;
that is, the treatment of tonsillitis is variable and character-
ized by high antibiotic prescribing.
The available evidence about adherence to tonsillectomy
guidelines is more limited and not specific to the pediatric
population. In Scotland, 87% of tonsillectomies between
2007 and 2008 were performed in line with the guidelines
of the Scottish Intercollegiate Guidelines Network.25,44
However, a study of Israeli military personnel found that
only 1 of 44 patients who had a tonsillectomy between
April and July 2004 met the Paradise criteria45 for surgical
intervention.45,46
Practical Implication and Next Steps
There are many potential reasons for the observed high
rates of inappropriate antibiotic prescribing for tonsillitis:
clinical time constraints; diagnostic uncertainty; clinicians’
perceptions of parental expectations; parental health beliefs
and literacy; the knowledge, skill, and attitude of clinicians;
and risk aversion and avoidance due to, for example, fear of
litigation.47-50 The Australian primary care fee-for-service
model may help mitigate some of these factors, as it pro-
vides opportunities for diagnostic testing and remunerates
general practices to undertake review appointments, which
may allow greater diagnostic certainty and more time with
the patient.5,51
Numerous interventions have been trialed to reduce inap-
propriate antibiotic prescribing, with variable success.49
Intervention components include automatic computerized pre-
scribing prompts, audit and feedback, provider education, and
enhancing provider communication skills.40,42,52 Multifaceted
5
interventions that target providers, patients, and the public in
a variety of venues and formats seem to be more successful
in reducing inappropriate antibiotic prescribing as compared
with simple interventions.52 ‘‘Passive’’ interventions and
interventions directed at patients, such as pamphlets in clinic
waiting rooms, were shown to be ineffective.49,53
The provider-patient interaction is a crucial part of any
intervention. Explanations to patients should provide infor-
mation that is relevant to their situations, with reassurance
and taking the patients’ expectations seriously.49,53 These
can include reassurance that antibiotics are not immediately
necessary, provision of information on symptomatic man-
agement, advice on when to commence antibiotics if symp-
toms persist, and an invitation to reconsult if symptoms
worsen significantly.5,54 Communication training and use of
patient decision support tools can provide physicians with
concise strategies to communicate with their patients, and
they appear to be effective in reducing antibiotic usage.49,55
Our multistage indicator development methodology
aimed to validate recommendations published in CPGs to
the clinical setting. These validated indicators can be used
to measure performance in Australia and potentially interna-
tionally. The set of indicators also has the potential to
inform quality improvement projects at the practice,
regional, and national levels. However, as with CPGs, one
of the challenges is to keep the indicators up-to-date with
contemporary evidence.56 Three of the 11 indicators attempt
to measure inappropriate overuse of antibiotics. This reflects
the growing international recognition of overuse or low-
value care, being wasteful, costly, and potentially harmful
to patients and society as a whole—for example, through
antibiotic resistance.57,58
Structured medical records should be based on agreed stan-
dards of care for common conditions such as tonsillitis.56,59
They should be designed according to best practice human
factors principles that were outlined .25 years ago.60,61
Strengths and Limitations
Our study has several strengths and limitations. A key strength
is the large sample size collected from 113 health care provi-
ders in 3 Australian states. A further strength was the excellent
rate of participation among hospitals that were invited, with
only 3 of 37 (8%) declining; conversely, the high refusal rate
of GPs (76%) leads to a possibility of self-selection, which
may bias the estimated rate of adherence. The set of indicators
was developed and validated through an independent process
that avoided groupthink but incorporated consensus-building
methods. In addition, the application of the indicators and the
review of the medical records were performed with a struc-
tured approach. A weakness of the indicator development pro-
cess was that reviewers were GPs and pediatricians and did
not include ear, nose, and throat surgeons.
One of the criteria assessed by reviewers determining
indicator inclusion was whether care was likely to be docu-
mented. It should nevertheless be acknowledged that care
that was documented may not reflect the care that was
delivered. Care delivered but not documented was reported
6 Otolaryngology–Head and Neck Surgery
has been estimated to account for around 10% of nonadher-
ence.59 Due to self-selection, it is possible that enrolled GPs
were more likely to provide appropriate care than those who
chose not to participate. Another limitation was that insuffi-
cient data were collected to measure the quality (adherence) of
care in relation to tonsillectomies, likely due to the ICD-10-
AM code for acute tonsillitis being used to search for medical
records in hospitals. Given that the hospital readmission rate
for tonsillectomies is relatively high,62 mainly due to dehydra-
tion, bleeding, and pain, determining compliance with these
indicators is an important research gap that remains.63
As compared with retrospective electronic harvesting, the
CareTrack method offers more information about the individual
circumstances of decision making and actions related to clinical
care, such as the circumstances that lead to tests being per-
formed or medications prescribed. This results from the detailed
review and interpretation of records by trained clinical surveyors
to determine whether indicators were met. However, the process
is expensive and logistically difficult with barriers to access,
including ethics and privacy requirements, geographic distances,
and disruption to clinical staff.56 In addition, a high degree of
variability exists in the level of detail, structure, and quality of
the underlying records among organizations or health care pro-
viders.56,64 The potential barriers to access mean that it is likely
to be prohibitively expensive to use the current unstructured
medical records to assess quality of care on an ongoing basis.
Electronic harvesting offers much promise in relation to effi-
ciency of data collection and analysis; however, this relies on
more structure within the medical record.
Our main findings are broadly consistent with the interna-
tional literature, lending further support to the notion that the
management of many children who present with confirmed
or suspected tonsillitis is inconsistent with current guidelines.
In particular, many children are inappropriately prescribed
antibiotics, or, if antibiotics are indicated, there is no clinical
justification documented in patients’ records. In addition,
only a minority of clinicians recorded providing parents with
advice about self-care options. This study therefore provides
a strong rationale for prioritizing interventions to improve the
management of tonsillitis. The set of validated indicators that
we developed could be used for one of the crucial first steps
to achieve this aim, which is to reliably measure clinical per-
formance over time. While unstructured reviews of medical
records are unlikely to be cost-effective or sustainable, struc-
tured and automated reviews of samples of medical records
may provide a feasible and acceptable way forward.
Author Contributions
Peter Hibbert, was program manager of the CareTrack Kids study
and was heavily involved in all aspects of the research and leader-
ship of the study, he conceived and drafted the work with Prof
Braithwaite, revised the work critically for important intellectual
content, analyzed the data, and gave final approval of the version to
be published, agreeing to be accountable for all aspects of the work;
Jacqueline H. Stephens, made substantial contributions to the
design of the work, providing her expertise in the epidemiology of
childhood ear, nose, and throat surgery in Australia, revising the
work critically for important intellectual content and final approval
of the version to be published, agreeing to be accountable for all
aspects of the work; Carl de Wet, made substantial contributions to
the design of the work and provided his expertise as an academic
general practitioner, revising the work critically for important intel-
lectual content and final approval of the version to be published,
agreeing to be accountable for all aspects of the work; Helena
Williams, made substantial contributions to the design of the work,
providing expertise both as a general practitioner and as a key con-
tributor to quality and safety activities in Australia, revising the
work critically for important intellectual content and final approval
of the version to be published, agreeing to be accountable for all
aspects of the work; Andrew Hallahan, made substantial contribu-
tions to the design of the work, providing his expertise as a pediatri-
cian and leader in clinical patient safety and quality improvement
services, revising the work critically for important intellectual con-
tent and final approval of the version to be published, agreeing to be
accountable for all aspects of the work; Gavin R. Wheaton, made
substantial contributions to the design of the work, providing his
expertise and input as a specialist pediatrician, revising the work cri-
tically for important intellectual content and final approval of the
version to be published, agreeing to be accountable for all aspects of
the work; Chris Dalton, made substantial contributions to the
design of the work, providing his clinical expertise as a practicing
pediatric ear, nose, and throat surgeon, revising the work critically
for important intellectual content and final approval of the version
to be published, agreeing to be accountable for all aspects of the
work; Hsuen P. Ting, made substantial contributions to the acquisi-
tion, analysis, and interpretation of data for the work, providing
essential contributions and expertise in biostatistics, revising the
work critically for important intellectual content and final approval
of the version to be published, agreeing to be accountable for all
aspects of the work; Gaston Arnolda, made substantial contribu-
tions to the acquisition, analysis, and interpretation of data for the
work, providing essential contributions and expertise in pediatric
epidemiology, revising the work critically for important intellectual
content and final approval of the version to be published, agreeing
to be accountable for all aspects of the work; Jeffrey Braithwaite,
was chief investigator A on the CareTrack Kids study and responsi-
ble for the leadership and conception of the main study of which
this work originates, he conceived and drafted the work with Mr
Hibbert, revised the work critically for important intellectual con-
tent, analyzed the data, and gave final approval of the version to be
published, agreeing to be accountable for all aspects of the work.
Disclosures
Competing interests: None.
Sponsorships/funding source: This research was funded as an
Australian National Health and Medical Research partnership grant
(APP1065898), with contributions by the National Health and
Medical Research Council, Bupa Health Foundation, Sydney
Children’s Hospital Network, New South Wales Kids and Families,
Children’s Health Queensland, and South Australian Department of
Health (SA Health). The funding organizations approve and endorse
the writing and submission of this manuscript.
References
1. Al-Yaman F, Bryant M, Sargeant H. Australia’s Children:
Their Health and Wellbeing 2002. Canberra, Australia:
Australian Institute of Health and Welfare; 2002.
2. Medicare Australia Statistics. All Medicare Items in ‘‘Group:
A1 General Practitioner, Subgroup: 1 GP Attendances’’
Hibbert et al
Processed from January 2016 to December 2016. Canberra,
Australia: Department of Human Services; 2016.
3. Australian Institute of Health and Welfare National Hospital
Morbidity Database. Procedures and Healthcare Interventions
(ACHI 8th Edition), Australia 2013-14 to 2014-15. Canberra,
Australia: Australian Institute of Health and Welfare; 2014.
4. Independent Hospital Pricing Authority. National Hospital
Cost Data Collection: Australian Public Hospital Cost Report
2013-2014 Round 18. Canberra, Australia: Independent
Hospital Pricing Authority; 2014.
5. Biezen R, Pollack AJ, Harrison C, et al. Respiratory tract
infections among children younger than 5 years: current man-
agement in Australian general practice. Med J Aust. 2015;202:
262-266.
6. Australian Institute of Health and Welfare. Young Australians:
Their Health and Wellbeing 2011. Canberra, Australia:
Australian Institute of Health and Welfare; 2011.
7. Sargent L, McCullough A, Del Mar C, Lowe J. Is Australia
ready to implement delayed prescribing in primary care? A
review of the evidence. Aust Fam Physician. 2016;45:688-690.
8. Dunn N, Lane D, Everitt H, Little P. Use of antibiotics for
sore throat and incidence of quinsy. Br J Gen Pract. 2007;57:
45-49.
9. Georgalas CC, Tolley NS, Narula A. Tonsillitis. BMJ Clin
Evid. 2009;10:503.
10. Ear Nose Throat Infections Expert Group. Acute Pharyngitis
and/or Tonsillitis. Melbourne, Australia: Therapeutic
Guidelines Limited; 2014.
11. Royal Children’s Hospital Melbourne. Sore throat. http://www.
rch.org.au/clinicalguide/guideline_index/Sore_throat/.
Published 2017. Accessed August 15, 2017.
12. Antibiotics Expert Group. Therapeutic Guidelines Antibiotic
Version 15. Melbourne, Australia: Therapeutic Guidelines
Limited; 2014.
13. Royal Children’s Hospital Melbourne. Antibiotics: Clinical
Practice Guidelines. Melbourne, Australia: Royal Children’s
Hospital; 2017.
14. Australian Department of Health. Antimicrobial resistance
(AMR). http://www.health.gov.au/internet/main/publishing.nsf/
Content/ohp-amr.htm. Published 2016. Accessed August 9,
2017.
15. Australian Government. Responding to the Threat of
Antimicrobial Resistance: Australia’s First National Anti-
microbial Resistance Strategy 2015-2019. Canberra, Australia:
Australian Government; 2015.
16. Braithwaite J, Hibbert PD, Jaffe A, et al. Quality of health
care for children in Australia, 2012-2013. JAMA. 2018;319:
1113-1124.
17. Hunt TD, Ramanathan SA, Hannaford NA, et al. CareTrack
Australia: assessing the appropriateness of adult healthcare.
Protocol for a retrospective medical record review. BMJ Open.
2012;2:e000665.
18. Hooper TD, Hibbert PD, Mealing N, et al. CareTrack Kids—
part 2. Assessing the appropriateness of the healthcare deliv-
ered to Australian children: study protocol for a retrospective
medical record review. BMJ Open. 2015;5:e007749.
7
19. Wiles LK, Hooper TD, Hibbert PD, et al. CareTrack Kids—
part 1. Assessing the appropriateness of healthcare delivered to
Australian children: study protocol for clinical indicator devel-
opment. BMJ Open. 2015;5:e007748.
20. Baugh RF, Archer SM, Mitchell RB, et al. Clinical practice
guideline: tonsillectomy in children. Otolaryngol Head Neck
Surg. 2011;144(1):S1-S30.
21. NSW Ministry of Health. Children and Infants with Sore
Throats—Acute Management. Sydney, Australia: Ministry of
Health; 2006.
22. Pelucchi C, Grigoryan L, Galeone C, et al. Guideline for the
management of acute sore throat. Clin Microbiol Infect. 2012;
144(1)(suppl):1-28.
23. Royal Australasian College of Physicians, Australian Society
of Otolaryngology Head and Neck Surgery. Indications for
Tonsillectomy and Adenotonsillectomy in Children. Sydney,
Australia: Royal Australasian College of Physicians and
Australian Society of Otolaryngology Head and Neck Surgery;
2008.
24. The Royal Children’s Hospital Melbourne. Paediatric
Handbook. 8th ed. Melbourne, Australia: Wiley-Blackwell;
2009.
25. Scottish Intercollegiate Guideline Network. Management of
Sore Throat and Indications for Tonsillectomy. Scotland, UK:
NHS Quality Improvement Scotland; 2010.
26. Cardiac Society of Australia and New Zealand, New Zealand
Heart Foundation. Guide for Sore Throat Management.
Wellington, New Zealand: Cardiac Society of Australia and
New Zealand and New Zealand Heart Foundation; 2007.
27. Urquhart DS, Montague ML. When to remove tonsils and the
alternatives. Paediatrics and Child Health. 2012;22:37-41.
28. Fitch K, Bernstein SJ, Aguilar MD, et al. The RAND/UCLA
Appropriateness Method User’s Manual. Santa Monica, CA:
RAND Corporation; 2001.
29. Hemingway H, Crook AM, Feder G, et al. Underuse of coron-
ary revascularization procedures in patients considered appro-
priate candidates for revascularization. N Engl J Med. 2001;
344:645-654.
30. Kravitz RL, Park RE, Kahan JP. Measuring the clinical consis-
tency of panelists’ appropriateness ratings: the case of coron-
ary artery bypass surgery. Health Policy. 1997;42:135-143.
31. Selby JV, Fireman BH, Lundstrom RJ, et al. Variation among
hospitals in coronary-angiography practices and outcomes
after myocardial infarction in a large health maintenance orga-
nization. N Engl J Med. 1996;335:1888-1896.
32. Shekelle PG, Chassin MR, Park RE. Assessing the predictive
validity of the RAND/UCLA appropriateness method criteria
for performing carotid endarterectomy. Int J Technol Assess
Health Care. 1998;14:707-727.
33. Shekelle PG, Kahan JP, Bernstein SJ, Leape LL, Kamberg CJ,
Park RE. The reproducibility of a method to identify the over-
use and underuse of medical procedures. N Engl J Med. 1998;
338:1888-1895.
34. Korn EL, Graubard BI. Confidence intervals for proportions
with small expected number of positive counts estimated from
survey data. Surv Methodol. 1998;24:193-201.
8 Otolaryngology–Head and Neck Surgery
35. National Health and Medical Research Council. National
Statement on Ethical Conduct in Human Research 2007
(Revised March 2014). Canberra, Australia: Commonwealth
Government; 2014.
36. National Health and Medical Research Council. Guidelines
Approved under Section 95A of the Privacy Act 1988 (Revised
March 2014). Canberra, Australia: Commonwealth
Government; 2014.
37. Ivanovska V, Hek K, Mantel Teeuwisse AK, Leufkens HGM,
Nielen MMJ, van Dijk L. Antibiotic prescribing for children in
primary care and adherence to treatment guidelines. J
Antimicrob Chemother. 2016;71:1707-1714.
38. Uijen JH, Bindels PJ, Schellevis FG, van der Wouden JC.
ENT problems in Dutch children: trends in incidence rates,
antibiotic prescribing and referrals 2002-2008. Scand J Prim
Health Care. 2011;29:75-79.
39. McCullough AR, Pollack AJ, Plejdrup Hansen M, et al.
Antibiotics for acute respiratory infections in general practice:
comparison of prescribing rates with guideline recommenda-
tions. Med J Aust. 2017;207:65-69.
40. Murphy M, Bradley CP, Byrne S. Antibiotic prescribing in pri-
mary care, adherence to guidelines and unnecessary
prescribing—an Irish perspective. BMC Fam Pract. 2012;13:
43.
41. Fossum GH, Lindbæk M, Gjelstad S, Dalen I, Kværner KJ.
Are children carrying the burden of broad-spectrum antibiotics
in general practice? Prescription pattern for paediatric outpati-
ents with respiratory tract infections in Norway. BMJ Open.
2013;3:e002285.
42. Malo S, Bjerrum L, Feja C, Lallana M-J, Moliner J,
Rabanaque M-J. Compliance with recommendations on outpa-
tient antibiotic prescribing for respiratory tract infections: the
case of Spain. Basic Clin Pharmacol Toxicol. 2015;116:337-
342.
43. Engström S, Mölstad S, Nilsson G, Lindström K, Borgquist L.
Data from electronic patient records are suitable for surveil-
lance of antibiotic prescriptions for respiratory tract infections
in primary health care. Scand J Infect Dis. 2004;36:139-143.
44. Yeo JCL, Ah-See KW, MacKenzie K. Variation in practice:
an analysis of Scottish surgical profiles ENT data. Scott Med
J. 2013;58:22-25.
45. Paradise JL, Bluestone CD, Bachman RZ, et al. History of
recurrent sore throat as an indication for tonsillectomy. N Engl
J Med. 1978;298:409-413.
46. Katzenell U, Bakshi E, Ashkenazi I, Bar-Dayan Y, Yeheskeli
E, Eviatar E. A retrospective study of the eligibility for tonsil-
lectomy. Isr Med Assoc J. 2010;12:681-683.
47. Belongia EA, Naimi TS, Gale CM, Besser RE. Antibiotic use
and upper respiratory infections: a survey of knowledge, atti-
tudes, and experience in Wisconsin and Minnesota. Prev Med.
2002;34:346-352.
48. Moro ML, Marchi M, Gagliotti C, Di Mario S, Resi D
‘‘Progetto Bambini a Antibiotici’’ Regional Group. Why do
paediatricians prescribe antibiotics? Results of an Italian
regional project. BMC Pediatr. 2009;9:69.
49. van der Velden AW, Pijpers EJ, Kuyvenhoven MM, Tonkin-
Crine SKG, Little P, Verheij TJM. Effectiveness of physician-
targeted interventions to improve antibiotic use for respiratory
tract infections. Br J Gen Pract. 2012;62:e801-e807.
50. Wood F, Simpson S, Butler CC. Socially responsible antibiotic
choices in primary care: a qualitative study of GPs’ decisions
to prescribe broad-spectrum and fluroquinolone antibiotics.
Fam Pract. 2007;24:427-434.
51. Gosden T, Forland F, Kristiansen IS, et al. Capitation, salary,
fee-for-service and mixed systems of payment: effects on the
behaviour of primary care physicians. Cochrane Database Syst
Rev. 2000;(3):CD002215.
52. Arnold SR, Straus SE. Interventions to improve antibiotic pre-
scribing practices in ambulatory care. Cochrane Database Syst
Rev. 2005;(4):CD003539.
53. Vodicka TA, Thompson M, Lucas P, et al. Reducing antibiotic
prescribing for children with respiratory tract infections in pri-
mary care: a systematic review. Br J Gen Pract. 2013;63:
e445-e454.
54. National Institute for Health and Clinical Excellence.
Respiratory Tract Infections—Antibiotic Prescribing.
Prescribing of Antibiotics for Self-limiting Respiratory
Infections in Adults and Children in Primary Care. London,
UK: National Institute for Health and Clinical Excellence;
2008.
55. Cals JW, Butler CC, Hopstaken RM, Hood K, Dinant GJ.
Effect of point of care testing for C reactive protein and train-
ing in communication skills on antibiotic use in lower respira-
tory tract infections: cluster randomised trial. BMJ. 2009;338:
b1374.
56. Runciman WB, Coiera EW, Day RO, et al. Towards the deliv-
ery of appropriate health care in Australia. Med J Aust. 2012;
197:78-81.
57. Keyhani S, Falk R, Howell EA, Bishop T, Korenstein D.
Overuse and systems of care: a systematic review. Med Care.
2013;51:503-508.
58. Saini V, Brownlee S, Elshaug AG, Glasziou P, Heath I.
Addressing overuse and underuse around the world. Lancet.
2017;390:105-107.
59. McGlynn EA, Asch SM, Adams J, et al. The quality of health
care delivered to adults in the United States. N Engl J Med.
2003;348:2635-2645.
60. Zahabi M, Kaber DB, Swangnetr M. Usability and safety in
electronic medical records interface design. Hum Factors.
2015;57:805-834.
61. Molich R, Nielsen J. Improving a human-computer dialogue.
Commun ACM. 1990;33:338-348.
62. Australian Safety and Efficacy Register of New Interventional
Procedures–Surgical. Rapid Review: Tonsillectomy,
Adenoidectomy and Adenotonsillectomy for Obstructive Sleep
Apnoea: Review of Clinical Evidence and Guidelines.
Adelaide, Australia: ASERNIP-S and Royal Australasian
College of Surgeons; 2015.
63. Johnson RF, Chang A, Mitchell RB. Nationwide readmissions
after tonsillectomy among pediatric patients—United States.
Int J Pediatr Otorhinolaryngol. 2018;107:10-13.
64. Hibbert PD, Molloy CJ, Hooper TD, et al. The application of
the Global Trigger Tool: a systematic review. Int J Qual
Health Care. 2016;28:640-649.