OFFI

CIAL

Title

The diagnostic accuracy of Gram stain on formalin-fixed paraffin-embedded tissue sections in the
diagnosis of bacterial skin infection.

Abstract

Background and objective: To evaluate the sensitivity, specificity, and likelihood ratios of Gram stain
on formalin-fixed paraffin-embedded sections of skin (the index test) in diagnosing bacterial skin
infection.

Methods: We reviewed a retrospective consecutive series of 168 skin biopsy specimens reported at
our institution wherein histopathological assessment included a Gram stain and fresh tissue was
concurrently submitted for microscopy and culture. The reference standard was defined as
clinicopathological correlation drawing upon all available clinical information

Results: Our sample included 105 cases positive for infection and 63 cases negative for infection. The
index test showed a sensitivity of 0.43 (95% CI 0.29, 0.57), specificity of 0.98 (0.95, 1.01), a positive
likelihood ratio of 21.50, (19.76, 23.24) and a negative likelihood ratio of 0.58, (0.41, 0.75).

Conclusions: The index test has poor sensitivity, and a negative result should not be used as
evidence to exclude infection. In contrast, with excellent specificity, unless the pre-test probability of
infection is relatively low, a positive result would make infection much more likely. The value of the
index test lies in cases where either sterile tissue was not submitted for microbiological studies, or
sterile tissue culture was negative, and there is at least a low to moderate pre-test probability of
infection.

Registration: ANZCTR: ACTRN12621001637831

Introduction

First reported by Christian Gram in 1883, Gram stain has become an important tool for visualisation
and classification of bacteria.(1)

When applied to formalin-fixed paraffin-embedded (FFPE) tissue sections, the potential advantages
of Gram stain include: diagnosing infection when tissue has not been submitted for microbiological
assessment (microscopy and culture);(2) detection of organisms which are difficult to culture such as
rickettsiae and B. henselae (detection rates with Gram stain are low for these organisms);(2,3)
assessing the tissue plane within which bacteria are present (e.g. when culture has isolated
 OFFI
CIAL
organisms which are normal flora or potential contaminants);(2) using staining characteristics and
morphology to determine the pathogenic organism when culture has identified multiple bacterial
species.(2,3)

Its potential limitations include: non-specific retention of stains by debris;(4) lack of contrast
between bacteria (especially Gram-negative bacteria) and background tissue;(4) inability to provide
species specific information;(5) questionable utility in distinguishing bacterial colonization from
genuine infection;(2) the time intensive nature of searching for small numbers of bacteria; and low
diagnostic accuracy.(5)

Gram stain appears to have been accepted as an ancillary test in anatomical pathology laboratories
despite a lack of evidence regarding its test performance. 

In a retrospective study designed to evaluate S pneumoniae immunohistochemistry (IHC), Guarner et

al evaluated 46 cases (26 positive for infection) from a combination of pre- and post-mortem tissues
from a variety of organ systems. Cases were included if S pneomoniae IHC was performed, and Gram
stain was performed on blocks where inflammation was identified. Using tissue culture as the
reference standard, Gram stain showed a sensitivity of 0.61 and specificity of 0.50.(5) 

Renshaw reviewed a retrospective series of 110 lung biopsy specimens to compare the sensitivity of
Gram stain against tissue culture.(6) The reported sensitivity of Gram stain was 100%, but the
number of cases positive for infection was low (n = 5), and the author selectively reinterpreted index
test results in cases with positive culture, positive special stain, or histological features of infection.

In evaluating concordance between skin histopathology and culture with a retrospective series of
179 cases, Shaigany et al reported that histopathological assessment ± histochemical stains had a
sensitivity of 0.38 and specificity of 0.96, using each patient's ultimate diagnosis as the reference
standard.(7) The authors combined all methods of histopathological assessment (H&E, PAS-D, Gram,
ZN and Fite) and all types of infection (bacterial, mycobacterial, fungal, and parasitic) and the specific
test performance of Gram stain was not reported.

High quality evidence regarding Gram stain test performance will assist anatomical pathologists in
evaluating the appropriateness of performing this test and aid dermatologists in interpreting
histopathology reports. As such, the aim of this study was to evaluate the performance (sensitivity,
specificity, and likelihood ratios) of Gram stain on FFPE tissue sections of skin (i.e. the index test) in
the diagnosis of bacterial skin infection. To estimate our target sample size, our hypothesis was that
the index test would show poor performance as defined by a positive likelihood ratio (LR+) of less
 OFFI
CIAL
than 2 and a negative likelihood ratio (LR-) of more than 0.5. Our objective was achieved by
comparing the index test result against the result of sterile fresh tissue microbiological studies in
concert with clinicopathological correlation (i.e. the reference standard).

Methods

Ethics approval was granted by the Central Adelaide Local Health Network Human Research Ethics
Committee, and this study was made possible through in-kind support from SA Health staff.

The pre-specified research protocol was registered with Australian New Zealand Clinical Trials
Registry in November 2021, ACTRN12621001637831, accessible at
https://www.australianclinicaltrials.gov.au/

Study design and participants

Using our Enterprise Pathology Laboratory Information System (EPLIS) database, we retrieved a
retrospective consecutive series of all skin specimens reported at our institution for which
histopathological assessment included a Gram stain, with a date range extending from the activation
of this database (September 2017) to the start of this study (January 2022). Our institution serves
patients from metropolitan and rural areas of South Australia in both outpatient and inpatient
settings. Cases were included if skin tissue specimens were concurrently submitted for
microbiological assessment. Cases were excluded if: the result of the Gram stain was not available
(i.e. the result was not described in the anatomical pathology report); the patient had an infection
due to bacteria which cannot reasonably be identified using Gram stain (e.g. mycobacteria) or fungi;
or the case notes did not include sufficient information to perform clinicopathological correlation.

Test methods

The index test is performed at our institution by bringing sections to water, flooding with crystal
violet (1 minute), wash (with water), flooding with Gram’s iodine (1 minute), wash, decolorization
with acetone (approximately 1 - 2 seconds), wash, counterstaining with 1% safranin (9 seconds),
dehydration, clearing, and mounting. The crystal violet-iodine complex binds to phosphate groups
on bacterial nucleotides. Gram-positive bacteria retain the stain due to the amount and
configuration of peptidoglycan in their cell wall and appear purple, whereas Gram-negative bacteria
are decolorized by the clearing agent and appear red due to the counterstain.(8–10)
 OFFI
CIAL
The index test was recorded as positive if the anatomical pathology report described visualization of
bacteria on Gram-stained sections. Equivocal index test results (for example, “There are small
numbers of scattered Gram-positive staining circular bodies”) were recorded as negative.

The reference standard was defined as clinicopathological correlation after incorporating all
available clinical information (except index test results), including blood test (namely white cell
count (WCC) and C-reactive protein (CRP)), tissue microbiological studies and, ultimately, the
diagnosis documented in each patient’s case notes.

Tissue culture had been considered as an alternative reference standard but has shown suboptimal
test performance in diagnosing bacterial skin infection. In a review of 16 studies including 808
patients with cellulitis, tissue culture (via fine needle aspiration or punch biopsy of skin) was positive
in just 15.7 - 16.0% of cases.(11)

The amount of clinical information available to those interpreting the index test (i.e. anatomical
pathologists) varied and was not consistently documented. By virtue of the retrospective nature of
this study and the definition of the reference standard (i.e. clinicopathological correlation),
pathologists did not have access to reference standard results. In contrast, those who assessed the
reference standard (i.e. study investigators) had access to clinical information and index test results.

Other pre-specified data for collection included: specimen size (e.g. punch vs incisional biopsy), prior
use of antibiotics prior, anatomical site of specimen, semi-quantitative assessment of bacterial load
on microscopy and culture, and the isolated pathogen(s). Data with potential to reflect disease
severity included WCC, CRP, and whether the patient: was treated with topical, oral, or parenteral
antibiotics; was treated in intensive care unit (ICU); or died from disease. Other factors with
potential to affect test performance could include reagent batch variation, laboratory technician
experience, anatomical pathologist experience, and time spent evaluating Gram stain, however,
because this information is not consistently documented, we did not attempt to record it.

Analysis

With an estimated sensitivity of 0.61,(5) upon setting alpha at 0.05 and accepting a margin of error
of 0.11, the target number of cases positive for infection was 75. With an estimated prevalence of
skin and soft tissue infection among hospitalized patients of 7 - 10%,(12) the proposed target sample
size for calculating sensitivity was 750 cases. With an estimated specificity of 0.50,(5) accepting a
margin of error of 0.05, the target number of negative cases was 384, and the estimated total
sample size for calculating specificity was 426.(13) Deferring to the larger of these two estimates,
 OFFI
CIAL
our target sample size was 750. The target sample size was also checked against formulae relevant
to estimated likelihood ratios (as per our research hypothesis), and this yielded similar sample size
targets to those described above.(14)

Point estimates for sensitivity and specificity were calculated by constructing a standard 2 x 2
contingency table (Table 1) allowing subsequent calculation of LR+ and LR-. 95% confidence intervals
were calculated using formulae relevant to each test performance parameter.(14–16)

Results

Participants

Between 4th September 2017 and 14th January 2022, Gram stain was performed on 749 skin biopsy
specimens. Of these, sterile tissue was submitted for concurrent microbiological assessment in 193
cases. 14 cases were excluded due to insufficient clinical information to perform clinicopathological
correlation, 3 due to Gram stain not being described in the anatomical pathology report, and 8
because of infection due to pathogen(s) which cannot reasonably be identified using Gram stain. The
final sample size was 168. (Figure 1)

Figure 1. Flow of participants diagram. FFPE, formalin-fixed paraffin-embedded tissue

The study included 115 male and 53 female cases, with a mean age of 66 years (range 2 - 91). The
most common clinical presentations were ulcer or wound (n = 43, 26%), suspected soft tissue
infection (n = 35, 21%), and pustular or bullous rashes (n = 19, 11%). Other clinical presentations
included discrete lesion (n = 16), panniculitis (n = 8), and petechial or purpuric rashes (n = 6).

Among cases scored as positive for infection (n = 105), 72 (68%) had features of severe disease
including 19 cases of necrotising soft tissue infection. Nineteen (18%) required treatment in ICU for
 OFFI
CIAL
their infection, while 53 (50%) other cases were treated with intravenous antibiotics. Three patients
passed away due to their illness.

Among cases negative for infection, the most common diagnoses were neutrophilic dermatoses (n =
18, 29%) and non-infectious wounds (16, 25%).

The time interval between the index test and the point at which treating clinicians would have been
able to complete clinicopathological correlation varied, but in general was less than one week.

Test results

Table 1 shows index test results in relation to the reference standard. Gram stain showed a
sensitivity of 0.43 (95% CI 0.29, 0.57), a specificity of 0.98 (95% CI 0.95, 1.01), LR+ of 21.50, (95% CI
19.76, 23.24) and LR- of 0.58, (95% CI 0.41, 0.75).

Reference standard
+ +
+ 45 1
Index test

- 60 62

Table 1. 2 x 2 contingency table comparing results of the index test (Gram stain on Formalin Fixed
Paraffin Embedded sections of skin) against the reference standard (clinicopathological correlation)
in the diagnosis of bacterial skin infection. + positive test result, - negative test result.

Predictive values are not reported as the prevalence of infection in our sample (i.e. cases where the
pathologist opted to perform Gram stain on FFPE tissue and the clinician opted to submit fresh
tissue for culture) may not be representative of the population where the index test is usually
applied (i.e. any case where the pathologist opts to perform Gram stain). The prevalence of infection
in our sample was 62.5%.

Data pertaining to adverse events from obtaining tissue for histological and microbiological
assessment were not recorded. One would expect these to be similar to those of obtaining skin
tissue specimens for other indications.
 OFFI
CIAL
Discussion

This study used a large sample of cases spanning a range of healthcare settings to evaluate the
diagnostic accuracy of Gram stain on FFPE tissue sections in the diagnosis of bacterial skin infection
and is reported in accordance with STARD 2015 guidelines.(17,18)

A number of sources of potential bias have been considered including selection, spectrum, and
verification biases.

First, the prevalence of infection in our sample was relatively high, and our inclusion criteria appears
to have captured a cohort of patients with a higher pre-test probability of infection. Although this
potential selection bias could, in principle, lead to higher sensitivity results, the sensitivity of the
index test remained poor.

The rate of severe infection was relatively high, and although this type of spectrum bias could
exaggerate sensitivity, the sensitivity of the index test remained poor. Of interest, among cases of
severe infection (n = 72), the index test was positive in 32 cases, equating to a sensitivity of 0.44.

Although verification bias could exaggerate all parameters of test performance, the amount of
clinical information provided to or sought by reporting pathologists was variable, not consistently
documented, and could not be recorded in a consistent manner.

The handling of equivocal index test results (i.e. being recorded as negative) had potential to
decrease estimated sensitivity and increase specificity. This occurred in two cases: one case of
neutrophilic dermatosis of the dorsal hand (scored as true negative rather than false positive), and
one case of S aureus panniculitis (recorded as false negative rather than true positive). Cases with an
equivocal reference standard (i.e. insufficient clinical information to perform adequate
clinicopathological correlation) were excluded (n = 14), but we do not believe this resulted in
significant bias.

We did not attempt to distinguish between test performance for Gram-positive and -negative
bacteria, but we predict that index test sensitivity would be lower for Gram-negative bacteria due to
lack of contrast with background tissue.(2,3)

We reached our target number of cases positive for infection (n = 105, target 75), and the margin of
error for sensitivity was 0.14 (target 0.11). Although 384 cases negative for infection was proposed,
the margin of error for specificity was similar to our target (0.03 vs 0.05).
 OFFI
CIAL
Although the inclusion criteria might have produced a cohort of patients with a higher pre-test
probability for infection, we expect the findings of this study to have reasonable generalizability to
routine practice. We suspect the findings of this study could be extrapolated to specimens from
other organ systems (e.g. lung specimens), but further study may be warranted.

Power et al highlight that a test with a combined sensitivity and specificity of 1 is useless, and argue
that for a test to be useful this value should be at least 1.5. In our study, the index test had a
combined sensitivity and specificity of 1.41 (95% CI 1.24, 1.58). According to thresholds reported by
Schechter and Sheps, the index test LR+ could be described as good and the LR- as poor.(19)

Our results indicate that Gram stain on FFPE sections of skin has poor sensitivity, and a negative
result should not be used as evidence to exclude infection. Formalin is bactericidal and can affect cell
wall structure, and these factors might account for the poor sensitivity of the index test.(20,21) In
contrast, the index test has excellent specificity, such that unless the pre-test probability of infection
is relatively low, a positive result would make a diagnosis of infection much more likely.

In reviewing case notes for this study, our impression was that among cases which were positive for
infection, the index test had little to no bearing on the overall diagnosis. That is, the diagnosis of
infection was substantiated on the basis of other clinical and pathological parameters.

Some authors have proposed that when tissue culture is positive, Gram stain could be applied to
FFPE to distinguish commensal from pathogenic bacteria, however, its sensitivity is such that its
results could be misleading in these situations. Similar issues should be considered if the index test
were used to identify a causative pathogen when culture had identified multiple bacteria. Gram stain
is unlikely to be helpful for early identification of organisms which are slow to culture, and tests such
as Warthin-Starry stain, PAS-D, immunohistochemistry or PCR are more likely to be relevant,
depending on the suspected pathogen. In cases where tissue was not submitted for culture, a
positive Gram stain result might have some relevance to antibiotic selection based on pathogen
morphology.

Our overall impression is that the value of the index test lies in cases where either sterile tissue was
not submitted for microbiological studies or sterile tissue culture was negative, provided there is at
least a low to moderate pre-test probability for infection, noting that although a positive result
would make a diagnosis of infection much more likely, a negative result should not be used as
evidence to exclude infection.
 OFFI
CIAL
References

1. Bartholomew JW, Mittwer T. The Gram stain. Bacteriol Rev. 1952 Mar;16(1):1–29.

2. Procop GW, Wilson M. Infectious disease pathology. Clin Infect Dis Off Publ Infect Dis Soc
Am. 2001 Jun 1;32(11):1589–601.

3. Woods GL, Walker DH. Detection of infection or infectious agents by use of cytologic and
histologic stains. Clin Microbiol Rev. 1996 Jul;9(3):382–404.

4. Becerra SC, Roy DC, Sanchez CJ, Christy RJ, Burmeister DM. An optimized staining technique
for the detection of Gram positive and Gram negative bacteria within tissue. BMC Res Notes. 2016
Apr 12;9:216.

5. Guarner J, Packard MM, Nolte KB, Paddock CD, Shieh WJ, Tondella ML, et al. Usefulness of
immunohistochemical diagnosis of streptococcus pneumoniae in formalin-fixed, paraffin-embedded
specimens compared with culture and gram stain techniques. Am J Clin Pathol. 2007
Apr;127(4):612–8.

6. Renshaw AA. The relative sensitivity of special stains and culture in open lung biopsies. Am J
Clin Pathol. 1994 Dec;102(6):736–40.

7. Shaigany S, Steuer A, Seminara N, Brinster N, Femia A. Comparison between organismal

staining on histology and tissue culture in the diagnosis of cutaneous infection: A retrospective
study. J Am Acad Dermatol. 2020 Jun;82(6):1400–8.

8. Salton MR. The relationship between the nature of the cell wall and the Gram stain. J Gen
Microbiol. 1963 Feb;30:223–35.

9. Davies JA, Anderson GK, Beveridge TJ, Clark HC. Chemical mechanism of the Gram stain and
synthesis of a new electron-opaque marker for electron microscopy which replaces the iodine
mordant of the stain. J Bacteriol. 1983 Nov;156(2):837–45.

10. Beveridge TJ. Structures of gram-negative cell walls and their derived membrane vesicles. J
Bacteriol. 1999 Aug;181(16):4725–33.

11. Chira S, Miller LG. Staphylococcus aureus is the most common identified cause of cellulitis: a
systematic review. Epidemiol Infect. 2010 Mar;138(3):313–7.
 OFFI
CIAL
12. Ki V, Rotstein C. Bacterial skin and soft tissue infections in adults: A review of their
epidemiology, pathogenesis, diagnosis, treatment and site of care. Can J Infect Dis Med Microbiol J
Can Mal Infect Microbiol Medicale. 2008 Mar;19(2):173–84.

13. Hajian-Tilaki K. Sample size estimation in diagnostic test studies of biomedical informatics. J
Biomed Inform. 2014 Apr;48:193–204.

14. Simel DL, Samsa GP, Matchar DB. Likelihood ratios with confidence: sample size estimation
for diagnostic test studies. J Clin Epidemiol. 1991;44(8):763–70.

15. Hess AS, Shardell M, Johnson JK, Thom KA, Strassle P, Netzer G, et al. Methods and
recommendations for evaluating and reporting a new diagnostic test. Eur J Clin Microbiol Infect Dis
Off Publ Eur Soc Clin Microbiol. 2012 Sep;31(9):2111–6.

16. Ying GS, Maguire MG, Glynn RJ, Rosner B. Calculating Sensitivity, Specificity, and Predictive
Values for Correlated Eye Data. Invest Ophthalmol Vis Sci. 2020 Sep 1;61(11):29.

17. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an
updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015 Oct
28;351:h5527.

18. Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015
guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016
Nov 14;6(11):e012799.

19. Schechter MT, Sheps SB. Diagnostic testing revisited: pathways through uncertainty. Can
Med Assoc J. 1985 Apr 1;132(7):755–60.

20. Sagripanti JL, Eklund CA, Trost PA, Jinneman KC, Abeyta C, Kaysner CA, et al. Comparative
sensitivity of 13 species of pathogenic bacteria to seven chemical germicides. Am J Infect Control.
1997 Aug;25(4):335–9.

21. Fukunaga H, Murakami T, Gondo T, Sugi K, Ishihara T. Sensitivity of Acid-Fast Staining for
Mycobacterium tuberculosis in Formalin-fixed Tissue. Am J Respir Crit Care Med. 2002 Oct
1;166(7):994–7.