COVID-19

Analytical performances of SARS-Co-2 diagnostic tests

Mariano Bizzarri
Dept. of Experimental Medicine
Systems Biology Group,
WASPALM - Board
University La Sapienza
PATHOLOGY
 Coronaviruses form enveloped and spherical particles of 100e160 nm in diameter.
 They contain a positive sense single stranded RNA (ssRNA) genome of 26e32 kb in size.
 In SARS-CoV, MERS-CoV and SARS-CoV-2, the 50-terminal two-thirds of the genome
ORF1a/b encodes polyproteins, which form the viral replicase transcriptase complex.
 The other ORFs on the one-third of the genome encode four main structural proteins:
spike (S), envelope (E), nucleocapsid (N) and membrane (M) proteins, as well as several
accessory proteins.
 Schematic representation of the progression of COVID-19
infection and potential adjuvant interventions.
 After an incubation period, the invading COVID-19 virus
causes non-severe symptoms and elicits protective
immune responses.
 The successful elimination of the infection relies on the
health status and the HLA haplotype of the infected
individual. In this period, strategies to boost immune
response can be applied.

If the general health status and the HLA haplotype of

the infected individual do not eliminate the virus, the
patient then enters the severe stage, when strong
damaging inflammatory response occurs, especially
in the lungs.
At this stage, inhibition of hyaluronan synthase and
elimination of hyaluronan can be prescribed.
Cytokine activated mesenchymal stem cells can be
used to block inflammation and promote tissue
reparation.
 Isolation and antigenic characterization of
2019-nCoV.
 a, b, Vero E6 cells are shown at 24 h after
infection with mock virus (a) or 2019-nCoV
(b). c, d, Mock-virus-infected (c) or 2019-
nCoV-infected
 (d) samples were stained with rabbit serum
raised against recombinant SARSr-CoV Rp3
N protein (red) and DAPI (blue).
 The experiment was conducted twice
independently with similar results.
 e, The ratio of the number of reads related to
2019-nCoV among the total number of virus-
related reads in metagenomics analysis of
supernatants from Vero E6 cell cultures.
 f, Virus growth in Vero E6 cells.
 g, Viral particles in the ultrathin sections were
imaged using electron microscopy at 200 kV.
The sample was from virus-infected Vero E6
cells. The inset shows the viral particles in an
intra-cytosolic vacuole.
  On gross examination
(Figure 1), the surface of
the whole lung was
bronzing, and showed
diffuse congestive
appearance.
 Most of them had punctate
hemorrhage and partly
haemorrhagic necrosis.
The haemorrhagic necrosis
was mainly present in outer
edge of the right lower lobe,
middle lobe and upper lobe
of the lung.
 The bronchi had swollen
mucosal surfaces covered
with haemorrhagic
exudation.
 The cut surfaces of the
lung displayed severe
congestive and
haemorrhagic changes.
LUNG PATHOLOGY
  Histopathological findings showed
extensive pulmonary interstitial
fibrosis with partly hyaline
degeneration, and pulmonary
hemorrhagic infarct.
 Small vessels showed hyperplasia,
vessel wall thickening, and lumen
stenosis and occlusion.
 Interstitial infiltration of inflammatory
cells including lymphocytes, plasma
cells and mononuclear cells.
 On the other side, pulmonary
interstitial fibrosis was confirmed by
Masson staining, and no other
bacterial and fungal infections were
found by special staining.

HISTOLOGY
  a Case 1: thick hyaline
 membrane mixed with
desquamative pneumocytes and
mononuclear inflammatory cells.
 b Case 2: more delicate hyaline
membranes without evident
inflammatory infiltration.
 c Case 3: focal hyaline
membrane, type II pneumocyte
hyperplasia, and mild interstitial
thickening.
 d Case 4: alveolar spaces were
filled with red blood cell
exudation, and small fibrin plugs
seen in adjacent alveoli.
 e Organization with intra-alveolar
fibroblasts mixed with fibrin and
inflammatory cellular infiltration.
Diffuse type II pneumocyte
hyperplasia in the background
(inset: fibrinoid vascular necrosis,
arrow heads).
 f Changes of bronchopneumonia
with prominent neutrophilic
infiltration filling up alveolar
spaces.

HISTOLOGIC CHANGES IN THE LUNGS

  Representative images of chest computed tomography scan.
 (A) Case 1: image on postoperative day 1 revealing changes in the
right lung and increased ground-glass opacities bilaterally (arrows);
 (B) case 2: foci of ground-glass opacity seen bilaterally (arrows).

CLINICS – RADIOLOGICAL FINDINGS

  Chest CT from Case1
showed multiple patchy
Ground-glass opacity
(GGO) in the bilateral
upper lobes of the lungs
and appeared more
prominent in the right
upper lobe (RUL)(A1).
 Repeat CT showed
similar changes as A1 but
with thickened fascicles
of vasculatures and
bronchi (A2).

 In Case 2, patchy GGO,

consolidation, and air
bronchogram can be
seen in RUL; scattered
GGO can be identified in
the left upper lobe (LUL)
(B1).
 Repeat CT showed
additional consolidation in
LUL (B2).

Radiographic images of chest CT scan and X-ray from the four patients.
For each patient, the left and right images represent an earlier and latter time-point, respectively.
 In Case 3, X-ray
showed patchy high-
density shadows in
both lungs, which
were more prominent
in the lower lobes
(C1) and worsened
during the couple of
days before death
(C2).

 In Case 4, diffuse
GGO is seen in both
lungs as well as
consolidation in the
posterior segment
(D1), and additional
air bronchogram can
be detected in the
later radiography
(D2).
 (A) The elevated baseline IL-6 was correlated with the severity assessed by chest computed
tomography (CT) scan, and the decrease in IL-6 after treatment was positively correlated with the
improvement in chest CT images

Variation in IL-6 and chest computed tomography in severe COVID-19

patients
 Correlation between baseline IL-6 level and clinical features of severe COVID-19 patients. (A) The baseline IL-6
level was positively correlated with the maximal body temperature during hospitalization.
 (B-E) The baseline IL-6 level was positively correlated with C-reactive protein (CRP), lactate dehydrogenase
(LDH), ferritin and D-dimer.
 Il cosiddetto “citokine storm” o  Lo studio è estremamente rilevante perché
“tempesta citochinica” si riferisce a individua un bersaglio terapeutico
una sindrome comune a diverse potenzialmente “salvavita” per le acuzie, tale
patologie ma particolarmente bersaglio potrebbe essere ‘colpito’ anche da
severa nelle patologie respiratorie farmaci con minori effetti collaterali rispetto
acute in cui si produce un aumento al Tolicizumab (e.g. la sostanza naturale
vertiginoso (più di un ordine di inositolo, un normale integratore alimentare
grandezza della concentrazione di mostra un effetto importante su IL6).
varie interleuchine ma soprattutto
della inteleuchina 6 (IL6) che
sembra essere il segnale iniziatore
della ‘tempesta’.
 Quasi tutti i casi gravi di COVID-
19 che hanno bisogno di terapia
intensive ed eventualmente
intubazione presentano questo
quadro.
 Il Tolicizumab è un anticorpo
monoclonale specifico contro il
ricettore di IL6 può quindi essere
molto utile in questi casi

IL-6
DIAGNOSIS
SARS-CoV-2
Envelope
E protein
Nucleocapsid
N protein Spike
S protein

NSP
S protein
viral
RNA

RBD

Membrane
M protein

ACE2
Cell membrane
 The current diagnostic procedures are
twofold.
 First there is the direct detection of (parts
of) the virus. This can be done by culture
of the virus, detection of one or more of
its proteins and, the method used most
frequently during the present pandemic,
direct detection of nucleic acids or
detection via amplification of nucleic
acids. The latter are what are currently
called ‘molecular tests’.

 Immunological tests detect the

consequences of infection by the virus in
the host. This is most frequently focused
on the detection of virus-specific
antibodies, whereas some specialized
laboratories may also be capable of
defining the cellular immune response.
 More than 160 tests have been
approved and provided with the EUA
(Emergency Use Authorization) label,
and more than 80% of these were
molecular tests, with the rest being  In general, antigen tests are
antibody detection assays and a small not as sensitive as molecular
number of antigen detection tests
tests. Due to the potential for
 In comparison with rtPCR, rapid decreased sensitivity compared
antigen detection tests lack sensitivity, to molecular assays, negative
and owing to the increased risk of
false-negative results, they are results from an antigen test may
considered as an adjunct to rtPCR need to be confirmed with a
tests.
molecular test prior to making
 Olfactory tests using electronic ‘noses’ treatment decisions
or even dogs have also been
presented, but these tests are not yet
directly applied in patient care.
 SARS-CoV-2 is an RNA virus, and thus  A number of different primer-probe
all available RNA detection formats can sets for use in SARS-CoV-2
potentially be applied to detect the virus. detection assays and SARS-CoV-2
For adaption towards the more
testing kits have been developed
frequently
and are now available. the
performance characteristics of
 used diagnostic DNA detection formats,
assays using these primer-probe
the viral genome needs to be
transcribed into a DNA complement by sets and testing kits are variable.
reverse transcriptase  While all assays evaluated were
highly specific, some, such as those
 Currently, the preferred SARS-CoV-2 using the CDC N2 and the Corman
test is DNA amplification by PCR, and E-gene sets, were found to be more
the real-time versions of such tests were sensitive than others.
among the earliest available. Such tests
were previously developed during the
emergence of SARS-CoV and Middle
East respiratory syndrome coronavirus
(MERS-CoV),
 - Pre-analytical errors

- Sensitivity and specificity

- False positive results

- False negative results

- (mis)use of analytical data

DIAGNOSTIC ACCURACY
  There is now
incontrovertible evidence
that the preanalytical phase
is the major source of errors
in laboratory testing
Preanalytical issues affecting the diagnosis of COVID-19
PREANALYTICAL ERRORS
– Lack of identification/misidentification
– Inadequate procedures for specimen
(e.g. swab) collection,
handling, transport and storage
– Collection of inappropriate or
inadequate material for quality
or volume
– Presence of interfering substances
– Manual (pipetting) errors
- Sample contamination by external
DNA
- Antiretroviral therapy
  The procedure for collecting
oropharyngeal (e.g., throat) specimens
entails swabbing the posterior pharynx,
avoiding the tongue, and immediate
placement of the swab into another
separate sterile tube, also containing 2–
3 mL of viral transport media

The recommended procedure for collecting a

quality nasopharyngeal specimen entails
 Failure to comply
inserting the swab into the nostril parallel to straightforwardly with the
the palate, maintaining the swab in place for recommended procedures
few seconds for enabling secretion may be a significant cause of
absorption and immediate placement of the
swab into a sterile tube, containing 2–3 mL
diagnostic errors
of viral transport media

.
 The analytical specificity of a molecular
COVID-19 test is its ability to determine
exclusively the analyte it intends to
measure in the presence of off-target
templates or interfering substances.
 For molecular COVID-19 tests, the
quality and relevant abundance of RNA
in collected samples (which is heavily
dependent on the type and site of
collection) are crucial for the sensitivity
of the assays.
 For example, the rate of rtPCR detection
of SARS-CoV-2 in patients with COVID-
19 is as high as 93% in bronchoalveolar
lavage fluid. Yet is 72% in sputum and
63% in nasopharyngeal swabs, while it
is only 32% in pharyngeal swabs and
29% in stool.
 Though a number of studies had to
use target material from cultures of
Vero cells or synthetic viral DNA
fragments owing to the regulatory
inability to access samples from the
early infected populations in China
 For none of the currently used
COVID-19 tests is the absolute
sensitivity (RNA genomes per
millilitre) known because there
simply is not a clear gold standard
for testing available for a pathogen
that has been known for about half a
year

SENSITIVITY and SPECIFICITY

  In general, the limitations of many of
 As the current gold standard the published validation studies for
for the etiological diagnosis of COVID-19 diagnostic tests were low
sample numbers, the differences in
SARS-CoV-2 infection is (real the processes for collection, storage
time) reverse transcription and processing of samples before
polymerase chain reaction the diagnostic tests (preanalytical
(rRT-PCR) on respiratory tract bias) and the lack of validation by
specimens, the diagnostic independent third parties
accuracy of this technique
 Finally, it is very important to note
shall be considered a that no biological tests have 100%
foremost prerequisite. sensitivity and 100% specificity,
which needs to be considered when
diagnostic results are translated into
clinical practice

RELIABILITY of RT-PCR TESTS

 Overall, 88% (888/1014) of patients
had positive chest CT scans, whilst
RT-PCR positivity was found in 59%
(601/1014) of all cases. As many as
34.7% of patients with positive chest
CT findings had negative RT-PCR
results of throat swab samples
 As many as 93% of all patients
whose RT-PCR became positive for
SARSCoV-19 after an initially
negative test result had CT features
suggestive of COVID-19, with a
mean interval period of 5.1 ± 1.5
 TIME-WINDOW RELIABILITY
The generation of false-positive or
false-negative test results not only
jeopardizes the health of the individual
patient but may also derange and
disrupt the efficacy of public health
policies, emergency plans and
restrictive measures established by
national and international authorities
for containing the outbreak.

FALSE NEGATIVE RESULTS

 Negative
The CONUNDRUM: the window during which polymerase chain IgG
Positive viral load
reaction (PCR) detects infections before infectivity (green) is
short, whereas the corresponding postinfectious but PCR-
detectable window (purple) is long
Symptoms onset IgM

Virus not found

in culture
Low sensitivity methods

High sensitivity methods (PCR)

False negative Infectious False positive

weeks from
the infection
1 2 3 4 5 6 7

 false positive results can occur with
antigen tests, including when users do
not follow the instructions for use of
antigen tests for the rapid detection of
SARS-CoV-2
 If the test components are not stored
properly, this can affect the performance
of the test
 Reading the test before or after the
specified time could result in false
positive or false negative results
 Be careful to minimize the risks of cross-
contamination when testing patient
specimens, which can cause false
positive results
FALSE POSITIVE RESULTS
o For example, a test with 98% specificity
would have a PPV of just over 80% in a
population with 10% prevalence, meaning 20
out of 100 positive results would be false
positives.
The same test would only have a PPV of
o
approximately 30% in a population with 1%
prevalence, meaning 70 out of 100 positive
results would be false positives. This means
that, in a population with 1% prevalence,
only 30% of individuals with positive test
results actually have the disease.
o At 0.1% prevalence, the PPV would only be
4%, meaning that 96 out of 100 positive
results would be false positives.
 Health care providers should take the
local prevalence into consideration
when interpreting diagnostic test
results: as disease prevalence
decreases, the percent of test results
that are false positives increase
Even if a test were 98%
sensitive and 99% specific,
it would still produce a false
negative result in 2 of every
100 people infected. If we
test 5 million Americans
daily and only 1% of them
have Covid-19, a total of
1000 positive cases will be
missed, which increases the
risk of spread, and another
49,500 people will receive
false positive results.
  Antibody testing could also help to address a
potential unintended consequence of receiving
convalescent plasma or hyperimmune globulin.
 It’s possible that some COVID-19 survivors who
undergo these treatments won’t develop their
own immunity, putting them at risk for reinfection.
 Serologic testing could be used to check their
antibody status after they’ve recovered; those
with low or no immunity would be prime
candidates for a vaccine when one becomes
available.
 Most people with SARS-CoV-2 don’t start
producing antibodies - or seroconvert -until at
least 11 to 12 days after symptom onset.
 Not only are antibody tests likely to report false-
negatives early on, they’ll also miss infections
among people who are immunocompromised
and don’t produce antibodies

ANTIBODIES
Lateral flow test
 Diagnostics can be used in various
manners, the so-called use cases
 triage of symptomatic individuals in
an epidemic or endemic setting,
triage of at-risk presymptomatic and
symptomatic individuals in endemic
settings, confirmatory testing
 The use case determines the
way in which diagnostic tests
are used optimally
WRONG and RIGHT WAYS
 However, In this setting, the
best test is not necessarily
one that determines whether a
person has any evidence of
SARS-CoV-2, but one that
quickly and accurately
identifies individuals who are
capable of transmitting the
infection to others. This
means that a reliable test
should identify those
individuals than can really
transmit the virus
CONTAGIOUSNESS
To capture the nonlinear relationship between days post symptom onset (days) and Ct value, a fractional
polynomial model was used, indicating predictors days2 and days2ln(days). These were fitted in a random
intercept regression model with ln(Ct value) as the outcome variable. Analysis accounted for multiple samples
from the same individuals. The random intercept for individuals was not statistically significant, providing no
evidence for dependencies within person, thus each individual sample was treated as being independent.
A major issue related to the outbreak has been to correlate viral RNA load obtained
after reverse-transcription polymerase chain reaction (RT-PCR) and expressed as
the cycle threshold (Ct) with Contagiousness.

A strong inverse relationship between Ct value and ability to recover infectious

virus has been recorded. The estimated OR of recovering infectious virus
decreased by 0.67 for each unit increase in Ct value (95% CI: 0.58–0.77)
The estimated probability of recovery of virus from samples with Ct > 35 was 8.3%
(95% CI: 2.8%–18.4%).

We should propose a cutoff Ct value and duration of eviction, with a

consensus at approximately Ct >30 and at least 10 days,
respectively
We need common approaches
to validating test design
and performance, regardless of
whether there is an emergency.
Our experience with Covid-19
highlights the need for a common
legislative framework to ensure
that all clinical tests are accurate
and reliable.
CONCLUSIONS
We need to develop a reference panel
of samples with different amounts of
inactivated virus and samples whose
SARS-CoV-2 status is unknown to
developers. This procedure will allows
the determination of relative limits of
detection (LODs).
Once an international reference
standard is finalized, the panel can be
anchored to it and assessments can
be updated to represent absolute LOD.
This approach provides a workable
solution for assessing tests’
comparative performance
The COVID-19 pandemic has repositioned
laboratory medicine at the center of health-care
systems. The further expansion of testing
capacity at the primary care level will be a key
step for the rapid detection and identification of
individuals who have COVID-19 and will thus
help to prevent onward community transmission

TAKE HOME MESSAGE