Papers in Press. Published October 23, 2018 as doi:10.1373/clinchem.2016.
266536
The latest version is at http://hwmaint.clinchem.aaccjnls.org/cgi/doi/10.1373/clinchem.2016.266536
Clinical Chemistry 65:3
000 – 000 (2019)
Next-Generation Sequencing for Biodefense:
Biothreat Detection, Forensics, and the Clinic
Timothy D. Minogue,1*† Jeffrey W. Koehler,1† Christopher P. Stefan,1† and Turner A. Conrad1
BACKGROUND: Next-generation sequencing (NGS) is
revolutionizing a variety of molecular biology fields in-
cluding bioforensics, biosurveillance, and infectious dis-
ease diagnostics. For pathogen detection, the ability to
sequence all nucleic acids in a sample allows near limitless
multiplexability, free from a priori knowledge regarding
an etiologic agent as is typically required for targeted
molecular assays such as real-time PCR. Furthermore,
sequencing capabilities can generate in depth genomic
information, allowing detailed molecular epidemiologi-
cal studies and bioforensics analysis, which is critical for
source agent identification in a biothreat outbreak. How-
ever, lack of analytical specificity, inherent to NGS, pres-
ents challenges for regulated applications such as clinical
diagnostics and molecular attribution.
CONTENT: Here, we discuss NGS applications in the con-
text of preparedness and biothreat readiness. Specifically, we
investigate current and future applications of NGS technol-
ogies to affect the fields of biosurveillance, bioforensics, and
clinical diagnostics with specific focus on biodefense.
SUMMARY: Overall, there are many advantages to the im-
plementation of NGS for preparedness and readiness
against biowarfare agents, from forensics to diagnostics.
However, appropriate caveats must be associated with
any technology. This includes NGS. While NGS is not
the panacea replacing all molecular techniques, it will
greatly enhance the ability to detect, characterize, and
diagnose biowarfare agents, thus providing an excellent
addition to the biodefense toolbox of biosurveillance,
bioforensics, and biothreat diagnosis.
© 2018 American Association for Clinical Chemistry
Following the terrorist attacks of September 11, 2001, an
unidentified individual sent letters containing live Bacil-
1
Diagnostic Systems Division, United States Army Medical Research Institute of Infectious
Diseases, Fort Detrick, MD.
* Address correspondence to this author at: United States Army Medical Research Institute
of Infectious Diseases, Diagnostic Systems Division, 1425 Porter Street, Fort Detrick, MD
21702. Fax 301-619-2492; e-mail timothy.d.minogue.civ@mail.mil.
†
These authors contributed equally to the writing of this review.
Received May 2, 2018; accepted June 22, 2018.
Previously published online at DOI: 10.1373/clinchem.2016.266536
© 2018 American Association for Clinical Chemistry
Copyright (C) 2018 by The American Association for Clinical Chemistry
Reviews
lus anthracis spores to various media outlets and political
figures, causing infections in 22 individuals and 5 deaths
(1 ). This event resulted in (a) a 7-year investigation and
the treatment of over 10000 people with antibiotic pro-
phylaxis, (b) the collection and testing of 120000 clinical
and environmental samples, and (c) the quarantine of 1.8
million mailed items. These attacks resulted in billions of
research dollars being allocated toward extant and nas-
cent biodefense efforts (2 ), including efforts to enhance
bioforensics to support criminal prosecutions. Termed
the Amerithrax case, this investigation was instrumental
in demonstrating capabilities and readiness in the fields
of biosurveillance, clinical diagnostics, and bioforen-
sics in response to a bioterrorism event. While next-
generation sequencing (NGS)2 was not used directly
in this case, the lineage of NGS applications in the
biothreat surveillance and diagnostics context are a
direct result of increased research funding stemming
from this incident.
Staple assays for rapid pathogen detection at the on-
set of a biothreat public health crisis, such as the Ameri-
thrax case, can include PCR assays (3, 4 ), enzyme-linked
immunosorbent assays (5 ), and lateral flow immunoas-
says (6 ). While these approaches are excellent for patho-
gen identification and provide clinically actionable data,
these techniques have limited multiplexability and re-
quire a priori knowledge of the causative agent. These
methods can also leave an information gap concerning
the genetic makeup of an organism, a critical piece to
bioforensics solutions. Sequencing specific genes or
genomic regions can bridge this forensics gap, but, before
the advent of NGS technology, prior detection assays
failed to provide the needed capabilities of producing
high levels of quality genetic information rapidly.
Historically, modern sequencing began in 1977
with the development of Sanger sequencing (7, 8 ) (Fig.
1). Since the introduction of sequencing by synthesis,
advances in the technology have outpaced Moore’s law.
Second-generation technologies followed Sanger in the
mid-2000s, with pyrosequencing, sequencing by liga-
tion, and reversible terminator chemistry sequencing.
This technology development allowed efficient sequenc-
2
Nonstandard abbreviations: NGS, next-generation sequencing; EVD, Ebola virus dis-
ease; GMOs, genetically modified organisms; LDT, laboratory-developed test; FDA, Food
and Drug Administration.
Reviews

Fig. 1. Advancements in sequencing technologies.

Considered the first generation of sequencers (blue), Sanger sequencing in 1977 provided the foundation for all sequencing applications.
New innovations with clonal amplification, parallel sequencing, and sequence by synthesis resulted in the second-generation technologies
(green). The introduction of long-read technologies led to the advent of third-generation sequencers (purple) with PacBio and Nanopore.

ing of all the nucleic acids within a sample, revolutioniz-
ing how researchers approached biothreat and infectious
disease detection. Subsequent technology maturation re-
sulted in the current wave of third-generation sequencers
focusing on single-molecule nucleic acid detection in real
time (Fig. 1). This approach obviates clonal amplifica-
tion requirements common to previous generations of
sequencers, reducing both amplification errors and time
to answer (9 ). As technology continues to evolve, NGS is
beginning to find a role in regulated clinical laboratories
as an in vitro diagnostic. Overall, wide adoption and
application of this technology is transitioning from basic
research applications to more regulated applications such
as biosurveillance, bioforensics, and clinical diagnostics
(Fig. 2).
NGS for Biosurveillance
Biosurveillance is a critical component of emerging infec-
tious diseases and biothreat preparedness. Without cur-
rent, real-time knowledge of the endemic circulating
pathogens, it is impossible to prepare and/or appropri-
ately respond to an outbreak or biological event. In this
context, biosurveillance is defined as gathering, amal-
gamating, and interpreting data to communicate and es-
tablish actionable results for emerging and biothreat dis-
eases. Pathogen-agnostic NGS fits into this current
paradigm for biosurveillance, augmenting and, in some
cases replacing, traditional targeted assays such as real-
time PCR and immunoassays to increase the detection
capability for unexpected or novel organisms.

2 Clinical Chemistry 65:3 (2019)

Biothreat Applications of Next-Generation Sequencing
Fig. 2. Application of NGS for biothreat agents.
The 3 main biothreat NGS applications discussed here include
biosurveillance (blue), bioforensics (green), and clinical use or
diagnostics (purple). For each use case, the general definition is
provided with the projected outputs for any NGS biothreat assays
highlighted. Biosurveillance principally deals with environmen-
tal biothreat detection and surveillance with the mitigation and
tracking of an outbreak in as close to real time as possible. Biofo-
rensics is more focused on meeting evidentiary standards to pro-
vide attributional information. Biothreat clinical NGS diagnostics
is centered on generating actionable information to affect patient
treatment.
The discovery of a new rhabdovirus, Bas Congo vi-
rus, from an acute hemorrhagic fever patient in the Dem-
ocratic Republic of Congo, exemplifies this tiered-assay
approach (10 ). Initial testing by real-time PCR for
known hemorrhagic fever viruses was negative, necessi-
tating NGS for pathogen identification. Deep sequenc-
ing of the patient sample identified Bas Congo virus, a
previously unknown virus, generating a near complete
genome that formed the basis of a specific real-time RT-
PCR diagnostic assay and a pseudotyped virus for sero-
logical studies (10 ). In a similar case from 2008, Briese et
al. used NGS to identify a novel arenavirus, Lujo virus,
from a cluster of acute hemorrhagic fever patients in
South Africa following transport of the index case from
the Zambia (11 ).
NGS Biosurveillance for Biothreats
Perhaps the best illustration of NGS biothreat biosurveil-
lance is the most recent Ebola virus disease (EVD) out-
break in West Africa, where researchers applied NGS in 2
ways: (a) characterizing the genome of the emerging
pathogen, and (b) determining the transmission dynam-
ics of the virus.
Reviews
At the inception of the 2013–2015 outbreak, re-
searchers urgently sought to characterize the emerging
Ebola virus variant to better understand the virus’
genomic plasticity with regard to fielded diagnostic assays
and countermeasures. NGS-based sequencing early in
the outbreak identified a higher degree of diversity in this
isolate than expected, suggesting a negative effect on di-
agnostic assay performance (12 ). A larger characteriza-
tion of circulating Ebola virus isolates showed this plas-
ticity was not as significant as earlier believed (13 ). This
information coupled with empiric evaluations of existing
diagnostics (14 ) found the effect on fielded Ebola virus
assays was minimal. However, the initial genomic char-
acterizations raised critical concerns that, when ad-
dressed, resulted in higher confidence in assay results.
As the outbreak progressed, NGS capabilities be-
came available in EVD-affected countries. Implementa-
tion of NGS in near real time allowed for a better under-
standing of the Ebola virus transmission dynamics and
aided in controlling the outbreak. For example, Christie
et al. (15 ) and Mate et al. (16 ) documented a suspected
case of Ebola virus sexual transmission from an appar-
ently healthy survivor to his female partner. NGS and
genomic analysis confirmed the epidemiological hypoth-
esis of sexual transmission. These findings resulted in
changes to CDC recommendations related to sexual be-
havior of survivors to help end Ebola virus transmission
chains. Postoutbreak NGS sequencing of large infected
cohorts allowed detailed molecular epidemiological
tracking of the different transmission chains and helped
to understand the stability of the Ebola virus genome
during a sustained, large-scale outbreak (17–20 ). In Ni-
geria, for example, genomic sequencing and detailed con-
tact tracing from 20 EVD patients confirmed the Nige-
rian outbreak resulted from a single introduction rather
than multiple introduction events (18 ).
One important concern in the biosurveillance arena
is the potential for accidental or intentional release of
genetically modified biothreats designed for enhanced
pathogenicity, spread, or evasion of fielded countermea-
sures (i.e., diagnostics, therapeutics, and vaccines). Tech-
nology is advancing at a dynamic rate, so much so that
researchers have been able to generate an infectious
horsepox virus from synthetic DNA (21 ); this ability
raises the risk of someone using a similar strategy to re-
constitute other poxviruses. Such methods could be com-
bined with further genetic modifications to increase
pathogenicity, as others demonstrated for myxoma virus
by adding interluken 4 to modulate the host immune
response (22 ). While these technological advances gen-
erally improve public health, this benefit comes with an
increased risk of nefarious use. In cases in which these
manipulations would be unexpected, NGS coupled with
appropriate genome assembly could identify modifica-
Clinical Chemistry 65:3 (2019) 3
Reviews
tions that would be important to guide countermeasure
use and to begin assigning molecular attribution.
There are no known instances of genetically modi-
fied biothreat agents found in circulation; however,
the identification of genetically modified organisms
(GMOs) in the agricultural field provides a framework
for how NGS could be implemented for biothreats. The
European Union has strict GMO regulations and testing
mechanisms in place, such as real-time PCR detection of
GMO-specific signatures (23, 24 ), to determine unau-
thorized GMOs and to ensure compliance with regula-
tions. Rapid development of different GMOs now re-
quires a large number of assays to ensure regulatory
compliance. NGS, using its broad sequencing capability,
is beginning to fill this role (25–27 ). In this context,
Paracchini et al. used NGS to characterize an unauthor-
ized, genetically modified B. subtilis found in feed addi-
tives imported into Germany from China. Their analyses
identified multiple genetic modifications and the inclu-
sion of 4 recombinant plasmids used to increase ribofla-
vin production (25 ). Although agricultural GMOs do
not induce the same level of angst as an Ebola virus out-
break, there is a defined effect on economies and animal
welfare.
Challenges for NGS Biosurveillance
A major challenge for bringing NGS to the point of care
is the fielding of sequencing instruments in austere envi-
ronments where most natural biothreats occur. In the case
of the West Africa EVD outbreak, there were substantial
hurdles for NGS implementation including (a) instrument
shipping, (b) setup and validation of functionality, (c) gen-
eral maintenance, and (d) an uninterrupted power supply to
complete the sequencing run. Furthermore, preparing se-
quenceable libraries that employ enrichment or host back-
ground reduction steps is highly technical, thus reducing
assay utility in resource-limited, austere environments.
While NGS implementation challenges exist, these hurdles
are not insurmountable.
Advances in NGS technologies, sequencing capac-
ity, and analysis methods continue to expand the avail-
ability of this technology. For instance, the portability
and low cost of the MinION nanopore sequencer offers a
potential solution to several of the resource limitations
for biosurveillance applications. Greninger et al. demon-
strated a proof-of-concept study to identify Chikungu-
nya virus, Ebola virus, and hepatitis C virus from human
clinical samples by using the MinION and a web-based
analysis program (28 ). In this case a high-titer sample
(107 copies/mL) from an Ebola virus–infected patient
and a low titer (105 copies/mL) hepatitis C virus sample
had the first virus-specific reads identified within 10 and
40 min of sequencing, respectively (28 ). Another excel-
lent example is from Quick et al., who built and deployed
4 Clinical Chemistry 65:3 (2019)
a genome surveillance system to Guinea during the West
Africa Ebola virus outbreak (29 ). This system was highly
portable and could be transported by commercial airline
to Guinea, where it generated data within 2 days of ar-
rival. With use of PCR for genome enrichment before
sequencing, Ebola virus genomic information was avail-
able in ⬍24 h of sample receipt (29 ). This rapid time to
answer could affect fielded diagnostic assays during the
outbreak, providing real-time analysis of sequence vari-
ance affecting diagnostic assay efficacy.
While resolving these technological challenges is an
ongoing process, surveillance is the first important step in
a preparedness and readiness paradigm. Specifically, in
the context of a biological incident, there is a defined
need to determine attribution of the source of the insult-
ing agent.
NGS Bioforensics
Forensic microbiology is a relatively new scientific disci-
pline created, for attributional purposes, to analyze evi-
dence of bioterrorism, biocrime, or the inadvertent
release of an etiologic agent. In bioforensics, agent iden-
tification is the starting point of an investigation. All
subsequent efforts seek to link organism-specific genetic
information to a particular isolate, bacterial strain, or
even an individual person with sufficient confidence for
legal review. The inception of modern bioforensic meth-
ods began with Alec Jeffery’s development of human
DNA fingerprinting, precipitating the current attribu-
tional use of DNA for forensic samples (30 –32 ). Restric-
tion fragment length polymorphism and Sanger sequenc-
ing of short tandem repeats are recent advances in this
field (33 ), but, similar to biosurveillance, resolving com-
plex DNA mixtures or identifying genetic variations can
be challenging with these methods (34 ). Some examples
of modern biocrimes, in which bioforensic experts used
the microorganism’s genetic material for attribution, in-
clude Salmonella typhimurium poisoning of salad bars
(35 ) and attempted murder with HIV-infected blood
(36 ) (Fig. 3). Resolution of these cases demonstrated the
merit of microbial bioforensics as a scientific discipline.
However, it was not until the 2001 intentional release of
B. anthracis spores in the Amerithrax letters that the value
of biodefense microbial forensics came to fruition.
NGS found minimal use in the Amerithrax case pri-
marily due to the nascent nature of the technology; how-
ever, it is worth discussing the application of Sanger se-
quencing in this case because this provides an excellent
example of how NGS would be implemented currently. In
the Amerithrax investigational timeline, multiple locus
variable-number tandem repeat analysis subtyped the B. an-
thracis used to the Ames strain (37 ). Subsequent Sanger-
based whole-genome sequencing narrowed the isolate on
the basis of the presence of single-nucleotide polymorphism
Biothreat Applications of Next-Generation Sequencing
Fig. 3. Timeline of the commercialization of sequencing
platforms alongside instances of notable biological events.
Technological advances in sequencing allowed pathogen charac-
terization in notable biological events with the exact methods be-
ing limited to the technology available at the time. Sanger se-
quencing was used with the Amerithrax attacks and NGS in the
West Africa Ebola virus outbreak. Applying sequencing in 1 plat-
form generation or another resulted in characterizing an orga-
nism’s genetic information to better understand the effect and
causality of the biological event.
genotypic variants to a specific laboratory-cultivated stock
(38 ). Specifically, 4 phenotypic morphotypes in the sample
were genotypically linked to a parent stock, RMR-1029,
resulting in the identification of potential suspects who had
access to this material. Ultimately, the Amerithrax investi-
gation led to a nondefinitive conclusion due to circum-
stances outside context of forensics-based sequencing
(39, 40 ). All aspects of the sequence analysis met stringent
standards of scientific rigor; however, perceived biases in
sample collection, inconsistencies in sampling methods, cul-
turing before genetic analysis, and limited sample metadata
contributed to the inability to clearly resolve this case
(39, 40 ). The National Biodefense Analysis and Counter-
measures Center, an agency created for bioforensic applica-
tions, applies lessons learned from the Amerithrax case to the
current application of NGS for bioterrorism and biocrimes
investigations.
Reviews
NGS-based bioforensic applications include out-
break tracing (41– 44 ) and isolate characterization (45 ).
For example, NGS identified causality and traceability in
a hospital outbreak of carbapenem-resistant Klebsiella
pneumoniae and determined how this organism was
transmitted despite rigorous infection-control mecha-
nisms and policies (41 ). NGS linked the index and sub-
sequent cases of K. pneumoniae infection through whole-
genome sequencing characterization and variant analysis.
Epidemiological findings showed the isolate was highly
stable in the hospital environment, thus allowing the bac-
terium to persist on medical equipment despite rigorous
equipment cleaning procedures (41 ). These findings
highlighted the value of NGS in tackling nosocomial
infections through forensic identification of transmission
links and determining gaps in infection-control mea-
sures. Another application of bioforensic NGS is high-
lighted by a study determining the source of a Vibrio
cholerae outbreak in Haiti following the 2010 earth-
quake. Genomic sequencing and other molecular epide-
miological analyses found the outbreak source was likely
Nepalese peacekeepers deployed to Haiti to assist in the
earthquake response (42– 44 ). These data described an
inadvertent, likely single introduction event that resulted
in a rapid outbreak expansion and the first cholera out-
break in Haiti in approximately 100 years. Lastly, surveil-
lance networks are beginning to use forensic NGS in a
broader application of determining transmission of viru-
lence factors such as antimicrobial resistance. For exam-
ple, the Multidrug Resistant Organism Repository and
Surveillance Network used NGS whole-genome se-
quencing to identify the first instance of the mcr-13 colis-
tin resistance gene in US hospitals (45 ).
Bioforensic Challenges
All NGS applications struggle with the signal-to-noise or
pathogen-to-background ratio to obtain informative se-
quence information. However, a unique bioforensics
challenge is the requirement for precise assay standard-
ization and validation, as well as chain of custody to meet
legal evidentiary standards and withstand challenge in a
court of law. Validation of techniques must demonstrate
the information obtained can answer investigative ques-
tions by assigning the source of a sample while excluding
other sources to the highest degree possible (46, 47 ).
Standard operating procedures are absolutely required as
all components, from sample collection to library prepa-
ration methods, are highly specific to the NGS applica-
tion. Sequence analysis pipelines including evaluation of
base calling, raw sequence quality, variance analysis, and
3
Gene: mcr-1, mobilized colistin resistance gene.
Clinical Chemistry 65:3 (2019) 5
Reviews
genome assembly also need to be technically validated.
This can be a highly onerous process because validation
includes the definition of parameters such as analytical
specificity and sensitivity, reproducibility, precision, ac-
curacy, and robustness (46 ). Unlike many clinical and
biosurveillance samples, forensic samples tend to be the
most challenging for NGS because of such factors as lim-
ited sample volume, exposure to and degradation by en-
vironmental elements, and potential mixing with other
nucleic acid sources (48 ). While challenging, these issues
are addressable and are similar to the standard of valida-
tion required for clinical application and regulatory com-
pliance of biothreats diagnostics.
NGS for Biothreat Diagnosis
Clinical applications, perhaps most relevant for this au-
dience, are where NGS finds its greatest potential to shift
the diagnostic work flow. NGS infectious diseases diag-
nostics are less mature than biosurveillance or bioforen-
sics applications; however, NGS is positioned to be a
prominent tool in the molecular diagnostic toolbox. As
mentioned before, NGS requires little to no prior infor-
mation regarding the etiologic agent to identify a patho-
gen or even a mixed population in a culture or sample.
The ability to simultaneously sequence all nucleic acids
within a sample without analytical specificity is a key
NGS advantage over traditional targeted approaches
such as real-time PCR. However, this lack of specificity is
also a primary obstacle in clinical NGS implementation.
Specifically, in an NGS assay, any nucleic acid in the
sample is sequenced (pathogen, commensals, back-
ground biota, and host nucleic acid) and contributes se-
quence information to the overall diagnostic output.
Similar to bioforensics and biosurveillance, most clinical
samples are highly skewed toward host background nu-
cleic acid compared to the pathogen of interest in terms
of concentration, thus resulting in a poor signal-to-noise
ratio. Numerous groups have found innovative and ele-
gant solutions to this issue including host subtraction and
pathogen enrichment (49 –51 ). These solutions and the
resulting clinical NGS applications generally focus on 2
primary implementation strategies: diagnostic of last re-
sort or laboratory-developed tests (LDTs).
Wilson et al. efficiently demonstrated the “diagnos-
tic of last resort” use case by applying NGS to resolve a
clinical diagnosis when all other diagnostics failed (52 ).
In this study, NGS resolved a cryptic case of leptospirosis
that was previously unidentifiable before NGS applica-
tion. Researchers solved the signal-to-noise issue by ap-
plying high-density sequencing (Illumina HiSeq), pro-
viding sufficient coverage to detect the rare pathogen
reads among substantial human host background (52 ).
Specifically, NGS sequencing produced over 3 million
noninformative background reads with only 475 map-
6 Clinical Chemistry 65:3 (2019)
ping to the causative agent, Leptospirosis, from a cerebral
spinal fluid sample. Accomplished within 48 h, this
method provided actionable information toward patient
treatment in a reasonable timeframe, thus allowing incor-
poration into a standard clinical work flow. However,
deep sequencing is relatively expensive compared to most
diagnostics, effectively pricing its use out of routine clin-
ical applications. In addition, cerebrospinal fluid is a rel-
atively low background clinical specimen compared to
other complex diagnostic matrices. One might expect a
lower signal-to-noise ratio from whole blood in which
host nucleic acid is at a substantially higher concentra-
tions. Issues aside, use of this technology currently runs
the spectrum of applications, with documented use cases
for diagnosis of Zika and Chikungunya virus (53 ), Bru-
cella spp. (54 ), and Ebola virus (28 ) infections.
Introduction of new technologies into the clinic typ-
ically follows a path of adoption from Research Use
Only, to LDT, and finally to Food and Drug Adminis-
tration (FDA)-cleared regulatory compliance. Currently,
NGS is being adopted as an LDT in several clinical sites
for detection or identification of infectious diseases (55–
57 ). Excellent examples of NGS LDT implementation
include works from the University of Washington
(55, 56 ) and Houston Methodist Research Institute
(57 ). These efforts successfully insert NGS into the clin-
ical work flow as a whole-genome characterization tool to
identify and speciate positive bacterial cultures. Similarly,
in the biothreat context, efforts to transition research
methods to the clinic have followed the LDT path
(50, 58, 59 ). To facilitate the adoption of NGS LDTs,
the Clinical Laboratory Improvement Amendments
(60 ), College of American Pathologists (61 ), and other
accrediting bodies (62, 63 ) published roadmaps detail-
ing the validation data sets required to establish an NGS
LDT within the respective clinical laboratory accrediting
authorities.
Regulatory Challenges
Ideally, NGS assays will transition from the LDT status
to full FDA approval, allowing distribution and use of
these assays in any accredited clinical laboratory without
revalidation. To facilitate this transition, the FDA re-
leased draft guidance entitled “Infectious Disease Next
Generation Sequencing Based Diagnostic Devices: Mi-
crobial Identification and Detection of Antimicrobial
Resistance and Virulence Markers” (62 ). This document
details a de novo submission for both the modular NGS
components FDA considers regulated as well as the ana-
lytical and clinical data sets requested for both targeted
and whole-genome sequencing (Fig. 4). As part of the
guidance, FDA defines NGS as a whole system including
the wet-laboratory component through the bioinformat-
ics process to result in a diagnostic output. Perhaps the
Biothreat Applications of Next-Generation Sequencing
Fig. 4. Data sets required for an FDA de novo submission for NGS diagnostics.
FDA guidance defines an NGS assay as a whole system from specimen collection through data analysis (blue). All these steps must be
incorporated into the validation studies, generating both the analytical and clinical data sets (gray) required for approval.
most important aspect of this guidance is that it provided
a potential relief in the scope of clinical trials, reducing
the data generation burden with a proposed method of in
silico, as opposed to wet laboratory, comparison for pos-
itive and negative predictive values. Specifically, assay
results from the clinical trial would not need to be com-
pared to a gold standard comparator, such as real-time
PCR or other molecular assay, for every clinical sample
but instead could be compared to a vetted FDA database
of regulatory-grade sequences (FDA-ARGOS, (64 )).
This database is publicly available through NCBI Bio-
project PRJNA231221.
A substantial regulatory challenge for the clinical
adoption of NGS is false-positive reporting due to either
nucleic acid contamination of sample processing reagents
or inaccurate index demultiplexing (see section below on
bioinformatics considerations). Nucleic acid reagent
contamination is common for many molecular biological
products, e.g., enzymes. Adding steps to the overall sam-
ple processing, such as extraction and enrichment, adds
additional points of potential nucleic acid contamina-
tion, especially when applying nonspecific amplification
strategies (65 ) and/or ultradeep, unbiased NGS (66 ).
Targeted amplification of unique sequences of interest
and physical removal of contaminating nucleic acids via
enzymatic, radiation, and chemical treatments can miti-
Reviews
gate reagent contamination (66 ). In this context, FDA
draft guidance addresses the necessity for evaluation of
carryover, cross-contamination, and the essential nature
of nontemplate controls inserted throughout all aspects
of sample preparation and library generation. These con-
trols inform the end user of any contamination events in
the sequencing reaction, which can later be addressed
bioinformatically with appropriate data analysis.
Bioinformatics Considerations
NGS for biothreat characterization or clinical identifica-
tion is only as reliable as the data analysis implemented
after sequencing. There are a number of different se-
quencing analysis programs, several of which are tied spe-
cifically to individual sequencing platforms. Others cover
the scope of NGS and bioinformatics in broader detail
(67–71 ); however, there are several biothreat-centric
considerations for bioinformatics including (a) near-
neighbor differentiation and the implications of false
positives, (b) biodefense-specific bioinformatics solu-
tions, and (c) standardization of bioinformatics pipelines
to facilitate use of NGS in biothreat-regulated spaces.
The consequence of misidentifying a sample as a
biothreat, when in fact the organism is simply a near
neighbor with a similar genetic composition, can have
Clinical Chemistry 65:3 (2019) 7
Reviews
significant fiscal, logistical, and public perception effects.
Accurately differentiating a biothreat from near neigh-
bors with a limited number of pathogen reads, typical of
unbiased sequencing assays, remains a principal objective
for most biodefense bioinformatics software. Because
many biothreats are genetically similar to less pathogenic
ancestors, accurately differentiating between the 2 is crit-
ically important. For example, a recent report described
traces of Yersinia pestis and B. anthracis, bacteria that
cause plague and anthrax, respectively, in the New York
subway system (72 ). However, the group and others re-
alized after publication that these signatures were not
actually specific to these biothreat agents but instead de-
rived from phylogenetically similar organisms (73 ). Sim-
ilarly, in the process of developing a novel bioinformatics
software, Greninger et al. found Ebola virus reads in an
influenza case despite the laboratory not having ever pro-
cessed Ebola virus material (74 ).
Metagenomic sequencing often relies on detection
of low numbers of pathogen-specific reads, tens to hun-
dreds, to make diagnostic calls. As mentioned above,
intra- and interassay errors and inaccurate index demul-
tiplexing can result in false positives due to misassign-
ment of reads from the same or even previous sequencing
runs. Intraassay errors stem from hardware and software
issues resulting in misbinning of adjacent clusters into
incorrect FASTQ files or misidentification of closely re-
lated indices during demultiplexing (75 ). Interassay con-
tamination can occur on a sequencing run conducted
after sequencing a high concentration sample, leading to
pathogen-specific read cross-contamination and false-
positive results. Given the relatively low number of reads
used to make a detection call, these issues highlight the
need for assay pipeline validation and confirmatory test-
ing for any application in which actionable results would
be required, such as with all NGS biothreat approaches.
The biodefense industry created multiple open-
source bioinformatics suites, such as Sequence-based
UltraRapid Pathogen Identification (SURPI) (76 ) and
Empowering the Development of Genomics Expertise
(EDGE) (77 ), for highly reliable applications to solve
some the issues of parsing biothreats from near neighbors
and begin to move toward fieldable bioinformatics solu-
tions. Though different in execution, these products
strive for the most accurate and reliable answer while
minimizing the technical expertise required to run
these software programs. SURPI achieves high result
fidelity by liberal sequential removal of noninforma-
tive sequence reads before matching sequences to
pathogens. This high fidelity minimizes the potential
for spurious misassignment of sequence reads, thus
reducing potential for false positives. EDGE ap-
proaches this problem from a different perspective.
After removing host or other problematic signatures,
this software uses no fewer than 5 independent metag-
8 Clinical Chemistry 65:3 (2019)
enomics programs to answer fidelity through consen-
sus. We have found that both approaches are reliable
but are primarily limited by the scope of current, pub-
licly available sequence databases. As researchers strive
to push NGS further toward the point of care, the issue
of technical expertise becomes increasing important as most
hospitals and field sites do not have access to or funds to
support trained bioinformatic specialists. Both these soft-
ware programs are potential solutions to this issue.
Standardization of NGS, the wet laboratory, and
bioinformatics is perhaps the 1 common thread for all
biothreat applications owing to the necessity for regu-
lated and validated methods for actionable results. To
help standardize NGS, multiple governmental agencies
including FDA and National Institutes of Standards and
Technology are establishing procedural (64 ) and physi-
cal standards (78 ) for metagenomics that ease the surety
burden placed on computational tools. Using these
standards, we developed the Diagnostic targETEd
seQuencing adjudicaTion (DETEQT) software (79 ),
solidifying targeted infectious disease sequencing data
analysis for LDT applications while minimizing false-
discovery rates. In other efforts, FDA and other research-
ers proposed procedural standards for genome finishing
(80 ) and the generation of high-quality databases, an
equally important consideration for accurate data analy-
sis as assay standardization. Misannotations and poor-
quality sequences can markedly affect the bioinformatics
process. Culmination of these standards coupled with
existing validation procedures will eventually push data-
bases and NGS further toward the point of care because
of credible results and ease of use.
Conclusions
Over the past 15 years, there have been many technolog-
ical advancements in biothreat diagnostics and character-
ization. In particular, NGS has greatly expanded the ca-
pabilities of detecting biothreats, unexpected pathogens,
or even the completely novel, previously unknown
threat. Improvements in cost, throughput, quality,
and time to answer brought NGS from highly special-
ized sequencing centers into smaller laboratories, aus-
tere environments, and even the clinic. This broader
NGS availability has enabled many improvements to
be made across the NGS pipeline, including sample
processing, library preparation, and simplified large-
scale data analysis.
As NGS moves into the highly regulated realms of
biothreat diagnostics and bioforensics, substantial efforts
will need to be undertaken to meet regulatory and legal
requirements, including procedure and bioinformatics
standardization and validation of sequencing and analysis
pipelines. Improvements in the technology, specifically
with the third generation of sequencers, bring closer to
Biothreat Applications of Next-Generation Sequencing
Reviews

reality the concept of a pathogen-agnostic metagenomic and (d) agreement to be accountable for all aspects of the article thus
diagnostic usable in any location and providing answers ensuring that questions related to the accuracy or integrity of any part of the
within a clinically actionable timeline. This capability article are appropriately investigated and resolved.
will positively affect the public health with better infec- T.D. Minogue, financial support.
tious disease outbreaks response and biosurveillance ef-
Authors’ Disclosures or Potential Conflicts of Interest: Upon man-
forts. In addition, NGS biothreat applications will be a uscript submission, all authors completed the author disclosure form. Dis-
contributing factor for detection and containment of any closures and/or potential conflicts of interest:
biological incident with a high probability of generating
Employment or Leadership: J.W. Koehler, contractor with
attributional sequence information.
USAMRIID.
Consultant or Advisory Role: None declared.
Stock Ownership: None declared.
Author Contributions: All authors confirmed they have contributed to Honoraria: None declared.
the intellectual content of this paper and have met the following 4 require- Research Funding: Defense Threat Reduction Agency project
ments: (a) significant contributions to the conception and design, acquisi- funding.
tion of data, or analysis and interpretation of data; (b) drafting or revising Expert Testimony: None declared.
the article for intellectual content; (c) final approval of the published article; Patents: None declared.

1. Rasko DA, Worsham PL, Abshire TG, Stanley ST, Bannan
JD, Wilson MR, et al. Bacillus anthracis comparative ge-
nome analysis in support of the Amerithrax investiga-
tion. Proc Natl Acad Sci USA 2011;108:5027–32.
2. Reich ES. Science after 9/11: how research was changed
by the September 11 terrorist attacks. https://
www.scientificamerican.com/article/how-research-was-
changed-by-september-11-terrorist-attacks/ (Accessed
January 2018).
3. Rogan DT, Kochar MS, Yang S, Quinn JV. Impact of rapid
molecular respiratory virus testing on real-time decision
making in a pediatric emergency department. J Mol Di-
agn 2017;19:460 –7.
4. Towner JS, Rollin PE, Bausch DG, Sanchez A, Crary SM,
Vincent M, et al. Rapid diagnosis of Ebola hemorrhagic
fever by reverse transcription-PCR in an outbreak set-
ting and assessment of patient viral load as a predictor
of outcome. J Virol 2004;78:4330 – 41.
5. Bausch DG, Rollin PE, Demby AH, Coulibaly M, Kanu J,
Conteh AS, et al. Diagnosis and clinical virology of Lassa
fever as evaluated by enzyme-linked immunosorbent
assay, indirect fluorescent-antibody test, and virus iso-
lation. J Clin Microbiol 2000;38:2670 –7.
6. Broadhurst MJ, Kelly JD, Miller A, Semper A, Bailey D,
Groppelli E, et al. ReEBOV antigen rapid test kit for
point-of-care and laboratory-based testing for Ebola vi-
rus disease: a field validation study. Lancet 2015;386:
867–74.
7. Sanger F, Air GM, Barrell BG, Brown NL, Coulson AR,
Fiddes CA, et al. Nucleotide sequence of bacteriophage
phi ×174 DNA. Nature 1977;265:687–95.
8. Sanger F, Nicklen S, Coulson AR. DNA sequencing with
chain-terminating inhibitors. Proc Natl Acad Sci USA
1977;74:5463–7.
9. Heather JM, Chain B. The sequence of sequencers: the
history of sequencing DNA. Genomics 2016;107:1– 8.
10. Grard G, Fair JN, Lee D, Slikas E, Steffen I, Muyembe JJ,
et al. A novel rhabdovirus associated with acute hemor-
rhagic fever in central Africa. PLoS Pathog 2012;8:
e1002924.
11. Briese T, Paweska JT, McMullan LK, Hutchison SK,
Street C, Palacios G, et al. Genetic detection and charac-
terization of Lujo virus, a new hemorrhagic fever-
associated arenavirus from southern Africa. PLoS Pat-
hog 2009;5:e1000455.
12. Gire SK, Goba A, Andersen KG, Sealfon RS, Park DJ,
Kanneh L, et al. Genomic surveillance elucidates Ebola
References
virus origin and transmission during the 2014 out-
break. Science 2014;345:1369 –72.
13. Hoenen T, Safronetz D, Groseth A, Wollenberg KR, Koita
OA, Diarra B, et al. Virology. Mutation rate and geno-
type variation of Ebola virus from Mali case sequences.
Science 2015;348:117–9.
14. Sozhamannan S, Holland MY, Hall AT, Negron DA, Ivan-
cich M, Koehler JW, et al. Evaluation of signature ero-
sion in Ebola virus due to genomic drift and its impact
on the performance of diagnostic assays. Viruses 2015;
7:3130 –54.
15. Christie A, Davies-Wayne GJ, Cordier-Lassalle T, Black-
ley DJ, Laney AS, Williams DE, et al. Possible sexual
transmission of Ebola virus—Liberia, 2015. MMWR
Morb Mortal Wkly Rep 2015;64:479 – 81.
16. Mate SE, Kugelman JR, Nyenswah TG, Ladner JT, Wiley
MR, Cordier-Lassalle T, et al. Molecular evidence of sex-
ual transmission of Ebola virus. N Engl J Med 2015;
373:2448 –54.
17. Park DJ, Dudas G, Wohl S, Goba A, Whitmer SL, Ander-
sen KG, et al. Ebola virus epidemiology, transmission,
and evolution during seven months in Sierra Leone.
Cell 2015;161:1516 –26.
18. Folarin OA, Ehichioya D, Schaffner SF, Winnicki SM,
Wohl S, Eromon P, et al. Ebola virus epidemiology and
evolution in Nigeria. J Infect Dis 2016;214:S102–S9.
19. Lau MS, Dalziel BD, Funk S, McClelland A, Tiffany A,
Riley S, et al. Spatial and temporal dynamics of super-
spreading events in the 2014 –2015 West Africa Ebola
epidemic. Proc Natl Acad Sci USA 2017;114:2337– 42.
20. Holmes EC, Dudas G, Rambaut A, Andersen KG. The
evolution of Ebola virus: insights from the 2013–2016
epidemic. Nature 2016;538:193–200.
21. Noyce RS, Lederman S, Evans DH. Construction of an
infectious horsepox virus vaccine from chemically
synthesized DNA fragments. PLoS One 2018;13:
e0188453.
22. Kerr PJ, Perkins HD, Inglis B, Stagg R, McLaughlin E,
Collins SV, Van Leeuwen BH. Expression of rabbit IL-4
by recombinant myxoma viruses enhances virulence
and overcomes genetic resistance to myxomatosis. Vi-
rology 2004;324:117–28.
23. Hernandez M, Pla M, Esteve T, Prat S, Puigdomenech P,
Ferrando A. A specific real-time quantitative PCR detec-
tion system for event MON810 in maize yieldgard
based on the 3`-transgene integration sequence. Trans-
genic Res 2003;12:179 – 89.
24. Permingeat HR, Reggiardo MI, Vallejos RH. Detection
and quantification of transgenes in grains by multiplex
and real-time PCR. J Agric Food Chem 2002;50:
4431– 6.
25. Paracchini V, Petrillo M, Reiting R, Angers-Loustau A,
Wahler D, Stolz A, et al. Molecular characterization of an
unauthorized genetically modified Bacillus subtilis pro-
duction strain identified in a vitamin B2 feed additive.
Food Chem 2017;230:681–9.
26. Willems S, Fraiture MA, Deforce D, De Keersmaecker
SC, De Loose M, Ruttink T, et al. Statistical framework for
detection of genetically modified organisms based on
next generation sequencing. Food Chem 2016;192:
788 –98.
27. Liang C, van Dijk JP, Scholtens IM, Staats M, Prins TW,
Voorhuijzen MM, et al. Detecting authorized and unau-
thorized genetically modified organisms containing
vip3A by real-time PCR and next-generation sequenc-
ing. Anal Bioanal Chem 2014;406:2603–11.
28. Greninger AL, Naccache SN, Federman S, Yu G, Mbala
P, Bres V, et al. Rapid metagenomic identification of
viral pathogens in clinical samples by real-time nano-
pore sequencing analysis. Genome Med 2015;7:99.
29. Quick J, Loman NJ, Duraffour S, Simpson JT, Severi E,
Cowley L, et al. Real-time, portable genome sequenc-
ing for Ebola surveillance. Nature 2016;530:228 –32.
30. Jeffreys AJ, Brookfield JF, Semeonoff R. Positive iden-
tification of an immigration test-case using human DNA
fingerprints. Nature 1985;317:818 –9.
31. Jeffreys AJ, Wilson V, Thein SL. Individual-specific ‘fin-
gerprints’ of human DNA. Nature 1985;316:76 –9.
32. Jeffreys AJ, Wilson V, Thein SL. Hypervariable ‘minisat-
ellite’ regions in human DNA. Nature 1985;314:
67–73.
33. Roewer L. DNA fingerprinting in forensics: past, pres-
ent, future. Investig Genet 2013;4:22.
34. Yang Y, Xie B, Yan J. Application of next-generation se-
quencing technology in forensic science. Genomics
Proteomics Bioinformatics 2014;12:190 –7.
35. Torok TJ, Tauxe RV, Wise RP, Livengood JR, Sokolow R,
Mauvais S, et al. A large community outbreak of salmo-
nellosis caused by intentional contamination of restau-
rant salad bars. JAMA 1997;278:389 –95.
36. Metzker ML, Mindell DP, Liu XM, Ptak RG, Gibbs RA,
Hillis DM. Molecular evidence of HIV-1 transmission in
a criminal case. Proc Natl Acad Sci USA 2002;99:
14292–7.

Clinical Chemistry 65:3 (2019) 9

Reviews
37. Hoffmaster AR, Fitzgerald CC, Ribot E, Mayer LW, Pop-
ovic T. Molecular subtyping of Bacillus anthracis and
the 2001 bioterrorism-associated anthrax outbreak,
United States. Emerg Infect Dis 2002;8:1111– 6.
38. Read TD, Salzberg SL, Pop M, Shumway M, Umayam L,
Jiang L, et al. Comparative genome sequencing for dis-
covery of novel polymorphisms in Bacillus anthracis.
Science 2002;296:2028 –33.
39. Gilchrist CA, Turner SD, Riley MF, Petri WA, Jr., Hewlett
EL. Whole-genome sequencing in outbreak analysis.
Clin Microbiol Rev 2015;28:541– 63.
40. Committee on Review of the Scientific Approaches
Used During the FBI’s Investigation of the 2001 Bacillus
anthracis mailings. Review of the scientific approaches
used during the FBI’s investigation of the 2001 anthrax
letters. Washington (DC): National Academies Press;
2011.
41. Snitkin ES, Zelazny AM, Thomas PJ, Stock F, Group
NCSP, Henderson DK, et al. Tracking a hospital out-
break of carbapenem-resistant Klebsiella pneumoniae
with whole-genome sequencing. Sci Transl Med 2012;
4:148ra16.
42. Eppinger M, Pearson T, Koenig SS, Pearson O, Hicks N,
Agrawal S, et al. Genomic epidemiology of the Haitian
cholera outbreak: a single introduction followed by
rapid, extensive, and continued spread characterized
the onset of the epidemic. MBio 2014;5:e01721.
43. Hendriksen RS, Price LB, Schupp JM, Gillece JD, Kaas
RS, Engelthaler DM, et al. Population genetics of Vibrio
cholerae from Nepal in 2010: evidence on the origin of
the Haitian outbreak. MBio 2011;2:e00157–11.
44. Chin CS, Sorenson J, Harris JB, Robins WP, Charles RC,
Jean-Charles RR, et al. The origin of the Haitian cholera
outbreak strain. N Engl J Med 2011;364:33– 42.
45. McGann P, Snesrud E, Maybank R, Corey B, Ong AC,
Clifford R, et al. Escherichia coli harboring mcr-1 and
blaCTX-M on a novel IncF plasmid: first report of mcr-1
in the United States. Antimicrob Agents Chemother
2016;60:4420 –1.
46. Budowle B, Schutzer SE, Morse SA, Martinez KF,
Chakraborty R, Marrone BL, et al. Criteria for validation
of methods in microbial forensics. Appl Environ Micro-
biol 2008;74:5599 – 607.
47. Budowle B, Connell ND, Bielecka-Oder A, Colwell RR,
Corbett CR, Fletcher J, et al. Validation of high through-
put sequencing and microbial forensics applications.
Investig Genet 2014;5:9.
48. Kuiper I. Microbial forensics: next-generation sequenc-
ing as catalyst: the use of new sequencing technologies
to analyze whole microbial communities could become
a powerful tool for forensic and criminal investigations.
EMBO Rep 2016;17:1085–7.
49. Gardner SN, Frey KG, Redden CL, Thissen JB, Allen JE,
Allred AF, et al. Targeted amplification for enhanced
detection of biothreat agents by next-generation se-
quencing. BMC Res Notes 2015;8:682.
50. Koehler JW, Hall AT, Rolfe PA, Honko AN, Palacios GF,
Fair JN, et al. Development and evaluation of a panel of
filovirus sequence capture probes for pathogen detec-
tion by next-generation sequencing. PLoS One 2014;9:
e107007.
51. Depledge DP, Palser AL, Watson SJ, Lai IY, Gray ER,
10 Clinical Chemistry 65:3 (2019)
Grant P, et al. Specific capture and whole-genome se-
quencing of viruses from clinical samples. PLoS One
2011;6:e27805.
52. Wilson MR, Naccache SN, Samayoa E, Biagtan M,
Bashir H, Yu G, et al. Actionable diagnosis of neurolep-
tospirosis by next-generation sequencing. N Engl
J Med 2014;370:2408 –17.
53. Sardi SI, Somasekar S, Naccache SN, Bandeira AC, Tauro
LB, Campos GS, Chiu CY. Coinfections of Zika and Chi-
kungunya viruses in Bahia, Brazil, identified by metag-
enomic next-generation sequencing. J Clin Microbiol
2016;54:2348 –53.
54. Mongkolrattanothai K, Naccache SN, Bender JM, Sa-
mayoa E, Pham E, Yu G, et al. Neurobrucellosis: unex-
pected answer from metagenomic next-generation se-
quencing. J Pediatric Infect Dis Soc 2017;6:393– 8.
55. Roach DJ, Burton JN, Lee C, Stackhouse B, Butler-Wu
SM, Cookson BT, et al. Correction: a year of infection in
the intensive care unit: prospective whole genome se-
quencing of bacterial clinical isolates reveals cryptic
transmissions and novel microbiota. PLoS Genet 2017;
13:e1006724.
56. Salipante SJ, Sengupta DJ, Rosenthal C, Costa G, Span-
gler J, Sims EH, et al. Rapid 16S rrna next-generation
sequencing of polymicrobial clinical samples for diag-
nosis of complex bacterial infections. PLoS One 2013;
8:e65226.
57. Long SW, Williams D, Valson C, Cantu CC, Cernoch P,
Musser JM, Olsen RJ. A genomic day in the life of a
clinical microbiology laboratory. J Clin Microbiol 2013;
51:1272–7.
58. Stefan CP, Koehler JW, Minogue TD. Targeted next-
generation sequencing for the detection of ciprofloxa-
cin resistance markers using molecular inversion
probes. Sci Rep 2016;6:25904.
59. Stefan CP, Hall AT, Minogue TD. Detection of 16S rRNA
and KPC genes from complex matrix utilizing a molec-
ular inversion probe assay for next-generation se-
quencing. Sci Rep 2018;8:2028.
60. Gargis AS, Kalman L, Berry MW, Bick DP, Dimmock DP,
Hambuch T, et al. Assuring the quality of next-
generation sequencing in clinical laboratory practice.
Nat Biotechnol 2012;30:1033– 6.
61. Aziz N, Zhao Q, Bry L, Driscoll DK, Funke B, Gibson JS, et
al. College of American pathologists’ laboratory stan-
dards for next-generation sequencing clinical tests.
Arch Pathol Lab Med 2015;139:481–93.
62. United States Food and Drug Administration. Draft
guidance: infectious disease next generation sequenc-
ing based diagnostic devices: microbial identification
and detection of antimicrobial resistance and virulence
markers. https://www.fda.gov/downloads/Medical
Devices/DeviceRegulationandGuidance/Guidance
Documents/UCM500441.pdf (Accessed Jan. 2013).
63. Gargis AS, Kalman L, Lubin IM. Assuring the quality of
next-generation sequencing in clinical microbiology
and public health laboratories. J Clin Microbiol 2016;
54:2857– 65.
64. Sichtig H. Infectious disease next generation sequenc-
ing based diagnostic devices: microbial identification
and detection of antimicrobial resistance and virulence
markers. FDA; 2016.
65. Salter SJ, Cox MJ, Turek EM, Calus ST, Cookson WO,
Moffatt MF, et al. Reagent and laboratory contamina-
tion can critically impact sequence-based microbiome
analyses. BMC Biol 2014;12:87.
66. Laurence M, Hatzis C, Brash DE. Common contami-
nants in next-generation sequencing that hinder dis-
covery of low-abundance microbes. PLoS One 2014;9:
e97876.
67. Valdivia-Granda WA. Biodefense oriented genomic-
based pathogen classification systems: challenges and
opportunities. J Bioterror Biodef 2012;3:1000113.
68. Karlsson OE, Hansen T, Knutsson R, Lofstrom C, Gran-
berg F, Berg M. Metagenomic detection methods in
biopreparedness outbreak scenarios. Biosecur Bioter-
ror 11 Suppl 1:S146 –57, 2013.
69. Minogue TD, Koehler JW, Norwood DA. Targeted next-
generation sequencing for diagnostics and forensics.
Clin Chem 2017;63:450 –2.
70. Fricke WF, Rasko DA, Ravel J. The role of genomics in
the identification, prediction, and prevention of biolog-
ical threats. PLoS Biol 2009;7:e1000217.
71. Greene JM, Collins F, Lefkowitz EJ, Roos D, Scheuer-
mann RH, Sobral B, et al. National Institute of Allergy
and Infectious Diseases bioinformatics resource
centers: new assets for pathogen informatics. Infect Im-
mun 2007;75:3212–9.
72. Afshinnekoo E, Meydan C, Chowdhury S, Jaroudi D,
Boyer C, Bernstein N, et al. Geospatial resolution of hu-
man and bacterial diversity with city-scale metagenom-
ics. Cell Syst 2015;1:97– e3.
73. Ackelsberg J, Rakeman J, Hughes S, Petersen J, Mead
P, Schriefer M, et al. Lack of evidence for plague or an-
thrax on the New York city subway. Cell Syst 2015;1:
4 –5.
74. Greninger AL, Chen EC, Sittler T, Scheinerman A, Rou-
binian N, Yu G, et al. A metagenomic analysis of pan-
demic influenza a (2009 H1N1) infection in patients
from North America. PLoS One 2010;5:e13381.
75. Kircher M, Sawyer S, Meyer M. Double indexing over-
comes inaccuracies in multiplex sequencing on the Il-
lumina platform. Nucleic Acids Res 2012;40:e3.
76. Naccache SN, Federman S, Veeraraghavan N, Zaharia
M, Lee D, Samayoa E, et al. A cloud-compatible bioin-
formatics pipeline for ultrarapid pathogen identifica-
tion from next-generation sequencing of clinical sam-
ples. Genome Res 2014;24:1180 –92.
77. Li PE, Lo CC, Anderson JJ, Davenport KW, Bishop-Lilly
KA, Xu Y, et al. Enabling the democratization of the
genomics revolution with a fully integrated web-based
bioinformatics platform. Nucleic Acids Res 2017;45:
67– 80.
78. Mason CE, Afshinnekoo E, Tighe S, Wu S, Levy S. Inter-
national standards for genomes, transcriptomes, and
metagenomes. J Biomol Tech 2017;28:8 –18.
79. Conrad TA. Diagnostic targeted sequencing adjudication
(deteqt). https://www.mathworks.com/matlabcentral/
fileexchange/66892-deteqt.
80. Chain PS, Grafham DV, Fulton RS, Fitzgerald MG,
Hostetler J, Muzny D, et al. Genomics. Genome project
standards in a new era of sequencing. Science 2009;
326:236 –7.