[ Supplement An Overview of Study Design and Statistical Considerations ]

A Bioinformatics Crash Course for

Interpreting Genomics Data
Daniel M. Rotroff, PhD

Reductions in genotyping costs and improvements in computational power have made

conducting genome-wide association studies (GWAS) standard practice for many com-
plex diseases. GWAS is the assessment of genetic variants across the genome of many
individuals to determine which, if any, genetic variants are associated with a specific
trait. As with any analysis, there are evolving best practices that should be followed to
ensure scientific rigor and reliability in the conclusions. This article presents a brief
summary for many of the key bioinformatics considerations when either planning or
evaluating GWAS. This review is meant to serve as a guide to those without deep
expertise in bioinformatics and GWAS and give them tools to critically evaluate this
popular approach to investigating complex diseases. In addition, a checklist is provided
that can be used by investigators to evaluate whether a GWAS has appropriately
accounted for the many potential sources of bias and generally followed current best
practices. CHEST 2020; 158(1S):S113-S123

KEY WORDS: bioinformatics; genomics; statistics

General Overview of Study Design Unlike previous approaches (eg, candidate

Recent technologic advancements have led to
unprecedented volumes of data, bringing
renewed hope for medical breakthroughs.
However, the generation of large volumes of
data has also brought new challenges for
analysis and interpretation. Omics has
recently become a popular term in biological
research and refers to technologies that aim to
comprehensively evaluate a group of
molecular or biochemical constituents.
gene studies), a major benefit to omics
technologies is that one does not need a priori
knowledge of what targets may be involved.
The ability to agnostically evaluate potential
targets opens the door to discovering novel
targets involved in disease pathogenesis.
Genome-wide association studies (GWAS)
have become increasingly popular and
involve the assessment of genetic variants
across the genome of many individuals to

ABBREVIATIONS: GWAS = genome-wide association studies; LD =
linkage disequilibrium; MAF = minor allele frequency; PRS = poly-
genic risk score; SKAT = sequence kernel association test; SNP = single
nucleotide polymorphism; QQ = quantile-quantile
AFFILIATIONS: From the Department of Quantitative Health Sciences,
Lerner Research Institute, Cleveland Clinic, Cleveland, OH.
FUNDING/SUPPORT: This study was supported in part by grants
provided by the Clinical and Translational Science Collaborative of
Cleveland (KL2TR002547 to D. M. R.) from the National Center for
Advancing Translational Sciences.
CORRESPONDENCE TO: Daniel Rotroff, PhD, Department of Quanti-
tative Health Sciences, Lerner Research Institute, Cleveland Clinic,
9500 Euclid Ave, Cleveland, OH 44195; e-mail: rotrofd@ccf.org
Copyright Ó 2020 American College of Chest Physicians. Published by
Elsevier Inc. All rights reserved.
DOI: https://doi.org/10.1016/j.chest.2020.03.004

chestjournal.org S113
determine which, if any, genetic variants are associated
with a specific trait.
Precision pulmonology will require a deep
understanding of disease heterogeneity. Whereas
some hereditary pulmonary diseases (eg, cystic
fibrosis, alpha1-antitrypsin deficiency) have relatively
well-understood pathologies, other lung diseases (eg,
COPD, asthma) are highly complex and
heterogeneous. Furthermore, because the respiratory
system is integrally tied to our physiology, many
diseases can result in pulmonary comorbidities (eg,
pulmonary hypertension in patients with valvular
heart disease, pulmonary fibrosis in patients with
rheumatoid arthritis). There is heterogeneity in the
presentation of these secondary disorders due to
complex genetic and environmental factors. As with
many complex diseases, many primary and
secondary pulmonary diseases are likely to be
influenced by multiple molecular and biochemical
effects with modest effect sizes, making them
difficult to detect. Clever bioinformatics
approaches will be needed to identify the many
contributing factors for the diseases that fall within
this category.
The aim of the current review was to describe key
bioinformatics considerations when analyzing and
interpreting genomics data. As a companion to this
review, it is recommended that readers use the
glossary of common genomics terms provided by
the Journal of the American Medical Association,
which is available at: https://jamanetwork.com/
journals/jama/fullarticle/1677346.1 Although this
review is not meant to serve as a detailed guide
for performing the analyses, it is meant to
provide clinicians and investigators without
bioinformatics expertise with the knowledge to
identify potential sources of bias, critically
evaluate genomics research, and better understand
how to effectively use genomics to advance chest
medicine in their domain. To this end, Table 1
contains a checklist for researchers and statisticians
to use during planning and performance of GWAS.
The short list of questions for reviewers contains
guiding questions that can be used by clinical
reviewers to critically evaluate GWAS. In addition, a
table representing key considerations for various
stages of genomics study development is included
(Table 2).
S114 Supplement
Considerations Prior to Initiating a Genomics Study
There is no one-size-fits-all analysis plan, and each
analysis plan should consider potential impacts from the
cohort selection (eg, sex, race, socioeconomic status, age,
medication usage), study design (eg, cross-sectional,
longitudinal), technical considerations (eg, batch effects,
biological and technical replicates), and traits (eg,
quantitative, dichotomous). The way these factors are
incorporated into the analysis can affect the success of
the study. For instance, is it appropriate to include sex as
a model covariate or should the cohort be stratified
according to sex? Stratifying according to sex requires
performing tests separately on each group, reducing
the sample size but providing separate effect size
estimates for each group. It is easy to imagine other
study factors that may need to be considered in the same
way.
Genomics Studies Involve Many Statistical Tests,
Requiring Special Attention: Due to the nature of
genomics technologies, a large number of features are
evaluated, single nucleotide polymorphisms (SNPs),
resulting in potentially millions of individual statistical
tests. It is essential that adjustments for multiple testing
are applied to limit the likelihood of observing large
numbers of false-positive results. Bonferroni corrections
use a family-wise error rate approach, which aims to
minimize the probability of having false-positive
findings (type I error). It is easy to calculate: just divide
the desired statistical threshold (eg, P < .05) by the
number of independent statistical tests to determine the
adjusted significance threshold. However, this approach
is conservative and is prone to producing false-negative
findings.2 A false discovery rate approach can also be
calculated and aims to adjust the number of false
discoveries. It is less conservative and allows the
investigator to select the threshold for how many false-
positive findings are expected to minimize false-negative
findings.3 The choice between Bonferroni and false
discovery rate approaches will depend on how
conservative the investigators desire is to minimize false-
positive findings.
Although performing multiple testing adjustment is
necessary in GWAS, performing the adjustment based
on the total number of SNPs is inappropriate because
the SNPs are not independent due to linkage
disequilibrium (LD). LD is the nonrandom arrangement
of alleles across the genome and results in SNPs being
correlated with other nearby SNPs. LD is a consequence of
[ 158#1S CHEST JULY 2020 ]
TABLE 1 ] Checklist for Researchers and Statisticians to Evaluate the Performance of Genome-Wide Association
Studies
Ensure the following
The sample is size appropriate for the study
The phenotype is well defined
There is evidence that the phenotype is heritable
The statistical model is appropriate for the phenotype
Dichotomous trait, no covariates: Fisher exact test, c2 test
Dichotomous trait: logistic regression model
Quantitative trait: linear regression model
Multivariate trait: MANOVA/MANCOVA
Categorical/ordinal trait: multinomial logistic regression
Other ______________________________
Sex was appropriately accounted for.
Incorporated as a covariate
Cohort stratified according to sex
Population stratification was appropriately accounted for.
Principal component analysis
Genetic relationship matrix incorporated into statistical model
Cohort stratified
Other ____________________________
Linkage disequilibrium is accounted for in the population stratification analysis, if applicable
If imputation was performed
An appropriate reference population was used for imputation, if applicable
Study-specific features were accounted for (eg, batch effects, medications)
An appropriate genetic model was selected
Additive
Recessive
Dominant
Multiple models considered
Study covariates were appropriately selected
Expert opinion
Selection procedure
An appropriate minor allele frequency was used for the common variant analysis
The results of the covariate selection were adequately described
A quantile-quantile plot was presented, and minimal inflation was observed
A multiple test correction was applied.
Standard genome-wide threshold of P < 5
10–8
Bonferroni adjustment
False discovery rate approach
Other ______________________
If the goal was to create a prediction model
Cross-validation was performed to evaluate overfitting and tested in an independent cohort
If the goal was to perform an association analysis
The associations were replicated in an independent cohort or were functionally validated in an appropriate model
system

MANCOVA ¼ multivariate analysis of covariance; MANOVA ¼ multivariate analysis of variance;

chestjournal.org S115
genetic distance, history of mutation events, and changes
in population dynamics. Studies investigating the number
of unique LD “blocks” across the genome, an estimated
one million independent regions, have determined that
5
10–8 is a reasonable Bonferroni-adjusted threshold for
a type I error rate of 0.05 and has since become the
standard convention for GWAS.4 Multiple testing has
garnered attention because it has been identified as a likely
contributor for why many published studies have failed to
be reproducible.5,6 Due to the aforementioned challenges,
it has now become expected that replication be performed
to verify associations observed in genomics studies. This is
often done through replication in a similar but
independent cohort, or functional validation in an in vivo
or in vitro model system. It is important to consider
options for validating findings prior to initiating the study
because additional funds and expertise may be needed to
confirm discoveries.
The Phenotype Should Be Well Defined: Phenotypic
heterogeneity is increasingly being realized as a challenge
for omics studies. A trait with multiple underlying
etiologies can result in a loss of statistical power. In fact,
Manchia et al7 determined that 50% phenotypic
heterogeneity in GWAS increases the necessary sample
size by three times, suggesting that appropriate
phenotype classification may be more important than
increased sample sizes. Phenotypic heterogeneity can
also pose a challenge for replicating findings; the
cohort and phenotype used for replication must be
appropriately similar to the discovery cohort to be
successful.

TABLE 2 ] Key Stages of Genomics Study Development and Analysis

Stage
Study design and preparation
Data collection
Data processing
Postprocessing data analysis-
common variant analysis
Optional follow-up analyses
Important Considerations
 Determine if evidence exists that the phenotype is heritable: genetics contributes
to phenotypic variation
 Consider whether the phenotype is clearly defined and whether the phenotype
definition will be applied consistently across the entire cohort
 Analysis plan must be appropriate for the study design (eg, cross-sectional,
longitudinal)
 Consider cohort factors that may affect results and ensure that information will be
collected to evaluate or adjust for these impacts (eg, sex, race, socioeconomic
status, age, medications)
 Consider technical factors that may affect results (eg, batch effects, technical
and/or biological replicates)
 Many statistical tests will be conducted; consider impacts on statistical power
after correcting for multiple testing
 Collect data uniformly across the study
 Evaluate phenotypes and/or survey responses periodically for any systematic bias
or incomplete information
 Consider excluding SNPs and individuals with extensive missingness or poor-
quality control metrics
 Test for relatedness if not performing a family-based study
 Account for population stratification
 Select variables that will be adjusted in models (eg, sex, batch, age, medications)
 Identify a minor allele frequency threshold for common and rare variants
 Perform genomic imputation, if desired
 Select an appropriate genetic model (ie, additive, recessive, dominant)
 Select appropriate statistical model based on trait (eg, linear regression, logistic
regression) and add selected variables to model.
 Test each SNP that is above minor allele frequency in the statistical model.
 Create a quantile-quantile plot to assess whether the type I error rate inflation is
observed in the calculated P values
 Perform multiple test correction and create a Manhattan plot to visualize results
 GWAS replication in an external cohort
 Rare variant analysis
 Pathway-based or network-based analysis
 Polygenic risk scores
 Integration with other omics platforms
 Functional follow-up in in vitro or in vivo models

GWAS ¼ genome-wide association studies; SNP ¼ single nucleotide polymorphism.

S116 Supplement [ 158#1S CHEST JULY 2020 ]

TABLE 3 ] Example of SNP Dichotomization for Classical Genetic Modelsa
Unaffected Partially Affected Fully Affected
Model Genotype Genotype Genotype
Recessive AA or AB . BB
Dominant AA . AB or BB
Additive AA AB BB
See Table 2 legend for expansion of abbreviation.
a
In this example, B is the “risk allele” (the allele increasing the trait effect). 0 encoding ¼ AA genotype; 1 encoding ¼ AB genotype; 2 encoding ¼ BB genotype.
There Should Be Evidence of a Genetic Contribution
to the Phenotype: Heritability is the extent of trait
variation due to genetic factors. Pividori et al8 estimated
the heritability of childhood-onset and adult-onset
asthma to be 0.33 and 0.10, respectively. The remaining
fraction of the disease variation is expected to be due to
environmental factors. To observe a genetic association
with a trait, the trait must be heritable. Although having
a heritable trait does not guarantee that genetic
associations will be observed, it is prudent to establish
that the trait is heritable prior to performing GWAS. For
many complex diseases, SNPs explain a relatively small
portion of the overall disease heritability, the remaining
of which is often referred to as “missing heritability” and
is well described by Manolio et al.9 As larger GWAS are
conducted, additional variants with small effect sizes are
being discovered, suggesting that a lack of statistical
power may be the source of missing heritability, which
has been supported by other studies as well.10,11
Although it may be surprising that studies with >
100,000 individuals would be underpowered to find all
the contributing genetic variants, this is possible due to
the potential for thousands of variants to contribute very
small effects on the phenotype and the substantial
multiple testing burden due to millions of statistical tests
being performed. This may be further affected by
phenotype heterogeneity, which can significantly
influence statistical power, as described earlier. Lastly,
with the exception of whole-genome sequencing,
genotyping technologies do not genotype every SNP in
the genome, potentially missing variants contributing to
disease. Another possibility is that heritability methods
overestimate genetic contribution. New bioinformatics
approaches are being developed to try to gain more
precise and accurate measures of heritability.12,13
Important Considerations When Analyzing
and Evaluating Genomics Data
GWAS are now performed routinely and have led to a
substantially improved understanding of disease etiology
as well as a new appreciation for the complexity of many
chestjournal.org
Unaffected SNP Partially Affected SNP Fully Affected SNP
Coding Encoding Encoding
0 or 1 . 2
2 . 1 or 2
0 1 2
diseases. There are now many publicly available
resources for genomics data. The National Institutes of
Health database of Genotypes and Phenotypes is a
searchable repository of downloadable genotype data for
a wide range of diseases (https://www.ncbi.nlm.nih.gov/
gap/). The Cancer Genome Atlas is a cancer-focused
resource that contains genomic data, in addition to
multiple other omics data (https://www.cancer.gov/
about-nci/organization/ccg/research/structural-
genomics/tcga). Lastly, the UK Biobank Resource,14
which recruited 500,000 people with genetic data,
biospecimens, and many other clinical and demographic
measures, has provided a significant resource for
conducting large GWAS. Information regarding the UK
Biobank can be found at https://www.ukbiobank.ac.uk/.
A genome-wide association study of 35,735 patients with
COPD and 222,076 control subjects from the UK
Biobank identified > 80 genome-wide significant loci,
including 35 loci not previously known to be associated
with COPD.15 A recent study of 376,358 individuals
without asthma and those with childhood-onset asthma,
young adults with asthma, or adults with asthma from the
UK Biobank discovered 61 independent asthma loci, with
some loci shared across disease groups and some specific
for childhood-onset or age of onset.8 In addition,
approximately 27 identified loci seem to be novel.
Although these studies represent some of the largest
GWAS conducted to date for disease risk, many others
have been conducted for treatment response.16-18 Several
software tools are available with the capability to perform
analyses like those mentioned earlier. PLINK is a widely
used software tool that uses standard file formats and has
the functionality to perform many of the analyses
described here.19 Additional tools for analyzing
sequencing data include the Broad Institutes, GATK
pipeline (https://software.broadinstitute.org/gatk/), and
Illumina’s DRAGEN Bio-IT Platform.20
Impacts of Racial and Ethnic Heterogeneity
Although the UK Biobank8,15 is an incredibly valuable
resource, it is composed mostly of individuals of European
S117
ancestry and therefore does not adequately represent all
ethnicities. SNPs have different allele frequencies in
different populations. The importance of this can be seen
in the study by Hirota et al,21 which found SNPs
associated with asthma in a Japanese cohort but only a
single SNP they investigated was associated with asthma
in a non-Japanese cohort. Further information about the
challenges of genomics in multi-ethnic cohorts can be
found in the article by Medina-Gomez et al.22 If a
population of individuals has increased disease risk, SNPs
overrepresented in that population, but unrelated to the
disease, may appear enriched for the disease. One solution
is to stratify the cohort according to racial or ethnic
groups; however, it can also be advantageous to analyze an
entire cohort to leverage a larger sample size. Importantly,
even a cohort with the same self-reported race or ethnicity
can be admixed due to nonrandom ancestral mating.23
Studies have been conducted to identify “ancestry
informative markers” or SNPs that display substantial
frequency differences for specific populations, and these
markers have been proposed as a way to better
characterize populations included in a cohort.24-26
Two common approaches are used to account for
population stratification: (1) principal component
analysis27; and (2) mixed models. It is important to
perform SNP pruning, which will filter out highly
correlated SNPs due to LD. Principal component
analysis clusters individuals, and the first few principal
components can be incorporated into regression models
(described later in the text) to adjust for population
stratification.27 Mixed model approaches rely on the
incorporation of a genetic relationship matrix into the
regression model.28 The genetic relationship matrix
consists of pairwise genetic relationship coefficients of
each subject in the analysis. Inadequate control of
population stratification can manifest in type I error rate
inflation. Quantile-quantile (QQ) plots (described later
in the text) can provide critical information regarding
inflation and can highlight potential problems such as
inadequate adjustment for population stratification.
Description of Subtypes of Study Design
SNPs occur at different allele frequencies in various
populations, and SNPs with rare minor allele frequencies
(MAF) pose a statistical challenge because there are
often too few subjects expressing these alleles to be well
powered. For this reason, different approaches are
required to analyze these SNPs resulting in two
categories of genome-wide association study analyses:
common variant analyses and rare variant analyses.
S118 Supplement
The Genome-Wide Association Model: Common
Variants
Associations of commonly occurring genetic variants
with a phenotype is the heart of GWAS and are the
typical analysis presented in the literature regarding
GWAS. Examples of GWAS relevant for chest medicine
have been published previously.8,15-17,21 Building the
appropriate statistical model for GWAS depends on
three main factors: (1) the type of dependent variable (ie,
outcome or trait of interest); (2) the inclusion of
covariates to limit confounding; and (3) the selection of
an appropriate genetic model. If the outcome or trait is
continuous, then a linear regression model is likely the
best choice. However, for binary traits or for time-to-
event data, a logistic regression and Cox proportional
hazards models, respectively, may be more appropriate.
Detailed criteria for selecting a statistical model have
been reported by Long and Freese.29
Software is also available to perform GWAS if you have
a multivariate trait (ie, concentration response data).30
Many study designs contain aspects that could confound
results if not appropriately accounted for in the
statistical model. Examples may include sex, population
stratification, medication usage, socioeconomic status,
and many others. Including these in the model as
covariates can be effective at preventing confounding;
however, including unimportant or correlated variables
can result in a loss of statistical power and unstable
regression coefficients. For this reason, it is necessary to
select an optimal set of variables, while minimizing
model complexity. Multiple approaches exist for this
approach, and a common strategy is to exclude one
variable from pairs of correlated variables. A more
sophisticated approach that has gained favor is the least
absolute shrinkage and selection operator,31 which aims
to minimize the mean squared error (or some other
measure of fit); it does this by penalizing some model
coefficients by shrinking them to zero. Selected features
are those with nonzero coefficients.
Once the proper statistical model has been identified
and features have been selected, the genetic model must
be specified. The three classic genetic models are
dominant, recessive, and additive, although there is no
established mode of inheritance for complex diseases.
Simulation studies indicate that the optimal approach
may be to evaluate all three models together, but special
consideration must be paid to account for multiple
testing with correlated models.32 Table 3 shows the
appropriate SNP encoding based on each of the three
models.
[ 158#1S CHEST JULY 2020 ]
A model must then be created for each individual SNP,
and the statistical test will be to evaluate the
significance of the SNP with the trait while accounting
for the other covariates in the model. SNPs with rare
MAF should be excluded from this analysis due to a
lack of statistical power and their ability to produce
false-positive findings. MAF thresholds typically range
from 1% to 5%, and some approaches have used
inflation factors to determine the optimal thresholds
for filtering rare variants.33 Due to the many statistical
tests being performed, it is critical that multiple test
corrections are applied, and as described earlier, the
standard convention is to use a threshold of P < 5

10–8. Lastly, QQ plots should be developed to
investigate the extent of the type I error rate inflation
present in the results, which could point to potential
confounding. QQ plots provide information regarding
the distribution of observed test results over what
would be expected by chance. Additional information
about QQ plots has been provided by Voorman et al.34
Genomic control is often an effective approach to
quantifying and adjusting results for inflation.35 The
qqman R package is a popular tool to easily create QQ
plots and Manhattan plots.36
The Genome-Wide Association Model: Rare
Variants
Missing heritability may be partly due to large effects
from rare disease-causing variants.9,37 However,
discovering rare disease-causing variants is challenging
because the rarity of the risk allele results in a small
sample size that limits statistical power. Most studies use
an MAF threshold of 1% to 5% to identify rare SNPs.
When testing across the entire genome, it is likely that for
some rare SNPs, the few individuals with the risk allele
may end up all being in the disease group by chance,
resulting in a false-positive result. Many approaches have
been developed attempting to address this challenge, and
two popular classes of methods are burden and variance
component tests. Both of these involve collapsing rare
variants from a region of interest (eg, a gene) into a single
variable. This single variable can then be tested for
association with a phenotype. Simple burden methods use
an a priori selected indicator and a proportion or
weighted approach to collapsing variants. These are often
easily computed from the genotype data.38-40
A major limitation of simple burden tests, however, is
that they cannot account for the direction (positive or
negative association) of a rare variant effect.41 In other
words, two variants in the same region with opposing
chestjournal.org
effects will not cancel each other out. More sophisticated
approaches, such as sequence kernel association tests
(SKATs) allow for different effect directions and
magnitudes for each variant.41 When variants have
different directions of effects, SKATs are more powerful
than burden approaches; however, they are less powerful
when all variant effects are in the same direction. This
led to the development of SKAT-O, an approach aiming
to balance SKATs and burden tests.42 Additional
approaches have been developed for performing SKAT
in the context of family-based study designs
(famSKAT),43 combining rare and common variants
(SKAT-RC)44, and for multiple phenotypes (Multi-
SKAT).45 Methods for rare variant analysis are an active
area of research, and other approaches are also being
considered, such as the Protein Structure Guided Local
Test,46 which aim to improve associations by
incorporating information from protein structure.
What Types of Analyses Can Be Done After
Obtaining the Genome-Wide Association
Study Results?
What Is the Difference Between Association and
Prediction?
Although routinely conflated, association and prediction
models have different requirements and serve different
purposes. Association models determine which factors
explain the most variation in a trait and test for
statistical significance to determine causal associations.
Prediction models try to predict a future event. In
genomics, association models are often used to
determine etiology, whereas prediction models do not
need to inform etiology and are sometimes lacking
interpretability. A strong association between a feature
and a trait does not necessarily indicate that feature will
be able to accurately discriminate individuals, which is
needed for prediction.47 Associations should be
replicated in an independent dataset. Prediction models
should be built into a cross-validation framework to
prevent overfitting and then tested in an independent
test set. Metrics to evaluate predictive performance
include sensitivity, specificity, positive predictive value,
and negative predictive value. Model performance across
various thresholds is usually visualized by using a
receiver-operating characteristic curve. An optimal
threshold might then be selected by maximizing the
sensitivity and/or specificity, based on the clinical need
and an understanding of the cost of misclassification.
The review by Shmueli48 provides additional
information about this concept.
S119
Polygenic Risk Scores Can Be Effective Approaches
for Prediction
Complex diseases and traits are influenced by many
genetic factors. A polygenic risk score (PRS) is designed
to predict an outcome by incorporating information
from multiple genetic variants. Constructing a PRS
consists of taking SNPs above a certain statistical
threshold in a genome-wide association study, and
weighting them by their coefficient from the genome-
wide association study regression models. Each subject
receives a score based on the weighted sum of the risk
alleles.49 It is also important that a PRS is evaluated in a
cross-validation framework to evaluate predictive
performance. A guide to performing PRS has been
developed by Choi et al.50
Pathway Analysis Can Help Put Genomics Studies
Into Biological Context
Results generated from genomics studies can be
notoriously difficult to put into biological context.
Pathway analysis aims to reduce complexity and
improve interpretability of results. Pathway analysis has
been used to identify potential pathways putatively
involved in asthma susceptibility.51,52 Pathway analysis is
an expansive topic, and the following is a brief overview of
the major concepts and considerations with commonly
used approaches. Early pathway analysis methods focused
on gene expression. Pathways, or gene sets, consist of
groups of genes with a coordinated effect on a biological
function. Now pathways can consist of groups of
metabolites, proteins, microRNAs, and other molecular or
biochemical features. Numerous open source knowledge
bases are available that contain pathway information, such
as Gene Ontology53 and Kyoto Encyclopedia of Genes and
Genomes,54 among many others.
Khatri et al55 describes three generations of pathway
analyses. First-generation approaches take features such as
statistically significant genes and then test for enrichment
in a given pathway. A limitation of these approaches is
that they rely on an arbitrary statistical threshold to
determine which features should be evaluated. In addition,
this approach favors large effects being tested for pathway
impacts and has limited ability to detect multiple small
coordinated effects on a pathway. Second-generation
approaches (eg, gene set enrichment analysis56) are
designed to address this limitation by ranking all features
and testing if the ranks are enriched in a pathway.55
Third-generation methods are still evolving and use
topologic information by incorporating the correlation
structure of the features into the pathway. There has been
S120 Supplement
an expansion of these approaches, and a detailed review
and comparison can be found in Ihnatova et al.57
Simulated datasets have been created to properly evaluate
which methods have the best statistical power.58
Genomics data pose an additional challenge for pathway
analysis: knowledge bases contain information at the gene
level, and optimal methods for integrating SNP-level data
with gene sets are still evolving. Methods for aggregating
causal SNPs while minimizing the influence of noncausal
SNPs are needed to effectively determine the impacts on
biological pathways. Several approaches for aggregating
SNP-level data have been developed, such as gene set
analysis-single nucleotide polymorphisms,59 adaptive sum
of powered score,60 and hybrid set-based test.61 More
recently, open-source software, eXploring Genomic
Relations offers a user-friendly suite of tools to leverage
prior biological knowledge and improve interpretability of
genomics data.62 eXploring Genomic Relations does this
by accepting differential expression, SNP, or expression
quantitative trait loci and mapping to multiple knowledge
bases, including gene ontologies, gene networks, gene/SNP
annotations, and genomic annotations.
Additional Considerations Exist
The aim of the current article was to provide a brief
overview of important bioinformatics considerations for
genomics studies. However, this is not a fully
comprehensive evaluation, and there are additional
considerations that should be accounted for. For
example, data-processing strategies for sequencing-
based platforms differ from those of array-based
platforms. These are highly platform specific and
depend on the technology used for the study.
Imputation may be necessary to gain information for
variants not included on the genotyping platform.
Various considerations such as the reference population
ethnicity and quality metrics are important to obtain
reliable imputed variants.63 In addition, family-based
designs have not been discussed here, but it is important
that family structure is accounted for when a cohort
consists of related individuals, and additional software
tools are available to incorporate pedigree information.64
It is likely that epistasis plays an important role in
complex diseases, and approaches are also available that
search for gene-gene interactions. These methods
continue to evolve and have been reviewed by Niel
et al65 and Chatelain et al.66 To obtain additional
statistical power, genomics studies are routinely being
meta-analyzed; a review on meta-analysis approaches
can be found in Evangelou and Ioannidis.67 As
[ 158#1S CHEST JULY 2020 ]
technology continues to advance, analyzing multiple
omics platforms in the same individuals (multi-omics) is
shedding new light on disease mechanisms. Approaches
to simultaneously analyzing multiple omics can be
found by using tools such as Data Integration Analysis
for Biomarker Discovery Using Latent Components.68,69
As the cost of data generation continues to decline, the
importance of bioinformatics is growing. Although
many automated tools and pipelines make data
processing and analysis easily accessible to a wide range
of investigators, it remains important to incorporate
bioinformatics expertise into the study design and
analysis to ensure best practices.
Short List of Questions to Guide the Reviewer
When reviewing a GWAS, consider commenting on the
following:
1. Sample selection, description, and analysis. What
considerations were made when selecting individuals
to be included in the study? Was the sample size
appropriate? Was the race and ethnicity of partici-
pants well described and justified? Were study
covariates appropriately selected and described? Was
sex appropriately accounted for?
2. The phenotype included in the study. Was the
phenotype well-defined and described? Is there evi-
dence that the phenotype is heritable? Could the
phenotype represent multiple underlying disease
etiologies? If so, was this addressed by the authors?
3. The statistical approach. Was the statistical
model appropriate for the study design? Did the
authors account for possible sources of con-
founding (eg, sex, demographics, age, batch ef-
fects)? Was it clearly defined whether the purpose
of the analysis was for association (ie, determining
causality) or prediction of an outcome? Was
multiple test correction applied?
4. The quality control and processing of genomic
data. Was quality control and processing of
genomic data properly described? Were rare var-
iants handled appropriately?
5. Validation of the findings. If the goal was to create
a prediction model, was cross-validation performed
to evaluate overfitting, and was the model tested in
an independent cohort? If the goal was to perform
an association analysis, were the associations repli-
cated in an independent cohort or were they func-
tionally validated in an appropriate model system?
chestjournal.org
Acknowledgments
Financial/nonfinancial disclosures: None declared.
Role of sponsors: The sponsor had no role in the design of the study,
the collection and analysis of the data, or the preparation of the
manuscript.
References
1. Glossary of Genomics Terms. JAMA. 2013;309(14):1533-1535.
2. Andrade C. Multiple testing and protection against a type 1 (false
positive) error using the Bonferroni and Hochberg corrections.
Indian J Psychol Med. 2019;41(1):99.
3. Storey JD. A direct approach to false discovery rates. J R Stat Soc
Series B Stat Methodol. 2002;64(3):479-498.
4. Fadista J, Manning AK, Florez JC, Groop L. The (in) famous GWAS
P-value threshold revisited and updated for low-frequency variants.
Eur J Hum Genet. 2016;24(8):1202.
5. Ioannidis JP. Why most published research findings are false. PLoS
Med. 2005;2(8):e124.
6. Moonesinghe R, Khoury MJ, Janssens ACJ. Most published research
findings are false—but a little replication goes a long way. PLoS Med.
2007;4(2):e28.
7. Manchia M, Cullis J, Turecki G, Rouleau GA, Uher R, Alda M. The
impact of phenotypic and genetic heterogeneity on results of genome
wide association studies of complex diseases. PloS One. 2013;8(10):
e76295.
8. Pividori M, Schoettler N, Nicolae DL, Ober C, Im HK. Shared and
distinct genetic risk factors for childhood-onset and adult-onset
asthma: genome-wide and transcriptome-wide studies. Lancet Respir
Med. 2019;7(6):509-522.
9. Manolio TA, Collins FS, Cox NJ, et al. Finding the missing
heritability of complex diseases. Nature. 2009;461(7265):747.
10. Kruglyak L. Quantitative genetics and the missing heritability
problem. Bulletin of the Am Phys Soc. 2018;7:63.
11. Shirali M, Knott SA, Pong-Wong R, Navarro P, Haley CS. Haplotype
heritability mapping method uncovers missing heritability of
complex traits. Sci Rep. 2018;8(1):4982.
12. Young AI. Solving the missing heritability problem. PLOS Genet.
2019;15(6):e1008222.
13. Hou K, Burch KS, Majumdar A, et al. Accurate estimation of SNP-
heritability from biobank-scale data irrespective of genetic
architecture. bioRxiv. 2019:526855.
14. Sudlow C, Gallacher J, Allen N, et al. UK Biobank: an open access
resource for identifying the causes of a wide range of complex diseases
of middle and old age. PLoS Med. 2015;12(3):e1001779.
15. Sakornsakolpat P, Prokopenko D, Lamontagne M, et al. Genetic
landscape of chronic obstructive pulmonary disease identifies
heterogeneous cell-type and phenotype associations. Nat Genet.
2019;51(3):494-505.
16. Hernandez-Pacheco N, Farzan N, Francis B, et al. Genome-
wide association study of inhaled corticosteroid response in
admixed children with asthma. Clin Exp Allergy. 2019;49(6):
789-798.
17. Dahlin A, Litonjua A, Irvin CG, et al. Genome-wide association
study of leukotriene modifier response in asthma.
Pharmacogenomics J. 2016;16(2):151.
18. Dahlin A, Litonjua A, Lima JJ, et al. Genome-wide association study
identifies novel pharmacogenomic loci for therapeutic response to
montelukast in asthma. PloS One. 2015;10(6):e0129385.
19. Chang CC, Chow CC, Tellier LC, Vattikuti S, Purcell SM, Lee JJ.
Second-generation PLINK: rising to the challenge of larger and
richer datasets. Gigascience. 2015;4(1):7.
20. Goyal A, Kwon HJ, Lee K, et al. Ultra-fast next generation human
genome sequencing data processing using DRAGENTM bio-IT
processor for precision medicine. Open J Genet. 2017;7(1):9-19.
S121
21. Hirota T, Takahashi A, Kubo M, et al. Genome-wide association
study identifies three new susceptibility loci for adult asthma in the
Japanese population. Nature Genet. 2011;43(9):893.
22. Medina-Gomez C, Felix JF, Estrada K, et al. Challenges in
conducting genome-wide association studies in highly admixed
multi-ethnic populations: the Generation R Study. Eur J Epidemiol.
2015;30(4):317-330.
23. Hellwege JN, Keaton JM, Giri A, Gao X, Velez Edwards DR,
Edwards TL. Population stratification in genetic association studies.
Curr Protocols Human Genet. 2017;95(1):1-22.
24. Halder I, Shriver M, Thomas M, Fernandez JR, Frudakis T. A panel
of ancestry informative markers for estimating individual
biogeographical ancestry and admixture from four continents: utility
and applications. Hum Mutation. 2008;29(5):648-658.
25. Enoch MA, Shen PH, Xu K, Hodgkinson C, Goldman D. Using
ancestry-informative markers to define populations and detect
population stratification. J Psychopharmacol. 2006;20(suppl 4):19-26.
26. Galanter JM, Fernandez-Lopez JC, Gignoux CR, et al. Development
of a panel of genome-wide ancestry informative markers to study
admixture throughout the Americas. PLoS Genet. 2012;8(3):
e1002554.
27. Price AL, Patterson NJ, Plenge RM, Weinblatt ME, Shadick NA,
Reich D. Principal components analysis corrects for stratification in
genome-wide association studies. Nature Genet. 2006;38(8):904-909.
28. Shin J, Lee C. A mixed model reduces spurious genetic associations
produced by population stratification in genome-wide association
studies. Genomics. 2015;105(4):191-196.
29. Long JS, Freese J. Regression Models for Categorical Dependent
Variables Using Stata. College Station, TX: Stata Press; 2006.
30. Brown CC, Havener TM, Medina MW, et al. Multivariate methods
and software for association mapping in dose-response genome-wide
association studies. BioData Mining. 2012;5(1).
31. Tibshirani R. Regression shrinkage and selection via the lasso. J R
Stat Soc Series B Stat Methodol. 1996;58(1):267-288.
32. Lettre G, Lange C, Hirschhorn JN. Genetic model testing and
statistical power in population-based association studies of
quantitative traits. Genet Epidemiol. 2007;31(4):358-362.
33. Marvel SW, Rotroff DM, Wagner MJ, et al. Common and rare
genetic markers of lipid variation in subjects with type 2 diabetes
from the ACCORD clinical trial. PeerJ. 2017;5:e3187.
34. Voorman A, Lumley T, McKnight B, Rice K. Behavior of QQ-plots
and genomic control in studies of gene-environment interaction.
PloS One. 2011;6(5):e19416.
35. Devlin B, Roeder K. Genomic control for association studies.
Biometrics. 1999;55(4):997-1004.
36. Turner SD. qqman: an R package for visualizing GWAS results using
QQ and Manhattan plots. BioRciv. 2014005165.
37. Gibson G. Rare and common variants: twenty arguments. Nature
Rev Genet. 2012;13(2):135.
38. Morris AP, Zeggini E. An evaluation of statistical approaches to rare
variant analysis in genetic association studies. Genet Epidemiol.
2010;34(2):188.
39. Morgenthaler S, Thilly WG. A strategy to discover genes that carry
multi-allelic or mono-allelic risk for common diseases: a cohort
allelic sums test (CAST). Mutat Res. 2007;615(1):28-56.
40. Li B, Leal SM. Methods for detecting associations with rare variants
for common diseases: application to analysis of sequence data. Am J
Hum Genet. 2008;83(3):311-321.
41. Wu MC, Lee S, Cai T, Li Y, Boehnke M, Lin X. Rare-variant
association testing for sequencing data with the sequence kernel
association test. Am J Hum Genet. 2011;89(1):82-93.
42. Lee S, Wu MC, Lin X. Optimal tests for rare variant effects in
sequencing association studies. Biostatistics. 2012;13(4):762-775.
43. Chen H, Meigs JB, Dupuis J. Sequence kernel association test for
quantitative traits in family samples. Gen Epidemiol. 2013;37(2):196-
204.
S122 Supplement
44. Ionita-Laza I, Lee S, Makarov V, Buxbaum JD, Lin X. Sequence
kernel association tests for the combined effect of rare and common
variants. Am J Hum Genet. 2013;92(6):841-853.
45. Dutta D, Scott L, Boehnke M, Lee S. Multi-SKAT: general
framework to test for rare-variant association with multiple
phenotypes. Genet Epidemiol. 2019;43(1):4-23.
46. Marceau West R, Lu W, Rotroff DM, et al. Identifying individual risk
rare variants using protein structure guided local tests (POINT).
PLOS Computational Biol. 2019;15(2).
47. McLeod HL, Mariam A, Schveder KA, Rotroff DM. Assessment of
adverse events and their ability to discriminate response to anti-PD-
1/PD-L1 antibody immunotherapy. J Clin Oncol. 2019:JCO1901712.
48. Shmueli G. To explain or to predict? Statistical Sci. 2010;25(3):289-
310.
49. Simonson MA, Wills AG, Keller MC, McQueen MB. Recent
methods for polygenic analysis of genome-wide data implicate an
important effect of common variants on cardiovascular disease risk.
BMC Medical Genet. 2011;12(1):146.
50. Choi SW, Mak TSH, O’Reilly P. A guide to performing polygenic
risk score analyses. BioRxiv. 2018:416545.
51. Song GG, Lee YH. Pathway analysis of genome-wide association
study on asthma. Hum Immunol. 2013;74(2):256-260.
52. Ding L, Abebe T, Beyene J, et al. Rank-based genome-wide analysis
reveals the association of ryanodine receptor-2 gene variants with
childhood asthma among human populations. Hum Genomics.
2013;7(1):16.
53. Gene Ontology Consortium. The gene ontology resource: 20
years and still GOing strong. Nucleic Acids Res. 2018;47(D1):
D330-D338.
54. Kanehisa M, Sato Y, Furumichi M, Morishima K, Tanabe M. New
approach for understanding genome variations in KEGG. Nucleic
Acids Res. 2018;47(D1):D590-D595.
55. Khatri P, Sirota M, Butte AJ. Ten years of pathway analysis: current
approaches and outstanding challenges. PLoS Comput Biol.
2012;8(2):e1002375.
56. Subramanian A, Tamayo P, Mootha VK, et al. Gene set enrichment
analysis: a knowledge-based approach for interpreting genome-wide
expression profiles. Proc Natl Acad Sci U S A. 2005;102(43):15545-
15550.
57. Ihnatova I, Popovici V, Budinska E. A critical comparison of
topology-based pathway analysis methods. PloS One. 2018;13(1):
e0191154.
58. Mathur R, Rotroff D, Ma J, Shojaie A, Motsinger-Reif A. Gene set
analysis methods: a systematic comparison. BioData Mining.
2018;11(1):8.
59. Nam D, Kim J, Kim SY, Kim S. GSA-SNP: a general approach for
gene set analysis of polymorphisms. Nucleic Acids Res. 2010;38(suppl
2):W749-W754.
60. Pan W, Kwak IY, Wei P. A powerful pathway-based adaptive test for
genetic association with common or rare variants. Am J Hum Genet.
2015;97(1):86-98.
61. Li MX, Kwan JS, Sham PC. HYST: a hybrid set-based test for
genome-wide association studies, with application to protein-protein
interaction-based association analysis. Am J Hum Genet. 2012;91(3):
478-488.
62. Fang H, Knezevic B, Burnham KL, Knight JC. XGR software for
enhanced interpretation of genomic summary data, illustrated by
application to immunological traits. Genome Med. 2016;8(1):129.
63. Shriner D, Adeyemo A, Chen G, Rotimi CN. Practical considerations
for imputation of untyped markers in admixed populations. Genet
Epidemiol. 2010;34(3):258-265.
64. Ott J, Kamatani Y, Lathrop M. Family-based designs for genome-
wide association studies. Nat Rev Genet. 2011;12(7):465-474.
65. Niel C, Sinoquet C, Dina C, Rocheleau G. A survey about methods
dedicated to epistasis detection. Frontiers Genet. 2015;6:285.
[ 158#1S CHEST JULY 2020 ]
66. Chatelain C, Durand G, Thuillier V, Augé F. Performance of
epistasis detection methods in semi-simulated GWAS. BMC
Bioinformatics. 2018;19(1):231.
67. Evangelou E, Ioannidis JPA. Meta-analysis methods for genome-
wide association studies and beyond. Nature Rev Genet. 2013;14(6):
379-389.
chestjournal.org
68. Rohart F, Gautier B, Singh A, Lê Cao KA. mixOmics: an R package
for ‘omics feature selection and multiple data integration. PLoS
Comput Biol. 2017;13(11):e1005752.
69. Singh A, Shannon CP, Gautier B, et al. DIABLO: an
integrative approach for identifying key molecular drivers from multi-
omics assays. Bioinformatics. 2019;35(17):3055-3062.
S123