Download as pdf or txt
Download as pdf or txt
You are on page 1of 37

bioRxiv preprint doi: https://doi.org/10.1101/587360.

The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

1 Cell BLAST: Searching large-scale scRNA-seq database via


2 unbiased cell embedding

3 Zhi-Jie Cao, Lin Wei, Shen Lu, De-Chang Yang, Ge Gao*


4 1
Biomedical Pioneering Innovation Center (BIOPIC), Beijing Advanced Innovation Center for
5 Genomics (ICG), Center for Bioinformatics (CBI), State Key Laboratory of Protein and Plant Gene
6 Research, School of Life Sciences, Peking University, Beijing, 100871, China
7 2
College of Life Sciences, Beijing Normal University, Beijing, 100875, China

8 * To whom correspondence should be addressed. Tel: +86-010-62755206; Email: gaog@mail.cbi.pku.edu.cn


bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

9 Abstract

10 Large amount of single-cell RNA sequencing data produced by various technologies is


11 accumulating rapidly. An efficient cell querying method facilitates integrating existing data
12 and annotating new data. Here we present a novel cell querying method Cell BLAST based
13 on deep generative modeling, together with a well-curated reference database and a user-
14 friendly Web interface at http://cblast.gao-lab.org, as an accurate and robust solution to large-
15 scale cell querying.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

16 Main Text

17 Technological advances during the past decade have led to rapid accumulation of large-scale
18 single-cell RNA sequencing (scRNA-seq) data. Analogous to biological sequence analysis1,
19 identifying expression similarity to well-curated references via a cell querying algorithm is
20 becoming the first step of annotating newly sequenced cells. Tools have been developed to
21 identify similar cells using approximate cosine distance2 or LSH Hamming distance3, 4
22 calculated from a subset of carefully selected genes. Such intuitive approach is efficient,
23 especially for large-scale data, but may suffer from non-biological variation across datasets,
24 i.e. batch effect5, 6. Meanwhile, multiple data harmonization methods have been proposed to
25 remove such confounding factors during alignment, for example, via warping canonical
26 correlation vectors7 or matching mutual nearest neighbors across batches6.While these
27 methods can be applied to align multiple reference datasets, computation-intensive
28 realignment is required for mapping query cells to the (pre-)aligned reference data space.
29
30 Here we introduce a new customized deep generative model together with a cell-to-cell
31 similarity metric specifically designed for cell querying to address these challenges (Figure
32 1A, Method). Differing from canonical variational autoencoder (VAE) models8-11, adversarial
33 batch alignment is applied to correct batch effect during low-dimensional embedding of
34 reference datasets. Query cells can be readily mapped to the batch-corrected reference space
35 due to parametric nature of neural networks. Such design also enables a special “online
36 tuning” mode which is able to handle batch effect between query and reference when
37 necessary. Moreover, by exploiting the model’s universal approximator posterior to model
38 uncertainty in latent space, we implement a distribution-based metric for measuring cell-to-
39 cell similarity. Last but not least, we also provide a well-curated multi-species single-cell
40 transcriptomics database (ACA) and an easy-to-use Web interface for convenient exploratory
41 analysis.
42
43 To assess our model’s capability to capture biological similarity in the low-dimensional latent
44 space, we first benchmarked against several popular dimension reduction tools8, 12, 13 using
45 real-world data (Supplementary Table 1), and found that our model is overall among the best
46 performing methods (Supplementary Figure 1-2). We further compared batch effect
47 correction performance using combinations of multiple datasets with overlapping cell types
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

48 profiled (Supplementary Table 1). Our model achieves significantly better dataset mixing
49 (Figure 1B) while maintaining comparable cell type resolution (Figure 1C). Latent space
50 visualization also demonstrates that our model is able to effectively remove batch effect for
51 multiple datasets with considerable difference in cell type distribution (Supplementary Figure
52 3). Of note, we found that the correction of inter-dataset batch effect does not automatically
53 generalize to that within each dataset, which is most evident in the pancreatic datasets
54 (Supplementary Figure 3C-D, Supplementary Figure 4A-C). For such complex scenarios, our
55 model is flexible in removing multiple levels of batch effect simultaneously (Supplementary
56 Figure 4D-H).
57
58 While the unbiased latent space embedding derived by nonlinear deep neural network
59 effectively removes confounding factors, the network’s random components and nonconvex
60 optimization procedure also lead to serious challenges, especially false positive hits when
61 cells outside reference types are provided as query. Thus, we propose a novel probabilistic
62 cell-to-cell similarity metric in latent space based on posterior distribution of each cell, which
63 we term “normalized projection distance” (NPD). Distance metric ROC analysis (Method)
64 shows that our posterior NPD metric is more accurate and robust than Euclidean distance,
65 which is commonly used in other neural network-based embedding tools (Figure 1D).
66 Additionally, we exploit stability of query-hit distance across multiple models to improve
67 specificity (Methods, Supplementary Figure 4L). An empirical p-value is computed for each
68 query hit as a measure of “confidence”, by comparing posterior distance to the empirical
69 NULL distribution obtained from randomly selected pairs of cells in the queried database.
70
71 The high specificity of Cell BLAST is especially important for discovering novel cell types.
72 Two recent studies (“Montoro”14 and “Plasschaert”15) independently reported a rare tracheal
73 cell type named pulmonary ionocyte. We artificially removed ionocytes from the “Montoro”
74 dataset, and used it as reference to annotate query cells from the “Plasschaert” dataset. In
75 addition to accurately annotating 95.9% of query cells, Cell BLAST correctly rejects 12 out
76 of 19 “Plasschaert” ionocytes (Figure 1E). Moreover, it highlights the existence of a putative
77 novel cell type as a well-defined cluster with large p-values among all 156 rejected cells,
78 which corresponds to ionocytes (Figure 1F-G, Supplementary Figure 6A, also see
79 Supplementary Figure 5 for more detailed analysis on the remaining 7 cells). In contrary,
80 scmap-cell2 only rejected 7 “Plasschaert” ionocytes despite higher overall rejection number
81 of 401 (i.e. more false negatives, Supplementary Figure 6B-E).
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

82
83 We further systematically compared the performance of query-based cell typing with scmap-
84 cell2 and CellFishing.jl4 using four groups of datasets, each including both positive control
85 and negative control queries (first 4 groups in Supplementary Table 2). Of interest, while Cell
86 BLAST shows superior performance than scmap-cell and CellFishing.jl under the default
87 setting (Supplementary Figure 7A-C, 8-10), detailed ROC analysis reveals that performance
88 of scmap-cell could be further improved to a level comparative to Cell BLAST by employing
89 higher thresholds, while ROC and optimal thresholds of CellFishing.jl show large variation
90 across different datasets (Supplementary Figure 7D). Cell BLAST presents the most robust
91 performance with default threshold (p-value < 0.05) across multiple datasets, which largely
92 benefits real-world application. Additionally, we also assessed their scalability using
93 reference data varying from 1,000 to 1,000,000 cells. Both Cell BLAST and CellFishing.jl
94 scale well with increasing reference size, while scmap-cell’s querying time rises dramatically
95 for larger reference dataset with more than 10,000 cells (Supplementary Figure 7E).
96
97 Moreover, our deep generative model combined with posterior-based latent-space similarity
98 metric enables Cell BLAST to model continuous spectrum of cell states accurately. We
99 demonstrate this using a study profiling mouse hematopoietic progenitor cells (“Tusi”16), in
100 which computationally inferred cell fate distributions are available. For the purpose of
101 evaluation, cell fate distributions inferred by authors are recognized as ground truth. We
102 selected cells from one sequencing run as query and the other as reference to test if we can
103 accurately transfer continuous cell fate between experimental batches (Figure 2A-B). Jensen-
104 Shannon divergence between prediction and ground truth shows that our prediction is again
105 more accurate than scmap (Figure 2C).
106
107 Besides batch effect among multiple reference datasets, bona fide biological similarity could
108 also be confounded by large, undesirable bias between query and reference data. Taking
109 advantage of the dedicated adversarial batch alignment component, we implemented a
110 particular "online tuning" mode to handle such often-neglected confounding factor. Briefly,
111 the combination of reference and query data is used to fine-tune the existing reference-based
112 model, with query-reference batch effect added as an additional component to be removed by
113 adversarial batch alignment (Method). Using this strategy, we successfully transferred cell
114 fate from the above “Tusi” dataset to an independent human hematopoietic progenitor dataset
115 (“Velten”17) (Figure 2D). Expression of known cell lineage markers validates the rationality
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

116 of transferred cell fates (Supplementary Figure 11A-F). In contrary, scmap-cell incorrectly
117 assigned most cells to monocyte and granulocyte lineages (Supplementary Figure 11G). As
118 another example, we applied “online tuning” to Tabula Muris18 spleen data, which exhibits
119 significant batch effect between 10x and Smart-seq2 processed cells. ROC of Cell BLAST
120 improves significantly after “online tuning”, achieving high specificity, sensitivity and
121 Cohen’s κ at the default cutoff (Supplementary Figure 11H, last group in Supplementary
122 Table 2).
123
124 A comprehensive and well-curated reference database is crucial for practical application of
125 Cell BLAST. Based on public scRNA-seq datasets, we developed a comprehensive Animal
126 Cell Atlas (ACA). With 986,305 cells in total, ACA currently covers 27 distinct organs
127 across 8 species, offering the most comprehensive compendium for diverse species and
128 organs (Figure 2E, Supplementary Figure 12A-B, Supplementary Table 3). To ensure unified
129 and high-resolution cell type description, all records in ACA are collected and annotated with
130 a standard procedure (Method), with 95.4% of datasets manually curated with Cell Ontology,
131 a structured controlled vocabulary for cell types.
132
133 A user-friendly Web server is publicly accessible at http://cblast.gao-lab.org, with all curated
134 datasets and pretrained models available, providing “off-the-shelf” querying service. Of note,
135 we found that our model works well on all ACA datasets with minimal hyperparameter
136 tuning (latent space visualization available on the website). Based on this wealth of resources,
137 users can obtain querying hits and visualize cell type predictions with minimal effort
138 (Supplementary Figure 12C-D). For advanced users, a well-documented python package
139 implementing the Cell BLAST toolkit is also available, which enables model training on
140 custom references and diverse downstream analyses.
141
142 By explicitly modeling multi-level batch effect as well as uncertainty in cell-to-cell similarity
143 estimation, Cell BLAST is an accurate and robust querying algorithm for heterogeneous
144 single-cell transcriptome datasets. In combination with a comprehensive, well-annotated
145 database and an easy-to-use Web interface, Cell BLAST provides a one-stop solution for
146 both bench biologists and bioinformaticians
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

147 Software availability

148 The full package of Cell BLAST is available at http://cblast.gao-lab.org

149 Acknowledgements

150 The authors thank Drs. Zemin Zhang, Cheng Li, Letian Tao, Jian Lu and Liping Wei at
151 Peking University for their helpful comments and suggestions during the study.
152 This work was supported by funds from the National Key Research and Development
153 Program (2016YFC0901603), the China 863 Program (2015AA020108), as well as the State
154 Key Laboratory of Protein and Plant Gene Research and the Beijing Advanced Innovation
155 Center for Genomics (ICG) at Peking University. The research of G.G. was supported in part
156 by the National Program for Support of Top-notch Young Professionals.
157 Part of the analysis was performed on the Computing Platform of the Center for Life
158 Sciences of Peking University, and supported by the High-performance Computing Platform
159 of Peking University.

160 Author contributions

161 G.G. conceived the study and supervised the research; Z.J.C. and L.W. contributed to the
162 computational framework and data curation; S. L., Z.J.C., and D.C.Y designed, implemented
163 and deployed the website; Z.J.C. and G.G. wrote the manuscript with comments and inputs
164 from all co-authors.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A
1 Unbiased cell embedding
ACA
Genes Exprs
A1BG 0

Indexing
A1CF 2
... ...
ZZZ3 0

Compute posterior distance


2 and empirical p-values 3 Neighbor based predictions

Annotate existing cell types

Prioritize new cell types


Querying
Transfer continuous cell fate

......

B C 1.00 D

0.75
0.75
Mean Average Precision
Seurat Alignment Score

Method Method
PCA PCA
0.50 MNN MNN
0.50
CCA CCA
scVI scVI
Cell BLAST Cell BLAST

0.25 0.25
N.A.
N.A.

N.A.
N.A.

0.00 0.00

Baron_human Montoro_10x Baron_human Quake_Smart−seq2 Baron_human Montoro_10x Baron_human Quake_Smart−seq2


Baron_mouse Plasschaert Muraro Quake_10x Baron_mouse Plasschaert Muraro Quake_10x
Enge Enge
Segerstolpe Segerstolpe
Xin_2016 Xin_2016
Lawlor Lawlor
Dataset Dataset

E Plasschaert to Montoro_10x_no_ionocyte
F G

basal cell of epithelium of trachea


basal cell of epithelium of trachea

brush cell of trachea


brush cell of trachea
rejected

ciliated columnar cell of tracheobronchial tree ciliated columnar cell of tracheobronchial tree

club cell club cell

ionocyte ambiguous
tracheal goblet cell
lung neuroendocrine cell
lung neuroendocrine cell

165 Figure. 1 Cell BLAST benchmarking and application to trachea datasets.


166 (A) Overall Cell BLAST workflow. (B) Extent of dataset mixing after batch effect correction in four groups of
167 datasets, quantified by Seurat alignment score. High Seurat alignment score indicates that local neighborhoods
168 consist of cells from different datasets uniformly rather than the same dataset only. Methods that did not finish
169 under 2-hour time limit are marked as N.A. (C) Cell type resolution after batch effect correction, quantified by
170 cell type mean average precision (MAP). MAP can be thought of as a generalization to nearest neighbor
171 accuracy, with larger values indicating higher cell type resolution, thus more suitable for cell querying. Methods
172 that did not finish under 2-hour time limit are marked as N.A. (D) ROC curve of different distance metrics in
173 discriminating cell pairs with the same cell type from cell pairs with different cell types. (E) Sankey plot
174 comparing Cell BLAST predictions and original cell type annotations for the “Plasschaert” dataset. (F) t-SNE
175 visualization of Cell BLAST rejected cells, colored by unsupervised clustering.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A B C

D E
#$%&'()"('')
!"! /(01(,& #"*+,-.'(% 2345!#(678
*+,-.'
/90.% : ; < = =
>+92( ?? @A ; ; B
C$D.,E @ B B B B
F9,-'( @ B B B B
G(1,.H$2I < @ B B B
*'.%.,$. @ B B B B
7,+2+JI$'. @ @ B B B
5(0.-+E( @ B B B B
/KE,. B @ B B B

176 Figure. 2 Application to hematopoietic progenitor datasets.


177 (A, B) UMAP visualization of latent space learned on the “Tusi” dataset, colored by sequencing run (A) and cell
178 fate (B). Model is trained solely on cells from run 2 and used to project cells from run 1. Each one of the seven
179 terminal cell fates (E, erythroid; Ba, basophilic or mast; Meg, megakaryocytic; Ly, lymphocytic; D, dendritic;
180 M, monocytic; G, granulocytic neutrophil) are assigned a distinct color. Color of each single cell is then
181 determined by the linear combination of these seven colors in hue space, weighed by cell fate distribution
182 among these terminal fates. (C) Distribution of Jensen-Shannon divergence between predicted cell fate
183 distributions and author provided “ground truth”. (D) UMAP visualization of the “Velten” dataset, colored by
184 Cell BLAST predicted cell fates. (E) Number of organs covered in each species for different single-cell
185 transcriptomics databases, including Single Cell Portal (https://portals.broadinstitute.org/single_cell); Hemberg
186 collection2; SCPortalen19; scRNASeqDB20.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

187 Methods

188 The deep generative model

189 The model we used is based on adversarial autoencoder (AAE)21. Denote the gene expression
190 profile of a cell as 𝒙 ∈ ℝ$ . The data generative process is modeled by a continuous latent
191 variable 𝒛 ∈ ℝ& (𝐷 ≪ 𝐺) with Gaussian prior 𝒛 ∼ 𝒩(𝟎, 𝑰& ), as well as a one-hot latent
192 variable 𝒄 ∈ {0,1}7 , 𝒄8 𝒄 = 1 with categorical prior 𝒄 ∼ 𝐶𝑎𝑡(𝐾), which aims at modeling
193 cell type clusters. A unified latent vector is then determined by 𝒍 = 𝒛 + 𝐻𝒄, where 𝐻 ∈
194 ℝ&×7 . A neural network (decoder, denoted by Dec below) maps the cell embedding vector 𝒍
195 to two parameters of the negative binomial distribution 𝝁, 𝜽 = Dec(𝒍) that models the
196 distribution of expression profile 𝒙:
$

𝑝(𝒙|𝒛, 𝒄; Dec) = 𝑝(𝒙|𝝁, 𝜽) = J 𝑝K𝑥M N𝜇M , 𝜃M Q (1)


MRS

𝑝K𝑥M N𝜇M , 𝜃M Q =
W YX (2)
ΓK𝑥M + 𝜃M Q 𝜇M X
𝜃M
U V U V
ΓK𝜃M QΓK𝑥M + 1Q 𝜃M + 𝜇M 𝜃M + 𝜇M
197 Where 𝝁 and 𝜽 are mean and dispersion of the negative binomial distribution. Theoretically,
198 the negative binomial model should be fitted on raw count data8, 13, 22. However, for the
199 purpose of cell querying, datasets have to be normalized to minimize the influence of capture
200 efficiency and sequencing depth. We empirically found that on normalized data, negative
201 binomial still produced better results than alternative distributions like log-normal. To
202 prevent numerical instability caused by normalization breaking the mean-variance relation of
203 negative binomial, we additionally include variance of the dispersion parameter as a
204 regularization term.
205 Training objectives for the adversarial autoencoder are:
min −𝔼𝒙∼defgf(𝒙) h𝔼𝒛∼i(𝒛|𝒙;`a_),𝒄∼i(𝒄|𝒙;`a_) log 𝑝(𝒙|𝒛, 𝒄; Dec) + 𝜆n
]^_,`a_
(3)
⋅ 𝔼𝒛∼i(𝒛|𝒙;`a_) log Dp (𝒛) + 𝜆q ⋅ 𝔼𝒄∼i(𝒄|𝒙;`a_) log D_ (𝒄)r
max 𝜆n ⋅ K𝔼𝒛∼d(𝒛) [log Dp (𝒛)] + 𝔼𝒙∼defgf(𝒙) 𝔼𝒛∼i(𝒛|𝒙;`a_) hlogK1 − Dp (𝒛)QrQ (4)
]u

max 𝜆q ⋅ K𝔼𝒄∼d(𝒄) [log D_ (𝒄)] + 𝔼𝒙∼defgf(𝒙) 𝔼𝒄∼i(𝒄|𝒙;`a_) hlogK1 − D_ (𝒄)QrQ (5)


]x

206 𝑞(𝒛|𝒙; Enc) and 𝑞(𝒄|𝒙; Enc) are “universal approximator posteriors” parameterized by
207 another neural network (encoder, denoted by Enc). Expectations with regard to 𝑞(𝒛|𝒙; Enc)
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

208 and 𝑞(𝒄|𝒙; Enc) are approximated by sampling 𝒙’ ∼ 𝑃𝑜𝑖𝑠𝑠𝑜𝑛(𝒙) and feeding to the
209 deterministic encoder network. Dn and Dq are discriminator networks for 𝒛 and 𝒄 which
210 output the probability that a latent variable sample is from the prior rather than the posterior.
211 Effectively, adversarial training between encoder (Enc) and discriminators (Dn and Dq ) drives
212 encoder output to match prior distributions of latent variables 𝑝(𝒛) and 𝑝(𝒄). 𝜆n and 𝜆q are
213 hyperparameters that control prior matching strength. The model is much easier and more
214 stable to train compared with canonical GANs because of low dimensionality and simple
215 distribution of 𝒛 and 𝒄.
216 At convergence, the encoder learns to map data distribution to latent variables that follow
217 their respective prior distributions and the decoder learns to map latent variables from prior
218 distributions back to data distribution. The key element we use for cell querying is vector 𝒍 on
219 the decoding path, as it defines a unified latent space in which biological similarities are well
220 captured. The model also works if no categorical latent variable is used, in which case 𝒍 = 𝒛
221 directly.
222 Some architectural designs are learned from scVI8, including logarithm transformation before
223 encoder input, and softmax output scaled by library size when computing 𝝁. Stochastic
224 gradient descent with minibatches is applied to optimize the loss functions. Specifically, we
225 use the “RMSProp” optimization algorithm with no momentum term to ensure stability of
226 adversarial training. The model is implemented using Tensorflow23 python library.

227 Adversarial batch alignment

228 As a natural extension to the prior matching adversarial training strategy described in the
229 previous section, also following recent works in domain adaptation24-26, we propose the
230 adversarial batch alignment strategy to align the latent space distribution of different batches.
231 Below, we extend derivation in the original GAN paper27 to show that adversarial batch
232 alignment converges when embedding space distributions of different batches are aligned.
233 In the case of multiple batches, we assume that marginal data distribution can be factorized as
234 below:
𝑝•‚ƒ‚ (𝒙, 𝒃) = 𝑝(𝒃)𝑝(𝒙|𝒃) (6)

235 𝒃 ∈ {0,1}… , 𝒃8 𝒃 = 1 denotes a one-hot batch indicator, and batch distribution 𝑝(𝒃) is
236 categorical:

𝑝(𝑏‡ = 1) = 𝑤‡ , ‰ 𝑤‡ = 1 (7)
‡RŠ
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

237 Adversarial batch alignment introduces an additional loss:


min 𝔼𝒃∼d(𝒃),𝒙∼d(𝒙|𝒃) 𝔼𝒍∼‹(𝒛Œ𝒄•Ž𝒍)i(𝒛|𝒙)i(𝒄|𝒙) [ℒ •‚‘’ + 𝜆• ⋅ 𝒃8 log D“ (𝒍)] (8)
]^_,`a_

max 𝔼𝒃∼d(𝒃),𝒙∼d(𝒙|𝒃) 𝔼𝒍∼‹(𝒛Œ𝒄•Ž𝒍)i(𝒛|𝒙)i(𝒄|𝒙) [𝜆• ⋅ 𝒃8 log D“ (𝒍)] (9)


]”

238 ℒ •‚‘’ denotes the loss function in (3). D• is a multi-class batch discriminator network that
239 outputs the probability distribution of batch membership based on embedding vector 𝒍. 𝜆• is a
240 hyperparameter controlling batch alignment strength. Additionally, the generative
241 distribution is extended to condition on 𝒃 as well:
𝑝(𝒙|𝒛, 𝒄, 𝒃; Dec) = 𝑝(𝒙|𝝁, 𝜽)
𝝁, 𝜽 = Dec(𝒍, 𝒃) (10)

𝒍 = 𝒛 + 𝒄𝐻
242 Now, we focus on batch alignment and discard the first ℒ •‚‘’ term and scaling parameter 𝜆• .
243 To simplify notation, we fuse data distribution and encoder transformation, and replace the
244 minimization over encoder to minimization over batch embedding distributions:

min ‰ 𝑤‡ 𝔼𝒍∼d•(𝒍) log D“ ‡ (𝒍) (11)


d(𝒍)
‡RS

max ‰ 𝑤‡ 𝔼𝒍∼d• (𝒍) log D“ ‡ (𝒍) (12)


]”
‡RS

245 Here D•• (𝒍) denotes the i dimension of discriminator output, i.e. probability that the
th

246 discriminator “thinks” a cell is from the ith batch. D• is assumed to have sufficient capacity,
247 which is generally reasonable in the case of neural networks. Global optimum of (12) is
248 reached when D• outputs optimal batch membership distribution at every 𝒍:

max 𝑤‡ 𝑝‡ (𝒍) log D“ ‡ (𝒍) , 𝑠. 𝑡. ‰ D“ ‡ (𝒍) = 1 (13)


]” • (𝒍)
‡RS

249 Solution to the above maximization is given by:


𝑤‡ 𝑝‡ (𝒍)
D“ ∗ ‡ (𝒍) = (14)
∑…‡RS 𝑤‡ 𝑝‡ (𝒍)
250 Substituting D• ∗ (𝑙) back into (11), we obtain:
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.


𝑤‡ 𝑝‡ (𝒍)
‰ 𝑤‡ 𝔼𝒍∼d•(𝒍) log
∑…‡RS 𝑤‡ 𝑝‡ (𝒍)
‡RS
… …
𝑝‡ (𝒍)
= ‰ 𝑤‡ 𝔼𝒍∼d• (𝒍) log … + ‰ 𝑤‡ 𝔼𝒍∼d•(𝒍) log 𝑤‡
∑‡RS 𝑤‡ 𝑝‡ (𝒍)
‡RS ‡RS
(155)
… … …

= ‰ 𝑤‡ ⋅ KL œ𝑝‡ (𝒍) ∥ ‰ 𝑤‡ 𝑝‡ (𝒍)ž + ‰ 𝑤‡ log 𝑤‡


‡RS ‡RS ‡RS

≥ ‰ 𝑤‡ log 𝑤‡
‡RS

251 Thus ∑…‡RS 𝑤‡ log 𝑤‡ is the global minimum, reached if and only if 𝑝‡ (𝒍) = 𝑝M (𝒍), ∀𝑖, 𝑗. The
252 minimization of (11) is equivalent to minimizing a form of generalized Jensen-Shannon
253 divergence among multiple batch embedding distributions.
254 Note that in practice, model training looks for a balance between ℒ •‚‘’ and pure batch
255 alignment. Aligning cells of the same type induces minimal cost in ℒ •‚‘’ , while cells of
256 different types could cause ℒ•‚‘’ to rise dramatically if improperly aligned. During training,
257 gradient from both batch discriminators and decoder provide fine-grain guidance to align
258 different batches, leading to better results than “hand-crafted” alignment strategies like CCA7
259 and MNN6. Empirically, given proper values for 𝜆• , the adversarial approach correctly
260 handles difference in cell type distribution among batches. If multiple levels of batch effect
261 exist, e.g. within-dataset and cross-dataset, we use an independent batch discriminator for
262 each component, providing extra flexibility.

263 Data preprocessing for benchmarks

264 Most informative genes were selected using the Seurat7 function “FindVariableGenes”. We
265 set the argument “binning.method” to “equal_frequency” and left other arguments as default.
266 If within dataset batch effect exists, genes are selected independently for each batch and then
267 pooled together. By default, a gene is retained if it is selected in at least 50% of batches.
268 Downstream benchmarks were all performed using this gene set, except for scmap and
269 CellFishing.jl which provide their own gene selection method. GNU parallel28 was used to
270 parallelize and manage jobs throughout the benchmarking and data processing pipeline.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

271 Benchmarking dimension reduction

272 PCA was performed using the R package irlba29 (v2.3.2). ZIFA12 was downloaded from its
273 Github repository, and hard coded random seeds were removed to reveal actual stability.
274 ZINB-WaVE13 (v1.0.0) was performed using the R package zinbwave. scVI8 (v0.2.3) was
275 downloaded from its Github repository, and minor changes were made to the original code to
276 address PyTorch30 compatibility issues. Our modified versions of ZIFA and scVI are
277 available upon request.
278 For PCA and ZIFA, data was logarithm transformed after normalization and adding a
279 pseudocount of 1. Hyperparameters of all methods above were left as default. For our model,
280 we used the same set of hyperparameters throughout all benchmarks. 𝜆n and 𝜆q are both set to
281 0.001. All neural networks (encoder, decoder and discriminators) use a single layer of 128
282 hidden units. Learning rate of RMSProp optimizer is set to 0.001, and minibatches of size
283 128 are used. For comparability, target dimensionality of each method was set to 10. All
284 benchmarked methods were repeated multiple times with different random seeds. Run time
285 was limited to 2 hours, after which the job would be terminated.
286 Cell type nearest neighbor mean average precision (MAP) was computed with K nearest
287 neighbors of each cell based on low dimensional space Euclidean distance. Denote cell type
288 of a cell as 𝑦, and cell type of its ordered nearest neighbors as 𝑦S , 𝑦£ , … 𝑦¥ . Average precision
289 (AP) for that cell is defined as:

7
∑¥¥ ª RS 1¨R¨©ª
∑¥RS 1¨R¨© ⋅
AP = 𝑘 (16)
7
∑¥RS 1¨R¨©
290 Mean average precision is then given by:
®
1
MAP = ‰ AP‡ (17)
𝑁
‡RS

291 Note that when 𝐾 = 1, MAP reduces to nearest neighbor accuracy. We set 𝐾 to 1% of total
292 cell number throughout all benchmarks.

293 Benchmarking batch effect correction

294 We merge multiple datasets according to shared gene names. If datasets to be merged are
295 from different species, Ensembl ortholog31 information is used to map them to ortholog
296 groups. To obtain informative genes in merged datasets, we take the union of informative
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

297 genes from each dataset, and then intersect the union with the intersection of detected genes
298 from each dataset.
299 CCA7 and MNN6 alignments were performed using R packages Seurat7 (v2.3.3) and scran32
300 (v1.6.9) respectively. Hard coded random seeds in Seurat were removed to reveal actual
301 stability. The modified version of Seurat is available upon request. For comparability, we
302 evaluated cell type resolution and batch mixing in a 10-dimensional latent space. For MNN
303 alignment, we set argument “cos.norm.out” to false, and left other arguments as default. PCA
304 was applied to reduce dimensionality to 10 after obtaining the MNN corrected expression
305 matrix. For CCA alignment, we used the first 10 canonical correlation vectors. Run time was
306 limited to 2 hours, after which the job would be terminated. Seurat alignment score was
307 computed exactly as described in the CCA alignment paper7.

308 Cell querying based on posterior distributions

309 We evaluate cell-to-cell similarity based on posterior distribution distance. Like in the
310 training phase, we obtain samples from the “universal approximator posterior” by sampling
311 𝒙’ ∼ 𝑃𝑜𝑖𝑠𝑠𝑜𝑛(𝒙) and feeding to the encoder network. In order to obtain robust estimation of
312 distribution distance with a small number of posterior samples, we project posterior samples
313 of two cells onto the line connecting their posterior point estimates in the latent space, and
314 use projected scalar distribution distance to approximate true distribution distance.
315 Wasserstein distance is computed on normalized projections to account for non-uniform
316 density across the embedding space:
1
𝑁𝑃𝐷(𝑝, 𝑞 ) = ⋅ °𝑊S ²𝑧d (𝑝), 𝑧d (𝑞 )´ + 𝑊S ²𝑧i (𝑝), 𝑧i (𝑞 )´µ (18)
2
317 Where

318 𝑊S (𝑢, 𝑣) = inf ½|𝑥 − 𝑦|𝑑𝜋(𝑥, 𝑦)


¹∈º(»,¼)

𝑣 − 𝔼( 𝑢 )
319 𝑧» (𝑣) =
À𝑣𝑎𝑟(𝑢)
320 We term this distance metric normalized projection distance (NPD). By default, 50 samples
321 from the posterior are used to compute NPD, which produces sufficiently accurate results
322 (Supplementary Figure 4I-J). The definition of NPD does not imply an efficient nearest
323 neighbor searching algorithm. To increase speed, we first use Euclidean distance-based
324 nearest neighbor searching, which is highly efficient in the low dimensional latent space, and
325 then compute posterior distances only for these nearest neighbors. Empirical distribution of
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

326 posterior NPD for a dataset is obtained by computing posterior NPD on randomly selected
327 pairs of cells in the reference dataset. Empirical p-values of query hits are computed by
328 comparing posterior NPD of a query hit to this empirical distribution.
329 We note that even with the querying strategy described above, querying with single models
330 still occasionally leads to many false positive hits when cell types that the model has not been
331 trained on are provided as query. This is because embeddings of such untrained cell types are
332 mostly random, and they could localize close to reference cells by chance. We reason that
333 embedding randomness of untrained cell types could be utilized to identify and correctly
334 reject them. Practically, we train multiple models with different starting points (as determined
335 by random seeds), and compute query hit significance for each model. A query hit is
336 considered significant only if it is consistently significant across multiple models. To acquire
337 predictions based on significant hits, we use majority voting for discrete variables, e.g. cell
338 type, or averaging for continuous variables, e.g. cell fate distribution.

339 Distance metric ROC analysis

340 Our model and scVI8 were fitted on reference datasets and applied to positive and negative
341 control query datasets in the pancreas group of Supplementary Table 2. We then randomly
342 selected 10,000 query-reference cell pairs. A query-reference pair is defined as “positive” if
343 the query cell and reference cell are of the same cell type, and “negative” otherwise. Each
344 benchmarked similarity metric was then computed on all sampled query-reference pairs, and
345 used as predictors for “positive” / “negative” pairs. AUROC values were computed for each
346 benchmarked similarity metric. In addition to Euclidean distance, we also computed posterior
347 distribution distances for scVI (Supplementary Figure 4K). NPD is computed as described in
348 (18), based on samples from the posterior Gaussian. JSD is computed in the original latent
349 space (without projection).

350 Benchmarking cell querying

351 For each of the four querying groups, three types of datasets were used, namely reference,
352 positive query and negative query (see Supplementary Table 2 for a detailed list of datasets
353 used in each querying group). For each querying method, cell type predictions for query cells
354 are obtained based on reference hits with a minimal similarity cutoff. Cell ontology
355 annotations in ACA were used as ground truth. Cells with no cell ontology annotations were
356 excluded in the analysis. Predictions are considered correct if and only if it exactly matches
357 the ground truth, i.e. no flexibility based on cell type similarity. This prevents unnecessary
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

358 bias introduced in the selection of cell type similarity measure. Cells were inversely weighed
359 by size of the corresponding dataset when computing sensitivity, specificity and Cohen’s κ.
360 AUROC was computed using linear interpolation. For scmap2, we varied minimal cosine
361 similarity requirement for nearest neighbors. For Cell BLAST, we varied maximal p-value
362 cutoff used in filtering hits. For CellFishing.jl4, the original implementation does not include
363 a dedicated cell type prediction function, so we used the same strategy as for our own method
364 (majority voting after distance filtering) to acquire final predictions, in which we varied the
365 Hamming distance cutoff used in distance filtering. Lastly, 4 random seeds were tested for
366 each cutoff and each method to reflect stability. Several other cell querying tools
367 (CellAtlasSearch3, scQuery33, scMCA34) were not included in our benchmark because they
368 do not support custom reference datasets.

369 Benchmarking querying speed

370 To evaluate scalability of querying methods, we constructed reference data of varying sizes
371 by subsampling from the 1M mouse brain dataset35. For query data, the “Marques” dataset36
372 was used. For all methods, only querying time was recorded, not including time consumed to
373 build reference indices.

374 Application to trachea datasets

375 We first removed cells labeled as “ionocytes” in the “Montoro_10x”14 dataset, and used
376 “FindVariableGenes” from Seurat to select informative genes using the remaining cells. Four
377 models with different starting points were trained on the tampered “Montoro_10x” dataset.
378 We computed posterior distribution distance p-values as mentioned before, and used a cutoff
379 of p-value > 0.1 to reject query cells from the “Plasschaert”15 dataset as potential novel cell
380 types. We clustered rejected cells using spectral clustering (Scikit-learn37 v0.20.1) after
381 applying t-SNE38 to latent space coordinates. Average p-value for a query cell was computed
382 as the geometric mean of p-values across all hits.

383 Online tuning

384 When significant batch effect exists between reference and query, we support further aligning
385 query data with reference data in an online-learning manner. All components in the
386 pretrained model, including encoder, decoder, prior discriminators and batch discriminators
387 are retained. The reference-query batch effect is added as an extra component to be removed
388 using adversarial batch alignment. Specifically, a new discriminator dedicated to reference-
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

389 query batch effect is added, and the decoder is expanded to accept an extra one-hot indicator
390 for reference and query. The expanded model is then fine-tuned using the combination of
391 reference and query data. Two precautions are taken to prevent decrease in specificity caused
392 by over-alignment. Firstly, adversarial alignment loss is constrained to cells that have mutual
393 nearest neighbors6 between reference and query data in each SGD minibatch. Secondly, we
394 penalize the deviation of fine-tuned model weights from original weights.

395 Application to hematopoietic progenitor datasets

396 For within- “Tusi”16 query, we trained four models using only cells from sequencing run 2,
397 and cells from sequencing run 1 were used as query cells. PBA inferred cell fate distributions,
398 which are 7-dimensional categorical distributions across 7 terminal cell fates, were used as
399 ground truth. We took the average cell fate distributions of significant querying hits (p-value
400 < 0.05) as predictions for query cells. As for scmap-cell, we filtered nearest neighbors
401 according to default cosine similarity cutoff of 0.5. Jensen-Shannon divergence (JSD)
402 between true and predicted cell fate distribution was computed as below:
1 𝑝‡ 𝑞‡
𝐽𝑆𝐷 (𝒑 ∥ 𝒒) = ⋅ ‰ 𝑝‡ log 𝑝 + 𝑞 + 𝑞‡ log 𝑝 + 𝑞 (19)
2 ‡ ‡ ‡ ‡
‡∈{Æ,…‚,Ç’È,ɨ,&,Ç,$} 2 2
403 For cross-species querying between “Tusi” and “Velten”17, we mapped both mouse and
404 human genes to ortholog groups. Online tuning with 200 epochs was used to increase
405 sensitivity and accuracy. Latent space visualization was performed with UMAP39, 40.

406 ACA database construction

407 We searched Gene Expression Omnibus (GEO)41 using the following search term:
408 (
409 "expression profiling by high throughput sequencing"[DataSet Type] OR
410 "expression profiling by high throughput sequencing"[Filter] OR
411 "high throughput sequencing"[Platform Technology Type]
412 ) AND
413 "gse"[Entry Type] AND
414 (
415 "single cell"[Title] OR
416 "single-cell"[Title]
417 ) AND
418 ("2013"[Publication Date] : "3000"[Publication Date]) AND
419 "supplementary"[Filter]
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

420 Datasets in the Hemberg collection (https://hemberg-lab.github.io/scRNA.seq.datasets/) were


421 merged into this list. Only animal single-cell transcriptomic datasets profiling samples of
422 normal condition were selected. We also manually filtered small scale or low-quality data.
423 Additionally, several other high-quality datasets missing in the previous list were included for
424 comprehensiveness.
425 The expression matrices and metadata of selected datasets were retrieved from GEO,
426 supplementary files of the publication or by directly contacting the authors. Metadata were
427 further manually curated by adding additional description in the paper to acquire the most
428 detailed information of each cell. We unified raw cell type annotation by Cell Ontology42, a
429 structured controlled vocabulary for cell types. Closest Cell Ontology terms were manually
430 assigned based on Cell Ontology description and context of the study.

431 Building reference panels for ACA database

432 Two types of searchable reference panels are built for the ACA database. The first consists of
433 individual datasets with dedicated models trained on each of them, while the second consists
434 of datasets grouped by organ and species, with models trained to align multiple datasets
435 profiling the same species and the same organ.
436 Data preprocessing follows the same procedure as in previous benchmarks. Both cross-
437 dataset batch effect and within-dataset batch effect are manually examined and removed
438 when necessary. For the first type of reference panels, datasets too small (typically < 1,000
439 cells sequenced) are excluded because of insufficient training data. These datasets are still
440 included in the second type of panels, where they are trained jointly with other datasets
441 profiling the same organ in the same species. For each reference panel, four models with
442 different starting points are trained. Latent space visualization, self-projection coverage and
443 accuracy on all reference panels are available on our website.

444 Web interface

445 For conveniently performing and visualizing Cell BLAST analysis, we built a one-stop web
446 interface. The client-side was made from Vue.js, a the single-page application Javascript
447 framework, and D3.js for cell ontology visualization. We used Koa2, a web framework for
448 Node.js, as the server-side. The Cell BLAST web portal with all accessible curated datasets is
449 deployed on Huawei Cloud.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

N.A.
Macosko N.A.

Bach N.A.

Baron_human
Method
PCA
Dataset

ZIFA
Plasschaert
ZINB WaVE
scVI
Cell BLAST
Guo

Adam

Muraro

0.900 0.925 0.950 0.975 1.000


Mean Average Precision

450 Supplementary Fig. 1 Comparing low dimensional space cell type resolution of different dimension
451 reduction methods.
452 Nearest neighbor cell type mean average precision (MAP) is used to evaluate how well biological similarity is
453 captured. MAP can be thought of as a generalization to nearest neighbor accuracy, with larger values indicating
454 higher cell type resolution, thus more suitable for cell querying. Methods that did not finish under 2-hour time
455 limit are marked as N.A.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

456 Supplementary Fig. 2 t-SNE visualization of latent spaces learned by our model.
457 (A) “Muraro”43, (B) “Adam”44, (C) “Guo”45, (D) “Plasschaert”15, (E) “Baron_human”46, (F) “Bach”47, (G)
458 “Macosko”48.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

459 Supplementary Fig. 3 t-SNE visualization of latent spaces learned by our model on combinations of
460 multiple datasets, with batch effect corrected.
461 Figures in the left column color cells by their cell types, while figures in the right column color cells by dataset.
462 (A-B) “Baron_human”46 and “Baron_mouse”46; (C-D) “Baron_human”46, “Muraro”43, “Enge”49,
463 “Segerstolpe”50, “Xin_2016”51 and “Lawlor”52; (E-F) “Montoro_10x”14 and “Plasschaert”15; (G-H)
464 “Quake_Smart-seq2”18 and “Quake_10x”18.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

465 Supplementary Fig. 4 Multi-level batch correction and Cell BLAST strategy optimization.
466 (A-C) Latent space learned only with cross-dataset batch correction, colored by (A) donor in “Baron_human”46,
467 (B) donor in “Enge”49, (C) donor in “Muraro”43, respectively. (D-H) Latent space learned with both cross-
468 dataset batch correction and within-dataset batch correction, colored by (D) donor in “Baron_human”46, (E)
469 donor in “Enge”49, (F) donor in “Muraro”43, (G) cell type, (H) dataset, respectively. (I) Standard deviation
470 decreases as number of samples from the posterior increases. (J) Relation between Euclidean distance and
471 posterior distance on “Baron_human”46 data. Orange points represent cell pairs that are of the same cell type
472 (“positive pairs”), while blue points represent cell pairs of different cell types (“negative pairs”). (K) AUROC of
473 different distance metrics in discriminating cell pairs with the same cell type from cell pairs with different cell
474 types. Note that posterior distribution distances for scVI only lead to decrease in performance, possibly due to
475 improper Gaussian assumption in the posterior. (L) Accuracy, Cohen’s κ, specificity and sensitivity all increase
476 as the number of models used for cell querying increases, among which improvement of specificity is most
477 significant.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

B C D

E F G

478 Supplementary Fig. 5 Ionocytes predicted to be club cells are potentially doublets or of intermediate cell
479 state.
480 (A) Cell-cell correlation heatmap for several cell types of interest. Cells labeled as “<X>” are reference cells in
481 the “Montoro”14 dataset. Cells labeled as “X->Y” are cells annotated as “X” in the original “Plasschaert”15
482 dataset but predicted to be “Y”. (B-D) Expression levels of several club cell markers in cell groups of interest.
483 (E-G) Expression levels of several ionocyte markers in cell groups of interest.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A B Plasschaert to Montoro_10x_no_ionocyte (scmap)

basal cell of epithelium of trachea


basal cell of epithelium of trachea

unassigned
brush cell of trachea
brush cell of trachea
ciliated columnar cell of tracheobronchial tree
ciliated columnar cell of tracheobronchial tree

club cell
club cell

ionocyte
lung neuroendocrine cell lung neuroendocrine cell

C D

484 Supplementary Fig. 6 Rejected cells in the “Montoro” - “Plasschaert” query.


485 (A) t-SNE visualization of Cell BLAST rejected cells, colored by cell type. (B) Sankey plot of scmap prediction.
486 (C, D) t-SNE visualization of scmap rejected cells, colored by unsupervised clustering (C) and cell type (D). (E)
487 scmap similarity distribution in each cluster of scmap rejected cells. The rejected ionocytes do not have the
488 lowest cosine similarity scores to draw enough attention.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A B C
Mammary Gland Mammary Gland Mammary Gland

Pancreas Method Pancreas Method Pancreas Method


Organ

Organ

Organ
scmap scmap scmap
CellFishing.jl CellFishing.jl CellFishing.jl
Cell BLAST Cell BLAST Cell BLAST
Trachea Trachea Trachea

Lung Lung Lung

0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00
Specificity Sensitivity Kappa

D Lung Mammary Gland E


1.00 ●

200

0.75

Time per query (ms)


150
0.50 Method
● scmap

AUC: 0.911, Kappa: 0.958 AUC: 0.943, Kappa: 0.680 100 ● CellFishing.jl
0.25
AUC: 0.933, Kappa: 0.963 AUC: 0.765, Kappa: 0.456 Method ● Cell BLAST
scmap
AUC: 0.930, Kappa: 0.966 AUC: 0.909, Kappa: 0.668
CellFishing.jl
0.00
Sensitivity

50 ●
Cell BLAST
Pancreas Trachea

1.00 Default Cutoff ● ●


● ●
● ● ● ● ● ●
● ● ● ●
FALSE 0 ● ● ● ●

TRUE 1e+03 1e+04 1e+05 1e+06


0.75 Reference size

0.50

AUC: 0.954, Kappa: 0.914 AUC: 0.995, Kappa: 0.972


0.25
AUC: 0.663, Kappa: 0.859 AUC: 0.760, Kappa: 0.966

AUC: 0.967, Kappa: 0.921 AUC: 0.933, Kappa: 0.977

0.00

0.25 0.50 0.75 1.00 0.25 0.50 0.75 1.00


Specificity

489 Supplementary Fig. 7 Benchmarking cell querying.


490 (A-C) Querying specificity (A), sensitivity (B) and Cohen’s κ (C) for different methods under the default
491 setting. (D) ROC curve of cell querying in four different groups of test datasets. Cohen’s κ values in bottom left
492 of each subpanel correspond to the optimal point on ROC curve. Points corresponding to each method’s default
493 cutoff (scmap: cosine distance = 0.5, CellFishing.jl: Hamming distance = 110, Cell BLAST: p-value = 0.05) are
494 marked as triangles. Note that CellFishing.jl does not provide a default cutoff, so we chose Hamming distance
495 of 110 which the closest to balancing sensitivity and specificity, but it’s still far from being stable across
496 different datasets. (E) Querying speed on references of different sizes subsampled from the 1M mouse brain
497 dataset35.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A Pancreas positive control


B Pancreas negative control

endothelial cell kidney pelvis urothelial cell


kidney collecting duct epithelial cell
endothelial cell ureter urothelial cell
dendritic cell
glomerular endothelial cell
pancreatic A cell hematopoietic precursor cell pancreatic ductal cell
pancreatic A cell kidney loop of Henle epithelial cell
fibroblast endothelial cell
vasa recta ascending limb cell
pancreatic D cell vasa recta descending limb cell
mesangial cell T cell
pancreatic D cell glomerular visceral epithelial cell Schwann cell
pancreatic PP cell pancreatic stellate cell
epithelial cell of proximal tubule pancreatic acinar cell
pancreatic PP cell ambiguous ambiguous
fibroblast of dermis
pancreatic epsilon cell natural killer cell
pancreatic acinar cell regulatory T cell
pancreatic acinar cell memory T cell
pancreatic ductal cell T-helper 2 cell rejected
pancreatic endocrine cell cytotoxic T cell
B cell
rejected naive thymus-derived CD4-positive, alpha-beta T cell
pancreatic ductal cell
macrophage naive thymus-derived CD8-positive, alpha-beta T cell
mast cell Schwann cell monocyte
endothelial cell
professional antigen presenting cell mast cell renal alpha-intercalated cell
kidney loop of Henle descending limb epithelial cell
pancreatic stellate cell pancreatic stellate cell renal principal cell
renal beta-intercalated cell pancreatic D cell
kidney resident macrophage
kidney connecting tubule epithelial cell
type B pancreatic cell type B pancreatic cell kidney loop of Henle ascending limb epithelial cell
kidney distal convoluted tubule epithelial cell

C Trachea positive control


D Trachea negative control

retinal rod cell

basal cell of epithelium of trachea epithelial cell of proximal tubule


basal cell of epithelium of trachea

kidney distal convoluted tubule epithelial cell


rejected
luminal epithelial cell of mammary gland
ambiguous myoepithelial cell of mammary gland
retinal bipolar neuron
rejected amacrine cell
mammary alveolar cell
basal cell lung neuroendocrine cell
ciliated columnar cell of tracheobronchial tree
ciliated columnar cell of tracheobronchial tree Muller cell ambiguous
T cell
endothelial cell
kidney loop of Henle epithelial cell
renal intercalated cell
retinal cone cell
B cell
Schwann cell
astrocyte
club cell club cell blood vessel endothelial cell
fibroblast
glomerular visceral epithelial cell
kidney resident macrophage
renal principal cell
leukocyte
macrophage
microglial cell
natural killer cell
neutrophil
ionocyte tracheal goblet cell pancreatic A cell
type B pancreatic cell
brush cell of trachea ionocyte pancreatic D cell
pancreatic ductal cell
pancreatic PP cell
lung neuroendocrine cell brush cell of trachea pancreatic stellate cell
pericyte cell
retina horizontal cell
lung neuroendocrine cell retinal ganglion cell

E Mammary positive control F Mammary negative control

retinal rod cell


basal cell

basal cell
epithelial cell of proximal tubule

luminal epithelial cell of mammary gland kidney distal convoluted tubule epithelial cell rejected
retinal bipolar neuron
amacrine cell
luminal epithelial cell of mammary gland Muller cell
T cell
mammary alveolar cell kidney loop of Henle epithelial cell
retinal cone cell
renal intercalated cell
mammary gland epithelial cell B cell
myoepithelial cell of mammary gland fibroblast
glomerular visceral epithelial cell
kidney resident macrophage
endothelial cell mammary alveolar cell ionocyte
natural killer cell
type B pancreatic cell
ambiguous leukocyte
mammary stem cell macrophage
microglial cell
pancreatic A cell
neutrophil
pancreatic D cell
stromal cell retinal ganglion cell
pancreatic PP cell
retina horizontal cell
rejected endothelial cell
astrocyte
ciliated columnar cell of tracheobronchial tree
T cell club cell
brush cell of trachea
renal principal cell
lung neuroendocrine cell luminal epithelial cell of mammary gland
macrophage pericyte cell
pancreatic stellate cell basal cell
basal cell of epithelium of trachea ambiguous
blood vessel endothelial cell
B cell pancreatic ductal cell
Schwann cell

G Lung positive control H Lung negative control

natural killer cell natural killer cell


T cell T cell
fibroblast non-classical monocyte
lung endothelial cell B cell classical monocyte
neutrophil B cell
macrophage leukocyte
lung endothelial cell kidney resident macrophage mast cell
ciliated columnar cell of tracheobronchial tree myeloid cell
retinal rod cell
ciliated columnar cell of tracheobronchial tree T cell
classical monocyte epithelial cell of proximal tubule
T cell
B cell kidney distal convoluted tubule epithelial cell
classical monocyte rejected
luminal epithelial cell of mammary gland
rejected
B cell retinal bipolar neuron
amacrine cell
leukocyte leukocyte Muller cell
kidney loop of Henle epithelial cell
mast cell retinal cone cell ciliated columnar cell of tracheobronchial tree
Schwann cell
myeloid cell astrocyte
myeloid cell leukocyte ambiguous
microglial cell
natural killer cell pancreatic PP cell
natural killer cell renal principal cell
retina horizontal cell
monocyte alveolar macrophage renal intercalated cell
type B pancreatic cell
pancreatic ductal cell
non-classical monocyte pancreatic A cell
stromal cell glomerular visceral epithelial cell
ambiguous mammary alveolar cell
pancreatic D cell
retinal ganglion cell
pericyte cell
stromal cell pancreatic stellate cell lung endothelial cell
epithelial cell of lung endothelial cell
blood vessel endothelial cell
myoepithelial cell of mammary gland stromal cell
type II pneumocyte basal cell

498 Supplementary Fig. 8 Sankey plots for Cell BLAST in the cell querying benchmark.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A Pancreas positive control


B Pancreas negative control

kidney pelvis urothelial cell


kidney collecting duct epithelial cell
ureter urothelial cell
fibroblast
pancreatic A cell fibroblast of dermis
pancreatic A cell glomerular endothelial cell pancreatic ductal cell
kidney loop of Henle epithelial cell
vasa recta ascending limb cell pancreatic stellate cell
endothelial cell
pancreatic epsilon cell vasa recta descending limb cell
mesangial cell
professional antigen presenting cell endothelial cell glomerular visceral epithelial cell
endothelial cell epithelial cell of proximal tubule type B pancreatic cell
pancreatic D cell pancreatic A cell
pancreatic D cell hematopoietic precursor cell
natural killer cell pancreatic D cell
pancreatic PP cell
pancreatic endocrine cell naive thymus-derived CD8-positive, alpha-beta T cell
unassigned
cytotoxic T cell
pancreatic PP cell
memory T cell unassigned
mast cell pancreatic acinar cell
T-helper 2 cell
naive thymus-derived CD4-positive, alpha-beta T cell
pancreatic acinar cell regulatory T cell
pancreatic ductal cell
B cell pancreatic acinar cell
kidney loop of Henle ascending limb epithelial cell macrophage
pancreatic ductal cell macrophage renal alpha-intercalated cell
mast cell endothelial cell
kidney distal convoluted tubule epithelial cell
pancreatic stellate cell pancreatic stellate cell kidney connecting tubule epithelial cell
renal principal cell
renal beta-intercalated cell
kidney loop of Henle descending limb epithelial cell
type B pancreatic cell type B pancreatic cell kidney resident macrophage
monocyte
dendritic cell

C Trachea positive control


D Trachea negative control

basal cell
myoepithelial cell of mammary gland
Schwann cell basal cell of epithelium of trachea
pancreatic ductal cell
luminal epithelial cell of mammary gland ciliated columnar cell of tracheobronchial tree
basal cell of epithelium of trachea pancreatic stellate cell ionocyte
pancreatic D cell club cell
basal cell of epithelium of trachea lung neuroendocrine cell
mammary alveolar cell tracheal goblet cell
blood vessel endothelial cell
type B pancreatic cell
pancreatic PP cell
retinal rod cell
unassigned
brush cell of trachea astrocyte
brush cell of trachea
epithelial cell of proximal tubule unassigned
ciliated columnar cell of tracheobronchial tree
ciliated columnar cell of tracheobronchial tree pancreatic A cell
endothelial cell
macrophage
Muller cell
pericyte cell
kidney distal convoluted tubule epithelial cell
retinal bipolar neuron
amacrine cell brush cell of trachea
retinal cone cell
club cell club cell glomerular visceral epithelial cell
leukocyte
microglial cell
neutrophil
T cell
natural killer cell
fibroblast
kidney loop of Henle epithelial cell
B cell
kidney resident macrophage
ionocyte ionocyte retinal ganglion cell
renal principal cell
retina horizontal cell
lung neuroendocrine cell lung neuroendocrine cell renal intercalated cell

E Mammary positive control F Mammary negative control

club cell
ciliated columnar cell of tracheobronchial tree
lung neuroendocrine cell
basal cell ionocyte
pancreatic ductal cell
renal intercalated cell
basal cell basal cell of epithelium of trachea
pancreatic A cell
Schwann cell
type B pancreatic cell
pancreatic PP cell
renal principal cell
luminal epithelial cell of mammary gland pancreatic stellate cell luminal epithelial cell of mammary gland
glomerular visceral epithelial cell

luminal epithelial cell of mammary gland epithelial cell of proximal tubule


mammary alveolar cell brush cell of trachea basal cell
kidney distal convoluted tubule epithelial cell
myoepithelial cell of mammary gland pancreatic D cell
stromal cell kidney loop of Henle epithelial cell
pericyte cell
endothelial cell
unassigned
endothelial cell unassigned retinal rod cell
macrophage
retinal ganglion cell
blood vessel endothelial cell
mammary stem cell natural killer cell mammary alveolar cell
B cell
fibroblast
macrophage
mammary alveolar cell neutrophil
astrocyte
T cell amacrine cell
T cell
retinal bipolar neuron myoepithelial cell of mammary gland
mammary gland epithelial cell leukocyte
Muller cell
retina horizontal cell
retinal cone cell
B cell microglial cell
kidney resident macrophage

G Lung positive control H Lung negative control

B cell neutrophil leukocyte


B cell natural killer cell mast cell
T cell myoepithelial cell of mammary gland natural killer cell
basal cell stromal cell
classical monocyte luminal epithelial cell of mammary gland
T cell pancreatic stellate cell
leukocyte T cell T cell
classical monocyte fibroblast non-classical monocyte
B cell classical monocyte
mast cell endothelial cell B cell
leukocyte macrophage
Schwann cell ciliated columnar cell of tracheobronchial tree
pericyte cell
blood vessel endothelial cell lung endothelial cell
leukocyte myeloid cell
renal intercalated cell alveolar macrophage
lung endothelial cell astrocyte
lung endothelial cell pancreatic ductal cell
renal principal cell
pancreatic D cell
pancreatic PP cell
kidney resident macrophage
unassigned mammary alveolar cell
type B pancreatic cell type II pneumocyte
myeloid cell myeloid cell kidney distal convoluted tubule epithelial cell
glomerular visceral epithelial cell
alveolar macrophage kidney loop of Henle epithelial cell
natural killer cell epithelial cell of proximal tubule
natural killer cell
monocyte unassigned
non-classical monocyte pancreatic A cell
ciliated columnar cell of tracheobronchial tree
ciliated columnar cell of tracheobronchial tree retinal rod cell

Muller cell
stromal cell stromal cell microglial cell
retinal bipolar neuron
retinal ganglion cell
amacrine cell
epithelial cell of lung type II pneumocyte retinal cone cell
retina horizontal cell

499 Supplementary Fig. 9 Sankey plots for scmap in the cell querying benchmark.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A Pancreas positive control


B Pancreas negative control

endothelial cell endothelial cell fibroblast


fibroblast of dermis
vasa recta descending limb cell pancreatic stellate cell
glomerular endothelial cell endothelial cell
pancreatic A cell vasa recta ascending limb cell
pancreatic A cell dendritic cell
hematopoietic precursor cell
endothelial cell
renal principal cell
kidney loop of Henle ascending limb epithelial cell
pancreatic epsilon cell rejected renal alpha-intercalated cell
kidney loop of Henle descending limb epithelial cell
pancreatic PP cell kidney resident macrophage
kidney connecting tubule epithelial cell
kidney distal convoluted tubule epithelial cell rejected
pancreatic D cell renal beta-intercalated cell
pancreatic PP cell epithelial cell of proximal tubule
pancreatic endocrine cell mesangial cell pancreatic A cell
ambiguous
mast cell kidney collecting duct epithelial cell pancreatic D cell
pancreatic D cell glomerular visceral epithelial cell
ureter urothelial cell
pancreatic acinar cell mast cell kidney pelvis urothelial cell
natural killer cell
pancreatic acinar cell monocyte macrophage
kidney loop of Henle epithelial cell pancreatic ductal cell
pancreatic ductal cell pancreatic acinar cell
regulatory T cell T cell
pancreatic ductal cell
T-helper 2 cell
ambiguous
macrophage B cell
type B pancreatic cell
cytotoxic T cell
type B pancreatic cell memory T cell type B pancreatic cell
professional antigen presenting cell naive thymus-derived CD4-positive, alpha-beta T cell
pancreatic stellate cell pancreatic stellate cell naive thymus-derived CD8-positive, alpha-beta T cell

C Trachea positive control


D Trachea negative control

ciliated columnar cell of tracheobronchial tree


ciliated columnar cell of tracheobronchial tree retinal rod cell

brush cell of trachea club cell


epithelial cell of proximal tubule
brush cell of trachea
kidney distal convoluted tubule epithelial cell
pancreatic PP cell
club cell retina horizontal cell
leukocyte
type B pancreatic cell
retinal bipolar neuron
club cell retinal cone cell rejected
pancreatic D cell
kidney loop of Henle epithelial cell
macrophage
ambiguous amacrine cell
tracheal goblet cell retinal ganglion cell
astrocyte
pancreatic ductal cell tracheal goblet cell
neutrophil ionocyte
luminal epithelial cell of mammary gland brush cell of trachea
lung neuroendocrine cell
myoepithelial cell of mammary gland ambiguous
ciliated columnar cell of tracheobronchial tree
renal principal cell
renal intercalated cell
basal cell of epithelium of trachea rejected Muller cell
endothelial cell
fibroblast
pancreatic stellate cell
Schwann cell
glomerular visceral epithelial cell
microglial cell
T cell
kidney resident macrophage
pancreatic A cell
basal cell of epithelium of trachea natural killer cell
ionocyte B cell
basal cell basal cell of epithelium of trachea
ionocyte pericyte cell
mammary alveolar cell
lung neuroendocrine cell lung neuroendocrine cell blood vessel endothelial cell

E Mammary positive control F Mammary negative control

retinal rod cell


basal cell

basal cell
epithelial cell of proximal tubule

retinal bipolar neuron


luminal epithelial cell of mammary gland
amacrine cell rejected
luminal epithelial cell of mammary gland basal cell of epithelium of trachea
retinal cone cell
brush cell of trachea
ionocyte
mammary alveolar cell leukocyte
lung neuroendocrine cell
T cell
renal intercalated cell myoepithelial cell of mammary gland
glomerular visceral epithelial cell ambiguous
astrocyte
T cell natural killer cell mammary alveolar cell
neutrophil
rejected B cell
retina horizontal cell
club cell
B cell ciliated columnar cell of tracheobronchial tree
pancreatic A cell
Muller cell
kidney loop of Henle epithelial cell
mammary gland epithelial cell renal principal cell luminal epithelial cell of mammary gland
retinal ganglion cell
pancreatic ductal cell
mammary stem cell mammary alveolar cell kidney distal convoluted tubule epithelial cell
pancreatic PP cell
type B pancreatic cell
macrophage pancreatic D cell
macrophage
ambiguous fibroblast
endothelial cell
stromal cell Schwann cell basal cell
blood vessel endothelial cell
myoepithelial cell of mammary gland kidney resident macrophage
microglial cell
endothelial cell pancreatic stellate cell
pericyte cell

G Lung positive control H Lung negative control

B cell T cell T cell


B cell neutrophil mast cell
natural killer cell leukocyte
ciliated columnar cell of tracheobronchial tree fibroblast non-classical monocyte
ciliated columnar cell of tracheobronchial tree microglial cell classical monocyte
T cell kidney resident macrophage
B cell myeloid cell
endothelial cell natural killer cell
T cell blood vessel endothelial cell B cell
classical monocyte macrophage lung endothelial cell
mammary alveolar cell
classical monocyte leukocyte leukocyte alveolar macrophage
ambiguous
leukocyte mast cell retinal rod cell

luminal epithelial cell of mammary gland


lung endothelial cell
lung endothelial cell
epithelial cell of proximal tubule
rejected
rejected retinal bipolar neuron
amacrine cell
Schwann cell
myeloid cell myeloid cell glomerular visceral epithelial cell
retinal cone cell
ambiguous basal cell ciliated columnar cell of tracheobronchial tree
renal intercalated cell
alveolar macrophage astrocyte stromal cell
natural killer cell retinal ganglion cell
pancreatic stellate cell
natural killer cell Muller cell
monocyte kidney distal convoluted tubule epithelial cell
non-classical monocyte myoepithelial cell of mammary gland type II pneumocyte
pancreatic A cell
kidney loop of Henle epithelial cell
stromal cell stromal cell renal principal cell
retina horizontal cell
pancreatic ductal cell
pericyte cell
pancreatic PP cell
epithelial cell of lung type II pneumocyte pancreatic D cell
type B pancreatic cell

500 Supplementary Fig. 10 Sankey plots for CellFishing.jl in the cell querying benchmark.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A B C

D E F

G H
1.00

Default Cutoff
0.75 FALSE
TRUE
Sensitivity

0.50 Method
scmap
CellFishing.jl
AUC: 0.961, Kappa: 0.888
Cell BLAST
0.25 AUC: 0.643, Kappa: 0.770
Cell BLAST aligned
AUC: 0.891, Kappa: 0.843
AUC: 0.642, Kappa: 0.856
0.00

0.25 0.50 0.75 1.00


Specificity

501 Supplementary Fig. 11 Using “online tuning” in hematopoietic progenitor and Tabula Muris18 spleen
502 data.
503 UMAP visualization of the “Velten”17 dataset, colored by expression of lineage markers, including CA1 for
504 erythrocyte lineage (A), GP1BB for megakaryocyte lineage (B), DNTT for B cell lineage, TGFBI for monocyte,
505 dendritic cell lineage (D), CLC for eosinophil, basophil, mast cell lineage (E), MPO for neutrophil lineage (F),
506 and scmap predicted cell fate distribution (G). (H) ROC curve of cell querying in Tabula Muris18 spleen data.
507 Cohen’s κ values in bottom left of each subpanel correspond to the optimal point on ROC curve. Points
508 corresponding to each method’s default cutoff (scmap: cosine distance = 0.5, CellFishing.jl: Hamming distance
509 = 110, Cell BLAST: p-value = 0.05) are marked as triangles.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

A B

ACA ScarTrace (2.3%) sci−RNA−seq (1.15%)

SMARTer (2.3%) CEL−Seq2 (1.15%)


QUARTZ−seq (1.15%)
Single Cell Portal STRT/C1 (3.45%)

Database
inDrop (4.6%)
Smart−Seq2 (35.63%)

Hemberg Drop−seq (13.79%)

SCPortalen

scRNASeqDB
10x Genomics (34.48%)

0.0 2.5 5.0 7.5 10.0


Cell number (x10e5)

C D

510
511 Supplementary Fig. 12 ACA database and Cell BLAST Web portal.
512 (A) Comparison of cell number in different single-cell transcriptomics databases. (B) Composition of different
513 single-cell sequencing platforms in ACA. (C) Home page of the Web interface. (D) Web interface showing
514 results of an example query.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

Experimental
Dataset Name Organism Organ Profiled Used In
Platform
Guo45 Testis 10x53 DR

Muraro43 CEL-Seq254 DR, BC

Xin_201651 SMARTer55 BC

Lawlor52 Human SMARTer55 BC

Segerstolpe50 Pancreas Smart-seq256 BC

Enge49 Smart-seq256 BC

Baron_human46 inDrop57 DR, BC

Baron_mouse46 inDrop57 BC

Adam44 Kidney Drop-seq48 DR

Plasschaert15 inDrop57 DR, BC


Trachea
Montoro_10x14 10x53 BC
Mouse
Macosko48 Retina Drop-seq48 DR
Mammary
Bach47 10x53 DR
Gland
Quake_Smart-
20 Organs Smart-seq256 BC
seq218
Quake_10x18 12 Organs 10x53 BC
515
516 Supplementary Table. 1 Datasets used in dimensionality reduction and batch effect
517 correction benchmarking.
518 DR, dimension reduction benchmarking; BC, batch effect correction benchmarking
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

Organ Experimental
Group Role Dataset Name Organism
Profiled Platform
Baron_human46 inDrop57

Reference Xin_201651 SMARTer55

Lawlor52 SMARTer55
Pancreas
43
Muraro CEL-Seq254
Positive
Pancreas control Segerstolpe50 Human Smart-seq256
query
Enge49 Smart-seq256

Wu_human58 Kidney 10x53


Negative
control Zheng53 PBMC 10x53
query
Philippeos59 Skin Smart-seq256

Reference Montoro_10x14 10x53


Positive Trachea
15
control Plasschaert inDrop57
query
Baron_mouse46 Pancreas inDrop57
Trachea Mouse
Negative Park60 Kidney 10x53
control
Mammary
query Bach47 10x53
Gland
Macosko48 Retina Drop-seq48

Reference Bach47 10x53

Giraddi_10x61 10x53
Mammary
Positive Quake_Smart-
Gland
control seq2_ Smart-seq256
query Mammary_Gland18
Mammary
Quake_10x_ Mouse
Gland 10x53
Mammary_Gland18
Baron_mouse46 Pancreas inDrop57
Negative
control Park60 Kidney 10x53
query
Plasschaert15 Trachea inDrop57
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

Macosko48 Retina Drop-seq48


Quake_10x_
Reference 10x53
Lung18
Positive Quake-Smart- Lung
control seq2_ Smart-seq256
query Lung18
Baron_mouse46 Pancreas inDrop57
Lung Mouse
Negative Park60 Kidney 10x53
control
Mammary
query Bach47 10x53
Gland
Plasschaert15 Trachea inDrop57
Quake_10x_
Reference 10x53
Spleen18
Positive Quake_Smart- Spleen
control seq2_ Smart-seq256
query Spleen18
Baron_mouse46 Pancreas inDrop57
Spleen Mouse
Negative Park60 Kidney 10x53
control
query Macosko48 Retina Drop-seq48
Mammary
Bach47 10x53
Gland
519
520 Supplementary Table. 2 Datasets used in cell query benchmarking.
521
522 Supplementary Table. 3 Raw data of benchmarking results.
523
524 Supplementary Table. 4 Datasets in ACA.
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

525 Main text references

526 1. Altschul, S.F., Gish, W., Miller, W., Myers, E.W. & Lipman, D.J. Basic local
527 alignment search tool. J Mol Biol 215, 403-410 (1990).
528 2. Kiselev, V.Y., Yiu, A. & Hemberg, M. scmap: projection of single-cell RNA-seq data
529 across data sets. Nat Methods 15, 359-362 (2018).
530 3. Srivastava, D., Iyer, A., Kumar, V. & Sengupta, D. CellAtlasSearch: a scalable search
531 engine for single cells. Nucleic Acids Res 46, W141-W147 (2018).
532 4. Sato, K., Tsuyuzaki, K., Shimizu, K. & Nikaido, I. CellFishing.jl: an ultrafast and
533 scalable cell search method for single-cell RNA sequencing. Genome Biol 20, 31
534 (2019).
535 5. Tung, P.Y. et al. Batch effects and the effective design of single-cell gene expression
536 studies. Sci Rep 7, 39921 (2017).
537 6. Haghverdi, L., Lun, A.T.L., Morgan, M.D. & Marioni, J.C. Batch effects in single-
538 cell RNA-sequencing data are corrected by matching mutual nearest neighbors. Nat
539 Biotechnol 36, 421-427 (2018).
540 7. Butler, A., Hoffman, P., Smibert, P., Papalexi, E. & Satija, R. Integrating single-cell
541 transcriptomic data across different conditions, technologies, and species. Nat
542 Biotechnol 36, 411-420 (2018).
543 8. Lopez, R., Regier, J., Cole, M.B., Jordan, M.I. & Yosef, N. Deep generative modeling
544 for single-cell transcriptomics. Nat Methods 15, 1053-1058 (2018).
545 9. Ding, J., Condon, A. & Shah, S.P. Interpretable dimensionality reduction of single
546 cell transcriptome data with deep generative models. Nat Commun 9, 2002 (2018).
547 10. Grønbech, C.H. et al. scVAE: Variational auto-encoders for single-cell gene
548 expression data. bioRxiv preprint, 318295 (2019).
549 11. Wang, D. & Gu, J. VASC: Dimension Reduction and Visualization of Single-cell
550 RNA-seq Data by Deep Variational Autoencoder. Genomics, proteomics
551 bioinformatics (2018).
552 12. Pierson, E. & Yau, C. ZIFA: Dimensionality reduction for zero-inflated single-cell
553 gene expression analysis. Genome Biol 16, 241 (2015).
554 13. Risso, D., Perraudeau, F., Gribkova, S., Dudoit, S. & Vert, J.P. A general and flexible
555 method for signal extraction from single-cell RNA-seq data. Nat Commun 9, 284
556 (2018).
557 14. Montoro, D.T. et al. A revised airway epithelial hierarchy includes CFTR-expressing
558 ionocytes. Nature 560, 319-324 (2018).
559 15. Plasschaert, L.W. et al. A single-cell atlas of the airway epithelium reveals the CFTR-
560 rich pulmonary ionocyte. Nature 560, 377-381 (2018).
561 16. Tusi, B.K. et al. Population snapshots predict early haematopoietic and erythroid
562 hierarchies. Nature 555, 54-60 (2018).
563 17. Velten, L. et al. Human haematopoietic stem cell lineage commitment is a continuous
564 process. Nat Cell Biol 19, 271-281 (2017).
565 18. Tabula Muris, C. et al. Single-cell transcriptomics of 20 mouse organs creates a
566 Tabula Muris. Nature 562, 367-372 (2018).
567 19. Abugessaisa, I. et al. SCPortalen: human and mouse single-cell centric database.
568 Nucleic Acids Res 46, D781-D787 (2018).
569 20. Cao, Y., Zhu, J., Jia, P. & Zhao, Z. scRNASeqDB: A Database for RNA-Seq Based
570 Gene Expression Profiles in Human Single Cells. Genes (Basel) 8 (2017).
571 Supplementary references
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

572 21. Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I. & Frey, B. Adversarial
573 autoencoders. arXiv preprint (2015).
574 22. Love, M.I., Huber, W. & Anders, S. Moderated estimation of fold change and
575 dispersion for RNA-seq data with DESeq2. Genome Biol 15, 550 (2014).
576 23. Abadi, M. et al. in 12th USENIX Symposium on Operating Systems Design and
577 Implementation 265-283 (2016).
578 24. Ganin, Y. & Lempitsky, V. Unsupervised domain adaptation by backpropagation.
579 arXiv preprint (2014).
580 25. Xie, Q., Dai, Z., Du, Y., Hovy, E. & Neubig, G. in Advances in Neural Information
581 Processing Systems 585-596 (2017).
582 26. Tzeng, E., Hoffman, J., Saenko, K. & Darrell, T. in Proceedings of the IEEE
583 Conference on Computer Vision and Pattern Recognition 7167-7176 (2017).
584 27. Goodfellow, I. et al. in Advances in neural information processing systems 2672-2680
585 (2014).
586 28. Tange, O. Gnu parallel 2018. (2018).
587 29. Baglama, J., Reichel, L. & Lewis, B.J.R.p.v. irlba: Fast truncated singular value
588 decomposition and principal components analysis for large dense and sparse matrices.
589 2 (2017).
590 30. Paszke, A. et al. Automatic differentiation in pytorch. (2017).
591 31. Herrero, J. et al. Ensembl comparative genomics resources. Database (Oxford) 2016
592 (2016).
593 32. Lun, A.T., McCarthy, D.J. & Marioni, J.C. A step-by-step workflow for low-level
594 analysis of single-cell RNA-seq data with Bioconductor. F1000Res 5, 2122 (2016).
595 33. Alavi, A., Ruffalo, M., Parvangada, A., Huang, Z. & Bar-Joseph, Z. A web server for
596 comparative analysis of single-cell RNA-seq data. Nat Commun 9, 4768 (2018).
597 34. Han, X. et al. Mapping the Mouse Cell Atlas by Microwell-Seq. Cell 172, 1091-1107
598 e1017 (2018).
599 35. 10x-Genomics in 1.3 Million Brain Cells from E18 Mice (2017).
600 36. Marques, S. et al. Oligodendrocyte heterogeneity in the mouse juvenile and adult
601 central nervous system. Science 352, 1326-1329 (2016).
602 37. Pedregosa, F. et al. Scikit-learn: Machine learning in Python. 12, 2825-2830 (2011).
603 38. Maaten, L.v.d. & Hinton, G. Visualizing data using t-SNE. Journal of machine
604 learning research 9, 2579-2605 (2008).
605 39. McInnes, L., Healy, J. & Melville, J. Umap: Uniform manifold approximation and
606 projection for dimension reduction. arXiv preprint (2018).
607 40. Becht, E. et al. Dimensionality reduction for visualizing single-cell data using UMAP.
608 Nat Biotechnol (2018).
609 41. Barrett, T. et al. NCBI GEO: archive for functional genomics data sets--update.
610 Nucleic Acids Res 41, D991-995 (2013).
611 42. Diehl, A.D. et al. The Cell Ontology 2016: enhanced content, modularization, and
612 ontology interoperability. J Biomed Semantics 7, 44 (2016).
613 43. Muraro, M.J. et al. A Single-Cell Transcriptome Atlas of the Human Pancreas. Cell
614 Syst 3, 385-394 e383 (2016).
615 44. Adam, M., Potter, A.S. & Potter, S.S. Psychrophilic proteases dramatically reduce
616 single-cell RNA-seq artifacts: a molecular atlas of kidney development. Development
617 144, 3625-3632 (2017).
618 45. Guo, J. et al. The adult human testis transcriptional cell atlas. Cell Res 28, 1141-1157
619 (2018).
bioRxiv preprint doi: https://doi.org/10.1101/587360. The copyright holder for this preprint (which was not peer-reviewed) is the
author/funder. It is made available under a CC-BY-NC-ND 4.0 International license.

620 46. Baron, M. et al. A Single-Cell Transcriptomic Map of the Human and Mouse
621 Pancreas Reveals Inter- and Intra-cell Population Structure. Cell Syst 3, 346-360 e344
622 (2016).
623 47. Bach, K. et al. Differentiation dynamics of mammary epithelial cells revealed by
624 single-cell RNA sequencing. Nat Commun 8, 2128 (2017).
625 48. Macosko, E.Z. et al. Highly Parallel Genome-wide Expression Profiling of Individual
626 Cells Using Nanoliter Droplets. Cell 161, 1202-1214 (2015).
627 49. Enge, M. et al. Single-Cell Analysis of Human Pancreas Reveals Transcriptional
628 Signatures of Aging and Somatic Mutation Patterns. Cell 171, 321-330 e314 (2017).
629 50. Segerstolpe, A. et al. Single-Cell Transcriptome Profiling of Human Pancreatic Islets
630 in Health and Type 2 Diabetes. Cell Metab 24, 593-607 (2016).
631 51. Xin, Y. et al. RNA Sequencing of Single Human Islet Cells Reveals Type 2 Diabetes
632 Genes. Cell Metab 24, 608-615 (2016).
633 52. Lawlor, N. et al. Single-cell transcriptomes identify human islet cell signatures and
634 reveal cell-type-specific expression changes in type 2 diabetes. Genome Res 27, 208-
635 222 (2017).
636 53. Zheng, G.X. et al. Massively parallel digital transcriptional profiling of single cells.
637 Nat Commun 8, 14049 (2017).
638 54. Hashimshony, T. et al. CEL-Seq2: sensitive highly-multiplexed single-cell RNA-Seq.
639 Genome Biol 17, 77 (2016).
640 55. Verboom, K. et al. SMARTer single cell total RNA sequencing. bioRxiv preprint,
641 430090 (2018).
642 56. Picelli, S. et al. Full-length RNA-seq from single cells using Smart-seq2. Nat Protoc
643 9, 171-181 (2014).
644 57. Klein, A.M. et al. Droplet barcoding for single-cell transcriptomics applied to
645 embryonic stem cells. Cell 161, 1187-1201 (2015).
646 58. Wu, H. et al. Comparative Analysis and Refinement of Human PSC-Derived Kidney
647 Organoid Differentiation with Single-Cell Transcriptomics. Cell Stem Cell 23, 869-
648 881 e868 (2018).
649 59. Philippeos, C. et al. Spatial and Single-Cell Transcriptional Profiling Identifies
650 Functionally Distinct Human Dermal Fibroblast Subpopulations. J Invest Dermatol
651 138, 811-825 (2018).
652 60. Park, J. et al. Single-cell transcriptomics of the mouse kidney reveals potential
653 cellular targets of kidney disease. Science 360, 758-763 (2018).
654 61. Giraddi, R.R. et al. Single-Cell Transcriptomes Distinguish Stem Cell State Changes
655 and Lineage Specification Programs in Early Mammary Gland Development. Cell
656 Rep 24, 1653-1666 e1657 (2018).
657

You might also like