VOLUME 40, NUMBER 5 SEPTEMBER/OCTOBER 2020

Machine Learning for Systems

Mini-Theme: Biology and Systems Interactions

www.computer.org/micro
IEEE
Computer
Society Has
You Covered!
WORLD-CLASS CONFERENCES — Stay
ahead of the curve by attending one of our
200+ globally recognized conferences.

DIGITAL LIBRARY — Easily access over 780k

articles covering world-class peer-reviewed
content in the IEEE Computer Society
Digital Library.

CALLS FOR PAPERS — Discover

opportunities to write and present your
ground-breaking accomplishments.

EDUCATION — Strengthen your resume

with the IEEE Computer Society Course
Catalog and its range of offerings.

ADVANCE YOUR CAREER — Search the

new positions posted in the IEEE Computer
Society Jobs Board.

NETWORK — Make connections that count

by participating in local Region, Section,
and Chapter activities.

Explore all of the member benefits

at www.computer.org today!
EDITOR-IN-CHIEF IEEE MICRO STAFF
Lizy K. John, University of Texas, Austin Journals Production Manager: Joanna Gojlik,
j.gojlik@ieee.org
EDITORIAL BOARD Peer-Review Administrator:
micro-ma@computer.org
R. Iris Bahar, Brown University
Publications Portfolio Manager: Kimberly Sperka
Mauricio Breternitz, University of Lisbon
Publisher: Robin Baldwin
Christopher Batten, Cornell University
Senior Advertising Coordinator: Debbie Sims
Bronis de Supinski, Lawrence Livermore
IEEE Computer Society Executive Director:
National Lab
Melissa Russell
Yasuko Eckert, AMD Research
Natalie Enright Jerger, University of Toronto IEEE PUBLISHING OPERATIONS
Maya Gokhale, Lawrence Livermore National
Laboratory Senior Director, Publishing Operations:
Shane Greenstein, Harvard Business School Dawn Melley
Hyesoon Kim, Georgia Institute of Technology Director, Editorial Services: Kevin Lisankie
John Kim, Korea Advanced Institute of Science Director, Production Services: Peter M. Tuohy
and Technology Associate Director, Editorial Services:
Hsien-Hsin (Sean) Lee, Taiwan Semiconductor Jeffrey E. Cichocki
Manufacturing Company Associate Director, Information Conversion
Richard Mateosian and Editorial Support: Neelam Khinvasara
Tulika Mitra, National University of Singapore Senior Art Director: Janet Dudar
Trevor Mudge, University of Michigan, Ann Arbor Senior Manager, Journals Production:
Vijaykrishnan Narayanan, The Pennsylvania Patrick Kempf
State University CS MAGAZINE OPERATIONS COMMITTEE
Richard H. Stern, George Washington
University Law School Sumi Helal (Chair), Irena Bojanova,
Sreenivas Subramoney, Intel Corporation Jim X. Chen, Shu-Ching Chen,
Carole-Jean Wu, Arizona State University Gerardo Con Diaz, David Alan Grier,
Lixin Zhang, Chinese Academy of Sciences Lizy K. John, Marc Langheinrich, Torsten Möller,
David Nicol, Ipek Ozkaya, George Pallis,
ADVISORY BOARD VS Subrahmanian
David H. Albonesi, Erik R. Altman, Pradip Bose, CS PUBLICATIONS BOARD
Kemal Ebcioglu, Lieven Eeckhout,
Fabrizio Lombardi (VP for Publications),
Michael Flynn, Ruby B. Lee, Yale Patt,
Alfredo Benso, Cristiana Bolchini,
James E. Smith, Marc Tremblay
Javier Bruguera, Carl K. Chang, Fred Douglis,
Subscription change of address: Sumi Helal, Shi-Min Hu, Sy-Yen Kuo,
address.change@ieee.org Avi Mendelson, Stefano Zanero, Daniel Zeng
Missing or damaged copies:
COMPUTER SOCIETY OFFICE
help@computer.org
IEEE MICRO
c/o IEEE Computer Society
10662 Los Vaqueros Circle
Los Alamitos, CA 90720 USA +1 (714) 821-8380

IEEE Micro (ISSN 0272-1732) is published bimonthly by the IEEE Computer Society. IEEE Headquarters, Three Park Ave., 17th Floor, New York,
NY 10016-5997; IEEE Computer Society Headquarters, 2001 L St., Ste. 700, Washington, DC 20036; IEEE Computer Society Publications Office,
10662 Los Vaqueros Circle, PO Box 3014, Los Alamitos, CA 90720. Postmaster: Send address changes and undelivered copies to IEEE, Member-
ship Processing Dept., 445 Hoes Ln., Piscataway, NJ 08855. Periodicals postage is paid at New York, NY, and at additional mailing offices. Canadian
GST #125634188. Canada Post Corp. (Canadian distribution) Publications Mail Agreement #40013885. Return undeliverable Canadian addresses
to 4960-2 Walker Road; Windsor, ON N9A 6J3. Printed in USA. Reuse rights and reprint permissions: Educational or personal use of this material is
permitted without fee, provided such use: 1) is not made for profit; 2) includes this notice and a full citation to the original work on the first page of the
copy; and 3) does not imply IEEE endorsement of any third-party products or services. Author and their companies are permitted to post the accepted
version of IEEE-copyrighted material on their own webservers without permission, provided that the IEEE copyright notice and a full citation to the
original work appear on the first screen of the posted copy. An accepted manuscript is a version which has been revised by the author to incorporate
review suggestions, but not the published version with copy-editing, proofreading, and formatting added by IEEE. For more information, please go to
ieee.org/publications_standards/publications/rights/paperversionpolicy.html. Permission to reprint/republish this material for commercial, advertising,
or promotional purposes or for creating new collective works for resale or redistribution must be obtained from IEEE by writing to the IEEE Intellectual
Property Rights Office, 445 Hoes Lane, Piscataway, NJ 08854-4141 or pubs-permissions@ieee.org. ©2020 by IEEE. All rights reserved. Abstracting
and library use: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy for private use of patrons, provided the per-copy
fee indicated in the code at the bottom of the first page is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.
Editorial: Unless otherwise stated, bylined articles, as well as product and service descriptions, reflect the author’s or firm’s opinion. Inclusion in IEEE
Micro does not necessarily constitute an endorsement by IEEE or the Computer Society. All submissions are subject to editing for style, clarity, and
space. IEEE prohibits discrimination, harassment, and bullying. For more information, visit ieee.org/web/aboutus/whatis/policies/p9-26.html.
 September/October 2020
Volume 40 Number 5

6 M Guest Editors’ Introduction

achine Learning for
Systems
46 E nhancing Model
Parallelism in Neural
Architecture Search for
Heiner Litz and Milad Hashemi Multidevice System
Cheng Fu, Huili Chen, Zhenheng
Theme Articles Yang, Farinaz Koushanfar,
Yuandong Tian, and Jishen Zhao

8 A
Published by the IEEE Computer
Society Taxonomy of ML for

56 T
Systems Problems SA-NoC: Learning-
Martin Maas
Based Threat
Detection and

17 A Programmable
Approach to Neural
Mitigation for Secure
Network-on-Chip
Network Compression Architecture
Vinu Joseph, Ganesh L. Ke Wang, Hao Zheng, and
Gopalakrishnan, Saurav Ahmed Louri
Muralidharan, Michael Garland,
and Animesh Garg

64 B Guest Editor’s Introduction

26 A
iology and Systems
Single-Shot
Interactions
Generalized Device
Abhishek Bhattacharjee
Placement for Large
Dataflow Graphs
Yanqi Zhou, Sudip Roy, Amirali Theme Articles
65 A
Abdolrashidi, Daniel Lin-Kit Wong, ccelerating Genome
Peter Ma, Qiumin Xu, Azalia Analysis: A Primer
Mirhoseini, and James Laudon
on an Ongoing Journey

37 R
Mohammed Alser, Z€ ulal Bing€ol,
ELEQ:
A Damla Senol Cali, Jeremie Kim,
Reinforcement Saugata Ghose, Can Alkan, and
Learning Approach Onur Mutlu
for Automatic Deep
Quantization of Neural
Networks 76 P urpleDrop: A Digital
Microfluidics-Based
Ahmed T. Elthakeb, Prannoy Platform for Hybrid
Pilligundla, Fatemehsadat Molecular-Electronics
Mireshghallah, Amir Yazdanbakhsh, Applications
and Hadi Esmaeilzadeh
Ashley Stephenson, Max Willsey, Jeff
McBride, Sharon Newman, Bichlien
Nguyen, Christopher Takahashi,
Karin Strauss, and Luis Ceze
 Image credit: shutterstock.com/whiteMocca.

COLUMNS AND DEPARTMENTS

From the Editor-in-Chief Micro Economics
4 Machine Learning 88 Triggers, Transmissions,
for Systems, Biological and Adjustments
Computing, and More Shane Greenstein
Lizy Kurian John
 From the Editor-in-Chief

Machine Learning for Systems,

Biological Computing, and More
Lizy Kurian John
The University of Texas at Austin

& ARTIFICIAL INTELLIGENCE (AI) and machine for automatic neural network compression, parti-
learning (ML) are influencing everything on the tioning of large DNNs using ML, automatic deep
planet. With the capability of modern com- quantization of DNNs using ML, automatic optimi-
puters, it has become feasible to employ ML for zation of deep neural networks, and learning-based
many everyday applications. Deep neural net- threat detection. Enjoy these six papers and hope-
works are demonstrated to surpass human capa- fully they help you to incorporate such techniques
bility for image detection and recognition into your research and/or products.
applications. Can ML augment human ability to Litz and Hashemi present a guest editorial
design efficient computer systems? Can ML be giving a more detailed introduction to the six
used to design the very same systems that are articles on this topic. Use it to get an overview to
running the ML applications? guide your reading.
IEEE Micro presents six articles on how the In addition to ML for systems theme, this issue
computer architecture community has explored also presents a minitheme: biological computing.
the use of ML models to improve and optimize the The efficiency in many processes in nature has
computing systems that we build. been admirable. It is not uncom-
ML for system design is a recent Can ML augment mon to try to achieve the same in
research direction. Prof. Heiner Litz human ability to design man-made systems. Computing
of the University of California Santa efficient computer occurring in the human brain has
Cruz and Dr. Milad Hashemi of Goo- systems? Can ML be admirably higher energy effi-
gle guest edited the ML for systems used to design the very ciency. Two interesting questions
theme. Prof. Litz and Dr. Hashemi same systems that are are: Can computing help biology?
spent a signficant amount of time running the ML Can biology help computing? Dr.
aquiring submissions and reviewing applications?
Abhishek Bhattacharjee of Yale
them. At the end, they chose six University presents a minitheme
papers that cover topics such as a on biological computing. Articles were solicited on
taxonomy for ML for system problems, use of ML topics that lie at the intersection of biology and
computing. After a peer-review process, two
articles were selected for this minitheme. The first
Digital Object Identifier 10.1109/MM.2020.3017880 article is on the possibilities offered by accelera-
Date of current version 1 September 2020. tors to advance life sciences. It presents genome

4
0272-1732 ß 2020 IEEE Published by the IEEE Computer Society IEEE Micro
analysis algorithms and opportunities that modern
hardware offers to enhance them. The second arti-
cle in this theme is on DNA storage by using a
microfluidic platform. DNA storage is certainly
unconventional computing. Prof. Bhattacharjee
has written an introduction to the minitheme.
Read his introduction and the two articles in this
theme.
I take this opportunity to thank all the three
guest editors for their efforts in getting many
submissions and in getting them reviewed. It is
the effort of such dedicated guest editors and
authors that result in the excellent articles that
we present to you.
The aforementioned eight articles describe
some interesting developments that are happen-
ing in computer system design. There are many
more interesting developments happening
around us. A field known as neurosymbolic AI
is emerging. The “neuro” part of the term
“neurosymbolic” refers to deep learning neural
networks that have experienced unprecedented
growth in the last ten years and the “symbolic”
part of the term refers to the early mainstream
approach to create AI using logical relations.
Essentially, the development is to combine the
two branches of AI to help each other. Symbolic
AI can inject common sense reasoning and
domain knowledge into deep learning, helping
large neural networks to train faster with less
data. Neural networks can make symbolic AI sys-
tems smarter by breaking the world into symbols.
There is ongoing work at MIT, Google, etc.,
around this topic, which is also referred to by
terms such as neural logic machines and logical
neural networks.1,2,3
In addition to the top pick articles, this issue
also features a Micro Economics column by
Shane Greenstein of Harvard Business School,
titled “Triggers, Transmissions, and Adjust-
ments,” describing the impact of COVID on the
economy and comparing it to the two previous
economic busts, the dot com bust of 2000, and
the financial meltdown of 2008. He observes that
every crisis leads to many adjustments and
notes the current redirection toward remote
work and telemedicine that is happening now.
Many companies are extending remote work to
September/October 2020
their employees until the end of the year. Some
of the regulatory hurdles toward telemedicine
disappeared in the light of the pandemic. But as
the need for both these suddenly arose with
COVID, it was interesting to note that hardware
and software advances to enable these were in
good shape. He likes to give credit to all the
investment that had occurred in the hardware
and software advances that made remote work
and telemedicine possible.
Hope AI for systems and biological comput-
ing enables you to design more awesome sys-
tems that help humanity to get through the next
global challenge. I hope that this special issue
takes our readers to unconventional and new
areas. I hope you architect new and unconven-
tional chips, using established and unconven-
tional methodologies. Perhaps AI may assist you
in ways you were not thinking of earlier. Let us
embark on the opportunities presented to us by
the new and emerging technologies, whether
they are biological or artificial.
Best wishes dealing with your personal and
professional challenges as the new school year
starts in the new online hybrid or in-person
environment.
Until next time.
& REFERENCES
1. J. Mao, C. Gan, P. Kohli, and J. B. Tenenbaum, and
J. Wu, “The neuro-symbolic concept learner: Interpreting
scenes, words, and sentences from natural supervision,”
in Proc. Int. Conf. Learn. Representations, 2019.
[Online]. Available at: https://arxiv.org/abs/1904.12584
2. H. Dong, J. Mao, T. Lin, C. Wang, L. Li, and D. Zhou,
“Neural logic machines,” in Proc. Int. Conf. Learn.
Representations, 2019. [Online]. Available at: https://
arxiv.org/abs/1904.11694
3. R. Riegel et al., “Logical neural networks – IBM,”
Jun. 2020. [Online]. Available: https://arxiv.org/pdf/
2006.13155.pdf
Lizy Kurian John is a Cullen Trust for Higher Educa-
tion Endowed Professor with the Electrical and Com-
puter Engineering Department, The University of
Texas at Austin. Contact her at ljohn@ece.utexas.edu.
5
 Guest Editors’ Introduction

Machine Learning for Systems

Heiner Litz Milad Hashemi
University of California Santa Cruz Google

& SPECIALIZED COMPUTER SYSTEMS have driven that hit many different applications of what is
the performance and capability of deep learning possible in this area:
over the past decade.1 However, as machine The first article sets the stage on machine learn-
learning models and systems improve, there is a ing in systems by introducing “A Taxonomy of ML
growing opportunity to also use these models to for Systems Problems.” It provides a classification
improve how we design, architect, optimize, and of problems in the system’s domain and analyzes
automate computer systems and software. This their applicability to machine learning. The article
is a challenging area, both from correctly observes that machine learn-
a learning and a systems per- ing is not a one-size-fits-all technique
As machine learning
spective. Systems often impose and that it needs to be carefully
models and systems
tight size, latency, or reliability designed for a particular problem. An
improve, there is a
constraints on learning mecha- growing opportunity to important contribution of this article
nisms that do not arise in other also use these models is that it provides advice to the read-
applications of machine learn- to improve how we ers about what problem types are
ing, such as computer vision or design, architect, amenable to machine learning and
natural language processing. optimize, and automate which ones are rather to be addressed
From a learning perspective, computer systems and with classic approaches.
systems is a challenging appli- software. This is a The second article, “A Programma-
cation, where input features are challenging area, both
ble Approach to Neural Network
often large and sparse, action from a learning and a
Compression,” explores compression
systems perspective.
spaces are gigantic, and gener- and pruning techniques to reduce the
alization is a key attribute. size and computational complexity of
Yet, despite these chal- deep neural networks. While in many cases com-
lenges, combining learned systems with com- pression can reduce the size of DNNs significantly
puters has the potential to revolutionize the field. with almost no loss in accuracy, determining the
This is a young area, and interdisciplinary optimal technique and intensity of pruning and
research provided the historical constraints of compression is challenging. This article introduces
both systems and ML is challenging. Therefore, a programmable system to easily explore and auto-
we are excited to highlight six excellent papers matically optimize the compression strategy
enabling significant performance gains and storage
savings.
Digital Object Identifier 10.1109/MM.2020.3016551 In “A Single-Shot Generalized Device
Date of current version 1 September 2020. Placement for Large Dataflow Graphs,” a new

6
0272-1732 ß 2020 IEEE Published by the IEEE Computer Society IEEE Micro
approach is proposed to automatically optimize
the training phase of large deep neural networks.
Large DNNs are generally trained on networks of
GPUs or other training accelerators and hence
the optimal partitioning of the DNN among these
devices while considering the communication
patterns is challenging. This article proposes
machine learning techniques to address this
challenging systems problem, significantly out-
performing classical graph partitioning and
placement techniques.
In “RELEQ: A Reinforcement Learning Appro-
ach for Automatic Deep Quantization of Neural
Networks,” the authors describe a reinforcement
learning mechanism to optimize deep neural net-
work architectures. The technique automatically
explores the best bit precisions for storing
weights in neural networks, achieving compres-
sion with minimal loss of accuracy. The tech-
nique leverages proximal policy optimization to
skip the large design space of possible encodings.
“Enhancing Model Parallelism in Neural
Architecture Search for Multidevice Systems” is
another article that analyzes automatic optimi-
zation techniques for improving the perfor-
mance of deep neural networks. In this case, the
article optimizes neural network architecture
September/October 2020
depending on the target hardware. The appro-
ach trades accuracy against inference latency to
achieve high performance on a range of hard-
ware including mobile devices and GPUs.
Our issue concludes with an article proposing
machine learning techniques to improve the secu-
rity of hardware systems, particularly networks-
on-chip. In “TSA-NoC: Learning-Based Threat
Detection and Mitigation for Secure Network-on-
Chip Architecture,” the authors describe how
machine learning techniques can be used to
detect abnormal network behavior induced by
hardware trojans and how to automatically
reconfigure the network to address security
vulnerabilities.
& REFERENCE
1. D. Hernandez and T. B. Brown, “Measuring the
algorithmic efficiency of neural networks,” 2020,
arXiv:2005.04305.
Heiner Litz is with the University of California Santa
Cruz. Contact him at: hlitz@ucsc.edu.
Milad Hashemi is with Google. Contact him at:
miladhashemi@utexas.edu.
7
 Theme Article: Machine Learning for Systems

A Taxonomy of ML for
Systems Problems
Martin Maas
Google Research, Brain Team

Abstract—Machine learning has the potential to significantly improve systems, but only
under certain conditions. We describe a taxonomy to help identify whether or not machine
learning should be applied to particular systems problems, and which approaches are
most promising. We believe that this taxonomy can help practitioners and researchers
decide how to most effectively use machine learning in their systems, and provide the
community with a framework and vocabulary to discuss different approaches for applying
machine learning in systems.

& MACHINE LEARNING (ML) has transformed
many research areas, from image recognition to
natural language processing. ML has also had a
significant impact on computer systems and
inspired the development of new systems for
designing and training ML models (e.g., Tensor-
Flow), as well as new hardware (e.g., TPUs).
In contrast to such Systems for ML research,
ML for Systems is only now seeing more attention.
While ML has long been used in areas such as
branch prediction, recent work has shown prom-
ising results in caching, compilers, and cluster
scheduling. These advances indicate that ML
could hold the key to improving many areas in
computer systems. However, these successes
Digital Object Identifier 10.1109/MM.2020.3012883
Date of publication 30 July 2020; date of current version
1 September 2020.
hide the fact that ML does not always lead to the
immediate wins that its popularity promises.
Applying ML to systems does not always outper-
form highly tuned non-ML solutions, and even if
ML improves a particular metric, its resource
cost does not always justify the improvement.
This article makes the case that while ML
has the potential to improve systems, it does so
only in certain cases. Furthermore, different ML
techniques are suitable for different problems.
We therefore categorize systems problems and
develop a taxonomy for identifying whether ML
can be applied, and what strategies might be
suitable. We also provide a bibliography1 that
matches existing work to this taxonomy. We
believe that our approach can help practi-
tioners and researchers decide how to most
effectively use ML in their systems and provide
the research community with a framework to
discuss ML for Systems strategies.

8 This work is licensed under a Creative Commons

Attribution 4.0 License. For more information, see
https://creativecommons.org/licenses/by/4.0/
Published by the IEEE Computer Society IEEE Micro
BACKGROUND policies collect databases of examples and learn
“Systems” is a broad term. To ground discus- from them, either by exploring a search space
sion in a common terminology, we therefore focus (e.g., autotuners) or by building a lookup table
on “system policies.” Given a software or hard- and using it in future executions (e.g., profile-
ware component that makes decisions related to guided optimization). However, we typically start
the execution of computer programs, a system explicitly calling this approach ML only when we
policy describes how these decisions are made. use tools from the ML literature, such as support
Compiler passes, branch predictors and memory vector machines, decision trees, or neural
allocators are all examples of system policies. networks.
When we talk about ML for Systems, we therefore What sets ML approaches apart from lookup
mean “using ML in the implementation of a sys- tables is that they
tem policy.” Specifically, we focus on supervised can potentially gener-
This article makes the
and reinforcement learning (RL); other techni- alize to unseen cases.
case that while ML has
ques such as learning-to-learn, transfer learning, An early example is
the potential to improve
or representation learning have applications in systems, it does so neural branch predic-
systems as well,2 but are out of scope for this only in certain cases. tors. Recently, there
taxonomy. Furthermore, different has been an explo-
System policies typically fall into four cate- ML techniques are sion of such techni-
gories that oftentimes correspond to the suitable for different ques, ranging from
degree to which a system has been optimized. problems. We therefore learning compiler
categorize systems optimizations3 to

An ad hoc policy based on assumptions at problems and develop cluster scheduling.4
the time of development. Consider an inlin- a taxonomy for
Note that a compli-
ing policy in a compiler: An ad-hoc heuris- identifying whether ML
cated ML technique
can be applied, and
tic could inline all functions with less than is not a necessary
what strategies might
10 instructions. requirement to gener-
be suitable.

An empirically tuned policy that has been alize. For example,
optimized for a set of benchmarks. This is work on index struc-
the type of policy often published in research tures has shown that while it is possible to learn
papers. For example, such a policy could a key distribution using neural networks, a simi-
consider the call graph and apply carefully lar goal can be achieved by fitting splines to its
crafted inlining rules, chosen to optimize per- cumulative distribution function.5
formance across a set of benchmarks (e.g., Note that highly tuned and complex heuris-
SPEC). tics are similar to data-driven policies. For exam-

A data-driven policy that optimizes towards a ple, compilers have cost models to accurately
specific target. In contrast to an empirically- predict performance and generalize to unseen
tuned policy that uses benchmarks as a proxy programs, but only recently has this problem
for the real target, this policy is tuned to the been revisited using modern ML techniques
target itself (e.g., feedback-directed inlining). such as neural networks.6 A corollary is that if a

An adaptive data-driven policy that does not system has been well-tuned, it can be difficult to
make the same decision for the same target improve the baseline with a learned policy. One
every time, but adapts online in response to interpretation is that engineers have used a
its own decisions. An example is trace-based "real-life version of gradient descent" to move
JIT compilers that re-evaluate and revise the system to a local optimum, not unlike what
inlining decisions over time. ML would do. This shows that applying ML to
systems does not represent a fundamental
ML is often defined as the ability of a program departure from systems research, but only pro-
to learn from experience. By this definition, data- vides a new set of tools.
driven policies are a form of ML, although poten- This aspect is often obscured in the discus-
tially a rudimentary one. Most data-driven sion of ML for Systems, in part due to the

September/October 2020
9
 Machine Learning for Systems
popularity of end-to-end learning. Modern ML
techniques can learn very complex behavior,
and it is therefore possible to train models that
learn complex policies end-to-end. We have seen
this approach in areas ranging from RL for
caches, to end-to-end cluster scheduling, to RL
for compilers. While these approaches often
work, they are not always data efficient, con-
sume large amounts of resources, and some-
times do not conclusively outperform strong
baselines. Many problems have a known struc-
ture that can be captured in a handwritten heu-
ristic. However, end-to-end learning has to learn
this structure from scratch and may re-learn
known facts, at the cost of maximizing perfor-
mance on the otherwise intractable part.
We therefore argue that effectively applying
ML to systems requires identifying which part of a
systems policy requires ML, and developing spe-
cific ML techniques for this part. This is supported
by the fact that many recent successes of ML for
Systems have focused on specific subproblems
rather than end-to-end learning (e.g., learning-
based index structures, learned cost models6).
Note that these learning techniques were used in
areas that were already data-driven. As such,
there was already an interface for the learning
technique to fit into the conventional portion of
the system, as well as strong baselines. Mean-
while, when learning replaces an end-to-end heu-
ristic, it can be hard to attribute which gains are
due to shifting to a data-driven approach versus
ML. The resulting tradeoff space can be difficult to
reason about, in part because the vocabulary to
discuss the way ML is used is often missing.
CONSTRAINTS AND TRADEOFFS
When considering whether to apply ML, sev-
eral tradeoffs need to be considered: The prob-
lem needs to be well-suited to ML, the
deployment constraints need to allow for ML,
and suitable data needs to be available. We now
discuss these considerations in turn.
Problem Suitability

Target: System policies have different optimi-
zation metrics (e.g., resource utilization or
the worst case latency). ML can help for
10
metrics that are difficult to reason about,
while metrics that can be optimized analyti-
cally might be addressed without ML.

Data-driven baselines: If the main benefit of
ML is to make a heuristic data-driven, simple
data-driven methods should be tried first. In
some cases, this is ML (e.g., because the
input features are too complex), but in
others, a lookup table might yield most of
the benefits. For example, table-based
branch predictors are competitive with ML
approaches.

High-dimensional input space: If the number
of possible inputs is small, a lookup table can
be used to memorize all predictions instead
of using ML for generalization.
Deployment Constraints

Latency: ML for Systems differs from areas
such as NLP in its ultralow latency require-
ments. OSs and runtimes often need to make
decisions in micro/nanoseconds, branch pre-
dictors in cycles. In contrast, even small neu-
ral networks often take hundreds of
microseconds and GBRTs take tens of micro-
seconds. More complex models can take tens
of milliseconds. Before applying ML, it is
therefore necessary to identify the latency
requirements—offline policies (e.g., in com-
pilers) are often more latency-tolerant than
online policies (e.g., in schedulers).

Space/time overheads: Even if prediction
latency is low, models that run often can con-
sume significant resources (CPU/DRAM). In
our case study, we use a model for memory
allocation, with 250 000 allocations/second.
The model takes >100 ms; running it at every
allocation is thus infeasible. To use ML in
such a setting, formulating the problem so
that prediction results can be cached is
crucial.

Custom hardware: Scenarios where latencies
are large (e.g., compilers) can use GPUs/
TPUs, and hardware policies (e.g., branch
predictors, prefetchers) can have custom
implementations. Other models typically run
on the CPU, which can be inefficient for neu-
ral networks. Compiling and inlining the
model directly into the code can help.
IEEE Micro

Risk/robustness/interpretability: Models
sometimes mispredict and systems need to
adapt. The specific use case dictates the
robustness requirements and risk, and many
non-ML systems policies are not 100% robust
themselves. However, many ML models are
opaque, which makes problems more difficult
to track down when they occur. In high-risk
scenarios, it can therefore be required to rely
on interpretable approaches, even if they
yield lower accuracy (e.g., decision trees).
Data Availability

Privacy/Security: It needs to be ensured that
the data used for learning does not expose
sensitive data (e.g., encryption keys).


Offline Versus Online Learning: Some policies
learn online (e.g., branch predictors) while
others train offline. For the latter, ML needs
to be integrated into the development and
QA cycle (e.g., is the model deployed with a
new binary or updated dynamically?). Mean-
while, online training can be challenging for
expensive models that require accelerators
to train.


Distribution Shifts: Models take time to train
and require quality control. ML therefore
introduces challenges in cases where the
output distribution of the policy shifts
quickly. Such cases may require an online
approach.
Applying ML requires trading off these priori-
ties. For example, it is sometimes more impor-
tant to be robust than achieve perfect accuracy.
Even if ML achieves better accuracy on a given
task, it may therefore not be suitable.
TAXONOMY OF ML FOR SYSTEMS
To give researchers and practitioners a
framework to reason about ML for Systems, we
divide ML for Systems problems into five catego-
ries that we believe capture most problems
(they can overlap). We start by classifying exist-
ing work.

Anomaly detection: Detect when a system
does not behave as expected (e.g., system
September/October 2020
failures, security incidents, interference, per-
formance regressions7).

Forecasting: Predict future behavior of a sys-
tem. This includes program speculation (e.g.,
prefetching and branch prediction), object
lifetime prediction,8 cardinality estimation in
databases, and system or network resource
demand forecasting.

Extrapolation: Given a policy for a known sub-
set of inputs, extend it to unseen inputs. This
can include classification tasks in schedulers
(e.g., whether a program is scale-up or scale-
out4), cost models in compilers, or selecting
configuration parameters.

Discovery: Generate a new/better policy.
This includes policies that could not be
handwritten, either because the rules are
too counterintuitive for a human to come
up with (e.g., caching policies identified
through learning) or because they are based
on a large amount of data. Examples include
custom inlining heuristics based on perfor-
mance profiles or data-specific index struc-
tures for databases.

Optimization: Exploring a potentially large
space to find a good or optimum solution
(e.g., ML for hardware design, autotuners).
The full bibliography is available online.1
Note that all of these use cases can and are
being solved without ML. We argue that this
list represents a hierarchy of how hard it is to
achieve improvements with ML, from easiest
to hardest.
Anomaly Detection
Detecting faults in mechanical and control
systems is one of the traditional uses of AI.
Anomaly detection is an attractive target for ML
because it is often data-driven already: Many
anomaly detectors start from a set of examples
and cluster them to detect outliers. Modern ML
adds new tools, such as autoencoders, where
the reconstruction error measures anomaly.7
Baselines: Simple data-driven baselines
should be tried first (e.g., clustering). If a base-
line is used that is not data-driven, the main lift
from ML might be the result of switching from a
heuristic to a data-driven approach.
11
 Machine Learning for Systems
Strategies: The first step is to select input fea-
tures and model their statistical properties.
Before using complex techniques, it is worth try-
ing clustering approaches that show a difference
between normal and anomalous examples. Com-
plex techniques can be appropriate when the
features cannot be readily encoded for cluster-
ing (e.g., if they represent graphs), or if it is
unknown which of a large set of features to use.
In such cases, complex neural networks such as
graph neural networks or recurrent neural net-
works can be appropriate, either used as an
embedding, or in an autoencoder setting.
Forecasting
Most computer systems use some form of
forecasting, either implicitly or explicitly. Any
form of hardware speculation relies on forecast-
ing (e.g., prefetching), and schedulers adjust
resources based on predicting future resource
usage from current/past usage.
Baselines: Because most systems use fore-
casting, it is important to determine whether
these baselines are data-driven or heuristic-
based. If they are not data-driven, prior to creat-
ing a complex ML-based approach, a simpler
data-driven baseline should be tried. In many
cases, the simplest baseline is a lookup table
that maps input features to predictions (e.g.,
branch predictors or inline caches).
Strategies: ML models can provide benefits
over table-based forecasting when generaliza-
tion is required. For example, recent work2 that
learned from application-level features in stor-
age shows that a lookup table approach
degrades over time as previously unseen fea-
tures appear. In such cases, different models can
be applied, from decision trees to neural
networks.
Extrapolation
Most extrapolation in systems is heuristic-
driven and often performs classification. For
example, a scheduler might compare counters
against hard-coded thresholds.
Baselines: Extrapolation should start from a
data-driven baseline. However, as previously
observed, highly tuned heuristics are similar to
data-driven policies and may therefore be appro-
priate baselines as well.
12
Strategies: Extrapolation strategies are often
problem-specific. Collaborative filtering4 has
been used (e.g., for workload classification), but
supervised learning also works for many areas,
such as predictions based on stack traces8 or
learning memory access patterns.9
Discovery
Discovery is about designing previously
unknown policies, such as new caching strate-
gies.10 There are two variants: Discovering a
new general policy that is intended to be uni-
versally used, and learning a specialized pol-
icy for a particular set of workloads (i.e., data-
driven).
Baselines: Since the goal of discovery is to
find a new algorithm, it should be evaluated in
the same way as existing approaches of the
same (data or nondata-driven) type.
Strategies: A common strategy is RL, where a
policy is learned by exploring different deci-
sions in a simulator or real environment and
updating the policy based on a reward. While
most problems can be framed as RL, it is in
practice harder to train a model using it. Sim-
pler approaches are available: One approach is
to use an expensive method to solve instances
of the problem offline (e.g., SAT/ILP solvers)
and use them as training inputs for imitation
learning.3 Another alternative is to design sev-
eral parameterized subpolicies and learn a pol-
icy that picks the best one (i.e., a bandit-based
approach).
Optimization
In some cases, ML can be used to solve a
static optimization problem (e.g., a complex
scheduling problem). In contrast to discovery,
this approach is not learning a general policy
to solve new problem instances, but is using
ML to explore the search space of a specific
instance.
Baselines: There are many well-known optimi-
zation techniques, including genetic algorithms
and simulated annealing, some of which have
been shown to work in the same areas as ML
(e.g., playing video games11). While ML techni-
ques can be used for optimization (e.g., gradient
descent is a form of optimization), this alone
IEEE Micro
Figure 1. How to decide which ML approach to use.
does not constitute learning and is not necessar-
ily better than alternatives.
When applying ML to an optimization prob-
lem, it is important to identify whether the goal
is to learn a policy that transfers to new problem
instances (i.e., discovery) or whether it is to
solve a standalone instance. While discovery
problems need to be evaluated based on their
zero-shot performance for a previously unseen
example, optimization problems need to be eval-
uated against other optimization techniques,
such as genetic algorithms. Both baseline and
ML approach need to be evaluated with the
same amount of resources.
Strategies: RL has been successfully used in
optimization problems, by learning a policy that
selects the next design decision, combined with
a value function that estimates the quality of a
particular choice. The policy function and value
function can potentially be reused when solving
a new optimization problem, leading to transfer-
ability. It is, however, possible that these func-
tions overfit to a particular optimization
example, so transferability cannot be taken for
granted.
For design spaces that are low-dimensional,
Bayesian optimization frameworks can work
well. It is also important to not dismiss alterna-
tive optimization strategies in favor of ML, par-
ticularly if transferability is not required.
CHOOSING AN ML STRATEGY
We now discuss how to determine whether a
problem is suitable for ML. While major
September/October 2020
improvements from ML have been shown in all
five areas, the further we go from “anomaly
detection” to “optimization,” the more a model
has to learn about how the system works, and
the more data/examples are required.
The first step is to check whether the input
features are predictive of the output. For low-
dimensional prediction problems, this can be
done with a lookup table baseline that stores a
prediction for every possible input. For higher
dimensional problems, replaying a run with an
oracle (e.g., in a simulator) is an alternative. This
indicates the headroom.
The next decision is the scope of the learned
system policy. Almost every system could be
framed as an end-to-end RL problem. However,
such a model needs to not only learn the statisti-
cal properties of the data but also everything
about its environment. It can therefore be advan-
tageous to separate the prediction problem from
the rest of the system, to limit the complexity of
the function that needs to be learned. As shown in
the next section, it is sometimes possible to
decompose an end-to-end problem into a super-
vised learning portion and a (traditional) algorith-
mic portion that consumes these predictions.
Alternatively, the latter (reduced) problem may
be solved with ML itself (e.g., RL).
Once the learning problem has been defined,
an ML technique needs to be chosen. We frame
our recommendations as a decision diagram
(see Figure 1). For each presented ML type, dif-
ferent learning strategies are available. For
example, supervised learning could use neural
networks or decision trees. One important factor
13
 Machine Learning for Systems

Figure 2. ML for memory allocation.8 (a) Visualizing memory fragmentation. (b) Reinforcement learning
approach. (c) Imitation learning approach. (d) Decomposed approach using supervised learning and a new
type of memory allocator. (e) Cþþ server workload memory reduction (running against synthetic requests).8
(f) Synthetic memory trace. (g) Final steady-state fragmentation. (h) Allocation/inference latency. (i) ILP solver
scalability.

is how to deploy a model within a system, based
on resource constraints. Deployment ranges
from compiling a model directly into code to
running the model offline (e.g., at compile time).
The uniqueness of systems problems and their
constraints may necessitate new ML techniques.
CASE STUDY
We demonstrate how the previous insights
apply to recent work on ML for memory allo-
cation.8 The goal of this work was to reduce
fragmentation in Cþþ workloads with huge
(2 MiB) pages and varying memory footprints.
Since Cþþ cannot move objects, long-lived
objects can prevent entire pages from being
released to the OS [see Figure 2(a)], causing
fragmentation. To solve this problem, a mem-
ory allocator needs to reason about object
lifetimes and group objects with similar life-
times together.
This requires the memory allocator to pre-
dict future behavior of the application, which
suggests that this could be a target for ML. The

14 IEEE Micro
allocator represents the following system policy:
Given a sequence of allocation requests (each
with a size and an unknown lifetime), the alloca-
tor needs to place objects in virtual address
space such that the number of live 2 MiB pages
(i.e., pages containing at least one object) is min-
imized. At the time of allocation, we know the
current stack trace and the size of the allocation.
As such, we need to solve a forecasting problem.
Our baseline is TCMalloc, which organizes
objects by size but ignores lifetime. Figure 2(e)
shows used memory and actual memory foot-
print for running a server workload against syn-
thetic inputs. The baseline incurred over 2
fragmentation (footprint/used) on average, over
4 at low memory usage. Our goal was to reduce
this fragmentation. Since the trace is large (110M
allocations), we also generated a synthetic
driver and baseline that replicates similar behav-
ior with 5000 allocations [see Figure 2(f)].
Reinforcement Learning
This looks like a perfect setup for RL, with a
sequence of decisions (where to place each
object) and a reward function (memory fragmen-
tation). We therefore started collecting traces
and built a simulator to replay these traces,
which would allow an RL policy to learn a good
allocation strategy. However, several constraints
make RL challenging for this scenario. First, the
state space is large and complex, as there are 264
addresses. Furthermore, the number of alloca-
tions is large (millions/min), and rewards are
sparse, creating credit assignment challenges.
We implemented a naive DQN model that
observes the state of all hugepages and picks an
allocation target for our synthetic trace. This
simple model outperformed the baseline, but
needs to run every allocation and takes 2 ms
(TCMalloc’s fast path takes 8.3 ns). Even if opti-
mizations improved this latency by 1000, this
approach would thus be impractical.
Imitation Learning
Given full allocation traces of a program, the
problem can be solved offline. It reduces to a
two-dimensional packing problem, which can
be solved retroactively using an ILP solver.
Given such a solution, we could train a policy
against it. This yields low fragmentation [see
September/October 2020
Figure 2(g)], but the ILP approach does not
scale [see Figure 2(i)], as applications have
>10M allocations/minute (the solver did not
even solve the full synthetic trace). Further-
more, just like with RL, we would need a DQN
that learns from these solutions, and running
such a model every allocation is impractical.
Decomposed Supervised Learning
We tackle these challenges by breaking up
the prediction problem. Instead of learning
object placement end-to-end, we decomposed it
into a supervised learning portion that predicts
(potentially incorrectly) the lifetime of an object
based on its stack trace. We built a new mem-
ory allocation algorithm (LLAMA) that relies on
these predictions and can detect when past pre-
dictions were incorrect. Using this approach,
we can avoid running the model at every alloca-
tion, because results can be cached (we use a
hash table and identify stack traces based on a
cheap fingerprinting mechanism).
Following the rules laid out in this article, we
validated our approach by using it with a lifetime
oracle in the simulator. We then replaced the
oracle with a lookup table, but found that this
table did not generalize across different versions
of an application. We then decided to use an
LSTM neural network that we compiled directly
into the CPU code to maximize performance (a
similar approach could be used for a DQN).
This model is easier to train than RL, because
training is supervised. This also means that train-
ing does not require full traces from entire runs
(like RL or imitation learning). We can instead
sample individual allocations. Even so, running
the LSTM every allocation takes 144 ms and is
therefore impractical [see Figure 2(h)]. However,
since predictions now depend only on the current
stack trace and no state, we can cache predic-
tions, bringing the predictions down to 20 ns.
LLAMA reduces steady-state fragmentation by 43%
in Figure 2(e) (up to 78% for other workloads8).
Lessons Learned
This approach shows that while it is often pos-
sible to apply end-to-end learning to an ML for Sys-
tems problem, it is not always the best approach.
The successful solution required us to apply the
rules laid out in this article: We moved from an
15
 Machine Learning for Systems
empirically-tuned policy (i.e., past memory alloca-
tors) to a data-driven policy. Since a lookup table
alone was not sufficient, we applied ML to learn an
embedding of stack traces, using supervised learn-
ing. The other parts of the problem (selecting how
to allocate memory based on the prediction) were
solved with conventional heuristics that tolerate
mispredictions.
CONCLUSION
ML for Systems is an emerging area. To
maximize its potential, ML needs to be used
in the most effective way and evaluated against
suitable baselines. Meanwhile, systems-specific
requirements such as low latency and large
input/output spaces necessitate systems-
specific innovation
on the ML side. Our ML for Systems is an
goal is to establish emerging area. To
a framework and maximize its potential,
vocabulary to dis- ML needs to be used in
cuss these alterna- the most effective way
tives and tradeoffs. and evaluated against
suitable baselines.
We see this article
and the accompa-
nying bibliography1 as a contribution toward
this discussion and believe that these points are
going to be increasingly important as we are see-
ing more ML for Systems work.
ACKNOWLEDGMENTS
The author would like to thank A. Klimovic, A.
Goldie, A. Mirhoseini, C. Raffel, H. Lim, J. Laudon,
M. Phothilimthana, M. Abadi, R. Singh, R. Frostig,
and S. Roy for their feedback.
& REFERENCES
1. Supplementary material/bibliography, 2020. [Online].
Available: https://github.com/google-research/ml-for-
systems-taxonomy
2. G. Zhou and M. Maas, “Multi-task learning for storage
systems,” in Proc. ML Syst. Workshop, 2019. [Online].
Available: http://mlforsystems.org/assets/papers/
neurips2019/multi_task_zhou_2019.pdf
16
3. C. Mendis, C. Yang, Y. Pu, S. Amarasinghe, and
M. Carbi, “Compiler auto-vectorization with imitation
learning,” in Proc. Adv. Neural Inf. Process. Syst.,
2019, pp. 14625–14635.
4. C. Delimitrou and C. Kozyrakis, “Quasar: Resource-
efficient and QoS-aware cluster management,” in
Proc. 19th Int. Conf. Archit. Support Program. Lang.
Oper. Syst., 2014, pp. 127–144.
5. T. Neumann, “The case for B-tree index structures,”
2017. [Online]. Available: http://databasearchitects.
blogspot.com/2017/12/the-case-for-b-tree-index-
structures.html
6. C. Mendis, A. Renda, S. Amarasinghe, and
M. Carbin. “Ithemal: Accurate, portable and fast basic
block throughput estimation using deep neural
networks,” in Proc. Int. Conf. Mach. Learn., 2019,
pp. 4505–4515.
7. M. Alam, J. Gottschlich, N. Tatbul, J. S. Turek,
T. Mattson, and A. Muzahid, “A zero-positive learning
approach for diagnosing software performance
regressions,” in Proc. Adv. Neural Inf. Process. Syst.,
2019, pp. 11627–11639.
8. M. Maas, D. G. Andersen, M. Isard, M. M. Javanmard,
K. S. McKinley, and C. Raffel, “Learning-based
memory allocation for Cþþ server workloads,” in Proc.
25th Int. Conf. Archit. Support Program. Lang. Oper.
Syst., 2020, pp. 541–556.
9. M. Hashemi et al., “Learning memory access
patterns,” in Proc. Int. Conf. Mach. Learn., 2018,
pp. 1919–1928.
10. D. S. Berger, “Towards lightweight and robust
machine learning for CDN caching,” in Proc.
17th ACM Workshop Hot Topics Netw., 2018,
pp. 134–140.
11. F. P. Such, V. Madhavan, E. Conti, J. Lehman, K. O.
Stanley, and J. Clune, “Deep neuroevolution: Genetic
algorithms are a competitive alternative for training
deep neural networks for reinforcement learning,”
2017, arXiv:1712.06567.
Martin Maas is a Research Scientist at Google. His
research interests span language runtimes, computer
architecture, systems, and machine learning. Maas
received the Ph.D. degree in computer science from
UC Berkeley. Contact him at mmaas@google.com.
IEEE Micro
 Theme Article: Machine Learning for Systems

A Programmable
Approach to Neural
Network Compression
Vinu Joseph and Ganesh L. Gopalakrishnan Animesh Garg
University of Utah University of Toronto and Vector Institute
Saurav Muralidharan and Michael Garland
NVIDIA

Abstract—Deep neural networks (DNNs) frequently contain far more weights, represented
at a higher precision, than are required for the specific task, which they are trained to
perform. Consequently, they can often be compressed using techniques such as weight
pruning and quantization that reduce both the model size and inference time without
appreciable loss in accuracy. However, finding the best compression strategy and
corresponding target sparsity for a given DNN, hardware platform, and optimization
objective currently requires expensive, frequently manual, trial-and-error experimentation.
In this article, we introduce a programmable system for model compression called
CONDENSA. Users programmatically compose simple operators, in Python, to build more
complex and practically interesting compression strategies. Given a strategy and user-
provided objective (such as minimization of running time), CONDENSA uses a novel Bayesian
optimization-based algorithm to automatically infer desirable sparsities. Our experiments
on four real-world DNNs demonstrate memory footprint and hardware runtime throughput
improvements of 188
and 2.59
, respectively, using at most ten samples per search.

& MODERN networks (DNNs) are

DEEP NEURAL
complex and often contain millions of parameters
Digital Object Identifier 10.1109/MM.2020.3012391
Date of publication 28 July 2020; date of current version
1 September 2020.
spanning dozens or even hundreds of layers.1; 2
This complexity translates into substantial mem-
ory and runtime costs on hardware platforms at
all scales. Recent work has demonstrated that
DNNs are often overprovisioned and can be com-
pressed without appreciable loss of accuracy.
Model compression can be used to reduce both

September/October 2020 Published by the IEEE Computer Society 0272-1732 ß 2020 IEEE
17
 Machine Learning for Systems
model memory footprint and inference latency
using techniques such as weight pruning, quanti-
zation, and low-rank factorization.3 5 Unfortu-
nately, the requirements of different compression
contexts—DNN structure, target hardware plat-
form, and the user’s optimization objective—are
often in conflict. The recommended compression
strategy for reducing inference latency may be
different from that required to reduce total
memory footprint. For example, in a convolu-
tional neural network (CNN), reducing inference
latency may require pruning filters to realize
speedups on real hardware,4 while reducing
memory footprint may be accomplished by zero-
ing out individual weights. Similarly, even for the
same optimization objective, say reducing infer-
ence latency, one may employ filter pruning for a
CNN, while pruning 2-D blocks of nonzero weights
for a language translation network such as Trans-
former, since the latter has no convolutional
layers. Thus, it is crucial to enable convenient
expression of alternative compression schemes;
however, none of today’s model compression
approaches help the designer tailor compression
schemes to their needs.
Current approaches to model compression
also require manual specification of compres-
sion hyperparameters, such as target spar-
sity—the proportion of zero-valued parameters
in the compressed model versus the original.
However, with current approaches, finding the
best sparsity often becomes a trial-and-error
search, with each such trial having a huge cost
(often multiple days for large models such as
BERT), as each such trial involves training the
compressed model to convergence, only to find
(in most cases) that the compression objec-
tives are not met. The main difficulty faced by
such unguided approaches is that sparsities
vary unpredictably with changes in the com-
pression context, making it very difficult to pro-
vide users with any guidelines, whatsoever.
Therefore, automatic and sample-efficient appr-
oaches that minimize the number of trials are
crucial to support the needs of designers who
must adapt a variety of neural networks to a
broad spectrum of platforms targeting a wide
range of tasks.
To address the above-mentioned problems
of flexible expression of compression strategies,
18
automated compression hyperparameter infer-
ence, and sample efficiency, we make the follow-
ing contributions.
1) We present CONDENSA, a new framework for
programmable neural network compression.
CONDENSA supports the expression of the
overall compression strategy in Python using
operators provided by its compression library.
Since each strategy is a Python function, users
are able to programmatically compose ele-
mentary schemes to build much more com-
plex and practically interesting schemes.
2) We present a novel sample-efficient algorithm
based on Bayesian optimization (B.O.) in
CONDENSA for automatically inferring optimal
sparsities based on a user-provided objective
function. Given CONDENSA’s ability to support
the expression of meaningful high-level objec-
tive functions—for example, the throughput
(images per second) of a CNN—users are freed
from the burden of having to specify compres-
sion hyperparameters manually.
3) We demonstrate the effectiveness of
CONDENSA on three image classification and
language modeling tasks, resulting in mem-
ory footprint reductions of up to 188
and
runtime throughput improvements of up to
2:59
using at most ten samples per search.
BACKGROUND
For a given task such as image classification,
assume we have trained a large reference model
w ¼ argminw LðwÞ, where LðÞ denotes a loss func-
tion (e.g., cross-entropy on a given training set),
and w 2 RP . Model compression refers to finding
a smaller model Q that can be applied to the
same task and ideally achieves the same accu-
racy as w. Model compression can be performed
in various ways, and CONDENSA currently sup-
ports two commonly used techniques: pruning
and quantization. In pruning, nonzero values
from w are eliminated or “pruned” to obtain Q.
Pruning is usually performed using some kind of
thresholding (for e.g., magnitude-based) and can
be unstructured (prune any nonzero value) or
structured (prune only blocks of nonzeros).4; 3
On the other hand, quantization retains the num-
ber of parameters in Q but assigns parameters in
IEEE Micro
w one of K codebook values, where the code- Listing 1. Example usage of the CONDENSA
book may be fixed or adaptive. CONDENSA sup- library.
ports low-precision approximation, which refers
to assigning each parameter in w a correspond-
ing lower-precision representation (for example,
converting from 32-bit to 16-bit floating-point).
Automating model compression involves
finding both an optimal compression strategy
for a given w, along with its corresponding com-
pression hyperparameters such as target spar-
sity with minimal manual intervention. Current
state-of-the-art frameworks in this domain
include AMC3 and AutoCompress,5 which use
reinforcement learning and simulated annealing,
respectively, to automatically find desirable tar-
get sparsities for a fixed pruning strategy. CON-
DENSA, in contrast, supports the programmable
expression of a wide variety of compression
strategies (not just pruning). Also, in the context
of automated model compression, each sample
corresponds to training the compressed model
to convergence, and can be extremely expensive
to compute; unfortunately, techniques such as
reinforcement learning, which is used in AMC,3 CONDENSA System Design
can be highly sample-inefficient.6 To minimize Figure 1 provides a high-level overview of the
the number of samples drawn, CONDENSA uses CONDENSA framework. As shown on the left-hand
a novel and sample-efficient B.O.-based algo- side of the figure, a user compresses a pretrained
rithm for automatically arriving at desirable tar- model w by specifying a compression scheme and
get sparsities. an objective function f. Both the compression
scheme and objective are specified in Python
using operators from the CONDENSA library; alter-
CONDENSA FRAMEWORK natively, users may choose from a selection of
In CONDENSA, users specify compression commonly used built-in schemes and objectives.
schemes to systematically describe how a given Apart from the operator library, the core frame-
reference model w is transformed into a com- work, shown in the middle of the figure, consists
pressed version. Schemes are expressed as primarily of two components: 1) the constrained
Python functions and can utilize a rich set of Bayesian optimizer for inferring optimal target
compression operators provided by the CON- sparsities; and 2) the “learning-compression” (L-
DENSA library. Integrating with the Python eco- C) optimizer7 for accuracy recovery. The remain-
system makes the expression of common der of this section describes both the Bayesian
compression patterns more natural. For exam- and L-C optimizers in more detail.
ple, operators can be combined with conditional
statements to selectively compress layers based Sample-Efficient Bayesian Optimization
on properties of the input DNN and/or target It is intuitive to split the problem of finding
hardware platform. The CONDENSA Library pro- optimal target sparsities into two stages: 1) find
vides operators and prebuilt schemes for the highest target sparsity that loses at most 
unstructured and structured (filter and block) accuracy w.r.t the original uncompressed model
pruning, and quantization. Listing 1 provides a w; and 2) in a constrained sparsity regime
concrete example of invoking CONDENSA to com- obtained from stage 1), optimize a user-provided
press a model. objective function f (e.g., throughput, or

September/October 2020
19
 Machine Learning for Systems
Figure 1. CONDENSA framework overview. The user provides the pretrained model (w), a compression scheme, and an
objective function f. CONDENSA uses the Bayesian and L-C optimizers to infer an optimal target sparsity s and
corresponding compressed model Q.
memory footprint) and return the solution as the
final sparsity. For both stages, CONDENSA uti-
lizes B.O., as shown in Figure 1.
B.O. is an optimization framework based on
continually updating a probabilistic model with
measurements of a function to be optimized.8
Given a set of parameters to be optimized, B.O.
makes black-box calls to the objective, updates the
probabilistic model with the new information, and
selects the next point to evaluate using an acquisi-
tion function that combines information about
the expectation and uncertainty of a function
value under the probabilistic model. CONDENSA
employs a Gaussian Process (G.P.) model for B.O.
due to its favorable statistical and computational
characteristics.9 It is worth highlighting that B.O.
leverages principled Bayesian inference to tradeoff
exploration and exploitation and is sample-effi-
cient for nonconvex black-box functions such as
the ones optimized by CONDENSA.8
In CONDENSA’s two-stage optimization pipe-
line, we first find a sparsity ratio sacc that constrains
the model accuracy function A to the provided .
We then constrain the sparsity search space to
ð0; sacc Þ while optimizing the user-provided objec-
tive function f. Note that we assume that A
decreases monotonically w.r.t. sparsity in the
region ð0; sacc Þ. For each stage, CONDENSA uses a
distinct acquisition function to guide the next best
point for function evaluation.
Stage 1: Solving Accuracy Constraints: Recall that
in the first stage of the sparsity inference process,
we aim to find the highest sparsity sacc that loses at
most  accuracy w.r.t. the original reference model
w. To this end, we first define a Level-Set L that
20
represents AccðwÞ   and aim to find the point on
the accuracy curve of the compressed model that
intersects with L; the sparsity corresponding to
this solution will be sacc . We propose a novel acqui-
sition function to find sacc named domain-restricted
upper confidence bound (DR-UCB).
DR-UCB builds upon an existing level-set black-
box optimization technique named ILS-UCB,10
which is characterized by two properties. 1) It pri-
oritizes searching in the region where the level set
intersects the accuracy curve. 2) It does not seek
to precisely learn the shape of the entire accuracy
curve. However, in CONDENSA, since accuracy
values can be safely assumed to decrease mono-
tonically with increasing sparsity, we notice that it
is also possible to progressively restrict the
search domain of sparsities based on whether the
currently sampled point meets the level-set con-
straints. In DR-UCB, we exploit this property to
greatly improve sample efficiency over ILS-UCB.
Mathematically, we define st , the sparsity value
sampled at iteration t using DR-UCB, as follows:
st ¼ argmax ð1  gÞsðsÞ  gjmðsÞ  Lj
s (1)
s.t. st > si 8i 2 ½0; t  1; Bf ðst Þ  L:
Here, Bf represents the L-C accuracy function, and
st is 1) greater than all the previous sparsities si ;
and 2) satisfies the level set constraint Bf ðst Þ  L.
We achieve this by minimizing the difference
between the GP’s mean curve mðsÞ and the level
set using the term jmðsÞ  Lj in (1); the parameter
g controls the tradeoff between exploitation and
exploration. Algorithm 1 illustrates how DR-UCB is
employed to efficiently find sacc .
IEEE Micro
Algorithm 1. Bayesian Sparsity Inference with
Domain Restriction
1: procedure BODRUCB ðBf , L, T Þ
" B f : Function to optimize
" L: Level set
" T : # Iterations
2: GP GP-Regressor.initialize()
3: s0 0; D ð0; 1Þ; X ;
4: for t 1; 2; . . . . T  1 do
5: st argmaxD DR-UCBðsjX0:t1 Þ
6: yt Bf ðst Þ
7: if st > st1 and yt  L then
8: D ðst ; 1Þ
9: end if
10: X0:t fX0:t1 ; ðst ; yt Þg
11: GP.Update(X0:t )
12: end for
13: return sT 1
14: end procedure
Stage 2: Optimizing the User-Defined Objective:
Once we find a sparsity sacc that satisfies the user-
provided accuracy constraints in stage 1, our next
objective is to find the final sparsity s that opti-
mizes the user-defined objective function f in the
constrained sparsity domain ð0; sacc Þ. For this, we
employ the upper and lower confidence bound
(UCB/LCB) acquisition functions for function max-
imization and minimization, respectively.9
Accuracy Recovery Using L-C
As described earlier in this section, given a ref-
erence model w, compression scheme, and target
sparsity (obtained automatically by the Bayesian
optimizer), CONDENSA tries to recover any accu-
racy lost due to compression. In this article, we
use the recently proposed L-C algorithm7 for accu-
racy recovery, which formulates model compres-
sion as a constrained optimization problem. L-C
naturally supports all the compression operators
supported by CONDENSA while providing opti-
mality guarantees whenever possible. Due to
space restrictions, we refer the reader to the arti-
cle by Carreira-Perpina n and Idelbayev7 for a
more detailed description of the L-C algorithm.
EVALUATION
We conduct extensive experiments and fully
analyze CONDENSA on three tasks: 1) image classi-
fication on the CIFAR-10 data set11 using the VGG-
192 and ResNet561 neural networks; 2) image clas-
sification on the ILSVRC (ImageNet) task12 using
September/October 2020
the VGG-16 neural network2; and 3) language
modeling on WikiText-213 using a two-layer LSTM
network described by Yu et al.14 We optimize the
networks in each task for two distinct objectives
described below.
Objective 1: Minimize Memory Footprint: The
memory footprint of a model is defined as the
number of bytes consumed by the model’s non-
zero parameters. Reducing the footprint below a
threshold value is desirable, especially for mem-
ory-constrained devices such as mobile phones,
and can be accomplished through either pruning
or quantization, or both. For reducing footprint,
we define a compression scheme that performs
unstructured pruning of each learnable layer
(except batch normalization layers), and then
quantizes it to half-precision floating-point, yield-
ing an additional 2
reduction. We denote this
scheme by P+Q and implement it using the
CONDENSA library as follows:
from schemes import Compose; Prune; Quantize
scheme ¼ Composeð½PruneðÞ; Quantize ðfloat16ÞÞ
Objective 2: Maximize Throughput: Inference
throughput is defined as the number of input
samples processed by a model per second and
is commonly used for measuring real-world per-
formance. For CIFAR-10 and ImageNet, we mea-
sure hardware inference throughput of the
compressed model in the objective function. We
use an NVIDIA Titan V GPU with the TensorRT 5
framework to obtain throughput data. For Wiki-
Text-2, due to the lack of optimized block-sparse
kernels for PyTorch, we measure the floating-
point operations (FLOPs) of the compressed
model instead as a proxy for inference perfor-
mance. To improve throughput, we focus on
removing entire blocks of nonzeros, such as con-
volutional filters, since they have been shown to
improve performance on real-world hardware.4
For CIFAR-10 and ImageNet, we use filter prun-
ing, since all the networks we consider are
CNNs. In WikiText-2, we employ block pruning
with a block size of 5.
Bayesian Optimizer Settings: We use a Gaussian
Processes prior with the Matern kernel (n ¼ 2:5),
length scale of 1.0, and a value of 0.1 with normali-
zation of the predictions. For the DR-UCB acquisi-
tion function, we use a g value of 0.95 for all our
21
 Machine Learning for Systems

Table 1. CONDENSA performance results on CIFAR-10, ImageNet, and WikiText-2.

Method Dataset Network s Accuracy rc Throughput

Baseline CIFAR-10 VGG19-BN 92:98% 1
1

CONDENSA P+Q CIFAR-10 VGG19-BN 0.99 93:26% 188:23
N/A

CONDENSA Filter CIFAR-10 VGG19-BN 0.79 93:34% 1:35
2:59

Baseline CIFAR-10 ResNet56 92:75% 1
1

AMC 3
CIFAR-10 ResNet56 N/A 90:1% N/A sF ¼ 2

CONDENSA P+Q CIFAR-10 ResNet56 0.95 91:42% 31:14
N/A

CONDENSA Filter CIFAR-10 ResNet56 0.63 93:18% 1:14
1:17

Baseline ImageNet VGG16-BN 91:50% 1
1

Filter Pruning4 ImageNet VGG16-BN 89:80%  4
N/A

AutoCompress 5
ImageNet VGG16-BN N/A 90:90% 6:4
N/A

AMC3 ImageNet VGG16-BN N/A 90:1% N/A sF ¼ 1:25

CONDENSA P+Q ImageNet VGG16-BN 0.93 89:89% 29.29 N/A

CONDENSA Filter ImageNet VGG16-BN 0.12 90:25% 1
1:16

Baseline WikiText-2 LSTM Log-Perplexity: 4.70 1
1

Lottery Ticket14 WikiText-2 LSTM N/A Log-Perplexity: 4.70  10
N/A

CONDENSA P+Q WikiText-2 LSTM 0.92 Log-Perplexity: 4.75 4:2
N/A

CONDENSA Block WikiText-2 LSTM 0.60 Log-Perplexity: 4.62 1:1
sF ¼ 2:14


Here, s represents the target sparsity obtained by CONDENSA, rc is the memory footprint reduction, and sF the FLOP
reduction. The level-set, represented by , is set to 2% below baseline in all experiments.

experiments with a bias toward sampling more in
the area of level set, with the intention that the
Bayesian optimizer results in a favorable sparsity
level in as few samples as possible. We imple-
mented DR-UCB using the fmfn/BO package.
Results
We present the memory footprint reductions
and inference throughput improvements obtai-
ned by CONDENSA for each of the three tasks we
evaluate in Table 1. For each task, we list the tar-
get sparsity obtained by the CONDENSA Bayesian
optimizer (s in the table), its corresponding
accuracy/perplexity (top-1 accuracy, top-5 accu-
racy, and log perplexity for CIFAR-10, ImageNet,
and WikiText-2, respectively), memory footprint
reductions using pruning and quantization
(column labeled rc ), and inference throughput/
FLOP improvements using filter/block pruning.

https://github.com/fmfn/BayesianOptimization
We also compare our approach with recent work
on automated model compression. For CIFAR-10
and ImageNet, we compare our results with AMC3
and AutoCompress,5 and for WikiText-2, we com-
pare with Yu et al.14 Since AMC3 and Yu et al.14 do
not report actual runtime numbers on hardware,
we report the corresponding FLOP improve-
ments instead (values marked sF ). We also use
FLOP reduction as a metric for LSTM block prun-
ing, as described above.
Using the P+Q scheme designed to minimize
memory footprint, CONDENSA is able to obtain
compression ratios up to 188
, which surpasses
those of frameworks such as AutoCompress. While
AMC and AutoCompress only report theoretical
FLOP improvements on CIFAR-10 and ImageNet,
the filter pruning strategy implemented using
CONDENSA yields real-world runtime improve-
ments of up to 2:59
on an NVIDIA Titan V GPU.
Since AMC and AutoCompress do not report the
number of samples evaluated to arrive at their

22 IEEE Micro
Figure 2. CONDENSA sparsity profiles for VGG19-BN and ResNet56 for CIFAR-10. Column 1 shows the
problem of the form “minimize memory footprint with a lower bound on accuracy,” while Column 2 illustrates
“maximize throughput with a lower bound on accuracy.” The dc line (gray) shows accuracy values if no
accuracy recovery with L-C is performed. Note that the x-axis ranges are different: the plots on the left have
sparsities ranging from 0.9 to 1.0 while those on the right have values ranging from 0 to 1.

solutions, we are unable to directly compare sam-
ple efficiencies with these frameworks; however,
we notice that CONDENSA obtains desirable model
sparsities using a fixed ten iterations per search
in all experiments. Finally, while we set the level
set to be 2% below the accuracy of the reference
model in all our experiments, we notice that
CONDENSA-compressed models often exceed base-
line accuracy.
Sparsity Profile Analysis
Figure 2 illustrates how a compressed model’s
accuracy, inference performance, and memory
footprint varies w.r.t. sparsity for the CIFAR-10
task. All three of these functions are assumed to be
unknown in our problem formulation, but we com-
pute them explicitly here to better understand the
quality of solutions produced by CONDENSA. For
each figure, compression accuracies (shown in
green) are obtained by running the L-C algorithm
to convergence for 100 sparsities ranging from 0.9
to 1.0 (for pruning + quantization), and from 0 to 1
for the filter and block pruning schemes; collect-
ing each such point requires between 30 min and
8 h of time on a single NVIDIA Tesla V100 GPU.
Inference throughput, FLOPs, and memory foot-
print data are collected for each compressed
model and depicted by red lines in the figures
(right-hand-side y-axis). We also show direct com-
pression (D-C) accuracies in gray for comparison;

September/October 2020
23
 Machine Learning for Systems

right y-axis) for filter pruning. We only show data

for convolutional layers as they dominate compu-
tation time for this network. We make two key
observations: 1) runtime speedups on real hard-
ware are largely correlated with compression
ratios, but may be affected by hardware and
implementation details (e.g., compare conv13
with conv14 in the Figure); and 2) higher com-
Figure 3. TensorRT runtimes and compression pression ratios and corresponding speedups for
ratios of convolutional layers in VGG19-BN (filter the later layers of the network, which indicates
pruning). that distributing a given global sparsity evenly
across network layers may not always be optimal,
and algorithms such as L-C are essential to auto-
direct compression refers to applying a compres- matically finding desirable distributions of spar-
sion scheme to a model without any subsequent sity across layers.
attempt at accuracy recovery. In each figure, the
sparsity found by CONDENSA is shown as a black
CONCLUSIONS
vertical dashed line.
This article has presented CONDENSA, which
We notice three important trends in Figure 2.
is a flexible programming system for DNN compres-
sion and corresponding hyperparameter opti-
1) CONDENSA consistently finds solutions near
mization. We have demonstrated CONDENSA’s
the “knee” of the L-C accuracy curves, signify-
effectiveness and ease-of-use on a range of
ing the effectiveness of the DR-UCB acquisi-
state-of-the-art DNNs for image classification
tion function.
and language modeling, and achieved memory
2) Local minima/maxima is avoided while opti-
footprint reductions of up to 188
and runtime
mizing the objective function, demonstrating
throughput improvements of up to 2:59
using
that the UCB acquisition func-
at most ten samples per search.
tion for objective function opti-
mization is working as expected. This article has
presented
3) The knees of the D-C accuracy ACKNOWLEDGMENTS
curves occur at significantly CONDENSA, which is a
This work was supported in
flexible programming
lower sparsities; the L-C opti- part by DARPA under Contract
system for DNN
mizer, on the other hand is able HR0011-18-3-0007; in part by
compression and
to recover accuracies up to National Science Foundation
corresponding
much higher target sparsities. hyperparameter (NSF) under Award CCF-1704715;
optimization. and in part by a CIFAR AI Chair
Layerwise Compression Analysis award. Any opinions, findings,
In this section, we analyze how and conclusions or recommenda-
improving throughput using compression trans- tions expressed in this material are those of the
lates to execution time improvements for each author(s) and do not necessarily reflect the
layer on actual hardware. For this experiment, we views of the U.S. Government. Distribution State-
focus on VGG-19 on CIFAR-10, since it has a rela- ment “A” (Approved for Public Release, Distribu-
tively simple structure and is easy to analyze on a tion Unlimited).
layer-by-layer basis. We use filter pruning with a
target sparsity of 0.79 (found by the Bayesian opti-
mizer, as decribed in Table 1) for this experiment.
& REFERENCES
Figure 3 shows layer-by-layer mean runtimes col- 1. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual
lected over 100 runs using TensorRT (blue bars, learning for image recognition,” in Proc. IEEE Conf.
left y-axis), and compression ratios (green line, Comput. Vis. Pattern Recognit., 2016, pp. 770–778.

24 IEEE Micro
 2. K. Simonyan and A. Zisserman, “Very deep convolutional
networks for large-scale image recognition,” in Proc. Int.
Conf. Learn. Representations, 2015.
3. Y. He, J. Lin, Z. Liu, H. Wang, L.-J. Li, and S. Han,
“AMC: AutoML for model compression and
acceleration on mobile devices,” in Proc. 10th Eur.
Conf. Comput. Vis., 2018, pp. 784–800.
4. Y. He, X. Zhang, and J. Sun, “Channel pruning for
accelerating very deep neural networks,” in Proc. IEEE
Int. Conf. Comput. Vis., 2017, vol. 2, pp. 1398–1406.
5. N. Liu, X. Ma, Z. Xu, Y. Wang, J. Tang, and J. Ye,
“AutoCompress: An automatic DNN structured
pruning framework for ultra-high compression rates,”
in Proc. AAAI Conf. Artif. Intell., 2020, pp. 4876–4883.
6. V. Mnih et al., “Playing Atari with deep reinforcement
learning,” 2013, arXiv:1312.5602.
n and Y. Idelbayev, ““Learning-
7. M. A. Carreira-Perpina
compression” algorithms for neural net pruning,” in
Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2018,
pp. 8532–8541.
8. D. R. Jones, M. Schonlau, and W. J. Welch, “Efficient
global optimization of expensive black-box functions,”
J. Global Optim., vol. 13, no. 4, pp. 455–492, 1998.
9. N. Srinivas, A. Krause, S. M. Kakade, and M. Seeger,
“Gaussian process optimization in the bandit setting:
No regret and experimental design,” in Proc. 27th Int.
Conf. Mach. Learn., 2010.
10. A. Garg et al., “Tumor localization using automated
palpation with Gaussian process adaptive sampling,”
in Proc. IEEE Int. Conf. Autom. Sci. Eng., 2016,
pp. 194–200.
11. A. Krizhevsky, V. Nair, and G. Hinton, “The CIFAR-10
dataset,” [Online]. Avaliable: http://www. cs. toronto.
edu/kriz/cifar. html, vol. 55, 2014.
September/October 2020
12. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and
L. Fei-Fei, “ImageNet: A large-scale hierarchical image
database,” in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit., 2009, pp. 248–255.
13. S. Merity, C. Xiong, J. Bradbury, and R. Socher,
“Pointer sentinel mixture models,” 2016,
arXiv:1609.07843.
14. H. Yu, S. Edunov, Y. Tian, and A. S. Morcos, “Playing
the lottery with rewards and multiple languages:
Lottery tickets in RL and NLP,” 2019,
arXiv:1906.02768.
Vinu Joseph is currently working toward the Ph.D.
degree in computer science at the University of
Utah. Contact him at vinu@cs.utah.edu.
Ganesh L. Gopalakrishnan is a Professor of
computer science with the School of Computing, Uni-
versity of Utah. Contact him at ganesh@cs.utah.edu.
Saurav Muralidharan is a Senior Research Scientist
in the Programming Systems & Applications research
group with NVIDIA. Contact him at sauravm@nvidia.com.
Michael Garland is the Senior Director for Pro-
gramming Systems & Applications research with
NVIDIA. Contact him at mgarland@nvidia.com.
Animesh Garg is an Assistant Professor of computer
science with the University of Toronto, and a Faculty
Member with the Vector Institute, Canada. Contact him
at garg@cs.toronto.edu.
25
 Theme Article: Machine Learning for Systems

A Single-Shot
Generalized Device
Placement for Large
Dataflow Graphs
Yanqi Zhou, Sudip Roy,
Amirali Abdolrashidi, Daniel Lin-Kit Wong,
Peter Ma, Qiumin Xu, Azalia Mirhoseini, and
James Laudon
Google

Abstract—With increasingly complex neural network architectures and heterogeneous

device characteristics, finding a reasonable graph partitioning and device placement
strategy is challenging. There have been prior attempts at learned approaches for solving
device placement, these approaches are computationally expensive, unable to handle large
graphs consisting over 50 000 nodes, and do not generalize well to unseen graphs. To
address all these limitations, we propose an efficient single-shot, generalized deep RL
method (SGDP) based on a scalable sequential attention mechanism over a graph neural
network that is transferable to new graphs. On a diverse set of representative deep learning
models, our method on average achieves 20% improvement over human placement and 18%
improvement over the prior art with 15
faster convergence. We are the first to demonstrate
super human performance on 8-layer recurrent neural network language model and 8-layer
GNMT consisting of over 50 000 nodes, on 8-GPUs. We provide rationales and sensitivity
study on model architecture selections.

& NEURAL demonstrated rema-

NETWORKS HAVE
rkable scalability–improved performance can
usually be achieved by training a larger model
Digital Object Identifier 10.1109/MM.2020.3015188
Date of publication 7 August 2020; date of current version
1 September 2020.
on a larger dataset.6,7 Training such large models
efficiently while meeting device constraints, like
memory limitations, necessitate partitioning of
the underlying dataflow graphs for the models
across multiple devices. However, devising a
good partitioning and placement of the dataflow
graphs requires deep understanding of the
model architecture, optimizations performed by

26
0272-1732 ß 2020 IEEE Published by the IEEE Computer Society IEEE Micro
domain-specific compilers, as well as the device Instead of generating placement decisions one
characteristics, and is therefore extremely hard node a time,1,3,16 the policy network generates
even for experts. decisions for the entire graph in a single shot
Graph partitioning and device placement can fashion. In order to handle large graphs consist-
be specified through a programming interface ing of over 50 000 nodes, we use a Transformer-
or through compiler optimizations. ML practi- XL based on segmented recurrent attention10,18
tioners often rely on their domain knowledge to that partitions the input sequences and gener-
determine a reasonable partitioning and device ates placement decisions one sequence each of
mapping for computational graphs. For example, the time while using caching to track interse-
programmers can manually assign devices to quence dependencies. The segmented Trans-
operations through a programming interface former-XL removes any hard constraints such
such as Tensorflow and Mesh-Tensorflow. How- as hierarchical grouping of operations3 or colo-
ever, relying solely on the model architecture cation heuristics to reduce the placement com-
while ignoring the effect of the partitioning on plexity.1 For generalization, we apply a graph
subsequent compiler optimizations like opera- neural network (GNN) to encode operation fea-
tion scheduling can lead to suboptimal place- tures and dependencies into a trainable graph
ments and consequently under-utilization of representation, and learn the graph representa-
available devices. Alternatively, a compiler can tion end-to-end with the placement policy
apply heuristics to annotate graphs and assign decisions.
devices to Tensors or operations. The heuristics Both our graph-embedding network and place-
not only lead to suboptimal configurations but ment network can be jointly trained in an end-to-
also need to be constantly modified to accom- end fashion using a
modate new cases arising from previously supervised reward,
We empirically show
unseen model architectures. without the need to
that the network learns
The goal of an automatic device placement is manipulate the loss
flexible placement
to find the optimal assignment of operations to policies at a per-node
functions at multiple
devices such that the end-to-end execution time granularity and can levels. We empiri-
for a single step is minimized and all device con- scale to problems over cally show that the
straints like memory limitations are satisfied. 50 000 nodes. By network learns flexi-
Since this objective function is nondifferentiable, transferring the learned ble placement poli-
prior approaches1,3,11 have explored solutions graph embeddings cies at a per-node
based on reinforcement learning (RL). However, and placement granularity and can
these RL policies are impractical to be used in a policies, we are able to scale to problems
real production compiler for several reasons. achieve faster over 50 000 nodes.
First, they are designed for small to medium sized convergence and thus By transferring the
use less resources to
computation graphs and do not demonstrate learned graph emb-
obtain high-quality
strong performance on large graphs where device eddings and place-
placements.
placement is truly needed. Second, these RL poli- ment policies, we are
cies are usually not transferable and require able to achieve faster
training a new policy from scratch for each indi- convergence and thus use less resources to obtain
vidual graph. This makes such approaches high-quality placements.
impractical due to the significant amount of com- Our contributions can be summarized as
pute required for the policy search itself, at times follows.
offsetting gains made by the reduced step time.
In this article, we propose an end-to-end, 1) An end-to-end deep RL framework to auto-
single-shot deep RL method for device place- matically learn graph partitioning and device
ment (SGDP) where the learned policy is gener- placement in a single-shot fashion. Our
alizable to new graphs. For the speed of method is demonstrated 15
faster than the
training, we propose a single-shot placement prior SoTA based on a hierarchical LSTM
using a re-engineered Transformer-XL network. model.1,3

September/October 2020
27
 Machine Learning for Systems
2) A scalable placement network with an effi-
cient recurrent attention mechanism, which
eliminates the need for an explicit grouping
stage before placement. Our method handles
large graphs consisting over 50 000 nodes
and is the first to demonstrate superhuman
placement performance on large problems
such as 8-layer GMNT and 8-layer recurrent
neural network language model (RNNLM).
3) An end-to-end device placement network
that can generalize to arbitrary and held-out
graphs. This is enabled by jointly learning a
transferable GNN along with the placement
network.
4) Superior empirical performance over a wide
set of important workloads in computer vision,
speech, and NLP (InceptionV3, AmoebaNet,
RNNs, GNMT, Transformer-XL,10 WaveNet).
5) Detailed rationales and sensitivity studies on
model architecture selections for the policy
network. Compared against LSTMs, MLPs,
and graph attention networks (GANs).20
RELATED WORK
Model-Level Parallelism
Model-level parallelism partitions a neural net-
work model among multiple devices and each
device is responsible for the weights updates of
the assigned operations or layers. Model-level
parallelism enables training large models exceed-
ing the size constraint of the device memory.
There are different forms of model-level paral-
lelism and many of them are supported at the pro-
gramming language level. Mesh-TensorFlow13 is a
language that built on top of Tensorflow that pro-
vides a general class of distributed tensor compu-
tations. While data-parallelism can be viewed as
splitting tensors and operations along the “batch”
dimension, in Mesh-TensorFlow the user can
specify any tensor-dimensions to be split across
any dimensions of a multidimensional mesh of
processors. FlexFlow2 introduces SOAP, a more
comprehensive search space of parallelization
strategies for DNNs which allows parallelization
of a DNN in the Sample, Operator, Attribute, and
Parameter dimensions. It uses guided random-
ized search to find a parallelization strategy.
GPipe12 proposed pipeline parallelism, by
automatically splitting a minibatch of training
28
examples into smaller micro-batches. By pipe-
lining the execution across micro-batches, accel-
erators can operate in parallel. PipeDream19
introduces pipeline parallelism with more flexi-
bility, allowing gradient updates of multiple mini-
batches to happen in parallel. However, this
introduces the staleness and consistency issue
for weigh updates. In addition, both GPipe and
PipeDream partition a model at the granularity of
layers instead of operations. Instead of proposing
different parallelism strategies or programming
primitives, our work focuses on a general deep
RL algorithmic solution for automating device
placement at operation granularity.
Automatic Device Placement
RL has been used for device placement of a
given dataflow graph1 and demonstrated runtime
reduction over human crafted placement and
conventional heuristics. For improved scalability,
a hierarchical device placement (HDP) strategy3
has been proposed that clusters operations into
groups before placing the operation groups onto
devices. Spotlight11 applies proximal policy opti-
mization (PPO) and cross-entropy minimization
to lower training overhead. Both HDP and Spot-
light rely on LSTM controllers that are difficult to
train and struggle to capture very long-term
dependencies over large graphs. In addition, both
methods are restricted to process only a single
graph at a time, and cannot generalize to arbitrary
and held-out graphs. Placeto16 represents the first
attempt to generalize device placement using a
graph embedding network. But like HDP, Placeto
also relies on hierarchical grouping and only gen-
erates placement for one node at each time step.
Our approach leverages a recurrent attention
mechanism and generates the whole graph place-
ment at once. This significantly reduces the train-
ing time for the controller. We also demonstrate
the generalization ability of SGDP over a wider set
of important workloads.
Compiler Optimization
REGAL8,9 uses deep RL to optimize the execu-
tion cost of computation graphs in a static com-
piler. The method leverages the policy’s ability
to transfer to new graphs to improve the quality
of the genetic algorithm for the same objective
budget. However, REGAL does not show strong
IEEE Micro
Figure 1. Overview of SGDP: An end-to-end placement network that combines graph embedding and
sequential attention. N: Number of nodes. h: Hidden size. d: Number of devices.
empirical results on large graphs, especially
graphs consisting of over 50 000 nodes such as
8-layer GNMT and 8-layer RNNLM. In addition,
REGAL relies on a performance model to app-
roximate the rewards, while we demonstrate
superior performance using real machine meas-
urements in an online learning fashion.
END-TO-END PLACEMENT POLICY
Given a dataflow graph GðV; EÞ where V rep-
resents atomic computational operations (ops)
and E represents the data dependence, our goal
is to learn a policy p : G 7! D that assigns a place-
ment D 2 D for all the ops in the given graph G 2
G, to maximize the reward rG;D defined based on
the runtime. D is the allocated devices that can
be a mixture of CPUs and GPUs. In this article,
we represent policy pu as a neural network
parameterized by u.
Unlike prior works that focus on a single
graph only, the RL objective in GDP is defined to
simultaneously reduce the expected runtime of
the placements over a set of N dataflow graphs
JðuÞ ¼ EGG;Dpu ðGÞ ½rG;D 
1X
 EDpu ðGÞ ½rG;D : (1)
N G
In the following, we refer to the case when
N ¼ 1 as individual training and the case when
N > 1 as batch training. We optimize the objec-
tive above using PPO15 for improved sample
efficiency.
September/October 2020
Figure 1 shows an overview of the proposed
end-to-end device placement network. Our pro-
posed policy network pu consists a graph
embedding network that learns the graphical
representation of any dataflow graph, and a
placement network that learns a placement
strategy over the given graph embeddings. The
two components are jointly trained in an end-to-
end fashion. The policy pðajGÞ is applied to make
a set of decisions at each node. These decisions,
denoted as av for each v 2 V across all nodes,
form one action a ¼ fav2V g. One decision corre-
sponds to playing one arm of a multibandit prob-
lem, and specifying the entire a corresponds to
playing several arms together in a single shot.
Note the architecture is designed to be invariant
over the underlying graph topology, enabling us
to apply the same learned policy to a wide set of
input graphs with different structures.
Graph Embedding Network
We leverage GNNs4,5 to capture the topologi-
cal information encoded in the dataflow graph.
Most graph embedding frameworks are inher-
ently transductive and can only generate embed-
dings for a given fixed graph. These transductive
methods do not efficiently extrapolate to handle
unseen nodes (e.g., in evolving graphs), and can-
not learn to generalize to unseen graphs.
GraphSAGE4 is an inductive framework that lev-
erages node attribute information to efficiently
generate representations on previously unseen
data. While our proposed framework is generic,
29
 Machine Learning for Systems
Figure 2. Normalized runtime to SGDP-one (the
lower the better). Comparison of different policy
network architectures: MLPs consists of graph
embedding network with MLPs. GATs incorporates
an attention of neighbor nodes into a GNN. SGDP-
one is our method. Results for GATs on all 8-layer
input graphs, GNMT, and AmoebaNet are missing
as GATs fails to generate valid placements.
we adopt the feature aggregation scheme pro-
posed in GraphSAGE to model the dependencies
between the operations and build a general, end-
to-end device placement method for a wide set
of dataflow graphs.
In SGDP, nodes and edges in the dataflow
graph are represented as the concatenation of
their meta features (e.g., operation type, output
shape, adjacent node ids) and are further
encoded by the graph embedding network into a
trainable representation. The graph embedding
process consists of multiple iterations, and the
computation procedure for the lth iteration can
be outlined as follows.
First, each node v 2 V aggregates the feature
representations of its neighbors, fhðlÞ u ; 8u 2
ðlÞ
N ðvÞg, into a single vector hN ðvÞ . This aggrega-
tion outcome is a function of all previously gen-
erated representations, including the initial
representations defined based on the input node
features. In this article, we use the following
aggregation function with max pooling:
ðlÞ
hN ðvÞ ¼ maxðsðW ðlÞ hðlÞ ðlÞ
u þ b Þ; 8u 2 N ðvÞÞ
(2)
ðlÞ ðlÞ
where ðW , b Þ define an affine transform and s
stands for the sigmoid activation function. We
then concatenate the node’s current representa-
tion, hðlÞ
v , with the aggregated neighborhood vec-
ðlÞ
tor, hN ðvÞ , and feed this concatenated vector
30
through a fully connected layer f ðlþ1Þ
ðlÞ
hðlþ1Þ
v ¼ f ðlþ1Þ ðconcatðhðlÞ
v ; hN ðvÞ ÞÞ: (3)
Different from GraphSAGE, parameters in our
graph embedding network are trained jointly
with a placement network via stochastic gradi-
ent descent with PPO, in a supervised fashion, as
described in the “End-to-End Placement Policy”
section. That is, we replace the unsupervised
loss with our task-specific objective.
Placement Network
The GNN works as a feature aggregation net-
work that learns a trainable feature representa-
tion for the computational graph, we still need a
policy network that produces actions on a per
node basis. Given hv ’s, the policy network produ-
ces av ’s through conditionally independent pre-
dictions, where the prediction for one node v
does not depend on the prediction of other nodes
Y Y
pðajGÞ ¼ pðav jGÞ ¼ pðav jfðhv ÞÞ: (4)
v v
Next, we discuss the selection of a proper neu-
ral network model to create a policy network.
Multilayer Perceptron (MLPs) While f can
be represented using MLPs, where the MLPs is
shared across all nodes for prediction the place-
ment output distributions. However, MPLs lack a
dependence tracking mechanism across nodes.
In practise, the placement of one node can be
determined by the placement of another node,
where the placed node may consume a large size
of data produced by the other node.
LSTM The conventional LSTM models proposed
for language tasks usually target a shorter
sequence length. For example, in a language
task, a typical sequence length is between a few
hundred to a thousand. However, in the device
placement problem, models truly needs device
placement can consists of over 50 000 of nodes.
HDP3 has been proposed to address this issue,
however, the proposed grouper network comes
with limited flexibility and generality. For exam-
ple, the grouper network leverages an aggre-
gated feature representation by averaging
feature vectors for nodes within the same group.
IEEE Micro
The nondifferentiable grouping procedure pre-
vents training the graph-embedding and place-
ment networks end-to-end.
Graph Attention An attention network can
learn internode dependence and the relative
importance of dependencies across an entire
graph. An intuitive way is to incorporate an atten-
tion mechanism into the GNN to enable nodes to
attend over their neighborhood’s features, follow-
ing a self-attention strategy. GATs20 has several
strengths: first, it is efficient and is parallelizable
across node-neighbor pairs; second, it can specify
arbitrary weights to the neighbors; third, the
model can generalize to completely unseen
graphs. However, GATs only supports local atten-
tion, as contrary to the long-term global attention
using a conventional Transformer network. More-
over, it has limited scalability that cannot handle
large graphs consisting of over 10 k’s of nodes.
Our Method As we increase the input graph
size, the complexities of a GNN and an attention
network can scale up. However, due to the reduc-
tion used in the aggregation layers in GraphSAGE,
the complexity usually scales linearly with the
number of nodes. But for an attention network,
the complexity is OðN 2 Þ. Therefore, designing a
more scalable attention network is critical to
large input graphs.
We propose to use a transformer-based atten-
tive network to generate operation placements
in an end-to-end fashion. As the graph embed-
ding already contains spatial (topological) infor-
mation for each node, we remove the positional
embedding in the original transformer to pre-
vent the model from overfitting node identifica-
tions. To capture long-term dependencies
efficiently among a large set of nodes, we adopt
segment-level recurrence introduced in Trans-
former-XL,10,17 where hidden states computed
for the previous set of nodes are cached (with
gradient flows disabled) and reused as an
extended context during the training of the next
segment. This reduces the complexity to
OðN 2 =kÞ, where k is the number of segments.
Besides achieving extra long context than a
GAN, we empirically find the segment-level recur-
rent attention much faster than a conventional
LSTM-based GNMT model. In our experimental
September/October 2020
evaluation, we compare both the performance
and speedup of our placement network with that
of the LSTM-based HDP.
To improve policy optimization for the big
search space [OðdN Þ], where d is the number of
devices and N is the number of nodes, we apply
an additional mask attention to the last layer of
feature map generated by the recurrent atten-
tion policy network. The generated actions mask
is position-wise multiplied with the actions to
selectively choose nodes to place. Intuitively,
this enables selecting important ops in the graph
to change placements while minimizing the cuts
for the entire graph.
EXPERIMENT
Experimental Setup
Workloads: We evaluate SGDP using the
computational graphs of six diverse architectures
from different domains. To create a larger set of
workloads, we vary architectural parameters like
the number of layers for each of these workloads.
All our workloads are implemented in TensorFlow.
Further details about the graphs is in Appendix A.
Runtime Measurement: For placement task,
where TensorFlow provides an API for device
assignment, our experiments are evaluated on
actual hardware with configuration of one Intel
Broadwell CPU and up to eight Nvidia P100 GPUs.
Baselines: We choose three different base-
lines against which we compare the performance
of SGDP along various metrics in the “Perfor-
mance on Individual Graphs” section. They are
the default heuristics-based optimizations used
in TensorFlow (METIS), a human-expert solution,
and finally solutions found using a learning
based strategy like HDP.3 For sensitivity study
in Section refsubsec:sensitivity, we compare
against MPLs and GATs.20
Performance on Individual Graphs
We evaluate SGDP by training the model sepa-
rately on six important deep learning computa-
tion graphs, including RNN Language Modeling,
GNMT, Transformer-XL, Inception, AmoebaNet,
and WaveNet. We name this approach SGDP-one.
Since TensorFlow provides an API for assigning
operation placement, all reported measurements
for placement task are on real hardware. As
31
 Machine Learning for Systems

Table 1. Runtime comparison between SGDP-one, human expert, TensorFlow METIS, and HDP on six graphs (RNNLM, GNMT,
Transformer-XL, Inception, AmoebaNet, and WaveNet). Search speedup is the policy network training time speedup compared to
HDP (reported values are averages of six runs).

SGDP-one HP METIS HDP Runtime speedup over HP / Search speedup over

Model (#devices)
(s) (s) (s) (s) HDP HDP
2-layer RNNLM (2) 0.173 0.192 0.355 0.191 9.9% / 9.4% 2.95x

4-layer RNNLM (4) 0.210 0.239 0.503 0.251 13.8% / 16.3% 1.76x

8-layer RNNLM (8) 0.320 0.332 OOM 0.764 3.8% / 58.1% 27.8x

2-layer GNMT (2) 0.301 0.384 0.344 0.327 27.6% / 14.3% 30x

4-layer GNMT (4) 0.350 0.469 0.466 0.432 34% / 23.4% 58.8x

8-layer GNMT (8) 0.440 0.562 OOM 0.693 21.7% / 36.5% 7.35x

2-layer Transformer-XL
0.223 0.268 0.37 0.262 20.1% / 17.4% 40x
(2)

4-layer Transformer-XL
0.230 0.27 OOM 0.259 17.4% / 12.6% 26.7x
(4)

8-layer Transformer-XL
0.350 0.46 OOM 0.425 23.9% / 16.7% 16.7x
(8)

Inception (2) b32 0.229 0.312 OOM 0.301 26.6% / 23.9% 13.5x

Inception (2) b64 0.423 0.731 OOM 0.498 42.1% / 29.3% 21.0x

AmoebaNet (4) 0.394 0.44 0.426 0.418 26.1% / 6.1% 58.8x

2-stack 18-layer WaveNet

0.317 0.376 OOM 0.354 18.6% / 11.7% 6.67x
(2)

4-stack 36-layer WaveNet

0.659 0.988 OOM 0.721 50% / 9.4% 20x
(4)

GEOMEAN - - - - 20.5% / 18.2% 15x

shown in Table 1, SGDP-one consistently outper-
forms human expert placement (HP), TensorFlow
METIS placement, and HDP. Overall, SGDP-one
achieves on average 20.5% and 18.2% run time
reduction across the evaluated 14 graphs, com-
pared to HP and HDP, respectively. Importantly,
with the efficient end-to-end single-shot place-
ment, SGDP-one has a 15
speedup in conver-
gence time of the placement.
Scalability Analysis: SGDP is designed in a way
to scale up to extremely large graphs, consisting of
over 80 000 nodes (8-layer GNMT). Therefore,
unlike any of the prior works including HDP,3
REGAL,9 and Placeto,16 we can demonstrate super
human performance on large graphs such as 8-
layer GNMT (21.7%/36.5% better than HP/HDP)
and 8-layer RNNLM (3.8%/58.1% better than HP/
HDP). For all of the related SoTA work, Placeto16
and REGAL9 do not provide any results on 8-layer
RNNLM or 8-layer GNMT (more than 50 000 nodes).
HDP3 reports inferior performance on 8-layer
RNNLM and 8-layer GNMT than human placement.
Sensitivity Study on Model Architectures
We compare our method with two alternative
architectures (MLPs and GATs), as explained in
“Placement Network” section. An LSTM-based
HDP has been compared in Table 1 and we will
leave it out in this section. SGDP-one consis-
tently outperforms both MLPs and GATs by an
average of 10% and 7%. (We only include valid
placements for GATs). GATs fails to generate
valid placements for large graphs consisting of
over 10 000 nodes, such as 8-layer RNNLM, 2-
layer GNMT, 8-layer Transformer-XL, and Amoe-
baNet. The results imply that SGDP yields better
performance with an attention network, as com-
pared to MLPs, and provides better scalability
with a decoupled segmented attention network,
as compared to GATs.

32 IEEE Micro
Generalization
Table 2. Runtime comparison on SGDP-batch versus SGDP-one.
SGDP enables the training of multiple heteroge-
neous graphs in a single batch (GDP-batch). We Model Speedup Model Speedup
empirically show that GDP-batch generates better 2-layer RNNLM 0 Inception 0
placements for many workloads such as Trans-
4-layer RNNLM 5% AmoebaNet 5%
former-XL (7.6%), WaveNet (15%), and 8-layer
4-stack 36-layer
GNMT (8%). Table 2 compares the run time of 11 2-layer GNMT 0 3.3 %
WaveNet
tasks using SGDP-batch. SGDP-batch yields
2-stack 18-layer
slightly better runtime compared to SGDP-one in 4-layer GNMT 0 15%
WaveNet
majority of the tasks, while being only slightly
2-layer 8-layer Transformer-
worse on AmoebaNet. Compared to training 7.6% 1.5%
Transformer-XL XL
graphs separately, SGDP-batch reduces network
parameters and enables transfer learning among 4-layer
3%
Transformer-XL
different graphs.
We test on unseen graphs from different work-
loads by pretraining the SGDP on different subsets super human performance on large graphs con-
of five workloads, excluding the entire workload of sisting of over 50 000 nodes.
the unseen graph. For example, for an RNNLM
input graph, all RNNLM models are excluded from APPENDIX A
the pretraining dataset. During pretraining, we per-
INPUT GRAPHS
turb the number of layers, hidden size, and batch
size of the input graphs to augment the data and We used a variety of widely used workloads from
computer vision, speech, and NLP. In this sec-
add more randomness to the input data. SGDP-
tion, we give a detailed explanation on the
zeroshot directly runs inference on the pre-trained
selected models and hyperparameters.
SGDP model to generate placements for the target
graph. SGDP-finetune further trains the pre-trained
SGDP model for an additional 50 training steps and Inception-V3
generates placements using the fine-tuned SGDP Inception-V3 is a multibranch convolutional net-
model. We find that SGDP-finetune almost matches work used for a variety of computer vision tasks,
the performance of SGDP-one, degrading the place- including classification, recognition, or generation.
ment runtime on average by only 1.2% compared The network consists of blocks made of multiple
to SGDP-one and outperforms both human place- branches of convolutional and pooling operations.
Within a block, the branches of ops can be executed
ment and HDP significantly. SGDP-zeroshot
in parallel. However, the model is mostly sequential
completely eliminates the training for the target
as the outputs of each block are concatenated
unseen graphs, while being only 3.7% worse on together to form the input to the next block. We use
average than SGDP-one and being over 10% better a batch size of 64. The TensorFlow graph of this
than human placement. This indicates that both model contains 24 713 operations.
graph embedding and the learned policies transfer
and generalize to the unseen data.
AmoebaNet
AmoebaNet is an automatically designed neural
CONCLUSION network that yields SOTA performance on Image-
We propose an efficient single-shot, general- Net. Similar to Inception-V3, it contains Inception-
ized deep RL method (SGDP) and demonstrate like blocks called cells, which receives a direct
superior performance on a wide set of represen- input from the previous cell and a skip input from
tative deep learning models, including Inception- the cell before it. We use a batch size of 64. The
v3, AmoebaNet, RNNLM, GNMT, Transformer-XL, TensorFlow graphs contains 9430 operations.
and WaveNet. Our method on average achieves
20% improvement over human experts and 18% Recurrent Neural Network Language Model
improvement over the prior art with 15
faster RNNLM is made of many LSTM cells organized in
convergence, being the first to demonstrate a grid structure. The processing of each LSTM

September/October 2020
33
 Machine Learning for Systems

Table 3. Hyperparameters for policy network. gs layers: longer than vanilla Transformers. We use a
GraphSAGE layers, gs knn: GraphSAGE maximum Transformer-XL with batch size of 64, sequence
neighbors, trf d model: Dimension of the Transformer- length of 256, segment length of 64, model hid-
XL model, trf n head: Number of attention heads, den dimension of 500 and feed forward hidden
trf layers: Number of Transformer-XL layers, dimension of 1000, 10 heads, and head dimen-
trf d heads: Dimension of each attention head,
trf d inner: Dimension of inner hidden size in
sion of 50. The 2-layer Transformer-XL contains
positionwise feed-forward. 2618 operations. The number of ops grow
roughly proportional with the number of layers.
Parameters Value Parameters Value
gs layers 4 gs dim 128 WaveNet
gs knn 5 trf layers 2 WaveNet is a generative model for speech synthe-
sis. The model is fully probabilistic and autore-
trf d model 128 trf n head 5
gressive, with the predictive distribution for each
trf d head 25 trf d inner 256 audio sample conditioned on all previous ones.
We use a WaveNet model with batch size 64 and a
receptive field size of 2048 (9-layers per stack). An
Table 4. Hyperparameters for PPO. 5-stack WaveNet contains 4374 operations and a
10-stack WaveNet contains 8516 operations.
Parameters Value Parameters Value
learing rate 0.5 num of rollouts 800 APPENDIX B
minibatches 40 epochs 20
HYPERPARAMETERS
We list out all the selected hyperparameters in
epsilon 0.2 entropy 0.5
our experiments for reproducibility in Tables 3
optimizer Adam and 4.

cell only depends on the results of 2 other cells ACKNOWLEDGMENTS

(from the previous layer, and from the previous
time step), which make the concurrent execu-
tion of many LSTM cells possible given enough
hardware resources. We use batch size 64 and a
hidden size of 2048. The corresponding Tensor-
Flow graph contains 9021 operations for a 2-layer
model. The number of ops grow roughly propor-
tional with the number of layers.
GNMT
Neural machine translation with attention mech-
anism has an architecture similar to that of
RNNLM, but its many hidden states make it far
more computationally expensive than RNNLM.
We use batch size 64. The original 2-layer
encoder-decoder consisting of 28 044 opera-
tions. An extended 4-layer version consisting of
46 600 operations, An even larger 8-layer version
consisting of 83 712 operations.
This work was done during internship at
Google.
& REFERENCES
1. A. Mirhoseini et al., “Device placement optimization
with reinforcement learning,” in Proc. Int. Conf. Mach.
Learn., 2017.
2. Z. Jia et al., “Beyond data and model parallelism for
deep neural networks,” in Proc. 35th Int. Conf. Mach.
Learn., 2018.
3. A. Mirhoseini et al., “A hierarchical model for device
placement,” in Proc. Int. Conf. Learn. Representations,
2018.
4. W. L. Hamilton et al., “Inductive representation
learning on large graphs,” in Proc. Conf. Neural Inf.
Process. Syst., 2017.
5. K. Xu et al., “How powerful are graph neural
networks?,” in Proc. Int. Conf. Learn. Representations,

Transformer-XL
Transformer-XL10 is an modified version of
Transformer18 that supports segment-level
recurrence and a novel positional encoding
scheme. This innovation enables learning depen-
dence that is 80% longer than RNNs, and 450%
2019.
6. H. Joel et al., “Deep learning scaling is predictable,
empirically,” 2017, arXiv:1712.00409.
7. S. Noam et al., “Outrageously large neural networks:
The sparsely-gated mixture-of-experts layer,” in Proc.
Int. Conf. Learn. Representations, 2017.

34 IEEE Micro
 8. A. Paliwal et al., “REGAL: Transfer learning for fast
optimization of computation graphs,” Knowl. Discovery
Database, 2019.
9. A. Paliwal et al., “Reinforced genetic algorithm learning
for optimizing computation graphs,” in Proc. Int. Conf.
Learn. Representations, 2020.
10. Z. Dai et al., “Transformer-XL: Attentive language
models beyond a fixed-length context,” in Proc. 57th
Annu. Meeting Assoc. Comput. Linguistics, 2019,
pp. 2978–2988.
11. I. Sutskever et al., “Spotlight: Optimizing device
placement for training deep neural networks,” in Proc.
Conf. Neural Inf. Process. Syst., 2018.
12. Y. Huang et al., “GPipe: Efficient training of giant
neural networks using pipeline parallelism,” in Proc.
NeurIPS, 2019.
13. N. Shazeer et al., “Mesh-tensorflow: Deep learning for
supercomputers,” in Proc. NeurIPS, 2018.
14. B. Cheung et al., “Superposition of many models into
one,” in Proc. NeurIPS, 2019.
15. J. Schulman et al., “Proximal policy optimization
algorithms,” 2017, arXiv:1707.06347.
16. R. Addanki et al., “Placeto: Learning generalizable
device placement algorithms for distributed machine
learning,” in Proc. NuerIPS, 2019.
17. Z. Dai, “Improving deep generative modeling with
applications,” 2019.
18. V. Ashish et al., “Attention is all you need,” in Proc.
Conf. Neural Inf. Process. Syst., 2017.
19. D. Narayanan et al., “PipeDream: Generalized pipeline
parallelism for DNN training,” in Proc. Symp. Oper.
Syst. Principles, 2019.
kovic
20. P. Velic  et al., “Graph attention networks,” in
Proc. Int. Conf. Learn. Representations, 2018.
Yanqi Zhou is currently a Research Scientist with
Google Brain. She works on generalizing machine
learning to optimize systems problems, including
compiler graph optimizations and hardware acceler-
ator design. In addition, she builds large-scale deep
learning models for speech and language tasks.
Zhou received her Ph.D. degree from Princeton Uni-
versity, working on configurable computer architec-
tures and resource provisioning for clouds. Contact
her at yanqiz@google.com.
September/October 2020
Sudip Roy is currently a Senior Research Scien-
tist with Google AI. He is interested in design and
development of systems for machine learning and
also in applying machine learning to solve optimi-
zation problems that arise in systems. He has
worked on a variety of problems in this space
including infrastructure for large-scale distributed
machine learning, data management solutions for
managing data lakes, using reinforcement learning
to solve optimization problems in machine learning
compilers, and geo-replicated transaction proc-
essing systems. Roy received a Ph.D. degree in
computer science from Cornell University. Contact
him at sudipr@google.com.
Amirali Abdolrashidi is currently working toward
the Ph.D. degree in computer science and engineer-
ing from the University of California, Riverside, where
he works on speeding up data-dependent workloads
on GPU architectures. Abdolrashidi received the
M.S. degree in electrical engineering from New York
University, in 2014, and during his Software Engi-
neering Internship with Google, he worked to
improve the performance of deep learning applica-
tions through prioritized fusion of operations. Contact
him at abdolrashidi@gmail.com.
Daniel Lin-Kit Wong is currently working toward
the Ph.D. degree with the Computer Science
Department, Carnegie Mellon University. He is a
systems builder and hacker with a focus on systems
design and distributed systems. His research focus
has been machine learning for caching. Contact
him at wonglkd@gmail.com.
Peter Ma is currently a Software Engineer with Goo-
gle, and he works on machine learning accelerator
architecture and machine learning platforms perfor-
mance. Ma received the Ph.D. degree in mechanical
engineering with a Ph.D. Minor in computational and
mathematical engineering from Stanford University.
Contact him at pcma@google.com.
Qiumin Xu is currently a Senior Software Engineer
with Google Brain, working on the performance of
machine learning accelerators. Xu received the
Ph.D. degree in electrical engineering from the
University of Southern California. Contact her at
qiuminxu@google.com.
35
 Machine Learning for Systems
Azalia Mirhoseini is currently a Senior Research
Scientist with Google Brain, where she works on
deep reinforcement learning based approaches to
solve problems in computer systems. Mirhoseini
received the Ph.D. degree in electrical and computer
engineering from Rice University. She was the recipi-
ent of a number of awards, including the MIT Tech-
nology Review 35 under 35 award, the Best Ph.D.
Thesis Award at Rice, and a Gold Medal in the
National Math Olympiad in Iran. Her work has been
covered in various media outlets including MIT Tech-
nology Review and IEEE Spectrum. Contact her at
azalia@google.com.
36
James Laudon is currently an Engineering Director
with Google Brain. His research interests focus on hard-
ware and software co-design for high-performance sys-
tems and he is currently working on domain-specific
computer architectures for machine learning. Before
joining Google Brain, he was Founder and Site Director
for the Google Madison office. He has contributed to the
architecture and implementation of multiple computer
systems including the Stanford DASH, SGI Origin 2000,
and Sun UltraSPARC T1. Laudon received the B.S.
degree in electrical engineering from the University of
Wisconsin—Madison and the M.S. and Ph.D. degrees
in electrical engineering from Stanford University.
Contact him at jlaudon@google.com.
IEEE Micro
 Theme Article: Machine Learning for Systems

RELEQ : A Reinforcement
Learning Approach for
Automatic Deep
Quantization of Neural
Networks
Ahmed T. Elthakeb, Prannoy Pilligundla, Hadi Esmaeilzadeh
and Fatemehsadat Mireshghallah University of California San Diego
University of California San Diego
Amir Yazdanbakhsh
Google Brain

Abstract—Deep Quantization (below eight bits) can significantly reduce the DNN
computation and storage by decreasing the bitwidth of network encodings. However,
without arduous manual effort, this deep quantization can lead to significant accuracy
loss, leaving it in a position of questionable utility. We propose a systematic approach to
tackle this problem, by automating the process of discovering the bitwidths through an
end-to-end deep reinforcement learning framework (RELEQ). This framework utilizes the
sample efficiency of proximal policy optimization to explore the exponentially large space
of possible assignment of the bitwidths to the layers. We show how RELEQ can balance
speed and quality, and provide a heterogeneous bitwidth assignment for quantization of a
large variety of deep networks with minimal accuracy loss (
0.3% loss) while minimizing
the computation and storage costs. With these DNNs, RELEQ enables conventional
hardware and custom DNN accelerator to achieve 2:2 speedup over 8-bit execution.

Digital Object Identifier 10.1109/MM.2020.3009475

Date of publication 15 July 2020; date of current version
1 September 2020.

September/October 2020 Published by the IEEE Computer Society 0272-1732 ß 2020 IEEE
37
 Machine Learning for Systems

& DEEP NEURAL NETWORKS (DNNs) have made

quantization below eight bits is considered. For
waves across a variety of domains,1 however their example, ResNet-20 exposes a hyper-parameter
compute efficiency has become a major constraint space of size 8l ¼ 820 > 1018 , where l ¼ 20 is the
in unlocking further applications and capabilities. number of layers and 8 is the possible bitwidths.
To this end, quantization of neural networks pro- This exponentially large hyperparameter space
vides a path forward as it reduces the bitwidth of grows with the number of layers making it imprac-
operations and memory footprint. For example, in tical to exhaustively assess and determine the bit-
many scenarios, the bottleneck of running DNNs width for each layer.
is in transferring the weights and data between We develop an end-to-end framework, dubbed
main memory and compute cores. Using 8-bit inte- RELEQ, which exploits the sample efficiency of
ger rather than 32-bit, we instantly speed up the the proximal policy optimization4 to explore
memory transfer by 4. the quantization hyper-parameter space. The RL
Albeit alluring, quantization can lead to signifi- agent starts from a full-precision previously
cant accuracy loss if not employed trained model and learns the sen-
with diligence. To that end, two fun- sitivity of final classification accu-
We develop an end-to-
damental problems need to be racy with respect to the bitwidth
end framework, dubbed
addressed: 1) developing learning
RELEQ, which exploits of each layer, determining its bit-
techniques that can perform quan- the sample efficiency of width while keeping classification
tized training of DNNs; and 2) the proximal policy accuracy almost intact. Observ-
designing algorithms that identify optimization to explore ing that the quantization bitwidth
appropriate bitwidth per-layer while the quantization hyper- for a given layer affects the accu-
preserving accuracy. This article parameter space. racy of subsequent layers, our
takes on the second challenge as framework implements a long
there are inspiring efforts that have short-term memory (LSTM)-based RL framework,
developed techniques for quantized training.2,3 which enables selecting bitwidths with the con-
However, this possibility (discovering bit- text of previous layers’ bitwidths. Rigorous evalu-
widths) is manually laborious as to preserve accu- ations with a variety of networks (AlexNet, CIFAR,
racy, the bitwidth varies across individual layers LeNet, SVHN, VGG-11, ResNet-20, and MobileNet)
and different DNNs.2,3 Each layer has a different show that RELEQ can effectively find heteroge-
role and unique properties in terms of weight nous bitwidths with minimal accuracy loss
distribution; hence, displays different sensitivity (
0.3% loss) while minimizing the computation
toward quantization. Nonetheless, considering and storage cost. The results (see Table 1) show
layerwise quantization opens a rather exponen- that there is a high variance in bitwidths across
tially large hyperparameter space, specially when the layers of these networks. With the seven

Table 1. Benchmark DNNs and their deep quantization with RELEQ.

Average Accuracy Loss

Network Dataset Quantization Bitwidths
Bitwidth (%)
AlexNet ImageNet {8, 4, 4, 4, 4, 4, 4, 8} 5 0.08

SimpleNet CIFAR10 {5, 5, 5, 5, 5} 5 0.30

LeNet MNIST {2, 2, 3, 2} 2.25 0.00

{8, 5, 6, 6, 4, 4, 7, 8, 4, 6, 8, 5, 5, 8, 6, 7, 7, 7, 6, 8, 6, 8, 8, 6, 7, 5, 5,
MobileNet ImageNet 6.43 0.26
7, 8, 8}

ResNet-20 CIFAR10 {8, 2, 2, 3, 2, 2, 2, 3, 2, 3, 3, 3, 2, 2, 2, 3, 2, 2, 2, 2, 2, 8} 2.81 0.12

10-Layers SVHN {8, 4, 4, 4, 4, 4, 4, 4, 4, 8} 4.80 0.00

VGG-11 CIFAR10 {8, 5, 8, 5, 6, 6, 6, 6, 8} 6.44 0.17

VGG-16 CIFAR10 {8, 8, 8, 6, 8, 6, 8, 6, 8, 6, 8, 6, 8, 6, 8, 8} 7.25 0.10

38 IEEE Micro
benchmark DNNs, RELEQ enables conventional while it explores the search space such as
hardware5 as well as a custom DNN accelerator6 state of quantization and relative accuracy.
to achieve 2.2  2:7 speedup over 8-bit execu-
tion. These results suggest that RELEQ takes an State of quantization and relative accuracy:
effective first step toward automating the deep The “NetworkSpecific” parameters reflect some
quantization of neural networks. indication of the state of quantization and rela-
tive accuracy. StateofQuantization is a metric to
evaluate the benefit of quantization for the net-
RL FOR DEEP QUANTIZATION OF work and it is calculated using the compute and
DNNs memory costs of each layer. For a neural net-
work with L layers, we define compute cost of
Method Overview layer l as the number of Multiply-Accumulate
RELEQ trains an RL agent that determines the (MAcc) operations (nMAcc ), where (l ¼ 0; . . .; L).
l
bitwidth for each layer of the network. RELEQ Additionally, since RELEQ only quantizes
explores the search space of the bitwidths, layer weights, we define memory cost of layer l as the
by layer. The underlying optimization problem is number of weights (nwl ) scaled by the ratio of
multiobjective (higher accuracy, lower compute, memory access energy (EMemoryAccess ) to MAcc
and reduced memory); however, preserving the computation energy (EMAcc ), which is estimated
accuracy is the primary objective. With this for- to be around 120 .
mulation of the RL problem, RELEQ employs It is intuitive to consider that the sum of mem-
the state-of-the-art proximal policy optimization ory and compute costs linearly scale with the
(PPO)4 to train its policy and value networks. This number of bits for each layer (nbits bits
l ). The nmax term
section details the components and the research is the maximum bitwidth among the predefined
path we have examined to design them. set of bitwidths that are available for the RL agent
to pick from. Finally, the StateofQuantization
(StateQuantization ) is the normalized sum over all
State Space Embedding
layers (L) that accounts for the total memory and
Interplay between layers: We design the
compute costs of the entire network
state space to consider sensitivities and inter-
play between layers by including the knowledge
about the bitwidth of previous layers, the index StateQuantization
  
of the layer-under-quantization, layer size, and PL w EMemoryAccess MAcc bits
l¼0 nl  EMAcc þ nl  nl
weights statistics (e.g., standard deviation).
¼   :
However, this information is incomplete with- PL w EMemoryAccess MAcc bits
l¼0 n l  E þ nl  n max
out knowing the accuracy of the network given a MAcc

set of bitwidths and state of quantization for (1)

the entire network. As such, the parameters used
to embed the state space of RELEQ agent are Besides the potential benefits, captured by
categorized across two different axes. StateQuantization , RELEQ considers the
StateofRelativeAccuracy to gauge the effects of
1) “Layerspecific” parameters, which are unique
quantization on the classification performance.
to the layer (layer index, layer dimensions,
To that end, the StateofRelativeAccuracy
weight statistics) versus “NetworkSpecific”
(StateAccuracy ) is defined as the ratio of the cur-
parameters that characterize the entire net-
rent accuracy (AccCurr ) with the current bit-
work as the agent steps forward during training
widths for all layers during the RL training, to
process (state of quantization and relative
accuracy of the network when it runs with full
accuracy).
precision (AccFullP )
2) “Static” parameters that do not change dur-
ing the training process versus “Dynamic”
parameters that change during training AccCurr
StateAccuracy ¼ : (2)
depending on the actions taken by the agent AccFullP

September/October 2020
39
 Machine Learning for Systems

Figure 1. (a) Action spaces: (i) flexible action space (used in RELEQ); (ii) alternative action space with
restricted movement. (b) Reward shaping with three different formulations as functions of the optimization
objectives: state of relative accuracy and state of quantization: (i) proposed formulation; (ii) direct division; and
(iii) direct subtraction. The color palette shows the intensity of the reward. (c) Overview of RELEQ, which starts
from a pretrained network and delivers its corresponding deeply quantized network.

Given these embeddings of the observations convergence is much longer than the aforemen-
from the environment, the RELEQ agent can take tioned flexible action space, which is used, as it
actions, described next. encourages more exploration.

Flexible Actions Space Asymmetric Reward Formulation for Accuracy

The RELEQ agent steps through each layer
sequentially and chooses the bitwidth of a layer
from a discrete set of bitwidths which are pro-
vided as possible choices.
Figure 1(a)-(i) shows the representation of
action space in which the set of bitwidths is
f1; 2; 3; 4; 5; 6; 7; 8g. As depicted, the agent can flex-
ibly choose to change the bitwidth of a given layer
from any bitwidth to any other bitwidth. An alter-
native [Figure 1(a)-(ii)] that we experimented
with was to only allow RELEQ agent to increment/
decrement/keep the current bitwidth of the
layer (BðtÞ ). The experimentation showed that the
While the state space embedding focused on
interplay between the layers and the action space
provided flexibility, reward formulation for RELEQ
aims to preserve accuracy and minimize bitwidth
of the layers simultaneously. This requirement cre-
ates an asymmetry between the accuracy and bit-
width reduction, which is a core objective of
RELEQ. The following RewardShaping formulation
provides the asymmetry and puts more emphasis
on maintaining the accuracy as illustrated with dif-
ferent color intensities in Figure 1(b)-(i). This
reward uses the same terms of StateQuantization and
StateAcc from the “State Space Embedding” section.

40 IEEE Micro
 Reward Shaping:
reward ¼ 1:0  ðStateQuantization Þa
if ðStateAcc < thÞ then
reward ¼ 1:0
else
Accdiscount ¼ StateAcc ðb=StateAcc Þ
reward ¼ reward  Accdiscount
end if
This used formulation 1) produces a smooth
reward gradient as the agent approaches the
optimum quantization combination; and 2) the
varying 2-D gradient speeds up the agent’s conver-
gence time. In the reward formulation, th is thresh-
old for relative accuracy below, which the
accuracy loss may not be recoverable and those
bitwidths are completely unacceptable. After
some trials, we observe that a ¼ 0:2; b ¼ 0:4; th ¼
0:4 provide reasonable convergence times and
accuracy-quantization tradeoff; thus, we fixed
them throughout the experiments. While Figure 1
(b)-(i) shows the aforementioned formulation,
Figures 1 (b)-(ii) and (iii) depict two other
alternatives. Figure 1(b)-(ii) is based on
StateAcc =StateQuantization while Fig. 1 (b)-(iii) is
based on StateAcc  StateQuantization . In summary,
based on our experiments, the formulation for
Figure 1(b)-(i) offers faster convergence.
Network Architecture of Policy and Value
Networks
Both Policy and value are functions of state, so
the state space, defined in the “State Space
Embedding” section, is encoded as a vector and
fed as input to LSTM layer, which acts as the first
hidden layer for both Policy and Value networks.
Apart from the LSTM, policy network has two fully
connected hidden layers of 128 neurons each and
the number of neurons in the final output layer is
equal to the number of available bitwidths the
agent can choose from, whereas the value network
has two fully connected hidden layers of 128 and
64 neurons each. Based on our evaluations, LSTM
enables the RELEQ agent to converge almost 1:33
faster in comparison to a network with only fully
connected layers.
PUTTING IT ALL TOGETHER: RELEQ
IN ACTION
As discussed in the “RL for Deep Quantiza-
tion Of DNNs” section, state, action, and reward
September/October 2020
Figure 2. Action (bitwidths selection) probability
evolution over training episodes for LeNet.
enable the RELEQ agent to maneuver the search
space with an objective of quantizing the neural
network with minimal loss in accuracy. We use
linear quantization as proposed by Mishra et al.7
Figure 1(c) depicts the entire workflow for RELEQ
and this section gives an overview of how every-
thing fits together in practice.
Learning the policy. Policy in terms of neural
network quantization is to learn to choose the
optimal bitwidth for each layer in the network.
Figure 2 shows the evolution of RELEQ agent’s
bitwidth selection probabilities for all layers of
LeNet over training episodes, which reveals how
the agent’s policy changes with respect to select-
ing a bitwidth per-layer. As indicated on the
graph, the end results suggest the following
quantization patterns, 2; 2; 2; 2 or 2; 2; 3; 2 bits.
For the first two convolution layers, the agent
ends up assigning the highest probability for
two bits. For the third layer (FC1), the probabili-
ties of two bits and three bits are very close.
Finally, for the fourth layer (FC2), the agent again
tends to select two bits, however, with relatively
smaller confidence compared to layers one and
two. With these observations, we can infer
that bitwidth probability profiles are not uni-
form across all layers. As such, the agent dis-
tinguishes between the layers, understands
the sensitivity of the objective function to dif-
ferent layers, and accordingly chooses the
bitwidths.
41
 Machine Learning for Systems
Figure 3. (a) Performance: quantization space and its Pareto frontier for (i) CIFAR-10; (ii) LeNet; (iii) SVHN;
and (iv) VGG-11. (b) Convergence: the evolution of reward and its basic elements: State of Relative Accuracy
for (i) CIFAR-10; (ii) SVHN. State of Quantization for (iii) CIFAR-10; (iv) SVHN, as the agent learns through the
episodes. The last plot (v) shows an alternative view by depicting the evolution of reward for MobileNet. The
trends are similar for the other networks.
EXPERIMENTAL RESULTS
Quantization Levels With RELEQ
Table 1 provides a summary of results with
respect to layerwise quantization bitwidths achi-
eved by RELEQ. At the onset of the agent’s explora-
tion, all layers are initialized to 8 bits. As the agent
learns the optimal policy, each layer converges
with a high probability to a particular bitwidth. As
shown in the “QuantizationBitwidths” column of
Table 1, RELEQ quantization policies show a spec-
trum of varying bitwidth assignments to the
layers. The bitwidth for MobileNet varies with an
irregular pattern, which averages to 6.43. ResNet-
20 achieves mostly 2 and 3 bits, again with an irreg-
ular interleaving that averages to 2.81. In many
cases, there is significant heterogeneity in the bit-
widths and a uniform assignment of the bits is not
always the desired choice to preserve accuracy.
These results demonstrate that RELEQ automati-
cally distinguishes different layers and their vary-
ing importance with respect to accuracy while
choosing their respective bitwidths. As shown in
the “AccuracyLoss” column of Table 1, the deeply
quantized networks with RELEQ have less than
0.30% loss in accuracy. To assess the quality of
these bitwidths assignments, we conduct a Pareto
analysis on the DNNs for which we could populate
the search space.
Validation: Pareto Analysis
Figure 3(a) depicts the solutions space for
four benchmarks (CIFAR10, LeNet, SVHN, and
42
VGG11). Each point on these charts is a unique
combination of bitwidths that are assigned to
the layers of the network. The boundary of the
solutions denotes the Pareto frontier and is
highlighted by a dashed line. The solution
found by RELEQ is marked out using an arrow
and lays on the desired section of the Pareto
frontier, where the accuracy loss can be recov-
ered through fine-tuning, which demonstrates
the quality of the obtained solutions. It is
worth noting that as a result of the moderate
size of these four networks, it was possible to
enumerate the design space, obtain Pareto
frontier, and assess ReLeQ quantization policy
for each network. However, such enumeration
is infeasible for state-of-the-art deep networks
(e.g., MobileNet, AlexNet), which further
stresses the importance of automation and effi-
cacy of RELEQ.
Learning and Convergence Analysis
An appropriate evidence for the correctness of
a formulated RL problem is the ability of the agent
to consistently yield improved solutions. Figure 3
(b) shows [through different quantities (i)–(v)]
that RELEQ consistently yields solutions that
increasingly preserve the accuracy (maximize
rewards), while seeking to minimize the number
of bits assigned to each layer (minimizing the state
of quantization) and eventually converges to a
rather stable solution. The trends are similar for
other networks.
IEEE Micro
Table 2. Speedup and energy reduction with RELEQ over ADMM.8
Network Dataset Technique Bitwidth
RELEQ {8,4,4,4,4,4,4,8}
AlexNet ImageNet
ADMM {8,5,5,5,5,3,3,8}
RELEQ {2,2,3,2}
LeNet MNIST
ADMM {5,3,2,3}
Execution Time and Energy Benefits With RELEQ
Deep quantization with conventional hard-
ware. RELEQ’s solution can be deployed on con-
ventional hardware, such as general purpose CPUs
to provide improvements. To manifest this, we
evaluated RELEQ using TVM5 on an Intel Core i7-
4790 CPU. Figure 4(a) shows the speedup for each
benchmark using TVM compiler. The baseline is
the 8-bit runtime for inference. RELEQ’s solution
offers, on average, 2:2 speedup over the baseline
as the result of merely quantizing the weights that
reduces the amount of computation and data trans-
fer during inference.
Deep quantization with custom hardware
accelerators: To further demonstrate the energy
and performance benefits of the solution found
by RELEQ, we evaluate it on Stripes,6 a custom
accelerator designed for DNNs, which exploits
bit-serial computation to support flexible bit-
widths for DNN operations. Figure 4(b) shows
the speedup and energy reduction benefits of
RELEQ’s solution on Stripes. Baseline is the 8-bit
inference execution. RELEQ’s solutions yield, on
Figure 4. (a) Speedup with RELEQ for conventional
hardware using TVM over the baseline run using 8
bits. (b) Energy reduction and speedup with RELEQ
for Stripes over the baseline execution when the
accelerator is running 8-bit DNNs.
September/October 2020
RELEQspeedup RELEQspeedup on Energy Improvement of
on TVM Stripes RELEQon Stripes
1.20X 1.22X 1.25X
1.42X 1.86X 1.87X
average, 2:0 speedup and an additional 2:7
energy reduction. MobileNet achieves 1:2
speedup, which is coupled with a similar degree
of energy reduction. On the other end of the
spectrum, ResNet-20 and LeNet achieve 3:0 and
4:0 benefits, respectively.
Speedup and Energy Reduction Over ADMM
We compare RELEQ’s solution in terms of
speedup and energy reduction against ADMM,8
another procedure for finding quantization bit-
widths. As shown in Table 2, RELEQ’s solution
provides 1:25 energy reduction and 1:22 aver-
age speedup over ADMM with Stripes for Alex-
Net and the benefits are higher for LeNet.
RELATED WORK
RELEQ is the initial step in utilizing reinforce-
ment learning to automatically find the bitwidth
for the layers of DNNs such that their accuracy is
preserved.
Reinforcement learning for automatic tuning:
RL-based methods have attracted much attention
within neural architecture search (NAS) after
obtaining the competitive performance on the
CIFAR-10 dataset employing RL as the search strat-
egy.9 Different RL approaches differ in how they
represent the agent’s policy. Zoph and Le9 used
an RNN trained by policy gradient to sequentially
sample a string that in turn encodes a neural
architecture.
Aside from NAS, He et al.10 employed RL to
prune existing architectures, where a policy gradi-
ent method is used to automatically find the com-
pression ratio for different layers of a network.
Techniques for selecting bitwidths: Recent
work ADMM8 runs a binary search to minimize the
total square quantization error in order to decide
the bitwidths for the layers. Then, they use an
43
 Machine Learning for Systems
iterative optimization technique for fine-tuning.
Other work11 focused on binarized neural net-
works. There is a concurrent work HAQ,12 which
also uses RL in the context of quantization. The fol-
lowing highlights some of the differences. RELEQ
uses a unique reward formulation and shaping
that enables simultaneously optimizing for two
objectives (accuracy and reduced computation
with lower bitwidth) within a unified RL process.
In contrast, HAQ utilizes accuracy in the reward
formulation and then adjusts the RL solution
through an approach that sequentially decreases
the layer bitwidths to stay within a predefined
resource budget. This approach also makes HAQ
focused more toward a specific hardware platform
whereas we are after
a strategy that can
generalize. Addition- This article offered the
RL framework that was
ally, we also provide
able to effectively
a systemic study of
navigate the huge
different design deci-
search space of
sions, and have sig- quantization and
nificant performance automatically quantize
gain across diverse a variety of networks
benchmarks. The ini- leading to significant
tial version of our performance and
work,13 predates energy benefits.
HAQ, and it is the
first to use RL for quantization. Later HAQ was pub-
lished in CVPR, and we published initial version of
RELEQ in NeurIPS ML for Systems Workshop.
CONCLUSION
This article set out to define the automated dis-
covery of bitwidths for the layers while complying
to the constraint of maintaining the accuracy. As
such, this article offered the RL framework that
was able to effectively navigate the huge search
space of quantization and automatically quantize
a variety of networks leading to significant perfor-
mance and energy benefits. The results suggest
that a diligent design of our RL framework, which
considers multiple concurrent objectives can
automatically yield high accuracy, yet deeply
quantized, networks.
ACKNOWLEDGMENTS
This work was supported in part by the
National Science Foundation (NSF) under Awards
44
CN#1703812, ECCS#1609823, CCF#1553192; in
part by the Semiconductor Research Corporation
(SRC) under Contract #2019-SD-2884; in part by
the Air Force Office of Scientific Research
(AFOSR) Young Investigator Program (YIP)
under Award #FA9550-17-1-0274; in part by the
National Institute of Health (NIH) under Award
#R01EB028350; in part by the Air Force Research
Laboratory (AFRL) and the Defense Advanced
Research Project Agency (DARPA) under Agree-
ment #FA8650-20-2-7009 and #HR0011-18-C-0020;
and in part by gifts from Microsoft, Google, Qual-
comm, Xilinx. The U.S. Government is authorized
to reproduce and distribute reprints for Govern-
mental purposes notwithstanding any copyright
notation thereon. The views and conclusions
contained herein are those of the authors and
should not be interpreted as necessarily repre-
senting the official policies or endorsements,
either expressed or implied, of Microsoft, Google,
Qualcomm, Xilinx, SRC, NSF, AFSOR, NIH, AFRL,
DARPA or the U.S. Government.
& REFERENCES
1. Y. LeCun, Y. Bengio, and G. E. Hinton, “Deep learning,”
Nature, vol. 521, no. 7553, pp. 436–444, 2015.
2. S. Zhou, Z. Ni, X. Zhou, H. Wen, Y. Wu, and Y. Zou,
“DoReFa-Net: Training low bitwidth convolutional
neural networks with low bitwidth gradients,” CoRR,
2016.
3. C. Zhu, S. Han, H. Mao, and W. J. Dally, “Trained
ternary quantization,” in Proc. ICLR, 2017.
4. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and
O. Klimov, “Proximal policy optimization algorithms,”
2017, arXiv:1707.06347.
5. T. Chen et al., “TVM: End-to-end optimization stack for
deep learning,” 2017, arXiv:1802.04799.
6. P. Judd, J. Albericio, T. H. Hetherington, T. M. Aamodt,
and A. Moshovos, “Stripes: Bit-serial deep neural
network computing,” in Proc. 49th Annu. IEEE/ACM
Int. Symp. Microarchit., 2016, pp. 1–12.
7. A. K. Mishra, E. Nurvitadhi, J. J. Cook, and D. Marr,
“WRPN: Wide reduced-precision networks,” in Proc.
ICLR, 2018.
8. S. Ye et al., “A unified framework of DNN weight
pruning and weight clustering/quantization using
ADMM,” 2018, arXiv:1811.01907.
9. B. Zoph and Q. V. Le, “Neural architecture search with
reinforcement learning,” in Proc. ICLR, 2017.
IEEE Micro
10. Y. He, J. Lin, Z. Liu, H. Wang, L.-J. Li, and S. Han,
“AMC: AutoML for model compression and
acceleration on mobile devices,” in Proc. 13th Eur.
Conf. Comput. Vision, 2018.
11. I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and
Y. Bengio, “Quantized neural networks: training neural
networks with low precision weights and activations,”
J. Mach. Learn. Res., vol. 18, 2017.
12. K. Wang, Z. Liu, Y. Lin, J. Lin, and S. Han, “HAQ:
Hardware-aware automated quantization,” Nov. 21,
2018, arXiv:1811.08886.
13. A. T. Elthakeb, P. Pilligundla, F. Mireshghallah,
A. Yazdanbakhsh, and H. Esmaeilzadeh, “Releq: A
reinforcement learning approach for deep quantization
of neural networks,”, Nov. 5, 2018, arXiv:1811.01704.
[Online]. Available: http://arxiv.org/abs/1811.01704
Ahmed T. Elthakeb is currently working toward the
Ph.D. degree in the Alternative Computing Technolo-
gies (ACT) Lab, University of California San Diego. His
current research interests include developing cross-
stack solutions to improve the performance and
energy efficiency of machine learning algorithms.
September/October 2020
Prannoy Pilligundla is currently working toward the
Master’s degree in computer science with the Univer-
sity of California San Diego. His research focus is on
designing frameworks and cross platform solutions for
accelerating machine learning applications.
Fatemehsadat Mireshghallah is currently work-
ing toward the Ph.D. degree in computer science with
the University of California San Diego. Her research
focuses on deep learning and privacy.
Amir Yazdanbakhsh joined Google Brain as a
Research Scientist in 2019 following a one year AI resi-
dency. Yazdanbakhsh received the Ph.D. degree in
computer science from Georgia Institute of Technol-
ogy. His research interests include machine learning,
computer architecture, and programming language
for hardware design.
Hadi Esmaeilzadeh is currently the inaugural holder
of Halicioglu Chair in Computer Architecture with the
rank of Associate Professor in computer science and
engineering with the University of California San Diego,
where he was awarded early tenure. His research inter-
ests include the intersection of architecture, machine
intelligence, system design, and software engineering.
45
 Theme Article: Machine Learning for Systems

Enhancing Model
Parallelism in Neural
Architecture Search for
Multidevice System
Cheng Fu and Huili Chen Yuandong Tian
University of California, San Diego Facebook AI Research
Zhenheng Yang Jishen Zhao
Facebook AI Research University of California, San Diego
Farinaz Koushanfar
University of California, San Diego

Abstract—Neural architecture search (NAS) finds favorable network topologies for better
task performance. Existing hardware-aware NAS techniques only target to reduce inference
latency on single CPU/GPU systems and the searched model can hardly be parallelized. To
address this issue, we propose ColocNAS, the first synchronization-aware, end-to-end NAS
framework that automates the design of parallelizable neural networks for multidevice
systems while maintaining a high task accuracy. ColocNAS defines a new search space with
elaborated connectivity to reduce device communication and synchronization. ColocNAS
consists of three phases: 1) offline latency profiling that constructs a lookup table of
inference latency of various networks for online runtime approximation; 2) differentiable
latency-aware NAS that simultaneously minimizes inference latency and task error; and 3)
reinforcement-learning-based device placement fine-tuning to further reduce the latency of
the deployed model. Extensive evaluation corroborates ColocNAS’s effectiveness to reduce
inference latency while preserving task accuracy.

Digital Object Identifier 10.1109/MM.2020.3004538

Date of publication 26 June 2020; date of current version
1 September 2020.

46
0272-1732 ß 2020 IEEE Published by the IEEE Computer Society IEEE Micro
& DEEP NEURAL NETWORKS (DNNs) are increas-
ingly adopted in various fields due to their
unprecedented performance. Enormous archi-
tecture-level advancements have been proposed
to improve the performance and efficiency of
DNN execution.1 In the meantime, neural archi-
tecture search (NAS)2,3 enables a new line of
research that aims to design efficient DNN topol-
ogy for target hardware platforms. Existing hard-
ware friendly NAS techniques4–6 target to
identify efficient models on single CPU/GPU sys-
tems by introducing the notion of FLOPs, model
size, or predicted latency into the searching pro-
cess. However, in modern real-time and cloud
computing scenarios, multiple computation
components co-exist and are shared by different
tasks or users.
There are two types of parallelism when exe-
cuting an neural network (NN) on a multiple-
device platform: 1) model parallelism partitions
operations into different parts and assign them
onto different devices; and 2) data parallelism
distributes the inference data onto multiple devi-
ces and duplicate the entire model on each
device. However, data parallelism has several
limitations. 1) Data parallelism can increase the
throughput of the application but not the end-to-
end inference latency, thus it is not suitable for
real-time applications. 2) Data parallelism may
incur severe communication overhead during
the setup stage since the inputs need to be dis-
tributed to each available device. 3) Data paral-
lelism is infeasible when the model size is too
large to fit on a single device with constrained
memory. With the increasing model size of SOTA
NNs, the fact in 3) is becoming more stringent,
especially for edge devices with very limited
memory. In contrast, emerging multidevice com-
puting systems and device-to-device communi-
cation techniques (e.g., 5G communication,
NVLINKs, and PCIe Gen 4) can benefit model par-
allelism. Because our goal is to reduce applica-
tion latency, model-level parallelism is a
promising candidate.
The designed search spaces of conventional
NAS can be classified into two main categories
(see Figure 2). 1) Layer-wised search. Existing
latency-oriented model searching methods4,5
explore model architectures that are analogous
to chain-like wired mobilenetV2 with the building
September/October 2020
block shown in Figure 2(a). This topology is hard
to be paralleled across multiple computation
components due to data dependence. 2) Cell-
structure search. Another line of methods2,7 pro-
pose to search a building block, called cell, on
small tasks (e.g., CIFAR-10) and transfer the
searched cell for training on a large task (e.g.,
ImageNet). The model in this space have certain
levels of parallelism due to the concurrent exe-
cution of different blocks inside each cell. How-
ever, the concatenation (concat) operations
between cells prevents further parallelism since
they synchronize the execution and increase
communication overhead. To overcome these
constraints, ColocNAS is motivated to automate
the design of suitable NN architectures for multi-
device platforms to achieve low inference
latency and high task accuracy.
Designing an efficient model to achieve model
parallelism across devices is a nontrivial task. It
is challenging because of the following. C1) The
searched architecture needs to keep a high accu-
racy, while reducing its runtime overhead. C2)
While NNs with more complicated wiring pat-
terns have higher chance for model parallelism,
predicting their execution time is challenging for
latency-aware NAS. C3) The actual latency of a
specific network architecture in a multidevice
Figure 1. (a) ColocNAS design overview and (b) roadmap
between accuracy and latency of NAS methods. DP+NN refers to
device placement (DP) policy and neural network (NN)
architecture for deploying. ColocNAS significantly outperforms
other NAS methods, when running on multiple devices.
47
 Machine Learning for Systems
Figure 2. Example of basic building blocks in three types of search space. (a) Layer-wise searched model.
(b) Cell-based searched model. (c) New search space proposed in ColocNAS. The yellow circle refers to the
concatenation node. bin denotes the nth block in ith cell (i.e., layer).
setting depends on the device placement policy,
which is unknown ahead of time.
To tackle these challenges, we propose Coloc-
NAS, a differentiable NAS framework that effec-
tively searches network architectures for the
given hardware environment. ColocNAS resolves
all the aforementioned challenges (C1–C3) using
the following solutions [as shown in Figure 2 (a)].
S1) ColocNAS designs a new search space with
less synchronization and communication over-
head by exploring elaborate connectivity patterns
in the NN. S2) Based on the observation that simi-
lar computation graphs yield closer latency,
ColocNAS leverages offline latency profiling and k-
nearest neighbors (kNN) with graph similarity to
predict latency for searched model in the online
phase. S3) ColocNAS employs uniform workload
distribution as the expert placement policy to
facilitate offline latency profiling and to guide
online latency-aware NAS. This policy leverages
the intrinsic structure features of the DNN in
the new search space. After online NAS, the
placement of the searched model is further fine-
tuned using a reinforcement learning (RL)-based
algorithm to reduce its runtime on the target
platform.
Our framework offers a holistic solution to
designing efficient and accurate NNs on mul-
tidevice systems. Extensive experiments show
that ColocNAS reduces inference latency by a
large margin, while achieving a competitive accu-
racy and latency as shown in Figure 1(b). Our
work sheds light on a new dimension (device-level
parallelism) of hardware-aware NAS algorithms.
48
ColocNAS DESIGN
ColocNAS uses differentiable neural architec-
ture search by combining two gradient-based
methods4,7 to solve the problem of topology
design. We formulate the neural architecture
search problem as a nonconvex optimization
problem as shown in
min min Lða; wa Þ: (1)
a2A wa
A promising architecture yields small latency
and high task accuracy. Here, A is a new search
space proposed in ColocNAS, a 2 A is a set of
continuous variables that specify a possible
architecture, wa is the weight parameter of the
network. L is the loss function that penalizes
both accuracy degradation as well as the
increase of inference latency. The design flow of
ColocNAS is detailed in Algorithm 1.
Algorithm 1. ColocNAS Design Flow.
INPUT: Prototyping Hardware Devices ðtÞ; Device
Number ðN t Þ; ArchTable Size (Narch ); Cell
Number Nc ; Possible Operations (Ops).
OUTPUT: Fine-tuned Device Placement Policy
PRL ; Model Mout .
1: Offline Profile:
Mi Random GenerateðN t ; Ops; Nt ; Nc Þ
Pi Expert Placement PolicyðMi Þ
Arch Table Hardware ProfileðMi ; Pi ; tÞ
for i ¼ 1; :::; Narch
2: Online Searching and Training:
March SearchingðArch Table; Ops; Nt ; Nc Þ
Mout Training ModelðMarch Þ
3: RL Fine-Tuning:
PRL RL FineTuningðMout ; tÞ
IEEE Micro
Search Space
Previous works search models with layer-
wise or cell-based structures as shown in Figure 2
(a) and (b). The layer-wise structure is suitable
for a single CPU device as the identified model
topology is very regularized and its execution is
fast due to data locality. However, the resulting
chain-like model is not suitable when multiple
devices are available for parallelism, as each
layer can only start execution when the outputs
of all its previous layers are computed. For the
cell structure search space, its evaluation can be
paralleled. Yet, the concatenation at the end of
each cell node works as a synchronization unit
that collects all the results inside the cell blocks
before starting the computation in the next cell.
This synchronization incurs severe communica-
tion overhead (the red arrow in Figure 2) and
hinders computation parallelism. As such, we
proposed a new search space that reduces the
participants of the synchronization unit in order
to minimize the delay of computation as shown
in Figure 2(c). ColocNAS uses the same definition
of search blocks and cells while introducing a
new block connectivity pattern by “duplicating”
the concatenation nodes.
The traditional cell consists of an ordered
sequence of N blocks where each block has two
input edges ði; jÞ as shown in Figure 2(b). An
edge can be selected from two sources. i) previ-
ous blocks in the current cell. ii) The output of
the previous two cells. Each edge is associated
with an operation that is determined after the
search [oði; jÞ] is performed. Assuming the
inputs of the block are x1 and x2 , the output of
the block is computed as follows:
X
2
bj ¼ oði;jÞ ðxi Þ:
i¼1
The intermediate results from the N blocks are
concatenated together to form a single cell output
xi . Since we identify that this concatenation node
is the bottleneck for achieving device-level paral-
lelism, the new search space is defined as follows.
Instead of reducing towards a single concate-
nation node in each cell, the blocks of the ith
cell will be evenly connected to C different con-
catenation nodes, resulting in outputs xi ¼ ½xi1 ;
xi2 ; :::; xic T . For the nth block in the ith cell, we
September/October 2020
define block connectivity parameters bi1 as the
n
probability over each possible connection
between the current block to any concatenation
node in ði  1Þth layer. It will be used to compute
the weighted sum input xi1 from previous layer
n
i  1 as
i1 i1
xi1
n ¼ softmaxðbn Þ  x
X
C expðbi1
n;c Þ (3)
¼ PC i1
xi1
c :
c¼1 c0 ¼1 expðb n;c0 Þ
Here, bi1
n is a vector of size 1-by-C. xi2
n is com-
i2
puted from bn using the same method as (3).
Note that ColocNAS explores optimal net-
work architectures by solving the bilevel optimi-
zation problem defined in (1). The optimization
variable a ¼ fb; opsg where b is the block con-
nectivity parameter and the softmax of ops is the
probability vectors over the predefined set of DL
operations for every possible connection (the
candidate operations are the same as given by
Cai et al.5). At the end of the search, we choose
the connectivity of the nth block in ith cell from
the previous two cells as xi1 c = arg maxc bi1
n and
i2
xi2
c = arg max b
c n .
Through comparison experiments, we find
that by using Gumbel softmax4 over bic in (3) can
improve both the accuracy and latency of the
searched model compared to the softmax
function.
Given xi1 i2 as the computed input from
n ; xn
the previous two cells for the ith cell, the output
of the each blocks bni would be the weighted sum
over ops7 during architecture search.
Offline Hardware Bounded Latency Profiling
ColocNAS integrates a latency-aware, gradi-
ent-based NAS for the target devices into the
(2)
searching process to make the low latency model
preferable. Recall that the optimization variable
a in our search space is a set of probabilistic vari-
ables, thus the inference latency given a specific
choice of a is also a random variable lat. We use
the expectation value of the latency variable to
assess the quality of the architecture choice a.
However, computing the latency expectation
over a architecture probability Elat ðpa Þ on the
given devices is challenging because of the fol-
lowing. 1) Measuring the real latency value of
each architecture during the searching process
49
 Machine Learning for Systems
is infeasible due to the prohibitive latency cost.
2) The latency of a complex graph is hard to pre-
dict while considering the delay incurred by data
movement. 3) The latency value of an architec-
ture choice is highly dependent on the placement
policy, which makes it hard to compare the opti-
mal latency values of different network topolo-
gies. To resolve the above problems, the first
stage of ColocNAS is an offline latency profiling
step that measures the inference time of diverse
network architectures on the target devices.
Besides, to disentangle the complexity of
model placement on the given hardware, we use
a predefined expert placement policy for all
searched architectures. In particular, our expert
placement policy uniformly distributes each
concatenation node and the associated opera-
tions onto all existing devices. By doing so, we
reduce the overhead of synchronization and the
cross-device communication compared to the
traditional cell-based search space as shown in
Figure 2(c).
It is also intractable to profile the latency val-
ues of all the architectures in the entire search
space ( 1020 models). Leveraging the observa-
tion that similar NNs yield closer runtime
latency, we propose a method to approximate
the latency of a complicate wired NN using kNN.
We randomly generate architectures and profile
their latency values on the target hardware using
expert placement policy to obtain a lookup table
ArchTable. The distance between two NNs can
be measured by using graph edit distance8. To
approximate the latency expectation over an
architecture parameter, we first convert a into
the corresponding probability variable pa which
includes the following two parts:
pops ¼ SoftmaxðopsÞ (4)
pb ¼ GumbelSoftmaxðbÞ: (5)
Given a specific choice of the probability
parameter pa ¼ fpb ; pops g, we sample N neural
network architectures arch1 ; arch2 ; :::; archN . The
expectation value of the latency variable is then
approximated as
1X N
Elat ðpa Þ  kNNðarchn ; ArchTableÞ (6)
N n¼1
50
where the kNNðcotÞ function returns the average
latency of the top-k closest architecture laten-
cies for input archn .
Online Latency-Aware Architecture Searching
ColocNAS aims to find a network architecture
with low latency and high task accuracy. As
such, we define the loss function of ColocNAS’s
online searching phase as follows:
Lða; wa Þ ¼ CEða; wa Þ þ 1 Llat (7)
Llat ¼ expðjjpa  pba jj2 Þ  Elat ðpa Þ: (8)
Here, CEða; wa Þ is the cross-entropy loss given
the architecture parameter a and the weights
wa . 1 is the scaling factor within the range [0,1]
that controls the tradeoff between accuracy and
latency. Elat ðpa Þ is the expected latency of the
specific architecture parameter obtained from
(6). For the latency term, we add a regularization
P
term expðjjpa  pba jj2 Þ where pba ¼ N1 Nn¼1 archn
to make the latency term differentiable to the
architecture parameters a. If the sample mean
value of the architecture probability pba is far
from the true one pa , the corresponding latency
term will be weighted less. Note that after the
model is searched, we train the model from
scratch to obtain the final accuracy.
RL-Based Device Placement Fine Tuning
Once the model is searched and trained, the
model can be directly deployed on to the target
hardware systems using the expert placement
policy that uniformly distributes the workload
to each available device. However, this heuristic-
based device placement policy may not be opti-
mal considering the heterogeneous property of
the underlying computing platforms. To further
optimize the latency of the searched architec-
ture G, the last step of the ColocNAS leverages
policy gradient for device placement on the
given hardware. We apply the RL-based method
proposed by Mirhoseini et al.9 where the policy
network is an attentional autoencoder that
takes the NN graph as input. Each input in the
encoding sequence is the concatenated embed-
ding of operation type, connectivity and output
shape of each block in the graph. The decoder
sequentially generates the placement policy P .
IEEE Micro
The goal is to learn the policy pðP jG; uÞ to mini-
mize the objective JðuÞ ¼ EP pðP jG;uÞ ½RðP ÞjG.
Here, RðP Þ is the expectation of the reward
(execution time).
The policy gradients are computed via the
REINFORCE equation10 as follows:
ru JðuÞ ¼ EP pðP jG;uÞ ½RðP Þ  ru logp ðP jG; uÞ:
By sampling K placements following the
placement policy probability P  pðjG; uÞ, the
expectation of reward RðP Þ can be estimated
and the policy gradient is computed as
1X K
ru JðuÞ  ðRðPi Þ  BÞ  ru logp ðP jG; uÞ:
K i¼1
Here, B is the mean value of the rewards com-
puted from the K sampled placements. Coloc-
NAS leverages the intrinsic structure of the
architecture in the new search space and per-
forms device placement only for blocks and con-
catenation nodes [see Figure 2 (c)]. As a result,
the training of the policy network pðP jG; uÞ con-
verges much faster compared to the original one
by Mirhoseini et al.9
EXPERIMENTS
In this section, we demonstrate ColocNAS’s
effectiveness of model-level parallelism and
accurate classification performance on CIFAR-10
and ImageNet tasks compared to the state-of-
the-art NAS methods.
Experimental Setup
We run ColocNAS on different device config-
urations and evaluate the searched architec-
tures in terms of its test accuracy and inference
latency.
Searching phase setup. ColocNAS searches two
types of cells, namely, a normal cell and a reduction
cell. We put the reduction cell at 1=3 and 2=3 loca-
tion of the neural architectures. After each reduc-
tion cell, we double the number of output
channels in the network. The operations searched
in the reduction cell has stride¼ 2.
The model that uses on the searching phase
has eight cells. Larger networks can be built by
stacking multiple normal cells in between the
reduction cells. We alternatively update the two
September/October 2020
variables (a and wa ) to solve the bilevel optimiza-
tion problem in (1). In particular, we use second-
order approximations to update the architecture
parameter a. All possible operations in the
search space and relevant searching details are
the same as DARTS.7
Hardware platforms and corresponding search
(9) space. To prove the generality of ColocNAS on
different platforms, we test our method on two
types GPUs and six different device configura-
tions. 1) 2/3/4 Tesla K80 GPUs with Gen3 PCIe
device connection. 2) 2/3/4 Tesla V100 GPUs
with NVlink connections. The device placement
algorithm and latency profiling for cell structure
are implemented in Tensorflow v1.14. For each
(10)
device setting, we set the number of concatena-
tion nodes to be the same as the number of devi-
ces to facilitate the expert placement policy.
Since the device configuration determines the
search space as shown in Figure 2, we name the
search space with 2/3/4 GPUs as ColocNAS-SP-2/
ColocNAS-SP-3/ColocNAS-SP-4. We denote the
searched architectures as ColocNAS-b4c2/Coloc-
NAS-b6c3/ColocNAS-b4c4 which have 4/6/4 num-
ber of blocks and 2/3/4 concatenation nodes in
each cell for hardware settings with 2/3/4 GPUs,
respectively.
Latency Prediction Results
Recall that ColocNAS applies the kNN
method to estimate end-to-end latency. We sam-
ple 10 000 architectures that are mapped using
the expert placement policy to available GPUs
on both CIFAR-10 (32323) and ImageNet
(2242243) datasets. We set the total number
of cells to be 14 and 20 for ImageNet and CIFAR-
10, respectively. For CIFAR-10 and ImageNet for
evaluation, we added one/two initial stem cells
to downscale the raw images, respectively.
Figure 3 shows the effectiveness of our kNN-
based latency predictor. The latency is profiled in
search space ColocNAS-SP-4 with ImageNet input
(batch size is 32) on 4 Tesla K80 GPUs. We use 5%
of the profiled data for testing. The profiling takes
2.5 days on 8 GPUs. The RMSE error is 0.22 ms on
the test data, which is adequate for the latency
prediction of complicated graphs. Note that this
offline profiling is a one-time profiling process,
that can be used to search different NN architec-
tures when tuning the 1 parameters in (7).
51
 Machine Learning for Systems

Results of Latency-Aware NAS

Cifar-10 Benchmark After the architecture is
searched, we train the weight parameters of the
resulting network for 1500 epochs using a batch
size of 96 and Adam optimizer. As shown in
Table 1, ColocNAS maintains a competitive test
error rate (2.36% on average) in different hard-
ware settings. In the meantime, ColocNAS
Figure 3. Effectiveness of kNN-based end-to-end
reduces the inference latency by a large margin
latency predictor on the ImageNet data with 4 Tesla
(1.24/1.34/1.50 faster on 2/3/4 Tesla K80
K80 GPUs. We determine K ¼ 5 using cross-
GPUs compared to DARTS 2-GPU) on multiple-
validation, which yields RMSE 0.22 ms on the test set.
devices thanks to the parallelizable search space
and differentiable latency guidance. ColocNAS
uses a search space with elaborated connectiv- searching step, we stack 14 cells to build an NN
ity and an online kNN-based latency lookup, thus for ImageNet evaluation. The network is trained
the search cost is higher than DARTS (see for 360 epochs with power cosine learning rate
Table 1). Empirical results show that ColocNAS and SGD optimizer with nesterov-momentum.
is still among the fastest NAS techniques due to The comparison results are shown in Table 2.
our gradient-based method and CIFAR-10 proxy. Compared to the state-of-the-art NAS methods,
our model shows a competitive error rate on the
ImageNet Results Using the normal and test dataset (25.47% on average). Furthermore,
reduction cell found by the latency-aware our model outperforms all other state-of-the-art

Table 1. Performance comparison between ColocNAS and the state-of-the-art NAS methods classification. The test error and
inference latency on CIFAR-10 benchmarks are shown here. For cell-based method, the latency is tested by reconstructing the
model using open-source code from DARTS. 7 “-” indicates the model for CIFAR-10 is not publicly available. We use a batch size of
32 for all latency evaluation. The latency of ColocNAS is tested on 2/3/4 Tesla K80 GPUs after RL fine-tuning and “DARTS 2-GPU”
are tested on 2 GPUs using RL placement. Other baselines are tested on a single Tesla K80 GPU.

Search Search Search Cost (GPU # Test Error Latency (per Normalized
Model
Space Method hours) Params (%) image) Reduction (%)
DenseNet-
- manual - 25.6M 3.46 14.8 ms 0.0
BC

NAONet cell gradient 4.8K 10.6M 3.18 8.4 ms -43.2

PNAS cell SMBO 6K 3.2M 3.41 3.14 ms -78.8

Amoeba-
cell evolution 76K 3.2M 3.12 3.10 ms -79.1
A 11

Amoeba-
cell evolution 76K 2.8M 2.55 3.04 ms -79.4
B 11

DARTS
cell gradient 24 3.2M 2.76 3.12 ms -78.9
(2nd) 7

NASNet-A 2 cell RL 48K 3.3M 2.65 3.24 ms -78.1

DARTS 2-
cell gradient 24 3.2M 2.76 3.01 ms -79.7
gpu 7

ColocNAS- new
gradient 41 3.4M 2.31 2.43 ms -83.6
b4c2 space

ColocNAS- new
gradient 49 3.5M 2.35 2.24 ms -84.9
b6c3 space

ColocNAS- new
gradient 51 3.4M 2.42 2.01 ms -86.4
b4c4 space

52 IEEE Micro
Table 2. Transferability classification error on ImageNet benchmarks. For NAS in layer-wise search space, the latency is tested by
using open-source evaluation code from the papers. “-” indicates the number is not shown in the original paper or the model is not
publicly available. The latency results in the table are tested on Tesla K80 GPUs. The RL setting for the baseline is evaluated on 2-
GPU. With more GPUs, the baseline would yield same latency as the models can hardly be parallelized across devices.

Search Search Search cost # # Test accuracy Latency Normalized

Model
space method (GPU hours) Params FLOPs (Top-1 %) (-RL/+RL) reduction (%)
3.51/3.51
MobileNetV2 - - - 3.4M 300M 72.0 0.0
ms

ShuffleNetV2 3.32/3.32
- - - 3.5M 299M 72.6 -5.4
(1.5) ms

3.13/ 3.02
Amoeba-A 11 Cell Evolution 756K 5.1M 555M 74.5 -14.0
ms
3.22/ 3.10
NASNet-A 2 Cell RL 48K 5.3M 564M 74.0 -11.7
ms
3.09/2.98
DARTS (2nd) 7 Cell Gradient 24 4.9M 595M 73.1 -15.1
ms
Layer
MnasNet-65 6 RL 91K 3.6M 270M 73.0 - -
wise
Layer
MnasNet 6 RL 91K 4.2M 317M 74.0 - -
wise
Layer 2.93/2.93
FBNet-A 4 Gradient 216 4.3M 249M 73.0 -16.5
wise ms
Layer 3.41/3.41
FBNet-B 4 Gradient 216 4.5M 295M 74.1 - 2.8
wise ms

ProxylessNAS- Layer 3.95/3.95

Gradient 200 6.9M - 74.6 +12.5
mobile 5 wise ms

ProxylessNAS- Layer 2.80/2.80

Gradient 200 7.1M - 75.1 -20.2
GPU 5 wise ms

ColocNAS- New 2.67/2.51

Gradient 41 4.7M 546M 74.9 -28.5
b4c2 space ms

ColocNAS- New 2.41/2.23

Gradient 49 4.8M 572M 74.6 -36.5
b6c3 space ms

ColocNAS- New 2.18/2.07

Gradient 51 4.8M 566M 74.1 -41.0
b4c4 space ms

NAS methods in terms of the inference latency
when deploying in a multiple-device hardware
environment. With 2/3/4 GPU settings, Coloc-
NAS achieves 1.57/1.77/1.91 execution
speedup compared to the ProxylessNAS-mobile
on Tesla K80. For Tesla V100 GPUs, ColocNAS
yields a similar speedup compared to DARTS
and ProxylessNAS. Applying the RL placement,
we directly tested models searched for Tesla
K80 onto the new hardware systems and the
latency per image for ColocNAS-b4c2/b6c3/b4c4
are 0.66/0.61/0.54 ms on 2/3/4 Tesla V100 GPUs,
which are 1.12/1.21/1.35 faster than DARTS
(0.74 ms) and 1.28/1.39/1.57 faster than
ProxylessNAS-mobile (0.85 ms). This corrobo-
rates that ColocNAS preserves a high task accu-
racy while reducing the latency overhead. We
also notice that if without the guidance of the
hardware-bounded latency estimation [1 ¼ 0 in
(7)], the latency incurred by ColocNAS will
increase by +0.23 ms (ColocNAS-b4c4) on Tesla
K80 GPUs. Based on our observation, the
latency regularization term encourages compu-
tation-efficient operations on the timing critical
paths in the model. In addition, it explicitly
helps to reduce the number of connectivity
across blocks compared to model searching
without latency guidance.

September/October 2020
53
 Machine Learning for Systems
Performance Discussion One can observe
the trade-off between accuracy and latency
from Table 1 and 2. ColocNAS aims to achieve
hardware-aware NAS for fast model inference by
reducing the communication overhead between
multiple computing platforms. However, the
reduction in communication would incur
accuracy degrada-
tion since the inf-
ormation exchange In this article, we
between sub-graphs propose ColocNAS,
for each device is an end-to-end,
differentiable neural
reduced.
architecture searching
Tables 1 and 2
framework that is aware
show that the
of the hardware-
latency improve- bounded latency.
ment of ColocNAS
is sublinear with
the number of computing devices. This is caused
by the nonnegligible communication overhead
between devices and system control overhead.
CONCLUSION
In this article, we propose ColocNAS, an end-
to-end, differentiable neural architecture search-
ing framework that is aware of the hardware-
bounded latency. ColocNAS introduces an inno-
vative parallelizable search space that redu-
ces synchronization/device communication for
model parallelism. For the first time, ColocNAS is
able to maximize device utilization of multiple
computing platforms by decoupling the task of
hardware-aware NAS into three steps: offline
hardware latency profiling, online latency-aware
searching, and RL-based device placement fine-
tuning. As a result, ColocNAS automates the
design of neural architectures that achieve high
task accuracy and low runtime latency for a given
hardware setting. We perform extensive experi-
ments and corroborate that ColocNAS reduces
the inference latency by a large margin in a
multidevice environment while maintaining a
competitive task accuracy compared to the
state-of-the-art hardware-aware NAS methods.
ACKNOWLEDGMENTS
This work was supported by NSF 1829525 and by
the SRC/DARPA Center for Research on Intelligent
Storage and Processing-in-Memory.
54
& REFERENCES
1. V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer,
“Efficient processing of deep neural networks: A
tutorial and survey,” Proc. IEEE, vol. 105, no. 12,
pp. 2295–2329, Dec. 2017.
2. B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le,
“Learning transferable architectures for scalable
image recognition,” in Proc. IEEE Conf. Comput. Vision
Pattern Recognit., 2018, pp. 8697–8710.
3. B. Zoph and Q. V. Le, “Neural architecture search with
reinforcement learning,” in Proc. 5th Int. Conf. Learn.
Representations, 2017. [Online]. Available: https://
openreview.net/forum?id=r1Ue8Hcxg
4. B. Wu et al., “Fbnet: Hardware-aware efficient convnet
design via differentiable neural architecture search,” in
Proc. IEEE Conf. Comput. Vision Pattern Recognit.,
2019, pp. 10 734–10 742.
5. H. Cai, L. Zhu, and S. Han, “Proxylessnas: Direct neural
architecture search on target task and hardware,” in
Proc. Int. Conf. Learn. Representations, 2019. [Online].
Available: https://arxiv.org/pdf/1812.00332.pdf
6. M. Tan et al., “Mnasnet: Platform-aware neural
architecture search for mobile,” in Proc. IEEE Conf.
Comput. Vision Pattern Recognit., 2019, pp. 2820–2828.
7. H. Liu, K. Simonyan, and Y. Yang, “Darts: Differentiable
architecture search,” in Proc. 7th Int. Conf. Learn.
Representations, 2019. [Online]. Available: https://
openreview.net/forum?id=S1eYHoC5FX
8. X. Gao, B. Xiao, D. Tao, and X. Li, “A survey of graph
edit distance,” Pattern Anal. Appl., vol. 13, no. 1,
pp. 113–129, 2010.
9. A. Mirhoseini et al., “Device placement optimization
with reinforcement learning,” in Proc. 34th Int. Conf.
Mach. Learn. 2017, vol. 70, pp. 2430–2439, [Online].
Available: JMLR.org.
10. R. J. Williams, “Simple statistical gradient-following
algorithms for connectionist reinforcement learning,”
Mach. Learn., vol. 8, no. 3/4, pp. 229–256, 1992.
11. E. Real, A. Aggarwal, Y. Huang, and Q. V. Le,
“Regularized evolution for image classifier architecture
search,” in Proc. AAAI Conf. Artif. Intell., 2019, vol. 33,
pp. 4780–4789.
Cheng Fu is currently working toward the Ph.D.
degree with the University of California, San Diego. His
research interests include efficient ML algorithms for
hardware and machine learning for binary analysis. His
advisor is Professor Jishen Zhao. He interned at Face-
book AI research and worked on automating machine
learning project. Contact him at cfu@eng.ucsd.edu.
IEEE Micro
Huili Chen is currently working toward the Ph.D.
degree with the University of California, San Diego.
Her research interests span diverse aspects of
machine learning (ML) systems, including intellectual
property (IP) protection, adversarial attack detection,
and application of advanced ML algorithms to solve
long-standing problems in other domains. Contact
her at huc044@ucsd.edu.
Zhenheng Yang is currently a Research Scientist
with Facebook AI. His research interests include
computer vision and machine learning, specifically
weakly supervised learning, 3-D perception, and
activity reasoning. Yang received the Ph.D. degree
from the University of Southern California. Contact
him at zhenhengy@gmail.com.
Farinaz Koushanfar is currently a Professor and
Henry Booker Faculty Scholar with the Department of
Electrical and Computer Engineering, University of
California, San Diego, where she directs the Adap-
tive Computing and Embedded Systems Lab. She is
the Co-Founder and Co-Director of the University of
California, San Diego, Center for Machine-Integrated
September/October 2020
Computing and Security. Koushanfar received
the Ph.D. degree from the University of California,
Berkeley. She is a Fellow of the Kavli Foundation
Frontiers of the National Academy of Engineering.
Contact her at farinaz@ucsd.edu.
Yuandong Tian is currently a Research Scientist
and Manager with Facebook AI Research. His
research interests include theory and practice of deep
learning, sequential decision making, and computer
vision. Tian received the Ph.D. degree from the
Robotics Institute, Carnegie Mellon University. Contact
him at yuandong@fb.com.
Jishen Zhao is currently an Assistant Professor with
the Computer Science and Engineering Department,
University of California, San Diego. Her research
spans and stretches the boundary between computer
architecture and system software, machine learning,
and system codesign. Zhao received the Ph.D.
degree in computer science and engineering from
the Pennsylvania State University. Contact her at
jzhao@eng.ucsd.edu.
55
 Theme Article: Machine Learning for Systems

TSA-NoC: Learning-Based
Threat Detection and
Mitigation for Secure
Network-on-Chip
Architecture
Ke Wang, Hao Zheng, and Ahmed Louri
George Washington University

Abstract—Networks-on-chip (NoCs) are playing a critical role in modern multicore

architecture, and NoC security has become a major concern. Maliciously implanted
hardware Trojans (HTs) inject faults into on-chip communications that saturate the
network, resulting in the leakage of sensitive data via side channels and significant
performance degradation. While existing techniques protect NoCs by detecting and
isolating HT-infected components, they inevitably incur occasional inaccurate detection
with considerable network latency and power overheads. We propose TSA-NoC, a learning-
based design framework for secure and efficient on-chip communication. The proposed
TSA-NoC uses an artificial neural network for runtime HT-detection with higher accuracy.
Furthermore, we propose a deep-reinforcement-learning-based adaptive routing design for
HT mitigation with the aim of minimizing network latency and maximizing energy efficiency.
Simulation results show that TSA-NoC achieves up to 97% HT-detection accuracy, 70%
improved energy efficiency, and 29% reduced network latency as compared to state-of-the-
art HT-mitigation techniques.

& AS TECHNOLOGY SCALES, modern multiproces- computation-centric to communication-centric

sors have pushed for a paradigm shift from
Digital Object Identifier 10.1109/MM.2020.3003576
Date of publication 19 June 2020; date of current version
1 September 2020.
systems. Networks-on-Chip (NoCs) are becoming
increasingly critical yet vulnerable to various
security threats,1–4 especially to hardware trojans
(HTs).4 Maliciously implanted HTs inject transient
faults in transmitted flits/packets, causing mis-
routing and unnecessary packet re-transmissions.

56
0272-1732 ß 2020 IEEE Published by the IEEE Computer Society IEEE Micro
These retransmissions consume massive NoC
resources (e.g., link and buffer) and saturate the
transmission channels, resulting in data leakage,
significant performance degradation, and even a
denial-of-service.2,3
Significant research has been devoted to
securing NoCs against HT-based attacks.5,6 A
majority of these countermeasures mitigate HTs
by detecting and isolating the HT-infected NoC
components. In the HT-detection aspect, the
existing works use fault history logging (FHL),7
runtime threshold monitoring (RTM) on link-
error/packet-injection rates, or built-in self-test-
ing hardware.6 These techniques monitor fault-
related NoC attributes (e.g., temperature and
buffer/link utilization) at runtime and label the
corresponding component as HT-infected if any
attribute value exceeds its corresponding manu-
ally set threshold. However, since massive NoC
attributes are correlated with transient faults
and interact with each other,8 designing the
thresholds for HT detection is complicated.
These thresholds, if selected carelessly, can
cause false/misdetection, additional power con-
sumption, and increased network latency.
In the HT-isolation aspect, conventional solu-
tions use regional routing algorithms2,3 to isolate
the HT-infected components. Unfortunately,
these techniques limit the network throughput,
forbid communication via certain channels, and
detour the packets to avoid infected regions.
This inevitably increases network latency.
Therefore, the challenge is to design a secure
architecture that promptly and accurately
detects and isolates HTs with minimal perfor-
mance loss.
In this article, we propose TSA-NoC, a learn-
ing-based, high-performance, and energy-effi-
cient NoC design for secured on-chip
communication. In TSA-NoC, we enhance the
router architecture with a learning-based per-
router HT-detection (DetectANN) module,
bypass channel, and a SmartRoute controller for
HT-isolation. The major contributions of this
article include the following.

Improved HT detection using DetectANN:
DetectANN uses an artificial neural network
(ANN) to automatically identify HT-injected
faults by recognizing abnormal network
September/October 2020
behaviors (e.g., unexpected high error rate)
and improve the accuracy of HT detection.

Improved HT-isolation using SmartRoute: After
HT detection, the routers are dynamically cat-
egorized into HT-free and HT-infected routers.
A low-cost bypass channel using simple
switch logic is proposed to bypass the HT-
infected routers while maintaining network
connectivity. To balance traffic loads among
low-throughput bypass channels and high-
throughput routers and improve the overall
network performance, SmartRoute controller
uses DRL to handle diverse traffic patterns by
dynamically applying the most suitable rout-
ing algorithms thus minimizing network
latency and power consumption.
We evaluate the performance of the proposed
TSA-NoC architecture with PARSEC benchmark.
Simulation results show that TSA-NoC provides
enhanced HT-detection accuracy and improved
HT mitigation with reduced power consumption
and network latency as compared to conven-
tional HT-mitigation techniques.
BACKGROUND AND MOTIVATION
Transient Faults in NoC
Transient faults may manifest during any
stage of transmission8 in NoCs. To mitigate these
faults in typical NoC architecture, each flit is
encoded with error correction codes (ECCs)
before being propagated to the downstream
router. At each hop of transmission, the ECC
decoder examines the correctness of each flit
and initiates a negative-acknowledgment signal
for data retransmission if an error occurs.
HT Attacks on NoCs
HTs have been shown to infect NoC links9 and
router microarchitectures.5 Typically, HTs are
inserted in the layout during the fabrication phase
of the IC design cycle. After being implanted, they
usually remain dormant and slip past hardware
diagnosis until they are activated by attackers.
Once activated, they inject transient faults by forc-
ing bit-flips in links or disrupting ECC in routers.
These faults can lead to massive retransmission
traffic, back pressures, and network saturation.
57
 Machine Learning for Systems
Figure 1. Proposed TSA-NoC router architecture.
The router comprises a DetectANN design for
HT detection, a bypass channel for HT isolation, and a
SmartRoute controller. Solid and dashed arrows
represent data paths and control signals, respectively.
Conventional HT-Mitigation Techniques
A majority of the existing HT-mitigation techni-
ques use a detect–isolate approach.2,3,6,7 For HT
detection, built-in diagnosis hardware6 periodically
checks the correctness of the circuitry’s logic
operations, which inevitably stalls the application
execution to retain HT-detection accuracy. To miti-
gate this limitation, runtime software-based mech-
anisms such as FHL7 and RTM are proposed to
detect suspicious HT-infected components by
dynamically capturing abnormalities in NoC
behavior. FHL records the pattern of transient
faults in all the transmitted packets to identify HT-
infected channels. Similarly, RTM monitors the
retransmission rate and marks out suspicious HT-
infected components that exceed a certain retrans-
mission rate. However, both techniques rely on
manually-set thresholds to identify suspicious NoC
behavior, thus, often leading to false HT detection.
For HT isolation, SurfNoC3 eliminates the
transmissions between high-security and low-
security domains by alternatively reserving and
scheduling transmission channels in each
dimension exclusively for a specific domain.
NIBR2 partitions the virtual channels of each
router to transmit data flows exclusively with
high-security demands. However, the static net-
work partition results in poor network utiliza-
tion, thus, affecting network performance.
PROPOSED TSA-NoC DESIGN
We propose TSA-NoC, a learning-based HT-
detection and mitigation framework for secure
58
Figure 2. Overview of the proposed TSA-NoC
design.
NoC architecture. In TSA-NoC, we propose an
enhanced router architecture, as shown in
Figure 1, consisting of an improved HT-detection
design using an ANN (DetectANN), a bypass chan-
nel, and an enhanced HT-mitigation mechanism
(SmartRoute) to improve network performance
with DRL.
The overview of the proposed TSA-NoC
framework is shown in Figure 2. DetectANN mon-
itors network attributes, learns from runtime
network activities, and automatically identifies
HT-infected components by recognizing abnor-
mal network behaviors (e.g., high error rate and
retransmission). Based on the HT-detection
results from DetectANN, SmartRoute controller
categorizes packets into ones whose source and
destination nodes are HT-free (high-security
packets) and ones whose source and/or destina-
tion nodes are HT-infected (low-security pack-
ets). For high-security packets, all the
components in the transmission path should be
HT-free for security, while the transmission
paths of the latter packets are allowed to contain
HT-infected nodes. To isolate the HT-infected
routers to protect the high-security packets, a
bypass channel is proposed to bypass the HT-
infected routers while maintaining network con-
nectivity. Since the simple switch logic of the
bypass channel inevitably degrades the network
IEEE Micro
throughput, a routing mechanism that avoids
transmitting intense traffic through bypass
channels should be applied. Moreover, to better
utilize network resources, especially the isolated
routers, a different routing algorithm may be
needed for the low-security packets. To this end,
we propose SmartRoute, which proactively
selects the most suitable routing algorithm to bal-
ance traffic-loads in the low-throughput bypass
channel and high-throughput routers, respec-
tively, and improve overall network performance.
Learning-Based Runtime HT-Detection Using
DetectANN
We implement a per-router DetectANN to per-
form runtime HT-detection with improved accu-
racy with minimized timing and power overheads.
Unlike the static thresholds used by FHL and
RTM, DetectANN eschews the human engineering,
monitors NoC attributes, and automatically
detects HTs by learning how to accurately recog-
nize the abnormal behavior(s) of the local router
through complex and interrelated NoC attributes.
Since HTs are hard to detect when dormant, to
identify activated HTs in a timely manner while
reducing the computational overhead of Detec-
tANN, HT-detection is performed iteratively.
Construction of DetectANN. DetectANN is a
fully connected ANN with an input layer, a mid-
dle layer, and an output layer. Previous works
have shown that some system attributes are
highly correlated with transient faults in NoCs.8
In TSA-NoC, we explore 12 fault-related NoC
attributes as inputs, including buffer utilization
(number of occupied virtual channels) for each
input port (+x, x, +y, y, and local core), link
utilization (value of input-flits per cycle) for each
input port (+x, x, +y, y, and local core), local
operation temperature, and the previous tran-
sient error rate in the last epoch. The middle layer
utilizes all the attribute values and maps them to
the classification of whether the router is HT-
infected. As HT-detection is a binary classification
problem, a single-hidden-layer construction can
mitigate overfitting and is sufficient to deliver
desired accuracy. For optimized detection accu-
racy and computational/storage overhead, we
implement 30 neurons in the single hidden-layer
(detailed discussion is in the “Evaluation and Ana-
lysis” section). The middle layer uses Sigmoid
September/October 2020
activation function. For each neuron j in the mid-
dle layer, the output yj ¼ sigmoidðS12
i¼0 xi  wi Þ. The
output layer indicates the binary classification
result: HT-free or HT-infected.
Training the Proposed DetectANN. The pro-
posed DetectANN is trained offline. To build the
training set, applications are first executed in an
HT-free system while runtime attributes are
recorded. These attributes are used for the input
layer of DetectANN, and the desired output is
HT-free. Then, the same applications are exe-
cuted multiple times with HT-infected NoC com-
ponents. For a better training result, HTs are
randomly implanted each time. DetectANN moni-
tors the same attributes, and routers with
implanted HTs are labeled as HT-infected.
Avoidance of False-Positives/Negatives. As dis-
cussed, inaccurate HT-detection can lead to per-
formance degradation. False-positives can be a
problem when an HT-free router is always labeled
as HT-infected. In TSA-NoC, even if the HT-free
router is mistakenly labeled as HT-infected, the
detection result will be updated at the next epoch.
As the trained DetectANN has a high HT-detection
accuracy, the wrongly labeled router has a high
chance to be labeled correctly at the next epoch.
By doing so, the penalty of isolating that HT-free
router will be limited to one epoch. Therefore, the
false-positive problem can be mitigated. False-neg-
atives are common in conventional designs, in
which an HT-infected router is labeled as HT-free
when the HT is not activated. The proposed Detec-
tANN resolves this problem by monitoring the
runtime NoC behaviors consecutively and provid-
ing HT-detection results every 2000 cycles. As the
DetectANN utilizes the average attribute values
within the epoch, it is able to sensitively capture
the anomaly behavior of HT-infected routers, even
if the HTs are triggered on for a short period of
time. Therefore, the false-negative problem is
resolved.
Learning-Based Dynamic HT-Mitigation Using
SmartRoute
We propose a learning-based HT-mitigation
mechanism for efficient HT-isolation. We imple-
ment a bypass channel and a per-router Smart-
Route controller to dynamically route high-
security packets without traversing HT-infected
59
 Machine Learning for Systems
components and utilize the bypassed routers to
propagate low-security packets without degrad-
ing network performance. There is no need to
restrict the transmission paths of the low-secu-
rity packets, since they are already HT-infected.
When isolating HT-infected routers with
bypass channels, the simple switch logic of the
bypass channel could limit the throughput of
given path directions. TSA-NoC addresses this
problem by intelligently balancing traffic-loads
with various routing algorithms (O1TURN, West-
First, and Negative-First) using a SmartRoute con-
troller. The rationale behind this design is two-
fold: 1) avoid injecting into bypass channels and
2) optimize the worst case throughput of different
NoC traffic patterns.10 The O1TURN routing
dynamically applies XY or YX routing for each
packet to better utilize the network spatially under
normal traffic loads. West-First and Negative-First
restrict different types of turns and achieve lower
latency and less dynamic power consumption
than O1TURN under intense traffic loads.10 Note
that the TSA-NoC router has multiple virtual chan-
nels to avoid protocol and routing deadlocks.
Since the HT-detection results from Detec-
tANN vary periodically during runtime, selecting
the most suitable routing algorithm that can han-
dle the dynamic interactions between diverse
traffic patterns and limited NoC resources is
complex. Therefore, we propose the use of DRL
to automatically balance the tradeoffs among
the different routing algorithms to achieve bet-
ter system-level performance for high-security
and low-security packets.
DRL-Based Control Policy. The adaptive routing
algorithm is applied iteratively to avoid the timing
overhead incurred by NoC reconfiguration and
packet draining. The length of each iteration
(epoch) is identical to that of DetectANN. At each
epoch, the DRL-based SmartRoute controller moni-
tors NoC attributes and suggests an action (apply-
ing one of the routing algorithms) with the highest
expected long-term return11 in terms of network
performance and energy efficiency. The network
attributes will change with the action selection,
resulting in a new state at the next epoch. The
changes in performance and energy metrics are
evaluated to update the reward accordingly. The
DRL-based control policy continues to evolve
based on the NoC historical activities and
60
generates a direct map between the optimal action
and a given state. The problem formulation is as
follows.
State and Action Space. We select a set of net-
work-related attributes to represent the state
vector s for SmartRoute, which include the 12
attributes used in DetectANN, local router label
(HT-free or HT-infected), and packet injection rates
of different network dimensions. The action space
{a1 ; a2 ; a3 } comprises three routing algorithms:
O1TURN, West-First, and Negative-First.
Reward Function. The goal of the DRL agent is
to select actions that can maximize the long-term
return R for any given state. In SmartRoute, we
use Q-learning11 to estimate the expected long-
term return for each state-action pair, recorded as
R ¼ Qðs; aÞ. At each epoch, the agent selects the
action with the highest Qðs; aÞ. Next, it observes
the immediate reward r and the new state s0 . The
Qðs; aÞ value is updated using the following rule:
Qðs; aÞ ¼ ð1  aÞQðs; aÞ þ a½r þ g max Qðs;0 a0 Þ:
a0
(1)
The variables a and g are DRL parameters
called learning rate and discount rate, respec-
tively. The immediate reward r implies minimizing
the latency and power consumption. Therefore,
we define the immediate reward rt in (1) at epoch
t as follows:
rt ¼ ðPowert  Latencyt Þ1 (2)
The Latencyt and Powert values are obtained
by average end-to-end latency and power con-
sumption (static and dynamic), respectively.
The DRL agents select actions according to
the Q-table. To eliminate storing overheads, the
Q-table in conventional RL is replaced with a
neural network.
The working of the DRL of SmartRoute con-
troller is shown in Figure 3. At each epoch, the
router first uses the feature values in the state
vector s as inputs of the expanded ANN. The
ANN then calculates the Q-values of all the possi-
ble state-action pairs in the state entry. The
router suggests an action a, which has the maxi-
mum Qðs; aÞ-value for the next epoch. All routers
vote with their selected actions for packets that
require HT-free transmission paths, and the rout-
ing algorithm with the highest score is selected.
IEEE Micro
Figure 3. Working of DRL of SmartRoute.
Upon taking the action a, the NoC system transi-
tions to a new state s’. The NoC system then pro-
vides an immediate reward r, which is used to
update Qðs; aÞ. We implement a five-cycle win-
dow between two consecutive epochs to inform
routers of the upcoming actions and store on-
the-fly flits in router buffers.
EVALUATION AND ANALYSIS
Simulation Setup
We implement the proposed TSA-NoC architec-
ture in the Gem5 full-system simulator. We imple-
ment 64 out-of-order CPUs with 2-level cache in an
8  8 2D-mesh network. Additionally, we imple-
ment a runtime error injection module consisting
of NoC fault and thermal models (DSENT, HotSpot,
and VARIUS) to realistically simulate transient
errors. We compare the performance of TSA-NoC
(DetectANN+SmartRoute) with three techniques,
namely FHL+SurfNoC, DetectANN+SurfNoC, and
DetectANN+NIBR, with the PARSEC benchmark.
We train the DetectANN and SmartRoute with a
semi-real dataset generated with synthetic traffic
and part of PARSEC benchmark applications
(dedup, facesim, freqmine, and swaption). The
rest of real applications in PARSEC benchmark are
used in the testing phase.
Before executing each benchmark application,
we randomly select 10% of the total transmission
links in the NoC and implant HTs in them for run-
time fault injection. The testing phase of each
application lasts for the entire application execu-
tion time. The area overhead of TSA-NoC is evalu-
ated using the Synopsys design compiler with the
32-nm library.
September/October 2020
Figure 4. Simulation results of the proposed
TSA-NoC: (a) average end-to-end latency,
(b) average power consumption, and (c) average
energy efficiency. Results are normalized to the
FHL+SurfNoC baseline.
For DetectANN and SmartRoute, we set the
epoch size to 2000 cycles. The Q-values are ini-
tialized to 0. The learning rate a and the discount
rate g are set to 0.1 and 0.9, respectively. Addi-
tionally, the DRL agents have a small probability
of  = 0.05 to select a random action for exploring
unvisited state–action pairs.
Performance Analysis
Average Network Latency: Figure 4(a) shows
the normalized average end-to-end packet latency
of all the transmitted packets. TSA-NoC achieves
an average of 29% end-to-end latency reduction
over the baseline. Note that the proposed TSA-
NoC using SmartRoute can improve upon Detec-
tANN+SurfNoC by an additional 13% over baseline.
This illustrates that DRL-based dynamic routing
can further improve overall network latency.
Power Consumption: We evaluate static
and dynamic power consumption for all the
techniques used. For TSA-NoC, the power con-
sumption of the learning-based TSA-router (with
DetectANN and SmartRoute), intermediate links,
and bypass channels are included. We first model
the static power of all components with Synopsys
Design Compiler. Afterward, the captured power
61
 Machine Learning for Systems
Figure 5. Sensitivity study: (a) HT-detection
accuracy, (b) epoch size, (c) discount rate g, and
(d) -greedy factor.
parameters are fed to the full-system simulator for
accurate dynamic power simulation. Figure 4(b)
shows that TSA-NoC reduces overall power con-
sumption by an average of 18% over the baseline.
The majority of power saving is from dynamic
power reduction.
Energy Efficiency: Energy efficiency is defined
as: packetsenergy1 , where energy equals overall
power consumption (of all NoC components, the
proposed DetectANN, and SmartRoute) multiplies
benchmark execution time. Figure 4(c) shows that
the proposed framework improves energy effi-
ciency by an average of 70% compared to baseline.
HT-Detection Accuracy: The HT-detection accu-
racy is calculated with the ratio of the number of
identified HTs to the total number of implanted
HTs within a full execution of each benchmark. In
this simulation, we vary the middle layer size of
DetectANN. The DetectANN is trained with random
distributed HT-generated bit-flips, while in the test-
ing phase, the HT-generated bit-flips follow three
different distributions: normal, uniform, andPois-
son distribution. Figure 5(a) shows that, for all
distributions, the proposed DetectANN improves
HT-detection accuracy by 39% on average over the
FHL baseline, with 30 neurons in the middle layer.
Sensitivity Analysis
Impact of Middle Layer Size of DetectANN: We
vary the size of the middle layer to study its
impact on HT-detection accuracy in Figure 5(a).
The HT-detection accuracy improves as the size
of the middle layer increases. For the best accu-
racy, area consumption, and timing overhead,
we use 30 neurons in the middle layer.
62
Impact of Epoch Size of DRL: In this test, we
vary the length of DRL epoch from 1000 to
10 000 clock cycles. As shown in Figure 5(b),
increasing the epoch size negatively impacts
network latency and energy-delay product
(EDP) due to coarse-grain control (lower net-
work latency and EDP indicate better perfor-
mance). Alternatively, aggressively reducing
the length of epochs also leads to performance
degradation, as the timing overhead of DRL
would be notable.
Impact of Discount Rate g of DRL: Figure 5(c)
depicts the impact of the discount rate g on net-
work performance. As shown in Figure 5(c), both
network latency and EDP are initially improved
with larger g. However, aggressively increasing g
can also result in slow DRL convergence, which
leads to performance degradation. The best per-
formance is achieved when g equals 0.9.
Impact of Exploration Factor  of DRL: The
impact of  on network performance is shown in
Figure 5(d). As  increases, the agent explores
unvisited state-action pairs more frequently, which
is beneficial for training DRL. However, when 
equals 1, the router will take actions completely at
random. As shown in Figure 5(d), the best perfor-
mance is achieved when  equals 0.05.
Overhead Analysis
Timing Overhead: The timing overhead is
induced by calculating and updating the weights
for DetectANN and SmartRoute. In the worst case,
for each epoch, the computation overheads are 90
and 150 cycles for DetectANN and SmartRoute,
respectively. We use two sets of different epochs
for the monitoring and controlling to minimize the
negative effect of this latency. The two sets of
epochs are offset by the ANN computation time,
which can pipeline the overhead effectively. By
doing so, the calculation of ANNs does not block
monitoring or controlling. Therefore, the use of
ANN does not negatively impact the overall
performance.
Area and Power Overhead: The proposed
DetectANN and Smart-Route require additional
ALUs (adders, multipliers, and Sigmoid function)
and SRAM storage in each router. The proposed
DetectANN consumes additional 425.2-mm2 area
for ALUs and 718.7-mm2 area for SRAM, incurring
0.9% area overhead over a conventional router
IEEE Micro
in total. The DRL logics consume 956.7-mm2 ALU
area and 1617.2-mm2 SRAM area, which implies
2.1% area overhead. Furthermore, the power
overheads of DetectANN and DRL are 0.086 and
0.195 MW, respectively.
CONCLUSIONS
In this article, we proposed TSA-NoC, a learning-
enabled, high-performance, and energy-efficient
design framework for secure on-chip communica-
tion. The TSA-NoC consists of an ANN-based HT-
detection design (DetectANN) and a DRL-based
adaptive routing mechanism (SmartRoute). The
proposed DetectANN detects HT-infected NoC
components promptly and accurately at runtime.
SmartRoute isolates the HT-infected components
and deploys dynamic routing algorithms to opti-
mize system-level performance. Full-system evalua-
tions show that TSA-NoC improves HT-detection
accuracy, network latency, and energy efficiency
over existing techniques.
ACKNOWLEDGMENTS
This work was supported by NSF under
Grants CCF-1547035 and CCF-1702980.
& REFERENCES
1. H. Wassel et al., “Networks on chip with provable
security properties,” IEEE Micro, vol. 34, no. 3,
pp. 57–68, May/Jun. 2014.
2. T. Boraten and A. K. Kodi, “Securing NoCs against
timing attacks with non-interference based adaptive
routing,” in Proc. 12th IEEE/ACM Int. Symp.
Network-on-Chip, 2018, pp. 1–8.
3. H. Wassel et al., “SurfNoC: A low latency and provably
non-interfering approach to secure networks-on-chip,”
in Proc. 40th Annu. Int. Symp. Comput. Archit., 2013,
pp. 583–594.
4. M. Tehranipoor and F. Koushanfar, “A survey of
hardware trojan taxonomy and detection,” IEEE Des.
Test Comput., vol. 27, no. 1, pp. 10–25, Jan./Feb. 2010.
5. D. M. Ancajas, K. Chakraborty, and S. Roy, “Fort-NoCs:
Mitigating the threat of a compromised NoC,” in Proc.
51st ACM/EDAC/IEEE Des. Autom. Conf., 2014, pp. 1–6.
September/October 2020
6. K. Xiao and M. Tehranipoor, “BISA: Built-in self-
authentication for preventing hardware Trojan
insertion,” in Proc. IEEE Int. Symp. Hardware-Oriented
Secur. Trust, 2013, pp. 45–50.
7. H. Salmani, “COTD: Reference-free hardware trojan
detection and recovery based on controllability and
observability in gate-level netlist,” IEEE Trans. Inf.
Forensics Secur., vol. 12, no. 2, pp. 338–350, Feb.
2017.
8. K. Wang et al., “IntelliNoC: A holistic design
framework for energy-efficient and reliable on-chip
communication for manycores,” in Proc. 46th Annu.
Int. Symp. Comput. Archit., 2019, pp. 1–12.
9. Q. Yu and J. Frey, “Exploiting error control approaches
for hardware Trojans on network-on-chip links,” in
Proc. IEEE Int. Symp. Defect Fault Tolerance VLSI
Nanotechnol. Syst., 2013, pp. 266–271.
10. A. Singh. “Load-balanced routing in interconnection
networks,” Ph.D. thesis, Stanford Univ., Stanford, CA,
USA, 2005.
11. R. Sutton et al., Reinforcement Learning: An
Introduction. Cambridge, MA, USA: MIT Press, 2018.
Ke Wang is currently working toward the Ph.D. degree
in computer engineering with George Washington Uni-
versity. His research work focuses on high-perfor-
mance, energy-efficient, and reliable interconnect
designs using machine learning-based optimization.
Contact him at cory@gwu.edu.
Hao Zheng is currently working toward the Ph.D.
degree in computer engineering with George Washing-
ton University. His research interests are in the areas of
computer architecture and parallel computing, with
emphasis on on-chip interconnects. Contact him at
haozheng@gwu.edu.
Ahmed Louri is currently the David and Marilyn
Karlgaard Endowed Chair Professor of Electrical and
Computer Engineering with George Washington Uni-
versity. Louri received the Ph.D. degree in computer
engineering from the University of Southern California
in 1988. He conducts research in the broad area of
computer architecture and parallel computing. He is
a Fellow of IEEE, and currently serves as the Editor-
in-Chief of the IEEE TRANSACTIONS ON COMPUTERS. Con-
tact him at louri@gwu.edu.
63
 Guest Editor’s Introduction
Biology and Systems Interactions
Abhishek Bhattacharjee
Yale University
& BIOLOGICAL SYSTEMS CARRY out computation
and store information with efficiency levels and
complexity that far exceeds those of synthetic com-
puter systems. Consider, for example, that biologi-
cal neural networks are estimated to offer at least
four orders of magnitude with better ops/Joule
than synthetic neural networks, or the storage
advantages offered by molecular and DNA sub-
strates, or the fact that sophisticated molecular
pathways and mechanisms permit biological sys-
tems to interface with the most efficient sensing sys-
tems that we know of. At the same time, advancing
our understanding of complex biological systems
rests on continued scaling of the computational
capabilities of high-performance and efficient com-
puter systems, hardware, and software.
This special issue contains two articles on
this intersection of systems and biology. In the
first article, Alser, Bingol, Kim, Cali, Ghose,
Alkan, and Mutlu present an overview of the field
of genome analysis and the role of algorithmic
methods, hardware accelerators, and their inter-
play in their advancement. Genome analysis—
with its emphasis on measurement and compari-
son of features such as DNA sequence struc-
tures, variations, expressions, and regulation—
is an incredibly important and exciting field criti-
cal to the existential questions facing humankind
(not least because of its implications on the
COVID-19 pandemic). The authors focus on the
Digital Object Identifier 10.1109/MM.2020.3016852
Date of current version 1 September 2020.
64
0272-1732 ß 2020 IEEE Published by the IEEE Computer Society
read mapping stage of genome analysis pipe-
lines, because of its computational require-
ments, and offer researchers’ perspective on key
problems that may be ripe for future studies.
While the first article tackles the possibilities
offered by emerging hardware to advance the life
sciences, the second article studies the design
of tools and infrastructure to promote next-gen-
eration hybrid molecular–electronic systems.
This article, authored by Stephenson, Willsey,
McBride, Newman, Nguyen, Takahashi, Strauss,
and Ceze, focuses on a full-stack digital microflui-
dic platform for hybrid molecular–electronic sys-
tem studies, culminating in a study of DNA data
storage. The authors offer a vertical approach by
describing the design principles of the system,
its closed-loop operation with vision and capaci-
tive sensing, on-board magnetic bead extraction,
and much more. Of particular interest to com-
puter, architects may be the principles by which
the authors express computation and protocols
and use it to bridge the molecular and computa-
tional components of their microfluidic platform.
These two articles address but a small sample
of the myriad research questions at the intersec-
tion of the life sciences and computer systems. We
hope, however, that they spur research discus-
sions and studies that help propel a virtuous
research cycle between biology and computer sys-
tems design, where both fields advance from inno-
vations and ties with the other.
Abhishek Bhattacharjee is an Associate Profes-
sor of Computer Science at Yale University. Contact him
at yxie4@miners.utep.edu.
IEEE Micro
 Theme Article: Biology and Systems Interactions

Accelerating Genome
Analysis: A Primer on
an Ongoing Journey
Mohammed Alser Saugata Ghose
€rich
ETH Zu University of Illinois at Urbana–Champaign and
Carnegie Mellon University
€ lal Bingo
Zu €l
Bilkent University Can Alkan
Bilkent University
Damla Senol Cali
Carnegie Mellon University Onur Mutlu
ETH Zurich, Carnegie Mellon University, and
Jeremie Kim
Bilkent University
ETH Zurich and Carnegie Mellon University

Abstract—Genome analysis fundamentally starts with a process known as read mapping,

where sequenced fragments of an organism’s genome are compared against a reference
genome. Read mapping is currently a major bottleneck in the entire genome analysis pipeline,
because state-of-the-art genome sequencing technologies are able to sequence a genome
much faster than the computational techniques employed to analyze the genome. We
describe the ongoing journey in significantly improving the performance of read mapping. We
explain state-of-the-art algorithmic methods and hardware-based acceleration approaches.
Algorithmic approaches exploit the structure of the genome as well as the structure of the
underlying hardware. Hardware-based acceleration approaches exploit specialized
microarchitectures or various execution paradigms (e.g., processing inside or near memory).
We conclude with the challenges of adopting these hardware-accelerated read mappers.

& GENOME ANALYSIS is the foundation of many
scientific and medical discoveries, and serves as
a key enabler of personalized medicine. This
Digital Object Identifier 10.1109/MM.2020.3013728
Date of publication 3 August 2020; date of current version
1 September 2020.
analysis is currently limited by the inability of
modern genome sequencing technologies to
read an organism’s complete genome. Instead,
sequencing machines extract smaller random
fragments of an organism’s DNA sequence,
known as reads. While the human genome con-
tains over three billion bases (i.e., A, C, G, T in
DNA), the length of a read is orders of magnitude

September/October 2020 Published by the IEEE Computer Society 0272-1732 ß 2020 IEEE
65
 Biology and Systems Interactions
smaller, ranging from a few hundred bases (for
short reads) to a few million bases (for long
reads). Computers are used to perform genome
assembly, which reassembles read fragments
back into an entire genome sequence. Genome
assembly is currently the bottleneck to quickly
and accurately determining an individual’s entire
genome, due to the complex algorithms and large
datasets used for assembly.
A widely used approach for genome assembly
is to perform sequence alignment, which com-
pares read fragments against a known reference
genome (i.e., a complete representative DNA
sequence for a particular species). A process
known as read mapping matches each read gener-
ated from sequencing to one or more possible
locations within the reference genome, based on
the similarity between the read and the reference
sequence segment at that location. Unfortu-
nately, the bases in a read may not be identical
to the bases in the reference genome at the loca-
tion that the read actually comes from. These dif-
ferences may be due to 1) sequencing errors (up
to 0.1% in short reads and up to 20% in long
reads) during extraction; and 2) genetic muta-
tions that are specific to the individual organ-
ism’s DNA and may not exist in the reference
genome. Due to these potential differences, the
similarity between a read and a reference
sequence segment must be identified using an
approximate string matching (ASM) algorithm.
The possible genetic differences between the ref-
erence genome and the sequenced genome are
then identified using genomic variant calling
algorithms.
The ASM performed during read mapping
typically uses a computationally expensive
dynamic programming (DP) algorithm. This
time-consuming algorithm has long been a major
bottleneck in the entire genome analysis pipe-
line, accounting for over 70% of the execution
time of read mapping.1 The vast majority of read
mappers, such as the widely used minimap2,2
are implemented as software running on CPUs.
We refer readers to a comprehensive survey3 for
a discussion of state-of-the-art CPU-based read
mappers. Accelerating ASM can help bridge the
wide performance gap between sequencing
machines and CPU-based read mapping algo-
rithms, but faces three key challenges.
66
1) Due to the large datasets that a read mapper
operates on, it generates a large amount of
data movement between the CPU and main
memory. The CPU accesses off-chip main
memory through a pin-limited bus known as
the memory channel, and a high amount of
data movement across the memory channel
is extremely costly in terms of both execu-
tion time and energy.4,5
2) Modern sequencing machines generate read
fragments at an exponentially higher rate
than prior sequencing technologies, with
their growth far outpacing the growth in
computational power in recent years.6 For
example, the Illumina NovaSeq 6000 system
can sequence about 48 human whole
genomes at 30x genome coverage (the aver-
age number of times a genomic base is
sequenced) in about two days. However,
analyzing (performing mapping and variant
calling) the sequencing data of a single
human genome requires over 32 CPU hours
on a 48-core Intel Xeon processor, 23 of
which are spent on read mapping.7
3) The first two challenges worsen when a
metagenomic sample is profiled, where the
sample donor is unknown. This requires
matching the extracted reads to thousands
of reference genomes. Increasing the num-
ber of CPUs used for genome analysis
decreases the overall analysis time, but sig-
nificantly increases energy consumption
and hardware costs. Cloud computing plat-
forms are a potential alternative to distrib-
ute the workload at a reasonable cost, but
are disallowed due to data protection
guidelines in many countries.26
As a result, there is a dire need for new compu-
tational techniques that can quickly process and
analyze a tremendous number of extracted reads
in order to drive cutting-edge advances in the
genetic applications space.8 Many works boost
the performance of existing and new read map-
pers using new algorithms, hardware/software
co-design, and hardware accelerators. Our goal
in this work is to survey a prominent set of these
three types of acceleration efforts for guiding the
design of new highly efficient read mappers. To
this end, we 1) discuss various state-of-the-art
IEEE Micro
Figure 1. (a) Three steps of read mapping in genome analysis: 1) indexing, 2) pre-alignment filtering, and
3) sequence alignment. (b) Overview of the existing approaches to accelerating each step of read mapping.
mechanisms and techniques that improve the
execution time of read mapping using different
modern high-performance computing architec-
tures; and 2) highlight the challenges, in the last
section, that system architects and programmers
must address to enable the widespread adoption
of hardware-accelerated read mappers.
READ MAPPING
The main goal of read mapping is to locate
possible subsequences of the reference genome
sequence that are similar to the read sequence
while allowing at most E edits, where E is the
edit distance threshold. Commonly allowed edits
include deletion, insertion, and substitution of
characters in one or both sequences. Mapping
billions of reads to the reference genome is com-
putationally expensive.1,8,9 Therefore, most read
mapping algorithms apply two key heuristic
steps, indexing and filtering, to reduce the num-
ber of reference genome segments that need to
be compared with each read.
The three steps of read mapping are shown in
Figure 1(a). First, a read mapper indexes the
September/October 2020
reference genome by using substrings (called
seeds) from each read to quickly identify all
potential mapping locations of each read in the
reference genome. Second, the mapper uses fil-
tering heuristics to examine the similarity for
every sequence pair (a read sequence and one
potential matching segment in the reference
genome identified during indexing). These filter-
ing heuristics aim to eliminate most of the dis-
similar sequence pairs. Third, the mapper
performs sequence alignment (using ASM) to
check whether or not the remaining sequence
pairs that are identified by filtering to be similar
are actually similar. The alignment step exam-
ines all possible prefixes of two sequences and
tracks the prefixes that provide the highest pos-
sible alignment score (known as optimal align-
ment). The alignment score is a quantitative
representation of the quality of an alignment for
a given user-defined scoring function (computed
based on the number of edits and/or matches).
Alignment algorithms typically use DP-based
approaches to avoid re-examining the same pre-
fixes many times. These DP-based algorithms
provide the most accurate alignment results
67
 Biology and Systems Interactions
compared to other non-DP algorithms, but they
have quadratic time and space complexity [i.e.,
Oðm2 Þ for a sequence length of m]. Sequence
alignment calculates information about the align-
ment such as the alignment score, edit distance,
and the type of each edit. Edit distance is defined
as the minimum number of changes needed to
convert a sequence into the other sequence.
Such information is typically output by read
mapping into a sequence alignment/map (SAM)
file. Given the time spent on read mapping, all
three steps have been targeted for acceleration.
Figure 1(b) summarizes the different accelera-
tion approaches, and we discuss a set of such
works in the following sections.
ACCELERATING INDEXING
The indexing operation generates a table that
is indexed by the contents of a seed, and identi-
fies all locations where the seed exists in the ref-
erence genome. Indexing needs to be done only
once for a reference genome, and eliminates the
need to perform ASM across the entire genome.
During read mapping, a seed from a read is
looked up in the table, and only the correspond-
ing locations are used for ASM (as only they can
match the entire read). The major challenge
with indexing is choosing the appropriate length
and number of to-be-indexed seeds, as they can
significantly impact the memory footprint and
overall performance of read mapping.2 Querying
short seeds potentially leads to a large number
of mapping locations that need to be checked
for a string match. The use of long reads requires
extracting from each read a large number of
seeds, as the sequencing error rate is much
higher in long reads. This affects 1) the number
of times we query the index structure; and 2) the
number of retrieved mapping locations. Thus,
there are two key approaches used for accelerat-
ing the indexing step [see Figure 1(b)].
Reducing the Number of Seeds
Read mapping algorithms (e.g., minimap22)
typically reduce the number of seeds that are
stored in the index structure by finding the mini-
mum representative set of seeds (called minimiz-
ers) from a group of adjacent seeds within a
genomic region. The representative set can be
68
calculated by imposing an ordering (e.g., lexico-
graphically or by hash value) on a group of adja-
cent seeds and storing only the seed with the
smallest order. Read mappers also apply heuris-
tics to avoid examining the mapping locations of
a seed that occur more times than a user-defined
threshold value.2 Various data structures have
been proposed and implemented to both reduce
the storage cost of the indexing data structure
and improve the algorithmic runtime of identify-
ing the mapping locations within the indexing
data structure. One example of such data struc-
tures is FM-index (implemented by Langarita
et al.10), which provides a compressed representa-
tion of the full-text index, while allowing for query-
ing the index without the need for decompression.
This approach has two main advantages.
1) We can query seeds of arbitrary lengths, which
helps to reduce the number of queried seeds.
2) It typically has less (by 1:5  2) memory
footprint compared to that of the indexing
step of minimap2.2
However, one major bottleneck of FM-indexes is
that locating the exact matches by querying the
FM-index is significantly slower than that of clas-
sical indexes.10,11 BWA-MEM211 proposes an
uncompressed version of the FM-index that is at
least 10 larger than the compressed FM-index
to speed up the querying step by 2 .
Reducing Data Movement During Indexing
RADAR12 observes that the indexing step is
memory intensive, because the large number of
random memory accesses dominates computa-
tion. The authors propose a processing-in-mem-
ory (PIM) architecture that stores the entire
index inside the memory and enables querying
the same index concurrently using a large num-
ber of ASIC compute units. The amount of data
movement is reduced from tens of gigabytes to a
few bytes for a single query task, allowing
RADAR to balance memory accesses with com-
putation, and thus provide speedups and energy
savings.
ACCELERATING PRE-ALIGNMENT
FILTERING
After finding one or more potential mapping
locations of the read in the reference genome,
IEEE Micro
 Sidebar: Related Works on Pre-alignment Filtering Using
the Pigeonhole Principle
igeonhole-filtering-based pre-alignment filtering can
P accelerate read mappers even without specialized
hardware. For example, the adjacency filter [1] accelerates
sequence alignment by up to 19. The accuracy and
speed of pre-alignment filtering with the pigeonhole princi-
ple have been rapidly improved over the last seven years.
Shifted Hamming Distance (SHD) [2] uses SIMD-capable
CPUs to provide high filtering speed, but supports a
sequence length up to only 128 base pairs due to the SIMD
register widths. GateKeeper [3] utilizes the large amounts of
parallelism offered by FPGA architectures to accelerate SHD
and overcome such sequence length limitations. MAG-
NET [4] provides a comprehensive analysis of all sources of
filtering inaccuracy of GateKeeper and SHD. Shouji [5] lever-
ages this analysis to improve the accuracy of pre-alignment
filtering by up to two orders of magnitude compared to
both GateKeeper and SHD, using a new algorithm and a
new FPGA architecture. SneakySnake [6] achieves up to
four orders of magnitude higher filtering accuracy
compared to GateKeeper and SHD by mapping the
pre-alignment filtering problem to the single net routing
(SNR) problem in VLSI chip layout. SNR finds the shortest rout-
ing path that interconnects two terminals on the boundaries of
a VLSI chip layout in the presence of obstacles. SneakySnake is
the only pre-alignment filter that works on CPUs, GPUs, and
FPGAs. GenCache [7] proposes to perform highly parallel pre-
alignment filtering inside the CPU cache to reduce data move-
ment and improve energy efficiency, with about 20% cache
area overhead. GenCache shows that using different existing
pre-alignment filters together, each of which operates only for
a given edit distance threshold (e.g., using SHD only when E is
between 1 and 5), provides a 2.5 speedup over GenCache
with a single pre-alignment filter.
& REFERENCES
1. Hongyi Xin et al. Accelerating read mapping with
FastHASH. BMC Genomics, 2013.
2. Hongyi Xin et al. Shifted Hamming Distance: A fast and
accurate SIMD friendly filter to accelerate alignment
verification in read mapping. Bioinformatics, 2015.
3. Mohammed Alser et al. GateKeeper: A new hardware
architecture for accelerating pre-alignment in dna short
read mapping. Bioinformatics, 2017.
4. Mohammed Alser et al. MAGNET: Understanding and
improving the accuracy of genome pre-alignment filtering.
Transactions on Internet Research, 2017.
5. Mohammed Alser et al. Shouji: A fast and efficent
pre-alignment filter for sequence alignment. Bioinformatics,
2019.
6. Mohammed Alser et al. SneakySnake: A fast and accurate
universal genome pre-alignment filter for CPUs, GPUs, and
FPGAs. arXiv:1910.09020 [q-bio.GN], 2019.
7. Anirban Nag et al. GenCache: Leveraging in-cache operators
for efficient sequence alignment. in Proc. 52nd Int. Symp.
Microarchitecture, 2019.

the read mapper checks the similarity between
each read and each segment extracted at these
mapping locations in the reference genome.
These segments can be similar or dissimilar to the
read, though they share common seeds. To avoid
examining dissimilar sequences using computa-
tionally expensive sequence alignment algo-
rithms, read mappers typically use filtering
heuristics that are called pre-alignment filters. The
key idea of pre-alignment filtering is to quickly
estimate the number of edits between two given
sequences and use this estimation to decide
whether or not the computationally expensive
DP-based alignment calculation is needed—if not,
a significant amount of time is saved by avoiding
DP-based alignment. If two genomic sequences
differ by more than the edit distance threshold,
then the two sequences are identified as dissimi-
lar sequences and hence DP calculation is not
needed. In practice, only genomic sequence pairs
with an edit distance less than or equal to a user-
defined threshold (i.e., E) provide useful data for
most genomic studies.1,3,13 Pre-alignment filters
use one of four major approaches to quickly filter
out the dissimilar sequence pairs: 1) the pigeon-
hole principle; 2) base counting; 3) q-gram filter-
ing; or 4) sparse DP. Long read mappers typically
use q-gram filtering or sparse DP, as their perfor-
mance scales linearly with read length and inde-
pendently of the edit distance.
Pigeonhole Principle
The pigeonhole principle states that if E
items are put into E+1 boxes, then one or more

September/October 2020
69
 Biology and Systems Interactions
boxes would be empty. This principle can be
applied to detect dissimilar sequences and dis-
card them from the candidate sequence pairs
used for ASM. If two sequences differ by E edits,
then they should share at least a single subse-
quence (free of edits) among E+1 nonoverlap-
ping subsequences,1 where E is the edit
distance threshold. For a read of length m, if
there are no more than E edits between the read
and the reference segment, then the read and
reference segment are considered similar if they
share at most E+1 nonoverlapping subsequen-
ces, with a total length of at least m  E. The
problem of identifying these E+1 nonoverlap-
ping subsequences is highly parallelizable, as
these subsequences are independent of each
other. Shouji1 exploits the pigeonhole principle
to reduce the search space and provide a scal-
able architecture that can be implemented for
any values of m and E, by examining common
subsequences independently and rapidly with
high parallelism. Shouji accelerates sequence
alignment by 4.2-18.8 without affecting the
alignment accuracy. We refer the reader to the
sidebar for a brief discussion of several other
related works.
Base Counting
The base counting filter compares the num-
bers of bases (A, C, G, T) in the read with the cor-
responding base counts in the reference
segment. If one sequence has, for example, three
more Ts than another sequence, then their align-
ment has at least three edits. If the difference in
count is greater than E, then the two sequences
are dissimilar and the reference segment is dis-
carded. Such a simple filtering approach rejects
a significant fraction of dissimilar sequences
(e.g., 49.8%–80.4% of sequences, as shown in
GASSST14) and thus avoids a large fraction of
expensive verification computations required by
sequence alignment algorithms.
Q-Gram Filtering Approach
The q-gram filtering approach considers all of
the sequence’s possible overlapping substrings
of length q (known as q-grams). Given a sequence
of length m, there are m  q þ 1 overlapping
q-grams that are obtained by sliding a window of
length q over the sequence. A single difference in
70
one of the sequences can affect at most q over-
lapping q-grams. Thus, E differences can affect
no more than q  E q-grams, where E is the edit
distance threshold. The minimum number of
shared q-grams between two similar sequences
is therefore ðm  q þ 1Þ  ðq  EÞ. This filtering
approach requires very simple operations (e.g.,
sums and comparisons), which makes it attrac-
tive for hardware acceleration, such as in GRIM-
Filter.13 GRIM-Filter exploits the high memory
bandwidth and computation capability in the
logic layer of 3-D-stacked memory to accelerate
q-gram filtering in the DRAM chip itself, using a
new representation of reference genome that is
friendly to in-memory processing. q-gram filter-
ing is generally robust in handling only a small
number of edits, as the presence of edits in any
q-gram is significantly underestimated (e.g.,
counted as a single edit).
Sparse Dynamic Programming
Sparse DP algorithms exploit the exact
matches (seeds) shared between a read and a
reference segment to reduce execution time.
These algorithms exclude the corresponding
locations of these seeds from estimating the
number of edits between the two sequences, as
they were already detected as exact matches
during indexing. Sparse DP filtering techniques
apply DP-based alignment algorithms only
between every two nonoverlapping seeds to
quickly estimate the total number of edits. This
approach is also known as chaining, and is used
in minimap2.2
ACCELERATING SEQUENCE
ALIGNMENT
After filtering out most of the mapping loca-
tions that lead to dissimilar sequence pairs, read
mapping calculates the sequence alignment infor-
mation for every read and reference segment
extracted at each mapping location. Sequence
alignment calculation is typically accelerated
using one of two approaches: 1) accelerating the
DP-based algorithms using hardware accelerators
without altering algorithmic behavior; and
2) developing heuristics that sacrifice the opti-
mality of the alignment score solution in order to
reduce alignment time. With the first approach, it
IEEE Micro
is challenging to rapidly calculate sequence align-
ment of long reads with high parallelism. As long
reads have high sequencing error rates (up to
20% of the read length), the edit distance thresh-
old for long reads is typically higher than that for
short reads, which results in calculating more
entries in the DP matrix compared to that of short
reads. The use of heuristics (i.e., the second
approach) helps to reduce the number of calcu-
lated entries in the DP matrix and hence allows
both the execution time and memory footprint to
grow only linearly with read length (as opposed
to quadratically with classical DP). Next, we
describe the two approaches in detail.
Accurate Alignment Accelerators
From a hardware perspective, sequence align-
ment acceleration has five directions: 1) using
SIMD-capable CPUs; 2) using multicore CPUs and
GPUs; 3) using FPGAs; 4) using ASICs; and 5) using
PIM architectures. Traditional DP-based algo-
rithms are typically accelerated by computing
only the necessary regions (i.e., diagonal vectors)
of the DP matrix rather than the entire matrix, as
proposed in Ukkonen’s banded algorithm.27 This
reduces the search space of the DP-based algo-
rithm and reduces computation time. The num-
ber of diagonal bands required for computing the
DP matrix is 2E+1, where E is the edit distance
threshold. For example, the number of entries in
the banded DP matrix for a 2 Mb long read can be
1.2 trillion. Parasail15 and KSW2 (used in mini-
map22) exploit both Ukkonen’s banded algorithm
and SIMD-capable CPUs to compute banded align-
ment for a sequence pair with a configurable scor-
ing function. SIMD instructions offer significant
parallelism to the matrix computation by execut-
ing the same vector operation on multiple oper-
ands at once. KSW2 is nearly as fast as Parasail
when KSW2 does not use heuristics (explained in
the next section).
The multicore architecture of CPUs and GPUs
provides the ability to compute alignments of
many independent sequence pairs concurrently.
GASAL216 exploits the multicore architecture of
both CPUs and GPUs for highly parallel computa-
tion of sequence alignment with a user-defined
scoring function. Unlike other GPU-accelerated
tools, GASAL2 transfers the bases to the GPU,
without encoding them into binary format, and
September/October 2020
hides the data transfer time by overlapping GPU
and CPU execution. GASAL2 is up to 20 faster
than Parasail (when executed with 56 CPU
threads). BWA-MEM211 accelerates the banded
sequence alignment of its predecessor (BWA-
MEM) by up to 11:6 , by leveraging multicore
and SIMD parallelism. However, to achieve such
levels of acceleration, BWA-MEM2 builds an index
structure that is 6 larger than that of minimap2.
Other designs, such as FPGASW,17 exploit the
very large number of hardware execution units
in FPGAs to form a linear systolic array. Each
execution unit in the systolic array is responsi-
ble for computing the value of a single entry of
the DP matrix. The systolic array computes a sin-
gle vector of the matrix at a time. The data
dependency between the entries restricts the
systolic array to computing the vectors sequen-
tially (e.g., top-to-bottom, left-to-right, or in an
antidiagonal manner). FPGASW has a similar exe-
cution time as its GPU implementation, but is 4
more power efficient.
Specialized hardware accelerators (i.e., ASIC
designs) provide application-specific, power- and
area-efficient solutions to accelerate sequence
alignment. For example, GenAx18 is composed of
SillaX, a sequence alignment accelerator, and a
second accelerator for finding seeds. SillaX sup-
ports both a configurable scoring function and
traceback operations. SillaX is more efficient for
short reads than for long reads, as it consists of
an automata processor whose performance
scales quadratically with the edit distance.
GenAx is 31.7 faster than the predecessor of
BWA-MEM2 (i.e., BWA-MEM) for short reads.
Recent PIM architectures such as RAPID19
exploit the ability to perform computation inside
or near the memory chip to enable efficient
sequence alignment. RAPID modifies the DP-
based alignment algorithm to make it friendly to
in-memory parallel computation by calculating
two DP matrices: one for calculating substitu-
tions and exact matches and another for calcu-
lating insertions and deletions. RAPID claims
that this approach efficiently enables higher lev-
els of parallelism compared to traditional DP
algorithms. The main two benefits of RAPID and
such PIM-based architectures are higher perfor-
mance and higher energy efficiency,4,5 as they
alleviate the need to transfer data between the
71
 Biology and Systems Interactions
main memory and the CPU cores through slow
and energy hungry buses, while providing high
degree of parallelism with the help of PIM. RAPID
is on average 11:8 faster and 212:7 more
power efficient than 384-GPU cluster of GPU
implementation of sequence alignment, known
as CUDAlign.20
Heuristic-Based Alignment Accelerators
The second direction is to limit the functional-
ity of the alignment algorithm or sacrifice the opti-
mality of the alignment solution in order to reduce
execution time. The
use of restrictive
functionality and heu- Despite more than two
ristics limits the pos- decades of attempts,
sible applications of bridging the
the algorithms that performance gap
utilize this direction. between sequencing
Examples of limiting machines and read
mapping is still
functionality include
challenging.
limiting the scoring
function, or only tak-
ing into account accelerating the computation of
the DP matrix without performing the backtracking
step.21 There are several existing algorithms and
corresponding hardware accelerators that limit
scoring function flexibility. Levenshtein distance
and Myers’s bit-vector algorithm are examples of
algorithms whose scoring functions are fixed, such
that they penalize all types of edits equally when
calculating the total alignment score. Restrictive
scoring functions reduce the total execution time
of the alignment algorithm and reduce the bit-
width requirement of the register that accommo-
dates the value of each entry in the DP matrix.
ASAP22 accelerates Levenshtein distance calcula-
tion by up to 63:3 using FPGAs compared to its
CPU implementation. The use of a fixed scoring
function as in Edlib,23 which is the state-of-the-art
implementation of Myers’s bit-vector algorithm,
helps to outperform Parasail (which uses a flexible
scoring function) by 12–1000. One downside of
fixed function scoring is that it may lead to the
selection of a suboptimal sequence alignment.
There are other algorithms and hardware
architectures that provide low alignment time
by trading off accuracy. Darwin8 builds a cus-
tomized hardware architecture to speed up the
alignment process, by dividing the DP matrix
72
into overlapping submatrices and processing
each submatrix independently using systolic
arrays. Darwin provides three orders of magni-
tude speedup compared to Edlib.23 Dividing the
DP matrix (known as the Four-Russians Method)
enables significant parallelism during DP matrix
computation, but it leads to suboptimal align-
ment calculation.14 Darwin claims that choosing
a large submatrix size ( 320  320) and ensur-
ing sufficient overlap (128 entries) between
adjacent submatrices may provide optimal align-
ment calculation for some datasets.
There are other proposals that limit the num-
ber of calculated entries of the DP matrix based
on one of two approaches: 1) using sparse DP; or
2) using a greedy approach to maintain a high
alignment score. Both approaches suffer from
providing suboptimal alignment calculation.24,25
The first approach uses the same sparse DP algo-
rithm used for pre-alignment filtering but as an
alignment step, as done in the exonerate tool.24
The second approach is employed in X-drop,25
which 1) avoids calculating entries (and their
neighbors) whose alignment scores are more
than X below the highest score seen so far
(where X is a user-specified parameter); and
2) stops early when a high alignment score is not
possible. The X-drop algorithm is guaranteed to
find the optimal alignment between relatively
similar sequences for only some scoring func-
tions.25 A similar algorithm (known as Z-drop)
makes KSW2 at least 2:6 faster than Parasail.
DISCUSSION AND FUTURE
OPPORTUNITIES
Despite more than two decades of attempts,
bridging the performance gap between sequenc-
ing machines and read mapping is still challeng-
ing. We summarize four main challenges below.
First, we need to accelerate the entire read
mapping process rather than its individual steps.
Accelerating only a single step of read mapping
limits the overall achieved speedup according to
Amdahl’s Law. Illumina and NVIDIA have recently
started following a more holistic approach, and
they claim to accelerate genome analysis by more
than 48 , mainly by using specialization and hard-
ware/software codesign. Illumina has built an
FPGA-based platform, called DRAGEN (https://
IEEE Micro
www.illumina.com/products/by-type/informatics- within a portable sequencing device can poten-
products/dragen-bio-it-platform.html), that accel- tially enable new applications of genome sequenc-
erates all steps of genome analysis, including read ing (e.g., rapid surveillance of new diseases such
mapping and variant calling. DRAGEN reduces the as COVID-19, near-patient testing, bringing preci-
overall analysis time from 32 CPU hours to only sion medicine to remote locations). Unfortunately,
37 min.7 NVIDIA has built Parabricks, a software efforts in this direction remain very limited.
suite accelerated using the company’s latest Third, we need to develop flexible hardware
GPUs. Parabricks (https://developer.nvidia.com/ architectures that do not conservatively limit
clara-parabricks) can analyze whole human the range of supported parameter values at
genomes at 30 coverage in about 45 min. design time. Commonly used read mappers (e.g.,
Second, we need to reduce the high amount of minimap2) have different input parameters, each
data movement that takes place during genome of which has a wide range of input values. For
analysis. Moving data 1) between example, the edit distance thresh-
compute units and main memory; old is typically user defined and can
The acceleration efforts
2) between multiple hardware acc- be very high (15%–20% of the read
we highlight in this
elerators; and 3) between the length) for recent long reads. A con-
article represent state-
sequencing machine and the com- figurable scoring function is
of-the-art efforts to
puter performing the analysis reduce current another example, as it determines
incurs high costs in terms of execu- bottlenecks in the the number of bits needed to store
tion time and energy. These costs genome analysis each entry of the DP matrix (e.g.,
are a significant barrier to enabling pipeline. We hope that DRAGEN imposes a restriction on
efficient analysis that can keep up these efforts and the the maximum frequency of seed
with sequencing technologies, and challenges we discuss occurrence). Due to rapid changes
some recent works try to tackle provide a foundation in sequencing technologies (e.g.,
this problem.4,5,13 GenASM9 is a for future work in high sequencing error rate and lon-
framework that uses bitvector- accelerating read ger read lengths),28 these design
mappers and
based ASM to accelerate multiple restrictions can quickly make spe-
developing other
steps of the genome analysis pipe- cialized hardware obsolete. Thus,
genome sequence
line, and is designed to be imple- read mappers need to adapt their
analysis tools.
mented inside 3-D-stacked algorithms and their hardware
memory. Through a combination architectures to be modular and
of hardware–software co-design to unlock parallel- scalable so that they can be implemented for any
ism, and PIM to reduce data movement, GenASM sequence length and edit distance threshold
can perform 1) pre-alignment filtering for short based on the sequencing technology.
reads; 2) sequence alignment for both short and Fourth, we need to adapt existing genomic
long reads; and 3) whole genome alignment, data formats for hardware accelerators or
among other use cases. For short/long read align- develop more efficient file formats. Most sequenc-
ment, GenASM achieves 111/116 speedup over ing data is stored in the FASTQ/FASTA format,
state-of-the-art software read mappers while where each base takes a single byte (8 bits) of
reducing power consumption by 33/37. DRA- memory. This encoding is inefficient, as only
GEN reduces data movement between the 2 bits (3 bits when the ambiguous base, N, is
sequencing machine and the computer perform- included) are needed to encode each DNA base.
ing analysis by adding specialized hardware sup- The sequencing machine converts sequenced
port inside the sequencing machine for data bases into FASTQ/FASTA format, and hardware
compression. However, this still requires move- accelerators convert the file contents into unique
ment of compressed data. Performing read map- (for each accelerator) compact binary representa-
ping inside the sequencing machine itself can tions for efficient processing. This process that
significantly improve efficiency by eliminating requires multiple format conversions wastes
sequencer-to-computer movement, and embed- time. For example, only 43% of the sequence align-
ding a single specialized chip for read mapping ment time in BWA-MEM211 is spent on calculating

September/October 2020
73
 Biology and Systems Interactions
the DP matrix, while 33% of the sequence align-
ment time is spent on preprocessing the input
sequences for loading into SIMD registers, as pro-
vided by Ahmed et al.11 To address this ineffi-
ciency, we need to widely adopt efficient
hardware friendly formats, such as UCSC’s 2bit
format (https://genome.ucsc.edu/goldenPath/
help/twoBit), to maximize the benefits of hard-
ware accelerators and reduce resource utilization.
We are not aware of any recent read mapper that
uses such formats.
The acceleration efforts we highlight in this
article represent state-of-the-art efforts to
reduce current bottlenecks in the genome analy-
sis pipeline. We hope that these efforts and the
challenges we discuss provide a foundation for
future work in accelerating read mappers and
developing other genome sequence analysis
tools.
ACKNOWLEDGMENTS
The work of Onur Mutlu’s SAFARI Research
Group was supported by funding from Intel, the
Semiconductor Research Corporation, VMware,
and the National Institutes of Health (NIH).
& REFERENCES
1. M. Alser, H. Hassan, A. Kumar, O. Mutlu, and C. Alkan,
“Shouji: A fast and efficient pre-alignment filter for
sequence alignment,” Bioinformatics, vol. 35,
pp. 4255–4263, 2019.
2. H. Li, “Minimap2: Pairwise alignment for nucleotide
sequences,” Bioinformatics, vol. 34, pp. 3094–3100,
2018.
3. M. Alser et al., “Technology dictates algorithms:
Recent developments in read alignment”, 2020,
arXiv:2003.00110.
 mez-Luna, and
4. O. Mutlu, S. Ghose, J. Go
R. Ausavarungnirun, “Processing data where it makes
sense: Enabling in-memory computation,”
Microprocessors Microsyst., vol. 67, pp. 28–41, 2019.
 mez-Luna, and
5. S. Ghose, A. Boroumand, J. S. Kim, J. Go
O. Mutlu, “Processing-in-memory: A workload-driven
perspective,” IBM J. Res. Develop., vol. 63, no. 6,
pp. 3–1, 2019.
6. Z. D. Stephens et al., “Big data: Astronomical or
genomical?,” PLoS Biol., vol. 13, 2015, Art. no.
e1002195.
74
7. A. Goyal et al., “Ultra-fast next generation human
genome sequencing data processing using
DRAGENÒ Bio-IT processor for precision medicine,”
Open J. Genetics, vol. 7, pp. 9–19, 2017.
8. Y. Turakhia, G. Bejerano, and W. J. Dally, “Darwin: A
genomics co-processor provides up to 15,000x
acceleration on long read assembly,” in Proc.
Archit. Support Program. Lang. Oper. Syst., 2018,
pp. 199–213.
9. D. Senol Cali et al., “GenASM: A low-power, memory-
efficient approximate string matching acceleration
framework for genome sequence analysis,” in Proc.
53rd Int. Symp. Microarchitecture, 2020.
10. R. Langarita et al., “Compressed sparse FM-index:
Fast sequence alignment using large k-steps,” IEEE/
ACM Trans. Comput. Biol. Bioinformatics, to be
published, doi: 10.1109/TCBB.2020.3000253.
11. M. Vasimuddin, S. Misra, H. Li, and S. Aluru, “Efficient
architecture-aware acceleration of BWA-MEM for
multicore systems,” in Proc. IEEE Int. Parallel Distrib.
Process. Symp., 2019, pp. 314–324.
12. W. Huangfu, S. Li, X. Hu, and Y. Xie, “RADAR: A
3D-ReRAM based DNA alignment accelerator
architecture,” in Proc. Des. Autom. Conf., 2018, pp. 1–6.
13. J. S. Kim et al., “GRIM-filter: Fast seed location filtering in
DNA read mapping using processing-in-memory
technologies,” BMC Genomics, vol. 19, 2018, Art. no. 89.
14. G. Rizk and D. Lavenier, “GASSST: Global alignment
short sequence search tool,” Bioinformatics, vol. 26,
pp. 2534–2540, 2010.
15. J. Daily, “Parasail: SIMD C library for global, semi-
global, and local pairwise sequence alignments,” BMC
Bioinformatics, vol. 17, 2016, Art. no. 81.
vy, S. Ren, H. Mushtaq, K. Bertels, and
16. N. Ahmed, J. Le
Z. Al-Ars, “GASAL2: A GPU accelerated sequence
alignment library for high-throughput NGS data,” BMC
Bioinformatics, vol. 20, 2019, Art. no. 520.
17. X. Fei, Z. Dan, L. Lina, M. Xin, and Z. Chunlei,
“FPGASW: Accelerating large-scale Smith-Waterman
sequence alignment application with backtracking on
FPGA linear systolic array,” Interdisciplinary Sci.:
Comput. Life Sci., vol. 10, pp. 176–188, 2018.
18. D. Fujiki et al., “GenAx: A genome sequencing
accelerator,” in Proc. 45th Annu. Int. Symp. Comput.
Archit., 2018, pp. 69–82.
19. S. Gupta, M. Imani, B. Khaleghi, V. Kumar, and
T. Rosing, “RAPID: A ReRAM processing in-memory
architecture for DNA sequence alignment,” in Proc.
IEEE/ACM Int. Symp. Low Power Electron. Des., 2019,
pp. 1–6.
IEEE Micro
20. E. F. de Oliveira Sandes, G. Miranda, X. Martorell,
E. Ayguade, G. Teodoro, and A. C. Magalhaes Melo,
“CUDAlign 4.0: Incremental speculative traceback for
exact chromosome-wide alignment in GPU clusters,”
IEEE Trans. Parallel Distrib. Syst., vol. 27, no. 10,
pp. 2838–2850, Oct. 2016.
21. P. Chen, C. Wang, X. Li, and X. Zhou, “Accelerating
the next generation long read mapping with the
FPGA-based system,” IEEE/ACM Trans. Comput. Biol.
Bioinformatics, vol. 11, no. 5, pp. 840–852,
Sep.–Oct. 2014.
22. S. S. Banerjee et al., “ASAP: Accelerated short-read
alignment on programmable hardware,” IEEE Trans.
Comput., vol. 68, no. 3, pp. 331–346, Mar. 2019.
 
23. M. Sosic  c
 and M. Siki  , “Edlib: A C/C++ library for fast,
exact sequence alignment using edit distance,”
Bioinformatics, vol. 33, pp. 1394–1395, 2017.
24. G. S. C. Slater and E. Birney, “Automated generation of
heuristics for biological sequence comparison,” BMC
Bioinformatics, vol. 6, 2005, Art. no. 31.
25. Z. Zhang, S. Schwartz, L. Wagner, and W. Miller,
“A greedy algorithm for aligning DNA sequences,”
J. Comput. Biol., vol. 7, pp. 203–214, 2000.
26. L. Ben and N. Abhinav, “Cloud computing for genomic
data analysis and collaboration,” Nature Reviews
Genetics, vol. 19, no. 4, p. 208, 2018.
27. U. Esko, “Algorithms for approximate string matching,”
Inform. control, vol. 64, no. 1–3, pp. 100–118, 1985.
September/October 2020
28. D. Senol Cali, J. S. Kim, S. Ghose, C. Alkan, and
O. Mutlu, “Nanopore sequencing technology and tools
for genome assembly: Computational analysis of the
current state, bottlenecks and future directions,”
Briefings Bioinf., vol. 20, no. 4, pp. 1542–1559,
2019.
€rich.
Mohammed Alser is currently with ETH Zu
Contact him at alserm@inf.ethz.ch.
Zu€ lal Bingo € l is currently with Bilkent University. Con-
tact her at zulal.bingol@bilkent.edu.tr.
Damla Senol Cali is currently with Carnegie Mellon
University. Contact her at dsenol@andrew.cmu.edu.
Jeremie Kim €rich
is currently with ETH Zu
and Carnegie Mellon University. Contact him
at jeremie.kim@inf.ethz.ch.
Saugata Ghose is currently with the University of
Illinois at Urbana–Champaign and Carnegie Mellon
University. Contact him at ghose@illinois.edu.
Can Alkan is currently with Bilkent University. Con-
tact him at calkan@cs.bilkent.edu.tr.
Onur Mutlu is currently with ETH Zu €rich, Carnegie
Mellon University and Bilkent University. Contact him
at omutlu@gmail.com.
75
 Theme Article: Biology and Systems Interactions

PurpleDrop: A Digital
Microfluidics-Based
Platform for Hybrid
Molecular-Electronics
Applications
Ashley Stephenson, Max Willsey, Karin Strauss
Jeff McBride, and Sharon Newman University of Washington and Microsoft
University of Washington
Luis Ceze
Bichlien Nguyen University of Washington
University of Washington and Microsoft
Christopher Takahashi
University of Washington

Abstract—Molecular manipulation and analysis are the cornerstone of life sciences.

With the recent advances in molecular data storage and computing, it has become an
increasingly exciting and viable alternative for the post-CMOS scaling era. Widespread
use of molecular manipulation/analysis and data storage/computing requires a scalable
and low-cost platform for hybrid molecular-electronics systems. This enables us to build
on the best of what molecular and electronics systems can offer. In this article, we
present PurpleDrop, a full-stack digital microfluidic platform for hybrid molecular-
electronic systems in multiple domains, and focus on DNA data storage as a use case.
We describe its design principles and relevant aspects such as closed-loop operation

Digital Object Identifier 10.1109/MM.2020.3005615

Date of publication 30 June 2020; date of current version
1 September 2020.

76
0272-1732 ß 2020 IEEE Published by the IEEE Computer Society IEEE Micro
with computer vision and capacitive sensing, on-board magnetic bead extraction, and
polymerase chain reaction, among other primitives. Importantly, we emphasize the ability
to express and execute protocols and computation that include molecular and
computational components.
& THE SEMICONDUCTOR INDUSTRY has seen a
profound trend in miniaturization driven by sci-
entific and technological advancements that
have enabled us to generate, manipulate, and
store massive amounts of data. However, as we
approach the physical limits of conventional
storage and computing hardware, alternative
approaches must be investigated.
DNA data storage systems are a prime exam-
ple of molecule-based systems capable of achiev-
ing far greater information density (1018 bytes/
mm3 ) and stability (thousands of years) than con-
ventional magnetic, optical, and solid state stor-
age technologies. DNA is also likely to have
eternal format relevance, owing to interest in
DNA for health and biotechnology applications.
Molecular systems uniquely require macro-
level physical storage and manipulation, e.g.,
mixing, splitting, heating, or diluting various
molecules in solution. At present, these opera-
tions are most commonly manually performed
by humans in laboratories. Automation is key if
we are to develop sophisticated and robust
hybrid molecular-electronic systems1 that
combine the best features of electronic com-
puting with the benefits of molecular density
and stability.
We have developed a digital microfluidic
(DMF) automation platform, PurpleDrop, in
response to this need and demonstrated its
potential in hybrid molecular-electronic sys-
tems. PurpleDrop aims to 1) minimize cost and
hardware complexity, 2) maximize scalability,
and 3) maximize modularity. Furthermore, Pur-
pleDrop is designed with a larger system in
mind: we have designed a complementary soft-
ware stack to make the most of DMF’s flexibility
without sacrificing reliability.
In this article, we present the PurpleDrop
platform and demonstrate its application in
molecular computing. Using data storage as a
case study, we first provide background on mod-
ern DNA data storage and present the need for
future automation and scale. Building on
September/October 2020
previous work demonstrating DNA data storage
with microfluidic retrieval via PurpleDrop,2 we
now turn our attention to automating subse-
quent steps in the DNA data storage pipeline.
Finally, we provide a detailed description of
the PurpleDrop platform and explain the func-
tional considerations that informed its design.
HYBRID MOLECULAR-ELECTRONIC
SYSTEMS
Electronic systems are well understood, and
can perform a wide variety of tasks in extremely
specific and controllable ways. Molecular sys-
tems, however, offer remarkable advantages in
performance and energy efficiency.
In the molecular domain, computation occurs
in the same solution in which data-containing
molecules reside, making it possible to perform
many molecular
computational pro-
In this article, we cesses in parallel.
present the PurpleDrop
Performing an opera-
platform and
tion on a few mole-
demonstrate its
cules takes the same
application in
molecular computing. amount of time as for
trillions of molecules,
with the caveat that
some operations could take hours to complete.
Given these qualities, molecular systems are par-
ticularly well suited for applications with large
data sets that can handle the high latency that
comes along with the high throughput.
In the case of DNA data storage, combining
the extreme density and parallelism of the
molecular domain with the precision of elec-
tronic systems could yield a more efficient
and robust system than either domain in
isolation.
Archival DNA Data Storage
Figure 1 shows a DNA data storage system.
Such systems encode binary data into DNA
sequences of nucleotides (A, C, G, and T) that are
77
 Biology and Systems Interactions

Figure 1. DNA data storage workflow. Digital data is encoded into a nucleotide sequence used to synthesize
DNA “files.” DNA files can be stored in pools containing many files if they are tagged with unique identifiers
(corresponding to PCR primers). To recover the original digital data, random access selectively retrieves and
amplifies only the file(s) of interest from the pool. The selected files are then sequenced and decoded. A
notable gap between synthesis and sequencing exists, and automation of this workflow requires an efficient
solution for the physical storage and retrieval of DNA data.

translated into manufactured molecules (or oligo-
nucleotides) by a chemical process and stored in
molecular pools. To recover the stored informa-
tion, the DNA molecules must be sequenced (read)
and decoded back into digital data. Previous work
in DNA data storage has covered synthesis, encod-
ing and decoding, DNA preservation, random
access protocols, and sequencing technologies.
However, achieving high density data storage
in practice will require an automated mechanism
for physical storage, organization, and retrieval
of files in molecular form, a need illustrated by
the gap in the modern DNA data storage pipeline
depicted in Figure 1.
Physical Storage and Random Access
The manner in which DNA data is physically
stored in a system is an essential consider-
ation in how it is retrieved. DNA data must be
in solution for sample preparation and
sequencing steps, but storing DNA in liquid
form requires separate vessels that reduce
information density. DNA can be easily isolated
and more densely stored when dehydrated,
and multiple file pools can be stored in close
proximity if dehydrated onto a substrate such
as glass.2
To “write” DNA files to a storage substrate,
samples must be deposited in a spatially isolated
arrangement, treated with preservatives, and
dehydrated onto the surface for indefinite stor-
age. The “read” process involves rehydrating a
sample with water, performing random access
via polymerase chain reaction (PCR) (or other
molecular methods), and preparing the sample
for sequencing via PCR and other heat depen-
dent operations.
Random access allows specific data to be
retrieved without wastefully sequencing an
entire pool of data. DNA files are tagged during
synthesis with unique addresses (short strands
of DNA corresponding to specific files) that
enable retrieval of only those strands. Larger
addresses use more nucleotides in the strand,
but allow more data to be stored in a pool.
Storing DNA in physically isolated pools ena-
bles information density gains because physi-
cally isolated pools can share the same
molecular address space. This leads to a hierar-
chical addressing scheme where the address of
a file encodes both the physical location of the
pool and molecular address within that pool.2
Molecular Random Access Methods
Random access can be implemented in vari-
ous ways in molecular systems.
PCR is widely used in molecular biology to
rapidly make many copies of specific DNA sam-
ples in solution. It involves mixing a number of
reagents, including enzymes, small sequences of
single-stranded DNA called primers, individual
nucleotides, and the DNA to be copied, and con-
trolling their temperature, typically done by an
instrument called a thermocycler. When the tem-
perature is raised, DNA “melts,” and the two
sides of its double helix come apart. As the solu-
tion cools, primers attach to the single-stranded
sides of the DNA, creating double-stranded sec-
tions of DNA. This enables enzymes to also
attach at the double-stranded to single-stranded

78 IEEE Micro
Figure 2. (a) PurpleDrop’s modular features enable fluidic operations essential to hybrid molecular-
electronic systems such as automated DNA data storage. (PurpleDrop in a hybrid molecular-electronic
system.) (b) PurpleDrop is comprised of two main PCBs: a parent board that houses electronic components
(A), and a removable child board that contains the electrode surface (B). Peripheral hardware systems include
three on-board heaters (F), a computer vision system consisting of a camera on 3-D printed mount (C), and a
fluid input-output system powered by miniature peristaltic pumps (D) that pump fluids onto and off the board
via capillary tube ports (E). A motorized magnet (not pictured) is mounted on the underside of the electrode
board. (The PurpleDrop DMF device.)

transitions and perform a copy of the DNA by PURPLEDROP: A SIMPLE, FLEXIBLE

adding complementary individual nucleotides
as it traverses the molecule. In DNA data sto-
rage systems, molecules representing a file are
tagged with a unique primer target site (address).
PCR-based random access selects specific DNA
files from a pool (molecular random access)
using the unique primers associated to those
files, after the pool is physically retrieved from a
library containing multiple DNA pools (physical
coordinate random access).
Separation-based purification methods such as
magnetic bead wash can also be used to accom-
plish random access. In this modality, magnetic
beads coated with DNA strands that selectively
bind to a specific file in the pool are used to
extract files via separation and washing. Purified
samples are amplified via PCR and sequenced.
At present, retrieval, purification, and amplifi-
cation steps are typically performed by humans
and expensive, bulky thermocyclers. Stream-
lined, scalable DNA data storage will require
fluid handling solutions for integrated molecular
processing steps such as file retrieval and PCR.
DMF DEVICE
Microfluidic technology has potential appli-
cations in a broad range of areas such as syn-
thetic biology, medical diagnostics, automated
experimentation, and molecular computing. Pur-
pleDrop was inspired by its DNA data storage
use case, but designed with a greater vision in
mind: a full-stack microfluidic platform that is
cost effective, reliable, and capable enough to be
the foundation of scalable computer systems
with molecular components.
The PurpleDrop hardware, shown in Figure 2
(b), and complementary software stack, Puddle,3
manage design tradeoffs by keeping hardware
costs low and compensating with sophisticated
software. PurpleDrop is comprised of consumer
materials, and can be assembled outside of a
clean room, with an assembled form factor of
approximately 110  140  100 mm.
PurpleDrop’s modular features, depicted in
Figure 2(a), can be combined in numerous ways
to enable operations such as those required for
DNA data storage. Together, PurpleDrop and

September/October 2020
79
 Biology and Systems Interactions
Puddle provide a platform that could allow users
to easily combine computation and fluidic opera-
tions to implement hybrid molecular-electronic
systems.
Microfluidics Background
Microfluidic devices can perform chemical
and biological protocols at smaller scales, lower
cost, and higher precision than humans.4,5 These
devices all center around the manipulation of
small fluid volumes, but different approaches
offer different tradeoffs among size, cost, flexibil-
ity, and reliability.
Channel-based devices are manufactured
ahead-of-time as a set of channels designed to
meet the needs of a particular protocol. These
devices can offer some flexibility by including
configurable valves,6,7 but their design is largely
fixed-function. On the other hand, liquid han-
dling robots feature mechanical arms or gantries
that hold pipettes. The robots can be program-
matically controlled, offering a great deal of flexi-
bility, but they are large and expensive, costing
from many thousands to hundreds of thousands
of dollars.
A more recent technology, DMF moves drop-
lets around a grid of electrodes using a phenom-
enon called electrowetting on dielectric
(EWOD).4,5 In EWOD DMF, droplets sit on a sub-
strate material that is patterned with electrodes,
insulated by a dielectric layer, and coated with a
hydrophobic layer. The dielectric layer prevents
electrolysis of droplets and the hydrophobic
layer reduces the surface energy in contact with
the droplets, making them easier to move. Some
devices have a second plate with a hydrophobic
coating added on top, effectively sandwiching
the droplets between the two plates. The top
plate typically consists of a continuous ground
electrode made from conductive, transparent
indium tin oxide glass.
Applying voltages to electrodes changes the
wettability of droplets on the surface. To move a
droplet, an electrode adjacent to it is activated
while the electrode directly beneath it is deacti-
vated, generating an electrowetting force that
pulls the droplet onto the active electrode. This
process is repeated to move droplets along any
path of adjacent electrodes.
80
DMF devices can be highly programmable,
offering flexibility at small size and low cost.
These features are highly desirable features in
the molecular computing space, where automa-
tion and parallelization are essential.
Key PurpleDrop Features and Operations
PurpleDrop is designed to support several
operations required by automated DNA data
storage while navigating tradeoffs among cost,
reliability, and scalability.
Moving Droplets Electrode substrates are
commonly made from silicon or glass making
them smooth and level, but requiring costly
clean room processes. Printed circuit boards
(PCBs) can also be used as cheaper, more dura-
ble substrates, sacrificing some smoothness.8
PCBs can also accommodate multiple electrical
wiring layers, increasing addressablility, while
glass and silicon substrates (outside of expen-
sive integrated circuit-style fabrication) are typi-
cally limited to one layer with conduction lines
patterned onto the surface.
PurpleDrop follows OpenDrop’s9 design with
a removable “child” electrode board that con-
sists of a two layer PCB with an array of 127 elec-
trodes. The parent PCB contains the electrical
components. Child electrode boards can be eas-
ily recoated or replaced upon wear or contami-
nation with minimal cost and effort.
The dielectric and hydrophobic layers have
similarly significant influence over performance.
PurpleDrop’s electrode PCB is coated with a
25.4-mm-thick polyimide tape dielectric layer
(Caplinq, PIT0.5S-UT/100), with a top coat of
Teflon AF 1600 (spin-coated at 98 xg for 60 s,
DuPont/Chemours) for hydrophobicity. We
chose polyimide tape over more traditional DMF
dielectric materials such as Parylene due to its
advantages in cost, dielectric strength, durabil-
ity, and fabrication complexity. Polyimide tape
is inexpensive, costing about $0.07 per electrode
board before scale. In contrast, Parylene C costs
approximately $100 per board.
Furthermore, in previous design iterations
using Parylene C (5 mm applied via vapor deposi-
tion), we faced significant challenges with the
gaps between electrodes interfering with droplet
movement. The copper electrodes are patterned
IEEE Micro
Figure 3. (a) Block diagram of capacitance sensing circuitry: An analog integrator circuit and microcontroller
are used to measure the total charge transferred while selected electrodes are charged and (b) the measured
capacitance plotted for varying droplet sizes.
with 1oz of copper, which corresponds to a
height of approximately 1.4 mil above the FR4.
Droplets frequently snagged on electrode edges
when transitioning between electrodes as a
result, resulting in droplet fragmentation/immo-
bilization. In contrast to conformal coatings like
Parylene, the polyimide tape forms a relatively
level surface that covers the gaps in between
electrodes and allows droplets to smoothly tran-
sition between electrodes.
Fluid Management We have implemented
computer vision and capacitance sensing sub-
systems that provide dynamic droplet tracking,
volume detection, and error correction to
enhance reliability.
Volume sensing and droplet tracking provide
critical information about the number and size of
droplets on the board. This allows a higher level
system like Puddle to perform fluid resource man-
agement by automatically placing and routing
droplets. Furthermore, these features enable
dynamic error correction where the system vali-
dates that the operations occurring on the board
match those executed by the software.
The computer vision system3 consists of a
Raspberry Pi camera on a 3-D-printed mount.
Droplets are detected based on color.
The presence of a conductive fluid between an
electrode and the top-plate increases the capaci-
tance between the two conductors, with the
amount of the increase depending on the area
covered by the fluid. The capacitance sensing
system measures this capacitance to detect the
September/October 2020
presence of a droplet and measure its volume.
The PurpleDrop electrodes and top-plate are
switched between the high voltage supply (VHV )
and 0 V at 500 Hz to generate an ac actuation volt-
age. During each cycle, the total charge transfer
is measured while the active electrodes are
driven from 0 V to VHV , by integrating the voltage
across a sense resistor connected between the
top-plate and its driver. This provides capaci-
tance measurement at a rate of 500 Hz, which is
significantly faster than a typical camera and
image processing. Figure 3(a) shows a diagram of
the capacitance measurement system.
In order to measure the relationship of capac-
itance to drop volume, a drop of water was
placed on PurpleDrop and slowly enlarged by
adding approximately 0.5 uL at a time. At each
step, a camera was used to measure the area
covered by the drop, and capacitance was
recorded. Volume was calculated using the
known height of the gap between the top-plate
and the electrode board. Results for this test are
shown in Figure 3(b).
Capacitance sensing has several advantages
over the computer vision system: lower latency,
lower computational load, and robustness to
lighting changes. Using capacitance sensing in
conjunction with computer vision could lead to
more robust and precise sensing than either sys-
tem alone.
Polymerase Chain Reaction Filler fluids
such as silicone oil are commonly used in two-
plate DMF devices to reduce droplet resistance,
81
 Biology and Systems Interactions

Figure 4. DNA data storage pipeline with microfluidic storage, retrieval, and sequencing preparation. Digital
data is encoded into nucleotide sequences and synthesized into DNA. DNA files are stored in spatially
isolated pools of dehydrated DNA, organized into a “library” of pools. Each library is contained on a glass
cartridge with a unique ID, and these cartridges are stored in deck. An address system mapping files to
specific libraries, pools, and PCR primers is used to locate files. Cartridges are loaded onto PurpleDrop for
microfluidic file retrieval with magnetic-bead-based random access. Retrieved files are amplified via PCR and
output to a sequencer for data recovery.

lower the driving voltage requirements, and slow
evaporation;4 however, fluid media complicate
certain operations. Inputting or removing drop-
lets from the device is much simpler in an air
medium, where introducing unwanted air bub-
bles is not a concern. We found that using an air
medium was also advantageous for thermocy-
cling protocols like PCR. When using a silicone
oil medium (Clearco PSF–2cSt), we observed
bubbles beginning to form around droplets at
the heating site around 80 C, with significant
bubbling and fragmentation of the droplet above
90C. This made it impossible to move droplets
away from the heaters for the remaining steps in
the protocol. We decided instead to operate in
air and utilize the computer vision and fluid
input/output systems to provide automated
droplet replenishment.3
PurpleDrop’s fluid input/output system is
driven by small peristaltic pumps (Takasago Flu-
idic Systems), each fitted with a capillary tube
that serves as the interface between the pump
and the electrode surface. The other port of the
pump is fitted with tubing that can be coupled to
arbitrary reservoirs or devices.
PurpleDrop has three heaters used for ther-
mocycling, and compensates for the increased
droplet evaporation in air by using the compu-
ter vision system (capacitance sensing could
also be used) to monitor droplet volumes and
provide real-time feedback to Puddle. When
droplets fall below a threshold volume, a replen-
ishment water droplet is pumped onto the board
and carried to the reaction site. This process
could extend the number of cycles of PCR possi-
ble indefinitely.
Bead Extraction The magnetic-bead-based
random access implementation discussed ear-
lier is well matched with PurpleDrop’s DNA data
storage architecture and hardware. PurpleDrop
is equipped with a motorized magnet that facili-
tates magnetic bead wash. A small servo motor
mounted near the underside of the electrode
board moves a magnet underneath droplets
containing magnetic beads. Once in place, the
magnet causes the beads to aggregate at the bot-
tom of the droplet. With the magnet held in
place, the supernatant droplet (i.e., the fluid
without the beads being held by the magnet)
is moved away via electrowetting until comp-
lete separation from the pellet [see Figure 6(a)].
After separation, washing and resuspension steps
complete the purification process.
Implementing Hybrid Molecular-Electronic
Primitives With PurpleDrop
To demonstrate how PurpleDrop can automate
DNA data storage and random access, we imple-
mented several key operations: file storage and
retrieval in a DNA library, magnetic-bead-based
random access for file selection from a pool, and
file amplification for sequencing preparation.
DNA Data Storage and Retrieval We previ-
ously evaluated PurpleDrop as a file retrieval mech-
anism in a high density DNA data storage library.2
Libraries consist of dehydrated pools of DNA files
organized in arrays on the droplet-facing sides of
PurpleDrop’s glass top plates, referred to as car-
tridges when containing DNA data (see Figure 4).
Water droplets are used to retrieve and prepare
files for sequencing, as depicted in Figure 5.

82 IEEE Micro
Figure 5. Puddle code snippet (a) and associated diagram (b) depicting DNA file location and retrieval via
PurpleDrop. File locations are given in cardinal coordinates with C referring to the central file in the
configuration shown. A key maps each file to a physical location in library and a PCR primer. Retrieved files
can be output directly to a sequencer. (c) Sequencing results from the work by Amin et al.2 show successful
recovery of the retrieved central file. (a) Puddle retrieval code. (b) DNA file retrieval. (c) Sequencing coverage
of each discovered file in cardinal coordinates (log scale).

Using this architecture, we demonstrated that
it is possible to store dehydrated files in DNA on
PurpleDrop (with an estimated storage density of
50 TB per 50-  50-mm cartridge), and success-
fully retrieve and recover the files without con-
tamination. This storage scheme allows for
physical isolation, enabling pools to share the
same addressing space and increasing the stor-
age capacity of the system. Microfluidic retrieval
provides the needed actuation for accessing and
transporting data, with the added benefit of intro-
ducing near-data processing opportunities.

Figure 6. (a) Motorized magnet is moved underneath the electrode board and positioned under the droplet
containing DNA and functionalized magnetic beads. The magnetic beads are pulled out of solution with
specific DNA attached and the supernatant droplet is moved away. (b) Retrieval with file A-specific beads
results in qPCR amplification of File A that takes place in fewer cycles than for file B and the control groups.
We attribute the latter to primer–dimer formation during qPCR. PCR amplification on PurpleDrop could be
calibrated to amplify only file A by limiting the number of cycles to 30, i.e., after amplification of file A and
before amplification of file B. (a) Magnetic bead wash on PurpleDrop. (b) Quantitative PCR (qPCR)
amplification for magnetic-bead-based random access.

September/October 2020
83
 Biology and Systems Interactions

For DNA data storage to become a reality out- shorter than file strands, and can be detected
side of the research laboratory, scalable physical by measuring the strand lengths of the qPCR
storage with automated file retrieval, as well as reaction products. The control bead experiments
automated file preparation for sequencing are showed comparable late-cycle amplification of
imperative. We envision PurpleDrop filling the both files, which confirms that no file selection
gap in the data pipeline between DNA synthesis took place. These results suggest that Purple-
and sequencing, as illustrated in Figure 1. In addi- Drop could be used to locate and selectively
tion to file retrieval, random access, PCR, and retrieve files stored in high density libraries.
other sequencing preparation steps are promis-
ing candidates for DMF automation.
PCR PurpleDrop is well positioned to automate
Magnetic-Bead-Based Random Access protocols such as PCR at miniaturized scales
To demonstrate magnetic-bead- that reduce reagent consumption, equipment
based random access on Pur- cost, and human involvement. Using
We have presented the automated replenishment
pleDrop, we synthesized a pool
PurpleDrop, a DMF approach described in “Key Purple-
containing two unique DNA
platform designed with Drop Features and Operations” sec-
files, files A and B, in equal con-
hybrid-molecular
centrations. Magnetic beads tion, we demonstrated the first fully
systems in mind.
(Invitrogen, 65601) functional- automated execution of PCR with
PurpleDrop provides
ized with (i.e., attached to) pri- replenishment in air.3 We have also
necessary features for
mers for file A were synthesized implementing these demonstrated basic sequencing
to selectively extract file A from systems, i.e., storing, preparation steps on PurpleDrop,
the pool containing both files retrieving, and with automated output to a
via bead wash. Control beads manipulating DNA, and sequencer through the fluidic input/
functionalized with a dummy integrates with output system.3 PurpleDrop’s versa-
primer (i.e., coated with a computing systems tile features enable a diverse set of
that can provide higher fluidic operations that can be com-
primer not found in the pool to
level control. bined to implement automated DNA
reduce nonspecific binding that
can occur with uncoated beads) storage and retrieval in a low-cost,
were used to confirm selectivity. scalable, and efficient way.
All reagents were loaded onto the electrode
board, and the magnetic beads were mixed and
incubated with the file pool. The magnet was DISCUSSION
then used to extract the beads from the droplet We have presented PurpleDrop, a DMF plat-
as described in the “Key PurpleDrop Features form designed with hybrid-molecular systems
and Operations” section. Following separation, in mind. PurpleDrop provides necessary fea-
multiple washing steps with 80% ethanol were tures for implementing these systems, i.e.,
performed to remove any unbound molecules storing, retrieving, and manipulating DNA, and
from the pelleted beads. Finally, resuspension integrates with computing systems that can
buffer was used to elute and output the purified provide higher level control. We expect this
sample. Quantitative PCR (qPCR) was used to combination of programmability and biological
assess random access performance [see Figure 6 primitives to be critical for future hybrid-
(b)]. qPCR of the PurpleDrop-purified sample molecular systems.
showed amplification of file A, with some amplifi- While this work focused on data storage, we
cation of file B appearing to occur multiple cycles have designed PurpleDrop to capture the flexi-
later. We attribute the latter to primer–dimer for- bility and accessibility of DMF technology in a
mation during qPCR. Primer–dimers are common general way. Combined with future work, Purple-
PCR byproducts that form when primers attach Drop and other DMF devices could impact a
to each other and undergo enzymatic strand diverse range of domains from diagnostics to
extension. These strands are generally much high throughput experimentation.

84 IEEE Micro
ACKNOWLEDGMENTS
This work was supported in part by a Grant
from DARPA under the Molecular Informatics
program, in part by the National Science Founda-
tion EAGER Grant 1841188, and in part by a spon-
sored research agreement and gifts from
Microsoft.
& REFERENCES
1. D. Carmean, L. Ceze, G. Seelig, K. Stewart, K. Strauss,
and M. Willsey, “Dna data storage and hybrid
molecular-electronic computing,” Proc. IEEE, vol. 107,
no. 1, pp. 63–72, Jan. 2019.
2. S. Newman et al., “High density DNA data storage
library via dehydration with digital microfluidic
retrieval,” Nature Commun., vol. 10, no. 1, 2019,
Art. no. 1706.
3. M. Willsey et al., “Puddle: A dynamic, error-correcting,
full-stack microfluidics platform,” in Proc. 24th Int. Conf.
Architectural Support Program. Lang. Oper. Syst.,
2019, pp. 183–197.
4. K. Choi, A. H. Ng, R. Fobel, and A. R. Wheeler, “Digital
microfluidics,” Annu. Rev. Analytical Chem., vol. 5,
no. 1, pp. 413–440, 2012.
5. R. B. Fair, “Digital microfluidics: Is a true lab-on-a-chip
possible?” Microfluidics Nanofluidics, vol. 3,
pp. 245–281, 2007.
6. A. M. Amin, M. Thottethodi, T. Vijaykumar, S. Wereley,
and S. C. Jacobson, “Aquacore: A programmable
architecture for microfluidics,” ACM SIGARCH
Comput. Architecture News, vol. 35, no. 2,
pp. 254–265, 2007.
7. J. Paul Urbanski, W. Thies, C. Rhodes, S. Amarasinghe,
and T. Thorsen, “Digital microfluidics using soft
lithography,” Lab. Chip, vol. 6, no. 1, 2006,
Art. no. 96104.
8. J. Gong and C. Kim, “Direct-referencing two-
dimensional-array digital microfluidics using multilayer
printed circuit board,” J. Microelectromech. Syst., vol.
17, no. 2, pp. 257–264, 2008.
9. M. Alistar and U. Gaudenz, “Opendrop: An integrated
do-it-yourself platform for personal use of biochips,”
Bioengineering, vol. 4, no. 2, 2017, Art. no. 45.
September/October 2020
Ashley Stephenson is currently working toward
the Ph.D. degree in computer science and engineer-
ing with the University of Washington, Seattle. Her
research is focused on applying microfluidic technol-
ogy and machine learning for intelligent automation
in synthetic and molecular biology. Stephenson
received the B.S. degree in biomedical engineering
and electrical engineering from the University of
Michigan, Ann Arbor. Contact her at ashsteph@cs.
washington.edu.
Max Willsey is currently working toward the Ph.D.
degree in computer science and engineering with the
University of Washington, Seattle. His research inter-
ests center around programming languages, with a
current focus on developing programming models for
microfluidic chips. Willsey received the B.S. degree in
computer science from Carnegie Mellon University.
Contact him at mwillsey@cs.washington.edu.
Jeff McBride is currently a Research Engineer with
the Molecular Information Systems Lab, University of
Washington, Seattle. His research is focused on
developing robust and affordable microfluidic tech-
nologies for wet lab automation. McBride received
the M.S. degree in computer engineering from Vir-
ginia Commonwealth University. Contact him at
mcbridejc@gmail.com.
Sharon Newman is currently working toward a
Ph.D. degree in bioengineering with Stanford Univer-
sity. Her research interests are focused on increas-
ing access to medical technologies, with a focus on
combining bioinformatics, image processing, and
biological assay development for early disease
detection. Newman received the B.S. degree in bio-
engineering from the University of Washington, Seat-
tle. Contact her at newmans@stanford.edu.
Bichlien Nguyen is currently a Senior Researcher
with Microsoft Research. Her research interests
include sustainable computing and DNA data storage
systems. Nguyen received the Ph.D. degree in chem-
istry from Washington University in St. Luis. Contact
her at bnguy@microsoft.com.
85
 Biology and Systems Interactions
Chris Takahashi is currently a Senior Research
Scientist with the Molecular Information Systems Lab,
University of Washington, Seattle. His research inter-
ests include DNA data storage systems and wet lab
automation. Takahashi received the Ph.D. degree in
synthetic biology from the University of Washington.
Contact him at cnt@cs.washington.edu.
Karin Strauss is currently a Principal Research
Manager with Microsoft, and an Affiliate Professor in
computer science and engineering with the Univer-
sity of Washington, Seattle. Her research lies at the
intersection of computer architecture, systems, and
86
biology. Strauss received the Ph.D. degree in com-
puter science from the University of Illinois at
Urbana-Champaign. Contact her at kstrauss@micro-
soft.com.
Luis Ceze is currently a Professor in computer sci-
ence and engineering with the University of Washing-
ton, Seattle. His research focuses on the intersection
of computer architecture, programming languages,
machine learning, and molecular biology. Ceze
received the Ph.D. degree in computer science from
the University of Illinois at Urbana-Champaign, Cham-
paign. Contact him at luisceze@cs.washington.edu.
IEEE Micro
Evolving Career
Opportunities
Need Your Skills
Explore new options—upload your resume today

Changes in the marketplace shift demands for vital skills and talent. The
IEEE Computer Society Jobs Board is a valuable resource tool to keep job
seekers up to date on the dynamic career opportunities offered by employers.
Take advantage of these special resources for job seekers:

JOB ALERTS TEMPLATES WEBINARS

CAREER RESUMES VIEWED

ADVICE BY TOP EMPLOYERS

No matter what your career level, the IEEE Computer Society Jobs Board
keeps you connected to workplace trends and exciting career prospects.

www.computer.org/jobs
 Department: Micro Economics
Triggers, Transmissions,
and Adjustments
Shane Greenstein
Harvard Business School
& TAKE A STEP back from the daily details of
events. Compare the recession unfolding in the
United States at this moment with the two previ-
ous downturns. Today’s economic events share
a surprising set of common features with the
dot-com boom and bust of 1997 to 2001, and
fewer similarities with the financial meltdown of
2008–2009. For reasons I will explain later, we
ought to be thankful for the latter.
To be sure, these are not obvious similarities.
How will be arrive at such an observation? These
similarities arise from the economic processes
behind triggers, transmissions, and adjustments.
While that might sound like the vocabulary of a
foreign language, the explanation should be intui-
tive to many at technology firms, who tend to be
first responders to the triggers, transmissions,
and adjustments of economic recessions.
SKELETON OF RECESSIONS
A trigger is the event that interrupts an econ-
omy, usually moving it away from a path of eco-
nomic growth. Money can be saved or made from
foreseeing a trigger sooner than later, so analysts
often jump the gun on their arrival. That said, trig-
gers do not arise out of the ether. Usually a trigger
Digital Object Identifier 10.1109/MM.2020.3014536
Date of current version 1 September 2020.
88
0272-1732 ß 2020 IEEE Published by the IEEE Computer Society
runs counter to a prevailing view. A prevailing
view is a general set of opinions and theories held
by most analysts. Until the trigger arrives, the pre-
vailing view remains largely consistent with
observable facts and trends.
Triggers are never completely surprises,
however. Usually somebody can see the disaster
coming and, unwittingly, plays the role of a mod-
ern Cassandra, with their warnings not heeded.
For example, some analysts warned about unjus-
tified valuations of startups during the dot-com
boom, and some professionals foresaw the
issues inherent from nontransparent and mis-
priced securities for subprime mortgages. After
the fact, dissertations have been written about
why their warnings caught the attention of so
few listeners.
Transmission mechanisms turn triggers into
recessions by either lowering demand in many
parts of the economy or interrupting supply, or
both, and at the same time. As an example, the
dot-com bust eliminated wealth of many invest-
ors, and—oversimplifying for the sake of brev-
ity—reduced consumption and shuttered value
chains affiliated with many online businesses.
That led to a redirection of aggregate invest-
ment. As another example, the collapse of Leh-
man Brothers in 2008 generated a liquidity crisis
at virtually every U.S. bank, and—oversimplify-
ing again—that hindered lending activity every-
where, because no bank was sure how to price
IEEE Micro
risky assets. Shuttered value chains and less Despite that complexity, one thing always hap-
lending led to unemployment. pens during these moments: news outlets and ana-
There are professional economists who spend lysts find ample audiences for publically accessible
their entire career studying every angle of every information and analysis, because readers and
trigger and transmission in these types of events viewers urgently need to adjust their behavior to
in every country that has ever experienced them. match new trends. Analysts forecast, learn to reas-
So it is necessarily an oversimplification to charac- sess as events unfold, update, and keep forecast-
terize what can happen. That said, based on ing. The hunger for updated information reaches a
watching many other countries, analysts had spec- rare intensity.
ulated that a major pandemic could act as a trigger Nobody can predict when the present hunger
for an economic recession in the United States. for information will subside, but, as in every
As this pandemic played out, the Corona other prior situation, it will do so eventually. I
Virus generated a rapid decline in consumption in should stress “eventually.” In complicated eco-
the spring at restaurants, bars, stadiums, stages, nomic times, such as in the United States right
parks, schools, and other places where people now, slow adjustment toward a prevailing view
gather. It also interru- is more likely than fast adjustment.
pted supply of plenty Nobody can predict As a historical illustration, consider the
of entertainment, am- when the present emergence of a new prevailing view around the
ong other immediate hunger for information dot-com boom and bust. It is hard to remember,
effects, as well as im- will subside, but, as in but the new prevailing view took a long time to
posed costs on medi- every other prior emerge. The first downturn in spring of 2000
cal services in the situation, it will do so met with considerable resistance among opti-
areas where infection eventually. I should mistic investors, who stuck to the old prevailing
spiked. The reactions stress “eventually.” In view that things would bounce back. It took
complicated economic
to this trigger have more than half and a year to convince most
times, such as in the
concentrated in some investors that revenue would not cover the
United States right
regions and not costs in most of these businesses. Then invest-
now, slow adjustment
others, and at differ- toward a prevailing ors had to relearn which metrics were prescient
ent times, as if Jack- view is more likely than and which were misleading. It took several
son Pollock painted fast adjustment. years before new startups of online firms grew
the economic effects again.
all over a U.S. map.
There have been myriad reactions, also typically UPDATES
localized. It will be a long time before analysts settle
Broadly construed, few of the adjustments to most open questions, and a new prevailing view
this transmission surprised anybody. Someone emerges. As illustration for why, let us tally
who loses a job usually looks for alternatives, some of the changes in the prevailing view in the
and if they fail, they cut back on their own con- last few months, and recognize the challenges
sumption. If investors perceive a decline in for technology firms.
opportunity, they usually redirect their efforts Since March there have been vociferous pub-
to new opportunities with better returns, or cut lic debates about the trigger itself. Every firm in
back on investments. Firms who foresee a social media has had to follow these develop-
decline in demand usually cut back on produc- ments, and decide what action, if any, to take
tion, and reduce staff. against misinformation. What is the rate of infec-
However, the economy experienced a gazil- tion? (It varies by location.) Does it spread by
lion adjustments all at once. Everywhere you air? (Yes, primarily this way). Do masks slow
look in all directions. Because everyone adjusts down the spread? (Yes, but some designs are
at the same time, that makes the sum total of better than others.) Does the summer heat kill
this behavior difficult to forecast in the aggre- it? (No, it is not like the flu.) Can bad cases be
gate. The whole differs from the sum of its parts. treated in a hospital? (Some can, but not easily.)

September/October 2020
89
 Micro Economics

The misinformation will not cease anytime soon, conversation is urgent, extensive, constantly
least of all when a vaccine emerges, so these shifting, and unceasing. It is not confined to
firms will continue to be challenged. experts. It infuses every public square that hosts
Another set of open question haunts the the conversation. Many technology firms find
firms who supply software to organizations who themselves altering their core strategies and
permitted remote work on a temporary basis, actions due to changes in the frontier of that
and have begun to consider sustaining those conversation.
practices. For example, Google just told its
employees to work remotely through January,
REACHING A PREVAILING VIEW
2021. More broadly, the extent of experimenta- Frequent changes in the prevailing view, as
tion has yielded a new stream of insights about
we presently are experiencing, is a symptom of
best practices, and firms in the office software bad economic times. Good economies are much
business have been scrambling to keep up. Do
more boring.
not expect the implications for product design It is not as if Cassandras took over the prevail-
to settle anytime soon.
ing view today, but most professionals now believe
A related debate has been taking place among that most parts of the economy have started to
data carriers, who built capacity to support heavy
decline. Here are some grim facts: The second
data loads in downtown areas. quarter of 2020 was the worst eco-
Remote work this spring traffic
nomic quarter in the United States
moved data loads to residential Remote work this since the 1940s, and every large
areas during the day time. Mainte- spring traffic moved
bank in the United States has put
nance and investment had to be data loads to
aside tens of billions of dollars for
redirected, and quickly. Those car- residential areas during
anticipated bankruptcies.
riers have begun to plan for next the day time.
Maintenance and Unlike the financial crisis of 2008,
year. Should they expect more of however, there has not been any
investment had to be
the same next spring? Once again,
redirected, and destruction from a financial panic.
that is an open question.
quickly. Those carriers That could have happened, but did
As it turned out, some open
have begun to plan for not. This is the sense in which the
questions about adjustments res- next year. Should they present economy resembles the
olved themselves quickly. Just not expect more of the financial crisis. In the spring, the
many. For example, many medical same next spring? Federal Reserve implemented a pol-
practices have adopted a range of icy to introduce liquidity into the
communications technologies for
financial system, and to help localities with their
telemedicine consults. In light of the years of resis- financing. The situation contained the same poten-
tance to telemedicine, feel free to blame your favor-
tial for disaster. Why not the same outcome? The
ite government entity for needed a crisis to finally Fed learned from the experience of 2009.
get around to approving it. I prefer to give credit to
If policy makers had not learned, it could
all the effort that went into hardware and software have been even worse. It is one of the few bright
investment that finally made this last step
spots in an otherwise bleak situation.
incremental.
In other words, just as occurred 20 years ago Shane Greenstein is a Professor at the Harvard
during the dot-com boom and its aftermath, the Business School. Contact him at sgreenstein@hbs.edu.

90 IEEE Micro
PURPOSE: The IEEE Computer Society is the world’s largest
association of computing professionals and is the leading provider
of technical information in the field.
MEMBERSHIP: Members receive the monthly magazine
Computer, discounts, and opportunities to serve (all activities
are led by volunteer members). Membership is open to all IEEE
members, affiliate society members, and others interested in the
computer field.
COMPUTER SOCIETY WEBSITE: www.computer.org
OMBUDSMAN: Direct unresolved complaints to
ombudsman@computer.org.
CHAPTERS: Regular and student chapters worldwide provide the
opportunity to interact with colleagues, hear technical experts,
and serve the local professional community.
AVAILABLE INFORMATION: To check membership status, report
an address change, or obtain more information on any of the
following, email Customer Service at help@computer.org or call
+1 714 821 8380 (international) or our toll-free number,
+1 800 272 6657 (US):
• Membership applications
• Publications catalog
• Draft standards and order forms
• Technical committee list
• Technical committee application
• Chapter start-up procedures
• Student scholarship information
• Volunteer leaders/staff directory
• IEEE senior member grade application (requires 10 years
practice and significant performance in five of those 10)
PUBLICATIONS AND ACTIVITIES
Computer: The flagship publication of the IEEE Computer Society,
Computer publishes peer-reviewed technical content that covers
all aspects of computer science, computer engineering,
technology, and applications.
Periodicals: The society publishes 12 magazines‫ژ‬ƌȄư‫ژז׏ژ‬DZȏɓȵȄƌǹȽ.
Refer to membership application or request information as noted
above.
Conference Proceedings & Books: Conference Publishing
Services publishes more than 275 titles every year.
Standards Working Groups: More than 150 groups produce IEEE
standards used throughout the world.
Technical Committees: TCs provide professional interaction in
more than 30 technical areas and directly influence computer
engineering conferences and publications.
Conferences/Education: The society holds about 200 conferences
each year and sponsors many educational activities, including
computing science accreditation.
Certifications: The society offers three software developer
credentials. For more information, visit
www.computer.org/certification.
BOARD OF GOVERNORS MEETING
‫ژגא‬٫‫ דאژ‬°ƷȲɋƷȂƨƷȵ‫ژ׎א׎אژ‬ǠȄ‫ژ‬uƩkƷƌȄً‫ژ‬ÝǠȵǒǠȄǠƌً‫ژ‬Å°

revised ‫ڳת‬uƌɲ‫ש׫ש׫ڳ‬
EXECUTIVE COMMITTEE
President: kƷǠǹƌ‫ژ‬%Ʒ‫ژ‬FǹȏȵǠƌȄǠ
President-Elect: FȏȵȵƷȽɋ‫ژ‬°ǚɓǹǹ
Past President: ƷƩǠǹǠƌ‫ژ‬uƷɋȵƌ
First VP: ¨ǠƩƩƌȵưȏ‫ژ‬uƌȵǠƌȄǠ; Second VP: °ɲ‫ٯ‬äƷȄ‫ژ‬hɓȏ ‫ژژژژژ‬
Secretary: %ǠȂǠɋȵǠȏȽ‫ژ‬°ƷȵȲƌȄȏȽ; Treasurer: %ƌɫǠư‫ژ‬kȏȂƷɋ
VP, MemberȽǚǠȲ & Geographic Activities: Yervant Zorian
VP, Professional & Educational Activities: °ɲ‫ٮ‬äƷȄ‫ژ‬hɓȏ ‫ژژژژژژژژژژژ‬
VP, Publications: Fabrizio Lombardi
VP, Standards Activities: Riccardo Mariani
VP, Technical & Conference Activities: William D. Gropp‫ژ‬
2019–2020 IEEE Division VIII Director: Elizabeth L. Burd‫ژ‬
‫ژ׏א׎אٮ׎א׎א‬U---‫ژ‬%ǠɫǠȽǠȏȄ‫ژ‬Ý‫ژ‬%ǠȵƷƩɋȏȵ‫¾ژي‬ǚȏȂƌȽ‫ژ‬uِ‫ژ‬ȏȄɋƷ‫ژژژژژژژژژژ‬
‫ژ׎א׎א‬U---‫ژ‬%ǠɫǠȽǠȏȄ‫ژ‬ÝUUU‫ژ‬%ǠȵƷƩɋȏȵ‫ٮ‬-ǹƷƩɋ‫ژي‬ǚȵǠȽɋǠȄƌ‫ژ‬uِ‫ژ‬°ƩǚȏƨƷȵ
BOARD OF GOVERNORS
¾ƷȵȂ‫ ژ‬-ɱȲǠȵǠȄǒ‫
ژي׎א׎א ژ‬Ȅưɲ‫ ژِ¾ ژ‬ǚƷȄً‫ ژ‬eȏǚȄ‫ ژ‬%ِ‫ ژ‬eȏǚȄȽȏȄً‫ژ‬
°ɲ‫ٮ‬äƷȄ‫ ژ‬hɓȏً‫ ژ‬%ƌɫǠư‫ ژ‬kȏȂƷɋً‫ ژ‬%ǠȂǠɋȵǠȏȽ‫ ژ‬°ƷȵȲƌȄȏȽً‫ژژژژژ‬
Oƌɲƌɋȏ‫ژ‬äƌȂƌȄƌ
¾ƷȵȂ‫ ژ‬-ɱȲǠȵǠȄǒ‫ ژي׏א׎א ژ‬uِ‫ ژ‬ȵǠƌȄ‫ ژ‬ǹƌǵƷً‫ ژ‬FȵƷư‫ ژ‬%ȏɓǒǹǠȽً‫ژ‬
ƌȵǹȏȽ‫ ژ‬-ِ‫ ژ‬eǠȂƷȄƷɼ‫ٮ‬GȏȂƷɼً‫¨ ژ‬ƌȂƌǹƌɋǚƌ‫ ژ‬uƌȵǠȂɓɋǚɓً‫ژژژژژژژژژژژ‬
-ȵǠǵ‫ ژ‬eƌȄ‫ ژ‬uƌȵǠȄǠȽȽƷȄً‫ ژ‬hɓȄǠȏ‫ ژ‬ÅƩǚǠɲƌȂƌ
¾ƷȵȂ‫ ژ‬-ɱȲǠȵǠȄǒ‫ ژيאא׎א ژ‬wǠǹȽ‫
ژ‬ȽƩǚƷȄƨȵɓƩǵً‫ژ‬
-ȵȄƷȽɋȏ‫ ژ‬ɓƌưȵȏȽ‫ٯ‬ÝƌȵǒƌȽً‫ ژ‬%ƌɫǠư‫ ژ‬°ِ‫ ژ‬-ƨƷȵɋً‫ ژ‬ÞǠǹǹǠƌȂ‫ ژ‬GȵȏȲȲً‫ژ‬
GȵƌƩƷ‫ ژ‬kƷɬǠȽً‫ ژ‬°ɋƷǑƌȄȏ‫ژ‬îƌȄƷȵȏ
EXECUTIVE STAFF
Executive Director: Melissa
ِ‫ژ‬Russell
Director, Governance & Associate Executive Director:
Anne Marie Kelly
Director, Finance & Accounting: Sunny Hwang
Director, Information Technology & Services: Sumit Kacker
Director, Marketing & Sales: Michelle Tubb
Director, Membership Development: Eric Berkowitz
COMPUTER SOCIETY OFFICES
Washington, D.C.: 2001 L St., Ste. 700, Washington, D.C.
20036-4928ٕ Phone: +1 202 371 0101ٕ Fax: +1 202 728 9614ٕ‫ژ‬
Email: ǚƷǹȲ‫ۮ‬ƩȏȂȲɓɋƷȵِȏȵǒ
Los Alamitos: 10662 Los Vaqueros Cir., Los Alamitos, CA 90720ٕ‫ژ‬
Phone: +1 714 821 8380ٕ Email: help@computer.org
u-u-¨°OU¥‫ژۯژ‬¥ÅkU
¾U‚w‫¨‚ژ‬%-¨°‫ژ‬
¥ǚȏȄƷ‫ژٕבבבגژזוהژ׎׎זژ׏ڹژي‬Fƌɱ‫ژٕ׏גהגژ׏אזژג׏וژ׏ڹژي‬
-ȂƌǠǹ‫ژي‬ǚƷǹȲ‫ۮ‬ƩȏȂȲɓɋƷȵِȏȵǒ
IEEE BOARD OF DIRECTORS
President: ¾ȏȽǚǠȏ‫ژ‬Fɓǵɓưƌ
President-Elect: °ɓȽƌȄ‫ژ‬hِ‫ٹژ‬hƌɋǚɲ‫ژٺ‬kƌȄư
Past President: eȏȽƸ‫ژ‬uِFِ‫ژ‬uȏɓȵƌ
Secretary: Kathleen
ِ‫ژ‬Kramer
Treasurer: Joseph V. Lillie
Director & President, IEEE-USA: eǠȂ‫ژ‬ȏȄȵƌư‫ژ‬
Director & President, Standards Association: Robert S. Fish‫ژ‬
Director & VP, Educational Activities: °ɋƷȲǚƷȄ‫ژ‬¥ǚǠǹǹǠȲȽ‫ژ‬
Director & VP, Membership ‫ ۯ‬Geographic Activities:‫ژژژژژژژ‬
hɓǵDZǠȄ‫ژ‬ǚɓȄ
Director & VP, Publication Services & Products: ¾ƌȲƌȄ‫ژ‬°ƌȵǵƌȵ‫ژ‬
Director & VP, Technical Activities: hƌɼɓǚǠȵȏ‫ژ‬hȏȽɓǒƷ
 T
EN
EV
EW

IEEE
N

QUANTUM
WEEK

20
20
ER
B
TO
C
O
6
–1
12

REGISTRATION IS OPEN!
qce.quantum.ieee.org
The Future Directions Quantum Initiative invites you to IEEE Quantum Week
2020—the inaugural IEEE International Conference on Quantum Computing
and Engineering (QCE).

Council on Superconductivity