Artificial Intelligence and Machine Learning for Quantum Technologies

Mario Krenn,1, ∗ Jonas Landgraf,1, 2 Thomas Foesel,1, 2 and Florian Marquardt1, 2

1
Max Planck Institute for the Science of Light, Erlangen, Germany.
2
Department of Physics, Friedrich-Alexander Universität Erlangen-Nürnberg, Germany.
(Dated: August 9, 2022)
In recent years, the dramatic progress in machine learning has begun to impact many areas of
science and technology significantly. In the present perspective article, we explore how quantum
technologies are benefiting from this revolution. We showcase in illustrative examples how scientists
in the past few years have started to use machine learning and more broadly methods of artificial
intelligence to analyze quantum measurements, estimate the parameters of quantum devices, dis-
cover new quantum experimental setups, protocols, and feedback strategies, and generally improve
aspects of quantum computing, quantum communication, and quantum simulation. We highlight
open challenges and future possibilities and conclude with some speculative visions for the next
arXiv:2208.03836v1 [quant-ph] 7 Aug 2022

decade.

CONTENTS with an amazing vision of applications (in the 1950s

and 1980s, respectively), went through a series of chal-
I. Introduction 1 lenges, and are currently extremely hot research topics.
Of these two, machine learning has firmly taken hold
II. Basic Techniques of Machine Learning and beyond academia and beyond prototypes, triggering a
Artificial Intelligence 2 revolution in technological applications during the past
A. Evolutionary Algorithms 2 decade. This perspective article will be concerned with
B. Neural Networks: Structure 2 shining a spotlight on how techniques of classical ma-
C. Neural Networks: Training 3 chine learning (ML) and artificial intelligence (AI) hold
D. Unsupervised Learning 4 great promise for improving quantum technologies in the
E. Reinforcement Learning 4 future. A wide range of ideas have been developed at this
F. Automatic differentiation and gradient-based interface between the two fields during the past five years,
optimization 5 see Fig. 1. Whether one tries to understand a quantum
G. How to get started 5 state through measurements, discover optimal feedback
strategies or quantum error correction protocols, or de-
III. Applications of Machine Learning for Quantum sign new quantum experiments, machine learning can
Technologies 5 yield efficient solutions, optimized performance and, in
A. Measurement data analysis and quantum the best cases, even new insights.
state representation 5 With the present review, we aim to take physicists with
B. Parameter estimation: learning the properties a background in quantum technologies on a tour of this
of quantum systems 8 rapidly growing area at the interface to classical machine
C. Discovering strategies for hardware-level learning. The readers are not expected to have a back-
quantum control 10 ground in machine learning and will get a state-of-the-art
D. Discovering quantum experiments, protocols, view into how machine learning techniques are applied to
and circuits 12 quantum physics. We should state right away that in this
E. Quantum Error Correction 16 perspective article we tried to achieve our goal by focus-
ing in each application domain on some selected illus-
IV. Outlook 17 trative examples. Our selection is necessarily subjective.
We thus make no claim as to providing a comprehensive
V. Acknowledgments 18 list of the literature and apologize to anyone who misses
some favorite work.
References 18 We hope that after seeing the examples discussed in
our review, the reader will appreciate how useful ma-
chine learning techniques could be for quantum technolo-
I. INTRODUCTION
gies. At the same time, we also want to make the reader
aware of how crucial it is to choose the right AI approach
The fields of machine learning [1–3] and quantum tech- for a problem. It is by no means necessary that the most
nologies [4–7] have a lot in common: Both started out advanced AI method is the most suited tool for a given
task. Often, modern deep-learning methods can be sig-
nificantly outperformed by rather simple methods if ap-
plied to the wrong task, as seen in other areas [8]. Here it
∗ ML4qtech@mpl.mpg.de is crucial to analyze the scope of the problem and decide

FIG. 1. Overview of tasks in the area of quantum technologies that machine learning and artificial intelligence can help solve
better, as explained in this perspective article.
on the best-suited algorithms.
Once the reader wants to understand how to apply
these tools in practice, we recommend the very educa-
tional lecture notes introducing machine learning tech-
niques with a view to their application to quantum de-
vices, both brief [9], and very extended [10]. In addition,
there are several reviews from recent years with a some-
what different focus than ours, e.g. about machine learn-
ing applied to physics in general [11] or machine learning
for quantum many-body physics [12].
We also remark that there is the whole field of quan-
tum machine learning, which tries to discover potential
quantum advantages when implementing new learning
algorithms on quantum platforms. This also includes
variational quantum circuits, which are very promising
in the context of noisy intermediate-scale quantum com-
puters (NISQ). We do not cover these developments here,
and we refer the interested reader to reviews for quan-
tum machine learning [13–16] and variational quantum
algorithms and NISQ devices [17, 18]. While we focus
on quantum technology here, we want to mention that
there is also considerable work on ML for foundational
quantum science [19].
In the following section, we will briefly introduce the
basics of neural networks and other machine learning
techniques, aiming to set the stage for subsequent dis-
cussions. The bulk of our review is contained in section
III, where we discuss the various applications of machine
learning to quantum technologies. In each case, we aim
to remark on some of the challenges and potential future
research directions. Finally, in the outlook, we specu-
late about how machine learning might have transformed
quantum technologies in a dozen years from now.
II. BASIC TECHNIQUES OF MACHINE
LEARNING AND ARTIFICIAL INTELLIGENCE
The purpose of this section is merely to provide a
glimpse of the essential basics so that the subsequent dis-
cussion of applications becomes intelligible to the reader
without prior exposure. Machine learning techniques
have been around for several decades and involve many
efficient approaches that predate the recent deep learning
revolution, like ”support vector machines” or ”decision
2
trees” [20]. However, the flexibility of neural networks
has made them a popular general-purpose choice, so we
focus on those in our brief introduction.
A. Evolutionary Algorithms
One set of algorithms that often been used in opti-
mization tasks in the domain of artificial intelligence are
evolutionary (genetic) algorithms [21, 22]. There, the
idea is to deal with a set of candidate solutions, each
of them described via a suitable vector. These solutions
can be randomly changed (”mutated”), two solutions can
be combined to form a new candidate (”crossover”), and
finally only the best solutions can be kept for the next
round of evolution (”selection”). Such methods can be
surprisingly effective, and their applications to new prob-
lems is often straight forward. As we will describe in fur-
ther sections, genetic algorithms have successfully been
applied to quantum-technology-related tasks.
B. Neural Networks: Structure
The structure of artificial neural networks, which can
be trained to approximate arbitrary functions, is loosely
motivated by neurons in the human brain. Each neuron
receives multiple inputs and generates an output signal
which serves as input for other neurons. During learning,
the strength of connections between neurons changes.
More precisely, each artificial neuron receives N inputs
xj which are summed according to some weights wj , rep-
resenting connection strengths, see Fig. 2a. After adding
a bias (shift) b, a simple nonlinear ”activation function”
f is applied to yield the output y:
y = f (z) (1a)
N
X
z= w j xj + b (1b)
j=1
Without the nonlinear activation function between each
layer, the whole neural network could be compressed into
one single linear transformation, with very limited com-
putational power. Popular activation functions are the

FIG. 2. Basics of Neural Networks and Machine Learning
Techniques. (a) Operation of a single artificial neuron. (b)
Structure of a neural network with dense layers. (c) Evolu-
tion of the network’s parameters θ in the cost function land-
scape using stochastic gradient descent. For every step (or-
ange arrows) the gradients of the averaged cost function with
respect to θ (gray dashed line) are approximated by by the
gradient averaged over a random batch. The parameters θ̂
minimize the averaged cost function. (d) Classification of un-
labeled data. (e) Reinforcement learning problem modelled
as Markov decision process.
“ReLU” (f (z) = z for z ≥ 0 and f (z) = 0 for z ≤ 0)
and the “sigmoid” function (f (z) = 1/(e−z + 1)), which
represents a smoothed step-function.
Multiple connected neurons form a neural network as
shown in Fig. 2b. These networks are typically struc-
tured into several layers, where the output from each
layer serves as input for the next layer. Overall, there
is one input layer, one or more intermediate ”hidden”
layers, and an output layer. The input being fed into
such a network could be, e. g. pixel data from an image,
and after successful training, the output neurons encode
e. g. the category of the image (cat vs. dog). A network
with only one hidden layer and suitably many neurons
is already able to approximate arbitrary functions with
arbitrary precision [23]. However, networks with more
hidden layers (called “deep” neural networks) may be
able to fulfill the same task with fewer neurons.
In the network in Fig. 2b, all outputs of one layer serve
as input for all neurons of the next layer. Such a struc-
ture is called a dense layer, or ”fully connected network.”
However, this is often not the best choice, especially when
the input has certain symmetries. For example, an im-
age classification task can have a translational symme-
try since the category of an object shown in a picture
is independent of the object’s precise location. Convo-
lutional neural networks (CNNs) use this translational
invariance by convolving the input with a ”kernel” that
can be learned.
Time series have a temporal structure that can
be utilized using so-called ”recurrent” neural networks
(RNNs). The key idea is that the RNN receives the in-
formation for each subsequent time step sequentially as
input but also keeps some memory about previous inputs.
One of the most advanced and commonly-used RNNs are
3
so-called long short-term memory (LSTM) [24] networks.
C. Neural Networks: Training
A network can be made to approximate any function
by suitably choosing its weights and biases, often sum-
marily denoted as a parameter vector θ. To produce
correct predictions with a neural network, its parameters
have to be trained for the given problem. Supervised
learning (SL) is a training method to learn a desired tar-
get function f (x) from a dataset containing pairs (”sam-
ples”) of inputs and associated outputs, (x, f (x)). A
neural network represents an approximate input-output
relation fθ (x). The deviation between the network’s pre-
diction and the correct output in the dataset is quantified
by a ”cost” (or ”loss”) function. For any given input x,
this cost function can be calculated, C(fθ (x), f (x)), and
eventually, it will be averaged over all samples in the data
set. The trainable parameters θ of the NN fθ (x) have to
be chosen so as to minimize this (sample-averaged) cost
function.
The choice of cost function depends on the problem.
Typical fields of application for neural networks are re-
gression and classification tasks. For regression, the tar-
get function f (x) is a continuous function of the input x.
Then, the so-called ”mean-square error” is the canonical
choice, which for n output neurons reads:
n
X
2
CMSE (x) = (fθ,i (x) − fi (x)) (2)
i=1
On the other hand, in classification tasks, the input
should be assigned to certain predefined classes (e.g. cat-
egories of images). This is solved by having each output
neuron correspond to one of the n classes. Each neuron
value (or ”activation”) can then be interpreted as the
probability that the input is assigned to the correspond-
ing class. In that situation, the typical cost function is
the so-called ”cross-entropy,” a means to compare prob-
ability distributions.
In any case, neural-network training relies on the min-
imization of the cost function using gradient descent.
However, evaluating the cost function averaged over all
samples of the data set is infeasible. Rather, in each
update step, the cost function is averaged over a batch
of randomly selected inputs. In the simplest version of
the resulting ”stochastic gradient descent” scheme (see
Fig. 2c), the parameters θ are updated in each step ac-
cording to θ → θ − ηg with the gradient g:
∂hCibatch
g= (3)
∂θ
h·ibatch denotes the average over the random batch, and
η is the ”learning rate,” controlling the update’s size.
A neural network has many parameters (even small
networks usually have thousands of parameters, the
largest published networks have hundreds of billions of

parameters). Therefore, it is crucial that there exists
a highly efficient approach to calculating the gradient:
the so-called ”backpropagation” scheme. As its name
implies, this algorithm calculates the gradients layer by
layer, starting from the output layer. Remarkably, it is
computationally not more demanding than the original
evaluation of the neural network (also called the forward
propagation). A significant hardware advance for train-
ing neural networks are GPUs (graphical processor units)
that are optimized to perform highly-efficient manipula-
tions of large matrices. This efficiency is essential for the
success of ML in many applications.
For more details on neural network architectures, back-
propagation, and gradient descent, the interested reader
is referred to the many existing excellent introductions
[3, 9, 10, 25].
D. Unsupervised Learning
Unsupervised learning approaches can learn the struc-
ture of unlabelled data sets on their own, for example, as
shown in Fig. 2d. Typical fields of application are fea-
ture learning, where the machine is asked to find a com-
pact representation of the data, and clustering, where
the computer has to sort on its own samples into classes
with similar properties. Generative models are also often
attributed to the field of unsupervised learning. Their
purpose is to stochastically create new samples that fol-
low the same distribution as the previously observed data
set (e.g. images of the same type, though never seen be-
fore). The simplest approaches to feature learning still
rely on the techniques of supervised learning (namely,
”self-supervised” learning in so-called ”autoencoders”).
However, clustering and generative models are typically
implemented using distinct techniques (see, e.g. [3]).
E. Reinforcement Learning
Some of the most important problems in artificial intel-
ligence and machine learning can be seen as the attempt
to find an optimal strategy to cope with a certain task,
where the optimal strategy is unknown (thus, supervised
learning cannot be applied). This is the domain of Re-
inforcement Learning (RL), which aims to discover the
optimal action sequences in decision-making problems.
Its power has been famously illustrated in board games
like chess or Go [26] or video games[27], in all of which
RL is able to reach superhuman performance. In many
applications, RL can find the optimal strategy without
prior knowledge about the actual dynamics of the sys-
tem. When that is the case, we speak of ”model-free”
RL. The goal in any RL task is encoded by choosing a
suitable ”reward,” a quantity that measures how well the
task has been solved.
The typical RL problem can be understood as a so-
called ”Markov Decision Process” (MDP), see Fig. 2e.
4
An MDP consists of an ”agent” (the controller) and
an ”environment” (the world, or the system to be con-
trolled), and both interact in multiple time steps. In
each time step t, the environment’s state st is observed.
Solely based on this observation, the agent decides on
its next action at , which will change the environment’s
state. The agent’s behavior is defined by the ”policy”
π(a|s), which denotes the probability of choosing the ac-
tion a given the observation s. For each action at , the
agent receives a reward rt . For example, in a game, this
reward could be +1/ − 1 at the last time step, when the
agent has won/lost the game, and otherwise 0 in all pre-
vious steps. RL aims to maximize the cumulative reward
R (also called ”return”)
T
X
R= rt , (4)
t=1
where T is the total number of time steps.
Three major branches of RL algorithms exist: pol-
icy gradient, Q-learning, and actor-critic methods. For
policy-gradient methods, the agent directly sets the pol-
icy πθ (a|s). In deep RL, this agent is realized by a deep
neural network with trainable parameters θ. To find the
optimal strategy, policy-gradient estimates the gradient
of the average return hRi with respect to θ. Here, h·i
denotes the average over all trajectories for the current
policy. At first sight, it is unclear how to take the gra-
dients through the reward without knowing the model.
However, one can compute the gradients of the frequency
of a certain reward via the policy function πθ . Thus, the
gradient turns out to be:
* T +
∂hRi X ∂
g= = R ln πθ (at |st ) (5)
∂θ t=1
∂θ
It is important to note that R depends on the full trajec-
tory and the parameters θ (we suppressed these depen-
dencies for brevity). Updates based on this equation will
increase (”reinforce”) the probabilities of actions that oc-
cur predominantly in high-reward trajectories. Another
approach to finding the optimal policy is so-called ”Q-
learning.” It employs the ”Q function” Qπ (s, a) that tries
to estimate the quality of an action a: it is defined as
the average expected future return starting from a state
s and action a for a policy π. The optimal policy is
then, by definition, to choose the action that maximizes
Q in a given state s. In practice, the Q function is ini-
tially unknown. During training, an approximation to
this function is learned, often using a deep neural net-
work to represent Q.
Finally, the third group of RL algorithms are the so-
called ”actor-critic” methods. These try to combine
the benefits of both policy-gradient and Q-learning ap-
proaches. The basic idea behind actor-critic methods is
to estimate the expected reward given the current state,
the so-called value function V . The success of any ac-
tion is then measured by comparing the resulting reward

against V . As might be expected, the value function is
represented by a neural network (the ”critic” network)
which is trained by SL to approximate the true value
function for the current policy.
For an introduction to the different RL algorithms, the
interested reader is referred to [28].
F. Automatic differentiation and gradient-based
optimization
Deep learning is efficient because of the backpropa-
gation technique that can efficiently calculate gradients
with respect to all the hundreds or millions of parame-
ters in a neural network. This technique more generally
leads to the concept of automatic differentiation. There,
the idea is to obtain the exact gradient with respect to
any variable appearing in any kind of numerical calcu-
lation. As a numerical approach, this is distinct from
symbolic differentiation applied in computer algebra pro-
grams. Modern frameworks used for neural networks of-
fer various modes of automatic differentiation. This of-
fers the chance to employ them for arbitrary gradient-
based continuous optimization tasks, especially those in-
volving many parameters, where efficiency is of concern.
We will later show examples of how this can be used
to discover new quantum experiments, quantum circuits,
and in other contexts.
G. How to get started
Only a few years ago, implementing neural networks
still required quite some effort. Nowadays, libraries like
Tensorflow, PyTorch, JAX, and many more offer pow-
erful tools to set up and train neural networks. We
have created an online collection of resources to start
with ML, that contains information on different popu-
lar frameworks, as well as helpful tutorials, lecture notes,
and reviews for machine learning in general1 : https:
//github.com/ML4QTech/Collection.
III. APPLICATIONS OF MACHINE LEARNING
FOR QUANTUM TECHNOLOGIES
A. Measurement data analysis and quantum state
representation
An important direct application of machine learning
to quantum devices is the interpretation of measurement
data. The application ranges from an improved under-
standing of the measurement apparatus itself to extract-
ing some high-level properties of the quantum system to
1 We plan to update this collection continuously and are happy
about contributions from the community.
5
FIG. 3. State Estimation via Neural Networks. (a) Measure-
ments on many identical copies of a quantum state can be
processed to produce an estimate of the quantum state. (b)
A continuous weak measurement on a single quantum system
can be used to update the estimated state. Both in (a) and
(b), a single network is trained to estimate arbitrary states
correctly. (c) One can also train a network-based generative
model to reproduce the statistics of a quantum state, i.e. to
sample from the probability distribution. Training requires
many identical copies that can be measured, so the statistics
can be learned. Here, one network represents only a single
quantum state. It can be extended to handle measurements
in arbitrary bases.
the full reconstruction of the measured quantum state.
In many cases, this can be phrased as a supervised learn-
ing task. One example might be to extract an approxi-
mate description of a quantum state from a sequence of
measurement results on identically prepared copies, see
Fig. 3a. Provided the actual quantum state is known for
each training example, the machine learning algorithm
will learn to provide the best possible approximation to
the quantum state. The same is true if the goal is to
reconstruct certain properties of the state or the device
instead of reconstructing the full quantum state. The
choice of the cost function is essential, a simple exam-
ple being the infidelity between the predicted and the
actual state. Other choices will lead to slightly different
optimal approximations predicted by the ML algorithm.
Crucially, the algorithm can easily deal with distortions
of the measurement data (such as extra technical noise),
as it will learn the properties of these distortions and how
to undo them.
Interpreting Measurements – An interesting early
example used machine learning to improve the readout
fidelity of a qubit in a superconducting quantum device.
There, the noisy measurement trace, obtained from a
microwave signal passing through a readout resonator
interacting with the qubit, can be used to deduce the
qubit’s logical state. However, classifying the qubit state
is challenging. The authors of [29] use a basic machine

learning technique called support-vector machine (SVM)
to perform clustering of measurement traces in an un-
supervised fashion, outperforming classical clustering al-
gorithms. The idea of a nonlinear SVM is to map data
points to a higher-dimensional space and to find the best
hyperplane for separating two classes of data points in
that space. In this specific work, each measurement tra-
jectory, which consists of hundreds of individual data
points, is interpreted as a point in a high-dimensional
space. The SVM’s goal was to separate curves that orig-
inate from a zero-state and one from a one-state. The
readout fidelity was improved compared to non-ML clus-
tering techniques. Furthermore, this analysis has shown
that the main noise contribution comes from physical bit-
flips (either heating or relaxation of the qubit). Without
such events, the classification of the SVM becomes near-
perfect. A similar approach has been demonstrated – us-
ing neural networks – to enhance the readout capability
of trapped-ion qubits [30]. Here, the authors show that
the readout fidelity improves significantly compared to
a non-ML clustering method, especially when the effec-
tive amount of data per measurement increases. Similar
techniques have also been applied to NV center quantum
devices [31].
In a pioneering experimental work, a neural network
reconstructed the quantum dynamics of a quantum sys-
tem directly from measurement data [32]. There, the au-
thors considered again a superconducting qubit coupled
to a readout resonator, whose noisy measurement trace
is fed as input into the network (together with the initial
preparation state and the final measurement basis). The
network’s task was to predict the statistics of arbitrary
measurements at some given time during the evolution,
i.e. effectively predict the evolution of the density matrix
given the measurement record, see Fig. 3b. The kind of
network most suited to this task is a so-called recurrent
network, i.e. a network able to process a time series (orig-
inally used for text or speech processing). The resulting
fully trained network can map any measurement data to
a quantum state.
Furthermore, machine learning techniques can be ex-
ploited to analyze the statistics of measurement outcomes
in quantum experiments where the aim is to demonstrate
the classical complexity of sampling from the quantum
distribution. Boson sampling is the most well-known
such scenario. In [33], unsupervised machine learn-
ing techniques (various clustering approaches) were em-
ployed to identify and rule out possible malfunction sce-
narios that would lead to a noticeably different distribu-
tion.
One exciting work from theoretical computer science
connects the question of quantum state tomography with
the theory of computational learning in the supervised
setting [34]. Here, a learner uses the training dataset to
produce a hypothesis about future measurements. Quan-
tum state tomography, which requires a number of mea-
surements that is exponential in the number of particles,
can be seen as a learner that produces a hypothesis for ev-
6
ery possible measurement on the quantum system. This
might, however, not be necessary in most cases. The
theory of probably approximately correct models (PAC)
provides a hypothesis for every measurement close to the
dataset. Surprisingly, it was found that this question can
be solved with a linear scaling of measurements. This
computational learning strategy has been first demon-
strated in quantum optics experiments with up to six
photons. The experiment has confirmed the scaling be-
havior of PAC, even in the presence of realistic experi-
mental noise [35].
Another very interesting recent development shows
that joint quantum measurements of several individual
copies of a many-particle quantum state can lead to
an exponential improvement over classical learning al-
gorithms [36]. The authors show experimentally on a
platform of 40 superconducting qubits that tasks such as
predicting properties of the physical system can be signif-
icantly improved when the results of such joint measure-
ments are fed as an input to a classical RNN. The result
is particularly remarkable as it shows a clear advantage
already for current, noisy, and not error-corrected quan-
tum computers.
Many other interesting examples exist that use neu-
ral networks to analyze simulated or measured data of
quantum systems. For example, it has been demon-
strated that the Wigner negativity of a multimode quan-
tum state can be approximated well even in the low-data
regime [37], with important consequences for quantum
technologies. Another vivid field is the neural-network-
based detection of quantum phase transitions and classi-
fication of quantum phases in condensed matter physics.
We will not go into detail here. Instead, we point to
some exciting early works in this field [38–40], as well
as some very modern applications that, for instance, use
anomaly detection to find phase transitions in an unsu-
pervised way [41]. Anomaly detection has already been
applied to detect phases directly from experimental mea-
surements [42]. See a recent review on this topic in [12].
Approximation of Quantum States – The direct
application of numerical techniques to quantum devices
requires, in many situations, the storage, and processing
of the system’s full quantum state. As the quantum state
grows exponentially with the number of particles, the
memory requirements quickly become enormous, even for
moderately large quantum systems. For example, stor-
ing the full quantum state of a 42-qubit system requires
35 TByte of memory. As demonstrated in the earliest
quantum advantage experiments [43], this is directly re-
lated to the power of quantum computers and quantum
simulators. However, it poses a significant problem for
classical computational approaches that deal with large
quantum systems and therefore for developing new large-
scale quantum technologies.
To overcome this challenge, memory-efficient approxi-
mations of the quantum wave function are indispensable.
Neural networks are one key candidate to approximate
the quantum wave function. This approach has some-

times been called Neural Quantum State (NQS).
A prominent approach tries to represent the quantum
state in terms of a neural network [44]. This implies
that for each new quantum state, another network will
be trained, based on the associated measurement data
for that state. In principle, that is considerably easier
than asking a single network to be responsible for arbi-
trary states, i.e. the task considered above. As a con-
sequence, much more complicated many-body states can
be accessed. The whole approach can be seen as a neural-
network-based version of quantum state tomography.
Several different ways exist to use a single neural net-
work to represent a single quantum state. One straight-
forward approach, first introduced in [44] and then ex-
tended in subsequent works, employs a network that
directly represents the wave function. Given a multi-
particle configuration x as input, the network has to pro-
duce the wave function amplitude for that configuration
as output: Ψθ (x).
Different structures can be used for the network, with
a restricted Boltzmann machine (RBM) being a popu-
lar choice since it also allows direct sampling from the
probability distribution of observations [45]. In a tradi-
tional RBM, the aim is to learn to sample from some
observed probability distribution, see Fig. 3(c). It con-
sists of binary visible units and hidden units connected
to each other, and the statistics of these units are sam-
pled from a Boltzmann distribution with an energy E
that contains interaction terms bilinear
P in the P hidden (h)
and visible (v) unit values: −E = j aj vj + k bk hk +
P
j,k wjk vj hk . During training, the coupling constants
are updated to obtain the desired probability distribution
of v (observed in samples provided during training). A
simple physics example would be a 1D spin chain, whose
configurations are identified as sample vectors v. More
generally, other so-called generative deep learning meth-
ods (such as normalizing flows, variational auto-encoders,
and generative adversarial networks) can be used to learn
probability distributions, including those representing
the statistics of observables in quantum states in a given
basis.
Quantum state tomography using an RBM-style
ansatz for the wave functions was introduced in [46].
Since one wants to keep the wave function’s phase ϕ as
well as the
p probability p, the ansatz is now of the type
Ψ(x) = p(x)eiϕ(x) , where both p and ϕ are represented
as networks and x corresponds to the visible units. A cru-
cial idea in this approach is to match the probability dis-
tributions obtained from the experiment for observables
in more than one basis (e.g. σ z and σ x etc. for qubits).
The evaluation of different bases can be carried out via
unitary transformations acting on the wave function Ψ
that is expressed in a single reference basis.
In [47], this approach was applied to experimental data
from snapshots of many-particle configurations taken
in a Rydberg atom quantum simulator. The resulting
network-based wave-function ansatz could then be used
to reconstruct other expectation values and observables
7
that were not directly accessed in the experiment.
The idea has been extended to mixed states [48–52].
In that way, NQS can be used to efficiently approximate
open quantum systems which are notoriously difficult to
capture.
Approximating Quantum Dynamics – Once a
suitable quantum state representation is available, it can
be exploited to evolve the state in time. This enables
potentially efficient simulation of quantum many-body
time evolution, which is important for predicting and
benchmarking the dynamics of quantum simulators and
quantum computing platforms. For the general case of
dissipative quantum many-body dynamics, i.e. the time
evolution of mixed states, this has been explored in [50].
Rather than explicitly storing the entire quantum
state, another technique shows how one can directly com-
pute a quantum state’s complex properties just from the
state’s construction rules, i.e. the quantum experimen-
tal circuit. In [53], the authors show how a recurrent
neural network [24] can approximate the properties that
emerge from quantum experiments without ever storing
the intermediate quantum state directly. These systems
could then directly be applied for complex quantum de-
sign tasks, a topic we cover in chapter III D. Another
approach [54] also foregoes the representation of quan-
tum states and instead trains a recurrent network to
predict the evolution of observables under random ex-
ternal driving of a quantum many-body system (either
based on simulated or possibly even experimental data).
The trained network can then predict the evolution un-
der arbitrary driving patterns (e.g. quenches). In a sim-
ilar spirit, neural ordinary differential equations (neural
ODEs) can be used to approximate the dynamics of quan-
tum systems directly, again without storing the explicit
information about the quantum wave function [55]. In-
terestingly, the approximation is of high enough quality
that it is possible to rediscover some fundamental prop-
erties of quantum physics, such as the Heisenberg uncer-
tainty relation.
Future Challenges and Opportunities –
Improved data efficiency (both for the training but also
when applying ML to interpret the data) will be an im-
portant challenge for the future, especially when the de-
vices scale to more complex quantum systems.
It will be interesting to co-discover measurement strat-
egy together with the data interpretation strategy. This
might be particularly interesting if the AI algorithm is
allowed to employ quantum measurements on numerous
copies of the same state, as pioneered in [36].
When a neural network can find a suitable approxima-
tion for the computation of complex quantum systems,
such as an NQS, it has learned a theoretical technique
that might be useful for humans too. It will be interest-
ing to learn how to extract the per-se inaccessible knowl-
edge from the weights and biases of the neural network.
One method is so-called symbolic regression.
The extensions of NQS to complex quantum systems,
such as higher dimensions and spins beyond qubits, will

FIG. 4. Machine Learning for Parameter Estimation in
Quantum Devices. (a) A typical scenario, with the measure-
ment result statistics depending both on some tuneable mea-
surement setting and the unknown parameter(s), here rep-
resented as phase shifts in a Mach-Zehnder setup. (b) An
adaptive measurement strategy can be illustrated as a tree,
with branches on each level corresponding to different mea-
surement outcomes. Depending on those outcomes, a certain
next measurement setting (indicated as ”αj ”) needs to be se-
lected. Finding the best strategy is a challenging task, as it
corresponds to searching the space of all such trees. (c) Neu-
ral generative models can be used to randomly sample possi-
ble future measurement outcomes (here: 2D current-voltage
maps as in [56]) that are compatible with previous measure-
ment outcomes. This is helpful for selecting the optimal next
measurement location. Different random locations in latent
space result in different samples. (d) Measurement outcome
vs. measurement setting for 5 possible underlying parameter
values (different curves; measurement uncertainty indicated
via thickness). We aim to maximize the information gain,
i.e. choose the setting which best pinpoints the parameter
(which is not equivalent to maximizing the uncertainty of the
outcome).
allow for more interesting applications.
B. Parameter estimation: learning the properties
of quantum systems
In this section, we will discuss machine-learning-
based approaches for estimating experimental parame-
ters. These could either be system parameters for the
calibration of quantum devices or the estimation of ex-
ternal parameters in the form of quantum metrology.
Quantum metrology – Machine learning can be
helpful for various challenges in quantum metrology [57].
Quantum metrology deals with the resource-efficient
measurement of external parameters acting on a quan-
tum system, like magnetic fields or optical properties
of a material sample. Broadly, the field can be sepa-
rated into two different branches. First, non-adaptive
approaches exploit how complex quantum entanglement
can be exploited to reduce the required resources (for in-
8
stance, the number of photons that interact with a sam-
ple) without changing the experimental setup or input
states throughout the measurement sequence. The sec-
ond class of approaches exploits feedback, meaning they
employ adaptive strategies that change either the input
or the measurement setting, depending on the previous
measurement outcome. Naturally, adaptive strategies re-
quire more advanced experimental implementations, in-
cluding fast switching or long-term stability of setups.
For that reason, non-adaptive quantum metrology is so
far much more explored in laboratories, while adaptive
approaches are still at the stage of proof-of-principle ex-
periments. AI has contributed to both approaches.
An example of the application of neural networks to the
estimation of unknown parameters in a photonic experi-
ment is provided in [58]. There the goal is to calibrate a
device via neural-network training that can later be used
to estimate an unknown phase shift. The authors first ac-
cumulate a large set of calibration measurements using
a controlled phase plate. In addition, the data is aug-
mented to account for the statistical noise contribution.
The neural network receives the measurement data (the
number of detected photons) and has to estimate the cor-
responding phase. As soon as the neural network is able
to model the connection between measurement data and
calibration phases, it can be used to estimate unknown
phases as well. This task shows that device calibration
and estimation of external parameters for quantum sen-
sors are closely related.
In addition, also still for non-adaptive approaches, var-
ious projects have explored the discovery of new exper-
imental setups for the resource-efficient measurement of
parameters of an external system [59, 60]. We will talk
about this approach in section III D.
We now turn to focus on feedback-based quantum
metrology schemes and how they could be improved with
ML algorithms. A pioneering non-ML approach for such
a task is called BWB (Berry-Wiseman-Breslin) strategy,
after the authors of [61, 62]. In the original setting of
that strategy, the authors consider a Mach-Zehnder in-
terferometer with two detectors at the two outputs, see
Fig. 4a. One of the arms contains an unknown phase
shift, and the other arm contains a controllable phase
that is modified depending on the previous measurement
results. One can think of this decision-making problem
as a decision tree (see Fig. 4b), where the actions must
be chosen based on the measurement outcomes to max-
imize the information gain. In general, the expected in-
formation gain is defined as the reduction of entropy of
the parameter distribution, averaged over possible mea-
surement outcomes. The authors derived the feedback
algorithm by applying Bayes theorem to the distribution
of the unknown phase, which is then used to choose sub-
sequent measurement settings with a large information
gain. This strategy has been found not only to be highly
resource-efficient, but also practically implementable in
the laboratory [63].
Interestingly, greedy adaptive strategies (i.e. to choose

in every step the reference phase that yields the largest
immediate information gain) do not necessarily lead to
the largest information gain in the long run. To over-
come this effect, one of the seminal early contributions
to ML-based quantum metrology used particle swarm op-
timization of the feedback strategy[64]. In particle swarm
optimization, a collection of different feedback strategies
(each called “particle”) iteratively moves in the space
of all possible strategies. In each iteration, the particle
moves towards a combination of the best local optimum
and the currently best known global optimum known by
the whole swarm. The experimental setting of [64, 65]
is the same as for the BWB strategy – a Mach Zehnder
interferometer with an unknown phase in one arm and an
adaptive phase in the other arm. Indeed, the swarm opti-
mization algorithm finds (slighly) better strategies than
the greedy Bayesian BWB approach. Interestingly, nei-
ther BWB nor swarm optimization can find the optimal
strategy, which was identified for small photon numbers
via an extensive computation of all possible strategies.
Other early ML algorithms in this domain have applied
evolutionary approaches to approximate the ideal feed-
back strategy[66, 67].
Discovering a strategy is a problem that can directly
be formulated as a reinforcement learning task. An early
application of RL to quantum parameter estimation was
provided in [68], with frequency estimation of a qubit as
a test case. In that work, the idea was to optimize the
quantum Fisher information for the parameter of inter-
est. This can be done by finding a sequence of suitable
control pulses applied during the noisy evolution of the
quantum probe. No feedback is involved in this simple
setting since the measurement itself is not part of the
evolution controlled by RL.
However, the quantum Fisher information is only use-
ful in cases where one is already fairly certain of the true
parameter value. RL can be employed to study more
complex situations, where updates are performed using
the Bayes rule, starting from an arbitrary prior param-
eter distribution, and where the strategy is not greedy
(i.e. more than a single step of the sequence is optimized).
In [69], the authors provided information about the cur-
rent Bayes distribution of the unknown parameter (as
extracted from previous measurement results) and the
previous measurement choices as input to an RL agent
implemented by a neural network. It then has to sug-
gest the next measurement. After the whole sequence
of measurements, the agent is rewarded according to the
total reduction in parameter variance. It was shown that
this approach performs very competitively for an impor-
tant test case, namely parameter estimation for a qubit
of unknown frequency in the presence of dephasing.
Device calibration – Future large-scale quantum de-
vices will consist of a large number of components with
adjustable parameters that need to be characterized and
tuned automatically. A complete characterization of the
device via quantum process tomography quickly becomes
impractical. To find the actual parameters or the ideal
9
operating point of the quantum device, it is therefore
necessary to extract the relevant data with a very lim-
ited amount of information. This task can be formulated
as a machine learning task, specifically applying “active
learning” or “Bayesian optimal experimental design” [70]
where the algorithm chooses the most informative mea-
surement autonomously (see also [71] for a review of such
methods in the context of quantum devices). Naturally,
this is closely related to the estimation of external pa-
rameters in quantum metrology as discussed above.
We illustrate these techniques via a pioneering exper-
imental application to quantum devices. This experi-
ment [56] considered the calibration and measurement
of a semiconductor quantum dot. Such a device can be
tuned via applied gate voltages, and its resulting proper-
ties can be measured via a transport current. Here, the
goal was to explore the properties of the quantum dot, as
defined by its current-voltage map I(V1 , V2 ), where the
voltages include a bias voltage driving the current and
a gate voltage deforming the dot’s potential. In a naive
approach, even if only two voltages were scanned with
100 discretization steps each, one would need to perform
10,000 measurements to get a suitable resolved device
characteristic.
To reduce the required number of current measure-
ments, the authors tried to estimate which measurement
(in the 2D voltage space) would yield the ”maximum
amount of additional information.” In practical terms,
this would be the measurement that is expected to place
the tightest constraints on the current-voltage maps that
are still compatible with all the observed values of the
current (observed in this and prior measurements). It
is obvious how this setting translates to other quantum
platforms, e.g. measurements of microwave transmission
through superconducting circuits controlled via gate volt-
ages and magnetic fields or the optical response of tune-
able atomic systems.
The first step towards this goal is to efficiently rep-
resent all current-voltage maps that might be observed,
given the general physics of such a device, the assumed
prior distribution of device parameters, and all previous
measurement results. In general, this is the domain of
”generative models,” which can sample from a probabil-
ity distribution that is learned. In the case of [56], the
authors used such a generative model, in their case a so-
called ”constrained variational autoencoder” (cVAE), to
randomly create realistic current-voltage map that follow
the probability distribution of the actual physical system,
see Fig. 4c. Additional input into the generative model
provides a constraint, in the form of a few initially exist-
ing measurement results, and guides the reconstruction
to sample only maps compatible with those constraints.
In each step, 100 different voltage maps are sampled.
Those maps are used to find the next measurement point
in voltage-space that would lead to the maximum infor-
mation gain, see Fig. 4d. With this technique, the total
number of necessary measurements for the characteriza-
tion of the device is reduced by a factor of 4. This clearly

shows that the overhead of the deep-learning algorithm
is more than compensated by its efficiency improvement
compared to the naive approach. A benefit of this tech-
nique is that generating new samples with the cVAE is
very efficient. Thus it can be scaled to much larger de-
vices, where even more significant efficiency gains are ex-
pected.
Another comparatively straightforward way to use ma-
chine learning in device characterization consists in train-
ing a network-based classifier to recognize ”interesting”
measurement results. This then allows to tuning param-
eters until those results are obtained. Such an approach
has been demonstrated in [72] for navigating charge-
stability diagrams of multi-quantum-dot devices. In that
setting, the algorithm’s goal was to automatically tune
the charge occupation of the double quantum dot. The
task is reformulated as a classification task, where the
algorithm recognizes individual charge transitions when
presented with a charge-stability diagram. Since such a
diagram constitutes an image, CNNs are a suitable choice
for the task.
Quantum Hamiltonian Learning – Imagine the fol-
lowing parameter-estimation problem: One wants to es-
timate the parameters x0 that affect the evolution of a
quantum state under a quantum many-body Hamiltonian
H(x0 ) [73]. Unfortunately, even the task of comput-
ing the dynamics scales exponentially with the system
size (number of qubits) when tackled using a classical
machine. The idea of Quantum Hamiltonian Learning
(QHL) is to enlist the help of a quantum simulator to
overcome this problem. The parameters x0 can then be
estimated with standard Bayesian methods. Thereby,
the quantum simulator is used like a subroutine inside
a classical ML approach. The first experimental imple-
mentation of this idea was demonstrated in 2017 [74]. In
that work, the authors wanted to estimate the parame-
ters of an electron spin in a nitrogen-vacancy center, and
they used a quantum simulator on an integrated pho-
tonics platform to perform the QHL. Interestingly, not
only did the approach lead to a high-quality estimation
of the dynamic system parameters, but it also indicated
when the initial Hamiltonian model had deficits. In these
cases, the learning method informed the user that there
are other dynamics in play that have not been consid-
ered, which inspires an improvement of the underlying
Hamiltonian model.
While the QHL method indicates that when the model
Hamiltonian is not ideal, it cannot adapt it. To overcome
this hurdle and to learn the entire Hamiltonian structure
(not only its parameters), the authors of [75] have in-
troduced the idea of a Quantum Model Learning Agent
(QMLA). This agent not only finds the parameters of a
predefined Hamiltonian, but discovers the whole Hamil-
tonian that describes the dynamics of a system. The
approach iteratively refines the initial Hamiltonian and
uses QHL as a subroutine for finding suitable parameter
settings. This approach has also been demonstrated in
a hybrid quantum system involving a nitrogen-vacancy
10
center. The underlying learning mechanism is very gen-
eral, and it thus could become a powerful tool for learning
the dynamics of unknown quantum systems.
Future Challenges and Opportunities – For adap-
tive approaches, one needs to consider the trade-off be-
tween speed and the sophistication of the approach. In
these tasks, the time between measurement and feedback
is often very short, so the decision must be taken quickly.
An interesting future approach for advanced quan-
tum metrology approaches is to simultaneously co-design
the experimental setup and the feedback strategy, rather
than solving these tasks individually.
C. Discovering strategies for hardware-level
quantum control
Challenges like quantum computing and quantum sim-
ulation are leading to rapidly increasing demands on the
efficient and high-fidelity control of quantum systems.
Tasks range from the preparation of complex quantum
states and the synthesis of unitary gates via suitable
control-field pulses all the way up to goals like feedback-
based quantum state stabilization and continuously per-
formed error correction. In trying to solve these tasks,
the specific capabilities and restrictions of any hardware
platform, from superconducting circuits to cavity quan-
tum electrodynamics, need to be considered.
In this section, we will highlight specifically how rein-
forcement learning has come to help with many of these
challenges. In the form of model-free RL, it promises
to discover optimal strategies directly on an experiment,
which can be treated as a black box, see Fig. 5a. All
its unknowns and non-idealities will then be revealed
only via its response to the externally imposed control
drives. But even when used in a model-based way, us-
ing simulations, RL can be more flexible than simpler
approaches. In particular, it offers ways to discover feed-
back strategies, i.e. strategies conditioned on measure-
ment outcomes. These were not previously accessible to
the usual numerical optimal control techniques.
The present section is firmly concerned with hardware-
level control that is continuous in the time domain, dis-
covering pulse shapes or feedback strategies based on
time-continuous noisy measurement traces as they would
emerge from weak measurements of quantum devices.
There are some connections to the next section, but there
we will be concerned with the discovery of protocols, con-
trol strategies, and whole experimental setups that are
described on a higher level, composed of discrete build-
ing blocks like gates or experimental elements.
Quantum control tasks without feedback (open-
loop control) –
Prior to the application of machine learning techniques
in this field, the focus was essentially on tasks without
feedback, which was solved by direct optimization tech-
niques, adapting the shape of control pulses applied to
the quantum system to maximize some quantity (like the

state fidelity). These direct optimization techniques in-
clude gradient-based approaches, with GRAPE [76] and
the Krotov method [77, 78] the most prominent exam-
ples, as well as approaches that do not rely on access
to gradients, such as CRAB [79]. At the time of writing,
these techniques still form the default toolbox for the case
of open-loop control, even while the first applications of
machine learning (described below) are taking hold. Evo-
lutionary algorithms define another class of (stochastic)
approaches that have been used successfully to find op-
timal control sequences [80].
State preparation is the most common quantum con-
trol problem, and yet it can already be challenging, es-
pecially for multi-qubit settings. In probably the ear-
liest application of RL to quantum physics, pure-state
preparation with discrete control pulses was shown us-
ing a version of Q-learning for a spin-1/2 system and a
three-level system [81]. A few years later, RL-based state
preparation was demonstrated for a many-qubit system
[82], also using Q-learning and discrete bang-bang type
actions, with particular emphasis on analyzing the com-
plexity of the control problem showing up in the form of a
glassy control landscape. Both of these works used some
version of table-based Q-learning, which works well for a
restricted number of states and actions. The first work
to employ deep (i.e. neural-network-based) RL methods
to open-loop control of quantum systems was [83], with
both discrete and continuous controls and a recurrent
network as an agent, as applied to dynamical decoupling
and again state preparation, followed shortly afterwards
by [84].
The RL approach can be used successfully to find suit-
able pulse sequences for unitary gates and optimize for
the gate fidelity, as shown first in [85], and analyzed later
also in [86].
Recently, deep RL has been applied for the first time
to learn control strategies for a real quantum comput-
ing experiment [87]. The authors trained on a cloud-
based quantum computing platform, collecting data for
the current control policy, extracting rewards, and up-
dating the policy. The goal was unitary gate synthesis,
and the lack of real-time access to the device was not a
concern since the task required only open-loop control.
This first demonstration of RL-based quantum control
on a real quantum experiment helped to illustrate the
possibilities and challenges in this new approach.
Even though open-loop control pulse design means
that the actual strategy in the experiment is not con-
ditioned on any measurement outcomes, RL training for
such tasks (when done on a computer simulation) may
still benefit from the agent receiving input information
like the current quantum state. Experience shows that
this makes it easier to find a good strategy. Otherwise,
only very sparse nominal information like the current
time step and possibly the most recent selected action
would be fed into the agent. In any case, however, once
RL has found a control sequence, it could in principle be
stored (e.g. as a waveform or pulse sequence) and sent to
11
an experiment whenever needed. In other words, there
is no need for the agent to be running during the actual
experiment, which strongly relaxes requirements for the
hardware: no real-time control is necessary.
Quantum feedback control (closed-loop control)
– The successful control of quantum systems subject to
noise, decay and decoherence requires either reservoir en-
gineering (autonomous feedback) or active feedback con-
trol. The space of active feedback strategies is expo-
nentially larger than that of open-loop control strategies
(i.e. without feedback), owing to the number of potential
measurement outcome sequences growing exponentially
with time (each such sequence may require a different
response). It is here that it is almost inevitable to use
the power of RL, particularly deep RL, with its ability
to process high-dimensional observations.
The first work to apply deep RL to feedback-based con-
trol of quantum systems was [88]. It employed discrete
gates for quantum error correction, and we will discuss
it in sections III D and III E. State preparation and sta-
bilization in the presence of noise or an uncertain ini-
tial state are other natural candidates for RL feedback
strategies. Examples include quantum state engineering
via feedback [89], as well as control of a quantum particle
in an unstable potential [90] and a double-well potential
[91]. In some quantum systems, control may be very lim-
ited (e.g. only linear manipulations), but measurements
can introduce nonlinearity and their exploitation through
RL-based feedback strategies can enable powerful con-
trol, as shown in [92]. One challenge for model-free RL
as applied to experiments is to make sure rewards can be
extracted directly and reliably from experimental mea-
surements, and to use a training procedure that really
treats the quantum device as a black box (not relying,
e.g. on simulations). These aspects were emphasized in
[93], where state preparation in a cavity coupled to a
qubit was analyzed.
Model-free vs model-based RL – Applying model-
free RL techniques, as described above, has a great ad-
vantage: the experimental quantum device can be treated
as a black box, and its inner parameters and distortions of
the control and measurement signals need not be known a
priori. However, this also means that part of the training
effort is spent on effectively learning an implicit model of
the quantum device since that is the basis for a good
control strategy.
In many situations relevant to modern quantum tech-
nologies, though, a good model is known since the Hamil-
tonian and Lindblad dissipation terms have been care-
fully calibrated. This allows to consider model-based
techniques explicitly, see Fig. 5b. In principle, these can
simply consist in applying model-free approaches to an
RL-environment that is represented by a simulation of
the model. However, this is only useful if running the ex-
periment often would be expensive or time-consuming. A
more direct approach takes gradients directly through the
model dynamics. In the absence of feedback, this is what
well-known approaches like GRAPE offer. In an inter-
 12

esting recent development, automatic differentiation (the

cornerstone of deep-learning frameworks) has been used
to easily get access to the gradients needed for model-
based control optimization. This was first presented in
[94] and then applied to various quantum control tasks,
especially for qubit systems [95, 96], also employing neu-
ral networks to generate the control pulses [97, 98].
Until recently, it was unclear, however, how to extend
these ideas naturally to situations with feedback. The
reason is that the stochastic choice of measurement out-
comes is not directly compatible with taking gradients
through smooth dynamics, unless special care is taken.
A first example, applied to feedback based on weak lin-
ear measurements, was provided in [99], using automatic
differentiation. The continuous measurement outcomes FIG. 5. (a) The eventual goal of model-free reinforcement
in such situations can be written as a simple function learning is the direct application to experiments, which then
of a given Gaussian noise process. Very recently, a fully can be treated as a black box. Many actual implementations,
general approach, termed ‘feedback-GRAPE,’ was pre- however, use model-free RL techniques applied to model-
sented and analyzed in [100]. There, it was pointed out based simulations. (b) Model-based reinforcement learning
how the effect of discrete stochastic measurements can directly exploits the availability of a model, e.g. taking gradi-
ents through differentiable dynamics.
be properly considered in such a gradient-based setting.
This enables model-based optimization for feedback in-
volving arbitrary strong discrete measurements, where
the response to those outcomes can be represented via the simulation as an “observation” into the agent. Since
neural networks or trainable lookup tables. that would not be available in a real experiment, one pos-
sible solution is the use of a stochastic master equation
Future Challenges and Opportunities – As
to deduce the quantum evolution of the state based on
of the time of writing, experimental applications of
noisy measurement traces during the experimental run
reinforcement-learning-based quantum control are still in
in real time (see e.g. [92] for comments on this). An-
their infancy. Even for the easier, open-loop control case,
other option is a “two-stage learning” procedure, where
one has to set up a full pipeline where control sequences
the successful RL agent, which still uses the state as in-
are delivered to the setup and a suitable reward is ex-
put, is used for supervised training of another network
tracted from experimental measurement data, before be-
that only uses measurement results as input. This new
ing processed (e.g. externally, in a PC, implementing the
network can then be applied to an experiment [88]. How-
RL algorithm). In a quantum system, where measure-
ever, both approaches require some calibration of exper-
ments collapse the state, this often means the reward
imental parameters since the original training still relies
can only be obtained at the end of an experimental run,
on simulations.
making it harder to guide training. By contrast, having
an immediate reward after each time step would improve
training success because it assigns credit to the actions
that immediately preceded a high reward. D. Discovering quantum experiments, protocols,
The challenge becomes even larger when feedback con- and circuits
trol is called for. Then, we require an agent (a neural
network) that can process in real time the incoming mea- Inventing a blueprint for a quantum experiment, cir-
surement signals and decide on the subsequent actions. cuits or protocols requires ensuring that complex quan-
Depending on the hardware platform, this imposes se- tum phenomena play together to produce a quantum
vere constraints: for superconducting qubits, the time state or a quantum transformation from a limited set of
afforded to one such evaluation may be on the order of basic building blocks. In this section we will discuss de-
only a few hundred nanoseconds. This kind of challenge sign questions that involve discrete building, such as op-
is specific to real-time feedback and does not exist for tical elements, superconducting circuit elements or quan-
any of the other machine learning applications to quan- tum gates. While these systems also contain additional
tum devices. continuous parameters, the overarching discrete nature
Almost all of the applications of model-free RL tech- gives in some cases the possibility – as we will see – to
niques mentioned above are numerical and rely on some understand and learn from the solutions.
simulation of the quantum device (if only because realiz- Discovery of Quantum Experiments – An impor-
ing these approaches in an experiment is still technically tant and natural playground for discovery based on dis-
very demanding). In many publications, access to the crete building blocks is the invention of new experimental
simulation is furthermore utilized to make learning easier, setups.
e.g. by feeding the current quantum state obtained from In the field of quantum optics, those building blocks

FIG. 6. Discovery of Quantum Experiments. Quantum op-
tics experiments can be represented by colored graphs. Using
the most general, complete graph as a starting representa-
tion, the AI’s goal is to extract the conceptual core of the
solution, which can then be understood by human scientists.
The solution can then be translated to numerous different
experimental configurations[101].
contain lasers, nonlinear crystals, beam splitters, holo-
grams, photon detectors, and alike. Put together in a
specific way, those systems can lead to the generation
of complex quantum entanglement or perform quantum
teleportation – but exchange one single element, and the
setup produces something entirely different and most
likely not useful. Conventionally, experienced and cre-
ative scientists use their intuition and insights to design
new quantum experiments. They translate an abstract
task into a concrete layout that can be built in an ex-
perimental laboratory. However, human researchers are
struggling to find suitable experimental setups for more
complex quantum states and transformations.
This challenge was met recently with the introduction
of automated discovery of quantum experiments from
scratch[102, 103]. The general idea is that an algorithm
combines building blocks from a toolbox to produce a
suitable experimental setup while optimizing for certain
desired characteristics. In the setting of [102, 103], the al-
gorithm (here called Melvin), puts optical elements from
a toolbox onto a virtual optical table. The toolbox con-
sists of the experimental components available in the lab-
oratory (such as lasers and crystals). The algorithm ini-
tially starts to put elements from the toolbox in random
order on the table. If the candidate setup satisfies all
sanity checks, a simulator computes the full experimen-
tal output. If the setup produces the desired quantum
state, it is automatically simplified and reported to the
user. In addition, the setup is then stored as a new part
of the toolbox. In that way, over time, the algorithm
learns useful macro components which it can use in sub-
sequent iterations. Thereby, it can already access useful
operations that significantly speed up the discovery pro-
cess.
This algorithm has produced experimental blueprints
that enabled the observation of numerous new quantum
13
phenomena in laboratories [103]. Furthermore, new con-
cepts and ideas have been discovered, understood, and
generalized from some of the surprising solutions of the
algorithm, such as an entirely new way to multi-photon
interactions [104].
Sequentially building an experimental setup can be for-
mulated as a reinforcement learning problem. This pos-
sibility has been explored in [105]. The approach led to
the rediscovery of several experimental setups and to the
automated simplification of experimental setups (which
before was done using hand-crafted algorithms).
An alternative approach, called Theseus, which is or-
ders of magnitudes faster than the previous techniques,
is based on a new abstract representation of quantum ex-
periments [101]. Here, quantum experimental setups are
translated into a graph-based representations, see Fig.6.
Any quantum optical experiment that can be built in the
laboratory can be represented with a colored weighted
graph, which translates an in principle infinite search
space into a finite space with continuous (thus differen-
tiable) parameters. In that way, the question of finding
a certain quantum setup can be directly translated into
discovering a graph with certain properties. As the pa-
rameters are continuous, highly efficient gradient-based
optimization algorithms can be used to find the solutions.
In addition, the graph’s topology is eventually simplified,
such that the human researcher is not only presented with
a solution, but can immediately understand why and how
the solution works. The algorithm has been used to an-
swer numerous open questions and has led to new con-
cepts. It showcases that – when a simulator is available
(i.e. a model) and no feedback needs to be taken into ac-
count, gradient-descent optimization at a large abstract
representation often outperforms approaches such as ge-
netic algorithms networks or reinforcement learning.
A different approach, denoted Tachikoma, aims at
the discovery of new experimental setups for quantum
metrology[59, 60] – using an evolutionary learning ap-
proach. The goal of Tachikoma is to find setups for
quantum states that can measure phase shifts efficiently
and with high precision. It uses a toolbox of optical
elements from which it builds up a pool of candidate
solutions. The next generation is produced from the
best-performing parent setups. Those are merged and
mutated to create the new pool of candidates. In that
way, the population improves its performance over time
and leads to numerous counter-intuitive and exotic solu-
tions. One of the computationally expensive operations
and bottlenecks is the computation of the fidelity of the
quantum state. For that, the authors have extended the
approach by using a neural network that can classify the
quantum states from the setup. This combination of an
evolutionary algorithm with a neural network has discov-
ered experimental blueprints with yet unachieved quan-
tum metrology advantage. While much effort has been
put into constraining the system to realistic solutions,
it remains to be seen whether these experiments can be
built in the laboratory and achieve the expected quality.

An entirely different approach uses logical artificial
intelligence for designing quantum optical experiments
[106]. While the credo of the deep learning community
is to build large neural networks that can solve arbitrary
tasks given large enough training examples and compute
power, this is not the only way towards ”intelligent” algo-
rithms. An alternative is logic AI [107]. Here, the idea is
to translate arbitrary problems to Boolean satisfiability
expressions and solve them with powerful SAT solvers. In
[106], the question of designing quantum experiments has
been rephrased into logical expressions and solved with
MiniSAT. It is shown that in some problems, a combina-
tion of Theseus and the logical approach is faster than
the continuous optimization itself. The reason is that the
unsatisfiability of candidate solutions is detected quickly
with a logical approach, thus guiding the continuous opti-
mizer towards more promising candidates. This approach
is in its infancy. Given that the field of logic AI is grow-
ing fast due to computational and algorithmic advances,
we expect a large increase in interest in this topic.
Deep generative models such as variational autoen-
coders became a standard tool in fields such as mate-
rial design [108]. Here, an encoder network transforms
a (potentially discrete) representation into a continuous
latent space. The decoder network is trained to take a
point in the latent space and translate it back to the dis-
crete structure. The encoder and decoder together are
trained to perform an identity transformation, which by
itself is not that interesting. However, as an exciting
side effect, the system builds up an internal, continu-
ous latent space that can be shaped during the training
and used for gradient-based optimization. For the first
time, such a system was demonstrated for quantum op-
tics in [109]. The work focuses on understanding what
the neural networks have learned and how they store the
information in their internal latent space. The structure
of the latent space shows surprising discrete structures
that were then identified with concrete properties of the
experimental setups. It will be interesting to see more
advanced ways to investigate, navigate and understand
the high-dimensional internal representations of neural
networks that are built autonomously during training.
A conceptually related task is the design of supercon-
ducting circuits. The quantum behavior of superconduct-
ing circuits is defined by a network of inductances, ca-
pacitances, and Josephson junctions. As with quantum
optical experiments, those systems are conventionally de-
signed by experienced human researchers who aim to
find suitable configurations for complex quantum trans-
formations, such as coupling between two well-defined
qubits in quantum computers. The search space of pos-
sible structures grows exponentially with the number of
elements, and thus it quickly becomes infeasible for hu-
mans to find solutions for complex tasks. In [110], the
authors addressed the question of designing supercon-
ducting circuits for the first time with a fully automated
closed-loop optimization approach and designed a 4-local
coupler by which four superconducting flux qubits inter-
14
act. The algorithm SCILLA starts with a discrete circuit
topology. The best candidates are further parametrically
optimized, either with a direct gradient-based optimiza-
tion or with an evolutionary approach (to avoid local
minima). The final design outperforms the only other
(hand-crafted) 4-local coupler in terms of noise resilience
and coupling strength.
Discovering Quantum Protocols and Discrete
Feedback Strategies –
Discrete building blocks occur not only naturally in the
construction of experiments, but also as part of discrete
temporal sequences, specifically sequences of quantum
gates and other operations. These sequences can rep-
resent quantum protocols or higher-level control strate-
gies for quantum devices. The hardware-level control dis-
cussed in the previous section III C could then be consid-
ered as a tool to implement the individual building blocks
(e.g. an individual gate). In the following we will deal
with protocols that also contain elements of feedback or
other actions that are not merely unitary gates. The task
of quantum circuit synthesis (building up unitaries out of
elements) will be discussed further below.
Reinforcement learning is one suiting tool for the au-
tomated discovery of such sequences. This was first an-
alyzed in [88], using deep RL, where the goal was to dis-
cover a strategy for quantum error correction in a quan-
tum memory register made of a few qubits. This involves
applying discrete unitary gates, which are conditioned on
the outcomes of measurements, i.e. ’real-time’ feedback
executed during the control sequence, Fig. 7a. Since the
aim of [88] was primarily to find quantum error correction
strategies, we will discuss some more aspects separately
in the upcoming section III E.
A reinforcement learning technique was subsequently
also applied to the rediscovery of implementations for
quantum communication protocols [111]. There, the au-
thors set up the task as an RL problem and explain the
similarity of quantum communication and RL with the
following intuition: A quantum communication protocol
is a sequence of operations that leads to the desired out-
come. Similarly, an RL agent learns a policy, that is, to
perform sequences of operations that maximize a reward
function. The authors task the RL agent to rediscover
several important quantum communication schemes such
as quantum teleportation, quantum state purification, or
entanglement swapping. Each of these tasks can be writ-
ten as a simple network, where the nodes stand for the
involved parties and edges indicate classical or quantum
correlations between them. Let us take the quantum tele-
portation protocol as an example (the others follow simi-
lar ideas). The environment is a three-node network (the
incoming unknown quantum state A, the sender B, and
the receiver C). The environment starts with pre-shared
entanglement between B and C. The agent now has to
find a correct sequence of local measurements and clas-
sical communication steps that teleports the quantum
state from A to C. After performing up to 50 opera-
tions, the transformations are evaluated, and the agent

FIG. 7. Discovery of Quantum Circuits and Feedback Strate-
gies with Discrete Gates. (a) A reinforcement-learning agent
acts on a multi-qubit system by selecting gates, potentially
conditioned on measurement outcomes, finding an optimized
quantum circuit or quantum feedback strategy. (b) A fixed
layout quantum circuit with adjustable parameters that can
be optimized via gradient ascent to achieve some goal like
state preparation or variational ground state search (possibly
including feedback).
gets a (binary) reward for whether it succeeded or not.
Over 100k trials, the agent finds with high probability
an action-efficient strategy to perform the task. As there
is no feedback from the environment and a model of the
system exists, RL agents are not the only option to solve
these questions, and direct gradient-based methods can
be used for discovering quantum protocols.
Quantum Circuits – The design of quantum circuits
has some relation to the design of quantum experiments.
In both situations, a discrete set of parametrized ele-
ments are carefully connected to form the topology of the
circuit or experiment, while the continuous parameters of
the elements (such as phases) are optimized. In contrast
to quantum circuits that can use a universal gate set, for
experimental design or the quantum protocols considered
above it is not clear in the beginning whether certain tar-
gets can be reached with the available resources.
Quantum circuit design problems broadly fall into two
branches: First, quantum circuit synthesis (QCS) (some-
times called quantum circuit compilation) addresses the
problem of how to build from scratch a circuit that per-
forms a specific task. Second, quantum circuit optimiza-
tion (QCO) aims at turning a given circuit into a simpli-
fied, logically equivalent circuit.
The problem of quantum circuit synthesis is translat-
ing an algorithm into elementary gates from a finite uni-
versal set. There are two situations – the fully discrete
case, where gates are fixed, and the case where gates
can be tuned via continuous parameters (e.g. rotation
angles). The first application of machine-learning tech-
niques for the de-novo generation of quantum circuits
used genetic algorithms [112]. The algorithm had access
to a set of single- and two-qubit gates and was tasked
to rediscover the quantum circuit for quantum teleporta-
tion. It indeed found the correct circuit with significantly
15
less evaluations than an exhaustive search and could also
present different solutions not discussed in the literature
before. Genetic algorithms are a powerful tool for dis-
crete discovery. Thus, related approaches are still being
used two decades later [113, 114].
The problem of designing a quantum circuit from an
unparameterized set of elementary gates can be formu-
lated as a reinforcement learning problem. In [115], the
authors introduce a deep reinforcement learning algo-
rithm that translates an arbitrary single-qubit gate into
a sequence of elementary gates from a finite universal
set for a topological quantum computer. The authors
apply their algorithms to Fibonacci anyons and discover
high-quality braiding sequences. Similar machine learn-
ing techniques have been applied to the quantum circuits
for gate-based quantum computers [116]. The design of
unparameterized quantum circuits has an important ap-
plication for fault-tolerant quantum computation, where
only a finite list of unparameterized gates can be applied
to the logical qubits.
In general, quantum circuits can have tunable param-
eters – for instance, parameterized X-gates. In that case,
the problem has both a discrete and continuous element.
An important application for this task is the hardware-
aware design of circuits[117, 118]. Here, the algorithms
consider the circuit’s connectivity on the one hand and
noise contributions on the other. The main goal is to find
shallow circuits that are more noise-resistant than other
(textbook) implementations of circuits. The authors of
[117, 118] present an algorithm that can significantly out-
perform textbook solutions for various state generation
tasks in terms of fidelity under realistic noise conditions.
The design of discrete circuits with continuous parame-
ters has also been approached with deep reinforcement
learning [119].
A crucial and heavily investigated topic in the area of
quantum circuit design is the parametric optimization of
a constant circuit topology. This task is essential for hy-
brid quantum-classical variational quantum algorithms
(VQA) that can run on near-term quantum computing
hardware [17, 18], as well as for quantum machine learn-
ing, where such parameterized circuits are used as quan-
tum neural networks [15]. We will not go into the details
of these topics, but refer the reader to the excellent re-
views on this topic.
We will mention only a few selected but important
results and ideas for completeness. First, it has been
discovered that the gradients in a randomly initialized
parametrized quantum circuit vanish exponentially with
the number of qubits [120], which has become one of the
main challenges in the field of quantum machine learning
and VQA, Fig. 7b. An important question then is how
an expressive initial state of the circuit (ansatz) allows
for the efficient machine-learning-based optimization of
the circuits [121]. Some exciting approaches involve re-
inforcement learning that explores economic and expres-
sive initial ansätze [122] or ideas that are inspired from
neural network architecture search [123]. Besides the di-

rect gradient-based optimization of parametrized quan-
tum circuits, different approaches try to avoid the prob-
lem of vanishing gradients, by employing reinforcement
learning [124], using ML-based prediction of suitable ini-
tial parameters (rather than optimizing the parameters
directly) [125] or advanced gradient-free approaches that
are naturally not susceptible to the barren plateau prob-
lem [126].
An interesting recent application of VQA-based sys-
tems is the quantum-computer-aided design of quantum
hardware[127–129]. As described at the beginning of
this chapter, the AI-based design of new quantum hard-
ware on a classical computer has the problem of memory
requirements increasing exponentially with the system
size. One way to overcome this problem is to outsource
the computation of the expensive quantum system to a
quantum computer. Here, the problem of designing new
multi-qubit couplers for superconducting quantum com-
puters or the design of new quantum optics hardware
can be rephrased as in a VQA-style problem. A clas-
sical AI algorithm changes the parameter of a parame-
terized quantum circuit to minimize a fidelity function
computed from the outcome of the quantum computa-
tion. After convergence, a mapping translates the fi-
nal parametrized quantum circuit into the specific quan-
tum hardware. This approach has been experimentally
demonstrated in a proof-of-principle three-qubit super-
conducting circuit [129].
In general, it is not guaranteed that a direct compi-
lation of an algorithm already yields the most efficient
implementation of a quantum circuit. A powerful clas-
sical method to simplify (compile) quantum circuits is
the ZX formalism [130], which reformulates the circuit
into a graph, where predefined rules identify simplifica-
tions. However, this and similar approaches have been
formulated in a hardware-independent way, operating on
a global level. Alternatively, this problem can be ap-
proached by RL algorithms [131] that can autonomously
simplify circuits, for example, in terms of circuit depth or
gate counts, and this enables easily taking into account
concrete hard-ware constraints. In [131] this approach
was developed and found superior to simulated anneal-
ing (tested for circuits of up to 50 qubits). It has the
potential to become an important tool for simplifying
quantum circuits in the future.
Future Challenges and Opportunities – The sim-
ulation of quantum experiments becomes expensive as
soon as the system grows in size. Neural networks could
autonomously find approximate predictions for the dy-
namics of the quantum system. Such supervised systems
need a lot of training examples. Thus the trade-off be-
tween the creation of training examples and the compu-
tational benefit of an approximation needs to be investi-
gated.
The design of new experiments or hardware can not
only be seen as optimization (in the sense of making an
existing structure better) but as discovery in which we
create new ideas that did not exist before, as shown in
16
[101]. This point of view shows how machines can cre-
atively contribute to science and act as an inspiration
for human scientists. There will be a great potential for
expanding these ideas. Automatic extraction of under-
standable building blocks (’subroutines’) can help with
this challenge.
E. Quantum Error Correction
The ability to correct errors in a quantum computing
device will be indispensable to realizing beneficial ap-
plications of quantum computation since real-world de-
vices are not coherent enough to run an error-free cal-
culation. The basic conceptual ideas in this domain are
known since the pioneering work of Shor [132] and sub-
sequent developments, most notably the surface code. In
any case, the idea is to encode logical qubit information
in many physical qubits robustly and redundantly. The
presence of errors (like qubit dephasing and decay) must
be detected via measurement of so-called syndromes,
i.e. suitably chosen observables (often multi-qubit opera-
tors). Finally, a good way to interpret the observed syn-
dromes and apply some error correction procedure must
be found. Despite the knowledge of good encodings and
suitable syndromes, it remains a challenging problem how
to best implement those in practice, for a given quantum
device, with its available gate set and topology of con-
nections between the qubits, and how to optimize them
for a given noise model.
It has been recognized early on that machine learning
methods could potentially be of great help in this domain.
The tasks can naturally be divided into three categories.
Syndrome interpretation – On the simplest level,
we already assume an existing encoding and a fixed set
of syndromes. The task then is to find the optimal way
to interpret the observed syndrome, e.g. deciding which
qubits are likely erroneous and must be corrected, see
Fig. 8. This can be phrased as a supervised learning
problem, where some errors are simulated, the syndrome
is fed into a network, and the network must announce
the location of the errors. In practice, the surface code
is the most promising QEC architecture, and deducing
the error from the syndrome is not trivial, though non-
ML algorithms exist. Multiple works therefore trained
neural networks to yield “neural decoders” [133–136]. In
one early example [133], a modified, restricted Boltzmann
machine was used, with two types of visible units, cor-
responding to syndrome and underlying error configura-
tion. This was then trained on a data set of such pairs.
Afterward, the machine could be used to sample the er-
rors compatible with an observed syndrome. It is also
possible to use reinforcement learning to discover better
strategies in more complicated situations. In [137], this
was applied to the surface code, exemplified in a situation
with faulty syndrome measurements.
Code search – Going one step further, the question
arises whether a machine can also find better codes. It is

helpful to take an existing code and modify it. The sur-
face code is usually formulated on a simple square lattice,
but it can be implemented on more complicated geome-
tries, both periodic and even more generally a-periodic.
In [138], a reinforcement learning agent was asked to opti-
mize the connectivity of a surface code given a number of
data qubits. It was able to find the best performing code
in scenarios important for real experiments. These in-
cluded biased noise (not all error channels equally strong)
or spatially localized noise (higher error rates in the vicin-
ity of some qubits). The agent found interesting nontriv-
ial connectivities as optimal solutions. Due to the avail-
ability of highly efficient simulation tools for establishing
the performance of surface codes, the authors of [138]
were able to go up to 70 data qubits.
Autonomous quantum error correction consists of an
experimental configuration that can intrinsically correct
certain types of errors without active feedback. This idea
can be implemented by introducing carefully additional
drives and dissipation. The discovery of such mechanisms
in real physical systems, under strict experimental con-
straints, is highly nontrivial. The authors of [139] show
the automated discovery of an autonomous QEC that
could be applied to Bosonic systems. The goal is to find
an encoding of a logical qubit that is robust under the dy-
namics of the system. The algorithm denoted AutoQEC
can then discover such an encoding by maximizing the
average fidelity of the logical qubit. AutoQEC is further
constrained to consider only systems within experimental
capabilities. Indeed,
√ the authors discover a new quantum
code, denoted 3-code, that has a longer lifetime than
previously studied systems with the same concrete ex-
perimental constraints. The authors go on and, inspired
by their numerical discovery, derive the analytical, logi-
cal state and analyze the new autonomous QEC system
further.
Full QEC protocol discovery – Finally, one can
adopt the attitude that neither the code itself, the code
family, or any other ingredients are assumed. In that
case, one starts from scratch, and the goal of the machine
is to discover ways to preserve the quantum information
with high fidelity for as long as possible. In other words,
it (re-)discovers all aspects of QEC and error mitigation,
adapted to the given platform and noise model. Such an
ab-initio approach was demonstrated in [88], which we
already mentioned above in the context of RL for feed-
back. Given a few qubits with arbitrary connectivity and
gate set, as well as an arbitrary noise model, the agent
is asked to preserve the quantum information as long as
possible. To solve this challenging task, additional gen-
erally applicable insights were required, e.g. introduc-
ing a reward that can measure the amount of surviving
quantum information without having discovered a proper
decoding sequence. Beyond approaches that fall in the
family of stabilizer codes, the same agent also discovered
noise mitigation techniques based on adaptive measure-
ments. The advantage of such an ab-initio approach is
its flexibility, but the price to pay is that so far, it only
17
FIG. 8. Quantum Error Correction. Syndrome interpretation
in a surface code as a task that a neural network can be
trained to perform.
works for a handful of qubits due to the effort required
in simulations. A future challenge would be finding ways
to make the RL work directly on the experiment.
Future Challenges and Opportunities – Regard-
ing syndrome measurements, an important challenge for
the future is ensuring that the neural networks inter-
preting those measurement results can be deployed in
an actual device at sufficient speed: even for the clas-
sical algorithms, this is nontrivial. Another challenge
is that ab-initio discovery of quantum error correction
strategies still relies on simulations, whose numerical ef-
fort scales exponentially in the number of qubits (for gen-
eral quantum dynamics). An interesting direction could
also be the co-discovery of autonomous QEC experiments
together with QEC feedback strategies in these systems.
IV. OUTLOOK
With all these promising ideas in mind, let us look
forward to the year 2035: how do we imagine machine
learning to contribute to quantum technologies by that
time?
Fully controlled and error-corrected quantum
systems – As quantum platforms scale towards ever
greater numbers of components and connections, ma-
chine learning will provide a way to harness this com-
plexity – by automatically calibrating and fine-tuning the
resulting huge number of parameters adaptively, by dis-
covering optimized experimental setups in the first place,
by extracting the maximum amount of information from
rather indirect measurements of the underlying phenom-
ena via complex observables, and by finding smart con-
trol strategies. Errors in such systems could be corrected
by fully automated quantum error correction schemes
discovered by an AI. When we think of quantum com-
puters with 1000s of qubits or quantum simulators with
even more degrees of freedom, all of this will be a crucial
part of the community’s toolbox.
Specifying goals, not algorithms – One important
aspect here will be that instead of defining an algorithm
that tells the computer how to achieve some goal, we will
typically define the goal itself. Such goals could be to
retain a large fidelity during quantum operations, pro-
duce highly entangled states, or have a strong sensitivity

to some external signal. The details of how to reach this
goal will be left for the computer to discover. This change
of perspective will enable a much higher level of descrip-
tion, which is one way to keep ahead of the growing com-
plexity. Ultimately, one might expect that the machine
has access to the scientific literature and suggests goals
and new experiments autonomously, as demonstrated in
material science[140].
Discovering new Algorithms – Rather than discov-
ering experiments or feedback strategies, it will be very
interesting to see whether ML agents can autonomously
discover other higher-logic quantum programs such as
quantum algorithms. This task is recently been tack-
led by large language models for classical algorithms
[141, 142], and we expect that similarly quantum algo-
rithms can be discovered with classical machine learning
models.
How can the human learn? – Suppose that com-
puters will be able to help us find solutions for many
of the lower-level and even some higher-level, more con-
ceptual tasks in the domain of quantum technologies.
That raises the following notorious question, pervasive
throughout machine learning and artificial intelligence:
How can we human scientists understand what the ma-
chine has learned? Do we need to open the black box of
neural networks, or can we use the algorithms as a source
of inspiration in a different way [143]? We argue that
while improved performance in the task at hand is great,
being able to understand the essence of what the machine
has discovered is crucial for the result to become of much
wider applicability. In general, gaining understanding
has been called the essential aim of science[144]. Here
approaches where the solution involves discrete steps (e.g.
discrete actions of an agent) or logic-based AI seem to be
easier to interpret, explain and understand than results
from deep-learning-based methods. The field of symbolic
regression (which extracts discrete explanations of neu-
ral network predictions) might be very fruitful in this
approach.
What needs to be done? – To attain the visions
described above, our community may adopt some proven
methodologies from other areas. The idea of fair bench-
marks and competitions is one of the powerful driving
forces in the development of ML algorithms. One of
[1] M. I. Jordan and T. M. Mitchell, Machine learning:
Trends, perspectives, and prospects, Science 349, 255
(2015).
[2] Y. LeCun, Y. Bengio, and G. Hinton, Deep learning,
Nature 521, 436 (2015).
[3] I. Goodfellow, Y. Bengio, and A. Courville, Deep learn-
ing (MIT press, 2016).
[4] J. P. Dowling and G. J. Milburn, Quantum technology:
the second quantum revolution, Philosophical Transac-
tions of the Royal Society of London. Series A: Math-
ematical, Physical and Engineering Sciences 361, 1655
18
the most famous examples being the ImageNet data set
which provided the basis for a revolution of ML-based
computer vision systems [145]. This idea was adapted in
other fields of science that apply AI methodologies, such
as material discovery[146, 147]. In contrast, the field
of AI in quantum technology, at the moment, appears
more like the wild west. There are no clear ways how to
compare approaches from different papers, because most
works apply their approaches to slightly different tasks,
making them incomparable. We believe that fair and
suitably curated benchmark data sets will steer the devel-
opment of powerful and ever more generally applicable AI
algorithms in quantum technology. The data sets could
consist of simulated or (in the best case) experimental
data for data interpretation tasks. Likewise, to facili-
tate the discovery of experimental setups and protocols,
the community can develop a selection of well-curated
objective functions and a set of simulated environments
describing important prototypical quantum devices (see
SciGym for a first attempt at this [148]). In a similar
direction, we expect that cloud access to real quantum
experiments will become available for significantly more
systems. AI algorithms can then be trained on the data
from these real machines with specific experimental con-
straints (such as connectivity or noise). This will boost
the capabilities of algorithms that deal with important,
real-world systems.
Finally, what Alan Turing remarked in his visionary
article on intelligence and learning machines [149] is also
valid here, in the field of machine learning applied to
quantum technologies: ”We can only see a short dis-
tance ahead, but we can see plenty there that needs to
be done.”
V. ACKNOWLEDGMENTS
The authors thank Sören Arlt and Xuemei Gu for help-
ful comments on the manuscript. F.M. acknowledges
funding by the Munich Quantum Valley, which is sup-
ported by the Bavarian state government with funds from
the Hightech Agenda Bayern Plus.
(2003).
[5] F. Flamini, N. Spagnolo, and F. Sciarrino, Photonic
quantum information processing: a review, Reports on
Progress in Physics 82, 016001 (2018).
[6] I. M. Georgescu, S. Ashhab, and F. Nori, Quantum sim-
ulation, Reviews of Modern Physics 86, 153 (2014).
[7] J. Preskill, Quantum Computing in the NISQ era and
beyond, Quantum 2, 79 (2018).
[8] M. C. Angelini and F. Ricci-Tersenghi, Cracking nuts
with a sledgehammer: when modern graph neural
networks do worse than classical greedy algorithms,

arXiv:2206.13211 (2022).
[9] F. Marquardt, Machine learning and quantum devices,
SciPost Physics Lecture Notes , 29 (2021).
[10] A. Dawid, J. Arnold, B. Requena, A. Gresch,
M. Plodzień, K. Donatella, K. A. Nicoli, P. Stornati,
R. Koch, M. Büttner, et al., Modern applications of
machine learning in quantum sciences, arXiv:2204.04198
(2022).
[11] G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld,
N. Tishby, L. Vogt-Maranto, and L. Zdeborová, Ma-
chine learning and the physical sciences, Reviews of
Modern Physics 91, 045002 (2019).
[12] J. Carrasquilla, Machine learning for quantum matter,
Advances in Physics: X 5, 1797528 (2020).
[13] M. Schuld, I. Sinayskiy, and F. Petruccione, An intro-
duction to quantum machine learning, Contemporary
Physics 56, 172 (2015).
[14] J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost,
N. Wiebe, and S. Lloyd, Quantum machine learning,
Nature 549, 195 (2017).
[15] V. Dunjko and P. Wittek, A non-review of Quantum
Machine Learning: trends and explorations, Quantum
Views 4, 32 (2020).
[16] L. Lamata, Quantum machine learning and quantum
biomimetics: A perspective, Machine Learning: Science
and Technology 1, 033002 (2020).
[17] M. Cerezo, A. Arrasmith, R. Babbush, S. C. Benjamin,
S. Endo, K. Fujii, J. R. McClean, K. Mitarai, X. Yuan,
L. Cincio, et al., Variational quantum algorithms, Na-
ture Reviews Physics 3, 625 (2021).
[18] K. Bharti, A. Cervera-Lierta, T. H. Kyaw, T. Haug,
S. Alperin-Lea, A. Anand, M. Degroote, H. Heimonen,
J. S. Kottmann, T. Menke, W.-K. Mok, S. Sim, L.-C.
Kwek, and A. Aspuru-Guzik, Noisy intermediate-scale
quantum algorithms, Reviews of Modern Physics 94,
015004 (2022).
[19] K. Bharti, T. Haug, V. Vedral, and L.-C. Kwek, Ma-
chine learning meets quantum foundations: A brief sur-
vey, AVS Quantum Science 2, 034101 (2020).
[20] S. J. Russell and P. Norvig, Artificial Intelligence: A
Modern Approach, 4th ed. (Pearson, 2021).
[21] D. Whitley, A genetic algorithm tutorial, Statistics and
computing 4, 65 (1994).
[22] S. N. Sivanandam and S. N. Deepa, Introduction to Ge-
netic Algorithms (Springer Berlin, Heidelberg, 2008).
[23] G. Cybenko, Approximation by superpositions of a sig-
moidal function, Mathematics of Control, Signals and
Systems 2, 303 (1989).
[24] S. Hochreiter and J. Schmidhuber, Long Short-Term
Memory, Neural Computation 9, 1735 (1997).
[25] M. A. Nielsen, Neural Networks and Deep Learning (De-
termination Press, 2015).
[26] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre,
G. van den Driessche, J. Schrittwieser, I. Antonoglou,
V. Panneershelvam, M. Lanctot, et al., Mastering the
game of Go with deep neural networks and tree search,
Nature 529, 484 (2016).
[27] O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Math-
ieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell,
T. Ewalds, P. Georgiev, et al., Grandmaster level in
StarCraft II using multi-agent reinforcement learning,
Nature 575, 350 (2019).
[28] R. S. Sutton and A. G. Barto, Reinforcement Learning:
An Introduction (MIT Press, Cambridge, MA, 2018).
19
[29] E. Magesan, J. M. Gambetta, A. D. Córcoles, and J. M.
Chow, Machine Learning for Discriminating Quan-
tum Measurement Trajectories and Improving Readout,
Physical Review Letters 114, 200501 (2015).
[30] A. Seif, K. A. Landsman, N. M. Linke, C. Figgatt,
C. Monroe, and M. Hafezi, Machine learning assisted
readout of trapped-ion qubits, Journal of Physics B:
Atomic, Molecular and Optical Physics 51, 174006
(2018).
[31] G. Liu, M. Chen, Y.-X. Liu, D. Layden, and P. Cappel-
laro, Repetitive readout enhanced by machine learning,
Machine Learning: Science and Technology 1, 015003
(2020).
[32] E. Flurin, L. S. Martin, S. Hacohen-Gourgy, and I. Sid-
diqi, Using a Recurrent Neural Network to Reconstruct
Quantum Dynamics of a Superconducting Qubit from
Physical Observations, Physical Review X 10, 011006
(2020).
[33] I. Agresti, N. Viggianiello, F. Flamini, N. Spagnolo,
A. Crespi, R. Osellame, N. Wiebe, and F. Sciarrino, Pat-
tern Recognition Techniques for Boson Sampling Vali-
dation, Physical Review X 9, 011013 (2019).
[34] S. Aaronson, The learnability of quantum states, Pro-
ceedings of the Royal Society A: Mathematical, Physical
and Engineering Sciences 463, 3089 (2007).
[35] A. Rocchetto, S. Aaronson, S. Severini, G. Carvacho,
D. Poderini, I. Agresti, M. Bentivegna, and F. Sciar-
rino, Experimental learning of quantum states, Science
Advances 5, eaau1946 (2019).
[36] H.-Y. Huang, M. Broughton, J. Cotler, S. Chen,
J. Li, M. Mohseni, H. Neven, R. Babbush, R. Kueng,
J. Preskill, et al., Quantum advantage in learning from
experiments, Science 376, 1182 (2022).
[37] V. Cimini, M. Barbieri, N. Treps, M. Walschaers,
and V. Parigi, Neural Networks for Detecting Multi-
mode Wigner Negativity, Physical Review Letters 125,
160504 (2020).
[38] J. Carrasquilla and R. G. Melko, Machine learning
phases of matter, Nature Physics 13, 431 (2017).
[39] E. P. L. van Nieuwenburg, Y.-H. Liu, and S. D. Huber,
Learning phase transitions by confusion, Nature Physics
13, 435 (2017).
[40] S. J. Wetzel, Unsupervised learning of phase transitions:
From principal component analysis to variational au-
toencoders, Physical Review E 96, 022140 (2017).
[41] K. Kottmann, P. Huembeli, M. Lewenstein, and
A. Acı́n, Unsupervised Phase Discovery with Deep
Anomaly Detection, Physical Review Letters 125,
170603 (2020).
[42] N. Käming, A. Dawid, K. Kottmann, M. Lewenstein,
K. Sengstock, A. Dauphin, and C. Weitenberg, Unsu-
pervised machine learning of topological phase transi-
tions from experimental data, Machine Learning: Sci-
ence and Technology 2, 035037 (2021).
[43] F. Arute, K. Arya, R. Babbush, D. Bacon, J. C. Bardin,
R. Barends, R. Biswas, S. Boixo, F. G. S. L. Brandao,
D. A. Buell, et al., Quantum supremacy using a pro-
grammable superconducting processor, Nature 574, 505
(2019).
[44] G. Carleo and M. Troyer, Solving the quantum many-
body problem with artificial neural networks, Science
355, 602 (2017).
[45] R. G. Melko, G. Carleo, J. Carrasquilla, and J. I. Cirac,
Restricted Boltzmann machines in quantum physics,

Nature Physics 15, 887 (2019).
[46] G. Torlai, G. Mazzola, J. Carrasquilla, M. Troyer,
R. Melko, and G. Carleo, Neural-network quantum state
tomography, Nature Physics 14, 447 (2018).
[47] G. Torlai, B. Timar, E. P. L. van Nieuwenburg,
H. Levine, A. Omran, A. Keesling, H. Bernien,
M. Greiner, V. Vuletić, M. D. Lukin, et al., Integrating
Neural Networks with a Quantum Simulator for State
Reconstruction, Physical Review Letters 123, 230504
(2019).
[48] M. Schuld, I. Sinayskiy, and F. Petruccione, Neural Net-
works Take on Open Quantum Systems, Physics 12, 74
(2019).
[49] A. Nagy and V. Savona, Variational Quantum Monte
Carlo Method with a Neural-Network Ansatz for
Open Quantum Systems, Physical Review Letters 122,
250501 (2019).
[50] M. J. Hartmann and G. Carleo, Neural-Network Ap-
proach to Dissipative Quantum Many-Body Dynamics,
Physical Review Letters 122, 250502 (2019).
[51] F. Vicentini, A. Biella, N. Regnault, and C. Ciuti, Vari-
ational Neural-Network Ansatz for Steady States in
Open Quantum Systems, Physical Review Letters 122,
250503 (2019).
[52] N. Yoshioka and R. Hamazaki, Constructing neural sta-
tionary states for open quantum many-body systems,
Physical Review B 99, 214306 (2019).
[53] T. Adler, M. Erhard, M. Krenn, J. Brandstetter,
J. Kofler, and S. Hochreiter, Quantum Optical Experi-
ments Modeled by Long Short-Term Memory, Photonics
8, 535 (2021).
[54] N. Mohseni, T. Fösel, L. Guo, C. Navarrete-Benlloch,
and F. Marquardt, Deep Learning of Quantum Many-
Body Dynamics via Random Driving, Quantum 6, 714
(2022).
[55] M. Choi, D. Flam-Shepherd, T. H. Kyaw, and
A. Aspuru-Guzik, Learning quantum dynamics with la-
tent neural ordinary differential equations, Physical Re-
view A 105, 042403 (2022).
[56] D. T. Lennon, H. Moon, L. C. Camenzind, L. Yu, D. M.
Zumbühl, G. A. D. Briggs, M. A. Osborne, E. A. Laird,
and N. Ares, Efficiently measuring a quantum device
using machine learning, npj Quantum Information 5,
79 (2019).
[57] E. Polino, M. Valeri, N. Spagnolo, and F. Sciarrino,
Photonic quantum metrology, AVS Quantum Science 2,
024703 (2020).
[58] V. Cimini, I. Gianani, N. Spagnolo, F. Leccese, F. Scia-
rrino, and M. Barbieri, Calibration of Quantum Sen-
sors by Neural Networks, Physical Review Letters 123,
230502 (2019).
[59] P. A. Knott, A search algorithm for quantum state en-
gineering and metrology, New Journal of Physics 18,
073033 (2016).
[60] R. Nichols, L. Mineh, J. Rubio, J. C. F. Matthews, and
P. A. Knott, Designing quantum experiments with a
genetic algorithm, Quantum Science and Technology 4,
045012 (2019).
[61] D. W. Berry and H. M. Wiseman, Optimal States
and Almost Optimal Adaptive Measurements for Quan-
tum Interferometry, Physical Review Letters 85, 5098
(2000).
[62] D. W. Berry, H. Wiseman, and J. Breslin, Optimal in-
put states and feedback for interferometric phase esti-
20
mation, Physical Review A 63, 053804 (2001).
[63] B. L. Higgins, D. W. Berry, S. D. Bartlett, H. M. Wise-
man, and G. J. Pryde, Entanglement-free Heisenberg-
limited phase estimation, Nature 450, 393 (2007).
[64] A. Hentschel and B. C. Sanders, Machine Learning for
Precise Quantum Measurement, Physical Review Let-
ters 104, 063603 (2010).
[65] A. Hentschel and B. C. Sanders, Efficient Algorithm for
Optimizing Adaptive Quantum Metrology Processes,
Physical Review Letters 107, 233601 (2011).
[66] N. B. Lovett, C. Crosnier, M. Perarnau-Llobet, and
B. C. Sanders, Differential Evolution for Many-Particle
Adaptive Quantum Metrology, Physical Review Letters
110, 220501 (2013).
[67] P. Palittapongarnpim, P. Wittek, E. Zahedinejad,
S. Vedaie, and B. C. Sanders, Learning in quantum
control: High-dimensional global optimization for noisy
quantum dynamics, Neurocomputing 268, 116 (2017).
[68] H. Xu, J. Li, L. Liu, Y. Wang, H. Yuan, and X. Wang,
Generalizable control for quantum parameter estima-
tion through reinforcement learning, npj Quantum In-
formation 5, 82 (2019).
[69] J. Schuff, L. J. Fiderer, and D. Braun, Improving the
dynamics of quantum sensors with reinforcement learn-
ing, New Journal of Physics 22, 035001 (2020).
[70] E. G. Ryan, C. C. Drovandi, J. M. McGree, and A. N.
Pettitt, A Review of Modern Computational Algorithms
for Bayesian Optimal Design, International Statistical
Review 84, 128 (2016).
[71] V. Gebhart, R. Santagati, A. A. Gentile, E. Gauger,
D. Craig, N. Ares, L. Banchi, F. Marquardt,
L. Pezze, and C. Bonato, Learning Quantum Systems,
arXiv:2207.00298 (2022).
[72] R. Durrer, B. Kratochwil, J. V. Koski, A. J. Landig,
C. Reichl, W. Wegscheider, T. Ihn, and E. Greplova,
Automated Tuning of Double Quantum Dots into Spe-
cific Charge States Using Neural Networks, Physical Re-
view Applied 13, 054019 (2020).
[73] N. Wiebe, C. Granade, C. Ferrie, and D. G. Cory,
Hamiltonian Learning and Certification Using Quantum
Resources, Physical Review Letters 112, 190501 (2014).
[74] J. Wang, S. Paesani, R. Santagati, S. Knauer, A. A.
Gentile, N. Wiebe, M. Petruzzella, J. L. O’Brien, J. G.
Rarity, A. Laing, et al., Experimental quantum Hamil-
tonian learning, Nature Physics 13, 551 (2017).
[75] A. A. Gentile, B. Flynn, S. Knauer, N. Wiebe, S. Pae-
sani, C. E. Granade, J. G. Rarity, R. Santagati, and
A. Laing, Learning models of quantum systems from
experiments, Nature Physics 17, 837 (2021).
[76] N. Khaneja, T. Reiss, C. Kehlet, T. Schulte-
Herbrüggen, and S. J. Glaser, Optimal control of cou-
pled spin dynamics: design of NMR pulse sequences by
gradient ascent algorithms, Journal of magnetic reso-
nance 172, 296 (2005).
[77] V. F. Krotov, Global methods in optimal control theory,
Monographs and textbooks in pure and applied mathe-
matics, Vol. 195 (Marcel Dekker, New York, 1996).
[78] J. Somlói, V. A. Kazakov, and D. J. Tannor, Controlled
dissociation of I2 via optical transitions between the
X and B electronic states, Chemical Physics 172, 85
(1993).
[79] P. Doria, T. Calarco, and S. Montangero, Optimal
Control Technique for Many-Body Quantum Dynamics,
Physical Review Letters 106, 190501 (2011).

[80] R. S. Judson and H. Rabitz, Teaching lasers to control
molecules, Physical Review Letters 68, 1500 (1992).
[81] C. Chen, D. Dong, H.-X. Li, J. Chu, and T.-J. Tarn,
Fidelity-Based Probabilistic Q-Learning for Control of
Quantum Systems, IEEE Transactions on Neural Net-
works and Learning Systems 25, 920 (2013).
[82] M. Bukov, A. G. R. Day, D. Sels, P. Weinberg,
A. Polkovnikov, and P. Mehta, Reinforcement Learn-
ing in Different Phases of Quantum Control, Physical
Review X 8, 031086 (2018).
[83] M. August and J. M. Hernández-Lobato, Taking Gra-
dients Through Experiments: LSTMs and Memory
Proximal Policy Optimization for Black-Box Quantum
Control, in High Performance Computing, edited by
R. Yokota, M. Weiland, J. Shalf, and S. Alam (Springer
International Publishing, Cham, 2018) pp. 591–613.
[84] X.-M. Zhang, Z.-W. Cui, X. Wang, and M.-H. Yung,
Automatic spin-chain learning to explore the quantum
speed limit, Physical Review A 97, 052333 (2018).
[85] M. Y. Niu, S. Boixo, V. N. Smelyanskiy, and H. Neven,
Universal quantum control through deep reinforcement
learning, npj Quantum Information 5, 33 (2019).
[86] Z. An and D. L. Zhou, Deep reinforcement learning for
quantum gate control, Europhysics Letters 126, 60002
(2019).
[87] Y. Baum, M. Amico, S. Howell, M. Hush, M. Liuzzi,
P. Mundada, T. Merkh, A. R. R. Carvalho, and M. J.
Biercuk, Experimental Deep Reinforcement Learning
for Error-Robust Gate-Set Design on a Superconducting
Quantum Computer, PRX Quantum 2, 040324 (2021).
[88] T. Fösel, P. Tighineanu, T. Weiss, and F. Mar-
quardt, Reinforcement Learning with Neural Networks
for Quantum Feedback, Physical Review X 8, 031084
(2018).
[89] J. Mackeprang, D. B. R. Dasari, and J. Wrachtrup, A
reinforcement learning approach for quantum state en-
gineering, Quantum Machine Intelligence 2, 5 (2020).
[90] Z. T. Wang, Y. Ashida, and M. Ueda, Deep Reinforce-
ment Learning Control of Quantum Cartpoles, Physical
Review Letters 125, 100401 (2020).
[91] S. Borah, B. Sarma, M. Kewming, G. J. Milburn,
and J. Twamley, Measurement-Based Feedback Quan-
tum Control with Deep Reinforcement Learning for a
Double-Well Nonlinear Potential, Physical Review Let-
ters 127, 190403 (2021).
[92] R. Porotti, A. Essig, B. Huard, and F. Marquardt, Deep
Reinforcement Learning for Quantum State Preparation
with Weak Nonlinear Measurements, Quantum 6, 747
(2022).
[93] V. V. Sivak, A. Eickbusch, H. Liu, B. Royer, I. Tsiout-
sios, and M. H. Devoret, Model-Free Quantum Control
with Reinforcement Learning, Physical Review X 12,
011059 (2022).
[94] N. Leung, M. Abdelhafez, J. Koch, and D. Schuster,
Speedup for quantum optimal control from automatic
differentiation based on graphics processing units, Phys-
ical Review A 95, 042318 (2017).
[95] M. Abdelhafez, D. I. Schuster, and J. Koch, Gradient-
based optimal control of open quantum systems us-
ing quantum trajectories and automatic differentiation,
Physical Review A 99, 052327 (2019).
[96] M. Abdelhafez, B. Baker, A. Gyenis, P. Mundada, A. A.
Houck, D. Schuster, and J. Koch, Universal gates for
protected superconducting qubits using optimal control,
21
Physical Review A 101, 022321 (2020).
[97] F. Schäfer, M. Kloc, C. Bruder, and N. Lörch, A dif-
ferentiable programming method for quantum control,
Machine Learning: Science and Technology 1, 035009
(2020).
[98] L. Coopmans, D. Luo, G. Kells, B. K. Clark, and J. Car-
rasquilla, Protocol Discovery for the Quantum Con-
trol of Majoranas by Differentiable Programming and
Natural Evolution Strategies, PRX Quantum 2, 020332
(2021).
[99] F. Schäfer, P. Sekatski, M. Koppenhöfer, C. Bruder, and
M. Kloc, Control of stochastic quantum dynamics by
differentiable programming, Machine Learning: Science
and Technology 2, 035004 (2021).
[100] R. Porotti, V. Peano, and F. Marquardt, Gra-
dient Ascent Pulse Engineering with Feedback,
arXiv:2203.04271 (2022).
[101] M. Krenn, J. S. Kottmann, N. Tischler, and A. Aspuru-
Guzik, Conceptual Understanding through Efficient
Automated Design of Quantum Optical Experiments,
Physical Review X 11, 031044 (2021).
[102] M. Krenn, M. Malik, R. Fickler, R. Lapkiewicz, and
A. Zeilinger, Automated Search for new Quantum Ex-
periments, Physical Review Letters 116, 090405 (2016).
[103] M. Krenn, M. Erhard, and A. Zeilinger, Computer-
inspired quantum experiments, Nature Reviews Physics
2, 649 (2020).
[104] X. Gao, M. Erhard, A. Zeilinger, and M. Krenn,
Computer-Inspired Concept for High-Dimensional Mul-
tipartite Quantum Gates, Physical Review Letters 125,
050501 (2020).
[105] A. A. Melnikov, H. P. Nautrup, M. Krenn, V. Dunjko,
M. Tiersch, A. Zeilinger, and H. J. Briegel, Active learn-
ing machine learns to create new quantum experiments,
Proceedings of the National Academy of Sciences 115,
1221 (2018).
[106] A. Cervera-Lierta, M. Krenn, and A. Aspuru-Guzik, De-
sign of quantum optical experiments with logic artificial
intelligence, arXiv:2109.13273 (2021).
[107] M. J. H. Heule and O. Kullmann, The science of brute
force, Communications of the ACM 60, 70 (2017).
[108] B. Sanchez-Lengeling and A. Aspuru-Guzik, Inverse
molecular design using machine learning: Generative
models for matter engineering, Science 361, 360 (2018).
[109] D. Flam-Shepherd, T. C. Wu, X. Gu, A. Cervera-Lierta,
M. Krenn, and A. Aspuru-Guzik, Learning interpretable
representations of entanglement in quantum optics ex-
periments using deep generative models, Nature Ma-
chine Intelligence 4, 544 (2022).
[110] T. Menke, F. Häse, S. Gustavsson, A. J. Kerman, W. D.
Oliver, and A. Aspuru-Guzik, Automated design of su-
perconducting circuits and its application to 4-local cou-
plers, npj Quantum Information 7, 49 (2021).
[111] J. Wallnöfer, A. A. Melnikov, W. Dür, and H. J. Briegel,
Machine Learning for Long-Distance Quantum Commu-
nication, PRX Quantum 1, 010301 (2020).
[112] C. P. Williams and A. G. Gray, Automated Design
of Quantum Circuits, in Quantum Computing and
Quantum Communications, edited by C. P. Williams
(Springer Berlin, Heidelberg, 1999) pp. 113–125.
[113] R. B. McDonald and H. G. Katzgraber, Genetic braid
optimization: A heuristic approach to compute quasi-
particle braids, Physical Review B 87, 054414 (2013).

[114] U. Las Heras, U. Alvarez-Rodriguez, E. Solano, and
M. Sanz, Genetic Algorithms for Digital Quantum Sim-
ulations, Physical Review Letters 116, 230504 (2016).
[115] Y.-H. Zhang, P.-L. Zheng, Y. Zhang, and D.-L. Deng,
Topological Quantum Compiling with Reinforcement
Learning, Physical Review Letters 125, 170501 (2020).
[116] L. Moro, M. G. A. Paris, M. Restelli, and E. Prati,
Quantum compiling by deep reinforcement learning,
Communications Physics 4, 178 (2021).
[117] L. Cincio, Y. Subaşı, A. T. Sornborger, and P. J. Coles,
Learning the quantum algorithm for state overlap, New
Journal of Physics 20, 113022 (2018).
[118] L. Cincio, K. Rudinger, M. Sarovar, and P. J. Coles,
Machine Learning of Noise-Resilient Quantum Circuits,
PRX Quantum 2, 010324 (2021).
[119] J. Yao, P. Kottering, H. Gundlach, L. Lin, and
M. Bukov, Noise-Robust End-to-End Quantum Control
using Deep Autoregressive Policy Networks, in Proceed-
ings of the 2nd Mathematical and Scientific Machine
Learning Conference, Proceedings of Machine Learning
Research, Vol. 145, edited by J. Bruna, J. Hesthaven,
and L. Zdeborova (PMLR, 2022) pp. 1044–1081.
[120] J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Bab-
bush, and H. Neven, Barren plateaus in quantum neural
network training landscapes, Nature Communications
9, 4812 (2018).
[121] S. Sim, P. D. Johnson, and A. Aspuru-Guzik, Ex-
pressibility and Entangling Capability of Parameterized
Quantum Circuits for Hybrid Quantum-Classical Al-
gorithms, Advanced Quantum Technologies 2, 1900070
(2019).
[122] M. Ostaszewski, L. M. Trenkwalder, W. Masarczyk,
E. Scerri, and V. Dunjko, Reinforcement learning for op-
timization of variational quantum circuit architectures,
Advances in Neural Information Processing Systems,
34, 18182 (2021).
[123] S.-X. Zhang, C.-Y. Hsieh, S. Zhang, and H. Yao, Neural
predictor based quantum architecture search, Machine
Learning: Science and Technology 2, 045027 (2021).
[124] J. Yao, L. Lin, and M. Bukov, Reinforcement Learning
for Many-Body Ground-State Preparation Inspired by
Counterdiabatic Driving, Physical Review X 11, 031070
(2021).
[125] G. Verdon, M. Broughton, J. R. McClean, K. J. Sung,
R. Babbush, Z. Jiang, H. Neven, and M. Mohseni,
Learning to learn with quantum neural networks via
classical neural networks, arXiv:1907.05415 (2019).
[126] A. Anand, M. Degroote, and A. Aspuru-Guzik, Natural
evolutionary strategies for variational quantum compu-
tation, Machine Learning: Science and Technology 2,
045012 (2021).
[127] T. H. Kyaw, T. Menke, S. Sim, A. Anand, N. P. Sawaya,
W. D. Oliver, G. G. Guerreschi, and A. Aspuru-Guzik,
Quantum Computer-Aided Design: Digital Quantum
Simulation of Quantum Processors, Physical Review
Applied 16, 044042 (2021).
[128] J. S. Kottmann, M. Krenn, T. H. Kyaw, S. Alperin-
Lea, and A. Aspuru-Guzik, Quantum computer-aided
design of quantum optics hardware, Quantum Science
and Technology 6, 035010 (2021).
[129] F.-M. Liu, M.-C. Chen, C. Wang, S.-W. Li, Z.-X.
Shang, C. Ying, J.-W. Wang, C.-Z. Peng, X. Zhu, C.-
Y. Lu, et al., Quantum Design for Advanced Qubits,
arXiv:2109.00994 (2021).
22
[130] R. Duncan, A. Kissinger, S. Perdrix, and J. van de We-
tering, Graph-theoretic Simplification of Quantum Cir-
cuits with the ZX-calculus, Quantum 4, 279 (2020).
[131] T. Fösel, M. Y. Niu, F. Marquardt, and L. Li, Quantum
circuit optimization with deep reinforcement learning,
arXiv:2103.07585 (2021).
[132] P. W. Shor, Scheme for reducing decoherence in quan-
tum computer memory, Physical Review A 52, R2493
(1995).
[133] G. Torlai and R. G. Melko, Neural Decoder for Topolog-
ical Codes, Physical Review Letters 119, 030501 (2017).
[134] S. Krastanov and L. Jiang, Deep Neural Network Prob-
abilistic Decoder for Stabilizer Codes, Scientific reports
7, 11003 (2017).
[135] S. Varsamopoulos, B. Criger, and K. Bertels, Decoding
small surface codes with feedforward neural networks,
Quantum Science and Technology 3, 015004 (2017).
[136] P. Baireuther, T. E. O’Brien, B. Tarasinski, and
C. W. J. Beenakker, Machine-learning-assisted correc-
tion of correlated qubit errors in a topological code,
Quantum 2, 48 (2018).
[137] R. Sweke, M. S. Kesselring, E. P. L. van Nieuwenburg,
and J. Eisert, Reinforcement learning decoders for fault-
tolerant quantum computation, Machine Learning: Sci-
ence and Technology 2, 025005 (2021).
[138] H. P. Nautrup, N. Delfosse, V. Dunjko, H. J. Briegel,
and N. Friis, Optimizing Quantum Error Correction
Codes with Reinforcement Learning, Quantum 3, 215
(2019).
[139] Z. Wang, T. Rajabzadeh, N. Lee, and A. H. Safavi-
Naeini, Automated Discovery of Autonomous Quantum
Error Correction Schemes, PRX Quantum 3, 020302
(2022).
[140] V. Tshitoyan, J. Dagdelen, L. Weston, A. Dunn,
Z. Rong, O. Kononova, K. A. Persson, G. Ceder, and
A. Jain, Unsupervised word embeddings capture la-
tent knowledge from materials science literature, Nature
571, 95 (2019).
[141] S. d’Ascoli, P.-A. Kamienny, G. Lample, and F. Char-
ton, Deep Symbolic Regression for Recurrent Sequences,
arXiv:2201.04600 (2022).
[142] P. Veličković, A. P. Badia, D. Budden, R. Pascanu,
A. Banino, M. Dashevskiy, R. Hadsell, and C. Blun-
dell, The CLRS Algorithmic Reasoning Benchmark,
arXiv:2205.15659 (2022).
[143] M. Krenn, R. Pollice, S. Y. Guo, M. Aldeghi,
A. Cervera-Lierta, P. Friederich, G. d. P. Gomes,
F. Häse, A. Jinich, A. Nigam, et al., On scientific under-
standing with artificial intelligence, arXiv:2204.01467
(2022).
[144] H. W. de Regt, Understanding Scientific Understanding
(Oxford University Press, 2017).
[145] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and
L. Fei-Fei, Imagenet: A large-scale hierarchical image
database, in 2009 IEEE Conference on Computer Vi-
sion and Pattern Recognition (2009) pp. 248–255.
[146] N. Brown, M. Fiscato, M. H. S. Segler, and A. C.
Vaucher, GuacaMol: Benchmarking Models for de Novo
Molecular Design, Journal of Chemical Information and
Modeling 59, 1096 (2019).
[147] D. Polykovskiy, A. Zhebrak, B. Sanchez-Lengeling,
S. Golovanov, O. Tatanov, S. Belyaev, R. Kurbanov,
A. Artamonov, V. Aladinskiy, M. Veselov, et al., Molec-
ular sets (MOSES): a benchmarking platform for molec-
 23

ular generation models, Frontiers in Pharmacology 11, [149] A. Turing, Computing Machinery and Intelligence,
565644 (2020). Mind LIX, 433 (1950).
[148] Reinforcement learning for science, https://www.
scigym.net/, accessed: 2022-08-08.