Available online at www.sciencedirect.
com
ScienceDirect
Computation noise in human learning and
decision-making: origin, impact, function
Charles Findling1,2 and Valentin Wyart1,2
Making sense of uncertain and volatile environments, a
cognitive process modeled across domains as statistical
inference, constitutes a difficult yet ubiquitous challenge for
human intelligence. Beside sensory errors and exploratory
choices, recent research has identified the limited
computational precision of cognitive inference as a surprisingly
large contributor to the variability and suboptimality of
perceptual and reward-guided decisions made under
uncertainty. This focused review discusses the theoretical and
experimental evidence scattered across psychology and
neuroscience which, taken together, provides key insights into
the origin, impact and function of this ‘computation noise’ for
learning and decision-making. Moving beyond the classical
description of internal noise as performance-limiting constraint
on neural function and cognition, we outline the possible
emergent benefits of computation noise for adaptive behavior
in adverse conditions and highlight open questions for future
research.
Addresses
1
Laboratoire de Neurosciences Cognitives et Computationnelles, Insti-
tut National de la Santé et de la Recherche Médicale, Paris, France
2
Département d’Études Cognitives, École Normale Supérieure,
Université PSL, Paris, France
Corresponding authors: Findling, Charles (charles.findling@gmail.com),
Wyart, Valentin (valentin.wyart@ens.fr)
Current Opinion in Behavioral Sciences 2021, 38:124–132
This review comes from a themed issue on Computational cognitive
neuroscience
Edited by Angela J Langdon and Geoffrey Schoenbaum
For a complete overview see the Issue and the Editorial
Available online 12th March 2021
https://doi.org/10.1016/j.cobeha.2021.02.018
2352-1546/ã 2021 The Author(s). Published by Elsevier Ltd. This is an
open access article under the CC BY-NC-ND license (http://creative-
commons.org/licenses/by-nc-nd/4.0/).
Prominent variability of human decisions
made under uncertainty
Humans routinely navigate uncertain and volatile envir-
onments in everyday life, from a changing weather to
unexpected incidents on our regular metro line. In such
conditions, adaptive behavior requires making dynamic
probabilistic inferences about external events (e.g. mak-
ing accurate predictions about the weather) and action-
outcome contingencies (e.g. choosing a detour which
minimizes additional delays on our way to work).
Current Opinion in Behavioral Sciences 2021, 38:124–132
In the laboratory, these dynamic inferences are investi-
gated using controlled paradigms generating different
forms of uncertainty (Figure 1a). The ‘weather pre-
diction’ task [1–3] probes the ability to combine multiple
sources of uncertain information about the location of a
hidden reward. After learning probabilistic associations
between a set of symbols — presented one at a time —
and the location of the reward, subjects are asked to
predict the location of the reward based on sequences
of symbols. Each symbol taken in isolation predicts the
same reward location as before, but this uncertain infor-
mation has to be combined across symbols to predict
accurately the location of the reward. In this task, even
after reaching ceiling performance at predicting the loca-
tion of the reward based on symbols presented one at a
time, humans (and non-human primates) show a large
trial-to-trial variability in their predictions based on
sequences of symbols [3,4]. In other words, subjects
sometimes choose a reward location which is not associ-
ated with the highest posterior probability of reward given
the presented sequence of symbols.
Another widely used paradigm, the ‘reversal learning’
task [5–7], probes the ability to monitor changes in the
reward probabilities associated with choice options (e.g.
the two arms of a bandit, Figure 1b). Reward probabilities
being uncertain, subjects need to distinguish ‘false
alarms’ — missing rewards when choosing the option
associated with the highest reward probability — from
genuine changes in reward probabilities. Humans
engaged in this task (or one of its variants) make a
substantial fraction of ‘non-greedy’ choices — choices
which do not maximize expected reward but reduce the
uncertainty about recently unchosen options [5,8].
Therefore, and despite the several differences between
these different experimental paradigms, human decisions
made in uncertain and volatile environments exhibit a
pervasive variability which limits their accuracy.
Large contribution of inference errors to
human decision variability
The origin of decision variability under uncertainty has
usually been assigned to a single source, located either at
the input or output of the probabilistic inference process
used to update an internal model of the environment –
the location of the hidden reward in the weather predic-
tion task, or the reward probabilities associated with each
choice option in the reversal learning task. However,
these accounts fail to explain empirical observations, such
as why decision variability grows linearly with the number
www.sciencedirect.com
 Computation noise in human cognition Findling and Wyart 125

Figure 1

(a)

(b)

(c)

Current Opinion in Behavioral Sciences

(a) Description of experimental paradigms studying dynamic inferences under uncertainty. Left: visual variant of the weather prediction task. Each
trial consists of a sequence of oriented patterns, drawn from one of two generative probability distributions (sources A and B). At sequence offset,
subjects are prompted to indicate the source from which they believed the oriented patterns were drawn. Right: restless variant of the reversal
learning task. On each trial, subjects are asked to choose between two colored symbols (options), and then obtain its associated reward. The
mean rewards associated with the two options drift continuously and randomly over time (thick lines represent the drifting mean rewards
associated with the two options, whereas thin lines represent rewards sampled the drifting means). (b) Contributions of distinct sources of errors
to human behavioral variability in the weather prediction task (left) and the reversal learning task (right). Left: inference errors explain about 90% of
the observed behavioral variability in the weather prediction task. Right: inference errors in reinforcement learning explain more than 60% of
seemingly non-greedy decisions when the reward associated with the foregone option is not observed (partial outcome condition, left), and more
than 85% when the foregone reward is observed and there is thus equal uncertainty about chosen and unchosen options (complete outcome
condition, right). (c) Decomposition of human inference errors in terms of a computation bias-variance trade-off in the weather prediction task (left)
and the reversal learning task (right). The bias term (left) corresponds to predictable errors across repetitions of the same trial, whereas the
variance term (right) corresponds to unpredictable errors across trial repetitions – that is, computation noise. In the weather prediction task, the
bias term is split into temporal biases (green), perceptual biases (blue), trial history biases (brown), and other unspecified biases (gray).
Computation noise explains about two thirds of human inference errors in the two tasks.

of presented symbols in the weather prediction task – at a
rate which greatly exceeds sensory variability [4]. Or
why subjects make non-greedy decisions even in condi-
tions where they observe after each choice the foregone
reward associated with the unchosen option, and there is
thus equal uncertainty about chosen and unchosen
options [8,9,10].
These different effects can be readily explained by the
presence of significant inference errors – that is, errors in
the updating of the internal model of the environment
(Figure 1b). In the weather prediction task [4], these
cognitive errors accumulate across successive inference
steps and explain why decision variability grows with the
number of stimuli in the sequence. Strikingly, inference
errors account for more than 85% of the observed decision
variability in this task. This result means that subjects
almost never choose a reward location which is not
associated with the highest perceived probability of
reward. Rather, they make substantial cognitive errors
inferring the most probable reward location based of the
presented sequence of stimuli. In a ‘random walk’ variant
of the reversal learning task where non-greedy decisions
reduce the uncertainty about recently unchosen options,
inference errors still make up more than 60% of these
decisions [8]. This observation means that a substantial

www.sciencedirect.com Current Opinion in Behavioral Sciences 2021, 38:124–132

126 Computational cognitive neuroscience
fraction of these adaptive decisions is not driven by an
explicit arbitration between exploration and exploitation,
but by cognitive errors when inferring the reward proba-
bilities associated with the different choice options. In
other words, many of the non-greedy decisions labeled as
‘exploratory’ (uncertainty-minimizing) when assuming
noise-free inference reflect ‘exploitative’ (reward-maxi-
mizing) decisions based on misestimated reward proba-
bilities caused by inference noise. Accounting for infer-
ence errors during reward-guided learning also explains
the non-greedy decisions observed when foregone
rewards are presented together with obtained rewards –
that is, conditions where these decisions have no adaptive
value in terms of reward maximization.
Computation bias-variance structure of
human inference errors
The surprisingly large contribution of inference errors to
human decision variability requires qualifying their
nature. The statistical signatures of inference errors differ
from those of stochastic choice policies (e.g. softmax,
Thompson sampling) in the sense that inference errors
committed at one point in time corrupt (in a normative
sense) the internal model of the environment and thus
propagate forward – within a sequence of stimuli in the
weather prediction task [4], or across successive trials in
sequential reinforcement paradigms such as the reversal
learning task [8]. In contrast to stochastic choice policies
[11], inference errors do not reflect a sampling-based
‘read-out’ of the internal model of the environment,
but fluctuations of the internal model itself.
A key question concerns the statistical structure of
inference errors — in particular, whether they produce
random (unpredictable) or systematic (predictable) var-
iability in behavior — a decomposition known as the
‘bias-variance’ trade-off. Inference errors being defined
as deviations from optimal (or near-optimal) inference
[4,8], they may be caused by different cognitive
biases known to affect probabilistic inference under
uncertainty. For example, evidence integration shows
idiosyncratic temporal biases (e.g. primacy or recency)
that can produce large inference errors [12–15]. Simi-
larly, confirmation biases and other forms of ‘trial history’
effects have been reported across decision domains [16–
19]. A fundamental difference between such cognitive
biases and computation ‘noise’ — that is, stochastic
variability in the updating of the internal model of the
environment — is that biases should produce correlated
inference errors across repetitions of the same trial
[4,8,20]. Instead of performing an exhaustive search
of all possible cognitive biases [21], leveraging the
consistency of human decisions across repeated trials
[22] revealed that random variance accounts for 65% of
inference errors across perceptual and reward-guided
decisions [4,8] (Figure 1c). In other words, about
two thirds of inference errors arise from the limited
Current Opinion in Behavioral Sciences 2021, 38:124–132
precision of underlying computations rather than from
biased (systematically wrong) inference [23].
In volatile environments where the state of the environ-
ment changes over time, computation noise shows a
scaling variance which matches the Weber’s law of per-
ceptual discrimination prevalent in numerous sensory
modalities. Across different variants of the reversal learn-
ing task [8,24], computation noise grows with the
prediction error between observed and expected reward
– the ‘temporal difference’ (surprise) signal which drives
model updating in reinforcement learning (RL) [25]. This
Weber scaling structure of computation noise is not
observed in stable environments where the state of the
environment is uncertain but fixed, as in the weather
prediction task [4]. These differences may be due to the
distinct roles of surprise for learning in these two types of
uncertain environments. Inferring the state of volatile
environments constitutes an estimation task where sur-
prise indicates a possible change in the current state of the
environment – and is thus the primary signal for updating
the internal model of the environment [26–28]. By con-
trast, inferring the state of stable environments relies on
the integration of uncertain information over much longer
(ideally infinite) time constants [29]. The relevance of
surprise for inferring the state of a stable environment
decays over time as the amount of integrated information
grows, and optimal inference does not require computing
surprise to update the internal model of the environment.
Future research should examine further the relation
between surprise, model updating and computation
noise.
Despite these fine-grained differences, computation
noise accounts for a dominant fraction of inference errors
across perceptual and reward-guided decisions – two
canonical types of decisions studied and theorized using
different paradigms and cognitive models [4,8]. This
pattern of findings sets the limited precision of probabi-
listic inference as an upper bound on the accuracy and
trial-to-trial consistency of human decisions made in
uncertain environments.
Constraints of computation noise on learning
and decision-making
Computation noise, defined above as random variability
in the updating of the internal model of the environment
(the location of the hidden reward in the weather predic-
tion task, or the reward probabilities associated with each
choice option in the reversal learning task), can have very
different substrates in the brain. Computation noise can
reflect genuine neural noise such as stochastic synaptic
release at the cellular level [30], random fluctuations in
the tight excitation-inhibition balance required by neural
populations to perform precise computations [31,32], or
the variable pooling of task-relevant neural responses by
top-down attention across cortical hierarchies [33]. But
www.sciencedirect.com

computation noise can also arise from task-independent
(background) input to neural circuits implementing prob-
abilistic inference [34,35,36] – that is, ‘effective’ noise
which may not be random in an absolute sense, but
generates trial-to-trial variability in the updating of the
internal model. And irrespective of its precise substrates,
this type of internal noise is widely seen as a performance-
limiting constraint which neural systems have evolved to
cope with using ‘efficient’ mechanisms.
At the neural level, correlated noise across neurons
(which does not average out by pooling neural
responses) has been theorized [37] and recently shown
[38,39] to constrain the precision of neural representa-
tions. Because this correlated noise aligns strikingly
well with the neural ‘dimensions’ (in population space)
which predict decisions on a trial-by-trial basis [40],
its impact on task performance is expected to be sub-
stantial, and consistent with the large contribution of
computation noise to decision variability. In agreement
with this view, correlated noise has been shown to be
the prime target of top-down attentional modulation
[41–43], by decreasing pairwise correlations between
sensory neurons tuned to attended locations or features,
and increasing the gain of neural representations at the
population level [44]. At the theoretical level, attention
has been described as an evolved mechanism for
approximate inference which can increase the precision
of task-relevant computations in the presence of lim-
ited processing resources [45].
Other neural constraints which may give rise to compu-
tation noise include the multiplexing of several cognitive
tasks by context-dependent computations in shared neu-
ral circuits in parietal and prefrontal cortices [36,46].
Concurrent, irrelevant input that has not been fully
suppressed by context-dependent computations would
produce task-independent variability with the same sta-
tistical signatures as the computation noise observed in
the weather prediction task [4].
In terms of cognition, different lines of research have
identified ‘efficient coding’ mechanisms for dealing with
external and internal sources of noise [47]. Human per-
ceptual biases running opposite to prior expectations (in
apparent contradiction with statistical inference) can be
explained by efficient coding principles [48]. Similarly,
the variability and occasional irrationality of human pref-
erence-based decisions may arise from strikingly similar
mechanisms applied to value signals [49]. The notion of
limited processing resources, which lies at the heart of
efficient coding theories, has recently been instantiated as
a finite number of ‘particles’ in a sampling-based descrip-
tion of statistical inference [50,51–53]. As shown by
these two examples, certain cognitive biases may be
the consequence of computation noise. Sampling-based
computations in a hierarchical inference circuit can
www.sciencedirect.com
Computation noise in human cognition Findling and Wyart 127
produce either primacy or recency effects depending
on the main source of uncertainty [15]. And more com-
plex, ‘winner-take-all’ biases in information integration
may have emerged to mitigate the impact of computation
noise on performance [54].
Together, these recent behavioral findings blur the line
between bias and variance in the statistical sense [23]:
certain cognitive biases may correspond to direct mecha-
nistic consequences of computation noise, while others
may guard against the variability and suboptimality trig-
gered by computation noise. In either case, this deep
interplay between cognitive bias and variance emphasizes
the importance of characterizing computation noise for
understanding even seemingly unrelated aspects of
human cognition [20,22,23].
Regulation of computation noise by
noradrenergic neuromodulation
The large contribution of computation noise to human
learning and decision-making suggests that neural sys-
tems should be involved in its active regulation [4,8].
Previous research which does not consider explicitly
computation noise has identified the anterior cingulate
cortex (ACC) as a selective neural correlate of behavioral
variability in uncertain and volatile environments which
require arbitrating between exploration and reward max-
imization [55,56]. Accounting for the presence of compu-
tation noise in reinforcement learning has revealed that
trial-to-trial fluctuations in ACC activity covary with
variability in the updating of choice values based on
obtained rewards, even when it does not result in a
behavioral switch (i.e. overt exploration) on the subse-
quent trial (Figure 2a) [8]. This finding indicates that
the phasic ACC activity observed preceding exploratory
choices may reflect a genuine resetting of the internal
model of the environment rather than a temporary
‘release’ of the reliance on the internal model for guiding
behavior.
Beside the ACC, the magnitude of computation noise in
the updating of choice values correlates also robustly with
trial-to-trial fluctuations in pupil dilation [8], a non-
invasive proxy of locus coeruleus-norepinephrine (LC-
NE) activity [57,58,59]. Noradrenergic neuromodulation
has been linked to the regulation of arousal, but also
strategic exploration through the adaptive gating of neural
variability in the ACC and other decision circuits in the
prefrontal cortex [60]. In volatile environments, pupil
dilation reflects not only the updating of the internal
model following surprising events [61], but also the
variability of the resulting behavior [62]. Accounting
for the presence of computation noise in similar condi-
tions showed that pupil-linked variability in behavior
arises from computation noise rather than fluctuations
in the exploration-exploitation trade-off (Figure 2b) – an
Current Opinion in Behavioral Sciences 2021, 38:124–132
128 Computational cognitive neuroscience

Figure 2

(a) anterior cingulate cortex (ACC) (c) reanalysis of Jepma et al., 2010
contributions to behavioral variability
parameter estimate (a.u.)

parameter estimate (a.u.)

(b) phasic pupil dilation (d) computation noise estimates
parameter estimate (a.u.)

parameter estimate (a.u.)

posterior density

Current Opinion in Behavioral Sciences

(a) Regression of trial-to-trial fluctuations of computation noise with BOLD activity in the anterior cingulate cortex (ACC) locked to the onset of the
choice period. Left: the correlations of ACC activity with computation noise (blue) and prediction error triggered by the previous reward (orange)
emerge before choice onset and follow similar time courses. Right: fluctuations of ACC activity predict trial-to-trial changes in behavioral variability
to the same extent in the partial outcome condition (left bar) and the complete outcome condition (right bar) where exploration has no adaptive
value in terms of reward maximization. (b) Regression of trial-to-trial fluctuations of computation noise with phasic pupil dilation locked to the
onset of the choice period. Left: the correlation of pupil dilation with learning noise (blue) emerges before choice onset and precedes the
correlation of pupil dilation with choice value (purple). Right: like ACC activity, fluctuations of pupil dilation predict trial-to-trial changes in
behavioral variability to the same extent in the partial and complete outcome conditions. (c) Reanalysis of behavioral data from a human
pharmacological study of the effect of reboxetine (noradrenaline reuptake inhibitor) on reinforcement learning in a restless four-armed bandit task.
Contribution of inference errors in reinforcement learning to behavioral variability for the placebo group (left) and the reboxetine group (right).
Computation noise in reinforcement learning explains a larger fraction of behavioral variability in the reboxetine group (80%) than the placebo
group (53%). (d) Posterior distributions of computation noise estimates for the placebo group (gray) and reboxetine group (dark blue). Reboxetine
increases the magnitude of computation noise during reinforcement learning.

effect observed even when there is equal uncertainty
about chosen and unchosen options [8]. Recent simul-
taneous recordings from the LC and the ACC have
further shown that phasic LC activation increases pair-
wise correlations between ACC neurons [63], in a way
that resembles the attentional modulation of correlated
noise in visual cortex [41–44].
The idea that noradrenergic neuromodulation may reg-
ulate computation noise rather than the exploration-
exploitation trade-off is also supported by causal phar-
macological manipulations of the LC-NE system in
humans. Small doses of atomoxetine, a noradrenaline
reuptake inhibitor, increase the rate of endogenous
perceptual alternations of bistable stimuli [64], in a task
which does not involve any form of exploration. Fur-
thermore, small doses of reboxetine, another noradrena-
line reuptake inhibitor, do not produce exploratory
behavior [65] but seem to increase the magnitude of
computation noise in a preliminary reanalysis of this
dataset using a reinforcement learning model which
accounts for the presence of computation noise
(Figure 2c,d) [8,66]. Future research should validate
this preliminary finding and investigate further the pos-
sible involvement of the noradrenergic system in the
active regulation of computation noise.

Current Opinion in Behavioral Sciences 2021, 38:124–132 www.sciencedirect.com


Emergent benefits of computation noise for
learning and decision-making
The research discussed above describes computation
noise as an important constraint on neural function and
cognition, whose impact on performance may be con-
trolled by a specific neuromodulatory pathway. A remain-
ing puzzle concerns the reasons of its very existence. If
computation noise drives such a large fraction of human
behavioral variability and suboptimality in uncertain
environments, why hasn’t it been suppressed or reduced
to a larger extent during evolution?
A first possible answer is that computation noise may
optimize a trade-off between the marginal payoff of a
computation and the cost associated with performing the
computation at a certain precision [8]. The ACC has
recently been hypothesized to arbitrate such a cost-ben-
efit trade-off by computing an ‘expected value of control’
– defined as the difference between an expected payoff
and its associated cost in terms of cognitive conflict [67].
The cost associated with a computation is likely to grow
with its precision, such that the ACC may optimize this
computation cost-benefit trade-off by regulating compu-
tation noise [8] (possibly, through bidirectional connec-
tivity with the LC-NE system [58]). This proposal
provides a natural explanation for the existence of com-
putation noise, but also makes testable predictions. In
particular, computation noise should increase in highly
volatile conditions where the marginal payoff of main-
taining a precise internal model of the environment
decreases – a prediction which has recently been vali-
dated experimentally [24]. The Weber scaling structure
of computation noise also drives transient increases in
behavioral variability following surprising events by reset-
ting the current state of the internal model [8,24].
Although this cost-benefit description of computation
noise in volatile environments has its merits, it fails to
explain why computation noise is similarly large in uncer-
tain but fixed environments where behavioral variability
is neither required nor useful, as in the weather prediction
task [4]. This pervasiveness of computation noise sug-
gests that it has broader emergent benefits for cognition.
Recent research has provided insights regarding what
these emergent benefits may be. First, two neurophysio-
logical correlates of computation noise (pupil-linked
arousal and large-scale neural variability) have been pro-
posed to counteract idiosyncratic biases during perceptual
decisions [68–70,71]. This means that increases in com-
putation noise may be accompanied by less biased per-
ception, a cognitive trade-off which has also been
reported in the reversal learning task [72]. Another possi-
ble benefit of computation noise is that it produces less
predictable behavior – a key advantage in competitive
social contexts. Behavioral experiments in rodents have
shown that the animals can purposefully increase their
www.sciencedirect.com
Computation noise in human cognition Findling and Wyart 129
behavioral variability when confronted with a computer-
simulated competitor that is capable of predicting their
upcoming choices [73,74].
Finally, and more generally, computation noise may
produce beneficial stochasticity for the attractor-like
dynamics of neural circuits implementing probabilistic
inference [34,35,36,46]. Their dynamics are typically
low-dimensional and converge on strong attractor states,
such that computation noise could serve to destabilize
these attractor states and allow for more flexible transi-
tions between them. Such flexibility is particularly wel-
come in uncertain and volatile environments where the
state of the environment can change unpredictably, and
the internal model (reflected in the current attractor state
of the circuit) needs to be updated following each change.
This flexibility is typically implemented in cognitive
models through explicit sophistication (e.g. hierarchical
inference) [6,75,76]. Whether computation noise may
provide such cognitive flexibility at zero cost therefore
constitutes another important open question for future
research.
Conflict of interest statement
Nothing declared.
Acknowledgements
We thank Marieke Jepma and Sander Nieuwenhuis for sharing the
behavioral data of their pharmacological study, and Vasilisa Skvortsova for
useful discussions. This work was supported by a starting grant from the
European Research Council (OPTIMIZERR, ERC-StG-759341) awarded
to V.W., and an institutional grant from the Agence Nationale de la
Recherche (FrontCog, ANR-17-EURE-0017) awarded to the Département
d’Études Cognitives of the École Normale Supérieure.
References and recommended reading
Papers of particular interest, published within the period of review,
have been highlighted as:
 of special interest
 of outstanding interest
1. Knowlton BJ, Mangels JA, Squire LR: A neostriatal habit learning
system in humans. Science 1996, 273:1399-1402.
2. Poldrack RA, Clark J, Paré-Blagoev EJ, Shohamy D, Creso
Moyano J, Myers C, Gluck MA: Interactive memory systems in
the human brain. Nature 2001, 414:546-550.
3. Yang T, Shadlen MN: Probabilistic reasoning by neurons.
Nature 2007, 447:1075-1080.
4. Drugowitsch J, Wyart V, Devauchelle A-D, Koechlin E:
 Computational precision of mental inference as critical source
of human choice suboptimality. Neuron 2016, 92:1398-1411
This study investigates the distinct sources of human decision variability
and suboptimality in a visual variant of the weather prediction task. They
introduce a cognitive modeling framework which decomposes decision
variability into sensory, inference and selection errors based on their
respective statistical signatures. By measuring the bias-variance trade-
off of inference errors, they find that computation noise in inference
accounts alone for more than two thirds of human decision variability.
5. Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ: Cortical
substrates for exploratory decisions in humans. Nature 2006,
441:876-879.
Current Opinion in Behavioral Sciences 2021, 38:124–132
130 Computational cognitive neuroscience
6. Behrens TEJ, Woolrich MW, Walton ME, Rushworth MFS:
Learning the value of information in an uncertain world. Nat
Neurosci 2007, 10:1214-1221.
7. Izquierdo A, Brigman JL, Radke AK, Rudebeck PH, Holmes A: The
neural basis of reversal learning: an updated perspective.
Neuroscience 2017, 345:12-26.
8. Findling C, Skvortsova V, Dromnelle R, Palminteri S, Wyart V:
 Computational noise in reward-guided learning drives
behavioral variability in volatile environments. Nat Neurosci
2019, 22:2066-2077
This study investigates the contribution of computation noise in reinfor-
cement learning to human decision variability in a random-walk variant of
the reversal learning task. They find that computation noise drives a
dominant fraction of non-greedy decisions otherwise assigned to the
exploration-exploitation trade-off, and identify its neurophysiological
correlates in the anterior cingulate cortex and phasic pupil dilation during
reinforcement learning rather than choice.
9. Boorman ED, Behrens TEJ, Woolrich MW, Rushworth MFS: How
green is the grass on the other side? Frontopolar cortex and
the evidence in favor of alternative courses of action. Neuron
2009, 62:733-743.
10. Palminteri S, Khamassi M, Joffily M, Coricelli G: Contextual
modulation of value signals in reward and punishment
learning. Nat Commun 2015, 6:8096.
11. Gershman SJ: Deconstructing the human algorithms for
exploration. Cognition 2018, 173:34-42.
12. Kiani R, Hanks TD, Shadlen MN: Bounded integration in parietal
cortex underlies decisions even when viewing duration is
dictated by the environment. J Neurosci 2008, 28:3017-3029.
13. Nienborg H, Cumming BG: Decision-related activity in sensory
neurons reflects more than a neuron’s causal effect. Nature
2009, 459:89-92.
14. Wyart V, Myers NE, Summerfield C: Neural mechanisms of
human perceptual choice under focused and divided
attention. J Neurosci 2015, 35:3485-3498.
15. Lange RD, Chattoraj A, Beck JM, Yates JL, Haefner RM: A
confirmation bias in perceptual decision-making due to
hierarchical approximate inference. bioRxiv 2020 http://dx.doi.
org/10.1101/440321.
16. Talluri BC, Urai AE, Tsetsos K, Usher M, Donner TH: Confirmation
bias through selective overweighting of choice-consistent
evidence. Curr Biol 2018, 28:3128-3135.
17. Akrami A, Kopec CD, Diamond ME, Brody CD: Posterior parietal
cortex represents sensory history and mediates its effects on
behaviour. Nature 2018, 554:368-372.
18. Lak A, Okun M, Moss MM, Gurnani H, Farrell K, Wells MJ,
Reddy CB, Kepecs A, Harris KD, Carandini M: Dopaminergic and
prefrontal basis of learning from snsory confidence and
reward value. Neuron 2020, 105:700-711.e6.
19. Lak A, Hueske E, Hirokawa J, Masset P, Ott T, Urai AE, Donner TH,
Carandini M, Tonegawa S, Uchida N et al.: Reinforcement biases
subsequent perceptual decisions when confidence is low, a
widespread behavioral phenomenon. eLife 2020:9.
20. Wyart V, Koechlin E: Choice variability and suboptimality in
uncertain environments. Curr Opin Behav Sci 2016, 11:109-115.
21. Rahnev D, Denison RN: Suboptimality in perceptual decision
making. Behav Brain Sci 2018, 41:e223.
22. Wyart V: Leveraging decision consistency to decompose
suboptimality in terms of its ultimate predictability. Behav Brain
Sci 2018, 41:e248.
23. Beck JM, Ma WJ, Pitkow X, Latham PE, Pouget A: Not noisy, just
 wrong: the role of suboptimal inference in behavioral
variability. Neuron 2012, 74:30-39
This review discusses the contribution of inference biases to decision
variability based on uncertain sensory evidence. They propose that
decision variability arises dominantly from biased computations which
can amplify sensory noise, rather than from neural noise– expressed at
the level of individual neurons. They ground their hypothesis on theore-
tical considerations: 1. that neural noise should average out at the
Current Opinion in Behavioral Sciences 2021, 38:124–132
population level, and 2. that sensory variability dominates over motor
variability in most perceptual decision-making tasks.
24. Findling C, Chopin N, Koechlin E: Imprecise neural
 computations as source of human adaptive behavior in
volatile environments. Nat Hum Behav 2021, 5:99-112
This study investigates the contribution of computation noise to human
adaptive behavior in volatile environments. They show that computation
noise during low-level, single-stage inference leads to the same near-
optimal performance as normative hierarchical inference in volatile envir-
onments. They also find that noisy single-stage inference accounts for
human behavioral variability better than noise-free hierarchical inference.
25. Sutton RS, Barto AG: Reinforcement Learning: An Introduction.
MIT Press; 1998.
26. Rushworth MFS, Noonan MP, Boorman ED, Walton ME,
Behrens TE: Frontal cortex and reward-guided learning and
decision-making. Neuron 2011, 70:1054-1069.
27. O’Reilly JX, Schüffelgen U, Cuell SF, Behrens TEJ, Mars RB,
Rushworth MFS: Dissociable effects of surprise and model
update in parietal and anterior cingulate cortex. Proc Natl Acad
Sci U S A 2013, 110:E3660-3669.
28. Farashahi S, Donahue CH, Khorsand P, Seo H, Lee D, Soltani A:
Metaplasticity as a neural substrate for adaptive learning and
choice under uncertainty. Neuron 2017, 94:401-414.e6.
29. Waskom ML, Kiani R: Decision making through integration of
sensory evidence at prolonged timescales. Curr Biol 2018,
28:3850-3856.
30. Stevens CF: Neurotransmitter release at central synapses.
Neuron 2003, 40:381-388.
31. Boerlin M, Machens CK, Denève S: Predictive coding of
dynamical variables in balanced spiking networks. PLoS
Comput Biol 2013, 9 e1003258.
32. Denève S, Machens CK: Efficient codes and balanced
 networks. Nat Neurosci 2016, 19:375-382
This review discusses the advantages of a tight excitation-inhibition
balance for precise computations in neural circuits. They describe how
such this tight balance can be used by associative cortical circuits to
construct high-dimensional representations and learn complex functions
of their input.
33. Pestilli F, Carrasco M, Heeger DJ, Gardner JL: Attentional
enhancement via selection and pooling of early sensory
responses in human visual cortex. Neuron 2011, 72:832-846.
34. Wong K-F, Wang X-J: A recurrent network mechanism of time
integration in perceptual decisions. J Neurosci 2006, 26:1314-
1328.
35. Soltani A, Wang X-J: Synaptic computation underlying
probabilistic inference. Nat Neurosci 2010, 13:112-119.
36. Mante V, Sussillo D, Shenoy KV, Newsome WT: Context-
 dependent computation by recurrent dynamics in prefrontal
cortex. Nature 2013, 503:78-84
This study investigates how cortical circuits in the prefrontal cortex can
perform context-dependent computations based on multidimensional
stimuli. They find that population-level activity shows low-dimensional
trajectories characteristic of the selection and integration of context-
dependent decision signals. They show that the attractor-like dynamics
of recurrent neural networks trained on the same task show the same
characteristic features.
37. Moreno-Bote R, Beck J, Kanitscheider I, Pitkow X, Latham P,
 Pouget A: Information-limiting correlations. Nat Neurosci 2014,
17:1410-1417
This theoretical study describes the precise statistical structure of cor-
related noise which limit information processing in populations of neurons
tuned to a stimulus feature. They show that decorrelation alone does not
necessarily increase the precision of sensory representations.
38. Rumyantsev OI, Lecoq JA, Hernandez O, Zhang Y, Savall J,
Chrapkiewicz R, Li J, Zeng H, Ganguli S, Schnitzer MJ:
Fundamental bounds on the fidelity of sensory cortical coding.
Nature 2020, 580:100-105.
39. Kafashan M, Jaffe A, Chettih SN, Nogueira R, Arandia-Romero I,
Harvey CD, Moreno-Bote R, Drugowitsch J: Scaling of
information in large neural populations reveals signatures of
www.sciencedirect.com

information-limiting correlations. bioRxiv 2020 http://dx.doi.
org/10.1101/2020.01.10.902171.
40. Ni AM, Ruff DA, Alberts JJ, Symmonds J, Cohen MR: Learning
 and attention reveal a general relationship between
population activity and behavior. Science 2018, 359:463-465
This study investigates how correlated noise in visual cortex covaries with
behavioral performance as a function of attention and perceptual learn-
ing. They find that correlated noise projects dominantly on the neural
dimensions which predict decisions at the trial-by-trial level, and that
attention and perceptual learning decrease correlated noise in similar
ways over distinct time scales.
41. Cohen MR, Maunsell JH: Attention improves performance
primarily by reducing interneuronal correlations. Nat Neurosci
2009, 12:1594-1600.
42. Cohen MR, Kohn A: Measuring and interpreting neuronal
correlations. Nat Neurosci 2011, 14:811-819.
43. Engel TA, Steinmetz NA, Gieselmann MA, Thiele A, Moore T,
Boahen K: Selective modulation of cortical state during spatial
attention. Science 2016, 354:1140-1144.
44. Rabinowitz NC, Goris RL, Cohen M, Simoncelli EP: Attention
stabilizes the shared gain of V4 populations. eLife 2015, 4
e08998.
45. Whiteley L, Sahani M: Attention in a Bayesian framework. Front
Hum Neurosci 2012, 6:100.
46. Yang GR, Joglekar MR, Song HF, Newsome WT, Wang X-J: Task
representations in neural networks trained to perform many
cognitive tasks. Nat Neurosci 2019, 22:297-306.
47. Barlow HB: Possible principles underlying the transformations
of sensory messages. In Sensory Communication. Edited by
Rosenblith WA. MIT Press; 1961:217-234.
48. Wei X-X, Stocker AA: A Bayesian observer model constrained
by efficient coding can explain “anti-Bayesian” percepts. Nat
Neurosci 2015, 18:1509-1517.
49. Polanı́a R, Woodford M, Ruff CC: Efficient coding of subjective
value. Nat Neurosci 2019, 22:134-142.
50. Sanborn AN, Chater N: Bayesian brains without probabilities.
 Trends Cogn Sci 2016, 20:883-893
This review discusses how Bayesian accounts of human cognition do not
imply the explicit representation and use of probabilities. Rather, the
authors describe the brain as a Bayesian sampler which performs
approximate (yet asymptotically normative) computations based on lim-
ited cognitive resources. They show how this hypothesis can account for
human biases in probabilistic reasoning.
51. Bhui R, Gershman SJ: Decision by sampling implements
efficient coding of psychoeconomic functions. Psychol Rev
2018, 125:985-1001.
52. Heng JA, Woodford M, Polania R: Efficient sampling and noisy
decisions. eLife 2020, 9:e54962.
53. Lieder F, Griffiths TL: Resource-rational analysis:
understanding human cognition as the optimal use of limited
computational resources. Behav Brain Sci 2019, 43:e1.
54. Tsetsos K, Moran R, Moreland J, Chater N, Usher M,
 Summerfield C: Economic irrationality is optimal during noisy
decision making. Proc Natl Acad Sci U S A 2016, 113:3102-3107
This study investigates the origin of economic irrationality during the
sequential comparison of choice options. They find that humans integrate
information in a ‘winner take all’ fashion which can explain the observed
irrationality of human behavior. They further show that this seemingly
suboptimal integration strategy mitigates optimally the negative effects of
computation noise on performance.
55. Karlsson MP, Tervo DGR, Karpova AY: Network resets in medial
prefrontal cortex mark the onset of behavioral uncertainty.
Science 2012, 338:135-139.
56. Donoso M, Collins AGE, Koechlin E: Foundations of human
reasoning in the prefrontal cortex. Science 2014, 344:1481-
1486.
57. Usher M, Cohen JD, Servan-Schreiber D, Rajkowski J, Aston-
Jones G: The role of locus coeruleus in the regulation of
cognitive performance. Science 1999, 283:549-554.
www.sciencedirect.com
Computation noise in human cognition Findling and Wyart 131
58. Joshi S, Li Y, Kalwani RM, Gold JI: Relationships between pupil
 diameter and neuronal activity in the locus coeruleus, colliculi,
and cingulate cortex. Neuron 2016, 89:221-234
This study investigates the relation between pupil dilation and neural
activity in the locus coeruleus and the cingulate cortex. They find that
changes in pupil dilation can reflect neural activity in the locus coeruleus,
and that pupil-linked arousal may coordinate neural activity in the cingu-
late cortex.
59. Joshi S, Gold JI: Pupil size as a window on neural substrates of
cognition. Trends Cogn Sci 2020, 24:466-480.
60. Aston-Jones G, Cohen JD: An integrative theory of locus
coeruleus-norepinephrine function: adaptive gain and optimal
performance. Annu Rev Neurosci 2005, 28:403-450.
61. Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI:
Rational regulation of learning dynamics by pupil-linked
arousal systems. Nat Neurosci 2012, 15:1040-1046.
62. Filipowicz AL, Glaze CM, Kable JW, Gold JI: Pupil diameter
encodes the idiosyncratic, cognitive complexity of belief
updating. eLife 2020:9.
63. Joshi S, Gold JI: Context-dependent relationships between
locus coeruleus firing patterns and coordinated neural activity
in the anterior cingulate cortex. bioRxiv 2020 http://dx.doi.org/
10.1101/2020.09.26.314831.
64. Pfeffer T, Avramiea A-E, Nolte G, Engel AK, Linkenkaer-Hansen K,
 Donner TH: Catecholamines alter the intrinsic variability of
cortical population activity and perception. PLoS Biol 2018, 16
e2003453
This study investigates the effects of catecholamines on the internal
variability of large-scale neural activity and the perception of bistable
visual stimuli. They find that low doses of atomoxetine, a noradrenaline
reuptake inhibitor, increase the rate of spontaneous perceptual alterna-
tion and the temporal structure of ‘scale-free’ population activity across
cortex.
65. Jepma M, Te Beek ET, Wagenmakers E-J, van Gerven JMA,
Nieuwenhuis S: The role of the noradrenergic system in the
exploration-exploitation trade-off: a psychopharmacological
study. Front Hum Neurosci 2010, 4:170.
66. Findling C, Skvortsova V, Wyart V: A role for the noradrenergic
system in the precision of reward-guided learning. Symposium
on the Biology of Decision Making. 2019. (Oxford, UK), 9:33.
67. Shenhav A, Botvinick MM, Cohen JD: The expected value of
control: an integrative theory of anterior cingulate cortex
function. Neuron 2013, 79:217-240.
68. de Gee JW, Colizoli O, Kloosterman NA, Knapen T, Nieuwenhuis S,
Donner TH: Dynamic modulation of decision biases by
brainstem arousal systems. eLife 2017:6.
69. de Gee JW, Tsetsos K, Schwabe L, Urai AE, McCormick D,
McGinley MJ, Donner TH: Pupil-linked phasic arousal predicts a
reduction of choice bias across species and decision
domains. eLife 2020:9.
70. Kloosterman NA, de Gee JW, Werkle-Bergner M, Lindenberger U,
Garrett DD, Fahrenfort JJ: Humans strategically shift decision
bias by flexibly adjusting sensory evidence accumulation.
eLife 2019:8.
71. Kloosterman NA, Kosciessa JQ, Lindenberger U, Fahrenfort JJ,
 Garrett DD: Boosts in brain signal variability track liberal shifts
in decision bias. eLife 2020:9
This study investigates the relation between fluctuations in the entropy of
frontal brain signals and adjustments in conservative biases during
perceptual decision-making. They find that liberal shifts in decision biases
are accompanied by increases in the variability of frontal brain signals.
They propose that the regulation of neural variability may support the
moment-to-moment adaptation of decision biases to environmental
demands.
72. Glaze CM, Filipowicz ALS, Kable JW, Balasubramanian V, Gold JI:
A bias–variance trade-off governs individual differences in on-
line learning in an unpredictable environment. Nat Hum Behav
2018, 2:213-224.
73. Tervo DGR, Proskurin M, Manakov M, Kabra M, Vollmer A,
 Branson K, Karpova AY: Behavioral variability through
Current Opinion in Behavioral Sciences 2021, 38:124–132
132 Computational cognitive neuroscience
stochastic choice and its gating by anterior cingulate cortex.
Cell 2014, 159:21-32
This study investigates whether animals can purposefully generate beha-
vioral variability in competitive social settings. They find that rats increase
their behavioral variability when confronted with a computer-simulated
competitor that is capable of predicting their upcoming choices, a
mechanism reflected in the relation between anterior cingulate activity
and behavior.
74. Belkaid M, Bousseyrol E, Durand-de Cuttoli R, Dongelmans M,
Duranté EK, Ahmed Yahia T, Didienne S, Hanesse B, Come M,
Current Opinion in Behavioral Sciences 2021, 38:124–132
Mourot A et al.: Mice adaptively generate choice variability in a
deterministic task. Commun Biol 2020, 3:34.
75. Meyniel F, Schlunegger D, Dehaene S: The sense of confidence
during probabilistic learning: a normative account. PLoS
Comput Biol 2015, 11 e1004305.
76. Meyniel F, Dehaene S: Brain networks for confidence weighting
and hierarchical inference during probabilistic learning. Proc
Natl Acad Sci U S A 2017, 114:E3859-E3868.
www.sciencedirect.com