Learning & Behavior (2022) 50:153–166

https://doi.org/10.3758/s13420-021-00505-1

Cognitive flexibility in the wild: Individual differences in reversal

learning are explained primarily by proactive interference,
not by sampling strategies, in two passerine bird species
Julie Morand‑Ferron1 · Michael S. Reichert2 · John L. Quinn3,4

Accepted: 30 November 2021 / Published online: 11 January 2022

© The Psychonomic Society, Inc. 2021

Abstract
Behavioural flexibility allows animals to adjust to changes in their environment. Although the cognitive processes that explain
flexibility have been relatively well studied in psychology, this is less true for animals in the wild. Here we use data collected
automatically during self-administered discrimination-learning trials for two passerine species, and during four phases
(habituation, initial learning, first reversal and second reversal) in order to decompose sources of consistent among-individual
differences in reversal learning, a commonly used measure for cognitive flexibility. First, we found that, as expected, proactive
interference was significantly repeatable and had a negative effect on reversal learning, confirming that individuals with poor
ability to inhibit returning to a previously rewarded feeder were also slower to reversal learn. Second, to our knowledge for the
first time in a natural population, we examined how sampling of non-rewarding options post-learning affected reversal-learning
performance. Sampling quantity was moderately repeatable in blue tits but not great tits; sampling bias, the variance in the
proportion of visits to each non-rewarded feeder, was not repeatable for either species. Sampling behaviour did not predict
variation in reversal-learning speed to any significant extent. Finally, the repeatability of reversal learning was explained
almost entirely by proactive interference for blue tits; in great tits, the effects of proactive interference and sampling bias on
the repeatability of reversal learning were indistinguishable. Our results highlight the value of proactive interference as a more
direct measurement of cognitive flexibility and shed light on how animals respond to changes in their environment.

Keywords  Behavioural flexibility · Discrimination learning · Individual variation · Adjusted repeatability · Perseveration ·

Exploration · Paridae

Introduction explain behavioural plasticity are the focus of much research

Animals adjust to changes in their environment using a vari-
ety of mechanisms during their lives. Behavioural plastic-
ity allows animals to adjust their behaviour in response to
environmental change, and the diversity of mechanisms that
* John L. Quinn
J.Quinn@ucc.ie
1
Department of Biology, University of Ottawa, Ottawa, ON,
Canada
2
Department of Integrative Biology, Oklahoma State
University, Stillwater, OK, USA
3
School of Biological Earth and Environmental Sciences,
University College Cork, Cork, Ireland
4
Environmental Research Institute, University College Cork,
Cork, Ireland
in many fields (Lee & Thornton, 2021). In neurosciences and
comparative psychology, reversal-learning tasks are widely
used to assay the executive cognitive function ‘cognitive
flexibility’, defined as the ability to update behaviour in
accordance with changes in stimulus-reward contingencies
(Izquierdo et al., 2017). In these tasks, individuals first need
to learn that one stimulus is associated with a reward while
another (or several others) is not. The contingencies are then
reversed, so that the previously rewarded option becomes
non-rewarding, and a previously non-rewarding stimulus
becomes associated with a positive outcome.
Increasingly reversal-learning tasks are being applied
to evolutionary ecological studies of individual cognitive
performance in wild populations (reviewed in: Boogert
et al., 2018; Morand-Ferron et al., 2016). Cognitive flex-
ibility could be important for tracking changing environ-
ments (Shettleworth, 2010; Sol, 2009). For instance, lizards

13
Vol.:(0123456789)
154
from suburban areas, where ecological conditions vary
spatially and temporally, were quicker to learn and rever-
sal learn an association between refuges and predatory cues
than lizards from rural areas (Batabyal & Thaker, 2019).
Performance in serial reversal learning, in which reward
contingencies were repeatedly switched, was predicted by
an index of sociality among three species of corvids, sug-
gesting that dynamic social interactions might select for
greater cognitive flexibility (Bond et al., 2007). Reversal
learning performance did not significantly predict overwin-
ter survival in wild mountain chickadees Poecile gambeli
(Sonnenberg et al., 2019), but negatively impacted survival
after release in captive-raised pheasant Phasianus colchi-
cus (Madden et al., 2018). Whether and how natural selec-
tion acts on the observed individual variation depends on
the mechanisms underlying reversal-learning performance,
which in reality is a composite trait (Nilsson et al., 2015).
In neuroscience, identifying the underlying mechanisms of
reversal-learning performance helps to identify pathologies
associated with cognitive inflexibility, and to examine the
efficacy of drugs administered to alleviate specific issues
(Nilsson et al., 2015). In evolutionary ecological studies it
is also potentially important to do so because the individual
mechanisms could vary in their additive genetic variation
and in their links with fitness through independent associa-
tions with other behaviours. However, few ecological studies
have examined what drives variation in reversal-learning. A
series of studies on mountain chickadees examined spatial
reversal-learning performance at an array of eight feeders
in the field, and specifically examined proactive interfer-
ence errors, i.e., visits to the previously rewarded option
(Croston et al., 2017; Tello-Ramos et al., 2019). While these
studies indicated that proactive interference can vary among
populations (but see Hermer et al., 2021), we still have lit-
tle information on individual variation, and no repeatability
estimates, for proactive interference in natural populations.
Studies on both humans and non-human animals suggest
that reversal-learning tasks involve at least four cognitive
phenomena: (1) the formation of an excitatory association
between the new rewarded stimulus and the reward; (2) the
formation of an independent, inhibitory association, between
the newly non-rewarded stimulus and lack of reinforcement;
(3) attention allocation to the relevant stimulus dimension
(e.g. colour, space); and (4) proactive interference, in which
the similarity of the previous and new association to be
learned results in perseverative choices towards the previ-
ously rewarded stimulus, which may prevent or slow down
new learning (Crossley et al., 2019; Lewis & Kamil, 2006),
although, counter-intuitively, over the course of several
reversals proactive interference may in fact reduce perse-
veration and speed up reversal learning (Mackintosh et al.,
1968; Strang & Sherry, 2014). While the first three pro-
cesses are common to discrimination and reversal learning,
13
Learning & Behavior (2022) 50:153–166
proactive interference is specific to reversal learning, and
plays an important role in cognitive flexibility (Nilsson et al.,
2015). Proactive interference is itself likely an outcome of
multiple cognitive mechanisms (Anderson & Neely, 1996);
for simplicity, here we refer to the outcome of choosing a
previously rewarded stimulus as proactive interference, and
acknowledge that the specific mechanism responsible for this
phenomenon is unknown in our study system.
When more than two options are available in a reversal-
learning experiment, an additional process that may affect
reversal-learning performance can be identified: exploration
behaviour or information sampling, hereafter, ‘sampling’,
by examining choices made for unrewarded stimuli (e.g.,
Jentsch et al., 2002). Optimal performance in reversal tests
is attained with a win-stay/lose-shift rule: stay with the
rewarded option until it stops paying, then shift to another
alternative until you find a reward (Shettleworth, 2010).
Sampling a non-rewarded option is typically viewed as a
‘learning error’, but this overlooks the possibility that this
behaviour may allow the animal to obtain information on the
characteristics of the alternatives, the value of which may
change – and potentially increase – over time. Individuals
who make many visits to non-rewarded options may tend to
favour sampling over exploitation, a strategy that could be
adaptive in natural conditions (Reader, 2015). For instance,
bees that made more errors during learning were faster to
discover new food sources (Evans et al., 2017). Sampling
allows for tracking changes in the environment (Shettle-
worth et al., 1988), but repeatable individual differences in
such behaviour have been reported in only a few studies
(Morand-Ferron et al., 2011; Smit & van Oers, 2019), and
have not been investigated in the context of reversal-learning
tasks in wild populations. We thus know very little about the
impact of sampling tendencies on reversal performance in
non-human animals. Information reduces uncertainty and
is thus expected to increase fitness (Dall et al., 2005). Here
we predict that individuals who sample at a low rate (which
we term ‘sampling quantity’), and those that sample a nar-
rower range of options (which we term ‘sampling bias’) are
less able to reduce uncertainty about the available options,
resulting in reduced reversal-learning performance.
We used the data generated by a published reversal learning
experiment on free-ranging great and blue tits foraging in
mixed-species flocks (Reichert et al., 2020). Both species
readily visit feeders and interact with cognitive tasks. Although
Reichert et al. (2020) showed substantial individual variation
in learning speed, and a bias towards previously rewarded
feeders indicative of proactive interference, the role of
sampling and proactive interference on learning performance
were unexplored. The experiment used a linear array of five
automated feeders, thus allowing an opportunity to examine
sampling separately from proactive interference (multi-option
test; Izquierdo et al., 2017). Trials were run on the same
Learning & Behavior (2022) 50:153–166
individuals over four different phases – habituation, initial
learning, first reversal learning, and second reversal learning
– which allowed us to examine the repeatability of individual
behaviour expressed over time. The effect of sampling in
preceding stages could be examined for the last three phases,
while a test for an independent effect of proactive interference
on the second reversal learning was possible using proactive
interference measured during the first reversal learning. Thus,
we explored the following questions: (1) Are learning speed
and reversal learning speed associated with sampling quantity
and sampling bias? If greater sampling provides individuals
with more information about the environment, then we would
expect that individuals that sample more would be faster to
respond and learn more quickly when reward contingencies
change in the reversal learning experiments. (2) Does
reversal learning speed correlate negatively with proactive
interference? If individuals tend to visit previously rewarded
feeders more extensively, then we would expect them to take
longer to learn an association with a new rewarding feeder. (3)
Are individual differences in proactive interference, sampling
quantity, and sampling bias consistent over time during the
learning trials, that is, are they repeatable? (4) And in the case
of sampling quantity and bias, are they also repeatable across
the two different contexts of learning and the prior habituation
phase? (5) Is the repeatability of reversal-learning performance
explained by proactive interference, sampling quantity or
sampling bias? If these effects explain among individual
differences in performance during the reversal learning trials,
then their addition to a model should lead to a lower ‘adjusted
repeatability’ estimate (Nakagawa & Schielzeth, 2010). If
they are entirely unrelated to among-individual differences
in reversal-learning performance, there should be no change.
However, it is possible that proactive interference, sampling
quantity and sampling bias could mask individual variation
in performance, in which case accounting for these variables
would lead to a higher adjusted repeatability estimate.
Methods
Study site and species
The data for this experiment came from a study of associa-
tive learning in a wild population of PIT-tagged (passive
integrated transponder) great tits, Parus major, and blue
tits, Cyanistes caeruleus, in Wytham Woods, Oxfordshire,
UK. For full details on the study design, see Reichert et al.
(2020). Briefly, we set out arrays of five programmable
sunflower-seed feeders in each of eight locations within the
wood (four locations in November-December 2017, and four
different locations in January-February 2018). At each site,
feeders were arranged linearly and spaced at 1-m intervals.
Feeders were equipped with a radio-frequency identification
155
(RFID) antenna placed on a perch in front of the only feeder
opening. Access to the feeders was controlled by a sole-
noid connected to a printed circuit board (‘Darwin Board’,
Stickman Technologies Inc., UK). If a PIT-tagged bird was
detected by the antenna at the perch, the solenoid block-
ing the door was released, and a transparent door could be
pecked open to gain access to the sunflower seeds. The board
recorded the RFID code and time of day of each individual
visit.
The project received ethical approval from the Animal
Welfare Body at University College Cork (HPRA license
number AE19130-P017), and was carried out in accordance
with the ASAB Guidelines for the Treatment of Animals in
Behavioural Research and Teaching. All research was con-
ducted under BTO licenses as part of ongoing research in
this population.
Learning experiment and criterion
Prior to the associative learning experiments, any PIT-
tagged individual could obtain food from any of the five
feeders at a site for a period of 4 days (habituation phase).
The habituation phase allowed the birds an opportunity to
become accustomed to gaining access to food at the feeders,
and provided data on the relatively uninhibited sampling
tendencies of individuals prior to the learning experiments.
The initial learning phase took place immediately after
the habituation phase. Each individual was randomly
assigned to one of the five feeders at the site, and the feed-
ers were programmed such that individuals could only gain
access to food at their single assigned feeder. Individu-
als were considered to have learned the association with
their assigned feeder when they made 16 ‘correct’ visits to
the assigned feeder out of 20 consecutive visits, with the
requirement that the first visit within that window was a cor-
rect visit (Reichert et al., 2020). The individual’s learning
speed was then defined as the number of visits until the first
visit at which this criterion was met (i.e., trials to criterion).
The initial learning phase took place for 8 days. Imme-
diately afterwards we began a reversal learning phase, in
which we randomly assigned individuals to a different
rewarded feeder in the array. Although laboratory tests of
reversal learning typically involve only two choices, reversal
learning paradigms can involve more choices, which allows
examining proactive interference and sampling as distinct
phenomena (Izquierdo et  al., 2017; Tello-Ramos et  al.,
2019). Furthermore, we use the term ‘reversal learning’
here to maintain consistency with previous work (Reichert
et al., 2020). The assignment to a new feeder was completely
random by individuals for four of the eight sites, and for the
other four sites, the entire set of birds assigned to one feeder
was reassigned as a group to a randomly selected new feeder.
The purpose of this treatment was to test that our design
13
156
prevented social learning (which seemed to be the case;
Reichert et al., 2020), and to examine possible effects on
the social network, which will be reported elsewhere and are
therefore not explored further here. We measured learning
speed during the first reversal phase as in the initial learning
phase (trials to criterion). The first reversal learning phase
lasted for 8 days in 2017 and 10 days in 2018, when extra
time was needed to repair backup feeders. We then repeated
the randomization procedure a second time, assigning birds
to another new feeder (no bird was assigned to a feeder that
it had already been assigned to in a previous phase). This
second reversal learning phase also lasted for 8 days in 2017
and 10 days in 2018, and we again measured trials to crite-
rion. All of our analyses are for a dataset including the 183
individuals that met the learning criterion for all three of
these phases (n = 115 blue tits and n = 68 great tits; Reichert
et al., 2020).
Sampling and proactive interference
As a first step in examining sampling behaviour expressed
by wild birds in this experiment, we examined visits occur-
ring during the habituation phase (when birds could feed
from any feeder) in three ways. First, we calculated the total
number of visits to all of the feeders during this phase, the
sampling quantity. Second, we calculated the variance in
visits to the different feeders in the array, which we used to
define sampling bias. To do this, we determined the propor-
tion of the total visits that were made to each feeder and
then calculated the variance of these proportions across the
five feeders. This variance ranged from a minimum of zero,
indicating that the individual visited each feeder an equal
number of times (i.e., sampling broadly), to a maximum of
0.2, indicating that the individual exclusively visited just
one feeder in the array and therefore was likely to have
acquired less information about the array of feeders as a
whole. Third, we estimated the strength of preference for the
most preferred feeder, by calculating the proportion of visits
directed towards the most preferred feeder during the habitu-
ation phase. Six individuals that met the learning criterion
in all three learning phases did not visit the feeders during
the habituation phase.
We defined sampling during the three learning phases
as visits to non-rewarding feeders that took place after
the learning criterion was met. Visits prior to the learn-
ing criterion being met were not included because these
were more likely to have resulted from errors made while
searching for the rewarding feeder rather than sampling.
For the reversal phases, we also excluded visits to the
feeder assigned in the previous phase, because these were
considered to have resulted from proactive interference
(i.e., three feeders could be sampled in each of the rever-
sals, whereas four feeders could be sampled in the initial
13
Learning & Behavior (2022) 50:153–166
phase). Visits that took place to non-rewarded feeders
when the individual’s assigned feeder was malfunction-
ing were not included. Sampling quantity and sampling
bias were otherwise defined the same as for the habitu-
ation phase. Individuals that did not make any sampling
visits were excluded from analyses of sampling bias (n =
2, 6, and 10 for initial learning, first reversal, and second
reversal, respectively). Unlike the analyses of the habitu-
ation phase, for the learning phases we did not calculate
the third metric of sampling, the strength of preference for
the most preferred feeder, because birds directed most of
their visits in the learning phases to the rewarded feeder,
which is not considered in analyses of sampling. Note that
we use an operational definition of sampling, and can-
not discriminate between visits to non-rewarded feeders
that occurred ‘intentionally’ to sample information, and
those that were simply errors. However, from the point of
view of our hypotheses on information gathering by visit-
ing non-rewarding feeders and its effects on subsequent
reversal learning, all such visits can potentially provide
information, whether they resulted from a ‘true error’ or
from ‘sampling’.
Proactive interference during the reversal phases was
quantified as the number of visits to the feeder during
those phases that had been rewarded in the most recent
previous phase. For the first reversal, proactive interfer-
ence was thus defined as taking place when individuals
visited the feeder during the first reversal learning phase
that they had been assigned to during the initial learn-
ing phase. For the second reversal, proactive interference
was defined as taking place when individuals visited the
feeder during the second reversal learning phase that they
had been assigned to during the first reversal phase. Note
that we found no evidence that birds were biased during
the second reversal towards visiting the assigned feeder
from the initial learning phase, so these visits were not
classified as proactive interference. We excluded visits
when the individual’s assigned feeder was malfunction-
ing (some feeders temporarily failed to open or register
visits due to electrical issues; see Reichert et al., 2020).
Proactive interference visits for a given reversal phase
included visits both before and after the learning criterion
was met, because such interference could continue to bias
visits through the entire phase (i.e., early and late errors;
Nilsson et al., 2015), and because we had observed a sig-
nificant bias towards the previous feeder across all visits
in both the first and second reversal (Reichert et al., 2020).
When examining the effect of proactive interference on
reversal-learning performance, we used proactive inter-
ference errors registered in the first reversal as a predictor
of trials-to-criterion in the second reversal. This ensured
independence of the data used to quantify each variable
because they occurred in distinct experimental phases.
Learning & Behavior (2022) 50:153–166
Statistical analyses
Effect of sampling and proactive interference on learning
Objective 1, the test of relationships between sampling and
trials to criterion, was explored in separate models for initial,
first reversal, and second reversal learning. The model for
the second reversal also included proactive interference to
address Objective 2. For the initial learning phase, we used a
general linear model with trials to criterion (ln-transformed)
as the dependent variable and the following measures of
sampling during the habituation phase (see above): sampling
quantity, sampling bias, and the proportion of visits directed
to the most visited feeder. We included interactions between
species and these three variables. The following additional
potentially confounding effects were explored in a previ-
ous study, and are therefore not discussed here, but were
included in the model because they were sometimes related
to behaviour at the feeders: sex, age, a binary feeder position
variable indicating whether the feeder was at the edge of the
array or not, and the time in hours that either the individual’s
own assigned feeder or any of its non-assigned feeders were
malfunctioning prior to it reaching the learning criterion
(for further details, see Reichert et al., 2020). We removed
non-significant interaction terms before calculating the final
model coefficients so that we could interpret main effects.
The general linear models run for the first and second
reversal phases were similar. The dependent variable was the
number of trials to criterion (ln-transformed) and the main
explanatory variables were sampling quantity and sampling
bias in the previous phase, and the interactions between each
of these variables and species. As for the initial learning
model, we also included feeder location, sex, age, and the
malfunctioning times of feeders, along with the number of
trials to criterion in the previous phase (ln-transformed) and
the number of rewarded visits post-criterion in the previous
phase. The latter variable was included because the dura-
tion of each phase was the same for all birds, regardless of
when they met the learning criterion. Thus, this variable
accounts for variation among birds in the delay, and num-
ber of post-criterion rewards, between reaching the crite-
rion and the change in experimental contingencies occurring
at the start of the next phase, which could impact reversal
learning speed (Mackintosh, 1974). For the second reversal
phase, proactive interference visits in the previous phase,
and the interaction between this variable and species, was
also included to test the second objective.
Repeatability of sampling and proactive interference
In Objective 3, we tested for consistent individual differ-
ences in sampling behaviours and proactive interference
using a repeatability analysis. For sampling quantity and
157
sampling bias, we did this across the three learning phases,
and across the two reversal learning phases alone, and did so
for the two different species separately. We modelled sam-
pling quantity as a Poisson variable, and included the phase
and the number of correct visits made after the learning cri-
terion was met (to control for overall visit numbers) as fixed
effects, and individual identity as a random factor. Sampling
bias was modelled as a Gaussian variable (the best fit we
could achieve, though the errors departed from normality),
and we included phase, the number of correct visits made
after the learning criterion was met, and sampling quantity
as fixed effects, and individual identity as a random factor.
The repeatability value was calculated using mixed models
in the rptR version 0.9.22 package (Stoffel et al., 2017) in R
version 4.1.0 software (R Development Core Team, 2021).
Because we included fixed effects in these models, the esti-
mates were adjusted repeatabilities (Nakagawa & Schielzeth,
2010). Note that for these and all other analyses, we did not
include a separate random effect of site because of low rep-
lication at some sites resulting in poor model performance.
The repeatability of proactive interference was tested using
a Poisson distribution with individual identity as a random
factor, and with the following fixed effects: phase (first rever-
sal or second reversal; note that proactive interference can-
not be measured for initial learning), the total number of
correct visits in that phase (to control for overall activity at
the feeders), and the total number of rewarded visits made
after the learning criterion was met in the previous phase.
Separate models were run for great tits and blue tits.
For Objective 4, to assess whether sampling behaviour
was consistent across contexts, we tested whether sam-
pling during the habituation phase (when all feeders were
rewarding) was predictive of sampling during the initial
learning phase (when only one feeder was rewarding).
Note that we did not use a repeatability approach here
because the sampling behaviours were from very differ-
ent contexts, and were effectively different variables, so
could not be treated as the same response variable. We
performed separate generalized linear models for sampling
quantity (Poisson) and sampling bias (Gaussian) during
the initial learning phase as the dependent variable, and
the equivalent variables during the habituation phase as
fixed effects. Specifically, we included three variables
representing sampling during the habituation phase: the
total number of visits, the proportion of visits to the pre-
ferred feeder, and the variance in feeder visits (further
details above). We also included interactions between spe-
cies and these three variables to test the generality of any
observed association, along with main effects of sex and
age. We focus our analyses of interactions on the species
variable, rather than examining all possible interactions,
because we expected the most pronounced differences in
sampling to be at the species level, and because sex and
13
158 Learning & Behavior (2022) 50:153–166

age never explained variation in learning performance Results

(Reichert et al., 2020). In addition, for the analyses of
sampling bias during initial learning as the dependent
variable, we included sampling quantity during the initial
learning phase (i.e., the number of sampling visits) as a
fixed effect, to account for the possibility that sampling
bias would be skewed for individuals with low visit num-
bers (because it is calculated as a variance). For sampling
quantity, we included the total number of rewarded visits
after the learning criterion was met to control for total
visit effort. Non-significant interaction terms were dropped
before calculating the final model.
For Objective 5, we examined whether proactive
interference, sampling quantity, or sampling bias observed
during the two reversal learning phases affected the
repeatability estimate of trials to criterion during these
phases. We estimated repeatabilities separately for each
species using rptR. Trials to criterion (ln-transformed) was
the dependent variable, individual identity was the random
effect, and phase was an additional fixed effect along with
assigned feeder location and the malfunctioning times
of the feeders as described above. We then calculated
repeatability values with and without the number of
proactive interference visits during this phase as an
additional fixed effect. We compared the fit of these models
using Wald tests with the anova function in R. We repeated
these procedures but with sampling quantity or sampling
bias as the additional fixed effect, rather than proactive
interference. We then examined whether including the two
sampling variables in addition to proactive interference
resulted in additional effects on repeatability of trials to
criterion in the reversal phases.
Descriptives of visit behaviour
In the initial learning phase, individuals took a mean (SD)
of 60.0 (100.4) trials to reach criterion, had a mean sam-
pling quantity of 40.7 (46.2) visits, and had a mean of 428.1
(133.3) total visits to feeders across the entire phase. In the
first reversal, the mean trials to criterion was 59.1 (70.7),
mean sampling quantity was 36.9 (41.5), the mean num-
ber of proactive interference visits was 45.5 (50.2), and the
mean total visits was 488.7 (114.1). In the second reversal,
individuals took a mean of 70.6 (98.7) trials to reach crite-
rion, had a mean sampling quantity of 29.4 (36.1), a mean
number of proactive interference visits of 37.1 (34.6), and
had a mean of 517.8 (147.2) total visits.
1) Is trials to criterion related to sampling behaviour?
In the initial learning phase, trials to criterion was
not predicted by sampling bias (Estimate: −0.69 ± 8.14,
P = 0.93), sampling quantity (Estimate: −0.0007 ±
0.002, P = 0.69), or feeder preference (Estimate: −0.57
± 1.89, P = 0.76), during the habitation phase (Table 1).
Similarly, there was no effect of sampling quantity on
trials to criterion in the next phase for either the first
reversal (Estimate: −0.003 ± 0.002, P = 0.12; Table 2) or
the second reversal (Estimate: 0.003 ± 0.002, P = 0.22;
Table 3; Fig. 1b). However, there was a non-significant
trend for individuals that were more biased in their
sampling to take fewer trials to meet criterion in the next
phase, both for the first reversal (Estimate: −2.38 ± 1.37, P

Table 1  Trials to criterion during initial learning, the dependent variable, in relation to sampling during the preceding habituation phase
Estimate SE t P

Intercept 4.00 0.68 5.84 <0.001

Species (great tit) −0.70 0.19 −3.59 <0.001
Sampling quantity in the habituation phase −0.001 0.002 −0.40 0.69
Sampling bias in the habituation phase −0.69 8.14 −0.08 0.93
Proportion visits to preferred feeder in habituation phase −0.57 1.89 −0.30 0.76
Own feeder malfunction time 0.07 0.02 3.46 <0.001
Other feeders malfunction time 0.06 0.01 4.37 <0.001
Feeder position (edge) −1.61 0.19 −8.55 <0.001
Sex (male) 0.05 0.18 0.27 0.79
Age (first year) −0.03 0.18 −0.16 0.88

Results from a linear mixed model with trials to criterion in the initial learning phase (ln-transformed) as the dependent variable. Note that high
values of trials to criterion indicate slow learning speed, and therefore learning is slower (more trials to criterion) with increasing values of any
given variable if the variable’s estimate is positive, while learning is faster (fewer trials to criterion) with negative estimates. The reference cat-
egory for species was blue tit, for sex was female, for age was adult, and for feeder position was centre. The interactions between species and
variance in feeder visits during habituation phase, and between species and proportion visits to preferred feeder during habituation phase were
not significant and were dropped before calculating these estimates. N = 177

13
Learning & Behavior (2022) 50:153–166 159

Table 2  Trials to criterion in the first reversal learning phase, the = 0.08; Table 2) and the second reversal (Estimate: −1.84
dependent variable, in relation to sampling in the preceding, initial ± 1.02, P = 0.07; Table 3; Fig. 1c).
learning phase

Estimate SE t P 2) Is trials to criterion related to proactive interference?

Intercept 3.58 0.4 9.01 <0.001 There was a significant positive relationship between tri-
Species (great tit) −0.09 0.19 −0.49 0.62 als to criterion for the second reversal and the number of
Sampling quantity in initial −0.003 0.002 −1.57 0.12 proactive interference visits in the first reversal (Estimate ±
Sampling bias in initial −2.38 1.37 −1.74 0.08 SE: 0.007 ± 0.003, P = 0.017; Table 3; Fig. 1a). This indi-
Own feeder malfunction time 0.13 0.03 4.16 <0.001 cates that a stronger tendency to use the previously rewarded
Other feeders malfunction time 0.09 0.02 4.97 <0.001 feeder during the first reversal predicted slower learning in
Feeder position (edge) −0.49 0.16 −2.96 <0.001 the second reversal.
Sex (male) −0.20 0.16 −1.21 0.23
Age (first year) 0.05 0.17 0.31 0.76 3) Do individuals differ consistently in sampling behaviour
Trials to criterion in initial (ln) −0.02 0.06 −0.38 0.71 and proactive interference?
Number rewards post−criterion 0.001 0.001 1.28 0.20
in initial
Across the two reversal-learning phases, sampling quan-
Results from a linear mixed model with trials to criterion (ln) in the tity was not significantly repeatable in great tits (R = 0.129,
first reversal phase as the dependent variable. Note that high values of CI = [0, 0.365], P = 0.16) but was moderate and significant
trials to criterion indicate slow learning speed, and therefore learning
in blue tits (R = 0.309, CI = [0.124, 0.475], P < 0.001).
is slower (more trials to criterion) with increasing values of any given
variable if the variable’s estimate is positive, while learning is faster Results were similar when including sampling during initial
(fewer trials to criterion) with negative estimates. The reference cat- learning along with both reversal-learning phases (great tits:
egory for species was blue tit, for sex was female, for age was adult, R = 0.081, CI = [0, 0.238], P = 0.14; blue tits: R = 0.327,
and for feeder position was centre. The interactions between spe-
CI = [0.211, 0.452], P < 0.001).
cies and sampling quantity in the initial phase, and between species
and sampling bias in the initial phase, were not significant and were Sampling bias was not significantly repeatable across
dropped before calculating these estimates. N = 181 the two reversal learning phases for either species (great
tits R = 0.118, CI = [0, 0.406], P = 0.19; blue tits R =
0, CI = [0, 0.199], P = 0.5). This was also true when
including initial learning along with both reversal learning

Table 3  Trials to criterion in the second reversal learning phase, the dependent variable, in relation to proactive interference and sampling dur-
ing first reversal
Estimate SE t P

Intercept 4.5 0.6 7.51 <0.001

Proactive interference in first reversal 0.01 <0.001 2.41 0.02
Species (great tit) 0.21 0.2 1.01 0.31
Sampling quantity in first reversal 0.003 0.002 1.23 0.22
Sampling bias in first reversal −1.84 1.02 −1.81 0.07
Own feeder malfunction time 0.1 0.03 3.87 <0.001
Other feeders malfunction time 0.04 0.02 2.44 0.02
Feeder position (edge) −0.52 0.19 −2.72 0.01
Sex (male) −0.23 0.18 −1.23 0.22
Age (first year) 0.08 0.2 0.39 0.69
Trials to criterion in first reversal (ln) −0.06 0.09 −0.62 0.54
Number rewards post−criterion first reversal −0.003 0.001 −2.89 0.004

Results from a linear mixed model with trials to criterion (ln) in the second reversal phase as the dependent variable. Note that high values of
trials to criterion indicate slow learning speed, and therefore learning is slower (more trials to criterion) with increasing values of any given vari-
able if the variable’s estimate is positive, while learning is faster (fewer trials to criterion) with negative estimates. The reference category for
species was blue tit, for sex was female, for age was adult, and for feeder position was centre. The interactions between species and sampling
quantity in the first reversal, sampling bias in the first reversal, and proactive interference were not significant and were dropped before calculat-
ing these estimates. N = 177

13
160 Learning & Behavior (2022) 50:153–166

A B C

Trials to criterion 2nd reversal (ln)

Trials to criterion 2nd reversal (ln)

Trials to criterion 2nd reversal (ln)

6 6
6

4 4 4

2 2 2

0 0 0
0 50 100 150 200 0 50 100 150 200 0.0 0.1 0.2 0.3 0.4
Less exclusive More exclusive
Proactive interference 1st reversal Sampling quantity 1st reversal Sampling bias 1st reversal

Fig. 1  a Significant relationship between the number of proactive between sampling quantity during the first reversal phase and the
interference visits during the first reversal phase and the number of number of trials to criterion during the second reversal phase. This
trials to criterion during the second reversal phase (higher values = relationship was not significant. c Marginally non-significant relation-
slower learning). Line and shaded region represent the model pre- ship between sampling bias (higher values = feeders are sampled less
diction (± 95% confidence interval) and dots represent the estimated broadly) during the first reversal phase and the number of trials to cri-
marginal means of the data points for each individual. b Relationship terion during the second reversal phase

phases (great tits R = 0.082, CI = [0, 0.252], P = 0.15; The number of sampling visits after reaching criterion
blue tits R = 0.044, CI = [0, 0.165], P = 0.22). (i.e., sampling quantity) in the initial learning phase was
For both great tits and blue tits, proactive interfer- predicted by the sampling quantity to all feeders during the
ence was moderately and significantly repeatable across habituation phase (Estimate: 0.005 ± 0.0003, P < 0.001),
the first and second reversal (Great tits: R = 0.383, CI and by sampling bias (Estimate: −5.04 ± 1.34, P < 0.001),
= [0.145, 0.577], P = 0.001; blue tits R = 0.385, CI = but not by the proportion of visits to the most preferred
[0.214, 0.538], P < 0.001). This indicates that individuals feeder during the habituation phase (Estimate: 0.13 ± 0.29,
differed consistently in their tendency to inhibit responses P = 0.65; Table 4). Sampling bias during initial learning
that were no longer rewarded. was not related to sampling quantity during the habitua-
tion phase (Estimate: −0.00003 ± 0.00009, P =0.72), but
4) Is sampling behaviour consistent across contexts? there was a marginally non-significant effect of sampling
bias during habituation suggesting that birds who sampled

Table 4  Consistency in sampling quantity across contexts. The relationship between sampling quantity post-learning criterion during the initial
learning phase (dependent variable) and sampling variables during the habituation phase

Estimate SE t P

Intercept 3.4 0.09 36.05 <0.001

Species (great tit) −0.29 0.03 −10.01 <0.001
Sampling quantity during habituation phase 0.005 0 18.3 <0.001
Sampling bias during habituation phase −5.04 1.34 −3.75 <0.001
Proportion visits to preferred feeder during habituation 0.13 0.29 0.45 0.65
Number correct visits after initial learning −0.001 0 −9.89 <0.001
Sex (male) 0.06 0.02 2.5 0.01
Age (first year) 0.4 0.02 16.02 <0.001

Results from a generalized linear model with sampling quantity (the number of visits that were to the non-rewarding feeders after the learning
criterion was met) in the initial learning phase as the dependent variable. The reference category for species was blue tit, for sex was female, and
for age was adult. The interactions between species and either the number of habituation phase visits (sampling quantity), the variance in feeder
visits during habituation phase (sampling bias), or the proportion of visits to the preferred feeder during the habituation phase were not signifi-
cant and were dropped before calculating these estimates. N = 177

13
Learning & Behavior (2022) 50:153–166 161

Table 5  Consistency in sampling bias across contexts. The relationship between sampling bias (dependent variable) during initial learning phase
and sampling variables during the habituation phase
Estimate SE t P

Intercept 0.1 0.03 2.97 <0.001

Species (great tit) 0.04 0.01 4.33 <0.001
Sampling quantity during habituation phase −0.0003 0.0001 −0.36 0.72
Sampling bias during habituation phase 0.82 0.43 1.92 0.06
Proportion visits to preferred feeder during habituation −0.14 0.1 −1.48 0.14
Number sampling visits during initial learning −0.0002 0.0001 −1.54 0.12
Sex (male) 0.01 0.01 1.14 0.25
Age (first year) −0.005 0.01 −0.49 0.62

Results from a general linear model with sampling bias (the variance in visits among the four non-rewarding feeders) in the initial learning phase
as the dependent variable. The reference category for species was blue tit, for sex was female, and for age was adult. The interactions between
species and either the number of habituation phase visits (sampling quantity), the variance in feeder visits during habituation phase (sampling
bias), or the proportion of visits to the preferred feeder during the habituation phase were not significant and were dropped before calculating
these estimates. N = 175

feeders broadly during habituation continued to do so later in
the experiment (Estimate: 0.82 ± 0.43, P = 0.057; Table 5;
Online Supplemental Material (OSM) Fig. S1).
5) Is the repeatability of reversal learning speed explained
by proactive interference and sampling?
The adjusted repeatability of trials to criterion in the
reversal phases was lower for both species when proactive
interference in those phases was included as a fixed effect
than when proactive interference was not included (Table 6).
However, including sampling quantity as a fixed effect had
only a very minor effect on the repeatability of trials to cri-
terion in the reversals (Table 6). When sampling bias was
included as a variable, the adjusted repeatability of trials to
criterion in the reversal phases was lower for great tits, but
not for blue tits, compared to when sampling bias was not
included (Table 6). Models of the trials to criterion in the
reversal phases had a better fit when proactive interference
was included as a variable than when proactive interfer-
ence was not included (great tits: Wald test, Χ12 = 44.7, P
< 0.001; blue tits: Wald test, Χ12 = 68.9, P < 0.001). Like-
wise, models with sampling bias included had a better fit
than when sampling bias was not included (great tits: Χ12 =
8.17, P = 0.004; blue tits: Χ12 = 4.28, P = 0.04). However,

Table 6  Estimates of trials to criterion repeatability across the two reversal phases, either excluding, or including, proactive interference and
sampling variables from the current stage
R 95% CI P

Great tit a) Proactive interference and sampling variables not included 0.273 0.055, 0.501 0.01
b) Proactive interference included 0.079 0, 0.347 0.26
c) Sampling quantity included 0.247 0.049, 0.492 0.02
d) Sampling bias included 0.069 0, 0.378 0.30
e) Proactive interference and sampling variables included 0.066 0, 0.414 0.31
Blue tit f) Proactive interference and sampling variables not included 0.138 0, 0.323 0.07
g) Proactive interference included 0 0, 0.212 0.5
h) Sampling quantity included 0.1 0, 0.294 0.15
i) Sampling bias included 0.161 0.001, 0.364 0.04
j) Proactive interference and sampling variables included 0 0, 0.215 1.0

Repeatability estimates (R; 95% confidence interval; P-value) from mixed model analyses. All ten (a)–(j) models had fixed effects of phase,
feeder location, and the malfunctioning times of the feeders. Models (b) and (g) additionally included the number of proactive interference visits
prior to the learning criterion being met during each phase as a fixed effect. Models (c) and (h) included sampling quantity as a fixed effect (but
not proactive interference). Models (d) and (i) include sampling bias as a fixed effect. Models (e) and (j) included proactive interference, sam-
pling quantity and sampling bias during each phase as fixed effects. N = 68 great tits, N = 115 blue tits. Note that for models (d), (e), (i) and (j),
there were 13 great tits and three blue tits that did not have a measurement of sampling bias for one phase, because they made no sampling visits
during that phase

13
162
including sampling quantity did not improve model fit for
great tits (Χ12 = 1.80, P = 0.18) but it did for blue tits (Χ12
= 6.30, P = 0.01). Estimates of repeatability adjusting for
sampling bias and sampling quantity in addition to proac-
tive interference were very similar to the estimates when
these sampling variables were not included (Table 6). This
indicates that proactive interference and sampling bias, but
not sampling quantity, explains among-individual variation
in reversal learning speed for one or both species.
Discussion
While studies on humans and animal models are revealing
the neurological and genetic bases of reversal-learning per-
formance (Nilsson et al., 2015), as well as those specifically
associated with perseveration and sampling errors (Jentsch
et al., 2002; Odland et al., 2021), we still know little about
individual variation in these processes in wild populations.
The repeatability of reversal learning has been assessed in
a few species, with results ranging from no repeatability to
large and statistically significant repeatabilities (reviewed by
Cauchoix et al., 2018). These individual differences could
be due to a number of cognitive mechanisms and behav-
iours known to contribute to reversal-learning performance
(Nilsson et al., 2015), but the extent to which each of these
mechanisms varies among-individuals has not been exam-
ined before, to the best of our knowledge. Here we used a
multi-option discrimination reversal-learning task to assess
the contribution of (1) sampling behaviour, as measured by
errors to options that had never been rewarding during the
learning phases, and (2) proactive interference, as meas-
ured by errors to the previously rewarded option, to vari-
ation in reversal-learning performance. With the exception
of sampling quantity in blue tits, sampling quantity and bias
were not significantly repeatable, and neither variable had
a strong relationship with reversal-learning performance.
Proactive interference was moderately repeatable and nega-
tively impacted reversal-learning performance. Furthermore,
proactive interference explained all of the among-individ-
ual variation in reversal-learning performance for blue tits,
though sampling bias and proactive interference had similar
but indistinguishable effects in great tits.
Sampling quantity
We predicted a positive relationship between sampling
quantity post-learning and the subsequent reversal-learning
performance, i.e., that more sampling would lead to better
performance. This prediction was not supported in either
of the two reversals (Tables 2 and 3). However, there was
evidence that sampling quantity was repeatable within and
across contexts (see below), which suggests this variation is
13
Learning & Behavior (2022) 50:153–166
biologically relevant, and specifically we speculate that sam-
pling quantity was more likely to reflect motivation to use
the feeders. In mice engaged in a sequential go/no-go rever-
sal task, exploration errors were related to the motivation
of individuals to interact with the test device (Odland et al.,
2021). If sampling quantity is indeed underpinned by moti-
vation rather than an attempt to gather information as such,
then the lack of a relationship between sampling quantity
and trials to criterion would suggest that motivation to visit
the feeders is not a driver of differences in reversal-learning
performance, thereby ruling out an important confound in
cognitive experiments (Cooke et al., 2021; Rowe & Healy,
2014). One limitation of our experiment is that it involved
only two reversals, and each time the rewarded feeder was
constant across a period of several days. Thus, sampling
may have been of only limited utility in this relatively sta-
ble environment, in which case one might not readily detect
any association with reversal learning. Future studies in the
wild could easily increase the number of reversals and vary
the rate at which they take place, which may affect the ben-
efits of sampling and reveal additional individual variation
in sampling that may affect reversal-learning performance.
Once the learning criterion was reached, our ‘sampling
quantity’ variable was significantly and moderately repeat-
able in blue tits but not in great tits. Thus, in great tits, vari-
ation in this behaviour was mostly at the within-individual
level, while some variation in blue tits was explained by
consistent among-individual variation. Motivation to use
freely available feeders has previously been shown to be
moderately repeatable in this population for both great tits
and blue tits (Crates et al., 2016), so it is unclear why we
detect repeatability only in blue tits. There was also a spe-
cies difference in sampling quantity itself: blue tits made
more sampling visits than great tits. Furthermore, males and
first-year birds sampled more than females and older birds.
The reasons for these demographic differences despite a
similar foraging ecology could be examined in future stud-
ies that manipulate the social environment at the feeders,
perhaps revealing a role for competition among species, ages
and sexes.
Both great tits and blue tits showed consistency in sam-
pling quantity across contexts early in the experiment (habit-
uation vs. initial learning). For great tits, the consistency
in sampling in the habituation versus initial learning phase
but lack of consistency across the learning phases could
potentially be explained by a change in motivational and/or
attentional state due to the reversal of contingencies, which
may bring in other sources of variation in the tendency to
visit non-rewarding feeders, such as a decrease in the ability
to recognize and remember the actually rewarding feeder,
even after the learning criterion was met (i.e., true errors).
Future studies could investigate the relative contribution of
learning errors and information gathering by manipulating
Learning & Behavior (2022) 50:153–166
the predictability and rate of reversal and the difficulty of
the learning task.
Sampling bias
Sampling bias was unrelated to the total number of visits
during habituation, suggesting it is not a general expres-
sion of motivation to use the feeders, but might indeed be
more closely related to the tendency to collect information,
or the way information is collected. Sampling bias during
the habituation phase was a marginally non-significant pre-
dictor of sampling bias post-criterion of the initial learning
phase, suggesting some cross-contextual consistency in how
broadly birds sampled the different options. However, sam-
pling bias during the learning phases of the experiment, or
solely during the reversal learning phases, was not signifi-
cantly repeatable in any of the two species. It is thus possible
that this process of information gathering is different in the
initial phases of the experiment (habituation and initial dis-
crimination) than in the reversal phases, a change that could
be prompted by variation in the predictability of rewards
(Keasar et al., 2013).
The lack of repeatability of sampling bias means that
most variation is due to within-individual adjustment in
sampling, or errors made, in response to the environmental
or test context. A potential source of within-individual vari-
ation in sampling bias is the energetic state of the animal,
which has been shown to influence the propensity to sample
novel options (Katz & Naug, 2015) or options that have not
paid before (Arvidsson & Matthysen, 2016). If the state of
individuals varies in a manner that is not consistent over
the phases of the experiment (e.g., stochasticity in foraging
success at the experimental feeders or other sources), this
could explain the lack of repeatability of state, and thus of
sampling tendencies. Greater experimental control over the
energetic state of individuals in the laboratory might explain
why the tendency to sample was found to be significantly
repeatable in artificial selection lines of great tits (Smit &
van Oers, 2019) but not here in this field-based study.
We predicted that if sampling many options is beneficial
by providing information that can be used to inform future
choices, then individuals with a smaller sampling bias, indi-
cating a tendency to distribute sampling visits more evenly
across the feeders, would require fewer trials to meet the
learning criterion. However, we found no support for this
prediction in either reversal. In natural conditions, sampling
a range of options might be an adaptive strategy in some
ecological conditions (Evans et al., 2017). This might not be
the case in reversal tasks, where some information becomes
irrelevant after the reversal; in our experiment, two of five
feeders have a different payoff after a reversal. However, it
is not clear why information on the other three feeders was
not used by individuals to reduce errors and improve their
163
performance (Pike et al., 2016). Other studies have reported
that sampling does not always lead to the maximization
of payoffs, for instance in bees learning to use two feed-
ers that varied in their probability of reward (Dunlap et al.,
2017). Increasing the costs of sampling – for example, by
increasing the distance between feeders or adding a task to
complete to have access to the food – could lead to more
targeted information-gathering visits and, potentially, a posi-
tive impact on learning. Although a bias for the previously
rewarded feeder during the reversals demonstrates that great
and blue tits used some information from the previous phase
in guiding their current decisions (Reichert et al., 2020),
it is possible that birds mostly memorized information on
rewarded, rather than non-rewarded, feeders. If this was the
case, it could suggest that in our system reversal-learning
performance is mainly driven by the formation and inhi-
bition of positive associations, as opposed to learned non-
reward (Nilsson et al., 2015).
Proactive interference
Proactive interference was moderately and significantly
repeatable in both species, pointing to intrinsic differences
among individuals. Repeatability is a prerequisite for, and
sets the upper limit to, heritability in a trait. Equally most
of the intrinsic differences could be caused by non-genetic
effects (Kruuk, 2004) – for example, permanent environment
or maternal effects (e.g., Quinn et al., 2009). Our results re-
iterate previous findings that reversal-learning performance
is negatively impacted by proactive interference (Crossley
et al., 2019; Lewis & Kamil, 2006), but in our experimental
design it was also possible for birds to make non-proactive
interference errors (i.e., visits to other feeders than those
made by visiting the previously rewarded feeder), which is
not the case in two-option tasks. Note that our results indi-
cate a robust link between proactive interference and rever-
sal speed, as we examined proactive interference errors and
reversal performance independently, instead of during the
same reversal. Moreover, the criterion for reversal learning
was attained more slowly than that for initial learning, an
effect that has often been used to detect proactive interfer-
ence in reversal learning studies (e.g., Mancini et al., 2019).
Proactive interference captured relatively more among-
individual variance than reversal-learning performance
itself, that is, it was more repeatable. This difference was
especially pronounced for blue tits, for whom proactive
interference was moderately repeatable (R = 0.39), but trials
to criterion in reversals was not (Reichert et al., 2020). Com-
bined with the finding that proactive interference explains
all of the repeatability in reversal-learning performance for
blue tits (Table 6), this suggests that examining proactive
interference itself could be a valid avenue for future stud-
ies on the evolution of cognitive flexibility (Tello-Ramos
13
164
et al., 2019). This would bring the benefit of examining
a less composite trait than reversal-learning performance
itself, which leaves out little or no among-individual varia-
tion to be explained by other variables – at least for blue tits
in our study system. This was not true for the great tit, for
which the effects of sampling bias and proactive interference
on the repeatability of reversal learning were similar; note
that when sampling bias and proactive interference were
included in the same model, the order in which they were
entered determined which explained repeatability, so their
effects on repeatability were indistinguishable. Nevertheless,
we argue that the proactive interference measure was likely
the most important because, in the second reversal learning
model, proactive interference was a significant fixed effect,
but sampling bias had only a non-significant tendency to
predict performance and this was in the opposite direction to
that predicted, i.e., reversal-learning performance decreased
with increasing sampling bias.
Conclusion
Our results demonstrate that an automated system can be
used to provide novel insight into the mechanisms that cause
consistent among-individual differences in cognitive flex-
ibility in the wild. Most of our findings were similar for both
species, suggesting that the mechanisms driving cognitive
flexibility are likely to be similar among species. Decom-
posing among-individual variation into its constituents is
an important step towards identifying heritable variation in
traits potentially targeted by natural or sexual selection. It
may also help explain why some tasks that seem to concep-
tually measure the same ability do not covary (e.g., detour-
reaching and reversal learning, both implicated as measures
of behavioural flexibility; Audet & Lefebvre, 2017; Troisi
et al., 2021; Völter et al., 2018).
Here, we have identified proactive interference as a
repeatable trait that underlies most of the among-individual
variation in reversal-learning performance, at least for one
of two species, and probably the other. This suggests that
other processes that may contribute to variation in reversal-
learning performance, such as the formation of an excitatory
association with the currently rewarded feeder and of inhibi-
tory associations with the non-rewarded feeders, or the utili-
zation of working memory (Hassett & Hampton, 2017), do
not do so at the among-individual level, although they could
be repeatable traits in other test contexts (e.g., discrimina-
tion tests; Cauchoix et al., 2018). A natural extension of this
work would examine whether proactive interference errors
can themselves be decomposed into further constituents with
significant among-individual variation (e.g., perseveration
vs. learned non-reward; Nilsson et al., 2015), as well as their
covariation with other phenotypic and life-history traits. Our
13
Learning & Behavior (2022) 50:153–166
results on sampling behaviour gave equivocal results, possi-
bly because sampling is also a composite trait, with elements
of exploration, motivation, and other unidentified processes.
Additionally, the relative importance of these effects is likely
to be dynamic across phases of the experiment. Examining
decision making by fitting competing models of learning
onto trial-by-trial choice data (Daw et al., 2006; Metha et al.,
2020) could be a valuable approach to examine individual
variation in exploration-exploitation in reversal-learning
tasks. Furthermore, the costs and benefits of proactive
interference and sampling may depend on the permanence
of environmental change: proactive interference may act
as a buffer when changes are transient, while sampling in
such conditions may provide a misleading characterization
of environmental conditions. In contrast, when changes are
relatively long-lasting, sampling would enable individu-
als to adjust to the new situation quickly, while proactive
interference would slow them down. Thus, investigations
of individual variation and covariation in these traits under
different patterns of environmental change will contribute
to the understanding of reversal-learning performance in an
ecological context.
Supplementary Information  The online version contains supplemen-
tary material available at https://d​ oi.o​ rg/1​ 0.3​ 758/s​ 13420-0​ 21-0​ 0505-1.
Acknowledgements  Sam Crofts and Keith McMahon assisted with
fieldwork. Martin Whitaker helped design the selective feeders; Josh
Firth, Gabrielle Davidson, Ipek Kulahci, James Savage and Iván de
la Hera helped with construction. Ben Sheldon provided access and
funding to maintain the study system. JMF would like to thank Ethan
Hermer for useful discussions. Robert Hampton and two anonymous
referees provided very useful comments. Funding was provided by
the European Research Council under the European Union’s Horizon
2020 Programme (FP7/2007-2013)/ERC Consolidator Grant ‘EVOLE-
COCOG’ Project No. 617509 and a Science Foundation Ireland ERC
Support Grant 14/ERC/B3118, awarded to JLQ.
References
Anderson, M. C., & Neely, J. H. (1996). Interference and inhibition in
memory retrieval. In E. L. Bjork & R. A. Bjork (Eds.), Memory.
Handbook of perception and cognition (2nd Edi, pp. 237–313).
Academic Press. https://​doi.​org/​10.​1016/​B978-​01210​2570-0/​
50010-0
Arvidsson, L. K., & Matthysen, E. (2016). Individual differences in
foraging decisions: Information-gathering strategies or flexibility?
Behavioral Ecology, 27(5), 1353–1361. https://​doi.​org/​10.​1093/​
beheco/​arw054
Audet, J. N., & Lefebvre, L. (2017). What’s flexible in behavioral flex-
ibility? Behavioral Ecology, 28(4), 943–947. https://​doi.​org/​10.​
1093/​beheco/​arx007
Batabyal, A., & Thaker, M. (2019). Lizards from suburban areas learn
faster to stay safe. Biology Letters, 15(2), 20190009. https://​doi.​
org/​10.​1098/​rsbl.​2019.​0009
Bond, A. B., Kamil, A. C., & Balda, R. P. (2007). Serial reversal learn-
ing and the evolution of behavioral flexibility in three species of
North American corvids (Gymnorhinus cyanocephalus, Nucifraga
Learning & Behavior (2022) 50:153–166
columbiana, Aphelocoma californica). Journal of Comparative
Psychology, 121(4), 372–379. https://d​ oi.o​ rg/1​ 0.1​ 037/0​ 735-7​ 036.​
121.4.​372
Boogert, N. J., Madden, J. R., Morand-Ferron, J., & Thornton, A.
(2018). Measuring and understanding individual differences in
cognition. Philosophical Transactions of the Royal Society B, 373,
20170280.
Cauchoix, M., Chow, P. K. Y., van Horik J. O., Atance, C. M., Bar-
beau, E. J., Barragan-Jason, G., … Morand-Ferron, J. (2018). The
repeatability of cognitive performance: a meta-analysis. Philo-
sophical Transactions of the Royal Society B, 373, 20170281.
https://​doi.​org/​10.​1098/​rstb.​2017.​0281
Cooke, A. C., Quinn, J. L., Davidson, G. L., & Oers, K. van. (2021).
Motivation , accuracy and feedback through experience explain
innovative problem solving and its repeatability. Animal Behav-
iour, 174, 249–261. https://​doi.​org/​10.​1016/j.​anbeh​av.​2021.​01.​
024
Crates, R. A., Firth, J. A., Farine, D. R., Garroway, C. J., Kidd, L.
R., Aplin, L. M., … Sheldon, B. C. (2016). Individual variation
in winter supplementary food consumption and its consequences
for reproduction in wild birds. Journal of Avian Biology, 47(5),
678–689. https://​doi.​org/​10.​1111/​jav.​00936
Crossley, M., Lorenzetti, F. D., Naskar, S., O’Shea, M., Kemenes, G.,
Benjamin, P. R., & Kemenes, I. (2019). Proactive and retroactive
interference with associative memory consolidation in the snail
Lymnaea is time and circuit dependent. Communications Biology,
2(1), 242. https://​doi.​org/​10.​1038/​s42003-​019-​0470-y
Croston, R., Branch, C. L., Pitera, A. M., Kozlovsky, D. Y., Bridge, E.
S., Parchman, T. L., & Pravosudov, V. V. (2017). Predictably harsh
environment is associated with reduced cognitive flexibility in
wild food-caching mountain chickadees. Animal Behaviour, 123,
139–149. https://​doi.​org/​10.​1016/j.​anbeh​av.​2016.​10.​004
Dall, S. R. X., Giraldeau, L. A., Olsson, O., McNamara, J. M., & Ste-
phens, D. W. (2005). Information and its use by animals in evolu-
tionary ecology. Trends in Ecology & Evolution, 20(4), 187–193.
https://​doi.​org/​10.​1016/j.​tree.​2005.​01.​010
Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J.
(2006). Cortical substrates for exploratory decisions in humans.
Nature, 441(7095), 876–879. https://d​ oi.o​ rg/1​ 0.1​ 038/n​ ature​ 04766
Dunlap, A. S., Papaj, D. R., & Dornhaus, A. (2017). Sampling and
tracking a changing environment: Persistence and reward in
the foraging decisions of bumblebees. Interface Focus, 7(3),
20160149. https://​doi.​org/​10.​1098/​rsfs.​2016.​0149
Evans, L. J., Smith, K. E., & Raine, N. E. (2017). Fast learning in
free-foraging bumble bees is negatively correlated with lifetime
resource collection. Scientific Reports, 7(1), 496. https://​doi.​org/​
10.​1038/​s41598-​017-​00389-0
Hassett, T. C., & Hampton, R. R. (2017). Change in the relative con-
tributions of habit and working memory facilitates serial reversal
learning expertise in rhesus monkeys. Animal Cognition, 20(3),
485–497. https://​doi.​org/​10.​1007/​s10071-​017-​1076-8
Hermer, E., Murphy, B., Chaine, A. S., & Morand-Ferron, J. (2021).
Great tits who remember more accurately have difficulty for-
getting, but variation is not driven by environmental harsh-
ness. Scientific Reports, 11(1), 10083. https://​doi.​org/​10.​1038/​
s41598-​021-​89125-3
Izquierdo, A., Brigman, J. L., Radke, A. K., Rudebeck, P. H., & Hol-
mes, A. (2017). The neural basis of reversal learning: An updated
perspective. Neuroscience, 345, 12–26. https://​doi.​org/​10.​1016/j.​
neuro​scien​ce.​2016.​03.​021
Jentsch, J. D., Olausson, P., De La Garza, R., & Taylor, J. R. (2002).
Impairments of reversal learning and response perseveration
after repeated, intermittent cocaine administrations to monkeys.
Neuropsychopharmacology, 26(2), 183–190. https://​doi.​org/​10.​
1016/​S0893-​133X(01)​00355-4
165
Katz, K., & Naug, D. (2015). Energetic state regulates the explora-
tion-exploitation trade-off in honeybees. Behavioral Ecology,
26(4), 1045–1050. https://​doi.​org/​10.​1093/​beheco/​arv045
Keasar, T., Motro, U., & Shmida, A. (2013). Temporal reward vari-
ability promotes sampling of a new flower type by bumblebees.
Animal Behaviour, 86(4), 747–753. https://​doi.​org/​10.​1016/j.​
anbeh​av.​2013.​07.​010
Kruuk, L. E. B. (2004). Estimating genetic parameters in natural
populations using the ‘animal model’. Philosophical Transac-
tions of the Royal Society B: Biological Sciences, 359(1446),
873–890. https://​doi.​org/​10.​1098/​rstb.​2003.​1437
Lee, V. E., & Thornton, A. (2021). Animal cognition in an urbanised
world. Frontiers in Ecology and Evolution, 9, 633947. https://​
doi.​org/​10.​3389/​fevo.​2021.​633947
Lewis, J. L., & Kamil, A. C. (2006). Interference effects in the
memory for serially presented locations in clark’s nutcrackers,
Nucifraga columbiana. Journal of Experimental Psychology:
Animal Behavior Processes, 32(4), 407–418. https://​doi.​org/​10.​
1037/​0097-​7403.​32.4.​407
Mackintosh, N. J. (1974). The Psychology of Animal Learning. Aca-
demic Press.
Mackintosh, N. J., Mcgonigle, B., Holgate, V., & Vanderver, V.
(1968). Factors underlying improvement in serial reversal learn-
ing. Canadian Journal of Psychology, 22(2), 85–95. https://​doi.​
org/​10.​1037/​h0082​753
Madden, J., Langley, E., Whiteside, M., Beardsworth, C., & van
Horik, J. (2018). The quick are the dead: Pheasants that are slow
to reverse a learned association survive for longer in the wild.
Philosophical Transactions of the Royal Society B, 373, 20170297.
Mancini, N., Hranova, S., Weber, J., Weiglein, A., Schleyer, M., Weber,
D., … Gerber, B. (2019). Reversal learning in Drosophila larvae.
Learning and Memory, 26(11), 424–435. https://d​ oi.o​ rg/1​ 0.​1101/​
lm.​049510.​119
Metha, J. A., Brian, M. L., Oberrauch, S., Barnes, S. A., Featherby, T.
J., Bossaerts, P., … Jacobson, L. H. (2020). Separating probability
and reversal learning in a novel probabilisticreversal-learning task
for mice. Frontiers in Behavioral Neuroscience, 13, 270. https://​
doi.​org/​10.​3389/​fnbeh.​2019.​00270
Morand-Ferron, J., Varennes, E., & Giraldeau, L. A. (2011). Individual
differences in plasticity and sampling when playing behavioural
games. Proceedings of the Royal Society B, 278(1709), 1223–
1230. https://​doi.​org/​10.​1098/​rspb.​2010.​1769
Morand-Ferron, J., Cole, E. F., & Quinn, J. L. (2016). Studying the
evolutionary ecology of cognition in the wild: a review of practi-
cal and conceptual challenges. Biological Reviews, 91, 367–389.
https://​doi.​org/​10.​1111/​brv.​12174
Nakagawa, S., & Schielzeth, H. (2010). Repeatability for Gaussian and
non-Gaussian data: a practical guide for biologists. Biological
Reviews, 85(4), 935–956. https://​doi.​org/​10.​1111/j.​1469-​185X.​
2010.​00141.x
Nilsson, S. R. O., Alsiö, J., Somerville, E. M., & Clifton, P. G. (2015).
The rat’s not for turning: Dissociating the psychological compo-
nents of cognitive inflexibility. Neuroscience and Biobehavioral
Reviews, 56, 1–14. https://​doi.​org/​10.​1016/j.​neubi​orev.​2015.​06.​
015
Odland, A. U., Sandahl, R., & Andreasen, J. T. (2021). Sequential
reversal learning: a new touchscreen schedule for assessing cogni-
tive flexibility in mice. Psychopharmacology, 238(2), 383–397.
https://​doi.​org/​10.​1007/​s00213-​020-​05687-6
Pike, R. K., McNamara, J. M., & Houston, A. I. (2016). A general
expression for the reproductive value of information. Behavio-
ral Ecology, 27(5), 1296–1303. https://​doi.​org/​10.​1093/​beheco/​
arw044
Quinn, J. L., Patrick, S. C., Bouwhuis, S., Wilkin, T. A., & Sheldon, B.
C. (2009). Heterogeneous selection on a heritable temperament
13
166
trait in a variable environment. Journal of Animal Ecology, 78(6),
1203–1215. https://​doi.​org/​10.​1111/j.​1365-​2656.​2009.​01585.x
R Development Core Team. (2021). R: A language and environment
for statistical computing. R Foundation for Statistical Computing.
Reader, S. M. (2015). Causes of individual differences in animal explo-
ration and search. Topics in Cognitive Science, 7(3), 451–468.
https://​doi.​org/​10.​1111/​tops.​12148
Reichert, M. S., Crofts, S. J., Davidson, G. L., Firth, J. A., Kulahci, I.
G., & Quinn, J. L. (2020). Multiple factors affect discrimination
learning performance, but not between-individual variation, in
wild mixed-species flocks of birds. Royal Society Open Science,
7, 192107. https://​doi.​org/​10.​1098/​rsos.​192107
Rowe, C., & Healy, S. D. (2014). Measuring variation in cognition.
Behavioral Ecology, 25, 1287–1292. https://​doi.​org/​10.​1093/​
beheco/​aru090
Shettleworth, S. J. (2010). Cognition, Evolution and Behavior (2nd
ed.). Oxford University Press.
Shettleworth, S. J., Krebs, J. R., Stephens, D. W., & Gibbon, J. (1988).
Tracking a fluctuating environment: a study of sampling. Ani-
mal Behaviour, 36(1), 87–105. https://​doi.​org/​10.​1016/​S0003-​
3472(88)​80252-5
Smit, J. A. H., & van Oers, K. (2019). Personality types vary in their
personal and social information use. Animal Behaviour, 3–11.
https://​doi.​org/​10.​1016/j.​anbeh​av.​2019.​02.​002
Sol, D. (2009). Revisiting the cognitive buffer hypothesis for the evolu-
tion of large brains. Biology Letters, 5(1), 130–133. https://​doi.​
org/​10.​1098/​rsbl.​2008.​0621
Sonnenberg, B. R., Branch, C. L., Pitera, A. M., Bridge, E., & Pra-
vosudov, V. V. (2019). Natural selection and spatial cognition in
wild food-caching mountain chickadees. Current Biology, 29(4),
670–676. https://​doi.​org/​10.​1016/j.​cub.​2019.​01.​006
13
Learning & Behavior (2022) 50:153–166
Stoffel, M. A., Nakagawa, S., & Schielzeth, H. (2017). rptR: repeatabil-
ity estimation and variance decomposition by generalized linear
mixed-effects models. Methods in Ecology and Evolution, 8(11),
1639–1644. https://​doi.​org/​10.​1111/​2041-​210X.​12797
Strang, C. G., & Sherry, D. F. (2014). Serial reversal learning in bum-
blebees (Bombus impatiens). Animal Cognition, 17(3), 723–734.
https://​doi.​org/​10.​1007/​s10071-​013-​0704-1
Tello-Ramos, M. C., Branch, C. L., Kozlovsky, D. Y., Pitera, A. M.,
& Pravosudov, V. V. (2019). Spatial memory and cognitive flex-
ibility trade-offs: to be or not to be flexible, that is the question.
Animal Behaviour, 147, 129–136. https://d​ oi.o​ rg/1​ 0.1​ 016/j.​anbeh​
av.​2018.​02.​019
Troisi, C. A., Cooke, A. C., Davidson, G. L., de la Hera, I., Reichert,
M. S., & Quinn, J. L. (2021). No evidence for cross-contextual
consistency in spatial learning and behavioural flexibility in a pas-
serine. Animal Behavior and Cognition, 8(3), 446–461.
Völter, C., Tinklenberg, B., Call, J., & Seed, A. M. (2018). Compara-
tive psychometrics: establishing what differs is central to under-
standing what evolves. Philosophical Transactions of the Royal
Society B, 373, 20170283. https://d​ oi.o​ rg/1​ 0.1​ 3140/R
​ G.2.2​ .1​ 5283.​
73764
Open Practices Statement The datasets generated during and/
or analysed during the current study are available in the Figshare
repository: https://​doi.​org/​10.​6084/​m9.​figsh​are.​14823​588.​v1
None of the experiments were preregistered.
Publisher’s note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.