A Journal of Conservation Biogeography
) (2016) 22, 249–262
Diversity and Distributions
Diversity and Distributions, (Diversity Distrib.
BIODIVERSITY
RESEARCH
1
School of Geography, Planning and
Environmental Management, Global Change
Institute, The University of Queensland,
Brisbane, Qld 4072, Australia, 2School of
Botany, The University of Melbourne,
Melbourne, Vic. 3010, Australia, 3Department
of Biosciences, University of Helsinki, Helsinki
00014, Finland, 4School of Geography Planning
and Environmental Management, The
University of Queensland, Landscape Ecology
and Conservation Group, Brisbane, Qld 4072,
Australia, 5School of Botany, Environmental
Science, The University of Melbourne,
Melbourne, Vic. 3010, Australia, 6Endeavour
Veterinary Ecology, 1695 Pumicestone Rd,
Toorbul, Qld 4510, Australia, 7The Australian
National University, Research School of
Biology, Canberra 0200, Australia, Niche
Environment and Heritage, PO Box 2443,
North Parramatta, NSW 2150 Australia,
8
Wildlife Heritage and Marine Division,
Department of the Environment, Marine and
Freshwater Species Conservation Section,
Canberra ACT 2700, Australia, 9Office of
Environment and Heritage NSW, Hurstville,
NSW 2220, Australia, 10School of Biological
Sciences, University of Sydney, Sydney, NSW
2006, Australia, 11Koala Research Centre of
Central Queensland, School of Medical and
Applied Sciences, CQ University,
Rockhampton, Qld 4702, Australia,
12
Department of Environment, Land, Water &
Planning, Arthur Rylah Institute for
Environmental Research, Heidelberg, Vic. 3084,
Australia, 13Department of Environment,
Water and Natural Resources, PO Box 39,
Kingscote, SA 5223, Australia, 14Hawkesbury
Institute for the Environment, Western Sydney
University, Locked Bag 1797, Penrith, 2751
NSW, Australia, 15Faculty of Veterinary
Science, University of Sydney, Sydney, NSW
2006, Australia, 16ARC Centre of Excellence for
ª 2016 John Wiley & Sons Ltd
Use of expert knowledge to elicit
population trends for the koala
(Phascolarctos cinereus)
Christine Adams-Hosking1*, Marissa F. McBride2,3, Greg Baxter4,
Mark Burgman5, Deidre de Villiers6, Rodney Kavanagh7, Ivan Lawler8,
Daniel Lunney9,10, Alistair Melzer11, Peter Menkhorst12, Robyn Molsher13,
Ben D. Moore14, David Phalen15, Jonathan R. Rhodes16,18, Charles Todd12,
Desley Whisson17 and Clive A. McAlpine18
ABSTRACT
Aim The koala is a widely distributed Australian marsupial with regional popu-
lations that are in rapid decline, are stable or have increased in size. This study
examined whether it is possible to use expert elicitation to estimate abundance
and trends of populations of this species. Diverse opinions exist about estimates
of abundance and, consequently, the status of populations.
Location Eastern and south-eastern Australia
Methods Using a structured, four-step question format, a panel of 15 experts
estimated population sizes of koalas and changes in those sizes for bioregions
within four states. They provided their lowest plausible estimate, highest plausi-
ble estimate, best estimate and their degree of confidence that the true values
were contained within these upper and lower estimates. We derived estimates
of the mean population size of koalas and associated uncertainties for each
bioregion and state.
Results On the basis of estimates of mean population sizes for each bioregion
and state, we estimated that the total number of koalas for Australia is 329,000
(range 144,000–605,000) with an estimated average decline of 24% over the
past three generations and the next three generations. Estimated percentage of
loss in Queensland, New South Wales, Victoria and South Australia was 53%,
26%, 14% and 3%, respectively.
Main conclusions It was not necessary to achieve high levels of certainty or
consensus among experts before making informed estimates. A quantitative,
scientific method for deriving estimates of koala populations and trends was
possible, in the absence of empirical data on abundances.
Keywords
bioregions, Delphi process, expert elicitation, iconic, knowledge, koala popula-
tions, Phascolarctos cinereus, threatened species, uncertainty.
DOI: 10.1111/ddi.12400
http://wileyonlinelibrary.com/journal/ddi 249
C. Adams-Hosking et al.
Environmental Decisions, The University of
Queensland, Brisbane, Qld 4072, Australia,
17
School of Life and Environmental Sciences,
Faculty of Science Engineering & Built
Environment, Deakin University, Burwood,
Vic. 3125, Australia, 18Landscape Ecology and
Conservation Group, School of Geography,
Planning, and Environmental Management,
The University of Queensland, Brisbane, Qld
4072, Australia
*Correspondence: Christine Adams-Hosking,
School of Geography, Planning and
Environmental Management and Global
Change Institute, The University of
Queensland, Brisbane, Qld 4072, Australia.
E-mail: c.hosking@uq.edu.au
INTRODUCTION
For many species, the available evidence in the form of field
surveys and modelling capabilities to predict population
numbers leaves gaps in the capacity to make timely conserva-
tion decisions. In such contexts, managers have long relied
on experts to translate data and available evidence into
advice (Sutherland, 2006). Expert knowledge can be defined
as substantive information on a particular topic that is not
widely known by others (Martin et al., 2012). Experts are
distinguished from non-experts by having training or experi-
ence with respect to a topic of interest (Fazey et al., 2006).
Unsystematic methods for eliciting and applying expert
knowledge may limit the validity of this knowledge; however,
recent advances in structured, evidence-based methods for
expert elicitation in ecology that incorporate expert knowl-
edge that is not necessarily based on empirical data (e.g.
Perera et al., 2011) have provided information for decision-
makers (Kuhnert et al., 2010; Martin et al., 2012). The use
of expert knowledge is growing as a tool for synthesizing
diverse sources of information into a reliable representation
of the current state of scientific knowledge, including its
uncertainties when elicited with the same level of rigour pro-
vided in the collection and use of empirical data (Kuhnert
et al., 2010; Martin et al., 2012). Structured expert elicitation
has been used to address a broad range of ecological topics
(e.g. Grech et al., 2012; Marcot et al., 2012; McDaniels et al.,
2012), including potentially dealing with uncertainty in an
adaptive management context (Keith et al., 2011). This has
led to its increasing recognition as a valid method for cap-
turing and summarizing knowledge and characterizing
uncertainty when empirical data on species distributions and
abundances are limited or dispersed (Cooke, 2013). Further-
more, expert elicitation is increasingly being accepted as a
tool for decision support, for example, to elicit estimates of
the status of Australian birds (McBride et al., 2012a), to rank
threats to sea turtle populations world-wide (Donlan et al.,
250
2010) and to develop management strategies for endangered
whooping cranes (Grus americana) (Runge et al., 2011).
Use of expert elicitation to consolidate the current state of
knowledge is valuable when the status of a species is uncer-
tain, yet there is general recognition that the species is sub-
ject to multiple, often synergistic, threats to its persistence in
the wild. Consolidated knowledge can assist in the develop-
ment of adaptive management strategies for a species, setting
management goals (Lindenmayer & Burgman, 2005) and
decision-making, particularly when it complements other
information (Krueger et al., 2012). For high-profile species,
structured elicitation can be important in establishing the
credibility of expert knowledge, particularly where outputs
inform management or policy decisions (Cooke & Goosens,
2000; Burgman, 2004; Burgman et al., 2005) or where stake-
holders are concerned about potential bias and manipulation
of expert judgments (e.g. U.S. EPA, 2011).
The koala is an arboreal and folivorous marsupial (Melzer
et al., 2000). It is widely distributed in eastern mainland
Australia with range boundaries closely linked to climatic
conditions (Adams-Hosking et al., 2011, 2012). At extents of
1000s of ha, its occurrence is associated with the distribution
of the genera Eucalyptus and Corymbia (Myrtaceae) (DeGab-
riel et al., 2010) which occur on relatively fertile soils (Moore
et al., 2004; Crowther et al., 2009). Since European settle-
ment in Australia in 1788, the clearing and fragmentation of
forests and woodlands (McAlpine et al., 2006; Santika et al.,
2014) and hunting for its fur have affected the species. In
more recent times, indirect threats from urbanization (Dique
et al., 2003), disease (Wan et al., 2011; Jobbins et al., 2012;
Polkinghorne et al., 2013), dogs, motor vehicle collisions
(Rhodes et al., 2011, 2014) and fire have become major
threats to the koala’s survival (Dique et al., 2003; Lunney
et al., 2007, Commonwealth of Australia, 2009, 2011). Now,
climate change (Gordon et al., 1988; Adams-Hosking et al.,
2011, 2012; Seabrook et al., 2011; Lunney et al., 2012, 2014)
is an additional threat. Knowledge of the status of koala
Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd

populations across its geographical range is needed to plan
for conservation and management actions (Commonwealth
of Australia, 2011).
Current data on koala abundance are patchy, and there is no
consistent monitoring method; hence, they have a high level of
uncertainty when extrapolated to a bioregional scale. There are
significant geographical gaps in abundances and trends that
need addressing so that koala conservation management
actions can be confidently employed to secure koala popula-
tions across their broad geographical range. There also are
diverse expert opinions on koala numbers in various regions.
The aim of this study was to evaluate a structured, expert
elicitation procedure for a species for which empirical data are
limited, and where conflict may exist about estimates of abun-
dances and population trends, and consequently its status.
METHODS
We conducted an elicitation workshop over 4 days in July
2012 to assess koala population sizes and trends throughout
the species’ range. The elicitation process took place in the
context of a risk analysis workshop, facilitated by an elicita-
tion specialist (MB) and assistant (MFM), neither of whom
are involved in koala research. They used a modified version
of the Delphi process for structured group decision-making
(Dalkey & Helmer, 1963; Burgman et al., 2011). The 15
expert participants, who also co-authored this paper, are
active koala researchers in the Australian states and territory
where koalas occur and have knowledge of, and experience
with, koala ecology, conservation practice and modelling.
This allowed for a holistic examination and synthesis of data
and analyses as well as the identification of knowledge gaps.
The ability of groups of diverse but knowledgeable individu-
als to improve on the accuracy of one or a few specialist
experts has been well documented (e.g. Arlinghaus & Krause,
2013). The duration of the elicitation process was 6 months.
We used a structured, four-step process (Speirs-Bridge
et al., 2010; McBride et al., 2012b) to question the experts
(Table 1). This procedure helps improve the accuracy of
estimates (Burgman et al., 2011), promotes the pooling of
individual knowledge and reduces overconfidence (Speirs-
Bridge et al., 2010; McBride et al., 2012b).
Stage 1: Pre-workshop estimation
The first round of the elicitation process was conducted
remotely. We sent the experts spreadsheets containing
questions regarding the status of koala populations. The
questions were drawn from the IUCN Red List criteria
(IUCN, 2012). While the round 1 questionnaire asked
experts to assess koala subpopulations against the full set of
IUCN Red List criteria, the decision was made prior to the
workshop to proceed with further elicitation for only the
two key criteria questions deemed most relevant for the
koala (population size and percentage change in population
size) due to time constraints (see Queensland example Data
Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd
Expert knowledge estimates koala numbers
S1 in Supporting Information). Some participants were not
able to complete their first-round questionnaires. To be
listed as vulnerable, the species must have a maximum
observed, estimated, inferred, projected or suspected reduc-
tion in population size of ≥30% over any 10 years or three
generations, whichever is longer (up to a maximum of
100 years in the future) (sometimes known as the sliding
or moving window, e.g. RAMAS (2014)), and where the
reduction or its causes may not have ceased, may not be
understood or may not be reversible.
Experts were asked to estimate koala population
abundances and trends for each bioregion. Koalas occur in
the states of Queensland, New South Wales [including the
Australian Capital Territory (ACT)], Victoria and South Aus-
tralia. There are multiple bioregions within each state and
some overlap two states. Bioregions were considered the
most appropriate classification of the diverse environments
over which the koala is distributed (Fig. 1). Bioregions are
large, geographically distinct areas of land with common
characteristics, such as geology, landform patterns, climate,
ecological features and plant and animal communities
(Thackway & Cresswell, 1995; IBRA 2012) and thus present
a consistent geographical unit for eliciting estimates.
The experts were asked to estimate population size for each
bioregion, and declines or increases in koala numbers that
capture the largest change in any three-generation period of
three past and three future koala generations (15–21 years).
These estimates were based on a combination of past observa-
tions or knowledgeable estimations and future knowledgeable
projections. The past time frame included 8 years of severe
drought. Along with their best estimate for each bioregion,
experts also provided upper and lower bounds and an associ-
ated confidence level (Table 2). Experts were directed to spec-
ify confidence levels such that for a 90% confidence level, the
true value would be expected to fall between their stipulated
lower and upper limits in 9 of 10 cases.
Although each expert had expertise for specific states and
bioregions, they were asked to complete questions for all
states and bioregions. Responses were compiled, and linear
extrapolation (Bedford & Cooke, 2001; Hora, 2007) was used
to convert the experts’ elicited lower (l) and upper (u)
bounds for each parameter to standardized 80% credible
bounds. The adjusted lower (ladj) and upper (uadj) bounds
were calculated as:
ladj ¼ b  ðb  lÞðcadj =cÞ
uadj ¼ b þ ðu  bÞðcadj =cÞ
where b is the expert’s best estimate, c is their assigned
confidence for their estimates, and cadj = 0.8 is the desired
confidence level (i.e. 80%). Standardizing the expert credible
intervals enabled easy viewing and comparison at the feed-
back stage of the elicitation. The choice to convert to 80%
credible intervals (as opposed to 90% or 100%, for example)
was arbitrary, but allowed for more informative intervals to
251
C. Adams-Hosking et al.

Table 1 The structured elicitation procedure (after McBride et al., 2012b)

Elicitation

Pre-elicitation Pre-workshop Workshop Post elicitation

Background information Estimate Feedback Estimate Mean and median of all experts’
compiled second round responses calculated.
Experts contacted and All experts All experts shown All experts Experts review individual and
briefed on elicitation individually answer anonymous answers individually make group status outcomes via email,
process emailed questions and discuss in second and final provide feedback and make individual
face-to-face workshop anonymous estimates corrections if needed, and sign-off on
final results

Figure 1 The bioregions outlined in

black depict where the koala occurs in
the States of Queensland, New South
Wales (including the Australian Capital
Territory), Victoria and South Australia
(Source: IBRA, 2012).

be displayed to participants than the much wider bounds assessments. These revised assessments were used in the final
typically required to encapsulate 90% or 100% confidence estimation of bioregional koala population sizes and trends
would allow. (e.g. Fig. 2).

Stage 2: Within-workshop feedback and Stage 3: Post-elicitation

re-estimation
The second round of the elicitation process was a 4-day,
face-to-face workshop, led by the facilitator (MB) with the
assistance of MFM. Each expert’s first-round answers from
the email questionnaire were displayed anonymously to the
group by the facilitator. On the first day of the workshop,
the first round of elicited answers was discussed in detail
within the group, with experts sharing knowledge from their
respective regions of expertise. All experts were given the
opportunity to participate in the discussion of each region
and, in particular, to query experts with local knowledge.
The experts agreed that this information was influential in
guiding their revisions of their estimates in cases where they
did not possess knowledge. On the second day of the work-
shop, experts were asked to reconsider their previous assess-
ments in the light of the group discussions. They were given
the opportunity to anonymously revise their first-round
Following the workshop, the second round expert estimates
were compiled and used to generate aggregate estimates of
population size and the percentage change for each biore-
gion. Group aggregate estimates were calculated by taking
the average of expert lower, best and upper estimates, respec-
tively (e.g. Lichtendahl et al., 2013). These estimates were
then summed across bioregions to generate overall estimates
of population size and percentage change for each state.
Bioregional population size estimates were combined
directly, but estimates of population change were combined
by first weighting individual bioregional estimates of
population change by the number of koalas in that
bioregion. Aggregation is not always appropriate for a group
of judgments (Morgan & Henrion, 1990; Kriegler et al.,
2009). For example, opinions may differ on the basis of dif-
ferent interpretations of data (Morgan, 2014). However, here
the patterns of elicited responses for each bioregion were suf-

252 Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd

Table 2 The parameters elicited for the koala. Experts provided
assessments for population size and greatest population change
by bioregion.
What is your estimate of the current population size (number of
mature individuals)?
1. Realistically, what is the lowest value it could be?
2. Realistically, what is the highest value it could be?
3. What is your best estimate (the most likely value)?
4. How confident are you that the interval you provided contains
the truth (provide an answer in the range of 50–100%).
What do you think is the biggest change in population size in any
three generation period over the immediate past three generations
(15–21 years ago) and immediate future three generations (15-21
years hence)? (Answer in terms of % change)
1. Realistically, what is the lowest value it could be?
2. Realistically, what is the highest value it could be?
3. What is your best estimate (the most likely value)?
4. How confident are you that the interval you provided contains
the truth (Provide an answer in the range of 50–100%)
ficiently consistent that, in most cases, the calculation of a
mean and median response for the group was deemed
appropriate.
The experts’ final assessments and the aggregate bioregion
and state values were circulated to the group for comment
and final review of their individual inputs. For population
size, comparative levels of uncertainty for each aggregate
state and bioregion estimate were assessed as:
Uncertainty r ¼ ðUr  Lr Þ=Ur  100;
for Ur and Lr the aggregate upper and lower bounds for
region or state of interest r, respectively. For percentage
change, as estimates were already on a common, inter-
pretable scale without the need for standardization, the
comparative uncertainty levels were just assessed as the
difference between aggregate upper and lower bounds (i.e.
Uncertaintyr = [Ur  Lr]). To provide a measure of the
interexpert variation in population size estimates, the coeffi-
cient of variation (CoV) was calculated (von der Gracht,
2012) on the basis of the standard deviation of expert
lower, best and upper estimates and the means for each
bioregion:
r½br 
CoV½br  ¼
l½br 
where br is the estimate type for a given region or state r
(best estimate, upper or lower bound), r is the standard
deviation, and l is the mean of these estimates across the
expert group. For percentage change, we used the standard
deviation as a measure of inter-expert variation. This choice
was made because the percentage change estimates were
already on a common scale, and because the calculation of
the coefficient of variation is inappropriate for non-ratio
scale data.
Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd
Expert knowledge estimates koala numbers
RESULTS
Bioregional population estimates (group means) ranged from
55 to almost 68,000 koalas. The state mean population esti-
mates of the group of experts (i.e. the group mean) for
Queensland, New South Wales [including the (ACT)], Vic-
toria and South Australia were 79,624, 36,350, 182,505 and
33,319, respectively (Table 5). The total national aggregated
mean population change was a decline of 24% (Table 5).
Proportional levels of uncertainty varied between 55% and
100%, with coefficient of variation values calculated for the
experts’ best estimates only (Table 3). The calculated average
80% lower bounds and upper bounds (Table 3) correspond
to an average of the group’s assessment of the uncertainty of
the responses, rather than a measure of group agreement/
disagreement. These values appear high when represented in
percentage form, but they reflect a level of uncertainty that
is common for expert elicitation exercises. Standard devia-
tion values were for the experts’ best estimates only
(Table 4).
For Queensland’s bioregions, the mean population esti-
mates ranged between 1,943 and 15,821 (Fig. 3). Levels of
uncertainty were between 63% and 92% (Table 3). The pre-
dicted koala population changes were from 20% to 73%
(Table 4), and the population change for the whole of
Queensland, aggregated from bioregional estimates, was a
decline of 53% (Table 5).
For the New South Wales bioregions (including the ACT),
the mean population estimates ranged between 55 and
11,133 (Fig. 4). Levels of uncertainty were between 78% and
100% (Table 3). The predicted koala population changes
were from 4% to 50% (Table 4), and the population
change for the whole of New South Wales and the ACT,
aggregated from bioregional estimates, was a decline of 26%
(Table 5).
In the Victorian bioregions, the mean population estimates
ranged between 117 and 67,917 (Fig. 5). Levels of uncertainty
were between 63% and 95% (Table 3). The predicted koala
population changes were from 2% to 28% (Table 4), and
the population change for the whole of Victoria, aggregated
from bioregional estimates, was a decline of 14% (Table 5).
In the South Australian bioregions, the mean population
estimates ranged between 76 and 19,800 (Fig. 6). Levels of
uncertainty were between 55% and 89% (Table 3). The pre-
dicted koala population changes were from 29% to +16%
(Table 4), and the population change for the whole of South
Australia, aggregated from bioregional estimates, was a
decline of 3% (Table 5).
Coefficients of variation were higher for estimates of
upper and lower bounds than for best estimates, and
higher for the first- round estimates than for second-
round estimates, indicating a shift towards agreement
among experts (von der Gracht, 2012). Bioregions for
which the level of variation among experts was relatively
low included the Lofty Block bioregions (Kangaroo Island
and Mainland) and the majority of the Victorian biore-
253
C. Adams-Hosking et al.

Figure 2 Comparison of experts’ first and second round estimates for the Mulga Lands (NSW) and Lofty Block (Mainland) bioregions,
with round 1 responses shown on the left, for each bioregion pair. Uncertainty bars have been adjusted to reflect 80% credible intervals
for each expert’s response. The mean (solid black line) and median (dashed black line) were calculated using the experts’ estimates.

gions. Bioregions with high levels of variation between

expert responses included the Murray Darling Depression
and Mulga Lands in New South Wales. Disagreement in
uncertainty intervals among experts, measured in terms of
the proportion of non-overlap of expert intervals (for each
variable calculated as the proportion of all possible pair-
wise comparisons between experts for which intervals did
not overlap), decreased from an average of 35% for initial
population estimates to 4% for final estimates, indicating
that a reasonable agreement in the expert responses was
achieved when taking into account their uncertainty. A
similar result was achieved for estimates of percentage
change in population size, with a shift from 44% for ini-
tial responses down to 16% non-overlap in the final round
estimates.
DISCUSSION
The expert elicitation process
The main achievement of this expert elicitation process was
a koala population estimate, at 2012, for each bioregion
through the application of expert elicitation that did not
require formal consensus.
Lack of agreement among scientists is often seen as
problematic by policymakers and the public. However,
experts are often hesitant to offer quantitative estimates
due to the frequently high uncertainties involved. Past
work on koalas, including the 2005 IUCN expert work-
shops on the status of Australian mammals, did not seek
estimates of population size, information about uncertainty

254 Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd
 Expert knowledge estimates koala numbers

Table 3 Group mean population estimates, average of 80% lower and upper bounds, percentage uncertainty and coefficient of variation
of the best estimates of each bioregion’s koala population size (mature individuals only). The estimates of mean population size were
used to calculate an aggregate mean density estimate for each state.

Mean Average of Average of Uncertainty Coefficient

Queensland bioregions population lower bounds upper bounds (%)* of variation†

Brigalow Belt North 15,179 6541 28,810 77 0.3

Central Mackay Coast 8857 3196 18,777 83 0.4
Desert Uplands 6357 1138 14,490 92 0.6
Einasleigh Uplands & Wet Tropics 4750 942 12,426 77 0.2
Mitchell Grass Downs 1943 364 4366 92 0.5
Mulga Lands 15,286 8234 22,476 63 0.5
South Brigalow 11,071 3719 27,608 87 0.3
Southeast Queensland 15,821 9194 224,871 63 0.2
Queensland 79,264

Mean Average of Average of Uncertainty Coefficient

New South Wales bioregions population lower bounds upper bounds (%) of variation

Murray-Darling Depression 55 0 298 100 2.5

South East Corner 655 182 1454 88 0.8
Cobar Peneplain & Riverina 2354 864 5326 84 0.5
Darling-Riverine Plains 964 238 2699 91 0.9
Mulga Lands 711 94 2033 95 0.9
New England Tableland 2771 468 5831 92 0.6
NSW North Coast 8367 4048 14,618 72 0.4
NSW Southwestern Slopes 2310 870 4895 82 0.6
South Brigalow & Nandewar 11,133 4262 21,604 80 0.4
South-Eastern Highlands 1363 363 2950 88 0.5
Sydney Basin 5667 2436 11,307 78 0.4
New South Wales 36,350

Mean Average of Average of Uncertainty Coefficient

Victorian bioregions population lower bounds upper bounds (%) of variation

Murray-Darling Depression 117 16 319 95 0.6

Naracoorte Coastal Plain 4233 1812 7941 77 0.2
Furneaux 330 89 620 86 0.4
Riverina 2833 864 5265 84 0.3
South East Coastal Plain (East) 11,357 6264 18,464 66 0.3
South East Coastal Plain (West) 6000 2872 10,511 73 0.3
South East Corner 2768 463 6897 93 0.4
Otways 67,917 29,762 103,746 71 0.3
Strezleckis 4667 1869 10,739 83 0.2
South-Eastern Highlands/ Northern Ranges 46,667 19,988 85,710 77 0.3
Victorian Midlands 24,231 6747 57,286 88 0.3
Victorian Volcanic Plain 11,385 7017 19,017 63 0.2
Victoria 182,505

Mean Average of Average of Uncertainty Coefficient

South Australian bioregions population lower bounds upper bounds (%) of variation

Murray Darling Depression 475 218 905 76 0.6

Eyre and Yorke Blocks (West) 76 33 168 81 0.7
Lofty Block (Kangaroo Island) 12,600 8158 18,119 55 0.2
Lofty Block (Mainland) 19,800 10,512 30,969 66 0.2
Naracoorte Coastal Plain 368 116 1059 89 0.8
South Australia 33,319

*Uncertainty %: (calculated as [(Upper bound – Lower bound)/Upper bound*100].

†Coefficient of variation values reported here are for the experts’ best estimates only.

Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd 255
C. Adams-Hosking et al.

Table 4 Estimates of percentage change in population size in each bioregion either in the past or future, 80% lower and upper bounds,
percentage uncertainty and inter-expert standard deviation.

Percentage Average of Average of Uncertainty Inter-expert

Queensland bioregions change (%) lower bounds upper bounds (%)* standard deviation†

Brigalow Belt North 63 87 29 58 9

Central Mackay Coast 35 64 5 59 26
Desert Uplands 20 46 1 45 19
Einasleigh Uplands & Wet Tropics 41 75 4 79 29
Mitchell Grass Downs 39 74 2 76 43
Mulga Lands 73 88 41 47 50
South Brigalow 56 78 22 56 47
Southeast Queensland 51 74 20 54 41

Percentage Average of Average of Uncertainty Interexpert

New South Wales bioregions change (%) lower bounds upper bounds (%) standard deviation

MurrayDarling Depression 12 29 4 33 24

South East Corner 46 71 16 55 40
Cobar Peneplain & Riverina 9 34 15 49 16
DarlingRiverine Plains 34 69 57 126 21
Mulga Lands 31 69 1 70 22
New England Tableland 6 19 28 47 10
NSW North Coast 50 72 32 40 18
NSW Southwestern Slopes 23 59 14 73 21
South Brigalow & Nandewar 35 64 7 71 30
SouthEastern Highlands 19 40 9 49 20
Sydney Basin 4 33 15 48 15

Percentage Average of Average of Uncertainty Interexpert

Victorian bioregions change (%) lower bounds upper bounds (%) Standard deviation

MurrayDarling Depression 22 52 1 18 17

Naracoorte Coastal Plain 9 40 19 18 16
Furneaux 9 37 6 10 8
Riverina 9 36 22 20 17
South East Coastal Plain (East) 2 20 17 25 25
South East Coastal Plain (West) 28 58 8 18 22
South East Corner 5 27 15 13 14
Otways 6 51 34 25 26
Strezleckis 20 57 4 0 13
SouthEastern Highlands/ Northern Ranges 21 61 4 10 14
Victorian Midlands 20 53 6 16 13
Victorian Volcanic Plain 15 47 5 19 17

Percentage Average of Average of Uncertainty Interexpert

South Australian bioregions change (%) lower bounds upper bounds (%) standard deviation

Murray Darling Depression 26 56 23 79 51

Eyre and Yorke Blocks (West) 10 32 7 39 37
Lofty Block (Kangaroo Island) 29 51 13 64 24
Lofty Block (Mainland) 16 15 53 68 50
Naracoorte Coastal Plain 5 33 21 54 27

*Uncertainty %: calculated as [(Upper bound-Lower bound)]. Note that this differs from the method used for calculating the uncertainty for
population size estimates, as reported in Table 3.
†Standard deviation values reported here are for the experts’ best estimates only.

in the form of a range of possible values or information the koala as “Least Concern” in view of its wide distribu-
at the regional level from the expert group, two of whom tion, presumed large population (Gordon et al., 2008). It
are co-authors on this current paper. That workshop listed became apparent that much more detail was required for

256 Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd
 Expert knowledge estimates koala numbers

Table 5 State and national koala population sizes and trends for 2012, aggregated from bioregional estimates. The group mean koala
population sizes and uncertainty range are rounded to the nearest thousand. The mean population change and estimated total national
population for 2012 are based on the Red List criterion that captures the largest change in any three generation period, that is in the
past three koala generations (from 15 to 21 years ago) to the future three koala generations (15–21 years into the future).

Queensland New South Wales Victoria South Australia National total

Aggregated mean population 79,000 36,000 183,000 33,000 331,000

Aggregated range ~33,000–153,000 ~14,000–73,000 ~77,000–327,000 ~19,000–51,000 ~144,000–605,000
Aggregated mean population change (%) 53 26 14 3 24

Mean populaon Populaon change (%) Uncertainty for mean pop. size
(past and future) (%)

Figure 3 Estimated mean koala population size, population change and uncertainty for each bioregion in Queensland.

this widespread species to enable appropriate and effective

conservation priority setting (Commonwealth of Australia,
2011) and that is now available as a result of this expert
elicitation process.
In this study, we observed broad agreement among all
experts, which allowed for the calculation of aggregate group
estimates. However, in situations where experts have made
very different judgments about the relevant underlying
science, or if the variable being assessed will be used as an
input to a nonlinear model, aggregation may not be suitable
and could lead to a misleading picture of the true nature of
expert opinion (Keith, 1996; O’Hagan et al., 2006; Morgan,
2014). In these cases, running separate analyses using each of
the key schools of thought could be used to gain insight into
the impact that different opinions have on the resulting deci-
sion or model outputs (Morgan & Henrion, 1990). In Crome
et al. (1996), for example, rather than generate an aggregate
prior, the authors compared the effects of using priors eli-
cited from proponents in the logging industry, and priors
elicited from a conservationist in their model predicting the
impacts of logging on birds. Given that the two groups held
such vastly different opinions about the impact of logging,
investigating the sensitivity of the model outputs to their dif-
ferent opinions was considerably more informative than
attempting to combine opinions from individuals with
conflicting worldviews.
Contribution to koala conservation and management
Expert knowledge plays a role in conservation decision-mak-
ing and is regularly utilized to inform listing decisions under
the IUCN Red List criteria (McBride et al., 2012a). This
study is the first to apply this type of structured expert elici-
tation procedure to quantifying the status of widely dis-
tributed koala populations. It can inform the mapping of the
koala’s conservation status at a regional, state and national
scale, supply estimates of uncertainty and contribute to
setting priorities for data acquisition. The koala population
elicitation process incorporated published accounts of local
koala population sizes and trends (e.g. Kavanagh & Barrott,
2001; Seabrook et al., 2011; Kavanagh & Barrott-Brown,
2014; Lunney et al., 2014; Santika et al., 2014; McAlpine
et al., 2015).
Our results also highlight the differing levels of uncertainty
associated with populations within each of the bioregions. In
South Australia, for example, the management goal for the
Kangaroo Island koala population (55% uncertainty) is a
decrease in abundance to reduce browsing levels, and the
abundance is regularly monitored. Even so, estimates for this
bioregion had upper and lower bounds of 30,000 and 5000,
respectively. Although these bounds appear to represent high
levels of uncertainty when expressed as percentages, they are
typical of such elicitation exercises (e.g. O’Neill et al., 2008;

Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd 257
C. Adams-Hosking et al.

Mean populaƟon PopulaƟon change (%) Uncertainty for mean pop. size
(past and future) (%)

Figure 4 Estimated mean koala population size, population change and uncertainty for each bioregion in New South Wales.

Mean populaƟon PopulaƟon change (%) Uncertainty for mean pop. size
(past and future) (%)

Figure 5 Estimated mean koala population size, population change and uncertainty for each bioregion in Victoria.

Mean populaƟon PopulaƟon change (%) Uncertainty for mean pop. size
(past and future) (%)

Figure 6 Estimated mean koala population size, population change and uncertainty for each bioregion in South Australia.

258 Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd

Czembor & Vesk, 2009; Morgan, 2011; Cooke, 2013). There
was a reasonably high level of best -estimate agreement
among experts, with 7,000–15,000 for Kangaroo Island and
7,000–20,000 for south-east Queensland.
Structured elicitation workshops are used for assessing the
current state of expert knowledge and providing a forum for
discussion and a baseline for further research and assessment
(e.g. Bamber & Aspinall, 2013; Durant et al., 2014). In addi-
tion to highlighting areas of agreement, our process also
highlighted knowledge gaps. The knowledge from such exer-
cises always needs to be treated as a snapshot of current sci-
entific opinion and used with the intent of periodic revision
as new information comes to light. This means that expert
knowledge, including the uncertainties, and how and when
they were reported, must be properly documented.
The workshop presented here suggested regional variations
in koala population numbers and trends, including the
disparity between the northern and southern states and among
bioregions within states. The mostly abundant southern koala
populations are not necessarily secure because of excessively
high levels of herbivory (Duka & Masters, 2005), low genetic
diversity (Cristescu et al., 2009) and, in more recent times, cli-
mate change, including extreme weather events and more fre-
quent fires (CSIRO, 2014). Estimates of the rate of loss, what
the rate is likely to be in the future and where changes are tak-
ing place may inform the drafting and implementation of a
National Recovery Plan for this species now that the National
Koala Conservation and Management Strategy 2009–2014
(Commonwealth of Australia, 2009) has expired.
By providing population abundances and trends, the
expert elicitation process we used has informed the debate
over how to manage koalas. This is demonstrated by post-
workshop consultations with decision-makers in the state
and Commonwealth governments that informed the Com-
monwealth listing on the koalas as vulnerable in Queensland,
New South Wales and the ACT (Australian Government,
2012). While the coastal populations have previously been
the public and policy focus, because that is where most peo-
ple live, the status of koalas at the more arid, inland edge of
their range has become clearer.
ACKNOWLEDGEMENTS
We are grateful to Alison Specht and the Australian Centre
for Ecological Analysis and Synthesis (ACEAS) a Facility
within the Terrestrial Ecosystem Research Network (TERN)
for providing funding and logistic support for this
workshop.
REFERENCES
Adams-Hosking, C., Grantham, H.S., Rhodes, J.R., McAl-
pine, C. & Moss, P.T. (2011) Modelling climate-change-
induced shifts in the distribution of the koala. Wildlife
Research, 38, 122–130.
Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd
Expert knowledge estimates koala numbers
Adams-Hosking, C., McAlpine, C., Rhodes, J.R., Grantham,
H.S. & Moss, P.T. (2012) Modelling changes in the distri-
bution of the critical food resources of a specialist folivore
in response to climate change. Diversity and Distributions,
18, 847–860.
Arlinghaus, R. & Krause, J. (2013) Wisdom of the crowd and
natural resource management. Trends in Ecology and
Evolution, 28, 8–11.
Australian Government (2012) Koala populations in Queens-
land, New South Wales and the Australian Capital Territory
and national environment law. Department of Sustainability,
Environment, Water, Population and Communities. Available
at: https://www.environment.gov.au/system/files/resources/1b
863ab8-39ae-4f21-bd27-7e18952daad6/files/bio220-0213-koa
la-listing-factsheet-general.pdf (accessed May 2014).
Bamber, J.L. & Aspinall, W. (2013) An expert judgement
assessment of future sea level rise from the ice sheets.
Nature Climate Change, 3, 424–427.
Bedford, T. & Cooke, R.M. (2001) Mathematical tools for
probabilistic risk analysis. Cambridge University Press,
Cambridge.
Burgman, M. (2004) Expert frailties in conservation risk
assessment and listing decisions. Threatened species legisla-
tion: is it just an Act? (ed. by P. Hutchings, D. Lunney and
C. Dickman), pp. 20–29. Royal Zoological Society of NSW,
Mosman, NSW, Australia.
Burgman, M.A., Lindenmayer, D.B. & Elith, J. (2005)
Managing landscapes for conservation under uncertainty.
Ecology, 86, 2007–2017.
Burgman, M.A., McBride, M., Ashton, R., Speirs-Bridge, A.,
Flander, L., Wintle, B., Fidler, F., Rumpff, L. & Twardy, C.
(2011) Expert status and performance. PLoS ONE, 6, e22998.
Commonwealth of Australia (2009) National Koala Conserva-
tion and Management Strategy 2009–2014. Department of
the Environment, Water, Heritage and the Arts, Canberra.
Commonwealth of Australia (2011) The koala – saving our
national icon. The Senate Environment and Communica-
tions References Committee, Canberra.
Cooke, R. (2013) Expert judgement assessment: quantifying
uncertainty on thin ice. Nature Climate Change, 3, 311–312.
Cooke, R.M. & Goosens, L.H.J. (2000) Procedures guide for
structured expert judgement in accident consequence mod-
elling. Radiation Protection Dosimetry, 90, 303–309.
Cristescu, R., Cahill, V., Sherwin, W.B., Handasyde, K., Car-
lyon, K., Whisson, D., Herbert, C.A., Carlsson, B.L.J., Wil-
ton, A.N. & Cooper, D.W. (2009) Inbreeding and testicular
abnormalities in a bottlenecked population of koalas
(Phascolarctos cinereus). Wildlife Research, 36, 299–308.
Crome, F.H.J., Thomas, M.R. & Moore, L.A. (1996) A novel
Bayesian approach to assessing impacts of rain forest log-
ging. Ecological Applications, 6, 1104–1123.
Crowther, M.S., McAlpine, C.A., Lunney, D., Shannon, I. &
Bryant, J.V. (2009) Using broad-scale, community survey
data to compare species conservation strategies across
regions: a case study of the Koala in a set of adjacent
259
C. Adams-Hosking et al.
‘catchments’. Ecological Management and Restoration, 10
(Suppl s1), S88–S96.
CSIRO (2014) State of the Climate 2014. Australian Govern-
ment Bureau of Meteorology, Canberra.
Czembor, C.A. & Vesk, P.A. (2009) Incorporating between-
expert uncertainty into state-and-transition simulation
models for forest restoration. Forest Ecology and Manage-
ment, 259, 165–175.
Dalkey, N. & Helmer, O. (1963) An experimental application
of the Delphi method to the use of experts. Management
Science, 9, 458–467.
DeGabriel, J.L., Moore, B.D., Marsh, K.J. & Foley, W.J.
(2010) The effect of plant secondary metabolites on the
interplay between the internal and external environments
of marsupial folivores. Chemoecology, 20, 97–108.
Dique, D.S., Thompson, J., Preece, H.J., Penfold, G.C.,
De Villiers, D.L. & Leslie, R.S. (2003) Koala mortality on
roads in south-east Queensland: the koala speed-zone trial.
Wildlife Research, 30, 419–426.
Donlan, C.J., Wingfield, D.K., Crowder, L.B. & Wilcox, C.
(2010) Using expert opinion surveys to rank threats to
endangered species: a case study with sea turtles. Conserva-
tion Biology, 24, 1586–1595.
Duka, T. & Masters, P. (2005) Confronting a tough issue:
fertility control and translocation for over-abundant Koalas
on Kangaroo Island, South Australia. Ecological Manage-
ment and Restoration, 6, 172–181.
Durant, S.M., Wacher, T., Bashir, S., Woodroffe, R., De
Ornellas, P., Ransom, C., Newby, J., Ab
aigar, T., Abdelga-
dir, M., El Alqamy, H., Baillie, J., Beddiaf, M., Belbachir,
F., Belbachir-Bazi, A., Berbash, A.A.., Bemadjim, N.E., Beu-
dels-Jamar, R., Boitani, L., Breitenmoser, C., Cano, M.,
Chardonnet, P., Collen, B., Cornforth, W.A.., Cuzin, F.,
Gerngross, P., Haddane, B., Hadjeloum, M., Jacobson, A.,
Jebali, A., Lamarque, F., Mallon, D., Minkowski, K., Mon-
fort, S., Ndoassal, B., Niagate, B., Purchase, G., Sama€ıla, S.,
Samna, A.K., Sillero-Zubiri, C., Soultan, A.E., Stanley Price,
M.R. & Pettorelli, N. (2014) Fiddling in biodiversity hot-
spots while deserts burn? Collapse of the Sahara’s mega-
fauna. Diversity and Distributions, 20, 114–122.
Fazey, I., Fazey, J.A., Salisbury, J.G., Lindenmayer, D.B. &
Dovers, S. (2006) The nature and role of experiential
knowledge for environmental conservation. Environmental
Conservation, 33, 1–10.
Gordon, G., Brown, A.S. & Pulsford, T. (1988) A koala
(Phascolarctos cinereus Goldfuss) population crash during
drought and heatwave conditions in south-western
Queensland. Austral Ecology, 13, 451–461.
Gordon, G., Menkhorst, P., Robinson, T., Lunney, D., Mar-
tin, R. & Ellis, M. (2008) Phascolarctos cinereus. In: IUCN
2013. IUCN Red List of Threatened Species. Version
2013.1. Available at: http://www.iucnredlist.org (accessed
June 2014)
von der Gracht, H.A. (2012) Consensus measurement in Del-
phi studies: review and implications for future quality
260
assurance. Technological Forecasting and Social Change, 79,
1525–1536.
Grech, A., Chartrand-Miller, K., Erftemeijer, P., Fonseca, M.,
McKenzie, L., Rasheed, M., Taylor, H. & Coles, R. (2012)
A comparison of threats, vulnerabilities and management
approaches in global seagrass bioregions. Environmental
Research Letters, 7, 024006.
Hora, S. (2007) Eliciting probabilities from experts. Advances
in decision analysis: from foundations to applications (ed. by
E. Ward, R.F. Miles Jr and D. von Winterfeldt), pp. 129–
153. Cambridge University Press, Cambridge.
IBRA (2012) Australia’s Bioregions (IBRA). Australian
Government, Department of the Environment.
IUCN (2012) IUCN red list categories and criteria: version
3.1, 2nd edn. Gland, Switzerland and Cambridge, UK:
IUCN. iv + 32 pp.
Jobbins, S.E., Sanderson, C.E., Griffith, J.E., Krockenberger,
M.B., Belov, K. & Higgins, D.P. (2012) Diversity of mhc
class ii dab 1 in the koala (Phascolarctos cinereus).
Australian Journal of Zoology, 60, 1–9.
Kavanagh, R., & Barrott, E., (2001) Koala populations in the
Pilliga forests. Pp. 93–103 Perfumed Pineries: Environmental
history of Australia’s Callitris forests, ed. by J. Dargavel, D.
Hart and B. Libbis. CRES, Australian National University,
Canberra.
Kavanagh, R. & Barrott-Brown, G. (2014) Pilliga Koala Sur-
veys 2013. Report to NSW Office of Environment and Her-
itage. Niche Environment and Heritage Pty Ltd,
Parramatta.
Keith, D.W. (1996) When is it appropriate to combine
expert judgments? Climatic Change, 33, 139–143.
Keith, D.A., Martin, T.G., McDonald-Madden, E. & Wal-
ters, C. (2011) Uncertainty and adaptive management for
biodiversity conservation. Biological Conservation, 144,
1175–1178.
Kriegler, E., Hall, J.W., Held, H., Dawson, R. & Schellnhu-
ber, H.J. (2009) Imprecise probability assessment of tipping
points in the climate system. Proceedings of the National
Academy of Sciences USA, 106, 5041–5046.
Krueger, T., Page, T., Hubacek, K., Smith, L. & Hiscock, K.
(2012) The role of expert opinion in environmental mod-
elling. Environmental Modelling and Software, 36, 4–18.
Kuhnert, P.M., Martin, T.G. & Griffiths, S.P. (2010) A guide
to eliciting and using expert knowledge in Bayesian ecolog-
ical models. Ecology Letters, 13, 900–914.
Lichtendahl, K.C. Jr, Grushka-Cockayne, Yael & Winkler,
R.L. (2013) Is it better to average probabilities or quantiles?
Management Science, 59, 1594–1611.
Lindenmayer, D. & Burgman, M. (2005) Practical conserva-
tion biology. CSIRO Publishing, Vic., Australia.
Lunney, D., Gresser, S., O’Neill, L.E., Matthews, A. &
Rhodes, J. (2007) The impact of fire and dogs on koalas
at Port Stephens, New South Wales, using population
viability analysis. Pacific Conservation Biology, 13,
189–201.
Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd

Lunney, D., Crowther, M.S., Wallis, I., Foley, W.J., Lemon,
J., Wheeler, R., Madani, G., Orscheg, C., Griffith, J.E.,
Krockenberger, M., Retamales, M. & Stalenberg, E. (2012)
Koalas and climate change: a case study on the Liverpool
Plains, north-west NSW. Wildlife and climate change:
towards robust conservation strategies for Australian fauna
(ed. by D. Lunney and P. Hutchings), pp. 150–168. Royal
Zoological Society of NSW, Mosman NSW, Australia.
Lunney, D., Stalenberg, E., Santika, T. & Rhodes, J. (2014)
Extinction in Eden: identifying the role of climate change
in the decline of the koala in south-eastern NSW. Wildlife
Research, 41, 22–34.
Marcot, B.G., Allen, C.S., Morey, S., Shively, D. & White, R.
(2012) An expert panel approach to assessing potential
effects of bull trout reintroduction on federally listed sal-
monids in the Clackamas River, Oregon. North American
Journal of Fisheries Management, 32, 450–465.
Martin, T.G., Nally, S., Burbidge, A.A., Arnall, S., Garnett,
S.T., Hayward, M.W., Lumsden, L.F., Menkhorst, P.,
McDonald-Madden, E. & Possingham, H.P. (2012) Acting
fast helps avoid extinction. Conservation Letters, 5, 274–280.
McAlpine, C.A., Rhodes, J.R., Callaghan, J.G., Bowen, M.E.,
Lunney, D., Mitchell, D.L., Pullar, D.V. & Possingham,
H.P. (2006) The importance of forest area and configura-
tion relative to local habitat factors for conserving forest
mammals: a case study of koalas in Queensland, Australia.
Biological Conservation, 132, 153–165.
McAlpine, C.A., Lunney, D., Melzer, A., Menkhorst, P., Ste-
phen Phillips, S., Phalen, D., Ellis, W., Foley, W., Baxter,
G., de Villiers, D., Kavanagh, R., Adams-Hosking, C.,
Todd, C., Whisson, D., Molsher, R., Walter, M., Lawler, I.
& Close, R. (2015) Conserving koalas: a review of the con-
trasting regional trends, outlooks and policy challenges.
Biological Conservation, 192, 226–236.
McBride, M.F., Garnett, S.T., Szabo, J.K., Burbidge, A.H.,
Butchart, S.H.M., Christidis, L., Dutson, G., Ford, H.A.,
Loyn, R.H., Watson, D.M. & Burgman, M.A. (2012a)
Structured elicitation of expert judgments for threatened
species assessment: a case study on a continental scale
using email. Methods in Ecology and Evolution, 3, 906–920.
McBride, M.F., Fidler, F. & Burgman, M.A. (2012b) Evaluat-
ing the accuracy and calibration of expert predictions
under uncertainty: predicting the outcomes of ecological
research. Diversity and Distributions, 18, 782–794.
McDaniels, T., Mills, T., Gregory, R. & Ohlson, D. (2012)
Using expert judgments to explore robust alternatives for
forest management under climate change. Risk Analysis, 32,
2098–2112.
Melzer, A., Carrick, F., Menkhorst, P., Lunney, D. & St.
John, B. (2000) Overview, critical assessment, and conser-
vation implications of koala distribution and abundance.
Conservation Biology, 14, 619–628.
Moore, B.D., Wallis, I.R., Wood, J.T. & Foley, W.J. (2004)
Foliar nutrition, site quality, and temperature influence
foliar chemistry of tallowwood (Eucalyptus microcorys).
Ecological Monographs, 74, 553–568.
Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd
Expert knowledge estimates koala numbers
Morgan, M.G. (2011) Certainty, uncertainty, and climate
change. Climatic Change, 108, 707–721.
Morgan, M.G. (2014) Use (and abuse) of expert elicitation
in support of decision making for public policy. Proceed-
ings of the National Academy of Sciences USA, 111, 7176–
7184.
Morgan, M.G. & Henrion, M. (1990) Uncertainty: a guide to
dealing with uncertainty in quantitative risk and policy anal-
ysis. Cambridge University Press, New York, NY.
O’Hagan, A., Buck, C.E., Daneshkhah, A.R., Eiser, J.R.,
Garthwaite, P.H., Jenkinson, D.J., Oakley, J.E. & Rakow, T.
(2006) Uncertain judgments: eliciting expert probabilities.
John Wiley and Sons Ltd., Chichester.
O’Neill, S.J., Osborn, T.J., Hulme, M., Lorenzoni, I. &
Watkinson, A.R. (2008) Using expert knowledge to assess
uncertainties in future polar bear populations under cli-
mate change. Journal of Applied Ecology, 45, 1649–1659.
Perera, A.H., Drew, C.A. & Johnson, C.J. (eds) (2011) Expert
knowledge and its application in landscape ecology. Springer-
Verlag, New York, NY.
RAMAS (2014) Red List version 3.0. Available at: http://
www.ramas.com/redlist.htm (accessed June 2014).
Rhodes, J.R., Fei Ng, C., de Villiers, D.L., Preece, H.J., McAl-
pine, C.A. & Possingham, H.P. (2011) Using integrated
population modelling to quantify the implications of mul-
tiple threatening processes for a rapidly declining popula-
tion. Biological Conservation, 144, 1081–1088.
Rhodes, J., Lunney, D., Callaghan, J. & McAlpine, C. (2014)
A few large roads or many small ones? How to accommo-
date growth in vehicle numbers to minimise impacts on
wildlife. PLoS ONE, 9, e91093.
Runge, M.C., Converse, S.J. & Lyons, J.E. (2011) Which
uncertainty? Using expert elicitation and expected value of
information to design an adaptive program. Biological
Conservation, 144, 1214–1223.
Santika, T., McAlpine, C.A., Lunney, D., Wilson, K.A. &
Rhodes, J.R. (2014) Modelling species distributional shifts
across broad spatial extents by linking dynamic occupancy
models with public-based surveys. Diversity and Distribu-
tions, 20, 786–796.
Seabrook, L., McAlpine, C., Baxter, G., Rhodes, J., Bradley,
A. & Lunney, D. (2011) Drought-driven change in wildlife
distribution and numbers: a case study of koalas in south
west Queensland. Wildlife Research, 38, 509–524.
Speirs-Bridge, A., Fidler, F., McBride, M., Flander, L., Cum-
ming, G. & Burgman, M. (2010) Reducing overconfidence
in the interval judgments of experts. Risk Analysis, 30,
512–523.
Sutherland, W.J. (2006) Predicting the ecological conse-
quences of environmental change: a review of the methods.
Journal of Applied Ecology, 43, 599–616.
Thackway, R. & Cresswell, I.D. (1995) An Interim Biogeo-
graphic Regionalisation for Australia: a framework for setting
priorities in the National Reserves System Cooperative Pro-
gram. Reserve System Unit, Australian Nature Conserva-
tion Agency, Canberra.
261
C. Adams-Hosking et al.
U.S. EPA (2011) Expert elicitation task force white paper.
Science and Technology Policy Council, U.S. Environmen-
tal Protection Agency, Washington, DC. 20460
Wan, C., Loader, J., Hanger, J., Beagley, K., Timms, P. &
Polkinghorne, A. (2011) Using quantitative polymerase
chain reaction to correlate Chlamydia pecorum infectious
load with ocular, urinary and reproductive tract disease in
the koala (Phascolarctos cinereus). Australian Veterinary
Journal, 89, 409–412.
SUPPORTING INFORMATION
Additional Supporting Information may be found in the
online version of this article:
Data S1. Queensland example of the questionnaire provided
to the experts.
262
BIOSKETCH
Christine Adams-Hosking Christine’s main research inter-
ests have involved using a range of systematic conservation
planning tools to model the predicted distribution of terres-
trial species under future climate change and identify ‘cli-
mate change refugia’, using koalas and their key Eucalyptus
food trees as case study species. Currently, she is investigat-
ing the potential impacts of climate change on future agri-
culture to assist Natural Resource Management bodies in
developing climate change adaptation plans.
Author contributions: All co-authors participated equally in
the workshop and contributed to the writing which was led
by Christine Adams-Hosking.
Editor: David Richardson
Diversity and Distributions, 22, 249–262, ª 2016 John Wiley & Sons Ltd