Global Catastrophic Risk Annual Report 2016 FINAL

Global
Challenges
Foundation
Global Catastrophic Risks

2016
© Global Challenges Foundation/Global Priorities Project 2016

GLOBAL CATASTROPHIC RISKS 2016
The views expressed in this report are those of the authors. Their
statements are not necessarily endorsed by the affiliated organisations.
Authors:
Owen Cotton-Barratt*†
Sebastian Farquhar*
John Halstead*
Stefan Schubert*
Andrew Snyder-Beattie†
* = The Global Priorities Project

† = The Future of Humanity Institute, University of Oxford
Graphic design:
Accomplice/Elinor Hägg
Global
Challenges
Foundation
in association with
4 Global Catastrophic Risks 2016

THE GLOBAL CHALLENGES FOUNDATION works to raise awareness of the
Global Catastrophic Risks. Primarily focused on climate change, other en-
vironmental degradation and politically motivated violence as well as how
these threats are linked to poverty and rapid population growth. Against this
background, the Foundation also works to both identify and stimulate the
development of good proposals for a management model – a global gover-
nance – able to decrease – and at best eliminate – these risks.
THE GLOBAL PRIORITIES PROJECT helps decision-makers effectively prior-

itise ways to do good. We achieve his both by advising decision-makers on
programme evaluation methodology and by encouraging specific policies. We
are a collaboration between the Centre for Effective Altruism and the Future
of Humanity Institute, part of the University of Oxford.
Global Catastrophic Risks 2016 5

Contents
Foreword8
Introduction10
Executive summary 12
1. An introduction to global catastrophic risks 20
2. What are the most important global
catastrophic risks? 28
Catastrophic climate change 30
Nuclear war 36
Natural pandemics 42
Exogenous risks 46
Emerging risks 52
Other risks and unknown risks 64
Our assessment of the risks 66
3. Risk factors and interactions between risks 72
Drivers of individual risks 74
Shared risk factors and interactions between risks 78
4. Do institutions collectively underinvest
in global catastrophic risk? 82
Market and political failures 84
Which actors can help reduce global catastrophic risk? 88
5. What can the world do to reduce
global catastrophic risk? 94
Endnotes 100
Acknowledgements107
Contact info 107

Definition: Global
Catastrophic Risk
– risk of events or
processes that would
lead to the deaths of
approximately a tenth of
the world’s population, or
have a comparable impact.

FOREWORD
Dear Reader!
N
early four years ago
when the Global
Challenges Foundation
was established, we
decided on a direction
with two parallel strategies. The first
is increasing the knowledge about
Global Catastrophic Risks (GCRs),
which with our terminology means
threats that can eliminate at least 10%
of the global population. This knowl-
edge is an important prerequisite for
the Foundation’s second strategy: to
encourage debates and proposals as
to how we can effectively and fairly
reduce – and preferably eliminate –
these catastrophic risks.
This publication, the Foundation’s
Annual Report for 2016, is the re- this. In addition to the risks involved
sult of a collaboration between the in the Annual Report for 2016, the
Foundation and the Future of Hu- Foundation actively works with envi-
manity Institute (FHI) and the Global ronmental degradation, weapons of
Priorities Project at Oxford University mass destruction, population growth
in the U.K., which has now lasted for (that exacerbates several risks), and
over two years. A big group of re- political violence which is behind
searchers at the FHI, commissioned many of the world’s current problems.
by the Foundation, summarized Political violence comes in many
where research, focused on charting forms. Various kinds of weapons of
some of the greatest global risks, cur- mass destruction represent poten-
rently stands. tially devastating weaponry. Further,
In addition to describing the risks, political violence creates uncon-
their effects and their likelihood of trolled migration and we receive
occurring, this year’s Annual Report repeated reminders that there is
takes one step further and try’s to also “digital violence” in the form of
show how different risks relate to one cyber-attacks. Together, this takes up
another, what can be done to combat a significant amount of space on the
the risks and who can and should do political agenda, thus stealing atten-

tion from other important risks. And in favor of one or more organizations
above all, the defense against various that have a mandate to decide on how
forms of political violence requires to mitigate GCRs.
a grotesquely large share of public Would this be possible? My counter
resources. Each day, the world spends question is whether there are any
over SEK 40 billion on defence alternatives? To continue relying on
expenditure – money that would be multilateral negotiations increases
needed to fight poverty and prevent the probability that decisions and
catastrophic risks. actions are insufficient and executed
My personal opinion is that in order too late. This means that the likeli-
to drastically minimize GCRs we hood of GCRs continues to escalate.
must develop a model where a major- I hope that this publication can
ity of the world’s nations, with strong deepen the understanding of GCRs
support from leading nations, can and that these insights provide a
make binding decisions which can fertile ground for both debates and
be enforced in an effective and fair proposals on how we can develop a
way. This would imply that individ- better way of managing and address-
ual nations waive their sovereignty ing these risks.
Stockholm, April 2016
Laszlo Szombatfalvy
Founder of Global Challenges Foundation

INTRODUCTION
Global catastrophic risks

pose a pressing challenge
T
his report addresses one
of the most important
issues of our age – global
catastrophic risk. Over
the last decades, behav-
ioural psychology has taught us that,
as a species, we are bad at assessing
scope. Issues that affect ten people do
not intuitively seem ten times more
important than those that affect one
person. Global catastrophic risks are
one area where our scope insensitivi-
ty might prove the most dangerous.
These risks can’t just be treated as
problem for the future, even though
we might well expect them not to ma-
terialise this year or the next. At the
Future of Life Institute, my team and of which has inspired my own deep
I have been calling for global leaders curiosity in cosmology.
to address critical global risk issues This technological power is an
including nuclear weapons, biotech- enormous force for good, but carries
nology and artificial intelligence. This its own risks. Although consuming
builds on existing risk reduction work fossil fuels was critical in creating the
led by institutions such as the United thriving and wonderful civilization
Nations. we live in today, we’ve come to learn
Over the last centuries, humanity that there are potentially catastroph-
has achieved incredible things. New ic long-term consequences from
medical technologies save millions of climate change. Other technologies,
lives every year. Agricultural science more powerful than combustion en-
allows billions to be fed who might gines, might also offer huge benefits
otherwise not exist. And we have and carry unforeseen risks. If we fail
begun to explore the very foundations to manage this risk well, we might be
of our universe itself – the beauty caught out by consequences that fol-

low from the technology more rapidly means being realistic in weighing
than climate change has. these factors against each other. This
As a global community, we need report offers an excellent background
to win the race between the growing to the underlying issues of global
power of our technology and the catastrophic risks, and is an outstand-
wisdom with which we manage it. ing starting point for policy-makers
This requires a nuanced approach developing an interest in the area or
towards technological developments, researchers considering how their
acknowledging both that technol- own work might be brought into the
ogy carries huge potential to make study of global catastrophic risks.
lives better and also that it carries
some risks. Smart risk management
Max Tegmark
Co-founder of the Future of Life Institute
Professor of Physics at MIT

EXECUTIVE SUMMARY
Executive Summary
M
ost generations generations. Reducing these risks is
never experience a therefore both a global and an inter-
global catastrophe. generational public good.
However, the idea of The ever-evolving landscape of
such catastrophes technology and society compounds
is not fanciful: plagues have killed these challenges. Technological
over 10% of world’s population and and economic forces can create new
we came close to nuclear war several global catastrophic risks, such as
times in the 20th century. anthropogenic climate change and
Despite their scale, the risks of the 20th century’s nuclear arms race.
global catastrophes receive limited But technology can also reduce risk,
attention. One reason is that many for example through better vaccines
of these risks are unlikely in any or clean energy.
given decade. But even when the We believe the global community
probability is low, the sheer magni- should work together to harness new
tude of an adverse outcome warrants tools to address global catastrophic
taking these risks seriously. A global risks. It is possible that, collectively,
catastrophic risk not only threatens we significantly under-invest in glob-
everyone alive today, but also future al catastrophic risk reduction.

We believe the global
community should work
together to harness new
tools to address global
catastrophic risks.

EXECUTIVE SUMMARY
What are the biggest threats?

T
he global catastrophic risks in
this report can be divided into
two categories. Some are on-
going and could potentially occur in
any given year. Others are emerging
and may be very unlikely today but
will become significantly more likely
in the coming decades. The most
significant ongoing risks are natural
pandemics and nuclear war, whereas
the most significant emerging risks
are catastrophic climate change and
risks stemming from emerging tech-
nologies. Even where risks remain in
the future, there are things we can do
today to address them.
The Spanish influenza pandemic
of 1918 may have killed as much as
5% of the world population. Some
outbreaks since then infected over a
third of the world’s population (e.g.,
pandemic influenza), whereas others
killed over half of people infected
(e.g., Ebola or SARS). If a disease
were to emerge that was as transmis-
sible as the flu and as lethal as Ebola,
the results could be catastrophic.
Fortunately, this rarely transpires,
but it is possible that it could, for ex-
ample with the H5N1 influenza virus.
The invention of nuclear weap-
ons ushered in a new era of risks
created by human action. A large
nuclear war between major powers
would likely kill tens or hundreds of
millions in the initial conflict, and
perhaps many more if a nuclear win-
ter were to follow. During the Cuban

EXECUTIVE SUMMARY
Missile Crisis, President Kennedy from what is expected or if positive

estimated the chance of nuclear environmental feedback loops occur.
conflict as “between one in three and Catastrophic risks from emerging
even”. Tensions have eased some- technology are less well understood.
what since the Cold War, but could Emerging technologies promise sig-
recur. Moreover, accidents or mis- nificant benefits, but a handful could
calculation with nuclear weapons also create unprecedented risks to
continue to pose a risk. civilisation and the biosphere alike.
Climate change is a well-known Biotechnology could enable the crea-
anthropogenic risk. Even if we suc- tion of pathogens far more damaging
ceed in limiting emissions, scientists than those found in nature, while in
expect significant climate change the longer run, artificial intelligence
to occur. This could bring a host of could cause massive disruption.
global challenges including environ- The relative likelihood and ur-
mental degradation, migration, and gency of the different risks matters
the possibility of resource conflict. when deciding how to respond. Even
But this is not the worst-case sce- though the level of uncertainty is
nario. Although it receives far less extreme, rational action requires
attention, scientists also acknowl- explicit assessments of how much
edge the possibility of catastrophic attention the different risks deserve,
climate change. There is a small and how likely they are. The views of
likelihood that warming could even the authors on these vexed ques-
exceed 6 °C, leaving large swathes of tions, based on our reading of the
the planet dramatically less habita- scientific evidence, are summarised
ble. This could occur if emissions are in the following table. More informa-
not cut sufficiently, if the sensitivity tion can be found in Chapter 2.
of the climate system is different

FIGURE 1. OUR ASSESSMENT OF GLOBAL CATASTROPHIC RISKS
Natural pandemic
Unknown risks Nuclear war

over next 5 years
Higher likelihood
Engineered pandemic
Catastrophic
Asteroid impact climate change
Supervolcanic Catastrophic
eruption disruption from AI
over next 5 years
Lower likelihood
Failure of geo-engineering
Low High APPROPRIATE LEVEL

OF ATTENTION
EXECUTIVE SUMMARY
Humanity can respond to these risks

F
or each of the risks in this To reduce the risk of global ca-
report, we consider actions tastrophe caused by nuclear war:
available to avoid or mitigate • The international community
the risk, and which actors are best- should continue the policy of
placed or responsible for taking that nuclear non-proliferation, and nu-
action. For the most significant risks, clear states can continue to reduce
some of the most promising opportu- stockpiles.
nities are listed here. • Nuclear-weapon states should con-
tinue to work to reduce the chance
To reduce the risk of global ca- of accidental launch or escalation.
tastrophe caused by pandemic:
• The World Health Organisation, To reduce the risk of global ca-
nation states, and other bodies tastrophe caused by emerging
should increase their planning for technologies:
extremely bad pandemics. • Research communities should
• The global health community further investigate the possible
should improve developing world risks from emerging capabilities in
capacity for response, for example biotechnology and artificial intel-
by ensuring that vaccine produc- ligence, and possible solutions.
tion facilities are well-distributed • Policymakers could work with
around the world. researchers to understand the
issues that may arise with these
To reduce the risk of global ca- new technologies, and start to lay
tastrophe caused by climate groundwork for planned adaptive
change: risk regulation.
• Research communities should in-
crease their focus on understand- To reduce global catastrophic risk
ing the pathways to and likelihood in a cross-cutting way:
of catastrophic climate change, • Research communities should fo-
and possible ways to respond. cus greater attention on strategies
• Nations should continue to imple- and technologies for resilience to
ment and improve mechanisms and recovery from global catastro-
for emissions abatement such as phe, for example by developing
carbon taxes or tradable emissions alternate food sources.
quotas, as for non-catastrophic • Nations should work to incorpo-
climate change. rate the interests of future gener-
ations into their decision-making
frameworks.

Research communities
should further investigate
the possible risks from
emerging capabilities
in biotechnology and
artificial intelligence,
and possible solutions.
CHAPTER 1 – AN INTRODUCTION TO GLOBAL CATASTROPHIC RISKS
Chapter 1
An Introduction to
Global Catastrophic Risks
O
ver the course of history, times been uncomfortably high.
the world has suffered Throughout the Cold War, the
disasters of such mag- threat of nuclear warfare loomed
nitude that human civ- large. The United States and the So-
ilisation itself has been viet Union possessed tens of thou-
threatened. Warfare and pandem- sands of high yield nuclear warheads,
ics have caused especially significant and their retaliatory strike systems
damage. Originating in 541-542, the were programmed to respond to any
initial outbreak of the ‘Great Plague of attack within minutes. The world has
Justinian’ killed 25-33 million people come close to the nightmare scenar-
– between 13% and 17% of the world io on a number of occasions. Perhaps
population at the time.1 The plague the narrowest escape came on 27th
had trans-generational consequenc- October 1962. Two Russian B-59 sub-
es: many historians believe that it marine commanders off the coast
weakened the Byzantine Empire at of Cuba gave the order to launch a
a crucial time, undermining its at- nuclear strike against the United
tempts to reconquer Europe.2 States, on the mistaken assump-
In recent times, humanity tion that war had already start-
has not endured catastroph- ed. The launch of a nuclear
ic events on the propor-
As torpedo required the con-
tionate scale of Plague of many sent of all three officers
Justinian. However, the as on board; the second
17%
risk of global catastro- in command, Vasili
phe, which is deter- Arkhipov, was alone
mined by the po- in refusing permis-
tential damage of sion.3
of the world’s population was
the event and While the nu-
killed in the initial outbreak of the
its probabili- ‘Great Plague of Justinian’. That equals clear threat
ty of occur- 25-33 million people has receded
ring, has at since the

The most deadly
event of the 20th century
was probably the Spanish
influenza pandemic of
1918-1920 which killed
between 2.5% and 5% of
the world population… Our
focus here is on even more
extreme possibilities which
receive less attention.
end of the Cold War, the risk remains. machine intelligence which could be-
In addition, ongoing economic and have in a manner incompatible with
technological developments bring, human values.
alongside their benefits, a range of However, our governments and in-
new unprecedented anthropogen- stitutions, whose primary focus is
ic risks: for example, catastrophic understandably on more day-to-day
climate change, pandemics of glob- concerns, may systematically be ne-
al proportions, and the potential for glecting global catastrophic risks.

1.1. Defining Global

Catastrophic Risk
G
lobal catastrophes are events None of the various humanitarian
or processes that would inflict disasters of the 20th century killed
serious damage to humanity more than 10% of the world popula-
on a global scale, such as all-out nu- tion. Around 1% of the world popu-
clear war or a pandemic killing hun- lation died in the First World War,
dreds of millions. The severity of a while up to 3% died in the Second
risk is a function of its scope (the size World War.5 The most deadly event
of the population at risk), intensity of the 20th century was probably the
(how badly this population would be Spanish influenza pandemic of 1918-
affected), and probability (how likely 1920 which killed 50 - 100 million
the disaster is to occur).4 people – between 2.5% and 5% of the
A fatal car crash is a personal ca- world population.6 Although these
tastrophe: a small number of unfor- were huge tragedies, our focus here
tunate victims suffer a severe harm. is on even more extreme possibilities
Most genocides are examples of local which receive less attention. In Chap-
catastrophes: thousands or millions ter 2 we give an overview of the ma-
of people within a country or region jor risks of global catastrophe, and in
lose their lives. The focus of this re- Chapter 3 we look at the factors which
port is catastrophic risks with global may increase or decrease these risks.
scope. We define a global catastrophe Limited historical evidence makes
as a possible event or process that, it very difficult to provide a definitive
were it to occur, would end the lives list of past global catastrophes. There
of approximately 10% or more of the have been at least two in the past two
global population, or do comparable millennia – the Plague of Justinian
damage. Extinction risks are a sub- and the Black Death. Some scholars
set of global catastrophic risks, which have argued that more than 10% of
would end the human race. the world population lost their lives
It is important to put the scale of in pre-industrial wars, though this is
global catastrophic risks in context. heavily disputed.7

FIGURE 1.1. QUALITATIVE RISK CATEGORIES 8
SCOPE
Global
Global warming Depletion of the

Nuclear war
by 0.001 C° ozone layer
Recession in one National

Local
Minute increase
in regional air pollution country genocide
Personal
Loss of one hair Loss of one arm Fatal car crash
Imperceptible Damaging Catastrophic

SEVERITY
1.2. Why Global

Catastrophic Risks matter
A
lthough the chance of dy- Now let’s suppose that the chance
ing in a car crash is small, we of extinction were 0.1% per year and
each take steps to mitigate consider the consequences. It may
the risk such as wearing seat belts and seem at first glance that this would
driving safely. National governments be an acceptable level of risk. How-
take steps to mitigate the risk of rare ever, that would mean an individual
natural disasters, such as earthquakes would be more than five times as like-
and hurricanes. Similarly, it is impor- ly to die in an extinction event than a
tant that the global community works car crash.11 Moreover, these small an-
to reduce the risk of catastroph- nual probabilities add up, so that the
ic events which would have a global chance of extinction within the next
scope. century under this scenario is 9.5%.12
The probabilities of these cat- A global catastrophe, which involves
astrophic events are low but not the death of 10% of the global popu-
negligible. Moreover, small annual lation, is more likely than an event
probabilities compound significantly that involves human extinction. As a
over the long term. result, even if 0.1% were on the high
We do not know of a robust estimate side for extinction risk, it might be of
of the annual probability of global the appropriate order of magnitude
catastrophic risk. Nor do we believe for global catastrophic risk.
that we are able to create a robust Reducing these risks has obvious
estimate because the uncertainties in humanitarian benefits for those alive
key parameters are so large. However, today. But we should also consider
for extinction risks some experts have the welfare of future generations. A
suggested that a 0.1% annual chance global catastrophe could reduce the
of extinction is within the range of standards of living for many genera-
plausible orders of magnitude. A 2008 tions to come, while outright human
Oxford survey of expert judgement on extinction denies existence to all fu-
the topic implied an average annual ture generations. Many leading mor-
extinction risk over the next centu- al philosophers have argued that the
ry of around 0.2%.9 The UK’s Stern welfare of these future generations is
Review on the Economics of Climate of utmost importance.13
Change used 0.1% as an upper bound Global catastrophic risks are also
modeling assumption for annual likely to be politically neglected.14
extinction risk. 10 Global catastrophic risk reduction is

The probabilities of
these catastrophic events
are low but not negligible.
Moreover, small annual
probabilities compound
significantly over the
long term.
a global public good, as even a large beneficiaries are future people who
country would only capture a small have no voice in the political process.
portion of the total benefit of risk mit- For these reasons and others, nation-
igation. Moreover, it is an intergen- al and international actors are likely
erational public good, as many of the to underinvest in risk reduction.

1.3. Why Global Catastrophic

Risks are especially relevant today
N
ow more than ever before, most important future risks may be at
global catastrophic risks de- present unknown. Just as in the early
serve attention. Prior to the 20th century it would have been im-
20th century, the main global cata- possible to predict nuclear weapons,
strophic risks that humankind faced catastrophic climate change, or bio-
were natural pandemics and conven- technology risks, it may be that many
tional warfare. However, economic of the future leading global cata-
and technological development have strophic risks are not yet within sight.
brought a range of new anthropogen- Moreover, to reduce these new an-
ic risks. thropogenic risks we may need lev-
The first of these new risks was nu- els of international coordination that
clear weapons, which gave states un- existing institutions are not designed
precedented destructive power and to produce, something we discuss in
emerged very rapidly: the bombings Chapter 4. A good illustration of this
of Hiroshima and Nagasaki came only is the threat of catastrophic climate
six years after Einstein’s letter to Roo- change. Atmospheric concentrations
sevelt warning of the dangers of nu- of greenhouses gases are now at their
clear fission.15 Other anthropogenic highest level for hundreds of thou-
risks might also mature quickly giv- sands of years,18 and if the interna-
ing us little time to prepare. Advances tional community fails to take strong
in certain kinds of biotechnology, for action soon there is a worryingly high
example, might at some point in the chance of warming in excess of 6°C
next few decades give states, or even (compared to pre-industrial levels)
terrorist groups, the capacity to cre- by the end of the century.19 But be-
ate devastating designer pathogens.16 cause unilateral action is costly for
Likewise, experts warn of the longer- any state, and the benefits are felt by
term risks associated with power- everyone regardless of their contribu-
ful machine intelligence, which may tion, action on greenhouse gas emis-
prove hard to control safely.17 sions has been slow in coming.
Indeed, experience over the last
century suggests that many of the

1.4. What can be done?
W
hen dealing with global policy-makers, industries, research
catastrophic risks we can- communities, and citizens can take
not generally rely on his- preemptive steps to limit global cat-
torical experience or trial and error. astrophic risks. This report outlines
Given the severity of global catastro- the key features of the world’s most
phes, learning from experience would significant global catastrophic risks
be extremely costly or, in the event and identifies, in Chapter 5, some
of human extinction, impossible. But strategies for limiting them.
1.5. How to read this report

C
hapter 2 gives a comprehen- Chapter 3 discusses the causes of
sive overview over the main global catastrophic risks. These in-
global catastrophic risks, from clude both factors that increase the
catastrophic climate change and nu- likelihood or impact of individu-
clear war to risks associated with al risks, as well as factors that affect
emerging technologies such as bio- multiple risks, such as poor govern-
technology, artificial intelligence and ance.
geo-engineering. We discuss the po- Chapter 4 discusses why we cur-
tential impact and the likelihood of rently collectively underinvest in
each risk, as well as the main actions global catastrophic risk. It also dis-
to limit them. Finally, we attempt to cusses what actors are best placed to
compare the risks in terms of how overcome this neglect.
likely they are and in terms of how Finally, in Chapter 5 we briefly dis-
much attention we ought to pay them cuss a number of concrete steps to re-
at present. duce global catastrophic risk.20

CHAPTER 2 – WHAT ARE THE MOST IMPORTANT GLOBAL CATASTROPHIC RISKS?
Chapter 2
What are the Most

Important Global
Catastrophic Risks?
F
or most of human histo- sion. The burning of fossil fuels has
ry, humanity has had to brought huge improvements in hu-
contend with a relatively man welfare, but unless strong action
narrow range of global cat- is taken soon, there is an unaccept-
astrophic risks. Super-vol- able chance that our children and
canic eruptions and large asteroid im- grandchildren will face catastrophic
pacts were possible but very unlikely, global warming. Rapid developments
so only natural pandemics and ex- in biotechnology could enable scien-
treme conventional warfare seriously tists to develop new therapies to re-
threatened the complete destruction duce the global burden of disease and
or permanent stagnation of human feed a growing population, but might
civilisation. also in the future give malicious
Today, thanks to economic and groups the capacity to synthesise dev-
technological progress, global living astating pathogens.
standards have never been higher, This chapter surveys currently the
but unfortunately, for the same rea- most important global catastrophic
son, we face a number of new anthro- risks by examining expert scientific
pogenic global catastrophic risks. opinion on the two determinants of
Some of these appear to be at least as risk: potential impact and likelihood.
threatening to human civilisation as There is also a brief discussion of ac-
natural pandemics and convention- tions available to limit each of these
al warfare. Splitting the atom brought risks, which is summarised in Chap-
the promise of clean power, but also ter 5. In the final section, we give a
led to the nuclear bomb, which has comparative assessment of the differ-
brought humanity to the brink of ca- ent catastrophic risks.
tastrophe on more than one occa-

Splitting the atom
brought the promise of
clean power, but also
led to the nuclear bomb,
which has brought
humanity to the brink of
catastrophe on more than
one occasion.

2.1. Catastrophic
climate change
A
s a result of human activity or strong positive feedback loops in
ty since the Industrial Rev- the carbon cycle. For example, global
olution, atmospheric con- warming might cause the melting of
centrations of Greenhouse Gases arctic permafrost, which would re-
(GHGs) are now at their highest level lease substantial amounts of meth-
for hundreds of thousands of years,21 ane – a potent GHG – into the at-
which has caused global surface and mosphere.22 This process could itself
ocean warming. Continued increas- trigger other positive feedback loops.
es in GHG emissions are very likely Catastrophic warming could also oc-
to cause future warming. The eventu- cur even without these feedback ef-
al level of warming depends on total fects, if climate sensitivity turns out
GHG emissions and on the sensitiv- to be higher than median estimates.
ity of the climate to GHG emissions. Alternatively, it might occur simply
The reports of the Intergovernmental because we are less able to coordinate
Panel on Climate Change (IPCC) focus internationally to reduce emissions
on the most likely levels of warming than we expect.
given a particular emissions path- It is impossible to say with confi-
way. It is widely agreed that the neg- dence exactly what level of warming
ative consequences of the most likely would bring about global catastrophe,
levels of warming will be substantial. in the sense we are interested in here.
However, it is important to take ac- The IPCC states:
count the ‘tail risk’ of lower probabili- “Global climate change risks are
ty, but potentially catastrophic, levels high to very high with global mean
of warming. Of course, what we refer temperature increase of 4°C or more
to in this section as ‘non-catastroph- above preindustrial levels in all rea-
ic climate change’ would still have se- sons for concern, and include severe
vere consequences, but it would not and widespread impacts on unique
constitute a global catastrophe, on and threatened systems, substan-
our definition. tial species extinction, large risks to
global and regional food security, and
POTENTIAL IMPACT the combination of high temperature
OF THE CATASTROPHE and humidity compromising normal
Increasing GHG emissions could po- human activities, including growing
tentially trigger catastrophic climate food or working outdoors in some ar-
change due to high climate sensitivi- eas for parts of the year. The precise
30
levels of climate change sufficient to most likely to follow a medium-high

trigger tipping points (thresholds for emissions scenario, and there is some
abrupt and irreversible change) re- chance we will follow a high emis-
main uncertain, but the risk associ- sions scenario.27 However, the situ-
ated with crossing multiple tipping ation may have changed as a result
points in the earth system or in inter- of the December 2015 Paris Agree-
linked human and natural systems ment.28
increases with rising temperature.”23 In estimating the impact of differ-
The latest IPCC report focuses on ent emissions scenarios, the IPCC
the impacts of warming of 1°C to 4°C does not set out the probability of all
above pre-industrial levels, and sug- possible levels of warming, but in-
gests that the impacts corresponding stead only sets out the likely range of
to more extreme levels of warming warming, where ‘likely’ is defined as
are relatively unstudied.24 However, it having a greater than 66% chance of
is likely that damages increase signif- occurring.29 However, in order to pri-
icantly at higher temperatures,25 and oritise resources effectively govern-
possible that warming of 6°C or more ments need to take into account the
above pre-industrial levels may be whole probability distribution, in-
catastrophic. Warming of this magni- cluding lower probability but extreme
tude is, for example, likely to render levels of warming. Indeed, these may
most of the tropics substantially less constitute the majority of the expect-
habitable than at present.26 ed (probability-weighted) costs of cli-
mate change.
LIKELIHOOD OF Some scholars have provided esti-
THE CATASTROPHE mates of the probability of extreme
The probability of catastrophic cli- warming. The economists Ger-
mate change depends on the lev- not Wagner and Martin Weitz-
el of GHGs in the atmosphere man have inferred estimates
and on the sensitivity of the of the probability of warming
climate to cumulative GHGs. of more than 6°C relative to
According to a report by pre-industrial levels from
King et al for the Cen- the IPCC figures. They
$40
tre for Science Policy argue that even on a
at the University of low-medium emis-
Cambridge, if ma- sions scenario, there
jor countries and is at least a 3%
regions contin- per tonne chance of even-
ue with cur- Is the recommended global price tual 6°C warm-
rent plans and of carbon by some leading economists. ing (with
projects, the Current price is significantly less. significant
world is uncertain-

Governments need to
take into account lower
probability but extreme
levels of warming. Indeed,
these may constitute
the majority of the
probability-weighted
costs of climate change.

ty). On the medium-high emissions approximately linear relationship

scenario, the chance could be around between the total amount of carbon
10%.30 These figures are of course emitted and the resulting tempera-
speculative, but they do provide some ture increase.34 The majority of the
reason to believe that the probabili- carbon still underground is in the
ty of catastrophic climate change is form of coal.35 If we were to avoid
non-negligible, unless strong action is burning the remaining global coal
taken on GHG emissions. reserves we would likely avoid cata-
strophic levels of climate change.36
MAIN ACTIONS AVAILABLE States could commit to building no
TO LIMIT THE RISK new coal-fired power stations without
There are three main ways to reduce carbon capture and sequestration to
the risks from climate change: adap- limit the fraction of the global coal re-
tation to climate change, abatement serves which are burned.
of GHG emissions, and geo-engineer- Geo-engineering – the deliberate
ing. It is likely to be very costly if not use of technology to alter the world’s
impossible to to avoid many of the climate – in the form of Carbon Di-
impacts of non-catastrophic climate oxide Removal (CDR) or Solar Radia-
change by adaptation alone,31 and tion Management (SRM), could also
adapting to warming of 6°C or more is help to reduce the risk of catastroph-
likely to be even more costly and dif- ic climate change, as a complement
ficult. to GHG abatement. CDR techniques,
Turning to abatement, most econ- such as carbon sequestration or iron
omists agree that the best way to re- fertilisation of the oceans, would re-
duce GHG emissions is to impose a move CO2 from the atmosphere and
carbon tax or a cap and trade sys- thereby help us move towards net
tem.32 At present, the global price of neutral or net negative emissions.37
carbon is approximately $4 per tonne, SRM techniques, such as the injection
whereas according to Wagner and of sulphates into the stratosphere,
Weitzman, to fully price in the exter- cause global cooling by reflecting
nalities from catastrophic climate sunlight. The benefits and risks of
change, a price of at least $40 could geo-engineering are discussed in
be required.33 more detail in section 2.5.38
Scientists have demonstrated an

FIGURE 2.1. THE CHANCE OF EXTREME CLIMATE CHANGE
The probability of warming of 6°C for different atmospheric concentrations of greenhouse gases.39
Probability of warming >6°C

Percent
20
15
10
0
400 450 500 550 600 650 700 750 800
CO2e concentration
Parts per million
2.2. Nuclear war

T
he invention of nuclear weap- Russia has 4,300, with yields ranging
ons gave humanity the tech- from 50 to 800 kilotons.42 For compar-
nical capacity to cause dev- ison, the Little Boy bomb dropped on
astation on a hitherto unseen scale. Hiroshima had a 15 kiloton yield.44
Although there have been no nucle- The damage from nuclear war can
ar attacks since the Second World be divided into two main categories.
War, we have come close to inadvert- Firstly, there is the damage from the
ent and intentional nuclear war on a blast, fire and radiation. A 1979 re-
number of occasions. A worrying pos- port by the U.S. Office of Technolo-
sible consequence of nuclear war is gy Assessment estimated that, in an
a nuclear winter with global climat- all-out war between the US and Rus-
ic implications. While the chance of sia involving thousands of nuclear
nuclear war may appear to have de- weapons, 35-77% of the U.S. popula-
clined since the end of the Cold War, tion and 20-40% of the Soviet popula-
tensions between nuclear states per- tion would die within the first 30 days
sist. Reducing the likelihood of nu- of the attack, and millions would die
clear war is a serious ongoing global globally in the following years due to
challenge. the radioactive dust cloud.45 The pro-
Despite these risks, some have ar- portionate death toll today is likely to
gued that the deterrence effects of be lower because nuclear arsenals at
the nuclear bomb have been, and will that time were five times larger.46
be, very valuable in terms of ensur- The second category of damage is
ing global peace.40 How to make the a possible nuclear winter, which is
trade-off between lower likelihood caused by the burning of cities, in-
of nuclear or conventional conflict is dustrial facilities and other flamma-
unclear. ble materials, sending smoke into the
atmosphere. Scientists have applied
POTENTIAL IMPACT modern climate models to predict the
OF THE CATASTROPHE scale of nuclear winter, though these
The scale of the damage done by nu- predictions are uncertain. According
clear war obviously depends on the to one model, an all-out exchange of
scale of the nuclear war itself. In 2014, 4,000 nuclear weapons would release
there were 9,920 stockpiled nuclear 150 teragrams of smoke, leading to a
warheads in the world (down from a 8°C fall in global temperature.47 Due
peak of 65,000 in 1986).41 The United to the fall in temperature and the loss
States has 4,760, with yields ranging of sunlight and growing food would
from 5 kilotons to 455 kilotons;42 and be virtually impossible for 4-5 years,
36
creating an unprecedented famine. between one out of three and even”.5!

According to some models, even a Although tensions have decreased
smaller scale regional war between since the end of the Cold War, in
India and Pakistan involving fifty 15 the wake of the crisis in Ukraine the
kiloton weapons would cause global chance of confrontation has risen,52
temperatures to fall by around 1.25°C and the geopolitical situation could
in the first year. Some have suggested become more unstable over the next
that this could disrupt agriculture so few decades.
significantly that one billion people According to many experts, the
would be at risk of starvation, though most likely intentional nuclear war
this has been criticised as overly pes- is between India and Pakistan. In-
simistic.48 dia and Pakistan have had numerous
wars in the past and there have been
LIKELIHOOD OF THE CATASTROPHE various terrorist attacks against India
It is very difficult to estimate the by Pakistani groups.53 Pakistan has
probability of a nuclear war in the pledged to meet any Indian attack on
next century. However, the chance its territory with a retaliatory nuclear
may be too high to ignore. Over the strike.54
course of the nuclear age, we have
come close to inadvertent nuclear war MAIN ACTIONS AVAILABLE
on numerous occasions. For exam- TO LIMIT THE RISK
ple, in 1995 Russian systems mistook There is some disagreement about
a Norwegian weather rocket for a po- how to reduce the risk of nuclear war.
tential nuclear attack. Russian Pres- One policy option favoured by many
ident Boris Yeltsin retrieved launch is continued reduction of global nu-
codes and had the nuclear suitcase clear arsenals.55 There has been a
open in front of him. Thankfully, Rus- great deal of progress on this front:
sian leaders decided the incident was in the 1960s 23 countries had weap-
a false alarm.49 In a 2013 paper Barrett ons or were pursuing programmes,
et al estimated that the 90% confi- whereas today only nine countries
dence interval of the annual probabil- have weapons.56 However, reduction
ity of accidental nuclear war between of nuclear arsenals does not guaran-
the US and Russia is from 0.001% to tee safety.57
7%.50 This covers a significant range, Weapons systems could also be al-
but a substantial portion of the possi- tered to reduce the risk of acciden-
bilities are uncomfortably high. tal and intentional war. For instance,
We have also come close to inten- decision times could be increased
tional nuclear war on many occa- by each side locating their weapons
sions. President Kennedy said that further from each other’s borders.58
the chance of nuclear war during the More broadly, improved internation-
Cuban Missile Crisis was “somewhere al relations would help to decrease

FIGURE 2.2.1. NUCLEAR ARSENALS OF THE US, RUSSIA,
AND THE WORLD FROM 1945 UNTIL TODAY 59
Number of nuclear warheads

Thousands
80
70 Russia
US
60
Global
50
40
30
20
10
0
1945 1950 1960 1970 1980 1990 2000 2010 2014
Year
NUCLEAR NEAR MISSES

• In 1973, when Israel encircled the Egyptian Third Army, the Soviets
threatened to intervene, leading to implied nuclear threats.
• In September 1983, a Soviet early warning satellite showed that the
United States had launched five land-based missiles at the Soviet
Union. The Soviet officer on duty, Stanislav Petrov, had only minutes
to decide whether or not the satellite data were a false alarm. Since
the satellite was found to be operating properly, following procedures
would have led him to report an incoming attack. Going partly on gut
instinct and believing the United States was unlikely to fire only five
missiles, he told his commanders that it was a false alarm before he
knew that to be true. Later investigations revealed that reflection of
the sun on the tops of clouds had fooled the satellite into thinking it
was detecting missile launches.
• The Able Archer incident of November 1983 was, in the words of
former US Secretary of Defense Robert Gates, “one of the potential-
ly most dangerous episodes of the Cold War”. With talk of fighting
and winning a nuclear war emanating from Washington, the Soviets
reasoned that the West would mask preparations for a nuclear attack
as a military exercise. The Able Archer exercise simulated the coordi-
nated release of all NATO nuclear weapons. In response, the Soviets
readied their nuclear forces and placed air units in East Germany and
Poland on alert.
There have been numerous other examples of nuclear near misses.
the likelihood of international nu- could work to ensure that terrorists

clear conflict. Research into nucle- cannot acquire nuclear weapons by
ar winter would improve our knowl- theft or on illegal markets. Glob-
edge and allow nuclear states to al enforceable standards for nuclear
change their strategy accordingly. weapon security would help achieve
Finally, state and non-state actors this goal.60

FIGURE 2.2.2. GLOBAL TEMPERATURE ANOMALY FROM NUCLEAR WINTER
Global average surface air temperature change from a release of 5 Teragram, 50 Teragram, and 150 Teragram
of particular matter in the context of the climate change of the past 125 years.61
Temperature anomaly
°C +5 Tg +50 Tg +150 Tg
-1
-2
-3
-4
-5
-6
-7
-8
1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010
Year
2.3. Natural pandemics

U
ntil the discovery of nucle- H5N1 would have to be easily transmis-
ar weapons, pandemics were sible between humans, which is not
the most important global currently the case, though occasionally
catastrophic risks, and it is plausible humans become infected through ani-
they still are. Plague, HIV, smallpox mal vectors. Influenza pandemics have
and other diseases have killed mil- previously infected about 24-38% of the
lions upon millions of people. There world population.65 The case fatality of
remains a serious chance that a pan- a novel strain of H5N1 is unpredicta-
demic could kill a huge portion of the ble, but estimates for the H5N1 case fa-
world population, with pandemic in- tality rate until today vary particularly
fluenza the most serious threat. A va- widely – from 1% to 60%.66 A very rough
riety of measures, including improv- estimate of the death toll of a H5N1
ing international coordination and pandemic can be found by multiplying
data sharing, and increasing the pro- the usual pandemic attack rate (24% to
duction of drugs and vaccines, would 38%) by the global population (rough-
help to reduce this risk. ly 7 billion) times the case-fatality ratio
(1% to 60%). This produces an estimat-
POTENTIAL IMPACT ed death toll of between 16.8 million
OF THE CATASTROPHE and 1.7 billion fatalities. The develop-
The Black Death plague in the 14th ing world would probably bear an over-
century killed between 11% and 17% whelming part of this burden.67
of the global population over the
course of a decade.62 In the most dev- LIKELIHOOD OF THE CATASTROPHE
astating pandemic of the last 100 Influenza pandemics occur relatively
years, the 1918 Spanish influenza frequently: there have been ten in the
outbreak, 50-100 million people lost last 300 years, though none of these
their lives – between 2.5 and 5% of have killed more than 5% of the world
the global population.63 Both of these population.68 It is difficult to know
pandemics occurred before the age where and when the next natural pan-
of modern medicine, so it is unlike- demic will occur. Globalisation and
ly these diseases would have simi- increased interaction between hu-
lar impact today, although our more mans and animals increase the risk of
connected society may increase the pandemics, but improved health and
spread of pandemics. sanitation would lessen their impact.
Currently, H5N1 avian influenza is According to the UK National Risk
thought to be the greatest pandemic Register there is between a 1 in 20 and
threat.64 To develop into a pandemic a 1 in 2 chance of a pandemic killing up
42
to 1% of the UK population in the next bodies such as the WHO can stockpile
five years. Such a pandemic would go drugs and vaccines to counter disease
on to have global effects. Global cata- outbreaks. However, stockpiles may
strophic risk-level pandemics (killing only be effective for some but not all
more than 10% of the global popula- pandemic pathogens.72 Thirdly, since
tion) are, however, further down the developing countries will probably
tail of the probability distribution. Re- face the highest burden from future
sults from expert surveys have put a pandemics, continued improvements
15% chance of an H5N1 pandemic over in developing world health systems
a three year period.69 in accordance with the WHO’s Inter-
national Health Regulations, and im-
MAIN ACTIONS AVAILABLE provements in the global distribution
TO LIMIT THE RISK of drugs and vaccines, would limit the
There are numerous ways to limit the risk from pandemics.73 Fourthly, dis-
risk from pandemics.70 Firstly, pharma- ease surveillance and response sys-
ceutical companies could focus atten- tems could be improved so that nov-
tion on developing vaccines and drugs el threats in both animal and human
for very damaging pandemics, and the populations are detected and respond-
international community can remove ed to quickly.74 Rapid dissemination
barriers to development with measures of relevant data between countries is
such as the establishment of a glob- also essential for effective outbreak
al vaccine development fund.71 Sec- response, because of the exponential
ondly, governments and international spread of infection in an outbreak.75

FIGURE 2.3.1. HISTORIC PLAGUES AND PANDEMICS 76
PANDEMIC DATE(S) LETHAL IMPACT
Plague of Justinian AD 541-542 25-33m (13-17% of the world population)
Black Death 14th Century 50-75m (11-17% of world population)
Smallpox 1520-1527 200,000 deaths within the Aztec population

(75% of population in some areas)
Spanish influenza 1918 - 1919 50 - 100m (2.5-5% of population)
Smallpox 20th century 300m over the course of the 20th Century
HIV/AIDS 1981 - present 34m

2.4. Exogenous risks

E
xogenous risks are those which est mass extinction in history and re-
arise independent of human duced the human ancestor popula-
activity (whereas even natu- tion from around 100,000 to around
ral epidemics are spread by humans). 4,000 people for approximately
They include such possibilities as 20,000 years, though this is contro-
super-volcanic eruptions and large versial.81 According to a report by the
asteroid and comet impacts, which Geological Society of London:
are believed by some to have caused “A layer of ash estimated at 15 cen-
mass extinctions.77 The likelihood of timetres thick fell over the entire
exogenous risks is better understood Indian sub-continent, with similar
than that of many other global cata- amounts over much of SE Asia. Just
strophic risks because the underlying one centimetre of ash is enough to
dynamics have been unchanged for devastate agricultural activity, at least
a very long time. The historical evi- when it falls in the growing season.
dence suggests that exogenous global An eruption of this size would have
catastrophic risks cannot be too fre- catastrophic consequences. Many
quent, and may therefore be much millions of lives throughout most
less likely than some of the anthropo- of Asia would be threatened if Toba
genic risks.78 erupted today.”82
A. SUPER-VOLCANOES LIKELIHOOD OF THE CATASTROPHE

Super-volcanoes are volcanoes ca- In order to assess the likelihood of
pable of producing at least 1012 m3 super-volcano eruptions, we have
bulk volume of fragmental material.79 to rely on a relatively limited set of
Some experts believe that the erup- observations of past super-volcan-
tion of the Toba super-volcano in ic eruptions, which makes any esti-
Indonesia around 70,000 years ago mates very uncertain. Existing data
brought humanity to the brink of ex- suggest that there will be a super-vol-
tinction, though there is significant canic eruption roughly every 30,000-
disagreement about this. 50,000 years on average.83
POTENTIAL IMPACT OF THE RISK MAIN ACTIONS AVAILABLE

The Toba eruption ejected large TO LIMIT THE RISK
amounts of dust and sulphates into At present, humanity lacks the tech-
the Earth’s atmosphere, which caused nical capacity to prevent volcanic
global cooling of 3-5°C for sever- eruptions. Consequently, improving
al years and led to enormous loss of resilience to catastrophe is the main
plant and animal life.80 Some have way to limit the risk from super-vol-
argued that Toba caused the great- canoes. We discuss this in more detail

in Chapter 3. Increased investment being released into the atmosphere.86

and research might also improve our The damage that would be caused by
ability to predict volcanic eruptions, a multi-kilometre asteroid or comet
which would in turn improve prepar- impact is only modestly understood
edness.84 at present.87 An asteroid winter would
undermine agriculture at least for an
B. ASTEROIDS entire growing season and so could
AND COMETS cause the deaths of billions of peo-
Around 66 million years ago, an as- ple.88 According to a comprehensive
teroid of around 10km in diameter report by the US National Research
struck Chicxulub in Mexico. This im- Council, “above the conservatively
pact probably caused one of the three assumed global catastrophe thresh-
largest mass extinctions in history old from a 1.5km-diameter impac-
and may have abruptly ended the age tor, the number of fatalities ramps up
of the dinosaurs.85 Today, an impact from 10% of the world’s population to
by a Near Earth Object (NEO) – an as- the entire population for impactors
teroid or comet – larger than 1.5km in above 10km in diameter”.89
diameter would kill millions largely
by causing global cooling and agricul- LIKELIHOOD OF THE CATASTROPHE
tural disruption. The likelihood of an NEOs are a comparatively well-un-
NEO impact is reasonably well-understood global catastrophic risk. NA-
derstood and important steps have SA’s Spaceguard system has mapped
been taken to monitor the risk they more than 90% of asteroids with a di-
pose. ameter of more than 1km.90 Reinhardt
et al argue that the total probabili-
POTENTIAL IMPACT ty in a 100-year period of an aster-
OF THE RISK oid encounter with the Earth that
Contemporary studies of aster- might cause a globally signifi-
1
oids and comets conservative- cant effect is approximately
ly assume that all objects 1 in 1,250.91
greater than 1.5km in di- One can calculate the ex-
ameter would be capa- pected cost of NEOs by
ble of causing damage in multiplying the like-
1,250
on a global scale, via ly deaths per event
firestorms generated by the frequency of
by impact debris events of a cer-
and a so-called tain size. As of
approximate risk of an asteroid
asteroid win- encounter with the Earth that might 2010, the annu-
ter caused by cause a globally significant effect al expected
dust and in a 100-year period (probabili-
sulphates ty-weight-

FIGURE 2.4. RISK BEFORE AND AFTER THE SPACEGUARD SURVEY 92
Statistical fatalities per year
250
Intrinsic risk before survey
200 Residual risk after 10 years of surveying
150
100
50
0
0,01 0,1 1 10
Diameter of impactor
Kilometres
“The likelihood of an NEO

impact is reasonably well-
understood and important
steps have been taken to
monitor the risk they pose.”
ed) cost of all NEOs is currently 91 fa- detection of asteroids would enable
talities.93 humanity to understand the risks it
Once ongoing asteroid surveys faces and to take appropriate coun-
are completed, long-period comets ter-measures. NASA’s ongoing Space-
– comets which take more than 200 guard Survey has so far reduced the
years to orbit the Sun – may domi- expected cost a large asteroid impact
nate the remaining unknown impact by more than an order of magnitude.95
threat from NEOs.94 Secondly, technological research could
help us to find ways to deflect threat-
MAIN ACTIONS AVAILABLE ening NEOs.96 Thirdly, improvements
TO LIMIT THE RISK in resilience could be made to ensure
There are three main ways to limit the human survival in the event of a large
risk from asteroids. Firstly, improved asteroid or comet impact.
50
2.5. Emerging risks

I
n the coming decades, emerging come increasingly streamlined, the
technologies will provide major tools enabling such risks may become
benefits to society, but they may widely accessible.99 Previously benign
also create significant and unprece- digital information, such as the wide-
dented risks. Certain types of bio- ly available online genetic data for
technology, if more widely accessi- smallpox, will become more hazard-
ble, could give terrorist groups the ous.
access to pathogens as dangerous as
smallpox. Geoengineering technol- POTENTIAL IMPACT OF THE RISK
ogies could give single countries the Evolutionary pressure generally con-
power to unilaterally alter the earth’s strains the lethality of pathogens, as
climate. Finally, artificial general in- most highly lethal pathogens fail to
telligence could, if developed, leave spread far before killing their host.
human control. There is thus some evidence for an
Technological risks could emerge inverse relationship between a path-
very quickly and give certain groups ogen’s lethality and transmissibility,
large and perhaps unprecedented thereby limiting the damage from a
destructive power. Moreover, some naturally occurring pandemic. Bio-
emerging technologies may be par- technology has the potential to break
ticularly difficult to control because this correlation, allowing organisms
barriers to access the technology may with extraordinarily high lethality
be quite low. and transmissibility. In 2001 Austral-
ian researchers accidentally created
A. ENGINEERED a highly lethal and vaccine resistant
PANDEMIC form of mousepox.100 Similar tech-
The past decades have seen rapid ad- niques could potentially be applied
vances in biotechnology, in part due to smallpox.101 Two recent controver-
to the falling costs of gene sequenc- sial papers have shown how to create
ing and synthesis.97 Improvements in a version of H5N1 which is potential-
ease-of-use of certain specific kinds ly transmissible between humans.102
of biotechnology bring increased con- Basic calculations based on H5N1 pa-
cerns about bioterrorism. Gene syn- rameters suggest that a single release
thesisers have the capacity to turn of these modified viruses could cause
digital sequence data into physical hundreds of millions of casualties.103
genetic sequences, enabling individ- Engineered pathogens with danger-
uals to create viruses from digital files ous features could be released acci-
(as was done with the 1918 Spanish dentally from a lab or intentionally
Flu virus).98 Should gene synthesis be- by states or terrorist groups. The legal

theorist Richard Posner has described As biotechnology develops, the

a scenario in which terrorists synthe- level of expertise required to create
sise a version of smallpox that is in- dangerous pathogens will fall. In the
curable, immune to vaccine and kills longer term, if biotechnology ma-
all its victims, which is then released tures sufficiently and gene synthesis
in a large city via aerosol.104 Although is not well-regulated, states and small
such scenarios are not currently feasi- groups will find it increasingly easy to
ble, eminent figures including George synthesise and alter dangerous path-
Church,105 Nathan Myhrvold,106 and ogens.112 This poses a serious risk of a
Martin Rees107 have all argued that bi- global catastrophe.
otechnology poses serious risks.
MAIN ACTIONS AVAILABLE
LIKELIHOOD OF THE CATASTROPHE TO LIMIT THE RISK
There is a real possibility that a dan- Many of the measures which limit the
gerous engineered pathogen could risk from natural pandemics would
be released by accident. The H1N1 also limit the risk from synthesised or
influenza strain, responsible for sig- modified pathogens. However, other
nificant morbidity and mortality measures would help specifically with
around the world from 1977 to 2009, is biotechnology risks. Fostering a cul-
thought to have originated from a lab- ture of safety in the relevant research
oratory accident.108 As of 2012, there and technical communities is proba-
were at the very least 42 laboratories bly very valuable. Researchers could
engaged in live research on potential be encouraged to take significant pre-
pandemic pathogens, though this will cautions with this research and to
have been affected by the US mora- avoid disseminating research when
torium on ‘gain of function’ research doing so brings major risks. With re-
in 2014.109 Lipsitch and Inglesby esti- gard to regulation, a licensing regime
mate that “work with a novel, trans- for DNA synthesis could be one first
missible form of influenza virus car- step,113 while mandatory liability in-
ries a risk of between 0.01% and 0.1% surance for dual-use research - which
per laboratory-year of creating a pan- helps medical progress but could be
demic... or between 0.05% and 0.6% used maliciously - would ensure that
per full-time worker-year”.110 These researchers have incentives to main-
estimates are illustrative of the prob- tain high levels of laboratory biosafe-
ability that existing ‘gain of function’ ty.114 Governments could also require
influenza research would produce a journal editors to consider whether
pandemic, though there is significant publication of research could lead to
uncertainty about them.111 adverse outcomes.115

FIGURE 2.5.1. COST PER GENOME 116
Cost of sequencing a human genome

Dollars (logarithmic scale)
100M
10M Moore’s law equivalent
1M
100K
10K
1K
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Year
B. ARTIFICIAL influence over the future. It has been

INTELLIGENCE argued that if these goals were not
Prominent figures uch as Stuart Rus- aligned with human values, the conse-
sell, Professor of Computer Science at quences could be truly catastrophic.121
Berkeley, Peter Norvig, Director of Re- It is difficult not only to specify human
search at Google, and Nick Bostrom, values in a robust, machine-interpret-
author of Superintelligence, recent- able way, but also to agree on human
ly signed a letter warning of the risks values in the first place. Moreover, if
posed by Artificially Intelligent (AI) even if the values of very powerful ar-
systems.117 Past experience demon- tificially intelligent systems can be
strates that AI systems can go from aligned with their creators’, such sys-
significantly subhuman to superhu- tems might destabilise the geopolitical
man relatively quickly in narrow do- balance in a destructive way.
mains, such as recently in the game
Go. In the coming decades, we may LIKELIHOOD OF THE CATASTROPHE
create AI systems which surpass hu- It is widely accepted that we will be
mans in all relevant domains. If this able to create AI systems that are ca-
were to happen, the effects would be pable of performing most tasks as
uniquely transformative. However, well as a human at some point (‘hu-
AI also has huge potential benefits, man-level AI’). Experts disagree
as well as risks.118 It could, for exam- about when this will occur. According
ple, greatly reduce the cost of many to the median surveyed expert, there
goods, and allow us to solve other is a roughly 50% chance of such AI
global problems. by 2050.122 The median surveyed ex-
pert believes that there is at least a 5%
POTENTIAL IMPACT OF THE RISK chance of superintelligent AI within
Some risks of AI likely fall short of two years after human-level AI, and
posing a global catastrophic risk. a 50% chance within thirty years.123
Widespread automation could cause Assuming that human-level AI is de-
significant economic and social dis- veloped, its long-term social impact
ruption, which has only a relative- is unclear. According to the medi-
ly small chance of leading deaths at an surveyed expert, there is around a
the scale of other global catastrophic 7% chance that it would be ‘extreme-
risks.119 ly bad’.124 The extreme uncertainty
In the longer term, AI may enable surrounding these estimates should,
important new capabilities, perhaps however, be strongly emphasised.
extremely quickly if it turns out that we
can automate AI development.120 If we MAIN ACTIONS AVAILABLE
have powerful and generalisable auto- TO LIMIT THE RISK
mated systems, the goals they are pro- Different challenges from AI systems
grammed with may exert significant will require different responses, and
56
in many cases it will be more appro- iron or urea could pose major risks to
priate to respond closer to the time marine ecosystems.125 However, most
the technologies mature. However, forms of SRM are thought to carry
there is a strong case for early work much greater risks than most forms
preparing for possibly rapid changes of CDR,126 and worries about civilisa-
brought about by AI systems because tion-threatening consequences have
a reactive response won’t succeed generally focused on SRM (and in
when the change is very rapid. In par- particular on currently the leading
ticular, important research could be form of SRM: the injection of sulphate
done on how to give AI systems de- particles into the stratosphere).127 The
sirable goals. Foresight work could be remainder of this section will there-
done to better understand which oth- fore focus on SRM only.
er issues require advance preparation, SRM is the only known technique
or where there may be an eventual for quickly stopping (or even revers-
role for other responses. Work could ing) the rise in global temperatures.
also be done to encourage under- This means that it could be used as
standing of the risks among AI devel- a complement to GHG reduction, to
opers, especially around automated manage temperatures while the world
AI system development, which might phases out fossil fuels. Some have
enable very fast transitions. However, proposed that SRM could provide in-
the benefits of AI could be very great surance against a ‘climate emergen-
so it is important not to unnecessarily cy’, such as unexpected abrupt and
impede AI development. extreme warming.128 Thus, on some
scenarios, failing to use SRM could
C. GEOENGINEERING constitute a global catastrophic risk.
As mentioned previously in this chap- Moreover, SRM has the potential to
ter, geo-engineering – in the form of reduce the costs of warming at very
Carbon Dioxide Removal (CDR) or low cost. Some estimate that the an-
Solar Radiation Management (SRM) nual cost of stratospheric aerosols
– could help to limit the risks of cata- could be less than $10 billion per year,
strophic climate change. CDR tech- which is orders of magnitude less
niques, such as ocean fertilisation or than the costs of climate change miti-
carbon sequestration, remove Car- gation strategies.129
bon Dioxide from the atmosphere,
whereas SRM techniques, such as POTENTIAL IMPACT OF THE RISK
cloud brightening or the injection of Four main arguments have been giv-
sulphates or other particles into the en for the view that SRM brings global
stratosphere, reflect the sun’s light catastrophic risks. Firstly, while ex-
and heat back into space. Certain isting models suggest that SRM could
forms of CDR could carry major risks. reduce the catastrophic effects of cli-
For example, ocean fertilisation using mate change and will not bring their

2.5.2. GLOBAL TEMPERATURE ANOMALY FROM GEO-ENGINEERING
FOLLOWED BY TERMINATION
Global average surface air temperature change from business as usual emissions, injection of 3 mega-
tons/annum (Mt/a) of SO2 in the Arctic, 5 Mt/a of SO2 in the tropics, and 10 Mt/a SO2 in the tropics.130
+Anthro Forcing +3 Mt/a Arctic

Temperature anomaly
°C +5 Mt/a Tropical +10 Mt/a Tropical
1.4
Geoengineering Geoengineering
1.3 starts ends
1.2
1.1
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
-0.0
-0.1
-0.2
-0.3
-0.4
-0.5
1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 2030 2040
Year
own catastrophic impacts, it may nev- it could discourage GHG reduction

ertheless bring currently unknown efforts.136 All major reports studying
risks, particularly through impacts SRM have concluded that it does not
on global precipitation.131 The climate present an alternative to emissions re-
system is imperfectly understood and ductions, as it only masks the effects
the deployment of a novel technol- of GHGs and does nothing to coun-
ogy with global effects is inherent- ter ocean acidification.137 Therefore, it
ly risky. Whether the use of SRM is could be problematic if geo-engineer-
more risky than allowing the planet to ing drew a disproportionate amount
warm is one of the key questions for of policy attention. However, it is im-
anyone considering using SRM. Sec- portant to bear in mind that there are
ondly, because SRM is so cheap, in- countervailing reasons in favour of
dividual states could feasibly deploy SRM research and advocacy.
it and unilaterally bring about global
climatic impacts. Moreover, individ- LIKELIHOOD OF THE CATASTROPHE
ual states might also have the incen- The probability that there will be a
tive to do this because they could be global catastrophe brought about by
particularly badly affected by climate SRM depends on many factors, in-
change.132 Individual states acting cluding the timing, speed and severi-
alone may be less likely to properly ty of global warming; the state of SRM
take into account the interests of oth- technology and our knowledge of the
er states and may be concerned about climate system; the response of the
catastrophic consequences in other climate system to SRM; the form of
regions. SRM deployed and how it is used; and
Thirdly, sudden termination of SRM how well the world does at organising
would lead to rapid and severe global governance of SRM. There is obvious-
warming.133 There are some reasons ly very large uncertainty about all of
to think that an SRM system could be these factors.
very resilient against external shocks We argued above that unless strong
and termination. If the current de- action is taken soon, there is a sizea-
ployer were to suddenly stop SRM ble chance of catastrophic warming.
for some reason, every other country Catastrophic warming would create
would have strong incentives to re- very strong incentives to use SRM.
sume SRM.134 Thus, sudden termina- Therefore, unless strong GHG reduc-
tion might only be likely in the event tion action is taken soon, the chance
of a severe global catastrophe which that SRM is used will increase. The
undermines the capacity of all coun- greatest chance of catastrophe prob-
tries to use SRM.135 ably comes from poorly planned and
Finally, research into geo-engineer- governed the use of SRM, perhaps
ing or advocacy for geo-engineering by an individual state or small group
could present a ‘moral hazard’ in that of states. Since most of the risks of a

well-governed and well-planned form tant for policymakers to be aware of

of SRM are unknown, it is very dif- these issues.
ficult to say how likely that form of
SRM is to produce a global catastro- THE CHALLENGE OF
phe. EMERGING RISKS
Finally, it is unclear the impact re- The catastrophic risks from emerging
search into, advocacy for, or prepara- technology are particularly challeng-
tion for, geo-engineering would have ing. Firstly, we have no track record
on global willingness to cut green- of dealing with these emerging tech-
house gas emissions. Therefore, it is nological risks and, as we discuss in
not clear to what extent geo-engineer- chapter 4, existing national and inter-
ing is a moral hazard. national institutions are not designed
to deal with them. It is therefore less
MAIN ACTIONS AVAILABLE likely that our eventual responses will
TO LIMIT THE RISK be effective.
The main way to reduce the risks Secondly, these technologies might,
from SRM would be to take strong like nuclear weapons, reach maturi-
action on GHGs to reduce the ex- ty more quickly than expected. If so,
pected costs of climate change and the nature of the problem would only
thereby reduce the incentives to become fully apparent over a short
geo-engineer. Further research into time-frame and it would be difficult
the different kinds of SRM and into to make an appropriate response.
the response of the climate to them Thirdly, some of these technologies
would reduce the unknown risks of could be harder to control than nucle-
SRM. Finally, working to develop ar weapons. Nuclear weapons require
geo-engineering governance through the rare and controllable resources
climate treaties and through global of uranium-235 or plutonium-239. In
institutions might limit the risks of contrast, if some of these technolo-
unilateral SRM.138 However, both of gies reach full maturity, they could be
these actions might also have moral accessible to small countries or even
hazards. We cannot settle the moral terrorist groups.
hazard debate here, but it is impor-

In the coming decades,
emerging technologies
will provide major
benefits to society,
but they may also create
significant and
unprecedented risks.

2.6. Other risks and

unknown risks
T
here is a broad spectrum of unforeseeable a few decades before
possible global catastroph- they became apparent. Forty years
ic risks, from very salient and before the discovery of the nuclear
well-understood threats to those bomb, few could have predicted that
which are extremely low probabili- nuclear weapons would come to be
ty and highly speculative. In compil- one of the leading global catastroph-
ing this report, we had to make deci- ic risks. Immediately after the Second
sions about which risks to investigate World War, few could have known
carefully, and which to ignore. Our that catastrophic climate change, bio-
guiding principle was to include risks technology, and artificial intelligence
which could clearly cause a global would come to pose such a significant
catastrophe in our definition if they threat.
were to happen, which could be spec- These risks emerged due to rapid
ulative but not too speculative, and economic and technological devel-
which were not too low in probability. opment, which looks set to contin-
There are several risks we didn’t ue apace in the coming century. That
include for these reasons. For exam- might create a number of new risks.
ple, gamma ray bursts are fairly well Therefore, it seems likely that some
understood and would be catastroph- future global catastrophic risks are at
ic, but have a very low annual likeli- present unknown.
hood.139 A global totalitarian state is Detailed planning for unknown
a possible scenario, but it is unclear risks is of course impossible, but steps
what proportion of the scenarios can be taken to improve our prepar-
would meet the threshold for a global edness. Bodies specifically tasked
catastrophe and how likely it would with horizon scanning and the dis-
be to arise given the current geopoliti- covery of new global catastroph-
cal environment.140 Conventional and ic risks would give the international
chemical warfare are real threats, but community more time to craft an ap-
quite unlikely to reach the scale of a propriate response. Measures can also
global catastrophe. be taken to improve general societal
It is important to remember, howev- resilience to catastrophe. These are
er, that nearly all of the most threat- discussed in chapter 5.
ening global catastrophic risks were

2.7. Our assessment

of the risks
W
e’ve given qualitative de- two dimensions are linked, but they
scriptions of each of these can come apart, as in the case of cli-
risks. It is useful to con- mate change whose main harms are
sider comparisons between them as not likely to be felt soon, but which
well. Unfortunately there is a great demands action today. We think that
deal of uncertainty around the level this split should be explicit.
of many of the risks, and around how To assess current risk, we consid-
much can usefully be done. Nonethe- ered the likelihood over the next five
less, some attempt must be made in years. Our upper band consists of
order to prioritise the world’s scarce events which we consider to be dis-
resources, and there are some areas tinct possibilities. The lower band
where we can be more confident than consists of risks which appear low in
others. absolute likelihood, either because
In this section, therefore, we make the base rate is low enough or because
some comparative assessments of the five years is too short a timescale for
risks. Because of the high level of un- the risk to develop. Because of the
certainty we categorise the risks into large scale of the potential catastro-
broad bands rather than attempt pre- phes, even risks in the lower category
cise assessment. The categorisation may still be significant over this time
remains our subjective judgement, period.
and it is possible that some risks It is even harder to assess how large
should be in different bands. But we the response should be. We consid-
believe that it is better to offer even ered how much risk was posed, both
such an imperfect assessment than at present and in years to come. We
nothing. also considered how good the oppor-
tunities to reduce the risk appear, and
A. OUR METHOD for emerging risks how much they
We have assessed the risks on two dif- benefit from early responses. Our up-
ferent dimensions. First, the current per band indicates risks where we
likelihood: how likely is it to materi- think a significant global response is
alise in the next few years? Second, likely appropriate, perhaps involv-
how much work should be given to re- ing thousands of people, or billions of
ducing the risk in the next few years dollars. There is a wide range with-
(according to our judgement)? These in this band, and we do not think all

Bodies specifically
tasked with horizon
scanning and the
discovery of new global
catastrophic risks would
give the international
community more time
to craft an appropriate
response.

of the risks there should receive the have decreased since the Cold War,
same amount of attention. For risks nuclear war appears to remain a real
in the lower band, we still think re- possibility. We have corresponding-
sponding is appropriate, but perhaps ly placed it in the upper category for
at a smaller scale for the time being. likelihood. It is the only catastrophe
Our assessment is summarised in that is definitely within the reach and
this diagram: control of humanity today, and we
think it is therefore appropriate that
B. HOW WE MADE THESE the response be high.
ASSESSMENTS ABOUT Asteroid impacts and supervolcan-
PARTICULAR RISKS ic eruptions are caused exogenous-
Catastrophic climate change would ly to human actions. This means we
likely have effects decades out, and have better estimates of their rates,
we consider the likelihood in the next through knowledge of historical in-
five years to be small unless the sci- cidents. A supervolcanic eruption
entific community has significant- (which may or may not cause a glob-
ly mis-modelled climate dynamics. al catastrophe) is estimated to oc-
Nonetheless, because of cumulative cur very approximately every 30,000
nature of greenhouse gas emissions, years. An asteroid impact large
and the relatively well understood dy- enough to cause a global catastrophe
namics, we believe a large response is is estimated every 120,000 years. We
justified. are therefore reasonably confident
Natural pandemics have been re- that they belong to the lower category
sponsible for past global catastro- of current risk. Because of this rela-
phes, and are seen high on lists of tively low risk, we have also put them
national risk assessments. We are in the low category for attention.
not confident assigning a par- We are confident that this is justi-
ticular probability to a pandem- fied for supervolcanic eruptions,
1
ic, but we do think it is among where there are few clear ac-
the most likely risks in the tions to reduce the risk. As-
next five years. Internation- teroids are exceptional in
al work in reducing the that we have relatively
risk from pandemics every well-understood ways
30,000
seems both important to reduce the risk, and
and effective, and this could mean that
we think a large re- the attention cat-
sponse is appro- years egory should be
priate. is the estimated occurrence of a higher.
Although supervolcanic eruption (which may or Engineered
the likeli- may not cause a global catastrophe). pandem-
hood may ics are an

FIGURE 2.7. OUR ASSESSMENT OF GLOBAL CATASTROPHIC RISKS
Natural pandemic
Unknown risks Nuclear war

over next 5 years
Higher likelihood
Engineered pandemic
Catastrophic
Asteroid impact climate change
Supervolcanic Catastrophic
eruption disruption from AI
over next 5 years
Lower likelihood
Failure of geo-engineering
Low High APPROPRIATE LEVEL

OF ATTENTION
emerging risk in that future tech- the speed of technological progress.

nology will make them more plausi- In the longer term the risks could be
ble. But even existing experiments extremely significant, and experts
have been claimed to pose real risk of have identified useful work that can
sparking a pandemic, so we put this be done today, so we have put it in the
in the upper band for likelihood. We upper category of attention.
also think it is appropriate that it gets Finally, risks from geoengineering
significant amounts of attention, par- are another emerging technological
ticularly as the unmanaged risk may risk where the current risks seem low.
be increasing. But because geoengineering may be
Risks from artificial intelligence ap- employed to tackle climate change,
pear to be low in the short-term, since it seems important to build a more
we most likely are some way off the comprehensive understanding of the
kind of capabilities that might cause a risks it poses in itself. For this reason
global catastrophe. We have therefore we have put it in the higher category
put them in the lower category of cur- for attention.
rent risk, although it is hard to predict

There is a great deal of
uncertainty around many
of the risks, and around
how much can usefully be
done. Nonetheless, some
attempt to estimate must
be made in order to
prioritise the world’s
scarce resources.

CHAPTER 3 – RISK FACTORS AND INTERACTIONS BETWEEN RISKS
Chapter 3
Risk factors and

interactions between risks
I
n this chapter we discuss some Global catastrophic risks are parts
of the factors that affect the like- of extremely complicated causal net-
lihood of global catastrophic works, and the drivers of each one
risks. We first discuss some of the could quite sensibly be the subject of
key factors influencing individ- multiple books. Here, we provide a
ual risks, before turning to factors that brief overview where we highlight a
the risks have in common, and ways in few key drivers. This serves as a back-
which catastrophic events could trigger ground to the more action-oriented
further catastrophes. discussions of Chapters 4 and 5.

3.1. Drivers of
individual risks
NUCLEAR WAR ty which could result from significant
Increasing the number of nucle- climate change may increase the ge-
ar-armed actors -- horizontal prolif- opolitical tensions that drive nuclear
eration -- probably increases the like- risk.147
lihood of nuclear conflict. Increasing
the size and sophistication of their PANDEMICS
arsenals -- vertical proliferation -- in- The factors driving pandemic risk di-
creases the amount of damage that vide into those affecting the likeli-
could occur in a given conflict.141 hood of potentially pandemic patho-
These factors are, in turn, influenced gens arising, and those affecting how
by political factors such as leading much society will be affected by such
nations’ ability to control rogue ac- pathogens. Whether a given pathogen
tors, and the level of conflict between could cause a pandemic depends on
nuclear states.142 its natural parameters, such as trans-
The possibility of false alarms or mission vector, resistances, and le-
miscalculation between nuclear pow- thality, over which we generally have
ers is another driver of risk, exacer- little control.148 However, concentrat-
bated further when geopolitical ten- ed populations of poultry or other
sions are high.143 There is also the animals can increase the probabili-
unresolved question of exactly how ty of zoonotic spillover, raising the
sensitive the climate system is to dust risk.149 The transmission of pathogens
and sulphates. The higher the sensi- through society is increased by global
tivity, the greater the risk of a nuclear travel and dense populations, but de-
winter scenario.144 creased by factors such as improved
Our response to climate change hygiene.150 The effectiveness of our
may also have significant effects on countermeasures constitutes anoth-
the risk of nuclear war. Many author- er important factor, ranging from dis-
ities on climate change argue that it ease surveillance to healthcare access
will be hard to reduce GHG emissions in the developing world.151
sufficiently without nuclear power
being part of the energy mix.145 Unfor- SUPER-VOLCANOES,
tunately, rising use of nuclear power ASTEROIDS, AND COMETS
may increase the chance of nuclear The probability of super-volcanic
weapon proliferation.146 Moreover, the eruptions and asteroid and comet im-
mass migration and resource scarci- pacts are driven by natural process-

“If ‘positive feedback loops’ prove
to be worse than anticipated,
the risk of catastrophic climate
change will be higher.”
es outside of human control. Howev- CATASTROPHIC CLIMATE CHANGE

er, our ability to predict, and, in the & GEO-ENGINEERING
case of asteroids and comets, to pre- The risk of catastrophic climate
vent these catastrophes could have a change and the risk of a geoengineer-
significant bearing on their eventu- ing catastrophe are strongly influ-
al impact. Because we have less con- enced by the level of GHG emissions.
trol over the likelihood of an adverse These depend on factors including
event with these than with most glob- the amount of energy the world con-
al catastrophic risks, resilience is a sumes and the proportion of this
more central part of the risk manage- which comes from fossil fuels. The
ment strategy. A particularly impor- total energy consumption is shaped
tant factor is the level of food secu- by, among other things, population
rity, including stockpiles and the growth, economic development, and
ability to switch to less sunlight-de- energy efficiency. The proportion of
pendent food sources.152 Better food energy which comes from fossil fuels
security means a lower chance that a depends on other factors such as the
particulate winter would have cata- rate of progress in clean energy tech-
strophic consequences.153 nology development and the preva-

lence of economic incentives to adopt just their risk management mecha-

clean energy technologies.154 nisms.159
Another important set of factors Formal regulation of technology
concern how the climate is likely has an unclear impact—in some cas-
to react to increased levels of GHG es it could prevent risky activities, but
emissions. If the climate turns out to in other cases it could stifle critical
be generally more sensitive to GHG innovation or, if implemented before
emissions than expected, or if “pos- the issues are properly understood,
itive feedback loops” prove to be even increase risk.160 On the positive
worse than anticipated, the risk of side, technology developed in a cul-
catastrophic climate change will in- ture of responsible innovation will
crease.155 likely be safer.
Finally, the development of new For some of the emerging techno-
geoengineering techniques might ei- logical risks we may also hope to de-
ther decrease or increase global cat- velop solutions. It may reduce risk to
astrophic risk. They could decrease make progress on developing safety
global catastrophic risk if they prove aspects of a technology faster relative
to be an effective tool to mitigate to progress on the technology itself.
catastrophic climate change.156 But For example having excellent diag-
they could increase risk if they have nosis and treatment tools to contain
a high chance of causing a catastro- outbreaks before there is significant
phe of their own, or if they prove to risk from artificial pandemics could
be ineffective while at the same time reduce the total risk significantly.
leading countries to avoid emission Similarly, a solution to the problem
abatement (as discussed in Chapter of aligning advanced AI with human
2).157 values before we are able to create ad-
vanced AI would reduce risk.161 The
EMERGING TECHNOLOGY proposal of aiming for safety-enhanc-
The factors that impact the risks of ing technologies ahead of risk-in-
emerging technology are surround- creasing technologies in order to re-
ed by a high degree of uncertainty. duce global catastrophic risk is called
However, one plausible risk factor is ‘differential technological develop-
misaligned incentives, which might ment’.162
cause nation-states to engage in arms
races or under-regulate risky tech- UNKNOWN RISKS
nology for economic reasons.158 The It is of course impossible to provide
speed and suddenness of technolog- a detailed account of the drivers of
ical breakthroughs could also be a currently unknown risks. Unknown
risk factor, as sudden breakthroughs risks could in principle include both
might leave inadequate time for so- exogenous and anthropogenic risks.
cial and political institutions to ad- We have access to a long historical

The speed and suddenness
of technological
breakthroughs could
also be a risk factor, as
sudden breakthroughs
might leave inadequate
time for social and political
institutions to adjust
their risk management
mechanisms.
record, which might have brought risks in the future (although techno-
many exogenous risks to our atten- logical progress could also reduce
tion already. The historical record global catastrophic risk; see section
offers us little or no guidance on an- 3.2.). Another important factor in-
thropogenic risks. They will often fluencing these unknown risks is the
be driven by social or technological quality of our foresight work, since
change, so a higher rate of change the ability to plan for a risk could help
could mean more currently unknown both avoidance and mitigation.

3.2. Shared risk factors and

interactions between risks
S
o far we have mainly discussed might also be reversed. A global ca-
each risk in isolation. But the tastrophe could lead to a breakdown
global catastrophic risks share of social and political institutions,
numerous risk factors, and also inter- which in turn could cause an outbreak
act with each other. In this section, of violence. In a 2008 paper, Nel and
we discuss some salient examples of Righarts argued that even smaller nat-
shared risk factors - variables associ- ural disasters such as earthquakes,
ated with an altered level of risk - and tsunamis, and heat waves significant-
interactions between risks. ly increase the risk of violent civil con-
One important shared risk factor is flict in the short and medium term,
governance. Good governance may especially in low- and middle-income
help timely and appropriate prepa- countries.163 Similarly, there has been
ration and response to risks. By the significant attention on the relation-
same token, bad governance, nation- ship between climate change and po-
ally or internationally, could increase litical conflict. Many scholars and se-
the likelihood and potential impact of curity experts have argued that even
every risk. Lax oversight might lead moderate climate change could in-
to the accidental release of a danger- crease the risk of political violence
ous engineered pathogen from a lab- because of conflicts over dwindling
oratory. Dysfunctional governments natural resources (such as food and
would be less able to intervene early water), massive international migra-
in an outbreak to prevent it becom- tion and a range of other factors.164
ing a pandemic. Poor international Greater disasters are likely to lead to
coordination might radically wors- even more upheaval, and might signif-
en our prospects to avoid catastroph- icantly weaken the defenses against
ic climate change, and internation- further catastrophic events. In this
al tensions could increase the risk of way, one global catastrophe might
nuclear war. Bad governance could in trigger another.
principle also lead to over-prioritising Another set of factors which affect
catastrophic risk reduction, but be- many global catastrophic risks is tech-
cause of the political distortions dis- nological. Technological advances
cussed in Chapter 4 we think this is change the system we live in by giv-
less likely. ing actors new powers. These in turn
While bad governance can cause could increase or decrease global cat-
global catastrophes, the causality astrophic risk, even aside from cases

Food stockpiles and the
ability to rapidly increase
production of alternate
sources of food would
increase resilience to a
broad range of risks.

where the technology directly poses unscrupulous states to control their

risk. It is hard to fully predict the ef- citizens, which makes it easier for
fects of technological trends, but we states to engage in extreme behav-
will highlight three trends which may iour. It might also weaken trust be-
be relevant: economic productivity tween states, which could cause po-
gains, surveillance, and distributed litical tension. However, surveillance
manufacturing. may make it more difficult for mali-
Global productivity has increased cious states or terrorist groups to act
dramatically in the last few decades. in secret. This will become increas-
Partly, this is because of technological ingly important as barriers to access
progress and organisational efficien- destructive weapons fall. Surveil-
cy growth. Partly, it is due to automa- lance between countries could even
tion, a form of technological progress facilitate international cooperation
which we expect to continue. by making the actions of states more
This growth may help us reduce transparent. It is not clear what over-
risk because it lets us spend more on all effect surveillance has on levels of
prevention and resilience. Measures global catastrophic risk.
like clean energy and food stockpiles Distributed manufacturing is a set of
are costly, and more likely to receive technologies that allow products to be
investment when people are wealthy. designed and built without centralised
In the tail case, with very powerful factories. They offer many benefits,
artificial intelligence, there could be and may increase resilience to cata-
a radical improvement in our ability strophe by spreading out production
to manage other global catastrophic capacity. However, they also bypass
risks. However, economic develop- some government controls designed
ment may exacerbate some risks, at to prevent the construction of destruc-
least in the short term, for example tive weapons. 3D printing, an early
by increasing greenhouse gas emis- form of such technology, has already
sions. generated security risks by allowing
The global surveillance disclosures people to create functional homemade
by Edward Snowden and others re- firearms. More powerful forms of dis-
vealed that state surveillance is now tributed manufacturing could increase
extensive. In addition, the use of risk caused by malicious actors, by in-
CCTV is expanding, and police and creasing access to powerful weaponry
other law enforcement agencies are such as bioweapons.
making increasing use of cameras in Nuclear war, geo-engineering, su-
their day-to-day work. per-volcanoes, asteroids, and com-
Increased surveillance has the po- ets all pose global catastrophic risk in
tential both to exacerbate and reduce significant part because of the ‘par-
global catastrophic risk. Extensive ticulate winter’ scenarios they might
surveillance could make it easier for produce. By ejecting large amounts

FIGURE 3.2. A WORKING GUN MADE FROM
PLASTIC ON A 3D PRINTER165
of smoke, dust, and/or sulphates into tegies for resilience that address one
the stratosphere they could cause risk address several. Food stockpiles
global cooling, sunlight loss, ozone and the ability to rapidly incerase
loss, and subsequent agricultural production of alternate sources of
disruption.166 Because so many risks food would increase resilience to a
share this mechanism, many stra- broad range of risks.167

CHAPTER 4 – DO INSTITUTIONS COLLECTIVELY UNDERINVEST IN GLOBAL CATASTROPHIC RISK?
Chapter 4
Do institutions collectively
underinvest in global
catastrophic risk?
I
n Chapters 2 and 3 we gave an of attention to the risks, and suggests
overview of several global cat- some mechanisms for reducing risk
astrophic risks, the main fac- by countering the distortions.
tors that affect their likelihood In the second half of the chapter, we
and impact, and some of the examine the kinds of actors or insti-
levers available to influence them. In tutions that may be well-placed to act
the first half of this chapter, we argue on global catastrophic risks, or have
that market and political distortions a responsibility to do so. We look at
mean that these risks are likely to be how they can help to correct the mar-
systematically neglected by many ac- ket and political failures we consider
tors. This increases the importance in the first half.

We should expect
global catastrophic risks to
systematically receive less
attention than they merit.

4.1. Market and

political failures
I
t may seem surprising that rela- Supra-national institutions are gener-
tively little effort has gone into ally weaker and international com-
global catastrophic risk mitiga- munities less cohesive, and therefore
tion. However, we should expect glob- are less able to implement these solu-
al catastrophic risks to systematically tions to collective action problems.
receive less attention than they merit, Sometimes, such as with the World
for structural reasons described be- Trade Organisation, nations do col-
low. lectively give up some of their auton-
omy in order to provide a public good,
4.1.1. GLOBAL but this process requires a great deal
PUBLIC GOODS of negotiation and trust. Thus, insti-
Global catastrophic risk reduction is tutions for aligning the incentives of
a global public good – the benefits of nation-states to provide global public
reduction spill over to other countries goods are typically less mature and
and it is hard (and arguably unethi- less effective than those that provide
cal) to only protect the countries that national public goods.
contribute to risk reduction. Many
global public goods tend to be under- 4.1.2. INTERGENERA-
provided, primarily because coun- TIONAL PUBLIC GOODS
tries try to free-ride.168 Every country Many of the benefits of global cata-
might hope that other countries in- strophic risk reduction accrue to fu-
vest in risk reduction, so they can get ture generations. However, the in-
protected for free. If everyone shares terests of future generations tend to
this hope, no one will invest. To over- be systematically neglected because
come this, there is a need for good co- they cannot vote and have no direct
ordination between actors. voice in the political process.171 For ex-
In well-functioning states, na- ample, reducing the risk from cata-
tional public goods, such as defence strophic climate change may provide
and clean air,169 are provided despite only small benefits to many people
the difficulties of collective action. alive today, but could be very valua-
Sometimes, central regulation sup- ble to people who will exist in sixty
plies these goods, sometimes mar- years’ time If we do not have formal
ket mechanisms are employed, and processes for taking the interests of
sometimes cooperative institutions future generations into account, we
for supplying the goods emerge.170 may under-invest in risk reduction.

4.1.3. SPECIAL INTERESTS 4.1.4. UNPRECEDENTED

The costs of regulatory actions to re- RISKS
duce global catastrophic risks will Global catastrophic events occur very
typically be concentrated on par- infrequently. In the last two millen-
ticular industries, whereas the bene- nia, there may have only been two
fits are dispersed. The small affected such events – the Plague of Justinian
groups have strong incentives to lob- and the Black Death. Anthropogen-
by and campaign, while those gaining ic global catastrophes are complete-
the benefits will not regard those ben- ly unprecedented. Because unprec-
efits as a voting priority. This means edented events are typically less
that industry lobbying could wield salient, it is less likely that govern-
disproportionate power over the reg- ments and voters will pay appropri-
ulatory process.172 Consequently, the ate attention to them, in spite of their
trade-offs made between econom- very high costs in expectation.173
ic profit and risk reduction could be
skewed towards underprovision of
risk reduction.

FIGURE 4.1. PREPAREDNESS IN THE AFRICA REGION
Multi-hazard national public health emergency preparedness & response plan in the African Region. 174
Implementation status (%)
0-24
25-49
50-74
75-100
Data not available
Not part of WHO African region
Not applicable

4.2. Which actors

can help reduce global
catastrophic risk?
T
here is a broad range of ac- the risk of nuclear war. International
tors that can help us over- coordination can aim to reduce the
coming the current neglect of risk of nuclear conflict between these
global catastrophic risk. In this sec- major actors.177
tion, we give an overview of some There have been many agreements
ways in which different groups, intended to reduce global catastroph-
ranging from the international com- ic risk. Two of the more important
munity to individuals, can contrib- examples178 are the Treaty on the
ute to the mitigation of global cata- Non-Proliferation of Nuclear Weap-
strophic risks. ons179 and the 1992 United Nations
Framework Convention on Climate
4.2.1. THE INTERNA- Change, with its objective of pre-
TIONAL COMMUNITY venting “dangerous” anthropogen-
As discussed in section 4.1, reduc- ic climate change.180 Similarly, there
tion of global catastrophic risk is a are several permanent internation-
global public good. This means that al bodies, such as the International
the international community will Atomic Energy Agency and the World
probably need to play a major role in Health Organisation, which are (part-
reducing global catastrophic risk. In ly) concerned with reducing global
some cases, the global level of risk is catastrophic risks (nuclear war181 and
significantly dependent on the ca- pandemics, respectively). The inter-
pabilities of the countries that are national community often collabo-
the weakest links.175 For example, to rates productively with relevant parts
safeguard against the risk of a global of the academic community, as it did
pandemic, the least prepared coun- when it set up the Intergovernmen-
tries should be enabled to strengthen tal Panel on Climate Change.182 Many
their health systems.176 In other cas- of the global catastrophic risks - not
es, the risk of a catastrophe is mainly least those associated with emerg-
dependent on a few major actors. For ing technologies - require a deep level
example, the nuclear-weapon states of technical expertise to be properly
have a disproportionate influence on managed.
88
4.2.2. NATION-STATES how to respond. For instance, we

Although the international commu- would benefit from more resources
nity has a crucial role in the reduc- devoted to the study of global cata-
tion of global catastrophic risk — strophic risks such as tail risk climate
because this is a global public good change183 and particulate winter sce-
— much power currently resides in narios.184 We would also benefit from
the hands of nation-states. In prac- more research on technical solutions
tice, international bodies have limit- that could reduce global catastroph-
ed power unless they are backed up ic risks, such as clean energy sourc-
by nation-states (especially the more es, or how to how to align the actions
powerful ones). This means that na- of above-human-level AI with our
tion-states are essential to galvanis- values.185 Since catastrophic risk re-
ing action. They also need to lead the duction is a public good, it is unlike-
process of implementation. For in- ly that this research will be carried
stance, it is up to nation-states to see out by companies under competitive
that emissions actually are cut to the pressure. Instead, it will most likely
extent required by the Paris agree- require public or charitable funding.
ment. Similarly, it is the responsibili- In fact, several academic institutions
ty of nuclear-weapon states to guard that focus on global catastrophic risk
against the possibility of accidental have already been set up through
launch. public and philanthropic funds.
Nation-states can also reform their Among them are Oxford’s Future of
internal political processes in ways Humanity Institute and Cambridge’s
that are conducive to reduction of Centre for the Study of Existential
global catastrophic risks. Ensuring Risk.
that decisions take account of the in- Beyond this, the research commu-
terests of future generations is, for nity should contribute to global cat-
instance, likely to lead to a more ap- astrophic risk reduction by promot-
propriate degree of focus on glob- ing a culture of safety within areas of
al catastrophic risk. Similarly, con- research that could have the poten-
straining the excessive power of tial to cause a catastrophe through
special interests could decrease dis- accident or misuse. This is especially
tortions on decision-making and so in relevant for emerging technologies,
turn reduce global catastrophic risk. where it is not always clear in advance
whether there are any risks.186
4.2.3. THE RESEARCH
COMMUNITY 4.2.4. INDUSTRY
Many of the global catastrophic risks Competitive pressures mean that it
are not well-understood, and more is often hard for companies to make
research would allow more appropri- large moves on issues which do not
ate decisions about when to act and improve their profits. They can, how-

ever, show leadership on reducing the 4.2.6. THE NON-PROFIT

risk of catastrophic climate change SECTOR
by choosing low carbon options at The non-profit sector contributes sig-
the margin, or by developing clean nificantly to global catastrophic risk
energy solutions. Just like the wider reduction. Non-profits are often less
research community, they can and constrained than companies and na-
should promote a culture of safety in tional governments, which means
biotechnology and AI research as, for that they are free to work on intergen-
example, Google DeepMind has done erational global public goods such as
by setting up an AI ethics board.187 global catastrophic risk reduction --
if they can find donors that support
4.2.5. GENERAL PUBLIC that cause. There are many charities
Some global catastrophic risks can be working on individual global cata-
addressed by individual action. For strophic risks. In recent years, donors
instance, individuals can decrease have been increasingly interested in
the risk of catastrophic climate global catastrophic risk reduction as
change (if by ever so little) by mak- a general category. This has led to the
ing low-carbon consumption choic- creation of groups such as the Future
es. Perhaps the more promising route of Life Institute in Boston.
to reduce global catastrophic risk for Besides supporting research, chari-
individuals is, however, by exerting ties can also support direct interven-
political pressure on policymakers. tions, e.g. to increase pandemic pre-
For instance, voters could try to in- paredness in developing countries.188
fluence politicians to agree to carbon They can also exert political pressure,
emissions cuts, and then to actually as International Physicians for the
implement the agreements once they Prevention of Nuclear War did to re-
are in place. duce the risk of a nuclear exchange
during the cold war, winning the No-
bel Peace Prize in 1985.189

The research community
should contribute to
global catastrophic risk
reduction by promoting a
culture of safety within
areas of research that could
have the potential to cause
a catastrophe through
accident or misuse.

CHAPTER 5 – WHAT CAN THE WORLD DO TO REDUCE GLOBAL CATASTROPHIC RISK?
Chapter 5
What can the world

do to reduce global
catastrophic risk?
I
n Chapter 2, we looked at dif- in order to reduce global catastroph-
ferent global catastrophic risks ic risk. Some of these steps pertain to
and their mechanisms. There individual risks, whereas others are
was also some preliminary dis- cross-cutting opportunities which
cussion of actions available to may reduce the chance or impact of
limit the risks. In Chapter 3 we ex- several different risks at once.
plored the different factors which af- Our aim here is to offer some start-
fect these risks. In Chapter 4 we con- ing points for considering action on
sidered why global catastrophic risk is risks of global catastrophe and to
probably neglected, and how differ- demonstrate that there are real ave-
ent actors can help with it. Finally in nues to making progress. For more
this chapter we draw these strands detailed discussion of the actions
together and outline a few of the most available, there exists a rich literature
promising steps that existing commu- on most of the specific risks.
nities can take or are already taking

Our aim here is to
offer some starting
points for considering
action on risks of
global catastrophe
and to demonstrate that
there are real avenues to
making progress.

CATASTROPHIC CLIMATE CHANGE NUCLEAR WAR

• Research communities should in- • The international community
crease their focus on understand- should continue the policy of nu-
ing the pathways to and likelihood clear non-proliferation, and nu-
of catastrophic climate change, and clear states can continue to reduce
possible ways to respond.190 stockpiles.
• Nations should continue to imple- • Nuclear-weapon states should
ment and improve mechanisms continue to work to reduce the
for emissions abatement such as chance of accidental launch or
carbon taxes or tradable emissions escalation.193
quotas, as for non-catastrophic cli- • Nuclear weapon states can adopt
mate change.191 military strategies that reduce the
• Developed nations could commit chance of nuclear winter.194
to the goal of building no new coal-
fired power stations without carbon
capture and sequestration.192

NATURAL PANDEMICS
• The World Health Organisation,
nation states, and other bodies
should increase their planning for
extremely bad pandemics.195
• International and research com-
munities could improve disease ASTEROIDS AND COMETS
surveillance, for example by reduc- • Research communities should con-
ing the delay between scientific tinue working to detect and track
breakthrough and the availability asteroids and commets with a di-
of diagnostic tools. ameter of 1 km or more.197
• The global health community
should improve developing world SUPERVOLCANIC ERUPTIONS
capacity for response, for example • Research communities should con-
by ensuring that vaccine produc- tinue to work on understanding
tion facilities are well-distributed their causes, to increase predicta-
around the world.196 bility.198

BIOTECHNOLOGY • Research communities can further

• Research communities should fur- investigate the potential impacts of
ther investigate the possible risks solar radiation management.202
from emerging capabilities in bio-
technology, and develop counter- UNKNOWN RISKS
measures.199 • Research communities can contin-
• Policymakers should continue to ue to develop methods and tools
work with researchers to understand for horizon scanning and reduc-
the biosafety and biosecurity issues tion of unknown unknowns.
that are likely to arise, and build • Research communities should
planned adaptive risk regulation.200 identify and carefully investigate
speculative threats.
ARTIFICIAL INTELLIGENCE
• Research communities should fur- CROSS-CUTTING OPPORTUNITIES
ther investigate the possible risks • Research communities should fur-
from artificial intelligence, and ther investigate ‘particulate winter’
work on developing possible solu- scenarios, both in forecasting and
tions.201 mitigation strategies.203
• Policymakers can work with re- • Nations and local communities can
searchers to understand the impli- continue to take steps to build their
cations of advanced artificial intel- resilience to catastrophe.204
ligence. • Research communities should fo-
cus greater attention on strategies
GEO-ENGINEERING and technologies for resilience to
• The international community and recovery from global catastro-
should continue work to stabilise phe, for example by developing al-
and eventually reduce concentra- ternate food sources.205
tions of greenhouse gases in the • Nations should work to incorporate
atmosphere in order to reduce the the interests of future generations
incentives to use solar radiation into their decision-making frame-
management. works.206

Research communities
should focus greater
attention on strategies
and technologies for
resilience to and
recovery from global
catastrophe, for example
by developing alternate
food sources.

ENDNOTES
Endnotes
1. William Rosen, Justinian’s Flea : Plague, Empire, and 15. http://www.atomicarchive.com/Docs/Begin/Einstein.
the Birth of Europe (New York: Viking, 2007), 3, 209. shtml
Percentages deduced from Demographic Internet Staff 16. A
li Nouri and Christopher F. Chyba, “Biotechnology
US Census Bureau, “International Programs, World and Biosecurity,” in Global Catastrophic Risks, ed. Nick
Population,” accessed February 12, 2016, https://www. Bostrom and Milan M. Ćirković (Oxford University Press,
census.gov/population/international/data/worldpop/ 2008).
table_history.php. Some earlier estimates put the death
toll as high as 100 million. See B. Lee Ligon, “Plague: 17. F
uture of Life Institute, “AI Open Letter,” accessed
A Review of Its History and Potential as a Biological February 3, 2016, http://futureoflife.org/ai-open-letter/.
Weapon,” Seminars in Pediatric Infectious Diseases, 18. N
ASA, “The Relentless Rise of Carbon Dioxide,” Climate
Return of the Gram-Positives, 17, no. 3 (July 2006): Change: Vital Signs of the Planet, accessed February 2,
161–70. 2016, http://climate.nasa.gov/climate_resources/24/.
2. Rosen, Justinian’s Flea, 3–5, 321–324. 19. G
ernot Wagner and Martin L. Weitzman, Climate Shock :
3. dward Wilson, “Thank You Vasili Arkhipov, the Man Who
E The Economic Consequences of a Hotter Planet (Princeton:
Stopped Nuclear War,” The Guardian, October 27, 2012, Princeton University Press, 2015).
http://www.theguardian.com/commentisfree/2012/ 20. C
learly, enormous literatures exist in each of these
oct/27/vasili-arkhipov-stopped-nuclear-war. areas which can inform policy. We suggest only a small
4. his is adapted from Nick Bostrom, “Existential Risk
T selection of the policy options which are available.
Prevention as Global Priority,” Global Policy 4, no. 1 21. NASA, “The Relentless Rise of Carbon Dioxide,”.
(February 1, 2013): 16.
22. S
ergey A. Zimov, Edward AG Schuur, and F. Stuart Chapin
5. Steven Pinker, The Better Angels of Our Nature : A History of III, “Permafrost and the Global Carbon Budget,” accessed
Violence and Humanity (London: Penguin, 2012),195. Note January 18, 2016, http://www.imedea.uib-csic.es/master/
that although the fatalities are too low, the non-fatality cambioglobal/Modulo_V_cod101619/Permafrost%20
impacts mean it could be argued that the Second World response.pdf.
War was just about large enough to count as a global
23. C
hristopher B. Field et al., Climate Change 2014:
catastrophe in our sense.
Impacts, Adaptation, and Vulnerability: Summary for
6. iall Johnson and Juergen Mueller, “Updating the
N Policymakers (Contribution of Working Group II to the
Accounts: Global Mortality of the 1918-1920 ‘Spanish’ Fifth Assessment Report of the Intergovernmental Panel
Influenza Pandemic,” Bulletin of the History of Medicine 76, on Climate Change, 2014), 14, http://epic.awi.de/37531/.
no. 1 (2002): 105–15.
24. D
avid King et al., “Climate Change–a Risk Assessment”
7. or further discussion see the sections on natural
F (Centre for Science Policy, University of Cambridge,
pandemics, nuclear war and super-volcanic eruptions in 2015), 45–46, www.csap.cam.ac.uk/projects/climate-
Chapter 2. change-risk-assessment/.
8. dapted from Nick Bostrom, “Existential Risk Prevention
A 25. G
ernot Wagner and Martin L. Weitzman, Climate
as Global Priority,” Global Policy 4, no. 1 (February 1, Shock : The Economic Consequences of a Hotter Planet
2013): 15–31, doi:10.1111/1758-5899.12002. (Princeton: Princeton University Press, 2015), 53–56.
9.
Sandberg, A. & Bostrom, N. (2008): “Global Catastrophic 26. K
ing et al., “Climate Change–a Risk Assessment,” chap.
Risks Survey”, Technical. Report #2008-1, Future of 10.
Humanity Institute, Oxford University: pp. 1-5.
27. King et al., “Climate Change–a Risk Assessment,” pt. 1.
10. N
. H. Stern, The Economics of Climate Change : The Stern
28. “ Green Light,” The Economist, December 19, 2015, http://
Review (Cambridge: Cambridge University Press, 2007), 47.
www.economist.com/news/international/21684144-what-
11. “ Institute for Health Metrics and Evaluation (IHME). GBD expect-after-deal-exceeded-expectations-green-light.
Compare. Seattle, WA: IHME, University of Washington,
29. R
ajendra K. Pachauri et al., Climate Change 2014: Synthesis
2015. Available from http://vizhub.healthdata.org/gbd-
Report. Contribution of Working Groups I, II and III to the
compare” from ‘ibid.’
Fifth Assessment Report of the Intergovernmental Panel on
12. A
dapted from Nick Bostrom, “Existential Risk Prevention Climate Change, 2014, 10-13
as Global Priority,” Global Policy 4, no. 1 (February 1,
30. Wagner and Weitzman, Climate Shock, 53–56.
2013): 15–31, doi:10.1111/1758-5899.12002.
31. F
ield et al., Climate Change 2014: Impacts, Adaptation,
13. Derek Parfit, Reasons and Persons (Oxford: Oxford
and Vulnerability: Summary for Policymakers, chap. 16.
University Press, 1984); Nick Bostrom, “Astronomical
Waste: The Opportunity Cost of Delayed Technological 32. N
icholas Stern, “The Economics of Climate Change,” The
Development,” Utilitas 15, no. 03 (November 2003): American Economic Review 98, no. 2 (May 1, 2008): 1–37;
308–14; Nick Bostrom, “Existential Risk Prevention as William D. Nordhaus, “A Review of the ‘Stern Review on
Global Priority”. the Economics of Climate Change,’” Journal of Economic
Literature 45, no. 3 (September 1, 2007): 686–702;
14. We discuss this in more detail in chapter 4.
Wagner and Weitzman, Climate Shock.

33. Wagner and Weitzman, Climate Shock, chap. 3. 50. A
nthony M. Barrett, Seth D. Baum, and Kelly Hostetler,
34. H
. Damon Matthews et al., “The Proportionality of Global “Analyzing and Reducing the Risks of Inadvertent Nuclear
Warming to Cumulative Carbon Emissions,” Nature 459, War Between the United States and Russia,” Science &
no. 7248 (June 11, 2009): 829–32. Global Security 21, no. 2 (May 1, 2013): 106–33, doi:10.10
80/08929882.2013.798984.
35. M
alte Meinshausen et al., “Greenhouse-Gas Emission
Targets for Limiting Global Warming to 2 °C,” Nature 458, 51. G
raham Allison, “At 50, the Cuban Missile Crisis as
no. 7242 (April 30, 2009): 1158–62. Guide,” The New York Times, June 15, 2012.
36. Ibid. 52. Z

achary Keck, “Russia Threatens Nuclear Strikes Over
Crimea,” July 11, 2014; National Institute for Public Policy,
37. J. G. Shepherd, Geoengineering the Climate: Science, “Russia’s Nuclear Posture” (Fairfax, VA, 2015).
Governance and Uncertainty (Royal Society, 2009), chap. 2.
53. H
ellman, “How Risky Is Nuclear Optimism?”; Tom
38. Ibid., chap. 3. Hussain, “Are India and Pakistan Heading for a Nuclear
39. G
ernot Wagner and Martin L. Weitzman, Climate Shock : Showdown?,” Al Jazeera, March 2016, http://www.
The Economic Consequences of a Hotter Planet (Princeton: aljazeera.com/indepth/opinion/2016/03/india-pakistan-
Princeton University Press, 2015), 54. heading-nuclear-showdown-160303053541342.html.
40. S
ee Martin E. Hellman, “How Risky Is Nuclear Optimism?,” 54. M
ichael Krepon, “Pakistan’s Nuclear Strategy and
Bulletin of the Atomic Scientists 67, no. 2 (March 1, 2011): Deterrence Stability,” Stimson Center 10 (2012), http://
47–56. indianstrategicknowledgeonline.com/web/Krepon_-_
Pakistan_Nuclear_Strategy_and_Deterrence_Stability.pdf.
41. B
ulletin of the Atomic Scientists, “Nuclear Notebook,”
Bulletin of the Atomic Scientists, accessed February 55. S
ee Cirincione, “The Continuing Threat of Nuclear War”;
3, 2016, http://thebulletin.org/nuclear-notebook- Hellman, “How Risky Is Nuclear Optimism?”
multimedia. 56. Cirincione, “The Continuing Threat of Nuclear War,” 396.
42. H
ans M. Kristensen and Robert S. Norris, “US Nuclear 57. M
atthew R. Costlow, “Do More Nukes Really Mean More
Forces, 2015,” Bulletin of the Atomic Scientists 71, no. 2 Nuclear Crises? Not Necessarily,” Bulletin of the Atomic
(2015): 108. Scientists, December 8, 2015, http://thebulletin.org/
43. H
ans M. Kristensen and Robert S. Norris, “Russian do-more-nukes-really-mean-more-nuclear-crises-not-
Nuclear Forces, 2015,” Bulletin of the Atomic Scientists necessarily8965.
71, no. 3 (2015): 85. 58. F
or other suggestions for weapons systems see Seth
44. “ A Nuke by Any Other Name,” Bulletin of the Atomic D. Baum, “Confronting the Threat of Nuclear Winter,”
Scientists, May 17, 2012, http://thebulletin.org/nuke-any- Futures, Confronting Future Catastrophic Threats To
other-name. Humanity, 72 (September 2015): 69–79, doi:10.1016/j.
futures.2015.03.004.
45. U
S Office of Technology Assessment, “The Effects of
Nuclear War,” 1979, 8–9, http://ota.fas.org/reports/7906. 59. http://thebulletin.org/nuclear-notebook-multimedia
pdf; Joseph Cirincione, “The Continuing Threat of Nuclear 60. S
ee the work of the Nuclear Threat Initiative at http://
War,” in Global Catastrophic Risks, ed. Nick Bostrom www.nti.org/.
and Milan M. Ćirković (Oxford University Press, 2008),
386–390. 61. A
lan Robock, Luke Oman, and Georgiy L. Stenchikov,
“Nuclear Winter Revisited with a Modern Climate
46. T
his area has received less attention since the end of Model and Current Nuclear Arsenals: Still Catastrophic
the Cold War. Other estimates give death tolls within the Consequences,” Journal of Geophysical Research:
same order of magnitude. Ira Helfand et al., “Projected Atmospheres 112, no. D13 (July 16, 2007): 10,
US Casualties and Destruction of US Medical Services doi:10.1029/2006JD008235.
from Attacks by Russian Nuclear Forces,” Medicine &
Global Survival 7, no. 2 (2002): 68–76; WHO, “Health 62. O
le J. Benedictow, “The Black Death: The Greatest
Effects of the Chernobyl Accident and Special Health Care Catastrophe Ever,” History Today, 2005, http://www.
Programmes” (WHO, 2006), http://www.who.int/ionizing_ historytoday.com/ole-j-benedictow/black-death-
radiation/chernobyl/who_chernobyl_report_2006.pdf; greatest-catastrophe-ever; Edwin Dennis Kilbourne,
Owen B. Toon et al., “Atmospheric Effects and Societal “Plagues and Pandemics: Past, Present, and Future,”
Consequences of Regional Scale Nuclear Conflicts and in Global Catastrophic Risks, ed. Nick Bostrom and
Acts of Individual Nuclear Terrorism,” Atmospheric Milan M. Ćirković (Oxford: Oxford University Press,
Chemistry and Physics 7, no. 8 (2007): 1973–2002. 2008), 295. Percentages deduced using population data
from Demographic Internet Staff US Census Bureau,
47. F
or a good summary see Seth D. Baum, “Winter-Safe “International Programs, World Population,” accessed
Deterrence: The Risk of Nuclear Winter and Its Challenge February 12, 2016, https://www.census.gov/population/
to Deterrence,” Contemporary Security Policy 36, no. 1 international/data/worldpop/table_history.php.
(January 2, 2015): 126, doi:10.1080/13523260.2015.101
2346. 63. N
iall Johnson and Juergen Mueller, “Updating the
Accounts: Global Mortality of the 1918-1920 ‘Spanish’
48. A
lan Robock and Owen Brian Toon, “Local Nuclear War, Influenza Pandemic,” Bulletin of the History of Medicine
Global Suffering,” Scientific American 302, no. 1 (2010): 76, no. 1 (2002): 105–15.
74–81. See the discussion in Givewell, “Nuclear Weapons
Policy,” September 2015, http://www.givewell.org/labs/ 64. W
HO, “Current WHO Phase of Pandemic Alert for Avian
causes/nuclear-weapons-policy. Influenza H5N1,” WHO, accessed February 15, 2016,
http://apps.who.int/csr/disease/avian_influenza/phase/
49. Martin E. Hellman, “How Risky Is Nuclear Optimism?” en/index.html; Declan Butler, “Death-Rate Row Blurs

ENDNOTES
Mutant Flu Debate,” Nature 482, no. 7385 (February 13, History of Medicine 76, no. 1 (2002): 105–15; Ole J.
2012): 289–289, doi:10.1038/482289a. Benedictow, “The Black Death: The Greatest Catastrophe
65. M
arc Lipsitch and Thomas V. Inglesby, “Moratorium on Ever,” History Today, 2005, http://www.historytoday.com/
Research Intended To Create Novel Potential Pandemic ole-j-benedictow/black-death-greatest-catastrophe-ever;
Pathogens,” mBio 5, no. 6 (December 31, 2014): WHO and others, Bugs, Drugs and Smoke: Stories from
e02366–14, doi:10.1128/mBio.02366-14. Public Health, 2011, chap. 1, http://apps.who.int/iris/
handle/10665/44700; WHO, “HIV/AIDS Fact Sheet,” WHO,
66. Butler, “Death-Rate Row Blurs Mutant Flu Debate.” November 2015, http://www.who.int/mediacentre/
67. C
hristopher JL Murray et al., “Estimation of Potential factsheets/fs360/en/.
Global Pandemic Influenza Mortality on the Basis of Vital 77. A
rnon Dar, “Influence of Supernovae, Gamma-Ray
Registry Data from the 1918–20 Pandemic: A Quantitative Bursts, Solar Flares, and Cosmic Rays on the Terrestrial
Analysis,” The Lancet 368, no. 9554 (December 23, 2006): Environment,” in Global Catastrophic Risks, ed. Nick
2211–18, doi:10.1016/S0140-6736(06)69895-4. Bostrom and Milan M. Ćirković (Oxford: Oxford
68. M
ichael T. Osterholm, “Preparing for the Next Pandemic,” University Press, 2008).
New England Journal of Medicine 352, no. 18 (May 5, 78. S
ee the draft manuscript Toby Ord, “Will We Cause Our
2005): 1839–42, doi:10.1056/NEJMp058068; Neil M. Own Extinction? Natural versus Anthropogenic Extinction
Ferguson et al., “Strategies for Mitigating an Influenza Risks” 2014. Ord’s argument is about extinction risks and
Pandemic,” Nature 442, no. 7101 (July 27, 2006): 448–52, not the broader category of global catastrophic risks, but
doi:10.1038/nature04795. it is very plausible that similar arguments would also show
69. B
ruine de Bruin, Wändi, et al. “Expert judgments of that exogenous non-extinction global catastrophic risks
pandemic influenza risks.” Global Public Health 1.2 are less likely than anthropogenic non-extinction global
(2006): 179-194. catastrophic risks.
70. F
or two recent overviews see Peter Sands, Carmen 79. M
ichael Rampino, “Super-Volcanism and Other
Mundaca-Shah, and Victor J. Dzau, “The Neglected Geophysical Processes of Catastrophic Import,” in Global
Dimension of Global Security — A Framework for Catastrophic Risks, ed. Nick Bostrom and Milan M.
Countering Infectious-Disease Crises,” New England Ćirković (Oxford: Oxford University Press, 2008).
Journal of Medicine 0, no. 0 (January 13, 2016): null, 80. G
. A. Zielinski et al., “Potential Atmospheric Impact of
doi:10.1056/NEJMsr1600236; WHO and Report of the the Toba Mega-Eruption 71,000 Years Ago,” Geophysical
High-level Panel on the Global Response to Health Crises, Research Letters 23, no. 8 (April 15, 1996): 837–40,
“Protecting Humanity from Future Health Crises” (WHO, doi:10.1029/96GL00706; Rampino, “Super-Volcanism and
January 25, 2016), http://www.un.org/News/dh/infocus/ Other Geophysical Processes of Catastrophic Import,”
HLP/2016-02-05_Final_Report_Global_Response_to_ 209–210.
Health_Crises.pdf.
81. R
ampino, “Super-Volcanism and Other Geophysical
71. S
tanley A. Plotkin, Adel A.F. Mahmoud, and Jeremy Processes of Catastrophic Import,” 211–212; Christine
Farrar, “Establishing a Global Vaccine-Development S. Lane, Ben T. Chorn, and Thomas C. Johnson, “Ash
Fund,” New England Journal of Medicine 373, no. 4 (July 23, from the Toba Supereruption in Lake Malawi Shows No
2015): 297–300, doi:10.1056/NEJMp1506820. Volcanic Winter in East Africa at 75 Ka,” Proceedings of
72. C
atherine Yen et al., “The Development of Global the National Academy of Sciences 110, no. 20 (May 14,
Vaccine Stockpiles,” The Lancet Infectious Diseases 2013): 8025–29, doi:10.1073/pnas.1301474110.
15, no. 3 (March 2015): 340–47, doi:10.1016/S1473- 82. S
. Sparks et al., Super-Eruptions: Global Effects and
3099(14)70999-5. Future Threats. London, Report of a Geological Society of
73. W
HO, “International Health Regulations: Support to London Working Group, 2005, 6.
Global Outbreak Alert and Response, and Building 83. W
. Aspinall et al., “Volcano Hazard and Exposure in
and Maintaining National Capacities,” 2015, http:// GFDRR Priority Countries and Risk Mitigation Measures,”
apps.who.int/iris/bitstream/10665/199747/1/WHO_ Volcano Risk Study 0100806-00-1-R, 2011, 15; Rampino,
HSE_GCR_2015.7_eng.pdf; WHO, “Pandemic Influenza “Super-Volcanism and Other Geophysical Processes of
Preparedness Framework,” 2011, http://www.who.int/ Catastrophic Import,” 212–213; Susan Loughlin et al.,
influenza/resources/pip_framework/en/. Global Volcanic Hazards and Risk (Cambridge University
74. E
mily H. Chan et al., “Global Capacity for Emerging Press, 2015), 97.
Infectious Disease Detection,” Proceedings of the 84. Sparks et al., Super-Eruptions, 20.
National Academy of Sciences 107, no. 50 (December 14,
2010): 21701–6, doi:10.1073/pnas.1006219107. 85. P
eter Schulte et al., “The Chicxulub Asteroid Impact and
Mass Extinction at the Cretaceous-Paleogene Boundary,”
75. S
ands, Mundaca-Shah, and Dzau, “The Neglected Science 327, no. 5970 (March 5, 2010): 1214–18,
Dimension of Global Security — A Framework for doi:10.1126/science.1177265.
Countering Infectious-Disease Crises.”
86. N
ational Research Council (U. S.). Committee to Review
76. E
dwin Dennis Kilbourne, “Plagues and Pandemics: Past, Near-Earth-Object Surveys and Hazard Mitigation
Present, and Future,” in Global Catastrophic Risks, ed. Strategies, Defending Planet Earth: Near-Earth Object
Nick Bostrom and Milan M. Ćirković (Oxford: Oxford Surveys and Hazard Mitigation Strategies (Washington,
University Press, 2008); William Rosen, Justinian’s DC: National Academies Press, 2010), 21.
Flea : Plague, Empire, and the Birth of Europe (New
York: Viking, 2007); Niall Johnson and Juergen Mueller, 87. N
ational Research Council (U. S.). Committee to Review
“Updating the Accounts: Global Mortality of the 1918- Near-Earth-Object Surveys and Hazard Mitigation
1920 ‘Spanish’ Influenza Pandemic,” Bulletin of the Strategies, Defending Planet Earth, 21.

88. C
lark R. Chapman, “The Hazard of near-Earth Asteroid Church_Biohazard04c.htm.
Impacts on Earth,” Earth and Planetary Science 106. Nathan Myhrvold, “Strategic Terrorism: A Call to Action,”
Letters 222, no. 1 (May 15, 2004): 11, doi:10.1016/j. SSRN Scholarly Paper (Rochester, NY: Social Science
epsl.2004.03.004. Research Network, July 3, 2013), http://papers.ssrn.com/
89. N
ational Research Council (U. S.). Committee to Review abstract=2290382.
Near-Earth-Object Surveys and Hazard Mitigation 107. Martin J. Rees, Our Final Century : Will Civilisation Survive
Strategies, Defending Planet Earth, 23. the Twenty-First Century? (London: Arrow, 2004).
90. A
lan Harris, “What Spaceguard Did,” Nature 453, no. 7199 108. Marc Lipsitch and Alison P. Galvani, “Ethical Alternatives
(June 26, 2008): 1178–79, doi:10.1038/4531178a; NASA, to Experiments with Novel Potential Pandemic
“NASA Space Telescope Finds Fewer Asteroids near Pathogens,” PLoS Med 11, no. 5 (May 20, 2014):
Earth,” September 2011, http://www.nasa.gov/mission_ e1001646, doi:10.1371/journal.pmed.1001646.
pages/WISE/news/wise20110929.html.
109. Lynn C. Klotz and Edward J. Sylvester, “The Unacceptable
91. Jason C. Reinhardt et al., “Asteroid Risk Assessment: A Risks of a Man-Made Pandemic,” Bulletin of the Atomic
Probabilistic Approach,” Risk Analysis, July 1, 2015, 1-18, Scientists, August 7, 2012, http://thebulletin.org/
doi:10.1111/risa.12453. unacceptable-risks-man-made-pandemic. http://www.
92. S
ource: Alan Harris, “What Spaceguard Did,” Nature 453, the-scientist.com/?articles.view/articleNo/41263/title/
no. 7199 (June 26, 2008): 1178–79, doi:10.1038/4531178a. Moratorium-on-Gain-of-Function-Research/
93. N
ational Research Council (U. S.). Committee to Review 110. Lipsitch and Inglesby, “Moratorium on Research Intended
Near-Earth-Object Surveys and Hazard Mitigation To Create Novel Potential Pandemic Pathogens,” 2.
Strategies, Defending Planet Earth, 23. 111. Ibid.
94. Ibid., 22. 112. Gautam Mukunda, Kenneth A. Oye, and Scott C. Mohr,
95. Harris, “What Spaceguard Did.” “What Rough Beast?,” Politics and the Life Sciences 28,
96. A
. Harris et al., “Asteroid Impacts and Modern Civilization: no. 2 (September 1, 2009): 8, doi:10.2990/28_2_2.
Can We Prevent a Catastrophe,” Asteroids IV. University 113. Church, “A Synthetic Biohazard Non-Proliferation
of Arizona Press, Tucson, AZ, 2015, http://elib.dlr. Proposal.”
de/100120/1/3004_Harris_reprint.pdf. 114. Owen Cotton-Barratt, Sebastian Farquhar, and Andrew
97. R
obert Carlson, “The Changing Economics of DNA Snyder-Beattie, “Beyond Risk-Benefit Analysis: Pricing
Synthesis,” Nature Biotechnology 27, no. 12 (December Externalities for Gain-of-Function Research of Concern”
2009): 1091–94, doi:10.1038/nbt1209-1091. (Future of Humanity Institute, Global Piorities Project,
98. T
errence M. Tumpey et al., “Characterization of the March 2016).
Reconstructed 1918 Spanish Influenza Pandemic Virus,” 115. Chyba and Greninger, “Biotechnology and Bioterrorism,”
Science 310, no. 5745 (2005): 77–80. 151–153.
99. M
arkus Schmidt, “Diffusion of Synthetic Biology: A 116. “DNA Sequencing Costs,” accessed March 31, 2016,
Challenge to Biosafety,” Systems and Synthetic Biology 2, http://www.genome.gov/sequencingcosts/.
no. 1–2 (June 2008): 1–6, doi:10.1007/s11693-008-9018-z. 117. Future of Life Institute, “AI Open Letter,” accessed
100. Ronald J. Jackson et al., “Expression of Mouse February 3, 2016, http://futureoflife.org/ai-open-letter/.
Interleukin-4 by a Recombinant Ectromelia Virus 118. Eliezer Yudkowsky, “Artificial Intelligence as a Positive
Suppresses Cytolytic Lymphocyte Responses and and Negative Factor in Global Risk,” in Global Catastrophic
Overcomes Genetic Resistance to Mousepox,” Journal of Risks, ed. Nick Bostrom and Milan M. Ćirković, vol. 1
Virology 75, no. 3 (2001): 1205–10. (Oxford University Press, 2008), 303.
101. Christopher F. Chyba and Alex L. Greninger, 119. Carl Benedikt Frey and Michael A. Osborne, “The Future
“Biotechnology and Bioterrorism: An Unprecedented of Employment” 2013; Nick Bostrom, Superintelligence :
World,” Survival 46, no. 2 (2004): 148–149. Paths, Dangers, Strategies (Oxford: Oxford University
102. Sander Herfst et al., “Airborne Transmission of Influenza Press, 2014), chap. 11.
A/H5N1 Virus between Ferrets,” Science 336, no. 6088 120. Bostrom, Superintelligence, chap. 4.
(2012): 1534–41; Masaki Imai et al., “Experimental
Adaptation of an Influenza H5 HA Confers Respiratory 121. For extensive discussion of the ‘AI control problem’ see
Droplet Transmission to a Reassortant H5 HA/H1N1 Virus Bostrom, Superintelligence.
in Ferrets,” Nature 486, no. 7403 (June 21, 2012): 420–28, 122. Bostrom, Superintelligence, 18–21. For more detail about
doi:10.1038/nature10831. the surveys see Vincent C. Müller and Nick Bostrom,
103. Nicholas Greig Evans, Marc Lipsitch, and Meira “Future Progress in Artificial Intelligence: A Survey of
Levinson, “The Ethics of Biosafety Considerations in Expert Opinion,” in Fundamental Issues of Artificial
Gain-of-Function Research Resulting in the Creation Intelligence, ed. Vincent C. Müller (Berlin: Synthese
of Potential Pandemic Pathogens,” Journal of Medical Library; Springer, 2014).
Ethics 41, no. 11 (November 1, 2015): 901–8, doi:10.1136/ 123. Bostrom, Superintelligence, 18–21
medethics-2014-102619.
124. Bostrom, Superintelligence, 20–21.
104. Richard A. Posner, Catastrophe : Risk and Response
125. Shepherd, Geoengineering the Climate, chap. 2.
(Oxford: Oxford University Press, 2004), 78–79.
126. Shepherd, Geoengineering the Climate.
105. George Church, “A Synthetic Biohazard Non-Proliferation
Proposal,” 2004, http://arep.med.harvard.edu/SBP/ 127. Alan Robock, Kirsten Jerch, and Martin Bunzl, “20 Reasons

ENDNOTES
Why Geoengineering May Be a Bad Idea,” Bulletin of the factor, and is discussed in section 3.2.
Atomic Scientists 64, no. 2 (2008): 14–59. 145. Centre for Science Policy, University of Cambridge et al.,
128. Shepherd, Geoengineering the Climate, chap. 3. For “Climate Change–a Risk Assessment,” 38; International
criticism of the ‘climate emergency’ argument for Energy Agency and Nuclear Energy Agency, “Technology
SRM see Oliver Morton, The Planet Remade : How Roadmap: Nuclear Energy,” 2015; Ottmar Edenhofer and
Geoengineering Could Change the World (London: Intergovernmental Panel on Climate Change. Working
Granta, 2015), chap. 6; Jana Sillmann et al., “Climate Group III, Climate Change 2014 : Mitigation of Climate
Emergencies Do Not Justify Engineering the Climate,” Change : Working Group III Contribution to the Fifth
Nature Climate Change 5, no. 4 (2015): 290–92. Assessment Report of the Intergovernmental Panel on
129. Scott Barrett, “The Incredible Economics of Climate Change (New York, NY: Cambridge University
Geoengineering,” Environmental and Resource Press, 2015), 516–518.
Economics 39, no. 1 (2008): 45–54. 146. Edenhofer and Intergovernmental Panel on Climate
130. Alan Robock, Luke Oman, and Georgiy L. Stenchikov, Change. Working Group III, Climate Change 2014, 531.
“Regional Climate Responses to Geoengineering 147. According to Sailor et al “development of nuclear weapons
with Tropical and Arctic SO2 Injections,” Journal of has been aided in at least three countries (India, Iraq, and
Geophysical Research: Atmospheres 113, no. D16 Israel) by use of research reactors obtained under the
(August 27, 2008): D16101, doi:10.1029/2008JD010050. cover of peaceful research programs”.William C. Sailor et
131. Morton, The Planet Remade, chap. 4.. al., “A Nuclear Solution to Climate Change?,” Science 288,
no. 5469 (2000): 1177–78. For a very good overview of the
132. Shepherd, Geoengineering the Climate, chap. 4. technical and political aspects of the impact of nuclear
133. Ibid., chap. 3. power on nuclear proliferation see A. Adamantiades and
I. Kessides, “Nuclear Power for Sustainable Development:
134. Morton, The Planet Remade, chap. 4.
Current Status and Future Prospects,” Energy Policy
135. Seth D. Baum, Timothy M. Maher Jr, and Jacob Haqq- 37, no. 12 (December 2009): 5149–66, doi:10.1016/j.
Misra, “Double Catastrophe: Intermittent Stratospheric enpol.2009.07.052.
Geoengineering Induced by Societal Collapse,”
148. Edwin Dennis Kilbourne, “Plagues and Pandemics: Past,
Environment Systems & Decisions 33, no. 1 (January 8,
Present, and Future,” in Global Catastrophic Risks, ed.
2013): 168–80, doi:10.1007/s10669-012-9429-y.
Nick Bostrom and Milan M. Ćirković (Oxford: Oxford
136. Shepherd, Geoengineering the Climate, chap. 4. University Press, 2008).
137. See for example Shepherd, Geoengineering the Climate; 149. James M. Hughes et al., “The Origin and Prevention of
National Academy of Sciences, Climate Intervention: Pandemics,” Clinical Infectious Diseases 50, no. 12 (June
Reflecting Sunlight to Cool Earth (Washington, D.C.: 15, 2010): 1636–40, doi:10.1086/652860.
National Academies Press, 2015), http://www.nap.edu/
150. Kate E. Jones et al., “Global Trends in Emerging Infectious
catalog/18988; Solar Radiation Management Governance
Diseases,” Nature 451, no. 7181 (February 21, 2008):
Initiative et al., Solar Radiation Management: The
990–93, doi:10.1038/nature06536.
Governance of Research (SRMGI, 2012).
151. Peter Sands, Carmen Mundaca-Shah, and Victor J.
138. Shepherd, Geoengineering the Climate; Barrett, “The
Dzau, “The Neglected Dimension of Global Security — A
Incredible Economics of Geoengineering.”
Framework for Countering Infectious-Disease Crises,”
139. Arnon Dar, “Influence of Supernovae, Gamma-Ray New England Journal of Medicine 0, no. 0 (January 13,
Bursts, Solar Flares, and Cosmic Rays on the Terrestrial 2016): null, doi:10.1056/NEJMsr1600236.
Environment,” in Global Catastrophic Risks, ed. Nick
152. David Charles Denkenberger and Joshua Pearce, Feeding
Bostrom and Milan M. Ćirković (Oxford: Oxford
Everyone No Matter What : Managing Food Security after
University Press, 2008).
Global Catastrophe (Amsterdam: Academic Press, 2015).
140. Bryan Caplan, “The Totalitarian Threat,” in Global
153. S. Sparks et al., Super-Eruptions: Global Effects and
Catastrophic Risks, ed. Nick Bostrom and Milan M.
Future Threats. London, Report of a Geological Society of
Ćirković (Oxford: Oxford University Press, 2008).
London Working Group, 2005; National Research Council
141. Seth D. Baum, “Winter-Safe Deterrence: The Risk of (U. S.). Committee to Review Near-Earth-Object Surveys
Nuclear Winter and Its Challenge to Deterrence,” and Hazard Mitigation Strategies, Defending Planet
Contemporary Security Policy 36, no. 1 (January 2, 2015): Earth: Near-Earth Object Surveys and Hazard Mitigation
123–48, doi:10.1080/13523260.2015.1012346. Strategies (Washington, DC: National Academies Press,
142. Joseph Cirincione, “The Continuing Threat of Nuclear 2010), chap. 2 and 5.
War,” in Global Catastrophic Risks, ed. Nick Bostrom and 154. David King et al., “Climate Change–a Risk Assessment”
Milan M. Ćirković (Oxford University Press, 2008). (Centre for Science Policy, University of Cambridge,
143. Pavel Podvig, “Reducing the Risk of an Accidental 2015), pt. 1, www.csap.cam.ac.uk/projects/climate-
Launch,” Science & Global Security 14, no. 2–3 (December change-risk-assessment/.
1, 2006): 75–115, doi:10.1080/08929880600992990. 155. Sybren Drijfhout et al., “Catalogue of Abrupt Shifts in
144. For an overview of the literature on ‘nuclear winter’ Intergovernmental Panel on Climate Change Climate
see Seth D. Baum, “Winter-Safe Deterrence: The Risk Models,” Proceedings of the National Academy of
of Nuclear Winter and Its Challenge to Deterrence,” Sciences 112, no. 43 (2015): E5777–86
Contemporary Security Policy 36, no. 1 (January 2, 2015): 156. For a qualified defence of this position see Oliver Morton,
123–48, doi:10.1080/13523260.2015.1012346. The The Planet Remade : How Geoengineering Could Change
resilience of the food supply is also an important risk the World (London: Granta, 2015).

157. J. G. Shepherd, Geoengineering the Climate: Science, Global Public Goods (Oxford: Oxford University Press,
Governance and Uncertainty (Royal Society, 2009). 2007).
158. For discussions of worries about arms races and emerging 169. Clean air can of course also be influenced by pollution in
technologies see Christopher F. Chyba, “Biotechnology and neighbouring countries.
the Challenge to Arms Control,” Arms Control Today 36, 170. Elinor Ostrom, Governing the Commons: the evolution
no. 8 (2006), http://search.proquest.com/openview/3251 of institutions for collective action. 1990. Cambridge
1e461cdf35678bea5679269ac45b/1?pq-origsite=gscholar; University Press.
Future of Life Institute, “Open Letter on Autonomous
Weapons,” accessed March 18, 2016, http://futureoflife. 171. It is hotly disputed how we should go about taking
org/open-letter-autonomous-weapons/; Gernot Wagner the interests of future generations into account, but it
and Martin L. Weitzman, Climate Shock : The Economic seems plausible that they should be given at least some
Consequences of a Hotter Planet (Princeton: Princeton consideration in public policy. Indeed, this is a standard
University Press, 2015), chap. 3. assumption in many governmental cost-benefit analyses.
159. Nick Bostrom, Superintelligence : Paths, Dangers, 172. Mancur Olson, The Logic of Collective Action : Public
Strategies (Oxford: Oxford University Press, 2014); Ali Goods and the Theory of Groups, Harvard Economic
Nouri and Christopher F. Chyba, “Biotechnology and Studies ; v. 124 (Cambridge, Mass: Harvard University
Biosecurity,” in Global Catastrophic Risks, ed. Nick Press, 1965).
Bostrom and Milan M. Ćirković (Oxford University Press, 173. This is an example of availability bias, which is
2008). discussed along with other cognitive biases that can
160. For a discussion of this issue with respect to skew perceptions of global risks in Eliezer Yudkowsky,
biotechnology and geo-engineering risks respectively see “Cognitive Biases Potentially Affecting Judgment of Global
W. Paul Duprex et al., “Gain-of-Function Experiments: Risks,” in Global Catastrophic Risks, ed. Nick Bostrom
Time for a Real Debate,” Nature Reviews Microbiology 13, and Milan M. Ćirković, vol. 1 (Oxford: Oxford University
no. 1 (January 2015): 58–64, doi:10.1038/nrmicro3405; Press, 2008).
Oliver Morton, The Planet Remade : How Geoengineering 174. Barrett, Why Cooperate?, chap. 2.
Could Change the World (London: Granta, 2015).
175. WHO, “Summary of States Parties 2013 Report on IHR
161. Bostrom, Superintelligence. Core Capacity Implementation,” 2014, 6, http://www.who.
162. Bostrom, N. “Analyzing Human Extinction Scenarios and int/ihr/publications/WHO_HSE_GCR_2014.10/en/.
Related Hazards”. Journal of Evolution and Technology. 176. Lawrence O. Gostin and Devi Sridhar, “Global Health and
Vol. 9. No. 1. (2002) the Law,” New England Journal of Medicine 370, no. 18
163. Philip Nel and Marjolein Righarts, “Natural Disasters and (May 1, 2014): 1732–40, doi:10.1056/NEJMra1314094.
the Risk of Violent Civil Conflict,” International Studies 177. Barrett, Why Cooperate?
Quarterly 52, no. 1 (March 1, 2008): 159–85, doi:10.1111/
178. Treaty on the Non-Proliferation of Nuclear Weapons,
j.1468-2478.2007.00495.x.
Washington/Moscow/London, 1 July 1968, in force 5
164. For an overview see Halvard Buhaug, Nils Petter March 1970, 729 UNTS 161
Gleditsch, and Ole Magnus Theisen, “Implications of
179. United Nations Framework Convention on Climate
Climate Change for Armed Conflict” (World Bank, 2009);
Change, Rio de Janeiro, 9 May 1992, in force 21 March
Ragnhild Nordås and Nils Petter Gleditsch, “Climate
1994, 1771 UNTS 107. This convention provided the
Change and Conflict,” Political Geography, Climate
platform for the Kyoto and Paris climate change
Change and Conflict, 26, no. 6 (August 2007): 627–38,
agreements.
doi:10.1016/j.polgeo.2007.06.003.
180. The IAEA’s mandate is limited to civil nuclear applications,
165. BBC, “Working Gun Made with 3D Printer,” BBC News,
but it has probably reduced the likelihood of nuclear
accessed February 22, 2016, http://www.bbc.co.uk/news/
terrorism by imposing detailed nuclear fissile accounting.
science-environment-22421185.
This verification process as well as on-site inspection also
166. See Chapter 2 and Alan Robock, Luke Oman, and make it very difficult for non-nuclear-weapon states to
Georgiy L. Stenchikov, “Nuclear Winter Revisited with a develop nuclear weapons without detection.
Modern Climate Model and Current Nuclear Arsenals:
181. United Nations General Assembly, “Protection of Global
Still Catastrophic Consequences,” Journal of Geophysical
Climate for Present and Future Generations of Mankind,”
Research: Atmospheres 112, no. D13 (July 16, 2007):
December 1988, http://www.ipcc.ch/docs/UNGA43-53.pdf.
D13107, doi:10.1029/2006JD008235; National Research
Council (U. S.). Committee to Review Near-Earth-Object 182. Gernot Wagner and Martin L. Weitzman, Climate
Surveys and Hazard Mitigation Strategies, Defending Shock : The Economic Consequences of a Hotter Planet
Planet Earth: Near-Earth Object Surveys and Hazard (Princeton: Princeton University Press, 2015).
Mitigation Strategies (Washington, DC: National 183. See for example Alan Robock, “Nuclear Winter,” Wiley
Academies Press, 2010), chap. 2; S. Sparks et al., Super- Interdisciplinary Reviews: Climate Change 1, no. 3 (May 1,
Eruptions: Global Effects and Future Threats. London, 2010): 418–27, doi:10.1002/wcc.45.
Report of a Geological Society of London Working Group,
184. Nick Bostrom, Superintelligence : Paths, Dangers,
2005.
Strategies (Oxford: Oxford University Press, 2014).
167. Denkenberger and Pearce, Feeding Everyone No Matter
185. For example, see the discussion of the responsibility
What.
of research laboratories conducting gain of function
168. For an excellent discussion of global public goods see research in W. Paul Duprex et al., “Gain-of-Function
Scott Barrett, Why Cooperate? : The Incentive to Supply Experiments: Time for a Real Debate,” Nature

ENDNOTES
Reviews Microbiology 13, no. 1 (January 2015): 58–64, 69–79, doi:10.1016/j.futures.2015.03.004.

doi:10.1038/nrmicro3405. 195. The WHO’s International Health Regulations were an
186. Charles Arthur, “DeepMind: ‘Artificial Intelligence Is a Tool important development in this area, but the rules and
That Humans Can Control and Direct,’” The Guardian, their implementation could be improved. See Rebecca
June 9, 2015, sec. Technology, http://www.theguardian. Katz and Scott F Dowell, “Revising the International
com/technology/2015/jun/09/deepmind-artificial- Health Regulations: Call for a 2017 Review Conference,”
intelligence-tool-humans-control. The Lancet Global Health 3, no. 7 (July 2015): e352–53,
187. Anja Wolz, “Face to Face with Ebola — An Emergency doi:10.1016/S2214-109X(15)00025-X.
Care Center in Sierra Leone,” New England Journal of 196. Ibid.
Medicine 371, no. 12 (September 18, 2014): 1081–83, 197. See National Research Council (U. S.). Committee to
doi:10.1056/NEJMp1410179. Review Near-Earth-Object Surveys and Hazard Mitigation
188. T
. Frängsmyr, and I. Abrams, I., 1997. Peace, 1981-1990 Strategies, Defending Planet Earth, chap. 2.
(Vol. 5). World Scientific. 198. Sparks et al., Super-Eruptions..
189. National Research Council (U. S.). Committee to Review 199. See the discussion in Duprex et al., “Gain-of-Function
Near-Earth-Object Surveys and Hazard Mitigation Experiments.”.
Strategies, Defending Planet Earth: Near-Earth Object
Surveys and Hazard Mitigation Strategies (Washington, 200. Nouri and Chyba, “Biotechnology and Biosecurity.”
DC: National Academies Press, 2010), chap. 2. 201. Bostrom, Superintelligence.
190. For a defence of the importance of tail risk climate 202. Shepherd, Geoengineering the Climate.
change see Martin L. Weitzman, “A Review of ‘The Stern
203. For discussion of different particulate winter scenarios
Review on the Economics of Climate Change”, Journal of
see Timothy M. Maher and Seth D. Baum, “Adaptation to
Economic Literature 45, no. 3 (2007): 703–24.
and Recovery from Global Catastrophe,” Sustainability 5,
191. Wagner and Weitzman, Climate Shock, chap. 3. no. 4 (March 28, 2013): 1461–79, doi:10.3390/su5041461.
192. See Meinshausen et al., “Greenhouse-Gas Emission 204. Ibid.
Targets for Limiting Global Warming to 2 °C.”.
205. See for example David Charles Denkenberger and Joshua
193. On this see Pavel Podvig, “Reducing the Risk Pearce, Feeding Everyone No Matter What : Managing
of an Accidental Launch,” Science & Global Food Security after Global Catastrophe (Amsterdam:
Security 14, no. 2–3 (December 1, 2006): 75–115, Academic Press, 2015).
doi:10.1080/08929880600992990.
206. Nick Bostrom, “Existential Risk Prevention as Global
194. On this see Seth D. Baum, “Confronting the Threat Priority,” Global Policy 4, no. 1 (February 1, 2013): 15–31,
of Nuclear Winter,” Futures, Confronting Future doi:10.1111/1758-5899.12002.
Catastrophic Threats To Humanity, 72 (September 2015):

ACKNOWLEDGEMENTS
Acknowledgements
CONTACT INFO
Authors:
Global Priorities Project
16-17 St. Ebbes Street
Oxford, OX1 1PT
United Kingdom
contact@globalprioritiesproject.org
The Global Challenges Foundation:

Strandvägen 7A
114 56 Stockholm
Sweden
info@ globalchallenges.org
+46 (0) 709 98 97 97


Global Catastrophic Risk Annual Report 2016 FINAL

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Global Catastrophic Risk Annual Report 2016 FINAL

Uploaded by

Copyright:

Available Formats

Global

Global Catastrophic Risks

© Global Challenges Foundation/Global Priorities Project 2016

* = The Global Priorities Project

4 Global Catastrophic Risks 2016

THE GLOBAL PRIORITIES PROJECT helps decision-makers effectively prior-

Global Catastrophic Risks 2016 5

6 Global Catastrophic Risks 2016

Global Catastrophic Risks 2016 7

8 Global Catastrophic Risks 2016

Stockholm, April 2016

Global Catastrophic Risks 2016 9

Global catastrophic risks

10 Global Catastrophic Risks 2016

Global Catastrophic Risks 2016 11

12 Global Catastrophic Risks 2016

Global Catastrophic Risks 2016 13

What are the biggest threats?

14 Global Catastrophic Risks 2016

Missile Crisis, President Kennedy from what is expected or if positive

16 Global Catastrophic Risks 2016

Unknown risks Nuclear war

Low High APPROPRIATE LEVEL

Humanity can respond to these risks

18 Global Catastrophic Risks 2016

20 Global Catastrophic Risks 2016

Global Catastrophic Risks 2016 21

1.1. Defining Global

22 Global Catastrophic Risks 2016

Global warming Depletion of the

Recession in one National

Loss of one hair Loss of one arm Fatal car crash

Imperceptible Damaging Catastrophic

1.2. Why Global

24 Global Catastrophic Risks 2016

Global Catastrophic Risks 2016 25

1.3. Why Global Catastrophic

26 Global Catastrophic Risks 2016

1.5. How to read this report

Global Catastrophic Risks 2016 27

What are the Most

28 Global Catastrophic Risks 2016

Global Catastrophic Risks 2016 29

levels of climate change sufficient to most likely to follow a medium-high

32 Global Catastrophic Risks 2016

Global Catastrophic Risks 2016 33

ty). On the medium-high emissions approximately linear relationship

34 Global Catastrophic Risks 2016

Probability of warming >6°C

2.2. Nuclear war

creating an unprecedented famine. between one out of three and even”.5!

38 Global Catastrophic Risks 2016

Number of nuclear warheads

NUCLEAR NEAR MISSES

There have been numerous other examples of nuclear near misses.

the likelihood of international nu- could work to ensure that terrorists

40 Global Catastrophic Risks 2016

2.3. Natural pandemics

44 Global Catastrophic Risks 2016

PANDEMIC DATE(S) LETHAL IMPACT

Plague of Justinian AD 541-542 25-33m (13-17% of the world population)

Black Death 14th Century 50-75m (11-17% of world population)

Smallpox 1520-1527 200,000 deaths within the Aztec population