Professional Documents
Culture Documents
Super Intelligence
Super Intelligence
Basel, Switzerland
22 November 2014
Superintelligence
Our Final Invention
Outline
Introduction
Singularity
Superintelligence
Strategy
Sources
Summary
Superintelligence
Our Final Invention
Introduction
What are we talking about?
Superintelligence
Our Final Invention
Crucial Consideration
an idea or argument that entails a
major change of direction or priority.
Superintelligence
Introduction
Evolution
Fitness
Function
Global Optimum
Feature
Space
Replicator
x
x
x
x
x
x x x x
Local Optimum
Superintelligence
Introduction
Stability
Fitness
Fitness
Disturbance
Variation
Disturbance
Variation
Attractors
Maturity
Past
Technological Maturity
(Singleton)
Instability
Life
Extinction
0
Big Bang
Time
9 billion years
13.8 billion
15 20 billion
Superintelligence
Introduction
Singleton
ultimate fate?
arms races
Darwinism
Disadvantages:
It might result in a
dystopian world
durable lock-in
Superintelligence
Introduction
~ 3 1023 stars
100000000000
300000000000000000000000
Universe
en.wikipedia.org/wiki/Universe
Superintelligence
Introduction
10
Fermi Paradox
Where are they? (Extraterrestrial Life)
Superintelligence
Introduction
11
Great Filter
Life
Were rare:
Transitions
Us
Were doomed:
The Fermi Paradox
waitbutwhy.com/2014/05/fermi-paradox.html
!
Superintelligence
Introduction
12
Major Transitions
Sexual reproduction
Multi-cell organisms
Tool-using animals
Space colonization
Superintelligence
Introduction
13
Anthropic Principle
How probable are these transitions?
Superintelligence
Introduction
14
Technologies
?
By definition:
Superintelligence
Introduction
15
Candidates
Nuclear Weapons (still possible)
Molecular Nanotechnology
Machine Intelligence
Geoengineering
Unknown
Global Catastrophic Risks
www.global-catastrophic-risks.com
Superintelligence
Introduction
16
Intelligence
Intelligence measures an agents
ability to achieve its goals in a wide
range of unknown environments.
(adapted from Legg and Hutter)
Intelligence =
Universal Intelligence
arxiv.org/pdf/0712.3329.pdf
Optimization Power
Used Resources
Superintelligence
Introduction
17
Ingredients
Epistemology: Learn model of world
Superintelligence
Introduction
18
Consciousness
is a completely separate question!
Consciousness is
reducible or
fundamental
and universal
How Do You Explain Consciousness?
David Chalmers: go.ted.com/DQJ
Superintelligence
Introduction
19
Machine Sentience
Open questions of immense importance:
If yes:
Superintelligence
Introduction
20
Singularity
What is the basic argument?
Superintelligence
Our Final Invention
21
Feedback
Systems can feed back into themselves
and thus must be analyzed as a whole!
Feedback
en.wikipedia.org/wiki/Feedback
Feedback is either:
Positive (reinforcing)
Negative (balancing)
Superintelligence
Singularity
22
Exponential Functions
If increase is linear to current amount:
d
f (x) = c f (x)
dx
solved by
f (x) = e
cx
(0,1)
(1,e)
Superintelligence
Singularity
23
Climate Change
Absorb less CO2
Warmer oceans
Rising temperature
Less reflection
Less ice
Melting ice
Climate Change Feedback
en.wikipedia.org/wiki/Climate_change_feedback
Superintelligence
Singularity
24
Superintelligence
Singularity
25
Accelerating Change
Progress feeds on itself:
Knowledge
Technology
Technology
?
Rate of Progress
in the year 2000
0
2000 2100
Time in years AD
20000
Superintelligence
Singularity
26
Moores Law
Superintelligence
Singularity
27
Artificial Mind
Superintelligence
Singularity
28
Hyperbolic Growth
Second-order positive feedback loop
d
1
2
f (t) = c f (t) ) f (t) =
dt
ct
f(t)
Singularity
t
Mathematical Singularity
en.wikipedia.org/wiki/Singularity_(mathematics)
0
Superintelligence
Singularity
29
Speed Explosion
Time
Speed
2 years
Objective Time
1 year
6m 3m
Superintelligence
Singularity
30
Population Explosion
Quan
titativ
2 years
Time
1 year
6m 3m
31
Intelligence Explosion
Qual
itativ
Intelligence Explosion
intelligence.org/files/IE-EI.pdf
Superintelligence
Singularity
32
more people
Intelligence
more speed
more research
better algorithms
more time
Population
better research
Superintelligence
Singularity
33
Technological Singularity
Theoretic phenomenon: There are
arguments why it should exist but it has
not yet been confirmed experimentally.
Superintelligence
Singularity
34
Superintelligence
What are potential outcomes?
Superintelligence
Our Final Invention
35
Definition of Superintelligence
An agent is called superintelligent if
it exceeds the level of current human
intelligence in all areas of interest.
Rock
Chimp Genius
Superintelligence
Mouse
Fool
Superintelligence
Superintelligence
36
biological cognition
brain-computer interfaces
Strong Superintelligence
artificial intelligence
neuromorphic
synthetic
Weak Superintelligence
Pathways to Superintelligence
Superintelligence
Superintelligence
37
Size
Speed
Memory
Software:
Editability
Copyability
Expandability
Human Brain
86 billion neurons
Effectiveness:
Rationality
Coordination
Communication
Modern Microprocessor
1.4 billion transistors
4400000000 Hz
300000000 m/s
Superintelligence
Superintelligence
38
Cognitive Superpowers
Intelligence amplification: bootstrapping
Superintelligence
Superintelligence
39
Orthogonality Thesis
Intelligence
likelier because easier
Paperclip
Maximizer
Adolf
Hitler
Mahatma
Gandhi
Final Goals
Superintelligence
Superintelligence
40
necessary to
achieve goal
Resource Accumulation
Intelligence Accumulation
to achieve
goal better
Superintelligence
Superintelligence
41
Single-Shot Situation
Our first superhuman AI must be a safe
one for we may not get a second chance!
Were good at iterating with testing and feedback
Superintelligence
Superintelligence
42
Takeoff Scenarios
Intelligence
Physical Limit
Superintelligence
Separate Questions!
Human Level
?
Feedback
now
takeo duration
Time
Superintelligence
Superintelligence
43
Potential Outcomes
Fast Takeoff
Slow Takeoff
Unipolar Outcome
Multipolar Outcome
Singleton
(Slide 9)
Second Transition
Unification by Treaty
44
Superintelligence
Our Final Invention
45
Computer
Consciousness sequential
Software parallel
Mindware parallel
Hardware sequential
easy
Pattern Recognition
hard
hard
easy
GPU
Superintelligence
State and Trends
46
Superhuman
Backgammon Superhuman
Stanley: 2005
Schmidhuber: 2011
Othello
Superhuman
Chess
Superhuman
Crosswords
Expert Level
Scrabble
Superhuman
Bridge
Equal to Best
Jeopardy!
Superhuman
Poker
Varied
FreeCell
Superhuman
Go
Strong Amateur
47
Consumer Products
Raffaello D'Andrea
go.ted.com/xeh
Superintelligence
State and Trends
48
Military Robots
P.W. Singer
go.ted.com/xe3
Superintelligence
State and Trends
49
Financial Markets
High-frequency trading (HFT): Buy and sell
securities within millseconds algorithmically
Superintelligence
State and Trends
50
Machine Learning
Superintelligence
State and Trends
51
o k rk
... max
am
(rk + ... + rm )
o m rm
l(q)
Superintelligence
State and Trends
52
Predicting AI Timelines
Great uncertainties:
10%
50%
90%
AI scientists, median
2024
2050
2070
2030
2070
2140
Superintelligence
State and Trends
53
Speed Bumps
Depletion of low-hanging fruit
Societal collapse
Disinclination
Superintelligence
State and Trends
54
Accelerators
Faster hardware
Better algorithms
Massive datasets
+ enormous incentives!
Superintelligence
State and Trends
55
Economic Incentives
more data
more users
better AI
Superintelligence
State and Trends
56
Economic Consequences
The living costs of digital workers
are drastically lower (just energy)
Superintelligence
State and Trends
57
more
funding
better
intelligence
Daniel Suarez
go.ted.com/Brd
better
predictions
Superintelligence
State and Trends
58
Egoistic Incentives
Intelligence
Wellbeing
Longevity
Willing to
take risks
But with great power comes great responsibility!
PostHuman: An Introduction to Transhumanism
www.youtube.com/watch?v=bTMS9y8OVuY
Superintelligence
State and Trends
59
Strategy
What is to be done?
Superintelligence
Our Final Invention
60
Prioritization
Scope: How big/important is the issue?
Superintelligence
Strategy
61
Flow-Through Effects
Going meta: Solve the problem-solving problem!
Extreme Poverty
Factory Farming
Climate Change
could
solve
other
issue
Artificial Intelligence
Holden Karnofsky: Flow-Through Effects
blog.givewell.org/2013/05/15/flow-through-effects/
Superintelligence
Strategy
62
Controlled Detonation
Difficulty:
Friendly AI >> General AI
AI as a Positive and Negative Factor in Global Risk
intelligence.org/files/AIPosNegFactor.pdf
Superintelligence
Strategy
63
Control Problem
Will AI o
utsmar
t us?
Capability Control
Motivation Selection
Boxing
Direct Specification
Stunting
Indirect Normativity
Tripwires
Incentive Methods
Superintelligence
Strategy
64
Superintelligence
Our Final Invention
65
Value Loading
Utility function of AI?
Perverse instantiation
Moral blind-spots?
Superintelligence
Strategy
66
Superintelligence
Strategy
67
Stable Self-Improvement
Friendly
MIRI Research Results
intelligence.org/research/
Friendly?
Superintelligence
Strategy
68
12'000
5'000
10'000
15'000
Superintelligence
Strategy
69
Order of Arrival
100%
75%
50%
25%
0%
Nanotechnology
Superintelligence
Total
100%
75%
50%
25%
0%
AI determines transition
Superintelligence
Existential Risk
www.existential-risk.org
Biotechnology
Nanotechnology
=
Total
Superintelligence
Strategy
70
Information Hazards
Research can
Superintelligence
Strategy
71
Creating Awareness
Outreach can
create awareness
Superintelligence
Strategy
72
Prisoners Dilemma
Dicult to prevent arms races
Superintelligence
Strategy
73
International Cooperation
We are the ones who will
create superintelligent AI
International regulation?
In face of uncertainty, cooperation is robust!
Lower Bound on the Importance of Promoting Cooperation Superintelligence
foundational-research.org/[]/[]-promoting-cooperation/
Strategy
74
Moral Trade
Compromise!
Superintelligence
Strategy
75
Superintelligence
Strategy
76
Sources
Where to learn more?
Superintelligence
Our Final Invention
77
Nick Bostrom
Eliezer Yudkowsky
Brian Tomasik
Superintelligence
Sources
78
Talks
Daniel Dewey
TEDxVienna
Jrgen Schmidhuber
TEDxLausanne
Superintelligence
Sources
79
Papers
Intelligence Explosion by Luke
Muehlhauser and Anna Salamon
Superintelligence
Sources
80
Books
Superintelligence
Sources
81
Summary
What have we learned?
Superintelligence
Our Final Invention
82
Crucial Crossroad
Instead of passively drifting,
we need to steer a course!
Philosophy
Mathematics
Cooperation
with a deadline.
Luke Muehlhauser: Steering the Future of AI
intelligence.org/[]Steering-the-Future-of-AI.pdf
Superintelligence
Our Final Invention
83
Superintelligence
Our Final Invention
84
Discussion
www.superintelligence.ch
Basel, Switzerland
22 November 2014
85