Why cant I innovate

in my wiring closet?
MIT, April 17, 2008

Nick McKeown
nickm@stanford.edu

The Stanford Clean Slate Program Funded by: Cisco, DT

http://cleanslate.stanford.edu DoCoMo, NEC, Xilinx
 Outline
Routing doesnt belong in routers

Substrates to enable innovation

OpenFlow as a streamlined substrate

2
 Internet Innovation
End to end arguments led to the principle:
Move function up out of the network to the end-points

Led to streamlined substrate of links and

routers
Even congestion control done at end-points

Led to enormous innovation on top

Email, WWW, VOIP, P2P, Social networks,

3
 A fixed substrate
Fierce protection against overloading
network, unless necessary

Change happens slowly or surreptitiously

IPv6, Multicast, NAT
An era of fixed substrates
IP, Intel instruction set, ms-dos
Stable fixed substrates
Standards that bred innovation
Its hard to imagine it having been any other way.

4
 How streamlined is the substrate?
Feature lists as long as your arm
Routers with >10M lines of source code,
datapath with 100M gates & 1Gbyte RAM.

Many complex functions in the infrastructure

OSPF, BGP, multicast, differentiated services,
Traffic Engineering, NAT, firewalls, MPLS, redundant layers,

Claim
Most complexity is about routing: picking paths,
managing location, identity and access.
The current network controls the routing.
Routing doesnt belong in the boxes.
If we control routing, we can innovate.

Uh-oh, looks like another GENI talk

5
 Routing inside routers
It made sense at the time
Early on, seemed necessary for scalability & robustness

Routing doesnt need to be in the network

Trivial example: source routing

Routing doesnt belong in the network

Aside: Electricity grid of lines, transformers and switches

Stakeholders should control routing and innovation

Why cant I add my own routing protocol? Or mobility
management protocol?
Why do you and I have to run the same routing protocol?

4D [Zhang], RCP [Rexford] and others

6
 Local equilibrium
Inelegantclosed design (like a bad API)
leads to complexity, and fragility. Hard to
innovate.
Resistant to change

High
Inelegance Industry, IETF, Barrier to
Entry

Add complexity

Not a conspiracy, just a local equilibrium

7
 Why should we care?
There are basic things the Internet cant do well
Mobility, access control/security, management,

Why not just add more features to fix it?

An uneasy feeling that we are piling more and more
complexity into the network, making it overloaded,
fragile, less elegant and harder to innovate.

We need a simple, reliable, constant substrate

Oh my god it really does look like another GENI talk

8
 Current substrate
On a PC substrate, we can choose an OS.
And now we have virtualization too.

In the network world, your switch/router comes

bundled and you have no choice.

So
1. Alternative OS on your switch/router?
2. Virtualization of your switch/router?

9
 Outline
Routing doesnt belong in routers

Substrates to enable innovation

1. Open OSs on switches/routers
2. Virtualization of switches/routers

OpenFlow as a streamlined substrate

10
 Innovation by ripping off the lid

XORP (and Zebra; or just Linux)

Open-source router code for Linux, FreeBSD, etc.
Flexible, easy to use
Software-based; limited performance
Point solution

NetFPGA
Small (4x1GE) router; open-source hardware and software
Flexible, low-cost, medium ease of use
Hardware-based; line-rate performance
Point solution, limited port count (4-8)

OpenWRT
Embedded Linux for wireless APs and routers

11
 Open source networking hardware

CPU Memory
Memory

Hardware available from 3rd party ~$500 (universities)

PCI PC with NetFPGA
Line-rate forwarding
PCI board, or pre-built system (desktop or rack-mount)
500
1GEboards, 15 countries; expect 2,000 this year
FPGA 1GE
Free reference designs, gateware and courseware
1GE
Memory
Memory
1GE Rack of 1U servers
96 x 1GE ports

NetFPGA Board

12
 Outline
Routing doesnt belong in routers

Substrates to enable innovation

1. Open OSs on switches/routers
2. Virtualization of switches/routers

OpenFlow as a streamlined substrate

13
 Innovation by changing the substrate

Diverse applications Diverse applications

Diverse transport layers Diverse transport layers

IP X Y Z
IP Virtualization layer

Diverse link layers Diverse link layers

Diverse physical layers Diverse physical layers

This is the GENI approach: Innovation from the top down

e.g. WUSTL SPP, VINI,

Proposed approach: OpenFlow innovation from the bottom up

14
 OpenFlow Model
Allow lots of innovative research experiments

Diverse applications
Routing, Mobility,
Diverse transport layers
Naming/Addressing,
Ethernet IP X Y Z
Access Control,
Flow layer
Management,
Monitoring Collapse difference; ease use of optical circuits
Packet switching and circuit switching
Diverse link layers

Diverse physical layers

15
 OpenFlow Switching
A way to innovate in the networks we use everyday.

A pragmatic compromise
Allow researchers to run experiments in their network
without requiring vendors to expose internal workings.

1. Work with switch and AP vendors to add OpenFlow to

their products
2. Deploy on university campuses
3. Stand back and watch students innovate

Basics
An Ethernet switch (e.g. 128-ports of 1GE)
Use flow-table already in every switch and chipset
An open protocol to remotely add/remove flow entries
16
 What wed like
Isolation: Regular production traffic untouched
Virtualized and programmable: Different flows
processed in different ways
Equipment we can trust in our wiring closet
Open development environment for all
researchers (e.g. Linux, Verilog, etc).
Flexible definitions of a flow
Individual application traffic
Aggregated flows
Alternatives to IP running side-by-side

17
 OpenFlow Switching Controller
OpenFlow Switch specification

OpenFlow Switch n Flow PC

Ope ocol
t
Pro
SSL
sw Secure
Channel
Add/delete flow entry
hw Flow
Encapsulated packets
Controller discovery
Table

18
 Flow Table Entry
Type 0 OpenFlow Switch

Rule Action Stats

Packet + byte counters

1. Forward packet to port(s)

2. Encapsulate and forward to controller
3. Drop packet
4. Send to normal processing pipeline

Switch MAC MAC Eth VLAN IP IP IP TCP TCP

Port src dst type ID Src Dst Prot sport dport
+ mask

19
 OpenFlow Type 1
(future)
More flexible header
Allow arbitrary matching of first few bytes
Additional actions
Rewrite headers (e.g. NAT, or address
obfuscation)
Map to queue/class
Encrypt
Support multiple controllers
Load-balancing and robustness

20
 OpenFlow Consortium
http://OpenFlowSwitch.org

Goal: Evangelize OpenFlow to vendors

Free membership for all researchers

Whitepaper, OpenFlow Switch Specification,

Reference Designs

Licensing: Free for research and commercial use

21
 OpenFlow: Status
Commercial Ethernet switches and routers
Working with six vendors to add to existing products
Expect OpenFlow Type 0 to be available in 2008-09

Reference switches
Software: Linux and OpenWRT (for access points)
Hardware: NetFPGA (line-rate 1GE; available soon)
Working on low-cost 48-port 1GE switch based on Quanta switch (Broadcom
chips)

Reference controller
Simple test controller
NOX controller (available soon)

22
 Deployment at Stanford
Stanford Computer Science Department
Gates Building
~1,000 network users
23 wiring closets

Stanford Center for Integrated Systems (EE)

Allen Building
~200 network users
6 wiring closets

Working with HP Labs and Cisco on deployment

23
 Deployment in Internet2
NetFPGA routers deployed

s
link
E
1G

24
 OpenFlow Usage Models
1. Experiments at the flow level
Layer 2 or Layer 3 (same!)
User-defined routing protocols
Experiment-specific controllers
Admission control Static or dynamic flow-entries
Network access control
Network management
Energy management
VOIP mobility and handoff

2. Experiments at the packet level

Slow: Controller handles packet processing
Fast: Redirect flows through programmable hardware
Modified routers, firewalls, NAT, congestion control

3. Alternatives to IP

25
 OpenFlow Switch
Usage example: Production Traffic Unchanged
Policy Rule
Hari: Use production network
Commercial
Switch or AP User Space
Open API Hari
Controller
e n F low Llinux kernel
Op
r ot ocol
P
Normal
sw Secure SSL Linux PC
Software Channel

Normal Flow
hw
datapath Table

26 Hari
 OpenFlow Switch
Usage example: New Protocols Isolated, But Alongside
Policy Rule
Dina: Use Dinas protocol
Commercial
Switch or AP User Space
Open API Dina
Controller
e n F low Llinux kernel
Op
r ot ocol
P
Normal
sw Secure SSL Linux PC
Software Channel

Normal Flow
hw
datapath Table

27 Dina
 Example Experiment at the flow level
Mobility

Lots of interesting questions

Management of flows
Control of switches
Access control of users and devices
Tracking user location and motion

28
 Innovation in the datapath
Layerless plumbing
Controller
Line-rate packet processing
1. Encryption PC
OpenFlow-enabled
2. IP++ Commercial Switch
3. Packet inspection
4. Congestion control
Normal Secure
Secure
(XCP etc.) Software Channel
Channel

5. ? Normal Flow
Flow
Datapath Table
Table

Laboratory

NetFPGA
29
 Controller Innovation

Controller Innovation Layer

Experiment
Static
Controllers
Specific 4D NOX
Controllers

OpenFlow Substrate Layer

TDM/WDM
IP Ethernet
Alongside
Circuits
Alongside Alongside
Spanning tree,
Normal routing, circuit routing,
VLANs, mgmt,
mgmt, inside box mgmt, inside box
inside box

30
 Controller Open Questions
Scalability
of controller
Load-balanced and redundant controllers
Aggregation of flows

31
 Conclusion
Networking has to work harder than most fields
to enable innovation.
The good news is that industry is open to the
idea.
And government agencies are on our side.
Lets work together to do it lets deploy
OpenFlow widely in universities.
If you are interested, please contact me
nickm@stanford.edu
http://OpenFlowSwitch.org

32