Cloud Computing

• Elastic resources
– Expand and contract resources
A survey study on Data Center – Pay-per-use
– Infrastructure on demand
Traffic Engineering • Multi-tenancy
– Multiple independent users
– Security and resource isolation
– Amortize the cost of the (shared) infrastructure
• Flexibility service management
– Resiliency: isolate failure of servers and storage
– Workload movement: move work to other locations

DCN 1 DCN 2

Multi-Tier Applications Enabling Technology: Virtualization

• Applications consist of tasks
Front end
– Many separate components Server
– Running on different machines
• Commodity computers
– Many general-purpose computersAggregator
– Not one big mainframe
……
Easier scaling
–Aggregator Aggregator Aggregator • Multiple virtual machines on one physical machine
… … • Applications run unmodified as on real machine
Worker Worker Worker Worker Worker • VM can migrate from one computer to another
DCN 3 DCN 4

1
 Data center networks
• 10’s to 100’s of thousands of hosts, often closely coupled, in
close proximity:
– e-business (e.g. Amazon)
Data Center Network – content-servers (e.g.,YouTube,Akamai, Apple,
Microsoft)
– search engines, data mining (e.g., Google)
Few slides from
V. Arun  challenges:
College of Computer Science
University of Massachusetts Amherst
 multiple applications,
each serving massive
numbers of clients
 managing/balancing load,
avoiding processing, Inside a 40-ft Microsoft container,

DCN 5 networking, data DCN

Chicago data center

bottlenecks

Data center networks Data center networks

load balancer: application-layer routing
 receives external client requests  rich interconnection among switches, racks:
 directs workload within data center
 returns results to external client (hiding data center  increased throughput between racks (multiple
internals from client)
routing paths possible)
Internet  increased reliability via redundancy
Border router
Load Load
Access router Tier-1 switches
balancer balancer

Tier-1 switches
B Tier-2 switches

A C Tier-2 switches

TOR TOR
switches switches
Server racks Server racks

1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
DCN 8
DCN

2
 Broad questions
• How are massive numbers of commodity
machines networked inside a data center?
• Virtualization: How to effectively manage
physical machine resources across client
virtual machines?
• Operational costs:
– Server equipment
– Power and cooling

DCN 9 DCN 10
Source: NRDC research paper

Breakdown wrt DC size Virtual Switch in Server

Source: NRDC research paper

DCN 11 DCN 12

3
 Top-of-Rack Architecture Data Center Network Topology
Internet
• Rack of servers CR CR

– Commodity servers ...

AR AR AR AR
– And top-of-rack switch
S S
• Modular design
– Preconfigured racks S S S S
...
– Power, network, and Key
… …
storage cabling •
•
CR = Core Router
AR = Access Router
~ 1,000 servers/pod • S = Ethernet Switch
• Aggregate to the next level • A = Rack of app. servers

DCN 13 DCN 14

Popular topologies Canonical DC topology

DCN 15 DCN 16

4
 Fat-Tree topology for DCs PortLand topology
• Fat-Tree: a special type of Clos Networks (after C. Clos)
• Introduce “pseudo MAC addresses” to servers to balance the pros and cons of flat- vs.
K-ary fat tree: three-layer topology (edge, aggregation and core)
topology-dependent addressing
– each pod consists of (k/2)2 servers & 2 layers of k/2 k-port switches
– each edge switch connects to k/2 servers & k/2 aggr. switches
– each aggr. switch connects to k/2 edge & k/2 core switches
– (k/2)2 core switches: each connects to k pods

pod

Fat-tree
with K=4 position

DCN 17 DCN 18

HyperX topology [8]

Clos Network Topology
Offer huge aggr capacity & multi paths at modest cost

Int ...
D Max DC size
(# of 10G ports) (# of Servers)
48 11,520
Aggr ...
96 46,080
K aggr switches with D ports
144 103,680
. . . TOR ......

... 20 ........
Servers
20*(DK/4)
DCN Servers 19 DCN 20

5
 Data-Center Routing Load Balancers
Internet
CR CR • Spread load over server replicas
DC-Layer 3 – Present a single public address (VIP) for a service
AR AR ... AR AR
– Direct each request to a server replica
DC-Layer 2
SS S
S 10.10.10.1
Virtual IP (VIP)
192.121.10.1
S
S S
S SS SS
...
Key 10.10.10.2
… … • CR = Core Router (L3)
• AR = Access Router (L3)
~ 1,000 servers/pod == IP subnet • S = Ethernet Switch (L2)
• A = Rack of app. servers

10.10.10.3

DCN 21 DCN 22

Wide-Area Network Wide-Area Network: Ingress Proxies

... Data
... ... ...
Data
Servers Centers Servers Servers Centers Servers
Router Router Router Router

DNS
Server Internet Proxy Proxy

DNS-based
Clients Clients
site selection
DCN 23 DCN 24

6
 Traffic Engineering Challenges
• Scale
– Many switches, hosts, and virtual machines
• Churn
Data Center Traffic Engineering – Large number of component failures
– Virtual Machine (VM) migration
• Traffic characteristics
Challenges and Opportunities – High traffic volume and dense traffic matrix
– Volatile, unpredictable traffic patterns
• Performance requirements
– Delay-sensitive applications
– Resource isolation between tenants

DCN 25 DCN 26

Traffic Engineering Opportunities

• Efficient network
– Low propagation delay and high capacity
• Specialized topology
– Fat tree, Clos network, etc. A survey on DC traffic engg.
– Opportunities for hierarchical addressing
• Control over both network and hosts
– Joint optimization of routing and server placement From different existing works
– Can move network functionality into the end host
• Flexible movement of workload
– Services replicated at multiple servers and data centers
– Virtual Machine (VM) migration

DCN 27 DCN 28

7
 Improving Data Center Network Utilization
Using Near-Optimal Traffic Engineering [1]
• [1] constructs a routing algorithm that
provides path diversity over non-uniform link
weights (i.e., unequal cost links), simplicity in
path discovery and optimality in minimizing
maximum link utilization (MLU) is nontrivial
• [1] implemented and evaluated the Penalizing
Exponential FlowspliTing (PEFT) algorithm in a
cloud DC environment based on two
dominant topologies, canonical and fat tree
DCN
[1 contd.]
• [1] proposes online PEFT for more adaptive TM (traffic
matrix) to compute link weights.
• Simulation has been carried out by NS-3 with the help
IPOPT.
• Comparison made between PEFT and ICMP over the
three topologies.
• Modification of canonical tree structure to further
reduce MLU
DCN
ECMP [1 contd.]
• Most of currentDCnetworks employ equal cost
multipath (ECMP) forwarding to leverage the
path diversity provided by topological
redundancy, by splitting traffic across multiple
paths through hashing packets’ headers.
• Per-flow ECMP is link load and flow agnostic in
which resulting hash collision can hinder load-
balancing among links
29 DCN 30
Intra-Data-Center Traffic Engineering with
Ensemble Routing [2]
• The key challenge of intra-data-center traffic
engineering with ensemble routing is the
combinatorial optimization of VLAN assignment, i.e.,
optimally assigning flow ensembles to VLANs to
achieve load balancing and low network costs.
• Based on the Markov approximation framework, [2]
solves the VLAN assignment problem with a general
objective function and arbitrary network topologies by
designing approximation algorithms with close-to-
optimal performance guarantees.
• TCP congestion and QoS issues were also dealt for
VLAN assignment.
31 DCN 32
8
 OpenFlow based Load Balancing for
Fat-Tree Networks with Multipath Support [3]
• [3] present a load balancer for OpenFlow based fat-tree
data center networks and implements a dynamic routing
algorithm in the load balancer.
• [3] prposes dynamic load balancer (DLB) and compares it
with the results of three flow scheduling algorithms: none
load balancing (NLB) algorithm, static load balancing (SLB)
algorithm
• One characteristic of routing in fat-tree network is: once a
flow reaches the highest layer that it accesses, the path
from that switch to the destination host is deterministic.
Only upward traffic is handled by DLB algorithm. The
downward traffic is automatically determined
corresponding to highest layer switch and destination.
• Beacon openflow controller with Mininet emulator.
DCN
Current DC Multi path approaches [4]
• Valiant Load Balancing (VLB), which essentially
guarantees equal spread load-balancing in a
mesh network by bouncing individual packets
from a source switch in the mesh off of
randomly chosen intermediate “core”
switches, which finally forward those packets
to their destination switch.
• ECMP
DCN
Hedera: Dynamic Flow Scheduling for
Data Center Networks [4]
• It is a dynamic flow scheduling system that adaptively schedules a multi-
stage switching fabric to efficiently utilize aggregate network resources.
• Dynamic Flow Demand Estimation: The demand estimator performs
repeated iterations of increasing the flow capacities from the sources and
decreasing exceeded capacity at the receivers until the flow capacities
converge
• Schedular algo.s:
– Global First Fit: When a new large flow is detected, (e.g. 10% of the host’s link
capacity), the scheduler linearly searches all possible paths to find one whose
link components can all accommodate that flow. Forwarding entries are also
scheduled.
– Simulated Annealing: performs a probabilistic search to efficiently compute
paths for flows. The key insight of this approach is to assign a single core
switch for each destination host rather than a core switch for each flow. This
reduces the search space significantly.
• Implementation uses NetFPGA OpenFlow switches which have 2 hardware
tables: a 32-entry TCAM (that accepts variable-length prefixes) and a 32K
entry SRAM that only accepts flow entries with fully qualified 10-tuples.
33 DCN 34
Other works in summary
• Devoflow [5] is a modication of the OpenFlow model which
gently breaks the coupling between control and global
visibility, in a way that maintains a useful amount of
visibility without imposing unnecessary costs. It focuses on
handling most of the micro-flows in data plane
• [6] proposes LABERIO (a novel path-switching algorithm)
which solves the problem of OpenFlow which only try to
find a static routing path during initialization step while the
static routing path often suffers from poor performance
since the network configuration may change during the
data transmission.
• [7] discusses of Aster*x controller built on top of NOX
which is a distributed load balancer for large cloud.
35 DCN 36
9

Ongoing Research
DCN
Research Questions
• Rethinking TCP congestion control?
– Low propagation delay and high bandwidth
– “Incast” problem leading to bursty packet loss
• Division of labor for TE, access control, …
– VM, hypervisor, ToR, and core switches/routers
• Reducing energy consumption
– Better load balancing vs. selective shutting down
• Wide-area traffic engineering
– Selecting the least-loaded or closest data center
• Security
– Preventing information leakage and attacks
DCN
Research Questions
• What topology to use in data centers?
– Reducing wiring complexity
– Achieving high bisection bandwidth
– Exploiting capabilities of optics and wireless
• Routing architecture?
– Flat layer-2 network vs. hybrid switch/router
– Flat vs. hierarchical addressing
• How to perform traffic engineering?
– Over-engineering vs. adapting to load
– Server selection, VM placement, or optimizing routing
• Virtualization of NICs, servers, switches, …
37 DCN 38
Data center
• A data center (DC) is a facility used to house
computer systems and associated
components, such as telecommunications and
storage systems.
• It consists of lots of switches , routers, servers,
cables, racks to place the servers, cooling
module and finally a proper data network.
• Networking may be different inside and
outside of DC.
39 DCN 40
10
 42

Fat-Tree topology for DCs

Recent DC architectures
(topologies)
Fat-tree
with K=4

DCN 41 DCN

Facebook 4-post

DCN 43 DCN 44

11
Cisco virtualized fabric DC design
Amazon EC2 high availability design

DCN 45 DCN 46

References
1. “Improving Data Center Network Utilization Using Near-Optimal Traffic
Engineering”, Fung Po Tso, Dimitrios P. Pezaros, IEEE TRANSACTIONS ON
PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 24, NO. 6, JUNE 2013
2. “Intra-Data-Center Traffic Engineering with Ensemble Routing”, Ziyu Shao, Xin Jin,
Wenjie Jiang, Minghua Chen, Mung Chiang, 2013 Proceedings IEEE INFOCOM
3. “OpenFlow based Load Balancing for Fat-Tree Networks with Multipath Support”,
Yu Li, Deng Pan, IEEE ICC 2013
4. “Hedera: Dynamic Flow Scheduling for Data Center Networks”, Mohammad Al-
Fares, Sivasankar Radhakrishnan, Barath Raghavan, Nelson Huang, Amin Vahdat,
NSDI'10 Proceedings of the 7th USENIX conference on Networked systems design
and implementation
5. “DevoFlow: Scaling Flow Management for High-Performance Networks”, Andrew
R. Curtis, Jeffrey C. Mogul, Jean Tourrilhes, Praveen Yalagandula, Puneet Sharma,
Sujata Banerjee, Proceedings of the ACM SIGCOMM 2011 conference Pages 254-
265
6. “LABERIO: Dynamic load-balanced routing in OpenFlow-enabled networks”, Hui
Long, Yao Shen, Minyi Guo, Feilong Tang, 2013 IEEE 27th International
Conference on Advanced Information Networking and Applications
7. “Aster*x: Load-Balancing as a Network Primitive”, Nikhil Handigol, Mario Flajslik,
Srini Seetharaman, Ramesh Johari, Nick McKeown, Stanford University, Deutsche
Telekom R&D Lab USA, 9th GENI Engineering Conference, 2010
8. “HyperX: Topology, Routing, and Packaging of Efficient Large-Scale Networks”,
Jung Ho Ahn, Nathan Binkert, Al Davis, Moray McLaren, Robert S. Schreiber, HP
Laboratories , HPL-2009-184
DCN 47 DCN 48

12