Scaling the IP NGN with

Unified MPLS
Istvan Kakonyi
Vertical Solutions Architect
September 2012

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1
• Introduction – Challenges ahead of Service Providers

• Evolution of MPLS Technology

• UMMT Architecture
Architecture Overview
UMMT Application in Mobile Networks
Resiliency

• Q and A

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 2
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
 A 18x increase in mobile
data traffic over the next
5 years

Video to comprise >70%

of mobile data traffic by
2016

Machine-to-machine
Traffic to increase 22x
by 2016

Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 2011-2016

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4
• High capacity requirements from edge to core:
100Mbps eNB, 1Gbps Access, 10Gbps Aggregation, 100Gbps Core

• Higher scale as LTE drives ubiquitous mobile broadband

Tens- to hundred-of-thousands of LTE eNBs and associated CSGs

• Support for multiple and mixed topologies

Fiber and microwave rings in access, fiber rings, and hub-and-spoke in
aggregation and core networks

• Need for graceful LTE introduction to existing 2G/3G networks

Coexistence with GSM Abis, TDM backhaul, and ATM for UMTS IuB

• Need to support transport for all services from all locations

Residential and business, retail and wholesale, L2 and L3 services from cell
site where this is the most cost effective location for the customer

• Optimized operations with consistent packet transport

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
 Reduction in BGP routes towards Access
Aggregation Node Aggregation Node

Core
Core
~45
~45 ~45
~45
RAN ~Aggregation
~254 2,500
IGP Routes
Domain ~70~MPLS/IP
Core67,000
IGPDomain
Routes ~254~Aggregation
2,500
IGP Routes
Domain
IGP
IGP
MPLS/IP
~ 6,020
MPLS/IP
BGP Routes ~ 67,000 BGP MPLS/IP IGP
IGP
RAN
IGPIGPRoutes! IGP Area Routes
IGPRoutes! ~ 6,020
IGPIGPBGP
Routes! Routes
Area/Process MPLS/IP
Area/Process
Routes
Routes
IGP Area/Process
Aggregation
Aggregation
Routes
IGPRoutes
Area/Process
Node
Node
Core
Core

Aggregation Node Aggregation Node

LDP LSP
LDP LSP
25 Aggregation Domains attached to the core!
LDP LSP
iBGP Hierarchical LSP
LDP LSP
LDP LSP LDP LSP
LDP LSP
LDP LSP

120 Access Rings / Aggregation Domain !

Node Access Domain Aggregation Domain Network Wide
Cell Site Gateways 20 2,400 60,000

Pre-Aggregation Nodes 2 240 6,000

Aggregation Nodes NA 12 300

Core ABRs NA 2 50

Mobile transport Gateways NA NA 20

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 7
 IGP area 3 IGP area 1 IGP area 2

ABR-RR1 ABR-RR2

Aggregati Aggregati
on PE2
PE1 on Core
Domain 1 Domain 2

ABR-RR3 ABR-RR4

BGP AS

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
 ABR-RR1 ABR-RR2
iBGP iBGP iBGP
peers peers peers
PE1 PE2
IGP 1 IGP 2 IGP 3

Next-Hop-Self Next-Hop-Self

 ABRs are also Route Reflectors

 PEs in the same segment peer with ABR-RRs
 RRs are inserted in data path by setting next-hop-self
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
 iBGP + Labels (RFC 3107)
iBGP IPv4 iBGP IPv4
Place of controlled
redistribution:
update: update:
Local IGP->BGP PE1 PE1
Core IGP->Local IGP
Label=(L2) Label=(L1)
NH=ABR-RR1 NH=ABR-RR2

Aggregation ABR-RR1 Core Aggregation

ABR-RR2
PE1 PE2
iBGP IPv4
iBGP IPv4 update:
update:
PE2
PE2
Label=(L3)
Label=(L4)
NH=ABR-RR2
NH=ABR-RR1

 BGP updates include labels for IPv4 prefixes

 Only share PE loopbacks with other segments
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
 PW

VCID:X
Label:Z

GE0/1
VCID:X
PE1 PE2
GE0/1
VCID:X
ABR-RR1 ABR-RR2

VCID:X
Label:Y

 PE1 and PE2 exchange PW Virtual Circuit labels as

usual
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
 IGP Label
BGP Label
PW VC Label
Payload

Aggregation Core Aggregation

ABR-RR1 ABR-RR2 PE2
PE1

21 22 23
L4 L3 Z
Z Z

 Egress PE pops VC label

 IGP and BGP labels are exchanged
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
 Supported
RR Today in
IOS/XR/XE

ABR-RR1 ABR-RR2
PE1 PE2

Core Aggregation
Aggregation

PE4

ABR-RR3 ABR-RR4
PE3

(cluster-id 1) (cluster-id 2)
BGP Additional-path:
RR sends all paths for ABRs, and they perform path selection
RR performs path selection, sends path + additional path

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
 • Access, Aggregation and Core are in different IGP areas

• No or very limited IGP route redistribution from Core towards

Aggregation areas
• Supports both single AS / multiple AS models

• RFC 3107 for label distribution (prefix+label through BGP):

PE loopbacks
Central Infrastructure: Edge Nodes, etc

• ABRs between IGP areas also act as BGP RRs

Next-hop self for inserting ABRs into the Data Path
Loop avoidance via Cluster-id

• BGP Additional-path + existing mechanisms for Fast

convergence
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15
 Operational Points

LER LSR LER

Access AGG AGG AGG AGG Access

MPLS MPLS
MPLS MPLS

• In general transport platforms, a service has to be configured on every

network element via operational points. The management system has to
know the topology.
• Goal is to minimize the number of operational points

• With the introduction of MPLS within the aggregation, some static

configuration is avoided.
• Only with the integration of all MPLS islands, the minimum number of
operational points is possible.

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16
 TDM BTS/ATM NodeB

TDM/ATM
BSC
ATM RNC
CSG
MTG

S1-U SGW
eNodeB
MTG MME
S1-C MTG

V4 or v6 MPLS VPN
CSG
X2-C, X2-U
MTG
SGW

Mobile Access Network Aggregation Network Core Network

Mobile Transport
Gateway (MTG) ASR-9000

IP/MPLS Transport IP/MPLS Transport IP/MPLS Transport

Cell Site Gateway (CSG) Pre-Aggregation Node Aggregation Node Core Node Core Node
ASR-901 3800X, 3600X-24CX, ASR-903 ASR-9000 CRS-3, ASR-9000 CRS-3, ASR-9000
Fiber or uWave Link, Ring DWDM, Fiber Rings, H&S, Hierarchical Topology DWDM, Fiber Rings, Mesh Topology

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18
 Validated in
UMMT 3.0

Aggregation Node
Aggregation Node

Mobile
Transport GW Core Node CSG
CSG Core Node
Core Core
Node Node RAN
RAN Aggregation Network Core Network Aggregation Network IP/MPLS
IP/MPLS IP/MPLS IP/MPLS Domain IP/MPLS domain
domain Domain Domain
CSG
CSG Pre-Aggregation Core Pre-Aggregation
Node Core
Node Node
Mobile Node
Core Node Transport GW Core Node

CSG
CSG Aggregation Node
Aggregation Node

iBGP (eBGP across ASes) Hierarchical LSP

LDP LSP LDP LSP LDP LSP LDP LSP LDP LSP

• The Mobile Core, Aggregation, Access Network enable Unified MPLS Transport
• The Core, Aggregation, Access are organized as independent IGP/LDP domains
• Core and Aggregation Networks may be in different Autonomous Systems, in which case the inter-
domain LSP is enabled by labeled eBGP in between ASes
• The network domains are interconnected with hierarchical LSPs based on RFC 3107, BGP
IPv4+labels. Intra domain connectivity is based on LDP LSPs
• The Access Network Nodes learn only the required labelled BGP FECs, with selective distribution of
the MPC and RAN neighbouring labelled BGP communities

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19
 Unified MPLS Transport
Inline RR External RR
Inline RR Inline RR
 NHS 
 NHS  RR  NHS 

iBGP
iBGP iBGP IPv4+label
IPv4+label PE IPv4+label ABR IPv4+label
IPv4+label
IPv4+label PE
BNG, MSE

Example: IP RAN VPNv4 Service External RR

Inline RR RR
Inline RR Inline RR

VPNv4 PE iBGP
iBGP iBGP
CSG VPNv4 VPNv4
VPNv4
VPNv4 PE
MTG (EPC GW)

Mobile Access Network Aggregation Network Core Network

Mobile Transport Gateway
(MTG) ASR-9000

IP/MPLS Transport IP/MPLS Transport IP/MPLS Transport

Cell Site Gateway (CSG) Pre-Aggregation Node Aggregation Node Core ABR Core ABR
ASR-901 ME-3800X, 3600-X, ASR-903 ASR-9000 CRS-3, ASR-9000 CRS-3, ASR-9000
Fiber or uWave Link, Ring
© 2010 Cisco and/or its affiliates. All rights reserved. DWDM, Fiber Rings, H&S, Hierarchical Topology DWDM, Fiber Rings, Mesh Topology
Cisco Confidential 20
 LDP Label

BGP Label
LSPs between CSG and MTG Loopbacks
NHS NHS NHS NHS
iBGP IPv4+label iBGP IPv4+label
1- Control
iBGP IPv4+label

NHS NHS NHS NHS

iBGP IPv4+label iBGP IPv4+label iBGP IPv4+label
2- Control Imp-Null
CSG PAN CN-ABR MTG CN-ABR
Inline RR Inline RR Inline RR
RAN IGP Domain Aggregation IGP Domain Core IGP Domain

Central RR

iBGP iBGP
iBGP

Mobile Transport GW

push swap pop push swap pop

1- Forwarding
push swap swap swap pop

LDP LSP LDP LSP

iBGP Hierarchical LSP LDP LSP

pop swap push pop swap push

pop swap swap swap push

2 - Forwarding

LDP LSP LDP LSP

LDP LSP iBGP Hierarchical LSP

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
 Labeled BGP LSPs between Remote Access Nodes LDP Label

BGP Label

Control

Next-Hop-Self Next-Hop-Self Next-Hop-Self Next-Hop-Self Next-Hop-Self

iBGP IPv4+label iBGP IPv4+label iBGP IPv4+label iBGP IPv4+label iBGP IPv4+label
Imp-Null

Access IGP Domain Aggregation IGP Domain Core IGP Domain Aggregation IGP Domain Access IGP Domain

AN PAN-ABR CN-ABR CN-ABR PAN-ABR AN

Inline-RR Inline-RR Inline-RR Inline-RR
Central RR

iBGP iBGP iBGP iBGP

iBGP

MTG

push swap pop push swap pop push swap pop push swap pop

push swap swap swap swap swap pop

LDP LSP LDP LSP LDP LSP LDP LSP

iBGP Hierarchical LSP LDP LSP

Forwarding
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
• Only the MPC community is distributed to RAN access

• The RAN Common Community is only distributed to MTGs

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
• Unified MPLS transport with a common MPLS VPN for LTE S1 from all CSGs and X2 per LTE region.

• Mobile Transport GWs import all RAN & MPC Route Targets, and export prefixes with MPC Route Target

• CSGs (and/or Pre-Aggregation Node) in a RAN region import the MPC and regional RAN Route Targets:
Enables S1 control and user plane with any MPC locations in the core
Enables X2 across CSGs in the RAN region

• MPLS VPN availability based on BGP PIC Edge and infrastructure LSP based LFA FRR

• Pre-Aggregation Nodes and Core POP Nodes form inline RR hierarchy for the MPLS VPN service
Core ABRs perform BGP community based Egress filtering to drop unwanted remote RAN VPNv4 prefixes
Pre-Aggregation Nodes implement RT Constrained Route Distribution towards CSR VPNv4 clients
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 25
 Access Network Aggregation Network Core Network Aggregation Network Mobile Access Network
OPSF 0 / IS-IS L2 IS-IS L1 IS-IS L2 IS-IS L1 OPSF 0 / IS-IS L2

BGP Egress filter towards CSGs: BGP Egress filter towards CSGs:
1) Allow MTG community 1001:1001 1) Allow MTG community 1001:1001
2) Allow common wireline community 20:20 2) Allow common wireline community 20:20
3) Drop 3) Drop

PAN CN-ABR CN-ABR PAN

Inline RR Inline RR Inline RR Inline RR
MTG
CSG
CSG iBGP iBGP
IPv4 + label EoMPLS Pseudowire IPv4 + label iBGP
iBGP IPv4 + label
IPv4 + label
RR
CN-RR CSG
CSG iBGP
IPv4 + label

Advertise loopback in iBGP with Advertise loopback in iBGP with

Local RAN community 10:0101, Local RAN community 10:0201,
Common RAN community 10:10, MTG Common RAN community 10:10,
CSG and Common Wireline Community and Common Wireline Community CSG
20:20 20:20

BGP Inbound Route Filter: BGP Inbound Route Filter

1) Accept MTG community 1001:1001 1) Accept MTG community 1001:1001
2) Accept remote loopbacks for configured wireline services 2) Accept remote loopbacks for configured wireline services
3) Drop 3) Drop

• MTG and Common wireline communities distributed to RAN access

• Common RAN Community is only distributed to MTGs

• CSG accepts MTG & remote loopbacks for configured wireline services

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26
• What is LFA FRR? 2
Well known (RFC 5286) basic fast re-route mechanism to provide C D
local protection for unicast traffic in pure IP and MPLS/LDP 10
2
networks 2
A B 1
Path computation done only at “source” node
4 8
Backup is Loop Free Alternate (C is an LFA, E is not)
E F
• No directly connected Loop Free Alternates (LFA) in some
topologies

• Ring topologies for example:

Consider C1-C2 link failure A1 A2
If C2 sends a A1-destined packet to C3, C3 will send it back to C2

• However, a non-directly connected loop free alternate node C1 C5

(C5) exits
C2 C4

C3

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 27
2
http://tools.ietf.org/html/draft-shand-remote-lfa
• Remote LFA uses automated IGP/LDP behavior to extend
basic LFA FRR to arbitrary topologies
Backbone
• A node dynamically computes its remote loop free
alternate node(s)
Done during SFP calculations using PQ algorithm (see draft) A1 A2

• Automatically establishes a directed LDP session to it

C1 C5
The directed LDP session is used to exchange labels for the Directed LDP
FEC in question session

• On failure, the node uses label stacking to tunnel traffic to C2 C4

the Remote LFA node, which in turn forwards it to the
destination C3

• Note: The whole label exchange and tunneling

mechanism is dynamic and does not involve any manual Access Region
provisioning

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 28
2
• C2’s LIB
C1’s label for FEC A1 = 20
C3’s label for FEC C5 = 99 Backbone

C5’s label for FEC A1 = 21

A1 A2
• On failure, C2 sends A1-destined traffic onto an LSP
destined to C5
Swap per-prefix label 20 with 21 that is expected by C5 for that C1 Directed LDP C5
E1
session
prefix, and push label 99 20
21

• When C5 receives the traffic, the top label 21 is the one C2 21 C4

that it expects for that prefix and hence it forwards it onto
the destination using the shortest-path avoiding the link 99 C3
C1-C2. 21 X

21 99
Access Region

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29
2
 Odd Ring
• MPLS-TE FRR 1-hop Link
AG1-1 AG1-2
14 primary TE tunnels to operate
tLDP session for
14 backup TE tunnels to operate CSS-1
link CSS 2-3
CSS-5
No node protection

• MPLS-TE FRR Full-Mesh CSS-2 tLDP session for

link CSS 1-2
CSS-4

42 primary TE tunnels to operate

CSS-3
14 backup TE tunnels to operate for Link protection
28 backup TE tunnels to operate for Link & Node protection
Even Ring
• Remote LFA
AG1-1 AG1-2
Fully automated IGP/LDP behavior
tLDP session
tLDP session dynamically set up to Remote LFA Node for links
CSS 1-2 and 2-3
Even ring involves 1 directed LDP sessions per node CSS-1 CSS-4

Odd ring involves 2 directed LDP sessions per node

No tunnels to operate
CSS-2 CSS-3

*For the count, account that TE tunnels are unidirectional

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30
3
 • BGP Fast Reroute (BGP FRR)—enables
BGP to use alternate paths within sub-
seconds after a failure of the primary or
active paths
• PIC or FRR dependent routing protocols
(e.g. BGP) install backup paths
• Without backup paths
Convergence is driven from the routing
protocols updating the RIB and FIB one
prefix at a time - Convergence times directly
proportional to the number of affected
prefixes
• With backup paths
Paths in RIB/FIB available for immediate use
Predictable and constant convergence time
independent of number of prefixes

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 32
 PIC Core PIC Edge
msec
1000000
Core
100000
100000
250k PIC
10000 10000
250k no PIC
PIC 500k PIC
LoC (ms)

1000 1000
no PIC 500k no PIC

100 100

10
10

1 P 1
12 0

15 0

17 0

20 0

22 0

25 0

27 0

30 0

32 0

35 0
00
0

10 0
1

0
00

00

00

50000

100000

150000

200000

250000

300000

350000

400000

450000

500000
00

50

00

50

00

50

00

50

00

50

00
25

50

75

Prefix
Prefix

• Upon failure in the core, without Core PIC,
convergence function of number of affected
prefixes
• With PIC, convergence predictable and
remains constant independent of the number
of prefixes
 Upon failure at the edge, without edge PIC,
convergence function of number of affected
prefixes
 With PIC, convergence predictable and
remains constant irrespective of the number of
prefixes

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
RAN IGP Process Aggregation Domain Core Domain Aggregation Domain RAN IGP Process
OSPF/ ISIS (OSPFx/ISIS1) OSPF0/ISIS2 (OSPFx/ISIS1) OSPF/ ISIS

Redistribute MPC Aggregation Node

iBGP Aggregation Node
iBGP community
into RAN Access IGP IPv4+Label
iBGP RR Core
Core

RAN
RAN iBGP Core iBGP Access
Access
Aggregation
Node Aggregation Aggregation Aggregation
Node
BGP Community Core MPC BGP Community
PE Core
Redistribute
CSN Loopbacks Aggregation Node
Aggregation Node iBGP Hierarchical LSP
into 3107 iBGP

LDP LSP LDP LSP LDP LSP LDP LSP LDP LSP

LFA L3 convergence < 50ms

BGP PIC Core L3

convergence < 100ms
BGP PIC Edge L3
convergence < 100ms

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34
• Cisco Unified MPLS helps in building highly scalable MPLS
networks where:
IGP domains are kept small
Number of operation points are minimized
Operation and troubleshooting kept simple

• Cisco Unified Transport for Mobile Networks provides:

An implementation of Unified MPLS
Tested, validated design with extensive support and documentation
Optimized for Mobile and converged fixed / mobile networks

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
Thank you.