BRKSP-2612

Modernizing Broadband Services
with Cisco Subscriber Edge

Architecture and EVPN access
Gurpreet Dhaliwal, Provider Connectivity TME
BRKSP-2612
Let’s Baseline …
Policy Layer
Subscriber Services Network

Components
AN
SE SE: Subscriber Edge
Policy Layer: AAA, PCRF
CDN: Content Delivery Network/ Content

Caching
CDN
BRKSP-2612 © 2024 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Agenda
• Overview
• Subscriber Edge Redefined
• Cisco Subscriber Edge aka cnBNG
• Resiliency and Operations
• Access Auto-signaling
• Demonstration
• Conclusion
Overview
Service Provider Challenges Today
Complex Higher Flat Complex

Operations CapEx & OpEx Revenue Integration
New Revenue
Opportunities?
We will discuss ☺
Modernize Broadband Networks
Simplify Lower Increase Simplify

Operations CapEx & OpEx Revenue Integration
What is modernization?
*definition from oxfordlearnersdictionaries.com
Let’s understand “Present
time” for broadband networks…
Demand for bandwidth will continue to increase
110 Mbps
average broadband
>75%
traffic is Video
Source: Cisco Annual Internet Report
Which design you select?
PEERING
PeeringTraf
SE
fic
Traffic
PRE-AGG AGG EDGE CORE
CDN
DSL/PON/FWA
Caching Centralized
with centralized caching and peering
Distributed PEERING
PeeringTr
with distributed caching and peering
PEERING
affic
CUPS to accelerate
further distribution
PRE-AGG AGG EDGE COR
Traffic
DSL/PON/FWA
CDN
SE Caching E
How about Subscriber Edge?
Business L3
VPNs IP/MPLS
SR/SRv6
L2VPN/
EVPN
Bigger More Multiservice
• Higher starting CapEx • Lower starting CapEx

• Non-optimal Centralized Design • Optimal distributed design
• Widespread failure impact • Limited failure impact
• CUPS to address Manageability and integration
Subscriber Edge
Redefined
Multi-Service Disaggregated BNG with CUPS
• CUPS means Control and User Plane Separation
• Defined by Broadband Forum Standard TR-459
Control Plane functions

• Authentication
• Authorization
• Accounting-data reporting
• Address assignment
• Security and policy management
• Centralized Management
Data Plane functions

• Access termination
• Accounting data collection
• Security and Policy enforcement
from TR459i2 • Quality of service (QoS)
Control and User Plane Separation (CUPS)
Charging Captive
AAA Policy OSS
/Billing Portal
CapEx and
OpEx
Centralized CP CP
Control-Plane(s)
Active Standby
Easier OSS/ BSS
Integration
m x n x 100’s
Simplified
Operations
Distributed
User-Planes
Control and User Plane Separation Benefits
I have a centralized
User Plane Selection Control Plane view, let me
Function *
CSE CP
Ops Center North Bound Interfaces CDL DB anchor subscriber
(UPSF) on less loaded
User Plane!
DHCPv4 DHCPv6 L2TP IPoE PPPoE
Traffic Steering IPAM Monitoring GTP-u PFCP LI-CP
Control Function *
SMI
SCi CPRi Mi
Keepalive PFCP GTP-u LI-UP Local CP
Access AN Traffic Steering

Function * Mgmt CSE UP PPPoE IPoE FIB Routing
Internet
XR User Plane
New Revenue Streams Optimized Operations
• Service offerings based on UP capabilities • Loadbalancing

• Differentiated Plans: • Intelligent Offload
• Night plan vs Day plan • Controlled Rollout
• Platinum, Gold and Best-effort plans
*from BBF WT-474 Subscriber Session Steering

Cisco Subscriber Edge
(aka Cloud Native BNG or cnBNG)
High Level Cisco Subscriber Edge Architecture
Aligned with Broadband Forum TR-459
Charging Captive
AAA Policy OSS
/Billing Portal
Control Plane:
Control Plane • Built as cloud-native application for greater resiliency
• Key functions split into multiple containers, which allows:
rolling upgrade, independent patching, easy scale-in/out
of services, faster feature delivery
Subscriber Microservices Infrastructure • Varying deployment models: VM Based, Baremetal, Public
(Cisco Ultra Cloud Core) cloud
SCi CPRi Mi
XR User Plane User/Data Plane:

• ASR9k* IOS-XR based UP
ASR9k
SCi: State Control Interface (PFCP)
CPRi: Control Packet Redirect Interface (GTPu)
Mi: Management Interface
Cisco Subscriber Edge: Simplifying Operations
Charging Captive
AAA Policy OSS
/Billing Portal
Control Plane
• CSE Ops Center:
• Provides netconf/ restconf/ cli interface
• Abstracts K8s details for day2day operations
CSE Ops Center IPAM Monitoring

• IPAM:
• IP Address Manager
• Provides a centralized mechanism to manage IP address
• Subscriber Monitoring:
SCi CPRi Mi • A unified view of BNG Deployment
• Exposes 100s of metrics and KPIs for BNG monitoring
X • Includes an in-built Grafana dashboard
R User Plane • Can generate alerts through Alert Manager
SCi: State Control Interface (PFCP)

CPRi: Control Packet Redirect Interface (GTPu)
Mi: Management Interface
for reference
Cisco Subscriber Edge Ops Center

• Ops Center embeds Cisco Conf-D
• Easier operations for Network Engineers
• Standard NETCONF, RESTCONF, and CLI

interfaces
• YANG Models for configurations and operations

• Supports Native and Openconfig models
for reference
IP Pool and Address Management: IPAM

ipam
• IPAM manages address pools dynamically across all User Planes instance 1
source local
• Larger pools can be split into multiple sub-pools or chunks address-pool pool_1
vrf-name default
ipv4
• Pool allocation can be static or dynamic split-size
per-dp 512
• Pools can be selected based on dhcp option value or radius exit
address-range 192.168.110.2 192.169.110.254
exit
• Pool allocation can be monitored from Ops Center or Grafana ipv6
address-ranges
split-size
[cnbng-tme-lab/bng] bng# show ipam pool
per-dp 512
========================================================================================== exit
PoolName Ipv4Utilization Ipv6AddrUtilization Ipv6PrefixUtilization address-range 2001::1 2001::1:100
==========================================================================================
POOL_2 1.46% 4.01% 1.00%
exit
POOL_1 1.46% 4.01% 1.00% prefix-ranges
POOL_3 2.86% 18.30% 1.14% split-size
==========================================================================================
per-dp 512
exit
prefix-range 2001:1:: length 48
exit
exit
exit
for reference
Performance Monitoring
Collection Visualization Alerts
• Control Plane metric collection inbuilt • Metrics Visualization on Inbuilt Dashboard (Grafana) • Alert Manager for generating alerts
• Model Driven Telemetry for UP stats collection • Integration with external visualization also supported • Alerts via Email, Webhooks, SNMP or on
• 100s of subscriber metrics already available Grafana dashboard
from Control Plane • Netconf/ Restconf/ CLI to set alerts
for reference
Performance Monitoring (contd)
for reference
Performance Monitoring (contd)
Resiliency and
Operations
Control Plane Layered Architecture
OAM Layer
CSE Ops Center Smart Agent
Multiple k8s Nodes can be deployed per Layer
Protocol Layer
UDP-Proxy Radius-EP BNG-N7
K8s Node labeling is used to separate
BNG-N4 Cache-POD deployments
Service Layer
Multi-replicas of similar container are possible
Node Mgr DHCP PPPoE within each Layer
BNG-SM BNG-Charging
All containers are stateless except CDL which

Session Layer
can be Local or Geo Redundant
CDL-EP CDL-Index CDL-Slot
Intra Cluster Resiliency: Container Failure
• Multi-replicas of similar containers are supported
• Whenever a Container fails

• Container running on other server provides
service to ensure continued operations
• Kubernetes re-starts it automatically
• Containers in all layers except Session are
stateless
• No impact in CP operations
Intra Cluster Resiliency: Node Down/ Failure
• Multiple Servers can be deployed per Layer
• Kubernetes node labelling is used to separate
the deployments
• Multi-replicas of similar containers run per
Layer
• Whenever a Server fails or is down for
operations
• Containers running on other server provides service
to ensure continued operations
• When Server is brought back to operations

• Kubernetes creates Containers automatically based
on Labels
Control Plane Rolling Upgrade
• Utilizes K8s rolling update strategy to upgrade POD images
CP Upgrade Procedure
• PODs of a StatefulSet are updated sequentially
• This ensures ongoing control-plane operations remains unaffected • One-by-one PODs in StatefulSets are terminated
• Update can be controlled from Deployer and is fully automated • Terminating PODs exit gracefully after completing ongoing
processing
StatefulSet-1 StatefulSet-2
• New PODs with updated images take the place of
Initial State DHCP POD-0 DHCP POD-1 PPPoE POD-0 PPPoE POD-1 PPPoE POD-2 terminated PODs
• This process continues till all the replicas of the StatefulSet

Step-1 DHCP POD-0 DHCP POD-1 PPPoE POD-0 PPPoE POD-1 PPPoE POD-2 are updated
• During upgrade other PODs continue to function

seamlessly without causing service disruption
Step-2 DHCP POD-0 DHCP POD-1 PPPoE POD-0 PPPoE POD-1 PPPoE POD-2
Step-3 DHCP POD-0 DHCP POD-1 PPPoE POD-0 PPPoE POD-1 PPPoE POD-2
Cisco CSE CP supports Rolling Upgrade from Day1

Final State DHCP POD-0 DHCP POD-1 PPPoE POD-0 PPPoE POD-1 PPPoE POD-2
Control Plane and User Plane Geo-redundancy
Better Customer Experience Stateful Redundancy
Multiple Geographically Separated Control Planes Cisco CSE CP and UP can be geographically spread out with L3
connectivity. No dedicated/direct link required for synchronization
CP CP
1:1, M:1 and M:N Redundancy support for User Planes
1:1 Active-Active Redundancy support for Control Plane
Multiple Geographically Separated User Planes
Multiple Technology Support

Access Nodes (DSLAM/OLT/A6) Access Nodes are dual/multi-homed for redundancy using a
variety of technologies – EVPN, MCLAG, Dual Homed, Ring
(G.8032), PWHE etc.
Transparent Switching
CPE
CPEs see one Gateway
User Plane Geo-Redundancy Unified
Northboun
d interface
CP
• SRG is a set of subscriber sessions Active for
Activ
• SRG could be resilient or non-resilient SRG1 e
• A resilient SRG is Standby CP

• A unit of failover for geo redundancy for SRG1 Standb
UP 1 y
• Defined as a set of access interfaces: Port, VLAN, BE, SCi/CPRi
SRG 1
PWHE, Mix
• Provisioned in consideration of the redundancy design in
access & core networks – capacity planning, load balance,
routing preference, etc.
Access Core
• Active and Standby state is defined at Control Plane CPE
UP 2
• Control Plane determines the state of SRGs on User
Planes and controls subscriber traffic flows
• No Accounting change at backend SRG 2
UP 3
Active for
SRG2
Standby
for SRG2
Access Auto-signaling
Auto-signaling Call Flow
EVPN Route Type – 1 (steady state)
RP/0/RP0/CPU0:APE1#show bgp l2vpn evpn rd 172.0.0.1:1 [1][0002.0101.0101.0101.0102][3001]/120
Wed Sep 20 11:07:40.526 UTC Control-Word(C) = 4
BGP routing table entry for [1][0002.0101.0101.0101.0102][3001]/120, Route Distinguisher: 172.0.0.1:1 Primary(P) = 2
Versions:
Backup(B) = 1
Process bRIB/RIB SendTblVer RT-1 ESI PW-Ether1 OLT
Speaker 155 155 UP1/UP2 SVLAN
Last Modified: Sep 14 12:35:38.738 for 5d22h
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer Control-Word +
Local
172.0.0.2 (metric 10) from 172.0.0.2 (172.0.0.2) Primary (Based on SRG State) UP1 -> 172.0.0.2
UP2 -> 172.0.0.3
Received Label 24009
Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 1, version 153
Extended community: EVPN L2 ATTRS:0x06:0 RT:100:1
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 172.0.0.2:1
Path #2: Received by speaker 0
Not advertised to any peer
Local Control-Word +
172.0.0.3 (metric 10) from 172.0.0.3 (172.0.0.3)
Received Label 24004 Backup (Based on SRG State)
Origin IGP, localpref 100, valid, internal, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 0, version 0
Extended community: EVPN L2 ATTRS:0x05:0 RT:100:1
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 172.0.0.3:1
Single Home OLT Scenario
CP evpn
interface PW-Ether1
ethernet-segment
identifier type 0 2.1.1.1.1.1.1.1.2
service-carving preference-based
VLAN: A A
srg-driven
B
Home1 OLT1 APE1 UP1
IGW Internet
EVPN Core
Access
Home2 OLT2 APE2 UP2

A
VLAN: B B Active Path

Backup Path
• Each OLT is Single Home

• Failures at BNG are protected using SRG
• VLAN A (OLT1) is active on BNG-UP1 and VLAN B (OLT2) is active on BNG-UP2
• Only Flexible-xconnect-service is supported on APEs
Dual Home OLT: Scenario-1
Loadbalncing fromfrom
Loadbalancing PWHE towards
EVPNHE
CP AG1s is notAPEs
towards supported. Lowest IP NH
is not supported.
is selected
Lowest IP NHbyisdefault as preferred
selected by default
NH
as preferred NH
A
VLAN: A
B
Home1 OLT1 APE1 UP1
IGW Internet
EVPN Core
Access
Home2 OLT2 APE2 UP2

A
B
Active Path
VLAN: B
Backup Path
• Each OLT is Dual Home

• Only all-active is supported for dual home OLT scenario
Dual Home OLT: Scenario-2
Loadbalancing
Loadbalncingfrom
from EVPNHE towards
PWHE towards
APEs is not supported. Lowest IP NH
AG1s
CP is selected by default as preferred
NH
A
VLAN: A
B
Home1 OLT1 APE1 UP1
IGW Internet
EVPN Core
Access
Home2 OLT2 APE2 UP2

A
B
Active Path
VLAN: B
Backup Path
• Each OLT is Dual Home

• Only all-active is supported for dual home OLT scenario
Demonstration
Objective
• Showcase
• Geo-Redundant Subscriber Edge Provisioning
• L2 Service auto-signaling to connect SRGs to Active User Plane
Demo Topology
Telco Cloud
Radius
• Active for OLT1

• Standby for OLT2 CSE/cnBNG
Control Plane
SCi/ CPRi
OLT-BRU1
svlan.3001/ srg.3001 APE1 UP1
IPoE
L3
Access Core
IGW
Internet
OLT-BRU2 APE2 UP2

IPoE svlan.3002/ srg.3002
Spirent 1/1
Spirent 1/3 • Standby for OLT1
• Active for OLT2
Access Network Brief Telco Cloud
Radius
EVPN flexible-
xconnect-service CSE/cnBNG
Control Plane
evpn
interface PW-Ether1-2
OLT-BRU1 ethernet-segment
svlan.3001 APE1 UP1 service-carving preference-based
srg-driven
L3
Access Core
IGW
Internet
OLT-BRU2 APE2 UP2

svlan.3002
EVPN flexible-
xconnect-service
EVPN
Day-1 Provisioning Cisco Subscriber Edge
Telco Cloud
• Objective:
Radius
• Control Plane and User Plane Day 1 provisioning
CSE/cnBNG
• Steps: Control Plane (CP)
• Control Plane Provisioning

UP1
• Verify Control Plane K8s Cluster
• User Planes (UP1 and UP2) Provisioning Core
• Verify association state on CP

• Verify association state on UP1 and UP2 UP2
netconf
Provisioning
System
Subscriber Redundancy Group Brief
Telco Cloud
• Objective:
Radius
• Verify User Plane Geo Redundancy • SRG3001 Active
for OLT1
• SRG 3002 Standby CSE/cnBNG
• Steps: for OLT2 Control Plane (CP)
• Verify Subscriber Redundancy Groups

SCi/ CPRi
(SRG) State on Control Plane
UP1
• Verify SRG State on User Planes (UP1 and

UP2)
Core
• SRG3001 Standby UP2

for OLT1
• SRG 3002 Active
for OLT2
OLT L2-Service Auto-Signaling Telco Cloud
EVPN Route Adv
• Objective: • SRG.3001: EVPN L2 ATTRS: Primary-
• SRG.3001: EVPN L2 ATTRS: Primary Radius
> 0x06:0
• SRG.3002: EVPN L2 ATTRS: Backup
• Verify Auto Signalling of L2-Service • SRG.3002: EVPN L2 ATTRS: Backup-
based on SRG State > 0x05:0
CSE/cnBNG
Control Plane
• Steps:
SCi/ CPRi
• Verify EVPN routes on APE1 and APE2 OLT-BRU1
• Notice how EVPN L2 Attributes are

influenced by SRG state for
corresponding route L3
Access Core
OLT-BRU2
Traffic
User Plane Geo Redundancy
• Objective:
Telco Cloud
• User Plane Geo Redundancy Demo
Radius
• Steps:
CSE/cnBNG
• Bringup Subscribers for SRG.3001 and SRG.3002 Control Plane
• Verify Subscribers on CP, UP1 and UP2 SCi/ CPRi

OLT- BRU1
• Start traffic for subscribers

IPoE
L3
Core
Access
• Switchover SRG.3001 on UP1 to UP2 IGW
Internet
• Verify state on CP IPoE

OLT- BRU2
svlan.3002/ srg.3002
APE2 UP2
• Verify state on UP1 and UP2 Spirent 1/ 1

Spirent 1/ 3
• Verify EVPN route on APE1*.
Note: EVPN L2 Attributes value is influenced by SRG state
Conclusion
Service Provider Challenges
Simplify and Modernize Subscriber Edge
Cisco Subscriber Edge and Resiliency
New Revenue Opportunities and Access Auto-signalling
References
• XRDocs Tutorials/Blogs
• https://xrdocs.io/cnbng/
• Cisco.com
• https://www.cisco.com/c/en/us/products/routers/cloud-native-broadband-network-gateway-bng/index.html
• Broadband Forum
• TR-459 Issue 2 : Multi-Service Disaggregated BNG with CUPS. Reference Architecture,
Deployment Models, interface, and Protocol Specifications (https://www.broadband-
forum.org/pdfs/tr-459-2-0-0.pdf)
• WT-474: Subscriber Session Steering
Thank you

BRKSP-2612

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

BRKSP-2612

Uploaded by

Copyright:

Available Formats

Modernizing Broadband Services

with Cisco Subscriber Edge

Gurpreet Dhaliwal, Provider Connectivity TME

Subscriber Services Network

Policy Layer: AAA, PCRF

CDN: Content Delivery Network/ Content

Complex Higher Flat Complex

Simplify Lower Increase Simplify

*definition from oxfordlearnersdictionaries.com

Source: Cisco Annual Internet Report

Bigger More Multiservice

• Higher starting CapEx • Lower starting CapEx

• Defined by Broadband Forum Standard TR-459

Control Plane functions

Data Plane functions

Traffic Steering IPAM Monitoring GTP-u PFCP LI-CP

Keepalive PFCP GTP-u LI-UP Local CP

Access AN Traffic Steering

New Revenue Streams Optimized Operations

• Service offerings based on UP capabilities • Loadbalancing

*from BBF WT-474 Subscriber Session Steering

XR User Plane User/Data Plane:

CSE Ops Center IPAM Monitoring

SCi: State Control Interface (PFCP)

Cisco Subscriber Edge Ops Center

• Easier operations for Network Engineers

• Standard NETCONF, RESTCONF, and CLI

• YANG Models for configurations and operations

IP Pool and Address Management: IPAM

Performance Monitoring (contd)

Performance Monitoring (contd)

All containers are stateless except CDL which

• Whenever a Container fails

• When Server is brought back to operations

• This process continues till all the replicas of the StatefulSet

• During upgrade other PODs continue to function

Cisco CSE CP supports Rolling Upgrade from Day1

Multiple Geographically Separated User Planes

Multiple Technology Support

• A resilient SRG is Standby CP

Home2 OLT2 APE2 UP2

VLAN: B B Active Path

• Each OLT is Single Home

Home2 OLT2 APE2 UP2

• Each OLT is Dual Home

Home2 OLT2 APE2 UP2

• Each OLT is Dual Home

• Active for OLT1

OLT-BRU2 APE2 UP2

OLT-BRU2 APE2 UP2

• Control Plane Provisioning

• Verify Control Plane K8s Cluster

• User Planes (UP1 and UP2) Provisioning Core

• Verify association state on CP

• Verify Subscriber Redundancy Groups

• Verify SRG State on User Planes (UP1 and

• SRG3001 Standby UP2

• Notice how EVPN L2 Attributes are

• Verify Subscribers on CP, UP1 and UP2 SCi/ CPRi

• Start traffic for subscribers

• Verify state on CP IPoE

• Verify state on UP1 and UP2 Spirent 1/ 1

• WT-474: Subscriber Session Steering

You might also like