Tungsten Fabric

Microservice Architecture
& Role in Edge Computing

Qasim Arham
Nov 6, 2018 (OSN Day Dallas)
 Tungsten Fabric Linux Foundation Project

https://tungsten.io/
COMMUNITY MEMBERS

your logo here

Networking for Edge Computing

Networking is most overlooked and underestimated component in any stack

Networking is focal point for most of the security and scalability issues

Tungsten Fabric is fully distributed and Microservices based SDN controller

addressing security, scale and advance networking services

Production grade networking stack for Data Center and Public & Edge cloud

Highly available and ISSU (In Service Software Upgrade) support

Full Fabric Management – Overlay & Underlay Networks

Tungsten Fabric as SDN Controller

RULE THEM ALL WITH ONE

automated secure open SDN Controller

CaaS & PaaS

Public & Private VMs or Metal
IaaS
Architecture Overview
 Visualizing Tungsten Fabric’s Operational Effects
Service Chain
TF Security Policy
Policy with a
Security (e.g. allow only HTTP traffic)
Firewall VNF
(Policy Definition)

Groups VIRTUAL VIRTUAL VIRTUAL

NETWORK NETWORK NETWORK
LOGICAL

GREEN BLUE YELLOW

G1 G2 G3 Non-HTTP B1 B2 B3 Y1 Y2 Y3
traffic

Intra-network traffic Inter-network traffic traversing a service

VM and virtualized Network

function pool
(Policy Enforcement)

G1 B3
PHYSICAL

G2
IP fabric
Y1 G3 B1
(switch underlay) Y2
B2 Y3

Host + Hypervisor
Host + Hypervisor
… …
Tungsten Fabric Multi Cloud
Multi-Cloud Networking for Converged Operators
 Tungsten Fabric vRouter Architecture & Overview
vRouter Agent
• Exchanging control state such as routes with the Control nodes
using XMPP.
Host Compute • Receiving low-level configuration state such as routing instances
vRouter Agent
and forwarding policy from the Control nodes using XMPP
• Reporting analytics state such as logs, statistics, and events to the
Config analytics nodes.
• Installing forwarding state into the forwarding plane
Policy • Discovering the existence and attributes of VMs in cooperation
VRFs
Table Virtual with the Nova agent.
Virtual
Machine Machine • Applying forwarding policy for the first packet of each new flow
(Tenant A) (Tenant B) and installing a flow entry in the flow table of the forwarding
plane.
User Netlink • Proxying DHCP, ARP, DNS
space vRouter Kernel/DPDK
• Encapsulating packets sent from the overlay network and de-
capsulating packets received for the overlay network.
pkt0 tap-abc tap-xyz
• Packets received from the overlay network are assigned to a
Routing Routing
vRouter Kernel routing instance based on the MPLS label or Virtual Network
Instance Instance
Identifier (VNI).
vhost0 • Doing a lookup of the destination address of the in the Forwarding
Information Base (FIB) and forwarding the packet to the correct
destination. The routes may be layer-3 IP prefixes or layer-2 MAC
XMPP ethX OR bondX
Kernel space addresses.
• Doing RPF check before sending Virtual machine traffic to
Control Node destination. This is configurable.
TF VROUTER DEPLOYMENT MODELS
Tungsten Fabric Evolution to Microservices
● Contrail-Control (5 daemons) DaemonSet, Ingress Services with Host
● Contrail-Config ( 8 daemons) Multiple Process running in one
Container (FAT Containers)
Networking
● Contrail-Analytics (5 daemons)
● Contrail-WebUI ( 4 daemons) with choice of run single or multiple
● Contrail-DB (3 daemons) containers per PODs
● Contrail-vRouter (3 D) + Kernel/DPDK (FP)

Contrail Controller: 2n+1

VM VM VM 27-30 Containers Images

OR Analytics Config +
Analytics
DB Control

Kube HA vRouter
BMS MGR Proxy Agent

Contrail 1.X/2.X/3.X
BMS or VMs base
(SDN Controller) Contrail 4.X (Containers) Contrail 5.X (Containers)
BMS or VMs base Microservices
(SDN Controller) (SDN Controller)
TF Helm Microservices Architecture (Helm Charts)
Container (1/4) Kubernetes Cluster
Ingress POD – DaemonSet (3/3) contrail-control-nodemgr POD - DaemonSet
analytics-api Host-Networking contrail-config
Cluster-SVC-Networking
Container (1/4) analyticsdb
contrail-control Ingress Host-Networking contrail-control
Service Host-Networking config-api POD - DaemonSet contrail-webui
analytics-ingress Container (2/4) Cluster-SVC-Networking
analyticsdb-nodemgr contrail-analytics
Cluster-SVC-Networking contrail-dns Ingress Host-Networking
Host-Networking
Service webui contrail-vrouter
analytics-api Container (3/4) Cluster-SVC-Networking POD - DaemonSet
Cluster-SVC-Networking contrail-named Service analytics-zookeeper
Host-Networking Host-Networking
config-api
POD – DaemonSet (7/7) Cluster-SVC-Networking POD - DaemonSet
Container (1/7) POD – DaemonSet (5/5) Service kafka
contrail-analytics-api Container (1/5) config-ingress
Host-Networking
Host-Networking contrail-config-api Cluster-SVC-Networking
POD - DaemonSet
Container (2/7)
Host-Networking
Container (2/5)
Service configdb Contrail Helm
contrail-analytics-nodemgr web-controller Host-Networking
Host-Networking contrail-config-nodemgr Cluster-SVC-Networking
POD - DaemonSet
Toolkit
Host-Networking
Container (3/7) Service configdb-nodemgr
Container (3/5) web-ingress
contrail-collector Host-Networking
Host-Networking contrail-svc-monitor Cluster-SVC-Networking
Host-Networking POD - DaemonSet
Container (4/7) POD – DaemonSet (2/2) config-zookeeper Other Containers:
Container (4/5)
contrail-snmp-collector
contrail-schema-transf Container (1/2) Host-Networking • Contrail-status
Host-Networking
Host-Networking contrail-webui POD - DaemonSet • node-init
Container (5/7) Host-Networking
Container (5/5) redis • vrouter-init-kernel
contrail-query-engine Container (2/2) Host-Networking
Host-Networking contrail-device-mgr • vrouter-init-dpdk
Host-Networking contrail-webui-middleware
Container (6/7)
Contrail-Controller Host-Networking Contrail-Third-Party
Contrail-topology
Host-Networking
Container (1/2) Container (1/3)
Container (7/7) contrail-vrouter-agent POD contrail-vrouter-agent-dpdk POD DaemonSet (2/2) Contrail-
Contrail-alarm-gen Host-Networking
DaemonSet Host-Networking
Host-Networking Container (2/2) Container (2/3) Container (3/3) vRouter
Contrail-Analytics contrail-vrouter-nodemgr
Host-Networking
(2/2) contrail-vrouter-dpdk
Host-Networking
contrail-vrouter-nodemgr
Host-Networking
Tungsten Fabric Integration with ONAP

Orchestration E2E monitoring

Config REST API Analytics REST API
(MSO) (DCAE)
 Edge Computing (Today & Tomorrow)

Today
Cell Sites Core Sites Internet

Tomorrow
BMS,VM,Containers Core Sites Internet
Cell/Edge
Sites

Distributed Core Sites

Colo/Internet
Colocation Data Center

1
4
 TF Distributed Compute Architecture
Light version: BGP not
extended to Edge sites only
Core/Distributed Core Site Core/Distributed Core Site XMPP to Edge Sites
Distributed Compute (Sub-Cluster-ID) Distributed Compute Light (Sub-Cluster-ID)
Local Compute Local Compute
Local Compute Local Compute
Local Compute Local Compute
TF SDN Controller (Cluster) TF SDN Controller (Cluster)

Sub-Cluster Controller (1) Sub-Cluster Controller (n) Sub-Cluster Controller (1) Sub-Cluster Controller (n)

BGP
BGP
XMPP XMPP
IP/MPLS IP/MPLS
Backbone/RAN Transport
Backbone/RAN Transport

Local Compute Local Compute Local Compute Local Compute

Local Compute Local Compute Local Compute Local Compute
Distributed Compute Distributed Compute Distributed Compute Distributed Compute

Edge Site (1) Edge Site (n) Edge Site (1) Edge Site (n)
TF Distributed Compute Architecture
Core/Distributed Core Site
Distributed Compute (Sub-Cluster-ID) with MC-GW
Local Compute
Local Compute
Local Compute
Contrail SDN Controller (Cluster)

Sub-Cluster Controller (1) Sub-Cluster Controller (n)

IPsec/SSL
XMPP
Multi-Cloud
Gateway
Colo/Internet/IP/MPLS
Backbone/RAN Transport

Local Compute Local Compute

Local Compute Local Compute
Distributed Compute Distributed Compute

Edge Site (1) Edge Site (n)

1
6
TF as Single SDN for VMs, PODs & BMS
BMS
Virtual Network-01
10.1.1.100/24

BMS
Virtual Network-02
20.1.1.100/24 On-Prem:
BMS & Fabric
●
●
Core Site
Core Distributed Site Manager
VM ● Edge Site
Virtual Network-01
10.1.1.3/24

VM
Neutron/CNI/DM/Fabric
SDN Controller
Kubernetes
Virtual Network-02
20.1.1.3/24 Edge/MC-GW CNI
POD
Virtual
Network-01
10.1.1.5/24
Basic Networking:
L2/L3 or L2/L3 Network, IPAM/DHCP, DNS, Multi-Tenancy
Advance Networking:
OpenStack
POD
Virtual
VLAN-ID, VRRP, VIP, Load Balancer, Routes Advertisement,GW
Function, Service Chaining, Traffic Steering, Flow
awareness,QoS, vRouter Kernel/DPDK, SR-IOV,, BGP-
Neutron Plugin
Network-02 VPN,Inter Site Federation DCI, Health Checks, FW, IPSec/SSL
20.1.1.5/24
Support, Distributed Compute, Edge Fabric Management, Multi-
Cloud support, Multi-tenancy (to support network slicing)
 5G Edge Computing and Encryption
Edge Site (Data Center) Centralized
Data Center
APP
VNFs UPF
Cell Sites IPSec or SSL Tunnel
CU
DU (Central Unit)
(Distributed Unit)
PPF UPF CCF
RRU RPF RCF VNFs
Core Network VNFs
VNFs

APP Application
CCF Core Control Function (Core Network)
UPF User Plane Function (Core Network)
Secure RAN to CN RCF Radio Control Function (RAN)
● Use Contrail Encryption to secure Remote Edge and Central DC connection. PPF Packet Processing Function (RAN)
RPF Radio Processing Function (RAN)
● Secure Overlay site to site communication via Contrail encryption support RRU Remote Radio Unit (RAN)
● Policy based encryption model
SOFTWARE DEFINED SECURE NETWORKING
Tungsten fabric provides a rich, consistent set of security policy capabilities across multiple platforms.

1. Simplified Manageability (change control, etc. is much easier)

2. Improved Scalability
App1, Deployment = Dev
3. Define / Review / Approve Once → Use Everywhere

Web App db

App1, Deployment = Dev-K8s

App1, Deployment = Staging
Network Policy Web App db
Web App db
vR
ou
ter
Se
cur

Mana
Devic r
App1, Deployment = Prod ity
Gr
ou
ps

ge
e
Web App db
App1, Deployment = Dev-

… Bare Metal Server

s
App1, Deployment = Staging-BMS Web
Mesos
Ap
db
p

Web App db
Tungsten Fabric INSTALLATION

• Ansible playbook to flexibly deploy Tungsten Fabric binaries

• Helm charts to easily operate Tungsten Fabric components on Kubernetes

• Install-time option with OpenShift to deploy with Tungsten Fabric

• Tungsten Fabric binaries available on DockerHub and we’re improving

CI/CD

• Commercial integrations into lifecycle tools like RH OpenStack Director

 Tungsten Fabric K8s CNI (A single YAML Install & CARBIDE)

CentOS Single YAML

Ubuntu Single YAML

Reference: https://github.com/Juniper/contrail-kubernetes-docs
 Carbide Sandbox Environment

Tungsten Fabric + Kubernetes on AWS

https://tungsten.io/start/
0-60 in 15 Minutes Flat w/Carbide (TF+k8s on
AWS)
0-60 in 15 Minutes Flat w/Carbide (TF+k8s on
AWS)
0-60 in 15 Minutes Flat w/Carbide (TF+k8s on AWS)
Carbide EC2 Instances overview

tungstenfabric-k8s-aws-master-node

tungstenfabric-k8s-aws_control1

tungstenfabric-k8s-aws_compute1

tungstenfabric-k8s-aws_compute2
0-60 in 15 Minutes Flat w/Carbide (TF+k8s on AWS)
Try Tungsten Fabric

https://tungstenfabric.github.io/website/Tungsten-Fabric-15-minute-
deployment-with-k8s-on-AWS.html