Technical Proposal - Solution Overview - Printed

Engineering, Procurement and Construction of MAN
Response to
Invitation to Tender

Ethernet - Package 2
Vietnam Posts and Telecommunications Group

(VNPT)
Technical Proposal - Item 1
Technical Proposal (Solution

Overview)
3rd July 2008 Commercial-In-Confidence Page 1

Table of Content
1. Executive Summary 4
1.1 Cisco Carrier Ethernet Solution 4
1.2 Cisco Solution Architecture Advantages 4
1.3 Cisco presence in Vietnam 5
1.4 Cisco Differentiators 5
1.5 Conclusion 6
2. VNPT Network Architecture 7
2.1. Physical Network Design 7
2.2. Proposed MAN Equipment 8
2.3. VNPT MAN Transport Options 9
2.4. Integrating ME61 and VN2 10
2.5. Integrating ME61 to Hanoi PT and HCMC PT MAN 12
2.6. MAN Transport Protocol Architecture 12
2.6.1. MAN IP/MPLS Architecture 12
2.6.2. MAN IGP OSPF Architecture 13
2.6.3. MP-BGP4 (Multiprotocol BGP) 14
2.6.4. BGP Route Reflectors for VNPT MAN Network 14
2.6.5. Using BGP for VPLS Auto Discovery 15
2.6.6. LDP (Label Distribution Protocol) 15
2.7. VNPT MAN Service Architecture 16
2.7.1. MSAN Voice Service Architecture 16
2.7.1.1. IP VPN Transport Option for MSAN Voice Service 17
2.7.1.2. E-LAN Transport Option for MSAN Voice Service 18
2.7.2. Residential Service Logical Architecture 18
2.7.2.1. Residential (& Business) High Speed Internet Access 18
2.7.2.2. Residential IPTV 20
2.7.2.3. Video on Demand—IP Unicast Routing 22
2.7.3. Business Service Logical Architecture 23
2.7.3.1. E-Line 23
2.7.3.2. E-LAN 25
2.7.3.3. Layer 3 VPN Services 26
2.7.3.4. Circuit Emulation Services 27
3. Resiliency Strategy 29
3.1. Service Level Resiliency 29
3.2. Network Level Resiliency 29
3.3. Device Level Resiliency 30
3.4. In Service Software Upgrade 31
3.5. Link Level Resiliency 32
3.6. The truth about 50ms resiliency 32
3.7. Ensuring High Availability Of Physical Component 37
4. QoS Architecture 45
4.1. Cos/Qos Mechanisms 45
4.2. QoS Features in VNPT MAN 46
4.3. MAN Core routers 47
4.4. Application Versus Transport Services 47
5. Security Architecture 49
5.1. Control, Data and Management Planes 49
5.2. Security Threats 50
5.3. Trust Model 51
5.4. Baseline Infrastructure Protection Leading Practices 52
6. IP Version 6 55
6.1. Planning to Deploy IPv6 55

6.2. IPv6 Over MPLS 56
7. Element Management System 58
7.1. Introduction 58
7.2. Integrated Metro E aggregation Management Solution 58
7.3. Integrated Metro E Management System Functional Design 60
7.4. Deployment Strategy 63
7.5. Conclusion 63
7.6. Appendix A 64
8. MAN-E Sparing Strategy 70
9. Total Power Consumption for Proposed system of Cisco 7600 73

1. Executive Summary
Cisco Systems is pleased to have this opportunity to present VNPT with a proposal for implementing the
Cisco® Carrier Ethernet.
We are confident that VNPT will find the Cisco Carrier Ethernet a comprehensive and industry-leading
solution with a compelling value proposition. Cisco is committed to delivering innovative technology and
business strategies that enable our customers to drive growth and profitability.
Cisco‘s recommended solution will meet/exceed VNPT’s transport and service needs besides enabling them to
lay the foundations of an IP Next-Generation Network.
Cisco believes that in selecting the proposed Cisco solution, VNPT will position itself to provide a wide range
of Next-Generation IP-based services to Enterprise, SME and Residential customers, delivered on a highly
resilient, highly available Carrier Ethernet network supporting multiple access methods and multiple types of
end-devices.
The integrated design and operation of the proposed Carrier Ethernet architecture, tried and tested in the
Service Provider environment, allows a high degree of operational flexibility and simplification previously not
possible. This in turn translates into reduced operational cost, increased speed of application deployment,
greater customer attraction and retention, reliability and longevity.
1.1 Cisco Carrier Ethernet Solution

The Cisco Carrier Ethernet Solution will enable VNPT to seamlessly offer:
Video services – In broadband environments it is common to receive television and different kinds of
on-demand programming through IP. This transition will continue and also involve interactive video
services.
Broadband networking – While broadband deployments are rapidly expanding, the transition from being
used as simple access to providing for networked home environments has just started.
Wireline data services – Most business data services are still being delivered through last-mile Frame
Relay or ATM services. The IP infrastructure will offer these plus other services on the same
infrastructure, meeting the same customer delivery requirements.
Telephony services – Traditional carrier telephony has been moving toward packet technology for some
time and is now poised for a breakthrough as several traditional circuit-switching vendors are shipping
packet-based voice solutions. In addition, the convergence between fixed and mobile voice services is
just about to begin.
Wireless and mobile services – As mobile services converge with fixed voice services, the mobile
handset is turning into a multimedia window into the converged voice, video, and data world, with real-
time video and online gaming ready to take off on this communication device.
1.2 Cisco Solution Architecture Advantages

Supports all the required services by VNPT including E-Line, ELAN, E-Tree, CES, IP
Voice, IPTV, IPv4 VPN and IPv6 VPN
Positioned Cisco 7600 Series Router for Both Core and Access, for ease of operation,
provisioning and management. Efficient sparing and lowere training cost for VNPT
staff
Support L2 and L3 VPN in Both Core and Access, flexible service depolyment for
VNPT
Supports IPv6 in hardware for Both Core and Access
Supports E1, E3 and STM-1 Circuit Emulation on Both Core and Access

Ensure Network High Availability by providing In Chassis Hardware Redundancy,

Link Redundancy and PE redundancy
Ensure Service High Availability using None Stop Forwarding, Stateful Switch Over,
Traffic Engineering Fast Reroute, Fast IGP Convergence, Psuedowaire Redundancy,
Gateway Redundancy etc
Provides Quality of Service for Voice, Video and critical business traffic
Protects Network Against Securty Threats
1.3 Cisco presence in Vietnam

Cisco places significant emphasis on building our existing strong relationship with VNPT.
The following are some of the key areas where we think VNPT can benefit from as a result of the strong Cisco
presence in Vietnam
Account team resources focused on providing value to VNPT.
Open access to Cisco technical specialist and subject matter experts.
Access to Demo and Evaluation equipment as required and appropriate.
Advanced Services Team to support technical knowledge transfer and on-going operational assistance.
Labs, equipment, tools, and other resources in various Cisco offices in Hanoi and HCMC ready to
support VNPT.
1.4 Cisco Differentiators

Cisco’s vision, technology innovations, and extensive experience make us uniquely qualified to deliver on the
complex and wide-ranging needs of VNPT has to make in the transition to IP NGN. Our key strengths
include:
• Comprehensive Solution Portfolio—Many vendors have narrowly defined
the term NGN to address only a small portion of the significant network,
such as IPTV or Metro-E. Cisco takes a comprehensive view of the
intelligent IP-based NGN that encompasses the service provider’s entire
network (edge-to-core) and most importantly, its business.
• State-of-the-art Service Exchange Framework (SEF)—Cisco’s SEF is the
only comprehensive set of service-enabling technologies that provide the
four critical building blocks for service control: Subscriber awareness and
identity management, policy and resource management, dynamic session
management, and industry-leading subscriber management. Through
Cisco’s SEF, service providers gain maximum control over user sessions on
both broadband and mobile networks, allowing true subscriber awareness
and visibility into network activity.
• Widespread Adoption—In the global migration towards IP NGNs, service
providers worldwide are rapidly adopting Cisco IP Carrier Ethernet
solutions. Early adopters include French Telecom, Italia Telecom, Sprint,
British Telecom, Comcast, Kabel Deutschland, NTT, and other leading
providers around the world.
• Most Technological Innovation—Since 2000, Cisco has spent an averaged
of $3.5 billion annually in technology R&D with 50 percent dedicated to
developing service provider solutions—and the investment is paying off.
Our recent innovative and industry-leading NGN solutions include Next
Generation Routing Software (IOS-XR), Cisco Carrier Routing System
(CRS-1), Cisco Ethernet Service modules for Cisco 7600 Carrier Ethernet

platform, Cisco Distributed Denial of Service (DDoS) Protection, Cisco

WebVPN Services Module, and the Cisco Reconfigurable Add/Drop
Multiplexing (ROADM).
• Leader in IP Experience—With Cisco’s 20-year history of IP innovation,
successful implementations and the broadest intelligent IP product portfolio
in the industry, we are uniquely positioned to lead the way in bringing about
your network transformation.
• Carrier- Class Approach—Our mission is to lead the industry in the
reliability, availability, and serviceability of today’s and tomorrow’s IP
networks to ensure that the solutions we develop meet or exceed customer
expectations. To do so, Cisco continues to make significant investments to
provide quality in four key service provider areas: hardware and software,
commitment and compliance management, end-to-end system architecture,
and service and support.
• Standards-Based for Easy Integration—Cisco knows standards are what its
customers want and need for cost-effective and easy-to-manage
communications systems. To be the leader in IP solutions for service
providers, we support multi-vendor interoperability with open standards
interfaces to allow for the easiest integration with your enterprise
environment. We maintain global awareness of activities that could affect
product development and ensure that we expedite standards production life
cycles.
• Solid Professional Implementation—Cisco provides the tools, resources,
expertise, and best practice methodologies needed to deliver its IP NGN
solution faster and more cost-effectively to market. Services include network
strategy, architectural design, project management, site surveys, staging,
installation, and test and deployment.
1.5 Conclusion
As the leader in networking, Cisco can provide VNPT solutions at every step of its IP NGN transformation.
We can assist you in building an efficient, long-lasting network infrastructure that will help VNPT deliver
unique and competitively differentiated services in the market. Working with Cisco, VNPT can look forward
to building a next-generation architecture that offers the revenue-producing and innovative services that your
customers demand from you.

2. VNPT Network Architecture
VNPT Network Architecture
VN2 Core Internet

Gateway
VN2
DIA
Backbone
VPN
ME
BRAS
BRAS Large
ME
VPN
VPN VPN
VNPT_ME_Design © 2006 Cisco Systems, Inc. All rights reserved. Cisco Confidential 2
Figure 1 – VNPT Target Network Architecture
2.1. Physical Network Design

The physical network design for VNPT MAN encompasses the following design criteria :
1) Hierarchical – A hierarchical approach allows for the network to be broken into smaller and
manageable pieces
2) Simplicity – A simple approach to the design avoids unnecessary complication which ultimately
affects availability
3) Modular – The various components of the network is built in a modular fashion, so that addition and
deletion of a component does not affect the rest of the network. This applies to the construction of the
individual services
4) Repeatable – Since the MAN is going to span a wide geographical area, the sub-networks have to be
constructed in a repeatable fashion so as to avoid delay and mistake
With the above criteria in mind, it is recommended that the physical network of VNPT MAN be made up of
the following different tiers of network :
Core Ring – Built using MAN Core router to aggregate the Access Rings and to interface the province MAN
to the VN2 Core
Access Ring – To aggregate access connections to MAN, including DSLAM, Ethernet and future Wimax and
mobile.
The Core Ring and Access Ring connections are built out of GE or Ten GE, depending on the bandwidth
requirement as per “Bandwidth Calculation” in the attached appendix.

2.2. Proposed MAN Equipment
Cisco 7600 series Carrier Ethernet Router is proposed as the Core and Access router for VNPT MAN solution.
Cisco 7600 series Carrier Ethernet Router enable high-performance IP/MPLS features as well as scalable
personalized IP services at the network edge, improve operational efficiency, and maximize return on network
investments. Cisco 7600 series router is proven and currently operational in Hanoi PT MAN and HCMC PT
MAN.
• The benefits of positioning the same platform for both Core and Access Router includes,
• Ease of configuration, provisioning and management
• Efficient and common equipment sparing
• Flexible equipment deployment for Core and Access
• Lower training cost for VNPT staff
The Cisco 7600 Series is the industry's first carrier-class edge router to offer integrated, high-density Ethernet
switching, carrier-class IP/MPLS routing, and 10-Gbps interfaces, benefiting enterprises and helping enable
service providers to deliver both consumer and business services over a single converged Carrier Ethernet
network.
Important Features:
• High performance, with up to 720 Gbps in a single chassis, or 80 Gbps capacity per slot. Optional
32Gbps SC for Access.
• A choice of form factors purpose-built for high availability
• Cisco I-Flex design: A portfolio of shared port adapters (SPAs) and SPA interface processors (SIPs)
that controls voice, video, and data experiences
• Scalable and extensible suite of hardware and software capabilities to enable intelligent Carrier
Ethernet services
• Integrated Video Call Admission Control with innovative visual quality of experience for both
broadcast and video on demand (VoD)
• Intelligent Services Gateway, providing scalable subscriber and application awareness with
multidimensional identity capabilities and policy controls
• Integrated Session Border Control with quality of experience in both Session Initiated Protocol (SIP)
and non-SIP applications.
• Share the same Shared Port Adapter with Cisco CRS-1, Cisco GSR12000, Cisco 10000 and Cisco
ASR 1000 series router
The Cisco 7600 chassis accommodates a broad selection of line cards supporting numerous applications,
including:
• SPAs and SIPs (Cisco 7600 Series SPA Interface Processor-200 [SIP-200], SIP-400, and SIP-600)
• Channelized T1/E1, Channelized T3, and Channelized OC-3/STM-1
• Circuit Emulation T1/E1, E3 and STM-1

• OC-3/STM-1, OC-12/STM-4, OC-48/STM-16 Packet over SONET/SDH (PoS), and OC-192/STM-64

PoS
• OC-3/STM-1 ATM, OC-12/STM-4 ATM, and OC-48/STM-16 ATM
• Fast Ethernet, Gigabit Ethernet, and 10 Gigabit Ethernet
• Enhanced FlexWAN module: Supporting Cisco 7200 and 7500 WAN Port Adapters from DS-0 to
OC-3 for channelized and ATM interfaces and also Fast Ethernet port adapters
• High-density Ethernet services modules: 10/100 Mbps, Gigabit Ethernet, and 10-Gigabit Ethernet
• Services modules: IP Security (IPsec), firewall, distributed denial of service, intrusion detection
systems, network analysis, and content switching commonly used, for example, in the Cisco Mobile
Exchange solution
2.3. VNPT MAN Transport Options

The benefits of an IP MAN are well understood: rapid delivery of rich multimedia services, increase
efficiency, enhanced resource management, carrier class resilence and availability, and superior service and
network control – a complete business transformation. Flexible infrastructure becomes essential to meet those
goals. The proposed IP NGN Carrier Ethernet based MAN should support all residential quadruple-play
services and standard Layer 2 MEF point-to-point and multipoint business services plus Layer 3 IP
MPLS/Multicast VPNs for business customers.
The two most deployed transport architectures are native 802.1ad (or QinQ) and MPLS. The native 802.1ad
uses S-VLAN for service delivery and STP for resilence support. Before the 3-play and mission critical VPN
are transported by the MAN, this architecture was widely deployed since double tagging expands the services
significantly beyond what 802.1q supports. However, the nature of the slow convergence of STP is challenged
if voice and video services are added. It also constrains the network intelligence due to the L2 limitation. For
instance, bandwidth management can’t be supported without a topology aware NMS. On the other hand,
MPLS MANs have been built by major SPs in the world to roll out triple-play services for their residential
and enterprise customers.
The capabilities of NGN MAN will transform the legacy MAN from the transport Ethernet to service enabled
Ethernet. Cisco IP NGN CE based MAN architecture is that multiple Layer 2-3 network technologies are used
to provide optimal flexibility for VNPT current and next-generation service offerings. These technologies
and protocols include native Ethernet VLAN(802.1q), 802.1ad, EoMPLS, Layer 3 PIM-SSM, MPLS VPN,
and H-VPLS. This allows VNPT to support a broad range of applications while minimizing capital and
operating expenses associated with the network infrasturcture.
The NGN MAN architecture also achieves carrier-class resilience and troubleshooting requirements.
Standards-based OAM end-to-end across the access and core switches allow VNPT to manage Ethernet
connectivity all the way to the customer premises. This includes 802.1ag for service verification, 802.3ah for
link-layer troubleshooting, and E-LMI for service status and auto provisioning of customer equipment. Cisco
IP SLA integration with 802.1ag enables VNPT to offer Ethernet layer SLA. The architecture provides a
highly available solution that supports sub-second multicast convergence and link restoration across the end-
to-end network. Using FRR for the aggregration and distribution EoMPLS tunnels, 50 ms restoration can be
achieved. This enables VNPT to deploy NGN voice with carrier-class protection.
Bundled services are offered at attractive price points so as to encourage VPNT subscribers to purchase all
services from a single vendor. Multimedia service integration is an important factor for IP convergnece in the
network.Video services delivered using IPTV and IP VoD will consume the largest portion of NGN MAN
bandwidth. It is vital that the NGN MAN be able to scale to tens and even hundreds of Gbps. The Cisco Video
CAC solution based on RSVP signaling provides comprehensive and reliable admission control to enable
VNPT to effieciently manage bandwidth growth generated by the video services. The proposed NGN MAN

architecture employs Layer 3 in the video distribution network to take advantage of PIM-SSM and IGP
enhancements that enable fast network-level convergence for greater network efficiency and scale. More
importantly, these Layer 3 fast convergence techniques provide consistent sub-second network and application
recovery for all failure modes.
While Cisco’s proposed architecture is field proven, two emerging packet transport architectures have been
articulated to VNPT by other vendors. They are T-MPLS and PBT/PBB-TE. First, T-MPLS, a connection-
oriented packet standard formulated by the ITU-T specifically for application in transport network, is
officially dead. After a lot of debate and concerns related to possible interoperability issues with the widely
deployed MPLS networks, a new Joint Working Team, consisting of members from both the IETF and ITU-T,
has been formed to work on extending the current IETF defined MPLS functionality and to develop a
transport profile for MPLS which will be referred to as MPLS-TP.
The MPLS-TP will use current existing MPLS data plane architecture while allowing service providers to
statically provision LSPs or tunnels, use traditional protection schemes like 1:1, 1+1 and ring topologies and
transport-centric OAM tools that line up with established architecture and support Performance Monitoring
and Fault, Configuration, Accounting and Performance management. This change in events with respect to the
old T-MPLS standard and the initiation of the MPLS-TP is a good endorsement for matured and this proven
IP/MPLS technology will be used for the VNPT NGN MAN proposed by Cisco.
Second, PBT, a vendor proprietary protocol, has been widely over hyped as the next major protocol to deliver
capex and opex reductions. PBT was created to use Ethernet for connection-oriented purposes similiar to T-
MPLS. PBT strips the complexity out of Ethernet by removing spanning tree, flooding, and broadcasting.
Service providers have to employ a proprietary network management system to provision point-to-point PBT
tunnel across the Ethernet network. PBT employs the data encapsulation mechanism standardized by 802.1ah
(PBB). It doesn’t add any new levels of overall network scalability beyond PBB. Although there is some
appeal to deploying PBT point-to-point Ethernet transport services, organizations must continue to apply
additional rigor in the examination and standardization of PBT to determine its effective benefits to service
providers and their customers. When examined in the context of deploying multiple services over a converged
network from both a cost and simplicity perspective, PBT does not offer a compelling advantage over
IP/MPLS technologies, which have already matured and been adapted to deploy highly scalable, reliable, and
cost-effective Carrier Ethernet deployments worldwide.
Cisco continues to innovate and promote standard development in IP/MPLS, Ethernet, OAM, Multicast, QoS,
and video networking technologies. VNPT can choose the amount of intelligent participation by the Carrier
Ethernet networking platforms to simplify the deployment of VNPT bandwidth-intensive business and
entertainment-grade consumer services. To meet the ever growing customer and service expectations, Ethernet
must be pragmatically adapted and combined with IP/MPLS technologies in VNPT NGN MAN to efficiently
optimize the quality of experience for all services and customers. A combination of IP/MPLS and native
Ethernet technologies employed in Cisco proposed architecture will deliver on the promise of scalable and
reliable converged VNPT NGN MAN.
2.4. Integrating ME61 and VN2

The back-to-back interconnect shown in Figure 2 is the most popular design choice in existing deployment.
The 802.1q or 802.1ad trunk provides VLANs for interconnected services. It preserves existing rich sets of
QoS applied to IP traffic on per VLAN basis. The 802.1q trunk can provide 4096 VLANs and the 802.1ad
trunk can be used to support more VLANs. All inter VN2 services in the MAN will be terminated to VLANs
on the MAN PE. Each VLAN on VN-2 PE can be configured as a VRF for L3 VPN service, a L3 interface for
IGP connectivity, or a VLAN for L2 VPN service. If there is no IGP connectivity between VN-2 and MAN,
each MAN MPLS domain can be managed separately. The MPLS control plane scalability will be less of a
concern. Furthermore, this design can interconnect two different transport architectures like MPLS core and
802.1ad MAN since it doesn’t require L3 connectivity.

VNPT Network Transport Architecture

802.1ad Interconnect VN2 Domain and ME61 Domains
VN2
Backbone
VN2 MPLS
802.1ad
802.1ad
MAN
MAN MPLS MAN
MAN MPLS
Figure 2 – MAN & VN2 integration using IEEE 802.1ad option
VNPT Network Transport Architecture

Single MPLS Domain for VN2 and ME61
VN2
Backbone
MPLS
MPLS
MAN
MPLS
MAN
Figure 3 – MAN & VN2 integration using MPLS option

The design above shows that the L3 VPN service requires one eBGP session per VRF on the VN-2 PE. The
VN-2 PE control may not scale as well as the required number of eBGP sessions is equal to the number of
VRFs for L3 VPN service. A design alternative shown in Figure 3 provides end-to-end MPLS connectivity.
However, complex multiple AS and inter-AS designs are required to scale the IGP routing across MPLS
domains. Unless a large number of VRFs (> 2000) is needed per VN-2 PE, this design is not recommended.
2.5. Integrating ME61 to Hanoi PT and HCMC PT MAN

The existing MAN and Hanoi and HCMC are built out of Cisco 7600 Series routers. Each of this MAN is
going through upgrade and expansion to canter for the ever increasing requirement for bandwidth and
services.
The current HSI model deployed in Hanoi PT is 1:1 VLAN mapping for DLSAM QinQ and N:1 model is
used in HCMPT. With the experience and knowledge Cisco and our partners have in configuring and
deploying this 2 MANs, we are confident that we’ll be able to integrate the new ME61 with HCMC PT and
Hanoi PT MAN to offer seamless intra provinces between this MANs.
2.6. MAN Transport Protocol Architecture

This section deals with the protocols that the routers will require to provide an MPLS network and associated
IP services for VNPT’s MAN.
2.6.1. MAN IP/MPLS Architecture

The MAN network architecture for each province should be built based upon a single Autonomous System
(AS) design. Within this single AS, the network is then logically segmented into multiple IGP areas for
administration and scalability reasons. The single AS approach not only allows VNPT to exercise a consistent
network policy, it also deals with critical implementation issues that comes with a multi-AS approach. Some
of these issues include :
Multicast service – it is not a straight forward design for multicast service to run across multiple AS(s). This
means added complexity in rolling out services like IPTV
VPN services – whether it is point to point or multipoint-to-multiple Layer 2 service, stitching different layer
2 circuits across multi-AS network is not trivial. Same goes for layer 3 VPN service as well
Resiliency – the resiliency strategy of VNPT’s MAN is based upon fast detection and recovery mechanisms.
The problem with a multi-AS network is that the signaling protocol used (or the lack of it) between AS(s)
adds complexity and challenges in achieving the desired recovery time
Besides the engineering problems associated with a multi-AS design, new hardware architecture and
improvement made to routing protocols allows VNPT to build its MAN based on a single AS. For example, it
is now possible to include hundreds of routers within an OSPF area. In this case, each VNPT MAN network
can be built by segmenting the network into multiple areas.
The following routing protocols will operate in VNPT MAN networks :
OSPF will function as the Interior Gateway routing Protocol (IGP) providing IP connectivity amongst all
routers (Core & Access).
MP-BGP4 (Multi-protocol Border Gateway Protocol V4) is used to distribute VPN routing information
amongst the PE routers. It also distributes VPN-related labels to PE routers.
LDP (Label Distribution Protocol) is responsible for distributing label information for IP destinations within
the global routing table only. That is, P and PE addresses that is not part of a VPN.

2.6.2. MAN IGP OSPF Architecture

An Interior Gateway Routing Protocol (IGP) must be operating amongst the Core and Access’s routers before
the label distribution (LDP) and VPN routing distribution (BGP4) can execute.
Cisco recommends that the interior routing protocol for the MAN’ global routing table be OSPF. Many
enhancements have been made to the protocol, such as in the area of resiliency and optimization to support a
large network.
OSPF is particularly suited to hierarchical routing environments, where portions of the network can be
partitioned into areas. This can help reduce very large routing tables through route summarization at area
border routers (ABR). It also assists in reducing the number of link state changes propagating through a
network as they will be contained to that area. The following is an example of how the VNPT OSPF network
can be divided :
VNPT MAN OSPF Design
Access Ring
Area 3
Core Ring
Area 0
Access Ring
Area 1
Access Ring
Area 2
Figure 4 - Sample OSPF design
The VNPT’s OSPF network architecture should consists of an area 0 containing all the routers in the Core
Ring. Fanning out of these individual routers are multiple sub-areas, each containing Access routers. These
sub-areas contain routers that belong to a specific geographic region in the metro area network. Depending on
location and traffic pattern, some large metro area may even be within its own sub-area. By carving the entire
MAN network into multiple areas, one can keep the global routing table small for efficiency and link flaps can
be contain within a geographic region, so as not to affect other parts of the network.
One thing must be considered when using multiple areas, and that is route summarization must never be done
on the loopback addresses (or the network they reside in to be specific) of the Core and Access routers as this
will cause the label switching path to terminate where the summarization occurs. This will result in loss of
connectivity.
Since all circuits are provisioned predominantly on an existing SDH infrastructure, it is highly unlikely that
circuit reliability will cause excessive link state changes due to flapping circuits. Hence the OSPF protocol
should remain fairly stable for VNPT’s MAN networks.

As has been discussed in the Physical design, resiliency and redundancy to circuit failure is provided by the
convergence capabilities of the OSPF routing protocol. This is in contrast to the layer 1 and layer 2
redundancy provided by an SDH or SONET ring using Auto-protection switching. The key to fast
convergence is in the detection of failure and re-calculation activity. Improvements have been made in these
areas, and they are listed in the appendix.
2.6.3. MP-BGP4 (Multiprotocol BGP)

The Multi-Protocol BGP (MP-BGP) is necessary to provide VPN services to VNPT’s MAN Layer 3 VPN
customers. The protocol is used to propagate VPN routing information among the PE routers.
In the MPLS network, each PE defined VPN consists of VPN Routing and forwarding tables (VRF’s) which
are associated with each customer interface. The VRF tables consist of unique VPN-IPv4 addresses, (they are
unique as each is prefixed with the Route Distinguisher). Since these are not IPv4 addresses, BGP provides
multiprotocol extensions that allow the distribution of these VPN-IPv4 routes.
BGP propagates VPN-IPv4 information using the BGP multiprotocol extensions for handling these extended
addresses. (See RFC 2283, Multiprotocol Extensions for BGP-4.) It propagates reachability information,
expressed as VPN-IPv4 addresses, among the PE routers only. The reachability information for a given VPN
is propagated only to other members of that VPN. The BGP multiprotocol extensions identify the valid
recipients for VPN routing information. All the members of the VPN learn routes to other members.
A Core of Access router can function as a IP VPN PE routers to provide the most flexible L3 architecture. A
detail discussion in the area of IP VPN architecture in later section.
For VNPT MAN, each MAN network will be built upon a single AS.
2.6.4. BGP Route Reflectors for VNPT MAN Network

BGP Route Reflectors will be an integral part of VNPT’s MAN network. The basic rule for IBGP peering
within a network is that all peers need to form full mesh connections with the other peers. Without route
reflectors, whenever a new PE is introduced, each existing PE in the VNPT network will need to have an extra
BGP neighbor command added pointing to the new PE.
When the number of PE’s becomes too great to make this operation practical (that is, adding neighbor
commands in every PE) then BGP Route Reflectors should be introduced. Route Reflectors obviate the need
to fully mesh the BGP peers and avoid adding the neighbor commands to each PE.
With Route Reflectors, the PE’s would only require neighbors defined for each route reflector. Any updates,
including VRF information, would be sent to the Route reflectors only. The Route Reflectors are then
responsible for propagating any information received from PE’s to all other PE’s. Each time a new PE is
added, only the Route Reflectors would need to be updated with neighbor statements.
Route Reflectors are also useful for a customer that has connections to several PE’s (dual homing). In the
situation where a route change occurs in the customer network, the PE that terminates that part of the
customer network, would have to update every PE peer participating in that VPN. Route Reflectors would
remove the burden of BGP updates from the PE.
The following diagram shows the effect of Route Reflectors. Each of the PE routers, without Route
Reflectors, requires 5 peers. Adding Route Reflectors in this example reduce each PE peerings to 2.

Figure 5 - MPLS VPN with Route Reflector design
For the VNPT MPLS network:

Only PE routers participate as peers in the MP-BGP network. The P routers do not run BGP for VPN route
distribution.
In the initial implementation, each PE router should peer to a pair of route reflectors, located in major PoP of
the province. The route reflectors should be reconfigured to support rr-groups and initially all VRF’s should
export the same RR route-target so that both route reflectors hold all the VPN routes. As the networks grows
additional route reflectors could be introduced by simply changing the route-target on the VRF to export to
the new route reflectors.
The AS number used between BGP peers can be assigned by VNPT during planning stage and need not be an
existing AS number.
All routers will peer, using the BGP neighbor statement, to the loopback address of the adjacency. The
loopback address is used to make sure the IP address of interface stays up and is independent of an interface
that might be unstable.
VNPT should consider to position a pair of dedicated route reflector for each province’s MAN.
BGP and route reflector can also be use for VPLS auto discovery.
2.6.5. Using BGP for VPLS Auto Discovery

VPLS Autodiscovery enables each Virtual Private LAN Service (VPLS) provider edge (PE) router to discover
which other PE routers are part of the same VPLS domain. VPLS Autodiscovery also automatically detects
when PE routers are added to or removed from the VPLS domain. VNPT no longer need to manually
configure the VPLS and maintain the configuration when a PE router is added or deleted. VPLS
Autodiscovery uses the Border Gateway Protocol (BGP) to discover the VPLS members and to set up and tear
down pseudo wires in the VPLS.
2.6.6. LDP (Label Distribution Protocol)

LDP is responsible for distributing the labels associated with every IP destination prefix in the MPLS
network. Labels will be assigned to every address that is in the OSPF global routing table. Essentially all IP
destination prefixes will be either a loopback or circuit interface address. The Label Distribution Protocol co-
ordinates the allocation and propagation of labels.
The P routers perform the label switching/forwarding part of the MPLS network. The core only has an
understanding of labels that are associated with IP destinations in the OSPF internal routing table. They have
no knowledge of labels related to routes in customer’s VPN network . Therefore the labeled IP packet is label
switched to the next-hop destination, that is, the PE where the customer network is connected either within
the same PoP or via other connections.

2.7. VNPT MAN Service Architecture

The following is an overview of the service architecture design for VNPT MAN. Business and residential
customers can be connected to the MAN using various access technology including DSL, Ethernet, FTTx and
Wireless. This access equipment is aggregated at the MAN Access Routers using Gigabit Ethernet. Service
separation is achieved via by running IEEE 802.1q and IEEE 802.1ad trunk between the MAN Access
Routers and the access equipment. In the MAN Access Routers, different services are then being cross
connected to the MAN Core routers or being terminated locally.
802.1
Core VN2
Figure 6 – MAN Service Architecture
The Service Architecture allows VNPT to deliver various classes of service over a distributed service edge
architecture. The following lists some of the key capabilities that this architecture will deliver :
• Converged MAN to deliver both residential and business service offerings
• Mux Uni enable flexibility of Layer 2 and Layer 3 services deployment
• IP/MPLS Metro transport architecture
• MSAN Voice
• Intra MAN layer 2 and layer 3 Services
• Inter MAN layer 2 and layer 3 Services
• Voice, Video and Data for broadband 3 Play services
2.7.1. MSAN Voice Service Architecture

The design consideration for MSAN voice service including,
• Scalable for the MAN to be able to aggregate all the current and future MSAN
• Efficient voice payload routing capability, local voice call should be able to route directly with the
province city
• Ability to provide good QoS scheme for the Voice Services
• No single point of failure for the province voice
• Ability to support current VoIP over IPv4 and support migration to IPv6 in the future

2.7.1.1. IP VPN Transport Option for MSAN Voice Service

2 Level of IP VPN hierarchy is required in this option. A national Voice over IP VPN is setup at the
VN2 network to provide transport for inter-province voice calls and a second level of IP VPN is setup
at the province MAN level to provide transport for the intra-province voice calls. The 2 level of IP
VPN is connected at the MAN Core routers and VN2 PE routers via IEEE 802.1q VLANs. OSPF will
be used for routes exchange between the VN2 IP VPN and the province level IP VPN.
The MSANs will be aggregated to the MAN Access router using IEEE 802.1q VLAN and all the
MSAN Voice VLAN will be terminated into the Province level Voice VPN within the same Access
Router. The IP Address of the vrf in the Access Router will be the default gateway for the MSAN
Voice services.
Voice calls within the province can be routed directly within the province MAN and toll calls will be
routed to VN2 Voice VPN via the MAN Core routers. Service Resiliency is achieved by creating
redundant links between MAN Core Routers and VN2 PE routers, there is always a redundant link
and node available should the primary routers or links fail.
MSAN Voice via IP VPN
VN2 MSAN Voice VPN
VN2 PE VN2 PE
1q 1q
DLSAM DLSAM
Access Access
POTS 1q 1q POTS
VRF VRF VRF VRF
Core Core
VRF
VRF
Access Core Access
Figure 7 – NGN Voice Service Architecture Option 1
The advantages of using 2 Levels of IP VPN for MSAN Voice Transport includes,
1. Does not consume and MAC resources of the MAN, architecture is scaleable to address the
current and future VoIP requirements.
2. Local Voice Calls can be routed directly at the Access Ring without going through the Core Ring
and VN2.
3. Consistent Voice Architecture across VN2 and ME61
4. Secure, Layer 3 VPN does not subject to MAC layer attack and broadcast storm
5. Every Access Routers is able to support Layer 3 VPN service, no single point of failure

6. One time setup of Voice IP VPN, no reconfiguration or maintenance is required
2.7.1.2. E-LAN Transport Option for MSAN Voice Service

In this transport option, a Voice IP VPN is required at VN2 and a E-LAN is required at each of the
province’s MAN. The Voice IP VPN at the VN2 national level will carry voice calls between
provinces and the E-LAN at the province MAN will transport local voice calls. The national Voice
VPN and the province E-LAN is interconnected at the province Core Router and the VN2 PE using
IEEE 802.1q VLAN.
The MSANs will be aggregated to the MAN Access router using IEEE 802.1q VLAN and the IP
address of the IP VPN at VN2 PEs will be the default gateway for the MSAN Voice Services. Service
resiliency can be done by creating redundant links between the Core Routers and VN2 PE routers and
a redundant default gateways for the VoIP Services at the VN2 PEs. A pair of PEs at VN2 will be
configured for vrf and setup redundant gateways using VRRP protocol. All the MSAN in the province
will be setup to point their default gateway to this virtual gateway IP address. If the primary PE fails
or link to the primary VN2 PE is broken, the backup PE will take over the routing of voice calls
between VN2 and MAN.
Residential MSAN Voice via E-LAN
VN2 MSAN Voice VPN
VN2 PE VN2 PE
1q 1q
DLSAM DLSAM
Access Access
POTS 1q PW PW PW 1q POTS
VFI VFI
Core Core
PW PW
VFI
Access Core Access
Figure 8 – NGN Voice Service Architecture Option 2
2.7.2. Residential Service Logical Architecture

As VNPT embarked on its NGN roadmap, its ability to provide triple play services will have the greatest
impact on its residential users. As illustrated under the section, Service Architecture, there many types of
services that VNPT’s residential get to enjoy. As such, the logical architecture for the access network for its
residential has to be clearly defined, as illustrated in the following section :
2.7.2.1. Residential (& Business) High Speed Internet Access

Cisco Systems proposed to leverage VNPT MAN network to offer new revenue generating services to
both business and residential customers. VNPT could also consider migrating the existing ATM-based
DSL services to the MAN infrastructure. The goal is to build one network (based on Ethernet and IP

infrastructure) that can offer traditional and new services for business and residential customers. This
will reduce CAPEX and make it easy for VNPT to manage a single network instead of managing two
networks.
VNPT new MAN will be used as transport to carry the PPPoE backhaul traffic from the IP DLAM to
terminates at the BRAS. The network architecture for VNPT is a hybrid of centralize and distributed
architecture. BRAS is distributed to many parts of the VNPT network while it is centralize in each
province. Cisco is proposing to use the BRAS as the High Speed Internet Access Concentrator only
but TV Broadcast, Voice over IP and Video-on-Demand traffic will be transported using the IP and
MAN directly towards the subscribers.
A Typical Network Topology as following,
Residential HSI
BRAS VN2 Internet VPN
QinQ
DLSAM DLSAM
Access Access
VC QinQ PW PW QinQ
Core Core
DLSAM
PW QinQ
Access Core Access
Figure 9 – Residential High Speed Internet Architecture
As shown in the figure above, in each remote CO a MAN Access router aggregates the GE DSLAM
traffic from both residential and business customers. The business customers’ CPE can be directly
connected to the MAN Access router via Ethernet or can be DSL attached. All traffic is backhauled
over a ring or star topology to an MAN Core router in a main CO where the traffic is handled
differently based on the subscriber service. The IP DLAM will be configured to map each user DSL
VC to a unique 802.1qinq VLAN on its uplink on a 1:1 basis.
The 1:1 mapping of subscriber to VLAN PPPoEoQinQ Model is currently being used in Hanoi PT. It
could be utilized for both Business and Residential services. This is a “VLAN Tag Stacking” model
where a two VLAN tags are used per subscriber. The outer and inner VLAN tags are used to uniquely
identify a subscriber line.

Port 0
VLAN 100
Ethernet VLAN 101 PPPoX or IP
VLAN 102
VLAN 103
Port 3
DSLAM 100
Figure 10 – Single outer tag with unique inner tag per subscriber
Figure above shows a unique inner VLAN Tag per subscriber and the outer VLAN Tag represent the
DSLAM or U-PE. This is similar to current ATM deployment where the outer tag is equivalent to a
VP (per DSLAM) and inner Tag is equivalent to a VC. This model is being proposed for both
Business and Residential services.
At province MAN, the Access router will be configured to map all the users HSI VLANs on the
DSLAM to a single EoMPLS EVC and cross connect the EVC to the MAN Core router. The MAN
Core router will be mapping the EVC to a 802.1qinq double taq VLAN to the BRAS. This require the
MAN Core router to be able to push additional VLAN taq on to the egress traffic to the BRAS. The
outer taq of the VLAN identifies the DSLAM and the inner taq of the VLAN identifies the user within
that DSLAM. In this configuration, no MAC learning is required on the Metro Ethernet and no Global
VLAN resources is consumed for the EVC. Only one EVC is required for HIS service per DSLAM.
2.7.2.2. Residential IPTV

With the popularity of IPTV as a service, special mention need to be made with regards to the
technical requirement of delivering good quality IPTV service. The IP Multicast technology is used to
deliver broadcast services over an IP infrastructure. At a very high level, the multicast packets are IP
routed end-to-end from the multicast source (Video Head End) to the MAN Access Routers. Hence
the standard and proven IP and Multicast routing technology delivers the IP Multicast traffic to each
MAN Access aggregation router aggregating IP DLSAMs. This section details the functional
components involved in delivering broadcast IPTV services.
From a functional overview perspective, the IP Multicast routing is run from the video head-end to
MAN Access routers. ALL multicast streams (broadcast TV channels) are injected at the PE located
nearest to the Super headend, which in most cases, located at the VN2 network, and IP routed to the
MAN Access routers by leveraging the benefits of the Layer 3 routing network all the way to the edge.
To improve the channel change latency quality of experience (QoE), every MAN Access router is
configured to statically join all the multicast groups supported in the network. IGMP snooping is
required at the IP DSLAM to prevent unnecessary flooding on the customer facing links. Overall,
statically delivering all multicast streams to the Layer 3 edge devices improves channel change
latency experienced by the end user when a large number of multicast streams are present in the
aggregation network. The MAN Access routers functioning as the IGMP querier intercepts all the
IGMP join requests from the clients and pushes the requested multicast streams downstream towards
clients to the IP DSLAMs. With the IGMP snooping capability on the Layer 3 edge device and the IP
DSLAM ensures that no multicast stream is duplicated and flooded unnecessarily. Delivering
multicast streams over routed IP all the way to the edge allows us to leverage enhancements in the fast
IGP, known as incremental SPF (Shortest Path Forwarding) algorithm. The fast IGP allows for
expedient service restoration of the broadcast services. The resiliency section details the fast IGP
features that can help improve scale, resiliency, and latency performance of broadcast services.
PIM is an IP independent routing protocol used for routing multicast groups. PIM leverages the
unicast routing protocols (IS-IS, OSPF, etc.) to populate the unicast routing information base (RIB)

and to perform the multicast forwarding functions. PIM uses the RIB to perform the Reverse Path
Forwarding (RPF) check function instead of building an independent multicast routing table. Based
on the RIB, PIM builds the multicast route table (mroute) with the multicast source and group
(destination) information. To deliver multicast stream over IP infrastructure, the service providers
typically employ IGMP (as discussed earlier) for user-to-network signaling mechanism and the PIM
Sparse Mode (SM) routing protocol for distribution of multicast traffic.
PIM SM is a valid technique for a source-unaware multicast implementation where the receivers are
unaware of the existence and location of active sources, and hence must relay in a Rendezvous Point
(RP) mechanism. This however is not applicable in the VNPT deployments where VNPT can either
own the content (retail services) or are intimately aware of the placements of sources within a VPN
(wholesale services).
In the case where VNPT own the content, the multicast sources are well known and are limited to a
set of devices that reside in a Broadcast TV Headend, which mean that the PIM-SM implementation
becomes an inefficient mechanism for delivering multicast streams. Furthermore, with PIM-SM any
device is capable of becoming a source, and can start streaming multicast content to group(s), thus
opening security issues resulting from malicious attacks or simple miss-configurations.
To counter these gaps, Source Specific Multicast (SSM) is chosen over SM. The SSM feature is an
extension of IP multicast where datagram traffic is forwarded to receivers from only those multicast
sources to which the receivers have explicitly joined. For multicast groups configured for the SSM,
only source-specific multicast distribution trees (no shared trees) are created. In contrary to the PIM-
SM implementation where each multicast speaking PE must maintain knowledge about which hosts in
the network that are actively sending multicast traffic, with SSM, this information is provided by
receivers through the explicit source address(es) relayed to the last hop routers by IGMPv3 join
messages. From the control plane perspective, with the SSM implementation only (S,G) states are
created on the multicast speaking routers versus in addition to (*,G) state. Hence, every broadcast TV
channel that is statically joined on the PE routers will need to create and maintain (S,G) state per each
channel.
As noted in preceding sections, the user-to-network (CPE-PE) signaling can be accomplished by
either IGMPv2 or IGMPv3 messages. We also noted that not all STBs are capable of supporting
IGMPv3 messages in which case the PE will be responsible for making necessary source mapping for
incoming IGMPv2 messages that do not include source(s) in the IGMP join requests. The source
mapping can be accomplished using the SSM-map feature in the Cisco IOS which allows a device to
automatically determine the source of a given group when the received IGMP messages are in the v2
format.
The SSM-map feature will enable a PE-aggregation node to map the broadcast-TV channel multicast
groups to the video server (multicast source) IP address. The SSM-map will translate the IGMPv2
membership reports into (S,G) PIM join messages which will be sent directly via the source tree
towards the source of the multicast stream.
When the MAN Access routers is configured as the Layer 3 edge for broadcast video, the distribution
network can take advantage of “anycast” support for either load balancing of video encoders or fast
fail over of video encoders. IP multicast technology natively supports the ability for “anycasting” of
IP multicast sources. With anycasting, one can configure 2 or more multicast sources that are sending
to the same IP multicast group (same multicast destination address) and have the same IP source
address. When used with PIM sparse mode, IP multicast technology uses a reverse path lookup to
determine which IP source is closest to any particular PIM edge node. The result is that the replication
path for a single multicast group can consist of a separate multicast tree for each broadcast encoder.
Using anycasting, the network can be configured to load share between multiple broadcast encoders.
The following illustrates the use of anycasting for load sharing between multiple video encoders.

Figure 11 - Redundant Multicast sources design
2.7.2.3. Video on Demand—IP Unicast Routing

The delivery of Video on Demand (VOD) services in our architecture is constructed over the
MPLS/IP aggregation architecture. The VOD service is a IP unicast application whereby residential
subscribers requests the service on as needed demand basis. In this model, the VOD service is
delivered to the residential subscribers via IP over the MPLS/IP transport aggregation architecture.
The BRAS functionality for the VOD service is distributed whereby the subscribers requesting for the
VOD service do not need to be aggregated and controlled through the BRAS, unlike the centralized
Internet Access service.
To better understand end-to-end delivery of the Video On Demand service, the section breaks down
the critical functional components involved.
• Residential Network Gateway (Various Models Considered)
• VNPT Access Network
• VNPT Aggregation Network
• ARP scaling design consideration
• Service Delivery Guarantee (QoS)
The Residential Network Gateway is essentially a CPE that connects the residential network and
associated IP appliances (PC, IP Phone, STB, etc.) to the access network. The VOD services are
offered using an STB appliance located in the subscriber home network. The VOD service can be
controlled and authenticated by the ASP offering the VOD service by use of video middleware or
simply by the VOD server. The service authentication and/or authorization are performed at the
application level (STB and video server/middleware). There are basically two flavors of CPE
connectivity models namely bridged CPE and routed CPE connectivity models. With the bridged CPE
model the DHCP server functionality is offered through the BRAS functioning as the DHCP relay
agent whereas in the routed CPE model the DHCP functionality is localized to the home network.
The VNPT MAN Network enables and aggregates support for various last mile technologies. We
explore both the DSL and Ethernet aggregation as the last mile technologies.
In our solution, the distributed VOD service offering is delivered through the DSLAM. The DSLAM
can bridge (802.1q) the VOD service for multiple subscribers over a shared video service VLAN and
also implements the split horizon like feature to isolate subscribers over a shared video VLAN. The
residential VOD service is a N:1 offering whereby multiple subscribers requesting a VOD service are

supported over a particular DSLAM node are aggregated and bridged over a shared video service
VLAN. The subscriber VOD session line identification is supported on the DSLAM through the
DHCP relay functions supported through the IP DHCP option 82.
The VNPT MAN Network provides a flexible a transport architecture that supports both the native
Ethernet as well as the MPLS/IP transport options. The aggregation transport network is composed of
the Access routers responsible of aggregating all the DSLAMs and the MAN Core routers are
responsible of aggregating all the Access routers. The distributed VOD service delivery is a point-to-
point data service and hence can be supported over MPLS/IP network. The shared video service
VLAN is IP routed between the Access routers and the video server.
The ARP Scaling Design Consideration in critical factor since the VOD service is IP routed, the
Access routers will be required to build and maintain many ARP entries in the ARP table. This can be
a limiting factor in the Access routers.
The following example illustrates the ARP scaling design consideration:
Total ARP Entries = M * W * Y where

• M = number of DSLAMs attached to the PE
• W = number of subscribers per DSLAM
• Y = number of Layer 3 terminated services supported per PE
• Layer 3 Terminated Services Include: VoIP, VoD/B-TV
For example, the router aggregates 40 DSLAMs each supporting 300 subscribers. Each active
subscriber supports 2 services (VoIP & VOD) that are Layer 3 terminated at the PE. The number of
ARP entries required in the ARP table in this case is (40*3000*2) = 24K.
The ARP scaling issues are not of concern at the MAN Core routers since it does not directly
terminate any Layer 3 services or connect hosts.
The Service Delivery Guarantee (QoS) is responsible for ensuring the differentiated services is
provided for various traffic classes and policed accordingly across the end-to-end Broadband network.
The QoS section details various traffic profile and characteristics associated with different classes of
service and various hardware platforms.
Our solution is architected to support the multiple PVC model. In other words, each residential service
offering gets its own PVC connecting to the DSLAM. Hence the video (B-TV & VOD) service
offering will have its own PVC. The VOD service offering is mapped to the shared video service
VLAN on the DSLAM as described above and than the 802.1p markings are applied for each service
VLAN for upstream queuing. In the downstream direction, the DLSAM will send the VOD traffic
based on the ATM queuing structure from the DSLAM.
2.7.3. Business Service Logical Architecture

The business service logical architecture is different from that of the residential, as its is targeted at a totally
different group of users. The following are some of the services that can be supported :
2.7.3.1. E-Line
P2P Layer 2 VPN Business services are possible with the use of Ethernet over MPLS (EoMPLS).
EoMPLS is one of Cisco’s Any Transport Over MPLS (AToM) transport types. AToM transports
Layer 2 packets over a MPLS backbone using a directed LDP session between edge routers for setting
up and maintaining connections. Forwarding occurs through the use of two level labels that provide
switching between the edge routers. The outer MPLS label (tunnel label) routes the packet over the

MPLS backbone from ingress to the egress PE. The inner MPLS label (VC label) is a demuxing label
that determines the connection at the tunnel endpoint (the particular egress interface on the egress PE
as well as the VLAN identifier for an Ethernet frame).
EoMPLS works by encapsulating Ethernet PDUs in MPLS packets and forwarding them across the
MPLS network. Each PDU is transported as a single packet. The virtual connection created for the
transport of frames is referred as a Pseudo-wire (PW).
EoMPLS PWs must be configured on the MAN Access routers that connect to the customer
equipment. According to the type of transport desired, EoMPLS can carry Ethernet frames associated
with a particular VLAN ID (VLAN-based EoMPLS) or Ethernet frames regardless of the VLAN ID
(Port-based EoMPLS).
Business E-Line (Intra Province)
Access Access
Core Core
CE Access Core Access CE
Figure 12 - Intra-Province Business E-Line Architecture
An EoMPLS pseudo-wire will be created directly across the MAN MPLS network between 2 E-Line
Access routers for Intra-province E-Line service.

Business E-Line (Inter Province)
VN2 Core
PW
VN2 PE PW
.1q
Access Access
PW
Core Core
PW
Access Core Access CE
Figure 13 - Inter-Province Business E-Line Architecture
As for Inter-province E-Line service, an EoMPLS pseudo-wire is created from the MAN Access
router and terminated at the MAN Core router. The pseudo-wire is then connected to VN2 PE router
via the IEEE 802.1q GE connection. A second EoMPLS pseudo-wire is required at the VN2 network
to interconnect the E-Line service to another province.
2.7.3.2. E-LAN
VPLS is the technology enabler for multipoint Layer 2 VPN services over an MPLS network. H-
VPLS adds to the scaling of VPLS by allowing a multi-tier architecture that lowers the capabilities
and scale requirements at the smaller access nodes.

Business E-LAN (Intra Province)
Access Access
VFI VFI
VFI
Core Core
VFI
VFI
CE Access Core Access CE
Figure 14 - Intra-Province Business E-LAN Architecture
Business E-LAN (Inter Province)
VN2 E-LAN
VN2 PE
1q
Access Access
PW PW PW
VFI VFI
Core Core
PW PW
PW PW
VFI
CE
Access Core Access
CE
Figure 15 - Inter-Province Business E-LAN Architecture
2.7.3.3. Layer 3 VPN Services

Layer 3 VPN services can be offered to both Business DSL or Ethernet customers. In the first case,
they attach to the VNPT network through the DSLAM and in the second they connect directly or

indirectly via Ethernet technology to the aggregation or distribution nodes (MAN Access router and
MAN Core router).
The following are two (2) implementation models for Layer 3 VPN services:
• Centralized Layer 3 VPN PE—In this case, the Layer 3 VPN PE functions are handled outside the
MAN Access router. The PE in the MAN Access router does not act as a Layer 3 VPN PE, but
rather an already existent device takes care of the function. For this environment, the MAN
Access router rely on EoMPLS PWs to backhaul the traffic from the business customer to the
Layer 3 VPN PE.
• Distributed Layer 3 VPN PE—In this case, the Layer 3 VPN PE functions are placed in the
devices that built the Ethernet aggregation network. So here either the MAN Access router or the
MAN Core routers are also Layer 3 VPN capable
2.7.3.4. Circuit Emulation Services

The Circuit Emulation Service(CES) Capability of the proposed solution provides a new service
offering opportunity for VNPT. CES provides bit-transparent data transport that is completely
protocol independent. This allows VNPT to leverage on the MAN IP/Multiprotocol Label Switching
(MPLS) network to provide leased-line emulation services or to carry data streams or protocols that
do not meet the format requirements of other multiservice platform interfaces. In addition, CES can be
used for low-speed ATM services, including IMA, E1/T1, E3/T3 and STM-1 interfaces.
Some primary applications of the CES (Figure xx) include:
• Transporting 2G and 3G network traffic over packet networks. VNPT could be implementing
High-Speed Data Networks (HSxPA) to support new revenue-generating services. The CES
Service can be positioned for multigenerational migration of mobile networks (2G and 3G),
simultaneously carrying TDM and ATM traffic over IP/MPLS MAN networks. This
technology provides a mechanism to enable IP/MPLS to the cell site, which can eventually be
in place to transport the mobile traffic over IP from end to end.
• T1/E1 circuit emulation for leased-line replacement.
• PBX to PBX connectivity over PSN.
• High-density SS7 backhaul over IP/MPLS.
• Inter-MSC connectivity.
• Pre-encrypted data for government, defense, or other high-security applications.
• Proprietary synchronous or asynchronous data protocols used in transportation, utilities, and
other industries.
• Leased-line emulation service offerings in MAN
For circuit emulation services, In order to achieve bit-transparent circuit emulation without bit errors,
it is imperative that both endpoints of the circuit use the same bit clock frequency. The network
should be synchronized end to end for proper operation. Three options are available for achieving
proper clocking and synchronization of network when deploying circuit emulation services over
packet network. They are:
Synchronous mode: In this option, a GPS or BITS clock source is available to be fed into the edge router to
clock the packets for transmission. The clock is received from a line interface and is used by the router to
transmit the TDM frames, received from the packet network to the final destination.

Differential clocking mode: Often a GPS or a BITS clock source may not be available for service providers at
every possible site, such as a remote cell site. However, they may have a common clock source that is fed into
all the elements of the network. In this scenario, the system will use the common clock source as well as
observe the timestamps received from the CEoP PWE packets received from the packet network and calculate
the differential to recover an accurate clock. This recovered clock reference is then used to transmit the TDM
frames.
Adaptive clocking mode: In some deployments, there is no common clock or a GPS/BITS source available at
the remote site. The edge router has to completely rely on the incoming packet stream from the IP/MPLS
network to calculate the clocking reference. The clock accuracy, thus derived, should be of very high quality,
compliant to the 3GPP mobile standards (accuracy of 15ppb or higher). This is called the adaptive clock
recovery mode. The central office will be using a primary clock source reference, and the receiving site will
derive the clock based on the incoming CEoP PWE packets.

3. Resiliency Strategy
This section takes a look at the topic of resiliency for VNPT MAN. Before we begin our discussion about
resiliency, it is important to understand the important concepts of reliability and availability.
3.1. Service Level Resiliency

It is important to note that in the discussion of network resiliency, it is the service that VNPT’s customers care
about. And the service, be it IPTV or high speed Internet access, requires more than just having a resilient
network infrastructure. For example, the IPTV requires the servers housing the middleware to be running, and
the high speed internet access need the DHCP server to be able to give out IP address.
Therefore, it is important that VNPT approach its resiliency strategy on a per-service basis, so as to ensure all
grounds are covered. The following shows just one example of what can be done to improve the availability of
the video source in an IPTV implementation :
Figure 16 - Video High Availability
3.2. Network Level Resiliency

Network level resiliency relies mostly on the routing protocol to route traffic away from a failure domain.
There are several features that improve the convergence times of the underlying Interior Gateway Protocol
(IGP). The following lists some of the fast convergence features available today:
• IGP (OSPF/ISIS) Fast Hellos
• IP Event Dampening
• IGP Exponential Backoff
• Incremental SPF
The sub-second IGP convergence will be critical in order to deliver time sensitive multicast applications to the
end user. The multicast packets for the proposed broadband models are IP routed end-to-end. Hence, the
link/node/path failure will depend on the fast IGP for sub-second convergence of service restoration.

3.3. Device Level Resiliency

While network level resiliency deals with diverting traffic away from a failure domain, device level resiliency,
in the most stringent case, deals with continual forwarding of traffic even in the light of failure.
Figure 17 - Network and Device Resiliency
Device level resiliency can be addressed in two components, namely the hardware and software component.
For hardware component, we look at the redundany level of the following parts :
• Route processor, or sometime referred to as supervisor module
• Cooling Fan
• Power supply
• Switch fabric, if any
While the redundant implementation of redundant cooling fan and power supplies is more specific to
hardware component only, redundancy of route processor or supervisors require the corresponding software
features to be available.
Route processor redundancy implementation may be implemented in various stages, namely :
1. High System Availability (HSA)

In this mode, a secondary route processor (RP) is introduced but not booted. It just sit there waiting for the
primary RP to fail. When the primary RP fails, the entire router is booted via the secondary RP. This mode is
also call Cold-Standby because the secondary RP is not booted. It may take a few minutes for the secondary
RP to be completely booted and take over a fail primary RP
2. Route Processor Redundancy (RPR)

RPR is an improvement over HSA. In this mode, the secondary RP is pre-booted. When the primary RP fails,
the secondary RP takes over by resetting only the line cards. This is also called Warm-Standby. Since the
secondary RP need not go through the booting process, RPR allows the router to be functional in a shorter
time.

3. Route Processor Redundancy Plus (RPR+)

RPR+ is a further improvement over RPR. In RPR+ mode, the secondary RP is fully initialized. The primary
RP dynamically synchronizes startup and the running configuration changes to secondary RP.Additionally,
the line cards are not reset in RPR+ mode. This functionality provides a much faster switchover between the
processors. Information synchronized to the standby RP includes running configuration information, startup
information, and changes to the chassis state such as online insertion and removal (OIR) of hardware. Line
card, protocol, and application state information however, is not synchronized to the secondary RP.
4. Stateful Switch Over (SSO)

SSO mode provides all on the functionality of RPR+ and supports synchronization of line card, protocol, and
application state information between RPs for supported features and protocols (a “hot standby”). Therefore
RP redundancy is of highest level when it is configured in SSO mode.
5. Non-Stop Forwarding (NSF)

The Non Stop Forwarding (NSF) feature works with the SSO feature in the Cisco IOS software. While SSO
solves an internal problem (RP failure), NSF prevents some external event from occurring, which may be
harmful to the network.
Usually, when a networking device restarts, all routing peers of that device detect that the device went down
and then came back up. This transition results in what is called a routing flap, which could spread across
multiple routing domains. Although the device may be forwarding traffic, routing flaps caused by the
switchover create routing instabilities, which are detrimental to the overall network performance. NSF helps to
suppress routing flaps in SSO-enabled devices, thus reducing network instability.
NSF allows for the forwarding of data packets to continue along known routes while the routing protocol
information is being restored following a switchover. With NSF, routing peer do not experience routing flaps.
Data traffic is forwarded through the line cards while the standby RP assumes control from the failed active
RP during a switchover. The ability of line cards to remain up through a switchover and to be kept current
with the Forwarding Information Base (FIB) on the active RP is key to NSF operation.
For NSF to function, protocols and applications must be high-availability (HA)-aware. A feature or protocol is
HA aware if it maintains, either partially or completely, undisturbed operation through an RP switchover. For
some HA aware protocols and applications, state information is synchronized from the active to the standby
processor. For NSF to work, enhancements to the routing protocols like Cisco Express Forwarding (CEF),
OSPF, IS-IS and BGP have been made.
With the enhancements, these protocols have been made NSF-capable and aware, which means that routers
running these protocols can detect a switchover and take the necessary actions to continue forwarding network
traffic and to recover route information from the peer devices. For example, the IS-IS protocol can be
configured to use state information that has been synchronized between the active and the standby RP to
recover route information following a switchover instead of information received from peer devices.
A device is said to be NSF-capable if it has a redundant RP and has been configured to support NSF;
therefore, it would rebuild routing information from NSF-aware or NSF-capable neighbors. A networking
device is said to be NSF-aware if it is running NSF-compatible software but it does not have redundant RP in
its hardware architecture.
3.4. In Service Software Upgrade

The ISSU process allows VNPT to perform a Cisco IOS software upgrade or downgrade while the system
continues to forward packets. Cisco IOS ISSU takes advantage of the Cisco IOS high availability
infrastructure—Cisco NSF with SSO and hardware redundancy—and eliminates downtime associated with
software upgrades or version changes by allowing changes while the system remains in service. Cisco IOS
software high availability features combine to lower the impact that planned maintenance activities have on
network service availability, with the results of less downtime and better access to critical systems.

3.5. Link Level Resiliency

Link protection is based on capabilities of the individual hardware platforms. Link protection mechanisms are
those network and platform characteristics that enable link redundancy and provide for the rapid re-route of
traffic in the event of a failed link. This includes physical hardware redundancy and software protocol
capabilities.
The commonly used mechanisms in IP-environments dictates that an IGP extended for Fast Convergence
together with convergence enhancements for BGP will provide the overall protection at restoration function.
Traditional MPLS Protection and Restoration mechanisms, such as Traffic Engineering Fast ReRoute (TE
FRR), provides for excellent abilities to circumvent node or link failures, although scale and design of TE as
well as interoperability with other control plane mechanisms, such as BGP, can be challenging.
The following steps define the functional aspects of restoration that are used to provide for link protection:
Link failure detection—Various mechanisms are in place to provide a fast detection of link failure, both
generic and media dependent. The fastest mechanism by far is the integrated OAM mechanism of
SONET/SDH framing. Since a frame is sent every 125 µs, detection and propagation of link problems are
extremely quick. Only media types that use this type of framing is can provide this type of rapid detection.
Other mechanisms include Loss of optical Signal (LOS), PPP keepalives and various LMI mechanisms.
Bidirectional Forwarding Detection (BFD) is a generic lightweight hello-based mechanism that will be used in
conjunction with GE/10GE media. Although faster than most other mechanisms, it still does not have the
performance of SONET/SDH. The detection delay of SONET/SDH is generally in the order of 100s of
microseconds, while BFD typically operates in the range of 50-200 milliseconds (depending on
implementation and scale).
Failure propagation—Depending on the Protection and Restoration mechanism being used, there may not be
an associated propagation delay before the backup for a failed facility is installed. This is the case with TE
FRR. In our design IGP is used and therefore, the updated network information has to be flooded throughout
the network, with an associated delay of 1-2 ms per hop in a network tuned for fast convergence.
Updated network view—Upon receiving updated topology information, an IGP will compute a new network
view through the means of performing an SPF operation. The time consumed by this operation is a direct
consequence of the size of the network topology. Once that operation is completed, updated routing
information will be installed in the RIB. In an MPLS network protected by TE FRR, this operation still takes
place, but the service restoration is not dependent on its completion. In our design, incremental OSPF can be
applied to optimize SPF operation. Optionally, if IS-IS were the IGP, incremental IS-IS would similarly be
applied to SPF optimize operation.
Updating the forwarding plane—After the RIB has been updated the associated FIB also has to be updated so
that the forwarding plane can make use of the updated information. This operation will occur in a single
location on a centralized platform and on a per linecard basis in a distributed platform. The platform
architecture in conjunction with the size of the update (number of forwarding entries that need updating)
determines the how much time is spent on this operation.
Therefore, the total restoration time then is computed as follows:
Service Restoration Time =
Failure Detection Delay + Failure Propagation Delay + Update Network Topology Delay + Update
Forwarding Plane Delay
3.6. The truth about 50ms resiliency

Another concept that plagues the networking community is the requirement for a resilient network to mean
having the ability to reroute within 50ms limit. A little history lesson will serve to shed light on this subject.
In the 1980s, the ITU-T introduced the G.841 document which specifies the SDH network protection
standard. The 50ms protection time was introduced so that a network fault can be corrected within that short
duration. The reason for introducing the 50ms protection time has to do with the standard of the voice channel

banks technology. In the early days, the voice channel banks would terminate all calls that are being carried
over a trunk if a failure lasts more than 200ms. Taking other activities like fault detection into consideration,
the 50ms was adopted and has since been the ‘standard’. With newer technology being introduced, for
example the new generation digital phone, the tolerance for failure has been increased to even 2 seconds.
There is no longer the need for the 50ms reroute capability. However, the original requirement of 50ms still
stays in the document.
In the data world, things work in a different way. If one looks at the way a TCP application behaves, it is
elastic in nature in a sense that it could tolerate high failure duration and can recover by itself. In other words,
a network that recovers in 50ms and another one that recovers in a few seconds would probably be the same to
a TCP application. While new applications like VoIP may require a more stringent resiliency, due to the
nature of human conversation, the effect of a 50ms failure recovery and one that takes 1 second may still be
the same - users at both end of a VoIP call may not notice it at all.
One thing to note is that to build a network with 50ms recovery capability, a lot more resources have to be
spent. These include money and human engineering resources. It can be very expensive and the difference in
cost can easily be 50% or even double. The design philosophy behind such a network is very different from
the rest of the network as well. Network manager should therefore, understand the implication of requesting
for a network with 50ms recovery. They should only do so if they absolutely understand the traffic nature and
fully understand the cost involve. Failing to do so may result in costly expenditure or in the worst case
scenario, network inefficiency. Depending on the network design, some may end up with 50% wastage of
bandwidth just to achieve a protection capability that applications may not need.
There are of course, cases where 50ms recovery seems a goal worth shooting for. For example, if you are
running a network with strict SLA to upkeep. And considering the number of end users that are going to be
affected by your network and the potential penalty, you have no other choice. Another example is when the
link speed of your network gets faster and faster, like the case of an OC-192 or OC-768, one may truly
requires such resiliency.
1. A Note On IPTV Service Resiliency

Due to the common notion of associating resilient network with 50ms reroute capability, this gets into the
technical requirement of building an IPTV service. However, deeper inspection into the IPTV world reveals
that the problem is of a completely different nature.
In the discussion of building resilient IPTV service, one ought to understand the nature of the MPEG
technology. As it goes, the quality experience by the users is dependant on a couple of factors. Among them is

how the so-called Group-Of-Picture (GOP) make up a block of information in the video stream. Within a
GOP, there is an I-frame, which act as a reference frame, and it is followed by an intersparse of multiple B and
P-frames. The B and P-frames cannot be viewed without the presence of an I-frame. Therefore, the effect of
loosing a packet that contains a B-frame and that of lossing a packet that contains an I-frame is very different.
In a worse case scenario, loosing an I-frame is equivalent to loosing a string of 36 frames, which lasts up to
1200msec. In a situation like this, even a 50ms reroute capability does not buy any protection. The impact of
lossing the frames is illustrated in the following table :
Figure 18 - Service level Resiliency, the Myth of 50 ms recovery
The effects of loosing a frame in the IPTV service demonstrates that a plain 50ms reroute capability may not
be a one-size-fits-all solution for an NGN. As mention, different services will react differently to different
duration of outages and recovery timing, as illustrated in the following table :
Figure 19 - Service Level Resiliency strategies
Though various implementation of IPTV services throughout the world, a lot of knowledge has been learned
with respect to ensuring good Quality of Experience (QoE) for IPTV service. In fact, there are many aspects

within the network architecture that needs to be addressed, rather than simply adopting strategy like a layer 2
infrastructure with 50ms reroute capability. One good example is how the multicast network is architected for
the IPTV service has a tremendous impact on the capability of the network.
As shown in the following table, in the multicast architecture, a failure of the Designated Router (DR) takes
more than a couple of seconds to recover. This event is a layer 3 event and is totally independent of the layer 2
reroute capability. Another event that affect service is the time when the source fails. Dependant on the
redundant strategy, it may not be possible to recover a source failure to maintain good QoE.
Figure 20 - Impact of Network Failure
From the above understanding and through implementation experience, it has been proven that a layer 2
network, even with a 50ms reroute capability, does not guarantee a high QoE. In fact, there are many failure
scenarios that will render the service unacceptable.

Figure 21 – Intelligent Layer 3 Multicast
In the above diagram, a DR failure in a layer 2 network will render massive outages, since a secondary DR
needs to take over the forwarding of multicast traffic to the downstream. How fast a DR takes over is totally
independent of how fast the layer 2 network can reroute.
It has been proven through implementation that a layer 3 network can better react to similar failure due to the
resultant traffic flow. At the very least, the damage area could be controlled and will not wide spread, as in the
layer 2 network.
The IPTV failure scenarios points to the fact that resiliency in the NGN network may not be as simple as it
seems. It points to extensive failure scenario analysis and requires an architectural approach. The following
example is what needs to be done.
2. PIM Fast Hellos

The PIM enabled routers periodically exchange PIM hellos with other PIM enabled routers on the same
subnet to form PIM adjacencies. The PIM hellos act as a keepalive mechanism between PIM routers on the
same subnet. When more than one PIM routers are available on the same subnet, it provides for redundancy
for the multicast traffic. Hence the PIM hello mechanism enables VNPT to offer resilient multicast service
offerings. By default the PIM hello messages are exchanged between the PIM speaking routers every 30-
seconds. It takes three PIM hello misses for the PIM neighbor to be announced dead (3xPIM Query Interval)
and before another PIM router on the same subnet assumes responsibility.
To improve multicast stream switchover in case of PIM node or link failure, PIM hellos need to be tuned with
aggressive timers. However PIM depends on the unicast routing protocol (IGP) to converge first. Hence
aggressive timers for PIM should be in accordance with the fast-IGP convergence timers.
The PIM hello timers can be tuned down by using the ip pim query-interval. The recommendation is to tune
the PIM query interval down to 500 ms or lower.

3.7. Ensuring High Availability Of Physical Component

In the discussion of high availability of physical component within ME61, it is important to first get the
concept of reliability and availability right.
1. Reliability vs Availability
Reliability is the probability that a product can perform a required function for a given time interval. It is
generally used to describe the quality of a product through the following data provided by an equipment
vendor :
Mean Time Between Failure (MTBF) - The average time taken for a component to transit from an operation
state to a failure state
On the other hand, availability is the total amount of time a system is up, and is functioning properly to deliver
its mission. When one talks about ‘Five-Nines’, it is availability that we are interested in. But bear in mind
that reliability is also an important contributing factor. Those who prefer the classical approach would suggest
the following formula for availability:
MTBF
Availability = --------------------------------
(MTBF + MTTR)
where
MTBF = Mean Time Between Failure - The average time taken for a component to transit from an operation
state to a failure state
MTTR = Mean Time To Restore - The average time taken to reinstate a failed component to a functioning
state
It is then possible to achieve high availability with a minimal value of MTTR. In this case, it points to sparing
strategy for ME61, with backup components readily available in the event of a failure.
The classical approach focuses on calculating a theoretical availability of a system. And in doing so, we look
at how a system is constructed out of its components. These components are inevitably arranged in two
fashions : series or parallel. The overall availability of a system is the summation of the availability of these
components :

Series Vs Parallel Components
B1
A B A C
B2
Series Parallel
The overall availability of a system using components arrange in a series have a different availability from
those that are arranged in a parallel fashion. Ultimately, we will be able to arrive at a figure which indicates
the overall availability number.
Availability of a group of components arrange in a serial fashion is given by
n
SerialAvailability = ∏ ComponentAvailability
i =1
(i )
Where
i = component
n = total number of components
For example, if we have two individual components each with an availability of 0.99999 and 0.99994. And
we build a system with these two components by lining them up in a serial fashion, then the availability of the
system is as follows :
System availability = 0.99 * 0.999 = 0.98901
For a system made up of components arranged in a serial fashion, the resultant system availability is less than
any of the individual component.

Availability of a group of components arranged in a parallel fashion is given by
n
ParallelAvailability = 1 - ( ∏ (1 − ComponentA vailabilit y
i =1
))
(i )
For example, if we arrange the previous two components in a parallel fashion, the resultant system has an
availability of
System availability = 1 - ( (1-0.99) * (1-0.999) )

= 1 - ( 0.01 * 0.001 )
= 1 - 0.00001
= 0.99999
For a system made up of components arranged in a parallel fashion, the resultant system availability is more
than any of the individual component. It is also interesting to note that a ‘five-nines’ system can be
constructed out of less reliable components. In other words, one of the primary resiliency strategy applied for
ME61 network should be the deployment of dual-device design in certain part of the network. For example, in
major cities, there should be a pair of PE routers instead of one. Deploying the PE routers in a parallel fashion
greatly enhances the availability of these major cities.
Based on the above examples, the availability of the ME61 network can also be derived recursively after some
calculation. Another important activity that needs to be performed is failure analysis, so that impact to
customers can be determined in the event of a failure. Understanding these impacts is important as it will help
VNPT in determining its service offering to its customers.
It is important to note that the exact calculation has not been done as it would require information that is not
available as this time – reliability figures of the underlying system for example.
2. Device Level Resiliency
For device level resiliency, we look at the hardware architecture of the proposed products. We are interested
in the redundant components like power supply and route processors. We are also interested in how a
particular device behaves under certain physical conditions like heat and humidity. This is when certification
like NEBS comes in. The devices proposed for the ME61 network are NEBS certified and supports redundant
hardware configurations.
At the hardware level, the Cisco 7600 Routers are each built for very high reliability. They have fully passive
backplanes, redundant power supplies, and redundant control and switching. They also employ sophisticated
techniques to minimize or eliminate packet drops in the event of a switchover between primary and redundant
control systems. This is accomplished using stateful switchover (SSO), a feature in Cisco IOS Software. With
SSO, the control and switching state are continuously maintained between primary and secondary control
complexes. This enables in-service software upgrades and nonstop forwarding, since state does not need to be
relearned following a failover.

SSO protects from hardware or software faults on an active Route Switch Processor (RSP) by synchronizing
Layer 2 protocol and state information with a standby router processor. This ensures zero interruption of L2
connections in the event of a switchover. The SSO feature takes advantage of route processor redundancy by
establishing one of the SUP as the active processor while the other route processor is designated as the
standby processor, and then synchronizing critical state information between them. Following an initial
synchronization between the two processors, SSO dynamically maintains route processor state information
between them. A switchover from the active to the standby processor occurs when the active route processor
fails, is removed from the networking device, or is manually taken down for maintenance. Since the standby
route processor contains L2 protocol state information, it can communicate to its neighboring routers after it
takes control and becomes the active route processor. At this time, packet forwarding continues while route
convergence is completed on the newly active route processor. This continuous forwarding technique is
accomplished via the Non-Stop-Forwarding (NSF) feature.
NSF works with SSO to minimize the amount of time a network is unavailable to its users following a
switchover. NSF helps to suppress routing flaps, thus improving network stability within ME61. NSF allows
for the forwarding of data packets to continue along known routes while the routing protocol information is
being restored on the newly active route processor following a switchover via IETF graceful restart extensions
for the routing protocols.
In-Service Software Upgrade (ISSU) minimizes the impact of upgrading or downgrading Cisco IOS Software
images on Cisco 7600 Series Routers with redundant route processors. Software upgrades are accomplished
by loading the new release onto the standby supervisor, then performing a hot switchover from the old, active
supervisor. The line cards automatically undergo a warm reload to activate the new software, minimizing the
outage.
Benefits of ISSU feature within ME61 network include :
• Provides the ability to upgrade/downgrade a particular software feature with minimal system downtime
• Delivers a comprehensive upgrade solution covering maintenance-fixes as well as new features-rapid
deployment of new features/services within ME61
• Reduces planned downtime and operational expenses for VNPT
By relying on the high reliability features on the cisco 7600 routers, the proposed design for ME61 takes a
step further, by introducing dual-box strategy in critical part of the network.

As shown in the above diagram, there is always at least a pair of 7600 Core CES routers within the major
cities and there is also at least a pair of access CES routers forming a access ring. The design of the ME61
network is that these devices function as a parallel system. With its already high reliability figure, putting
these devices in a parallel fashion makes the ME61 core even more resilient to failure.
3. Link Level Resiliency
For link level redundancy, we look at how deploying multiple links between two network nodes within ME61
helps in improving availability. We look into areas like number of links required, and how they map to the
logical design. For example, one may choose to have multiple Ethernet links between two routers within the
ME61 core. If one chooses to implement link bundling technology, then these links appear as one logical
interface in the logical network. On the other hand, if these links are used individually, then there will be
multiple logical links in the logical design. One thing for sure is that, it may not be the case of the more the
merrier when it comes to link resiliency. For one, CAPEX may be prohibitive, especially the high speed
interfaces, or some protocols may impose limit on the number of physical links that it can support.
The proposed design for ME61 can incorporates the link level resiliency strategy for its MPLS network. One
important characteristic of this is that the network design demonstrates a symmetrical graph within the
network.

As illustrated in the above diagram, within the ME61 network can deploy in future multiple links to its
neighbor in the Core CES Rings. With this design, there is always more than one link between two nodes,
protecting the network from connection failure and also ensure sufficient ring bandwidth in even of links
failure.
Another important aspect of link resiliency is in the ability to detect link failure in a fast manner in the first
place. With Ethernet as a link technology, then additional features like Bi-Directional Forward Detection
(BFD) may be required to achieve fast detection of failure.
4. Site Level Resiliency
This is the highest echelon of the physical resiliency exercise. With device level and link level resiliency
addressed, the next thing to look into is whether there is a need for the entire site to be protected from disaster.
Another area that this idea is applicable is the data centre facility for ME61. A remote site may also be
required as part of a disaster recovery purpose for customer providing mission critical services.
While it is beyond the scope of this proposal to dwell in site level resiliency, it is our opinion that this topic be
further discussed in the near future.

5. Proposed Solution Availability
The Following is the MTBF and Availability of each node in the network.
MTBF
Product Description Quantity Availability
Hours
Core CES
CISCO7609-S Cisco 7609-S Chassis 176,382 1
7609S-RSP720C- Cisco 7609S Chassis,9-slot,Redundant System,2RSP720-
176,382
R 3C,2PS 1 99.998%
4000W-DC 4000W DC Power Supply (select cable) 331,945 2 99.999%
Cisco 7600 Route Switch Processor 720Gbps fabric, PFC3C,
RSP720-3C-GE 86,555
GE 1 99.995%
RSP720-3C-GE 86,555
GE 1 99.995%
WS-CAC-3000W Catalyst 6500 3000W AC power supply 331,945 2 99.999%
7600-ES20-GE3C 7600 ES20 Line Card, 20xGE SFP with DFC 3C 97,500 1 99.996%
SFP-GE-L 1000BASE-LX/LH SFP (DOM) 1,000,000 2 100.000%
SFP-GE-Z 1000BASE-ZX Gigabit Ethernet SFP (DOM) 1,000,000 12 100.000%
7600-ES20-10G3C 7600 ES20 Line Card, 2x10GE XFP with DFC 3C 94,000 5 99.996%
XFP-10GLR-
Multirate XFP module for 10GBASE-LR and OC192 SR-1 1,131,606
OC192SR 1 100.000%
XFP-10GER-
10GBASE-ER and OC192 IR2 XFP Module 1,131,606
OC192IR 7 100.000%
XFP-10GZR-
10GBASE-ZR and OC192 LR2 XFP Module 3,039,506
OC192LR 1 100.000%
TOTAL Node Availability 99.972%
Access CES
CISCO7604 Cisco 7604 Chassis 131,000 1 99.997%
Cisco 7604 Chassis,4-slot,Redundant System,2RSP720-
7604-RSP720C-R 131,000
3C,2PS 1 99.997%
2700W-DC 2700 W DC Power Supply for 7604 367,850 2 99.999%
RSP720-3C-GE 86,555
GE 1 99.995%
RSP720-3C-GE 86,555
GE 1 99.995%
7600-ES20-GE3C 7600 ES20 Line Card, 20xGE SFP with DFC 3C 97,500 1 99.996%
SFP-GE-L 1000BASE-LX/LH SFP (DOM) 1,000,000 15 100.000%
SFP-GE-Z 1000BASE-ZX Gigabit Ethernet SFP (DOM) 1,000,000 0 100.000%
WS-X6704-10GE Cat6500 4-port 10 Gigabit Ethernet Module (req. XENPAKs) 94,000 1 99.996%
XENPAK-10GB-
10GBASE-LR XENPAK Module with DOM support
LR+ 2
XENPAK-10GB-
10GBASE-ER XENPAK Module with DOM support
ER+ 0
XENPAK-10GB-
10GBASE-ZR XENPAK Module
ZR 0
WS-F6700-
Catalyst 6500 Dist Fwd Card, 256K Routes for WS-X67xx 330,980
DFC3B 1 99.999%
TOTAL Node Availability 99.989%
Per Node Availability in the network.

With the availability is with the assumption that the MTRS (Mean Time to Restore) is 4 hrs. As the
Availability design has at least 2 nodes in parallel in the ring and the access CES have connection to at these 2
nodes. The availability of the network increases.
Core CES node availability =1 - ( (1-99.972%) * (1-99.972%) )

= 99.999%
Access CES node availability =1 - ( (1-99.989%) * (1-99.989%) )

= 99.999%
Thus the availability of the nodes in the network deployed in the parallel design coupled with the HA features
in the system are able to provide a network availability of 99.999%

4. QoS Architecture
This section provides the framework that can be used to deliver differentiated services for VNPT’s MAN. It is
expected that as VNPT build up its MAN, some fine tuning of CoS parameters will be necessary over the
initial implementation of the network to gain the optimal settings.
QoS in IP networks gives devices the intelligence to preferentially handle traffic as dictated by network policy.
QoS is defined as those mechanisms that provide the ability to control the mix of bandwidth, delay, jitter, and
packet loss in the network. It has to be stressed that QoS is not a device feature, but it is an end-to-end system
architecture.
Figure 22 - QoS Strategy
4.1. Cos/Qos Mechanisms

The VNPT network should consider the following features to deliver class of service. These are :
DSCP – Identifying packet priorities
Committed Access rate – Rate Limiting and Classifying packets
Weighted Random Early Detection – Congestion Avoidance
Class Based Weighted fair Queuing – Congestion Management
These are discussed in the following sections.
1. Service Classes and DSCP

VNPT can broadly define several service classes for its MAN. The following table defines how each of these
services might be mapped to the equivalent DSCP values :

Table 1 - QoS Recommendations in Aggregation / Distribution:
4.2. QoS Features in VNPT MAN

The application of the above features differs with respect to
1. MAN Access Router
The Access routers are responsible for examining IP packets arriving from customer devices for various
characteristics such as application type (Internet, IPTV, Voice) and destinations of the traffic. The packets
can then be classified, using for example, IP precedence, according to the SLA agreed with the customer. For
example, all Voice traffic to a certain destination should be given a “gold” classification, which would mean it
would be given priority over all other traffic of lesser priorities: silver; bronze.
The Access routers also provide ingress bandwidth management from the CE interface and appropriate
queuing on egress to the core network. In this way the edge device ensures that no one customer can flood
the network to the detriment of others. The process of applying CoS/QoS requires intensive CPU processing
and is confined to the edge. It is important to select an appropriate device with the right hardware architecture,
such as ability to enable QoS without sacrificing performance etc.
Based on the mechanisms described, The following illustrates the CoS/QoS operation in a PE router
Figure 1
Figure 23 – QoS at PE device
Typical CoS processing in the Access routers would be as follows:
On ingress to the Access routers, customer traffic is subjected to bandwidth policing.

Dependent upon the customer contract, specific actions can be taken such as dropping the packet or setting an
appropriate precedence bit. Each Access router interface can have multiple rate policies testing for different
types of traffic and classifying them accordingly. For instance, customer traffic could be classified into
various service classes; “”Gold”, “Silver”, “Bronze”. Once the packet has been classified it is passed for
queuing into the core network WRED applies the rules as described in the previous section. If the packet
passes WRED, it is offered for queuing in the appropriate class queue.

4.3. MAN Core routers

The MAN Core routers expedite forwarding of packets while enforcing QoS levels assigned at the edge. The
MAN Core router does this by associating the CoS fields in the label headers with various egress queues on
transmission, which provide the appropriate class of service. The MAN Core router therefore, is freed up from
understanding customer SLA requirements as all the classification has been done once, at the edge.
Figure 24 - CoS Operation between 02 CEs
A typical CoS operation in the would be as follows;

The customer sends an IP packet to the PE. The PE performs packet classification, and applies the appropriate
label pertaining to the service class for that destination. CBWFQ/WRED is applied on the egress to the GSR.
The packets could be MDRR queued through the GSR core based on their class of service in the label header.
Initially, this would not be necessary as the core is over provisioned at present CBWFQ/WRED (using MDRR)
can applied on the egress to the PE from the GSR. The destination PE, forwards the IP packet, based on the
second level label to its correct destination. Egress queuing from the PE would be based on the IP precedence
set at ingress to the MPLS network.
4.4. Application Versus Transport Services

The key to an effective, network-wide IP QoS plan is scalability. Applying QoS on a flow-by-flow basis
throughout VNPT’s NGN is not practical because of the huge numbers of IP traffic flows in its network.
There are actually different QoS strategy within the network, depending on the services.

Figure 25 - QoS for Application services
For application services like IPTV and VoD, QoS is applied on a per class network wide basis whereby
individual devices within the network allocate certain amount of bandwidth on a per hop manner. There is no
need to apply QoS for these applications on an individual user basis, since it does not make sense. On the
other hand, transport services, like high speed internet access may be subjected to per user basis, since
depending on the subscription, different users may be allocated with different bandwidth. Both these strategies
can co-exist within the same network, as illustrated in the following diagram:
Figure 26 - QoS for Transport Services

5. Security Architecture
In order to deliver revenue-generating services for its business and residential customers, it is critical for
VNPT to maintain its network availability. Besides those points that have discussed in the previous section,
one critical factor that needs to be considered in network availability is the security posture of VNPT’s MAN.
VNPT needs to maintain a security strategy for its NGN in order tackle the ever increasing threat of malicious
activities that pose a threat to its business.
Figure 27 - Sercurity Roles in IP Networks
Not surprising to the telecommunication industry, service disruption attacks are on the rise. What is more
alarming is that many of the attackers are motivated by monetary gain, and will often try to hold networks for
ransom. Clearly now, in coming out with VNPT’s MAN architecture, securing the infrastructure of the
network is a top priority.
To sustain service uptime, many factors need to be considered not the least being infrastructure security.
Infrastructure security is a methodology for applying tools and techniques to preserve the integrity of the
network. The methodology as described in this section focuses on securing the control, data, and management
planes.
5.1. Control, Data and Management Planes

All networking devices have functional components that can be divided into three distinct parts :
• Control Plane – the brain of the hardware
• Data Plane – the portion of the router where most of the user traffic traverses
• Management Plane – provides a way to configure and management the device’s resources

Figure 28 - Three planes security
The Control Plane handles the well being of the router and is responsible for activities like routing updates,
keepalive and housekeeping of the many processes running. In normal operation, part of the processor’s job is
to maintain the Forwarding Information Base (FIB) and adjacency tables. This is an example of a control
plane function. Of course, the rest of the caches like ARP is also part of the function. These tables, and rest of
the processes that are running in the CPU of the router, keeps the router running properly and maintain states
with the rest of devices. Without these being properly maintained, the router will fail, and this in turn affects
the integrity of the network. Since control plane is such an important function, any disruption to it will have
detrimental effect. An example of an event that causes disruption of the control plane is a Denial of Service
(DoS) attack. Such activity almost always cause one of the following to occur :
• Near or 100% CPU utilization, which inhibits the router to function properly
• Loss of routing protocol keepalives, which causes route flaps and network stability
• Loss of packets due to buffer exhaustion, causing dropping of legitimate IP traffic
Therefore, with more sophisticated attacks, more and more network managers are paying attention to control
plane protection.
The Data Plane handles most of the traffic forwarding function. For a router that support distributed
forwarding architecture, the line cards are forwarding traffic among the interfaces, with minimal processor
intervention. This is an example of a data plane activity. Most of the activities that happen within the data
plane are value added services like inspection, filtering, marking or translation. For increase performance,
most of these activities are done with the help of ASIC within the line cards, or sometime called hardware-
assist. In the event that a packet it is not handled by the ASIC, it will be passed to the processor for
processing. Of course, this is what we learned about process switching, an event that some refers to as
punting. Any activities that cause punting will affect performance and this is where we want to avoid.
The management plane provides a way to configure and manage the network. Because it can change the way
the network behaves, protecting it from unlawful use if of paramount importance. The management plane
plays an important role in maintaining the resiliency of the network, because it is also responsible for
performance information gathering.
In launching an attack on a network, hackers will always target one of these planes for exploit. Therefore, it is
important to work out a strategy and apply appropriate protection mechanism for each planes.
5.2. Security Threats

This following sections focus on threats that are aimed at the infrastructure of a Service Provider network. The
five major threats are as follows:

• Reconnaissance
• Distributed Denial of Service
• Unauthorized Access/Break-in/Takeovers
• Collateral Damage
• Service/Application Abuse
Although these threat differs how they function, they ultimately target of the three planes that we have just
discussed.
5.3. Trust Model

The trend of increasing attacks has forced service providers to move from an Internet model where it started
with implicit trust, to a model of pervasive distrust. This new approach segments the network into domains
that are trusted, untrusted, or mostly trusted.
The trust boundaries are the demarcation points that segment the network into regions with different trust
levels. The trust level determines how traffic transiting that region should be treated. Typically the network
becomes less trustful as one gets closer to the edge, whereas, the level of trust increases as the network
elements are located closer to the core.
The following diagram illustrates the trust model for VNPT’s NGN :
Figure 29 - Trusted Model
As depicted in the diagram, all network elements at the customer premises are considered "untrusted." Clearly,
VNPT cannot guarantee the enforcement of a security feature on any device out of its control. For this reason,
CPEs and all other elements out of the scope of VNPT administration need to be treated as untrusted.
In the proposed design, the outer trust boundary is enforced at the DSLAM or Metro-edge device equipment,
providing the first line of protection for the MAN infrastructure. The DSLAMs interface directly with
untrusted CPEs, but they provide a “moderate” amount of access security, such as protection against MAC
spoofing and theft-of service, as well as support for user separation. For this reason, they are treated with
moderate trust and another inner trust boundary is implemented at the MAN Access router.

The MAN Access router should provide advanced security features that allow the enforcement of a strong
trust boundary. The DSLAMs and the links connecting them to the MAN Access router are treated with a
medium level of trust (mostly trusted), while all elements from the MAN Access router to the interior of the
MAN network are considered trusted. It is also possible for CPE to connect directly to the MAN Access router,
in which case they, and the links connecting them to the PE are regarded as untrusted.
1. Residential Services
The residential services follow the common trust model, where the DSLAM is viewed as "mostly trusted"
because it will generally support a subset of the common leading practices. So to be safe, it is recommended to
implement an inner line of protection by applying most of the standard security measures on the MAN Access
router that connects to the DSLAMs. Since the DSLAM connects directly to untrusted subscriber equipment,
certain recommendations can be made with regard to protecting the device. Also, it is assumed that residential
customers will only interface directly to the DSLAM and not any other VNPT equipment. Here the DSLAM
interfaces via a 802.1q trunk to the MAN Access router, which terminates the link onto a Layer 3 interface or
sub-interface.
The service VLANs coming from the DSLAM are broken out and mapped to dot1q encapsulations on Layer 3
sub-interfaces on the MAN Access router. Depending on the VLAN service, it can be routed as IP or multicast,
or mapped directly to a pseudo-wire. In either case, Layer 2 MAC storms should have little to no effect on the
MAN Access router. Note that MACs should be dropped on routed interfaces and passed (and unlearned) on
pseudo-wires, and out of profile traffic should be policed.
2. Business Services
The trust model described in the previous section applies for business services as well, that is, the inner trust
boundary starts at the MAN Access router, the DSLAM is mostly trusted, and business CPEs are regarded as
untrusted. The business services differ from the residential ones in that they are may be primarily Layer 2 end-
to-end and often provide business inter-site connectivity both for point-to-point and multipoint scenarios.
They can be transparent offerings suitable for interconnecting remote switches, and/or non-transparent
services that interconnect routers. As with the residential side, voice and data can be typical service
applications.
In terms of security, the business solution can potentially expose the VNPT network to more direct Layer 2
threats than in the residential case. For business services, the untrusted business CPEs can connect directly to
the MAN Access router, whereas residential services can benefit from the security features on the DSLAM
prior to reaching the PE. Note that business services can also be provisioned off the DSLAMs, in which case
they also inherit the same security protection as with the residential offerings.
5.4. Baseline Infrastructure Protection Leading Practices

This section lists some of the common practice for the protection of the NGN infrastructure. These tools and
techniques are organized as follows into a common set of leading practices:
• Disabling unnecessary services
• Controlling device access
• Securing unused ports and interfaces
• Secure the routing and switching infrastructure
• Controlling resource exhaustion
• Policy Enforcement
1. Disabling Unnecessary Services

Routers and switches, as well as other infrastructure devices, typically come out of the box with a list of
services turned on by default that are considered appropriate for most network environments. While default

services certainly ease deployment, there is always a risk that these services could potentially present a
vulnerability that could be used maliciously to gain unauthorized access, or to generate a denial of service. For
this reason, it is a good practice to disable all "unnecessary" services.
Not all networks have the same requirements, and many times some default services are not required, in which
case they should be disabled. Disabling unnecessary services not only eliminates the potential for security
exploits on the disabled services, but also helps preserve system resources. This becomes critical for services
known to be used for malicious purposes. Some of the default services can be used by attackers to obtain
network and user information, bypass security controls, and even generate a denial of service. For example, IP
Source Routing is a default service found in routers that could be used to bypass security controls. It allows
the sender of an IP packet to specify the route that the datagram will follow. An attacker could make use of IP
Source Routing to force the route of the packet, and bypass the security controls that network administrators
might have implemented in the normal routing path.
Once the unnecessary services are identified, it is a good practice to enable them only where they are needed.
Most network devices allow a selective configuration of services. Some services can be activated globally for
the entire system, or per component, typically at a module or interface level. Services known to be prone to
abuse should be deployed only when absolutely necessary.
Finally, disabling services is an activity that requires some planning. Prior to disabling services one should
check for dependencies, as some services may depend on each other. This helps avoid cases where one service
unexpectedly breaks because another service was disabled. Some network devices provide configuration tools
that facilitate the process of disabling unnecessary services. AutoSecure, which is available on Cisco IOS-
based platforms is one such tool. AutoSecure is a CLI set of commands that guide the user in the process of
turning off common IP services that can be exploited, as well as enabling other services and features that can
aid in the defense of the network.
2. Controlling Device Access

Securing the network infrastructure implies establishing the appropriate methods to prevent unauthorized
access to routers, switches, and all the other elements that comprise the infrastructure. To that end it is critical
to understand what access mechanisms are available on each device, how they work, which ones come on by
default, and how they can be protected.
Network infrastructure equipment often provides more access mechanisms than we realize, from console and
asynchronous connections, to remote sessions based on protocols such as telnet, rlogin and SSH. Some of
these mechanisms may even come enabled by default in some platforms. Clearly, establishing controls on
these access mechanisms is fundamental to prevent unauthorized access and device misuse. Anyone who gains
access to a router or switch may obtain critical information about the network, reconfigure the device, and
even take the device out of service. For this reason, each infrastructure device should be carefully configured
to secure all the access mechanisms enabled on the system.
3. Securing Ports and Interfaces

For ease of deployment for some switches, their ports and interfaces are pre-configured with a set of factory
defaults. Unfortunately, some of these defaults do not provide a secure configuration, and as a result, they
might facilitate malicious activities. For this reason, ports and interfaces need to be properly secured.
4. Securing the Routing and Switching Infrastructure

Routing is one of the key components that keeps a network running and as such, it is absolutely critical to take
the necessary measures to secure it. There are different ways routing can be compromised, such as the
injection of illegitimate updates and DoS attacks specifically designed to disrupt routing.
In addition to routing, the access portion of the network supports Layer 2 forwarding by defining dot1q
encapsulation on Layer 3 sub-interfaces, and then mapping the designated VLANs to pseudo-wires. In some

designs, switched interfaces could also be used, in which case they need to be secured against potential DoS
attacks.
5. Controlling Resource Exhaustion

Routers and Switches are often targeted by attacks that directly or indirectly affect their operation. On one
hand, a growing number of attacks specifically target these components by overwhelming the CPU, input
queues, memory and other limited resources. On the other hand, worms and DDoS, which are generally
designed to target end systems, end up generating large volumes of traffic that quite often exhaust most of the
available resources in the infrastructure equipment.
The following security leading practices help protect the infrastructure by controlling the utilization of the
limited resources in routers and switches:
• Monitor CPU and memory usage and set a notification mechanism to alarm for unusual levels.
• Filter traffic sent to the control plane making sure only the expected protocols are allowed.
• Rate limit the traffic sent to the control plane making sure permitted traffic never reaches levels that could
overwhelm the control plane.
• Control traffic that requires the CPU to generate packets.
• Tune input hold queues.
Features such as Control Plane Policing (CoPP) has been researched extensively and made available. Such
features should be a mandatory tool in protecting the control plane of the NGN infrastructure.
6. Policy Enforcement
In a typical network, most infrastructure elements do not need access from outside the network, and only a
few components may require some external connectivity, such as routers with external BGP peers. A security
policy should be built with this in mind, making sure that access to the infrastructure is granted only when
needed, and only for trusted sources, protocols, and ports.
This type of security policy can be enforced by the implementation of packet filters at the edge of the network.
These filters act as the first line of protection against external threats. Therefore, they need to be configured at
the network ingress points, or more precisely, at the ingress interfaces that provide the first line of access to
the network.
At a minimum the packet filters should be configured to provide the following:
• Make sure only external authorized sources can talk to the infrastructure elements, and only for the
expected protocols and ports.
• Block any IP traffic recognized as invalid, i.e., packets with private IP addresses (RFC 1918) or special
use addresses (RFC 3330).
• Provide basic IP anti-spoofing services (RFC 2827).
• Block any non-tunneled Layer 2 PDU traffic at the UNI.

6. IP Version 6
The continuous growth of the global Internet requires that its overall architecture evolve to accommodate the
new technologies that support the growing numbers of users, applications, appliances, and services. IPv6 is
designed to meet these requirements and allow a return to a global environment where the addressing rules of
the network are again transparent to the applications.
The current IP address space is unable to satisfy the potential huge increase in the number of users or the
geographical needs of the Internet expansion, let alone the requirements of emerging applications such as
Internet-enabled personal digital assistants (PDAs), home area networks (HANs), Internet-connected
automobiles, integrated telephony services, and distributed gaming. IPv6 quadruples the number of network
address bits from 32 bits (in IPv4) to 128 bits, which provides more than enough globally unique IP addresses
for every network device on the planet. The use of globally unique IPv6 addresses simplifies the mechanisms
used for reachability and end-to-end security for network devices, functionality that is crucial to the
applications and services that are driving the demand for the addresses.
The lifetime of IPv4 has been extended using techniques such as address reuse with translation and
temporary-use allocations. Although these techniques appear to increase the address space and satisfy the
traditional client/server setup, they fail to meet the requirements of the new applications. The need for always-
on environments (such as residential Internet through broadband, cable modem, or Ethernet-to-the-Home) to
be contactable precludes these IP address conversion, pooling, and temporary allocation techniques, and the
"plug and play" required by consumer Internet appliances further increases the address requirements. The
flexibility of the IPv6 address space provides the support for private addresses but should reduce the use of
Network Address Translation (NAT) because global addresses are widely available. IPv6 reintroduces end-to-
end security and quality of service (QoS) that are not always readily available throughout a NAT-based
network.
Standards bodies for the wireless data services are preparing for the future, and IPv6 provides the end-to-end
addressing required by these new environments for mobile phones and residential Voice over IP (VoIP)
gateways. IPv6 provides the services, such as integrated autoconfiguration, QoS, security, and direct-path
mobile IP, also required by these environments.
Although the success of IPv6 will depend ultimately on the availability of applications that run over IPv6, a
key part of the IPv6 design is its ability to integrate into and coexist with existing IPv4 networks. It is
expected that IPv4 and IPv6 hosts will need to coexist for a substantial time during the steady migration from
IPv4 to IPv6, and the development of transition strategies, tools, and mechanisms has been part of the basic
IPv6 design from the start.
6.1. Planning to Deploy IPv6

One strategy that VNPT can consider about its transition strategy from IPv4 to IPv6 is to begin from the
edges of the network and moves in toward the core. This strategy allows VNPT to control the deployment cost
and focus on the needs of the applications, rather than complete a full network upgrade to a native IPv6
network at this stage. This strategy permits the first stage of the transition to IPv6 to happen now, whether as a
trial of IPv6 capabilities or as the early controlled stages of major IPv6 network implementations later on.
VNPT should evaluate and assess IPv6 requirement now because the current public IP address space may not
be able to satisfy the potential huge increase in the number of users or the demand for new technologies.
Using globally unique IPv6 addresses simplifies the mechanisms used for reachability and end-to-end security
for networked devices, functionality that is crucial to the emerging applications such as Internet-enabled
PDAs, HANs, Internet-connected automobiles, integrated telephony services, and distributed gaming.
VNPT can look at the deployment of IPv6 in three key phases:
Providing an IPv6 service at the customer access level
Running IPv6 within the core infrastructure itself
Interconnecting with other IPv6 service providers

Starting the deployment of IPv6 at the customer access level permits an IPv6 service to be offered now
without a major upgrade to the core infrastructure and without an impact on current IPv4 services. This
approach allows an evaluation of IPv6 products and services before full implementation in the network, and
an assessment of the future demand for IPv6 without substantial investment at this early stage.
At the end of this initial evaluation and assessment stage, as support for IPv6 within the entire infrastructure
improves, and as applications fully embrace IPv6, the core network infrastructure can be upgraded to support
IPv6. This upgrade path could involve use of dual-stack routers (a technique for running both IPv4 and IPv6
protocols in the same router), or eventually use of IPv6-only routers as the IPv6 traffic becomes predominant.
Interconnections with other IPv6 service providers or with the 6bone allow further assessment and evaluation
of IPv6, and a better understanding of the requirements for IPv6.
6.2. IPv6 Over MPLS

IPv6 over MPLS MAN enables isolated IPv6 domains to communicate with each other over an MPLS IPv4
core network. This implementation requires far fewer backbone infrastructure upgrades and lesser
reconfiguration of core routers because forwarding is based on labels rather than the IP header itself,
providing a very cost-effective strategy for the deployment of IPv6.
Additionally, the inherent Virtual Private Network (VPN) and traffic engineering services available within an
MPLS environment allow IPv6 networks to be combined into VPNs or extranets over an infrastructure
supporting IPv4 VPNs and MPLS-TE.
A variety of deployment strategies are available as follows:
• IPv6 using tunnels on the customer edge (CE) routers
• IPv6 over a circuit transport over MPLS
• IPv6 on the provider edge (PE) routers (known as 6PE)
• IPv6 VPN on the provider (PE) routers (known as 6VPE)
The first of these strategies has no impact on and requires no changes to the MPLS provider (P) or PE routers
because the strategy uses IPv4 tunnels to encapsulate the IPv6 traffic, thus appearing as IPv4 traffic within the
network. The second of these strategies also requires no change to the core routing mechanisms. The third
strategy just requires changes to the PE routers to support a dual-stack implementation, but all the core
functions remain IPv4. The last strategy takes advantage of operational IPv4 MPLS backbones and is more
like a regular IPv4 MPLS-VPN provider edge, with an addition of IPv6 support within Virtual Routing and
Forwarding (VRF). It provides logically separate routing table entries for VPN member devices.
The proposed Cisco MAN Architecture supports the introduction of IPv6 into VNPT’s network using any or
all of the above options.

Figure 30 - 6VPE Deployment

7. Element Management System
7.1. Introduction
Building a next generation Metro Ethernet network and its network management system are part of the effort
to transition the current VNPT networks into a single pervasive IP Next Generation Networks (NGN) network;
enabling convergence of legacy data, traditional voice and many other services.
Cisco understands the requirements and has herein presented a complete integrated end to end Element&
Network Management System for the proposed aggregation network. The system would have complete fault,
troubleshooting, performance, provisioning and planning capabilities that will satisfy the VNPT requirements.
7.2. Integrated Metro E aggregation Management Solution

The proposed Element Management System is designed to manage 7609 Internet routers. VNPT understands
the challenges for a network management system in such a large scale network. Cisco is proposing an
combined EMS and NMS solution with a strong touch in provisioning to manage the proposed networks.
The solution proposed by Cisco would leverage existing applications as an extension of the end to end
management system. This would greatly benefit VNPT’s network manageability in the following ways,
1) Reduce the operation risk. The proposed EMS product are all commercially off-the-shelf and provide
management functions out of box, with enough flexibility to address VNPT’s long term gaol. These will
results significant OPEX savings, such as training, process integration and testing.
2) Lower the CAPEX. Cisco IS Solution Centre is an industry leader and Cisco’s flagship MPLS
provisioning tool. VNPT has invested ISC in both Hanoi and HCM Metro E project. By re-using ISC not
only VNPT can reduce upfront investment; but also reduce the cost in integrating it with upper layer OSS
applications.
Currently Cisco has proposed the applications to address specifically the tender requirements. The proposed
applications are similar to the one existing therefore during the implementation some applications may be able
to leverage exiting applications.
1. Solution Architecture
The proposed Network Management System supports Configuration, Fault and Performance management.
Cisco provisioning solutions provision services in an automated, "flow-through" fashion on an end-to-end
basis, speeding deployment and reducing operations costs. Through an integrated validation function, these
solutions eliminate a wide range of potential provisioning errors. Service Fault and Performance application
provide tools for effectively and efficiently monitoring performance and faults in the network from a service-
oriented perspective; SLA monitoring, customer and administrative partitioning, and flow-through integration
to other systems.
Implementing the Role Based Administration and Control (RBAC) features provide VNPT a multi-tier, multi-
region and multi-tenant security regime. A complete and true geographical redundancy system ensures the
constant network operation even during or after a catastrophic event.

Figure 31 Proposed Provisioning & EMS Overall Architecture
2. Solution Components
The proposed network management solution for the ME61 network is based on five Cisco modules, Cisco IP
Solution Centre, Cisco Info Centre 7.0 (CIC 7.0), CiscoWorks Health Utilization Manager, CiscoWorks QPM,
CiscoWorks Internet Performance Monitor.
The IP Solution Centre 5.x is Cisco’s flagship provisioning application. The Cisco ISC Layer 2 VPN
Management application provides the tools for VNPT to effectively manage the entire lifecycle of L2 Ethernet
PW, Any Transport over MPLS (AToM), L2TPv3, and Metro Ethernet services. Management features such as
policy based VPN and quality of service (QoS) provisioning help minimize the cost of deploying Layer 2
VPN services. The management features reduce errors and increase the efficiency of service deployment and
management.
Figure 32 Proposed EMS/NMS Application Components
CiscoWorks is a bundle of tools that enables ME61 to measure the network performance, construct the reports
based on IP SLA MIBs available in the proposed devices; and to plan and construct QoS commands during
the commissioning of the network. The CiscoWorks LMS 2.5 is a web-based and management product for
managing Cisco networks and devices. Current CiscoWorks 2.5 offerings include Resource Manager
Essentials and CWSI Campus. These products provide inventory, configuration and software management
capabilities, traffic management, and analysis tools, as well as integrated views and reports of network
information with built-in access to Internet resources such as Cisco Connections Online (CCO). This coupling
of Cisco device management call device centre with web-based technologies, allows the CiscoWorks LMS 2.5
deliver on Cisco’s vision of a comprehensive Device Management.
The Cisco Info Centre is a Service-Level Management (SLM) system that provides a consolidated view of
multi-vendor events and status information. It collects event streams or messages from many different data
sources and presents a single, consistent view of the current state of all Network Elements. It correlates and

store the alarms in Oracle repository using Reporter before it distributes the event information to the operators
and administrators responsible for monitoring service levels.
All five applications are integrated using the common inventory. While the deployment of the applications
could spread to several servers to increase the scaling, the applications would function as a single software
package.
For a detail product feature description of Cisco Craft Work Interface EMS ,CiscoWorks, and CIC, please
refer to the attached Appendix A.
7.3. Integrated Metro E Management System Functional Design

Network Management is more an operational best practice than the software application. Cisco recommends
VNPT take a functional approach to design the integrated system in order to overcome the day to day
operational challenges. The following sections would discuss typical management functions whereby the SP
operations are defined
Configuration and Device Management

Fault isolation and Operational Management
Performance Monitoring
Security and Role based Administration
Redundancy Strategy
1. Configuration and Device Management

Configuration and Device management function is offered by the Cisco IP solution Centre and CiscoWorks
LMS. VNPT operator can use NetConfig application in CiscoWorks LMS to compose, apply and commit a
portion of a CLI configuration for multiple devices simultaneously. Working in this way, VNPT would have
full access to command-completion, help and syntax checking capabilities. Cisco IP Solution Centre provides
service aware configurations management. All syntax and options are verified against the devices prior to the
deployment. ISC also offers full service inventory support to ensure that all services are reflected on the
devices. Errors at any stage are clearly displayed in an error logs and are separated so that you can quickly
determine which devices had which errors.

Figure 33 Configuration Function
CiscoWorks Device Centre and CiscoView provides a graphical representation of the Device configuration
and line card allocation. It is used to verify the configuration performed by either Netconfig or ISC. Using ISC
VPN Viewing application an operator can have a service view of the provisioned topology .
2. Fault management and Operation

Cisco Info Centre (CIC) offers a combination of alarm processing rules, filtering, customisable alarm viewing,
and partitioning. CIC can consolidate, de-duplicate, filter, and correlate fault information from multiple
network layers. CIC interacts with Cisco SNMP and Cisco Syslog mediator to effectively retrieve fault events
from the proposed Cisco Routers.
Figure 34 Fault Management and Operation Function
Cisco Info Centre collects network events and stores them in a high performance distributed database. It then
presents event information to users through customized filters and views. Once a view has been defined, the
Cisco Info Centre software consolidates all events that match a defined filter into one single event. This
process is called event deduplication. Using deduplication, events from single or multiple Cisco devices, for
example, can be consolidated into one unified event. By consolidating multiple events that represent
symptoms of the same or related network faults into one event, Cisco Info Centre reduces the volume of

network and alarm messages. This frees network operators from having to look at masses of SNMP messages
and other events to analyze the network.
Once an event is received, it is broken down into data fields and the Cisco Info Centre interface allows
operators to view the events that have been consolidated into a unified event.
All events would be received directly from Cisco
The CIC Reporter is a real-time, web-based client-server application that provides accurate, historical
reporting on network events data forwarded from the CIC server. It provides VNPT with not only real-time
but also long term, retrospective information about the behavior of devices, links, and services within the
networks.
The CIC product suite captures event data from more than 150 network management environments and
devices, through a wide range of probes. As a result, the CIC Reporter allows operators to analyze and
display service-level reports from diverse environments,
3. Security and Role Based Administration Management

Management application and network access security is implemented using the Role Based Administration
and Control features. These features are available in both CIC and CiscoWorks. Cisco proposes VNPT to
implement two layers of RBAC security –
1) Geographical
2) Functional.
Combining the two layers of RBAC security, VNPT would receive the most robust, granular and yet flexible
NMS security access control.
Geographical RBAC - Cisco recommends VNPT to divide devices into groups according to the geo location
of the devices. To manage ME-61 Metro E aggregation network we can have a total of 63 groups - 61 groups
for provincial devices, one group for the devices within HCM city and one group for devices in Hanoi city.
Figure 35 Security and RBAC Design
The Groups feature in CiscoWorks can be used for this purpose. It partition the network managed by
CiscoWorks applications. It helps in creating, managing, and sharing groups of devices. The groups created
using this feature are shared across applications. The groups created in applications can also be viewed from
Common Services too.
Functional RBAC - Cisco recommends VNPT to assign users to role groups. Each role group would have the
defined access rights to the devices on the predefined region. To manage ME61 network we can have 4 role
groups per region – Administrator, Provisioning, Operator and Read Only. In addition to this, we propose
creating a “ME61 Admin Role” which would have full right to the entire network. There will be a total of

4x63+1 =253 user role groups. Each NOC operator would be required to join one or more role groups to gain
access rights to the proposed NMS.
CiscoWorks and Cisco Info Centre both support above mentioned RBAC features.
4. Geographic Redundancy
By leveraging the built-in redundancy features of CIC and a combination full active-active capability
CiscoWorks; Cisco proposed a complete geographic redundancy design. VNPT would be setting up two
geographically redundant network management systems to ensure continuous operation should catastrophic
event occurs on either site.
Figure 36 Geo Redundancy Design
As depicted in the diagram, two sets of Geographical Management Sites (GMSs) and two sets of NetWork
Operation Centres (NOCs) would be installed. The GMS is used to host servers for network management
applications. The NOC is used to house network operators and their workstations. These two GMSs are
designed for load sharing purpose during normal operation. Each NOC would access there local GMS during.
Should there be any total shutdown at any GMS, the other GMS would react and support both NOCs.
Both CIC and CiscoWorks are passive monitoring tools. These applications do not interfere with real device
operation while running. Cisco recommends installing active copies of CIC and CiscoWorks on both GMSs,
each equip with a complete set of devices. In other words CIC and CiscoWorks would manage the devices
simultaneously from different GMSs.
7.4. Deployment Strategy

Cisco proposes all NMS applications to install on 8 x T2000 servers. CIC would be deployed on 4 servers, 2
x object servers and 2 x reporter servers. Oracle for reporter would be installed on both reporter servers.
CiscoWorks would be deployed on 2 x T2000.
In addition to the production servers, there would be 2 x T2000 lab servers for CIC and CiscoWorks. IP
Solution Centre would be installed on 2 x T2000 servers.
7.5. Conclusion
Cisco propose a full feature, fully redundant, modular, scalable, complete end to end Metro Ethernet
management solution that enables VNPT to offer the most competitive business Ethernet services and the
most feature rich subscriber triple play services in the market. Cisco is confident the proposed Provisioning &
EMS solution not only offers unprecedented value but also is one of the most cost effective solutions.

7.6. Appendix A
1. Cisco IP Solution Centre

Cisco IP Solution Center (ISC) is a family of intelligent network management applications that help reduce
overall administration and management costs by providing automated resource management and rapid profile-
based provisioning capabilities. ISC enables fast deployment and time to market of Multiprotocol Label
Switching (MPLS) and Metro Ethernet technologies. There are four applications in ISC 5.0, which can
operate alone or as a suite in an MPLS Management Solution:
The Cisco ISC MPLS VPN Management (ISC:MPLS) application helps Service Providers offering MPLS
VPN Services by providing the provisioning, planning, and troubleshooting features essential to manage the
entire-life cycle of MPLS VPN Services. MPLS management features include policy-based VPN,
Management VPN and MPLS VPN routing audit. These features help to guarantee accurate service
deployment and to reduce the cost of deploying new and revenue-producing MPLS VPN services.
Cisco IP Solution Center supports a wide range of Cisco devices, from Cisco 2800/3800 ISR all the way up to
CRS-1, across a wide range of IOS and IOS XR releases where appropriate. Cisco ASR 1000 is supported in
ISC L3 Provisioning in ISC 5.0.1 and above, and in MDE 2.1.2 (ISC 5.0.2) and above.
This application can also work with the Cisco MPLS Diagnostics Expert product for VPN post-provisioning
check. The Cisco MPLS Diagnostics Expert is an automated, workflow-based network management product
that troubleshoots and diagnoses problems in MPLS VPN deployments.
- The Cisco ISC Layer 2 VPN and Metro Ethernet Management (ISC:L2VPN/ME) application helps
enterprises and service providers offering Layer 2 VPN services by providing the provisioning, planning, and
troubleshooting features essential to manage the entire lifecycle of Layer 2 VPNs, Any Transport over MPLS
(AToM) and Metro Ethernet services. Management features such as policy-based VPN and management VPN
help minimize the cost deployment of Layer 2 VPN services, and guarantee the accuracy of service
deployment.
- Cisco ISC Traffic Engineering Management (ISC:TEM) is Cisco's exclusive planning and provisioning
application for Cisco MPLS Traffic Engineering (MPLS-TE)-enabled routers. Cisco ISC:TEM enables
superior Fast Re-Route (FRR) protection and Bandwidth Guarantees by generating the paths for tunnels that
meet constraints, including bandwidth, DiffServ-aware Traffic Engineering (DS-TE) pool, affinity, delay, and
protection. ISC:TEM uses world-class hybrid optimization techniques to provide better network protection
and major network utilization improvements. It automatically discovers, audits, optimizes, and deploys
MPLS-TE tunnels including tunnels in Cisco devices that reside in a multivendor environment. Graphical and
table-based displays of MPLS-TE tunnels and MPLS-TE-enabled devices and interfaces give the user full
access to all MPLS-TE configurations.
2. CiscoWorks
CiscoWorks provides the integrated management tool sets needed during the initial deployment and low level
commissioning phase. The purpose of the tool sets are to simplify the configuration, administration,
monitoring, and troubleshooting of VNPT ME61 networks. It provides an integrated system for sharing
device information across management tool sets , automation of device management tasks, visibility into the
health and capability of the network, and identification and localization of network trouble. By using common
centralized systems and network-inventory knowledge, CiscoWorks delivers a unique platform of cross-
functional management capabilities that reduces network administration overhead and provides upper-layer
systems integration.
The proposed CiscoWorks includes:

• CiscoWorks Device Fault Manager (DFM) 2.0

• CiscoWorks Campus Manager 4.0

• CiscoWorks Resource Manager Essentials (RME) 4.0
• CiscoWorks Internetwork Performance Monitor (IPM) 2.6
• CiscoWorks Common Services 3.0 with CiscoView 6.1
• CiscoWorks Health and Utilization Monitor 1.0
• CiscoWorks QoS Policy Manager 3.2
The CiscoWorks uses a centralized system for sharing device information across all applications, improving
manageability and allowing the management system to more dynamically adjust to changes. CiscoWorks also
offers a new lightweight desktop interface that facilitates rapid navigation between tools and that can be
modified to individual workflow needs. CiscoWorks utilizes security information maintained in Cisco Secure
Access Control Server (ACS) to simplify the management of user privileges. Cisco Secure ACS integration
provides flexibility in defining user roles, and supports secured user views of specific devices, groups of
devices, or by geographic or logical network segments. Significant improvement in performance, such as
multithreading background tasks, has reduced the time needed to deliver updates to the network, as well as
generate reports. Efficient task processing and its shared database of managed devices allow CiscoWorks to be
deployed into larger networks.
CiscoWorks Quality of Service Policy Manager (QPM) provides centralized management of quality of service
(QoS) policy creation, validation, deployment, and monitoring to enable the secure and predictable delivery of
business applications.
Designing, deploying, and monitoring QoS is a complex process that requires automation. CiscoWorks QPM
provides network administrators with comprehensive QoS provisioning and monitoring capabilities allowing
them to manage and fine-tune the delay, delay variation (jitter), bandwidth, and packet loss parameters
required for successful end-to-end deployment and optimal utilization of network resources. The end result is
networkwide intelligent, consistent, and sophisticated QoS that allows performance protection for voice, video,
and Internet business applications while reducing costs and optimizing the utilization of network resources.
In the latest version of QPM, the new QoS features such as hierarchical QoS, network-based application
recognition packet description language module (NBAR PDLM), virtual channel bundle, and time-based
access control lists (ACLs), are all supported; A simplified high-level workflows for network selection, QoS
provisioning, QoS monitoring, and reporting are also included.
Cisco Heath and Utilization Monitor (CHUM) allows VNPT to

Measures network performance by monitoring device components such as memory, CPU, interface ports for
their Utilization and Availability levels.
Pre-defined MIB templates for user convenience - CiscoWorks HUM provides System-defined MIB templates
that allows network administrators to create Pollers easily. System-defined MIB templates provide most of the
common network parameters that is needed to monitor a device connected to the network.

User-defined MIB templates -CiscoWorks HUM allows network administrators to create custom or user-
defined MIB templates by leveraging MIB variables from System-defined MIB templates or by grouping new
MIB variables.
Supports polling of additional SNMP data - CiscoWorks HUM allows network administrators to add any
other Cisco MIB apart from the standard MIBs provided as part of the application to create pollers.
Cisco devices support a large number of performance MIBs which can all be used to construct the
performance reports. Popular MIB are 1) IP SLA MIB;2) Interface Stat MIB;3)NBAR MIB; 4) RMON MIB;
5) Class Based QoS MIB;6)CISCO-ENHANCED-MEMPOOL-MIB,CISCO-ENVMON-MIB,CISCO-
MEMORY-POOL-MIB,CISCO-PROCESS-MIB,ENTITY-MIB,OLD-CISCO-CHASSIS-MIB,RFC1213-
MIB. All of these MIB can be working with external probes.
Using CHUM, VNPT can monitor the following performance parameters on the proposed NE,
• Monitoring Devices for Availability
• Monitoring Devices for CPU, Memory and Interface Utilization
• Monitoring Device Interface Errors
• Creating Custom Templates for Polling
• Creating Threshold Rules for MIB Variables
• Generating Periodic Reports
• Viewing Reports from LMS Applications
• Compiling New MIB Files
The following are some of the default reports, this report can be in 24 hours, weekly, monthly or yearly,
• CPU Utilization
• Device Availability
• Interface Utilization
• Memory Utilization
• Interface Availability
• Interface Error Rate
• Threshold Violation
3. Cisco Info Centre

CIC is a Service-Level Management (SLM) system that provides a consolidated view of enterprise-wide
events and status information. It collects event streams or messages from many different data sources and
presents a single, consistent view of the current state of all Cisco Info Centre managed systems. It distributes
the event information to the operators and administrators responsible for monitoring service levels. This
information can then be:
• assigned to operators
• passed on to help desk systems
• logged in a database
• replicated on a remote Cisco Info Centre system
• used to trigger automatic responses to certain events.
Cisco Info Centre allows diverse management platforms, applications and Internet protocols to be brought
together to provide the administrator a single point of monitoring those platforms and applications. Cisco Info
Centre does not replace the management platforms. It instead compliments them by providing an enterprise
wide event/fault and status exchange. Cisco Info Centre can also tie together domain limited network
management platforms in remote locations.
Cisco Info Centre tracks the state of events in a high performance distributed database and presents
information of interest to specific users through individually configurable filters and views. Cisco Info Centre

automation functions can also be used to perform intelligent processing on the current state of managed
objects. Cisco Info Centre has the following key features,
• Integrates different and multiple event streams or messages from the source of the data such as Cisco
WAN Manager and consolidates them into a single view.
• Provides a way of normalizing data from different sources and data mapping techniques.
• VPN and CNM support for alarm and event management from diverse sources and different vendor’s
products.
• Substantial reduction in event and alarm volume due to sophisticated deduplication and correlation
mechanisms.
• Requires little or no configuration to automatically start gathering all messages and events, not just a
subset of them
• Distributes state information from existing management systems to staff across the enterprise
• Transforms fragmented tactical management domains into coherent strategic management domains
• Rapid, non-disruptive deployment across the enterprise
• Derives state (for example, up or down) from events, forming data into information
• An open and flexible main-memory database, capable of storing and processing all events received,
which allows operators to key in on any field describing important aspects of the service they need to
monitor
• Sophisticated filtering provides views customized to individual user requirements
• Provides a single tool to view status information and launch other management applications
• Builds on existing systems and expertise, ensuring short learning curves
• Enterprise-scale solutions can be installed and productive in hours using plug-and-play components
• Multi-domain management using peer-to-peer, hierarchical, and Web topologies
• Incorporates high performance distributed Cisco Info Server database technology
• Implemented using open protocols
• Real-time data available to other applications
• An optional Java based Event List providing consistent, current, web-accessible views of services
• Operational facilities include control management, recovery and restore management, and component
fail over support
• Security facilities for user and group management
• Provides an audit trail of actions taken on events through journal facilities.
Cisco Info Centre can also be used to support service level monitoring. The flexible definition of partitions
and event filters within Cisco Info Centre allows Service Providers to monitor the status of services that
encompass multiple technologies and resources. Services sold to customers typically include multiple
resources. Monitoring of customer services inherently requires the ability to monitor all elements that
comprise a service.
For example, a managed network service might include a variety of WAN, LAN, and computing resources.
With Cisco Info Centre, it is possible to create an abstract view of services that provides the status of each
customer's services as a whole. This simplifies the monitoring of customer services, and allows the Service
Provider to create a management environment that best suits their management model or understanding of
what constitutes a service.
The Cisco Info Centre solution can support event monitoring on a global network scale, as well as on a user-
defined partition of the network.

CNM
Admin
CNM
Admin Center
Desktop Partitioning
Java
Info Gateway
Event
Web
Center
Info Server
Info Expert
Info VPN
Server Partitioning
Info Info Info Remedy

Mediator Mediator Mediator TT
System
SV+ CW EMS Service
Support
RDBMS System
MultiService Network
ASCII/
SNMP
Figure 317 CIC Internal Architecture
Info Centre, which is represented in the figure above, collects data from Info Mediators. The mediators
available are for Cisco WAN Manager and HP OpenView. In addition all the Syslog messages from loghost
can be sent to Info Centre.
The Info Centre application is based upon a distributed client/server model with the clients being info
mediators which communicate with the network elements and which in turn communicate with a central info
server.
The Info Mediator is a collector of fault and performance data from management systems, such as
CiscoWorks (syslog). Info Mediators are supported for other technologies and vendors as well. The
mediators convert technology or vendor specific information into a canonical form, which the Info Server can
understand. Mediators can perform local filtering and translation of information, and shield the Info Server
from technology or vendor specific details.
The Info Server is the core of the product architecture, and receives fault and performance data from multiple
Info Mediators simultaneously. The Info Server is a memory resident, real-time database that de-replicates
the fault data it receives (i.e., it records the number of times an event has occurred) and correlates the
information according to rules that are easily specifiable by the administrator. A key feature of the product is
the easy-to-configure filters and correlation rules and event-driven automation that can be created in real time
by non-programmers. This approach allows for customization of the application based on operator needs
without lengthy development times.
Persistent storage of events requires the use of a database gateway known as Info Gateway that interacts with
relational database systems, and is discussed in the section on Info Gateways. The configuration of the rules
system is achieved internally by Boolean algebraic expressions and SQL, but the set of customization GUI
tools provided by Info Centre hides the details and provides an easily configurable user interface.
The correlation rules and filters that are interpreted by the Info Server are Boolean and SQL based, and are
evaluated for all incoming events to the Info Server. Rules facilitate the isolation of the real faults within the
network that operators might be concerned with, distinguishing them from other less important events. Info
Centre supports dynamic creation of filters and rules via intuitive administration tools. This capability is
important in today's dynamic network environments, where rapid changes in the management paradigm are
required to match the changing behavior of large networks. The administrative tools allow the user to rapidly
deploy usable solutions without programming knowledge.
The events that are collected by the Info Server can be correlated and applied to user-defined entities,
representing anything from physical elements to logical resources. These elements are defined as visual
objects within the Admin Desktop using a tool called ObjectiveView. ObjectiveView allows users to create

views representing any resources and their relationships and status. For administrative VPN's, this view might
be a physical view of a specific geographical partition of the network, where entities represent nodes and links
between nodes. Entities can also represent servers, management platforms, customer services, customers,
departments, etc. Both X11/Motif and WEB/Java based desktops are supported within Info Centre.
The Info Gateways provide the linking and filtering mechanism to the world outside of the Info Server. Info
Gateways dynamically filter and forward events to other systems, and can provide either uni- or bi-directional
information transport. The information passed by an Info Gateway is customizable to the user's needs.
Several Info Gateway products are available, including ones that forward filtered information to:
• other Info Servers for filtered event distribution,
• Sybase and Oracle RDBMS (Informix to come),
• Remedy Action Request System,
• ASCII log files (in the form of text), and
• other management systems (in the form of SNMP traps).
The Info Gateway is a key component to creating customized views of the network and resources. The Info
Gateway defines what information is passed to other. Info Gateways can also be used for consolidation of
information from multiple Info Servers, as well as for information partitioning and distribution to multiple
Info Servers.
The Cisco Info Centre supports both X11/Motif and Java/WEB based client front-ends. The Java Event List
Server is a Java daemon that runs on a WEB server and brokers the communication between an Info Server
and the Java applets. The Java applets can be displayed in any WEB page and viewed by a Java enabled
WEB browser. Several different styles of applets are supported, and can be customized to fit the customer's
WEB environment. Authentication is also supported to control access or modification of information. The
Java Event List applets currently provide filtered and partitioned event list information. Future applets will
provide performance and availability information on selected resources. The result is a collection of
customizable, modular applets that can be easily assembled by the customer into any WEB page.
In summary, CIC provides:

• Filtering, correlation and partitioning of network information
• Rules-based alarm reduction and customizable event triggered actions
• VPN, CNM, and resource partitioning support
• Scalable and robust product components
• Powerful administrative desktop views

8. MAN-E Sparing Strategy

For Vung Tau, Can Tho, Binh Duong, Dong Nai, Ha Tay, Quang Ninh, Hai Phong, Hue, Da Nang, Nghe An
We proposed Cisco 7600 for both Core and Access Node of Metro Ethernet network required in the RFP. The
proposed Cisco 7600 platform included 7604, 7606, 7609 and 7613 uses all the same cards (line cards,
fabric/route processors cards...) and interchangable between all model. This help VNPT reduce the number of
spare parts across all platform. For example: instead of having different spared switch fabric cards for Core
and Access nodes. With Cisco 7600, it doesn’t matter whether it is 7604, 7606, 7609 or 7613 using in Core
and Access nodes, only one (1) spare part is required for the switch fabric.
Spare parts is very important element in service providers to ensure continuity of operation, service level
agreement to the end customers and optimize the operation cost. The geographical locations of 10 provinces in
the required RFP included Vung Tau, Can Tho, Binh Duong, Dong Nai, Ha Tay, Quang Ninh, Hai Phong,
Hue, Da Nang, Nghe An are stretch from the North to the South of Vietnam. Taking to consider that those 10
provinces are belong to 4 different management centers of VNPT namely Hanoi, Danang, HCMC and Can
Tho. We are proposing that at least 4 pieces of spares need to be included for each type of technical
component. It means that each management region of VNPT will have at least one piece of spare for each type
of component. This allocation of spare help reducing the repair time for each fault caused by network
hardware by shortening the time to reach the failure site.
The proposing number of spare is considered as hot spares in VNPT stock, beside that Cisco also have a store
in Vietnam to provide Next Bussiness Day (NBD) service to VNPT according the professional service
proposal. This service cover an immediate shipment of a part in Cisco stock after 1 working day upon the
receival of failure notice from the customers. This service is under maintenance and warranty contract and
will last for 2 years. With this superior service, the quantity of spare (considered as hot spare) will be filled by
the Cisco stock in maximum 5-7 days for the worst case scenario considering other factors such as
transportation, handover procedures, late failure notice receival...
We use a special Cisco tool to calculate the number of spare based on the requirement in RFP from VNPT as
below:
• Clause 98 Overall Technical Condition

• Clause 115 RFP for MAN
Quantities of spare parts should be based on the MTBF & MTBR figures for the individual parts involved,
weighted by the quantity of parts required in the system and a factor included to ensure that there is at least a
95% chance that the required spare part will be available during the first two years of operation.
As baseline the tool utilizes the Poisson distribution to calculate the likelihood (probability) that a certain
number of failures will occur over a given period of time. It takes as input, the number of units in service, the
unit MTBF, and the time period being considered. the Poisson distribution is a discrete probability distribution
that expresses the probability of a number of events occurring in a fixed period of time if these events occur
with a known average rate and independently of the time since the last event.
The calculator also calculates the sparing needs to satisfy a set of given conditions. The formula is as follows:

where:
n = number of Line Replacement Units in service
λ = failure rate per 1 million operating hours
R = repair time in hours
s = minimum number of spares required
CL = confidence level
For the paritcular requirement of VNPT we will have
n = proposed quantity of equipment part

λ = failure rate per million hours calculated from given MTBF figures
R = The maximum time to deliver the Cisco stock spare to VNPT spares (7 days)
CL = 95%, confident that the number of hot spares provided will be adequate to proposed quantity.
S = How many spare assemblies should be on hand
Cisco spare calculation tool is a reverse calculation of above formula to get the S numbers, and show in the
table below .
The result showed that all number of needed spare for each part by the calculation is less than the allocation
by the geographical (managerial) region. So we proposed that the number spare for each part will be 4 pieces
to guarantee the repair time in emergency.

Engineering, Procurement and Construction of MAN Ethernet -
ACTUAL SPARE CALCULATION FOR PROPOSED QUANTITY OF EQUIPMENT

FOLLOWED POISON DISTRIBUTION FUNCTION
Product Description Total MTBF figures R=7 days
quantity (hours)
PWR-2700-DC/4= 2700W DC Power Supply for Cisco 7604/6504-E 234 367,850 1

PWR-2700-DC= 2700W DC power supply for CISCO7606 182 367,850 1
7600-ES20-10G3C= 7600 ES20 Line Card, 2x10GE XFP with DFC 3C 104 94,000 1
7600-ES20-GE3C= 7600 ES20 Line Card, 20xGE SFP with DFC 3C 411 97,500 2
CISCO7604= Cisco 7604 Chassis 117 131,000 1
CISCO7606-S= Cisco 7606-S Chassis 91 529,758 1
CISCO7609-S= Cisco 7609-S Chassis 50 176,382 1
PWR-4000-DC= 4000W DC PS for CISCO7609-S/CISCO7609/13, Cat6509/13 100 300,000
chassis 1
RSP720-3C-GE= Cisco 7600 Route Switch Processor 720Gbps fabric, PFC3C, GE 86,555
516 3
SFP-GE-L= 1000BASE-LX/LH SFP (DOM) 5610 1,000,000 3
SFP-GE-Z= 1000BASE-ZX Gigabit Ethernet SFP (DOM) 160 1,000,000 1
WS-F6700-DFC3B= Catalyst 6500 Dist Fwd Card, 256K Routes for WS-X67xx 184 100,000 1
WS-X6704-10GE= Cat6500 4-port 10 Gigabit Ethernet Module (req. XENPAKs) 184 110,000 1
XENPAK-10GB-ER+= 10GBASE-ER XENPAK Module with DOM support 141 1,000,000 1
XENPAK-10GB-LR+= 10GBASE-LR XENPAK Module with DOM support 192 1,000,000 1
XENPAK-10GB-ZR= 10GBASE-ZR XENPAK Module 37 1,000,000 1
XFP-10GER-OC192IR= 10GBASE-ER and OC192 IR2 XFP Module 79 1,131,606 1
XFP-10GLR-OC192SR= Multirate XFP module for 10GBASE-LR and OC192 SR-1 66 1,131,606 1
XFP-10GZR-OC192LR= 10GBASE-ZR and OC192 LR2 XFP Module 41 3,039,506 1
FAN-MOD-9SHS= High Speed Fan Module for CISCO7609-S Chassis 50 773,874 1
For consideration of additional factors to calculate spare such as statistic of actual returned in-service unit in VNPT network.over specific period..., Cisco
is happy to work with VNPT to optimize the number of spare units.

9. Total Power Consumption for Proposed system of Cisco 7600

We calcualed the power for proposed system based on the power consumed by each type of active components.
The list of component and their respective power consumption are as below
Power
Consumption
Equipment
Per Card
(Watts)
Cisco 7609-S Chassis (included 2 FAN modules) 484.00
Cisco 7604 Chassis (included 2 FAN modules) 60.06
Cisco 7606-S Chassis (included 2 FAN modules) 311.00
Cisco 7600 Route Switch Processor 720Gbps fabric, PFC3C, GE 309.96
7600 ES20 Line Card, 20xGE SFP with DFC 3C 277.00
1000BASE-LX/LH SFP (DOM) 1.05
1000BASE-ZX Gigabit Ethernet SFP (DOM) 1.05
7600 ES20 Line Card, 2x10GE XFP with DFC 3C 277.00
Multirate XFP module for 10GBASE-LR and OC192 SR-1 8.00
10GBASE-ER and OC192 IR2 XFP Module 8.00
10GBASE-ZR and OC192 LR2 XFP Module 8.00
Cat6500 4-port 10 Gigabit Ethernet Module (req. XENPAKs) 377.16
10GBASE-LR XENPAK Module with DOM support 8.00
10GBASE-ER XENPAK Module with DOM support 8.00
10GBASE-ZR XENPAK Module 8.00
The figures in above list are retrieved from the latest information of about power consumption of Cisco 7600. This is the maximum power consumption
which could be drawed from DC power plant in worst case scenario.

The total power consumption is calculated by multiply with the total quantity of proposed system. The total value is 442,026.32 W
TOTAL POWER CONSUMPTION FOR PROPOSED SYSTEM

Power
Total proposed Extended
Consumption
Product Equipment quantities Power
Per Card
(Units) (Watts)
(Watts)
CISCO7609-S Cisco 7609-S Chassis (included 2 FAN modules) 484.00 50 24,200.00
CISCO7604 Cisco 7604 Chassis (included 2 FAN modules) 60.06 117 7,027.02
CISCO7606-S Cisco 7606-S Chassis (included 2 FAN modules) 311.00 91 28,301.00
RSP720-3C-GE Cisco 7600 Route Switch Processor 720Gbps fabric, PFC3C, GE 309.96 516 159,939.36
7600-ES20-GE3C 7600 ES20 Line Card, 20xGE SFP with DFC 3C 277.00 411 113,847.00
SFP-GE-L 1000BASE-LX/LH SFP (DOM) 1.05 5,610 5,890.50
SFP-GE-Z 1000BASE-ZX Gigabit Ethernet SFP (DOM) 1.05 160 168.00
7600-ES20-10G3C 7600 ES20 Line Card, 2x10GE XFP with DFC 3C 277.00 104 28,808.00
XFP-10GLR-
Multirate XFP module for 10GBASE-LR and OC192 SR-1 8.00
OC192SR 66 528.00
XFP-10GER-
10GBASE-ER and OC192 IR2 XFP Module 8.00
OC192IR 79 632.00
XFP-10GZR-
10GBASE-ZR and OC192 LR2 XFP Module 8.00
OC192LR 41 328.00
WS-X6704-10GE Cat6500 4-port 10 Gigabit Ethernet Module (req. XENPAKs) 377.16 184 69,397.44
XENPAK-10GB-LR+ 10GBASE-LR XENPAK Module with DOM support 8.00 192 1,536.00
XENPAK-10GB-ER+ 10GBASE-ER XENPAK Module with DOM support 8.00 141 1,128.00
XENPAK-10GB-ZR 10GBASE-ZR XENPAK Module 8.00 37 296.00
TOTAL POWER CONSUMPTION FOR PROPOSED SYSTEM (Watts) 442,026.32

Technical Proposal - Solution Overview - Printed

Uploaded by

Copyright:

Available Formats

You might also like

Technical Proposal - Solution Overview - Printed

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Technical Proposal - Solution Overview - Printed

Uploaded by

Copyright:

Available Formats

Engineering, Procurement and Construction of MAN

Engineering, Procurement and Construction of MAN

Vietnam Posts and Telecommunications Group

Technical Proposal - Item 1

Technical Proposal (Solution

3rd July 2008 Commercial-In-Confidence Page 1

6.1. Planning to Deploy IPv6 55

3rd July 2008 Commercial-In-Confidence Page 3

1.1 Cisco Carrier Ethernet Solution

1.2 Cisco Solution Architecture Advantages

3rd July 2008 Commercial-In-Confidence Page 4

 Ensure Network High Availability by providing In Chassis Hardware Redundancy,

1.3 Cisco presence in Vietnam

1.4 Cisco Differentiators

3rd July 2008 Commercial-In-Confidence Page 5

platform, Cisco Distributed Denial of Service (DDoS) Protection, Cisco

3rd July 2008 Commercial-In-Confidence Page 6

2. VNPT Network Architecture

VNPT Network Architecture

VN2 Core Internet

Figure 1 – VNPT Target Network Architecture

2.1. Physical Network Design

3rd July 2008 Commercial-In-Confidence Page 7

2.2. Proposed MAN Equipment

• Ease of configuration, provisioning and management

• Efficient and common equipment sparing

• Flexible equipment deployment for Core and Access

• Lower training cost for VNPT staff

• A choice of form factors purpose-built for high availability

• Channelized T1/E1, Channelized T3, and Channelized OC-3/STM-1

• Circuit Emulation T1/E1, E3 and STM-1

3rd July 2008 Commercial-In-Confidence Page 8

• OC-3/STM-1, OC-12/STM-4, OC-48/STM-16 Packet over SONET/SDH (PoS), and OC-192/STM-64

• OC-3/STM-1 ATM, OC-12/STM-4 ATM, and OC-48/STM-16 ATM

• Fast Ethernet, Gigabit Ethernet, and 10 Gigabit Ethernet

2.3. VNPT MAN Transport Options

3rd July 2008 Commercial-In-Confidence Page 9

2.4. Integrating ME61 and VN2

3rd July 2008 Commercial-In-Confidence Page 10

VNPT Network Transport Architecture

Figure 2 – MAN & VN2 integration using IEEE 802.1ad option

VNPT Network Transport Architecture

Figure 3 – MAN & VN2 integration using MPLS option

3rd July 2008 Commercial-In-Confidence Page 11

2.5. Integrating ME61 to Hanoi PT and HCMC PT MAN

2.6. MAN Transport Protocol Architecture

2.6.1. MAN IP/MPLS Architecture

3rd July 2008 Commercial-In-Confidence Page 12

2.6.2. MAN IGP OSPF Architecture

VNPT MAN OSPF Design

Figure 4 - Sample OSPF design

3rd July 2008 Commercial-In-Confidence Page 13

2.6.3. MP-BGP4 (Multiprotocol BGP)

2.6.4. BGP Route Reflectors for VNPT MAN Network

3rd July 2008 Commercial-In-Confidence Page 14

Figure 5 - MPLS VPN with Route Reflector design

For the VNPT MPLS network:

2.6.5. Using BGP for VPLS Auto Discovery

Ensure Network High Availability by providing In Chassis Hardware Redundancy,