Nokia Advanced Troubleshooting SG v3.0.2

Module 0 | 1
Advanced Troubleshooting v3.0.2

© 2016 Nokia
This Nokia Advanced Troubleshooting v3.0.2 is for the exclusive use of Dandan HUA - ALCATEL-LUCENT
This course is part of the Nokia Service Routing Certification (SRC) Program. For more information on the SRC
program, see https://networks.nokia.com/src
To locate additional information relating to the topics presented in this manual, refer to the following:
 Technical Practices for the specific products referenced in this course
 Internet Standards documentation such as protocol standards bodies, RFCs, and IETF drafts
 Technical support pages of the Nokia website located at: https://networks.nokia.com/support
Module 0 | 2 Advanced Troubleshooting v3.0.2 © 2016 Nokia

Module 0 | 3
© 2016 Nokia
Module 0 | 4
© 2016 Nokia
Module 0 | 5
© 2016 Nokia
Module 0 | 6
© 2016 Nokia
Module 0 | 7
© 2016 Nokia
Module 0 | 8
© 2016 Nokia
Module 0 | 9
© 2016 Nokia
Module 0 | 10
© 2016 Nokia
Module 0 | 11
© 2016 Nokia
Module 0 | 12
© 2016 Nokia
Module 0 | 13
© 2016 Nokia
Module 0 | 14
© 2016 Nokia
Module 0 | 15
© 2016 Nokia
Module 0 | 16
© 2016 Nokia
Module 0 | 1
© 2016 Nokia
Module 0 | 2
Advanced Troubleshooting v.3.0.2
© 2016 Nokia
Module 0 | 3
© 2016 Nokia
Module 0 | 4
© 2016 Nokia
Module 0 | 5
© 2016 Nokia
Module 0 | 6
© 2016 Nokia
Module 0 | 7
© 2016 Nokia
Nokia Advanced Troubleshooting (AT)
This course is part of the Nokia Service Routing Certification (SRC) Program. For more information on the SRC program,
see www.Nokia.com/src.
To locate additional information relating to the topics presented in this manual, refer to the following:
 Technical Practices for the specific product.

 Internet Standards documentation such as protocol standards bodies, RFCs, and IETF drafts.
 Technical support pages of the Nokia website located at: http://www.Nokia.com/support.
Module 0 | 8 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 0 | 9
© 2016 Nokia
Factors Impacting Availability
The landscape is set so that there is an expectation of availability by consumers for the networks they use. What can a
network operator do to increase their network availability?
Three areas that require special attention are:
1. Hardware Stability - How redundant is your system? Do you have redundant control processors that support a full
chassis with negligible packet loss and non-stop services?
2. Network Resiliency – Does your network deploy fast re-reroute, standby secondary path, or sonnet protection? Is
the physical transport geographically dispersed? Are links identified as shared risk in your routing protocol? Is your
redundant site on the same power source? Have you deployed UPS?
3. Process and Support - Do you have an escalation process defined? Do you have a support contract for your
network equipment? How easily can you detect performance degradation and isolate or resolve network outages?
What sort of Network Management tools do you have available? How quickly can you recover from failures in your
network?

Network Availability:
A network’s reputation depends on a number of factors, one of which is network availability. The greater the availability
of a network is, the better its reputation is. Downtime means more than just outage time for customers; on the
operator’s side, it also means extra support and sales efforts.
The Dilemma:
Network availability became a contentious topic since there was no separation among the network, hardware, and
application. For example, a network failure could exist while the hardware and application were running. Were they really
available? Ways of counting it became debatable. If a router showed up for a full year, the uptime was reported as
100%. However, what if an interface was down? Approaching the formula from a per port basis resulted in missing the
5 9s target on a fair amount of customers while the majority experienced 100% availability. Would the average across
all the ports be a more accurate estimation of someone’s experience with the network?
Even though there is disagreement on the validity of the numbers, network operators always strive for the 5 9s target
based on their own internal rules.
SLAs – Always On:

The shift was made to providing “always on” service to consumers and with this came Service Level Agreements. From
users’ perspectives, they are really concerned with access to the services when they want them. If there is a network
failure and the network re-routes in under 50ms, it is of little consequence to them as long as they get the service they
requested.
With understanding from customers that repair items that require a truck roll will inevitably miss a 5 9s agreement, a
new set of realistic goals were established. They typically cover response times and Mean Time to Repair (MTTR) metrics
that if exceeded would result in credits to the customer.
An example of an SLA might be 15 minutes to respond to a support request and 4 hours to repair. After 8 hours of
outage, there is a 1% credit on the next bill for every hour of outage.

The impact of downtime is experienced in many different ways by both the customer and the provider.
As a consequence of an outage, both the customer and provider are in the danger of losing revenue.
 Most SLAs will include credits for outage time which can be very costly.
• It is possible to lose a large portion of the margins gained during a single significant outage.
 While an outage is taking place, both the customer and provider require an increase in their support
activities.
• On-call, overtime hours paid, and time off in lieu.
 Word of mouth brand degradation is a likely consequence when an outage is prolonged or mishandled.
• Major outages have their way of making themselves into viral media and news channels with
today’s Internet and may impact the future and renewal of sales.
• Having proper escalation channels, including the time spent managing these channels to ensure
that the best service possible is provided, comes at a cost.

MTBF provides an indicator to the expected frequency of outages within a network.
It is common practice to remove non service effecting issues and scheduled maintenance windows from the
calculations.
Improved by the following factors:
1. Hardware Stability - How redundant is your system? Do you have redundant control processors that support a full
chassis with negligible packet loss, non-stop services, and other HA features?
2. Network Resiliency – Does your network deploy fast re-reroute, standby secondary path, or sonnet protection? Is
the physical transport geographically dispersed? Are links identified as shared risk in your routing protocol? Is your
redundant site on the same power source? Have you deployed UPS?

MTTR is an indicator of how long it will take for a problem to be resolved. The lower the number the better.
MTTR includes the total outage time regardless of when it was detected. The pressure is on the network provider to be
proactive in the network to find and repair issues.
MTTR is a KPI in many support groups and included in most SLAs with customers.
1. Process and Support - Do you have an escalation process defined? Do you have a support contract for your
network equipment? How easily can you detect performance degradation and isolate or resolve network outages?
What sort of Network Management tools do you have available? How quickly can you recover from failures in your
network?
Can range from a few milliseconds, as in the case of

an uninterrupted power supply, to many hours or
even days.

MTBF and MTTR are one of the most common methods of determining network availability.

Module 0 | 16
© 2016 Nokia
Network Readiness is a term that refers to the following items:
• Building a redundant network and using the resilient hardware to achieve higher MTBF.
• Creating a network baseline to measure anomalies against.
• Implementing a network monitoring and reporting solution to reduce MTTR.
• Ensuring that service routers and switches are protected from security attacks to achieve a higher MTBF.
• Protecting customers from security threats to improve their quality of experience.

Module 0 | 18
© 2016 Nokia
Various tools exist to protect the routers as well as user traffic.
In the event of DoS (Denial of Service) attacks, Management Access Filters, CPM Queues, and CPM Filters maintain the
integrity of services using the Nokia SR portfolio.

Module 0 | 20
© 2016 Nokia
The Nokia SR platform provides numerous tools to perform baseline testing.
For example, an SAA test can be performed to measure the latency and jitter across a network before turning a service
over to a customer.
As another example, a network operator may turn on cflowd to baseline the current internet peering traffic distribution
before adjusting policies. They would also use cflowd to monitor the impacts of changes made.

The Nokia SR platform provides numerous tools to isolate network incidents.
For example, while isolating an incident, an operator may use an OAM command to locate a specific MAC address in a
VPLS service. Once located, the operator may monitor the rate of specific traffic on a port, and use show commands to
view the queue statistics.
Debug commands are often used to isolate control plane issues. They are very useful for session establishment issues.
Cflowd may be consulted to detect any traffic pattern changes that may isolate an upstream network failure.

Self monitoring is an approach to network management that leverages the use of each individual router to monitor
itself and report to a collector, or series of collectors.
The Nokia SR platform supports this approach by allowing various tools. For example, a network manager can use “cron”
to schedule an SAA probe to perform an OAM test every 5 minutes and generate an “SNMP trap” if a threshold is
crossed.
SAM can be used as a collector to correlate multiple events and notify network managers of incidents.

Module 0 | 24
© 2016 Nokia
Module 0 | 25
© 2016 Nokia
Module 0 | 26
© 2016 Nokia
Module 2 | 1
© 2016 Nokia
Module 2 | 2
© 2016 Nokia
Module 2 | 3
© 2016 Nokia
Module 2 | 4
© 2016 Nokia
Logs provide data which ultimately reduces the amount of time spent resolving network issues, thus indicating an
improved MTTR value.
Logs are the first place to look when investigating an issue. The outputs, for the most part, are comprehensible in
English and thus provide a strong indicator of where to start your isolation attempts.
Logs also aid in the documentation process. They are time-stamped to allow for accurate MTTR and MTBF recording.
When deploying logs throughout a network, timing synchronization becomes very important to the correlation of
events.

To configure the SR product to synchronize timing with a network time source, an sntp protocol is supported. Having all
network devices synchronized to the same clock reference allows events generated from different nodes to be
correlated, thus reducing MTTR.
The NTP algorithm is much more complicated than the SNTP algorithm. NTP normally uses multiple time servers to
verify the time and then controls the slew rate of the PC. The algorithm determines if the values are accurate using
several methods including fudge factors and identifying time servers that don't agree with the other time servers. It
then speeds up or slows down the PC's drift rate so that (1) the PC's time is always correct and (2) there won't be any
subsequent time jumps after the initial correction. Unlike NTP, SNTP usually uses just one Ethernet Time Server to
calculate the time and then it "jumps" the system time to the calculated time. It can, however, have back-up Ethernet
Time Servers in case one is not available. During each interval, it determines whether the time is off enough to make a
correction and if it is, it applies the correction.

7x50 has two default memory logs (Log-id 99 & 100) containing all events from the “main” application. All severity
levels of alarms are recorded in log-id 99 where as log-id 100 only contains serious errors.
Log 98 is created by SAM for SNMP events and is typically reserved as a best practice if a network has not deployed
SAM.
Operators can create 97 additional custom logs from various sources and have them sent to various destinations. It is
important to create custom logs before you need the events you are trying to collect.
In addition to event logs, the Nokia 7x50 also has special filter logs that can be used to send a copy of frames matching
a specific entry in an ip-filter or a mac-filter.

Event Logs
Event logs are a means of recording system generated events for later analysis. Should there be a fault within a 7X50 system, event
logs can be used for troubleshooting. Events are messages generated by the system by applications or processes within the 7X50.
7X50 OS supports event logging. Event logging controls the generation, dissemination, and recording of system events for monitoring
status and troubleshooting faults within the system. The logging:
 Provides you with logging information for monitoring and troubleshooting.
 Allows you to select the types of logging information to be recorded.
 Allows you to assign a severity to the log messages.
 Allows you to select the source and destination of logging information.
Event logs are a means of recording system generated events for later analysis. Events are messages generated by the system by
applications or processes within the 7X50.
Event sources are the main categories of events that feed the log manager. The 7X50 groups events into four major categories.
 Security Events — Events that pertain to attempts to breach system security. The security event source indicates all events
that affect attempts to breach system security such as failed login attempts, attempts to access MIB tables to which the user
is not granted access, or attempts to enter a branch of the CLI to which access has not been granted. Security events are
generated by the SECURITY application.
 Change Events — Events that pertain to the configuration and operation of the node. The change activity event source
indicates all events that directly affect the configuration or operation of the node. Change events are generated by the USER
application.
 Debug-Trace Events — Debug and trace messages that have been enabled for applications or processes. The debug event
source indicates all debugging and trace messages that have been enabled on the system. Debug events are generated by
the DEBUG application.
 Main events — Events that pertain to 7X50 OS applications that are not assigned to other event categories/sources.
Before an event can be associated with a log-id, the from command, identifying the source of the event, must be configured. Only
one destination can be specified for a log-id. The destination of an event stream can be in memory buffer, console, session, snmp-
trap-group, syslog, or file. Use the event-control command to suppress the generation of events, alarms, and traps for all log
destinations.

Event Control
Event control pre-processes the events generated by applications before the events are passed into the main event
stream. Event control assigns the severity for each application event and determines whether the event should be
generated or suppressed. The severity numbers and names supported in 7X50 OS conform to ITU standards M.3100
X.733 & X.21 (as shown above).
Events that are suppressed by event control will not generate any event log entries since they never reach the log
manager. Event control maintains a count of the number of events generated (logged) and dropped (suppressed) for
each application event. The severity of an application event can be configured in event control.
Application events contain an event number and description that explains why the event is generated. The event
number is unique within an application; however, the number can be duplicated in other applications.
The following example, generated by querying event control for application events, displays a partial list of event
numbers and names.

Log Manager
Events that are forwarded by event control are sent to the log manager. The log manager manages the event logs in the
system as well as the relationships between the log sources, event logs and log destinations, and log filter policies.
An event log has the following properties:
 A unique log ID
• The log ID is a short, numeric identifier for the event log.
 One or more log sources
 A source stream (or streams) to be sent to the log destination can be specified. The source must be identified
before the destination can be specified. The events can be from the main event stream, the security event
stream, the user activity stream, or all debug-trace messages in the debug stream.
 One event log destination
 A log can only have a single destination. The destination for the log ID destination can be that of a console,
session, syslog, snmp-trap-group, memory, or file on the local file system.
 An optional event filter policy
• An event filter policy defines whether to forward or drop an event or trap based on match criteria.

Event Filter Policies
The log manager uses event filter policies to allow fine control over which events are forwarded or dropped based on
various criteria. Filter policies have a default action. The default actions are to either:
 Forward
 Drop
Filter policies also include a number of filter policy entries that are identified with an entry ID and define specific match
criteria including a forward or drop action for the match criteria. Each entry contains a combination of matching criteria
that define the application, event number, severity, and subject conditions. The entry’s action determines how the
packets should be treated if they have met the match criteria.
Entries are evaluated in order from lowest to highest entry ID. The first matching event is subject to the forward or drop
action for that entry.
A match criteria entry can include combinations of:
 Equal to or not equal to a given system application.
 Equal to, not equal to, less than, less than or equal to, greater than or greater than or equal to an event number
within the application.
 Equal to, not equal to, less than, less than or equal to, greater than or greater than or equal to a severity level.
 Equal to or not equal to an event subject string.

Log Destinations
An event log within 7X50 OS associates the event sources with logging destination. 7X50 OS supports the following log
destinations:
 Console
 Session
 Memory logs
 Log files
 SNMP trap group
 Syslog
Only a single log destination can be associated with an event log or an accounting log. An event log can be associated
with multiple event sources, but it can only have a single log destination.
A file destination is the only type of log destination that can be configured for an accounting log.
Console
Sending events to a console destination means the message will be sent to all active console sessions. If there are no
active console sessions, the event log entries are dropped. The console device can be used as an event log destination.
Session
A session destination is a temporary log destination which directs entries to the active console session for the duration
of that session. When the session is terminated, the event log is removed. Event logs with a session destination are not
stored in the configuration file. Event logs can direct log entries to the session destination.

The show log log-id command proves a first look summary of all logs created on the node.
Log 99 records all events across all severities from the source of main (M). Note that it has logged a much larger number
of event logs since inception. It is important to note that even though log-id 99 has logged 17,056 events, only the last
500 are viewable.
Log-id 100 only logs events with a severity greater than major.
log filter 1001
default-action drop
description "Collect events for Serious Errors Log"
entry 10
action forward
description "Collect only events of major severity or higher"
match
no application
no number
severity gte major
no router
no subject
exit
exit

Information to view show command
Displays a list of all application names that show log applications

can be used in event-control and
filter commands.
Displays event control settings for events show log event-control

including whether the event is suppressed [application [event-name |
or generated and the severity level for the event-number]]
event.
Displays event file log information. show log file-id [file-id]
Displays log collector statistics for the main, show log log-collector
security, change, and debug log
collectors.
Displays an event log summary with show log log-id [log-id]

settings and statistics or the contents of a [severity severity-level][application
specific log file, SNMP log, or memory application] [sequence from-seq [toseq]]
log. [count number] [subject subject]
[ascending | descending]

Event logs can be filtered using the application attribute on the command line.
<application>: application_assurance|aps|atm|bgp|cflowd|chassis|
debug|dhcp|dhcps|efm_oam|elmi|ering|eth_cfm|etun|
filter|gsmp|igh|igmp|igmp_snooping|ip|ipsec|isis|
l2tp|lag|ldp|li|lldp|logger|mcpath|mc_redundancy|
mirror|mld|mld_snooping|mpls|msdp|nat|ntp|oam|ospf|
pim|pim_snooping|port|ppp|pppoe|rip|route_policy|
rsvp|security|snmp|stp|svcmgr|system|user|video|vrrp|
vrtr

Module 2 | 17
© 2016 Nokia
Rollover Interval – defined in minutes and determines how long a file will be used before it is closed and a new log file is
created.
Logs can consume lots of flash space thus it is advised to never log to compact flash 3 (cf3:).

A best practice is to configure the log-id and file-id to be equal.

Note: The log naming on the file system contains 3 sections: Log information-date-timestamp
In this example,
log0601 represents log-id 06, file-id 06.
20110331 represents the calendar date of file creation on the node.
124450 represents a timestamp.

Debug can be very useful when troubleshooting a problem. To activate the “debug” attributes on an Nokia router, the
administrator must first create a log-id and define where the debug information will be posted. The options range from
sending debug traffic directly to your session to logging the traffic to buffers of the router.
In the example above, the administrator creates a log-id for debug traffic. The output of the debug is posted to the
session. Upon creation, the administrator then activates the debug he/she wants. In this case, it is OSPF packets. Once
activated, the output is placed to the session.
To turn off the debug, use the “no debug” command. When sending debug traffic to the session, the prompt may not
be visible. That is “OK”, since the router will still accept your typing. If lots of information is being sent to the session,
you may not see what you are typing. That too is “OK”, since the router will keep track of the input. When the debug is
turned off, the router will send the following message:
A:R1# no debug
Trace disabled for all existing and future clients
Debug output can also be turned off by shutting down the log that was created for the debug. The debug statement will
remain and if the log is “no shutdown”, the debug output will resume.

An event log has been created to keep track of all OSPF attributes in the router. Note that the filter was created first
and then applied to the log-id. Also, note that the log ID is using a value between 1-100.
The numeric range of event logs is from 1-100.
The following is a list of match criteria that can be used as of FG8.0R4:
*A:R6>config>log>filter>entry# match
- match
- no match
[no] application - Specify application id to match

[no] number - Specify event-id number to match
[no] router - Specify router context string to match
[no] severity - Specify severity level to match
[no] subject - Specify subject string to match
Note:
If no action is set within a log filter entry, the entry will follow the action of the default-action.

Debug is available for an extensive subset of protocol and hardware processes operating on the Nokia SR.
Consult the user guides specific to the version you are running for a complete list.

Filter logs allow for the trapping of information about a specific device or application based on the use of an Access
Control List. The list defines what traffic is placed into the log file. In the above example, a simple filter has been
created to match against the specific IP address of 105.0.0.1. Note that all traffic is permitted with this filter. The
function of the filter is not to block traffic from entering the router, but is to identify traffic that should be placed into
log 150. Once created, the filter is applied to the interface.
These log files are numerically limited from 101-199 .

To verify that the filter works correctly, the administrator can enter the command “show filter ip 50” and examine the
output. Note that the filter is using “log id” 150; however, this does not show the total number of entries that have
been placed.
Each entry within the filter is shown in detail. This command is very useful for verifying the accurate configuration of an
ACL.

To examine the information that has been placed into the log, the administrator should use the “show filter log”
command. In this case, two entries are shown in the log. There are a total of 198 entries out of a total of 500 allowed.
When the 501st is placed into the log, the first entry is removed.
The “IP Filter” statement defines the filter number and entry number within the filter that qualifies the trapped
information being entered into the log file.

Syslog can be deployed in various architectures.
For a fully redundant system, a network operator would deploy either a Multiple Destinations architecture or a
Distributed Relay architecture. In both cases, in the event of a backhaul network failure, syslog messages have a greater
probability of being tracked.
In a smaller installation, only a single central server may be deployed.

Syslog
Syslog destinations have the following properties:
 A syslog server IP address.
 A UDP port used to send syslog messages.
 A Syslog Facility Code (0 - 23) (default 23 - local7).
 A Syslog Severity Threshold (0 - 7) - events exceeding the configured level are sent.
Syslog uses eight severity levels whereas the 7750 SR-series uses six. This results in a mapping of SR severity levels to
syslog severities .

The following syntax is used to configure a syslog:
configure log syslog syslog-id address ip-address description description-string facility syslog-facility level syslog-level
log-prefix log-prefix-string port port
address – Specifies the target syslog host address.

description – Description for the target syslog host address.
facility – Configures the facility code that is used for the messages associated with the syslog-id.
level – Configures the syslog message severity level threshold.
log-prefix – Configures a string to be prepended to every log message sent to the target syslog host.
port – Configures the UDP port used to generate syslog messages.

To limit the number of messages sent to the syslog server, a filter can be applied to the log.

Module 2 | 31
© 2016 Nokia
Module 2 | 32
© 2016 Nokia
OAM information is conveyed in slow protocol frames called OAM protocol data units (OAMPDUs). OAMPDUs contain the
appropriate control and status information used to monitor, test and troubleshoot OAM-enabled links. OAMPDUs
traverse a single link, being passed between peer OAM entities, and as such, are not forwarded by MAC clients (like
bridges or switches).

802.3ah enabled ports can be in a specific mode – Active or Passive
 Default is Active
— At least one port must be in Active mode.
 Passive mode ports prevents specific actions as mentioned above
— It is recommended to have it enabled for managed CPE devices that support 802.3ah (7250 SAS)

Module 2 | 35
© 2016 Nokia
Customers who subscribe to Epipe service treat the Epipe as a wire, so they demand the ability to run 802.3ah between
their devices which are located at each end of the Epipe.
Note:
This feature only applies to port-based Epipe SAPs because 802.3ah runs at port level not VLAN level. Hence, such
ports must be configured as null encapsulated SAPs.

Module 2 | 37
© 2016 Nokia
Module 2 | 38
© 2016 Nokia
Module 2 | 39
© 2016 Nokia
Parameters:
accept-remote-loopback - Enables reactions to loopback control OAM PDUs from peers.
hold-time - This command configures efm-oam operational transition dampening timers which reduce the number of
efm-oam state transitions reported to upper layers. (Values 0-50)
mode - This command configures the mode of OAM operation for this Ethernet port.
transmit-interval - This command configures the transmit interval of OAM PDUs. (Values 1-600 (in 100 milliseconds))
multiplier - Specifies the multiplier for transmit-interval to set local link down timer. (Values 2-5)
tunneling - This command enables EFM OAM PDU tunneling. Enabling tunneling will allow a port mode Epipe SAP to pass
OAM frames through the pipe to the far end.
Note:
•802.3ah must be enabled on both ends of a link for the physical ports to go operationally up
•When turning on 802.3ah the port may go to status (Link Up) until the discovery process is completed.

Module 2 | 41
© 2016 Nokia
Below is the remainder of the output;
…
Peer Mac Address : 8e:4c:01:01:00:02
Peer Vendor OUI : 00:16:4d
Peer Vendor Info : 00:00:00:00
Peer Mode : active
Peer Pdu Size : 1518
Peer Cfg Revision :0
Peer Support : LB
Loopback State : None

Loopback Ignore Rx : Process
===============================================================================
Ethernet Oam Statistics
===============================================================================
Input Output
-------------------------------------------------------------------------------------------------------------------
Information 1297335 1297388
Loopback Control 0 0
Unsupported Codes 0 0
Frames Lost 0
===============================================================================

Flow – a uni-directional traffic stream defined by several characteristics such as source and destination IP addresses,
source and destination ports, inbound interface, IP protocol, and TOS bits.
When a router receives a packet for which it currently does not have a flow entry, a flow structure is initialized to
maintain state information regarding that flow. Subsequent packets matching the same parameters (bytes, IP
addresses, port numbers, and so on) of the flow contribute to the byte and packet count of the flow until it is
terminated and exported by the router to a Cflowd collector.
Cflowd is also useful for web host tracking, accounting, network planning and analysis, network monitoring, developing
user profiles, data warehousing and mining, as well as security-related investigations. Collected information can be
viewed in several ways such as in port, AS, network matrices, and/or pure flow structures. The amount of data stored
depends on the cflowd configurations.

Note:
Excessive sampling over an extended period of time, for example, more than every 1000th packet, can burden router processing
resources.
The following data is maintained for each individual flow in the raw flow cache:
• Source IP address
• Destination IP address
• Source port
• Destination port
• Input interface
• Output interface
• IP protocol
• TCP flags
• First timestamp (of the first packet in the flow)
• Last timestamp (timestamp of last packet in the flow prior to expiry of the flow)
• Source AS number for peer and origin (taken from BGP)
• Destination AS number for peer and origin (taken from BGP)
• IP next hop
• BGP next hop
• ICMP type and code
• IP version
• Source prefix (from routing)
• Destination prefix (from routing)
• MPLS label stack from label 1 to 6
Within the raw flow cache, the following characteristics are used to identify an individual flow:
• Ingress interface
• Source IP address
• Destination IP address
• Source transport port number
• Destination transport port number
• IP protocol type
• IP TOS byte
• Virtual router ID
• ICMP type and code
• MPLS labels

Cflowd can be deployed in a similar method to syslog servers.
As a network operator, certain information that would result in significant data traffic may be stored locally in a region.
Other statistical data that may be of use for real time monitoring in the network management center may be sent
directly to a centralized location.
In some smaller installations, only a single central server may be deployed. A final alternative is available to store
information locally at each site.

1. As a packet ingresses a port, a decision is made to forward or drop the packet.
2. If the packet is forwarded, it is then decided if the packet should be sampled for cflowd.
3. If a new flow is found, a new entry is added to the cache. If the flow already exists in the cache, the flow statistics are
updated.
4. If a new flow is detected and the maximum number of entries are already in the flow cache, the earliest expiry entry
is removed. The earliest expiry entry/flow is the next flow that will expire due to the active or inactive timer
expiration.
5. If a flow has been inactive for a period of time equal to or greater than the inactive timer (default 15 seconds), then
the entry is removed from the flow cache.
6. If a flow has been active for a period of time equal to or greater than the active timer (default 30 minutes), then the
entry is removed from the flow cache.
When a flow is exported from the cache, the collected data is sent to an external collector which maintains an
accumulation of historical data flows that network operators can use to analyze traffic patterns.

When a flow is exported from the cache, the collected data is sent to an external collector which maintains an
accumulation of historical data flows that network operators can use to analyze traffic patterns.
Data is exported in one of the following formats:

• Version 5 — Generates a fixed export record for each individual flow captured.
• Version 8 — Aggregates multiple individual flows into a fixed aggregate record.
• Version 9 — A more flexible format that allows for different templates or sets of cflowd data to be sent based on the
type of traffic being sampled and the template set configured.
V5, V8, and V9 Flow Processing

1. As flows are expired from the active flow cache, the export format must be determined, either Version 5, Version 8,
or Version 9.
2. If the export format is Version 5 or Version 9, no further processing is performed and the flow data is accumulated to
be sent to the external collector.
3. If the export format is Version 8, then the flow entry is added to one or more of the configured aggregation matrices.
As the entries within the aggregate matrices are aged out, they are accumulated to be sent to the external flow
collector in Version 8 format. The sample rate and cache size are configurable values. The cache size default is 64K
flow entries.
A flow terminates when one of the following conditions is met:

• When the inactive timeout period expires (default: 15 seconds). A flow is considered terminated when no packets are
seen for the flow for N seconds.
• When an active timeout expires (default: 30 minutes). A flow terminates according to the time duration regardless of
whether or not there are packets coming in for the flow.
• When the user executes a clear cflowd command.
• When other measures are met that apply to aggressive age flows as the cache becomes too full (such as an overflow
percent).

There are three modes in which cflowd can be enabled to sample traffic on a given interface:
1. Cflowd interface, where all traffic entering a given port will be subjected to sampling at the configured sampling rate.
2. Cflowd interface plus the definition of IP filters which specify an action of interface disable-sample. Traffic that
matches these filter entries will not be subject to cflowd sampling.
3. Cflowd ACL, where IP filters must be created with entries containing the action filter sampled. In this mode only
traffic matching these filter entries will be subject to the cflowd sampling process.
By enabling cflowd at the interface level, all IP packets forwarded by the interface are subject to cflowd analysis. By
setting cflowd as an action in a filter, only packets matching the specified filter are subject to cflowd analysis. This
provides the network operator greater flexibility in the types of flows that are captured.
The following cflowd components must be configured for cflowd to be operational:

• Cflowd is enabled globally.
• At least one collector must be configured and enabled.
• A cflowd option must be specified and enabled on a router interface.
• Sampling must be enabled on either:
 An IP filter which is applied to a port or service.
 An interface on a port or service.

AS Matrix — Flows are aggregated based on source and destination AS and ingress and egress interface.
Protocol-Port — Flows are aggregated based on the IP protocol, source port number, and destination port number.
Source Prefix — Flows are aggregated based on source prefix and mask, source AS, and ingress interface.
Destination Prefix — Flows are aggregated based on destination prefix and mask, destination AS, and egress interface.
Source-Destination Prefix — Flows are aggregated based on source prefix and mask, destination prefix and mask,
source and destination AS, and ingress interface and egress interface.
Raw — Flows are not aggregated but are sent to the collector in a V5 record.

CLI Syntax: config>cflowd#
active-timeout minutes
cache-size num-entries
inactive-timeout seconds
template-retransmit seconds
overflow percent
rate sample-rate
collector ip-address[:port] {version [5 | 8 | 9]}
aggregation
as-matrix
destination-prefix
protocol-port
raw
source-destination-prefix
source-prefix
template-set {basic | mpls-ip}
autonomous-system-type [origin | peer]
description description-string
no shutdown
no shutdown
active-timeout – Specifies, in minutes, how long cflowd samples an active flow before terminating the flow. The range is 1 to 600.
The default is 30.
cache-size – Specifies the maximum number of active flows in the flow cache table. The range is 1000 to 131072. The default is
65536.
inactive-timeout – Specifies how long, in seconds, cflowd waits for a matching flow packet before it considers the flow inactive and
terminates it. The range is 10 to 600. The default is 15.
overflow – Specifies the percentage of flow cache entries cflowd removes when the number of cache entries exceeds the cache-size
value.
rate - Specifies the rate at which cflowd samples packets. Cflowd samples one packet out of every N packets where N is the parameter
value. For example, a value of 100 specifies that one of every 100 packets is to be sampled whereas a value of 1 means that all
packets are to be sampled. The range is 1 to 10 000. The default is 1000.
template-retransmit - Specifies how often, in seconds, cflowd sends cflowd template definitions to collectors. The parameter is
configurable when the Version parameter, which is set when configuring the collectors, is set to version-9. The range is 10 to 600.
The default is 600.

CLI Syntax: config>cflowd#
collector ip-address[:port] [version version]
aggregation
as-matrix
destination-prefix
protocol-port
raw
source-destination-prefix
source-prefix
autonomous-system-type [origin | peer]
no shutdown
template-set {basic | mpls-ip}
Ip-address - Specifies the cflowd collector host address. Also specifies an IPv4 address in dotted-decimal format.
port - Specifies the cflowd collector UDP port. The range is 1 to 65 535. The default is 2055.
description - Specifies a description for the created object. The range is 0 to 80 characters.
version - Specifies the cflowd version of the collector. The options are 5, 8 or 9.
aggregation - Specifies the type of aggregation scheme to export. The parameter is configurable when the version parameter is set
to 8. The following are types of options:
 raw - Flows are not aggregated but sent to the collector in a V5 record.
 destination-prefix - Flows are aggregated based on the destination prefix and mask, destination AS, and egress
interface.
 protocol-port - Flows are aggregated based on the IP protocol type, source port number, and destination port
number.
 source-destination-prefix - Flows are aggregated based on the source prefix and mask, destination prefix and mask,
source and destination ASs, ingress interface, and egress interface.
 source-prefix - Flows are aggregated based on the source prefix and mask, source AS, and ingress interface.
 as-matrix - Flows are aggregated based on the source and destination AS and ingress and egress interfaces.
autonomous-system-type - Specifies whether the AS information that cflowd sends for analysis is from the originating AS or peer AS.
The parameter is configurable when the Version (version) parameter is set to 5 or 8. The following options are:
 origin (default)
 peer
template-set - Specifies which set of templates cflowd sends to the collector. The parameter is configurable when the Version
(version) parameter is set to 9. The options are:
 basic
 mpls-ip

If the collector is configured to use Version 9, the flow data is sent to the designated collector using one of the pre-
defined templates. The template used is based on the type of flow for which the data was collected (IPv4 or MPLS) and
the configuration of the template-set parameter.
The following table indicates the relationship between these values and the corresponding template used to export the
flow data:
Traffic Type Basic MPLS-IP

IPv4 Basic IPv4 MPLS-IPv4
MPLS Basic MPLS MPLS-IP
Please consult the 7750 SR OS Router Configuration Guide section “Configuring Cflowd Collectors” for a list of
attributes included in each template type.

When the cflowd interface option is configured in the config>router>interface context, the following requirements
must be met to enable traffic sampling on the specific interface:
1. Cflowd must be enabled.
2. At least one cflowd collector must be configured and enabled.
3. The interface>cflowd interface option must be selected.
4. To omit certain types of traffic from being sampled when the interface sampling is enabled, the config>filter>ip-
filter>entry>interface-disable-sample option may be enabled via an ip-filter or ipv6-filter. The filter must be
applied to the service or network interface on which the traffic to be omitted is to ingress the system.
CLI Syntax: config>router>if#

cflowd {acl|interface}
no cflowd
Services Interfaces CLI Syntax: config>service>vprn service-id# interface ip-int-name

cflowd {acl|interface}

CLI Syntax: config>filter>ip-filter>entry#
[no] filter-sample
[no] interface-disable-sample
To enable for filter traffic sampling, the following requirements must be met:
1. Cflowd must be enabled globally.
2. At least one cflowd collector must be configured and enabled.
3. On the IP interface being used, the interface>cflowd acl option must be selected. (See Interface Configuration) For
configuration information, refer to the IP Router Configuration Overview section of the 7750 SR OS Router
Configuration Guide.
4. On the IP filter being used, the entry>filter-sample option must be explicitly enabled for the entries matching the
traffic that should be sampled. The default is no filter-sample. (See Filter Configuration for more information).
5. The filter must be applied to a service or network interface. The service or port must be enabled and operational.
When a filter policy is applied to a service or network interface, sampling can be configured so that traffic matching the
associated IP filter entry is sampled when the IP interface is set to cflowd ACL mode and the filter-sample command
is enabled. If cflowd is neither enabled (no filter-sample) nor set to the cflowd interface mode, then sampling does
not occur.
Since a filter can be applied to more than one interface (when configured with a scope template), the interface-disable-
sample option is intended to enable or disable traffic sampling on an interface-by-interface basis. The command can
be enabled or disabled as needed instead of creating numerous filter versions.
 When the interface-disable-sample command is enabled, traffic matching the associated IP filter entry is
not sampled if the IP interface is set to cflowd ACL mode.

The combination of interface and filter entry configurations determine if and when flow sampling occurs.
The following table displays the expected results when specific features are enabled and disabled:
Interface Settings cflowd [acl | interface] ip-filter entry Expected Results

IP-filter mode acl filter-sampled Traffic matching is sampled at
specified rate.
IP-filter mode acl no filter-sampled No traffic is sampled on this
interface.
IP-filter mode or acl interface-disable-sample Command is ignored. No sampling
cflowd not enabled occurs.
Interface mode interface interface-disable-sample Traffic matching this IP filter entry
is not sampled.
Interface mode interface none All IP traffic ingressing the
interface is subject to sampling.
Interface mode interface filter-sampled Filter level action is ignored. All
traffic ingressing the interface is
subject to sampling.

Above is an example of how a cflowd collector could group together traffic rates based on IP addresses, application
types, ports or protocol. This type of information can be very useful in ways such as networking monitoring, network
troubleshooting and capacity planning.

This command displays basic information regarding the administrative and operational status of cflowd.
The following describes the show cflowd status output fields:
Cflowd Admin Status - The desired administrative state for the cflowd remote collector host.
Cflowd Oper Status - The current operational status of the cflowd remote collector host.
Active Timeout - The maximum amount of time, in minutes, before an active flow is exported. If an individual flow is
active for this amount of time, the flow is exported and a new flow is created.
Inactive Timeout - Inactive timeout in seconds.
Template Retransmit - The time in seconds before template definitions are sent.
Cache Size - The maximum number of active flows to be maintained in the flow cache table.
Overflow - The percentage number of flows to be flushed when the flow cache size has been exceeded.
Sample Rate - The rate at which traffic is sampled and forwarded for cflowd analysis.
 One (1) — All packets are analyzed.
 1000 (default) — Every 1000th packet is analyzed.
Active Flows - The current number of active flows being collected.
Total Pkts Rcvd - The rate at which traffic is sampled and forwarded for cflowd analysis.
Total Pkts Dropped - The total number of packets dropped.
Aggregation Info:
Type - The type of data to be aggregated and sent to the collector.
Status
 enabled — Specifies that the aggregation type is enabled.
 disabled — Specifies that the aggregation type is disabled.

This command displays the administrative and operational status of a data collector configuration.
The following describes the show cflowd collector output fields:
Host Address - The IP address of a remote cflowd collector host to receive the exported cflowd data.
Port - The UDP port number on the remote cflowd collector host to receive the exported cflowd data.
AS Type - The style of AS reporting used in the exported flow data.
 origin — Reflects the endpoints of the AS path which the flow follows.
 peer — Reflects the AS of the previous and next hops for the flow.
Version - Specifies the configured version for the associated collector.
Admin - The desired administrative state for the cflowd remote collector host.
Oper - The current operational status of the cflowd remote collector host.
Recs Sent - The number of cflowd records that have been transmitted to the remote collector host.
Collectors - The total number of collectors using the IP address.

The following describes the show cflowd collector detail output fields:
Address - The IP address of a remote cflowd collector host to receive the exported cflowd data.
Port - The UDP port number on the remote cflowd collector host to receive the exported cflowd data.
Description - A user-provided descriptive string for the cflowd remote collector host.
Version - The version of the flow data sent to the collector.
AS Type - The style of AS reporting used in the exported flow data.
 origin — Reflects the endpoints of the AS path which the flow follows.
 peer — Reflects the AS of the previous and next hops for the flow.
Admin State - The desired administrative state for the cflowd remote collector host.
Oper State - The current operational status of the cflowd remote collector host.
Records Sent - The number of cflowd records that have been transmitted to the remote collector host.
Last Changed - The time when the row entry was last changed.
Last Pkt Sent - The time when the last cflowd packet was sent to the remote collector host.
Aggregation Type - The bit mask which specifies the aggregation scheme(s) used to aggregate multiple individual flows
into an aggregated flow for export to the remote host collector.
 none — No data is exported for the remote collector host.
 raw — Flow data is exported without aggregation in Version 5 format.
 All other aggregation types use Version 8 format to export the flow data to the remote host collector.
Collectors - The total number of collectors using the IP address.

Displays the administrative and operational status of the interfaces with cflowd enabled.
The following describes the show cflowd interface output fields:
Interface - Displays the physical port identifier.

IP Address - Displays the IP address.
Mode - Displays the mode.
Admin - Displays the administrative state of the interface.
Oper - Displays the operational state of the interface.
A:R1# clear cflowd

• Clears the raw and aggregation flow caches which send flow data to the configured collectors. This action triggers all
the flows to be discarded. The cache restarts flow data collection from a fresh state. This command also clears
global collector stats listed in the cflowd show commands.

Accounting statistics typically record service or subscriber usage data for billing or to ensure SLA compliance.
The supported accounting statistics types are:
Service Accounting Statistics - Collected on every queue on every SAP that is linked to an accounting policy. Service
accounting statistics provide queue throughput and drop information and can be used for billing and SLA purposes.
Network Accounting Statistics - Collected on every queue on every SDP or on network ports that are linked to an
accounting policy. Network accounting statistics measure forwarding class queue usage. This information is used to
monitor link utilization and identify network traffic patterns and trends for capacity planning and traffic engineering.
Subscriber Accounting Statistics – Collected on a subscriber profile for residential subscriber instances. Subscriber
accounting statistics are used for billing and SLA purposes.
Application Assurance, or AA, Accounting Statistics - Collected on applications, application groups, and protocols.
Note: SDP statistics collection is supported only on devices that are in chassis mode B, C, or D.

Cflowd can be deployed in a similar method to syslog servers.
As a network operator, certain information that would result in significant data traffic may be stored locally in a region.
Other statistical data that may be of use for real time monitoring in the network management center may be sent
directly to a centralized location.
In some smaller installation, only a single central server may be deployed. A final alternative available is to store
information locally at each site.

Network Elements (NE) collect accounting statistics using an accounting policy and an associated file that are assigned
to a SAP, SDP, network port, or subscriber profile.
An accounting policy specifies an accounting statistics record type, a collection interval, an administrative state, and a
file.
A NE collects accounting statistics based on a specified collection interval and writes the statistics data in XML format to
a file on the NE. After the rollover period, the NE closes and compresses the file. The NE then notifies the 5620 SAM
that a new file is ready for processing.

config>log
file-id log-file-id
location cflash-id [backup-cflash-id]
rollover minutes [retention hours]
Rollover Interval – Defined in minutes and determines how long a file will be used before it is closed and a new log file is
created.
Retention Interval – Determines how long a file will be stored on the CF before it is deleted.
A file ID can only be assigned to one accounting log.
The 7750 SR-Series creates two directories on the compact flash to store the files. The following output displays a
directory named act-collect that holds accounting files that are open and actively collect statistics. The directory
named act stores the files that have been closed and are awaiting retrieval.
A:R1>file cf2:\# dir act*

12/19/2006 06:08a <DIR> act-collect
12/19/2006 06:08a <DIR> act
A:R1>file cf1:\act-collect\ # dir

Directory of cf1:\act-collect#
12/23/2006 01:46a <DIR> .
12/23/2006 12:47a <DIR> ..
12/23/2006 01:46a 112 act1111-20031223-014658.xml.gz
12/23/2006 01:38a 197 act1212-20031223-013800.xml.gz
Accounting files always have the prefix act followed by the accounting policy ID, log ID, and timestamp.

When creating accounting policies, one service accounting policy and one network accounting policy can be defined as
default. If statistics collection is enabled on a SAP or network port and no accounting policy is applied, then the
respective default policy is used. If no default policy is defined, then no statistics are collected unless a specifically
defined accounting policy is applied.
Predefined Record-Names:
aa-app-group|aa-application|aa-protocol|aa-subscriber-application|aa-subscriber-protocol|combined-ldp-lsp-
egress|combined-mpls-lsp-egress|combined-mpls-lsp-ingress|combined-network-ing-egr-octets|combined-queue-
group|combined-sdp-ingress-egress|combined-service-ing-egr-octets|combined-service-ingress|compact-service-
ingress-octets|complete-sdp-ingress-egress|complete-service-ingress-egress|complete-subscriber-ingress-
egress|custom-record-aa-sub|custom-record-service|custom-record-subscriber|network-egress-octets|network-
egress-packets|network-ingress-octets|network-ingress-packets|queue-group-octets|queue-group-
packets|saa|service-egress-octets|service-egress-packets|service-ingress-octets|service-ingress-packets|video
Each accounting record name consists of one or more sub-records which in turn consists of multiple fields.
Below is a description of one accounting record:
Sub-record Field Field Description
Service-ingress-Octets sio svc SvcID
sap SapID
qid QueueID
hoo OfferedHiPrioOctets
hod DroppedHiPrioOctets
loo LowOctetsOffered
lod LowOctetsDropped
uco UncoloredOctetsOffered
iof InProfileOctetsForwarded
oof OutOfProfileOctetsForwarded

Custom Accounting Records (custom-record)
You can customize the record in an accounting policy to only contain the accounting data you require. This reduces the
statistics-collection processing load and the volume of data that the collection generates. An accounting policy can
contain one custom record.

The collect-stats command enables accounting and statistical data collection. When applying accounting policies, the
data, by default, is collected in the appropriate records and written to the designated billing file.
When the no collect-stats command is issued, the statistics are still accumulated by the IOM cards. However, the CPU
will not obtain the results and will not write them to the billing file. If a subsequent collect-stats command is issued, the
counters written to the billing file will include all the traffic while the no collect-stats command was in effect.

Syntax: accounting-policy policy-id
no accounting-policy
This command assigns the accounting policy to a SAP, SDP, or pseudowire template. An accounting policy must be
defined before it can be associated. If the policy-id does not exist, an error message is generated.
A maximum of one accounting policy can be associated at one time. Accounting policies are configured in the
config>log context.
Pseudowire Template
The pw-template is defined under the top level service command (config>service# pw-template) and specifies whether
to use an automatically generated SDP or manually configured SDP. It also provides the set of parameters required for
establishing the pseudowire (SDP binding).

Syntax: accounting-policy policy-id
no accounting-policy
Context: config>port>ethernet>access>egr>qgrp
config>port>ethernet>access>ing>qgrp
config>port>ethernet>network>egr>qgrp
config>port>ethernet>network
config>port>sonet-sdh>path>network
config>port>tdm>ds1>network
config>port>tdm>ds3>network
config>port>tdm>e1>network
config>port>tdm>e3>network

Module 2 | 70
© 2016 Nokia
The Nokia 5620 SAM can provide a tabular and graphical view of accounting statistics. This is very useful when spotting
traffic patterns (as seen in the slide above). The screen captured above shows an example of increasing traffic followed
by a complete stoppage.
Note:
The results shown above are absolute values.

Please refer to the lab guide for further instructions.

Service Mirroring Overview
When troubleshooting complex operational problems, customer packets can be examined as they traverse the network.
One way to accomplish this is to establish an overlay of network analyzers at multiple PoPs along with skilled technicians
to operate them and decode the date provided. This method of traffic mirroring often requires setting up complex
filters in multiple switches and/or routers. These, at best, are only able to mirror from one port to another on the same
device.
Nokia’s 7X50 routers support service-based mirroring. While some Layer 3 switches and routers can mirror on a per-
port basis within the device, Nokia 7X50 routers can mirror on an n-to-1 unidirectional service basis and re-encapsulate
the mirrored data for transport through the core network to another location, using either IP or MPLS tunneling as
required.
Original packets are forwarded while a copy is sent out the mirrored port to the mirroring (destination) port. Service
mirroring allows an operator to see the actual traffic on a customer’s service with a ‘sniffer’ sitting in a central
location. In many cases, this reduces the need for a separate, costly overlay sniffer network.
The mirrored frame size that is to be transmitted to the mirror destination can be explicitly configured by using slicing
features. This enables mirroring only the parts needed for analysis. For example, only the headers can be copied for
analysis, protecting the integrity and security of customer data, or conversely, copying the full packet including
customer data.
Service mirroring is supported on any interface type as well as mixed interface types. For example, a service that uses
only Ethernet service interfaces can be mirrored to a SONET/SDH network port, transported across the core network,
and delivered on either Ethernet or SONET/SDH egress ports at the location where service analysis is performed. The
packet traffic is uninterrupted and packets flow normally through the mirrored port.

Local and Remote Mirroring
Mirrored frames can be copied and sent to a specific local destination or service on the 7X50 router (local mirroring) or
copies can be encapsulated and sent to a different 7X50 router (remote mirroring). This functionality allows network
operators to centralize not only network analyzer (sniffer) resources but also the technical staff who operates them.
The 7X50 allows multiple concurrent mirroring sessions so traffic from more than one ingress mirror source can be
mirrored to the same or different egress mirror destinations.
Remote mirroring uses a service distribution path (SDP) which acts as a logical way of directing traffic from one SR-
Series router to another through a uni-directional (one-way) service tunnel. The SDP terminates at the far-end 7X50
which directs packets to the correct destination on that device.
The SDP configuration from the mirrored device to a far-end 7X50 requires a return path SDP from the far-end 7X50
back to the mirrored router. Each device must have an SDP defined for every remote router for which it wants to
provide mirroring services. SDPs must be created first before services can be configured.

Mirroring performance
Replication of mirrored packets can typically affect performance and should be used carefully. Nokia 7X50 routers
minimize the impact of mirroring on performance by taking advantage of its distributed Flexible Fast Path technology.
Flexible Fast Path forwarding allows efficient mirror service scaling and, at the same time, allows a large amount of data
to be mirrored with minimal performance impact. When a mirror destination is configured, the packet slice option can
truncate mirrored packets to the destination which minimizes replication and tunneling overhead. The mirroring
architecture also supports mirror rate limiting both at the ingress and egress Flexible Fast Path NPA. This rate limiting is
accomplished through a shaping queue and is settable according to the maximum amount of mirroring desired.
Mirroring can be performed based on the following criteria:
 Port
 SAP
 MAC filter
 IP filter
 Ingress label

Mirror Destination — Sets up a service which allows the mirrored packets to be directed locally or over the core of the
network and have a far end 7X50 decode the mirror encapsulation. The service ID must match in the mirror-destination
and the mirror-source context.
SAP (Mirror Destination) — Creates a service access point (SAP) which defines the port and encapsulation parameters
to which the mirrored source packets are sent. The sniffer is physically connected to this port.
SDP — For remote mirrored service. Binds an existing (mirror) service distribution path (SDP) to the mirror destination
service ID to transport the source mirrored traffic to the destination.
Remote Source — For remote mirrored services. Specifies the remote (source) allowed to mirror traffic to the device
for mirror service egress.
Mirror Source — Configures packet mirroring match criteria for a mirror destination service. The same mirror
destination service ID and the mirror source service ID must be configured.
Port — A packet mirroring option which defines ingress and/or egress traffic monitoring by port.
SAP (Mirror Source) — A packet mirroring option which defines ingress and/or egress traffic monitoring by SAP defined
by the port-id:encap-val or portid.channelid: encap-val.
IP Filter — A packet mirroring option which specifies that packets matching the IP filter are mirrored to a mirror
destination.
MAC Filter — A packet mirroring option which specifies that packets matching the MAC filter are mirrored to a mirror
destination.
Ingress Label — A packet mirroring option which defines packets with a specific MPLS label to a mirror destination.
NOTE: To stop a mirror source, use the following command:

debug no mirror-source <id>
The “no debug” command does not impact mirror-sources

Module 2 | 77
© 2016 Nokia
Module 2 | 78
© 2016 Nokia
CLI Syntax: config>mirror mirror-dest service-id [type {ether|frame-relay|
ppp|ip-only|atm-sdu|satop-e1|satop-t1|cesopsn|cesopsn-cas}] [create]
description string
fc fc-name
sap sap-id [create]
slice-size bytes
no shutdown
For a remote mirror configuration, the slice-size parameter is set on the source router. This allows conservation of
mirroring resources by limiting the size of the stream of packet through the 7750 SR-Series and the core network. For
example, if the value of 256 bytes is defined, up to the first 256 bytes of the frame are transmitted to the mirror
destination. The original frame is not affected by the truncation.
Note that slice-size is not supported by CEM encap-types or IP-mirroring.
The transmission of a sliced or non-sliced frame is also dependent on the mirror destination SDP path MTU and/or the
mirror destination SAP physical MTU. Packets that require a larger MTU than the mirroring destination supports are
discarded if the defined slice size does not truncate the packet to an acceptable size.
Parameters bytes — The number of bytes to which mirrored frames will be truncated, expressed as a decimal integer.
Values 128 — 9216


In order to verify that a service is operational, a set of in-band, packet-based Operation, Administration, and
Maintenance (OAM) tools is required with the ability to test each of the individual packet operations.
For in-band testing, the OAM packets closely resemble customer packets to effectively test the customer's forwarding
path. However, they are distinguishable from customer packets and are thus kept within the service provider's network
and not forwarded to the customer.
The suite of OAM diagnostics supplement the basic IP ping and traceroute operations with diagnostics specialized for
the different levels in the service delivery model. There are diagnostics for MPLS LSPs, SDPs, services and VPLS MACs
within a service.

clear>saa context
 saa [test-name [owner test-owner]]
 Use this command to clear SAA results
 saa test configuration does not get deleted.

Note:
A probe is one instance of the test.
jitter-event - Specifies that the calculated jitter value, at the termination of an SAA test probe, is evaluated against the
configured rising and falling jitter thresholds.
latency-event - Specifies that the calculated latency event value, at the termination of an SAA test probe, is evaluated
against the configured rising and falling latency event thresholds.
loss-event - Specifies that the calculated loss event value, at the termination of an SAA test run, is evaluated against
the configured rising and falling loss event thresholds.
trap-gen - This command enables the context to configure trap generation for the SAA test.
probe-fail-enable - This command enables the generation of an SNMP trap when probe-fail-threshold consecutive
probes fail during the execution of an SAA ping test.
test-completion-enable - This command enables the generation of a trap when an SAA test completes.
test-fail-enable - This command enables the generation of a trap when a test fails. In the case of a ping test, the test is
considered a failure (for the purpose of trap generation) if the number of failed probes is at least the value of the test-
fail-threshold parameter.
type - This command creates the context to provide a test type for the named test. Only a single test type can be
configured.
continuous - Specifies whether the SAA test is continuous.
When you configure a test, use the config>saa>test>continuous command to make the test run continuously. Use the
no continuous command to disable continuous testing and shutdown to disable the test completely. Once you have
configured a test as continuous, you cannot start or stop it by using the saa test-name [owner test-owner] {start | stop}
[no-accounting] command.

test-name — Name of an SAA test. The test name must already be configured in the config>saa>test context.
owner test-owner — Specifies the owner of an SAA operation up to 32 characters in length.
Values:
If a test-owner value is not specified, tests created by the CLI will have a default owner “TiMOS CLI”.
start — This keyword starts the test. A test cannot be started if the same test is still running. A test cannot be started if
it is in a shut-down or continuous state. An error message and log event will be generated to indicate a failed attempt to
start an SAA test run.
stop — This keyword stops a test in progress. A test cannot be stopped if it is not in progress or is in a continuous
state. A log message will be generated to indicate that an SAA test run has been aborted.
no-accounting — This parameter disables the recording results in the accounting policy. When specifying no-
accounting, the MIB record produced at the end of the test will not be added to the accounting file. It will however use
up one of the three MIB rows available for the accounting module to be collected.
Syntax: saa [test-name] [owner test-owner]
Use this command to display information about the SAA test. If no specific test is specified, a summary of all configured
tests will be displayed. If a specific test is specified, then detailed test results for that test will be displayed for the last
three occurrences that the test has been executed or since the last time the counters were reset via a system reboot or
clear command.
For each defined SAA test, the router will keep the oldest 3 results with a max of 50 result entries.
If a test-owner value is not specified, tests created by the CLI will have a default owner “TiMOS CLI”.

Looking further into the SAA results file (above) is a listing of the occurrences of a threshold value including the value
that was measured. A time stamp is also provided.

The final section of the report shows a summary of the various tests performed through SAA.

Much information can be gathered from the log events above:
•OAM test “ATS_sdp_ping” was run and completed.
•A falling rtJitter (round trip Jitter) threshold, which is set at 5000, was crossed with a value of 509.
•A falling rtLoss (round trip Loss) threshold, which is set at 1, was crossed with a value of 0.
•A rising rtLatency (round trip Latency) threshold, which is set at 1000, was crossed with a value of 3531.


Element Management Systems (EMSs) are responsible for the configuration management of one or more products.
They also include features to view an administrative settings and make changes.
Network Management Systems (NMSs) are responsible for monitoring the health of the elements in a node and the
services that run on top of them in addition to gathering logs and notifying operators.
Control Plane Managers (CPAMs) are responsible for monitoring the control plane processes on an element, allowing
the operator to assess the status of individual tunnels, routing protocols, and so on.

SAM is both an EMS (element manager) and a NM (Network Manager).
As an EMS, SAM offers the ability to configure nodal elements and services from end to end network perspective.
As an NM, SAM provides topology maps, event correlation, OAM test facilities, and other monitoring and reporting
capabilities.
The advantage of SAM is in its awareness of the entire discovered network topology as opposed to the CLI, which
provides a very detailed view of the local router.

Network Planning - Planning activities are optimized with real-time topology and strong linkages between services and
infrastructure layers in the 5620 SAM GUI and 5620 SAM-O OSS interfaces.
Network Operations - Real-time topology and multi-layer highlighting allows you to rapidly assess the state of services,
tunnels, and routing on the IGP and IP/MPLS maps.
Network Troubleshooting - Historical OAM trace, SPF and RSVP path, and checkpoints allow you to rapidly detect and
resolve service level issues whose root cause is in IP or MPLS layers.
Network Restoration - Checkpoints and real-time views of IP/MPLS and service and tunnel infrastructure allow you to
restore and plan networks.
Proactive Assurance - 5650 CPAM alarms, network route and tunnel inspection lists, validation functions, checkpoints,
and multi-layer views allow you to detect routing faults.

Module 2 | 92
© 2016 Nokia
Module 2 | 93
© 2016 Nokia
Module 2 | 94
© 2016 Nokia
Over the years, we have witnessed a significant increase in network based attacks such as DoS, Worms and Viruses,
L2/L3 attacks, and OS vulnerabilities.
Due to the influx of attacks, carriers must focus on the identification and elimination of security threats to ensure
that SLAs can be met.
The Nokia SR-OS offers a broad range of features to ensure that security threats are mitigated in a carrier network
and to protect itself from DOS attacks.

Peakflow SP can utilize cflowd v5, v9 to build its traffic models, which can then be used to visualize and report on
network traffic behaviors as well as pro-actively detect abnormal traffic that may impact services.
Misuse Detection
Threats are detected against individual /32 hosts and cover the most common attack types. Misuse anomalies are
generated for ICMP, TCP NULL, TCP SYN, TCP RST, IP NULL, IP Fragment, IP Private Address Space, UDP, DNS and Total
Traffic flood attacks.
Profiled Detection
Threats are detected against Managed Objects configured within the Peakflow SP system. Profiled Anomalies are
generated when traffic for a given Managed Object deviates from normal traffic levels. Peakflow SP builds ‘baselines’
for the Managed Objects configured within the SP system.
These ‘baselines’ represent:

 The amount of TCP, UDP, AH, GRE, ESP, ICMP and Multi-Protocol traffic seen at each router for each object;
 Baseline data which is stored for in / out and in bps and pps; and
 The total amount of traffic in / out of each monitored router interface for each object.
Fingerprint Detection
Threats are detected when traffic exceeds user defined bps / pps thresholds for either a user defined detection
‘signature’ based on Layer 3 / 4 traffic parameters or an Arbor Active Threat Feed (ATF) policy.
ATF - Provided to all Peakflow SP deployments (will periodically download signature updates from an Arbor server over
SSL) and contains Layer 3 / 4 signature information for identifying both malicious and non-malicious network behaviors
which may be of interest to network operators (for example, host connections to known Botnet CnC servers, traffic to
Dark IP addresses, worm infection, and so on).

Peakflow TMS includes the following range of counter-measures:
•Global exception list
•Per mitigation black/white list
•Zombie Removal
•TCP SYN, HTTP, and DNS authentication
•Payload RegExp
•Baseline enforcement
•FCAP rate limiting
•Idle connection reset
•SIP counter measures
 Malformed SIP filtering
 SIP request filtering
•HTTP counter-measures
 HTTP request / Object limiting
 Malformed HTTP
 HTTP header RegExp
•DNS counter-measures
 Shaping DNS field RegExp counter-measures
 DNS rate limiting counter-measure
 DNS cache poison detection and counter-measure
 Malformed DNS filtering

Filter policies, also referred to as Access Control Lists (ACLs), are templates applied to services or network ports to
control network traffic into (ingress) or out of (egress) a service access port (SAP) or network port.
Service and Network Port-based Filtering

IP filter policies specify either a forward or a drop action for packets based on information specified in the match
criteria. You can create up to 2047 IP, 2047 IPv6, and 2047 MAC filter policies per node although your network can
handle up to 65,535 policies including policies pushed out globally or to specific nodes. Within each filter policy, you can
create up to 16,384 entries.
Filter entry matching criteria can be as general or specific as required; however, all conditions in the entry must be met
in order for the packet to be considered a match and the specified entry action performed. The process stops when the
first complete match is found and executes the action defined in the entry - either to drop or forward packets that
match the criteria.

A filter policy compares the match criteria specified within a filter entry to packets coming through the system in the
order the entries are numbered in the policy. When a packet matches all the parameters specified in the entry, the
system takes the specified action to either drop or forward the packet. If a packet does not match the entry
parameters, the packet continues through the filter process and is compared to the next filter entry and so on. If the
packet does not match any of the entries, the system executes the default action specified in the filter policy.

Module 2 | 100
© 2016 Nokia
Matching criteria to drop or forward IP traffic include:
 Source IP address and Mask — Source IP address and mask values can be entered as search criteria. Address
ranges are configured by specifying mask values - the 32-bit combination used to describe the address portion
which refers to the subnet and which portion refers to the host. The mask length is expressed as an integer (range
0 to 32). The IP Version 6 (IPv6) addressing scheme consists of 128 bits expressed in compressed representation of
IPv6 addresses (RFC 1924).
 Destination IP address and Mask — Destination IP address and mask values can be entered as search criteria.
 Protocol — Entering a protocol (such as TCP, UDP, and so on) allows the filter to search for the protocol specified in
the field.
 Protocol (IPv6) — Entering a next header allows the filter to match the first next header following the IPv6 header.
 Source port/range — Entering the source port number or port range allows the filter to search for matching TCP or
UDP port and range values.
 Destination port/range — Entering the destination port number or port range allows the filter to search for
matching TCP or UDP values.
 DSCP marking
 ICMP code — Entering an ICMP code allows the filter to search for matching ICMP code in the ICMP header.
 ICMP type — Entering an ICMP type allows the filter to search for matching ICMP types in the ICMP header.
 Fragmentation (IPv4 only) — Enables fragmentation matching. A match occurs if packets have either the MF (More
Fragment) bit set or the Fragment Offset field of the IP header set to a non-zero value.
 Option Value — Entering an option value enables the first filter to search for a specific IP option.
 TCP-ACK/SYN flags — Entering a TCP-SYN/TCP-ACK flag allows the filter to search for TCP flags specified in these
fields.

Matching criteria to drop or forward IP traffic include:
 Source MAC address and Mask — Entering the source MAC address range allows the filter to search for a matching
source MAC address and/or range. Enter the source MAC address and mask in the form of xx:xx:xx:xx:xx:xx or xx-xx-
xx-xx-xx-xx; for example, 00:dc:98:1d:00:00.
 Destination MAC address and Mask — Entering the destination MAC address range allows the filter to search for a
matching destination MAC address and/or range. Enter the destination MAC address and mask in the form of
xx:xx:xx:xx:xx:xx or xx-xx-xx-xx-xx-xx; for example, 02:dc:98:1d:00:01.
 Dot1p and Mask — Entering an IEEE 802.1p value or range allows the filter to search for a matching 802.1p frame.
The Dot1p and mask accept decimal, hex, or binary in the range of 0 to 7.
 Ethertype — Entering an Ethernet type II Ethertype value to be used as a filter match criterion. The Ethernet type
field is a two-byte field used to identify the protocol carried by the Ethernet frame. The Ethertype accepts decimal,
hex, or binary in the range of 1536 to 65535.
 IEEE 802.2 LLC SSAP — Specifying an Ethernet 802.2 LLC SSAP value allows the filter to match a source access
point on the network node designated in the source field of a packet. The SSAP and mask accept decimal, hex, and
binary in the range of 0 to 255.
 IEEE 802.2 LLC DSAP — Specifying an Ethernet 802.2 LLC DSAP value allows the filter to match a destination
access point on the network node designated in the destination field of a packet. The DSAP and mask accept
decimal, hex, and binary in the range of 0 to 255.
 IEEE 802.3 LLC SNAP PID — Specifying an Ethernet IEEE 802.3 LLC SNAP PID allows the filter to match the two-
byte protocol ID that follows the three-byte OUI field. The DSAP and mask accept decimal and hex in the range of 0
to 65535.
 IEEE 802.3 LLC SNAP OUI — Specifying an Ethernet IEEE 802.3 LLC SNAP OUI allows the filter to match the three-
byte OUI field.
 ISID — Specify an ISID match condition.

The following information describes filter implementation caveats:
• Creating a filter policy is optional.
• Associating a service with a filter policy is optional.
• When a filter policy is configured, it must be defined as either having an exclusive scope for one-time use or a
template scope indicating that the filter can be applied to multiple SAPs.
• A specific filter must be explicitly associated with a specific service in order for packets to be matched.
• Each filter policy must consist of at least one filter entry. Each entry represents a collection of filter match criteria.
When packets enter the ingress or egress ports, packets are compared to the criteria specified within the entry (or
entries).
• When you configure a large (complex) filter, it may take a few seconds to load the filter policy configuration and to be
instantiated.
• The action keyword must be entered for the entry to be active. Any filter entry without the action keyword will be
considered incomplete and inactive.

In the above configuration, RTR-A is configured to deny all traffic sourced from IP subnet 1.2.3.4, using TCP at the
transport layer and specifically using application ports ranging from 265 to 1024. All other traffic received on the
interface will be allowed to enter since the default action is set to forward.

In the above sample configuration, IP filter 20 is created. Within the filter, the default action is to forward IP packets
that do not meet the more explicit “match” settings. If those criteria are met, then the packet will be dropped. Once
the filter is created, it must be associated to the ingress or egress of an interface.

The next-hop for VPLS policy enables a filter policy that has a forward next-hop SAP or SDP action specified to be used
in a VPLS service to direct traffic with specific match criteria to a SAP or SDP. This allows traffic destined for the same
gateway to be split and forwarded differently based on the policy.
Policy routing is a popular tool used to direct traffic in Layer 3 networks. As Layer 2 VPNs become more popular,
especially in network aggregation, policy forwarding is required. Many providers are using methods such as DPI servers,
transparent firewalls, or Intrusion Detection/Prevention Systems (IDS/IPS) for policy routing. Since these devices are
limited by bandwidth, providers want to limit the traffic that is forwarded through them. To accomplish this, a
mechanism is required to direct some of the traffic coming from a SAP to the DPI without learning and to direct other
traffic coming from the same SAP directly to the gateway uplink based on learning.
VPLS policy-based forwarding allows the provider to create a filter that will forward packets to a specific SAP or SDP.
The packets are then forwarded to the destination SAP regardless of a learned destination or lack thereof. The SAP can
either terminate a Layer 2 firewall or Deep Packet Inspection (DPI) directly or be configured to be part of a cross-
connect bridge into another service. This is useful when running the DPI remotely using a VPWS.
If an SDP is used, the provider can terminate it in a remote VPLS or VPWS service where the firewall is connected. The
filter can be configured under a SAP or SDP in a VPLS service. All packets (unicast, multicast, broadcast, and unknown)
can be delivered to the destination SAP or SDP. The filter may be associated with SAPs or SDPs belonging to a VPLS
service only if all actions in the policy forward to SAPs or SDPs within the context of that VPLS.

Module 2 | 107
© 2016 Nokia
The configuration shown in this slide causes all traffic with the source IP 192.168.1.1 and destination IP 192.168.1.2 to be forwarded out
SAP 1/1/4:100 when ingressing on SAP 1/1/3:1, regardless of the FDB.
A filter must be created for traffic in the reverse direction which will ingress on the SDP.
Syntax action [drop]
action forward [next-hop {ip-address | indirect ip-address | interface ip-int-name | }]
action forward [redirect-policy policy-name]
action forward [sap sap-id | sdp sdp-id]
action http-redirect url
no action
Context config>filter>ip-filter>entry
This command specifies to match packets with a specific IP option (or a range of IP options) in the first option of the IP header as an IP
filter match criterion. The action keyword must be entered and a keyword must be specified in order for the entry to be active. Note that
action forward next-hop cannot be applied to multicast traffic.
Multiple action statements entered will overwrite previous actions parameters when defined.
The no form of the command removes the specified action statement. The filter entry is considered incomplete and hence rendered
inactive without the action keyword.
Default
No action is specified thus rendering the entry inactive.
Parameters
drop — Specifies packets matching the entry criteria to be dropped.
forward — Specifies packets matching the entry criteria to be forwarded.
If neither drop nor forward is specified, the filter action is No-Opand and is rendered inactive.
sap sap-id — Specifies the physical port identifier portion of the SAP definition. Only Ethernet SAPs are supported (including QinQ,
BCP, bridged Ethernet in Frame Relay, or ATM).
• Values sap-id null [port-id | bundle-id | lag-id | aps-id]
dot1q [port-id | bundle-id | lag-id | aps-id]:qtag1
qinq [port-id | bundle-id | lag-id]:qtag1.qtag2
sdp sdp-id — The SDP identifier.
Values 1 to 17407

The configuration shown in this slide causes all traffic with the source IP 192.168.1.2 and destination IP 192.168.1.1 to be
forwarded out SAP 1/1/5:100 when ingressing on SDP 100:1, regardless of the FDB.

SAPs 1/1/4:100 and 1/1/5:100 only accept traffic going between 192.168.1.1 and 192.168.1.2

This filter prevents traffic from being looped during FDB learning.
Note: Disabling the learning function on SAP 1/1/5:100 and SAP 1/1/4:100 is an important step in this example. SAP
1/1/5:100 and SAP 1/1/4:100 see traffic with the source MAC from IP addresses 192.168.1.1 and 192.168.1.2. Disabling
the learning function prevents incorrect information from being populated in the FDB.

Module 2 | 112
© 2016 Nokia
Nokia ‘protocol’ signatures
• Provided as router independent SW upgrades for the MS-ISA card.
• For example: bittorrent, skype, msexchange, pop3, sip, and so on.
Applications
• Operator configured for maximum flexibility.
• Specific match criteria such as signature, traffic direction, server-subnet, TCP/UDP ports , and
string matches.
• For example: Gmail, Database, WebTraining, and so on.
Application Groups
• Multiple applications can be grouped together.

Module 2 | 114
© 2016 Nokia
Resource Elimination attacks are aimed at a specific target such as a network device or server. The goal is to prevent
legitimate users from accessing the Internet resource.
Resource Starvation attacks attempt to consume bandwidth or resources such as session handling capabilities.
Regardless of the DoS attack used, it exists as a flood of packets such as SYNs, SYN-FIN, ICMP, TCP, or UDP.
Amplifier sites will typically consist of zombie computers on xDSL, cable, or dialup connections (both residential and
business systems).
DoS will affect customers and the network in several ways such as:
• The inability to gain access to the network.
• Response times producing a less than satisfactory user experience.

Module 2 | 116
© 2016 Nokia
Surgical Mitigation
• Baseline enforcement
• Black and white lists
• Payload Filtering
• HTTP, DNS, VOIP specific
• Rate limiting
• Network “Fingerprints”
Arbor
• Set baseline service-level characteristics
• Identify potential attacks and send alarms
• Fine tune mitigation tools
• View what is being blocked

Module 2 | 118
© 2016 Nokia
Creating and implementing management access filters is optional. Management access filters control all traffic entering
the CPM, including all routing protocols. They apply to packets from all ports. The filters can be used to restrict
management of the 7750 SR router by other nodes outside either specific (sub)networks or through designated ports.
By default, there are no filters associated with security options. The management access filter and entries must be
explicitly created on each router. These filters apply to the management Ethernet port.
The 7750 SR OS implementation exits the filter when the first match is found and executes the actions according to the
specified action. For this reason, entries must be sequenced correctly from most to least explicit.
An entry may not have any match criteria defined (in which case, everything matches) but must have at least the
keyword action to be considered complete. Entries without the action keyword are considered incomplete and will be
rendered inactive.

Use the following CLI commands to configure a management access filter.
CLI Syntax: config>system

security
management-access-filter {ip-filter|ipv6-filter|mac-filter}
default-action {permit|deny|deny-host-unreachable}
renum old-entry-number new-entry-number
no shutdown
entry entry-id
src-port {port-id cpm|laglag-id}
src-ip {ip-prefix/mask | ip-prefix netmask}
protocol protocol-id
router {router-name |service-id}
dst-port port [mask]
action {permit|deny|deny-host-unreachable}
log

CPM filters and queues control all traffic going in to the CPM, including all routing protocols. They apply to packets from
all network and access ports but not to packets from a management Ethernet port. CPM packet filtering and queuing is
performed by a network processor hardware, using no resources on the main CPUs. CPM filters and queues are not
configurable on the one-slot chassis.

Use the following CLI commands to configure a CPM filter.
CLI Syntax: config>system>security
cpm-filter
default-action {accept | drop}
ip-filter
entry entry-id
action {accept | drop | queue queue-id}
log log-id
match [protocol protocol-id]
dscp dscp-name
dst-ip {ip-address/mask|ip-address netmask}
dst-port [tcp/udp port-number] [mask>
fragment {true|false}
icmp-code icmp-code
icmp-type icmp-type
ip-option ip-option-value [ip-option-mask]
multiple-option {true|false}
option-present {true|false}
router [router-name |service-id]
src-ip {ip-address/mask|ip-address netmask}
src-port src-port-number [mask]
tcp-ack {true|false}
tcp-syn {true|false}
renum old-entry-id new-entry-id
no shutdown

Control Processor Module Queuing (CPMQ) implements separate hardware-based CPM queues which are allocated on a
per-peer basis. CPMQ allocates a separate queue for each LDP peer and ensures that each queue is served in a round
robin fashion. This mechanism guarantees fair and "non-blocking" access to shared CPM resources across all peers. This
would ensure, for example, that an LDP-based DOS attack from a given peer would be mitigated and
compartmentalized so that the overwhelming control traffic sent by that specific peer would not use all the CPM
resources.
CPM filters can be used in conjunction with queues to mitigate the effects of Denial-of-Service threats. You must
determine the CIR (Committed Information Rate), PIR (Peak Information Rate), CBS (Committed Burst Size), and MBS
(Maximum Burst Size) before the CPM-queue can be configured.
The following CPM traffic management features are supported:

• Traffic classification using CPM filters:
 Packets going to the CPM are first classified by the IOM into forwarding classes (FCs) before CPM
hardware sees them.
 CPM filters can be used to further classify the packets using Layer 3/Layer 4 information (for example,
destination IP, DSCP value, TCP SYN/ACK, and so on).
• Queue allocation:
 Allocable queues: 33 — 2000
 Queues 1 — 8 are default queues. They cannot be modified or deleted.
 Queues 9 — 32 are reserved for future use.
 Queues 2001 — 8000 are used for per-peer queuing.
• Specifying PIR, CIR, CBS, and MBS for the queues.
• The queuing scheduler works in such a way that the queues within the CIR will be scheduled first in a round robin
fashion, followed by the queues above the CIR.
• Unclassified traffic is directed to default queues (1 — 8).

Use the following CLI commands to configure a CPM queue. The first queue available is 33.
CLI Syntax: config>system>security# cpm-queue
queue queue-id
cbs cbs
mbs mbs
rate rate [cir cir]


Module 2 | 126
© 2016 Nokia
Module 2 | 127
© 2016 Nokia
Module 2 | 128
© 2016 Nokia
Module 2 | 129
© 2016 Nokia
Module 2 | 130
© 2016 Nokia
Module 3 | 1
© 2016 Nokia
Module 3 | 2
Module 3 - 2
Module 3 | 3
Module 3 - 3
Module 3 | 4
© 2016 Nokia
Having a thorough knowledge of your network and how it functions under normal conditions is essential if you want to
be efficient in troubleshooting problems as it allows for rapid and easy identification that a fault exists in your network.
It is essential that a sound baseline of your network and services be established and rigorously maintained since a
network is never a static environment, for example: Customer churn, new service introductions, new service points of
presence are added, links fail, etc…

Troubleshooting Process
Troubleshooting and problem solving is basically the same thing. In either case, there is the acknowledgment that
something in the network, a component of the network or a service within the network, is not operating within expected
operating parameters. The problem can result in a total or catastrophic failure in the network, or the problem can
manifest itself intermittently, or then again, the problem may have resulted in degradation of how the service is
performing.
There are many accepted methodologies for troubleshooting a problem and they all must naturally start with the
identification that a problem exists. This implies a certain level of understanding of the designed state and behavior of
a network and the services that are using that network as well as an identification of a symptom that the desired
behavior is no longer there. This identification can come in the form of an alarm received from a network component,
through the analysis of network capacity and performance data or even from a call from a customer reporting a
problem with their service.

Maintaining a detailed history of problems, their symptoms, how the root cause was identified and how the problem was
resolved is a powerful troubleshooting tool. The more detailed the documented symptoms, the easier it is to identify
the root cause of the problem. It is important to remember that in many cases the individual or the team that is
recording the problem symptoms may not be the same people who will be finding the root cause and resolving the
problem, therefore close attention to detail in recording the problem symptoms is crucial to rapid problem resolution.
Check the following for possible information regarding the problem(s);

• Alarm files
• Error logs
• Network statistics
• Network analyzer traces
• Tech Support Files
• Serial line traces
• Stack dumps
• Output of various show commands in CLI (current configuration)
• Accounting logs
• Customer trouble reports
Some questions to answer and conditions to investigate are:

 Is it an intermittent problem, or is the problem static in nature?
 If the problem is intermittent, how often has it happened, is there a pattern?
 What alarms or network events are associated with the problem?
 Can you identify any congestion in routers or network links?
 Identify and record any changes that have taken place since the network was last functioning properly.

Module 3 | 8
© 2016 Nokia
Prior to changing any attribute in the router you should execute a “admin tech-support <file>” command.
NOTE: This command should always be executed prior to any additional intrusive troubleshooting. This would include
shutting down an interface, resetting a routing protocol or any other changes that could alter the status of the router
as it is not operating correctly.
NOTE: This command should only be used with authorized direction from the Nokia Technical Assistance Center (TAC).

It is entirely possible to fix the problem by trying a variety of actions, such as resetting a network link, rebooting a
router, reseating an IO module, in the general case the intended solution will arrive more rapidly by following a
systematic approach to troubleshooting. A systematic approach to isolating and identifying the root cause of the
problem includes the following elements:
 Once the symptoms have been identified and thoroughly documented, first try to identify if they have anything
in common. Focus on the common issues first and work out from there.
 Alarms available through the 5620 SAM contain vendor-specific and X.733 standardized probable cause that
can be very useful in identifying the root cause.
 Statistics on alarms available from the 5620 SAM tell you how often an alarm has been raised based on specified
scenarios, which can be helpful in identifying the root cause of a problem.
 If the symptoms are present in different areas of the network try to identify what is common across these
areas.
 Work on one problem at a time, fix that problem, then move on to the next.
 Divide the problem space into natural segments and try to isolate the problem to one of the segments. One
way of segmenting the network is:
• LAN switching (edge access).
• LAN routing (distribution, core).
• Metropolitan-area networks.
• WAN (national backbone).
• Partner services (extranet).
• Remote access services.
 Try to determine the precise network state that existed before the problem appeared.
 Identify which specific functions are not working properly and focus on those.
 Extrapolate from the network alarms and network events what conditions could result in the observed
symptoms. Test for these to see if the problem can be reproduced.

1. More times than not the problem exists at the physical layer and not in the logic of the routers. As the problem is
more defined, use divide and conquer.
2. Ensure to document all aspects of your network, including existing topologies, logs, configuration files, etc.
3. Nokia has two default logs that are automatically configured in the router. They are log 99 and 100. Use these logs
and ones you create to identify the problem.
4. Isolate any faults you identify and actively use show commands to confirm your beliefs. Do not change any aspects
of the router until you have logically decided the problem. Randomly changing aspects of the router will generally
compound the problem and result in longer down time.

Module 3 | 12
© 2016 Nokia
Module 3 | 13
© 2016 Nokia
Module 3 | 14
© 2016 Nokia
Module 3 | 15
© 2016 Nokia
Plan your actions & Resolve the Problem
The actions you take will depend on the type of problem that you are trying to resolve. Critical problems that are
affecting a wide range of services for a large number of gold service level customers require a different tactic from
minor problems affecting a small number of best-effort service customers. The former situation will by necessity
require drastic and immediate actions to restore service while the latter can afford to take a little more time to ensure
that the actions will not put any other services at risk. The key is to balance the risk of creating further service
interruptions while attempting to restore service in the shortest possible timeframe. Whatever corrective action is
planned, you should:
 Reproduce the problem
 Document each step of the corrective action
 Test the corrective action
 Use CLI to verify behavior changes for each step
After testing your hypothesis and verifying that the corrective action will correct the problem and not introduce any new
symptoms, the next step is to apply the corrective action to the live network. When doing so, it is recommended to
resolve the easiest problem, in terms of risk, effort and time, first.

Verify Solutions
After having taken corrective action to resolve the problem it is important to verify that the changes have not
introduced new symptoms and that the original problem has been completed corrected. If new symptoms are detected
or if the problem has only been mitigated, you need to start the troubleshooting process again.
Save changes once operational

Once the problem has been resolved ensure that any network equipment that was modified has its active configuration
saved not only on the device but also to the off-site location where configurations are saved.
Document resolution
Create a detailed report as to what the problem was, what times your team was notified and what and when actions
were taken. This will become valuable in the next step.

Module 3 | 18
© 2016 Nokia
It is important to escalate to ensure that resources are used as allocated. First levels of support are typically staffed to
handle a large volume of incidents. A typical goal is to resolve 80% of the issues at first line.
An escalation plan usually includes, but is not limited to;

•Description of the escalation process
•Support level hierarchy
•A list of information that needs to be included such as problem status, suspected causes and troubleshooting
information.
•Contact Information and notification instructions
•Problem severity or priority levels
•A description of common issues followed by recommended procedures for each
•Optimal Repair times

Module 3 | 20
© 2016 Nokia
Module 3 | 21
© 2016 Nokia
Module 3 | 22
© 2016 Nokia
Module 3 | 23
© 2016 Nokia
Using the “show boot-messages” command can be useful when diagnosing problems related to bootup. The command
will provide a trap of the entire bootup procedure. This includes all tests that are accomplished at bootup along with
information about the individual blades inserted into the chassis.
When an IOM is installed in a slot and enabled, the system verifies that the installed IOM type matches the allowed IOM
type. The IOM will remain offline if the parameters do not match. To see the IOM configuration at system initialization
use the “show boot-messages” command. To display the current IOM configuration use the “show card” command.
The following is an example of the output for this command.
Using the “show mda” command will define what MDA is located in what slot of the chassis. This also allows the
administrator to ensure operability.
The “show system information” command will allow the administrator to verify basic operational ability and up time.
The “show chassis environment” command will display the current status of the router fans indicating any error
conditions. The status field should say “up” and the speed should be “half speed”.
The “show chassis power-supply” command displays general power supply information.

Troubleshooting a console connection
If you are unable to bring up a management session through the console port connection, the most likely source of the
problem is the console configuration. You should also verify the DTE/DCE setting of the terminal and select the
appropriate setting for the console port.
Troubleshooting a telnet or ssh connection

If you are unable to establish a telnet or ssh connection through the management port, verify that the management
port has been assigned an IP address by issuing a show bof command from a session established using the console port
or by a telnet connection to an IP interface on the router. In the default configuration, the telnet capability is disabled
on the Nokia 7750 SR and only ssh connections are allowed.

The “show chassis” command will identify the type of chassis you are connected too. It will also provide much more
information such as the MAC address of the router.

Indicator Category Potential Problem Indication
StatusAmber: Operationally down but administratively up.
Unlit: Not operational, shutdown, or administratively down. M/S
(Master/Slave) Ctl Green (blinking): Indicates that the SF/CPM is operating as the secondary SF/CPM
in a redundant configuration.
M/S
(Master/Slave) Ref Green (blinking): Indicates that the SF/CPM is operating as the secondary clocking
reference in a redundant system.
Unlit: Clock not initialized.
Timing Green (blinking): Clock in (internal) holdover state
Amber (blinking): Clock in free running state
Unlit: Clock not initialized.
Reference Amber: The reference is enabled (no shutdown) but not qualified.
Unlit: Not in use, not configured.
Reference BITS Status
:Amber: The reference is enabled (no shutdown) but not qualified.
Unlit: Not in use, not configured.
Power Supply Amber: Indicates an error condition with an installed power entry module in the
associated slot.
Unlit: Indicates that a power entry module is not installed or not recognized.
Fan Status Amber: Indicates a fan tray failure.
Unlit: Indicates that a fan tray is not installed.
Compact Flash Amber (blinking): Error condition exists.
Amber (solid): Indicates that the slot is in an operationally down mode. (This is
the only mode to safely remove the flash card.)
Unlit: A flash card is not installed in the slot.

When an IOM is installed in a slot and enabled, the system verifies that the installed IOM type matches the allowed IOM
type. The IOM will remain offline if the parameters do not match. To see the IOM configuration at system initialization
use the “show boot-messages” command. To display the current IOM configuration use the “show card” command.

The “show card detail” will document all major aspects of the individual cards in the chassis. The highlights define the
temperature, alarm state, MAC address and memory capacity. These are just some of the important bits of information
that can be acquired from this command.

Using the “show mda” command will define what MDA is located in what slot of the chassis. This also allows the
administrator to ensure operability.
The “show mda detail” command provides detailed information about each MDA configured on the router

The “show system information” command will allow the administrator to verify basic operational ability and up time.
System Up Time is calculated as the total time from restart or power on. If a CPM switch-over occurs, or it is forced, the
node is considered up, so System Up Time is not reset.

Power Supplies
7X50 OS supports a power-supply command to configure the type and number of power supplies present in the
chassis. The operational status of a power source is always displayed by the LEDs on the Control Processor/Switch
Fabric Module (CP/SFM), but the power supply information must be explicitly configured for a power supply alarm to be
generated if a power source becomes operationally disabled.
By default, 7X50 Series routers are configured as DC-input devices. Traps and alarms are automatically sent if DC power
supplies are installed in the power supply slots. In order to generate traps and alarms when AC power supplies are
installed in 7X50 models (except the 7450 ESS-1 model) the power-supply command must be modified. 7450 ESS-1
power supply parameters cannot be modified.
Configuring an existing power supply to none prior to powering off the unit will prevent an alarm from being generated.

The show chassis environment command will display the current status of the router fans indicating any error
conditions. The above is an example of the output of these commands. The status field should say “up” and the speed
should be “half speed”. If the speed is “double speed”, environment temperature may be an issue.
The 7750 SR-1 chassis supports redundant fans. There is a single fan tray containing six fans. Normal system
operation continues if one of the fans fails. (Not field replaceable)
The 7750 SR-7 chassis supports dual redundant fans. One fan is required for normal operation. Normal operations will
be maintained if a single fan unit is removed or fails.
The 7750 SR-12 chassis supports three redundant fans. Two fans of the three fans are required for normal operation.
Normal operations will be maintained if a single fan unit is removed or fails.
The 7750 SR continuously monitors the temperature of the system and adjusts their speed accordingly. In the event of
a cooling tower failure or if the chassis/line card temperature rises above a threshold, the appropriate chassis alarms
and SNMP traps and events are triggered.
NOTE: The fan tray increases speed as the egress air temperature rises and is forced to high speed if or when any of the
internally monitored temperatures on any of the SF/CPMs, IOMs, or MDAs exceed 154º F (68º C). NOTE: There are
temperature sensors on the IOM, SF/CPM and on each MDA. An alarm (trap) is generated when temperatures exceed
167° F (75° C). If any slot exceeds 230º F (110º C) then the slot will shut down to avoid damage.

On boot-up and during convergence events, it is common to see high CPU utilization. It is best to determine a steady
state baseline average to understand what is considered normal operating levels. High CPU levels rarely impact the data
forwarding of the SR platform. Even 99% for brief periods will not adversely affect the performance and may simply be
a convergence event.
CPU Time – The time each process or protocol has used in the specified sample period.
CPU Usage – The total CPU usage of the process or protocol.
Capacity Usage - Displays the level the specified process or protocol is being utilized. If this number hits 100%, this
part of the system is busied out. There may be extra CPU cycles still left, but this process or protocol is running at
capacity.

“Current Size” is the amount of memory allocated to the specific “Name” function. “Max Allowed” is the maximum
allocated memory size. “Max So Far” identifies the maximum amount of memory used by the specific function since the
router was activated. “In Use” denotes the amount of memory being used by the function at the time of entering the
command.
Remainder of the “show system memory-pools”.

…
CFLOWD No limit 0 1,048,576 0
IGMP No limit 1,048,576 1,048,576 1,028,920
PIM No limit 0 0 0
ATM No limit 4,320,656 4,320,656 3,329,952
MMPI No limit 1,048,576 1,048,576 132,072
MFIB No limit 1,048,576 1,048,576 32,024
PIP No limit 3,145,728 3,145,728 2,796,400
MBUF No limit 5,837,312 5,837,312 4,831,760
IGMP Snooping No limit 1,048,576 1,048,576 342,448
TLS MFIB No limit 1,048,576 1,048,576 514,880
-------------------------------------------------------------------------------------------------------------
Current Total Size : 313,037,408 bytes
Total In Use : 293,913,136 bytes
Available Memory : 636,102,720 bytes
===========================================================================

Module 3 | 37
© 2016 Nokia
Module 3 | 38
© 2016 Nokia
Module 3 | 39
© 2016 Nokia
Module 3 | 40
© 2016 Nokia
Look at the device to ensure that the connectivity LEDs are illuminated. If they are not then the interface might have a
cable connected to it, but the cable is not connected on the other end to an active device. This could be that the device
in question is not powered on, or it’s interface is inactive. It could also mean that the cable is broken. – Try replacing the
cable if all else seems operational.
If the connection is slow there could be a duplex mismatch on the interface. Check the interface with the “show
interface” command and the default logs for errors being detected on the interface.
If excessive FCS errors are detected ensure that the MTU of the end device is configured correctly. Also, ensure that
the cable connecting the two devices together is not near any devices that might be injecting electrical interference.
Late collisions occur after the standard collision detection time (5.12us for Fast Ethernet). The smallest a frame for
Ethernet is 64 bytes, anything less than that is considered a RUNT and discarded. The largest frame on an Ethernet is
1518 bytes in size. Anything greater than 1518 bytes is considered a giant and discarded. All of these issues can be
results of faulty NIC cards, poor cabling or non-compliant network configuration using excessive amounts of hubs.

There are multiple possible reasons for a data-link layer (layer 2) failure. Detecting them and quickly fixing the problem
depends on the ability of the technician.
Ways to Identify problems at the data-link layer vary. Below are some examples of how to detect problems:
•MTU failures, encapsulation failures, and framing errors all are great indicators that there are problems at the link-layer
•Late collisions are commonly seen when a duplex mis-match exists. Runts are frames that are too small for the
protocol, and giants are frames that are too large.
•Router log files (log files 99 and 100) should be checked to ensure that there are no critical errors.
•MAC database corruption will cause frames to be forwarded out ports that should not receive the frames. This can be
catastrophic and contribute to an STP loop.
•Excessive broadcast traffic can saturate a network. Keep in mind that a switch will forward broadcast traffic out all
ports, less the one the traffic is received on, as long as they belong to the same VLAN.
•STP failures and loops can bring a network to its knees. Methodical design and meticulous configuration will avoid this
issue.
•MAC filters when not documented and configured correctly will cause frames to be blocked from being sent out
interfaces without an administrators knowledge.

This command displays port or channel information. If no command line options are specified, the command port
displays summary information for all ports on provisioned MDAs.
Troubleshooting note: Ports by default are administratively down. If a port is correctly configured but not up, most
likely the port is administratively down.
The highlighted parts outline just some of the information that can be used in troubleshooting a physical layer problem.
Note that the specific port and operational speed is defined. In addition the administrative and operational status,
along with the duplex settings are stated.
Several lines have been removed to be able to show the “Transceiver Type” entry of the “show port x/x/x” command. In
this case the transceiver in use is an “SFP” type.

Several lines have been removed to be able to show the errors. In this case the port is showing no errors of any kind.
Alignment Errors - The total number of packets received that had a length (excluding framing bits, but including FCS
octets) of between 64 and 1518 octets, inclusive, but had either a bad Frame Check Sequence (FCS) with an integral
number of octets (FCS Error) or a bad FCS with a non-integral number of octets.
FCS Errors - The number of frames received on a particular interface that are an integral number of octets in length but
do not pass the FCS check.
SQE Test Errors - The number of times that the SQE TEST ERROR is received on a particular interface.
CSE - The number of times that the carrier sense condition was lost or never asserted when attempting to transmit a
frame on a particular interface.
Too Long Frames - The number of frames received on a particular interface that exceed the maximum permitted frame
size.
Symbol Errors - For an interface operating at 100 Mb/s, the number of times there was an invalid data symbol when a
valid carrier was present.
Sngl Collisions - The number of frames that are involved in a single collision, and are subsequently transmitted
successfully.
Mult Collisions - The number of frames that are involved in more than one collision and are subsequently transmitted
successfully.
Late Collisions - The number of times that a collision is detected on a particular interface later than one slotTime into
the transmission of a packet.
Excess Collisions - The number of frames for which transmission on a particular interface fails due to excessive
collisions.
Int MAC Tx Errors - The number of frames for which transmission on a particular interface fails due to an internal MAC
sublayer transmit error.
Int MAC Rx Errors - The number of frames for which reception on a particular interface fails due to an internal MAC
sublayer receive error.

Clear counters and statistics using the following commands:
A:Rtr-1# clear port 1/1/1 statistics

Clients are complaining that the server, located at 10.1.2.1, is not responding to requests. In addition, other clients are
complaining that other severs on that same network are not responding.
The administrator tries to ping the server that most people are upset about and notes that the ping fails. This step
verifies the issue.

A ping from P3 succeeds which proves the issue is somewhere between routers P1 and P3

In the above example the administrator has performed the “show port” command on the CLI of the router. This
summary page does not show enough information to verify that the interface is working correctly. What is does show is
that the router thinks that all ports are operational and active. Further analysis is required to determine what the
problem is.

In the above example the administrator has performed the “show port 1/1/1 detail” command. This command shows
in-depth information about the port at a physical and link layer. Note the excessive errors on the inbound of the
interface. Also, note that the CRC/Align errors are identical to the errors received. Either the remote routers port
connected to this router has problems or the cable connecting the two routers needs to be replaced.

The administrator checks the remote end port connecting to building 2. Note that the port has not detected any
errors. The administrator decides that the cable could be the problem. In checking the cable they note that one end
appears to be damaged.

After replacing the cable the administrator verifies that the connection is operational.

Clients are complaining about the slowness of the network between building-1 and building-3. The administrator
decides to ping the remote building to verify the slowness of the link.
Upon executing the ping the administrator notes that only a few of the pings are getting across the link.

One possible reason would be the duplex settings on the links between the routers. To eliminate that possibility the
administrators are checking the settings and current operational setting of the ports.
Note the operational and administrative state of the interface is “up”, meaning operational. In addition the configured
and operational duplex settings are set to “full” on router P1.
Also note that there are no detected errors on the port from this side of the link between P1 and P4.
Note: The Errors counter is a running total. Either clear the port statistics before troubleshooting, or run the command
multiple times and check for an incrementing value.

Having checked the one end of the link the administrator is now checking the other side. Note that the duplex setting is
“half” and not “full”. This mismatch will result in one side simultaneously transmitting and receiving data, while the
other side will only send data when there is no traffic being received. If P4 is sending data and at the same time receives
data, the router will log the errors since it would violate CSMA/CD. These errors are commonly called “late collisions”
since they occur well after the normal collision time.
To solve this problem the administrator must change P4 to “full-duplex”.

Once the link is reactivated the ping works to the remote end.

When troubleshooting use the following steps:
 Check the log files to see if the router has detected any problems
 Using show commands verify that the port and interface is active
 Verify that the encapsulation is correctly configured on the interface
 Check the MTU of the interface and that of the remote end. Ensure it’s correctly configured.
 Check for FCS errors. If many are detected then examine the cable, NIC cards and path of the cable.
 As always, document your findings.


Maximum Transmission Unit defines the size of packet, in bytes, that a device can send across a segment. The MTU
varies based upon the type of network in use. For example, Ethernet’s commonly accepted MTU is 1518 bytes, but
when sending frames across a trunk, a protocol such as 802.1q increases the MTU to 1522 bytes. This is due to the
additional 4 bytes of overhead used by 802.1q for tagging the VLAN association to the frames being sent.
NOTE: When calculating the MTU for a port do not include the FCS, although some other vendors do. With the FCS an
Ethernet frame is 1518 bytes in length. The FCS is the last 4 bytes. Since the FCS is not included in the MTU
calculation, a standard frame is 1514 bytes.

MTU Configuration Guidelines
Observe the following general rules when planning your service and physical MTU
configurations:
 The SR must contend with MTU limitations at many service points. The physical (access and network) port,
service, and SDP MTU values must be individually defined.
 Identify the ports that will be designated as network ports intended to carry service traffic.
 MTU values should not be modified frequently.
 MTU values must conform to both of the following conditions:
• The service MTU must be less than or equal to the SDP path MTU.
• The service MTU must be less than or equal to the access port (SAP) MTU.
Modifying MTU Defaults

MTU parameters should be modified on the service level as well as the port level.
 The service-level MTU parameters configure the service payload (Maximum Transmission Unit – MTU) in bytes
for the service ID overriding the service-type default MTU.
 The port-level MTU parameters configure the maximum payload MTU size for an Ethernet port or SONET/SDH
SONET path (sub-port).
The default MTU values should be modified to ensure that packets are not dropped due to frame size limitations. The
service MTU must be less than or equal to both the SAP port MTU and the SDP path MTU values. When an SDP is
configured on a network port using default port MTU values, the operational path MTU can be less than the service MTU.
In this case, enter the show service sdp command to check the operational state. If the operational state is down, then
modify the MTU value accordingly.

Excellent tool for viewing bandwidth utilization.
The command will default to monitoring for 50 seconds; within that timeframe the router will use a 5 second window to
provide the analysis and update the window. This timeframe can be modified if necessary to adapt to the
administrators requirements.

The “debug sync-if-timing” command allows a user to force the system synchronous timing output to use a specific
reference. When the command is executed, the current system synchronous timing output is immediately referenced
from the specified reference input. If the command forces the BITS input, then both CPMs will select their local BITS
input ports; otherwise, the standby CPM locks to the output of the active CPM clock.
The force-reference command affects both the central clock and the BITS output. The 7750 SR-c4 has two BITS input
ports on the cfm. The force reference command on this system allows the selection of the specific port.
NOTE:
The debug sync-if-timing force-reference command should only be used to test and debug problems. Network
synchronization problems may appear if network elements are left with this manual override setting. Once the system
timing reference input has been forced, it may be cleared using the no force-reference command.

The debug system command will allow all system level information to be analyzed in a real time perspective.
The persistence feature allows information learned through DHCP snooping across reboots to be kept. This information
can include data such as the IP address, MAC binding information, lease length information, and ingress sap information
(required for VPLS snooping to identify the ingress interface). This information is referred to as the DHCP lease-state
information.

When a link is not working, one of the best ways to test the interface is to logically configure the interface to loop back
upon itself. Nokia supports both software and hardware based loopback implementations. For software loopback there
is no need for physical cables to be connected to the interface to make it logically active.
Ethernet ports:
You can NOT loop Ethernet ports using CLI commands.
line — Places the associated port or channel into a line loopback mode. A line loopback, loops frames received on the
corresponding port or channels back to the remote router.
internal — Places the associated port or channel into an internal loopback mode. An internal loopback loops the frames
from the local router back at the framer.
fdl-ansi — requests FDL line loopback according to ANSI T1.403.
fdl-bellcore — requests FDL line loopback according to Bellcore TR-TSY-000312.
payload-ansi — requests payload loopback using ANSI signaling.
inband-ansi — requests inband line loopback according to ANSI T1.403.
inband-bellcore — requests inband line loopback according to Bellcore signaling.

Module 3 | 64
© 2016 Nokia
Module 3 | 65
© 2016 Nokia
Module 3 | 66
© 2016 Nokia
Module 3 | 67
© 2016 Nokia
Module 3 | 68
© 2016 Nokia
Module 3 | 69
© 2016 Nokia
Module 3 | 70
© 2016 Nokia
When troubleshooting layer 3 issues, it is a best practice approach to first check for log entries associated with the
protocol in question. Due to the large amount of events logged, it may be useful to use a filter such as the following;
show log log-id 99 application ospf (filters for OSPF events)

show log log-id 99 application isis (filters for ISIS events)
show log log-id 99 application bgp (filters for BGP events)

Module 3 | 72
© 2016 Nokia
The OSPF environment is organized using two primary elements.
 Area — An area is a grouping of contiguous OSPF networks and hosts. OSPF areas are logical subdivisions of
OSPF autonomous systems. The topology of each area is invisible to entities in other areas, and each area
maintains its own topological database.
 Autonomous System — a group of networks, and network equipment under common administration.
BACKBONE
The OSPF backbone area, area 0.0.0.0, must be contiguous and all other areas must be connected to the backbone
area. The backbone distributes routing information between areas. If it is not practical to connect an area to the
backbone then the ABRs must be connected via a virtual link.
STUB AREA
A stub area is a designated area that does not allow external route advertisements. Routers in a stub area do not
maintain external routes. A single default route to an ABR replaces all external routes. This OSPF implementation
supports the optional summary route (type-3) advertisement suppression from other areas into a stub area. This
feature further reduces topological database sizes and OSPF protocol traffic, memory usage, and CPU route calculation
time.
NSSA
Another OSPF area type is called a Not-So-Stubby area (NSSA). NSSAs are similar to stub areas in that no external
routes are imported into the area from other OSPF areas. External routes learned by OSPF routers in the NSSA area are
advertised as type-7 LSAs within the NSSA area and are translated by ABRs into type-5 external route advertisements
for distribution into other areas of the OSPF domain. An NSSA area cannot be designated as the transit area of a virtual
link.

Module 3 | 74
© 2016 Nokia
In the figure above, the two routers have not formed an adjacency. Both routers are in a down state; neither router has
sent any OSPF-related packets. The following steps describe how the two routers reach the 2-way state and the actions
that are required:
1. The router on the left sends a hello packet with the standard header. In the hello information, the router inserts its
RID and leaves the neighbor field blank because it does not know of any other router on the Ethernet segment.
The router now is in the initializing state.
2. The right-side router responds with its own hello packet. However, this router’s hello contains not only its RID, but
also the RID of the left-side router.
3. The left-side router responds with another hello packet, containing the RID of the right-side router. Now that each
router sees that the other router acknowledges its existence, the state changes from initializing to 2-way.

In the figure above, the two routers have not formed an adjacency. The following steps describe how the adjacency is
created and the actions that are required.
1. The neighboring routers establish a master/slave relationship. During this step, the initial DBD sequence number is
determined for the exchange state. The router with the highest RID becomes the master, and its initial sequence
number is used.
2. This is part of step 1.
3. The slave (left-side) router sends its DBD packet, describing its link-state database. The sequence number
negotiated in step 1 is used.
4. The master (right-side) router increments the sequence number and sends its DBD packet, describing its link-state
database.

The adjacency continues to be created with the following steps:
1. Each router is responsible for maintaining a bit of reliability. Each responds to the DBD with an ACK packet. This
ensures that each knows the other has received the information without error.
2. In the example, the right side router asks for explicit information with the use of an LSR. Both routers would
actually be sending LSRs. When the LSR is sent, the exchange state changes to the Loading state.
3. Each router responds to the LSR with one or more LSU packets. These packets contain explicit details about the
requested networks.

The final steps for creating the adjacency are described below:
1. The LSUs are sent and acknowledged by each router.
2. After all LSUs have been received and ACKs sent, each router now has an identical link-state database. The state
changes from loading to full. This means that each router is fully converged with the other’s database.
3. To maintain the adjacency, the routers send periodic hellos to each other. The default interval is 10 seconds. If
something changes, only that change in the database is sent to the neighbor.

When troubleshooting Layer 3 issues, it is useful to try and pinpoint if the problem is related to the Control Plane or
Data Plane. When checking for OSPF control plane issues, be sure to check the following;
•OSPF status
•OSPF adjacencies
•OSPF interface status
•OSPF Configuration (Areas, ABR, ASBR, etc.)
•OSPF Database (LSAs)
•Management Access or CPM Filters

Suggested actions to resolve the problem.
 Link/Interface Status – Verify that the link is operational by using the “show router interface” command.
 MTU Mismatch – The MTU can be set at the port level or at the IP level. To verify the MTU settings use the “show
router ospf interface” command. Interface MTU is advertised in the Database Descriptor Packet.
 Mismatched Interface type – To verify how the interface is configured use the “show router ospf interface”
command. The adjacency will not get past the DR election process. The interface that is configured as broadcast
will be stuck in two-way and the interface that is configured for point-to-point will be stuck in exchange.
 Mismatched subnet mask or IP address - Check the router and its neighbor’s interface to see if the subnet mask
or IP address matches each other. Use the “show router interface” command. The network mask is advertised in
the OSPF Hello Packet
 Interface not configured for OSPF – To verify that OSPF has been configured on the interface use the “show
router ospf interface” command.
 Router-ID not unique - Make sure the router has a unique Router ID. Normally a router uses its system interface as
its Router ID. A router ID can also be configured specifically. If neither the system interface or router ID are
implicitly specified, then the router ID is inherited from the last four bytes of the MAC address. Use the “show
router ospf status” command to verify the router-id. The router-id is advertised in the OPSF hello Packet
 Neighbor is configured for authentication - If the router’s OSPF neighbor is configured for authentication, the
router must be configured to match the authentication. To view the authentication configuration of an interface,
use the “config>router>ospf>area if# info detail” command. Authentication is advertised in the OSPF Hello
Packet
 OSPF parameters do not match neighbor – Mismatched values, such as hello and dead timers will result in OSPF
not operating correctly. To view the settings of the router use the “show router ospf interface <int-name> detail”
command. OSPF Parameters are advertised in the OSPF Hello Packet.
 Area ID mismatch- The interfaces must be configured in the same area for the adjacency to come up. To view the
settings of the router use the “show router ospf interface <int-name> detail” command. Area id is Advertised in
the OSPF hello Packet

Module 3 | 81
© 2016 Nokia
A neighbor is a router configured with an interface to a common network. In broadcast networks, a designated router
and a backup designated router are elected. The designated router is responsible for sending LSAs describing the
network. For point-to-point networks, no designated or backup designated router is elected.
This command will display all neighbor information. To reduce the amount of output the user many opt to select the
neighbors on a given interface by address or name. The detail option will produce a large amount of data for each
neighbor.
When troubleshooting, this command is an excellent way to quickly check the status of all OSPF neighbors on a router.

An OSPF interface includes state information that was obtained from the underlying lower level protocols and from the
routing protocol itself.
This command displays the details of the OSPF interfaces. An interface can be identified by ip-address or ip interface
name. When neither is specified, all in-service interfaces are displayed.
The output above shows that interface “to-R5” is Up and configured as Point-to-Point which, in this example, is correct.
Interface “to-R1” on R5 is also in the same status.

Adding the “detail” option will output detailed information on the interface. This is very useful when verifying values
such as the IP address, Hello interval, Authentication, MTU, etc.
When troubleshooting adjacency issues, be sure to run this command on both interfaces.

Always check log-id 99 for messages that may help to locate the problem. Use a filter (such as the OSPF filter used
above) to look for specific events.
In the above log, R1 is stating it is receiving a conflicting hello interval from 10.1.5.5 than what is configured on interface
“to-R5”.

The debug router ospf command allows the user to troubleshoot an OSPF related issue in many circumstances. The following are the
choices of events that can be logged;
- [no] area - [no] neighbor
- [no] area-range - [no] nssa-range
- [no] cspf - [no] packet
- [no] interface - [no] rtm
- [no] leak - [no] spf
- [no] lsdb - [no] virtual-neighbor
- [no] misc
The Hello packet is used to establish adjacencies with other routers that speak OSPF. It is also used to maintain neighbor connectivity
by being propagated periodically. Viewing hello packets can help in recognizing the following adjacency issues;
- Mismatched subnet mask or IP address
- Router ID not unique
- Authentication problems
- OSPF timer do not match
- Area IDs do not match
The output above verifies there is a hello interval mismatch.
Important Notes:
1) Before enabling “debug”, the user must ensure a log is created to view the debug result. The following is a log created to view
debug results. Note that if the log destination is session, when the session is closed, the log (log-id) will not be saved.
B:R1>config>log>log-id 2
B:R1>config>log>log-id$ from debug-trace
B:R1>config>log>log-id$ to session
2) Use either of the following commands to stop the debug at different levels:
debug router ospf no packet - Disable debugging for OSPF packets
debug router no ospf - Disable debugging for all OSPF messages
no debug - Disable debugging for all applications

The hello interval should be set to 5 which is the default for OSPF. Changing this value back to 5 for interface “to-R1”
on router R5 will allow the OSPF adjacency come up.

This command displays information about the OSPF link state database (LSDB). For more detailed information on each
LSA, use the “detail” command.
Syntax:
database [type {router | network | summary | asbr-summary | external | nssa | all}] [area area-id] [adv-router router-id]
[link-state-id] [detail]

Module 3 | 89
© 2016 Nokia
The show router ospf spf command can be used to determine if there is network instability caused by networks either
internal or external to OSPF. The total number of spf runs should not continually increment when the output is
refreshed in a stable network.

The show router ospf database command displays information about the OSPF link state database (LSDB). For more
detailed information on each LSA, use the “detail” command.
Checking the age of an LSA is a good way of isolating the location of the instability. A router that is experiencing link
flapping will have to generate a new LSA for every time the link flaps. This can be noticed as an LSA, or multiple LSAs in
the database with a continual low age, each time the command is executed.
Note:
In multi area environments it will be required to check the source of the summary LSAs. Once the source of the
summary LSAs has been determined, run the OSPF commands from the source of the summary LSAs.

The debug router ospf packet dbdescr is very useful in debugging MTU issues.



Module 3 | 95
© 2016 Nokia
IS-IS — Intermediate System to Intermediate System
 ISO standard 10589, subsequently RFC 1142
 Link-state
 Highly scalable (1000 routers per area)
 Areas are connected by Level 2 routers in a mesh.
 The network between level 2 routers must be highly available
 Routing protocol engine almost identical to OSPF, except area boundaries are on links between routers rather
than through a border router (an ISIS router is always in one area only)
All routers within an ISIS topology are identified as Level 1, Level 2, or Level 1/2 routers.
 Level 1 routers exchange topology information for the local area.
 Level 2 routers exchange topology information between the different areas.
 Level 1/2 routers exchange information between Level 1 and Level 2 routing domains

When checking for ISIS control plane issues, be sure to check the following;
•ISIS status
•ISIS adjacencies
•ISIS interface status
•ISIS Configuration (Levels, Authentication, etc.)
•ISIS Database

Suggested actions to resolve the problems.
 Link/Interface Status — Verify that the link is operational by using the “show router interface” command.
 Mismatched Interface type — To verify how the interface is configured use the “show router isis interface”
command. The ISIS routers will be sending different ISIS Hello packets
 Mismatched subnet mask or IP address — Check the router and its neighbor’s interface to see if the subnet mask
or IP address matches each other. Use the “show router interface” command.
 Interface not configured for ISIS — To verify that ISIS has been configured on the interface use the “show router
isis interface” command.
 System-ID not unique — Make sure the router has a unique System ID. Normally a router uses its system interface
as its System ID. A router ID can also be configured specifically. If neither the system interface nor router ID are
implicitly specified, then the system ID is inherited from the last four bytes of the MAC address. Use the “show
router isis status” command to verify the system-id. The system id advertised in the Hello packet
 Neighbor is configured for authentication — If the router’s ISIS neighbor is configured for authentication, the
use the “config>router>isis> if# info detail” command. There are multiple authentication levels for ISIS (hello,
csnp, psnp, etc.)
 Level Capability — The Level Capability must be configured properly based on the area the interfaces are in.

Module 3 | 99
© 2016 Nokia
An adjacency is a portion of the local routing information which pertains to the reachability of a single neighboring end
or intermediate system over a single circuit. Adjacencies are used as input to the decision process to form paths
through the routing domain. A separate adjacency is created for each neighbor on a circuit and for each level of routing
(Level 1 and Level 2) on a broadcast circuit.
When troubleshooting, this command is an excellent way to quickly check the status of all ISIS adjacencies on a router.

There are no interfaces associated with IS-IS by default. An interface belongs to all areas configured on a router and
therefore cannot belong to separate areas.
This command displays the details of the ISIS interfaces. An interface can be identified by ip-address or ip interface
name. When neither is specified, all in-service interfaces are displayed.
The output above shows that all the correct interfaces are configured for ISIS level-capability 1 and 2 and are
operationally up.

This command displays information about the ISIS link state database. For more detailed information use the “detail”
command.
An L1/L2 router sets the ATT bit when it has routes to another area. The L1 routers in an area create a default route
to the nearest router with the ATT bit set.
The Level 2 database above should be showing Level 2 capable interfaces in all areas. Seeing as how R2 is the gateway
for area 49.0001, this is the reason why the client can not route from area 49.0001 to any other area.

Always check log-id 99 for messages that may help to locate the problem. Use a filter (such as the ISIS filter used
In the above log, R2 is stating that it has L2 authentication failures on interfaces to-R1, to-R3 and to-R6.

The debug router isis command allows the user to troubleshoot an ISIS related issue in many circumstances. The following are the
- [no] adjacency - [no] cspf
- [no] graceful-restart - [no] interface
- [no] leak - [no] lsdb
- [no] misc - [no] packet
- [no] rtm - [no] spf
- [no] tunnel-endpoint
The following PDUs are used by IS-IS to exchange protocol information:

1. IS-IS hello PDU — Routers with IS-IS enabled send hello PDUs to IS-IS-enabled interfaces to discover neighbors and establish
adjacencies.
2. Link-state PDUs — Contain information about the state of adjacencies to neighboring IS-IS systems. LSPs are flooded periodically
throughout an area.
3. Complete sequence number PDUs — In order for all routers to maintain the same information, CSNPs inform other routers that
some LSPs can be outdated or missing from their database. CSNPs contain a complete list of all LSPs in the current IS-IS database.
4. Partial sequence number PDUs (PSNPs) — PSNPs are used to request missing LSPs and acknowledge that an LSP was received.
The output above shows an authentication password mismatch. R2 is showing a password with 4 bytes (when converted is equal to
“lala”) whereas all its’ neighbours are showing an authentication password with 7 bytes (when converted is equal to “alcatel”). This
information confirms the authentication mismatch issue.

The level 2 authentication-key is set to “alcatel” to match the other level 2 neighbours . Once this is completed, the
“show router isis database” command on R2 will show entries from other level 2 capable routers in the level 2
database. Clients in area 49.0001 can not route to the other areas. The ping command can also be used to verify the
correct operation.

Module 3 | 106
© 2016 Nokia
The show router isis statistics displays information regarding IS-IS traffic statistics. It can be used to determine if
there is network instability. The total number of spf runs should not continually increase when the output is refreshed
in a stable network.

The show router isis database command displays the entries in the IS-IS link state database (LSDB). For more detailed
information on each entry, use the “detail” command.
Checking the “Lifetime” of an entry is a good way of isolating the location of the instability. A router that is
experiencing link flapping will have to generate a new entry for every time the link flaps. A newly created entry will have
an age of 1200 and decreases to 0.
There should not be continual new events when the output is refreshed in a stable network.



Module 3 | 111
© 2016 Nokia
BGP is an enhanced distance vector protocol, specifically called a path vector protocol.
Neighbor relationships in BGP are somewhat different from what is normal in the IGP context. Traditionally, a neighbor is
always a directly connected router. With BGP, this is not the case. Neighbors may be directly connected, but this is not
required. Therefore, BGP relies on an IGP to route between peers that are not directly connected.
BGP uses unicast TCP/IP for neighbor establishment. It is possible for neighbor relationships to be established with any
device that is IP-reachable. There is no guarantee that the neighbor relationship will succeed, because factors such as
firewalls or access control lists may prevent certain types of traffic from passing, but the relationship is possible and
likely to occur.
At the application layer, BGP functions similarly to TCP/IP applications such as Telnet, FTP, and HTTP. BGP is viewed as
an application because it uses registered port number 179 in the TCP/IP model.
Generic TCP/IP applications use a 3-way handshake for session establishment. After the session is established, the
applications exchange or negotiate a set of parameters for the session. In Telnet, for example, parameters such as
terminal types and passwords are typically negotiated. If application-level parameters are also acceptable, a session is
established at the application layer and data is exchanged. Periodic user data keeps the session alive and, when the
session is to be terminated, either user input or an inactivity timeout will cause the application session to be torn down.
TCP/IP initiates the 4-way session teardown.

Under normal neighbor establishment procedures, BGP peers can exist in one of six defined states. Idle is the initial
state, established is the operational state, and all other states are transitional. Peers that exist in one of these
transitional states for extended intervals usually indicate a connection problem.
A neighbor may also be in the Idle state, if it is administratively shut down.

1. BGP process receives BGP updates (from TCP-stack), and are collected in the RIB-IN (RIB = routing information
base).
2. After applying import policies (if configured) the routes are sent to the RIB.
3. If routes are considered being “best”, and if they have to be advertised to one or more peers…, they are sent to
the RIB-OUT, and export policies are applied.
4. BGP routes are sent out to the peers who need to receive the routes (BGP UPDATE).
5. RIB does the BGP selection process, and sends the “best” routes to the RTM.
6. RTM collects all “best” routes from all protocols, and selects the best, based on the preference.
7. RTM sends all “best” routes to the FIB, on all forwarding complexes of the IOM.
8. The incoming IP traffic, will do LPM (longest prefix match) on the FIB, to take a forwarding decision.
Note: It is not because a route is indicated as BEST in BGP, that it is actually used for IP-forwarding.
Note: BGP forwards routes only when they are ‘best’ and ‘used’. This is different for VPRN routes.

When checking for BGP control plane issues, make sure to check the following;
•BGP status
•BGP neighbours
•BGP interfaces
•BGP Configuration (AS number, Authentication, etc.)
•BGP Policies

Suggested actions to resolve the problems;
 Link/Interface Status — Verify that the link is operational by using the “show router interface” command.
 Router-ID not unique — Make sure the router has a unique Router ID. Normally a router uses its system interface
as its Router ID. A router ID can also be configured specifically. If neither the system interface or router ID are
implicitly specified, then the router ID is inherited from the last four bytes of the MAC address. Use the “show
router bgp status” command to verify the router-id.
 Neighbor is configured for authentication — If the router’s OSPF neighbor is configured for authentication, the
use the “show router bgp” command.
 Neighbor statements not accurate — If either neighbor does not correctly list the other router as a neighbor, the
bi-directional TCP handshake will not take place. If that occurs BGP will not operate correctly. Check both routers
configurations to ensure they are working correctly.
 Local or Peer AS incorrect – Use the “show router bgp summary” command to view the configured AS. The BGP
configuration will also show which peer AS is configured as well.
 Incorrect Route Policy – Use the “show router policy <policy-name>” command to view a policy’s configuration.
 Route Reflector Cluster ID incorrect – Use the “show router bgp neighbor” or “show router bgp group” to view
the Cluster ID.

Module 3 | 117
© 2016 Nokia
The show router bgp summary command displays a summary of BGP information.
When troubleshooting, this command is an excellent way to quickly check information such as the BGP router ID,
autonomous system, BGP operational state and the status of all BGP neighbours on a router.
The output above shows the neighbour 10.10.10.2 is currently in the “Active” state, which confirms what the client is
seeing.

Peer relationships are defined by configuring the IP address of the routers that are peers of the local BGP system. When
neighbour and peer relationships are configured, the BGP peers exchange update messages to advertise network
reachability information.
A peer is operational when the State is showing as “Established”
In the above output, peer 10.10.10.2 is currently in the “Active” state with an error of “Bad BGP Identifier”

Always check log-id 99 for messages that may help to locate the problem. Use a filter (such as the BGP filter used
In the above log, R6 is showing an error of “INCORRECT_BGPID” when trying to associated with peer 10.10.10.2

The debug router bgp command allows the user to troubleshoot an BGP related issue in many circumstances. The following are the
- [no] events - [no] graceful-restart
- [no] keepalive - [no] notification
- [no] open - [no] outbound-route-filtering
- [no] packets - [no] route-refresh
- [no] rtm - [no] socket
- [no] timers - [no] update
Four message types are used by BGP to negotiate parameters, exchange routing information and indicate errors. They are:
1. Open Message — After a transport protocol connection is established, the first message sent by each side is an Open message. If
the Open message is acceptable, a Keepalive message confirming the Open is sent back. Once the Open is confirmed, Update,
Keepalive, and Notification messages can be exchanged.
2. Update Message — Update messages are used to transfer routing information between BGP peers. The information contained in
the packet can be used to construct a graph describing the relationships of the various autonomous systems. By applying rules,
routing information loops and some other anomalies can be detected and removed from the inter-AS routing.
3. Keepalive Message — Keepalive messages, consisting of only a 19 octet message header, are exchanged between peers frequently
so hold timers do not expire. The keepalive messages determine if a link is unavailable.
4. Notification — A Notification message is sent when an error condition is detected. The peering session is terminated and the BGP
connection (TCP connection) is closed immediately after sending it.
The output above shows router 10.10.10.6 is sending a BGP ID of 10.10.10.2, which is incorrect.

Confirm that the system interface is configured correctly by running the show router interface command. If the
interface is configured correctly, check to see if a router-id is configured in the BGP configuration. When the no router-
id command is issued, BGP will use the system interface address as the BGP Router ID by default.

Module 3 | 123
© 2016 Nokia
Module 3 | 124
© 2016 Nokia
It is important to note that the flags are very useful for debugging.
Flags:
used: route is in use in the RTM
suppressed: route is suppressed by route-flap-damping
history: route has been withdrawn, but logged
decayed: route damping entries that are not valid, but not suppressed
valid: no inconsistency in the BGP attributes
I/e/?: BGP attribute ORIGIN
best: the route has been selected as “best” by the BGP selection process

A:PEESMON01# show router bgp routes 12430:700002:10.184.135.0/24 detail
…
Original Attributes
Network : 10.184.135.0/24
Nexthop : 10.51.0.68
Route Dist. : 12430:700002 VPN Label : 131060
From : 10.51.0.4
Res. Nexthop : 10.49.128.1
Local Pref. : 100 Interface Name : port-1/1/3
Aggregator AS : none Aggregator : none
Atomic Aggr. : Not Atomic MED : none
Community : 12430:2 target:12430:700000
Cluster : 1.1.1.1
Originator Id : 10.51.0.68 Peer Router Id : 10.51.0.4
Flags : Used Valid Best IGP
AS-Path : No As-Path
Modified Attributes
Network : 10.184.135.0/24
Nexthop : 10.51.0.68
From : 10.51.0.4
Res. Nexthop : 10.49.128.1
Local Pref. : 100 Interface Name : port-1/1/3
Community : 12430:2 target:12430:700000
Cluster : 1.1.1.1
Originator Id : 10.51.0.68 Peer Router Id : 10.51.0.4
Flags : Used Valid Best IGP

-------------------------------------------------------------------------------
RIB Out Entries
-------------------------------------------------------------------------------
Network : 10.185.70.0/25
Nexthop : 10.51.0.129
To : 10.51.0.20
Res. Nexthop : n/a
Local Pref. : 100 Interface Name : NotAvailable
Community : target:12430:600000
Cluster : No Cluster Members
Originator Id : None Peer Router Id : 10.51.0.20
Origin : IGP
Network : 10.185.70.0/25
Nexthop : 10.51.0.129
To : 10.51.0.5
Res. Nexthop : n/a
Origin : IGP
Network : 10.185.70.0/25
Nexthop : 10.51.0.129
To : 10.51.0.4
Res. Nexthop : n/a
Origin : IGP

Module 3 | 127
© 2016 Nokia
:PEESMON01# show router route-table
- route-table [<family>] [<ip-prefix[/prefix-length]>
[longer|exact]]|[protocol <protocol-name>]|[summary]
<ip-prefix[/prefix*> : ipv4-prefix a.b.c.d (host bits must be 0)

ipv4-prefix-length [0..32]
ipv6-prefix x:x:x:x:x:x:x:x (eight 16-bit pieces)
x:x:x:x:x:x:d.d.d.d
x [0..FFFF]H
d [0..255]D
<protocol> : keyword
<summary> : keyword
<longer> : keyword
<exact> : keyword
<family> : ipv4|ipv6|mcast-ipv4
<protocol-name> : bgp|bgp-vpn|isis|local|ospf|rip|static|aggregate|ospf3
|sub-mgmt
A:PEESMON01# show router fib

- fib <slot-number> [<family>] [<ip-prefix/prefix-length> [longer]]
<slot-number> : [1..10]
<ip-prefix/prefix-*> : ipv4-prefix a.b.c.d (host bits must be 0)
ipv6-prefix x:x:x:x:x:x:x:x (eight 16-bit pieces)
x:x:x:x:x:x:d.d.d.d
x [0..FFFF]H
d [0..255]D
<longer> : keyword
<family> : ipv4|ipv6

Policies can be as simple or complex as required. A simple policy can block routes for a specific location or IP address.
More complex policies can be configured using numerous policy statement entries containing matching conditions to
specify whether to accept or reject the route, control how a series of policies are evaluated, and manipulate the
characteristics associated with a route.
There are no default route policies. Each policy must be created explicitly and applied to a policy, a routing protocol, or
to the forwarding table. Policy parameters are modifiable.

The above output will show entries that have only been committed on the router.




Module 3 | 134
© 2016 Nokia
Filter policies, also referred to as Access Control Lists (ACLs), are templates applied to services or network ports to
control network traffic into (ingress) or out of (egress) a service access port (SAP) or network port based on IP, IPv6, and
MAC matching criteria.
A filter policy compares the match criteria specified within a filter entry to packets coming through the system, in the
order the entries are numbered in the policy. When a packet matches all the parameters specified in the entry, the
system takes the specified action to either drop or forward the packet. If a packet does not match the entry
parameters, the packet continues through the filter process and is compared to the next filter entry, and so on.

A filter is needed on all network interfaces on R6 that will drop all ICMP traffic in the 10.10.10.0/24 subnet, except from the core
routers (R1, R2, R3 and R4). The filter on the left is the correct one, which inputs the drop on 10.10.10.0/24 after the more specific
allow entries. The filter on the left, however, drops all ICMP traffic in 10.10.10.0/24 from the first entry and therefore will never reach
the more specific entries below.

Many policies have an enormous amount of entries, which can make it extremely difficult to pinpoint if the filter is
dropping packet. Using a filter log to capture events can help with this.
To use the filter log for troubleshooting policies, perform the following;
1. Create a filter log
2. Add the log to the entries you wish to view
3. View the log events
If traffic is running and the log shows no events or no new events, then you know the packets didn’t even reach that entry. However, if
packets did reach the entry, they can be viewed in the log contents.
Example Above:
ip-filter 1 is applied on the ingress of all network interfaces on R6. It is configured incorrectly because the more generic drop entry is
before the specific forward ones.
•In step 1, a filter log is created.
•In step 2, the filter log is applied to an entry to see if the traffic reaches that entry or not. If the traffic is be processed by a certain
entry, the output will be logged in the log file. If the traffic does not reach the entry, the number of entries logged will equal 0. The
above output shows that entry 5 is dropping the ICMP traffic from 10.10.10.4.
•In step 3, the filter log is confirming that entry 5 in the policy is dropping ICMP traffic from 10.10.10.4.
Information on creating and applying filter logs can be found in Module 2.

Module 3 | 138
© 2016 Nokia
Module 3 | 139
© 2016 Nokia
Module 3 | 140
© 2016 Nokia
Customer edge devices
A customer edge (CE) device resides on the customer premises. The CE device provides access to the service provider
network over a link to one or more provider edge (PE) routers. The end user typically owns and operates these devices.
The CE devices are unaware of tunneling protocols or VPN services that are provided by the service provider.
Provider edge devices

A provider edge (PE) device has at least one interface that is directly connected to the CE devices. In addition, a PE
device usually has at least one interface that connects to the service provider’s core devices or routers. The PE device
must be able to connect to different CE devices over different access media, so it is usually able to support many
different interface types. The PE device is the customer's gateway to the VPN services offered by the service provider.
Provider router
Provider (P) routers are located in the provider core network. The P router supports the service provider’s bandwidth
and switching requirements over a geographically dispersed area. The P router does not connect directly to the
customer equipment.
LER (Label Edge Router)

The LER MPLS router resides in the boundary between the MPLS domain and the customer domain (hence the keyword
“edge”). In this sense, it is similar to the PE. This naming convention refers to router’s function in the MPLS datagram
forwarding process. An LER may be an:
 Ingress LER (iLER): Non-MPLS traffic enters the MPLS domain through the iLER. The iLER adds a label to the non-
MPLS traffic and sends it to the next hop LSR.
 Egress LER (eLER): MPLS traffic exits the MPLS domain through the eLER. The eLER removes the label from the
MPLS packet and forwards the unlabeled packet to the CE router.
LSR (Label Switched Router)

The LSR resides within the MPLS domain. It connects the iLER and eLER to form a path for forwarding labeled traffic
through the MPLS domain. When an LSR receives labeled traffic, it replaces the incoming (ingress) label with an outgoing
(egress) label and forwards the labeled packet to the next hop router.
Whether a router is iLER, eLER, or LSR depends on where that router resides in the MPLS domain as well as the direction
in which traffic flow. A different CE-CE pair or traffic flow direction could change these roles.

A Label Switched Path (LSP) can be defined as a sequence of labels and label actions performed on MPLS routers to
forward data packets from point A to point B, using label switching. A Label Switched Path always starts from an iLER
and ends at an eLER.
In the above slide, traffic flows from left-to-right. The flow of MPLS labeled packets in the other direction, that is, right-
to-left, would be represented by another LSP pointing in the reverse direction. In this case, the roles of the iLER and
eLER routers in the figure would be swapped.
The encapsulation and forwarding of packets using labels is also referred to as tunneling; thus, LSP’s are often called as
tunnels. Tunnels must be established prior to the arrival of data packets. Label negotiation and distribution protocols
are used to build the tunnels with negotiated label values. The details of these control processes and exact mechanisms
of MPLS protocols will be covered in the upcoming modules.
The following are LSP types:

1. Static LSPs
 A static LSP specifies a static path. All routers that the LSP traverses must be configured manually with labels.
No signalling such as RSVP or LDP is required.
2. Signalled LSP
 LSPs are set up using a signalling protocol such as RSVP-TE or LDP. The signalling protocol allows labels to be
assigned from an ingress router to the egress router. Signalling is triggered by the ingress routers.
Configuration is required only on the ingress router and is not required on intermediate routers. Signalling also
facilitates path selection.

The Resource Reservation Protocol (RSVP) is a network control protocol used by a host to request specific qualities of
service from the network for particular application data streams or flows. RSVP is also used by routers to deliver quality
of service (QoS) requests to all nodes along the path(s) of the flows and to establish and maintain state to provide the
requested service. RSVP requests generally result in resources reserved in each node along the data path. MPLS
leverages this RSVP mechanism to set up traffic engineered LSPs. RSVP is not enabled by default and must be explicitly
enabled.
Label Distribution Protocol (LDP) is a protocol used to distribute labels in non-traffic-engineered applications. LDP
allows routers to establish label switched paths (LSPs) through a network by
mapping network-layer routing information directly to data link layer-switched paths.
The table above provides a high level comparison of LDP and RSVP-TE as label signaling protocols.

Module 3 | 144
© 2016 Nokia
Module 3 | 145
© 2016 Nokia
The list above can be used as a checklist when troubleshooting MPLS LDP issues.

Module 3 | 147
© 2016 Nokia
The 'show router ldp status' command displays overall LDP parameters including the administrative and operational
state of the protocol, the number of active and inactive neighbor relationships and protocol counter and error
statistics.
Running this command on all other routers within the network reveals LDP is up and all required sessions and peers are
operationally up as well.

The 'show router ldp parameters' command displays a summary of LDP parameters, including the default LDP settings
of Downstream Unsolicited label distribution, liberal label retention and Ordered Control Mode.
For this example, all LDP parameters throughout the network are correct.

The 'show router ldp interface' command displays the interfaces on which LDP has been configured and their state.
All LDP interfaces throughout the network are operational.

The 'show router ldp session' command displays the LDP Identifier, the state of the LDP session with each peer and
session related statistics. Use the ‘detail’ command to output more information of each session. A link adjacency type
indicates an interface (or direct) based LDP neighbor relationship.
Another option is to use the ‘show router ldp discovery’ command as it displays the status of the Hello adjacency
discovery and whether any neighbors have been discovered on those interfaces.
R5 shows three link sessions to neighboring routers and 1 targeted session to the far-end of SDP ID 8, which his correct.
All other routers in the network have the correct LDP sessions in the Established state.

The 'show router ldp bindings' command displays the contents of the LIB (Label Information Base) and should contain
all labels locally generated and those received from any LDP neighbors, no matter if they are in use or not.
The output above shows there is no entry for 10.10.10.8/32 in the LIB which is why the SDP is operationally down.
Follow the best path to R8 to see where the label is not forwarded.

R1 is showing entries in its’ LIB for 10.10.10.8. Next step is to check the LFIB on R1 to see if these entries are being
forwarded.

The 'show router ldp bindings active' command displays the contents of the LFIB (Label Forwarding Information Base)
and should contain all labels used for label switching packets and the associated label action.
The output above shows there is no entry for 10.10.10.8/32 which means R1 is aware of R8, but is not forwarding its’
label to any of the neighbors.

Module 3 | 155
© 2016 Nokia
To be able to “resolve” LDP prefixes and install forwarding entries in the LFIB, router R1 looks for an exact match for all
prefixes in the FIB, by default. In this case, R1 is looking for 10.10.10.8/32, but could not find an exact match in the
routing table and is therefore not installing in the LFIB.

The aggregate-prefix-match feature allows the more specific LDP prefixes to be resolved by an aggregate prefix in the FIB, and
overrides the default “exact” prefix match rule. Aggregate prefixes increase scalability by reducing the topology database and route
table sizes in large multi-area IGP deployments.

10.10.10.8/32 was forwarded from R1 to R5 successfully, however it is still only in the LIB. It must be installed in the LFIB for the SDP
to be operationally Up.

Enabling aggregate-prefix-match on R5 will install the 10.10.10.8/32 entry into the LFIB. SDP ID 8 will then go
operationally Up.

Module 3 | 160
© 2016 Nokia
The 'show router tunnel-table' command displays the tunnels that are enabled.

Important Notes:
1) Before enabling “debug”, the user must make sure a log is created to view the debug result. The following is an
example log created to view debug results. Note that if the log destination is session, when the session is closed, the
log (log-id) will not be saved.
SR12>config>log>log-id 3
SR12>config>log>log-id$ from debug-trace
SR12>config>log>log-id$ to session
SR12>config>log>log-id$ no shutdown
2) To stop the “debug”, use either of the following commands to stop the debug at different levels:
debug router bgp no packet - Disable debugging for OSPF packets
debug router no bgp - Disable debugging for all OSPF messages
no debug - Disable debugging for all applications
3) The “debug” will stop if a router is rebooted for some reason.


Module 3 | 164
© 2016 Nokia
Originally, the Resource Reservation Protocol (RSVP) was developed as a network control protocol that a host would use
to request specific qualities of service from the network for particular application data streams or flows. In addition,
RSVP has also been used by routers to deliver quality of service (QoS) requests to all nodes along the paths of the flows,
and to establish and maintain a state that provides the requested service. RSVP requests usually result in the
reservation of resources in each node along the data path. When used with MPLS, RSVP leverages this mechanism to set
up traffic engineered LSPs.
RSVP requests resources for simplex flows but only in one direction (unidirectional). Therefore, RSVP treats a sender as
logically distinct from a receiver, although the same application process may act as both a sender and a receiver at the
same time. Duplex flows require two LSPs, to carry traffic in each direction.
RSVP is not a routing protocol. It operates with unicast and multicast routing protocols that determine where packets
are forwarded. RSVP consults local routing tables to relay its messages.

The RSVP protocol has been extended to support the traffic engineering requirements in MPLS based networks.
RSVP-TE brings major benefits to MPLS networks. They:
 Provide access to detailed and customized path information to signal LSPs, which can be completely different from
IGP best path decisions.
 Facilitate the use of additional administratively defined attributes for links that enable more complex dynamic path
calculations, to increase resiliency and resource efficiency. This is an improvement to IGP shortest path calculations,
which are restricted by their use of a single parameter or metric, called the link cost, for Link-State Protocols.
 Provide access to preferred features, such as secondary path or Fast Reroute protection, which help to improve the
convergence times offered by standard routing protocols.
 Take resource reservation information into account during the LSP establishment process, which ensures that LSPs
only traverse routers that have sufficient resources available. This is called Connection Admission Control (CAC).
Using CAC allows operators to prevent resource overbooking.

By default, once every 30 seconds (the refresh-time), a Path and Resv refresh message will be sent. If the Path or Resv
message fails to reach its intended destination and the hold down timer (keep-multiplier) expires (by default 3 missed
updates, or 90 seconds), then the LSP will attempt to re-signal a secondary LSP if one is defined. If no secondary LSP is
defined then the Primary LSP simply terminates its session and drops the RSVP connection sending a Path tear down
message along the path.
This animated slide shows the Resv message being somehow dropped and therefore never reaches the tunnel head.
After a hold down timer (90 seconds), the router will tear down the primary path and use a secondary one if one is
available. If no secondary path is available, then the primary path is torn down and then attempted to be re-signaled. If
that is not possible then the path remains down.
refresh-time
Syntax refresh-time seconds
Context config>router>rsvp
Description The refresh-time controls the interval, in seconds, between the successive Path and Resv refresh
messages. RSVP declares the session down after it misses keep-multiplier number consecutive refresh messages.
Default 30 seconds
Parameters seconds — The refresh time in seconds.
Values 1 – 65535
keep-multiplier
Syntax keep-multiplier number
Context config>router>rsvp
Description The keep-multiplier number is an integer used by RSVP to declare that a reservation is down or the
neighbor is down..
Default 3
Parameters number — The keep-multiplier value.
Values 1 - 255

This list is an excellent troubleshooting checklist when dealing with MPLS/RSVP-TE signalling issues.

Module 3 | 169
© 2016 Nokia
Module 3 | 170
© 2016 Nokia
The command show router mpls lsp detail can be used to determine that the LSP is both administratively and operational “up”. Also,
this command can be used to determine which path is in active state.

The ‘show router status’ command is used to verify the active protocols in use on the router. OSPF is enabled for
provider core IGP routing. LDP was enabled in the previous exercise for label exchange.
RSVP is automatically enabled when MPLS is enabled.

The show router mpls interface command can be used to verify that MPLS is operational on the selected interfaces.

The show router rsvp interface command can be used to verify that RSVP is operational on the selected interfaces.

The output of the ‘show router mpls lsp <lsp-name> path detail’ shows the status and parameters associated with the
configured LSP, as well as the hop specified in the configuration, the actual hop taken by the LSP and the labels at each
hop.
Refer to the LAB guide for suggested actions based on the error Codes displayed in the above command for LSPs that
are in the down state.
Note:
When you are reading the Failure Code of a LSP, wait for the Retry period because the Failure Code could change for the
next retry period since for every retry period we will attempt to compute a path again if the last attempt to setup a LSP
failed.

The Lab Guide recommends the following actions for a Failure Code of “badNode”:
For a non cspf LSP with invalid strict/loose path

•Verify that all the routes/hops in the strict/loose path are valid and are in correct sequence according to the network
topology.
•Go to the node that is being reported by this failure code and verify the configuration.
For a cspf LSP

•The node that is reported will always be itself. This is because CSPF was unable to return a path to meet the
constraints of the LSP.
•Verify the TE database. Use the “tools perform router mpls cspf” command to further isolate the cause of the
problem.

Module 3 | 177
© 2016 Nokia
When using system IP address in the MPLS path hops, the IGP metrics will be taken into consideration when computing
the route to the next hop. Using the next-hop IP interface address instead of system IP addresses, will bypass the IGP
metrics and force a direct hop.

Module 3 | 179
© 2016 Nokia
The 7750 SR OS LSP diagnostics are implementations of LSP ping and LSP traceroute based on Internet Draft draft-
ietf-mpls-lsp-ping-xx.txt. LSP ping, as described in the draft, providing a mechanism to detect data plane failures in
MPLS LSPs. LSP ping and LSP traceroute are modeled after the ICMP echo request/reply used by ping and traceroute to
detect and localize faults in IP networks.
For a given FEC, LSP ping verifies whether the packet reaches the egress label edge router (LER), while in LSP traceroute
mode, the packet is sent to the control plane of each transit label switched router (LSR) which checks to see if it is
actually a transit LSR for the path.
lsp-ping
In-band LSP ping utility to verify LSP connectivity.
lsp-trace
In-band LSP traceroute command to determine the hop-by-hop path for an LSP.

Module 3 | 181
© 2016 Nokia
This command can only be used if Traffic Engineering is enabled in the IGP as it relies on the traffic engineering
database. The Traffic Engineering database will not be fully populated if TE is not enabled on all nodes in the network
and if interfaces are not MPLS or RSVP enabled. The output of the command is strictly informational and has no impact
on any LSPs. If the CSPF calculation fails it indicates that there is no path in the TE database to reach the endpoint.
This tools command can be used similar to ping to isolate the source of the problem. For example, if the CSPF
calculation fails to find the desired endpoint the next node in the path can be specified as the endpoint. This can be
continued until the failing node is isolated.

Please refer to the lab guide. A full mesh of LSPs are created within the network.

Module 3 | 184
© 2016 Nokia
Module 3 | 185
© 2016 Nokia
Module 3 | 186
© 2016 Nokia
The Nokia Router is based on the service model where service edge routers are deployed at the provider edge. Services
are provisioned on the router and transported across an IP and/or IP/MPLS provider core network in encapsulation
tunnels created using Generic Router Encapsulation (GRE) or MPLS Label Switched Paths (LSPs).
The service model uses logical service entities to construct a service. The logical service entities are designed to provide
a uniform, service-centric configuration, management, and billing model for service provisioning.
The Service model is based on the following components:
 Subscribers
 SAP
 Customer
 Service
 SDP
 Service Tunnels

Module 3 | 188
© 2016 Nokia
Useful troubleshooting commands:
 show service id <id> base
 show service id <id> sap <sap-id> detail
 show port <port-id>
 show service sap-using

The show service id <service-id> base command is excellent for viewing the overall status of a service and its objects
(SAPs, SDP bindings, etc.) In the output above, we can see that SAP 1/1/2:1 is administrative up, but operationally
down.

Show service id <service-id> sap <sap-id> detail: This command displays information about the specified SAP for the
service.
The Flag value is very important for any troubleshooting of SAPs. Refer to Appendix B of the Lab guide for possible
problems and suggested actions based on the Flag Values.
In the output above, the Flag is set to “PortMTUTooSmall”. The previous slide showed the service MTU set at 1514 and
this slide shows the SAP at an MTU size of 1514 as well. Because dot1q encapsulation is being used, the SAP must be
greater than the service MTU by 4 bytes.

The physical port MTU is set to 1514 which is too low. To fix this issue, change the MTU size of port 1/1/2 to 1518 or
greater.

Module 3 | 193
© 2016 Nokia
Useful Troubleshooting Commands:
 show service id <id> base
 show service id <id> sdp <sdp-id> detail
 show service id <id> labels
 show service sdp-using
 show router ldp bindings
 show service sdp

The show service sdp-using command displays SDP Binding information. If no optional parameters are specified, a
summary output for all SDP Bindings is displayed.

Show service id <service-id> sdp <sdp-id> detail: This command displays information about the specified SDP binding
for the service.
The Flag value is very important for any troubleshooting of sdp bindings. Refer to Appendix B of the Lab guide for
possible problems and suggested actions based on the Flag Values.

Useful troubleshooting commands:
 show service sdp <sdp-id>
 show router ldp session
 show router route-table
 show router mpls lsp
 show router ldp bindings

The show service sdp command displays SDP information. If no optional parameters are specified, a summary output
for all SDPs is displayed.

show service sdp <sdp-id> detail: This command displays information about the specified SDP.
The Flag value is very important for any troubleshooting of sdp bindings. Refer to Appendix B of the Lab guide for
possible problems and suggested actions based on the Flag Values.

Module 3 | 200
© 2016 Nokia
Module 3 | 200
© 2016 Nokia
Module 3 | 201
© 2016 Nokia
•VPLS is a class of L2 VPN service that allows multiple customer sites to be connected in a single bridged domain
•Customer sites in the VPLS appear to be on the same LAN, even if the sites are geographically dispersed
•Devices perform MAC learning
•Uses T-LDP signaling, and MPLS or GRE as the service tunnel transport
•Can use HVPLS to eliminate the need for a full mesh of virtual circuits between devices in a VPLS

Module 3 | 203
© 2016 Nokia
Module 3 | 204
© 2016 Nokia
The cpe-ping extends the MAC Ping OAM tool to detecting end-station IP addresses inside a VPLS. A CPE ping for a
specific destination IP address within a VPLS will be translated to a MAC-ping towards a broadcast MAC address. Upon
receiving such a MAC ping, each peer PE within the VPLS context will trigger an ARP request for the specific IP address.
The PE receiving a response to this ARP request will report back to the requesting device.
Note:
It is important to use a non-existent source address so the customer ARP cache is not polluted with invalid ARP entries.

Show service id <service-id> base: This command is very useful when trying to get a snapshot of the service and status
of SAPs and SDPs.
In the output above, sdp:443:4000 to R7 (10.10.10.7) is operationally Down. This is why the CPE ping command in the
previous slide fails.

The details of Mesh-SDP 443:4000 includes a Flag value set to SdpOperDown, which means that the associated SDP is
Down. Troubleshooting efforts must now turn to SDP ID 443.

The details of SDP ID 443 includes a Flag value set to TranspTunnDown. Reference to Appendix B in the lab guide
recommends to view the status of the associated LSP.

The associated LSP “R5-R7-strict-B” is operationally Down. Troubleshooting efforts must now turn to LSP “R5-R7-
strict-B”.

The output is showing a Failure Code of badNode with an associated Failure Node of 10.1.5.1. IP address 10.1.5.1 is
located on router R1.

The MPLS interfaces shows “to-R3” operationally Down. The network interfaces shows “to-R3” (10.1.3.1) is
administratively Down. The previous slide showed 10.1.3.3 as the second strict hop in the LSP path. 10.1.3.1 is the
direct neighbour of 10.1.3.3, therefore meaning that there is no connectivity for this strict hop.
To fix the problem, set the administrative state of interface to-R3 to UP. Once this is completed, view the service to see
if the mesh-sdp binding is operationally UP.

Module 3 | 212
© 2016 Nokia
Module 3 | 213
© 2016 Nokia
The show service id <id> 4000 fdb detail command displays the MAC entries contained within the FDB.
The show service id <id> fdb command displays FDB info for a particular service. The output above shows the table size
set to 5, while there are currently 4 entries in the FDB. The high watermark is set to 80% which means an alarm should be
present. Alarms can be viewed in log-id 99.
*A:R5# show log log-id 99
===============================================================================
Event Log 99
===============================================================================
Description : Default System Log
Memory Log contents [size=500 next event=516 (wrapped)]
514 2011/08/18 21:23:12.07 UTC MINOR: SVCMGR #2104 Base

"FDB table utilization of service 4000 (customer 1) crossed its high watermark“
Use the command show service fdb-info to display global FDB info.
Use the command show service fdb-mac to display global FDB entries or an FDB entry for a particular MAC.
To clear the FDB, use the clear service id <id> fdb command.

Module 3 | 215
© 2016 Nokia
The Path MTU Discovery tool provides a powerful tool that enables service providers to get the exact MTU supported
between the service ingress and service termination points (accurate to one byte).
It is important to understand the MTU of the entire path end-to-end when provisioning services.

SDP Ping performs in-band uni-directional or round-trip connectivity tests on SDPs. The SDP Ping OAM packets are sent
in-band, in the tunnel encapsulation, so it will follow the same path as traffic within the service. The SDP Ping response
can be received out-of-band in the control plane, or in-band using the data plane for a round-trip test.
For a unidirectional test, SDP Ping tests:

 Egress SDP ID encapsulation
 Ability to reach the far-end IP address of the SDP ID within the SDP encapsulation
 Path MTU to the far-end IP address over the SDP ID
 Forwarding class mapping between the near-end SDP ID encapsulation and the far-end tunnel termination

For round trip connectivity testing, the resp-sdp keyword must be specified. If resp-sdp is not specified, a uni-
directional SDP test is performed.
For a round-trip test, SDP Ping uses a local egress SDP ID and an expected remote SDP ID. Since SDPs are unidirectional
tunnels, the remote SDP ID must be specified and must exist as a configured SDP ID on the far-end 7750 SR. SDP round
trip testing is an extension of SDP connectivity testing with the additional ability to test:
 Remote SDP ID encapsulation
 Potential service round trip time
 Round trip path MTU
 Round trip forwarding class mapping

Nokia's Service Ping feature provides end-to-end connectivity testing for an individual service.
Service Ping operates at a higher level than the SDP diagnostics in that it verifies an individual service and not the
collection of services carried within an SDP.
Service Ping is initiated from a 7750 SR router to verify round-trip connectivity and delay to the far-end of the service.
Nokia's implementation functions for both GRE and MPLS tunnels and tests the following from edge-to-edge:
 Tunnel connectivity
 Service (VC) label mapping verification
 Service existence
 Service provisioned parameter verification
 Round trip path verification
 Service dynamic configuration verification
SDP Path Used determines whether the Originating 7750 SR used the originating SDP-ID to send the svc-ping request.
If a valid originating SDP-ID is found operational and has a valid egress service label, the originating 7750 SR should use
the SDP-ID as the requesting path. If the originating 7750 SR uses the originating SDP-ID as the request path, “Yes” is
displayed. If the originating 7750 SR does not use the originating SDP-ID as the request path, “No” is displayed. If the
originating SDP-ID is non-existent, N/A is displayed.

For a MAC ping test, the destination MAC address (unicast or multicast) to be tested must be specified. A MAC ping
packet can be sent through the control plane or the data plane. When sent by the control plane, the ping packet goes
directly to the destination IP in a UDP/IP OAM packet. If it is sent by the data plane, the ping packet goes out with the
data plane format.
In the control plane, a MAC ping is forwarded along the flooding domain if no MAC address bindings exist. If MAC address
bindings exist, then the packet is forwarded along those paths (if they are active). Finally, a response is generated only
when there is an egress SAP binding to that MAC address. A control plane request is responded to via a control reply
only.
In the data plane, a MAC ping is sent with a VC label TTL of 255. This packet traverses each hop using forwarding plane
information for next hop, VC label, etc. The VC label is swapped at each service-aware hop, and the VC TTL is
decremented. If the VC TTL is decremented to 0, the packet is passed on to the management plane for processing. If
the packet reaches an egress node, and would be forwarded out a customer facing port, it is identified by the OAM label
below the VC label and passed to the management plane.
MAC pings are flooded when they are unknown at an intermediate node. They are responded to only by the egress
nodes that have mappings for that MAC address.

A MAC trace functions like an LSP trace with some variations. Operations in a MAC trace are triggered when the VC TTL is
decremented to 0. Like a MAC ping, a MAC trace can be sent either by the control plane or the data plane.
For MAC trace requests sent by the control plane, the destination IP address is determined from the control plane
mapping for the destination MAC. If the destination MAC is known to be at a specific remote site, then the far-end IP
address of that SDP is used. If the destination MAC is not known, then the packet is sent unicast, to all SDPs in the
service with the appropriate squelching.
A control plane MAC traceroute request is sent via UDP/IP. The destination UDP port is the LSP ping port. The source
UDP port is whatever the system gives (note that this source UDP port is really the demultiplexor that identifies the
particular instance that sent the request, when correlating the reply). The source IP address is the system IP of the
sender.
When a traceroute request is sent via the data plane, the data plane format is used. The reply can be via the data plane
or the control plane. A data plane MAC traceroute request includes the tunnel encapsulation, the VC label, and the
OAM, followed by an Ethernet DLC, a UDP and IP header. If the mapping for the MAC address is known at the sender,
then the data plane request is sent down to the known SDP with the appropriate tunnel encapsulation and VC label. If it
is not known, then it is sent down every SDP (with the appropriate tunnel encapsulation per SDP and appropriate egress
VC label per SDP binding).
The tunnel encapsulation TTL is set to 255. The VC label TTL is initially set to the min-ttl (default is 1). The OAM label
TTL is set to 2. The destination IP address is the all-routers multicast address. The source IP address is the system IP of
the sender. The destination UDP port is the LSP ping port. The source UDP port is whatever the system gives (note that
this source UDP port is really the demultiplexor that identifies the particular instance that sent the request, when
correlating the reply). The Reply Mode is either 3 (i.e., reply via the control plane) or 4 (i.e., reply via the data plane),
depending on the reply-control option. By default, the data plane request is sent with Reply Mode 3 (control plane
reply).
The Ethernet DLC header source MAC address is set to either the system MAC address (if no source MAC is specified) or
to the specified source MAC. The destination MAC address is set to the specified destination MAC. The Ethertype is set
to IP.

MAC Populate is used to send a message through the flooding domain to learn a MAC address as if a customer packet
with that source MAC address had flooded the domain from that ingress point in the service. This allows the provider to
craft a learning history and engineer packets in a particular way to test forwarding plane correctness.
The MAC populate request is sent with a VC TTL of 1, which means that it is received at the forwarding plane at the first
hop and passed directly up to the management plane. The packet is then responded to by populating the MAC address
in the forwarding plane, like a conventional learn although the MAC will be an OAM-type MAC in the FIB to distinguish it
from customer MACs addresses.
This packet is then taken by the control plane and flooded out the flooding domain (squelching appropriately, the
sender and other paths that would be squelched in a typical flood).
This controlled population of the FIB is very important to manage the expected results of an OAM test.
The same functions are available by sending the OAM packet as a UDP/IP OAM packet. It is then forwarded to each hop
and the management plane has to do the flooding.
Options for MAC Populate are: 1. to force the MAC in the table to type OAM (in case it already existed as dynamic or
static or an OAM induced learning with some other binding), 2. to prevent new dynamic learning to over-write the
existing OAM MAC entry, 3. to allow customer packets with this MAC to either ingress or egress the network, while still
using the OAM MAC entry.
Finally, an option to flood the MAC Populate request causes each upstream node to learn the MAC (i.e., populate the
local FIB with an OAM MAC entry), and to flood the request along the data plane using the flooding domain.
An age can be provided to age a particular OAM MAC after a different interval than other MACs in a FIB.
The oam mac-purge command is used to clear the FIBs of any learned information for a particular MAC address. This
allows one to do a controlled OAM test without learning induced by customer packets. In addition to clearing the FIB of a
particular MAC address, the purge can also direct the control plane not to allow further learning from customer packets.
This allows the FIB to be clean, and be populated only via a MAC Populate.




Module 3 | 226
© 2016 Nokia
Virtual Private Routed Network (VPRN) is the Nokia Service implementation of MPLS Layer 3 VPN
which is based on RFC4364
Virtual Private Routed Networks allow multiple customer sites to communicate securely at the IP
level over a provider-managed MPLS network.
A VPRN service distributes its customer’s routing information using MP-BGP and forwards their data
packets using MPLS (or GRE ) tunnels.
As shown in the slide, a single provider-managed IP/MPLS infrastructure permits the deployment of
multiple, distinct customer routed networks that are fully isolated from each other.

VPRN service uses VPN routing and forwarding instances (VRFs) within a PE device to maintain forwarding information on
a per-site basis. A VRF is a logical private forwarding (routing) table created within a PE router. It securely isolates the
routing information of one customer from the next, and also from the routes of the provider core network.
Each PE can maintain multiple separate VRFs based on the number of customer sites it connects to.

All customer routes are propagated across the provider core as VPN-IPv4 routes, advertised in multiprotocol BGP
sessions between PE devices. This ensures the scalable propagation of customer VPRN routing information.
Note: there is one MP-BGP session between PEs, and the same session is used to forward routing updates for different
VRFs.
The receiving PE will receive a VPN-IPv4 route from a remote PE and install the IPv4 route into the corresponding VRF.
The receiving PE must have a criteria on which to base this decision, since it is receiving routes from many different PEs,
each serving many different customers.
.
The route distinguisher is defined solely to create unique VPN-IPv4 addressing, which allows overlapping addresses
from different customers to be transported uniquely across the provider core. It is never intended to define VPRN
membership and ,therefore, a new identifier, separate from the route distinguisher, is required to associate a route to a
VRF and define the VPRN membership of the route.
A route target is defined to address this issue. A route target (RT) is the closest approximation to a VPRN membership
identifier in the VPRN architecture, and identifies the VRF table that a prefix is associated with to the receiving PE. One
or more route targets can be associated to any route.
In simple VPRN cases and for provisioning consistency, the route target value chosen is often the same as the route
distinguisher value, however, they should not be interpreted as meaning the same thing.

Standard iBGP design rules apply to the MP-BGP sessions. To ensure proper VPRN route and label propagation between
all sites, the MP-BGP sessions should exist in a full mesh (or equivalent) arrangement between PE devices.
A properly designed and implemented core of route reflectors, or a combination of full mesh and route reflectors, is
considered to be the equivalent of a full mesh.

After the establishment of the routing topology in the provider core, the network must be MPLS-enabled to support
services such as VPRN.
Transport tunnels must be created between the PEs. The transport tunnel is either a MPLS LSP or a GRE point-to-point
tunnel between PEs.
These tunnels serve as the label switched paths the customer packets will take as they cross the provider core network.
Each PE involved in a given VPRN service must be configured with a tunnel to every other PE participating in the same
VPRN service in order to transport a customer’s VPRN traffic from one site to another.
A full mesh of transport tunnels is required between participating PE devices.

Notes:
1. Enable the VPRN if it is administratively down.
2. Enable the interface. Check the status of the port that the interface/SAP is bound to.
3. Confirm that BGP sessions are established and both local and remote family is configured for MP-BGP (VPN-IPv4)
4. Configure a Route Distinguisher. An RD must be configured even if the VPRN is local.
5. Import targets on the local PE must match the export targets on the remote PEs. Verify the route targets being
advertised by the local node by running show router bgp routes <prefix> hunt. Also check that the remote routes
are in the RIB-IN by using the show router bgp routes command.

Notes:
6. The VPRN must be bound to an operational SDP for the next-hop of the VPRN route OR Auto-Bind LDP OR GRE must
be configured.
 If spoke-sdp are used: show service sdp <sdp-id>
 If auto-bind LDP is used: show router ldp bindings prefix <vpn next-hop/32> active
 If auto-bind GRE is used: show router route-table <vpn next-hop/32>
7. Confirm the policy configuration is correct.

Module 3 | 234
© 2016 Nokia
The ping router <router-instance> command confirms there is no connectivity from R5 to 192.168.40.1.

The output above shows the IP Address 192.168.40.1 is missing from the VPRN’s routing table.

Use the show service <id> base command for an overview of the service and its’ components.
• The service and all components are operational on routers R5 and R8
Use the show router <service-id> interface command to view interfaces for a particular service.
• All VPRN interfaces on R5 and R8 are operationally up.

show router bgp neighbor – From the output of this command, confirm that the BGP sessions are established and both
local and remote family is configured for VPN-IPv4.
show router bgp summary - This command displays a summary of BGP neighbor information. It is an excellent
command to use when checking which BGP neighbors are operational due to the smaller output as compared to the
show router bgp neighbor command.
Executing these commands on routers R5, R6, R7 and R8 show that all required BGP neighbors are operationally up and
VPN-IPv4 (MP-BGP) is running.

The show service <id> base command can be used to view the configured Route Distinguisher.
The Route Distinguisher in the output above is correct.

Import targets on the local PE must match the export targets on remote PEs.
Verify that the expected routes from remote PEs are in the global BGP routing table (BGP RIB-IN). If there are no VPRNs
with an import target on the PE that matches the target in a received route the PE router will silently drop the
advertise route.
Use the show router bgp routes <prefix> hunt command to verify the networks and communities within both the RIB-
IN and RIB-OUT.
The output from R5 above shows a missing RIB-IN entry from 192.168.40.1/24. The RIB-OUT is advertising entries to
all 3 other routers within the VPRN.

The output from R8 above shows no RIB-OUT entries which would explain why there is no RIB-IN entry for 192.168.40.1
in R5 (on the previous slide).

A transport tunnel to the VPN routes next hop must be active before the route will get installed into the VRF. There
were no entries in the RIB-OUT of R8 so start with that router. Check to see if there is a transport tunnel to R5.
Use the following steps to check if the transport tunnel to the next-hop of the VPRN route is operational.
1. show service id <id> base

 A spoke-sdp, auto-bind ldp or auto-bind gre must be configured.
2. If spoke-sdps are used for the transport, use show service sdp <sdp-id> to verify the status of the associated SDP.
3. If auto-bind LDP is configured, confirm that there is an active binding for the VPN next-hop address by running
show router ldp bindings prefix <vpn next-hop/32> active.
4. If auto-bind GRE is configured, confirm that there is a route in the routing table for the VPN next-hop address by
running show router route-table <vpn next-hop/32>.
From the output of show service id 100 base, auto-bind LDP can be seen as the type of transport being used.
Verify that R5 has an active binding to R8 and R8 has an active binding to R5. From the output above R5 and R8 have an
active binding to each other.

Check to see if there are any import or export policies configured and if so, verify the policy configuration.
show service <id> base will show if there are any import or export policies configured for the service.
From the output above, the VPRN 100 on both R5 and R8 have an import and export policy configured.
Use the show router policy <policy-name> to view the configuration of each policy. There seems to be an error in the
configuration of the policy “export-VPRN100” on R8. When adding a community, the name of the community must
be specified, not the route target identifier. R8 above is showing a route target identifier of “target:65535:100” as
the community. This needs to be changed to the correct community name of “exportVPRN100”.

Apply the fix to policy “export-VPRN100” on R8 and verify the fix. Using vprn-ping shows that R5 is now able to reach all
the other sites within VPRN 100.

Module 3 | 245
© 2016 Nokia



Module 3 | 249
© 2016 Nokia
Module 3 | 250
© 2016 Nokia
Module 3 | 251
© 2016 Nokia
Module 3 | 252
© 2016 Nokia
This process happens twice, first on the ingress Fast Flexible Forwarding Complex (FFPC), and then on the egress FFPC.

QoS involves mapping the appropriate bits in a protocol data unit header at OSI layer 2, 3, or 4, to indicate different
levels of service. It is up to the administrator to ensure that no errors arise while classifying and marking packets in a
QoS-aware network.
Queues used for different types of traffic can have properties tailored to the type of traffic for which they are used, for example:
• Length/size of queue, based on delay requirements and burstiness of traffic.
• Discard policies, based on application’s sensitivity to packet loss.
Scheduling packets out of their queues is an extremely complex and variable process. There are a few types of
scheduling available. Strict Priority and Round Robin are example of simple schedulers, which service queues in order,
regardless of the amount of traffic transmitted from them. Fair Queuing is an adaptive scheduler, which dynamically
adapts the allocation of servicing time to each queue, based on the amount of traffic (for example, the number of bits)
recently transmitted from that queue.

Nokia divides the bandwidth over the queues and offers three possible options from which a designer or operator can
choose.
The first possibility is the Default Scheduler. This scheduler, enabled by default, gives priority first to the Expedited
queues within CIR, then to the Best Effort Queues within CIR and then, finally, to all queues above CIR. The default
scheduler is also called a single-tier hardware scheduler because all the queues are linked directly into the FFPC
scheduler.
The second possibility (Release 4 onwards) is the Hierarchical Scheduler, which overrides the default scheduler and can
only be implemented on access ports (ingress and/or egress). This type of scheduling distinguishes eight strict priority
levels among queues serviced within CIR, and eight levels above CIR. Each level then divides the bandwidth, using
Weighted Fair Queuing.
The third type of scheduler (Release 6 onwards) is the Egress Port Scheduler, which can only be implemented on egress
ports (access and/or network). It overrides the Default Scheduler and can be used in combination with hierarchical
scheduling. This type of scheduling can be seen as an advanced technique, if compared to the hierarchical scheduling
method. The two approaches are very similar, but the Egress Port Scheduler takes the real speed of the line and the
lower OSI-levels’ overhead into consideration, and delivers a much more granular division of bandwidth to the network
egress port (previously only default scheduling).
The Hierarchical and Egress Port schedulers are multi-tiered virtual schedulers. Queues are linked in a family-like
structure so that “child” queues or schedulers can only receive bandwidth from their virtual “parent” schedulers, which
exist between the queues and the master hardware scheduler of the FFPC. Through hierarchical virtual schedulers, the
Nokia 7750 allows service providers to design customized and sophisticated scheduling. By associating queues with a
hierarchical virtual scheduler, the PIR and CIR of multiple queues can be dynamically modified. Ultimately, all queues and
schedulers are linked to the FFPC hardware scheduler.
The Nokia 7750 SR/ESS supports the three types of scheduling.

Module 3 | 256
© 2016 Nokia
The show service id <id> sap <sap-id> stats command will output traffic statistics per QoS queues. Dropped packets
signal a potential problem with the assigned QoS policy.
Because these values are running totals, it is a good idea to clear the statistics before running these commands.

The show service id <id> sap <sap-id> qos command gives an overview of all QoS related policies that are assigned to
the SAP. In the example above, sap 1/1/4 is using Ingress qos-policy = 77. The next step is to verify the configuration
of this QoS policy.

The show qos sap-ingress <policy-id> detail command outputs the configuration of a specific QoS policy. The above
R5 output shows traffic directed to destination port 1234 is assigned to forwarding class ef. FC ef is assigned to Queue
3 which is assigned a CIR and PIR of 200.
Comparing this to R7, we see FC ef also assigned to Queue 3, however the CIR and PIR are set much higher at 2000.
From this output, it seems the value of 200 for CIR and PIR on R5 is incorrect and too low.

Module 3 | 260
© 2016 Nokia
Module 3 | 261
© 2016 Nokia
The source node network QoS policy:
egress
fc ef
dscp-out-profile ef
lsp-exp-out-profile 5
…
The destination node network QoS policy:

ingress
dscp ef fc ef profile in
lsp-exp 5 fc l2 profile out
…

Module 3 | 263
© 2016 Nokia
Module 3 | 264
© 2016 Nokia
Multicast is the answer to efficient, scalable, large-scale data delivery.
The source sends a single copy of a packet that is addressed to a group of receivers. This group is a logical entity to
which any device can choose to listen (or not) at anytime. Efficient, low-overhead transport layer services are provided
by UDP.
Unlike unicast traffic, only one copy of any packet is required regardless of the number of receivers, as many receivers
can join the same group and therefore all can receive a copy of the same packet.
Unlike broadcast traffic, multicast will span across routers if configured to do so. Devices that do not want the packet
may not receive the data at all, and if they do receive the frame, they will discard it at Layer 2.

Shown here is an illustration of a multicast network. The source and receivers are now separated by several routers.
There are multiple LAN segments, each containing multiple potential receivers.
The lines representing the data flow show only a sample of how multicast data would be distributed in this network if
the routers were multicast-enabled and Receivers 1, 3 and 6 had joined the multicast group.
It is important to note that the source is sending a single copy of a packet, but it is being replicated at several points in
the network. Router B receives one packet from the source and sends out two. Similarly, Router D receives a single
packet and sends out two.
This packet replication by the routers is something not typically seen in a unicast environment.

Multicast Distribution Trees
Multicast data forwarding is based on the behavior of trees. The logical path taken through the network is called the Multicast
Distribution Tree (MDT). The type of MDT will vary depending on the choice of multicast routing protocol configured. The role of the
multicast routing protocol is to create the MDT by signaling, so that packets may then follow this logical path.
In a Dense Mode protocol implementation, the type of tree used at all times is called a Source Path Tree. The root of the tree is at the
source device, and data flows from the root to the leaves via the branches. This tree is alternatively referred to as a Shortest Path
Tree. This is because the path taken by the multicast packets (created by the protocol signaling) is the shortest path as determined by
the underlying unicast routing protocol.
Sparse Mode protocols (PIM SM) may use a Source Path Tree, but it must first start with the Shared Path Tree. The Shared Path Tree
is a tree rooted at the Rendezvous Point (RP), which is the meeting point in the network where the source and receivers must first
meet to establish the multicast flow. It is only after this initial meeting has been completed that Sparse Mode protocols will utilize the
Source Path Tree.
RPF Check
Multicast packets are handled in a different fashion than unicast packets when they are received by a router. Once the frame has been
received and the packet is accepted by the router, a validation is always performed on the source IP address of each received packet.
This check is called the Reverse Path Forwarding (RPF) check. The packets that fail are silently discarded. This is in contrast to the
unicast IP world, where a router commonly will generate an Internet Control Message Protocol (ICMP) message when it discards a
packet.
Designated Router and Neighbors

As an initial verification of the multicast topology, it is usually a good plan to first verify that neighbors have been established and that
all expected neighbors are seen on the interfaces expected. When PIM is enabled, PIM Hellos are sent to 224.0.0.13 with a TTL of 1
every 30 seconds. Any directly connected PIM-enabled router should be listed as a neighbor. After neighbors have been discovered, or
some time has passed without hearing any Hellos, all multi-access networks have a Designated Router Election. The DR will be elected
based on the configured interface DR-priority value, which by default is set to 1, and is equal on all routers. Since all routers have equal
priority, the highest interface address on the multi-access network is used to determine the election winner.
Querier Election
Host Membership Query messages are sent by routers to the local network control block address of 224.0.0.1 for all multicast-enabled
devices. In order to discover which multicast groups have receivers present, the router issues a query on each of its attached
interfaces on which IGMP is enabled. The queries are sent every 125 seconds by default. The router elected as the Querier is the one
that controls the query messages.

Module 3 | 268
© 2016 Nokia
The show router pim group command displays the summarized group forwarding table.
The output above shows we have a group, however it is showing as (*,G) which means it does not know about the
source. It should be showing (S,G) instead.

The show router pim group detail command displays detailed group information. This includes data that can be used
for verifying if traffic is being received (Curr Fwding Rate, Forwarded Packets, etc.)
The output above shows a Curr Fwding Rate = 0.0 kbps, which means there is no traffic flowing. This command can also
be used to verify if traffic is flowing, once the issue is fixed.

The show router pim interface command verifies that PIM is enabled on the appropriate interfaces and the address of
the DR for each multi-access network.
The above output shows that PIM is enabled on all appropriate interfaces for R7.

This command displays PIM neighbor information. This can be important if an interface has more than one adjacency;
for example, a LAN-interface configuration with three routers connected all running PIM on their LAN interfaces. These
routers then have two adjacencies on their LAN interfaces, each with different neighbors. If the address ip-address
parameter is not defined in this example, then the show command output would display two adjacencies.
The output above shows R7 having two neighbours that are operational.

The show router pim rp command verifies the RP-set of the local router, which should be consistent on all routers.
The output above shows R7 has an RP configured with an address of 10.10.10.1 and group address of 239.0.0.0/8
which is correct.
At this point, all configuration on R7 looks correct so the troubleshooting efforts will turn to the RP, which is R1
(10.10.10.1).

The above command shows that the RP is also showing a type of (*,G) instead of the desired (S,G). Because of this,
check the PIM interfaces.

Module 3 | 275
© 2016 Nokia
Verification on R7 now shows a type of (S,G) with a source address of 192.168.11.100 as well as a Forwarding Rate of
1026.8 kbps.

Module 3 | 277
© 2016 Nokia
The show router pim statistics command displays counters for all PIM message types.

This command traces the multicast path from a source to a receiver by passing a trace query hop-by-hop along the
reverse path from the receiver to the source. At each hop, information such as the hop address, routing error
conditions and packet statistics is gathered and returned to the requestor. Network administrator can determine where
multicast flows stop and verify the flow of the multicast stream. The “?” is displayed if the reverse DNS lookup fails.

This command is used to display relevant multicast information from the target multicast router. Information displayed
includes adjacency information, protocol, metrics, thresholds and flags from the target multicast router.
Flag Values:
tunnel - a multicast path exists between the source (10.10.10.66 in the example above) and the named host via
encapsulated IP. Lines with no “tunnel” are native multicast links to other multicast routers within the network.
pim - router is running the PIM protocol.
querier - router is an IGMP querier.
leaf - router is a leaf router (directly attached to the receivers).

This command traces a multicast path from a source to a receiver and displays multicast packet rate and loss
information. The mstat command adds the capability to show the multicast path in a limited graphic display and
provide drops, duplicates, TTLs and delays at each node. This information is useful to network operator because it
provides nodes with high drop and duplicate counts. Duplicate counts are shown as negative drops.

Additional commands and options are available to debug, verify and troubleshoot multicast on the 7750 SR.

Module 3 | 283
© 2016 Nokia

Module 3 | 285
© 2016 Nokia
Module 3 | 286
© 2016 Nokia
Module 3 | 287
© 2016 Nokia
Module 3 | 288
© 2016 Nokia

Nokia Advanced Troubleshooting SG v3.0.2

Uploaded by

Copyright:

Available Formats

You might also like

Nokia Advanced Troubleshooting SG v3.0.2

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nokia Advanced Troubleshooting SG v3.0.2

Uploaded by

Copyright:

Available Formats

Module 0 | 1

Advanced Troubleshooting v3.0.2

Module 0 | 2 Advanced Troubleshooting v3.0.2 © 2016 Nokia

 Technical Practices for the specific product.

Module 0 | 8 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Three areas that require special attention are:

Module 0 | 10 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

SLAs – Always On:

Module 0 | 11 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 0 | 12 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Improved by the following factors:

Module 0 | 13 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Can range from a few milliseconds, as in the case of

Module 0 | 14 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 0 | 15 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 0 | 17 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 0 | 19 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 0 | 21 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 0 | 22 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 0 | 23 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 5 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 6 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 7 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 8 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 9 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 10 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 11 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 12 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

log filter 1001

Module 2 | 14 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Displays a list of all application names that show log applications

Displays event control settings for events show log event-control

Displays event file log information. show log file-id [file-id]

Displays an event log summary with show log log-id [log-id]

Module 2 | 15 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 16 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 18 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 19 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 20 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 21 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

The numeric range of event logs is from 1-100.

The following is a list of match criteria that can be used as of FG8.0R4:

[no] application - Specify application id to match

Module 2 | 22 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 23 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

These log files are numerically limited from 101-199 .

Module 2 | 24 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 25 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 26 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

In a smaller installation, only a single central server may be deployed.

Module 2 | 27 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 28 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

address – Specifies the target syslog host address.

Module 2 | 29 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 30 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 33 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 34 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 36 Advanced Troubleshooting v.3.0.2 © 2016 Nokia

Module 2 | 40 Advanced Troubleshooting v.3.0.2 © 2016 Nokia