IPanema Troubleshooting Guide V5

IPANEMA
Troubleshooting
guideline
th
10 of may 2011
Version 5.0
Troubleshooting guideline IPANEMA
document control
date version no. author change/addition
24 of march
th
1 A.SEITZ creation
2006
20 of oct th
2 A.SEITZ updates
2008
16 of march
th
3 A.SEITZ - debug command (uncomplete)
2009 - new hardwares (5v2, 120ax, 140ax, 1000axT,
1800axT and 1800axSX)
06 of july th
4 A.SEITZ updates (smartpath), Xcomp SRE & ZRE , Xapp,
2010 Cluster
10 of may
th
5 A.SEITZ debug show flow
2011
© copyright, Equant 2006

All rights reserved.
The information contained in this document is the property of Equant and its affiliates and subsidiary companies
forming part of the Equant group of companies (individually or collectively). No part of this document may be
reproduced, stored in a retrieval system, or transmitted in any form or by any means; electronic, mechanical,
photocopying, recording, or otherwise, without the prior written permission of Equant. Legal action will be taken
against any infringement.
Equant is part of the France Telecom group and operates under the name, Orange Business Services.
© Copyright Equant 10th of may 2011

Internal Use Only 2 of 109
table of contents
1 INTRODUCTION......................................................................................................................................... 5
1.1 PURPOSE OF THIS DOCUMENT ................................................................................................................... 5
1.2 INTENTED AUDIENCE ............................................................................................................................... 5
2 QUALITY OF SERVICE TROUBLESHOOTING ................................................................................... 6
2.1 DIAGNOSE THE PROBLEM ......................................................................................................................... 6
2.2 SITE TROUBLESHOOTING .......................................................................................................................... 7
2.3 QOS PROFILE TROUBLESHOOTING ........................................................................................................... 8
2.4 SUPERVISION ............................................................................................................................................ 9
2.5 CONFIGURATION .................................................................................................................................... 10
3 IP|BOSS TROUBLESHOOTING.............................................................................................................. 11
3.1 IP|BOSS FILE SYSTEM ............................................................................................................................. 11
3.2 TCP/IP COMMUNICATIONS TO OR FROM IP|ENGINES ............................................................................. 11
4 IP|ENGINES TROUBLESHOOTING ...................................................................................................... 13
4.1 INTERFACE MAPPING .............................................................................................................................. 13
4.2 PROCESS CHAIN INSIDE IPENGINES ......................................................................................................... 14
4.3 PHYSICAL LAYER CONNECTIVITY PROBLEM ........................................................................................... 17
4.4 NETWORK LAYER CONNECTIVITY PROBLEM .......................................................................................... 22
4.5 CONNECTIVITY WITH IPBOSS .................................................................................................................. 27
4.6 SYNCHRONISATION PROBLEM ................................................................................................................ 28
4.7 APPLICATION LAYER CONNECTIVITY PROBLEM ...................................................................................... 30
4.8 IP|TRUE TROUBLESHOOTING .................................................................................................................. 35
4.9 IP|FAST TROUBLESHOOTING................................................................................................................... 49
4.10 IP|XCOMP ZRE TROUBLESHOOTING....................................................................................................... 62
4.11 IP|XCOMP SRE TROUBLESHOOTING ....................................................................................................... 66
4.12 IP|XCOMP REPORTING ............................................................................................................................ 74
4.13 IP|XAPP TROUBLESHOOTING ................................................................................................................... 76
4.14 SMARTPATH TROUBLESHOOTING ........................................................................................................... 83
4.15 REAL TIME GRAPH PROBLEM .................................................................................................................. 87
4.16 OTHER USEFUL TOOLS ............................................................................................................................ 89
4.17 IPENGINES CLUSTER ............................................................................................................................... 93
5 IN WHICH ORDER PROCESSES ARE EXECUTED?......................................................................... 95
5.1 IP|TRUE ONLY ...................................................................................................................................... 95
5.2 IP|TRUE AND IP|FAST.......................................................................................................................... 95
5.3 IP|FAST AND IP|XCOMP...................................................................................................................... 95
5.4 IP|FAST AND IP|XTCP.......................................................................................................................... 95
5.5 IP|FAST AND IP|XCOMP AND IP|XTCP............................................................................................... 95
6 ADVANCED TROUBLESHOOTING ...................................................................................................... 96
6.1 SHELL .................................................................................................................................................. 96
6.2 WHICH CERTIFICATE IS USED BY IP|ENGINE ? ......................................................................................... 96
7 UNSUPPORTED COMMANDS BY ORANGE BUSINESS SERVICES IN IPANEMA CLI ............ 97
7.1 NETFLOWCONFIG ................................................................................................................................... 97
7.2 SSLPASSPHRASE ..................................................................................................................................... 97
8 DEBUG COMMAND ................................................................................................................................. 98
8.1 CHECK IPBOSS CONFIGURATION ............................................................................................................. 99
8.2 “DEBUG FAST”...................................................................................................................................... 100
8.3 “DEBUG XCOMP” .................................................................................................................................. 100
8.4 OPTIONS OF DEBUG COMMAND ............................................................................................................ 101
9 DRAW_HTB COMMAND....................................................................................................................... 104
10 FAQ (EXTRACT FROM IPANEMA SUPPORT SITE).................................................................. 105

10.1 WHY I HAVE UNCORRELATED TRAFFIC ?.............................................................................................. 105

10.2 HIGH CPU USAGE IN AN IP|ENGINE........................................................................................................ 106
10.3 IMPACT OF MISSING SUBNET IN IP|BOSS CONFIGURATION ..................................................................... 106
10.4 MYSELF NOT FOUND IN CONFIG ............................................................................................................ 107
10.5 HOW IDENTIFY TRAFFIC ENCAPSULATED IN GRE TUNNEL?.................................................................. 107
10.6 FLOWS WRONGLY TAGGED AS OUT OF DOMAIN .................................................................................... 108
10.7 WHAT IS UNKNOWN TRAFFIC IN QUALITY EVOLUTION SLM REPORT? ................................................ 108
10.8 IP|ENGINE SYNCHRONIZATION PROBLEM .............................................................................................. 109

1 introduction
1.1 Purpose of this document
This document is designed to help support engineers to troubleshoot IPANEMA system. Two
major sections are detailed, one for IP|Boss and another for IP|engine probes.
1.2 Intented Audience
Support engineers.

2 Quality Of Service Troubleshooting

2.1 Diagnose the problem
START
Equipped Site?
YES
Performance
NO YES
problem?
3 IP BOSS
REALTIME
Supervision FLOWS
Are the flows Start DISCOVERY

YES NO
5 listed? on IP|engine
Display
Real time
flows in 2
ways
YES
6
QOS problem? YES
7
NO
NO WAN Problem?
YES
9
YES LAN Problem?

NO
11
YES Congestion?
10 8
13
NO
Criticial flow? YES
Check Load of
Check WAN
Client/Server?
14
Is the flow
NO 12
protected?
Check Load of Check QOS

Client/Server? Profile
15
NO
END

2.2 Site troubleshooting
START
Site's PM-
Detailed per
UC report
NO Congestion? YES
Site's SA- Site-PM-UC

Site summary
Throughput per
report direction
Are there Are criticals

YES unmeasured applcations NO
traffic? suffering?
Check Topology
subnet
NO
Site's PM-
Are there
Time
some
Evolution
complains from
WAN YES users?
Report
During non
Strong
Congestion,
congestion
QOS problem?
YES
YES
YES
Check Ethernet
config Application still
necessary?
Check WAN
Access config in YES
IP|Boss NO
Check WAN rate

of CE-Router
Upgrade the
line
NO
NO
NO
END

2.3 QOS Profile troubleshooting
START
PM-Time
Evolution
UC
YES TCP? NO
High Average
WAN loss?
Delay?
YES
NO
Critical? NO
LAN loss?
YES
YES
NO
YES
Critical?
Strong
Bursty?
congestion?
YES
NO
Increase
Increase Increase
bandwidth Modify QOS
bandwidth Upgrade Site bandwidth Check WAN
objective with Profile
objective objective
care
NO
Check Client/
Server Load
END

2.4 Supervision
START
IP|Boss IP|Boss
IP|Boss Map
Main IP|engine Trap/Mail
Supervision
Screen status
IP|engine
problem?
YES
IP|Boss
IP|engine
Status
Reachability
NO
problem
YES
NO
CE-router
Check Routing NO Time
reachable?
synchronisatio NO
n problem?
YES
YES
Are time
Check IP|engine is Is IP|engine Overload
NO servers NO
not blocking traffic reachable problem?
synchronized?
YES YES YES
Check IP NO
Is IP|engine Check IP|engine's Service non
connectivity
Reboot IP|engine NO run? check type and actual started
between I|engine
lights WAN rate problem?
and TIme servers
YES YES
Check LAN &

Is problem still WAN connector Check TIme
Since Last
here? servers YES
Update?
YES
search for the last
NO type of update
IP|reporter IP|Boss
NO
problem problem ?
Are LAN &
WAN
YES
Interfaces
YES YES
plugged?
check IP|reporter
NO
server
Call N+1 support
check Ethernet
Call N+1 support
connectivity
check infovista NO
login
END

2.5 Configuration
START
SA-Site
throughput
Egress/
Ingress
Abnormal
Start discovery
unmeasured
for these sites
traffic?
PM-Detailed
per user
class
Too much Improve the user

other? class description
PM-Detailed
per
Application
Improve the
Too much
application
TCP?
dictionnary
PM-Time
Evolution
Check the
High rate of
ethernet
loss?
configuration
Sites list
is list
Add missing site
correct?
PM-site
summary
Null Check IP address

bandwidth? site
END

3 IP|Boss troubleshooting
3.1 IP|Boss file system
IP|Boss server is multi domain, it means is can manage multiple IPANEMA domains at same
time. Each IPANEMA domain has it own configuration file stored in its own directory. IP|Boss
directory is organized as following:
├───gui
├───server
│ ├───bin IP Boss binaries
│ ├───conf IP Boss configuration files ipboss.conf
│ ├───domains IP Boss domains
│ │ ├───FT-BAM FT-BAM domain’s repository
│ │ │ ├───catalog
│ │ │ ├───conf
│ │ │ ├───config In this directory is stored __active__.ipmconf file.
│ │ │ ├───log
│ │ │ ├───security
│ │ │ ├───temp
│ │ │ │ └───ipanema-dump Here are stored scripts results
│ │ │ └───uninst
│ │ └───FT-BAM2 FT-BAM2 domain’s repository
│ │ ├───catalog
│ │ ├───conf
│ │ ├───config
│ │ ├───log
│ │ ├───security
│ │ ├───temp
│ │ └───uninst
│ ├───graph
│ ├───interface
│ ├───languages
│ │ ├───english
│ │ └───french
│ └───script
├───uninst
└───web_server
Most important files are:
__active__.ipmconf configuration file for each domain
ipboss.conf IP|Boss configuration where is stored tcp/ip port used for IP|Boss
3.2 TCP/IP communications to or from IP|Engines
In this section are listed all IP connections used inside IPANEMA system:

From To Default ports Description Configurable

(client) (server)
Management traffic
ip|boss-server ip|engines TCP/443 management traffic permanent
TCP/19996 upgrade status on-demand
TCP/19990 Realtime graphs opened from on-demand
to ipboss
TCP/19993
TCP/20 FTP transfert for upgrade on-demand
TCP/21
Inter ipengine traffic
ip|engine ip|engine TCP/19999 measurement traffic permanent
UDP/19999 optimization traffic if ip|fast ON
UDP/19997 Cluster traffic if cluster
UDP/123 Synchronization NTP permanent
UDP/19989 Xcomp dictionnary if ip|xcomp
synchronization ZRE ON
UDP/19988 Xcomp tunnel
UDP/19987 Signal from decompressor to
compressor
TCP option 26 Transparent compression if ip|xcomp
SRE ON
UDP/20000- Smartpath probing if smartpath
20001

4 IP|Engines troubleshooting
In all following subsections, troubleshooting requires to be remotely connected to IP|engine via
telnet, ssh, reverse telnet or console.
4.1 Interface mapping
4.1.1 Single instance software
It applies to single instance software earlier than 5.0.
APPLICATIONS
Egress HTB Ingress HTB
Bond X BRG0 Bond Y
Line shaper
Eth X Eth Y
LAN WAN
Eth interface are directly connected to hardware and a Line shaper is associated to WAN
interface, it permits to shape traffic at WAN access rate.
Bond interfaces hold HTB (“Hierachical Tree Based”) optimization, also seen as IBA in IP|fast
vocabulary.
Brg:
Brg is the bridge interface which interconnect LAN and WAN, Egress and Ingress interfaces.
4.1.2 Multi-instances software
It applies to multi-instances software earlier than 5.0.mi

APPLICATIONS
Egress HTB Ingress HTB
Brg0.0
Egr 0 Egr N Ingr 0 Ingr N
Brg0.N
BRG0
Lap 0 Lap N Wap 0 Wap N
Line shaper 0 Line shaper N

Eth X Eth Y
LAN WAN
Eth interface are directly connected to hardware.
Lap and Wap:

Wap are Line shapers associated to LAN and WAN interfaces, it permits to shape
traffic at WAN access rate of different instances.
Ingr
Ingr and Egr : those interfaces hold HTB (“Hierachical Tree Based”) optimization, also seen as
IBA in IP|fast vocabulary.
Brg:
Brg is the bridge interface which interconnect LAN and WAN, Egress and Ingress interfaces.
So for each instance (n), a 5-tuple of interfaces is created: LapN, EgrN, Brg0.N, IngrN, WapN.
4.2 Process chain inside ipengines

4.2.1 From LAN to WAN interfaces

Xapplication
PROXY
UC
classifier
-liste de
LAN Layer 7 Topology
servicesell XTCP FAST XCOMP
classifier
classification igibles WAN
(capacité
distante)
Tag Appli Tag Iba Tag UC
Session
context Session
Session context updating context
finding
restoring
Cache session
4.2.2 From WAN to LAN interfaces

Cache session
Session
Session context updating context
restoring
Tag Appli Tag Uc Tag Appli
LAN Layer 7 FAST XTCP UC Topo WAN

classif classif XCOMP
classification
Xapplication
© Copyright Equant PROXY 10th of may 2011

4.3 Physical Layer connectivity problem
4.3.1 Brcount
CONFIGURATION TROUBLESHOOTING
NO YES
brcount [ -d { lan | wan | all } ]

without args: count the number of MAC addresses per bridge port
with -d arg: display the known MAC addresses on the given port or all ports
An easy method to verify if IP|engine is correctly installed is to use brcount command. This
command helps to know if LAN and WAN interfaces aren’t exchanged, it means LAN plugged
on router and WAN plugged on customer LAN.
[ipe]$ brcount
Bridge has 5 (LAN) + 2 (WAN) = 7 MAC addresses
[ipe]$
Typically, there should more MAC address on LAN interface than on WAN interface. If not,
there could be a problem.
To request a detailled status, use the following command:
[ipe10]$ brcount -d lan

intf. MAC Address local ageing
LAN fe:fd:81:44:ea:da yes 0.00
LAN 00:11:92:62:fc:0a no 1.20
LAN 00:e0:81:44:ea:da yes 0.00
[ipe10]$ brcount -d wan
WAN 00:0b:be:16:ba:8a no 23.38
WAN 00:0b:be:16:ba:80 no 0.06
WAN 00:e0:81:44:ea:db yes 0.00
[ipe10]$ brcount -d all
LAN fe:fd:81:44:ea:da yes 0.00
LAN 00:11:92:62:fc:0a no 0.88
WAN 00:0b:be:16:ba:8a no 27.99
WAN 00:0b:be:16:ba:80 no 0.05
WAN 00:e0:81:44:ea:db yes 0.00
LAN 00:e0:81:44:ea:da yes 0.00
[ipe10]$
4.3.2 Ifconfig
NO YES
ifconfig [-a {interface} ]

without args: display all interfaces
with –a args: display only given interface
To check IP configuration and interfaces counters use the ifconfig command.

The following extract, shows an example of ip|engine configured with only a private ip address
(ip address that belong to customer network).
[ipe]$ ifconfig
brg0 Link encap:Ethernet HWaddr FE:FD:4B:0D:50:BF
inet addr:10.0.14.253 Bcast:10.0.14.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:422121 errors:0 dropped:0 overruns:0 frame:0
TX packets:404714 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:39047261 (37.2 MiB) TX bytes:49216429 (46.9 MiB)
eth0 Link encap:Ethernet HWaddr 00:E0:4B:0D:50:BF

Interrupt:9 Base address:0x3000
eth1 Link encap:Ethernet HWaddr 00:E0:4B:08:90:E9
Interrupt:10 Base address:0x5000
lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
[ipe]$
IP|engine’s IP address is configured on brg0 interface. Brg0 is a virtual bridge instantiated in
IP|engine’s Linux OS.
Check errors counters ( bold blue characters), if erros counters increase, the problem could
be Ethernet configuration mistake or hardware problem
Check collisions counters in both physical interface Eth0(LAN) and Eth1(WAN)
The next extract, shows an example of ip|engine configured with a private ip address with vlan
and an alias ip address.
[ipe10]$ ifconfig
bond0 Link encap:Ethernet HWaddr 00:E0:81:44:EA:DA
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
bond1 Link encap:Ethernet HWaddr 00:E0:81:44:EA:DB

UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
brg0 Link encap:Ethernet HWaddr FE:FD:81:44:EA:DA

brg0.100 Link encap:Ethernet HWaddr FE:FD:81:44:EA:DA



brg0.100: Link encap:Ethernet HWaddr FE:FD:81:44:EA:DA

eth0 Link encap:Ethernet HWaddr 00:E0:81:44:EA:DA

UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
eth1 Link encap:Ethernet HWaddr 00:E0:81:44:EA:DB

UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
[ipe10]$
Private IP address is configured on brg0.100 interface because we use vlan 100

Alias IP address is also automatically configured on vlan 100, so on interface brg0.100:1
To filter on a specific interface, use:
[ipe10]$ ifconfig brg0.100:1

brg0.100: Link encap:Ethernet HWaddr FE:FD:81:44:EA:DA
[ipe10]$
4.3.3 Ethconfig
YES YES
ethconfig
Manage ethernet controllers modes
Usage: ethconfig [-h] Print this help and exit
ethconfig -d Display current modes
ethconfig <if> <mode> Change mode for a particular interface
ethconfig all <mode> Change mode for all interfaces
With:
<if>=lan, wan or mgt
<mode>=10HD, 10FD, 100HD, 100FD, 1000FD or auto

Ethconfig is useful to force speed and duplex of LAN, WAN and MGT IP|engine’s interfaces.
This tools can also be used to display current settings.
[ipe]$ ethconfig –d
Copyright (c) Ipanema Technologies 2000-2005
eth[lan] : 100FD
eth[wan] : 100FD
eth[mgt] : AUTO
[ipe]$
Ethconfig –d displays current settings, it doesn’t display the real interface’s status specially
when interfaces are configured in AUTO-NEGOCIATION
If an interface needs a configuration changes, use the command:
[ipe10]$ ethconfig wan 100FD

eth[lan] : 100FD
eth[wan] : 100FD
eth[mgt] : AUTO
Configuration done...
Please reboot to apply these modifications...
[ipe10]
Each time, a speed or duplex change occurs on interface, you must reboot ip|engine.
4.3.4 Eth-diag
NO YES
eth-diag
Eth-diag command displays information about interfaces’s controller and interfaces’s status.
[ipe]$ eth-diag
---- Lan Interface ----
Basic registers of MII PHY #1: 3000 782d 02a8 0154 05e1 45e1 0001 0000.
The autonegotiated capability is 01e0.
The autonegotiated media type is 100baseTx-FD.
Basic mode control register 0x3000: Auto-negotiation enabled.
You have link beat, and everything is working OK.
This transceiver is capable of 100baseTx-FD 100baseTx 10baseT-FD 10baseT.
Able to perform Auto-negotiation, negotiation complete.
Your link partner advertised 45e1: Flow-control 100baseTx-FD 100baseTx 10baseT-
FD 10baseT, w/ 802.3X flow control.
---- Wan Interface ----
Basic registers of MII PHY #1: 3000 782d 02a8 0154 05e1 0021 0000 0000.
Basic mode control register 0x3000: Auto-negotiation enabled.
You have link beat, and everything is working OK.
This transceiver is capable of 100baseTx-FD 100baseTx 10baseT-FD 10baseT.
Able to perform Auto-negotiation, negotiation complete.
Your link partner is generating 10baseT link beat (no autonegotiation).
[ipe]$
4.3.5 Failsafe
On all new “AX” ipengines, it is possible to (un)configure failsafe directly through the CLI. With
old ipengine, you had to choose failsafe circuit or not during the installation on site. This old
method is over with new ipengines (5v2, 120ax, 140ax, 1000ax and 1800ax).
Additionaly, this modification is applied on the fly, it is not necessary to reboot.
To enable failsafe circuit

[ipe]$ customize +failsafe

Programming failsafe to bypass LAN & WAN when set
Configuration saved. Changes applied on the fly.
Current configuration:
. TrafficFilter : IP
. UseEADI : no
. UseSoftWdg : yes
. UseHardWdg : yes
. ExcludeMode : standard
. LanDownToWan : no
. AcceptICMPRedirect : yes
. IpxTunnelMode : in
. NoPMTUDiscovery : no
. Bypass disabled : no
. SkipGRE : no
[ipe]$
To disable failsafe circuit:
[ipe]$ customize -failsafe

Programming failsafe to disconnect LAN & WAN when set
Configuration saved. Changes applied on the fly.
. UseEADI : no
. UseSoftWdg : yes
. UseHardWdg : yes
. LanDownToWan : no
. Bypass disabled : yes
. SkipGRE : no
[ipe]$
On older ipengines (ipe5, 120, 120V2, 140, 1000, 1200 and 1800), you have to choose the
correct couple of LAN/WAN interfaces if you want to enable or disable failsafe. Because it is a
physical configuration you can’t change the failsafe mode remotely.
4.3.6 LanDownToWan
Use customize command. It works in interactive mode only:
[ipe10]$ customize
. UseEADI : no
. UseSoftWdg : yes
. UseHardWdg : yes
. LanDownToWan : no
. Failsafe Disabled : no
. SkipGRE : no
-------------------
f set filter mode
e set eadi usage
d set softDog usage
w set watchdog usage
m set exclude Mode
l copy Lan status to wan

r accept ICMP redirect

t ip|xcomp tunnel mode
p path MTU discovery
b enable/disable bypass (Failsafe)
g skip IP+GRE headers
-------------------
v show current Values
s Save new values
x save and eXit
q Quit without saving
? print help
-------------------
Custom ('?' for help):l
(copy Lan status to wan)
Current value for "Copy Lan status to Wan" : no
Select "Copy Lan status to Wan" (y/n) [n]? y
Modified configuration:
. UseEADI : no
. UseSoftWdg : yes
. UseHardWdg : yes
. LanDownToWan : yes [*]
. Failsafe Disabled : no
. SkipGRE : no
[*] indicates a modified value (not saved).
Custom ('?' for help): x
(save and eXit)
Configuration saved. Please reboot to apply these modifications...
[ipe10]$
Reboot is necessary to apply modifications.
4.4 Network Layer connectivity problem
4.4.1 Ipconfig
YES YES
ipconfig
Manage network interfaces
Usage: ipconfig Print this help and exit
ipconfig -d Display current configuration
ipconfig [lan|mgt] [none | -a <IPaddr> [-m <IPmask>]] [-vlan [<Id>|none]] [-
mtu <MTU>]
Set local settings for this ip|engine
ipconfig alias [none | -a <IPaddr> [-m <IPmask>]]
Configure a 2nd address on the same interface
as the main address
ipconfig -g <Gateway> Set IP address of the gateway
ipconfig -h <Hostname> Local name assigned to this engine
ipconfig [no] serial Set the ip|engine in parallel/serial mode
ipconfig [no] dual Set the ip|engine in single/dual parallel mode
ipconfig reset Reset configuration parameters (no IP address
nor mask, serial mode)
Options:
-a <IPaddr> IP address

-m <IPmask> Subnet mask (optional)

-vlan <Id> Local 802.1Q vlan Id
-mtu <MTU> Interface MTU [576..1500]
To check IP config, use ipconfig command.
Example with private ip address only:
[ipe]$ ipconfig –d
Current Configuration:
IP addr : 10.0.14.253
IP mask : 255.255.255.0
Gateway : 10.0.14.254
Hostname : ipe
Serial mode : yes
[ipe]$
Example with private and alias ip address :
[ipe7]$ ipconfig -d
[LAN] IPaddr : 10.10.70.240
IPmask : 255.255.255.0
intfMTU : 1500
[alias] IPaddr : 10.0.18.7
IPmask : 255.255.255.255
Gateway : 10.10.70.254
Hostname : ipe7
Serial mode : yes
[ipe7]$
Example with private and alias ip address on a vlan 802.1Q:
[ipe7]$ ipconfig -d
[LAN] IPaddr : 10.10.70.240
IPmask : 255.255.255.0
Vlan id : 100
intfMTU : 1500
[alias] IPaddr : 10.0.18.7
IPmask : 255.255.255.255
Gateway : 10.10.70.254
Hostname : ipe7
Serial mode : yes
[ipe7]$
4.4.2 Ping
NO YES
ping
usage: ping [-LRdfnqrv] [-c count] [-i wait] [-l preload]
[-p pattern] [-s packetsize] [-t ttl] [-I interface address] host
Useful to test IP connectivity.
4.4.3 Traceroute
NO YES

traceroute
Version 1.4a5
Usage: traceroute [-dFInrvx] [-g gateway] [-i iface] [-f first_ttl] [-m max_ttl]
[ -p port] [-q nqueries] [-s src_addr] [-t tos] [-w waittime]
host [packetlen]
Useful to test IP path connectivity.
4.4.4 Arp
NO YES
[ipe]$ arp
Address HWtype HWaddress Flags Mask Iface
10.0.14.254 ether 00:04:C0:5D:57:E0 C brg0
[ipe]$
Useful to verify IP connectivity on customer’s LAN.
4.4.5 Route
YES YES
[ipe]$ route –e
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
10.0.14.0 * 255.255.255.0 U 40 0 0 brg0
default 10.0.14.254 0.0.0.0 UG 40 0 0 brg0
[ipe]$
or
[ipe]$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.0.14.0 * 255.255.255.0 U 0 0 0 brg0
default 10.0.14.254 0.0.0.0 UG 1 0 0 brg0
[ipe]$
Columns Description Without With –e option
option
Destination Subnet or host YES YES
Gateway Gateway’s IP address or * if undefined YES YES

GenMask Netmask of recipient YES YES
Flags U route is up YES YES
H recipient is a host
G route is default
R
D dynamic route learnt via ICMP redirect
M
! reject packets

Metric Distance to reach recipient YES NO

Ref Unused YES NO
Use Unused YES NO
Iface Outgoing interface YES YES
MSS Default TCP Maximum Segment Size for this NO YES
route
Windows Default TCP window size for this route NO YES
Irtt Initial RTT NO YES
When ip|engine is installed with IP alias mode, it is necessary to configure at least a static route:
route add -net 193.105.90.0 netmask 255.255.255.0 gw Adr_Gateway_Mgt
To delete an old route :
route del -net 193.105.90.0 netmask 255.255.255.0 gw Adr_Gateway_Mgt
4.4.6 Ip
Since release 5.1.4 and 6.0 the IP command is available.
It permits to display several information about network interface and routing.
To display ip routes:
[ipe]$ shell
bash-2.01$ ip route list
172.10.30.241 via 10.10.30.254 dev brg0 src 172.10.30.241
10.10.30.0/24 dev brg0 proto kernel scope link src 10.10.30.241
default via 10.10.30.254 dev brg0 metric 1
bash-2.01$
To dump the routing cache:
bash-2.01$ ip route show cache
To find a route matching an ip address or a subnet part:
bash-2.01$ ip route show to match 195.6.0.177/32

or
bash-2.01$ ip route show to match 192.168/16
To display interfaces configuration
bash-2.01$ ip link show

1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether 00:e0:81:46:61:ac brd ff:ff:ff:ff:ff:ff
link/ether 00:e0:81:46:61:ad brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,PROMISC,UP> mtu 1500 qdisc pfifo qlen 1000
link/ether 00:0c:bd:01:29:06 brd ff:ff:ff:ff:ff:ff


link/ether 00:e0:81:46:61:67 brd ff:ff:ff:ff:ff:ff
7: brg0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue
link/ether fe:fd:bd:01:29:06 brd ff:ff:ff:ff:ff:ff
8: wap0: <NOARP,UP> mtu 1500 qdisc htb qlen 1000
link/ether
9: ing0: <NOARP,UP> mtu 1500 qdisc htb qlen 1000
link/ether
10: lap0: <NOARP,UP> mtu 1500 qdisc pfifo qlen 1000
link/ether
11: egr0: <NOARP,UP> mtu 1500 qdisc htb qlen 1000
link/ether
bash-2.01$
To display interface configuration with detailed statistics
bash-2.01$ ip -s link show

1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
RX: bytes packets errors dropped overrun mcast
253826249 4764943 0 0 0 0
TX: bytes packets errors dropped carrier collsns
253826249 4764943 0 0 0 0
link/ether 00:e0:81:46:61:ac brd ff:ff:ff:ff:ff:ff
0 0 0 0 0 0
0 0 0 0 0 0
link/ether 00:e0:81:46:61:ad brd ff:ff:ff:ff:ff:ff
0 0 0 0 0 0
0 0 0 0 0 0
21728506 187779 0 0 0 26825
64732869 836706 0 0 0 0
135627028 1693032 0 0 0 576757
155436953 1152173 0 0 0 0
link/ether 00:e0:81:46:61:67 brd ff:ff:ff:ff:ff:ff
0 0 0 0 0 0
0 0 0 0 0 0
7: brg0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue
link/ether fe:fd:bd:01:29:06 brd ff:ff:ff:ff:ff:ff
123501686 1880826 0 0 0 0
127705743 964407 0 0 0 0
8: wap0: <NOARP,UP> mtu 1500 qdisc htb qlen 1000
link/ether
0 0 0 0 0 0
20977270 187777 0 0 0 0
9: ing0: <NOARP,UP> mtu 1500 qdisc htb qlen 1000
link/ether

0 0 0 0 0 0
20977270 187777 0 0 0 0
10: lap0: <NOARP,UP> mtu 1500 qdisc pfifo qlen 1000
link/ether
0 0 0 0 0 0
61384715 836684 0 0 0 0
11: egr0: <NOARP,UP> mtu 1500 qdisc htb qlen 1000
link/ether
0 0 0 0 0 0
61384715 836684 0 0 0 0
bash-2.01$
4.5 Connectivity with ipboss
4.5.1 Debug show config
Check if ipengine has correctly received its configuration, check DomaineName, the ip address
of ipboss
debug show config
[ CONFIGURATION ]
ConfigName =
__active__@a399e2f3228e768bc3a66b22bfbb49fdaf8c3009@9e90509f8e8d3e192d2e29171c074dc
Domain e5d10e36e
name DomainName = BAM
Bridge = brg0
Cap100Mbs[yes] CapGigabit[no] CapFullD[yes]
CapSerial[yes] CapDual[no] CapEadi[no] Ipboss connected to
CapGps[yes] CapExt[yes] CapXcomp[yes] CapXtcp[yes] ip|engine
SerialMode[yes] DualMode[no] EadiMode[no]
Boss IP addr = 10.0.16.19 ExcludeMode = sentry<=>any
Locl IP addr = 10.10.50.241 Inb MAC addr = FE:FD:0B:07:67:11
LAN MAC addr = 00:90:0B:07:67:11 WAN MAC addr = 00:90:0B:07:67:12
DrvNbTickets = 16384 MinToRead = 50 MaxAcqFrames = 256
MaxDefragCtxt= 2048 MaxFragments = 8096
TkgMaxFlows = 64000 ArchMaxFlows = 12000 RTMMaxFlows = 8
RTMPeriod = 10 ArchPeriod = 60 MaxCrPerFlow = 6
TickDownDelay= 50 TickDownPer. = 2 CnxRetryPer. = 4
LossThreshold= 5 TTP_Port = 19999 TTP_MaxLength= 1293
SynchroThresh= 10000 us (+10%:11000 us) SpuriousDelay= 5000
NbThreshDlay = 8
ThreshDlay = 10000 20000 50000 100000 200000 500000 1000000
ITP port2000000
: should be 123
Options = 0x000a SerialMode Softdog
NbTickets = 64000, 63891 free (28725 + 35166 in TKG)
and not 19995
TicketCrcNb = 65536
RxTtpBuffers = 1000 (1000 free) TxTtpBuffers = 1000 (999 free)
UflowAging = 10 ConCnxTcpAging = 30 SentryTTPAging = 600 ConCnxTcpDeadAging
= 300
Port ITP: 123 ITP Mode active, alpha 0.000020, beta 0.000500
ArpCpeInitPeriod = 10 ArpCpePeriod = 60 DeadCpeAgingDelay = 1200
ForceCrc24 thresholds low 10000, high 15000
Engine capacity advertising = yes (on UDP port 19996)
MaxValidSRT = 20000000
RT flows ports : 19990-19993;
Expected DR (0x1E) = CorrRecord SiteRecord TopHostAppli ExtSiteRecord
Known DR (0x1E) = CorrRecord SiteRecord TopHostAppli ExtSiteRecord
Expected CR (0x7F) = LANVol LANQual WANVol WANQual SmartPlan XComp TCP
Known CR (0x7F) = LANVol LANQual WANVol WANQual SmartPlan XComp TCP
[ CRC INFOS ]

Using: IP.PaquetID: no TCP/UDP.Ports: no TCP.Seq/Ack: yes TCP.Window:

yes
4.5.2 Debug show global Services enabled (ip|true, ip|fast,
ip|xcomp and ip|xtcp)
debug show global
[ GLOBAL VARIABLES ]
NbSpurious = 4965 NbAmbiguous = 142272 NbLost = 59763
Here probe is part of a cluster
ArchLostLost = 0 RTMLostLost = 12384
ip|true? yes ip|fast? yes ip|xcomp? yes ip|disc? no ip|xtcp? no
Sessions = 209 (max 96000) Flow = 18 in 13 variants (max 12000)
Cluster: 2 contexts (max 4096)
TopCtxt Hashing in 4096 buckets: 11 items in 11 buckets, min/avg/maxProbe
1 1 1 is synchronized
TkgFlow Hashing in 8192 buckets: 209 items in 174 buckets, min/avg/max server
on 1 1 2
ClsUFlow Hashing in 8192 buckets: 187 items in 167 buckets, min/avg/max 1 1 2
ArchFlow Hashing in 2048 buckets: 18 items in 18 buckets, min/avg/max 1 1 1
Sync'ed ? yes(30) SyncSrc Server 10.10.50.240
NbSatellites = 0
PositionHold ? no (Oncore GPS receiver) Display alarms (out of
SynchroThresh= 10000 us (+10%:11000 us) SyncKernelOffset= 6 correlation ticket,
SyncSrcOffset= 7 SyncFreq= -12 SyncSrcDelay= 204 interface down,
UseFlowId? yes StopOnIntfDown? yes LanDownToWan? yes overloaded…etc)
Current Alarms = (none)
Previous Alarms = (none)
CPU Load = 8% (Bounds 0..10000)
CPU load
Hardware tag and model

4.5.3 Debug show version or version
debug show version
[ VERSION ]
Hardware: Tag = 10AR, Name = 0120-512-CF64-3-X-G1, Rev = XX
Software: Version = 5.0.mi, Date = Oct 8 2008, Time = 16:39:57
Packages: Ipe = 5.0.mi.6, Kernel = 5.0.mi.5, Tools = 5.0.mi.5
Software version and its
packages
4.6 Synchronisation problem
Inside a same domain, IP|engines must be synchronized. Synchronization is done via a

modified NTP protocol, ITP (Ipanema Time Protocol).
First check:
Time server is reachable from IP|engine

Time server is synchronized.
Then check, directly on IP|engine:
4.6.1 Itpq
YES YES

ITPQ command is useful to check Time Synchronization. It displays available ITP servers and
ITP server elected as the time source.
[ipe]$ itpq –p
remote refid st t when poll reach delay offset jitter
==============================================================================
*LOCAL(0) LOCAL(0) 3 l 17 16 377 0.000 0.000 0.000
10.0.13.253 10.0.14.253 5 u 3 16 377 7.591 0.107 0.177
[ipe]$
4.6.2 Itpconfig
Itpconfig command is deprecated.
4.6.3 Date
YES YES
To check that date is correct, use date command.
[ipe]$ date
Tue Mar 28 07:26:41 UTC 2006
[ipe]$
4.6.4 Debug dump global
NO YES
To debug network time synchronization, check the red bold characters.
[ipe]$ debug dump global

ip|true? yes ip|fast? yes ip|xcomp? no ip|disc? No
TkgFlow Hashing in 4096 buckets: 6 items in 6 buckets, min/avg/max 1 1 1
ClsUFlow Hashing in 4096 buckets: (empty)
ArchFlow Hashing in 2048 buckets: (empty)
Sync'ed ? yes(30) SyncSrc server 10.0.13.253
NbSatellites = 0
PositionHold ? yes (no GPS receiver)
SynchroThresh= 10000 us (+10%:11000 us) SyncKernelOffset= 1416
SyncSrcOffset= 1412 SyncFreq= 45 SyncSrcDelay= 7704
UseFlowId? yes StopOnIntfDown? yes LanDownToWan? No
CPU Load= 1.22 (Bounds: 0.00 .. 10000.00)
[ipe]$

4.6.5 Debug show itp
debug show itp
[ ITP SERVERS (ITP Port 19995) ]

10.10.50.240 SYNC engine=ipe51
10.238.38.226 TIME
4.7 Application layer connectivity problem
This section is dedicated to ipboss connectivity problem, I mean when ipboss can connect to
ip|engine but when when problem is not due to a network outage.
4.7.1 Netstat
NO YES
Use netstat to verify that udp and tcp ports are opened.
[ipe]$ netstat
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 10.0.14.253:https 10.238.39.75:4102 ESTABLISHED 1
tcp 0 126 10.0.14.253:telnet 10.238.39.75:1050 ESTABLISHED
udp 0 0 localhost:32769 *:*
udp 0 0 10.0.14.253:19995 *:* 2
udp 0 0 localhost:19998 *:* 3
udp 0 0 10.0.14.253:19999 *:* 4
udp 0 0 localhost:ntp *:* 5
Active UNIX domain sockets (w/o servers)

Proto RefCnt Flags Type State I-Node
Path
unix 3 [ ] STREAM CONNECTED 24295358 /dev/log
unix 3 [ ] STREAM CONNECTED 24295357
[ipe]$
1 IP|Boss is connected to IP|engine with HTTPS connection
IPBOSS BAM is translated with ip address = 195.6.0.116
IPBOSS NETWORK BOOST is translated with ip address on scope 2-3= 193.105.90.9
IPBOSS NETWORK BOOST is translated with ip address on scope 0-1= 57.7.44.65
2 ITP network synchronization
3 Compression process is listening
4 Optimization process connected with remote ip|engine 10.0.14.253
5 NTP process is listening

4.7.2 Ps –a
NO YES
With ps –a command, we can display all process started on ip|engine.
Before using it, enter command shell to enter shell command
[ipe]$ shell
bash-2.01$ ps -a
PID Uid VmSize Stat Command
1 0 492 S init [2]
2 0 SW [keventd]
3 0 RWN [ksoftirqd_CPU0]
4 0 SW [kswapd]
5 0 SW [bdflush]
6 0 TW [kupdated]
284 0 460 S /usr/local/bin/ipm_survey -
f/etc/ipm_survey.co
286 0 536 S ipm_upgrade
289 0 7660 S /usr/local/bin/ip_true
299 0 584 S /sbin/syslogd -m 0
306 0 536 S /sbin/klogd
309 0 780 S /usr/sbin/xinetd
313 0 720 S -interp
314 0 1312 S ipm_security
315 0 380 S ipm_script
317 0 1060 S /usr/local/bin/ip_fast
321 0 596 S /usr/local/bin/ip_xcomp
23650 0 2072 S /usr/local/apache/bin/httpd -DSSL
9230 0 1572 S /usr/local/bin/itpd
9362 0 592 S in.telnetd: 10.238.39.75
9363 0 724 S -interp
9373 1000 1192 S /bin/bash
9374 1000 552 R ps -a
bash-2.01$exit
[ipe]$
Verify that the following process are running:
Ipm_survey : is the software watchdog process

Ip_true : is the IP|true process
Ip_fast : is the IP|Fast process
IP_xcomp : is the IP|Xcomp process
Ipm_security :
Ipm_upgrade : is the process responsible for upgrading IP|agent
Ipm_script : is the process which executes IP|Boss scripts

Httpd : is the web server process, on which IP|Boss connect.

Xinetd : is the process which manage telnet and ssh sessions.
Itpd : is the ITP process for time synchronization
4.7.3 Cat /tmp/survey.*
NO YES
This command is accessible after entering shell command.
It displays content of files:
Survey.httpd
Survey.ip_fast
Survey.ip_true
Survey.ip_xcomp
Survey.ipm_script
Survey.ipm_security
Survey.ipm_upgrade
Survey. Tpd
[ipe7]$ shell
bash-2.01$ cat /tmp/survey.*
Launch #1 of httpd at Mon Mar 13 10:02:35 2006
Launch #2 of httpd at Thu Mar 23 10:09:10 2006
Launch #1 of ip_fast at Mon Mar 13 10:02:24 2006
Launch #1 of ip_true at Mon Mar 13 10:02:10 2006
Launch #1 of ip_xcomp at Mon Mar 13 10:02:30 2006
Launch #1 of ipm_script at Mon Mar 13 10:02:19 2006
Launch #1 of ipm_security at Mon Mar 13 10:02:14 2006
Launch #1 of ipm_upgrade at Mon Mar 13 10:02:10 2006
Launch #1 of itpd at Mon Mar 13 10:02:40 2006
Launch #2 of itpd at Tue Mar 28 07:21:59 2006
bash-2.01$
It is useful to verify when a process was started or restarted. Check also with uptime
command.
4.7.4 Netconfig
YES YES

netconfig -h
Manage network services
Usage: netconfig [-h] Print this help and exit
netconfig -d Display current configuration
netconfig [+/-]<service>... Enable (+) or disable (-) one or more
services
netconfig restart Stop all running network services and
restart
enabled services
netconfig reset Enable ssh, disable telnet and ftp
With:
<service>=ssh, telnet, ftp
Netconfig command permits to check if telnet or ssh are enabled. If you want to enable one use
netconfig:
Netconfig +telnet : enables telnet netconfig –telnet : disables telnet
Netconfig +ssh : enables ssh netconfig –ssh : disables ssh
[ipe]$ netconfig –d
service telnet enabled
service ssh enabled
[ipe]$
Since release 4.3.0, ftp service is available on all ip|engine models. FTP could be used to
upgrade ip|agents.
Netconfig +ftp : enables ftp netconfig –ftp : disables ftp
[ipe]$ netconfig –d
current configuration:
service ssh : enabled
service telnet : enabled
service ftp : disable
[ipe]$
Note: after being reconfigured, it is necessary to restart all network services with the command:
[ipe]$ netconfig restart

New configuration:
Service ssh : enabled
Service telnet : enabled
Service ftp : enabled
Restarting network services...
Your connection (telnet or ssh or ftp) is reset, but ip|engine is not rebooting.
4.7.5 Debug dump global
NO YES
To debug status of IP|True, Fast, Xcomp and Xtcp processes, check the red bold characters.
[ipe]$ debug dump global

ip|true? yes ip|fast? yes ip|xcomp? no ip|disc? no ip|xtcp? no
ClsUFlow Hashing in 4096 buckets: (empty)
ArchFlow Hashing in 2048 buckets: (empty)
Sync'ed ? yes(30) SyncSrc server 10.0.13.253
NbSatellites = 0
PositionHold ? yes (no GPS receiver)
SynchroThresh= 10000 us (+10%:11000 us) SyncKernelOffset= 1416
SyncSrcOffset= 1412 SyncFreq= 45 SyncSrcDelay= 7704
UseFlowId? yes StopOnIntfDown? yes LanDownToWan? No
CPU Load= 1.22 (Bounds: 0.00 .. 10000.00)
[ipe]$
4.7.6 Certificates
Certificates could be cause of problem between Ip|engines and IP|Boss. To check that every
IP|engines use the same certificate, use in System Provisioning profile, Tools button then
Security Status tab.
Select all IP|engines and click status. Wait for result to be displayed and check certificates are
the same.
4.7.7 Uptime
NO YES
You can check last time ip|engine has rebooted.
With uptime command, it is possible to check load of an ip|engine. If load is greater than 2,
then ssh connection are refused.
Load is in red bold characters.
[ipe]$ uptime
08:57:45 up 14 days, 22:55, load average: 0.00, 0.00, 0.00
[ipe]$
4.7.8 Snmpconfig
This feature is available since release 4.3.0.
YES YES
Snmpconfig is used to enable MIB2 agent on ip|engine and to configure a community string.
To configure a snmp read-only community string, proceed as following:

[ipe]$ snmpconfig –c publicstring

SNMP Configuration:
status = enabled
state = stopped
community = publicstring
[ipe]$
In default configuration, MIB2 agent is stopped. After any modification, it is necessary to restart
MIB2 agent.
[ipe]$ snmpconfig start

SNMP Configuration:
status = enabled
state = running
[ipe]$
To display current configuration:
[ipe]$ snmpconfig –d
SNMP Configuration:
status = enabled
state = running
[ipe]$
4.8 IP|True troubleshooting
4.8.1 Ipconfig -d
YES YES
First, it is necessary to check if IP configuration is correct. So verify Gateway IP address.
The command ipconfig –d displays all IP configuration.
[ipe]$ ipconfig –d
Current Configuration:
IP addr : 10.0.14.253
IP mask : 255.255.255.0
Gateway : 10.0.14.254
Hostname : ipe
Serial mode : yes
[ipe]$
4.8.2 Arp
NO YES
With arp command, it is possible to check that gateway MAC address is correct.
[ipe]$ arp
Address HWtype HWaddress Flags Mask Iface
10.0.14.254 ether 00:04:C0:5D:57:E0 C brg0
[ipe]$

4.8.3 Debug dump tkg
NO YES
Displays information about correlation tickets:
Number of processed frames

Upstream (from local to remote) and Downstream (from remote to local) tickets
Ip|engine limits after which it becomes overloaded, here 24Mbps for a ipe120v1
Average throughput for last 0 to 30 seconds (dynamic) on WAN interface
Average throughput for last 30 seconds on WAN interface
Maximum reached since last reboot on WAN interface
debug show tkg
[ TICKETGEN ]
Processed Frames: 95105970 NoMoreTkUp: 0
Upstream: 54825004 Downstream : 38836485
Ignored : 468157 OutOfDomain: 35863707
Frames/Ioctl LAN: Min=1, Max=255, Avg=1, Total=39528158
Frames/Ioctl WAN: Min=1, Max=256, Avg=1, Total=55573904
TopCtxt Hashing in 4096 buckets: 11 items in 11 buckets, min/avg/max 1 1 1
Throughput: (Bits and Packets per sec, averaged and checked every 30 sec)
Limits : Ingress 24.000 Mb/s Egress 24.000 Mb/s Packets 0
Last23s: Ingress 821.184 Kb/s Egress 410.360 Kb/s Packets 2641
LastAvg: Ingress 1.048 Mb/s Egress 385.896 Kb/s Packets 2470
MaxUsed: Ingress 7.994 Mb/s Egress 615.192 Kb/s Packets 4131
4.8.4 Debug dump engine
NO YES
With debug dump engine command, we can display counters of received tickets per
destinations.
[ipe]$ debug dump engine

[ 8 IP_ENGINES ]
Local engine:
BRICOMAN-ESPAGNE (10.0.14.253): LOCL FAST XCOMP DCOMP
Other engines:
Out of domain (240.0.0.0): TOOD FOOD VIRT
TkDown= 0 Spur= 0 Ambig= 0 Lost= 0
CAP-GEMINI2 (10.0.15.249): FAST XCOMP DCOMP
CAP-GEMINI (10.0.15.250): FAST XCOMP DCOMP
FRANCE-LEZENNES (10.0.11.253): FAST XCOMP DCOMP
FRANCE-NEWDATA (10.0.12.253): FAST XCOMP DCOMP


OBI-FRANCE (10.0.13.253): FAST XCOMP DCOMP
*FastEgressInDomain* (255.255.255.255): FEID VIRT
[ipe]$
TkDown: received Tickets
Spur or spurious: unwaitted tickets
Ambig or ambiguous: many packets have the same CRC signature.
Lost: tickets not received.
Spurious and ambiguous tickets could be due to encryption or tests with the same packet.
4.8.5 Debug dump cor
NO YES
[ipe]$ debug dump cor

[ CORRELATOR ]
FRANCE-LEZENNES => INTERNET No Cnx, CurrTkDown = 0
NbCorr = 0
FRANCE-LEZENNES => OBI-FRANCE Connected, CurrTkDown = 14647
NbCorr = 2548
[ipe]$
Displays current correlators with downstream tickets and total upstream tickets.
Synthax used is :
<Local Ipengine> => <remote Ipengine> <connexion status>, CurrTkDown = <number of

tickets sent to remote Ipengine>, NbCorr = <number of tickets correlated (received and
corresponding to config) from all other remote ipengine>
4.8.6 Debug dump flow
Displays every flows measured by ip|true engine and provides topology information such as:
upstream (go toward IPVPN), downstream (come from IPVPN), multicast, redirected, transit.
NO YES
Table 4.8-1
Number of measured flows

And maximum supported by ip|engine
debug show flow
[ 220 FLOWS, limit is 96000 ]

Current flows, > Ingress, < Egress, oldest last, age in msec, life in sec:
> UDP, 10.10.50.250:20068 - 10.10.110.61:10012, appli 92(G729): Upstream_Cluster,
50 pkts CORREL(10), age 0, life 1


< UDP, 10.10.110.191:10056 - 10.10.50.250:20050, appli 92(G729):
Downstream_Cluster, 653 pkts, age 0, life 16
< UDP, 10.10.110.49:10036 - 10.10.50.250:20062, appli 92(G729):
< UDP, 10.10.110.43:10000 - 10.10.50.250:20056, appli 92(G729):
< TCP, 10.10.110.51:34439 - 10.10.50.251:80, appli 11(HTTP): Downstream_Cluster,
52 pkts, age 10, life 2
< TCP, 10.10.110.60:6788 - 10.10.50.251:80, appli 11(HTTP): Downstream_Cluster, 44
pkts, age 10, life 2
Layer 4 Dest IP + Layer 7 direction

dest port
protocol protocol
Source IP + Number of
source port packets
> UDP, 10.10.50.121:10024 - 10.10.20.204:20008, appli 96(VOICE): Upstream, 1490

pkts CORREL(8), age 3030, uc 27, life 33
Age 3030 : means the last packet was seen 3030msec ago (so 3.03sec)
Life 33: means the connection has started 33 sec ago
Life – age = duration of connection
CORREL : connection is correlated, since age is > 10 000 then CORRELATION stops
< UDP, 10.10.20.204:20022 - 10.10.50.160:10010, appli 93(G711a): Downstream, 625

pkts, age 0, uc 27, life 12
Age 0 : means the last packet was seen 0 msec ago (so connection is alive)
Life 12 : means the connection has started 12 sec ago
Life – age = duration of connection: connection is still alive
CORREL not displayed because it is an Egress flow, only Ingress flow are correlated.
If you need more details, use “-d” with previous command:
debug show flow -d |more

< UDP, 10.10.110.27:10094 - 10.10.50.250:20054, appli 92(G729):
EngineUp/Down: ipe11/ipe52, priv 103(rtp/rtcp), client, iba 10.10.110.240,
flowId 433010, tkCount 1
TopoSrc/Dst: NET110/NET50 UserSrc/Dst: *Other*/*Other*



Another example with filtering
[ipe]$ shell
ipe:~# debug show flow -d | grep -A 3 FTP
> TCP, 10.10.50.201:51990 - 10.10.0.2:5248, appli 10(FTP): Upstream_Cluster, 44400
pkts CORREL(9), age 0, life 394
EngineUp/Down: ipe52:1/ipe0a:1, priv 38(ftp), server, tcpFlags SYN+ACK, iba
10.10.0.241, flowId 43227, Xcomp, XTCP, tkCount 40
TopoSrc/Dst: NETP10.50/NET0 UserSrc/Dst: 10.10.50.0/*Other*
< TCP, 10.10.0.2:5248 - 10.10.50.201:51990, appli 10(FTP): Downstream_Cluster,

20185 pkts, age 0, life 394
EngineUp/Down: ipe0a:1/ipe52:1, priv 38(ftp), client, tcpFlags SYN+ACK, iba
10.10.0.241, flowId 43227, XTCP, tkCount 1
TopoSrc/Dst: NET0/NETP10.50 UserSrc/Dst: *Other*/10.10.50.0
ipe:~#
Can be compressed Can be accelerated

Classification tags
classification tag description status in SA-site-throughput report Measured Optimized

Upstream Ingress measured traffic. If correlated To physical ip|engine (if CORREL) else YES YES
(CORREL tag) quality+volume measured No correlation (if no CORREL tag)
otherwise is only volume.
UpstreamToVirtual Ingress traffic to virtual ip|engine. Only To virtual ip|engine or to out of domain. YES YES
reported in volume + TCP metrics when
available
Upstream_Cluster Ingress traffic to a cluster. If correlated To physical ip|engine (if CORREL) else YES YES
(CORREL tag) quality+volume measured No correlation (if no CORREL tag)
otherwise is only volume.
Downstream Egress traffic from physical ip|engine. From physical ip|engine. YES YES
DownstreamFromVirtual Egress traffic from virtual ip|engine. Only From virtual ip[engine YES YES
measured in volume.
From Out Of Domain
Downstream_Cluster Egress traffic which source subnet is behind From physical ip|engine YES YES
an ip|engine cluster.
IngressTransit(OwnNoSubnet) Ingress traffic (from the LAN to the WAN) for Transit NO YES
which netiher the source or destination
subnets are managed by local ip|engine.
EgressTransit(OwnNoSubnet) Egress traffic (from the WAN to the LAN) for Transit NO YES
which netiher the source or destination
subnets are managed by local ip|engine.
Excluded(OwnNoSubnet) Traffic for which netiher the source or Transit NO YES
destination subnets are managed by local
ip|engine.
Excluded(OwnBothSubn) Traffic where both source and destination Other NO NO
subnets are managed by local ip|engine

classification tag description status in SA-site-throughput report Measured Optimized

Excluded(IngrToLocal) Traffic coming the LAN port to WAN but with a Other NO NO
source IP address belonging to another
ip|engine and a destination IP address owned
by itself.
Excluded(EgrFromLocal) Traffic coming from the WAN port to the LAN Other NO NO
but with a source IP address owned by itself
and a destination IP address owned by
another ip|engine.
Excluded(Multicast) Multicast traffic Other NO NO
Excluded(Redirected) Packets seen LAN to WAN and then WAN to Locally rerouted NO NO
LAN
IngressTransit(IpeExcluded) Traffic to an ip|engine of the Domain or to Transit NO NO
ip|boss where neither the source nor the dest
IP addresses are managed by the ip|engine
(in optimisation only)
EgressTransit(IpeExcluded) Traffic from an ip|engine or ip|boss where Transit NO NO
neither the source nor the the dest IP
addresses are managed by the ip|engine (in
optimisation only).
Excluded(IpeExcluded) Traffic from/to an ip|engine of the Domain or Other NO NO
from/to ip|boss (in measurement only)
WithManager Traffic between ip|engines and ip|boss when Other NO NO
the ip|engine was customized with the
ExcludeMode (command customize). It
excludes NAT traffic.

4.8.7 Debug show rtm
Displays a list of all realtime flows measured by a user on ipboss GUI. Each ip|engine is limited
to 4 simultaneous flows.
Used slot unused slot
debug show rtm
[ REAL TIME MONITORING ]

RT ports:
19990 [1] 19991 [1] 19992 [0] 19993 [0]
RT flows:
-- RTM flow 1 Flows
Engine Up= 10.10.50.241 Engine Down= 10.10.110.240 measured by
IPSrc = 0.0.0.0/255 IPDst = 0.0.0.0/255 a user in ipboss gui
Application = G729(92) TOS = 0
Socket=49(connected) TCPPort=19990 LastTicket=1225184552
-- RTM flow 2 Engine Up= 10.10.50.241 Engine Down= 10.10.110.240
IPSrc = 0.0.0.0/255 IPDst = 0.0.0.0/255
Application = HTTP(11) TOS = 0
Socket=50(connected) TCPPort=19991 LastTicket=1225184552
4.8.8 Debug show disc
Displays same information than in ipboss gui under Discovery tool.
debug show disc
[ DISCOVERY ] Discovery agent state
[status]
running = yes
overload = no
rule = 0|||0|1|||0|0|0|1||||0|100
Rule used for discovery
[Rule]
Local network filter: none
Remote network filter: none
Application filter: none
61 Discovery session contexts created
Showing 100 contexts (maxtop is 100)
CONNEXIONS: Hashing in 8192 buckets: 61 items in 61 buckets, min/avg/max 1 1 1
[Outgoing]
10.10.50.251|10.10.110.5|11-36~HTTP (http)|106/154358/1|55/2313/1|refcnt=2
10.10.50.251|10.10.110.6|11-36~HTTP (http)|106/154358/1|55/2313/1|refcnt=2
10.10.50.251|10.10.110.8|11-36~HTTP (http)|106/154358/1|55/2313/1|refcnt=2
10.10.50.251|10.10.110.7|11-36~HTTP (http)|106/154358/1|55/2313/1|refcnt=2
10.10.50.251|10.10.110.11|11-36~HTTP (http)|106/154358/1|55/2313/1|refcnt=2
10.10.50.251|10.10.110.10|11-36~HTTP (http)|106/154358/1|55/2313/1|refcnt=2
[Incoming]
10.10.50.250|10.10.110.3|92-103~G729 (rtp/rtcp)|216/12364/1|216/12364/1|refcnt=2
10.10.50.250|10.10.110.119|92-103~G729 (rtp/rtcp)|219/12350/1|219/12350/1|refcnt=2
10.10.50.250|10.10.110.168|92-103~G729 (rtp/rtcp)|211/12252/1|210/12192/1|refcnt=2
10.10.50.250|10.10.110.104|92-103~G729 (rtp/rtcp)|208/11800/1|208/11800/1|refcnt=2
10.10.50.250|10.10.110.140|92-103~G729 (rtp/rtcp)|178/9810/1|179/9870/1|refcnt=2
10.10.50.250|10.10.110.204|92-103~G729 (rtp/rtcp)|170/9224/1|171/9284/1|refcnt=2

4.8.9 Debug show cluster
Displays every flows involving at least a source and/or destination cluster.
4.8.10 Debug show drv
Displays low level information about ip packet:
Local traffic : traffic passing through ip|engine but which stays local to LAN
Ingress traffic: traffic going from local ipengine managed LAN subnet to remote subnet
Egress traffic: traffic going from remote subnet to local ipengine managed LAN subnet
Transit traffic: traffic going from a not locally managed subnet to a not locally managed
subnet, but passing through ip|engines.
debug show drv
[ Local traffic ]
CurrentAlarms= (none)
Traffic : 0
DefragError : 0
FragDatagrams : 0
Numb.Fragments : 0
[ Ingress traffic ]
Traffic : 57481993
DefragError : 0
FragDatagrams : 0
Numb.Fragments : 0
[ Egress traffic ]
Traffic : 40891069
DefragError : 0
FragDatagrams : 0
Numb.Fragments : 0
[ Transit traffic ]
Traffic : 0
DefragError : 0
FragDatagrams : 0
Numb.Fragments : 0

4.8.11 Debug show wan
Displays usefull information about current traffic. Displayed traffic rate are measured on LAN
interface. The value should be the same than the reported one in SA-site-throughput.
debug show wan Ingress and egress bytes

processed by ipengine
[ WAN COUNTERS ]
Ingress bytes: 2878824842 Egress bytes: 1397891640
INGRESS LAYER2 THROUGHPUT (delta time 24 sec):
IPv4 16069 pkt 8.703 MB 362.656 Kb/s
SNA 0 pkt 0.000 B 0.000 b/s
IPv6 0 pkt 0.000 B 0.000 b/s Ingress layer 2 throughput
IPX 0 pkt 0.000 B 0.000 b/s sorted by L2 protocols
ApTalk 0 pkt 0.000 B 0.000 b/s
Others 15 pkt 7.200 KB 296.000 b/s
EGRESS LAYER2 THROUGHPUT (delta time 24 sec):
IPv4 22388 pkt 157.949 MB 6.581 Mb/s
SNA 0 pkt 0.000 B 0.000 b/s
IPv6 0 pkt 0.000 B 0.000 b/s
Ingress layer 2 throughput
IPX 0 pkt 0.000 B 0.000 b/s
ApTalk 0 pkt 0.000 B 0.000 b/s
sorted by L2 protocols
Others 3 pkt 1.440 KB 56.000 b/s
INGRESS LAYER3 THROUGHPUT (delta time 24 sec):
InDomainQual 8882 pkt 4.137 MB 172.392 Kb/s sess 667
InDomNotCorr 5880 pkt 1.903 MB 79.304 Kb/s sess 0
InDomainVol 0 pkt 0.000 B 0.000 b/s sess 0
OutOfDomain 0 pkt 0.000 B 0.000 b/s sess 0
InterEngines 398 pkt 2.284 MB 95.200 Kb/s sess 189
Transit 0 pkt 0.000 B 0.000 b/s Ingress
sesslayer
0 3 throughput
Redirect 0 pkt 0.000 B 0.000 b/s sess 0
Ignored 0 pkt 0.000 B 0.000 b/s sess 0
EGRESS LAYER3 THROUGHPUT (delta time 24 sec):
InDomainQual 21500 pkt 154.546 MB 6.439 Mb/s sess 1727
InDomNotCorr 0 pkt 0.000 B 0.000 b/s sess 0
InDomainVol 0 pkt 0.000 B 0.000 b/s sess 0
OutOfDomain 0 pkt 0.000 B 0.000 b/s
Ingress layer 3 throughput
sess 0
InterEngines 354 pkt 893.688 KB 37.232 Kb/s sess 261
Transit 0 pkt 0.000 B 0.000 b/s sess 0
Redirect 0 pkt 0.000 B 0.000 b/s sess 0
Ignored 0 pkt 0.000 B 0.000 b/s sess 0debug show wan
4.8.12 Debug show engine
Displays all ip|engines created in ip|engine provisionning into ipboss configuration. We have
following information for each probe:
Public/private ip address
Capabilities FAST / XCOMP / DCOMP…Etc
TkDown is the number of correlation tickets received from remote ip|engine
Spur is the number of correlation tickets received but not attempted
Ambig is the number packets which have the same signature
Lost is the number of lost packet, no correlation ticket received

Remark: signature size depends of WAN access bandwidth, it is between 4 and 8 bytes.
debug show engine
[ 6 IP_ENGINES ]
Local engine:
ipe52 (10.0.18.52,10.10.50.241): LOCL FAST XCOMP DCOMP
Other engines:
ipe51 (10.0.18.51,10.10.50.240): FAST XCOMP DCOMP
ipe8 (10.0.18.8,10.10.82.240): FAST XCOMP DCOMP
ipe11 (10.0.18.11,10.10.110.240): FAST XCOMP DCOMP TSWAN
Out of domain (240.0.0.0): TOOD FOOD VIRT
*FastEgrInDomain* (255.255.255.255): FEID VIRT
4.8.13 Debug show subnet
Displays:
topology subnets created in topology provisionning with their associated ip|engines.

User subnets created in Application provisionning “REPORTING SUBNETS”
debug show subnet
[ 6 TOPOLOGY SUBNETS ]
NET120 (10.10.120.0/24) Out of domain
NET110 (10.10.110.0/24) ipe11
NET90 (10.10.90.0/24) Out of domain
NET82 (10.10.82.0/24) ipe8
NET50 (10.10.50.0/24) ipe52 - ipe51 MINE
Out of domain (0.0.0.0/0) Out of domain
[ 7 REPORTING SUBNETS ]
10.10.105.0 (10.10.105.0/24)
10.10.104.0 (10.10.104.0/24)
10.10.103.0 (10.10.103.0/24)
10.10.102.0 (10.10.102.0/24)
10.10.101.0 (10.10.101.0/24)
10.10.100.0 (10.10.100.0/24)
*Other* (0.0.0.0/255)
4.8.14 Debug show appli
Displays all applications enabled in Application provisionning.
debug show appli
[ APPLICATIONS (with default port ranges) ]

SAP (91): sap
PCAnywhere (90): pcanywhere
SOCKS (89): socks
SOAP (88): soap
RDP (86): rdp
Q931 (85): q931
DICT (84): dict
IRC (83): irc

………………….
G729 (92): rtp/rtcp Codec=[audio/G729]
G711a (93): rtp/rtcp Codec=[audio/PCMA]
G711u (94): rtp/rtcp Codec=[audio/PCMU]
G723 (95): rtp/rtcp Codec=[audio/G723]
RTP/RTCP (62): rtp/rtcp
4.8.15 Debug show uc
Displays User Class defined in ipboss GUI
debug show uc
[ USER CLASSES ]
G729(23): BwObj=2125 Crit=Top
Appli=G729
HTTP(24): BwObj=5000 Crit=Hig XCOMP XTCP
Appli=HTTP
other(0): BwObj=3750 Crit=Med XTCP
4.8.16 Abc
ABC is the process in charge of layer 7 recognizion, it manages L7 plugins.
4.8.17 Debug abc show filter
Check if application filter is enabled on the current ipengine
[ipe] debug.abc> show filter

show filter
[ FILTERS ]
521: SAP (base.*:application_id = 107)
522: PCAnywhere (base.*:application_id = 214)
523: SOCKS ((base.*:application_id = 116) or (base.*:application_id = 117))
524: SOAP (base.*:application_id = 115)
525: RDP (base.*:application_id = 90)
526: Q931 (base.*:application_id = 86)
527: DICT (base.*:application_id = 21)
528: IRC (base.*:application_id = 53)
529: Directconnect (base.*:application_id = 22)
530: X11 (base.*:application_id = 135)
531: RFB (base.*:application_id = 91)
532: POSTGRES (base.*:application_id = 84)
………………………………………………….
……………………………………………………..
580: HTTP-INDX (^.http.*:uri ~ /indx*)
581: HTTP-INX (^.http.*:uri ~ /inx*)
582: HTTP-IX (^.http.*:uri ~ /ix*)
583: HTTP-X (^.http.*:uri ~ /1x*)
584: HTTP-XA (^.http.*:uri ~ /2x*a)
585: HTTP-XAA (^.http.*:uri ~ /3x*aa)
586: HTTP-XAAA (^.http.*:uri ~ /4x*aaa)
587: HTTP-XAAAA (^.http.*:server ~ "*5x*aaaa")
588: HTTP-XAAAAA (^.http.*:uri ~ /6x*aaaaa)
589: HTTP-XAAAAAa (^.http.*:uri ~ /7x*aaaaaA)
590: HTTP-XAAAAAaa (^.http.*:uri ~ /8x*aaaaaAa)
591: HTTP-XAAAAAaaa (^.http.*:uri ~ /9x*aaaaaAaa)
592: HTTP-XAAAAAaaaa (^.http.*:uri ~ /10x*aaaaaAaaa)
593: HTTP-XAAAAAaaaaa (^.http.*:uri ~ /11x*aaaaaAaaaa)
594: HTTP-XAAAAAaaaaaa (^.http.*:uri ~ /12x*aaaaaAaaaaa)
595: HTTP-XAAAAAaaaaaaa (^.http.*:uri ~ /13x*aaaaaAaaaaaa)
596: HTTP-XAAAAAaaaaaaaa (^.http.*:server ~ "*14x*aaaaaAaaaaaaa")
597: HTTP-XAAAAAaaaaaaaaa (^.http.*:uri ~ /15x*aaaaaAaaaaaaaa)

598: HTTP-XAAAAAaaaaaaaaaa (^.http.*:uri ~ /16x*aaaaaAaaaaaaaaa)

599: HTTP-XAAAAAaaaaaaaaaaa (^.http.*:uri ~ /17x*aaaaaAaaaaaaaaaa)
600: HTTP-XAAAAAaaaaaaaaaaaa (^.http.*:uri ~ /18x*aaaaaAaaaaaaaaaaa)
601: HTTP-XAAAAAaaaaaaaaaaaaa (^.http.*:uri ~ /19x*aaaaaAaaaaaaaaaaaa)
602: HTTP-XAAAAAaaaaaaaaaaaaaa (^.http.*:uri ~ /20x*aaaaaAaaaaaaaaaaaaa)
603: HTTP-XAAAAAaaaaaaaaaaaaaaa (^.http.*:uri ~ /21x*aaaaaAaaaaaaaaaaaaaa)
604: HTTP-XAAAAAaaaaaaaaaaaaaaaa (^.http.*:uri ~ /x*aaaaaAaaaaaaaaaaaaaaa)
…………………………………………
………………………………………..
[ipe] debug.abc>
4.8.18 Debug abc show protocol
Check which protocols are current enabled in configuration:
bash-2.01$ debug abc show protocol | grep USED

17: cups (Common Unix Printer System), USED
18: dcerpc (Distributed Computing Environment Remote Procedure Call), USED
20: dhcp (Dynamic Host Configuration Protocol), USED
21: dict (Dictionary Server Protocol), USED
22: directconnect (DirectConnect), USED
23: dns (Domain Name Service), USED
27: epm (End Point Mapper), USED
29: ftp (File Transfer Protocol), USED
31: giop (General Inter-ORB Protocol (Corba)), USED
34: gre (Generic Routing Encapsulation), USED
37: http (HyperText Transfer Protocol), USED
38: ica (Independant Computing Architecture (Citrix)), USED
40: icmp (Internet Control Message Protocol), USED
41: icq (ICQ), USED
44: imap (Internet Message Access Protocol version 4), USED
48: ipp (Internet Printing Protocol), USED
…………………………………………
117: socks5 (SOCKSv5), USED
118: srvloc (Service Location Protocol), USED
119: ssdp (Simple Service Discovery Protocol), USED
120: ssh (Secure Shell), USED
121: ssl (Secure Socket Layer), USED
124: syslog (Syslog), USED
125: tcp (Transport Control Protocol), USED
126: tds (Tabular Data Stream), USED
127: telnet (Telnet), USED
128: tftp (Trivial File Transfer Protocol), USED
129: tns (Transparent Network Service (Oracle)), USED
131: udp (User Datagram Protocol), USED
135: x11 (X-Window), USED
146: l2tp (Level 2 Tunneling Protocol), USED
168: nbns (Netbios Name Service), USED
188: https (Secure HTTP), USED
214: pcanywhere (PCAnywhere), USED
bash-2.01$
4.8.19 Debug abc show uc
Display how much packets matched every UC
[ipe]$ debug abc show uc
[ ABC - USER CLASSES ]

UC 13(BusinessApp) Transac,top,Routine(preferred) matched 0
UC 15(ThinClient) Transac,top,Routine(preferred) matched 2264
UC 16(Voip G729) RealTim,top,Routine(preferred) matched 0
UC 17(VideoStreaming) RealTim,hig,Routine(preferred) matched 0
UC 20(BackOffice) Bckgrnd,med,Routine(preferred) matched 60
UC 21(NetworkServices) Bckgrnd,med,Routine(preferred) matched 145790
UC 19(MailCollaborative) Bckgrnd,low,Routine(preferred) matched 0
UC 18(Internet) Transac,low,Routine(preferred) matched 28898

UC 0(other) Bckgrnd,med,Routine(preferred) matched 65010

[ipe]$
4.8.20 Display all available plugins
For advanced user only:
[ipe6]$ shell
ipe6:~$ abc_plugin show all
base is enabled
unknown is enabled
malformed is enabled
incomplete is enabled
fragmented is enabled
8021q is enabled
aim is enabled
apollo is enabled
arp is enabled
atalk is enabled
bgp is enabled
bittorrent is enabled
cdp is enabled
cotp is enabled
cups is enabled
dcerpc is enabled
dec is enabled
dhcp is enabled
dict is enabled
directconnect is enabled
4.8.21 Enable or disable plugin
Reserved for advanced users with Ipanema support authorization only.
To enable a plugin:
ipe:~$ abc_plugin +mpls

mpls is enabled
ipe:~$debug_more stop
…OK
ipe:~$debug_more config
ipe:~$debug_more start
To disable a plugin:
ipe:~$ abc_plugin -mpls

mpls is disabled
ipe:~$debug_more stop
…OK
ipe:~$debug_more config
ipe:~$debug_more start

4.9 IP|Fast troubleshooting
4.9.1 Prerequisite
Verify that classification is correct with IP|true troubleshooting section.
4.9.2 Debug show uc
Check User Class are correctly defined in configuration
debug show uc
[ USER CLASSES ]
G729(23): BwObj=2125 Crit=Top
Appli=G729
HTTP(24): BwObj=5000 Crit=Hig XCOMP XTCP
Appli=HTTP
other(0): BwObj=3750 Crit=Med XTCP

4.9.3 Generality about the TC tree
All shaping control is based on TC tree
INGRESS /
EGRESS
IBA 0
IBAICOSQOSP 0.1.1
ICOS 0.1
IBAICOSQOSP 0.1.2
ICOS 0.2
IBAICOSQOSP 0.2.1
IBAICOSQOSP 0.2.2
User Class
ICOS 0.11
IBAICOSQOSP 0.11.1
Directions IBAICOSQOSP 0.11.2

ICOS 0.12
IBAICOSQOSP 0.12.1
IBA 1 IBAICOSQOSP 0.12.2
ICOS 1.1 IBAICOSQOSP 1.1.1
IBAICOSQOSP 1.1.2
Criticity X traffic ICOS 1.2

IBAICOSQOSP 1.2.1
type
IBAICOSQOSP 1.2.2
ICOS 1.11
IBAICOSQOSP 1.11.1
IBAICOSQOSP 1.11.2
ICOS 1.12
IBAICOSQOSP 1.12.1
IBAICOSQOSP 1.12.2
IBA 0 is always Out of domain direction
There are always 12 ICOS even if they are not instantiated every time. 12 because there are 4 level of
criticity and 3 kind of traffic so 4 x 3 = 12.
IBAICOSQOSP depends of domain configuration and traffic crossing the ipengine.

4.9.4 Debug show uflow
Suppose we want to troubleshoot SMTP traffic, use command:
bash-2.01$ debug show uflow | grep SMTP | more

> TCP, 205.223.229.101:25 - 10.74.100.19:49894, appli 32(SMTP), iba 10.48.89.231, id 464943005, Flow 0x88a9068, IbaGroup
0x080d7700, IcosGroup 0x08144d88, age 90
< TCP, 10.74.100.19:49894 - 205.223.229.101:25, appli 32(SMTP), iba 10.48.89.231, id 464943005, Flow 0x8f202c8, IbaGroup
0x080e2718, IcosGroup 0x080f9858, age 20
< TCP, 10.67.152.211:4962 - 205.223.229.51:25, appli 32(SMTP), iba 10.67.252.22, id 466395023, Flow 0x840ddec, IbaGroup
0x080e2718, IcosGroup 0x080f9858, age 180
> TCP, 205.223.229.51:25 - 10.67.152.211:4962, appli 32(SMTP), iba 10.67.252.22, id 466395023, Flow 0x852a6f0, IbaGroup
0x080d6fd0, IcosGroup 0x08132018, age 1580
< TCP, 10.41.36.51:4218 - 205.223.229.51:25, appli 32(SMTP), iba 10.208.101.62, id 466502995, Flow 0x81df470, IbaGroup 0x080e2718,
IcosGroup 0x080f9858, age 10
> TCP, 205.223.229.51:25 - 10.41.36.51:4218, appli 32(SMTP), iba 10.208.101.62, id 466502995, Flow 0x91998dc, IbaGroup 0x080d4be0,
IcosGroup 0x080f5c18, age 130
This command displays every ingress or egress flows, if you want to filter on the way add a grep “>” for ingress or grep “<” for egress
bash-2.01$ debug show uflow | grep SMTP | grep ">" | more

> TCP, 205.223.229.101:25 - 10.74.100.19:49894, appli 32(SMTP), iba 10.48.89.231, id 464943005, Flow 0x88a9068, IbaGroup
0x080d7700, IcosGroup 0x08144d88, age 510
> TCP, 205.223.229.51:25 - 10.67.152.211:4962, appli 32(SMTP), iba 10.67.252.22, id 466395023, Flow 0x852a6f0, IbaGroup
0x080d6fd0, IcosGroup 0x08132018, age 1390
> TCP, 205.223.229.51:25 - 10.41.36.51:4218, appli 32(SMTP), iba 10.208.101.62, id 466502995, Flow 0x91998dc, IbaGroup 0x080d4be0,
IcosGroup 0x080f5c18, age 40
Here we have filter ingress SMTP traffic for all directions (iba).
This result display some information about:
The direction
Iba 10.48.89.231 refers to a remote ipengine
Ibagroup 0x080d6fd0 refers to the direction in the TC tree of the ipengine you are connected on.
IcosGroup 0x08132018 refers to the Criticity x Traffic type in the TC tree of the ipengine you are connected on.
It is possible to find the same values in the debug fast show ingress or debug fast show egress command to improve troubleshooting.

4.9.5 Debug fast show { ingress | egress }
This command displays the current TC tree instantiated in the ipengine you logged on.
bash-2.01$ debug fast show ingress | more

[NAP 1 INGRESS IBA ICOS]
--------------------------------------------------------------------------------
IBA 0 (Out of domain), active, Ingress Uncontrolled, (dynIbaId 1, class 64:40)
RBP=194.89Mbs/64.00Kbs/194.89Mbs, MinBw=0.00 bs MaxBw=200.00Mbs
ABmin=48.00Kbs ABmax=200.00Mbs, WWHmin=47.03Kbs WWHmax=195.99Mbs, cfactor=1.00
GROUP 0x080d8aa8
PRIO:1 remote:240.0.0.0
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ICOS 0.1 (class 64:41), RBP=46.20Mbs/42.00Mbs/4.20Gbs, Prio=7, Qos2Cos=72
GROUP 0x080e5710, QosProfile=GQ_MAIL Client, Crit=Med, Type=Other
IBAICOSQOSP 0.1.1 (class 64:42), RBP=46.20Mbs/46.20Mbs/46.20Mbs, Prio=1
………………………………………………
--------------------------------------------------------------------------------
IBA 3 (GE-AM-CA-MISS001-BAM), active, Ingress Controlled, (dynIbaId 64, class 64:1000)
RBP=510.88Kbs/64.00Kbs/510.88Kbs, MinBw=0.00 bs MaxBw=28.00Mbs XCOMP-active
GROUP 0x080d4060
PRIO:1 remote:206.245.17.143
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ICOS 3.1 (class 64:1001), RBP=26.05Mbs/26.05Mbs/2.60Gbs, Prio=7, Qos2Cos=72
GROUP 0x08173e20, QosPr
……………………………………………….
--------------------------------------------------------------------------------
IBA 4 (GE-AM-CA-MARK001-BAM), active, Ingress Controlled, (dynIbaId 6, class 64:180)
RBP=30.73Mbs/64.00Kbs/30.73Mbs, MinBw=0.00 bs MaxBw=30.00Mbs XCOMP-active
GROUP 0x080d4118
PRIO:1 remote:206.245.13.143
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ICOS 4.1 (class 64:181), RBP=8.58Mbs/7.80Mbs/780.00Mbs, Prio=7, Qos2Cos=72
We recommend to use | more because the verbosity of this command is important. This command displays every directions and their associated UC
instantiated at the moment you are issuing the command.
IBA are the directions and ICOS refer to criticity x traffic type. IBA and ICOS are nodes but IBAICOSQOSP are the leafs of TC tree.

If you use the Hexa code of previous command debug show uflow inside this command, you’ll find the direction and the UC matched.
bash-2.01$ debug fast show ingress | grep -B 4 0x080d6fd0 | more

--------------------------------------------------------------------------------
IBA 69 (GE-EU-FR-RENN001-BAM), active, Ingress Controlled, (dynIbaId 29, class 64:740)
RBP=18.81Mbs/64.00Kbs/18.81Mbs, MinBw=0.00 bs MaxBw=8.00Mbs XCOMP-active
GROUP 0x080d6fd0
bash-2.01$
This IBA is number 69 and is matching the GE-EU-FR-RENN001-BAM ipengine. The class number is 64:740, this Id permits to watch tc shaping in
realtime.
bash-2.01$ debug fast show ingress | grep -A 2 0x08132018 | more

GROUP 0x08132018, QosProfile=GQ_Mail Srv, Crit=Med, Type=Other
IBAICOSQOSP 69.1.2 (class 64:743), RBP=3.00Mbs/3.00Mbs/8.66Mbs, Prio=1
PRIO:35 appli: (SMTP|TcpMail|mail-adminTMCM) xcomp
bash-2.01$
The class number is 64:743, this Id permits to watch tc shaping in realtime.
4.9.6 Tc
To display the shaper rate for a direction you can use previous class number of the IBA:
bash-2.01$ tc -s class show dev `getintf ing` parent 64: | grep -A 2 "htb 64:740"
class htb 64:740 root rate 200000Kbit ceil 200000Kbit burst 257561b cburst 257561b
Sent 185362203 bytes 293455 pkts (dropped 0, overlimits 0)
lended: 0 borrowed: 0 giants: 0
bash-2.01$
Here you can observe that direction is shaped at 200’000Kbit, 185’362’203 bytes in 293’455 pkts have matched this directions and no drop occur.
To display the shaper rate for a ICOS inside a direction you can use previous class number of the ICOS:
class htb 64:743 parent 64:741 leaf 2ce: prio 0 rate 3000Kbit ceil 6921Kbit burst 5439b cburst 10457b
bash-2.01$
Here you can see that parent is 64:741 and is not 64:740, because 64:741 is the ICOS (criticity X traffic type).

You can observe that the shaper regulate traffic at 3’000kbit and have never drop any packet.
Execute the same command with 64:741, you should have parent 64:740.
class htb 64:741 parent 64:740 rate 8298Kbit ceil 8298Kbit burst 12218b cburst 12218b
bash-2.01$
4.9.7 Debug fast show engine
Displays physical and virtual ip|engines, their ip addresses and their capabilities.
debug fast show engine

[ENGINES]
*EgrInDomain* (255.255.255.255) VIRTUAL EGR_IND
Out of domain (240.0.0.0) VIRTUAL NoFAST OWN_OOD
ipe11 (10.0.18.11,10.10.110.240) XCOMP DCOMP MYSELF
ipe8 (10.0.18.8,10.10.82.240) XCOMP DCOMP
ipe52 (10.0.18.52,10.10.50.241) XCOMP DCOMP
ipe51 (10.0.18.51,10.10.50.240) XCOMP DCOMP
4.9.8 Debug fast show subnet
debug fast show subnet

[4 TOPOLOGY SUBNETS (4 significant)]
NET110 10.10.110.0/24, Owners: ipe11
NET82 10.10.82.0/24, Owners: ipe8
NET50 10.10.50.0/24, Owners: ipe51,ipe52
Out of domain 0.0.0.0/0, Owners: Out of domain

[6 REPORTING SUBNETS]
10.10.105.0 10.10.105.0/24,
10.10.104.0 10.10.104.0/24,
10.10.103.0 10.10.103.0/24,
10.10.102.0 10.10.102.0/24,
10.10.101.0 10.10.101.0/24,
10.10.100.0 10.10.100.0/24,
4.9.9 Debug fast show wan
Displays all WAN access configured into ipboss GUI.
debug fast show wan

[WAN ACCESS]
SDSL 1024, INGBW=0.00 bs/1.02Mbs EGRBW=0.00 bs/1.02Mbs, Type=WanOther, Sla=none
LL 512 with ISDN backup, INGBW=0.00 bs/512.00Kbs EGRBW=0.00 bs/512.00Kbs, Type=WanOther, Sla=none
LL 2x1024, INGBW=-2000.00 bs/2.04Mbs EGRBW=-2000.00 bs/2.04Mbs, Type=LeasedLine, Sla=none
LL 128, INGBW=-2000.00 bs/128.00Kbs EGRBW=-2000.00 bs/128.00Kbs, Type=LeasedLine, Sla=none
FR 512, INGBW=0.00 bs/512.00Kbs EGRBW=0.00 bs/512.00Kbs, Type=FR_Access, Sla=none
FR 256, INGBW=0.00 bs/256.00Kbs EGRBW=0.00 bs/256.00Kbs, Type=FR_Access, Sla=none
ADSL 2048, INGBW=0.00 bs/512.00Kbs EGRBW=0.00 bs/2.04Mbs, Type=WanOther, Sla=none
ADSL 1024, INGBW=0.00 bs/256.00Kbs EGRBW=0.00 bs/1.02Mbs, Type=WanOther, Sla=none
8M-MCS, INGBW=0.00 bs/7.60Mbs EGRBW=0.00 bs/7.60Mbs, Type=WanOther, Sla=PLATINIUM
10M, INGBW=0.00 bs/10.00Mbs EGRBW=0.00 bs/10.00Mbs, Type=WanOther, Sla=PLATINIUM ** MINE **
100M, INGBW=92.00Mbs/92.00Mbs EGRBW=92.00Mbs/92.00Mbs, Type=Reserved2, Sla=PLATINIUM

4.9.10 Debug fast show tracking
bash-2.01$ debug fast show tracking |more

[NAP 1 TRACKING VARIABLES (Kbs)]
================================================================================
Ingress VP.1: ABmin 48.00Kbs ABmax 200.00Mbs Ingress / Egress
vpWWH 125.77Mbs vpWTU 113.78Mbs vpWTUave 106.05Mbs (margin= 2.1 %) Line shaper
NAS 378.93 TCNUM 423.43 balance 20.00
TRACK 15 [20 ] SMOTH 0 [150] BREATHE 0 [10 ]
Egress VP.1: ABmin 48.00Kbs ABmax 200.00Mbs
vpWWH 106.15Mbs vpWTU 59.99Mbs vpWTUave 53.34Mbs (margin= 2.0 %)
NAS 277.77 TCNUM 61.63 balance 20.00
================================================================================
Ingress IBA on NAP 1: Margin = bandwidth
ingIBA 0 (Out of domain), ABmin 48.00Kbs ABmax 200.00Mbs reserved by ipengine to
ibWWH 5.64Mbs ibWTU 3.48Mbs ibWTUave 3.20Mbs (margin= 4.0 %) protect new critical
TRACK 19 [20 ] SMOTH 0 [150] BREATHE 0 [10 ] traffic. % decrease when
ingIBA 2 (CAPGEMINI global), ABmin 48.00Kbs ABmax 200.00Mbs banwidth increase.
ibWWH 22.03Mbs ibWTU 18.42Mbs ibWTUave 16.08Mbs (margin= 2.4 %)
ingIBA 3 (GE-AM-CA-MISS001-BAM), ABmin 48.00Kbs ABmax 28.00Mbs
ibWWH 55.45Kbs ibWTU 300.00 bs ibWTUave 900.00 bs (margin= 10.0 %)
TRACK 4 [20 ] SMOTH 0 [150] BREATHE 0 [10 ] Internal metrics used for
ingIBA 4 (GE-AM-CA-MARK001-BAM), ABmin 48.00Kbs ABmax 30.00Mbs bandwidth tracking
ibWWH 4.46Mbs ibWTU 3.92Kbs ibWTUave 366.45Kbs (margin= 3.1 %)
ingIBA 5 (GE-EU-GB-WARD001-BAM), ABmin 48.00Kbs ABmax 24.00Mbs
ibWWH 2.06Mbs ibWTU 44.48Kbs ibWTUave 64.18Kbs (margin= 9.4 %)
TRACK 0 [20 ] SMOTH 0 [150] BREATHE 0 [10 ] ibWWH = iba What We Have
ingIBA 6 (GE-EU-GB-WOKI001-BAM), ABmin 48.00Kbs ABmax 28.00Mbs Estimated bandwidth for the
ibWWH 1.46Mbs ibWTU 3.68Kbs ibWTUave 50.00Kbs (margin= 7.4 %) specified IBA (direction)
ingIBA 7 (GE-EU-GB-LOND001-BAM), ABmin 48.00Kbs ABmax 177.00Mbs
ibWWH 61.37Kbs ibWTU 420.00 bs ibWTUave 399.00 bs (margin= 10.0 %)
TRACK 2 [20 ] SMOTH 0 [150] BREATHE 0 [10 ] ibWTU = iba What They Use
ingIBA 8 (GE-EU-GB-ASTO001-BAM), ABmin 48.00Kbs ABmax 12.00Mbs Consumed bandwidth for the
ibWWH 779.63Kbs ibWTU 854.00 bs ibWTUave 1.13Kbs (margin= 5.9 %)
specified IBA (direction)
ingIBA 9 (GE-EU-GB-WYNY001-BAM), ABmin 48.00Kbs ABmax 3.20Mbs

ibWWH 327.21Kbs ibWTU 584.00 bs ibWTUave 604.00 bs (margin= 4.1 %) ibWTUave = iba average of
ingIBA 10 (SO-EU-ES-BARC402-BAM), ABmin 48.00Kbs ABmax 3.00Mbs
What They Use
ibWWH 2.28Mbs ibWTU 5.24Kbs ibWTUave 5.52Kbs (margin= 4.5 %) Average of Consumed
TRACK 10 [20 ] SMOTH 0 [150] BREATHE 0 [10 ] bandwidth for the specified IBA
ingIBA 11 (GE-BK-CN-SHAN002-BAM), ABmin 48.00Kbs ABmax 2.00Mbs (direction)
ibWWH 271.81Kbs ibWTU 424.00 bs ibWTUave 1.90Kbs (margin= 8.8 %)
4.9.11 Debug fast show ibadyn
4.9.12 Debug fast show icos
4.9.13 Debug fast show qosp
4.9.14 Debug fast show ingress
[ipe52]$ shell
bash-2.01$ debug fast show ingress | more
[NAP 1 INGRESS IBA ICOS]
--------------------------------------------------------------------------------
IBA 0 (Out of domain), active, Ingress Uncontrolled, (dynIbaId 1, class 64:40)
RBP=600.00Kbs/64.00Kbs/600.00Kbs, MinBw=0.00 bs MaxBw=7.60Mbs
GROUP 0x081951a8
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ICOS 0.1 (class 64:41), RBP=660.00Kbs/600.00Kbs/60.00Mbs, Prio=7, Qos2Cos=40
GROUP 0x08181770, QosProfile=Mail, Crit=Med, Type=Other
IBAICOSQOSP 0.1.1 (class 64:42), RBP=660.00Kbs/660.00Kbs/660.00Kbs, Prio=1
PRIO:7 appli: SMTP
GROUP 0x081818e8, QosProfile=default, Crit=Med, Type=Other
IBAICOSQOSP 0.1.2 (class 64:43), RBP=660.00Kbs/660.00Kbs/660.00Kbs, Prio=1
PRIO:14 (default)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
GROUP 0x081817f0, QosProfile=FileTransfert, Crit=Low, Type=Other
PRIO:9 appli: FTP
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
GROUP 0x081815f0, QosProfile=G711, Crit=Top, Type=RTime
IBAICOSQOSP 0.4.1 (class 64:46), RBP=120.00Kbs/120.00Kbs/7.60Mbs, Prio=1

PRIO:1 appli: (G711a|G711u)

GROUP 0x08181670, QosProfile=G729, Crit=Top, Type=RTime
PRIO:3 appli: G729
GROUP 0x08181870, QosProfile=streaming, Crit=Top, Type=RTime
PRIO:11 appli: (RTP/RTCP|RTSP)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
GROUP 0x081816f0, QosProfile=PERF, Crit=Hig, Type=Trans
PRIO:5 appli: HTTP
--------------------------------------------------------------------------------
IBA 1 (virt203.255), NOT active, Ingress Uncontrolled Virtual-MD, (dynIbaId 0, class 0:0)
RBP=7.60Mbs/7.60Mbs/7.60Mbs, MinBw=0.00 bs MaxBw=1.00Mbs
ABmin=48.00Kbs ABmax=1.00Mbs, WWHmin=45.11Kbs WWHmax=939.84Kbs, cfactor=1.00
GROUP 0x080e64b8
Display Ingress TC tree, IBA (active or not)
4.9.15 “debug fast show egress”
4.9.16 “debug fast show sla”
4.9.17 “debug fast show monitor”
It displays ingress and egress IBA id (IBA = direction) according to configuration file (“__active__.ipmconf”) in section [INGRESS IBA] and [EGRESS
IBA].
It displays association between IBA/ICOS/QOSP for ingress and egress, refer to section [INGRESS IBAICOSQOSP] and [EGRESS IBAICOSQOSP].
Then, it displays on which IBA the ipengine detects activity (packets that match IBA) below lines “#groupes de detection d’activite”.
ipe:~# debug fast show monitor | more

[MONITOR]
[INGRESS IBA]
#columns = ibaId|name|mainPublicIp|mainPrivateIp
item = 0|Out of domain|240.0.0.0|240.0.0.0

item = 1|virt203.255|240.0.3.255|240.0.3.255
item = 2|virt203.254|240.0.3.254|240.0.3.254
………………….
[EGRESS IBA]
#columns = ibaId|name|mainPublicIp|mainPrivateIp
item = 0|Out of domain|240.0.0.0|240.0.0.0
item = 1|*EgrInDomain*|255.255.255.255|255.255.255.255
item = 2|virt203.255|240.0.3.255|240.0.3.255
…………………….
[INGRESS IBAICOSQOSP]
#columns = ibaId|icosId|crit|qosp|group|class
item = 0|1|Med|Mail|0x08206920|ing_64:42
item = 0|1|Med|default|0x08206a98|ing_64:43
item = 0|3|Low|FileTransfert|0x082069a0|ing_64:44
item = 0|4|Top|G711|0x082067a0|ing_64:46
item = 0|4|Top|G729|0x08206820|ing_64:47
item = 0|4|Top|streaming|0x08206a20|ing_64:48
item = 0|10|Hig|PERF|0x082068a0|ing_64:49
#groupes de detection d'activite
item = 0|-1|-1|-1|0x081231a0|ing_64:10
item = 1|-1|-1|-1|0x080f0fe8|ing_64:20
item = 2|-1|-1|-1|0x080f10a0|ing_64:30
………………………..
[EGRESS IBAICOSQOSP]
#columns = ibaId|icosId|crit|qosp|group|class
item = 0|1|Med|Mail|0x08207318|egr_64:42
item = 0|1|Med|default|0x08207490|egr_64:43
item = 0|3|Low|FileTransfert|0x08207398|egr_64:44
item = 0|4|Top|G711|0x08207198|egr_64:46
item = 0|4|Top|G729|0x08207218|egr_64:47
item = 0|4|Top|streaming|0x08207418|egr_64:48
item = 0|10|Hig|PERF|0x08207298|egr_64:49
item = 1|1|Med|Mail|0x08207d10|egr_64:82
item = 1|1|Med|default|0x08207e88|egr_64:83
item = 1|3|Low|FileTransfert|0x08207d90|egr_64:84
item = 1|4|Top|G711|0x08207b90|egr_64:86
item = 1|4|Top|G729|0x08207c10|egr_64:87
item = 1|4|Top|streaming|0x08207e10|egr_64:88
item = 1|10|Hig|PERF|0x08207c90|egr_64:89
#groupes de detection d'activite
item = 0|-1|-1|-1|0x081abe68|egr_64:10
item = 1|-1|-1|-1|0x0817df50|egr_64:20
item = 2|-1|-1|-1|0x0817e148|egr_64:30

…………………………………
[ICOS]
#columns = icosId|icosName
item = 0|other_high
item = 1|other_low
item = 2|other_med
item = 3|other_none
item = 4|realtime_high
item = 5|realtime_low
item = 6|realtime_med
item = 7|realtime_none
item = 8|transac_high
item = 9|transac_low
item = 10|transac_med
item = 11|transac_none
This command is useful when you troubleshoot ip|fast because you can easily find a IBA with a hexa code.
Example:
ipe:~# debug show uflow
[ 2 CLS MICRO-FLOWS ]
Current flows, > Ingress, < Egress, age in msec:
< UDP, 10.10.110.242:123 - 10.10.100.241:123, appli 39(NTP), iba 255.255.255.255, id 251625, Flow 0x81bf6c4, IbaGroup 0x080fb288,
IcosGroup 0x08200328, age 11740
< UDP, 10.10.90.242:123 - 10.10.100.241:123, appli 39(NTP), iba 255.255.255.255, id 251633, Flow 0x81bf6c4, IbaGroup 0x080fb288,
IcosGroup 0x08200328, age 860
ipe:~# debug fast show monitor | grep 080fb288
item = 1|-1|-1|-1|0x080fb288|egr_64:20
with < indicator, you know it is an EGRESS flow

it matches IBA 1 (which stands for OUT OF DOMAIN, iba 255.255.255.255)
its tc class is 64:20

4.9.18 “debug fast show ingltc”
4.9.19 “debug fast show egrltc”
4.9.20 Debug fast show rcg
RCG stands for Remote Coordination Group.

Virtual ip|engine
[anvpn-ctinip01]$ debug fast show rcg
[RCG CONFIGURATION (ref 1226938510)]
INGRESS (egress-proxy):
RCG 240.0.4.149, members: 10.110.3.194(10.1.10.82)
RCG 240.0.3.151, members: 10.110.3.194(10.1.10.82)
RCG 240.0.3.51, members: 10.110.3.194(10.1.10.82)
RCG 240.0.3.34, members: 10.110.3.194(10.1.10.82)
RCG 240.0.3.25, members: 10.110.3.194(10.1.10.82)
RCG 240.0.2.193, members: 10.110.3.194(10.1.10.82)
RCG 240.0.1.226, members: 10.110.3.194(10.1.10.82) Remote
RCG 240.0.0.5, members: 10.24.0.40 Coordinator
EGRESS (ingress-proxy): Member
RCG 240.0.3.51, members: 10.110.3.194(10.1.10.82)
RCG 240.0.0.5, members: 10.24.0.40
[anvpn-ctinip01]$
4.9.21 “debug fast show ingrcg”
4.9.22 “debug fast show egrrcg”

4.10 IP|Xcomp ZRE troubleshooting
4.10.1 About ZRE compression
4.10.1.1 Generalities
o ZRE: Zero delay Redundancy Elimination is available on all ipengines except old ipe10
or ipe14.
o Packet per packet compression, uses UD¨P tunnel, compress TCP and UDP traffic
o Typical compression ratio is between 1 and 10
o It introduce zero delay, it compress packet per packet, so it doesn’t wait to fill a buffer
before doing compression
o It is based on UDP tunneling
o Half-connections are independents
o ZRE compress only packets > 128bytes
4.10.1.2 Micro-tunnels
A compression tunnel is established between two IP|engines. The colour of each IP packet is
replicated in the UDP packet used to carry the compressed payload. Compression tunnel rules
are:
One UDP session per Client/Server communication

Source IP = source IP|engine
Destination IP = destination IP|engine
Destination Port = 19988
Source Port = Hash( @ClientIP, @ServerIP, IP protocol)
Source port is generated to keep advantage of Weighted Fair Queuing configured in router.
Session 1 Session 1 Session 1

Customer
Session 2 IP/VPN Session 1 workstation
workstation
IP|e IP|e Session 2
UDP sessions
between ip|e for
compression purpose
4.10.1.3 Restrictions
o Doesn’t work with vlan

4.10.2 Xcompconfig
Verify if a lan gateway is necessary.
YES YES
To display current configuration:
[ipe]$ xcompconfig -d
LAN Gateway : none
[ipe]$
To add a lan gateway:
[ipe]$ xcompconfig –l <LanGateway>

New configuration:
LAN Gateway : <LanGateway>
[ipe]$
Most part of time, LAN gateway is not required:
Sometimes, customers have cascaded IP subnets behind a Layer 3 equipment (like a router of
Layer 3 switch):
In default configuration, ipengine has no default gateway and every decompressed traffic which
destination ip address is not in the same subnet than ipengine subnet is sent back to the CE
router:
It works but generate more traffic to redirect by CE router and to switch (ignore) by ipengine.
A solution consists to configure a LAN gateway:

4.10.3 Debug abc show host
To display which host is elligible to ZRE compression (RAM based)
bash-2.01$ debug abc show host | more
[ XCOMP HOSTS ]
host 10.69.202.47 ttl 0
host 10.67.172.9 ttl 1
host 10.48.237.135 ttl 1
host 10.147.65.135 ttl 1
host 10.69.144.79 ttl 2
host 10.48.215.190 ttl 3
host 145.247.217.132 ttl 6
host 10.68.16.31 ttl 6
host 10.69.148.232 ttl 9
host 192.168.1.193 ttl 9
host 10.74.153.220 ttl 11
host 10.147.64.172 ttl 11
host 10.48.88.12 ttl 12
host 10.58.147.187 ttl 15
host 10.147.80.167 ttl 16
host 10.69.17.39 ttl 17
host 10.48.73.232 ttl 19
host 10.58.16.74 ttl 20
host 10.147.65.30 ttl 21
host 10.58.213.66 ttl 21
…………
4.10.4 Debug xcomp show config
With this command it is possible to have an overview of compression tunnel status
[ipe140axT]$ debug xcomp show config

[CURRENT CONFIGURATION] Global configuration
Connect ipe with private: true
Xcomp on domain: true
Tunnel mode: 0
UDP tunnel ports: com=19987 dec=19988
TCP dictionary port: dec=19988
Local IP addr: 10.10.60.241
Local mask: 255.255.255.0
Local default gateway: 10.10.60.253
Local lan gateway: 10.10.60.250
VRF1-SITE6-A (172.10.60.241,10.10.60.241):
- Administrative state = 1
- Optimizing = true
- Xcomp = true Dcomp = true
- Current comZiba = 1

- Current decZiba = 1
VRF1-SITE1-A (172.10.10.241,10.10.10.241):
- Dcomp = true comZiba = none
Tunnel to VRF1-SITE2-
- Xcomp = true decZiba = none A status
VRF1-SITE2-A (172.10.20.241,10.10.20.241):
- Dcomp = true comZiba = up
tunState = valid tunRetry = 0 tunAbort = 0
tryState = yes tryRetry = 0 alg/cap/msk = 7/1/0x0
cntState = connected cntRetry = 0 cntBreak = 0
tunMTU = 1500
- Xcomp = true decZiba = up
A tunnel status is composed of 2 way: Dcomp stands for decompression, Xcomp stands for
compression.
VRF1-SITE2-A (172.10.20.241,10.10.20.241):
- Dcomp = true comZiba = up
tunState = valid tunRetry = 0 tunAbort = 0
tryState = yes tryRetry = 0 alg/cap/msk = 7/1/0x0
tunMTU = 1500
- Xcomp = true decZiba = up
we can observe:
Decompression and Compression are enable
The direction is UP (comZiba / decZiba = up)
Compression and Decompression tunnels are OK (contState = connected)
Decompression tunnel never had abort (tunAbort=0)
4.10.5 Debug show flow –x
With debug show flow –x, it is possible to display details about compression:
o XCOMP for ZRE
o DBC for SRE
ipe140axT:~# debug show flow -x

> TCP, 10.10.60.210:445 - 10.10.20.210:1125, appli 42(SMB): Upstream_Cluster,
122541477 pkts CORREL(10), age 0, uc 30, life 76841
XCOMP IN:429232193 OUT:766456988 ratio:0.56 rate:78.56%
< TCP, 10.10.80.201:25 - 10.10.60.167:17694, appli 32(SMTP): Downstream_Cluster,
26 pkts, age 0, uc 29, life 0
< TCP, 10.10.20.210:1125 - 10.10.60.210:445, appli 42(SMB): Downstream_Cluster,
66636759 pkts, age 0, uc 30, life 76841
> TCP, 10.10.60.168:20650 - 10.10.80.201:25, appli 32(SMTP): Upstream_Cluster, 30
DBC IN:1052619 OUT:17339 ratio:60.71 rate:-98.35%
< TCP, 10.10.80.201:25 - 10.10.60.169:20196, appli 32(SMTP): Downstream_Cluster, 5


DBC IN:168 OUT:444 ratio:0.38 rate:164.29%
Let’s take 2 examples:

Here an ingress SMB half-connection is badly compressed with ZRE:
o Ratio = IN/OUT 429’232’193/766’456’988=0.56
o Rate = (OUT – IN) / IN (766’456’988-429’232’193) / 429’232’193 =78.56%

66636759 pkts, age 0, uc 30, life 76841
Here an egress SMB half-connection is badly compressed with ZRE:
o Ratio = IN/OUT 1’598’243’967/3’797’773’642=2.38
o Rate = (IN – OUT) / OUT (1’598’243’967-3’797’773’642)/ 3’797’773’642=57.92%
In the previous example, compression is very bad and negative because we generate more
traffic on WAN side.
Typically when we have more traffic on WAN side than on LAN side, it is probably due to
uncompressible traffic sent inside full length MTU packet. The ipengine will try to compress it,
then will have to encapsulate compressed packet inside a UDP tunnel. Finally
compressed/encapsulated packet is greater than MTU, so we have to fragment packet.
4.11 IP|xcomp SRE troubleshooting
4.11.1 About SRE compression
4.11.1.1 Generalities
Standard Redundancy Elimination (SRE) is available with ipengines AX series only running at
least v6.0.
It compresses TCP streams only and is transparent to Layer 2 (@MAC), Layer 3 (@IP) and Layer
4 (TCP port).
Every TCP connection compressed with SRE mechanism is splitted in 3 parts by 2 TCP
proxies:
• 1st third-connection between client and compressor
• 2nd third-connection between compressor and decompressor
• 3nd third-connection between decompressor and server

The second third-connection is TCP optimized.

SRE compression works at steam level, it means for N LAN ingress packets, you have only 1
WAN ingress packet. N can be 1 or more.
4.11.1.2 Implementation inside ip|agent
Poxification must be done on the first packet (SYN packet), classification process has not
enough time to complete correctly. Usually it needs few data packets to classify correctly (for
example, when you identify an application with an URL).
4.11.1.3 Dictionaries
With ip|agent v6.0 and v6.1, each ipengine has:
• only 1 compression dictionary (if the same data is compressed to several directions, it
is stored once only)
• 1 decompression dictionary per direction
Ipanema planned to merge all decompression dictionaries to only 1.
4.11.1.4 Transparency
Layer 2 transparency :
Conservation des @MAC
Conservation des VLAN
Layer 3/4 transparency :
Conservation des @IP
Conservation des n° de port TCP
Embedded proxy do not multiplex tcp connection :

1 TCP connection between client and compressor = 1 TCP connection between compressors
Ipengines do TCP optimization between themselves :
Window scaling
Optimization of TCP buffers
4.11.1.5 Proxy exceptions
Not all TCP connections can be proxified, there are some exceptions :
ipe140axT:~# cat /proc/ipanema/tcprox/exceptions

tcpcompd:
source ports:
dest ports:
21 563 1080 585 636 684 443 465 989 990
993 995
caxd:
source ports:
dest ports:
ipe140axT:~#
Actually, we know there is no gain to try to compress specific traffic like:
• Port 21 : control connection for FTP traffic
• Port 443: mainly used for HTTPS connections which are not compressible
• etc
4.11.1.6 Recommendations
This mechanism can introduce some delay, so it not recommended to use it with delay
sensitive traffic.
Prefer to enable SRE compression on file sharing, background traffic.
4.11.1.7 Restrictions
• Even if SRE mechanism is Layer 4 transparent, it uses the TCP option number 26, so if
you have firewall, it must permit TCP option 26 else proxification won’t work correctly.
• It is not possible to do for a same remote HTTP server A:
o SRE compression on a specific URL of server A
o And disable SRE compression on another specific URL of server A
o This can be explained because TCP proxification is done on the first TCP
packet = SYN and because we receive only one packet we can’t use Layer 7
engine to differentiate 2 URL traffic.
• SRE compression doesn’t work on half-connections
• IP fragmentation:
o Ip fragmentation / reassembly is not supported by the proxy
o MSS can be locally supported by ipagent to prevent fragmentation

o IP packet generated by tcp proxy have the DF bit set to 1

• Each time a half-connection is detected toward a server, the ipengine puts the ip
address of the server to its blacklist, in order to stop trying to proxyfied future TCP
connection toward this ip address. Timeout of blacklist is 10 minutes.
4.11.2 Measurement
4.11.3 Ipboss configuration
4.11.3.1 Ipagent
It is recommended to have at least v6.1 to run SRE compression.

4.11.3.2 Service activation
If you have the feeling SRE doesn’t work, verify if service is correctly enabled

4.11.3.3 Ipengine provisioning
Verify that ip|xcomp compress/decompress is checked for ipengines.
Remember SRE is only available for AX engines.

4.11.3.4 User Class provisioning
Compress option must be checked at User Class level, then click on “Advanced” tab.
Now verify that Standard is checked.

When you create a new User Class, default configuration of the advanced tab depend of QoS
profile selected:
o Realtime : ZRE and SRE not checked: compression is not enabled
o Transactional : ZRE only
o Background : ZRE and SRE
4.11.4 cat /proc/ipanema/tcprox/dump
This command displays which TCP connections are currently proxified.

ipe5ax$ cat /proc/ipanema/tcprox/dump
003033 WAN 0.0.0.0:35609 10.10.60.203:44618
004870 proxy 10.10.90.27:26057 10.10.60.203:44616
007016 proxy 10.10.90.30:45979 10.10.60.203:44619
007846 LAN 0.0.0.0:63335 10.10.90.27:26057
012372 WAN 0.0.0.0:35608 10.10.60.203:44617
012377 WAN 0.0.0.0:35606 10.10.60.203:44615
013333 LAN 0.0.0.0:63335 10.10.90.25:59256
014803 proxy 10.10.90.21:11112 10.10.60.203:44610
016661 LAN 0.0.0.0:63335 10.10.90.26:7663
018431 WAN 0.0.0.0:35607 10.10.60.203:44616
020786 LAN 0.0.0.0:63335 10.10.90.21:11112
021384 LAN 0.0.0.0:63335 10.10.90.24:47957
021565 LAN 0.0.0.0:63335 10.10.90.28:29146
023708 WAN 0.0.0.0:35602 10.10.60.203:44611
027905 LAN 0.0.0.0:63335 10.10.90.22:16493
033573 proxy 10.10.90.23:20467 10.10.60.203:44612
034070 WAN 0.0.0.0:35610 10.10.60.203:44619
034075 WAN 0.0.0.0:35604 10.10.60.203:44613
034665 proxy 10.10.90.22:16493 10.10.60.203:44611
037816 proxy 10.10.90.28:29146 10.10.60.203:44617
041736 WAN 0.0.0.0:35603 10.10.60.203:44612
043253 proxy 10.10.90.26:7663 10.10.60.203:44615
048790 WAN 0.0.0.0:35605 10.10.60.203:44614
050071 LAN 0.0.0.0:63335 10.10.90.30:45979
053843 WAN 0.0.0.0:35601 10.10.60.203:44610
056384 proxy 10.10.90.24:47957 10.10.60.203:44613
058941 LAN 0.0.0.0:63335 10.10.90.23:20467
059722 LAN 0.0.0.0:63335 10.10.90.29:7995
060646 proxy 10.10.90.29:7995 10.10.60.203:44618
064528 proxy 10.10.90.25:59256 10.10.60.203:44614
ipe5ax$
4.11.5 debug show flow –x
With debug show flow –x, it is possible to display details about compression:
o XCOMP for ZRE
o DBC for SRE
ipe140axT:~# debug show flow -x


< TCP, 10.10.80.201:25 - 10.10.60.167:17694, appli 32(SMTP): Downstream_Cluster,

26 pkts, age 0, uc 29, life 0
66636759 pkts, age 0, uc 30, life 76841
< TCP, 10.10.80.201:25 - 10.10.60.169:20196, appli 32(SMTP): Downstream_Cluster, 5
Let’s take 2 examples:

Here an SMTP connection is highly compressed with SRE:
o Ratio = IN/OUT 1’052’705/17’338=60.72
o Rate = (OUT – IN) / IN (17’338-1’052’705)/1’052’705=-98%
When Ratio > 1 and Rate < 0% , compression is useful. Else it means than compression
generate more traffic on WAN side than it received on the LAN side, however most part of time
you can’t prevent this phenomenon, typically when ipengine compresses upstream
acknownledgement like in the following example:

You can notice we have 168 IN packets for 444 OUT packets.
4.11.6 Common problems encountered with SRE
4.11.6.1 Traffic is not compressed with SRE, but it is with ZRE
• Sometime, it occurs when traffic is coming from or going to an ipengine cluster, as long as the
ipengine cluster is not resolved by the remote ipengine, then the traffic is not proxified and
can’t be compressed with SRE. And in this case, ZRE can compress traffic instead of SRE.
4.11.6.2 Out of domain traffic
In default configuration, Out of domain traffic is not compressible, however it is possible to

compress it by modifying default topology subnet associated to Out Of Domain.
Create 2 new topology subnets: 0.0.0.0/1 and 128.0.0.0/1both topology subnets are
equivalent to 0.0.0.0/0. Then you have to associate both new subnets to an ipengine.
4.12 IP|xcomp reporting
SRE and ZRE compression statistics are merged into the same reports.

4.12.1 Reporting
4.12.2 Potential cases
On the following example, compressor sends more traffic on the WAN side, than it receives on
the LAN side. Maybe, It can be explained because ipengines tries to compress small packet
like acknowledgement, here we can notice that compress traffic is very small compared to
decompress traffic, so it is probably ack packet.

4.13 IP|xapp troubleshooting
This section is usefull for CIFS protocol only.
Remember that a CIFS connection is really long, I mean each time you open a shared (ex: a
folder) resources on a network, the connection is established and is kept opened untill you
decide to remove the network mount.
For example, a employee begins its works at 9:00AM, and is going to finish at 6:00PM, if this
user has network drives on its workstation :
A TCP connection is opened at 9:00AM and is closed at 6:00PM, so the same connection is
kept opened all day long, even if there is no traffic during the lunch time and if computer is still
running.
CIFS optimization works at protocol level and need to proxify TCP connection on 1 ipengine,
only ipengine installed on the LAN of the workstation will proxify the connection.
Remark: Xapp feature must be enable before the CIFS connection is established, else TCP
connection won’t be proxified by CIFS proxy.

4.13.1 About CIFS acceleration
4.13.1.1 Protocol optimized
The CAX engine is the CIFS optimization process. It can optimize only dialect “NT LM 0.12”
with is mainly used in today Windows workstation.
It is based on use of TCP proxy embedded in ipengine and also used for SRE compression.
It will try to optimize every TCP connection running on port 139 (NetBIOS) or 445 (SMB).
Port 139 is used by the old implementation of File sharing using NetBIOS
Port 445 is used for transport of SMB directly over TCP
4.13.1.2 Implementation inside ip|agent
CIFS optimization runs before every other optimization or measurement processes inside
ipengine, it means:
o Ip|true can only see the packets actually sent over the WAN by the CAX engine, packets
generated locally can’t be seen by ip|true
o Ip|fast will regulate only CIFS optimized traffic, it means CIFS client may experience a
higher bandwidth than the one enforced by ip|fast.
o Ip|xcomp will compress actuallly sent CIFS bytes
4.13.1.3 CIFS acceleration technics
Acceleration technics used by CAX:
o By default CAX engine uses 60Kbytes CIFS block instead of 4Kbytes (16Kbytes) used by
Windows NT/XP (Windows 2000)
o Read-ahead: anticipate read blocks
o Caching: dedicated CIFS cache
o Xapp can be coupled by SRE compression
Tele CIFS optimization is not possible.
4.13.1.4 Restrictions of CIFS acceleration
• Asymetric routing not support due to TCP proxy

• Dialect other than NTLM 0.12 or SMBv2
• SMB signing or sealing used
• CIFS sessions are processed independantly, optimization of a session can’t benefit to another
session
• Linux-based CIFS connection are not accelerated or gain is light.
• If a network drive is mounted before Xapp is enabled, then running sessions won’t be
accelerated by ipengine.

4.13.2 Ipboss configuration
In order to accelerated CIFS traffic, the Ipanema domain must be well configured.
4.13.2.1 Ipagent
Ipengine running CIFS acceleration must runs at least ipagent 6.1
4.13.2.2 Service activation

4.13.2.3 Ipengine provisioning
4.13.2.4 User Class
It is not mandatory to have a specific User Class with SMB protocol. Once you have enabled
Xapp on an ipengine and enabled global the Xapp service, each SMB/Netbios sessions passing
through the ipengine will be accelerated.
However, you can create a specific User Class for file sharing to apply QoS objective to this
traffic.
4.13.2.5 Realtime graphs
CIFS throughput measured displayed in ipboss realtime graph is measured by ip|true process,
it means sometimes it won’t displayed the exact throughput like the one experience by CIFS
user.

4.13.3 Reporting
Throughput displayed here is measured at the CAX engine, it is not measured by ip|true and so
reflect the bandwidth experienced by end user.

4.13.4 Debug cifs show version
Useful to display version of CIFS optimization engine.
140T:~# debug cifs show version
[CIFS Version ]
CAX Version: 1.20
140T:~#
Version 1.20 is embedded in ipagent 6.1.4
4.13.5 Debug cifs show state
To display current status of CIFS optimization.
Is it configured in ipboss ? Is it enabled in ipboss ? Is it running on ipengine ?
140T:~# debug cifs show state
[CIFS Status ]
Configured? yes Enabled? yes Running? yes
140T:~#
4.13.6 Debug cifs show conns
Display information about CIFS connections.
• Here maximum optimized CIFS connections is 200
• It displays every CIFS connections currently optimized and which ipengine is currently optimizing
the connection.
• If some CIFS connection are not accelerated, they are listed below. Typically, when the maximum
of CIFS optimized connections is reached
• Refused connections are connections which can’t be optimized. Typically when CIFS is sealed or
when dialect is not supported by CIFS engine.
140T:~# debug cifs show conns
[CIFS Conns ]
Maintaining 1 connections (max is 200)
Accelerated Connections:
0001: Accelerated: 10.10.20.210:1125 -> 10.10.60.210:445 (sentry 10.10.60.241
UC 30 AC 42) lanRTT 18 wanRTT 50 GMT 20100705 16:04:56
Number of Accelerated Connections: 1
Non-Accelerated Connections:
Number of Non-Accelerated Connections: 0
Summary of Refused Connections:
140T:~#
4.13.7 Debug cifs show conns –d
Display the same information than “debug cifs show conns” but with more details.
140T:~# debug cifs show conns -d

[CIFS Conns ]
Maintaining 1 connections (max is 200)
Accelerated Connections:
0001: Accelerated: 10.10.20.210:1125 -> 10.10.60.210:445 (sentry 10.10.60.241
UC 30 AC 42) lanRTT 18 wanRTT 38 GMT 20100705 16:04:56
LAN Bytes sent: 127579963061
LAN Bytes recv: 360981635
WAN Bytes sent: 208896374
WAN Bytes recv: 127415955371
-- Number of CIFS requests from client total: 5106363 prev.hour: 325087
curr.hour: 217365 last.five: 22715curr.five: 28627
-- Number of CIFS responses from server total: 2800768 prev.hour: 178743
-- Number of CIFS bytes from client total: 360983725 prev.hour: 22980293
-- Number of CIFS bytes from server total: 712296638 prev.hour:
1691687921 curr.hour: 1022618600 last.five: 656115351curr.five: 583240818
Number of Accelerated Connections: 1
Non-Accelerated Connections:
Number of Non-Accelerated Connections: 0
Summary of Refused Connections:
140T:~#
For each CIFS connection the following detail is displayed:
• LAN Bytes sent : sent by ipengine to CIFS client

• LAN Bytes received : sent by CIFS client to ipengine
• WAN Bytes sent : sent by ipengine to CIFS server
• WAN Bytes received: sent by CIFS server to ipengine
• Number of CIFS requests (response) from Client (Server)
• Number of CIFS bytes from Client and Server
This command is useful to see which connection is active of not, refer to curr.five or last.five
counters.
4.13.8 Debug cifs show global
140T:~# debug cifs show global
[CIFS Statistics]
** CAX Statistics:
-- Max number of concurrent connections total: 1 prev.hour: 0
curr.hour: 0 last.five: 0 curr.five: 0
-- Number of accepted connections total: 1 prev.hour: 0 curr.hour:
0 last.five: 0 curr.five: 0
-- Number of refused connections total: 0 prev.hour: 0 curr.hour:
0 last.five: 0 curr.five: 0
-- Number of connections refused because CIFS connection was signed/sealed
total: 0 prev.hour: 0 curr.hour: 0 last.five: 0 curr.five: 0
-- Number of connections refused because lack of resources total: 0
prev.hour: 0 curr.hour: 0 last.five: 0 curr.five: 0
-- Number of connections refused because unsupported CIFS dialect total: 0
prev.hour: 0 curr.hour: 0 last.five: 0curr.five: 0
-- Number of CIFS requests from client total: 5144111 prev.hour: 325051
-- Number of CIFS responses from server total: 2821168 prev.hour: 178745
-- Number of CIFS bytes from client total: 363579461 prev.hour: 22978025
-- Number of CIFS bytes from server total: 1731268622 prev.hour:
1691852147 curr.hour: 187338965 last.five: 672869461curr.five: 525986112
** end CAX statistics

ACCELERATION RATIO: 1.901705

140T:~#
Displays information about CIFS optimization engine activity. And also give an acceleration ratio
for the ipengine.
4.13.9 cat /proc/ipanema/tcprox/dump
This command can be used to display CIFS connections proxified like TCP connection proxified
for SRE purpose.
140T:~# cat /proc/ipanema/tcprox/dump

006381 LAN 0.0.0.0:32808 10.10.20.210:1125
058747 WAN 0.0.0.0:32810 10.10.60.210:445
128546 proxy 10.10.20.210:1125 10.10.60.210:445
140T:~#
4.14 Smartpath troubleshooting
SmartPath is available in 2 modes : Smartpath V1 (TOS) and Smartpath V2(MAC).

Troubleshooting commands are identical between both modes.
4.14.1 Display global options
Permit to display on the ipengine “advanced parameters” define in Ipboss:
Sticky_choice :
• Yes : path decision is done on the first packet and all following packets will use the
same path.
• No: path decision is done per packet, for every packets
Slave return:
• Yes: half-connection (SYN+ACK) will always use the same path than the half-
connection (SYN)
• No: both half-connections are independents
Sens_policy:
The policy determine the behaviour of the ipengine
• Prefered: Business traffic Business NAP, Routine traffic Routine NAP, except if
NAP is down or if bandwidth/QoS criteria are not met
• Strict: Business traffic Business NAP, Routine traffic Routine NAP, if a NAP is
down then ipengine doesn’t take any decision (TOS mask xxxx00xx)
• Protected: Business traffic Business NAP, Routine traffic Routine NAP, but
Business traffic can fallback to Routine NAP
• Ordered: Business traffic Business NAP, Routine traffic Routine NAP, but Routine
traffic can fallback to Business NAP. The WAN access must offer at least the same
trust level than required in the UC

• Backup: Business traffic Business NAP, Routine traffic Routine NAP, fallback
possible if connectivity is down.
ipe140axT:~# debug sph show conf
[ OBPS CONFIGURATION ]
Options:
sticky_choice=no slave_return=no sens_policy=backup
Parameters:
sizingCoef=100 normConst=3 fitThreshold=0
qosTimer=300 qosAgeout=10800
eddPeriod=60 eddTimer=120 eddObjective=1.0
goodQosWeight=1 badQosWeight=1
Edd/Qos Weights:
Bckgrnd={8/2} RealTim={2/8} Transac={5/5}
ipe140axT:~#
Weight of criteria for smartpath decision according UC type:
• Background: 80% bandwidth / 20% QOS criteria (loss, delay, jitter, rtt)
• Transactional : 50% bandwidth / 50% QOS criteria (loss, delay, jitter, rtt)
• Real Time : 20% bandwidth / 80% QOS criteria (loss, delay, jitter, rtt)
4.14.2 Display NAPs configuration
Naps configuration for Smartpath V1: TOS mode
ipe140axT:~# debug sph show naps
[ OBPS NAPS ]
Number of NAPS: 2
NAP 1 (20MBPS-SMART-B): trust_level=Business default
NAP 2 (20MBPS-SMART-R): trust_level=Routine
[ OBPS NAP MAPPINGS ]

NAP 1 (20MBPS-SMART-B) tos marking value=04/mask=0c
NAP 2 (20MBPS-SMART-R) tos marking value=08/mask=0c
ipe140axT:~#
nap1 is tagged 0x04 with a mask of 0x0c which means xxxx01xx
nap2 is tagged 0x08 with a mask of 0x0c which means xxxx10xx
Naps configuration for Smartpath V2: MAC mode
ipe:~# debug sph show naps
[ OBPS NAPS ]
Number of NAPS: 2
NAP 1 (10MBPS): trust_level=Business default
NAP 2 (2MBPS): trust_level=Business
[ OBPS NAP MAPPINGS ]

NAP 1 (10MBPS) ingress=00:05:9A:06:48:00 egress=()
NAP 2 (2MBPS) ingress=00:04:C0:5D:57:E0 egress=()
ipe:~#
According to what was configured in ipengine provisionning module on ipboss, the ipengine
has resolved both IP addresses of CE routers in front of which it is installed.

4.14.3 Display NAPs status
When everything goes well: Naps are UP, there is no error or broken path
ipe140axT:~# debug sph show conn
[ OBPS NAP STATES ]

LocalNAP 1 (20MBPS-SMART-B) state is up
LocalNAP 2 (20MBPS-SMART-R) state is up
[ OBPS CONNECTIVITY ]
ipe140axT:~#
When a problem is detected on a NAP, it is always according a destination
ipe140axT:~# debug sph show conn
[ OBPS NAP STATES ]

LocalNAP 1 (20MBPS-SMART-B) state is up
LocalNAP 2 (20MBPS-SMART-R) state is up
[ OBPS CONNECTIVITY ]
Remote VRF1-SITE2-A (10.10.20.241) from LocalNAP 1: OK 2: BROKEN
ipe140axT:~#
Here we can observe destination “VRF1-SIET2-A” is only reachable through NAP1 because
NAP2 is broken.
4.14.4 Display flows forwarding
ipe140axT:~# debug show flow -d | more

pkts CORREL(10), age 0, uc 29, life 6
EngineUp/Down: VRF1-SITE6-A:2/VRF1-SITE2-A, priv 111(smtp), client, tcpFlags
SYN+ACK, iba 10.10.20.241, uc 29(UC-MAIL), flowId 17675239, Xcomp, tkCount 14
TopoSrc/Dst: VRF1-NET6/VRF1-NET2 UserSrc/Dst: *Other*/*Other*
< UDP, 10.10.20.205:20036 - 10.10.60.193:10042, appli 93(G711a):

Downstream_Cluster, 1110 pkts, age 0, uc 28, life 22
EngineUp/Down: VRF1-SITE2-A/VRF1-SITE6-A:1, priv 104(rtp/rtcp), client, iba
10.10.20.241, uc 28(UC-VOIP), flowId 17675051, tkCount 0
Here we can observe 2 flows:
• An ingress TCP flow forwarded to NAP2 (VRF1-SITE6-A:2)
• An egress UDP flow coming from NAP1 (VRF1-SITE6-A:1 )
4.14.5 Reporting
4.14.5.1 Default reporting with smartpath
Each time, smartpath is enabled on an ipengine, corresponding site folder is split in N+1 folder
(N: number of WAN accesses declared in ipengine provisionning):

For example, the office is called

“RENNES1” and is controlling 2 WAN
accesses, after enabling smartpath, you
have 3 folders:
• RENNES1: aggregation of all NAP
• RENNES1 x NAP id: 1 : display

statistics corresponding to NAP1
• RENNES1 x NAP id: 2 : display

statistics corresponding to NAP2
4.14.5.2 Advanced reporting for troubleshooting
In default configuration, NAP0 is not displayed but it can be interesting to report statistics
corresponding to NAP0. To proceed, you need to create a metaview on ipboss:

Like many other parameters, Wan Access Id (refering to NAP) are now available in Metaview
definition.
Then, you can instantiate a site report on this metaview.
4.15 Real time graph problem
4.15.1 DEBUG DUMP CONFIG
NO YES
The debug dump config command displays configuration information, here in bold blue are the
TCP dynamic ports used.
[ipe]$ debug dump config

[ CONFIGURATION ]
ConfigName = __active__@2@17
Interfaces: /dev/sentry0 (eth0) and /dev/sentry1 (eth1) Bridge = brg0
CapGigabit[no] Cap100Mbs[yes] CapFullD[yes] CapGps[yes]
CapSerial[yes] CapEadi[no] CapDual[no] CapXcomp[yes]
SerialMode[yes] EadiMode[no] DualMode[no]

ip|boss address = 192.130.18.64 ExcludeMode: sentry<=>any

Local IP address= 10.0.14.253 MAC address = FE:FD:4B:0D:50:BF
DrvNbTickets = 4096 MinToRead = 50 MaxAcqFrames = 256
MaxDefragCtxt= 2048 MaxFragments = 8096
TkgMaxFlows = 16000 ArchMaxFlows = 8192 RTMMaxFlows = 8
RTMPeriod = 10 ArchPeriod = 60 MaxCrPerFlow = 6
TickDownDelay= 50 TickDownPer. = 2 CnxRetryPer. = 4
LossThreshold= 5 TTP_Port = 19999 TTP_MaxLength= 1293
SynchroThresh= 10000 us (+10%:11000 us) SpuriousDelay= 5000
NbThreshDlay = 8
ThreshDlay = 10000 20000 50000 100000 200000 500000 1000000 2000
000
Options = 0x001b SchedPrio SerialMode Softdog Watchdog
NbTickets = 30000, 30000 free (1275 + 28725 in TKG) TicketCrcNb = 4096
UflowAging = 10 ConCnxTcpAging = 30 SentryTTPAging = 600 ConCnxTcpDeadAgi
ng = 300
Port ITP: 19995 ITP Mode active, alpha 0.000020, beta 0.000500
ArpCpeInitPeriod = 10 ArpCpePeriod = 60 DeadCpeAgingDelay = 1200
ForceCrc24 thresholds low 10000, high 15000
Engine capacity advertising = yes (on UDP port 19996)
RT flows ports : 19990-19993;
[ CRC INFOS ]
Using PaquetID ? no Using TCP/UDP Ports ? no Using TCP seq/ack ? yes
[ipe]$
This TCP port range must be identical to the one configured on IP|Boss domain’s configuration
file.
4.15.2 DEBUG DUMP RTM
NO YES
This debug dump rtm displays each TCP port used for Real time monitoring.
A [0] and [1] respectively mean it not used or it is used.
[ipe]$ debug dump rtm

[ REAL TIME MONITORING ]
RT ports:
19990 [1] 19991 [0] 19992 [0] 19993 [0]
RT flows:
[ipe]$
Here a graph is open on port 19990.
4.15.3 NETSTAT –TNA
NO YES
The netstat –tna command displays every active/listening connections. Process responsible for
realtime graphs must be listening on tcp port 19994.
[ipe]$ netstat -tna

Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:23 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:19994 0.0.0.0:* LISTEN

tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN

tcp 0 0 0.0.0.0:19996 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:19999 0.0.0.0:* LISTEN
tcp 0 0 10.0.14.253:443 10.238.39.75:4102 ESTABLISHED
tcp 0 0 10.0.14.253:80 10.238.39.75:1563 ESTABLISHED
tcp 0 132 10.0.14.253:23 10.238.39.75:4500 ESTABLISHED
[ipe]$
4.16 Other useful tools
4.16.1 Password recovery
When the ip|engine password is lost, the only way to recover password is connect it via the
console port.
It is possible to connect it via a reverse telnet (through a CISCO router) or directly with a PC
connected with a console cable.
Then follow the procedure:
Connect to IP|engine to its console port (9600/8/no/1)

Power off, power on
When message “LILO boot:” appears, enter the ‘rescue’ command (characters are echoed
two time)
IP|engine will restore default password “ipanema” to “ipanema" user.
After a second reload, Ip|engine is operational.
LILO boot: rreessccuuee
4.16.2 Tcpdump tool
Tcpdump tools is a linux tool to analyze IP packet on-demand.
Becarefull, if you want to capture packets and save them into a file, you need to
to store capture
file into /tmp directory else ip|engine
ip|engine could crash.
crash.
Synthax is tcpdump [primitives ]
Primitves description
host IPHOST Displays IP packets where host IPHOST is source or destination
dst host IPHOST Displays IP packets where host IPHOST is destination
src host IPHOST Displays IP packets where host IPHOST is source
ether host ETHERHOST Displays IP packets where MAC address host ETHERHOST is source or destination
ether dst ETHERHOST Displays IP packets where MAC address host ETHERHOST is destination
ether src ETHERHOST Displays IP packets where MAC address host ETHERHOST is source
port TCPUDPPORT Displays IP packets where tcp or udp port is equal to TCPUDPORT
src port TCPUDPPORT Displays IP packets where tcp or udp source port is equal to TCPUDPORT
dst port TCPUDPPORT Displays IP packets where tcp or udp destination port is equal to TCPUDPORT
less LENGTH Displays IP packets if length is less than LENGTH

greater LENGTH Displays IP packets if length is greater than LENGTH

ip proto PROTO Displays IP packets if protocol is PROTO possible are tcp, udp, icmp, igmp, igrp,pim,
ah , esp, vrrp
ether proto ETHPROTO Displays Ethernet frames if Layer 3 protocol is ETHPROTO. Possible values are
icmp Displays icmp packets
arp Displays arp packets
vlan VLANID Dispays packets from vlan VLANID
Usefull templates:
Listen on brg0 interface, filter on host 10.10.110.250 and write packets into a file ftp.cap
tcpdump -i brg0 ip host 10.10.110.250 -w /home/ipanema/ftp.cap
/home/ipanema/ftp.cap
To stop, you must to CRTL-C
Listen on brg0 interface, filter on port 443. Command exits after 102400 bytes
tcpdump -i brg0 port 443 -C 102400
Listen on brg0 interface and display vlan tag, limit on port 443 and display only first hundred
packets
tcpdump -i brg0 vlan –e and port 443 -c 100
Available options
options description
-s <size> capture only <size> bytes for each packet
-s 0 capture the whole packet
-S display TCP sequence number
-i <intf> listen on <intf> interface
-e display MAC addresses
-c <num> display only <num> packets and finishes
-w /tmp/<file> write capture into a file . Warning: write only on /tmp directory.
Advanced filters
filter description
‘ip[1]=0xb8’ filter ip packet with DSCP = 101110xx
‘tcp[tcpflags] = tcp-syn’ filter only SYN packets
‘tcp[tcpflags] & tcp-syn = tcp-syn’ filter SYN packets (SYN or SYN-ACK..)
'tcp[tcpflags] & (tcp-syn|tcp-ack) == (tcp-syn|tcp-ack)' filter only SYN-ACK packets
ether proto 0x0806 filter ARP packets
'ip[6] & 0x20 = 0x20' or 'ip[7] != 0x00' filter fragments
Useful keywords (that can be used in tcpdump filters)
keywords
icmp-echoreply icmp-tstampreply
icmp-unreach icmp-ireq

icmp-sourcequench icmp-ireqreply
icmp-redirect icmp-maskreq
icmp-echo icmp-maskreply
icmp-routeradvert tcp-fin
icmp-routersolicit tcp-syn
icmp-timxceed tcp-rst
icmp-paramprob tcp-push
icmp-tstamp tcp-ack
tcp-urg
Tcpdump command is rich of many options. Some useful are liste above, but many other are
available. To go furthermore, visit : http://www.tcpdump.org/
4.16.3 Reboot and shutdown
Can be used to reboot ip|engine. Takes no argument but ask for confirmation.
[ipe]$ reboot
please confirm: really reboot now [y/n] ? y
rebooting...
goodbye.
ask
Warning: if you entered shell before rebooting, the command doesn’t a sk you a confirmation.
[ipe]$ shell
bash-2.01$ reboot
WARNING: could not determine runlevel - doing soft reboot
(it's better to use shutdown instead of reboot from the command line)
bash-2.01$
4.16.4 Version
Permits to display hardware and software version running:
[ipe10]$ version
*** Hardware Version ***
Tag : 10I0
Name : 1800T-4GB-CF512-3-X-G1
Rev. : 01
*** Software Version ***
Global : 5.0.3
Kern : 5.0.3.6
Ipe : 5.0.3.6
Tool : 5.0.3.0
[ipe10]$
Hardware Tag, Name and Rev help Ipanema to know which device you have.
Software is split in 3 components: Kern (linux kernel), Ipe (Ipanema software), Tool (Ipanema
Toolbox).

4.16.5 Upgrade
Since version 4.3.7, ip|engine can be manually upgraded. With earlier release, an ipboss was
necessary to upgrade ip|engines.
upgrade
Usage: upgrade [-h]
upgrade status
upgrade cancel
upgrade -s <server> -d <directory> [-l <login> -p <password>]
[-c none|alias] [-v]
Options:
-c none Connect the server with the private address (default)
-c alias Connect the server with the public address (if configured)
-v Active the verbose mode
To upgrade manually an ip|engine, a FTP server with appropriate ip|agent images is necessary.
FTP server can be a ip|engine.
Tipically, we use command:
upgrade -s <server> -d <directory> [-l <login> -p <password>] -v
<server> is your FTP server.
<directory> is the directory where is stored file ipengine.ipmsys
Optionnaly if user and password is not ipanema/ipanema specify credentials with –l <login> -p
<password>
-v is the verbose option, it displays upgrade follow-up.
To check upgrade status, you can use command
upgrade status
To cancel upgrade, use:
upgrade cancel
4.16.6 Getintf
getintf
Display device name of logical interfaces
Usage: getintf [-h] Print this help and exit
getintf <intf> Print device name of a logical interface
getintf all Print device names of all logical interfaces
With:
<intf>=lan, wan, mgt, egress, ingress, brg
Permits to map device name to Ipanema logical interfaces.
[ipe10]$ getintf all

LAN is eth0
WAN is eth1
MGT is eth2
EGR is bond0
ING is bond1

BRG is brg0
[ipe10]$
4.16.7 Customize
Customize command permits to configure other feature than LanDownToWan and failsafe (for
ipengine 5v2, 120ax, 140ax, 1000ax and 1800ax), but must be reserved for Orange Business
Services experts support or Ipanema support only.
4.17 Ipengines cluster
When multiple ipengines control the same ip topology subnet(s), we call this set of ipengines a
cluster. Usually, we install cluster on central office where we need redundancy.
A good understanding of how works a cluster can help to understand and troubleshoot
ipengine behavior.
Each time, an ipengine detect that a flow is send to a remote cluster, it will try to resolve the
cluster, it means the probe will exchange packet with all ipengines of the cluster to know which
device in the cluster is seing the traffic. Once the cluster is resolved, then end-to-end
optimisation begins. Cluster resolution can take up to 2x40 sec (1min20sec) with 2 ipengines in
the cluster.
Typically, cluster must be first resolved before:
- SRE compress cluster traffic
- Xapp accelerate cluster traffic
4.17.1 Unresolved cluster
When a flow passing through an ipengine cluster begins, on the remote site, the command
“debug show cluster” displays the subnet is owned by several ipengines
140T:~# debug show cluster
[ CLUSTERING-FLOWS ]
*** SRC VRF1-NET2 (10.10.20.0/24) MINE
DST VRF1-NET6 (10.10.60.0/24) owned by <several> (C)
declared by VRF1-SITE6-B(10.10.60.242):28 VRF1-SITE6-A(10.10.60.241):68
RemoteResult= yes RefCount= 95
*** SRC VRF1-NET6 (10.10.60.0/24) owned by <several> (C)
declared by VRF1-SITE6-B(10.10.60.242):28 VRF1-SITE6-A(10.10.60.241):69
DST VRF1-NET2 (10.10.20.0/24) MINE
140T:~#
During resolution laps, some traffic can be classified like OutOfDomain
140T:~# debug show flow -d | more


< TCP, 10.10.60.95:35946 - 10.10.20.204:80, appli 11(HTTP):
Downstream_Cluster_NoBossRep, 4 pkts, age 0, uc 24, life 0
EngineUp/Down: Out of domain:0/VRF1-SITE2-A, priv 37(http), client, tcpFlags
SYN+ACK, iba 255.255.255.255, uc 24(UC-HTTP), flowId 6704, tkCount 0
< UDP, 10.10.60.77:10008 - 10.10.20.205:20054, appli 93(G711a):

Downstream_Cluster_NoBossRep, 201 pkts, age 0, uc 28, life 4
EngineUp/Down: Out of domain:0/VRF1-SITE2-A, priv 104(rtp/rtcp), client, iba
4.17.2 Resolved cluster
After a while, which can be up to 80 seconds. The ipengines cluster is resolved
140T:~# debug show cluster
[ CLUSTERING-FLOWS ]
*** SRC VRF1-NET2 (10.10.20.0/24) MINE
DST VRF1-NET6 (10.10.60.0/24) owned by VRF1-SITE6-A(10.10.60.241) (C)
*** SRC VRF1-NET6 (10.10.60.0/24) owned by VRF1-SITE6-A(10.10.60.241) (C)
DST VRF1-NET2 (10.10.20.0/24) MINE
140T:~#
Now traffic is correctly classified
140T:~# debug show flow -d | more

> UDP, 10.10.20.205:20080 - 10.10.60.127:10002, appli 93(G711a): Upstream_Cluster,
> UDP, 10.10.20.205:20072 - 10.10.60.57:10004, appli 93(G711a): Upstream_Cluster,


5 In which order processes are executed?

5.1 IP|TRUE only
LAN IP|TRUE WAN
5.2 IP|TRUE and IP|FAST
LAN IP|TRUE IP|FAST WAN
5.3 IP|FAST and IP|XCOMP
LAN IP|TRUE IP|FAST IP|XCOMP WAN
IP|XCOMP gives the possibility to IP|FAST, to burst at the WWH x Compression factor.
WWH = “what we have” = the available bandwidth detected for a direction.
5.4 IP|FAST and IP|XTCP
LAN IP|TRUE IP|XTCP IP|FAST WAN
5.5 IP|FAST and IP|XCOMP and IP|XTCP
LAN IP|TRUE IP|XTCP IP|FAST IP|XCOMP WAN
IP|XCOMP gives the possibility to IP|FAST, to burst at the WWH x Compression factor.
WWH = “what we have” = the available bandwidth detected for a direction.

6 Advanced troubleshooting
This section must be used only when necessary because we’ll use command that grant access
to the whole file system.
6.1 SHELL
After being logged on an ip|engine, several commands are available to configure and basically
troubleshoot ip|engine. To have access to more linux command, it is necessary to use the
hidden command “shell
shell”.
shell
The shell command gives access to a bash shell interpreter.
[ipe]$ shell
bash-2.01$
6.2 Which certificate is used by ip|engine ?
A script developed by Ipanema is stored in ip|engine filesystem to get the certificate name.
/tmp/cgi-bin/ipm_cgi_get_certificate_name
bash-2.01$ /tmp/cgi-bin/ipm_cgi_get_certificate_name
Content-type: text/plain
certificatenamebash-2.01$
Name of certificate used by ip|engine is printed just in front of the bash prompt. (in red on the
preceding screen-shot)

7 Unsupported commands by Orange Business

Services in Ipanema CLI
7.1 Netflowconfig
Not supported in standard Orange Business Services
7.2 Sslpassphrase
Not supported in standard Orange Business Services

8 Debug command
Debug command is not documented and can be modified without notifications from by
IPANEMA.
Debug command can be run in context mode (you can enter successively into sub-menu) or
directly from prompt.

8.1 Check ipboss configuration
Ipboss configuration is stored in ip|engine. It is possible to draw ipboss configuration with the
following commands.
8.1.1 “debug show ltc”
Display Local Traffic Limiting if configured in ipboss GUI.
debug show ltc
[ LOCAL TRAFFIC LIMITING ]

Ingress
Flags=0x00 default
Egress
Flags=0x00 default

8.2 “debug fast”
8.3 “debug xcomp”
8.3.1 “debug_more xcomp show state”
Display the state of the probe: compressor and/or decompressor side
[ipe]$ shell
bash-2.01$ debug_more xcomp show state
[CURRENT STATE]
Compress: yes
Decompress: yes
Internal status: XMG (ready/ok) XCO (ready/ok) DCO (ready/ok)
bash-2.01$
8.3.2 “debug_more xcomp show config”
xcomp global configuration : optim, default gateway, lan gateway, capacities xcomp, displays
current zibas
bash-2.01$ debug_more xcomp show config

[CURRENT CONFIGURATION]
Connect ipe with private: true
Xcomp on domain: true
Tunnel mode: 0
UDP tunnel ports: com=19987 dec=19988
TCP dictionary port: dec=19988
Local IP addr: 192.168.1.253
Local mask: 255.255.255.0
Local default gateway: 192.168.1.254
Local lan gateway: 0.0.0.0
IPE1 (10.42.84.3,192.168.1.253):
- Administrative state = 1
- Optimizing = true
- Xcomp = true Dcomp = true
- Current comZiba = 0
- Current decZiba = 0
IPE2 (10.113.142.39,192.168.2.253):
- Dcomp = true comZiba = none
- Xcomp = true decZiba = none
Hashed IP engines:
- 151: IPE2
- 210: IPE2
bash-2.01$
8.3.3 “debug_more xcomp show stats”
Displays statistics about utilization of diferents algorithm of compression (no/Dict/Zlib/Zlib+Dict)
bash-2.01$ debug_more xcomp show stats | more

[CURRENT STATISTICS]
Compression with IPE1:
- Running NAPs : (1,1)

- Backlog packets: 18199, 1082 bytes, 0.4%

- Processed packets: 4238367, 835 bytes
no = 83.1% 776-> 776 0 (01.00) dct = 12.0% 1055-> 341 714 (03.09)
zlb = 0.0% 0-> 0 0 (00.00) dct+zlb = 0.0% 0-> 0 0 (00.00)
oso = 4.9% 1287-> 796 491 (01.62) dct+oso = 0.0% 0-> 0 0 (00.00)
- Link ratio: 835->725 109 (01.15, -13.17%)
Compression with IPE2:
- Running NAPs : (1,1)
- Backlog packets: 76864, 805 bytes, 0.5%
no = 81.2% 644-> 644 0 (01.00) dct = 13.9% 1020-> 335 685 (03.04)
zlb = 0.0% 0-> 0 0 (00.00) dct+zlb = 0.0% 0-> 0 0 (00.00)
oso = 4.9% 1197-> 713 483 (01.68) dct+oso = 0.0% 0-> 0 0 (00.00)
- Link ratio: 723->604 118 (01.20, -16.43%)
Decompression with IPE1:
no = 89.0% 783-> 783 0 (01.00) dct = 9.1% 155-> 515 359 (03.30)
zlb = 0.0% 0-> 0 0 (00.00) dct+zlb = 0.0% 0-> 0 0 (00.00)
oso = 1.9% 446-> 884 438 (01.98) dct+oso = 0.0% 0-> 0 0 (00.00)
- Link ratio: 720->761 40 (01.06, -5.37%)
bash-2.01
8.4 Options of debug command
Sub-menu level1 Sub-menu level2 Sub-menu level 3 Description

Abc ABC Sub-menu relative to traffic classification
Show|dump|more Command to display information (more displays page
per page)
Device Device list (typically brg0 which stands for the bridge
inside ip|engine)
Filter Ip|engine applications dictionnary
Link List of the configured links
Protocol List of the built-in protocols
Iba Instantiated classification tree
Host List of xcomp eligible hosts
all Eq show device+filter+link+protocol+iba+host
Sleep <x seconds> Wait x seconds
Loop x... loop end Execute commands between loop and loop end X
times
Quit | exit Exit from the current sub-menu
Gps Displays debug information about GPS
Show|dump|more Command to display information (more displays page
per page)
Version [–r] [–d] Harware, software and package information
Config Configuration information
Global Global variables
Engine Known ip|engines inside domain
Subnet Topology and user subnets
Proto !!!
Tos TOS configuration
Appli Current application dictionary and port range
Crc (anymore Information about how ip|engine computes CRC

available since 4.3.0) ticket

Tkg Information about ticketing
Cor Information about correlation between ip|engines
Cls
Rtm Status of four possible concurrent realtime graphs
Top Information about top listeners and top talkers
Opt
Disc Status of discovery agent
Flow Current ip flows
Uflow Current micro-flows
Drv | Intf
Ltc Status of local traffic limiting
itp ITP servers and their status
Eadi
Wan WAN counters
Cpe CPE list configured on Ip|engine
Uc Configured UC
Cluster Current cluster flows
Monitor UC and flows ticketing
Pool
All Eq version+config+global+....+cluster+monitor
Fast
Show|more|dump
State Current state
Config All parsed and generated data
Engine List of the known engines
Icos Description of the ICOS
Qosp Description of the QosProfiles
Sentry List of the known sentries
Subnet List of the known subnets
Wan Description of the local WAN access
Ingltc Description of the Ingress LTC
Egrltc Description of the Egress LTC
ingress Description of the Ingress Iba-ICOS
Egress Description of the Egress Iba-ICOS
Sla Description of the local SLA
Tracking Bandwidth tracking variables
Ibadyn Dynamic IBA variables
Config All parsed and generated data
Monitor Monitor flows and classes
Ingltc Description of the Ingress LTC
Egrltc Description of the Egress LTC
Ingrcg Active Ingress RCG* (virtual engine Egress proxy)
Egrrcg Active Egress RCG* (virtual engine Egress proxy)
Rcg RCG configuration (accepted from ip boss)

Xcomp
Show|more|dump
State Show current state
Config Show current configuration
Sleep
Loop
Help
Exit
Quit
*RCG: Remote Coordinator Group

9 Draw_htb command
Draw_htb command is useful to display TC class based tree when ip|fast is running.
Draw_htb –d [ing | egr ] option
[ipe]$ shell
bash-2.01$ draw_htb -d ing -qrfA
parsed 46 lines, rejected 49 lines
1:0
+-- 1:1 1e+06Kbit - 1e+06Kbit (u32: match 00000000/00000000 at 16)
| +-- 64:0
| +-- 64:1 1e+06Kbit - 1e+06Kbit (iba: remote 10.0.16.105 rnap any matched 519 )
| +-- 64:40 (iba: remote 240.0.0.0 rnap 1 matched 0 )
| +-- 64:44 155000Kbit - 155000Kbit (icos: appli FTP matched 0 )
| +-- 64:45
| | +-- 64:46 120Kbit - 155000Kbit (icos: appli (G711a|G711u) matched 0 )
| | +-- 64:47 30Kbit - 155000Kbit (icos: appli G729 matched 0 )
| | +-- 64:48 300Kbit - 155000Kbit (icos: appli (RTP/RTCP|RTSP) matched 0 )
| +-- 64:41
| | +-- 64:42 1000Kbit - 155000Kbit (icos: appli SMTP matched 0 )
| | +-- 64:43 600Kbit - 155000Kbit (icos: default matched 0 )
| +-- 64:49 155000Kbit - 155000Kbit (icos: appli HTTP matched 0 )
+-- 1:2 1000Mbit - 1000Mbit (u32: match 0a001069/ffffffff at 12)
bash-2.01$
IBA = direction Criticalities

UCs
The subtree IBA is repeated each time a new direction is mounted.

10 FAQ (extract from ipanema support site)

10.1 Why I have uncorrelated traffic ?
When an ip|engine sends a packet, it waits for a ticket record. This ticket record is sent by the
destination ip|engine. It contains the information to calculate the delay, jitter and loss.
Where do I see uncorrelated traffic?
The volume of uncorrelated traffic is availble in the SA reports : SA - site throughput (for a
physical site), SA - site summary ingress (for the domain).
What is the consequence of uncorrelated traffic?
The traffic correlation allows the ip|engine to calculate the quality index of a flow. The correlation
is the process that is used to calculate delay, jitter and losses. The volume is always measured
whether the traffic is correlated or not. Correlation has no impact on optimisation.
Why do I have uncorrelated traffic?
There are several reason. It can be either that :
the ip|engines are not synchronised

The ticket records are stopped by a firewall (click here for more details )
A topology subnet is not associated to the right ip|engine. Let's see what happens in that
case:
When a topology subnet is not associated with the right ip|engine, you may sometimes
see a few flows going to specific directions that are not correlated. The solution is of
course to set the correct topology subnets. Now, if you do not know to topology, how
can you find the correct destination ip|engine?
First, we must understand why a wrong topology subnet assocation ends up in no
correlation. after a packet is sent, the source ip|engine waits for a correlation record from
the destination ip|engine. If the source ip|engine receives the correlation record from
another ip|engine, it will be discarded. So the situation is basically this one:

Here is the procedure to track the not correlated flow:
In the real-time window, filter the flows which accuracy is low. these flows correspond to
non correlated traffic.
Which topology subnet do they match? Do you see the flows when you run a discovery on
the destination ip|engine?
If you do not see the flow, the topology subnet association is wrong. If you want to current
the real destination site, you must look for sites with transit flow.
Run a discovery on a site with transit traffic and tick the check box "out of local config". If
you find your flow, you have found the real site where the flows is going to.
10.2 High Cpu usage in an ip|engine
Report "fi - availability overview" shows the CPU usage of the ip|engines.
A high CPU usage can be explained by many reasons. Note that compression needs an
important CPU share, specially when the ip|engine works close to the limits for its bandwidth
and/or number of tunnels. Locally rerouted traffic can impact ip|engine performances, so we
recommend checking in SA-SiteThroughput that this kind of traffic does not reach the 10% of
bandwidth usage. In this same report we can see the totallity of flows that cross an ip|engine,
however in the rest of reports we will see only REPORTED flows.
10.3 Impact of missing subnet in ip|boss configuration
The fact of not declaring a subnet in ip|bossTopology Subnets will make the flows from/towards
that subnet to be seen as OutOfDomain.
An example will help explaining the impact of this issue:
LAN_A is physically added to the network but has not been declared in TOPOLOGY SUBNETS.
Traffic flows are sent in both ways between ipeA and ipeB
10.3.1 Flow identification
Flow A -> B
- ipeA will identify this flow as coming from OOD and going to LAN_B, then the flow will
be seen as TRANSIT and not reported.
IpeB will identify this flow as coming from OOD and going to its LAN_B, then the flow
will be reported
Flow B -> A
ipeB will identify this flow as coming from LAN_B and going to OOD, then the flow will
be reported
- ipeA will identify this flow as coming from LAN_B and going to OOD, then the flow will
be seen as TRANSIT and not reported
10.3.2 Optimization
Flow A-> B

- ipeA will optimise this flow LOCALLY, which means that it will only check congestion in
its own WAN access. If there is congestion in site_B, ipeA will not optimise the traffic to
avoid it and sessions will be limited to their MAX instead of their OBJ
-ipeB will class this flow in its Egress_IBA OutOfDomain and will perform the optimisation
LOCALLY.
Flow B-> A
ipeB will class this flow in its Egress_IBA OutOfDomain and will perform the optimisation
LOCALLY.
ipeA will optimise this flow LOCALLY
10.3.3 Coloring
Flow A-> B
ipeA will not apply coloring in the traffic
Flow B-> A
ipeB will apply coloring
10.3.4 Compression
Compression will not be enabled, as ipeA sees the flows as TRANSIT and ipeB sees the
flows as going to OOD.
10.3.5 Summary
REPORTING COLORING:
- ipeA: no reporting Only applied in the flow B-
- ipeB: report as OOD >A
OPTIMISATION: COMPRESSION
- locally performed in each ipe Not performed
10.4 Myself not found in config
The user notices one of the ip|engines in a 'down:not configured' state, and finds the message
'myself not found in config' in ip|boss logs.
This message reveals a problem in the IP configuration of the ip|engine. The user should check
whether the IP configuration in ip|boss and in the ip|engine are the same.
In most of the cases, the problem is related to a wrong configuration of MGT interface. If the
MGT is not intended to be used disable it using the command 'ipconfig mgt none'.
10.5 How identify traffic encapsulated in GRE tunnel?
All flows crossing an ip|engine in a GRE tunnel will be reported as GRE traffic, and will be
treated according to ip parameters of the tunnel.
See in the following drawing an example. Note that the ip|engine will report this flow with the
following data:

source: ip_R1, destination: ip_R2, application GRE
However the user could configure the ip|engine to ignore IP and GRE headers and treat the
https header. In that case the flow will be reported as:
source: ip_client, destination: ip_server, application HTTPS
To enable this functionality in your ip|engine use the command 'customize +GRE'. Afterwards,
reboot the ip|engine or launch the script 'restart iptrue'. Note this functionality is only available
for versions newer than 4.3.4.2
10.6 Flows wrongly tagged as out of domain
When performing a DISCOVERY and filtering to see just Out Of Domain flows, the user could
notice some flows which are sent to an existing Topology Subnet.
During the time the source ip|engine looks for the destination ip|engine in its configuration, the
packets will be classified as Out Of Domain. This can take around 5 seconds, and the impact is
just some packets being incorrectly classified.
In case of a Cluster architecture, it takes one additional polling period to perform cluster
resolution. If after that time the flow is still tagged as OutOfDomain we recommend to perform
the following tests:
Can the 2 ip|engines communicate together? Use command: 'debug dump engine'
What is the state of the cluster for this flow? Use commands: 'debug dump flow -d' and
'debug dump cluster'
Check that source/destination Topology Subnets are correctly defined
10.7 What is unknown traffic in Quality Evolution SLM Report?
In SLM-Site Synthesis and SLM-Application Synthesis graph Quality Evolution shows quality
calculated for all User Classes measured.
The quality is classified in three levels: good, average and bad. However a fourth value is
available in this graph: unknown.
This tag identifies all flows which could not be measured for different reasons. Remind that
metrics used to calculate quality are:

- Delay/Jitter/Losses/SRT/RTT/TCPretransmission for TCP flows in versions 4.2 and newer

- Delay/Jitter/Losses for non-TCP flows in versions 4.2 and newer.
- Delay/Jitter/Losses for all flows in version 4.0.
Then, if a TCP flow is not correlated, ip|engines can still measure SRT/RTT/TCPretransmission.
However, if a non-TCP flow (e.g. UDP) is not correlated it will be marked as 'unknown' in our
Quality Evolution graph.
10.8 Ip|engine synchronization problem
In order to provide quality metrics on a flow (delay, jitter,loss information), both source and
destination ip|engines must have the same clock. To achieve this we use Network Time
Protocol to synchronize to a time server (which can be an ip|engine synchronized by a router).
For a domain, we can have up to 8 NTP servers which respond to the time request of the other
ip|engines. The client ip|engine will select the best source for it: with less jitter.
If the ip|sync server is not anymore synchronized, it will not answer to time requests from the
other ip|engines. For the server and clients, check the Offset column in ip|engine status
window. The value should be lower than 10ms to be considered as synchronized.
If an ip|sync client has still GPS as source, that means either the quality of network is too low for
the ITP traffic either the ip|engine can’t dialog on the port UDP/123.


IPanema Troubleshooting Guide V5

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

IPanema Troubleshooting Guide V5

Uploaded by

Copyright:

Available Formats

IPANEMA

© copyright, Equant 2006

© Copyright Equant 10th of may 2011

© Copyright Equant 10th of may 2011

10.1 WHY I HAVE UNCORRELATED TRAFFIC ?.............................................................................................. 105

© Copyright Equant 10th of may 2011

1.2 Intented Audience

© Copyright Equant 10th of may 2011

2 Quality Of Service Troubleshooting

Are the flows Start DISCOVERY

YES LAN Problem?

Check Load of Check QOS

© Copyright Equant 10th of may 2011

2.2 Site troubleshooting

Site's SA- Site-PM-UC

Are there Are criticals

Check WAN rate

© Copyright Equant 10th of may 2011

2.3 QOS Profile troubleshooting

© Copyright Equant 10th of may 2011

YES YES YES

Check LAN &

© Copyright Equant 10th of may 2011

Too much Improve the user

Null Check IP address

© Copyright Equant 10th of may 2011

Most important files are:

__active__.ipmconf configuration file for each domain

3.2 TCP/IP communications to or from IP|Engines

© Copyright Equant 10th of may 2011

From To Default ports Description Configurable

© Copyright Equant 10th of may 2011

4.1 Interface mapping

4.1.1 Single instance software

It applies to single instance software earlier than 5.0.

Egress HTB Ingress HTB

Bond X BRG0 Bond Y

4.1.2 Multi-instances software

It applies to multi-instances software earlier than 5.0.mi

© Copyright Equant 10th of may 2011

Egress HTB Ingress HTB

Lap 0 Lap N Wap 0 Wap N

Line shaper 0 Line shaper N

Eth interface are directly connected to hardware.

Lap and Wap:

4.2 Process chain inside ipengines

© Copyright Equant 10th of may 2011

4.2.1 From LAN to WAN interfaces

Tag Appli Tag Iba Tag UC

4.2.2 From WAN to LAN interfaces

© Copyright Equant 10th of may 2011

Tag Appli Tag Uc Tag Appli

LAN Layer 7 FAST XTCP UC Topo WAN

© Copyright Equant PROXY 10th of may 2011

4.3 Physical Layer connectivity problem

brcount [ -d { lan | wan | all } ]

To request a detailled status, use the following command:

[ipe10]$ brcount -d lan

ifconfig [-a {interface} ]

© Copyright Equant 10th of may 2011

eth0 Link encap:Ethernet HWaddr 00:E0:4B:0D:50:BF

lo Link encap:Local Loopback

bond1 Link encap:Ethernet HWaddr 00:E0:81:44:EA:DB

active.ipmconf configuration file for each domain