MDS Slow Drain Device Training: © 2009 Cisco Systems, Inc. All Rights Reserved. Cisco Confidential

MDS Slow Drain
Device Training
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 1

What’s the Issue?
Today almost all of data traffic between end devices is carried by FC Class 3
service that uses link level, per hop based, buffer to buffers flow control.
There is no End-to-End flow control with Class 3 services and in presence of
slow devices, that are not accepting frames at the originating device rate, the
FC networks are prone to ISL credit starvation and may choke the links.
Frames destined to slow devices have to be carefully isolated in separate
queues and switched over to egress queues without congesting the backplane.
Eventually decision has to be made if the frames are considered stuck and
when to drop them.

Fibre Channel Class of Service 3
Class 3 services is defined as connectionless service based on frame

switching. There is no acknowledgment of a frame delivery or rejection.
Frames could be held by the Fabric or by the end devices without any
notification.
Upper layer protocols must guarantee the transmission and handle the lost
frames efficiently.
It uses buffer-to-buffer flow control system that works only on per hop basis.

Buffer to Buffer Flow Control
B2B flow control is a link-level flow control system that manages the rate of frame transmission
between two FC ports on a link.
Upon receiving (1) a B2B credit signal from the FC switch the server will transmit (2) a frame to the
switch.
The switch will hold the frame in its buffer (3) until it receives (4) a B2B credit signal from the
destination device. After a credit is received, switch then transmits (5) the frame to its destination
and will release (6/1) a B2B credit to the server so it can send another frame.
Devices that do not release the B2B credits in timely fashion are termed as slow drain devices.
FC Switch
1
Server Receive 4 Disk
B2B Credit Receive
6
B2B Credit
Transmit
Transmit Buffers
Frame
Buffers 3
2 Frame
5

ISL Blockage
B2B credits are on per hop basis and in a multi hop networks this limitation may create a problem.
Consider the example below where yellow disk is not releasing (1) a B2B credit while blue disk is
behaving fine.
The frames going from the yellow server to the yellow disk will be held (2) in the buffer at “FC switch
2” and eventually it will stop generating the B2B credit (3) signal to the adjacent “FC switch 1”,
attached via ISL.
This would cause blockage for all flows (yellow & blue in this case) going over that ISL. In other
words ISL would be blocked until the frames are released from the buffers and B2B credit is
released.
FC Switch 1 FC Switch 2
ISL
Slow
NO
X Drain
B2B Credit
Buffers
Full X Device
NO
3 2 B2B Credit
1
MDS Solution to ISL
Blockage Problem

What’s the Solution?
The ISL blockage problem requires three steps solution.

First, all the frames related to slow drain devices should be held in separate
queues.
Second, frames destined to slow drain devices should not be switched over
the backplane as this would cause backplane congestion.
Third, if a stuck frame is held in the queue for more than “Drop_Timer” value
then it should be dropped.
The first two steps are done automatically in MDS architecture and the 3rd step
would require some manual fine tuning of the “Drop_Timer” value.

Step1: Isolate Slow Drain related flows
Cisco MDS architecture implements a Virtual Output Queuing (VoQ) architecture. Upon receiving a
frame the source index (SI), destination index (DI) and the QoS flag is checked and the frame is
placed in a unique queue.
To accommodate all possible permutations MDS VoQ architecture can dynamically provide up to
96,000 queues. (24src X 1024dst X 4QoS = 98,304 queues).
On average it maintains about 4,096 queues. Frames related to slow drain device are held in
separate ingress queues (S2-D1 & S3-D3) at each switch. This step is done automatically in MDS by
VoQ technology.
96,000 VoQ FC Switch 1 96,000 VoQ FC Switch 2
S1-D1
ISL
S1 D1 S3 S3-D2
D2
Slow
Drain
D3
S2 Buffers Buffers X Device
S3-D3
S2-D1
NO
B2B Credit

Step2: Avoid Backplane Congestion
Once the frame arrives in the VoQ it generates a request signal “R” to the central arbiter asking for
its permission to transmit that frame over the backplane to its destination.
If a buffer space is available in the destination queue a token signal “T” is generated to the arbiter
indicating that it can accept a frame. At this point arbiter sends the grant signal “G” to the VoQ
indicating it to transmit the frame to its destination.
This three way hand shake is done at line rate for every frame that is switched within MDS
architecture. This ensures that the backplane is not congested by stuck frames taking up buffers and
blocking other flows. Again this step is done automatically by MDS central arbiter.
G
G
T
ISL
R T
Slow
R Drain
Buffers Buffers X Device
NO
Frames Holdup Frames Holdup B2B Credit

Step3: Drop the Stuck Frames
If the slow drain device continues to hold the release of B2B credit the switch cannot hold frames
indefinitely.
Currently within MDS if a frame is held for more than 500ms it is flagged as stuck frame and is
dropped automatically.
This releases the buffer space and allows the switch to generate a B2B credit signal on the blocked
ISL, clearing the ISL blockage. This drop timer is called “Drop_Timer” and it is configurable by the
end user.
G
G
T
ISL
R T
Slow
R Drain
B2B Credit
Buffers Buffers X Device
NO
Drop Frames
B2B Credit
after “Twait” expires

Step3a: Identify the blocked port
Check if any ports have timeout drops:
 switch# show hardware internal packet-flow dropped
 module-x# show hardware internal packet-flow dropped
 module-x# show hardware internal fc-mac port <y> statistics

The above CLI provides the timestamp dropped frame count and also provides the excessive TX
credit transition to zero.
Check if any ports are experiencing credit loss and recovery

 module-x# show process creditmon statistics
The above CLI provides the credit loss/recovery count.
 module-x# show process creditmon event-history

The above CLI provides the credit loss/recovery timestamp.

Step3a: Identify the blocked port continued
Check Request Timeouts
 module-x# show hardware internal up-xbar <0|1> asic-info
This CLI captures the DI/GI (Destination Index or Global Index) to which the requests are timing out.

Check Queued Frames
 module-x# show hardware internal up-xbar <0|1> queued-packet-info
This CLI captures the DI/GI (Destination Index or Global Index) to which the frames are destined to if
they are queued for an extended time.
GI/DI to Interface Conversion:

 switch# show system internal fcfwd idxmap port-to-interface
This CLI helps to find the interface associated with the DI/GI (Destination Index or Global Index). DI
and GI refer to the same index. FCIP links will not have the timestamp drops; take a look at the
queued frame count to find out if the FCIP link is congested.
If the timestamp drop or the credit recovery is seen on FC ISLs, follow that link to the next switch
and repeat the steps mentioned. These steps have to be followed till an end device or the end ISL
that is congested is found.

Step3b: Reconfigure the timeout value
 Timeout values are configured on per port basis. The blocked port is identified by Step 3a. You
may have to try different values based on your network characteristics.
 Gen-2:
 module-x# debug fc-mac port <y> reg-wr 0x566 <to-val-1>
 module-x# debug fc-mac port <y> reg-wr 0x5a6 <to-val-2>
 Gen-3:
 module-x# debug fc-mac port <y> reg-wr 0x3cb <to-val-1>
 module-x# debug fc-mac port <y> reg-wr 0x3cc <to-val-1>
 Fabric Switches: 9222i:

 Same as Gen-2 above
 9124 and 9134:


New Features Available
in 4.2(7)

Slow Drain Device
Credit issues at this Host

“Slow Drain Device”

Detection:
Average Credit Non-Available Duration
Sys Log/Trap
(Switch/Interface)
show system internal snmp credit-not-available

show port-monitor
show logging logfile
 0 sec  credits
 10% (100
ms)
e.g. 10%
Enabled by default, threshold: 10% in 1s, action: syslog and trap

 1 sec

Detection:
Average Credit Non-Available Duration Configuration
Switch Level, per Port Mode (F/E)/ All Ports

Threshold: 10% - 100% , 10% increment, default: 10%
Interval: 1 s – 1 hour, default: 1 s
Events and actions include
Threshold configuration will
1. Syslog (default) be in % of credit non-available
2. Trap (default) duration in a specified
interval(s), default 10% (100
3. Error Disable or Flap the port
ms) in 1 second.
switch(config)# port-monitor name Cisco
switch(config-port-monitor)# counter tx-credit-not-available poll-
interval 1 delta rising-threshold 10 event 4 falling-threshold 0
event 4 portguard errordisable

Level 0 – No Recovery Attempt “Error Disable”
Sys Log, Trap

(Switch/Interface)

show port-monitor
Show interface fcx/y
X
 0 sec  credits
 10% (100ms)
e.g.10%
Action (Error Disable) and Threshold depends on topology and customer

requirements and are customer driven configs. Error Disabled port(s) can
be recovered by shut/no shut.  1 sec

Level 1 - Recovery “Congestion Drop Timeout”
Switch Level, per Port Mode
switch(config)# system timeout congestion-drop <value> mode (F) / (E)
switch(config)# system timeout congestion-drop default mode (F) / (E)
show logging onboard timeout-drops

Show logging onboard request-timeout
show process creditmon credit-loss-events
Frees up credits
e.g. drop timeout = 200 ms

 0 sec ---
 Check Timestamp
Timeout Value (100 ms - 1000 ms),  200 ms  of each frame
10 ms increment, default value – 500 ms
 Drop the frame

 from the Queue
Congestion Drop Timeout is enabled by default.  credit

Timeout Value change is dependent on topology and customer
requirements.

Level 2 – Recovery “No-Credit Preemptive drops”
Switch Level, F mode only
switch(config)# system timeout no-credit-drop <value> mode F
switch(config)# system timeout no-credit-drop default mode F
show logging onboard timeout-drops

Show logging onboard request-timeout
Frees up credits
 0 sec ---
e.g. No-Credit Timeout = 300 ms
 300 ms  --  NO credit
(100 ms – 1000 ms), 100 ms increments  ------
 ------
 ----- 
 Frames
Start dropping
 -----
 the frames
 -----
* No-Credit Preemptive Drops is disabled by default.  at the line rate till

 Credits are received. ------
Timeout Value and Enable Decision is based on topology and  ------
 --------
customer requirements.  credit
 --------
 --------

Level 3 – Recovery “Credit Loss”  Link Reset
 Automatic – Enabled by Default
 1 sec for F Ports, 1.5 sec for E Ports
(Sys Log/Trap)
 These timers are not configurable (Switch/Interface)

show port-monitor
 credits
 0 sec --
 LR
 shut/no shut  No Credits (Dead)
 1 sec --  LR
 No Response
 shut
The interface should run out of credits for 1 s (F) or 1.5 s (E) continuously
 no shut
for LR to kick-in.

Level 4 - Recovery “Credit Loss”  Error Disable
(Sys Log/Trap)
(Switch/Interface)

show port-monitor
X  credits
* Error Disable  0 sec --
 No Credits (Dead)
 1 sec --  LR
Error Disable X  No Response
anytime after 1 sec  Shut

 no shut
Error Disable action config is determined by the topology
and customer requirements. Error Disabled port(s) can be
recovered by shut/no shut.

Level 4 - Recovery “Credit Loss”  Error Disable
Action Configuration
Switch Level, per Port Mode (F/E)/ All Ports

Events and actions include
1. Syslog (default)
2. Trap (default)
3. Error Disable or Flap the port
switch(config)# port-monitor name Cisco
switch(config-port-monitor)# counter credit-loss-reco poll-interval 60
delta rising-threshold 4 event 4 falling-threshold 1 event 4 portguard
errordisable

Debug Commands
switch# show logging onboard request-timeout [module <module number>]
This command will display the source interface(s) and destination interface pair for which the
requests are getting timed-out (requests take > 90 ms to be granted). Destination Interface displayed
in this command output is the slow interface.
switch# show logging onboard request-timeout

----------------------------
Module: 2
----------------------------
--------------------------------------------------------------------------------
| Dest | Source |Events| Timestamp | Timestamp |
| Intf | Intf | Count| Latest | Earliest |
--------------------------------------------------------------------------------
|fc2/1 |fc2/19, | 77 |Mon May 31 13:12:22 2010|Mon May 31 13:12:39 2010 |
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
switch#

Debug Commands (contd…)
switch# show process creditmon credit-loss-events [module <module number>]
This command will display the interface on which the credit loss events are seen with the total number
of credit loss events and upto 10 latest timestamp of credit loss events for each interfaces.
switch# show process creditmon credit-loss-events module 2

Module: 02 Credit Loss Events: YES
-----------------------------------------------------------------
| Interface | Total | Timestamp |
| | Events | |
-----------------------------------------------------------------
| fc2/1 | 444748 | 1. Tue Jun 8 20:22:02 2010 |
| | | 2. Tue Jun 8 20:22:01 2010 |
| | | 3. Tue Jun 8 20:22:00 2010 |
| | | 4. Tue Jun 8 20:21:59 2010 |
----------------------------------------------------------------
| fc2/19 |2 | 1. Wed Jun 9 10:15:49 2010 |
| | | 2. Wed Jun 9 09:07:45 2010 |
-----------------------------------------------------------------
switch#

switch# show process creditmon credit-loss-event-history [module <module number> ]
This command will display the credit loss event-history for all the interfaces in a specified module.
switch# show process creditmon credit-loss-event-history module 2

1) Event:CREDITMON_EVENT_CREDIT_LOSS, length:4, at 854490 usecs after Wed Jun 9
10:15:49 2010
interface = fc2/19
2) Event:CREDITMON_EVENT_CREDIT_LOSS, length:4, at 205391 usecs after Wed Jun 9
09:07:45 2010
interface = fc2/19
3) Event:CREDITMON_EVENT_CREDIT_LOSS, length:4, at 730422 usecs after Tue Jun 8
20:22:02 2010
interface = fc2/1
4) Event:CREDITMON_EVENT_CREDIT_LOSS, length:4, at 630446 usecs after Tue Jun 8
20:22:01 2010
interface = fc2/1
switch#

switch# show logging onboard credit-loss [last mm minutes] [last hh hours] [last dd days]
[module <module number>]
This command will display the total credit loss events per interface and the timestamp in the
latest specified duration.
switch# show logging onboard credit-loss

----------------------------
Module: 2
----------------------------
-----------------------------------------------------------------------------------------------------
ERROR STATISTICS INFORMATION FOR DEVICE ID 118 DEVICE Aakash
------------------------------------------------------------------------------------------------------
Interface | | | Time Stamp
Range | Error Stat Counter Name | Count | MM/DD/YY HH:MM:SS
| | |
----------------------------------------------------------------------------------------- ------------
fc2/1 | AK_FCP_CNTR_CREDIT_LOSS | 647 | 05/25/10 23:53:34
fc2/7 | AK_FCP_CNTR_CREDIT_LOSS | 400 | 05/25/10 23:50:34
------------------------------------------------------------------------------------------------------
switch#

switch# show logging onboard timeout-drops [last mm minutes] [last hh hours] [last dd days]
[module <module number>]
This command will display the total timeout drops per interface and the timestamp in the
latest specified duration.
switch# show logging onboard timeout-drops

----------------------------
Module: 2
----------------------------
--------------------------------------------------------------------------------------------------------------------
ERROR STATISTICS INFORMATION FOR DEVICE ID 118 DEVICE Aakash
---------------------------------------------------------------------------------------------------------------------
Interface | | | Time Stamp
Range | Error Stat Counter Name | Count | MM/DD/YY HH:MM:SS
| | |
--------------------------------------------------------------------------------------------------------- ------------
fc2/1 | AK_FCP_CNTR_LAF_TOTAL_TIMEOUT_FR | 647 | 05/25/10 23:53:34
fc2/2 | AK_FCP_CNTR_LAF_TOTAL_TIMEOUT_FR | 200 | 05/25/10 20:53:34
-----------------------------------------------------------------------------------------------------------------------
switch#

switch# show system internal snmp credit-not-available [module <module number>]
This command will display the rising and falling thresholds, timestamp and the interval/% duration
of time credit not available per interface.
switch# show system internal snmp credit-not-available module 2
Module: 2 Number of events logged: 5

---------------------------------------------------------------------------------------------------------------------------------------
Port Threshold Rising Interval(s) Event Time Type Duration of time not available
--------------------------------------------------------------------------------------------------------------------------------------
fc2/1 20/10(%) 1 Tue Jun 1 16:27:24 2010 Falling 0%
fc2/13 20/10(%) 1 Tue Jun 1 16:27:43 2010 Rising 100%
switch#

Example command
output

CLI Commands
 How to check ISLs for lack of transmit credits.In the below example, fc2/1
has no tx credits available and fc3/1 has available tx credits, but it may be
below its full complement. This may indicate congestion in the fabric:
switch# sh int | i "fc|Belong|low pri" | ex "descript" | ex "Peer" | i "trunking" n 2
fc2/1 is trunking Belongs to portchannel 20
0 low priority transmit B2B credit remaining
fc3/1 is trunking Belongs to portchannel 20
185 low priority transmit B2B credit remaining
 How to check for issues queuing frames to a port on the ingress port
switch# attach module < modulenumber>
modulex# show hardware internal upxbar 0 queuedpacketinfo
...
++++++
| GI (Hex) | Prio 0 | Prio 1 | Prio 2 | Prio 3 |
++++++
| c | 000000 | 000000 | 000000 | 001005 |
| 1c | 000000 | 000000 | 000000 | 00200a |
^ ^
| |
egress port (slow) ingress port

CLI Commands (continued)
 How to quickly check for transmit frame drops

switch# show hardware internal packetflow drop
Module: 03 Dropped Packets: YES
++++
| DEVICE NAME | PORTS | DROPPED COUNT |
| | | RX (Hex) | TX (Hex) |
++ |+
| Aakash | 3 | 0 | 1c |
| Aakash | 4 | 0 | 10 |
^ ^ ^ |
| | |
MAC device name egress port count

 How to check frame drop reason on egress port

switch# show hardware internal packetdroppedreason
Module: 02 Dropped Packets: YES
+++++
| | | DROPS | |
| PORTS | DEVICE NAME || COUNTER NAME |
| | | Rx(Hex) | Tx(Hex) | |
+ +++++
| 2 |Stratosphere | | 849 |FCP_CNTR_LAF_C3_TIMEOUT_FRAMES_DISCARD
| ^ ^ ^
| | |
egress port index (slow) count timeout discard reason

 How to check frame transmit timeout history on egress port

switch# show logging onboard errorstats

Module: 1

ERROR STATISTICS INFORMATION FOR DEVICE ID 63 DEVICE Stratosphere

| | Time Stamp |In|Port
Error Stat Counter Name | Count |MM/DD/YY HH:MM:SS |st|Rang |
| | Id|e

FCP_CNTR_LAF_C3_TIMEOUT_FRAMES_DISCARD|ff |02/17/10 16:26:03 |00|1
FCP_CNTR_LAF_C3_TIMEOUT_FRAMES_DISCARD|2fd |02/17/10 16:26:13 |00|1
FCP_CNTR_LAF_C3_TIMEOUT_FRAMES_DISCARD|4fb |02/17/10 16:26:23 |00|1
FCP_CNTR_LAF_C3_TIMEOUT_FRAMES_DISCARD|6f9 |02/17/10 16:26:33 |00|1
FCP_CNTR_LAF_C3_TIMEOUT_FRAMES_DISCARD|8f7 |02/17/10 16:26:43 |00|1
FCP_CNTR_LAF_C3_TIMEOUT_FRAMES_DISCARD|af5 |02/17/10 16:26:53 |00|1
FCP_CNTR_LAF_C3_TIMEOUT_FRAMES_DISCARD|10ef |02/17/10 16:27:23 |00|1

 How to check frame transmit timeout history on egress port -

continued
switch# attach module < modulenumber>
modulex# show hardware internal upxbar 0 asicinfo
...
DI priority qlen time

21 (0x015) 0 1 812028 usecs on Mon Oct 19 22:06:05 2010
239 (0x0ef) 3 1 935355 usecs on Mon Oct 19 22:05:44 2010
18 (0x012) 3 1 972427 usecs on Mon Oct 19 22:05:42 2010
34 (0x022) 3 1 931536 usecs on Mon Oct 19 22:05:42 2010
^ ^
| |
egress port index (slow) timestamp of timeout

 How to check for credit loss on an egress port

switch# attach module < module_number>
modulex# show process creditmon statistics
***** Credit Monitor Info *****
Port Mode Monitor VL Cr Thresh Cr Loss
Exceeded Event

1 E yes NA 0 7
^ ^
| |
egress port count
switch# attach module < module_number>
modulex# show process creditmon eventhistory | include CREDIT_LOSS next 1
Event:CREDITMON_EVENT_CREDIT_LOSS, length:1, at 818948 usecs after Sat Nov 21 06:33:33 2010 port = 18


 Please refer to the following extensive “Slow Drain

Receiver Troubleshooting” document prepared by Tim
Greenland for a detailed explanation of troubleshooting
slow drain related problems.
 The previous command outputs were all taken from this
document.
 http://bock-bock/wiki/User:Smarkovi:Troubleshooting:Identify_Slow_Draining_Device


MDS Slow Drain Device Training: © 2009 Cisco Systems, Inc. All Rights Reserved. Cisco Confidential

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MDS Slow Drain Device Training: © 2009 Cisco Systems, Inc. All Rights Reserved. Cisco Confidential

Uploaded by

Copyright:

Available Formats

MDS Slow Drain

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 1

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 2

Class 3 services is defined as connectionless service based on frame

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 3

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 4

FC Switch 1 FC Switch 2

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 6

The ISL blockage problem requires three steps solution.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 7

96,000 VoQ FC Switch 1 96,000 VoQ FC Switch 2

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 8

FC Switch 1 FC Switch 2

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 9

FC Switch 1 FC Switch 2

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 10

 module-x# show hardware internal packet-flow dropped

 module-x# show hardware internal fc-mac port <y> statistics

Check if any ports are experiencing credit loss and recovery

 module-x# show process creditmon event-history

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 11

GI/DI to Interface Conversion:

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 12

 module-x# debug fc-mac port <y> reg-wr 0x5a6 <to-val-2>

 module-x# debug fc-mac port <y> reg-wr 0x3cc <to-val-1>

 module-x# debug fc-mac port <y> reg-wr 0x446 <to-val-2>

 Fabric Switches: 9222i:

 9124 and 9134:

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 13

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 14

Credit issues at this Host

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 15

show system internal snmp credit-not-available

Enabled by default, threshold: 10% in 1s, action: syslog and trap

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 16

Switch Level, per Port Mode (F/E)/ All Ports

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 17

Sys Log, Trap

show system internal snmp credit-not-available

Action (Error Disable) and Threshold depends on topology and customer

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 18

show logging onboard timeout-drops

e.g. drop timeout = 200 ms

 Drop the frame

Congestion Drop Timeout is enabled by default.  credit

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 19

show logging onboard timeout-drops

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 20

show process creditmon credit-loss-events

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 21

show process creditmon credit-loss-events

Error Disable X  No Response

anytime after 1 sec  Shut

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 22

Switch Level, per Port Mode (F/E)/ All Ports

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 23

switch# show logging onboard request-timeout

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 24

switch# show process creditmon credit-loss-events module 2

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 25

switch# show process creditmon credit-loss-event-history module 2

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 26

switch# show logging onboard credit-loss