Basic KPI Troubleshooting

Basic Key Performance Indicators

• RRC Success Rate - This is to evaluate Accessibility of the network.

“ConnectionEstablishmentSuccess/ConnectionEstablishmentAttempt”

• Connection Drop Rate (CDR) – This is evaluate disconnections/drops for all kind of services (QCI1 to QCI9)
“E-RAB Release whilst Active/(Initial ERAB Setup Success+ Additional ERAB Setup Success)”

• Voice over LTE Drop Rate (VDR) - This is to evaluate call drops during Volte call.
“E-RAB Release whilst Active*/(Initial ERAB Setup Success*+ Additional ERAB Setup Success*)”
*Use values from only QCI1

• Handover Success Rate (HOSR) – This is to evaluate mobility performance of the network.
“Inter eNB handover Success/Inter eNB handover Attempt”

• Throughput DL/UL - Cell level Data speed in DL and UL

“DL Throughput_HSI =(('DlMacTrafficBytes.9'[eCell,Time]*((8.0*0.743)*1000.0))/(('DlActTti.9'[eCell,Time]*1024.0))”
“UL Throughput_HSI = ((('UlMacTrafficBytes.9'[eCell,Time]*(8.0*0.2*1000))/(('UlActTti.9'[eCell,Time])*1024.0)”

Copyright 2016 AIRSPAN NETWORKS INC. 2

RRC Success Rate Troubleshooting
1. PCI & RSI Planning Validation- Refer Basic Parameter Planning Guideline *
1. *If Auto PCI and Auto RSI is enabled (Check SON Profile from Netspan) then ignore this step.

2. Clock Issue: Refer “Alarm Document” for clock related issues.

1. If No clock alarms but RRC Success Rate < 50% then swap L2 Switch & monitor the performance
2. If multiple cells are having poor RRC Success Rate then try CSS reboot.

3. Congestion : If “Max. Connected Users” > 30 then

1. Check if neighbouring cells are OOS - resolve it and monitor the performance.
2. Check # connected users at NBR cells, if possible increase the power of NBR cell & monitor the performance.
3. If SMCs are not planned tightly at affected floor then add small cell for traffic offload and monitor the performance.
4. If SMCs are tightly planned then do re-design the floor with additional capacity and get validated from NPE team .

4. Planning and design validation

1. Check if eNBs placed in close proximity is causing SMC to SMC interference.
2. Physical optimization if installed locations are different than plan
3. If above two conditions are met then re-design.

5. Walk Test: Do CVT/Walk to validate coverage gap

1. Increasing in eNB power can help to improve RRC success rate.
2. If power is at maximum, plan additional small cells.
3. if the outdoor signal strength is high and causing interference then increase SMC powe rif not improved, coordinate with Samsung
Optimization team to do physical optimization
Copyright 2016 AIRSPAN NETWORKS INC. 3
RRC Success Rate Flow Chart

Copyright 2016 AIRSPAN NETWORKS INC. 4

HOSR Troubleshooting
1. PCI & RSI Planning Validation- Refer Basic Parameter Guideline *
1. *If Auto PCI and Auto RSI is enabled (Check SON Profile from Netspan) then ignore this step.

2. Clock Issue: Refer “Alarm Document” for clock related issues

1. If No clock alarms but RRC Success Rate < 50% then swap L2 Switch & monitor the performance
2. If multiple cells are having poor RRC Success Rate then try CSS reboot.

3. Audit NRT table –

1. Validate SMC installation plan along with surrounding Macro locations.
2. Check if all possible neighbours are reported in NRT and their X2 links are setup to form required NBR relationship.
3. Check stats to find out specific NBR relationship causing handover failures.

4. Neighbour cell Availability- Check if neighbouring cells are OOS and causing handover failure because of coverage gaps or poor
coverage areas generated.

5. Neighbour cell Accessibility – Check if the neighbouring cells are having Accessibility issues, Refer to RRC Success Rate
troubleshooting module.

6. Walk Test
A. Pingpong HO
1. To identify Ping-Pong Handover, do walk test and check if Handovers are happening back n forth at same locations.
2. If both SMCs are strong (RSRP) then 1. Reduce power or 2. Physical optimization required.
3. If Macro interference causing this then increase power to make SMC dominant.
Copyright 2016 AIRSPAN NETWORKS INC. 5
HOSR Troubleshooting

4. else coordinate with Samsung Optimization to optimize the macro coverage

B. Coverage Gaps
1. Do walk test to identify if Handover failures are happening in poor coverage and bad SINR
2. Increase eNB power to coverage and check HOSR.
3. If power is at maximum, additional small cell planning is required.

Copyright 2016 AIRSPAN NETWORKS INC. 6

HOSR Flow Chart

Copyright 2016 AIRSPAN NETWORKS INC. 7

CDR & VDR Troubleshooting

1. PCI & RSI Planning Validation- Refer Basic Parameter Guideline *

1. *If Auto PCI and Auto RSI is enabled (Check SON Profile from Netspan) then ignore this step.

2. Check HO Performance - Need to check if HO failures are increased.

1. Refer HO Success Rate module

3. Planning and design validation

1. Check if eNBs placed in close proximity is causing SMC to SMC interference.
2. Physical optimization if installed locations are different than plan
3. Else do redesign and NPE will validate.

4. Walk Test: Do CVT/Walk to validate coverage gap

1. Increasing in eNB power can help to improve CDR & VDR.
2. If power is at maximum, plan additional small cells. Raise to NPE for redesign

Copyright 2016 AIRSPAN NETWORKS INC. 8

CDR & VDR Flow Chart

Copyright 2016 AIRSPAN NETWORKS INC. 9

Throughput DL/UL Troubleshooting

1. PCI Planning Validation- Refer Basic Parameter Planning Guideline *

1. *If Auto PCI is enabled (Check SON Profile from Netspan) then ignore this step.

2. Congestion : If “Max. Connected Users” > 30 or “DL PRB %” > 75% & “UL PRB %” > 75% then
1. Check if neighbouring cells are OOS - resolve it and monitor the performance.
2. Check # connected users at NBR cells, if possible increase the power of NBR cell & monitor the performance.
3. If SMCs are not planned tightly at affected floor then add small cell for traffic offload and monitor the performance.
4. If SMCs are tightly planned then raise to NPE/NOC team for re-design the floor with additional capacity.

3. Backhaul Testing: Run Iperf test with maximum data load.

1. If throughput lower than data sent then raise issue to Backhaul team

4. Planning and Design validation

1. Check if eNBs placed in close proximity is causeing SMC to SMC interference.
2. Physical optimization if installed locations are different than plan
3. Else raise to NPE/NOC team for re-design.

5. Walk Test: Do CVT/Walk to validate coverage gap

1. Increasing in eNB power can help to improve Throughput DL/UL.
2. If power is at maximum, plan additional small cells.

Copyright 2016 AIRSPAN NETWORKS INC. 10

Throughput Flow Chart

Copyright 2016 AIRSPAN NETWORKS INC. 11

Alarm & Troubleshooting
Small Cell Alarm List
Sr. No Alarm ID Alarm Description Severity Probable Cause Action

1. Check Reachability through CSS

1 100 Node Lost Comms Critical Power Failure / Backhaul Issue 2. Check power availability on site
3. Check physical backhaul issue

1. Check clock status

2 557 Channel Out of Service Critical Clock Issue / MME not reachable 2. Make sure MME connectivity is OK
3. Check for any PCI Allocation Failure alarm

Loss of the S1AP SCTP connection to 1. Verify the extent of the alarms on the nearby nodes
3 556 MME connectivity lost Major
a specific MME 2. Contact NOC for further advise
1. Verify the extent of the alarms on the nearby nodes
4 580 SGW connectivity lost Major SGW is not reachable
2. Contact NOC for further advise
No PCI resource available in the PCI
5 565 Auto PCI Allocation fail Major Report to NOC for further advise
pool
Wi-Fi Board Wi-Fi board is not active, Wi-Fi
6 587 Major Report to NOC for further advise
Connectivity software version is not updated
eSON Server
7 579 Major Server unreachable Report to NOC
connection Lost

Copyright 2016 AIRSPAN NETWORKS INC. 13

Small Cell Alarm List – PTP / GPS
Sr. Alarm
Alarm Description Severity Probable Cause Action
No ID
The node does not have a valid sync 1. For ISC, check the GM connectivity
1 572 Node Loss of Sync Critical
source and remain in the OOS state 2. For OSC, check GPS connectivity
Node PTP Grand Master 1. Check status in Netspan connection to GM
2 570 Major GM unreachable
Lost 2. Report to IP team to check connectivity to the GM
1. Verify additional alarms involving GPS or GM.
PTP Grand Master Lost
3 571 Node in Holdover Major 2. Rectify physical issues for GPS alarms
PTP Degraded Stream
3. Report to IP team for GM-related alarms

1. Check additional alarms and events in Netspan

6 599 PTP Degraded Stream Warning High PDV
2. Report to NOC.

1. Check onsite if GPS location has clear skyview

Skyview partially obstructed
2. Check connectors, they should free from moist
4 213 GPS SNR Low Minor GPS antenna faulty
3. Check gps cable on both ends make sure properly tighten
GPS antenna cable faulty
4. Try to swap new gps antenna
1. Check onsite if GPS location has clear skyview
GPS Signal 2. Check connectors, they should free from moist
5 224 Minor No satellite detected
Unacceptable 3. Check gps cable on both ends make sure properly tighten
4. Try to swap new gps antenna

Copyright 2016 AIRSPAN NETWORKS INC. 14

Node Lost Comms
Description – The node is not reachable from Netspan
Probable Cause : Power / IP / Backhaul issue Yes From CSS
Check
Netspan No
Impact : Services will be suspended , No Ping possible. Reachabil
ity

Actions to be taken : Yes Check

eNB Check
Reachabi Routes in
lity CSS
1. Management VLAN (IP) Issue
No
a. Request IP team to check reachability from CSS to Netspan Check
Report to
Power No concern
team
b. If not reachable check Netspan route in CSS
c. If Netspan is reachable from CSS, check reachability from CSS to eNB
Yes
d. Check reachability of L2 SW from CSS
2. Power Failure:
Check
Physical
Yes Take
necessary
connecti actions
vity
a. Check power on the site
b. LED should be ON in the node No
3. Backhaul Issue Report to
NOC team
a. Check port status on L2 (Up / Down)
b. Check the physical connections between CSS to L2 Switch and L2 Switch to SMC e-NB. (Loose
connection check )
c. Clean SFPs used at L2 Switch and SMC e-NB. (Dust /Moisture check )
d. Check fiber/CPRI cable (if damaged, bent)

Copyright 2016 AIRSPAN NETWORKS INC. 15

Channel OOS
Description – The node is manageable but RF is not radiating
Probable Cause : Clock Issue, MME connectivity, PCI allocation failure
Impact : Services will be suspended Yes
No Check
PTP
Actions to be taken : Alarms

1 . Clock Issue
Check Report to IP

a. In Netspan, clock status should be connected & Locked

MME
Alarm
Yes Team

b. If it is not Locked, check Grand Master (ISC) or GPS (OSC) connectivity or

refer to PTP & GPS alarm chart
2. MME Connectivity Issue No
Check PCI failure
a. Check MME Alarm (Details in MME Connectivity Lost Alarm) Events

3. PCI Allocation Failure

a. Check PCI alarms / events (Details in PCI Allocation failure)
b. If alarm is present, possible no PCI resources available then report to SON Report to
NOC team
team

Copyright 2016 AIRSPAN NETWORKS INC. 16

MME Connectivity Lost
Description – Loss of the S1AP SCTP connection to a specific MME

Probable Cause : MME is not reachable

Impact : If alarm present on all defined MMEs then service affecting (Channel OOS)

Actions to be taken :
1. Verify other nodes at the same site with the same MME connection if it has the same alarm
a. If it has same alarm report to IP team and check MME status
b. Otherwise, report to IP team to check:
i. Check 602 vlan tagging on L2 switch
ii. Check 602 gateway reachability with source as eNb on CSS
2. If everything is OK contact NOC team

Copyright 2016 AIRSPAN NETWORKS INC. 17

SGW Connectivity Loss
Description – Loss of S1 connection to the SGW

Probable Cause : SGW is not reachable

Impact : If alarm present on all defined SGWs then service affecting

Actions to be taken :
1. Verify other nodes with the same SGW connection if it has the same alarm
a. If it has same alarm report to IP team and check SGW status
b. Otherwise, report to IP team to check :
i. Check 601 vlan tagging on L2 switch
ii. Check 601 gateway reachability with source as eNb on CSS
2. If everything is OK contact NOC team

Copyright 2016 AIRSPAN NETWORKS INC. 18

Auto PCI Allocation Fail Alarm

Description – Automatic allocation of the PCI failed

Probable Cause : No PCI resource available in the PCI pool ( Channel OOS)

Impact : Services will be affected

Actions to be taken :
1. Report to NOC for further advise

Copyright 2016 AIRSPAN NETWORKS INC. 19

Wi-Fi Board Connectivity Alarm
Description – The unit is not able to communicate with the Wi-Fi board (applies to AirVelocity with Wi-Fi only)

Probable Cause : Wi-Fi board is not active, Wi-Fi software version is not updated

Impact : Wi-Fi services will be affected

Actions to be taken :
1. Report to NOC for further advise

Copyright 2016 AIRSPAN NETWORKS INC. 20

eSON Server Connection Lost
Description – Connection with eSON server is lost

Probable Cause : eSON server unreachable

Impact : Not service affecting but some SON performance will be affected

Actions to be taken :
1. Check Netspan status
2. Report to IP team to check connection to eSON server

Copyright 2016 AIRSPAN NETWORKS INC. 21

PTP and GPS Alarm Chart

PTP Degraded GPS SNR LOW

stream

High PDV
(Event)

Check Sky,
GPS Signal antenna no
Node in Holdover Unacceptable obstruction
2.5 Min

Node PTP Grand Node in Holdover

Master Lost

8 Hrs

Node Loss Sync

Node Loss Sync Channel OOS

Copyright 2016 AIRSPAN NETWORKS INC. 22

Node Loss of Sync

Description – The maximum holdover time has expired, node has no valid sync source

Probable Cause : GM is not reachable or GPS connectivity issue

Impact : Services will be suspended (Channel OOS)

Actions to be taken :
1 . For ISC, check the GM connectivity
a. In Netspan, clock status should be connected
b. If it is not Locked, check Grand Master or refer to PTP & GPS alarm chart
c. Report to IP team

2. For OSC, check GPS connectivity

a. In Netspan, clock status should be locked
b. Ensure GPS is well installed having sky clearance
c. Check physical connectivity between eNB and GPS antenna (Moist, connector)
d. If problem persist then try to replace GPS antenna

Copyright 2016 AIRSPAN NETWORKS INC. 23

Node PTP Grand Master Lost

Description – Node has no valid 1588 clock source

Probable Cause : GM not reachable

Impact : Service will be suspended (Channel OOS)

Actions to be taken :
1. Check status in Netspan connection to GM
2. Report to IP team to check connectivity to the GM.

Copyright 2016 AIRSPAN NETWORKS INC. 24

Node in Holdover
Description – Node has no valid clock source

Probable Cause : GM not reachable, High PDV

Impact : May lead to Holdover expire and probable service-affecting

Actions to be taken :

1. Verify additional alarms in Netspan like GPS SNR Low and Node PTP Grand Master Lost and events like
High PDV
2. Report to IP team to check connectivity to the GM.
3. If GPS problem, refer to the GPS SNR Low action item.

Copyright 2016 AIRSPAN NETWORKS INC. 25

PTP Degraded Stream
Description – PDV conditions in the 1588 PTP stream have changed

Probable Cause : PDV reaches the threshold level

Impact: May lead to Holdover and probable service-affecting

Actions to be taken :
1. Check additional alarms and events in Netspan
2. Report to NOC.

Copyright 2016 AIRSPAN NETWORKS INC. 26

GPS SNR Low
Description – Low SNR of the GPS signal

Probable Cause : Sky view partially obstructed ,GPS antenna faulty , GPS antenna cable faulty

Impact : May lead to GPS lock loss

Actions to be taken :

1. Check onsite if GPS location has clear skyview

2. Check connectors, they should free from moist
3. Check gps cable on both ends make sure properly tighten
4. Try to swap new gps antenna

Copyright 2016 AIRSPAN NETWORKS INC. 27

GPS Signal Unacceptable
Description – Satellite not detected

Probable Cause : Sky view partially obstructed ,GPS antenna faulty , GPS antenna cable faulty

Impact : May lead to GPS lock loss

Actions to be taken :

1. Check onsite if GPS location has clear sky view

2. Check connectors, they should free from moist
3. Check GPS cable on both ends make sure properly terminated
4. Try to swap new gps antenna

Copyright 2016 AIRSPAN NETWORKS INC. 28

THANK YOU