Professional Documents
Culture Documents
ACME Checks and Learnings - 2 PDF
ACME Checks and Learnings - 2 PDF
V to V
•VOLTE to VOLTE
V to C
•VOLTE to CS
C to V
•CS to VOLTE
V to P
•VOLTE to PSTN
9
VoLTE – Logical Architecture
ENUM – Electronic Number Mapping System
To translate a telephone number into IP address
Part 2 of 2
1. Equal priority Band and higher priority Bands, are always measured
2. UE on 1.8 G will always measure 2.3G
3. The UE starts measuring other lower priority band neighbors when Serving Cell < S_nonintrasearch
4. UE on 2.3G measures 1.8G when 2.3G is lower than -104 dBm
5. UE does cell reselection from 2.3G to 1.8G when 2.3G is lower than -104 dBm and 1.8G is better
than -104 dBm
6. UE returns to 2.3G when 2.3G is better than -100 dBm
2016 © Samsung Electronics 24
Idle Mode Parameters Behavior
-75dBm -80dBm -85dBm -90dBm -95dBm -100dBm -105dBm -110dBm -115dBm
Band 40
(2300MHz)
• The UE starts measuring other lower 1. For Cell Reselection from 2300 to 1800 happens when
priority band neighbors when Serving 2300 is weaker than Thresh_Serving_Low and 1800 is
Cell < S_nonintrasearch. stronger than Thresh_Xlow (higher to lower priority)
2. For Cell Reselection from 1800 to 2300 happens when
• Equal priority band And higher priority 2300 is stronger than ThreshX_High (lower to higher
Bands, are always measured. priority)
Example
Band 3
Idle Mode (1800MHz)
Band 40
(2300MHz)
than threshold
RSRP
TimetoTrigger
Thresh
Hys
Hys
RSRP
Event A2
Serving Cell
Serving Cell
Time
Leave A1
Condition Enter A1
Enter A1 Condition Hys
Condition
Event A1 Thresh
Hys TimetoTrigger
Leave A2
2016 © Samsung Electronics Condition Event A2 27
LTE Handover Events
Event A3
Neighbor becomes offset better than serving
RSRP/RSRQ
Serving Cell
Neighbor Cell
TimetoTrigger
A3 Offset
Hys
Hys
Hys
Time
Enter A3 Condition
When Ofn, Ocn, Hys,
Event A3
Ofs, Ocs, Off are all zero
Leaving A3 Condition Enter A3 Condition
When Ofn, Ocn, Hys, When Ofn, Ocn, Hys,
Ofs, Ocs, Off are all zero Ofs, Ocs, Off are all zero
Thresh1 Hys
Hys
Time
Leave Condition1 Serving Cell weaker
Enter Condition1 Enter Condition2
Event A5
than Threshold 1
Enter Condition1
1. For GBR for transition from 2.3G to 1.8G, as 2.3G becomes than weaker than -105 for
GBR, UE starts measuring other band and transitions when 1.8G is stronger than
2.3G by A3 Offset which is 3 dBm
2. For Non GBR - A5 for transition from 1.8G to 2.3G (as 2.3G would be weaker than
1.8G in most cases ) i.e. when serving cell in 1.8G is weaker than -60 dBm and the
target cell in 2.3G is better than -110 dBm
3. A2 and A3 are used for GBR for intra & inter band handover and for intra band
handover for non GBR
4. A5 is used for inter band handover for non GBR
2016 © Samsung Electronics 30
Active Mode Parameters Behavior
-75dBm -80dBm -85dBm -90dBm -95dBm -100dBm -105dBm -110dBm -115dBm
Band 40
(2300MHz)
Example
Parameter Template 2: Default Parameter implementation during Golden Parameter audit & corrections
Audit & corrections carried out on 24-Oct over-ride A2 value earlier implemented as a part of Multi-band Parameter list
Technical Analysis:
Default Parameter List vs Multi-band Services Implementation
(Used in Work Stream 1) (Used in Work Stream 2) Remarks
A1: Not Set A1: -100dBm ( value: 40 ) If RSRP is better than A1, it
A2: -90dBm ( value: 50 ) A2: -113dBm ( value: 27 ) If RSRP is lower than A2, UE has
to start measurement of neighbor band
Result:
A2 got configured as -90dBm in place of -113dBm, with A1 as -100dBm
When UE RSRP goes less than -90dBm (A2), inter-band measurements will trigger.
However since RSRP is better than A1 ( -100dBm ), UE will be instructed to stop the inter-band measurements.
Thus conflicting settings for A1 & A2, leads to high messaging & thus eNB going into Racing condition.
2016 © Samsung Electronics 33
Learnings – from incorrect EARFCN settings
EARFCN settings for eNode B and for Multiband should be the same
Else Handovers would get impacted
6 6 6
A3 Threshold
3 dB 3 dB 3 dB
A5 Threshold (Th1) -113 dBm -113 dBm -60 dBm -113 dBm -60 dBm -113 dBm
A5 Threshold (Th2) -110 dBm -93 dBm -110 dBm -110 dBm -110 dBm -110 dBm
36
Neighbor Deletion Based on HO Success
Neighbor Table
Auto build by LSMR and UE based . Max NBR in table are 256
Key parameter are Neighbor cell PCI (Unique/Band), Cell ID , TAC, eNB ID
PCI Confusion
Neighbor Cell Neighbor Cell PCI Reallocation
Case1 Source cell
PCI : “A” PCI : “A” SON PCI needs to be ON
Neighbor Deletion based on HO Success This feature removes Neighbor Relation based on HO Success
Rate
Deployed in network
Neighbor Deletion based on NBRDEL-CAUSE This feature removes Neighbor Relation based on 2 cause
reported Cell not available & Unknown Target ID
Deployed in network
Manual Neighbor Deletion This is based on troubleshooting HO call failure observed in CSL/Truecall.
Not Applicable
Enabled MR based NR deletion on 13th Jan . Before enabling this feature the total NRs in Ahmedabad city are 407192.
on 14th Jan the MR based deletion algorithm deleted the most of the NBRs and the count on 15th Dec is 44K.
Feature Configuration
nrDelFlag = True
thNumMrNrDel = 5 in Ujjain (Range 0 to 1000K)
thTimeNrDel = 1 in Ujjain ( Range 1 to 7 days)
Ujjain Observations
Total eNB 84 , Neighbors 40,000
18Dec - Enabled MR based NR deletion
20Dec - MR based deletion algorithm deleted the most of the Neighbors.
24Dec – 3413 Neighbor created mostly by UE based ANR.
InterX2 HO Attempts
Ujjain Analysis – MR Based Neighbor Deletion HO Performance Stats InterX2 HO Failure
100.00%
45000 40000 15
26 18
40000 99.80% 24
35000
70
InterX2HOSuccRate
91
Number Of NBRs
99.60% 103
30000
25000 99.40%
20000
99.20%
15000
10000 99.00%
2100 3413 3083
5000 400 1200
98.80%
0
7557 5627 7107 12110 12880 9315 11888
98.60%
22-Dec-15 23-Dec-15 24-Dec-15 25-Dec-15 26-Dec-15 27-Dec-15 28-Dec-15
Neighbor Deletion threshold of HO Attempt GT 10 and HO Success less than 5 are not met in most of the cases
1 to 4% of neighbors having HO Attempt greater than 10 indicates less users.
Next Step – Review the HO Attempts and HO Success every month for any optimization of threshold.
46
Pre-requisites/ Checklist (RAN) …..Contd.
1. SCFT: 100% completed and all sites to be radiating.
• Run command “RTRV-CELL-STS” in all eNBs.
• Cells with Status = DISABLE needs to be checked.
2. No Service affecting alarm in eNB
• Run command “RTRV-ALM-LIST” in all eNBs.
• Service impacting alarms needs to be rectified.
3. eNB on 5.0.0. – all patches to be loaded , SW & FW Audits
• Go to Configuration -> Software -> Verify. Select all eNBs and verify the Software and Firmware.
• eNBs with “NOK” status, should be noted and packges should be upgraded in planned event.
• Please refer Software/Firmware Verification in attached pre-requisite file.
4. OCNS to be removed
• Run command “MON-TEST” in all eNBs.
• eNBs with OCNS running should be noted. Run command “TERM-TEST” to remove OCNS, in all eNBs.
• Please refer OCNS Check section for procedure in attached pre-requisite file.
5. Multiband/Golden Parameter Settings
• Go to Configuration -> Parameter Manager -> Select appropriate parameter planner.
Golden Parameter: pkg_500new
Multiband Parameter : Multi_Carrier_23n18_CircleName (For eNBs having 2300 MHz and 1800 MHz bands)
Multi_Carrier_3band_CircleName (For eNBs having 2300 MHz and 1800 MHz bands)
• Select all eNBs and compare the parameter manger.
• eNBs with “NOK” status, should be noted and parameter planner should be applied in planned event.
• Please refer Golden Parameter/Multiband Parameter Settings section for procedure in attached pre-requisite file.
6. PCI SON Notifications to be resolved
• Go to SON -> Log Dump Management. Select the time period and get the dump.
• From dump, check source and target eNBs and solve the PCI collision if both neighbors are required.
• Please refer PCI SON Notifications section for procedure in attached pre-requisite file.
10. PCI repetition within 2 km to be removed for all cases, Additional Audit for 4 km and first tier /second tier neighbor -
all PCI conflict cases to be plotted on map info and corrective actions identified.
— Go to SON -> Log Dump Management. Select the time period and get the dump.
— This dump needs to be provided to RF team, RF team will map conflict cases on map .
— RF will recommend to change PCI of some sites.
— PCI of sites can be changed using GUI.
— Please refer PCI/EARFCN Change section for procedure in attached pre-requisite file.
13. EARFCN definitions to be audited for multiband operation for all elements (Macro & Small cells).
— Run command “RTRV-CELL-IDLE” in all eNBs.
— Check UL and DL EARFCN of all eNBs with EARFCN allocated to respective circle.
— Please refer PCI/EARFCN Change section for procedure in attached pre-requisite file.
50
True Call Report – Cell Wise RLF & X2 Fail (Sample)
eNB ID Cell Name
Total ACME Zone
RLF
5 16 17 18 35 36 37 48 49
1076 13 14 56 112 22 20 104 12 353 New Mumbai JNPT
4147 109 1 11 96 3 4 76 39 339 kalyan
2915 147 80 8 66 2 10 313 New Mumbai JNPT
2166 78 104 35 24 26 16 283 New Mumbai JNPT
2790 17 4 19 124 41 1 61 1 268 New Mumbai JNPT
936 13 24 21 86 12 48 13 15 3 235 New Mumbai JNPT
2918 11 40 56 53 29 13 17 14 233 New Mumbai JNPT
318 38 70 2 1 114 225 New Mumbai JNPT
1556 98 19 8 28 28 23 9 213 kalyan
2919 5 79 61 22 7 174 New Mumbai JNPT
4092 2 15 125 4 146 thane central
4242 123 4 4 2 1 134 south
1337 5 21 30 24 9 7 1 21 9 127 New Mumbai JNPT
1418 13 3 4 27 3 55 20 125 kalyan
1348 1 25 5 7 50 1 17 18 124 west
GTP Pa th Fa i l ure
Abnormal Release Cause
RLC - Inva l i d Pa ra meter
eNB - S1 SCTP Out of Servi ce eNB Ti meout - S1 MME Status Tra ns fer Not Recei ved
MAC - Inva l i d Pa ra meter
X2AP - Ca us e Mi s c Uns peci fi ed
RRM - MME Not Ava i l a bl e for PLMN
eNB Ti meout - RRC UE Ca pa bi l i ty Informa tion Mes s a ge Not Recei ved
eNB Ti meout - Interna l Res ource Setup
eNB Ti meout - S1 Ini tia l Context Setup Reques t
GTP Setup Fa i l ure
eNB Ti meout - S1 Pa th Swi tch Reques t Ack Not Recei ved
PDCP - Inva l i d RB Id
eNB - Recei ved Cel l Rel ea s e Indi ca tion from eNB
X2AP - Cel l Not Ava i l a bl e
S1AP - Ra di o Connection Wi th UE Los t
S1AP - Loa d Ba l a nci ng TAU Requi red
S1AP - CS Fa l l ba ck Tri ggered
eNB - Recei ved Res et Reques t From eNB
RRM - CQI Res ource Fa i l ure
GTP Recei ve Error Indi ca tion From GW
X2AP - Trel ocprep Expi ry
Release Count
% Abnormal Failure
4.60
39154166 4.24 4.22
4.40 40000000
4.20 3.98
4.00 30000000
3.80
3.60 20000000
3.40 10000000
3.20 1843142 1980235 2289216 2218316
3.00 0
Dec Week3 Dec Week4 Jan Week1 Jan Week2
800000
700000
600000
Absolute Failures
500000
Dec Week3
400000
Dec Week4
300000
Jan Week1
200000 Jan Week2
100000
0
Gujrat Mumbai Delhi AP Karnataka Kolkata MP
Circles With True Call Implemented
1 eNB - Radio Link Failure 21.59% 25.98% 16.90% 31.32% 20.15% 25.48% 27.68%
2 S1AP - Reduce Load In Serving Cell 21.03% 2.89% 3.11% 5.45% 6.74% 6.88% 5.94%
3 RRM - UE S-TMSI Duplicate 14.80% 15.81% 21.90% 17.40% 17.09% 16.24% 18.63%
4 eNB Timeout - Inter X2 Handover Command Complete 10.03% 9.71% 14.53% 6.77% 9.35% 12.68% 8.09%
7 eNB Timeout - Intra Handover Command Complete 5.23% 3.76% 7.39% 4.04% 5.27% 8.53% 4.90%
8 eNB Timeout - X2 RelocOverall Expiry 4.52% 5.38% 6.83% 5.09% 5.07% 3.84% 6.26%
10 eNB - ARQ Maximum Retransmission 1.26% 1.81% 2.63% 2.00% 1.80% 2.42% 1.96%
11 X2AP - Cause Misc Unspecified 0.28% 7.93% 0.19% 0.66% 4.20% 0.04% 1.89%
12 GTP Path Failure 0.05% 1.41% 0.32% 0.17% 1.16% 3.04% 0.30%
Out of 79516 Abnormal release more than 95% failures contributed by below shown 10 Cause codes.
25.00
20.99 20.87
20.00
% Contribution to Total Failures
15.00 14.02
11.92
10.00
7.14 6.63
4.56 4.56
5.00
2.79
1.61
0.00
S1AP - Reduce eNB - Radio Link RRM - UE S-TMSI eNB Timeout - eNB Timeout - RRC eNB Timeout - RRC eNB Timeout - X2 eNB Timeout - eNB Timeout - RRC S1AP -
Load In Serving Failure Duplicate Inter X2 Handover Connection Connection Setup RelocOverall Intra Handover Connection Authentication
Cell Command Reconfiguration Complete Message Expiry Command Reestablishment Failure
Complete Complete Message Not Received Complete Complete Message
Not Received Not Received
MOTOROLA MOBILITY LLC A LENOVO COMPANY MOTO E (2ND GEN) WITH 4G LTE ST12424645 1818
Call Flow
MR EVENT A2(LteHo)
RRC Conn ReConfiguration
MR EVENT A5(IntraLteHandover)
As UE does not support Inter frequency HO or
RRC Connection Release (with redirectedCarrierInfo) FDD TDD handover then eNB will initiate redirection
4294.16 473
Top 3 Cell Analysis 2761.50 217
4294.16 is IBS of TC22, and supporting data for further analysis not available
2257.16 201
from CSL for 05/01/2016.
405874000026883 199
No site upto 1 km and very high rise 75.35 108
clutter observed. Failures from far and
2257.16
high-rise indoor location suspected. Last 1076.37 104
I-MU-MUMB-ENB-7334 405874000062987 1
CQI Reported is Of Value 3 (SINR < 0).
Also fails observed in one specific IMSI
405874000016480 1
only. Total 1924
2761.50
> 1 km
Non-Integrated Site
> 1 km
2217.56
1350.18 1953
For remaining fails in this cell, wrong neighbor deletion/PCI conflict to be checked.
1350.18 405874000051815 61
I-MU-MUMB-ENB-1327 405874000064118 53
As seen below, in TrueCall Logs, there are no events between 01/06/16 1:26:56 to 1:28:41, whereas, as per CSL logs, 3
events occurred during same time.
TrueCall Logs
CSL Logs
eNB_1484 0x00000FFF 2016-01-06 10:55:48:650 10:55:48:744 40 405 874 379921 76 2016-01-06 10:56:17:923 76 405 874 379921 405874000017389
eNB_2166 0x00000FFF 2016-01-06 11:19:49:869 11:19:49:962 3 405 874 554532 208 2016-01-06 11:20:21:337 208 405 874 554532 405874000017389
eNB_4241 0x00000FFF 2016-01-06 12:04:18:839 12:04:18:944 40 405 874 1E+06 332 2016-01-06 12:05:08:922 332 405 874 1085714 405874000017389
eNB_4103 0x00000FFF 2016-01-06 13:12:01:035 13:12:01:190 40 405 874 1E+06 501 2016-01-06 13:12:27:062 501 405 874 1050384 405874000017389
eNB_3417 0x00000308 2016-01-06 13:27:25:845 255:255:255:65535 40 405 874 874768 252 2016-01-06 13:27:28:855 252 405 874 874768 405874000017389
eNB_964 0x00000324 2016-01-06 13:26:57:707 13:26:57:801 40 405 874 246802 236 2016-01-06 13:27:30:859 236 405 874 246802 405874000017389
eNB_4091 0x00000236 2016-01-06 13:28:15:081 255:255:255:65535 40 405 874 1E+06 501 2016-01-06 13:28:16:620 501 405 874 1047313 405874000017389
eNB_1662 0x00000343 2016-01-06 13:28:28:082 13:28:28:188 40 405 874 425488 3 2016-01-06 13:28:41:234 3 405 874 425488 405874000017389
eNB_2686 0x00000308 2016-01-06 13:30:44:682 255:255:255:65535 40 405 874 687633 181 2016-01-06 13:30:47:693 181 405 874 687633 405874000017389
64
[0x0308]EccTmout_InterX2HandoverCmdComplete – Case Study
[0x0308] Tags When A call is released when the target eNB cannot receive the RRC Connection Reconfiguration Complete message
during the X2 handover.
As Per UE Logs UE has sent MR For 118 PCI which It Found better so A3 Event Triggered. But After that UE has Started PRACH
Procedure but failed as done on wrong Cell.
A3 Event Triggered
MR HO Preparation
HO Command
EccTmout_InterX2HandoverCmdComplete
2016 © Samsung Electronics 65
How To Identify Wrong Nbr Based Upon CSL – (1/2)
Target CI Source CI
Call Release
Call Attempt Time Call Setup Time cgi.CI[0] cgi.CI[1] IMSI PCI
Cause
By CSL we get Cause Code/ Cell Id From UE History, so that we can Find Source & Target eNB .
Please Check Call Release Cause “0x0000 308” is for A call is released when the target eNB cannot receive the RRC
Connection Reconfiguration Complete message during the X2 handover
By True call Please check Column Name “Last HO Source Cell ID” & “Last HO Target Cell ID”, For Call
Release Casue “eNB Timeout – eNB Timeout - Inter X2 Handover Command Complete
2016 © Samsung Electronics 66
How To Identify Wrong Nbr Based Upon CSL – (2/2)
eNB275
Wrong Id
eNB1231
Correct Id
eNB2155
eNB2155
Wrong Total
Source Id Distance (Km) Source PCI Target PCI Nearst eNB Nearst Distance Remarks
Target Id Failure
2155.37 275.17 5.07 172 53 118 1231 0.76 DELETE
Ue has sent measurement report for eNB1231 PCI 118 but UE Does RACH With eNB275, Due to Wrong eNB
Hence Handover Fail So we can Check Nearby Same PCI & Delete Wrong One To Reduce Drop Rate by Handover Fail
S1 AP Links – Uptime Very Critical
21-Jan-2016
68
UE Temp ID’s – S1AP, GTP-C, GTP-U
GTP Loss
GTP Loss
Root Cause of GTP Loss
Statistic & Troubleshooting
X2 Communication Fail
X2 communication Fail
Root Cause of X2 Communication Fail
Statistic & Troubleshooting
2- Physical drops-
The next common issue that can lead to drops would be a physical component
that is malfunctioning. Physical issues like SFP module problem, cable problem,
loosed connection, line card problem, duplex mismatch, etc. are the most
common reasons for packet drop in network.
These physical drops and backhaul media Buffer overflow results in low
throughput.
Cisco team can only identify case # 1 in the above 3 cases. So Cisco team
should not declare that backhaul is clean without buffer size and rate limitation
or bottleneck bandwidth confirmation. Only by ping test, buffer size issue and
wrong rate limitation issue cannot be identified.
Circle AP AS BR DL GJ HR JK KA KO MH MP MU PB RJ TN UE UW WB Total
Count 12 3 21 12 150 20 64 11 13 76 8 185 326 3 10 35 31 137 1,117
Intensive inspections and improvements required
(GJ, MU, PB, WB)
1. Missing MME Interface for 1100 eNB added.
2. MME Interface not required in MME Pool deleted as Database cleanup activity.
3. eNB grown before MME Pool activity having Long time IND MME alarms since they were grown with old MME Pool.
Next Step – Above step to continue for 1 week to clean IND communication alarm
GTP-U
GTP-C
GTP-U MPLS Network
GTP-U
FTP
If you find out the Low Throughput somewhere during test in the field.
As you well known, Please check the RF Condition including eNB status (Is it down or not ?)
Please check the current alarm (Related BH, Related RU, Related RF, Etc.)
Check the Rate limitation using shared MOP
If you do not find out the reason, then call the RAN BH TF
It provides mobility. When UE is mobile, the IP address remains same and packets are still forwarded
since tunneling is provided between PGW and eNB via SGW.
GTP Loss can happen only by user traffic but cannot occur by ping or traceroute betwee
n SAEGW and eNB.
GTP tunnel is formed only between SAE-GW and eNB so GTP loss has no relation with UE
RF conditions.
When this data comes to SAE-GW from server then It encapsulates this data using GTP, UD
P, IP, L2 and L1 headers and forwards this data to eNB through GTP tunnel.
GTP header consists of Sequence no. & tunnel end point Id (TE ID) which is assigned to each
packet.
This encapsulated data transfers through GTP tunnel with Sequence no. 1,2,3,4,5….. and so
on and reaches to eNB.
If after 2nd packet eNB receives 4th packet instead of 3rd packet then this event would be cou
nted as loss in GTP loss counter but after receiving 3rd packet this event would be counted as
Out of Sequence packet(OOS) as shown below-
That means this packet was not lost but came after words (Out of sequence packet) so this will
not contribute to GTP loss. This can be seen in GtpSnEnbDeltaLoss column which is calculated as-
Buffer Overflow-
Rate Mismatch-
Case 1 Case2
-Physical issues like SFP module problem, cable problem, loosed connection, line card proble
m, duplex mismatch, etc. are the most common reasons for packet drop in a network.
-If hardware is not working properly, it will usually lead to error messages being seen on the con
sole of the device or within system logs but there are other software issues also which can lead t
o packet drop and main concern is there is no alarm indication mechanism for these issu
es in the system
GTP loss can’t be avoided in commercial mode when multiple TCP session
GTP Loss cannot be used as any index or indicator which means backhaul normal
ity because of speed mismatch in backhaul environment (10G -> 1G on AG1 to CSS
ring, 1G -> 470, 250, 150, 100, 45 on Microwave or Leased line), GTP Loss by bo
ttleneck buffer overflow cannot be avoidable in commercial mode where
Multi TCP sessions keep trying to increase TCP window size so congestion happ
ens in a bottleneck point and so buffer overflow occurs.
NOTE-GTP Loss must be less than equal to 0.01% for proper throughput. (>50Mbps)
Note- Please note that the GTP loss counter is only valid for downlink throughput
analysis.
- Perform IPERF test between SGW and eNB on lower B.W.(less than equal to 10Mbps)
- If packet drop is observed while pumping lower B.W from SGW to eNB then that means there is
some physical issue in backhaul.
-Ask cisco team to check backhaul.
Note- Please note that on Lower B.W. (<=10Mbps) packet loss should be less the 0.01%.
If no drop observed on lower B.W, proceed further to check if any bottleneck present in backha
ul by performing iperf test on higher B.W.
-Once you calculated the RTT, find Buffer size using Buffer estimation chart shown below-
Suppose you got rate limit (Bottleneck B.W.) as 237Mbps and RTT as 0.271 sec then in above c
hart search for RTT equal to or nearer to 0.271 in the row of 250Mbps and look corresponding
Buffer size vertically. Here Buffer size is 8192KBytes i.e. around 8MB.
eNB Side :
Iperf.octeon64 -c SGW_IP -V -B eNB_bearer_IP -u -l 8000 -i 1 -t 999 -b 10m -p 8001
SGW(LENA) :
iperf.octeon64 -s -V -u -l 8000 -i 1 -p 8001
If no loss found till 10Mbps in uplink then we can say that uplink path is clean.
NOTE-
You can use any port (“-p 5001”) for iperf test as per your wish but port number must be same
at both client and server end.
-Even though low throughput is observed in the field, if the below 3 cases are not in backh
aul, definitely there's no backhaul issue causing low throughput.
- Cisco team can only identify case # 1 in the above 3 cases. So Cisco team should not declar
e that backhaul is clean without buffer size and rate limitation or bottleneck bandwidth co
nfirmation. Only by ping test, buffer size issue and wrong rate limitation issue cannot be iden
tified. Cisco team needs to find out a method to improve their backhaul check.
Observed packet drop in iperf test and asked cisco team to check backhaul.
NOTE-
Packet loss got cleared after clearing physical issue in backhaul by cisco circle team.
115 115
X2 Disabled Status Summary
Nexus
SAR
AG3
AG2 AG2
X2 Request to Target eNB
AG1 AG1
CSS CSS
UE
X2 Disabled cases decreased significantly due to X2 routing done through SAR instead of AG2 Router.
Out of 12,11,000 X2 links, only 4892 links (0.40%) are in disabled state.
These are primarily in Tamilnadu , UP(East), Karnataka which are deployed with juniper routers and Delhi,
Maharashtra with Cisco routers, Which we are jointly troubleshooting
X2 disabled cases is currently reduced to ~1%. Will further troubleshoot and resolve to make it nil
Confidential
2016 © Samsung Electronics Reliance Jio Infocomm Limited 116 116
Pre-checks for X2 Interface
From Cisco Side –
MTU Configuration Check & Correction .
Two IP addresses for single eNB, removing the multiple IP's.
AG1 Config change for Boundary Cases
Cisco ASR 901 ( Old Cisco Router) Image Up-grade to handle Neighbor Discovery Protocol Issue.
Correction of routing issue at CSS and other nodes.
From Juniper Side –
Juniper Router (ACX2200) Image Up-grade to handle Neighbor Discovery Protocol Issue.
Correction of routing issue at CSS
From eNB Side
"LTE_SIGNAL_X2 " needs to be true only for VLAN 101.
eNB IP Correction in eNB Neighbor list.
eNB Administrative State in both Neighbors to be "Unlocked"
"NO_X2State" should be "False" instead of "True" in both the neighbors.
eMBMS IP change in order to make subnet mask "/126" from "/64" as it was falling under same n/w
of neighbor eNB making wrong entries in eNB routing table.
Confidential
2016 © Samsung Electronics Reliance Jio Infocomm Limited 117 117
S1 AP Bearer Preservation
118
S1 AP Preservation – To improve VoLTE performance
Call Drop Analysis of ACME drive Test
Sr. No. Time Samsung NEO RF/eNB S1AP Cause code Time taken to
recover (sec)
1 15:39:38.259 A call is released when the target eNB cannot receive the RRC radioNetwork: ho-failure-in-target-EPC- 0.22
Connection Reconfiguration Complete message during the S1 eNB-or-target-system (6)
handover.
2 19:08:59.258 RLF at MT due to poor SINR. MT send measurement report radioNetwork: ho-failure-in-target-EPC- 2.10
continuously but no HO command received from eNB eNB-or-target-system (6)
3 15:34:25.566 RLF happened due to HO failure at MO, poor SINR radioNetwork: failure-in-radio-interface- 0.30
procedure (26)
4 16:27:08.406 MO logs are incomplete and no drop at MT found. radioNetwork: ho-failure-in-target-EPC- 0.10
eNB-or-target-system (6)
5 16:37:09.774 No RF issue. No uplink RSSI issue found. Call is released when ARQ radioNetwork: unspecified (0) 2.90
ACK is not received from the UE after the eNB RLC performs
transmission as many as the Max Retransmission to the UE. UE issue
suspected.
6 16:55:17.871 HO Failure due to poor SINR at MT. A call is released when the target radioNetwork: ho-failure-in-target-EPC- 0.30
cell cannot receive the RRC Connection Reconfiguration Complete eNB-or-target-system (6)
message during the intra-handover
7 18:21:42.528 RLF happened due to poor SINR at MT radioNetwork: failure-in-radio-interface- 0.70
procedure (26)
8 18:32:21.596 Call dropped because of HO failure. High DL BLER observed No NW logs available 0.10
9 18:22:00.177 UE lost UL synch and trying to access eNB again. But RACH fails and radioNetwork: failure-in-radio-interface- 0.50
RLF happened at MO. Poor SINR. procedure (26)
Analysis
• MME deleting the QCI-1 bearer after receiving UE Context Release request with a cause of HO failure
and UE lost from eNB.
Solution
• Since HO failure and UE lost due to RLF is being observed in the field and it is also observed that the
recovery process is also happening is less than 1 sec.
• It is suggested to preserve the QCI-1 bearer in case of UEContextRelease with HO failure MME
deleting the QCI-1 bearer after receiving UE Context Release request with a cause of HO failure and
UE lost from eNB
• Preservation ON is implemented in IDC on 1st Sep. Performance will be monitored for ACME .
Equal SAEGW Traffic distribution
TA/TAL/BTAL Configuration Audited
GBR Bearer preservation set to “ON” (i.e. In Test Area Serving 7 MMEs Only )
S1AP - Cause
Cause Description Old Value New Value Benefit
Code Value
Handover Failure In Target EPC/eNB Or Target
06 OFF ON
System
Failure in the Radio Interface Reduction in Call Drops due to RLF
26 OFF ON for fractions of seconds
Procedure
21 UE Connection Lost OFF ON
Clear Test Area Service Impacting Alarms
Check Test Area Availability Status
2016 © Samsung Electronics 121
S1AP preservation option in MME
When eNB detects any RAN layer problems, eNB can send ‘UE Context Release Request’
message to MME to release the UE-associated S1 connection
As per 3GPP standard, MME should release the UE context as per the request
eNB MME
UE Context Release Request (Cause)
However, eNB’s local decision may be wrong in case of some causes, for example,
UE gets lost from an eNB, but the UE can re-attach to some other eNB
During HO, UE is not responding and X2 timer expires, but the UE can re-attach to some other eNB
For some cases, it can be better to preserve S1 connection rather than releasing UE context
Samsung MME provides such options (On/Off) for every S1AP cause code
2016 © Samsung Electronics 122
Ref: S1AP cause code
3GPP TS 36.413 defines the following cause codes
Code Cause Code Cause
0 Unspecified 20 User Inactivity
1 TX2RELOCOverall Expiry 21 Radio Connection With UE Lost
2 Successful Handover 22 Load Balancing TAU Required
3 Release due to E-UTRAN generated reason 23 CS Fallback triggered
4 Handover Cancelled 24 UE Not Available for PS Service
5 Partial Handover 25 Radio resources not available
6 Handover Failure In Target EPC/eNB Or Target System 26 Failure in the Radio Interface Procedure
7 Handover Target not allowed 27 Invalid QoS combination
8 TS1RELOCoverall Expiry 28 Inter-RAT redirection
9 TS1RELOCprep Expiry 29 Interaction with other procedure
10 Cell not available 30 Unknown E-RAB ID
11 Unknown Target ID 31 Multiple E-RAB ID instances
12 No radio resources available in target cell 32 Encryption and/or integrity protection algorithms not supported
13 Unknown or already allocated MME UE S1AP ID 33 S 1 intra system Handover triggered
14 Unknown or already allocated eNB UE S1AP ID 34 S1 inter system Handover triggered
15 Unknown or inconsistent pair of UE S1AP ID 35 X2 Handover triggered
16 Handover Desirable for Radio Reasons 36 Redirection towards 1xRTT
17 Time Critical Handover 37 Not supported QCI Value
18 Resource Optimization Handover 38 Invalid CSG Id
19 Reduce Load in Serving Cell
2016 © Samsung Electronics 123
Preservation option case study
RJio network has the following S1AP preservation options = ON
ID DESC KT RJio
1 Tx2relocoverallExpiry ON ON
3 ReleaseDueToEutranGeneratedRsn ON -
6 HoFailInTargetEpcEnborTargetSy ON ON
8 TS1relocoverallExpiry ON -
16 HoDesirableForRadioReason ON -
20 UserInactivity ON -
21 RadioConnectionWithUeLost - ON
26 FailureInTheRadioInterfaceProcedure - ON
28 InterRatRedirection ON -
125
X2 HO - SGW change Between Noida/Vikaspuri
• Possible Impacted Case:
For Single PLMN, two Sites
distributing traffic with S5.
• Applicable Sites - Mumbai,
Delhi and Chennai.
• Scenario:
• On Registration Co-located
SAEGW get selected
• Initial PGW provides IP to
user
• Once HO happens
between SGW Group,
New SGW will get selected
but PGW remains same.
• Signaling & Traffic moves
from new SGW to old PGW
on S5 interface.
Chennai
Hyderabad
Hyderabad Guwahati
Kolkata
Bangalore Shilong
Bangalore
Guwahati
130
ACME Related Issue Analysis Summary
Test Scope
• Test Drive Location - Bhopal , Ahmedabad, Kolkata, Analysis Summary
Mumbai, Delhi
1 MGCF Issue resulting in SIP "480" MGCF Faulty Card replaced. Close MCGF
Other
2 MGCF Issue resulting in SIP “500" MGCF getting release from Other Operator POI Close
Operator
8 IMS Failure SIP "480" Patch applied to fix the same. Close IMS
Resolution: Issue got fixed after changing A2 Parameter configuration change in eNB which was
generating race condition within eNB and were not responding to MME for ERAB Setup requests.
Issue Description: Delay from OCS for CCA response resulted in VoLTE Call setup Failure with ASR error
“Session released - service based local policy function aborted session".
Issue Description: IMS is not forwarding 200 Ok for Invite towards MO UE.
City – Mumbai
Date – 06-Jan-2016
City – Mumbai
Date – 06-Jan-2016
City – Mumbai
Date – 07-Jan-2016