Course RF151 Recognizing Common System Problems Recognizing Common System Problems From Drive-Test Files From Drive-Test

Files

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1-1

Common System Problems

What's a good Call? Pilot Pollution Weak Coverage Forward Link Interference Reverse Link Interference Troubleshooting Dropped Calls Search Window Problems Troubleshooting Access Failures References Normal Call Processing Event Templates Basic Steps in Post-Processing Drive Files with Actix Analyzer

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1-2

What is a Good CDMA Call?

In a good call: FER is low, typically 2% or less Receive power is moderate Not stronger than -40, that would cause RX overload Not weaker than -100 Composite Ec/Io is -10 or above Transmit Gain Adjust is between -20 and 0 Transmit Power Output is reasonable for the mobile's location -40, -50 near a base station +23 max at edge of coverage
1, 2, or 3 Sectors Dominant
Traffic Channels

SIGNATURE: GOOD CALL

FFER
100%

RXL
-30 -40

EC/IO
0

TxGa
+25

TxPo
+23 +10

-6 50% -10

+10 0 -10

0 -10 -20 -30 -40

-15 10% 5% 2% 0% -90 -100 -110 -20 -20 -25

-50

FFER

RXL

EC/IO

TxGa

TxPo

Io = -90 dbm Ec = -96 dbm Ec/Io = -6 db

4w 1.5w 0.5w 2w

Paging Sync Pilot

I0 EC

BTS

Messaging

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1-3

What is Pilot Pollution?

Pilot Pollution: Many Signals, None Dominant

In Pilot Pollution, the mobile hears many sectors all about the same strength No single sector is strong enough to provide a good call by itself To survive, the mobile must be in soft handoff with several sectors Access failures are usually higher than normal System Capacity is wasted since each call uses so many sectors

Io = 10 signals each -90 dbm = -80 dbm Ec of any one sector = -96 Ec/Io = -16 db

Traffic Sync & Paging Pilot Traffic Sync & Paging Pilot Traffic Sync & Paging Pilot Traffic Sync & Paging Pilot Traffic Sync & Paging Pilot Traffic Sync & Paging Pilot Traffic Sync & Paging Pilot Traffic Sync & Paging Pilot Traffic Sync & Paging Pilot Traffic Sync & Paging

BTS10 BTS9 BTS8 BTS7 BTS6 BTS5 BTS4 BTS3 BTS2

I0

BTS1

Pilot

EC

Solving Pilot Pollution

Pilot Pollution is caused by too much coverage overlap by multiple base stations. The solution is to reduce the coverage of base stations, usually by height reduction, antenna downtilting, and/or power reduction. 1-4

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

Weak Coverage
Weak coverage areas have a peculiar "signature" of conditions Low receive power, causing Low Ec/Io, causing High FER High Mobile Transmit Power High Transmit Gain Adjust Difficulty hearing complete messages in each direction Missing messages sometimes prevent call events from happening
Solving Weak Coverage
The cure for weak coverage is additional base stations, increased height, higher-gain antennas, etc. October, 2001
BTS

SIGNATURE:
DROPPED CALL, BAD COVERAGE
FFER
100%

RXL
-30 -40

EC/IO
0

TxGa
+25

TxPo
+23 +10

-6 50% -10

+10 0 -10

0 -10 -20 -30 -40

-15 10% 5% 2% 0% -90 -100 -110 -20 -20 -25

-50

FFER

RXL

EC/IO

TxGa

TxPo

Messaging

RF151 v1.0 (c) 2001 Scott Baxter

1-5

Forward Link Interference

Forward Link Interference happens when the mobile is listening to one sector or sectors but there are even stronger interfering signals present Mobile Receive power (Io) is raised by the interfering signals Ec/Io is driven much more negative by the increased Io FER increases Mobile transmit power is driven down by higher Io, but Transmit Gain Adjust rises to cancel out the decrease and keep mobile transmit power normal Reverse link works fine, forward link has trouble
Solving Forward Link Interference
Adding the forward link interfering signal into soft handoff resolves the forward link interference. Anything which prevents a needed handoff can cause Fwd. Link Interf. October, 2001
BTS

SIGNATURE:
FORWARD LINK INTERFERENCE
FFER
100%

RXL
-30 -40

EC/IO
0

TxGa
+25

TxPo
+23 +10

-6 50% -10

+10 0 -10

0 -10 -20 -30 -40

-15 10% 5% 2% 0% -90 -100 -110 -20 -20 -25

-50

FFER

RXL

EC/IO

TxGa

TxPo

Messaging

RF151 v1.0 (c) 2001 Scott Baxter

1-6

Reverse Link Interference

SIGNATURE:
Reverse Link Interference happens when a foreign signal or signals are received at the base station This degrades base station reception of mobile signals Mobile transmit power is adjusted higher to compensate for interference In severe cases, far-out mobiles can't increase enough to be heard, and drop
Solving Reverse Link Interference
The solution is to identify and remove the strong interfering signal(s) on the reverse link. "Rogue" mobiles are the most common cause of reverse link interference; these mobiles should be in handoff with the victim base station, but the handoff is somehow prevented. October, 2001

REVERSE LINK INTERFERENCE

FFER
100%

RXL
-30 -40

EC/IO
0

TxGa
+25

TxPo
+23 +10

-6 50% -10

+10 0 -10

0 -10 -20 -30 -40

-15 10% 5% 2% 0% -90 -100 -110 -20 -20 -25

-50

FFER

RXL

EC/IO

TxGa

TxPo

BTS

Messaging

RF151 v1.0 (c) 2001 Scott Baxter

1-7

A Common Cause of both Forward and Reverse Link Interference

FORWARD LINK DIES ns tio c tru s B Ob
BTS

A
BTS

REVERSE LINK DIES B

BTS

s Ob

ns tio c tru
l ve a

Tr

l ve a

Tr

Common cause: a handoff is needed, but somehow can't happen Possible factors which could prevent the handoff: the new sector is not on the neighbor list of the old the neighbor search window is set too small problem happens so fast, mobile and system cant react in time the new sector is in forward power overload, or TCEs used up the new cell is an island due to timing errors
October, 2001 RF151 v1.0 (c) 2001 Scott Baxter 1-8

Setting Pilot Search Window Sizes

When the handset first powers up, it does an exhaustive search for the best pilot. No windows are used in this process. On the paging channel, the handset learns the window sizes SRCH_WIN_A, N, R and uses them when looking for neighbors both in idle mode and during calls. When a strong neighbor is requested in a PSMM, the former neighbor pilot is now a candidate. Its offset is precisely remembered and frequently rechecked and tracked by the phone. Window size for actives and candidates can be small, since the windows float, drifting with the observed pilot energy of the signal. Only search wide enough to include multipath energy! This greatly speeds up overall searching! Most post-processing tools deliver statistics on the spread (in chips) between fingers locked to the same pilot. These statistics literally show us how wide the SRCH_WIN_A should be set. Neighbor and Remaining search windows should be set to accommodate the maximum intercell distances which a mobile might experience

SEARCH WINDOW SETTINGS AND PROPAGATION DISTANCES

Window Datafill N,R Delta Distance Size (Chips) Value Miles KM. 14 (7) 20 (10) 28 (14) 40 (20) 60 (30) 80 (40) 100 (50) 130 (65) 160 (80) 226 (113) 320 (160) 452 (226) 4 5 6 7 8 9 10 11 12 13 14 15 1.06 1.52 2.12 3.03 4.55 6.07 7.59 9.86 12.1 17.1 24.3 34.3 1.71 2.44 3.42 4.88 7.32 9.77 12.2 15.9 19.5 27.6 39.1 55.2

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1-9

Handoff Problems: Window Dropped Calls

Calls often drop when strong neighbors suddenly appear outside the neighbor search window and cannot be used to establish soft handoff. Neighbor Search Window SRCH_WIN_N should be set to a width at least twice the propagation delay between any site and its most distant neighbor site Remaining Search Window SRCH_WIN_R should be set to a width at least twice the propagation delay between any site and another site which might deliver occasional RF into the service area
SITUATION 1 A
BTS

12 80 mile Ch s ips

Locked to distant mo site, cant see un one nearby tai ns B

BTS SRCH_WIN_N = 130 BTS A is reference. 1 mi. BTS B appears (7-80) chips 7 Chips early due to its closer distance. vel This is outside the 65-chip window.Tra Mobile cant see BTS Bs pilot, but its strong signal blinds us and the call drops.

SITUATION 2 A
BTS

12 80 mile Ch s ips SRCH_WIN_N = 130 BTS BTS B is reference. 1 mi. BTS A appears (80-7) chips 7 Chips late due to its farther distance. l This is outside the 65-chip window. Trave Mobile cant see BTS As pilot.
1 - 10

Locked to nearby mo site, cant see un distant one tai ns B

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

Optimizable Dropped Calls: Slow Handoff

When the mobile is suddenly confronted with a strong new signal, or when the signal it is using takes a sudden deep fade, it will have poor Ec/Io and high forward FER. The call will drop unless it gets help quickly. Several steps which must occur without delay: The mobile search correlator must first notice the new pilot and send a PSMM to the system. BTS The system must set up the soft handoff and notify the mobile. The mobile must acquire the new signal by locking a finger
BTS

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 11

Sources of Delay Causing Slow Handoff

Every step in the handoff process can suffer delay if were not careful to control conditions: Mobile search correlator notices new pilot Window sizes too large, searching is slow Multi-sector soft handoff already underway, many active pilots, searching is slow Interferor not a neighbor, must find in remaining set: slow, DIE! System cannot currently set up true remaining-set handoffs Mobile reports PSMM to system. Reverse link noisy, PSMM must be re-requested & repeated System sets up handoff, sends EHDM to mobile Resource congestion: no TCEs, or other problems Forward link is noisy, mobile doesnt hear EHDM, must repeat

Fortunately, these problems do not have to happen.

October, 2001 RF151 v1.0 (c) 2001 Scott Baxter 1 - 12

Troubleshooting Dropped Calls Troubleshooting Dropped Calls

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 13

Dropped Call Troubleshooting - Mobile Side

Just arrived on sync channel! Is this a drop? yes Were there release messages? no This is a drop! yes Check for: yes Weak Signal/Coverage Hole? Strong Fwd/Rev interference? Is T-1unstable/blocking? Why didnt handoff happen? PN not in neighbor list Weak Signal/Coverage Hole? FER already too bad? Border configuration problems Fast-rising pilot, slow reaction Forward Power Channel Elements Rev. Link Noise Add PN to Nbr List! Add coverage Push earlier Debug, reconfigure Incr Sector Overlap Speed up searcher Optmz Fpwr DGUs Add chan cards Identify, fix source Report/repair Improve coverage Identify, eliminate Report/repair OK, normal end of call

Was the Sync Channel PN Active before the drop? no Did mobile request Sync CH PN in PSMM before drop? no Is PN in neighbor list? yes no yes Repair/Re-initialize Cell! Is SRCH_WIN_N adequate? yes Is cell in island Mode? no Is T-1unstable/blocking?

no Add PN to Neighbor List!

Widen SRCH_WIN_N!

Blocking

Is T-1unstable/blocking? More information needed. Collect system logs and merge with mobile data, analyze

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 14

Troubleshooting Access Failures Troubleshooting Access Failures

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 15

Troubleshooting Access Failures & TCCFs

Troubleshooting access failures (Traffic Channel Confirmation Failures) can be difficult There are many steps in the access process Finding which step failed is not easy Rarely, circumstantial evidence points clearly to the problem Usually, it is necessary to debug the process leading up to the access failure Consider each step in the access process Get evidence to determine whether this step occurred successfully Move on to the next step and keep checking steps until the unsuccessful step is found Determine why this step failed The following slides describe the steps in the access process, where they take place, and some of the factors which may cause them to fail This narrative might be useful as a template for organizing your own thinking as you investigate access failures you are tracking! Go out and capture actual drive tests of failed origination attempts If possible, also collect system logs (RF call trace, etc.) for the same event
October, 2001 RF151 v1.0 (c) 2001 Scott Baxter 1 - 16

Troubleshooting Access Failures (1)

BTS

Steps in the Access Process

Access Channel
Origination Msg. Probe #1 Mobile waits to see if the BTS hears and acknowledges its probe within the time ACC_TMO. If not, the mobile must transmit the message again in another probe, this time PI db. louder. Origination Msg. Probe #2 Mobile waits again to see if the BTS hears and acknowledges its probe within the time ACC_TMO. If not, the mobile must transmit the message again in another probe, this time PI db. louder. Origination Msg. Probe #3 The mobile keeps probing until NUM_STEP probes have been sent, then repeats the probe sequence again until Max_Probe_Sequences have been sent.

Troubleshooting Comments
If the mobile does not hear acknowledgment from the BTS within ACC_TMO, this could mean either: The BTS did not hear the mobile Maybe the mobile collided with another mobile transmitting at the same time Maybe mobile was too weak to overcome the existing reverse noise level at the BTS In either case another probe should solve the problem, provided PI is set reasonably and additional probes are allowed (check the Access Parameters Message to see if Num_Step and the power parameters make sense; be sure also the cell size or Access Channel acquisition search width is set large enough and the number of access preamble frames is large enough for the cell size) The BTS is acknowledging but the mobile cannot hear the acknowledgment If the mobile cant hear the BTS acknowledging, Ec/Io is likely quite poor. If so, check whether this is due to weak signal (poor coverage) or pilot pollution (lots of pilots all weak but no dominant server) Collect system logs if necessary to determine definitely whether the system heard the mobiles origination or not

Paging Channel

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 17

Troubleshooting Access Failures (2)

BTS

The Access Process

Access Channel

Troubleshooting Comments
If this problem happens frequently, the BTS traffic overload must be relieved. Here are some steps to try: Investigate BTS TX hardware to ensure everything is working correctly and properly calibrated, particularly gain settings in the TX chain To free up more forward power for traffic channels, try: Reduce PTXstart (initial traffic channel DGU) watching for less forward power control overloads. If you go too far, you will notice access failures increase. Reduce PTXmax (maximum traffic channel DGU) watching for less forward power control overloads. If you go too far, dropped calls will increase. Reduce sector traffic by reorienting the sectors to more closely balance the load carried by each Or, add another carrier Or split cells

Paging Channel

One Dreaded Possibility: Reorder

Mobile beeps and displays Call Failed - System Busy

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 18

Troubleshooting Access Failures (3)

BTS

The Access Process

Access Channel

Troubleshooting Comments
After hearing the BTS acknowledgment, the mobile will stop probing and wait for further instructions on the paging channel. If the mobile does not hear the Channel Assignment Message within 12 seconds, the mobile will beep and display Call Failed. Possible causes: The BTS did not transmit the Channel Assignment Message Check system logs to see if this was not transmitted. If not transmitted, get troubleshooting help from the system manufacturer -- this should never occur The BTS did transmit the Channel Assignment Message, but the mobile did not hear it Was this because the paging channel faded? (Did the Ec/Io drop momentarily)? If so, see If this is a recurring problem such as a coverage hole or severe pilot pollution Finally! The mobile hears the Channel Assignment Message! Now it will immediately leave the paging channel and start trying to hear the new Forward Traffic Channel.

Paging Channel
Base Station Acknowledgment

Channel Assignment Message

STOP! Leave the Paging Channel, and dont transmit again on the access channel. The mobile now goes to try to hear the Forward Traffic Channel.

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 19

Troubleshooting Access Failures (4)

BTS

The Access Process

Troubleshooting Comments
The mobile listens to the Walsh Code # given in the Channel Assignment Message. It should hear N5M good frames full of all zeroes within T2M seconds (usually 2 frames in 10 frames). If the mobile does not hear the required number of good empty frames, it will beep and give an error message, then reacquire the system. If the mobile hears the required number of good empty frames, it starts transmitting its own Reverse Traffic Channel Preamble of empty allzero frames. If the BTS does NOT hear the mobiles access preamble within a prescribed delay, it will abort the process and release all the resources, and the mobile will reacquire the system. . This is what Lucent terms a Traffic Channel Confirmation Failure (TCCF).

FWD Traffic Channel REV Traffic Channel

00000000000000000000 00000000000000000000 00000000000000000000 Mobile beeps and displays Call Failed

00000000000000000000 00000000000000000000 00000000000000000000

Base Station Acknowledgment Mobile Station Acknowledgment

If the BTS DOES hear the mobiles access preamble, it will send an acknowledgment. The mobile responds with an acknowledgment, or maybe even a pilot strength measurement message if it already needs a handoff.

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 20

Troubleshooting Access Failures (5)

BTS

The Access Process

Service Connect Message

Troubleshooting Comments
Now that the BTS and mobile see each other on the traffic channels, the next step is service negotiation. The BTS sends a Service Connect message listing the type and rate set of the vocoder or other primary traffic source. The mobile either accepts the proposal with a Service Connect Complete message, or counterproposes a different mode.

FWD Traffic Channel REV Traffic Channel

Service Connect Complete Message This is still just an ongoing access attempt Base Station Acknowledgment Now this is officially a call in progress

The BTS acknowledges the Service Connect Complete message.

The call is now officially in progress. If anything happens to interrupt it after this point, that is considered a dropped call. If any of these steps is unsuccessful, the call attempt will probably fail. Suspect RF conditions on the link which was supposed to carry the unsuccessful command. Look at system logs and message logs from mobile drive testing to pin down just what happened.

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 21

Access Failure/TCCF Troubleshooting

Access Attempt Failed Were any probes acknowledged? Yes, BS Ack No, Nothing Yes, Reorder Blocking Forward Power Channel Elements Rev. Link Noise Optmz Fpwr DGUs Add chan cards Identify, fix source Add coverage Identify, eliminate Report/repair Identify, fix source Ensure reasonable values Ensure reasonable values for cell size Software problem Resource blocking

Weak Signal/Coverage Hole? Paging Channel faded, lost no Strong Fwd interf / pollution? Is T-1unstable/blocking? Check System Logs. Was mobile heard? yes no Was Channel Assignment Message heard? yes Did mobile see N5M good frames on F-TCH? yes Check System Logs. Did BTS see mobile preamble? yes Did mobile see BS Ack? yes Check System Logs. Did BTS see mobile Ack? OK yes Check System Logs. Was CH ASN sent? no no Check System Logs. CH EL initialized OK? yes no no F-TFC Channel faded, lost Rev Link Overload? Num_Step, Pwr_Step appropriate? Sector Size, Acq Width appropriate? System Problem. Investigate why

no

Rev. Link Noise Init TCH DGU large enough? Weak Signal/Coverage Hole? Strong Fwd interf / pollution? Is T-1unstable/blocking? Weak Signal/Coverage Hole? Strong Rev Noise? Is T-1unstable/blocking?

Identify, fix source Raise DGU Improve coverage Identify, eliminate Report/repair Improve coverage Identify, eliminate Report/repair

no

R-TFC Channel faded, lost

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 22

Normal Call Processing Normal Call Processing Event Templates Event Templates

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 23

Registration
Registration Message (by PROBING)
BTS

Base Station Acknowledgment Order Access Channel

Paging Channel

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 24

Voice Mail Notification

Feature Notification Message
BTS

Mobile Station Ackmt. (by PROBING) Base Station Acknowledgment Order Access Channel

Paging Channel

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 25

Incoming Call Delivery Scenario

General Page Message Page Response Message (by PROBING)
BTS

Base Station Acknowledgment Order Channel Assignment Message Continuous frames of all 000s Traffic Channel Preamble: Frames of 000s Access Channel

Paging Channel

Forward Traffic Channel

Base Station Acknowledgment Order Mobile Station Acknowledgment Order Service Connect Message Service Connect Complete Message The Call is now officially Established!

Reverse Traffic Channel

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 26

Mobile-Originated Call Scenario

Origination Message (by PROBING)
BTS

Base Station Acknowledgment Order Channel Assignment Message Continuous frames of all 000s Traffic Channel Preamble: Frames of 000s Access Channel

Paging Channel

Forward Traffic Channel

Base Station Acknowledgment Order Mobile Station Acknowledgment Order Service Connect Message Service Connect Complete Message The Call is now officially Established!

Reverse Traffic Channel

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 27

The Handoff Process

The handset pilot searcher notices energy from another sector or BTS, meeting any of these criteria: Neighbor or Remaining Pilot Ec/Io stronger than T_Add Candidate Pilot just got T_Comp better than an ac tive An Active Pilot stayed below T_DROP for T_TDROP time
BTS

Pilot Strength Measurement Message Base Station Acknowledgment Order

Forward Traffic Channel

Selector arranges channel elements/Walsh codes in requested sectors and begins using them, too.

Extended Handoff Direction Message Mobile Station Acknowledgment Order

Handset verifies which assigned PNs it can now hear.

Reverse Traffic Channel

Handoff Completion Message Base Station Acknowledgment Order Neighbor List Update Message Mobile Station Acknowledgment Order The new Handoff condition is now officially Established!
October, 2001 RF151 v1.0 (c) 2001 Scott Baxter 1 - 28

Normal Call Processing Normal Call Processing Event Templates Event Templates

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 29

Basic Actix Analyzer Post-Processing

Preparation: Obtain a current site database from your market's RF engineer Actix Cellrefs.txt format Collect mapinfo maps for your market Analyzing files in Actix Analyzer: Open a workspace Load the drive-test files; select the drives and devices to view Open a message browser window Plot desired indications on the map Plot events of interest on the map Investigate bad events Correlate map timeline, events, messages Match up the observed conditions with problem signatures Solve Problems: Take action to resolve obvious problems; refer less obvious problems to RF or switching personnel, as appropriate

October, 2001

RF151 v1.0 (c) 2001 Scott Baxter

1 - 30