TEMS Pocket

Quick Report Flow and

NR21 Test Case MoP & Manual check guideline

Marselinus Magtal (NPO)

1 © 2023 Nokia
 Quick Report
Process Flow (1 of 2)
The Quick Report function already deploy for TEMS Pocket. This will help DT engineer to do verification on site whether the log that they
collected is PASS or NOT PASS. Please use the naming convention base on Workshop document provided in the workshop. As the quick report
need the naming of the log should be correct.

A site can have more than one sector, and one sector
consist of two location. Every time you finished all the
location within the sector, and finished all sector within
the site you MUST UPLOAD ALL PASSED LOG
USING QUICK REPORT

2 © 2023 Nokia
 Quick Report
Process Flow (2 of 2)
When you taking a log in a certain sector and location you can do it using 2 options below.

Option 1: Option 2:
You can do one test case at a time and You do all the test case first, then
then upload the result. Do the test upload it using upload script and see
case until it PASS and the continue to the quick report result. You redo test
next text case. case that not passed. Then upload it
again the PASS log and re-test log
Continue the process for all the test
after ALL IS PASS.
case one by one.
In the end after all the test case has
been done select all the pass log then
upload it using the upload script and
see the quick report result. You need
to make sure all is PASS.

3 © 2023 Nokia
 NR21 Test Case MoP & Manual Check Guidance
Test case on how to do and collect the log is available in the next slide.
Next slide also will contain how to do manual check in TEMS Pocket.
ATTENTION!!! If you already do all test and do verification to see the logs PASS or NOT PASS. You need to
compile all the PASS logs the upload it using upload script provided in the TEMS Pocket and see the quick report
result. ALL TEST MUST PASS.
Please follow also MoP provided by InfoVista team in below documents. This document also provide the naming
method used for conduct a test.

Nokia Singapore
Indoor Testing Workshop

4 © 2023 Nokia
 Attach Test (1 of 6)
Accessibility, Retainability, Call Setup Time
The attach test procedure will generate result for Attach Success Rate, Call Setup Success Rate, Drop Call, and Call Setup Latency. Due to
control function in TEMS Pocket is having problem the attach/detach process is done by airplane mode.
Problem in Samsung UE: If using airplane mode on/off with the below method Samsung UE will generate 5 additional RRC:RRCSetupRequest,
RRC:RRCSetup, RRC:RRCSetupComplete, and 5GMM:RegistrationRequest (total 10 messages each for 5x iteration).
Oppo UE: Oppo UE does not have problem like Samsung UE, recommend to use Oppo UE to do Attach test.
The Airplane Mode ON should be done before inactivity timer to trigger 5GMM:DeRegistrationRequest.
1 sample per sector per MNO, 1 sample 1 logfile (for NR21).

5 © 2023 Nokia
 Attach Test (2 of 6)
Clear Events and Layer 3 messages

1. Camp to NR21 (Lock to NR21) and Find location with

NR RSRP value better than -85dBm.
2. Set airplane mode ON
3. Go to messages menu ([Events] and [Layer 3
Messages]) and tap+hold in one of the event message
until popup menu show. Then tap [Clear list]
4. Start Log file recording
5. Turn flight mode OFF.
6. Run HTTP DL script (or external FDM App). Wait until
finished download.
7. Go back to Point 5, until 5 loops.

6 © 2023 Nokia
 Attach Test (3 of 6)
Checking the Log - Accessibility
Go to Layer 3 Tap and hold the Go to Events
Messages in Message “RRC Setup Request Message and wait
menu then find “RRC (UL-CCCH)”. After until “Script End
Setup Request (UL- popup menu showed Event” showed.
CCCH)” Message. tap “Show only this
There will 5
message type”. Tap
RRC:RRCSetupRequ
on the [magnifier
est messages for
glass/search icon]
Oppo phone. But for
twice to see the
Samsung phone there
result. Seems a bug in
will be 10 messages.
software after tap
“Show only this
message type” the
search result not
shown directly.

7 © 2023 Nokia
 Attach Test (4 of 6)
Checking the Log - Accessibility
Do the same with:
• “RRC Setup (DL-CCCH)” Message.
• “RRC Setup Complete (UL-DCCH)
• “Registration Request”
• “Registration Accept”
There will 5 messages for Oppo phone for
each of the message. But for Samsung
phone there will be 10 messages, except
“Registration Accept”, Samsung and Oppo
showing the same 5 messages.

8 © 2023 Nokia
 Attach Test (5 of 6)
Checking the Log - Retainability

Since we do RRC Make sure all the 5

release triggered by messages has following
UE (with PS Detach de-registration type
or Flight Mode) please “Switch Off”
check there is 5 “De-
registration Request
(UE Originating)”
messages.

9 © 2023 Nokia
 Attach Test (6 of 6)
Call Setup Time
Call setup time to be calculate from RRC setup to RRC setup complete
Find the “RRC Setup Calculate time interval between “RRC
(DL-CCCH)” Setup (DL-CCCH)” and “RRC Setup
message in Layer 3 Complete (UL-CCCH)” message as Call
Messages menu. Setup time. Tap [Down] button to search
for the next “RRC Setup (DL-CCCH”
messages (until all 5 messages). Average
the result.

10 © 2023 Nokia
 Latency (1 of 2)
• Ping IP 8.8.8.8
• Packet size 32byte
• Ping repetition 10 times
• KPI is based on the 10 samples average

Recommend to use script in TEMS Pocket, to see

the result on the spot (sample script attached on
slide XXX).

1. Turn on flight mode

2. Go to messages menu and tap+hold in one of
the event message until popup menu show.
Then tap [Clear list]
3. Turn OFF flight mode and camp to NR21
(Lock to NR21) and find location with NR
RSRP value better than -85dBm (to prevent
HO).
4. Start Recording and run a Ping Test script in
the TEMS Pocket

11 © 2023 Nokia
 Latency (2 of 2)

6. Wait until “Ping End Event” show in the events

messages panel. Then stop the recording.
7. Check the result by tap the “Ping End Event”.
Success Count should be = 10 and check the
Average RTT value.

12 © 2023 Nokia
 Download and Upload Test

Download and Upload will be done using SpeedTest APP.

And cannot done in TEMS Pocket. You will need Quick
Report function to see the result PASS or NOT PASS.

1. Camp to N21 with NR RSRP value better than -85dBm.

2. Start recording.
3. Run SpeedTest APP.
4. After finished stop recording.
5. Run script for uploading the log.
6. Wait and check the quick report result.

13 © 2023 Nokia
 Intra-Frequency Handovers (1 of 2)
• This test is to test the Intra-Frequency handover from
NR21 to NR21 with service continuous (PSDL).
• 1 sample per sector per MNO, 1 sample 1 logfile (for
NR21).

1. Turn on flight mode

2. Go to messages menu and tap+hold in one of the event
message until popup menu show. Then tap [Clear list]
3. Turn off flight mode and camp to NR21 (Lock to
NR21) and find location with NR RSRP value better
than -85dBm.
4. Run a Continuous HTTP DL Test script (or external
FDM App)
5. Make sure NR Throughput is running and Start
Recording.

14 © 2023 Nokia
 Inter-Frequency Handovers (2 of 2)
6. Move to area with signal weaker than -85dBm, observe in the
Events messages panel for message “NR Intra-frequency
Handovers”. You can do this by type “NR Intra-frequency
Handover” in the filter input then press [DOWN] repeatedly if
the messages goes to fast.
7. Wait 1-2 second after the HO then stop the test
8. Tap the message to see the one NR-ARFCN value for N21 and
one source PCI and target PCI. If correct, then stop the test.
9. See also the Handover Completion Time to see the Handover
Time values.

15 © 2023 Nokia
 Inter-Frequency Handovers (1 of 2)
• This test is to test the IFHO handover from NR21 to
NR35 with service continuous (PSDL).
• Event A2 threshold set example RSRP = -85dBm,
without CA service
• 1 sample per sector per MNO, 1 sample 1 logfile (for
NR21).

1. Turn on flight mode

2. Go to messages menu and tap+hold in one of the event
message until popup menu show. Then tap [Clear list]
3. Turn off flight mode and camp to NR21 (Lock to
NR21) and find location with NR RSRP value better
than -85dBm
4. Run a Continuous HTTP DL Test script (or external
FDM App), then release band lock.
5. Make sure NR Throughput is running and Start
Recording.

16 © 2023 Nokia
 Inter-Frequency Handovers (2 of 2)
6. Move to area with signal weaker than -85dBm, observe in the
Events messages panel for message “NR Inter-frequency
Handovers”. You can do this by type “NR Inter-frequency
Handover” in the filter input then press [DOWN] repeatedly if
the messages goes to fast.
7. Wait 1-2 second after the HO then stop the test
8. Tap the message to see the source (n1) and target (n78)
information. If correct, then stop the test.

17 © 2023 Nokia
 Inter-RAT Handovers (1 of 2)
• This test is to test the IRAT handover from 5G to 4G
with service continuous (PSDL).
• Event B2 threshold set example RSRP = -85dBm.
• 1 sample per sector per MNO, 1 sample 1 logfile (for
NR21).

1. Turn on flight mode

2. Go to messages menu and tap+hold in one of the event
message until popup menu show. Then tap [Clear list].
3. Turn off flight mode and camp to NR21 (Lock to NR21
+ LTE Bands) and find location with NR RSRP value
better than -85dBm.
4. Run a Continuous HTTP DL Test script (or external
FDM App)
5. Make sure NR Throughput is running and Start
Recording.

18 © 2023 Nokia
 Inter-RAT Handovers (2 of 2)
6. Move to area with signal weaker than -85dBm, observe in the
Events messages panel for message “Handover to LTE”. You can
do this by type “Handover to LTE” in the filter input then press
[DOWN] repeatedly if the messages goes to fast.
7. Wait 1-2 second after the HO then stop the test
8. Tap the message to see the source (n1) and target (LTE)
information. If correct, then stop the test.

19 © 2023 Nokia
 Script Samples

20 © 2023 Nokia
 XML Script Sample for TEMS Pocket

TEMS Pocket path in Android “Internal storage\pocket\private\scripts”

Samsung Galaxy S22+

Oppo Find X2 Pro
With Control Function Available

Ping_10x_32Bytes_8.8.8.8.xml Ping_10x_32Bytes_8.8.8.8_Oppo.xml

If you removed the control function in

TEMS Pocket you can use Oppo script
instead.

21 © 2023 Nokia