UMTS terminal testing: A

practical perspective

Olaf Bergengruen
May, 2003

Agere Systems - Optimay GmbH May, 2003 -1-

We will not survive without a
proper regression test system,
an Optimay engineer

Agere Systems - Optimay GmbH May, 2003 -2-

 Overview
Virtual testing
Logical view (what we want to test)
Why virtual testing
Requirements to the test system

3GPP approach
Test model
Test cases

Optimays implementation
GSM/GPRS virtual test system
GSM/GPRS/UMTS virtual test system

Agere Systems - Optimay GmbH May, 2003 -3-

 Test environment / logical view
(conformance testing, FTA)
GSM/GPRS tests
GSM 11.10
SAT tests
GSM 11.10-4

GERAN
simulation
Um
Cu
SIM
simulation
MS
UMTS tests
3GPP 34.123

UTRAN
Uu
simulation

Agere Systems - Optimay GmbH May, 2003 -4-

Test environment / logical view (2)
WAP
TCP/IP USB
test cases 802.11
Bluteooth
simulation

GERAN
simulation
Um
WAP Cu
tcp/ip
SIM
MS simulation

MMI

UTRAN
Uu simulation

simulation

Agere Systems - Optimay GmbH May, 2003 -5-

 Why virtual test environment

We need a SW only environment (no specific HW)

HW is expensive or does not exist
HW debugging is difficult and time consuming
SW developers dont want to be bothered with
HW issues or dont have the specific skills
We need to be much faster than real time
Developers want to use standard tools (compilers,
editors, debuggers)

Agere Systems - Optimay GmbH May, 2003 -6-

 Requirements to the virtual test system

Test system shall be much faster than real

time
Support the design of test scenarios at a high
level of abstraction
All test suites need to run over night:
GSM/GPRS tests (GSM 11.10), UMTS test
(3GPP 34.123), STK tests (GSM 11.10-4),
MMI test cases and possibly other suites
(TCP/IP, WAP, ...)
Test system shall be available to anybody, at
anytime, on any PC

Agere Systems - Optimay GmbH May, 2003 -7-

GSM/GPRS test architecture

Agere Systems - Optimay GmbH May, 2003 -8-

GSM/GPRS test architecture

Optimay test scripts

(GSM/GPRS test cases)

MS
MMI
Phone Apps
control
SS GMM, MM, CC, etc

GRR RR
MAC/RLC L2

RLC/MAC L2 Logical L1
Logical L1 DSP emulation

L1 simulation
(Radio Block transfer)

Agere Systems - Optimay GmbH May, 2003 -9-

 Great approach: L1 simulated
interface

Basic communication with the Tester via three

functions: RxRadioBlock, TxRadioBlock,
TimerTick
These functions can be extended as needed to
simulate DSP code or other HW related
functionality
MS is the master of system ticks, about 100
times faster than real time
Tester and MS are tightly coupled which
allows debugging using standard tools

Agere Systems - Optimay GmbH May, 2003 - 10 -

 Debugging GMM code ...
static void GMMDoRoutingAreaUpdate ( GM_IE_UPDATE_TYPE RauType, BOOLEAN FirstAttempt )

{
MMRAUpdPending = FALSE;
GMRauType = RauType;
GMStopT3302( );
GMStopT3311( );
if ( ! FirstAttempt )
{
GMT3330Expiries = 0;
if ( GMActState == GMActDetachInit )
{
GMStopT3321( );
GMDetachPending = TRUE;

}
}
if ( GMTriggerAction == GM_GS_ATTACH_COMP )
{
/* Reinitialise the trigger action variable. */
GMTriggerAction = GM_NO_TRIGGER_ACTION;

/* Confirm the end of the ATTACH procedure to GS */

SEND( GSGMEstCnf ) ( );
} Agere Systems - Optimay GmbH May, 2003 - 11 -
 Highlights of our GSM/GPRS
virtual test system

Extremely efficient performance (around 1300

tests in 20 minutes)
Includes testing of DSP code
About 30 man-years development effort
(including test case development)

Agere Systems - Optimay GmbH May, 2003 - 12 -

 Drawbacks of our GSM/GPRS
simulation

Tester uses proprietary (in house) test script

languages and encoders which are difficult to
maintain
It is difficult to write test cases
Tester is not nicely layered as shown in the
previous slide, so it is difficult to extend and
maintain
Summarizing: the system can not be
extended to implement an UMTS System
Simulator

Agere Systems - Optimay GmbH May, 2003 - 13 -

 3GPP test model
(TS 34.123-3)

TTCN test scenarios

UE
SS MMI / E-MMI
control
Phone Apps

NAS (GMM, MM, CC, etc)

RRC- RRC RR / GRR

PDCP SNDCP
adapt. RLC
RLC LLC MAC/RLC
MAC L2
MAC RLC/MAC L2
PHY (UTRAN) L1 (GERAN) PHY L1

RF cable

Agere Systems - Optimay GmbH May, 2003 - 14 -

 Why we need a Test Model
Main reason for us: If we comply with the
3GPP Test Model we can use the TTCN test
cases developped here at ETSI / STF 160
In general:
Clear and stable interfaces to the SS enables
manufacturers to develop the test equipment
It reduces ambiguities within test case scenarios
It enables outsourcing of testing activities and SW re-
use
It provides a common consens or understanding of the
complete system to manufacturers and operators

Agere Systems - Optimay GmbH May, 2003 - 15 -

 A test case (1/2) ...

Configure SS
Configure PHY / L1
Configure MAC
Configure RLC
Schedule and send System Information Blocks
Test case preamble
Bring UE into initial state (e.g. CS and PS registration)
Perform Location Update procedure
Perform GPRS Attach procedure

Agere Systems - Optimay GmbH May, 2003 - 16 -

 A test case (2/2) ...

Test case body

Stimulate the UE, e.g.
Send X message to the UE, or
Change cell power levels, or
Trigger the UE via AT command to initiate a call
Verify responses
Match messages from the UE to expected values
Assign a verdict (PASS, FAIL, INCONC)
Test case postamble
Complete signalling to bring UE into stable state

Agere Systems - Optimay GmbH May, 2003 - 17 -

So, how to design a Virtual Tester
which matches the 3GPP model
and our requirements ?

Agere Systems - Optimay GmbH May, 2003 - 18 -

 Problems designing an overall
GSM/GPRS/UMTS Test Env.

The clean 3GPP model is not enough for a

complete MS test environment
We need to re-use tools, tracers, and most of
all test cases developed for GSM in the new
environment
We can not afford to develop the UMTS test
suite in house, we need to re-use the 3GPP
tests
The L1 and DSP simulation strategies for GSM
do not match those of UMTS

Agere Systems - Optimay GmbH May, 2003 - 19 -

 Current approach (a compromise)

Multi-threaded system: The proper MS, the

GSM SS, the UMTS SS and the SIM simulator
All communicating via shared buffers
The tick-master is the MS, when all jobs for
current frame are completed, it issues a tick
and all System Simulators prepare data (radio
blocks) for next frame.
Each SS will send a tick-ack to the MS when
it is ready for next frame
When all SSs are done: the MS L1 engine will
fetch or store a radio block

Agere Systems - Optimay GmbH May, 2003 - 20 -

GSM/GPRS/UMTS Test Environment
.bst, .asc files TTCN suite
(Optimay tests) (3GPP tests)

Thread 1
Thread 2

GERAN simulation UTRAN simulation

SNDCP/
LLC PDCP
RLC/ RLC
L2
MAC
Thread 0 MAC
L1 emulation PHY emulation
Um Uu
MS
Thread 4
Cu
Tracer
.sat files SIM
(Optimay) simulation
Thread 3

Agere Systems - Optimay GmbH May, 2003 - 21 -

 Summary

A Virtual Test Environment is an essential tool

for development
for quality assurance

3GPP test model

A rough overview of our implementation

Agere Systems - Optimay GmbH May, 2003 - 22 -

Thank you.

Agere Systems - Optimay GmbH May, 2003 - 23 -