Csci 565

Spring 2009
H.Reza

 Event-based
GUI

systems

Testing

Concepts, strategies, etc

The

Currency Conversion Program

To show unit testing for GUI

To show integration testing for GUI
To show system testing for GUI

 What

is GUI?

Universal client alternative to traditional

forms/commands based user interfaces

that accepts as input user-generated
and/or system-generated events from a
set of finite events and produces
predicable graphical outputs

GUIs are organized in hierarchical fashion and

Composed of objects:
Objects has a set of fixed properties having discrete

values
Menu

provide almost continuous availability of the various features of the

systems;

Buttons /keyboard shortcuts

enable the user to navigate and access the various functions of their
application.

User interacts and manipulate with objects using events (or

sequence of events)
Windows allow multiple windows to be displayed at the same
time
Windows provides functionality using fields in which text or numeric data
can be entered
Windows operate in two mode:
Modal (restrict the user's foucs and interactions to one window)
Modalless ( does not restrict monopolozes the GUI interactions)

 The

GUI events can be classified as

Restricted-focus events: open modal windows

E.g. Set Language

Unrestricted-focus events open modeless

windows
E.g., Replace in MS World

Termination Events : close modal windows

E.g., OK and Cancel

System- interaction events: interact with

underlying software to perform some action

E.g., Copy event used for coping objects to the clipboard

GUI testing issues

The event-driven nature of GUIs

presents the first serious testing difficulty because users can click on
any pixel on the screen and there so many of them
Unsolicited events
E.g., when the printer is out of papers
The infinite sequence of events (or permutations of events)
GUI can be represented by Object oriented paradigm
Hidden synchronization and dependencies (objects in the same windows
and different windows)
E.g., if a check box is set to true, a text box intended to accept a
numeric value elsewhere in the window may be made inactive or
invisible.
Many events in/Many events out
Testability (difficulty to specify, to execute and to analyze test cases)
event-based system +hidden interactions
Coverage criteria??? (memon et al uses events, and events
sequences)
Suffers from some of problems related to COTs (or precompiled
components)

 There

are serious legal issues related

to both design and testing of GUI
Legal issues
Privacy/security
E.g., access to the privileged/stored data

Safety/ reliability
E.g., GUI for Safety critical systems can affect life-ordeath decisions (functionality and ease of use)

Accessibility
Equal access for disabled users

 Focus

on errors to reduce the

scope of tests
Separation of concerns (divide
and conquer)
Use Test design techniques when
it is applicable (e.g., blackbox)

GUI errors include

Data validation
Incorrect field defaults
Mishandling of server process failures
Mandatory fields, not mandatory
Wrong fields retrieved by queries
Incorrect search criteria
Field order
Multiple database rows returned, single row expected
Currency of data on screens
Window object/DB field correspondence
Correct window modality?
Window system commands not available/dont work
Control state alignment with state of data in window?
Focus on objects needing it?
Menu options align with state of data or application mode?
Action of menu commands aligns with state of data in window
Synchronization of window object content
State of controls aligns with state of data in window?

 Testing Strategies
Functionality

include

Unit testing (Buttons)

Integrations Testing (synchronizations)
System Testing (functional behaviors according

to requirements or functional testing)

Non-Functionality(Quality)
Usability testing
Security testing

 Model-Based

requires:

Approach to test GUI

Identify components and Events

Select a hierarchal formal model (e.g., EFSM)

to represent GUI elements (components)

Intra vs. inter components
Use model to generate threats or event
sequences of various lengths corresponding to
visible & observable system outputs
Total number of events sequences increases when the
length increase
It might impractical to test for ALL possible event
sequences
be selective?

 Unit

and Integration testing

minimally needed
Main focus is on System Testing
Model?
Event-driven Petri nets (EDPNs)
Testing threads

 Event driven
Unit button level testing
Not much integration testing
System level testing: Models
UML Behavioral models
StateCharts
Finite State Machines

 Starts

testing (functional and structural)

of user-supplied code for buttons
Compute
Valid and invalid data (e.g., non-numerical, etc)
Clear
Quit

Two

approaches

Using Drivers (good)

Using GUI (system level for input/output

events)

 Not too much of Integration

Three possible options:

testing

Focus on all logic in one place and use the value

of the option buttons as conditions in IF tests

Use OO approach using methods invocation to
exchange the rate value
Use visual Basic
Event procedures for each option button uses
constant value for specific exchange rate
(e.g. Private Sub OptEU_click () exchange =1.35 End
sub)

 Heavily usage of System testing

Need event-driven model
Event-Driven PN (EDPN) to specify and

to generate scenarios/threads
Tables 19.1 and Table 19.2

The process to generate threads

(positive/negative scenarios) include
Map the elements of CCP to EDPN

Map input/out events to input /output events of EDPN

Map Data to EDPN data places
Map each Action to the transition in EDPN
Create EDPN for each ASF (tasks)
Compose the EDPNs
Use ASF graph

Generate a subset (or suitable?) of threads

 How

many threads are needed?

Coverage criteria?

Reasonable

coverage includes

Test ALL port inputs

Test All port outputs
Test All ASF (tasks or requirements)
Difficult to test each ASF
Test Threats
There are too many
Need to be selective

 Definition:

EDPN is tripartite-directed
graph (P, D, S, In, Out)
P: a set of port events
D: a set of data places
S: a set of transitions
In: a set of ordered Paris from (P D) S
Out: a set of ordered pairs from S (P D)

P,

D, S correspond to

Data, Input/output Events, Action,

Tuple

(p4, d2, p12,

p17,p21)

description

m1

(1,0,0,0,0)

S4 enabled; s4 fired

m2

(0,1,1,1,1)

Nothing enabled

m3
..
mn

Represent the snap shot of the

simulating EDPN

 Port

Input Events

P1: Enter US Dollar Amount

P2: Click on Brazil
P3: Click on Canada
P4: Click on European Union
P5: Click on Japan
P6: Click on Compute Button
P7: Click on Clear Button
P8: Click on Quit Button

 Output Events
P9: Display U.S. dollar amount
P10: Display Brazilian Reals
P11: Display Canadian dollars
P12 :Display European Community Euros
P13: Display Japanese Yen
P14: Display ellipsis
P15: Indicate Brazil
P16: Indicate Canada
P17: Indicate European Community
P18: Indicate Japan

 Output Event
P19: Reset selected country
P20: Reset Brazil
P21: Reset Canada
P22: Reset European Community
P23: Reset Japan
P24: Reset equivalent currency amount
P25: End application

 Atomic

System Functions (or actions)

maps to transition
S1: Store US Dollar amount
S2: Sense Click on Brazil
S3: Sense click on Canada
S4: Sense click on EU
S5: Sense click on Japan
S6: Sense click on clear button
S7: sense click on the Clear button
S8: Sense click on Quit button

 D1:
D2:

US dollar amount entered

country selected

 The

composition is not defined

Place fusions?
Looks like it
Transition fusions?

Consider a set of threads T = {T1, T2, T3, T4}

Scenario 1: 1) enter US amount, 2) select EU, 3) press

compute button, 4) press quit

T1 = {s1, s4, s6, s8}

Scenerio2: 1) User enters US$, 2) Click on Brazil, 3)

Clicks on Compute, Clicks on Clear button)

T2 = {s1, s2, s6, s7}

Scenario 3:
T3 = {s3, s1, s6, s7}

Scenario 4:

T4 = {s5, s1, s7, s8}

In general the set of T covers (minim set of usages

for CCP)
Every action (or S)
Every port input (every input data and input event)
Every port output (every output data and output event)

 Next-level user behavior

Common usage scenario
Converts amount to all 4 currencies,

clears and quits

T5 = [s1, s2, s6, s3, s6, s4, s6, s5, s6, s7,
s8]

 Unexpected

user behavior (negative

scenario)
Neg-Sencario1: User changes his/her mind

about currency
T6 = [s1, s2, s3, s4, s5, s6, s7, s8]
unexpected

behavior (negative scenario)

Neg-Scenario2:User toggles between 2

countries, then quits

T7 = [s1, s2, s3, s2, s3, s2, s3, s8]

 Unit

testing & integration testing not

important
System testing most complicated
test.
ASFs and EDPNs can help identify
and validate test cases
Valid usages (positive scenarios)
Invalid usages (negative scenarios)

How to use MBT to test GUI?

How to measure the benefit and cost of using MBT in

GUI?

Usability testing/evaluation: requires specified

requirements or formal models of GUI
Expert reviews
Heuristic evaluation
Guidelines review
Consistency inspection
Cognitive walkthrough
Formal usability inspection
Usability Testing and Laboratories
Surveys Instruments
Acceptance tests
Evaluation during Active Use

 Software Testing by P. Jorgensen

Model-Based Testing by Utting