Functional Testing II

Dragos Dragan
QA Test Consultant
dragos.dragan@qualitance.com

© Copyright 2017 QUALITANCE.

The information contained herein is subject to change without notice.
 3. Black-Box Testing Techniques
Agenda
a) Equivalence partitioning
1. Test Classification b) Boundary value analysis
a) Functional / Non-Functional
c) Decision table testing
b) Black box / White box
d) State transition testing
2. Test Levels e) Use case testing
a) Unit testing
b) Integration testing
c) System testing 4. White-Box Techniques
d) Acceptance testing
a) Statement coverage

b) Branch/Decision coverage

c) Path coverage
Test Classification
Functional / Non-Functional
 Functional / Non-functional

 Test activities are focused on verifying a system based on a

specific reason/target/objective

– Testing the functionality

– Testing non-functional characteristics

– Reliability, usability, performance

Functional Testing
 Functional Testing

 Functions: “what” the system does

 Functions are described in
• Requirements specification
• Use cases
• Functional Specifications
• Folklore
 External behavior of the system: black box
 Use specification-based techniques to derive test conditions and test cases
Non-Functional Testing
 Non-Functional Testing

 Testing of system characteristics, ‘how’ the system works

 Non-Functional Testing

The “Performance” group

 Performance - Expected normal behavior

 Load testing - Behavior of system under increasing load

 Stress testing - Behavior of system under unreasonable load

 Non-Functional Testing

 Usability
Ease with which users can engage with the system

 Maintainability
Ability to introduce changes into the system

 Reliability
Ability of the system to perform required functions over time
 Non-Functional Testing

 Portability
Operation on different platforms

 Recovery
Recovery procedures on failure

 Instal-ability
Installation procedures
 Non-Functional Testing

 Security
Ensures that no unauthorized access can be gained to the system or it's
components

 Interoperability
Operation of system when interacting in various environments

 Accessibility
Focus on people with disabilities
Test Classification
Black-box and White-box
 Black-box Testing

 Black-box:

– the subject of a test is

treated as a unit with an
unknown internal structure

– inputs and outputs

 White-box Testing

 White-box:

– based on knowledge of the

internal structure of the
test subject
 Gray-box Testing

 Gray-box:

– This is where you'll test

your application from an
external and user-facing
perspective along with
focusing on the internals.

– There are different shades

of gray
Test Levels
 Black-box and white-box testing

 Stage/Level: indication of the focus of testing

 Typical test levels

– Unit Testing (development driven)

– Integration Testing

– System Testing

– (User/Business) Acceptance Testing

Remember the V-Model ?
Test Levels
Unit Testing
 Unit Testing

 Code is written in (isolated) parts / units

 happens prior to integration with other units
 Objective: isolate a unit and validate its correctness
 verifies:
– Code can be executed
– Code written meets its specification
 Typically performed by the developer who wrote the code
 Usually automated: Unit Testing Frameworks
 Defects found and fixed are usually NOT logged
 Unit Testing: Example

public class Math {

static public int add(int a, int b) {
return a + b;
}
}
import junit.framework.*;
public class TestMath extends TestCase {
public void testAdd() {
int num1 = 3;
int num2 = 2;
int total = 5;
int sum = 0;
sum = Math.add(num1, num2);
assertEquals(sum, total);
}
}
 Unit Testing: Best Practices

 Keep tests independent of each other

 Think black-box AND white-box
 Test the trivial cases too
 Cover boundary cases
 Test each feature once
 Provide negative tests
 Write tests to reproduce bugs
Test Levels
Integration Testing
 Integration Testing

 Integration strategies in Software Development

– Big-Bang integration

– Top-Down integration

– Bottom-Up integration

– Hybrid integration (Sandwich integration)

 Big Bang Integration

 All units are linked at once, resulting in a complete system

 Works only for very simple systems

 Otherwise regarded as a poor choice

 Disadvantages:

– Difficult to isolate errors

– Introduces the risk of discovering issues late in the project

 Top-Down Integration

 Top-Down Control Structure

Top Component (1)
 Integration order:
– 1,2
– 1,3 Stub(2) Stub(3)
– 2,4
– 2,5
– 3,6
Stub(4) Stub(5) Stub(6) Stub(7)
– 3,7
 Bottom-Up Integration

 Bottom-Up Control Structure

Driver (1)
 Integration order:
– 4,2
– 5,2 Driver(2) Driver(3)
– 6,3
– 7,3
– 2,1
Component(4) Component(5) Component(6) Component(7)
– 3,1
Test Levels
System Testing
 System Testing

 End-To-End functionality

 Focuses on behavior of the entire system/product in a

representative environment

 Targeted to reveal errors that are due to

– Interactions across the entire system

– Environmental issues
 System Testing

 Both functional and non-functional

 Based on written specification

Test Levels
Acceptance Testing
 Acceptance Testing

 Purpose:
– Provide users with confidence that system will function according
to their expectations

– Demonstrate system conformance to customer requirements

 No reference to the code

 Unlike system testing, acceptance testing should be

INDEPENDENT of any other testing carried out
Testing Techniques
Black-Box Techniques
 Black-Box Techniques

A) Equivalence partitioning

B) Boundary value analysis

C) Decision table testing

D) State transition testing

E) Use case testing

Black-Box Techniques
Equivalence partitioning
 Equivalence partitioning

 Inputs of a program can be chunked into groups of similar

inputs
– expected to exhibit similar behavior
– likely to be processed in the same way
 For each partition select one representative input
 For both valid and invalid input data
 Invalid inputs are important!
– Far more possible invalid inputs
– May have to limit test cases to invalid inputs that are most likely
 Equivalence partitioning: Example

Program that takes as a valid input an integer between 100 and 999

– Valid partition: 100-999

– Non-valid: <100 , >999 , Real , non-numeric

 Equivalence partitioning: Example

Test Case 1 2 3 4 5

Input 150 50 5000 48.7 q

Partition 100 ≤ i ≤ i = real i = non-

i < 100 i > 999
tested 999 number numeric

Exp.
OK NOK NOK NOK NOK
Output
 Equivalence partitioning: Example 2

Program that takes as a valid input 20 alphabetic char names

• Valid partition: strings up to 20 alphabetic char

• Non-valid:
 20+ char

 contain non-alphabetic chars

 null
Black-Box Techniques
Boundary value analysis
 Boundary value analysis

 Behavior at the edge of each equivalence partition is more

likely to be incorrect

 Maximum and minimum values of a partition

 Both valid and invalid boundary values

 Boundary value analysis can be applied at all test levels

 Boundary value analysis : Example 1

• Water boiling point

• Boundary 100 C

• Boundary values:

• 100 C, 101 C or

• 100.0 C, 100.1 C
 Boundary value analysis: Example 2

• Exam pass
• Pass limit: 40%

• Merit limit: 60%

• Distinction limit: 80%

• Boudary values:
• 39%, 40%, for pass

• 59%, 60%, for merit

• 79%, 80%, for distinction

Black-Box Techniques
Decision table testing
 Decision table testing

Useful in systems where combinations of input conditions

produce actions
A way to capture system requirements that contain
logical conditions
From the model (specification) conditions and actions of
the system are identified
 Input conditions and actions are most often stated boolean
 Decision table testing

 The decision table rows

– contain the triggering conditions
– the resulting actions for each combination of conditions

 The decision table columns

– rule that defines a unique combination of conditions that
result in the execution of the actions associated with that rule
 Decision table testing

• When Condition1 and Condition2 and Condition3, then just Action1

• When Condition2 and Condition3 but not Condition1, then just Action2
• When Condition1 and Condition2 but not Condition3, then Action1&2

Rule 1 Rule 2 Rule 3

Condition 1 T F T
Condition 2 T T T
Condition 3 T T F
Action 1 Y N Y
Action 2 N Y Y
 Decision table testing

If you are a new customer and you want to open a credit card
account then there are three conditions:
•first you will get a 15% discount on all your purchases today,
•second if you are an existing customer and you hold a loyalty card, you get a 10%
discount
•third if you have a coupon, you can get 20% off today (but it can’t be used with the ‘new
customer’ discount).

Discount amounts are added, if applicable.

 Decision table testing

Conditions Rule1 Rule2 Rule3 Rule4 Rule5 Rule6 Rule7 Rule8

New customer (15%) T T T T F F F F

Existing customer T T F F T T F F
Loyalty card (10%)

Coupon (20%) T F T F T F T F

Actions
Discount 15% - - - YES - - - -
Discount 10% - - - - YES YES - -
Discount 20% - - YES - YES - YES -
Total Discount (%) x x 20 15 30 10 20 0
Black-Box Techniques
State transition testing
 State transition testing

 Useful in systems in which behavior depends on current state and

incoming events

 Create state transition diagrams

 Derive state tables

 Use the state tables to derive test cases

– cover a typical sequence of states

– cover every state

– exercise every transition

 State transition diagrams

 Transitions: changes of system State

 Transitions: triggered by Events

 Events: an input to the system

 Events
– Can generate output

– Can trigger Transitions – changes of State

 State transition diagrams

A B C D E
S1 S2 - - - S5
S2 - - - - S3 / X
S3 - - S4 - -
S4 - - - S2 -
S5 - S1 - - -
 State transition diagrams

Which transitions would be valid?

1. ABCDE

2. FEABC

3. ABCEF

4. EFADC
Black-Box Techniques
Use Case Testing
 Use cases

 description of a system's behavior as it responds to

a request that originates from outside of that system

 describe interactions between actors, including

users and the system, which produce a result of
value to a user

 describe the “process flows” through a system

based on its intended use
 Use case testing

 Write a test case to represent every use case

 Most useful in uncovering defects in the process

flows during real-world use of the system

 Useful for designing acceptance tests

 Uncover integration defects

 Use case testing: Example

ATM authentication workflow:

 Insert card
 If card is valid: type tour PIN code
 If PIN is valid: user is redirected to ATM menu

Exceptions:
• If card is invalid: Error message and card is rejected
• If PIN is invalid: Error message, user can re-type the PIN
• If PIN is typed invalid 3 times: Error message and ATM eat card
 Use case testing: Example
Main Step Description
Success
Scenario 1 A: Inserts card

A: Actor
2 S: Validates card and asks for PIN
S: System
3 A: Enters PIN

4 S: Validates PIN

5 S: Allows access to ATM menu

Exceptions 2a Card not valid.

S: Display message and reject card
4a PIN not valid.
S: Display message and ask for retry (twice)
4b PIN invalid 3 times.
S: Eat card and exit.
Testing Techniques
White-Box Techniques
 White-Box Techniques

 Statement Coverage

 Branch/Decision Coverage

 Path Coverage
White-Box Techniques
Statement coverage
Statement Coverage

 test cases are designed and Read P

executed in such a way that every Read Q
statement of the code is executed at IF P+Q > 100 THEN
Print “Large”
least once.
ENDIF
If P > 50 THEN
 to calculate Statement Coverage, Print “P Large”
find out the shortest number of paths ENDIF
following which all the nodes will be
covered.
White-Box Techniques
Branch / Decision Coverage
Branch/Decision Coverage

 requires enough test cases such that Read P

each condition in a decision takes on all Read Q
possible outcomes at least once, and each IF P+Q > 100 THEN
point of entry to a program or subroutine is Print “Large”
invoked at least once. ENDIF
If P > 50 THEN
 every branch (decision) taken each way, Print “P Large”
true and false. ENDIF

 to calculate Branch Coverage, find out

the minimum number of paths which will
ensure covering of all the edges.
White-Box Techniques
Path Coverage
Path Coverage

 Test cases are designed and executed in Read P

such a way that every path is executed at Read Q
least once. IF P+Q > 100 THEN
Print “Large”
 All possible control paths taken, ENDIF
including all loop paths taken zero, once, If P > 50 THEN
and multiple (ideally, maximum) items in Print “P Large”
path coverage technique. ENDIF

 Path Coverage ensures covering of all

the paths from start to end.
 White-Box Techniques

 100% Path coverage will imply 100% Statement coverage

 100% Decision coverage will imply 100% Statement coverage
 100% Path coverage will imply 100% Decision coverage
 Branch coverage and Decision coverage are same.
Questions and Answers
 The end...
Thank you for your time!
Laboratory
Black-Box Techniques
 Equivalence partitioning: Exercise 1

A mail order company charges $2.95 postage for deliveries if

the package weighs less than 2 kg, $3.95 if the package weighs
2 kg or more but less than 5 kg, and $5 for packages weighing 5
kg or more. Generate a set of test cases using equivalence
partitioning.
 Equivalence partitioning: Exercise 2

Suppose you have a bank account that offers variable interest rates:
• 0.5% for the first $1,000 credit;
• 1% for the next $1,000;
• 1.5% for the rest.
If you wanted to check that the bank was handling your account
correctly what valid input partitions might you use?
 Equivalence partitioning: Exercise 3

A mail-order company selling flower seeds charges $3.95 for

postage and packing on all orders up to $20 value and $4.95
for orders above $20 value and up to $40 value. For orders
above $40 value there is no charge for postage and packing.
If you were using equivalence partitioning to prepare test
cases for the postage and packing charges what valid
partitions would you define? What about non-valid partitions?
 Equivalence partitioning: Exercise 4

Consider a component, generate_grading, with the following specification:

The component is passed an exam mark (out of 75) and a coursework (c/w) mark (out of 25), from
which it generates a grade for the course in the range 'A' to 'D'. The grade is calculated from
the overall mark which is calculated as the sum of the exam and c/w marks, as follows:
– greater than or equal to 70 - 'A'
– greater than or equal to 50, but less than 70 - 'B'
– greater than or equal to 30, but less than 50 - 'C'
– less than 30 - 'D'
Where a mark is outside its expected range then a fault message ('FM') is generated. All inputs are
passed as integers.
 Boundary value analysis: Exercise 1

Consider a component, generate_grading, with the following specification:

The component is passed an exam mark (out of 75) and a coursework (c/w) mark (out of 25), from
which it generates a grade for the course in the range 'A' to 'D'. The grade is calculated from
the overall mark which is calculated as the sum of the exam and c/w marks, as follows:
– greater than or equal to 70 - 'A'
– greater than or equal to 50, but less than 70 - 'B'
– greater than or equal to 30, but less than 50 - 'C'
– less than 30 - 'D'
Where a mark is outside its expected range then a fault message ('FM') is generated. All inputs are
passed as integers.
 Boundary value analysis: Exercise 2

A program calculates Christmas bonuses for employees

depending on the affiliation to the company:
•More than 3 years = 30% bonus

•More than 5 years = 50% bonus

•More than 8 years = 80% bonus

 Boundary value analysis: Exercise 3

In an flight reservation application, we test the Tickets field. If

a value between 1-10 is entered the flight is considered
booked. If a value between 11-99 is entered, the application
displays an error message (“Only ten tickets can be ordered at
a time”). When entering above 100, the field defaults to a two
digits number and when entering values below 1, the field
default to 1.
 Boundary value analysis: Exercise 4

A store offers different discounts depending on the purchases

made by the individual. In order to test the software that
calculates the discounts, we can identify the ranges of purchase
values that earn the different discounts. For example, if a
purchase is in the range of $1 up to $50 has no discounts, a
purchase over $50 and up to $200 has a 5% discount, and
purchases of $201 and up to $500 have a 10% discounts, and
purchases of $501 and above have a 15% discounts.
 Decision table testing: Exercise 1

In order to log in to his email account, the user has to provide a

valid email account and its associated password. If the user
provides invalid credentials, he should remain on the current
page (Login page) and an error message should be displayed. If
the user provides valid credentials he should be taken to the
Inbox page.
 Decision table testing: Exercise 2

He/she wants to borrow a book. The library management

system has three conditions: first of all it checks whether the
user is registered or not, second whether there are no
outstanding fees, and last if the user is under the borrow limit
(5 books).
 Decision table testing: Exercise 3

 A supermarket has a loyalty scheme that is offered to all customers.

 Loyalty card holders enjoy the benefits of either additional discounts on
all purchases if they spend more than $100 or the acquisition of loyalty
points, which can be converted into vouchers for the supermarket or to
equivalent points in schemes run by partners.
 Customers without a loyalty card receive an additional discount only if
they spend more than $100 on any one visit to the store, otherwise only
the special offers offered to all customers apply
 Decision table testing: Exercise 4

 Mutual insurance company

A mutual insurance company has decided to float its shares on the stock
exchange and is offering its members rewards for their past custom at the
time of notation.
- Anyone with a current policy will benefit provided it is a “with-profits”
policy and they have held it since 2001. Those who meet these criteria can
opt for either a cash payment or an allocation of shares in the new
company;
- those who have held a qualifying policy for less than the required time
will be eligible for a cash payment but not for shares.
 State transition testing: Exercise 1

Story: hill-walker’s watch

2 modes:
- Time
- Press Set -> transition to Set Hrs
- Press Set -> inc Hrs
- Press Mode -> Set Mins
- Press Set -> inc Mins
- Press Mode -> transition to Time Mode
- Altimeter
 State transition testing: fitness wristband
• When it is worn the default screen is Time where user can see actual time clock, if he clicks the button the
Date will be displayed and after 3 seconds time is displayed again.
• If the user clicks the button while Date is on screen he will be redirected to Steps view, steps are
incremented with every step.
• Another push will redirect user to Remaining steps screen. Remaining steps are decremented while
walking.
• If the user pushes again the button he will be redirected to Time view.
• From any view if the user hold the button for 5 seconds, wristband will move to sync state in order to sync
progress with pc/smartphone. After syncing or 30sec in case of any error user is redirected to previous
screen.
• From any view, if the user hold the button for 10 seconds, wristband will move to Sleep state in order to
monitor user’s sleep. User can move to previous screen by pushing the button for 5 seconds.
• From any view, if the user hold the button for 20 seconds wristband will move to Pairing mode in order to
pair with another pc/smartphone.
 White-Box Techniques
Given the following program fragment:

if day = Monday then statement a

else statement b
end if
if day = Tuesday then statement c
end if

What is the minimum number of test cases needed to achieve:

-100% statement coverage
-100% decision coverage
-100% path coverage
Questions and Answers
 The end...
Thank you for your time!