Software Testing and Quality Assurance

Software Testing and Quality Assurance
Test Design
AHMED A. QAZZAZ OCT 2021

Index
• Test Design
• Test Case
• White Box Testing
• Branch Coverage
• Statement Coverage
• Mutation testing

Test Design
is a process that describes “how” testing should be done
A test design basically helps us to select a good set of tests from the total number of all possible tests for a
given system

Following are the typical design techniques in Dynamic testing:
1. Deriving test cases directly from a requirement specification or black box test design technique. The Techniques
include:
•Boundary Value Analysis (BVA)
•Equivalence Partitioning (EP)
•Decision Table Testing
•State Transition Diagrams
Use Case Testing

!2. Deriving test cases directly from the structure of a component or system (White box testing):
•Statement Coverage
•Branch Coverage
•Mutation Testing
!3. Deriving test cases based on tester's experience on similar systems or testers intuition:
•Error Guessing
•Exploratory Testing
!
•


!A test case is a document which consists of a set of conditions or actions which are performed on the software
application in order to verify the expected functionality of the feature.
•

Test Case Documentation:
Document contains a complete set of test cases for a system
- Along with other information about the testing process.
- Who does what; when; how; anticipated results;

Project Name:
Test case Id: Test case name:
Module Name: Test Priority:
Designed By: Designed Date:
Executed by: Execution Date:
Short Description:
Precondition:
Step Status (Pass/ Comments

Action Expected Result Actual Result
Fail)
1
2
Postcondition:

Test Case Document Details
• Project Name: Name of the project you want to test.
• Example: Airline reservation system
• Test case Id: String of letters and numbers that uniquely identify this test case document
• For example: c_01.1, c_01.1a, c_01.2,…


• Test Case Name: This field is the main way to identify a test case. It is a short, direct
description about what the test is intended for.
• You use this name to identify the test case and search for it if you have to.
• For Examples Create user, Login
• Or Track and process Invoiceable Expenses

• Module Name: Mention name of main module or sub module you want to write the test cases.
• For Examples
• Authentication
• Products Filter


• Test priority: Out of a large cluster of test cases in our hand, we need to scientifically decide
their priorities of execution based upon some rational, non-arbitrary, criteria.
• popular three-level priority categorization scheme is described as under
• High: Allocated to all tests that must be executed in any case.
• Medium: Allocated to the tests which can be executed, only when time permits.
• Low: Allocated to the tests, which even if not executed, will not cause big upsets.

• Test Designed By: test’s name.
• For example: Mohamed Ahmed
• Test Designed Date: Date when wrote

• For example: Sep13, 2020

• Test Executed By: Name of tester who executed this test. To be filled after test execution.
• For example: Mohamed Ahmed
• Test Execution Date: Date when test executed.

• For example: Oct 31, 2020


• Short Description: describes the purpose of a specific test.
• For example: For Test Case Description: To verify that Login name on login page must be
greater than 3 characters

• Preconditions: which describe the state of the software before the test case. It is the settings/
conditions required to execute the test case.
• For Example: Software should be install on system, login page should be available on
desktop.

• Step: describes the step sequence
• Action: Actions that describe the specific steps which make up the interaction.
It describes in a detailed sequence what have to be done in order to achieve the test case
results.
• For example: Enter login name less than 3 chars (i.e. “a”) and password and click Submit
button

 

• Expected Results which describe the expected state of the software after the test case is
executed. Describe the expected result in detail including message/error that should be
displayed on screen.
• For example:
• an error message “Login not less than 3 characters” must be displayed

• Actual Results which describe the actual state of the software after the test case is executed.
• The actual result of the test; to be filled after executing the test.


• Status : Does the case pass or fail.
• Comments: Any additional comments about this step.

• Post-condition: What should be the state of the system after executing this test case?
• For example: The user will login inside the app.

Example

White Box Testing
White Box Testing
• This testing approach will use the internal structure of the code to derive test data.
• also called, glass box testing, clear box testing, and structural testing.

White Box Testing

White Box Testing - Steps
• Understand the languages and tools used in development of the software
• Understand the Source Code
• Write test cases for every flow and coverage possibility
• Execute test cases.
• Analyze and record the results

Control Flow Testing
• structural testing strategy that uses the program’s control flow as a model
• Control-flow testing techniques are based on wisely selecting a set of test paths through the
code

• Test cases examples
Test case will be
{(x=3, y=2); (x=2, y=3)}


:

• in the previous code the number of possible test cases is 264, Assuming that integer is 32 bit
• Exhaustive testing is very hard.
• This {(x=3,y=2),(x=4,y=3),(x=5,y=1),{(x=100,y=99)} test cases set has more cases but it will not
detect the error.

How to Control Flow Testing

• Draw a graph for the code flow based on the source code. (control flow graph)
• Design the test cased to cover all elements in the graph

Control Flow Graph
• Control graph will consists of three primitives:
• decisions
• junctions
• process block

Decision
• is a program point at which the control can diverge, (e.g., if and case statements).

Junctions
• is a program point where the control flow can merge, (e.g., end if, end loop, goto label)

Process Block
• s a sequence of program statements uninterrupted by either decisions or junctions. (i.e.,
straight-line code)

Control Flow Graph
C
C C
S1 S2 S1
Sequential statement block
if C then S1else S2 if C then S1


Control Flow Graph
C
for loo
for i = 1 to n do s
S S1
while C do S1

p

Example

Example
B1
B1
B3 B2 B3 B2
B4
B4

Example.2
B1
B2
B3
B4
B5
B6
AHMED A. QAZZAZ
B7 OCT 2021

Example.2
B1 B2 B3 B4 B5 B6 B7

Example.3
B1
B2
B3
B4
B5
B6
B7
B8

Example.3 - Test cases
• lets try to call
• check2(3, 2)

check2(3,2) test case#1 B1
B2 x=
y=
d=
B3
s=0
B4
B5 B6
B7
B8
2

B2 x=
y=
d=
B3
s=0
B4
B5 B6
B7
B8
2

B2 x=
y=
d=
B3
s=0
B4
B5 B6
B7
B8
2

Example.3 - Test cases
• lets try to call
• check2(2,4)

B2 x=
y=
d=
B3
s=0
B4
B5 B6
B7
B8
4

B2 x=
y=
d=
B3
s=0
B4
B5 B6
B7
B8
4

B2 x=
y=
d=
B3
s=0
B4
B5 B6
B7
B8
6

B2 x=
y=
d=
B3
s=0
B4
B5 B6
B7
B8
6

B2 x=
y=
d=
B3
s = 11
B4
B5 B6
B7
B8
6

B2 x=
y=
d=
B3
s = 11
B4
B5 B6
B7
B8
6

B2 x=
y=
d=
B3
s = 11
B4
B5 B6
B7
B8
6

B2 x=
y=
d=
B3
s = 11
B4
B5 B6
B7
B8
6

B2 x=
y=
d=
B3
s = 11
B4
B5 B6
B7
B8
8

B2 x=
y=
d=
B3
s = 11
B4
B5 B6
B7
B8
8

B2 x=
y=
d=
B3
s = 27
B4
B5 B6
B7
B8
8

B2 x=
y=
d=
B3
s = 27
B4
B5 B6
B7
B8
8

B2 x=
y=
d=
B3
s = 27
B4
B5 B6
B7
B8
8

B2 x=
y=
d=
B3
s = 27
B4
B5 B6
B7
B8
8

Results
• Problem: node 6 is never executed, so we don’t have 100% branch coverage or statement
coverage
• Note d is 0 which means statement
• d= d+x-y;
• Code that has not been executed during testing is more likely to contain errors

Branch Coverage
• write test cases that cover all branches of decision nodes
• True and false branches of each IF
• The two branches corresponding to the condition of a loop
• All alternatives in a SWITCH

Branch coverage
• We need to cover all the nodes with branches 1,2,6 1 6 8
and 8 for both True and False branches
2 7
3 9
5 10

Branch Coverage
• We need to cover the paths through the nodes 1-3-5-6 in both cases for node 3
• abc
• adc
1 3 5 6
a b c

Branch Coverage
• abc
PATH NODE 3 INPUT(X) OUTPUT
• adc
negative
d ABC TRUE 2
integer -2
1 3 5 6
a b c
positive
ADC FALSE 2
integer 2

Statement Coverage
• write test cases that cover all nodes 1 6 8
• We need to cover all the nodes from 1, to 10
2 7
3 9
5 10

Statement Coverage
• We need to cover all nodes by checking that each link is covered at least once
• links are: a, b, c, and d
1 3 5 6
a b c

Statement Coverage
• abc Links Test Cases
Path
• adc a b c d Input(x) Output
abc ✓ ✓ ✓ -2 2
d adc ✓ ✓ ✓ 2 2
1 3 5 6
a b c

EXAMPLE
True True
a b c d e
1 3 4 5 6 2
h f
i g
l k j
10 9 8 7
True True

EXAMPLE
True True
a b c d e
1 3 4 5 6 2 Decision Nodes
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
-

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e
a, b, c, d, f

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e
a, b, c, d, f, j

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e
a, b, c, d, f, j, g, d, e

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e
a, b, c, d, f, j, g, d, e
a, b, c, d, f, m

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e
a, b, c, d, f, j, g, d, e
a, b, c, d, f, m, i, b, c, d, e

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e
a, b, c, d, f, j, g, d, e
a, b, c, d, f, m, i, b, c, d, e
a, b, h

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e
a, b, c, d, f, j, g, d, e
a, b, c, d, f, m, i, b, c, d, e
a, b, h, k, g, d, e

EXAMPLE
True True
a b c d e
i
h
g
f 4, 6, 7, and 9
l k j
10 9 8 7 Paths
True True
a, b, c, d, e
a, b, c, d, f, j, g, d, e
a, b, c, d, f, m, i, b, c, d, e
a, b, h, k, g, d, e
a, b, h, l, i, b, c, d, e

Results
Links Decision Nodes

Path
a b c d e f g h i j k l m 4 6 7 9
abcde ✓ ✓ ✓ ✓ ✓ T T
abcdfjgde ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ T F-T T
abcdfmibcde ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ T-T F-T F
abhkgde ✓ ✓ ✓ ✓ ✓ ✓ F T F
abhlibcde ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ F-T T T

Example 2

Example 2
Decision Nodes

Example 2
Other Nodes

1
2
3
4
5
6
7
8
9

EXAMPLE 1
True
a
2 b
3
True
c
4 d
m e
f g
5 6
h
i j
7 8
1
2 k
3 9
4
5 l
6
7
8
9

True
Links Decision Nodes

Path
a b c d e f g h i j k l m 1 3 5 7
m ✓ F
abcd ✓ ✓ ✓ ✓ T T
abefg ✓ ✓ ✓ ✓ ✓ T F T
abehij ✓ ✓ ✓ ✓ ✓ ✓ T F F T
AHMED A. QAZZAZ abehkl ✓ ✓ ✓ ✓ ✓ ✓ T F FOCT 2021
F

Test Caeses
Links Decision Nodes Test Cases

Path Input Output
a b c d e f g h i j k l m 1 3 5 7
isSocial email passwor
d
m ✓ F TRUE any value any value return;
empty
abcd ✓ ✓ ✓ ✓ T T FALSE “”/“a@b.c” “123”/“”
error
abefg ✓ ✓ ✓ ✓ ✓ T F T FALSE “abc” any value email error
abehij ✓ ✓ ✓ ✓ ✓ ✓ T F F T FALSE “ab@aa.com” “123” pass error
abehkl ✓ ✓
AHMED A. QAZZAZ ✓ ✓ ✓ ✓ T F F F FALSE “ab@aa.com“ “a12345A” request
OCT 2021

Example
• Classify triangles by the length of the side
Equilateral Isosceles Scalene


Example
1
2
5
6
7

Example
True
b d f h
a c e g i j
1 2 3 4 5 6 7
False

Example
Links Decision Nodes Test Cases
Path Input Output
a b c d e f g h i j 2 3 4 5
a b c
ab ✓ ✓ T 0 0 0 4
acd ✓ ✓ ✓ F T 1 1 3 4
acef ✓ ✓ ✓ ✓ F F T 2 2 2 1
acegh ✓ ✓ ✓ ✓ ✓ F F F T 2 2 3 2
acegij ✓ ✓ ✓ ✓ ✓ F F F F 2 3 4 3

Check this Mistake
• if we type a*b instead of a + b in the second condition by mistake or internally.
• by using the previous test cases we will not be able to detect this mistake
• So we need to add new test case such as (1,1,1) it will give invalid while the expected is
equilateral.

Mutation Test
• Mutation is a small change to a program
• Mutation test based on creating faults to script, and check it the test cases can detect these
faults or not.
• Randomly make changes to the script

Example
a = 6, b = 1 result = 1 result = 1
a = 1, b = 6 result = 1 result = 6

Mutation Test
• Focuses on measuring the adequacy of test cases
• Is NOT a testing strateg
• Should be used in conjunction with traditional testing techniques, not instead of them

y

Mutations
Process
Mutation Test
Mutation
Mutation TEST New data
Mutation
YES
Kill mutant when its behavior
differ from the original any
process n mutant Complete
at i o NO
u t alive ?
M

Mutation Score
• The mutation score for a set of test cases is
• mutationScore = 100 * D / (N - E)
• D : Dead mutants
• N : Number of mutants
• E : Number of Equivalent mutants
• When score is 100% the mutations are adequate.

Equivalent mutants
• Mutants that can not be killable

Software Testing and Quality Assurance - Chapter 04 - Test Design

Uploaded by

Copyright:

Available Formats

You might also like

Software Testing and Quality Assurance - Chapter 04 - Test Design

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Software Testing and Quality Assurance - Chapter 04 - Test Design

Uploaded by

Copyright:

Available Formats

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

Step Status (Pass/ Comments

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

Test Case Document Details

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

Test Case Document Details

AHMED A. QAZZAZ OCT 2021

• Test Designed Date: Date when wrote

AHMED A. QAZZAZ OCT 2021

• Test Execution Date: Date when test executed.

AHMED A. QAZZAZ OCT 2021

Test Case Document Details

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

Test Case Document Details

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

Test case will be

{(x=3, y=2); (x=2, y=3)}

AHMED A. QAZZAZ OCT 2021

How to Control Flow Testing

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

Sequential statement block

if C then S1else S2 if C then S1

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021

AHMED A. QAZZAZ OCT 2021