4.1 Software Testing

Software Testing
Classical Waterfall Model
CA3401 Software Engineering Dr. Ashish Kumar Sahu 2

Software Testing
 Executing a program with the intent of finding an error.
 To check if the system meets the requirements and be executed

successfully in the Intended environment.
 To check if the system is “ Fit for purpose”.
 To check if the system does what it is expected to do.

Some popular errors: Ariane 5

Some popular errors: Therac 25

Some popular errors: Y2K
 The Year 2000 problem (also known as the Y2K problem,
the millennium bug, and Y2K Bug) was a computer problem that
affected lots of computer systems.
 Early computer programs were made to handle only years containing
two digits (for example '92 instead of 1992).

Objective of Software Testing
 A good test case is one that has a probability of finding an as yet
undiscovered error.
 A successful test is one that uncovers a yet undiscovered error.
 A good test should neither be too simple nor too complex.
 A good test is not redundant.
 A good test should be “best of breed”.

Some Terminologies
 People make errors. A good synonym is mistake. This may be a syntax
error or misunderstanding of specifications. Sometimes, there are logical
errors.
 When developers make mistakes while coding, we call these mistakes
“bugs”.
 Fault : Fault is a mistake or error caused by misjudgment, carelessness and
forgetfulness.
 Failure : It is the inability of a system or component to perform required
function according to its specification.
 Or condition or state of not being able to meet intended objectives.
 A failure occurs when a fault executes. Deviation of the software from its
expected result.

Test case
• A test case is a specification of the inputs, execution conditions, testing procedure,

and expected results that define a single test to be executed to achieve a
particular software testing objective.
• Test suite is collection of test cases.
Test Case Expected

Test Case # Test Data Actual Result Pass/Fail
Description Result
Check
response Email: guru99
when valid @email.com Login should Login was
1 Pass
email and Password: be successful successful
password is lNf9^Oti7^2h
entered
Classical Waterfall Model

V-model
(Verification & Validation Model)
Verification Validation

VERIFICATION & VALIDATION
 Verification - typically involves reviews and meeting to evaluate
documents, plans, code, requirements, and specifications.
 This can be done with checklists, issues lists, walkthroughs, and
inspection meeting.
 Verification answers the question: Am I building the product right?
 Validation - typically involves actual testing and takes place after

verifications are completed.
 Validation answers the question: Am I building the right
product?
 Testing= Verification + Validation.

Levels of Testing
 Unit (Module) testing
 testing of a single module in an isolated environment
 Integration testing
 testing parts of the system by combining the modules
 To find the interface errors
 System testing
 testing of the system as a whole after the integration phase
 Acceptance testing
 testing the system as a whole to find out if it satisfies the requirements specifications

Unit Testing
 Done on individual units (Class or small cluster classes in OO System
and a function or modules in Procedural Systems)
 Test unit w.r.t unit specification
 Mostly done by developers/programmers requires stubs and drivers
 Further Popularized by the availability of Unit Testing Frameworks-
JUnit, NUnit

int add(int n1, int n2) int multiply(int n1, int n2)
{ {
int result; int result;
result = n1 + n2; result = n1 * n2;
return result; return result;
} }
int subtract(int n1, int n2)
{ int divide(int n1, int n2)
int result; {
result = n1 + n2; int result;
result = n1 / n2;
return result;
return result;
}
}
Unit Testing
How can you execute these individual modules.
Required main function
int main()
{
int num1, num2;
printf("Enter two numbers: ");

scanf("%d %d", &num1, &num2);
printf("%d + %d = %d\n", num1, num2, add(num1, num2));

printf("%d - %d = %d\n", num1, num2, subtract(num1, num2));
printf("%d * %d = %d\n", num1, num2, multiply(num1, num2));
printf("%d / %d = %d\n", num1, num2, divide(num1, num2));
return 0;
}
Driver is required to execute below modules

Drivers and Stubs
 Driver: A program that calls the interface procedures of the

module being tested and reports the results
 A driver simulates a module that calls the module currently being tested
 Stub: A program that has the same interface as a module that is

being used by the module being tested, but is simpler.
 A stub simulates a module called by the module currently being tested
 Mock objects: Create an object that mimics only the behavior needed
for testing
Integration Testing
 Integration testing: Integrated collection of modules tested as a group or partial
system
 To find interface errors.
 Integration plan specifies the order in which to combine modules into partial
systems
 Different approaches to integration testing

 Bottom-up
 Top-down
 Big-bang
 Sandwich

Integration Testing
int Get_Radius(int R)
{
printf("Input the Radius of a circle:\n");
scanf("%d",&R);
return R;
}
float ACIRCLE()
{
float R;
R=Get_Radius(R);
A=R*R*3.14;
return A;
}
int main(){
int R,L;
float Area;
Area=ACIRCLE(R );
printf(“%d”, Area);
return 0;
}
Module Structure
level 3 A B
• We assume that
the uses hierarchy is
a directed acyclic graph.
level 2 • If there are cycles merge
D them to a single module
C H
level 1
level 0 E F G I
• A uses C and D; B uses D; C uses E and F; D uses F, G, H and I; H uses I

• Modules A and B are at level 3; Module D is at level 2
Modules C and H are at level 1; Modules E, F, G, I are at level 0
• level 0 components do not use any other components
• level i components use at least one component on level i-1 and no
component at a level higher than i-1
Bottom-Up Integration
 Only terminal modules (i.e., the modules that do not call other modules) are
tested in isolation
 Modules at lower levels are tested using the previously tested higher level
modules
 Non-terminal modules are not tested in isolation
 Requires a module driver for each module to feed the test case input to the
interface of the module being tested
 However, stubs are not needed since we are starting with the terminal modules and
use already tested modules when testing modules in the lower levels

Bottom-up Integration
A B
C H
E F G I

Top-down Integration
 Only modules tested in isolation are the modules which are at the highest
level
 After a module is tested, the modules directly called by that module are
merged with the already tested module and the combination is tested
 Requires stub modules to simulate the functions of the missing modules that
may be called
 However, drivers are not needed since we are starting with the modules which is not
used by any other module and use already tested modules when testing modules in the
higher levels

Top-down Integration
A B
C
H
E F G I

Other Approaches to Integration
 Sandwich Integration
 Compromise between bottom-up and top-down testing
 Simultaneously begin bottom-up and top-down testing and meet at a
predetermined point in the middle
 Big Bang Integration

 Every module is unit tested in isolation
 After all of the modules are tested they are all integrated together at once and
tested
 No driver or stub is needed
 However, in this approach, it may be hard to isolate the bugs!

System Testing
 Process of attempting to demonstrate that the program or
system does not meet its original requirements and
objectives as stated in the requirements specification.
 Test cases derived from

 requirements specification
 system objectives, user documentation

Types of System Tests
 Functional Testing
 Testing Functionality to be delivered by the system
 Specialized Testing
 Volume testing
 to determine whether the program can handle the required volumes of data, requests,
etc.
 Load/Stress testing
 to identify peak load conditions at which the program will fail to handle required processing
loads within required time spans
 Usability (human factors) testing
 to identify discrepancies between the user interfaces of a product and the human engineering
requirements of its potential users.
 Security Testing
 to show that the program’s security requirements can be subverted

 Performance testing
 to determine whether the program meets its performance requirements (eg. response
times, throughput rates, etc.)
 Recovery testing
 to determine whether the system or program meets its requirements for recovery after
a failure
 Installability testing
 to identify ways in which the installation procedures lead to incorrect results
 Configuration Testing
 to determine whether the program operates properly when the software or hardware
is configured in a required manner
 Reliability/availability testing
 to determine whether the system meets its reliability and availability requirements

Acceptance Testing
 Acceptance Testing
 Performed by the customer or end user
 Compare the software to its initial requirements and needs of its end users
 Alpha testing
 conducted at the developer’s site (part of organization)
 tests conducted in a controlled environment
 Beta testing
 conducted at one or more User sites by the end user of the SW
 it is a “live” use of the SW in an environment over which the developer has no
control

Regression testing
 You should preserve all the test cases for a program
 During the maintenance phase, when a change is made to the program, the test
cases that have been saved are used to do regression testing
 figuring out if a change made to the program introduced any faults
 Regression testing is crucial during maintenance

 It is a good idea to automate regression testing so that all test cases are run after each
modification to the software
 When you find a bug in your program you should write a test case that exhibits the
bug
 Then using regression testing you can make sure that the old bugs do not reappear

Advantages of the V-Model
 This is a highly-disciplined model and Phases are completed one at a
time.
 Works well for smaller projects where requirements are very well
understood.
 Simple and easy to understand and use.
 Easy to manage due to the rigidity of the model. Each phase has
specific deliverables and a review process.

Disadvantages of the V-Model
 Not a good model for complex and object-oriented projects.
 Not suitable for the projects where requirements are at a moderate to
high risk of changing.
 Once an application is in the testing stage, it is difficult to go back and
change a functionality.
 No working software is produced until late during the life cycle.
 High risk and uncertainty.

Static Testing
 Static Testing, a software testing technique in which the software is
tested without executing the code.
 It has two parts.
 Review - Typically used to find and eliminate errors or ambiguities in
documents such as requirements, design, test cases, etc.
 Static analysis - The code written by developers are analyzed (usually by
tools) for structural defects that may lead to defects.

Dynamic Testing
 Involves interaction with the program while it runs.
 Dynamic testing involves testing the software for the input values and
output values are analyzed.

Basic Testing Strategies/Methods
 Function Versus Structure: Tests can be designed from a functional
or a structural point of view.
 In functional testing, the program or system is treated as a blackbox.
It is subjected to inputs, and its outputs are verified for conformance to
specified behaviour. Functional testing takes the user point of view-
bother about functionality and features and not the program's
implementation.
 Structural testing does look at the implementation details. Things
such as programming style, control method, source language, database
design, and coding details dominate structural testing.
Basic Testing Strategies
Black-box testing White-box testing
Tests that validate business Tests that validate internal
requirements (what the system program logic (control flow ,
is supposed to do) data structures, data flow)
Test cases are derived from Test cases are derived by
the requirements specification. examination of the internal
No knowledge of structure.
internal program structure is
used.
Also known as -- functional, Also known as -- structural or
data-driven, or Input/Output logic-driven testing
Testing.
Basic Testing Strategies
 Black box testing (Specification Based)
 Equivalence Class Partitioning
 Boundary Value Analysis
 Decision Table
 Cause-effect graphing
 White box testing (Program Based)
 Control Flow Based
 Data Flow based
 Mutation Testing

Equivalence Class Partitioning
 Partition the program input domain into equivalence classes

(according to the specifications)
 The rationale is that test of a representative value of each class is
equivalent to a test of any other value of the same class.
 identify valid as well as invalid equivalence classes
 One test case from each equivalence class

Example
 Example: input condition 0 <= x <= 100
 valid equivalence class : 0 <= x <= 100 (50)
 invalid equivalence classes : x < 0  -1, x > 100 101
 3 test cases

 Consider a program for determining the Previous date. Its
input is a triple of day, month and year with the values in the
range 1 <= month <= 12, 1<= day<= 31 & 1900<= year
<=2025.
 The possible outputs would be Previous date or invalid input
date. Identify the equivalence class test cases for output &
input domains.
Output domain equivalence class are:
O1={<D,M,Y>: Previous date if all are valid inputs}
O2={<D,M,Y>: Invalid date if any input makes the
date invalid}

 I1={month: 1 <= m <= 12}
 I2={month: m < 1}
 I3={month: m > 12}
 I4={day: 1 <= D <= 31}
 I5={day: D < 1}
 I6={day: D > 31}
 I7={year: 1900 <= Y <= 2025}
 I8={year: Y < 1900}
 I9={year: Y > 2025}

 Consider a simple program to classify a triangle. Its inputs is a
triple of positive integers (say x, y, z) and the date type for
input parameters ensures that these will be integers greater
than 0 and less than or equal to 100. The program output may
be one of the following words: [Scalene; Isosceles;
Equilateral; Not a triangle]
 Identify the equivalence class test cases for output & input
domains.
Output domain equivalence classes are:
O1={<x,y,z>: Equilateral triangle with sides x,y,z}
O2={<x,y,z>: Isosceles triangle with sides x,y,z}
O3={<x,y,z>: Scalene triangle with sides x,y,z}
O4={<x,y,z>: Not a triangle with sides x,y,z}

 Input domain based classes are:
 I1={x: x < 1}
 I2={x: x > 100}
 I3={x: 1 <= x <= 100}
 I4={y: y < 1}
 I5={y: y > 100}
 I6={y: 1 <= y <= 100}
 I7={z: z < 1}
 I8={z: z > 100}
 I9={z: 1 z 100}

 Also identify output equivalence classes, and write test cases to
generate the output equivalence classes.
 O1= Not a triangle, when any of the sides is greater than the sum of
the other
 O2 = Equilateral Triangle
 O 3= Isosceles Triangle
 O 4= Scalene Triangle

 Some inputs domain test cases can be obtained using the relationship
amongst x,y and z.
 I10={< x,y,z >: x = y = z}
 I11={< x,y,z >: x = y, x <= z}
 I12={< x,y,z >: x = z, x <= y}
 I13={< x,y,z >: y = z, x >=y}
 I14={< x,y,z >: x>= y, x>= z, y >=z}
 I15={< x,y,z >: x = y + z}
 I16={< x,y,z >: x > y +z}
 I17={< x,y,z >: y = x +z}
 I18={< x,y,z >: y > x + z}
 I19={< x,y,z >: z = x + y}
 I20={< x,y,z >: z > x +y}

Boundary Value Analysis
 The basis of Boundary Value Analysis (BVA) is testing the boundaries
at partitions.
 Design test cases that exercise values that lie at the boundaries of an
input equivalence class.
 Example: input condition 0 <= x <= max
Test for values : 0,1, x, max-1, max ( valid inputs)
: -1, max+1 (invalid inputs)

BVA: 4n+1 test cases

Robust BVA: 6n+1 test cases
CA3401 Software Engineering -1, 0 , 1, mid, max-1, max, max+1 Dr. Ashish Kumar Sahu 50
Worst BVA: 5n test cases

BVA:Example-1
 The boundary value analysis test cases for our
program with two inputs variables (x and y) that may
have any value from 100 to 300 are:
 (200,100),(200,101), (200,200), (200,299), (200,300),
(100,200), (101,200), (299,200) and (300,200).
 Thus, for a program of n variables, boundary value
analysis yield 4n + 1 test cases.

BVA:Example-2
 Consider a program for determining the Previous date. Its
input is a triple of day, month and year with the values in the
range 1 <= month <= 12, 1<= day<= 31 & 1900<= year
<=2025.
 The possible outputs would be Previous date or invalid input
date. Design the boundary value test cases.

BVA:Example-3
 Consider a simple program to classify a triangle. Its
inputs is a triple of positive integers (say x, y, z) and
the date type for input parameters ensures that these
will be integers greater than 0 and less than or equal to
100. The program output may be one of the following
words: [Scalene; Isosceles; Equilateral; Not a triangle]
 Design the boundary value test cases.

Decision Table
 Decision table showing the combinations of input
conditions that make an effect true.
 Test applications where the action is basis of multiple
input combinations.
 A decision table is a good way to deal with different
combination inputs with their associated outputs.

Decision Table
CONDITIONS 1st Input 2nd Input 3rd Input

Salaried? Y N Y
Monthly Income > 25000 N Y Y
IT Returns available? Y N Y
ACTIONS
Loan Eligibility N N Y

Conditions Rule 1 Rule 2 Rule 3 Rule 4
Username F T F T
Password F F T T
Output E E E H
In the above example,

•T – Correct username/password
•F – Wrong username/password
•E – Error message is displayed
•H – Home screen is displayed
Cause Effect Graphing Technique
 A technique that aids in selecting test cases for
 combinations of input conditions in a systematic way
Steps:
 1. Identify the causes (input conditions) and effects
(output conditions) of the program under test.
 2. For each effect, identify the causes that can produce
that effect. Draw a Cause-Effect Graph.
 3. Generate a test case for each combination of input
conditions that make some effect to be true.

Example
 Consider a program with the following:
 input conditions Output conditions
 C1: command is credit E1: print invalid command
 C2: command is debit E2: print invalid A/C
 C3: A/C is valid E3: print debit amount not valid
 C4: Transaction amount E4: debit A/Cnot valid
E5: credit A/C

Structural /White Box Testing
 A complementary approach to functional testing is called structural /
white box testing.
 It permits us to examine the internal structure of the program.
 Involves tests that concentrate on close examination of procedural
detail
 Logical paths through the software are tested
 Test cases exercise specific sets of conditions and loops

White Box Testing
 Control-Flow Testing
 Statement coverage
 Decision coverage
 Condition coverage
 Basis Path Testing
 Data Flow Testing

 All p-use
 All c-use
 All d-use
 All uses

White Box Testing
 The program structure used in structural testing is the CFG i.e. control flow
 Nodes represent statements

 Edges represent the flow
White Box Testing
 Statement coverage
 write enough test cases to execute every statement at least once
void function eval (int A, int B, int X )

{
if ( A > 1) and ( B = 0 )
then X = X / A;
if ( A = 2 ) or ( X > 1)
then X = X + 1;
}
Statement coverage test cases:
1) A = 2, B = 0, X = ? ( X can be assigned any value)

White Box Testing
 Decision coverage
 write test cases to exercise the true and false outcomes of every decision
void function eval (int A, int B, int X )

{
if ( A > 1) and ( B = 0 ) then Decision coverage test cases:
X = X / A; 1) A = 3, B = 0, X = 1 (acd)
if ( A = 2 ) or ( X > 1) then 2) A = 2, B = 1, X = ? (abe)
X = X + 1;
}

White Box Testing
 Condition coverage
 write test cases such that each condition in a decision takes on all possible
outcomes at least once
 may not always satisfy decision coverage
void function eval (int A, int B, int X ) Condition coverage test

{ cases must cover conditions
if ( A > 1) and ( B = 0 ) then A>1, A<=1, B=0, B !=0
X = X / A; A=2, A !=2, X >1, X<=1
if ( A = 2 ) or ( X > 1) then
X = X + 1;
}

Path Testing
 The objective of path testing is to ensure that the set of test cases is
such that each path through the program is executed at least once.
 The starting point for path testing is a program flow graph that shows
nodes representing program decisions and arcs representing the flow
of control.
 Statements with conditions are therefore nodes in the flow graph.

Path testing – Control Flow Graph
 White-box technique is based on the program flow graph (CFG)
Many paths between 1 (begin) and 6 (end)
1
1, 2, 5, 6
2 1, 2, 3, 4, 2, 6
1, 2, 3, 4, 2, 3, 4, 2, 5, 6
3
…
 Prepare test cases that will force the execution of each path in the basis set.
 Test case : ((inputs …) , (expected outputs …))

Flow Graph
 The control flow of a program can be analysed using a graphical
representation known as flow graph.
 The flow graph is a directed graph in which nodes are either entire
statements or fragments of a statement, and edges represents flow of
control.

Cyclomatic Complexity
 Cyclomatic Complexity determines the basis set of linearly
independent paths and tries to measure the complexity of a
program
 McCabe’s cyclomatic metric V(G) = e – n + 2P.
 For example, a flow graph shown in in Fig. 21 with entry node
‘a’ and exit node ‘f’.
 The value of cyclomatic complexity can be calculated as : V(G)
=9–6+2=5
 Here e = 9, n = 6 and P =1
 Path 1 : a c f
 Path 2 : a b e f
 Path 3 : a d c f
 Path 4 : a b e a c f or a b e a b e f
 Path 5 : a b e b e f
Basis Path Testing
 Two alternate methods are available for the complexity calculations.
 1. Cyclomatic complexity V(G) of a flow graph G is equal to the
number of predicate (decision) nodes plus one.
V(G)= decision nodes+1
Where is the number of predicate nodes contained in the flow graph
G.
 2. Cyclomatic complexity is equal to the number of regions of the flow
graph.

 Consider a flow graph given and calculate the cyclomatic complexity
by all three methods.
 1. V(G) = e – n + 2P
 = 13 – 10 + 2 = 5
 2. V(G) = + 1
= 4 + 1 = 5
 3. V(G) = number of regions
= 5

Graph Matrices
 A graph matrix is a square matrix with one row and one column for
every node in the graph. The size of the matrix (i.e., the number of
rows and columns) is equal to the number of nodes in the flow graph.
Some examples of graphs and associated matrices are shown in fig.

Graph Matrices

Graph Matrices

Graph Matrices

Graph Matrices

Graph Matrices
 A graph matrix is a square matrix with one row and one
column for every node in the graph. The size of the matrix
(i.e., the number of rows and columns) is equal to the number
of nodes in the flow graph. Some examples of graphs and
associated matrices are shown in fig. 24.

Graph Matrices

Graph Matrices

Loop Testing
 Loop testing is a white-box testing technique that
focuses exclusively on the validity of loop constructs.

Loop Testing-Simple Loop
 The following set of tests can be applied to simple loops, where n is
the maximum number of allowable passes through the loop.
 1. Skip the loop entirely. -0

 2. Only one pass through the loop. 1
 3. Two passes through the loop. 2
 4. m passes through the loop where m <n. 50
 5. n - 1, n, n+ 1 passes through the loop. 99,100,101

Loop Testing-Nested Loop
 The following set of tests can be applied to simple loops, where n is the
maximum number of allowable passes through the loop.
 1. Start at the innermost loop. Set all other loops to minimum values.
 2. Conduct simple loop tests for the innermost loop while holding the outer
loops at their minimum iteration parameter (e.g., loop counter) values. Add
other tests for out-of-range or excluded values.
 3. Work outward, conducting tests for the next loop, but keeping all other
outer loops at minimum values and other nested loops to “typical” values.
 4. Continue until all loops have been tested.
Loop Testing
 Concatenated loops. Concatenated loops can be tested using the
approach defined for simple loops, if each of the loops is independent
of the other.
 Unstructured loops. Whenever possible, this class of loops should be

redesigned to reflect the use of the structured programming constructs.

Data Flow Testing
 Data Flow Testing is a type of structural testing.
 Used to find the test paths of a program according to the locations of

definitions and uses of variables in the program.
 Data flow testing is used to analyze the flow of data in the program.
DEF(S) = {X | statement S contains the definition of X}

USE(S) = {X | statement S contains the use of X}
Data Flow Testing
1. read x, y;
2. if(x>y)
3. a = x+1
else
4. a = y-1
5. print a;
DEFINED AT USED AT
VARIABLE
NODE NODE
x 1 2, 3
du pairs-X- (1,2), (1,3)
y 1 2, 4
a 3, 4 5 du pairs-y- (1,2), (1,4)
du pairs-a- (3,5), (4,5)

Model based Testing
 Model based testing is a software testing technique where run time
behavior of software under test is checked against predictions made by
a model.

Generating Test cases from State Model
 Finite state machines are good at representing behavior
 System/Subsystem
 Object
 State models can be derived from formal specifications
 Some formal specifications are state based
 Ex: Z, Object-Z, AsmL, VDM, OCL
 Test sequences can easily be identified from the state
models

Test Case Generation

State based coverage criteria's
 State Coverage
 ctor – addDrink(2) - addQtr() – addQtr()
 All- Transition Coverage
 ctor – addDrink(2) - addQtr() – addQtr() – addQtr() – vend()
 ctor – addDrink(2) - addQtr() – addQtr() – retQtrs()
 ctor – addDrink(2) - addQtr() – retQtrs()
 ctor – addDrink(1) - addQtr() – addQtr() – vend()
 All- Transition Pair Coverage
 ctor – addDrink(2)
 ctor – addDrink(2) - addQtr() – addQtr() – vend – addDrink()
 ctor – addDrink(2) - addQtr()
State based coverage criteria's

Regression testing
 You should preserve all the test cases for a program
 During the maintenance phase, when a change is made to the program, the test
cases that have been saved are used to do regression testing
 figuring out if a change made to the program introduced any faults
 Regression testing is crucial during maintenance

 It is a good idea to automate regression testing so that all test cases are run after each
modification to the software
 When you find a bug in your program you should write a test case that exhibits the
bug
 Then using regression testing you can make sure that the old bugs do not reappear

Test Strategies for
Object-Oriented Software

Test Strategies for
Object-Oriented Software
 With object-oriented software, you can no longer test a single operation in
isolation (conventional thinking)
 Traditional top-down or bottom-up integration testing has little meaning
 Class testing for object-oriented software is the equivalent of unit testing
for conventional software
 Focuses on operations encapsulated by the class and the state behavior of the
class
 Drivers can be used
 To test operations at the lowest level and for testing whole groups of classes
 To replace the user interface so that tests of system functionality can be
conducted prior to implementation of the actual interface
 Stubs can be used
 In situations in which collaboration between classes is required but one or
more of the collaborating classes has not yet been fully implemented

Test Strategies for Object-Oriented Software
(continued)
 Two different object-oriented integration testing strategies
 Thread-based testing
 Integrates the set of classes required to respond to one input or event for the system
 Each thread is integrated and tested individually
 Regression testing is applied to ensure that no side effects occur
 Use-based testing
 First tests the independent classes that use very few, if any, server classes
 Then the next layer of classes, called dependent classes, are integrated
 This sequence of testing layer of dependent classes continues until the entire system is constructed

System Testing
 Process of attempting to demonstrate that the program or
system does not meet its original requirements and
objectives as stated in the requirements specification.
 Test cases derived from

 requirements specification
 system objectives, user documentation

 Functional Testing
 Testing Functionality to be delivered by the system
 Specialized Testing (Testing NFRs)
 Volume testing
 to determine whether the program can handle the required volumes of data, requests,
etc.
 Load/Stress testing
 to identify peak load conditions at which the program will fail to handle required processing
loads within required time spans
 Usability (human factors) testing
 to identify discrepancies between the user interfaces of a product and the human engineering
requirements of its potential users.
 Security Testing
 to show that the program’s security requirements can be subverted

Acceptance Testing
 Acceptance Testing
 Performed by the customer or end user
 Compare the software to its initial requirements and needs of its end users
 Alpha testing
 conducted at the developer’s site by a User
 tests conducted in a controlled environment
 Beta testing
 conducted at one or more User sites by the end user of the SW
 it is a “live” use of the SW in an environment over which the developer has no
control

Test case Template
Test Case ID Test Case Name Inputs Expected Actual Test Result
output output (Pass/Fail)
1 login Uid Login Login Pass

&pwd successful successf
ul
2 Login Uid&pw Login error Fail

d Succssful

Test case Template

The Art of Debugging

Debugging Process
 Systematic process of spotting and fixing the number of bugs,
or defects, in a piece of software so that the software is
behaving as expected.
 Debugging is harder for complex systems in particular when
various subsystems are tightly coupled as changes in one
system or interface may cause bugs to emerge in another.
 Debugging is a developer activity and effective debugging is
very important before testing begins to increase the quality of
the system.
 Debugging will not give confidence that the system meets its
requirements completely but testing gives confidence

Debugging Process

Debugging Process
 Debugging occurs as a consequence of successful testing
 The debugging process begins with the execution of a test case

 Results are assessed and the difference between expected and actual
performance is encountered
 This difference is a symptom of an underlying cause that lies hidden
 The debugging process attempts to match symptom with cause,

thereby leading to error correction

Why is Debugging so Difficult?
 The symptom and the cause may be geographically remote
 The symptom may disappear (temporarily) when another error is
corrected
 The symptom may actually be caused by nonerrors (e.g., round-off
accuracies)
 The symptom may be caused by human error that is not easily traced
(continued on next slide)

Why is Debugging so Difficult?
(continued)
 The symptom may be a result of timing problems, rather than processing
problems
 It may be difficult to accurately reproduce input conditions, such as
asynchronous real-time information
 The symptom may be intermittent such as in embedded systems involving
both hardware and software
 The symptom may be due to causes that are distributed across a number of
tasks running on different processes

Debugging Strategies
 Objective of debugging is to find and correct the cause of a software error
 Bugs are found by a combination of systematic evaluation, intuition, and luck
 Debugging methods and tools are not a substitute for careful evaluation based on a complete
design model and clear source code
 There are three main debugging strategies
 Brute force
 Backtracking
 Cause elimination
 Program slicing

Strategy #1: Brute Force
 Most commonly used and least efficient method
 The program loaded with print statement to print the intermediate values
will help to identify the error.
 Involves the use of memory dumps, run-time traces, and output statements
 Leads many times to wasted effort and time
1. int i;
1. int i; 2. int sum = 0;
2. int sum = 0; 3. int product = 1;
3. int product = 1; 4. int w = 7;
4. int w = 7; 5. for(i = 1; i < N; ++i) {
5. for(i = 1; i < N; ++i) { 6. sum = sum + i + w;
6. sum = sum + i + w; 7. Printf(“%d”, sum);
7. product = product * i; 8. product = product * i;
8. } 9. Printf(“%d”, product);
9. Printf(“%d”, sum); 10. }
CA3401 Software10.Printf(“%d”,
Engineering product); 11.Printf(“%d”, sum);
Dr. Ashish Kumar Sahu 109
Strategy #2: Backtracking
 Can be used successfully in small programs
 The method starts at the location where a symptom has been uncovered
 The source code is then traced backward (manually) until the location of the cause is found
 In large programs, the number of potential backward paths may become unmanageably large

Strategy #3: Cause Elimination
 Involves the use of induction or deduction and introduces the concept of
binary partitioning
 Induction (specific to general): Prove that a specific starting value is true; then
prove the general case is true
 Deduction (general to specific): Show that a specific conclusion follows from a
set of general premises
 Data related to the error occurrence are organized to isolate potential causes
 A cause hypothesis is devised, and the aforementioned data are used to
prove or disprove the hypothesis
 Alternatively, a list of all possible causes is developed, and tests are
conducted to eliminate each cause
 If initial tests indicate that a particular cause hypothesis shows promise, data
are refined in an attempt to isolate the bug

Strategy #4: Program Slicing
 program slicing is the computation of the set of
program statements, the program slice, that may
affect the values at some point of interest, referred to
as a slicing criterion.
 Program slicing can be used in debugging to locate
source of errors more easily. The static executable slice for criteria
1. int i; (write(sum), sum) is the new
program shown below.
2. int sum = 0;
3. int product = 1;
4. int w = 7; 1. int i;
5. for(i = 1; i < N; ++i) 2. int sum = 0;
{ 3. int w = 7;
6. sum = sum + i + w; 4. for(i = 1; i < N; ++i){
7. product = product * i; 5. sum = sum + i + w;
8. } 6.
9. write(sum); 7. }
CA3401 Software Engineering
10. write(product); 8. write(sum);
Coding Standards
 Coding standards are a set of guidelines, best practices,
programming styles and conventions that developers adhere to when
writing source code for a project.

Coding Practices
 Code readability is a universal subject in the world of computer
programming.
1. Commenting & Documentation
 Name of the module
 Date of module creation
 Author of the module
 Modification history
 Synopsis of the module about what the module does
 Different functions supported in the module along with their input output parameters
function foo() {
if ($maybe) {
do_it_now();
2. Consistent Indentation again();
} else {
abort_mission();
}
CA3401 Software Engineering finalize();
}
Coding Practices
3. Avoid Obvious Comments
4. Code Grouping
// get list of forums
$forums = array();
$r = mysql_query("SELECT id, name, description FROM forums");
while ($d = mysql_fetch_assoc($r)) {
$forums []= $d;
}
// load the templates

load_template('header');
load_template('forum_list',$forums);
load_template('footer');
5. Consistent Naming Scheme

6. DRY Principle
 DRY stands for Don't Repeat Yourself. Also known as DIE: Duplication is Evil.

Coding Practices
7. Avoid Deep Nesting
8. Limit Line Length
// bad
$my_email->set_from('test@email.com')->add_to('programming@gmail.com')->set_subject('Methods Chained')->set_body('Some long message')->send();
// good
$my_email
->set_from('test@email.com')
->add_to('programming@gmail.com')
->set_subject('Methods Chained')
->set_body('Some long message')
->send();
9. File and Folder Organization

10. Consistent Temporary Names
11. Capitalize SQL Special Words
12. Separation of Code and Data
13. Alternate Syntax Inside Templates
14. Object Oriented vs. Procedural
15. Read Open Source Code
Coding Practices- Refactoring
 Code refactoring is the process of restructuring
existing computer code—changing the factoring—
without changing its external behavior.



Refactoring
 The purposes of refactoring according to M. Fowler are stated in the
following:
 Refactoring Improves the Design of Software
 Refactoring Makes Software Easier to Understand
 Refactoring Helps Finding Bugs
 Refactoring Helps Programming Faster

Coding Standards
 Advantages of Coding Guidelines:
 Coding guidelines increase the efficiency of the software and reduces
the development time.
 Coding guidelines help in detecting errors in the early phases, so it
helps to reduce the extra cost incurred by the software project.
 If coding guidelines are maintained properly, then the software code
increases readability and understandability thus it reduces the
complexity of the code.
 It reduces the hidden cost for developing the software.

4.1 Software Testing

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

4.1 Software Testing

Uploaded by

Copyright:

Available Formats

Software Testing

Classical Waterfall Model

CA3401 Software Engineering Dr. Ashish Kumar Sahu 2

 To check if the system meets the requirements and be executed

 To check if the system is “ Fit for purpose”.

 To check if the system does what it is expected to do.

CA3401 Software Engineering Dr. Ashish Kumar Sahu 3

CA3401 Software Engineering Dr. Ashish Kumar Sahu 4

CA3401 Software Engineering Dr. Ashish Kumar Sahu 5

CA3401 Software Engineering Dr. Ashish Kumar Sahu 6

CA3401 Software Engineering Dr. Ashish Kumar Sahu 7

CA3401 Software Engineering Dr. Ashish Kumar Sahu 8

• A test case is a specification of the inputs, execution conditions, testing procedure,

Test Case Expected

CA3401 Software Engineering Dr. Ashish Kumar Sahu 10

CA3401 Software Engineering Dr. Ashish Kumar Sahu 11

 Validation - typically involves actual testing and takes place after

 Testing= Verification + Validation.

CA3401 Software Engineering Dr. Ashish Kumar Sahu 13

CA3401 Software Engineering Dr. Ashish Kumar Sahu 14

printf("Enter two numbers: ");

printf("%d + %d = %d\n", num1, num2, add(num1, num2));

Driver is required to execute below modules

 Driver: A program that calls the interface procedures of the

 Stub: A program that has the same interface as a module that is

 A stub simulates a module called by the module currently being tested

 To find interface errors.

 Different approaches to integration testing

CA3401 Software Engineering Dr. Ashish Kumar Sahu 19

• A uses C and D; B uses D; C uses E and F; D uses F, G, H and I; H uses I

 Non-terminal modules are not tested in isolation

CA3401 Software Engineering Dr. Ashish Kumar Sahu 22

CA3401 Software Engineering Dr. Ashish Kumar Sahu 23

CA3401 Software Engineering Dr. Ashish Kumar Sahu 24

CA3401 Software Engineering Dr. Ashish Kumar Sahu 25

 Big Bang Integration

CA3401 Software Engineering Dr. Ashish Kumar Sahu 26

 Test cases derived from

CA3401 Software Engineering Dr. Ashish Kumar Sahu 27

CA3401 Software Engineering Dr. Ashish Kumar Sahu 28

CA3401 Software Engineering Dr. Ashish Kumar Sahu 29

CA3401 Software Engineering Dr. Ashish Kumar Sahu 30

 Regression testing is crucial during maintenance

CA3401 Software Engineering Dr. Ashish Kumar Sahu 31

CA3401 Software Engineering Dr. Ashish Kumar Sahu 32

CA3401 Software Engineering Dr. Ashish Kumar Sahu 33

CA3401 Software Engineering Dr. Ashish Kumar Sahu 34

CA3401 Software Engineering Dr. Ashish Kumar Sahu 35

CA3401 Software Engineering Dr. Ashish Kumar Sahu 38

 Partition the program input domain into equivalence classes

CA3401 Software Engineering Dr. Ashish Kumar Sahu 39

CA3401 Software Engineering Dr. Ashish Kumar Sahu 40

CA3401 Software Engineering Dr. Ashish Kumar Sahu 42

CA3401 Software Engineering Dr. Ashish Kumar Sahu 43

CA3401 Software Engineering Dr. Ashish Kumar Sahu 44

CA3401 Software Engineering Dr. Ashish Kumar Sahu 45

CA3401 Software Engineering Dr. Ashish Kumar Sahu 46

CA3401 Software Engineering Dr. Ashish Kumar Sahu 47

CA3401 Software Engineering Dr. Ashish Kumar Sahu 48

CA3401 Software Engineering Dr. Ashish Kumar Sahu 49

CA3401 Software Engineering Dr. Ashish Kumar Sahu 51

CA3401 Software Engineering Dr. Ashish Kumar Sahu 52

CA3401 Software Engineering Dr. Ashish Kumar Sahu 53

CA3401 Software Engineering Dr. Ashish Kumar Sahu 54