SE_340 Software Quality Engineering

Spring 2024

Course Instructor:
Ms. Seemab Khan- Lecturer (Software Engineering )
Department of Creative Technologies
Air University - Islamabad
Seemab.khan@mail.au.edu.pk
Lecture 11

REGRESSION
TESTING
Regression Testing
 What is Regression Testing ?
● Testing done to verify that
– Code changes in software do not impact existing
functionality
– Goal is to make sure that product works fine as
previously
● with the newly added functionality
● or any change in the existing feature
● or once the bug fix is done
– Previously executed test cases are re-executed in
order to verify the impact of change.
 When to perform Regression Testing ?

● Regression testing is initiated when a programmer fixes any

bug or adds a new code for new functionality to the system

● It is a quality measure to check

– whether the new code complies with the old code
– that the unmodified code is not getting affected.
 Regression Testing

● To maintain effectiveness of regression test suite a tester

may
– Reduce the size of test suite
– Reduce the number of test case executions
– Reduce test case execution time
– Select a subset of test cases
– Prioritize test cases
 Regression Testing

● Testing changed software

– Unit Regression
– Retest all
– Selective regression testing
● Regression test suite
– Set of test cases that are to be re-executed
● Types of changes
– Defect fixes
– Enhancements
– Adaptations
– Perfective maintenance
 Unit Regression

● Unit Regression is done during the unit testing phase

– code is tested in isolation i.e.. any dependencies on the unit to
be tested are blocked
– the unit can be tested individually without any discrepancy
 Retest all
● Assumption:
– Changes could have introduced errors anywhere in
the code
– Expensive, prohibitive for large systems
– Reuse existing test suite
– Add new tests as needed
– Remove obsolete tests
 Selective regression testing
● Impact analysis
● Only code impacted by change needs to be retested
● Select tests that exercise such code
● Add new tests if needed
● Remove obsolete tests
 Regeneration strategy

● Often for defect fixes when unsure whether defect is really fixed
● Develop new tests
– Probe around defect fix
– Test affected functionality
● Comes in retest all, and selective regression testing flavors
 Regression testing process
● Identify changes
● Determine which tests will remain valid: T’
● Test modified software with T’
● If T’ does not meet test criterion, generate new test cases T’’
● Execute modified software with T ’’
 Regression testing process (2)

● T’:
– test whether modifications have broken anything
– Determined either by retest all, or selective strategy
● T’’:
– test whether modified code works
● Use scope appropriate criteria
 Classification of tests in T

● Reusable:
– does not test software modification
– Should produce same result
– Need not be rerun
● Retestable:
– Tests software modification
– Must be rerun
● Obsolete: no longer applies
Regression Testing
 Approaches
● Full new test cycle
– Redesign entire test plan
– Rebuild whole test suite
● • Minimum regression test
– Reuse test plan as much as possible
– Rerun minimum number of retestable test cases
– Generate minimum number of new test cases for
changes
 Approaches
● Expanded scope of regression testing
– Reuse part of test plan
– Rerun all retestable test cases
– Generate new test cases on the full scope of
changes
● Full reuse of existing test suite
● Reuse test plan and test suite
Practical regression testing
 Attributes of regression tests

● Safe:
– All tests that could possibly exhibit different outputs on
the modified software
– Is 100% inclusive
● Precision:
– Percentage of tests in a reduced test suite that cannot
reveal regression faults, and
– Are not selected for the reduced test suite
 Attributes of regression tests

● Efficiency:
● cost of identifying a reduced regression test suite
● Generality:
● range of application for the selection strategy
 Regression test patterns

● Retest changed code

– Class, cluster, subsystem scope
– Identify segment coverage by test case
– Use version control tool to identify changed segments
– Select tests covering changed/deleted segments
– Develop new tests for new segments
 Retest by profile

● Assumes operational profile

● Include critical use cases
● Include other use cases by frequency in operational
profile No impact analysis
● Better: determine impact on use case
 Retest risky use cases

● Suspicious use cases

– Use cases that depend on components, objects, middleware,
resources that are
● Unstable, unproven
● Have not been shown to work together before
● Implement complex business rules
● Are complex
● Were fault-prone during development
● Critical use cases
● Unsafe
 Retest within Firewall

● Firewall: set of components whose test cases will be

included in a regression test
● External interface, contract change
● Internal implementation change
 Regression Testing Research

● Regression Test case selection

● Test suite reduction
● Difference ?
 Regression Testing Research

● Regression Test case Selection

– Based on code modifications in SUT
– Selects test cases that are based on modifications between the
two versions
● Test suite reduction
– Relies on coverage information of (a version) of SUT
● Test case prioritization
– Discover faults as early as possible
– Focus on fault detection rather than on what has changed
specifically in the code
 State of art in Regression Testing

● Classification of techniques
– Mining and Machine learning approaches
– Model Based Testing
– Program Slicing
– Control Flow Graphs
– FireWall
 Practice Task

● You are a software testing student tasked with regression

testing a simple calculator application.
● The application allows users to perform basic arithmetic
operations such as addition, subtraction, multiplication, and
division.
● You have been provided with a set of test cases that cover the
existing functionalities of the calculator. Recently, a
modification was made to the application's code to optimize
the division operation.
● Your task is to verify that this modification has not introduced
any regressions in the existing functionalities.
 Practice Task

● Addition Test Cases:Test Case 1: Add two positive integers

(e.g., 5 + 3)
● Test Case 2: Add a positive integer and zero (e.g., 7 + 0)
● Test Case 3: Add a positive integer and a negative integer
(e.g., 10 + (-3))
● Test Case 4: Add two negative integers (e.g., -8 + (-4))
● Test Case 5: Add two large positive integers to check for
overflow (e.g., 99999999 + 99999999)
 Practice Task

● Subtraction Test Cases:

● Test Case 1: Subtract a positive integer from another positive
integer (e.g., 10 - 3)
● Test Case 2: Subtract zero from a positive integer (e.g., 7 - 0)
● Test Case 3: Subtract a negative integer from a positive integer
(e.g., 10 - (-3))
● Test Case 4: Subtract a positive integer from a negative integer
(e.g., -8 - 4)
● Test Case 5: Subtract two negative integers (e.g., -10 - (-5))
 Practice Task

● Multiplication Test Cases:

● Test Case 1: Multiply two positive integers (e.g., 6 * 4)
● Test Case 2: Multiply a positive integer by zero (e.g., 9 * 0)
● Test Case 3: Multiply a positive integer by a negative integer
(e.g., 7 * (-2))
● Test Case 4: Multiply two negative integers (e.g., -5 * (-3))
● Test Case 5: Multiply two large positive integers to check for
overflow (e.g., 9999999 * 9999999)
 Practice Task

● Division Test Cases:

● Test Case 1: Divide a positive integer by another positive
integer with no remainder (e.g., 8 / 2)
● Test Case 2: Divide a positive integer by one (e.g., 5 / 1)
● Test Case 3: Divide a negative integer by a positive integer
(e.g., -10 / 2)
● Test Case 4: Divide a positive integer by a negative integer
(e.g., 15 / (-3))
● Test Case 5: Divide by zero (expecting an error or infinity
result) (e.g., 10 / 0)