Software Testing, Validation and

Verification
Lecture 3

Presented by:
Dr. Yasmine Afify
yasmine.afify@cis.asu.edu.eg
KINDLY
 Reference
https://www.istqb.org/certifications/certified-tester-foundation-level

The ISTQB® Certified Tester Foundation

Level (CTFL) certification provides
essential testing knowledge that can be
put to practical use and, very
importantly, explains the terminology
and concepts that are used worldwide in
the testing domain. CTFL is relevant
across software delivery approaches and
practices including Waterfall, Agile,
DevOps, and Continuous Delivery. CTFL
certification is recognized as a
prerequisite to all other ISTQB®
certifications where Foundation Level
is required. 3
4
5
Software Testing Types
Functional Testing
Non-functional Testing
Structural/White-box Testing
Change-related Testing (Re-testing and Regression)
Functional
Testing

7
1. Functional/Dynamic Testing Process Steps

8
 1.Functional/
Dynamic Testing
• Tests that evaluate functions
that the system should perform.
• For every test level, a suitable
test environment is required.
• In acceptance testing, for
example, a production-like test
environment is ideal, while in
component testing the
developers typically use their
own development
environment.

9
Component/unit/module testing
• Component testing is the process of testing individual components in
isolation.
• Carried out by the team developing the system.

• Components may be:

• Individual functions
• Methods within an object
• Classes
• Database modules
• Examples of typical defects and failures:
• Incorrect functionality (e.g., not as described in design specifications)
• Data flow problems
• Incorrect code and logic
10
Advantages of Unit Testing
• The earlier a problem is identified, the fewer compound
errors occur.
• Fixing problems early is usually cheaper than fixing them
later in development.
• Easier debugging processes.
• Developers can quickly make changes to the codebase.
• Developers can reuse code and migrate it to new projects.

11
Integration testing
• A level of software testing where individual units are combined and
tested as a group.
• Its purpose is to expose faults in the interaction between integrated
units.
• Take into account the broader context of the application: database,
external APIs, filesystem.
• Interfacing errors which were not detected in unit testing may appear.
• Timing problems (in real-time systems) can be detected (are not
detectable by unit testing).
• It is assumed that component tests on the individual objects within the
component have been completed.
• Unlike unit tests, which should run at build-time and should be very
quick to run, integration tests require a deployment step.
12

Integration Testing
• Sample Integration Test Cases for the following scenario:
Application has 3 modules; say ‘Login Page’, ‘Mailbox’ and ‘Delete
emails’ and each of them is integrated logically.
• Here do not concentrate much on the Login Page testing as it’s
already been done in Unit Testing. But check how it’s linked to the
Mail Box Page.
• Similarly Mail Box: Check its integration to the Delete Mails
Module.

14
Integration Testing

15
Integration Testing

16
Decomposition-based Integration
Big-bang Integration
• This approach waits until all the components arrive, and one
round of integration testing is done. This checking technique
states that the integration procedure will not proceed until all the
elements have been integrated.
• Big Bang Testing is an Integration testing approach in which all the
components or modules are integrated together at once and then
tested as a unit. This combined set of components is considered as
an entity while testing.
• Big-bang integration is ideal for a product where the interfaces are
stable with few number of defects.

17
18
Big-bang Integration Testing Workflow 1
• For the system, modules A and D are tested individually.
• Module C is combined with module B and F is combined with
module E to perform a group test.
• The first module is examined, followed by module B, then
module C, and finally module F.
• The modules are examined for their functionality and
compatibility with each other.
• The testing of modules and their integration into the system is
complemented by the testing of the entire system for
functionality.
19
20
Decomposition-based Integration
Big-bang Integration
Advantages:
• Convenient for small
systems.
• Simple. The amount
of planning needed
for this type of
testing is almost
non-existent.

21
Decomposition-based Integration
Big-bang Integration
Disadvantages:
• Fault Localization is difficult (it may prove challenging to detach all
relevant modules to investigate its underlying cause).
• Given the large number of interfaces that need to be tested in this
approach, some interfaces link to be tested could be missed easily.
• Since the Integration testing can commence only after “all” the
modules are designed, the testing team will have less time for
execution in testing phase.
• Since all modules are tested at once, high-risk critical modules are
not isolated and tested on priority. Peripheral modules which deal
with user interfaces are also not isolated and tested on priority.
• There is a higher chance of failure since all the modules are tested
together. 22
DO WE HAVE
TO WAIT??

23
24
Stubs and Drivers
• Stubs and Drivers are the dummy programs in Integration testing
used to facilitate the software testing activity.
• These programs act as a substitutes for the missing modules in the
testing.
• They do not implement the entire programming logic of the
software module; but they simulate data communication with the
calling module while testing.

• Stub: Is called by the Module under Test.

• Driver: Calls the Module to be tested.

25
Decomposition-based Integration
Bottom-up Integration Approach
• Bottom-up Integration Testing is a strategy in which the lower-level
modules are tested first. These tested modules are then further
used to facilitate the testing of higher-level modules.
• The process continues until all modules at top level are tested.

26
Decomposition-based Integration
Bottom-up Integration Approach
Advantages:
• Fault localization is easy
• No time is wasted waiting for all modules to be developed (unlike
Big-bang approach)

Disadvantages:
• Critical modules (at the top level of software architecture) which
control the flow of application are tested last and may be prone to
defects
• An early prototype is not possible
27
Decomposition-based Integration
Top-down Integration Approach
• Top-down Integration Testing is a method in which integration testing takes
place from top to bottom following the control flow of software system.
• The higher-level modules are tested first, and then lower-level modules are
tested and integrated to check software functionality.
• Stubs are used for testing if some modules are not ready.
• Top-down integration follows a breadth-first traversal of the functional
decomposition tree.

28
Decomposition-based Integration
Top-down Integration Approach
Advantages:
• Fault localization is easy.
• Critical modules are tested on priority; major design flaws could be
found and fixed first.
• Possibility to obtain an early prototype.

Disadvantages:
• Needs many stubs.
• The basic functionalities of the software are tested at the end of
the cycle.
29
Decomposition-based Integration
Sandwich/Hybrid Integration Approach
• Sandwich Testing is a strategy in
which top level modules are
tested with lower-level
modules, at the same time
lower modules are integrated
with top modules and tested as
a system.
• It is a combination of Top-down
and Bottom-up approaches
therefore it is called Hybrid
Integration Testing.
30
Decomposition-based Integration
Sandwich Integration Approach
Advantages:
• It can reduce the time and effort of testing the system as a whole,
as it can identify and fix the integration errors earlier and faster.
• It can facilitate the collaboration and feedback between the
developers and testers, as it can expose the dependencies and
interfaces of the modules.

Disadvantages:
• It requires a lot of coordination and communication between the
developers and testers who work on different levels of the system.
• It may involve creating and maintaining many stubs and drivers.
• It may not detect all the integration errors, especially those that
occur in the intermediate modules, which are tested last.
31
Call Graph-based Integration - Pairwise Integration

Four pairwise integration sessions are shown in Figure 7.7 32

Call Graph-based Integration
Pairwise Integration
• Instead of developing stubs and drivers, why not use the actual code?
• At first, this sounds like big-bang integration, but we restrict a session
to only a pair of units in the call graph.
• The end result is that we have one integration test session for each
edge in the call graph.
• This is not much of a reduction in sessions from either top-down or
bottom-up, but it is a drastic reduction in stub/driver development.

33
Call Graph-based Integration
Pairwise Integration
How many test
Example cases can you get
rid of?

34
Call Graph-based Integration
Pairwise Integration
Example

35
Call Graph-based Integration
Pairwise Integration
Example

36
Call Graph-based Integration
Neighbourhood Integration
• The neighbourhood of a node in a graph is the set of
nodes that are one edge away from the given node.
• In a directed graph, this includes all of the immediate
predecessor nodes and all of the immediate successor nodes.
• We can always compute the number of neighbours for a given
call graph. Each interior node will have one neighbourhood plus
one extra in case leaf nodes are connected directly to the root
node.
• Note that these correspond to the set of stubs and drivers of the
node. 37
 Call Graph-
based
Integration -
Neighbourhood
Integration

For node-16, neighbourhood nodes are 9, 10, and 12

nodes as successors and node-1 as predecessor node.
39
System testing
• System testing focuses on a complete, integrated system to
evaluate compliance with specified requirements.
• Tests are made on characteristics that are only present when the
entire system is run.
• The purpose of a system test is to evaluate the end-to-end
system specifications.
• System testing is the only testing phase that tests both
functional and non-functional aspects (quality factors) of the
product.
• Test product behaviour in a complete and realistic environment.
The test environment should ideally correspond to the final
production environment.
• Independent testers typically carry out system testing. 40
System testing
Typical test objects for system testing include:
• Applications
• Hardware/software systems
• Operating systems
• System under test (SUT)
• System configuration and configuration data

Examples of typical defects and failures for system testing include:

• Incorrect or unexpected system functional or non-functional behaviour
• Failure to properly and completely carry out end-to-end functional tasks
• Failure of the system to work properly in the production environment(s)
• Failure of the system to work as described in system and user manuals
41
42
Release testing
• Release testing is the process of testing a particular release of a
system that is intended for use outside of development team.
• The primary goal of the release testing process is to convince
the supplier of the system that it is good enough for use.
• Release testing has to show that the system delivers its
specified functionality, performance and dependability, and
that it does not fail during normal use.
• Release testing is usually a black-box testing process where
tests are only derived from the system specification.

43
Release testing
• Release testing is a form of system testing.
• Important differences:
• A separate team that has not been involved in the system
development, should be responsible for release testing.
• System testing by the development team should focus on
discovering bugs in the system (defect testing). However, the
objective of release testing is to check that the system meets
its requirements and is good enough for external use
(validation testing).

44
Smoke Testing

45
Smoke Testing
• A type of software testing that comprises of a non-
exhaustive set of tests that aim at ensuring that the most
critical/important functions work.
• It is executed "before" any detailed functional or
regression tests are executed on the software build.
• The purpose is to reject a badly broken application so that
the Quality team does not waste time installing and testing
the software application.
• The result of this testing is used to decide if a build is
stable enough to proceed with further testing.
• Conducted by developers or testers.

46
Examples of smoke test scenarios
Web Applications:
• Verify that the login/authentication process is working correctly.
• Access the main pages and make sure they load without errors.
• Submit a form and validate successful data submission.
• Perform basic navigation and confirm smooth transitions between pages.

Mobile app:
• Install the app and make sure it launches without any crashes.
• Navigate through the core screens and validate their responsiveness.
• Perform required interactions such as button clicks or form inputs.
• Verify functionality of key features such as push notifications or location services.

47
Sanity Testing
• Conducted for a build with minor bug fixes or new code added.
• It targets specific areas affected by bug fixes or new code.
• If rejected, then the build won’t proceed to further testing.

48
Sanity Testing

49
50
User acceptance testing (UAT)
• UAT is a stage in the testing process in which users
provide input and advice on system testing.
• Main objective is building confidence that users can use
the system to meet their needs, fulfil requirements, and
perform business processes with minimum difficulty,
cost, and risk.
• It is important because users have different perspective
than the developers. Moreover, the influences from the
user’s working environment have a major effect on the
reliability, performance, usability and robustness of a
system. These cannot be replicated in a testing
environment.
51
User acceptance testing (UAT)

• The quality team has a meeting with the client, with "UAT test
cases" which are the basic scenarios the client should run himself.
• The client will then give feedback: bugs or approval and a sign off
that "UAT has passed successfully".
• This is a very crucial activity done for all projects in all IT
companies and the quality team is responsible for managing it.

52
Business acceptance testing (BAT)
• It helps to check whether the system satisfies the business
requirements and specifications.
• The main aim of this testing is to help in increasing business
profits by considering market strategies and technologies.

53
Alpha and Beta Testing
• Both used for commercial off-the-shelf (COTS) software to get
feedback from potential/existing users/customers before the
software product is put on the market.
• Alpha testing
• Production team test the software in a lab environment at the
developer’s site.
• Beta testing
• After alpha testing is done, a beta version of the software is
made available to users to allow them to experiment and to
raise problems that they discover in their own environment.
54
55
Operational Acceptance Testing (OAT)
Acceptance testing of the system by operations/administration
staff is usually performed in a (simulated) production
environment.
The tests focus on operational aspects:
• Backup and restore
• Installing, uninstalling and upgrading
• Disaster recovery
• User management
• Maintenance tasks
56
Contractual acceptance testing (CAT)
• Contractual acceptance testing is performed against a contract’s
acceptance criteria for producing custom-developed software.
• Acceptance criteria should be defined when the parties agree to
the contract.

57
Regulatory acceptance testing (RAT)
• Regulatory acceptance testing is performed against any
regulations that must be adhered to, such as government, legal,
or safety regulations.
• If testing undergoes all the rules and regulations that are
released by the government then it is said to be RAT.
• Not even a single violation has not to be found while
performing this testing if any occurs then the whole product
will be going under failure. So, it is the most important and
complicated testing type.

58
59
60
Answer:
D
Answer:
A
Answer:
C