SOFTWARE TESTING

DEPARTMENT OF
COMPUTER SCIENCE & ENGINEERING
1
 • Testing is the process of • Software testing can be stated as the process of
evaluating a system or its verifying and validating that a software or
component(s) with the intent to application is bug free, meets the technical
find whether it satisfies the requirements as guided by it’s design and
specified requirements or not. development and meets the user requirements
In simple words, testing is effectively and efficiently with handling all the
executing a system in order to exceptional and boundary cases.
identify any gaps, errors, or
missing requirements in • The process of software testing aims not only at
DEFINITI contrary to
requirements.
the actual finding faults in the existing software but also at
finding measures to improve the software in terms of
ON OF • According to ANSI/IEEE 1059
efficiency, accuracy and usability. It mainly aims at
measuring specification, functionality and
TESTING standard, Testing can be
defined as - A process of performance of a software program or application.
analyzing a software item to • Software Testing is a process of evaluating the
detect the differences between
functionality of a software application to find any
existing and required
conditions (that is software bugs. It checks whether the developed
defects/errors/bugs) and to software met the specified requirements  and
evaluate the features of the identifies any defect in the software in order to
software item. produce a quality product. It is basically executing
a system in order to identify any gaps, errors, or
missing requirements in contrary to the actual
requirements.
Who does Testing?
It depends on the process and the associated stakeholders of the project(s). In the IT industry, large companies have a team with
responsibilities to evaluate the developed software in context of the given requirements. Moreover, developers also conduct
testing which is called Unit Testing. In most cases, the following professionals are involved in testing a system within their
respective capacities −
•Software Tester
•Software Developer
•Project Lead/Manager
•End User
Different companies have different designations for people who test the software on the basis of their experience and
knowledge such as Software Tester, Software Quality Assurance Engineer, QA Analyst, etc.
It is not possible to test the software at any time during its cycle. The next two sections state when testing should be started and
when to end it during the SDLC.
When to Start Testing?
An early start to testing reduces the cost and time to rework and produce error-free software that is delivered to the client.
However in Software Development Life Cycle (SDLC), testing can be started from the Requirements Gathering phase and
continued till the deployment of the software.
It also depends on the development model that is being used. For example, in the Waterfall model, formal testing is conducted
in the testing phase; but in the incremental model, testing is performed at the end of every increment/iteration and the whole
application is tested at the end.
Testing is done in different forms at every phase of SDLC −
•During the requirement gathering phase, the analysis and verification of requirements are also considered as testing.
•Reviewing the design in the design phase with the intent to improve the design is also considered as testing.
•Testing performed by a developer on completion of the code is also categorized as testing.
When to Stop Testing?
It is difficult to determine when to stop testing, as testing is a never-ending process and no one can claim that a software is
100% tested. The following aspects are to be considered for stopping the testing process −
•Testing Deadlines
•Completion of test case execution
•Completion of functional and code coverage to a certain point
•Bug rate falls below a certain level and no high-priority bugs are identified
•Management decision
Let’s break down the basic definition of Software testing into the following parts:
1)  Process:  Testing is a process rather than a single activity.
2)  All Life Cycle Activities: Testing is a process that’s take place throughout the Software Development Life Cycle (SDLC).
•The process of designing tests early in the life cycle can help to prevent defects from being introduced in the code. Sometimes it’s referred as  “verifying
the test basis via the test design”.
•The test basis includes documents such as the requirements and design specifications.
3)  Static Testing:  It can test and find defects without executing code. Static Testing is done during verification process. This testing includes reviewing
of the documents (including source code) and static analysis. This is useful and cost effective way of testing.  For example:
reviewing, walkthrough, inspection, etc.
4)  Dynamic Testing:  In dynamic testing the software code is executed to demonstrate the result of running tests. It’s done during validation process. For
example: unit testing, integration testing, system testing, etc.
 5)  Planning:  We need to plan as what we want to do. We control the test activities, we report on testing progress and the status of the software under
test.
6)  Preparation:  We need to choose what testing we will do, by selecting test conditions and designing test cases.
7)  Evaluation:  During evaluation we must check the results and evaluate the software under test and the completion criteria, which helps us to decide
whether we have finished testing and whether the software product has passed the tests.
8)  Software products and related work products:  Along with the testing of code the testing of requirement and design specifications and also the related
documents like operation, user and training material is equally important.
 Why do we need Software Testing?
The software application’s success rate controls the growth of our business. It plays an important role for the development
of software applications and products.

We need software testing for the following reasons:

1.Cost Effective – Testing has many benefits and one of the most important ones is cost-effectiveness. Testing our project on time can save
money in the long run. Software development consists of many stages and if bugs are caught in the earlier stages it costs much less to fix
them.
2. Security – This is the most sensitive and vulnerable part of software testing. Users are always looking for trusted products that they can
rely on. It helps in removing problems and risks beforehand.

3. Product Quality – In order to make your product vision come to life, it has to work as planned. It is important to follow
the product requirements because it helps you get the required end results.

4. Customer Satisfaction – The ultimate goal for a product owner is to give the best customer satisfaction.  Software should be tested in order
to bring the best user experience possible. Being the best product in this saturated market will help you gain trustworthy clients which will
have great long-term effects.
 Difference between Testing and Debugging

What is Testing?

Testing: Fundamentally, testing is a process to check if the system is working same as it was supposed to do, and
not working as it was not supposed to do. It is done by the tester to identify the defects in the system (actual
result of test case execution is not matching with expected result). It can be done using Manual and automated
way. The issues are logged against the all failed cases and communicate to the dev team for debugging and
fixing. After the bug fixes then tester retest the bug and checks if the issue is no more exists. You may interest in
– Defect Life Cycle.

Testing involves identifying software bug/error/defect without its correction. Usually professionals with a
background of quality assurance are involved in the process of bugs identification. This process is performed in
the testing phase.
 
 
What is Debugging?
Debugging is the activity performed by developers to fix the bug found in the system. This is manual step by step unstructured
and unreliable process to find and removes a specific bug from the system.
It is done by the developers to fix the defect in the system. Based on the defects reported, the developer starts analyzing the
root cause of the defect to make system defect free. The developers need to fix the code to make actual result same as expected
result. With the help of details added in the defect developer follow steps mentioned and check the loopholes in the particular
code area. They may use a different tool for debugging. On fixing the defect developer send back the defect to a tester for
retesting.

Debugging involves identifying, isolating, and fixing the bugs/problems. Developers who code the software carry out
debugging in order to encounter an error in the code. This process is a part of Unit Testing or White Box Testing. Debugging
can be performed either while conducting Unit Testing, i.e. development phase or while fixing the reported bugs.
Debugging is deeper process than the  Testing. Debugging involves identifying, isolating and fixing the bugs.
 Following are the important differences between Testing and Debugging.

Sr. No. Key Testing Debugging

1 Definition Technically Testing is a process to check if the On other hand Debugging is the
application is working same as it was supposed to do, activity performed by developers to
and not working as it was not supposed to do. fix the bug found in the system.

2 Objective Main objective of Testing is to find bugs and errors in On other hand the main objective of
an application which get missed during the unit Debugging is to find the exact root
testing by the developer. cause at code level to fix the errors
and bugs found during the testing.
3 Perform As Testing is mainly to find out the errors and bugs is While on other hand Debugging is to
mainly performed by the testers. Also if testing is at find the missing or de-faulty code in
developer end known as unit testing then it is an application hence major
performed by the Developer. performed by the developers only.

4 Knowledge As Testing covers the functional and behavioural flow On other hand Debugging is to find
Required of an application so only functional knowledge is the error at code level so technical
required for the tester to perform the testing. and code level knowledge is required
for the developer to perform
debugging.
5 Automation Testing can be manual or made automated with the On other hand Debugging can't be
help of different tools. get automated it is always be the
manual.
6 Level Testing on basis of level of performing is at different On other hand no such level of
level i.e., unit testing, integration testing, system Debugging is possible.
testing, etc.
What are Software Testing Objectives?
•To evaluate the work products such as requirements, design, user stories, and code:
The work products such as Requirement document, Design, and User Stories should be verified before the developer picks it up for development.
Identifying any ambiguity or contradicting requirements at this stage saves considerable development and test time. The static analysis of the code (reviews,
walk-thru, inspection, etc.) happens before the code integrates/is ready for testing.  This idea of testing is known as Verification. It is the process of
evaluating the product during the development phase of each work product.

•To verify the fulfillment of all specified requirements:

This objective reveals the fact that the essential elements of testing should be to fulfill the customer’s needs. Testers test the product and ensure the
implementation of all the specified requirements have. Developing all the test cases, regardless of the testing technique ensures verification of the
functionality for every executed test case. The Tester should also create a requirement traceability matrix (RTM), which will ensure the mapping of all the test
cases to requirements. RTM is an effective way to ensure that test cases have got the right requirement coverage.
 
•To validate if the test object is complete and works as per the expectation of the users and the stakeholders:
Testing ensures the implementation of requirements along with the assurance that they work as per the expectation of users. This idea of testing is
called Validation. It is the process of checking the product after development. Validation can be a manual or automation. It usually employs various types of
testing techniques, i.e., Black Box, White Box, etc. Generally, testers perform validation, whereas customers can also validate the product as part of User
acceptance testing. Every business considers the customer as the king. Thus the customer’s satisfaction is a predominant need for any business. 
For example, customer satisfaction and loyalty in online shopping and e-commerce environments is a useful indicator for long-term business success.
•To build confidence in the quality level of the test object:
One of the critical objectives of software testing is to improve software quality. High-Quality software means a lesser number of
defects. In other words, the more efficient the testing process is, the fewer errors you will get in the end product. Which, in turn,
will increase the overall quality of the test object. Excellent quality contributes to a significant increase in customer satisfaction as
well as lower maintenance costs. 
 
•To prevent defects in the software product:
One of the objectives of software testing is to avoid the mistakes in the early stage of the development. Early detection of errors
significantly reduces the cost and effort. The prevention of defects involves doing a root cause analysis of the defects found
previously and after that, taking specific measures to prevent the occurrence of those types of errors in the future. Efficient testing
helps in providing an error-free application. If you prevent defects, it will result in reducing the overall defect count in the product,
which further ensures a high-quality product to the customer.
 
•To find defects in the software product:
Another essential objective of software testing is to identify all defects in a product. The main motto of testing is to find maximum
defects in a software product while validating whether the program is working as per the user requirements or not. Defects should
be identified as early in the test cycle as possible.
E.g., a defect found in the UAT phase will be much costlier to fix than the same defect found in the Sprint testing phase. 
 
•To provide sufficient information to stakeholders to allow them to make informed decisions, especially regarding the level of quality of
the test object:
The purpose of testing is to provide complete information to the stakeholders about technical or other restrictions, risk factors, ambiguous
requirements, etc. It can be in the form of test coverage, testing reports covering details like what is missing, what went wrong. The aim is
to be transparent and make stakeholders fully understand the issues affecting quality.
 
•To reduce the level of risk of insufficient software quality:
The possibility of loss is also known as risk. The objective of software testing is to reduce the occurrence of the risk. Each software project
is unique and contains a significant number of uncertainties from different perspectives, such as market launch time, budget, the technology
chosen, implementation, or product maintenance. If we do not control these uncertainties, it will impose potential risks not only during the
development phases but also during the whole life cycle of the product. So, the primary objective of software testing is to integrate the Risk
management process to identify any risk as soon as possible in the development process.
 
•To comply with contractual, legal, or regulatory requirements or standards, and to verify the test object’s compliance with such
requirements or standards:
This objective ensures that software developed for a specific region must follow the legal rules and regulations of that region. Moreover,
the software product must be compatible with the national and international standards of testing. We have ISO/IEC/IEEE 29119 standards
that deal with the software testing concept.
E.g., each country has laws specific to accessibility requirements which must be fulfilled to avoid legal implications. The European Union
has strict rules on how the Personal Identifiable Information (PII) like Social security number etc. should be handled. Failure to adhere to
such requirements will lead to failure of the product, no matter how defect-free it has been working!
 
 Software Testing - Types of Testing
1) Manual Testing
Manual testing includes testing a software manually, i.e., without using any automated tool or any script. In this type, the
tester takes over the role of an end-user and tests the software to identify any unexpected behavior or bug. There are
different stages for manual testing such as unit testing, integration testing, system testing, and user acceptance testing.
Testers use test plans, test cases, or test scenarios to test a software to ensure the completeness of testing. Manual testing
also includes exploratory testing, as testers explore the software to identify errors in it.
2) Automation Testing
Automation testing, which is also known as Test Automation, is when the tester writes scripts and uses another software to
test the product. This process involves automation of a manual process. Automation Testing is used to re-run the test
scenarios that were performed manually, quickly, and repeatedly.
What to Automate?
It is not possible to automate everything in a software. The areas at which a user can make transactions such as the login
form or registration forms, any area where large number of users can access the software simultaneously should be
automated.
Furthermore, all GUI items, connections with databases, field validations, etc. can be efficiently tested by automating the
manual process.
When to Automate?
Test Automation should be used by considering the following aspects of a software −
•Large and critical projects
•Projects that require testing the same areas frequently
•Requirements not changing frequently
•Accessing the application for load and performance with many virtual users
•Stable software with respect to manual testing
•Availability of time
How to Automate?
Automation is done by using a supportive computer language like VB scripting and an automated software application. There are
many tools available that can be used to write automation scripts. Before mentioning the tools, let us identify the process that can be
used to automate the testing process −
•Identifying areas within a software for automation
•Selection of appropriate tool for test automation
•Writing test scripts
•Development of test suits
•Execution of scripts
•Create result reports
•Identify any potential bug or performance issues

Software Testing Tools

The following tools can be used for automation testing −
•HP Quick Test Professional
•Selenium
•IBM Rational Functional Tester
•SilkTest
•TestComplete
•Testing Anywhere
•WinRunner
•LoadRunner
•Visual Studio Test Professional
•WATIR
 Software Testing - Myths
Myth 1: Testing is Too Expensive
Reality − There is a saying, pay less for testing during software development or pay more for maintenance or correction later. Early testing saves both
time and cost in many aspects, however reducing the cost without testing may result in improper design of a software application rendering the product
useless.
Myth 2: Testing is Time-Consuming
Reality − During the SDLC phases, testing is never a time-consuming process. However diagnosing and fixing the errors identified during proper
testing is a time-consuming but productive activity.
Myth 3: Only Fully Developed Products are Tested
Reality − No doubt, testing depends on the source code but reviewing requirements and developing test cases is independent from the developed code.
However iterative or incremental approach as a development life cycle model may reduce the dependency of testing on the fully developed software.
Myth 4: Complete Testing is Possible
Reality − It becomes an issue when a client or tester thinks that complete testing is possible. It is possible that all paths have been tested by the team but
occurrence of complete testing is never possible. There might be some scenarios that are never executed by the test team or the client during the
software development life cycle and may be executed once the project has been deployed.
Myth 5: A Tested Software is Bug-Free
Reality − This is a very common myth that the clients, project managers, and the management team believes in. No one can claim with absolute
certainty that a software application is 100% bug-free even if a tester with superb testing skills has tested the application.
Myth 6: Missed Defects are due to Testers
Reality − It is not a correct approach to blame testers for bugs that remain in the application even after testing has been performed. This myth relates to Time, Cost,
and Requirements changing Constraints. However the test strategy may also result in bugs being missed by the testing team.
Myth 7: Testers are Responsible for Quality of Product
Reality − It is a very common misinterpretation that only testers or the testing team should be responsible for product quality. Testers’ responsibilities include the
identification of bugs to the stakeholders and then it is their decision whether they will fix the bug or release the software. Releasing the software at the time puts
more pressure on the testers, as they will be blamed for any error.
Myth 8: Test Automation should be used wherever possible to Reduce Time
Reality − Yes, it is true that Test Automation reduces the testing time, but it is not possible to start test automation at any time during software development. Test
automaton should be started when the software has been manually tested and is stable to some extent. Moreover, test automation can never be used if requirements
keep changing.
Myth 9: Anyone can Test a Software Application
Reality − People outside the IT industry think and even believe that anyone can test a software and testing is not a creative job. However testers know very well
that this is a myth. Thinking alternative scenarios, try to crash a software with the intent to explore potential bugs is not possible for the person who developed it.
Myth 10: A Tester's only Task is to Find Bugs
Reality − Finding bugs in a software is the task of the testers, but at the same time, they are domain experts of the particular software. Developers are only
responsible for the specific component or area that is assigned to them but testers understand the overall workings of the software, what the dependencies are, and
the impacts of one module on another module.
Estimation Techniques in Software Testing

18
One of the most important factors while estimating testing efforts is the hands-on experience on varied
projects for the software test life cycle. No longer can one just take a guessing approach about the number
of days for any task or working on the old-time formula of one-third of the development effort.
Although this method is not based on any scientific principle or technique, it is one of the most widely used
estimation technique by companies offering software testing services. Unfortunately, the development
versus testing effort method has given many failures in software projects in the past, thereby compromising
the software or mobile apps on quality.
In recent years there have been many techniques that have been developed for estimating the software
testing timeframe. These techniques are: 3-Point Software Testing Estimation Technique, Use-Case Point
Method and Wide Band Delphi Method.
 
3-Point Software Testing Estimation Technique
3-Point Software Testing Estimation Technique is based on statistical methods in which each testing task is
broken down into sub-tasks and then three types of estimation are done on each tasks.
The formula used by this technique is:
Test Estimate = P + (4*N) + E / 6
Where,
•P = Positive Scenarios
•N = Negative Scenarios
•E = Exceptional Scenarios
 
Standard deviation for the technique is calculated as,
•Standard Deviation (SD) = (N – E)/6
 

19
Use – Case Point Method
Use-Case Point Method is based on the use cases, where we calculate the unadjusted actor weights and unadjusted use
case weights to determine the software testing estimation.
The formula used for this technique is:
•Unadjusted actor weights = Total no. of actors (positive, negative and exceptional)
•Unadjusted use case weight = Total no. of use cases
•Unadjusted use case point = Unadjusted actor weights + Unadjusted use case weight
•Adjusted use case point = Unadjusted use case point * [0.65+ (0.01 * 50]
•Total Effort = Adjusted use case point * 2
 
Wideband Delphi
In Wideband Delphi Method, work structure is broken down for each task and is distributed to a team comprising of 3-7
members for re-estimating the task. The final estimate is the result of the summarized estimates based on the team
consensus. This method speaks more on experience rather than any statistical formula. The Wideband Delphi testing
estimation technique logically estimates the group iteration efforts required in a visual manner for the testing team. This
test was coined by Barry Boehm and is widely accepted software testing estimation technique to solve complex
problems.
How to evaluate a software testing estimation technique?
For any software testing estimation technique, it is highly recommended that the following factors should be taken into
account:
1.Domain knowledge and core requirements
2.Risks and complexity of the application
3.Team knowledge on the subject/skills
4.Historical data for the previous estimation for improvement and accuracy
5.Estimation should include buffer time
6.Bug cycles for the project
7.Resources availability 20
Aravali College of Engineering And Management
Jasana, Tigoan Road, Neharpar, Faridabad, Delhi NCR
Toll Free Number : 91- 8527538785
Website : www.acem.edu.in
21