Software Testing

What is software quality

-Achieving high level of product or service quality is the objective

of most organizations.

-It is no longer acceptable to deliver poor quality products, and fix

deficiencies after they have been shipped to the customer.

-However, software quality is a complex notion that cannot be

defined in a simple way, unlike for other manufactured products
such as cars, televisions, or computers.

- Quality product is expected to comply with customer requirements.

- It is an ongoing challenge to precisely specify and check certain quality

requirements (e.g., usability, maintainability, reusability etc.)
 What is Software Quality

qFailure:

qThe inability of a system or component to perform required function according to its

specification.

qFault:

qIncorrect step in computer program

qIt is a condition that causes the software to fail to perform its required function.

qError:

qRefers to difference between Actual Output and Expected output

qIncorrect human action.

Error, fault, and failure collectively refer to as defects

What is Software Quality

errors à faults à failures

qErrors may cause faults to be injected into the software, and

faults may cause failures when the software is executed.

qA single error may cause many faults, such as in the case that
a wrong algorithm is applied in multiple modules and causes
multiple faults, and a single fault may cause many failures in
repeated executions.
What is Software Quality

qFaults which do not cause a failure under the

given scenarios are typically called dormant or
latent faults, which may still cause problems
under a different set of scenarios or
circumstances.
Why do we need to engineer software quality

q Software defects are extremely costly in

term of:
q Resources
q Time
q Money
q Reputation
q Safety Critical System
q Loss of life

q How to avoid Failure ?

q Quality Engineering Management
Why do we need to engineer software quality

q How Faults Happen?

q Failures Cost?

q Inserting faults is the norm

q High reliance on human
communication,
q Lack of Training,
q High complexity,
q Missing Docs,
q Ignorance of standard,
q Missing reviews,
q Insufficient testing etc.
*As the amount of project size/complexity
increases, the number of faults will increase
Why do we need to engineer software quality

Quality Engineering
Why do we need to engineer software quality

What to do for preventing fault introduction ?

Why do we need to engineer software quality

- Quality Assurance (QA) is fault prevention through process

design and auditing
- Creating processes, procedures, tools, etc. to prevent faults from occurring
- Examples: Templates, checklists, guides
- Main goal: Prevent as much as possible defect injection
Why do we need to engineer software quality

What to do for existing faults and faults that are carried forward ?
Why do we need to engineer software quality

- Quality Control (QC) is fault/failure detection through static

and/or dynamic testing of artifacts

- Examining the artifact against pre-determined criteria to measure conformance

Difference in (QA) , (QC) & (Testing) ?

- Quality Assurance (QA)

-Prevention process –those activities which develop/modify

processes to prevent introduction of defects

- Things done before starting to develop software & related produces

- Recruit Good People

- Identify standards, Processes, Tools

- Train People to use Processes & Tools

- Plan for QC
Difference in (QA) , (QC) & (Testing) ?

-Quality Control (QC)

-Detective process –those activities which find the flaws
-Things to be done after developing software & related
products

- Since Software development is a human dominating

activity and humans are error prone, some errors will occur.

- QC is focused on error detection.

- Focus on finding & detecting the defects in specific

deliverables.
Difference in (QA) , (QC) & (Testing) ?

-Testing ----- The process of executing a system with

the intent of finding defects

- Note: Process of executing a system includes test

planning prior to the execution of the test cases

-Testing is one example of a QC activity, but there

are others such as formal and Informal technical
reviews, inspections etc..
Software Quality Engineering

- Prevention - Quality Assurance (QA)

- fault prevention through process design and auditing

- Detection - Quality Control (QC)

- fault detection through review & testing of artifacts & program
Software Quality
qAn inherent or distinguish characteristic, a personal trait, degree or
grade of excellence The American Heritage Dictionary of the English Language.

qQuality is conformance to requirements and prevention of defects.

qThe degree to which a system, component, or process meets

specified requirements.

qThe degree to which a system, component, or process meets

customer or user needs or expectations.
Quality Views

- Quality is several attributes (portability, reliability, efficiency,

usability, testability, understandability, modifiability) …… Glass
- Quality is conformance to requirements ………………… Crosby
- Quality is fitness for use …………………………………... Deming
- Quality is value to some person …………………………. Weinberg
- Quality is whatever the customer decides………….……. Ginac
- Quality is an attitude or state of mind ……………...…….. Juran
Other Views …….. Garvin
- Manufacturing View….Conformance to specification
- Value-based View…How much customer is willing to pay
Quality Goals
- What are Goals?
- User View, Producer View, Quality Attributes: Reliability, Usability ….?

- A goal is S M A R T
Specific.… Measurable/Testable.... Attainable.… Relevant….Time-bound

-Danger of Lack of Goal

Repair-service behavior
- Without any clear idea of what the benchmarks are, we go in search of
things that are broken and our goal becomes fixing them
Know-how behavior
- We often don’t solve the problems that need to be solved but the ones
we know how to solve
No justification for any quality action
- “I’m doing it for quality” anarchy , wasting a lot of time with no alignment
Quality Plan
-A quality plan is a document setting out the specific quality practices,
resources and activities, relevant to particular product, process,
service, contract or project.
- Defines the Quality Goals
-Selects appropriate prevention & detection methods.
- Set out the desired product qualities, and define how these are to be
assessed ?
- Select those organizational standards that are appropriate to a
particular process and development process.
Software Quality Attributes Cavanno & MC call, 1978

qProduct Revision : This identifies quality factors that

influence the ability to change the software product.

qProduct Transition : This identifies quality factors that

influence the ability to adapt the software to new
environments.

qProduct Operations : This identifies quality factors that

influence the extent to which the software fulfills its
specification.
 Software Quality Attributes

Maintainability - Can I fix it? Portability - Will I be able to use

Flexibility - Can I change it? it on another machine?
Testability - Can I test it? Reusability - Will I be able to
reuse some of the software?
Product Product Interoperability - Will I be able to
Revision Transition interface it with another machine?

Product
Operations

Correctness - Does it do what I want?

Reliability - Does it do it accurately all the time?
Efficiency - Will it run on my machine as well as it can?
Integrity - Is it secure?
Usability - Can I easily use it?
Software Quality Characteristics and Attributes – ISO 9126-
1998

qReliability Characteristics relating to capability of software to

maintain its level of performance under stated conditions for a
stated period of time
qFault tolerance is the ability to maintain a specified level of
performance in cases of software faults or unexpected inputs
qRecoverability is the capability and effort needed to re
establish level of performance and recover affected data after
possible failure
qCompliance is the adherence to application-related
standards, conventions, regulations in laws and protocols
Software Quality Characteristics and Attributes – ISO
9126-1998
qUsability Characteristics relating to the effort needed for use, and
on the individual assessment of such use, by a stated or implied set
of users.
qUnderstand ability is the effort required for a user to recognize
the logical concept and its applicability
qLearnability is the effort required for a user to learn its
application, operation, input, and output
qOperability is the ease of operation and control by users
qAttractiveness is the capability of the software to be attractive to
the user
qCompliance is adherence to application-related standards,
conventions, regulations in laws and protocols
Software Quality Characteristics and Attributes –
ISO 9126-1998

qEfficiency Characteristic related to the relationship between

the level of performance of the software and the amount of
resources used, under stated conditions
qTime behaviour is the speed of response and processing
times and throughput rates in performing its function
qResource utilization is the amount of resources used and
the duration of such use in performing its function
qCompliance is adherence to application-related standards,
conventions, regulations in laws and protocols
 Software Quality Characteristics and Attributes – ISO 9126-
1998

qPortability Characteristics related to the ability to transfer the

software from one organization or hardware or software
environment to another
qInstallability is the effort needed to install the software in a
specified environment
qCo-existence is the capability of a software product to co-exist
with other independent software in common environment
qReplaceability is the opportunity and effort of using it in the
place of other software in a particular environment
qCompliance is adherence to application-related standards,
conventions, regulations in laws and protocols