Defect

Management
Defect Management is the process of recognizing and
recording defects, classifying them, investigating them,
taking action to resolve them, and disposing of them
when resolved

An organization’s defect management process and the

tool used to manage this work are of critical

importance:

Information gathered by effective management of

defects allows to gain insight on the state of a project

throughout the development lifecycle

By collecting and analyzing data over time, this can

help locate areas of potential improvement for testing

and development processes

The Test Manager must:

be familiar with what data is critical

to capture

be an advocate of proper usage of

both the process and the selected

defect management tool

Organizations should:

establish a defect management process which includes a

workflow and rules for classification

define standards while other might track defects

informally

Testers should attempt to minimize the number of false

positives reported as defects as some reports may describe

false positives

Defects found during testing should be logged

Typical defect reports have the following objectives:

Provide developers and other parties with information

about any adverse event that occurred

enables to identify specific effects and to isolate the

problem with a minimal reproducing test

enables to correct the potential defect(s) as needed

or to otherwise resolve the problem

Provide test managers a means of tracking:

the quality of the work product

the impact on the testing

Provide ideas for development and test process

improvement

When logging defects, you should consider:

the context or component/system under test

the test level

the software development lifecycle

Defects may be reported during:

coding

static analysis

reviews

dynamic testing

use of a software product

Defects may be reported for issues in:

code or working system

documentation

requirements

user stories

development or test documents

user manuals or installation guides

Any defects identified should be investigated and

should be tracked from discovery and classification to
their resolution.

Dynamic defect reports may sometimes differ. Defects

found during static testing (particularly reviews) will

be documented in a different way.

An example of contents of a defect report can be found

in ISO standard (ISO/IEC/IEEE 29119-3) and refers to

it as incident reports

The defect management tools used may automatically

manage and fill in some details

Defect report = Documentation of the occurrence, nature,

and status of a defect.

Whatever the specific information determined as necessary

for defect reports, it is critical that testers enter

information that is complete, concise, accurate,

objective, relevant and timely.

Defect report filed during dynamic testing includes:

Identifier Title

Short Summary

Date Organization Author

Item tested Environment Test Phase

Severity Priority Status

Expected Results Actual Results

Description of the issue

to enable reproduction and resolution
includes logs, dumps, screenshots, recordings

Other aspects
Conclusions, recommendations and approvals
References (ex: test case that revealed the issue)
Other areas that may be affected
Defect change history
When a defect is detected (static testing), or a failure is

observed (dynamic testing) data should be gathered by the

person(s) involved and included in the defect report.

Information from a defect report should suffice for:

1. Management of the report

2. Assessment of project status (terms of product quality

and test progress)

3. Assessment of process capability

Core information gathered should be consistent across the

lifecycle and ideally across all projects in order to  allow

for meaningful comparison of process defect data within

and across all projects.

Defect data may also include the following (on top of the

already presented data for a Defect Report)

The role of the author (end user, tester, business

analyst, technical support, etc.)

Type of testing performed (usability, performance,

regression, etc.)

Sub-system or component where the issue lies besides

the System under test

Project activity occurring when the issue was found

Type of defect (based on defect taxonomy)

Quality characteristics affected by the defect

Project or product in which the problem lies

Current owner (assignee)

Business Impact

Risks, costs, opportunity and benefits for fix/no fix

Description of how the defect was resolved

Defect information should support:

defect density analysis

trend analysis of defects detected and resolved

average time from defect detection to resolution

failure intensity like mean time between failures

Even though the collection of data can assist in:

test progress monitoring

control

exit criteria evaluation

There can be a lot of challenges in properly assessing

project status, test progress and process capability when

defect report data is of low quality and not properly

managed.

Various standards as ISO 9126 being replaced by ISO

25000, IEEE 829, IEEE 1044 and Orthogonal Defect

Classification exist to help the Test Manager determine

which information to gather for defect reporting.

Defects are introduced when a person makes a mistake

during the creation of a work product:

requirements specification

a user story

a technical document

a test case

the program code

any other work product produced during a software

development or maintenance process

Defects may be introduced at any point in the software

development lifecycle.

each phase should include activities to detect defects

the earlier a defect is identified and removed, the lower

the cost if quality

Cost of quality for a given level of defects is minimized

when each defect is removed in the same phase in which it

was introduced.

Static testing
finds defects directly, rather than finding failures, and

thus the cost of removing the defect is lower because

debugging activities are not required to isolate the defect.

Dynamic testing
During activities such as unit testing, integration testing,

and system testing, the presence of a defect is revealed

when it causes a failure, which results in a discrepancy

between the actual results and the expected results of a

test (anomaly).

If the tester does observe the anomaly, a situation has

occurred which requires further investigation. This

investigation starts with the filing of a defect report.

In test driven development automated unit tests are used

and executed on top of new developed code for each

complete unit. These failures do not constitute a defect

and are typically not tracked as they will occur until the

development is complete
A defect report typically progresses through a workflow

and moves through a sequence of states as it continues

through the defect lifecycle. In most of these states a

single defect participant owns the report and is

responsible for carrying out a task which will cause the

defect report to be moved to the next state.

Once the defect report reaches a terminal state like closed,

cancelled, irreproducible or deferred then it does not have

an owner anymore as no further action is required.

Defect workflow and states

The INITIAL state

One or more testers gather the information necessary

for the person responsible for resolving the defect to

reproduce the anomaly

also be referred to as the “open” or “new” state

The RETURNED state

The receiver of the report has rejected the report or is

asking the tester to supply further details

May indicate a shortfall in the initial information-

gathering process or of the testing itself.

This may also be referred to as the “rejected” or

“clarification” state
 The CONFIRMATION state

The tester will run a confirmation test to determine

whether the fix has solved the problem.

The defect can then be "Closed" or "Re-Open"

This may also be referred to as the “resolved” or

“verification” state

In some cases, an anomaly occurs not as the symptom of a

defect, but rather due to a problem with:

test environment

test data

other element of the testware

tester’s own misunderstanding

If the tester opens a defect report that is found not to

relate to a defect in the product under test, that is a false-

positive result. Such reports are cancelled or closed as

invalid defect reports.

If two or more defect reports are filed and found to

relate to the same root cause, one of the defect reports is

typically retained while the others are closed as duplicate

defect reports.

A Test Manager should:

monitor for excessive rates of return

should accept some amount of invalid and duplicated

defects (inevitable)

be wary of false-negative failures when

attempting to eliminate invalid and duplicate reports