Defect Management

Create Defect Reports

Level: Practitioner

Press F11 to view this course in Full Screen

 Learning Objectives

After completing this session, you will be able to review :

Defect reports and its related details.

Defect severity and priority.

The status of the defect and test environment details.

To ensure steps to reproduce are given appropriately.

The Rejected and deferred defects report.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

2
 Do you know?

Do you know?

 How to report a defect and its significance?

 How to extract different reports from any tools?

 Different defect logs that need to be collected?

3
 Learning Objective 1 of 5

After completing this session, you will be able to review ::

Defect reports and its related details.

Defect severity and priority.

The status of the defect and test environment details.

To ensure steps to reproduce are given appropriately.

The Rejected and deferred defects report.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

4
 Defect Report

Components of a defect report

 Project Name
 Module Name
 Detected build version
 Test Environment
 Defect Summary
 Defect Description (Defect Description, Expected and actual result, steps to reproduce)
 Test Data
 Defect Severity
 Defect Priority
 Defect status
 Test Case ID
 Attachment of Screenshot is captured
 Assigned to
 Submitted by
 Logged Date
 Release version

5
 Daily Defects Report - Sample
A sample consolidated defect summary with list of all defects found during testing phase:

Defect
Project Defect ID Detected By Status Environment Summary Severity Priority Introduced
Phase
BDW_SIT_Informatica_M1_The Job
Client.Informatica.DS1.FSS.BDW.M1_Cod
DW BI 1000 Adam Open Test e_Translation_Extract has failed to run 5-Urgent 5-Urgent Deployment
BDW_SIT_M1_Code Records link in the
DW BI 1001 Susan Open Test BDW is throwing an error. 5-Urgent 5-Urgent Deployment
BDW_SIT_Informatica_M1_XREFX_MAPP
DW BI 1002 Adam Open Test ING table not been loaded with data 3-High 3-High Coding
BDW_SIT_M1 - The values in the drop
down of the local system field in the
selection panel are not being populated
DW BI 1003 Susan Open Test correctly from the database. 3-High 3-High Coding
BDW_SIT_M1 - Drop down values of the
Object filed in the selection panel are not
DW BI 1004 Susan Open Test in sync with the database object values. 3-High 3-High Coding
BDW_SIT_M1 - Unable to edit the code
DW BI 1005 Susan Closed Test records 4-Very High 4-Very High Coding
BDW_SIT_M1 - Unable to block the code
DW BI 1006 Brian Closed Test records 4-Very High 4-Very High Coding

6
 Daily Defects Report - Sample
Projected Actual
TC Actual Projected
Actual
Date Attempted TC Projected Attempted TC Actual TC Actual Defects
(Pass + Fail + Partial) Passed (Pass + Fail + Passed Failed (ratio=20%) Defects
Partial)
21-Sep-13 8.7 6.96 0 0 0 1.74 0
22-Sep-13 17.4 13.92 0 0 0 3.48 0
23-Sep-13 26.1 20.88 1 0 1 5.22 1
24-Sep-13 34.8 27.84 1 0 1 6.96 1
25-Sep-13 43.5 34.8 13 8 4 8.7 9
26-Sep-13 43.5 34.8 13 8 4 8.1 9
27-Sep-13 43.5 34.8 13 8 4 8.1 9
28-Sep-13 52.2 41.76 13 8 4 10.44 9
29-Sep-13 60.9 48.72 41 20 21 12.18 19
30-Sep-13 69.6 55.68 55 39 16 13.92 20
1-Oct-13 78.3 62.64 68 52 16 15.66 28
2-Oct-13 87 69.6 68 52 16 17.4 28
3-Oct-13 87 69.6 68 52 16 17.4 28
4-Oct-13 87 69.6 68 52 16 17.4 28
5-Oct-13 87 87 72 69 3 17.4 29
6-Oct-13 87 87 87 82 2 17.4 31
7-Oct-13 87 87 87 86 1 17.4 33
8-Oct-13 87 87 87 86 1 17.4 33
9-Oct-13 87 87 87 86 1 17.4 33

7
 When & How to review-Defect Report Effectively

 Before sharing with the customer.

Review the defect  If rejected defects are high in previous releases.
report of your project  In case of escalation from development / client team on
defect report quality.

Before sharing the defect report to customer ensure the following details are appropriate.

 Consolidation of all logged defects.

 Check if the steps in the defect are detailed enough to reproduce the defect.
 Check the traceability between requirement, test cases, test data and defect maintained.
 Ensure if the defect severity and priority are assigned as per the defect classification guideline defined in
Test Strategy/Test Plan.

Please refer the checklist in the upcoming slides to review each defect in the defect report.

8
 Defects Reporting Review Checklist

Sl.No Checklist Items Yes No

(Defect ID)

1 Logged defect is simple and easy to understand

2 Defect logged is reproducible?

3 Is the same defect is logged by another team member/Check the logged defect is not included in
deferred defect list /Defect raised in previous build release which is not yet fixed?
(Duplicate Defect)

4 Reference id of all test cases is provided

5 Defect Description Test environment, build version, project id , module id , submitted by is given

6 Is the Severity and Priority of the defect is given based on the defect complexity as per the guideline
provided in Test strategy/Test plan document ?
7 Does the Screen shot or other reference documents is attached to substantiate the defect?

8 Does the used test data for identifying the defect is provided?

9 Verify for build release 1 and 2 – High priority / Critical functionality defects are identified and
logged.?

10 Verify Spell check is done?

9
 Learning Objective 2 of 5

After completing this session, you will be able to review ::

Defect reports and its related details.

Defect severity and priority.

The status of the defect and test environment details.

To ensure steps to reproduce are given appropriately.

The Rejected and deferred defects report.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

100
 Defect Severity
Severity is the degree of impact that a defect has on the system.

Severity can be of following types:

Minor
Moderate The defect that
Major The defect that does not result
Critical
The defect that does not result in the Cosmetic
The defect that
results in the in the termination and The defect that is
results in the
termination of the termination, but does not related to the
termination of
one or more causes the damage the enhancement of
the complete
component of the system to usability of the the system where
system and
system and causes produce system and the the changes are
causes extensive
extensive incorrect, desired results related to the look
corruption of the
corruption of the incomplete or can be easily and feel of the
data.
data. inconsistent obtained by application .
results. working around
the defects.

For example:
Severity Critical: If an application or web page crashes that is impact of  application crashing is severe.

111
 Defect Priority
Priority defines the order in which we should resolve a defect. This priority status is set by the tester to the
developer mentioning the time frame to fix the defect. The priority status is set based on the customer
requirements.

Priority can be of following types:

High: The defect must be

Low: The defect is an irritant
which should be repaired, but
repair can be deferred until after
more serious defect have been
fixed.
Medium: The defect should be
resolved in the normal course of
development activities. It can
wait until a new build or version
is created.
resolved as soon as possible
because the defect is affecting
the application or the product
severely. The system cannot be
used until the  repair has been
done.

For example: If the company name is misspelled in the home page of the website, then the priority is
high and severity is low to fix it.

122
 Defect Severity and Defect Priority

Let us understand defect severity and defect priority with some examples

Defect Scenario 1:
 A education site maintaining the student details, on saving record if it, doesn’t allow to save the record
then this is high priority and high severity defect.

 Adding a student details is important and key functionality so the severity of the defect is high. It is of
high priority to fix the defect because after fixing this defect other areas in the site can be tested.

Defect Scenario 2:
 The spelling mistakes that happens on the cover page or heading or title of an application. Spelling
mistakes in the title of application is High Priority. This depends on the internal and external application.

 External application , no of users who view this site will have bad impression about the company. This is of
low severity as it does not impact your testing . You can proceed and test all the functionalities.

13
 Defect Severity and Defect Priority

Defect Scenario 3:
 An error which occurs on the functionality of the application. Delete functionality is not working.
 This is High Severity because there is no workaround to test this functionality.

 This is of Low priority because this function is rarely used by the customer.

Defect Scenario 4:
 Any cosmetic or spelling issues which is within a paragraph or in the report.
 This is of low priority as this is not in customer heading or main page.
 This is of Low severity as it is not impacting/stopping other areas to test the application.

144
 Review – Severity & Priority of Defect

As per the guidelines defined in Test Strategy - Review the defects mapped for severity and priority.

Daily Defects Summary by severity by plotting Daily Defect Summary reports by priority by plotting
‘total no of defects found with respective severities’ ‘total no of defects found with respective priorities’
on Y axis and ‘Day of Testing’ on X axis. on X axis and ‘Day of Testing’ on Y axis ports.

25
No of Defects found #

20
5-Dec
15
Urgent Pri
10 High ort
3-Dec y3
Medium
5
Low
0 1-Dec
D ec Dec Dec Dec Dec Dec 0 5 10 15 20 25
1- 2- 3- 4- 5- 6-

Date

155
 Learning Objective 3 of 5

After completing this session, you will be able to review ::

Defect reports and its related details.

Defect severity and priority.

The status of the defect and test environment details.

To ensure steps to reproduce are given appropriately.

The Rejected and deferred defects report.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

166
 Review - Defect status & Test environment

The following report can be extracted to understand the Test Environment : Test Environmental details
status of the defects. of the type of application being tested along
 Daily Defects Summary by status with the
 Defects reported vs. closed  The version of the software being tested
 Defects summary by status – Module-wise  Operating System
 Firm Ware Version
 Database Version
Open Defects
140 100
 Build Version
127
10
120 80
76

100 Environment Availability

60

% of Environment Availability
100
Number of Defects

et

in
ef

ct
N

D
C
h

n
e

e
a
g

80 77 s 90
40
60
80
70
65 60
60 51
7
25
20
60
50 40
40 34 12
3
11
25 26 31 20
40
30 20
4 4 1 0 20
20
15 -9
12 11 10
18 1 6
5
1 0
0 2 1 -20 Mon Tue Wed Thu Fri
15-Oct 22-Oct 29-Oct 5-Nov 12-Nov
Critical High Medium Low Awaiting business priority Totals
Highlights the test environment up/down time
Net Change
* Represents data for the last 5 days of testing
Indicates the trend of open defects that need to be
driven to closure * Represents data for a max of previous 5 weeks

177
 Learning Objective 4 of 5

After completing this session, you will be able to review ::

Defect reports and its related details.

Defect severity and priority.

The status of the defect and test environment details.

To ensure steps to reproduce are given appropriately.

The Rejected and deferred defects report.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

188
 Review - Steps to reproduce

As this is the most important part of the defect report

 Ensure the tester has provided exact, simple, detailed steps on how to reproduce the defect.
 Along with the steps to reproduce, Ensure the tester has provided the expected result and the actual result.
 Ensure “test data” is provided.
 Verify by repeating the same steps given and check whether the defect can be reproduced.

Example:
Defect: On clicking “Inbox” in Microsoft mail box displays error message “.pst is not available in the right path”.

Steps to reproduce:
• Double Click on Microsoft Outlook Icon
• Click on Inbox < March 2013>
• Error message is displayed.

19
 Learning Objective 5 of 5

After completing this session, you will be able to review ::

Defect reports and its related details.

Defect severity and priority.

The status of the defect and test environment details.

To ensure steps to reproduce are given appropriately.

The Rejected and deferred defects report.

Indicates the topic in discussion

Indicates topics to be covered
Indicates topics covered

20
 Typical Causes of Defect Rejection or Deferred

Typical causes of Defect Rejection:

 No documented requirement which mentions the result expected.

 Lack of knowledge on requirement/domain for team.
 Changes to the requirements are not communicated to team.
 Defect logged relates to upcoming build, build is not available for testing.
 Invalid/Dummy test data in the application.
 Defects logged for Out of scope requirements. E.g. external systems, external content etc.
 Defect for a missing requirement on business functionality.
 Inadequate documentation of Steps to reproduce.

The below are some instances for defect rejection in case of Duplicate.
 Complete Duplicate of another defect.
 Describes the same problem in another area of the application as given in a valid defect.
 Describes the problem that is a part of the problem described in another valid defect.
 The defect report describes a problem that is due to another defect.

21
 Typical Causes of Defect Rejection or Deferred

Let us understand the typical reasons for Deferred Status

 An abandoned defect, which has no feasibility of effort to fix it.

 Defect will be considered for next release as they foresee no major problem in this release.

 Low priority Defect.

 Due to strategic changes in the plan, all defects related to specific application functionality are deferred.

22
 Review Rejected Defects

After analyzing the defect report, Follow

the below steps.

1. Read the Defect Report and try to

reproduce the defect.
2. Analyze the reason for the Defect
Rejection.
3. Establish whether the defect is valid or
not and then take appropriate action.

Invalid Defect – Update checklist or

guideline document appropriately.

Valid Defect – Take further action in

support to fix the defect.

23
 Defect Reports – Case study 1

Project X is in the Test execution phase and the Test manager finds out that there are lots of open
defects pending for quite some time in the project. On discussion with the Test Lead, he finds out that
though the testers have raised the defects and assigned it to the development team, the development
team comes back with lot of queries but does not give a fix to the defects. As the end Cycle 1 of the Test
Execution is nearing, the client is raising concerns on the huge number of unresolved defects.

What can the Test Manager do to bring down the number of open defects?

24
 Defect Reports – Case study 1

Test Manager reviews the defect reports created by the Testers. He discovers the following
issues in the defect report.

 The “Steps to reproduce” does not give a sequential explanation of the activities
performed to arrive at the defect. This made the developers come back to testers
again and again for more information.

 In most of the defects, the team has not followed the severity guidelines when
assigning the severity of defects. Even minor defects were assigned Sev2. Hence
the Development team was not sure on prioritizing the defects.

 The summary statement did not indicate the exact functionality where the defect
was found. Hence the defects often got assigned to the wrong developer and later
it was re-assigned to another developer after the developer went through the
details of the defects. This caused a time lag in fixing the defect.

The manager ensured that all the corrections are made to the defects and set clear
guidelines when raising defects in future.

25
 Defect Reports – Case study 2

Scenario : Project Y team has been set up in a short span of time and the team comprises of 50%
fresher. During execution, the Test manager is under pressure as there are lots of rejected defects and
the metric is showing the testing team in poor light. The senior management has asked the Test
manager to look into the issue and sort it out as quickly as possible.

What would be the solution provided by the Test manager?

26
 Defect Reports – Case study 2

The Test Manager has to perform the following steps to resolve the issue.

 The Test manager has to come up with a standard defect log template
to maintain consistency in the team.

 Before raising a new defect, the new team members should check the
defect management tool for any duplicates.

 For a brief period, the defects raised by the new team members have to
be reviewed by the senior personnel before sending to the
development team. The review should particularly focus on the clarity
of steps to reproduce and the appropriate screen shots.

27
 Summary
The key points covered in this session are...

 Importance of Defect Summary, Defect description, Defect Severity , Defect Priority

 Components of a defect report
 Sample defect reports
 Points to consider for defect report review
 Defects Reporting Review Checklist
 Understanding of Defect Severity and Priority
 Severity types - Critical, Major, Moderate, Minor, & Cosmetic
 Priority types – Low, Medium & high
 Review of Defect severity and priority
 Review of the Status of the Defect & Test Environment
 Review of Steps to reproduce
 Review the rejected and deferred defects to understand the typical Causes

33
Create Defect Reports
Thank You