ISYS6264 – Testing and System Implementation

Template and Models

in Test Management
 Learning Outcomes

• LO 2: Design the testing management plan for a software

• LO 3: Design the testing implementation plan for a software
 References

• Black, Rex. (2009). Managing the testing process : practical

tools and techniques for managing hardware and software
testing. 03. Wiley. Indianapolis. ISBN: 9780470404157.

• Homès, Bernard. (2012).Fundamentals of Software Testing. ISTE

– Wiley. London – Hoboken. ISBN: 978-1-84821-324-1
 Sub Topics

• Master Test Plan – IEEE

• Test Plan – IEEE
• Test Design Document – IEEE
• Test Case – IEEE
• Test Procedure – IEEE
• Test Log – IEEE
• Bug Report – IEEE
• Test Report – IEEE
Master Test Plan – IEEE
 IEEE 829 Master Test Plan
Template
• 1. Introduction
– 1.1. Document identifier
– 1.2. Scope
– 1.3. References
– 1.4. System overview and key features
– 1.5. Test overview
– 1.5.1. Organization
– 1.5.2. Master test schedule
– 1.5.3. Integrity level schema
– 1.5.4. Resources summary
– 1.5.5. Responsibilities
– 1.5.6. Tools, techniques, methods, and metrics
 IEEE 829 Master Test Plan
Template (Cont.)
• 2. Details of the master test plan
– 2.1. Test processes including definition of test levels
– 2.1.1. Process: management
– 2.1.1.1. Activity: management of test effort
– 2.1.2. Process: acquisition
– 2.1.2.1. Activity: acquisition support test
– 2.1.3. Process: supply
– 2.1.3.1. Activity: planning test
– 2.1.4. Process: development
– 2.1.4.1. Activity: concept
– 2.1.4.2. Activity: requirements
– 2.1.4.3. Activity: design
– 2.1.4.4. Activity: implementation
 IEEE 829 Master Test Plan
Template (Cont.)
– 2.1.4.5. Activity: test
– 2.1.4.6. Activity: installation/checkout
– 2.1.5. Process: operation
– 2.1.5.1. Activity: operational test
– 2.1.6. Process: maintenance
– 2.1.6.1. Activity: maintenance test
– 2.2. Test documentation requirements
– 2.3. Test administration requirements
– 2.4. Test reporting requirements

• 3. General
– 3.1. Glossary
– 3.2. Document change procedures and history
Test Plan – IEEE
IEEE 829 Standard Test Plan
Template

Source: Black (2009, pg. 72)

 Introduction Section

• The Document Identifier section includes information such as

the date of release for the test plan, the organization that
released the test plan, the author, the author’s manager, the
approvers, the reviewers, the document ID number or official
name, and the version number.

• Scope section describes the purpose, goals, and scopes of the

test effort in detail.

• References section in this template is basically the same as

the one in my template, though the IEEE recommends
segmenting the documents into external and internal
references.
 Introduction Section
(Cont.)
• The System Overview and Key Features section can serve as
an executive summary for senior managers reading the
document.

• Test Overview section can include a description of the

relationship of the test processes to other processes, such as
programming, project management, the larger quality
assurance processes, configuration management or release
engineering, integrity level, resources,
 Details of The Master Plan
Section
• Test Process section discusses test activities and tasks for
each level. The assumption here is that this master test plan
describes all test levels, and might be complemented by level
or detail test plans.
• The section on test documentation specifies the purpose,
format, and content of the test documentation produced.
• The section on test administration deals with a number of
important issues. Some of those issues are reporting,
management of bugs, approach for regression testing,
confirmation testing, deviation from test plan, control
procedures, and standard & practices.
• Test Reporting section discusses the purpose, content, format,
recipients, and timing of all test reports. This includes all test
logs, bug reports (which they call anomaly reports), and interim
and final test status reports.
 General Section

• It includes a glossary, which cover definitions and terms used.

• It also includes a discussion on change procedures and

history.
Test Design Document – IEEE
 IEEE 829 Test Design
Document Template
• 1. Introduction
– 1.1. Document identifier
– 1.2. Scope
– 1.3. References
• 2. Details of the level test design
– 2.1. Features to be tested
– 2.2. Approach refinements
– 2.3. Test identification
– 2.4. Feature pass/fail criteria
– 2.5 Test deliverables
• 3. General
– 3.1. Glossary
– 3.2. Document change procedures and history
Test Case – IEEE
 IEEE 829 Test Case Document
Template
• 1. Introduction (once per document)
– 1.1. Document identifier
– 1.2. Scope
– 1.3. References
– 1.4. Context
– 1.5. Notation for description

• 2. Details (once per test case)

– 2.1. Test case identifier
– 2.2. Objective
– 2.3. Inputs
– 2.4. Outcome(s)
– 2.5. Environmental needs
 IEEE 829 Test Case Document
Template (Cont.)
– 2.6. Special procedural requirements
– 2.7. Intercase dependencies

• 3. Global (once per document)

– 3.1. Glossary
– 3.2. Document change procedures and history
Test Procedure – IEEE
 IEEE 829 Test Procedure
Document Template
• 1. Introduction
– 1.1. Document identifier
– 1.2. Scope
– 1.3. References
– 1.4. Relationship to other procedures
• 2. Details
– 2.1. Inputs, outputs, and special requirements
– 2.2. Ordered description of the steps to be taken to
execute the test cases
• 3. General
– 3.1. Glossary
– 3.2. Document change procedures and history
Test Log – IEEE
IEEE 829 Test Log Template

Source: Black (2009, pg 222)

 Introduction Section

• The first section, the Introduction, document ID is much the

same, though IEEE notes that the log ID should have some
clear mapping to the test procedure ID so that the
relationship between the test resulting in the log entry is clear.

• The IEEE says the Scope section should describe the scope of
the log, both in terms of its focus and its relationship to other
test documents. The header rows on the test case summary
worksheets and test suite summary worksheets show this
information.
 Details Section

• Details section is the core of the test log and provides the
essential information. It starts with a Description section,
which provides general information for all entries in the log.
IEEE suggests the following topics:
– The test items, including version numbers. In the case of the worksheets
shown earlier, this is provided indirectly by the date of the test, which
should suffice if the test releases are weekly. If that’s not precise enough,
you can add a column for this information to the test case summary
worksheet.
– Information about the test environment, which is explicitly documented
in the test case summary worksheet.
– The date and time of starting and stopping the test, which is a bit more
fine-grained than I tend to get.
– The name of the tester who ran the test, for which you can use initials.
– Any blocking issues that stop the test from completing, which you would
show in the comments field.
 General Section

• Finally, the template includes a General section, which has a

glossary in it. The IEEE describes these sections as having the
same information as similar sections in previously discussed
templates.
Bug Report – IEEE
 IEEE 829 Standard’s
Anomaly (Bug) Report Template

Source: Black (2009, pg. 177)

 Introduction Section

• Section 1, the introduction, includes a report ID number,

which is pretty obvious. It includes a section listing references,
which is useful when a bug report is pointing out a clear
discrepancy between some documents that describe the
expected behavior and whatever the actual behavior is. The
Scope section is less obvious, and the standard doesn’t help
much when it says that this section should ‘‘briefly describe
any contextual information not covered elsewhere in the AR
[anomaly report] that is needed to make this AR
understandable,’’ which might lead you to ask, ‘‘Well, what
wouldn’t fit into section 2.3 that would require it to go here?’’
 Details Section

• Getting into the second section, you’ll see some fields that
look familiar. The Summary and Date Anomaly Discovered
fields are exactly as I discussed earlier in this chapter.

• The Context field here makes sense. This includes the

information about the particular item or system being tested,
and the version of it, when we saw the bug. It also includes
information about the software or system life cycle process
underway at that time.

• It also includes references to other test documents, especially

the test procedure or test case the tester was running when
we saw the bug, and any test results logs that document the
test case result. It should also identify the phase or level of
testing underway.
 Details Section
(Cont.)
• The Description of Anomaly field is basically the same as the
failure description I described earlier in this chapter. It should
indicate reproducibility, and provide detailed steps to
reproduce. It should provide or reference the inputs given, the
expected results, the actual results, what was unexpected
about the outcome, the specific step in the test cases or
procedure that failed, the test environment used, how many
times the failure was reproduced, who ran the test, and who
observed the results.

• The 829 standard encourages you to include a section on

impact, which is not quite the same as the priority
classification information. Here, they are looking for the way
the problem affects various business and technical
considerations, in narrative form. They are also looking for
workarounds, if they exist. Once the development
 Details Section
(Cont.)
• The section on the originator’s assessment of urgency is
actually closer to priority. They also include the idea of risk
related to the bug, possibly involving a reference back to the
quality risk item associated with the bug.. The IEEE 829
standard encourages the use of a consistent set of
classifications for urgency.

• The Description of the Corrective Action section corresponds

to resolution.

• The Status of the Anomaly field is as we discussed in the

section on life cycles.

• The Conclusions and Recommendations field includes

recommendations for improving the development or testing
 General Section

• Finally, the template includes a General section, which has a

document change procedures and history in it. The IEEE
describes these sections as having the same information as
similar sections in previously discussed templates.
Test Report – IEEE
 IEEE 829 Interim Test Report
Template
• 1. Introduction
– 1.1. Document identifier
– 1.2. Scope
– 1.3. References

• 2. Details
– 2.1. Test status summary
– 2.2. Changes from plans
– 2.3. Test status metrics

• 3. General
– 3.1. Document change procedures and history
 IEEE 829 Level Test Report
Template
• 1. Introduction
– 1.1. Document identifier
– 1.2. Scope
– 1.3. References

• 2. Details
– 2.1. Overview of test results
– 2.2. Detailed test results
– 2.3. Rationale for decisions
– 2.4. Conclusions and recommendations

• 3. General
– 3.1. Glossary
– 3.2. Document change procedures and history
 IEEE 829 Master Test Report
Template
• 1. Introduction
– 1.1. Document identifier
– 1.2. Scope
– 1.3. References

• 2. Details of the master test report

– 2.1. Overview of all aggregate test results
– 2.2. Rationale for decisions
– 2.3. Conclusions and recommendations

• 3. General
– 3.1. Glossary
– 3.2. Document change procedures and history
Thank