SSE4306

Chapter 3
Product Assurance
 Learning Outcomes

At the end of this chapter student should be

able
– Understand the concept of product assurance
– Can differentiate between V&V and QA
– Describe the techniques of product assurance

2
 Defining product assurance
• An important aspect of SQA is the establishment of
confidence in the quality of the software products produced
by the project.
• The products are the software and related documentation.
Related documentation may include all those documents
associated with the development, operation, support,
maintenance, and retirement of the software, including
installation and administration.
• A product may also be a software service provided to the
acquirer.

3
 Defining product assurance
• The outcomes of the product assurance activities provides
evidence that the software services, products, and any
related documentation are identified in and comply with the
contract and any non-conformances are identified and
addressed.
• Product Assurance comprises these activities:
– Evaluate plans for conformance
– Evaluate product for conformance
– Evaluate product for acceptability
– Evaluate product life cycle support for conformance
– Measure products
• The SQA function confirms that software products are in
conformance with established requirements.
4
 Product Assurance
• Purpose:
– provide adequate assurance that the software product
and processes conform to their specified requirements
and adhere to the established plans, by measuring and
controlling quality of product.
• Product Assurance deals with the assurance of the product by
– Defining product quality objectives and metrication
– Defining product standards and procedures
– Ensuring product dependability* and safety.

*the quality of being trustworthy and reliable

 Product Assurance
• Software Product Assurance Plan shall specify or reference:
– Quality objectives (measurable whenever possible)
– The software development life cycle, (including milestones and the
input and output criteria for each phase)
– Verification and validation activities (including definition, schedules,
resources and approval authorities)
– Responsibilities for quality activities such as reviews and tests
configuration management and change control, non conformance
control and corrective action
– Methods, tools and rules to be used

6
 V&V Definitions
• V&V – a system engineering discipline
employing a rigorous methodology for
evaluating and assessing the correctness and
quality of software throughout the software
life cycle.

• Verify a developers process is technically

sound.
 V&V Definitions
• V&V processes are used to determine whether the development products
of a given activity conform to the requirements of that activity and
whether the product satisfies its intended use and user needs.
• V&V life cycle process requirements are specified for different integrity
levels. The scope of V&V processes encompasses systems, software, and
hardware, and it includes their interfaces.
• This standard applies to systems, software, and hardware being
developed, maintained, or reused [legacy, commercial off-the-shelf
(COTS), nondevelopmental items].
• The term software also includes firmware and microcode, and each of the
terms system, software, and hardware includes documentation. V&V
processes include the analysis, evaluation, review, inspection, assessment,
and testing of products.

8
 V&V
• V&V is an extension of program management and systems engineering
that employs a rigorous methodology to identify objective data and
conclusions to provide feedback about quality, performance, and
schedule to the supplier.
• This feedback consists of anomaly resolutions, performance
improvements, and quality improvements not only for expected operating
conditions but also across the full spectrum of the system and its
interfaces.
• Early feedback results allow the development organization to modify the
products in a timely fashion and thereby reduce overall project and
schedule impacts.
• Without a proactive approach, the anomalies and associated system
changes are typically delayed to later in the program schedule, resulting in
greater program costs and schedule delays.

9
 V&V
• Example: SDLC – V Model

10
 V&V and QA
• V&V and QA are not the same, but compliment each
other.
• V&V usually focuses on ensuring the requirements
are being met, the overall project is focused on the
correct objectives, and risk is being managed.
• QA is focused on the day to day aspects of a project
and is used to determine if procedures are followed
 V&V Principles (1)
• Errors/defects should be detected as early as
possible.
• V&V MUST be conducted throughout the entire life-
cycle.
• The outcome of V&V should not be considered a
binary variable.
• V&V requires ‘some’ independence to prevent
developer’s bias.

12
 V&V Principles (2)
• V&V is difficult in general. Creativity and insight are
required.
• Credibility can be claimed ONLY for the prescribed
conditions, which have been validated/verified.
• Complete model testing is not possible.
– Testing demonstrates the presence of defects, not their
absence!
• V&V must be planned and documented.

13
 Benefits of V&V
• Early detection leads to a better solution
rather than quick fixes
• Validating the solution is solving the “right
problem” against software requirements
• Objective evidence of software and system
compliance to quality standards
• Support process improvements with an
objective feedback on the quality of
development process and products
 V&V Techniques
• Informal techniques.
– Tools rely heavily on human reasoning and subjectivity.
– This does not imply the lack of structure or guidelines.
• Static techniques.
– Concerned with accuracy assessment based on static design
(mental execution).
• Dynamic techniques.
– Require instrumentation, execution and analysis.
• Formal techniques.
– Based on mathematical correctness proofs.

15
 V&V Techniques
Static: Dynamic: Formal:
Informal: Cause-effect Acceptance tests Induction
Inspections graphs Assertion Inductive assertions
Documentation Control analysis checking Inference
checks Data analysis Comparison tests Logical deduction
Reviews Fault/Failure Compliance tests Logical abduction
Walkthroughts analysis Execution profiling Correctness proofs
Interface analysis Fault injection Convergence proofs
Semantic analysis Interface tests Stability proofs
Traceability Partition tests
analysis Regression tests
Sensitivity
analysis
Special input tests
Statistical
techniques
Visualization and
animation
16
 V&V
Conclusions
• The V&V methodology and measurements are
outlined in ANSI/IEEE Standard 1012.
• Provides the framework for achieving an effective
V&V effort
• V&V is part of the software quality management
process as defined in the IEEE SWEBOK.
• Complimentary to and supportive of the software
quality assurance, reviews, and inspections.

17
 What is Software Inspection?
• Developed in the 1970’s at IBM by Fagan
• Definition:

Inspection process are a means of verifying

intellectual products by manually examining the
development product, a piece at a time, by small
groups of peers to ensure that it is correct and
conforms to product specifications and requirement.

18
 Software Inspection
• The goal of software inspections is to remove as
many defects as possible prior to product release.
• Defect removal efficiency (DRE) = Number of defects
resolved by the development team / total number of
defects at the moment of measurement.
• Inspections contribute to high defect removal
efficiency.

19
 Why effective?
• Most software problems can be traced to
requirements.
– Requirements are usually written as English (or local
language) prose.
– Personnel training problems.
• Requirements elicitation, analysis, negotiation, specification,
validation, management, etc.
– Imprecise, ambiguous, nondeterministic.
– Software, by its formal nature, is precise, unambiguous,
deterministic (?).

20
 Primary Purpose
• To remove defects as early as possible in the
development process
• The purpose of the inspection preparation and meeting is
to:
– Identify potential defects during preparation and
validate them at the meeting
– Validate the fact that identified items are actual
defects
– Record the existence of the defect
– Provide the record to the developer to use in making
fixes.
 Secondary Purpose
• The secondary purposes resulting from the inspection
process are:
– Provide traceability of requirements to design
– Provide a technically correct base for the next phase of
development
– Increase programming quality
– Increase product quality at delivery
– Achieve lower life-cycle cost
– Increase effectiveness of test activity
– Provide a first indication of program maintainability
– Encourage entry/exit-criteria software management
 Inspection Phases
• The moderator of an inspection is responsible for the
entire inspection process for the software product.
• There are six distinct inspection phases:
– Planning
– Overview
– Preparation
– Inspection meeting
– Rework
– Follow-up
 Participants of an Inspection
• Moderator – Coordinates the inspection and
leads the discussion
• Author/Producer – Responsible for the work
being inspected
• Reader – Paraphrases the work inspected
• Inspector – Inspects the product
• Recorder – Records problems discussed
• Manager – Supervises the producer

24
 1. Planning
• Moderator establishes the conduct and progress for the
entire inspection. This requires that the moderator :
– Assure the identification of the inspection team
– Assure that the team members will be able to adequately prepare for
the inspection
– Assure that the materials to be inspected are available and conform
to standards
– Determine whether the entry criteria have been met
– Determine the need for an overview
– Assure that the place for the inspection meeting is available and
reserved for the inspection.
– Schedule the inspection meeting time and place
– Give all inspection team members and other interested parties
notice of the inspection meeting time and place.
– Assign the role of reader to a selected member of the inspection
team
 2. Overview
• An overview meeting is an educational meeting usually
conducted prior to a design inspection
• It is a short meeting in which the author of the product to
be inspected gives the inspection team members and
others who will be interfacing with author’s product, a
brief description of the software, what task is being
performed, how it will perform that task, what interfaces
are to be active, and a description of the interface
functions.
• The overview meeting is held at the beginning of the
preparation phase and it is at this meeting, if held that the
moderator gives the inspection materials to the
inspection team members and also gives them the
inspection meeting notice.
 3. Preparation
• The preparation phase begins when the inspection team
members receive the inspection materials and the notice
of the inspection meeting.
• The reader prepares to present the material to the
inspection team.
• The reader make particular note of any difficulty in
understanding the design, code or commentary.
• Each inspector examines the material for all possible
defects.
• The defects are recorded on the inspection-defect log
form.
• Each inspector also keep track of his/her preparation
time and records it on the inspection defect log
4. Inspection meeting
• The team members come together to discuss the
discrepancies which have been detected.
• The moderator is responsible for the proper conduct of
the meeting and assuring that the team members
approach this task in a professional manner
• The reader is responsible for presenting the product to
the team in a logical and orderly manner.
• During this phase, the moderator calls the meeting to
order, records the preparation time for each team
member and directs the reader to begin the discussion.
• As defects are identified, they are discussed and
recorded.
• The activity during the meeting is limited to finding
defects, not solution.
4. Inspection meeting (cont.)
• When defects are repeated in a product, the team should
wait until reader gets to the place each repeated defect is
found before mentioning its repetition
• When the inspection meeting is over, the moderator
collects the individual defects log from the team.
• Moderator need to decide whether re-inspection is
required or not.
• When the inspection has been dismissed, the moderator
summarizes on the defect summary log form the defects
found, records the preparation and inspection time, notes
the author, requirement for re-inspection, type of
inspection and so on.
 5. Rework
• During the rework phase, the author examines the defects
found and makes the necessary corrections.
• After the corrections are verified, the author discusses the
corrections with the moderator.
• If the moderator determines that a reinspection is required,
the author begins preparation for the reinspection.
6. Follow-up
• To assure that all of the defects detected during the
inspection have been corrected.
• The moderator is responsible for the activity in this phase
and the inspection is not completed until the follow-up
phase has been completed.
• When verification is completed, the moderator sends a
copy of the summary with the notation that this is a fix
notification.
• When this last task is accomplished, the task of the
moderator for the inspection is completed, and the
inspection itself for the software product is also
completed.
 Type of inspection
• Producer and manager make the choice.
– Critical product functions.
– Complex modules.
– Modules that have been “problematic” in the past.

32
33
 Requirements Inspections
“If you can only afford to do one inspection on a
project, you will get the biggest return on
investment from a requirements inspection. A
requirements inspection should be the one
inspection that is never skipped.”
- Steven R. Rakitin

34
 Attributes of Good Requirements
Specifications
• Unambiguous
• Complete
• Verifiable
• Consistent
• Modifiable
• Traceable
• Usable

35
 Requirements Inspections
• Objectives
– Is every requirement traceable to the preceding
document? Is every requirement clear and concise,
internally consistent, unambiguous,
testable/demonstrable?
➢ Make sure each requirement in the Software Requirements
Specification (SRS) is consistent and traceable to the document that
preceded the SRS
➢ Make sure each requirement in the SRS is clear, concise, internally
consistent, unambiguous, and testable

36
 Requirements Inspection
Prerequisites
• All inspection team members must receive
appropriate training
• The document(s) that preceded the SRS must have
been reviewed and approved
• The SRS must have been internally reviewed
• A requirements inspection checklist must be
available
• Guidelines for a good SRS must be available

37
 Requirements Inspections (2)
• Planning phase.
– Diversity of inspector’s backgrounds, include clients (if
possible).
– Identification of the SRS, other documentation.
• Preparation phase.
– Self-study, inspector-author discussions.
• Inspection meeting.
– Checking preparedness, discussing discrepancies.
• Follow-up phase

38
 Design Inspection
• Objectives
– SRS-SDD compliance, traceability, design
conformance to standards.
• Prerequisites
– SRS has been inspected and completed, SDD
internally reviewed, availability of checklists,
availability of design documentation (CASE
tools).

39
 Design Inspection (2)
• Planning phase.
– Inspector’s backgrounds include software engineers, QA,
hardware engineers.
– Identification of the SRS, SDD other documentation.
• Overview meeting phase.
– Producer’s presentation, inspectors ask questions.
• Preparation phase.
• Inspection meeting.
– Checking preparedness, discussing discrepancies, going
through checklists.
• Follow-up phase
40
 Code Inspections
• Objectives and Prerequisites
– SDD inspected and accepted.
– Compiled code.
• 50-100 lines of C code per hour of
preparations.
• 100-200 lines per hour of inspection.

41
 Test Script Inspections
• Objectives:
– Accurate validation of requirements in SRS, taking
advantage of known design decisions.
• Prerequisites:
– Internally reviewed and executed tests/scripts,
checklists, acceptable test results.

42
Inspection Defect Types and
Definitions
• The types of defects and their definitions follow:
– Design defect – function description does not meet the
requirements specification
– logic defect – data is missing; wrong or extra
information
– syntax defect – does not meet the software standards
requirements.
– Data defect – missing, extra, or wrong data definition
or usage
Inspection Defect Types and
Definitions…
– Interface defect – incompatible definition/ format of
information exchanged between two module
– Return code/message defect – incorrect or missing
values/ message sent
– Prologue/message – the explanation accompanying
the design/ code language is incorrect, inexplicit or
missing
– Requirements change defect – change in the
requirements specification which is the direct and
proximate reason for the required change in the design
or code.
– Performance improvement defect – code will not
perform in the amount of time/space/CPU allowed.
 Defect Category in SRS
• Major—A defect that if not corrected prevents the product
from meeting its requirements. Requires correction before
the work product may be promoted to the next phase in the
lifecycle.
• Minor—A defect that if not corrected means the product
will not meet specifications, design, or standards completely,
but will not prevent the product from otherwise operating
or serving its purpose.
• Grammatical—format/spelling/grammar errors or other
typos that do not affect meaning
• Question – statement that need clarification by the author

45
 Inspection Initiation
• Inspection are initiated upon the completion of software
design, either high or low level, or upon the completion of
the first clean compilation of code.
• Developers should not spend any time doing desk-
checking of the product if these conditions have been
met.
• The inspectors do not really care how many defects are
found.
• They are there to do a job, do it professionally and then
return to their assigned task.
• If the defects are not found in inspections, they must be
found in test.
 Inspection Prerequisites
• The requirements for the proper conduct of an inspection
are:
– A team of technically competent, trained inspectors.
– A trained moderator
– Proper planning and distribution of material.
– A good professional attitude
– Full preparation prior to the inspection meeting
– Completed design or cleanly compiled code
– Updated resource requirements
Inspection Teams
• Establishing the team
– The moderator has the responsibility for identifying
the inspection team.
– The inspection team should be limited to no more
than five(5) people unless special conditions warrant
an increase.
• Inspection team and duties
– Inspection team include the moderator, the author and
depending on which type of inspection.
– Responsibilities of the moderator
• Prior to the inspection meeting
Inspection Teams…
– Responsibilities of the moderator (continue)
• Completion of the SWQE training course
• Determination whether the entry criteria for this
level of inspection has been met
• Work with author to establish the team
membership.
• Preview the material for conformance to standard
• Insure the team size and mix is proper.
• Insure there are at least 5 working days of
preparation
• Insure proper materials are distributed
Inspection Teams…
• During the inspection meeting
– assure adequate attendance
– assure adequate preparation; if not, postpone to a
later time.
– Lead the inspection meeting
– Log defects and all open items.
– Require inspection for major defects
Inspection Teams…
• After the inspection meeting
– Review result with author
– Provide manager with estimate of the rework
completion data.
– Eliminate duplicate defect log entries and send
inspection summary to SWQE.
– Add inspection summary and detail report to the
software quality notebook kept by the department or
author.
– Verify correction notice to quality notebook and send
to SWQE.
– Log open items in software quality notebook or open
issues log.
Responsibilities of the author
• Prior to the inspection
– Prepare material to address all inspection level
checklist items.
– Review material with moderator for completeness
– Provide cover page with all included material.
– Prepare for review if one is to be held.
– Work with moderator to schedule time and place for
meeting
– Work with moderator to establish team membership
– Produce materials distribution package in timely
manner.
 Responsibilities of the author…
• After the inspection meeting
– Complete all rework required
– Verify fixes will correct problem and not cause any
additional problem.
– Verify to moderator that changes have been made.
Responsibilities of the reader
• Guide the team through the material during the meeting ;
paraphrase or verbalize the review material
• Present material with clarity and understand
• Be able to tie-back to specification or design
• Fulfill normal inspector responsibilities.

Note: The reader should not be the author

Responsibilities of all inspectors
• Attend the SWQE training class
• Thoroughly review all material against the checklist
• Assure understanding of function; consult with author if
necessary.
• Record detected defects on inspection defect log form
prior to the inspection meeting
• Record the inspection-meeting preparation time.
Responsibilities of the manager
• Establish schedules which allow adequate review time
and resulting follow-up.
• Insure all team members are aware of inspection
procedures
• Meet with moderator and author to review open items
and obtain rework estimate
• Monitor individual inspection time.
• Review SWQE defect summary report for defect trends
and perform defect-trends analysis.
 Mechanics of inspections
• The team must reach consensus on issues recorded as
errors and defects.
– Error is a problem (lack of compliance with the
requirements) identified at the point of origin.
– Defect is found beyond the point of origin (ex. Design
problem identified in code).
– The author doesn’t get to vote.

57
 Mechanics of inspections (2)
• Inspection meetings limited to 2 hours.
• Posting results of individual inspection meetings is
controversial.
– Consider posting aggregate results.
– Supports quality improvement without personalizing the
guilt.
• Inspection is complete when all the problem reports
are closed.

58
Topic Discussion

59
Topic Discussion

60