Formal Inspections

 Types of Inspection
 Benefits of Inspection
 Inspection is more cost effective than testing

 How to conduct an inspection

 who to invite
 how to structure it

 Some tips
 Reviews, Walkthroughs, Inspections…
 “Management reviews”
 E.g. preliminary design review (PDR), critical
design review (CDR), …
 These
These definitions
definitions are
are not
not
 Used to provide confidence that the design is widely
widely agreed!
agreed!
sound
Other
Otherterms
termsused:
used:
 Attended by management and sponsors (customers)
Formal Technical Reviews (FTRs)
 Often just a “dog-and-pony show” Formal Technical Reviews (FTRs)
 Formal Inspections
 Formal Inspections
 “Walkthroughs” “Formality”
“Formality” can
can vary:
vary:

 developer technique (usually informal)
 used by development teams to improve quality of informal:
informal:
product  meetings over coffee,
 meetings over coffee,
 focus is on finding defects  regular team meetings,
 regular team meetings,
 etc.
 etc.
 “(Fagan) Inspections” formal:
formal:
 a process management tool (always formal)  scheduled meetings,
 scheduled meetings,
 used to improve quality of the development prepared participants,
prepared participants,
process defined agenda,
defined agenda,
 collect defect data to analyze the quality of the  specific format,
 specific format,
process  documented output
 documented output
 written output is important
 major role in training junior staff and transferring
expertise
 Benefits of formal inspection
Source: Adapted from Blum, 1992, Freedman and Weinberg, 1990, & notes from Philip Johnson.

 Formal inspection works well for programming:

 For applications programming:
 more effective than testing
 most reviewed programs run correctly first time
 compare: 10-50 attempts for test/debug approach
 Data from large projects
 error reduction by a factor of 5; (10 in some reported cases)
 improvement in productivity: 14% to 25%
 percentage of errors found by inspection: 58% to 82%
 cost reduction of 50%-80% for V&V (even including cost of inspection)
 Effects on staff competence:
 increased morale, reduced turnover
 better estimation and scheduling (more knowledge about defect profiles)
 better management recognition of staff ability

 These benefits also apply to requirements inspections

 Many empirical studies investigated variant inspection processes
 Mixed results on the relative benefits of different processes
 Inspection Constraints
Source: Adapted from Blum, 1992, pp369-373 & Freedman and Weinberg, 1990.

 Size  Scope
 “enough people so that all the  focus on small part of a design, not
relevant expertise is available” the whole thing
min: 3 (4 if author is present)  Fagan recommends rates:
max: 7 (less if leader is inexperienced) 130-150 SLOC per hour

 Duration  Timing
 never more than 2 hours  Examines a product once its author
concentration will flag if longer has finished it
 not too soon
 Outputs product not ready - find problems the
 all reviewers must agree on the author is already aware of
result  not too late
accept or re-work or re-inspect product in use - errors are now very
 all findings should be documented costly to fix
summary report (for management)
detailed list of issues
 Purpose
 Remember the biggest gains come
from fixing the process
collect data to help you not to make
the same errors next time
 Choosing Reviewers
Source: Adapted from Freedman and Weinberg, 1990.

 Possibilities
 specialists in reviewing (e.g. QA people)
 people from the same team as the author
 people invited for specialist expertise
 people with an interest in the product
 visitors who have something to contribute
 people from other parts of the organization

 Exclude
 anyone responsible for reviewing the author
 i.e. line manager, appraiser, etc.
 anyone with known personality clashes with other reviewers
 anyone who is not qualified to contribute
 all management (?)
 anyone whose presence creates a conflict of interest
 Roles
Source: Adapted from Blum, 1992, pp369-373

Formal Walkthrough Fagan Inspection

 Review Leader  Moderator
 chairs the meeting  must be a competent programmer
 ensures preparation is done  should be specially trained
 keeps review focussed  could be from another project
 reports the results
 Designer
 Recorder  programmer who produced the design
 keeps track of issues raised being inspected

 Reader  Coder/Implementor
 summarizes the product piece by  programmer responsible for
piece during the review translating the design to code
 Author  Tester
 should actively participate (e.g. as  person responsible for
reader)
writing/executing test cases
 Other Reviewers
 task is to find and report issues
 Guidelines
Source: Adapted from Freedman and Weinberg, 1990.

 Prior to the review

 schedule Formal Reviews into the project planning
 train all reviewers
 ensure all attendees prepare in advance

 During the review

 review the product, not its author
 keep comments constructive, professional and task-focussed
 stick to the agenda
 leader must prevent drift
 limit debate and rebuttal
 record issues for later discussion/resolution
 identify problems but don’t try to solve them
 take written notes

 After the review

 review the review process
 Opening Moments
Source: Adapted from Wiegers 2001.

1) Don’t start until everyone is present

2) Leader announces:
 “We are here to review product X for purpose Y”

3) Leader introduces the reviewers, and explains the

recording technique
4) Leader briefly reviews the materials
 check that everyone received them
 check that everyone prepared

5) Leader explains the type of review

Note: The review should not go ahead if:
 some reviewers are missing
 some reviewers didn’t receive the materials
 some reviewers didn’t prepare
 Structuring the inspection
 Checklist
 uses a checklist of questions/issues
 review structured by issue on the list

 Walkthough
 one person presents the product step-by-step
 review is structured by the product

 Round Robin
 each reviewer in turn gets to raise an issue
 review is structured by the review team

 Speed Review
 each reviewer gets 3 minutes to review a chunk, then passes to the next
person
 good for assessing comprehensibility!
 Fagan Inspection Process
Source: Adapted from Blum, 1992, pp374-375

1 Overview 4 Rework
 communicate and educate about  All errors/problems addressed by
product author
 circulate materials  Rate: 16-20 hours per 1000 SLOC
 Rate: 500 SLOC per hour
5 Follow-up
2 Preparation  Moderator ensures all errors have
 All participants perform individually been corrected
 review materials to detect defects  if more than 5% reworked, product is
 Rate: 100-125 SLOC per hour re-inspected by original inspection
team
3 Inspection
 a reader paraphrases the design
 identify and note problems (don’t
solve them)
 Rate: 130-150 SLOC per hour
 Tactics for problematic review meetings
 Devil’s advocate
 deliberate attempt to adopt a contrary position

 Bebugging
 put some deliberate errors in before the review
 with prizes for finding them!

 Money bowl
 if a reviewer speaks out of turn, he/she puts 25c into the drinks kitty

 Alarm
 use a timer to limit ‘speechifying’

 Issues blackboard
 appoint someone to keep an issues list, to be written up after the review

 Stand-up review
 no tables or chairs!
 Summary
 Inspections are very effective
 Code inspections are better than testing for finding defects
 For Specifications, inspection is all we have (you can’t “test” a spec!)

 Key ideas:
 Preparation: reviewers inspect individually first
 Collection meeting: reviewers meet to merge their defect lists
 Log each defect, but don’t spend time trying to fix it
 The meeting plays an important role:
 Reviewers learn from one another when they compare their lists
 Additional defects are uncovered
 Defect profiles from inspection are important for process improvement

 Wide choice of inspection techniques:

 What roles to use in the meeting?
 How to structure the meeting?
 What kind of checklist to use?
 References
Freedman, D. P. and Weinberg, G. M. “Handbook of Walkthroughs,
Inspections and Technical Reviews”. Dorset House, 1990.
Good practical guidebook, full of sensible advice about conducting reviews. Not so strong
on the data collection and process improvement aspects of Fagan inspections, though.

Ackerman, A. F. “Software Inspections and the Cost Effective Production

of Reliable Software”. From “Software Engineering”, Dorfman & Thayer,
eds., IEEE Computer Society Press, 1997.
This paper summarizes some of the practical aspects of introducing inspections, including
how inspectors are trained.

Karl E. Wiegers, "Peer Reviews in Software: A Practical Guide", Addison-

Wesley, 2001
We’ll be using the forms from this book for the practical inspection exercise.

Blum, B. “Software Engineering: A Holistic View”. Oxford University

Press, 1992
Section 5.2 provides one of the best overview of walkthroughs and inspections anywhere.
Blum manages to cut through a lot of the confusion about ‘walkthroughs’, ‘inspections’ and
‘reviews’ managing to get to the key issues.