Professional Documents
Culture Documents
SQA-ppt II Unit
SQA-ppt II Unit
SQA-ppt II Unit
Software Quality
Assurance:
III Quality Control
Dr. Holger Giese
Software Engineering Group
Room E 3.165
Tel. 60-3321
Email: hg@upb.de
University of Paderborn
Software Engineering Group
Outline
I Introduction
II Software Life Cycle
III Quality Control
IV Infrastructure
V Management
VI Standards
VII Conclusion & Summary
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-2
University of Paderborn
Software Engineering Group
III.1 Foundations
Quality Control is the series of
inspections, reviews and tests used
throughout the development cycle to
ensure that each work product meets the
requirements placed upon it.
[Pressman2004]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-4
University of Paderborn
Software Engineering Group
Boehm [Boehm81]:
Verification: “Are we building the product right?”
Validation: “Are we building the right product?”
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-6
University of Paderborn
Software Engineering Group
Static analysis:
The process of evaluating a system or component based on
its form, structure, content, or documentation.
Contrast with: dynamic analysis.
See also: inspection; walk-through.
[IEEE-610.12-1990]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-7
University of Paderborn
Software Engineering Group
Terminology: Testing
Several definitions:
“Testing is the process of executing a program or system with the intent of finding errors.”
[Myers1979]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-8
University of Paderborn
Software Engineering Group
Static analysis:
investigation without operation;
pencil and paper reviews etc.
Modelling (mathematical representation)
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-9
University of Paderborn
Software Engineering Group
[Storey1996]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-10
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-11
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-12
University of Paderborn
Software Engineering Group
Purpose/Objectives [SWENE]
Verify that
software meets its requirements
software follows predefined standards
software is developed in uniform manner
Catching errors
Sooner
More and different
Breaking frame of reference
Make projects more manageable
To identify new risks likely to affect the project
Improving communication
Crossing organization boundaries
Providing Education
Making software visible
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-13
University of Paderborn
Software Engineering Group
Features
Informal
Done by the producer
Implications
Not objective
Available to any developer
Different mindset
Need for review Limited screening efficiency!
Product completion
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-14
University of Paderborn
Software Engineering Group
Features
Team reviews materials separately
Team and producers meet to discuss
May review selected product aspects only
Implications
Focus on important issues
If you know what they are
More material per meeting
Less preparation time
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-15
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-16
University of Paderborn
Software Engineering Group
Features
Lessformal
Producer presents or provides information
Implications
Larger groups can attend (education)
More material per meeting
Less preparation time
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-17
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-18
University of Paderborn
Software Engineering Group
Inclusion: Omission:
“Straightforward” sections
Sections of (no complications)
complicated logic Sections of a type already
reviewed by the team in
Critical sections, similar past projects
where defects Sections that, if faulty, are
not expected to effect
severely damage functionality
essential system Reused design and code
Repeated parts of the
capability design and code
Sections dealing
with new
environments
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-19
Sections designed
University of Paderborn
Software Engineering Group
Features
Formal
Scheduled event
Defined procedure
Reported result
Independent review team
Producers not present
Implications
More preparation time
Less material per meeting
Product must stand or fall on its See also: IEEE Standard for
own Software Reviews and Audits
[IEEE 1028, 1988]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-20
University of Paderborn
Software Engineering Group
Managers
Manager assigns
Vestedinterest in a good outcome
Review as delegation of manager’s responsibility
Technical competence
Current technology
Objectivity
Best buddies and “outsiders”
User involvement
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-22
University of Paderborn
Software Engineering Group
Smaller for
Focus
Scheduling
Reasonable
Larger for
output volume
per person-hour 3 6
Expertise
Making review public
Non-participating
observers
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-23
University of Paderborn
Software Engineering Group
Team Characteristics
Experienced senior technical staff
Representatives of
team that created the document
client representative
team for next development phase
software quality assurance group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-24
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-25
University of Paderborn
Software Engineering Group
Scheduling reviews
10 AM
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-26
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-27
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-28
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-29
University of Paderborn
Software Engineering Group
Technical competence
General strength
Credibility
Able to understand the issues
Personal skills
Willing to confront people
Willing to report failure
Able to step back from the heat of discussion
Administrative skills
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-30
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-31
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-32
University of Paderborn
Software Engineering Group
Act as facilitator
Controlling level of participation
Enough but not too much
Conflict resolution
Terminate the meeting if unproductive
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-33
University of Paderborn
Software Engineering Group
Prepare before
Thorough review of materials
Participate
Be there
Coming late; leaving early
Act professionally
Personal agendas
Big egos and shyness
Reviewer - Preparation
Be prepared - evaluate product before review
be sure that you understand the context
first, skim all the product material to understand
location and format of the information
next, read product material and annotate hardcopy
avoid issues of style
inform the review leader if you can’t prepare
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-35
University of Paderborn
Software Engineering Group
Reviewers – Participation
Review the product, not the producer
keep your tone mild, ask questions instead
of making accusations
stick to the review agenda
pose your written comments as questions
raise issues, don’t resolve them!
limit discussions (do them off line!)
avoid discussions of style - stick to technical
correctness
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-36
University of Paderborn
Software Engineering Group
Recorder [SWENE]
Selection
Any competent reviewer
Single or multiple recorders
Rotating responsibility within a meeting
Leaders as recorders
Having too much to do
Separation of power
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-37
University of Paderborn
Software Engineering Group
Issues
PublicVs. private notes
Speed and accuracy
Usefulness after the meeting
Media
Flipcharts; posting prior pages
Blackboards, overheads, PC and projector
Video and audio recording
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-38
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-39
University of Paderborn
Software Engineering Group
Process [Galin2004]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-40
University of Paderborn
Software Engineering Group
Purpose
Tellmanagers the outcome
Early warning system for major problems
Provide historical record
For process improvement
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-41
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-42
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-43
University of Paderborn
Software Engineering Group
Post-Review Activities
Prepare the review report, including the action items
Establish follow-up to ensure the satisfactory performance of
all the list of action items
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-44
University of Paderborn
Software Engineering Group
Comparison [Galin2004]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-45
University of Paderborn
Software Engineering Group
TYPES OF REVIEWS IEEE STD 1028-1988
Category/Attribute Management Technical Inspection Walkthrough
Objective Ensure Progress. Evaluate conformance to Detect and identify Detect defects.
Recommend corrective specifications and plans. defects. Examine alternatives.
action. Ensure proper Ensure change integrity. Verify resolution. Forum for learning.
allocation of resources.
Delegated Controls
Decision making Management team charts Review team petitions Team chooses All decisions made by
course of action. management of technical predefined product producer. Change is the
Decisions leadership to act on dispositions. Defects prerogative of the
are made at the meeting recommendations. must be removed. producer.
or as a result of
recommendations.
Change verification Change verification left to Leader verifies as part of Moderator verifies rework.Change verification left to
other project controls. review report. other project controls.
Group Dynamics
Recommended size 2 or more people 3 or more people 3-6 people 2-7 people
Attendance Management, technical Technical leadership and College of peers meet Technical leadership and
leadership, and peer mix. peer mix. with documented peer mix.
attendance.
Leadership Usually the responsible Usually the lead engineer. Trained moderator. Usually producer.
manager.
Procedures Moderate to high, Moderate to high,
Material volume depending on the specific depending on the specific Relatively low. Relatively low.
“statement of objectives” “statement of objective”
at the meeting. for the meeting.
Presenter Project representative Software element Presentation by “reader” Usually the producer
representative. other than producer.
Data collection As required by applicable Not a formal project Formally required. Not a formal project
policies, standards, or requirement. May be requirement. May be
plans. done locally. done locally.
Outputs
Reports Management review Technical review report. Defect list. Defect Walkthrough report.
report. summary. Inspection
report.
Database entries Any schedule changes No formal database Defect counts, No formal database
must be entered into the required. characteristics, severity, required.
project tracking database. and meeting attributes
are kept.
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-46
University of Paderborn
Software Engineering Group
Inspections “Proof of
correctness”
Walkthroughs
“Detection of flaws”
Personal Review team-oriented
“Improved design”
individual initiative
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-47
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-49
University of Paderborn
Software Engineering Group
Effectiveness of Inspections
[Fagan 1976] inspections of design & code
67%-82% of all faults were found by inspections
25% time saved on programmer resources (despite
inspections)
[Fagan 1986]
93% of all faults were found by inspections
Cost reduction for fault detection
(compared with testing)
[Ackerman+1989]: 85%
[Fowler1986]:90%
[Bush1990]: 25.000US$ saved PER inspection
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-50
University of Paderborn
Software Engineering Group
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-51
University of Paderborn
Software Engineering Group
[Collofello&Woodfield1989]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-52
University of Paderborn
Software Engineering Group
Summary [SWENE]
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-53
University of Paderborn
Software Engineering Group
Further Readings
[Ackerman+1989] A.F. Ackerman, L.S. Buchwald, F.H. Lewski, “Software inspections: an effective verification process”, IEEE
Software 6 (May 1989), pp. 31-36
[Bush 1990] M. Bush, “Improving software quality: the use of formal inspections at Jet Propulsion laboratory”
Proceedings of the 12th International Conference on Software Engineering, Nice, France, March 1990, pp.
196-199
[Collofello&Woodfield1989] J.S. Collofello and S. N. Woodfield. “Evaluating the Effectiveness of Reliability-Assurance
Techniques.” Journal of Systems and Software, March 1989.
[Cusumano1991] M. A. Cusumano. Japan‘s Software Factories – A Challange to u.S. Management. Oxford University
Press, New York, 1991.
[Dobbins1998] J. H. Dobbins. Inspections as an up-front quality technique. In G. G. Schulmeyer and J. i. McManus (eds),
handbook of Software Quality Assurance, Prentice Hall, Harlow, Essex, UK, 1998.
[Doolan1992] Doolan, E. P. “Experience with Fagan’s Inspection Method,” Software-Practice and Experience, 22, 2,
February 1992: 173-182.
[Fagan 1976] M.E. Fagan, “Design and code inspections to reduce errors in program development”, IBM Systems
Journal 15 (No. 3, 1976), pp. 182-211
[Fagan 1986] M.E. Fagan, “Advances in software inspections”, IEEE Transactions on Software Engineering, SE-12 (July
1986), pp. 744-751
[Fowler 1986] P.J. Fowler, “In-process inspections of workproducts at AT&T”, AT&T Technical Journal 65 (March/April
1986), pp. 102-112
[Kelly+1992] J. C. Kelly, J. S. Sherif, and J. Hops. “An Analysis of Defect Densities Found During Software
Inspections.” Journal of Systems and Software, 1992.
[Parnas&Weiss1985] Parnas, D. L. & Weiss, D. M. “Active Design reviews: Principles and Practices,” Proceedings of the 8th
Conference on Software Engineering, pp. 215-222, August, 1985.
[Parter+1994] Porter, A. A.; Votta, Jr., L. G.; & Basili, V. R. Comparing Inspection Methods for Software Requirements
Inspections: A Replicated Experiment, University of Maryland Technical Report (CS-TR-3327), 1994. [E&S
Tech Rept MDU 3327]
[Weller1993] E. F. Weller. “Lessons From Three Years of Inspection Data.” IEEE Software, Sept. 1993.
Dr. Holger Giese WS04/05 – Software Quality Assurance – III Quality Control III-54