Process Improvement

CIS 376
Bruce R. Maxim
UM-Dearborn
 Process Improvement Goals
Understanding existing processes
Introduce process changes to improve quality,
reduce costs, or accelerate schedules
Industry is demanding increased attention to
quality in general
Most process improvement work focuses on defect
reduction and prevention
There are other process attributes that deserve our
attention
 Process Improvement Attributes - part 1

Understandability - degree to which a process is

well defined and understood
Visibility - process activities have results that are
externally recognizable
Supportability - process activities supported by
CASE tools
Acceptability - defined processes are used and
accepted by software engineers
Process Improvement Attributes - part 2

Reliability - process is defined so that errors are

avoided or trapped before product errors result
Robustness - process can continue despite
unexpected problems
Maintainability - process can evolve to reflect
changing organizational requirements or identified
process improvements
Rapidity - the time required to complete a system
from specification to delivery
 Process Improvement Stages
Process analysis
modeling and quantitative analysis of existing processes
Improvement identification
quality, cost, and scheduling bottlenecks located
Process change introduction
modify process to remove bottlenecks
Process change training
train staff involved in process revision proposals
Change tuning
process improvements are revised and allowed to evolve
 Process Improvement Activities

Introduce
processchange
Analyse Identify Tune
process improvements processchanges
Train
engineers

Process Processchange Training Feedbackon Revisedprocess

model plan plan improvements model
 Process and Product Quality
Closely related to one another
Good processes are usually required to
produce good products
In manufacturing applications, process is
principle determinant of quality
For design-based activities, the capabilities
of the designers are also important
 Product Quality Factors
Development technology
for large projects with average capability this is the main
determinant of product quality
Quality of people involved
for small projects the developer capability is the main determinant
of product quality
Process quality
significant for both small and large projects
Cost, time, and schedule constraints
unrealistic schedules can doom the quality of most products
 Process Analysis and Modeling
Process analysis
study of existing processes to understand relationships among
process components
allows comparisons with other processes
Process modeling
documentation of process in which the tasks, roles, and entities
used are recorded
best to represent models graphically
several different perspectives may be used (e.g. activities,
deliverables, etc.)
model should be examined for weaknesses, this involves
discussion with stakeholders
 Process Model Elements - part 1
Activity - (round edged rectangle)
has clearly defined objective, entry, and exit conditions
Process - (round edged rectangle with shadow)
set of coherent activities with agreed upon objective
Deliverable - (rectangle with shadow)
tangible output of an activity predicted by project plan
Condition - (parallelogram)
process or activity pre- or post-conditions
 Process Model Elements - part 2
Role - (circle with shadow)
defined and bounded area of responsibility
Exception - (double edged box))
description of how to modify the process if anticipated
or unanticipated events occur
Communication - (arrow)
exchange of information between people and/or
machines
 Process Model Example
Rle
Postcond ition
Precondition Test Alldefinedtests
engineer runonmodule
Modulecompiles
withoutsyntax
errors Responsible
for Outputs
Test Signedof f test
module record
Input
Process
Module Moduletest
specification data
 Process Exceptions
Process models cant represent how to handle
exceptions
key people are lost prior to a critical review
failure of e-mail server for several days
organizational reorganization
request to respond to change requests
General procedure is to suspend the process model
and follow RMMM plans augmented with the
managers own initiatives
 Process Measurement
Wherever possible quantitative process data should
be collected
Organizations without process standards may have to
be define processes before measurements can be
made (since they wont know what to measure)
Process measurements should be used to assess
process improvements
Organization objectives drive process improvement,
not measurements
 Process Measurement Classes

Time taken to complete process activities

e.g. calendar time to complete a milestone
Resources required to complete processes
or activities
e.g. person months
Number of event occurrences
e.g. number of defects found
 Goal Question Metric Paradigm
Goals
What is the organization trying to achieve?
Process improvement deals with goal satisfaction.
Questions
Concerned with areas of uncertainty related to goals.
You need process knowledge to derive questions.
Metrics
Measurements collected to answer questions
 SEI Process Maturity Model
Level 1 - Initial
essentially uncontrolled
Level 2 - Repeatable
project management procedures defined and used
Level 3 - Defined
process management strategies defined and used
Level 4 - Managed
quality management strategies defined and used
Level 5 - Optimizing
process improvement strategies defined and used
 SEI Process Model Problems
Focuses on project management rather than project
development
Ignores the use of strategies like rapid prototyping
Model is intended to represent organizational
capability and not practices used on particular
projects
There may be wide variation in the practices used in
a single organization
Capability assessment is questionnaire-based
 Capability Assessment Process

Selectprojects Distribute Analyse Clarify

forassessment questionnair es responses responses

Identifyissues Interview Interview Interview

fordiscussion projectmanagers engineers mana gers

Brief managers Present Writereport

andengineers assessment
 Process Classification
Informal
No detailed process model, developers created their own
way of doing things
Managed
defined model drive development process
Methodical
processes supported by standard development method
Supported
processes supported by automated CASE tools
 Process Tool Support

Informal Managed Methodical Improving

process process process process

Configuration Project Analysisand Specialized

Generic management design
management tools
tools tools workbenches
tools
 Defect Removal Effectiveness

Defect removal is central to software

development
One of the top expense items
Affects project scheduling
Improves product quality
 PSP - Defect Density
This is the primary defect measure used in
PSP

Dd = 1000 * D/N

D = total number of defects found in all

phases of the process
N = number of new and changed lines of
code in the program
 Defect Density Example
For a program with 96 new or changed lines
of code and 14 defects

Dd = 1000 * (14/96) = 145.83 defects/KLOC

 Defect Metrics - part 1
Error Detection Efficiency
100%*(#errors found in 1 inspection)/(#errors in product before inspection)

Defect Removal Efficiency

100%*(#defects found now)/(#defects found now + #defects found later)

Error Detection Percentage

100%*(#inspection errors)/(#inspection errors + #valid discrepancy reports)
 Defect Metrics - part 2
Total Defect Containment Effectiveness (TDCE)
(#prerelease defects)/(#prerelease defects + #post-release defects)

Phase Containment Effectiveness (PCE)

(#phase(i) defects)/(#phase(i) defects + #phase(i+x) defects)

Effectiveness (E)
100%*N/(N + S)
N = #defects found by an activity
S = #defects found in subsequent activities
Phase-based Defect Removal Model
Defects present at exit of each development phase
are estimated
This allows us to set realistic targets and assess the
costs of reducing error injection rates
This is a quality management tool and not a device
for estimation of software reliability
How would this work in practice?
 Assumptions
Suppose we decide to create two broad
defect removal classes
activities that handle defects before code is
integrated into the system library (design
reviews, inspections, unit testing)
formal machine tests after code integration
Also assume the same defect removal
effectiveness for each phase
 Example - part 1
MP = major problems found in before integration
PTR = errors found during formal machine tests
mu = MP/PTR
the higher the value of mu the better
Q = defects found after release to customer
TD = (MP + PTR + Q)
total defects for life of software
 Example - part 2
Phase 1 effectiveness
E1 = MP/TD
MP = E1 * TD
Phase 2 effectiveness
E2 = PTR/(TD - MP)
PTR = E2 * (TD - MP)
 Example - part 3

Some equations that can be useful in quality

planning (assuming that E1 = E2)
Q = PTR /(mu - 1)
Q = MP / [mu * (mu - 1)]
Q = TD / (mu * mu)
These equations work with either raw or
normalized defect values
 PSP Phase Yield
Phase yield =
100 * (defects removed during phase)/
(defects in product at phase entry)

Note: cannot be computed until project is

completed
 Phase Yield - Example
5 defects found during code review
3 defects found during compile
2 defects found during unit testing
2 defects found during integration testing

Phase yield for compile =

100 * 3 / (3 + 2 + 2) = 42.9 %
Phase yield for code review =
100 * 5 /(5 + 3 + 2 + 2) = 41.7 %
Seven Basic Software Quality Tools
Checklists (paper forms)
used to gather data for later analysis
used to confirm that process tasks are complete
both simple yes/no and branching questions
Seven Basic Software Quality Tools
Pareto Diagram
bar chart sorted in descending height order
vertical axis labeled with # defects
horizontal axis (nominal) labeled with defect
cause types
software defects tends cluster near related
causes
Seven Basic Software Quality Tools
Histogram
frequency bar graph
vertical axis is # defects
horizontal axis has ordinal or interval type
labels
Seven Basic Software Quality Tools

Flowchart
pictorial representation of a process
breaks down process into its constituent steps
can be useful in identifying were errors are
likely to be found in the system
Seven Basic Software Quality Tools
Scatter diagram (point plots)
used with correlation, regression, or statistical
modeling
vertical axis is # defects
horizontal axis some metric (e.g. McCabes
index)
Seven Basic Software Quality Tools
Run chart
line graph showing performance of dependent
variable (y) over time (x)
best used for trend analysis (e.g. arrival of
defects during formal machine testing)
can plot cumulative dependent variables (S
curves)
Seven Basic Software Quality Tools
Control chart
advanced form of run chart where capability is defined
upper and lower control limits (dashed lines) are drawn
to alert the user when dependent measure is out of
control
can plot cumulative dependent variables (S curves)
C chart based on # conforming or not
R chart based on subgroup ranges (max min)
X bar chart based on subgroup means
Control Chart (C)
Seven Basic Software Quality Tools
Cause and effect (fish bone) diagram
not widely used in software development, but can be
useful
shows effect between quality variable and the factors
affecting it
Fishbone Diagram