Software Quality Engineering

BS(SE)-VI

Dr. Assad Abbas

Department of Computer Science

COMSATS University Islamabad, Islamabad Campus
assadabbas@comsats.edu.pk
Outline
n Software process improvement
n Case studies

12/21/20 2
Process Fundamentals (Recap)
n Process: The means by which people, procedures,
and tools are integrated to produce a product (or an
end result)
n Software Process: The set of all tasks involved in the
production and evolution of a software product
5 tasks are organized and sequenced
5 tasks are performed in accordance with a procedure

12/21/20 3
Process Management Principles
n The quality of a product is largely determined by the
quality of the process used to build it [1]
n By extension, the quality of a software product is
largely determined by the quality of the software
process used for developing and maintaining it
n To improve the quality of a software product, the
process for producing it must improved
n FACT: the majority of software problems or the
causes of software crisis (e.g., budget overrun, lack
of quality, late delivery) are managerial, not technical
(Humphrey 1989)

12/21/20 4
Examples of Software Crisis and Problems
n A review of 17 major Department of Defense (DoD) software
projects revealed that (Humphrey 1993):
5 One project was not delivered for 7 years
5 No project was on time
n Nine DoD contracts totaling $6.8 million
5 47% software delivered but never used
5 29.7% software paid for but never delivered
5 19% software used but later reworked or abandoned
5 ≈ 3% software could be used after changes
5 ≈ only 2% software could be used as delivered

12/21/20 5
Examples of Software Crisis and Problems
n Industry results are not any better (Gibbs 1994)
5 For every six new large software systems put into operation, two
others are canceled
5 Average software project overruns its schedule by half; large projects
do worse
5 Many projects are terminated after millions of dollars invested, e.g.,
CONFIRM project (over $200 M), AAS ($144 M), DMV ($44.3 M)
5 DIA’s Baggage Handling System, delayed for more than a year at
$1.1 million/day in interests and operating costs

12/21/20 6
Characteristics of Immature Organizations
n Processes are generally unplanned during the
project
5 Specific processes are not rigorously followed
5 Schedules and budgets are routinely exceeded
5 The organization is conservative
5 Product quality suffers

12/21/20 7
CMM’s Five-Level Framework (A reorientation)
n Initial: unpredictable, poorly controlled
n Managed (Repeatable): basic process management
practices are established; organization can repeat
previously mastered tasked
n Defined: the software process for both management and
engineering activities is documented and well
understood
n Quantitatively Managed: detailed measures of the
software process and product quality are collected; both
are understood and controlled
n Optimizing: focus on process improvement; feedback
from piloting innovative ideas and technologies

12/21/20 8
How Maturity Affects Project Results?
n Level 1: Initial schedule and cost targets are typically
overrun
n Level 2: Managed (Repeatable) Plans based on past
performances are more realistic
n Level 3: Defined With well-defined processes,
performance improves
n Level 4: Quantitatively Managed Based on
quantitative understanding of process and product,
performance continues to improve
n Level 5: Optimizing Performance continuously
improves

12/21/20 9
How Maturity Affects Project Results?

12/21/20 10
Case Studies of Applying the CMM
n Studies based on 13 organizations (Herbsleb et al 1995)
n Organizations involved:
5 DoD contractors (e.g., Hughes Aircraft)
5 Commercial organizations (e.g., HP, AT&T, Bull HN, Schlumberger, TI)
5 Military organizations (e.g., OC–ALC)
n Data collected on organizational characteristics, SPI efforts, results of SPI
efforts, other elements

12/21/20 11
A Specific Case: The Hughes Aircraft
n Organization description
5 Software Engineering Division, 500 Employees
5 US DoD Contracts
n SEI Efforts
5 First assessment in 1987 — Level 2
5 Second assessment in 1990 — Level 3
n Costs
5 Level 2 to Level 3: 75 person-months
5 ≈$450,000 ($400,000 + $45,000)
5 ROI: First year benefits: $2,000,000

12/21/20 12
Lessons Learned
n Substantial increase in productivity (as much as
67%)
n Substantial improvement in quality

n High ROI

n Management involvement is important

n Developing action plan is essential

n Many intangible benefits (less stress, higher morale,

few crisis)
“Process improvement pays off and is cost-effective”

12/21/20 13
Another Case: DataStream Content Solutions (DSCS) [2]
n In 1999, DataStream Content Solutions (DSCS) began providing a
service that converts large data files from one format to another.
Some of their contracts require immediate data turnaround (for
example, within two hours), while others require them to convert
data at set intervals (for example, monthly). From an initial staff of
four, they’ve grown to 28, with plans to almost double in size over
the next few years. As with many small companies, the founders
were experts in their application domain with little experience in
computer software technology. From 1999 until 2003, they
expanded their business, with the company president providing
marketing support and the chief programmer providing
programming and project management support. By early 2003, they
realized that their company had grown too large to continue
handling new business in this manner. Both realized that they
needed to change their business model, but interestingly, each had
a different reason for the change.

12/21/20 14
DSCS: Why change a successful business?
n At DSCS, the president secured contracts and the chief
programmer built each system.
5 Each system had the same basic architecture: read in textual data,
error-check and convert the data, and write out text in a new format.
n Once the programmer had written the program, he gave it to an
analyst, whose job was to process each day’s textual input and
produce the required output.
n An error during a day’s run meant that either the input text or the
software contained an error. The company handled each error the
same way
n If the input data was incorrect, the analyst could change it and
continue processing. If fixing the data wasn’t possible, the chief
programmer had to fix and debug the source program. Because
some contracts required the company to process data by 7:30 each
morning, defects could be stressful for the staff

12/21/20 15
DSCS: Why change a successful business?
n Both the president and chief programmer saw the need for change.
n The president wanted the organization to be compliant with CMM Level 3.
5 New contracts might be available from customers who insisted on CMM
Level 3 from their suppliers. Such a rating seemed like a good discriminator
for a small company in their market place.
n On the other hand, the chief programmer saw a business with multiple
independent systems.
5 Each time a program failed, he had to modify and debug the program. He was
an expert in the application domain but not in software development.
5 The chief programmer saw himself as the potential single point of failure.
g For example, when he learned about syntax processing, context-free grammars,
and programming tools such as Lex and Yacc, he was amazed at how much easier
and more uniform they could make textual processing. He believed that a process
improvement approach should eliminate the multiple versions of software
conversion tools that the company was developing and would simplify
development.
n Expertise in effort estimation were also needed at the time of contract
bidding/securing

12/21/20 16
DSCS: Measures Taken
n System architecture
5 Centralized servers to manage the data to be processed
n Software Architecture
5 Most DSCS programs have a similar function: converting text
g So, redesigning the software to use a common structure (for
example, using Yacc to parse input data) should let programmers
handle multiple programs more easily.
g Or creating a single set of programs to produce the output
formats.
g Differentiating between software architectural design and a
software development improvement plan
u Several benefits of requirements specification, a software design
document, software test plan, configuration management, version
control, software change, file backup, archiving.

12/21/20 17
DSCS: Measures Taken
n Development Process
5 Recognition of key aspects
g understand the differences among system architecture, software
architecture, and supporting organizational and development processes
5 Commitment of Resources
g developing practices and software process improvement after assigning a
dedicated person to the cause
5 Process Asset Library
g DSCS spent time and effort to create a process assets repository for
organizational and project information
u A Web interface on the library makes it easier for end users to locate
information. Also, library links are included in work products
5 Periodic Reviews
g Monthly meetings to review the status of projects and changes.
5 Terminology and Process Definition Standards
5 Other Factors
g Managing the workflow
g Unplanned work management
12/21/20 18
DSCS: Success Factors
n The organization now understands that it needs to
address system architecture, software architecture,
and development processes if they are to grow
successfully.
n Their revised goal is to have new development
processes in place so that they can fully realize
gains next year. They’re beginning to rewrite existing
programs to use a common software architecture
n Mainly two types of success factors
5 Process related
5 Organizational

12/21/20 19
Success Factors
n Process-related Success Factors
5 Process isn’t just for the sake of process; rather, it should
address the organization’s business goals and not just CMM
compliance goals. For DSCS, it was a to implement a set of
software engineering best practices.
5 Ordering the process development and implementation
activities on the basis of benefits and process groupings that
support particular needs, rather than being strictly Key Process
Areas (KPA) focused.
5 You can’t sustain process improvement without dedicated
resources (even if only part-time).
5 First test newly developed processes on selected projects, then
refine these processes on the basis of the lessons you learned
and expand them to other projects in the organization.
5 Plan the introduction of new processes; consideration for
ongoing activities is a determining factor.
12/21/20 20
Success Factors
n Organizational Success Factors
5 Identified 4 goals and 10 KPAs from the CMM to achieve the
goals
5 Goal1
g to improve the company’s ability to accurately forecast software
development projects’ costs and schedules.
u use project plans that include work breakdown structure, schedule,
and estimates for initial projects and implement a change control
process to manage scope, schedule, and cost changes to existing
projects.
5 Goal1
g to improve the company’s ability to reduce software development
time to market.
u software cost, schedule, and progress metrics and monitor them to
see if they’re consistent with plans.

12/21/20 21
Success Factors
5 Goal 3:
g improve the company’s ability to achieve competitive-edge
quality.
u define industry best practices to support and manage the
rapid growth of the company’s operations and to position the
company for future procurements. They also plan to identify
roles and responsibilities to allow for quicker, better decision
making.
5 Goal 4
g to achieve CMM Level 3 on a fast track through templates,
train-the-trainer materials, access to experts, and other
resources.
u conduct senior management and project reviews, assign
responsibility for the process improvement effort, and track
and manage an improvement plan.

12/21/20 22
Effects on the Organization
n The company’s leaders now realize that because the
company is too large for one manager to do it all, they must
delegate tasks. All three separate, distinct areas (system
architecture, software architecture, and development
processes) must work in unison to support corporate goals.
n Providing process structure and the ability to understand,
manage, and measure development risks has a cost, and
DSCS’s leaders are willing to spend the funds to achieve
those goals.
n DSCS has established repeatable processes that more than
one person knows. All employees clearly understand what
they need to do their jobs. DSCS is able to create software
more quickly and cost effectively.

12/21/20 23
Sources
n [1]https://people.eecs.ku.edu/~hossein/811/Papers/c
mm.pdf
n [2]Dangle, Kathleen Coleman, Patricia Larsen,
Michele Shaw, and Marvin V. Zelkowitz. "Software
process improvement in small organizations: a case
study." IEEE software 22, no. 6 (2005): 68-75.

12/21/20 24