83

The Concept of Quality Information System (QIS)

Ninoslav Slavek
Faculty of Electrical Engineering and Computing, University of Osijek, Croatia
Phone: (0385) 03 1 208 900, e-mail: ninoslav.slavek@etfos.hr

Abstract - The product quality characteristics should be the prime drivers when assessing
and improving the quality of software development process as we are concerned with the
product quality. The quality of software product is determined by the quality of software
process. This seems intuitive but there is no empirical evidence to prove its validity yet. QIS
establishes a system that enables to analyse the relation between base practices and processes
of the SPICE model and the eleven product quality factors and criteria of McCall's model for
software product evaluation. The main goal of QIS is to evaluate and verify benefits gained
by improving the process maturity level. In front line of both, the process model and product
quality model is the software product improvement, resulting in a high quality software
product delivered on time and at less cost.
Keywords: Process assessment, software quality improvement, international standard

1. INTRODUCTION

In software engineering quality, costs and time are interrelated and cause the main problems
with software as it imposes significant cost and time overruns and fails to meet the user
satisfaction.
A defined quality for a specified date can only be achieved if the adequate budget is available.
For most development contracts, the budget and costs are fixed and software quality suffers
from these restrictions [l]. The result of the development process should be a high-level
quality product delivered on time with low costs. The relation between process quality and the
quality of products is based on the assumption that the software product quality can be
achieved by improving the quality of the software process [4]. It seems intuitive but there is
no empirical evidence to prove this assumption yet. Measurable data showing evidence of
positive correlation between process maturity levels and project results is required. Product
and process are closely linked and cannot be separated when quality is analysed. The
certification of the process according to I S 0 9000 cannot give a guarantee of the quality of
the product. The result of the process, i.e., the product, is more important than process itself,
and the coupled certification is necessary for both the software process and software product.
Previous work. In references [ll], [14] it is held that a direct mapping between product
quality attributes and Base Practices of SPICE model [9] does exist. In the reference [ 121, the
Management Quality Information System for Software Quality Improvement was explained.
This work is continuation of the cited work. The Bootstrap project [5] has been used as the
process model. A set of statistical data and rules allowed analysing the relations between
general data and process maturity profile, and the analysis of maturity profile and process and
product specific data. The analysis showed the positive impact of process maturity profile on
software product specific values. The difference between previous reported work and work
reported in this paper are:
- Base Practice and Processes of the SPICE model is used as the process model,
- The tool Medusa [ 6 ] has been developed that is used (i) to support the evaluation of the
existence or adequacy of practices within the scope of SPICE assessment, and (ii) to
support the quality evaluation processes of QIS.
The Medusa tool was published as a departmental report at the Faculty of Electrical
Engineering and Computing in Osijek. The approach to the model solving ability of the
Medusa is to combine quantitative and qualitative description of the target hnction. This tool

23'd Int. Conf. Information Technology Interfaces /TI 2007, June 19-22, 2001, Pula, Croatia
 84

enable to set up front the target hnction that is constructed according to the target rate values
in QIS. Target hnction is compared with hnction of combined and achieved measured rate
values.

2. SOFTWARE PRODUCT QUALITY MODEL

A common approach to formulating a model for product quality is to identify a set of high-
level quality attributes and then in top-down fashion decompose it into sets of subordinate
attributes. Typical of this approach are McCall's model [7] and ISO-9126 Standard [13].

2.1 International Standard ISO-9126

International Standard ISO-9126 has been put forward as a high-level framework for
characterising product quality. While this standard can provide high-level guidance, it does
not go far enough to support building quality into software. According to Dromey's criticism
[2] this standard does not emphasise the reusability of software that is an important quality
attribute and deserves a similar status as the other attributes in ISO-9126.

2.2 McCall's model

FACTOR CRITERIA FACTOR CRITERIA

Traceability Simplicity
Correctness Consistency Testability Modularity
Completeness Instrumentation
Self-descriptiveness
Error tolerance Modularity
Reliability Consistency Portability Self-descriptiveness
Accuracy HW independence
Simplicity Independence
Efficiency Storage efficiency Generality
Execution efficiency Reusability Modularity
SW independence
Operability Modularity
Usability Training Interoperability Communication commonality
Communicativeness Data commonality
Consistency Access control
Maintainability Simplicity Integrity Access audit
Conciseness
Modularity
Self-descriptiveness
Modularity
Flexibility Generality
Expandability

23'd Int. Conf. information Technology lnterfaces /TI 2007, June 19-22, 2001, Pula, Croatia
 85

These factors are hrther decomposed into lower level attributes called quality criteria that are
related to the various factors by definition (Table 1.). In fhrther level of decomposition quality
criteria are associated with a set of directly measurable attributes called quality metrics.

2.3 Software quality attributes measurement

In software three classes of entities are measured: (i) processes, (ii) products, and (iii)
resources. Attributes are either internal or external: (a) internal attributes are those that can be
measured purely in terms of the process, product, or resource itself, (b) external attributes are
those which can only be measured with respect to how the process, product, or resource
relates to its environment. The external attributes are synonymous with particular attributes of
software quality. Thus software quality is made up of a range of different external attributes.
External attributes are the most difficult to measure due to their nature. There are no agreed
definitions of these attributes. External attributes cannot be measured directly and we are
forced to use internal attributes that can be measured directly to support the external attribute
measurement.

3. SPICE - SOFTWARE PROCESS MODEL

ISO/IEC 15504 (SPICE) is an emerging standard concerned with software process

assessment, process improvement and supplier capability determination. The Baseline f

Practices Guide (BPG) is the main document that defines the activities fhndamental for good
software engineering, and as a basis to assess the capability of an organisation software
process.

3.1 Software process dimension

The SPICE model has two dimensions, the process dimension and the capability dimension.
The process dimension defines 35 processes grouped in 5 process categories: (1) Customer -
supplier, (2) Engineering, (3) Support, (4) Project, and (5) Organisation. A process is a set of
activities, the base practices that contribute to achieve the purpose of the process. Each
process has a between 3 and 13 base practices. The capability dimension of the model differs
from that of CMM [8]. Five levels composing a scale of requirements from the lowest level 1,
to the level 5 provide it. Each level is composed of common feature formed with generic
practices.

3.2 Process assessment

By combining the two dimensions of the model, SPICE allows to analyse software process
with the requirements of the levels in order to determine the process capability. Process
assessment will assess a subset of 35 processes in each of the five process categories, for each
of the attributes associated with a continuous subset of the capability levels. Assessment
involves the comparison of the actual situation of the processes to a process model.
Evaluating compliance to the requirements of the model gives the assessment result.

4. DESIGN OF QIS

QIS consists of three primary entities: ( 1 ) a set of quality factors according to McCall's
categorisation, (2) a base practices and processes of the SPICE model, and (3) a set of

.. -

23'' Int. Conf. lnformation Technology interfaces /TI 2007, June 19-22, 2001, Pula, Croatia
 86

software metrics, namely (i) product metrics, (ii) process metrics, and (iii) resource metrics
(Fig. 1).
McCall' s model concentrated on the design of a detailed product high-level quality attribute
hierarchy resulting in a product quality model.
SPICE model defines, at a high-level, the process quality model.
QIS establishes a system that enables to analyse the relation between base practices and
processes of the SPICE model and the eleven product quality factors and criteria of McCall' s
model for software product evaluation. The main goal of QIS is to evaluate and verify
benefits gained by improving the process maturity level. In front line of both, the process
model and product quality model is the software product improvement, resulting in a high
quality software product delivered on time and at less cost.

ORGANISATION
1t
Process maturity Process metrics
J"Maturity profile Product metrics
2 Resource metrics
~~ ~ ~~~

PROCESS QUALITY MODEL

Process quality attributes PRODUCT QUALITY MODEL

Organisation process category
Customer-supplierprocess category Product quality attributes
Engineering process category
Project process category
3
' Quality factors
~ualitycriteria
Support process category Quality metrics

Fig. 1 Qualityfactors and characteristics

5. GENERAL CONCEPT OF QIS

SYSTEM EXPERIMENTAL, SAFETY REAL-TIME LONG-TIME

CHARACTERISTICS SYSTEMS CRITICAL SYSTEMS USAGE
SYSTEMS SYSTEMS
Flexibility Reliability Efficiency Correctness
QUALITY FACTORS Reusability Correctness Reliability Maintainability
Reliability Testability Correctness Reliability

Only meaningfil quality factors will be measured for a project, i.e., with the avoidance of
unnecessary measurements for less meaninghl quality factors. Time and cost for testing are
reduced in this way.

23rdInt. Conf. lnformation Technology lnteffaces IT1 2007, June 19-22, 2001, Pula, Croatia
 87

5.1 Quality evaluation process

The quality evaluation processes take the following steps (Fig. 2): (1) System characteristic
definition- the basic characteristics for evaluating target system are defined. Next, related
quality factors and criteria for a project are selected and defined. (2) Metrics selection- in order
to measure each quality factor metrics are selected and defined. (3) Target rate definition- the
decision-maker together with the user sets target rates for defined quality factors. (4)
Measurement and assessment- the quality factors are measured. The quality requirement level
for each quality factor is clarified in order to meet users or application needs. Each quality
factors achieved value is compared with the target rate values set up-front that must be
achieved. The developed Medusa tool enable to set up front the target function that is
constructed according to the target rate values. Target function is compared with function of
combined and achieved measured rate values showing the differences and pointing to the
quality factors that comprise these differences. The process maturity level of attributes that
influence product quality factors should be improved in order to achieve needed value. The
evaluation is repeating until satisfying all values resulting in a high-quality software product
delivered on time with the low cost.

Quality
requirements
definition Metrics selection Target rate

Software
development Product Result

Measurement b Assessment
I J I I

Figure 2. Quality evaluation process

5.2 The software reliability in safety-critical systems

Every system has a true reliability that is generally unknown. The use of software in safety-
critical and real-time embedded systems requires the achievement of high software reliability.
The reliability is quality factor that appears in almost all quality models. The only type of
product for which this attribute is relevant is executable code (Fenton, 1995). Reliability is
defined as the probability of no occurrence of a software failure during certain period on a
specified condition. SoRware failure is defined as an acceptable departure from normal
program operation caused by a software error latent in the software (Shigeru, 1991). The
reliability function,

Ri(t) = P(Ti>t) = 1- Fi(t), Ill

is the probability that the program will survive for a time “t” before failing next. “P” will
denote probability, “Fi(t)” is the distribution function of random variable “T”. The techniques
for the quantitative measurement and assessment of reliability are based on the reliability
growth model characterised by a non-homogeneous Poisson process. During testing software is

23‘‘ Int. Conf. lnfurmafiunTechnulugy Interfaces /TI 2007, June 19-22, 2001, Pula, Croatia
 88

subject to failures, test data can than be observed. Assuming that the correction of errors does
not introduce new errors, the cumulative number of detected errors increases as they are
corrected and the mean time interval between errors becomes longer. This means that the
reliability increases with the progress of testing. Generally, software reliability assessment
during testing is closely related to the quantity and quality of executed tests. Software process
attribute testing (unit testing, integration testing, acceptance testing) influences reliability.

6. Conclusion

In QIS the relationship between the software process quality attributes and software product
quality attributes is examined and it is argued that both the software process quality and
software product quality should be integrated in order to achieve the improvement of sofiware
quality. In particular, the process quality profile of the Base Practices of the SPICE model are
compared with the eleven product quality factors and criteria of McCall’ s model for software
quality evaluation showing evidence of positive correlation between process quality levels and
project results. The result of QIS is a high-quality software product delivered on time with the
low cost.

References:
1. Bullinger H. J., Fahnrich K. P.: “Managing Lean Enterprises and Lean Software
Development”, Conference on Lean Software Development, Germany, 1992.
2. Dromey R. G.: “A Model for Software Product Quality”, IEEE Transactions on
Software Engineering, Vol. 21, No, 2, 1995.
3. Fenton N. E.: “ SoRware Metrics, A Rigorous Approach”, International Thompson
Computer Press, 1995.
4. Haase V. H. : “Software process assessment concepts” ISSN 1383-762 1/0165-6074,
Journal of Systems Architecture 42, No. 8, 1996.
5. Haase V. H., Messnarz R., Koch G., Kugler H-J., Decrinis P.: “Bootstrap: Fine Tuning
Process Assessment”, IEEE Software, 1994.
6. JoviC F. : “Qualitative reasoning and a circular information processing algebra”,
Informatica, 2 1, 1997.
7. McCall J., Richards P., Walters G.: “Factors in Software Quality”, NTIS AD-A049-
14, 1977.
8. Paulk M.C., B. Curtis, M. B. Chrisis, C. V. Weber: “Capability maturity level for
software”, Ver. 1.1, CMU/SEI-93-TR-24, Research assess, 1993,
9. Simon J-M.: “SPICE: Overview for software process improvement” ISSN 1383-
762110165-6074, Journal of Systems Architecture 42, No. 8, 1996.
10. Shigeru Y. : “Software QualityReliability Measurement and Assessment: Software
Reliability Growth Models and Data Analysis”, Information Processing, Vol. 14, 1991.
11. Slavek N., JoviC F.: “Process Versus Product Quality System (PWQS), in publishing
proceeding, Zagreb, 200 1.
12. Slavek N.: “Concept of a Management Quality Information System (MQIS) for
Software Quality Improvement” ISBN 953-6042-40- 1, M P R O ‘97, Opatija, Croatia,
1997.
13. Standard I S 0 9126, Information Technology - Software Product Evaluation - Quality
Characteristics and Guidelines for Their Use, 1991.
14. Woodman I. H. G.: “Relationship between the activities of the software process and
the quality of the software product”, Research report, University of Strathclyde, 1994.

23‘dInt. Conf. information Technology lnterfaces IT/ 2007, June 19-22, 2001, Pula, Croatia