2010 Seventh International Conference on Information Technology
A Methodology for Assessment Database According to ISO 9126
Michelle Dias de Andrade Alves Danilo Douradinho Fernandes
Brazilian Aeronautics Brazilian Aeronautics
Institute of Technology - ITA Institute of Technology - ITA Institute of Technology - ITA
Sergio Roberto M. Pelegrino Paulo Marcelo Tasinaffo
Brazilian Aeronautics Brazilian Aeronautics
Institute of Technology – ITA Institute of Technology - ITA
Abstract
A methodology for implement, design and test, and an
instrument for quality inspection complying with the ISO
9126 standard was developed by the Research Group on
Software Engineering, GPES, and TATA Consultancy
Services at the Aeronautics Institute of Technology, ITA.
GPES is a research group based on ITA. The
methodology and the instrument use the Causal Analysis
and Resolution, CAR, process area, based on the
approach Goal, Question, indicator, and Measures
(GQ(i)M). The main contributions were: a) the creation
and implementation of indicators, funded on metrics, for
a decision process, which are based on the construction
of a quality assessment methodology for a software
product, in a model of a information system; and b) a
prototype of an instrument for quality, which involves the
approach GQ(i)M in the definition of quantitative and
qualitative metrics, by means of a Microsoft Excel
spreadsheet. The methodology represents the
sophistication of the Causal Analysis and Resolution
technique for gain identification, using database
techniques. In this article the methodology was applied to
two different case studies.
Keywords: databases, FI, ISO 9126, CAR and CMMi.
1. Introduction
The CAR Causal Analysis and Resolution is a process
area of Capability Maturity Model integrated, (CMMi)
[1], developed for system design software. The
methodology is used to identify specific cases gain
adherence in relation to ISO 9126, with the use of
techniques of database by means of a quality instrument.
This instrument is an element to support CAR and a
process to analyze the databases vulnerabilities, checking
the gain / loss on realization of these activities. The model
978-0-7695-3984-3/10 $26.00 © 2010 IEEE
DOI 10.1109/ITNG.2010.80
Denis Ávila Montini
Brazilian Aeronautics
Luiz Alberto Vieira Dias
Brazilian Aeronautics
Institute of Technology - ITA
analysis provided the identification of the main
vulnerabilities for each technical approach Goal,
Question, indicator, and Measure GQ(i)M and CAR [2].
The model was implemented through a quality
instrument prototyped in a Microsoft Excel spreadsheet.
The prototype was based on the techniques of GQ(i)M
and ISO 9126, focused on verification tests and provided
an appraisement of the quality verification and was
applied on two different database. This instrument
allowed the categorization of quality attributes of software
features, functionality, reliability, usability, efficiency,
maintainability, portability, which were individually,
subdivided into sub-characteristics derived from ISO
9126. The use of the composition of the methodology for
product support, derived from CAR, was used to render
the instrument-based approach GQ(i)M. It was defined a
instrument, called Prototype Information System 9126,
PSI-9126, which was inserted to be used in a Final
Inspection [3] process for support of understanding about
the gain or loss of adhesion in relation to the quality
models chosen. The PSI-9126 instrument was used to test
each set of database techniques that have been
implemented in two projects of software applications.
In the first half of 2009 the authors of this work,
members of the Research Group on Software Engineering
(GPES), at the Aeronautics Institute of Technology, ITA,
and the company TATA Consultancy Services (TCS) used
the Technological Development Laboratories at the
Casimiro Montenegro Filho Foundation (LABTEC-
FCMF) to develop a methodology, a instrument for
inspection of quality of the ISO 9126 [4], and an use
process. The instrument was developed by deriving from
the ISO 9126 and indicators from (GQ(i)M) [5] [PARK et
al., 1996].
This article proposes a methodology to support the
Causal Analysis and Resolution (CAR) within a
Verification (VER) and Validation (VAL) process; a
1137
prototype of a quality instrument that involves the
approach GQ(i)M to detail a Final Inspection [3] [6]. For
the VER and VAL of databases were used: CAR, VER
and VAL, which are Process Area (PA) of the Capability
Maturity Model integrated [1].
In order to support the testing and homologation
phases of software, this research work focused on the
creation of some indicators GQ(i)M [5] [PARK et al.,
1996] to compose an analytical tool, to identify what are
the strengths and vulnerabilities of each techniques being
used in databases. Within this context, to optimize the
choice of techniques and through GAP Analysis [3] [TCS,
2006], it was carried out a series of activities to collect
data and information.
Initially, it was proposed the used of PA CAR and
GQ(i)M to generate an assessment instrument. It was
defined the main operational objectives for the established
of the guidelines of ISO 9126. Finally, from the main data
and information obtained from the two implemented
databases, and techniques were used for two different
databases were evaluating their strengths and weaknesses,
for each type of implementation.
For the development of this research it was used the
Windows environment and the paradigm of relational
databases used in the generation of the two databases
studied. The use of a System Manager Database such as
SQL Server 2003 did not represent a significant
investment of development.
The main contributions were: a) the integration of ISO
9126 with GQ(i)M, which constitutes a prototype
information system; and b) a process of product analysis.
The achievement of this process was only possible after
the design of a model of information system derived from
the guidelines of GQ(i)M. The prototype information
system was used to support the creation and use of
quantitative and qualitative metrics and the subsequent
implementation.
The prototype information system of ISO 9126 [3]
designed and developed, was implemented in a software
lab to performance a GAP Analysis. The PSI-9126 and its
process have represented the foundations for the
comparison between the two software applications [7],
demonstrating their strengths and vulnerabilities.
The research focused primarily on implementation of
GQ(i)M, the definition of quantitative and qualitative
metrics, and derivation from the ISO 9126 for the creation
of a prototype information system. The PSI-9126 was
used to identify the need to correct and resolve the main
difficulties founded within the use of databases.
2. CAR, Gap Analysis, ISO 9126 and GQ(i)M
The extent of these specific concepts, CAR, GAP, ISO
9126, and GQ(i)M, suited to refine the functional
1138
specifications and also to identify the gain in each of the
different implementation form.
A CAR was relevant because it provides the feasibility
of the Gap Analysis [3], through a set of specific activities
such as collecting data and information. This process used
to be the basis for better understanding of each solution in
detail. The quality instrument created, the PSI-9126, has
enabled the identification of vulnerabilities and their
relationships with their causes. The purpose of the use of
PSI-9126 facilitated the identification of defects and the
technical features affected the ISO 9126 standard, found
in the applications database studied. The identification,
analysis and correction of defects, from the heuristic
techniques of database analysis are supported by the field
of study focused on statistical methods for quality control
[8]. This area was used by means of indicators GQ(i)M
[5], an evolution of the Goal, Question, and Metric
(GQM) [2].
PSI-9126 is an instrument for collecting primary data
and its use is based on the application of a digital
questionnaire for obtaining the metrics necessary to
answer the questions set by the GQ(i)M. The use of PSI-
9126 facilitated the understanding of each group of
technical database defined and implemented. From this
point, a set of primary metrics according to the ISO 9126
were developed from the indicators GQ(i)M. Thus, the
activities under the PSI-9126 were: a) data accounting;
and b) the identification by means of graphics and
obtained result visualizations.
The use of PSI-9126 as times goes by provides a better
selection of groups of technical databases in accordance
with the established standard of quality for each new
project. The effectiveness of this choice restrained
spending and the researchers had the consciousness of the
risks taken in the choice for each set of chosen techniques.
3. Recommendation for Measurement and
Analysis (MA) for CMMi Level 2
The primary data collection began with the
identification procedures of GQ(i)M [6] [9]. From the
collection of primary data, it was possible to analyze in
the instrument the degree of compliance in relation to
guidelines of the ISO 9126 standard. Figure 01 shows the
comparison of the results between the two processes.
The goals and specific practices for Process Area
(PA), of the Measurement & Analysis (MA), listed in
Table 01 introduced the concept of integration of
activities around goals [9]. The planning, estimates, and
goals have been adjusted by the indicators information, in
order to locate any changes or deviations.
The GQ(i)M for the information system prototype,
PSI-9126, has implemented the various dimensions of
product performance, this GQ(i)M process application is At that time two databases, from the academic
in Table 02, which illustrates. program at ITA, were selected as case studies and later
analyzed in the LAB TEC FMCF for this article.

3.1. Measurement and Analysis supported in

GQ(i)M

The origin of the formally established concept of

quantitative and qualitative control was the report of the
Department of Defense (DoD) [10] that created a
manageable environment. For the effectiveness of the
management, it is necessary to collect and analyze metrics
that are significant to the measurement process. At this
point, it was necessary to use the concepts of PA MA
Figure 01- CMMi Measurement & Analysis PA supported by the use of GQ(i)M.
Following the suggestion of GQ(i)M contained in
Table 01- Specific Practices for CMMi specialized work, such as the Goal-Driven Software
Measurement & Analysis PA Measurement, it was found that the use of GQ(i)M
complements and relates to the PA of the CMMi MA [5].
Figure 02 shows the levels of GQ(i)M that can be
Specific Practices for CMMi Measurement & Analysis PA
Goals Specific Practices
applied to products, processes and resources of the life
Aligning Activities, Establish measurement objectives; cycle of the system. The vision of GQ(i)M has been
Measurement, and Specify measurements; applied to control the measurement and analysis in the
Analysis Specify the data collection and storage; and dimensions of products database. GQ(i)M reference is an
Specify analysis.
Generating Collect measurement data; approach top-down.
Measurement Analyze measurement data;
Results Store the data and results; and
Communicate results.
Establishing a Establish organizational policy;
Managed Business Plan processes;
Process Provide resources;
Determine responsibilities;
Coach the team;
Manage software configurations; and
Identify and involve the stakeholders that are
relevant;
Monitor and control the process;
Objectively evaluate adherence; and Figure 02 - The metamodel of GQ(i)M [5].
Report the project status to upper management.
Establishing a Collect information for improvement. However, in the CMMi there is not yet a direct
Defined Process mention of GQ(i)M. It was adopted in this proposal that
Establishing a Establishes the quantitative goals for the
Quantitatively process; and the use of GQ(i)M, goals and practices described above
Managed Process Stabilize the performance of sub-processes. do not compete or detract from the principles of CMMi.
Establishing a Ensure continuous improvement process; and Through the use of GQ(i)M it has been focused on
Optimized Process Correct the problem roots.
specific problems for database implementation. For the
Adapted from: identification of the goals an inference process took place
www.osqa.org/documents/siviy.pdf [13] in order to characterize, understand, and use questions and
answers. This process was based on the derivation of
Its use has served to provide the composition of metrics GQ(i)M for each indicator of ISO 9126.
database measures and their graphic representations. These concepts were developed for software factories
The process of analysis of the databases by PSI-9126 and the set of indicators created were incorporated into
enabled the identification of adhesion relationship in each the processes to improve control of the Software
dimension of ISO 9126. After filling the indicators Development Life Cycle (SDLC). These needs have been
GQ(i)M the prototype allowed the identification of the met during the selection process metrics and practices
best areas ranked, which allowed the comparison between proposed by Basili [2]. To analyze the vulnerabilities of
the two databases and how the study of the dimensions the two databases and verify what gains/losses were when
were affected by the two sets of techniques. realizing these activities, two case studies were used.

1139
 These studies made possible to relate the concepts of
GQ(i)M with the guidelines of the ISO 9126 standard,
identified by the CAR. In order to integrate the goals of
these concepts some questions, metrics and indicators
were developed to monitor the testing phase and
homologate production lines of software [11]. For this
phase, it was considered the process in the same
development environment and not selected metrics in
which data were difficult to collect [2].

Table 02 – GQ(i)M process Application.

Procedure for Application of GQ(i)M Phase Test
Activities Actions Goals
1 - Plan 1. Analyze the semantic similarity of 1. For each type of requirement present an array that lists the components
requirements for Business, Financial, that contain at least one of the requirements on one axis and all the
Functional, non-Functional and attributes on the other axes.
Detailed.
2. Create the arrays Attribute
Requirements.
2 - Measure 1 - See collection procedures defined for Measurement procedure must:
GQ(i)M. 1. Provide a basis for continuous process improvement.
2. Quantify the quality and productivity.
3. Integrate with the life cycle of the development.
4. Measure the impact with methods, tools and techniques
improvements available.
3 - Analyze the 1. Tabulate data in accordance with the 1. Generate a technical report showing the inferences made based on primary
data. technology that implements the data in order to make them information.
information system of GQ(i)M. 2. Identify in scalar form the deterministic behavior that a metric has
2. Check the limits found by the studied for a class of phenomena evidenced by a basic measure.
indicator.
4 - Make 1. Detail the critical points. 1. Identify in scalar form the deterministic behavior that a metric has
decisions 2. Define the actions to be performed studied for a class of phenomena evidenced by a basic measure.
based on the in accordance with the availability of 2. Map a traceability tree through a graphical display, showing the points of
analysis. planning. customization and traceability relationships affected.
5 - Implement 1. Find and make available to the team 1. Correct the distortions to return to the initial planning.
the decisions. results and make them aware of the
interruption of the process and
reasons for the new shares to the
alignment.
6 - Back to 1. See collection procedures defined for Measurement procedure must:
plan and the GQ(i)M, in order to verify the 1. Provide a basis for continuous process improvement.
measure. effectiveness of the actions taken. 2. Quantify the quality and productivity.
3. Integrate with the life cycle of development.
4. Measure the impact with methods, tools and techniques improvements
available.
7 - Conclude 1 - Review the documentation. 1. The developed product must be stored in a database data, a repository of
of the Project data and information requirements, attributes and project dependencies.
for reuse. 2. Review and validate the previous steps and documentation.

This approach began with the synchronization of the
goals of the organization, identified through questions and
ending with the identification of metrics. The indicators
are intended to elicit responses about the goals of the
organization. It is important to keep this objective when
measuring the results that will be used by means of
indicators and issues identified.
4. The instrument of quality of ISO 9126 for
the testing phase
The refinement of GQ(i)M MA PA is in Table 03,
which illustrates the twenty-one areas represented by their
respective metadata. Based on ISO 9126 standard it was
selected a possible indicator for each dimension. Table 05
provides a better understanding and visualization; it was
implemented as one of the indicators in the worksheet
PSI-9126.
The two case studies for implementation of GQ(i)M
were defined from a clear business purpose for the
homologation of the generated product. They were
extracted from the questions presented in Table 04. When
an identified Goal (G) was derived.
The ISO 9126 standard was studied for the creation of
a scientific test Question (Q) to represent how that
question is intended to be met in practice.
With the metric definition, it were created indicators, and
given a qualitative sense and classes for the data ranges.
Additionally it was added information about the
procedures for data collection.

1140
 The guidelines of the Measurement Plan established . The information system represented by the PSI-9126
for each technical objective, all other dimensions followed prototype with their guidelines outlined in Table03, was
the same form of design: design and implementation in conducted by the team that streamlines the testing phase
prototype. In this context, the two databases were tested and approval so that the indicators of GQ(i)M developed
on the lab, from the perspective of GQ(i)M. represent the needs in this step. Table 04 presents an
indicator implemented and comparison with the results of
Table 03 - Dimensions of ISO 9126 selected for the research.
the assembly of GQ(i)M
Table 04 - Application process of GQ(i)M.
Functionality ϭ͘ Suitability
Ϯ͘ Accuracy
ϯ͘ Interoperability
G 1. Suitability
ϰ͘ Compliance
Q 1. Implements the requirements requested?
ϱ͘ Access Security
Reliability (is immune to Count the number of requirements implemented
failure?) ϭ͘ Maturity in accordance with the Software Requirement
Ϯ͘ Fault tolerance M Specification (SRS) and homologation
ϯ͘ Recoverability 1 - Suitability --------------------------------------------------------
Usability (easy to use?) ϭ͘ Understandability Count amount of requirements requested
Ϯ͘ Learn ability Suitability
ϯ͘ Operability 450 / 450
= 100%
ϰ͘ Attractiveness D. ( 10 -
Efficiency (it's fast and ϭ͘ Time behavior B. (80 -
C. (51 - 50) E. ( 0 - 9)
"lean"?) Ϯ͘ Resource utilization 94)
A. (95 - 79) Implement Does not
Maintainability (easy to ϭ͘ Analyzability Impleme
100) Implement s almost implements
change?) Ϯ͘ Changeability I nts
Implement s half of all nothing of the
ϯ͘ Stability almost
s all the requiremen any requiremen
ϰ͘ Testability all of
requiremen ts are requiremen ts
Portability (it's easy to use in ϭ͘ Adaptability what is
ts appropriate ts that is appropriate
another environment?) Ϯ͘ Install ability appropria
requested. . appropriate ly.
ϯ͘ Co-existence te
.
ϰ͘ Replace ability
1 - Among the system's database;
It was adopted a reduction in scope to restrict the 2 - Select all tables in the database, verifying that the system
will display the names of tables;
specific features of software and hardware needed for the P 3 - View a table of contents;
testing of databases. The following guidelines include 4 - Try removing a record from the table;
analysis of known defects during the homologation phase. 5- Change a record in the table;
The reduction in scope was set to meet the rigorous 6 - Insert a record in the table;
academic research project, so it was used the minimum
necessary and sufficient for a behavioral and The prototype information system PSI-9126 was
phenomenological study. produced with the resources of PivotTables software
spreadsheet from Microsoft Excel [12]
4.1. Sentences of GQ(i)M generated from the
analysis of ISO 9126 standard 5. Conclusion

This phase of testing and approval of the databases
searched is a disciplined application of the model GQ(i)M
implemented by means of the quality instrument PSI-
9126. The fundamentals of quantified and qualified
control could only be obtained during the data collection
procedure.
The design and implementation of a practical case for
the use of GQ(i)M was obtained with the detailed product
requirements derived from ISO 9126 standard and a set of
complementary activities carried out in the laboratory for
the use of GQ(i)M. In the laboratory, it was considered
that the procedure of GQ(i)M supplemented Operational
Process for Iterative Development (Unified Process) of
the Integrated Quality Management System (IQMS) [3]
The contribution of this research was obtained from
the implementation of CAR, VER, VAL, Entry-Task-
Validation-Exit (ETVX) and GQ(i)M for the
identification of vulnerabilities in the testing phase and
homologation noting that it is necessary to stabilize a
defined process of work. Without the formal process
presented, the use of this methodology in the production
line may become unviable. It needs the support of the area
of quality in order to be implemented.
The experiment consisted of applying the PSI-9126 in
two case studies databases. Both used the modeling
techniques of database, such as the Model Entity
Relationship (MER) for the same information system with
the same features of a company.

1141
 The first used common modeling techniques
(CMT) as an end product that generated a MER. The
second addressed heuristic modeling techniques (HMT)
[7] that produced another MER. The database CMT has
75 tables with redundant information and the bank HMT
has 57 tables without redundant information. The
prototype information system PSI-9126 was designed to
have tracks of qualification for each of the studied
dimensions of ISO 9126. Each sub-feature of ISO 9126
included a set of features such as: Functionality,
Reliability, Usability, Efficiency, Maintainability and
Portability.
Based on the answers obtained by the use of GQ(i)M-
PSI-9126, results were obtained, expressed in percentage
terms. Each result percentage was grouped with the
following percentage A (95 -100), B (80-94), C (51-79),
D (10-50) and E (0-9). When using the PSI-9126 to
analyze the databases, it was observed that the instrument
applied to the database CMT found that there were
improvements in the visualization, removal, alteration,
insertion, import and export, update, installation and use
of machine. Most of the gain is contained in the feature
Functionality and few are present in the Portability and
Efficiency features classified in the percentage range A
(95 -100).
In the database HMT it was found that there was
improvement in the understanding of the entities, their
relationships, better processing in data recovery, ease of
maintenance. Most of the gain of the HMT was
concentrated in Maintainability, Functionality, Efficiency,
Portability, and Reliability (Maturity and Recoverability).
The features were classified in the percentage range A.
The use of prototype information system PSI-9126
allowed the comparison of the same project database,
which was implemented in two different ways, and found
the strengths and vulnerabilities of each of the two
approaches so that from the two approaches (CMT and
HMT) CMT obtained a performance of 43.8% of the
performance of the HMT. It (HMT) was classified in the
percentage range A.
7. Acknowledgements
The maintenance of an infrastructure for research in
software engineering is a crucial point. Institutions and
research groups like the ITA, FCMF and GPES have
contributed to the advancement of this specialized work.
The participation of TATA Consultancy Services is
also acknowledged.
8. References
[1] CMMi Product Team “Capability Maturity Model
Integration (CMMi) for Development”. Version 1.2.
1142
Software Engineering Institute, Carnegie Mellon
University, Pittsburgh, PA, 2006.
[2] V.R.Basili, G.Caldeira, and H.D.Rombach “The Goal
Question Metric Approach”. 5th ACM-IEEE International
Symposium on Empirical Software Engineering (ISESE
’06), Rio de Janeiro, 2006.
[3] TCS, Tata Consultancy Services, “Operational Process
for Iterative Development (Unified Process) Integrated
Quality Management System IQMS, Air India Building
Mumbai, India October of 2006.
[4] International Organization for Standardization /
International Electrotechnical Commission, ISO/IEC 9126
– Software Engineering – Product Quality, Geneve, 2001.
[5] R.E.Park, W.B.Goethert, and Florac; CMU/SEI-96-
HB-002 - Goal- Driven Software Measurement - A
Guidebook. August 1996.
[6] C.G. Bernardo, D.Á. Montini, D.D. Fernandes, D.A.
Silva, L.A.V. Dias, and A.M. Cunha, “Using GQM for
testing of design patterns in real time and embedded
systems on a software production line”.
[7] A.M.Cunha, “CE-240 Projeto de Sistemas de Banco
de Dados - Notas de Aula”, Instituto Tecnológico de
Aeronáutica – ITA. In Portuguese. Available in:
http://www.ita. br/~cunha, It is available in July 2009.
[8] Shewhart, W.A. (1939). “Statistical Method from the
Viewpoint of Quality Control”, Dover Publications
December of 1996.
[9] Software Acquisition Gold Practice. Goal Question
Metric. Available in: https://www.goldpractices.com/p
ractices/gqm/. Available 11May2007.
[10] Measuring Software Product Quality: The ISO 25000
Series and CMMi by the U.S Department of Defense,
Carnegie Mellon University, 2004.
[11] D.Á. Montini, “Model of risk indicators for the
budget of software components for cell manufacturing.
Master's Thesis in Production Engineering”, Universidade
Paulista, São Paulo, 2005.
[12] D.Á. Montini, “Universidade Excel”. Publisher
Digerati, Brasil, SP, São Paulo, 2003.
[13] Siviy, J. M. “The Effective Application of Six Sigma
in Software Engineering”. Carnegie Mellon Software
Engineering Institute, Pittsburgh, PA 15213-3890, 2004.