The Goal Question

Metric Approach
Victor R. Basili,
Gianluigi Caldiera
and H. Dieter Rombach

Presenter:
Jose Calvo-Villagran
jcalvovi@uwaterloo.ca

Overview
Motivations
GQM
Definition
Process

Examples
Extensions
Conclusions

Motivations [1/2]
Measurement

Planning
Quality Control
Refinements
Progress

The case of Amisoft [2]

CPI/SPI
Volatility
Client incidents

Motivations [2/2]
Effective measurement must be [1]:
Focused on specific goals
Applied to all cycles of production
Embedded into an specific context

Top down (?)

Quantitative model

GQM: Definition [1/2]

Measurement system
Set of rules for interpretation of measurement data
Three levels
Conceptual Level [Goal]
Objects

Operation Level [Question]

Assessment

Quantitative Level [Metric]

Objective/Subjective

(improve), a process (change request processing), a viewpoint (project manager), and a

quality issue (timeliness). This goal can be refined to a series of questions, about, for
instance, turn-around time and resources used. These questions can be answered by
metrics comparing specific turn-around times with the average ones. The complete
Goal/Question/Metric Model is shown in Figure 2.

GQM: Definition [2/2]

Figure 2

Goal

Purpose
Issue
Object (process)
Viewpoint

Question

Improve
the timeliness of
change request processing
from the project manager's viewpoint

Metrics

What is the current change request processing

speed?
Average cycle time
Standard deviation
% cases outside of the upper limit

Question

Is the performance of the process improving?

Metrics

Current average cycle time

* 100
Baseline average cycle time
Subjective rating of manager's satisfaction

3.

THE GOAL QUESTION METRIC PROCESS

GQM: Process
Identify goals
Policies/Strategies
Description of process/product
Model of the organization

Derive questions
How to characterize the object?
How to characterize the attributes?
How to evaluate the characteristics?

Specify metrics
Amount and quality of data
Maturity of objects
Learning process

GQM: Remarks
Simple, intuitive approach for specifying metrics
Metrics can be reused by several questions
Four phases: [3]

Planning
Definition
Collection
Interpretation

Goals include different viewpoints (stakeholders)

Subjective metrics

Examples [1/2]

[4]

Examples [2/2]

The purpose is to characterize the effect of pair programming

on programmer effort and program quality from the point of
view of software managers in the context of a small webdevelopment company [5].

*Side notes
Applicable on mature organizations:
Employee involvement (resistance)
Clear goals
Gradual/Slow changes

Does GQM evolve over time? How fast?

Does GQM apply to more agile environments?

Why GQMs main focus is SE?

Simplicity: Double edged sword
Relation between goals
How do GQMs goals align with organizations goal?

Extensions
GQM+Strategies [6]

Conclusions

Top-down approach for definition and collection of metrics

3 levels & 4 phases
Applied to all phases of software development
Requires organizational maturity

Bibliography
[1] Victor Basili, Gianluigi Caldiera, Dieter Rombach. 1994.
The Goal Question Metric Approach.
[2] Romain Robbes, Rene Vidal, Maria Cecilia Bastarrica.
2013. Are software analytics efforts worthwhile for small
companies? The case of Amisoft. IEEE
[3] Irene Eusgeld, Feliz Freiling, Ralf Reussner. 2008.
Dependability Metrics: Advanced Lectures.
[4] D.Ernst, F. Houdek. GQM Method Application. Taken on
27-Jan-2013. http://www-ivs.cs.uni-magdeburg.de/sweng/us/java/GQM/link3.shtml
[5] http://en.wikipedia.org/wiki/GQM
[6] Victor Basili, et. al. 2010. Linking software development
and business strategy through measurement. IEEE