02/Mar/2020

Contents
CMU-CS 462:
1. Software measure classification
Software Measurement
2. Goal-based paradigms:
and Analysis Goal-Question-Metrics (GQM)
Spring 2019-2020 Goal-Question-Indicator-Metrics (GQIM)
Goal-based framework for software measurement (Cont’d) 3. Applications of GQM and GQIM

Nguyen Duc Man

E: mannd@duytan.edu.vn
T: 0904235945

02/Mar/2020 1 02/Mar/2020 2

1 2

1
 02/Mar/2020

Review
◼ The primary question in goal-based measurement:
◼ “What do we want to know or learn?”
instead of “What metrics should we use?”
Goal Based Software
◼ Because the answers depend on your goals, no fixed
Measurement: GQ(I)M Process set of metrics is universally appropriate.
◼ Instead of attempting to develop general-purpose
measures, one has to describe an adaptable process
that users can use to identify and define measures
that provide insights into their own development
problem.

02/Mar/2020 3 02/Mar/2020 4

3 4

2
 02/Mar/2020

GQM Approach: Review GQ(I)M: Concept

◼ Goal: List major goals of development or ◼ A methodology to convert a business goal
maintenance project. into a measurement plan.
◼ Question: Derive from each goal the
Business
questions that must be answered to determine Measurement
Goal GQIM Process Plan
if the goals are being met. Measurement Plan:
1…………………….
◼ Metrics: Decide what must be measured in Business Goal:
Reduce cost?
2…………………….

order to be able to answer the questions

adequately.

02/Mar/2020 5 02/Mar/2020 6

5 6

3
 02/Mar/2020

GQ(I)M: Precepts GQ(I)M Process

IdentifyGoals

◼ The GQ(I)M method is based on 3 precepts, ◼ 10 steps of 1) Identify Business

Goals BG
2) Identify what to know 1

and it consists of 10 steps the GQIM 3) Identify subgoals MG MG

1 1

◼ The three precepts are: process 4) Identify entities &

attributes
5) Formalize
Measurement Goals

◼ Measurement goals are derived from business ◼ A home G MG MG

1 1

goals brewed tool Define Indicators

6) Identify questions
and indicators Q Q Q Q

◼ Evolving mental models provide context and to automate 7) Identify measures

1 2 3

focus this process 8) Define measures I I

1
I
2
I
3
I
4

◼ GQ(I)M translates informal business goals into exists (ISMS) CreateAction Plan M M
1
M
2
M
3

executable measurement structures 9) Identify actions

10) Create plan Action Plan

02/Mar/2020 7 02/Mar/2020 8

7 8

4
 02/Mar/2020

GQ(I)M: Steps /1 GQ(I)M: Steps /2

1. Identify business goals 6. Identify quantifiable questions and indicators that
will be used to achieve measurement goals
2. Identify what you want to know or learn
7. Identify data elements that will be collected to
3. Identify subgoals construct the indicators to answer questions
4. Identify entities and attributes related to 8. Define measures to be used, and make these
subgoals definitions operational
5. Formalize measurement goals 9. Identify actions that will be taken to implement the
measures
10. Prepare a plan for implementing the measures
Business Measurement
goal goal Measurement Measurement
goal plan

02/Mar/2020 9 02/Mar/2020 10

9 10

5
 02/Mar/2020

GQ(I)M: Steps /3 Step 1: Identify BusinessGoals

The GQ(I)M method 1
◼ ◼ The business goal can
begins with identifying 1
business goals and 2 be initiated at any
breaking them down into organizational level
manageable subgoals. 3 4 ◼ Examples:
◼ It ends with a plan for
5 ◼ Reduce time to market 2
implementing well-defined
measures and indicators (TTM)
6
that support the goals. ◼ Improve customer
◼ It can also maintain satisfaction
7
traceability back to the 9 10 ◼ Improve quality of the
8
business goals. code

02/Mar/2020 11 02/Mar/2020 12

11 12

6
 02/Mar/2020

Templates for Goal Definition Templates for Goal Definition

(1)Purpose: To (characterize, evaluate, predict, motivate, etc.) (3)Environment: The environment consists of the following:
the (process, product, model, metric, etc.) in order to process factors, people factors, problem factors, methods,
(understand, assess, manage, engineer, learn, improve, etc.) tools, constraints, etc.
it.
◼ Example: Evaluating maintenance process in order to improve it. ◼ Example: The maintenance staff are poorly motivated programmers
who have limited access to tools.
(2)Perspective: Examine the (cost, effectiveness, correctness,
defects, changes, product measures, etc.) from the viewpoint
of the (developer, manager, customer, etc.)
◼ Example: Examine the cost of software development from the
viewpoint of the manager.

02/Mar/2020 13 02/Mar/2020 14

13 14

7
 02/Mar/2020

Step 1: Identify Goals Example

◼ The output of Step 1 is a Business Goal #1: Primary Business Goal:
sorted checklist of
business goals (i.e., Saving money by improving productivity of
management goals, Business Goal #2:
software development team
development goals, and
maintenance goals, etc.) Business Goal #3:
along with their
definitions.
◼ If more than one goal: Examine the effect of productivity of software
generate plan for each development team on project costs
goal separately. Business Goal #n:
◼ Typical expansion (Analyzed from project manager’s perspective)
rate: 1-4 times
02/Mar/2020 15 02/Mar/2020 16

15 16

8
 02/Mar/2020

Step 2: Identify What to Know Step 2: Identify What to Know

◼ Identifying what is needed to be known in order to
understand, assess, predict, or improve the activities
related to achieving goals by asking questions such as:
◼ “What activities do we manage or execute?”
◼ “What do we want to achieve or improve?”

◼ and by completing statements such as

◼ “To do this, we will need to…”

◼ Repeat these questions several times and break top-level

goals down into specific things should be accomplished and
issues that are needed to be addressed.

02/Mar/2020 17 02/Mar/2020 18

17 18

9
 02/Mar/2020

Step 2: Identify What to Know Entity-Question Checklist /1

◼ The output of this step is the entity-question checklist. Make 1. Start with one of the top-level goals identified in
sure that entities addressed should be in four categories:
◼ input and resources,
Step 1.
◼ products and by-products, 2. Identify the persons or groups whose concerns
◼ internal artefacts,
will be addressed. (i.e., manager, developer,
◼ and activities and flowpaths.
customer, etc.) This defines the perspective and
For each entity, list questions that can help us address
the roles that you and the team will assume in
those business goals. Tasks 3 through 6 here and in the remaining steps
of the process.
3. Create rough sketches of the relevant processes
that you, in your role, manage or affect. As you do
this, be guided by what you want to achieve and
the issues you will have to address to achieve it.

02/Mar/2020 19 02/Mar/2020 20

19 20

10
 02/Mar/2020

Entity-Question Checklist /2 Entity-Question Checklist /3

4. List the important things (entities) in your 5. For each entity, list questions that, if answered,
processes that you, in your role, manage or would help you, in your role, plan and manage
influence. Make sure that you address the four progress toward your goals.
kinds of process entities below: For example:
Inputs and resources How big is it?
Products and by-products How much is there?
Internal artifacts such as inventory and work in process How many components?
Activities and flowpaths How fast is it?
How long does it take?
◼ You may also want to list some of the environmental
How much does it cost?
entities outside your processes that affect your work.

02/Mar/2020 21 02/Mar/2020 22

21 22

11
 02/Mar/2020

Entity-Question Checklist /4 Example (cont’d)

6. Step back and look at your process as a whole to see if Entities of Interest Questions Related to Business Goal
anything is missed. By asking questions such as: Inputs People • How productive are people right now?
Is the process stable? How is it performing now? and • In what areas do they need
What limits our capability? What determines quality? Resources improvement?
• Extent of individual morale?
What determines success? What things can we control?
• Do we need more team members?
What do our customers want? What limits our performance? • Is our staff over-worked?
What might signal early warnings? Computer Aided Software • Are the tools sufficient or do they
Where is backlog occurring? How big is our backlog? Engineering Tools need to be upgraded?
What could go wrong? How will we know? • How do tools affect productivity?
Subcontractors • Would subcontracting aid our
productivity?
◼You may discover additional entities whose properties • Is it worth it?
may be worth measuring.
Customer Requests for • How much do changes in the project
7. Repeat Tasks 1–6 for your other goals. Change affect productivity?

02/Mar/2020 23 02/Mar/2020 24

23 24

12
 02/Mar/2020

Example (cont’d) Example (cont’d)

Entities of Interest Questions Related to Business Goal Entities of Interest Questions Related to Business Goal
Activities Development •Are we using the most productive development Products Documents Are the documents we produce readable?
and Flow methodology? and By- Is it possible to trace system features from one
Paths •Do we effectively structure the development team? products document to the next?
•Are we spending too much time in the development
phase? Source Code Is the source code consistent with the documents?
Is the source code error free?
Testing • Are we using manual testing or automated testing?
Does source code follow programming standards?
• Are the tests finding enough defects?
• Are we spending too much time on testing? Plans Do the plans change too much?
Budget Do we have enough money to increase salaries and
invest in tools?
Fixing • Is the response time for fixing bugs reasonable?
•Are high-priority changes getting implemented in a
timely fashion?
• Are we spending too much time debugging?

02/Mar/2020 25 02/Mar/2020 26

25 26

13
 02/Mar/2020

Step 3: Identify Subgoals/1

Step 3: Identify Subgoals/2
◼ Grouping related questions helps identify subgoals
◼ In Step 3, you identify the questions that you have
about the entities, then group them and identify the
issues they address.
◼ Then groupings of issues and questions translate
naturally into candidate subgoals.
◼ Sometimes, you may find several issues mapping
into a single subgoal, or single issues mapping into
several subgoals.

02/Mar/2020 27 02/Mar/2020 28

27 28

14
 02/Mar/2020

Example (cont’d) Example (cont’d)

Questions Related to Development Team Productivity
GROUPING #1 • How productive are our employees right now?
(People) • In what areas do they need improvement?
• Extent of overall office morale?
• Do we need more team players?
• Is our staff over-worked?
GROUPING #2 • Are the tools sufficient or do they need to be upgraded?
(Development) • How do tools affect productivity?
• Would subcontracting aid our productivity? Is it worth it?
• Are we using the most productive development
methodology?
• Are we spending too much time in the development phase?
• Is the source code consistent with the documents?
• Is the source code error free?
• Does source code follow programming standards?
• Do we have enough money to invest in tools?
Questions Related to Development Team Productivity
GROUPING #3 • How much do changes in the project affect productivity?
(Response to Change) • Do the plans change too much?
GROUPING #4 • Are we using manual testing or automated testing?
(Quality Assurance) • Are the tests finding enough defects?
• Are we spending too much time on testing?
• Is the response time for fixing bugs reasonable?
•Are high-priority changes getting implemented in a timely
fashion?
• Are we spending too much time debugging?
• Do we need better testing to be done?
• Are the documents we produce readable?
•Is it possible to trace system features from one document to
the next?

02/Mar/2020 29 02/Mar/2020 30

29 30

15
 02/Mar/2020

Example (cont’d) Step4: Identify Entities & Attributes /1

◼ Derived Subgoals (from Project Manager’s

Perspective)
◼ Subgoal #1 Improve performance of staff
◼ Subgoal #2 Improve code development
processes
◼ Subgoal #3 Minimize the negative effects of
project changes in productivity 4
◼ Subgoal #4 Improve quality assurance

02/Mar/2020 31 02/Mar/2020 32

31 32

16
 02/Mar/2020

Step4: Identify Entities & Attributes /2 Example (cont’d)

◼ Once having a list of questions, you should examine each ◼ Subgoal 1: Improve the performance of our staff
question and identify entities implicit in it. Then list ◼ Question 1:
pertinent attributes associated with each entity. ◼ Currently, how productive is our development team?
◼ Pertinent attributes are those which, if quantified, help ◼ Entity:
answer your question or establish a context for interpreting
the answers. ◼ Development team
◼ Pertinent attributes are usually cited in the question, either ◼ Attributes:
explicitly or implicitly. ◼ Team personality factors
◼ List of entities and the attributes for each entity are the ◼ Expertise of the development organization
principal outputs of this step. ◼ Team’s analysis and design techniques
◼ The attributes will become candidates for the things that ◼ Knowledge of programming languages
should be measured.

02/Mar/2020 33 02/Mar/2020 34

33 34

17
 02/Mar/2020

Example (cont’d) Example (cont’d)

◼ Subgoal 1: Improve the performance of our staff ◼ Subgoal 2: Improve Code Development Process
◼ Question 2: ◼ Question 1:
◼ How is the overall office’s morale? ◼ Are we spending too much time in the development phase?
◼ Entity:
◼ Entity:
◼ Working Environment
◼ The development process
◼ Attributes:
◼ Accommodations ◼ Attributes:
◼ Incentives ◼ Average duration of development process, per line of code
◼ Extra-curricular activities ◼ Percentage of development process’ duration for average project
◼ Hardware and software used ◼ Company expectations for percentage duration of code development
◼ Work time to break time ratio
◼ Workspace (room and desk size, ventilation)

02/Mar/2020 35 02/Mar/2020 36

35 36

18
 02/Mar/2020

Example (cont’d) Example (cont’d)

◼ Subgoal 3: Minimize the negative effects of ◼ Subgoal 4: Improve quality assurance
project changes in productivity
◼ Question 1: ◼ Question 1:
◼ How much do changes in the project affect productivity? ◼ Are we using manual testing or automated testing?
◼ Entity: ◼ Entity:
◼ Set of change requests received from the customer ◼ Testing method
◼ Attributes: ◼ Attributes:
◼ Number of change requests
◼ Estimated total effort to satisfy change requests ◼ Type (name of type)
◼ Total effort for project ◼ Is used? Yes/no (a binary classification)

02/Mar/2020 37 02/Mar/2020 38

37 38

19
 02/Mar/2020

Example (cont’d) Step 5: FormalizeMeasurement Goals /1

◼ Subgoal 4: Improve quality assurance

◼ Question 2:
◼ How effective are the tests? Are we spending too much
time on testing?
◼ Entity:
◼ Testing
◼ Attributes:
◼ Volume number of testsscheduled
◼ Progress number of tests executed
number of tests passed 5

02/Mar/2020 39 02/Mar/2020 40

39 40

20

What is a Measurement Goal?
◼ Business goals (or subgoals) are usually represented
by a phrase or sentence in natural language.
◼ A measurement goal (or subgoal) is a semi-formal
representation of a business goal (or subgoal),
composed of 4 components:
◼ An object of interest (entity)
◼ A purpose
◼ A perspective
◼ A description of environment & constraints
02/Mar/2020
41
02/Mar/2020
Active & PassiveMeasurement Goals
◼ Active measurement goals are directed toward
controlling processes or causing changes to
products, processes, resources, or environments.
These are the kinds of goals that are found in
project management and process improvement
activities.
◼ Passive measurement goals are meant to enable
learning or understanding. Passive goals are often
accomplished by characterizing objects of interest
according to some productivity or quality model.
41 02/Mar/2020 42
42
21

Active & PassiveMeasurement Goals: Examples
Active Goals
◼ Meet the scheduled
completion date
◼ Reduce variability
◼ Improve product reliability
◼ Improve productivity of
the process
◼ Improve time-to-market
◼ Reduce employee turnover
02/Mar/2020
43
02/Mar/2020
Measurement Goal: Object
Passive Goals ◼ Anything real or abstract, that we want to describe
◼ Understand the current or know more about is a potential object for
development process measurement.
◼ Identify root causes ◼ An object is an entity we want to describe with
◼ Assess product maintainability measured values.
◼ Identify capabilities and trends,
so that we can better predict Object of interest (entity):
future performance a process, product,
◼ Understand relationships among resource, task,
attributes, so that we can develop activity, agent,
models for predicting and artefact, metric,
estimating
environment, etc.
43 02/Mar/2020 44
44
22
 02/Mar/2020

Measurement Goal: Purpose

◼ Purpose of a measurement activity may be to understand, predict,
plan, control, compare, assess, or improve some productivity or
quality aspect of the object.
◼ Examples include: cost, size, reliability, test coverage,
responsiveness, peer review effectiveness, process compliance,
time to market, quality, customer satisfaction.

characterize, analyze,
evaluate, etc.

the <entity>, <aspect>,

<attribute(s)>, etc.
understand, baseline, predict,
in order to it. plan, control, assess, compare,
improve, etc.

02/Mar/2020 45 02/Mar/2020 far@ucalgary.ca 46

45 46

23
 02/Mar/2020

Measurement Goal: Perspective Measurement Goal: Environment

◼ The perspective identifies who is interested in the ▪ List or otherwise describe the environmental factors & related
measurement results, such as: developer, maintainer,
manager, or customer. parameters that should be understood to put the observed
◼ The perspective is stated to clarify the purpose of the results in context.
measurement activity. ▪ Focus on describing similarities to (and differences from) other
Perspective: familiar products, processes, and settings. This information
Examine the modifiability, quality, changes, becomes part of the database for future comparisons.
defects, defect types, backlog, ▪ Factors and parameters to consider include:
behaviour, stability, progress,
<specific attribute(s)>, etc.
- application - customer
from the point of view factors factors
of (the) . developer, manager, customer, - people factors
engineer, process improvement
- methods
team, senior management, etc.
- resource factors - tools
- process factors - constraints
02/Mar/2020 47 02/Mar/2020 48

47 48

24
 02/Mar/2020

Measurement Goal: Template

◼ Template for measurement (sub)goal:
Object of interest:

Purpose:
the in order to it.

Perspective:
Examine the
from the point of view of (the) .

Environment:
, , , ,
, , , ,

02/Mar/2020 49 02/Mar/2020 50

49 50

25
 02/Mar/2020

Example (cont’d) Example (cont’d)

Object of Interest: Development Team Object of Interest: Development Team
◼ Purpose:

◼ Determine a way to improve the productivity of our

◼ Environment & Constraints:
◼ Payroll applications programming in C++
development team by evaluating their current
◼ 100 software developers with 5 or more years
productivity.
◼ Perspective:
experience in C++
◼ Customers are businesses
◼ Examine the team personality factors, expertise of
◼ Do not maintain a reusable module database.
the development organization, team’s analysis
◼ Examine new projects completed and sold from
and design techniques, knowledge of
1/1/1995 to 31/12/2002.
programming languages of our development team
from project manager’s point of view.

02/Mar/2020 51 02/Mar/2020 52

51 52

26
 02/Mar/2020

Example (cont’d) Example (cont’d)

Object of Interest: Working Environment Object of Interest: Working Environment
◼ Purpose:

◼ Evaluate the working environment in order to identify

◼ Environment & Constraints:
◼ Payroll applications programming in C++
opportunities in improving the productivity of our
◼ 100 software developers with 5 or more years experience
development team.
◼ Perspective:
in C++
◼ Customers are businesses
◼ Examine the ratio of work time to break time of our
◼ Do not maintain a reusable module database.
employees, the accommodations, incentives, and extra-
◼ Examine new projects completed and sold from 1/1/1995
curricular activities offered to our employees, and the
workspace (room and desk size, ventilation) where our to 31/12/2002.
employees work
from the point of view of the employees themselves.

02/Mar/2020 53 02/Mar/2020 54

53 54

27
 02/Mar/2020

Example (cont’d) Example (cont’d)

Object of Interest: CASE Tools
◼ Purpose:
◼ Evaluate the impact of various CASE tools on the
productivity of the development team.
◼ Perspective:
◼ Examine the effectiveness of using various CASE
tools to help in the development of our product
from the point of view of the developers and testers.

02/Mar/2020 55 02/Mar/2020 56

55 56

28
 02/Mar/2020

Example (cont’d) Readings and Assignment 4

Object of Interest: CASE Tools
◼ Environment & Constraints:
◼ Payroll applications programming in C++
◼ 100 software developers with 5 or more years
◼ You are required to read the the links below and
Answer the following questions. The answers will
be submitted on e-Learning system (Assignment 4)
before 17:00, 20-Mar, 2020.

experience in C++ ◼ https://www.youtube.com/watch?v=7nhD__7-FE4

◼ Customers are businesses

◼ Do not maintain a reusable module database. ◼ Question 1

◼ Examine new projects completed and sold from ◼ What is GQM (according to the video)?
1/1/1995 to 31/12/2002.

02/Mar/2020 57 02/Mar/2020 58

57 58

29