Agile (C): Managing

and Reporting
Project Agility

Linda had been the head of her company’s project management office (PMO) for
the past five years. All projects were managed using traditional project manage-
ment practices and a single project management methodology established by the
PMO and updated continuously. But now, with her company’s adoption of agile
project management techniques on several of the projects, many of the activities
in her PMO would change.
During the past decade, there had been a rapid growth in agile project man-
agement practices, not just in IT, but in other types of projects as well. Most of the
principles of agile project management practices had provided beneficial results
when applied to non-IT projects. While all of this sounds good, there were also
some headaches that accompany the growth. Linda understood the changes that
needed to be made regarding use of a different approach, but her biggest headache
would be with metrics. Using just time, cost, and scope metrics would no longer
be sufficient. What metrics would be needed to track and report agility?
There is an old adage in project management: “You cannot manage what you
cannot measure.” Therefore, to manage projects using agile techniques, Linda
understood that she must establish metrics to confirm that the benefits are being
realized and agile practices are being executed correctly. Fortunately, accompany-
ing the growth in agile practices had been growth in metric measurement tech-
niques. Today Linda believed that just about anything could be measured and
there were good metrics for reporting performance.

777
778 Project Management Case Studies

METRIC MANIA
Linda was convinced that her biggest challenge would be to combat metric mania,
the insatiable desire to create metrics for metrics’ sake rather than for what is
really needed. Linda knew that there were pros and cons to having both too many
metrics and too few metrics. There is always confusion in what metrics to choose,
as Linda found out.
The result of having too many metrics is that:

●● We steal time from important work to measure and report these metrics.
●● We provide too much data, and the stakeholders and decision makers find
it difficult to determine what information is in fact important.
●● We provide information that has little or no value.
●● We end up wasting precious time doing the unimportant.
●● Too many metrics can open the door for unnecessary questions from
stakeholders and business owners and eventually lead to a micromanage-
ment environment.

The result of having too few metrics is that:

●● We may not be providing the right information for stakeholders to make

informed decisions based on evidence and facts.
●● Stakeholders may be confused as to what is really happening on the pro-
ject until the project is over.
●● Stakeholders and business owners may be misled as to the true status of
the project. This could lower customer satisfaction and risk the loss of
follow-on work with this client.

In traditional project management using waterfall charts, Linda’s reporting

had always been done around the metrics of time, cost, and scope. With the use
of the earned value measurement system, the number of metrics soon increased
to 12 to 15. Linda feared that when a new approach appears, such as agile and/or
Scrum, metric mania would set in, and people would believe that they must create
significantly more metrics than were actually needed.
Linda found numerous publications that discussed the 10 or 20 important
metrics that needed to be reported in agile project management. Some publica-
tions stated that there may be as many as 50 different metrics, and there appeared
to be some disagreements as to what metrics were really important. Initially, with
the acceptance of new approaches, Linda felt that it might be necessary to allow
metrics mania to exist until she could filter out those metrics that did not provide
informational value. As companies become mature in using any new approach,
the number of metrics reported is usually reduced.
Agile (C): Managing and Reporting Project Agility 779

Having a good metric selection process would ease the impact of metric
mania and allay many of Linda’s fears. Linda understood that the use of a PMO
and the use of organizational metric owners who report solid to functional manag-
ers and dotted to the PMO would certainly help.

METRIC MANAGEMENT
Having a good metric management program can minimize the damage of metric
mania but cannot always eliminate it. Linda established a four-step metric man-
agement program that would be supported by her PMO and used on all projects
where agile concepts were required:

1. Metric identification
2. Metric selection
3. Metric measurement
4. Metric reporting

Linda then sent out a memo to her PMO team explaining her thoughts on
metric management:

Metric identification is the recognition of those metrics needed for fact-based or

evidence-based decision making. Today’s project management methodologies,
such as agile and scrum, function more as frameworks or flexible methodologies
than rigid methodologies. These flexible methodologies can be customized to a
particular project’s or business owner’s needs. Our external clients would prefer
to work with a contractor who has a flexible methodology that can be customized
to the client’s business model than to work with a rigid methodology that sup-
ports only our business model. Flexible methodologies that are adaptable to any
situation are the future and we must be prepared.
When dealing with flexible methodologies, the notion of standardized metrics
disappears. Each project can have a set of metrics that are unique to that par-
ticular project. Metrics can also be unique to individual parts of a project. Since
each stakeholder and business owner may have different needs and make differ-
ent decisions in support of the project, it is advisable at the onset of a project to
ask each stakeholder and business owner what metrics they would like to have
reported. It is possible that each stakeholder on the same project will request
a different set of metrics based on their own needs and the decisions they are
expected to make.
Metric selection is when you decide how many of the identified metrics are
actually needed. Some companies maintain metric libraries and show the metric
library to the stakeholders and business owners. It is not uncommon for the first
response to be “I would like to have all of the metrics in the library reported on
my project.” This is when the devastating effects of metric mania set in. We must
create a metric library, but do it the correct way.
780 Project Management Case Studies

Metric selection is the first step in resolving metric mania issues. Ground
rules for metric selection might include the following:

●● There is a cost involved to track, measure, and report metrics even if we use
a dashboard reporting system rather than written reports.
●● If the intent of a good project management approach such as agile or Scrum
is to reduce or eliminate waste, then the number of metrics selected should
be minimized.
●● We should encourage viewers of metrics to select the metrics they need, not
the metrics they want. There is a difference!
●● Asking for metrics that seem nice to have but provide no informational value,
especially for decision making, is an invitation to create waste. This is some-
thing we must avoid.

There is no point in selecting metrics that are difficult and costly to meas-
ure. We can be supported by the use of metric owners. These are people who
track the use of one or more metrics on all applicable projects and seek out best
practices for improvements in the way the metric is measured and reported.
This information will be periodically provided to the PMO, and updates will
then made to the metric library. Metric measurements can become costly when
the measurement data needed does not come from our computerized informa-
tion system.
Paperwork is the greatest detriment to most project managers. The future of
project management practices is to create a paperless project management envi-
ronment. This does not mean that we will be 100% paperless since some reporting
is mandatory, but it does mean that we recognize that unnecessary paperwork is
waste that needs to be eliminated. To do so, we will rely heavily on dashboard
project performance reporting.
Each stakeholder or business owner can have dashboards that are custom-
ized with the metrics they selected. This customization may seem like an added
expense, but we must also consider that, once the dashboards are designed,
updating the information is relatively fast if we use Excel spreadsheets as the
drivers for the data for the images on the dashboards. Although customized dash-
board reporting may appear as an added expense initially, we must also consider
the long-term impact it can have on business owner, stakeholder, and customer
satisfaction.
We have been discussing that some flexible methodologies may have as many
as 50 metrics. Dashboard reporting systems can force viewers to be selective in
the metrics they wish to see on the dashboard. A typical dashboard screen has a
limited amount of real estate, namely the space for usually only six to 10 metrics
that are aesthetically pleasing to the eyes and can be easily read. Therefore, tell-
ing stakeholders or business owners that we wish to provide them with one and
only one dashboard screen may force them to be selective in determining the
metrics they actually need.
Agile (C): Managing and Reporting Project Agility 781

THE LOVE/HATE RELATIONSHIP

Linda wanted to make sure that her company did not end up the way other companies
did with a love/hate relationship with metrics, especially metrics related to agility.
She wanted metrics that could be used to shine a light on the accomplishments of the
team by tracking performance, reporting the creation of business value, and identify-
ing ways to reduce waste. Linda also wanted metrics that could be used to identify
pain points, or are situations that bring displeasure to business owners, stakeholders,
and clients. The team then would look for ways to reduce or eliminate the pain points.
The hate relationship occurs when metrics become a weapon used to enforce
a certain behavior. While good metrics can drive the team to perform well, the
same metrics can create a hate relationship if management uses them to pit one
team against another. Another hate relationship occurs when the metrics are used
as part of an employee’s performance review. From her research, Linda discov-
ered that this type of hate relationship arises when:

●● Metrics are seen as the beginning of a pay-for-performance environment.

●● Metrics are the results of more than one person’s contribution, and it may
be impossible to isolate individual contributions.
●● Unfavorable metrics may be the result of circumstances beyond the
employee’s control.
●● Employees may fudge or manipulate the numbers in the metrics to look
good during performance reviews.
●● The person doing the performance review does not understand that the true
value of the metric may not be known and the stakeholders may not get a
true representation of the project’s status until some time in the future.
●● Employees working on the same project may end up competing with one
another rather than collaborating, and the project’s results could end up
being suboptimized.

LINDA’S CONCLUSIONS AND RECOMMENDATIONS

Linda believed that she just scratched the surface in the identification of metrics. Met-
rics would be a necessity with any and all project management approaches, including
agile and Scrum. However, given the number of possible metrics that could be identi-
fied, she had to establish some guidelines to avoid metric mania conditions and love/
hate relationships. Linda prepared the following list as a starting point:

●● Select metrics that are needed rather what people think they might want
without any justification
●● Select metrics that may be useful to a multitude of stakeholders, clients,
and business owners.
782 Project Management Case Studies

●● Make sure the metrics provide evidence and facts that can be used for
decision making.
●● Make sure the metrics are used rather than just nice to have displayed.
●● Do not select metrics where data collection will be time-consuming and
costly.
●● Do not select metrics that create waste.
●● Do not use metrics where the sole purpose is for performance reviews
and comparing one team against another.
●● Make sure that the metrics selected will not demoralize the project teams.

QUESTIONS
1. Should metric identification and selection be a structured process?
2. Is it better to allow and even encourage metric mania to exist and then reduce the
number of metrics or to let the number of metrics grow incrementally?
3. If you started small, what would be a reasonable number of metrics?
4. What metrics would be appropriate for agile project management, at least as a
starting point?