Measuring the value of KM

David Gurteen
Gurteen Knowledge
www.gurteen.com
 Objectives

• Take a look at the many things you need to

think about when ‘measuring’ anything but in
particular KM

• Look at the role of rewards in motivating

people
– Create measures, set targets, reward people when
they meet them!
 Two things can be measured

1. Measure the knowledge of an organization

– Intellectual capital
2. Measure outcomes of ‘knowledge in action’
– Say a KM initiative

• My interest is in outcomes – demonstrating the value

of a KM initiative
• The value of knowledge is contextual!
– You can have all the knowledge in the world and still do
nothing with it
Some considerations when
measuring things!
 Beware the word “Measure”

• Beware the word “measure”!

• Other words: performance indicator, metric,
ROI
• Performance indicator is usually best
• Measures and metrics imply an exactness
that is usually not achievable
• Its ok 'to measure' but you don't create 'a
measure‘ but an ‘indicator’
 Why measure?

• Could be for one of several purposes

– To conform to laws or regulations
– To deliver on a promise or agreement
– To meet a service level agreement
– To justify your existence
– To enforce performance
– You have been told to
– To prove to yourselves you are not wasting time
– To provide feedback that facilitates learning
 Keep asking the question why!

• What are your real objectives and are

measurements the best way of achieving
them?

• Don’t measure for its own sake!

 Beware not measuring

• Your manager may not have asked for

measures
• But when the going gets touch you had better
be able to justify your existence
• Change of Management!!
• Your manager’s manager may have different
ideas
• When cuts have to be made – KM is high on
the list! (KM means Kill Me!)
 Plan to measure before you start

• You should plan to measure before you start

a project or initiative- not after

• What you measure and how you measure will

affect how you do things!!

• Too often Knowledge Managers don’t plan to

measure up front!
 There are multiple stakeholders

• Multiple stakeholders will 'measure' you or want to

see different measurements from you depending on
their perspective
• Don’t assume:
– you know what the right measures are. Ask your
stakeholders!
– they will tell the truth or measure you by the measures you
have agreed to provide them
• You need to understand what is important to them
• At the end of the day you will probably be measured
on their 'gut feel‘ (How might you influence this!)
 Two Stakeholders

• Senior management
– Sell & measure on business outcomes

• Staff
– Sell & measure on what's in it for them
 Two types of measure

• There are two types of measure

– you can measure activity
– or you can measure business outcomes
• Business outcomes are best by FAR
• Setting up say 10 communities of practice is
an outcome but is NOT a business outcome
• Business outcomes: increased revenue;
decreased cost; improved bid to win ratio
 Focus on Outcomes not Activities

• Too often we measure activity rather than outcome

• And we try to measure & justify KM initiatives on

activity rather then outcome

• Sometimes activity is the only proxy we have but too

often we focus on activities at the expense of
outcomes
 Examples of Measure of Activity

• Number of documents captured in a database

• Number of times a document is read
• Number of meetings held
• Number of active communities of practice
• Time taken to complete a task
 Examples of Measure of Outcome

• Bid to win ratio

• Increased sales
• Decreased costs
• Improved quality
• Reduced development time
• Reduced staff turn over
• Percentage of customers happy with service
according to customer satisfaction survey
 Beware of Targets

• People are often given ‘targets’ by which they

are ‘measured’
• Often a command & control way of trying to
force people to change
• All imposed measures or targets will be
gamed
• They rarely work well
– e.g. NHS
• Targets need to be agreed and bought into
Examples of Poor Measures/Targets

• First piece of baggage should arrive in arrival hall

within 10 minutes of plane touching down

• Patients when booking an appointment should not

have to wait more than 3 days to see their doctor

• In-patients in A&E should be seen by someone within

30 minutes of arrival
 General Rule of Measurement

• Any measure that is based on a simple metric

such as a number or time interval is probably
a bad one as it can be too easily “gamed”
– too simple to reflect the complexity - the
multidimensional & contextual aspects of life
– a simple ‘satisfaction survey’ that measure
‘customer perception’ is better
– difficult to game ‘perception’
 Some things cannot be measured

• Cynefin domains
– Simple
– Comlpicated
– Complex
– Chaotic
• You cannot correlate cause & effect in the complex
domain
• You cannot say this was an outcome of my activity
• And thus things cannot be directly measured
 Cannot measure the new

• If doing something new

• Cannot plan to measure
• As unsure of the outcomes
• Act on gut feel
• Can only measure in retrospect
 Measures Distort

• Measures distort behaviour

– and have unintended and unimagined side-effects
– always detrimental to the whole!
 Softer Measures

• Outcome based measures

• Activity based measures

• Anecdotal stories
– Success stories
– Focused on outcomes
• Surveys and polls
– Can provide numbers
 Best Measures

• To provide feedback to facilitate learning

• NOT for control or conformance

• Must be developed, owned and bought into by the

people involved otherwise they will be ‘gamed’

• They are personal learning tools!

When a measure becomes an objective it
ceases to be a good measure!
 What and How?

• Only now think about what you are going

measure and how you plan to do it!!
Rewards?
 Don’t reward people!

• Rewarding people for meeting targets is

detrimental to
– Quality
– Motivation
– Pride in work
• Leads to gaming
• What about sales?
– Highly quantifiable outcome
– But even then has side-effects
Here is what Alfie Kohn has to say
about rewards

To the best of my knowledge, no

controlled scientific study has
ever found a long-term
enhancement of the quality of
work as a result of any reward
system

http://
www.alfiekohn.org
 Rewards Punish

• Threats & coercion destroy motivation and so do

rewards
• Rewards are manipulative
• “Do this and you will get that” is not much different to
“Do this else here is what will happen to you”
• When people do not get the reward they hoped for
they feel punished
• The more desirable the reward the more demoralizing
it is to miss out
 Rewards rupture relations

• Excellence depends on teamwork

• Rewards destroy cooperation
– Especially if scarce or valuable
• Incentive driven employees will not ask for
help from their manager when they need it
• They will conceal problems from their
manager to appear infinitely competent
 Rewards ignore reasons

• To solve problems people must understand

the causes
• They ignore the complexities of the problems
• Each situation calls for a different response
• Rewards tend to blindly promote a single
solution
 Rewards deter risk-taking

• People are less likely to take risks; to explore

possibilities; to play hunches
• The No. 1 casualty of rewards is creativity
 Rewards undermine interest

• Loving what you do is a more powerful

motivator than any goody including money
• Rewards are controlling!
• If people focus on getting a reward they tend
to feel their work is no longer freely chosen
and directed by them
• If they have to bribe me to do it - it must be
something I don’t want to do!
 Summary

• Beware of measures!
• Think carefully!
• Measure outcomes not activities
• Chose measures that can not be easily
gamed
• Measures are best as learning tools 
• Don’t reward outside the normal ‘appraisal
system’
David Gurteen
• Knowledge Networker
• Knowledge Website
– www.gurteen.com
• Knowledge Community and
Knowledge Letter
– 13,000 people
– 145 countries
• Knowledge Cafés
– London, Liverpool, Bristol, New
York, Adelaide, Zurich
• Educator, speaker, facilitator,
coach, consultant
www.gurteen.com

David Gurteen
Gurteen Knowledge
United Kingdom
Tel: +44 1252 812 878
Email: david.gurteen@gurteen.com