Professional Documents
Culture Documents
An Effectiveness Measurement Model For Knowledge Management
An Effectiveness Measurement Model For Knowledge Management
An Effectiveness Measurement Model For Knowledge Management
Knowledge-Based Systems
journal homepage: www.elsevier.com/locate/knosys
a r t i c l e i n f o a b s t r a c t
Article history: The aim of this study is to develop a model for the measurement of the effectiveness of knowledge man-
Received 26 November 2005 agement in Taiwanese high-tech enterprises. Following a survey of the relevant literature on the subject,
Received in revised form 12 August 2008 the study describes the construction of the model – including the opinions of specialists, scholars, and
Accepted 17 February 2009
practitioners of knowledge management practice among Taiwan’s high-tech firms, the use of focus
Available online 26 February 2009
groups, the application of analytic hierarchy process (AHP), and a questionnaire analysis of qualitative
and quantitative methods. The study summarizes the experts’ opinions, selects the measurement indica-
Keywords:
tors, and calculates the weightings of dimensions and items. An empirical study is then conducted to test
Knowledge management
Effectiveness
the validity and reliability of the model, and its suitability for improving the measurement of knowledge
Measurement management effectiveness in high-tech firms.
Analytic hierarchy process (AHP) Ó 2009 Elsevier B.V. All rights reserved.
In today’s competitive business environment, knowledge man- KM can be defined as the creation, acquisition, sharing, and uti-
agement (KM) is increasingly recognized as a significant factor in lization of knowledge for the promotion of organizational perfor-
gaining a competitive advantage [8,9,14–16,18,23]. To obtain such mance [12]. Arthur Anderson Business Consulting [4] proposed a
a competitive advantage, companies must know how to manage schematic representation of the relationships among data, infor-
organizational knowledge by expanding, disseminating, and mation, knowledge, and wisdom (see Fig. 1), and stated that data,
exploiting it effectively [6,21]. information, and knowledge are necessary for dealing with regular
Assessing the effectiveness of KM operations is thus an impor- affairs, whereas wisdom is necessary for dealing with irregular af-
tant issue, but the measures that are available to evaluate the fairs and adopting appropriate actions when faced with a changing
effectiveness of KM are generally unsatisfactory. There is a need environment. According to their view, KM not only manages
to develop an assessment model that can be used to make an accu- knowledge, but also encourages individuals to utilize knowledge
rate assessment of the effectiveness of KM. This paper establishes effectively while working.
such a model. The model can be used to evaluate KM performance The term ‘performance’ refers to a measurement of extent to
in terms of the control and coordination of organizational which an organization reaches a given objective; and the term
knowledge. ‘operational performance’ refers to the measured effect of each
The steps adopted in the construction of the model include: (i) operational variable within overall ‘performance’. To evaluate the
consulting with specialists, scholars, and practitioners in knowl- ‘operational performance’ of KM, the American Productivity and
edge management among Taiwan’s high-tech enterprises; (ii) Quality Center [3] and Arthur Anderson Business Consulting [4]
obtaining the views of a focus group; (iii) the application of ana- developed a knowledge management assessment tool (KMAT) in
lytic hierarchy process (AHP); and (iv) a questionnaire survey of 1995. KMAT can be used by enterprises to select the appropriate
qualitative and quantitative methods used in KM assessment. type of KM. KMAT is composed of five fundamental elements: (i)
The validity and reliability of the model is then tested through strategy and leadership; (ii) culture; (iii) technology; (iv) measure-
empirical tests in high-tech firms. ment; and (v) knowledge management process. Four key success
An objective and comprehensive KM effectiveness assessment factors were proposed: (i) procedures of KM adopted; (ii) persons
model is thus established and presented for use by organizations involved in KM; (iii) supporting organizational structure for KM;
engaged in knowledge management. and (iv) information technology utilized in KM. However, at pres-
ent, the structure, processes, and procedures of KM have not been
defined as a concrete standard, and it is difficult to find compre-
hensive and explicit reference criteria.
* Tel.: +886 7 3617141x3461; fax: +886 7 3617141x3451. Andrew et al. [2] identified a strong relationship between orga-
E-mail address: ywen@mail.nkmu.edu.tw nizational effectiveness and KM capability (see Fig. 2).
0950-7051/$ - see front matter Ó 2009 Elsevier B.V. All rights reserved.
doi:10.1016/j.knosys.2009.02.007
364 Y.-F. Wen / Knowledge-Based Systems 22 (2009) 363–367
Table 1
Create value via action and application Organizational KM activity measures.
Culture
Organizational Table 3
Acquisition Effectiveness Judgment scores for the importance/preference of criteria using AHP.
H2 (0.058)
I1 (0.078)
five main criteria: (i) human resources; (ii) information; (iii) data; I2 (0.047)
(iv) knowledge; and (v) wisdom. The third level consists of 30 I3 (0.049)
Information
alternatives. (Fig. 3) shows the performance hierarchy designed (0.277)
for this problem. I4 (0.034)
I5 (0.036)
4.2. Pair-wise comparison of criteria and calculation of relative
I6 (0.033)
weights
the results over all levels, the priorities at each level were weighted Table 4
by the priority of the higher-level criterion with respect to which The weight of each index (overall weights).
the comparison was made. The eigenvector scaling technique of Index Weights
AHP then models the relative weights for each category (priorities) Staff (H1–H6)
and for each ratio (local weights). Global weights for each ratio 1. Rate of staff severance (H1) 0.276
were calculated as the product of its local weight and its category’s 2. Work attitude of staff (H2) 0.260
priority. 3. Average work seniority of staff (H3) 0.094
4. Number of staff who have obtained professional a certificate (H4) 0.054
There were 5 criteria and 6 alternatives, and then there were 5 5. Number of KM staff (H5) 0.169
principal vectors, all of which had 6 elements. Once the matrices in 6. Cost of education & training (H6) 0.147
each level were completed, the relative importance of the elements Information (I1–I6)
in that level was given by the principal right eigenvector of the ma- 1. Flow and utilization rate of network (I1) 0.281
trix of judgments. The number of eigenvectors (that is, local prior- 2. Periodical evaluation and knowledge update rate (I2) 0.170
ity vectors) was therefore equal to the number of criteria. The 3. Degree of information network system construction (I3) 0.180
4. Coordinating and integrating through interior information (I4) 0.121
results quantified the decision-maker’s preference for each alterna-
5. Information management capability (I5) 0.130
tive and provided a means for answering the ‘type-of-manage- 6. Information resources application effectiveness and data 0.118
ment’ question. completeness (I6)
The procedures used to solve this hierarchy system were as Data (D1–D6)
follows. 1. Time validity of customer complaint response 0.241
2. Number of improvement proposals by staff 0.148
A pair-wise comparison was made of the criteria with respect to 3. Information-based degree of knowledge 0.092
4. Knowledge document standardization 0.213
the goal. In the problem there were 5 criteria in level 2. The
5. Plan of KM 0.143
experts made three pair-wise judgments among 10 with respect 6. Establishment of customer knowledge database 0.163
to level 1. After the construction of the pair-wise comparison Knowledge (K1–K6)
matrix, the next step was to retrieve the weights of each ele- 1. Return rate of innovation 0.200
ment in the matrix. The comparison results and the weights of 2. Acquiring and utilizing extent of knowledge 0.160
three criteria are shown in Fig. 4. The principal vector was com- 3. Support of KM by high-level superintendent 0.330
4. Knowledge sharing on staff 0.144
puted and can be interpreted as the relative importance of each
5. Repaid by superintendent for working performance 0.102
of the criteria. 6. Contribution of technology innovation 0.064
A pair-wise comparison was made of the 30 alternatives in level
Wisdom (W1–W6)
3 with respect to 5 criteria in level 2. The comparison and the 1. Respect for intellectual property rights 0.254
relative contributions (that is, weights) among 30 alternatives 2. Innovation capability of staff 0.241
with respect to the three criteria results are shown in Fig. 4. 3. Investment on professional staff 0.184
The final stage of the AHP was to compute the contribution of 4. Number of staff who obtained qualified international accreditation 0.100
5. Operating income of innovation staff 0.087
each alternative to the overall goal (that is, effectiveness of
6. Number of patent obtained 0.134
KM). The overall priority for each alternative was obtained by
totalling the product of the criteria weight and the contribution
of the alternative, with respect to that criterion. The final practice, the importance of each KM effectiveness measurement
weights were obtained, and a ranking was made of the alterna- index is not exactly the same. Therefore, 10 experts were asked
tives with respect to the goal. The results are shown in Fig. 4. to answer an AHP questionnaire and the relative weights of KM
effectiveness were compared.
4.4. Measurement of pair-wise comparison consistency
4.5. Results of the AHP application
To control the result of the method, the consistency ratio for
each of the matrices and overall inconsistency for the hierarchy Following the procedure described above, the relative ranking
were calculated. of the five criteria was then determined to facilitate the selection
The deviations from consistency are expressed by the following of the best alternative.
equation, and the measure of consistency is called the consistency On the basis of the preference vectors, the five criteria of KM
index (CI) effectiveness were ranked as follows: information (0.277), staff
(0.223), wisdom (0.213), knowledge (0.184), and data (0.103)
kmax n
CI ¼ : ð3Þ (see Fig. 4 and Table 4).
n1
To synthesize the opinion of academic and enterprise experts,
The consistency ratio (CR) is used to estimate directly the con- ‘information’ and ‘staff’ were the most important criteria. ‘Rate of staff
sistency of pair-wise comparisons. The consistency ratio (CR) is severance’ was the most important alternative among those in the
computed by dividing the CI by a value obtained from a table of ‘Staff’ criterion. ‘Respect for intellectual property rights’ was the most
the random consistency index (RI). important alternative among those under the criterion of ‘wisdom’.
CI
CR ¼ : ð4Þ 4.6. Calculation of KM effectiveness
RI
If the CR is less than 0.10, the comparisons are acceptable. For After obtaining the weights of constructs and criteria of KM
this application, all CR inconsistency ratios values were less than effectiveness, 30 indices were normalized to obtain a standardized
0.1 (CR < 0.1); therefore, all the judgments were consistent. Each value (that is, Z value). The purpose of normalization of these indi-
alternative possesses a score on all criteria. The criteria scores were ces was to deal with the problem caused by different units. The
combined into an overall score. The overall score indicates the rel- Z value is between 0 and 1. The sum of each index multiplied
ative importance of each alternative. by the corresponding weight was the standardized value of
In the past, most of the relative weights of a KM effectiveness constructs. The sum of each construct multiplied by the corre-
measurement index were said to be of equal weight. However, in sponding weight created a score of KM effectiveness.
Y.-F. Wen / Knowledge-Based Systems 22 (2009) 363–367 367