See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/233832692

An Empirical Test of the DeLone-McLean Model of Information System Success

Article  in  ACM SIGMIS Database · June 2005

DOI: 10.1145/1066149.1066152 · Source: DBLP

CITATIONS READS
346 4,862

1 author:

Juhani Iivari
University of Oulu
125 PUBLICATIONS   5,302 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

DDSR project - Discourses on design science research View project

Slowly retiring View project

All content following this page was uploaded by Juhani Iivari on 31 May 2014.

The user has requested enhancement of the downloaded file.

 Abstract
An Empirical This paper tests the model of information system

Test of the success proposed by DeLone and McLean using a

field study of a mandatory information system. The re-
sults show that perceived system quality and
DeLone-McLean perceived information quality are significant predictors
of user satisfaction with the system, but not of system
Model use. Perceived system quality was also a significant
predictor of system use. User satisfaction was found

of Information to be a strong predictor of individual impact, whereas

the influence of system use on individual impact was
insignificant.
System Success ACM Categories: J.1, K.6.2
Keywords: Information System Success, Infor-
mation System Quality, System Quality, Information
Juhani Iivari Quality, User Satisfaction, Use, Individual Impact
University of Oulu
Introduction
Seddon et al. (1999) estimate that the total annual
worldwide expenditure on information technology (IT)
probably exceeds one trillion US dollars per year and
is growing at about 10% annually. At the same time,
information systems are pervading almost all aspects
of human life. In view of the high investments in IT
and its ubiquity, the success of such investments and
the quality of the systems developed is of the utmost
importance both for research and in practice.
This paper focuses on the success of individual
information system applications. Following
Gustafsson et al. (1982), we interpret an information
system (IS) as a computer-based system that
provides its users with information on specified topics
in a certain organizational context. DeLone and
McLean (1992) proposed in their influential paper a
framework for IS success measures that distinguishes
system quality, information quality, user satisfaction,
use, individual impact and organizational impact.
They also suggested a causal model for the success
measures.
Despite the considerable interest in the DeLone-
1
McLean model , there is a dearth of studies that test it
empirically. DeLone and McLean (2002) identify only
sixteen empirical studies that have explicitly tested
some of the associations of the original DeLone-
McLean model. Among them Seddon and Kiew
Acknowledgment (1994) revised it considerably, by deleting system use
and substituting perceived usefulness. In our view
I wish to express my gratitude to Minna Perälä, M.Sc.,
perceived usefulness reflects more the individual
for the data collection, and especially to Prof. Wynne
impact (Rai et al., 2002), i.e. the impact of the system
Chin for his comments and for helping me to use
PLS.
1
The Science Citation Index, Social Science Citation Index and
This paper was submitted in February of 2002. Wynne Chin
Arts & Humanities Index identify 235 references to the article (as
served as the Senior Editor.
of January 10, 2002).

8 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
 2
on a user’s performance of his/her job. The idea of
this paper is to test the DeLone-McLean model while
sticking more faithfully to its original form. Leidner
(1998) reports a partial test of the model in the case
of Executive Information Systems, and more recently,
Rai et al. (2002) tested both the DeLone-McLean
(1992) model and the Seddon (1997) model, reporting
reasonable support for both.
The composition this paper is as follows: Section 2
discusses the theoretical background; Section 3
introduces the research method; Section 4 describes
the results; Section 5 discusses the results; and
Section 6 concludes the paper.
Theoretical Background
The DeLone-McLean Model for IS Success
The DeLone-McLean model for IS success, described
in Figure 1, assumes that system quality and
information quality, individually and jointly, affect user
satisfaction and use. It also posits use and user satis-
faction to be reciprocally interdependent, and
presumes them to be direct antecedents of individual
impact, which should also have some organizational
impact.
DeLone and McLean (1992) characterize system
quality as desired characteristics of the information
system itself, and information quality as desired
characteristics of the information product. More
concretely, they incorporate four scales from the
Bailey-Pearson (1983) instrument into system quality
(convenience of access, flexibility of the system, inte-
gration of the system and response time) and nine
scales into information quality (accuracy, precision,
currency, timeliness, reliability, completeness,
conciseness, format and relevance).
Much of the research on User Information
Satisfaction has concerned users’ satisfaction with
specific features of a system (Doll & Torkzadeh,
1988; Iivari & Koskela, 1987) or IS function (Bailey &
Pearson, 1983; Baroudi & Orlikowski, 1988), covering
features of both system quality and information
quality. Even though the inclusion of service quality in
the updated DeLone and McLean (2002) model
reflects IS functions or IS organizations rather than IS
application, the following will focus on the success of
IS applications only. User satisfaction in DeLone and
McLean (1992) refers to the overall user satisfaction
(Seddon & Kiew, 1994) measured independently of
2
Davis’ original measure for perceived usefulness was developed
to assess ex ante expectations of individual impact. We focus here,
however, on ex post individual impact measured after six months’
experience of use.
The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
system quality and information quality. Otherwise the
relationship between system/information quality and
user satisfaction would be an artifact of measure-
ment.
Seddon (1997) claims that the DeLone-McLean
model is ambiguous in the sense that one component
of it, use, has three potential meanings (Table 1). His
conclusion is that only Meaning 1 is justified in the
light of the objections listed in the second column of
Table 1. To me all these points of criticism seem
questionable. His criticisms of Meaning 2 and
Meaning 3 refer to the distinction between a variance
3
model and a process model (Mohr, 1982). Without
going into the details of this distinction, it is obvious
that even though IS use as a process is assumed to
lead to individual impact and organizational impact, it
is not necessary to regard it as a discrete event to be
stated (use vs. non-use), as implied by process
4
theories (Mohr, 1982). This paper interprets use as
the amount of use, which may be considered one
measure of IS success.
DeLone and McLean (1992) characterize individual
impact as “an indication that an information system
has given a user a better understanding of the
decision context, has improved his or her decision-
making productivity, has produced a change in user
activity, or has changed the decision maker’s
perception of the importance or usefulness of the in-
formation system” (p. 69). Seddon (1997) reinterprets
individual impact to mean benefits accruing to
individuals from use. Even though both DeLone and
McLean (1992) and Seddon (1997) implicitly
presuppose that individual impact is of benefit to the
user, this paper interprets individual impact as
referring to a unit of analysis rather than the benefi-
5
ciary of the impact.
3
As Mohr (1982, p.44) points out, he uses the term “process
theory” in a highly specific meaning. “Process theory” does not
imply that variance theories cannot address processes (e.g. IS
acceptance).
4
We interpret the DeLone-McLean model as based on the
reasoning that a system that is not used at all does not have any
individual or organizational impact. On the other hand, the DeLone-
McLean model also allows the hypothesis that more use is
associated with more individual impact, which follows the “logic” of
variance theories.
5
The likely explanation for this assumption is that the above
authors implicitly assume that use of the system is voluntary. In that
case a user will hardly continue to use a system if he or she does
not perceive its use as beneficial. However, according to my
reading, DeLone and McLean (1992) do not explicitly restrict their
model to voluntary systems, although they do note that actual use
makes sense only when system use is voluntary.
9
 System
Use
Quality

Individual Organizational
Impact Impact

Information User
Quality Satisfaction

Figure 1. The DeLone-McLean Model for IS Success

Meaning Seddon’s objections Counter-objections

Meaning 1: Benefits (The only justified mean- What is the meaning of “use” in “benefits from
from use ing) use”?
Meaning 2: Use as IS success must bring Can’t a system (e.g. a piece of free software
the dependent vari- benefits to somebody. such as Linux) be genuinely considered a suc-
able in a variance cess when it is widely used without any consi-
model of future use deration of its benefits or disadvantages to
different stakeholders?
Meaning 3: Use as IS use is a process con- Even though IS use as a process is assumed
an event in a pro- struct that should not have to lead to individual impact and organizational
cess leading to indi- any place in a variance impact, it is not necessary to regard it as a
vidual or organiza- model predicting IS discrete event to be stated (use vs. non-use),
tional impact success. as implied by process theories.

Table 1. The Three Meanings of IS Use in the DeLone-McLean Model, According to Seddon (1997)

Following DeLone and McLean (1992) and Rai et al.
(2002), we will specifically focus in this paper on the
effect of an information system on the work perform-
ance of individual users as measured by perceived
6
usefulness.
Hypotheses
DeLone and McLean (1992) introduce the model
shown in Figure 1 primarily as a causal-explanatory
model of how system quality and information quality
affect use and user satisfaction, how use and user
satisfaction, affecting each other reciprocally, are
direct antecedents of individual impact, and how
individual impact leads to organizational impact. As
an alternative, one could emphasize more the
predictive nature of the model, how the preceding
variables help to predict the dependent variables,
6
We do claim that perceived usefulness covers all aspects of
individual impact. DeLone and McLean (1992) specifically focus on
decision-makers as users of an information system. Assuming that
the work of decision-makers is to make decisions, perceived
usefulness essentially covers the impact on decision-making
productivity. Perceived usefulness nevertheless misses those
aspects of individual impact which do not directly concern work
performance, e.g. the impact on the quality of work (Iivari, 1997).
even though the causal explanation of the relationship
is not totally clear. The criticism of Seddon (1997),
even though we do not accept it in its entirety, shows
that some of the assumed causal relationships in the
DeLone-McLean model are arguable and the model
is incomplete. In particular, the model misses the
feedback loops from individual impact and organiza-
tional impact to user satisfaction and use. We
interpret the DeLone-McLean model primarily as a
predictive one that is worth testing empirically. Based
on the DeLone-McLean model, we propose to test the
hypotheses depicted in Figure 2 in the present paper.
It is hypothesized in Figure 2 that system quality and
information quality are positively associated with user
satisfaction. Hypothesis H1 assumes that ceteris
paribus the higher the system quality is perceived to
be by users, the more satisfied they are with the
system. Similarly, Hypothesis H2 posits that ceteris
paribus the higher the information quality is perceived
to be by users, the more satisfied they are with the
system. If user satisfaction is interpreted as an
attitude (Baroudi et al., 1986), hypotheses 1-2
essentially argue that the attitude is dependent on
perceptions of the attitude object (Fishbein & Ajzen,

10 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
1975; McGuire, 1969). There is considerable
empirical evidence for these hypotheses.
H3:+ Actual use
System
quality
H7:+
H1:+
H5a:+ H5b:+ Individual
impact
H4:+
H6:+
User
Iinformation
quality H2:+ satisfaction
Figure 2. The Model to Be Tested
Many of the instruments developed to measure User
Information Satisfaction (UIS) in terms of attributes
such as system quality and information quality (e.g.
Bailey & Pearson, 1983; Ives et al., 1983; Doll &
Torkzadeh, 1988) have used an independent
measure of overall user satisfaction to test the
predictive validity of the measure. They have
consistently reported significant correlations between
UIS or its factors and the independent measure of
overall satisfaction. Doll and Torkzadeh (1988), for
example, found correlations varying between 0.51
and 0.65 between the twelve items of their end-user
satisfaction measure and the criterion variable that
can be interpreted as overall satisfaction. The
correlations between the five factors (content,
accuracy, format, ease of use and timeliness) and the
criterion varied between 0.55 and 0.69, and that
between the 12-item instrument and the criterion
variable was 0.76. Further, Seddon and Kiew (1994)
found in their path analysis that information quality
and system quality are significant determinants of
overall user satisfaction (both path coefficients
7
significant at the level 0.001). Similarly Rai et al.
(2002) report significant path coefficients between
ease of use (used to measure system quality) and
user satisfaction (ß = 0.30, p ≤ 0.01) and between
information quality and user satisfaction (ß = 0.52, p ≤
0.01) in their LISREL analysis of the DeLone-McLean
model.
DeLone and McLean (1992) hypothesize that the
higher the system quality, the more the system is
used (Hypotheses H3) and the higher the information
quality, the more the system is used (Hypothesis H4).
7 paper is confined to a single point in time, however,
System quality was measured using two items from Doll and
Torkzadeh (1988), four items of ‘ease of use’ from Davis (1989)
and three additional items. The measure of information quality
consisted of ten items from the Doll and Torkzadeh (1988)
instrument. (Overall) satisfaction with the system was measured
using four items: how adequately the application meets the
information processing needs, how efficient it is, how effective it is
and overall satisfaction.
The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
At a general level, there is a considerable body of em-
pirical research on the relationship between UIS
(measured in terms of attributes such as system
quality and information quality) and IS use, which
suggests that the relationship is positive but relatively
weak (Amoroso & Cheney, 1991; Barki & Huff, 1985;
Baroudi et al., 1986; Ginzberg, 1981; Igbaria, 1990;
Igbaria & Zviran, 1991; Nelson & Cheney, 1987;
Srinivasan, 1985). Baroudi et al. (1986), for example,
found a correlation of 0.28 between UIS and IS use,
and Barki and Huff (1985) 0.39. More specifically re-
lated to Hypothesis H3, the Technology Acceptance
Model (Davis et al., 1989) predicts that perceived
ease of use, as an aspect of system quality (DeLone
& McLean, 1992), is a significant direct and indirect
determinant of use, the indirect effect being
channelled through perceived usefulness. After a de-
cade of intensive research into TAM, there is
significant empirical evidence of the indirect effect of
perceived ease of use (Davis, 1989; Davis et al.,
1989; Mathieson, 1991; Adams et al., 1992; Davis et
al., 1992; Igbaria et al., 1995; Igbaria & Iivari, 1995;
Chau, 1996; Igbaria et al., 1996; Szajna, 1996; Taylor
& Todd, 1995, Gefen & Straub, 1997; Igbaria et al.,
1997; Straub et al., 1997; Gefen & Keil, 1998;
Karahanna & Straub, 1999; Karahanna et al., 1999;
Venkatesh & Davis, 2000), whereas the direct effect
is much more controversial (Gefen & Straub, 2000).
Consistent with the above, Rai et al. (2002) report a
relatively low path coefficient between ease of use
and system dependence (used to measure system
use) (ß = 0.09, p ≤ 0.10) and a somewhat higher
coefficient between information quality and system
dependence (ß = 0.18, p ≤ 0.01). Leidner (1998)
reports positive associations between perceived EIS
information quality and frequency of EIS use (ß =
0.38, p ≤ 0.01) and between perceived EIS
information quality and EIS use for internal monitoring
(ß = 0.34, p ≤ 0.01), but not between EIS information
quality and EIS use for external monitoring nor
between EIS information quality and EIS use for
communication. Despite these somewhat inconsistent
findings, we will follow the original model of DeLone
and McLean (1992) as expressed in Hypotheses H3
and H4.
As mentioned above, DeLone and McLean (1992)
posit that use and user satisfaction are reciprocally
interdependent. To test this reciprocal dependence
fully, one should have a piece of research in which
use and user satisfaction are followed over time. This
although hypotheses 5a and 5b, which are to be
tested separately, do attempt to capture this
reciprocal dependence.
Hypothesis H5a predicts that the more satisfied users
are with the system, the more they will use it. Baroudi
11
et al. (1986) suggest that if user satisfaction is
interpreted as an attitude, the Theory of Reasoned
Action (Fishbein & Ajzen, 1975) supports the model
that user satisfaction will influence intentions to use
the system and actual use (Hypothesis 5a). As an
alternative, they identify Dissonance theory (Fishbein
& Ajzen, 1975), which suggests that IS use leads to
user satisfaction (Hypothesis H5b). The results of
path analysis supported the model that user
satisfaction leads to system use rather than vice
versa (Baroudi et al., 1986). Interestingly, they explain
the model as follows: “The model assumes that as
use demonstrates that a system meets a user’s
needs, satisfaction with the system should increase,
which should further lead to greater use of that
system. Conversely, if system use does not meet the
user’s needs, satisfaction will not increase and further
use will be avoided.” This explanation suggests that,
causally, IS use precedes user satisfaction, or that the
relationship is reciprocal, as assumed by DeLone and
McLean (1992). In fact, Torkzadeh and Dwyer (1994)
found a path coefficient of 0.21 (p < .05) from user
satisfaction to usage and a coefficient of 0.37 (p <
.05) from usage to user satisfaction in their LISREL
analysis. Rai et al. (2002) report a significant path
coefficient (ß = 0.35, p ≤ 0.01) from user satisfaction
to system dependence, used to measure system use.
The relatively low association between actual use and
user satisfaction may be explained by the complexity
of the relationship. Chin and Lee (2000) propose that
overall user satisfaction is composed of expectation-
based satisfaction and desire-based satisfaction. Ex-
pectation-based satisfaction is a direct and
multiplicative combination of the overall expectation
discrepancy between prior expectation and post hoc
perceptions of the system and the overall evaluation
of this expectation discrepancy. Similarly, desire-
based satisfaction is a direct and multiplicative
combination of the overall desire discrepancy be-
tween prior desires and post hoc perceptions of the
system and the overall evaluation of this desire
discrepancy.
Seddon (1997) reasons that although user
satisfaction may have an association with IS use in a
steady state, this will break down in a situation of
system replacement. Favourable satisfaction with the
old system is not sufficient “to cause use of the old
system”. Although this may be true, it is obvious that
user satisfaction is never the only cause of system
use. At a minimum, the system must be accessible to
a user. Seddon’s (1997) reasoning assumes that user
satisfaction with the old system is not affected by the
new system. To the author’s knowledge, this issue
has not been studied empirically, but one could
speculate that a user would compare the old and new
systems. After this comparison it seems quite unlikely
12 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
that a user’s satisfaction will not be affected by the
new system. One could conjecture that dissatisfaction
with the new system might lead to higher satisfaction
with the old system, whereas high satisfaction with
the new system may reduce satisfaction with the old
8
system. Further, if the user is dissatisfied with the
new system after the replacement, high satisfaction
with the old system may explain the likelihood of its
future use.
Contrary to Chin and Lee (2000), Seddon (1997) also
assumes that user satisfaction reflects past
experience with the system and does not include
expectations. Iivari (1987) points out, however, that if
user satisfaction is interpreted as the user’s belief in
the degree to which the information system (at the IS
schema level) is capable of satisfying his or her
information requirements (at the IS schema level)
(Ives, et al., 1983), the distinction between ex post
(experience-based) and ex ante (expectation-based)
interpretation disappears. At the same time, he
argues that in this case user satisfaction is
consequent upon acts of using a system rather than
being an antecedent, especially when one takes into
account alternative and complementary information
systems. The explanation is that information needs in
a specific use situation may differ from information
requirements imposed on the system at the schema
level. A manager, when he or she has time, may
prefer to visit the shop floor to inquire directly from the
employees about the state of production, because he
or she wishes to receive “soft” knowledge of a kind
that a formal information system is assumed to be
incapable of supplying. He or she may nevertheless
be totally satisfied with the reporting system and use it
when there is no opportunity to visit the shop floor. At
the same time, Iivari (1987) acknowledges that user
satisfaction may correlate highly with IS use. We also
claim that it may predict IS use.
Hypothesis 6 predicts that user satisfaction will be
positively associated with individual impact. The
meaning of this hypothesis depends on the
interpretation of user satisfaction which users employ.
Goodhue (1986) defines “IS satisfactoriness” as the
individual’s belief in the correspondence or fit
between job requirements and IS functionality, and
“IS satisfaction” as the correspondence between the
information system’s intrinsic benefits of use, such as
providing a sense of accomplishment due to a crisp,
attractive output, and the needs of the individual. Iivari
8
Relative advantage, i.e. “the degree to which an innovation is
perceived as being better than the idea it supersedes” (Rogers,
1995, p. 212) captures this idea of comparing the old and new
systems. Our conjecture above would suggest that relative
advantage is a dynamic concept that emerges in the form of an
interaction between the levels of satisfaction with the two
systems.
and Ervasti (1994) propose that when user
satisfaction is interpreted in the sense of “IS
satisfactoriness” rather than “IS satisfaction,” the link
between user satisfaction and IS effectiveness can be
established directly without the intervening variable of
IS use. If user satisfaction is the user’s best estimate
of the match between the requirements imposed on
the system by his or her work and the system capa-
bilities, the positive association between user
satisfaction and individual impact is quite
understandable. If the user’s interpretation of these
requirements and estimate of the match between
them and the system capabilities are correct,
increased user satisfaction should be positively
associated with task performance.
There are relatively few studies that have investigated
empirically the relationship between user satisfaction
and individual impact, especially when we focus on
the effect on individual job performance. DeLone and
McLean (1992) identify four papers that comprise
both user satisfaction and individual impact criteria,
among which only Cats-Baril and Huber (1987)
actually address the relationship, reporting
correlations between two measures of satisfaction on
the one hand and the quality and productivity of
decision-making on the other hand. Productivity was
measured in terms of the number of objectives
generated, the number of alternatives generated and
the number strategies prioritized. The authors do not
report the significances, but all the correlations were
negative, albeit relatively low in absolute terms. One
of the eight correlations was just over 0.30 in absolute
terms. More recently, Gatian (1994) conducted a
LISREL analysis of the relationship between user
satisfaction, decision performance and user efficiency
in the case of direct and indirect users of the same
system in 39 organizations. In the case of direct
users, she found a close association between user
satisfaction and decision performance (r = 0.64, ß =
0.64) and similarly between user satisfaction and
efficiency (r = 0.68, ß = 0.97), but in the case of
indirect users only the relationship between user
satisfaction and user efficiency was of interest. This
was also found to be significant (r = 0.81, ß = 0.81).
Her findings may partly be inflated by the fact that her
user satisfaction measure was adapted from the
Jenkins-Ricketts (1979) instrument and decision
performance from Sanders (1985). User efficiency
was also evaluated in terms such as user’s data
processing correctness, report preparation and
distribution timeliness. Etezadi-Amoli and Far-
hoomand (1996) report that six factors of end user
computing satisfaction (documentation, ease of use,
functionality, quality of output, support and security)
explained 50% of the variance in end user
performance. They also found that satisfaction with
the quality of the output and satisfaction with the
The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
functionality of the system were the most significant
predictors, whereas documentation was the least
significant. The paper of Seddon and Kiew (1994)
also analyses the relationship between user
satisfaction and individual impact when the latter is
interpreted as perceived usefulness. They report a
9
correlation 0.70 (p < 0.001) between them.
Finally, Hypothesis 6 predicts that IS use is positively
associated with individual impact. Theoretically, this
relationship can be argued based on the reasoning
that a system that is not used at all will not have any
impact on individual performance. Furthermore, one
could also expect that a system that is used more will
have higher impact on users’ performance. Chin and
Marcolin (2001) point out, however, that the impact of
initial usage on individual productivity may differ from
that of continued usage. The interest of the present
paper lies in the continued use of a system rather
than its initial use. There seems to be a paucity of
research into the relationship between IS use and
individual impact, but the existing evidence seems to
support this hypothesis. DeLone and McLean (1992)
identify seven studies that address both system use
and individual impact. Among these, Srinivasan
(1985) reports time per session (connect time) and
user type (light, average, heavy), among four
indicators of system use, to be significantly correlated
with the problem solving capabilities of the user (p ≤
0.10 and p ≤ 0.01, respectively). Snitkin and King
(1986) found a significant association between sys-
tem usage and perceived effectiveness (p ≤ 0.05).
More recently, Iivari (1996) found CASE usage to
have a significant effect on the productivity of
individual users (systems developers) and on the
quality of their products. Leidner (1998) reports that
EIS use to monitor internal information and EIS use to
monitor external information were both positive and
significant predictors of individual decision making
speed (ß = 0.34, p ≤ 0.05; ß = 0.24, p ≤ 0.05,
respectively), mental mode enhancement (ß = 0.38, p
≤ 0.01; ß = 0.28, p ≤ 0.05, respectively) and extent
analysis in decision making (ß = 0.36, p ≤ 0.01; ß =
0.25, p ≤ 0.05, respectively). On the other hand,
frequency of EIS use was not a significant predictor of
any of the three aspects of individual impact.
On the other hand, there is an abundant literature on
the effect of perceived usefulness on system use
(Lee, 2003; Ma and Liu, 2004), where perceived
usefulness is interpreted as an ex ante belief
(expectation) regarding “the degree to which a person
believes that using a particular system would enhance
9
The high correlation between overall satisfaction and perceived
usefulness may be explained by the fact that two of the four items
of user satisfaction, concerning the efficiency and effectiveness of
the application, may be interpreted in terms of perceived
usefulness.
13
his or her job performance” (Davis, 1989). As noted
above, there is significant empirical evidence that per-
ceived usefulness has a positive effect on attitudes
towards use, behavioural intention to use and actual
use. Perceived usefulness as an antecedent of
system use serves to point out that the relationship
between system use and individual impact is probably
not uni-directional but that a user’s experience of
individual impact (in terms of its effect on his/her job
performance) will affect his/her belief in the perceived
usefulness of a system and in that way its use.
Research Methodology
The Field Study
The model of Figure 2 was tested as a part of a
larger, longitudinal field study in Oulu City Council,
which is a municipal organization of about 7500
employees. The organization in question formed a
concrete setting where some of its employees were
working on the adoption of a new information system
and shaping its organizational acceptance. We saw it
as an opportunity to study a specific case of “real”
users’ acceptance of a “real” system in “real” time.
Even though the study focuses only on one system in
one organization, the research setting can be
expected to increase the internal validity of the study
in particular and to some extent its external validity as
well (Jenkins, 1985). The choice of one organization
controls for possible confounding effects of organiza-
tional level variables such as institutional constraints
and infrastructure arrangements, which may have an
influence on individual adoption and acceptance,
making it more likely that micro-level effects will be
detected (Karahanna et al., 1999).
Oulu City Council renewed its financial and account-
ing systems at the beginning of 1997 as an outcome
of a nation-wide reform of municipal financial and
accounting systems. As a part of this reform, it
acquired an application package from a major vendor
in Finland, including accounting, sales receivable,
payments receivable and invoicing. The field study
was targeted at about 100 primary users of the sys-
tem who participated in the training provided by the
vendor in October and November 1996, of whom 78
agreed to participate. Data collection, based on
questionnaires, was conducted during summer 1997
after half a year of experience with the system.
Measurement of the Variables
The questionnaire was based as much as possible on
standard measures (see Appendix A), the questions
being translated into Finnish. System quality was
measured using six scales adopted from Bailey and
Pearson (1983): flexibility of the system, integration of
the system, response/turnaround time, error recovery,
14 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
convenience of access, and language. Similarly,
information quality adopted six scales from Bailey and
Pearson (1983): completeness, precision, accuracy,
reliability, currency, and format of output. Each scale
was measured using four items, as proposed by the
source authors.
User satisfaction was measured using the six items of
general reactions suggested by Chin et al. (1988) and
actual use in terms of daily use time and frequency of
use. Individual impact was confined to impact on the
user’s work performance and was measured with an
adaptation of the 6-item instrument for perceived
usefulness suggested by Davis (1989).
10
Data Analysis
Gefen et al. (2000) provide a recent comparison of
traditional regression analysis and two classes of
structural equation modelling, covariance-based and
partial-least-square-based, which are potentially
suitable for testing the model of Figure 2. Their
guidelines suggest that covariance-based structural
equation modelling approaches such as LISREL,
EQS and AMOS do not suit the case at hand, for two
major reasons. Firstly, they are suitable for
confirmatory rather than exploratory analyses and
require a strong underlying theory. Actually, the
covariance-based structural equation modelling is
oriented towards causal modelling and theory testing
rather than prediction (Chin & Newsted, 1999), which
is the major, purpose of the present paper. As pointed
out by Seddon (1997), the underlying theory of the
DeLone-McLean model is not very strong. Secondly,
the number of cases in the present material, 68, is
quite small for covariance-based structural equation
modelling methods, which also impose tighter
statistical assumptions than regression analysis and
partial-least-square-based methods.
The hypothesized relationships among the study
variables depicted in Figure 2 were tested by the
Partial Least Squares (PLS) method, which is
particularly well suited for predictive applications and
theory building (Chin & Newsted, 1999). It does not
imply parametric assumptions of multivariate normal
distribution, and the sample size can be small, the
minimum being ten times of the number of items in
the most complex construct in the model (Chin, 1998;
Gefen et al., 2000).
PLS recognizes two components of a causal model:
the measurement model and the structural model. A
structural model consists of the unobservable, latent
constructs and the theoretical relationships among
10
Observe, however, that perceived usefulness is mainly used to
measure ex ante expectations of the system’s impact in terms of
speed of accomplishing tasks, job performance, productivity,
effectiveness, ease of job and usefulness in work (Davis, 1989).
them. Testing this includes estimating the path coeffi-
cients, which indicate the strengths of the
relationships between the independent and de-
pendent variables. Furthermore, for each construct in
a structural model, there is a related measurement
model which links the latent construct in the diagram
with a set of observed items. The measurement
model consists of the relationships between the
observed variables (items) and the latent constructs
which they measure. The characteristics of this model
demonstrate the construct validity of the research
instruments, i.e. the extent to which the operationali-
zation of a construct actually measures what it pur-
ports to measure. Two important dimensions of
construct validity are (a) convergent validity, including
reliability, and (b) discriminant validity. Together, the
structural and measurement models form a network
of constructs and measures. The item weights and
loadings indicate the strengths of the measures, while
the estimated path coefficients indicate the strengths
and signs of the theoretical relationships.
More specifically, a molar approach (Bagozzi, 1985;
Chin & Gopal, 1995) was adopted to the testing of the
model in Figure 2. Both system quality and
information quality were considered to be second
order concepts influenced by six scales each (see
Figure 3). To measure system quality, the 24 items
covering flexibility, system integration, response time,
recoverability, convenience, and common language
were used as reflective measures (Chin, 1998).
Similarly, the 24 items covering completeness, preci-
sion, accuracy, consistency, currency, and format
were used to measure information quality.
Results
The model of Figure 2 was tested using PLS-Graph,
version 03.00 software. As Hypothesis H5 in Figure 2
includes a mutual influence between use and user
satisfaction that could not be tested at the same time,
we tested two models: Model 1, which assumed the
influence to be from user satisfaction to actual use
(H5a), and Model 2 which worked from actual use to
user satisfaction (H5b).
Measurement Models
To test the measurement models Model 1 and Model
2, we examined (1) individual item loadings, (2)
internal consistency (reliability of measures), (3)
convergent validity, and (4) discriminant validity. All
the item loadings except for five of the 24 items
concerning perceived system quality and four of the
24 items concerning perceived information quality ex-
ceeded the threshold value of 0.70. The internal con-
sistencies of the latent variables, the average
variances explained (on the diagonal) and the
The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
correlations between latent variables are listed in
Table 2. The two models gave very similar results.
Only a few correlations between latent variables
11
differed by the absolute value of 0.01.
The weights and loadings of the indicators in both
models are reported in Appendix B.
The internal consistencies of all the latent constructs,
examined using the formula developed by Fornell and
Larcker (1981), clearly exceeded the cut-off value
0.70 proposed by these authors. Convergent validity
is considered adequate when the average variance
extracted is 0.50 or more (Fornell & Larcker, 1981).
As Table 2 shows, the minimum average variance
extracted was 0.54. For satisfactory discriminant
validity, the average variance shared between a
construct and its measures should be greater than the
variance shared by the construct and any other
constructs in the model (Chin 1998). As Table 2
shows, the square roots of the average variance
extracted (the lower values on the diagonal) exceed
the corresponding off-diagonal correlation values in
the corresponding rows and columns in all cases
except for perceived system quality and perceived
information quality, where the violations concern their
antecedents in the molar model of Figure 3. This
indicates adequate discriminant validity for the
constructs.
As shown in Table 2, the correlations between
flexibility, system integration, response time,
recoverability, convenience and command language
vary between 0.35 and 0.76. Analysis of collinearity
between them showed that the lowest tolerance value
was 0.22, which clearly exceeds the cut-off value 0.10
recommended by Hair et al. (1992). Similarly, the
correlations were between 0.51 and 0.93. The
tolerance value was 0.12, which is very close to the
above cut-off value.
Structural Models
The tests performed on the structural models gave
the results depicted in Figure 3. The upper path coef-
ficients and R2 values give the results for Model 1
and the lower ones for Model 2. The path coefficients
from the antecedents of perceived system quality and
perceived information quality were the same in both
models and are therefore not repeated. The bootstrap
resampling technique (500 resamples) was used to
determine the significances of the paths within the
structural model. As shown in Figure 3, perceived
system quality is a very significant predictor of user
satisfaction in both models (ß = 0.55 and 0.48,
11
In these two cases, Table 1 lists the higher correlations in
absolute values.
15
 The paths from system quality and information quality
Flexi- to user satisfaction and from user satisfaction to
Inte- bility individual impact in particular emerged in the manner
gration
hypothesized by the DeLone-McLean model. On the
0.19***
Res- 0.21*** 0.18 other hand, the paths from system quality and in-
ponse 0.14
time formation quality to actual use and from actual use to
0.19*** Per-
ceived individual impact were not significant. This negative
0.30* Actual
Re-
system
quality
0.45** use finding may be explained by the mandatory nature of
cover- 0.23*** the system, which may inflate the significance of
ability
0.22*** 0.15 actual use in the model. Our test of the DeLone-
0.15
Con.
0.16***
0.55***
McLean model was therefore limited, and there is
veni.
ence
0.48*** need to test it in the case of more voluntary systems.
Langu- Indivi-
age dual As explained in section 3.2, we do not claim that the
Impact
DeLone and McLean model provides a complete
0.28Õ 0.14Õ
Comple-
0.35
picture if interpreted as a causal-explanatory model. It
teness
Preci-
0.35 is unclear, for example, whether our finding that user
-0.19
sion
-0.12 0.52*** satisfaction predicts individual impact implies that
0.19*** 0.52***
0.20***
user satisfaction in some sense explains individual
Accu-
racy
impact or vice versa. Even though Seddon (1997)
0.18*** Per-
User
claims that “in the long run it is people’s observations
ceived
information
0.26*
0.28*
satis- of the outcomes of use, the impacts, that determine
Con. quality faction
sis- 0.23*** their satisfaction with the system, not vice versa” (p.
tency
0.57
243), we see this more as a research issue that is
0.22***
0.16***
0.58 quite complicated by nature. Our results may be
Cur- interpreted as indicating that users perceive user
rency
Format satisfaction in terms of “IS satisfactoriness” or as a
weighted combination of “IS satisfactoriness” and “IS
satisfaction” rather than “IS satisfaction” alone
(Goodhue, 1986). Assuming that user satisfaction is
Figure 3. Results of the Structural Analyses the user’s best estimate of the match between the
requirements imposed on the system by his or her
respectively; p ≤ 0.001 in both cases), while perceived work and the system’s capabilities, a positive relation-
information quality is also a significant predictor (p ≤ ship between user satisfaction and individual impact
0.05) of user satisfaction in both models. Perceived is quite understandable. If the user’s interpretation of
system quality is also a significant direct predictor of those requirements and estimate of the match
system use both in Model 1 (ß = 0.30, p ≤ 0.05) and between requirements and the system’s capabilities
in Model 2 (ß = 0.45, p ≤ 0. User satisfaction is an are correct, increased user satisfaction should be
almost significant predictor of actual use in Model 1 (ß positively associated with task performance.
= 0.28, p ≤ 0.10), and conversely actual use is an al-
most significant predictor of user satisfaction in Model Our test of the DeLone-McLean model was
2 (ß = 0.14, p ≤ 0.10). Most notably, user satisfaction incomplete in the sense that we did not include
is a strong predictor of individual impact in both organizational impact. In addition to the data analysed
models (ß = 0.52, p ≤ 0.001 in both), whereas actual above, we had access to managers’ evaluations of
use is insignificant as a predictor of individual impact. the organizational impact of the system (n = 38) in 15
12
Overall, both models explain a considerable portion of user departments. Organizational impact was
the variance in both user satisfaction (R2 = 0.57 in evaluated in terms of the impact of the system on the
unit’s output, the quality of this output and the unit’s
Model M1 and R2 = 0.58 in Model 2) and individual innovativeness, reputation for excellence and morale
impact (R2 = 0.35 in both models). (Van de Ven & Ferry, 1980). Factor analysis of the
five items based on the 38 responses gave only one
factor.
Discussion
The findings regarding the seven hypotheses
derived from the DeLone-McLean model (1992)
proposed in section 3.2 are summarized in Table 3.
12
Overall, the results support the reasonableness of The 38 managers’ evaluations were largely independent of the
the DeLone-McLean model as a predictive model. 78 user responses. There were only four common respondents in
the two samples.

16 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
 Inter-correlations of latent constructs and average variances extracteda

Int. Consistency
No of Items

Com language

Completeness
Recoverability

Convenience
Construct

Consistency
Ind Impact

Integration

Response

Accuracy
Precision
Flexibility

Currency

Format
PSQ

USE
PIQ

UIS
Perceived 24 0.97 0.54
Syst. Quality 0.74
(PSQ)
Perceived Inf. 24 0.98 0.62
Quality (PIQ) 0.71 0.79
User Inf. 6 0.91 0.62
Satisfaction 0.73 0.65 0.80
(UIS)

Use 2 0.92 0.85

0.37 0.21 0.38 0.92
Individual 6 0.95 0.78
impact 0.56 0.45 0.58 0.34 0.88
Flexibility 4 0.96 0.85
0.78 0.57 0.57 0.29 0.55 0.92

Integration 4 0.95 0.83

0.79 0.52 0.53 0.38 0.38 0.59 0.92
Response 4 0.93 0.77
time 0.88 0.65 0.61 0.25 0.50 0.66 0.58 0.88
Recover- 4 0.95 0.82
ability 0.90 0.59 0.65 0.30 0.52 0.61 0.70 0.76 0.90
Conven- 4 0.94 0.79
ience 0.86 0.60 0.76 0.42 0.56 0.64 0.52 0.71 0.72 0.89
Command 4 0.91 0.73
language 0.76 0.62 0.51 0.19 0.28 0.35 0.52 0.71 0.65 0.65 0.85
Complete- 4 0.97 0.89
ness 0.48 0.83 0.47 0.15 0.29 0.47 0.33 0.47 0.37 0.37 0.40 0.94
Precision 4 0.98 0.91
0.67 0.86 0.59 0.11 0.38 0.63 0.51 0.58 0.56 0.51 0.56 0.72 0.95

Accuracy 4 0.97 0.89

0.64 0.89 0.56 0.11 0.38 0.48 0.45 0.59 0.56 0.55 0.59 0.62 0.71 0.94
Consistency 4 0.97 0.90
0.67 0.90 0.59 0.18 0.44 0.48 0.52 0.63 0.58 0.57 0.58 0.63 0.69 0.93 0.95
Currency 4 0.98 0.93
0.51 0.85 0.58 0.31 0.41 0.36 0.35 0.47 0.42 0.44 0.44 0.72 0.59 0.71 0.73 0.96
Format 4 0.95 0.83
0.68 0.71 0.54 0.21 0.41 0.55 0.54 0.58 0.57 0.56 0.56 0.51 0.65 0.51 0.51 0.51 0.91
a The upper bold number on the diagonal is the average variance extracted and the lower one its square root.

Table 2. Internal Consistencies, Average Variances Extracted and Inter-Correlations between Constructs,
Part 2 of 2.

The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2) 17
 Hypothesis
H1: Perceived system quality predicts user sa-
tisfaction
H2: Perceived information quality predicts user
satisfaction
H3: Perceived system quality predicts actual use
H4: Perceived information quality predicts actual
use
H5a: User satisfaction predicts actual use
H5b: Actual use predicts user satisfaction
H6: User satisfaction predicts individual impact
H7: Actual use predicts individual impact
Table 3. Summary of the Support Obtained for the Hypotheses
At the aggregated level of the 15 departments, user
satisfaction, use and individual impact had corre-
lations of 0.40, -0.36 and 0.41 with the overall
measure of organizational impact (none of them
significant at the level p ≤ 0.10). Even though not
significant in this small population, the correlations are
still notable, as user satisfaction, use and individual
impact together explained 43.5% of the variance in
organizational impact (p ≤ 0.10). Of the individual
regression coefficients, the negative coefficient of
actual use with organizational impact (ß = -0.38) was
almost significant (p ≤ 0.10), suggesting that use may
be a problematic predictor of organizational impact.
One reason for this unexpected finding may be that
one should not analyse the relationship between
system use and organizational impact cross-
sectionally across departments. It may be that if one
could analyse variations in system use within depart-
ments, use would have a positive association with
organizational impact. This would require more
longitudinal research.
From a more practical viewpoint, the power of
perceived system quality and perceived information
quality as predictors of user satisfaction suggests that
they provide an effective diagnostic framework in
which to analyse system features that may “cause”
user satisfaction and dissatisfaction. The close
association between user satisfaction and individual
impact also suggests that user satisfaction may serve
as a valid surrogate for individual impact.
18 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
Model 1 Model 2
Supported (p ≤ 0.001) Supported (p ≤ 0.001)
Supported (p ≤ 0.05) Supported (p ≤ 0.05)
Supported (p ≤ 0.05) Supported (p ≤ 0.01)
Not supported Not supported
Supported (p ≤ 0.10) N/A
N/A Supported (p ≤ 0.10)
Supported (p ≤ 0.001) Supported (p ≤ 0.001)
Not supported Not supported
Conclusions
The above paper reports on an empirical test of the IS
success model of DeLone and McLean (1992) as a
predictive model. As pointed out in the Introduction,
there is a dearth of empirical tests of the model.
Overall, the present findings supported the model,
but, as implied in the discussion in section 3.2, there
is much ambiguity related to the DeLone –McLean
model as a causal-explanatory model. Much of this
culminates in the ambiguity of the concept of user
(information) satisfaction (e.g. Goodhue, 1986; Iivari,
1987; Melone, 1990; Iivari, 1997). It seems that little
progress has been made on this front since the
1980’s. Strengthening of the underlying theory of the
DeLone-McLean model would require some attention
to be paid to this component. On the positive side, the
findings suggest that user satisfaction may be a
reasonably good surrogate for individual impact as
long as it is confined to impact on work performance.
It is an open question, however, whether it is a valid
surrogate for organizational effectiveness, as
suggested by Ives et al. (1983).
The present paper has its limitations. Since the
results are based on a field study of one mandatory
information system in one specific organizational
context, the first question is whether the findings are
specific to that system and its organizational context.
A second question is whether they may be explained
by the nature of the system (a mandatory operational
level system based on an application package). As
pointed out above, we suspect that the relatively
insignificant role of actual use in the whole framework
may be explained by this mandatory nature of the
system. This may also explain the fact that perceived
system quality emerged as more significant than
perceived information quality. Empirical testing of the
DeLone-McLean model should therefore be extended
to cover a wider variety of systems.
References
Adams, D.A., Nelson, R.R. and Todd, P.A. (1992).
“Perceived Usefulness, Ease of Use, and Usage
of Information Technology: A Replication,” MIS
Quarterly, Vol. 16, No. 2, pp. 227-247.
Amoroso, D.L. and Cheney, P.H. (1991). “Testing a
Causal Model of End-User Application Effecti-
veness,” Journal of Management Information
Systems, Vol. 8, No. 1, pp. 63-89.
Bagozzi, R.P. (1985). “Expectancy-Value Attitude
Models: An Analysis of Critical Theoretical
Issues,” International Journal of Research
Marketing, Vol. 2, pp. 43-60.
Bailey, J.E. and Pearson, S.W. (1983). “Developing
a Tool for Measuring and Analyzing Computer
User Satisfaction,” Management Science, Vol.
29, No. 5, pp. 530-545.
Barki, H. and Hartwick, J. (1989). “Rethinking the
Concept of User Involvement,” MIS Quarterly,
Vol. 13, No. 1, pp. 53-63.
Barki, H. and Huff, S.L. (1985). “Change, Attitude to
Change, and Decision Support System
Success,” Information & Management, Vol. 9,
No. 5, pp. 261-268.
Baroudi, J.J. and Orlikowski, W.J. (1988). “A Short-
Form Measure of User Information Satisfaction:
A Psychometric Evaluation and Notes on Use,”
Journal of Management Information Systems,
Vol. 4, No. 4, pp. 44-59.
Baroudi, J.J., Olson, M.H. and Ives, B. (1986). “An
Empirical Study of the Impact of User
Involvement on System Usage and Information
Satisfaction,” Communications of the ACM, Vol.
29, No. 3, pp. 232-238.
Cats-Baril, W.L. and Huber, G.P. (1987). “Decision
Support Systems for Ill-Structured Problems: An
Empirical Study,” Decision Sciences, Vol. 18,
No. 3, pp. 350-372.
Chau, P.Y.K. (1996). “An Empirical Assessment of a
Modified Technology Acceptance Model,”
Journal of Management Information Systems,
Vol. 13, No. 2, pp. 185-204.
Chin, J.P., Diehl, V.A. and Norman, K.L. (1988).
“Development of an Instrument Measuring User
Satisfaction of the Human-Computer Interface,”
in Soloway, E., Frye, D. and Sheppard, S.B.
(eds.), CHI’88 Conference Proceedings: Human
Factors in Computing Systems, NY: Association
for Computing Machinery, pp. 213-218.
The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
Chin, W.W. (1998). “The Partial Least Squares
Approach to Structural Equation Modelling,” in
Marcoulides, G.A. (ed.), Modern Methods for
Business Research, Mahwah, NJ: Lawrence
Erlbaum Associated, pp. 295-336.
Chin, W.W. and Gopal, A. (1995). “Adoption
Intention in GSS: Relative Importance of
Beliefs,” The Data Base for Advances in
Information Systems, Vol. 26, No. 2&3, pp. 42-
63.
Chin, W.W. and Lee. M.K.O. (2000). “On the
Formation of End-User Computing Satisfaction:
A Proposed Model and Measurement
Instrument,” in Orlikowski, W., Ang, S., Weill, P.,
Krcmar, H.C. and DeGross, J.I. (eds.),
Proceedings of the Twenty-First International
Conference on Information Systems, Brisbane,
Australia, pp. 553-563.
Chin, W.W. and Marcolin, B.L., (2001). “The Future
of Diffusion Research,” The Data Base for
Advances in Information Systems, Vol. 32, No.
3, pp. 8-12.
Chin, W.W. and Newsted, P.R. (1999). “Structural
Equation Modeling Analysis with Small Samples
Using Partial Least Squares,” in Rick Hoyle
(Ed.), Statistical Strategies for Small Sample Re-
search, Thousand Oaks, CA: Sage Publications,
pp. 307-341.
Davis, F.D. (1989). “Perceived Usefulness,
Perceived Ease of Use, and User Acceptance of
Information Technology,” MIS Quarterly, Vol. 13,
No. 3, pp. 319-339.
Davis, F.D., Bagozzi, R.P. and Warshaw, P.R.
(1989). “User Acceptance of Computer
Technology: A Comparison of Two Theoretical
Models,” Management Science, Vol. 35, No. 8,
pp. 982-1003.
Davis, F.D., Bagozzi, R.P. and Warshaw, P.R.
(1992). “Extrinsic and Intrinsic Motivation to Use
Computers in the Workplace,” Journal of Applied
Social Psychology, Vol. 22, pp. 1111-1132.
DeLone, W.H. and McLean, E.R. (1992).
“Information Systems Success: The Quest for
the Dependent Variable,” Information Systems
Research, Vol. 3, No. 1, pp. 60-95.
DeLone, W.H. and McLean, E.R. (2003). “The
DeLone and McLean Model of Information
Systems Success: A ten-Year Update,” Journal
of Management Information Systems, Vol. 19,
No. 4, pp. 9-30.
Doll, W.J. and Torkzadeh, G. (1988). “The
Measurement of End-User Computing
Satisfaction,” MIS Quarterly, Vol 12, No. 2, pp.
259-274.
Etezadi-Amoli, J. and Farhoomand, A.F. (1996). “A
Structural Model of End-User Computing
19
 Satisfaction and User Performance,” Information
& Management, Vol. 30, No. 2, pp. 65-73.
Fishbein, M. and Ajzen, I. (1975). Belief, Attitude,
Intention and Behavior: An Introduction to
Theory and Research, Reading, MA: Addison-
Wesley.
Fornell, C. and Larcker, D.F. (1981). “Evaluating
Structural Equation Models with Unobservable
Variables and Measurement Error,” Journal of
marketing Research, Vol. 18, pp. 39-50.
(reprinted in Fornell, C. (ed.), A Second
Generation of Multivariate Analysis, Volume 2,
Measurement and Evaluation, New York, NY:
Praeger Publishers, pp. 289-316.)
Gatian, A.W. (1994). “Is User Satisfaction a Valid
Measure of System Effectiveness,” Information
& Management, Vol. 26, No. 3, pp. 119-131.
Gefen, D. and Keil, M. (1998). “The Impact of
Developer Responsiveness on Perceptions of
Usefulness and Ease of Use: An Extension of
the Technology Acceptance Model,” The DATA
BASE for Advances in Information Systems, Vol.
29, No. 2, pp. 35-49.
Gefen, D. and Straub, D.W. (1997). “Gender
Differences in the Perception and Use of E-Mail:
An Extension to the Technology Acceptance
Model,” MIS Quarterly, Vol. 21, No. 4, pp. 389-
400.
Gefen, D. and Straub, D.W. (2000). “The Relative
Importance of Perceived Ease of Use in IS
Adoption: A Study of E-Commerce Adoption,”
Journal of the Association for Information
Systems, Vol. 1, Article 8, pp. 1-28.
Gefen, D., Straub, D.W. and Boudreau, M.-C.
(2000). “Structural Equation Modelling and
Regression: Guidelines for Research Practice,”
Communications of the Association for
Information Systems, Vol. 1, Article 7, pp. 1-76.
Ginzberg, M.J. (1981). “Early Diagnosis of
Implementation Failure: Promising Results and
Unanswered Questions,” Management Science,
Vol. 27, No. 4, pp. 459-478.
Goodhue, D. (1986). “IS Attitudes: Toward
Theoretical and Definition Clarity,” in Maggi, L.,
Zmud, R. and Wetherbe, J. (eds.), Procedings of
the Seventh International Conference on
Information Systems, San Diego, CA, pp. 181-
194.
Gustafsson, M.R., Karlsson, T. and Bubenko, J.A.
Jr. (1982). “A Declarative Approach to
Conceptual Information Modeling,” in Olle, T.W.,
Sol, H.G. and Verrijn-Stuart, A.A. (eds.),
Information Systems Design Methodologies: A
Comparative Review, North-Holland,
Amstredanm, pp. 93-142.
20 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
Hair, J.F. Jr., Anderson, R.E., Tatham, R.L. and
Black, W.C. (1992). Multivariate Data Analysis
with Readings, New York, NY: Macmillan.
Igabria, M. (1990). “End-User Computing
Effectiveness: A Structural Equation Model,”
Omega, Vol. 18, No. 6, pp. 637-652.
Igbaria, M., Guimaraes, T. and Davis, G.B. (1995).
“Testing the Determinants of Microcomputer
Usage via a Structural Equation Model,” Journal
of Management Information Systems, Vol. 11,
No. 4, pp. 87-114.
Igbaria, M. and Iivari, J. (1995). “The Effects of Self-
Efficacy on Computer Usage,” Omega, Vol. 23,
No. 6, pp. 587-605.
Igbaria, M., Parasuraman, S. and Baroudi, J.J.
(1996). “A Motivational Model of Microcomputer
Usage,” Journal of Management Information
Systems, Vol. 13, No. 1, pp. 127-143.
Igbaria, M., Zinatelli, N., Cragg, P. and Cavaye,
A.L.M. (1997). “Personal Computing Acceptance
Factors in Small Firms: A Structural Equation
Model,” MIS Quarterly, Vol. 21, No. 3, pp. 279-
305.
Igbaria, M. and Zviran, M. (1991). “End-User
Effectiveness: A Cross-Cultural Examination,”
Omega, Vol. 19, No. 5, pp. 369-379.
Iivari, J. (1987). “User Information Satisfaction (UIS)
Reconsidered: An Information System as the
Antecedent of UIS.” in DeGross, J.I. and Kriebel,
C.H. (eds.), Proceedings of the Eighth Interna-
tional Conference on Information Systems,
Pittsburgh PA, pp. 57-73.
Iivari, J. (1996). “Why Are CASE Tools Not Used?,”
Communications of the ACM, Vol. 39, No. 19,
pp. 94-103.
Iivari, J. (1997). “User Information Satisfaction: A
Critical Review,” Encyclopedia of Library and
Information Science, Vol. 60, Supplement 23,
pp. 341-364.
Iivari, J. and Ervasti, I. (1994). “User Information
Satisfaction: IS Implementability and
Effectiveness,” Information & Management, Vol.
27, No. 4, pp. 205-220.
Iivari, J. and Koskela, E. (1987). “The PIOCO Model
for Information System Design,” MIS Quarterly,
Vol. 11, No. 3, pp. 401-419.
Ives, B., Olson, M.H. and Baroudi, I.J. (1983). “The
Measurement of User Information Satisfaction,”
Communications of the ACM, Vol. 26, No. 10,
pp. 785-793.
Jenkins, A.M. (1985). “Research Methodologies and
MIS Research,” in Mumford E. et al. (eds.),
Research Methods in Information Systems,
North-Holland, Amsterdam, pp. 103-117.
Jenkins, A.M. and Ricketts, J.A. (1979).
Development of an Instrument to Measure User
Information Satisfaction with Management
 Information Systems, Working paper, Bloom-
ington, Indiana: Indiana University.
Karahanna, E. and Straub, D.W. (1999). The
Psychological Origins of Perceived Usefulness
and Ease-of-Use,” Information & Management,
Vol. 35, No. 4, pp. 237-250.
Karahanna, E., Straub, D.W. and Chervany, N.L.
(1999). “Information Technology Adoption
Across Time: A Cross-Sectional Comparison of
Pre-Adoption and Post-Adoption Beliefs,” MIS
Quarterly, Vol. 23, No. 2, pp. 183-213.
Lee, Y., Kozar, K.A. and Larsen K.R.T. (2003) “The
technology acceptance model: past, present,
and future,” Communications of the Association
for Information Systems, Vol. 12, No. 50, pp.
752-780
Leidner, D.E. (1998). “Mexican Executives’ Use of
Information Systems: An Empirical Investigation
of EIS Use and Impact,” Journal of Global
Information Technology Management, Vol. 1,
No. 2, pp. 19-36.
Ma, Q. and Liu, L. (2004). “The Technology
Acceptance Model: A Meta-Analysis of Empirical
Findings,” Journal of Organizational and End-
User Computing, Vol. 16, No. 1, pp. 59-72
Mathieson, K. (1991). “Predicting User Intentions:
Comparing the Technology Acceptance Model
with the Theory of Planned Behavior,”
Information Systems Research, Vol. 2, No. 3,
pp. 173-191.
McGuire, W.J. (1969). “The Nature of Attitudes and
Attitude Change,” in Lindzey, G. and Aronson,
E. (eds.), The Handbook of Social Psychology,
Volume 3: The individual in social context, 2nd
ed., Reading, MA: Addison-Wesley, pp. 136-
314.
Melone, N.P. (1990). “A Theoretical Assessment of
the User-Satisfaction Construct in Information
Systems Research,” Management Science, Vol.
36, No. 1, pp. 76-91.
Mohr, L.B. (1982). Explaining Organizational
Behavior, the Limits and Possibilities of Theory
and Research, San Fransisco, CA: Jossey-Bass
Publishers.
Nelson, R.R. and Cheney, P.H. (1987). “Training
End Users: An Exploratory Study,” MIS
Quarterly, Vol. 11, No. 4, pp. 547-559.
Rai, A., Lang, S.S. and Welker, R.B. (2002).
“Assessing the Validity of IS Success Models:
An Empirical Test and Theoretical Analysis,”
Information Systems Research, Vol. 13, No. 1,
pp. 50-69.
Sanders, G.L. (1984). “MIS/DSS Success Measure,”
Systems, Objectives, Solutions, Vol. 4, pp. 29-
34.
The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
Seddon, P.B. (1997). “A Respecification and
Extension of the DeLone and McLean Model of
IS Success,” Information Systems Research,
Vol. 8, No. 3, pp. 240-253.
Seddon, P.B. and Kiew, M.-Y. (1994). “A Partial Test
and Development of the DeLone and McLean
Model of IS Success,” in DeGross, J.I., Huff,
S.L. and Munro, M.C. (eds.), Proceedings of the
Fifteenth International Conference on
Information Systems, Vancouver, Canada, pp.
99-110.
Seddon, P.B., Staples, S., Patnayakuni, R. and
Bowtell, M. (1999). “Dimensions of Information
Systems Success,” Communications of the
Association of Information Systems, Vol. 2,
Article. 20, pp. 1-39.
Snitkin, S.R. and King, W.R. (1986). “Determinants
of the Effectiveness of Personal Decision
Support Systems,” Information & Management,
Vol. 10, No. 2, pp. 83-89.
Srinivasan, A. (1985). “Alternative Measures of
System Effectiveness: Associations and
Implications,” MIS Quarterly, Vol. 9, No. 3, pp.
243-253.
Straub, D.W. (1989). “Validating Instruments in MIS
Research,” MIS Quarterly, Vol. 13, No. 2, pp.
147-169.
Straub, D.W., Keil, M. and Brenner, W. (1997).
“Testing the Technology Acceptance Model
Across Cultures: A Three Country Study,”
Information & Management, Vol. 33, No. 1, pp.
1-11.
Straub, D., Limayem, M. and Karahanna-Evaristo, E.
(1995). “Measuring System Usage: Implications
for Theory Testing,” Management Science, Vol.
41, No. 8, pp. 1328-1342.
Szajna, B. (1996). Empirical Evaluation of the
Revised Technology Acceptance Model,”
Management Science, Vol. 42, No. 1, pp. 85-92.
Torkzadeh, G. and Dwyer, D.J. (1994). “A Path
Analytical Study of Determinants of Information
System Usage,” Omega, Vol. 22, No. 4, pp. 339-
348.
Taylor, S. and Todd, P.A. (1995). “Understanding
Information Technology Usage: A Test of
Competing Models,” Information Systems
Research, Vol. 6, No. 2, pp. 144-176.
Van de Ven, A.H and Ferry, D.L. (1980). Measuring
and Assessing Organizations, Chichester: John
Wiley & Sons.
Venkatesh, V. and Davis, F.D. (2000). “A Theoretical
Extension of the Technology Acceptance Model:
Four Longitudinal Field Studies,” Management
Science, Vol. 46, No. 2, pp. 186-204.
21
About the Author
Juhani Iivari is a Professor in Information Systems
at the University of Oulu, Finland, and the Scientific
Head of the INFWEST.IT Postgraduate Education
Program of five Finnish Universities in the area of
information systems. He received his M.Sc. and
Ph.D. degrees from the University of Oulu. Iivari
serves in editorial boards of seven journals. His
research has broadly focused on theoretical
22 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
foundations, development methodologies and
approaches, organizational analysis, implementation
and acceptance, and the quality of information
systems. Iivari has published in journals such as
AJIS, BIT, Communications of the ACM, Data Base,
European JIS, Information & Management, Informa-
tion and Software Technology, Information Systems,
ISJ, ISR, JMIS, JOCEC,MISQ, Omega, SJIS and
others. Email: juhani.iivari@oulu.fi
 Appendix A: Measures
System quality Complex __ __ __ __ __ __ __ Simple
Please assess the flexibility of the system to change Weak __ __ __ __ __ __ __ Powerful
in response to new demands
Difficult __ __ __ __ __ __ __ Easy
Rigid __ __ __ __ __ __ __ Flexible
Hard-to-use__ __ __ __ __ __ __ Easy-to-use
Limited __ __ __ __ __ __ __ Versatile
Insufficient__ __ __ __ __ __ __ Sufficient
Information quality
Low __ __ __ __ __ __ __ High
Please assess the volume of output information (re-
ports and queries)
Please assess the ability of the system to Concise __ __ __ __ __ __ __ Excessive
communicate with other information systems
Insufficient__ __ __ __ __ __ __ Sufficient
Incomplete __ __ __ __ __ __ __ Complete
Unnecessary__ __ __ __ __ __ __ Necessary
Insufficient__ __ __ __ __ __ __ Sufficient
Unreasonable__ __ __ __ __ __ __ Reasonable
Unsuccessful__ __ __ __ __ __ __ Successful
Please assess the completeness of the output
Bad __ __ __ __ __ __ __ Good information
Incomplete__ __ __ __ __ __ __ Complete
Please assess the response and turnaround time of Inconsistent__ __ __ __ __ __ __ Consistent
the system
Insufficient__ __ __ __ __ __ __ Sufficient
Slow __ __ __ __ __ __ __ Fast
Inadequate__ __ __ __ __ __ __ Adequate
Bad __ __ __ __ __ __ __ Good
Inconsistent__ __ __ __ __ __ __ Consistent
Please assess the precision of the output information
Unreasonable__ __ __ __ __ __ __ Reasonable
Insufficient __ __ __ __ __ __ __ Sufficient
Inconsistent__ __ __ __ __ __ __ Consistent
Please assess the ability of the system to recover
Low __ __ __ __ __ __ __ High
from errors
Uncertain__ __ __ __ __ __ __ Certain
Slow __ __ __ __ __ __ __ Fast
Inferior __ __ __ __ __ __ __ Superior
Please assess the accuracy of the output information
Incomplete__ __ __ __ __ __ __ Complete
Inaccurate __ __ __ __ __ __ __ Accurate
Complex __ __ __ __ __ __ __
Simple Low __ __ __ __ __ __ __ High
Inconsistent__ __ __ __ __ __ __ Consistent
Please assess the convenience of use of the system Insufficient __ __ __ __ __ __ __ Sufficient
Inconvenient __ __ __ __ __ __ __ Convenient
Bad __ __ __ __ __ __ __ Good Please assess the consistency of the output
information
Difficult __ __ __ __ __ __ __ Easy
Inconsistent__ __ __ __ __ __ __ Consistent
Inefficient__ __ __ __ __ __ __ Efficient
Low __ __ __ __ __ __ __ High
Inferior __ __ __ __ __ __ __ Superior
Please assess the commands used to interact with
the system Insufficient __ __ __ __ __ __ __ Sufficient

The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2) 23
Please assess the currency of the output information Frequency of use: How often on average do you use
the system?
Bad __ __ __ __ __ __ __ Good
Less than once a month 1
Untimely__ __ __ __ __ __ __ Timely
Once a month 2
Inadequate__ __ __ __ __ __ __ Adequate
A few times a month 3
Unreasonable __ __ __ __ __ __ __ Reasonable
A few times a week 4
Once a day 5
Please assess the format of the output
Several times a day 6
Bad __ __ __ __ __ __ __ Good
Complex__ __ __ __ __ __ __ Simple
Individual impact
Readable__ __ __ __ __ __ __ Readable
Useless __ __ __ __ __ __ __ Useful Using the system in my job enables me to accomplish
tasks more quickly.

Fully disagree__ __ __ __ __ __ __ Fully agree

User satisfaction
Please assess the system Using the system improves my job performance.

Terrible __ __ __ __ __ __ __ Wonderful Fully disagree__ __ __ __ __ __ __ Fully agree

Difficult __ __ __ __ __ __ __ Easy Using the system in my job increases my productivity.
Frustrating __ __ __ __ __ __ __ Satisfying
Fully disagree__ __ __ __ __ __ __ Fully agree
Inadequate __ __ __ __ __ __ __ Adequate
Using the system enhances my effectiveness in my
Dull __ __ __ __ __ __ __ Stimulating job
Rigid __ __ __ __ __ __ __ Flexible
Fully disagree__ __ __ __ __ __ __ Fully agree

Using the system makes it easier to do my job

Actual use
Fully disagree__ __ __ __ __ __ __ Fully agree
Daily use: How much time do you spend with the
system during an ordinary day when you use comput- I find the system useful in my job
ers?
Fully disagree__ __ __ __ __ __ __ Fully agree
Scarcely at all 1
Less than 1/2 hours 2
1/2- 1 hours 3
1-2 hours 4
2-3 hours 5
More than 3 hours 6

24 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
 Appendix B: Weights and loadings on the indicators

Indicator Model 1 Model 2

Weight Loading Weight Loading
Flexibility
- item 1 0.2651 0.8913 0.2651 0.8913
- item 2 0.2468 0.9108 0.2468 0.9108
- item 3 0.2839 0.9473 0.2839 0.9473
- item 4 0.3042 0.9276 0.3042 0.9276
System integration
- item 1 0.2666 0.9208 0.2666 0.9208
- item 2 0.2730 0.8956 0.2730 0.8956
- item 3 0.2778 0.8851 0.2778 0.8851
- item 4 0.2924 0.9356 0.2924 0.9356
Response time
- item 1 0.2605 0.8308 0.2605 0.8308
- item 2 0.2716 0.8588 0.2716 0.8588
- item 3 0.3134 0.8887 0.3134 0.8887
- item 4 0.3118 0.9194 0.3118 0.9194
Recoverability
- item 1 0.2761 0.8757 0.2761 0.8757
- item 2 0.2800 0.9366 0.2800 0.9366
- item 3 0.2809 0.9236 0.2809 0.9236
- item 4 0.3858 0.8815 0.3858 0.8815
Convenience
- item 1 0.2992 0.9352 0.2992 0.9352
- item 2 0.3173 0.9489 0.3173 0.9489
- item 3 0.2782 0.9012 0.2782 0.9012
- item 4 0.2306 0.7590 0.2306 0.7590
Command language
- item 1 0.1303 0.6100 0.1303 0.6100
- item 2 0.3190 0.9027 0.3190 0.9027
- item 3 0.3612 0.9509 0.3612 0.9509
- item 4 0.3478 0.9123 0.3478 0.9123
Completeness
- item 1 0.2556 0.9408 0.2556 0.9408
- item 2 0.2652 0.9310 0.2652 0.9310
- item 3 0.2749 0.9589 0.2749 0.9589
- item 4 0.2671 0.9436 0.2671 0.9436
Precision
- item 1 0.2465 0.9385 0.2465 0.9385
- item 2 0.2687 0.9623 0.2687 0.9623
- item 3 0.2689 0.9596 0.2689 0.9596
- item 4 0.2716 0.9522 0.2716 0.9522
Accuracy
- item 1 0.2662 0.9596 0.2662 0.9596
- item 2 0.2595 0.9533 0.2595 0.9533

The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2) 25
 - item 3 0.2729 0.9300 0.2729 0.9300
- item 4 0.2710 0.9231 0.2710 0.9231
Consistency
- item 1 0.2601 0.9277 0.2601 0.9277
- item 2 0.2621 0.9630 0.2621 0.9630
- item 3 0.2620 0.9505 0.2620 0.9505
- item 4 0.2710 0.9585 0.2710 0.9585
Currency
- item 1 0.2606 0.9590 0.2606 0.9590
- item 2 0.2567 0.9675 0.2567 0.9675
- item 3 0.2549 0.9567 0.2549 0.9567
- item 4 0.2692 0.9644 0.2692 0.9644
Format
- item 1 0.2726 0.9236 0.2726 0.9236
- item 2 0.2838 0.9361 0.2838 0.9361
- item 3 0.3020 0.9648 0.3020 0.9648
- item 4 0.2510 0.8004 0.2510 0.8004
Perceived system quality
- item 1 0.0586 0.6701 0.0587 0.6702
- item 2 0.0487 0.6238 0.0487 0.6239
- item 3 0.0551 0.7177 0.0552 0.7177
- item 4 0.0608 0.7688 0.0608 0.7689
- item 5 0.0530 0.6915 0.0530 0.6916
- item 6 0.0564 0.7082 0.0565 0.7083
- item 7 0.0586 0.7206 0.0586 0.7207
- item 8 0.0619 0.7586 0.0619 0.7586
- item 9 0.0485 0.6846 0.0485 0.6846
- item 10 0.0536 0.7138 0.0537 0.7138
- item 11 0.0660 0.8238 0.0660 0.8237
- item 12 0.0668 0.8196 0.0668 0.8195
- item 13 0.0590 0.7930 0.0589 0.7930
- item 14 0.0635 0.8042 0.0635 0.8042
- item 15 0.0642 0.8066 0.0642 0.8067
- item 16 0.0698 0.8208 0.0697 0.8207
- item 17 0.0699 0.8041 0.0698 0.8040
- item 18 0.0720 0.8525 0.0719 0.8524
- item 19 0.0672 0.7475 0.0671 0.7475
- item 20 0.0559 0.6197 0.0560 0.6197
- item 21 0.0220 0.2841 0.0220 0.2841
- item 22 0.0481 0.6957 0.0480 0.6956
- item 23 0.0604 0.7877 0.0603 0.7876
- item 24 0.0595 0.7584 0.0595 0.7583
Perceived information quality
- item 1 0.0502 0.7418 0.0502 0.7419
- item 2 0.0489 0.7697 0.0490 0.7698
- item 3 0.0502 0.7977 0.0502 0.7978
- item 4 0.0513 0.7752 0.0514 0.7753

26 The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2)
 - item 5 0.0502 0.7548 0.0502 0.7548
- item 6 0.0563 0.8226 0.0563 0.8226
- item 7 0.0561 0.8234 0.0561 0.8233
- item 8 0.0558 0.8317 0.0558 0.8316
- item 9 0.0543 0.8303 0.0542 0.8302
- item 10 0.0527 0.8096 0.0527 0.8095
- item 11 0.0584 0.8514 0.0584 0.8513
- item 12 0.0540 0.8453 0.0540 0.8453
- item 13 0.0559 0.8346 0.0559 0.8345
- item 14 0.0577 0.8408 0.0577 0.8407
- item 15 0.0578 0.8406 0.0578 0.8405
- item 16 0.0581 0.8694 0.0581 0.8693
- item 17 0.0565 0.8142 0.0566 0.8143
- item 18 0.0560 0.8021 0.0560 0.8022
- item 19 0.0560 0.7964 0.0561 0.7965
- item 20 0.0584 0.8410 0.0584 0.8410
- item 21 0.0450 0.6233 0.0450 0.6232
- item 22 0.0467 0.6488 0.0467 0.6488
- item 23 0.0491 0.6905 0.0491 0.6905
- item 24 0.0408 0.5739 0.0408 0.5739
User information satisfaction
- item 1 0.2162 0.7257 0.2099 0.7219
- item 2 0.1982 0.7409 0.1937 0.7384
- item 3 0.2346 0.8521 0.2386 0.8542
- item 4 0.2226 0.8100 0.2294 0.8131
- item 5 0.2075 0.8287 0.2084 0.8296
- item 6 0.1943 0.7720 0.1921 0.7720
Use
- item 1 0.5897 0.9349 0.6080 0.9400
- item 2 0.4951 0.9063 0.4761 0.9001
Individual impact
- item 1 0.1362 0.8517 0.1358 0.8515
- item 2 0.1814 0.9088 0.1814 0.9088
- item 3 0.1968 0.8976 0.1967 0.8975
- item 4 0.2161 0.9213 0.2162 0.9213
- item 5 0.2098 0.9169 0.2100 0.9170
- item 6 0.1901 0.7948 0.1903 0.7949

The DATA BASE for Advances in Information Systems - Spring 2005 (Vol. 36, No. 2) 27

View publication stats