See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/318576818

National e-government performance and citizen satisfaction: a multilevel

analysis across European countries

Article  in  International Review of Administrative Sciences · July 2017

DOI: 10.1177/0020852317703691

CITATIONS READS

39 1,680

2 authors, including:

Liang Ma
Renmin University of China
67 PUBLICATIONS   989 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Mapping the evolution of the central government apparatus View project

Policy Change and Policy Entrepreneurship in Asia View project

All content following this page was uploaded by Liang Ma on 12 December 2017.

The user has requested enhancement of the downloaded file.

 International
Review of
Administrative
Article Sciences
International Review of
Administrative Sciences
0(0) 1–21
National e-government ! The Author(s) 2017
performance and citizen Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
satisfaction: a multilevel DOI: 10.1177/0020852317703691
journals.sagepub.com/home/ras
analysis across European countries
Liang Ma
Renmin University of China, China

Yueping Zheng
Sun Yat-sen University, China

Abstract
Are citizens more satisfied with e-government ranked higher in league tables? In this
article, we empirically examine the relationship between objective e-government per-
formance on the supply side and the perceptions of citizens on the demand side.
A multilevel analysis of over 28,000 respondents across 32 European countries reveals
that highly ranked e-government is warmly welcomed by citizens, suggesting that
the supply and demand sides of e-government are, in part, consistent. Specifically, the
e-government performance–satisfaction correlations in e-service and e-participation
are more prominent than that of e-information. The results also show that citi-
zens’ perceived e-government benefits are mainly from using online services. While
e-government rankings are reasonably predictive of citizen satisfaction, they should
be referred to with caveats in e-government policies.

Points for practitioners

The empirical findings reveal that objective e-government performance is partially con-
gruent with citizens’ satisfaction and perceived benefits. While e-government rankings
may not be good predictors of citizen use, they do coincide, in part, with citizen
satisfaction. Ubiquitous e-government benchmarks can be referred to as reliable
gauges of citizen satisfaction, though their susceptibility varies across the purposes of
e-government use. The various benefits that citizens perceived from e-government are
primarily derived through online services instead of electronic information or partici-
pation, and the government should pay more attention to e-service development in
order to bring more benefits to its users.

Corresponding author:
Yueping Zheng, Sun Yat-Sen University No. 135, Xingang Xi Road, Guangzhou, 510275, P. R. China
Guangzhou, 510275, China.
Email: zheng_yueping@126.com
2 International Review of Administrative Sciences 0(0)

Keywords
citizen satisfaction, cross-country analysis, e-government, Europe, multilevel model,
public service performance, supply–demand gap

Introduction
The rapid development and extensive penetration of information and communica-
tion technologies (ICTs) help governments around the world to innovate and
improve public services through various e-government features (West, 2005).
In comparison with traditional channels of service delivery and interaction,
e-government is much more convenient, useful, and beneficial, and citizens, busi-
nesses, and other clients of e-government have benefited tremendously from these
online services (Reddick and Turner, 2012). It is thus understandable that several
studies find a strong relationship between e-government use and satisfaction
(Tolbert and Mossberger, 2006; Welch et al., 2005).
Some countries outperform others in harnessing ICTs, and the last two decades
have witnessed the burgeoning of ratings and rankings of e-government maturity,
development, and performance, for example, the Global E-Government Report
from Brown University (West, 2005), the Waseda International E-Government
Rankings Survey (Obi, 2008), Rutgers University’s Digital Governance in
Municipalities Worldwide (Holzer and Manoharan, 2016), and the United Nations
E-Government Survey (UN, 2012). By benchmarking e-government best practices
and highlighting performance gaps, these rankings inform governments as to how
to improve online appearances and services.
The e-government rankings ubiquitously used among researchers and practi-
tioners, however, are predominantly derived from the supply side, for example,
the delivery and quality of e-government features (Seri et al., 2014). These rankings
often draw on objective indicators or experts’ judgments to gauge e-government
performance, for example, whether specific features are equipped and how many
steps or clicks are needed to process a request (Rorissa et al., 2011). The under-
standing of e-government from the demand side, in contrast, is largely limited.
Specifically, the users’ perceived quality, subjective judgment, and satisfaction
have not yet been systematically incorporated into these rankings (Barnes and
Vidgen, 2006), though their importance has been increasingly acknowledged in
the literature (Bertot et al., 2008).
The gap between the supply/objective and demand/perceptual sides of
e-government has not been fully examined (Gauld et al., 2010). Government
websites score well in objective indicators from the supply side; however, this
may not hold true in terms of perceptional assessments from the users’ perspective
(Park et al., 2013), which are determined by various factors. Could a country’s
excellent performance in e-government rankings result in higher citizen satisfac-
tion? If the answer is ‘yes’, through what mechanisms? If the answer is ‘no’,
then why?
Ma and Zheng 3

These questions deserve critical attention since the ubiquitous use of e-government
rankings may bias public managers in navigating public investments and ICT man-
agement. Given their increasing importance and amplification, these e-government
rankings should be critically scrutinized for validity and usability (Bannister, 2007).
Citizens are end users of e-government. If these rankings cannot predict citizen sat-
isfaction well, which is among the core purposes of e-government development
(Verdegem and Verleye, 2009), then their theoretical foundations and methodological
specifications should be overhauled. If they are significantly consistent with citizen
satisfaction, which provides empirical support for their social legitimacy, then more
studies are needed to identify the mechanisms through which the two sides of
e-government are linked.
In this article, we aim to empirically examine the effect of e-government
performance on citizen satisfaction by using multilevel data across 32 European
countries. The results partially support our hypotheses, suggesting objective
e-government performance largely resonates with users’ perceptions. To the best
of our knowledge, this study is among the first to link country-level e-government
performance with individual-level satisfaction by employing multilevel modeling
across multiple countries.
In the remainder of this article, we first review the literature and propose the
theoretical hypotheses to be tested in this study. We then introduce the data collec-
tion and methods used, followed by the presentation of empirical results. We finally
discuss the theoretical and policy implications of our findings for e-government
development, and conclude with limitations and future research directions.

Contexts and hypotheses

The supply and demand of e-government
Over the past two decades, e-government has made tremendous progress and been
adopted all over the world (UN, 2012; West, 2005). Government and equivalent
agencies at various levels supply e-government features, such as information,
services, and interaction opportunities, which are intended to meet citizens’ increas-
ingly diversified and heightened service demands. As one of the key customers
and stakeholders of e-government, citizen satisfaction is critical to e-government
success and sustainability (Osman et al., 2014). Promoting citizen-centric
e-government services is among the top priorities of e-government development
(Bertot et al., 2008).
The existing studies on e-government satisfaction are mainly individual-level
analyses of self-reported, perceptual, and common-source data prone to model
estimation biases. A review of 114 studies on e-government satisfaction reveals
that the existing research is limited by small sample sizes, a single-country focus,
and common method bias (Zahir et al., 2012). Alawneh et al. (2013) summarized
the determinants of e-government satisfaction, and most are perceptions at the
individual level. Few studies employ multilevel models to explain micro-level
4 International Review of Administrative Sciences 0(0)

citizen perceptions by macro-level e-government characteristics. The literature

commonly focuses on individual-level variables, without equivalent attention
paid to government-level variables.
The existing studies repeatedly reveal that perceived e-government quality is
positively related to citizen satisfaction and trust in government, but these percep-
tions may be driven by objective e-government characteristics. A recent study has
shown that highly ranked central agencies in South Korea are not more frequently
accessed by citizens, suggesting that e-government rankings from the supply
side may generate misleading information (Park et al., 2013). Cross-country
e-government rankings have also been scrutinized by recent studies, and mixed
and even inconclusive results suggest that government website evaluations are sus-
ceptible to biases, if not problematic (Bannister, 2007; Barnes and Vidgen, 2006;
Rorissa et al., 2011; Seri et al., 2014). Rankings and ratings play important roles in
information provision, comparison, benchmarking, and knowledge transfer, and it
is unrealistic and impossible to abolish all of them just because of low levels of reli-
ability and validity. An alternative way to use e-government rankings is to fine-tune
the extent to which they are relevant, reliable, and valid in gauging e-government
performance, which can be used with caveats to inform e-government investments,
policies, and development.

Theories of citizen satisfaction with e-government

E-government is among various public services delivered by the government, and
their measurement is controversial and inconclusive. In the literature on citizen
surveys and performance measurement, there are lingering debates about the con-
gruence between perceptual and archival performance indicators (Schachter, 2010).
While some studies find strong linkages between objective and subjective per-
formance indicators (Kelly and Swindell, 2002), others fail (Stipak, 1979). Due
to various reasons, such as misinformation, misunderstanding, stereotypes, and
preconceptions, citizens often under- or overestimate public service performance
(Miller and Miller, 1991). In measuring governance across countries, it is often
found that experts’ judgments and national statistics are inconsistent with pub-
lic opinion (Norris, 2011). Also, it is a fact that a country’s performance
in e-government varies a lot in different rankings as they adopt different
methodologies. It is thus interesting and meaningful to examine the nexus between
e-government rankings and citizen perceptions, which is theoretically linked but
empirically untested.
To explore the mechanisms linking e-government performance and citizen sat-
isfaction, we draw on the customer satisfaction model (CSM), the ‘‘management
matters’’ theory, and the technology acceptance model (TAM). Citizen satisfaction
with public services is primarily shaped by the (dis)confirmation of prior expect-
ations by actual experiences (Van Ryzin, 2006). Given the same level of expect-
ation, citizens are more content with higher-quality services, and vice versa. Service
quality is usually gauged by specific attributes, including benevolence, efficiency,
Ma and Zheng 5

responsiveness, customization, and so forth. These attributes all contribute to

perceived performance, which is linked with citizen satisfaction. Ceteris paribus,
citizens who perceive a higher level of performance are more satisfied with govern-
ment agencies (Morgeson and Petrescu, 2011).
The ‘‘management matters’’ theory argues that the quality of public manage-
ment is positively related to public service performance (Boyne, 2004). Public man-
agement is primarily gauged by managerial quality, processes, capacities, and
leadership, which are considered to be the key drivers of performance improvement
(Moynihan and Pandey, 2005). E-government performance reflects organizational
resources and capacities in maintaining and innovating online service features,
which can be regarded as good proxies of management quality. In this regard,
e-government performance is expected to be positively associated with government
performance measured by citizen satisfaction.
The TAM is one of the most salient models used to explain users’ intention to
accept and use new technologies, particularly ICTs. This model emphasizes the
important role of users’ perceptions, arguing that citizens’ perceived performance
of e-government is positively related to their use intention and actual use, which
is linked with continuous use, satisfaction, and trust (Lee et al., 2003). Users’
satisfaction is implicit albeit presumed in the TAM, and the theory uncovers
the mechanism between citizens’ perceived performance of and satisfaction with
e-government (Lee et al., 2003).

Citizen satisfaction with e-government

E-government rankings are professionally developed by experts, and their compo-
nents and indicators, in theory, reflect the core attributes of e-government.
Government websites ranked highly in these league tables are well-designed,
easily navigated, user-friendly, and comprehensive in coverage, and these features
should generate quality online services to please their target users. Despite being
from different sides, objective indicators of these rankings and subjective percep-
tions of citizens could generate consistent patterns.
ICT has opened up many possibilities for improving internal managerial
efficiency and the quality of public services delivery to citizens (Moon, 2002).
Supported by advanced ICTs, e-government has various advantages that
traditional channels lack. It is found that citizens with experience of the use of
government websites are more satisfied with e-government (Welch et al., 2005).
Better services with convenient delivery and lower costs meet citizens’ expectations
well and lead to higher satisfaction. Evidence from the US reveals that the use of
government websites, particularly those of federal agencies (instead of local and
state ones), is positively related to citizen evaluation of government performance
(Tolbert and Mossberger, 2006).
In addition, e-government performance could indirectly contribute to the rise of
citizen satisfaction through government trust. Welch et al. (2005) found that the
relationships among e-government, satisfaction, and trust are positively related to
6 International Review of Administrative Sciences 0(0)

each other. Well-designed e-government brings citizens the impression of transpar-

ency, responsiveness, and efficiency, generating a higher level of trust and satisfac-
tion. Thus, e-government enables citizens to better understand and scrutinize
government operations, making government more responsible and responsive, as
indicated from principal–agent theories:

Hypothesis 1: E-government performance is positively related to citizens’ satisfaction

with e-government features.

Although researchers have different understandings of e-government, and various

models, like the four-stage model (Layne and Lee, 2001) and the two-stage model
(Reddick, 2004), have been proposed, it is widely accepted that e-government can
primarily be used for three purposes: transparency, transaction, and interactivity
(Welch et al., 2005). We predict that performance in e-information, e-services, and
e-participation is positively related to citizens’ satisfaction with the corresponding
e-government features, but the strength of the linkages may vary.
Government websites have increasingly become the primary source of gov-
ernment information, which largely strengthens government openness and trans-
parency, particularly computer-mediated transparency (Meijer, 2009). Citizens
access government websites mainly for the purpose of searching for information,
and e-information is among the key features of e-government development in its
nascent stage. As Reddick (2004) discussed, the first stage of e-government focuses
on online presence, presenting information about the government and its activities
on the web. Citizens are likely to be more satisfied with the government web portals
releasing more essential information and being easy to navigate.
There is no question that fully functional e-government will make service deliv-
ery more efficient and increase savings for both the government and the
citizen (Layne and Lee, 2001). The second stage emphasizes services delivery.
Government websites are widely equipped with various online services and trans-
action channels enabling citizens to download and submit forms, pay taxes, file
and update identification, and process various administrative approvals. These
one-stop services are much more convenient, time-saving, and user-friendly.
We expect government web portals that are highly ranked in e-services to more
likely satisfy citizens.
E-government involves electronic relationships between the government and
different levels of constituents, an important one being that between the govern-
ment and individuals in a political process (Reddick, 2004) . These websites and
other online facilities have also increasingly been utilized for political and demo-
cratic purposes, such as dialogue, voting, deliberation, consultation, and petitions
(Zheng, 2015). These features facilitate instant two-way interactions between the
government and its constituency, enabling citizens to better participate and play
their roles in government decision-making processes. Thus, government web por-
tals with advanced e-participation functions are expected to be highly praised by
citizens.
Ma and Zheng 7

The mediating effect of perceived e-government benefits

With digital tools, e-government benefits citizens more than traditional means of
contacting government agencies. Citizens using e-government get various benefits,
which lays the foundations of citizen satisfaction. Through e-government, citizens
achieve higher-quality, more transparent, convenient, and simplified public ser-
vices. However, different users may have different perceptions of the benefits
achieved by the same e-government as they have their own expectations and under-
standing. Thus, we hypothesize that perceived benefits play a mediating effect on
the relationship between e-government performance and citizen satisfaction.
User satisfaction with e-government is one of the key approaches to citizen-
centric e-government development (Verdegem and Verleye, 2009). Customers
frame their assessments of online services by the benefits they perceive from
using these services (Udo et al., 2010). Citizens will be satisfied with e-government
only when they perceive benefits from using the online features, which is quite
different due to their own characteristics and backgrounds. In prior studies, per-
ceived ease of use and usefulness are found to be among the key drivers of user
acceptance of e-government services (Hung et al., 2006). Citizens adopting and
accepting e-government are found to be more satisfied with government websites
(Chan et al., 2010). Citizens perceiving quality information and services provided
by government websites are found to be generally satisfied with e-government
(Morgeson and Petrescu, 2011).
Highly ranked e-government services are more likely to benefit citizens through
the convenient, efficient, and useful process of service delivery. Well-designed and
efficiently operated government websites are more appreciated by citizens due to
various benefits perceived through the process of online service delivery and inter-
action. Citizens gaining the aforementioned benefits are more likely to be satisfied
with e-government:

Hypothesis 2: Perceived e-government benefits mediate the relationship between

e-government performance and citizen satisfaction.

Data and methods

Context and data sources
We test the hypotheses in the context of Europe, which permits us to tap into the large
variations in e-government development and satisfaction across its member countries.
The data on e-government performance across the world are well documented by the
proliferation of cross-country benchmarks and rankings, but representative surveys of
e-government use and satisfaction are disproportionally scarce. Fortunately, the data
on e-government development and satisfaction across European countries permit us
to empirically test the aforementioned hypotheses in this study.
8 International Review of Administrative Sciences 0(0)

The data on e-government satisfaction are from the 2012 EU eGovernment survey
(EU, 2013). Over 28,000 citizens across 32 European countries were randomly
sampled, and the sample is proportionally representative of national populations.
We use the United Nations (UN, 2012) e-government survey to gauge national
e-government performance. As a robustness check, we also include the EU’s
e-government rankings and other indicators, and the results are substantially similar.

Dependent and mediating variables

We measure citizen satisfaction with e-government by multiple items from the 2012
EU eGovernment survey (EU, 2013). The respondents were asked about their
satisfaction with Internet applications for a list of purposes over a year, and
10-point Likert scale items were used, ranging from 0 (strongly dissatisfied) to
10 (strongly satisfied). The respondents were asked to what extent they were satis-
fied ‘‘to consult the national government portal,’’ which is used to gauge citizens’
overall satisfaction with national e-government.
Citizen satisfaction with e-information is measured by the respondents’ satisfac-
tion with obtaining information and consulting policy documents or decisions from
government websites. The two items are highly correlated and we average them to
measure e-information satisfaction (Cronbach’s F ¼ 0.783).
E-service satisfaction is gauged by their satisfaction with two features: downloading
official forms and sending (uploading) completed web forms. The two items are highly
correlated and an index is built up by their mean (Cronbach’s F ¼ 0.896).
E-participation satisfaction is operationalized by their satisfaction with three
online functions, including online consultation (to participate in online consult-
ations on policy issues organized by governments), online interaction (to partici-
pate in interactive discussions about policy issues), and online collaboration (to
participate in collaborative platforms). The three items are highly correlated and
we average them to form an index (Cronbach’s F ¼ 0.897).
The respondents were asked about their agreement with eight e-government
benefits, including saving time, saving money, flexibility, quality of the service,
simplified process, better control over service delivery, more transparent service
delivery, and increased trust in public administration. The responses range from
0 (strongly disagree) to 4 (strongly agree). These items are highly correlated, and we
average them to create an index (Cronbach’s F ¼ 0.901).

Independent variables
Despite e-information being among the core of e-government, relevant surveys
and assessments are scarcely used in e-government studies. Williams’s (2015) infor-
mation transparency index (ITI) is constructed by drawing data on government
information disclosure and openness from multiple sources, which is tested to be
reliable and valid in gauging government transparency. We use the ITI for 2010,
the latest available data.
Ma and Zheng 9

National performance in e-service and e-participation is measured by two

sub-components of the UN e-government survey (UN, 2012). We use the
Online Service Index (OSI) and E-Participation Index (EPI) to gauge e-service
and e-participation, respectively. The two measures are lagged by one year to the
dependent variables.

Control variables
Many individual-level factors may affect citizen satisfaction with e-government,
and we control for them in model estimates to mitigate omitted-variable bias.
The control variables include gender, age, formal education, occupation, and
e-government use. While economic income and political attitudes may also affect
e-government satisfaction, they are unfortunately unavailable across countries.
Gender is measured as a dummy variable, with males coded as 1 and females 0. We
measure age by an ordinal variable of five categories ranging from 0 (below 25) to 4
(above 55). We classify formal education into three categories: (1) primary or lower
secondary school, or no formal education; (2) upper secondary school; and (3) higher
education. We consider the effect of unemployment, which is measured by a dummy.
We control for the frequency of e-government use by one item: ‘‘How often,
during the past 12 months, did you use the Internet to consult the national gov-
ernment portal?’’ The responses range from 0 (not once) to 4 (every day), and only
those using at least once (>0) are included in the analysis. The same categories of
items are averaged to form the indexes of e-information use (Cronbach’s F ¼ 0.568),
e-service use (Cronbach’s F ¼ 0.859), and e-participation use (Cronbach’s
F ¼ 0.897).

Model specifications
The citizens sampled in the survey are nested in the 32 European countries, creating a
multilevel data structure that is suitable to be analyzed by a multilevel model (Aguinis
et al., 2013). Despite its advantages in simultaneously estimating context- and indivi-
dual-level effects, the multilevel model has scarcely been used in e-government studies.
We use a random intercept and fixed slope model (RIFSM) since our primary
focus is the effect of national-level e-government performance. Citizen satisfaction
and perceptions refer to Level 1 (L1) variables, while national e-government per-
formance refers to Level 2 (L2) variables. We center the L1 independent variables
by their mean within each country and center the L2 variables by their grand mean.

Results
Descriptive statistics
The results show that the respondents were moderately satisfied with national
e-government features (see Table 1). The overall and specific satisfaction scores
10 International Review of Administrative Sciences 0(0)

Table 1. The descriptive statistics of key variables.

Variable N Mean SD Min. Max.

Overall e-govt satisfaction 14,158 6.083 2.202 0 10

E-information satisfaction 22,115 6.203 2.148 0 10
E-service satisfaction 18,675 6.177 2.202 0 10
E-participation satisfaction 11,168 5.622 2.423 0 10
Perceived e-govt benefits 17,180 2.676 0.803 0 4
E-information 32 77.250 3.818 68 83
E-service 32 0.679 0.144 0.464 0.974
E-participation 32 0.389 0.266 0.026 1
Gender (male ¼ 1) 28,178 0.500 0.500 0 1
Age 28,178 1.853 1.352 0 4
Education 28,178 2.266 0.701 1 3
Jobless 28,178 0.083 0.276 0 1
E-govt use 24,429 0.906 0.991 0 4
E-information use 24,429 1.280 0.896 0 4
E-service use 24,429 1.055 0.931 0 4
E-participation use 24,429 0.560 0.875 0 4

are all higher than 6.0 except for e-participation (5.622). The respondents also
generally agreed that e-government use brings them various benefits.
The sampled countries vary substantially in terms of e-government performance
and citizen satisfaction, and the two variables are positively associated (see
Figures 1–3). In line with our predictions, the correlation between e-government
performance and citizen satisfaction is positive in all three domains (see Table 2).

Multilevel model estimates

We report the multilevel model estimates in Table 3. We first estimate the null
models without any independent variables (not shown to save space). The intra-
class correlation (ICC) is L2 variance as the share of L2 and L1 variances,
which suggests that the total variance in L1 e-government satisfaction and bene-
fits being explained by L2 variance ranges from 1.95% to 3.52%. It means
that 1.95–3.52% of the variation in e-government satisfaction and benefits can
be attributed to national-level variables. Given the significant explanatory power
of the variances at the national level (see log likelihood test), it is appropriate to use
multilevel models.
The coefficients and standard errors reported in multilevel models can be
interpreted in similar ways to ordinary linear regressions. The results reveal that
all three e-government performance indicators are positively related to overall
Ma and Zheng 11

Cyprus
7

United Kingdom
Lithuania Estonia

France
6.5
E-government satisfaction

Switzerland Belgium
Latvia
Luxembourg
Finland
Turkey
NetherlandsMalta
Denmark
Romania
Hungary
Italy Austria
Norway Czech Republic
6

Slovenia
Portugal Poland Germany
Slovakia Ireland
Bulgaria Sweden

Croatia Spain
5.5

Iceland
Greece
5

65 70 75 80 85
ITI

Figure 1. The relationship between e-information and citizen satisfaction.

Cyprus
7

Lithuania Estonia United Kingdom

France
6.5
E-government satisfaction

Belgium
Latvia Switzerland
Luxembourg
Finland
Turkey Malta Denmark Netherlands
Romania Hungary
Austria
Czech Republic Italy Norway
6

Slovenia

Poland Portugal Germany

SlovakiaIreland
Bulgaria Sweden

Croatia Spain
5.5

Iceland
Greece
5

.4 .6 .8 1
OSI

Figure 2. The relationship between e-service and citizen satisfaction.

12 International Review of Administrative Sciences 0(0)

Cyprus
7

Lithuania EstoniaUnited Kingdom

France
6.5
E-government satisfaction

Belgium
Latvia Switzerland
Luxembourg
Finland
Turkey Malta Denmark Netherlands
Romania Hungary
Italy Austria
Czech Republic Norway
6

Slovenia

Poland Portugal Germany

Slovakia
Ireland
Bulgaria Sweden

Croatia Spain
5.5

Iceland
Greece
5

0 .2 .4 .6 .8 1
EPI

Figure 3. The relationship between e-participation and citizen satisfaction.

Table 2. The correlation matrix of key variables at the national level.

Variable 1 2 3 4 5 6 7 8

1 Overall e-govt satisfaction 1 0.839 0.800 0.777 0.483 0.225 0.338 0.200
2 E-information satisfaction 0.847 1 0.908 0.794 0.425 0.346 0.330 0.226
3 E-service satisfaction 0.809 0.912 1 0.714 0.412 0.363 0.331 0.212
4 E-participation satisfaction 0.771 0.783 0.691 1 0.355 0.177 0.356 0.290
5 Perceived e-govt benefits 0.506 0.432 0.398 0.409 1 0.116 0.306 0.193
6 E-information 0.273 0.304 0.364 0.155 0.053 1 0.495 0.375
7 E-service 0.343 0.271 0.277 0.403 0.309 0.484 1 0.929
8 E-participation 0.253 0.209 0.196 0.382 0.188 0.351 0.931 1
Notes: N ¼ 32. The lower-left and upper-right triangulates are Pearson’s and Spearman’ rank correlation
matrices, respectively. The correlation coefficients larger than 0.350 are statistically significant at the 0.05
level.

satisfaction (see Models 1–3), but only e-information ( ¼ 0.0430, p < 0.01) and
e-service ( ¼ 1.179, p < 0.01) are statistically significant. In other words, citizens
primarily rely on e-information and e-service (instead of e-participation ( ¼ 0.458,
p > 0.1)) to perceive e-government.
Ma and Zheng 13

Table 3. The multilevel model estimates of citizen satisfaction with e-government.

Model 1 Model 2 Model 3 Model 4 Model 5 Model 6

Variable Total Total Total Information Service Participation

E-information 0.0430*** 0.0393***

(0.015) (0.009)
E-service 1.179*** 0.715**
(0.438) (0.341)
E-participation 0.458 0.584***
(0.281) (0.182)
Gender 0.320*** 0.320*** 0.320*** 0.392*** 0.415*** 0.385***
(0.050) (0.050) (0.050) (0.037) (0.040) (0.050)
Age 0.102*** 0.101*** 0.102*** 0.0384*** 0.120*** 0.0245
(0.018) (0.018) (0.018) (0.013) (0.017) (0.022)
Education 0.0401 0.0397 0.0394 0.0948*** 0.0127 0.0684*
(0.029) (0.029) (0.029) (0.024) (0.031) (0.039)
Jobless 0.291*** 0.292*** 0.292*** 0.180*** 0.153** 0.239**
(0.083) (0.083) (0.083) (0.063) (0.069) (0.105)
E-govt use 0.670*** 0.670*** 0.669*** 0.426*** 0.425*** 0.762***
(0.056) (0.056) (0.056) (0.045) (0.054) (0.061)
Constant 4.942*** 4.942*** 4.943*** 5.968*** 5.589*** 5.126***
(0.151) (0.146) (0.150) (0.115) (0.135) (0.161)
Variance 0.151*** 0.149*** 0.163*** 0.107*** 0.0873*** 0.0901***
(L2) (0.037) (0.038) (0.040) (0.031) (0.025) (0.020)
Variance 4.363*** 4.363*** 4.363*** 4.360*** 4.596*** 5.249***
(L1) (0.199) (0.199) (0.199) (0.170) (0.160) (0.187)
Log likelihood 30,560 30,560 30,561 47,706 40,779 25,134
N 14,158 14,158 14,158 22,115 18,675 11,168
L1 R2 0.071 0.071 0.068 0.034 0.036 0.090
L2 R2 0.114 0.127 0.050 0.056 0.073 0.282
Notes: Standard errors clustered at country level are reported in parentheses. *p < 0.10; **p < 0.05;
***p < 0.01. Snijders/Bosker R2 at L1 and L2 are reported. E-government use is operationalized in line
with the dependent variable.

The results show that performance indicators for all three e-government features
are positively and significantly associated with the respective citizen satisfac-
tion indexes (see Models 4–6), and H1 is supported. In terms of the magnitude
of the effect, e-information ( ¼ 0.393, p < 0.01) is much more weakly related to
citizen satisfaction than e-service ( ¼ 0.715, p < 0.05) and e-participation
( ¼ 0.548, p < 0.01), suggesting that citizens prioritize the latter two features in
e-government use.
14 International Review of Administrative Sciences 0(0)

The results show that objective performance indicators for the three features are
all positively related to perceived e-government benefits (see Table 4, Models 7–9),
but only e-service is significant ( ¼ 1.241, p < 0.05). In other words, e-government
benefits are primarily perceived through the use of various online services.
In Models 10–12, the mediation effect tests show that perceived benefits are sig-
nificantly and positively related to overall satisfaction, but the magnitude and sig-
nificance of three e-government performance indicators do not obviously decline.
The findings reveal that perceived e-government benefits do not mediate the
e-government performance–satisfaction relationship, and H2 is not supported.

Discussions
Contributions and implications
In this article, we hypothesize that national e-government performance is positively
related to citizen satisfaction, and the relationship varies across different purposes
of use (i.e. information, service, and participation). We also expect perceived bene-
fits to play a mediating role in the supply–demand linkage in e-government. We
empirically tested these hypotheses by a multilevel model across European coun-
tries. National e-government performance is positively related to citizen satisfac-
tion but the correlation differs among the purposes of use. The results suggest that
it is not sufficient to improve e-government solely by adding more features from the
supply side since citizens vary in their susceptibilities to e-government functions.
While perceived benefits are strongly related to e-service performance, they do not
mediate the e-government performance–satisfaction relationship.
The theoretical contributions of this study are threefold. First, we find that all
e-government performance measures are positively related to citizen satisfaction.
E-information, e-service, and e-participation are all positively related to overall
citizen satisfaction, but only the former two are statistically significant. The esti-
mates of e-government benefits are largely similar, and only e-service is significant.
We provide one of the first tests of the predictive validity of e-government rankings,
which may help consolidate their legitimacy in the field. The varying magnitude of
correlations between e-government performance indicators and citizen satisfaction
may be attributed to their differentiating priorities and relevance to users. Citizens
are more likely to be satisfied with e-government performing well in delivering
online services, followed by participation and information. In comparison
with other e-government features, achieving convenient and efficient services is
the priority of citizens.
Second, for the three e-government features, the results reveal that the supply
and demand sides are largely linked. E-participation performance is not signifi-
cantly related to overall satisfaction and perceived benefits, but its linkage with
e-participation satisfaction is significant. The results imply that it is more relevant
to assess e-government rankings by specific features and functions, though, overall,
e-government ratings are more eye-catching and media-attractive. National
Ma and Zheng 15

Table 4. The multilevel model estimates of e-government satisfaction and perceived benefits.

Model 7 Model 8 Model 9 Model 10 Model 11 Model 12

Variable Benefits Benefits Benefits Total Total Total

E-information 0.00186 0.0476***

(0.003) (0.014)
E-service 0.241** 1.176***
(0.119) (0.375)
E-participation 0.0736 0.526**
(0.073) (0.216)
Gender 0.0394*** 0.0394*** 0.0394*** 0.262*** 0.262*** 0.262***
(0.013) (0.013) (0.013) (0.033) (0.033) (0.033)
Age 0.0479*** 0.0476*** 0.0478*** 0.0739*** 0.0735*** 0.0740***
(0.007) (0.007) (0.007) (0.018) (0.018) (0.018)
Education 0.0126 0.0127 0.0126 0.0215 0.0206 0.0204
(0.011) (0.010) (0.011) (0.033) (0.033) (0.033)
Jobless 0.103*** 0.103*** 0.103*** 0.267*** 0.268*** 0.267***
(0.035) (0.035) (0.035) (0.079) (0.078) (0.078)
E-govt use 0.0734*** 0.0735*** 0.0734*** 0.581*** 0.580*** 0.580***
(0.011) (0.011) (0.011) (0.054) (0.054) (0.053)
Benefits 0.741*** 0.740*** 0.740***
(0.043) (0.043) (0.043)
Constant 2.515*** 2.514*** 2.515*** 3.176*** 3.179*** 3.178***
(0.041) (0.040) (0.041) (0.178) (0.176) (0.178)
Variance 0.0126*** 0.0112*** 0.0121*** 0.103*** 0.108*** 0.117***
(L2) (0.004) (0.003) (0.004) (0.027) (0.024) (0.025)
Variance 0.621*** 0.621*** 0.621*** 3.880*** 3.880*** 3.880***
(L1) (0.029) (0.029) (0.029) (0.198) (0.198) (0.198)
Log likelihood 20,328 20,327 20,328 22,887 22,888 22,889
N 17,180 17,180 17,180 10,898 10,898 10,898
L1 R2 0.017 0.019 0.018 0.148 0.147 0.145
L2 R2 0.059 0.148 0.089 0.303 0.273 0.222
Notes: Standard errors clustered at country level are reported in parentheses. *p < 0.10; **p < 0.05;
***p < 0.01. Snijders/Bosker R2 at L1 and L2 are reported. E-government use is operationalized in line with
the dependent variable.

performance in all three domains is significantly associated with citizen satisfaction

in corresponding features, but the magnitude of the linkages varies. The perform-
ance indicators for e-service and e-participation are more strongly correlated with
respective citizen satisfaction measures than that for e-information. Once govern-
ment information is widely circulated, citizens’ reliance on e-government may be
16 International Review of Administrative Sciences 0(0)

bifurcated by alternative channels. With more and more ways (e.g. social media) to
get information about government operations, government websites are no longer
the most popular channels. Thus, public expectations toward e-government lie
more in e-service and e-participation.
Finally, the mediating effect of perceived e-government benefits is not
supported, and e-government performance is directly related to citizen satisfaction.
Prior individual-level analyses find a positive relationship between perceived
benefits and citizen satisfaction, but our results reveal that objective e-government
performance can be directly linked with citizen satisfaction. The use of
e-government itself can directly fulfill citizens’ performance expectations, though
perceived benefits are also related to e-government satisfaction.
As one specific domain of public services, our findings also join the burgeoning
research stream of assessing the congruence of subjective and objective government
performance measures (Schachter, 2010). Despite the fact that citizen surveys are
subject to various perceptional biases, they still provide stable benchmarking for
public managers to evaluate and learn about public service quality (Miller and
Miller, 1991). Objective/archival and subjective/perceptional measures of public
service performance largely coincide, which strengthens public managers’ confi-
dence in the use of citizen surveys in decision-making. The objective–subjective
performance congruence has only been examined in municipal offline services
(e.g. street cleanliness), and we report one of the first studies to test this proposition
in the field of e-government.
The policy implications of our findings are threefold. First, while e-government
rankings may not be good predictors of citizen use, they do, in part, coincide with
citizen satisfaction and perceptions of benefits. Prior studies reveal that e-govern-
ment performance is not significantly related to citizen use (Park et al., 2013),
but we find that it is significantly associated with citizen satisfaction. The high
performance of e-government may not boost citizen use, which is determined by
various factors like personal needs, access to ICT facilities, capabilities in use, and
so forth. However, once they use e-government, they are more likely to be satisfied
with high-performing e-government features. The government should be confident
in the attractiveness of e-government, and pay more attention to marketing
e-government and attracting citizens to use it.
Second, ubiquitous e-government benchmarks can be referred to as reliable
gauges of citizen satisfaction, though we should be cautious in applying them in
policy design and implementation. The results reveal the varying strength of link-
ages between e-government performance and citizen satisfaction. E-service per-
formance is found to be significantly related to citizen satisfaction and perceived
benefits, suggesting that e-government benefits are primarily derived through
online services instead of e-information or e-participation. The government
should pay more attention to e-service development in order to bring users more
benefits. While many jurisdictions offer e-service portals, few of them provide sig-
nificant citizen participation opportunities (Zheng and Schachter, 2016). The gap
between e-services and e-participation has been reflected in recent reports
Ma and Zheng 17

suggesting that governments should perform better in engaging citizens online

(Holzer and Manoharan, 2016).
Third, the government should catch up with new developments in ICTs to facili-
tate user-centric e-government and retain citizen satisfaction. Given the unprece-
dented development and ubiquitous penetration of social media, the Internet of
things, and other Web 2.0 technologies, the web portals that largely replaced trad-
itional service channels have soon become outdated. It is imperative to include
these new developments in e-government evaluation and improvement. Citizen
satisfaction with government web portals, for instance, may be depressed in the
shadow of social media. The ramifications of these new technological developments
for e-government use and satisfaction should be prioritized in government agendas.
By incorporating these new developments in e-government facilities and evalu-
ations, the government can maintain and steadily increase citizen satisfaction.

Limitations and future research directions

We discuss the limitations of this study and suggest future research avenues in this
section. First, a caveat is needed here about generalizing the findings to other
contexts since e-government development in European countries may be different
from that in other countries and regions. Countries vary remarkably in terms
of geographical, political, cultural, and socio-economic attributes, and it is
meaningful to cluster them by similarities or patterns, which could moderate the
e-government performance–perception linkage. Furthermore, it would be interest-
ing to test the findings in sub-national and local governments since citizens have
distinct attitudes and expectations toward e-government features at different levels.
Second, the e-government performance–satisfaction nexus may be mediated
and moderated by other variables, and it would be promising to explore these
possibilities in future studies. From the perspective of contingency theory,
the e-government performance–satisfaction linkage can be moderated by other
variables. The psychological and social mechanisms between e-government per-
formance and satisfaction have not been fully disentangled.
Third, we use cross-sectional data in the study, which raise reverse causality con-
cerns and disable causal inference. We predict that the relationship between objective
performance and subjective perceptions of e-government is causal in theory (Park
et al., 2013), but it could be plausibly reversed, for example, citizen satisfaction drives
e-government performance. We mitigate reverse causality concerns by lagging inde-
pendent variables, but more robust methods (e.g. the two-stage model), longitudinal
data, and experimental design could be applied in future studies.
Finally, we call for more surveys directly targeting e-government users to bridge
the gaps in existing e-government rankings and ratings, which are predominantly
documented by archival data and expert opinions. E-government is multifaceted
and can be assessed from various perspectives and approaches. The existing evi-
dence on e-government performance consists of either objective indicators gleaned
from content analyses and expert judgments of government websites, or surveys of
18 International Review of Administrative Sciences 0(0)

convenient samples without the representation of and generalization to the popu-

lation. To enhance the validity and legitimacy of e-government benchmarking,
consensus among the two camps should be developed to help comprehensively
understand and prudently promote e-government development.

Conclusion
Do government websites topping league tables satisfy their intended users?
This straightforward albeit untested question was empirically scrutinized in this
study. By estimating multilevel models of over 28,000 citizens across 32 European
countries, this study reveals a nascent linkage between e-government performance
and citizen satisfaction. National e-government performance indicators are found
to be positively associated with citizen satisfaction and perceived benefits, but the
magnitude and significance of the linkages vary across the three key e-government
features. The performance–satisfaction nexus is more profound in e-service than in
e-participation and e-information. While their reliability, validity, and legitimacy
deserve more scrutiny, e-government rankings could be referred to in gauging
citizen satisfaction and navigating government policies.
As for future research, more factors need to be examined to uncover the mech-
anisms through which e-government influences citizen satisfaction. Our findings
can be replicated and extended to other contexts and other levels to see whether
and how e-government performance is linked with citizen satisfaction. In addition,
new approaches and perspectives could be introduced to enhance the validity and
legitimacy of e-government benchmarking, which will facilitate the understanding
of the linkage between e-government development and citizen satisfaction in vari-
ous contexts.

Funding
The authors would like to thank the Fundamental Research Funds for the Central
Universities and the Research Funds of Renmin University of China (No. 16XNB005),
Renmin University’s funding on ‘‘Building World Class University and Discipline’’ (No.
15XNL001), and the Ministry of Education (MOE) Project of Key Research Institute of
Humanities and Social Sciences in Universities ‘‘Scientific and Technological Revolution
and State Governance: A Study Based on China’s Smart Cities’’ (No. 15JJD630014) for
the financial support.

References
Aguinis H, Gottfredson RK and Culpepper SA (2013) Best-practice recommendations for
estimating cross-level interaction effects using multilevel modeling. Journal of
Management 39(6): 1490–1528.
Alawneh A, Al-Refai H and Batiha K (2013) Measuring user satisfaction from e-government
services: Lessons from Jordan. Government Information Quarterly 30(30): 277–288.
Bannister F (2007) The curse of the benchmark: An assessment of the validity and value of
e-government comparisons. International Review of Administrative Sciences 73(2): 171–188.
Ma and Zheng 19

Barnes SJ and Vidgen RT (2006) Data triangulation and web quality metrics: A case study in
e-government. Information & Management 43(6): 767–777.
Bertot JC, Jaeger PT, and McClure CR (2008) Citizen-centered e-government services:
Benefits, costs, and research needs. In: Proceedings of the 2008 International
Conference on Digital Government Research. Montreal, Canada: Digital Government
Society of North America, pp. 137–142.
Boyne GA (2004) Explaining public service performance: Does management matter? Public
Policy and Administration 19(4): 100–117.
Chan FKY, Thong JYL, Venkatesh V, et al. (2010) Modeling citizen satisfaction with
mandatory adoption of an e-government technology. Journal of the Association for
Information Systems 11(10): 519–549.
EU (European Union) (2013) Public Services Online ‘‘Digital by Default or by Detour?’’:
Assessing User Centric eGovernment Performance in Europe. eGovernment Benchmark
2012 Insight Report. Luxembourg: Publications Office of the European Union.
Gauld R, Goldfinch S and Horsburgh S (2010) Do they want it? Do they use it? The
‘‘demand-side’’ of e-government in Australia and New Zealand. Government
Information Quarterly 27(2): 177–186.
Holzer M and Manoharan A (2016) Digital Governance in Municipalities Worldwide (2015–16).
Newark, NJ: E-Governance Institute, National Center for Public Productivity, Rutgers
University, Campus at Newark.
Hung S-Y, Chang C-M and Yu T-J (2006) Determinants of user acceptance of the e-govern-
ment services: The case of online tax filing and payment system. Government Information
Quarterly 23(1): 97–122.
Kelly JM and Swindell D (2002) A multiple-indicator approach to municipal service evalu-
ation: Correlating performance measurement and citizen satisfaction across jurisdictions.
Public Administration Review 62(5): 610–621.
Layne K and Lee J (2001) Developing fully functional e-government: A four stage model.
Government Information Quarterly 18(2): 122–136.
Lee Y, Kozar KA, and Larsen KRT (2003) The technology acceptance model: Past, present,
and future. Communications of the Association for Information Systems 12(1): Article 50.
Meijer A (2009) Understanding modern transparency. International Review of Administrative
Sciences 75(2): 255–269.
Miller TI and Miller MA (1991) Standards of excellence: U.S. Residents’ evaluations of local
government services. Public Administration Review 51(6): 503–514.
Moon MJ (2002) The evolution of e-government among municipalities: Rhetoric or reality?
Public Administration Review 62(4): 424–433.
Morgeson FV and Petrescu C (2011) Do they all perform alike? An examination of perceived
performance, citizen satisfaction and trust with US federal agencies. International Review
of Administrative Sciences 77(3): 451–479.
Moynihan DP and Pandey SK (2005) Testing how management matters in an era of gov-
ernment by performance management. Journal of Public Administration Research and
Theory 15(3): 421–439.
Norris P (2011) Measuring governance. In: Bevir M (ed.) The Sage Handbook of Governance.
London: Sage, pp. 179–199.
Obi T (2008) 2008 Waseda University world e-government ranking released. I-Ways: The
Journal of E-Government Policy and Regulation 31(2): 51–74.
Osman IH, Anouze AL, Irani Z, et al. (2014) Cobra framework to evaluate e-government
services: A citizen-centric perspective. Government Information Quarterly 31(2): 243–256.
20 International Review of Administrative Sciences 0(0)

Park S, Choi Y-T and Bok H-S (2013) Does better e-readiness induce more use of e-government?
Evidence from the Korean central e-government. International Review of Administrative
Sciences 79(4): 767–789.
Reddick CG (2004) A two-stage model of e-government growth: Theories and empirical
evidence for U.S. cities. Government Information Quarterly 21(1): 51–64.
Reddick CG and Turner M (2012) Channel choice and public service delivery in Canada:
Comparing e-government to traditional service delivery. Government Information
Quarterly 29(1): 1–11.
Rorissa A, Demissie D and Pardo T (2011) Benchmarking e-government: A comparison of
frameworks for computing e-government index and ranking. Government Information
Quarterly 28(3): 354–362.
Schachter HL (2010) Objective and subjective performance measures. Administration &
Society 42(5): 550–567.
Seri P, Bianchi A and Matteucci N (2014) Diffusion and usage of public e-services in
Europe: An assessment of country level indicators and drivers. Telecommunications
Policy 38(5/6): 496–513.
Stipak B (1979) Citizen satisfaction with urban services: Potential misuse as a performance
indicator. Public Administration Review 39(1): 46–52.
Tolbert CJ and Mossberger K (2006) The effects of e-government on trust and confidence in
government. Public Administration Review 66(3): 354–369.
Udo GJ, Bagchi KK and Kirs PJ (2010) An assessment of customers’ e-service quality
perception, satisfaction and intention. International Journal of Information Management
30(6): 481–492.
UN (United Nations) (2012) United Nations E-Government Survey 2012: E-Government for
the People. New York, NY: United Nations.
Van Ryzin GG (2006) Testing the expectancy disconfirmation model of citizen satisfaction
with local government. Journal of Public Administration Research and Theory 16(4):
599–611.
Verdegem P and Verleye G (2009) User-centered e-government in practice: A comprehensive
model for measuring user satisfaction. Government Information Quarterly 26(3): 487–497.
Welch EW, Hinnant CC and Moon MJ (2005) Linking citizen satisfaction with e-government
and trust in government. Journal of Public Administration Research and Theory 15(3):
371–391.
West DM (2005) Digital Government: Technology and Public Sector Performance. Princeton,
NJ: Princeton University Press.
Williams A (2015) A global index of information transparency and accountability. Journal
of Comparative Economics 43(3): 804–824.
Zahir I, Vishanth W, Muhammad K, et al. (2012) An analysis of methodologies utilised in
e-government research: A user satisfaction perspective. Journal of Enterprise Information
Management 25(3): 298–313.
Zheng Y (2015) Explaining citizens’ e-participation usage: Functionality of e-participation
applications. Administration & Society 49(3): 423–442.
Zheng Y and Schachter HL (2016) Explaining citizens’ e-participation use: The role of
perceived advantages. Public Organization Review 17: 1–120.
Ma and Zheng 21

Liang Ma, PhD, is Associate Professor at the School of Public Administration and
Policy, Renmin University of China, China. His research interests include public
organizational innovation, government performance measurement and manage-
ment, and urban management and governance. His recent publications include
articles in Public Management Review, Public Administration, and Public
Performance & Management Review.

Yueping Zheng, PhD, is Assistant Professor at the Center for Chinese Public
Administration Research, School of Government, Sun Yat-sen University,
China. His research interests include e-government, digital governance, and
smart cities. His recent publications include articles in Administration & Society
and Government Information Quarterly.

View publication stats