Mechanisms Fostering the Sustainability of Actualized Value Propositions of

Implemented e-Government Projects in Sub-Saharan Africa: A Realist

Evaluation Case of Nigeria and Rwandan Implemented e-Government
Projects
Abstract: This study examines the mechanisms fostering the sustainability of actualized value
propositions of implemented e-Governments projects in sub-Saharan Africa (SSA). Value
propositions in this study identify how public organizations using e-Government fulfill citizens'
needs across different roles. Using a concept-centric systematic review, we identified such value
propositions as effective public organizations, quality service delivery, open government and
democratic, as well as social value and well-being. We used realist evaluation (RE) as a
methodology and combine it with affordance actualization, self-determination theory and
sustainability framework for e-Government success as theoretical lenses. To collect data, we
adopted a mixed method RE approach and used a contingent valuation method as quantitative
aspect to identify the actualized value propositions in operations as outcomes. We chose some
selected e-Government projects in Nigeria and Rwanda and sequentially followed by a realist
interview as a qualitative approach to collect and analyze data to uncover the generative
mechanisms. Our findings revealed technological cognizance and autonomy as well as
competence as individual mechanisms identified for sustaining such outcomes. While
organizational mechanisms revealed are: continuous availability of funds, institutional,
administrative, and coordinating capacity with leadership and political support. Other
organizational mechanisms include institutionalizing the technology within local settings with
ongoing monitoring and improvement evaluation enabled by both internal and external
contextual conditions. We synthesized our findings into an actualized value sustainability
framework for implemented e-Government projects in SSA to contribute to academics and
practice in ICT4D, particularly RE application in e-Government study, which is nascent.

KEYWORDS: Value propositions, implemented e-Government projects, sustainability, realist

evaluation, sub-Saharan Africa

INTRODUCTION
The devastating effect of Covid-19 on the global economy (Ceesay & Bojang, 2020) raises
citizens' expectation for a proactive multi-channel service delivery across sub-Saharan Africa
(SSA) (Verkijika & De Wet, 2018). Governments in SSA countries of Ghana, Nigeria, and
Rwanda have implemented e-Government projects. These e-Government implementations have
changed the narratives in incremental and disruptive quality service delivery. Such service
delivery is resilient to withstand the uncertain effects of disruptions and pandemic such as the
Covid-19 and deliver value propositions as outcomes (Ceesay & Bojang, 2020; Twizeyimana et
al., 2018). Prominent among these implemented e-Government projects is "Umucyo", an e-
Procurement platform in Rwanda (Harelimana, 2018). Irembo (meaning "access" in their local
language) is also in Rwanda (Bakunzibake, 2016). And surveillance outbreak response
1
management and analysis system (SORMAS), a disease surveillance platform in Nigeria (Tom-
Aba et al., 2020). Most of these implemented e-Government projects, if not all, are funded by
donor agencies such as the World health organization (WHO), World Bank, African
Development Bank (AfDB), and others (Ziba & Kang, 2020), as shown in Table 1. Moreover, it
is a common fact that these donor agencies demand accountability for effective operational
utilization of the implemented e-Government projects (Anthopoulos et al., 2016; Jaeger &
Bertot, 2011). Such demand is to continue delivering the envisaged value propositions of such
implemented e-Government projects.
Table 1: Some of the Key Implemented e-Government Projects in SSA
S/N Project Name Project Type Location Implementat Sponsors
ion Date
1 Electronic disease Disease Rwanda 2013 US Centers for
surveillance and response Surveillance Disease Control and
system (ElDSR) (Igihozo et System Prevention and U.S.
al., 2022; Thierry et al., 2014) Agency for
International
Development
(USAID)
2 Irembo (P. Bakunzibake et al., e-Government Rwanda 2017 Rwanda Online and
2019; Pierre Bakunzibake et platform CrimsonLogic
al., 2019) providing
access and
government
services
3 Umucyo e-Procurement Rwanda 2016 World Bank
(Bosco Harelimana, 2018; Platform
Twizeyimana & Andersson,
2019; WorldBank, 2018)
4 Surveillance Outbreak Disease Nigeria 2014 Helmholtz Centre for
Response Management and Surveillance Infection Research
Analysis System (SORMAS) Platform (HZI)
(Silenou et al., 2020; Tom-
Aba et al., 2020)
5 e-Passport (Agency Reporter, International Nigeria 2007 Iris Smart
2022; Okunola & Rowley, passport Technology/Iris
2019; Olafusi, 2021) services Corporation
Malaysia/IOM
6 e-Citizen (Ondego & Moturi, Digital service Kenya 2014 Kenyan Investment
2016; Riany, 2018) and payment Climate Program of
platform the World Bank
Group Trade and
Competitiveness
Global Practice
7 Ghana Integrated Financial Budget and Ghana 2009 World Bank, DFID,
Management Information Financial E.U., and Danish
System (GIFMIS) (Simpson Management International
et al., 2020; Yaokumah & Development Agency

2
 Biney, 2020)
8 Electronic National Transport Management of South 2007 Department of
Information System Licensing Africa Transport (DOT)
(eNATIS) (Rajapakse et al., Records
2012)

Value propositions in this study is defined as the way public organizations fulfill citizens' needs
across different roles using e-Government platforms (e.g., class, gender, ethnicity, and business
concerns). It is perceived to be a means of accommodating all dimensions of e-Government's
performance to prove its relevance to the stakeholders (Chircu, 2008; Flak & Rose, 2005). Such
value propositions are anchored on the capability of implemented e-Government projects to
actualize the different needs of stakeholders and deliver the services they value most
(Twizeyimana & Andersson, 2019). While legitimizing and sustaining the implementation of
such projects (Twizeyimana & Andersson, 2019).
However, literature revealed that after implementing and taking over the implemented e-
Government projects by the benefiting countries in SSA, the envisaged value propositions are
mostly not actualized (Aladwani, 2016; Twizeyimana & Andersson, 2019). Different IS Scholars
attributed these to be as a result of diverse reasons, which are not limited to; IS failure due to
design-versus-reality-gap. Such a gap is the ‘distance’ between IS designers and local contexts
(Anthopoulos et al., 2016; Heeks, 2002). It also includes conflicts between foreign consultants
and local stakeholders due to cultural differences and poor understandings of local norms
(Choudrie et al., 2017).In addition, digital divide, poor IT skills/capabilities by the users, lack of
trust in e-Government services by citizens to mention a few (de Caluwe et al., 2012).
Actualized here is the sum actions taken by the citizens based on their motivations and
capabilities to appropriate, use and exploit the already implemented e-Government projects to
access the e-services afforded by the technology and derive values (Dremel et al., 2020; Strong
et al., 2014). Besides, to move beyond actualizing the value propositions, the focus of most
implemented e-Government projects has shifted to sustainability. Sustainability here is defined
as leveraging the existing structures, processes, and resources associated with the implemented e-
Government projects (Pang et al., 2014), to continue to deliver the actualized value propositions
overtime to the citizens. Besides, the sustainability of such projects focuses on the scope for
adaptation and the resilience of the adopting organization. Such adaptation guarantees the
potential spread, scale-up and maintenance of the technology overtime within the resource
constraint environment (Greenhalgh et al., 2017; James et al., 2021). However, the sustainability
of these implemented e-Government projects has been a significant fall-out and concern to the
donor agencies (Labrique et al., 2018). It is because such agencies need to invest in initiatives
that provide measurable, long-term impact on the delivery of e-services afforded by the
technology to the citizens (Klischewski & Lessa, 2013; Ngulube, 2012; Opoku et al., 2019).
Studies have attributed the non-achievement of sustainability to cultural, political, and financial
constraints (Abaku et al., 2021; Klischewski & Lessa, 2013; Thomas et al., 2009; Yadav et al.,
2019). They also include widespread corruption, poor infrastructures, high inequalities, fragile

3
democracies, and information poverty within the context the innovations are embedded
(Aladwani, 2016; Heeks, 2003).
Prior studies on value propositions of implemented e-Government projects exist. However,
literature revealed the dearth of research focusing on the actualization and sustainability of such
value propositions studies (Bannister & Connolly, 2014; Heeks & Bailur, 2007). Scholars
reiterates the scarcity of such research due to: (1) paucity of appropriate theories-in-use
(Bannister & Connolly, 2014; Heeks & Bailur, 2007), (2) lack of proper identification and
measure of the actualized value propositions by the citizens (Twizeyimana & Andersson, 2019).
Such proper identification are captured through citizen’ feedback and response via
questionnaires and interviews (3) absence of clear delineation of underlying mechanisms and
contextual conditions fostering the sustainability of the actualized values (Yadav et al., 2019).
Identifying such mechanisms and contextual conditions will enable the donor agencies,
government and implementers with clear measures. Such clear measures are to ensure the
sustainability of the actualized value propositions of implemented e-Government projects.
Scholars (e.g.,Heeks & Bailur, 2007; Madsen et al., 2014; Sarantis, 2017) also harp on the
paucity of appropriate theories to evaluate implemented e-Government projects. They emphasize
that previous studies are mostly “theory-applying” rather than “theory-building” to elucidate
contemporary or topical blind spots in e-Government research. Subsequently, scholars (e.g.,
Bannister & Connolly, 2014; Rose et al., 2015; Twizeyimana & Andersson, 2019) call for a need
to measure the values of implemented e-Government projects, while observing bias in the
previous works done. To do this, it is evident that the conventional evaluation approaches
adopted by previous studies are not helpful. They were not helpful because most of the studies
attribute the success of e-Government projects to a yes or no outcome. Furthermore, these
outcomes are based on achieving the short term goals or not (Ojo et al., 2019) and such
characterization fail to be replicated in other contexts.
Consequently, we conduct an evidence-based theory-driven RE of implemented e-Government
projects in SSA within the perimeter of such stakeholders' operations and affordances of the
system (Mukumbang et al., 2016; Pawson & Tilley, 1997). We did this to fill in the gap in the
literature on the lack of actualization and sustainability of value propositions of implemented e-
Government projects in SSA. Evidence-based here suggests that the study obtains empirical
evidence on the relative operational effectiveness of the implemented e-Government projects
(Rycroft‐Malone et al., 2004). Moreover, such operational effectiveness spread across various
stakeholders, including what the technology holds for each stakeholder in their respective
contexts. The specific mechanisms and enabling conditions that triggered the identified
outcomes, in this case, the actualized value propositions, are of particular interest. Mechanisms,
often not directly observed or hidden, are defined as generative (i.e. trigger causation and
processes that produce events or pattern of events (Marchal et al., 2022; McEvoy & Richards,
2006; Pawson & Tilley, 1997). When particularized to e-Government projects, mechanisms
could take the shape of the choices and the reasoning of the stakeholders (Dalkin et al., 2016).
Also capacity buildings, socio-economic and institutional structures (Marchal et al., 2022;
Westhorp, 2014) and what affordances the system provide that makes things change (Strong et

4
al., 2014). Such stakeholders include the e-Government platform designers, donor agencies,
implementers, managers of public organizations and the citizens. To do this, we pose our
research questions as follows:
RQ1. How can the identified value propositions of implemented e-Government projects from
literature be actualized while in use by citizens of the host countries that adopted the
technology in SSA?
RQ2. Under what generative mechanisms and contextual conditions can such actualized value
propositions be sustained?

To answer the research questions and arrive at a more refined theory, we used a combination of
approaches. The approaches we used include a systematic literature review to identify the value
propositions and affordances of implemented e-Government projects. Thence, we use affordance
actualization theory (Strong et al., 2014), self-determination theory (SDT) (Deci & Ryan, 1985)
and sustainability framework for e-Government success (Lessa et al., 2015) . Additionally, we
used RE as a method and as a theoretical framework (Marchal et al., 2022).
We anchor our multi-theory approach on the fact that no single theory can delineate the value
propositions of implemented e-Government projects. In addition, the actualization of such value
propositions, and mechanisms fostering its sustainability as well as the contextual conditions. We
got to know this from the initial empirical evidence collected. After the collection, we matched
the tentative theoretical concepts of the theories-in-use in this study and RE (Mukumbang et al.,
2016) in an abductive manner (Ononiwu et al., 2018). Each theory represents a version of the
phenomenon in question (Ononiwu et al., 2018). Thus, to "develop deeper levels of explanation
and understanding" (McEvoy & Richards, 2006, p. 69), we need such theories to afford a more
compelling account of the actualized value propositions of implemented e-Government projects.
Moreover, to uncover the mechanisms, and contextual conditions that foster the sustainability of
actualized value propositions of the implemented e-Government projects. We are not alone in
doing this, as other IS scholars (e.g., Lee et al., 2005; Rai et al., 2002) deployed multiple
theoretical perspectives. They did this to study IS artifact, context, use, and employees
computing activities unrelated to work respectively in their study.
The motivation to carry out this research is centered on several novel contributions. First, our
study contributes to the growing literature on e-Government evaluation, RE, and information and
communication technology for development (ICT4D) studies. The remainder of the paper is
structured as follows: Section 2 present the systematic literature review summaries. Section 3
discusses the theoretical foundation and research methodology used in the study. Section 4
highlights the findings and results of the RE methodology we adopted. Section 5 presents our
discussion, itemizes our contributions, and Section 6 holds our conclusion and future research
directions..

5
LITERATURE REVIEW
To identify the value propositions and affordances of implemented e-Government projects from
literature. We conducted a concept-centric systematic literature review (Cram et al., 2017;
Okoli, 2015; Schryen et al., 2017; Vom Brocke et al., 2015; Webster & Watson, 2002). There are
different approaches to conducting literature reviews, such as meta-analysis or scoping review
(Paré et al., 2015; Rowe, 2014; Schryen et al., 2017). However, this study chose the concept-
centric systematic literature advocated by IS researchers (Cram et al., 2017; Okoli, 2015;
Schryen, 2015; Vom Brocke et al., 2015; Webster & Watson, 2002). We adopted the concept-
centric approach because it provides the standard of a literature review and stimulate it towards a
synthesis of what is known about the phenomenon of study. It is in contrast to a summary of
what previous authors have reported while forming the foundations of theory building, testing
and developing future research direction (Watson & Webster, 2020; Webster & Watson, 2002).
First, we did a keyword search in titles, abstracts, and references of research archived in
information systems (IS) and e-Government studies databases. Such databases include
Ebscohost, Wiley online, Emerald, Springer, Elsevier, Taylor and Francis, AISeL (hosting all
I.S. conference proceedings and journals).
Additionally, we searched the current e-Government Reference Library (EGRL) version 15.5,
the largest peer-reviewed e-Government repository. We combined the keywords together; "e-
Government value proposition, the public value of e-Government, e-Government evaluation, ICT
enabled public administration, sub-Saharan Africa, e-Government affordance, Affordance
actualization". The used keywords were augmented with the Boolean operators, wild card ("*"
was set to include segments after the phrase). We augment with the Boolean operators to harvest
relevant papers as much as possible to boost our search and to establish accuracy (Cram et al.,
2017).
We chose our keywords to limit our literature search to only articles relevant to our study. At the
same time, it eliminates articles not significant to our research. We followed up by applying our
laid down inclusion and exclusion criteria recommended by IS scholars (e.g., Paré et al., 2015;
Rowe, 2014) to sieve the papers. These inclusion criteria include (a) published peer-reviewed
articles, (b) articles written in the English language, (c) articles published from 2005-2020, (d)
theoretical articles or articles that tested and validated models and frameworks, (e) conceptually
rich articles. The exclusion criteria are (1) articles not written in the English language, (2)
articles published before 2005, (3) a-theoretical papers that are conceptually weak, and (d)
research in progress as well as working papers. We defined our inclusion list based on the critical
features that will enable us to answer our research question in a language that we can read and
understand. While the exclusion criteria include articles that meet the inclusion criteria, but we
feel they may interfere with the study's outcome. Such outcomes can be as a result of language
barrier, research in progress, and empirically weak articles that may not contribute to our
research. Next, we searched the reference lists (i.e., backward citations) of articles included after
the sieve. That further identified a handful of papers that escaped the net of the automatic search
from the database. Finally, we carried out a forward citation search to identify the most recent
6
articles not indexed. In all, we adopted 76 articles as the relevant materials in our systematic
literature review.
We conducted a thematic coding process where we treated the 76 adopted articles as a qualitative
data set (Aksulu & Wade, 2010; Bandara et al., 2015; Inuwa & Ononiwu, 2020; Roberts et al.,
2012). We chose this technique to discover the underlying concepts, themes, and overarching
themes (Aksulu & Wade, 2010; Cram et al., 2017). We adopted Braun and Clarke’s six stage
thematic coding process to accomplish this by following their systematic coding, interpretation,
and analysis process. This process involved (a) familiarization with data, (b) generation of initial
codes, (c) development of themes, (d) reviewing themes, (e) defining and naming themes, and
finally (f) report production. We are not alone in using Braun and Clarke (2006) thematic coding
process in literature reviews advocated by IS scholars (Inuwa & Ononiwu, 2020; Nan et al.,
2020). The six stages are discussed as seen in appendix A highlighting their application in
analyzing our data. At the same time, we now begin to discuss the value proposition gleaned
from the literature, as seen in Table A of Appendix A.

Value Propositions of Implemented e-Government Projects

Following the concept-centric systematic literature review and thematic coding process we
conducted. We categorized the value propositions of the implemented e-Government projects
into four themes: quality service delivery, effective public organizations, open government and
democratic values, as well as social value and wellbeing. (1) Quality service delivery reflects the
generation of values through prompt and proactive delivery in a multi-channel service
environment with easy access to such services (Bannister & Connolly, 2014; Castelnovo, 2013).
(2) Effective public organization delineates the capacity to improve services in a synergetic and
synchronized efficient manner to avoid duplication of tasks (Heeks & Bailur, 2007). (3) For
open government (OG) and democratic value (Deng et al., 2018; Harrison et al., 2012), its value
hinges on the transparency and accountability affordance derived from OG initiatives. Besides,
OG essence is to avail information and decision-making processes of government accessible to
the public for scrutiny and input (Deng et al., 2018). Moreover, in doing this facilitates citizens'
social and political engagements through e-participation in developing and executing
government policies. (4) Finally, social value and well-being; IS scholars always advocate that
the expectations of e-Government go beyond mere citizen satisfaction, but they should
encompass a desire for much broader social outcomes (Karunasena & Deng, 2012). Such social
outcomes include social inclusion, community development, well-being and sustainability
(Harrison et al., 2012). Besides, general well-being in the form of quality of healthcare, threshold
standards of education, and access to civil and criminal justice should not be left out
(Castelnovo, 2013).
Based on our literature review, gaps were identified providing the basis for the study. Such gaps
include: (1) scarcity of research that dwells on the actualization of value propositions of
implemented e-Government projects and their sustainability (Bannister & Connolly, 2014; Heeks
& Bailur, 2007). (2) inadequate identification of value propositions of implemented e-

7
Government projects that are sufficiently actualized while in use by the citizens (Twizeyimana &
Andersson, 2019). (3) lack of identified mechanisms, and contextual conditions fostering the
sustainability of such actualized values (Yadav et al., 2019). (4) and lastly, a few theoretical
applications and development in e-Government studies (Bannister & Connolly, 2014; Heeks &
Bailur, 2007).
e-Government Affordances
The concept of affordance originated from psychologist James Gibson (Gibson, 1979) and drawn
upon by IS scholars (e.g., Markus & Silver, 2008; Strong et al., 2014; Thapa & Sein, 2018).
They define affordance as the potential for behaviors associated with achieving an immediate
concrete outcome arising from the relation between an IT artifact and a goal-oriented actor or
actors. Taking the understanding of affordance theory further, Strong et al. (2014) came up with
the affordance actualization concept. This is defined as "the sum actions taken by actors as they
take advantage of one or more affordances through their use of technology to achieve immediate
concrete outcomes in support of organizational goals" (p. 15). Contextualizing it to our study,
and of particular interest are e-Government affordances that citizens and others users of the
implemented e-Government projects engage with.
The e-Government affordances gleaned from literature (as seen in Table B of appendix A)
includes: (1) value co-creation affordance, (2) accessibility affordance, (3) mobility affordance,
(4) dialogue affordance, and (5) transparency affordance. Value co-creation affordance is when
citizens take active roles and co-create value together with the public organization through the
implemented e-Government project (Fung, 2015). It is an intentional and goal-oriented process
where citizens deliberately engage in the implemented e-Government project when striving for
outcomes.
Such engagement with public organizations should also empower citizens in different co-
creation processes via user involvement, service support, and well-being (Wang et al., 2016). For
example, well-being occurs due to the interactions in the value co-creation process for citizens to
obtain adequate health and justice. Other well-being includes educational access, satisfaction,
happiness, harmony, and respect among citizens in individual and collective fashions (Anderson
et al., 2013). Consequently, value co-creation is essential to quality service delivery and social
well-being as an outcome.
Accessibility affordance enables citizens to be involved with a high degree of visibility and
persistence, making it possible for them to access services quickly and conveniently (Yang et al.,
2020). Accessibility affordance reflects services that are reachable and readily accessible to all
citizens irrespective of their physical, motoric, or perceptual disabilities (Avdic & Lambrinos,
2015),enhancing quality service delivery as an outcome.
Dialogue affordance enables deliberations and the exchange of views, creating and maintaining a
satisfied, loyal citizen (Giesbrecht et al., 2015). Creating dialogue affordance requires
interactivity, deep engagement, feedbacks, and responsive open communication. Such
communications give citizens a sense of being part of the project enhancing open government
and democratic values.

8
Transparency affordance makes information about services public and open (Stamati et al., 2015)
and eliminates information asymmetry between citizens and public organizations. Besides, it
increases the level of openness and helps in building citizens' trust. The transparency affordance
creates persistent, reliable information from public organizations, which provides citizens with
access to rich, reliable information. Citizens rely more on information-generated content to
support their decision-making (Dini et al., 2018), and transparency enhances effectiveness in
public organizations.
Mobility affordance captures a ubiquitous interface that ensures accessibility to information
while on the move (Dremel et al., 2020). The essence is to avail the citizens of varying
accessibility options irrespective of their locations, abilities, and competence (Twizeyimana et
al., 2018), thereby enhancing the quality of service delivery.
Understanding these e-Government affordances in our study is essential and beneficial in
contributing rich information to delineate an emergent e-Government artifact-in-use (Thomas et
al., 2009). Equally important is understanding the actualization processes in a complex and
dynamic service delivery systems (Lawan et al., 2020). e-Government affordance is the action
potential associated with achieving value propositions as outcomes (Majchrzak & Markus, 2012;
Strong et al., 2014; Volkoff & Strong, 2017; Wahid & Sæbø, 2015). However, e-Government
affordances exist independent of perception and remain latent until it is acted upon or actualized
by the citizens, and such affordances are underlying mechanisms in our context (Volkoff &
Strong, 2017). Consequently, the actualization process is the sum of cumulative actions taken by
the citizens as they take advantage of one or more e-Government affordances through their use
of the implemented e-Government project (Strong et al., 2014). The actualization process is
influenced by socio-cultural and technical factors. And it reflects the dynamism of the
relationship between the capabilities of the citizens and the features of the implemented e-
Government project (Anderson & Robey, 2017). Such actualization occurs in the presence of
enabling contextual conditions (Thapa & Sein, 2018).

THEORETICAL FOUNDATION
There are diverse theoretical perspectives that can give us insight into e-Government affordances
and actualization of the value propositions of implemented e-Government projects. We chose
affordance actualization theory (Strong et al., 2014) because it provided us a better guide in
investigating the process through which e-Government affordances are actualized to produce
value propositions as outcomes. Anderson and Robey (2017) use the affordance actualization
lens in their study to gain better theoretical insights into IS affordances and their implications in
IT-enabled work practices. Thus, the combination of affordance actualization and RE theory
empowers us to a five-fold view of the affordance actualization process. These are affordances,
context, mechanisms, actualization actions, and outcomes (Pawson & Tilley, 1997; Strong et al.,
2014). Besides, the processes of the actualization actions vouchsafe the human agencies'
(citizens) choice to engage in the implemented e-Government project. As well as their

9
motivations and capabilities to exploit, appropriate, and use the affordances offered by the
implemented e-Government projects (Thapa & Hatakka, 2017).
With this in mind, we used Self-determination theory (SDT) (Deci & Ryan, 1985) as a universal
theory of human motivation. We did this to investigate how individuals interact with the
implemented e-Government project. Following Rezvani et al. (2017) who deployed SDT to study
how supervisors' leadership approach influences ERP users' motivation to use the ERP system
continuously. We used SDT to understand the specific nature of individual motivations within
the experiences of choices that determine their actions. Such determinants are autonomy,
competency, and relatedness considered (Deci & Ryan, 1985). Autonomy is the need to feel free
and possess self-directed cognizance in the environment, which signifies one's sense of control
and agency (Deci & Ryan, 1985). Competency is a feeling of being effective, broadly, and
competent with tasks, activities, and engagements (Deci & Ryan, 1985). Finally, relatedness is a
feeling of being included and affiliated with others.
Lastly, we used the sustainability framework for e-Government success (Lessa et al., 2015). We
also used this to understand what fosters the sustainability of the value propositions as outcomes
from the affordance actualization process at the organizational level. Scholars (e.g.,Aljarallah &
Lock, 2020; Mtebe & Sausi, 2021) used the sustainability framework for e-Government success
to investigate a sustainable e-Government service in Saudi Arabia and a public value perspective
of an e-Payment Gateway system in Tanzania, respectively.
The sustainability framework for e-Government success (Lessa et al., 2015) affords us rich
concepts distinguishing the sustainability of the implemented e-Government project is in terms
of organizational innovation continuation. Such concepts include (a) developing a sense of
national ownership, (b) continuously meeting available resources, (c) independence from donor
agencies/external assistance, (d) continuous monitoring and evaluation, (e) institutionalizing e-
Government projects within local context (f) solid political support and leadership (g)
availability of institutional, administrative and coordinated capacity (h) meeting stakeholders
needs (Lessa et al., 2015). Relying on the literature review we conducted, we gleaned and
operationalize the sustainability of the implemented e-Government project into five dimensions.
Such dimensions include (a) endurance of the identified actualized values (Slaghuis et al., 2011).
(b) persistence of routinization or institutionalization of the initial e-Government projects (Akaka
et al., 2017). (c) continued adaptation within the context the innovation is embedded (Buchanan
et al., 2006). (d) evolutionary growth through configuration (Fleiszer et al., 2015), and (e)
maintenance and appropriation with local capacities over time (Heeks, 2005).
However, other theoretical perspective can be deployed to study the actualization and
sustainability of implemented e-Government projects. Few among these theories are adaptive
structuration theory (AST) which focuses on the faithful or unfaithful appropriation of the
technology by users (DeSanctis & Poole, 1994). We disregarded AST because it did not consider
the concept of affordance (Markus & Silver, 2008). Neither did AST consider that technology
appropriation by users can be modulated by the designed material properties (Markus & Silver,
2008). Similar reasons were also responsible for ignoring task-technology fit (Goodhue &
Thompson, 1995). Task-technology-fit postulates that technology can increase the user's

10
performance if the characteristics of the technology match the requirements of the task as well
abilities of the users (Goodhue & Thompson, 1995). Hence our theoretical choice is explained in
the introductory part of our theoretical foundation.

RESEARCH METHODOLOGY
Our study is anchored on RE as a methodology within a multiple case study approach
(Mukumbang et al., 2016; Pawson & Tilley, 1997). RE is a theory-driven evaluation approach
that emanates from a realist philosophical view primarily about theory development, testing, and
refinement (Pawson, 2006, p. 9). RE aims to understand the right contextual conditions that the
internal potential of a program or intervention produces intended and unintended outcomes.
Understanding the context, mechanisms, and outcomes configuration (CMOc) explains what
works, for whom, and under what conditions (Pawson & Tilley, 1997). It seeks to understand
how complex interventions work in real situations and how "real social systems often result in a
complex world" (Miller & Page, 2007, p. 26).
We hinged our rationale for using RE to develop a deeper understanding of the phenomenon
under study. We develop a program theory from the study's outcome that offers excellent
potential for translating the research findings into policy and practice. Besides, by adopting RE,
our study provides a sound level of specificity required to; (a) unravel a complex social
intervention such as implemented e-Government projects; (b) proffer solution that meets the
needs of specific interventions; (c) produce research findings well specified to further
improvements on other implemented e-Government projects in SSA.
However, the methodological approach in RE has some shortcomings, including; the inexplicit
nature of the RE approach. It also exposes the need for detailed descriptions and more
transparency in its data analysis (Gilmore et al., 2019). Additionally, the elicitation of CMOc and
the refinements of theories primarily depend on the researcher's judgment, existing knowledge,
and documenting detailed decision-making process within the analysis (Emmel et al., 2018).
Furthermore, there is a need for terminological clarity and consistency in RE because
interpretations of key realist terms are laborious and hard to comprehend. Such amendments will
make it more transparent, including sources, definitions, and applications (Marchal et al., 2012).
However, the critical strength of RE is the explanatory depth of the approach and how it
improves a better knowledge of the subject matter or phenomenon under investigation. Finally,
RE is pragmatic and produces policy-relevant findings easily transferred across policy and
practice settings (Fletcher et al., 2016; Salter & Kothari, 2014).
RE approaches of varying models differ from the original version developed by Pawson and
Tilley (1997). Thus, we adapted the Mukumbang et al. (2016) model that compacted the initial
six stages of RE of Pawson and Tilley (1997) into three phases. Such phases include (1)
developing the Initial Program Theory (IPT), (2) designing the case study and the case selection
process, and (3) cross-case analysis of integrated CMO elements across the three cases to form a
Refined Program Theory (RPT). Furthermore, the adapted Mukumbang et al. (2016) model is
placed on a multiple case study approach as shown in phase 2 of Figure 2, and such a model can
take a mixed-method (quantitative and qualitative) approach in collecting and analyzing data

11
(Ebenso et al., 2017; McGaughey et al., 2017). Consequently, we built upon methodological
insights from prior RE mixed-method multiple cases empirical studies (Abildgaard et al., 2019;
Andrikopoulou et al., 2020; Ebenso et al., 2017) and used Mukumbang et al. (2016) adapted RE
model as detailed next starting with Phase 1 - developing the IPT.
Phase 1-Gear towards Developing the Initial Program Theory (IPT)
We gained access to the field (the case organizations or the investigative contexts). Irembo e-
Government system, e-Procurement system implemented in Rwanda and the e-Passport
immigration system implemented in Nigeria. Based on the access to the organization, we
conducted an exploratory case study where we started with reviewing documents collated. Such
documents include the e-Government plans and designs, implementation documents, post-
implementation plans, and sustainability of the e-Government projects. Then we proceeded with
a systematic literature review as highlighted in the literature review section, from where we
identified the value propositions. We then followed up with a collation of initial empirical
evidence by conducting face-to-face interviews with managers of Irembo, e-Procurement, and
officials of NIS. Subsequently, we followed up with data triangulation of the interview extract
with (1) participant's observation in the case organizations and archival document analysis (2)
the synthesized literature review, (3) scoping review of the theories-in-use in this study, as seen
Figure 1.

Figure 1: Phase 1-Gear towards Developing the Initial Program Theory (IPT)
These theories-in-use are affordance actualization theory (Strong et al., 2014), SDT (Deci &
Ryan, 1985), and sustainability framework for e-Government success (Lessa et al., 2015). In
addition to the theories is RE’s context-mechanism-outcome-configuration (CMOc) (Pawson et

12
al., 2005). Such a data triangulation guided an understanding of human-IT interaction and
extractions of the affordances of the implemented e-Government projects. The actualization is
seen as the value propositions, the sustainability of such values and the generative mechanisms
and contextual conditions that could enable such sustainability (Layder, 1998). Following
Mukumbang et al. (2016), we used such a triangulation of data from both primary and secondary
sources to develop the initial program theory (IPT) as seen in Figure 2.

Figure 2: Initial Program Theory (IPT) expounded from literature review, and initial data
collected from multiple investigative contexts subject to refinement. Source: Authors
We anchored this process of IPT development through abductive data analysis that aligned with
Layder (1998) version of adaptive theory development through abductive reasoning. Such a
reasoning proceeds from (1) the retroductive observation of outcomes/events seen as empirical
evidence that suggest anomalies or surprising to (2) the delineation of a kind of explanatory
hypothesis, which (3) fits into an organized pattern of extant theoretical concepts (Capaldi &
Proctor, 2008; Råholm, 2010). Thus, we adopted and integrated the dialectical interplay where
theoretical concepts of the theories-in-use in this study are matched with the initial empirical
evidence. Then any of such concepts that are near the data are considered and those that not are
filtered out (Dubois & Gadde, 2002; Ononiwu et al., 2018). The considered concepts amid the
competing theories in use (i.e., integration of theories-in-use) (Capaldi & Proctor, 2008) are then
used to construct novel hypotheses that explain the pattern of data (Capaldi & Proctor, 2008; Dubois
& Gadde, 2002; Ononiwu et al., 2018). It also suggests that such hypotheses should be accepted as
inference to the best explanation.

13
Altogether, such integration of theories "shape and inform the analysis of data that emanate from
the ongoing research at the same time the emergent data itself shapes and molds the existing
theoretical materials" (Rambaree, 2018, p. 68). Our actions here are recursive between collection
and analysis of data and prior theoretical pattern matching iteratively until data saturation. Braun
and Clarke (2019) view of “saturation' means that no additional data are found whereby the
sociologist can develop properties of the category. As the coder sees similar instances repeatedly,
the researcher becomes empirically confident that a category is saturated." (p. 61). Abductive
thematic analysis is seen here as a pragmatic methodological approach incorporating an
abductive way of reasoning in studying and explaining linkages between emerging pre-existing
theoretical themes from gathered qualitative data sets (Rambaree & Faxelid, 2013). The IPT is
now subjected to further empirical corroboration which takes us to Phase 2 – designing the case
study and case selection.
Phase Two – Designing the Case study and Case Selection
As noted earlier, we adopted the multiple case study design because it allows for an in-depth
exploration of complex, multifaceted phenomena as they play out in real-world settings
(Marshall & Rossman, 2014). Such a case study considers the role of socio-economic, political,
and cultural contexts. It allows for flexible exploration of various levels of analysis from the
individual to the organizational level (Tomlinson et al., 2018). Evidence developed from
multiple case studies is considered reliable and robust (Baxter & Jack, 2008) while creating a
credible theory, remarkably when the data is grounded in empirical evidence. In addition to
gaining in-depth analysis of each case, the case study approach affords us an opportunity for
comparison between each case.
Adopting multiple investigative cases and organizational variation gives a diverse perspective of
how contextual factors influence implementation. Then they can be used to develop a better
evidence-based informed decision of change processes. Consequently, following RE scholars
(e.g., McGaughey et al., 2017; Mukumbang et al., 2016) who used RE multiple cases as a
theory-building approach, we selected three cases, as shown in Figure 3.

14
Figure 3: Phase 2- Designing the case study and case selection
As Figure 3 could attest, our choices are three dominant entities: a typical case, a deviant case,
and a crucial case. The "typical" case is in the form of an ideal one-stop platform that provides
government-to-citizen (G2C), government-to-business (G2B), government-to-government
(G2G), and government-society (G2S) services on a single platform. Irembo, in Rwanda, fits into
this specification, hence our choice. A "deviant" case takes the form of an isolated security-
sensitive document service. e-Passport of NIS is an ideal investigative context and selection.
Finally, the crucial case looks at isolated G2B services that provide critical services. For
example, the e-procurement platform used for supplies and contract bidding plays a pivotal role
in creating a level playing field, confidence building, and trust. Moreover, this satisfies our
criteria for selection as one of the case studies.
Investigative Context Embedded with Contextual Conditions
To focus on the boundary of the investigative contexts upon which the developed theory from
this research will be generalized. We selected SSA countries of Nigeria and Rwanda, as
mentioned earlier. Rwanda, a country classified as a developing economy, is leading the pack in
ease of doing business in Africa. It is also transforming to a knowledge-based economy
(Twizeyimana et al., 2018). On the other hand, Nigeria, the most populous nation in Africa with
the most significant ICT market, requires proactive and diverse e-service delivery channels to
manage its teeming population. Consequently, the Nigerian government has implemented e-
Government projects of various functions, as shown in Table 1.

15
We chose the investigative context of e-Passport introduced by the Nigerian Immigration
Services (NIS) since literature reiterated it as a pacesetter and the first e-Passport in Africa
(Okunola & Rowley, 2019). In the case of Rwanda, we selected Irembo because it is also touted
in literature as the best one-stop online platform for all government services in Africa
(Twizeyimana et al., 2018). Finally, the e-Procurement platform in Rwanda is also the pioneer
e-Procurement platform in Africa (Harelimana, 2018).
Irembo
Irembo (meaning "gateway" in local parlance) is a collaborative partnership between the
Government of Rwanda and Rwanda Online Platform Ltd (ROL). It is a public-private
partnership initiative that started in 2017 (P. Bakunzibake et al., 2019). The ROL and Rwandan
Government partnership is to provide a one-stop online platform for all government services. As
a technical partner, ROL will integrate all government services across G2C, G2B, G2G, and G2S
(Mukamurenzi et al., 2019). Services initially enrolled are registration such as birth, death,
marriages, driver's license, and business registration. Other benefits include tax filing, land
management, road traffic, and motor vehicle inspection (Twizeyimana, 2017). In addition,
Irembo provides multiple services accessible via mobile phones, USSD, computers, or a visit to
Irembo service centers and Irembo agents.
e-Passport
NIS introduced the e-Passport on the 27th of July 2007 to replace the Machine-Readable
Passport (MRP) (Okunola & Rowley, 2019). The NIS, with support from donor agencies,
installed the e-Passport machines in all36 states of the federation and the Federal Capital, Abuja
(Lawan et al., 2020). MRP was the first effort to implement of e-Government reforms in the NIS.
Then e-Passport followed as a global best practice with biometric details of holders as a security
feature to prevent identity theft, data alteration, and multiple passport acquisitions (Lawan et al.,
2020). With the introduction of the e-Passport, the NIS becomes the first immigration agency in
Africa to introduce the e-Passport technology and also among the first countries in the world to
do so (Okunola & Rowley, 2019).
e-Procurement
An e-Procurement platform called "Umucyo", translated as transparency (Harelimana, 2018) is
implemented by the government of Rwanda. The implementation was done to transform
procurement processes from traditional based to system based. It is the global best practice in
procurement, fostering the fundamental principles of transparency, accountability, competition,
equity, and fairness (Omwono et al., 2020). Owing to its deployment in 2016, e-Procurement
(Umucyo) is now the only avenue for all public procurement processes in Rwanda by both public
and private institutions (Harelimana, 2018). The initiative started with eight public institutions;
the Rwanda public procurement authority, the Rwanda development board, and ministry of
finance and economic planning. Other institutions include the ministry of justice, Rwanda
revenue authority, Rwanda social security board, banks, and insurance companies.

16
The contextual conditions explicated in our study to foster the sustainability of actualized value
propositions of implemented e-Government projects are as follows; (1) Societal factors that
encapsulate economic, social, political, and institutional parameters (Twizeyimana et al., 2018).
(2) External factors examine donor agencies' influence (Ziba & Kang, 2020). Under societal
factors, the economic consideration captures the reality of adopting an I.T. innovation,
specifically e-Government implementations that require financial muscle. For example, to
acquire hardware and software, support system integration, train employees and, hire people with
specialized skills (Twizeyimana et al., 2018).
However, it does not just stop at that, but creating an enabling environment where the
implemented e-Government project should be less dependent on host government funds, donor
agencies' funds and a revenue generation model. The social factor focuses on reducing the digital
divide, ICT illiteracy, and the ability to design e-Government systems for social inclusion (Choi
et al., 2017). Moreover, such a design reflects multiple accessibilities, multiple payment options
and availability of networks of agents to enhance access and ease of use. Political factor plays a
crucial role in the success of implemented e-Government projects. Because political leadership
guides in allocating resources and making all the necessary political and legislative reforms to
aid the smooth implementation of e-Government projects (Adu et al., 2018).
The greater the political support, the easier it is to overcome the complexities and difficulties in
e-Government implementation and continuous use (Lawan et al., 2020). The institutional factor
embodies all the legislative, political, financial, and technological factors that will enable solid
institutional support for implementing and continuously the implemented e-Government projects
(Verkijika & De Wet, 2018). The enactment of such institutional factors is vital in e-Government
project implementation in the adopting nations wherein the institutional framework derived from
such factors holds and epitomizes how the project will be implemented and managed.
External factor dwells on the roles and influences of donor agencies in the implementation of e-
Government projects. Donor agencies provide support in the form of loans, grants, aid, training,
and equipment while, in some instances, helping set up infrastructure and funding for the
projects (Thomas et al., 2009; Ziba & Kang, 2020). Nevertheless, the major drawbacks are the
over-reliance on the donor agencies for funding and support in SSA. Such over-reliance leads to
strict demands of operational and financial accountability from the donor agencies, which we
articulated as an external factor under contextual conditions.

DATA COLLECTION
We used quantitative and qualitative approaches to collate data longitudinally that spans two
years. In RE, the method is not something we decide a priori but is always appropriate to the
research process as it unfolds while in the field to answer the research questions. Consequently,
RE endorses methodological eclecticism and pluralism in mixed methods (Abildgaard et al.,
2018; Gilmore et al., 2016; Nurjono et al., 2018). We now discuss the quantitative data collection
that we adopted as detailed next.

17
The Quantitative Data Collection
We conducted a quantitative survey using the contingency evaluation method (CVM) from April
2018 to January 2019. CVM is a survey technique that elicits placement of citizen's value as
essential or nearly indispensable upon some service delivery based on their personal needs and
preferences in their everyday life activities (Nocera et al., 2003). When particularized to e-
Government services, measuring the citizen’s commitment to meet such needs through the high
value, they placed on e-Government services demands that citizens must show a willingness to
pay (WTP).
Thus, CVM is achieved by asking citizens in the survey how much they value the e-Government
services and what they are willing to pay for them. The CVM is a well-acknowledged research
method for estimating the value of intangibles (Cha et al., 2019). It allows the researcher to
create a hypothetical market scenario as if the market existed in the real sense and, using surveys.
The reason is to examine the price that consumers are WTP when seeking to consume a certain
non‐market goods (Cha et al., 2019). CVM is used here to confirm that the identified value
propositions – i.e., quality service delivery, effective public organizations, social value and
general well-being, as well as Open government and democratic values - from the literature
review were actualized by citizens in their contexts of use (Cha et al., 2020; Chung, 2008).
Following scholars (e.g., Cha et al., 2020; Chung, 2008), we operationalized the value
propositions (as seen in Table A of appendix A) to develop the CVM instrument in the form of
questionnaires. Such an instrument generates categorical data of Yes or No when the citizens
complete it. We distributed 630 questionnaires in a printed format to citizens of Rwanda and
Nigeria who used Irembo, e-Procurement systems, and e-Passport. We translated 250 out of the
630 questionnaires about Irembo to Kinyarwanda, the local language in Rwanda, with the help of
a paid professional translator. We noticed Kinyarwanda as the local language is widely used to
engage with the e-Government platforms by the citizens. Later, the translator re-translated the
answered questionnaires from Kinyarwanda back to the English language for our consumption.
One hundred and fifty (150) questionnaires also for Irembo were in English to cater to citizens
who might prefer to use English. We distributed 30 questionnaires in English for e-Procurement
since most businesses use English on the platform. For e-Passport users in Nigeria, we
distributed 200 out of the 630 questionnaires in the English language. The participants were
selected using snowball sampling techniques and referrals from friends, office/business
colleagues, acquaintances, and family members. The answered questionnaires that we got back
to use were 576 which is 90% of the total number sent out, suggesting that citizens,
organizations, and businesses of the host countries that adopted the technology are enthusiastic
about understanding the actualized value propositions.

The Qualitative Data Collection- the realist interview

18
In a realist interview, Connelly (2001) advocated for its use as a privileged data collection
method. Because 'it is the primary source of identifying outcomes [in our case actualized value
propositions] as well as to explicate the generative mechanisms at work [to produce such
outcomes] in the specific context understudy" (p. 12). Thus, our rationale for adopting realist
interviews is the crucial aspect of the 'knowledgeability' of the interviewees/participants who
knows what is in terms of effective use of the implemented e-Government projects.
Consequently, we conducted realist interviews with participants who were actual users of the
technology and implementers and selected our participants through snowball sampling and
referrals. We started by seeking approval to interview and collect data from the three (3)
investigative contexts. It began with preliminary interviews, participant observation, and archival
documents to enrich our data collection, as detailed in phase 1 in Figure 1 to generate the IPT.
Our data collection started with a familiarization visit and preliminary interviews at Irembo,
NIS, and e-Procurement in October 2019. We followed up with an in-depth realist interview
from January to April 2021. We interviewed fifteen (15) senior managers of Irembo, e-
Procurement, and NIS e-Passport managers comprising of 5 managers in each organization. All
the interviews lasted about 40 minutes with structured questions that aligned with the IPT.
However, we replaced the 'real' names of the managers interviewed in the organizations with
identification numbers for anonymity and confidentiality. A realist interview is where the IPT
was discussed with interviewees, audio-recorded, and transcribed verbatim.
However, unlike in Phase 1, where we conducted a face-to-face interview, the interviewing
process in a realist interview is strongly theory-driven (Pawson & Tilley, 1997). The IPT is the
subject matter of our discussion, and the participants confirm and refine our IPT (Pawson &
Tilley, 1997). We conducted the realist interview in two phases: (1) the teacher-learner function,
where we ensure that we enlighten our participants by availing them of the knowledge of our IPT
(Manzano, 2016). The teacher-learner approach is a process where the interview starts by
teaching the participants/interviewee the IPT under test. After they have understood the IPT
under test, they can inform us about the components of the implemented e-Government projects
in an informed way (Pawson & Tilley, 2004). The participants are custodians of facts and vital
details and willingly offered them in the interview (Verkooijen et al., 2020). Knowledge is
mutually transferred , whereas as the researchers, we hold the expertise of concepts while the
participants hold experience, actions, and apperceptions as expertise. (2) The conceptual
refinement process where the participants delivered their thoughts on their thinking in the
context of amending or correcting our IPT (Mukumbang et al., 2020).

DATA ANALYSIS

19
We conducted two phases of data analysis in line with the sequential approach of collecting data
earlier discussed: (1) the quantitative data analysis and (2) qualitative data analysis. We start
with the quantitative data analysis.
Quantitative Data Analysis
The data analysis for the quantitative data is the composite logistic regression (Dalkin et al.,
2016; Henseler & Schuberth, 2020). Such a regression allows the measurement to understand
what observable variables could constitute a dependent variable seen as emergent variable
(Benitez et al., 2020; Benitez et al., 2018; Dalkin et al., 2016). It is seen as a multi-dimensional
concept appraised through the composite indictor variables' degree of importance in the emergent
variable, the polarity of effects, and significant results (Benitez et al., 2020; Polites et al., 2012;
Yilmaz & Belbag, 2016).
The actualized value propositions here should be seen as the emergent variable. Such a variable
in this study comprises of indicators of quality service delivery, effective public organizations,
social value and general well-being, as well as open government and democratic values.
Consequently, the actualized value propositions in realist term are considered as the semi-
outcome patterns or the observable demi-regularities that emerged due to the interactions of the
generative mechanisms (Jagosh et al., 2012; Lawson, 1997).
Such regularities can be subjected to quantitative data collection. Moreover, the regression
analysis of such data in realist philosophy is possible since statistical models can be used to
represent observable actions of agency or to account for observable outcome patterns. These
observable outcome patterns give some clues regarding the emergent effects of mechanisms
(Iosifides, 2016; Mingers, 2006; Olsen, 2010). Besides, such outcome patterns indicate the
presence of active mechanisms that then triggers a search for potential, theoretically informed
explanations (Eastwood et al., 2014; Iosifides, 2016; Mingers, 2006; Olsen, 2010; Porpora, 2015;
Pratschke, 2003).
Consequently, the logistics regression used here accounts for the observable outcome patterns
and does not provide an explanatory model for variables that do not have causal powers (Olsen,
2010; Pawson, 2013). Instead, the logistics model provided a constitutive association of variables
that co-vary or a composite model (Benitez et al., 2020; Bollen & Bauldry, 2011; Dalkin et al.,
2016; Henseler, 2017). It empowers us to start “reasoning from correlational data patterns to
underlying common causes” (Haig, 2005, p. 308).
Thus, the composite model accounts for the various outcome patterns that make-up the
actualized value propositions and how they co-vary. However, since the existence of association
is necessary but not a sufficient condition for explanatory analysis (Eastwood et al., 2014;
Mingers, 2006; Modell et al., 2007), we adopt an explanatory qualitative approach. Such an
approach follows an abductive-retroductive mode of reasoning. This is done to uncover the
generative mechanisms and contextual conditions in Phase 2 of our data collection and analysis
(Mingers, 2006).

20
The mechanisms and contextual conditions so now identified are understood as what might cause
the various outcome patterns. Such patterns represent the actualized value propositions to occur,
hence, leveraging the power of mixed methods research in realist palace (Eastwood et al., 2014;
Nurjono et al., 2018; Zachariadis et al., 2013). Consequently, we adopt the qualitative data
collection (i.e., realist interview) and analysis most suitable for the explanatory qualitative
approach. Such approach takes the form of an intensive case study design aligned with an
abductive-retroductive mode of reasoning as shown in Figure 2 (Iosifides, 2016; Mukumbang et
al., 2016; Sayer, 1992).
The CVM data collection and its composite logistic regression of data analysis are crucial to this
study. They are crucial to the interpretation of binary and categorical data, to confirm the
identified value propositions of implemented e-Government projects. Moreover, what the
citizens and users of implemented e-Government projects in SSA are willing to pay to actualize
the value propositions in their contexts of use (Yilmaz & Belbag, 2016). In the logistic
regression analysis, we developed Goodness of Fit models that measure Likelihood, and the log-
likelihood (-2LL).
Additionally, the Nagelkerke R square (as seen in Table C, D and E in appendix B) for Irembo e-
Government platform, e-passport, and e-procurement, respectively. The Goodness of Fit models
shows a positive and higher percentage of the variation in the actualized values in quality service
delivery, effective public organizations, social value, and well-being as well as open government
and democratic values. Tables C, D, and E in appendix B show the result of the data analysis.
Our regression approach here is in line with other scholars (e.g., Dalkin et al., 2016; Eastwood et
al., 2014; Nurjono et al., 2018) who used regression analysis in their RE research to gain insights
into specific e-health interventions. Consequently, we used regression analysis in our study to
confirm whether the value propositions gleaned from literature are a constitutive aspect of the
citizens' actualized value of the implemented e-Government projects in SSA while in use.
Qualitative Data Analysis
For the qualitative data analysis, we aligned with the works of RE scholar (Manzano, 2016).
First, we extracted the data sets from the realist interview through an abductive-retroductive
strategy with a high degree of reflection and iteration. Then the affordances, contexts,
mechanisms, and sustainability concepts were captured by the IPT and matched with the data
available using counterfactual thinking (Meyer & Lunnay, 2013) . The counterfactual thinking
approaches in this study guide us to reflect on the different extra-discursive realities in the data
(Iosifides, 2016).
Or additionally, what is at the back of the data to allow such participants' verbalization or
researchers' observed events to happen (Iosifides, 2016; Meyer & Lunnay, 2013). At the same
time, we discovered that such extra-discursive realities are referenced in the IPT as affordances,
contexts, mechanisms and sustainability concepts. Our approach of extracting the data is
described by Schon (1990) as a 'move-testing' experiment, in which a theory developed through
reflection can be confirmed or negated in the course of action.

21
Empirical Findings & Results
We discussed the findings and results starting with the quantitative findings and results and
thereafter we did the one for qualitative findings.
Quantitative Finding and Results
We want to reiterate here that our findings derived from the regression analysis is just ‘a tool’ for
discerning patterns in the empirical survey material ‘in a more nuance and sophisticated way’ but
cannot establish causality (Næss, 2015, p. 1235). Thus, the findings are fundamentally
descriptive of the effect of the actualized value propositions at the empirical domain. Rather
than the causal that occur at the real domain where contextual conditions and mechanisms are
made manifest (Hastings, 2021). With the empirical identification and subsequent quantitative
validation of the factors that constituted the actualized values propositions in the investigative
context and explained further (see appendix B, Table C, D, E,F, G, & H ). We proceed with the
qualitative data analysis by starting with the sustainability of the actualized value propositions.
Qualitative Findings and Results
We discussed the qualitative findings and results by starting with sustainability of the actualized
value propositions followed by the affordances of the implemented e-Government Projects and
lastly the context-mechanism-outcomes
Sustainability of the Actualized Value Propositions
Our empirical qualitative data confirm the sustainability of actualized values. We present
excerpts of a few qualitative analyses/validations in Table 2 to ensure that: (a) Endurance of the
actualized values. (b) Persistence of routinization or institutionalization of the initial e-
Government projects. (c) Continued adaptation within the context the innovation is embedded.
(d) Evolutionary growth through configuration and (e) Maintenance and appropriation with local
capacities over time are factors that reflect the sustainability of the actualized values. Such
factors are identified empirically as what constitutes sustainability of the actualized values.
Table 2: Sustainability of Actualized Value Propositions
Factors Direct Quotes from Participants
Endurance of actualized values “We continue to use the platform components and activities after the
initial funding period for the achievement of our intended outcomes.”
# RWSM
Persistence of routinization or “We put in place structures and processes of innovation as a culture
institutionalization of the initial of practices by staff, the organizations and our systems.”# DCI.
e-Government projects
Continued adaptation within the “Our platform adapts to a distinct possible situation within the context
context of the innovation of the services we deliver.”# ACI.
Evolutionary growth through “We have in place organizational strategies, structures, and processes
configuration to grow and sustain the platform.” # ePM.
Maintenance and appropriation “We rebuilt and launched the platform to be fully managed by local
with local capacities over time staff” # RWM.

22
Besides, the actualized value propositions are the concrete outcome of acting on the affordance
of the implemented e-Government projects. It then behooves us to identify such affordances
helpful in realizing the sustainability of the actualized value propositions as detailed following.
Affordances of the implemented e-Government Projects
e-Government affordances as revealed by the data were value co-creation affordance (Wang et
al., 2016), accessibility affordance (Thapa & Sein, 2018), as well as transparency affordance
(Stamati et al., 2015), and are empirically validated as shown in Table 3.
Table 3: Affordance Interview Extract
Affordance Direct Quotes from Participants
Types
Value Co- “We are giving citizens satisfaction and enhanced general wellbeing through their
creation engagement with our platform by automating the process for citizens to apply for
Affordance their passport at the convenience of their homes and offices with the value co-
creation that automation brings” “We are increasing enrolment, reducing queue,
reducing multiple passport acquisitions” # CSI.
“So, I will say that the reason for implementing Irembo is to give citizens a better
experience and to engage and use the implemented e-Government project to
enhance value co-creation” # RWAM1.
Accessibility “The product team is in charge of understanding the users and designing the
Affordance product, the experience that will match the user’s requirement that incorporates
their peculiarities—such peculiarities in design capture the accessibility function
in using the platform. Irrespective of individual ability, their physical, motoric or
perceptual disabilities enhance ease of use and access to services. And multiple
services accessible via mobile phones, USSD, computers, or a visit to Irembo
service centers and Irembo agents” # RWAM2.
Transparency “And we also increased the security of tender documents of bids from suppliers,
Affordance with the inscription and automating the process so that when the bidder has
submitted his bid, and when the time to open the bid, the system opens it
automatically enhancing transparency” # ePDM1.
“The system opens the bids automatically, and everyone can see what has been
submitted to this tender; they have quoted this much so that by the time it goes to
evaluation, everyone is clear who is the winner. So, there is a lot of value that has
been added” # ePAM.

Under what contextual conditions are generative mechanisms fostering sustainability of the
actualized value propositions as outcomes are detailed next.
Context-Mechanism-Outcomes
As Table 4 could attest, our empirical data confirm the mechanisms are of two types; Individual
and organizational mechanisms. Individual mechanisms focus on citizen’s motivation,
experience, ability, choices, and willingness to engage in the implemented e-Government
projects. Organizational mechanisms on the other hand focus on the total available support
system and resources needed for the success of the implemented e-Government project. The
individual mechanisms are (1) Technological autonomy and cognizance, and (2) Competencye.
23
The organizational mechanisms are (1) Continuous availability of funds, (2) Institutional
administrative and coordinating capacity, (3) Sense of ownership, (4) Strong political support
and leadership, (5) Institutionalized in local settings, and (6) Ongoing monitoring and evaluation.
The contextual conditions are societal, covering economic, institutional, political and
institutional factors and external donor agencies. As Table 4 could attest we mapped the context
and mechanism identified to the actualized value propositions as outcomes validated by the
logistic regression analysis of CVM data.
Table 4: CMOc Interview Extract
Context Mechanism Actualized Value propositions as
Outcomes patterns validated by
the logistic regression analysis of
CVM data
Societal Continuous availability of funds Improved service delivery
-Economic
With the “deployment of the “We have a business model that
platform, it becomes the charges commission on every
central payment collection transaction” # RWAM2.
point for all government
services” # RWDM.
-Social Institutional administrative and Improved service delivery
“Irembo is a One-stop coordinating capacity
platform for all government “We are headed by a D.G.,
Opendata
services” # RWSM. directors of product services,
engineering, developers, and
support team thereby
institutionalizing our capacity to
Social group profile management
run the platform” # RWM.
Building trust between government
Sense of ownership “We have
and citizens
multiple service channels (website,
Status reports of contact tracing,
USSD, agents) in English, isolation, and quarantine
Kinyarwanda, and French making Status reports of surveillance and
the platform to have a sense of response
local ownership” ePDM2. Status reports on diseases and
pandemic

Political Strong political support and

“Part of a larger National
ICT roadmap and Master
Plan being supported by the
political leadership of council
of ministers” # ePDM1.
-Institutional
“The Rwanda Public
Procurement Law was
leadership “We have a dedicated
Minister of ICT who is responsible
for the success of the platform” #
ePSM.
Institutionalized in local settings Improved transparency, accuracy,
“We rebuilt and launched the and facilitating information between
platform to be fully managed by government and citizens

24
 revised in August 2018 (No
62/2018) to incorporate the
e-procurement system.”
# ePDM2.
External-Donor agencies
“World Bank sponsored the
platform.” # ePAM.
local staff” # SI.
Ongoing monitoring and
evaluation
“We have a support team for
feedback through social media
platforms, email, and toll-free call
centers to ensure monitoring and
evaluation.” # RWSM.
Accountability and transparency
Streamlining procedures
Procurement
Improved transparency, accuracy,
and facilitating information between
government and citizens
Source: Data from Contingent
Valuation Questionnaire

Technological autonomy and Source: Data from Contingent

cognizance “Knowing that the Valuation Questionnaire on the
platform is a secured automated rationale for choice under actualize
procurement process gives me the
value and willingness to pay
confidence and convenience to
select which tender to bid for
anywhere, anytime.” # ePM.
Competence Data from Contingent Valuation
“My IT skills and ability enable Questionnaire on the rationale for
easy use of the platform which in choice under actualize value and
turn save time, resources, and to willingness to pay
get an update on news and
information.” # CSI2.

Phase 3: Theory Validation, Refinement, and Consolidation

We subsequently conducted a cross-case comparison of the findings in an abductive manner, as
shown in Figure 4. In doing so, we identify, confirm, and disconfirm cases with the identified
affordances, sustainability concepts, and CMOc

25
Figure 4: Phases 3- Theory Validation, Refinement, and Consolidation
Thence, we related such affordances, sustainability concepts, and CMOc across the issues back
to the IPT for further participants’ validation. Consequently, the validated CMOc, affordances,
and sustainability concepts were synthesized into one model of a RPT, as shown in Figure 5.

Figure 5: Refined Model of Actualized Values Sustainability Framework of Implemented

e-Government Projects in SSA (Source: Authors).

26
The research model, as seen in Figure 5, is the explanatory model developed from the RE
conducted in this study. Being an explanatory model, it could be used to explain the investigative
context comprising the implemented e-Government projects, their actualized values and the
sustainability in Rwanda and Nigeria. The implemented e-Government projects under study in
both nations suggest delivering the value propositions as actualized outcome seen in Figure 5 of
our research model, which in RE is called The RPT. Such actualized value propositions are
quality service delivery, effective public organizations, open government and democratic value
as well as social value and wellbeing.
However, the actualized outcomes (i.e. the value propositions) at the empirical domain cannot
exist without the affordance as the arrow suggest. Affordances being a relational concept exist as
a result of the social and technical relations. The social relations represent the human agency
while the technical/material agencies (e-Government IT artifact) are in interactions for
affordances (i.e. accessibility, transparency, and value creation) to manifest. As the interactions
continues, it recursively yields to actualized value propositions, hence the input and output
arrows linking both affordances and the actualized values.
Now, such interactions could be further facilitated through individual and organizational
mechanisms and enabling contextual conditions for the sustainability of those actualized values.
The mechanisms when particularized to the individuals exist as; technological cognizance and
autonomy as well as competency as drawn from SDT as a motivational mechanisms. In SSA,
extant literature has revealed that e-Government projects have been implemented. But the
citizens are unaware that such services exist in their local context, hence the need for
technological cognizance.
Besides, an informed individual may lack autonomy and competency to engage with the
implemented e-Government project. Therefore, when such individual mechanisms are in place in
our investigative context, we see them to facilitate the affordance of the technology towards its
actualization, hence the arrows that are pointing towards the e-Government affordances and
organizational mechanisms. Apart from the individual mechanisms, there are other mechanisms
sprouting from the activities of the organizations, i.e. public sector that houses the implemented
e-Government projects as collaborated by our data. The organizational mechanisms include; are
(1) Continuous availability of funds, (2) Institutional administrative and coordinating capacity,
(3) Sense of ownership, (4) Strong political support and leadership, (5) Institutionalized in local
settings, and (6) Ongoing monitoring and evaluation.
They are in interactions with individual mechanisms to facilitate the affordances and
sustainability of the actualized values, hence the back and forth arrows emanating from
organizational mechanisms pointing towards individual mechanisms and actualized values
yielding to the sustainability of the projects in SSA.
Sustainability on its own is seen here in the investigative context as multidimensional interaction
in a recursive manners, such dimensions includes; (a) Endurance of the actualized values. (b)
Persistence of routinization or institutionalization of the initial e-Government projects. (c)
Continued adaptation within the context the innovation is embedded. (d) Evolutionary growth

27
through configuration and (e) Maintenance and appropriation with local capacities over. These
dimensions in operations influence the organizational mechanisms in a dynamic version as the e-
Government implemented projects unfold in the investigative context.
Apart from these internal mechanisms, they are other external mechanisms seen as enabling
contextual conditions in this study. They are in constant interactions with internal mechanisms to
trigger the actualized value propositions and the sustainability of the technology while in use.
Such contextual conditions are political, economic, social, cultural and institutional in the form
of the activities of the donor agencies such World Bank and AfDB (i.e. funding, provision of
expertise skills, and panoptic surveillance). These enabling contextual conditions in interactions
with the identified individual, organizational mechanisms facilitate the affordance of the
technology. Furthermore,, its actualizations, and the sustainability of the implemented e-
Government projects in Nigeria and Rwanda.
When placed on par with our IPT, these mechanisms identified in our final model suggest that
mechanisms that includes independence from external assistance, continuously meeting the
stakeholders' needs and relatedness, were eliminated due to insufficient empirical data to support
them. We eliminated the dialogue and mobility affordances because of inadequate data to keep
them in our refined model empirically. Such an elimination resonates with the abductive
reasoning, where theoretical concepts from theories-in-use in this study that are not near the data
overtime are filtered out (Dubois & Gadde, 2002; Ononiwu et al., 2018).
Mechanism can exist as tendencies (i.e., have the capability to be causal but not yet active); they
can also exist as causal powers (i.e., they are active). Both mechanisms exist in concerted
manner in the investigative context under study. Tendencies can be dormant pending when
enabling contextual conditions trigger them into operation. Even though they are being triggered,
they may not be actualized at the empirical domain in space and in time. At the time of our data
collection, the filtered mechanisms might be dormant or triggered by enabling conditions but not
actualized or were partially actualized.
Consequently, enough empirical evidence was not witnessed at the time of the field work, and
hence, they were dropped. At the time of data collection, independence from donor assistance
does hold because most of the projects were still being sponsored by the donor agencies.
Besides, continuously meeting stakeholder’s needs attenuates due to the project being in the last
lap of experimental phase with slight reconfiguration still going; therefore stakeholder’s needs
are not continuously met.
Equally, relatedness, dialogue, and mobility affordances are mostly social and relational
mechanisms that were visible in early stage of the implementation. However, when the
implemented e-Government projects in both countries (i.e., Rwanda and Nigeria) stabilized. The
dialogue among stakeholders wanes off, while the mobile interface of the technology is not yet in
operation or switched on, even though it is part of the successful project implementations in both
nations.

28
DISCUSSION
At the onset of this study, we set out to identify the value propositions of implemented e-
Government projects and how such values can be actualized while in use by the users.
Furthermore, we try to understand the mechanisms and contextual conditions fostering the
sustainability of the actualized value propositions of implemented e-Government projects.
Through a systematic literature review approach, our findings revealed that the value
propositions of implemented e-Government projects are: (a) Quality service delivery, as
corroborated by scholars (e.g., Bannister & Connolly, 2014; Castelnovo, 2013). (b) Effective
public organization (Heeks & Bailur, 2007). (c) Open government and democratic values (Deng
et al., 2018; Harrison et al., 2012), as well as (d) Social value and general well-being (Grimsley
& Meehan, 2008; Karunasena & Deng, 2012).
The actualization of such value propositions emerged through affordance actualization of the
implemented e-Government projects. Such actualization was made manifest by the citizens'
competencies in the technology as a unique individual mechanism necessary for the
actualization. Equally, citizens are equipped with technological autonomy and cognizance within
the e-Government environment while using the platforms as an individual mechanism. Besides,
organizational mechanisms endured with continuous availability of funds by the host
government. In addition to institutional administrative and coordinating capacity with a strong
leadership and political support from the government in power was revealed as organizational
mechanisms. Other organizational mechanisms include institutionalizing the technology within
local settings with its ongoing monitoring and evaluation to foster improvements.
Consequently, we empirically validated that the sustainability of the actualized values takes the
shape of endurance of the identified actualized values. It also includes persistence of
institutionalization of the e-Government projects, and continued adaptation within the context the
innovation is embedded. Other sustainability shapes are evolutionary growth through
configuration and maintenance and appropriation with local capacities over time. While
contextual conditions revealed are the external donor agencies such as EU, AfDB, World Bank
continuous support to the host country. Others include the enactment of institutional policies,
political and socio-economic enablement. We captured all these and consolidated them into our
research model shown in Figure 3 that we classified as the actualized values sustainability
framework of implemented e-Government projects in SSA.
An interesting fact is the application of RE in our study, which is still nascent in e-Government
evaluation research, making this study a valuable contribution to this area. The ultimate goal of
using RE is to produce a refined middle-range theory (MRT) of how interventions work by
identifying common patterns within reality. Our MRT of actualized value sustainability
framework of implemented e-Government projects in SSA is a refinement of our IPT. We
empirically tested this through multiple cases culminating into a contextually specific relevant
policy theory. Additionally, such theory describes the explanatory pathway of how change
occurs through citizens' actualization of implemented e-Government projects in SSA and the
mechanisms fostering its sustainability.

29
In contrast, other scholars have conducted different evaluations in e-Government field, but none
was able to itemize the value propositions of implemented e-Government projects as we did. In
addition, they fail to highlight the actualization process and the mechanisms fostering its
sustainability. IS Scholars (e.g.,Bannister & Connolly, 2015) examines the validity of long- held
beliefs and claim that e-government is under-theorized. They explore the counter-argument that,
far from being short of theory, a great deal of excellent and valuable theory can be found in the
e-government literature. But the study did not go further to develop a framework or model of any
kind as a contribution to theory and practice.

Alshawi and Alalwany (2009) developed an evaluation criterion for an effective, adaptable, and
reflective assessment of e-government systems from the citizen's perspective. The evaluation
criterion was due to developing countries poor citizen’s utilization of e-government initiatives.
Moreover, the criteria can be a means for providing valuable feedback for the planning of future
e-government initiatives. Nevertheless, the study fails to capture the value propositions of
implemented e-Government projects; neither did it specify the actualization of the implemented
e-Government projects and their sustainability.

Adu et al. (2018) also examine the implementation progress of e-government in Ghana where
quantitative surveys were conducted in 182 government agencies. Findings showed that public
sector organizations are still plagued with infrastructural, economic and legal challenges and
human resources in the development of e-government. It concluded with a recommendation for
the expansion of the use of online services by adopting a more citizen-centric approach to
promote e-government planning and implementation in infrastructure, law, human capacity.
However, the study did not come up with a framework or model that reflects the citizen-centric
approach.
We believe our study is an essential contribution to the growing literature on e-Government
evaluation, RE, and ICT4D research. It is specifically helpful because our research has the
potential to open up a plethora of approaches available to e-Government evaluation. At the same
time it encourages the use of RE methodology in studying other aspects of e-Government
implementations.
Overall, the theoretical contribution of our study is the development of an actualized value
sustainability framework for implemented e-Government projects in SSA which will benefit
both academics and practice. By developing the theoretical framework from RE, researchers
adopting such a framework can identify e-Government affordances, citizens' motivations and
capabilities as mechanisms to actualize such affordances lacking in the literature. Besides, the
identified mechanisms fostering the sustainability of the actualized value propositions of the
implemented e-Government projects will be explicated.
Furthermore, the framework will allow future studies to engage in the evaluation of the
operational effectiveness (i.e., what works) of the implemented e-Government projects. In
addition to the actualized value propositions derived from such projects to generalize it to other
contexts. In practice, the outcomes of our study present a great potential for translation and

30
transfer of our research outcome into policy and practice. While the findings are well specified to
further improvements on other implemented e-Government projects in SSA.
The study leading to the emergence of the framework will also : (1) build up citizens'
satisfaction, and public trust about government e-service delivery (Agbabiaka, 2018), (2) meet
the demands for external/donor accountability, and (3) establish a clear, strategic goal for the
organization. Other benefits to practitioners by the use of the framework are: (4) identifying the
relevant value propositions from the citizen's perspective to foster a strong sense of operational
accountability and how it could be sustained (Agbabiaka, 2018), and lastly (5) to increase
organizational performance for national development and well-being of the citizens of the host
nations (Panagiotopoulos et al., 2019).

CONCLUSION AND FUTURE RESEARCH DIRECTIONS

Our study led us to realize that the actualization of the value proposition of implemented e-
Government projects and their sustainability is critical in SSA. It is critical for donor agencies’
continuous support and accountability due to a high rate of e-Government implementations
failure in SSA. Citizens need to engage, appropriate and exploit the implemented e-Government
projects. In doing this, they fulfill their desired goal in their respective contexts and build up
confidence and trust for their individual implemented e-Government projects. Besides, other
stakeholders such as businesses must continuously use the e-Government projects to achieve the
cardinal objectives of their implementations.
Consequently, by developing the actualized values sustainability framework of implemented e-
Government projects, we have unraveled the actualized values and citizens' motivation to use the
platforms. Additionally, we unravel the generative mechanisms fostering the sustainability of the
value propositions to shape innovation continuation at both individual and organizational levels.
We deployed Mukumbang et al. (2016) modified RE methodology that resonated with multiple
theories; and understood the e-Government affordances and its actualized outcomes.
Furthermore, the enabling contexts, the concerted mechanisms in interaction to foster the
sustainability of the actualized value propositions as outcomes. Although, to the best of our
knowledge, less or nothing is known about how RE evaluation aligned with implemented e-
Government projects in SSA, suggesting the study's novelty.
There are questionable uses of evaluation to assess IT interventions, especially in e-health
systems, which have come to include ritual use. Nevertheless, such an evaluation shows the
effectiveness of the technology while-in-use, this can be extended to e-Government projects.
However, our RE here takes it further to contribute to decision-making and accountability for
implementers of e-Government projects in SSA and their respective donor agencies. It also
highlights the legitimacy of actions taken, knowledge of the quality and results of value
propositions of public services in SSA that adopted the technology.
Consequently, our RPT developed in this study could be generalized to other SSA countries that
are not part of the investigative contexts. It is needed to increase our understanding, utility, and

31
validity of the developed theory in future studies. Besides, every evaluation theory has
underlying assumptions of how such evaluation is supposed to change social programs and
contribute to social change, but our research is limited in this area. Therefore, future research
should clarify how RE shapes certain aspects of an evaluation context beyond their formal
intentions. As put forth by Nordesjö (2021, p. 210), ". . . evaluations and their impact may
exceed their formal intentions and must be understood in a social, organizational and political
context [of its use]”. Other future research should be to investigate and unraveled ‘what must
have been the case’ (Pawson, 2006; Pawson & Tilley, 1997) for lack of empirical support for
relatedness, mobility, and dialogue affordances in the investigative context to be possible.

LIMITATIONS
Despite the theoretical contribution, our approach has some weaknesses as well as benefits.
Some of the visible weaknesses we identified in our study are that; RE approach using a mixed
methods design is relatively new in e-Government studies and ICT4D. Being nascent and
lacking familiarity with using the RE and mixed methods in the field. The study design will
draw criticism for being complex and dense, particularly concerning multiple theoretical
applications. Data extraction is also complex and may bring about biases because of undesirable
judgments. Such judgments are at individual participants who might not open up vital
information about our research in interviews. Seemingly, there is difficulty on how to formulate
and ‘‘unpack’’ CMOc in researches/investigations. Nonetheless, we attempt to integrate our data
at multiple levels to connect and merge the methodology and findings as simple and effective as
possible.

REFERENCE
Abaku, T., Calzati, S., & Masso, A. (2021). Exploring digital sustainability of/through Estonia’s e-
residency: Africa’s case and the importance of culture for sustainability. Digital Policy,
Regulation and Governance, 23(3), 300-313. https://doi.org/10.1108/DPRG-10-2020-0153
Abildgaard, J. S., Nielsen, K., & Sverke, M. (2018). Can job insecurity be managed? Evaluating an
organizational-level intervention addressing the negative effects of restructuring. Work & Stress,
32(2), 105-123.
Abildgaard, J. S., Nielsen, K., Wåhlin-Jacobsen, C. D., Maltesen, T., Christensen, K. B., & Holtermann, A.
(2019). ‘Same, but different’–a mixed methods realist evaluation of a cluster randomized
controlled participatory intervention. Human Relations, 73(10), 1339-1365.
Adu, K. K., Patrick, N., Park, E. G., & Adjei, E. (2018). Evaluation of the implementation of electronic
government in Ghana. Information polity, 23(1), 81-94.
Agbabiaka, O. (2018). The public value creation of eGovernment: An empirical study from citizen
perspective. Proceedings of the 11th International Conference on Theory and Practice of
Electronic Governance,
Agency Reporter. (2022). What Nigerian passport applicants went through in 2021.
https://thenationonlineng.net/what-nigerian-passport-applicants-went-through-in-2021/

32
Akaka, M. A., Vargo, S. L., & Wieland, H. (2017). Extending the context of innovation: The co-creation
and institutionalization of technology and markets. In T. Russo-Spena, Mele, C. & M. Nuutinen
(Eds.), Innovating in Practice (pp. 43-57). Springer, Cham. https://doi.org/10.1007/978-3-319-
43380-6_3
Aksulu, A., & Wade, M. R. (2010). A comprehensive review and synthesis of open source research.
Journal of the Association for Information Systems, 11(11), 6.
Al-Ghamdi, A. S. (2002). Using logistic regression to estimate the influence of accident factors on
accident severity. Accident Analysis & Prevention, 34(6), 729-741.
Aladwani, A. M. (2016). Corruption as a source of e-Government projects failure in developing countries:
A theoretical exposition. International Journal of Information Management, 36(1), 105-112.
Aljarallah, S., & Lock, R. (2020). An investigation into sustainable e-Government in Saudi Arabia.
Electronic Journal of e-government, 18(1), 1-16.
Alshawi, S., & Alalwany, H. (2009). E-government evaluation: Citizen's perspective in developing
countries. Information Technology for Development, 15(3), 193-208.
https://doi.org/10.1002/itdj.20125
Anderson, C., & Robey, D. (2017). Affordance potency: Explaining the actualization of technology
affordances. Information and Organization, 27(2), 100-115.
Anderson, L., Ostrom, A. L., Corus, C., Fisk, R. P., Gallan, A. S., Giraldo, M., Mende, M., Mulder, M.,
Rayburn, S. W., & Rosenbaum, M. S. (2013). Transformative service research: An agenda for the
future. Journal of Business Research, 66(8), 1203-1210.
Andrikopoulou, E., Scott, P. J., & Herrera, H. (2020). Mixed methods protocol for a realist evaluation of
electronic personal health records design features and use to support medication adherence
(ePHRma). BMJ health & care informatics, 27(1).
Anthopoulos, L., Reddick, C. G., Giannakidou, I., & Mavridis, N. (2016). Why e-government projects fail?
An analysis of the Healthcare. gov website. Government information quarterly, 33(1), 161-173.
Avdic, A., & Lambrinos, T. (2015). Modeling and illustrating requirement prioritization in public e-service
development from a value-based perspective. Electronic Journal of E-Government, ECEG, 13(1),
1-15.
Bakunzibake, P. (2016, 5-8 September). Towards Organizational Transformation in Developing Countries:
Enterprise Content Management in Rwanda. Electronic Government and Electronic
Participation: Joint Proceedings of Ongoing Research and Projects of IFIP WG 8.5 EGOV and
ePart 2016, Guimarães, Portugal.
Bakunzibake, P., Klein, G. O., & Islam, S. M. (2019). E-government implementation process in Rwanda:
Exploring changes in a sociotechnical perspective. Business Systems Research: International
journal of the Society for Advancing Innovation and Research in Economy, 10(1), 53-73.
Bakunzibake, P., Klein, G. O., & Islam, S. M. (2019). E‐government implementation and monitoring: The
case of Rwanda's ‘one‐stop’ E‐government. The Electronic Journal of Information Systems in
Developing Countries, 85(5). https://doi.org/10.1002/isd2.12086
Bandara, W., Furtmueller, E., Gorbacheva, E., Miskon, S., & Beekhuyzen, J. (2015). Achieving rigor in
literature reviews: Insights from qualitative data analysis and tool-support. Communications of
the Association for Information Systems, 37(1), 8. https://doi.org/10.17705/1CAIS.03708
Bannister, F., & Connolly, R. (2014). ICT, public values and transformative government: A framework and
programme for research. Government information quarterly, 31(1), 119-128.
Bannister, F., & Connolly, R. (2015). The great theory hunt: Does e-government really have a problem?
Government information quarterly, 32(1), 1-11. https://doi.org/10.1016/j.giq.2014.10.003
Baxter, P., & Jack, S. (2008). The qualitative report qualitative case study methodology: Study design and
implementation for novice researchers. The Qualitative Report, 13 (4), 544–559.
https://doi.org/10.46743/2160-3715/ 2008.1573

33
Benitez, J., Henseler, J., Castillo, A., & Schuberth, F. (2020). How to perform and report an impactful
analysis using partial least squares: Guidelines for confirmatory and explanatory IS research.
Information & Management, 57(2), 103168.
https://doi.org/https://doi.org/10.1016/j.im.2019.05.003
Benitez, J., Ray, G., & Henseler, J. (2018). Impact of information technology infrastructure flexibility on
mergers and acquisitions. MIS quarterly, 42(1), 25-44.
Bollen, K. A., & Bauldry, S. (2011). Three Cs in measurement models: causal indicators, composite
indicators, and covariates. Psychological methods, 16(3), 265.
Bosco Harelimana, J. (2018). The Impact of E-Procurement on the Performance of Public Institutions in
Rwanda. Global Journal of Management And Business Research.
https://journalofbusiness.org/index.php/GJMBR/article/view/2475
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology,
3(2), 77-101.
Braun, V., & Clarke, V. (2019). To saturate or not to saturate? Questioning data saturation as a useful
concept for thematic analysis and sample-size rationales. Qualitative Research in Sport, Exercise
and Health, 13(2), 201-216.
Buchanan, D. A., Fitzgerald, L., & Ketley, D. (2006). The sustainability and spread of organizational
change: modernizing healthcare. Routledge.
Capaldi, E., & Proctor, R. W. (2008). Are theories to be evaluated in isolation or relative to alternatives?
An abductive view. The American journal of psychology, 121(4), 617-641.
Castelnovo, W. (2013). A country level evaluation of the impact of e-government: the case of Italy. In E-
Government Success around the World: Cases, Empirical Studies, and Practical
Recommendations (pp. 299-320). IGI Global.
Ceesay, L. B., & Bojang, M. B. S. (2020). Embracing e-Government during the covid-19 pandemic and
beyond: Insights from the Gambia. Global Journal of Management And Business Research.
Cha, J. H., Lee, J., Kim, T., & Chang, K. (2019). Value of college athletic programmes in the Korean Higher
Education: A contingent valuation study. Higher Education Quarterly, 74(3), 320-333.
Cha, J. H., Lee, J., Kim, T., & Chang, K. (2020). Value of college athletic programmes in the Korean Higher
Education: A contingent valuation study. Higher Education Quarterly, 74(3), 320-333.
Chircu, A. M. (2008). E-government evaluation: towards a multidimensional framework. Electronic
Government, an International Journal, 5(4), 345-363.
Choi, H., Park, M. J., & Rho, J. J. (2017). Two-dimensional approach to governmental excellence for
human development in developing countries: Combining policies and institutions with e-
government. Government information quarterly, 34(2), 340-353.
Choudrie, J., Zamani, E. D., Umeoji, E., & Emmanuel, A. (2017). Implementing E-government in Lagos
State: Understanding the impact of cultural perceptions and working practices. Government
information quarterly, 34(4), 646-657. http://search.ebscohost.com/login.aspx?
direct=true&db=lxh&AN=126514380&site=ehost-live
Chung, H. (2008). The contingent valuation method in public libraries. Journal of Librarianship and
Information Science, 40(2), 71-80.
Connelly, J. (2001). Critical realism and health promotion: effective practice needs an effective theory.
Health Education Research, 16(2), 115-120.
Cram, W. A., Proudfoot, J. G., & D’arcy, J. (2017). Organizational information security policies: a review
and research framework. European Journal of Information Systems, 26(6), 605-641.
Dalkin, S. M., Lhussier, M., Philipson, P., Jones, D., & Cunningham, W. (2016). Reducing inequalities in
care for patients with non-malignant diseases: insights from a realist evaluation of an integrated
palliative care pathway. Palliative medicine, 30(7), 690-697.

34
de Caluwe, C., Verdegem, P., & Van Dooren, W. (2012). The digital divide and/in e-government: a catch-
22? Tijdschrift Voor Communicatiewetenschap, 40(1), 46-+. 000301516400004
Deci, E. L., & Ryan, R. M. (1985). The general causality orientations scale: Self-determination in
personality. Journal of research in personality, 19(2), 109-134.
Deng, H., Karunasena, K., & Xu, W. (2018). Evaluating the performance of e-government in developing
countries. Internet Research, 28(1), 169-190. https://doi.org/10.1108/IntR-10-2016-0296
DeSanctis, G., & Poole, M. S. (1994). Capturing the complexity in advanced technology use: Adaptive
structuration theory. Organization science, 5(2), 121-147.
Dini, A. A., Sæbo, Ø., & Wahid, F. (2018). Affordances and effects of introducing social media within
eParticipation—Findings from government‐initiated Indonesian project. The Electronic Journal of
Information Systems in Developing Countries, 84(4).
Dremel, C., Herterich, M. M., Wulf, J., & Vom Brocke, J. (2020). Actualizing big data analytics
affordances: A revelatory case study. Information & Management, 57(1), 103121.
Dubois, A., & Gadde, L.-E. (2002). Systematic combining: an abductive approach to case research.
Journal of Business Research, 55(7), 553-560.
Eastwood, J. G., Jalaludin, B. B., & Kemp, L. A. (2014). Realist explanatory theory building method for
social epidemiology: a protocol for a mixed method multilevel study of neighbourhood context
and postnatal depression. SpringerPlus, 3, 12-12. https://doi.org/10.1186/2193-1801-3-12
Ebenso, B., Huque, R., Azdi, Z., Elsey, H., Nasreen, S., & Mirzoev, T. (2017). Protocol for a mixed-methods
realist evaluation of a health service user feedback system in Bangladesh. BMJ open, 7(6),
e017743.
Emmel, N., Greenhalgh, J., Manzano, A., Monaghan, M., & Dalkin, S. M. (2018). Doing realist research.
Sage.
Evanciew, C., & Rojewski, J. W. (1999). Skill and knowledge acquistion in workplace: A case study of
mentor-apprentice relationships in youth apprenticeship programs. Journal of STEM Teacher
Education, 36(2), 3.
Flak, L. S., & Rose, J. (2005). Stakeholder governance: Adapting stakeholder theory to e-government.
Communications of the Association for Information Systems, 16(1), 31.
Fleiszer, A. R., Semenic, S. E., Ritchie, J. A., Richer, M. C., & Denis, J. L. (2015). The sustainability of
healthcare innovations: a concept analysis. Journal of advanced nursing, 71(7), 1484-1498.
Fletcher, A., Jamal, F., Moore, G., Evans, R. E., Murphy, S., & Bonell, C. (2016). Realist complex
intervention science: applying realist principles across all phases of the Medical Research
Council framework for developing and evaluating complex interventions. Evaluation, 22(3), 286-
303.
Fung, A. (2015). Putting the public back into governance: The challenges of citizen participation and its
future. Public Administration Review, 75(4), 513-522.
Gibson, J. J. (1979). "The theory of affordances" The ecological approach to visual perception. Lawrence
Erlbaum Associates Inc.
Giesbrecht, T., Schenk, B., & Schwabe, G. (2015). Empowering front office employees with counseling
affordances. Transforming Government: People, Process and Policy, 9(4), 517-544.
Gilmore, B., Adams, B. J., Bartoloni, A., Alhaydar, B., McAuliffe, E., Raven, J., Taegtmeyer, M., & Vallières,
F. (2016). Improving the performance of community health workers in humanitarian
emergencies: a realist evaluation protocol for the PIECES programme. BMJ open, 6(8), e011753.
Gilmore, B., McAuliffe, E., Power, J., & Vallières, F. (2019). Data analysis and synthesis within a realist
evaluation: toward more transparent methodological approaches. International Journal of
Qualitative Methods, 18, 1609406919859754.
Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2013). Seeking qualitative rigor in inductive research: Notes
on the Gioia methodology. Organizational research methods, 16(1), 15-31.

35
Goodhue, D. L., & Thompson, R. L. (1995). Task-technology fit and individual performance. MIS
quarterly, 213-236.
Greenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter,
R., & Shaw, S. (2017). Beyond Adoption: A New Framework for Theorizing and Evaluating
Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of
Health and Care Technologies. J Med Internet Res, 19(11), e367.
https://doi.org/10.2196/jmir.8775
Grimsley, M., & Meehan, A. (2008). Gaining the world and losing the soul? trust change in electronic
government. European Conference on Information System (ECIS) 2008 Proceedings,
Haig, B. D. (2005). Exploratory Factor Analysis, Theory Generation, and Scientific Method. Multivariate
Behav Res, 40(3), 303-329. https://doi.org/10.1207/s15327906mbr4003_2
Harelimana, J. B. (2018). The impact of E-procurement on the performance of public institutions in
Rwanda. Global Journal of Management And Business Research.
Harrison, T. M., Guerrero, S., Burke, B., Cook, M., Cresswell, A., Helbig, N., Hrdinová, J., & Pardo, T.
(2012). Open government and e-government: Democratic challenges from a public value
perspective. Information polity, 17(2), 83-97.
Hastings, C. (2021). A critical realist methodology in empirical research: foundations, process, and
payoffs. journal of critical realism, 20(5), 458-473.
Heeks, R. (2002). e-Government in Africa: Promise and practice. Information Polity: The International
Journal of Government & Democracy in the Information Age, 7(2,3), 97-114.
Heeks, R. (2003). Most eGovernment-for-development projects fail: how can risks be reduced?
iGovernment Working Paper.
Heeks, R. (2005). e-Government as a carrier of context. Journal of Public Policy, 51-74.
Heeks, R., & Bailur, S. (2007). Analyzing e-government research: Perspectives, philosophies, theories,
methods, and practice. Government information quarterly, 24(2), 243-265.
Henseler, J. (2017). Bridging Design and Behavioral Research With Variance-Based Structural Equation
Modeling. Journal of Advertising, 46(1), 178-192.
https://doi.org/10.1080/00913367.2017.1281780
Henseler, J., & Schuberth, F. (2020). Using confirmatory composite analysis to assess emergent variables
in business research. Journal of Business Research, 120, 147-156.
Igihozo, G., Henley, P., Ruckert, A., Karangwa, C., Habimana, R., Manishimwe, R., Ishema, L., Carabin, H.,
Wiktorowicz, M. E., & Labonte, R. (2022). An environmental scan of one health preparedness
and response: the case of the Covid-19 pandemic in Rwanda. One Health Outlook, 4(1), 2.
https://doi.org/10.1186/s42522-021-00059-2
Inuwa, I., & Ononiwu, C. G. (2020). Traditional and information technology anti-corruption strategies for
curbing the public sector corruption in developing economies of sub-Saharan Africa: A
systematic literature review. The African Journal of Information Systems, 12(2), 5.
Iosifides, T. (2016). Qualitative methods in migration studies: A critical realist perspective. Routledge.
Jaeger, P. T., & Bertot, J. C. (2011). Responsibility rolls down: Public libraries and the social and policy
obligations of ensuring access to e-government and government information. Public Library
Quarterly, 30(2), 91-116.
Jagosh, J., Macaulay, A. C., Pluye, P., Salsberg, J., Bush, P. L., Henderson, J., Sirett, E., Wong, G., Cargo,
M., Herbert, C. P., Seifer, S. D., Green, L. W., & Greenhalgh, T. (2012). Uncovering the benefits of
participatory research: implications of a realist review for health research and practice. The
Milbank quarterly, 90(2), 311-346. https://doi.org/10.1111/j.1468-0009.2012.00665.x
James, H. M., Papoutsi, C., Wherton, J., Greenhalgh, T., & Shaw, S. E. (2021). Spread, Scale-up, and
Sustainability of Video Consulting in Health Care: Systematic Review and Synthesis Guided by
the NASSS Framework. J Med Internet Res, 23(1), e23775. https://doi.org/10.2196/23775

36
Karunasena, K., & Deng, H. (2012). Critical factors for evaluating the public value of e-government in Sri
Lanka. Government information quarterly, 29(1), 76-84.
Klischewski, R., & Lessa, L. (2013). Sustainability of e-government success: An integrated research
agenda. In E-Government Success Factors and Measures: Theories, Concepts, and Methodologies
(pp. 104-123). IGI Global.
Labrique, A. B., Wadhwani, C., Williams, K. A., Lamptey, P., Hesp, C., Luk, R., & Aerts, A. (2018). Best
practices in scaling digital health in low and middle income countries. Globalization and health,
14(1), 1-8.
Lawan, B. M., Ajadi, I. A., Kayode, A. A., & Yaru, U. A. (2020). E-government and public service delivery in
Nigeria. Journal of Social Science and Humanities, 17(5), 1-14.
Lawson, T. (1997). Economics and reality. Routledge.
Layder, D. (1998). Sociological practice: Linking theory and social research. Sage.
Lee, O.-K. D., Lim, K. H., & Wong, W. M. (2005). Why employees do non-work-related computing: an
exploratory investigation through multiple theoretical perspectives. Proceedings of the 38th
Annual Hawaii International Conference on System Sciences,
Lessa, L., Anteneh, S., Klischewski, R., & Belachew, M. (2015). Towards a conceptual framework for
pledging sustainable e-Government success: The case of G2G in Ethiopia. AFRICON 2015
Lincoln, Y. S., & Guba, E. G. (1985). Establishing trustworthiness. In Naturalistic inquiry (Vol. 289, pp.
289-327).
Madsen, C. Ø., Berger, J. B., & Phythian, M. (2014). The Development in Leading e-Government Articles
2001-2010: Definitions, Perspectives, Scope, Research Philosophies, Methods and
Recommendations: An Update of Heeks and Bailur.Lecture Notes in Computer Science Electronic
Government: Proceedings of the 13th IFIP WG 8.5 International Conference, EGOV 2014, Dublin,
Ireland.
Majchrzak, A., & Markus, M. L. (2012). Technology affordances and constraints in management
information systems (MIS). Encyclopedia of Management Theory,(Ed: E. Kessler), Sage
Publications, Forthcoming.
Manzano, A. (2016). The craft of interviewing in realist evaluation. Evaluation, 22(3), 342-360.
Marchal, B., Van Belle, S., Van Olmen, J., Hoerée, T., & Kegels, G. (2012). Is realist evaluation keeping its
promise? A review of published empirical studies in the field of health systems research.
Evaluation, 18(2), 192-212.
Marchal, B., Van Belle, S., Westhorp, G., & Peersman, G. (2022). Realist Evaluation. Retrieved 14-02-
2022 from https://www.betterevaluation.org/en/approach/realist_evaluation
Markus, M. L., & Silver, M. S. (2008). A foundation for the study of IT effects: A new look at DeSanctis
and Poole's concepts of structural features and spirit. Journal of the Association for Information
Systems, 9(10), 5.
Marshall, C., & Rossman, G. B. (2014). Designing Qualitative Research. Sage Publications.
McEvoy, P., & Richards, D. (2006). A critical realist rationale for using a combination of quantitative and
qualitative methods. Journal of research in nursing, 11(1), 66-78.
McGaughey, J., O'Halloran, P., Porter, S., Trinder, J., & Blackwood, B. (2017). Early warning systems and
rapid response to the deteriorating patient in hospital: a realist evaluation. Journal of advanced
nursing, 73(12), 3119-3132.
Meyer, S. B., & Lunnay, B. (2013). The application of abductive and retroductive inference for the design
and analysis of theory-driven sociological research. Sociological research online, 18(1), 86-96.
Miller, J. H., & Page, S. E. (2007). Social science in between, from complex adaptive systems: An
introduction to computational models of social life. Princeton University Press.

37
Mingers, J. (2006). A critique of statistical modelling in management science from a critical realist
perspective: its role within multimethodology. Journal of the Operational Research Society,
57(2), 202-219.
Modell, S., Morris, R., & Scapens, B. (2007). Mixing qualitative and quantitative methods in management
accounting research: A critical realist approach. Social Science Research Network, http://papers.
ssrn. com/sol3/papers. cfm.
Mtebe, J. S., & Sausi, J. (2021). Revolutionization of revenue collection with Government E-payment
gateway system in Tanzania: A public value creation perspective. East African Journal of Science,
Technology and Innovation, 2(3).
Mukamurenzi, S., Grönlund, Å., & Islam, S. M. (2019). Improving qualities of e‐government services in
Rwanda: A service provider perspective. The Electronic Journal of Information Systems in
Developing Countries, 85(5), e12089.
Mukumbang, F. C., Marchal, B., Van Belle, S., & van Wyk, B. (2020). Using the realist interview approach
to maintain theoretical awareness in realist studies. Qualitative Research, 20(4), 485-515.
Mukumbang, F. C., Van Belle, S., Marchal, B., & Van Wyk, B. (2016). Realist evaluation of the
antiretroviral treatment adherence club programme in selected primary healthcare facilities in
the metropolitan area of Western Cape Province, South Africa: a study protocol. BMJ open, 6(4),
e009977.
Næss, P. (2015). Critical realism, urban planning and urban research. European Planning Studies, 23(6),
1228-1244.
Nan, W., Zhu, X., & M., L. M. (2020). What we know and don't know about the socioeconomic impacts of
mobile money in Sub‐Saharan Africa: A systematic literature review. The Electronic Journal of
Information Systems in Developing Countries, e12155.
Ngulube, P. (2012). "Ghosts in our machines": preserving public digital information for the sustenance of
electronic government in sub-Saharan Africa. Mousaion, 30(2), 128-136.
Nocera, S., Telser, H., & Bonato, D. (2003). The contingent valuation method in health care: An economic
evaluation of alzheimer's disease (Vol. 8). Springer Science & Business Media.
Nordesjö, K. (2021). The constitutive effects of social investment evaluation. Evaluation, 27(2), 210-228.
Nurjono, M., Shrestha, P., Lee, A., Lim, X. Y., Shiraz, F., Tan, S., Wong, S. H., Foo, K. M., Wee, T., Toh, S.-
A., Yoong, J., & Maria Vrijhoef, H. J. (2018). Realist evaluation of a complex integrated care
programme: protocol for a mixed methods study. BMJ open, 8(3), e017111.
https://doi.org/10.1136/bmjopen-2017-017111
Ojo, A., Mellouli, S., & Ahmadi Zeleti, F. (2019, 18-20 June). A Realist Perspective on AI-era Public
Management*.dg.o 2019 20th Annual International Conference on Digital Government Research
(dg.o 2019), Dubai, UAE.
Okoli, C. (2015). A guide to conducting a standalone systematic literature review. Communications of the
Association for Information Systems, 37(1), 43.
Okunola, O. M., & Rowley, J. (2019). User experience of e-government: the Nigeria Immigration Service.
Library Hi Tech, 37(3), 355-373.
Olafusi, E. (2021). The federal government has launched an enhanced e-passport to improve application
process for Nigerians in the country and abroad. Retrieved 13-02-2022 from
https://www.thecable.ng/fg-launches-e-passport-to-curb-forgery-improve-application-process
Olsen, W. (2010). Editor's Introduction: Realist Methodology : A Review Series. . In Realist Methodology :
Benchmarks in Social Research Methods (pp. xix-xlvi.). Sage Publications.
Omwono, G. A., Mayanja, S. N., & Rodrigue, K. (2020). Factors influencing e-Procurement on the
implementation of public institution in Rwanda: A case of Kigali City (2014-2016). Randwick
International of Social Science, 1(2), 277-293.

38
Ondego, B., & Moturi, C. (2016). Evaluation of the Implementation of the e-Citizen in Kenya. Evaluation,
10(4).
Ononiwu, C. G., Brown, I., & Carlsson, S. (2018). Theory choice In critical realist information systems
research. UK Academy for Information Systems (UKAIS) Conference Proceedings 2018. 8.,
Opoku, D., Busse, R., & Quentin, W. (2019). Achieving Sustainability and Scale-Up of Mobile Health
Noncommunicable Disease Interventions in Sub-Saharan Africa: Views of Policy Makers in
Ghana. JMIR Mhealth Uhealth, 7(5), e11497. https://doi.org/10.2196/11497
Panagiotopoulos, P., Klievink, B., & Cordella, A. (2019). Public value creation in digital government.
Government information quarterly, 36(4). https://doi.org/10.1016/j.giq.2019.101421
Pang, M. S., Lee, G., & DeLone, W. H. (2014). IT resources, organizational capabilities, and value creation
in public-sector organizations: a public-value management perspective. Journal of Information
Technology, 29(3), 187-205.
Paré, G., Trudel, M. C., Jaana, M., & Kitsiou, S. (2015). Synthesizing information systems knowledge: A
typology of literature reviews. Information Management, 52(2), 183-199.
Pawson, R. (2006). Realist methodology: the building blocks of evidence.
Pawson, R. (2013). The science of evaluation: A realist manifesto. Sage.
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005). Realist review-A new method of systematic
review designed for complex policy interventions. Journal of health services research & policy,
10(1_suppl), 21-34.
Pawson, R., & Tilley, N. (1997). Realistic evaluation. Sage.
Pawson, R., & Tilley, N. (2004). Realist evaluation. British Cabinet Office.
Polites, G. L., Roberts, N., & Thatcher, J. (2012). Conceptualizing models using multidimensional
constructs: a review and guidelines for their use. European Journal of Information Systems,
21(1), 22-48. https://doi.org/10.1057/ejis.2011.10
Porpora, D. V. (2015). Do realists run regressions? In Reconstructing Sociology: The Critical Realist
Approach (pp. 31-64). Cambridge University Press. https://doi.org/DOI:
10.1017/CBO9781316227657.002
Pratschke, J. (2003). Realistic models? Critical Realism and Statistical Models in the Social Science.
Philosophica(71), 1-38.
Råholm, M. B. (2010). Abductive reasoning and the formation of scientific knowledge within nursing
research. Nursing Philosophy, 11(4), 260-270.
Rai, A., Lang, S. S., & Welker, R. B. (2002). Assessing the validity of IS success models: An empirical test
and theoretical analysis. Information systems research, 13(1), 50-69.
Rajapakse, J., Van Der Vyver, A., & Hommes, E. (2012). e-Government implementations in developing
countries: Success and failure, two case studies. 2012 IEEE 6th International Conference on
Information and Automation for Sustainability,
Rambaree, K. (2018). Abductive thematic network analysis (ATNA) using ATLAS-ti. In Innovative Research
Methodologies in Management (pp. 61-86). Springer.
Rambaree, K., & Faxelid, E. (2013). Considering abductive thematic network analysis with ATLAS-ti 6.2. In
Advancing Research Methods with New Technologies (pp. 170-186). IGI Global.
Ravasi, D., & Phillips, N. (2011). Strategies of alignment: Organizational identity management and
strategic change at Bang & Olufsen. Strategic Organization, 9(2), 103-135.
Rezvani, A., Khosravi, P., & Dong, L. (2017). Motivating users toward continued usage of information
systems: Self-determination theory perspective. Computers in Human Behavior, 76, 263-275.
Riany, G. (2018). Influence of e-Government Strategy Implementation on the Performance of Public
Service Delivery in Kenya. International Journal of Strategic Management, 7(2), 32-49.
Rizopoulos, D. (2006). ltm: An R package for latent variable modeling and item response theory analyses.
Journal of statistical software, 17(5), 1-25.

39
Roberts, N., Galluch, P. S., Dinger, M., & Grover, V. (2012). Absorptive capacity and information systems
research: review, synthesis, and directions for future research. MIS quarterly, 625-648.
Rose, J., Persson, J. S., & Heeager, L. T. (2015). How e-Government managers prioritise rival value
positions: The efficiency imperative. Information polity, 20(1), 35-59.
Rowe, F. (2014). What literature review is not: Diversity, boundaries and recommendations. European
Journal of Information Systems, 23(4), 241-255. https://doi.org/10.1057/ejis.2014.7
Rycroft‐Malone, J., Seers, K., Titchen, A., Harvey, G., Kitson, A., & McCormack, B. (2004). What counts as
evidence in Evidence‐based practice? Journal of advanced nursing, 47(1), 81-90.
Salter, K. L., & Kothari, A. (2014). Using realist evaluation yo open the black box of knowledge
translation: A state-of-the-art review. Implementation Science, 9(1), 1-14.
Sarantis, D. (2017, 8-9 November). Assessment of egovernment policies: Reflections on applied
frameworks. 30th International Business Information Management Association Conference -
Vision 2020: Sustainable Economic development, Innovation Management, and Global Growth,
IBIMA 2017, Madrid, Spain.
Sayer, R. A. (1992). Method in social science: A realist approach. Psychology Press.
Schon, D. A. (Ed.). (1990). The design process: Varieties of thinking: Essays from Harvard's philosophy of
education research center. VA Howard. New York, Routledge.
Schryen, G. (2015). Writing qualitative IS literature reviews—Guidelines for synthesis, interpretation,
and guidance of research. Communications of the Association for Information Systems, 37(1), 12.
Schryen, G., Benlian, A., Rowe, F., Shirley, G., Larsen, K., Petter, S., Wagner, G., Haag, S., & Yasasin, E.
(2017). Literature reviews in IS research: What can be learnt from the past and other fields?
Communications of the Association for Information Systems, 40(1).
Silenou, B. C., Tom-Aba, D., Adeoye, O., Arinze, C. C., Oyiri, F., Suleman, A. K., Yinka-Ogunleye, A.,
Dörrbecker, J., Ihekweazu, C., & Krause, G. (2020). Use of surveillance outbreak response
management and analysis system for human monkeypox outbreak, Nigeria, 2017–2019.
Emerging infectious diseases, 26(2), 345.
Simpson, S. N. Y., Tetteh, L. A., & Agyenim-Boateng, C. (2020). Exploring the socio-cultural factors in the
implementation of public financial management information system in Ghana. Journal of
Accounting & Organizational Change.
Slaghuis, S. S., Strating, M. M. H., Bal, R. A., & Nieboer, A. P. (2011). A framework and a measurement
instrument for sustainability of work practices in long-term care. BMC health services research,
11(1), 1-13.
Stamati, T., Papadopoulos, T., & Anagnostopoulos, D. (2015). Social media for openness and
accountability in the public sector: Cases in the Greek context. Government information
quarterly, 32(1), 12-29.
Strong, D. M., Volkoff, O., Johnson, S. A., Pelletier, L. R., Tulu, B., Bar-On, I., Trudel, J., & Garber, L.
(2014). A theory of organization-EHR affordance actualization. Journal of the Association for
Information Systems, 15(2), 2.
Templier, M., & Pare, G. (2018). Transparency in literature reviews: an assessment of reporting practices
across review types and genres in top IS journals. European Journal of Information Systems,
27(5), 503-550.
Thapa, D., & Hatakka, M. (2017, 4-7 January). Understanding ICT in ICT4D: An Affordance Perspective.
Proceedings of the 50th Hawaii International Conference on System Science (HICSS-50),
Waikoloa Village, Hawaii, USA.
Thapa, D., & Sein, M. K. (2018). Trajectory of affordances: insights from a case of telemedicine in Nepal.
Information Systems Journal, 28(5), 796-817.
Thierry, N., Adeline, K., Anita, A., Agnes, B., Baptiste, K. J., Pamela, J., & Kizito, K. (2014). A National
Electronic System for Disease Surveillance in Rwanda (eIDSR): Lessons Learned from a Successful

40
 Implementation. Online Journal of Public Health Informatics, 6(1), e118.
https://doi.org/10.5210/ojphi.v6i1.5014
Thomas, C. A., Mbarika, V. W., Nwogu, R., Musa, P. F., & Meso, P. (2009). Facilitating better governance
through E-government Initiatives: successful case in Sub-Saharan Africa. Information and
Communication Technology-Africa, 17.
Tolbert, C. J., & Mossberger, K. (2006). The effects of e‐government on trust and confidence in
government. Public Administration Review, 66(3), 354-369.
Tom-Aba, D., Silenou, B. C., Doerrbecker, J., Fourie, C., Leitner, C., Wahnschaffe, M., Strysewske, M.,
Arinze, C. C., & Krause, G. (2020). The surveillance outbreak response management and analysis
system (SORMAS): digital health global goods maturity assessment. JMIR public health and
surveillance, 6(2), e15860.
Tomlinson, J., Baird, M., Berg, P., & Cooper, R. (2018). Flexible careers across the life course: Advancing
theory, research and practice. Human Relations, 71(1), 4-22.
Twizeyimana, J. D. (2017). User-centeredness and usability in e-government: A reflection on a case study
in Rwanda. Proceedings of the Internationsl Conference on Electronic Governance and Open
Society: Challenges in Eurasia,
Twizeyimana, J. D., & Andersson, A. (2019). The public value of E-Government–A literature review.
Government information quarterly, 36(2), 167-178.
Twizeyimana, J. D., Larsson, H., & Grönlund, Å. (2018). E-government in Rwanda: implementation,
challenges and reflections. Electronic Journal of E-Government, ECEG, 16(1), 19-31.
Verkijika, S. F., & De Wet, L. (2018). A usability assessment of e-government websites in Sub-Saharan
Africa. International Journal of Information Management, 39, 20-29.
Verkooijen, K. T., Super, S., Mulderij, L. S., De Jager, D., & Wagemakers, A. (2020). Using realist
interviews to improve theory on the mechanisms and outcomes of sport for development
programmes. Social Inclusion, 8(3), 152-161.
Volkoff, O., & Strong, D. M. (2017). Affordance theory and how to use it in IS research. The routledge
companion to management information systems, 232-245.
Vom Brocke, J., Simons, A., Riemer, K., Niehaves, B., Plattfaut, R., & Cleven, A. (2015). Standing on the
shoulders of giants: Challenges and recommendations of literature search in information
systems research. Communications of the Association for Information Systems, 37(1), 9.
Wahid, F., & Sæbø, Ø. (2015). Affordances and effects of promoting eParticipation through social media.
International Conference on Electronic Participation,
Wang, P., Li, H., & Suomi, R. (2016). Value co-creation in business via social media: a technology
affordance approach. Pacific Asia Conference on Information Systems (PACIS) 2016 Proceedings,
Watson, R. T., & Webster, J. (2020). Analysing the past to prepare for the future: Writing a literature
review a roadmap for release 2.0. Journal of Decision Systems, 29(3), 129-147.
Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature
review. MIS quarterly, xiii-xxiii.
Westhorp, G. (2014). Realist impact evaluation: an introduction'. Methods Lab. London: Overseas
Development Institute. https://cdn.odi.org/media/documents/9138.pdf
WorldBank. (2018). How Rwanda became the first African country with an electronic procurement
system. Retrieved 13-02-2022 from
https://www.worldbank.org/en/news/feature/2018/09/05/how-rwanda-became-the-first-
african-country-with-an-electronic-procurement-system
Yadav, J., Saini, A. K., & Yadav, A. K. (2019). Analysing and securing the sustainability of e-Government
projects from technical aspect using employees perspective approach. International Journal of
Information Technology, 1-13.

41
Yang, Y., Liu, Y., Phang, C. W., & Wei, J. (2020). Using microblog to enhance public service climate in the
rural areas. Government information quarterly, 37(1), 101402.
Yaokumah, W., & Biney, E. (2020). Integrated financial management information system project
implementation in Ghana government ministries. International Journal of Information
Technology Project Management (IJITPM), 11(1), 17-31.
Yilmaz, K. G., & Belbag, S. (2016). Prediction of consumer behavior regarding purchasing remanufactured
products: a logistics regression model. International Journal of Business and Social Research,
6(2), 1-10.
Zachariadis, M., Scott, S., & Barrett, M. (2013). Methodological implications of critical realism for mixed-
methods research. MIS quarterly, 855-879.
Ziba, P. W., & Kang, J. (2020). Factors affecting the intention to adopt e-government services in Malawi
and the role played by donors. Information Development, 36(3), 369-389.

APPENDIX A
As highlighted earlier in our review section, the thematic coding data analysis process is
explained in detail herewith.
Stage 1: Data Familiarization
We read the 76 articles thoroughly for familiarization of our adopted articles where patterns and
meanings emerged to empower us to further engage with the literature.
Stage 2: Generation of Initial Codes
We separately identified concepts in the form of in-vivo codes and theory/theories used in the
case of the empirical papers while for conceptual papers, our primary articles revealed key
phrases and concepts. Subsequently, identical concepts from other studies were then merged into
one concept (Templier & Pare, 2018) and the collection of such concepts generates our first-
order themes (Gioia et al., 2013). The formation of the first-order categories elicits our upgrade
from tentative to substantive category (Ravasi & Phillips, 2011).
Stage 3: Development of Themes
We identified conceptual links from the first order categories when the in-vivo codes were
categorized where our first order categories were merged into more theoretically relevant and
broad second-order categories (Ravasi & Phillips, 2011).
Stage 4: Reviewing Themes
We iteratively resume the development of our second-order themes to a point of data saturation,
as well as persistently comparing the data (Braun & Clarke, 2006). The saturation became
apparent through the emergence of similar regularities in the data pattern (Lincoln & Guba,
1985). Particularly as the authors “began to see and hear similitude, repetition, and superfluity in
the data” (Evanciew & Rojewski, 1999). After that, the dimensions of the themes emerge with
the relationship between them evident and established.
Stage 5: Defining and Naming Themes

42
Our second order categories form distinct and contrastive broad themes for value propositions of
implemented e-Government projects in the form of; quality service delivery, effective public
organizations, open government, and democratic value, as well as social value and wellbeing.
Similarly, e-Government affordances: value co-creation affordance, accessibility affordance,
mobility affordance, dialogue affordance, and transparency affordance. Thus, our complete data
structure emerged through the above-explained processes, from first-order categories to their
relationship, second-order categories, and our final theme that emerged as shown in Figures A &
B respectively (as seen in appendix A).

Figure A: Thematic Data Analysis Process Deployed for Value Propositions of

Implemented e-Government Projects

43
 Figure B: Thematic Data Analysis Process Deployed for e-Government Affordances

Stage 6: Report Production

Finally, the inductive thematic process followed suggests that the descriptive/in-vivo codes as
first-order themes, interpreted categories as second-order themes and patterns came from the
literature that was considered to be a data set (Aksulu & Wade, 2010; Bandara et al., 2015). The
process was not imposed by theoretical underpinnings before data collection and analysis
(Bandara et al., 2015). Besides, we combined the thematic coding process with concept-centric
reviews (Cram et al., 2017; Inuwa & Ononiwu, 2020; Webster & Watson, 2002). Such a
combination focused on highlighting the key concepts and their relationships identified in each
article, as well as key characteristics, such as the theory used as shown in Table A and B. We
then considered the overarching themes as the value propositions of the implemented e-
Government projects and e-Government affordances, which we placed in the literature review
section as Table A and B respectively with their corresponding authors (as seen below).

Table A: Extracted Table from Primary Adopted Articles

Extracted Concepts from Articles Reviewed
Theme Sub- Theory/Models Concepts Authors
Theme

44
 New Public Management Services, outcomes, (Agbabiaka, 2018; Alruwaie et al.,
(NPM), Kearns conceptual efficiency, information 2012; Ashaye & Irani, 2019; Bai,
framework (KCF), Golubeva quality, system quality, 2013; Bonina & Cordella, 2009;
Framework (GF), Public Value intrinsic enhancement, Castelnovo & Simonetta, 2008; Cook
Management (PVM), New user orientation, & Harrison, 2014; Deng et al., 2018;
Public Service (NPS), IS accessibility, integrity, Flak et al., 2009; Golubeva &
Success model, Social usability, intention to Gilenko, 2018; Golubeva et al., 2019;
cognitive theory (SCT)P, use, use, Grimsley & Meehan, 2007; Grimsley
Expectation confirmation Complimentaries, lock- et al., 2006; Gupta & Suri, 2017;
theory (Berger & Sector), in, novelty, output, Hellang & Flak, 2012; Karkin &
Public Value Framework functional features, user Janssen, 2014; Karunasena & Deng,
(PVF), e-government acceptance, information 2010; Karunasena et al., 2011; Omar
framework, Situation actor et al., 2011; Pang et al., 2017; Persson
process (SAP), Learning action et al., 2017; Rose, Persson, &
Quality Service Delivery

performance (LAP), Value Heeager, 2015; Rose, Persson,

Positions Framework (VPF), Heeager et al., 2015; Scott et al.,
Value Propositions of Implemented E-government Projects

Stakeholder’s theory (ST) 2016; Sundberg, 2019; Twizeyimana

& Andersson, 2019).

NPM, KCF, GF, PVM, NPS,
ST, eGEP framework, IS
Success model, SCT, ECT,
PVF, E-government
framework, Public value of IT
Effective Public Organization
Effectiveness,
professionalism,
performance,
responsiveness,
experience, operational
(Agbabiaka, 2018; Alruwaie et al.,
2012; Ashaye & Irani, 2019; Avdic &
Lambrinos, 2015; Bai, 2013; Bonina
& Cordella, 2009; Castelnovo, 2013;
Chu et al., 2017; Cook & Harrison,

(PVIT), SAP, LAP, Value capability, feedback, 2014; Deng et al., 2018; Golubeva et
Assessment Framework (VAF), coordination, integration, al., 2019; Gupta & Suri, 2017;
VPF, Reference process infrastructure, skills, Harrison et al., 2012; Hellang & Flak,
framework staff, readiness, 2012; Jussila et al., 2017; Karunasena
organizational capability, & Deng, 2011; Karunasena et al.,
activity, agent, role, 2011; Persson et al., 2017; Rose,
artifact Persson, Heeager, et al., 2015;
Salvodelli et al., 2013; Scott et al.,
2016; Srivastava, 2011; Tsohou et al.,
2013; Twizeyimana & Andersson,
2019).

45
 NPM, KCF, GF, PVM, NPS, Trust, engagement, (Agbabiaka, 2018; Ashaye & Irani,
eGEP, IS Success, ST, PVF, e- democracy, openness, 2019; Avdic & Lambrinos, 2015; Bai,
Government Framework, PVIT, transparency, 2013; Chu et al., 2017; Cook &
participation, Harrison, 2014; Flak et al., 2009;
collaboration, equity, Golubeva & Gilenko, 2018; Golubeva
Open Government and Democratic Value

dialogue, equality, et al., 2019; Grimsley & Meehan,

cohesiveness, 2007; Grimsley et al., 2006; Harrison
legitimacy, awareness et al., 2012; Hellang & Flak, 2012; Hu
et al., 2019; Hussein 2018;
Karunasena & Deng, 2012;
Karunasena et al., 2011; Omar et al.,
2011; Pang et al., 2014; Persson et al.,
2017; Rose et al., 2018; Rose,
Persson, & Heeager, 2015; Rose,
Persson, & Heeager et al., 2015;
Salvodelli et al., 2013; Scott et al.,
Value Propositions of Implemented E-government Projects

2016; Subbiah & Ibrahim, 2011;

Sundberg, 2019; Twizeyimana &
Andersson, 2019; Wihlborg et al.,
2017).

e-GEP, Public sector IS/IT Environmental (Agbabiaka, 2018; Alruwaie et al.,

evaluation framework, NPM, IS sustainability, self- 2012; Ashaye & Irani, 2019; Chircu,
Success, ST, SCT, ECT, PVF, development, 2008; Golubeva & Gilenko, 2018;
Object boundary theory, IS satisfaction, self- Golubeva et al., 2019; Harrison et al.,
Success, Technology efficacy, financial & 2012; Hellang & Flak, 2012; Hu et
acceptance theory, Trust theory, organizational value, al., 2019; Hussein, 2018; Karunasena
Motivation theory, VAF political value, & Deng, 2012; Raus et al., 2009; Rose
Social Value and Wellbeing

constituency value, et al., 2018; Salvodelli et al., 2013;

quality of life, net Scott et al., 2016; Sigwejo & Pather,
benefit, effort 2016; Twizeyimana & Andersson,
expectancy, social 2019; Uppstrom & Lonn, 2017.
influence,

Table B: Extracted Table from Primary Adopted Articles

Extracted Concepts from Articles Reviewed
Theme Sub- Theory/Models Concepts Authors
Theme

46
 Actor-Network Theory (ANT),
Technology Affordances and
Constraints Theory (TACT),
Grounded Theory, Affordance
Value Co-Creation
Sensoriability, Actor, (Andrade & Urquhart, 2010:
Structure, Technology, Bjorneborn, 2017; Dremel et al., 2020;
Organization, Intrinsic Fromm et al., 2020; Majchrzak &
Motivation, Markus, 2012; Manoury et al., 2019;

Theory, Critical Realism (CR),
Technology Affordance,
Technology affordance theory,
Value co-creation theory
e-Government Affordances
Programmability, Pozzi et al., 2014; Riedl, 2003;
Standardization, Smorgunov, 2020; Strong et al., 2014;
Editability, Thapa & Hatakka, 2017; Thapa &
Interoperability, Sein, 2018; Vitari & Pigni, 2014;
Interactivity, Reciprocity Volkoff & Strong, 2017; Wang et al.,

2016; Zeng et al., 2020).

Affordance theory, User Diversifiability, Bjorneborn, 2017; Fromm et al., 2020;
Acceptance theory, Technology Accessibility, Hadidi & Carter, 2016; Hassanin et
Acceptance, and Use, IS Explorability, Effective al., 2021; Martinovic et al., 2013;
Success, TACT, CR, Use, Pattern of Use, Pozzi et al., 2014; Riedl, 2003;
Accessibility

Technology affordance theory, Mindfulness of Use, Rietveld & Brouwers, 2017;

Affordance

Value co-creation theory
Affordance Theory,
Technology affordance theory,
Value Co-Creation theory
Transparency
Visibility, Availability, Smorgunov, 2020; Strong et al., 2014;
Identifiability, Thapa & Hatakka, 2017; Thapa &
Multimediality Sein, 2018; Volkoff & Strong, 2017;
Wang et al., 2016;).
Trust, engagement, (Adams & Quadri, 2018; Chen et al.,
democracy, openness, 2016; Dremel et al., 2020; Fromm et
transparency, al., 2020; Giesbrecht et al., 2014;

participation, Riedl, 2003; Stamati et al., 2015;

Affordance

collaboration, equity, Wang et al., 2016).

equality, cohesiveness,
legitimacy, awareness

Affordance theory, ANT, CR, Traversability, Mobility, ( Ahad et al., 2021; Andrade &
IS Success, Technology Urquhart, 2010; Autio et al., 2018;
acceptance and use, TACT Bjorneborn, 2017; Fromm et al., 2020;
Affordance
e-Government Affordances

Kohut, 2018; Pozzi et al., 2014;

Mobility

Ranzini & Lutz, 2017; Riedl, 2003;

Technology affordance theory,
Value Co-creation theory,
Affordance
Strong et al., 2014; Thapa & Hatakka,
2017; Thapa & Sein, 2018).
Dialogue, Participation, (Chouikh et al., 2016; Dahlberg, 2011;
Openness, Conversation, Fung, 2015; Giesbrecht et al., 2015;

Affordance theory, Activity Community Goldkuhl, 2020; Mungai, 2018; Riedl,

Dialogue

theory connectedness 2003; Stamati et al., 2015; Vogl,

20202; Wang et al., 2016).

APPENDIX B
According to the understanding of logistics regression put forth by Tolbert and Mossberger
(2006), the Wald value should be ≥2 to depict the variable that influences the model (Al-
Ghamdi, 2002). The value of β informs the decision-maker about the influence direction of the

47
variable/factor. A negative value represents a negative effect, while a positive value represents a
positive effect. SIG represents the significant value that should be less than 0.05 to be important
in the model. At the same time, EXP (β) indicates that independent variables constitute the
dependent variable in a productive way (Tolbert & Mossberger, 2006). It also depicts the ranking
order level of importance of the constituted independent variables in the model. A variable with
the highest EXP (β) value is the first most crucial variable in the model (Rizopoulos, 2006).
Consequently, from the results of the quantitative data analysis as seen here in Tables C, D, and
E in appendix B, it is evident from the Wald value of ≥2 for all the actualized variables that the
factors influence the model. It is also confirm the actualized value propositions in the form of
quality service delivery, effective public organizations, open government and democratic values
as well as social value and general wellbeing. Additionally, it also shows SIG of less than 0.05
from most of the items representing the factors and a higher EXP (β) indicating that all the
elements constitute the actualized value propositions. However, we also observed that improved
service delivery attached to quality of service shows the highest EXP (β). It is indicative of being
the most desirable and crucial actualized value propositions as indicated by the respondents. This
gives an insight on the preferences of citizens, businesses and all the users of the implemented e-
Government projects and what they value most as the actualized value propositions of implement
e-Government projects.
Goodness-of-Fit of the Model
Goodness of Fit for Irembo models is the measure of Likelihood. Likelihood is the probability of
observed results, given the parameter estimates and -2 times the log-likelihood (-2LL). The -2log
likelihood is in turn, the measure of badness-of-fit, illustrating error remaining in the model after
accounting for all independent variables. The -2LL of 44.817 in the Irembo model indicated no
significant error remaining in the model. Table F in appendix B shows the Nagelkerke R square
variation of 80% of the actualized values. These are quality service delivery, effective public
organizations, social value and general wellbeing, as well as open government and democratic
values. This responsive perception from the citizens and users of the implemented e-Government
platform suggests a greater acceptance of our explicated value propositions of implemented e-
Government projects from literature.

For the e-Procurement model where the -2LL of 17.173 indicates no significant error remaining
in the model. The Nagelkerke R square shows 75% of the variation of actualized values
constituting quality service delivery and effective public organizations. As well as social value
and general well-being and open government and democratic values as seen in Table G in
section B. It is indicative of what our systematic literature review revealed as the value
propositions of implemented e-Government projects and citizens and users of the platform
perceived as the actualized values.
For e-Passport model also, the -2LL of 46.657 indicates no significant error remaining in the
model (see Table H of Appendix B). The Nagelkerke R square shows 82% of the variation in the
actualized values. These are in the form of quality service delivery, effective public

48
organizations social value and general wellbeing as well as open government and democratic
values.
Table C: Variables in equation Table for Irembo

Independent Service Item Β S.E. Wald Df SIG. Exp (β)

Variables
Quality Service Improved service delivery 4. 24 17.27 .025 74.99
Delivery 1.65 1
Improved access to services 3.75 .89 23.96 1 .033 54.76

Improved transparency, 1.36 .99 11.89 1 .001 13.91

accuracy, and facilitating
information between
government and customers
Registration and 1.36 1.07 1.61 1 0.35 13.89
certification
Effective Public Filing Tax -17.67 2.91 14.00 1 .097 1.00
Organizations News and announcements 8.19 .37 15.03 1 .018 16.76
Accountability and 6.64 .47 18.05 1 .027 56.07
transparency
Streamlining procedures -.62 1.26 16.00 1 .087 5.37
Procurement 4.34 1.68 11.23 1 .039 44.89
Horizontal integration -.071 2.85 11.00 1 1 .932
(different functions and
services)
Integrated information -16.87 1.72 7.10 1 .029 12.00
knowledge management
Social Value and Social group profile -1.36 1 1 6.53
2.96 8.00
Wellbeing management
Status reports of .35 1 .044 31.42
1.43 14.23
government projects
Building trust between .35 .97 16.13 1 .02 14.22
government and citizens
Open Government Opendata 3.23 1.57 14.22 1 .04 24.19
and Democratic
Value
Improved public -14.44 1.82 10.27 1 .023 6.08
engagement and enhanced
participation
Citizen’s voice (Medium to -16.77 .96 12.06 1 1.00 2.13
reach out to government
complain and feedback)

Constant 35.14 .99 4.901E+5

8

49
Table D: Variables in equation Table for e-Passport

Independent Service Item Β S.E. Wald Df SIG. Exp (β)

Variables
Quality Service Improved service delivery 4. 23 1.65 17.22 1 .022 73.12
Delivery
Improved access to services 3.64 .88 23.78 1 .032 53.54

Improved transparency, 1.48 .10 12.32 1 .001 14.57

accuracy, and facilitating
information between
government and customers
News and announcements 9.28 .38 15.04 1 .019 16.88
Accountability and 7.03 .49 18.50 1 .028 56.79
transparency
Streamlining procedures -.622 1.26 16.00 1 .087 5.37
Horizontal integration -.071 2.85 11.00 1 1 .93
(different functions and
services)
Integrated information -16.87 1.72 7.10 1 .029 12.00
knowledge management
Building trust between .35 1 .016 14.22
.97 16.13
government and citizens
Open Government Opendata 3.23 1.570 14.23 1 .040 24.20
and Democratic
Value
Citizen’s voice (Medium to -16.77 .962 12.06 1 1.00 2.13
reach out to government
complain and feedback)
Constant 37.53 .99 4.951E
+60

Table E: Variables in equation Table for e-Procurement

Independent Service Item β S.E. Wald Df SIG. Exp (β)
Variables
Quality Service Improved service delivery 4. 24 1.53 3.54 1 .025 74.98
Delivery Improved access to services 3.75 1.88 9.88 1 .033 54.76
Improved transparency, 1.36 1.03 7.03 1 .001 13.91
accuracy, and facilitating
information between government
and customers
Registration and certification 1.36 1.98 2.98 1 0.35 13.89
Effective Public Filing Tax -17.67 .53 3.03 1 .09 1.00
Organizations News and announcements 8.19 3.01 13.01 1 .02 16.76
Streamlining procedures -.62 1.83 14.73 1 .09 5.37

50
 Procurement 4.34 .24 19.40 1 .04 44.89
Social Value and Social group profile -1.36 .88 19.88 1 1 6.53
Wellbeing management
Status reports of government .35 .37 16.42 1 .04 31.42
projects
Building trust between
government and citizens .35 1.88 11.88 1 .02 14.22
Open Government Opendata
and Democratic 3.23 1.60 12.60 1 .04 24.19
Value
Citizen’s voice (Medium to -16.77 2.89 18.47 1 1.00 2.13
reach out to government
complain and feedback)
Constant 42.54 .99 50.86

Table F: Model summary table for Irembo

Step -2 Log likelihood Cox & Snell R Square Nagelkerke R Square
1 44.817 a
.043 .795

Table G: Model Summary for e-Procurement

Step -2 Log likelihood Cox & Snell R Square Nagelkerke R Square

1 17.173a .388 .754

Table H: Model Summary for e-Passport

Step -2 Log likelihood Cox & Snell R Square Nagelkerke R Square

1 46.657 a
.047 .820

APPENDIX C
Table I: Realist Interview Table
Country: Rwanda
S/N Investigative Service Type ID#s of Interviewee & Years of
Context Position Experience
Organization and Staff Interviewed (Those that run the Irembo Application)

51
 1 IREMBO e-Services #RWSM - Senior Manager 3
2 IREMBO e-Services #RWM - Manager 4
3 IREMBO e-Services #RWDM - Deputy Manager 2
4 IREMBO e-Services #RWAM1-Assistant Manager 4
5 IREMBO e-Services #RWAM2-Assistant Manager 2
Staff Interviewed in e-Procurement Office (office that runs the e-Procurement
Application)
1 e-Procurement Online bidding and #ePSM – Senior Manager 4
Contract information
services
2 e-Procurement Online bidding and #ePM - Manager 2
Business information
services
3 e-Procurement Online bidding and #ePDM1 - Deputy Manager 3
Business information
services
4 e-Procurement Online bidding and #ePDM2 -Deputy Manager 5
Business information
services
5 e-Procurement Online bidding and #ePAM - Assistant Manager 4
Business information
services
Country: Nigeria
Staff Interviewed in the e-Passport Office (Office that runs the e-Passport Application)
1 e-Passport International e- #DCI - Deputy Controller 5
Passport
2 e-Passport International e- #ACI- Assistant Controller 3
Passport
3 e-Passport International e- #CSI1 - Chief Superintendent 2
Passport
4 e-Passport International e- #CSI2 - Chief Superintendent 3
Passport
5 e-Passport International e- #SI - Superintendent 2
Passport

Table J: Sample Realist Interview Questions Used

Question / Question structure
Can you tell me how your engagement with
Irembo/e-Procumbent/e-Passport has been?
Rationale
Realist interviewing assumes that people know
different things according to their roles. Use the
answers to this question to tailor future
questions to what it is that the respondent can be

52

What do you consider as the outcomes of Irembo/e-
Procurement/e-Passport? (a) Quality service
delivery (b) Effective public organization (c) Open
government and democratic value (d) Social value
and wellbeing
Do you think that the outcomes have been the same
for all? (Businesses, citizens, government, society)
In what ways have they been different?
Have the outcomes been the same for citizens;
businesses, society, government
agencies/departments. In what ways have they been
different?
We are very curious about how Irembo/e-
Procurement/e-Passport causes its outcomes. How
do you think Irembo has caused or helped to cause
the outcome?
Do you think Irembo/e-Procurement/e-Passport
changed the way [citizens, businesses, and society]
thinks or feels about the objective of Irembo/e-
Procurement/e-Passport in any way? In what ways?
Can you provide examples?
For participants: Has Irembo/e-Procurement/e-
Passport changed the way you think or feel about it
in any way?
There are lots of ideas about how Irembo/e-
Procurement/e-Passport should work, and we think
it probably works differently in different places or
for different people. One of those ideas is [brief
description of the main mechanism]. Has it worked
at all like that here/for you? Can you give an
example?
What is it about the way Irembo/e-Procurement/e-
Passport was implemented that made a difference to
how it worked? Or ‘What is it about the way
Irembo/e-Procurement/e-Passport works that make a
difference to how it works?
expected to know about.
Realist evaluation assumes that programs have
different outcomes for different groups. The
question should be asked repeatedly for
different groups until the range of outcomes has
been identified.
Realist evaluation seeks to identify “in what
respects, for whom” and “to what extent, for
whom” programs achieve outcomes. The
question can be asked repeatedly for different
groups
The focus of a realist interview is the program
theory. The question is seeking more specific
information about “for whom” the program has
and has not been effective (in what respects, to
what extent), but probing specifically about
subgroups that are identified in the program
theory.
The initial question leading to the exploration of
mechanisms. Many participants will identify
program activities (e.g. training) or resources
(e.g. funding). It is ESSENTIAL to probe
further – e.g. – So what did the training initiate
that was new? Was it mainly about new skills,
do you think, or new attitudes? Or: So what
exactly was the outcome of the training? How
did that help cause (the later outcome)?
Realist evaluation uses the construct of
‘reasoning and resources’ to explain how
programs cause an outcome. This is an explicit
probe for ‘reasoning’ about a specific aspect of
program theory. E.G. – Program may expect
stakeholders to change their understanding of
their roles or responsibilities and adapt their
behavior accordingly. So the question might be
“Has the program affected how teachers think
about their role in any way?”
The subject of a realist interview is the program
theory. The aim is to get the respondent to
refine the program theory for the particular
context about which they know. This question
should be asked about at least two different
explanations of how the program might work
and elicit whether the program works differently
for different people.
Realist interviewing sees specific aspects of
implementation as aspects of context (i.e.
factors that affect whether and how mechanisms
are triggered). The aim is to understand how
implementation has affected mechanisms and
53

We’ve seen that Irembo/e-Procurement/e-Passport
works differently in different places. What is it
about this place that makes it work [so well, less
well]?
If you could change something about Irembo/e-
Procurement/e-Passport to make it work more
effectively here, what would you change and why?
What else do you think we need to know, to
understand how this program has worked here?
therefore outcomes. Probe for both positive and
negative aspects of implementation.
Realist evaluation assumes context does affect
outcomes (by affecting which mechanisms fire).
Probe for aspects of culture, local resources/lack
of them, local relationships, the relationship
between organization and participants, and so
on.
Often elicits understandings of why the program
has not worked as effectively as it might (i.e.
mechanisms not firing, aspects of context) as
well as strategies for improvement.
Open probe that enables participants to
comment on anything not covered by the
interview. The structure of the question keeps
the focus on ‘how the program works and ‘in
this context.

How do you seek feedback from users’ experience about accessibility, ease of use, awareness of
information and services? I want to understand and assess the reasons that may encourage or
discourage citizens and businesses from Irembo/e-Procurement/e-Passport. How do you
determine awareness of the government’s feedback and complaints review mechanism? I want to
understand how you facilitate partnership between the government and private sector in running
Irembo/e-Procurement/e-Passport

54
APPENDIX D
Table K: Sample Questionnaire used
Being a user of the Irembo platform for registration, certification, tax filing, and other services. I believe you have realized some values as a
result of using the platform. However, the success largely relies on citizens and businesses that use the platform to actualize the value
propositions that benefited from such projects. One way of doing this is by checking on any of these values you derive from the Irembo
platform and your willingness to pay to continue deriving the indicated value/values or not.

How often do you Daily ☐ Weekly ☐ Monthly ☐ Quarterly☐ Twice a year☐ Once a year☐
use the e-
Procurement
platform?

Dissonance-minimizing I support the services I support the services but I support the services I oppose the services
Bias Questions and can afford payment cannot afford payment ☐ depending on the condition☐ regardless of the cost☒
☐

Actualized Value Continue to The rationale for The rationale for

Actualize the your choice under your choice
value under actualized value under
condition of willingness to
willingness to pay pay or not
SERVICES SYSTEM VALUE Yes No Yes No
PROPOSITION
Improved G2G Quality Service ☐ ☐ ☐ ☐
communication Delivery
among government
agencies
G2C ☐ ☐ ☐ ☐
Improved services
Improved access to G2C ☐ ☐ ☐ ☐
services
Improved G2B ☐ ☐ ☒ ☐

55
transparency,
accuracy, and
facilitating
information between
government and
customers
Registration & G2C ☐ ☐ ☐ ☐
certification
Filling of taxes G2B ☐ ☐ ☐ ☐
News & G2C ☐ ☐ ☐ ☐
announcements
Opendata G2C ☐ ☐ ☐ ☐

Accountability and G2B ☐ ☐ ☐ ☐

transparency
Streamlining G2B ☐ ☐ ☐ ☐
Effective Public
procedures
Organizations
Procurement G2B ☐ ☐ ☐ ☐
Vertical integration G2G ☐ ☐ ☐ ☐
(local, state, federal
governments connect
for different services)
Horizontal G2G ☐ ☐ ☐ ☐
Integration (Different
functions and
services)
Integrated G2G ☐ ☐ ☐ ☐
information
knowledge
management
Shared information G2G ☐ ☐ ☐ ☐
and ideas between
government agencies
and departments to
form a mega database

56
Document G2G ☐ ☐ ☐ ☐
interchange
Registration & G2G ☐ ☐ ☐ ☐
certification
News & G2G ☐ ☐ ☐ ☐
announcements
Procurement G2G ☐ ☐ ☐ ☐
Social Value and
Monitoring & reports G2S Wellbeing ☐ ☐ ☐ ☐
of diseases outbreaks
Social group profile G2S ☐ ☐ ☐ ☐
management
Status reports of G2S ☐ ☐ ☐ ☐
government projects
Improved public G2S ☐ ☐ ☐ ☐
engagement and
enhanced
participation
Citizen’s voice G2S ☐ ☐ ☐ ☐
(medium to reach out
to government
complaints and
feedback)
Building trust G2C ☐ ☐ ☐ ☐
between government
and citizens

Improved public G2C Open ☐ ☐ ☐ ☐

engagement and Government and
enhanced Democratic
participation Value
Opendata G2G ☐ ☐ ☐ ☐
Accountability and G2B ☐ ☐ ☐ ☐
transparency

57
Have you actualized Actualized ☐ ☐
the values for the e- Values
Procurement
platform?
Are you willing to ☐ ☐
pay for the actualized
value?
Will you continue to ☐ ☐
actualize the value?

Age Gender Educational Level Employment Status Income Level

18-30 ☐ Male ☐ Secondary Level ☐ Part-Time ☐ $50-$99 Monthly ☐
31-40 ☐ Female ☐ Diploma/NCE ☐ Full Time ☐ $100-$199 Monthly ☐
41-50 ☐ First Degree ☐ Student ☐ $200-$299 Monthly ☐
51 and above ☐ Masters ☐ Contractor ☐ $300-399 Monthly ☐
Ph.D. and above ☐ Consultant ☐ $400-$499 Monthly ☐
Others ☐ $500 & above monthly ☐

58