See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/325566497

Evolution of ERP Systems in the Cloud: A Study on System Updates

Article  in  Systems · June 2018

DOI: 10.3390/systems6020022

CITATIONS READS

11 1,699

2 authors, including:

Moutaz Haddara
Kristiania University College
57 PUBLICATIONS   513 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Big Data Analytics & Industry 4.0 View project

Call for Papers: Special issue at Scandinavian Journal of Information Systems on Enterprise Systems View project

All content following this page was uploaded by Moutaz Haddara on 05 June 2018.

The user has requested enhancement of the downloaded file.

 systems
Article
Evolution of ERP Systems in the Cloud: A Study
on System Updates
Elise Bjelland 1 and Moutaz Haddara 2, *
1 Xledger AS, Østensjøveien 32, 0667 Oslo, Norway; elise.bjelland@xledger.no
2 Westerdals-Oslo School of Arts, Communication and Technology, Vulkan 19, 0186 Oslo, Norway
* Correspondence: hadmoa@westerdals.no




Received: 25 March 2018; Accepted: 28 May 2018; Published: 4 June 2018


Abstract: Cloud-based enterprise resource planning (ERP) systems emerged around the new
millennium, and since then there has been a lack of research regarding the evolution and update
processes of these systems. From the users’ perspective, updates in a traditional on-premise
ERP system are carried at their own request; while cloud-based ERPs are compulsory updated.
Through an established ERP lifecycle framework, this study investigates how the process of updates
is conducted in a cloud ERP context, from both the users’ and vendors’ perspectives. A multiple case
study was conducted in Norway at 10 client organizations, as well as a cloud ERP vendor. Our main
findings suggest that the vendor and the users view the process of updates differently. The main
challenges with the process of updates from the users’ perspective are the size and date of the updates,
lack of information and communication during the process, and extinction of certain functionalities.
Yet, the main advantages are that all system users will always have the same version of the system,
users do not need to spend time on updating the system and paying attention to the ERP market,
which leads to more focus on their core competences instead.

Keywords: cloud ERP; evolution; SaaS; ERP lifecycle; updates; Norway

1. Introduction
Heraclitus once said, “The only thing that is constant is change”. In line, enterprise resource planning
(ERP) system implementations have changed dramatically since the emergence of cloud-based ERP systems.
Esteves and Pastor [1] argue that change as such, seems to be the foremost fact associated with ERP
systems. Cloud-based ERP systems transformed the way systems are offered, acquired, implemented, used,
maintained, evolve, and even retire. The emergence of cloud computing technologies started around the
year 2000, and since, ERP systems and database servers started to move into the cloud [2]. While there have
been many studies on cloud-ERP, however, there is still lack of research on ERP post-implementation issues
in general [3], and evolution and updates in cloud-based ERP systems in particular [3–5].
ERP systems need to be updated regularly in order to fix bugs, tighten security, extend current
functionalities or modules [6], convene market/legal regulations, and to update processes and meet
the constantly evolving organizations [7]. However, in contrast to on-premise ERP, cloud ERP users
are not anymore in control of the updates/upgrades and maintenance decisions, so maintenance and
update decisions and efforts are solely triggered and conducted on chosen dates by ERP vendors [8].
In an on-premise ERP system context, the upgrades of the system are normally triggered by a request
from the software users. The vendor informs the users that a new version of the system is available,
and then the users can decide whether to buy and implement it [9]. An on-premise ERP system upgrade
is seen as a complex and resource consuming task [9]. If an organization decides to implement the
update, they will need a dedicated team, which requires time, resources, and skills [6]. These reasons
may explain why several organizations using an on-premise ERP system abstain to conduct major

Systems 2018, 6, 22; doi:10.3390/systems6020022 www.mdpi.com/journal/systems

Systems 2018, 6, 22 2 of 26

upgrades to their systems and continue to use an outdated version until they are forced to update
it or retire it. Organizations that have adopted a cloud-based ERP system, experience that the ERP
system is updated automatically on certain planned dates, regardless of their preferences, without the
need for an internal skilled team, and commonly without additional costs [10]. Hence, in a cloud ERP
system, the client is not in control of the updates, as the vendor is the sole entity in charge of the ERP
system update decisions. As the updates are made automatically, client organizations can potentially
interpret them as forceful, as systems are updated without their consent.
The ERP lifecycle framework developed by Esteves and Pastor [1] represents the lifetime of
an on-premise ERP system. One of the phases with paramount importance during the lifetime
of an ERP system in organizations is the evolution phase. Activities that are fulfilled during the
evolution phase are typically updates and upgrades of the system, additional training and user skill
and capacity building, and the continuous business improvements [1], which matches the process
of updates explored in this study. In addition, through adopting the ERP lifecycle framework by
Esteves & Pastor [1], the evolution phase will be analyzed according to four related issues, also called
dimensions: change management, people, process and product. Through these dimensions, this study
attempts to explore and answer the following main question:

• What are the organizational and contextual factors affected by the forced updates within the
evolution phase of cloud-based ERP systems?

In order to answer the above question, several sub-questions also need to be investigated. A list
of the sub-questions is provided below.

◦ How the cloud-ERP system is updated? And how frequent?

◦ Users’ thoughts and attitudes about the update process;
◦ Acceptance and readiness for the update process;
◦ User training activities after system updates;
◦ And finally, the benefits and challenges of forced updates within organizations.

The rest of the paper is organized as follows: a literature review and theoretical background are
presented in Section 2. Research methodology and target cases are presented in Section 3. Section 4
highlights the main findings, followed by a discussion of the main findings in Section 5. Section 6
provides an overview of the study limitations. Finally, our conclusions and recommendations for
future research are presented in Section 7.

2. Related Literature and Theoretical Background

2.1. Cloud ERP Systems

An ERP system is defined as a cross-functional and enterprise-wide system operating through
a bundle of integrated modules that support the standard business processes of an organization.
The ERP system contains standard functionalities that can be applied in different types of industries.
One of the main goals of an ERP system is to support the business’ core processes and routine
transactions through integrating both data and cross-functional processes into the system [3].
Today, ERP systems can be leased as hosted software in a cloud-based technology [8]. In a cloud,
the vendor provides the users access to a system, hardware, and storage capacity, where system and
hardware management are highly transparent to the users [11]. The motivation for organizations to
adopt an ERP system is typically to reduce costs, improve decision making through better reporting
capabilities, improve customer relationships, meet market and legal requirements, and to increase
process efficiency [4]. With a cloud-based ERP system, organizations have the possibility to access the
ERP system that is hosted at a vendor’s site via the Internet. In addition, the responsibility regarding
updates, managing servers, maintenance, and performing backups lies with the vendor [8]. In other
words, the cloud ERP vendor ensures the security, stability, and seamless operation of the system
Systems 2018, 6, 22 3 of 26

on behalf of its client users. In current literature, cloud-based ERP systems are gaining increased
attention from researchers. Since the emergence of cloud-based ERP systems, the majority of ERP
system implementations have been in the cloud [11]. The emergence of the cloud service model
gives the client the opportunity to get on-demand network access and to share a bundle of resources.
These resources can include networks, servers, data storage, applications (e.g., ERP), and others [11].
The most common type of cloud-based ERP delivery models, is the software as a service (SaaS)
model [12,13]. SaaS eliminates the need to physically install and run the server-side applications on the
customer’s own premises, and eliminates the need for back-end hardware and data centers required to
run the system [14], which in turn simplifies the application’s maintenance and support operations.
This makes it possible for small organizations to take advantage of ERP systems, as they do not require
neither the substantial resources nor the skills needed for a successful on-premise implementation [8].

2.2. The ERP Lifecycle Framework

The ERP lifecycle framework (Figure 1) consists of six phases which represents the life stages
of an ERP system within organizations [1]. It also constitutes four dimensions, which are seen as
viewpoints that can be used to analyze those phases. The phases are adoption decision, acquisition,
implementation, use and maintenance, and evolution and retirement. The dimensions, also seen as
important issues within the evolution phase are: change management, people, process and product.
As the main focus of this study is updates, the study will describe the four dimensions of the ERP
lifecycle framework in relation to the evolution phase for a cloud-based ERP system. The evolution
phase is about introducing and integrating more capabilities and functionalities into the new ERP
system [1]. The challenge with the evolution phase is that it seems like this phase is taken for granted.
Albadri and Abdallah [15] state that the challenges the users often meet during the evolution phase
are recognized as important, yet inadequately addressed in research. It is clear that this phase has not
been a center of attention in previous literature regarding ERP systems in general [3], and in the cloud
context in specific [6].

Figure 1. The ERP lifecycle framework. Adapted from [1].

According to Herbert Nathan and Co., the Norwegian market is covered by more than 30 vendors
and close to 40 different solutions and systems [16]. Notable differences between them evolve around
technology (on-premise vs. cloud-based), use of partners, business areas, multi-country use and
support [16]. Aberdeen group believes that the trend towards the implementation of cloud-ERP
solutions will continue predominantly [17]. To grab this momentum, almost all the major vendors in
the ERP market are now offering their systems under the SaaS model. In Norway, cloud-based ERP
has generally seen a rapid and wide acceptance [16].

Dimensions

• Product: product is about the hardware and software changes that are implemented during
updates. There are no proper guidelines or standards for ERP maintenance and update
Systems 2018, 6, 22 4 of 26

preparations [5]. Even if there is not much literature regarding how a cloud-based ERP system
is updated [5], however, the topic has been briefly discussed by few researchers. For example,
Juell-Skielse and Enquist [12] state that the cloud ERP vendors upgrade their software regularly.
There is no literature at this time on the ideal frequency of updates for an ERP system, as the
areas of cloud ERP maintenance and upgrades are understudied as compared to other ERP
implementation phases [3,8]. However, Calvert and Seddon [7] have conducted research on
the consequence of updates in an on-premise context, and they suggest that updates typically
cause a decline in both individual and organizational capacity for a certain period afterwards.
In addition, organizations will need to undertake extensive user re-training efforts to correspond
with the system upgrade [7]. This matches the findings of another study, which argues that client
organizations will most likely experience ripple effects after updates [18]. However, if the vendor
has thoroughly conducted tests prior to an update, the client organizations would largely be
shielded from these negative effects [18]. An overview of the issues discussed in literature in
relation to updating cloud-based ERP system is provided below:

- It is easier to introduce a new functionality [11];

- Less planning, testing, maintenance, and configurations from the customer’s side [8,11];
- Lower costs, both with maintenance and IT staff, and updates are free of costs [8,11,12];
- The vendor has the possibility to expand its potential customer base, and let the users
focus on their core competencies [11,12];
- It is the vendor who provides a helpdesk and maintenance support [5];
- The organizations are not the owners of the system, as they do not own the infrastructure
or run the applications [19];
- Cloud-based ERP vendors regularly release new versions [18];
- Cloud-based ERP systems usually get more frequent updates and new functionalities than
on-premise ERP systems [11].

As seen in the above list, one of the main advantages of cloud-based ERP updates is that they
are free of charge. There are several reasons why the vendor should invest time and recourses in
updating the system, even when it is free for users as technology updates are inescapable process
with dealing with any man-made creation utilized for productive gain [7]. Therefore, vendors should
update the ERP system regularly to allow their customer organizations to constantly evolve and
optimize their business processes, and improve decision-making [7]. Frequency of updates however,
is still an unanswered question in cloud ERP literature.

• People: This dimension refers to the users of the system, and their way of developing skills within
it, also called ERP training. If a user has good skills and much knowledge regarding the system,
it makes it possible for the user to become more effective and competitive, and a decrease in costs
will occur. In order to continually be able to be more efficient, the ERP system needs to be updated
with smarter functionality. In this case, the users of the system, the people, need to learn how to
use the new functionalities to take advantage of them, and therefore ERP training could become
relevant. Training is outlined as what enhances and facilitates the user’s capability to use the ERP
application [15]. The importance of ongoing (or continuous training) after the ERP system has
been implemented, has not been widely addressed in the academic arena [7,20]. Hence, there is
also very little research to assess the implications of continuous ERP upgrades with regard to ERP
training needs [7]. However, the importance of training before and during the implementation
phase has been well documented in earlier research [7,20,21]. ERP training is an important and
relevant topic in literature, as it is one of the most important critical success factors (CSF) identified
in ERP implementations [4,6,15,21]. Therefore, organizations have to prioritize training to be able
to enhance the skills of their employees, and to make it easier for users to change roles within the
organization [7]. If the users of the system spend time on training, they can more easily understand
Systems 2018, 6, 22 5 of 26

the importance of continuous training and improvement. As their environment is constantly

changing, this could motivate them to continue training, and being more active and cooperative
in collaboration with other departments [6]. To improve, or at least maintain, the users’ expertise
and capability of using the system, user training is necessary during the evolution phase [6,7,20].
Well-trained users, can take better advantage of the capabilities of the system and motivate and
initiate new users [7]. One of the challenges with ERP training is that it is typically under-budgeted,
and training is often the first thing to be cut in the budget, especially during economic downturns
or increasing competition [7,15]. Another challenge is that users do not see the need for changing
their work processes and usually resist this change [21]. According to Wheatley [22],companies
spend from 50 percent to as little as 5 percent of their ERP budget on training. He also states
that organizations have begun to realize the fact that training should be a key requirement [22].
While discussing how ERP training should be done most effectively, it is important to keep in
mind that the users should not only train in how to use the systems’ functionality, but they also
might need to relearn their jobs, which could be an even more challenging task, and is a relevant
problem for discussion in the change management dimension [23].
• Process: This dimension is about how the users need to re-engineer their processes to be able
to adapt to the new business models and functionalities [1]. This is a huge task when first
implementing a new system, but within a cloud-based ERP environment, the incremental changes
implemented in updates are typically small and do not introduce major changes to the existing
processes. In addition, as cloud-ERP mainly targets SMEs, customizations are (in most cases) very
rare in the cloud ERP domain [24], and vanilla implementations are commonly adopted.
• Change management: Change management is described as the process of administering and
coordinating changes introduced to business processes and applications [25]. Change management
also seeks to warrant the acceptance and readiness of the new system, in order to allow the organization
to foster the benefits of its use [1]. Hence, change management is considered one of the most important
success factors in ERP implementations [25]. This study will look into change management and process
from a cloud-based ERP system perspective and will look at how users accept new functionalities that
have been implemented during updates and if the users need to change their work practices after
updates. According to Foster et al. [25], 75% of organizational change management efforts involving
technology fail as a result of people’s negative reactions to making change in their work practices.
According to Seethamraju [19], the organization using a cloud-based ERP system should be aware
that there is no guarantee that the users will accept changes, and use the new functionality effectively.
The handling of change management issues is often viewed as more difficult to handle, than technical
issues [15]. In Seethamraju’s [19] study, it was reported that educating users and training them on how
to use both the existing and newly introduced functionalities, and then make them actually use them,
were identified as critical challenges in the post-implementation phase.
Based on our literature review at the time of this study, we were not able to identify neither any
research targeting cloud ERP evolution in general, nor the process of updates in particular.

3. Research Methodology and Cases

As mentioned in the introduction Section 1, this study is an effort to understand the cloud-ERP
update process in organizations, and the impact of updates on the organizations and users. As this
topic is under-researched in the existing literature, the authors sought that a qualitative research
methodology [26] would be more suitable in order to gather deep insights from informants in their
natural setting. In general, qualitative research aids in understanding the subjects at their natural
social and cultural context [27]. The researchers conducted in-depth interviews with Norwegian
clients/users of one cloud-based ERP (Xledger), to elucidate knowledge regarding their usage and
experience with the system. To get a broader understanding of the topic, employees at the ERP vendor
organization (Xledger AS) were also interviewed. According to Yin [28], every case study is unique,
and case study research allows researchers to retain the holistic and meaningful characteristics of
Systems 2018, 6, 22 6 of 26

real-life events. A case study is thus a revealing method when one has not previously studied the same
kind of problem formulation [26]. One potential strength of case study research, is the possibility to
develop a deeper understanding of the complexity of the case and a phenomenon [28]. Case studies
are among the most suitable and common qualitative methods adopted in information systems (IS)
research [27]. Thus, this study has employed an explorative multiple case study research design [26,28].
The advantage with a multiple case study design is that it helps the researcher to learn as much as
possible about the phenomena that is under investigation [28]. It also enables the researchers to get
insights about how the use and perception of the phenomena is experienced across the different target
cases. In addition, multiple case study findings could provide a better overview and build a more solid
foundation for analytical results than from a single case study [28], and can enable cross-case analysis.
In order to elucidate knowledge and have a deeper understanding of the research topic from both
perspectives, the first author conducted a total of 18 qualitative semi-structured interviews at the ERP’s
users and vendor organizations. Fifteen interviews were conducted with Xledger users, and three
interviews were conducted at the vendor. The data gathering process was conducted over a period of
five weeks where the interviews were evenly distributed over the period. Two iterations of interviews
were conducted. The first two weeks were the first iteration, and the last three weeks were the last
iteration. The reason for these two iterations was because of the opportunity to listen to the interviews
in the first iteration and improve the interview guide to better fit the research if necessary. The data
gathered represent a snapshot of today’s process of updates in Xledger from the users’ and vendor’s
point of view. The main focus in the interviews was on the users’ thoughts, feelings and experiences
about the process.

3.1. Target Cases

This research focuses on the users of the ERP system Xledger. Xledger ERP is only offered as a cloud
ERP solution, which was introduced by Xledger AS in the year 2005. Xledger AS has four main offices and
a large network of partners. The headquarters is in the United States, and the other offices are located in
Norway, Sweden, and the United Kingdom. Xledger serves more than 9000 customers worldwide and
operates in more than 60 countries. In addition to the traditional and standard ERP modules, Xledger
ERP includes a CRM module and business intelligence reporting capabilities. Xledger also supports
multi-currency and multi-company operations.
The users participating in this study are from ten different companies, which represent the target cases.
The interviews were conducted with six accountancy firms, one from the shipping industry, one from the
real estate and facility management industry, one from the course and conference industry (event planner),
and one kindergarten, which is a total of 10 target cases. The unit of analysis across the target cases is
that they all use the same system. Altogether 14 interviews were conducted and a total of 15 users were
interviewed. This is due to one interview being conducted with two users present. The duration users
have used the system range from 1+ years to 10+ years. In addition to the 10 target cases, this study
gathered additional data from the ERP vendor. Hence, three Xledger consultants were also interviewed.
The consultants had a deep knowledge about the process of ERP updates in Xledger. This enabled the
researchers to elucidate knowledge and insights on how similar or different are the perspectives on the ERP
update process between the vendor and its client organizations. An overview of the informants and target
cases are provided in Table 1.
The names of the client companies are fictitious, and users are named by their job title due to the
informants’ right to be anonymous. Xledger’s client informants were asked about the date, which their
organizations went live with Xledger in order to get an idea of the respondents’ knowledge towards
the system. However, since not all users have been using the system since the go-live date, as they
have been hired afterwards or already had knowledge about Xledger when hired, information about
how long the users have used the system is also provided. In total, 21 users were contacted, and they
all agreed to participate in the interviews. Due to important deadlines, holidays, and interruption
during the planned interviews period, six informants later canceled.
Systems 2018, 6, 22 7 of 26

Table 1. Overview of informants and case organizations

Company Modules Years Respondent Interview

Respondent Industry Type Go-Live Year
(Size) Implemented Has Used the System Duration
Thor All except
System consultant Accountancy 2004 5+ 42 min
(Large) purchasing
Thor All except
Accountant Accountancy 2004 5+ 32 min
(Large) purchasing
Thor All except
Financial consultant Accountancy 2004 1+ 25 min
(Large) purchasing
Odin CRM, Accounting
System consultant Accountancy 2012 8+ 43 min
(Large) and Project
Balder
IT director Event planner All except CRM 2015 1+ 36 min
(Small)
Orm
IT manager Accountancy All except CRM 2014 5+ 30 min
(Large)
Chief Executive Loke
Accountancy All 2007 10+ 36 min
Officer (CEO) (Large)
Loke
Senior consultant Accountancy All 2007 10+ 38 min
(Large)
Loke
Business manager Accountancy All 2007 10+ 32 min
(Large)
Idun
System consultant Accountancy All 2008 5+ 33 min
(Medium)
Narve All except
Consultant manager Shipping 2012 1+ 43 min
(Medium) purchasing
Accounting,
Heimdall Invoicing,
Deputy director Accountancy 2016 1+ 30 min
(Medium) Project, and
Personnel/Expenses
Chief Financial
Frøya All except
Officer (CFO) and Kindergarten 2011 5+ 42 min
(Medium) purchasing
Accounting manager
Real Estate
Chief Financial Sleipner All except CRM
and Facility 2016 1+ 38 min
Officer (CFO) (Small) and purchasing
Management
Cloud-ERP
Consultant 1 Xledger - 2003 1+ 31 min
Vendor
Consultant 2 Xledger Vendor - 2003 1+ 25 min
Consultant 3 Xledger Vendor - 2003 1+ 40 min

3.2. Sampling and Data Collection

The selection of target cases was completed through a snowball sampling method [28]. First a small
group of initial informants, the employees at Xledger were identified and interviewed. They, among other
employees, were used to recruit informants from their client base to the interviews. When interviewing
client users, the same sampling method was used to gather six more respondents to the study. The target
cases were therefore not preselected in this study. Potential informants were contacted both through phone
and e-mail. All the interviews conducted at Xledger were face-to-face semi-structured interviews. Four of
the interviews with Xledger client users were also conducted face-to-face, and the rest were conducted over
the phone. All interviews were digitally recorded with the consent of all informants. The interviews took
place in Norwegian language, as this was the mother tongue of the respondents, and then all the interviews
were transcribed and translated to English.
It is important as a researcher to consider how many interviews are enough to reach data
saturation. Data saturation is reached when the aptitude to gather additional new information
is limited [29]. It is important to reach data saturation, as it directly impacts the content quality and
validity [29]. The researchers experienced data saturation after the first 10 interviews, that was
confirmed when the remaining interviews were not yielding additional significant information.
Therefore, recruiting more informants was assumed unnecessary.
Systems 2018, 6, 22 8 of 26

The data gathering process was conducted over a period of five weeks where the interviews were
evenly distributed over the period. Two iterations of interviews were conducted. The first two weeks were
the first iteration, and the last three weeks were the last iteration. The reason for these two iterations was
because of the opportunity to listen to the interviews in the first iteration in order to improve the interview
guide if necessary, before conducting iteration two. The data gathered represent a snapshot of today’s
process of updates in Xledger from the users’ and vendors’ point of views. The main focus in the interviews
was the users’ thoughts, feelings and experiences about the process. As the study examines the present
circumstances, based on the current opportunities and constraints the research type is can be considered
static [26]. The length of the interviews varied from 25 min to 43 min.

3.2.1. Interview Guide

In qualitative research it is a good practice to create an interview guide, as an overview of
the topics that researchers want to go through during the interviews [26]. To have the opportunity
for an open dialogue and to go in-depth within the topic, the interviews were semi-structured [26].
The interview guide was divided based on the research constructs identified in literature and guided
by the dimensions of ERP lifecycle framework presented earlier. The different constructs were
‘ERP Updates’, ‘ERP Training’, and ‘Change Management’. The primary goal during the interviews
was to understand what happens in the users’ organization when Xledger is publishing an update,
like which and how much training they conduct, and how they handle changing their work practices.
The individual in-depth interviews were carried out with the aim of examining how the individuals
interpret this phenomenon with their attitudes and beliefs. This provided good opportunities for
follow-up questions. In a column to the right of the questions, potential follow-up questions were
listed. Because of the interview guide, there were some systematic controls during the interviews [26].
The interview guide was sent to other peers in order to get feedback on the questions and clarity of the
guide. Some important feedback was received about the vagueness of some questions, which were
then enhanced. Later, the interview guide was again revised and enhanced after the first interview.
The revision contained some rewording to some questions and also a change in the order of some
questions, as the researchers learned what was more natural to ask consecutively during the interviews.

3.2.2. Data and Cross-Case Analysis

Data analysis and specifically cross-case analysis should preferably be used when searching for
patterns among the various cases [30]. These patterns can be mainly identified by using three methods:
(a) the selection of categories and scanning for within-group similarities coupled with intergroup
differences; (b) the selection of pairs of cases and listing of the similarities and contrasts between
each pair; and (c) the classifying of data by data source to extract distinctive understandings from
the different types of data collected [30]. After the data collection process was completed, the data
had to be electronically organized in order to be ready for analysis. Hence, the authors independently
used coding and tagging techniques, whereby the data gathered in each interview was classified
according to the topic of discussion, and according to the question and interview guide’s part in
which it is situated. In addition, the authors also added notes, comments, and interpretations on some
data segments. It was then possible to generate matrices, which can be classified by topic, interview,
and/or case. This process eased the data analysis, because it enabled the authors to view the data
related to the focus and theme per question and per interview. The authors also used a system of color
coding [31] to show similarities and patterns across the data. This system facilitated the visualization
and identification of patterns across the data.
With regard to the data analysis, several topics emerged during the discussions with informants in
the interviews. Across all cases, data was usually analyzed on the basis of topic and focus. For example,
extracting the data from interviews that are related to training after updates. In some other cases,
the theme in the data collected has emerged, which is natural in semi-structured interview setting.
Thus, two coding strategies were applied in this study. These can be classified as selective and
Systems 2018, 6, 22 9 of 26

theoretical coding [32], in which the categories were predefined and coded, and in other cases they
emerged from the data.

4. Findings
In this section, we present our main research findings in detail. In addition, an overview and
summary of the findings are provided in Table 2.

Table 2. Summary of findings

The Process of Change Cloud-Based ERP Cloud-Based ERP

Implementing Updates
The system is updated on
a Sunday, which fits
most informants.
The date the system is
updated could crash with
important deadlines.
5/15 think the frequency is
too often.
User Training Issues
Management Issues Updates Advantages Updates Challenges
The release notes are
There is a need for users
Users usually think All users always have the challenging to read.
training and to
that updates same version of the system. The users do not have
stay updated.
are improvements. Users get to focus on their control over what
Users plan their day
Users think it is okay core competences. functionality is coming.
around an update, or else
to change their work Users do not need to think There is a lack of
they work overtime.
processes, but it is a bit about updates. information
Users do not
frustrating and could Knowledge is transferrable before updates.
prioritize training.
affect their motivation independent of The new functionality
If you spend time on
in a short time. organization. implemented is not placed
training, you gain
Some think that they More possibilities for in a context. What is
an advantage.
have an opportunity to a good price. promised is
Few users train, test new
have an influence and Users do not need to make not implemented.
functionalities, read the
feel heard during the big changes after updates Some functionalities do
release notes and
update process, like with on-premise not work after updates.
participate in webinar
and some do not. ERP systems. The system is usually slow
as training.
after updates.

4.1. The Process of Implementing Updates

In Xledger: “The update is kind of a hybrid between changes in existing functionality and new
functionality” (Consultant 1, Xledger). Xledger patches small error corrections and small updates all the
time, but there are four fixed marked days a year where Xledger publishes an update. “The four standard
system updates a year follow the seasons” (Consultant 1, Xledger). There is a fixed routine that Xledger
follows when they update the system. The first process is testing. “We make sure to keep our calendars clear
the weeks before the update. Then we have more time for testing. If we do not do proper testing, we need
to spend more time on fixing errors after the update, which could have been discovered during the testing”
(Consultant 1, Xledger). “We have three test cycles, that goes over a two-week period, and is completed
on a Friday”. The system is then updated on the night between Saturday and Sunday, which leads to that
the system is unavailable for some hours during that time. When the system is updated, a document that
addresses new functionality, a so-called release note, is published in the system as a notification. In the
following week, a live webinar is conducted. “The webinar that Xledger provides is open for both partners
and users to participate. At the webinar, the changes implemented are visualized” (Consultant 1, Xledger).
The employees were asked about the flow of information during updates, and if users experience
challenges after updates. The informants from Xledger mentioned that client organizations are supposed to
contact the support department in Xledger via e-mail, but that is not always practiced. “That users contact
us consultants directly, well, it is pleasant to talk to them, but formally that qualifies as a procedure violation.
If it is a question that needs to be handled by one of our consultants, we see that some users just send
them e-mail directly instead of going through support. That is a bit of a shame, because we lose a great
opportunity for internal training” (Consultant 1, Xledger).

4.1.1. Date and Time of Updates

The study investigated what the users thought about the time and date the update is undertaken.
Most of the users answered that they were happy with the time. “They are implemented where it is the
least unfortunate. I have never had trouble delivering any public reports such as tax reports, so it has not
affected us negatively” (Deputy director, Heimdall). The CFO at Sleipner said that “We have not had any
situations where we have had to use the system that Sunday. If they had done it in a weekday on, it would
Systems 2018, 6, 22 10 of 26

be a crisis”. Even companies with users abroad has not been negatively affected by the time of the update.
“It prompts very clear when you log into the system, when the downtime is expected. I have never received
any complaint from our offices in either the United States or Singapore that there has ever been a problem
with the time” (Consultant manager, Narve). On the other hand, other users pointed out that the date the
updates are published is, or could be critical for their business. “It would be very nice if they avoid updating
the system close to deadlines that we face in accounting, like VAT termination. That is because when they
update the system, there have been some small bugs that change critical functionalities for us, and that
creates major problems” (Financial consultant, Thor). “When it comes to the date, if I answer on behalf
of our accountants, sometimes they place updates in connection with, for example, a VAT term and such
things, and things like that create frustrations” (System consultant, Idun). If Xledger updates the system,
and there are errors that affects the accountants, they could in worst case get fines from public authorities.
“You can say that they implement the updates when it suits Xledger, and not when it suits the accountants.
Accountants have deadlines to deal with, whether it’s payroll, VAT reporting, payment of taxes, assembly
task, or what it may be, and if you implement an update shortly against such a date, and there’s something
wrong, it can be quite fatal, because we cannot deliver to public authorities or something like that, and now
they have started with giving you fines for that” (IT manager, Orm). “Last time, the update was the 8 April,
and the 10 April was the deadline of delivering VAT” (Accountant, Thor).

4.1.2. Frequency of Updates

As mentioned, updates are conducted four times a year. The employees at Xledger were asked why
this is the procedure. “Well, that’s a good question, many vendors do it differently. As I understand,
our developers prefer it, but four times a year is not really the standard in the industry. Maybe the reason is
because that is how we have always done it, and it works quite well” (Consultant 1, Xledger). The users
were asked what they thought about that Xledger have four updates a year. Ten out of 15 users thought
that four times a year is an appropriate frequency considering that Xledger then can spread the number of
new functionalities during the year. “It is frequent enough. I think it is better with several smaller updates,
than a huge one. I need time to readjust as well, so four times is reasonable” (Accounting manager, Frøya).
“I prefer four times a year that you do not need to pay for. That is much better than for example twelve
times a year, because then you do not have time to understand and learn how to use the changes every
time” (Consultant manager, Narve). “It is nice. In our old on-premise system, we had updates three times
a year, and we normally skipped a couple, but in Xledger you cannot” (Deputy director, Heimdall). On the
other hand, five informants thought that four updates per year is a bit too frequent, because they do not
have time to pay attention to what is new and learn how to use it. “It might be too often, and the reason for
that is that when there is an update, there is a lot of new things to grasp, and when that happens four times
a year, it is a lot. They should launch less functionality each time. Then it would not be so big changes in
our work processes” (IT director, Balder). “It is too often for us to learn what is really new, and learn how to
use it each time” (IT manager, Orm).

4.1.3. Users’ Influence on New Functionalities during Updates

What type of changes and functionalities that are prioritized to implement in each update is
determined by “Commercial considerations from important existing or new customers, our strategic
focus, and the thoughts of our developers and representatives from the management group”
(Consultant 1, Xledger).
The users were asked about their possibility to have an influence on updates and how they are
conducted. The users had mixed opinions regarding this issue. ”Some of the functionalities we have
requested are implemented and some are not. These are functionalities that we do not pay for, they are
a part of the package. If there is something we absolutely need by the next update, then we could pay
for it and jump the queue, but only if it is a functionality that could benefit all users, if not, it will not
be implemented. That is quite annoying from one point of view, but from my point of view it is a good
thing, because it lets the system gets updates regardless of the different users’ set-up” (IT director,
Systems 2018, 6, 22 11 of 26

Balder). Due to the lack of an information system consultant at Thor, they feel that it is difficult to have
an influence. “That update happens if you like it or not. So, you just have to accept that. As you do not
have so much information about the course of development, it is a bit difficult to have an influence on
choices that are made”. The CEO at Loke said that “I feel that we have the opportunity to influence
Xledger, at the same time as I do not feel like we actually influence anything. It is mainly the founders
that decides what is implemented and not. We have been so long in this game that we know how it
works and try to influence whenever we can”. The business manager at Loke said that, “Yes, we have
suggested some improvements that have been taken into account. Unfortunately, it is easy to forget
what you have got that you requested, because then you are happy. It is much easier to remember
what you suggested in 2008 that you still have not got”. Even if users have the possibility to have
an influence, it does not necessarily mean that they feel heard, so that was the next question they were
asked. Once again, the answerers were split: 8/15 said yes, 3/15 said no, and 4/15 where mixed:
both yes and no. “Right after we went live Xledger was very responsive, now it is more difficult to
get through.” (CFO, Sleipner). “Yes, but it is a fight. Sometimes things happen, and sometimes they
do not” (Senior consultant, Loke). “Yes, I have a good dialogue with Xledger” (IT director, Balder).
“No, we are not heard. I feel heard and prioritized if I talk to the CEO directly. I do not know if the
other employees do not want to listen, or if they have been told not to” (Business manager, Loke).

4.2. User Training

The employees at Xledger were asked to what extent is it important to them that users pay
attention to the updates implemented. “It is very crucial, because we wish that the new functionalities
get adopted. We use a lot of resources and time on the update, so we are happy when users take
advantage of new functionalities that are implemented” (Consultant 1, Xledger). As a follow up,
Xledger informants were asked about the attitudes they think users have towards updates. In general,
Xledger informants agreed that “some users are interested and want to know what we will be included
in the next update and so on, but most of the users have a non-existing relationship to the updates.
Reading the release notes ends up very far down on the to-do list among users. My experience is that
quite few users actually read the release notes, they just close it whenever it pops up in the system,
and most users do not participate in the webinars either” (Consultant 2, Xledger).

4.2.1. Need for User Training

According to the majority of Xledger informants, there is a need for training. “If you look at
the system in 2013 you would not recognize the user interface, as it would have different colors and
fewer options” (Consultant 2, Xledger). All the employees had experiences with users who ask for
functionalities that are already implemented and available in the system. “I often experience that users
says “Oh, I wish you had this and that functionality”, “But we got that three years ago?”. “Some users
do not pay attention, and that is quite unfortunate” (Consultant 3, Xledger). The first question the
users of the system were asked regarding training was if they thought it was necessary to stay updated
on what has been implemented after an update.
Most of the users (14 out of 15) agreed that it is necessary to stay updated on new functionalities to
be able to be effective. “Yes, I would say that. It is important to make sure that we are effective” (CFO,
Sleipner). “Yes, it is necessary to be able to work effectively in the system, but you do not have time,
so it’s like shooting yourself in the leg when I do not do it” (Accountant, Thor). The users were asked
if they planned their days around updates. The majority of the users answered that they did, but the
answers ranged from that no time is cleared in their schedule to handle an update, to that they cleared
a week. “On Monday after the update, I try to limit other things, so if there are any trouble, I have
time to handle it. I always use the first hour on Monday, and then it depends on the update what I do
after that” (System consultant, Thor). “I clear the week considering planning and testing of the system,
and then I need to use time to understand the changes which can be challenging, but I love new things
so it’s okay” (Consultant manager, Narve). Two users mentioned that in case of system error after
Systems 2018, 6, 22 12 of 26

an update, they complete important tasks prior to an update. “I do not really plan my day around
an update, I just run important processes like payroll and stuff” (Accountant, Thor). Those who did
not clear their schedule prior to the update, were prepared for working overtime instead. “Normally,
I do not clear my schedule after an update, I think that is something Xledger should have control
over. But, I have got some surprises before, so then I just have to adjust my days and work overtime”
(Deputy director, Heimdall).

4.2.2. Prioritization of Training

The next question the users were asked, was if they prioritize training after updates. Eight of 15 of the
target companies’ informants said that they do prioritize training. “Yes, it is highly prioritized” (Accounting
manager, Frøya). The rest of the users said that they do not due to different reasons. “I do not think that it is
prioritized as much as it should have among users” (System consultant, Thor). “We spent a lot of time on
training when we went live, but after each update we do not do anything in particular. We seldom test new
functionality, we just try it sometimes and see if we understand how it is supposed to be used, so the short
answer is no” (CFO, Sleipner). Some users defended the lack of training due to lack of time. “No, we do not
have time for that” (Accountant, Thor), and others mentioned that they do some training, but not enough.
“No, not sufficient” (CEO, Loke).

4.2.3. Advantage of Training

The users were asked if they thought that those who spend time on training receive an advantage.
14/15 of the users said that they think they do. “Yes, if there has been a change with leads to that you do
things wrong, you waste a lot of time trying to complete the task” (System consultant, Thor). “Yes, it is
an advantage to know which options you have” (CFO, Frøya), “No, I do not think so” (Accountant, Thor).

4.2.4. Training Completed by Users

The users were asked if they do any kind of training considering updates. “Training is something
we do sporadically, both group-training and self-training” (System consultant, Idun). Some do nothing:”
There is no time for that. It’s not so much stuff that is important when you start reading the release notes,
so you drop it. Then you hope someone will pick up what’s new. I just register that there is an update,
and wait for the mistakes” (Accountant, Thor). Some test what have been implemented, to check that
there are no errors, and to understand the impact of the updates. “I sit down and read the release notes
and note what is relevant and what’s not and test it. Based on how much errors there are with the update
depends on how much time we get to test the system” (System consultant, Odin). “We normally test the
new functionality to see of if works, because that is not always the case” (System consultant, Thor). Many of
the users said that they participate in the webinar. “We participate at the webinar after the update, if we have
time. If we do not, then we just assume that things are as it did before” (IT director, Balder). “The webinar is
our only way of teaching how to really us the system” (Accounting manager, Frøya).
Many of the users said that they take time to read through the release notes. “You may want
to read the release notes before going to work on Monday morning, so you are a little bit prepared”
(System consultant, Thor). “I always read the release notes. If there is anything relevant I look further
in to it” (CFO, Sleipner). “No one that I know read that release notes. I know one person which is
a work-addict, and she might read it at home, for fun” (Accountant, Thor). Talking on the behalf of
accountants, the system consultant in Thor said that: “I do not experience that accountants use much
time on training. I ask them to at least read thorough the release notes, so they can discover if there
is anything interesting. Normally I ask them “Have you seen this?”, and they say “Yes, I think I saw
something about that, but I have not had time to read about it. They have busy days, so training is not
prioritized. I think they would gain an advantage, spending 15 min reading it” (System consultant,
Thor). In total, 10 of 15 informants mentioned that they conduct complete training after updates,
while 5 asserted that they do not. Seven out of 15 said that they participate in post update webinars,
Systems 2018, 6, 22 13 of 26

and only 7 informants confirmed that they read the (update) release notes. In addition, only 8 out of 15
stated that they conduct post update testing.

4.3. Change Management and Process

The Xledger informants felt that the users are resistant to change when it comes to updates.
Consultant 2 at Xledger said that “in general, the accountants have developed one way of conducting
their work in Xledger, and are not interested in learning new ways of conducting their work—even
if it would speed up the process. I they feel that the time they spend on learning the new features is
longer than the time they would save”.

4.3.1. Need for Updating the System

When asked, all the informants from our target cases agreed that there is a need for updating the
system. “Yes, both because of legislative changes which you have to follow and because I experience
Xledger as a young system. Xledger aims to have advanced functionality, so it is almost endless
of options. It will never be fully developed. We just have to hang on” (CFO, Sleipner). “In that
moment, you stop developing the system, your done in this industry. So, yes, it is a natural need
for developing the system” (Senior consultant, Loke). One users explained the need for updating
the system, with how things we use in our private life also have continuous updates. “It is a part of
the concept of a web-based world which we live in. We do not work on different versions anymore.
From our private life, we are used to that there are constant improvements and changes on applications
we use on our phone for example. That is why I think that’s normal for an ERP system as well” (Senior
consultant, Loke).
The users were also asked if they think that the updates are improvements. The most common
answer was “yes, usually”. “Yes, usually, defiantly” (Business manager, Loke). “Yes, usually, but
there are some exceptions of new functionality that is not improvements” (Senior consultant, Loke).
“Yes, normally it is. Some new functionality that is implemented is maybe useful for those who need it,
but we do not” (System consultant, Thor).

4.3.2. Change of Work Practices/Processes

The employees were asked if they think users are willing to change their work practices. “Let’s say
that there is a change in a screen image, and that the users have got used to one way of doing their work,
which is not in accordance with our best practice, and is counterintuitive. If we make a reasonable
change, it might not work for them anymore and then they have to reconfigure their brain in a way.”
(Consultant 1, Xledger). To start with, the users were asked about if an update have led to that they
need to do a process in a different way than before, and if yes, what they felt about that. Most of
the users answered yes (13/15), and felt that it was okay. “Yes, but just smaller stuff” (Accounting
manager, Frøya). ”Yes, but as long as it is a positive change, I do not mind” (System consultant, Odin).
“Yes, but you just get used to work that way. Xledger forces us to work a certain way, so we just have
to pay attention to it. It is Xledger who runs the system, we do not get to control it the way we want to,
but we just have to live with that” (IT manager, Orm). Some users said that they feel that it is annoying
to keep changing, but they manage. “It’s like ‘Ok, I need to change this’, then you do it and forget
about how angry you were at that time. The challenge for Xledger is that accountants hate to change,
they might even be that type of persons who is most resistance to change. Accountants are sitting
with landlines on their desk. I do not think that it is an easy job for Xledger to change the way people
work. It is annoying in the moment when you have to change, and then it goes over” (Consultant
manager, Narve). The users were then asked if that they have to change their work processes affects
their motivation. Most of the users answered yes. “Yes, absolutely. It’s frustrating when I do not
agree with the changes implemented” (System consultant, Thor). “Yeas, in short-term it does. I think
it depends on how positive you are towards the system initially. If there are some stuff that annoys
Systems 2018, 6, 22 14 of 26

you, changes do not really help in a positive direction” (Senoir consultant, Loke). “Totally, it is really
annoying, but as long as I can figure it out I actually think it is a bit fun” (Accountant, Thor).
The users were thereafter asked if they felt that collogues and users are willing to change their work
processes. Mostly, they answered “it really depends on the person”. “Some users are used to work with
Visma, so there are used to do things in a different way, and think that everything was better before.
Others are positive and look forward to work more productive” (System consultant, Idun). “People are
creatures of habit. Often users acquire a certain way of working with the system when they start using
it, and as the years have gone by it might would have save both costs and time. But that requires that
the user and their organization is willing to use both time and resources on that. But I know users in the
entire specter from those who never changes their work processes, as they are happy with the way they are,
and do not see the changes as improvements, and those who change them immediately. I think that smaller
organizations tend to be the first type” (Senior consultant, Loke).

4.4. Cloud-Based ERP System Update Advantages

During interviews, none of our informants would prefer an on-premise system. For example, one of
the informants felt like cloud computing is the future, based on its quality and price. “I’ve worked in several
ERP systems, and definitely mean that the possibilities in a cloud-based system compared with the price,
is definitely the best you can get on the market today. An on-premise ERP system costs you the shirt in
proportion. SaaS solutions are the way to go” (System consultant, Odin). With a cloud-based ERP system
users do not need to face the same challenges as when using an on-premise system. “The challenge with
on-premise ERP systems is that when the vendor publishes an update you receive an e-mail, then you think
‘This is something I should do’, but you do not. It costs a lot of money, and takes a lot of time, and then
you need to understand the changes. Then time passes by, and there is a new update, and a new update,
and in the end, you have skipped a couple of updates, and that is horrible and a total chaos. So, to have
forced updates that you do not control is a bit ambivalent, but I feel that it is an advantage, because you do
not need to make the big changes, and it is an advantage that the system is always updated” (IT director,
Balder). Another advantage that was stipulated is that all the system users have the same version of the
application. “If I want to talk to a friend of mine that also uses Xledger, then they have the same version of
the system, so all knowledge regarding the system is transferable independent of organization. That makes
it easier to exchange competence” (IT director, Balder). “What’s nice that we do not need to think about
with a cloud-based system is that updates are done continuously, and that the system is always on the latest
version. We have other systems that are on-premise were we need to upgrade them ourselves which leads
to downtime of the system during work hours” (CFO, Sleipner). Also, users do not have to pay attention to
the accounting industry, as Xledger does it for them. “We do course activities, which we are excellent at!
We do not do accounting. That is not what our core competence is, there is someone else that is better than
us at that, so we let someone else take care of it. It is a typical example of where we can lean on someone
else’s best practice, instead of following markets that are not really relevant to our core business” (IT director,
Balder). The last advantage that was clarified during the interviews, was that the users do not need to think
about the updates. “It is clearly an advantage that you do not have to think about the updates. As a user
of a cloud-based service it just happens, and then, well, it should have been working” (System consultant,
Thor). “It’s really fabulous that you can get up on Sunday morning, have a coffee, read the newspaper and
then the system is updated, it doesn’t get better than that. I think it works excellent” (IT Manager, Orm).

4.5. Cloud-Based ERP System Update Challenges

A lot of challenges in relation to updates were discovered when talking to the informants. In this
section, all the challenges that were discovered during interviews are listed. When talking about
the release notes, 13/15 participants mentioned that they had challenges with it. The employees at
Xledger also talked about the problems with it. “The challenge with the release notes are that it is only
relevant for some types of users every time, and they need to read the entire document to see if there is
something in there that is relevant for them” (Consultant 2, Xledger). “I understand that users do not
Systems 2018, 6, 22 15 of 26

prioritize time on reading the release notes, they are pretty boring and there is not much stuff that is
relevant for them, maybe just a small sentence” (Consultant 3, Xledger).
The first challenge the users mentioned was that some release notes contain too much information.
“We take time to look through the release notes, they are quite complementary, but the load of
information is too big” (IT director, Balder). The release notes are published as a pop-up in the system,
which is shown when you first log on to the system after an update. “When you log into the system
you normally have a task to complete, so you do not have time to sit down and read the release notes”
(IT director, Balder). The majority of the users feel that it is hard to understand, so understanding
what changes has been done is challenging. “It contains a lot of information and it is very technical.
So, it tells what has been done, but very little how it can be used. So, it is up to us to interpret and
understand it. Even if you have been working in the system for almost 10 years, you might not
understand the range of a small change made in the system” (CEO, Loke). As this CEO also mentioned
here, is that many feel that the release notes are not written for them to understand. “It would be
a good idea for Xledger to be aware of the audience when they disclose what has happened in the
release notes. We are not just IT people and developers, this is the accounting industry, and we have
a different way of seeing and understanding things. It does not seem as if they’ve thought about that at
all” (Financial consultant, Thor). Some users also said that even if they spend time on understanding
changes implemented, its normally not possible to start using it. “You see that they spend some time on
writing the release note, but sometimes I do not think it is explained well enough. 75% of cases where
we try to use new functionality, an email goes to support. ‘No, it’s not launched yet’ or ‘Oh, you have
to do this and this for it to work’” (System consultant, Odin).

4.5.1. Users’ Control over Updates

Twelve of 14 users said that they did not feel like they have control over the updates in Xledger.
The users mentioned that due to the lack of information beforehand, it is not possible to have control.
“Not very much (in control). The webinars describe what have been implemented in the update,
but the challenge is that it recorded after the update. It is kind of like saying that ‘Well, now you have
crashed the car, just so you know there was a stop-sign there, we did put it up’. It is like driving the
car with the rearview mirror, you control after what have happened” (IT director, Balder). Users said
that they have no control over what is going to be implemented in the next update. “No, we do not
have control. The updates are often a surprise gift ‘What do we get this time, what did they manage
to include?’. Things that are said to come, do not. And new functionalities are going to have both
positive and negative effects, but we cannot prepare for anything” (CEO, Loke). “It should have been
a little bit more transparency regarding further development of the system. What are they thinking,
and what will be focused on this year and stuff like that” (CFO, Sleipner). Another user mentioned
that not being in control is no problem. “I do not think that we are not in control is not a problem at all.
If you have used the system for a while, you know there will be four updates, so that is no stress at all”
(Financial consultant, Thor). The employees were also asked about this. “A challenge is that the users
do not know what will be implemented in the next update, and what will affect them and not before
after the update, but we try our best, and I feel that we are getting better and better” (Consultant 1,
Xledger). Some informants mentioned that they hear about new functionalities randomly. “Sometimes
it’s simply that things change behavior in relation to how it uses to do” (IT director, Balder). “There is
something that requires that you to turn on some features to see the new things that have happened,
and then it’s very difficult to discover new things other than random. ‘Oh, what’s this?’, so I send
an email to support about ‘What does this do?’or ‘No, this function cannot do anything yet’” (System
consultant, Thor). The flow of information between Xledger and users was discussed during interviews
with employees at Xledger. “I think that users feel that they need more information” (Consultant 3,
Xledger). The reason for the assumption the employee had was both because of discussions with end
users, but also the opening rate on Xledgers newsletter. “In October 44% of everyone who received the
newsletter opened it, and in September 49% opened it. Based on opening rates from other companies
Systems 2018, 6, 22 16 of 26

I have worked in, this is really high” (Consultant 3, Xledger). So, Consultant 3 at Xledger would start to
provide more information to the end users, if there was something that should be changed. Consultant
3 at Xledger views were discussed with the other employees interviewed. The others agreed that the
information regarding updates could defiantly be enhanced, but the problem is that “you could end
up in a situation where you have promised one thing but fail to deliver. Xledger make an estimate
on what will come in the upcoming update, but the world is a chaotic place, and it is unfortunate to
promise something that do not happen, if we for example make incorrect estimates” (Consultant 1,
Xledger). Consultant 1 at Xledger states that is the main reason why external communication is not
a main focus.
After an update, some users said it was difficult to understand what has been actually
implemented. “The problem is that it just seems like they are just coughing up a bunch of things,
‘Now we’ve changed this and that’ without giving any context and explanation. It is therefore difficult
to make use of the release notes and these webinars. When they come with a new change, it would be
better if they put it in a context. Not only ‘Now you can tap this button’, but we must have a context,
so we can use it” (Financial consultant, Thor).

4.5.2. Functionalities That Were Promised but Not Implemented

Some frustrations associated with updates are that “Xledger announces what functionalities are coming,
but some of them might not be implemented before the next update. It creates more frustration than it is
useful” (Business manager, Loke). Many of the users felt like Xledger is not sure about what will be actually
implemented in the nearest future. “We just have to wait and see what is really going to be implemented
during the update and what won’t, as it doesn’t seem like Xledger know it either. I’ve grown so old that
I do not let it affect my sleep anymore, but it’s frustrating” (System consultant, Thor).

4.5.3. Functionality Does Not Work after Updates

The users said that after an update they often experience that certain functionalities do not work
anymore. That yields for both critical and cosmetic functionalities. “It’s terribly annoying. I cannot submit
the settlement now, since now have made a change that does not work, it works in two days . . . when
you are actually going on holiday, and do not have time to wait for two days. Thinking that you cannot
plan everything in case of errors after update, that’s not how it should be” (System consultant, Thor).
The users also said that a continuing problem is that one new implemented functionality often leads to
another old functionality either stop working, or do not work as it has to for them to conduct their work.
“After each update, a list of errors is published, sometimes it’s long, sometimes is short, but it’s always
errors after an update. There is new functionality implemented that negatively affects existing functionality.
There are often functionalities missing, that were said to be implemented but weren’t. It is even on the
release notes, and it says that it will be implemented during the next two to three weeks, but it never comes”
(IT manager, Orm). Twelve of 15 of our informants say that these errors have led to that they have been
hindered in their work. “It is clear that an update has hindered my work. As I work in support, I have
to drop everything else I work with. For example, because hourly reports did not work, I had to make
sure that 200 employees were paid in other ways than by hourly report. It is clear that we really notice
when things stop. We, who only use Xledger, are almost 100 people in a coffee break when it happens”
(System consultant, Thor). Errors can affect the performance of users. “One usually remembers the last
update, and there were a lot of bugs and ‘cabbage’ with it. So, what I think of when I talk about updates is
that there was a lot of errors, and that is always boring. Both performance went down, and we had a lot
of challenges” (IT director, Balder). Some of the users feel like Xledger does not always take into account
how they use the system. Therefore, some system updates have led to critical functionality for them have
stopped working, without them having a saying in the process. “Functionality that are do not seem to be
fully thought through have been implemented, and that have led to that the system actually works worse
for many users. They do some changes in the system without investigating how the different users use the
functionality, which leads to that the changes they implement get negative consequences they didn’t think
Systems 2018, 6, 22 17 of 26

of” (System consultant, Thor). One reason for the errors, could be that Xledger doesn’t test the functionalities
enough beforehand. “It is a challenge when a functionality do not work after an update. I understand that it
is difficult for Xledger to test the functionality in all possible scenarios before an update, especially because
they start testing in such a short time prior to an update, which I know that they do, that it is impossible for
them to get through everything. They do not have enough resources, maybe they should crowd-source
it” (System consultant, Odin). Almost all users argued that if Xledger had used more time on testing new
functionalities before publishing them in an update, they would avoid this problem.

4.5.4. The System Is Slow after Updates

The users mentioned that downtime of the system have not been a huge problem after an update,
but that the system is usually slower. “There has not been so much downtime of the system” (System
consultant, Odin). “There have been a couple of times were the system is slow, especially Monday
morning after the update, but this is not counted as downtime of the system” (CEO, Loke). “Downtime
of the system have not affected me, it is normally on Sundays and in the evening, but the system is
almost always slow after an update. The ‘lineman’ which we call him/her, appears when the system is
to slow to work in, and s/he is very often visiting us after an update” (Accountant, Thor).

5. Discussion
This study aimed at exploring and assessing the technological, organizational and contextual
factors affected by the forced updates within the evolution phase of cloud-based ERP systems among
Norwegian organizations. A questionnaire was created with the objective of identifying the issues that
were considered most beneficial and challenging within this context. In this section, we will discuss
the factors that are considered as advantageous and disadvantageous by our informants.

5.1. The System Updating Process

The findings show that the users and vendor similar perceptions about how the updates are
published. The vendor’s system in this case study is updated in alignment with the year’s four seasons,
four times a year, and is always updated on a Sunday. However, there are several points in the
process of updating the system, which the users and the vendor view differently. To be able to get
an overview of these differences and to discover the real challenges, Table 3 presents an overview on
these variations.

Table 3. Vendor and user contrasting statements

Vendor Users
The date of updates
The dates of the quarterly system updates follow The date the system is updated can crash with
the seasons. important deadlines for accountants.
The number of updates per year
10/15 of the users think that four times a year is
The system is updated four times a year because it
good. The rest of the users think that too much
fits the developers and it works well.
functionalities are implemented in each update.
The opportunity to have an influence
Important existing, and new customers are a part of
Users have split opinions regarding their
the group who influences what are implemented in
opportunity to have an influence. Some users do
an update.Users are supposed to go through
not feel heard through support, so they
support if they have a suggestion or an issue,
communicate with Xledger employees directly
but some directly talk to employees instead,
when they have issues.
which is a routine violation.
Systems 2018, 6, 22 18 of 26

Table 3. Cont.

Vendor Users
Users’ relationship with updates
Users clear their schedule before an update, and
Users have a non-existing relationship to updates those who do not, are usually prepared to work
and do not pay attention towards them. overtime. Users do not know what is coming in
an update, so it is hard for them to prepare.
User training
Few of the users read the release notes. 13/15 had different issues with the release notes.
7/15 users participate in the webinars. The webinars
Users do not participate in webinars. are not about what is coming for the users to prepare,
it is about what has already been done.
Users think that the time they spend on training
14/15 users think that spending time on training
and understanding new functionality are longer
should gain them an advantage.
than the time they would save.
System testing
We test the system in three test cycles that last for A challenge is that Xledger do not spend enough
two weeks before the update. time on testing.
Change management
Users are resistant to change and are not interested Users need to change their work processes due to
in learning new ways of conducting their work. updates, and most think it is okay.
Flow of information
Xledger announces new functionalities that are not
Our main focus is not on external communications,
there before the next update, or not at all. Users are
as we do not want to promise something that is
uncertain regarding what will be implemented in
not implemented.
the future, which they find challenging.

5.1.1. The Date of Updates

Xledger is updated according to the four seasons a year. The findings show that the date the
system is updated can potentially be challenging for users, if the system is updated at the wrong
date. Users that are accountants have important public deadlines to deal with such as summary
reports, payroll payments, VAT, and payment of tax. It is therefore important for the accountants that
Xledger do not put an update in relation to one of those deadlines, as it can lead to that accountants
are hindered in delivering these reports. In the worst case, the accountants are given a fine from the
authorities. To be able to solve this challenge the users said that Xledger should get to understand the
accountant industry a bit more, so they have an understanding of the effects of putting an update close
up to a deadline. Thereby, Xledger has the knowledge and potential to update the system without
interfering with the accountants’ important deadlines.

5.1.2. Number of Updates per Year

Informants at Xledger stated that the system is updated four times a year, because it works
well and fits their developers’ strategy. Five of 15 of the users saw this as a challenge, as they
think it is too much implemented each time, so they do not get time to readjust or spend enough
time on understanding the changes and maintain an effective way of working. This coincides with
Calvert and Seddon’s findings, that ERP updates can lead to a decrease in users capacity to work
effectively [7]. Based on the findings, it seems that updating the system four times a year is appropriate.
However, it seems to be an idea to implement less functionality in each update, for the users to
maintain their effective way of working in the system.
Systems 2018, 6, 22 19 of 26

5.1.3. The Opportunity to Have an Influence

According to Xledger, important customers have the possibility to influence what will be implemented
during updates. The users felt like they have some possibility to have an influence, but that due to the lack
of information about the course of development it is hard to be able to give input. The users also highlighted
that it is harder to remember the functionality they have requested that have been implemented, and that
they remember better what has not been implemented after a request. Most of the users said that they have
got some functionality that they have requested, but they would also like to be included, or at least get an
explanation for why things are done. In the end, users highlighted that they do not pay for the updates and
see it as an advantage that Xledger only implements functionality if it can be beneficial for several users.
The findings show the majority of the users (8/15) feel that they are heard and have varying influence on
the future functionalities, but some do not (3/15). Another challenge for Xledger, is that users do not use the
support system correctly. Instead of contacting support, they contact employees directly. According to the
users this was because then they have a better opportunity to be heard. A solution would seem to be to
provide more information about the process of updates, but then again as an employee in Xledger said,
Xledger could end up in a situation where they have promised new functionality that they fail to deliver.

5.1.4. Users Relationship towards Updates

Employees in Xledger claimed that most users do not have a relationship towards updates.
The findings however, show that many of the users clear their calendar before an update, and those
who do not are prepared for working overtime. Several users argue that it is hard to prepare for
updates, as they have no information about what is coming, it is like a surprise. Even functionalities
that are promised are sometimes not implemented. The majority of users (10/15) put some effort into
some form of training such as reading the release note, testing, and participating in webinars.

5.1.5. User Training

According to Gartner, each hour of effective user training saves the organization a worth of five
hours [23], and well trained users can better take advantage of the system [20]. One employee in Xledger
claimed that users do not think that they gain an advantage of training. However, 13/15 of the users think
that those users who spend time on training receive and advantage. They highlighted that training makes it
possible to spend less time doing things wrong. Another informant at Xledger claimed that most users do
not participate in the webinars, which present the news of each update, but 7/15 of the users for this study
answered that they participated in the webinar.
Calvert and Seddon [7] distinguish between formal and informal training. The findings show
that users do both. Most of the users said that they participate in webinars provided by Xledger that
qualifies for formal training. They also mentioned that they ask super users at their work if there
is anything they should pay attention to after an update, which is informal training. As training
documentation, Xledger provides release notes for the users to read after each update, which is also
a form of formal training, but state that few of the users actually read it. The release notes were
frequently mentioned by users as a challenge. Some of the users said that the release notes contain too
much information, so they stop reading it because there is very seldom information that is relevant
for them. Some users also have trouble understanding the content of the release notes, even when
working in the system for over 10 years, because it is too technical. According to literature, Xledger
provides untailored training material (release notes) as recommended by Scott [23], but the users seem
to argue that they want training material tailored for their organizational context, as recommended
by Wheatley [22]. Furthermore, the users feel that the changes that are explained in the release notes
should be put in a context for them to understand the value of the changes. Lastly, the release notes are
distributed as a pop-up in the system, which some users claimed to be a challenge as when they log on
to the system they have a task to do, and do not have time to read it then. A possible solution could be
to also distribute the release notes in another arena such as e-mail or as a news on their homepage.
Systems 2018, 6, 22 20 of 26

Furthermore, the release notes could be tailored by the roles of the users or divide it by the modules,
but at least it should be put in a context and written by someone with a non-technical background for
users to understand it better.

5.1.6. Change Management

According to literature, a challenge often phased with training is that the users do not see the need for
changing their work processes and that users are often resistant to change [21]. This was also the opinion of
one of the employees at Xledger. The findings show that most of the users feel that it is okay to change their
work processes, but they are frustrated at the moment when they have to change. When users were asked if
they think other users of the system is willing to change, they answered that it depends on the person and
that they know both users that like to change due to the advantages, and some that prefer to work in the
system as it is today, and do not view changes as an advantage in general.
The users were asked if they think that the updates are improvements. The majority answered
that they usually are, which could explain why they are not so resistant to change as Xledger think.

5.1.7. System Testing

Xledger tests the system in three cycles over a period of two weeks to see if the new functionality
works. The findings show that a challenge for users during the update process is that the system is
not tested enough. This leads to that new functionality does not work, and existing functionality that
worked previously do not work after updates. Claybaugh et al. [18] argue, the users could be shielded
from ripple effects of updates, if the vendor tests the updates properly beforehand. The users claim
that Xledger does not test the new functionalities in the way they as users use the system, so an update
may have negative effects on the users, which Xledger did not think of. Findings show that this has led
to users being hindered in their work. In addition, some informants stated that it is understandable
that they experience ripple effects, because Xledger may not have enough resources to test every
possible scenario in the system. A possible solution could be to provide more resources for user testing,
by for example inviting partners (Xledger resellers) to test the updates.

5.1.8. Summary of Challenges and Potential Solutions

Based on the findings, an overview on the main challenges of updating the system and the
potential solutions are listed below.

• The date of updates is sometimes challenging for accountants using the system. A possible
solution is for Xledger to talk to accountants, get informed about their deadlines, and update the
system at a time that not interfere with them.
• The size of the updates is according to half of the users challenging to maintain their effective
way of working. A possible solution is to introduce fewer new functionalities in each update.
• Lack of information about what is going to be implemented in the next update makes it hard for
users to both be prepared, and to have an influence. It is difficult to provide a possible solution for
this problem, as it is a challenge for Xledger to provide this information, as they might promise
a functionality that they will fail to deliver.
• Release notes are written too technical for the users to understand, so the users do not always
understand the meaning of the new functionality as it is not put in a context when presented.
Xledger provides an untailored release note, while the users request a tailored version for them to
better understand it. A possible solution could be to tailor the release notes to the different roles
of the users, or divide it by the modules, put the new functionality in a context and hand over the
job of writing the release note to someone with a non-technical background.
• Some functionalities do not work after updates. A possible solution could be to dedicate more
resources for user testing by, for example, inviting super users or partners to test the system prior
to updates.
Systems 2018, 6, 22 21 of 26

5.2. Users Are Not in Charge of Updates

The findings show that there are both advantages and disadvantages with not being in charge
of updates.

5.2.1. Advantages
According to Duan et al. [11], not being in charge of updates could potentially lead to less
planning and testing activities by the client organizations. However, our findings suggest that users
still have to plan before updates and test if the new functionality actually works. Duan et al. [11]
also state that not being in charge of updates leads to lower costs. The informants also considered
this as a huge advantage, because even though there might be challenges with the updates, they are
free. Another advantage, which was not identified in previous research, is that all users always
have the same and latest version of the system. This means that users do not have to think about
updating the system, like they do in an on-premise ERP system where users tend to typically skip
updates. This leads to users focusing on their core competences as also argued by [12], instead of
paying attention to for example, law changes in the accountancy industry, as these are automatically
implemented in the system by Xledger. In literature, another advantage is that new versions of the
system are regularly published [18], and the updates are more frequent [12]. Most of the users agreed
to that, but at the same time it leads to them being unable to use the system as effectively as before,
as a new update typically means more new things to learn. Frequent updates could therefore also be
interpreted as a disadvantage. Other advantages found include the system always being updated,
which means that the changes that the users need to do after each update is typically smaller than
with an on-premise system that is seldom updated. If users know other users of Xledger in other
organizations, they can share knowledge and information regarding Xledger as they are always on the
same version. A summary of advantages is listed below.

• Less planning and testing;

• Lower costs;
• All users always have the same version of the system;
• Users get to focus on their core competence;
• Updates are regular;
• The system is always updated;
• The changes that the users need to handle after each update is smaller;
• Sharing knowledge about the system is easier.

5.2.2. Disadvantages
The findings also show that there are some disadvantages when users are not in control of updates.
The first main challenge is the lack of information about what is going to be implemented in the future,
so users are not prepared for the changes, and have to handle them after they are implemented. The lack of
knowledge in relation to future development of the system makes it hard for the users to have a certain
control over the updates. The second main challenge is that the users do not decide when the system is
updated. This is a challenge because after the updates, the system might not work properly, and previous
work processes might have changed. This leads to users either being unable their work or having to spend
time on learning how to do their work in a different way. As accountants have important deadlines to deal
with, they do not have time to do either of those things when an update is published close to a deadline.
They have to deliver documents to the authorities such as the value added tax (VAT) terms, or other things
like payroll, and if they do not deliver these documents in time, they could end up getting a fine from the
authorities or that the employees are not paid for their work. The main disadvantages are therefore:

• Lack of information about the future plan of updates;

• The users do not and can’t decide when the system is updated.
Systems 2018, 6, 22 22 of 26

5.3. The Evolution Phase

The ERP lifecycle framework was initially developed for on-premise ERP systems. Based on the
findings of this study, this section will discuss how the evolution phase and its dimensions can fit with
a cloud-based ERP system.
In Figure 2, a visualization of the evolution phase and its dimensions in a cloud-based ERP context
is illustrated. As this study has only focused on the evolution phase, the figure is not an addition or
a change of the ERP lifecycle model, but rather an argument for the importance of change management
and training in particular after system updates. In the existing literature, studies focusing on training
usually focus on pre-implementation phases, and tend to ignore the need for re-training during
the post-implementation stages. In addition, in the ERP lifecycle framework, the dimensions are
visualized like layers to the phases that resemble a traditional waterfall model. The dimensions can
therefore be interpreted as non-iterative. The findings in this study suggest that, in a cloud-based ERP
system, the dimensions are iterative and are therefore visualized like a spiral model to emphasize the
continuous update process. During the evolution phase, there are more frequent updates in contrast to
on-premise ERP, which triggers the dimensions that are called ‘training’ (previously ‘people’), ‘change
management’, and ‘process’. This means that in the evolution phase of a cloud-based ERP system,
the dimensions’ cycles happen several times and for an extended time than traditional ERP systems.
This means that the evolution phase lasts a longer period of time than in on-premise ERP systems.
This also calls for research efforts on how and when cloud ERP will retire.

Figure 2. The evolution phase in a cloud-based ERP system.

6. Study Limitations

6.1. Method
One of the disadvantages of qualitative multiple case studies is that they can become increasingly
complex [33]. The amount of data is large, which can make it challenging to identify themes and
patterns, and the data interpretation process is more tied to the researcher(s) than in a quantitative
data analysis [34]. Hence, there is a risk that the researches view of reality would color the experiences
and points made, and that some assumptions may be of a subjective manner, especially when the
researchers have a certain relationship to the study [33] (e.g., Xledger). Thus, the authors of this study
attempted to minimize bias by analyzing the data and identifying the themes independently and
discussing the results together until they reached consensus.

6.2. Validity and Transferability

Validity in this context is about the relevance and validness of the study [33]. Internal validity is
often seen as the strength of qualitative studies [26]. It refers to the match between what is observed,
Systems 2018, 6, 22 23 of 26

and the theoretical ideas developed. In this context, it is important to note that what users answer
might not always be what they think. In an interview setting, it can be difficult for users to talk about
negative sides of the topic, as it is easier and more socially accepted to talk about positive aspects.
Furthermore, the quotes from the users are translated from Norwegian to English which creates
a problem with the accuracy of what the users actually said. While the transcriptions’ translation was
conducted by only one of the authors, which may weaken the validity of the study, as there exists
a possibility that what have been said during the interviews have been influenced by the researcher’s
subjective perception. However, respondent validation was employed in this research in order to
maintain ‘good practice’. Respondent validation is a process whereby the researcher shares an account
of the findings with the participants (e.g., organizations and interviewees) upon which the research is
based [26]. After the data analysis, an overview of the findings was sent to the relevant participants to
make sure that our understanding matches their statements and opinions.
External validity, which is most relevant for this qualitative study, refers to whether a study can
be generalized beyond the specific research context [26]. In this research, many of the informants were
recommended by Xledger, and therefore might have a closer relationship to Xledger than the average
user. Also, the roles of several of the users were consultants or managers within accounting firms,
which means that they might have different attitudes towards the process of updates than the average
user or users in other contexts. The study can therefore not claim the generalizability of the findings
to other ERP vendors or contexts. Another clear limitation of this study is that it is focused on one
ERP system. A replication of this study with another ERP vendor or the data collection from other
organizations using a different type of cloud-based ERP systems would be preferred in order to test
the generalizability of the findings of this study. External validity is known as being a problem for
qualitative research due to small samples [26], which is why this study attempted to study several
cases and collect data from both users and the vendor.
In general, generalizability and transferability from qualitative research and the case studies
may pose something of a challenge. Especially when there is a small number of informants in some
of the case studies. While there is no recommended number for interviewees in case studies [35]
however, it is usually recommended that the researchers can stop the interviews when they reach
data saturation [28]. In our case, one of the challenges was that we were not able to interview more
than one informant in some organizations. However, the authors can see that the main themes are
reoccurring in the data among the different target cases. The relatively small number of informants
provided in this research may pose a challenge to replicate findings in other contexts [26]. Nonetheless,
some academics have argued that it is feasible to generalize and develop theories from such case
studies [30,36]. Guba and Lincoln [37] argue that ‘thick descriptions’ of case studies could help other
researchers in judging the transferability of their descriptions to their own contexts and lexicons. In this
study, the authors sought to document and describe the context in detail, which may enable researchers
to relate the findings to their domains. As our cases are limited to Norway, however, any general
conclusions must be made with caution. The experience of Xledger ERP updates at our cases could be
then different from that of other contexts.

6.3. The Topic

The study focuses on an area of literature that has not been previously researched to the best of
the researchers’ knowledge. This can make both the literature review and the discussions weaker as
they are not totally based other researchers’ points of view or existing body of knowledge. On the
other hand, this study is a call for research and an attempt to pave the way for more research on cloud
ERP usage and implementations in organizations.

7. Conclusions and Future Research Avenues

The existing ERP literature mainly focuses on on-premise ERP context. On the other hand,
cloud-based ERP systems are becoming more frequently adopted and are experiencing increasing
Systems 2018, 6, 22 24 of 26

success. However, research has not been able to keep up with the diffusion, and the lack of relevant
empirical studies are clear. In specific, there has been little research on ERP maintenance, evolution,
and retirement phases in both on-premise and in-cloud ERP contexts. In an in-cloud context,
systems’ clients and users are not in control anymore of the systems updates and evolution processes.
Thus, this study focused on the evolution phase of the ERP lifecycle framework, which for cloud-based
ERP systems involve continuous forced updates of the system. The evolution phase has been analyzed
using the related dimensions: change management and process, people, and product. There is limited
literature about this phase. The continuous and forced updates in the cloud-based ERP context show
that the evolution process is probably longer and more comprehensive for users than in an on-premise
ERP system. Hence, this study explored how the users and vendors feel about and react to the forced
cloud ERP updates at several organizations across various industries.
Cloud-based ERP systems have already been present for almost two decades but are under-researched.
Future research has to grasp and analyze many aspects of these systems. Cloud-based ERP systems are
considered one of the most important trends in the recent years, and updates are an important aspect of the
cloud-based ERP system. Thus, this study was an attempt to contribute to a topic that has several gaps in
literature. Now, as more information about the process of updates in a cloud-based ERP system is presented
in our findings, this could pave the way for more research in the field. For example, our findings suggest
that some users have issues with the frequency of updates. Future studies should do further research on
this topic, to be able to pinpoint what the ideal frequency of cloud-based ERP updates are for different
industries? This research also proposes a new and deeper perspective on the evolution phase in the lifecycle
of cloud-based ERP systems. The aim is to provide practitioners with general guidelines and insights on
how to implement cloud ERP system updates. However, other users’ and vendors’ thoughts and challenges
about the process with cloud-based ERP system need to be further investigated in order to check conformity
or contradictions with our results. In addition, this study did not have space for another aspect of this
topic, which is ‘workarounds’. If the users do not engage in the new changes done in the system, this could
lead to workarounds. It is potentially interesting to investigate if users develop workarounds after changes
in processes after the frequent updates. Finally, in this study, the evolution phase of the ERP lifecycle
framework was the main target of focus. It is equally important to explore cloud-based ERP systems in the
context of the other lifecycle phases.

Author Contributions: M.H. identified the initial research problem, and both authors developed the research
scope, method, and the interview guide. E.B. handled the data collection process and wrote the initial draft of
the paper. Both authors analysed and categorized the collected data. M.H. edited and enhanced the initial draft.
M.H. added the figures and study limitations section, responded to and addressed the anonymous reviewers’
constructive comments and recommendations, and wrote the final version of this study.
Conflicts of Interest: The authors declare no conflicts of interest.

References
1. Esteves, J.; Pastor, J. An ERP lifecycle-based research agenda. In Proceedings of the First International
workshop in Enterprise Management and Resource Planning: Methods, Tools and Architectures‚ EMRPS,
Venice, Italy, 25-27 November 1999; pp. 359–371.
2. Peng, G.C.A.; Gala, C. Cloud ERP: A new dilemma to modern organizations? J. Comput. Inf. Syst. 2014, 54,
22–30. [CrossRef]
3. Kotb, M.T.; Haddara, M.; Kotb, Y.T. Back-propagation artificial neural network for ERP adoption cost
estimation. Commun. Comput. Inf. Sci. 2011, 220, 180–187.
4. Elragal, A.; Haddara, M. The Future of ERP Systems: Look backward before moving forward. Procedia Technol.
2012, 5, 21–30. [CrossRef]
5. Ng, C.S.P.; Gable, G.; Chan, T. An ERP maintenance model. In Proceedings of the 36th Annual Hawaii
International Conference on System Sciences, Big Island, HI, USA, 6–9 January 2003.
6. Ha, Y.M.; Ahn, H.J. Factors affecting the performance of Enterprise Resource Planning (ERP) systems in the
post-implementation stage. Behav. Inf. Technol. 2014, 33, 1065–1081. [CrossRef]
Systems 2018, 6, 22 25 of 26

7. Calvert, C.; Seddon, P.B. The importance of ongoing ERP training and support. AISeL 2006, 91.
8. Haddara, M.; Fagerstrøm, A.; Mæland, B. Cloud ERP Systems: Anatomy of Adoption Factors and Attitudes.
J. Enterp. Resour. Plan. Stud. 2015, 2015, 22. [CrossRef]
9. Ng, C.S.P.; Chan, T.; Gable, G.G. A client-benefits oriented taxonomy of ERP maintenance. In Proceedings
of the IEEE International Conference on Software Maintenance (ICSM’01), Washington, DC, USA,
7–9 November 2001; p. 528.
10. Arnesen, S. Is cloud ERP solution right for you. Strat. Finance 2013, 95, 45–50.
11. Duan, J.; Faker, P.; Fesak, A.; Stuart, T. Benefits and Drawbacks of Cloud-Based versus Traditional ERP
systems. In Proceedings of the 2012–13 Course on Advanced Resource Planning; 2013; Available online:
https://s3.amazonaws.com/academia.edu.documents/31000164/A.Fesak_et_al._Comparing_Cloud-based_
_Hosted__and_On-premises_ERP._SME_aspect._2012.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&
Expires=1527846824&Signature=%2BCsxtGXXh4wrS5IXWBonq7N7yIM%3D&response-content-disposition=
inline%3B%20filename%3DBenefits_and_Drawbacks_of_Cloud-Based_ve.pdf (accessed on 28 May 2018).
12. Juell-Skielse, G.; Enquist, H. Implications of ERP as Service. In Re-Conceptualizing Enterprise Information
Systems; Springer: Berlin, Germany, 2012; pp. 129–151.
13. Raihana, G.F.H. Cloud ERP—A solution model. Int. J. Comput. Sci. Inf. Technol. Secur. 2012, 2, 76–79.
14. Armbrust, M.; Fox, A.; Griffith, R.; Joseph, A.; Katz, R.; Konwinski, A.; Lee, G.; Patterson, D.; Rabkin, A.
A view of cloud computing. Commun. ACM 2010, 53, 50–58. [CrossRef]
15. Albadri, F.A.; Abdallah, S. ERP training and evaluation: ERP life-cycle approach to end-users’ characterization
and competency building in the context of an oil and gas company. Ibima Bus. Rev. 2009, 3, 19–26. [CrossRef]
16. Co, H. ERP Systems in Norway 2013; HerbertNathan and Co.: Oslo, Norway, 2013.
17. Castellina, N. SaaS and Cloud ERP Observations: Is Cloud ERP Right For You?;; Aberdeen Group:
Boston, MA, USA, 2012.
18. Claybaugh, C.C.; Ramamurthy, K.; Haseman, W.D. Assimilation of enterprise technology upgrades:
A factor-based study. Enterp. Inf. Syst. 2017, 11, 250–283. [CrossRef]
19. Seethamraju, R. Adoption of software as a service (SaaS) enterprise resource planning (ERP) systems in
small and medium sized enterprises (SMEs). Inf. Syst. Front. 2015, 17, 475–492. [CrossRef]
20. Haddara, M.; Hetlevik, T. Investigating the Effectiveness of Traditional Support Structures and
Self-organizing Entities within the ERP Shakedown Phase. Procedia Comput. Sci. 2016, 100, 507–516.
[CrossRef]
21. Haddara, M.; Moen, H. User Resistance in ERP Implementation: A Literature Review. Procedia Comput. Sci.
2017, 121, 859–865. [CrossRef]
22. Wheatley, M. ERP training stinks. CIO 2000, 13, 86.
23. Scott, J.E. Post-implementation usability of ERP training manuals: The user’s perspective. Inf. Syst. Manag.
2005, 22, 67–77. [CrossRef]
24. Goel, M.S.; Kiran, R.; Garg, D. Impact of cloud computing on ERP implementations in higher education.
Institutions 2011, 5, 8. [CrossRef]
25. Foster, S.; Hawking, P.; Zhu, C. The human side of ERP implementations: Can change management
really make a difference? In Research and Practical Issues of Enterprise Information Systems II; Springer:
Berlin, Germany, 2007; pp. 239–249.
26. Bryman, A. Social Research Methods; OUP Oxford: Oxford, UK, 2012.
27. Myers, M.D. Qualitative research in information systems. Manag. Inf. Syst. Q. 1997, 21, 241–242. [CrossRef]
28. Yin, R.K. Case Study Research: Design and Methods; SAGE Publications: Thousand Oaks, CA, USA, 2009.
29. Fusch, P.I.; Ness, L.R. Are we there yet? Data saturation in qualitative research. Qual. Rep. 2015, 20, 1408.
30. Eisenhardt, K.M. Building theories from case study research. Acad. Manag. Rev. 1989, 14, 532–550. [CrossRef]
31. Knafl, K.A.; Webster, D.C.; Benoliel, J.Q.; Morse, J.M. Managing and analyzing qualitative data a description
of tasks, techniques, and materials. West. J. Nurs. Res. 1988, 10, 195–218. [CrossRef] [PubMed]
32. Glaser, B.G. Conceptualization: On theory and theorizing using grounded theory. Int. J. Qual. Methods 2008,
1, 23–38. [CrossRef]
33. Andersen, I. Den Skinbarlige Virkelighed; Samfundslitteratur: Copenhagen, Denmark, 2013.
34. Oates, B.J. Researching Information Systems and Computing; Sage: Newcastle upon Tyne, UK, 2005.
 Systems 2018, 6, 22 26 of 26

35. Baker, S.E.; Edwards, R.; Doidge, M. How Many Qualitative Interviews Is Enough?: Expert Voices and Early
Career Reflections on Sampling and Cases in Qualitative Research; National Centre for Research Methods (NCRM)
Review Paper; NCRM: Southampton, UK, 2012.
36. Flyvbjerg, B. Five misunderstandings about case-study research. Qual. Inq. 2006, 12, 219–245. [CrossRef]
37. Guba, E.; Lincoln, Y. Competing paradigms in qualitative research. Handb. Qual. Res. 1994, 2, 163–194.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).

View publication stats