Soft Computing

https://doi.org/10.1007/s00500-020-05150-w (0123456789().,-volV)(0123456789().,-volV)

FOCUS

Identification and prioritization of DevOps success factors using fuzzy-

AHP approach
Muhammad Azeem Akbar1 • Sajjad Mahmood2 • Muhammad Shafiq3 • Ahmed Alsanad4 •

Abeer Abdul-Aziz Alsanad5 • Abdu Gumaei4

 Springer-Verlag GmbH Germany, part of Springer Nature 2020

Abstract
DevOps (development and operations) is a collaborative and multidisciplinary organizational effort to automate continuous
delivery of a software project with an aim to improve software quality. The implementation of DevOps practices is not
straightforward as there are several complexities associated with it. The aim of this study is to identify and prioritize the
factors that positively influence the DevOps practices in software organizations. Using a systematic literature review, 19
factors were identified. The identified factors were further validated with experts via a questionnaire survey study. Finally,
Fuzzy Analytical Hierarchy Process (FAHP) was used to prioritize the identified success factors. The results indicate that
‘‘DevOps security pipeline,’’ ‘‘use system orchestration’’ and ‘‘attempt matrix organization and transparency’’ factors are
the highest ranked success factors for the successful implementation of DevOps practices. The FAHP analysis is novel in
this research area as it provides the prioritization based taxonomy of the identified factors which will assist the researchers
and practitioners to focus on the critical areas that are significant for the successful adoption of DevOps practices.

Keywords DevOps  Fuzzy AHP  Success factors  Systematic literature review  Empirical investigation

1 Introduction

Lately, DevOps is one of the widely used development

Communicated by Hemen Dutta. methodology with an aim to minimize cost of software
development by adopting continues development and
& Muhammad Azeem Akbar
continues delivery (Céspedes et al. 2019; Sacks 2012). In
azeem.akbar@ymail.com
DevOps, the development and operation teams work
& Ahmed Alsanad
together to improve their delivery process. The DevOps
aasanad@ksu.edu.sa
focuses on sharing tasks and responsibilities within a team
1
College of Computer Science and Technology, Nanjing from development to deployment and support. Hence, in a
University of Aeronautics and Astronautics, Nanjing, DevOps paradigm, cross-functionality, shared responsibil-
People’s Republic of China
ities and trust between the development teams and clients
2
Information and Computer Science Department, King Fahd are encouraged (Farid et al. 2017). DevOps essentially
University of Petroleum and Minerals, Dhahran, Saudi extends the continuous development goals of the agile
Arabia
3
movement to continuous integration and release. To
School of Computer Science and Technology, Chongqing accommodate continuous releases, DevOps encourages
University of Posts and Telecommunications,
Chongqing 400065, China automation of the change, configuration and release pro-
4 cesses (Jabbari et al. 2016; Lwakatare et al. 2016). In this
Research Chair of Artificial Intelligence (RCAI), Department
of Information Systems, College of Computer and study, we adopted the definition of Leite et al. (2019):
Information Sciences, King Saud University, Riyadh, Saudi ‘‘DevOps is a collaborative and multidisciplinary effort
Arabia within an organization to automate continuous delivery of
5
Information Systems Department, College of Computer and new software versions while guaranteeing their correctness
Information Sciences, Imam Muhammad Ibn-Saud Islamic and reliability.’’
University, Riyadh, Saudi Arabia

123

The software organizations perceived several benefits by
adopting DevOps such as more concentration on imple-
mentation and frequent release. These perceived benefits
are powered by the automated build system, testing and
deployment process, which assists in channeling more
features into the production and delivery pipelines (Fors-
gren 2018). For example, Forsgren (2018) underlined that
the automated process also helpful to reduce the required
effort and capable of organizations to deploy the devel-
opment iteration with the frequency they scheduled.
Moreover, it is indicated that an effective automation
assists to develop good quality software (Virmani 2015).
The DevOps practices enable software organization to have
frequent and small releases as this helps improve visibility
of the implemented module to the end-users (Senapathi
et al. 2018). The frequent and small releases enable the
development teams to receive timely feedback from the
customers, which are useful for improving the overall
product quality (Olszewska and Waldén 2015). Besides the
advantages of DevOps, there are number of challenges
such as fear of change, risky deployment, blame game and
conceptual deficit (Gruver and Mouser 2015; Jabbari et al.
2018).
Despite the widespread adoption of DevOps, there is a
lack of understanding of the factors associated with suc-
cessful implementation of DevOps practices. In this study,
we have twofold objectives: (1) to identify the success
factors related to the DevOps via a systematic literature
review and feedback on the identified success factors via a
questionnaire survey study and (2) to develop a prioriti-
zation-based taxonomy for the identified success factors
using FAHP. This study provides a body of knowledge to
practitioners by presenting the success factors associated
with DevOps projects and a ranking of success factors from
people, change and business perspectives. In this study, we
address the following research questions:
[RQ1] What success factors of DevOps practices are
reported in the literature?
[RQ2] What do practitioners do think about the success
factors of DevOps practices?
[RQ3] How the identified success factors were prioritize
using fuzzy AHP?
[RQ4] What would be the prioritization-based taxonomy
of success factors?
2 Background
DevOps is a complementary set of agile practices that
combines development and operations with an aim to
reduce development life cycle and support continuous
delivery of high-quality software. DevOps promotes
123
M. A. Akbar et al.
collaboration and coordination between developers and
operators. DevOps focuses on improving efficiency
through automation with DevOps tools. DevOps supports
cross-functional mode of software development where
practitioners use different sets of tools for coding, building,
testing, packaging, releasing, configuring, and monitoring
aspects of development and delivery processes.
Initially, the term DevOps has ambiguous managing as
the literature interpret it in two different ways (Roche 2013;
Smeds et al. 2015). The term DevOps interpret as a job
position which requires both types of skill, i.e., develop-
ment and operations. Besides the others claims, DevOps is
an evolving need for software industry which is using
legacy software development paradigms (Senapathi et al.
2018; Callanan and Spillane 2016). The mainstream
researches have addressed this ambiguity by defining the
characteristics of DevOps methodology (Jabbari et al. 2016;
Senapathi et al. 2018; Roche 2013) and by highlighting the
pros and cons of implementing the DevOps strate-
gies(Shahin et al. 2017; Riungu-Kalliosaari et al. 2016).
Some of the existing studies emphasized that the col-
laboration, automation and services are the most significant
characteristics of DevOps (Senapathi et al. 2018; Erich
et al. 2014). Besides, we found that the DevOps is also
treated as the cultural movement that assists the develop-
ment environment concerning with effective communica-
tion, control and responsibilities(Senapathi et al. 2018).
Many researchers have carried out studies to better
understand the DevOps characteristics and practices used
in the industry. For example, Dyck et al. (2015) argued that
the cultural change contributed to enhancing the trust
between the DevOps teams that assist the transformation
and change in an organizational development environment.
Moreover, Smeds et al. (2015) stated that cultural change is
not a central feature of DevOps, but it provides assistance
to improve the development process. Several existing
studies highlighted the limitations and significance of
DevOps (Chen 2015; Chen et al. 2015; Leppänen et al.
2015). Banica et al. (2017) highlighted the key benefits of
DevOps, including quality products through quality ser-
vices and continues bonding.
Gupta et al. (2017) indicated that the DevOps assists in
developing the trust among teams, supporting their ideas
and innovations, and encourages the collaboration between
stakeholders. They further identify and prioritize the key
attributes of DevOps paradigm that are important to assess
the maturity level of an organization concerning to the
DevOps adoption. Similarly, Gill et al. (2018) highlighted
the DevOps works as a bridge to overcome the commu-
nication gap between the development and operation
teams. They also express that the DevOps contributes to
reducing human errors by automating the development and
operation activities.
Identification and prioritization of DevOps success factors using fuzzy-AHP approach
Wiedemann et al. (2019) stated that the DevOps pro-
vides the platform for project management with better
understandability, performance, integration and relation-
ships among teams. Roche (2013) argued that implicitly the
DevOps is not a stable development environment. How-
ever, it needs strong collaboration, skilled human resour-
ces, and effective automation to get the significant fruit of
DevOps practices.
However, no systematic study has been carried out to
identify the success factors of DevOps. Moreover, there is
no model that prioritizes the success factors which can help
practitioner’s better plan and carryout DevOps practices. In
this study, we identify the success factors associated with
DevOps and present a prioritization-based taxonomy for
the identified success factors using FAHP.
3 Research methodology
Systematic literature review (SLR) approach has been
adopted to collect the potential literature related to research
questions of this study. SLR is a protocol-based research
approach which assists in obtaining, verifying and ana-
lyzing the available literature systematically, using inclu-
sion and exclusion criteria(Kitchenham et al. 2009). The
extracted results using the SLR approach are less preju-
diced than informal literature review (Niazi et al. 2016a, b;
Akbar et al. 2019a, b). Various researchers of software
engineering domain have adopted the SLR approach. To
conduct this study, we have followed the step-by-step
instructions of Kitchenham and Charters (2007). The core
phases of SLR, i.e., planning the review, conducting the
review, and reporting the review, are carefully performed
in this paper. All the phases are diagrammatically pre-
sented in Fig. 1, and briefly discussed in subsequent
sections.
3.1 Planning the review
3.1.1 Research questions
The principal aim of this study is to explore the factors that
could positively impact the DevOps practices. However, to
address the study objective, we have proposed the fol-
lowing research questions (RQs):
[RQ1] What success factors of DevOps practices are
reported in the literature?
3.1.2 Data collection source
The selection of appropriate digital libraries is an essential
aspect of SLR study to collect the potential literature
related to research questions. However, by following the
suggestion of Chen et al.(2010) and Zheng et al.(2011), we
have selected the following digital repositories:
I. IEEE Xplore (http://ieeexplore.ieee.org)
II. ACM Digital Library (http://dl.acm.org)
III. ‘‘Springer Link (http://link.springer.com)
IV. Wiley Inter-Science (www.wiley.com)
V. Science Direct (http://www.sciencedirect.com)
VI. Google Scholar (http://scholar.google.com)
VII. IET Software (https://digital-library.theiet.org)
3.1.3 Search string
To collect the data from the selected digital libraries using
an automated search process, the search string plays a vital
role Zhang et al. (2011). To develop an appropriated search
string, we have collected the keywords, and their alterna-
tives form the published primary studies selected using
Quasi-Gold Standard (QGS) guidelines (White et al. 2001),
i.e., [PS1-PS5]. The collected keywords and their alterna-
tive were concatenated using the Boolean operator ‘‘OR’’
and ‘‘AND.’’ An example of the search string is given
below:
(‘‘success factors’’ OR ‘‘factors’’ OR ‘‘aspects’’ OR
‘‘items’’ OR ‘‘elements’’ OR ‘‘drivers’’ OR ‘‘motivators’’
OR ‘‘variables’’) AND (‘‘DevOps’’ OR ‘‘Development and
Operation’’, OR ‘‘Continues development and operation.’’
3.1.4 Initial inclusion criteria
We have specified some initial inclusion criteria, to decide
which part of the collected literature, in response to the
execution of search string on selected digital libraries was
used for further SLR process. The inclusion criteria were
specified based on the existing SLR studies Niazi et al.
(2016) and Inayat et al. (2015) are as follows: (1) the
articles published as a journal paper conferences paper or
book chapter; (2) the study that describes the success factor
of DevOps; (3) the findings of the paper based on empirical
evaluation; (4) the paper that provides sufficient detail
about the highlighted success factors of DevOps.
3.1.5 Initial exclusion criteria
For excluding the collected data, we have specified the
following protocols considering the suggestions of Niazi
et al. (2016) and Inayat et al. (2015) and Akbar et al. (2019)
as follows: (1) the studies that do not highlight the success
factors of DevOps; (2) the studies that are not in English;
(3) the redundant studies were not considered; (4) if two
studies are from a similar research project, only the most
completed one was considered; (5) the studies that do not
provide detail information about the study subject.
123

Fig. 1 The Adopted set of research methodologies
3.1.6 Study assessment
As part of the SLR process, quality assessment was carried
out to validate the relevance of primary studies for the
research questions. Table 1 shows the list of quality
assessment questions that were developed based on the
guidelines of Kitchenham and Charters (2007) and other
similar studies published in the software engineering
domain (Niazi et al. 2016; Khan et al. 2017; Khan et al.
2011). The quality assessment criteria consist of five
questions. Each question of the checklist was assessed
using the Likert scale, which is presented in the right-most
column of Table 1.
3.2 Conducting the review
3.2.1 Final study selection
The final papers were selected via three methods. Firstly,
the five studies were selected by considering the guidelines
of QGS guidelines (White et al. 2001). By executing the
search string (Sect. 3.3) on the selected data sources
(Sect. 3.2) and by applying the initial inclusion and
exclusion criteria (Sects. 3.4 and 3.5), we have collected a
Table 1 quality assessment criteria formal primary studies for
QA questions Checklist questions
QA1 ‘‘Does the used research approach address the research questions?’’
QA2 ‘‘Does the study discuss any success factors of DevOps?’’
QA3 ‘‘Does the study, discuss software development outsourcing by using cloud computing?’’
QA4 ‘‘Is the collected data related to DevOps practices?’’
QA5 ‘‘Are the identified results related to the justification of the research questions?’’
123
M. A. Akbar et al.
total of 713 pre-reviewed primary studies. To finally select
the primary studies, we have applied the tollgate approach
developed by Afzal et al. (2009). The tollgate approach
consists of five core phases, as presented in Fig. 2. All the
authors of this study carefully perform the phases of toll-
gate approach, and 56 studies were selected for data
extraction process (Fig. 2). We further performed snow-
balling (backwards and forward), and 20 studies were
selected for final data extraction process (Fig. 2). There-
fore, a total of 81 studies were considered for final data
extraction process. The quality assessment process was
also executed during the studies selection process. The
selected primary studies are given in ‘‘Appendix A’’ sec-
tion. The study ID ‘‘PS’’ is used to indicate the use of SLR
studies.
3.2.2 Data extraction and synthesis
The final selected 108 primary studies are used for data
extraction. All the selected studies were carefully reviewed
by the first two authors of this study and the data in the
form of success factors, main themes, statements, etc., were
collected. To validate the data extraction process, the third
author of this study arbitrary involve in the data extraction
Likert scale
Yes = 1, Partial = 0.5, NO = 0
Yes = 1, Partial = 0.5, NO = 0
Yes = 1, Partial = 0.5, NO = 0
Yes = 1, Partial = 0.5, NO = 0
Yes = 1, Partial = 0.5, NO = 0
Identification and prioritization of DevOps success factors using fuzzy-AHP approach
Fig. 2 Final refinement of
formal studies
process. During the data extraction process, we have col-
lected a total of 37 ideas and statements that reflect the
positive impact of DevOps activities. We have merged the
statements and ideas into the compact statement of success
factors. All the success factors were rephrased to remove
the plagiarisms. The finalized list of success factors is
given in Table 4.
Once the data extraction process finalized, we have
conducted an inter-rater reliability test (Afzal et al. 2009)
to check the research’s prejudice. To do this, five inde-
pendent experts were invited. They selected ten studies,
performed all the steps of the data extraction process, and
recorded the extracted data in a separate Microsoft Excel
Sheet. Based on the results extracted by researcher and
independents experts, we have calculated the nonpara-
metric Kendall’s coefficient of concordance (W). The value
of W = 1 shows the perfect agreement between the data
extracted by the researcher and indented experts, where is
the W = 0 indicted no agreement between the data
extracted by researchers and independent experts. The
calculated results (W = 0.87, p = 0.003) show the signed
agreement between data which is extracted by both teams.
3.3 Reporting the review
3.3.1 Quality of selected studies
To check the significance of the selected primary studies,
we have performed the quality assessment. The quality of
the selected studies was assessed based on the quality
assessment criteria discussed in Sect. 3.1.6. The detailed
quality assessment score is given in ‘‘Appendix B’’ sec-
tion. The cumulative results show that 60% of the selected
studies scored C 70%. This quality assessment result
shows the extent to with the selected studies stud eve is
significant to address the objective of the study. We have
used the 40% quality assessment score as a threshold value.
3.3.2 Temporal distribution of selected studies
and adopted research approaches
The publication years of the selected studies were also
extracted with the aim to check the frequency of publica-
tion in the current years. The results show that the selected
studies are published between 2013 and 2019, and it is
noted that there is a rapid increase of publication in recent
era (Fig. 3). This shows that the DevOps is the currently an
active research area in the domain of software engineering.
We further extracted the used research approaches of
used in the selected studies and their frequency analysis
shows that the questionnaire survey (QS) adopted by 18%,
case studies (CS) by 35%, grounded theory (GT) approach
by 17%, content analysis (CA) by 5%, Action research
(AR) 9% mixed method (MM) is adopted by 16% (Fig. 3).
It is summarized that CS is the most common adopted
research methodology by the selected SLR studies.
3.4 Empirical study
The empirical research approach is considered as the most
appropriate way to collect pure data from the targeted
population. Kitchenham and Pfleeger (2003) reported that
the selection of empirical data collection approach is based
on data type, ‘‘available data collection resources,’’ ‘‘con-
trolling mechanism of selected approach’’ and ‘‘skill to
operate the variable of interest.’’ The observational method
123

Fig. 3 Adopted research methodologies and temporal distribution
is considered very hard to collect the pure and represen-
tative data sets (Akbar et al. 2019a, b). However, to address
the research questions of this study, a questionnaire survey
approach is used to collect the data form practitioners.
3.4.1 Questionnaire development
The survey instrument was developed based on the findings
of SLR. To develop an online survey instrument, we use
the services of Google form (docs.google.com/forms). The
questionnaire survey approach is an effective way to reach
the target population at geographically distributed loca-
tions. The purpose of the developed survey questionnaire
was to validate the investigations of SLR and to identify
the additional success factors of DevOps. To do this, the
developed survey instrument was broadly categorized into
two core section: first part consist of biographic informa-
tion of the participant and the second part is further cate-
gorized into two sections (i.e., close-ended and open-
ended): the close-ended section consists of the questions
that are developed based on the findings of SLR (success
factors), and in second section (open-ended), we requested
from the respondents to provide the additional success
factor of DevOps which is not enlisted in questionnaire.
3.4.2 Pilot assessment of the questionnaire survey
Initially, the questionnaire was developed based on the
author’s experiences and by considering the suggestions of
the supervisor (Khan et al. 2017a, b; Noy 2008). A pilot
assessment was conducted with the three independent
experts: one from education institute (Nanjing University
of Aeronautics and Astronautics) and two from the soft-
ware industry (AMAZON India and QSoft-Vietnam). The
experts suggest some imported changing in the question-
naire related to improving the understanding of the ques-
tion. We addressed all the suggestions of experts to
123
M. A. Akbar et al.
increase the readability and understandability of the queries
which are requested in the questionnaire. The updated
questionnaire is used for the data collection process. A
sample of the survey questionnaire is presented in ‘‘Ap-
pendix B’’ section.
3.4.3 Ethics approval
To get the survey approval, we send the updated ques-
tionnaire to research supervisor and Research Ethics Board
of School of Big Data and Software Engineering-
Chongqing–China. After receiving the permission
response, we initiate the survey, and the survey instrument
makes available to the participants through this web-link.
The responses of the survey are hosted at Google Drive
(drive.google.com). The survey participants were requested
to mark the survey questions bestowing to their knowledge.
All the respondents contributed to the data collection pro-
cess voluntarily and anonymous. The respondents can exist
from the survey at any stage.
3.4.4 Data sources
The purpose of this survey was to validate the findings of
SLR (i.e., DevOps success factor). However, to validate the
findings of SLR, the opinions of experts are important. To
target the most potential population of survey study at the
geographically distributed development environment, the
snowball sampling strategy (Khan et al. 2017) was applied.
The snowball sampling is an efficient and cost-effective
way to collect the data from a physically distributed pop-
ulation. In snowball sampling, the participants are reques-
ted to share the survey questionnaire to their contact
researchers or practitioners. The snowball sampling is an
effective way to collect the data from a large and dispersed
targeted population (Easterbrook et al. 2008; Finstad
2010). Various methods were used to target the population,
including personal Email, organizational Email, LinkedIn
and Research-Gate. The data were collected during
September 2019 to December 2019. A total of 147
responses were collected in the form of an Excel sheet.
First two authors of this study manually reviewed all the
responses. During the manual review, we found 16
incomplete responses. However, while discussing with the
research supervisor, we decided to not include the incom-
plete responses in the data analysis process. Finally, a total
of 131 complete responses were entertained for future data
analysis. The demographics of the respondents are given at
this link: https://tinyurl.com/wpw3eb9.
Identification and prioritization of DevOps success factors using fuzzy-AHP approach
3.4.5 Survey data analysis
The frequency analysis method is applied to analyze the
collected responses statistically; the frequency analysis
method is an effective way to analyze the descriptive data
(Bland 2015). The frequency of occurrence and the per-
centage of each success factor are reported in tables. The
frequency approach is useful to compare the views and
values within groups of variables and across the groups of
variables. To check the significance of each success factor,
according to the survey respondents, the views of all the
respondents are calculated and presented in the form of
tables. Moreover, to check the relative importance of each
success factor, the frequency of occurrence of one factor is
compared with other factors. The same method is used by
other researchers in several other research domains (Akbar
et al. 2018; Keshta et al. 2017; Mahmood et al. 2017).
3.5 Phase 3: Fuzzy set theory and AHP
This section discusses the basics of fuzzy set theory and the
procedure to incorporate it in the classical AHP. The tra-
ditional AHP is a multi-criteria decision-making method
that assumes the expert judgment as it is and uses crisp
number leading to inconsideration of the uncertainty that
came from linguistic variable (Tiwari 2006). When using
AHP method, it is important to make sure that the opinions
given by the experts are full of proficiency, harmony, and
carefulness. If these criteria for the options are fulfilled,
then AHP is considered to be an effective problem solving
methodology (Mulubrhan et al. 2014). However, AHP
method does not consider the uncertainty of one’s judg-
ment. Hence, fuzziness and vagueness existing in many
decision-making problems may contribute to the imprecise
judgments of decision makers in conventional AHP
approaches (Bouyssou et al. 2000). The fuzzy AHP tech-
nique uses a range of values to incorporate the uncertainty
of the decision maker. The fuzzy AHP uses the concept of
fuzzy logic that deals with uncertain data and provides a
systematic base for handling situations which are
ambiguous or not well defined (Kahraman et al. 2004). As
a result, the decision maker can specify preferences in the
form of natural language expression about the importance
of each criteria (Yaghoobi 2018).
In this study, we have used questionnaire survey to
collect data from experts working in different development
environments, facilities, project size, organization size and
roles across the globe. The data collected from the ques-
tionnaire survey contain the uncertainty of human judg-
ments as experts evaluate various success factors
associated with DevOps projects. Hence, in this paper, we
apply the fuzzy AHP approach because it is an appropriate
methodology to handles uncertain and imprecise data.
3.5.1 Fuzzy set theory
Fuzzy set theory is an extension of classical set theory that
was initially introduced by Zadeh et al. (1996). That was
oriented to deal with uncertainties and vagueness in the
real-world problems and manage these ambiguities in the
multi-criteria decision-making problems. The primary
contribution of Fuzzy set theory is to represent the vague
data. In the fuzzy set, a membership function lF(x) is
characterized which maps object between 0 and 1. The
FAHP has been adopted by a number of studies to address
the multi decision problems in range of industries, for
example, supplier selection in electronics market
(Chamodrakas et al. 2010), gearmotor company (Ayhan
2013), personnel selection (Güngör et al. 2009), supplier
selection for developing the low carbon supply chain (Önüt
et al. 2009), selection of software process improvement
success factors (Khan et al. 2019) and for ranking chal-
lenging factors of distributed agile software development
paradigm (Shameem et al. 2020). The definitions and
preliminary of the fuzzy set theory are discussed in the
following sections.
3.5.2 Fuzzy analytical hierarchy process
The analytic hierarchy process (FAHP) is one of the most
powerful methods used multi-criteria decision-making
problems. The main advantages of FAHP are the relative
ease with which it handles multiple criteria, easier to
understand, and it can effectively handle both qualitative
and quantitative data.
Definition A triangular fuzzy number (TFN) F is denoted
by a set (fl, fm, fu) as shown in Fig. 4. The given Eq. (1)
defines the membership function lF(x) of F.
8 9
>
> x  fl >
>
> l m >
< fm  fl ; f xf >
> =
lF ðxÞ ¼ u
f x ð1Þ
>
> ; fm xfu >>
>
> f u  fm >
>
: ;
0; Otherwise
where fl, fm and fu are the crisp numbers denoting the
lowest, most promising and highest possible values
respectively.
The algebraic operational laws using two TFNs, namely
(F1, F2) are given in Table 2.
There is the following main step of FAHP method are
adopted:
Step 1 Decompose the complex decision problem into
the hierarchical structure (Fig. 5).
Step 2 Calculate priority vector at each level of hierar-
chy with the help of pairwise comparison.
123
 M. A. Akbar et al.

1 2 m
Fgi ; Fgi ; . . .; Fgi ; ð2Þ
i ¼ 1; 2; . . .; n ð3Þ
where all Fgij ,
(j = 1, 2,…, m) are fuzzy triangular numbers
(TFNs).
The following are the key steps of Chang’s extent
analysis method (Chang 1996):
Step 1 The value of a fuzzy synthetic extent concerning
the ith object can be defined using Eq. (4):
" #1
Xm
j
Xn X m
j
Si ¼ Fgi  Fgi ð4Þ
Fig. 4 Triangular fuzzy number j¼1 i¼1 j¼1
P
Step 3 Compute the consistency ratio of the pairwise To achieve the expression m j
j¼1 Fgi , execute the fuzzy
comparison. addition operation of m extent analysis such as:
!
Step 4 Calculate the final priority weight for the factors Xm
j
Xm X
m X
m
l m u
and the sub-factors (Fig. 5). Fgi ¼ fgi ; fgi ; fgi ð5Þ
j¼1 j¼1 j¼1 j¼1
Though the classical AHP has several benefits, it has
some limitation due to usability of AHP in Crisp envi- " #1
P
n P
m
ronment, Judgmental scale is unbalanced, and absence of and to achieve the expression Fgij , the fuzzy
uncertainty, selection of judgment is subjective. Therefore, i¼1 j¼1

fuzzy AHP, a fuzzy extension of AHP, was introduced to addition operation is executed on Fgij ðj ¼ 1; 2; . . .mÞ value,
solve more accurately for the real-time and uncertain as follow:
problem (Saaty 2013). The FAHP can capture uncertain, !
imprecise judgment of different experts by handling the Xn X m
j
Xn X
n X
n
l m u
Fgi ¼ fi ; fi ; fi ð6Þ
linguistic variables. Various researchers have followed the i¼1 j¼1 i¼1 i¼1 i¼1
FAHP methods in a variety of domains (Yaghoobi 2018;
Sloane et al. 2002; Wen-ying 2009; Kabra et al. 2015). In and finally, calculate the inverse of the vector with the help
our study, we have utilized the FAHP suggested by Chang of Eq. (7):
(Chang 1996), which provides more accurate and consis- 0 1
" #1
tent results as compared to other FAHP techniques. Xn X m B 1 1 1 C
Fgij ¼B
@P ; n ; n C ð7Þ
In a prioritization problem, let X = {x1, x2,…, xn} Akbar n
l
P m P uA
et al. (2019) represent the elements of main categories as i¼1 j¼1 fi fi fi
i¼1 i¼1 i¼1
an object set and U = {u1, u2,…, un} represent the elements
of each category as a goal set. According to the Chang Step 2 As Fa and Fb are two triangular fuzzy number
 
(Chang 1996) methodology, each object is considered, and then the degree of possibility of Fa ¼ fal ; fam ; fau  Fb ¼
 l m u
extent analysis for each goal (gi) is executed, respectively. fb ; fb ; fb is defined as follows.
Thus, for each object, there are (m) extent analysis values    
V ðFa  Fb Þ ¼ sup min lFa ð xÞ; lFb ð xÞ ð8Þ
that can be obtained with the following Eqs. (2) and (3):
Equation 8 can also be similarly specified as below:

Table 2 Triangular fuzzy

Operation Law Expression
numbers
 l m u  l m u  l 
Addition ðF1  F2 Þ f1 ; f1 ; f1  f2 ; f2 ; f2 ¼ f1 þ f2l ; f1m þ f2m ; f1u þ f2u
 l m u  l m u  l 
Subtraction ðF1  F2 Þ f1 ; f1 ; f1  f2 ; f2 ; f2 ¼ f1  f2l ; f1m  f2m ; f1u  f2u
 l m u  l m u  l 
Multiplication ðF1  F2 Þ f1 ; f1 ; f1  f2 ; f2 ; f2 ¼ f1  f2l ; f1m  f2m ; f1u  f2u
 l m u  l m u  l l m m u u
Division ðF1  F2 Þ f 1 ; f1 ; f1  f2 ; f2 ; f2 ¼ f 1 = f2 ; f1 = f 2 ; f1 = f 2
 l m u 1  
Inverse ðF1  F2 Þ f1 ; f1 ; f1 ¼ 1=f1l ; 1=f1m ; 1=f1u
 l m u
For any real number k (kF1) k f1 ; f1 ; f1 ¼ k f1l ; k f1m ; k f1u

123
Identification and prioritization of DevOps success factors using fuzzy-AHP approach

Fig. 5 FAHP decision hierarchy

V ðFa  Fb Þ ¼ 8
hgtðFa \ Fb Þ ¼ lFa ðdÞ Step 4 Via normalization, the normalized weight vectors
9
> 1 if fam  fbm > are in Eq. 13, and the result will be a non-fuzzy number
>
< >
=
fau  fbl l u
which represents the priority weight of the criteria:
¼ f b  f a
> u m m l >
: ðfa  fa Þ þ ðfb  fb Þ
> >
; W ¼ ðdðF1 Þ; dðF2 Þ; dðF3 Þ; . . .dðFn ÞÞ ð13Þ
0 Otherwise
where W is a non-fuzzy number.
ð9Þ
Step 5 Checking consistency ratio: The pairwise
Here, d represents the ordinate of the highest intersec- matrices should always be consistent in fuzzy AHP(-
tion point between D, lFa and lFb (Fig. 6). The values of Shameem et al. 2018). Therefore, it is necessary to check
V1 (Fa C Fb) and V2 (Fa C Fb) are mandatory for calcu- the consistency ratio of each pairwise comparison matrices.
lating the value of P1 and P2. To do so, the graded mean integration approach is utilized
Step 3 Calculate the overall degree of possibility of a for defuzzifying the matrix. A triangular fuzzy number,
convex fuzzy number and the other convex fuzzy numbers denoted as P = (l, m, u), can be defuzzified to a crisp
Fi (i = 1, 2,…, k) can be defined as follow. number as follows:
VðF  F1 ; F2 ; F3 . . .Fk Þ ¼ minVðF  Fi Þ ð10Þ ð4m þ l þ uÞ
Pcrisp ¼ ð14Þ
6
Assuming that,
0 After the defuzzification of each value in the matrix,
d ðFi Þ ¼ minV ðFi  Fk Þ ð11Þ
consistency ratio (CR) of the matrix can easily be calcu-
for k = 1, 2,…,n; k = i. lated and checked whether CR is smaller than 0.10 or not.
With the help of Eq. 12, calculate the weight vector For this, two basic parameters, i.e., consistency index (CI)
using Eq. 11. and consistency ratio (CR), are used, which are defined
using Eqs. 14 and 15, respectively.
W 0 ¼ ðd 0 ðF1 Þ; d 0 ðF2 Þ; d0 ðF3 Þ; . . .d 0 ðFn ÞÞ ð12Þ
Imax  n
where Fi (i = 1,2,…,n) are n distinct elements. CI ¼ ð15Þ
n1

Fig. 6 Triangular fuzzy number

123
 M. A. Akbar et al.

Table 3 Random consistency

Size of the matrix 1 2 3 4 5 6 7 8 9 10
index (RI) with respect to
matrix size Random consistency index (RI) 0 0 0.58 0.9 1.12 1.24 1.32 1.41 1.45 1.49

CI time-intensive and complicated activities, but these activ-

CR ¼ ð16Þ
RI ities are critical to gain the full participation of the DevOps
(Nogueira et al. 2018). However, the sharing of informa-
where Imax: the largest eigenvalue of the comparison
tion and the assurance of team participation in change
matrix, n: the number of items being compared in the
activities are important for culture change in an organiza-
matrix and RI: the random index and its value can be opted
tion (Snyder and Curtis 2017). The frequent meetings of
from Table 3.
the development team’s effective communication assist in
To be a consistent matrix, the computed value of CR
sharing the information and issues that are critical for
should less than 0.10. If the value of CR is found to be
culture change in an organization (Samarawickrama and
greater than 0.10, the decision maker must make the pair-
Perera 2017). The adoption of advance tools and automa-
wise judgments again.
tion appliances can only be fruitful while the effective
cultural change has been taken (Samarawickrama and
Perera 2017).
4 Findings
Fundamentally, each development team has their
understanding of each process by considering their spe-
4.1 SLR findings
cialization, experiences and strength. Smeds et al. (2015)
mention that to the successful implementation of DevOps
The minimization of the development cycle is not sufficient
activities, all the teams must have a common overall
to rationalize the processes. On the contrary, multiple
vision, language and contextualization of each practitioner
deliveries are each a potential source of friction. To min-
working style in the entire project. They further mention
imize this risk, it is significant to explore the optimizations
that the common understanding leads to the creation of
areas as early as possible. This is done in particular not
collaborative tools in the field, to identify and address the
only by modeling all the delivery processes but also by
bugs, manage delivery time, project management activities,
giving precise, unambiguous definitions of responsibilities
commitments of services, etc. Therefore, to successfully
and the deliverables associated with each step in the
employ the DevOps activities, the common working
cycle(Kamuto and Langerman 2017; McCarthy et al.
environment teams and easy access of information is
2015). In other words, the automation and rapidity of
important that assist the awareness grown for each practi-
DevOps do not do away with the need for formal delivery
tioner about the overall working environment and the team
and documentation processes and for the introduction of
members (Lwakatare et al. 2016). The key aim is to
human checkpoints. This effort to be transparent also
develop overlapping zones (developers who have access to
represents an opportunity to shed light on certain tasks, so
logs and monitoring, operators who carry out tests them-
that they are no longer sacrificed to planning require-
selves, etc.) to encourage mutual understanding among
ments(John et al. 2017). Of course, the tasks concerned are
practitioners and ensuring the accountability of all the
operating tasks performed right at the end of the production
participants (Rahman and Williams 2016).
chain, but in the same way as other processes they deter-
The transformation of agile to DevOps is a complicated
mine, the quality of the overall service rendered(Cois et al.
phenomenon. DevOps adoption aims to achieve continuous
2014).
development and operations by implementing the essential
The change in development methods is not sufficient for
steps, i.e., continuous integration, continuous delivery, and
the acquisition of newly adopted tools and technologies.
continuous deployment where is, each step required a
Wahaballa et al. (2015) underlined that the change in
specific predecessor. Gupta et al. (2017) underlined that to
culture is significant to ensure the success of DevOps
achieve a high frequency of deployment, it is necessary to
practices, as it is prerequisites and the tools are just a mean
apply continuous integration and delivery. They further
of implementation. Taryana et al. (2017) mention that once
mention that it is important to automate the deployment
the development tools put in place, then it is a time to
and operational processes for the sake of monitoring, dis-
emphasize on human resources (existence, training,
aster recovery plan, restoration, and administration. Mohan
recruitment, etc.), organization, governance and more
and Othmane (2016) argued that this makes no sense to put
prominently the information pass to the development team
every step of the software development process at one
about a certain objective to achieve. The complications to
point at the same time; that even causes the counter
entertain the change and re-organization of teams as it is

123
Identification and prioritization of DevOps success factors using fuzzy-AHP approach
predictive. They further mention that it is better to manage
and stabilize each phase that will assist with the success of
the next one. However, there is no need to wait for a step to
reach complete maturity before starting the next one. Right
from the start, the chain should be envisaged in its entirety,
to anticipate dependencies between tools, and to make
development teams aware of how their code will be used
(Benguria et al. 2018).
It is significant to systematically review each DevOps
process (sprint review, bug review (quality), deployment
review, incident review, performance review, etc.), as each
individual must have a contribution for continuous process
improvements. The summarized results of the review process
must share with all the teams that are monitored and priori-
tized at the project level. Chen (2017) suggested that it is
important to set up key performance indicators to monitor and
evaluate the project activities or a specific time to assure the
continues improvement loops. Snyder and Curtis (2017)
suggested that to improve the DevOps activities; it makes sure
when a problem is identified, a comprehensive decision must
be taken and automate the procedure to its future identifica-
tion. They further mention that the early investment to fix a
problem is better than its harmful effect if it is found at the
end of the development process.
It is noted that in DevOps, the clear definition of a
practitioner’s responsibilities assists in identifying their
key role in DevOps processes. Matrix organization is an
extremely effective means of going further and developing
the concepts of sharing and communication (Jiménez et al.
2018). A DevOps profile and a tester are associated with
each group of developers responsible for functionality or
block of components in the project, and in parallel; they
both belong to the Quality and Operation groups. This
guarantees ‘‘embodiment’’ of the overlaps and overall
cohesion (Chen 2017). It also reinforces communication,
which is vital for DevOps to succeed, since withholding or
concealing information can have a particularly severe
impact on this type of organization. Managers must
simultaneously impose very frequent communication and
behave positively: singling out responsibility is pointless.
Finding solutions together is much more effective. When it
comes to DevOps, transparency is not a recommendation; it
is a necessity (Elberzhager et al. 2017).
Until recently, agile methods and DevOps focused
mainly on integration and continuous deployment, but they
are used increasingly now in production phases. Operators
are now active participants in the system (Marijan et al.
2018). They must clearly state their demands, expectations,
and use cases, and make the other teams aware of the issues
encountered in production. This must be an integral part of
the non-functional requirement collection phases, particu-
larly within the context of operation automation(El-
berzhager et al. 2017). Similarly, support must not limit
itself to watching incidents occur in succession. Support
teams must enrich the use cases tested by the quality team’s
scripts and suggest new monitoring approaches to devel-
opers and operators. In brief, by collecting demands or
through continuous improvement, the organization must
make it possible to take into account this aspect of the
application’s life cycle(John et al. 2017).
Many of the biggest enterprises that shared their stories
explained that a key component of the culture shift at their
organization had been some large-scale internal event—
essentially in-house DevOps conferences. These events
give DevOps champions the chance to bring in outside
thought leaders and practitioners at other companies to
demonstrate to people within the host enterprise that there
is something to DevOps, automation, continuous delivery
and the like John et al. (2017), Jiménez et al. (2018), and
Elberzhager et al. (2017). For example, Capital One
recently hosted more than 1200 software engineers for an
in-house event it calls SECon and brought in many of the
same speakers we see this week at DOES. Target has also
hosted numerous internal DevOps days, as well as an
executive summit that Ross Clanton, senior group manager
at Target, called a ‘‘tipping point’’ for gaining top-down
support from executives(Benguria et al. 2018).
Cois et al. (2014) indicated that to implement the
DevOps practices, the additional and technical skill is
highly required from software industry experts. They fur-
ther underlined that the development team acquire skills
from expert members of the operational team and vice
versa. Mohan and Othmane (2016) emphasized on the
training of DevOps concerning to different tools. They
further mention that the training sessions should include
the concepts of DevOps culture, which assist in keeping the
pace of DevOps practices implementation. Organizational
management should consider the training of DevOps teams
on a priority basis (John et al. 2017).
The frequency of numbers of continues delivery possess
the enquiries of security in the domain of DevOps (Riungu-
Kalliosaari et al. 2016; Rajkumar et al. 2016). However,
the closed working environment between the practitioners
of DevOps provides the benefits for security perspective.
The duel relationship between the developers and operators
generates a dichotomy that needs further investigation. All
the investigated success factors are enlisted in Table 4.
The investigated factors were also mapped into three
core categories of software process improvement (SPI)
manifesto ‘‘People’’, ‘‘Business’’ and ‘‘Change’’ (Fig. 7).
The concept of SPI manifesto was developed by the experts
of software engineering to execute and to improve the
software development activities formally. The DevOps is a
new software process improvement methodology, and we
are motivated to map the investigated success factors of
DevOps into core categories of SPI manifesto(Pries-Heje
123
 M. A. Akbar et al.

Table 4 List of identified success factors

S. No. Success factors IDs of selected SLR studies

F1 Emphasize culture rather than [PS3], [PS7], [PS8], [PS9], [PS11], [PS13], [PS15], [PS18], [PS20], [PS22], [PS23], [PS25],
tools [PS26], [PS27], [PS30], [PS32], [PS34], [PS36], [PS37], [PS39], [PS40], [PS41], [PS43], [PS45],
[PS48], [PS49], [PS50], [PS53], [PS57], [PS58], [PS59], [PS61], [PS62], [PS64], [PS67], [PS69],
[PS70], [PS71], [PS73], [PS74], [PS76], [PS78], [PS79], [PS80], [PS81]
F2 Attempt matrix organization and [PS11], [PS13], [PS19], [PS21], [PS23], [PS25], [PS29], [PS32], [PS34], [PS37], [PS39], [PS46],
transparency [PS49], [PS53], [PS56], [PS57], [PS65], [PS67], [PS69], [PS73], [PS76], [PS79]
F3 Empowerment [PS5], [PS11], [PS12], [PS14], [PS15], [PS18], [PS20], [PS21], [PS22], [PS25], [PS26], [PS28],
[PS31], [PS33], [PS34], [PS37], [PS46], [PS47], [PS48], [PS49], [PS51], [PS54], [PS55], [PS60],
[PS62], [PS63], [PS69], [PS70], [PS76], [PS78], [PS81]
F4 Cross-functional team [PS6], [PS7], [PS10], [PS16], [PS18], [PS21], [PS27], [PS31], [PS35], [PS38], [PS41], [PS44],
[PS45], [PS52], [PS53], [PS58], [PS62], [PS63], [PS65], [PS68], [PS70], [PS81]
F5 Established skilled DevOps teams [PS2], [PS5], [PS8], [PS10], [PS12], [PS13], [PS14], [PS16], [PS19], [PS20], [PS21], [PS23],
[PS25], [PS29], [PS30], [PS31], [PS33], [PS35], [PS37], [PS39], [PS40], [PS42], [PS44], [PS46],
[PS48], [PS52], [PS55], [PS57], [PS58], [PS60], [PS61], [PS62], [PS65], [PS66], [PS67], [PS68],
[PS72], [PS73], [PS74], [PS76], [PS79], [PS80]
F6 Design a common baseline [PS1], [PS4], [PS9], [PS12], [PS14], [PS15], [PS18], [PS19], [PS20], [PS22], [PS23], [PS29],
[PS30], [PS32], [PS34], [PS35], [PS37], [PS38], [PS39], [PS41], [PS43], [PS45], [PS46], [PS47],
[PS48], [PS49], [PS54], [PS56], [PS57], [PS58], [PS59], [PS63], [PS64], [PS68], [PS69], [PS72],
[PS74]
F7 Keep to the sequencing of [PS4], [PS7], [PS9], [PS17], [PS19], [PS20], [PS21], [PS22], [PS25], [PS26], [PS29], [PS31],
the DevOps approach [PS37], [PS38], [PS41], [PS42], [PS44], [PS47], [PS49], [PS55], [PS56], [PS58], [PS60], [PS62],
[PS64], [PS69], [PS70], [PS74], [PS76], [PS79], [PS80], [PS81]
F8 Internal DevOps Events [PS1], [PS7], [PS14], [PS15], [PS20], [PS26], [PS27], [PS30], [PS33], [PS35], [PS38], [PS41],
[PS42], [PS46], [PS47], [PS53], [PS54], [PS58], [PS59], [PS60], [PS61], [PS65], [PS66], [PS68],
[PS69], [PS73], [PS74], [PS75], [PS79]
F9 Demonstrating Lean Leadership [PS7], [PS12], [PS16], [PS17], [PS20], [PS22], [PS25], [PS26], [PS29], [PS30], [PS34], [PS38],
behavior [PS39], [PS40], [PS42], [PS46], [PS49], [PS50], [PS57], [PS61], [PS63], [PS66], [PS69], [PS78]
F10 Continues integration and [PS7], [PS10], [PS17], [PS19], [PS21], [PS22], [PS27], [PS30], [PS33], [PS34], [PS37], [PS38],
deployment [PS40], [PS42], [PS46], [PS48], [PS49], [PS50], [PS51], [PS56], [PS57], [PS58], [PS59], [PS61],
[PS63], [PS64], [PS66], [PS69], [PS70], [PS71], [PS73], [PS76], [PS76], [PS78], [PS81]
F11 Measure progress and plan next [PS9], [PS13], [PS14], [PS19], [PS22], [PS23], [PS26], [PS27], [PS32], [PS35], [PS39], [PS40],
improvement [PS41], [PS44], [PS46], [PS48], [PS49], [PS50], [PS52], [PS56], [PS57], [PS59], [PS60], [PS64],
[PS65], [PS70], [PS73]
F12 Use modeling [PS2], [PS8], [PS9], [PS16], [PS17], [PS20], [PS22], [PS29], [PS31], [PS36], [PS38], [PS41],
[PS44], [PS49], [PS50], [PS57], [PS59], [PS64], [PS67], [PS69], [PS71], [PS76], [PS77]
F13 Integrate the challenges of [PS5], [PS6], [PS8], [PS11], [PS13], [PS16], [PS19], [PS20], [PS21], [PS23], [PS27], [PS29],
operations and support [PS30], [PS31], [PS33], [PS35], [PS36], [PS37], [PS40], [PS44], [PS45], [PS46], [PS47], [PS48],
[PS49], [PS50], [PS52], [PS55], [PS57], [PS58], [PS60], [PS61], [PS62], [PS67], [PS68], [PS69],
[PS75], [PS78], [PS79]
F14 Automatic testing techniques [PS9], [PS11], [PS17], [PS19], [PS24], [PS25], [PS29], [PS32], [PS34], [PS39], [PS42], [PS46],
[PS47], [PS51], [PS57], [PS60], [PS69], [PS71], [PS81]
F15 Accommodate legacy systems [PS1], [PS3], [PS9], [PS13], [PS17], [PS20], [PS23], [PS28], [PS30], [PS33], [PS40], [PS42],
[PS45], [PS47], [PS49], [PS51], [PS57], [PS59], [PS60], [PS62], [PS65], [PS67], [PS78]
F16 Use system orchestration [PS8], [PS9], [PS14], [PS15], [PS16], [PS17], [PS19], [PS20], [PS23], [PS24], [PS27], [PS29],
[PS30], [PS32], [PS34], [PS39], [PS41], [PS42], [PS44], [PS48], [PS49], [PS50], [PS51], [PS52],
[PS55], [PS57], [PS58], [PS59], [PS61], [PS64], [PS65], [PS66], [PS68], [PS74]
F17 Assess your DevOps strategy [PS7], [PS9], [PS11], [PS12], [PS15], [PS17], [PS18], [PS20], [PS22], [PS24], [PS26], [PS28],
[PS31], [PS32], [PS33], [PS34], [PS36], [PS38], [PS39], [PS41], [PS42], [PS43], [PS47], [PS48],
[PS49], [PS50], [PS51], [PS52], [PS54], [PS56], [PS57], [PS59], [PS60], [PS62], [PS63], [PS65],
[PS66], [PS68], [PS71], [PS76], [PS78]
F18 Real-Time Feedback [PS2], [PS3], [PS10], [PS17], [PS19], [PS24], [PS29], [PS31], [PS32], [PS34], [PS36], [PS37],
[PS39], [PS41], [PS47], [PS49], [PS43], [PS54], [PS56], [PS57], [PS58], [PS61], [PS62], [PS64],
[PS68], [PS72], [PS75], [PS76], [PS77], [PS79], [PS80]
F19 DevOps Security pipeline [PS1], [PS2], [PS4], [PS6], [PS7], [PS8], [PS11], [PS12], [PS13], [PS14], [PS16], [PS19], [PS20],
[PS22], [PS23], [PS24], [PS31], [PS33], [PS34], [PS35], [PS36], [PS38], [PS40], [PS42], [PS44],
[PS46], [PS47], [PS48], [PS49], [PS50], [PS52], [PS53], [PS55], [PS56], [PS57], [PS58], [PS59],
[PS60], [PS64], [PS66], [PS68], [PS69], [PS70], [PS73], [PS74], [PS77], [PS79]

123
Identification and prioritization of DevOps success factors using fuzzy-AHP approach
Fig. 7 SPI manifesto core
categories (Pries-Heje 2010)
Fig. 8 SPI manifesto key
principle (Pries-Heje 2010)
Fig. 9 Conceptual map of DevOps dimensions
2010). The classification of success factors that serve as a
knowledge base for practitioners and researchers to con-
sider the most important factors category for revising and
developing the strategies to improve the DevOps practices.
The mapping team consists of three team members that
include the first two authors of this study and an external
industry expert invited from the ‘‘Virtual-Force’’ software
development organization. The expert is a DevOps team
manager working in Bahrein (Fig. 8). By considering the
concept derived from the literature survey (by the first two
study authors) and based on the practical experience of
industry experts, we have developed a conceptual model
(Fig. 9). According to the conceptual model Fig. 9, the
change and people categories are consisting of manage-
ment and engineering perspective of DevOps; and business
category has an association with the management per-
spective of DevOps activities.
All the identified success factors were mapped into the
core categories of SPI manifesto by considering the percep-
tions of the developed conceptual model (Fig. 9). Based on
the mapping process results, we have developed a framework
that consists of the core categories and their related success
factors (Leite et al. 2019; Akbar 2019) (Fig. 10).
To check potential bias in the development of the con-
ceptual map of DevOps dimensions, we have performed
the inter-rater reliability. To conduct the reliability test,
three experts were invited from different empirical soft-
ware engineering laboratories. The experts classified all the
investigated success factors into the core three categories
of SPI manifesto. Using the mapping results of both study
123

Fig. 10 mapping of success
factors in the categories of SPI
manifesto
authors and independent researchers, we have determined
the nonparametric Kendall’s coefficient of concordance
(W) (Afzal et al. 2009), whereas the value of W = 1 indi-
cates the perfect agreement, and W = 0 indicates the
complete disagreement. The calculated results (W = 0.94,
p = 0.005) indicated that there is a perfect agreement
between the mapping results of study authors and external
experts. Hence, based on the results it is concluded that the
categorization process is unbiased and consistent.
DevOps adoption aims to significantly increase the
return on investment and the vision and interest of the
organization as well. The success factors related to the
business dimension of DevOps are mapped in the business
category. The mapping results show that out of 19, the 5
success factors are related to the business category
(Fig. 10). Consequently, the primary structure of DevOps
practices is based on deploying a pool of practitioners from
Dev (development) and Ops (operations) teams via col-
laborative culture. However, the 6 identified related suc-
cess factors are mapped into people category (Fig. 10).
Similarly, change is one of the key aspects of DevOps that
is significant to improve the delivery and deployment
process effective and reliable. The DevOps practices
require the change in automation for delivery and deploy-
ment pipeline. The identified 8 success factors are mapped
in the change category (Fig. 10). The success factors of
change category are significant to address the success and
improvement of DevOps practices in software
organizations.
4.2 Empirical findings
We have developed an online survey questionnaire link to
validate the success factors of DevOps identified during the
123
M. A. Akbar et al.
literature review. The responses of the survey participants
were classified in positive, negative, and neutral categories
(Table 5). The positive (‘‘strongly agree and agree’’) cat-
egory consist of the percentage of survey respondents who
considered the identified factors have a positive impact on
DevOps practices. The negative (strongly disagree, dis-
agree) category presents the results of the survey partici-
pants who did not consider the reported success factors
have any influence on DevOps implementation. The neutral
category presents the results of those respondents who were
not sure about the significance and impact of a success
factor (Table 5).
4.3 Application of fuzzy analytic hierarchy
process (FAHP)
In this section, we have performed all the steps of FAHP to
calculate the weights of success factors within their cate-
gories and in between the categories. The analysis was
conducted using MATLAB R2016b programming envi-
ronment developed by math works is an American pri-
vately held corporation, which was running on a personal
computer with an Intel Corei3 3.5-GHz processor and
8-GB memory. The step-by-step phases of FAHP are per-
formed in the subsequent sections.
Step 1 (decompose a complicated decision problem into
a hierarchical structure)
At this level, a complicated decision-making problem is
divided into interconnected decision elements Shameem
et al. (2018) and Albayrak and Erensal (2004). The hier-
archical structure of the problem divided at a minimum of
three stages, as indicated in Fig. 5. In this hierarchical
structure, the objective of the problem is mentioned in
stage 1. However, the factors and sub-factors are positioned
Identification and prioritization of DevOps success factors using fuzzy-AHP approach

Table 5 Results of the

Sr. no Success factors Empirical observations (n = 117)
questionnaire survey study
Positive Negative Neutral
S.A A % S.D D % N %

C1 Category: People 40 62 87 2 2 3 11 9
F1 Emphasize culture rather than tools 29 58 74 4 10 12 16 14
F2 Attempt matrix organization and transparency 40 62 87 1 4 4 10 9
F3 Empowerment 30 53 71 6 8 12 20 17
F4 Cross-functional team 37 60 83 2 7 8 11 9
F5 Established skilled DevOps teams 35 48 71 0 10 9 24 21
C2 Category: Business 38 57 81 2 7 8 13 11
F6 Design a common baseline 37 48 73 2 8 9 22 19
F7 Keep to the sequencing of the DevOps approach 42 61 88 0 0 – 14 12
F8 Internal DevOps events 28 59 74 4 8 10 18 15
F9 Demonstrating Lean leadership behavior 37 56 79 5 7 10 12 10
F10 Continues integration and deployment 31 54 73 4 8 10 20 17
F11 Measure progress and plan next improvement 29 54 71 2 13 13 19 16
C3 Category: Change 42 64 91 0 1 1 10 9
F12 Use modeling 33 50 71 3 9 10 22 19
F13 Integrate the challenges of operations and support 46 51 83 2 8 9 10 9
F14 Automatic testing techniques 32 52 72 2 12 12 19 16
F15 Accommodate legacy systems 28 61 76 2 12 12 14 12
F16 Use system orchestration 33 54 74 4 10 12 16 14
F17 Assess your DevOps strategy 26 58 72 4 9 11 20 17
F18 Real-time feedback 37 59 82 2 7 8 12 10
F19 DevOps security pipeline 30 61 78 4 7 9 15 13

determine the ranks of each investigated success factors

Fig. 11 Proposed hierarchal structure of the problem
at stage 2 and stage 3, respectively. Figure 11 shows the
hierarchy structure of the present study.
Step 2 Pairwise comparison
The key aim of FAHP analysis is to prioritize the
investigated success factors and their core categories con-
cerning the significance of DevOps practices in software
organizations. The pairwise comparison is performed to
and their core categories. To perform the pairwise com-
parison, the second survey was conducted with the experts.
We have developed a survey instrument and contact with
the respondents of the first survey study. A total of 34
respondents replied, and they agreed for participating in
pairwise comparison analysis. The survey questionnaire
developed for pairwise comparison was sent to the experts.
A sample of the used questionnaire for pairwise compar-
ison was given in ‘‘Appendix C’’ section. We received the
responses from the experts and manually reviewed to check
the uncompleted responses. During the manual review, we
did not find any uncompleted response, though all the 34
response were considered for FAHP analysis. The sample
size of pairwise comparison survey (34 responses) is small
and might not strong enough to generalize the results of
FAHP analysis. The literature renders that the FAHP is a
subjective methodology and the small data set is also
acceptable for results generalization, as the several existing
studies also consider the small sample size for AHP anal-
ysis. For example, Cheng and Li (2002) collected data from
nine experts to conduct the pairwise comparison of con-
struction partnering success factors prioritize the success
factors. Shameem et al. (2018) received five responses and

123
 M. A. Akbar et al.

Table 6 Triangular fuzzy

Linguistic scale Triangular fuzzy scale Triangular fuzzy reciprocal scale
conversion scale (Rahman and
Williams 2016) ‘‘Just equal (JE)’’ (1, 1, 1) (1, 1, 1)
‘‘Equally important (EI)’’ (0.5, 1, 1.5) (0.6, 1, 2)
‘‘Weakly important (WI)’’ (1, 1.5, 2) (0.5, 0.6, 1)
‘‘Strongly more important (SMI)’’ (1.5, 2, 2.5) (0.4, 0.5, 0.6)
‘‘Very strongly more important (VSMI)’’ (2, 0.5, 3) (0.3, 0.4, 0.5)
‘‘Absolutely more important (AMI)’’ (2.5, 3, 3.5) (0.2, 0.3, 0.4)

Table 7 Pairwise comparison of Success factors categories The priority vector of each main category of success
factors is listed in Table 7. Local priority weight (LPW) of
Success factors categories
all the main categories of the factors were calculated using
Change Business People Eq. 3. First, the synthetic extent values of four categories,
Change (1,1,1) (1.5, 2.5, 3) (1, 1.5, 2) i.e., Organizational Management, process, technology, and
Business (0.3, 0.4, 0.6) (1,1,1) (0.4, 0.5, 0.6) coordination in were determined, and the priority weight of
People (0.5, 0.6, 1) (1.5, 2, 2.5) (1,1,1) each category was calculated using Eq. 4. In the following,
we have provided the calculation of priority weight for all
the categories of the success factors.
X
n X
m
Fgij ¼ ð1; 1; 1Þ þ ð1:5; 2; 2:5Þ þ ð1; 1:5; 2Þ   
performed an AHP analysis to prioritize the success factors i j

of distributed agile software development methodology. þ ð0:5; 0:6; 1Þ þ ð1; 1; 1Þ ¼ ð14:1; 18:2; 22:8Þ
" #1  
Similarly, Wong and Li (2008) performed an AHP Xn Xm
1 1 1
analysis for the selection of intelligent buildings systems Fgij ¼ ; ;
i j
22:8 18:2 14:1
by considering the responses received from nine experts.
¼ ð0:04386; 0:054945; 0:070922Þ
Therefore, we have conducted a FAHP analysis based on X
m
j
the data received from 34 experts. By considering the data Fg1 ¼ ð1; 1; 1Þ þ ð1:5; 2:5; 3Þ þ ð1; 1:5; 2Þ ¼ ð5; 7; 8:5Þ
j¼1
sample of exiting studies, we can justify that the sample
X
m
size of 34 experts is acceptable for generalizing the study j
Fg2 ¼ ð0:3; 0:4; 0:6Þ þ ð1; 1; 1Þ þ ð0:4; 0:5; 0:6Þ ¼ ð2:2; 2:5; 3:2Þ
results. j¼1

The geometric mean is used to transform the survey X

m
j
Fg3 ¼ ð0:5; 0:6; 1Þ þ ð1:5; 2; 2:5Þ þ ð1; 1; 1Þ ¼ ð4; 5:1; 6:5Þ
response collected via the second survey (FAHP survey); j¼1
the geometric mean is an effective method to convert the
expert’s judgments into TFN numbers. The formula used to The ‘‘Change,’’ ‘‘Business,’’ and ‘‘People’’ represent the
calculate the geometric mean is given below: synthesis values of main categories which were calculated
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi using Eq. 4 as follow:
Geometric mean ¼ n t1 t2 t3. . .tn
" #1
X
m n X
X m
t ¼ Weight of each response Change ¼ j
Fg1  Fgij
n ¼ Number of responses j i j

¼ ð5; 7; 8:5Þ  ð0:04386; 0:054945Þ ¼ ð0:219298; 0:384615Þ

Step 3 Test the consistency of the pairwise matrix
In this section, we presented a step-by-step calculation
of the procedure followed to check whether a given pair-
wise matrix is consistent or not. For this, we have con-
sidered the Table of Categories (Table 6). A triangular
fuzzy number of the pairwise comparison matrix of the
main categories is defuzzified to crisp number using Eq. 14
and obtained the corresponding Fuzzy Crisp Matrix (FCM)
as shown in Table 7:
Step 4 Calculating the local priority weight of each
success factor and their respective categories
(i) A numerical example
Business ¼ ð2:2; 2:5; 3:2Þ  ð0:04386; 0:054945Þ ¼ ð0:096491; 0:137363Þ
People ¼ ð4; 5:1; 6:5Þ  ð0:04386; 0:054945Þ ¼ ð0:175439; 0:280220Þ
The degree of possibility using Eq. 6 is determined.
The minimum degree of possibility (priority weight) for
each pairwise comparison was calculated using Eq. 8.
Therefore, the weight vector was determined as W0 = (1,
0.030019, 0.69836) (Table 8). When these values were
normalized, the importance of attributes was calculated as
W = (0.4789, 0.01435, 0.3337). The given results reveal
that Organizational management is the most significant
category as it has highest priority weight as compared to
the other categories of the success factors.

123
Identification and prioritization of DevOps success factors using fuzzy-AHP approach

Table 8 Results of V values for

Change Business People d (priority weight)
criteria
V (Change C ….) – 1 1 1
V (Business C ….) 0.030019 – 0.26502 0.030019
V (People C ….) 0.69836 1 – 0.69836

(ii) Test the consistency of the pairwise matrix 0.9 for n = 4 (Table 5). Therefore, Eqs. 18 and 19 are used
to calculate the consistency index and consistency ration as
In this section, we presented a step-by-step calculation
follows:
of the procedure followed to check whether a given pair-
wise matrix is consistent or not. For this, we have con- kmax  n 3:0707  3
CI ¼ ¼ ¼ 0:035553 ð18Þ
sidered the Table of Categories (Table 9). A triangular n1 31
fuzzy number of the pairwise comparison matrix of the CI 0:035553
main categories are defuzzified to crisp number using CR ¼ ¼ ¼ 0:061 ð19Þ
RI 0:58
Eq. 14 and obtained the corresponding Fuzzy Crisp Matrix
The calculated value of CR is 0.061 \ 0.10; therefore,
(FCM) as shown in Table 9:
the pairwise comparison matrix developed for the cate-
The largest Eigenvector (kmax) value of the FCM matrix
gories of success factors is consistent and acceptable.
is calculated by calculating the column sum of each col-
Similarly, the consistency ratio for all the categories are
umn of FCM matrix (Table 9) and then divide each ele-
checked, and the results are given in Tables 11, 12, 13, 14
ment of FCM matrix by column sum. Moreover, the
and 15, respectively.
priority weight is calculated by taking the average of each
row, as shown in Table 10.
X hX i
4.4 Phase 5: Determining the ranking
kmax ¼ Cj fWg ð17Þ of the success factors
P
where Cj = sum of the columns of Matrix [C] (Table 7), Local weights (LW) of the factors were determined based
W = weight vector (Table 10), therefore on the weight vector value (W) in their respective cate-
gories. The LW shows the local rank (LR) order of a
kmax ¼ 3:6  0:37938 þ 5:5  0:14945 þ 3:2  0:27593 þ
specific success factor in a category. For example, in
¼ 3:0707
‘‘People’’ category, F1 (Emphasize culture rather than
tools, W = 0.42) is ranked as the highest priority factor
Based on the calculation, the largest Eigen value (kmax)
(Table 15), because its weight vector value is higher than
of the matrix FCM is 3.0707. The dimension of FCM is 4.
the other factors. Similarly, F3 (Empowerment, W = 0.241)
Therefore n = 4 and the Random Consistency Index (RI) is
and F5 (Established skilled DevOps Teams, W = 0.201)
are, respectively, 2nd and 3rd most top-ranked success
factors in the People category. The same approach was
Table 9 Fuzzy crisp matrix (FCM) for success factors categories
used to calculate the local ranks of all the other factors and
Change Business Process their categories (Table 15). The aim of calculating the local
Change 1.9 2.5 1.5 rank is to check the significance of the identified success
Business 0.9 1.0 0.7
factors in their specific categories. The local rank-based
Process 0.8 2.0 1.0
classification will assist the practitioners to consider the
Column sum 3.6 5.5 3.2
highest ranked success factors at the top priority and
interest.
We further determined the global weight (GW) of each
success factor to measure their overall impact on DevOps
practices. We globally rank (i.e., among all the identified
Table 10 Normalized matrix of success factors categories factors) each factor based on their global weight, which is
Change Business People Priority vector weight calculated by multiplying the value of the local weight of a
specific factor with the respective category weight. For
Change 0.37037 0.35714 0.40541 0.37938 example, the global weight of F1 = local weight of
Business 0.18519 0.14286 0.13514 0.14945 F1 9 category weight (i.e., C1: Peo-
People 0.25926 0.28571 0.27027 0.27593 ple) = 0.420 9 0.37938 = 0.159. Based on the given GW
value of F1, it is ranked as the most significant factor for

123
 M. A. Akbar et al.

Table 11 Pairwise comparison

C1
of ‘‘people’’ category
CH1 CH2 CH3 CH4 CH5

F1 (1,1,1) (0.3, 0.4, 0.5) (0.4, 0.5, 0.6) (1.5, 2, 2.5) (0.4, 0.5, 0.6)
F2 (2, 2.5, 3) (1,1,1) (2, 2.5, 3) (0.5, 1, 1.5) (1, 1.5, 2)
F3 (1.5, 2, 2.5) (0.3, 0.4, 0.5) (1,1,1) (2, 2.5, 3) (2.5, 3, 3.5)
F4 (0.4, 0.5, 0.6) (0.6, 1, 2) (0.3, 0.4, 0.5) (1,1,1) (0.5, 0.6, 1)
F5 (1.5, 2, 2.5) (0.5, 0.6, 1) (0.2, 0.3, 0.4) (1, 1.5, 2) (1,1,1)
Imax = 5.50, CI = 0.12, CR = 0.10

Table 12 Pairwise comparison

C2
of ‘‘business’’ category
F6 F7 F8 F9 F10 F11

F6 (1,1,1) (0.4, 0.5, 0.6) (1.5, 2, 2.5) (0.3, 0.4, 0.5) (0.4, 0.5, 0.6) (1, 1.5, 2)
F7 (1.5, 2, 2.5) (1,1,1) (2, 2.5, 3) (0.5, 1, 1.5) (1, 1.5, 2) (1, 1.5, 2)
F8 (0.4, 0.5, 0.6) (0.3, 0.4, 0.5) (1,1,1) (2, 2.5, 3) (2.5, 3, 3.5) (0.4, 0.5, 0.6)
F9 (2, 2.5, 3) (0.6, 1, 2) (0.3, 0.4, 0.5) (1,1,1) (0.5, 0.6, 1) (1, 1.5, 2)
F10 (1.5, 2, 2.5) (0.5, 0.6, 1) (0.2, 0.3, 0.4) (1, 1.5, 2) (1,1,1) (2, 2.5, 3)
F11 (0.5, 0.6, 1) (0.5, 0.6, 1) (0.5, 0.6, 1) (0.5, 0.6, 1) (0.5, 0.6, 1) (1,1,1)
Imax = 5.85, CI = 0.21, CR = 0.19

Table 13 Pairwise comparison of ‘‘change’’ category

C3
F12 F13 F14 F15 F16 F17 F18 F19

F12 (1,1,1) (1, 1.5, 2) (2.5, 3, 3.5) (0.6, 1, 2) (1.5, 2, 2.5) (1, 1.5, 2) (0.5, 0.6, 1) (0.3, 0.4, 0.5)
F13 (0.5, 0.6, 1) (1,1,1) (0.5, 0.6, 1) (1, 1.5, 2) (0.4, 0.5, 0.6) (1, 1.5, 2) (2, 2.5, 3) (1, 1.5, 2)
F14 (0.2, 0.3, 0.4) (1, 1.5, 2) (1,1,1) (0.5, 1, 1.5) (0.5, 0.6, 1) (0.5, 0.6, 1) (1, 1.5, 2) (0.5, 0.6, 1)
F15 (0.5, 1, 1.5) (0.5, 0.6, 1) (0.6, 1, 2) (1,1,1) (0.2, 0.3, 0.4) (2, 2.5, 3) (0.5, 1, 1.5 (2, 2.5, 3)
F16 (0.4, 0.5, 0.6) (1.5, 2, 2.5) (1, 1.5, 2) (2.5, 3, 3.5) (1,1,1) (0.4, 0.5, 0.6) (0.2, 0.3, 0.4) (1, 1.5, 2)
F17 (0.5, 0.6, 1) (0.5, 0.6, 1) (1, 1.5, 2) (0.3, 0.4, 0.5) (1.5, 2, 2.5) (1,1,1) (0.4, 0.5, 0.6) (2, 2.5, 3)
F18 (1, 1.5, 2) (0.3, 0.4, 0.5) (0.5, 0.6, 1) (0.6, 1, 2) (2.5, 3, 3.5) (1.5, 2, 2.5) (1,1,1) (0.4, 0.5, 0.6)
F19 (2, 2.5, 3) (0.5, 0.6, 1) (1, 1.5, 2) (0.3, 0.4, 0.5) (0.5, 0.6, 1 (0.3, 0.4,)0.5) (1.5, 2, 2.5) (1,1,1)
Imax = 10.4, CI = 0.17, CR = 0.11

Table 14 Pairwise comparison of in between the categories your DevOps strategy), GW = 0.13), is recognized as the
Change Business People
second highest priority success factor for cloud-based
software development outsourcing. Moreover, F18 (Real-
Change (1,1,1) (2, 2.5, 3) (1, 1.5, 2) Time Feedback, GW = 0.12) and F3 (Empowerment,
Business (0.3, 0.4, 0.5) (1,1,1) (0.3, 0.4, 0.5) GW = 0.09) are ranked as the 3rd and 4th most priority
People (0.5, 0.6, 1) (2, 2.5, 3) (1,1,1) factors for the DevOps practices.

4.5 Phase 6: Taxonomy of success factors

DevOps activities. Similarly, the GW values for all the The final phase of this study aim is the development of
other factors are identified and listed in Table 15. Based on prioritization-based taxonomy of the identified DevOps
the results given in Table 15, we could see that F17 (Assess success factors. The prioritization-based taxonomy consists
of calculated the local and global ranks of the success

123
Identification and prioritization of DevOps success factors using fuzzy-AHP approach

Table 15 Success factors ranking

Categories Categories weight Success Local weights Local ranking Global weights Global ranking
(CW) factors (LW) (LR) (GW) (GR)

People 0.3798 F1 0.42 1 0.16 1

F2 0.16 5 0.06 7
F3 0.241 2 0.09 4
F4 0.18 4 0.07 6
F5 0.201 3 0.08 5
Business 0.14945 F6 0.342 2 0.05 8
F7 0.176 4 0.03 10
F8 0.378 1 0.06 7
F9 0.104 6 0.02 11
F10 0.17 5 0.03 10
F11 0.32 3 0.05 8
Change 0.27593 F12 0.34 3 0.09 4
F13 0.191 5 0.05 8
F14 0.161 6 0.04 9
F15 0.21 4 0.06 7
F16 0.12 7 0.03 10
F17 0.487 1 0.13 2
F18 0.45 2 0.12 3
F19 0.063 8 0.02 11

factors (Table 14, Fig. 12). The developed taxonomy
shows that impact of success factors locally (within the
category) and globally (overall project). For example, the
taxonomies results presented in Fig. 12 shows that F1
(Emphasize culture rather than tools) is ranked as the 1st
highest ranked success factor in ‘‘People’’ category, and
globally, it is also ranked as the most significant success
factor for the success and progression of DevOps practices
in software organizations. This indicated that F1 is highly
important to address for the successfulness of People cat-
egory, as well as it is important for overall study objective.
This is because while pairwise comparison, the rest of the
success factors of other categories are more important to
address the overall study objective, i.e., successful imple-
mentation of DevOps practices in the software industry.
The results (Fig. 12) show that Based on the results given
in Table 15, we could see that F17 (Assess your DevOps
strategy) is recognized as the second highest priority suc-
cess factor for cloud-based software development out-
sourcing. Moreover, F18 (Real-Time Feedback) and F3
(Empowerment) are ranked as the 3rd and 4th most priority
factors for the DevOps practices. The developed taxonomy
will assist the industry practitioners and academic
researchers to develop new strategies for the success and
progression of DevOps practice.
4.6 Sensitivity analysis
We performed a sensitivity analysis by changing the
importance of different categories of the model and ana-
lyzed how the weights of success factors are impacted. We
individually maximized the importance of ‘‘business’’ and
‘‘change’’ categories and compared the ranking of the
success factors with results of the model. The maximum
importance of an individual category was assigned a value
of 0.3798, which the importance weight of highest ranked
category (i.e., ‘‘people’’) in the model.
The sensitivity analysis results indicate that when the
‘‘change’’ category is maximized, F1, F12, F17 and F18
still remain the top four ranked success factors. Similarly,
F7, F9, F10 and F19 also remain the lowest four ranked
success factors. Moreover, the middle-ranked success fac-
tors also change within a small interval.
The sensitivity analysis results also indicate that when
the ‘‘business’’ category is maximized and the ‘‘people’’
category is assigned the weight of ‘‘business’’ category,
F12, F17 and F18 remain one of the highest ranked success
factors except F1. This indicates that if the ‘‘business’’
category is the highest priority for stakeholders then F8
will be the highest ranked success factor instead of F1. On
the other hand, the lowest ranked success factors will be
F2, F5, F16 and F19. Moreover, the middle-ranked success

123

Fig. 12 Taxonomy of success
factors
Fig. 13 Sensitivity analysis
factors also change within a small interval. Figure 13
summaries the results of the sensitivity analysis.
5 Threats to validity
An important threat toward the validity of this study’s
findings could be the researcher’s bias and inconsistency in
the data extraction process. To address this threat, we have
123
M. A. Akbar et al.
conducted an inter-rater reliability test, and the results
show that the extracted data were unbiased and consistent.
Another internal threat to validity is that the developed
search string was executed on limited academic databases
and search engines. This may cause incomplete data col-
lection sources, and some related studies may be omitted.
Moreover, the developed search string was executed on
selected digital libraries and web-search engines. As the
search mechanism of every library and search engines vary,
Identification and prioritization of DevOps success factors using fuzzy-AHP approach

Table 16 Summary of research findings

Research questions Findings

[RQ1] What success factors of DevOps practices are
reported in the literature?
[RQ2] What does practitioners do think about the
success factors of DevOps practices?
[RQ3] How the identified success factors were
prioritize using fuzzy AHP?
[RQ4] What would be the prioritization-based
taxonomy of success factors?
Emphasize culture rather than tools (F1)
Attempt matrix organization and transparency (F2)
Empowerment (F3)
Cross-functional team (F4)
Established skilled DevOps Teams (F5)
Design a common baseline (F6)
Keep to the sequencing of the DevOps approach (F7)
Internal DevOps Events (F8)
Demonstrating Lean Leadership behavior (F9)
Continues integration and deployment (F10)
Measure progress and plan next improvement (F11)
Use modeling (F12)
Integrate the challenges of operations and support (F13)
Automatic testing techniques (F14)
Accommodate legacy systems (F15)
Use system orchestration (F16)
Assess your DevOps strategy (F17)
Real-Time Feedback (F18)
DevOps Security pipeline (F19)
According to the expert’s opinions, all the identified factors could positively impact the
DevOps practices
The step-by-step protocols of fuzzy AHP is applied to prioritize the investigated success
factors. The fuzzy AHP results show that F1 (Emphasize culture rather than tools),
F17 (Assess your DevOps strategy), F18 (Real-Time Feedback) and F3 (Empowerment)
are ranked as the 3rd and 4th most significant success factors for DevOps practices in
software organizations
The prioritization-based taxonomy of the success factors is developed (Fig. 12). The
developed taxonomy presents the local and global ranks of the success factors which
assists the practitioners and researchers concerning their interest

though there is a chance of improper execution of search 6 Study implication

string which may lead to omitting the related studies.
However, based on the existing literature review studies,
this is not a systematic omission (Niazi et al. 2016; Akbar
et al. 2019; Khan et al. 2017, 2018).
The DevOps success factors were extracted from the
available state-of-the-art literature and validated by con-
ducting the empirical study with the experts. The feedback
of the survey participants revealed that the reported success
factors are related to their work. It is noted that most of the
survey study respondents are from developing countries.
Moreover, external threat to validity focuses on general-
ization of the results. Most survey participants are from the
Asia and there is a representative set of participants from
non-Asian organizations. Hence, the findings of the study
are generalizable.
The study provides the state-of-the-art on the success fac-
tors associated with DevOps practices. The identified fac-
tors were also mapped to three core areas of SPI manifesto.
The identified success factors and their respective mapping
to core areas of SPI manifesto will help practitioners in a
successful implementation of DevOps-based projects.
Moreover, the study also addresses the multi-criteria-
based decision-making problem for the success factors
associated with DevOps. The prioritization-based taxon-
omy developed in this study provides the rank order for the
identified success factors and their mapping to three key
categories. We believe the taxonomy will serve as the
body-of-knowledge for the practitioners as it helps them
identify key success factors and their categories during the
implementation of a DevOps project.

123

7 Summary of research questions findings
To address the objective of this study is to identify and
prioritize the success factors of DevOps practices. A total
of 19 success factors were identified using the SLR
approach. The identified factors were mapped in SPI
manifesto. The identified factors and their mapping results
were empirically validated with experts. Finally, the fuzzy-
AHP approach has been applied to prioritize the investi-
gated success factors with respect to their significance for
DevOps practices. Therefore, the summarized answers of
research questions are presented in Table 16.
8 Conclusion and future directions
The importance of DevOps adoption has motivated us to
explore the factors that positively influence the DevOps
practices in the software industry. The systematic literature
review study was adopted, and a total 19 success factors
were identified. The identified factors were classified in the
core categories of software process improvement mani-
festo. Next, an empirical study was conducted to validate
the identified success factors and their mapping results. The
results show that the identified factors and their mapping is
related to industry practices. Finally, the fuzzy AHP tech-
nique was applied to p prioritize the identified success
factors concerning their significance for the success and
progression of DevOps practices. The fuzzy-AHP results
indicate that the ‘‘people’’ is the most significant category
of the success factors. Global ranks indicates that ‘‘em-
phasize culture rather than tools, ‘‘assess your DevOps
strategy,’’ ‘‘real-Time Feedback’’ and ‘‘empowerment’’ are
top four success factors that should be addressed for the
successful implementation of DevOps practices. Moreover,
the study provides the prioritization-based taxonomy of the
success factors which will assist both academic researchers
and industry practitioners to develop effective techniques
for the implementation of DevOps practices in a software
organization.
The scope of the SLR was carried out on six mainstream
research databases as well as Google scholar. Due to
dynamic nature of the research publications in DevOps
domain, some recent publications could have been missed
at the time of consolidating the findings of the study.
However, we believe that our results are comprehensive
and cover the relevant published literature. With respect to
the questionnaire survey, the pilot study results provided
internal validity as success factors included in the survey
were result of a systematic literature review. Another
potential limitation of the study could be related to external
threat to validity as most of survey participants are from
123
M. A. Akbar et al.
Asia. Hence, findings of the study can be extended by
incorporating input from organizations in USA and Euro-
pean regions in future studies.
In future, we plan to conduct a multivocal literature
review study to collect evidence from the gray literature for
the success factors for DevOps projects. We also plan to
conduct another multivocal literature review study to
identify challenges associated with the adoption of DevOps
practices.
Acknowledgements The authors are grateful to the Deanship of
Scientific Research, King Saud University for funding through Vice
Deanship of Scientific Research Chairs.
Compliance with ethical standards
Conflict of interest All authors declare that there is no conflict of
interest.
Appendix A
Selected primary studies and quality assessment score
(https://tinyurl.com/uq2vjc6).
Appendix B
Sample of survey questionnaire (https://tinyurl.com/
wcvf4xw).
Appendix C
Sample of Fuzzy-AHP questionnaire (https://tinyurl.com/
wwj9jvg).
References
Afzal W, Torkar R, Feldt R (2009) A systematic review of search-
based testing for non-functional system properties. Inf Softw
Technol 51:957–976
Akbar MA (2019) SRCMIMM: managing requirements change
activities in global software development: student research
abstract. In: Proceedings of the 34th ACM/SIGAPP symposium
on applied computing, pp 1633–1636
Akbar MA, Sang J, Khan AA, Amin F-E, Hussain S, Sohail MK et al
(2018) Statistical analysis of the effects of heavyweight and
lightweight methodologies on the six-pointed star model. IEEE
Access 6:8066–8079
Akbar MA, Sang J, Khan AA, Mahmood S, Qadri SF, Hu H et al
(2019a) Success factors influencing requirements change man-
agement process in global software development. J Comput
Lang 51:112–130
Identification and prioritization of DevOps success factors using fuzzy-AHP approach
Akbar MA, Sang J, Khan AA, Hussain S (2019b) Investigation of the
requirements change management challenges in the domain of
global software development. J Softw Evol Process 31:e2207
Akbar MA, Sang J, Khan AA, Shafiq M (2019c) Towards the
guidelines for requirements change management in global
software development: client-vendor perspective. IEEE Access
7:76985–77007
Albayrak E, Erensal YC (2004) Using analytic hierarchy process
(AHP) to improve human performance: an application of
multiple criteria decision making problem. J Intell Manuf
15:491–503
Ayhan MB (2013) A fuzzy AHP approach for supplier selection
problem: a case study in a gear motor company. arXiv preprint
arXiv:1311.2886
Banica L, Radulescu M, Rosca D, Hagiu A (2017) Is DevOps another
project management methodology? Informatica Economica
21:39
Benguria G, Alonso J, Etxaniz I, Orue-Echevarria L, Escalante M
(2018) Agile development and operation of complex systems in
multi-technology and multi-company environments: following a
DevOps approach. In: European conference on software process
improvement, pp 15–27
Bland M (2015) An introduction to medical statistics. Oxford
University Press, Oxford
Bouyssou D, Marchant T, Pirlot M, Perny P, Tsoukias A, Vincke P
(2000) Evaluation and decision models: a critical perspective,
vol 32. Springer, Berlin
Callanan M, Spillane A (2016) DevOps: making it easy to do the right
thing. IEEE Softw 33:53–59
Céspedes D, Angeleri P, Melendez K, Dávila A (2019) Software
product quality in DevOps contexts: a systematic literature
review. In: International conference on software process
improvement, pp 51–64
Chamodrakas I, Batis D, Martakos D (2010) Supplier selection in
electronic marketplaces using satisficing and fuzzy AHP. Expert
Syst Appl 37:490–498
Chang D-Y (1996) Applications of the extent analysis method on
fuzzy AHP. Eur J Oper Res 95:649–655
Chen L (2015) Continuous delivery: huge benefits, but challenges too.
IEEE Softw 32:50–54
Chen L (2017) Continuous delivery: overcoming adoption challenges.
J Syst Softw 128:72–86
Chen L, Ali Babar M, Zhang H (2010) Towards an evidence-based
understanding of electronic data sources
Chen H-M, Kazman R, Haziyev S, Kropov V, Chtchourov D (2015)
Architectural support for DevOps in a neo-metropolis BDaaS
platform. In: 2015 IEEE 34th symposium on reliable distributed
systems workshop (SRDSW), pp 25–30
Cheng EW, Li H (2002) Construction partnering process and
associated critical success factors: quantitative investigation.
J Manag Eng 18:194–202
Cois CA, Yankel J, Connell A (2014) Modern DevOps: optimizing
software development through effective system interactions. In
2014 IEEE international professional communication conference
(IPCC), pp 1–7
Dyck A, Penners R, Lichter H (2015) Towards definitions for release
engineering and DevOps. In: 2015 IEEE/ACM 3rd international
workshop on release engineering, pp 3–3
Easterbrook S, Singer J, Storey M-A, Damian D (2008) Selecting
empirical methods for software engineering research. In: Guide
to advanced empirical software engineering, pp 285–311.
Springer
Elberzhager F, Arif T, Naab M, Süß I, Koban S (2017) From agile
development to devops: going towards faster releases at high
quality–experiences from an industrial context. In: International
conference on software quality, pp 33–44
Erich F, Amrit C, Daneva M (2014) Report: Devops literature review.
University of Twente, Tech. Rep., Drienerlolaan
Farid AB, Helmy YM, Bahloul MM (2017) Enhancing lean software
development by using DevOps practices. Int J Adv Comput Sci
Appl 8:267–277
Finstad K (2010) Response interpolation and scale sensitivity:
evidence against 5-point scales. J Usability Stud 5:104–110
Forsgren N (2018) DevOps delivers. Commun ACM 61:32–33
Gill AQ, Loumish A, Riyat I, Han S (2018) DevOps for information
management systems. VINE J Inf Knowl Manag Syst
Gruver G, Mouser T (2015) Leading the transformation: applying
agile and devops principles at scale. IT Revolution, Melbourne
Güngör Z, Serhadlıoğlu G, Kesen SE (2009) A fuzzy AHP approach
to personnel selection problem. Appl Soft Comput 9:641–646
Gupta V, Kapur PK, Kumar D (2017) Modeling and measuring
attributes influencing DevOps implementation in an enterprise
using structural equation modeling. Inf Softw Technol 92:75–91
Inayat I, Salim SS, Marczak S, Daneva M, Shamshirband S (2015) A
systematic literature review on agile requirements engineering
practices and challenges. Comput Hum Behav 51:915–929
Jabbari R, Bin Ali N, Petersen K, Tanveer B (2016) What is DevOps?
A systematic mapping study on definitions and practices. In:
Proceedings of the scientific workshop proceedings of XP2016,
pp 1–11
Jabbari R, Bin Ali N, Petersen K, Tanveer B (2018) Towards a
benefits dependency network for DevOps based on a systematic
literature review. J Softw Evol Process 30:e1957
Jiménez M, Rivera LF, Villegas NM, Tamura G, Müller HA, Gallego
P (2018) DevOps’ shift-left in practice: an industrial case of
application. In: International workshop on software engineering
aspects of continuous development and new paradigms of
software production and deployment, pp 205–220
John W, Marchetto G, Nemeth F, Skoldstrom P, Steinert R, Meirosu
C et al (2017) Service provider devops. IEEE Commun Mag
55:204–211
Kabra G, Ramesh A, Arshinder K (2015) Identification and priori-
tization of coordination barriers in humanitarian supply chain
management. Int J Disaster Risk Reduct 13:128–138
Kahraman C, Cebeci U, Ruan D (2004) Multi-attribute comparison of
catering service companies using fuzzy AHP: the case of Turkey.
Int J Prod Econ 87:171–184
Kamuto MB, Langerman JJ (2017) Factors inhibiting the adoption of
DevOps in large organisations: South African context. In: 2017
2nd IEEE international conference on recent trends in electron-
ics, information and communication technology (RTEICT),
pp 48–51
Keshta I, Niazi M, Alshayeb M (2017) Towards implementation of
requirements management specific practices (SP1. 3 and SP1. 4)
for Saudi Arabian small and medium sized software develop-
ment organizations. IEEE Access 5:24162–24183
Khan SU, Niazi M, Ahmad R (2011) Factors influencing clients in the
selection of offshore software outsourcing vendors: an explora-
tory study using a systematic literature review. J Syst Softw
84:686–699
Khan AA, Keung J, Niazi M, Hussain S, Ahmad A (2017a)
Systematic literature review and empirical investigation of
barriers to process improvement in global software development:
client–vendor perspective. Inf Softw Technol 87:180–205
Khan AA, Keung JW, Abdullah-Al-Wadud M (2017b) SPIIMM:
toward a model for software process improvement implementa-
tion and management in global software development. IEEE
Access 5:13720–13741
Khan AA, Keung J, Hussain S, Niazi M, Kieffer S (2018) Systematic
literature study for dimensional classification of success factors
affecting process improvement in global software development:
client–vendor perspective. IET Softw 12:333–344
123

Khan AA, Shameem M, Kumar RR, Hussain S, Yan X (2019) Fuzzy
AHP based prioritization and taxonomy of software process
improvement success factors in global software development.
Appl Soft Comput 83:105648
Kitchenham B, Charters S (2007) Guidelines for performing system-
atic literature reviews in software engineering
Kitchenham B, Pfleeger SL (2003) Principles of survey research part
6: data analysis. ACM SIGSOFT Softw Eng Notes 28:24–27
Kitchenham B, Brereton OP, Budgen D, Turner M, Bailey J, Linkman
S (2009) Systematic literature reviews in software engineering–a
systematic literature review. Inf Softw Technol 51:7–15
Leite L, Rocha C, Kon F, Milojicic D, Meirelles P (2019) A survey of
DevOps concepts and challenges. ACM Comput Surv CSUR
52:1–35
Leppänen M, Mäkinen S, Pagels M, Eloranta V-P, Itkonen J, Mäntylä
MV et al (2015) The highways and country roads to continuous
deployment. IEEE Softw 32:64–72
Lwakatare LE, Kuvaja P, Oivo M (2016) Relationship of DevOps to
agile, lean and continuous deployment. In: International confer-
ence on product-focused software process improvement,
pp 399–415
Lwakatare LE, Karvonen T, Sauvola T, Kuvaja P, Olsson HH, Bosch
J et al (2016) Towards DevOps in the embedded systems
domain: why is it so hard? In: 2016 49th Hawaii international
conference on system sciences (HICSS), pp 5437–5446
Mahmood S, Anwer S, Niazi M, Alshayeb M, Richardson I (2017)
Key factors that influence task allocation in global software
development. Inf Softw Technol 91:102–122
Marijan D, Liaaen M, Sen S (2018) DevOps improvements for
reduced cycle times with integrated test optimizations for
continuous integration. In: 2018 IEEE 42nd annual computer
software and applications conference (COMPSAC), pp 22–27
McCarthy MA, Herger LM, Khan SM, Belgodere BM (2015)
Composable DevOps: automated ontology based DevOps matu-
rity analysis. In: 2015 IEEE international conference on services
computing, pp 600–607
Mohan V, Othmane LB (2016) Secdevops: is it a marketing
buzzword?-mapping research on security in devops. In: 2016
11th international conference on availability, reliability and
security (ARES), pp 542–547
Mulubrhan F, Mokhtar AA, Muhammad M (2014) Comparative
analysis between fuzzy and traditional analytical hierarchy
process. In MATEC web of conferences, p 01006
Niazi M, Mahmood S, Alshayeb M, Qureshi AM, Faisal K, Cerpa N
(2016a) Toward successful project management in global
software development. Int J Project Manag 34:1553–1567
Niazi M, Mahmood S, Alshayeb M, Riaz MR, Faisal K, Cerpa N et al
(2016b) Challenges of project management in global software
development: a client-vendor analysis. Inf Softw Technol
80:1–19
Nogueira AF, Ribeiro JC, Zenha-Rela M, Craske A (2018) Improving
la redoute’s ci/cd pipeline and devops processes by applying
machine learning techniques. In: 2018 11th international
conference on the quality of information and communications
technology (QUATIC), pp 282–286
Noy C (2008) Sampling knowledge: the hermeneutics of snowball
sampling in qualitative research. Int J Soc Res Methodol
11:327–344
Olszewska M, Waldén M (2015) DevOps meets formal modelling in
high-criticality complex systems. In: Proceedings of the 1st
international workshop on quality-aware DevOps, pp 7–12
Önüt S, Kara SS, Işik E (2009) Long term supplier selection using a
combined fuzzy MCDM approach: a case study for a telecom-
munication company. Expert Syst Appl 36:3887–3895
Pries-Heje JJ (2010) SPI manifesto. Version A.1.2.2010
123
M. A. Akbar et al.
Rahman AAU, Williams L (2016) Software security in devops:
synthesizing practitioners’ perceptions and practices. In: 2016
IEEE/ACM international workshop on continuous software
evolution and delivery (CSED), pp 70–76
Rajkumar M, Pole AK, Adige VS, Mahanta P (2016) DevOps culture
and its impact on cloud delivery and software development. In:
2016 international conference on advances in computing,
communication, and automation (ICACCA) (Spring), pp 1–6
Riungu-Kalliosaari L, Mäkinen S, Lwakatare LE, Tiihonen J,
Männistö T (2016) DevOps adoption benefits and challenges in
practice: a case study. In: International conference on product-
focused software process improvement, pp 590–597
Roche J (2013) Adopting DevOps practices in quality assurance.
Commun ACM 56:38–43
Saaty TL (2013) Analytic hierarchy process. In: Encyclopedia of
operations research and management science, pp 52–642013
Sacks M (2012) Devops principles for successful web sites. In: Pro
website development and operations, pp 1–14. Springer, Berlin
Samarawickrama SS, Perera I (2017) Continuous scrum: a framework
to enhance scrum with DevOps. In: 2017 Seventeenth interna-
tional conference on advances in ICT for emerging regions
(ICTer), pp 1–7
Senapathi M, Buchan J, Osman H (2018) DevOps capabilities,
practices, and challenges: insights from a case study. In:
Proceedings of the 22nd international conference on evaluation
and assessment in software engineering 2018, pp 57–67
Shahin M, Babar MA, Zahedi M, Zhu L (2017) Beyond continuous
delivery: an empirical investigation of continuous deployment
challenges. In: 2017 ACM/IEEE international symposium on
empirical software engineering and measurement (ESEM),
pp 111–120
Shameem M, Kumar RR, Kumar C, Chandra B, Khan AA (2018)
Prioritizing challenges of agile process in distributed software
development environment using analytic hierarchy process.
J Softw Evol Process 30:e1979
Shameem M, Kumar RR, Nadeem M, Khan AA (2020) Taxonomical
classification of barriers for scaling agile methods in global
software development environment using fuzzy analytic hierar-
chy process. Appl Soft Comput 90:106122
Sloane E, Liberatore M, Nydick R, Luo W, Chung Q (2002) Clinical
engineering technology assessment decision support: a case
study using the analytic hierarchy process (AHP). In: Proceed-
ings of the second joint 24th annual conference and the annual
fall meeting of the biomedical engineering society, engineering
in medicine and biology, pp 1950–1951
Smeds J, Nybom K, Porres I (2015) DevOps: a definition and
perceived adoption impediments. In: International conference on
agile software development, pp 166–177
Snyder B, Curtis B (2017) Using analytics to guide improvement
during an Agile–DevOps transformation. IEEE Softw 35:78–83
Taryana A, Setiawan I, Fadli A, Murdyantoro E (2017) Pioneering the
automation of internal quality assurance system of higher
education (iqas-he) using devops approach. In: 2017 interna-
tional conference on sustainable information engineering and
technology (SIET), pp 259–264
Tiwari N (2006) Using the analytic hierarchy process (AHP) to
identify performance scenarios for enterprise application. The
Computer Measurement Group, Google Scholar, San Diego
Virmani M (2015) Understanding DevOps and bridging the gap from
continuous integration to continuous delivery. In: Fifth interna-
tional conference on the innovative computing technology
(INTECH 2015), pp 78–82
Wahaballa A, Wahballa O, Abdellatief M, Xiong H, Qin Z (2015)
Toward unified DevOps model. In: 2015 6th IEEE international
conference on software engineering and service science
(ICSESS), pp 211–214
Identification and prioritization of DevOps success factors using fuzzy-AHP approach
Wen-ying L (2009) Application of AHP analysis in risk management
of engineering projects. J Beijing Univ Chem Technol Soc Sci
Ed 1:46–48
White VJ, Glanville JM, Lefebvre C, Sheldon TA (2001) A statistical
approach to designing search filters to find systematic reviews:
objectivity enhances accuracy. J Inf Sci 27:357–370
Wiedemann A, Forsgren N, Wiesche M, Gewald H, Krcmar H (2019)
Research for practice: the DevOps phenomenon. Commun ACM
62:44–49
Wong JK, Li H (2008) Application of the analytic hierarchy process
(AHP) in multi-criteria analysis of the selection of intelligent
building systems. Build Environ 43:108–125
Yaghoobi T (2018) Prioritizing key success factors of software
projects using fuzzy AHP. J Softw Evol Process 30:e1891
Zadeh LA, Klir GJ, Yuan B (1996) Fuzzy sets, fuzzy logic, and fuzzy
systems: selected papers, vol 6. World Scientific, Singapore
Zhang H, Babar MA, Tell P (2011) Identifying relevant studies in
software engineering. Inf Softw Technol 53:625–637
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
123