Annals of Nuclear Energy 151 (2021) 107969

Contents lists available at ScienceDirect

Annals of Nuclear Energy

journal homepage: www.elsevier.com/locate/anucene

Review

Two decades on human reliability analysis: A bibliometric analysis and

literature review
Lin-Xiu Hou a, Ran Liu a, Hu-Chen Liu b,⇑,1, Shan Jiang a
a
School of Management, Shanghai University, Shanghai 200444, PR China
b
School of Economics and Management, Tongji University, Shanghai 200092, PR China

a r t i c l e i n f o a b s t r a c t

Article history: Human reliability analysis (HRA) is a systematic technique to assess human systematic risk, and has been
Received 21 February 2020 widely used in various industries for enhancing the safety and reliability of complex socio-technical sys-
Received in revised form 19 October 2020 tems. However, there is a lack of a comprehensive bibliometric analysis of the literature on this topic. This
Accepted 20 October 2020
article aims to provide academics and practitioners with a bibliometric overview and visualization of the
Available online 9 November 2020
HRA field. For doing so, 271 publications derived from the Web of Science and published from 2009 to
2020 were analyzed by the CiteSpace regarding cooperation network, co-citation network, and keyword
Keywords:
co-occurrence network. The cooperation network analysis indicates that the most productive authors in
Human reliability analysis
Probabilistic safety assessment
the HRA area are Jinkyun Park, Wondea Jun and Yochan Kim from the Korea Atomic Energy Research
Performance shaping factor Institute, and the selected publications mainly from South Korea, USA, and China. By the co-citation net-
Human error probability work analysis, the core journals are identified as Reliability Engineering & System Safety, Safety Science,
Bibliometric analysis and Journal of Loss Prevention in the Process Industries; the influential authors are Alan D Swain, Erik
Literature review Hollnagel, and Barry A Kirwan; and the emerging research trends are determined as ‘‘decision making”,
‘‘humane performance data”, ‘‘dependence assessment”, and ‘‘ratio magnitude estimation”. The keyword
co-occurrence network analysis shows that the research hotspots in the HRA domain are ‘‘CREAM”,
‘‘quantification”, and ‘‘performance”. Based on these trends and blind spots in the literature, potential
future research directions are also suggested. The results provide valuable information for scholars to
grasp the current situations, hotspots, and future development trends of the HRA area.
Ó 2020 Elsevier Ltd. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Research methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3. Results and discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.1. Publication trend in the HRA field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2. Cooperation network analysis in the HRA field. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2.1. Author cooperation network analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2.2. Institution cooperation network analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2.3. Country cooperation network analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3. Co-citation analysis in the HRA field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3.1. Journal co-citation analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3.2. Author co-citation analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3.3. Document co-citation analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.4. Keyword co-occurrence analysis in the HRA field. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4. Suggestions for future research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

⇑ Corresponding author at: School of Economics and Management, Tongji University, 1239 Siping Road, Shanghai 200092, PR China.
E-mail address: huchenliu@tongji.edu.cn (H.-C. Liu).
1
ORCID: 0000-0003-4566-2107.

https://doi.org/10.1016/j.anucene.2020.107969
0306-4549/Ó 2020 Elsevier Ltd. All rights reserved.
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al. Annals of Nuclear Energy 151 (2021) 107969

5. Conclusions. . . . . . . . . . . . ....... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Declaration of Competing Interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Acknowledgments . . . . . . ....... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
References . . . . . . . . . . . . ....... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1. Introduction body of academic literature within a given research area (Yang

et al., 2020). Consequently, this paper aims to systematically
Human reliability analysis (HRA) is a systematic approach to review the HRA literature on the basis of bibliometric analysis. In
identify, quantify, and reduce the human errors of human–ma- this review, a total of 271 papers published in international jour-
chine systems. It has been performed as a vital part of probabilistic nals from 2009 to 2020 were identified from the database of
risk assessment that considers all possible accident scenarios to Web of Science (WOS). We analyze the scientific articles with
probabilistically evaluate the overall safety of a system. Human respect to cooperation network, co-citation network, and keyword
reliability is defined as the probability of successful performance co-occurrence network. The main objectives of this paper are to
of a task and human errors are the human behaviors that fall out- determine (1) the tendency in terms of the number of publications
side the tolerance scope of a system where a person operates on HRA, (2) the most productive and influential authors, institu-
(Porthin et al., 2020). Normally, five stages are included in an tions, and countries, and their cooperation relationships, (3) the
HRA process: problem definition or analysis scope specification, most impactful journals, articles, and authors based on their cita-
task modelling, human error analysis, human error quantification, tions and co-citations, and (4) the hot research topics and their
and error management recommendation (Sujan et al., 2020). Since trends in the research domain of HRA. In addition, the research
the advent of the first HRA method, i.e., technique for human error gaps, challenges, and future research directions are discussed
rate prediction (THERP) (Swain and Guttman, 1983), a lot of HRA based on the analysis of previous studies.
tools have been proposed for general use or for specific applica- The remaining part of this article is organized as follows: In
tions, which include human error assessment and reduction tech- Section 2, the research methodology and the review process of this
nique (HEART) (Williams, 1988), cognitive reliability and error study are introduced. In Section 3, the results of bibliometric anal-
analysis methods (CREAM) (Hollnagel, 1998), standardized plant ysis are presented with the consideration of the listed research
analysis risk human reliability analysis (SPAR-H) (Gertman et al., aims. Section 4 discusses the blind spots missed by HRA research-
2005), Phoenix (Ekanem et al., 2016), and Petro-HRA (Laumann ers and provides suggestions for the future work. Finally, Section 5
and Rasmussen, 2016). Over the past two decades, HRA has been summarizes and concludes this paper.
used as a systematic technique to mitigate the risk associated with
human errors in the operation of complex systems, and has been 2. Research methodology
applied to a variety of industries, including the nuclear (Kančev,
2020; Kang and Seong, 2020; Park et al., 2019b), petroleum The scientific publications used in this study were retrieved
(Abílio Ramos et al., 2020; Taylor et al., 2020), healthcare (Liu from the WOS database. The WOS was selected because it is a com-
et al., 2018; Pandya et al., 2020), and other safety–critical indus- monly accepted database of abstracts and references from high-
tries (Ung, 2019; Wang et al., 2020). quality and impactful scientific papers (Gaviria-Marin et al.,
To date, some literature reviews in relation to the HAR have 2019; Gil et al., 2020; Shukla et al., 2019), and offers the ability
been conducted from different perspectives. For example, Porthin to export bibliographic meta-data for use in the commonly used
et al. (2020) reviewed current knowledge about the effect of digital bibliometric software. To ensure the quality of data, only peer-
nuclear power plant (NPP) control rooms on human reliability and reviewed journal articles are considered in this study, excluding
system safety. Tao et al. (2020) carried out a bibliometric analysis textbooks, doctoral dissertations, letters, and conference proceed-
of the human reliability researches published between 1984 and ings. To ensure the comprehensiveness of the search results, the
2018. Alvarenga and Frutuoso e Melo (2019) reviewed the cogni- keywords of ‘‘human reliability analysis”, ‘‘human error probabil-
tive basis for HRA, and Pan et al. (2017) investigated the cognitive ity”, ‘‘human error analysis”, ‘‘probabilistic risk assessment”, ‘‘de-
models applied in HRA methods. Rasmussen et al. (2015) analyzed pendence assessment”, and ‘‘HRA data collection” are utilized for
different elements included in the task complexity as performance literature search in the ‘‘article titles, abstracts, and keywords”.
shaping factor (PSF) through a thematic analysis. (Mkrtchyan et al., In addition, the period of time limitation for the literature search
2015) surveyed the use of Bayesian belief network in HRA and is from 2009 to 2020, and the language of publication was limited
pointed out their different features. In Alvarenga et al. (2014), a to English. The search was conducted in May 2020, and a total of
critical evaluation of the methods for evaluating organizational 1945 documents were initially obtained from the WOS database.
factors in HRA was made. In French et al. (2011), a critical review After deleting duplicate records, 1742 different records were
of the HRA models was performed for managers to manage com- remained for further analysis. This study only focuses the
plex systems safely. For comparing and evaluating HRA methods, researches which applied HRA for reliability analysis and planning.
Boring et al. (2010) reviewed the past benchmarking studies in Finally, 271 papers were considered within the scope of this
the areas of psychology and risk assessment. Besides, a review of research after title, abstract and full-text screening. The search pro-
advances in the HRA of post-initiator errors of commission (EOCs), cess and the steps to obtain the final dataset is shown in Fig. 1.
consisting of EOC identification and quantification, has been car- In order to realize the quantitative analysis and visualization of
ried out and presented in Reer (2008a), Reer (2008b). the retrieved literature, CiteSpace was used in this study to per-
Although the above review studies provide valuable insights form the bibliometric analysis. The software is a visualization tool
and research directions on HRA, there is a lack of a comprehensive to detect and visualize the distribution features of a certain field
bibliometric analysis of the literature on this topic. Compared to (Chen, 2006), and has been widely used in many areas to identify
standard literature reviews, the bibliometric analysis is an effective emerging developments regarding trends and dynamics in scien-
way to obtain high-level insights in the characteristics of a large tific literature (Cong and Shi, 2019; He et al., 2020; Jin et al., 2017).

2
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al.
Fig. 1. Retrieval process of this study.
3. Results and discussions
3.1. Publication trend in the HRA field
Fig. 2 illustrates the frequency distribution of the selected 271
articles in line with the publication year. It can be seen that the
number of annually published HRA researches has overall steadily
increased since 2009. Moreover, the researches on HRA can be
roughly divided into three stages. The starting period from 2009
to 2012 was quite stable, with the annual number of publications
Fig. 2. Distribution of papers by publication year.
3
Annals of Nuclear Energy 151 (2021) 107969
being basically the same (on average 12 articles per year). In
2013, the number of publications increased to 20 articles, and for
the following years the publication maintained an average of 20
articles till 2016. The last period showed the greatest increase,
starting from 2017 with 31 articles published, and the number of
publications reached a peak of 49 in 2019. Note that only five
months are considered in this study, and thus the publication
quantity is 29 in 2020. With the rapid development of technologies
and devices in systems, the contribution of human factors to sys-
tem failures is becoming more salient. Therefore, it is predicted
that the field of HRA will attract growing attention from research
communities and the number of articles will continue to increase
in the next decade.
3.2. Cooperation network analysis in the HRA field
In this section, the cooperation networks of authors, institu-
tions, and countries are analyzed to obtain observed features of
evolution in the HRA field and find productive authors, influential
institutions and leading countries.
3.2.1. Author cooperation network analysis
Fig. 3 presents the author cooperation network of the HRA pub-
lications, where the thicknesses of links reflect the cooperation
intensity of two authors and the color reflects the year that they
were firstly co-authored. As can be seen, the authors who pub-
lished at least three papers on the topic can be segmented into
eight major research groups. For example, Vinh N Dang has
authored with Huafei Liao, Andreas Bye, John Forester, Erasmia Lois
and Yung Hsien Chang for three times in 2009 in the collaborative
network #1. For the collaborative network #3, Jinkyun Park has
authored with Wondea Jung for 13 times since 2009, with Yochan
Kim for 13 times since 2012, and with Jaewhan Kim for seven times
since 2009. In addition, we can find that some authors in the net-
work are not connected to each other and the communication
among the formed research groups is weak.
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al.
Fig. 3. Author cooperation network in the HRA area.
Next, a detailed analysis on the representative research groups
in Fig. 3 is presented. The group #1 has published five research
articles from 2009 to 2020. Kyriakidis et al. (2018) presented a gen-
eric framework for developing a standardized list of PSFs that could
be used across sectors to describe the immediate and latent factors
that affect human performance. In Liao et al. (2019a), Liao et al.
(2019b), Liao et al. (2019c), the international HRA empirical study
and the US HRA empirical study were analyzed to develop an
empirically-based understanding of the performance, strengths,
and weaknesses of HRA methods by comparing HRA method pre-
dictions against actual operator performance in simulated accident
scenarios on NPP simulators. Pandya et al. (2020) developed a
quantification framework of HRA method based on expert judg-
ment aggregation and applied it to the radiotherapy domain.
The group #3 has published 50 articles since 2009, and Jinkyun
Park and Wondea Jung contributed the most. In a nutshell, there
existed three research teams in this group, which were chaired
by Jinkyun Park, Jooyoung Park, and Katrina M Groth, respectively.
The first research team published 34 articles and they paid more
attention to PSF (i.e., performance influencing factor) analysis,
human error probability (HEP) estimation, HRA data collection,
and HRA method. First, Kim et al. (2016) analyzed and identified
the main drivers affecting the performance of human operators
during-low power and shutdown operation in NPPs. Kim et al.
(2017a) adopted a profiling technique to determine the weights
of PSFs when performing HRA during low power and shutdown
operations. Kim et al. (2015) proposed a statistical methodology
using a logistic regression and stepwise variable selection to esti-
mate the effects of PSFs on HEPs of soft controls. Kim and Park
(2019) proposed an analysis based on the Bayesian logistic regres-
sion to estimate the multipliers of PSFs on HEPs. Kim et al. (2017c)
proposed a practical framework to quantitatively measure the
levels of socio-psychological PSFs using human error data. Second,
Lee et al. (2013) suggested an evaluation framework for quantify-
ing the HEPs during soft control operations. Cho et al. (2020) sug-
gested a method for the estimation of HEPs using Monte Carlo
thermal–hydraulic simulation. Jang et al. (2013) investigated the
basic HEPs for advanced main control room (MCR) operation using
4
Annals of Nuclear Energy 151 (2021) 107969
soft control. Park et al. (2016b) proposed a framework to estimate
task opportunities in terms of off-normal tasks from the opera-
tional experience of NPPs. Based on this framework, Park et al.
(2018) further calculated the nominal HEPs from investigation
reports that reflect the operation experience of NPPs. Third, Ham
and Park (2020) applied a big data analysis technique to collect
HRA data from event investigation reports based on the Safety-II
concept. Jung et al. (2009) summarized the process and results of
human event repository based on the simulator data on a steam
generator tube rupture (SGTR) scenario. Jung et al. (2020) reported
a framework, named as human reliability data extraction (HuREX),
for the HRA data collection in NPPs. Kim et al. (2017b) presented a
scheme to classify the erroneous behaviors identified by the
HuREX framework for HEP estimations. In addition, Lee et al.
(2011) proposed a CREAM-based communication error analysis
method to analyze the communication errors in NPPs. Jang et al.
(2016) suggested an HRA framework to evaluate soft control exe-
cution error in advanced MCRs. Park et al. (2016a) used a process
mining technique to characterize the work process of MCR crews.
Park et al. (2015) put forward a conceptual framework to deter-
mine the response time data of human operators working in the
MCR of NPPs under a seismic event.
The second research team published five papers, which mainly
investigated the factors impacting HRA. Park et al. (2017) studied
the relationships between PSFs and operator performances in the
digital MCR by using an NPP simulator. Park et al. (2019a) per-
formed a comparison of the quantification aspects of four
typically-used HRA methods in NPPs. Park et al. (2019b) developed
an approach for the analysis of the human and organizational fac-
tors in a multi-unit HRA based on SPAR-H, and Arigi et al. (2019)
identified the human and organizational factors for the multi-
unit probabilistic risk assessment of NPPs based on literature
review. Park et al. (2020) investigated the inter-relationships
between PSFs for HRA in NPPs based on correlation and factor anal-
yses and treated them by a context-based approach.
The third research team published nine articles, which manly
focused on improving HRA with Bayesian networks or developing
advanced HRA methods. For example, Groth and Mosleh (2012)
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al. Annals of Nuclear Energy 151 (2021) 107969

suggested a methodology to develop a data-informed Bayesian
network of PSFs using multiple sources of HRA data. Groth and
Swiler (2013) built a Bayesian network version of the SPAR-H
method for bridging the gap between HRA research and HRA prac-
tice. Groth et al. (2014) provided a Bayesian methodology to
enhance the scientific basis of HRA by incorporate simulator data
into the estimation of HEPs, and Zwirglmaier et al. (2017) proposed
an approach to enhance the scientific basis of HRA by capturing
causal paths with Bayesian network models. In addition, Pandya
et al. (2018) presented a methodology to develop the generic task
type-PSF structure for the external beam radiotherapy domain.
Groth et al. (2019) defined a hybrid algorithm for developing cog-
nitively based third generation HRA methods using multisource
data, causal models, and cognitive science. Podofillini et al.
(2010) proposed a method based on fuzzy expert system for the
dependence assessment in HRA that can capture the rules used
by experts to assess dependence levels.
Table 1 lists the most productive authors in the HRA field based
on the total number of publications. In this table, the authors are
divided into three categories according to the number of publica-
tions, i.e., ‘‘A (20 or more publications)”, ‘‘B (10 or more publica-
tions and less than 20), and ‘‘C (5 or more publications and less
than 10)”. The most prolific author is Jinkyun Park, who published
24 articles in the HRA field, followed by Wondea Jun and Yochan-
Kim with 17 and 10 publications, respectively. They are the most
important scholars who promote communication between schol-
ars in the HRA field. In addition, eight of the 14 productive authors
come from South Korea; others come from Turkey, USA, Switzer-
land, and Canada, respectively.
3.2.2. Institution cooperation network analysis
The institution cooperation network of the retrieved HRA pub-
lications is shown in Fig. 4, where the size of nodes reflects the
number of articles, the connection between nodes reflects cooper-
ation relationship, and the thickness of connections reflects the
cooperation strength. The threshold for the number of publications
was set at three, with a total of 31 institutions meeting the thresh-
old. As shown in the figure, Korea Atom Energy Research Institute
is the centrality with the most collaborative publications; Paul
Scherrer Institute, Korea Advanced Institute of Science Technology,
and Chosun University have cooperated with it. Furthermore, the
Paul Scherrer Institute from Switzerland has cross-regional cooper-
ation with Polytechnic University of Milan, Sandia National Labora-
tory, United States Nuclear Regulatory Commission, Sandia
National Laboratory, Idaho National Laboratory, and Nanyang
Technological University. Korea Atom Energy Research Institute,
Paul Scherrer Institute, and Korea Advanced Institute of Science
Technology are three institutions with the largest number of
publications.
Table 2 summarizes the most prolific institutions based on the
number of published papers on HRA. These institutions con-
tributed 143 articles accounting for 52.8% of the total publications.
Further, the institutions are divided into three categories according
to the number of publications, i.e., ‘‘A (20 or more publications)”,

Table 1
Top 14 productive authors on the topic of HRA.

Ranking Publication Author Institution Country

A 24 Jinkyun Park Korea Atomic Energy Research Institute South Korea
B 17 Wondea Jung Korea Atomic Energy Research Institute South Korea
B 10 Yochan Kim Korea Atomic Energy Research Institute South Korea
C 9 Jaewhan Kim Korea Atomic Energy Research Institute South Korea
C 8 Emre Akyuz Istanbul Technical University Turkey
C 8 Poong Hyun Seong Korea Advanced Institute of Science Technology South Korea
C 8 Ali Mosleh University of California USA
C 7 Vinh N Dang Paul Scherrer Institute Switzerland
C 6 Faisal Khan Memorial University of Newfoundland Canada
C 5 Ar Ryum Kim Korea Advanced Institute of Science Technology South Korea
C 5 Inseok Jang Korea Atomic Energy Research Institute South Korea
C 5 Luca Podofillini Paul Scherrer Institute Switzerland
C 5 Seung Jun Lee Ulsan National Institute of Science and Technology South Korea
C 5 Sankaran Mahadevan Vanderbilt University USA

Fig. 4. Institution cooperation network in the HRA field.

5
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al. Annals of Nuclear Energy 151 (2021) 107969

Table 2
Top 16 productive institutions of the HRA field.

Ranking Institution Publication Country

A Korea Atom Energy Research Institute 39 Korea
B Paul Scherrer Institute 14 Switzerland
B Korea Advanced Institute of Science 12 Korea
Technology
C Memorial University of 8 Canada
Newfoundland
C Sandia National Laboratory 8 USA
C University of Maryland 7 USA
C Idaho National Laboratory 6 USA
C Istanbul Technical University 6 Turkey
C Vanderbilt University 6 USA
C Polytechnic University of Milan 6 Italy
C Chosun University 6 Korea
C Beihang University 5 China
C Southwest University 5 China
C Norwegian University of Science and 5 Norway
Technology
C The Ohio State University 5 USA
C Bursa Technical University 5 Turkey

‘‘B (10 or more publications and less than 20), and ‘‘C (5 or more
publications and less than 10)”. It can be observed that the Korea
Atom Energy Research Institute (39 articles) is the most productive
institution, followed by Paul Scherrer Institute (14 articles), and
Korea Advanced Institute of Science Technology (12 articles).
Among the 16 institutions, three are from Korea with a total of
57 articles, five from USA with 32 papers, and two from Turkey
with a total of 11 articles. Although only one institution is from
Switzerland, the Paul Scherrer Institute has the second largest
number of publications. Besides, 11 institutions are universities
in the 16 productive institutions. Hence, universities are of key
importance to drive and direct HRA researches.
3.2.3. Country cooperation network analysis
Fig. 5 shows the country cooperation network in the HRA field,
where the size of nodes represents the amount of published arti-
cles, the line between them reflects cooperation relationship, and
the thickness of the links reflects the level of cooperation between
countries. In this figure, the countries published more than 10 arti-
cles on the topic are displayed. As can be seen, there are strong col-
laborations between South Korea and Switzerland, Switzerland
and USA, England and Brazil. That is, these countries played a par-
ticularly important role in linking cross-country cooperation in the
HRA field. Furthermore, China with 60 articles is ranked as the first
Fig. 6. Distribution of articles with respect to continents.
in terms of the number of publications, and it is followed by South
Korea (49 articles) and USA (48 articles). This shows that these
three countries are the main research forces of the HRA field.
Fig. 6 further presents the distribution of publications on the
basis of continents. As it is seen from the figure, more than 77%
of the included publications are derived from Europe and Asia,
and Asia with 158 articles is ranked as the first based on the num-
ber of articles.
3.3. Co-citation analysis in the HRA field
Co-citation refers to the presence of two or more journals,
authors, and documents in the list of references for the third doc-
ument (Osareh, 1996). Co-citation networks can be analyzed to
explore the structure of science in certain aspects, including jour-
nals, authors, and documents.
3.3.1. Journal co-citation analysis
Fig. 7 shows the journal co-citation network in the HRA field,
where the size of nodes indicates the number of journal citations,
and the line between them indicates the co-citation frequency
between journals. As can be observed, the most cited journals are
Reliability Engineering & System Safety (238 citations), Safety
Science (149 citations), and Journal of Loss Prevention in the Pro-
cess Industries (102 citations). All the 10 most significant journals
on HRA are listed in Table 3. In terms of the link number, Journal of
Loss Prevention in the Process Industries (17 links) has the most
citation with others in the HRA area, followed by Annals of Nuclear
Energy (15 links), Reliability Engineering & System Safety (14
links), and Safety Science (13 links).

3.3.2. Author co-citation analysis

Fig. 8 shows the author co-citation network in the HRA field,
where the size of nodes reflects the quantity of author’s citations,
the link between them reflects co-citation relationship between
authors, the color reflects the year that the authors were firstly
co-cited, and the thicknesses of different links stand for the times
of two authors being co-cited. As shown in Fig. 8, Alan D Swain
from the Sandia National Laboratory is the most highly cited
author, who recievd 145 citations and focused on the research of
Fig. 5. Country cooperation network in the HRA field. PSFs (Swain, 1982). Erik Hollnagel from the Linköping University
6
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al. Annals of Nuclear Energy 151 (2021) 107969

Fig. 7. Journal co-citation network in the HRA field.

Table 3
The 10 most significant journals on HRA.

Ranking Journal Citation Impact factor Publication

1 Reliability Engineering & System Safety 238 5.040 75
2 Safety Science 149 4.105 42
3 Journal of Loss Prevention in the Process Industries 102 2.795 9
4 Annals of Nuclear Energy 94 1.378 29
5 Applied Ergonomics 75 3.145 1
6 Risk Analysis 75 3.137 6
7 Nuclear Engineering and Technology 52 1.846 7
8 Human Factors 51 3.165 2
9 Nuclear Engineering and Design 49 1.620 6
10 Fuzzy Sets and Systems 37 3.305 2

Fig. 8. Author co-citation network in the HRA field.

7
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al. Annals of Nuclear Energy 151 (2021) 107969

Table 4 cussed the scenario authoring, characterization, and debriefing

Top 10 cited authors in the HRA field. application (SACADA) data taxonomy, the theoretical foundation,
Ranking Author Citation Institution the prospective data to be generated from the SACADA raw data
1 Alan D Swain 145 Sandia National Laboratory to inform human reliability and human performance. Yang et al.
2 Erik Hollnagel 136 Linköping University (2013) proposed a modified CREAM to facilitate human reliability
3 Barry A Kirwan 114 Eurocontrol Experimental quantification in marine engineering by incorporating fuzzy evi-
Centre dential reasoning and Bayesian inference logic. Groth and Swiler
4 David I Gertman 83 Idaho National Laboratory
5 James Reason 63 University of Manchester
(2013) bridged the gap between HRA research and HRA practice
6 Janet C Williams 62 Det Norske Veritas by building a Bayesian network version of the SPAR-H method.
7 Katrina M Groth 57 University of Maryland Furthermore, Fig. 10 is the timeline view of document clusters,
8 Ronald Laurids 57 Idaho National Laboratory which shows that 440 references of the selected documents can be
Boring
clustered into eight clusters in the HRA area. The horizontal line
9 Jooyoung Park 55 Chosun University
10 Bruce P Hallbert 48 Idaho National Laboratory represents changes over time of the research hotspots, and the size
of the circle refers to the hot degree in a certain period.
As we can see, three clusters are formed by the researches
around 2007, i.e., #2 expert elicitation, #5 performance influencing
and Barry A Kirwan from the Eurocontrol Experimental Centre
factors, and #6 organizational factors. Each of the three clusters
come in second and third positions, and they received 136 and
shows only few citation tree-rings, which implies that they have
114 citations, respectively. Erik Hollnagel focused on the reliability
not received much attention in the literature. As far as the cluster
of cognition (Hollnagel, 1991, 1996), and Barry A Kirwan special-
#2, a method was developed by Čepin (2008) to consider the
ized in the validation of the HRA techniques THERP, HEART and
dependencies between human actions, which integrates the fea-
JHEDI (Kirwan, 1996, 1997; Kirwan et al., 1997). The 10 most cited
tures of existing methods and the experience from a full scope
authors also include David I Gertman, James Reason, and Janet C
plant simulator. Marseguerra et al. (2007) proposed a systematic
Williams (cf. Table 4). In terms of the link number, Erik Hollnagel
procedure for computing probabilities of operator action failure
(20 links) has co-citated with others most in the HRA area, fol-
in the CREAM. For the cluster #5, scholars discussed how to collect
lowed by Katrina M (19 links) and Groth David I Gertman (18
human performance data under the influence of various PSFs. For
links).
example, each individual operator’s behaviors were simulated
through a cognitive model under the influence of explicitly mod-
3.3.3. Document co-citation analysis
eled PSFs (Chang and Mosleh, 2007b). The information, decision,
Fig. 9 displays the document co-citation network of the HRA
and action in crew context (IDAC) model was described by Chang
researches. Every node represents a publication and the size of
and Mosleh (2007c) for HRA. Regarding the cluster #6, Chang
nodes denotes the number of citations a reference has been
and Mosleh (2007b) identified the IDAC set of PSFs, providing their
received. The links between nodes represent co-citation relation
definitions and causal organization in the form of a modular influ-
between two documents, and the color reflects the year that the
ence diagram. An integrated decision support system to aid opera-
documents were firstly co-cited. The documents with at least 10
tor cognitive processes was proposed by Lee and Seong (2007) for
co-citations are visible in the figure. It can be found that the top
advanced MCRs of future NPPs. Chang and Mosleh (2007a) devel-
three co-cited articles were published by Chang et al. (2014),
oped a model to probabilistically predict the responses of the con-
Yang et al. (2013), and Groth and Swiler (2013), which received
trol room operating crew in NPPs during an accident.
26, 23, and 21 co-citations, respectively. Chang et al. (2014) dis-

Fig. 9. Document co-citation network in the HRA field.

8
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al.
Fig. 10. Timeline view of document clusters in the HRA field.
Between 2010 and 2015, four areas have attracted a lot of atten-
tion from researchers, which include #0 decision-making, #1
human performance data, #3 dependence assessment, and #4
Dempster-Shafer evidence theory. First, the cluster #0 began to
appear in 2007 and had drawn plenty of interest in 2013. Many
researches attempted to use decision-making methods for HRA in
this area. Yang et al. (2013) proposed a modified CREAM to facili-
tate human reliability quantification in the marine engineering
by incorporating fuzzy evidential reasoning and Bayesian inference
logic. Akyuz and Celik (2015b) provided a methodological exten-
sion through the integration of analytic hierarchy process (AHP)
technique into HEART approach to systematically predict human
error for designated tasks and ascertain the desired safety control
level in crude oil tanker ships. Castiglia and Giardina (2013) mod-
ified the HEART method based on fuzzy set concept to evaluate the
probability of erroneous actions. Akyuz and Celik, 2016 proposed a
human reliability assessment approach by combining interval
type-2 fuzzy sets with HEART to enhance safety levels in the mar-
ine and off-shore industries.
Second, it is noteworthy that the cluster #1 had drawn plenty of
interest in 2012. Reliable human performance data is an essential
factor affecting HRA quality. Hence, many researchers have devel-
oped technical underpinnings that specify what and how HRA data
can be collected. In Ali (2006), simulators were used to collect the
competency data for quality training and assessment. A method
was introduced by Groth and Mosleh (2012) for integrating multi-
ple sources of empirical data and validated theoretical models to
improve both qualitative and quantitative HRA applications.
Groth and Swiler (2013) built a Bayesian network version of the
SPAR-H method and demonstrated how it can be used to incorpo-
rate data and information from research to advance HRA practice.
Chang et al. (2014) collected operator performance information by
the SACADA tool for its implementation in the NPPs’ operator
training program. A data collection approach was presented by
Musharraf et al. (2014) in order to create a virtual environment
to simplify Bayesian network model of offshore emergency
evacuation.
Third, it is noteworthy that the cluster #3 had received more
and attention since 2009. Dependence analysis is an evaluation
of the impact of an operator’s failure to perform a task on the fail-
ure probabilities of subsequent tasks. It is an HRA task strongly
based on analysts’ judgments. Su et al. (2015) put forward a com-
putational model based on Dempster-Shafer evidence theory and
9
Annals of Nuclear Energy 151 (2021) 107969
AHP method for dependence assessment among human errors.
Guo et al. (2017) adopted evidence credibility decay model to
assess the dependence between tasks in HRA. Zheng and Deng
(2018) used an evidential network approach extended by belief
rules and uncertainty measures to handle dependence in HRA.
Podofillini et al. (2010) presented a methodology for dependence
assessment in HRA that captures the rules used by experts to
assess dependence levels and incorporates this knowledge into a
software tool. Liu et al. (2018) developed a large group dependence
assessment model based on interval 2-tuple linguistic variables
and cluster analysis method to manage the dependence in HRA.
Deng and Jiang, 2018 presented a framework for dependence
assessment on the basis of belief rules and uncertainty measures.
Fourth, the cluster #4 had drawn plenty of interest from 2009 to
2017. Many computational methods based on Dempster-Shafer
evidence theory have been proposed to handle the dependence
in HRA. For example, Chen et al. (2017) proposed an evidential
AHP-based dependence assessment methodology to handle the
dependence in HRA. A computational model was proposed by Su
et al. (2015) by using Dempster-Shafer evidence theory and AHP
method to handle the dependence in HRA. Guo et al. (2017) sug-
gested a computational model based on Dempster-Shafer evidence
theory and evidence credibility decay model to assess the depen-
dence between tasks in HRA.
From 2016 to now, except for the clusters (#0, #1, and #3) that
are constantly being concerned, another one major cluster is #7
ratio magnitude estimation. These four areas may be potential
research topics in the near future. With regard to the cluster #7,
HRA relies on expert judgments to generate reliability data. As an
expert judgment technique, ratio magnitude estimation can be
used to verify the reliability data obtained from experts. For exam-
ple, two expert judgment techniques (absolute probability judg-
ment and ratio magnitude estimation) were applied by Liu et al.
(2020) to update the PSF multiplier design in the SPAR-H method.
Kim et al. (2018) collected the HRA data including human reliabil-
ity data and contextual data, and estimated the quantitative rela-
tions between PSFs and HEPs for the purpose of generating
empirical evidence.
3.4. Keyword co-occurrence analysis in the HRA field
Fig. 11 shows the keyword co-occurrence network analysis of
HRA articles. From this figure, we can see that ‘‘CREAM”, ‘‘quantifi-
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al. Annals of Nuclear Energy 151 (2021) 107969

Fig. 11. Keyword co-occurrence analysis of the HRA field.

cation”, ‘‘performance”, ‘‘probability”, ‘‘Risk”, ‘‘HEART”, ‘‘Bayesian
network”, ‘‘accident”, ‘‘performance shaping factor”, and ‘‘risk
assessment” are the keywords taking over notable positions. More-
over, ‘‘CREAM”, ‘‘HEART”, and ‘‘THERP” are HRA methods; ‘‘Baye-
sian network”, and ‘‘Bayesian belief network” are the techniques
employed to expand the scientific basis and traceability of HRA.
The ‘‘CREAM”, ‘‘quantification” and ‘‘performance” can be consid-
ered as three important research hotspots. The classifications with
these research hotspots and the related articles are presented in
Table 5.
For the CREAM method, we here further analyze its application
fields based on the related 30 articles. Fig. 12 provides the applica-
tion field distribution of the CREAM method. As we can see, the
CREAM method has been mainly utilized in the energy and chem-
ical industry, the maritime industry, and the transportation
industry.
4. Suggestions for future research
HRA is significantly important for enhancing the safety and reli-
ability of different complex systems, including design, manufactur-
ing, operation, and maintenance. Although breakthroughs have
been made in the research of HRA, many problems have still not
been fully solved, and some new problems are needed to be
addressed. Based on the literature survey of HRA articles, we sum-
marize the following possible directions for future research:
(1) The fundamental issue in reference to HRA is the lack of
objective data and lower level information of human perfor-
mance that can be collected from a real event. First, in future
research, it is suggested to use more or new system monitor-
ing equipment, such as sensors and cameras, to collect mul-
tiple source HRA data in a more efficient and effective way.

Table 5
Classifications with research hotspots.

Research Classification Article

hotspots
CREAM Applications of CREAM
Modification of CREAM
Quantification Quantification of HEPs
Quantification of human factors
Quantification of HRA methods
Quantification of organizational
influences
Weighting determination of PSFs
Weighting determination of error-
producing conditions
Performance Human performance
Classification of PSFs
Analysis of PSFs
Common performance condition
(Akyuz and Celik, 2015; Chen et al., 2019; Kirytopoulos et al., 2014; Kumar et al., 2017; Martins and Maturana,
2010; Pellegrino, 2011; Taga et al., 2012; Wang et al., 2011; Zhou et al., 2017, 2018)
(Akyuz and Celik, 2015; Chen et al., 2018, 2019; Guo et al., 2019; Lee et al., 2011; Monferini et al., 2013; Nan
and Sansavini, 2016; Pan et al., 2019; Petrillo et al., 2017; Sun et al., 2012, 2020, Ung, 2015, 2019; Wu et al.,
2017; Xi et al., 2017; Yang et al., 2013; Zhang et al., 2019; Zhang and Tan, 2018; Zhou et al., 2017; Zio et al.,
2009)
(Akyuz et al., 2018; Chen et al., 2019b, 2018; Guo and Sun, 2020; Pan et al., 2019; Ramezani et al., 2020; Steijn
et al., 2020; Sun et al., 2009; Taylor et al., 2020; Wang et al., 2011; Wu et al., 2017; Xi et al., 2017)
(Maturana and Martins, 2019; Myers, 2013; Noroozi et al., 2013)
(Pandya et al., 2020; Park et al., 2019a)
(Li et al., 2012; Pence et al., 2019)
(Kim et al., 2017a)
(Akyuz et al., 2016)
(Farcasiu and Prisecaru, 2014; Islam et al., 2018; Liao et al., b, c, 2019a; Nan and Sansavini, 2016)
(El-Ladan and Turan, 2012; Franciosi et al., 2019; Kyriakidis et al., 2018; Li et al., 2018a, 2018b; Rasmussen and
Laumann, 2020)
(Chen et al., 2018; Griffith and Mahadevan, 2015; Groth and Mosleh, 2012; Kim et al., 2017; Kumar et al., 2015;
Liu et al., 2017, 2020; Pandya et al., 2018; Park, 2011, 2014; Shirley et al., 2020)
(Wu et al., 2017; Zhou et al., 2017)

10
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al.
Fig. 12. Application field distribution of the CREAM method.
Second, more advanced (big data or small data) analytical
techniques and digital platforms should be developed in
the future to analyze the massive amount but poor quality
HRA data in a real-time way (Ham and Park, 2020; Jung
et al., 2020). Also, systematic experts’ judgments (expert
elicitation or expert estimation) can be used to generate
HRA data if collecting positivistic empirical data is challenge
(Laumann and Skogstad, 2020).
(2) Many tasks in the HRA (e.g., assigning values for PSFs and
assessing HEPs) heavily relies on expert judgments to gener-
ate reliability data. In many situations, the opinions of
experts are often vague, uncertain, or even incomplete,
because of time pressure, lack of knowledge, and their lim-
ited expertise related to the problem domain (Liu et al.,
2018; Yang et al., 2019). Therefore, it is promising to use var-
ious uncertainty theories, such as Pythagorean fuzzy sets
(Yager and Abbasov, 2013), probabilistic linguistic term sets
(Gou and Xu, 2016), and cloud model theory (Li et al., 2009),
to effectively manipulate the uncertainties from expert
judgements, and put forward a traceable and defendable
method to characterize experts’ experience and knowledge
for HRA. Further, experts may have different experience
and backgrounds, and thus conflict opinions are unavoidable
in the HRA process. Hence, as another direction for future
studies, consensus methods (Ureña et al., 2019; Zhang
et al., 2020) are suggested to be adopted to improve group
consistency and lead to efficiency improvement in HRA.
(3) PSFs are the factors that may affect human performance in
HRA. Currently, a lot of PSFs have been proposed to highlight
human error contributors and to adjust basic HEPs. How-
ever, there is not a standard set of PSFs used among HRA
methods, nor is there a framework available to compare
the PSFs used in various methods. Thus, future research
should develop a qualitative evaluation framework to collect
and analyze the frequently used PSFs in the literature. More-
over, practitioners constantly suffered a problem for select-
ing proper PISs for a specific HRA problem among a
diversified set of factors (Liu et al., 2017). Therefore, it is very
necessary to introduce a method to help determine a limited
number of key PSFs considering their inter-relationships and
the dependency between the states of them in the future.
11
Annals of Nuclear Energy 151 (2021) 107969
(4) Since the introduction of the first HRA method, a number of
research achievements have been made in qualitative and
quantitative HRA models. But the suitability of existing
HRA methods in the digital environment is often questioned
(Li et al., 2018b; Shirley et al., 2020). On the other hand, the
traditional HRA methods have many disadvantages for the
practical HRA (Abrishami et al., 2020; Ramos et al., 2020).
As the development of various computer and control tech-
nologies, the control of many high-risk systems has been
gradually digitized. As a result, new human reliability prob-
lems may emerge because of a series of new changes in the
cognitive processes, behavioral patterns, and error mecha-
nisms of operators. To address this situation, future research
should be directed towards developing advanced HRA mod-
els to conduct HRA of operators in the digital system but also
avoids disadvantages of the classic methods.
(5) The literature review shows that some multi-criteria
decision-making (MCDM) methods, which include AHP
(Petruni et al., 2019), analytic network process (ANP) (De
Ambroggi and Trucco, 2011), and multi-objective optimiza-
tion by ratio analysis plus full multiplicative form (MULTI-
MOOR) (Yazdi, 2020), have been used for HRA technique
selection, human error assessment, and intervention action
ranking. In the future, more researches should be conducted
to apply other new MCDM methods such as the best-worst
method (Rezaei, 2015), the alternative queuing method
(Gou et al., 2016), the multi-attributive border approxima-
tion area comparison (MABAC) (Pamučar et al., 2018), and
the measurement of alternatives and ranking according to
compromise solution (MARCOS) (Stević et al., 2020) to solve
the decision-making problems in HRA.
(6) Different HRA methods have different advantages and draw-
backs. The existing studies did not make sufficient compar-
isons among various HRA methods. As a result, it is
difficult for the practitioners to select a suitable HRA model
in real applications. Therefore, in future research, it is neces-
sary to conduct a detailed comparative study to adequately
evaluate and compare the advantages and disadvantages of
different HRA methods to aid practitioners and scholars in
selecting the most suitable one for a given application. Also,
to combine the superiorities of different HRA approaches,
future research is suggested to propose an integrated frame-
work by using multiple HRA methods.
(7) The emerging technologies have large room for improve-
ment regarding the field of HRA. So, another possible direc-
tion for future work would be to employ techniques such as
artificial intelligence tools to address the HRA problems. For
instance, deep learning algorithms (Lecun et al., 2015) can
be employed to learn the dependence levels of human fail-
ure events from the evaluation data of experts on each pair
of sequent tasks. The methods for the estimation of HEPs can
be empowered by neural networks (Liu et al., 2017) to con-
sider fluctuations in the assessment of human errors in the
way changes occur in human brain. Moreover, the develop-
ment of computer aided tools to execute the HRA algorithms
given in the literature is certainly an important direction in
order that HRA can be easily realized by practitioners to
manage large-scale complex systems efficiently.
To balance, HRA technology needs to be improved as a whole to
handle the dynamic and changeable application environment and
the complex system monitoring issues. Considering the advantages
and potential application of HRA in safety–critical domains, we
believe the number of articles on HRA will grow rapidly in the
coming years to assess human contribution to systemic risk and
improve the reliability management level of complex systems.
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al.
5. Conclusions
The HRA, as a way to analyze and reduce human errors of com-
plex systems and processes, has attracted extensive attention of
academics and practitioners. Correspondingly, a great number of
researches have been conducted in the area of HRA during the past
decades. In this article, we provided a holistic review of the HRA
papers published between 2009 and 2020 using statistical and bib-
liometric analysis methods. A total of 271 journal articles were
retrieved from the WoS database and used in the bibliometric anal-
ysis. The article publication trend indicated the researches on HRA
steadily increased from 2009 to 2020. Via the cooperation network
analysis, we find that Jinkyun Park, Wondea Jun, and YochanKim
from the Korea Atomic Energy Research Institute are the most pro-
lific authors. The Korea Atom Energy Research Institute is the cen-
trality with the most collaborative publications, followed by the
Paul Scherrer Institute from Switzerland, and the Korea Advanced
Institute of Science Technology. Furthermore, South Korea, USA
and China contribute to a great many HRA publications and play
pivotal roles to connect the cross-country cooperation in the field.
The co-citation network analysis showed that Reliability Engi-
neering & System Safety, Safety Science, and Journal of Loss
Prevention in the Process Industries are the most significant jour-
nals on HRA. According to the number of citations, Alan D Swain
from the Sandia National Laboratory, Erik Hollnagel from the Mac-
quarie University and Barry A Kirwan from the Eurocontrol Exper-
imental Centre are the highly cited authors in the HRA area. The
top three co-cited articles are those published by Chang et al.
(2014), Yang et al. (2013), and Groth and Swiler (2013). The docu-
ment cluster analysis displayed that ‘‘decision making”, ‘‘humane
performance data”, ‘‘dependence assessment”, and ‘‘ratio magni-
tude estimation” are the main research topics in the near future.
The keyword co-occurrence analysis revealed that ‘‘CREAM”,
‘‘quantification” and ‘‘performance” are three important research
hotspots. The CREAM is a second generation HRA technique that
enables qualitative opinions from experts to be converted into
quantitative human failure analysis results. In the literature, many
papers focused on the application and modification of the CREAM
method. There is an increasing trend in applying the THERP,
SPAR-H and EPRI methods for HRA as well as providing HEPs to
PRA model. Besides, the mainstream researches have shifted to
quantify HEPs and address the impact of human performance on
system risk over the past decades. In summary, this article pro-
vides theoretical and practical guidance for researchers and practi-
tioners who are interested in the HRA field to capture the current
research hotspots and potential research directions.
Declaration of Competing Interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared
to influence the work reported in this paper.
Acknowledgments
The authors are very grateful to the editor and reviewers for
their insightful and constructive comments and suggestions, which
are very helpful in improving the quality of the paper. This work
was supported by the National Natural Science Foundation of
China (Nos. 61773250 and 71671125 ) and the Fundamental
Research Funds for the Central Universities.
References
Abílio Ramos, M., López Droguett, E., Mosleh, A., Moura, D.C., M, 2020. A human
reliability analysis methodology for oil refineries and petrochemical plants
12
Annals of Nuclear Energy 151 (2021) 107969
operation: Phoenix-PRO qualitative framework. Reliab. Eng. Syst. Saf. 193,
106672.
Abrishami, S., Khakzad, N., Hosseini, S.M., van Gelder, P., 2020. BN-SLIM: A Bayesian
Network methodology for human reliability assessment based on success
likelihood index method (SLIM). Reliab. Eng. Syst. Saf. 193, 106647.
Akyuz, E., Celik, M., 2015a. Application of CREAM human reliability model to cargo
loading process of LPG tankers. J. Loss Prev. Process Ind. 34, 39–48.
Akyuz, E., Celik, M., 2015b. A methodological extension to human reliability
analysis for cargo tank cleaning operation on board chemical tanker ships. Saf.
Sci. 75, 146–155.
Akyuz, E., Celik, M., Akgun, I., Cicek, K., 2018. Prediction of human error probabilities
in a critical marine engineering operation on-board chemical tanker ship: The
case of ship bunkering. Saf. Sci. 110, 102–109.
Akyuz, E., Celik, M., Cebi, S., 2016. A phase of comprehensive research to determine
marine-specific EPC values in human error assessment and reduction
technique. Saf. Sci. 87, 63–75.
Akyuz, E., Celik, E., 2016. A modified human reliability analysis for cargo operation
in single point mooring (SPM) off-shore units. Appl. Ocean Res. 58, 11–20.
Ali, A., 2006. Simulator instructor - STCW requirements and reality. Pomorstvo 20
(2), 23–32.
Alvarenga, M.A.B., Frutuoso e Melo, P.F, 2019. A review of the cognitive basis for
human reliability analysis. Prog. Nucl. Energy 117, 103050.
Alvarenga, M.A.B., Melo, Frutuoso E, P.F., Fonseca, R.A, 2014. A critical review of
methods and models for evaluating organizational factors in human reliability
analysis. Prog. Nucl. Energy 75, 25–41.
Arigi, A.M., Kim, G., Park, J., Kim, J., 2019. Human and organizational factors for
multi-unit probabilistic safety assessment: Identification and characterization
for the Korean case. Nuclear Engineering and Technology 51 (1), 104–115.
Boring, R.L., Hendrickson, S.M.L., Forester, J.A., Tran, T.Q., Lois, E., 2010. Issues in
benchmarking human reliability analysis methods: A literature review. Reliab.
Eng. Syst. Saf. 95 (6), 591–605.
Castiglia, F., Giardina, M., 2013. Analysis of operator human errors in hydrogen
refuelling stations: Comparison between human rate assessment techniques.
Int. J. Hydrogen Energy 38 (2), 1166–1176.
Čepin, M., 2008. DEPEND-HRA-A method for consideration of dependency in human
reliability analysis. Reliab. Eng. Syst. Saf. 93 (10), 1452–1460.
Chang, Y.H.J., Mosleh, A., 2007a. Cognitive modeling and dynamic probabilistic
simulation of operating crew response to complex system accidents - Part 1:
Overview of the IDAC Model. Reliab. Eng. Syst. Saf. 92, 997–1013.
Chang, Y.H.J., Mosleh, A., 2007b. Cognitive modeling and dynamic probabilistic
simulation of operating crew response to complex system accidents - Part 2:
IDAC performance influencing factors model. Reliab. Eng. Syst. Saf. 92 (8), 1014–
1040.
Chang, Y.H.J., Mosleh, A., 2007c. Cognitive modeling and dynamic probabilistic
simulation of operating crew response to complex system accidents - Part 5:
Dynamic probabilistic simulation of the IDAC model. Reliab. Eng. Syst. Saf. 92
(8), 1076–1101.
Chang, Y.J., Bley, D., Criscione, L., Kirwan, B., Mosleh, A., Madary, T., Nowell, R.,
Richards, R., Roth, E.M., Sieben, S., Zoulis, A., 2014. The SACADA database for
human reliability and human performance. Reliab. Eng. Syst. Saf. 125, 117–133.
Chen, C., 2006. CiteSpace II: Detecting and visualizing emerging trends and
transient patterns in scientific literature. J. Am. Soc. Inform. Sci. Technol. 57
(3), 359–377.
Chen, D.K., Fan, Y., Li, W.H., Wang, Y.H., Zhang, S., 2019a. Human reliability
prediction in deep-sea sampling process of the manned submersible. Saf. Sci.
112, 1–8.
Chen, D.K., Fan, Y., Ye, C., Zhang, S., 2019b. Human reliability analysis for manned
submersible diving process based on CREAM and Bayesian network. Qual.
Reliab. Eng. Int. 35 (7), 2261–2277.
Chen, J.Y., Zhou, D., Lyu, C., Zhu, X., 2018. A method of human reliability analysis and
quantification for space missions based on a Bayesian network and the
cognitive reliability and error analysis method. Qual. Reliab. Eng. Int. 34 (5),
912–927.
Chen, L.Y., Zhou, X.Y., Xiao, F.Y., Deng, Y., Mahadevan, S., 2017. Evidential analytic
hierarchy process dependence assessment methodology in human reliability
analysis. Nucl. Eng. Technol. 49 (1), 123–133.
Cho, J., Kim, Y., Kim, J., Park, J., Kim, D.S., 2020. Realistic estimation of human error
probability through Monte Carlo thermal-hydraulic simulation. Reliab. Eng.
Syst. Saf. 193, 13.
Cong, W., Shi, L., 2019. Heterogeneity of industrial development and evolution of
cleaner production: Bibliometric analysis based on JCLP. J. Cleaner Prod. 212,
822–836.
De Ambroggi, M., Trucco, P., 2011. Modelling and assessment of dependent
performance shaping factors through analytic network process. Reliab. Eng.
Syst. Saf. 96 (7), 849–860.
Deng, X.Y., Jiang, W., 2018. Dependence assessment in human reliability analysis
using an evidential network approach extended by belief rules and uncertainty
measures. Ann. Nucl. Energy 117, 183–193.
Ekanem, N.J., Mosleh, A., Shen, S.H., 2016. Phoenix – A model-based human
reliability analysis methodology: qualitative analysis procedure. Reliab. Eng.
Syst. Saf. 145, 301–315.
El-Ladan, S.B., Turan, O., 2012. Human reliability analysis - Taxonomy and praxes of
human entropy boundary conditions for marine and offshore applications.
Reliab. Eng. Syst. Saf. 98 (1), 43–54.
Farcasiu, M., Prisecaru, I., 2014. MMOSA - A new approach of the human and
organizational factor analysis in PSA. Reliab. Eng. Syst. Saf. 123, 91–98.
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al.
Franciosi, C., Di Pasquale, D., Iannone, R., Miranda, S., 2019. A taxonomy of
performance shaping factors for human reliability analysis in industrial
maintenance. J. Ind. Eng. Manage. 12 (1), 115–132.
French, S., Bedford, T., Pollard, S.J.T., Soane, E., 2011. Human reliability analysis: A
critique and review for managers. Saf. Sci. 49 (6), 753–763.
Gaviria-Marin, M., Merigó, J.M., Baier-Fuentes, H., 2019. Knowledge management: A
global examination based on bibliometric analysis. Technol. Forecast. Soc.
Change 140, 194–220.
Gertman, D., Blackman, H., Marble, J., Byers, J., Smith, C., 2005. The SPAR-H human
reliability analysis method. US Nuclear Regulatory Commission 230, 35.
Gil, M., Wróbel, K., Montewka, J., Goerlandt, F., 2020. A bibliometric analysis and
systematic review of shipboard decision support systems for accident
prevention. Saf. Sci. 128, 104717.
Gou, X., Xu, Z., 2016. Novel basic operational laws for linguistic terms, hesitant
fuzzy linguistic term sets and probabilistic linguistic term sets. Inf. Sci. 372,
407–427.
Gou, X., Xu, Z., Liao, H., 2016. Alternative queuing method for multiple criteria
decision making with hybrid fuzzy and ranking information. Inf. Sci. 357, 144–
160.
Griffith, C.D., Mahadevan, S., 2015. Human reliability under sleep deprivation:
Derivation of performance shaping factor multipliers from empirical data.
Reliab. Eng. Syst. Saf. 144, 23–34.
Groth, K.M., Mosleh, A., 2012. Deriving causal Bayesian networks from human
reliability analysis data: a methodology and example model. Proc. Inst. Mech.
Eng. Part O-J. Risk Reliab. 226, 361–379.
Groth, K.M., Smith, C.L., Swiler, L.P., 2014. A Bayesian method for using simulator
data to enhance human error probabilities assigned by existing HRA methods.
Reliab. Eng. Syst. Saf. 128, 32–40.
Groth, K.M., Smith, R., Moradi, R., 2019. A hybrid algorithm for developing third
generation HRA methods using simulator data, causal models, and cognitive
science. Reliab. Eng. Syst. Saf. 191, 106507.
Groth, K.M., Swiler, L.P., 2013. Bridging the gap between HRA research and HRA
practice: A Bayesian network version of SPAR-H. Reliab. Eng. Syst. Saf. 115, 33–
42.
Guo, X.F., Zhou, Y.H., Qian, J., Deng, Y., 2017. Using evidence credibility decay model
for dependence assessment in human reliability analysis. Ann. Nucl. Energy
100, 107–118.
Guo, Y.D., Sun, Y.C., 2020. Flight safety assessment based on an integrated human
reliability quantification approach. PLoS ONE 15, (4) 231391.
Guo, Y.D., Sun, Y.C., Yang, X.F., Wang, Z.P., 2019. Flight safety assessment based on a
modified human reliability quantification method. Int. J. Aerosp. Eng. 2019,
2812173.
Ham, D.H., Park, J., 2020. Use of a big data analysis technique for extracting HRA
data from event investigation reports based on the Safety-II concept. Reliab.
Eng. Syst. Saf. 194, 106232.
He, S., Zhu, D., Chen, Y., Liu, X., Chen, Y., Wang, X., 2020. Application and problems of
emergy evaluation: A systemic review based on bibliometric and content
analysis methods. Ecol. Ind. 114, 106304.
Hollnagel, E., 1991. Cognitive ergonomics and the reliability of cognition. Travail
Humain 54 (4), 305–321.
Hollnagel, E., 1996. Reliability analysis and operator modelling. Reliab. Eng. Syst.
Saf. 52, 327–337.
Hollnagel, E., 1998. Cognitive reliability and error analysis method. Elsevier Science,
Oxford, UK.
Islam, R., Khan, F., Abbassi, R., Garaniya, V., 2018. Human error assessment during
maintenance operations of marine systems - What are the effective
environmental factors? Saf. Sci. 107, 85–98.
Jang, I., Kim, A.R., Al Harbi, M.A.S., Lee, S.J., Kang, H.G., Seong, P.H., 2013. An
empirical study on the basic human error probabilities for NPP advanced main
control room operation using soft control. Nucl. Eng. Des. 257, 79–87.
Jang, I., Kim, A.R., Jung, W., Seong, P.H., 2016. Study on a new framework of human
reliability analysis to evaluate soft control execution error in advanced MCRs of
NPPs. Ann. Nucl. Energy 91, 92–104.
Jin, Y., Ji, S., Li, X., Yu, J., 2017. A scientometric review of hotspots and emerging
trends in additive manufacturing. J. Manuf. Technol. Manage. 28 (1), 18–38.
Jung, W., Park, J., Kim, Y., Choi, S.Y., Kim, S., 2020. HuREX - A framework of HRA data
collection from simulators in nuclear power plants. Reliab. Eng. Syst. Saf. 194,
106235.
Jung, W., Whaley, A.M., Hallbert, B.P., 2009. Human errors during the simulations of
an SGTR scenario: Application of the HERA system. Nucl. Eng. Technol. 41 (10),
1361–1374.
Kančev, D., 2020. A plant-specific HRA sensitivity analysis considering dynamic
operator actions and accident management actions. Nucl. Eng. Technol. 52,
1983–1989.
Kang, S., Seong, P.H., 2020. Performance shaping factor taxonomy for human
reliability analysis on mitigating nuclear power plant accidents caused by
extreme external hazards. Ann. Nucl. Energy 145, 107533.
Kim, A.R., Park, J., Kim, J.T., Kim, J., Seong, P.H., 2016. Study on the identification of
main drivers affecting the performance of human operators during-low power
and shutdown operation. Ann. Nucl. Energy 92, 447–455.
Kim, A.R., Park, J., Kim, Y., Kim, J., Seong, P.H., 2017a. Quantification of performance
shaping factors (PSFs)’ weightings for human reliability analysis (HRA) of low
power and shutdown (LPSD) operations. Ann. Nucl. Energy 101, 375–382.
13
Annals of Nuclear Energy 151 (2021) 107969
Kim, Y., Park, J., 2019. Incorporating prior knowledge with simulation data to
estimate PSF multipliers using Bayesian logistic regression. Reliab. Eng. Syst.
Saf. 189, 210–217.
Kim, Y., Park, J., Jung, W., 2017b. A classification scheme of erroneous behaviors for
human error probability estimations based on simulator data. Reliab. Eng. Syst.
Saf. 163, 1–13.
Kim, Y., Park, J., Jung, W., 2017c. A quantitative measure of fitness for duty and work
processes for human reliability analysis. Reliab. Eng. Syst. Saf. 167, 595–601.
Kim, Y., Park, J., Jung, W., Choi, S.Y., Kim, S., 2018. Estimating the quantitative
relation between PSFs and HEPs from full-scope simulator data. Reliab. Eng.
Syst. Saf. 173, 12–22.
Kim, Y., Park, J., Jung, W., Jang, I., Seong, P.H., 2015. A statistical approach to
estimating effects of performance shaping factors on human error probabilities
of soft controls. Reliab. Eng. Syst. Saf. 142, 378–387.
Kirwan, B., 1996. The validation of three human reliability quantification techniques
THERP, HEART and JHEDI: Part 1 - Technique descriptions and validation issues.
Appl. Ergon. 27 (6), 359–373.
Kirwan, B., 1997. The validation of three human reliability Quantification
techniques THERP, HEART and JHEDI: Part III - practical aspects of the usage
of the techniques. Appl. Ergon. 28 (1), 27–39.
Kirwan, B., Kennedy, R., Taylor-Adams, S., Lambert, B., 1997. The validation of three
human reliability quantification techniques THERP, HEART and JHEDI: Part II -
results of validation exercise. Appl. Ergon. 28 (1), 17–25.
Kirytopoulos, K., Konstandinidou, M., Nivolianitou, Z., Kazaras, K., 2014. Embedding
the human factor in road tunnel risk analysis. Process Saf. Environ. Prot. 92 (4),
329–337.
Kumar, A.M., Rajakarunakaran, S., Prabhu, V.A., 2017. Application of fuzzy HEART
and expert elicitation for quantifying human error probabilities in LPG
refuelling station. J. Loss Prev. Process Ind. 48, 186–198.
Kumar, V.N.A., Gandhi, M.S., Gandhi, O.P., 2015. Identification and assessment of
factors influencing human reliability in maintenance using fuzzy cognitive
maps. Qual. Reliab. Eng. Int. 31 (2), 169–181.
Kyriakidis, M., Kant, V., Amir, S., Dang, V.N., 2018. Understanding human
performance in sociotechnical systems - Steps towards a generic framework.
Saf. Sci. 107, 202–215.
Laumann, K., Skogstad, M.R., 2020. Challenge to collect empirical data for human
reliability analysis-illustrated by the difficulties in collecting empirical data on
the performance-shaping factor complexity. Asce-Asme J. Risk Uncertainty Eng.
Syst. Part B-Mech. Eng. 6 (1), 11013.
Laumann, K., Rasmussen, M., 2016. Suggested improvements to the definitions of
standardized plant analysis of risk-human reliability analysis (SPAR-H)
performance shaping factors, their levels and multipliers and the nominal
tasks. Reliab. Eng. Syst. Saf. 145, 287–300.
Lecun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. Nature 521 (7553), 436–444.
Lee, S.J., Kim, J., Jung, W., 2013. Quantitative estimation of the human error
probability during soft control operations. Ann. Nucl. Energy 57, 318–326.
Lee, S.J., Seong, P.H., 2007. Development of an integrated decision support system to
aid cognitive activities of operators. Nucl. Eng. Technol. 39 (6), 703–716.
Lee, S.M., Ha, J.S., Seong, P.H., 2011. CREAM-based communication error analysis
method (CEAM) for nuclear power plant operators’ communication. J. Loss Prev.
Process Ind. 24 (1), 90–97.
Li, D.Y., Liu, C.Y., Gan, W.Y., 2009. A new cognitive model: Cloud model. Int. J. Intell.
Syst. 24 (3), 357–375.
Li, J., Yu, M.H., Wang, H.W., 2018a. A taxonomy of performance shaping factors for
shield tunnel construction. Eng. Constr. Arch. Manage. 25 (4), 574–596.
Li, P.C., Zhang, L., Dai, L.C., Li, X.F., Jiang, Y., 2018b. A new organization-oriented
technique of human error analysis in digital NPPs: Model and classification
framework. Ann. Nucl. Energy 120, 48–61.
Li, P.C., Chen, G.H., Dai, L.C., Zhang, L., 2012. A fuzzy Bayesian network approach to
improve the quantification of organizational influences in HRA frameworks. Saf.
Sci. 50 (7), 1569–1583.
Liao, H.F., Forester, J., Dang, V.N., Bye, A., Chang, Y.H.J., Lois, E., 2019a. Assessment of
HRA method predictions against operating crew performance: Part I: Study
background, design and methodology. Reliab. Eng. Syst. Saf. 191, 106509.
Liao, H.F., Forester, J., Dang, V.N., Bye, A., Chang, Y.H.J., Lois, E., 2019b. Assessment of
HRA method predictions against operating crew performance: Part II: Overall
simulator data, HRA method predictions, and intra-method comparisons.
Reliab. Eng. Syst. Saf. 191, 106510.
Liao, H.F., Forester, J., Dang, V.N., Bye, A., Chang, Y.H.J., Lois, E., 2019c. Assessment of
HRA method predictions against operating crew performance: Part III:
Conclusions and achievements. Reliab. Eng. Syst. Saf. 191, 106511.
Liu, H.C., Li, Z., Zhang, J.Q., You, X.Y., 2018. A large group decision making approach
for dependence assessment in human reliability analysis. Reliab. Eng. Syst. Saf.
176, 135–144.
Liu, P., Lyu, X., Qiu, Y.P., He, J.D., Tong, J.J., Zhao, J., Li, Z.Z., 2017a. Identifying key
performance shaping factors in digital main control rooms of nuclear power
plants: A risk-based approach. Reliab. Eng. Syst. Saf. 167, 264–275.
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., Alsaadi, F.E., 2017b. A survey of deep
neural network architectures and their applications. Neurocomputing 234, 11–
26.
Liu, P., Qiu, Y.P., Hu, J.T., Tong, J.J., Zhao, J., Li, Z.Z., 2020. Expert judgments for
performance shaping factors’ multiplier design in human reliability analysis.
Reliab. Eng. Syst. Saf. 194, 106343.
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al.
Marseguerra, M., Zio, E., Librizzi, M., 2007. Human reliability analysis by fuzzy
CREAM. Risk Anal. 27 (1), 137–154.
Martins, M.R., Maturana, M.C., 2010. Human error contribution in collision and
grounding of oil tankers. Risk Anal. 30 (4), 674–698.
Maturana, M.C., Martins, M.R., 2019. Technique for early consideration of human
reliability: Applying a generic model in an oil tanker operation to study
scenarios of collision. J. Offshore Mech. Arctic Eng.-Trans. Asme 141 (5), 51607.
Mkrtchyan, L., Podofillini, L., Dang, V.N., 2015. Bayesian belief networks for human
reliability analysis: A review of applications and gaps. Reliab. Eng. Syst. Saf. 139,
1–16.
Monferini, A., Konstandinidou, M., Nivolianitou, Z., Weber, S., Kontogiannis, T.,
Kafka, P., Kay, A.M., Leva, M.C., Demichela, M., 2013. A compound methodology
to assess the impact of human and organizational factors impact on the risk
level of hazardous industrial plants. Reliab. Eng. Syst. Saf. 119, 280–289.
Musharraf, M., Bradbury-Squires, D., Khan, F., Veitch, B., MacKinnon, S., Imtiaz, S.,
2014. A virtual experimental technique for data collection for a Bayesian
network approach to human reliability analysis. Reliab. Eng. Syst. Saf. 132, 1–8.
Myers, P.M., 2013. Layer of protection analysis - Quantifying human performance in
initiating events and independent protection layers. J. Loss Prev. Process Ind. 26
(3), 534–546.
Nan, C., Sansavini, G., 2016. Developing an agent-based hierarchical modeling
approach to assess human performance of infrastructure systems. Int. J. Ind.
Ergon. 53, 340–354.
Noroozi, A., Khakzad, N., Khan, F., MacKinnon, S., Abbassi, R., 2013. The role of
human error in risk analysis: Application to pre- and post-maintenance
procedures of process facilities. Reliab. Eng. Syst. Saf. 119, 251–258.
Osareh, F., 1996. Bibliometrics, citation analysis and co-citation analysis: A review
of literature II. Libri 46 (4), 217–225.
Pamučar, D., Stević, Ž., Zavadskas, E.K., 2018. Integration of interval rough AHP and
interval rough MABAC methods for evaluating university web pages. Appl. Soft
Comput. J. 67, 141–163.
Pan, X., Lin, Y., He, C., 2017. A review of cognitive models in human reliability
analysis. Qual. Reliab. Eng. Int. 33 (7), 1299–1316.
Pan, X., Wang, H.X., Lin, Y., Liu, T., Wang, X.X., 2019. HEP quantification strategy
based on modified CREAM. J. Syst. Eng. Electron. 30 (4), 815–822.
Pandya, D., Podofillini, L., Emert, F., Lomax, A.J., Dang, V.N., 2018. Developing the
foundations of a cognition-based human reliability analysis model via mapping
task types and performance-influencing factors: Application to radiotherapy.
Proc. Inst. Mech. Eng. Part O-J. Risk Reliab. 232 (1), 3–37.
Pandya, D., Podofillini, L., Emert, F., Lomax, A.J., Dang, V.N., Sansavini, G., 2020.
Quantification of a human reliability analysis method for radiotherapy
applications based on expert judgment aggregation. Reliab. Eng. Syst. Saf.
194, 106489.
Park, J., 2011. Scrutinizing inter-relations between performance influencing factors
and the performance of human operators pertaining to the emergency tasks of
nuclear power plant - An explanatory study. Ann. Nucl. Energy 38 (11), 2521–
2532.
Park, J., 2014. Investigating the TACOM measure as a general tool for quantifying the
complexity of procedure guided tasks. Reliab. Eng. Syst. Saf. 129, 66–75.
Park, J., Arigi, A.M., Kim, J., 2019a. A comparison of the quantification aspects of
human reliability analysis methods in nuclear power plants. Ann. Nucl. Energy
133, 297–312.
Park, J., Arigi, A.M., Kim, J., 2019b. Treatment of human and organizational factors
for multi-unit HRA: Application of SPAR-H method. Ann. Nucl. Energy 132, 656–
678.
Park, J., Jung, J.Y., Jung, W., 2016a. The use of a process mining technique to
characterize the work process of main control room crews: A feasibility study.
Reliab. Eng. Syst. Saf. 154, 31–41.
Park, J., Jung, W., Kim, J., 2020. Inter-relationships between performance shaping
factors for human reliability analysis of nuclear power plants. Nucl. Eng.
Technol. 52 (1), 87–100.
Park, J., Kim, Y., Jung, W., 2016b. A framework to estimate task opportunities from
the operational experience of domestic nuclear power plants. Saf. Sci. 88, 146–
154.
Park, J., Kim, Y., Jung, W., 2018. Calculating nominal human error probabilities from
the operation experience of domestic nuclear power plants. Reliab. Eng. Syst.
Saf. 170, 215–225.
Park, J., Kim, Y., Kim, J.H., Jung, W., Jang, S.C., 2015. Estimating the response times of
human operators working in the main control room of nuclear power plants
based on the context of a seismic event - A case study. Ann. Nucl. Energy 85, 36–
46.
Park, J., Lee, D., Jung, W., Kim, J., 2017. An experimental investigation on relationship
between PSFs and operator performances in the digital main control room. Ann.
Nucl. Energy 101, 58–68.
Pellegrino, O., 2011. Road context evaluated by means of fuzzy interval. Cogn.
Technol. Work 13 (1), 67–79.
Pence, J., Sakurahara, T., Zhu, X.F., Mohaghegh, Z., Ertem, M., Ostroff, C., Kee, E., 2019.
Data-theoretic methodology and computational platform to quantify
organizational factors in socio-technical risk analysis. Reliab. Eng. Syst. Saf.
185, 240–260.
Petrillo, A., De Felice, F., Falcone, D., Silvestri, A., Zomparelli, F., 2017. A hybrid
probabilistic model for evaluating and simulating human error in industrial
emergency conditions (HEIE). J. Fail. Anal. Prev. 17 (3), 462–476.
14
Annals of Nuclear Energy 151 (2021) 107969
Petruni, A., Giagloglou, E., Douglas, E., Geng, J., Leva, M.C., Demichela, M., 2019.
Applying analytic hierarchy process (AHP) to choose a human factors
technique: Choosing the suitable human reliability analysis technique for the
automotive industry. Saf. Sci. 119, 229–239.
Podofillini, L., Dang, V., Zio, E., Baraldi, P., Librizzi, M., 2010. Using expert models in
human reliability analysis-A dependence assessment method based on fuzzy
logic. Risk Anal. 30 (8), 1277–1297.
Porthin, M., Liinasuo, M., Kling, T., 2020. Effects of digitalization of nuclear power
plant control rooms on human reliability analysis – A review. Reliab. Eng. Syst.
Saf. 194, 106415.
Ramezani, A., Nazari, T., Rabiee, A., Hadad, K., Faridafshin, M., 2020. Human error
probability quantification for NPP post-accident analysis using Cognitive-Based
THERP method. Prog. Nucl. Energy 123, 103281.
Ramos, M.A., Droguett, E.L., Mosleh, A., Moura, M.D., 2020. A human reliability
analysis methodology for oil refineries and petrochemical plants operation:
Phoenix-PRO qualitative framework. Reliab. Eng. Syst. Saf. 193, 106672.
Rasmussen, M., Laumann, K., 2020. The evaluation of fatigue as a performance
shaping factor in the Petro-HRA method. Reliab. Eng. Syst. Saf. 194, 106187.
Rasmussen, M., Standal, M.I., Laumann, K., 2015. Task complexity as a performance
shaping factor: A review and recommendations in standardized plant analysis
risk-human reliability analysis (SPAR-H) adaption. Saf. Sci. 76, 228–238.
Reer, B., 2008a. Review of advances in human reliability analysis of errors of
commission, Part 2: EOC quantification. Reliab. Eng. Syst. Saf. 93 (8), 1105–
1122.
Reer, B., 2008b. Review of advances in human reliability analysis of errors of
commission, Part 1: EOC identification. Reliab. Eng. Syst. Saf. 93 (8), 1091–1104.
Rezaei, J., 2015. Best-worst multi-criteria decision-making method. Omega 53, 49–
57.
Shirley, R.B., Smidts, C., Zhao, Y.F., 2020. Development of a quantitative Bayesian
network mapping objective factors to subjective performance shaping factor
evaluations: An example using student operators in a digital nuclear power
plant simulator. Reliab. Eng. Syst. Saf. 194, 106416.
Shukla, A.K., Janmaijaya, M., Abraham, A., Muhuri, P.K., 2019. Engineering
applications of artificial intelligence: A bibliometric analysis of 30 years
(1988–2018). Eng. Appl. Artif. Intell. 85, 517–532.
Steijn, W.M.P., Van Kampen, J.N., Van der Beek, D., Groeneweg, J., Van Gelder, P.,
2020. An integration of human factors into quantitative risk analysis using
Bayesian Belief Networks towards developing a ‘QRA plus’. Saf. Sci. 122, 104514.
Stević, Ž., Pamučar, D., Puška, A., Chatterjee, P., 2020. Sustainable supplier selection
in healthcare industries using a new MCDM method: Measurement of
alternatives and ranking according to compromise solution (MARCOS).
Comput. Ind. Eng. 140, 106231.
Su, X.Y., Mahadevan, S., Xu, P.D., Deng, Y., 2015. Dependence Assessment in human
reliability analysis using evidence theory and AHP. Risk Anal. 35 (7), 1296–
1316.
Sujan, M.A., Embrey, D., Huang, H., 2020. On the application of human reliability
analysis in healthcare: Opportunities and challenges. Reliab. Eng. Syst. Saf. 194,
106189.
Sun, Y.H., Zhang, Q., Yuan, Z.M., Gao, Y., Ding, S.X., 2020. Quantitative analysis of
human error probability in high-speed railway dispatching tasks. IEEE Access 8,
56253–56266.
Sun, Z.Q., Li, Z.Y., Gong, E.L., Xie, H.W., 2012. Estimating human error probability
using a modified CREAM. Reliab. Eng. Syst. Saf. 100, 28–32.
Sun, Z.Q., Xie, H.W., Shi, X.J., Liu, F.Q., 2009. Engineering approach for human error
probability quantification. J. Syst. Eng. Electron. 20 (5), 1144–1152.
Swain, A., Guttman, H., 1983. Handbook of human reliability analysis with emphasis
on nuclear power plant applications. Sandia National Labs, Albuquerque, NM
(USA).
Swain, A.D., 1982. Modelling of human performance in complex system with
emphasis on nuclear power plant operations and probabilistic risk assessment.
Ergonomics 25 (6), 449.
Taga, H., Furuta, K., Kanno, T., 2012. Human reliability analysis of car drivers in
urban intersections. Cogn. Technol. Work 14 (4), 365–377.
Tao, J., Qiu, D., Yang, F., Duan, Z., 2020. A bibliometric analysis of human reliability
research. J. Cleaner Prod. 260, 121041.
Taylor, C., Oie, S., Gould, K., 2020. Lessons learned from applying a new HRA method
for the petroleum industry. Reliab. Eng. Syst. Saf. 194, 106276.
Ung, S.T., 2015. A weighted CREAM model for maritime human reliability analysis.
Saf. Sci. 72, 144–152.
Ung, S.T., 2019. Evaluation of human error contribution to oil tanker collision using
fault tree analysis and modified fuzzy Bayesian network based CREAM. Ocean
Eng. 179, 159–172.
Ureña, R., Chiclana, F., Melançon, G., Herrera-Viedma, E., 2019. A social network
based approach for consensus achievement in multiperson decision making.
Information Fusion 47, 72–87.
Wang, A.S., Luo, Y., Tu, G.Y., Liu, P., 2011. Quantitative evaluation of human-
reliability based on fuzzy-clonal Selection. IEEE Trans. Reliab. 60 (3), 517–527.
Wang, N., Du, X., Zhang, M., Xu, C., Lu, X., 2020. An improved weighted fuzzy CREAM
model for quantifying human reliability in subway construction: Modeling,
validation, and application. Hum. Factors Ergon. Manuf. Serv. Ind. 30, 248–265.
Williams, J.C., 1988. Data-based method for assessing and reducing human error to
improve operational performance. In: IEEE Conference on Human Factors and
Power Plants, pp. 436–450.
Lin-Xiu Hou, R. Liu, Hu-Chen Liu et al.
Wu, B., Yan, X.P., Wang, Y., Soares, C.G., 2017. An evidential reasoning-based CREAM
to human reliability analysis in maritime accident process. Risk Anal. 37 (10),
1936–1957.
Xi, Y.T., Yang, Z.L., Fang, Q.G., Chen, W.J., Wang, J., 2017. A new hybrid approach to
human error probability quantification-applications in maritime operations.
Ocean Eng. 138, 45–54.
Yager, R.R., Abbasov, A.M., 2013. Pythagorean membership grades, complex
numbers, and decision making. Int. J. Intell. Syst. 28 (5), 436–452.
Yang, Y., Chen, G., Reniers, G., Goerlandt, F., 2020. A bibliometric analysis of process
safety research in China: Understanding safety research progress as a basis for
making China’s chemical industry more sustainable. J. Cleaner Prod. 263,
121433.
Yang, Z.L., Abujaafar, K.M., Qu, Z.H., Wang, J., Nazir, S., Wan, C.P., 2019. Use of
evidential reasoning for eliciting bayesian subjective probabilities in human
reliability analysis: A maritime case. Ocean Eng. 186, 106095.
Yang, Z.L., Bonsall, S., Wall, A., Wang, J., Usman, M., 2013. A modified CREAM to
human reliability quantification in marine engineering. Ocean Eng. 58, 293–
303.
Yazdi, M., 2020. A perceptual computing-based method to prioritize intervention
actions in the probabilistic risk assessment techniques. Qual. Reliab. Eng. Int. 36
(1), 187–213.
Zhang, H., Zhao, S., Kou, G., Li, C.-C., Dong, Y., Herrera, F., 2020. An overview on
feedback mechanisms with minimum adjustment or cost in consensus reaching
in group decision making: Research paradigms and challenges. Inf. Fusion 60,
65–79.
15
Annals of Nuclear Energy 151 (2021) 107969
Zhang, R.Y., Tan, H., 2018. An integrated human reliability based decision pool
generating and decision making method for power supply system in LNG
terminal. Saf. Sci. 101, 86–97.
Zhang, S., He, W.P., Chen, D.K., Chu, J.J., Fan, H., 2019. A dynamic human reliability
assessment approach for manned submersibles using PMV-CREAM. Int. J. Nav.
Archit. Ocean Eng. 11 (2), 782–795.
Zheng, X.L., Deng, Y., 2018. Dependence assessment in human reliability analysis
based on evidence credibility decay model and IOWA operator. Ann. Nucl.
Energy 112, 673–684.
Zhou, Q.J., Wong, Y.D., Loh, H.S., Yuen, K.F., 2018. A fuzzy and Bayesian network
CREAM model for human reliability analysis - The case of tanker shipping. Saf.
Sci. 105, 149–157.
Zhou, Q.J., Wong, Y.D., Xu, H., Thai, V.V., Loh, H.S., Yuen, K.F., 2017a. An enhanced
CREAM with stakeholder-graded protocols for tanker shipping safety
application. Saf. Sci. 95, 140–147.
Zhou, T.Q., Wu, C.Z., Zhang, J.Y., Zhang, D., 2017b. Incorporating CREAM and MCS
into fault tree analysis of LNG carrier spill accidents. Saf. Sci. 96, 183–191.
Zio, E., Baraldi, P., Librizzi, M., Podofillini, L., Dang, V.N., 2009. A fuzzy set-based
approach for modeling dependence among human errors. Fuzzy Sets Syst. 160
(13), 1947–1964.
Zwirglmaier, K., Straub, D., Groth, K.M., 2017. Capturing cognitive causal paths in
human reliability analysis with Bayesian network models. Reliab. Eng. Syst. Saf.
158, 117–129.