Professional Documents
Culture Documents
Aggregating Risk Matrices Under A Normative Framework: Journal of Risk Research March 2019
Aggregating Risk Matrices Under A Normative Framework: Journal of Risk Research March 2019
Aggregating Risk Matrices Under A Normative Framework: Journal of Risk Research March 2019
net/publication/332117864
CITATIONS READS
5 1,951
4 authors:
Dengsheng Wu Jianping Li
Chinese Academy of Sciences Chinese Academy of Sciences
69 PUBLICATIONS 958 CITATIONS 233 PUBLICATIONS 2,967 CITATIONS
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Dengsheng Wu on 01 April 2019.
To cite this article: Chunbing Bao, Jie Wan, Dengsheng Wu & Jianping Li (2019):
Aggregating risk matrices under a normative framework, Journal of Risk Research, DOI:
10.1080/13669877.2019.1588912
1. Introduction
Risk aggregation is to composite individual risks to an overall one. In practice, decision makers
usually require the aggregated risk to make decisions at the whole organization level (Kouvelis
et al. 2012; Acharya, Almeida, and Campello 2013; Bernard, Jiang, and Wang 2014). In the cases
where data are sufficient, the quantitative methods are employed to obtain the overall risks.
For instance, in a bank, the overall risk should be the aggregated result of market risk, credit
risk, operational risk, and so on; and the result can be obtained using copulas (Li et al. 2015).
However, in practice, it is common that the data cannot meet the requirement of a quantita-
tive method to aggregate risks. In these cases, the quantitative methods will fail to work out the
problem (Wu et al. 2019). Some text mining algorithms are proposed to identify the risk factors
based on the risk disclosure text (Wei et al. 2019a; Wei et al. 2019b). In vague environment
where data are sparse, risk matrices have been popular and effective tools to assess risks (IEC
CONTACT Jianping Li ljp@casipm.ac.cn No. 15, Beiyitiao, Zhongguancun, Haidian District, Beijing 100190, China
ß 2019 Informa UK Limited, trading as Taylor & Francis Group
2 C. BAO ET AL.
2009; Iverson et al. 2012; Ruan, Yin, and Frangopol 2015; Ni, Chen and Chen 2010; Garvey and
Lansdowne 1998; Oliveira, Bana e Costa, and Lopes 2018; Hsu, Huang, and Tseng 2016). Risk
matrices are qualitative tools used to rate risks according to two dimensions (inputs), namely,
consequence and likelihood. When decision makers need to assess the severity of a risk using a
risk matrix, firstly, they should subjectively decide the levels of the consequence and likelihood
of the risk, then the risk matrix outputs the rating, denoted by a certain color, for example, red,
yellow, or green (Bao, Li, and Wu 2018a; Li, Bao, and Wu 2018; Cox 2008; Ball and Watt 2013).
Figure 1 presents a typical 3 3 risk matrix.
Though risk matrices have been widely used in many fields, few extant literature concern
about the aggregation of different risks measured with risk matrices. The difficulty of the aggre-
gation is twofold (Bao, Li, and Wu 2018a; Duijm 2015; IEC 2009). One is that it is difficult to
determine the magnitude of different risks in different scenarios since the risks are assessed with
subjective adjectives. For example, the risk rating ‘medium’ in a case may be more severe in
another case. The other is the types of risks may be different, such as casualties, economic loss,
and so on, making it difficult to compare different risks. Because of the two difficulties, even ISO
states that ‘risks cannot be aggregated’ (IEC 2009).
Bao et al. tried to resolve the aggregation problem of risk matrices (Bao, Li, and Wu 2018a). They
first conquered the difficulties by using quantitative risk matrices and normalizing the axes of
inputs. Then a fuzzy mapping framework was proposed to aggregate risks assessed by risk matrices.
The purposes of this paper are to find a more general framework of the aggregation and see if any
other methods are appropriate for resolving the problem. Specifically, four steps necessary are sum-
marized for the aggregation of risk matrices: (1) employ the risk matrices to assess individual risks,
(2) find appropriate expressions of the risk ratings, (3) aggregate individual risks by composition
methods, and (4) transform the composition results to specific values. Then two new methods,
namely, the interval number based method and the probability density function method, are pro-
posed. Lastly, an example is presented to show the difference between the three methods. The
three methods are compared from different aspects. Results reveal that the interval number method
is the least robust one, the fuzzy set method explains the risk matrix best, and the probability dens-
ity function could be a substitution for the fuzzy set method since they have similar results.
risks are measured by predesigned risk matrices, the concept of ‘aggregating risk matrices’ is
substituted for ‘aggregating individual risks measured by risk matrices’ for simplicity (Duijm 2015;
IEC 2009; Bao, Li, and Wu 2018a).
Generally speaking, four steps constitute the framework. Given several different risks assessed
by risk matrices, we should (1) assess individual risks according to the normalized quantitative
risk matrices. Then, (2) find appropriate expressions of the risk ratings. Next, (3) aggregate indi-
vidual risks by composition methods. And, (4) transform the results of aggregation to specific val-
ues respectively to make it able to compare different portfolios which consist of differently rated
risks. The framework is as Figure 2 shows. To obtain relative severity of a risk portfolio, we
should obtain all the values of different portfolios.
2.1. Assess individual risks according to the normalized quantitative risk matrices
As is suggested by the researchers that study on risk matrices should be conducted on quantita-
tive risk matrices, namely, the consequence and likelihood of a risk matrix should be described
with numerical intervals (Bao, Li, and Wu 2018b; Li, Bao, and Wu 2018; Cox 2008). For example,
the category ‘medium’ of consequence corresponds to the interval of ($10 M, $20M). Besides, it
is known that the dimension of different risks mostly varies from one to another. For example,
the types of consequence could be an economic loss, casualties, and so on. Given different types
of consequence, in this step, they should be normalized so that the quantitative intervals are
from 0 to 1, making different types of consequence comparable. Therefore, in this paper, we
aggregate different risks caring about their overall severity instead of the sum of quantitative values
of the risks. The usage of quantitative risk matrices and the normalization of consequence and
likelihood are two important settings that conquer the obstacles preventing the aggregation of
risks measured by risk matrices.
After obtaining the normalized quantitative risk matrices, the decision makers should assess
the consequence and likelihood of each risk based on their knowledge or experience. Different
decision makers may assign different categories of consequence and likelihood to the same risk.
However, in this step, the decision makers must reach a consensus on the categories of conse-
quence and likelihood to guarantee the uniqueness of the risk rating of an assessed risk.
Therefore, before finding an appropriate way to aggregate risks, the estimations to the conse-
quence and likelihood of these risks based on the risk matrices should be given in advance, as
shown to the left of the brace in Figure 2. The following steps are carried out based on these
assessed risks and the corresponding risk matrices.
4 C. BAO ET AL.
Since a point in the risk matrices represents the possible value of the assessed risk, we could
view a risk rating from the perspective of distribution. In general, we explore a probability dens-
ity function to reflect the information that a risk rating contains. We first obtain the cumulative
distribution function FX ðrÞ ¼ PðX rÞ; where r denotes the given risk value in rating G(green),
Y(yellow), or R(red). For example, in Figure 5, given any risk value r in rating G, the cumulative
probability of X r can be approximated by the proportion of the shaded area in the green
area. Then the probability density function is the derivation of the cumulative distribution func-
tion with respect to r.
The above is the introduction of the three possible expressions the risk rating. These expres-
sions are constructed because the risk rating contains infinite points instead of a single one. And
these expressions reflect the information of the risk rating from different perspectives. The next
part tells how to aggregate the risk ratings based on these expressions.
For the fuzzy set, the aggregation is by the composition method of membership functions of
fuzzy sets. For any two fuzzy sets, the most prevalent composition method is the max–min rule
shown in Equation (1) (Zimmermann 2001):
lAB ðzÞ ¼ max min lA ðxÞ; lB ðyÞ : (1)
z¼xþy
All risks are assumed to be independent of each other in this paper. Thus, by repeatedly
using Equation (1), the aggregation of two or more fuzzy sets is achieved by the following
rule:
lA1 þA2 þþAi þþAn ðzÞ ¼ max min lA1 ðx1 Þ; lA2 ðx2 Þ; :::; lAi ðxi Þ; :::; lAn ðxn Þ ; (2)
z¼x1 þx2 þxi þþxn
where lAi ðxi Þis the membership function, denoting the degree of xi in set Ai (or rating Ai ).
When the ratings of risks are denoted by interval numbers, aggregating risks is by the add-
ition of interval numbers according to the addition principle of interval number. The aggregated
result obtained is an interval number denoting the overall risk of the risk portfolio.
Lastly, if the ratings are expressed by the probability density functions, such as f(x),
aggregating different risks is to obtain the probability density function of the sum of the risks,
namely, f (x1þx2þ … ). This process can be carried out analytically or with the aid of Monte Carlo
simulation, details of which will be given in Section 3.3.
Obviously, the composition methods help aggregate several risks into an overall one which
has the same form of expression as the individual risks. For example, if the individual risk ratings
are expressed by fuzzy sets, the aggregated risk is still of the form of a fuzzy set.
centroid method, weighted average method, and so on (Zimmermann 2001). The most prevalent
one is the centroid method, which is defined as:
Ð
l ðzÞ zdz
z ¼ ÐC : (3)
lC ðzÞdz
Similarly, interval numbers and the probability density functions can be transformed into
comparable values. Details of the transformation will be introduced in Section 3.
Although the interval number cannot be directly transformed into a specific value, with other
interval numbers to be compared, the interval number can be transformed into a specific value
indirectly. In other words, the value does not exist where there is only one interval number, only
if there are several interval numbers to be compared, the value can be calculated accordingly.
The formula and detailed process are in the following part 3.2.
As for the method of probability density function, the expectation of the distribution is usu-
ally used to represent the distribution of an objective. The expectation of distribution can be
obtained analytically or using the Monte Carlo simulation method.
Remark: The above four parts are necessary when aggregate the risks measured by risk matri-
ces, constituting a more general framework than the fuzzy mapping framework proposed by
Bao, Li, and Wu (2018a). Under the framework, different methods are allowed to resolve the
aggregation problem. Furthermore, we can explore the difference between the methods and
show which one is more reliable.
~ ¼ bL ; b U ; l a
~ ¼ aL ; aU ; b
Let a ~ ¼ bU bL ;
~ ¼ aU aL ; lb
then
n o
min la ~ max aU bL ; 0
~ þ lb;
p a ~ ¼
~ b ~
þ lb; (6)
~
la
which represents the probability of a ~ b:~ The relation can be denoted by a ~ Suppose there
~ p b:
are N interval numbers, namely, a ~ i ¼ ½ai ; ai ; i 2 N; Formula (6) can be used to calculate
L U
Formula (7) represents the magnitudes of interval numbers a~i : vi can be used to compare the
magnitudes of different interval numbers.
Aggregating risk matrices by the aggregation method of interval number is fairly simple,
which is achieved just by using the addition operation of the interval numbers. Specifically,
each risk rating can be denoted by an interval number, by addition of interval numbers, we
can obtain the overall risk denoted by an interval number. What should be paid attention to
JOURNAL OF RISK RESEARCH 9
is an interval number itself does not have a v and thus there is not a crisp value to represent
its magnitude. Therefore, to obtain the crisp value of a portfolio, we should collect all the
interval numbers of all possible portfolios. The detailed process is as follows.
Step 1. Assess the n risks in their respective risk matrices to obtain the risk ratings.
Step 2. Denote the ratings of the evaluated risks by interval numbers, take the specific risk
matrix in Figure 1 for instance, g ~ ¼ ½0; 1=3; ~y ¼ ½1=9; 2=3; ~r ¼ ½4=9; 1 (g
~ ; ~y ; ~r represent interval
numbers of rating green, rating yellow, and rating red, respectively).
Step 3. According to the calculation principle of interval numbers, we can aggregate the
individual risks by the addition of interval numbers, the formula is a ~ þb ~ ¼ ½aL þ bL ; aU þ bU :
The use of interval numbers should correspond to the ratings of risks in the portfolio.
Step 4. Use formula (7) to calculate i to compare the different interval numbers.
Much attention should be paid on i ; because the results will differ when objects
of comparison are different. When the comparison is conducted, i is generated, and when there
is not a comparison, i does not exist.
estimation of the risks. Therefore, we employ the risk matrix shown in Figure 6 to assess these
risks (of course, different risk matrices can be used to assess different risks. For simplicity, the
same risk matrix is adopted in this example).
Firstly, the decision makers need to give an estimation to the consequence and likelihood
of each risk. The quantitative description of each category of consequence is provided in Table 1.
As is shown in the table above, R1 ;R2 ;R3 ; and R4 are schedule delay, casualty, misoperation, and
environmental pollution, respectively. Any infinite interval is not considered in the aggregation
because will have the highest priority. Notice that some of these risks are usually measured in the
discrete index, so we need to translate them into the continuous indexes, such as casualty (R2 ).
Casualty is usually measured by the number of dead or injured people so we translate it into the
expenses spent. Moreover, the likelihood is [0, 1/3], (1/3, 2/3], and (2/3, 1], which correspond to
‘Low’, ‘Medium’, and ‘High’, respectively. It has to be emphasized that the interval length does not
have to be equal, which depends on the subjective judgment of decision makers.
After giving the estimation to the consequence and likelihood of the 4 risks, the decision
makers obtain the risk rating of each risk according to the risk matrix in Figure 6. The risk ratings
of the 4 risks are reported in Table 2.
Based on the information, the proposed methods are employed to tell what the severity
of the overall risk is. Obviously, to obtain the severity of a possible portfolio, we must know the
magnitude of the overall risks of other portfolios. In theory, there are totally 81 (=3 3 3 3)
possible risk portfolios. Since the same colored risk ratings have the same consequence and like-
lihood in each risk matrix, they are identical. Therefore, we can compress the number of different
portfolios. After this operation, totally, there are 15 different risk portfolios.
Based on the aforementioned methods, the results are exhibited in Table 3.
JOURNAL OF RISK RESEARCH 11
Table 3. Ranking of the severity of different portfolios of the aggregated risks shown in Table 1 by different methods.
Portfolios Fuzzy set Interval number Cumulative distribution function Ranking
riskred ; riskred ; riskred ; riskred 2.776/1 17.056/1 2.776/1 1
riskred ; riskred ; riskred ; riskyellow 2.444/0.862 16.041/0.875 2.445/0.862 2
riskred ; riskred ; riskred ; riskgreen 2.175/0.750 15.453/0.803 2.182/0.752 3
riskred ; riskred ; riskyellow ; riskyellow 2.108/0.722 14.940/0.740 2.115/0.724 4
riskred ; riskred ; riskyellow ; riskgreen 1.847/0.613 14.258/0.656 1.844/0.611 5
riskred ; riskyellow ; riskyellow ; riskyellow 1.779/0.585 13.810/0.601 1.777/0.584 6
riskred ; riskred ; riskgreen ; riskgreen 1.579/0.501 13.494/0.562 1.575/0.500 7
riskred ; riskyellow ; riskyellow riskgreen 1.510/0.473 13.063/0.510 1.511/0.472 8
riskyellow ; riskyellow ; riskyellow ; riskyellow 1.441/0.444 12.679/0.463 1.445/0.445 9
riskred ; riskyellow ; riskgreen ; riskgreen 1.247/0.363 12.226/0.407 1.247/0.363 10
riskyellow ; riskyellow ; riskyellow ; riskgreen 1.178/0.334 11.868/0.363 1.178/0.334 11
riskred ; riskgreen ; riskgreen ; riskgreen 0.981/0.253 11.283/0.291 0.978/0.251 12
riskyellow ; riskyellow ; riskgreen ; riskgreen 0.910/0.223 10.959/0.251 0.910/0.223 13
riskyellow ; riskgreen ; riskgreen ; riskgreen 0.647/0.113 9.961/0.129 0.643/0.111 14
riskgreen ; riskgreen ; riskgreen ; riskgreen 0.375/0 8.913/0 0.376/0 15
Two values of risk portfolios are given. One is the original value and other is the normalized one.
Firstly, we notice that some results are consistent with common sense, like the scenario of
four ‘red’ risks has a higher priority than the one of four ‘yellow’ or ‘green’ risks; and the scenario
of three identical risks with the remaining risk of ‘red’ has the higher priority than with the
remaining risk of ‘yellow’ or ‘green’. Furthermore, the above tables provide some results
that cannot be estimated intuitively; for instance, the portfolio (riskred ; riskgreen ; riskgreen ; riskgreen )
has a higher ranking than the portfolio (riskyellow ; riskyellow ; riskgreen ; riskgreen ), which is one of the
main contributions of the methods.
Besides, we find that all the three methods output the same ranking of the risk portfolios.
However, the ranking is not the final result we are to obtain. In the next step, we will divide the
15 risk portfolios into several ratings for the purpose to obtain the severity of a risk portfolio.
In order to compare the results straightforwardly, the values of the aggregated risks are
normalized to ½0; 1 as shown in Table 3. Next, we divide all the 15 risk portfolios into three risk
levels according to the thresholds 1/3 and 2/3. Figures 7–9 report the division of the risk
portfolios calculated by different methods.
We see that even calculated by different methods, under the division criteria, the division of
the risk portfolios are the same, namely, the lowest 4 risk portfolios are assigned the level of 3,
the largest 4 are of risk level 1, and the remaining 2.
So far, by the aggregation methods, we reach our goal, namely, obtaining the severity (or risk
rating) of the overall risk. Therefore, in this example, since the 4 risks are estimated as ‘yellow,’
‘green,’ ‘green,’ and ‘red,’ respectively, the overall risk should have the risk rating of rating ‘2,’
which is at a lower–moderate level.
12 C. BAO ET AL.
It seems that in this example, namely, when this 3 3 risk matrix is adopted, the three meth-
ods could be substitutes for each other. However, we find some differences between the results.
The distributions of different risk scenarios obtained by a fuzzy set and cumulative distribution
function are almost the same. Furthermore, compared with the result obtained by a fuzzy set
and cumulative distribution function, the normalized values obtained by interval number are
larger. Does it raise the question that whether the interval number method overestimates the
risk of a portfolio? We wonder whether these methods are robust under other circumstances.
This will be discussed in the following section.
Table 4. Ranking of the severity of different portfolios of the aggregated risks measured by the risk matrix in Figure 10 by
different methods.
Cumulative
Fuzzy set distribution function Interval number
Portfolios Value Ranking Value Ranking Value Ranking
riskred ; riskred ; riskred ; riskred 3.275/1 1 2.778/1 1 17.827/1 1
riskred ; riskred ; riskred ; riskyellow 2.894/0.867 2 2.448/0.863 2 16.612/0.875 2
riskred ; riskred ; riskred ; riskgreen 2.548/0.746 3 2.175/0.749 3 15.960/0.809 3
riskred ; riskred ; riskyellow ; riskyellow 2.455/0.713 4 2.111/0.722 4 15.286/0.739 4
riskred ; riskred ; riskyellow ; riskgreen 2.163/0.611 5 1.843/0.611 5 14.501/0.659 5
riskred ; riskyellow ; riskyellow ; riskyellow 2.073/0.579 6 1.780/0.585 6 13.989/0.606 6
riskred ; riskred ; riskgreen ; riskgreen 1.818/0.490 7 1.577/0.500 7 13.501/0.556 7
riskred ; riskyellow ; riskyellow riskgreen 1.743/0.464 8 1.511/0.473 8 13.085/0.514 8
riskyellow ; riskyellow ; riskyellow ; riskyellow 1.553/0.398 9 1.444/0.445 9 12.773/0.482 9
riskred ; riskyellow ; riskgreen ; riskgreen 1.415/0.349 10 1.242/0.361 10 11.950/0.397 10
riskyellow ; riskyellow ; riskyellow ; riskgreen 1.329/0.319 11 1.177/0.333 11 11.792/0.381 11
riskred ; riskgreen ; riskgreen ; riskgreen 1.083/0.233 12 0.978/0.250 12 10.639/0.263 13
riskyellow ; riskyellow ; riskgreen ; riskgreen 0.993/0.202 13 0.910/0.222 13 10.640/0.263 12
riskyellow ; riskgreen ; riskgreen ; riskgreen 0.699/0.099 14 0.645/0.112 14 9.372/0.133 14
riskgreen ; riskgreen ; riskgreen ; riskgreen 0.417/0 15 0.377/0 15 8.074/0 15
Two values of risk portfolios are given. One is the original value and other is the normalized one.
Secondly, we observe that the values of the results of the fuzzy number and cumulative
distribution function are relatively close. In most cases, the values of the results of the fuzzy
number and cumulative distribution function are the same until the percentile of the number,
whereas the magnitude of i which is obtained by the method of interval number are far from
the magnitude which the former two methods get. We regard this as an implication that the
methods of fuzzy set and of cumulative distribution function are more stable.
The interval number method does not always perform well when the form of risk matrices
changes. It is because that interval numbers only record information of intervals from the small-
est risk value to the largest risk value of a rating. For example, in Figure 11, the two risk matrices
are different, but the interval numbers used to express each risk rating are the same. Therefore,
interval numbers reflect little information of the relative position of these ratings and lead to an
information loss. The characteristic forms of the matrices are not embodied at all, which makes
the method of interval numbers ineffective in some cases.
Next, similar to the previous part, in order to compare the results straightforwardly, the values
of the aggregated risks are normalized to [0, 1] and divided into three risk levels according to
the thresholds of 1/3 and 2/3. Figures 12–14 show the division of the risk portfolios calculated
by different methods.
14 C. BAO ET AL.
Figure 14. Risk levels of different scenarios (calculated by cumulative distribution function).
Figures 12–14 show that although the ranking derived from the aggregation approach of
interval number differs in the ranking of scenario (riskred ; riskgreen ; riskgreen ; riskgreen ) and scenario
(riskyellow ; riskyellow ; riskgreen ; riskgreen ), the bars of the two scenarios are so close that we consider
the aggregation approach of interval number does not perform well in the judgment of the
JOURNAL OF RISK RESEARCH 15
Table 5. Comparison of the aggregation methods of fuzzy set, interval number and probability density function.
Methods Comprehensibility Complexity Degree of explanation of risk matrix
Fuzzy set Hard to comprehend Most complex High
Interval number Easy to comprehend Easiest Low
Probability density function Medium Medium Medium
ranking of the two scenarios. Nevertheless, the two scenarios are distinguished by the other two
approaches well because of an appropriate distance between the two bars, making it possible to
rank the two scenarios.
4.3.1. Comprehensibility
Amongst the three approaches, the aggregation method based on interval numbers is the most
comprehensible one, attributing to its simplicity and feasibility. Decision makers only need to
estimate the consequence and likelihood in their respective risk matrices to obtain the respective
ratings and then get three interval numbers to add up.
Moreover, the aggregation method of probability density function is considered to be
relatively comprehensible whose comprehensibility is between fuzzy set and interval number.
The aggregation method of a fuzzy set is the least intuitive one, which includes several
important conceptions such as membership function and the rule to aggregate fuzzy sets,
making it hard to comprehend.
4.3.2. Complexity
Similar to the dimension of comprehensibility, the method of a fuzzy set is the most complex
one. The processes of obtaining fuzzy numbers, the aggregation, and the defuzzification are all
complicated.
The method of cumulative distribution function remains the middle one in complexity,
while the method of interval number is the simplest one.
of explanation of risk matrix, it could be used as an approximation for the result of fuzzy set
method for its simplicity.
Disclosure statement
No potential conflict of interest was reported by the authors.
Funding
This work was supported by the National Natural Science Foundation of China under Grant 71571179, 71425002,
and 71433001; the Key Research Program of Frontier Sciences of the Chinese Academy of Sciences under Grant
QYZDB-SSW-SYS036; the Youth Innovation Promotion Association of the Chinese Academy of Sciences under Grant
2012137 and 2013112.
ORCID
Chunbing Bao http://orcid.org/0000-0003-0569-3910
Dengsheng Wu http://orcid.org/0000-0002-6162-1287
Jianping Li http://orcid.org/0000-0003-4976-4119
References
Acharya, V. V., H. Almeida, and M. Campello. 2013. “Aggregate Risk and the Choice between Cash and Lines of
Credit.” Journal of Finance 68 (5): 2059–2116. doi:10.1111/jofi.12056.
Ball, D. J., and J. Watt. 2013. “Further Thoughts on the Utility of Risk Matrices.” Risk Analysis: An Official Publication
of the Society for Risk Analysis 33 (11): 2068–2078. doi:10.1111/risa.12057.
Bao, C., J. Li, and D. Wu. 2018a. “A Fuzzy Mapping Framework for Risk Aggregation Based on Risk Matrices.” Journal
of Risk Research 21 (5): 539–561. doi:10.1080/13669877.2016.1223161.
JOURNAL OF RISK RESEARCH 17
Bao, C., J. Li, and D. Wu. 2018b. “A Knowledge-Based Risk Measure from the Fuzzy Multi-Criteria Decision-Making
Perspective.” IEEE Transactions on Fuzzy Systems, Online. doi:10.1109/TFUZZ.2018.2838064.
Bernard, C., X. Jiang, and R. Wang. 2014. “Risk Aggregation with Dependence Uncertainty.” Insurance: Mathematics
and Economics 54: 93–108. doi:10.1016/j.insmatheco.2013.11.005.
Cox, L. A. 2008. “What’s Wrong with Risk Matrices?” Risk Analysis: An Official Publication of the Society for Risk
Analysis 28 (2): 497–512. doi:10.1111/j.1539-6924.2008.01030.x.
Duijm, N. J. 2015. “Recommendations on the Use and Design of Risk Matrices.” Safety Science 76: 21–31. doi:
10.1016/j.ssci.2015.02.014.
Durrett, R. 2005. Probability: Theory and Examples. Cambridge: Cambridge University Press.
Garvey, P. R., and Z. F. Lansdowne. 1998. “Risk Matrix: An Approach for Identifying, Assessing, and Ranking Program
Risks.” Air Force Journal of Logistics 22 (1): 18–21.
Hsu, W. K. K., S. H. S. Huang, and W. J. Tseng. 2016. “Evaluating the Risk of Operational Safety for Dangerous Goods
in Airfreights – A Revised Risk Matrix Based on Fuzzy AHP.” Transportation Research Part D 48: 235–247. doi:
10.1016/j.trd.2016.08.018.
IEC 2009. “Risk Management–Risk Assessment Techniques.” ISO 31010: 2009-11.
Iverson, L. R., S. N. Matthews, A. M. Prasad, M. P. Peters, and G. Yohe. 2012. “Development of Risk Matrices for
Evaluating Climatic Change Responses of Forested Habitats.” Climatic Change 114 (2): 231–243. doi:10.1007/
s10584-012-0412-x.
Kouvelis, P., L. Dong, O. Boyabatli, and R. Li. 2012. “Integrated Risk Management: A Conceptual Framework with
Research Overview and Applications in Practice.” The Handbook of Integrated Risk Management in Global Supply
Chains. Hoboken, New Jersey: Wiley; 2011.
Kumar, E. V., and S. K. Chaturvedi. 2009. “True Degradation Estimation of Industrial Equipment with Fuzzy Sets: A
Case Study.” Journal of Risk & Reliability 223 (2): 167–179. doi:10.1243/1748006XJRR201.
Li, J., C. Bao, and D. Wu. 2018. “How to Design Rating Schemes of Risk Matrices: A Sequential Updating Approach.”
Risk Analysis: An Official Publication of the Society for Risk Analysis 38 (1): 99–117. doi:10.1111/risa.12810.
Li, J., X. Yao, X. Sun, and D. Wu. 2017. “Determining the Fuzzy Measures in Multiple Criteria Decision Aiding from
the Tolerance Perspective.” European Journal of Operational Research 264 (2): 428–439. doi:10.1016/
j.ejor.2017.05.029.
Li, J., X. Zhu, C. F. Lee, D. Wu, J. Feng, and Y. Shi. 2015. “On the Aggregation of Credit, Market and Operational
Risks.” Review of Quantitative Finance and Accounting 44 (1): 161–189. doi:10.1007/s11156-013-0426-0.
Markowski, A. S., and M. S. Mannan. 2008. “Fuzzy Risk Matrix.” Journal of Hazardous Materials 159 (1): 152–157. doi:
10.1016/j.jhazmat.2008.03.055.
Ni, H., A. Chen, and N. Chen. 2010. “Some Extensions on Risk Matrix Approach.” Safety Science 48 (10): 1269–1278.
doi:10.1016/j.ssci.2010.04.005.
Oliveira, M. D., C. A. Bana e Costa, and D. F. Lopes. 2018. “Designing and Exploring Risk Matrices with Macbeth.”
International Journal of Information Technology & Decision Making 17 (1): 45–81. doi:10.1142/S0219622015500170.
Ruan, X., Z. Y. Yin, and D. M. Frangopol. 2015. “Risk Matrix Integrating Risk Attitudes Based on Utility Theory.” Risk
Analysis: An Official Publication of the Society for Risk Analysis 35 (8): 1437–1447. doi:10.1111/risa.12400.
Sengupta, A., and T. K. Pal. 2000. “On Comparing Interval Numbers.” European Journal of Operational Research 127
(1): 28–43. doi:10.1016/S0377-2217(99)00319-7.
Sengupta, A., and T. K. Pal. 2009. On Comparing Interval Numbers: A Study on Existing Ideas, 25–37. Berlin,
Heidelberg: Springer;.
Sun, H. L., and W. X. Yao. 2008. “The Basic Properties of Some Typical Systems’ Reliability in Interval Form.”
Structural Safety 30 (4): 364–373. doi:10.1016/j.strusafe.2007.05.003.
Wei L., G. Li, X. Zhu, X. Sun, and J. Li. 2019a. “Developing a hierarchical system of energy corporate risk factors
based on textual risk disclosures.” Energy Economics 80: 452–460. doi:10.1016/j.eneco.2019.01.020.
Wei L., G. Li, X. Zhu, X. Sun, and J. Li. 2019b. “Discovering bank risk factors from financial statements based on a
new semi-supervised text mining algorithm.” Accounting & Finance, online. doi:10.1111/acfi.12453.
Wu, D., X. Zhu, J. Wan, C. Bao, and J. Li. 2019. “A multiobjective optimization approach for selecting risk response
strategies of software project: From the perspective of risk correlations.” International Journal of Information
Technology & Decision Making 18 (1): 339–364. doi:10.1142/S0219622019410013.
Zadeh, L. A. 1965. “Fuzzy Sets.” Inform Control 8 (3): 338–353. doi:10.1016/S0019-9958(65)90241-X.
Zimmermann, H. J. 2001. Fuzzy Set Theory and Its Applications. Berlin: Springer.