Accepted Manuscript

An Integrated Decision Support System Based on ANN and

Fuzzy_AHP for Heart Failure Risk Prediction

Oluwarotimi Williams Samuel , Grace Mojisola Asogbon ,

Arun Kumar Sangaiah , Fang Peng , Guanglin Li

PII: S0957-4174(16)30551-6
DOI: 10.1016/j.eswa.2016.10.020
Reference: ESWA 10925

To appear in: Expert Systems With Applications

Received date: 25 February 2016

Revised date: 10 October 2016
Accepted date: 11 October 2016

Please cite this article as: Oluwarotimi Williams Samuel , Grace Mojisola Asogbon ,
Arun Kumar Sangaiah , Fang Peng , Guanglin Li , An Integrated Decision Support System Based
on ANN and Fuzzy_AHP for Heart Failure Risk Prediction , Expert Systems With Applications (2016),
doi: 10.1016/j.eswa.2016.10.020

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service
to our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and
all legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

Highlights
This study proposed a hybrid decision support method (ANN and Fuzzy_AHP) for heart failure prediction.

The performance of the proposed method was examined using three performance metrics.

From the evaluations results, the proposed method performed better than the conventional ANN approach

The proposed method would provide improved and realistic result for efficient therapy administration.

T
IP
CR
US
AN
M
ED
PT
CE
AC

1
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

An Integrated Decision Support System Based on ANN and Fuzzy_AHP

for Heart Failure Risk Prediction

Oluwarotimi Williams Samuela, b, , Grace Mojisola Asogbona, b, , Arun Kumar Sangaiahc,

Peng Fanga,b, and Guanglin Lia,b*

T
IP
samuel@siat.ac.cn1, mojiasho@yahoo.com2, arunkumarsangaiah@gmail.com3,

peng.fang@siat.ac.cn4, and gl.li@siat.ac.cn5

CR
a
CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institutes of

US
Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), Shenzhen, China
AN
b
Shenzhen College of Advanced Technology, University of Chinese Academy of Sciences,

Shenzhen 518055, China

M

c
School of Computing Science and Engineering, VIT University, Vellore-632014, Tamil Nadu,
ED

India.
PT

* Corresponding Author: Guanglin Li

CE

1068 Xueyuan Avenue, University Town of Shenzhen

AC

Xili, Nanshan, Shenzhen, 518055 China

Email: gl.li@siat.ac.cn or glli_10@yahoo.com

Phone: 86-755-86392219

The first two authors contributed equally to this work

2
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

Abstract: Heart failure (HF) has been considered as one of the deadliest human diseases

worldwide and the accurate prediction of HF risks would be vital for HF prevention and

treatment. To predict HF risks, decision support systems based on artificial neural networks

(ANN) have been widely proposed in previous studies. Generally, these existing ANN-based

systems usually assumed that HF attributes have equal risk contribution to the HF diagnosis.

T
However, several previous investigations have shown that the risk contributions of the attributes

IP
would be different. Thus the equal risk assumption concept associated with existing ANN

methods would not properly reflect the diagnosis status of HF patients. In this study, the

CR
commonly used 13 HF attributes were considered and their contributions were determined by an

experienced cardiac clinician. And Fuzzy analytic hierarchy process (Fuzzy_AHP) technique

US
was used to compute the global weights for the attributes based on their individual contribution.

Then the global weights that represent the contributions of the attributes were applied to train an
AN
ANN classifier for the prediction of HF risks in patients. The performance of the newly proposed

decision support system based on the integration of ANN and Fuzzy_AHP methods was
M

evaluated by using online clinical dataset of 297 HF patients and compared with that of the

conventional ANN method. Our result shows that the proposed method could achieve an average
ED

prediction accuracy of 91.10%, which is 4.40% higher in comparison to that of the conventional

ANN method. In addition, the newly proposed method also had better performance than seven
PT

previous methods that reported prediction accuracies in the range of 57.85 to 89.01%. The

improvement of the HF risk prediction in the current study might be due to both the various
CE

contributions of the HF attributes and the proposed hybrid method. These findings suggest that

the proposed method could be used to accurately predict HF risks in the clinic.
AC

Keywords: Heart failure; Risk prediction; Artificial neural networks; Fuzzy AHP; Clinical

decision support system;

3
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

1. Introduction

One of the most common and complex health problems around the world to date is heart failure

(HF) (McRae et al., 2016; Kumar et al., 2016; Bui et al., 2014), which usually occurs when the

heart fails to pump sufficient amount of blood to meet the needs of the body. The HF patients

often experience some symptoms such as shortness of breath, weakness of the body, and swollen

T
feet (Rahimi et al., 2012; Durairaj & Sivagowry, 2014). The findings from the previous

IP
investigations have identified HF and its resulting complications as one of the major causes of

CR
reduced quality of life as well as mortality in many technologically advanced countries (Hui &

Jonathan, 2015; Allen et al., 2012). The management of HF disease is quite complex and even

US
worse in developing countries where there are rare possibility of adequate diagnostic tools and

limited number of medical experts especially in some rural communities (Sameh et al., 2013).
AN
Therefore, accurately predicting the HF risk in patients/individuals would be essential for

reducing their associated risks of severe heart problems and increasing their safety and
M

productivity during daily life activities (Al-Shayea, 2011; Kumar, 2012).

ED

The conventional methods for the diagnosis of HF risks are mainly based on the analysis of

patients medical history, physical examination report, and review of relevant symptoms by a
PT

medical practitioner, which often lead to imprecise diagnosis and sometimes delay in diagnosis

outcome because of human errors (Yan et al., 2006; Vanisree & Singaraju, 2011). In an attempt
CE

to resolve these issues, medical decision support systems based on emerging computational
AC

methods such as support vector machine, k-nearest neighbor, decision tree, fuzzy logic, and

artificial neural network (ANN) have been proposed in previous studies for HF-risks prediction

(Samuel et al., 2013a; Anooj, 2012; Babaoglu et al., 2010; Shantakumar & Kumaraswamy, 2009;

Kemal & Seral, 2007). Among these computational approaches, ANN-based methods have been

widely adopted in medical diagnosis and other related fields due to their capability in handling

complex linear and non-linear problems (Olaniyi et al., 2015; Samuel et al., 2013b ; Babaoglu et
4
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

al., 2010). Mohd & Fadzilah developed a three phase model based on ANN technique to predict

HF diseases in angina, which achieved a prediction accuracy of 88.89% and could be easily

incorporated into hospital information systems (Mohd & Fadzilah, 2013). Resul et al. developed

an ANN ensemble based model for the diagnosis of heart disease by using the Statistical

Analysis System enterprise miner 5.2., and their method achieved 89.01% accuracy, 80.95%

T
sensitivity, and 95.91% specificity (Resul et al., 2009). Akhil et al. proposed a multilayer

IP
perception ANN driven by back propagation learning algorithm and feature selection technique

CR
to predict and rank HR risks in patients. Using data from Andhra Pradesh heart disease database,

their proposed method was better than the traditional classification techniques (Akhil et al,

US
2013). While these existing ANN-based decision support systems would be useful for HF risks

prediction, they suffer from a major limitation which is that the HF attributes are generally
AN
assumed to have equal contribution towards the diagnosis outcome. However, it has been

reported in a number of previous studies that HF attributes have a varying degree of

M

contributions to the overall diagnosis outcome, which is rarely considered in the existing ANN
ED

methods (Rahimi et al., 2014; Senni et al., 2013; Wedel et al, 2009). Thus, such ANN methods

may not actually reflect the diagnosis status of patients that may lead to inadequate treatment
PT

measures.

Considering various contributions of the different HF attributes in the ANN-based methods may
CE

improve the performance of HF-risks prediction. In order to assign a specific contribution to a set
AC

of competing alternatives (X1,..., Xn) such as HF attributes under a given criterion, the

framework of pairwise comparisons represented by a square matrix (n n) is generally used to

obtain a set of preference values for the alternatives. A number of multi-criteria decision making

techniques such as simple additive weighting (SAW), multi-attribute utility theory (MAUT),

decision-making trial and evaluation laboratory model (DEMATEL), analytic network process

(ANP), analytic hierarchy process (AHP), and Fuzzy_AHP, have been proposed for the
5
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

evaluation of preference values from a pairwise comparison matrix (Sangaiah et al, 2015a;

Sangaiah et al, 2015b; Velasquez & Hester, 2013; Wallenius et al., 2008). Among these

techniques, AHP has been widely applied due to its simplicity and scalability. In addition, it

allows decision makers to pair-wisely and easily compare variables by assigning weights to them

based on their significance (Sloane et al, 2002; Liberatore & Nydick, 2008). As a result, there

T
has been a growing interest in the medical applications of AHP technique to aid efficient

IP
diagnosis of patients in decision support platforms (Li et al., 2012a). Furthermore, AHP

CR
procedural simplicity, mathematical background, ability to assess qualitative and quantitative

factors, have made it a promising tool for resolving a wide range of decision problems in

US
different health related domains. For instance, Leandro et al. developed a home continuity care

system based on AHP technique to evaluate the effectiveness and efficacy of monitoring patients
AN
with congestive HF in their respective homes. They found that the system worked well due to

AHP contribution in quantifying decision-makers opinion (Leandro et al., 2009). However, to

M

the best of our knowledge, there is no method that exploits the individual contributive weight of
ED

HF attributes and considers such weight in predicting the HF risk in patients.

In this study, an integrated decision support system based on both ANN and Fuzzy_AHP is
PT

proposed with an attempt to efficiently predict HF risks in patients. The proposed method

involves two major sequential steps. Firstly, Fuzzy_AHP was used to properly rank and compute
CE

the local weights of HF attributes and the global weights for a set of given attributes, where the
AC

individual contribution of each attribute was determined by an experienced cardiac clinician.

Next, the global weights that represent the contributions of the attributes were applied to train an

ANN classifier for the prediction of HF risks in patients. The performance of the proposed

method (ANN and Fuzzy_AHP) was evaluated by using an online clinical dataset obtained from

the patients with a potential HF disease and compared with that of the previously proposed

methods by using different metrics. With considering various contributions of the different HF
6
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

attributes in the study, the proposed hybrid approach would have potential to improve the

performance of HF risk prediction in clinical application.

2. Methods and Materials

2.1 Dataset

In the study, the Cleveland dataset collected from an online data mining repository of the

T
University of California, Irvine (UCI) (Cleveland Heart Disease Dataset, 2016), was adopted to

IP
evaluate the performance of the proposed hybrid method in identifying HF risk in patients. The

CR
dataset contained 303 data samples of patients with some missing values and 76 attributes.

During the study, 297 input data samples (excluding patients data with missing values) with

US
their corresponding output (target) consisting of 13 key HF diagnosis attributes were extracted

and used. The output values were categorized into two classes denoted as absence (meaning HF
AN
is absent) and presence (meaning HF is present). The choice of the dataset was based on the fact

that it is one of the commonly used dataset for testing purpose as reported in previous studies
M

(Rajkumar & Reena, 2010; Bhatia et al., 2008; Detrano et al., 1989). The definitions and
ED

categories of the HF attributes are summarized in Table 1.

PT
CE
AC

7
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

Table 1
HF diagnosis attributes: Description and grading of the attributes alternatives.
S/N Attribute Description Attribute Code Alternatives Alternative Code Range
1 Young YNG < 33
Age (Years) Medium MED 34 40
AGE
Old OLD 41 52
Very Old VOLD >52
2 Sex Male M 1
SEX
Female F 0

T
3 Typical Angina TA 1
Atypical Angina ATA 2

IP
Chest Pain Type CPT
Non-angina Pain NAP 3
Asymptomatic ASY 4

CR
4 Low LOW <128
RBP Medium MED 128 - 142
Resting Blood Pressure
High HIGH 143 - 154
Very High VHIGH >154
5

Serum Cholesterol SCH

US Low
Medium
High
LOW
MED
HIGH
<188
189 - 217
218 - 281
AN
Very High VHIGH >281
6 True YES 1
Fasting Blood Sugar FBS
False NO 0
7 Resting Normal NOR 0
M

Electrocardiographic RES ST-T abnormal ST-AB 1

results
Hypertrophy HYPER 2
8 Low LOW <112
ED

Maximum Heart Rate

MHR Medium MED 112 152
Achieved
High HIGH >152
9 True YES 1
Exercise Induced Angina EIA
PT

False NO 0
10 Low LOW <1.5
Old peak OPK Risk RSK 1.5 - 2.55
CE

Terrible TER >2.55

11 Upsloping UPS 1
Peak Exercise Slope PES Flat FLT 2
Downsloping DWS 3
AC

12 Fluoroscopy-0 FL-0 0
Number of major Vessels Fluoroscopy-1 FL-1 1
VCA
Colored by fluoroscopy Fluoroscopy-2 FL-2 2
Fluoroscopy-3 FL-3 3
13 Normal NOR 3
Thallium Scan THA Fixed Defect FDE 6
Reversible Defect RDE 7

8
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

2.2 Proposed hybrid method

The proposed hybrid method based on ANN and Fuzzy_AHP techniques for HF risk prediction

had two sequential stages. In the first stage, the 13 HF attributes considered in this study as

shown in Table 1 were evaluated and ranked in order of their contribution (importance) towards

HF risk using Fuzzy_AHP technique. The application of the Fuzzy_AHP technique was

T
achieved in the following three steps:

IP
Step 1: The problem was represented using a hierarchical structure that shows the relationship

CR
between the goal (HF diagnosis), attributes (HF attributes) that contributes to the goal, and

attributes alternatives involved in the decision process (Fig. 1).

US
Heart failure diagnosis LEVEL 1
AN
M

AGE SEX CPT RBP SCH FBS RES MHR EIA OPK PES VCA THA LEVEL 2
ED

YNG M TA LOW LOW YES NOR LOW YES LOW UPS FL-0 NOR

MED F ATA MED MED NO HYPER MED NO RSK FLT FL-1 FDE LEVEL 3
PT

OLD NTP HIGH HIGH ST-TAB HIGH TER DWS FL-2 RDE

VOLD ASY VHIGH VHIGH FL-3

CE

Fig. 1. Hierarchical structure showing the goal (LEVEL 1), attributes (LEVEL 2), and attributes alternatives
(LEVEL 3)
AC

Step 2: Having decomposed the problem into goal, attributes, and alternatives, a pairwise

comparison of each attribute with respect to the others was carried out based on the experience

and judgments of medical practitioners. Then each attributes alternatives were represented by

using fuzzy triangular membership function prior to the pairwise comparison. The results

obtained from the pairwise comparison were afterwards expressed in semantic judgments and

9
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

converted to numerical values based on Saaty fundamental scale (Saaty, 1990), where

quantification of judgments ranges from 1 to 9 (Table 2).

Table 2
The AHP fundamental preference scale (Saaty, 1990)
AHP Scale of Importance Numeric Reciprocal
S/N
for comparison pair Rating (Decimal)
1 Extreme Importance 9 1/9 (0.111)

T
2 Very Strong to Extremely Importance 8 1/8 (0.125)
3 Very Strong Importance 7 1/7 (0.143)

IP
4 Strongly to very Strong 6 1/6 (0.167)
5 Strong Importance 5 1/5 (0.200)

CR
6 Moderately to Strong 4 1/4 (0.250)
7 Moderately Importance 3 1/3 (0.333)
8
9
Equally to Moderately
Equal Importance
US 2
1
1/2 (0.500)
1 (1.000)
AN
The procedure was represented by using a square matrix of preferences, where the number in

row i and column j of the matrix gives the importance of a certain attribute Ci in comparison to
M

another Cj, and the square matrix is shown in equation (1).

ED

[ ]
PT

[ ]
CE

Let X = [ ] (i, j = 1, 2 n) denote a square pairwise matrix, where gives the relative
AC

importance of the elements in row i and column j. Also, is represented by a quantified value

of judgment based on experiential knowledge of medical practitioners as reported in some

previous studies (Alonso & Lamata, 2006). This procedure is constrained by the conditions in

equation (2) (Vilas Boas, 2005).

10
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

If then ;

If (2)

Then .

Step 3: Thus far, the matrices of comparison have been obtained and then weights for each

T
criterion across the hierarchical levels in relation to the alternatives under consideration were

IP
computed using the eigenvector which is expressed in equation (3).

CR

[ ]

where
US
is the weight of the ith criteria and n is the number of criterion.

As earlier stated (Ferreira et al., 2014; Vilas Boas, 2005; Laininen & Hmlinen, 2003), the
AN
results of the eigenvector solution should be standardized. In this study, the standardization

process was achieved by computing the proportion of each attribute with respect to the sum of all
M

the attributes based on equation (4).

ED

where Y is the normalized eigenvector, and is the weight of the ith attribute (i = 1,2,..., n).
PT

This procedure leads to the ordering of the eigenvector with respect to the attribute priorities and
CE

was repeated until the difference between the normalized results of the last and current

computation is negligible (i.e., the difference should be 0.0001). Subsequently, the rank of the
AC

attributes based on their individual contribution towards HF diagnosis was obtained. Thereafter,

the weights of the attributes alternatives were determined and arranged in a matrix form. And

the values in each column of the matrix were multiplied by the rank of the corresponding

attribute to obtain the weights that were later used to train an ANN predictor for HF risk

prediction.

11
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

The second stage of the proposed hybrid method involved the development of the ANN predictor

whose architecture was conceptualized as shown in Fig. 2. A typical feed forward network model

based on backpropagation learning algorithm and scale conjugate gradient technique was built

for the prediction of HF risks. The network model consist of three layers namely input, hidden,

and the output layer (Fig. 2). The input layer had 13 nodes corresponding to the 13 HF attributes

T
that were fully connected to the nodes in the hidden layer. The 13 HF attributes were fed into the

IP
network through the input layer and then passed on to the hidden layer by multiplying the value

CR
of each attribute with their corresponding weight.

AGE SEX CPT RBP SCH FBS RES MHR EIA OPK PES VCA THA

US Input Layer
AN
M

Hidden Layer
ED
PT

Weighted Links
Output Layer
CE

Presence of Heart Failure Absence of Heart Failure

Fig. 2. Schematic diagram of the ANN predictor for HF risk prediction

AC

The nodes in the hidden layer were configured to process the incoming information by

computing the weighted sum and adding a bias ( ) as expressed in equation (5). It is important

to note that upon examining different number of nodes in hidden layer, a hidden layer with 10

nodes was considered in the study because it yielded a relatively stable performance.

12
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

where is the incoming data, is the bias, and is the weighted link between the nodes.

Thereafter, was transformed using a sigmoid transfer function presented in equation (6) and

the processed information was transferred to the output nodes to predict the status of patients

T
with respect to HF risk.

IP
CR
The nodes in the output layer represent the two possible classes of HF risk prediction: HF

US
presence and HF absence. And the network training process began with the initialization of the
AN
connection weights by choosing a set of random values. Subsequently, the data from the input

layer were transferred to the output layer with the aid of a backpropagation learning algorithm.
M

The back propagation algorithm was chosen because it allows the network to learn and store

large amounts of input-output mapping relations and also regulates the networks weight as well
ED

as threshold values in order to obtain minimum error (Li et al., 2012b). While training the

network, the calculated error was propagated backward and the weights were adjusted
PT

accordingly using equation (7).

CE
AC

where denotes the positive constant called the learning rate and E represents the error.

Based on the training error, all the weights were re-computed and updated, and then the output

values of the nodes in the output layer were re-evaluated. The process was repeated continuously

until the network converged to a point where the error obtained between the actual and target

output was relatively minimal. Furthermore, the error was determined using cross-entropy

measure whose formula is shown in equation (8). The cross-entropy measure was adopted
13
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

because it was previously identified as a better means of measuring a networks performance in

terms of error, especially when using ANN methods for classification tasks (Kline & Berardi,

2005; Nasr et al., 2002).

where CE is the cross entropy, t and y represent the target output and the actual output.

T
IP
2.3 Implementation and performance measure of the proposed method.

The proposed hybrid method (ANN and Fuzzy_AHP) was implemented with the aid of the

CR
Matrix Laboratory (MATLAB R2014a). For the purpose of comparison, a predictor based on

US
conventional ANN approach was also built to diagnose the HF risks in patients. The Microsoft

Excel (MS Excel 2010) and Statistical Package for Social Science (SPSS 21.0) were employed in
AN
the preprocessing of the 297 HF data samples considered in the study. The entire dataset had a

dimension of 13x297 that was partitioned into three subsets for training, testing, and validating,
M

respectively. With two output classes denoted as Class 1 (HF presence) and Class 2 (HF absence),

the corresponding target output data had a dimension of 2x297. According to the previous
ED

studies (Ehsani et al., 2016; Kalyan et al., 2014; Divide Data for Optimal Neural Network
PT

Training, 2016), 60% to 80% of the entire dataset are generally used as the training set, its 15%

to 25% as the validating set, and its 15% to 20% as the testing set. In the current study, 65%,
CE

20%, and 15% of the entire samples (297 samples) were chosen as the training set (193 samples),

the validating set (59 samples), and the testing set (45 samples), respectively.
AC

Three commonly used metrics, classification accuracy, receiving operating characteristic (ROC)

plot, and performance plot, were adopted in the study to test and compare the performance of the

proposed hybrid method with that of the conventional ANN approach. While the classification

accuracy presents the prediction accuracy to HF risks, the ROC analysis can be used to

independently determine an optimal network model from the cost context or the class

14
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

distribution. And the performance plot would graphically demonstrate the training, validating,

and testing errors of the ANN classifiers. To further quantitatively analyze the performance of

the proposed method, sensitivity (Recall), specificity, false positive rate, and false negative rate,

were also computed by using equations (9-12), respectively based on the testing results of the

classification accuracies.

T
IP
CR
US
AN
where TP (True positive) is the number of correctly classified patients with HF, FN (False

negative) is the number of patients without HF disease that were misclassified as having HF
M

disease, TN (True negative) denotes the number of patients without HF disease that were

correctly classified, and FP (False negative) represents the number of patients without HF that
ED

were misclassified as having HF.

PT
CE
AC

15
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

3. Results and Discussion

3.1 Results of HF attributes ranking based on Fuzzy_AHP

The values obtained after ranking the 13 HF diagnosis attributes and their respective alternatives

based on the contribution of each towards the final goal (HF diagnosis) based on Fuzzy_AHP

technique are presented in Table 3.

T
Table 3

IP
Summary of HF attributes ranking based on Fuzzy_AHP technique

Attribute Alternative weight (ranging from 1 4)

CR
S/N Attribute
Weight 1 2 3 4
1 AGE 0.0822 0.0082 0.0164 0.0247 0.0329
2 SEX 0.0287 0.0096 0.0191 - -
3 CPT 0.1333 0.0133 0.0267 0.0400 0.0533
4
5
6
7
RBP
SCH
FBS
RES
0.0645
0.0559
0.0531
0.0452
US
0.0065
0.0056
0.0177
0.0075
0.0129
0.0112
0.0354
0.0151
0.0194
0.0168
-
0.0226
0.0258
0.0224
-
-
AN
8 MHR 0.1235 0.0206 0.0412 0.0618 -
9 EIA 0.0696 0.0232 0.0464 - -
10 OPK 0.0997 0.0166 0.0332 0.0499 -
11 PES 0.0386 0.0064 0.0129 0.0193 -
12 VCA 0.0849 0.0085 0.0170 0.0255 0.0340
M

13 THA 0.1208 0.0201 0.0403 0.0604 -

From the ranking shown in Table 3, it could be seen that each HF diagnosis attribute has a local
ED

weight (Attribute weight) and their corresponding global weights (Alternative weight). For
PT

instance, AGE has a local weight of 0.0822 that is distributed across its four alternatives (YNG =

0.0082, MED = 0.0164, OLD = 0.0247, and VOLD = 0.0329) based on the level of significance
CE

of each of the alternative (Table 1 and Fig. 1). The summation of all the global weights for any

attribute must be equal to its local weight for an efficient distribution to be achieved (Local
AC

weight [AGE] = Global weight [YNG+MED+OLD+VOLD]). In addition, the summation of the

local weights across all attributes should be equal to 1.0 (Local weight of all attributes

[AGE+SEX+CPT++THA] = 1.0) (Saaty, 1990; Chatterjee, 2014) as shown in Table 3. A rank

of the HF attributes and their alternatives are shown in Fig. 3a and Fig. 3b, respectively.

16
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

0.14 0.07

0.12 0.06

Attribute alternative weights

0.10 0.05
Attribute weights

0.08 0.04

0.06 0.03

0.04 0.02

T
0.02 0.01

IP
0.00 0.00
AGE SEX CPT RBP SCH FBS RES MHR EIA OPK PES VCA THA AGE SEX CPT RBP SCH FBS RES MHR EIA OPK PES VCA THA

Heart failure diagnosis attributes Heart failure diagnosis attributes

CR
a) b)
Fig. 3. Results of Fuzzy_AHP ranking; a) HF diagnosis attribute ranking, b) Attributes
alternatives weights.

US
From Fig. 3a, it could be seen that chest pain type (CPT) contributes the most towards HF

prediction while maximum heart rate (MHR) is the next most significant contributor. On the
AN
other hand, sex (SEX) appears to be the least contributor while peak exercise slope (PES)

happens to be the next least contributor. The global weights obtained during Fuzzy_AHP
M

attribute ranking (as shown in the columns under Alternatives weights in Table 3b) were applied
ED

to the Cleveland dataset which was used to train an ANN model built based on the proposed

hybrid method. In order to avoid overfitting problem and improve the generalization capability
PT

of the ANN classifier, the training parameters such as number of hidden layers and epochs were
CE

carefully observed over several runs and the optimal parameter set was decided. These optimal

parameter set was also used to train the classifier that was built based on the conventional ANN
AC

approach. Furthermore, the two classifiers based on the proposed hybrid method and the

conventional ANN method were individually used to predict the HF status of patients and their

performances were observed by using the earlier described performance evaluation metrics,

respectively.

17
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

3.2 Classification performance of the proposed hybrid method

The classification accuracies of the proposed hybrid method in predicting the status of HF

patients are presented in Fig. 4. Fig. 4a shows the results obtained from the training stage by the

proposed method (ANN and Fuzzy_AHP) and Fig. 4b shows those obtained when the trained

ANN classifier was tested by the testing data set.

T
IP
CR
US
AN
M

Fig. 4. Results of HF risk prediction by the proposed hybrid method. a). Accuracy of the trained network

model; b). Accuracy of the tested network model

ED

From the test results shown in Fig. 4b, it could be seen that out of a total of 20 patients with

varying degree of HF disease (Class 1), the proposed hybrid method was able to correctly
PT

classify all the 20 patients. Meanwhile, out of a total of 25 patients without HF disease (Class 2),
CE

the proposed hybrid method was able to correctly classify 21 with 4 misclassified samples. For

comparison, the classification results obtained by the conventional ANN model are shown in Fig.
AC

5. It can be observed from Fig. 5b that out of 18 patients reported with HF disease (Class 1), 15

were correctly classified with three misclassified patients. Meanwhile, out of 27 patients without

any trace of HF disease (Class 2), 24 of them were correctly classified by the conventional ANN

method with three misclassified patients. Comparing the overall prediction performance of the

proposed hybrid method with that of the conventional ANN method, it could be seen that the

18
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

proposed method achieved an average prediction accuracy of 91.1% (Fig. 4b) across the two

categories of patients (class 1 and class 2) as against 86.7% (Fig. 5b) for the conventional ANN

method. This therefore suggests that the proposed hybrid method would outperform the

conventional ANN approach in predicting the status of HF patients.

T
IP
CR
US
AN

Fig. 5. Results of HF risk prediction by the conventional ANN method; a). Accuracy of the trained
M

network model; b). Accuracy of the tested network model

ED

3.3 Performance of the proposed hybrid method based on ROC plot

The ROC curve is a plot that shows the true positive rate versus the false positive rate as the
PT

threshold is varied. In a ROC curve, the perfect test would show the points in the upper-left

corner, with 100% sensitivity and 100% specificity. In such a case, the performance of a trained
CE

classifier would be considered as very good one and the classifier would yield good prediction

results (Beale et al., 2013). Hence, the ROC plot was used in this study to further examine the
AC

performance of the ANN classifiers in predicting the HF risks. Fig. 6 shows the ROC plots of the

proposed hybrid method and conventional ANN approach.

19
 0.2 Class 1 0.2 0.2 Class 1 0.2
Class 2 Class 2
ACCEPTED MANUSCRIPT
0 0 0 0
0 0.2 0.4 0.6 0.8 1 0 00.2 0.20.4 0.40.6 0.60.8 0.81 1 0 0.2 0.4
False Positive Rate False False
Positive Rate Rate
Positive False Posit
Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)
Test ROC
Training ROC All ROC
TestROC
Validation ROC All RO
Training ROC Validat
1 1 11
11 11

0.80.8 0.8 0.8

0.8 0.8 0.8
0.8
True Positive Rate

Rate

True Positive Rate

Rate
True Positive Rate

Positive Rate
True Positive Rate

Positive Rate
0.60.6 0.6
0.6 0.6
0.6 0.6
0.6

True Positive

True Positive
0.40.4 0.4 0.4
0.4 0.4 0.4
0.4

True
0.20.2 Class 1 0.2 0.2
0.2 0.2 Class 1 0.2
0.2

T
Class 2 Class 2

IP
0 0 00 0 0
0 0 0.2 0.4 0.6 0.8 1 00
0.2 0.4 0.6 0.8 1 00 0 00.2
0.2 0.2 0.4 0.4
0.20.4 0.6 0.6
0.40.6 0.60.80.8 0.8
0.81 1 11 00 0.2
0.2 0.4
0.4
False Positive Rate
False Positive Rate False
False Positive
False RateRate
Positive
Positive Rate False Po
a) False Positive Rate False Posit
b)

CR
Test ROCmeasure with the ROC plot. a). Proposed
Fig. 6 Performance All Test
ROC ROC method; b)
hybrid All
1 1 1 1
Conventional ANN method. Class 1 and Class 2 represent the patients with HF disease and
0.8 the ones without HF disease, respectively.

US 0.8 0.8 0.8

True Positive Rate

True Positive Rate

True Positive Rate

True Positive Rate

It could
0.6be observed from Fig. 6 that the ROC plots of the proposed method cover a larger area0.6
0.6 0.6
AN
between
0.4 the diagonal and the upper-left corner of the curves (Fig. 6a) than those of the0.4
0.4 0.4

conventional
0.2
ANN method (Fig. 6b), which suggests
0.2
that the proposed method would be better
0.2 0.2
M

than the conventional ANN method in the prediction of HF risks. Also, from the results
0 0 0 0
0 0.2 0.4 0.6 0.8 1 0 0 0.2 0.2 0.4 0.4 0.6 0.6 0.8 0.8 1 1 0 0.2 0.4
presented in False Positive Rate
Fig. 4b, a sensitivity of 100.0% and False
specificity of Positive
84.0%
False Rate
were
Positive Rate obtained for the False Po
ED

proposed hybrid method as against 83.3% and 88.9% for the conventional ANN approach (Fig.
PT

5b). A false positive rate of 16.0% and a false negative rate of 0.0% could be observed for the

proposed method (Fig. 4b) in comparison to the conventional ANN method with a false positive
CE

rate of 11.1% and a false negative rate of 16.7% (Fig. 5b).

AC

20
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

3.4 Performance plot of the proposed hybrid method

The performance plot is another important indicator that shows the training, validating, and

testing errors and can be used to estimate the reliability of a classifier. Fig. 7a and Fig. 7b show

the performance plots for the proposed hybrid method and conventional ANN approach,

respectively. It could be seen from Fig. 7b that the best validation performance for the proposed

T
hybrid method was achieved at an error rate of 0.35799 as against 0.45968 for the conventional

IP
ANN approach.

CR
Best Validation Performance is 0.35799 at epoch 7 Best Validation Performance is 0.44968 at epoch 4
1
10 0
Train 10 Train
Validation Validation
Test
Cross-Entropy (crossentropy)

Test
Cross-Entropy (crossentropy)

Best

US
Best

0
10
AN
M

-1
-1 10
10
0 2 4 6 8 10 12 0 1 2 3 4 5 6 7 8 9 10
13 Epochs 10 Epochs
a) b)
ED

Fig. 7. Performance plot; a). Proposed hybrid method; b). Conventional ANN method

In other words, the final error for the proposed hybrid method was relatively smaller than that for
PT

the conventional ANN method. Also, it could be observed from Fig. 7a that the testing set error

and validating set error had a similar characteristics at the 7th epoch which implies there was no
CE

significant overfitting unlike in Fig. 7b that has significant overfitting where the best validation
AC

performance occurred (at epoch 4). In summary, the obtained results shown in the performance

plot (Fig. 7) reveal that the proposed hybrid method would most likely perform better than the

conventional ANN approach. The performance plot of the proposed method was further

observed and compared with that of the conventional ANN approach based on the values of the

21
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

cross-entropy and percent error obtained for their training, validating, and testing phases as

shown in Table 4.

Table 4
Performance measure based on cross-entropy and percent error.
Conventional ANN diagnostic Proposed hybrid method
S/N Samples No. method
Cross-entropy Percent error Cross entropy Percent error
1 Training 193 6.51490e-1 20.72538e-0 6.61985e-1 18.65284e-0

T
2 Validation 59 9.44453e-1 18.64406e-0 1.08551e-1 15.25423e-0

IP
3 Testing 45 9.74461e-1 13.33333e-0 1.13366e-0 8.88888e-0
Mean SD 297 0.86 0.178 17.57% 3.81 0.63 0.51 14.27% 4.96

CR
Generally speaking, minimizing the cross-entropy and percent error values yields good

classification performance while zero values mean no error. It could be seen from Table 4 that

US
the proposed hybrid method had an average cross-entry value of 0.63 as against 0.86 for the

conventional ANN method. Similarly, an average percent error of 14.27% was recorded by the
AN
proposed method as against 17.57% for the conventional ANN approach. These results further
M

show that the proposed hybrid method would mostly likely be more efficient in predicting the

HF risks in patients than the conventional ANN method.

ED

In addition, the performance of the proposed hybrid method in terms of prediction accuracy was

also compared with that of some previously proposed methods that used similar dataset as shown
PT

in Table 5. The performance of all the compared methods was arranged in increasing order of
CE

classification accuracy in Table 5. We can see that the newly proposed method seems to have

better prediction performance with respect to others. Note that Sameh et al., 2013 used an ANN
AC

method for heart diseases diagnosis and obtained similar accuracy with the current study using

self-recorded heart sound signals of patients, instead of the commonly used online clinical

database such as University of California, Irvine Cleveland heart disease dataset. So it would not

be meaningful to compare the result of our method with that of Sameh et al. in Table 5. The

22
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

promising results obtained in this study suggest that our method could aid the development of

efficient clinical decision support system for HF risk prediction in patients.

Table 5
Prediction accuracy of the proposed method and results of other previously proposed methods.

S/N Authors Year proposed Method Accuracy

1 Anooj P.K. 2012 Weighted fuzzy rules 57.85%

T
2 Robert Detrano (Sahan et al., 2005) 2008 Logistic regression 77.00%

3 Newton Cheung (Das et al., 2009) 2001 Naive Bayes 81.48%

IP
4 Sahan et al. 2005 Attribute weighted artificial immune system 82.59%

CR
5 Polat et al. 2005 Artificial immune system 84.50%

6 Ozsen and Gunes 2009 Modified artificial immune system 87.43%

7 Das et al. 2009 Neural networks ensemble 89.01%

8 The proposed method* 2016

US ANN-Fuzzy_AHP 91.10%
AN
4. Conclusion

In spite of the recent technological advancements in the medical domain, the number of death
M

resulting from HF disease is still on the increase (McRae et al., 2016; Kumar et al., 2016; Bui et

al., 2014), and this could be attributed to inadequate diagnostic tools. As part effort to develop a
ED

clinical decision support platform for the accurate prediction of HF risks in patients, a hybrid
PT

approach based on both ANN and Fuzzy_AHP was proposed in this study. Unlike the

conventional ANN-based methods that assumed an identical risk contribution for all the
CE

attributes of HF, the newly proposed method considered each of the attributes as a different risk

contributor based on the medical knowledge of an experienced clinical expert. Using the UCI
AC

clinical dataset with a total of 297 samples obtained from potential HF patients, our results

demonstrated that the newly proposed method had a better performance in terms of prediction

accuracy, cross-entropy, ROC plot, and performance plot measures in comparison to the

conventional ANN approach. In addition, a comparative analysis of the prediction accuracy

between our proposed method and some previous studies also showed that our method could

23
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

achieve a higher performance in predicting HF risks. The improvement in prediction accuracy of

HF recorded by our method should be due to fact that the contribution of individual HF attributes

was considered in predicting the HF risks in patients. Thus with the results from the study, we

could say that the proposed hybrid method would be promising in developing a clinical decision

support system for the accurate prediction of HF risks in patients.

T
5. Limitations and future work

IP
While the proposed method could achieve higher prediction accuracy in identifying HF risks, the

CR
current study also suffers from some limitations. The prediction time of the proposed hybrid

method which could be considered as an important factor in clinical application remains

US
uninvestigated in the study. Despite the promising results recorded by the proposed hybrid
AN
method, automatic determination of the optimal sets of hidden layer nodes and their respective

weighted links still are a challenge. And as a result, the ANN classifiers in the study were trained
M

and re-trained until the performance of the networks were relatively stable. In the future studies,

the optimization technique such Ant colony or Genetic algorithm (Bhardwaj et al., 2016; Ldio et
ED

al., 2016 ) among others will be adopted to automatically generate the optimal set of hidden layer

nodes and the corresponding weighted links for the networks. Note that the performance of the
PT

proposed hybrid method seems satisfactory only based on a total of 297 data samples in the
CE

study. Clinical decision support systems of this kind would generally require extensive training

and testing with a larger number of samples consisting of data from all the possible categories of
AC

patients. Therefore in the future, the proposed method will be presented with a wide spectrum of

data samples pooled from all the possible categories of patients for both training and testing

purposes.

24
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

Acknowledgements

The work was supported in part by the National Key Basic Research Program of China

(2013CB329505), the National Natural Science Foundation of China under Grants (#91420301,

#61135004, #61203209, #51275101, and #61201114), the National High Technology Research

and Development Program of China (#2105AA042303), the Natural Science Foundation for

T
Distinguished Young Scholars of Guangdong Province, China (2014A030306029), the Special

IP
Support Program for Eminent Professionals of Guangdong Province, China (2015TQ01C399),

CR
and the Shenzhen Peacock Plan Grant (#KQCX2015033117354152). Lastly, I (O.W. Samuel)

sincerely appreciate the support of CAS-TWAS Presidents Fellowship to pursue a PhD degree

US
at the University of Chinese Academy of Sciences, Beijing, China.
AN
M
ED
PT
CE
AC

25
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

References
Akhil, M.J., Deekshatulu, B.L., & Priti, C. (2013). Classification of heart disease using artificial neural network and
feature subset selection. Global Journal of Computer Science and Technology Neural and Artificial
Intelligence, 13(3), 5-14.

Allen, L.A., Stevenson, L.W., Grady, K.L. et al. (2012). Decision making in advanced heart failure: a scientific
statement from the American Heart Association, Circulation, 125, 19281952.

Alonso, J., & Lamata, M. (2006). Consistency in the analytic hierarchy process: a new approach, International
Journal of Uncertainty. Fuzziness and Knowledge-Based Systems 14(4): 445459

T
Al-Shayea, Q.K. (2011). Artificial neural networks in medical diagnosis. International Journal of Computer

IP
Science, 8(2), 150-154.

Anooj, P.K. (2012). Clinical decision support system: Risk level prediction of heart disease using weighted fuzzy

CR
rules. Journal of Computer and Information Sciences, Elsevier, 24, 27-40.

Babaoglu, I., Findik, O., & lker, E. (2010). A comparison of feature selection models utilizing binary particle
swarm optimization and genetic algorithm in determining coronary artery disease using support vector machine.
Expert System with Applications, 37, 3177-3183.
US
Beale, M., Hagan, M., & Demuth, H. (2013). Neural network toolbox users guide, MATLAB, The Math Works.
AN
Bhatia, U., Prakash, P., & Pillai, G.N. (2008). SVM based decision support system for heart disease classification
with integer-coded genetic algorithm to select critical features. Proceedings of the World Congress on
Engineering and Computer Science, San Francisco, USA.
M

Bhardwaj, A., Tiwari , A., Bhardwaj, H., & Bhardwaj, Aditi. (2016). A genetically optimized neural network model
for multi-class classification. Expert Systems with Applications, Elsevier, 60, 211-221.
ED

Bui, A.L., Horwich, T.B., & Fonarow, G.C. (2014). Epidemiology and risk profile of heart failure. Nat Rev Cardiol,
8, 30-41.

Chatterjee B. (2014). Case study: Applying a Risk based decision making framework for outsourcing.
Pharmaceutical engineering, the official technical magazine of ISPE, vol. 34, no. 6, pp. 1-6.
PT

Cleveland Heart Disease Dataset available at http://archive.ics.uci.edu/ml/datasets/Heart+Disease Date last

accessed: 2016-01-05.
CE

Divide Data for Optimal Neural Network Training. Available at http://www.mathworks.com/help/nnet/ug/divide-

data-for-optimal-neural-network-training.html Date last accessed: 2016-01-11.
AC

Das, R., Turkoglu, I., & Sengur, A. (2009). Effective diagnosis of heart disease through neural networks ensembles.
Expert Systems with Applications, Elsevier, 36(4), 76757680.

Detrano, R., Janosi, A., Steinbrunn, W., Pfisterer, M., Schmid, J., et al. (1989). International application of a new
probability algorithm for the diagnosis of coronary artery disease. American Journal of Cardiology, 64,304-310.

Durairaj, M., & Sivagowry, S. (2014). A pragmatic approach of preprocessing the data set for heart disease
prediction. International Journal of Innovative Research in Computer and Communication Engineering,
2(11),6457- 6465.

Ehsani N., Fekete B.M., Vorosmarty C.J., & Tessler Z.D. (2016). A neural network based general reservoir
operation scheme. Stoch Environ Res Risk Assess, 30:11511166.
26
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

Ferreira, F. A. F., Santos, S.P., & Dias, V. M. C. (2014). An AHP-based approach to credit risk evaluation of
mortgage loans. International Journal of Strategic Property Management, 18(1): 3855.

Hui, Y., & Jonathan, M.G. (2015). A hybrid model for automatic identification of risk factors for heart disease.
Journal of Biomedical Informatics, 58, S171S182.

Kalyan K., Jakhia B., Lele R.D., Joshi M., & Chowdary A. (2014). Artificial Neural Network Application in the
Diagnosis of Disease Conditions with Liver Ultrasound Images. Advances in Bioinformatics, Volume 2014,
Article ID 708279, pp. 1-14

Kemal, P., & Seral, S. (2007). Automatic detection of heart disease using an artificial immune recognition system

T
(AIRS) with fuzzy resource allocation mechanism and K-NN (nearest neighbor) based weighting preprocessing.
Expert System with Applications, 32, 625-631.

IP
Kline D.M. & Berardi V.L. (2005). Revisiting squared-error and cross-entropy functions for training neural network

CR
classifiers. Neural Computing and Applications, 14(4), 310318.

Kumar, A.V.S. (2012). Diagnosis of heart diseases using fuzzy resolution mechanism. Journal of Artificial
Intelligence, 5(1), 47-55.

US
Kumar, M., Pachori, R.B., & Acharya, U.R. (2016). An Efficient Automated Technique for CAD Diagnosis Using
Flexible Analytic Wavelet Transform and Entropy Features Extracted from HRV signals. Expert Systems With
Applications, Elsevier, doi: 10.1016/j.eswa.2016.06.038
AN
Laininen, P. & Hmlinen, R. (2003). Analyzing AHP matrices by regression. European Journal of Operational
Research 148(3): 514524.

Leandro, P., Umberto, B., Paolo, M., Mario, S., & Marcello, B. (2009). AHP for health technology assessment, a
M

case study: Prioritizing care approaches for patients suffering from chronic heart failure. Proceedings of the
10th International Symposium on the Analytic Hierarchy/Network Process for Multi-criteria Decision Making,
1-9.
ED

Liberatore, M.J., & Nydick, R.L. (2008). The analytic hierarchy process in medical and health care decision making:
A literature review. European Journal of Operational Research, 189(1), 194207.
PT

Li, Q., Zhang, L., & Chu, H. (2012a). An AHP-based assessment model for clinical diagnosis and decision,
Springer-Verlag Berling Heidelberg, AICI 2012, LNAI 7530, pp. 271-278.

Li, J., Cheng, J.H., Shi, J.Y., & Huang, F. (2012b). Brief introduction of back propagation (BP) neural network
CE

algorithm and its improvement. Advances in Intelligent and Soft Computing, 2,553-558.

Ldio, M.L.C., Roberto C.L.O.,Mauro, R. (2016). Optimization of neural networks through grammatical evolution
and a genetic algorithm. Expert Systems with Applications, Elsevier,56, 368-384
AC

McRae, M.P. et al., (2016). Cardiac Score Card: A diagnostic multivariate index assay system for predicting a
spectrum of cardiovascular disease. Expert Systems With Applications, Elsevier, 54, 136-147.

Mohd, K.A., & Fadzilah, S. (2013). Utilization of an artificial neural network in the prediction of heart disease.
International Journal of Bio-science and Bio-technology, 5(4), 159-166.

Nasr G.E., Badr E.A., & Joun C. (2002). Cross entropy error function in neural networks: Forecasting gasoline
demand. FLAIRS-02 Proceedings, American Association for Artificial Intelligence, 381-384.

Olaniyi, E.O., Oyedotun, O.K., & Khashman, A. (2015). Heart diseases diagnosis using neural networks arbitration.
International Journal of Intelligent Systems and Applications, 12, 75 - 82.
27
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

Ozsen, S., & Gunes, S. (2009). Attribute weighting via genetic algorithms for attribute weighted artificial immune
system (AWAIS) and its application to heart disease and liver disorders problems. Expert Systems with
Applications, Elsevier, 36(1), 386392.

Polat, K., Sahan, S., Kodaz, H., & Gnes, S. (2005). A new classification method to diagnosis heart disease:
Supervised artificial immune system (AIRS). In Proceedings of the Turkish Symposium on Artificial
Intelligence and Neural Networks (TAINN).

Rahimi, K., Patel, A., & Macmahon, S. (2012). Two decades of research on innovative models of care delivery for
patients with heart failure: the end or just the beginning? Arch Iran Med., 15, 43945.

T
Rahimi, K., Bennett, D., Conrad, N., Williams, T.M., Basu, J., et al. (2014). Risk prediction in patients with heart
failure: a systematic review and analysis. JACC: Heart Failure, 2(5), 440-446.

IP
Rajkumar, A., & Reena, S. (2010). Diagnosis of heart disease using data mining algorithms. Global Journal of

CR
Computer Science and Technology, 10(10), 38-43.

Resul, D., Ibrahim, T., & Abdulkadir, S. (2009). Effective diagnosis of heart disease through neural networks
ensembles. Expert Systems with Application, 36(4), 7675-7680.

US
Saaty, T.L. (1990). How to make a decision: The analytic hierarchy process. European Journal of Operational
Research, 48(1), 9-26.
AN
Sahan, S., Polat, K., Kodaz, H., & Gunes, S. (2005). The medical applications of attributes weighted artificial
immune system (AWAIS): Diagnosis of heart and diabetes diseases. C. Jacob et al. (Eds.): ICARIS 2005, LNCS
3627, Springer-Verlag Berlin Heidelberg, 456-468.

Sameh, G., Adel, M., & Ali, A. (2013). Innovative artificial neural networks-based decision support system for heart
M

diseases diagnosis. Journal of Intelligent Learning Systems and Applications, 5(3), 176-183.

Samuel, O.W., Omisore, M.O., & Ojokoh, B.A. (2013a). A web based decision support system driven by fuzzy
ED

logic for the diagnosis of typhoid fever. Expert Systems with Applications, Elsevier, 40(10), 41644171.

Samuel, O.W. & Omisore, M.O. (2013b). Hybrid Intelligent System for the Diagnosis of Typhoid Fever. Journal of
Computer Engineering & Information Technology, 2(2), 1-9.
PT

Sangaiah, A.K., Subramaniam, P.R., & Zheng, X. (2015a). A combined fuzzy DEMATEL and fuzzy TOPSIS
approach for evaluating GSD project outcome factors. Neural Computing and Applications, 26(5), 1025-1040.
CE

Sangaiah, A.K., Gao, X.Z., Ramachandran, M., & Zheng, X. (2015b). A fuzzy DEMATEL approach based on
intuitionistic fuzzy information for evaluating knowledge transfer effectiveness in GSD projects. International
Journal of Innovative Computing and Applications, 6(3/4), 203215.
AC

Senni, M., Parrella, P., DeMaria, R., et al., (2013). Predicting heart failure outcome from cardiac and comorbid
conditions: the 3C-HF score. Int J Cardiol., 163,206-211.

Shantakumar, B.P., & Kumaraswamy, Y.S. (2009). Intelligent and effective heart attack prediction system using
data mining and artificial neural networks. European Journal of Scientific Research, 31(4), 642-656.

Sloane, E.B., Liberatore, M.J. & Nydick, R.L. (2002). Medical decision support using the analytic hierarchy process.
Journal of Healthcare Information Management, 16(4), 3843.

Vanisree, K., & Singaraju, J. (2011). Decision support system for congenital heart disease diagnosis based on signs
and symptoms using neural networks. International Journal of Computer Applications, 19(6), 6-12.

28
 ACCEPTED MANUSCRIPT

Second Revised Manuscript Submitted to Expert Systems With Applications (ESWA)

Velasquez, M., & Hester, P.T. (2013). An analysis of multi-criteria decision making methods. International Journal
of Operations Research, 10(2), 56-66.

Vilas Boas, C. (2005). Mtodo multicritrios de anlise de deciso (MMAD) para as decises relacionadas ao uso
mltiplo de reservatrios: analytic hierarchy process (AHP), in Pessoa, J. (Ed.). Simpsio brasileiro de recursos
hdricos, 6.

Wallenius, J., Dyer, J.S., Fishburn, P.C., Steuer, R.E., Zionts, S., & Deb, K. (2008). Multiple criteria decision
making, multi attribute utility theory: Recent accomplishments and what lies ahead. Management Science,
54(7), 1336-1349.

T
Wedel, H., McMurray, J.J., Lindberg, M., et al. (2009). Predictors of fatal and non-fatal outcomes in the Controlled
Rosuvastatin Multinational Trial in Heart Failure (CORONA): incremental value of Apo lipoprotein A-1, high-

IP
sensitivity C-reactive peptide and N-terminal pro B-type natriuretic peptide. Eur J Heart Fail., 11,281-291.

CR
Yan, H., Jiang, Y., Zheng, J., Pemg, C., & Li, Q. (2006). A multi-layer perceptron based medical decision support
system for heart disease diagnosis. Expert System with Applications, Elsevier, 30, 272-281.

US
AN
M
ED
PT
CE
AC

29