Original Research Article

CREDIT CARD FRAUD FEATURE SELECTION USING LINEAR DISCRIMINANT

ANALYSIS AND DEEP ARTIFICIAL NEURAL NETWORK FOR FRAUD
DETECTION

Abstract
Credit card scams are susceptible to exploitation and are frequently targeted due to their inherent
vulnerabilities. The proliferation of e-commerce and several other online platforms has led to an
expansion of online payment methods, thereby heightening the susceptibility to online fraudulent
activities. In response to the escalating incidence of fraudulent activities, scholars have turned to
various machine learning techniques as a means to identify and analyze instances of fraud within
the realm of online transactions. The primary objective of this study is to propose and implement
an innovative approach for detecting credit card fraudulent activities from the Kaggle European
cardholder dataset using a novel Deep Artificial Neural Network (DANN) by stacking multiple
hidden layers on top of each other. During the process, the Linear Discriminant Analysis (LDA)
algorithm and SMOTE-ENN were used for feature selection and data balancing respectively.
SMOTE-ENN is a hybrid sampling algorithm combining synthetic minority over-sampling
techniques and edited nearest neighbor. To analyze the performance of the DANN algorithm
various metrics, including precision, recall, f-measure, and accuracy were employed.
Experimental results demonstrated that our approach achieved an accuracy of 98%, a precision
of 100%, a recall of 98%, and also an f1-score of 99%.
1.0 Introduction
Technological improvements have brought about a substantial revolution in the banking
business, leading to the implementation of faster transaction processes that eliminate the need for
direct engagement with cashiers (Guerra and Castelli, 2021). This advancement has resulted in
the development of credit cards. These credit cards are designed to facilitate financial
transactions with the primary purpose of enabling various forms of transactions, including but
not restricted to fuel, groceries, televisions, travel costs, and retail spending. The primary
objective of this initiative is to cater to circumstances whereby prompt access to financial
resources is not readily available (Tiwari, Mehta, Sakhuja, Kumar, and Singh, 2021).
Credit cards have gained significant popularity in the realm of online banking, as they are
extensively utilized for conducting online transactions and engaging in electronic commerce
(Varmedja, Karanovic, Sladojevic, Arsenovic, Anderla, 2019). Nevertheless, the proliferation
and advancement of credit card utilization have resulted in the emergence of many
manifestations of fraudulent activities. The utilization of progressively intricate methods by
fraudulent individuals to carry out illicit transactions has led to substantial financial losses for
both cardholders and financial institutions (Varmedja et al., 2019). In contemporary times,
individuals engaged in fraudulent activities have gained increased ease in executing illicit
transactions through various means such as theft, phishing for credit card information, and the
creation of counterfeit cards that imitate genuine user behavior.
In the context of financial institutions, fraud can be defined as the unlawful exploitation of an
individual's confidential data, to engage in unauthorized transactions or remove funds from the
user's account (Tiwari et al., 2021). Credit card fraud can be categorized into two separate
manifestations: online and offline (Cherif, Badhib, Ammar, Alshehri, Kalkatawi, and Imine,
2022). Instances of internet fraud involve persons who participate in fraudulent operations by
carrying out false transactions associated with online purchases. In contrast, offline fraud refers
to the act of engaging in fraudulent activities by exploiting a stolen or lost credit card.
However, the growing adoption of neural networks has become imperative for card issuers and
banking services, necessitating the incorporation of artificial intelligence (AI) and deep learning
methodologies as AI is playing a significant role in the development of innovative approaches
for enhancing the efficacy of credit card fraud detection systems (Bashar, 2019). This is
accomplished by implementing measures that enhance the likelihood of approval, mitigate the
frequency of denied transactions, and enable the proactive surveillance of credit limits. The
credit card fraud detection procedure primarily entails the categorization of incoming
transactions into two separate groups: genuine transactions and fraudulent transactions (Carrasco
and Sicilia-Urbán, 2020). This categorization facilitates the recognition of potentially dubious
behaviours.
The issue of class imbalance in machine learning refers to a scenario in which the entire quantity
of data belonging to one class (referred to as the positive class) is significantly lower than the
total quantity of data belonging to another class (referred to as the negative class). The issue of
classifying unbalanced datasets has been extensively examined in numerous works (Makki,
Assaghir, Taher, Haque, Hacid, and Zeineddine, 2019). As a result, we propose the application
of the Synthetic Minority Over-Sampling Technique and the Edited Nearest Neighbors
(SMOTE-ENN) algorithm for data class balancing and hence modifying the artificial neural
network by incorporating supplementary layers to facilitate the extraction of features and the
classification of credit card transactions as either fraudulent or legitimate.
2.0 Literature Review
Prusti and Rath (2019) developed a fraud detection system that utilizes various machine learning
techniques, including Decision Tree (DT), k-Nearest Neighbors algorithm (KNN), Extreme
Learning Machine (ELM), Multilayer Perceptron (MLP), and Support Vector Machine (SVM),
to assess the effectiveness of fraud detection. Two web-based protocols, namely Simple Object
Access Protocol (SOAP) and Representational State Transfer (REST), were employed to
facilitate the effective flow of data across many heterogeneous systems. The researchers
conducted a comparative analysis of the outcomes of five distinct machine learning algorithms,
utilizing the accuracy metric as the basis for evaluation. The Support Vector Machine (SVM)
algorithm showed superior performance compared to other algorithms, achieving an accuracy
rate of 81.63%. However, the researchers' proposed hybrid system exhibited even higher
accuracy, reaching 82.58%.
Seera, Lim, Kumar, and Dhamotharan (2021) implemented an intelligent payment card fraud
detection system involving the utilization of a Genetic Algorithm (GA) for feature selection and
aggregation. The researchers employed various machine learning algorithms to assess the
efficacy of their proposed approach. The findings indicate that the GA-RF model reached an
accuracy rate of 77.95%, while the GA-ANN model acquired an accuracy rate of 81.82%.
Additionally, the GA-DT model attained an accuracy rate of 81.97%.
Kibria and Sevkli (2021) in their research applied a deep learning model for credit card approval
and then compared it with two Machine Learning techniques. The machine learning model used
by the authors for comparison was the Logistic Regression and Support Vector Machine on a
dataset obtained from the UCI machine learning repository. They evaluated the performance of
the models using accuracy score, precision, recall, and F1-measure. The result of their analysis
revealed that both the Support Vector Machine and the Logistic Regression achieved an accuracy
of 86.23% whereas the Deep Learning model emerged as the best-performing model of their
experiment with an accuracy score of 87.10%. Based on these results, the authors deduced that
the deep learning model performed slightly better than the other two models namely, the Logistic
Regression and Support Vector Machine
Khatri, Arora, and Agrawal (2020) undertook an analysis of the performance of machine
learning techniques in the context of credit card fraud detection. In this study, the authors
examined various machine learning (ML) methodologies, including Decision Trees (DT), k-
Nearest Neighbor (KNN), Logistic Regression (LR), Random Forest (RF), and Naive Bayes
(NB). To evaluate the efficacy of each machine learning technique, the researchers employed a
dataset characterized by a significant imbalance, which was derived from a pool of European
cardholders. One of the primary performance metrics included in the trials is precision, which
was acquired by each classifier. The results of the experiment indicate that the decision tree
(DT), k-nearest neighbours (KNN), logistic regression (LR), and random forest (RF) algorithms
achieved precision rates of 85.11%, 91.11%, 87.5%, and 89.77% respectively.
Bommala, Basha, Rajarao, and Sangeetha, (2022) in their study on an Innovative Model-Based
Approach for Credit Card Fraud Detection employed the viability of machine learning
techniques. The authors utilized the K-Nearest Neighbor as an intelligent approach that can
improve the detection of fraud in credit card transactions. The experimental result of the authors
gave an accuracy of 94%. The author further reported that the large percentage obtained was due
to inconsistency in the number of legal and illegal transactions.
3.0 Research Methodology
Credit card fraud (CCF) detection is a binary classification and transactions predicated and
classified as fraudulent transactions or legitimate transactions. This study has carried out a
variety of research activities; the first one is the collection of the dataset from the Kaggle
machine learning repository. The second one is data preprocessing to treat missing values,
perform feature selection using the Linear Discriminant Analysis (LDA) algorithm, data
balancing using SMOTE-ENN, and also scaling and encoding the data before feeding it to the
deep artificial neural network. In the third phase of the credit card fraud detection, some
evaluation metrics are chosen to measure the performance of the deep artificial neural network
classifiers on the European cardholder’s credit card dataset discussed and finally, the impact of
the FS method has been evaluated and results of the classifications approaches are compared.
The research methodology is shown in Figure 1.
 Figure 1: Research Methodology Framework
3.1 Dataset description
The dataset included in this study comprises credit card transactions conducted by European
cardholders during September in the year 2013. The dataset was obtained and analyzed as part of
a research partnership between Worldline and the Machine Learning Group of ULB (Université
Libre de Bruxelles) in the field of big data mining and fraud detection (Bertrand, Yann-Aël,
Liyun, Frederic, and Gianluca, 2019). The dataset provided encompasses a collection of
transactions spanning two days, comprising a total of 284,807 instances. Among these
transactions, there were 492 occurrences of fraudulent activity. The dataset exhibits a significant
class imbalance, with the positive class representing a mere 0.172% of the total transactions. The
dataset exclusively consists of numerical input variables that have been obtained by a Principal
Component Analysis (PCA) transformation. The principle components derived by PCA are
denoted as V1, V2, ..., and V28. It is important to note that the features 'Time' and 'Amount' have
not undergone any transformation using PCA. The 'Time' feature represents the duration in
seconds between each transaction and the initial transaction recorded in the dataset. The feature
'Amount' represents the monetary value of a transaction. This particular feature can be utilized in
scenarios that involve example-dependent cost-sensitive learning. The response variable, denoted
as 'Class', represents whether a fraudulent activity has occurred or not. It is assigned a value of 1
when fraud is detected and 0 when no fraud is detected. Figure 2 shows the imbalanced nature of
the adapted dataset.

Figure 2: imbalanced dataset

3.2 Data Preprocessing

3.2.1 Feature Selection
Linear Discriminant Analysis (LDA) is a reliable technique employed for feature selection in the
context of classification issues (Zhu, Gao, Yang, and Ye, 2022). The experimental results
indicate that this technique is particularly suitable for analyzing the credit fraud dataset due to its
high dimensionality. The primary objective of employing Linear Discriminant Analysis (LDA) is
to ascertain an optimal linear combination of features that enhances the differentiation between
separate classes, particularly in the domain of distinguishing fraudulent and non-fraudulent
transactions. The objective is to enhance the inter-class separation while concurrently reducing
the within-class variance. The selected feature based on the LDA algorithm is shown in Figure 3.

Figure 3: Selected Features Using LDA

As shown in Figure 2, from the credit card fraud dataset with 31 features, the LDA algorithm
selected 15 features considering that the k threshold was set to 15.
3.2.2 Data Balancing
Considering that the adapted dataset is highly imbalanced, we employed the viabilities of the
Synthetic Minority Over-Sampling Technique and the Edited Nearest Neighbors (SMOTE-ENN)
algorithm. The SMOTE-ENN method is a composite approach that integrates two strategies
commonly employed to address data imbalance in datasets. The Synthetic Minority Over-
Sampling Technique (SMOTE) is utilized to increase the representation of the minority class by
synthetic data generation (Yang, Wang, Sun, Zhai, Song, and Wang, 2022). Conversely, the
Edited Nearest Neighbors (ENN) approach is employed to enhance the dataset quality by
eliminating noisy samples from both the majority and minority classes. The utilization of these
two methodologies contributes to the enhancement of the machine learning model's performance
in the context of imbalanced datasets. Hence, the result of the SMOTE-ENN balancing
techniques is shown in Figure 4.

Figure 4: Balance Dataset Based on SMOTE-ENN

3.4 Proposed Algorithm
3.4.1 Artificial Neural Networks (ANNs) are intricate adaptive systems that possess the ability to
modify their internal structure in response to the information that is transmitted through them
(Bashar, 2019). The desired outcome of fraud classification is attained through the manipulation
of connection weights. Every link possesses a corresponding weight. Weight refers to a
numerical value that regulates the strength of the connection between two neurons. Artificial
neural networks (ANNs) possess the capacity to acquire and represent intricate and non-linear
relationships (Balakrishnan, Kathpalia, Saha, and Nagaraj, 2019). This attribute is of great
significance as several real-world scenarios involve complex and non-linear connections between
input and output variables. To enhance the performance of the ANN, the deep artificial neural
network (DANN) is designed by stacking multiple hidden layers on top of each other. Each
hidden layer consists of multiple neurons, and the ReLU (Rectified Linear Unit) activation
function is used to introduce non-linearity. The output layer has a single neuron with the sigmoid
activation function, making it suitable for binary classification problems. This structure allows
the neural network to capture and learn intricate patterns and complex relationships in the input
data. By adjusting the connection weights during the training process, the neural network can
iteratively fine-tune its internal structure to optimize its performance on the fraud classification
task. The goal is to improve the neural network's ability to accurately classify instances as either
fraudulent or non-fraudulent. Figure 5 shows the architecture of the DANN algorithm.

Figure 5: DANN algorithm.

3.5 Performance Evaluation Metric

For assessing the efficiency level of the proposed model, the following metrics were employed:
True Positive Ratio (TPR) or Recall (R): TPR or Recall for a certain class, is the number of
samples in the class that is correctly predicted, divided by the total number of samples in the
class.
TP
Recall= ………1
TP+ FN

Accuracy is quantified in percentage and pertains to the extent to which the instances are
predicted correctly. It is calculated as follows:
TP+TN
Accuracy= ………2
TP+TN + FP+ FN

Precision is represented by the ratio of the number of decisions that are considered correct.
Thus, precision can be calculated as:
TP
precision= ……3
TP+ FP
The area under the curve (AUC): measures the discriminative power of the models between
the class labels and is calculated using the equation:

AUC=
1
2(1+
TP
+
FP
TP+ FN FP+ FN
………4 )
4.0 Experimental Setup
The credit card fraud detection model was constructed using the Python programming language
within the Anaconda programming environment on a Windows operating system. The system
utilized an Intel Core I5 processor and 4GB of RAM (Random Access Memory). Throughout the
process of developing the model, various Python packages were installed, imported, and
employed. These packages include NumPy, which provides modules for performing numerical
and multi-dimensional vector operations, pandas, which facilitates the reading of phishing
datasets, and seaborn in conjunction with Matplotlib, which enables data manipulation and
graphical displays. It is noteworthy to mention that the utilization of the TensorFlow API was
based on the applicability within the domain of neural networks.
4.1 Parameter Setting
The table shows the respective parameter utilized and fined tuned to create the deep artificial
neural network algorithm.
Table 1: DANN parameters
Parameters Values
epoch 10
Batch size 100
Optimizer Adam
Activation function Relu, sigmoid
Loss function Binary cross entropy
Metric accuracy

4.2 Experimental Result

The results of the DANN algorithm on the credit fraud dataset are shown in Table 2. After the
conductance of feature selection and data balancing the DANN algorithm has an accuracy of
98%, a precision of 100%, a recall of 98%, and also an f1-score of 99%.
Table 2: Models (DANN and) performance evaluation report
S. No. Classification Evaluation Metrics for the Classification Models
Model Accuracy Precision Recall F1-Score
(%) (%) (%) (%)
2 DANN 98 100 98 99

To conduct a more comprehensive analysis of the DANN algorithm's efficacy on the modified
credit card fraud datasets, the Receiver Operating Characteristic (ROC) and confusion matrix
metrics were employed as a means of evaluating the model's performance. The ROC curve,
which visually represents the classification model's effectiveness in distinguishing between the
presence or absence of fraud, was utilized for this purpose. Hence, to assess the efficacy of the
model, the Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC) was
utilized. The evaluation of the model, based on the AUC, yielded an accuracy score of 94%, as
depicted in Figure 6. The confusion matrix allows us to visually inspect how well the model
performs in terms of correctly and incorrectly predicting each class, the result of the confusion
matrix is shown in Figure 7

Figure 6: DANN Receiver Operating Characteristic (ROC)

 Figure 7: DANN Confusion Matrix

4.3 Result Comparison

Table 3: Comparison with other algorithms
Authors Algorithm Used Best Accuracy
Algorithm
Prusti and Rath DT, KNN), ELM, MLP), and Hybrid 82.58
(2019) SVM,
Seera et al., (2021) GA-RF, GA-ANN, and GA- GA-DT 81.97
DT
Kibria and Sevkli DNN, SVM and LR DNN 87.10
(2021)
Bommala et al., KNN KNN 94
(2022)
Current Study DANN DANN 98

KEYS: Decision Trees (DT), k-Nearest Neighbor (KNN), Logistic Regression (LR), Random Forest (RF), Deep
Neural Network (PNN), Extreme Learning Machine (ELM), Multilayer Perceptron (MLP), Support Vector Machine
(SVM), Genetic Algorithm - Decision Tree (GA-DT), Genetic Algorithm – Random Forest (GA-RF), Genetic
Algorithm – Artificial Neural Network (GA-ANN)

Table 3 presents a comparison of the latest algorithms for the DANN algorithms, along with four
other methods employed by various academics. The table comprises five columns to compare the
state-of-the-art, which are authors, the algorithm implemented, the optimal algorithm selected by
each author, and the corresponding scores in the final column. Based on the findings presented in
Table 4, it is evident that all algorithms assessed in this study demonstrate a satisfactory level of
accuracy. Notably, the stacked DANN model, which was trained specifically for this research,
displayed the highest level of performance, obtaining an accuracy rate of 98%. Figure 8 shows
the graphical result comparison of the DANN algorithm with some other author's works

Figure 8: DANN Comparison With Other Authors

5.0 Conclusion
In this paper, we present a deep artificial neural network on an imbalanced European cardholder
dataset for the categorization of fraudulent and non-fraudulent transactions. In our DANN model,
we first propose an oversampling and undersampling strategy using SMOTE-ENN with LDA as
a feature selection methodology before feeding the selected and scaled feature to the DANN
algorithm. To measure the performance of the DANN model, the accuracy score, precision,
recall, and f1- were applied. Finally, we analyze the suggested technique using the balanced
European cardholder dataset. The results reveal that the DANN algorithm achieved an accuracy
of 98%, a precision of 100%, a recall of 98%, and also a f1-score of 99%.
Reference
Balakrishnan, H. N., Kathpalia, A., Saha, S., & Nagaraj, N. (2019). ChaosNet: A chaos-based
artificial neural network architecture for classification. Chaos: An Interdisciplinary
Journal of Nonlinear Science, 29(11).
Bashar, D. A. (2019). Survey on evolving deep learning neural network architectures. Journal of
Artificial Intelligence and Capsule Networks, 1(2), 73-82.
Bertrand, L., Yann-Aël, L., Liyun, H., Frederic, O., and Gianluca, B. (2019). Deep-Learning
Domain Adaptation Techniques for Credit Cards Fraud Detection, INNSBDDL 2019:
Recent Advances in Big Data and Deep Learning, pp 78-88, 2019
Bommala, H., Basha, R.M., Rajarao, B., and Sangeetha, K. (2022). An Innovative Model-Based
Approach for Credit Card Fraud Detection Using K-Nearest. https://doi.org/10.1007/978-
981-16-7389-4_19
Carrasco, R. S. M., & Sicilia-Urbán, M. Á. (2020). Evaluation of deep neural networks for
reduction of credit card fraud alerts. IEEE Access, 8, 186421-186432
Cherif, A., Badhib, A., Ammar, H., Alshehri, S., Kalkatawi, M., & Imine, A. (2022). Credit card
fraud detection in the era of disruptive technologies: A systematic review. Journal of
King Saud University-Computer and Information Sciences.
Guerra, P., & Castelli, M. (2021). Machine learning applied to banking supervision a literature
review. Risks, 9(7), 136.
Khatri, S., Arora, A., and Agrawal, AP (2020). Supervised machine learning algorithms for
credit card fraud detection: a comparison. In: 10th international conference on cloud
computing, data science & engineering (Confluence), 680-683.
Kibria, M. G., & Sevkli, M. (2021). Application of deep learning for credit card approval: A
comparison with two machine learning techniques. International Journal of Machine
Learning and Computing, 11(4), 286-290.
Makki, S., Assaghir, Z., Taher, Y., Haque, R., Hacid, M. S., & Zeineddine, H. (2019). An
experimental study with imbalanced classification approaches for credit card fraud
detection. IEEE Access, 7, 93010-93022.
Prusti, D., & Rath, S. K. (2019). Web service-based credit card fraud detection by applying
machine learning techniques. In TENCON 2019-2019 IEEE Region 10 Conference
(TENCON), 492-497.
Seera, M., Lim, C.P., Kumar, A., Dhamotharan, L., Tan, K.H. (2021) An intelligent payment
card fraud detection system. Ann Oper Res, 2021;1–23.
Tiwari, P., Mehta, S., Sakhuja, N., Kumar, J., & Singh, A. K. (2021). Credit card fraud detection
using machine learning: a study. arXiv preprint arXiv:2108.10005.
Varmedja, D., Karanovic, M., Sladojevic, S., Arsenovic, M., & Anderla, A. (2019). Credit card
fraud detection-machine learning methods. IEEE, 1-5.
Yang, F., Wang, K., Sun, L., Zhai, M., Song, J., & Wang, H. (2022). A hybrid sampling
algorithm combining synthetic minority over-sampling technique and edited nearest
 neighbor for missed abortion diagnosis. BMC Medical Informatics and Decision
Making, 22(1), 344.
Zhu, F., Gao, J., Yang, J., & Ye, N. (2022). Neighborhood linear discriminant analysis. Pattern
Recognition, 123, 108422.