Journal of Computer Virology and Hacking Techniques (2023) 19:217–239

https://doi.org/10.1007/s11416-022-00441-2

ORIGINAL PAPER

XAI for intrusion detection system: comparing explanations based on

global and local scope
Swetha Hariharan1 · R. R. Rejimol Robinson2 · Rendhir R. Prasad3 · Ciza Thomas4 · N. Balakrishnan1

Received: 6 January 2022 / Accepted: 14 June 2022 / Published online: 31 July 2022
© The Author(s), under exclusive licence to Springer-Verlag France SAS, part of Springer Nature 2022

Abstract
Intrusion Detection System is a device or software in the field of cybersecurity that has become an essential tool in computer
networks to provide a secured network environment. Machine Learning based IDS offers a self-learning solution and provides
better performance when compared to traditional IDS. As the predictive performance of IDS is based on conflicting criteria,
the underlying algorithms are becoming more complex and hence, less transparent. Explainable Artificial Intelligence is a
set of frameworks that help to develop interpretable and inclusive machine learning models. In this paper, we use Permu-
tation Importance, SHapley Additive exPlanation, Local Interpretable Model-Agnostic Explanation algorithms, Contextual
Importance and Utility algorithms, covering both global and local scope of explanation to IDSs on Random Forest, eXtreme
Gradient Boosting and Light Gradient Boosting machine learning models along with a comparison of explanations in terms
of accuracy, consistency and stability. This comparison can help cyber security personnel to have a better understanding of the
predictions of cyber-attacks in the network traffic. A case study focusing on DoS attack variants shows some useful insights
on the impact of features in prediction performance.

Keywords Intrusion detection system · RF · XGBoost · LightGBM · XAI · SHAP · LIME · Permutation importance ·
Contextual importance and utility

1 Introduction

The Internet on which we heavily depend for almost every-

thing is prone to various network-based attacks. Among the
B Ciza Thomas various systems designed to counter these attacks, Intrusion
cizathomas@gmail.com Detection Systems (IDS) aid the network by providing a
Swetha Hariharan second layer of defence against both external and internal
swethahariharan1810@gmail.com attacks. Accuracy is the key indicator of intrusion detection,
R. R. Rejimol Robinson while nuisance alerts or misdetection completely undermine
ashniya@gmail.com the security of the system [1]. Machine Learning (ML) algo-
Rendhir R. Prasad rithms used in IDSs identify the patterns present in network
rendhirrprasad@gmail.com traffic and detect anomalies as attacks. The broad categories
N. Balakrishnan of supervised and unsupervised algorithms comprise of Arti-
balki@iisc.ac.in ficial Neural Networks, SVM, Decision Trees, K-Nearest
neighbour, Random Forest (RF), eXtreme Gradient Boosting
1 Supercomputer Education and Research Center, Indian (XGBoost), Light Gradient Boosting Machine (Light GBM),
Institute of Science, Bangalore, Karnataka, India
Deep Learning (DL) etc., and K-Means clustering algorithms
2 SCT College of Engineering, Thiruvananthapuram, Kerala, respectively. These machine intelligence can categorize traf-
India
fic into normal traffic and attack traffic. Modern IDSs are
3 Government Engineering College, Barton Hill, equipped with deep learning (DL) techniques that learn from
Thiruvananthapuram, Kerala, India
patterns and discover novel attack patterns present in the net-
4 Directorate of Technical Education, Government of Kerala, work traffic.
Thiruvananthapuram, Kerala, India

123
218
Fig. 1 Overview of the scope of explainability methods
The complex machine learning models like RF, XGBoost,
Light GBM, and DL are observed to provide superior per-
formance over the traditional ML techniques. The more
sophisticated the system is, the more is the complexity, and
complex systems are less interpretative. These models are
referred to as “black box” in ML, as even their designers
cannot explain why the AI arrived at a specific decision.
The underlying mathematical principles of such a model lack
explicit declarative knowledge. We want to understand how
decisions are being made so that we can trust the decisions
made by these critical AI systems. However, there is always
a trade-off between predictive performance and transparency
of the systems [2].
The workshop titled “Implications of artificial intelli-
gence for Cybersecurity” explores certain issues, significant
concerns, and interests, among policymakers, security prac-
titioners, technologists, researchers, and the public about the
potential implications of AI and ML for cybersecurity [3].
The workshop concludes that the machine learning algo-
rithms that are applied presently can address only isolated
characteristics of network security. In addition, it brings
out the necessity for transparent and trustworthy systems.
Explainable Artificial Intelligence (XAI) is an emerging field
that gives importance to machine learning explainability.
In this work, we build models like RF, XGBoost, and
LightGBM for detecting anomalous activity in the network
traffic, with high prediction accuracy. Along with the intru-
sion detection, it is necessary to provide explanations as
to what influences the decision in the cybersecurity con-
text. This work is intended to solve the issues of ambiguity
in network intrusion detection by additionally providing an
explainer framework that can be implemented to explain the
predictions given by these less interpretive models. All the
methods implemented in this work improve the explainabil-
ity of the models as post-hoc explainability methods. These
methods cover local as well as global explanations of the
123
S. Hariharan et al.
models. Figure 1 represents the overview of the scope of
explainability methods used on these models.
Permutation Importance (PI), SHapley Additive exPlana-
tion (SHAP), Local Interpretable Model-Agnostic Explana-
tion (LIME), and Contextual Importance and Utility algo-
rithms CIU) are employed to establish the transparency of
the model as desired by the system analysts. These explain-
able methods can be compared based on certain measures
namely, accuracy, consistency and stability. Accuracy deter-
mines how well an explanation can predict unseen data.
Explanations that differ between models can be captured by
consistency measures. Stability can point out how similar are
the explanations for similar samples. The Global explanation
provides feature importance, using which predictive accu-
racy of the explanation is compared. The local explanation is
evaluated for consistency by comparing explanations of sim-
ilar instances for two different models to indicate whether
the models are stable or not.
This paper addresses the following research questions:
• How can the decisions taken by high performing tree-
based or tree boosting ML algorithms be explained using
the appropriate XAI methods?
• How can the different XAI methods be compared and
bench marked using various criteria?
Accordingly, the research contributions of this paper are as
follows:
• Explainable AI methods are used to explain the deci-
sions made by the tree-based classifier models of Random
forest and tree boosting algorithms of extreme gradient
boosting and light gradient boosting algorithms which are
applied on security data sets of NSL-KDD and Kaggle.
• Compared and benchmarked the current and popular
model-agnostic explanation methods from both global
and local scope.
• Global Model Agnostic explanation methods of Permu-
tation Importance and SHapley Additive exPlanation are
examined to obtain interpretability of prediction, specif-
ically from tree-based classifier model.
• Local Model Agnostic methods of Local Interpretable
Model-Agnostic Explanation (LIME) and Contextual
Importance and Utility algorithms (CIU) methods are
used to obtain interpretability of prediction specifically
from tree-based classifier model.
• A case study is done with NSL-KDD DoS attack variants
and studied the impact of features in prediction perfor-
mance.
• Investigated the algorithms and illustrated the efficiency
and quality of explanation provided by comparing them
on Global and Local scope by using accuracy, stability
and consistency as criteria.
XAI for intrusion detection system: comparing explanations based on global and local scope
The rest of the paper is organized as follows. Section 2 pro-
vides the literature review; Sect. 3 provides an exploratory
analysis of the dataset, Sect. 4 describes the explanation
methods used, Sect. 5 provides the details of the experiments
and discussion of results, and Sect. 6 is a case study on NSL-
KDD DoS attack variants and Sect. 7 is the discussion of
results obtained. Sect. 8 concludes the paper.
2 Literature review
In this section, a background study on the research progress in
IDS-based security using artificial intelligence and machine
learning is undertaken. The other areas covered in this lit-
erature review is the Explainable AI of models built for
anomaly-based IDSs, which is vital for improving practical
deployment of AI-based solutions.
2.1 AI and machine learning for IDS
A variety of machine learning algorithms are experimented
in literature to demonstrate the effectiveness of the model
in detecting attacks, and these models are built on net-
work traffic data available in different formats. Some of the
methodologies make use of feature selection algorithms to
reduce the complexity of the model. Preprocessing methods
like oversampling and undersampling are also employed to
combat the issues of a skewed distribution of attacks and
benign data. Othman et al. [4] investigate the importance of
information security and data analysis systems for Big Data,
as the IDS has to handle huge amounts of data. Their work
demonstrates intrusion detection model using support vector
machine (SVM) classifier on Apache Spark Big Data plat-
form and the results of the experiment show that this model
has high performance, reduces the training time and is effi-
cient for Big Data [4]. Security in the context of IoT is also
a major concern and the work of Costa et al. is a survey of
machine learning algorithms with a focus on the detection of
intrusions [5].
Hodo et al. use an Artificial Neural Network (ANN) model
to implement a threat analysis on IoT using network packet
traces [6]. Peng et al. propose a decision tree-based approach
and compare the detection time of the model. The experimen-
tal results demonstrate an effective model performance [7].
Zhang et al. propose a model for binary classification using
Support Vector Machine (SVM) with some text processing
techniques employed based on the characterization of the
frequencies of the system calls executed by the privileged
programs. 1998 DARPA BSM data set collected at MIT’s
Lincoln Labs were used for the experimentation [8].
The network security community is now using deep learn-
ing methods to combat ever-evolving attacks. Explanations
219
supporting these black box models have become a necessity
because of certain reasons. Explainable results ensure data-
driven decision-making and explainable reports facilitate in
improving the robustness of the model. Interpretability can
guarantee that only contextually correct variables infer the
output. The need for transparency in the new technology,
like AI in health care, is discussed in the work of Sharma
et al. [9]. In machine learning, these black box models are
created directly from data by an algorithm. Hence, it is very
difficult to understand how variables are being combined to
make predictions, even if one has a list of the input variables
[10].
Focus is on dependency of accuracy by selecting different
attack grouping options and influence of initial preprocessing
on accuracy by using Decision trees, Naive Bayes and Rule-
Based classifiers with NSL-KDD dataset [11]. Deshmukh
and Padiya [12] describe a set of data preprocessing activ-
ities including feature selection and discretization by using
Naive Bayes, NBTree and Hidden Naive Bayes on NSL-
KDD dataset.
2.2 Transparency of models
Transparency is one of the properties that can enable inter-
pretability [13]. Model transparency heavily depends on
the predictive performance and transparency trade off. The
explainability of the model can be increased by imposing
different constraints such as monotonicity, model size and
sparsity [14]. Even the choice of a model family like the
tree based model covering Decision Tree, Random Forest,
XGBoost etc can be considered a constraint on models that
affects the interpretability. SHapley Additive exPlanations
(SHAP) is a model agnostic post-hoc explainablility method
proposed by Lundberg and Lee [15]. SHAP is based on coali-
tion concept of game theory, a unified framework for value
estimation of additive feature attributions that generalizes
several works from the literature [16]. Both model-agnostic
and more efficient model-specific variants are proposed.
LIME [17] focuses on providing local explanations to a
specific instance. Visual diagnosis of LIME explanation is
discussed in the work of Goode and Hofmann [18]. Evalua-
tion of interpretability based on different grounds is done in
the work of Velez and Kim [19]. Visual explanation based on
dependencies of the input features is proposed in available
literature [20–22]. AI Explainability 360 [23], a toolkit with
eight XAI algorithms, provides a complete evaluation of the
explanations. Wang et al. [24] propose a framework to pro-
vide a better understanding and transparency of NSL-KDD
dataset using SHAP XAI Method which helps cybersecurity
experts to make good judgement.
123
220 S. Hariharan et al.

Table 1 Dataset Split of Kaggle

Dataset Classification type Total Train Test Target
and NSL-KDD dataset
Kaggle Binary 25192 20154 5038 Normal, Anomaly
NSL-KDD Binary 148517 125973 22544 Normal,Anomaly
NSL-KDD Multiclass 148517 125973 22544 Normal, DOS, Probe, R2L, U2R

Table 2 Feature description related to NSL-KDD dataset

Type Feature names

Binary features land, logged_in, root_shell, su_attempted, is_hot_login, is_guest_login

Continuous features duration, src_bytes, dst_bytes, wrong_fragment, urgent, hot, num_failed_logins, num_compromised, num_root,
num_file_creations, num_shell , num_access_files, num_outbound_cmds, count, srv_count, serror_rate, srv_serror_rate,
rerror_rate, srv_rerror_rate, same_srv_rate, diff_srv_rate, srv_diff_host_rate, dst_host_count, dst_host_srv_count,
dst_host_same_srv_rate, dst_host_diff_srv_rate, dst_host_same_src_port_rate, dst_host_srv_diff_host_rate,
dst_host_serror_rate, dst_host_srv_serror_rate, dst_host_rerror_rate, dst_host_srv_rerror_rate
Symbolic features protocol type: tcp , udp , icmp Flag:Oth , REJ, RSTO, RSTOS0, S0 .S1 . S2, S3, SF, SH Service: aol, auth, bgp, courier,
csnet_ns, ctf, daytime, discard, domain, domain_u, echo, eco_I, scr_I, efs, exec, finger, Ftp, Ftp_data , Gopher, Harvest
Hostnames. Http, Http_2784, Http_443, Http_8001, Imap4, Irc, Iso_tsap, Klogin, Kshell, Idap, Link, Login, Name,
Netbios_dgm, Netbios_ns, Netbios_ssn, Netstat, Nnsp, Nntp, Ntp_u, Pm_dump, Pop_2, Pop_3, Printer, Private, Red_I,
Mtp, Remote_job, Uucp, Uucp_path, Sql_net, Ssh, Sunrpc, Supdmp, Systat, Telnet, Tftp_u, Tim_I, Time, Whois, Smtp,
Urh_I, Urp_I, Rje, Shell, Vmnet, X11. Z39_50

3 Exploratory data analysis
Two data sets namely, the public network intrusion dataset
available on the Kaggle site [25] and the NSL-KDD [26]
dataset are used for experimentation. Kaggle IDS dataset
provides a wide variety of intrusions simulated in a military
network environment. It creates an environment by simulat-
ing US Air Force LAN and acquiring raw TCP/IP dump data.
The LAN was blasted with multiple attacks. A connection
consists of TCP packets starting and ending time duration,
source IP address and target IP address under some well-
defined protocol. Each connection is classified and named
either as normal or attack. NSL-KDD dataset is the bench-
mark for modern-day Internet traffic. Both Kaggle IDS and
NSL-KDD data set contain 42 features per record, with 41
of the features referring to the traffic input, and the last one
being the label.
The 41 features are classified as: 3 Symbolic features, 6
Binary features, and 32 Continuous features
• Symbolic features: One hot encoder is used to convert
categorical values to binary.
• Binary features: The binary values remain unchanged.
• Continuous features: Min-Max normalization was used
to scale the continuous values in the [0-1] range.
Table 1 represents the splitting of Kaggle and NSL-KDD
dataset. NSL-KDD dataset is used as a binary classification
as well as a multi classification problem. After preprocessing
the input feature increased from 41 to 122 features [24] for
multiclass recognition and the details are given in Table 2.
We have used google facets [27], an open-source visu-
alization tool, to understand and analyse datasets. Facets
consist of two visualization techniques:
• Facet’s dive – allows exploring a set of observations in
the dataset
• Facets Overview – provides an overview and understand-
ing of the distribution of the various values across the
features of the dataset.
Facets helps in identifying common data issues such as
unexpected feature values, features with high percentages of
missing values, and features with unbalanced distribution.
Table 3 shows the facets overview visualization of 7 con-
tinuous features. The number highlighted in bold indicates
possible trouble with the values, in this case, a numeric fea-
ture with a high percentage of values set to ‘0’ is highlighted.
Table 4 shows the percent of attack distribution in both
training and test dataset. The training dataset is made up of 21
different attacks out of the 37 present in the test dataset. The
attacks in the training dataset belonging to different attack
families are:
• DOS: back, land, Neptune, pod, smurf, teardrop
• Probe: ipsweep, nmap, portsweep, satan
• R2L: Spy, wareclient, ftp-write, guesspasswd, imap, mul-
tihop, phf, warezmaster
• U2R: bufferflow, loadmodule, perl, rootkit
The attacks in the test dataset belonging to different attack
families are:

123
XAI for intrusion detection system: comparing explanations based on global and local scope 221

Table 3 Facets overview

Numeric features (7)
visualization of seven numeric
Features Count Missing Mean Std dev Zeros Min Median Max
features
duration 126k 0% 287.14 2,604.52 92.05% 0 0 42.9k
src_bytes 126k 0% 45.6k 5.87M 39.21% 0 44 1.38B
dst_bytes 126k 0% 19.8k 4.02M 53.95% 0 0 1.31B
land 126k 0% 0 0.01 99.98% 0 0 1
wrong_fragment 126k 0% 0.02 0.25 99.13% 0 0 3
urgent 126k 0% 0 0.01 99.99% 0 0 3
hot 126k 0% 0.2 2.15 97.88% 0 0 77

Table 4 NSL-KDD attack distribution on dataset Algorithm 1: Compute Permutation Importance of fea-
Training dataset Testing dataset ture k
Class Distribution Class Distribution Require: Input: m - Trained Model, y - Target Vector, X - Feature
Matrix
Normal 67343 Normal 9711 Ensure: Estimate the original model error Ori = E(y, m(X ))
DOS 45927 DOS 7458
Probe 11656 Probe 2421 a. Generate feature matrix X per m by permuting feature K
in the data X . This breaks the association between feature k
R2L 995 R2L 2754 and true outcome y.
U2R 52 U2R 200 b. Estimate error Per m = E(Y , m(X per m ) based on the
Total 125973 Total 22544 predictions of the permuted data.
c. Calculate permutation feature importance
P I k = Per m/Ori.Alternatively, the difference can be used:
P I k = Per m − Ori. We are using P I k = Per m/Ori
• DOS: apache2, back, land, Mailbomb, Neptune, pod, pro-
Sort features by descending PI
cesstable, smurf, teardrop, udpstrom
• Probe: ipsweep, mscan, nmap, portsweep, saint, satan
• R2L: Spy, ftp-write, guesspasswd, httptunnel, imap,
multihop, named, phf, sendmail, snmpgetattack, warez-
It is based on the concept that randomly shuffling a single
master, xlock, xsnoop
column should cause a loss of accurate predictions since the
• U2R: bufferflow, loadmodule, perl, ps, rootkit, snmpguess,
resulting data no longer corresponds to the original observed
sqlattack, worm, xterm
target value. Model accuracy especially suffers if we shuffle
a column that the model depended on heavily for predic-
Correctly prepared data for processing guarantees precise tions. For stability of the results, any number from 50 to 100
and reliable results of data analysis. permutations is recommended [16].

4.2 SHapley Additive exPlanations (SHAP)

4 Explanation methods
The scope of explanations in this work, falls under global and
local post-hoc explanations. To obtain the global explanation
we have used Permutation Importance and SHAP explana-
tion algorithms, and for local explanations, we have used
LIME, SHAP and CIU [28].
4.1 Permutation importance (PI)
PI proposed in [16] normalizes the feature importance
thereby correcting the bias. Model reliance [29] is a model
agnostic way of obtaining permutation importance.
The permutation importance algorithm is given as Algo-
rithm 1:
SHAP, proposed by Lundberg and Lee [15], covers both
global and local scope of explanation based on Shapley value,
a method in coalitional game theory. SHAP needs to sat-
isfy the Shapely properties, namely the symmetry, wherein
features that contribute equally must get the same value,
dummy wherein the features that do not contribute to pre-
diction should get zero value, and additivity wherein the
features’ contribution must add up to the difference of pre-
diction and average. The algorithm for computing SHAP is
given as Algorithm 2:
First, select an instance of interest x of which explanation
need to be generated, a feature j and the number of iterations
needed K . For each iteration, a random instance t is selected
from the data and a random order of features is generated.

123
222
Algorithm 2: Compute Shapley value of feature j [30]
Result:  j , Shapley value for the value of the j-th feature
Input: Number of iterations K, instance of interest x, feature
index j, number of features p,data matrix X, and machine
learning model g;
for k=1,...K do
end
Draw random instance t from the data matrix X;
Choose a random permutation λ of the feature values;
Order instance x :
xλ = (x(1) , . . . , x( j) , . . . , x( p) );
 Order the feature values of instance x according to the random
permutation λ
Order instance t :
tλ = (t(1) , . . . , t( j) , . . . , t( p) );
 Order the feature values of random instance t according
to the random permutation λ
Construct two new instances:
With feature j:
x+ j = (x(1) , . . . , x( j−1) , x( j) , t( j+1) , . . . , t( p) );
 New instance formed which contains the feature values of x
upto jth feature and the remaining feature values from instance t,
while keeping the order of permutation
Without feature j:
x− j = (x(1) , . . . , x( j−1) , t( j) , t( j+1) , . . . , t( p) );
 New instance formed which contains the feature values of x
upto ( j − 1)th feature and from jth feature copy the values from
instance t, while keeping the order of permutation
Compute marginal contribution
kj = ĝ(x+ j ) − ĝ(x− j );
 ĝ(.) is the output of machine learning model
Compute Shapley value as the average:

 j (x) = K1 1K kj
Two new feature vectors are created by combining values
of the instance of interest x and the random instance t. The
instance x+ j is the instance of interest, but all feature values in
the order after the jth feature are replaced by feature values
from the sample t. The instance x− j is same as x+ j , but
feature j and other features in order are replaced by the value
for feature j and the values of other features in order from
the sample t. The difference in the prediction from the black
box is computed and all the differences are averaged. This
process is repeated for each and every feature, to find out its
Shapley value.
4.3 Local interpretable model-agnostic explanations
(LIME)
LIME, proposed by Ribeiro et al. [17] uses a surrogate model
to provide local explanations. LIME explains a model by
inspecting the changes that happen to the predictions when
123
S. Hariharan et al.
the model is given with variations of data. It generates a
new dataset having perturbed samples and the corresponding
predictions of the black box model. This perturbed dataset is
used for training the interpretable model, which is weighted
by the proximity of the sampled instances to the instance of
interest. The interpretability of local surrogate models with
constraints can be represented as given in Eq. 1.
E p(x) = arg min L( f , g, πx ) + (g) (1)
g∈G
E p(x) is the function representing explanation of our sample
of interest x and it tries to minimise the loss L due to mean
squared error. On the other hand we can say that, it measures
how close the explanation model g, is to the prediction of the
original model f , while maintaining the model complexity
(g) low. πx is the proximity measure, representing how
large the neighborhood around instance x is.
4.4 Contextual importance and utility (CIU)
The Contextual Importance and Utility (CIU) algorithm com-
prises of two algorithms to explain the model predictions, and
its working is based on decision-making theory [30], [31].
This algorithm is based on the fact that the significance of a
feature and its utility changes with respect to other feature
values. So, the Contextual Importance (CI) approximates the
overall importance of a feature in the current context, and
the Contextual Utility (CU) provides an estimation of how
favorable or not the current feature value is for a given output
class. A Context C defines the input values x that describe
the current situation or instance to be explained. CI and CU
are defined as follows:
max x (Ci ) − min x (Ci )
CI = (2)
AB Smax x − AB Smin x
yi − min x (Ci )
CU = (3)
max x (Ci ) − min x (Ci )
• max x (Ci ) and min x (Ci ) are the highest and the lowest
prediction values observed by varying the value of the
features, x
• AB Smax x and AB Smin x specifies the value range over
all predictions,
• Ci is the context such as the feature or set of features
being studied, where i is the prediction value.
• yi is the instance-specific prediction value.
The CIU algorithm is given as Algorithm 3:
XAI for intrusion detection system: comparing explanations based on global and local scope 223

Algorithm 3: Compute Contextual Importance and Util- Table 5 Permutation importance score using RF model on NSL-KDD
Binary classification dataset
ity [27]
Result: C I , CU Contextual Importance and Utility Weight Feature
Initialization: C is Context of the input values x, prediction value i,
data matrix K, and machine learning model g for k=1,...K do 0.0192 ± 0.0014 src_bytes
end 0.0153 ± 0.0007 Protocal_type_icmp
compute the prediction gk (x); 0.0092 ± 0.0014 logged_in
0.0081 ± 0.0006 Protocal_type_tcp
compute max x (Ci );
computemin x (Ci ); 0.0079 ± 0.0005 hot
compute AB Smax x ; 0.0071 ± 0.0008 dst_host_same_src_port_rate
compute AB Smin x ; 0.0061 ± 0.0007 srv_serror_rate
max x (Ci )−min x (Ci )
C I = AB Smax x −AB Smin x ; 0.0053 ± 0.0009 dst_bytes
yi −min x (Ci )
CU = max x (Ci )−min x (Ci ) ; 0.0051 ± 0.0006 dst_host_srv_serror_rate
0.0041 ± 0.0002 serror_rate
0.0039 ± 0.0006 dst_host_serror_rate
0.0038 ± 0.0015 dst_host_same_srv_rate
4.5 Methodology to compare explanations
0.0037 ± 0.0006 srv_count
0.0027 ± 0.0010 count
Both the global explanation methods, PI and SHAP, provide
the feature importance of all the input features. We have con- 0.0023 ± 0.0011 dst_host_srv_rerror_rate
sidered the top 15 features from the list, and compared the
accuracy by applying them to the RF model and XGBoost
model. The list of libraries used in ML algorithm and XAI methods
Both the local explanation methods, SHAP and LIME pro- are included here:
vide instance-based feature importance. For comparison of Machine Learning Algorithm:
local explanation, we are using Lipschitz continuity [32], a
robustness measure for a local explanation, by considering
• Random Forest classifier - Scikit learn
not just the single point, but also its neighbourhood. The
• XGBoost - Scikit learn
method to obtain the Lipschitz constant is given in equations
• Light Gradient Boosting - Scikit learn
4 and 5. We quantify the explanation model f in terms of its
Lipschitz constant L, Test data set X for every point xi of
interest as an optimization problem: XAI methods:
X = {xi }i=1
n denotes a sample of input.
• Permutation Importance - eli5 library
N (xi ) = {x j ∈X | xi − x j ≤} (4) • SHapley Additive exPlanations - shap python package
• Local Interpretable Model Agnostic Explanations - lime
where N (xi ) is a ball of radius  centered at xi , for every python package
xi ∈X . • Contextual Importance and Utility - Py-CIU python
library
f (xi ) − f (x j )
L̂(xi ) = argmax (5)
x j ∈N (xi )≤ xi − x j 5.1 Global interpretability

We are using Permutation Importance and SHAP to provide

5 Experiments and discussion of results
The primary focus of this work is to build a classifier for
network intrusion detection that can not only give a good
prediction performance but also augment the classifier model
with an explainer that can explain the predictions of the clas-
sifier. We are using the RF model, XGBoost and LightGBM
models on both the Kaggle IDS dataset for binary classi-
fication and the NSL-KDD dataset for binary as well as
multi-class classification problem.
a global scope of explanation of the model by obtaining the
feature importance.
Table 5 shows the top 15 important features obtained
by the permutation importance method on RF Model using
NSL-KDD dataset. The top three features are src_bytes, Pro-
tocol_type_icmp and logged in, as shown in the 2nd column
of the table.
Table 6 shows the top 15 important features obtained
by the permutation importance method on RF Model using
Kaggle dataset. The top three features are dst_host_count,

123
224 S. Hariharan et al.

Table 6 Permutation importance score using RF model on Kaggle reliable technique without the need to retrain the model at
Binary classification dataset each modification of the dataset. However, it is more compu-
Weight Feature tationally expensive than the default feature importance and
permutation importance, and is known to overestimate the
0.0058 ± 0.0008 dst_host_count
importance of correlated predictors.
0.0040 ± 0.0012 dst_bytes
We have iterated the permutation ranging from 10 to 500,
0.0032 ± 0.0010 dst_host_same_src_port_rate for the three datasets individually, to obtain a stabilized top
0.0029 ± 0.0005 src_bytes 15 features, which will be used in the comparison.
0.0026 ± 0.0011 service_private Figures 2 and 3 are the SHAP summary plots on NSL-
0.0010 ± 0.0003 dst_host_diff_srv_rate KDD dataset with multiclass classification and binary clas-
0.0008 ± 0.0002 hot sification respectively, where the X-axis is marked with
0.0008 ± 0.0006 dst_host_srv_count the mean SHAP value and the Y-axis lists the features in
0.0005 ± 0.0004 count the ascending order of importance. The top 3 features for
0.0005 ± 0.0006 dst_host_same_srv_rate binary classification dataset using RF model are logged_in,
0.0005 ± 0.0004 srv_count dst_host_error_rate and dst_host_diff_srv_rate. The top 3
0.0004 ± 0.0001 service_ftp_data features for multiclass classification dataset using RF model
0.0004 ± 0.0001 num_compromised are flag_S0, dst_host_rerror_rate and logged_in.
0.0004 ± 0.0001 Protocal_type_icmp Figure 4 shows tha SHAP summary plots on Kaggle binary
0.0003 ± 0.0003 dst_host_rerror_rate classification dataset. The top 3 features using RF Model are
src_bytes, dst_bytes and flag_s0.

dst_bytes and dst_host_same_src_port_rate as shown in the

5.1.1 Comparison of PI and SHAP based feature importance
2nd column of the table.
for global interpretability
The value in each row corresponds how much model per-
formance, in this case accuracy, decreases with a random
Table 7 shows the comparison table where in the list of fea-
permutation, i.e, the mean decrease in accuracy. The permu-
tures based on their importance obtained from PI and SHAP
tation importance is calculated by repeating it with multiple
XAI methods on RF and XGBoost model, we have selected
permutations. The value after the +/- measures how perfor-
the top 15 features and measured the performance metric
mance varied from across permutations. The negative values
accuracy, precision, recall and F1-score of the model. Both
for permutation importance are when the permutations are
the models perform better using only the top 15 features from
more accurate than the real data. This happens when the fea-
the PI explanation when compared to considering all input
ture should have had importance close to 0, but randomness
features. Both the models also perform better with multiclass
caused the predictions on permuted data to be more accurate.
dataset using the top 15 features from the SHAP explanation
One such feature that we identified in NSL-KDD binary clas-
when compared to considering all the input features.
sification is dst_host_count with PI value 0.0007 ± 0.0010.
The Permutation importance algorithm is model agnostic so
it can be applied to any model. It is a reasonably efficient and

Fig. 2 SHAP Summary plot

using RF model on NSL-KDD
multi-class classification dataset

123
XAI for intrusion detection system: comparing explanations based on global and local scope 225

Fig. 3 SHAP Summary plot

using RF model on NSL-KDD
binary-class classification
dataset

Fig. 4 SHAP Summary plot

using RF model on Kaggle
binary classification dataset

Table 7 Comparison table on NSL-KDD and Kaggle dataset

Dataset used Features Accuracy Precision Recall F1-score

Kaggle dataset on RF Model All features 0.999 1 1 1

Top 15 features from PI 0.996 1 1 1
Top 15 features from SHAP 0.998 0.99 0.99 0.99
Kaggle dataset on XGBoost Model All features 0.999 1 1 1
Top 15 features from PI 0.997 1 1 1
Top 15 features from SHAP 0.996 1 1 1
NSL-KDD Binary dataset on RF Model All features 0.726 0.814 0.726 0.719
Top 15 features from PI 0.75 0.823 0.75 0.74
Top 15 features from SHAP 0.707 0.756 0.707 0.704
NSL-KDD Binary dataset on XGBoost Model All features 0.769 0.84 0.769 0.765
Top 15 features from PI 0.772 0.769 0.837 0.772
Top 15 features from SHAP 0.764 0.832 0.764 0.76
NSL-KDD Multiclass dataset on RF Model All features 0.691 0.796 0.691 0.644
Top 15 features from PI 0.736 0.745 0.736 0.689
Top 15 features from SHAP 0.691 0.693 0.691 0.645
NSL-KDD Multiclass dataset on XGBoost Model All features 0.717 0.8 0.717 0.667
Top 15 features from PI 0.745 0.823 0.745 0.704
Top 15 features from SHAP 0.741 0.814 0.741 0.701

123
226
Fig. 5 SHAP waterfall plot for
misclassified instance using RF
model on NSL-KDD binary
classification dataset
5.2 Local interpretablility
LIME and SHAP are used to provide a local scope of expla-
nation for any specific instance. Figure 6 shows the SHAP
explanation of a misclassified sample, where an attack is clas-
sified as normal.
Figure 5 shows the SHAP explanation of a misclassified
sample, which is anomaly. It is observed that the predicted
SHAP value for an instance misclassified as anomaly in the
validation dataset is 0.89. Figure 6 shows the SHAP expla-
nation of a correctly classified sample, which is normal. It
is observed that the predicted SHAP value for an instance
classified as normal in the validation dataset is 0.89, whereas
the base value is 0.568. Feature values that cause an increase
in predictions are shown in black. Their visual size shows the
magnitude of the feature’s effect. Feature values that cause a
decrease in the prediction are in gray. The biggest impact is
for features logged_in = 1. Also, the service_http value has
a meaningful effect in decreasing the prediction. It can be
noted that if we subtract the length of the black bars from the
123
S. Hariharan et al.
length of the gray bars, it equals the distance from the base
value to the output.
LIME explanation of a correctly classified sample as
normal is shown in Figs. 7 and 8 for NSL-KDD and Kag-
gle dataset respectively. The feature value that contributes
in the prediction to be normal are shown in black. while
the features that contributes for predicting as an anomaly
are shown in gray. In Fig. 7 service_provider, logged_in
and wrong_fragment here contributes to normal prediction
and flag_S3, flag_SF and service_Idap here contributes to
anomaly prediction. In Fig. 8 service_pop_2, logged_in
and dst_bytes here contributes to normal prediction and
service_printer and count here contributes to anomaly pre-
diction.
5.2.1 Contextual importance and utility local explanation
This section provides explanations for a decision made by
a Random forest model on NSL-KDD dataset, for binary
XAI for intrusion detection system: comparing explanations based on global and local scope 227

Fig. 6 SHAP waterfall plot for

correctly classified normal
instance using RF model on
NSL-KDD binary classification
dataset

Fig. 7 LIME Explanation for

correctly classified normal
instance using RF model on
NSL-KDD binary classification
dataset

classification. A specific instance or group of instances can differently both the instances change with the new_impact
be taken from the dataset to represent current outcome of the feature.
model or a specific context. The Figs. 9 and 10 explain the
model based on CIU algorithm. For this particular example
5.2.2 Comparison of SHAP and LIME based on consistency
we have selected the features such as src_bytes and dst_bytes
and stability
to quantify the influence of interaction between these two
features. The two features are combined and represented as
We define consistency by comparing explanations between
a new feature called new_impact. Both the figures show how
similar instances for two different models. Figures 11

123
228
Fig. 8 LIME Explanation for
correctly classified normal
instance using RF model on
Kaggle classification dataset
Fig. 9 Contextual importance of a sample predicted as normal Random
Forest on NSL-KDD dataset
Fig. 10 Contextual utility of a sample predicted as normal Random
Forest on NSL-KDD dataset
and 12 illustrate the estimations performed for both RF
and XGBoost predictive models on test points of datasets
using LIME with  = 1 and with only 100 neighbouring
data points. In the random forest model, service_private,
logged_in and wrong fragment have a positive effect on pre-
dictions while flag_SF has a negative effect. In the XGBoost
model, src_bytes have a positive effect on predictions while
service_shell has a negative effect.
Figure 13 shows that the SHAP explanations main-
taining stability as the logged_in, dst_hosts_error_rate,
123
S. Hariharan et al.
dat_host_srv_count features have a positive impact on the
prediction except emph service_http.
Table 8 shows how the similar instance with correct pre-
diction as normal using SHAP Interpretability method. The
features that have a positive effect on RF Model using the
SHAP explanation for prediction is 100% of the RF model
using the SHAP explanation with the positive effect on pre-
diction. The features that have a negative effect on RF Model
using the LIME explanation for the prediction is 100% of
the RF model using the SHAP explanation with the negative
effect on prediction. Table 9 shows how the same instance
with correct prediction as normal using SHAP and LIME
Interpretability methods. The features that have a positive
effect on RF Model using the LIME explanation for predic-
tion is 71% of the RF model using the SHAP explanation
with the positive effect on prediction. The features that have
a negative effect on RF Model using the LIME explana-
tion for the prediction is 75% of the RF model using the
SHAP explanation with the negative effect on prediction.
Table 10 shows how the same instance with correct predic-
tion as normal using LIME Interpretability method on RF and
XGBoost model. The features that have a positive effect using
LIME explanation on RF Model for prediction is 61% of the
LIME explanation with positive effect on prediction using
XGBoost model. The features that have a negative effect
using LIME explanation on RF Model for prediction is 50%
of the LIME explanation with negative effect on prediction
using XGBoost model.
6 A case study on NSL-KDD DoS attack
variants
DoS attack category is the major subset of attack present in
the NSL-KDD dataset as a whole and hence it is important
to get the deeper understanding of predictions about DoS
attacks in global as well as local perspectives. That is the
reason why we have taken DoS attack as a case study for
analysis. The influential features in predicting the attacks can
XAI for intrusion detection system: comparing explanations based on global and local scope 229

Fig. 11 NSL-KDD binary

instance LIME explanation on
random forest with L = 4.64

Fig. 12 NSL-KDD binary

instance LIME explanation on
XGBoost with L = 6.47

be better explored using global explanation based on binary
and multiclass classification. The most important features
selected using SHAP and Permutation Importance as global
methods in-turn reduce the complexity of classification algo-
rithms. Comparison of detection of same attack variant using
two different models namely Random forest and LightGBM
increases the trustworthiness of the system. Local explana-
tions such as LIME and CIU contribute to study the feature
importance of specific DoS attacks such as land and nep-
tune. The security analysts can make use of the explanations
to explore the relationship between the values of features and
specific attack types. Furthermore, analysis of explanations
regarding a particular DoS attack variant in terms of stability

123
230
Fig. 13 NSL-KDD binary
instance SHAP explanation on
RF Model with L = 0.84
and consistency has been done to keep the trust of the end
users of the Internet.
We have formed a subset of training as well as test data set
from the NSL-KDD train and test data respectively by select-
ing the instances of normal and DoS attack variants only.
Table 11 explains the information regarding NSL-KDD DoS
attack only dataset. In the training set, 67343 connections
are labeled as ‘normal’and the rest of 45927 connections are
labeled as ‘DoS’attack. In the test set, 9711 connections are
‘normal’and 7460 connections are ‘DoS’. Experiments are
done in three different phases based on the abstract diagram
shown in Fig. 1. In the first phase, two class and multi-class
classifications are performed using Random forest and Light-
GBM algorithms. The performance metrics such as accuracy,
precision, recall and F- measure are evaluated. In the sec-
123
S. Hariharan et al.
ond phase global and local explanations are generated using
PI, SHAP and LIME. Then these Explanation methods are
compared based on the aspect of global and local explana-
tions generated. Global explanations mainly highlight the
feature importance of predictions caused by the correspond-
ing model. Hence, we use the important features recognised
by these methods to retrain the model and the performance
metrics are evaluated. It is done for both binary and multi-
class classification using LightGBM and Random forest.
Then the local explanations generated by LIME and SHAP
are compared based on consistency and stability aspects. In
the third phase, we have used CIU to generate explanations
as justifications that can be produced before intended clients
who wish to know the details of predictions of DoS attacks.
XAI for intrusion detection system: comparing explanations based on global and local scope 231

The features that have a positive and negative

6.1 Comparison of SHAP and PI based on feature

impact on the prediction of the model for

importance

This example provides explanations for a decision made by

a Random forest model on NSL-KDD dataset, for binary

both the instances are same

classification. Tables 12 and 13 explain the model based on
PI algorithm. A specific instance or group of instances can
be taken from the dataset to represent current outcome of
the model or a specific context. For this particular example
we have selected above 90% of the features to be the same
Conclusion

in both cases. In the case of binary, the features at the top

are the majority extracted from packet header fields namely
‘src_bytes’, ‘protocol’, ‘flag’and ‘dst_bytes’. In the case of
multi-class, the feature ‘service’at the top comes under the
feature from header field. Almost all the other important fea-
Number of features having positive effect: 46

Top 3 feature with positive effect: logged_in,

effect:12 Number of features having No

service_http, flag _RSTOS0, flag_SF tures are host based features.

Number of features having Negative

Figures 14 and 15 are the summary plots of SHAP based

Actual Classification: normal-Model

feature importance. From Fig. 14, it is inferred that the mean

Top 3 feature with negative effect:
dst_host _srv_count, dst_host

shap value of feature, flag = 0.07 for class normal and it is

Prediction Probability:90%

0.14-0.07=0.07 for class neptune. The contribution of fea-

ture, same_srv_rate is 0.04 for class normal and around 0.06
Prediction:normal

for class neptune in terms of mean shap values. However, for

the RF model based on binary classification given in Fig. 15,
_serror_rate

the influence of feature flag is almost equal for class normal

effect:64
Instance 2

and attack and the influence of feature dst_bytes is bit greater

for predicting class attack than normal. The important fea-
tures selected in binary and multi-class cases are almost the
Table 8 Comparison of NSL-KDD instances with SHAP Explainable method using RF Model

same. For comparing the effect of global explanation based

on PI and SHAP, we have done the experiments for binary and
Top 3 feature with negative effect: service_http,
Number of features having Negative effect:12

dst_host _srv_count, dst_host _serror_rate

multi-class, employing Random forest and LightGBM using

Number of features having positive effect: 46

Top 3 feature with positive effect:logged _in,

Number of features having No effect: 64

the top 15 features. The comparison table in terms of accu-

racy, precision, recall and F1-score is given in the Table 14.
Actual Classification: normal Model

An improvement in performance is prominent in the case of

PI based explanation.
Prediction Probability:89%

flag _RSTOS0, flag _SF

Prediction:normal

6.2 Comparison of SHAP and LIME based on

Similar instances

consistency and stability

Instance 1

Stability is the factor that measures how similar are the

explanations for similar instances for a particular model. Fig-
ures 16 and 17 represent the stability criteria of comparison
for LIME explanations for Random forest and LightGBM
respectively. Two samples labeled as normal are selected
XAI methods

for comparison and the figure lists the top twelve fea-
SHAP

tures which influences the prediction of the instance as

normal. From Fig. 16 it is evident that, the feature value
wr ong_ f ragment ≤ 0.00 has large impact on the predic-
tion in both the cases. The majority of the features are same in
Ran-dom forest

both the cases that positively and nagatively impact the pre-
ML model

dictions. Figures 18 and 19 illustrate the stability of SHAP

Case 1

explanations for the Random forest and LightGBM respec-

tively.

123
 232

123
Table 9 Comparison of NSL-KDD instances with SHAP and LIME Explainable method using RF Model
Case 2
ML model XAI methods Same instance Conclusion

Random forest SHAP Actual Classification: normal Model Prediction:-normal The features that have a positive
effect on RF Model using the
LIME explanation for prediction
is 71% of the RF model using the
SHAP explanation with the
positive effect on prediction.The
features that have a negative
effect on RF Model using the
LIME explanation for the
prediction is 75% of the RF
model using the SHAP
explanation with the negative
effect on prediction
Prediction Probability: 89%
Number of features having positive effect: 46 Number of features having Negative effect:12 Number of
features having No effect: 64
Top 3 features with positive effect:logged_in, dst_host_srv _count, dst_host _serror_rate
Top 3 features with negative effect: service_http, flag_RSTOS0, flag_SF
Random forest LIME Actual Classification: normal Model Prediction:-normal
Prediction Probability: 89%
Number of features having positive effect: 60 Number of features having Negative effect:48 Number of
features having No effect: 14
Top 3 feature with positive effect:service_private, logged_in, service_IRC
Top 3 feature with negative effect:flag_SF, flag_S2, service_domain_u
S. Hariharan et al.
 Table 10 Comparison of NSL-KDD instances with LIME Explainable method using RF Model and XGBoost
Case 3
ML model XAI methods Same instance Conclusion

Random forest LIME Actual Classification: normal Model Prediction: normal The features that have a positive
effect using LIME explanation
on RF Model for prediction is
61% of the LIME explanation
with positive effect on prediction
using XGBoost model. The
features that have a negative
effect using LIME explanation
on RF Model for prediction is
50% of the LIME explanation
with negative effect on prediction
using XGBoost model
Prediction Probability: 89%
Number of features having positive effect: 60 Number of features having Negative effect:48 Number of
features having No effect: 14
XAI for intrusion detection system: comparing explanations based on global and local scope

Top 3 feature with positive effect:service_private, logged_in, service_IRC

Top 3 feature with negative effect: flag_SF, flag_S2, service_domain_u
XGBoost LIME Actual Classification: normal Model Prediction:-normal
Prediction Probability: 89%
Number of features having positive effect: 68 Number of features having Negative effect:44 Number of
features having No effect: 10
Top 3 feature with positive effect:flag_S2, num_shells, src_bytes
Top 3 feature with negative effect:service_nntp, flag_OTH, service_harvest

123
233
234 S. Hariharan et al.

Table 11 Description of
Type of classification Train Test Total Target
NSL-KDD DoS attack only
dataset along with encoded label Binary 113270 17171 130441 DoS: 0, normal: 1
details
Multi-class 113270 17171 130441 apache2 : 0, back: 1, land:
2, mailbomb : 3, neptune:
4, normal: 5, pod : 6,
processtable: 7, smurf: 8,
teardrop: 9, udpstorm: 10,
worm: 11

Table 12 PI of features on NSL-KDD DoS attack variants using Light- Consistency of LIME and SHAP describes, how much
GBM Binary classification explanations agree with each other regarding LightGBM
Weight Feature and Random forest. Figures 18 and 19 represent the con-
sistency in explanations for making predictions of similar
0.2220 ± 0.0038 src_bytes
instances labeled as normal. The scale here represents model
0.0321 ± 0.0011 protocol
prediction. The output value is the calculated prediction for
0.0088 ± 0.0005 serror_rate a particular instance. The base value is the value that would
0.0080 ± 0.0006 flag be predicted if we do not know any features for the current
0.0037 ± 0.0011 count output [15]. This is the mean of the predictions of test data.
0.0034 ± 0.0009 dst_bytes Features shown in gray push the prediction to higher values
0.0028 ± 0.0010 same_srv_rate but the features shown in black push the prediction to lower
0.0016 ± 0.0005 srv_count values. The width of the coloured arrows show how much
0.0012 ± 0.0008 wrong_fragment influential the feature is.
0.0005 ± 0.0002 dst_host_srv_rerror_rate In Fig. 18 the base value of prediction is around 0.6
0.0004 ± 0.0003 dst_host_srv_serr_rate and the actual prediction value is 1.19. The feature value,
0.0003 ± 0.0003 dst_host_rerror_rate diff_srv_rate=0.0 influences the prediction positively. How-
0.0002 ± 0.0005 dst_host_serror_rate ever, the feature in black, ie serror_rate=0.0 pushes the
0.0002 ± 0.0002 dst_host_same_src_port_rate prediction in opposite direction. The Fig. 19 explains the
0.0001 ± 0.0001 num_compromised model LightGBM in predicting the same instance. The fea-
tures positively impact the predictions are almost same in
both the cases. Figures 16 and 17 explain the consistency of
Table 13 PI of features on NSL-KDD DoS attack variants using Light- LIME regarding Random forest and LightGBM respectively.
GBM multi-class classification
Weight Feature

0.1378 ± 0.0034 service 6.3 CIU of LightGBM on NSL-KDD (DoS) dataset

0.1172 ± 0.0020 logged_in
0.0886 ± 0.0042 dst_host_diff_srv_rate Local explanations based on the outcome of the black-box
0.0726 ± 0.0045 diff_srv_rate
for a specific instance based on the context is very beneficial
for the network analyst to take appropriate decisions. For an
0.0565 ± 0.0033 dst_host_rerror_rate
end user, it is not required to explain the underlying logic of
0.0499 ± 0.0013 count
the entire black-box but only the reason for the outcome on
0.0485 ± 0.0046 dst_host_srv_serr_rate
a specific input instance.The CIU can deal with this kind of
0.0406 ± 0.0028 same_srv_rate
explanations. The high importance of a feature means that
0.0403 ± 0.0025 src_bytes
perturbations in that feature results in highest changes in the
0.0352 ± 0.0037 flag
prediction value while the lower contextual importance value
0.0199 ± 0.0019 srv_count
implies that changes on this feature do not affect the outcome
0.0183 ± 0.0008 dst_host_same_src_port_rate significantly.
0.0170 ± 0.0005 dst_host_srv_rerror_rate The contextual importance and utility plots of specific land
0.0146 ± 0.0012 dst_host_srv_count attacks missclassified as neptune and normal respectively are
0.0133 ± 0.0014 dst_host_serror_rate given in Figs. 20, 21, 22 and 23. The length of the bar cor-
responds to the CI value. A configurable threshold value of
CI can be taken to divide the features into ‘defavorable’and
‘favorable’ ranges and here it is taken as 0.5. Hence the bar

123
XAI for intrusion detection system: comparing explanations based on global and local scope 235

Fig. 14 SHAP summary plot of

features on NSL-KDD DoS
attack variants using Random
forest Binary classification

Fig. 15 SHAP summary plot of

features on NSL-KDD DoS
attack variants using Random
forest multi-class classification

123
236 S. Hariharan et al.

Table 14 Comparison table of performance measures of RF and LightGBM on NSL-KDD(DoS) dataset

Dataset used Features Accuracy Precision Recall F1-score

NSL-KDD(DoS) Binary dataset on RF Model All features 0.884 0.90 0.88 0.88
Top 15 features from PI 0.908 0.92 0.91 0.91
Top 15 features from SHAP 0.902 0.91 0.90 0.90
NSL-KDD(DoS) Binary dataset on LightGBM Model All features 0.90 0.91 0.90 0.90
Top 15 features from PI 0.92 0.93 0.92 0.92
Top 15 features from SHAP 0.902 0.91 0.90 0.90
NSL-KDD(DoS) Multiclass dataset on RF Model All features 0.886 0.81 0.89 0.85
Top 15 features from PI 0.894 0.81 0.89 0.85
Top 15 features from SHAP 0.893 0.81 0.89 0.85
NSL-KDD(DoS) Multiclassdataset on LightGBM Model All features 0.89 0.81 0.89 0.84
Top 15 features from PI 0.894 0.81 0.89 0.85
Top 15 features from SHAP 0.893 0.81 0.89 0.85

Fig. 16 LIME explanation of

similar instances of NSL-KDD
(DoS) dataset using Random
forest Binary classification

plots show the features, with influence above 50%. A specific
instance or group of instances can be taken from the dataset to
represent current outcome of the model or a specific context.
CIU can assess the effects of feature interaction. For this
particular example we have selected the features such as
src_bytes and dst_bytes to quantify the influence of inter-
action between these two features. The two features are
combined and represented as a new feature called Fea-
ture_int1. Similarly the features such as wrong_fragment
and num_failed_logins are selected for assessing the feature
interaction and is represented as Feature_int2. This bar plot
can answer “why this sample is categorised as normal?” or
“Why it is not categorised as attack?”. Figure 20 shows that
the interdependent effects of src_bytes and dst_bytes have
greater importance on the prediction. The importance of fea-
ture interaction between src_bytes and dst_bytes is around
54.72% and flag is 51.25% in the context of predicting land
attack as neptune. The feature dst_host_srv_serror_rate has
an importance of 93.23% in the context of making prediction
of land attack as normal.
The contextual utility shows that the the grouped features
demonstrating the feature interactions, such as Feature_int
1 and Feature_int 2 are giving nearly 100% utility in the
context of predicting land attack as neptune. In predicting

123
XAI for intrusion detection system: comparing explanations based on global and local scope
Fig. 17 LIME explanation of similar instances of NSL-KDD (DoS)
dataset using LightGBM Binary classification
Fig. 18 SHAP explanation of of NSL-KDD (DoS) dataset using Ran-
dom forest Binary classification
Fig. 19 SHAP explanation of NSL-KDD (DoS) dataset using Light-
GBM Binary classification
land attack as normal, the CU of the feature interaction named
Feature_int 2 is around 100%.
7 Discussion
In this work, we compare the different explanation frame-
work focusing on model agnostic XAI methods covering
both global and local scope. We examine model agnostic
global XAI methods (PI, SHAP) and model agnostic local
XAI methods (SHAP, LIME, and CIU) explanation methods
to interpret the predictions for RF, XGBoost and LGBoost
models. We build classifiers by using Kaggle IDS classifica-
tion datasets and NSL-KDD IDS classification dataset. Our
results show that for the global comparison of explanation
methods we measured the robustness in terms of accuracy,
recall and precision considering the top 15 features obtained
237
Fig. 20 Contextual importance of land attack predicted as neptune
Fig. 21 Contextual importance of land attack predicted as normal
from global Permutation Importance and global SHAP inter-
pretability methods and we observed that the accuracy of the
model prediction for the top 15 features obtained using both
the global XAI method on NSL-KDD dataset ranges from
70% to 75% .
For the Local comparison of explanation methods we mea-
sured the robustness in terms of three cases and the findings
are as follows:
• Case 1: Compare similar instance using SHAP inter-
pretability method on RF model:
We observed that there is 100% match in the feature that
affects the prediction positively and negatively.
123
238
Fig. 22 Contextual utility of land attack predicted as normal
Fig. 23 Contextual utility of land attack predicted as normal
• Case 2: Compare same instance using SHAP and LIME
interpretability method on RF model:
We observed that there is an 71% match in the feature that
affects the prediction positively and 75% match in the feature
that affects the prediction negatively.
• Case 3: Compare same instance using LIME interpretability
method on RF and XGBoost model:
We observed that there is an 61% match in the feature that
affects the prediction positively and 50% match in the feature
that affects
the prediction negatively.
Our future work focuses on assessing the robustness of
explanations on models from different family and inspire
further exploration in the same area.
123
S. Hariharan et al.
8 Conclusion
In this paper, we have discussed the necessity of explain-
ing the machine learning model to the end user/analyst, and
the major hurdles of presenting a trustworthy model when
analysing the performance of anomaly based IDSs. Over
the years, one of the major challenges of ML-based IDSs
is the very high rate of false positives. The sophisticated ML
algorithms employed to improve the performance in turn
increases the complexity of the model. XAI methods pro-
vide us with interpretability to prediction, which can help in
redesigning the structure of IDSs to improve performance.
In this paper we have covered both global and local scope
of explanation along with providing a comparison with the
model evaluation metrics, namely, accuracy, precision, recall
and F1 score of the top 15 features obtained from PI and
SHAP. The local explanation comparison is made based on
consistency and stability. The case study with DoS attack
variants helps to study the impact of features on model perfor-
mance, specifically for DoS attacks. It is desirable to ensure
that the explanations provided by the models are acceptable
or it meets the requirements for which it is meant for, and
this is planned to be done in detail as a future work.
References
1. Hu, X., Li, T., Wu, Z., Gao, X., Wang, Z.: Research and application
of intelligent intrusion detection system with accuracy analysis
methodology. Infrared Phys. Technol. 88, 245–253 (2018)
2. Holzinger, A.: From machine learning to explainable AI. In:
World symposium on digital intelligence for systems and machines
(DISA), pp. 55–66 (2018)
3. National Academies of Sciences, Engineering, and Medicine et al.:
Implications of artificial intelligence for cybersecurity. In: Proceed-
ings of a Workshop. National Academies Press (2019)
4. Othman, S.M., Ba-Alwi, F.M., Alsohybe, N.T., Al-Hashida, A.Y.:
Intrusion detection model using machine learning algorithm on big
data environment. J. Big Data 5(1), 1–12 (2018)
5. Da Costa, K.A., Papa, J.P., Lisboa, C.O., Munoz, R., de Albu-
querque, V.H.C.: Internet of things: a survey on machine learning-
based intrusion detection approaches. Comput. Netw. 151, 147–
157 (2019)
6. Hodo, E. et al.: Threat analysis of IoT networks using artificial
neural network intrusion detection system, pp. 1–6. IEEE (2016)
7. Peng, K., et al.: Intrusion detection system based on decision tree
over big data in fog environment. Wirel. Commun. Mob. Comput.
2018 (2018)
8. Zhang, Z., Shen, H.: Application of online-training SVMs for real-
time intrusion detection with different considerations. Comput.
Commun. 28(12), 1428–1442 (2005)
9. Sharma, Y., Verma, A., Rao, K., Eluri, V.: Reasonable explainability
for regulating AI in health. ORF occasional paper (261) (2020)
10. Rudin, C., Radin, J.: Why are we using black box models in ai when
we don’t need to? A lesson from an explainable AI competition.
Harvard Data Sci. Rev. 1(2) (2019)
11. Paulauskas, N., Auskalnis, J.: Analysis of data pre-processing influ-
ence on intrusion detection using NSL-KDD dataset. In: Open
XAI for intrusion detection system: comparing explanations based on global and local scope
Conference of Electrical, Electronic and Information Sciences
(eStream), pp. 1–5. IEEE (2017)
12. Datta, H., Deshmukh, T.G., Puja Padiya, Y.: International Con-
ference on Communication, Information & Computing Technol-
ogy (ICCICT). Improving classification using preprocessing and
machine learning algorithms on NSL-KDD dataset
13. Lipton, Z.: The mythos of model interpretability. arXiv preprint
arXiv:1606.03490 (2016)
14. Freitas, A.A.: Comprehensible classification models: a position
paper. ACM SIGKDD Explor. Newsl. 15(1), 1–10 (2014)
15. Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting
model predictions, 4768–4777 (2017)
16. Altmann, A., Toloşi, L., Sander, O., Lengauer, T.: Permutation
importance: a corrected feature importance measure. Bioinformat-
ics 26(10), 1340–1347 (2010)
17. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?
Explaining the predictions of any classifier, 1135–1144 (2016)
18. Goode, K., Hofmann, H.: Visual diagnostics of an explainer model:
tools for the assessment of lime explanations. Stat. Anal. Data Min.
ASA Data Sci. J. 14(2), 185–200 (2021)
19. Doshi-Velez, F., Kim, B.: Towards a rigorous science of inter-
pretable machine learning. arXiv preprint arXiv:1702.08608
(2017)
20. Zhao, Q., Hastie, T.: Causal interpretations of black-box models.
J. Bus. Econ. Stat. 39(1), 272–281 (2021)
21. Goldstein, A., Kapelner, A., Bleich, J., Pitkin, E.: Peeking inside
the black box: visualizing statistical learning with plots of individ-
ual conditional expectation. J. Comput. Graph. Stat. 24(1), 44–65
(2015)
22. Apley, D.W., Zhu, J.: Visualizing the effects of predictor variables
in black box supervised learning models. J. R. Stat. Soc. Ser. B
(Stat. Methodol.) 82(4), 1059–1086 (2020)
239
23. Arya, V., et al.: One explanation does not fit all: a toolkit
and taxonomy of ai explainability techniques. arXiv preprint
arXiv:1909.03012 (2019)
24. Maonan Wang, Y.Y., Kangfeng Zheng, W.X.: An explainable
machine learning framework for intrusion detection systems. IEEE
Access 8(2020), 73127–73141 (2020)
25. Kaggle dataset. https://www.kaggle.com/sampadab17/network-
intrusion-detection
26. NSL-KDD data set for network-based intrusion detection systems.
https://www.unb.ca/cic/datasets/nsl.html
27. https://pair-code.github.io/facets/
28. Carvalho, D.V., Pereira, E.M., Cardoso, J.S.: Machine learning
interpretability: a survey on methods and metrics. Electronics 8(8),
832 (2019)
29. Fisher, A., Rudin, C., Dominici, F.: All models are wrong, but many
are useful: learning a variable’s importance by studying an entire
class of prediction models simultaneously. J. Mach. Learn. Res.
20(177), 1–81 (2019)
30. Anjomshoae, S., Främling, K., Najjar, A.: Explanations of Black–
Box Model Predictions by Contextual Importance and Utility, pp.
95–109. Springer, New York (2019)
31. Främling, K.: Decision Theory Meets Explainable AI, pp. 57–74.
Springer, New York (2020)
32. Alvarez-Melis, D., Jaakkola, T.S.: On the robustness of inter-
pretability methods. arXiv preprint arXiv:1806.08049 (2018)
Publisher’s Note Springer Nature remains neutral with regard to juris-
dictional claims in published maps and institutional affiliations.
Springer Nature or its licensor holds exclusive rights to this article
under a publishing agreement with the author(s) or other rightsholder(s);
author self-archiving of the accepted manuscript version of this article
is solely governed by the terms of such publishing agreement and appli-
cable law.
123