MANONMANIAM SUNDARANAR UNIVERSITY

Centre For Information Technology and Engineering

Mini Project - Viva Voce Examination

“EVALUATING DEEP NEURAL NETWORK AND CLASSICAL MACHINE LEARNING

ALGORITHMS FOR NETWORK INTRUSION DETECTION SYSTEM”

Guided By Presented by
Dr. C. Divya, B.E., M.E., Ph.D., J S Rishidev
Assistant Professor ,CITE,MSU 20204012533112
CONTENTS
● ABSTRACT
● OBJECTIVES
● INTRODUCTION
● REVIEW OF LITERATURE
● DEEP LEARNING ALGORITHMS
● EVALUATION OF DNN
● RESULT AND DISCUSSION
● CONCLUSION
● FUTURE WORK
ABSTRACT

● Explored a model of an intrusion detection system based on deep learning, and

a deep learning approach for intrusion detection using Deep neural networks
(DNN-IDS) .
● Moreover,the performance of the model in binary classification, and the number
of neurons and different learning rate impacts on the performance of the model
is studied.
● Compared it with those of, Naïve Bayes, Decision Tree, and other machine
learning methods proposed by previous researchers on the benchmark data set.
OBJECTIVES

● To find the accuracy,precision,F1 score and recall of Machine Learning

Algorithms such as Naive Bayes,Decision Tree and Logistic Regression.

● To explore the use of Deep Neural Networks to improve the accuracy of a

Network-based IDS (NIDS) and compare the results with classical Machine
Learning Algorithms.
1.INTRODUCTION
● In recent years, a large number of literatures have conducted research on the detection
performance of deep learning in IDS. However, implementation challenges are rarely
discussed. One of the most important issues is the complexity of the neural network,
including the computational requirements and the model size.
● The complexity of neural 4 networks greatly hinders the deployment of DL-based
algorithms in practical environments, especially the use of reinforcement learning and
federated learning.
● These models require continuous training after deployment, but the practical
deployment environment often does not have powerful computational resources.
Therefore, it is essential to design efficient and high-performance models for IDS.
2.REVIEW OF LITERATURE
● A highly scalable and hybrid DNN framework called scale-hybrid-IDS-AlertNet was proposed in the
study by Vinaykumar et al. [1] The framework may be used in real time to effectively monitor network
traffic to alert system administrators to possible cyberattacks. It was composed of a distributed deep
learning model with DNNs for handling and analyzing very large-scale data in real time. On NSL-
KDD, the best F-Measure for binary classification was 80.7% and 76.5% for multiclass classification.

● Yin et al.[2] proposed a model for intrusion detection using recurrent neural networks (RNNs). RNNs
are especially suited to data that are time dependent. The model consisted of forward and back
propagation stages. Forward propagation calculates the output 10 values, whereas back
propagation passes residuals accumulated to update the weights. The model consisted of 20 hidden
nodes, with Sigmoid as the activation function and Softmax as the classification function.

● Altwaijry et al.[3] developed an intrusion detection model using DNN. The proposed model consisted
of four hidden fully connected layers and was trained using NSL-KDD data set.
3.DEEP LEARNING ALGORITHMS
Hidden Markov Model (HMM)- is a statistical Markov model in which the system being modeled is assumed to be a Markov

process with unseen data. Prior research has shown that HMM analysis can be applied to identify particular kinds of malware
In this technique, a Hidden Markov Model is trained against known malware features (e.g., operation code sequence) and
once the training stage is completed, the trained model is applied to score the incoming traffic. The score is then contrasted to
a predefined threshold, and a score greater than the threshold indicates malware.

Support Vector Machines (SVM) - is a discriminative classifier defined by a splitting hyperplane. SVMs use a
kernel function to map the training data into a higher-dimensioned space so that intrusion is linearly classified.
SVMs are well known for their generalization capability and are mainly valuable when the number of attributes is
large and the number of data points is small.
3.DEEP LEARNING ALGORITHMS (Cont.)
Artificial Neural Network -An Artificial Neural Network (ANN) is an information processing paradigm that is inspired the
brain. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or
data classification, through a learning process. Learning largely involves adjustments to the synaptic connections that exist
between the neurons. An artificial neural network has three or more layers that are interconnected. The first layer consists of
input neurons. Those neurons send data on to the deeper layers, which in turn will send the final output data to the last output
layer.

RECURRENT NEURAL NETWORK-Recurrent neural networks recognize data's sequential characteristics and use
patterns to predict the next likely scenario. RNNs are used in deep learning and in the development of models that simulate
neuron activity in the human brain. They are especially powerful in use cases where context is critical to predicting an
outcome, and are also distinct from other types of artificial neural networks because they use feedback loops to process a
sequence of data that informs the final output. These feedback loops allow information to persist.
4.EVALUATION OF DNN
● Deep neural networks (DNNs) are Artificial Neural Network (ANN) with a multi-layered structure comprised within the
input-output layers. The extension of conventional artificial neural networks is deep neural networks.

● While traditional machine learning algorithms are linear, deep neural networks are stacked in increasing hierarchy
of complexity as well as abstraction. Each layer applies a nonlinear transformation onto its input and creates a
statistical model as output from what it learns. In simple terms, the input layer is received by the input layer and
passed onto the first hidden layer.
● These hidden layers perform mathematical computations on our inputs. One of the challenges in creating neural
networks is deciding the hidden layers’ count and the count of the neurons for each layer.
● Each neuron has an activation function which is used to standardize the output from the neuron. The “Deep” in
Deep learning refers to having more than one layer which is hidden.
● The output layer returns the output data. Until the output has reached an acceptable level of accuracy, epochs are
continued.
4.EVALUATION OF DNN(cont.)
4.1.Backpropagation

● Neural Networks are able to learn the desired function using big amounts of data and an iterative algorithm called backpropagation.

● Backpropagation, short for "backward propagation of errors," is an algorithm for supervised learning of artificial neural
networks using gradient descent. Given an artificial neural network and an error function, the method calculates the gradient of the
error function with respect to the neural network's weights.

● We feed the network with data, it produces an output, we compare that output with a desired one (using a loss function) and we re
adjust the weights based on the difference. And repeat. And repeat.

● The adjustment of weights is performed using a non-linear optimization technique called stochastic gradient descent.

The backpropagation algorithm consists of two phases:

1.The forward pass where our inputs are passed through the network and output predictions obtained (also
known as the propagation phase).

2. The backward pass where we compute the gradient of the loss function at the final layer (i.e., predictions
layer) of the network and use this gradient to recursively apply the chain rule to update the weights in our
network (also known as the weight update phase).
4.EVALUATION OF DNN(cont.)
4.2. Dataset description

● Since 1999, KDD’99 has been the most widely used data set for the evaluation of anomaly detection
methods. This data set is prepared by Stolfo et al and is built based on the data captured in DARPA’98
IDS evaluation program.
● DARPA 98 is about 4 gigabytes of compressed raw (binary) tcpdump data of 7 weeks of network
traffic, which can be processed into about 5 million connection records, each with about 100 bytes.
● The two weeks of test data have around 2 million connection records.
● KDD training dataset consists of approximately 4,900,000 single connection vectors each of which
contains 41 features and is labeled as either normal or an attack, with exactly one specific attack type.
4.EVALUATION OF DNN(cont.)
4.2.Dataset description

The simulated attacks fall in one of the following four categories:

1) Denial of Service Attack (DoS): is an attack in which the attacker makes some computing or memory resource too busy or
too full to handle legitimate requests, or denies legitimate users access to a machine.

2) User to Root Attack (U2R): is a class of exploit in which the attacker starts out with access to a normal user account on the
system (perhaps gained by sniffing passwords, a dictionary attack, or social engineering) and is able to exploit some
vulnerability to gain root access to the system.

3) Remote to Local Attack (R2L): occurs when an attacker who has the ability to send packets to a machine over a network
but who does not have an account on that machine exploits some vulnerability to gain local access as a user of that machine.

4) Probing Attack: is an attempt to gather information about a network of computers for the apparent purpose of
circumventing its security controls.
4.EVALUATION OF DNN(cont.)
4.3. PROPOSED ARCHITECTURE

Fig4.3 Proposed Architecture

5.RESULT AND DISCUSSION
● The KDDCup-’99’ dataset was fed into classical ML algorithms. After the training is
completed, all models were compared for f1-score, accuracy, recall and precision with the
test dataset.
● The scores for the accuracy of classical ML algorithms have been compared in detail in
Fig.5.1, and the Precision, Recall, F1 Score of the classical machine learning algorithms
are compared in Fig.5.2
● It is clear that the Naïve Bayes Algorithm has given the Highest Accuracy rate and the
DNN layer 3 gives an higher accuracy than the other classical machine learning
algorithms.
Fig.5.1. Accuracy of Classical Machine learning Algorithms
Fig.5.2 F1 score, Recall and Precision of Classical ML Algorithms
Fig.5.3. Accuracy of DNN layers
Fig.5.4. Recall values of DNN Layers
Fig.5.5. Precision and F1 Score of DNN layers
Fig.5.6. Overall Accuracy
6.CONCLUSION
● The publicly available KDDCup-’99’ dataset has been primarily used as the benchmarking tool
for the study, through which the superiority of the Deep Neural Networks (DNN) over the other
compared algorithms have been documented clearly.
● For further refinement of the algorithm, this work takes into account of DNNs with five counts of
hidden layers and it was concluded that a DNN layer 3 has been proven to be effective and
accurate of all, which has the accuracy of 0.930.
● From the empirical results of this thesis, we may claim that deep learning methods are a
promising direction towards Intrusion Detection tasks, but even though the performance on
artificial dataset is exceptional, application of the same on network traffic in the real-time which
contains more complex and recent attack types is necessary.
7.FUTURE WORK
➢ KDDCup ‘99’ and NSL-KDD Datasets are most well-known and outdated. Moreover, these
are not representative for today’s network traffic.

➢ Applying the proposed methodologies on the recent network traffic dataset such as
CICIDS2017 which is labelled based on the timestamp, source and destination IPs, source
and destination ports, protocols and attacks,

➢ A complete network topology was configured to collect this dataset which contains Modem,
Firewall, Switches, Routers, and nodes with different operating systems is essential. This will
be remained as one of significant future work direction
REFERENCES

[1] Vinayakumar R, Alazab M, Soman K, et al. Deep learning approach for

intelligent intrusion detection system. IEEE Access. 2019;7:41525–41550

[2] Yin C, Zhu Y, Fei J, et al. A deep learning approach for intrusion detection using
recurrent neural networks. IEEE Access. 2017; 5:21954–21961

[3] Altwaijry N, Alqahtani A, Al-Turaiki I. A deep learning approach for anomaly-

based network intrusion detection. In: Tian Y, Ma T, Khan MK (Eds.), First
International Conference on Big Data and Security, Nanjing, China: Springer, 2019