Machine Learning-Enhanced Detection of

Encrypted Malware Traffic Through

Graph-Based Network Analysis
By: LUCAS DAVAL-POMMIER
23142391

Abstract:

Traditional methods of intrusion detection has been struggling to identify new threats hidden within
encrypted traffic, which has made it imperative to develop more advanced methods for protecting
digital infrastructures. My project, "Machine Learning-Enhanced Detection of Encrypted Malware
Traffic Through Graph-Based Network Analysis," builds upon groundbreaking methodologies to
address these challenges. At the core of our project is the exploration of graph-based network
analysis, specifically through the study of the ST-Graph method [1]. This technique capitalizes on the
spatial and temporal characteristics of network behavior, employing graph representation learning to
map out and analyze network data. Furthermore, by employing a machine learning algorithm that is
both lightweight and statistical, alongside frequency domain analysis, our project seeks to detect
rogue devices efficiently. This approach is designed to address the shortcomings of prior models that
heavily rely on domain knowledge, aiming for high accuracy and reduced latency in identifying
threats. A significant part of our investigation also focuses on the detection of threats within encrypted
network traffic. This is achieved by adopting a broad-spectrum analysis of network flow data features,
including those derived from TLS (Transport Layer Security) handshake metadata and DNS
contextual flows [2]. Our methodology aims to distinguish between malicious and benign traffic
effectively, without compromising the integrity of encryption. This aspect is particularly crucial given
the growing use of encryption by malicious actors to conceal their activities. By integrating these
advanced techniques—graph-based analysis, machine learning algorithms, and in-depth examination
of encrypted traffic features—our project sets a new benchmark in the detection of encrypted malware
traffic.

References :
[1] - https://dl.acm.org/doi/pdf/10.1145/3545948.3545983
[2] - https://dl.acm.org/doi/epdf/10.1145/2996758.2996768