Information Sciences 619 (2023) 249–262

Contents lists available at ScienceDirect

Information Sciences
journal homepage: www.elsevier.com/locate/ins

Black-box attacks against log anomaly detection with

adversarial examples
Siyang Lu a,⇑, Mingquan Wang b, Dongdong Wang c, Xiang Wei b, Sizhe Xiao d, Zhiwei Wang b,
Ningning Han a, Liqiang Wang c
a
School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China
b
School of Software Engineering, Beijing Jiaotong University, Beijing, China
c
Department of Computer Science, University of Central Florida, Orlando, FL, USA
d
Beijing Research Institute of Automation for Machinery Industry, Beijing, China

a r t i c l e i n f o a b s t r a c t

Article history: Deep neural networks (DNNs) have been widely employed to solve log anomaly detection
Received 26 March 2022 and outperform a range of conventional methods. They have attained such striking success
Received in revised form 27 October 2022 because they can usually explore and extract semantic information from a large volume of
Accepted 4 November 2022
log data, which helps to infer complex log anomaly patterns more accurately. Despite its
Available online 9 November 2022
success in generalization accuracy, this data-driven approach can still suffer from a high
vulnerability to adversarial attacks, which severely limits its practical use. To address this
Keywords:
issue, several studies have proposed anomaly detectors to equip neural networks to
Log analysis
Big data
improve their robustness. These anomaly detectors are built based on effective adversarial
Anomalous detection attack methods. Therefore, effective adversarial attack approaches are important for devel-
Deep learning oping more efficient anomaly detectors, thereby improving neural network robustness. In
this study, we propose two strong and effective black-box attackers, an attention-based
and a gradient-based attacker, to defeat three target systems: MLP, AutoEncoder, and
DeepLog.
Our approach facilitates the generation of more effective adversarial examples with the
help of the analysis of vulnerable logkeys. The proposed attention-based attacker leverages
attention weights to achieve vulnerable logkeys and derive adversarial examples, which
are implemented using our previously developed attention-based convolutional neural
network model. The proposed gradient-based attacker calculates gradients based on
potential vulnerable logkeys to seek an optimal adversarial sample. The experimental
results showed that these two approaches significantly outperformed the state-of-the-
art attacker model log anomaly mask (LAM). In particular, owing to its optimization, the
proposed gradient-based attacker approach can significantly increase the misclassification
rate on three target models, yields a 70% successful attack rate on DeepLog and greatly
exceeds the baseline by 52%.
Ó 2022 The Author(s). Published by Elsevier Inc. This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

⇑ Corresponding author.
E-mail address: sylu@bjtu.edu.cn (S. Lu).

https://doi.org/10.1016/j.ins.2022.11.007
0020-0255/Ó 2022 The Author(s). Published by Elsevier Inc.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

1. Introduction

The data explosion has resulted in the wide usage of big data technologies to solve a number of practical problems. This
rapid growth demands support from highly reliable platforms owing to security concerns. Among several approaches to
enhance reliability, log analysis has been conducted to help detect potential issues due to being an effective approach.
Recently, owing to the success of deep neural networks (DNNs) in many application domains [1–5], several approaches inte-
grated with DNNs have been proposed to improve the log analysis performance and achieve significant performance gains
[6,7].
However, similar to other DNNs approaches, network security remains a critical concern in practical applications [8,9].
For example, adversarial attacks on DNNs can easily degrade the model and cause severe data breach problems. Several
approaches have been proposed to develop robust networks, such as [10–13], which can protect the networks from security
issues. Although they have successfully addressed robustness to some extent, there are still more challenging problems, such
as log anomaly attack, which remains unsolved. To tackle these challenges, a powerful adversarial attacker is critical to
reveal the vulnerability of neural networks and improve their robustness, which is a pressing concern in big data processing.
Among several attack strategies, this study focuses on the perturbation of anomalous log sessions. This scheme fools the
target models to identify anomalous logs as normal. It works effectively in that, compared to normal sessions, anomalous
sessions contain more information that reflects irregular behaviors during the system process. Hence, the neglect of anoma-
lous session logs owing to adversarial attacks can result in an accumulation of irregular behaviors, causing severe damage to
the system process.
In this paper, we propose two black-box adversarial attack approaches based on the aforementioned scheme, called the
gradient-based and attention-based attackers. We have assessed our proposed approaches on three strong log anomaly
detection models: two unsupervised and one supervised model. From comparative experiments, our proposed method with
only one key perturbation has been observed to significantly outperform state-of-the-art multiple key perturbation-based
approaches, such as LAM, on attack success metrics such as a decrease in the true positive rate of target models. In addition,
performance gain is consistently observed across all three models.
The contributions of our study are summarized as follows:

1. We propose two black-box adversarial attackers against log anomaly detection models. The gradient-based attacker is
based on gradient optimization while the attention-based attacker is based on attention weight information derived from
a prior attention network. Both attackers can identify the vulnerable logkeys in log data, effectively generate perturba-
tions, and successfully attack target models.
2. We conduct extensive experiments to evaluate the two proposed approaches across different benchmarks. Black-box
attack experiments are conducted on three benchmark target models. The evaluation results indicate, as expected, that
our proposed attackers consistently outperform relevant state-of-the-art approaches like LAM.
3. Our proposed approach takes only one key perturbation to achieve better performance than the other multiple-key per-
turbation approach. This saves perturbation cost while also being more effective.

The remainder of this paper is organized as follows. In Section 2, we survey anomalous log-detection methods and adver-
sarial attack approaches. Section 3 introduces the background information and architecture of the developed attacker and
the threat model employed. The details of the designed algorithms are presented in Section 4. Section 5 presents the exper-
imental results and analysis. Section 5.2 discusses ablation studies and briefly discusses observations. Section 6 extends our
work to real industrial data for a performance evaluation. Finally, conclusions are drawn in Section 7.

2. Related work

In this section, we briefly introduce the current deep-learning-based log analysis approaches and adversarial attack
methods.
Deep-Learning-based Log Analysis Approaches. A log file plays an prominent role in system diagnosis. Commonly, the
system log files can reflect the causes of errors in the system, which is useful for detecting hacking traces from a system
attacker. Given this merit, many traditional log analysis approaches have been proposed to detect anomalous logs more
effectively, including statistical approaches [6] and traditional machine learning-based approaches [7,14], among others.
Recently, DNNs have been incorporated as an effective approach for solving log analysis problems. It was found that deep-
learning-based approaches can capture more semantic information than traditional log analysis approaches, which helps
enhance the accuracy of log analysis. Given this advantage, a series of DNN-based approaches has been proposed that incor-
porate different neural networks into the framework to solve log analysis problems. For example, [15] incorporated a con-
volutional neural network (CNN) into a log anomaly detection framework and proposed two offline CNN-based approaches,
which achieved a significant performance gain. Compared to CNN, long short-term memory (LSTM), a recurrent neural net-
work, is more popular because it analyzes log files in a temporal manner, which is more effective. For instance, DeepLog [16]
comprehends a system log as a natural language sequence and leverages LSTM to mine log patterns, thus detecting log
anomalies when atypical logs arrive. Brown et al. [17] integrated an attention mechanism into LSTM to better interpret
250
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

the causes of anomalies in the system log. Xia et al. [18] further extended the LSTM-based approach by combining it with an
autoencoder and generative adversarial networks (GANs) to improve log anomaly detection accuracy. [19] proposes
attention-based GRU model to tackle the challenge of limited labelled data for log anomaly detection. Recently, Guo et al.
[20] developed LogBERT based on bidirectional encoder representations from transformers (BERT) [21], which is more pow-
erful due to learning the patterns of both normal and anomalous events, thereby further improving log anomaly detection.
To show the contributions of these effective approaches more clearly, we summarize this information in Table 1 for
reference.
Adversarial Example Attacks against Log Analysis. Although DNNs have shown success in adversarial training fields
[24], existing frameworks are still vulnerable and can be easily attacked by adversarial examples [25–29]. To better resolve
the vulnerability of deep models, a variety of adversarial attack approaches have been proposed to assess their robustness.
For example, [30] showed that adding imperceptible noise to an original image can easily mislead the classifier and cause
significant image recognition errors. Similar problems have been revealed and discussed in the NLP domain [31,32]. For
instance, Ebrahimi et al. [33] proposed a gradient-based method to attack text classifiers, achieving a higher attack success
rate. Cheng et al. [34] proposed a similar gradient-based method that yields a high success rate in attacks on text generators.
Wallace et al. [35] improved the gradient-based method using human-in-the-loop adversarial generation. Wallace et al. [36]
subsequently proposed several universal adversarial triggers and revealed that vulnerability can be transferred between NLP
models and datasets. In general, both gradient-based attack models and generated adversarial examples have been proven to
achieve higher attack success rates.
Log files, owing to its structural format, are vulnerable to system information disclosure and adversarial attack. Adversar-
ial samplers can be easily generated by quick analysis of log files and successfully attack file systems. Steverson et al. [22]
developed a framework to attack machine learning models which can be used to evaluate and enhance the robustness of log-
anomaly detectors. Recently, Herath et al. [23] proposed LAM, a real-time attack model for a distributed system with log files.
This model defeat the anomaly detection model by modifying multiple keys in log data, which degrades the log anomaly
detection in a distributed system. Although this work effectively performs attack, it expenses higher key perturbation cost
while yielding a lower success rate. To clearly present an overview of current adversarial attacking approaches, we summa-
rize the important methods and their contributions in Table 1.

3. Preliminaries

This section introduces data preprocessing, threat model selection, and the attacker model architecture. In the data pre-
processing subsection, the details of log file preparation are introduced for anomaly detection. In the subsection on threat
model selection, a prior network for adversarial example generation is discussed. More technical details are presented in
the subsection on attacker model architecture.

3.1. Data preprocessing

Although log files contain structural information, raw log entries from big data platforms are generated with texts which
are hardly for quantitative analysis. To quantitatively exploit the structural information, log file pre-processing is critical to
transform texts into numerical vectors. Typically, a log file contains multiple log lines, and each log line consists of a constant
part (log template) and variable parts. For example, as shown in Fig. 1, the Hadoop application yields this typical structure
line-by-line. In each line, a log template or logkey is presented as, ‘‘Task:
is done. And is in the process of committing”. In
this format, ‘‘
” denotes a placeholder of variables and is a variable part. In the example provided, the text ‘‘ attempt_14
45062781478_0019_m_000006_0 ” in the last line denotes the variable part or logvalue which may be nominal or numeric
data.
Given the above analysis, it has been determined that logkeys yield more structural patterns within log entry files, which
may be useful for log file mining. To leverage this property, we adopted the FT-Tree model [37,38] to parse log files and
extract logkeys for efficient analysis. The FT-Tree model parses log files into sentences and employs term frequency to
extract important word combinations from these sentences. These word combinations may be used as log templates. Owing

Table 1
A list of deep learning-based log analysis and adversarial attack approaches.

Approaches Authors Year Contributions

DeepLog [16] Min Du et al. 2017 A realtime LSTM-based model
ATT-LSTM [17] Brown et al. 2018 A LSTM-based model with attention mechanism
CNN [15] Lu et al. 2018 Two CNN-based models
LogGAN [18] Xia et al. 2019 A LSTM-based model with AutoEncoder and GANs
Transformer [22] Steverson et al. 2020 A transformer-based framework to improve model robustness
LogBERT [21] Guo et al. 2021 Two self-supervised training tasks with BERT
LAM [23] Herath et al. 2021 A realtime attack method

251
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

Fig. 1. Example log of Hadoop application.

to the limited logkey types, each type can be labeled with a unique number for classification analysis. Therefore, logkey and
logkey types are used as inputs and outputs, respectively, to carry out a logkey classification task.
To carry out this classification task, we prepared each log session with multiple log sentences, extracted logkeys using FT-
Tree, and then arranged these logkeys into temporal sequences for use as input data.

3.2. Threat model

To analyze the properties of the log analysis model, it is imperative to capture model behavior, such as model structure
(white-box) or input–output pairs (black-box). In the real world, it is usually difficult to achieve specifics of model architec-
ture which are highly protected. Therefore, black-box attack is more realistic setting. Our work was developed in this setting,
that is, with no prior knowledge about target models for attackers. To capture the behavior of the original model or target
model, it is necessary to leverage its input–output pairs. Usually, with these pairs of data, we can successfully invert the tar-
get model and achieve a new model called the threat model and can thus imitate the target model. With this threat model, a
more effective attack, that is, a black-box attack, can be carried out in different ways, such as adversarial example generation.
According to [23], following the black-box constraint, the generator used by the attacker can be completely different from
that used by the target model. Here, we develop a surrogate model as a threat model to generate adversarial examples. The
idea of the surrogate model was originally proposed in [39]; it aims to simplify the threat model while obtaining more effec-
tive samples. Inspired by this work, we employed a similar strategy to equip our attacker and render attacks more effective
at a lower cost.
We assume that the local database can be accessed by a potential attacker before the log entries are processed by the
target models. This assumption is practical in the real world because the security level of the local database is relatively
low. Given this assumption, offline adversarial examples can be effectively generated to perform black-box attacks. In par-
ticular, when offline log data are sufficiently large, a small perturbation in the input sequences can be considered impercep-
tible noise in the input sequence data.

3.3. Architecture

In this section, we introduce the architecture of the proposed attackers. The general pipeline is shown in Fig. 2. The attack
procedure consists of three steps. First, preprocessed log sessions are prepared and fed into a surrogate model to generate
candidate samplers. Next, the normal samplers are fed into a specific attacker, such as an attention-based or gradient-
based attacker, for adversarial sample generation. The attacker performs two functions: vulnerable logkey identification
and perturbation. Given different attackers, different schemes are used to identify vulnerable bits within the logkeys. After
identification, an attacker applies small perturbations to these bits to generate adversarial examples. Details are discussed in
the methodology section. After adversarial example generation, the samplers are sent to the target models to perform adver-
sarial attacks. Because no information on the target model is required, the entire pipeline follows a black-box attack.
Specifically, we trained a Text-CNN model that follows the logkey attention scheme stated in [15] as the surrogate model.
The model details are presented in Table 2. The convolution layer conducts convolution over the embedded vectors using
three one-layer independent convolutions. After the encoding layer, three convolutions are applied for CNN training, with
kernel sizes of 3  1, 4  1, and 5  1.

4. Methodology

In this section, we introduce more details regarding our proposed attackers. Two effective logkey-level attackers, an
attention-based attacker and a gradient-based attacker are proposed. These two attackers follow the similar general idea
to perform an attack. They identify the vulnerable logkeys in anomalous sessions and slightly perturb the anomalous log ses-
sions by dropping or replacing them to fool the model to misclassify these samples with no anomaly raised. Both attackers
act in a black-box manner, but one is based on the attention mechanism and the other is based on gradient optimization.
It must be noted that, to ensure that the attack is imperceptible, we added a constraint to the number of perturbations.
For HDFS log sessions with an average length of 25, we set 1 bit as the upper bound for the perturbation, which implies that
the modification percentage is not more than 4%. However, this is sufficiently small.
252
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

Fig. 2. General pipeline for anomalous log attack using the proposed attackers.

4.1. Attention-based attacker

In our first attacker design, we focused on improving attack success. Because this attacker leverages attention, it is called
an attention-based attacker (AA). Attention is an effective approach in deep learning. It is usually performed based on dif-
ferent weighting schemes for the input data or intermediate neural network features. Although simple, it effectively boosts
the performance of neural networks. In our proposed AA, extra weight vectors are added to the surrogate model to highlight
the important features and balance out minor noise within the data. Specifically, we add an extra weight vector in the neural
network, calculate the Hadamard product between it and intermediate features, and then pass the product matrix to the
active function. This straightforward approach results in effective feature importance clustering, which aids in anomalous
feature detection. For example, Fig. 3 illustrates the attention weight distribution. It is observed that within anomalous
log sessions, anomalous logkeys exhibit bits with larger attention weights, indicating those logkeys vulnerable to attack.
Further details of the AA pipeline are shown in Fig. 4. In addition to the attention mechanism, two more steps are
required, including vulnerable logkey identification and perturbation, to optimize the log session attack. First, the maximum
attention weights are calculated, and the corresponding logkeys are identified as vulnerable bits in the session. Next, adver-
sarial examples are generated by logkey perturbation, that is, modifying vulnerable bits within the logkeys.

Table 2
Network setting for the surrogate model of attention-based CNN. Concatenation input includes the outputs from average pooled Conv 1, 2, and 3. After input,
each convolution and the following average pooling compose one processing unit, which is separated from others. For example, Conv 1 with AvgPooling 1 is one
processing unit, and separate from Conv 2 with AvgPooling 2.

stage input output operations

Input 50  1 50  1  128 Embedding with code book size 29  128
Conv 1 50  1  128 48  1  128 Kernel = 31 Strides = 1 Padding = ‘‘VALID”
AvgPooling 1 48  1  128 1  1  128 Pooling Size = 48 Strides = 1
Conv 2 50  1  128 47  1  128 Kernel = 41 Strides = 1 Padding = ‘‘VALID”
AvgPooling 2 47  1  128 1  1  128 Pooling Size = 47 Strides = 1
Conv 3 50  1  128 46  1  128 Kernel = 51 Strides = 1 Padding = ‘‘VALID”
AvgPooling 3 46  1  128 1  1  128 Pooling Size = 46 Strides = 1
Concatenation 1  1  128  3 1  384 -
FC 1  384 12 dropout = 0.5

253
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

Fig. 3. The illustration of attention weight distribution in a log session. Each line represents a logkey in the log session, and the color symbolizes the value of
the attention weight. A deeper color corresponds to a larger weight, which indicates that the logkey is relatively anomalous, and vice versa.

Fig. 4. Overview of the proposed attention-based attacker for anomalous log attack. Attention mechanism is incorporated to optimize attack process. For
practical use, two extensions are implemented in logkey perturbation, including cosine-similarity-based replacement (AA-replace) and redundant key
dropping (AA-drop).

Vulnerable Logkey Identification. Anomalous logkeys are typically vulnerable to adversarial attacks. It is therefore crit-
ical to identify anomalous logkeys during a session for successful adversarial attacks. For AA, we optimized the anomalous
logkey search with the attention scheme in the surrogate model. The details of this process are discussed below.
The outputs of the CNN are used to calculate the attention weights. m is the number of filters in the convolution operation
and n is the number of logkey attention inputs after convolution. C nm denotes the attention input matrix, an1 denotes the
attention weight vector. The relationship between attention input C and attention output O can be described by:
O ¼ a ½ C; ð1Þ

254
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

where ½ is a penetrating product based on the Hadamard product, which is defined as an operation in which vector a mul-
tiplies the corresponding vector ci in C. Here, the attention vector a is derived from the feature of C using a lightweight CNN
(ConvNet):

a ¼ SoftmaxðConv Netðh; CÞÞ ð2Þ

In Eq. 2, h is a model parameter in the CNN, which is learnable and optimized by gradient descent using a cross-entropy
loss function. Given the analysis of the pattern in O, we choose the Mth position in a for the attack, which exhibits the largest
attention weight in a:

M ¼ arg max ai ð3Þ

i

As shown in Eq. 3, vulnerable logkey identification relies on the logkey attention weight derived from ConvNet in the sur-
rogate model. Therefore, more diverse attention weights indicate more potential vulnerable logkeys. Because ConvNet is
lightweight, we leverage it with three separate convolutions with kernel sizes of 3, 4, and 5 to generate more diverse atten-
tion results. Therefore, we can obtain three groups of solutions for vulnerable logkey identification, which increases attack
success.
Logkey Perturbation. Given the three vulnerable logkeys selected by ConvNets, perturbation is carried out based on a 1-
bit constraint. We set a hyperparameter b 2 f0; 1; 2g to select one bit for modification. A logkey with index M from the Con-
vNet is defined as the original key. Hence, the three vulnerable logkey indices can be presented as M þ b, where b takes three
different values. The perturbed bit is selected by adjusting the value of b.
Given the constraints and framework design, multiple perturbation schemes can be implemented to fulfill this attack
pipeline. We select two perturbation strategies for AA in this study: cosine-similarity replacement (AA-replace) and redun-
dant key dropping (AA-drop). These two extensions are described as follows:

1. (1) AA-replace. The selected vulnerable logkey is replaced by a logkey with the smallest semantic similarity to generate a
new sample. The semantic similarity of logkeys is calculated by cosine distance. This selection can increase attack success
because of the feature pattern between normal and anomalous logkeys. As the embedding vectors generated using Log-
key2Vec contain context information, normal and anomalous logkeys show more similar feature patterns between each
other. In contrast, there is a significant difference in the contexts between these two categories of logkeys, thus causing
adversarial examples. Thus, a replacement can easily achieve attack success. (2) AA-drop. The selected vulnerable logkey
is simply deleted to generate a new sample. As anomalous logkeys are redundant parts in normal log sessions, drop oper-
ations will not impact normal log classification, but will significantly weaken model recognition on anomalous log. Thus,
this will result in higher attack success.

4.2. Gradient-based attacker

Our next proposed attacker, the gradient-based attacker (GA), is developed based on the gradient-based attack. Although
AA is effective, its solution may not be optimal, which degrades adversarial example searches and adversely affects the attack
performance. Therefore, we leverage the surrogate model and use its gradients to optimize adversarial example search,
which follows a gradient-based adversarial attack. Although gradient-based attackers can be strong when images are inputs,
their performance is limited when inputs are sequential textual data because of their weaker spatial pattern. These sequen-
tial data cannot be directly modified by adding small perturbations to word-embedding vectors because the generated vec-
tors are usually outside the original semantic domain. Therefore, gradient-based attacks on sequential log entries are
challenging to implement. To address this challenge, we integrate a CNN with sequential textual data modeling to equip
the proposed GA.
Fig. 5 presents an overview of the GA for adversarial attacks. The GA performs adversarial attacks by leveraging the gra-
dient information in the surrogate model (our attention-based CNN). Given the gradients, the GA optimizes the vulnerable
logkey search and searches for better solutions. Moreover, it is found that one forward and backward propagation is suffi-
cient for the GA to generate more effective adversarial examples. This efficiency implies that the attacker has more potential
in practical problems where the processed data are large. In addition, similar to AA, GA modifies only one bit in the logkey
per session to meet the perturbation constraint.
Vulnerable Logkey Identification. The GA uses the gradient information of the surrogate model to optimize the vulner-
able logkey identification. The identification process is carried out based on surrogate model sensitivity analysis. Because the
gradient values can reflect the sensitivity of each parameter to the loss function, the GA leverages this property and conducts
a small perturbation on the input to locate the most vulnerable positions with the obtained gradients. This scheme not only
improves the effectiveness of the GA but also significantly reduces search costs. Compared to conventional methods, GA only
considers one bit instead of every bit in a session. The additional technical details are discussed below.
Anomalous and normal log sessions share the same structure, differing only in the extra anomalous logkeys. In terms of
normal inputs xn and the correct ground-truth label yn , the gradient rxn Lðx; xn ; yn Þ for every dimension of xn is close to zero.
This indicates that every logkey has normal behavior. However, when calculating the gradient rxa Lðx; xa ; yn Þ between
anomalous inputs xa and irrelevant normal label yn , normal bits still maintain small gradients and anomalous bits have sig-
255
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

Fig. 5. Overview of our proposed gradient-based attacker for anomalous log attack. Gradient descent mechanism is incorporated to optimize attack process.

nificantly larger gradients. This is because anomalous logkey values can affect the optimization process more. Hence, the lar-
gest gradient of loss can reflect the location of the most vulnerable part in the label, which is formulated as follows:
N ¼ arg max rxa Lðx; xa ; yn Þ ð4Þ

where N th denotes both the most anomalous logkey in a session and the most vulnerable logkey under perturbation. x is an
input vector that consists of a normal input vector xn and an anomalous input vector xa . y denotes the labels of the testing
input, which contains two values: normal label yn and anomalous label ya . x is the surrogate model parameter. Lðx; x; yÞ is
the loss function of the input x and label y.
Logkey Perturbation. The modification from the original anomalous logkey and the modified logkey can be represented
by the following equation:
v d0 ¼ v d þ rd ð5Þ
where d and d0 denote the original anomalous logkey and the modified logkey, respectively. v d denotes the original logkey
embedding vector. v d0 denotes the modified logkey-embedding vector. r d denotes the perturbation vector added to v d . The
influence of rd on the loss gradient of v d can be measured as the projection of rv d Lðx; xa ; yn Þ on rd :
@L rd
Prjrd ¼ rv d Lðx; xa ; yn Þ  ð6Þ
@v d jr d j
However, the rd with the largest projection may not be the best perturbation vector. Because d is an anomalous logkey
with a large gradient, a replacement can be regarded as an optimization through gradient descent:
@L @L rd
¼  rv d Lðx; xa ; yn Þ   rd ð7Þ
@ v d0 @ v d jr d j2
The loss gradient rv d0 Lðx; xa ; yn Þ is closest to the zero vector. Hence, we select logkey D to perform replacement with the
constraint,
 
D ¼ argmin rv d0 Lðx; xa ; yn Þ ð8Þ
By leveraging Eq. 8 as an objective, for each anomalous session, we need to calculate only a perturbation vector and a
projection for each candidate logkey v d0 after identifying the vulnerable logkey v d . The embedding vectors can be found
directly in the well-trained logkey2vec embedding layer, and the gradient information is obtained through only one back-
ward propagation.

4.3. Experiment setup

Dataset. This study adopted the dataset developed based on the Hadoop Distributed File System (HDFS) [40]. It collected
24 million lines of HDFS logs from 200 EC2 nodes. Moreover, the HDFS raw data were parsed with session windows, and each
line consisted of a unique blockId with related log keys in the parsed format. The parsed and labeled ground truth data used
in our experiment were prepared by [16]. This dataset contained an anomalous training set (1,638 parsed sessions), anoma-
lous testing set (15,200 parsed sessions), normal training set (4,855 parsed sessions), and normal testing set (553,366 parsed
sessions).
256
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

Target models. We selected three popular anomalous log detection models, including two unsupervised models (Dee-
pLog [16] and AutoEncoder [41]) and one supervised model (MLP [15]), as our target models. The settings of the three net-
works and their hyperparameters are listed in Table 3. DeepLog detects anomalies by calculating the probability of
occurrence of the next logkey for the current sliding window. AutoEncoder verifies the reconstruction error between the
original inputs and reconstructed outputs and sets a threshold to make comparisons with the reconstruction error. If the
error is larger than the threshold, the input window is identified as an anomaly. For online methods such as DeepLog and
AutoEncoder, the log sessions under detection are divided into multiple sliding windows, and the window size corresponds
to the input dimensions and observation domain of the models. For MLP, a classification probability distribution was pre-
dicted for each input log session. Table 4 presents the evaluation scores of the target models. All hyper-parameters were
well-tuned through the experiments.
Evaluation Metric. The evaluation metric in this experiment is the attack success rate (ASR), which quantifies attack
effectiveness. A higher ASR implies a stronger attacker. ASR is defined by the true positive rate (TPr) of the target model
before and after an attack, as follows:

TPr before  TPr after

ASR ¼ ; ð9Þ
TPr before

where TPrbefore and TPrafter denote the TPrs of the target model before and after an attack, respectively.
Online detection models usually use sliding windows to identify anomalies, whereas multiple malicious logkeys may
exist in one log entry. Because the entire entry is identified as an anomaly when any of these logkeys are detected in the
sliding window, the ASR cannot directly measure the attack effectiveness of these models. Hence, we propose the window
attack success rate (WASR), which is formulated as,

Sbefore  Safter
WASR ¼ ; ð10Þ
Sbefore

where Sbefore and Safter denote the total number of anomalous windows detected by the target model before and after the
attack, respectively. From the formulation, it can be observed that when we consider the entire input entry as one window
for offline detection models, WASR and ASR are numerically equivalent.

5. Experiments

We conducted experiments to validate our approach, and the results are discussed in this section. First, we introduce the
specifics of the dataset and target models in the experimental setup section. Next, we evaluate our proposed attackers, GA
and AA, on the target models with the dataset on different benchmarks. The performance of the attackers is compared with
that of the state-of-the-art LAM approach to justify its effectiveness. Here, we only select LAM for fair comparison because
there are no other adversarial evasion attacks against deep-learning-based anomaly detection on distributed system log [23].

Table 3
Network settings for the target models of anomalous log detection. Among the three models, two of them, including DeepLog and AutoEncoder, are
unsupervised models, and the other MLP model is supervised model.

Target Models Architecture (Layer with Output Size) Specifics

LSTM [input size = 1, time step = 10, window size = 10
DeepLog hidden state = 64, layers = 2]
(LSTM) FC [64  29]
Embedding [50  128]
Average Pooling [1  128]
MLP FC [128  128]
FC [128  64]
FC [64  32]
FC [32  2] Softmax for classifier
Input [1  290]
Encoder 1st FC [290  256]
Encoder 2nd FC [256  128]
Encoder 3rd FC [128  64]
Auto- Encoder 4th FC [64  32]
Encoder Decoder 1st FC [32  64]
Decoder 2nd FC [64  128]
Decoder 3rd FC [128  256] window size = 10
Decoder 4th FC [256  290] threshold = 32

257
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

Table 4
Evaluation scores of three target models.

Target Models TPr(%) FPr(%)

DeepLog 88.96 0.12
MLP 98.37 0.06
AutoEncoder 98.82 2.48

5.1. Experimental results

We compared GA and AA with black-box LAM [23], a state-of-the-art attacker that leverages reinforcement learning to
attack log anomaly detection on a distributed file system. For AA, because two schemes are implemented to fulfill this design
for practical use, these two extensions, i.e., AA-replace and AA-drop, are also evaluated.
Attack Success Rate. The ASR scores of the four attackers on the three target models are compared in Table 5. On Dee-
pLog, GA performed the best with an attack success rate over 70%, outperforming LAM significantly by 52.78%. AA-replace
outperformed LAM by 50.2% and AA-drop outperformed LAM by 43.81%. Compared with LAM, GA and the two AA strategies
significantly outperformed attack effectiveness. In particular, our attackers modify only one bit on average, but LAM modifies
1.58 bits. This implies that the proposed attacker has a higher imperceptibility.
For AutoEncoder, GA performed better than LAM, with an average modification decrease of 2.1 bits session. However,
both extensions of AA had lower ASR scores than those of LAM. The main reason is that LAM requires an average modification
of 3.1, whereas AA strategies are constrained by a 1-bit modification.
For MLP, although a black-box attack is a challenging task, GA and AA strategies still performed well. All attackers
achieved ASR scores over 55%. GA, in particular, reached a significantly higher ASR score of 60.68%.
In summary, across the three target models, GA and AA-replace performed comparably or better than the black-box LAM
method. This confirms the consistent performance gain of GA and AA. Compared to AA, GA shows a more significant perfor-
mance gain. This implies that gradient-based methods can search for more effective adversarial examples and can perform
stronger attacks. We also assessed the time consumption of our proposed strategies and LAM and found that they have very
similar performance, which indicates that our proposed strategies are competitive in real-time tasks.
Window Attack Success Rate. The experimental results for the WASR are shown in Fig. 6. For both online target models,
that is, DeepLog and AutoEncoder, the GA achieved the best performance. On DeepLog, the GA yielded a significantly higher
WASR of 58.45%, outperforming AA-replace and AA-drop by 9.04% and 12.65%, respectively. On AutoEncoder, the WASR of
the GA was 26.26%, outperforming the best AA strategy by 1.37%. The performance of the offline target model MLP on WASR
was the same as that of ASR.
Another interesting observation is that both online models showed lower WASR scores than ASR scores on the HDFS data-
set. This indicates that there are multiple anomalous windows in some log sessions, and a 1-bit perturbation has a limited
effect on these sessions. For instance, 41.55% of anomalous windows were concentrated in 29.64% of the anomalous sessions.

5.2. Ablation study

We conducted an ablation study to evaluate the proposed attackers further. The study was conducted using two compo-
nents: vulnerable logkey identification and logkey perturbation. For AA or GA, we assessed the effectiveness of vulnerable
logkey identification by comparing it with random search and that of logkey perturbation by comparing the proposed
schemes with random dropping. For the base model, we selected DeepLog with no attacks.
Ablation study on the attacker of AA. The experimental results are presented in Fig. 7, and the acronym for each case is
listed in Table 6. This indicates that the TPr of the base model increased by 4% and 3% for adversarial examples generated by
N-N and N-S, respectively. Meanwhile, only a 4% performance drop is observed for N-D. This indicates that a nontarget ran-
dom search may fail to find vulnerable logkeys, which causes ineffective logkey perturbation and attack failure. For example,
if an identified logkey is normal, it is changed to an anomalous one after perturbation, which may be more easily identified
by DeepLog, resulting in an increase in TPr. In contrast, when a vulnerable logkey is better identified, attack success is sig-
nificantly improved. For A-N, ATM-drop, and ATM-replace, TPr is reduced by 14%, 55%, and 60%, respectively, compared to

Table 5
The comparison between different black-box attack approaches on different target models based on Attack Success Rate (ASR). The best performance is
highlighted in bold.

Approach Average Modified Bit DeepLog AutoEncoder MLP

GA 1 70.36 45.70 60.68
AA-replace 1 67.78 44.89 58.63
AA-drop 1 61.39 41.18 55.43
LAM- 1.58 17.58 – –
black-box [23] 3.1 – 45.45 –

258
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

Fig. 6. The comparison among different black-box attack approaches on three target models based on Window Attack Success Rate (WASR).

the base model. This further justifies the fact that our developed vulnerable logkey identification is critical and primary to
successful attacks. Compared to A-N, ATM-drop and ATM-replace reduce TPr by 40% and 45%, respectively. This significant
decrease justifies the effectiveness of the proposed logkey perturbations. Compared to ATM-drop, ATM-replace yields a
lower TPr, which implies that replacement may boost attack success more than dropping.
Ablation study on the attacker of GA. Experimental results are presented in Fig. 8, and the acronym for each case is
explained in Table 7. Similar to AA, there was an insignificant change in TPr for N-N and N-S, but a significant drop in TPs
was observed for G-N and GAM. This indicates the fact that the proposed schemes for vulnerable logkey identification are
critical to successful attacks. Compared to G-N, GAM yields a 47% reduction in TPr. This demonstrates the importance of
our proposed logkey perturbation for an effective attack. Another interesting observation is that, compared to ATM-drop,
GAM shows an 8% reduction in TPr, which implies that gradient optimization can help boost attack success.

6. Case study

We also further challenged the attackers studied by evaluating practical problems. We applied the proposed adversarial
attack strategies to a new industry dataset and assessed their performance in a more realistic scenario. The logs were col-
lected from the Windows Event Tracing for Windows (ETW) framework, which records 135 different behaviors in the file
system. Each log entry corresponds to a label that indicates whether internal anomalies are detected. The dataset contains
4,878 anomalous log sessions and 130,411 normal log sessions. In the training stage, 8,000 log sessions were extracted as the
training set.
The experimental results are compared and listed in Table 8. We evaluated the performances of the three attack strategies
on the three target models using two metrics: ASR and WASR. For all three target models, GA achieved the best attack effec-
tiveness on ASR, with remarkable scores of 36.21%, 35.75%, and 35.16% on DeepLog, AutoEncoder, and MLP, respectively. In
particular, when attacking the AutoEncoder, GA outperformed AA-replace by 9.75%, which implies that GA is stronger than
AA. However, on the WASR scores, GA and AA-replace demonstrated very similar performances. A small difference of 0.48%
between GA and AA-replace was observed on DeepLog, and of 0.2% on AutoEncoder. The comparison among the three strate-
gies for ASR and WASR indicates that the GA has a stronger ability to locate and perturb anomalous logkeys. Given a similar
WASR, the GA can successfully attack more log sessions than the AA.

Fig. 7. Performance of our proposed AA in ablation studies.

259
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

Table 6
Acronyms for ablation study cases of Attention-based Attacker (AA). For example, N-N stands for the case with Nontarget-Random Vulnerable Logkey
Identification and Nontarget-Random Logkey Perturbation. Here Nontarget-Random Logkey Perturbation denotes random dropping.

Approach Vulnerable Logkey Identification Logkey Perturbation

Acronyms Attention-Based Nontarget-Random Semantics-Based Nontarget-Random Dropping
N-N U U
N-S U U
N-D U U
A-N U U
AA-drop U U
AA-replace U U

Fig. 8. Performance of our proposed GA in ablation studies.

Table 7
Acronyms for the ablation study cases of Gradient-based Attacker (GA). For example, N-N stands for the case with Nontarget-Random Vulnerable Logkey
Identification and Nontarget-Random Logkey Perturbation. Here Nontarget-Random Logkey Perturbation denotes random dropping.

Approach Vulnerable Logkey Identification Logkey Perturbation

Acronyms Gradient-Based Nontarget-Random Gradient-Based Nontarget-Random
N-N U U
N-G U U
G-N U U
GAM U U

Table 8
The result of different black-box attack approaches on the industry dataset. The best performance is highlighted in bold.

DeepLog AutoEncoder MLP

Approach ASR WASR ASR WASR ASR WASR
GA 36.21 40.36 35.75 51.74 35.16 35.16
AA-replace 32.14 40.84 26.00 51.94 32.33 32.33
AA-drop 30.87 34.73 25.91 48.11 32.32 32.32

7. Conclusion and future work

In this study, we have proposed two strong black-box attackers for log anomaly detection: an attention-based attacker
(AA) and a gradient-based attacker (GA). The proposed GA and AA approaches significantly increased the misclassification
rates for the three target models. The experimental results demonstrated that both proposed approaches achieved high per-
formance with a 1-bit logkey modification. AA achieved 67% ASR on DeepLog, 45% ASR on AutoEncoder, and 60% ASR on MLP.
Meanwhile, the GA achieved a higher ASR (70%) on DeepLog and significantly outperformed the state-of-the-art attacker
LAM by 52%.
In the future, more anomalous detection models will be studied as target models for black-box attack research. In addi-
tion, we will further examine the relationship between the robustness and accuracy of deep-learning-based anomalous
detection models.
260
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

CRediT authorship contribution statement

Siyang Lu: Funding acquisition, Methodology, Writing - review & editing. MingQuan Wang: Methodology, Writing -
review & editing. DongDong Wang: Validation, Writing - review & editing. Xiang Wei: Validation, Methodology, Writing
- review & editing. Sizhe Xiao: Investigation. Zhiwei Wang: Investigation, Writing - original draft. Ningning Han: Investi-
gation, Writing - original draft. Liqiang Wang: Supervision.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.

Acknowledgement

This research was supported by the National Natural Science Foundation of China (No.62006016).

References

[1] D. Wang, S. Zhang, L. Wang, Deep epidemiological modeling by black-box knowledge distillation: An accurate deep learning model for covid-19, arXiv
preprint arXiv:2101.10280.
[2] A. Bargiela, W. Pedrycz, Granular computing, Handbook on computer learning and intelligence, vol. 2, Deep Learning, Intelligent Control and
Evolutionary Computation, World Scientific, 2022, pp. 97–132.
[3] Y. Yang, W. Xing, D. Wang, S. Zhang, Q. Yu, L. Wang, Aevrnet: Adaptive exploration network with variance reduced optimization for visual tracking,
Neurocomputing 449 (2021) 48–60.
[4] X. Wei, X. Wei, X. Kong, S. Lu, W. Xing, W. Lu, Fmixcutmatch for semi-supervised deep learning, Neural Networks 133 (2021) 166–176.
[5] C. Johnson, B. Khadka, R.B. Basnet, T. Doleck, Towards detecting and classifying malicious urls using deep learning, J. Wireless Mobile Networks,
Ubiquitous Comput., Dependable Appl. (JoWUA) 11 (4) (2020) 31–48.
[6] S. Lu, B. Rao, X. Wei, B. Tak, L. Wang, L. Wang, Log-based abnormal task detection and root cause analysis for spark, in: Web Services (ICWS), 2017 IEEE
International Conference on, IEEE, 2017, pp. 389–396.
[7] S. Lu, X. Wei, B. Rao, B. Tak, L. Wang, L. Wang, Ladra: Log-based abnormal task detection and root-cause analysis in big data processing with spark,
Future Gener. Comput. Syst. 95 (2019) 392–403.
[8] X. Kong, X. Wei, X. Liu, J. Wang, S. Lu, W. Xing, W. Lu, 3lpr: A three-stage label propagation and reassignment framework for class-imbalanced semi-
supervised learning, Knowl.-Based Syst. 253 (2022) 109561.
[9] X. Wang, L. Ren, R. Yuan, L.T. Yang, M.J. Deen, Qtt-dlstm: A cloud-edge-aided distributed lstm for cyber-physical-social big data, IEEE Trans. Neural
Networks Learn. Syst.
[10] J. Yao, W. Xing, D. Wang, J. Xing, L. Wang, Active dropblock: Method to enhance deep model accuracy and robustness, Neurocomputing 454 (2021)
189–200.
[11] D. Wang, Y. Li, L. Wang, B. Gong, Neural networks are more productive teachers than human raters: Active mixup for data-efficient knowledge
distillation from a blackbox model, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 1498–1507.
[12] R. Casolare, C.D. Dominicis, G. Iadarola, F. Martinelli, F. Mercaldo, A. Santone, Dynamic mobile malware detection through system call-based image
representation, J. Wireless Mobile Networks, Ubiquitous Comput., Dependable Appl. (JoWUA) 12 (1) (2021) 44–63.
[13] J. Yang, L. Wang, S. Shakya, Modelling network traffic and exploiting encrypted packets to detect stepping-stone intrusions, J. Internet Services Inform.
Secur. (JISIS) 12 (1) (2022) 2–25.
[14] M. Komisarek, M. Pawlicki, R. Kozik, M. Choras, Machine learning based approach to anomaly and cyberattack detection in streamed network traffic
data, J. Wireless Mobile Networks, Ubiquitous Comput., Dependable Appl. (JoWUA) 12 (1) (2021) 3–19.
[15] S. Lu, X. Wei, Y. Li, L. Wang, Detecting anomaly in big data system logs using convolutional neural network, in: 2018 IEEE 16th Intl Conf on Dependable,
Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and
Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), IEEE, 2018, pp. 151–158.
[16] M. Du, F. Li, G. Zheng, V. Srikumar, Deeplog: Anomaly detection and diagnosis from system logs through deep learning, in: Proceedings of the 2017
ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 1285–1298.
[17] A. Brown, A. Tuor, B. Hutchinson, N. Nichols, Recurrent neural network attention mechanisms for interpretable system log anomaly detection, in:
Proceedings of the First Workshop on Machine Learning for Computing Systems, 2018, pp. 1–8.
[18] B. Xia, J. Yin, J. Xu, Y. Li, Loggan: a sequence-based generative adversarial network for anomaly detection based on system logs, in: International
Conference on Science of Cyber Security, Springer, 2019, pp. 61–76.
[19] L. Yang, J. Chen, Z. Wang, W. Wang, J. Jiang, X. Dong, W. Zhang, Semi-supervised log-based anomaly detection via probabilistic label estimation, in:
2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), IEEE, 2021, pp. 1448–1460.
[20] H. Guo, S. Yuan, X. Wu, Logbert: Log anomaly detection via bert, arXiv preprint arXiv:2103.04475.
[21] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint
arXiv:1810.04805.
[22] K. Steverson, J. Mullin, M. Ahiskali, Adversarial robustness for machine learning cyber defenses using log data, arXiv preprint arXiv:2007.14983.
[23] J.D. Herath, P. Yang, G. Yan, Real-time evasion attacks against deep learning-based anomaly detection from distributed system logs, in: Proceedings of
the Eleventh ACM Conference on Data and Application Security and Privacy, 2021, pp. 29–40.
[24] X. Wei, B. Gong, Z. Liu, W. Lu, L. Wang, Improving the improved training of wasserstein gans: A consistency term and its dual effect, in: International
Conference on Learning Representations, 2018.
[25] H. Xu, Y. Ma, H.-C. Liu, D. Deb, H. Liu, J.-L. Tang, A.K. Jain, Adversarial attacks and defenses in images, graphs and text: A review, Int. J. Autom. Comput.
17 (2) (2020) 151–178.
[26] X. Yuan, P. He, Q. Zhu, X. Li, Adversarial examples: Attacks and defenses for deep learning, IEEE Trans. Neural Networks Learn. Syst. 30 (9) (2019) 2805–
2824.
[27] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, A. Vladu, Towards deep learning models resistant to adversarial attacks, arXiv preprint arXiv:1706.06083.
[28] S. Narteni, I. Vaccari, M. Mongelli, M. Aiello, E. Cambiaso, Evaluating the possibility to perpetrate tunneling attacks exploiting short-message-service, J.
Internet Serv. Inform. Secur. (JISIS) 11 (3) (2021) 30–46.
[29] P. Kieseberg, J. Schantl, P. Frühwirt, E. Weippl, A. Holzinger, Witnesses for the doctor in the loop, in: International Conference on Brain Informatics and
Health, Springer, 2015, pp. 369–378.
[30] I.J. Goodfellow, J. Shlens, C. Szegedy, Explaining and harnessing adversarial examples, arXiv preprint arXiv:1412.6572.

261
S. Lu, M. Wang, D. Wang et al. Information Sciences 619 (2023) 249–262

[31] W.E. Zhang, Q.Z. Sheng, A. Alhazmi, C. Li, Adversarial attacks on deep-learning models in natural language processing: A survey, ACM Trans. Intell. Syst.
Technol. (TIST) 11 (3) (2020) 1–41.
[32] Y. Belinkov, J. Glass, Analysis methods in neural language processing: A survey, Trans. Assoc. Comput. Linguist. 7 (2019) 49–72.
[33] J. Ebrahimi, A. Rao, D. Lowd, D. Dou, Hotflip: White-box adversarial examples for text classification, arXiv preprint arXiv:1712.06751.
[34] M. Cheng, J. Yi, P.-Y. Chen, H. Zhang, C.-J. Hsieh, Seq2sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples, in:
Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34, 2020, pp. 3601–3608.
[35] E. Wallace, P. Rodriguez, S. Feng, I. Yamada, J. Boyd-Graber, Trick me if you can: Human-in-the-loop generation of adversarial examples for question
answering, Trans. Assoc. Comput. Linguist. 7 (2019) 387–401.
[36] E. Wallace, S. Feng, N. Kandpal, M. Gardner, S. Singh, Universal adversarial triggers for attacking and analyzing nlp, arXiv preprint arXiv:1908.07125.
[37] S. Zhang, W. Meng, J. Bu, S. Yang, Y. Liu, D. Pei, J. Xu, Y. Chen, H. Dong, X. Qu, et al., Syslog processing for switch failure diagnosis and prediction in
datacenter networks, in: 2017 IEEE/ACM 25th International Symposium on Quality of Service (IWQoS), IEEE, 2017, pp. 1–10.
[38] S. Zhang, Y. Liu, W. Meng, J. Bu, S. Yang, Y. Sun, D. Pei, J. Xu, Y. Zhang, L. Song, et al, Efficient and robust syslog parsing for network devices in datacenter
networks, IEEE Access 8 (2020) 30245–30261.
[39] C. Audet, J. Denni, D. Moore, A. Booker, P. Frank, A surrogate-model-based method for constrained optimization, in: 8th symposium on
multidisciplinary analysis and optimization, 2000, p. 4891.
[40] W. Xu, L. Huang, A. Fox, D. Patterson, M.I. Jordan, Detecting large-scale system problems by mining console logs, in: Proceedings of the ACM SIGOPS
22nd symposium on Operating systems principles, 2009, pp. 117–132.
[41] A. Borghesi, A. Bartolini, M. Lombardi, M. Milano, L. Benini, Anomaly detection using autoencoders in high performance computing systems, in:
Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 2019, pp. 9428–9433.

262