Thursday Seminars - IEIIT Youth

Online – 13/04/2023

“Countermeasures against adversarial machine learning

based on eXplainable and Reliable Artificial Intelligence ”
Alberto Carlevaro & Sara Narteni, Cnr-Istituto di Elettronica e di Ingegneria dell’Informazione e delle Telecomunicazioni
nome.cognome@ieiit.cnr.it
OUTLINE

• Adversarial Machine Learning: basic concepts and background

• Adversarial attacks generation
• Carlini-Wagner (CW)
• Fast-Gradient Sign Generation (FGSM)
• Jacobian-based Saliency Maps (JSMA)
• Defense strategies:
• Canonical Machine Learning
• Reliable Artificial Intelligence (AI): explainable AI-based or SVDD-based
• Practical application and performance analysis (accuracy, time and
memory consumption)

Thursday Seminars - IEIIT Youth, 13/04/2023

BACKGROUND: ADVERSARIAL MACHINE LEARNING

1 2 3
Training data ML model Prediction
poisoning corruption failure

Thursday Seminars - IEIIT Youth, 13/04/2023

ADVERSARIAL MACHINE LEARNING
Attack Concept Detection Concept

Considered attacks:
• Carlini-Wagner (CW)
• Jacobian based Saliency MAps (JSMA)
• Fast Gradient Sign Method (FGSM)
Thursday Seminars - IEIIT Youth, 13/04/2023
CARLINI WAGNER

N. Carlini and D. Wagner,

“Towards evaluating the
classifier robustness of neural
networks,” in 2017 ieee
optimal
symposium on security and
classification
privacy (sp). IEEE,
2017, pp. 39–57.

Generation of the Computational

malicious data is time can be very
very accurate long

Thursday Seminars - IEIIT Youth, 13/04/2023

 JACOBIAN BASED SALIENCY MAPS

Understand how inputs affect outputs by

N. Papernot, P. McDaniel, S. Jha, modifying samples through the most influential features and tune
M. Fredrikson, Z. B. Celik, and them to achieve the most subtle, yet detrimental, effect on
A. Swami, “The limitations of classification
deep learning in adversarial
settings,” in
2016 IEEE European symposium
on security and privacy
(EuroS&P).
IEEE, 2016, pp. 372–387

An attacker can exploit this saliency map by

targeting an adversarial class that does not
match the true class label of a given sample.

Thursday Seminars - IEIIT Youth, 13/04/2023

FAST GRADIENT SIGN METHOD

J. Goodfellow, J. Shlens, and C.

Szegedy, “Explaining and
harnessing
adversarial examples,” arXiv
preprint arXiv:1412.6572, 2014.

• adv_x : Adversarial image.

• x : Original input image.
• y : Original input label.
• 𝜖 : Multiplier to ensure the
perturbations are small.
• 𝜃 : Model parameters.
• 𝐽 : Loss.

Thursday Seminars - IEIIT Youth, 13/04/2023

DEFENSE STRATEGIES: OVERVIEW

Defensive technique through robustness enhancement outside the main

training model: definition of "adversarial regions" with minimum False Positives

Canonical Machine Learning Reliable AI

• Decision Tree • eXplainable AI-based:
• Gradient Boost • Reliability from outside
• K-Nearest Neighbors • Reliability from Inside
• Logistic Regression • LLM with Zero Error (LLM0%)
• Random Forest • SVDD-based:
• Support Vector Machine • ZeroFPRSVDD
• ExplainableSVDD

Thursday Seminars - IEIIT Youth, 13/04/2023

XAI-BASED RELIABLE AI: OUTSIDE AND INSIDE

Common algorithmic structure

Decision rules in
the if-
then format
Feature ranking

Most relevant
features
Value ranking

Combination of
the most relevant
intervals
Thresholds tuning

Adversarial region

Thursday Seminars - IEIIT Youth, 13/04/2023

XAI-BASED RELIABLE AI: OUTSIDE AND INSIDE

Reliability from Outside Reliability from Inside

• Start with the legitimate class (y = 0) • Start with the attack class (y = 1)
feature/value ranking feature/value ranking

• Find Δ∗ as: • Find Δ∗ as:

• Get the complementary to P as safety

region

Thursday Seminars - IEIIT Youth, 13/04/2023

XAI-BASED RELIABLE AI: LLM0%

Train Join Select Perturb Find

Train LLM Join in OR the m Select 𝑁𝐹𝑅 most Perturb their Find optimal
with 0% error top-covering relevant features thresholds with perturbations
for each rule rules for the quantities 𝛿 maximizing the
adversarial class, covering:
defining 𝑟Ƹ

Thursday Seminars - IEIIT Youth, 13/04/2023

SVDD BASED: ZEROFPRSVDD

Idea: execute successive SVDDs to avoid the presence of unsafe points

Thursday Seminars - IEIIT Youth, 13/04/2023

SVDD BASED: EXPLAINABLE SVDD

How can we make SVDD eXplainable ?

SVDD + LLM

Thursday Seminars - IEIIT Youth, 13/04/2023

APPLICATION SCENARIOS

• Cybersecurity: DNS tunneling detection (DNS)

target: tunneling vs no tunneling

• Automotive: collision avoidance in vehicle platooning (PLATOONING)

target: collision vs non collision

• Predictive maintenance: Remaining Useful Life estimation (RUL)

target: faulty vs healthy

Thursday Seminars - IEIIT Youth, 13/04/2023
CANONICAL ML PERFORMANCE

With hyperparameters optimization (via Optuna) https://optuna.org/

Thursday Seminars - IEIIT Youth, 13/04/2023

RESULTS: XAI-BASED RELIABLE AI

JSMA - DNS
FGSM - RUL
Thursday Seminars - IEIIT Youth, 13/04/2023
RESULTS: XAI-BASED RELIABLE AI

DNS case: better FPR on

CW attack than on JSMA
and FGSM!

LLM0%

Thursday Seminars - IEIIT Youth, 13/04/2023

RESULTS: XAI-BASED RELIABLE AI

Visual LLM rules comparison for the three adversarial attacks (DNS tunneling dataset)

Thursday Seminars - IEIIT Youth, 13/04/2023

RESULTS: SVDD-BASED RELIABLE AI

JSMA - DNS JSMA - Platooning CW-RUL

Thursday Seminars - IEIIT Youth, 13/04/2023

SCALABILITY EVALUATION

• Assessment of the detection capabilities on increasing test

set sizes, from 1% to 100% of the original size (with 1%
increments):
• Statistical analysis: mean and standard deviations of
FPR and TPR over the 100 test sets
• Runtime evaluation: CPU processing time (in seconds)
• Memory consumption: Resident Set Side (RSS), which
is related to the memory allocated to the process in RAM
Thursday Seminars - IEIIT Youth, 13/04/2023
STATISTICAL ANALYSIS: RELIABLE AI

Stable performance at the

increase of sample size
(low standard deviations)

Thursday Seminars - IEIIT Youth, 13/04/2023

TIME CONSUMPTION: DNS CASE

INSIDE OUTSIDE LLM0%

zeroFPRSVDD explainableSVDD
Thursday Seminars - IEIIT Youth, 13/04/2023
MEMORY CONSUMPTION

• Overall, canonical ML and XAI-based algorithms (python-

based) exhibited stable RSS with the increase of the size:
preallocation of memory resources

• SVDD-based methods (MATLAB-based) resulted in higher and

more variable RSS values

Thursday Seminars - IEIIT Youth, 13/04/2023

ON THE CHOICE OF THE BEST DEFENSE

Minimum FPR with good balance of FNR

DNS PLATOONING RUL

ZeroFPRSVDD (0.11, ZeroFPRSVDD (0.03, 0.97)

CW Inside (0.03, 0.55)
0.79) Inside (0.02, 0.99)
ZeroFPRSVDD (0.13, ZeroFPRSVDD (0.01, 0.01)
ZeroFPRSVDD (0.03, 0.22)
FGSM 0.86) Canonical ML (no SVM and
Inside (0.26, 0.71) LR)
All canonical (0, 0)
JSMA Inside (0.03, 0.07) Inside (0.07, 0.44) ZeroFPRSVDD (0.01, 0)
Inside (0.02, 0)

Can consider combinations of different detection approaches

Thursday Seminars - IEIIT Youth, 13/04/2023

CONCLUSIONS

• Innovative adversarial ML detection scheme: canonical ML & Reliable AI

• All methods should be considered for the problems under analysis

• Reliable AI are specifically designed to perform error control

• Native explainable AI or rule extraction from SVDD allowed to understand

useful knowledge about the adversarial behaviour

Thursday Seminars - IEIIT Youth, 13/04/2023

TO FIND OUT MORE

• I. Vaccari, A. Carlevaro, S. Narteni, E. Cambiaso and M. Mongelli, "eXplainable and

Reliable Against Adversarial Machine Learning in Data Analytics," in IEEE Access, vol. 10,
pp. 83949-83970, 2022, doi: 10.1109/ACCESS.2022.3197299.

• I. Vaccari, A. Carlevaro, S. Narteni, E. Cambiaso and M. Mongelli, "On The Detection Of

Adversarial Attacks Through Reliable AI," IEEE INFOCOM 2022 - IEEE Conference on
Computer Communications Workshops (INFOCOM WKSHPS), New York, NY, USA, 2022,
pp. 1-6, doi: 10.1109/INFOCOMWKSHPS54753.2022.9797955.

• Open data:

https://www.kaggle.com/datasets/cnrieiit/adversarial-machine-learning-dataset

Thursday Seminars - IEIIT Youth, 13/04/2023

 Thursday Seminars - IEIIT Youth
Online – 13/04/2023

Q&A

Thank you for your attention!

Alberto Carlevaro & Sara Narteni, Cnr-Istituto di Elettronica e di Ingegneria dell’Informazione e delle Telecomunicazioni
nome.cognome@ieiit.cnr.it