Professional Documents
Culture Documents
Leveraging Ferroelectric Stochasticity and In-Memo
Leveraging Ferroelectric Stochasticity and In-Memo
This is the author's version which has not been fully edited an
content may change prior to final publication. Citation information: DOI 10.1109/JXCDC.2022.3217043
the weights of the DNN model. In this work, we propose an Training data
energy-efficient defense technique that combines a Ferroelectric
Field Effect Transistor (FeFET)-based reconfigurable physically
unclonable function (PUF) with an in-memory FeFET XNOR to wk l
thwart model stealing attacks. We leverage the inherent stochas- i
wi
j
wj
k
ticity in the FE domains to build a PUF that helps to corrupt the Intellectual Property Steal weights Reverse
neural network’s weights when an adversarial attack is detected. engineer
We showcase the efficacy of the proposed defense scheme by
performing experiments on graph-neural networks (GNNs), a
particular type of DNN. The proposed defense scheme is a first Fig. 1: The setup and training of hardware NNs, either for a cloud-based
of its kind that evaluates the security of GNNs. We investigate the ML-as-a-Service model or for an on-chip neural core implementation, is
effect of corrupting the weights on different layers of the GNN on a resource-intensive and time-consuming process. This incentivizes NN IP
the accuracy degradation of the graph classification application stealing attacks. Here, the legitimate use cases are shown in green, whereas
for two specific error models of corrupting the FeFET-based
PUFs and five different bioinformatics datasets. We demonstrate the malicious attack is highlighted in red.
that our approach successfully degrades the inference accuracy
of the graph classification by corrupting any layer of the GNN the security of the deployed DNN hardware is of paramount
after a small re-write pulse. importance, the piracy of which can cause monetary loss or
Index Terms—Graph Neural Networks, Deep Neural Networks, result in the leaking of sensitive information.
Ferroelectric Field Effect Transistor (FeFET), Model stealing In particular, Graph Neural Network (GNN) is a class of
attacks, Hardware Security DNNs specifically designed to process data relationships that
can be expressed as graphs, e.g., datasets pertaining to molec-
ular chemistry and biology, social networks, and data mining,
I. I NTRODUCTION among others [5]. GNNs are typically utilized in applications
The demand for Artificial Intelligence (AI) and Machine involving non-Euclidean graph structures of various types,
Learning (ML) hardware for the edge computing paradigm including cyclic, acyclic, directed, and undirected graphs [6].
has burgeoned in recent times with the growth of the Internet They have recently gained traction because many relationships
of Things (IoT). Deep Neural Networks (DNN) are at the in the natural world occur in graph data, and Neural Networks
forefront of this revolution with applications in various do- (NNs) like Convolutional Neural Networks (CNNs) cannot
mains, including computer vision, big data, natural language process such graph data accurately. CNNs process the input
processing [1], [2] etc. However, constructing and setting data, such as images represented as tensors, and consider them
up a DNN incurs significant hardware costs and large-scale as ordered data. The change in the order of elements in a
training data, requiring considerable monetary and logistical tensor leads to a change in output of the CNN. This change
resources. Owing to this, cloud-based DNN applications and in the output with the representation order does not apply to
Machine Learning-as-a-Service (MLaaS) have become popular graphs. A graph representation does not require a fixed order;
commercial models, catering to a wide range of businesses [3], thus, the tensor-based representation is unsuitable for graphs.
[4]. Though performing complex DNN operations is computa- GNNs can process graph data irrespective of the order and are
tionally expensive, in certain scenarios like remotely deployed capable of learning the structural features of the overall graph.
IoT devices or security-critical military applications, it is
preferable to have an on-board hardware DNN processing sys-
tem. Hence, from both commercial and military standpoints, A. Hardware Security of Neural Networks
Manuscript received August 22, 2022; revised October 17, 2022; accepted Various attacks have been proposed against the confiden-
October 19, 2022. tiality of neural network (NN) systems. These attacks aim to
Likhitha Mankali is with the Department of Electrical and Computer reverse-engineer (RE) the hardware of NNs by stealing the
Engineering, Tandon School of Engineering, New York University, USA underlying model’s vital information, i.e., its weight mapping.
(email: lm4344@nyu.edu). In such attacks, an attacker queries the NN with various inputs
Nikhil Rangarajan and Ozgur Sinanoglu are with the Division and collects the corresponding output responses. Further, using
of Engineering, New York University Abu Dhabi, UAE (email: the input-output responses, an attacker can RE the weights of
nikhil.rangarajan@nyu.edu; ozgursin@nyu.edu).
Swetaki Chatterjee, Shubham Kumar, and Yogesh Singh Chauhan are with
the target network. Such attacks have been proposed against
the Department of Electrical Engineering, Indian Institute of Technology different types of DNNs. In [7]–[9], researchers have proposed
Kanpur, Kanpur, India (email: swetakic@iitk.ac.in; shubhamp@iitk.ac.in; an attack on black-box DNNs wherein they craft the inputs,
chauhan@iitk.ac.in). i.e., images to be queried in such a way that the output
Hussam Amrouch is with the Department of Computer Science, University predictions reveal the internal attributes of the underlying
of Stuttgart, Stuttgart, Germany (email: amrouch@iti.uni-stuttgart.de). network. There are different types of adversarial attacks that
aim to affect the confidentiality of GNNs. Such attacks either
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits. This is the author's version which has not been fully edited an
content may change prior to final publication. Citation information: DOI 10.1109/JXCDC.2022.3217043
aim to retrieve the important information of the training dataset (c) The attacker has access to the dataset that is part of the
or steal the GNN model itself. The attacks proposed in [10], original training dataset or is similar to the training dataset
[11] target membership inference which aims to find valid of GNN IP
data samples that are used for training, thus affecting the Our contributions of this work are as follows:
confidentiality of the training dataset. In [12], a link stealing (1) We exploit the randomness in FeFET devices to augment
attack has been proposed that aims to predict the existence the security of NN systems by amalgamating it with the in-
of links between two nodes in the training graph, thus, memory computation capabilities of FeFET XNOR gates.
leaking the training dataset. In [13], [14], property inference (2) We present a comprehensive analysis and modeling of the
attacks have been proposed against GNNs, which aim to infer randomness in FE domains and highlight the construction of
the properties of training datasets such as subgraphs, graph a reconfigurable PUF using this inherent randomness. This
density, etc. In [15]–[17], model extraction attacks have been FeFET-based reconfigurable PUF is pivotal to the weight
proposed against GNNs that aim to build a surrogate model corruption mechanism.
with an accuracy similar to the original GNN model that (3) To the best of our knowledge, this is the first work to
is under attack. In [16], researchers have proposed different demonstrate a defense against model piracy attacks specifically
model extractions considering different attack scenarios, such targeting GNNs.
as complete, partial, or no knowledge about the training (4) We explore the system-level implications of the GNN
dataset. However, this attack also extracts the model of GNN weight corruption on the accuracy of classification tasks and
using input and output (I/O) queries. In [15], researchers show how model piracy attacks can be foiled.
have proposed a model extraction attack that targets inductive
GNNs in an adversarial setting where they do not tamper
with the training process. The attack proposed here queries II. P RELIMINARY AND BACKGROUND
the GNN considering two scenarios – with and without the A. Introduction to Graph Neural Networks
structural information of the query graphs. In this work, we
focus on such model stealing or extraction attacks on GNNs. GNNs are rapidly gaining a large attraction and becoming
All these attacks target the software implementation of the NN the preferred network to analyze graph-based data structures.
and aim to generate adversarial examples using the knowledge GNNs use a message passing technique in which the informa-
of the extracted model. tion is passed from one node to its neighbors (i.e., connected
Several approaches have been proposed to defend against nodes). The input graph passes through various layers of the
the model extraction/stealing attacks, especially for DNNs. GNN. In each layer of the GNN, the node representation is
These techniques defend the NN architecture at the hardware- updated with the aggregation of messages from its neighbors.
level. In [19], researchers have demonstrated a technique that In this way, after passing through multiple layers, the feature
defends memristor-based NN architectures by leveraging the information of a node represents not only the node itself but its
memristor’s obsolescence effect. The continuous application of neighborhood as well. Thus, the GNN learns about the features
voltage causes an increase in memristance, which causes the of a complete graph and performs the classification. GNNs are
obsolescence effect in memristors. This solution thwarts the generally used for the applications such as graph classification,
attacker from querying the NN architecture to obtain enough node classification, and link prediction. In this work, we show
input-output pairs to replicate the target network model. But the performance of the proposed defense scheme on the Deep
this defense can be circumvented by controlling the obsoles- Graph Convolutional Neural Network (DGCNN) [18], used
cence effect through input voltage amplitude scaling. Later, for the application of graph classification. The architecture of
in [20], researchers proposed a superparamagnetic magnetic the DGCNN for graph classification is described next.
tunnel junction (s-MTJs)-based defense mechanism that lever- GNN Topology: Figure 2 demonstrates the architecture
ages the thermally-induced telegraphic switching property of of DGCNN. The DGCNN consists of three major stages. a)
s-MTJs to corrupt the weights. This defense is unlike [19], graph convolution layers (GCN), b) SortPooling layer, and c)
wherein the attacker cannot control the corruption of weights. traditional convolution and dense layers. Consider an input
However, the small retention time of s-MTJs warrants frequent graph G as an input to DGCNN ∼
with adjacency matrix A and
∼
diagonal degree matrix D where Dii = i Aij . X ∈ Rn×c is
P
refresh operations, leading to higher energy costs.
a feature vector matrix where n is the number of nodes in the
graph and c is a dimension of the feature of a single node. The
B. Key Contributions of this Work output Z of a GCN layer with non-linear activation function
In this work, we leverage two particular properties of f and weights matrix W is given by:
emerging FeFET devices to secure NN systems, namely, (i) ∼ −1
the inherent stochasticity in the spatial distribution of the Z = f (D AXW ) (1)
ferroelectric domains to corrupt the NN weights [21], and (ii)
the in-memory computation capability of FeFET to perform GCN layer gathers the information of the local neighborhood
efficient and compact XNOR-based logic-in-memory [22]. of each node and aggregates them. The updated graph with
We design a weight encryption scheme for protecting the aggregated node embeddings is propagated to the next GCN
confidentiality of hardware NNs by combining these two layer. The GCN layers help the GNN obtain the overall
effects. Specifically, we choose GNNs as the model network structural information and the nodes’ features in the graph.
to be protected, although the proposed scheme can be applied The output of the k + 1 GCN layer is given by:
to any DNN structure without loss of generality. Next, we ∼ −1
describe the threat model assumed for the target GNN. Z k+1 = f (D AZ k W k ) (2)
Threat model: Here, we outline the resources and capabilities
of the attacker considered in this work. where Z k and W k are the output and weight matrix at the pre-
th
(a) An attacker has (only) black-box access to the hardware vious GCN layer, i.e., k GCN layer. The node embeddings
GNN intellectual property (IP) that is either a part of the obtained by all GCN layers are concatenated before passing
cloud-based infrastructure or a part of an on-chip core. to the SortPooling stage. 1:K
The concatenated node embeddings
They do not have physical access to the individual internal matrix is given as Z := [Z 1 , ..., Z K ] where K is the
weights, to probe and find the programmed weights at any number of GCN layers. In Z 1:K , each row is the node’s
given instant. feature descriptor consisting of its neighborhood information,
(b) The attacker can apply any number of I/O queries to the and each column is a feature channel. SortPooling layer takes
GNN IP. an input of Z 1:K , sorts it row-wise according to the values
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits. This is the author's version which has not been fully edited an
content may change prior to final publication. Citation information: DOI 10.1109/JXCDC.2022.3217043
(a) Input graph (b) Graph convolution layers (c) SortPooling (d) 1-D convolution (e) Dense layers
Pooling R
R
R R R
S T S T R
T
T
R P Q Q T T T T
R P
S T S S S Sort S
U U S
S
R P Q P P P
T P
S Q Q Q P
U P
R P Q U U U Q
Q
Q
U
Fig. 2: The architecture of DGCNN (adopted from [18]). Consider (a) as an input graph G to the DGCNN. G is passed through the Graph convolutional
layers as shown in (b), where the neighborhood information of each node is gathered and then aggregated. The outputs of all graph convolutional layers are
concatenated. As shown in (c), the SortPooling layer sorts and performs pooling on the graph data. Finally, the output of the SortPooling layer is passed
through 1-D traditional convolutional layers and dense layers, as shown in (d) and (e), respectively.
IDS (A)
30
Count
MW=1.8 V
N+ Channel N+ N+ Channel N+ 10-7
20
High-VTH
BOX BOX 10-8 10
Low-VTH
Substrate Substrate
10-9 0
-1 0 1 -0.8 -0.4 0.0 0.4 0.8
Low- VTH High- VTH VFG (V) VTH (V)
Fig. 3: Schematic representation of the FeFET in (a) low-V TH state where Fig. 4: (a) I-V characteristics of the FeFET simulated in Sentaurus TCAD
the FE domains are polarized down, and (b) high-V TH state where the FE from Synopsys after a write voltage of ±4V. (b) Distribution of V TH for from
domains are polarized up. 0 % PFE+ (high-V TH ) to 100 % PFE+ (low-V TH ).
(a) 1.0 (b)
of Z K , and returns the first m rows of sorted input where 100
PFE+ (%)
converted to a one-dimensional matrix of size m×( 1 ck )×1 60
0.0
to pass through a third stage of the DGCNN, i.e., traditional 40
CNN and dense layers. Finally, the output of dense layers is
-0.5 20
passed through a softmax layer. The output of the softmax
layer is used to determine the class of the input graph in the -1.0
0
graph classification application. 0 20 40 60 80 100 -4.0 -2.0 0.0 2.0 4.0
PFE+ (%) WV (V)
B. FeFET Device Construction and Working Fig. 5: (a) Change of V TH with % PFE+ and (b) plot of % PFE+ against applied
write voltage (WV) for a pulse duration of 2 µs to set it into different V TH
Since the discovery of ferroelectricity in HfO2 in 2012, values and hence different stored states.
FeFETs have gained increased attention as a potential non-
volatile memory solution from industry and academia. This the case of FeFET, making it a lucrative option over other
is owing to its excellent CMOS compatibility and low-power emerging non-volatile memories. The FeFET is written by
operation [23]. The structure of a FeFET (shown in Figure 3) applying a voltage pulse of suitable magnitude and width at the
is similar to that of conventional MOSFET, except for the gate terminal. A positive pulse flips the domains downwards,
composition of the gate stack, where an extra Ferroelectric thereby reducing the V TH , whereas a negative pulse flips the
(FE) layer is incorporated on top of the oxide layer. The domains upwards and increases the V TH .
FE layer is composed of multiple domains which can be In this work, we consider FeFET devices with 50nm n-
either polarized down or up. The polarization describes the channel based on fully depleted silicon on insulator (FDSOI).
orientation of the permanent electric dipoles, which arises The gate stack comprises a 10 nm thick HfO2 layer and 0.6 nm
from the non-centrosymmetric arrangement of atoms in the SiO2 layer, and the FE layer is composed of multiple domains
FE material. Therefore, depending on the direction of the in which each domain has a size of 5 nm. The underlying
polarization, either up or down, a positive (QFIX+ ) or a negative FDSOI device has been fully calibrated against measurements
(QFIX- ) surface charge is developed respectively at the HfO2 - from 14 nm FDSOI technology node [24]. The ferroelectric
SiO2 interface. The developed surface charge controls the material parameters, namely: remnant polarization, saturation
channel conductivity and thus the electrical characteristics polarization, and coercive field, are calibrated using measured
of the FeFET. For example, in an n-channel FeFET, when QFE - VFE data from a metal-ferroelectric-metal capacitor [21].
the domains are polarized down, QFIX+ attracts more channel The resulting drain current(IDS ) against gate voltage sweep
electrons. This leads to higher conductivity and hence a lower after programming the FeFET with a pulse of ± 4 V in
threshold voltage (V TH ). Conversely, for up polarized domains, Sentaurus TCAD from Synopsys is shown in Figure 4(a).
the electrons are repelled away from the channel, and hence
conductivity in the channel decreases and V TH increases as a III. I MPLEMENTATION
result. Therefore, FeFETs can have at least two distinct V TH
states, which form the basis for the usage as a memory. A. Modeling the Inherent Randomness in FeFET
To read the stored memory state, a constant read-voltage It is noteworthy that not all the domains within the FE layer
(VREAD ) is applied to the gate of the FeFET. If the FeFET is switch at the same time [23]. Therefore, positive and negative
in the high V TH state, a low current (IOFF ) is sensed at the domains co-exist in the FE layer when a “weak” write voltage
drain terminal, whereas a high current (ION ) is sensed for the pulse (i.e., a write voltage pulse with a smaller amplitude
low V TH state. An excellent ION to IOFF ratio is observed in and/or smaller width than what is needed to completely switch
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits. This is the author's version which has not been fully edited an
content may change prior to final publication. Citation information: DOI 10.1109/JXCDC.2022.3217043
Sample 1
PFE- (Qfix-)
70 70 Iref 48% PFE+
Iref 50% PFE+
60 60
50 Vread = 0.5 V 50 Vread = 0.5 V
Count
Count
40 40
Sample 2
30 30
“0” “1” “0”
20 “1” 20
10 10
0 0
0.5 1.0 1.5 2.0 2.5 3.0 3.5 0.5 1.0 1.5 2.0 2.5 3.0 3.5
IDS (A) IDS (A)
Sample 3
10-6
50 % PFE+ 48 % PFE+
10-6
10-7 10-7
IDS (A)
IDS (A)
10-8 10-8
Sample 100 10-9 10-9
10-10
VDS = 50 mV VDS = 50 mV
10-10
10-11 10-11
-0.5 0.0 0.5 1.0 -0.5 0.0 0.5 1.0
VGS (V) VGS (V)
Fig. 6: The domain configuration and the distribution of IDS before and after reconfigure pulse. The tiles show the channel configuration and the random
distribution of the domains for a particular % PFE+ . IDS distribution for the corresponding state shows the variability due to the random distribution of domains.
all FE domains) is applied. Thus, depending on the percentage The maximum variation is observed at 50 % PFE+ , where there
of domains polarized up or down (% PFE+ ), the FeFET can be is an equal number of up and down polarized domains and thus
set into intermediate V TH states by controlling the write volt- maximum spatial variability. This variation in V TH also causes
age amplitude or pulse-width. The high V TH state corresponds a variation in IDS at a particular % PFE+ .
to 0 % PFE+ , where all the domains are polarized upwards Figure 5(a) plots the mean-V TH for each intermediate state
and vice-versa for the low V TH state (100 % PFE+ ), where with corresponding % PFE+ . In order to set the FeFET at a
all the domains are polarized downwards. For a sufficiently particular % PFE+ , we need to know the relationship between
long-channel FeFET, where domain size is much smaller than write voltage and % PFE+ . This relationship can be established
the channel dimensions, a gradual switching of the FeFET by measuring the residual polarization after a write pulse.
is observed and there can be many intermediate states of Once this value is known, it can be normalized between
polarization [21]. the minimum and maximum PFE and converted to % PFE+ .
For intermediate V TH states, the polarized domains can exist Our fixed-charge based modeling framework also measures
in any spatial orientation throughout the channel [25]. Also, the same maximum and minimum PFE , converts it into fixed
due to the stochastic switching time of the FeFET, it can charges, and distributes it among the domains according to a
not be predicted the exact domains that might be switched given % PFE+ . This allows us to link our fixed-charge based
even for the same pulse. This provides an additional source TCAD model with the already known Preisach model and
of variation in the distribution of the polarized domains along determine the write pulse magnitude and duration to set it
the channel. Thus, even for a fixed % PFE+ , we can have a into a particular % PFE+ . Figure 5(b) shows the relationship
different spatial distribution of the ferroelectric domains and between % PFE+ and write voltage for a fixed pulse width.
thus variability in the underlying channel electron density.
This can cause variability in the electrical characteristics of
the FeFET at a given intermediate state. Also, conventional B. FeFET-based Reconfigurable PUF
sources of variability in the underlying transistor, such as As discussed in the preceding section, FeFET shows varia-
Random Dopant Fluctuations, Metal Gate Work Function tion in V TH and correspondingly, the current flowing through it
Variation, and Line Edge Roughness, can cause additional even for a fixed polarization strength. This forms the basis for
variation in the electrical properties of the intermediate state. FeFET to be used as a PUF. The structure of our designed
To model the variability and randomness (inherent stochas- PUF is similar to the recently proposed 1 FeFET per cell
ticity) inside FeFET, we employ our in-hause TCAD-based reconfigurable PUF [26]. The PUF is programmed in 3 steps.
framework as in [21], [25]. It enables us to directly evaluate
the impact of random spatial fluctuation of the polarization 70 48% PFE+
through emulating the polarization charges (PFE ) with fixed Iref
50% PFE+
charges (QFIX ) at the HfO2 -SiO2 interface. In practice, each 60
52% PFE+
domain is assigned a particular QFIX depending on the polar- 50
40
residual PFE in the FE layer as 30
PFE 20
QFIX = . (3)
1.6 × 10−19 10
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits. This is the author's version which has not been fully edited an
content may change prior to final publication. Citation information: DOI 10.1109/JXCDC.2022.3217043
TABLE I: Reconfigure pulse magnitude and time to set into different % PFE+ . M-Line Output M-Line Output
% PFE+ Voltage (V) Time (ns) (a) (b)
45 -0.7 325 B A B A B
B
48 -0.7 145
49 -0.7 5 FeFET1 FeFET2 FeFET1 FeFET2
50 0 0
51 2.8 5 FeFET-based XNOR FeFET-based XNOR
52 2.8 190 storing A = 0 storing A = 1
55 2.8 no change A B FeFinFET1 FeFinFET2 Output
0 0 OFF OFF 1
TABLE II: Error probability for bitflip for reconfiguring to different % PFE+
at different VREAD 0 1 ON OFF 0
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits. This is the author's version which has not been fully edited an
content may change prior to final publication. Citation information: DOI 10.1109/JXCDC.2022.3217043
BL1
Driver / Controller
BL2 BLn FeFET
FeFET XNOR array
The m value of the SortPooling layer is set to 0.6. Further,
WL1
PUF
array
ωPUF1
ωint1=0
ωPUF1
ωint2=1
n the 1-D convolutional layers have 16 and 32 output channels,
respectively. Finally, the dense layer consists of 128 hidden
Challenge [ωfinal] units followed by a softmax layer as the output layer. Also,
Decoder
WL2
Reduction in Accuracy
40
and hence, the final weights from the XNOR operation will
35
be different from the original golden weights. This weight
30
corruption ensures that the attacker is unable to steal the GNN
25
IP (weight mapping). 20
We re-program the FeFET XNOR cell array and the FeFET 15
PUF after an attack as follows. We first retrieve the golden 10
weights ωf that are stored in a tamper-proof memory [29] and ENZYMES PROTEINS D&D MUTAG NCI1
XNOR them with the re-rolled PUF weight array i.e., ωPUF new .
This gives us the ωint new values, which are then updated in the Fig. 10: Reduction in the accuracy of GNN calculated over ten trials for all
FeFET XNOR memory cells, by setting them in high or low the considered bioinformatics datasets. The weights in all the GNN layers are
XNORed with the proposed FeFET-based PUF’s error model.
VT H . Now, by performing ωint new ⊙ ωPUF new , we can obtain
the golden weights back.
PROTEINS
30
Reduction in Accuracy
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits. This is the author's version which has not been fully edited an
content may change prior to final publication. Citation information: DOI 10.1109/JXCDC.2022.3217043
20 20
20 20
10 10 10 10
0 0 0 0
MES EINS TAG D&D NCI1 MES EINS TAG D&D NCI1 MES EINS TAG D&D NCI1 MES EINS TAG D&D NCI1
ENZY PROT MU ENZY PROT MU ENZY PROT MU ENZY PROT MU
Error ModelA Error ModelB
Fig. 13: Reduction in accuracy (average over ten trials) for corruption of weights separately across GNN layers (first and second GCN layers, first 1-D
Convolutional layer, and second Dense layer) for all the considered bioinformatics datasets for two error models.
B. Experimental Results model or the layer to be corrupted based on the dataset the
GNN model is designed for.
Considering the probabilistic nature of the error model, we
report an average reduction in accuracy of GNN over ten TABLE III: Estimated runtime (ns) for all the GNN layers
trials for all the results. Figure 10 demonstrates the reduction
in accuracy of the GNN for all the five datasets considered GNN layer Runtime (ns)
above over ten trials when all the weights of the GNN GCN Layer 1 2.5
are corrupted.2 The reduction in accuracy is the difference GCN Layer 2 2.5
between accuracy obtained upon weight corruption and the GCN Layer 3 2.5
accuracy of the GNN with golden/original weights. It can be GCN Layer 4 2.5
observed that the reduction in accuracy varies with the trial. 1-D Conv. Layer 1 14.5
This is observed because of the difference in the number of 1-D Conv. Layer 2 14.5
bit flips and corrupted weights. This observation is further Dense Layer 1 6.5
justified by observing the effect of the GNN layer on accuracy Dense Layer 2 2.5
degradation, which is discussed next.
The defender should consider the time taken to corrupt
Effect of GNN layer: To observe the impact of corruption the weights along with accuracy degradation. The runtime of
of weights in each layer of the GNN, we corrupt the weights of corruption is important since the defender has to ensure the
each layer separately and obtain the corresponding accuracies. weights are corrupted before an adversary collects a sufficient
Figure 11 demonstrates the accuracy reduction in the GNN number of input-output pairs of GNN. The time taken for
output when weights of individual layers are corrupted one corruption of weights depends on the magnitude of reconfigure
by one, across the eight layers of GNN (four GCN layers, pulse of the proposed FeFET-based PUF. Thus, the magnitude
two 1-D convolutional layers, and two hidden layers) for of reconfigure pulse should be chosen in such a way that
the PROTEINS dataset. It can be observed that the accuracy results in accuracy degradation and thwarts the attacker from
degradation varies with the GNN layer, i.e., the accuracy collecting the inference of a sufficient number of queries.
degradation due to the corruption in the second, third, and Further, the magnitude of reconfigure pulse also determines
fourth GCN layers is low compared to corruption in other the GNN layer to be corrupted. Next, we discuss the esti-
GNN layers. Thus, a defender need not corrupt all the weights mated runtime of the GNN model considered in this work.
of the GNN and can instead choose a particular layer to be As mentioned above, we consider an in-memory compute
corrupted, which lowers the power consumption. architecture of GNN, which is built using an array of multiply
Effect of error model: As described above, we consider and accumulate (MAC) instances. Table III reports runtime
two error models for the bit flipping or corruption of weights. required for operations of each GNN layer. We consider a
To observe the effect of the error model, we compare the fixed size of MAC array, i.e., 128x128, and runtime for the
accuracy degradation between the two error models for all the operations of a single cycle of MAC array as 1ns. A defender
GNN layers in the ENZYMES dataset as shown in Fig. 12. can set the magnitude of reconfigure pulse of the PUF based
There is a difference in the accuracy reduction between the on each layer’s runtime.
two error models for a considered layer. There is no particular
trend observed between the error models. In some layers, Error C. Detecting and Thwarting Physical Attacks
ModelA has a higher accuracy reduction than Error ModelB,
whereas it is the opposite for the remaining layers. Accuracy Physical attacks can be detected in the proposed solution
degradation depends on the weights’ value because the trend using resistance/capacitance sensor arrays [31] or cryptograph-
varies for the GNN layers between Error ModelA and Error ically secure mesh structures [32]. To deter an attacker from
ModelB. frequently querying the GNN, a mesh shield can be placed
over the input-output terminals of the FeFET array. Any
Effect of dataset: Along with the difference in accuracy attempt to apply inputs through external leads will alter the
degradation with the GNN layer and error model, we also data bit sequence through the mesh wires, thus detecting the
observe the variation in accuracy degradation with the dataset. incursion.
Figure 13 demonstrates the results of the accuracy degradation We note that physical incursions like cold-boot attacks are
for the datasets - ENZYMES, PROTEINS, MUTAG, D&D, dependent on the delay between the logical turn-off of the
and NCI1 for the first and second GCN layers, first 1-D memory cell and the time it takes to physically erase its
convolutional layer, and second Dense layer. This variation remnant state [33]. Attackers can further increase this latency
is observed because of the change in weights of the GNN using cryogenic cooling to reduce the data entropy. In this
model with the dataset. Thus, a defender can choose an error scenario, the defender could try to erase all the IP information
(stored weights) upon attack detection. However, data erasure
2 Note that all the results of accuracy degradation have been calculated over incurs a write time penalty (O(µs)), which is much larger than
ten trials. the miniscule time taken to corrupt the FeFET PUF array in the
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Journal on Exploratory Solid-State Computational Devices and Circuits. This is the author's version which has not been fully edited an
content may change prior to final publication. Citation information: DOI 10.1109/JXCDC.2022.3217043
proposed scheme (5 ns). Hence, attempting data erasure could VI. ACKNOWLEDGEMENT
still leave the attacker with ample time to obtain enough input- This work is supported partly by the Center for Cyber
output data, whereas the FeFET PUF-based weight corruption Security (CCS) at New York University Abu Dhabi (NYUAD).
will thwart such attacks. Furthermore, the authors would like to thank Kai Ni from
the Rochester Institute of Technology and Simon Thomann
D. Evaluation against Model Extraction Attack from the University of Stuttgart for their valuable help in FE
modeling.
Here, we firstly discuss the methodology of the considered
attack in [17] and evaluate the proposed obfuscation scheme
against it. The steps of the attack in [17] are as follows. R EFERENCES
(i) The attacker chooses a random network and default [1] K. He et al. Deep residual learning for image recognition. In CVPR,
pp. 770–778, 2016.
weights/connections as the starting point. [2] Y. Goldberg. A primer on neural network models for natural language
(ii) The attacker then repeatedly queries the golden GNN processing. Journal of Artificial Intelligence Research, 57:345–420,
model to build an I/O dataset. 2016.
[3] X. Zhang et al. A survey on privacy inference attacks and defenses in
(iii) After a sufficient number of I/O pairs are obtained, cloud-based deep neural network. Computer Standards & Interfaces,
the random network is trained with them to be almost pp. 103672, 2022.
similar to the original network. However, the individual [4] M. Xue et al. DNN intellectual property protection: Taxonomy, attacks
and evaluations. In GLSVLSI, pp. 455–460, 2021.
weights/connections inside this newly trained network will [5] F. Scarselli et al. The graph neural network model. IEEE transactions
be vastly different than the original network, even though on neural networks, pp. 61–80, 2008.
their I/O behavior is very similar. [6] J. Zhou et al. Graph neural networks: A review of methods and
applications. AI Open, 1:57–81, 2020.
[7] S. J. Oh et al. Towards reverse-engineering black-box neural networks.
In ICLR, 2018.
Error ModelA Error ModelB [8] M. Juuti et al. PRADA: Protecting against DNN model stealing attacks.
In EuroS&P, pp. 512–527, 2019.
80 80 [9] N. Papernot et al. Practical black-box attacks against machine learning.
No corruption No corruption [10] In AsiaCCS, pp. 506–519, 2017.
Accuracy (%)
Accuracy (%)
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/