Plasma Physics and Controlled Fusion

PAPER

Disruption prediction investigations using Machine Learning tools on

DIII-D and Alcator C-Mod
To cite this article: C Rea et al 2018 Plasma Phys. Control. Fusion 60 084004

View the article online for updates and enhancements.

This content was downloaded from IP address 128.111.121.42 on 19/06/2018 at 02:47

 Plasma Physics and Controlled Fusion

Plasma Phys. Control. Fusion 60 (2018) 084004 (13pp) https://doi.org/10.1088/1361-6587/aac7fe

Disruption prediction investigations using

Machine Learning tools on DIII-D and
Alcator C-Mod
C Rea1 , R S Granetz1 , K Montes1 , R A Tinguely1 , N Eidietis2 ,
J M Hanson3 and B Sammuli2
1
MIT Plasma Science and Fusion Center, Cambridge, MA 02139, United States of America
2
General Atomics, PO Box 85608, San Diego, CA 92186-5608, United States of America
3
Columbia University, New York, NY 10027-6900, United States of America

E-mail: crea@mit.edu

Received 27 February 2018, revised 21 April 2018

Accepted for publication 25 May 2018
Published 18 June 2018

Abstract
Using data-driven methodology, we exploit the time series of relevant plasma parameters for a
large set of disrupted and non-disrupted discharges to develop a classification algorithm for
detecting disruptive phases in shots that eventually disrupt. Comparing the same methodology
on different devices is crucial in order to have information on the portability of the developed
algorithm and the possible extrapolation to ITER. Therefore, we use data from two very different
tokamaks, DIII-D and Alcator C-Mod. We focus on a subset of disruption predictors, most of
which are dimensionless and/or machine-independent parameters, coming from both plasma
diagnostics and equilibrium reconstructions, such as the normalized plasma internal inductance ℓi
and the n=1 mode amplitude normalized to the toroidal magnetic field. Using such
dimensionless indicators facilitates a more direct comparison between DIII-D and C-Mod. We
then choose a shallow Machine Learning technique, called Random Forests, to explore the
databases available for the two devices. We show results from the classification task, where we
introduce a time dependency through the definition of class labels on the basis of the elapsed
time before the disruption (i.e. ‘far from a disruption’ and ‘close to a disruption’). The
performances of the different Random Forest classifiers are discussed in terms of several metrics,
by showing the number of successfully detected samples, as well as the misclassifications. The
overall model accuracies are above 97% when identifying a ‘far from disruption’ and a
‘disruptive’ phase for disrupted discharges. Nevertheless, the Forests are intrinsically different in
their capability of predicting a disruptive behavior, with C-Mod predictions comparable to
random guesses. Indeed, we show that C-Mod recall index, i.e. the sensitivity to a disruptive
behavior, is as low as 0.47, while DIII-D recall is ∼0.72. The portability of the developed
algorithm is also tested across the two devices, by using DIII-D data for training the forests and
C-Mod for testing and vice versa.

Keywords: cross-device study, Machine Learning, disruptions

(Some figures may appear in colour only in the online journal)

1. Introduction
The physics of disruptions in present tokamak devices still
remains a challenge for the fusion community. A thorough,
physical understanding of the transition mechanisms that
drive the plasma away from a stable phase into an
unstable regime has not yet been reached. Lacking a com-
prehensive theoretical model, scientists have addressed
disruptions using advanced statistical analysis [1], and
recent efforts have seen a boosted interest in the exploita-
tion of state-of-the-art Machine Learning techniques [2] to
develop data-driven predictors for disruption avoidance or

0741-3335/18/084004+13$33.00 1 © 2018 IOP Publishing Ltd Printed in the UK

Plasma Phys. Control. Fusion 60 (2018) 084004
mitigation [3–12]. Current devices are not extremely
affected by these unforeseen discharge terminations;
nevertheless, the consequences of disruption events for
future tokamaks and reactors could be disastrous, given the
energy scale and size.
Disruption precursors can be very different, and their
phenomenology strongly depends on the analyzed device.
Inspiring work has been published so far [13], presenting
manual classifications of the chain of events that can lead to
disruptions in tokamak plasmas. From these, one could
implement advanced statistical techniques to automatically
classify such transitions and thus define a possible disruption
predictor. Other approaches foresee the incorporation of
physics-based first-principle models in a complex statistical
learning architecture [14].
These studies require an intense and expensive human
intervention in the definition of the different cause-effect
sequences that can be identified during the transition to a
disruptive end, under the most varied operational conditions,
which might indeed be very device-dependent. Most fre-
quently, these studies are not affordable in terms of either
time or human resources. The approach we propose in this
paper follows the data science paradigm of letting the algo-
rithm learn from the data, with as little as possible human
interference. Statistical inference, developed after the algo-
rithm has learned from the provided data, can provide
important physics insights on the disruption dynamics,
especially when comparing a similar predictive methodology
on two very different devices.
In this paper we present the results regarding the appli-
cation of the Random Forests algorithm [15] to two datasets
coming from two very different tokamaks, DIII-D and Alcator
C-Mod. The construction of the databases, as well as the
selection of the relevant input features for our algorithms, will
be discussed in section 2. In section 3, we will discuss and
compare in detail the behavior of two plasma signals as dis-
ruption precursors, the normalized internal inductance ℓi and
the n=1 mode amplitude. We will then thoroughly describe
the Random Forests algorithm in section 4: this is a popular
and very powerful Machine Learning algorithm, widely used
in many different applications [16–18]. The results of its
application to DIII-D and C-Mod data will be presented in
section 5. Since a binary classification scheme is adopted, one
efficient way of displaying the results is through the utiliza-
tion of a confusion matrix. The number of correctly classified
samples as well as the misclassified ones are reported, from
which we can extract several performance metrics. We will
also discuss the ranking of the relatively most important input
variables in our binary classification scheme for both the
devices. In section 6 we will also test the portability of
Random Forests across the two different devices; we will
show the results coming from the algorithm trained on DIII-D
data and tested on C-Mod and vice versa. Finally, conclusions
are drawn in section 7, where we will discuss the reasons that
led to have such different predictive capabilities on DIII-D
and C-Mod.
2
C Rea et al
2. The DIII-D and C-Mod disruption databases
The physical processes that lead up to a disruption are com-
plex [19], but based on extensive empirical experience, it is
generally believed that changes in behavior of some routinely
measured plasma parameters are correlated with the approach
of a disruption. Therefore, our database currently consists of
the time series values of ∼45 disruption-relevant signals,
sampled simultaneously throughout the duration of all 2146
plasma discharges in the 2015 campaign on DIII-D, which
includes 678 disruptions, and all 1821 plasma discharges in
the 2015 campaign on Alcator C-Mod, which includes 643
disruptions. We include data from both disruptive and non-
disruptive discharges because, in addition to predicting
impending disruptions with high accuracy, we want to avoid
predicting disruptions on discharges that will not disrupt (i.e.
false positives).
The times at which the data are sampled consist of two
distinct sets for each machine. For all shots in the DIII-D
database we sample every 25 ms, starting at t=0.100 s and
continuing until the end of the discharge. (Typical DIII-D
shot durations range from 3 to 8 s.) For disruptive shots, we
add a set of samples taken at 2 ms intervals for the 100 ms
period preceding the disruption4. For all shots in the C-Mod
database, we sample every 20 ms, starting at t=0.060 s and
continuing until the end of the discharge. (The typical C-Mod
shot duration is 2 s.) For disruptive shots, we add a set of
samples taken at 1 ms intervals for the 20 ms period preceding
the disruption. On each machine, in order to avoid overlap
between the two sampling sets, we remove the slow-sampled
points during the pre-disruption period of high-frequency
sampling.
Our choice of parameters to include in the databases is
based partly on our own tokamak operational experience, and
partly on those specified in the relevant literature [12, 20].
These include plasma parameters directly measured by diag-
nostics, such as radiated power Prad and density, as well as
those derived from EFIT equilibrium reconstructions [21],
such as q95, elongation, and ℓi. We initially included a few
plasma parameters that are explicit time derivatives, such as
dWth dt (where Wth is the stored plasma energy), but we
found that the noise in these time derivative signals was much
larger than any observed changes due to impending disrup-
tions, and therefore not useful. There are also several control-
related parameters, such as the programmed plasma current
Iprog (useful for separating data into the rampup, flattop, and
rampdown phases of the discharges), a power supply status
flag, an intentional disruption flag, etc. Many of the plasma
parameters can be cast in a normalized form, such as
Greenwald fraction and Prad/Pinput, which is useful for cross-
machine analyses.
There are a number of additional factors that were con-
sidered in the design of the database. Since the ultimate goal
is to develop a real-time disruption warning algorithm, we
4
The disruption time tD is defined to be the time of max (∣dIp dt∣), which is
typically about halfway down the decay of the plasma current, i.e. current
quench.
Plasma Phys. Control. Fusion 60 (2018) 084004
Table 1. List of signals considered for Machine Learning
applications on DIII-D and C-Mod.
Signal description Variable name
Percent error between measured and
ip_error_frac
programmed plasma cur-
rent, (Ip - Iprog ) Ip
Poloidal beta, βp betap
Greenwald density fraction, n/nG n/nG
Safety factor at 95% of minor radius, q95
q95
Normalized internal inductance, ℓi li
Radiated power fraction, Prad/Pinput prad_frac
Loop voltage, Vloop (V) Vloop
Stored plasma energy, Wth (J) Wmhd
n=1 mode amplitude,
n_equal_1_normalized
normalized to Btor
Electron temperature profile width,
normalized to plasma minor radius Te_width_normalized
-not available for C-Mod-
chose only parameters that, in principle, can be available in
real-time to the plasma control system. This precludes the use
of parameters that may be very useful for disruption warning,
but which require extensive offline analysis to derive. On
DIII-D (and several other tokamaks) a highly optimized
version of EFIT runs in real-time in the plasma control system
[22], so we are justified in including EFIT-derived data in our
database. The goal of a real-time disruption predictor intro-
duces another constraint on the data, namely the avoidance of
non-causal filtering. Several of the desired parameters, such as
Prad, are available from the MDSplus archiving system [23],
but have been processed using non-causal smoothing win-
dows. Incorporation of these data into the disruption database
could lead to incorrect identification of these parameters as
useful for disruption prediction. Therefore we have re-ana-
lyzed the offending signals to avoid or minimize non-causal
filtering.
Although real-time EFITs are done on DIII-D, we have
elected to use EFIT-derived data from full EFIT reconstruc-
tions done after the shot, since they are more accurate.
However, the standard post-shot EFITs are not done at the
higher sampling rates that we desire prior to disruptions. In
order to avoid excessive interpolation, we have rerun EFIT on
all the shots in our database, using the sampling times that we
desire. For these custom EFITs, we also reduced the non-
causal filtering of the magnetic diagnostic signals upon which
the Grad–Shafranov reconstructions are based. The data from
these custom EFITs are archived in an alternative MDSplus
EFIT tree for each shot.
The data are stored in an SQL relational database and can
be retrieved by any analysis software that supports SQL
queries, including Matlab, IDL, Python, etc. Population of the
database is done with a Matlab main program that loops
through a specified list of shots, calling Matlab subroutines to
extract and process the relevant parameter data from
MDSplus for each shot, and interpolating data at the desired
times. This architecture makes it rather easy to incorporate
additional parameters, which we continue to do occasionally.
3
C Rea et al
Even adding data from other campaign years is feasible, with
the primary effort being the need to run dedicated EFITs as
described previously.
Each record in our SQL database consists of the values of
the ∼45 parameters at a single time on a single shot. The shot
number and the time are two of the parameters (the primary
keys) in each record. The time_until_disrupt is
another parameter in each record, but it is only defined for
shots that disrupt. (For non-disruptive shots, it has a null
value, or NaN in Matlab.) The ∼45 parameters can be thought
of as columns of the database. There are typically 200–250
records (i.e. time slices) for each of the 2146 shots in the DIII-
D database, totaling nearly 0.5 million records, and contain-
ing ∼22 million parameter values, and 80–100 records
for each of the 1821 shots in the C-Mod database, totaling
0.2 million records and containing ∼7.7 million parameter
values.
For the Machine Learning studies described in the
following sections of this paper, a subset of 10 particular
parameters was chosen, based on the literature on disruption
prediction [5, 7–12]. One of the chosen plasma parameters,
the electron temperature profile width normalized to the
plasma minor radius, was not available for most of the 2015
C-Mod database and was therefore neglected from the spe-
cific Machine Learning application on C-Mod. The subset of
chosen signals, reported in table 1, contains mostly machine-
independent and dimensionless parameters, which can there-
fore enable cross-device analysis and comparisons. Each
signal’s description is given together with the associated
name of the variable as it appears in the figures and tables of
the following sections.
We report in table 2 a schematic summary of the number
of disrupted and non-disrupted discharges used for Machine
Learning applications on DIII-D and C-Mod.
For these initial studies, we have chosen to concentrate
only on the flattop phase of discharges and on shots that
disrupted during flattop. Furthermore, we did not use the data
from disruptions that were caused by hardware (power sup-
ply) failures, nor intentionally-triggered disruptions (usually
for studies of disruption mitigation). These restrictions leave
us with a set of 194 discharges that disrupted during the
flattop on DIII-D and 189 discharges that disrupted during
flattop on C-Mod. We complemented these with data from
the flattop of 1366 non-disruptive DIII-D discharges and 1160
C-Mod discharges, preferentially from the same experimental
runs. This will ensure similar operational spaces for our
analyses. Thus for each of the 10 selected parameters, we are
using a total of ∼253 000 records from the DIII-D database,
selected from a total of 1562 discharges, and ∼74 000 records
from the C-Mod database, selected from a total of 1349
discharges.
3. Univariate data analysis
Before presenting the results related to the application of
Machine Learning techniques, we present a detailed analysis
on some of the parameters of interest for both devices.
Plasma Phys. Control. Fusion 60 (2018) 084004
Table 2. Number of DIII-D and C-Mod discharges considered for
Machine Learning applications during 2015 campaigns.
Disrupted Non-disrupted
DIII-D 194 1366
C-Mod 189 1160
Machine Learning is a powerful way to discern more com-
plicated relationships, but the observations from simple con-
ventional studies of the database should be reflected in the
Machine Learning results. Therefore, we highlight differences
and similarities in the behavior of disruption-relevant plasma
signals, such as the normalized internal inductance ℓi and the
n=1 mode amplitude.
3.1. The normalized internal inductance
The available databases for Alcator C-Mod and DIII-D can
also be queried in a conventional sense to study the behavior
of individual signals prior to disruptions.
An example of this is given in figure 1, which shows the
behavior of the normalized internal inductance5, ℓi, versus the
elapsed time before the disruption event, for all the considered
flattop disruptions in this cross-device study. To reduce the
clutter of the signal’s time traces we used different greyscale
colors depending on the initial value of ℓi.
The orange-colored time traces in figure 1 (top) highlight
that for a significant fraction of DIII-D flattop disruptions ℓi
starts to increase (i.e. the current density profile peaks up)
near the disruption event. The normalized internal inductance
is seen to rise from a plateau below 1 to a value between 1.1
and 1.3 starting around −0.35 s or earlier in ∼32% of all
considered DIII-D disruptive discharges. In contrast, when
analyzing non-disruptive discharges during the flattop phase,
a similar behavior is only reproduced in about 5% of the
cases. However, it is difficult to recognize any noticeable ℓi
rise in figure 1 (bottom) for C-Mod disruptive data.
To investigate the differences in the behavior of ℓi for
C-Mod and DIII-D, we analyzed the probability histograms
for the two different groups of discharges present in both
databases: disruptions and non-disrupted shots.
We report in figure 2 the relevant histograms for the
considered discharges. The blue distributions describe ℓi
behavior during the flattop phase of all non-disruptive dis-
charges. In orange and yellow we can see the probability
histograms for disrupted discharges (always during the flattop
of the plasma current) but at different times with respect to
disruption event. The threshold in time is device-dependent,
in particular for the DIII-D case it has been chosen on the
basis of figure 1, where a bifurcation can be seen around
350 ms before the disruption event. If we consider only DIII-
D histograms in figure 2(a), it is seen that there is an overlap
of the non-disruptive probability distribution and the dis-
tribution of ℓi for all times far from the disruption event, i.e.
5
The normalized internal inductance is an EFIT-derived measure of the
peakedness of the plasma current profile.
4
C Rea et al
Table 3. Performance metrics for DIII-D and C-Mod binary
classification tasks. All the indices are limited between 0 and 1, with
1 representing optimal performances.
Accuracy Precision Recall F1 score
DIII-D 0.983 6 0.819 6 0.720 5 0.766 8
C-Mod 0.978 5 0.814 6 0.468 8 0.595 1
where tD - t > 0.35 s and tD is the time of the disruption
event. These overlapping distributions are in turn fairly well
separated from the distribution related to all those time slices
close to the disruption event, i.e. tD - t < 0.35 s. This
behavior is also seen for the Greenwald density fraction and
the width of the normalized electron temperature profile (not
shown here). By contrast, for C-Mod the ℓi distribution for
times close to the disruption in figure 2(b) overlaps much
more with the other distributions and does not show the same
degree of separation.
In order to better understand the dependence on the
threshold in time for the discrimination of sample distribu-
tions that would be far from and close to the disruption event,
we analyzed the median of ℓi distributions at different time
ranges for both databases. Results are shown in figure 3.
For the DIII-D case, shown in red in figure 3 (left), we
have divided the ℓi data into 100 ms windows based on the
time before the disruption: 0.1–0.2 s, 0.2–0.3 s, 0.3–0.4 s, and
so on until 1.0–1.1 s range. Then we have computed the
histograms for each time range. However, a plot with all 10
histograms superimposed is too crowded to be useful. So we
computed the median value of the ℓi distribution for each
of the 10 time range distributions and plotted it versus
time_until_disrupt. We can see in figure 3 that the
median value of ℓi gradually increases until about 0.4 s before
a disruption, and then increases much more rapidly. This
corroborates our choice to divide the data into the time ranges
greater than 0.35 s and less than 0.35 s before the disruption.
A similar analysis was done for C-Mod using 100 ms
time windows from 0.2 to 1.1 s, but we can see from
figure 3(left) that the median of ℓi is varying slowly in this
time frame. However, an additional sorting was done into
higher resolution 20 ms frames from 20 to 200 ms before the
disruption time. It is possible to see from the zoomed-in view
in figure 3 (right) that there is indeed a noticeable increase in
the distribution of the median of ℓi, but such rise is smaller in
magnitude and localized much closer to the disruption event if
compared to DIII-D.
3.2. The n=1 mode indicator
Tearing modes that rotate very slowly (quasi-stationary
modes), or not at all (locked modes), can grow to large
amplitude, and frequently lead to disruptions. For example,
detailed studies on JET [13] have shown that mode-locking is
involved in approximately 90% of all disruptions. Modes with
poloidal/toroidal mode numbers m/n=2/1 are known to
strongly degrade confinement, not only in DIII-D and C-Mod,
Plasma Phys. Control. Fusion 60 (2018) 084004 C Rea et al

Figure 1. Normalized internal inductance, ℓi, as a function of time before disruption during the flattop phase of all the considered disrupted
discharges. To reduce the clutter of the signal’s time traces we used different greyscale colors depending on the initial value of ℓi. For DIII-D
(top), starting from approximately −0.35 s or earlier, ℓi starts to increase on a large fraction of discharges; the orange-colored time traces help
highlighting such behavior. For C-Mod (bottom), a less obvious increase in ℓi starts ∼60−50 ms before the disruption time.

but also in other tokamaks [24–26], and are responsible for
most locked mode disruptions.
Statistical analysis of m/n=2/1 modes conducted in
[27] on a database including 22 500 discharges on DIII-D
showed that more than 18% of disruptions between 2005 and
2014 were due to locked or slowly rotating modes.
For DIII-D, an estimate of the perturbed radial field of
non-rotating modes is provided in real-time to the Plasma
Control System (PCS) ‘Alarms’ category by the difference
pairs of the integrated external saddle loops (ESLDs). These
consist of a toroidal array of six external saddle loops, posi-
tioned at 60◦ intervals around the outboard midplane, capable
of resolving modes with toroidal number 0  n  2. A
compensation matrix is used to account for the pickup of the
driven non-axisymmetric coils (I-coils and C-coils) and the
vacuum coupling. Detailed discussion on the technique
adopted for the detection of n=1 modes can be found
in [28].
The raw ESLD saddle loop signals are archived for all
discharges. However, the real-time n=1 amplitude signal is
only available and archived for those discharges that require
the PCS ‘Alarms’ category to be enabled during the experi-
ment. For the analyzed 2015 database, the n=1 amplitude
indicator was missing on ∼38% of the discharges. For these
discharges, we reconstructed an equivalent n=1 amplitude
signal by performing the same computation on the ESLDs as
done in the PCS. To speed up the analyses of thousands of
discharges, we used the recently developed tool TokSearch
[29], which automates parallel processing of discharges on
multiple nodes.

5
Plasma Phys. Control. Fusion 60 (2018) 084004 C Rea et al

Figure 2. For (a) DIII-D and (b) Alcator C-Mod discharges, probability histograms of the normalized internal inductance, ℓi, are shown for all
non-disruptive discharges (blue) and disruptive discharges, where the latter data is split into times far from (orange) and close to (yellow) the
disruption. Note that the thresholds between ‘far from’ and ‘close to’ disruption data are different for C-Mod and DIII-D.

Figure 3. Median of ℓi probability distributions, for DIII-D (red) and Alcator C-Mod (blue) discharges, extracted at different time ranges
before the disruption event. On both machines there is an increase in the median value of ℓi before disruptions, but both the warning time and
the magnitude of the increase is noticeably smaller for C-Mod. The gray box in the left plot identifies the zoomed-in region in the right plot.

Alcator C-Mod did not have functional saddle loops, so
we used the analog-integrated signals from 4 poloidal
magnetic field Bp sensors located on the internal surface of the
vacuum vessel, near the outboard midplane. These sensors are
sensitive also to rotating modes and not only to locked ones,
unlike the saddle loops used for the DIII-D analysis. The 4 Bp
sensors were all at the same poloidal location, and were tor-
oidally distributed at roughly 90° intervals. (These 4 Bp
sensors were a subset of the 104 Bp sensors used for real-time
control and for EFIT equilibrium reconstructions.) The Bp
signals were compensated for baseline offsets, integrator
drifts, and toroidal field pickup. However, the contribution of
applied non-axisymmetric magnetic fields from external error
field coils has not yet been compensated for. At each sam-
pling time we used a least-squares fit to fit the 4 signals to the
series: A1 (t ) + A2 (t ) cos (f ) + A3 (t ) sin (f ). The n=1 mode
amplitude at each time is A22 + A32 . This amplitude is then
normalized by dividing by the toroidal field at each time.
A statistical analysis, similar to the one described above
for ℓi, can also be done for the n=1 mode amplitude nor-
malized to the value of the toroidal magnetic field on-axis.
Figure 4 (top) and (bottom) show the database parameter
n_equal_1_normalized as a function of the time before
the disruption event, for all the analyzed flattop disruptions on
DIII-D and C-Mod, respectively. There is an obvious differ-
ence in the average magnitude of Bpn = 1 Btor between the two
machines, with C-Mod values being about 3× higher. Both
machines also show a general trend of increasing Bpn = 1 Btor
as the disruption time is approached.

6
Plasma Phys. Control. Fusion 60 (2018) 084004
Figure 4. For DIII-D (top) and C-Mod (bottom), n=1 mode amplitude normalized to the value of the toroidal magnetic field as a function of
time before disruption during the flattop phase for the set of disrupted discharges.
Better quantification of this behavior can be seen in the
evolving probability distributions displayed in figure 5(a)
and (b). A significantly larger fraction of impending disrup-
tions on DIII-D exhibit an increase in Bpn = 1 Btor compared to
C-Mod, and the increases tend to occur much earlier before
the disruption.
Future work will focus on removing from the reconstructed
C-Mod signal the contribution of external error fields as well as
the contribution of rotating modes, in order to have a more
direct comparison with the DIII-D n=1 mode indicator.
4. Random Forests for disruption prediction
In this paper we present the application of a supervised,
classification algorithm, called Random Forests [15], to the
7
C Rea et al
available DIII-D and Alcator C-Mod databases. We chose to
adopt the scikit-learn [30] implementation: this is an
open-source Python library for Machine Learning that we
used through the OMFIT framework [31].
The Random Forests algorithm is among the most ver-
satile advanced statistical models. The details of such algo-
rithm have already been discussed in a previous paper [18], to
which we refer for a detailed discussion of the algorithmic
methodology. It belongs to the family of ensemble learners:
the forests are grown by developing parallel sets of predictors,
thus collecting a large number of independent and identically
distributed, de-correlated decision trees [32]. The trees are
usually fully grown and the final prediction is aggregated,
using majority voting, from a very large number of trees. A
representation of a single tree in a forest is given in figure 6.
These individual learners (i.e. the trees) can be defined as
Plasma Phys. Control. Fusion 60 (2018) 084004 C Rea et al

Figure 5. For (a) DIII-D and (b) C-Mod, histograms of n_equal_1_normalized during the flattop of non-disruptive discharges (blue
distribution), and disruptive discharges, separated into far from (orange) and close to (yellow) disruption datasets.

Figure 6. Graphical depiction of an individual tree in a Random Forest, zoomed-in the first three layers. We can see that in the root node, at the
very top, we have all the bootstrapped samples (100%) with a class composition given by [0.96, 0.04] reflecting the [‘non-disruptive’,‘disruptive’]
classes population in the whole DIII-D dataset. Nodes are then branched on the basis of real values of input features; the feature chosen as the best
candidate to reduce the impurity measure at each split can be picked from a random subset of the initially available input features. This
mechanism strongly reduces the correlation among the grown trees [15]. The blue and brown colors represent the two different classes in the
binary classification task. The classes population is reported in each node. The color at each node is assigned depending on the majority of the
classes of the samples populating that node.

hierarchical data structures that are grown through a divide- particular, it is defined as:
and-conquer strategy. When dealing with a supervised clas- K
sification problem, this means that the original input space is Gini = å pˆtk (1 - pˆtk ) , (1 )
recursively partitioned until the almost complete separation of k= 1

samples belonging to different labels. Starting from a root
node, the data is split on the input feature that results in the
largest information gain, and this process is recursively
repeated until the decision tree ends with nodes as pure as
possible. For the purpose of estimating the information gain,
several equivalent metrics can be adopted; in particular we
adopt the Gini impurity measure. Such measure is differ-
entiable, and hence amenable to numerical optimization. In
where pˆtk is the proportion of class k observations in node t. For
two classes, if p is the proportion in the positive class:
Gini = 2p (1 - p ). It can be seen in figure 6 that each node is
associated to a specific impurity measurement given by the Gini
index and a Gini closer to zero indicates that the node contains
almost all samples belonging to just one of the two classes.
Random Forests have the great advantage of being a
Machine Learning model with low bias and low variance due

8
Plasma Phys. Control. Fusion 60 (2018) 084004
Figure 7. Out-Of-Bag error rate for DIII-D (red) and Alcator C-Mod
(blue) as a function of the number of estimators, i.e. the individual
trees in the forest. The grey-shaded region highlights the chosen
number of estimators (500) for which the OOB error rate stabilizes.
OOB error rate curves were scaled by their mean values to be able to
represent both curves on the same plot scale. Even though the OOB
error rate stabilizes around 300 trees, we choose a high number of
trees, following Breiman’s recommendations in [15].
to the exploitation of bootstrap aggregation (bagging [33]).
By using this model, we do not need to adopt computationally
expensive techniques to assess hyperparameter values; the
main parameter that needs to be chosen is the number of trees
in the forest. For the application reported in this paper, we
decided to build a forest of 500 decision trees (or estimators),
and this number was chosen on the basis of the Out-Of-Bag
(OOB) error rate stabilization, shown in figure 7 for both
DIII-D and C-Mod.
Tree-based models are attractive due to their accessible
interpretability: the Gini index is the key metric to extract
useful information from the algorithm, such as the estimate of
the relative importance of the predictor variables. In sci-
kit-learn, the importance measure is implemented as
described by Breiman [15, 32]. The importance metric is
generally referred to as mean decrease impurity or Gini
importance, when the Gini index is used as the impurity
measure.
To evaluate the importance of the variable Vj, all the
nodes t in which Vj appears are considered:
1 M
⎡N ⎤ section 5, it is consequently possible to separate samples
importance (Vj ) =
M
å å ⎢⎣ Nt Di (st , t ) ⎥⎦ , (2 )
m = 1 t Î jm j
where N is the total number of samples and Nt is the number
of samples reaching node t. For such nodes, the impurity
decreases (Di (st , t )) after the split st are added up, weighted
by the proportion of samples that reach the node, and then
averaged over all the trees jm (for m=1, K, M, in our case
M=500) in the forest. Table 4 in section 5 reports on the
relative importance ranking extracted from the training set
used to develop the algorithms on the different devices.
In the study presented in this paper, we aim to develop an
analogous methodology for the C-Mod and the DIII-D dis-
ruption databases. The Machine Learning approach chases the
realization of a model being able to perfectly generalize its
predictions for unseen cases [34]. Therefore, the idea is that
we need to split the available data into a training set and a test
9
C Rea et al
set in order to be able to generate correct predictions to
unseen instances. The training ensemble is then used to build
the algorithm’s rules, and the test set is used to check the
generalization capabilities and the predictive power of the
constructed model. Data from time slices of each shot are
either all included in the training or the test set, so that data
belonging to the same discharge cannot be partly found both
in the training and in the test set.
We split the available discharges into a (train/test) ratio
of (80/20)%, keeping approximately 50% of the disrupted
discharges in each of the training and the test sets. Non-
disrupted discharges are treated in the same way. The per-
formances of the trained model, when analyzed using the test
set, are typically interpreted as an accuracy metric to deter-
mine the suitability of the constructed algorithm. One com-
mon way of displaying the results, when dealing with a binary
classification task, is through the utilization of a confusion
matrix, also called a contingency table [35].
In section 5 we show the confusion matrices resulting
from the application of the Random Forests classifier to both
DIII-D and C-Mod databases. A detailed discussion about the
performance metrics is also given.
It is important to notice that Random Forests are unaware
of any temporal dependence in both training and test obser-
vations. The training samples are passed to the forest under the
 
form of (x , y)train , where x is an N-dimensional feature vector
(one value for each of the N parameters listed in table 1 ) and
y is the associated class label. The test samples are the ones
for which we ask the Random Forest to guess the correct

prediction from the input data x test .
Through the appropriate definition of the class labels in
our binary classification tasks, we can introduce temporal
information to properly interpret the results of the Random
Forests predictions on the test set. Even though Random
Forests naturally present the capability of multiclass classifi-
cations, we decided to assign two different labels to our data
samples under this supervised classification scheme. The
basic assumption is that the discharges that eventually disrupt
present a transition from a safe to a disruptive phase in the
plasma parameter space. By discriminating a pre-disruptive
phase in disruptive discharges, as further discussed in
belonging to disruptive shots on the basis of a threshold in
time that differentiates between times close to and far from
the disruption event. We therefore decided to assign the
‘disruptive’ label to those samples of disruptive discharges
that are close to the disruption event, while the samples of
disruptive discharges far from the disruption and all the
samples belonging to non-disrupted discharges are labeled as
‘far from disruption or non-disruptive’.
5. Results from binary classification using DIII-D and
C-Mod databases
The definition of the threshold in time for the discrimination
of a disruptive versus a far from disruption phase during
Plasma Phys. Control. Fusion 60 (2018) 084004 C Rea et al

disrupted discharges varies when considering DIII-D or

C-Mod data.
For DIII-D, we chose to distinguish the observations on the
basis of a threshold in time fixed at 0.35 s before the disruption
event. This particular threshold was chosen also given the uni-
variate analysis conducted on parameters such as ℓi, n/nG, or the
normalized electron temperature width, which show different
patterns after 0.35 s, as already discussed in section 3 and as
reported in figures 1 and 2 for the normalized internal inductance.
For C-Mod, we conducted a similar univariate analysis
on all the signals used for training (table 1). From the uni-
variate analysis it emerges that only ℓi shows some detectable
change in its behavior starting from ∼0.06–0.05 s (see right
plot in figure 3), while all other signals show little change
until just 3–4 ms before disruptions occur. This is far too short
of a warning time for any kind of successful mitigation and/
or avoidance action to be taken, and therefore is not of much
practical use.
To define the threshold in time for the identification of the
‘disruptive’ class on C-Mod, we trained several Random Forests
at different time thresholds. Data not reported in this paper
Figure 8. Confusion matrix for the binary classification task using
shows that the model’s performances are insensitive to the DIII-D data. The number of samples correctly detected are displayed
choice of a specific threshold. Therefore we chose to adopt as diagonal elements in green, while the misclassifications are shown
40 ms as a conservative threshold, given that mitigation tech- in red as off-diagonal terms.
niques on C-Mod are estimated to take 10–20 ms to be effective.
As a comparison, on ITER the minimal response time for
the proposed mitigation system is 30–40 ms [36], and much
longer warning times are needed to successfully execute
disruption avoidance procedures.
For both DIII-D and C-Mod cases, the trained Random
Forests reach an overall model accuracy6 greater than 97%
when tested for the capability of discriminating between
samples belonging to a ‘disruptive’ versus a ‘far from disrup-
tion or non-disruptive’ class, ffd-nd for short, in the available
discharges. Nevertheless, the results are intrinsically very dif-
ferent when comparing the individual confusion matrices
obtained in the two cases. These are reported in figures 8 and 9.
Each row in a confusion matrix represents the actual or true
class, while each column represents the predicted class.
We use as a reference the data reported in figure 8 to
define some useful terminology. In figure 8 the first row of the
matrix considers the ffd-nd samples (the negative class): more
than 35 400 of them were correctly classified as ffd-nd sam-
ples (true negatives) while the remaining ∼200 were wrongly
classified as ‘disruptive’ (false positives). The second row
considers only the samples belonging to the ‘disruptive’ class
Figure 9. Confusion matrix for the binary classification task using
(the positive class): 388 were misclassified as ffd-nd samples
Alcator C-Mod data. The number of samples correctly detected are
(false negatives), while the remaining 1000 samples were displayed as diagonal elements in green, while the misclassifications
correctly classified as time slices close to the disruption event are shown in red as off-diagonal terms.
(true positives).
The numbers representing the true positives (TP), true models accuracies are comparable (see table 3). To be able to
negatives (TN), false positives (FP) and false negatives (FN) have a direct comparison between the model’s performances
substantially differ when we compare the two confusion on DIII-D and C-Mod, it is possible to adopt different per-
matrices for DIII-D and C-Mod, even though the reported formance indicators [37] that can be extracted directly from
6
In the binary classification context, the accuracy refers to the fraction of
the confusion matrix.
TP
correct predictions normalized to the total number of available sam- We first introduce the classifier’s precision, TP + FP , i.e. the
TP + TN
ples: TP + FP + TN + FN . fraction of positive predictions weighted by the number of false

10
Plasma Phys. Control. Fusion 60 (2018) 084004 C Rea et al

Table 4. Relative variable importance for binary classifications on DIII-D and C-Mod. C-Mod ranking is reported for completeness, but given
the poor performances of the classifier, the feature importance is not related to the true disruptivity on C-Mod.
DIII-D C-Mod
Parameter Importance Parameter Importance
q95 0.241 ip_error_frac 0.168
n_equal_1_normalized 0.235 n/nG 0.155
n/nG 0.170 li 0.123
li 0.079 n_equal_1_normalized 0.114
betap 0.066 q95 0.106
Wmhd 0.059 Vloop 0.096
Te_width_normalized 0.047 Wmhd 0.083
Vloop 0.047 betap 0.081
ip_error_frac 0.034 prad_frac 0.071
prad_frac 0.020

Table 5. The table reports on Random Forests performance when
trained on one device’s data and tested on the other. Performances
are reported in terms of the F1 score. We also tested the algorithm
against different thresholds in time for the discrimination of
‘disruptive’ class labels.
Train Test Thr (s) F1 score
C-Mod DIII-D 0.35 0.275 8
C-Mod DIII-D 0.04 0.177 2
DIII-D C-Mod 0.35 0.174 6
DIII-D C-Mod 0.04 0.011 4
positive misclassifications. In other words, it represents the
ability of the classifier not to label as ‘disruptive’ a sample that
belongs to the ‘far from disruption or non-disruptive’ class.
Precision needs to be evaluated along with another metric
named recall, also called sensitivity or true positive rate: this
represents the ratio of ‘disruptive’ class instances that are
TP
correctly detected by the classifier, i.e. TP + FN .
It is often convenient to combine precision and recall into
a single metric called F1 score, which is obtained from the
harmonic mean of precision and recall: F1 = 2TP +2TP . on the other, and vice versa. Results are reported in table 5;
FP + FN
Each of the performance metrics reported in table 3
provides a piece of useful information. It is often instructive
to report all of them to understand the classifiers differences.
From table 3 it is indeed possible to notice at first glance the
different performances of the Random Forest classifier on
DIII-D and C-Mod data. In particular, the recall index and the
F1 score reveal the very poor predictive capabilities of the
classifier for C-Mod disruptive samples. The recall shows that
a Random Forests trained on C-Mod data is capable of dis-
criminating ‘disruptive’ samples in not even half of the
available observations. The F1 score instead, provides a more
global view of the correct detection of the positive class,
taking into account the global misclassification error.
As introduced in section 4, Random Forests provide a
relative importance ranking for the input features, i.e. plasma
signals, used as training set to build the model. The relative
rank of a feature reflects the relative importance of that feature
with respect to the predictability of the target variable, in our
case the discrimination of ‘disruptive’ class labels.
For DIII-D (table 4), these relative importance measures
represent a well known correlation between disruptivity,
n=1 mode activity, and the safety factor.
Instead, given the poor classifier’s performances on
C-Mod data, we cannot rely on the variable ranking reported
in table 4. Since the Random Forests algorithm is capable of
discriminating ‘disruptive’ labels in less than 50% of the
cases, the relative ranking does not reflect any correlation
between the input features and the actual disruptivity on
C-Mod. It only reflects the relatively most important vari-
ables, e.g. ip_error_frac or n/nG, in detecting less than
a half disruptive samples.
6. The cross-device analysis
In order to test the portability of the Random Forests predictive
algorithm across different devices, we selected from table 1
those variables available for both DIII-D and C-Mod data,
trained the Random Forests on one device’s dataset, tested
we also trained the algorithm using the two different thresholds
in time that we adopted on DIII-D and C-Mod for the identi-
fication of a ‘disruptive’ phase in disrupted discharges.
In this analysis we used all the available dimensionless
signals: we discarded Te_width_normalized because it
was not available on C-Mod dataset and Vloop and Wmhd,
because they are not normalized quantities.
As we can see from the F1 score, the algorithm performs
rather poorly when trained on DIII-D data and tested on
C-Mod and vice versa. Relatively better models can be
obtained using C-Mod data as training set and predicting
DIII-D samples labels. Nevertheless, the predictive cap-
abilities of all the developed models are rather insufficient.
Apart from differences in geometry, configuration or material,
the two devices are characterized by intrinsically different
timescales for the evolution of the physics processes: the
confinement time on DIII-D is of the order of 0.1 s, whereas
on C-Mod is around 0.04 s [38]; while the current relaxation
time is about 1 s for DIII-D and ∼0.2 s on C-Mod [39, 40].
These differences in timescales are likely one reason for the

11
Plasma Phys. Control. Fusion 60 (2018) 084004
poor performance. Further work shall be devoted to improve
the methodology and understand how to conciliate such
intrinsic differences.
7. Discussion and conclusions
In this paper we presented a comparative study between DIII-
D and C-Mod using Machine Learning algorithms for dis-
ruption prediction. The chosen methodology is explained in
detail in section 4: Random Forests is a very powerful and
versatile Machine Learning technique, allowing the extraction
of valuable information from the database, such as the relative
importance ranking for the input features.
We chose to adopt the same set of features for the two
machines, listed in table 1, with the exception of the electron
temperature data, which was not available on a large fraction
of 2015 C-Mod data. Both the constructed algorithms show
accurate predictions when analyzing ‘far from disruption or
non-disruptive’ samples, incurring in a very low percentage of
false positive misclassifications. When asked to identify
‘disruptive’ time slices in discharges that eventually disrupt,
the classifier’s performances differ substantially with the
C-Mod case showing percentages of correct classifications
actually worse than a random guess. To better evidence this,
we show in table 3 the recall index for both DIII-D and
C-Mod cases.
The poor performance of disruption prediction in C-Mod
may reflect the fact that some C-Mod disruptions are thought
to be caused by tiny flecks of molybdenum penetrating into
the plasma and very effectively radiating away its thermal
energy on a millisecond timescale, due to the high atomic
number. Though not reported in this paper, the analysis of the
radiated power versus the time before the disruption shows
that on a fraction of C-Mod discharges, Prad has a very rapid
increase starting a few milliseconds before the disruption
event. The molybdenum injections presumably originate from
overheated edges/corners of the molybdenum tiles that
comprise the plasma facing surface of the divertor. This is not
surprising, since the energy density is higher than all other
currently operating tokamaks. Since the B-field is also high,
the scrape-off width is smaller than all other tokamaks,
resulting in parallel heat fluxes to the divertor of order
0.5 GW m−2 near the strikepoints. This can lead to the
overheating of tile edges/corners.
In contrast, DIII-D has lower thermal energy density and
lower B-field, and therefore less heat flux on the divertor. In
addition, the plasma facing surface is graphite, not high-Z
metal and graphite in a hot plasma does not radiate as
effectively as high-Z metals. It should be noted that ITER will
have a high-Z divertor (tungsten), like C-Mod, and its thermal
energy density and B-field and parallel heat flux to the
divertor will be very similar to C-Mod, and therefore dis-
ruptions on ITER may be more like disruptions on C-Mod
than on DIII-D.
All of these differences have to be taken into account
when testing the portability of the predictive model across
DIII-D and C-Mod. Results from the cross-device analysis
12
C Rea et al
have shown the very poor predictive capabilities of the
Random Forests algorithm, training on one device’s data and
testing on the other’s (and vice versa), even when using
dimensionless parameters as input variables.
This does not bode well for the realization of a ‘uni-
versal’ classifier, capable of predicting disruptions in ITER
with sufficient warning time. Nevertheless, different strategies
can still be explored. Different Machine Learning algorithms
dealing with data as a full time series could have better results
for predicting disruptions, especially on a device character-
ized by such fast timescales as C-Mod. Furthermore, the
incorporation of more dimensionless and device-independent
quantities might provide more information for developing an
efficient disruption warning algorithm. In terms of the
applicability to ITER, the experience on machines with
similar characteristics has to be explored to extrapolate all the
possible knowledge that can be used to predict disruptive
behaviors or assess the predictive algorithm itself. It may be
possible that all the efforts developed on currently available
data will have limited effect in the realization of a disruption
warning algorithm for ITER. Given the gradual transition to a
full performance, though, an adaptive retraining on initially
available low-power and low-current data could be exploited,
by including also data coming from simulations. Finally,
cutting-edge techniques in Artificial Intelligence can prove to
be beneficial in learning domain-invariant representations that
can be used to adapt the algorithm between different opera-
tional regimes and tokamaks.
Acknowledgments
This work was supported by the US Department of Energy
under DE-FC02-04ER54698, DE-SC0014264 and DE-FG02-
04ER54761.
DIII-D data shown in this paper can be obtained in digital
format by following the links at https://fusion.gat.com/
global/D3D_DMP.
Disclaimer
This report was prepared as an account of work sponsored by
an agency of the United States Government. Neither the
United States Government nor any agency thereof, nor any of
their employees, makes any warranty, express or implied, or
assumes any legal liability or responsibility for the accuracy,
completeness, or usefulness of any information, apparatus,
product, or process disclosed, or represents that its use would
not infringe privately owned rights. Reference herein to any
specific commercial product, process, or service by trade
name, trademark, manufacturer, or otherwise, does not
necessarily constitute or imply its endorsement, recommen-
dation, or favoring by the United States Government or any
agency thereof. The views and opinions of authors expressed
herein do not necessarily state or reflect those of the United
States Government or any agency thereof.
Plasma Phys. Control. Fusion 60 (2018) 084004
ORCID iDs
C Rea https://orcid.org/0000-0002-9948-2649
R S Granetz https://orcid.org/0000-0002-6560-1881
K Montes https://orcid.org/0000-0002-0762-3708
R A Tinguely https://orcid.org/0000-0002-3711-1834
N Eidietis https://orcid.org/0000-0003-0167-5053
References
[1] Wroblewski D, Jahns G and Leuer J 1997 Nucl. Fusion 37
725–41
[2] Baltz E A, Trask E, Binderbauer M, Dikovsky M, Gota H,
Mendoza R, Platt J C and Riley P F 2017 Sci. Rep. 7 6425
[3] Hernandez J, Vannucci A, Tajima T, Lin Z, Horton W and
McCool S 1996 Nucl. Fusion 36 1009–17
[4] Vannucci A, Oliveira K and Tajima T 1999 Nucl. Fusion 39
255–62
[5] Pautasso G et al (Team AU) 2002 Nucl. Fusion 42 100–8
[6] Yoshino R 2003 Nucl. Fusion 43 1771–86
[7] Windsor C, Pautasso G, Tichmann C, Buttery R, Hender T,
Contributors J E and (Team tAU) 2005 Nucl. Fusion 45
337–50
[8] Cannas B, Fanni A, Marongiu E and Sonato P 2004 Nucl.
Fusion 44 68–76
[9] Cannas B, Cau F, Fanni A, Sonato P, Zedda M and
Contributors J E 2006 Nucl. Fusion 46 699–708
[10] Cannas B, Delogu R, Fanni A, Sonato P and Zedda M 2007
Fusion Eng. Des. 82 1124–30
[11] Cannas B, Fanni A, Sonato P and Zedda M K 2007 Nucl.
Fusion 47 1559–69
[12] Vega J, Dormido-Canto S, López J M, Murari A, Ramírez J M,
Moreno R, Ruiz M, Alves D and Felton R 2013 Fusion Eng.
Des. 88 1228–31
[13] De Vries P, Johnson M, Alper B, Buratti P, Hender T,
Koslowski H and Riccardo V 2011 Nucl. Fusion 51 053018
[14] Berkery J W, Sabbagh S A, Bell R E, Gerhardt S P and
LeBlanc B P 2017 Phys. Plasmas 24 056103
[15] Breiman L 2001 Mach. Learn. 45 5–32
13
C Rea et al
[16] Saeys Y, Inza I and Larranaga P 2007 Bioinformatics 23
2507–17
[17] Danesi I L and Rea C 2016 A customer relationship
management case study based on banking data Lecture
Notes in Computer Science (Lecture Notes in Artificial
Intelligence and Lecture Notes in Bioinformatics) vol 10122,
pp 224–35
[18] Rea C and Granetz R S 2018 Fusion Sci. Technol. 00 1–12
[19] De Vries P, Johnson M and Segui I 2009 Nucl. Fusion 49
055011
[20] Gerhardt S, Darrow D, Bell R, LeBlanc B, Menard J,
Mueller D, Roquemore A, Sabbagh S and Yuh H 2013 Nucl.
Fusion 53 063021
[21] Lao L, John H St, Stambaugh R, Kellman A and Pfeiffer W
1985 Nucl. Fusion 25 1611–22
[22] Ferron J, Walker M, Lao L, John H S, Humphreys D and
Leuer J 1998 Nucl. Fusion 38 1055–66
[23] Stillerman J and Fredian T W 1999 Fusion Eng. Des. 43 301–8
[24] La Haye R J, Fitzpatrick R, Hender T C, Morris A W,
Scoville J T and Todd T N 1992 Phys. Fluids B 4 2098–103
[25] Buttery R J et al 2000 Plasma Phys. Control. Fusion 42
B61–73
[26] Wolfe S M et al 2005 Phys. Plasmas 12 056110
[27] Sweeney R, Choi W, La Haye R, Mao S, Olofsson K and
Volpe F 2017 Nucl. Fusion 57 016019
[28] Strait E J 2006 Rev. Sci. Instrum. 77 023502
[29] Sammuli B S, Barr J L, Eidietis N W, Olofsson K,
Flanagan S M, Kostuk M and Humphreys D 2018 Fusion
Eng. Des. 29 12–5
[30] Pedregosa F et al 2012 J. Mach. Learn. Res. 12 2825–30
[31] Meneghini O et al 2015 Nucl. Fusion 55 083008
[32] Breiman L 1984 Classification and Regression Trees (London:
Chapman and Hall)
[33] Breiman L 1996 Mach. Learn. 24 123–40
[34] Alpaydın E 2010 Introduction to Machine Learning 2nd edn
(Cambridge, MA: MIT Press)
[35] Stehman S V 1997 Remote Sens. Environ. 62 77–89
[36] IPEGoDPC, MHDBasis IP 1999 Nucl. Fusion 39 2251
[37] Powers D 2011 J. Mach. Learn. Technol. 2 37–63
[38] Sauter O and Martin Y 2000 Nucl. Fusion 40 955–64
[39] Greenfield C M et al (Team tDD) 2004 Plasma Phys. Control.
Fusion 46 B213–33
[40] Wallace G et al 2013 Nucl. Fusion 53 073012