Professional Documents
Culture Documents
Ziogas - Neurobiol.2023 5
Ziogas - Neurobiol.2023 5
Neuropsychobiology
Neuropsychobiology Received: December 11, 2022
Accepted: April 21, 2023
DOI: 10.1159/000530931 Published online: June 27, 2023
“Leave One Out Cross Validation.” This common practice splits EEG-Source Localization
data into k independent folds and subsequently trains in k−1 folds Source localization analysis was done using eLORETA software
and tests in the complementary fold until the model has been package, version 20221229 (Roberto Pascual-Marqui/The KEY
trained and tested in every fold (i.e., in the complete dataset). This Institute for Brain-Mind Research, Zurich). The eLORETA algo-
cross-validation is seen as the method of choice when it comes to rithm is an inverse solution for EEG signals that has no localization
deep learning with a limited amount of data and a separate training error. In contrast to other source localization techniques, this is
and test sets are not feasible [33]. also applicable in the presence of measurement and structured
biological noise [37]. The family of LORETA algorithms allows
Grad-CAM Feature Extraction imaging of current density distributions inside the human cortex
For feature extraction, the class activation maps allow to seek for via constraining the solution to only gray matter-dense regions
distinctive features in the input data for deep learning based networks. within 6,239 predefined 3D-voxels [38]. EEG assessment in con-
The class activation maps show weighted heatmaps of the input data junction with simultaneous recordings in other neuroimaging
that allow for the identification of the most important data points for modalities cross-validated the family of LORETA algorithm and
classification tasks, e.g., in medical imaging [34]. Their successor, showed overlapping results of intracortical EEG-source estimates
Grad-CAM allows for the same approach without the necessity of and the BOLD signal [39, 40] or glucose metabolic rate [41, 42]. To
global average pooling before the softmax layer by using gradient generate the input for the source localization, the projections of the
information that flows into the last convolutional layer of the network. network were trained on k−1 datasets and tested on the remaining
Grad-CAM also outperforms other heatmap algorithms in medical dataset repeatedly, until all subjects of one group (hetero- or
imaging [35]. To apply Grad-CAM method, a rectified linear unit is homosexual) were extracted from all layers for all EEG channels
used to focus on the decisive features for a certain class. This method and averaged over all subjects of the group. The electrode activa-
has been used to detect EEG channels with highest information load tion matrix for the LORETA analysis was thus fed with the weights
in brain-computer-interface scenarios [36]. Following this procedure, of the network, which allowed a correct classification. The LOR-
data from the subject to classify were fed into the network, and from ETA transformation matrix was computed from the EEG channel
each layer of the network, the decisive weights were exported and positions used in the EEG recordings and the electrode activation
averaged over all layers. This weight map then was projected on the matrix was projected into the intracortical solution space of the
input data, i.e., the EEG segments. Then, these 2-D maps were eLORETA software.
averaged over the time axis, yielding the 1-D activation array in
space, i.e., electrode positions. This process was repeated for all Frequency Analysis of the LORETA Regions
subjects. The output Grad-CAM maps were then averaged for all To determine which frequencies of the EEG input data were
homosexual and all heterosexual subjects. Feeding these arrays into a used for classification by the SexualOrientationNet, the output of
source localization algorithm such as the low-resolution electromag- all filters (ranging from 100–300 per layer) from all convolutional
netic tomography (eLORETA), it was possible to map the intracortical layers (n = 6) of the network were computed while using a loss
sources of electrophysiological activity that allowed the correct clas- function that maximized the activation of the filters. For each layer,
sification of a group. the filter with the largest activation was projected into a 24 × 256
Correct classification (binary) 97.5% (39/40) 92.3% (36/39) 89.2% (33/37) p = 0.30 p = 0.13 p = 0.64
Probability of correct 88.8% 83.5% (SD 18.5%) 75.8% p = 0.16 p = 0.00* p = 0.10
classification (SD 14.2%) (SD 21.6%)
Significant differences were only found when considering the general probability of a subject being classified correctly:
homosexual males were classified correctly with higher probabilities than a mixed male/female cohort.
Fig. 2. Top: illustration of the accuracies and losses of the training set and the test set by means of classification of
total segment count of each set. Results are averaged over 77 runs of the network with one subject left out every
time for training. Bottom: a four-fold table for the correct classification of homosexual and heterosexual men
using the SexOrientationNetwork, trained on data from homosexual and heterosexual men with a total accuracy
of 83%.
Fig. 3. Panel (a) shows averaged filter weights from layer 1–6, of 2 s. Red color codes for the areas with highest “attention” of the
superimposed on a single epoch input EEG time series. High network; blue color codes for the least “attention” for differ-
gradients are red; low gradients are blue. Since the network entiation into labels “heterosexual” or “homosexual” male EEG
convolutes and max-pooling is applied, the resolution of the layer segments. A clear focus on the parietal and occipital alpha waves
gets coarse toward the deeper layers. Panel (b) illustrates the grand can be seen. Panel (c) and (d) show the reconstruction of the
average of all filters, again superimposed on a single EEG segment associated sources of the weights in LORETA space.
gradient projections from all layer averages were summed lobe at Brodmann area 1 (X = −45, Y = −30, Z = 65, MNI
up to give a visualization of the gradients on the raw EEG cords, Fig. 4, panel b top). The region with the largest
(Fig. 3, panel b). The averaged layer weights then were discriminative activity for HoM was found at the inferior
convoluted toward the EEG channel axis and the weights parietal lobule at Brodmann area 40 (X = 40, Y = −55, Z =
were used for the LORETA source reconstruction. The 60, MNI cords, Fig. 4, panel b, bottom).
main focus of the differentiating features was found in the
Brodmann area 7, parietal lobe at x = −3, y = −53, z = 57) Identification of Frequency Properties for Classification
for the shown activity (Fig. 3, panel c and d). at Group Level
For assessment of the frequencies that are associated with
Visualization of Differentiating Features at the the identification of male sexual orientation, the filters of all
Group Level layers of the SexOrientationNet were fed backward with
As a next step, we trained 77 networks following the patterns that activated each filter the most, thus generating
LOO approach and calculated an averaged Grad-CAM map archetypical inputs for the decisions of the networks. These
for the remaining subject. Then, the maps were averaged for patterns (Fig. 4, panel c) can be seen as EEG traces that evoke
the two groups (homo- and heterosexual men, Fig. 3, panel the classifications “heterosexual” or “homosexual.” Since
a). Following the eLORETA source localization approach, these patterns have the same dimension as the input EEG
the region with the activity that allowed for best labeling segments (24 channels × 256 points in time), these time
HeMs was located at the postcentral gyrus at the parietal series were forwarded to Fourier transformation to reveal the
Fig. 4. Panel (a) visualizes the averaged Grad-CAM results for gyrus (panel b, bottom). From the backward activated filters of
the channels of highest activation for HeMs (top) and homo- the SexOrientationNet (panel c, ×3x3 filters with largest weights
sexual males (bottom). These matrices were averaged over time of 1–6), the time series of nearby electrodes was Fourier trans-
and used for the transformation matrices to identify the cortical formed to obtain information about the associated frequencies.
sources of these activities (panel c). While HeMs showed high- While early layers show peaks at 17 and 48 Hz (panel d, pictures
est weights in the inferior parietal gyrus (panel b, top), homo- 1 and 3), no clear peaks were found for later layers (panel
sexual males showed highest weights in the superior temporal d, picture 2 and 4).
frequency analysis to reveal associated oscillations at the trained on gender in another cohort. This counteracts
group level. These findings provide evidence that deep assumptions that homosexual males are “femalized” wom-
learning in combination with EEG data can successfully an. Interestingly, when not looking at the binary classifi-
classify sexual orientation at the group level and extract cation (yes/no) but into the probabilities for correct clas-
non-a priori-defined features on the single subject/ sification, the homosexual group even had a significant
segment and group level that might help to understand higher probability to be rated correctly in comparison to a
underlying electrophysiological and anatomical circuits. mixed sex group. It is further interesting that the overall
The further extraction of source-bound frequencies that accuracy of this trial was higher than the results that this
contributed to the network to learn revealed the impor- pretrained network achieved when being tested on 300
tance of specific EEG oscillations for differentiation. subjects from sample 3. The reason for a higher accuracy
Using the pretrained SexNet, all different groups from of 89–97.5% in comparison to 73% could be found in the
sample 1 and sample 2 showed very high classification fact that longer EEG epochs with a subsequently increased
accuracy for sex. However, contrary to the hypothesized segment number per subject were used. The sample 1 and
classification of homosexual males as more akin to females, sample 2 datasets consisted of a three-fold number of
following potential similarities of sexual stimulus processing segments per subject (240 s vs. 80 s) in comparison to
between homosexual males and females in contrast to the SexNet training (sample 3). A simulation on the
HeMs, there was no difference for classification of homo- association between numbers of segments for each subject
sexual males and HeMs. Strikingly, both variants of sexual (when classifying each subject into a specific group
orientations were classified as males when using a network with >50% of classified segments in a binary labeling
References
1 Bailey JM, Vasey PL, Diamond LM, Bree- 5 De Block A, Adriaens PR. Pathologizing Sex- 10 Ganna A, Verweij KJH, Nivard MG, Maier R,
dlove SM, Vilain E, Epprecht M. Sexual ori- ual Deviance: A History. J Sex Res. 2013; Wedow R, Busch AS, et al. Large-scale GWAS
entation, controversy, and science. Psychol 50(3–4):276–98. reveals insights into the genetic architecture
Sci Public Interest. 2016 Sep;17(2):45–101. 6 Bieber I, Bieber TB. Male homosexuality. Can of same-sex sexual behavior. Science. 2019
2 Gold JH. Homosexuality: a review. Can Fam J Psychiatry. 1979 Aug;24(5):409–21. Aug;365(6456):eaat7693.
Physician. 1983 Mar;29:521–4. 7 Independent Forensic Expert Group. State- 11 Ngun TC, Vilain E. The biological basis of
3 Groneberg M, Funke C. Combatting homo- ment on conversion therapy. J Forensic Leg human sexual orientation: is there a role
phobia: experiences and analyses pertinent to Med. 2020 May;72:101930. for epigenetics? Adv Genet. 2014;86:
education. Berlin: Auflage LIT; 2011. 8 Blanchard R, Bogaert AF. Homosexuality in 167–84.
4 Röll W. Homosexual inmates in the buchen- men and number of older brothers. Am 12 Savic I, Lindström P. PET and MRI show
wald concentration camp. J Homosex. 1996; J Psychiatry. 1996 Jan;153(1):27–31. differences in cerebral asymmetry and func-
31(4):1–28. 9 Gavrilets S, Rice WR. Genetic models of homo- tional connectivity between homo- and het-
sexuality: generating testable predictions. Proc erosexual subjects. Proc Natl Acad Sci U S A.
Biol Sci. 2006 Dec 22;273(1605):3031–8. 2008 Jul;105(27):9403–8.