ZIOGAS.NEUROBIOL.2023 พพ

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 12

Research Article

Neuropsychobiology
Neuropsychobiology Received: December 11, 2022
Accepted: April 21, 2023
DOI: 10.1159/000530931 Published online: June 27, 2023

Deep Learning in the Identification of


Electroencephalogram Sources
Associated with Sexual Orientation

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023


Anastasios Ziogas a Andreas Mokros b Wolfram Kawohl c, d
Mateo de Bardeci c Ilyas Olbrich e Benedikt Habermeyer d
Elmar Habermeyer a Sebastian Olbrich c
a
Department of Forensic Psychiatry, University Hospital of Psychiatry Zurich, Zurich, Switzerland;
b
FernUniversität in Hagen, Hagen, Germany; cDepartment of Psychiatry, Psychotherapy and Psychosomatics,
Psychiatric Hospital, University of Zurich, Zurich, Switzerland; dClienia Schlössli AG, Oetwil am See, Switzerland;
e
RG Gymnasium, Zurich, Switzerland

Keywords homosexual and heterosexual males. The newly trained net-


Electroencephalogram · Deep learning · Homosexual males · work was able, however, to correctly classify the cohorts with a
Sexual orientation total accuracy of 83%. The retrograde activation using Grad-
CAM technology yielded distinctive functional EEG patterns in
the Brodmann area 40 and 1 when combined with Fourier
Abstract analysis and a source localization. Discussion: This study shows
Introduction: It is unclear if sexual orientation is a biological that electrophysiological trait markers of male sexual orienta-
trait that has neurofunctional footprints. With deep learning, tion can be identified using deep learning. These patterns are
the power to classify biological datasets without an a priori different from the differentiating signatures of males and
selection of features has increased by magnitudes. The aim of females in a resting-state EEG. © 2023 The Author(s).
this study was to correctly classify resting-state electroence- Published by S. Karger AG, Basel

phalogram (EEG) data from males with different sexual ori-


entation using deep learning and to explore techniques to
identify the learned distinguishing features. Methods: Three
cohorts (homosexual men, heterosexual men, and a mixed sex Introduction
cohort), one pretrained network on sex classification, and one
newly trained network for sexual orientation classification were Sexual orientation is an important personal character-
used to classify sex. Further, Grad-CAM methodology and istic of every person. The individual orientation can be
source localization were used to identify the spatiotemporal placed on a continuum between homo- and heterosexual
patterns that were used for differentiation by the networks. alignment rather than applying a strict dichotomous
Results: Using a pretrained network for classification of males division [1]. Throughout history, persons with sexual
and females, no differences existed between classification of preferences for the same sex and thus a differing

karger@karger.com © 2023 The Author(s). Correspondence to:


www.karger.com/nps Published by S. Karger AG, Basel Sebastian Olbrich, sebastian.olbrich @ pukzh.ch
This article is licensed under the Creative Commons Attribution 4.0
International License (CC BY) (http://www.karger.com/Services/
OpenAccessLicense). Usage, derivative works and distribution are
permitted provided that proper credit is given to the author and the
original publisher.
orientation from the majority have been facing persecu- females based on their brain function patterns during
tion and violence, also in the Christian-coined western rest [16]. Although EEG research has been misused to
world [2–4]. The American Psychiatric Association only identify pathological EEG patterns in males with homo-
removed homosexuality from the category of psychiatric sexual orientation [17, 18], more recent research was
diseases in 1973 [5]. However, people always tried to more focused on mainly functional, non-pathological
understand the reason for these differences in sexual differences in groups of homosexual and heterosexual
orientation. One of the main questions thereby has males (HeMs) [19]. When EEG data were used in these
been if either homosexuality is a hereditary trait or studies to differentiate between males and females [20] or
environmental factors influence the development of these groups of homosexual and HeMs [21], the data were
preferences [1]. A main motivation for research in this always recorded during cognitive task performance (e.g.,
area has been the search for notorious preventive meas- mental rotation). Results could therefore be related to
ures for a long time [6]. Nowadays, in the western world, sex-dimorphic cognitive task properties and less to func-
the freedom of choice including the sexual preferences tional differences by means of basic central nervous
has evolved as the widely accepted social agreement, system activity during rest.

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023


although it is still a long way from providing safety Within recent years, new techniques derived from the
and participation for every person, regardless of their field of artificial intelligence research emerged that allow
sexual orientation. for powerful analysis of medical and electrophysiological
Having reached this cultural developmental stage, it data [22]. Especially so-called deep learning algorithms
still is of high scientific interest, whether there exist proved their superiority in the classification of time series
biological patterns that differ between persons with dif- data in comparison to conventional statistical analysis,
ferent sexual orientations. The authors are aware of the given that enough labeled data are available [23]. These
ethical questions arising from this research, as outlined deep learning techniques stem from the functional prin-
above. The approvement of biological traits in alignment ciple of natural neuronal networks as can be found in the
with sexual orientation still could help further withdraw human brain [24]. By changing the connection strength
the basis of involuntary behavioral measures against between different nodes, i.e., neurons during the training
persons with nonheterosexual orientation [7]. The fra- of the network, the labeled input data will be processed to
ternity hypothesis stated that the existence of older male result in the desired output, i.e., the classification label.
brothers increased the probability of being gay in men, The network then can be tested with previously unseen
thereby attributing a significant impact to environmental testing data. Since it is appealing to classify brain-derived
factors on sexual orientation [8]. Others found genetic [9, functional time series using a technique that follows the
10], epigenetic [11], brain structure [12–14], or immu- function of neuronal networks, in this study, we aimed to
nological [15] differences in subjects with same sex sexual test whether there are differences in the electrophysio-
orientation, supplying evidence for a biological trait. logical patterns during rest between homosexual and
Especially findings from large-scale studies using data heterosexual men using these deep learning. Assuming
from over 400.000 subjects from the UK Biobank and the that cues of biological sex are detected and processed in
23andMe database revealed that there are genetic pre- similar ways within individuals who are interested in the
dispositions for same-sex sexual behavior [10]. However, same sex of potential partners [25], we assumed a re-
the main outcome was that no genetic variant allowed for a semblance in the resting state activity between
prediction of the individual sexual orientation. The authors homosexual men (HoM) and heterosexual women.
underline that their findings shed light on the complexity of Some research of the past showed similar functional
sexual behavior because of still largely unknown interac- activations in homosexual males and heterosexual fe-
tions between genetics and environment. males [12]. Consequently, we at first tested if (1) HoM
In the light of structural differences between individ- might exhibit similar EEG patterns like heterosexual
uals with different variants of sexual orientation, it seems females in comparison to heterosexual men. That means
paramount to also search for possible functional differ- HoM could be classified via a network that was trained on
ences. Electrophysiology, and electroencephalogram sex and not sexual orientation, receiving the label “fe-
(EEG) in particular, provides a highly suitable framework male.” If this quite mechanistical hypothesis would not
for a noninvasive analysis of brain function for a large hold true, i.e., homosexual males would not exhibit
variety of conditions. In combination with advanced similar EEG patterns as heterosexual females, it was
technologies such as deep learning, EEG data recently further hypothesized that (2) HoM could be classified
have been used to differentiate between males and by applying a new network trained on HoM and

2 Neuropsychobiology Ziogas/Mokros/Kawohl/de Bardeci/


DOI: 10.1159/000530931 Olbrich/Habermeyer/Habermeyer/Olbrich
heterosexual men, i.e., being classified via a network for activation. A final dense layer with softmax activation was used
trained on sexual orientation and not sex. For testing for classification probabilities. The rest of the training procedure
was done as described in [16].
(1), a previously trained male/female classification deep
learning network (“SexNet”) [16] was applied to datasets
EEG Processing and SexOrientationNet
of HoM, heterosexual men, and a third group of mixed The EEGs were recorded at 2.5 kHz (BrainAmp, BrainProducts,
males and females. For testing (2), a new network was Germany) during a 5-min resting state with eyes closed using a 32-
trained to differentiate sexual orientation by using data electrode BrainCap (MR 32 standard, Easycap) referenced to FCz
from HoM or heterosexual men. To further elucidate the following the international 10 to 20 system. Impedances were kept
functional properties and anatomical distribution of po- below 20 kOhm. Eye movements were recorded using bipolar
diagonal electrooculogram channels. Software analysis was done
tential EEG patterns for differentiation between homo- using Brain Vision Analyzer 2.0, Gilching, Germany: raw data
sexual and heterosexual men at the individual level, a were downsampled to 128 Hz and band pass filtered between 0.5
gradient CAM (Grad-CAM) approach in combination and 25 Hz plus notch filter. To compare the results with the
with a source localization technique and Fourier trans- outcome of the SexNet, 24 channels were kept for further analysis
formation was used. as follows: Fp1, Fp2, F7, F3, Fz, F4, F8, FC3, FCz, FC4, T3, C3, Cz,

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023


C4, T4, CP3, CPz, CP4, T5, P3, Pz, P4, T6, O1. EEG data were
eye – artefact corrected using a regression-based approach Grat-
ton, Coles, and Donchin [29]. Artefact rejection was done using an
Materials and Methods automated pipeline following different voltage and power criteria,
detailed in [28]. Finally, the EEG was inspected visually by an
The study was approved by the Ethical Committee Zurich (EC- experienced EEG-rater, and artefacts were marked and rejected for
No 2014-0623). All participants gave written informed consent. consecutive 2 s segments (example Fig. 1). Raw data of 4-min
The study was conducted following the rules established by the recordings were imported to Matlab software (R2019b), where a
Declaration of Helsinki. three-dimensional matrix of channels (24) × segment-length (2 sec
with 2 × 128 = 256 timepoints) × epochs (120 with 2 sec duration)
Subjects Sample 1 and Sample 2: EEG Recording were stacked for each subject. Labeling of the subjects (homosexual
Sample 1 consisted of 39 HeMs and 38 HeMs recruited from man or heterosexual man in sample 1 and additionally hetero-
2017 to 2019 in Zurich and was reported in another study [26]. Ten sexual woman in sample 2 and sample 3) was done using one-hot
additional subjects had been recruited but needed to be excluded, coding. The matrix then was exported to the python environment.
four due to regular intake of medication. Two more subjects The SexOrientationNet was deployed in a python (3.8) Tensor-
showed a sexual orientation that showed no clear preference for Flow (version 1.2 GPU version) environment with a Keras backend
on sex (bisexuality), based on a Klein Sexual Orientation Grid (Version 2.2). The structure of the used network was identical to
(KSOG) Score between 3and 4.9. Four subjects had to be excluded the structure of the SexNet, only the number of filters was between
due to missing or bad EEG data. 100 and 300. However, training was not performed on sample 3
Sample 2 consisted of 37 healthy controls with mixed sex but on sample 1 (homosexual and heterosexual men) with a Leave
(female = 16) and unknown sexual orientation, recorded with One Out validation process (see below Sex Orientation network
the same equipment under the same conditions as sample 1. validation)
Sample 2 was recorded in year 2010 as control subgroup for
the ZINEP study [27]. Assessment of Sexual Orientation
The KSOG [30] was used to check for self-reported sexual
Subjects Sample 3: EEG Recording and SexNet Training preference. This scale uses seven items on sexual orientation (sexual
In total, 1,300 EEG datasets were used for the SexNet training attraction, sexual behavior, sexual fantasies, emotional preference,
and validation. They were recorded and preprocessed using a social preference, self-identification, and heterosexual/gay lifestyle
standardized methodology and platform (Brain Resource Ltd., outcomes) that are addressed with responses on three dimensions
Australia) for which full details have been published elsewhere (past, present, and future). Responses are given on a 7-point Likert
[28] and which is in large parts identical for sample 1 and 2. The scale ranging from 1 (other sex only/heterosexual only) to 7 (same sex
used SexNet has previously been reported to differentiate men and only/gay only). Cronbach’s alpha of 0.96 and significant correlation
women based on 80 s resting-state EEG recordings with an with self-reported sexual orientation (r = 0.71) have been reported for
accuracy of 84% [16], which is a highly significant finding, given the KSOG [31]. Average scores across the present time dimension were
a random guess distribution of 50%. To avoid overestimation for calculated. Values of 2.9 or less refer to heterosexual orientation; values
transfer learning, we did not apply the top performance net from of 5.0 or above refer to a homosexual orientation [32].
[16] but relied on an average model with a performance accuracy
of 73%. Therefore, SexNet was trained on 24 channel EEG seg- Deep Learning SexNet and SexOrientationNet Validation and
ments with a 128 Hz resolution from 1000 adults and tested on Feature Extraction
unseen 300 EEG datasets. The architecture of SexNet consisted of 9 Validation for the usage of the SexNet was done using separate
layers with 256 × 24 as input matrix, convolutional layers with training and test sets. Accuracy measures were used for assessing
decreasing number of filters (300–50) and pooling function after the quality of the classification in the test set.
each of the first four convolutional layers, and a dropout function Validation for the usage of the newly trained SexOrientation-
of 25% after each convolutional layer. A rectifier linear unit served Net on homosexual and heterosexual men was done using the

Deep Learning and EEG Patterns for Sexual Neuropsychobiology 3


Orientation DOI: 10.1159/000530931
Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023
Fig. 1. Example of cleaned 2 s segment of
EEG data used for classification of sexual
orientation.

“Leave One Out Cross Validation.” This common practice splits EEG-Source Localization
data into k independent folds and subsequently trains in k−1 folds Source localization analysis was done using eLORETA software
and tests in the complementary fold until the model has been package, version 20221229 (Roberto Pascual-Marqui/The KEY
trained and tested in every fold (i.e., in the complete dataset). This Institute for Brain-Mind Research, Zurich). The eLORETA algo-
cross-validation is seen as the method of choice when it comes to rithm is an inverse solution for EEG signals that has no localization
deep learning with a limited amount of data and a separate training error. In contrast to other source localization techniques, this is
and test sets are not feasible [33]. also applicable in the presence of measurement and structured
biological noise [37]. The family of LORETA algorithms allows
Grad-CAM Feature Extraction imaging of current density distributions inside the human cortex
For feature extraction, the class activation maps allow to seek for via constraining the solution to only gray matter-dense regions
distinctive features in the input data for deep learning based networks. within 6,239 predefined 3D-voxels [38]. EEG assessment in con-
The class activation maps show weighted heatmaps of the input data junction with simultaneous recordings in other neuroimaging
that allow for the identification of the most important data points for modalities cross-validated the family of LORETA algorithm and
classification tasks, e.g., in medical imaging [34]. Their successor, showed overlapping results of intracortical EEG-source estimates
Grad-CAM allows for the same approach without the necessity of and the BOLD signal [39, 40] or glucose metabolic rate [41, 42]. To
global average pooling before the softmax layer by using gradient generate the input for the source localization, the projections of the
information that flows into the last convolutional layer of the network. network were trained on k−1 datasets and tested on the remaining
Grad-CAM also outperforms other heatmap algorithms in medical dataset repeatedly, until all subjects of one group (hetero- or
imaging [35]. To apply Grad-CAM method, a rectified linear unit is homosexual) were extracted from all layers for all EEG channels
used to focus on the decisive features for a certain class. This method and averaged over all subjects of the group. The electrode activa-
has been used to detect EEG channels with highest information load tion matrix for the LORETA analysis was thus fed with the weights
in brain-computer-interface scenarios [36]. Following this procedure, of the network, which allowed a correct classification. The LOR-
data from the subject to classify were fed into the network, and from ETA transformation matrix was computed from the EEG channel
each layer of the network, the decisive weights were exported and positions used in the EEG recordings and the electrode activation
averaged over all layers. This weight map then was projected on the matrix was projected into the intracortical solution space of the
input data, i.e., the EEG segments. Then, these 2-D maps were eLORETA software.
averaged over the time axis, yielding the 1-D activation array in
space, i.e., electrode positions. This process was repeated for all Frequency Analysis of the LORETA Regions
subjects. The output Grad-CAM maps were then averaged for all To determine which frequencies of the EEG input data were
homosexual and all heterosexual subjects. Feeding these arrays into a used for classification by the SexualOrientationNet, the output of
source localization algorithm such as the low-resolution electromag- all filters (ranging from 100–300 per layer) from all convolutional
netic tomography (eLORETA), it was possible to map the intracortical layers (n = 6) of the network were computed while using a loss
sources of electrophysiological activity that allowed the correct clas- function that maximized the activation of the filters. For each layer,
sification of a group. the filter with the largest activation was projected into a 24 × 256

4 Neuropsychobiology Ziogas/Mokros/Kawohl/de Bardeci/


DOI: 10.1159/000530931 Olbrich/Habermeyer/Habermeyer/Olbrich
array, representing the EEG input with the highest potential of accuracy of 89.2–97.5% with no significant differences
layer activation. A Fourier transformation was applied for the between the three datasets (p > 0.05). Only when numeric
x-axis (i.e., EEG channel time series) to analyze the prevailing
frequencies in the corresponding layers.
probabilities of subject classification were tested for dif-
ferences, the HoM dataset showed a significant higher
Estimating Statistical Threshold probability for correctly classified sex in comparison to
For assessment of statistical significance of the SexualOrienta- the correct classification of the MixMF group (p = 0.00,
tionNet, we randomly assigned sexual orientation to each subject see Table 1).
in a test set (n = 20), using the prior sexual orientation distribution
(50% homosexual males). To set the p value for statistical sig-
nificance at p ≤ 0.01, we performed 100 simulations in Matlab, Classification of Sexual Orientation Using a New
following the principles of Monte Carlo simulation. The best Trained Deep Net
classification accuracy reached was 70%, which was subsequently To further test whether male subjects with different
considered the significance threshold for p ≤ 0.01. sexual orientation (homosexual or heterosexual) could be
correctly classified using deep learning, a new network
Comparison with fMRI Findings
(SexOrientationNet) was trained on data from sample 1.

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023


To compare the sources of differentiating EEG activity with
findings from neuroimaging studies using fMRI, the “Neurosynth” The used Leave One Out training and testing procedure
website (“neurosynth.org”) was used [43]. In total, 81 studies using of the network for classification of homosexual or het-
fMRI and the context “sexual” were included into the sample. erosexual men showed a performance of a 75.95% accu-
Works with the highest load in the analysis included studies on racy for the correct classification of HoM or heterosexual
functional endophenotypes for sexual orientation [44], neural
men for all segments of the dataset (Fig. 1). Following the
substrates of sexual desire [45], and correlates of gender differences
[46]. The results of the uniformity analysis (z-scores from a one- majority vote on the 120 epochs of each subject when
way ANOVA testing) were corrected for multiple comparisons trained on the rest of the subjects, 26 out of 39 hetero-
using the false discovery rate with a 0.01 criterion. sexual men were classified correctly and 38 out of 38
HoM. A four-fold table of the results can be found in
Figure 2 (bottom). The sensitivity to correctly classify a
Results homosexual man as homosexual was 100%; the specificity
was 67% with a positive predictive value of 76% and a
Sociodemography negative predictive value of 100%, yielding a total accu-
For sample 1 (n = 77), mean age of homosexual males (n = racy of 83%. Taking into account that an accuracy of 70%
38) was 24.7 years with SD 5.0, HeMs (n = 38) with 24.6 years is equivalent to a statistical threshold of p < 0.01 accord-
and SD 4.9. Mean KSOG score for homosexual males was 1.9 ing to the Monte Carlo simulation, this result can be seen
with SD 0.40, for HeMs 5.7 SD 0.39. For sample 2 (n = 37 as highly significant. The number of epochs of learning
with 16 females), mean age was 28.6 years (28.4 years for for each trial (with one subject left out and an early
females only) with SD 5.9. For sample 3 (n = 1,300), the mean stopping rule after four consecutive failed decreases of
age was 43.4 with 18.4 SD with 47% males [16]. Different validation loss) ranged from 6 to 51. The averaged train-
personal variables and performances, including handedness, ing and test accuracies and training and test losses for all
sexual desire, sexual excitation, and the attentional network segments (not only subjects) can be found in Figure 2.
task [47], were tested for differences between groups (homo-
sexual males vs. HeMs) with no significant alterations Visualization and Localization of Differentiating
between the groups (see [26] for details). Features at the Single EEG Segment Level
To analyze whether it is possible to extract the EEG
Classification of Homo- or Heterosexual Orientation patterns that showed discriminative features for sexual
Using a Pretrained SexNet orientation classification in single subjects, a Grad-CAM
The SexNet trained on 1,000 and tested on 300 EEGs approach was used. Therefore, the trained filters from all
from women and men to differentiate sex (accuracy of layers (ranging from 100–300 filters per layer) from a
73%) was applied to three different groups (two groups single leave-one-out run were averaged layer-wise, and
from sample 1 with HoM and HeM controls) and data the gradients of a single EEG epoch from the test subject
from sample 2 (mixed male female [MixMF]). In all were computed for each averaged layer filter (single layer
datasets, HoM, HeM, and MixMF, males irrespective averages can be found in Fig. 3, panel a). It is obvious that
of their sexual orientation were classified (binary classi- early layers focus on activity at 10 Hz at posterior sites
fication with a probability of being male >50% for the (Fig. 3, panel a), while later layers have less differentiation
majority of all 120 segments per subject) as male with an due to the max-pooling steps in the algorithm. The

Deep Learning and EEG Patterns for Sexual Neuropsychobiology 5


Orientation DOI: 10.1159/000530931
Table 1. Correct classification of sex of three different groups from sample 1 and 2. Men were classified as men irrespective of their
sexual orientation

HoM Heterosexual Mixed sex Ho Ho He


men versus He versus mix versus mix

Correct classification (binary) 97.5% (39/40) 92.3% (36/39) 89.2% (33/37) p = 0.30 p = 0.13 p = 0.64
Probability of correct 88.8% 83.5% (SD 18.5%) 75.8% p = 0.16 p = 0.00* p = 0.10
classification (SD 14.2%) (SD 21.6%)

Significant differences were only found when considering the general probability of a subject being classified correctly:
homosexual males were classified correctly with higher probabilities than a mixed male/female cohort.

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023

Fig. 2. Top: illustration of the accuracies and losses of the training set and the test set by means of classification of
total segment count of each set. Results are averaged over 77 runs of the network with one subject left out every
time for training. Bottom: a four-fold table for the correct classification of homosexual and heterosexual men
using the SexOrientationNetwork, trained on data from homosexual and heterosexual men with a total accuracy
of 83%.

6 Neuropsychobiology Ziogas/Mokros/Kawohl/de Bardeci/


DOI: 10.1159/000530931 Olbrich/Habermeyer/Habermeyer/Olbrich
a d

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023


c
b

Fig. 3. Panel (a) shows averaged filter weights from layer 1–6, of 2 s. Red color codes for the areas with highest “attention” of the
superimposed on a single epoch input EEG time series. High network; blue color codes for the least “attention” for differ-
gradients are red; low gradients are blue. Since the network entiation into labels “heterosexual” or “homosexual” male EEG
convolutes and max-pooling is applied, the resolution of the layer segments. A clear focus on the parietal and occipital alpha waves
gets coarse toward the deeper layers. Panel (b) illustrates the grand can be seen. Panel (c) and (d) show the reconstruction of the
average of all filters, again superimposed on a single EEG segment associated sources of the weights in LORETA space.

gradient projections from all layer averages were summed lobe at Brodmann area 1 (X = −45, Y = −30, Z = 65, MNI
up to give a visualization of the gradients on the raw EEG cords, Fig. 4, panel b top). The region with the largest
(Fig. 3, panel b). The averaged layer weights then were discriminative activity for HoM was found at the inferior
convoluted toward the EEG channel axis and the weights parietal lobule at Brodmann area 40 (X = 40, Y = −55, Z =
were used for the LORETA source reconstruction. The 60, MNI cords, Fig. 4, panel b, bottom).
main focus of the differentiating features was found in the
Brodmann area 7, parietal lobe at x = −3, y = −53, z = 57) Identification of Frequency Properties for Classification
for the shown activity (Fig. 3, panel c and d). at Group Level
For assessment of the frequencies that are associated with
Visualization of Differentiating Features at the the identification of male sexual orientation, the filters of all
Group Level layers of the SexOrientationNet were fed backward with
As a next step, we trained 77 networks following the patterns that activated each filter the most, thus generating
LOO approach and calculated an averaged Grad-CAM map archetypical inputs for the decisions of the networks. These
for the remaining subject. Then, the maps were averaged for patterns (Fig. 4, panel c) can be seen as EEG traces that evoke
the two groups (homo- and heterosexual men, Fig. 3, panel the classifications “heterosexual” or “homosexual.” Since
a). Following the eLORETA source localization approach, these patterns have the same dimension as the input EEG
the region with the activity that allowed for best labeling segments (24 channels × 256 points in time), these time
HeMs was located at the postcentral gyrus at the parietal series were forwarded to Fourier transformation to reveal the

Deep Learning and EEG Patterns for Sexual Neuropsychobiology 7


Orientation DOI: 10.1159/000530931
Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023
a b c d

Fig. 4. Panel (a) visualizes the averaged Grad-CAM results for gyrus (panel b, bottom). From the backward activated filters of
the channels of highest activation for HeMs (top) and homo- the SexOrientationNet (panel c, ×3x3 filters with largest weights
sexual males (bottom). These matrices were averaged over time of 1–6), the time series of nearby electrodes was Fourier trans-
and used for the transformation matrices to identify the cortical formed to obtain information about the associated frequencies.
sources of these activities (panel c). While HeMs showed high- While early layers show peaks at 17 and 48 Hz (panel d, pictures
est weights in the inferior parietal gyrus (panel b, top), homo- 1 and 3), no clear peaks were found for later layers (panel
sexual males showed highest weights in the superior temporal d, picture 2 and 4).

involved frequencies. Analysis was restricted to the closest Discussion


electrodes of the intracortical clusters (Fig. 4, panel b). This
process was done separately for heterosexual and homo- This study aimed at the differentiation of homosexual
sexual classification patterns and for all layers. While there and heterosexual orientation in men using neurophysio-
were clear peak frequencies around 17 Hz and 48 Hz in early logical resting state data. Further goal was to test feature
layers (Fig. 4, panel d, picture 1 and 3 for layer 2), later layers extraction from the used networks to gain new insights
resampled more complex patterns that did seem to comprise into electrophysiological patterns that differ between ho-
multiple frequencies (Fig. 4, panel d, picture 2 and 4). mosexual and heterosexual men. While the sexual
orientation in men could not be classified by a network
Comparison with fMRI Research previously trained on sex (SexNet), a newly trained net-
Comparing the sources of the discriminating EEG fea- work on sexual orientation (SexOrientationNet) was able
tures identified by the SexOrientationNet with meta- to distinguish between males with homo- or heterosexual
analytical findings from 81 fMRI studies that used a orientation with a very high accuracy of 83%. Further, the
sexuality context in their paradigms revealed an overlap Grad-CAM method was able to extract electrode-wise
mainly in the parietal cortices, i.e., at the superior parietal gradients that allowed for an identification of distinguish-
lobe (Brodmann x = −26, y = −60, z = 48). As can be seen in ing patterns at the temporal dimension, i.e., in the EEG
Figure 5, both groups, HoM and HeM, show source clusters traces and in the spatial dimension, i.e., the anatomical
with high filter weights in this area. source space. Additionally, these sources could be linked to

8 Neuropsychobiology Ziogas/Mokros/Kawohl/de Bardeci/


DOI: 10.1159/000530931 Olbrich/Habermeyer/Habermeyer/Olbrich
Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023
Fig. 5. Comparison of results from a Neurosynth meta-analysis of fMRI findings of studies related to sexual
content and the weighted sources of the SexOrientationNet on EEG data differentiating HeMs and homosexual
males. XYZ coordinates are all the same for top to bottom panels.

frequency analysis to reveal associated oscillations at the trained on gender in another cohort. This counteracts
group level. These findings provide evidence that deep assumptions that homosexual males are “femalized” wom-
learning in combination with EEG data can successfully an. Interestingly, when not looking at the binary classifi-
classify sexual orientation at the group level and extract cation (yes/no) but into the probabilities for correct clas-
non-a priori-defined features on the single subject/ sification, the homosexual group even had a significant
segment and group level that might help to understand higher probability to be rated correctly in comparison to a
underlying electrophysiological and anatomical circuits. mixed sex group. It is further interesting that the overall
The further extraction of source-bound frequencies that accuracy of this trial was higher than the results that this
contributed to the network to learn revealed the impor- pretrained network achieved when being tested on 300
tance of specific EEG oscillations for differentiation. subjects from sample 3. The reason for a higher accuracy
Using the pretrained SexNet, all different groups from of 89–97.5% in comparison to 73% could be found in the
sample 1 and sample 2 showed very high classification fact that longer EEG epochs with a subsequently increased
accuracy for sex. However, contrary to the hypothesized segment number per subject were used. The sample 1 and
classification of homosexual males as more akin to females, sample 2 datasets consisted of a three-fold number of
following potential similarities of sexual stimulus processing segments per subject (240 s vs. 80 s) in comparison to
between homosexual males and females in contrast to the SexNet training (sample 3). A simulation on the
HeMs, there was no difference for classification of homo- association between numbers of segments for each subject
sexual males and HeMs. Strikingly, both variants of sexual (when classifying each subject into a specific group
orientations were classified as males when using a network with >50% of classified segments in a binary labeling

Deep Learning and EEG Patterns for Sexual Neuropsychobiology 9


Orientation DOI: 10.1159/000530931
The second aim of this work was the identification
features that allowed discrimination between homosexual
and HeMs without a definition of an a priori feature space.
The extraction of features from the network by using the
Grad-CAM approach and backward filter activation from
discriminative tasks using deep learning allows the identi-
fication of nonlinear features from the raw data [49].
Following this path, the presented work was able to show
that the shape of the parietal and occipital alpha waves was
focused by the SexOrientationNet to label single subjects
according to their sexual orientation. This is in line with
previous EEG findings. One study found that heterosexual
men rated the erotic video with higher general and sexual
arousal than the homosexual participants. During observa-

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023


tion of the neutral and erotic videos, both groups showed
decreased amplitude of the alpha band in prefrontal and
Fig. 6. Increase of probability for a correct classification of a single
subject (for >50% of segments of one subject classified correctly)
parietal cortices, indicating increased attention [50]. Another
with different numbers of EEG segments (x-axis) from one record- study revealed less suppression of the mu-rhythm, a specific
ing and increasing overall probability for a correct classification of central rhythm in the frequency range of alpha, in HoM as
a single segment (colored graphs). Olbrich et al. [40]. reaction to painful actions in comparison to heterosexual
men [51]. Interestingly, Alexander and Sufka found in-
creased alpha activity in homosexual males over right parietal
approach) and the overall probability of one segment being cortices during cognitive tasks [19]. Ziogas et al. [52] related
classified correctly shows a sharp increase of correct clas- hemispheric alpha activity further to sexual arousal.
sification of subjects (Fig. 6) with increasing segment However, at the group level, the extracted frequencies
numbers. Hence, future studies on subject classification showed peaks in the beta and gamma range, not within
using deep learning and multiple EEG segments per subject the alpha band. Again, EEG beta activity has been asso-
should consider restricting the size of each segment to just ciated with processing of erotic content in homosexual
contain the essential information for increasing the overall males [50]. Further, at both, the single subject level and at
number of segments. This also strongly implies that deep the group level, the areas with the most important weights
learning approaches benefit from longer EEG recordings for differentiation by the network were in the parietal
since this allows for more data on each subject to be cortex. It is of special interest that the right Brodmann
classified. The increase of training samples has been found area 40 was source of EEG activity that yielded highest
to go in line with a logarithmical performance increase in discriminative power for homosexual males and Brod-
vison tasks [48]. mann area 1 had the highest load for HeMs. Parietal
When a new trained net was used for classification regions have been described in the context of sexual
instead (SexOrientationNet), the network was able to arousal in fMRI studies [53], while Brodmann area 1 is
accurately differentiate between homosexual males and part of the somatosensory cortex and thus highly related
HeMs. It is noteworthy that all homosexual subjects were to social perception [54] and sexual arousal [55]. The
classified correctly but only 2/3rd (26 out of 39) from the meta-analysis of 81 fMRI studies using paradigms with
HeMs. Although this result must be taken with caution, sexual content revealed a more likely activation of parietal
since no replication has yet been done, it might imply that regions in studies with sexual in comparison when no
most homosexual males show distinct features (or miss sexual content was present. Here, it must be mentioned
some features), while HeMs sometimes share these fea- that fMRI activation maps have to be regarded with
tures (or miss them, too) without being homosexual. caution since they cannot be interpreted as simple
However, only the heterosexual group reveals (or misses) maps of areas with higher or lower neuronal activity,
a feature in some of them (the correctly classified) that is given the nature of the underlying blood oxygenation
not there (or present) in the homosexual group. Thus, the dependent signal [56, 57]. However, the involvement of
SexOrientationNet revealed a very high specificity for the parietal and temporal cortex in activity generation in
classifying subjects as HeMs and a very high sensitivity both homosexual and HeMs during sexual exposure [58]
for homosexual orientation. has been linked to arousal and attention networks [59].

10 Neuropsychobiology Ziogas/Mokros/Kawohl/de Bardeci/


DOI: 10.1159/000530931 Olbrich/Habermeyer/Habermeyer/Olbrich
The findings of this study underline the presence of Acknowledgments
specific functional patterns at defined cortical areas
that are different for homosexual and HeMs. Sincere thanks go to Christine Wyss, Tania Villar de Araujo,
Xenia Binner, Laura Nanz, and Helena Pejic for their assistance.
Although this study clearly shows the potentials of
deep learning-guided EEG analysis for discriminative
tasks, it still bears some limitations. The first limitation Statement of Ethics
can be found in the relatively small number of homo-
sexual subjects and heterosexual controls in sample 1 for The study was approved by the Ethical Committee Zurich (EC-
No 2014-0623). All participants gave written informed consent.
the training of the SexOrientationNet. Although the The study was conducted following the rules established by the
analysis was enriched with data from other samples Declaration of Helsinki.
with heterosexual subjects, it was not possible to separate
a complete unseen test dataset; a Leave One Out approach
had to be used instead. Although this can be seen as the Conflict of Interest Statement
method of choice in an environment with limited data, a

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023


No conflicts of interest are reported by the authors.
larger dataset would have allowed for a more rigid testing
procedure. Moreover, following this limitation, all con-
clusions drawn from this study have to be considered with
Funding Sources
caution until proved in an independent sample.
Another limitation was the low sampling frequency of No external funding was provided for this study.
the finally used EEG datasets. Although higher sampling
rate was available for most datasets, at the point in time
where the analysis was carried out, the authors did not Author Contributions
have access to the computational power to calculate the
networks for higher frequencies. Anastasios Ziogas designed the study, performed data acquis-
ition, and helped in analyzing, drafting, revising, and approving
the manuscript. Andreas Mokros, Wolfram Kawohl, Benedikt
Habermeyer, and Elmar Habermeyer helped in the study design,
Conclusion result interpretation, and drafting and revising the manuscript.
Mateo de Bardeci and Ilyas Olbrich helped in analyzing the data,
Homosexual males did not show female-like patterns interpreting the results, and drafting and revising the manuscript.
in a pretrained network on sex but could be differentiated Sebastian Olbrich helped in the design of the study, performed
data analysis, and drafted and revised the manuscript.
from HeMs in a newly trained network. It was possible to
extract neurophysiological features and identify anatom-
ical regions of interest for further research. Thus, deep
Data Availability Statement
learning in the EEG domain has the potential to open new
avenues for research in the field by decoupling discrim- Data are not publicly available due to ethical reasons. Further
inative tasks from a priori-defined features. inquiries can be directed to the corresponding author.

References

1 Bailey JM, Vasey PL, Diamond LM, Bree- 5 De Block A, Adriaens PR. Pathologizing Sex- 10 Ganna A, Verweij KJH, Nivard MG, Maier R,
dlove SM, Vilain E, Epprecht M. Sexual ori- ual Deviance: A History. J Sex Res. 2013; Wedow R, Busch AS, et al. Large-scale GWAS
entation, controversy, and science. Psychol 50(3–4):276–98. reveals insights into the genetic architecture
Sci Public Interest. 2016 Sep;17(2):45–101. 6 Bieber I, Bieber TB. Male homosexuality. Can of same-sex sexual behavior. Science. 2019
2 Gold JH. Homosexuality: a review. Can Fam J Psychiatry. 1979 Aug;24(5):409–21. Aug;365(6456):eaat7693.
Physician. 1983 Mar;29:521–4. 7 Independent Forensic Expert Group. State- 11 Ngun TC, Vilain E. The biological basis of
3 Groneberg M, Funke C. Combatting homo- ment on conversion therapy. J Forensic Leg human sexual orientation: is there a role
phobia: experiences and analyses pertinent to Med. 2020 May;72:101930. for epigenetics? Adv Genet. 2014;86:
education. Berlin: Auflage LIT; 2011. 8 Blanchard R, Bogaert AF. Homosexuality in 167–84.
4 Röll W. Homosexual inmates in the buchen- men and number of older brothers. Am 12 Savic I, Lindström P. PET and MRI show
wald concentration camp. J Homosex. 1996; J Psychiatry. 1996 Jan;153(1):27–31. differences in cerebral asymmetry and func-
31(4):1–28. 9 Gavrilets S, Rice WR. Genetic models of homo- tional connectivity between homo- and het-
sexuality: generating testable predictions. Proc erosexual subjects. Proc Natl Acad Sci U S A.
Biol Sci. 2006 Dec 22;273(1605):3031–8. 2008 Jul;105(27):9403–8.

Deep Learning and EEG Patterns for Sexual Neuropsychobiology 11


Orientation DOI: 10.1159/000530931
13 Burke SM, Manzouri AH, Savic I. Structural 29 Gratton G, Coles MG, Donchin E. A new 44 Ponseti J, Bosinski HA, Wolff S, Peller M,
connections in the brain in relation to gender method for off-line removal of ocular artifact. Jansen O, Mehdorn HM, et al. A functional
identity and sexual orientation. Sci Rep. 2017 Electroencephalogr Clin Neurophysiol. 1983 endophenotype for sexual orientation in hu-
20;7(1):17954. Apr;55(4):468–84. mans. Neuroimage. 2006 Nov;33(3):825–33.
14 Abé C, Johansson E, Allzén E, Savic I. Sexual 30 Klein F, Sepekoff B, Wolf TJ. Sexual orientation: 45 Seok JW, Sohn JH. Neural substrates of sexual
orientation related differences in cortical a multi-variable dynamic process. J Homosex. desire in individuals with problematic hypersex-
thickness in male individuals. PLoS One. 1985;11(1–2):35–49. ual behavior. Front Behav Neurosci. 2015;9:321.
2014 Dec;9(12):e114721. 31 Qualls LR, Hartmann K, Paulson JF. Broad 46 Strahler J, Kruse O, Wehrum-Osinsky S,
15 Bogaert AF, Skorska MN, Wang C, Gabrie J, autism phenotypic traits and the relationship Klucken T, Stark R. Neural correlates of
MacNeil AJ, Hoffarth MR, et al. Male homo- to sexual orientation and sexual behavior. gender differences in distractibility by sexual
sexuality and maternal immune responsivity J Autism Dev Disord. 2018 Dec;48(12):3974–83. stimuli. Neuroimage. 2018 Aug;176:499–509.
to the Y-linked protein NLGN4Y. Proc Natl 32 Moore DL, Norris FH. Empirical investigation 47 Wang YF, Jing XJ, Liu F, Li ML, Long ZL, Yan
Acad Sci U S A. 2018 09;115(2):302–6. of the conflict and flexibility models of bisex- JH, et al. Reliable attention network scores
16 van Putten MJAM, Olbrich S, Arns M. Pre- uality. J Bisexuality. 2005 Apr;5(1):5–25. and mutually inhibited inter-network rela-
dicting sex from brain rhythms with deep 33 de Bardeci M, Ip CT, Olbrich S. Deep learn- tionships revealed by mixed design and
learning. Sci Rep. 2018 Feb;8(1):3069. ing applied to electroencephalogram data in non-orthogonal method. Sci Rep. 2015
17 Silverman D, Rosanoff WR. Electro- mental disorders: a systematic review. Biol May;5(1):10251.
encephalographic and neurological studies Psychol. 2021 May;162:108117. 48 Sun C, Shrivastava A, Singh S, Gupta A.

Downloaded from http://karger.com/nps/article-pdf/doi/10.1159/000530931/3931240/000530931.pdf by guest on 07 July 2023


of homosexuals. J Nerv Ment Dis. 1945; 34 Shinde S, Tupe-Waghmare P, Chougule T, Revisiting unreasonable effectiveness of
101(4):311–21. Saini J, Ingalhalikar M. Predictive and dis- data in deep learning era. 2017. p. 843–52.
18 Papatheophilou R, James S, Orwin A. Electro- criminative localization of pathology using [cited 2022 Feb 2].
encephalographic findings in treatment- high resolution class activation maps with 49 Najafabadi MM, Villanustre F, Khoshgoftaar
seeking hamosexuals compared with hetero- CNNs. Peerj Comput Sci. 2021;7:e622. TM, Seliya N, Wald R, Muharemagic E. Deep
sexuals: a controlled study. Br J Psychiatry. 35 Zhang Y, Hong D, McClement D, Oladosu O, learning applications and challenges in big
1975 Jul;127(1):63–6. Pridham G, Slaney G. Grad-CAM helps inter- data analytics. J Big Data. 2015 Feb;2(1):1.
19 Alexander JE, Sufka KJ. Cerebral lateraliza- pret the deep learning models trained to classify 50 Amezcua-Gutiérrez C, Marisela HG, Fernán-
tion in homosexual males: a preliminary EEG multiple sclerosis types using clinical brain dez Guasti A, Aguilar MAC, Guevara MA.
investigation. Int J Psychophysiol. 1993 Nov; magnetic resonance imaging. J Neurosci Meth- Observing erotic videos with heterosexual
15(3):269–74. ods. 2021 Apr;353:109098. content induces different cerebral responses
20 Razumnikova OM. Gender differences in hemi- 36 Li Y, Yang H, Li J, Chen D, Du M. EEG-based in homosexual and heterosexual men.
spheric organization during divergent thinking: intention recognition with deep recurrent- J Homosex. 2020;67(5):639–57.
an EEG investigation in human subjects. Neu- convolution neural network: performance 51 Lübke KT, Sachse C, Hoenen M, Pause BM.
rosci Lett. 2004 May;362(3):193–5. and channel selection by Grad-CAM. Neuro- Mu-suppression as an indicator of empathic
21 Wegesin DJ. Event-related potentials in ho- computing. 2020 Nov;415:225–33. processes in lesbian, gay, and heterosexual
mosexual and heterosexual men and women: 37 Pascual-Marqui RD. Discrete, 3D distributed, lin- adults. Arch Sex Behav. 2020 Feb;49(2):635–44.
sex-dimorphic patterns in verbal asymme- ear imaging methods of electric neuronal activity. 52 Ziogas A, Habermeyer E, Santtila P, Poeppl TB,
tries and mental rotation. Brain Cogn. 1998 Part 1: exact, zero error localization. 2007 Oct. Mokros A. Neuroelectric correlates of human
Feb;36(1):73–92. 38 Pascual-Marqui RD, Esslen M, Kochi K, sexuality: a review and meta-analysis. Arch Sex
22 Rajpurkar P, Chen E, Banerjee O, Topol EJ. Lehmann D. Functional imaging with low- Behav. 2023;52(2):497–596.
AI in health and medicine. Nat Med. 2022 resolution brain electromagnetic tomography 53 Seok JW, Sohn JH, Cheong C. Neural sub-
Jan;28(1):31–8. (LORETA): a review. Methods Find Exp Clin strates of sexual arousal in heterosexual
23 Craik A, He Y, Contreras-Vidal JL. Deep Pharmacol. 2002;24(Suppl C):91–5. males: event-related fMRI investigation.
learning for electroencephalogram (EEG) 39 Mulert C, Jäger L, Schmitt R, Bussfeld P, J Physiol Anthropol. 2016 Mar;35:8.
classification tasks: a review. J Neural Eng. Pogarell O, Möller H-J, et al. Integration of 54 Keysers C, Kaas JH, Gazzola V. Somatosen-
2019 Apr;16(3):031001. fMRI and simultaneous EEG: towards a com- sation in social perception. Nat Rev Neurosci.
24 LeCun Y, Bengio Y, Hinton G. Deep learning. prehensive understanding of localization and 2010 Jun;11(6):417–28.
Nature. 2015 May;521(7553):436–44. time-course of brain activity in target detec- 55 Mueller SC, Wierckx K, T’Sjoen G. Neural
25 Santtila P, Mokros A, Viljanen K, Koivisto M, tion. Neuroimage. 2004 May;22(1):83–94. and hormonal correlates of sexual arousal in
Sandnabba NK, Zappalà A, et al. Assessment 40 Olbrich S, Mulert C, Karch S, Trenner M, Leicht transgender persons. J Sex Med. 2020 Dec;
of sexual interest using a choice reaction time G, Pogarell O, et al. EEG-vigilance and BOLD 17(12):2495–507.
task and priming: a feasibility study. Leg effect during simultaneous EEG/fMRI measure- 56 Logothetis NK. What we can do and what we
Criminol Psychol. 2009;14(1):65–82. ment. Neuroimage. 2009 Apr;45(2):319–32. cannot do with fMRI. Nature. 2008 Jun;
26 Ziogas A, Habermeyer B, Kawohl W, Hab- 41 Pizzagalli DA, Oakes TR, Fox AS, Chung 453(7197):869–78.
ermeyer E, Mokros A. Automaticity of Early MK, Larson CL, Abercrombie HC, et al. 57 Logothetis NK, Pauls J, Augath M, Trinath T,
Sexual Attention: An Event-Related Potential Functional but not structural subgenual Oeltermann A. Neurophysiological investiga-
Study. Sex Abuse. 2022 Aug;34(5):507–6. prefrontal cortex abnormalities in melan- tion of the basis of the fMRI signal. Nature.
27 Ajdacic Gross V, Müller M, Rodgers S, cholia. Mol Psychiatry. 2004;9(4):325–405. 2001 Jul;412(6843):150–7.
Warnke I, Hengartner MP, Landolt K, 42 Guenther T, Schönknecht P, Becker G, Olbrich 58 Hu SH, Wei N, Wang QD, Yan LQ, Wei EQ,
et al. The ZInEP Epidemiology Survey: back- S, Sander C, Hesse S, et al. Impact of EEG- Zhang MM, et al. Patterns of brain activation
ground, design and methods. Int J Methods vigilance on brain glucose uptake measured during visually evoked sexual arousal differ be-
Psychiatr Res. 2014 Jun;23(4):451–68. with [(18)F]FDG and PET in patients with tween homosexual and heterosexual men. AJNR
28 Arns M, Bruder GE, Hegerl U, Spooner C, depressive episode or mild cognitive impair- Am J Neuroradiol. 2008 Nov;29(10):1890–6.
Palmer DM, Etkin A, et al. EEG alpha asymmetry ment. Neuroimage. 2011 May;56(1):93–101. 59 Stoléru S, Grégoire MC, Gérard D, Decety J,
as a gender-specific predictor of outcome to acute 43 Yarkoni T, Poldrack RA, Nichols TE, Van Lafarge E, Cinotti L, et al. Neuroanatomical
treatment with different antidepressant medica- Essen DC, Wager TD. Large-scale automated correlates of visually evoked sexual arousal in
tions in the randomized iSPOT-D study. Clin synthesis of human functional neuroimaging human males. Arch Sex Behav. 1999 Feb;
Neurophysiol. 2016;127(1):509–19. data. Nat Methods. 2011 Jun;8(8):665–70. 28(1):1–21.

12 Neuropsychobiology Ziogas/Mokros/Kawohl/de Bardeci/


DOI: 10.1159/000530931 Olbrich/Habermeyer/Habermeyer/Olbrich

You might also like