Emotion Recognition from Physiological

Signals Using Ensembled Machine

Learning Strategy
S. Fayaz Begum1 Fahimuddin Shaik2 Shaik Karimullah3 D. Mahalakshmi4,
Assistant Professor Professor Associate Professor UG Scholar
Dept. of ECE Dept. of ECE Dept. of ECE Dept. of ECE
AITS, Rajampet AITS, Rajampet AITS, Rajampet AITS, Rajampet
A.P, India A.P, India A.P, India A.P, India
Fayazbegums fahimaits munnu483 mahalakshmimahalaks
@gmail.com @gmail.com @gmail.com hmi16912@gmail.com

T. Lavanya5 M. Naveenkumarraju6 B. Mahendra7

UG Scholar UG Scholar UG Scholar
Dept. of ECE Dept. of ECE Dept. of ECE
AITS, Rajampet AITS, Rajampet AITS, Rajampet
A.P, India A.P, India A.P, India
lavanyathallapureddy3 naveenmeesara312 mahimynu
@gmail.com @gmail.com @gmail.com

ABSTRACT: being tested for its ability to accurately predict emotional

The field of emotion recognition by physiological signal
analysis is the subject of the proposed study. Its primary
objective is to evaluate how effectively various classifiers
predict two important emotional variables: valence and
arousal. Three distinct classifiers are used in the study:
kELM (Kernel Extreme Learning Machine), nELM (Neural
Extreme Learning Machine), and sSOM (Simplified Self-
Organising Map) in order to give a comprehensive
comparison analysis. The script contributes to the
advancement of emotion perception by carefully
contrasting the capabilities of multiple classifiers. By using
the dataset, the research hopes to clarify the benefits and
drawbacks of each classifier and contribute to the
development of more dependable and precise emotion
identification algorithms. The suggested study focuses on
the area of physiological signal processing for emotion
identification.. Its main goal is to assess how well different
classifiers perform in predicting valence and arousal, two
crucial emotional variables.
KEYWORDS: Emotion recognition, physiological signals,
valence, arousal, classifiers, kELM, nELM, sSOM.
characteristics from physiological information is kELM.
Another classifier that seeks to comprehend its potential for
emotion recognition is nELM. An artificial neural network
type called sSOM is made for pattern recognition and
unsupervised learning.
For emotion recognition from physiological signals, the
MATLAB script makes use of the Self-Organising Map
(SOM), including its simplified version (sSOM), and two
Extreme Learning Machine (ELM) variants: Kernel
Extreme Learning Machine (kELM) and Neural Extreme
Learning Machine (nELM). The script uses the
Concordance Correlation Coefficient (CCC) as the
performance indicator and preprocesses, interpolates, and
visualises the data using MATLAB. The performance of
the classifier is quantitatively evaluated through the
application of feature extraction techniques and statistical
analysis, namely the CCC calculation. The code addresses
model training and result visualisation in a unified
framework, allowing for a thorough investigation of
emotion identification through the smooth integration of
machine learning, signal processing, and visualisation
technologies.

I.INTRODUCTION: HCI implementation can lead to better user-friendly and

The process of recognising and analysing human emotions
is referred to as emotion recognition. Biological reactions
or activities that are detectable and quantifiable within the
human body are called physiological signals. The positive
or negative aspect of an emotional experience is
represented by valence. The term "arousal" describes the
degree of an emotional reaction. One of the classifiers
responsive interfaces, which increase user engagement. The
application of physiological signal-based emotion detection
in healthcare can improve monitoring and diagnostic tools
for mental health. Experiences can be personalised by
affective computing systems, which can adjust to users'
emotional states. Research helps create emotionally
engaging virtual environments for VR and gaming. By
adjusting the delivery of knowledge according to the
emotional states of students, educational technologies can
maximise engagement..
The research's emphasis, objectives, and methodology are
elucidated in the abstract, which makes a valuable
contribution. It highlights the three classifiers selected for
the comparative analysis and confirms the study's
importance in the field of emotion recognition. The abstract
highlights the potential effects of the research's findings
across a range of applications, including affective
computing, VR, gaming, HCI, healthcare, and educational
technology. It also offers a roadmap for the study.
II. LITERATURE REVIEW:
In 2023, Yujian Cai, et.al [1] suggested Emotion
Recognition with Diverse Sensors, Models, Methods, and
Datasets. Some limitations include data unreliability,
potential challenges extending results to other contexts, and
ethical problems.
In 2023, Anderson Faria Claret, et.al [2] Heart signals are
used to automatically classify the suggested emotions. This
study investigates signal collection techniques, feature
extraction tactics, and emotional elicitation protocols.
Limitations include the range of emotions and the
challenges associated in accurately classifying emotions
from cardiac data.
In 2022, Zhibin Zhu, et.al [3] highlighted the paper
Emotion Recognition Through Energy-Related Features of
Peripheral Physiological Signals. It ascertains the four
finest attributes through correlation analysis and decision
tree models for discrimination ability evaluation. One of
the strategy's drawbacks might be the difficulty of using it
with a larger range of emotions and demographics.
III. PROPOSED METHODOLOGY:
Fig.1:Block Diagram
Uses the RECOLA dataset's EEG, dual-channel peripheral
(EOG, EMG), and single-channel peripheral (GSR, skin
temperature, respiration, BVP) signals. Films were rated by
participants according to like, dominance, arousal, and
valence. Preprocessing prepares the dataset for analysis of
peripheral physiological signals by performing operations
such as segmentation, downsampling, filtering, and artefact
removal. Physiological signal recordings are split into
smaller temporal windows using this technique, and
In 2020, Jianhua Zhang, et.al [4] displayed Emotion
recognition uses multi-modal data and machine learning
techniques. It covers feature extraction techniques, feature
reduction, and machine learning classifiers. Interpretability
and data acquisition are the limitations.
In 2020, Vikrant Doma, et.al [5] suggested a comparison of
machine learning techniques for EEG and peripheral
physiological signals-based emotion identification. If brain
signals are utilised to identify emotions, then the diagnosis
and course of treatment for several medical conditions may
alter. Broad emotion identification software may have
problems and limitations because emotions may only be
concealed by a small number of facial expression triggers.
In 2019, Luca Romeo, et.al [6] recommended Emotion
recognition via multiple-instance learning based on
physiological cues. This method can be used in weakly
supervised circumstances because it does not require
continuous tagging. Rather, it records the existence or lack
of relevant states to model temporal intervals. Limitations
include dataset reliance and interpretability of the model.
In 2017, Salma Alhagry, et.al [7] described in detail An
LSTM recurrent neural network offers a deep learning
approach for emotion recognition is from raw EEG signals
by employing long-short-term memory (LSTM) to train
features. Limitations include complexity and training data
statistical properties are taken out of each window. In
feature extraction methods, statistical functions like mean,
maximum, minimum, entropy, skewness, kurtosis, median,
zero crossings, and standard deviation are commonly used.
Using structural functions, each feature is retrieved from
signal points. Eq. 1
 f ij=φi ( X j )=φ i (x1j , x 2j , x 3j , .. , x Kj )
(1)
Data segmentation is the process of dividing the dataset
into training and development sets. The training set is used
to train machine learning models, and the development set
is used to evaluate the models' performance and help them
become more generic. uses the Concordance Correlation
Coefficient (CCC) to assess classifier performance on both
the training and development sets. The logistic regression
(LR), random forest (RF), and support vector machine
(SVM) techniques are examples of classifiers for emotion
recognition. Each classifier receives the gathered
characteristics, as shown by Eq. 2
j j j j j j
M =¿Ф( f ¿ = Ф( f 1 , f 2, f 3 , .. , f L ¿ = Ф(
φ 1 ( X j ) , φ2 ( X j ) , φ3 ( X j ) , .. , φ L (X j )¿
For training and development purposes, the classifier,
target, outputs, references, and CCC are all kept in a cell
array together with the results. displays the CCC results for
valence and arousal for each classifier. Plots of the arousal
and valence classifier outputs relative to the reference are
produced. The graphical depiction of the trained sSOM
model provides an example of the mapping of emotions in
the arousal-valence space.

IV. RESULT AND ANALYSIS: Arousal

Tense Excited

Delighted

ANGRY HAPPY

Frustrated Clated

NEUTRAL Valence

Depressed Content

SAD RELAXED

Bored Calm

Tired

Fig .2 :Reference graph

The Fig. 2 is the reference graph for the result. By the Fig 2 we can get to know the emotion of the humans through
physiological signals.

S.NO VALENCE VALUES AROUSAL VALUES EMOTION

1. Positive Positive Happy
2. Negative Positive Angry
3. Positive Negative Relaxed
4. Negative Negative Sad

Table 1:Reference values of Valence and Arousal values.

3.(a) 3.(b)

3.(c) 3.(d)

3.(e)

Fig. 3: Comparative study arousal values of different classifiers with respect to reference value .

The fig.3 represents datasets of arousal of different human (Self-Organizing Map), kELM (Kernel Extreme Learning
emotions , that changes at every instant of time. The Machine), and nELM (Neural Extreme Learning Machine).
arousal is represented in 3 classifiers, which are sSOM The graph is drawn with respect to Mean rating and Time.
 4.(a) 4.(b)

4.(c) 4.(d)

4.(e)

Fig. 4: comparative study valence values of different classifiers with respect to reference value .
 The fig. 4 represents datasets of valence of different
human emotions , that changes at every instant of time.
The arousal is represented in 3 classifiers, which are
sSOM (Self-Organizing Map), kELM (Kernel Extreme
Learning Machine), and nELM (Neural Extreme
Learning Machine). The graph is drawn with respect to
Mean rating and Time.
Circumplex Model of Emotions:
According to the circumplex paradigm, different
emotions are arranged in a circular or two-dimensional
area based on their valence and arousal.
FIG: 5. MAPPING EMOTIONS INTO QUADRANTS
MAPPING EMOTIONS INTO QUADRANTS
0.5
0.3
VALENCE
Sections: Using Table 1, divide the circumplex space into
quadrants. Positive valence and high arousal: energising
and upbeat feelings (e.g., happiness, excitement).
Negative valence, high arousal: intensely felt emotions
(such as fear and rage). Positive valence, low arousal:
calm and happy feelings (such relaxation and happiness).
Low arousal, negative valence: Emotions that are calm
and negative (such as boredom and despair).
Table 2 displays the arousal and valence values for the
various classifiers as well as the dataset's dev1 reference.
Valence and arousal are commonly mapped onto a model
or framework that depicts various emotions in order to
facilitate the use of these two characteristics in emotion
recognition. Arousal indicates the degree of intensity or
activation of an emotion, whereas valence indicates how
pleasant or unpleasant the emotion is. This is a popular
method for mapping feelings according to their valence
and arousal. From fig 5 we can see that all the values
which are plotted , are present in the 1st
Quadrant .According to Fig 1 and Fig 2 we can say that
the emotion of the human is happy.

0.1
sSOM Table 3: Mean values of Arousal and Valence for
nELM different classifiers of fig. 3 and, fig. 4.
kELM
-0.5 -0.3 -0.1
-0.1 0.1 0.3 0.5
Ref-
er- Classifier Arousal Valence
-0.3 ence

-0.5 sSOM 0.402 0.354

AROUSAL
FROM TABLE 2 nELM 0.399 0.375

Harmony Axis (Tilted): Along the horizontal axis,

emotions range from pleasant (positive valence) to
unpleasant (negative valence). The Arousal Axis is a
vertical axis that shows how strongly or activatedly
emotions range from low arousal (calm or low energy) to
high arousal (excited or high energy).
Mapping Emotions :
Table 2: Comparative study arousal and valence values
of different classifiers with respect to reference value
from 3(a),4(a)
kELM 0.383 0.332
In Table 3 of the emotion recognition study, these
numbers correspond to the level of arousal and valence
that each classifier predicted for a certain input. Higher
arousal values usually correspond to stronger emotions,
even though higher valence levels might also suggest a
more positive emotional state.

Classifiers/Ref Arousal Valence

SSOM 0.063 0.038

NELM 0.122 0.188

KELM 0.07 0.123

Ref 0.07 0.188

 Fig. 4: Heatmaps of Arousal and Valence.
Arousal and valence heatmaps are graphical depictions of
these emotional states' degrees of intensity or level under
various circumstances, throughout time, or from multiple
data points.
Heatmap for Arousal:
Usually, arousal is a measure of an emotion's intensity or
degree of activation. Each cell in a heatmap represents a
distinct data point or situation, and the colour of the cell
reflects the arousal level. For example, higher arousal
may be represented by a darker colour, like range. A
lighter colour, like blue, may indicate a low level of
arousal. The heatmap's rows and columns can represent
various samples and time intervals, letting you see trends
and variances in arousal levels.
Heatmap for Valence:
[2] Anderson Faria Claret, Karina Rabello Casali,
Tatiana Sousa Cunha & Matheus Cardoso Moraes (2023)
Automatic Classification of Emotions Based on Cardiac
Signals: A Systematic Literature Review. Ann Biomed
Eng 51, 2393–2414 .
[3] Zhibin Zhu, Jingwen Feng, Xuanyi Wang, Yifei Xu,
Huiling Zhou, Jingjing Sun, Wenchen Jiang, Hang Chen,
(2022) . Emotion Recognition Based on Energy-related
Features of Peripheral Physiological Signals. pp. 1895-
1901, doi: 10.1109/EMBC48229.2022.9871935.
[4]
Jianhua Zhang , Zhong Yin , Peng Chen , Stefano Nichel
e (2020) Emotion recognition using multi-modal data
and machine learning techniques . pp.103-126.
[5] Vikrant Doma, , Pirouz, M. A comparative analysis of
machine learning methods for emotion recognition using
EEG and peripheral physiological signals. J Big Data 7,
18 (2020).

The positive or negative aspect of an emotion is
represented by its valence. Each cell in a heatmap
represents the degree of valence for a particular condition
or data point. In a similar vein, darker hues, like blue,
may indicate a more negative valence. Yellow and other
lighter hues could indicate a more positive valence.
Similar to the arousal heatmap, the valence heatmap's
rows and columns can stand for various samples,
intervals of time, or categories.
[6] Luca Romeo; Andrea Cavallo; Lucia Pepa, Nadia
Bianchi-Berthouze, Massimiliano Pontil
(2019) Multiple Instance Learning for Emotion
Recognition Using Physiological Signals. Vol. 13 ,pp.
389 – 407.
[7] Salma Alhagry, Aly Aly Fahmy, Reda A. El-Khoribi
(2017). Emotion Recognition based on EEG using LSTM
Recurrent Neural Network. Vol. 8, No. 10, pp. 355-358.

V. CONCLUSION:

In this work, we have effectively built an efficient

framework that identifies emotions by extracting valence
and arousal values from physiological signals. Accurate
prediction of two crucial characteristics of emotional
states—arousal and valence—was made achievable by
advanced machine learning techniques. Our approach
provided quantitative assessments, and we successfully
translated these values into defined emotional categories.
The arousal and valence values are represented by three
classifiers: the self-organizing map (sSOM), the kernel
extreme learning machine (kELM), and the neural
extreme learning machine (nELM). The optimal
classifier is identified by the highest CCC (concordance
correlation coefficient). Based on CCC, we conclude that
the kELM has the highest CCC value. Therefore, it is the
best classifier when compared to others.

REFERENCES:

[1] Yujian Cai, Xingguang Li, Jinsong Li (2023)

Emotion Recognition Using Different Sensors, Emotion
Models, Methods and Datasets. Sensors 2023, 23(5),
2455 [Google Scholar].