Computer Methods and Programs in Biomedicine 190 (2020) 105408

Contents lists available at ScienceDirect

Computer Methods and Programs in Biomedicine

journal homepage: www.elsevier.com/locate/cmpb

Towards an anxiety and stress recognition system for academic

environments based on physiological features
Jorge Rodríguez-Arce, Ph.D. a,∗, Liliana Lara-Flores, M.Sc. a, Otniel Portillo-Rodríguez, Ph.D. a,
Rigoberto Martínez-Méndez, Ph.D. a
a
Facultad de Ingeniería, Universidad Autónoma del Estado de México, Toluca, Mexico

a r t i c l e i n f o a b s t r a c t

Article history: Background and Objective: Traditional methods to determine stress and anxiety in academic environments
Received 6 April 2019 consist of the application of questionnaires, but the main disadvantage is that the results depend on the
Revised 17 November 2019
students’ self-perception. Being able to detect anxiety-related stress levels in a simple and objective way
Accepted 18 February 2020
contributes greatly to dealing with low performance and school drop-out by students.
Methods: The main contribution of this study is to identify the physiological features that could be used
Keywords:
as predictors of stressful activities and states of anxiety in academic environments using an Arduino
Stress
Anxiety board and low-cost sensors. A test with 21 students was conducted, and a stress-inducing protocol was
Classifiers proposed and 21 physiological features of five signals were analyzed. In addition, the State-Trait Anxiety
Physiological data Inventory (STAI) was used to assess the level of anxiety for each student. Four classifiers were compared
to find the physiological feature subset that provides the best accuracy to identify states of stress and
anxiety.
Results: The stress due to activities performed by students can be identified with an accuracy greater
than 90% (Kappa = 0.84) using the k-Nearest Neighbors classifier, using data from heart rate, skin tem-
perature and oximetry signals and four physiological features. Meanwhile, the identification of anxiety
was achieved with an accuracy greater than 95% (Kappa = 0.90) using the SVM classifier with data from
the galvanic skin response (GSR) signal and three physiological features.
Conclusions: The results provide a clue that anxiety detection in academic environments could be done
using the analysis of physiological signals instead of STAI test scores. Besides, the results suggest that
physiological features could be used to develop stress recognition systems to help teachers to identify the
stressful tasks in an academic environment or to develop anxiety recognition systems to help students to
control their level of anxiety when they are performing either academic tasks or exams.
© 2020 Elsevier B.V. All rights reserved.

1. Introduction
An academic environment is where students experience a vari-
ety of stress-related situations such as giving a class presentation,
solving problems against the clock, and dealing with tests and ex-
aminations. The stress that arises in an educational environment
is called academic stress [10], and refers to the stress-related sit-
uations during the educational process in which students usually
have no control and may cause students’ physiological reactions
(e.g. changes in their blood pressure, skin conductance, and res-
∗
Corresponding author.
E-mail addresses: jrodrigueza@uaemex.mx (J. Rodríguez-Arce),
lililafl@hotmail.com (L. Lara-Flores), oportillor@uaemex.mx (O. Portillo-Rodríguez),
rmartinezme@uaemex.mx (R. Martínez-Méndez).
piratory rate) [37]. In addition, Berrío and Mazo [5] define aca-
demic stress as the condition that students must endure as a re-
sult of a variety of stressors, related to the activities usually carried
out at school. Kadapatti and Vijayalxmi (Kadapatti & Vijayalaxmi,
2012) point to academic stress as the product of a combination
of academic demands that exceed the adaptive resources available
to an individual. In general, academic stress refers to the student’s
dealing with demands or stressors in academic environments that
cause physiological, emotional, cognitive or behavioral discomfort.
In recent years, some studies suggest that academic stress is
related to low performance and dropout, mainly among first-year
undergraduate students. According to García-Ros et al. [10], an
increase in stress levels of undergraduate students occurs more
frequently during the year of university admission, because stu-
dents are dealing with a process in which they must face greater

https://doi.org/10.1016/j.cmpb.2020.105408
0169-2607/© 2020 Elsevier B.V. All rights reserved.
2 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408
demands in school activities, improve their time management,
have greater autonomy, among others. Kitsantas, Winsler, and Huie
[20] report that only 73.6% of newly-enrolled undergraduate stu-
dents return to continue their studies the following year.
The early detection of academic stress in students represents
a challenge for universities. To face this challenge, the institution
must have enough trained personnel to detect the students most
likely to suffer from stress, and to implement preventive actions
and consequently reduce the dropout rate caused by academic
stress. Anxiety is one of the effects of stress, hindering student
concentration, memory capacity, and other processes causing lower
academic performance of students [10]. Some authors have studied
the relationship between academic stress, anxiety levels and the
academic performance of students [[7],[42]], and the results show
a negative correlation among them.
Traditional methods to determine stress and anxiety consist of
the application of questionnaires, self-reports (C. & K., 2016) and
clinical interviews [39]. Beiter et al. [4] used the 21-question ver-
sion of the Depression Anxiety Stress Scale (DASS-21) to evaluate
374 undergraduate students. Their results show that anxiety, aca-
demic performance and pressure to succeed are the main stressors
among students. The study concludes that problems derived from
the state of anxiety tend to hamper the success of university stu-
dents. It is therefore important for universities to continuously as-
sess the mental health of their students and implement prevention
methods to improve their academic performance.
In their study, DordiNejad et al. [7] discuss the effect of
test anxiety on medical students’ academic performances, us-
ing the Sarason Test Anxiety (STA) questionnaire to collect data.
Their data analysis shows a negative correlation between aca-
demic performance and test anxiety. Vitasari et al. used the
State-Trait Anxiety Inventory (STAI) to analyze the relation-
ship between anxiety levels and academic performance [41].
Their experiment shows a significant correlation between high-
level anxiety and low academic performance among engineering
students.
Other authors such as Jing [16], Peleg [27], and Onyeizugbo
[26] also show that undergraduate students with higher test anx-
iety have low academic performance. Reddy, Menon, and Thattil
[30] show that academic stress is a devastating problem affecting
the mental health of students and, thus, affecting their academic
performance. Their work shows that high stress levels were re-
ported in engineering and medical students, and it emphasizes the
need for medical interventions during the semester.
Although several authors have shown a significant correlation
between anxiety levels and the academic performance of students
using self-report questionnaires, the main disadvantages of this
method are the need for an expert (usually a psychologist) to in-
terpret the results of questionnaires, and the fact that the results
could be misleading since the answers on the questionnaires de-
pend on students’ perception.
Stress and anxiety detection methods may also be based on
the acquisition and analysis of subjects’ physiological information
(physiological signals). These methods are based on the fact that
the physiological signals present changes when the subject is un-
der stressful situations or activities [[12],[35]]. Physiological signals
are obtained from the human body by means of several types of
biomedical sensors, providing information of vital functions. These
signals can be described in terms of amplitude with respect to
time and frequency [29]. The physiological features (or character-
istics) are measurable variables related to the physiological signals,
which allow to monitor the body function. It is important to un-
derstand that physiological signals are raw signals acquired by sen-
sors, while physiological features are generally “hidden” in the raw
signals and they should be extracted by means of different signal
processing techniques [17]. Biomedical sensors or biosensors are
electronic instruments that allow to acquire physiological signals
for further processing and analysis.
It is widely known that biosensors can provide objective mea-
sures of stress response under specific tasks. For example, Plarre
et al. [28] exposed 21 subjects to a different kind of stressors, rep-
resentative of those experienced during daily activities. They pro-
posed two models to classify stress using wearable sensors; their
results showed a significant correlation between their physiological
classifiers and the participants’ self-reported ratings. Wijsman et al.
[43] used a wearable sensor system to measure physiological sig-
nals such as electrocardiogram (ECG), respiration, galvanic skin re-
sponse (GSR), and electromyography (EMG) to detect mental stress.
They presented three different stress conditions to a healthy sub-
ject group and calculated 19 physiological features from all signals.
Their results showed almost 80% of classification accuracy differen-
tiating stress and non-stress conditions using different classifiers.
Aigrain et al. [1] proposed a multi-assessment methodology to an-
alyze stress. They evaluated the predictive power of 101 physiolog-
ical features from EMG, GSR, temperature and heart rate (HR) sig-
nals. Their results showed that features related to body movement,
blood volume pulse, and heart rate are more useful for the classifi-
cation of stress. Vildjiounaite et al. [40] proposed a method to clas-
sify stress based on Hidden Markov Models and physiological data.
They tested the method and it achieved an average stress detec-
tion accuracy of 75%. Maaoui and Pruski [22] proposed a compar-
ative study of unsupervised approaches to detecting stress using
the heart rate signal analysis method. They employed three un-
supervised classification techniques (K-means, the Gaussian mix-
ture model and self-organizing map). Their results showed that
K-means clustering performs the most accurate stress detection.
Sevil et al. [34] focused on social and competitive stress detection
to discriminate between stress and non-stress states using wrist-
band physiological signals. Their results show that different types
of stress can be detected with an accuracy of over 87%.
Nevertheless, although previous studies show that the use of
some classifiers provides high accuracy of stress detection (in gen-
eral, accuracy of over 70% [[2],[12],[35]]), it should be noted that
these studies do not provide conclusive results on the stress in-
duction methods and physiological signals that can be useful for
stress and anxiety identification; it seems that the methods and
signals differ from one context to another (for example, between
driving activities and academic tasks). Thus, it is necessary to cor-
rectly define the stressor tasks, signal processing and classification
methodologies for each environment, activity and context. In ad-
dition, Giannakakis et al. [12] mention that most of the previous
experiments were performed in well-controlled environments, in
laboratory, these results could be different if the experiments are
executed in real-life environments; then, it is necessary to conduct
new studies with real-life conditions using induced stressors re-
lated to the task to be evaluated.
As a consequence, this research analyzes the use of Arduino
board and low-cost sensors to acquire physiological data in com-
bination with the STAI self-report questionnaire in an academic
environment (this questionnaire is the most widely used by the
counseling and psychological services office at the School of Engi-
neering, where this research was carried out, in order to evaluate
students’ anxiety levels). The advantage of this approach is to avoid
the results of the diagnosis depending on students’ self-perception.
The main contribution of this work is to identify the physiological
features that can be used to predict stressful activities and states
of anxiety in students in real academic environments.
2. Methods
Fig. 1 shows the architecture of the electronics platform used to
acquire physiological signals. The platform consists of: (a) a stress-
 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408 3

Fig. 1. Architecture of the electronics platform used in this study.

inducing protocol, (b) a data acquisition system (DAQ) and sensors

to acquire the psychological signals, (c) a database to store the sig-
nals for further analysis, and (d) a statistical model to identify the
stress and anxiety based on the STAI questionnaire results.

2.1. Stress-inducing protocol (stress conditions)

Determining whether an individual is experiencing stress or not

through physiological variables is not an easy task; it must be
guaranteed that the physiological changes detected at the time of
measurement are caused by stress and not by other factors (e.g.
other emotions, physical activity, and environmental factors). The
methodology used in similar studies to guarantee this was induc-
ing stress by presenting known stressful activities to the subject.
However, the number of stressful activities proposed is lim- Fig. 2. Screenshot of the tablet application used as the stress task (Rising Number
ited in the literature. Moya-Albiol and Salvador [24] mention that application by Wazo for Ipad).

among the activities most commonly used as psychological stres-

sors are: public speaking, arithmetic calculations, “the Stroop task”
simple questions such as “how do you feel?”, “how was your
[18], and other stressful tasks such as reaction time activities,
day?”; in addition, the instructions for the next task are ex-
video-games, and solving cognitive problems.
plained by the experimenter. The objective of this task is for
Sandulescu et al. (Sandulescu, Andrews, Ellis, Bellotto, & Mozos,
participants to rest between tasks.
2015) used a protocol based on the Trier Social Stress Test (TSST) • Non-stress task: this consists of listening to instrumental music
to identify stress in their experiment. The protocol consisted of a
while the subject is seated, as relaxed as possible. Salai et al.
neutral task (known as anticipatory stress phase), followed by the
[32] report that sometimes the stress-task could be more re-
tasks of speaking in public, one cognitive activity and again the
laxing than the non-stress task for some subjects, decreasing
neutral task until completion [19]. The neutral task consisted of
the stress level detection. In order to solve this issue, the sub-
two minutes only answering simple questions. The task of speak-
jects are also asked to close their eyes and to imagine a relaxing
ing in public consisted of a five-minute interview to obtain a job,
landscape, trying to minimize as much as possible any stressful
then each participant was asked to perform the cognitive task,
thoughts in order to enhance the relaxation state. In addition,
which consisted of counting backward in steps of 13 starting from
according to Ogba et al. [25], the fact of listening to music with
1022.
one’s eyes closed reduces levels of academic-related stress in
Salai, Vassányi, and Kósa [32] measured physiological parame-
undergraduate students.
ters of a control group while each individual listened to relaxing • Stress-task: a time-constrained arithmetic task was selected as
music. They recorded the values of the physiological signals and
a stressful task based on the works of Sandulescu [33] and
then measured the same parameters while the participants per-
Salai, Vassányi, and Kósa [32]. Students were asked to solve a
formed the “Stroop color test”. In the last task, the subject was
puzzle in which they had to add two numbers to calculate a
asked to achieve their best score in ten minutes; if an error was
specific value. The puzzle is a tablet-based application game,1
made, the game ended and the score restarted from 0.
based on the principle of addition, in which a set of different
Based on the previous work, in this study a stress-inducing pro-
positive numbers are presented in a random way on the left
tocol for undergraduate students was proposed using three differ-
and right edges of the screen, and a specific value (score) is
ent tasks: a neutral task, a non-stress task, and a stress task. Each
shown in the center of the screen (see Fig. 2). The students had
task is described in detail below:
to choose three numbers from the edges in such ways that,
• Neutral task: this task is similar to that proposed by
Kirschbaum et al. [19] and Sandulescu [33]. The students re- 1
Rising Number App by WAZO. https://appadvice.com/app/rising-number/
main seated without making any movement. They are asked 907455091 (accessed February 2, 2019).
4 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408
Fig. 3. On the left, the GSR sensor is placed on the index and middle fingers and the oximeter sensor is placed on the thumbs of the students. On the right, the breathing
sensor was placed near the nose.
when added, the result is the value shown in the center of
the screen (users must calculate the sum of three numbers).
Each time the students give the correct answer their score in-
creases and a new value is shown on the screen. In order to
increase the complexity of the task, the new value is greater
than the previous one. Each student must continue until they
reach a score equal to or higher than 100 points in five minutes.
During the task, if a mistake is made, the game ends and the
score will restart from zero. According to the participant’s per-
formance, they could obtain a candy (score equal to or higher
than 100 points) or an electrocutaneous stimulation (score less
than 100 points). According to Rhudy and Meagher [31], ex-
posure to electrocutaneous stimulation (or electric shock) in-
duces fear, while anticipation of shock (without exposure) in-
duces anxiety. In this study, the electrocutaneous stimulations
consisted of a brief shock delivered with 500 ms pulse dura-
tion and less than 5.0 mA (slight shock felt; not painful [9]).
2.2. Devices and sensors
The e-Health V2 (Cooking Hacks, Spain) shield and the Arduino
Uno board [3] were used as the DAQ system. The Arduino board
has a CPU speed of 16 MHz and a built-in 10-bits ADC (analog-to-
digital converter) that provides enough resolution to read the sub-
jects’ physiological signals. The e-Health V2 platform is a low-cost
platform that could be used to acquire raw physiological data in
real-time from nine sensors. In this study only four of them were
chosen to acquire five physiological signals from students:
• Body temperature: this allows skin temperature to be measured
with a 0.1 °C resolution; the sensor was placed in the axillary
area of each student. A piece of adhesive tape was used to at-
tach this sensor to the skin.
• Galvanic skin response: this sensor measures the electrical con-
ductance of the skin which is affected by the sweat, controlled
by the sympathetic nervous system. As a consequence, in
episodes of stress, the electrical resistance of the skin changes.
For this study, the sensor was placed on the index and middle
fingers (see Fig. 3) of the left hand. Adhesive tape was used to
attach the sensor to the skin of each finger.
• Pulse oximeter (for heart rate and oxygen saturation): this is a
sensor capable to measure the amount of oxygen dissolved in
the blood (oxygen saturation). This sensor can also measure the
heart rate and provides beats per minute (bpm) values. This
sensor was attached to the thumb of the left hand (see Fig. 3).
• Breath-rate sensor: this is a thermistor that was attached near
the nose (see Fig. 3) to measure the nasal thermal airflow
changes (the nasal temperature air). This sensor was used to
measure the students’ respiratory rate.
2.3. Experimental design
2.3.1. Experimental procedure
The experiment was conducted at the facilities of the School of
Engineering at the Universidad Autónoma del Estado de México.
Participants followed the protocol (see Fig. 4), as described below:
1 Familiarization session: In this stage, the objective and procedure
of this experiment were explained to each participant. They
were also notified about the prize that they will receive ac-
cording to their performance during the test. Participants who
agreed to continue filled out an identification sheet and signed
a consent form. Participants were given instructions regard-
ing to the two tasks that they had to complete, and also re-
ceived a brief demonstration of how to perform each task. For
example, in the case of the arithmetic task, each participant
had two minutes to practice the task and understand how the
tablet application works. In this session, the participants also
received an explanation about the sensors that were used to
collect the physiological data. At the end of this session, all sen-
sors were placed on the participant and their operation verified
(one-minute data acquisition). Participants remained seated for
the duration of the experiment; they could look at the screen
showing the time remaining for each activity. All participants
were right-handed, so the sensors were placed on their left
hand, enabling them to use their right hand to perform the ex-
perimental activities. Fig. 5 shows the experimental setup.
2 Test session: Before starting with the data acquisition, each par-
ticipant was encouraged to clarify any concerns because during
the experiment they would be not allowed to talk and had to
remain seated. They were also asked to be as quiet as possible,
making only the necessary movements to perform the experi-
mental tasks. The activities were carried out in the same order
and duration for each participant. The stress-inducing protocol
was composed of three main activities: a neutral task, a non-
stress task, and a stress task. The experiment started with a
neutral task for a period of three minutes. The experiment con-
tinued with the non-stress task. In this case, the selected mu-
sic was a piece of instrumental piano lasting 4:30 min. At the
beginning of the music, the participant was asked to close their
eyes and not open them until instructed to do so. Once the mu-
sic ended, the participant was asked to open their eyes and the
neutral task was repeated for three minutes. The next activity
was the stress-task. As mentioned in the previous section, each
 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408
Fig. 4. Experimental procedure.
Fig. 5. Experimental setup: a student is doing the arithmetic task.
participant had five minutes to achieve a score higher than 100
points in an arithmetic task.
3 STAI questionnaire: To collect data about anxiety, the State-Trait
Anxiety Inventory was applied to each participant. The ques-
tionnaire consists of 40 questions with a 4-point Likert scale;
higher levels of anxiety are positively correlated with higher
scores. This psychological inventory measures two types of anx-
iety: anxiety about an event (state-anxiety, 20 items) and anx-
iety level as a personal characteristic (trait-anxiety, 20 items).
The experiment ended by removing the sensors attached to the
participant.
2.3.2. Participants
Twenty-one right-handed students participated in this study.
All participants in this study were undergraduate engineering stu-
dents. There were 9 female students and 12 male students, with
ages ranging from 18 to 21 years. All participants reported normal
vision and hearing; none of them reported having consumed any
medication or being under any medical or psychological treatment.
They were chosen by random sampling, only considering the crite-
rion that they were in the first or second semester at university.
In order to minimize students’ potential concerns about their aca-
demic duties and/or exams, all the tests were carried out at the
end of the semester (after the end of the evaluation period).
5
Table. 1
Typical spectral characteristics of physiological sig-
nals (adapted from (Mendes, Figueiredo, Fernandes, &
Gama, 2011)).
Physiological signal and label Bandwidth (Hz)
Skin temperature (ST) 0–1
Oximetry (SpO2) 0–30
Breath-flow rate (BR) 0–10
Heart rate (HR) 0–5
Galvanic skin response (GSR) 0–5
2.4. Data acquisition and feature extraction
Five signals of each subject were recorded: heart rate (HR), skin
temperature (ST), galvanic skin response of the hand (GSR), oxy-
gen saturation (SpO2), and breath-flow rate (BR). The sampling rate
was set to 50 Hz. Table 1 shows the typical spectral characteristics
of the recorded signals; as a consequence, each signal was filtered
using a low-pass 6th-order Butterworth filter. Fig. 6 shows the raw
and filtered signals of “participant #5 .
MATLAB (2016a, MathworksTM ) software was used for process-
ing each signal (time-series data) and for feature extraction. The
signal of each subject was divided into individual segments ac-
cording to the experimental tasks (see Fig. 7); two segments be-
long to the neutral task (segments #1 and #3 of Fig. 7), one seg-
ment belongs to the non-stress task (segment #2 of Fig. 7), and
one segment belongs to the stress-task (segment #4 of Fig. 7). The
two neutral task segments were discarded for the data analysis.
The non-stress task and the stress-task segments were divided into
windows-size of 2500 samples intervals; this windows-size was
based on the feature extraction methodology proposed by other
authors [[8],[11]].
Four features were extracted for each of the following physio-
logical signals: HR, ST, GSR and SpO2. For the case of the BR signal,
five features were extracted. Consequently, a total of 21 features
were calculated for further analysis; the same feature extraction
procedures described in [[6],[14],[15],[36],[43]] were used. The fea-
tures for each physiological signal and its feature extraction proce-
dure are summarized in Table 2.
In order to evaluate the performance among the four classifiers
and the four datasets, the accuracy and the Cohen’s Kappa coef-
ficient were calculated. In this study, the accuracy is defined as
6 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408

Fig. 6. Left side, raw signals; right side, filtered signals of participant #5.

Fig. 7. Each signal was divided into five individual segments: segment numbers 1and 3 belong to the neutral task, segment number 2 belongs to the non-stress task, and
segment number 4 belongs to the stress task.

Table. 2
Feature extraction of each physiological signal (HR – heart rate, ST – skin temperature, GSR – galvanic skin response, SpO2 –
saturation of oxygen, BR- breath flow rate).

Derived from Signal HR Signal ST Signal GSR Signal SpO2 Signal BR

Mean normalization by Healey [14] HR1 ST1 GSR1 SpO2-1 BR1

Root mean square (RMS) [43] HR2 ST2 GSR2 SpO2-2 BR2
Means of differences between adjacent elements [15] HR3 ST3 GSR3 SpO2-3 BR3
Sum of local peak [36] —— —— —— —— BR4
Mean normalization by Iliou [15] HR5 ST5 GSR5 SpO2-5 BR5
 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408 7

Table. 3 bels were proposed based on the type of task, non-stress task, and
Interpretation of Cohen’s Kappa value (adapted from [[21],[23]]).
stress task.
Kappa value Interpretation % of data that are reliable In addition, a feature selection process was run to reduce the
<0 None or poor agreement 0–4% size of the feature vector that could provide a higher classifica-
0.0 – 0.20 Minimal agreement 4–15% tion value (thus reducing the number of sensors in comparison
0.21 – 0.40 Weak agreement 15–35% with the feature vector of the Full Dataset). Two correlation-based
0.41 – 0.60 Moderate agreement 35–63% attribute subset evaluators (CfsSubsetEval and CorrelationAttributeE-
0.61 – 0.80 Strong agreement 64–81%
val) and one information gain attribute evaluator (InfoGainAttribu-
0.81 – 1.00 Almost perfect agreement 82–100%
teEval) were used to create three new datasets. The CfsSubsetEval
evaluator assesses the worth of a subset of attributes by consider-
ing the individual predictive ability of each feature along with the
the percentage of testing set examples correctly classified; in other degree of redundancy between them [13]. The CorrelationAttribu-
words, it is the fraction of predictions the classifier got right. On teEval evaluator estimates the worth of an attribute by measuring
the other hand, the Cohen’s Kappa value is a statistical measure of the Pearson’s correlation between it and the class. On the other
inter-rater agreement between predicted and actual values, and it hand, the InfoGainAttributeEval evaluator measures each attribute
is useful to handle imbalanced class problems. In this study, the by evaluating their information gain with respect to the class. It
Cohen’s Kappa was used to evaluate the fitness of each classifier, discretizes numeric attributes first using the Minimum Descrip-
and its interpretation is based on the nomenclature proposed by tion Length method [44]. All these attribute evaluators are tools
Landis and Koch [21], where a perfect agreement would equate for data preparation provided by Weka 3.8 software. Table 5 shows
to a value of 1, and a non-agreement would equate to 0. The a summary of the results of each dataset.
nomenclature to describe the strength agreement associated with In Table 5 it is shown that the highest accuracy of 95.98% to
the Kappa’s value is shown in Table 3. McHugh [23] suggested that classify stressful activities in undergraduate students is obtained
80% of data that are reliable when the value Kappa>0.60, in other with the KNN method using the Full dataset (with a feature vector
words, if a classifier has a Kappa value lower than 0.60, it means with 13 features and five signals). On the other hand, the lowest
that the confidence value is low, thus it is not recommended to accuracy of 63.31% is obtained by the LogR algorithm (with a fea-
use it. ture vector with four features and three signals).

3. Results 3.2. Identification of the features influenced by students in states of

3.1. Identification of the features influenced by the non-stress and
stress tasks
In order to analyze and identify the features that are influenced
by the stress and the non-stress tasks, a two-sample t-student test
was done (N = 21) using Minitab 17 software (Minitab Inc, USA).
For this analysis, only the data of the non-stress and the stress
segments of each participant were used (see Figs. 4 and 7), and
in total 21 physiological features were calculated from five physi-
ological signals. Before running the t-student test, the mean value
of each feature in each segment was calculated (the mean of all
windows for each segment), for example, the mean value of HR1
for the non-stress task segment was compared with the HR1 mean
value of the stress task segment. A confidence interval of 95% was
selected. Table 4 shows the results of the t-student test.
According to Table 4, the mean differences for 13 out of 21 fea-
tures were statistically significant (p < 0.05). As a consequence, a
feature vector of 13 characteristics was chosen to classify different
activities that cause stress in undergraduate students.
Four classifiers – Support Vector Machine (SVM), k-Nearest
Neighbors (KNN), Random Forest and Logistic Regression (LogR) –
were used to find a feature subset that provides the best accu-
racy to classify stress. Each of these algorithms was tested with
10-fold cross-validation. The classifiers were implemented using
Weka 3.8 software (The University of Waikato, New Zealand) with
the default parameter settings [44]. In this case, two predictive la-
anxiety
Since one of the main objectives of this study is to define pre-
dictive features to detect anxiety in students in states of stress, the
participants were divided into two groups according to the score
obtained from the STAI questionnaire. The Mexican version of this
anxiety test is called IDARE [38], and it reports a mean value of
37.68 for male and 38.25 for female in the trait-anxiety question-
naire, and 36.35 for male and 35.12 for female in the state-anxiety
questionnaire as measures of anxiety reaction in undergraduate
students. In order to separate the students into two groups, the
mean values from the state-anxiety questionnaire of IDARE were
used. In this way, group A (labeled as non-stress students, N = 8)
included students with scores lower than 36.35 for male and 35.12
for female; on the other hand, group B (labeled as stress stu-
dents, N = 13) included students with scores equal to or higher
than 36.35 for male and 35.12 for female. The comparison between
group A and B was tested using a t-student test. For this case,
the test was run using only the physiological data of the stress-
task segment and the feature vector of 13 features identified in
Section 3.1. The results are shown in Table 6.
According to Table 6, the mean differences for 6 out of 13 fea-
tures were statistically significant (p < 0.05). This means that three
out of five physiological signals could be used to identify anxiety
according to the score from the STAI questionnaire. Once again, the
SVM, KNN, Random Forest and LogR classifiers were used to find
a feature subset that provides the best accuracy to classify anx-

Table. 4
Features influenced by the stress and non-stress tasks (the shadow values are the selected features for further analysis).

Heart rate (HR) Oxygen saturation (SpO2) Skin temperature (ST) Breath rate (BR) Galvanic skin response (GSR)

Features p-value Features p-value Features p-value Features p-value Features p-value

HR1 0.001 SpO2-1 0.721 ST1 0.048 BR1 0.094 GSR1 0.001
HR2 0.001 SpO2-2 0.005 ST2 0.538 BR2 0.56 GSR2 0.001
HR3 0.851 SpO2-3 0.006 ST3 0.001 BR3 0.407 GSR3 0.001
—— —— —— —— —— —— BR4 0.001 —— ——
HR5 0.001 SpO2-5 0.005 ST5 0.539 BR5 0.562 GSR5 0.001
8 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408

Table. 5
∗
Subsets of features that provide better accuracy to classify stress activities in undergraduate students (note: the symbol near each accuracy value indicates that the
dataset is significantly different from the Full dataset, p < 0.05).

Name Full Dataset Dataset #2 Dataset #3 Dataset #4

Feature
selection
methods None CorrelationAttributeEval CfsSubsetEval InfoGainAttributeEval

HR1, HR2, HR5, SpO2-2, SpO2-3, SpO2-5,

Feature vector ST1, ST3, GSR1, GSR2, GSR3, GSR5, BR4 GSR1, GSR2, GSR5, ST3, BR4 HR1, HR2, SpO2-2, ST3 GSR1, GSR2, GSR5, ST3, HR1

Characteristics 5 signals 13 features 3 signals 5 features 3 signals 4 features 3 signals 5 features

Performance Accuracy (95% CI) Kappa Accuracy (95% Kappa Accuracy Kappa Accuracy (95% Kappa
CI) (95% CI) CI)
KNN (K = 1) 95.98% (±4.73) 0.92 88.90% 0.78 91.85% 0.84 88.23% 0.76
(±7.35)∗ (±6.64) (±7.70)∗
SVM 86.88% (±8.62) 0.74 84.46% (±8.41) 0.69 67.18% 0.33 84.72% (±8.61) 0.69
(±9.00)∗
LogR 87.59% (±8.29) 0.77 83.52% (±9.22) 0.67 63.31% 0.26 85.37% (±8.28) 0.70
(±9.30)∗
RandomForest 94.44% (±5.85) 0.89 90.53% (±7.44) 0.80 84.39% 0.69 87.68% 0.75
(±8.57)∗ (±7.84)∗

Table. 6 from three signals (in this way, the hardware requirements and the
Features that are in-
number of physiological features to analyze are fewer).
fluenced by anxiety in
students (the shadow For the case of the feature vector from Dataset #3, its use to
values are the selected predict stress using either the SVM or LogR classifiers is not rec-
features for further ommended because in both cases the Kappa value is below 0.40,
analysis). meaning that more than 65% of the results are erroneous.
Feature p < 0.05 Vitasari et al. [41] demonstrate that Mathematics causes stress
among some engineering students, causing their academic perfor-
HR1 0.001
HR2 0.546 mance to be lower. Based on the results of Dataset #3 in Table 5,
HR5 0.539 a feature vector of four characteristics of three physiological sig-
SpO2-2 0.001 nals (HR, SpO2 and ST) with the KNN classifier could be used to
SpO2-3 0.437
develop a platform to help teachers to identify the stress tasks in
SpO2-5 0.001
GSR1 0.002 class. Consequently, teachers can help students in handling their
GSR2 0.001 stress due to different kinds of academic tasks to increase the qual-
GSR3 0.387 ity of learning.
GSR5 0.001 On the other hand, to identify the anxiety in undergraduate stu-
ST1 0.442
dents in states of stress the SVM classifier provides the best accu-
ST3 0.574
BR4 0.762 racy using three signals (HR, SpO2 and GRS) and a feature vec-
tor of six characteristics (see Table 7). However, the paired t-test
shows that there is not a significant difference of accuracy between
iety. The predictive labels proposed were without-anxiety (group Dataset #4 and the Full dataset; in this way, anxiety could be iden-
A) and with-anxiety (group B). The default parameter settings in tified with an accuracy of more than 95% and an almost perfect
Weka for each classifier were used with 10-fold cross-validation. agreement (Kappa = 0.90) using only one signal (GSR) and three
Table 7 shows a summary of the results. physiological features.
As shown in Table 7, the maximum accuracy of 98.89% was For the case of the feature vectors from datasets #2 and #3,
achieved using six features and three signals with the SVM clas- although their accuracy values are greater than 70%, there is a sta-
sifier. The lowest accuracy (72.78%) was achieved with the same tistically significant difference (p > 0.05) with the feature vector
classifier using three features and two signals. from the Full Dataset, and their Kappa statistics indicate a moder-
ate to weak agreement (Kappa<0.61). As a result, the use of these
4. Discussion two datasets as features to predict anxiety is not recommended.
Vitasari et al. [41] demonstrate that there is a negative corre-
For the identification of features to classify the non-stress and lation between anxiety levels and academic performance among
stress tasks, according to the results in Table 5, the Full dataset undergraduate engineering students (high-level anxiety results in
provides the best accuracy (95.98%) using the KNN classifier and lower academic performance). Consequently, the results of this re-
a Kappa value of 0.92 (almost perfect agreement). Nevertheless, it search could be used to develop a new ambient intelligence frame-
is necessary to use a feature vector with 13 physiological compo- work to support students effectively while carrying out stress ac-
nents from five signals. In order to determine whether the stress tivities and help them to reduce the level of anxiety to improve
influenced by the non-stress and stress tasks in students could their academic performance. For example, the SVM or KNN clas-
be identified using a feature vector with fewer physiological fea- sifiers with the GSR signal (accuracy of more than 95% to classify
tures, a paired-test with 20 repetitions of 10-fold cross-validation anxiety could be obtained by using three features of the GSR sig-
and a significance value of 0.05 was run. The paired t-test con- nal; see Table 7) could be used to develop a platform to help stu-
firms that the difference in accuracy is not statistically significant dents to control anxiety levels when they are performing academic
(p < 0.05) between Full dataset and Dataset #3 (accuracy=91.85% activities and/or exams. In this way, when the platform detects
and Kappa=0.84) using the KNN classifier; therefore, stress can be anxiety it will trigger an alarm to indicate the student to relax,
identified using a feature vector with four physiological features which means the students can receive a recommendation when
 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408 9

Table. 7
∗
Subsets of features to identify anxiety according to the STAI score (note: the symbol near each accuracy value indicates that the dataset is significantly different from
the Full dataset, p < 0.05).

Name Full Dataset Dataset #2 Dataset #3 Dataset #4

Feature
selection
method None CorrelationAttributeEval CfsSubsetEval InfoGainAttributeEval

HR1, SpO2-2, SpO2-5, GSR1,

Feature vector GSR2, GSR5 SpO2-2, SpO2-5, GSR2, GSR5 SpO2-2, GSR1, GSR2 GSR1, GSR2, GSR5

Characteristics 3 signals 6 features 2 signals 4 features 2 signals 3 features 1 signal 3 features

Accuracy (95% Accuracy (95%

Performance CI) Kappa CI) Kappa Accuracy (95% CI) Kappa Accuracy (95% CI) Kappa

KNN (K = 1) 95.56% (±6.32) 0.91 85.00% 0.65 82.56% (±13.14)∗ 0.60 95.22% (±7.45) 0.89
(±11.86)∗
SVM 98.89% (±3.35) 0.97 74.56% 0.34 72.78% (±12.27)∗ 0.33 95.56% (±6.51) 0.90
(±12.16)∗
∗
LogR 95.22% (±7.28) 0.89 75.11% 0.41 74.33% (±14.96) 0.43 87.56% (±6.23) 0.84
(±12.25)∗
RandomForest 88.22% 0.74 76.00% 0.38 74.22% (±13.01)∗ 0.35 83.78% (±7.79) 0.85
(±10.09) (±10.32)∗

they experience anxiety, so they could continue with the learning
activity to increase their learning engagement. Finally, Zhu, Ober,
and Jafari [45] propose the use of physiological signals to assist
professors with the assessment of the students’ attention levels.
Their results show an accuracy of more than 98% for predictions
of interest level in a reading activity. For future work, the authors
propose to run new experiments in order to evaluate and identify
the main physiological features that could be used as predictors
of students’ attention levels using another kind of academic activ-
ity (i.e. mathematical computations, solving engineering problems,
answering a test, etc.).
5. Conclusions
Selection - SFS) to compare with the statistical methods used in
this study to select the most relevant features.
Compliance with ethical standards
Ethical Approval: All procedures performed in studies involving
human participants were in accordance with the ethical standards
of the institutional and/or national research committee and with
the 1964 Declaration of Helsinki and its subsequent amendments
or comparable ethical standards.
Informed Consent: Informed consent was obtained from all indi-
vidual participants included in the study.

Academic stress is a cause of poor academic performance in un-
dergraduate students and results in dropout from university. Self-
reports are a tool to identify students prone to anxiety. However,
the interpretation of the test requires an expert and the diagnosis
depends on the veracity of the answers and perception of the sub-
jects. In this study, a stress-inducing protocol was proposed, and 21
features of five signals were analyzed from time-series physiolog-
ical data from 21 undergraduate students. In addition, the anxiety
due to an arithmetic task was corroborated through the application
of the STAI questionnaire for each student.
In order to identify stress due to activities performed by the
student, accuracy in classification greater than 90% and a Kappa
value > 0.80 were achieved by using the KNN classifier and only
three sensors (HR, SpO2 and ST) with four physiological features.
Meanwhile, the classification of anxiety with an accuracy of more
than 95% and Kappa = 0.90 can be done using only three char-
acteristics from the GSR signal and the SVM classifier. The main
advantage of the use of the GSR signal as a predictor of anxiety is
to avoid noise or artifacts due to subjects’ body part movements
(other signals could be affected by this kind of artifacts, i.e. EMG,
ECG, HR [12]). Thus, the results provide a clue that anxiety detec-
tion could be done using the analysis of the physiological signals
instead of the STAI test score.
One limitation of this study could be the experimental proce-
dure because only one type of stress task was tested. Nevertheless,
this research proposes to use another kind of academic task, to run
future experiments, and to evaluate the level of anxiety according
to the complexity of the task and the academic performance of the
students. In addition, future work derived from this study includes
the use of automatic feature selection methods (e.g. Minimum Re-
dundancy Maximum Relevance - mRMR and Sequential Forward
Declaration of Competing Interest
The authors declare no potential conflicts of interest with re-
spect to the research, authorship, and/or publication of this article.
Acknowledgments
The authors would like to thank the Autonomous University of
the State of Mexico (UAEM) for the financial support of this project
(4313/2017/CI). Liliana Lara-Flores acknowledges the Mexican Na-
tional Council for Science and Technology (CONACYT) for the schol-
arship CONACYT CVU 786012 to carry out this research.
References
[1] J. Aigrain, M. Spodenkiewicz, S. Dubuisson, M. Detyniecki, D. Cohen,
M. Chetouani, Multimodal stress detection from multiple assessments, IEEE
Trans. Affective Comput. (2016), doi:10.1109/taffc.2016.2631594.
[2] A Alberdi, A Aztiria, A. Basarab, Towards an automatic early stress recognition
system for office environments based on multimodal measurements: a review,
J. Biomed. Inform. 59 (2016) 49–75, doi:10.1016/j.jbi.2015.11.007.
[3] S.F. Barrett, Arduino, Microcontroller processing for everyone!, Synth. Lect.
Digit. Circuits Syst. 8 (4) (2013), doi:10.220 0/s0 0522ed1v01y201307dcs043.
[4] R Beiter, R Nash, M McCrady, D Rhoades, M Linscomb, M. Clarahan, S Sammut,
The prevalence and correlates of depression, anxiety, and stress in a sample of
college students, J. Affect. Disord. 173 (2015) 90–96, doi:10.1016/j.jad.2014.10.
054.
[5] N Berrío García, R Mazo Zea, Estrés académico, Revista de Psicología Universi-
dad de Antioquia 3 (2) (2011).
[6] Y Deng, Z Wu, C.-.H. Chu, Q Zhang, D.F Hsu, Sensor feature selection and com-
bination for stress identification using combinatorial fusion, Int. J. Adv. Rob.
Syst. 10 (8) (2013) 306, doi:10.5772/56344.
[7] F.G DordiNejad, H Hakimi, M Ashouri, M Dehghani, Z Zeinali, M.S Daghighi,
N Bahrami, On the relationship between test anxiety and academic perfor-
mance, Proc. Soc. Behav. Sci. 15 (2011) 3774–3778, doi:10.1016/j.sbspro.2011.
04.372.
10 J. Rodríguez-Arce, L. Lara-Flores and O. Portillo-Rodríguez et al. / Computer Methods and Programs in Biomedicine 190 (2020) 105408
[8] B Egilmez, E Poyraz, W Zhou, G Memik, P Dinda, N Alshurafa, UStress: under-
standing college student subjective stress using wrist-based passive sensing,
in: 2017 IEEE International Conference On Pervasive Computing and Commu-
nications Workshops, IEEE, 2017, doi:10.1109/percomw.2017.7917644.
[9] R.M. Fish, L.A. Geddes, Conduction of electrical current to and through the hu-
man body: a review, Open Access J. Plastic Surg. 9 (2009) 1–24.
[10] R García-Ros, F Pérez-González, J Pérez-Blasco, L.A Natividad, Academic stress
in first-year college students, Revista Latinoamericana de Psicología 44 (2)
(2012) 143–154.
[11] A Ghaderi, J Frounchi, A Farnam, Machine learning-based signal processing us-
ing physiological signals for stress detection, in: 2015 22nd Iranian Confer-
ence On Biomedical Engineering {ICBME}, IEEE, 2015, doi:10.1109/icbme.2015.
7404123.
[12] G. Giannakakis, D Grigoriadis, K Giannakaki, O Simantiraki, A Roniotis, M Tsik-
nakis, Review on psychological stress detection using biosignals, IEEE Trans.
Affective Comput. 1 (2019), doi:10.1109/taffc.2019.2927337.
[13] M.A Hall, Correlation-based Feature Subset Selection for Machine Learning,
University of Waikato, Hamilton, New Zealand, 1998.
[14] J.A Healey, R.W Picard, Detecting stress during real-world driving tasks using
physiological sensors, IEEE Trans. Intell. Transp. Syst. 6 (2) (2005) 156–166,
doi:10.1109/tits.2005.848368.
[15] T Iliou, G Konstantopoulou, M Ntekouli, C Lymperopoulou, K Assimakopou-
los, D Galiatsatos, G Anastassopoulos, ILIOU machine learning preprocess-
ing method for depression type prediction, Evol. Syst. (2017), doi:10.1007/
s12530- 017- 9205- 9.
[16] H. Jing, Analysis on the relationship among test anxiety, self-concept and aca-
demic competency, J. US-China Foreign Lang. 5 (1) (2007) 48–51.
[17] E Kaniusas, E. Kaniusas (Ed.), Springer-Verlag, 2012, doi:10.1007/
978- 3- 642- 24843- 6 1.
[18] P Karthikeyan, M Murugappan, S Yaacob, Analysis of stroop color word
test-based human stress detection using electrocardiography and heart rate
variability signals, Arab. J. Sci. Eng. 39 (3) (2013) 1835–1847, doi:10.1007/
s13369-013-0786-8.
[19] C Kirschbaum, K.-M Pirke, D.H Hellhammer, The trier social stress test a tool
for investigating psychobiological stress responses in a laboratory setting, Neu-
ropsychobiology 28 (1–2) (1993) 76–81, doi:10.1159/0 0 01190 04.
[20] A Kitsantas, A Winsler, F Huie, Self-Regulation and ability predictors of aca-
demic success during college: a predictive validity study, J. Adv. Acad. 20 (1)
(2008) 42–68, doi:10.4219/jaa- 2008- 867.
[21] J.R. Landis, G.G Koch, The measurement of observer agreement for categorical
data, Biometrics 33 (1) (1977) 159, doi:10.2307/2529310.
[22] C Maaoui, A Pruski, Unsupervised stress detection from remote physiological
signal, in: 2018 IEEE International Conference on Industrial Technology, IEEE,
2018, doi:10.1109/icit.2018.8352409.
[23] M.L McHugh, Interrater reliability: the kappa statistic, Biochemia Medica
(2012) 276–282, doi:10.11613/bm.2012.031.
[24] L Moya-Albiol, A Salvador, Empleo de estresores psicológicos de laboratorio en
el estudio de la respuesta psicofisiológica al estrés, Anales de Psicología 17 (1)
(2001) 69–81.
[25] F.N Ogba, M.O Ede, C.N Onyishi, P.U Agu, A.B Ikechukwu-Ilomuanya, J.N Igbo,
. . . S.C Ugwoke, Effectiveness of music therapy with relaxation technique on
stress management as measured by perceived stress scale, Medicine 98 (15)
(2019) e15107, doi:10.1097/md.0 0 0 0 0 0 0 0 0 0 015107.
[26] E.U Onyeizugbo, Self-Efficacy and test anxiety as correlates of academic per-
formance, J. Educ. Res. (2010).
[27] O Peleg, Test anxiety, academic achievement, and self-esteem among Arab ado-
lescents with and without learning disabilities, Learn. Disabil. Q. 32 (1) (2009)
11–20, doi:10.2307/25474659.
[28] K Plarre, A Raij, S.M Hossain, A.A Ali, M Nakajima, M Al’absi, . . . L.E Wittmers,
Continuous inference of psychological stress from sensory measurements col-
lected in the natural environment, in: Proceedings of the 10th ACM/IEEE In-
ternational Conference on Information Processing in Sensor Networks, 2011,
pp. 97–108.
[29] M.B.I Reaz, M.S Hussain, F Mohd-Yasin, Techniques of {EMG} signal analysis:
detection, processing, classification and applications, Biol. Proc. Online 8 (1)
(2006) 11–35, doi:10.1251/bpo115.
[30] K.J. Reddy, K.R. Menon, A. Thattil, Academic stress and its sources among uni-
versity students, Biomed. Pharmacol. J. 11 (1) (2018) .10.13005/bpj/1404.
[31] J.L. Rhudy, M.W. Meagher, Fear and anxiety: divergent effects on human pain
thresholds, Pain 84 (1) (20 0 0) 65–75, doi:10.1016/s0304-3959(99)00183-9.
[32] M. Salai, I. Vassányi, I. Kósa, Stress detection using low cost heart rate sensors,
J. Healthcare Eng. (2016) 1–13, doi:10.1155/2016/5136705.
[33] V. Sandulescu, S. Andrews, D. Ellis, N. Bellotto, O.M. Mozos, Stress detection
using wearable physiological sensors, in: Artificial Computation in Biology and
Medicine, Springer International Publishing, 2015, pp. 526–532, doi:10.1007/
978- 3- 319- 18914- 7_55.
[34] M. Sevil, I. Hajizadeh, S. Samadi, J. Feng, C. Lazaro, N. Frantz, . . . A. Cinar, Social
and competition stress detection with wristband physiological signals, in: 2017
IEEE14th International Conference On Wearable and Implantable Body Sensor
Network, IEEE, 2017, doi:10.1109/bsn.2017.7936002.
[35] N Sharma, T. Gedeon, Objective measures, sensors and computational tech-
niques for stress recognition and classification: a survey, Comput. Methods
Programs Biomed. 108 (3) (2012) 1287–1301, doi:10.1016/j.cmpb.2012.07.003.
[36] K. Soman, V. Alex, C Srinivas, Analysis of physiological signals in response to
stress using ecg and respiratory signals of automobile drivers, in: 2013 Interna-
tional Mutli-Conference on Automation, Computing, Communication, Control
and Compressed Sensing, IEEE, 2013, doi:10.1109/imac4s.2013.6526476.
[37] G. Spangler, R. Pekrun, K. Kramer, H. Hofmann, Students emotions, physiolog-
ical reactions, and coping in academic exams, Anxiety Stress Coping 15 (4)
(2002) 413–432, doi:10.1080/1061580021000056555.
[38] C.D. Spielberger, F. Gonzalez-Reigosa, A. Martinez-Urrutia, L. Natalicio, D. Na-
talicio, Development of the spanish edition of the state-trait anxiety inventory,
Int. J. Psychol. (1971).
[39] R.L. Spitzer, The structured clinical interview for DSM-III-R (SCID), Arch. Gen.
Psychiatry 49 (8) (1992) 624, doi:10.1001/archpsyc.1992.01820 080 0320 05.
[40] E. Vildjiounaite, J. Kallio, J. Mäntyjärvi, V. Kyllönen, M. Lindholm, G. Gimel’farb,
Unsupervised stress detection algorithm and experiments with real life data,
in: Progress in Artificial Intelligence, Springer International Publishing, 2017,
pp. 95–107, doi:10.1007/978- 3- 319- 65340- 2_9.
[41] P. Vitasari, T. Herawan, M.N.A. Wahab, A. Othman, S.K Sinnadurai, Explor-
ing mathematics anxiety among engineering students, Proc. Soc. Behav. Sci. 8
(2010) 482–489, doi:10.1016/j.sbspro.2010.12.066.
[42] P. Vitasari, M.N.A. Wahab, A. Othman, T. Herawan, S.K. Sinnadurai, The rela-
tionship between study anxiety and academic performance among engineer-
ing students, Proc. Soc. Behav. Sci. 8 (2010) 490–497, doi:10.1016/j.sbspro.2010.
12.067.
[43] J Wijsman, B. Grundlehner, H. Liu, H. Hermens, J. Penders, Towards mental
stress detection using wearable physiological sensors, in: 2011 Annual Interna-
tional Conference of the {IEEE} Engineering in Medicine and Biology Society,
IEEE, 2011, doi:10.1109/iembs.2011.6090512.
[44] I.H. Witten, E. Frank, M.A. Hall, Data Mining: Practical Machine Learning Tools
and Techniques, Morgan Kaufmann, 2011.
[45] Z. Zhu, S. Ober, R. Jafari,, Modeling and detecting student attention and interest
level using wearable computers, in: 2017 IEEE14th International Conference On
Wearable and Implantable Body Sensor Networks, IEEE, 2017, doi:10.1109/bsn.
2017.7935996.