1

MAJOR PROJECT REPORT

“Human Emotion Detection using Text and Questionnaire Recognition”

DEPARTMENT OF COMPUTER SCIENCE &

ENGINEERING
NATIONAL INSTITUTE OF TECHNOLOGY PATNA
(JANUARY-JUNE 2019)
__________________________

Submitted by:

SURAJ KUMAR (1506063)

In the shadow of
Dr. Ditipriya Sinha
(Assistant Professor)
Computer Science & Engineering Department,
National Institute of Technology Patna, Patna
Major Project Report P.T.O
 2

DECLARATION

I hereby declare that this project work entitled “ HUMAN EMOTION DETECTION
BASED ON TEXT AND QUESTIONNAIRE RECOGNITION” has been carried out by me in
the Department of Computer Science of Engineering in National Institute of
Technology Patna, under the guidance of Dr. Ditipriya Sinha, Assistant Professor,
Department of Computer Science of Engineering, National Institute of Technology
Patna. No part of this work had been submitted for the award of degree or diploma
to this institute.

_______________________________

SURAJ KUMAR (1506063)

Major Project Report P.T.O

 3

राष्ट्रीय प्रौद्योगिकी सस्ं थान, पटना

NATIONAL INSTITUTE OF TECHNOLOGY PATNA
(An institute under Ministry of HRD, Govt. Of India)
ASHOK RAJPATH, PATNA , BIHAR – 800 006
Patna, Bihar
06th May, 2019

CERTIFICATE

This is to certify that the project entitled “HUMAN EMOTION DETECTION BASED
ON TEXT AND QUESTIONNAIRE RECOGNITION” is a bonafide work carried
out by SURAJ KUMAR(1506063) at NATIONAL INTITUTE OF TECHNOLOGY PATNA
under my guidance and supervision in 8th semester of Bachelor of Technology
from January 2019 to May 2019.

________________ __________________

Dr. Ditipriya Sinha Dr. Prabhat Kumar

Assistant Professor Head of Department
Computer Science and Computer Science and
Engineering Department Engineering Department
NIT PATNA NIT PATNA
Major Project Report P.T.O
 4

Major Project Report P.T.O

 5

ACKNOWLEDGEMENT
I express my deepest gratitude towards my mentor at NIT Patna, Dr.Ditipriya Sinha for
her valuable suggestions, insightful criticisms and sections throughout the course of my
project. She always encouraged us get out of my shell and do something innovative
despite my limitations.
I also take this opportunity and thank each one of my professors under whom I
have had the privilege of pursuing our education. They are the reason I was technically
capable enough to indulge in such a endeavor.
I am also thankful to my immediate seniors for their special contribution in the
field of ‘Emotion Detection’, which I have carried forward to improve the system of
emotion detection. I am also thankful to all the non-teaching staffs of the department of
Computer Science and Engineering ,NIT Patna for their support.
Finally, I thank my friends and family members for being constant source of
encouragement, support and valuable suggestions during course of my project.

Major Project Report P.T.O

 6

ABSTRACT

As part of my final year major project I was required to work on a domain that was
both challenging and useful. After brain-storming through a plethora of ideas, I decided
to work on a ‘Human Emotion Detection’, an idea which has been the think-tank of
scientists for quite some time now.
The first step was to understand how human emotions works. For this, I had to
dive deep into psychology in human beings. In this sequence, I tried to know emotions
using text and their behavior which tested using questionnaire.
As part of the research, I finally concluded that human emotions are mostly
related to the human brain, the human brain which is considered to be the most complex
organ in the universe. I finally proceeded with a culmination of questionnaire and text
recognition approach to deduce human emotion.

Major Project Report P.T.O

 7

CONTENTS

1. Introduction 08
2. Plutchik’s Wheel of Emotion
2.1 Theory of Emotion 09
3. Literature survey 12
4. Scope of Work 14
5. Data Collection
5.1 Questionnaire Approach 15
5.2 Text Analysis 16
6. Proposed Approach
6.1 Text analysis Based Approach
6.1.1 Pre-processing 18
6.1.2 Data Set 18
6.1.3 Feature extraction 19
6.2 Questionnaire based Approach
6.2.1 Data Set Development 21
6.2.2 Feature Extraction 22
7. Classification
7.1 SVM ( Support Vector Machine ) 24
7.2 NN (Neural Network) 25
7.3 Training 27
7.4 Testing and Decision Making 27

Major Project Report P.T.O

 8

8. Proposed Combination of Two Approaches

8.1 Generation of feature vector by combining both
approaches 28
8.2 Combining the probability scores
of two classifiers 28
9. Result Analysis 31
10. Conclusion 34
11. References 35

Major Project Report P.T.O

 9

1. INTRODUCTION

Human emotions play a vital role in our lives. Emotion can be broadly defined as
“instinctive or intuitive feeling as distinguished from reasoning or knowledge”. They
affect an individual’s ability to reason various situations and also govern their reaction to
stimuli.
Emotion recognition finds its application in many areas such as medicine, law, marketing,
e-learning etc. Emotion recognition has for some time been a relatively untended area
but has gained traction in the recent past.
This is related to the premise that computers are logical machines that make
decisions based on logic and facts.
With the extensive researches in the fields such as Artificial Intelligence and
machine Learning, many works are being proposed to detect human emotion. Various
approaches have already been proposed for emotion recognition.
These include the analysis of brain signals, facial recognitions, etc.

The use of psychological features such as heart-rate, pupil dilation and skin
conductance has also been incorporated. Most approaches use the concepts of blogs and
social network to collect the data which might be infeasible for the illiterate people.
This paper analyzed a culmination of questionnaire and speech recognition to
determine the emotional state of a person.

The combination of questionnaire and speech recognition-based approaches has been

done using Dempster-Shafer theory. This whole process of collection along with analysis
was termed as emotion recognition.

Major Project Report P.T.O

 10

2. PLUTCHIK’S WHEEL OF EMOTIONS

2.1 THEORY OF EMOTIONS

Plutchik’s three-dimensional circumplex model describes the relations among emotion

concepts, in a similar way to the colours on a colour wheel. The cone’s vertical dimension
represents intensity – emotions intensify as they move from the outside to the center of
the wheel. As an example, a feeling of boredom can intensify to loathing if left
unchecked. This is an important rule about emotions to be aware of in relationships in
the workplace, with family and with friends.

The eight sectors are designed to indicate that there are eight primary emotion
dimensions. They are sadness, surprise, fear, anger, etc.

Plutchik's (1980) psycho evolutionary theory of basic emotions has ten postulates.

1. The concept of emotion is applicable to all evolutionary levels and applies to

animals as well as to humans.
2. Emotions have an evolutionary history and have evolved various forms of
expression in different species.
3. Emotions served an adaptive role in helping organisms deal with key survival
issues posed by the environment.
4. Despite different forms of expression of emotions in different species, there are
certain common elements, or prototype patterns, that can be identified.
5. There is a small number of basic, primary, or prototype emotions.
6. All other emotions are mixed or derivative states; that is, they occur as
combinations, mixtures, or compounds of the primary emotions.
7. Primary emotions are hypothetical constructs or idealized states whose properties
and characteristics can only be inferred from various kinds of evidence.
8. Primary emotions can be conceptualized in terms of pairs of polar opposites.
9. All emotions vary in their degree of similarity to one another.
10. Each emotion can exist in varying degrees of intensity or levels of arousal.

Major Project Report P.T.O

 11

Fig 1. Plutchik’s Wheel of Emotions

Basic emotion Basic opposite

Joy Sadness

Trust Disgust

Fear Anger

Surprise Anticipation

Major Project Report P.T.O

 12

Human feelings (results of Feelings Opposite

emotions)

Optimism Anticipation + Joy Disappointment

Love Joy + Trust Remorse

Submission Trust + Fear Contempt

Awe Fear + Surprise Aggression

Disappointment Surprise + Sadness Optimism

Remorse Sadness + Disgust Love

Contempt Disgust + Anger Submission

Fig 2. The result of Emotions

Major Project Report P.T.O

 13

3. LITERATURE SURVEY
Human emotion detection has gained an increasing attention in recent years due to the
wide variety of applications that benefit from such technology. Owing to this various
researches have been conducted so far using different mechanisms to detect emotions.

Most of the researches performed till now for detecting emotion mainly involve use of
facial features, acoustic data, physiological signals and lexical analysis. Bhatti et Al
presented an approach to language independent emotion recognition through speech
by extracting prosodic features from each utterance for the computational mapping
between emotions and speech patterns. Another similar work performed at HP
Laboratories for IVR applications focus on using features extracted from pitch contour,
energy contour and audible segment duration contours for distinguishing certain
emotions.

Liu et a worked on detection of inner emotions from electroencephalogram (EEG)

signals. Similar to this, Lisetti et al also conducted an experiment to elicit emotion and
capture physiological data which included signals from autonomous nervous system
arousal such as skin temperature, heart rate and Galvanic Skin Response (GSR).
Magglogianis et al stated the fact that emotions are most widely expressed through eye
and mouth expressions and hence emotion recognition was done through face detection
using Markov random fields.

Domains specific emotions were also detected using language and discourse information
in conjunction with acoustic correlates of emotion in speech signal. This work specifically
targeted call center application and detected negative and non-negative emotions using
spoken language data.

For building a social emotion detection system targeting online readers and writers many
approaches were suggested. The SWAT system exploits the sentiments of individual
words. It uses a supervised approach to develop a word emotion mapping dictionary
which is used to determine the emotion of unlabeled news headlines. Emotion topic
model proposed by Bao studies the sentiments of particular topics.

Major Project Report P.T.O

 14

Here a “topic” represents an object , abstract entity of real world event which is
representative of the subject or context of sentiments of particular topic. This model is
only suitable for annotating emotion from writers’ perspective and not from readers’
perspective. As compared to classical analysis conducted from the writers’ perspective,
analysis from readers’ perspective can be more meaningful. So , Yanghui et al developed
an effective topic model (ATM) to detect the emotion towards certain topic from readers’
perspective by introducing an intermediate layer.

Lie believed that the system performed better with word and part of speech combination
(POS). So the system was deployed using the document selection components , part of
speech tagging and social emotion lexicon generation . Later in 2015 , a rule based
approach to emotion causing component detection for Chinese micro blog was
proposed.

Major Project Report P.T.O

 15

4. SCOPE OF WORK

State of the art describes that the approaches designed till now for emotion detection
mainly involves either active internet users who are exposed to services such as social
networking , micro blogs , newsletters etc or complex equipments such as a high
definition camera to capture facial expressions , EEG machines to capture brain signals
and sensor enabled devices to capture physiological signals, which are highly expensive.
Such approaches can be used for a specific set of people especially the literate class. In
our system, we propose a questionnaire and speech recognition based human emotion
detection approach where data can be collected from every class of people.

Previous Work :

The research started with four emotions which were : Anger, Fear ,Happy, Sad and
Disgust. They have used speech-based approach to test the human emotion using
Support Vector Machine (SVM) classifier.

Major Project Report P.T.O

 16

5. DATA COLLECTION

5.1 Questionnaire Approach

The training data was collected via the Google cloud service I.e. Google Forms. The link
was shared and a comprehensive set of people with a mix bag of males/females, of
different age groups and of different professions were asked to fill the questionnaire.

Major Project Report P.T.O

 17

Fig 3 . Screenshots of Google Form

The responses of the emotional questionnaire were collected from around 109 persons
of varying age groups and occupations. The data collected was in the form of tabulated
sheets which was further converted to tab separated sheet (.csv) format so that data
could be utilized for mapping inti their corresponding mathematical value.

5.2 Text Analysis

The approach is to collect the text samples from different classes of people (viz. Different
age groups, gender) in all six moods that are under study . In this paper, a basic text in
that corresponding period was chosen. This sentence doesn’t convey any emotion until
written.

All the text are collected and converted to a text sentence corresponding of each
emotion of an individual.
Major Project Report P.T.O
 18

6. PROPOSED APPROACH

The paper aims at detecting emotion by conglomerating two approaches : Text analysis
wherein the acoustic features of the subject’s speech are analyzed and Questionnaire
approach where the subject is asked to fill a particular questionnaire . The Proposed
framework is shown in figure given below :

Fig 4. Framework of the Process

Detailed analysis of the above approaches is discussed in the following sections.

Major Project Report P.T.O

 19

6.1 Text Analysis Based Approach

6.1.1 Pre-processing:
For both training and testing datasets of text, first of all, we need to carry out some pre-
processing tasks on the collected data. Initially, each sample is converted to lowercase
and the characters other than a-z are removed from the sentences as they do not
influence the emotion or state of a person (”well it does”). Then, we have eliminated the
’stop-word’ (e.g. ’the’, ’that’ and lexical words) from the sample and at last checked the
negation in the sentence. If the word ’not’ comes with a verb, adjective or adverb then it
has been combined with the word for further consideration, otherwise the negation is
removed as again it is not going to influence the sentence for the emotion.

6.1.2. Data Set

The data set is taken from one globally ISEAR model :

Fig 5 . Snapshot of unprocessed raw data

Major Project Report P.T.O

 20

Now, after Pre-processing , the data looks something like the below given snap :

Fig 6 . Snapshot of Pre-processed data

6.1.3 Feature extraction:

The basic emotions are the only key features as the text consists of the basic emotions
whose values will be the probabilities of the emotion state in the sentence. These basic
emotions are Anger, Sad, Joy, Disgust and Fear.
In this approach, the text is converted into tokens applying Natural Language Toolkit
(NLTK) word tokenizer. Then, each token is added with the POS tag using the NLTK POS
tagger. Among the tags, only adjective, verb or adverb tokens are selected as these are
directly related to the emotion of human being. Now, using the National Research
Council (NRC), Canada word-emotion lexicon, we find the token(s)corresponding to each
emotion. Finally, we calculate the probability of each emotion using the following
equation:

C(i) = (Ni)* 𝛿/(N)

Major Project Report P.T.O
 21

where,
N(i) = Number of tokens related to (i)th emotion in each blog,
N = Total number of tokens in the blog, i = 1,...,5 and ,
𝛿 = any prime number scalar quantity to smooth the result.

Fig 7. Snaps of Feature value after feature extraction

Major Project Report P.T.O

 22

6.2 Questionnaire Based Approach

6.2.1 Data set Development

The dataset required for this approach has been created by us. Data have been collected
via the Google cloud service i.e. Google forms. The link was shared and a comprehensive
set of people with a mix bag of males/females, of different age groups and of different
professions were asked to fill the questionnaire.
The responses of the emotional questionnaire have been collected from about 100
different persons of varying age groups and occupations. The data collected are in the
form of tabulated sheet which is further converted to tab separated sheet (.csv) format
so that data could be utilized for mapping into their corresponding mathematical value.
Figure shows a snapshot of data collected in questionnaire approach.

Fig 8. Snaps of Collected questionnaire based dataset

In this figure, each row corresponds to the input from a person and each column defines
a particular feature e.g.-sleep, heart rate etc. It can also be noted from this figure that
for a particular feature there can be more than one emotions related to it. In order to
perform training these emotion strings were mapped to some numerical value. These
samples have been used only to train the system. For generating the testing dataset, 100
different persons from various backgrounds contributed their feedback against a similar
Major Project Report P.T.O
 23

form consisting of a set of questions and answers with options of ’yes’, ’no’ or ’cannot
say’. Here, similarly like training dataset the feedback of each person in response to the
questionnaire contributes one feature vector for each emotion class. Thus, we have
collected a total of 109 samples to test our proposed system.

6.2.2 Feature Extraction

There are several features/characteristics that correspond to a particular emotion. Upon

research and proper consultation with a practicing psychiatrist a set of most suitable
features has been created corresponding to each emotion. Thus, 30 most suitable
features came upon covering aforesaid five emotions. Those 30 features are shown in
below table :

Fig 9. Snaps of Feature values of each feature

Major Project Report P.T.O

 24

Mapping the emotion string to numerical values is done on the basis of data collection
format. Each of the features starting from heart rate to the last feature in the row is
assigned feature value from 1 to 30 based on their order of occurrence in the data
collection format. For example, ’Increase in heart rate’ is assigned the value of 1.

Each emotion string is assigned any class label between 1 and 5 depending on the
emotion class in which it belongs. The class label is assigned as follows: Anger-1, Sad-2,
Joy-3, Disgust-4 and Fear-5.

The following equation has been used to deduce the numerical value for each particular
entry in the cell of tabulated sheet that corresponds to that particular feature. The
equation is:

C(i) = ( Class Value )/ ( Feature value * Hash Value )

Where,
1.1.1. class value corresponds to that particular emotion,
1.1.2. feature value is the feature number of that particular feature,
1.1.3. hash value is any prime number which is used in all the
calculations to get a distinct numeric value for each value.
In this mapping criteria, hash value of 19 was taken as it gave the best result. The
mapping function was used to generate the training data so that the support vector
machine gets properly trained. For better results, these values were kept between –1 to
1.

Major Project Report P.T.O

 25

7. CLASSIFICATION

The Human Emotion Detection process is carried out using Support Vector Machine
(SVM).

7.1 SVM (Support Vector Machine)

In machine learning, SVMs are supervised learning models with associated learning
algorithms that analyze data and recognize patterns, used for classification and
regression analysis.
Given a set of training examples, each marked for belonging to one of two
categories, an SVM training algorithm builds a model that assigns new examples into one
category or the other, making it a non-probabilistic binary linear classifier.
An SVM model is a representation of the examples as points in space, mapped so
that the examples of the separate categories are divided by a clear gap that is as wide as
possible. New examples are then mapped into that same space and predicted to belong
to a category based on which side of the gap they fall on. SVM has been used successfully
for pattern recognition and regression tasks [33, 34, 35] .
SVM was originally defined for the problems of two-classes where it finds the optimal
hyper-planes that maximize the margin between the positive and negative data sets of
these classes.
This hyper-plane is characterized by the normal vector, which is expressed as linear
combination of the nearest examples of both classes, named support vectors.
In order to extend SVM to solve multi-class pattern recognition problem, kernels
technique is used. More formally, a support vector machine constructs a hyper-plane or
set of hyper-planes in a high or infinite-dimensional space, which can be used for
classification, regression, or other tasks.
Intuitively, a good separation is achieved by the hyper-plane that has the largest distance
to the nearest training data point of any class (so-called functional margin), since in
general the larger the margin the lower the generalization error of the classifier.

Major Project Report P.T.O

 26

Suppose, TD is a training dataset consists of pairs (xi, yi), i=1, 2,.., n, xi ∈ Rn and yi ∈ (-
1,1), where xi denotes input feature vector for ith sample and yi denotes the
corresponding target value. For a given input pattern x, the decision function of an SVM
binary classifier is

b is the bias, αi is the large range multiplier and K(x, xi) is the kernel function.
The input feature vector x is mapped into higher dimensional feature space using the
kernel function to make them linearly separable. Several kernel functions are used in
SVM. Some of those kernel functions are Gaussian (Radial Basis Function) kernel,
Polynomial kernel, Linear kernel etc. Studies have shown that RBF networks designed
through support vector (SV) method can produce better recognition performances
compared to those designed with traditional methodology for the same data set.

7.2 Neural Network

Neural Networks are a computational approach which is based on a large collection of

neural units loosely modeling the way a biological brain solves problems with large
clusters of biological neurons connected by axons. Each neural unit is connected with
many others, and links can be enforcing or inhibitory in their effect on the activation state
of connected neural units. Each individual neural unit may have a summation function
which combines the values of all its inputs together. There may be a threshold function
or limiting function on each connection and on the unit itself such that it must surpass it
before it can propagate to other neurons. These systems are self-learning and trained
rather than explicitly programmed and excel in areas where the solution or feature
detection is difficult to express in a traditional computer program.
Neural networks typically consist of multiple layers or a cube design, and the signal path
traverses from front to back. Back propagation is where the forward stimulation is used

Major Project Report P.T.O

 27

to reset weights on the ”front” neural units and this is sometimes done in combination
with training where the correct result is known.

More modern networks are a free flowing in terms of stimulation and inhibition with
connections interacting in a much more chaotic and complex fashion. Dynamic neural
networks are the most advanced in that they dynamically can, based on rules, form new
connections and even new neural units while disabling others. The goal of the neural
network is to solve problems in the same way that the human brain would, although
several neural networks are much more abstract. Modern neural network projects
typically work with a few thousand to a few million neural units and millions of
connections, which is still several orders of magnitude less complex than the human
brain and closer to the computing power of a worm. New brain research often stimulates
new patterns in neural networks. One new approach is using connections which span
much further and link processing layers rather than always being localized to adjacent
neurons.
Other research being explored with the different types of signal over time that axons
propagate which is more complex than simply on or off. Neural networks are based on
real numbers, with the value of the core and of the axon typically being a representation
between 0.0 and 1.
An interesting facet of these systems is that they are unpredictable in their success with
self learning. After training some become great problem solvers and others don’t
perform as well. In order to train them several thousand cycles of interaction typically
occur. Like other machine learning methods systems that learn from data neural
networks have been used to solve a wide variety of tasks, like computer vision and speech
recognition, that are hard to solve using ordinary rule-based programming. Historically,
the use of neural network models marked a directional shift in the late eighties from high-
level (symbolic) artificial intelligence, characterized by expert systems with knowledge
embodied in if-then rules, to low-level (sub-symbolic) machine learning, characterized by
knowledge embodied in the parameters of a dynamical system.

Major Project Report P.T.O

 28

7.3 Training
In order to perform training , the training data of questionnaire approach is converted to
the Support Vector Machine training format.
For text samples, the collected text samples from questionnaire were further
converted in form of feature vector, As stated above, the collected samples contained
answers of the questionnaire. The total length of text sample answers was 30 . So, we
get the feature vector size of 31 that includes 30 from questionnaire and 1 feature (i.e.
6 emotion + 1 neutral ) from text.
The training file was then given to Support Vector Machine . The training was done
on the default Support Vector Machine Kernel. Another feature of lib-SVM that us
training the system for probability generation was also incorporated while when training
was performed . The feature can be used running command svm-train-b 1
TraingFileName.

7.4 Testing and Decision-Making

Data were collected from each of the above approaches. It is preprocessed , the features
are extracted and the data is finally converted to the required SVM format to create the
testing data set. The two testing data formed are fed to the SVM individually. The
resulting probabilities from the above approaches are merged using the Dempster-
Shafer Theory . The resulting emotion is compared to the actual emotion of the subject
.

Major Project Report P.T.O

 29

8. PROPOSED COMBINATION OF TWO

APPROACHES
8.1 Generation of feature vector by combining both approaches

We have tested our system only by combining the feature vectors obtained from each of
the two approaches - questionnaire and text analysis-based to evaluate the performance
of the system. After combining, a 31 dimensionality feature vector has been generated
for each of the training and testing samples - 30 features from questionnaire-based
approach and 1 feature from text analysis-based approach.

8.2 Combining the probability scores of two classifiers

After generating the final feature vector by combining questionnaire and text analysis
based approaches, each feature vector is studied in both SVM and ANN-based platforms
to detect the emotional state of a person.
Both SVM and ANN classifiers return one probability score corresponding to each
emotion class and as a result a total of ten probability scores are obtained for five
different emotion classes from these two classifiers. Thus, the following two sets of
probability scores are generated from these two classifiers:

α = {SAnger, SSad, SJoy, SDisgust, SFear}

β = {NAnger, NSad, NJoy, NDisgust, NFear}

Here, α denotes the set of probability scores obtained from SVM classifier, whereas β
denotes the set of probabilities gained from ANN classifier.
DST has been used to combine the probability scores of these two classifiers. Fig. shows
the major steps of combining the probability scores of SVM and ANN classifiers using DST.

Major Project Report P.T.O

 30

The aim is to combine the outputs of SVM and ANN classifiers in the best way. To
accomplish this, it is required to convert the probabilistic output of each of these two
classifiers into a mass function, compute the conjunctive combination of the mass
functions, and design a decision function.
Initially, the probabilistic output of each of the two classifiers is converted into a mass
function. The inverse pignistic transform converts an initial probability distribution p into
a consonant mass function. The resulting consonant mass function is computed in the
following way:
Let pi be the probability value of a particular emotional approach set Ω corresponding to
a particular
emotion. First, the elements of Ω are ranked in decreasing order of probabilities as
defined in ,
p(e1) > ... > p(e|Ω|).

Next, mass function µ is described using following equation :

µ({e1,e2,...,e|Ω|}) = µ(Ω) = |Ω|×p(e|Ω|)

∀i < |Ω|,µ({e1,e2,...,ei}) = i×[p(ei)−p(ei+1)]

In our framework, six emotional classes have been considered, hence, |Ω| = 6. µ1(ei),
µ1(ei,ei+1), µ1(ei,ei+1,ei+2), µ1(ei,ei+1,ei+2,ei+3), µ1(ei,ei+1,ei+2,ei+3,ei+4) and
µ1(ei,...,ei+5) have been obtained from the resultant probability set of SVM classifier.
Here, each subset of µ1 is represented by X. Similarly, µ2(ei), µ2(ei,ei+1), µ2(ei,ei+1,ei+2),
µ2(ei,ei+1,ei+2,ei+3), µ2(ei,ei+1,ei+2,ei+3,ei+4) and µ2(ei,ei+1,ei+2,ei+3,ei+4,e have
been obtained from the resultant probability set of ANN classifier. Here, each subset of
µ2 is represented by Y . The above mass functions are combined using :

Major Project Report P.T.O

 31

where, A 6 = φ and A ⊆ Ω. For decision making, belief, plausibility and conflict are
computed for each emotion class E using following equation;

The calculated conflict for each of the emotions is compared with the threshold value T,
whose value is in [0, 1]. This threshold value is decided based on the experiment and
compared with the conflict values for the most suitable emotion. The emotions E,
satisfying the relation,
conflict(E) ≤ T
are the candidate emotion(s) to be accepted. Among these candidate emotions, the
emotion with the lowest conflict value is accepted as final emotional state of the person
and others are rejected. If it happens that none of the conflict(E) values is satisfying the
relation (17), then we will not accept any of the emotions as relevant and thus, reject all
the emotions E, for the result.

Major Project Report P.T.O

 32

9. RESULT ANALYSIS

We have tested our system using Linear kernel of SVM and multinomial naïve baise. The
accuracies are calculated using the individual approaches (text and questionnaire ) as
well as the combined approach. The detailed accuracies using individual and combined
approaches are shown in table given below.

Linear kernel of SVM

(%)
Text Analysis 59.48
Text+Questionnaire-Based
(Combined Approach) 81.74

Fig 10 . Principal Components Analysis

Major Project Report P.T.O

 33

To measure the performance of the proposed system when DST is applied,

we use Precision, Recall and F1-Score as performance measurement parameters using
following equations.

Fig 11 . Calculated Precision, recall and F1-score values

Major Project Report P.T.O

 34

And , then the related Confusion Matrix is Obtained:

Fig 12 : The Confusion Matrix

Major Project Report P.T.O

 35

10. CONCLUSION

Human emotion detection is a field that has tremendous potentials and is being
explored. Emotion plays an important role in human communication such as different
tweets and blogs written on social websites.
Most of the approaches for emotion detection such as facial recognition, brain signal
analysis etc. Are expensive and complex. A sound, reliable and simple emotion
detection approach would open the doors to solve the problems of multitude in this
society.
This article has explored a novel emotion detection model by applying the
combination of questionnaire and text analysis based approaches and then combining
probability scores of classifiers SVM using DST to determine the emotional state of
the subject with a certain accuracy. This type of emotion detection system can assist
the society to detect human emotions, especially of younger generations in order to
prevent them to commit suicide, apart from other societal benefits like human-
computer interaction. The novelty of this technique lies in its simplicity and feasibility
compared to other existing approaches.

Major Project Report P.T.O

 36

11. REFERENCES

[1]. A. Ortony, G.L. Clore, A. Collins, ”The cognitive structure of

emotions”, Cambridge University Press, 1998.
[2]. T. Partala, V. Surakka, ”Pupil size variation as an indication of
affective processing”, InternationalJournal of Human-Computer Studies,
2003.
[3]. Y. Zhang, Q. Ji, ”Active and dynamic information fusion for facial
expression understanding from image sequences”, IEEE Transactions on
Pattern Analysis and Machine Intelligence, 2005.
[4]. F. R. Chaumartin, ”A knowledge-based system for headline sentiment
tagging”, In Proceedings of the fourth international workshop on semantic
evaluations, 2007.
[5]. B. S. Tan, J. Zhang, ”An empirical study of sentiment analysis for
Chinese documents”, Expert Systems with Applications, 2008.
[6]. C.A. Frantzidis, C.D. Lithari, A.B. Vivas, C.L. Papadelis, C. Pappas,
P.D. Bamidis, ”Towards Emotion Aware Computing: a study of Arousal
Modulation with Multichannel Event-Related Poten tials, Delta Oscillatory
Activity and Skin Conductivity Responses”, In Proceedings of the 8th IEEE
International Conference on BioInformatics and BioEngineering, 2008.
[7]. R. Tokuhisa, K. Inui, Y. Matsumoto, ”Emotion classification using
massive examples extracted from the web”, In Proceedings of the 22nd
International Conference on Computational Linguistics, 2008.
[8]. A. Chakraborty, A. Konar, U.K. Chakraborty, A. Chatterjee, ”Emotion
Recognition From Facial Expressions and Its Control Using Fuzzy Logic”,
IEEE Transactions on Systems, Man, and Cybernatics, 2009.
[9]. H. Q. Ye, Z. Zhang, R. Law, ”Sentiment classification of online reviews
to travel destinations by supervised machine learning approaches”, Expert
Systems with Applications, 2009.
[10]. R.S. Smith, T. Windeatt, ”Facial Expression Detection using Filtered
Local Binary Pattern Features with ECOC Classifirs and Platt Scaling”,

Major Project Report P.T.O

 37

Journal of Machine Learning Research: Workshop and Conference

Proceedings, 2010.
[11]. U. Pal, P. P. Roy, N. Tripathy, J. Llads, ”Multi-Oriented Bangla and
Devanagari Text Recognition”, Pattern Recognition, Volume 43, 2010.
[12]. H. Binali, C. Wu, V. Potdar, ”Computational approaches for emotion
detection in text”, In Proceedings of the 4th IEEE International Conference
on Digital Ecosystems and Technologies, 2010.
[13]. A. Lanata, A. Armato, G. Valenza, E. P. Scilingo, ”Eye tracking and
pupil size variation as response to affective stimuli: A preliminary study”, In
Proceedings of the IEEE 5th International Conference on Pervasive
Computing Technologies for Healthcare, 2011 .
[14]. A. Martinez, S. Du, ”A Model of the Perception of Facial Expressions of
Emotion by Humans: Research Overview and Perspectives”, Journal of
Machine Learning Research, 2012.
[15]. G. E. Dahl, D. Yu, L. Deng, A. Acero, ”Context-Dependent Pre-Trained
Deep Neural Networks for Large-Vocabulary Speech Recognition”, IEEE
Transactions on Audio, Speech, and Language Processing, 2012.
[16]. A. Balahur, J. M. Hermida, A. Montoyo, ”Detecting implicit expressions
of emotion in text: A comparative analysis”, Decision Support Systems, 2012.
[17]. G. Valenza, A. Lanata, E. P. Scilingo, ”Oscillations of heart rate and
respiration synchronize during affective visual stimulation”, IEEE
Transactions on Information Technology in Biomedicine, 2012.
[18]. A. Martinez, S. Du, ”A Model of the Perception of Facial Expressions of
Emotion by Humans: Research Overview and Perspectives”, Journal of
Machine Learning Research, 2012.
[19]. M. A. Nicolaou, H. Gunes, M. Pantic, ”Output-associative RVM
regression for dimensional and continuous emotion prediction”, Image and
Vision Computing, 2012.
[20]. H. Gunes, B. Schuller, ”Categorical and dimensional affect analysis in
continuous input: Current trends and future directions”, Image and Vision
Computing, 2013.
[21]. T.E. Kontopoulos, C. Berberidis, T. Dergiades, N. Bassiliades,
”Ontology-based sentiment analysis of twitter posts”, Expert Systems with
Applications, 2013
Major Project Report P.T.O
 38

[22]. T. Baltrusaitis, N. Banda, P. Robinson, ”Dimensional affect recognition

using continuous conditional random fields,”, In Proceedings of the 10th
IEEE International Conference on Automatic Face and Gesture Recognition,
2013
[24]. S. Ramakrishnan, M.M. Iebrahiem, ”Speech emotion recognition
approaches in Human-Computer Interaction”, Telecommunication Systems,
2013.
[25]. G. Valenza, M. Nardelli, A. Lanata, C. Gentili, G. Bertschy, R.
Paradiso, E. P. Scilingo, ”Wearable monitoring for mood recognition in
bipolar disorder based on history-dependent longterm heart rate variability
analysis”, IEEE Journal of Biomedical and Health Informatics, 2014.
[26]. W. Li, H. Xu, ”Text-based emotion classification using emotion cause
extraction”, Expert Systems with Applications, 2014.
[27]. C. Aracena, S. Basterrech, V. Snael, ”Neural Networks for Emotion
Recognition Based on Eye Tracking Data”, In Proceedings of the IEEE
International Conference on Systems, Man, and Cybernatics, 2015.
[28]. S. L. Happy, A. Routray, ”Automatic Facial Expression Recognition
Using Features of Salient Facial Patches”, IEEE Transactions on Affective
Computing, 2015.
[29]. X. Li, J. Pang, B. Mo, Y. Rao, ”Hybrid neural networks for social
emotion detection over short text”,In Proceedings of the IEEE International
Joint Conference on Neural Networks, 2016.

Major Project Report P.T.O