SAI VIDYA INSTITUTE OF TECHNOLOGY

Accredited by NBA New Delhi (CSE, ECE, ISE, MECH, CIVIL), NAAC-“A” Grade

Department of Computer Science and Engineering

Rajanukunte, Bengaluru - 560064

Project exhibition 2023-24

EMO-VISION: Recognising Behavioral Patterns
By: Under the guidance of
Tanushree B. Sindagi Dr.A.M.Padma Reddy
(1VA22CS118) Mr. Kishor Patil
Guna M Mrs. Deepika G
(1VA22CS038) Mrs. Yashaswini D M
Samanth A.R
List of Contents

01
INTRODUCTION
02
PROBLEM-
03
LITERATURE
04
SYSTEM-
STATEMENT SURVEY REQUIREMENT

05
ARCHITECTURE
06
Where Ds is used?
07
IMPLEMENTATI
08
REFERENCES
ON

Dept. of CSE, SVIT

INTRODUCTION

who is guilty?
Dept. of CSE, SVIT
PROBLEM STATEMENT

"The current lack of effective systems for real-time behavioral pattern

analysis and emotion recognition hampers our understanding of human
experiences, contributing to challenges in communication, mental health,
and relationships. Developing accessible and reliable tools for accurately
identifying and interpreting behavioral patterns and emotions is crucial for
targeted interventions, personalized solutions, and fostering empathetic
environments."

TL;DR - BEHAVIOUR,
Dept. of CSE, SVIT
SOLUTION

Emo-
Dept. of CSE, SVIT
Vision
LITERATURE SURVEY

We refered many published papers on the topic

• ImageNet Classification with deep convolutional nural networks
-introduces CNN for recognition of Images from ImageNet Dataset
• Attention is all you Need
-Introduces Transformers for Data Procressing
• Kaggle Datasets for initial training

Dept. of CSE, SVIT

 SYSTEM REQUIREMENT

• Laptop
• i5+ CPU
• NVIDIA P100GPU
• Data-Sets

Dept. of CSE, SVIT

 ARCHITECTURE
Voice
Transformers
Adjustment
+ prediction
layer
RNN

Forward dialogue RNN cell

Backward dialogue RNN
cell
Dept. of CSE, SVIT
 ARCHITECTURE
image/1 frame of a video 1st layer
2nd layer
Angry
3rd layer 4th layer
48 Disgust

Fear

Happy
48

Neutral

Sad

pixeled form Surprise

Input Image

Iris movement up smell

1st layer u. left ext/int memory

2nd layer
3rd layer 4th layer lat. left auditory memory

l. left emotional memory

pixeled form down taste

l. right emotional memory

lat. right constructing words

u. right constructing images

Dept. of CSE, SVIT

 ARCHITECTURE
novel build
VOIC IMAG IRIS
E E

PREDICTIO
N

Dept. of CSE, SVIT

 ARCHITECTURE
Result

Dept. of CSE, SVIT

 WHERE Data-Structures IS USED?
• The collection of data (images/voice/iris) is done in ARRAYS (3D)
also known as TESNOR.

Dept. of CSE, SVIT

 WHERE Data-Structures IS USED?
• If we are having sequential data, such as a time series of iris
movements. Linked lists can be used in the pre-processing stage.

• As we are using a convolutional neural network (CNN), it contains

multiple neurons and these neurons are interconnected to each other
via GRAPHS.

Dept. of CSE, SVIT

IMPLEMENTATION
• hospital
• It can also be used in hospital areas, Such as observing our patient's
behaviors when they don't open up much and planning how we can
treat them.
• Very well used to determine the early symptoms of many diseases,
which also include Dementia, Brain Disorders etc.

Dept. of CSE, SVIT

IMPLEMENTATION
• hospital

Dept. of CSE, SVIT

IMPLEMENTATION
2. Criminology
• It can also be used in Interrogation rooms. we can run this system on
present suspects and find out their true behavior.
• We can classify our suspects whether they are guilty or not through
their Iris detection, Hand gestures, Body movements, and facial
movements as forms of input and narrow our interogation.

Dept. of CSE, SVIT

IMPLEMENTATION
3. Psychologists
• Can act as a personal assistant to a psychologist.
• Very helpful in judging a person’s character and giving the data and
psychologists move forward with their treatment of the person,
because even psychologists misjudge their patients few times.

Dept. of CSE, SVIT

IMPLEMENTATION
4. DRIVERS
• As it is a generation of electric cars and auto-driving cars, this system
can be used to analyze the emotions of vehicle drivers and collect data
on how their present emotions are and the car can stop automatically
if it's beyond the provided or safety data.
• This can prevent the possibility of accidents due to rash driving or any
such case.

Dept. of CSE, SVIT

IMPLEMENTATION
4. DRIVERS

Dept. of CSE, SVIT

REFERENCES
• Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh
, Emily Mower, Samuel Kim, Jeannette N. Chang, Sungbok
Lee and Shrikanth S. Narayanan for Speech Analysis and Interpretation Laboratory (SAIL) In
University of Southern California, Los Angeles, CA 90089
• Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton for Imagenet
Classification with deep Convolutional Neural Networks In University of Toronto.
• Cam-Van Thi
Nguyen, Anh-Tuan Mai, The-Son Le, Hai-Dang Kieu, Duc-Trong Le for Conversation Unders
tanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Int
eraction
In VNU University of Engineering, Vietnam, FPT AI center.
• Valentin Kassarnig, Enys Mones, Andreas Bjerre-Nielsen, Piotr Sapiezynski
for Academic Performance and behavioral patterns In Kssaening et al. EPJ DATA SCIENCE.
• Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion
Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
for attention is all you need in 31st conference on neural information processing systems, lon
g beach, CA, USA.
• Sail Chamishka, Ishra Madhavi, Rashmika Nawaratne, Damminda Alahakoon, Daswin
De Silva, Naveen Chiamkruti, Vishaka Nanayakkara
for A voice-based real-time emotion detection technique using recurrent neural network empo
wered feature modeling In Multimedia tools and applications (2022)
• What
Dept. of CSE, Every Body is saying - JOE NAVARO, PHD
SVIT