Context (3) (1) (AutoRecovered) (1) ... - Removed

Moodify: A Movie & Music Recommendation System Using Emotion Detection
CHAPTER 1
INTRODUCTION
1.1 Introduction
In an increasingly digitalized world where technology pervades nearly every

aspect of daily life, the intersection of artificial intelligence, multimedia content, and
user experience enhancement has become a focal point of research and innovation. One
particularly promising area within this domain is the development of recommendation
systems that leverage advanced techniques in computer vision and machine learning to
personalize content recommendations based on users' emotional states. Human
emotions play a pivotal role in shaping our interactions with various forms of media,
including music and movies. The ability to accurately discern and respond to users'
emotional cues presents a unique opportunity to tailor content recommendations in a
manner that resonates with their mood and preferences. Leveraging advancements in
facial emotion detection, coupled Using deep convolutional neural networks (DCNNs)
and deep learning algorithms, researchers have begun exploring innovative approaches
to enhance traditional recommendation systems.
This research paper aims to contribute to this evolving field by proposing a novel
approach to music and movie recommendation systems that integrates real-time facial
emotion detection. By harnessing the capabilities of CNN models trained with
frameworks such as Keras, we seek to develop a lightweight and efficient system
capable of analyzing users' facial expressions and predicting their emotional states with
high accuracy. In this paper, we will discuss the theoretical foundations of our proposed
system, including the underlying principles of facial emotion detection and CNN-based
image processing. We will also outline the methodology employed in the development
and training of our emotion detection model, as well as the integration ofthis model into
a music and movie recommendation framework. Furthermore, we will present
experimental results and performance evaluations to demonstrate the efficacy and
practicality of our approach.
Department of CSBS 1
Ultimately, we believe that the integration of facial emotion detection into music
and movie recommendation systems designs the caliber to innovate the way users
associate with digital content, offering a more immersive, personalized, and
emotionally resonant experience. Through this research, we aim to contribute to the
ongoing efforts to harness the power of artificial intelligence for the betterment of
human-computer interaction and content consumption experiences.
CHAPTER 2
LITERATURE SURVEY
2.1. Introduction
We went through a lot of different research papers to understand all the

previous work done on the project that we have undertaken.
2.1.1 Mood Indicator: Music and Movie Recommendation System Using

Facial Emotions
This paper analyses, compares, and contrasts the technological problems,

methodologies, and performance of colour picture text detection and recognition
studies. It highlights the underlying challenges and lists the things to consider when
fixing them. Existing techniques are classified as stepwise or integrated, with emphasis
on subproblems such as text localization, verification, segmentation, and recognition.
Special challenges relating to the enhancement of degraded text and the processing of
video text, multi-oriented, perspectival warped, and multilingual text arealso discussed.
The categories and subcategories of text are displayed, benchmark datasets are listed,
and the performance of the most representative algorithms is evaluated. This review
presents a fundamental comparison and analysis of the outstanding issues in the field.
In today's digital era, the demand for personalized content recommendation

systems is on the rise. Users increasingly expect tailored suggestions that resonate with
their current mood and emotional state. To meet this demand, the development of
innovative recommendation systems that leverage facial emotion analysis has gained
traction. The paper titled "Mood Indicator: Music and Movie Recommendation System
Using Facial Emotions" explores the design and implementation of such a system,
which aims to enhance user satisfaction and engagement by providing personalized
music and movie recommendations based on real-time facial expressions. Central to
the proposed system is the use of facial emotion analysis to detect users' emotional states
from their facial expressions.
Everyone strives for success in today's world, however many people suffer from
unhappiness as a result of this constant battle. Facial emotion recognition technology
has been given a lot of scope and interest by society and is being used in many industries
like security systems, DVP, and other technical breakthroughs. Mood Indicator was
developed to help people in recommending films and tunes based on their emotions.
60% of users say that the number of songs or videos in their collection is so big that
they can't pick which to play. By creating a recommendation system, we can help a user
in deciding which entertainment factor to listen to, which reduces the stress levels of
the client. It is time efficient too.
Once users' emotional states are detected, the system proceeds to recommend
music tracks and movie titles that align with their current moods. Leveraging extensive
databases of music and movies, such as Spotify and IMDb, the system curates tailored
recommendations that resonate with users' emotional states. Whether users are seeking
upbeat music to uplift their spirits or heartwarming movies to evoke nostalgia, the
system provides a diverse selection of content options to suit their preferences.
The "Mood Indicator" recommendation system prioritizes user experience and

engagement by offering a seamless and intuitive interface for interacting with the
system. Users can effortlessly navigate through recommended content, explore new
music and movies, and playback selected content with ease. By providing personalized
recommendations that cater to users' emotional needs, the system enhances user
satisfaction and fosters deeper engagement with multimedia content.
In conclusion, the "Mood Indicator: Music and Movie Recommendation System

Using Facial Emotions" paper presents an innovative approach to content
recommendation by integrating facial emotion analysis with music and movie
recommendations. By leveraging users' facial expressions to infer their emotional
states, the system offers personalized recommendations that resonate with users' moods.
Through this approach, the system enhances user satisfaction, engagement, and
enjoyment, paving the way for a more immersive and emotionally resonant multimedia
consumption experience.
2.1.2 Music Recommendation Based on Face Emotion Recognition
This paper proposed a new approach for playing music automatically using
facial emotion. Most of the existing approaches involve playing music manually, using
wearable computing devices, or classifying based on audio features. Instead, we advise
a modification to manual sorting and playing. We employed a Convolutional Neural
Network to detect emotions. Pygame and Tkinter are used to provide music
recommendations. Our proposed approach reduces the computational time required to
produce the results as well as the overall cost of the planned system, boosting the
system's overall correctness. The system is tested using the FER2013 dataset. The built-
in camera captures facial expressions. Feature extraction is used on input face photos to
detect emotions including happy, angry, sad, surprise, and neutral. Music playlists are
made automatically based on the user's current emotion. It performs better in terms of
computational time than the algorithm in the existing literature.
The paper presents a mobile application, EmoTunes, that detects the face
emotions of people and thereafter plays suitable music. The emotion detection system
works for any skin tone as the dataset introduced contains a mix of ethnicities. The
dataset was collected by asking friends, family members and acquaintances selfies of
themselves expressing the Ekman’s basic emotions. EmoTunes then used an ensemble
learning with MobileNetV2 and ResNet50 to detect the 7 Ekman’s emotions: Angry,
Disgust, Fear, Happy, Neutral, Sad and Surprise.
The system can detect single user as well as multi-user emotions. It has 2 types
of recommendation: Recommendation from server, where it fetches songs from the
database, and Recommendation via YouTube, where it takes only a single emotion and
fetches a YouTube song that best relates to the user’s preferences and liking.
EmoTunes uses 2 methods of discretizing multi-user emotions: either by finding the
most common emotion among them, or by finding the emotion of the face nearest to the
camera. During real-time emotion detection, the system detects the user’s emotion for
15 s and finds the most common emotion expressed within those15 s to play a music
directly from the server. A total validation accuracy of 99.64% was deduced from the
final model, making it one of the promising models when
compared to recent research work.
The system can be deployed to any part of the world as it adapts to each user’s
liking using the YouTube feature. While humans are social creatures, they do not like
to wait for services. EmoTunes can hence be deployed in waiting rooms such as for
healthcare facilities and at the metro/bus stations, or even in buses and trains in
Mauritius, or other parts of the world, to provide the users with some recomforting
music based on their current emotions. Future considerations might include further
working on the dataset to achieve a much bigger variation.
EmoTunes also addresses the challenge of discretizing multi-user emotions by

employing two methods: determining the most common emotion among users or
identifying the emotion of the face nearest to the camera. During real-time emotion
detection, EmoTunes analyzes users' emotions over a 15-second interval, selecting the
most common emotion expressed within that timeframe to play music directly from the
server. The final model achieves an impressive validation accuracy of 99.64%,
underscoring its efficacy in accurately discerning users' emotional states.
Beyond its technical capabilities, EmoTunes holds significant potential for

practical applications in various settings. The system's adaptability and real-time
response make it well-suited for deployment in waiting rooms, such as healthcare
facilities and transportation hubs like metro or bus stations. By providing users with
comforting music tailored to their current emotions, EmoTunes enhances the overall
user experience and offers a source of solace during moments of waiting or uncertainty.
Looking ahead, future considerations for EmoTunes may involve expanding and
refining the dataset to encompass an even broader variation of facial expressions and
emotions. This could further enhance the system's accuracy and effectiveness, ensuring
its continued relevance and utility in diverse environments and user scenarios. Overall,
EmoTunes represents a promising advancement in emotion-driven music
recommendation systems, offering both technical sophistication and practical
applicability in real-world settings
2.1.3 Music & Movie Recommendation System By FacialEmotions
In a digital age marked by the exponential growth of entertainment content, the

idea of tailoring recommendations to users' emotional states has been the driving force
behind our project. We embarked on a mission to create a recommendation system
leveraging emotion recognition technology to provide users with a more personalized,
engaging, and emotionally resonant entertainment experience. As we conclude our
project, we reflect on its accomplishments, potential, and future directions.
Our music and movie recommendation system based on emotions represents a

significant stride toward a more emotionally immersive and personalized digital
entertainment journey. While we celebrate our achievements, we are mindful of
addressing the challenges and limitations to enhance the system's accuracy, privacy,
and cultural inclusivity. This project sets the stage for a future where technology and
emotions converge, offering users an unparalleled exploration of the world of movies
and music, deeply aligned with their emotional states and preferences.
Music and movies play a fundamental role in shaping and elevating one's
emotional state, serving as powerful sources of entertainment and inspiration. Recent
research highlights the profound impact of music and movies on human emotions and
cognitive activity. In the contemporary digital landscape, characterized by an
abundance of content, the demand for personalized recommendations that align with
users' emotional states has taken center stage. This research project introduces an
innovative recommendation system that capitalizes on facial expressions to deliver real-
time, customized movie and music suggestions through emotion analysis. The system
employs facial feature detection techniques, incorporating both the Haar Cascade
algorithm and Convolutional Neural Networks (CNN).
The inclusion of an integrated camera for facial expression capturesignificantly

reduces the system's cost compared to alternative methods By fusing emotions with
cutting-edge technology, this system holds the potential to
revolutionize the entertainment experience by aligning content with human emotions.

It has the capacity to transform how individuals discover and engage with digital
entertainment, offering a more immersive and emotionally satisfying journey.
Addressing this demand, this research project introduces an innovative

recommendation system that leverages facial expressions to deliver real-time,
customized suggestions for music tracks and movie titles. By analyzing users' facial
expressions, the system can discern their emotional states and tailor recommendations
accordingly. This approach offers a more personalized and emotionally resonant
entertainment experience, enhancing user satisfaction and engagement.
including the Haar Cascade algorithm and Convolutional Neural Networks

(CNNs). These advanced technologies enable the system to accurately capture and
analyze users' facial expressions, providing valuable insights into their emotional states.
By integrating such cutting-edge methods, the system enhances its ability to understand
and respond to users' emotions effectively.
Moreover, the inclusion of an integrated camera for facial expression capture

represents a cost-effective solution compared to alternative methods. This aspect makes
the system accessible to a broader audience, democratizing the benefits of personalized
content recommendations based on emotional analysis. By making use of existing
hardware components, such as smartphone cameras, the system becomes more readily
available and scalable for widespread adoption.
By merging emotions with state-of-the-art technology, this recommendation

system has the potential to revolutionize the entertainment experience. It offers a
seamless integration of human emotions with digital content, paving the way for a more
immersive and emotionally satisfying journey through music and movies. Ultimately,
this research project seeks to transform how individuals discover, engage with, and
derive meaning from digital entertainment, ushering in a new era of emotionally
resonant content consumption.
2.1.4 Emotional Detection and Music Recommendation System based on User

Facial Expression
It is often confusing for a person to decide which music he/she must listen from
a massive collection of existing options. There have been several suggestion
frameworks available for issues like music, dining, and shopping depending upon the
mood of the user. The main objective of our music recommendation system is to provide
suggestions to the users that fit the user’s preferences. The analysis of the facial
expression/user emotion may lead to understanding the current emotional or mental
state of the user. Music and videos are one region where there is a significant chance to
prescribe abundant choices to clients considering their inclinations and also recorded
information. It is well known that humans make use of facial expressions to express
more clearly what they want to say and the context in which they meant their words.
More than 60 percent of the users believe that at a certain point of time the
number of songs present in their song library is so large that they are unable to figure
out the song which they must play. By developing a recommendation system, it could
assist a user in deciding which music one should listen to, helping the user to reduce
his/her stress levels. The user would not have to waste any time in searching or to look
up songs and the best track matching the user’s mood is detected, and songs would be
shown to the user according to his/her mood. The image of the user is captured with the
help of a webcam. The user’s picture is taken and then as per the mood/emotion of the
user an appropriate song from the playlist of the user is shown matching the user’s
requirement.
The analysis of facial expressions and user emotions serves as a valuable tool in
understanding the user's current emotional or mental state. Facial expressions are a
universal language through which individuals convey their feelings and intentions,
providing valuable insights into their mood and emotional well-being. Leveraging this
information, our recommendation system aims to offer personalized music suggestions
that resonate with the user's emotions and preferences.
Music and videos represent fertile ground for recommendation systems, as
they offer abundant choices that can cater to diverse tastes and preferences. However,
navigating through this vast sea of content can be overwhelming for users, particularly
when faced with a large and diverse music library. Studies have shown that a significant
portion of users struggle to select the perfect song from their extensive collection,
leading to frustration and stress.
By developing a recommendation system that takes into account the user's mood
and emotions, we aim to alleviate this burden and enhance the user's music listening
experience. Rather than spending time searching for the right song, our system
automates the process by capturing the user's facial image using a webcam and
analyzing their mood/emotion. Based on this analysis, the system selects an appropriate
song from the user's playlist that matches their emotional state and preferences.
This approach not only saves users time and effort but also helps reduce stress
levels by providing them with music that resonates with their current mood. By
seamlessly integrating facial emotion analysis with music recommendation technology,
our system offers a more intuitive and personalized music listening experience,
enhancing user satisfaction and engagement. With the user's emotional well-being at
the forefront, our recommendation system aims to revolutionize the way individuals
discover and enjoy music, providing a seamless and emotionally enriching journey
through their music libraries.
Furthermore, our recommendation system aims to bridge the gap between the
user's emotional state and their music preferences, creating a more seamless and
intuitive music listening experience. By understanding the user's mood and emotions,
the system can offer relevant music suggestions tha resonate with their current state
of mind. This not only enhances user satisfaction but also promotes emotional well-
being by providing music that aligns with the user's feelings and helps them navigate
through their emotional landscape.
2.1.5 Movie & Music Recommendation System Based on Facial Expressions
The proposed model depicts the recommendation system built to detect facial
expressions and automatically suggest the movies and music from the sample dataset.
There are many existing models which recommend music/movie based on genre, but
it accompanies manual sorting. So, to avoid manual sorting and reduce the browsing
time this model is proposed which is titled ‘Movie & Music recommendation system
based on facial expressions. We express our emotions using facial expressions.
Whenever we are in a happy, party, sad or gloomy mood we mostly prefer to listen some
songs or just sit and watch movies and spend our leisure time. But the most common
problem we all face is, whenever we hop to any movies or music platforms, we find
tons of options and get stuck & confused what to watch in terms of movies and spend
most of our time browsing. This utilizes a lot of our time. Music has always been known
to alter human emotions, enhancing their mood. In some cases mood alterations can
help to reduce depressions and sadness. So, to overcome manual sorting and save the
browsing time to play the music/movie with recommendation of our expressions,
‘Movie & Music Recommendation system based on Facial expressions’ model is
proposed. The model is named ‘Moodlift’.
The proposed model aims to capture the image of the user using an inbuilt
webcam. With the help of image segmentation and image processing techniques it
extracts features from the face of a target individual and tries to detect the emotion. The
project aims to lighten the mood of the user, by playing songs that match requirements
of the user. The Convolutional Neural Network will be used to detect emotions. It
consists of an input layer, convolutional layer, dense layer and output layer. CNN
extracts feature from the image and determines the specific expression. To accurately
detect emotions, we need to detect face within an image and run the model to detect
expressions. For music recommendation, Deep Learning and Spotify Apiwill be used
and for movie recommendation it is routed to IMDB site. It recognizes facial expression
based on the 5 categories i.e. angry, sad, happy, surprise and neutral. Based on the
emotion captured it will give user two choices either suggesting movies or songs. The
recommendation system is exhibited on a website created using Html and CSS with
Flask.
The emotions of an individual will be captured using an inbuilt webcam. The

emotions are then detected and analyzed by Convolution Neural Network which is used
for image detection & image processing. It has an input layer, convolutional layer, dense
layer and output layer. CNN extracts feature from images to determine emotion. Testing
of images is done from FER 2013 dataset. After the emotions are detected, mood
classification is done (for song: tone, valence, timber; for movie: happy, surprise, sad,
neutral, anger.) The model is displayed on a website created using Html and CSS with
Flask as interface. This system yields better performance results. The proposed system
has the potential to be used in a variety of applications, including personalized song
recommendations for individuals, song therapy, and even public spaces such as
shopping malls or airports, where music can be used to improve the mood of those
present. Overall, the proposed system is an innovation approach to personalized music
recommendations, which takes into account a person's emotional state, allowing them
to discover new music that resonates with their current mood. Integrate physiological
signals to improve accuracy and effectiveness in music recommendation.
Usually, the weighted-SoftMax loss approach is used to handle this problem by

weighing the loss term for each emotion class supported by its relative proportion within
the training set. However, this weighted-loss approach is predicated on the SoftMax
loss function, which is reported to easily force features of various classes to stay apart
without listening to intra-class compactness. One effective strategy to deal with the
matter of SoftMax loss is to use an auxiliary loss to coach the neural network. To treat
missing and Outlier values we have used a loss function named categorical Cross
entropy. For each iteration, a selected loss function is employed to gauge the error value.
So, to treat missing and Outlier values, we have used a loss function named categorical
cross entropy.
The extensive focus of the model was to classify music as per different
emotions. Main aspect for music recommendation was to train model to take different
features of song like acoustic, timber, tone, rhythm etc. and to determine if a song was
happy, sad, neutral, energetic. We found a playlist of songs as per genre and mood on
Spotify. For happy songs we used pop as EDM, for sad we used mellow and for neutral
we used Lofi.
More than 300 songs per genre were used to create a dataset of 1200 songs. For
movie recommendation we have trained the model with FER 2013 dataset images to
extract features to determine the genre depending on the five emotions. We have used
one hot encoding to give each song a label to use as training data for the model. Now
with two data frames containing song links and labels respectively, it was time to get
songs data we needed to make classifications. We used Spotify API to extract song
information and store it into the dataset. To train our classifier we used Tensorflow &
Keras. To make pandas data frames compatible with model, we had to convert them
into numpy model. Later, we used sklearn to split our dataset. The final layer was
created with keras which contained single dense layer with Relu activation function, it
was trained using scikit learn keras classifier function.
CHAPTER 3
EXISTING AND PROPOSED SYSTEM
3.1 Existing System
Existing music and movie recommendation systems predominantly rely on

Convolutional neural networks (CNNs) are used to study user preferences and make
personalized recommendations. These systems typically leverage users' search history,
listening habits, and ratings to generate music recommendations tailored to individual
tastes. However, while CNNs excel at processing and extracting features from music
data, they often overlook the realm of movie recommendations, leaving users without
comprehensive suggestions for their entertainment needs. Moreover, existing systems
tend to be complex, requiring extensive data processing and computational resources
to deliver accurate recommendations. Despite their effectiveness in music
recommendation, these systems have several limitations, including their inability to
offer movie recommendations, leading to a fragmented user experience.
Additionally, the reliance solely on user interaction data such as search history
and ratings may result in recommendations that lack diversity or fail to capture users'
evolving preferences accurately. This limitation highlights the need for more holistic
recommendation systems that consider a broader range of factors, including users'
emotional states and external influences such as trending topics or cultural events. By
incorporating additional sources of data and employing more sophisticated algorithms,
future recommendation systems could overcome these limitations and provide more
comprehensive and personalized recommendations for both music and movies.
In conclusion, while existing music recommendation systems leveraging CNNs

have demonstrated effectiveness in providing personalized music suggestions based on
user interactions, they often fall short in offering comprehensive movie
recommendations and may suffer from complexity and scalability issues. Addressing
these limitations requires the development of more integrated and user-centric
recommendation systems that consider a broader range of factors, including emotional
states and external influences, to deliver more relevant and engaging content
recommendations for users.
3.1.1 Disadvantages of Existing System
Some users may be resistant to the idea of using facial recognition technology
for music recommendations due to concerns about surveillance, security, or simply a
preference for more traditional and less invasive methods. A facial recognition system
relies solely on visual input, neglecting other important contextual factors that
influence musical preferences, such as mood, location, and social context.
Implementing facial recognition technology can be resource-intensive, requiring
significant computational power and storage. This may increase the cost of developing
and maintaining the music recommendation system.
3.2 Proposed System
This paper proposes an innovative approach to enhance music and movie

recommendation systems by integrating facial emotion analysis with content retrieval
from Spotify and IMDb. Leveraging advancements in computer vision and machine
learning, the proposed system aims to provide personalized recommendations tailored
to users' emotional states, thereby enhancing their entertainment experiences. Facial
emotion analysis techniques are employed to extract emotional cues from users' facial
expressions in real-time. These cues are then mapped to corresponding music tracks
and movie selections retrieved from Spotify and IMDb, respectively. The system
leverages the Spotify API to access a vast repository of music content, including
personalized playlists and recommendations,while also utilizing IMDb's
comprehensive database to retrieve movie information such as titles, synopses, genres,
and ratings. By seamlessly integrating emotion-driven content recommendations from
Spotify and IMDb within a unified interface, users are offered a tailored entertainment
experience that aligns with their emotional states and preferences. The proposed
system represents a novel approach to content recommendation that bridges the gap
between emotion analysis and multimedia content retrieval, ultimately enhancing user
satisfaction and engagement.
3.3 Flowchart of the System
Fig 3.3 Describes the Flowchart of System Architecture
An innovative approach to enhance music and movie recommendation systems

by integrating facial emotion analysis with content retrieval from Spotify and IMDb.
Leveraging advancements in computer vision and machine learning, the proposed
system aims to provide personalized recommendations tailored to users' emotional
states, thereby enhancing their entertainment experiences. Facial emotion analysis
techniques are employed to extract emotional cues from users' facial expressions in
real-time. These cues are then mapped to corresponding music tracks and movie
selections retrieved from Spotify and IMDb, respectively. The system leverages the
Spotify API to access a vast repository of music content, including personalized
playlists and recommendations, while also utilizing IMDb's comprehensive database
to retrieve movie information such as titles, synopses, genres, and ratings.
By seamlessly integrating emotion-driven content recommendations from

Spotify and IMDb within a unified interface, users are offered a tailored entertainment
experience that aligns with their emotional states and preferences. The proposed system
represents a novel approach to content recommendation that bridges the gap between
emotion analysis and multimedia content retrieval, ultimately enhancing user
satisfaction and engagement.
 Real-time video stream: The system first captures a video stream from a webcam.
 Capture Frame: An individual frame is captured from the video stream.
 Detecting face using Haar Cascade Classifier: The Haar cascade classifier is used
to detect faces within the captured frame. This classifier is a machine learning
model that’s trained to identify specific features within an image, such as edges and
lines. By identifying these features, the classifier can recognize the presence of a
face.
 Preprocessing and Pixelization of image: The captured frame is then pre-
processed, which involves preparing the image data for further processing. This may
involve resizing the image, converting it to grayscale, or applying other techniques
to improve the image quality for face detection. After preprocessing, the image is
pixelated, which reduces the resolution of the image. This is done to protect the
user’s privacy.
 Feature Extraction: Facial features are then extracted from the pre-processed frame.
These features may include the distance between the eyes, the shape of the nose,
and the curvature of the lips.
 Emotion Recognition using Trained Model: A trained model is used to recognize
the emotions from the extracted facial features. This model is likely a convolutional
neural network (CNN) that has been trained on a dataset of labelled faces. The CNN
is able to identify patterns in the facial features that correspond to different
emotions.
 Emotion Detected: Once the emotion is recognized, the system can take various
actions. In the flowchart you sent me, the system is designed to recommend music
and movies based on the user’s emotion. For instance, if the system detects that the
user is happy, it might recommend an upbeat song playlist or a comedy movie.
 Recommend Movies & Songs: The system recommends songs from Spotify and
movies & descriptions from IMDB based on the user’s detected emotion.
 Play suggested Spotify Playlist: If the user chooses to play the recommended
songs, the system will play the suggested playlist on Spotify.
 End: This is the final state of the flowchart.
The proposed approach represents a groundbreaking advancement in music and

movie recommendation systems by seamlessly integrating facial emotion analysis with
content retrieval from Spotify and IMDb. Leveraging state-of-the-art techniques in
computer vision and machine learning, this innovative system aims to deliver
personalized recommendations tailored to users' emotional states, thereby enhancing
their entertainment experiences to unprecedented levels. The process begins with real-
time video streaming captured from a webcam, allowing the system to analyze users'
facial expressions as they interact with the application.
Once the video stream is captured, the system proceeds to capture individual
frames, each of which contains valuable visual information for facial emotion analysis.
To detect faces within these frames, the system employs the Haar cascade classifier, a
machine learning model trained to identify specific features indicative of facial
structures. This classifier utilizes pattern recognition algorithms to identify edges, lines,
and other distinguishing features, enabling it to accurately recognize the presence of
faces within the captured frames.
Following face detection, the captured frame undergoes preprocessing to

enhance image quality and facilitate more accurate feature extraction. This
preprocessing may involve resizing the image, converting it to grayscale, or applying
other techniques to optimize image data for subsequent analysis. Additionally, to
protect users' privacy, the captured image is pixelated, reducing its resolution and
obscuring identifiable facial features.
With the pre-processed image in hand, the system proceeds to extract facial
features, such as the distance between the eyes, the shape of the nose, and the curvature
of the lips. These features serve as valuable inputs for emotion recognition, allowing
the system to discern users' emotional states from their facial expressions. Employing a
trained model, likely a convolutional neural network (CNN) trained on a dataset of
labelled faces, the system analyses these facial features to identify patterns
indicative of various emotions.
Upon successful emotion recognition, the system proceeds to recommend music

tracks and movie titles tailored to the user's detected emotion. Leveraging the vast
repositories of Spotify and IMDb, the system retrieves personalized playlists, movie
synopses, genres, and ratings corresponding to the user's emotional state. This
integration of emotion-driven content recommendations within a unified interface
offers users a tailored entertainment experience that resonates with their emotional
states and preferences, ultimately enhancing user satisfaction and engagement.
If the user chooses to play the recommended songs, the system initiates
playback of the suggested Spotify playlist, providing users with a seamless and
immersive music listening experience. This comprehensive process culminates in the
system's ability to provide personalized music and movie recommendations based on
the user's emotional state, enriching the entertainment experience and setting a new
standard for content recommendation systems in the digital age.
Upon emotion detection, the system recommends songs from Spotify and
movies from IMDb that align with the user's detected emotion, ensuring a highly
relevant and engaging entertainment selection. Users have the option to explore and
play the recommended content directly from the interface, facilitating a seamless
transition from recommendation to consumption.
Moreover, the system's ability to dynamically adjust recommendations based on

users' real-time emotional states enhances its adaptability and responsiveness to users'
evolving preferences. Whether users are seeking upbeat music to uplift their mood or
heartwarming movies to evoke nostalgia, the system offers a diverse range ofcontent
options to suit their emotional needs.
Overall, upon emotion detection, the system recommends songs from Spotify
and movies from IMDb tailored to the user's detected emotion. If the user opts to play
the recommended songs, the system initiates playback of the suggested Spotify playlist.
This comprehensive process culminates in the system's ability to provide personalized
music and movie recommendations based on the user's emotional state, enriching the
entertainment experience.
In conclusion, the facial emotion-driven music and movie recommendation

system presented herein represents a significant advancement in the field of content
recommendation. By harnessing the power of facial emotion analysis and leveraging
content from Spotify and IMDb, the system delivers personalized recommendations that
resonate with users' emotional states, ultimately enhancing their entertainment
experiences and setting a new benchmark for recommendation systems in the digital
era.
3.4 For the app.py code
3.4.1 Software Requirements
 Python 3. x installed on the system.
 Anaconda navigator latest version installed on the system.
 VS code installed on the system.
 Download dataset face-expression-recognition-dataset from Kaggle.
 Data files of music & movies in excel format.
 Required Python libraries installed, including:
o keras
o pandas
o opencv
o statistics
o tensorflow
o numpy
o flask
o matplotlib
3.4.2 Hardware Requirements
 A computer with sufficient CPU and memory resources to run Python and the
required libraries efficiently.
 Internet connectivity to access the URL posted.
CHAPTER 4
SYSTEM DESIGN
Designing a music recommendation system based on facial emotions involves

integrating various technologies such as facial recognition, emotion detection, and
machine learning algorithms.
4.1 Capture Image
Fig. 4.1 Describes the Design Phases of System
Collecting photos is an important step in facial expression recognition; with

computer vision libraries like OpenCV, capturing and processing photographs is
considerably easier and more efficient. Facial emotion recognition systems analyse the
many aspects of a person's face to determine the emotions they are experiencing. To
capture an image of a face, a camera is used to photograph the individual. The image is
then analysed with the OpenCV libraries to recognize face features such the eyes, nose,
and mouth. OpenCV has a few images processing capabilities, including face detection,
which can be used to recognize a face in an image. After detecting the face, numerous
features can be extracted, including the position of the eyes, mouth, and nose, as well
as the contour of the face. Using this information, we may determine the person's
emotion. For example, if the corners of the lips are turned up, the person
is probably happy. Similarly, if the brows are furrowed and the lips is directed
downward, the person may be feeling sorrowful.
Facial emotion recognition systems are designed to analyze various aspects of

a person's face to discern the emotions they are experiencing. To initiate this process,
a camera is employed to capture an image of the individual's face. This image is then
subjected to analysis using the functionalities offered by OpenCV libraries, which excel
in recognizing and processing facial features such as the eyes, nose, and mouth.
OpenCV encompasses a wide array of image processing capabilities, among

which face detection stands out as a fundamental feature utilized in facial emotion
recognition. Through face detection algorithms, OpenCV can effectively identify and
localize faces within an image, laying the groundwork for subsequent analysis. Once
the face is detected, the system can proceed to extract various features, including the
position of the eyes, mouth, and nose, as well as the contour of the face. In the proposed
system, the DCNN plays a central role in the detection of seven primary emotions—
anger, sadness, disgust, happiness, surprise, neutral, and fear—extracted from users'
facial expressions in real-time. Through the integration of state-of-the-art computer
vision techniques, the system achieves high levels of accuracy in identifying users'
emotional states and mapping them to corresponding content recommendations.
By leveraging the extracted facial features, the system can make inferences
about the person's emotional state. For instance, if the corners of the lips are turned up,
indicating a smile, the person is likely experiencing happiness. Conversely, if the brows
are furrowed and the lips are directed downward, it may suggest feelings of sorrow or
distress.
The ability to capture and analyze facial expressions in this manner opens up a
myriad of applications across various domains, ranging from psychology and human-
computer interaction to marketing and entertainment. By harnessing the capabilities of
OpenCV and facial emotion recognition systems, researchers and developers can gain
valuable insights into human emotions and behaviors, paving the way for innovative
solutions and experiences tailored to individual needs and preferences.
4.2 Face Identification Using Deep CNN
DCNNs are a sort of deep learning algorithm that is well-suited to image

recognition applications. The training phase entails feeding a large number of images
to the deep CNN and modifying the weights of the neurons in the network to reduce the
disparity between the expected and actual outputs. Once trained, the CNN can detect
faces in new photos. Face detection with deep CNNs has numerous advantages over
classic approaches like Haar cascades. Deep CNNs are far more accurate at detecting
faces in a variety of lighting conditions, orientations, and poses. They can also
recognize numerous faces in an image and generate a bounding box around each one.
The training phase of a DCNN involves exposing the network to a large dataset
comprising images containing faces. During training, the network adjusts the weights
of its neurons to minimize the difference between the expected outputs (i.e., the
presence or absence of a face) and the actual outputs generated by the network. Through
this iterative process, the DCNN learns to recognize and differentiate between facial
features, enabling it to accurately detect faces in new photos.
One of the key advantages of using DCNNs for face detection is their superior
accuracy compared to traditional approaches such as Haar cascades. DCNNs are
capable of detecting faces with high precision across a wide range of lighting
conditions, orientations, and poses. This robustness makes them highly effective in real-
world scenarios where faces may appear in varying contexts and environments.
Moreover, DCNNs excel at recognizing multiple faces within an image, a task

that can be challenging for conventional face detection methods. By leveraging the
hierarchical structure of the network and employing sophisticated feature extraction
techniques, DCNNs can identify and localize multiple faces simultaneously, generating
bounding boxes around each detected face.
The ability of DCNNs to accurately and efficiently detect faces in images has
numerous practical applications across various domains, including surveillance,
security, and human-computer interaction. In surveillance systems, for example,
DCNN-based face identification can be used to monitor crowded areas and identify
individuals of interest in real time. Similarly, in security applications, DCNNs can
enhance access control systems by verifying the identities of individuals based on their
facial features.
In summary, face identification using Deep Convolutional Neural Networks

offers a powerful and versatile approach to detecting faces in images. By leveraging the
capabilities of deep learning and sophisticated neural network architectures, DCNNs
provide unparalleled accuracy and robustness in face detection, making them invaluable
tools in a wide range of applications requiring reliable and efficient facial recognition
capabilities.
4.3 Features Extraction
Feature extraction is the process of detecting and extracting the important facial
features required for emotion recognition. The most prevalent facial emotion
identification features are the position and form of the eyes, brows, mouth, and nose.
Other characteristics, such as skin texture and colour, as well as face shape, may also
be considered. Haar cascades are one of the most often used feature extraction
approaches in facial emotion identification. Haar cascades are essentially a collection
of features used to recognize objects in an image. In the context of facial emotion
identification, these traits are intended to detect the eyes, nose, mouth, and other
significant facial features. Haar cascades work by scanning an image at various scales
and sizes to detect the presence of features. The features are essentially rectangular
sections of the image, containing dark and light areas. Combining numerous
characteristics allows for the detection of more complex objects, such as faces. Once
the features are identified, they can be used to train a machine learning model, such as
CNN, to distinguish emotions.
One of the most used approaches for feature extraction in facial emotion
recognition is the Haar cascade method. Haar cascades are essentially a collection of
features specifically designed to identify objects within an image. In the context of
facial emotion identification, Haar cascades are utilized to detect important facial
landmarks such as the eyes, nose, mouth, and other prominent features indicative of
emotional expressions.
The process of Haar cascade feature extraction involves scanning an image at

various scales and sizes to identify regions of interest that may contain the desired facial
features. These features are essentially rectangular sections of the image characterized
by alternating patterns of light and dark areas. By combining multiple features, Haar
cascades enable the detection of more complex objects, such as faces, within an image.
Once the relevant facial features are identified using Haar cascades, they can be
utilized to train machine learning models, such as Convolutional Neural Networks
(CNNs), to distinguish between different emotional expressions. CNNs are particularly
well-suited for this task, as they can effectively learn and extract intricate patterns and
features from raw image data, enabling them to accurately classify emotions based on
the detected facial features.
Overall, feature extraction plays a critical role in facial emotion recognition

systems by enabling the identification and extraction of key facial landmarks and
characteristics essential for accurately interpreting emotional expressions. By
leveraging techniques such as Haar cascades and machine learning models like CNNs,
facial emotion recognition systems can achieve high levels of accuracy and robustness
in detecting and classifying emotions based on facial cues.
4.4 Emotion Identification
Deep convolutional neural network (DCNN) is one of the most successful

machine learning methods for emotion classification based on face information. CNNs
are a sort of deep learning algorithm that is especially excellent at image recognition.
The emotion classification module can be built with a deep CNN algorithm trained on
a huge dataset of tagged facial expressions. Deep CNN learns to recognize patterns in
facial features linked with various emotions and can effectively classify the listener's
current emotional state using these aspects. One advantage of utilizing a deep CNN
algorithm for emotion classification is that it can detect both
local and global characteristics of the facial expression. For example, the algorithm can
learn to distinguish precise patterns in the position of the eyes or mouth, as well as more
general patterns in the overall form of the face.
One of the key advantages of employing a deep CNN algorithm for emotion
classification lies in its ability to detect both local and global characteristics of facial
expressions. This means that the algorithm can effectively capture subtle nuances in
facial features, such as the precise positioning of the eyes or mouth, while also
recognizing broader patterns in the overall structure of the face. By leveraging these
capabilities, the DCNN can accurately classify the listener's current emotional state
based on the extracted facial features, providing valuable insights into users' emotional
experiences.
By seamlessly integrating emotion-driven content recommendations within a

unified interface, the system delivers personalized entertainment experiences tailored
to users' current emotional states and preferences. Leveraging the capabilities of the
DCNN, users are presented with music tracks and movie selections retrieved from
Spotify and IMDb, respectively, that align with their emotional states. This holistic
approach to content recommendation ensures that users are provided with meaningful
and relevant entertainment options that resonate with their emotions, enhancing their
overall viewing and listening experiences.
The proposed system incorporates the detection of seven primary emotions—

anger, sadness, disgust, happiness, surprise, neutral, and fear—extracted from users'
facial expressions in real-time. Leveraging state-of-the-art computer vision techniques,
the system accurately identifies users' emotional states and maps them to corresponding
music tracks and movie selections retrieved from Spotify and IMDb, respectively.
4.5 Music & Movies
In the context of movie and song recommendations based on extracted

emotions, the system utilizes the detected emotional states to tailor content suggestions
that resonate with users' current moods and preferences. Once facial
expressions are analyzed and emotions are extracted using advanced computer vision
techniques, the recommendation system employs sophisticated algorithms to map these
emotional cues to appropriate movie genres or musical genres, styles, or specific tracks.
For example, if a user is identified as expressing joy or excitement, the system might
suggest upbeat and cheerful songs or movies with uplifting themes.
Conversely, if a user displays signs of sadness or melancholy, the system may

recommend soothing music or movies with poignant narratives. The recommendation
process is guided by a comprehensive understanding of the emotional content inherent
in both the user's expressions and the available media content. Through this
personalized approach, users are provided with tailored recommendations that not only
match their current emotional states but also enhance their overall viewing or listening
experience by fostering emotional resonance and engagement with the suggested
content.
4.6 Spotify Playlist and IMDB Info
In the subsequent phase of the recommendation system's operation, after

emotions have been extracted and content recommendations have been generated, the
system proceeds to retrieve relevant music tracks from Spotify playlists and movie
information from IMDb (Internet Movie Database). Leveraging APIs (Application
Programming Interfaces) provided by these platforms, the recommendation system
seamlessly accesses their vast repositories of music and movie data. For music
recommendations, the system interfaces with the Spotify API to fetch songs or playlists
that align with the user's emotional state and preferences. By accessing Spotify's
extensive catalog and utilizing features such as mood-based playlists or genre
categorization, the system ensures that the recommended music is not only emotionally
suitable but also reflective of the user's taste and interests.
Simultaneously, for movie recommendations, the system interacts with the

IMDb API to retrieve comprehensive information about movies matching the user's
emotional profile. This includes details such as movie titles, synopses, genres, ratings,
cast, and crew information. By tapping into IMDb's vast database of movies and
associated metadata, the system ensures that the recommended films cater to the user's
emotional needs while offering a diverse range of cinematic experiences.
4.7 Play The Songs & Movie Recommendation
In the final phase of the recommendation system's workflow, users are presented
with the curated music tracks sourced from Spotify playlists and movie
recommendations sourced from IMDb. Leveraging the seamless integration with these
platforms, the recommendation system delivers a tailored entertainment experience that
aligns with users' emotional states and preferences.
For music playback, the system accesses the selected tracks from Spotify
playlists directly within the application interface. Users can enjoy uninterrupted
streaming of the recommended songs without the need to switch between multiple apps
or interfaces. By harnessing Spotify's extensive library and personalized playlists, users
are presented with a diverse range of music that complements their current mood and
enhances their listening experience. Simultaneously, users are provided with detailed
information about the recommended movies sourced from IMDb. Leveraging IMDb's
rich database, the recommendation system displays movie titles, synopses, genres,
ratings, cast, and crew information directly within the application interface. Users can
explore additional details about the recommended films and make informed decisions
about their viewing preferences.
By seamlessly integrating music playback from Spotify and movie

recommendations from IMDb within the recommendation system, users are offered a
streamlined and immersive entertainment experience. The collaborative effort between
the recommendation system and these external platforms ensures that users have access
to a wide selection of high-quality content that resonates with their emotions and
preferences. This final phase represents the culmination of the recommendation
process, where users can seamlessly transition from content discovery to consumption
Within A Unified And User-Friendly Interface, Ultimately Enhancing Their Overall
Entertainment Satisfaction.
4.8 UML Diagrams
Unified Modelling Language (UML) diagrams are graphical representations

used in software engineering to visualize, specify, construct, and document the artifacts
of a software system. Developed by the Object Management Group (OMG), UML
provides a standardized notation for depicting various aspects of a software system's
structure, behaviour, and interactions.
UML diagrams serve as blueprints or visual models that help stakeholders,

including developers, designers, project managers, and clients, to understand,
communicate, and analyse the system's design and functionality more effectively. These
diagrams facilitate the process of software development by providing a common
language and visual framework for discussing and documenting system requirements,
architecture, and design decisions.There are several types of UML diagrams, each
focusing on different aspects of the software system. These diagrams allow for the
representation of static and dynamic aspects of the system, as well as its structure,
behaviour, and interactions. Each type of UML diagram serves a specific purpose and
provides a unique perspective on the system being modelled, helping stakeholders to
address different aspects of the system's development and maintenance.
In summary, UML diagrams are graphical tools used in software engineering to

depict various aspects of a software system's structure, behaviour, and interactions.
They play a crucial role in facilitating communication, analysis, design, and
documentation throughout the software development lifecycle.
4.8.1 Use Case Diagram
A use case diagram in the Unified Modeling Language (UML) is a visual

representation that captures the functionality of a system from the perspective of its
actors and their interactions with various use cases. Derived from use-case analysis,
this diagram serves as a pivotal tool in software engineering, offering stakeholders a
clear and concise overview of the system's behavior and requirements. Actors,
representing entities external to the system, interact with the system to accomplish
specific goals or tasks, which are encapsulated as use cases. These use cases delineate
the functionalities offered by the system to fulfill the needs of its actors. The primary
objective of a use case diagram is to delineate these interactions, depicting the roles of
actors, the functionalities provided by the system, and any dependencies or
relationships between them. By visualizing the system's functionality, stakeholders
gain insight into how the system supports the objectives of its users and can
communicate effectively about the system's requirements and behavior. Whether
illustrating simple systems with few actors and use cases or complex systems with
multiple interactions.
Fig 4.8 (a) Describes the Use Case Diagram for movie and music recommendation.
In the Fig 4.8 (a) describes use case diagram of facial emotion-driven
recommendation system captures the user's face, analyzes their emotional state, and
recommends content (like music or movies) that aligns with their mood, aiming to
provide a more personalized and emotionally fulfilling user experience.
4.8.2 Class Diagram:
Class diagrams illustrate the static structure of a system by showing the classes,
attributes, methods, and relationships between them. Classes represent objects in the
system, while attributes and methods representthe properties and behaviors of those
objects. Relationships such as association, aggregation, composition, inheritance,
and dependency depict how classes are connected to each other. Class diagrams
provide a blueprint for designing and understanding thestructure of the system's
objects and their relationships.
Fig4.8 (b) Describes the Class Diagram for movie and music recommendation.
In the Fig 4.8 (b) describes the diagram depicts a system recommending media
content based on a user's emotion. The user initiates the process by requesting a
recommendation. The system then captures the user's emotion through facial detection
and analysis. Based on the detected emotion (type: String), the system recommends
music or movies. Recommendations include title, artist/genre (for music) and title,
genre, rating (for movies). The user can then choose to play the recommended media.
4.8.3 Activity Diagram

Activity diagrams visualize the flow of control or the sequence of

activities within a system or a specific use case. They represent the workflow or
business processes by showing activities, decision points, and transitions between
them. Activities represent tasks or actions performed within the system, while
transitions represent the flow of control from one activity to another. Activity diagrams
help in understanding the dynamic behavior of the systemand identifying potential
bottlenecks or inefficiencies in the workflow.
Fig 4.8 (c) Describes the Activity Diagram for movie and music recommendation.
In the Fig 4.8 (c) describes the activity diagram that depicts a movie
recommendation system based on facial expressions. It starts by capturing the user's
facial expressions. If a face is detected, the system analyzes the emotions and
recommends movies based on the user's emotional state. If no face is detected or no
emotion is recognized, the system ends the recommendation process. The user can
then select a movie from the recommendations or end the process. If a movie is
selected, the system plays the chosen movie.
4.8.4 Sequence Diagram
Sequence diagrams illustrate the interactions between objects or components

ina particular scenario or use case. They show the sequence of messages exchanged
between objects over time, represented by vertical lifelines for each object and arrows
indicating message flow. Sequence diagrams help in understanding the timing and
order of interactions between objects and can be used to identify communication
patterns and potential issues in system behavior.
Fig 4.8 (d) Describes the Sequence Diagram for movie and music recommendation.
In the Fig 4.8 (d) describes the sequence diagram shows a facial emotion-driven
recommendation system. The user interacts with the system, which captures their facial
expression. The system then detects the emotion and retrieves music and movie
recommendations based on that emotion. The user can then see the recommendations
and choose to play either the recommended music or movie.
4.8.5 Collaboration Diagram
Collaboration diagrams depict the interactions between objects or

components in a system, like sequence diagrams. However, collaboration diagrams
focus more on the structural organization ofobjects and emphasize the relationships
between them. Objects are represented as nodes, and the messages exchanged
between themare shown as links. Collaboration diagrams provide a high-level view
of how objects collaborate to achieve certain functionalities and help in
understanding system architecture.
Fig. 4.8 (e) Describes the Collaboration Diagram for movie and music
recommendation.
In the Fig 4.8 (e) describes the collaboration diagram you sent depicts a facial
emotion-driven recommendation system that recommends music and movies based on
a user's emotional state. The user initiates the process by requesting a recommendation.
The facial emotion detection system captures the user's face and analyzes their
expression to detect their emotion. The recommendation engine then retrieves the
user’s emotional profile and queries external APIs (like Spotify and IMDb) for relevant
music tracks or movies based on the user's emotion. Finally, the system presents
recommendations to the user, who can then choose to play the selected music or movie.
4.8.6 Component Diagram
Component diagrams illustrate the physical components or modules of a

systemand their dependencies. They show the structural organization of the system
by representing components, interfaces, dependencies, and relationships between
components. Components represent the physical or logical units of the system, while
interfaces represent the interactions between components.
Fig. 4.8 (f) Describes the Component Diagram for movie and music
recommendation.
In the Fig 4.8 (f) describes the component diagram depicts a facial emotion-
driven recommendation system. The user interface allows users to interact with the
system by selecting music or movie recommendations. The recommendation engine
analyzes the user’s facial expressions captured through the user interface and sends
them to the emotion detection component. This component analyzes the expression
and sends the detected emotion back to the recommendation engine. The
recommendation engine then retrieves recommendations from external services like
IMDb and Spotify and displays them to the user through the user interface.
4.8.7 Deployment Diagram
Deployment diagrams depict the physical deployment of software

components and their relationships in a hardware environment. They show how
software components are distributed across hardware nodes, such as servers,
computers, or devices, and how they interact with each other. Nodes represent the
physical devices or execution environments, while components represent the software
modules or artifacts deployed on those nodes. Deployment diagrams help in
understanding the system's deployment architecture, scalability, and distribution of
components across different hardware.
Fig 4.8 (g) Describes the Deployment Diagram for movie and music recommendation
In the Fig 4.8 (g) describes the deployment diagram shows a facial emotion-
driven recommendation system. The user device, which can be a web browser, sends
a request to the recommendation engine. The recommendation engine then
communicates with the emotion detection component to analyze the user's facial
expressions captured by the user devic
CHAPTER 5
IMPLEMENTATION
For training purposes, download the dataset face expression recognition

dataset from Kaggle. Then train the module.
5.1. Training Data
train_datagen = ImageDataGenerator(
zoom_range = 0.2,
shear_range = 0.2,
horizontal_flip=True,
rescale = 1./255
)
train_data = train_datagen.flow_from_directory(directory=
"/content/images/images/train",
target_size=(224,224),
batch_size=32,
)
train_data.class_indices
5.2 Visualize the Images in Training Data
def plotImages(img_arr, label):

count = 0
for im, l in zip(img_arr,label) :
plt.imshow(im)
plt.title(im.shape)
plt.axis = False
plt.show()
count += 1
if count == 10:
break
#
# function call to plot the images
plotImages(t_img, label)
5.3 Using Mobilenet for Pre-Trained Weights
MobileNet, a computer vision model developed for training classifiers. It employs

depth wise convolutions to dramatically minimize the amount of parameters compared to
other networks, yielding a lightweight deep neural network. MobileNet is Tensorflow's
initial mobile computer vision model.
base_model = MobileNet( input_shape=(224,224,3), include_top= False )

for layer in base_model.layers:
layer.trainable = False
x = Flatten()(base_model.output)
x = Dense(units=7 , activation='softmax' )(x)
Creating our required model which is used to take the faces in the training dataset.
hist = model.fit_generator(train_data,
steps_per_epoch= 10,
epochs= 100,
validation_data= val_data,
validation_steps= 10)
5.4 Save the Model in h5 format
model.save('final_model.h5')
path = "/content/images/images/validation/angry/10052.jpg"
img = load_img(path, target_size=(224,224) )
i = img_to_array(img)/255
input_arr = np.array([i])
input_arr.shape
pred = np.argmax(model.predict(input_arr))
print(f" the image is of {op[pred]}")
# to display the image

plt.imshow(input_arr[0])
plt.title("input image")
plt.show()
Testing the model using the images:
op = dict(zip( train_data.class_indices.values(), train_data.class_indices.keys()))

path = "/content/images/images/validation/angry/10052.jpg"
img = load_img(path, target_size=(224,224) )
i = img_to_array(img)/255
input_arr = np.array([i])
input_arr.shape
pred = np.argmax(model.predict(input_arr))
print(f" the image is of {op[pred]}")
# to display the image
plt.imshow(input_arr[0])
plt.title("input image")
plt.show()
CHAPTER 6
CODING SPECIFICATIONS
6.1. app.py code
from future import division, print_function
#import sys
import os
import cv2
#import re
import numpy as np
import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.keras.preprocessing import image
from flask import Flask, request, render_template
from werkzeug.utils import secure_filename
import statistics as st
app = Flask( name )
@app.route("/")
def home():
return render_template("index1.html")
@app.route('/camera', methods = ['GET', 'POST'])
def camera():
i=0
GR_dict={0:(0,255,0),1:(0,0,255)}
model = tf.keras.models.load_model('final_model.h5')
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
output=[]
cap = cv2.VideoCapture(0)
while (i<=30):
ret, img = cap.read()
faces = face_cascade.detectMultiScale(img,1.05,5)
for x,y,w,h in faces:
face_img = img[y:y+h,x:x+w]
resized = cv2.resize(face_img,(224,224))
reshaped=resized.reshape(1, 224,224,3)/255
predictions = model.predict(reshaped)
max_index = np.argmax(predictions[0])
emotions = ('angry', 'disgust', 'fear', 'happy', 'sad', 'neutral', 'surprise')
predicted_emotion = emotions[max_index]
output.append(predicted_emotion)
cv2.rectangle(img,(x,y),(x+w,y+h),GR_dict[1],2)
cv2.rectangle(img,(x,y-40),(x+w,y),GR_dict[1],-1)
cv2.putText(img, predicted_emotion, (x, y-

10),cv2.FONT_HERSHEY_SIMPLEX,0.8,(255,255,255),2)
i = i+1
cv2.imshow('LIVE', img)
key = cv2.waitKey(1)
if key == 27:
cap.release()
cv2.destroyAllWindows()
break
print(output)
cap.release()
cv2.destroyAllWindows()
final_output1 = st.mode(output)
return render_template("buttons.html",final_output=final_output1)
@app.route('/templates/buttons', methods = ['GET','POST'])
def buttons():
return render_template("buttons.html")
@app.route('/movies/surprise', methods = ['GET', 'POST'])
def moviesSurprise():
return render_template("moviesSurprise.html")
@app.route('/movies/angry', methods = ['GET', 'POST'])
def moviesAngry():
return render_template("moviesAngry.html")
@app.route('/movies/sad', methods = ['GET', 'POST'])
def moviesSad():
return render_template("moviesSad.html")
@app.route('/movies/disgust', methods = ['GET', 'POST'])
def moviesDisgust():
return render_template("moviesDisgust.html")
@app.route('/movies/happy', methods = ['GET', 'POST'])
def moviesHappy():
return render_template("moviesHappy.html")
@app.route('/movies/fear', methods = ['GET', 'POST'])
def moviesFear():
return render_template("moviesFear.html")
@app.route('/movies/neutral', methods = ['GET', 'POST'])
def moviesNeutral():
return render_template("moviesNeutral.html")
@app.route('/songs/surprise', methods = ['GET', 'POST'])
def songsSurprise():
return render_template("songsSurprise.html")
@app.route('/songs/angry', methods = ['GET', 'POST'])
def songsAngry():
return render_template("songsAngry.html")
@app.route('/songs/sad', methods = ['GET', 'POST'])
def songsSad():
return render_template("songsSad.html")
@app.route('/songs/disgust', methods = ['GET', 'POST'])
def songsDisgust():
return render_template("songsDisgust.html")
@app.route('/songs/happy', methods = ['GET', 'POST'])
def songsHappy():
return render_template("songsHappy.html")
@app.route('/songs/fear', methods = ['GET', 'POST'])
def songsFear():
return render_template("songsFear.html")
@app.route('/songs/neutral', methods = ['GET', 'POST'])
def songsNeutral():
return render_template("songsSad.html")
@app.route('/templates/join_page', methods = ['GET', 'POST'])
def join():
return render_template("join_page.html")
if name == " main ":
app.run(debug=True)
CHAPTER 7
TESTING AND VALIDATION
Testing and validation are crucial phases in the development of any

recommendation system, especially one as complex and multifaceted as a facial
emotion-driven music and movie recommendation system. To ensure the accuracy,
reliability, and effectiveness of the system, rigorous testing procedures and validation
techniques must be employed. One aspect of testing involves assessing the accuracy of
the emotion detection module. This typically involves using labeled datasets of facial
expressions to evaluate the system's ability to correctly identify and classify emotions.
Through techniques such as cross-validation and confusion matrix analysis, developers
can determine the system's precision, recall, and F1 score for each emotion category.
Additionally, real-world testing scenarios may involve capturing facial expressions
from diverse users in different environments to evaluate the system's robustness and
generalization capabilities.
Another critical aspect of testing pertains to the recommendation algorithms

used to suggest music tracks and movie titles based on users' emotional states. This
involves evaluating the relevance and appropriateness of the recommendations
generated by the system. Test scenarios may involve presenting users with various
emotional stimuli and assessing whether the recommended content aligns with their
perceived emotional states. Additionally, user feedback and satisfaction surveys can
provide valuable insights into the quality of the recommendations and areas for
improvement.
7.1 Testing
Testing strategies for the facial emotion-driven music and movie

recommendation system encompass a comprehensive approach aimed at ensuring the
reliability, accuracy, and effectiveness of the system across various dimensions. One
key strategy involves the implementation of unit testing, where individual components
of the system, such as the emotion detection module, recommendation algorithms, and
API interactions, are tested in isolation. This allows developers to verify the
functionality of each component and identify any bugs or issues early in the
development process.
Additionally, integration testing is essential to evaluate the interaction and

compatibility of different system components when combined. By simulating real-
world usage scenarios and testing the integration points between modules, developers
can ensure seamless communication and data flow throughout the system. Integration
testing also enables the validation of end-to-end functionality, including the process of
capturing facial expressions, detecting emotions, retrieving content recommendations,
and presenting them to users.
Furthermore, usability testing plays a crucial role in assessing the user

experience and interface design of the recommendation system. This involves
soliciting feedback from end-users through user testing sessions, surveys, and usability
studies. By observing how users interact with the system and gathering their feedback
on aspects such as interface intuitiveness, content relevance, and overall satisfaction,
developers can identify usability issues and make iterative improvements to enhance
the user experience.
Performance testing is another critical aspect of the testing strategy, focusing

on evaluating the system's speed, scalability, and resource efficiency under various
load conditions. This includes stress testing to assess the system's stability and
responsiveness under heavy usage, as well as scalability testing to determine its ability
to handle increasing user loads without degradation in performance. Performance
testing helps identify bottlenecks, optimize system resources, and ensure that the
recommendation system can effectively serve its intended user base.
Finally, validation testing is conducted to verify that the recommendation

system meets its specified requirements and objectives. This involves comparing the
system's performance against predefined criteria, such as accuracy metrics for emotion
detection, relevance metrics for content recommendations, and user satisfaction
metrics. Validation testing helps confirm that the system delivers the desired outcomes
and meets the needs of its users effectively.
Overall, by employing a combination of unit testing, integration testing,
usability testing, performance testing, and validation testing, developers can ensure the
robustness, reliability, and usability of the facial emotion-driven music and movie
recommendation system, ultimately delivering a high-quality and engaging user
experience.
7.2 Validation
Validation is a crucial aspect of our facial emotion-driven music and movie

recommendation project, ensuring the reliability, accuracy, and effectiveness of the
system. One key aspect of validation revolves around the accuracy of our emotion
detection module. Through meticulous validation procedures, we assess the module's
ability to precisely identify and classify users' emotional states based on their facial
expressions. This validation process entails comparing the system's predictions against
ground truth labels obtained from meticulously labeled datasets of facial expressions.
Metrics like precision, recall, and F1 score are computed to evaluate the module's
performance across different emotion categories, providing insights into its
effectiveness in discerning subtle emotional nuances.
Furthermore, the validation extends to the recommendation algorithms

responsible for suggesting music tracks and movie titles based on users' emotional
states. Here, validation involves presenting users with recommended content and
soliciting feedback on its perceived relevance and alignment with their emotional
states. Through user satisfaction surveys and preference ratings, we gain valuable
insights into the quality of the recommendations and their impact on user engagement
and satisfaction.
Overall, through a comprehensive validation approach encompassing accuracy

validation of emotion detection, relevance validation of content recommendations,
usability testing, and performance testing, we ensure that our facial emotion-driven
music and movie recommendation system delivers a reliable, accurate, and user-
friendly experience that meets the needs and expectations of our users.
In our facial emotion-driven music and movie recommendation project, we

employ various types of validation to ensure the accuracy, reliability, and effectiveness
of the system. These validation techniques encompass both technical validation of the
system's algorithms and user-centric validation of the overall user experience. Here are
the types of validation used:
7.2.1 Emotion Detection Validation

This type of validation involves assessing the accuracy and effectiveness of
the emotion detection module. Through meticulous testing and validation procedures,
we verify the module's ability to accurately identify and classify users' emotional states
based on their facial expressions. Techniques such as precision, recall, and F1 score
calculation are used to evaluate the module's performance across different emotion
categories.
7.2.2 Content Recommendation Validation

Content recommendation validation focuses on evaluating the relevance and
appropriateness of the recommended music tracks and movie titles. This validation
process involves presenting users with recommended content and gathering feedback
on its perceived alignment with their emotional states and preferences. User
satisfaction surveys and preference ratings are utilized to assess the quality of the
recommendations.
7.2.3 Usability Testing

Usability testing is conducted to evaluate the user experience and interface
design of the recommendation system. This type of validation involves engaging end-
users in usability testing sessions, surveys, and usability studies to assess the system's
intuitiveness, ease of navigation, and overall user satisfaction. Feedback from users
helps identify usability issues and inform iterative improvements to the interface
design.
7.2.4 Performance Testing

Performance testing assesses the system's speed, scalability, and resource
efficiency under various load conditions. Stress testing evaluates the system's stability
and responsiveness under heavy usage, while scalability testing assesses its ability to
handle increasing user loads without degradation in performance. Performance metrics
such as response time, throughput, and resource utilization are measured to optimize
system performance.
7.2.5 Validation against Benchmarks

This type of validation involves comparing the performance of our
recommendation system against established benchmarks and industry standards. By
benchmarking our system's accuracy, effectiveness, and user satisfaction metrics
against existing recommendation systems, we gain insights into its competitive
position and identify areas for improvement.
SCREENSHOTS
Once link gets run then an interface visible which curates the option for Click
"Let's Start" to activate your live camera and unlock a world of immersive multimedia
experiences. With just a glance, our cutting-edge facial emotion analysis technology
will detect your current mood.
Fig 8.1 (a) Describes the Opening Interface of the System
In the Fig 8.1 (a) describes button in the center of the screen, a sleek interface
displays the title "Find Your Next Vibe." Below, rows of captivating movie posters and
music album covers entice you to explore. On the left, user profiles allow you to
personalize your experience. Feeling indecisive? A prominent "Lets Start" button
beckons you to discover your perfect movie and music pairing.
It detects the emotion as given below with the help of Haar Cascade Classifier
Algorithm for the frontal face. The facial emotion-driven music and movie
recommendation project employs the Haar Cascade Classifier Algorithm for frontal
face detection, a key component in the emotion detection process. This algorithm is
instrumental in identifying and localizing frontal faces within images or video streams,
providing a foundational step for subsequent emotion recognition.
The model was trained using a diverse dataset comprising facial expressions
representing seven primary emotions: anger, sadness, disgust, happiness, surprise,
neutral, and fear. Through meticulous training and validation, the model demonstrates
its capability to accurately predict users'. Through meticulous training and validation
procedures, the model undergoes rigorous refinement to ensure its capability to
accurately predict users' emotions. This involves iterative adjustments to the model's
parameters and architecture, as well as thorough evaluation against validation datasets
to assess its performance across different scenarios and variations in facial expressions.
Fig 8.1 (b) Describes the Emotion detection interface (Sad).
Figure 8.1 (b) illustrates the Emotion Detection interface focused on detecting
sadness. This interface serves as a crucial component within our facial emotion-driven
music and movie recommendation system, enabling real-time analysis of users' facial
expressions to identify the emotion of sadness.
Fig 8.1 (c) Describes the Emotion detection interface (Angry).
Fig 8.1 (c) illustrates the Emotion Detection interface focused on detecting
furious. This interface serves as a crucial component within our facial emotion-driven
music and movie recommendation system, enabling real-time analysis of users' facial
expressions to identify the emotion of angriness.
Fig 8.1 (d) Describes the Emotion detection interface (Happy).
Figure 8.1 (d) illustrates the Emotion Detection interface focused on detecting
happiness. This interface serves as a crucial component within our facial emotion-
driven music and movie recommendation system, enabling real-time analysis of users'
facial expressions to identify the emotion of happiness.
After successfully detecting the user's emotional state, the recommendation

system seamlessly transitions to the interface for suggesting music and movies tailored
to the detected emotions. In the music interface, users are presented with a curated
selection of music tracks, playlists, or albums that align with their current emotional
state. Leveraging the extensive catalog of Spotify, the interface offers a diverse range
of music genres and styles, ensuring that users find content that resonates with their
mood. Whether they seek upbeat and energetic tunes to match moments of happiness or
soothing melodies to soothe feelings of sadness, the music interface provides
personalized recommendations to enhance the user's emotionalexperience.
Upon completion of the training process, the model is equipped with the ability
to analyze facial expressions captured in real-time through the Haar Cascade Classifier
Algorithm. By leveraging the knowledge acquired during training, the model can
accurately identify and classify the emotional states of users based on their facial
expressions.
Simultaneously, the movie interface offers a curated list of movie

recommendations sourced from IMDb, each carefully selected to match the user's
detected emotional state. From heartwarming comedies to gripping thrillers, the
interface presents a diverse array of cinematic experiences tailored to the user's mood.
The user interface for both music and movies is designed with simplicity and
ease of use in mind, allowing users to effortlessly navigate through the recommended
content and explore additional options. Interactive features such as search functionality,
genre filters, and personalized recommendations enhance the user experience, enabling
users to discover new and relevant content aligned with their emotional states.
Furthermore, the interface may incorporate user feedback mechanisms to refine
recommendations over time, ensuring that the system continuously adapts to users'
evolving preferences and emotional states. Through this process, users can enjoy a more
immersive and emotionally resonant entertainment experience, enriched by content
recommendations that align with their current moods and preferences.
Fig 8.1 (e) Describes the Selection interface for detected emotion.
In the Fig 8.1 (e) depicts the Selection interface designed for the detected
emotion within our facial emotion-driven music and movie recommendation system.
This interface serves as a pivotal point where users are presented with curated
recommendations tailored to their detected emotional state. By leveraging advanced
emotion detection algorithms and user preferences, the system offers a selection of
music tracks and movie titles aligned with the user's current emotional state, whether
it be happiness, sadness, excitement, or any other emotion.
After the emotion detection phase, the recommendation system seamlessly

transitions to the music and movie recommendation interfaces, leveraging Spotify and
IMDb to provide tailored content suggestions based on users' emotional states.
In the music recommendation interface, users encounter a meticulously

curated selection of music tracks sourced from Spotify's vast and diverse library.
Drawing upon the emotional cues detected earlier in the process, the system adeptly
retrieves playlists and individual songs tailored to align with users' current emotional
states. This personalized approach ensures that users are presented with music options
that not only cater to their specific moods but also evoke the desired emotional
responses. Offering users a rich and varied assortment of genres, artists, and playlists
to explore, the interface provides ample opportunities for users to discover new music
that resonates with their feelings. By seamlessly integrating with Spotify, a platform
widely recognized and utilized by music enthusiasts worldwide, the interface delivers
a convenient and familiar environment for users to immerse themselves in a musical
journey curated to their emotional inclinations
Fig 8.1 (f) Describes the List of Playlists from Spotify
In the Fig 8.1 (f) describes the interface displays a comprehensive list of
playlists sourced from Spotify, an integral component of our facial emotion-driven
music recommendation system. This feature provides users with a diverse selection of
curated playlists tailored to their emotional states and preferences.
Fig 8.1 (g) Describes the Spotify interface for playing songs.
Fig 8.1 (g) illustrates the Spotify interface dedicated to playing songs within
our facial emotion-driven music recommendation system. This interface serves as a
pivotal component, seamlessly integrating with Spotify's extensive music library to
provide users with personalized music recommendations based on their emotional

states.
In the movie recommendation interface, users are provided with comprehensive

information about suggested movies sourced from IMDb. Leveraging the emotional
analysis conducted earlier, the system retrieves movie titles, synopses, genres, ratings,
cast, and crew information that match users' emotional states.
Users can explore detailed descriptions and reviews, enabling informed

decisions about their movie selections. By seamlessly integrating with IMDb, the
interface offers users access to a vast repository of cinematic content, ensuring a
personalized and enriching movie-watching experience.
Figure 8.1 (h) Describes the List of Movies based on selection.

Fig 8.1 (h) showcases the List of Movies based on selection, representing a
pivotal component within our facial emotion-driven music and movie recommendation
system. This interface provides users with a curated selection of movie titles tailored
to their emotional states and preferences. Leveraging advanced algorithms and data
from IMDb (Internet Movie Database), the system offers a diverse range of cinematic
experiences aligned with users' current moods.
Fig 8.1 (i) Describes the IMDB interface.

Figure 8.1 (i) presents the IMDB interface within our facial emotion-driven
music and movie recommendation system. This interface serves as a gateway for users
to explore movie recommendations sourced from the Internet Movie Database
(IMDB). By leveraging the vast repository of movie information available on IMDB,
users can access comprehensive details about various movie titles, including synopses,
genres, ratings, cast, and crew information.
Together, the music and movie recommendation interfaces offer users a

unified and immersive entertainment experience. By leveraging Spotify and IMDb's
extensive databases, the system provides users with a wide range of music and movie
options tailored to their emotional states and preferences. Whether users seek to uplift
their spirits with upbeat music or unwind with a captivating film, the recommendation
system ensures they have access to relevant and engaging content. This seamless
integration of music and movie recommendations enhances user satisfaction and
engagement, fostering a more enjoyable and personalized multimedia consumption
experience.
CONCLUSION AND FUTURE WORK
Conclusion
In conclusion, the advancement of facial emotion driven music and movie

recommendation ecosystem represents a significant advancement in the field of
multimedia content delivery. By integrating state-of-the-art facial emotion analysis
techniques with established recommendation algorithms, this project aims to offer users
a personalized and emotionally resonant entertainment experience. Through rigorous
research and implementation, the system has demonstrated its ability to accurately
detect users' emotional states from facial expressions and provide tailored
recommendations for music tracks and movie titles. Leveraging the vast repositories of
Spotify and IMDb, users are presented with a diverse selection of content that aligns
with their current moods and preferences.
Through rigorous research and implementation efforts, the system has

demonstrated its capability to accurately detect users' emotional states from facial
expressions. Leveraging this information, the system provides tailored
recommendations for music tracks and movie titles that align with users' current moods
and preferences. By tapping into the extensive repositories of Spotify and IMDb, users
are presented with a diverse selection of content curated to evoke specific emotions
and enhance their overall entertainment experience.
The project has also tackled various challenges inherent in developing such a
system, including ensuring data quality and diversity, enabling real-time processing,
addressing privacy considerations, and seamlessly integrating with external platforms.
Through interdisciplinary collaboration and innovative solutions, these obstacles have
been overcome, laying the groundwork for a robust and scalable recommendation
system that prioritizes user satisfaction and engagement.
In summary, the facial emotion-driven music and movie recommendation

system presented in this project represent a significant step towards personalized and
emotionally resonant content delivery. By harnessing the power of facial expression
analysis and leveraging vast content repositories, the system offers users a unique and
enriching entertainment experience tailored to their individual emotions and

preferences.
Future Work
Future work for the facial emotion-driven music and movie recommendation project
could involve several avenues of research and development to enhance the system's
capabilities, user experience, and overall effectiveness. Here are some potential areas
for future exploration:
1. Advanced Emotion Detection Techniques: Investigate and implement more

advanced facial emotion detection techniques, such as facial action unit detection,
deep learning-based approaches, or multimodal emotion recognition combining
facial expressions with other modalities like voice or physiological signals. This
could improve the accuracy and granularity of emotion detection, leading to more
precise recommendations.
2. Dynamic Content Adaptation: Explore methods for dynamically adapting content
recommendations based on real-time changes in users' emotional states or
contextual factors. This could involve continuously monitoring users' emotions
during content consumption and adjusting recommendations accordingly to
maintain relevance and engagement.
3. Integration of User Feedback: Incorporate mechanisms for collecting and
incorporating user feedback to refine the recommendation algorithms over time.
User interactions, ratings, and preferences could be leveraged to continuously
improve the system's ability to deliver personalized and satisfying
recommendations.
4. Multi-Modal Recommendations: Extend the recommendation system to provide
recommendations across multiple modalities, such as recommending music
playlists or movie genres based on users' emotions and preferences. This could offer
users a more diverse and immersive entertainment experience by catering to
different preferences and interests.
Another avenue for future exploration lies in the realm of dynamic content adaptation.
Here, the focus would be on developing mechanisms for dynamically adjusting content
recommendations in response to real-time changes in users’ motional states or
contextual factors. This continuous monitoring of users' emotions during content

consumption could enable the system to adapt recommendations on- the-fly, ensuring
that the content remains relevant and engaging to users' evolving emotional states.
REFERENCES
[1] Ramya Ramanathan, Radha Kumaran, Ram Rohan R, Rajat Gupta, and Vishalakshi
Prabhu, an intelligent music player based on emotion recognition.
[2] Shlok Gilda, Husain Zafar, Chintan Soni, Kshitija Waghurdekar, Smart musicplayer
integrating facial emotion recognition and music mood recommendation. India,
(IEEE),2017.
[3] Deger Ayata, Yusuf Yaslan and Mustafa E. Kamasak, Emotion-based music
recommendation system using wearable physiological sensors, IEEE transactions on
consumer electronics, vol. 14.
[4] Ahlam Alrihail, Alaa Alsaedi, Kholood Albalawi, Liyakathunisa Syed, Music
recommender system for users based on emotion detection through facial features.
[5] Research Prediction Competition, Challenges in representation learning: facial
expression recognition challenges, Learn facial expression from an image,
(KAGGLE).
[6] Preema J.S, Rajashree, Sahana M, Savitri H, Review on facial expression-based music
player, International Journal of Engineering Research & Technology (IJERT), ISSN-
2278-0181, Volume 6, Issue 15, 2018.
[7] AYUSH Guidel, Birat Sapkota, Krishna Sapkota, Music recommendation by facial
analysis, February 17, 2020.
[8] CH.Sadhvika, Gutta.Abigna, P.Srinivasreddy, Emotion-based music recommendation
system, (JETIR) Volume 7, Is-sue 4, April 2020.
[9] Zhuwei Qin, Fuxun Yu, Chenchen Liu, Xiang Chen. A survey of convolutional neural
network visualization methods. Mathematical Foundations of Computing, May 2018.
[10] Ahmed Hamdy AlDeeb, Emotion- Based Music Player Emotion Detection from Live
Camera, ResearchGate, June 2019.
[11] Frans Norden and Filip von Reis Marlevi, A Comparative Analysis of Machine
Learning Algorithms in Binary Facial Expression Recognition, TRITA-EECS-EX-
2019:143.
[12] P. Singhal, P. Singh and A. Vidyarthi (2020) Interpretation and localization of
Thorax diseases using DCNN in Chest X-Ray. Journal of Informatics Electrical and
Elecrtonics Engineering,1(1), 1, 1-7
[13] M. Vinny, P. Singh (2020) Review on the Artificial Brain Technology: BlueBrain.
Journal of Informatics Electrical and Electronics Engineering,1(1), 3, 1-11.
[14] N. K. Rao, N. P. Challa, S. S. Chakravarthi and R. Ranjana, "Movie
Recommendation System using Machine Learning," 2022.
[15] Chillara, S., Kavitha, A. S., Neginhal, S. A., Haldia, S., & Vidyullatha, K. S. (2019).
Music genre classification using machine learning algorithms: a comparison.
[16] J. Singh and V. K. Bohat, "Neural Network Model for Recommending Music Based
on Music Genres," 2021 International Conference on Computer Communication and
Informatics (ICCCI), Coimbatore, India, 2021, pp. 1-6, doi:
10.1109/ICCCI50826.2021.9402621.
[17] Reddy, S. R. S., Nalluri, S., Kunisetti, S., Ashok, S., & Venkatesh, B. (2019).
Content-based movie recommendation system using genre correlation. In Smart
Intelligent Computing and Applications: Proceedings of the Second International
Conference on SCI 2018, Volume 2 (pp. 391-397). Springer Singapore.Nici.
[18] S. Chawla, S. Gupta and R. Majumdar, "Movie Recommendation Models Using
Machine Learning," 2021 5th International Conference on Information Systems and
Computer Networks (ISCON), Mathura, India, 2021, pp. 1-6, doi:
10.1109/ISCON52037.2021.9702472.
[19] . J. Singh and V. K. Bohat, "Neural Network Model for Recommending Music Based
on Music Genres," 2021 International Conference on Computer Communication and
Informatics (ICCCI), Coimbatore, India, 2021, pp. 1-6, doi:
10.1109/ICCCI50826.2021.9402621.
[20] A. Singh and S. Gupta, "Recommendation System Algorithms For Music Therapy,"
2023 13th International Conference on Cloud Computing, Data Science &
Engineering (Confluence), Noida, India, 2023, pp. 138 143, doi:
10.1109/Confluence56041.2023.10048894

Context (3) (1) (AutoRecovered) (1) ... - Removed

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Context (3) (1) (AutoRecovered) (1) ... - Removed

Uploaded by

Copyright:

Available Formats

Moodify: A Movie & Music Recommendation System Using Emotion Detection

In an increasingly digitalized world where technology pervades nearly every

We went through a lot of different research papers to understand all the

2.1.1 Mood Indicator: Music and Movie Recommendation System Using

This paper analyses, compares, and contrasts the technological problems,

In today's digital era, the demand for personalized content recommendation

The "Mood Indicator" recommendation system prioritizes user experience and

In conclusion, the "Mood Indicator: Music and Movie Recommendation System

2.1.2 Music Recommendation Based on Face Emotion Recognition

compared to recent research work.

EmoTunes also addresses the challenge of discretizing multi-user emotions by

Beyond its technical capabilities, EmoTunes holds significant potential for

2.1.3 Music & Movie Recommendation System By FacialEmotions

In a digital age marked by the exponential growth of entertainment content, the

Our music and movie recommendation system based on emotions represents a

The inclusion of an integrated camera for facial expression capturesignificantly

revolutionize the entertainment experience by aligning content with human emotions.

Addressing this demand, this research project introduces an innovative

including the Haar Cascade algorithm and Convolutional Neural Networks

Moreover, the inclusion of an integrated camera for facial expression capture

By merging emotions with state-of-the-art technology, this recommendation

2.1.4 Emotional Detection and Music Recommendation System based on User

Music and videos represent fertile ground for recommendation systems, as

2.1.5 Movie & Music Recommendation System Based on Facial Expressions

The emotions of an individual will be captured using an inbuilt webcam. The

Usually, the weighted-SoftMax loss approach is used to handle this problem by

EXISTING AND PROPOSED SYSTEM

3.1 Existing System

Existing music and movie recommendation systems predominantly rely on

In conclusion, while existing music recommendation systems leveraging CNNs

recommendations for users.

3.1.1 Disadvantages of Existing System

3.2 Proposed System

This paper proposes an innovative approach to enhance music and movie

3.3 Flowchart of the System

Fig 3.3 Describes the Flowchart of System Architecture

An innovative approach to enhance music and movie recommendation systems

By seamlessly integrating emotion-driven content recommendations from

The proposed approach represents a groundbreaking advancement in music and

Following face detection, the captured frame undergoes preprocessing to

Upon successful emotion recognition, the system proceeds to recommend music

Moreover, the system's ability to dynamically adjust recommendations based on

In conclusion, the facial emotion-driven music and movie recommendation

3.4.1 Software Requirements

 Python 3. x installed on the system.

 Anaconda navigator latest version installed on the system.

 VS code installed on the system.

 Download dataset face-expression-recognition-dataset from Kaggle.

 Data files of music & movies in excel format.

 Required Python libraries installed, including:

3.4.2 Hardware Requirements

Designing a music recommendation system based on facial emotions involves

4.1 Capture Image

Fig. 4.1 Describes the Design Phases of System

Collecting photos is an important step in facial expression recognition; with

Facial emotion recognition systems are designed to analyze various aspects of

OpenCV encompasses a wide array of image processing capabilities, among

4.2 Face Identification Using Deep CNN

DCNNs are a sort of deep learning algorithm that is well-suited to image

Moreover, DCNNs excel at recognizing multiple faces within an image, a task

In summary, face identification using Deep Convolutional Neural Networks

4.3 Features Extraction

The process of Haar cascade feature extraction involves scanning an image at