Analysis of women safety in Indian cities using machine learning on

tweets

Abstract

Women and girls have been experiencing a lot of violence and harassment in public places in
various cities starting from stalking and leading to sexual harassment or sexual assault. This
research paper basically focuses on the role of social media in promoting the safety of women in
Indian cities with special reference to the role of social media websites and applications
including Twitter platform Facebook and Instagram. This paper also focuses on how a sense of
responsibility on part of Indian society can be developed the common Indian people so that we
should focus on the safety of women surrounding them. Tweets on Twitter which usually
contains images and text and also written messages and quotes which focus on the safety of
women in Indian cities can be used to read a message amongst the Indian Youth Culture and
educate people to take strict action and punish those who harass the women. Twitter and other
Twitter handles which include hash tag messages that are widely spread across the whole globe
sir as a platform for women to express their views about how they feel while we go out for work
or travel in a public transport and what is the state of their mind when they are surrounded by
unknown men and whether these women feel safe or not?
Introduction

Twitter has emerged as a major micro-blogging website, having over 100 million users
generating over 500 million tweets every day. With such large audience, Twitter has consistently
attracted users to convey their opinions and perspective about any issue, brand, company or any
other topic of interest. Due to this reason, Twitter is used as an informative source by many
organizations, institutions and companies.
On Twitter, users are allowed to share their opinions in the form of tweets, using only 140
characters. This leads to people compacting their statements by using slang, abbreviations,
emoticons, short forms etc. Along with this, people convey their opinions by using sarcasm and
polysemy.
Hence it is justified to term the Twitter language as unstructured.
In order to extract sentiment from tweets, sentiment analysis is used. The results from this can be
used in many areas like analyzing and monitoring changes of sentiment with an event,
sentiments regarding a particular brand or release of a particular product, analyzing public view
of government policies etc.
A lot of research has been done on Twitter data in order to classify the tweets and analyze the
results. In this paper we aim to review of some researches in this domain and study how to
perform sentiment analysis on Twitter data using Python. The scope of this paper is limited to
that of the machine learning models and we show the comparison of efficiencies of these models
with one another.
There are certain types of harassment and Violence that are very aggressive including staring and
passing comments and these unacceptable practices are usually seen as a normal part of the
urban life. There have been several studies that have been conducted in cities across India and
women report similar type of sexual harassment and passing off comments by other unknown
people. The study that was conducted across most popular Metropolitan cities of India including
Delhi, Mumbai and Pune, it was shown that 60 % of the women feel unsafe while going out to
work or while travelling in public transport.

Women have the right to the city which means that they can go freely whenever they want
whether it be too an Educational Institute, or any other place women want to go. But women feel
that they are unsafe in places like malls, shopping malls on their way to their job location
because of the several unknown Eyes body shaming and harassing these women point Safety or
lack of concrete consequences in the life of women is the main reason of harassment of girls.
There are instances when the harassment of girls was done by their neighbors while they were on
the way to school or there was a lack of safety that created a sense of fear in the minds of small
girls who throughout their lifetime suffer due to that one instance that happened in their lives
where they were forced to do something unacceptable or was sexually harassed by one of their
own neighbor or any other unknown person.

Safest cities approach women safety from a perspective of women rights to the affect the city
without fear of violence or sexual harassment. Rather than imposing restrictions on women that
society usually imposes it is the duty of society to imprecise the need of protection of women
and also recognizes that women and girls also have a right same as men have to be safe in the
City.

Analysis of twitter texts collection also includes the name of people and name of women who
stand up against sexual harassment and unethical behavior of men in Indian cities which make
them uncomfortable to walk freely. The data set that was obtained through Twitter about the
status of women safety in Indian society was for the processed through machine learning
algorithms for the purpose of smoothening the data by removing zero values and using Laplace
and porter’s theory is to developer method of analyzation of data and remove retweet and
redundant data from the data set that is obtained so that a clear and original view of safety status
of women in
Indian society is obtained.

People often express their views freely on social media about what they feel about the Indian
society and the politicians that claim that Indian cities are safe for women. On social media
websites people can freely express their view point and women can share their experiences
where they have faced sexual harassment or where we would have fight back against the sexual
harassment that was imposed on them.
The tweets about safety of women and stories of standing up against sexual harassment further
motivates other women data on the same social media website or application like Twitter. Other
women share these messages and tweets which further motivates other 5 men or 10 women to
stand up and raise a voice against people who have made Indian cities and unsafe place for the
women. In the recent years a large number of people have been attracted towards social media
platforms like Facebook, Twitter and Instagram point and most of the people are using it to
express their emotions and also their opinions about what they think about the Indian cities and
Indian society. There are several method of sentiment that can be categorized like machine
learning hybrid and lexicon-based learning.

Also there are another categorization presented with categories of statistical, knowledge-based
and age wise differentiation approaches. It is a common practice to extract the information from
the data that is available on social networking through procedures of data extraction, data
analysis and data interpretation methods. The accuracy of the Twitter analysis and prediction can
be obtained by the use of behavioral analysis on the basis of social networks.

Data Description
Twitter is the most popular social networking platform over which people express their views
and opinions about various trending topics with the help of short messages called as tweets.
These tweets are text messages with maximum length of 140 characters, because of this short
message service people make the use of acronyms, emoticons and other special characters with
special meanings. A brief description of all the terminologies associated with tweets.
Emoticons: A replacement to a group of words with a pictorial representation that express human
emotions like humor, temper, happiness, sadness etc.
Target: The tweet messages are sometimes targeted to a user letting them know about something
or expressing an opinion about the target user. The “@” symbol is used within the tweet to
address the target.
Hashtags: Hashtags within the tweets are used to make the tweet visible under a twitter topic,
there can be multiple hashtags within a tweet, making a tweet visible under multiple topics. If we
consider the tweets a source of information, we need to materialize this information so that it can
be extract some gains out of it.
ABOUT SENTIMENT ANALYSIS

Sentiment analysis is a process of deriving sentiment of a particular statement or sentence. It’s a

classification technique which derives opinion from the tweets and formulates a sentiment and
on the basis of which, sentiment classification is performed.
Sentiments are subjective to the topic of interest. We are required to formulate that what kind of
features will decide for the sentiment it embodies.
In the programming model, sentiment we refer to, is class of entities that the person performing
sentiment analysis wants to find in the tweets. The dimension of the sentiment class is crucial
factor in deciding the efficiency of the model.
For example, we can have two-class tweet sentiment classification (positive and negative) or
three class tweet sentiment classification (positive, negative and neutral).
Sentiment analysis approaches can be broadly categorized in two classes – lexicon based and
machine learning based.
Lexicon based approach is unsupervised as it proposes to perform analysis using lexicons and a
scoring method to evaluate opinions. Whereas machine learning approach involves use of feature
extraction and training the model using feature set and some dataset.
The basic steps for performing sentiment analysis includes data collection, pre-processing of
data, feature extraction, selecting baseline features, sentiment detection and performing
classification either using simple computation or else machine learning approaches.

Process Involved in Analyzing sentiment data

The complex process of measuring the sentiments of the tweets involves 5 major steps:
1) Data Extraction: This step involved in sentiment analysis consists of gathering the data from
social network site twitter source which helps in extracting the tweet text but also provides extra
information about the tweets like likes and retweets.
2) Text Cleaning: After the tweets for a topic are extracted before passing it to the classifier, we
need to clean the dataset to remove emoji’s, stop words so that the non-textual content not
pertinent to the analysis is identified and removed.
3) Sentiment Analysis: Once the data is cleansed it’s ready for classification, into positive,
negative and neutral tweets. There are various approaches to sentiment analysis like Machine
Learning, Lexicon based and Hybrid approach. Also there are some other approaches like
Natural Language Processing and Nero Linguistic Programming. Machine Learning involves a
training dataset and a testing dataset, where we used the training data and train the classifier
using one of many algorithms like Bayesian Networks, Naïve Bayes classification, Maximum
Entropy, Networks Support Vector Machine. The testing dataset is used to test the classifier for
its accuracy in the tweets. Lexicon approach does not use any training dataset, it makes the use
of an inbuilt dictionary where all words are associated with a human sentiment. The Hybrid
combines the Machine Learning and the Lexicon approach to improve performance of the
sentiment classifier.
4) Presentation Output: sentiment-analysis is a main aim to generate meaningful information of
raw data. After the analysis is complete, we can perform visualizations like creating bar graphs,
Time series and pie charts. Bar graphs can be used to measure the sentiment the tweet as
positive, negative and neutral. Time Series can be used to measure the likes, retweets and
average length of the tweet over a period. Pie charts can be used to analyze the source of the
tweet.

Approaches to sentiment classification

Sentiment Classification can be performed by classifying a feature in favorable as positive, as
well as unfavorable as negative too. Will classified as 3 levels:
1) Document-Level Classification: A document level classification helps in classifying an entire
opinion document into a positive or negative sentiment. When we are providing reviews for
products or services the entire review helps in expressing positive or negative sentiments of the
consumer towards the product.
2) Sentence-Level Classification: Sentence Level Classification involves breaking down a review
into sentences, calculating the polarity for each sentence and then accordingly calculating the
sentiments for each review.
3) Aspect-Level Classification: Aspect Level Classification judges various aspects of the entity
and giving different opinions about different aspects, it does not focus on the language
construction but focuses more on the opinion itself. The classification focuses around breaking
an opinion into sentiment of an opinion and target of opinion.

Twitter Sentiment Analysis

Twitter is a social networking and microblogging service that lets its users post real time
messages, called tweets. Tweets have many unique characteristics, which implicates new
challenges and shape up the means of carrying sentiment analysis on it as compared to other
domains.
The aim while performing sentiment analysis on tweets is basically to classify the tweets in
different sentiment classes accurately. In this field of research, various approaches have evolved,
which propose methods to train a model and then test it to check its efficiency.
Following are some key characteristics of tweets:
Message Length: The maximum length of a Twitter message is 140 characters. This is different
from previous sentiment classification research that focused on classifying longer texts, such as
product and movie reviews.
Writing technique: The occurrence of incorrect spellings and cyber slang in tweets is more often
in comparison with other domains. As the messages are quick and short, people use acronyms,
misspell, and use emoticons and other characters that convey special meanings.
Availability: The amount of data available is immense. More people tweet in the public domain
as compared to Facebook (as Facebook has many privacy settings) thus making data more
readily available. The Twitter API facilitates collection of tweets for training.
Topics: Twitter users post messages about a range of topics unlike other sites which are designed
for a specific topic. This differs from a large fraction of past research, which focused on specific
domains such as movie reviews.
Real time: Blogs are updated at longer intervals of time as blogs characteristically are longer in
nature and writing them takes time. Tweets on the other hand being limited to 140 letters and are
updated very often. This gives a more real time feel and represents the first reactions to events.
Performing sentiment analysis is challenging on Twitter data, as we mentioned earlier. Here we
define the reasons for this:
Limited tweet size: with just 140 characters in hand, compact statements are generated, which
results sparse set of features.
Use of slang: these words are different from English words and it can make an approach
outdated because of the evolutionary use of slangs.
Twitter features: it allows the use of hashtags, user reference and URLs. These require different
processing than other words.
User variety: the users express their opinions in a variety of ways, some using different
language in between, while others using repeated words or symbols to convey an emotion.
All these problems are required to be faced in the preprocessing section. Apart from these, we
face problems in feature extraction with less features in hand and reducing the dimensionality of
features.
Literature Survey

In paper [1], they present a classifier to predict contextual polarity of subjective phrases in a
sentence. Their approach features lexical scoring derived from the Dictionary of Affect in
Language (DAL) and extended through WordNet, allowing us to automatically score the vast
majority of words in our input avoiding the need for manual labeling. They augment lexical
scoring with n-gram analysis to capture the effect of context. They combine DAL scores with
syntactic constituents and then extract ngrams of constituents from all sentences. They also use
the polarity of all syntactic constituents within the sentence as features. Their results show
significant improvement over a majority class baseline as well as a more difficult baseline
consisting of lexical n-grams.

In this paper [2], we propose an approach to automatically detect sentiments on Twitter messages
(tweets) that explores some characteristics of how tweets are written and meta-information of the
words that compose these messages. Moreover, we leverage sources of noisy labels as our
training data. These noisy labels were provided by a few sentiment detection websites over
twitter data. In our experiments, we show that since our features are able to capture a more
abstract representation of tweets, our solution is more effective than previous ones and also more
robust regarding biased and noisy data, which is the kind of data provided by these sources.

In this paper [3], the author states that Microblogs as a new textual domain offer a unique
proposition for sentiment analysis. Their short document length suggests any sentiment they
contain is compact and explicit. However, this short length coupled with their noisy nature can
pose difficulties for standard machine learning document representations. In this work we
examine the hypothesis that it is easier to classify the sentiment in these short form documents
than in longer form documents. Surprisingly, we find classifying sentiment in microblogs easier
than in blogs and make a number of observations pertaining to the challenge of supervised
learning for sentiment analysis in microblogs.

In this paper [4], they demonstrate that it is possible to perform automatic sentiment
classification in the very noisy domain of customer feedback data. We show that by using large
feature vectors in combination with feature reduction, we can train linear support vector
machines that achieve high classification accuracy on data that present classification challenges
even for a human annotator. We also show that, surprisingly, the addition of deep linguistic
analysis features to a set of surface level word n-gram features contributes consistently to
classification accuracy in this domain.
According to [5], identifying sentiments (the affective parts of opinions) is a challenging
problem. We present a system that, given a topic, automatically finds the people who hold
opinions about that topic and the sentiment of each opinion. The system contains a module for
determining word sentiment and another for combining sentiments within a sentence. We
experiment with various models of classifying and combining sentiment at word and sentence
levels, with promising results.
References

1. Contextual phrase-level polarity analysis using lexical affect scoring and syntactic n-
grams.
2. Robust sentiment detection on twitter from biased and noisy data.
3. Classifying sentiment in microblogs: is brevity an advantage.
4. Sentiment classification on customer feedback data: noisy data, large feature
vectors, and the role of linguistic analysis.
5. Determining the sentiment of opinions.