2018 Conference on Information Communications Technology and Society (ICTAS)
A Framework for Sentiment Analysis with
Opinion Mining of Hotel Reviews
Kudakwashe Zvarevashe
ICT and Society Research Group,
Durban University of Technology,
P.O. Box 1334, Durban 4000, South Africa
kudakwashe.zvarevashe@gmail.com
Abstract — The rapid increase in mountains of unstructured
textual data accompanied by proliferation of tools to analyse
them has opened up great o pportunities and challenges for text
mining research. The automatic labelling of text data is hard
because people often express opinions in complex ways that are
sometimes difficult to comprehend. The labelling process
involves huge amount of efforts and mislabelled datasets usually
lead to incorrect decisions. In this paper, we design a framework
for sentiment analysis with opinion mining for the case of hotel
customer feedback. Most available datasets of hotel reviews are
not labelled which presents a lot of works for researchers as far
as text data pre-processing task is concerned. Moreover,
sentiment datasets are often highly domain sensitive and hard to
create because sentiments are feelings such as emotions,
attitudes and opinions that are commonly rife with idioms,
onomatopoeias, homophones, phonemes, alliterations and
acronyms. The proposed framework is termed sentiment
polarity that automatically prepares a sentiment dataset for
training and testing to extract unbiased opinions of hotel
services from reviews. A comparati ve analysis was established
with Naïve Bayes multinomial, sequential minimal optimization,
compliment Naïve Bayes and Composite hypercubes on iterated
random projections to discover a suitable machine learning
algorithm for the classification component of the framework.
Keywords: opinion mining, sentiment analysis, machine learning
algorithm, natural language processing, sentiment polarity,
dataset labelling
I. INTRODUCTION
In recent years, the world has experienced a tremendous
rise in the volu me of textual data especially for the
unstructured data generated from people who express
opinions through various web and social media platforms for
different reasons. Mountains of these textual data, in itially
could be equated to garbage which would need to be disposed
fro m t ime to time. However, with the advancement in storage
capacity accompanied by the increasing sophistication in data
mining tools, opportunities and challenges have been created
for analysing and deriv ing useful insights from these
mountains of data.
In this paper, we have chosen textual data in the form of
hotel reviews for sentiment analysis with opinion min ing
fro m customer perspectives. Sentiment analysis uses the
techniques of natural language processing and computational
linguistics to automate the classification of sentiments
generated from reviews. Hotels provide satisfaction, security,
ISBN 978-1-5386-1001-5/ 18/ $31.00 ©2018 IEEE
Oludayo O Olugbara
ICT and Society Research Group,
Durban University of Technology,
P.O. Box 1334, Durban 4000, South Africa
oludayoo@dut.ac.za
comfo rt, lu xu ry and lodging services for travellers and people
on vacation. Mining hotel reviews is desirable to gain deeper
knowledge of customer expectations and support effective
management of customer relationships. It would enable the
hotel managers to have a good understanding of customer
needs, discover areas for further improvement and improve
service quality. The hotel reviews are provided exclusively
by customers who have made reservations at a particular
hotel. Customers post feedback about hotels which include
hygiene, quality of food, location, customer service quality
and hospitality exh ibited by hotel staff. Moreover, sentiment
analysis of hotel reviews is crucial to understand hidden
patterns generated by data that would help to effect ively
improve performance [1].
II. RELATED LITERATURE
Sentiment analysis [2] and opinion mining [3] are terms
that refer to the field of study that analyses opinions,
evaluations, appraisals, attitudes and emotions of people
towards entities such as products, services, organizations,
individuals, issues, events, topics and their attributes [4].
These terms were used interchangeably to define opinions
that entail positive or negative sentiments [4, 5]. Sentiment
determines the polarity of opinions expressed in a given
review. Cambria et al. [6] disputed the interchange of these
concepts by classifying opinion mining as polarity detection
and sentiment analysis as focusing on emotion recognition.
The opinion min ing system only needs to understand polarity
that can be positive, negative or neutral sentiments depending
on the nature of sentences expressed in a review [7]. The
process of detecting polarity is strongly linked to analysing
sentiments on a particular subject.
Most researches on sentiment analysis are focused on
descriptive data. Manke and Shivale [8] exp lored the
significance of social networks as preferred environ ments for
opinion mining and sentiment analysis. They introduced the
original method of opinion classification and tested their
algorith m on real social network datasets. They concluded
fro m their findings that social networks exhib it properties
that make them suitable for opin ion min ing activities.
Co mprehensive surveys have been presented on various
methods used in opinion mining [9-11] with limited focus on
aspect oriented analysis.
The majority of current methods of sentiment analysis
attempt to detect the polarity of a review regardless of the
entities such as hotels and facilities with their respective
aspects such as food and internet access for instance. By
contrast, the task of this study is concerned with aspect based
sentiment analysis with the goal of identifying the aspects of
given target entities and sentiment expressed towards each
aspect. The aspect based sentiment analysis summarizes what
people like and dislike fro m reviews of products or services.
It has always been a difficu lt task [7] because several subtasks
such as feature extract ion, feature grouping, polarity
classification and evaluation measures have to be performed
to get an unbiased opinion, usually under the assumption of
grammar free errors which is not always realistic.
Hotel review is an important theme of natural language
processing (NLP), which is a discipline that deals with
processing of textual data [12]. It is at the intersection of
artificial intelligence and linguistics [6] which makes NLP
techniques amenable to sentiment analysis. The analysis of
sentiment can be performed using two basic approaches of
lexicon-based and machine learn ing [9, 10]. The mach ine
learning approach which this study is based upon utilizes t wo
main learn ing techniques of supervised and unsupervised
learning. Supervised learning algorith ms such as the support
vector mach ine, Naïve Bayes, K-nearest neighbour and
convolutional neural network deep learn ing have been
applied for sentiment analysis. Supervised learning
algorith ms require the train ing of machine using labelled
dataset. However, unsupervised learning algorith ms such as
the K-means and Fuzzy C-means clustering do not require
training datasets because they learn by observation.
The application of supervised learning algorith ms as
applied in this study to volumes of labelled train ing data on
hotel reviews can provide insightful in formation that would
help hotels imp rove performance and overall ratings amongst
competitors. Different evaluation measures such as true
positive rate, false positive rate, precision, recall, and F-
measure rate, receiver operating characteristic (ROC) area
and precision recall curve (PRC) area are the benchmark
metrics used to determine the accuracy of different
supervised learning algorith ms. However, to get accurate
results of these measures, there is the need to design a
framework that fosters the automatic creat ion of properly
labelled datasets containing actual sentiments expressed by
customers.
The success of sentiment analysis and opinion mining
hinges largely on the use of tools for executing d ifferent NLP
tasks. Tools that can be used for the NLP tasks include Red
Opal wh ich is used for enabling users to find products based
on aspects. Tools that are used to help companies extract and
analyse opinions of customers on products from blogs
include SenticNet, Luminoso, Factiva, Attensity and
Converseon. The NLTK, OpenNLP and Stanford CoreNLP
are widely used NLP toolkits to support the implementation
of basic NLP tasks such as POS tagging, named entity
recognition and parsing [11]. The W EKA system is a widely
used tool that contains a collection of techniques for data
analysis and predictive modelling. It supports several
standard data mining tasks that include data pre-processing,
clustering, classification, regression, visualization,
association rule min ing and feature selection [13]. Data can
be imported fro m an external source in formats such as
comma separated value (CSV) file. The data in its raw format
needs to be cleaned as it may not be compatible with the
processing required. Pre-processing is done to transform the
raw data into a format that can be manipulated by appropriate
tools. Furthermore, the larger the dataset, the more accurate
is the performance of the lean ing algorith ms that is inherently
measured in terms of the standard evaluation metrics.
III. METHODOLOGY
A. The Intuition Model
The conceptual view of the intuit ion model shown in
Figure 1, begins with the feedback collection. Customers
respond to questionnaires concerning their feelings about
services received from the selected hotels. This can be done
in a nu mber of ways, for example opening a web portal
through which customers can drop comments. The next step
will be to label the comments based on intuition. This will be
done by human agents who simply read the co mments and
assign labels based on perceptions. Once data are transformed
to a desire format, the next step will be to convert the labelled
text to feature vectors through the use of filters. This will
make it easier to imp lement a classification algorith m for
training and testing of data. The next step involves the
selection of an appropriate classificat ion algorith m whilst the
last step is the training and testing of the selected algorith m
on dataset and capturing of results.
B. The Sentiment Polarity Based Model
The research reported in this paper was done using the
sentimental polarity based model (SPBM ) as illustrated in
Figure 2. Just like the intuition based model (IBM), the
SPBM model begins with the elicitation of opinions which is
the step skipped because we used the raw OpinRank dataset
[14, 15]. Customers respond to questionnaires concerning
services received fro m the selected hotels through an
appropriate user interface. The next step will be to label the
comments based on sentiment polarity score using a
sentiment polarity algorith m. The score obtained will
determine whether a co mment is positive, negative or neutral.
Once the data are transformed, the next step is to convert the
labelled text to feature vectors through the use of filters. The
next step involves the selection of a suitable classification
algorith m. The last step is the training and testing of the
selected classification algorithm and capturing of results. The
distinguishing property of SPBM is labelling is that
automatic, it does not involve human intervention and it is
quite consistent in labelling sentiments. However, the IBM
relies heavily on human intervention to label sentiments
which sometimes may not be consistent and the labelling
process is intrinsically laborious and time demanding.
 IV. EXPERIM ENTA L RESULTS
Data fro m the Opin Rank opin ion based ranking dataset
was acquired to experimentally test the performance of the
investigated learning algorith ms to discover a suitable
algorith m for the classificat ion component of the SPBM
framework. We selected the OpinRan k dataset because it
contains unlabelled reviews that gave the flexib ility for a
custom experimentation. The Opin Rank dataset contains
approximately 259000 unlabelled reviews on cars and hotels
fro m 80 to 100 hotels in 10 different cities across the world.
These cities include Dubai, Beijing, London, New York City,
New Delhi, San Francisco, Shanghai, Montreal, Las Vegas
and Chicago. We selected hotel reviews fro m London,
Beijing and Montreal as a matter of choice to create a sub-
dataset for our experimentation. We labelled the hotel
features in the dataset using a sentiment polarity software
written in Python with the TextBlob which is a library for
processing textual data. The scores obtained from the
sentiment polarity were then used to automatically label the
data. After performing all the necessary processing s teps,
including labelling and filtering, the dataset was split into two
subsets to create testing and training datasets. We used four
classification algorithms wh ich are Naïve Bayes mu ltino mial
(NBM), Sequential min imal optimization (SMO),
Co mpliment Naïve Bayes (CNB) and Co mposite hypercubes
on iterated random projections (CHIRP) to train and test the
dataset. Figure 3 shows the co mparative results obtained after
experimentation. The Naïve Bayes mult inomial algorith m
had the highest precision which reached 80.9%. It was closely
followed by the compliment Naïve Bayes algorithm which
had 80.5%. CHIRP was the lowest performing algorithm and
it scored 75.6% in precision.
V. CONCLUSION
The sentiment analysis with opinion mining framework
reported in this paper can be incorporated into a hotel
technology system that can help improve customer
relationship management. What good is a system that predicts
the polarity of sentiments if it wo rks with the wrongly
labelled data? Fro m the sentiment polarity exercise that we
did, we found out that some co mments may be wrongly
viewed as neutral while they will be either positive or
negative. The following examp le was viewed as a neutral
comment. “That hotel is surely a HELLTEL!” This comment
is truly negative and sarcastic, but because the word
HELLTEL does not exist in the English vocabulary it was
classified under the neutral class. However, most comments
were labelled with a much better accuracy. We believe that a
lot of research can be done in this area especially in fine
tuning the feature extract ion algorith m of the framework so
that classification error is minimised.
The system is expected to determine sentiments the way
human beings do and labelled datasets are normally used for
the system to learn automat ically. The proposed framework
tries to make sure that sentences are correctly labelled such
that false information is not fed into the system. In a nutshell,
the proposed framework in this paper helps in automatic
labelling of sentiment datasets. The experimental result of
this study has indicated that Naïve Bayes multino mial
algorith m gave good performance when compared to other
classification algorithms in terms of the evaluation metrics
applied. In future work, we would like to improve on
automatic labelling, feature ext raction and perform
classification of customer responses based on emotions using
deep learning algorithms.
A CKNOWLEDGMENT
We would like to thank Prudence Kadebu and Innocent
Mapanga from Harare Institute of Technology, Belvedere,
Harare for their heartfelt assistance.
REFERENCES
[1] V. Dhanalakshmi, B. Dhivya and A.M. Saravanan, “ Opinion mining
from student feedback data using supervised learning algorithms”.
IEEE 3rd MEC International Conference on Big Data and Smart City.
, pp. 1-5, 2016.
[2] J. Yi, T . Nasukawa, R. Bunescu and W. Niblack, “ Sentiment analyzer:
Extracting sentiments about a given topic using natural language
processing techniques”. In Data Mining, 2003. ICDM 2003. Third
IEEE International Conference, pp. 427-434, 2003.
[3] K. Dave, S. Lawrence D.M. and Pennock, “Mining the peanut gallery:
opinion extraction and semantic classification of product reviews”. In
ACM Proceedings of the 12th international Conference on World Wide
Web, pp. 519-528, 2003.
[4] B. Liu, “ Sentiment analysis and opinion mining”. Synthesis Lectures
on Human Language Technologies, vol. 5, no. 1, pp. 1-167, 2012.
[5] A. Buche, D. Chandak and A. Zadgaonkar, “ Opinion mining and
analysis: a survey”. International Journal on Natural Language
Computing (IJNLC), vol. 2, no. 3, pp. 39-48, 2013.
[6] E. Cambria, B. Schuller, Y. Xia and C. Havasi, “New avenues in
opinion mining and sentiment analysis”. IEEE Intelligent Systems, vol.
28, no. 2, pp. 15-21, 2013.
[7] M.S. Akhtar, D.k Gupta, A. Ekbal and P. Bhattacharyya. "Feature
selection and ensemble construction: a two-step method for aspect
based sentiment analysis." Knowledge-Based Systems, vol. 125, pp.
116-135, 2017.
[8] S.N. Manke and N. Shivale, “ A review on: opinion mining and
sentiment analysis based on natural language processing”.
International Journal of Computer Applications, vol. 109, no. 4, pp.
29-32, 2015.
[9] W. Medhat, A. Hassan and H. Korashy, “ Sentiment analysis algorithms
and applications: a survey”. Ain Shams Engineering Journal, vol. 5,
no. 4, pp. 1093-1113, 2014.
[10] J.A. Balazs and J.D. Velásquez, “Opinion mining and information
fusion: a survey”. Information Fusion, vol. 27, pp. 95-110, 2016.
[11] S. Sun, C. Luo and J. Chen, “ A review of natural language processing
techniques for opinion mining systems”. Information Fusion, vol. 36,
pp.10-25, 2017.
[12] P.M. Nadkarni, L. Ohno-Machado and W.W. Chapman, “Natural
language processing: an introduction”. Journal of the American
Medical Informatics Association, vol. 18, no. 5, pp. 544-551, 2011.
[13] M.T. Khan, M. Durrani, A. Ali, I. Inayat, S. Khalid and K.H. Khan,
“ Sentiment analysis and the complex natural language”. Complex
Adaptive Systems Modeling, vol. 4, no. 1, pp. 1-19, 2016.
[14] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann and I.H.
Witten, “The WEKA data mining software: an update”. ACM SIGKDD
explorations newsletter, vol. 11, no. 1, pp. 10-18, 2009.
[15] K. Ganesan and C. Zhai, “ Opinion-based entity ranking”. Information
retrieval, vol. 15, no. 2, pp. 116-150, 2012.
 Figure 2. Sentiment polarity analysis framework

Figure 1. Intuitive sentiment analysis framework

NBM SMO CNB CHIRP

0.90
Weighted average score

0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00

Performance metric

Figure 3. Weighted average score against performance metrics