See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/362540336

Sentimental Analysis on Web Scraping Using Machine Learning Method

Article in Journal of Information and Computational Science · August 2022

DOI: 10.12733/JICS.2022/V12I08.535569.67004

CITATIONS READS

10 1,214

4 authors, including:

Dr. Yusuf Perwej Km Divya

ProfessorAmbalika Institute of Management & Technology Ambalika Institute of Management & Technology
103 PUBLICATIONS 1,286 CITATIONS 7 PUBLICATIONS 14 CITATIONS

SEE PROFILE SEE PROFILE

Puneet Kumar Yadav

Chandigarh University
5 PUBLICATIONS 21 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

An Experiential Study of the Big Data View project

Developing an Intelligent System for Medical Diagnosis View project

All content following this page was uploaded by Dr. Yusuf Perwej on 07 August 2022.

The user has requested enhancement of the downloaded file.

Journal of Information and Computational Science1 ISSN: 1548-7741

Sentimental Analysis on Web Scraping Using

Machine Learning Method
Saurabh Sahu1*, Km Divya2, Dr. Neeta Rastogi3, Puneet Kumar Yadav4, Dr. Yusuf Perwej5
1*
B.Tech Scholar, Computer Science & Engineering, Ambalika Institute of Management & Technology, Lucknow
2
Assistant Professor, Department of Computer Science & Engineering, Ambalika Institute of Management & Technology, Lucknow
3
Professor, Department of Computer Science & Engineering, Ambalika Institute of Management & Technology, Lucknow
4
Assistant Professor, Department of Computer Science & Engineering, Ambalika Institute of Management & Technology, Lucknow
5
Professor, Department of Computer Science & Engineering, Ambalika Institute of Management and Technology, Lucknow

Abstract — Most consumers look to online reviews when deciding which e-commerce services or items to purchase.
Unfortunately, the main problem with these reviews, which is not properly addressed, is that there are dishonest reviews. The
novel feature of the proposed approach is the application of opinion mining on customer evaluations to help companies and
organizations continually enhance their marketing strategies and obtain a comprehensive analysis of consumer perceptions of
their products and brands. In this study, the long short-term memory (LSTM) and deep learning convolutional neural network
integrated with LSTM (CNN-LSTM) models were used to analyses the sentiment of reviews in the e-commerce market. Data
preprocessing techniques including lowercase processing, stop word removal, punctuation removal, and tokenization were used
for data purification. Clean data was processed using the LSTM and CNN-LSTM models to identify and classify the customers'
sentiment as either positive or negative. The best accuracy achieved by our model was 97.8%.

Index Terms — Long Short Term Memory (LSTM), Convolutional Neural Network (CNN), Sentiment Analysis, Machine Learning,
Pre-processing, Tokenization.

1. Introduction
The ability of technology to transport information quickly
characterises the digital revolution, often known as the data
age [1]. In recent years, government agencies [2] and
corporate trade organisations have put a lot of effort into
learning what their target customers think. But more and more
individuals are freely sharing their thoughts online. The
availability of historical data online today makes obtaining it
straightforward. Individual opinions can be expressed by
participants verbally or via the use of evaluations or reviews.
Furthermore, information provided by a person is more
trustworthy than information provided by a producer [3].
Therefore, in order to market a product or develop new ones,
it is essential to know what the customer wants and dislikes
about it. Most companies or producers rely on consumer input
to increase sales. Word-of-mouth recommendations from
customers are exactly like product reviews posted online. The
study of sentiment is essential in the field of finance. The aim
of sentiment is to determine the direction of the emotions
suggested by the screenplay. The word "Subjectivity
Analysis" is a by-product of opinion mining. The study of
emotions, sentiments, views, behaviours, and assessments is
known as subjectivity analysis [4]. An opinion is a judgement
or a belief that lacks support from facts or evidence, according
to the definition. Rarely is the attitude expressed in a review
of a specific product expressly favourable or negative;
instead, consumers typically have a mixed opinion about
numerous qualities, some of which are positive and some of
which are bad. Analyse a review despite the poor battery life,
I appreciate Micromax's multimedia features [5]. This
sentence evokes conflicting feelings. This study suggests an
approach that uses dependency parsing to identify
relationships or patterns and also the views which correspond
to them, representing those relationships as a graph
comprising features and opinions. In order to get only those
thought expressions which are most closely connected to the
target feature (user-specified feature), clustering is done [6]
on the graph. The other expressions are pruned. Compared to
the baseline, we were highly accurate in every domain.
Compare this method to cutting-edge systems where, with
limited data [7], we attain a comparable level of accuracy. Our
model got highest accuracy as compare to other models
The remainder of this paper is structured in the following
manner. In section II, the related work is discussed. In
sections III and IV, we discuss the Sentiment Analysis and the
Web Scraping. The methodologies are presented in section V.
We present result and discussion in section VI. Section VII
brings this paper to a close
2. Related Work
A software agent that simulates the human experience of Web
surfing interaction is used in the web scraping process, which
includes systematic content extraction and combining from
the Internet [8]. It might be viewed as a manual task from an
operational perspective, which means copying and pasting the
data. The distinction in this case is that a virtual computer
agent is used to accomplish this operation automatically [9]. It
might also be viewed as an extraction procedure that converts
Internet material that is unstructured into a structured format
that is simple to read and utilize for various studies.
The web scraper uses a request to get access to the website,
and then uses the HTML code to discover specified

Volume 12 Issue 8 - 20221 24 www.joics.org

Journal of Information and Computational Science2
components, extract them, and store them into a structured
manner using a data frame. This method of removing
unstructured material from websites can be used in a variety
of contexts. It may be used, for instance, to compare product
pricing across several websites, to remove advertisements
from connected pages, to address issues with inadequate
statistics data, or to supplement official datasets with
additional data [10]. Web scraping may also be utilized in the
human resources field to locate open positions on various
websites and categorize those using Nave Bayes algorithms.
Another work on sentiment analysis for huge data was done
by Sharma et al. They have endeavored to present a summary
of the most recent developments in opinion mining. Although
we still have difficulties relating to unstructured data, they
found that sentiment analysis has grown to be quite popular in
this study sector and noted that a lot of research has already
been done. Furthermore, they claim that Wordnet is the most
frequently used lexicon source and that supervised rather than
dictionary procedures yield more accurate findings [11]. An
ensemble framework for Sentiment Classification, developed
by merging several feature sets and classification
methodologies, was employed by Xia et al. [12]. They
employed three basic classifiers (Naive Bayes, Maximum
Entropy, and Support Vector Machines), two kinds of feature
sets (Part-of-speech information and Word-relations), and
two different feature sets in their research. For sentiment
classification, they used ensemble techniques such fixed
combination, weighted combination, and meta-classifier
combination and got improved accuracy. Text having a
subjective context performs better in terms of sentiment
analysis than text with merely an objective context. This is
because, when a book's context or perspective is objective,
the text often portrays some commonplace assertions or facts
without evoking any sentiment [13].
Additionally, Dandannavar and Mangalwede offered an
overview on several methods for sentiment analysis that use
textual data. They analyzed the benefits and drawbacks of
each method in their study, and they came to the conclusion
that several ways could be used to do sentiment analysis,
some of which were based on lexicon, some on training sets,
and some of which employed both. The approaches that were
examined are domain-specific, and the majority of them focus
on English and Chinese. As a result, relatively few
investigations on sentiment categorization for other languages
have been done [14].
The bag-of-words technique was employed by Turney et al.
[15] for sentiment analysis, in which the connections between
words were not at all taken into account and a text was only
represented as a collection of words. Every word's emotion
was identified in order to assess the overall mood of the
document. These values were then combined using various
aggregation techniques. WordNet's lexical database was
utilized by Kamps et al. [16] to analyses the emotional
qualities of words. On WordNet, they created a distance
metric and identified the semantic polarity of adjectives. A
crucial method of the dictionary-based approach was also
described by Hu and Bing [17] in their final presentation.
In this one, a list of opinion words is manually compiled, and
this set is expanded by looking in other places (such as the
well-known corpora WordNet or thesaurus), in order to find
Volume 12 Issue 8 - 20222
ISSN: 1548-7741
more synonyms and potential antonyms. The following
iteration is then started when the new words are added to the
seed list. When no further words can be identified, the entire
process comes to an end.
3. Sentiment Analysis
The process of determining whether a block of text is positive,
negative or neutral is known as sentiment analysis. Sentiment
analysis is the contextual mining of words to determine the
social sentiment of a brand and to assist businesses in
determining whether or not the product they are producing
will find a market [18]. Sentiment analysis's objective is to
examine public opinion in a way that will support business
growth. It emphasizes emotions as well as polarity (positive,
negative, and neutral) (happy, sad, angry, etc.). Sentiment
analysis aids data analysts at major corporations in
understanding consumer experiences, doing complex market
research, monitoring brand and product reputation, and
gauging public opinion. In order to provide their own clients
with insightful information, data analytics [19] firms
frequently incorporate third-party sentiment analysis APIs
into their own platforms for customer experience
management, social media monitoring, and workforce
analytics.
4. Web Scraping
Automated web data and information collection is called web
scraping. In essence, it is the removal of online data.
Information retrieval, news collecting, website monitoring,
competitive marketing, and other topics are covered by web
scraping [20] shown in figure 1. Web scraping makes it quick
and straightforward to access the large quantity of
information available online. Compared to manually pulling
data from websites, it is far faster and easier. These days, web
scraping is more and more common. A lot of data collection
and information extraction may be done quickly and easily
using an online data scraping software. Compared to
manually pulling data from websites, it is far faster and easier.
These days, web scraping is more and more common. A lot of
data collection and information extraction may be done
quickly and easily using an online data scraping software.
However, when individuals use the term "web scrapers," they
often refer to computer programmes.
Figure 1 The Web Scraping
Web scraping software (sometimes known as "bots") is
designed to browse websites, scrape the pertinent pages, and
25 www.joics.org
Journal of Information and Computational Science3
extract meaningful data. These bots can quickly retrieve
enormous volumes of data by automating this process [21]. In
the digital era, when big data plays such a significant role and
is continuously updating and changing, this has obvious
advantages. Web scraping has several uses, particularly in the
area of data analytics.
5. Methodology
The main contributions of the proposed worked are the
following.
Figure 2 The Methodology Flow Chart
5. 1 Dataset and Description
To assess the proposed method, a dataset [22] in CSV file
format was collected from comments on the Amazon
website.com. The collection includes reviews of televisions,
video surveillance devices, cellphones, tablets, laptops, and
more shown in figure 2. A variety of procedures are involved
in the preparation of the data, including the processing of
lowercase data that contains meta-features such the reviewer's
ID, the product ID, and the review content.
5. 2 Data Preprocessing
We did a number of preprocessing procedures on the data as a
whole to make it simpler to process the review texts. To assess
our model for the best accuracy, we use 80% training and 20%
testing.
 Lower Case
It requires altering all of the words in the review text to
lowercase.
 Stopword Elimination
Volume 12 Issue 8 - 20223
ISSN: 1548-7741
Common words like "the," "a," "an," "is," and "are" are
examples of stop-words. Due to the fact that they don't
provide any knowledge that is crucial for the model, these
words were removed from the review's content.
 Remove all punctuation
There was no punctuation in the materials utilized for the
review.
 Contraction Removal
The full form is used instead of the abbreviated form when the
word is really written there; for instance, “when've” becomes
“when have”. Contraction removal is the name given to this
process.
 Tokenization
Tokens, or little bits of words, were used to separate the
sentences in the review texts.
 Part-of-Speech Tagging
In this stage, the phrase's words are each assigned a POS[23]
tag, such as "VB" for a verb, "AJJ" for an adjective, and "NN"
for a noun.
 Score Generation-
A score was produced after sentiment analysis of the review
text. The sentiment score was calculated by comparing the
dataset to the opinion lexicon [22], which contains 5,000
positive words and 4,500 negative keywords, along with their
associated ratings.
 Word Embeddings
We created numerical vectors for each preprocessed sentence
in the dataset of product reviews using the "Word
embeddings" approach.
To create word indices, we first turned each phrase in the
review text into a sequence. We use the Keras texts tokenize
to get those indexes [24]. We made sure that all terms and
words received a one index in tokenize and that the
vocabulary size was correctly updated. Then, for each word in
the training and testing sets, a unique index is created.
Figure 3 The CNN Model
26 www.joics.org
Journal of Information and Computational Science4 ISSN: 1548-7741

5. 3 Convolutional Layer
The most popular categorization models are CNN. CNNs are
well-liked because they can independently learn the features
for each class without the need for human feature extraction.
A CNN shown in figure 3 [25] building layer, the primary
convolutional layer, performs well with CSV and image data.
By mixing data with filters, CNN develops features that are
subsequently passed on to the following layer. The layer that
creates new features filters the previously created features
once more.

5. 3.1 Dense Layer

A layer of basic neurons known as the "dense layer" is

one in which every neuron gets input from every neuron
in the layer above. Images are referred to the dense
layer via the convolutional layer.

5. 3.2 Max Pooling Layer

With a particular kernel size (n*n), Max Pooling

evaluates the largest value from the image matrix and
uses it to build a down-sampled (pooled) feature map,
hence lowering the process time. It is regularly
employed following a convolutional layer. The 5 x 5
kernel has been assigned.

5. 3.3 Batch Normalization layer

The batch normalization approach boosts the neural

networks' speed and stability. The layer performs
operations such as standardization and normalization
on the inputs it receives from the layer above.
5. 3.4 Layer LSTM
The RNN [26] is a LSTM [27] version that can develop
long-term dependence. An LSTM layer was provided to 50
concealed units so they may enter the subsequent layer. One
of the most important advantages of employing a
convolutional neural network as a method for feature
extraction rather to a traditional LSTM is the decrease in the
aggregate number of features [28] shown in figure 4. These
features (words) from the feature extraction process are used
by a sentiment classification model to evaluate if the content
of the product review is favorable or negative.
Figure 4 The Summary of Model
6. Result and Discussion
The CNN-LSTM model's accuracy, which was fairly good at
97.8% [30] shown in figure 5. We are seeing that our model
has the greatest accuracy. When compared to other models
when I apply it to this dataset. Thus, we can claim that our
algorithm has the highest accuracy rate for sentiment analysis
of customer reviews. The confusion matrix is used to show the
sample's TP, FP, TN, and FN rates. These rates were used to
generate the performance metrics for the CNN-LSTM model
[31], which forecasts consumer sentiment using hidden data
[32]. These metrics include specificity, accuracy, recall,
precision, and F1-score.

5. 3.5 Sigmoid Activation Functionn

In this initial layer, the output classes are identified and

categorized as either positive or negative. The formulas below

Evaluation metrics, the suggested models were evaluated

using the metrics for accuracy, precision, recall, F1-score
[29], and specificity (CNN-LSTM and LSTM). Below are the
performance metrics
Figure 5 The Model Accuracy

Volume 12 Issue 8 - 20224 27 www.joics.org

Journal of Information and Computational Science5
7. Conclusion and Future Work
In order to assign weighted sentiment ratings to the entities,
topics, themes, and categories included inside a sentence or
phrase, sentiment analysis systems for text analysis integrate
natural language processing (NLP) with machine learning
approaches. The bulk of sentiment analysis methods used in
this study were created using machine learning techniques. In
comparison to other models, CNN-LSTM provides the best
accuracy. Based on a thorough examination, we can conclude
that our approach is most effective for sentiment analysis,
which has an accuracy rate of 97.8%. In the future, any author
could create sentiment analysis technology or robust machine
learning algorithms with great accuracy.
Reference
[1] Nikhat Akhtar, Firoj Parwej, Yusuf Perwej, “A Perusal
of Big Data Classification and Hadoop Technology”,
International Transaction of Electrical and Computer
Engineers System (ITECES), USA, Volume 4, No. 1,
Pages 26-38, 2017, DOI: 10.12691/iteces-4-1-4
[2] Al-Mushayt O., Haq Kashiful, Yusuf Perwej,
“Electronic-Government in Saudi Arabia; a Positive
Revolution in the Peninsula”, International Transactions
in Applied Sciences, India, ISSN-0974-7273, Volume 1,
Number 1, Pages 87-98, 2009
[3] Rui Xia, Feng Xu, Chengqing Zong, Qianmu Li, Yong Qi
and Tao Li, "Dual Sentiment Analysis: Considering Two
Sides of One Review", IEEE Trans. On Knowledge and
Data Engineering, 2015
[4] A. Pak and P. Paroubek, "Twitter as a Corpus for
Sentiment Analysis and Opinion Mining", LREc, vol. 10,
no. 2010, 2010
[5] anushka Bollegala, David Weir and John Carroll,
"Cross-Domain Sentiment Classification Using a
Sentiment Sensitive Thesaurus", IEEE trans. on
knowledge and data engineering, vol. 25, no. 8, August
2013
[6] Y. Dang, Y. Zhang, and H. Chen, “A lexicon-enhanced
method for sentiment classification: an experiment on
online product reviews,” IEEE Intelligent Systems, vol.
25, no. 4, pp. 46–53, 2010
[7] Yusuf Perwej, Bedine Kerim, Mohmed Sirelkhtem
Adrees, Osama E. Sheta, “An Empirical Exploration of
the Yarn in Big Data”, International Journal of Applied
Information Systems (IJAIS) – ISSN: 2249-0868,
Foundation of Computer Science FCS, New York, USA,
Volume 12, No.9, Pages 19-29, 2017, DOI:
10.5120/ijais2017451730
[8] Glez-Peña, D.; Lourenço, A.; López-Fernández, H.;
Reboiro-Jato, M.; Fdez-Riverola, F. Web scaping
technologies in an API world. Brienfings Bioinform.
2014, 15, 788–794. [CrossRef] [PubMed]
[9] Saurkar, A.V.; Pathare, K.G.; Gode, S.A. An overview on
web scraping techniques and tools. Int. J. Future Revolut.
Comput. Sci.Commun. Eng. 2018, 4, 363–367.
[10] Hillen, J. Web scraping for food price research. Br. Food
J. 2019, 121, 3350–3361.
[11] Sharma, D.; Sabharwal, M.; Goyal, V.; Vij, M.
Sentiment analysis techniques for social media data: A
review. In First International Conference on Sustainable
Volume 12 Issue 8 - 20225
ISSN: 1548-7741
Technologies for Computational Intelligence; Springer:
Singapore, 2020; pp. 75–90.
[12] R. Xia, C. Zong, and S. Li, “Ensemble of feature sets and
classification algorithms for sentiment classification,”
Information Sciences: an International Journal, vol. 181,
no. 6, pp. 1138–1152, 2011
[13] Sarkar, Dipanjan. Text Analytics with Python. Text
Analytics with Python: A Practical Real-World
Approach to Gaining Actionable Insights from your
Data. Apress, 2017
[14] Dandannavar, P.S.; Mangalwede, S.R.; Deshpande, S.B.
Emoticons and their effects on sentiment analysis of
Twitter data. In EAI International Conference on Big
Data Innovation for Sustainable Cognitive Computing;
Springer: Cham, Switzerland, 2020; pp. 191–201.
[15] P. D. Turney, “Thumbs up or thumbs down?: semantic
orientation applied to unsupervised classification of
reviews,” in Proceedings of the 40th annual meeting on
association for computational linguistics, pp. 417–424,
Association for Computational Linguistics, 2002.
[16] J. Kamps, M. Marx, R. J. Mokken, and M. De Rijke,
“Using wordnet to measure semantic orientations of
adjectives,” 2004.
[17] Hu, M.; Liu, B. Mining and summarizing customer
reviews. In Proceedings of the Tenth ACM SIGKDD
International Conf. on Knowledge Discovery and Data
Mining, Seattle, WA, USA, 22–25; pp. 168–177, 2004
[18] Svetlana Kiritchenko, Xiaodan Zhu and Saif M.
Mohammad, "Sentiment Analysis of Short Informal
Texts", Journal of Artificial Intelligence Research, vol.
50, pp. 723-762, 2014
[19] Firoj Parwej, Nikhat Akhtar, Yusuf Perwej, “A Close-Up
View About Spark in Big Data Jurisdiction”,
International Journal of Engineering Research and
Application (IJERA), ISSN: 2248-9622, Volume 8, Issue
1, (Part -I1), Pages 26-41, 2018, DOI:
10.9790/9622-0801022641
[20] ErdincUzun, A novel web scraping approach using the
additional information obtained from web pages,
Preparation of Papers for IEEE Transactions and
Journals, Vol 4, 2016
[21] Renita Crystal Pereira and T Vanitha, "Web Scraping of
Social Networks", International Journal of Innovative
Research in Computer and Communication Engineering,
vol. 3, pp. 237-239, Oct. 2018
[22] R. S. Jagdale, V. S. Shirsat, and S. N. Deshmukh,
“Sentiment analysis on product reviews using machine
learning techniques. Cognitive informatics and soft
computing,” Advances in Intelligent Systems and
Computing, vol. 768, 2018.
[23] Asif Perwej, Dr. Yusuf Perwej, Nikhat Akhtar, and Firoj
Parwej, “A FLANN and RBF with PSO Viewpoint to
Identify a Model for Competent Forecasting Bombay
Stock Exchange COMPUSOFT, An International
Journal of Advanced Computer Technology, 4 (1),
Volume-IV, Issue-I, Pages 1454-1461, 2015, DOI:
10.6084/ijact.v4i1.60
[24] Tensorflow and text preprocessing, Accessed 20 January
2022,https://www.tensorflow.org/api_docs/python/tf/ker
as/preprocess /text/Tokenize.
[25] Khadeeja Naqvi, Divyanshi Gautam, Ashish Kumar
Srivastava, Prof. (Dr.) Syed Qamar Abbas, Dr. Nikhat
Akhtar, “A Machine Learning-Based Rational Breast
28 www.joics.org
Journal of Information and Computational Science6 ISSN: 1548-7741

Cancer Diagnosis” , Journal of Emerging Technologies

and Innovative Research, Volume 9, Issue 7, Pages
558-567, 2022, DOI: 10.6084/m9.jetir.JETIR2207677
[26] Yusuf Perwej, “Recurrent Neural Network Method in
Arabic Words Recognition System”, International
Journal of Computer Science and Telecommunications
(IJCST), which is published by Sysbase Solution (Ltd),
UK, London, Volume 3, Issue 11, Pages 43-48, 2012
[27] S. N. Alsubari, S. N. Deshmukh, M. H. Al-Adhaileh, F.
W. Alsaade, and T. H. Aldhyani, “Development of
Integrated Neural Network Model for Identification of
Fake Reviews in E-Commerce Using Multidomain
Datasets,” Applied Bionics and Biomechanics, vol.
2021, Article ID 5522574, 11 pages, 2021
[28] Yusuf Perwej, “The Bidirectional Long-Short-Term
Memory Neural Network based Word Retrieval for
Arabic Documents”, Transactions on Machine Learning
and Artificial Intelligence (TMLAI), Society for
Science and Education, United Kingdom (UK), ISSN
2054-7390, Volume 3, Issue 1, Pages 16 - 27, 2015, DOI:
10.14738/tmlai.31.863
[29] Nikhat Akhtar, Devendera Agarwal, “An Efficient
Mining for Recommendation System for Academics”,
International Journal of Recent Technology and
Engineering (IJRTE), Volume-8, Issue-5, Pages
1619-1626, 2020, DOI: 10.35940/ijrte.E5924.018520
[30] Jorge E. Zambrano, Daniel P. Benalcazar, Claudio A.
Perez, Kevin W. Bowyer, "Iris Recognition Using
Low-Level CNN Layers Without Training and Single
Matching", IEEE Access, vol.10, pp.41276-41286, 2022
[31]J. Fan, W. Xu, Y. Wu and Y. Gong, "Human tracking
using convolutional neural networks", Neural Networks
IEEE Transactions, 2010
[32] Svetlana Kiritchenko, Xiaodan Zhu and Saif M.
Mohammad, "Sentiment Analysis of Short Informal
Texts", Journal of Artificial Intelligence Research, vol.
50, pp. 723-762, 2014

Volume 12 Issue 8 - 20226 29 www.joics.org

View publication stats