2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th
International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems
Fake Reviews Detection Based on Text Feature and
Behavior Feature
Yin Shuqin
College of Computer Science and Technology
Wuahn University of Technology
Wuhan, China
641981005@qq.com
Abstract—Considering the limitations of comment text in the
study of fake reviews recognition, this paper proposes to build a
classification model by integrating the features of comment text
and user behavior. However, the comment data obtained in
reality are mostly unlabeled. Therefore, this paper proposes an
MPINPUL (Mixing Population and Individual Nature PU
Learning) model based on multiple features to build a fake
reviews classification model. In this paper, the MPINPUL model
is divided into four steps. Firstly, the constrained k-means
algorithm is proposed to calculate a negative example of trust.
The advantage of constrained k-means is that it can expand the
set of positive examples while identifying the trusted negative
examples. Then, LDA and k-means are used to calculate
multiple representative samples for positive and negative
examples respectively. Then we use the idea of population and
individuality to determine the category label of the sample.
Finally, the classifier is established. Experimental results on real
data sets show that the recognition rate of the MPINPUL model
proposed in this paper is higher than that of other single features
under fusion feature conditions.
Keywords—fake reviews, fusion feature, PU-Learning,
constrained k-means, classification model
I. INTRODUCTION
Fake reviews identification was first proposed by Jindal
and Liu[1] in 2008. The difficulty of this research is how to
effectively extract or represent the features of the comment
text and user behavior, so as to achieve the purpose of fake
reviews recognition. Although the fabricator tries to simulate
the truth of the content as much as possible, there are some
verbal and behavioral details that may be flawed. The
researchers identified fake reviews based on different
research scenarios [2].
Ott[3,4] et al. used support vector machine classifier based
on word bag features to identify fake reviews on the gold data
set built by Amazon crowd-sourced platform with an
accuracy rate of 84%. However, fake reviews deliberately
imitate real comments in terms of language and vocabulary,
so the ability to identify fake reviews by word bag alone is
not strong. Li [5] et al. also used part of speech characteristics
in Amazon data set and found that comments constructed by
crowd-sourcing presented different characteristics from real
comments in terms of part of speech characteristics.
Crowdsourced fake reviews contained more verbs, adverbs
and pronouns. while real reviews contained more nouns,
adjectives, prepositions, qualifiers and conjunctions. Lau[6] et
al. believed that there was mutual copy between fake reviews,
978-1-7281-2058-4/19/$31.00 ©2019 IEEE
DOI 10.1109/HPCC/SmartCity/DSS.2019.00277
Feng Jing
College of Computer Science and Technology
Wuahn University of Technology
Wuhan, China
fengjing1805@163.com
and it could be identified by detecting semantically repeated
reviews. Mukherjee[7] et al. used the SVM classification
model to achieve 67.8% accuracy based on plain text features
in Yelp data sets. After the characteristics of the commenter
were integrated, the recognition accuracy was increased to
84.8%. Therefore, the behavior characteristics of reviewers
have a significant impact on the identification of fake reviews.
That's the state of the art for researchers trying to
identify fake reviews. In view of the limitations of text
features, this paper divides it into multiple behavioral features
from the perspective of reviewers for research. At the same
time, this paper proposes to use the MPINPUL (Mixing
Population and Individual Nature PU Learning[8]) model to
solve the problem that fake comments are widely distributed
in reality, have huge orders of magnitude, are difficult to be
accurately marked manually, and have some misjudgment
samples.
II. FEATURE CONSTRUCTION
Feature engineering plays an important role in the
research of natural language processing. Due to the
concealment and diversity of fake reviews, feature selection
is particularly important. In this paper, in order to integrate
multiple features for recognition, three major categories of
features including text, behavior and relationship are studied.
A. Text Features
The features extracted in this paper include Unigram
lexical features, POS features and LDA semantic features.
The comment text feature indicators and feature name
descriptions are shown in table ĉ:
TABLE I. COMMENTARY TEXT CHARACTERISTICS
Feature Name Feature Description
Unigram N-gram Lexical features
POS POS features
LDA LDA thematic features
B. Behavioral Characteristics of Reviewers
Compared with the real commenters, the behaviors of
the false commenters are often abnormal. Therefore, the
extraction of the behavior characteristics of the reviewers is
helpful to accurately identify the fake reviews. The
behavioral characteristics of reviewers extracted in this paper
are shown in table Ċ:
2007
 TABLE II. USER BEHAVIOR CHARACTERISTICS
Feature Name Feature Description
MCS Maximum text content similarity
RL Comment text length
MDN Maximum number of comments per day
PR Proportion of positive comments
MCS Commentator Scoring Deviation
Rating Score
C. Relationship Characteristics
The relationship between comment text, commenters
and merchants (commodities) is often not intuitive, so it
needs to be further explored. In this paper, FP-growth
algorithm is used to extract the commenters' frequency, and
collaborative filtering algorithm is used to analyze the target
similarity. Relationship feature names and descriptions are
shown in table ċ :
TABLE III. RELATIONSHIP CHARACTERISTICS
Feature Name Feature Description
FP Commenter frequency
IS Target item similarity
III. BUIDE MPINPUL CLASSIFICATION MODEL
Fake reviews recognition is regarded as a dichotomous
problem, and comments can be divided into true reviews and
fake reviews by classification model. The construction
process of the classification model is as follows:
Step1˖Prepare the data sets that can be entered into the
model;
Step2˖The data set is divided into training set and test
set;
Step3˖Using training set to train classification model
and output;
Step4˖The test set is used to evaluate the trained model.
Due to the large amount of unlabeled data obtained in
real applications, it is difficult to manually annotate them.
Therefore, this paper uses MPINPUL model to learn a small
number of marked samples and a large number of unmarked
samples. The MPINPUL model in this paper refers to the set
of real comments as the positive example set P, and a large
number of unlabeled data as the set U.
A. Extraction of Trusted Negative Examples Based on
Constrained K-Means Alogorithm
Based on semi-supervised clustering, this paper
proposes to use constrained k-means algorithm to extract
reliable negative examples. The clustering process is
influenced by users' needs, and the must-link constraint is
used to guide the clustering process. The advantage of the
constrained k-means algorithm is that the positive example
set is used to initialize the positive example center in the
clustering process. The positive example marker was used as
the must-link constraint to conduct constrained clustering. It
not only marks the reliable negative examples, but also
expands the positive examples.
The constrained k-means algorithm proposed in this
paper is based on the following two points:
2008
ķ Example set as a cluster of seed set;
ĸ Positive example mark as a Must-link constraint.
Algorithm 1: reliable negative example extraction based on
constrained k-means algorithm
Input˖Positive example set P
Unlabeled sample set U
Cluster number k
Output˖k disjoint clusters
Steps:
1. Initialize the clustering center
2. Set the positive example as cluster 0 seed set and
initialize the clustering center as:
1
= ;
| |
∀ ∈
3. Randomly select unlabeled samples to initialize
other clustering centers;
4. While clustering results are not changing
5. The positive sample text remains unchanged and
is allocated to cluster 0;
6. The text in the U set is allocated to cluster i*ˈIf
∗
= − , ∗ = 0,1, , … , − 1;
7. Modify the cluster centerˈ
1
= , = 0,1, … , − 1;
∀ ∈
8. End while
B. Calculation of Representative Samples
A certain accuracy can be obtained by training the
classifier with the set of trusted negative examples RN and
positive examples P obtained in the first stage. However, it
ignores a large number of spy samples in the unlabeled data
set, resulting in poor performance of the classifier. However,
the spy samples play an important role in improving the
performance of the classifier. In order to determine the
category label of spy samples, it is necessary to find samples
that can represent positive and negative examples
respectively. Therefore the first to use LDA algorithm to
achieve reliable negative set an RN distribution on different
topics, and then use the K - Means clustering algorithm for
reliable cases, let trusted negative case theme distribution of
the sample of relatively consistent category. Finally using
Rocchio classifier of positive and negative cases respectively
calculate the 10 representative sample.
C. Determine the Category Label for the Spy Sample
Determining the category label of the spy sample is the
most critical step in the MPINPUL model framework. The
purpose is to divide the set US into LP and LN. This step
requires 10 representative samples of positive and negative
samples to calculate the category label of the spy sample.
DPMM (Dirichlet Process Mixture Model) is adopted to
cluster the sample spies. The idea is: mixing spy sample
population sex and individuality, at the same time using the
probability model for spy sample calculate the probability
weights respectively belong to two categories. The
classification label error of the spy sample can be reduced to
a certain extent in order to train the classifier with higher
accuracy. The steps are as follows: first, calculate the
probability that a single sample belongs to positive example
and negative example. When the two probabilities are close, to learn a better classifier. SVM optimization is shown in
the category tag of the subclass of the sample is used to formula (8):
determine the category tag of the sample. | | | |
1
The idea of population is that samples in the same ( , , ) = || || + + ( ( )
)
2
subclass should have a high probability of belonging to the
same category. Formula (1) and (2) are used to calculate the | | | |

probability weight of each sample in the subclass belonging + ( ( )

) +
to two categories.
| | ˄8˅
_ ( ) = (1)
| |
Here C1ˈC2ˈC3 and C4 are the parameters that control
| |
_ ( ) = (2) classification errors and hyperplane boundaries. ˈ ˈ ˈ
| |
are error tolerance values, which control the tightness of
|USi| represents the total number of samples in a subclass, the boundary. ( ( )) , ( ( ) ) can be considered
|nump| represents the number of instances in subclass USi that error tolerances with different weights. It should be pointed
are temporarily labeled as positive examples, |numn|
out that ( ( ) ) larger a can increase the influence of
represents the number of instances in subclass USi that are
parameter , making spy sample ( ) more inclined to the
temporarily labeled as negative.
positive example.
The idea of individuality is to ignore the subclass of the
By solving the Lagrangian, the dual problem of the
sample and focus only on the relationship between a single
original optimization problem is shown in formula (9):
sample and the representative samples of all positive and
| ∗| | ∗|
negative examples. Using formulas (3) and (4), the category 1 () ( ) () ( )
probability of a single positive and negative example is W(α) = − < , >
2
calculated. ,
()
∑ ( , ) s. t. ≥ ≥ 0, ∈
_ ( )=∑ ( , )
(3)
( ( , )) ( )
≥ ≥ 0, ∈ (9)
∑ ( , )
_ ( )=∑ ( , )
(4) | | | |
( ( , ))
− =0
D. Calculate the Probability Weight of the Spy Sample
In order to improve the performance of the classifier, a
probabilistic model is proposed to represent a spy sample. Here, < ( ) , ( ) > is the inner product of ( ) and
( )
( ) and ( ) respectively represent the probability . In order to achieve better performance, the kernel
function can be used to map the input features to a higher
weights of positive and negative examples of a single sample.
dimensional feature space, so as to solve the problem of
The probability model is shown in formula (5): inconsistent data distribution and make the data have a better
{e, ( (e), (e))} , (e) + (e) = 1 (5) representation in the new space. By solving the above
optimization problem, the value of w is first calculated, and
For a spy example, the probability weights that then the Karush-Kuhntucker condition is used to obtain the
ultimately fall into two categories can be calculated by value of b, and then the prediction is made. For a test
mixing population and individuality. As shown in formula (6) sample , if + > 0 , it is proved to be a positive
and (7): example and a true comment; otherwise, it is a false comment.
(e) = ∙ _ ( )+ IV. EXPERIMENT
(1 − ) ∙ _ ( ) (6) In this paper, a total of three groups of comparative
(e) = experiments were completed. In order to prove the help of
∙ _ ( )+
fusion features for fake reviews recognition, the accuracy of
(1 + ) ∙ _ ( ) (7) MPINPUL model under different features was compared. In
order to verify the validity of the MPINPUL model proposed
The ¬ parameter in the above formula can be used to
in this paper. Two main PU learning algorithms LELC [9] and
balance the global and local information, and different values SPUL[10] are implemented respectively. In addition, two
will be selected for experiments, in order to obtain the overall additional multicore learning algorithms SILP [11] and
optimal. SimpleMKL[12] are implemented in the fourth stage of the
E. Establish a Classifier MPINPUL model to compare with the improved SVM
In this section, the probability weight of positive classifier in the fourth stage. At the same time, the MPINPUL
model was compared with the latest work on the Yelp website
example set P, trusted negative example RN, positive
dataset, namely SVM and XGBoost[13].
example spy sample set LP and negative example spy sample
set LN is fused to expand the optimization function of SVM

2009
A. Experiment Data Set
The data included 64,445 reviews from 99 restaurants
on Yelp. Among them, 8,035 fake comments were filtered,
while 56,410 were real. In this chapter, all 8,035 false
comments were selected and 56,410 real comments were
randomly selected. A total of 16,070 comments constitute the
data set of the experiment in this chapter. After obtaining the
experimental data set, the data was divided into 9 training set
and 1 test set by the sampling crossover method. The
proportion of positive examples in the training set was fixed
at 40%. Ten experiments were conducted in total, and
different test sets were selected for each experiment. Each
review contains a review ID, user ID, review content, stars,
date, "useful" count, whether or not it was filtered, and more.

Fig. 1. Partial Original Data Sets

B. Evaluation Index C. Analysis of Experimental Result

In this experiment, accuracy, precision, recall and F1 1) Extract the 5esults of 7rusted 1egative &ases for
values were used to evaluate the performance of the $nalysis
classification model. After carrying out the constrained k-means clustering,
the problem of selecting the mark proportion p value when
TABLE IV. PERFORMANCE EVALUATION LEAGUE TABLR the sample is marked is taken into consideration. When p is
Non-Fake Fake too large, more noise will be introduced, and when p value is
Predicted number of too small, the marked sample will be too small. Therefore, a
TN FN
true reviews simple search strategy is adopted in this paper, and the p value
Predicted number of fake changes from 1 to 30. The experimental results are shown in
FP TP
reviews
figure 1. It is not difficult to see from the figure that when the
1) AccuracyTake the proportion of true comments and proportion of positive examples is high, the difference of this
false comments with correct judgment in the total number of method under different p values is very obvious.
samples as the accuracy, as shown in formula (10) :
0.95
= (10)
0.9
2) PrecisionRefers to the correct number of true 0.85
comments in the sample judged to be true comments or false
comments, and the calculation formula is shown in formula 0.8
(11) and (12) : 0.75
( − )= (11) p=1
0.7
p=2
F1 Score

( )= (12) 0.65 p=4

0.6 p=6
3) Recall How many of the true or false comments in p=8
the data set are correctly predicted, as shown in formulas (13) 0.55 p=10
and (14) : p=20
0.5
( − )= (13) p=30
0.45
10% 20% 30% 40% 50% 60% 70% 80% 90%
( )= (14)
Percent of positive examples
4) F1 ScoreF1 Score refers to the harmonic average of
Fig. 2. Experimental Results with Different p Values
precision rate and recall rate, as shown in formula (15) :
× ×
1= (15)

2010
 2) Results of MPINPUL Model under Different
Characteristics
In the experiment of this chapter, the method of ten fold
crossing is adopted and ten experiments are carried out.
Different test sets were selected each time, and the average of
the results of 10 experiments was taken as the final
experimental results. The value of parameter s was selected
as 0.15, the value of a was 0.3, and the proportion of positive
examples was taken as 40% for the experiment. Table Ⅴ
shows the experimental results of MPINPUL model on Yelp
data set. It can be seen that the classification model trained on
fusion features is about 10% more accurate than that trained
on text features. It fully proves that the behavior
characteristics of commenters are helpful to the identification
of false comments.

TABLE V. CLASSIFICATION EXPERIMENTAL RESULT OF MPINPUL MODEL

Non-fake fake
Classification Accuracy
Features
model ˄%˅ Precision Recall F1 Precision Recall F1
(%) (%) (%) (%) (%) (%)
Unigram 77.31 77.89 78.25 78.07 78.93 76.15 77.52
POS 76.45 75.48 72.16 73.78 73.68 77.03 75.32
LDA 76.82 75.13 78.74 76.89 77.17 74.36 75.74
MPINPUL Behavioral 83.84 89.21 79.39 84.01 82.67 92.36 87.25
Behavioral+
82.65 89.68 75.68 82.09 78.14 92.07 84.53
Relational
fusion feature 87.51 89.92 83.67 86.68 86.32 90.36 88.29

3) Comparison of Experimental Results of Several PU model is MPINPUL>XGboost>SVM. It not only proves the
Learning Models importance of integrating text and behavior characteristics for
In order to verify the effectiveness of the algorithm false comment recognition, but also fully reflects the
proposed in this chapter, two additional mainstream PU effectiveness of MPINPUL classification model of mixed
learning models LELC and SPUL were implemented for population and individuality in fake reviews recognition.
comparison. In the fourth stage of MPINPUL model, two
additional multi-core learning algorithms, SILP and SVM XGBoost MPINPUL
90
SimpleMKL, were implemented to train the classifier and
compared with the improved SVM classifier. Firstly, the 85
comparison between the MPINPUL model proposed in this
80
chapter and the previous PU algorithm is discussed. Table Ⅵ
shows that the MPINPUL model designed in this chapter is 75
superior to the previous PU learning algorithm, and its 70
accuracy is as high as 87.51%. Based on the first three stages
of the MPINPUL model, the multi-core learning algorithms 65
SILP and SimpleMKL are trained in the fourth stage, and the
accuracy rate is 85.16% and 86.57% respectively, which is
also higher than the traditional PU learning algorithm LELC
and SPUL, fully proving the effectiveness of the MPINPUL
model designed in this chapter.

TABLE VI. COMPARISON OF EXPERIMENTAL RESULTS OF SEVERAL

PU LEARNING MODELS Fig. 3. Comparison of Experimental Results of Various Classification
Models
Accuracy Accuracy Accuracy
Algorithm
˄P=20%˅ ˅ ˄P=30%˅ ˅ ˄P=40%˅ ˅ V. CONCLUSION
LELC 80.54 82.19 84.35
SPUL 81.69 83.93 84.47 Aiming at the problem of single features of comment
SILP 82.67 84.23 85.16 text, this paper puts forward the fake reviews recognition by
SimpleMKL 83.21 84.83 86.57 integrating comment text and commenter behavior features.
MPINPUL 82.87 86.79 87.51
Considering the lack of a large number of annotated data in
the fake reviews recognition, a PU learning algorithm was
4) Comparison of Experimental Results of Classification proposed to identify the fake reviews text. Based on the
Model research of traditional PU learning algorithm, this paper
In order to verify the effectiveness of MPINPUL model proposes a PU learning model (MPINPUL) based on mixed
in the study of fake reviews recognition, this paper population and individuality. The MPINPUL model is
implemented two additional classification models, SVM and divided into four steps: extracting credible negative samples,
XGBoost. Figure  is a comparison diagram of the computing representative samples, determining the category
experimental results of SVM, XGboost and MPINPUL model. label of spy samples, and establishing the final classifier.
The proportion of positive examples is fixed at 40%. Figure Through three sets of comparative experiments, the
3 shows that, on the whole, fusion features are better than significance of the critic's behavior in the identification of
plain text features, and the performance of classification fake reviews and the feasibility and effectiveness of

2011
MPINPUL were confirmed from the two aspects of features
and classification model.
REFERENCES
[1] Jindal N,Liu B.Opinion Spam and Analysis. Proceedings of the 2008
International Conference on Web Search and Data Mining , 2008.
[2] Li Lu-Yang, Qin Bing,Liu Ting. Survey on Fake Review Detection
Research[J]. Chinese Journal of Computers, 2018(04)(in Chinese).
[3] Ott M, Cardie C, Hancock J T. Negative deceptive opinion spam.
Proceedings of the Conference of the North American Chapter of the
Association for Computational Linguistics:Human Language
Technologies ,2013.
[4] Ott M,Choi Y,Cardie C, et al.Finding deceptive opinion spam by any
stretch of the imagination. Meeting of the Association for
Computational Linguistics:Human Language Technologies,2011.
[5] Li J, Ott M, Cardie C, et al. Towards a General Rule for Identifying
Deceptive Opinion Spam. Meeting of the Association for
Computational Linguistics ,2014.
[6] Raymond Y. K. Lau,S. Y. Liao,Ron Chi-Wai Kwok,Kaiquan
Xu,Yunqing Xia,Yuefeng Li.Text mining and probabilistic language
.
2012
modeling for online review spam detection[J]. ACM Transactions on
Management Information Systems (TMIS) ,2012 (4).
[7] Mukherjee A, Venkataraman V,Liu B,et al.What Yelp Fake Review
Filter Might be Doing? Proceedings of the7th International AAAI
Conference on Weblogs and Social Media , 2013.
[8] Ren Ya-Feng, Ji Dong-Hong, Zhang Hong-Bin, et al. Deceptive
reviews detection based on positive and unalbeled Learning.Journal of
Computer Research and Development,2015,52(3):639-648(in
Chinese).
[9] Li X L, Philip S Y, Liu B, et al. Positive unlabeled learning for data
stream classification[C]. Proceeding of the 9th SIAM Int Conference
on Data Ming. Philadelphia, PA: SIAM, 2009.
[10] Xiao Yanshan, Liu Bing, Yin Jie, et al.Similarity-based approach for
positive and unlabeled learning[C]. Proceeding of the 22nd Int Joint
Conference on Artificial Intelligence. San Francisco: Morgan
Kaufmann,2011.
[11] Gert L, Nello C, Peter B,et al.Learning the Kernel matrix with semi-
definite programming [C]. Journal of Machine Learning Research,
2004.
[12] Alain R, Framcis R B, Stephane C, et al.SimpleMKL[J]. Journal of
Machine Learning Research,2008.
[13] Chen T, Guestrin C. XGBoost: A Scalable Tree Boosting System[J].
Proceedings of the 22nd ACM Sigkdd International Conference on
Knowledge Discovery and Data Mining. ACM, 2016:785-794.