Co-Extracting Opinion

Targets and Opinion

Words from Online
Reviews Based On
Word Alignment
Model
Presented by,
Rehna Kamil
M1 CSE. Roll No:15

Introduction
Data Mining

Text Mining

Web Mining

Web Usage
Mining

Web Content
Mining
Opinion
Mining
Fig1:Overview Of Data Mining

Web
Structure
Mining

Contd
Data mining - process of pattern discovering in large
data sets .
Text mining, refers to the process of deriving highquality information from text.
Web mining - is the application of data mining
techniques to discover patterns from the World Wide
Web. Web mining can be divided into three different
types Web usage mining, Web content mining and
Web structure mining.
Opinion mining is the field of study that analyses the
people opinions, sentiments, appraisals and emotion
towards the entities such as products, services.

Opinion Mining
Opinion mining is a technique which is used to detect and
extract subjective information in text documents.
Fig2:Workflow of Opinion Mining
Web
Users

Opinion
Impact

Comme
nt
Reviews

Input
Docume
nt

Positive
Opinions
Negative
Opinions

PreProcessin
g

Sentiment
Classificati
on

Opinion targets and Opinion words

Opinion target: Object about which users express
their opinions. Nouns or noun phrases are opinion
target.
Opinion words: The words that are used to express
users opinions. Adjectives/verbs are opinion words.
For example:
"This phone has a bright and big screen, but its
LCD resolution is very disappointing."
In the above example, three opinion words are
"bright", "big" and "disappointing" and two opinion
targets are "screen" and " resolution".

Different Levels of Opinion Mining

Document level Opinion Mining-To classify whether an
opinion of the document expresses a positive or
negative sentiment.
Sentence level Opinion Mining-Relation between
sentences are calculated.
Phrase/Feature level Opinion Mining-Phrases that
contain words are found out and a phrase level
classification.

Existing System
In previous methods, mining the opinion
relations between opinion targets and opinion
words was the key to collective extraction. The
most adopted techniques have been nearestneighbour rules and syntactic patterns.
Nearest neighbour rules regard the nearest
adjective/verb to a noun/noun phrase in a
limited window as its modifier.
Syntactic information, in which the opinion
relations among words are decided according to
their dependency relations in the parsing tree.

Disadvantages of Existing
System
Nearest neighbour rules strategy cannot obtain
precise results because there exist long-span
modified relations and diverse opinion
expressions.
Syntactic patterns are prone to errors.
The collective extraction adopted by previous
methods was based on a bootstrapping
framework, which has the problem of error
propagation

Proposed System
A simple method for extracting opinion target
and opinion word from online review based on
word alignment model.
To mine opinion relations in sentences through
partially-supervised word alignment model.
Then, a graph-based algorithm is to estimate the
confidence of each opinion target and opinion
word(candidate), and the candidates with higher
confidence than threshold will be extracted as
the opinion targets and opinion word.

Advantages of Proposed
System
WAM can obtain precise results ,since it scans
complex relations
WAM is more robust compared to syntactic
information, since it does not need to parse
informal text.
WAM integrate factors including word cooccurrence frequencies and word positions.

Overview of Method
Briefly, there are two important problems:
1) how to capture the opinion relations and
calculate the opinion associations between
opinion targets and opinion words
2) how to estimate the confidence of each
candidate with graph co-ranking.
Basic motivation follows here is :
If a word is likely to be an opinion word, the nouns/
noun phrases with which that word has a modified
relation will have higher confidence as opinion
targets. If a noun/noun phrase is an opinion target,
the word that modifies it will be highly likely to be an
opinion word.

Contd
Reviews
of
people

Extraction Of
Candidate with
Higher
Confidence

Extra
ct

Opinion
Words and
Opinion
Target

Estimation of
Confidence of
each
Candidate

Dete
ct

Opinion
Relations

Identifyin
g Opinion
Relations

Fig3:Extracting Opinion Word from review

System Architecture
System architecture is the design part which
defines structure and behaviour of the system.
Review source: In this user will load the reviews
from the online to the opinion processing engine.
Opinion processing Engine: This module has three
sub modules such as word alignment module,
partially supervised word alignment and graph coranking.
Partially supervised word alignment model: This
model which regards identifying opinion relations
as an alignment process.
Co-ranking algorithm: This algorithm is used to
exploit the estimate confidence of each candidate.

System Architecture
Opinion Processing Engine
Word
Alignment
Model
Review
Sources

Partially
o
Supervised
WAM

Graph CoRanking

Opinion
Targets

Opinion
Words

Word Alignment Model

Monolingual word alignment model to capture
opinion relations in sentences.
A noun/noun phrase can find its modifier through
word alignment.
Widely used in many tasks such as collocation
extraction and tag suggestion.
Both time consuming and manually impractical to
label all the alignmnets.

Contd

Here colorful and big are usually used to modify

screen in the cell-phone domain,.
If we know big to be an opinion word, then screen is
very likely to be an opinion target in this domain. Next,
the extracted opinion target screen can be used to
deduce that colorful is most likely an opinion word.

Contd..
Given a sentence with n words S = {w1,w2,wn}.The word
alignment A={(i,ai) | i [1,n] , ai [1,n]} can be written as:
A*=argmaxP(A|S)
Various models of WAM are IBM1,IBM2 AND IMB3 models.
IBM3 model is used here because of better performance.
The optimal alignment generated by IBM-3 is :
P(A|S) n(i,wi) t(wj|waj) d(j|aj,n),
where varies from (i=I to n )
3 main factors are n(i,wi) , t(wj|waj) and d(j|aj,n).
t(wj|waj) models the co-occurrence information of two
words.
d(j|aj,n) models word position information.
n(i,wi) describes the ability of a word for one-to-many
relation.

 In standard alignment model, an opinion target

candidate (noun/ noun phrase) may align with the
irrelevant words rather than potential opinion
words (adjectives/verbs), such as prepositions and
conjunctions.
Some constraints in the alignment model as
follows:
1)Nouns/noun phrases (adjectives/verbs) must be
aligned with adjectives/verbs (nouns/noun
phrases) or a null word. Aligning to a null word
means that this word either has no modifier or
modifies nothing.
2)Other unrelated words, such as prepositions,
conjunctions and adverbs, can only align with
themselves.

Fig. . Mining opinion relations between words

using the word alignment model under
constraints
Here This, a and and, are aligned with
themselves. There are no opinion words to modify
Phone and has modifies nothing; these two words
may align with NULL.

Partially Supervised WAM

To improve alignment performance, we perform a
partial supervision on the statistic model to
incorporate partial alignment links into the
alignment process.
To obtain partial alignments, we resort to
syntactic parsing.
Suppose partial alignment links A is
={(i,ai) | i [1,n] , ai [1,n]}
The optimal alignment A*=argmax P(A|S,)
Two steps involved:
Parameter estimation for the PSWAM.
Obtaining partial alignment links using High
Precision Syntactic Patterns.

Partially Supervised Alignment

Fig. Mining opinion relations between words using partially supervised

alignment model

Parameter Estimation for

PSWAM
The standard EM training algorithm is time
consuming and impractical.
A constrained EM algorithm based on hill-climbing
is then performed to determine all of the
alignments in sentences.
Hill-climbing algorithm is a local optimal solution
to accelerate the training process. It sequentially
trains the simple models (IBM-1, IBM-2) as the
initial alignments for the IBM-3 model.
Next, a greedy search algorithm is used to find
the optimal alignments iteratively.
The search space for the optimal alignment is
constrained on the neighbour alignments of the
current alignment.

Contd
The neighbour alignments could be generated
from the current alignment by one of the following
operators:
1)MOVE operator mi;j, which changes aj = i.
2)SWAP operator sj1;j2 , which exchanges aj1
and aj2 .
Two matrices are created, called the MOVE matrix
M and the SWAP matrix S, to record all possible
MOVE or SWAP costs, respectively, between two
different alignments.
The constrained hill-climbing algorithm involves
two primary steps involved.
1) Optimize toward the constraints.

1) Optimize toward the constraints:

First, the simpler alignment models (IBM-1, IBM-2)
are sequentially trained.
Second, partial alignment links is eliminated by
using the MOVE operator mi;j and the SWAP
operator sj1;j2 .
Third, the alignment is updated iteratively until no
additional inconsistent links can be removed .
2) Towards the optimal alignment under the
constraints:
Aims to optimize towards the optimal alignment
under the constraints.
The final alignment links have a high probability
of being consistent with the pre-provided partial
alignment links.

Constrained Hill Climbing Algorithm

Obtaining Partial Alignment Links

by Using High-Precision Syntactic
Patterns

For training the PSWAM, the other important issue

is to obtain the partial alignment links.
To fulfill this aim, we resort to syntactic parsing.
High-precision-low-recall syntactic patterns are
designed to capture the opinion relations among
words for initially generating the partial alignment
links.
Syntactic patterns are based on the direct
dependency relations.
A direct dependency indicates that one word
depends on the other word without any additional
words in their dependency path or that these two
words both directly depend on a third word.

Contd..

Fig. The types of the

used syntactic
patterns

Fig. Some Examples of Used Syntactic Patterns

Calculating the Opinion

Associations Among Words
From the alignment results, we obtain a set of
word pairs, each of which is composed of a
noun/noun phrase (opinion target candidate) and
its corresponding modified word (opinion word
candidate).
The alignment probabilities between a potential
opinion target and a potential opinion word are
estimated.
P(wt|wo) = Count(wt,wo)/Count(wo).
The opinion association between words are
calculated:
OA(wt,wo) = ( P(wt,wo) + (1)P(wt,wo))^-1
where is the harmonic factor used to combine

ESTIMATING CANDIDATE CONFIDENCE WITH

GRAPH CO-RANKING
After mining the opinion associations between
opinion tar-get candidates and opinion word
candidates, we complete the construction of the
Opinion Relation Graph.
Opinion Relation Graph is constructed to model all
opinion target/word candidates and the opinion
relations among them.
A random walk based co-ranking algorithm is then
proposed to estimate each candidates confidence
on the graph.
Finally, candidates with higher confidence than a
threshold are extracted.

Opinion Relation Graph

A bipartite undirected graph G=(V,E,W) named as

opinion relation graph.
V=Vt U Vo denotes the set of vertices.
E is the edge set of the graph, denotes that there is an
opinion relation between two vertices. There is no edge
between the two of the same types of vertices.
W means the weight of the edge, which reflects the
opinion association between these two vertices.

Estimating Candidate
Confidence by Using Random
Walking

The confidence of each candidate is estimated :

Ct(k+1^th) = (1-) Mto Co(k^th) + It
Co(k+1^th) = (1-) Mto(T) Ct(k^th) +
Io
Ct(k+1^th) and Co(k+1^th) are the confidence
of an opinion target candidate and opinion word
candidate in the k + 1 iteration.
Ct(k^th) and Co(k^th)are the confidence of an
opinion target candidate and opinion word
candidate in the k iteration.
Mto records opinion associations among
candidates.
It and Io denotes prior knowledge of candidates
being opinion targets and opinion words.

Penalizing on High-Degree
Vertices
The standard random walk algorithm could be
dominated by high-degree vertices, which may
introduce noise.
As high-degree vertices link with more vertices,
these high-degree vertices are prone to collecting
more information from the neighbours and have a
significant impact on other vertices when
performing random walks.
If a vertex connects with a high-degree vertex, it
have a larger possibility to be reached by a
walker.
In review texts, these high-degree vertices
usually represent general words

Contd
For example, good may be used to modify
multiple objects, such as good design, good
feeling and good things.
Good is a general word, and its degree in the
Opinion Relation Graph is high.
If design has higher confidence to be an
opinion target, its confidence will be propagated
to feeling and thing through good.
As a result, feeling and thing most likely have
higher confidence as opinion targets. This is
unreasonable.
Meanwhile, the same problem may occur in
opinion word extraction.

Contd
When the random walk reaches a vertex v,
There are three choices for the walker:
(a) continue the random walk to the neighbors
of v.
(b) abandon the random walk .
(c) stop the walk and emit a confidence
according to prior
knowledge.
Thus co-ranking algorithm eqn can be rewritten
using the above three events.
Finally, candidates with higher confidence are
extracted as opinion targets or opinion words.

Conclusion
A novel method for co-extracting opinion targets
and opinion words by using a word alignment
model.
This method captures opinion relations more
precisely and therefore is more effective for
opinion target and opinion word extraction.
An Opinion Relation Graph is constructed to model
all candidates and the detected opinion relations
among them.
Finally, a graph co-ranking algorithm to estimate
the confidence of each candidate. The items with
higher ranks are extracted out.

References
Kang Liu, Liheng Xu, and Jun Zhao, Co-Extracting Opinion
Targets and Opinion Words from Online Reviews Based on the
Word Alignment Model, IEEE TRANSACTIONS ON KNOWLEDGE
AND DATA ENGINEERING, VOL. 27, NO. 3, MARCH 2015.
Kang Liu, Liheng Xu, Yang Liu, and Jun Zhao. 2013.Opinion
target extraction using partially supervised word alignment
model.
M. Hu and B. Liu, Mining and summarizing customer reviews,
in Proc. 10th ACM SIGKDD Int. Conf. Knowl. Discovery Data
Mining, Seattle, WA, USA, 2004, pp. 168177.
M. Hu and B. Liu, Mining opinion features in customer reviews,
in Proc. 19th Nat. Conf. Artif. Intell., San Jose, CA, USA, 2004, pp.
755760.
Liu, Kang, LihengXu, Jun Zhao. Extracting opinion targets and
opinion words from online reviews with graph
coranking,Proceedings of the 52nd Annual Meeting of the
Association for Computational Linguistics. 2014;

Thank You