IMAGE, SPEECH AND LANGUAGE PROCESSING 5-11

Shape Contexts Incorporated to Extended Weighted-Tree Similarity Algorithm

for Agent Matching in Virtual Market

Budianto1 and Riyanarto Sarno2

1
Indonesia Informatics Institute
Jl. Biliton 21-23, Surabaya 60281, Indonesia Tel. +62 31 5046666 Fax. +62 31 5043334
Email: budianto98@yahoo.com
2
Faculty of Information Technology, Sepuluh Nopember Institute of Technology, Surabaya
Kampus ITS, Jl. Raya ITS, Sukolilo – Surabaya 60111, Indonesia Tel. + 62 31 5939214, Fax. + 62 31 5913804
Email: riyanarto@its-sby.edu

ABSTRACT
A weighted-tree similarity algorithm has been successfully used in the e-business environments and power grid transactions.
Using the weighted-tree similarity algorithms, we can compute the similarity of two agents having textual information. This
paper proposes the extension the weighted-tree similarity algorithm in order to compute the similarity of two agents having
visual information. The extension of the algorithm incorporates shape matching with Thin-Plate splines transformation. The
computation results show that the weighted-tree similarity algorithm can be employed in agent matching to obtain the most
appropriate garment.

Keywords: Multi agents system, Similarity Algorithms, Shape Context, Shape Matching , Tree Similarity Algorithm.

1. INTRODUCTION Later, the improvement on the set of keyword model

Distributed Artificial Intelligence (DAI) is a field of
artificial intelligent concern with how groups of agents
solve problem. In DAI a society of problem solver
composed of computers and people interacting each
other to solve a common problem. The society is
called Multi agent system which can be defined as a
network of problem solver that work together to solve
problems that beyond individual capabilities [7, 8, 12].
Moreover the development of the internet has
amplified researches on the virtual market places,
where agent buyer and agent seller meet to coordinate
their preferences and find agreement among them [6].
To enable negotiation among agents in the MAS,
agents bring the owner’s preferences and act on the
behalf of the owner. Many models can be used to
describe the information of agents. In ACORN [9], An
Agent-based Community Oriented Routing Network
use community based approach for searching
distribution and management information in network,
information is carried in the form of a set of key
words.
Using set of keyword can be used to carry the
information of the owner. For example, the owner
wants a blue shirt made from 100% cotton. Yet, set of
keyword model can not carry the preferences of the
owner which has a priority. Now, suppose the owner
wants a blue shirt made from 70% cotton and 30%
polyester; the set of keyword is difficult to represent
such preferences.
is made to carry well-descriptive information. Such
model is in the form of node-labelled, arc-labelled,
arc-weighted tree to represent the agent preferences in
the hierarchical model [4, 11]. In the tree, information
is carried as textual information stored in the node-
labeled with weighted-arc represents the level of
preference.
The form of node-labelled, arc-labelled, arc-weighted
tree has been successfully used in several applications
such as e-learning and power grid transaction. To
improve the capabilities of the tree, we add a node
shape, which carry bring not only textual information
but also visual information.
This paper is divided into 7 sections: section 2 reviews
Multi Agent System architecture; section 3 describes
the weighted-tree similarity algorithms. Next on section
4, we discuss about how node shape in the arc-labeled
arc-weighted tree carry visual information. Section 5
describe about shape context incorporated to weighted-
tree similarity algorithms. And finally, Section 6
discusses about the shape context and section 7
presents the experiment result and the conclusion
respectively.
2. MULTI AGENT SYSTEM ARCHITECTURE
ACORN (Agent-based Community Oriented Routing
Network) as one of multi-agents system use
community based approach for searching distribution
and management information in network [9]. The

Seminar Nasional “Soft Computing, Intelligent Systems and Information Technology” (SIIT 2005)
5-12 IMAGE, SPEECH AND LANGUAGE PROCESSING

information carried by agents is routed around 3. A WEIGHTED-TREE SIMILARITY ALGO-

networks of people. Such information can be any
electronic information such as document, images,
sound, and so on.
Figure 1 shows an ACORN architecture which adopts
client/server architecture. In ACORN system, a user
creates an agent as his/her representative on the
community through the web browser. The agent
works autonomously on the owner’s behalf based on
the information given. This information is stored in a
separate user profile file on the server site which
contains user information, user preferences, references
to agent sent and receive and about the people in the
user’s database. In addition, agent in the ACORN also
has capabilities to move from one server to another
server if needed.
Web Browser
RITHMS
In ACORN, information carried by agents is
represented as a set of key words with their weights
[9]. In the real world cases, such flat information can
not give the whole description about the agent’s
preferences. To solve the problem, bhasvar propose
the form of arc-labeled arc-weighted tree to represent
the agent preferences [4]. The description of seller and
buyer agents’s preferences are represented using node-
labelled, arc-labelled, arc-weighted trees. The arc-
labeled arc-weighted tree has been suitably applied as
representation of knowledge of an agent. Using
weighted-tree similarity algorithms, we can set up the
matching process between two trees and find the
similarity between them. Then, the pairing and the
negotiation process would be the next process between
two agents with best similarity to make an agreement
[11].
Garment Garment

Main Server Agent Type

Age Type Age 0.4
0.3 0.4 0.3 Occasion
Directory Server Agent Occasion 0.3

Anonymity Server Agent Adult Formal Shirt Adult Casual Shirt

User Info
User Profiles
Garment
. . . Type
. Age 0.4
User Agents . 0.3 Occasion
To 0.3
. . other
Acorn Adult Formal T-Shirt
Site
Cafe Figure 2. Example Trees of Garment Item
... In the arc-labeled arc-weighted tree, the arcs will
Café-1 Café-n always be labeled in alphabetical order. Moreover,
bold font is used to difference between arc labels and
node labels. Figure 2 shows the example trees of
garment.
Figure 1. ACORN Architecture (adopted from [9])
The depth of the tree is not limit with one or several
One capabilities of ACORN architecture is to provide level. Bhavsar et.al. then used RuleML to represent
information sharing among agents. Café is a place the tree in object orient model [4, 5]. Figure 3 shows
where agent can share its information among the RuleML of Figure 1(a). Element _r is a metarole
contains the names and weights of arc labels.
themselves. at a certain time, all agents give their
information to the café manager. The café manager <cterm>
compares all data, sharing out any relevant data to <_r n=”Age” w=”0.3”>
agent who would find it of use. Moreover, the café <ind>Adult</ind></_r>
manager maintains the mingling of agents in the café. <_r n=”Occasion” w=”0.3”>
Since the sharing information by café manager, each <ind>Formal</ind></_r>
agent will know about other agents and it can contacts <_r n=”Type” w=”0.4”>
interested agent to learn more. <ind>Shirt</ind></_r>
</cterm>
Figure 3. Symbolic Tree Representations [4]

Seminar Nasional “Soft Computing, Intelligent Systems and Information Technology” (SIIT 2005)
 IMAGE, SPEECH AND LANGUAGE PROCESSING 5-13

For similarity algorithms, the weighted-tree similarity
algorithm recursively traverses every pair of trees and
calculate the similarity on each node to find the
similarity between two trees. The algorithm compute
the average of weights from two trees, ( wi + wi' ) / 2 and
obtain the similarity si of tree t1 and tree t2 recursively
by multiply an arc function A(Si) and the averaged
weight. Then Sum of A( Si )( wi + wi' ) / 2 is divide by
sum of all averaged weights [4]:
describe his information not only using node label but
also using node color, node shape and node pattern.
Using the previous arc-labeled arc-weighted tree [4],
he would find difficulties to describe the shape using
node label. However, it is easy to describe the shape of
a dress using node shape. The shape matching using
shape context [1, 2, 3] used in this research is
employed to the problem.

∑ ( A(S )(w + w ) / 2 )/ ∑ (w + w ) / 2
i i
'
i i
'
i (1) 5. SHAPE CONTEXTS FOR SIMILARITY
ALGORITHM
Later the weighted-tree similarity algorithm is refined
by applying weight variance on the selection of the We propose the use of shape contexts for the tree
most preferable agent when several agents have the describing visual information of garment. Then Shape
same similarity [13]. Here is the formula of weighted- matching techniques can be used to find similarity.
tree similarity algorithm with weight variance [11]: According to Russel et.al. [10], shape matching in
n
= (∑ (( wi − wi ) 2 + ( wi' − wi ) 2 )) / n (2) computer vision can be categorized into two
var iance
1
approaches: (a) feature-based, and (b) brightness-
based.
where w = i ( wi − wi' ) / 2 and n = number of weight
pairs. Using variance to choose the preferable agents is
done by selecting the smaller variance.

4. NODE SHAPE ON TREE SIMILARITY

Garment

Figure 5. Examples of Edge Pixels Generated from

Dress
Age Fabric
Avail-
Gender
Price In this research, we propose the feature-based approach
ability Occasion Type using shape context matching developed by Belongie
{Adult, {1 day, {M, L} {Jacket, et.al. [1, 2, 3]. In Shape context matching, a set of
Baby, 2 days, Minor
{Forma, {Cheap, t-shirt, …} points defines the internal or external contour of shape
…} …} Casual, …}
Major …} is used. Figure 5 shows the example of set point from
shape of a dress. Given set P = {p1, p2,..,pn}, pi ∈ R2,
Button
Style
{Cotton, {Cotton, Sleeve each point pi in the first shape is matched to the point q i
Silk, …} Silk, …} Back Collar
Shape Front on the second shape which have the best
Shape correspondence each other. To match between two
{Circle, {Long,
correspondence points, we use the shape context, which
Square, {V-Style, Short, plays an important role for one-to-one point matching.
…} Circle, …}
…} Suppose we have a point used as a reference point, n-1
vector can be formed from the reference point. Now we
can identify the distribution over relative positions from
the reference point. The coarse histogram hi of the
Figure 4. Garment Tree as Representation of User’s relative coordinated of the remaining n-1 points is
Preferences employed to identify the distribution. The coarse
histogram hi is defined as [1, 2, 3]:
In this research, we build the arc-labeled arc-weighted
tree for garment as seen in Figure 4 and include shape Hi =#{q≠pi: (q-pi)∈ bin (k)} (3)
matching to unable weighted-tree similarity algorithm Histogram hi is called to be the shape context of pi.
to match the shape of a cloth or dress in the trees. With The shape context is counted using the log polar
the arc-labeled arc-weighted tree, a buyer/seller can coordinate with the diagram of log polar bins (see

Seminar Nasional “Soft Computing, Intelligent Systems and Information Technology” (SIIT 2005)
5-14 IMAGE, SPEECH AND LANGUAGE PROCESSING

figure 6), simply by counted the number of pixel in
each the polar bins.
After we finished calculating all histogram for each pi
in the first shape and the second shape, we need to find
cost function Cij = C (pi, qj) where pi is a point in the
first shape and q j is a point in the second shape. Based
on the histogram, now we can compute the matching
costs using Chi Squared Test χ2:
1 n [hi (k ) − h j (k )]
2
Similarity between two node shapes is counted based
on the shape context distance. Thus, we should
measure the context distance D(P,Q) between shapes
P and Q as the sum of shape context matching costs
(5) over the best matching points, as follows [1, 2, 3]:
1
D ( P, Q ) = ∑ arg min q∈Q C ( p, q)
n p∈P
(5)
where Dbe ( P, Q ) is the effort of transformation done
to adjust the shapes(see [9]).

Cij = ∑ (4)
2 k =1 hi (k ) + h j (k ) For finding the similarity between two shapes, we
modify the equation (6) by subtract 1 with the context
distance D (P, Q) such that the result will reflect the
similarity, as follows:
Sm i ( P, Q ) = 1 − D ( P, Q ) (6)

The weighted-tree similarity algorithm employs the

similarity between pair node shapes in the tree in order
to compute the similarity of the tree.

6. RESULTS

Count = 3

Figure 6. Examples of Edge Pixels Generated from

Dress

where hi(k) and hj(k) denote the K-bin normalized

histogram at pi and qi, respectively. For all pair i and j,
we compute the cost function Cij in order to find the Figure 8. Examples of Edge Pixels Generated from
minimum total cost of matching subject to the Dress
constraint that the matching be one-to-one. Figure 7
show the example of one-to-one point matching Here, we present results on the cloth shape data set
between two shape. from http://www.loeshinsedesign.com. In our
experiments, we use 8 shapes of coats. In the
experiments, we defined the similarity as equation 6.
Figure 8 shows all coats shape used in this experiment.
For each cloth, we provide the back and the front of
the coat respectively.
Table 1. The Similarity of Front Shape for Each Item
in figure 8
A B C D E F G H
A 0,025 0,029 0,028 0,025 0,023 0,028 0,035
B 0,025 0,024 0,025 0,022 0,025 0,027 0,040
C 0,029 0,024 0,023 0,027 0,027 0,028 0,040
D 0,028 0,025 0,023 0,026 0,025 0,028 0,036
E 0,025 0,022 0,027 0,026 0,015 0,023 0,042
F 0,023 0,025 0,027 0,025 0,015 0,024 0,036
G 0,028 0,027 0,028 0,028 0,023 0,024 0,038
Figure 7. Examples of Edge Pixels Generated from H 0,035 0,040 0,040 0,036 0,042 0,036 0,038
Dress

Seminar Nasional “Soft Computing, Intelligent Systems and Information Technology” (SIIT 2005)
 IMAGE, SPEECH AND LANGUAGE PROCESSING 5-15

Using shape contexts, the Table 1 and Table 2 show Table 4. Result of weighted-tree similarity algorithm
the similarity between the front of the cloth and the computation on each arc-labeled arc-
back of the cloth respectively. The computation of weighted tree for each item in figure 8
similarity between two node shapes is based on the A B C D E F G h
equation 3, 4, and 5. A 1.000 0.967 0.986 0.986 0.987 0.968 0.971 0.964

B 0.967 1.000 0.969 0.967 0.970 0.970 0.965 0.965

Table 2. The Similarity of Back Shape for Each Item
C 0.966 0.969 1.000 0.971 0.971 0.971 0.986 0.988
in figure 8.
D 0.986 0.967 0.971 1.000 0.989 0.989 0.966 0.969
A B C D E F G H E 0.987 0.970 0.971 0.989 1.000 0.998 0.967 0.968
A 0,057 0,059 0,062 0,055 0,055 0,030 0,063
F 0.987 0.970 0.971 0.989 0.998 1.000 0.967 0.968
B 0,057 0,047 0,055 0,039 0,039 0,064 0,052
G 0.971 0.965 0.986 0.966 0.967 0.967 1.000 0.985
C 0,059 0,047 0,034 0,031 0,031 0,061 0,036
H 0.964 0.965 0.988 0.969 0.968 0.968 0.985 1.000
D 0,062 0,055 0,034 0,042 0,042 0,061 0,033
E 0,055 0,039 0,031 0,042 0,000 0,060 0,036
F 0,055 0,039 0,031 0,042 0,000 0,060 0,036
G 0,030 0,064 0,061 0,061 0,060 0,060 0,056
7. CONCLUSION
H 0,063 0,052 0,036 0,033 0,036 0,036 0,056
We have improved the weighted-tree similarity
algorithm to match two agents having visual
After applying shape matching on each node shape on information. The visual information is carried in the
the tree, the similarity of two tree is counted using leaf nodes, which represent the shapes of garments.
equation 1. The computation result on weighted-tree The improvement of the algorithm incorporates shape
similarity algorithm is recorded in Table 3. context to find similarity between visual information.
The results of computation study show that the
Table 3. The Dataset Used for The Experiments. weighted-tree similarity algorithm can be employed in
A B C D E F G h agent matching to obtain the most appropriate garment.
age Adult Adult Adult Adult Adult Adult Adult Adult

avail 1day 1day 1day 1day 1day 1day 1day 1day

major polyester polyester polyester polyester polyester polyester polyester polyester REFERENCES
minor cotton cotton cotton cotton cotton cotton cotton cotton

gender F F F F M F F F [1] Belongie, S., Malik, J., and Puzicha, J., Matching
occasion Formal Formal Formal Formal Formal Formal Formal Formal Shape, Eight IEEE International conference on
price Normal Normal Normal Normal Normal Normal Normal Normal Computer Vision, July 2001.
type Coat Coat Coat Coat Coat Coat Coat Coat [2] Belongie, S., Malik, J., and Puzicha, J., Shape
button StyCircle Strap None Circle Circle Circle None None Context: A New Descriptor for Shape Matching
Back Shp Fig(a) Fig(b) Fig(c) Fig(d) Fig(e) Fig(f) Fig(g) Fig(h) and Object Recognition, In NIPS, November
Collar V-Style V-Style V-Style V-Style V-Style V-Style V-Style V-Style 2000.
Front Shp Fig(a) Fig(b) Fig(c) Fig(d) Fig(e) Fig(f) Fig(g) Fig(h)
[3] Belongie, S., Malik, J., and Puzicha, J., Shape
Sleeve Long Long Long Long Short Long Long Long
Matching and Object Recognition using Shape
In our experiment, we use build the arc-labeled arc- Context, Technical Report UCB//CSD-00-1128,
weighted tree for garment and include shape matching UC Berkeley, January 2001.
to unable weighted-tree similarity algorithm to match [4] Bhavsar, V.C., Boley, H., and Yang, L., A
the shape of a cloth or dress in the trees. The arc weighted-tree similarity Algorithm for Multi-
weighted in the tree is used to represent level of the Agent System in E-Business Environment, 2003.
preferences of the owner as shown in figure 4. The [5] Boley, H., The Rule Markup Language: RDF-
value of attribute in the arc-labeled arc-weighted tree XML Data Model, XML Schema Hierarchy, and
for each item in figure 2 is summarized in table 3. XSL Transformations, Invited Talk, INAP 2001,
Tokyo, October 2001. LNAI 2543, Springer-
The result of similarity on each item can be seen in Verlag, 2003.
table 4. The value in the table represents the degree of [6] Budianto, An Overview and Survey on Multi
similarity between two trees. Agent System, Soft Computing, Intelligent
Systems and Information Technology National
Seminar, July 2005.
[7] Green S. Hurst, L., Nangle, B. Cunningham, P.,
Somers, F., and Evans, R., Software Agents: A
Review, 1997.

Seminar Nasional “Soft Computing, Intelligent Systems and Information Technology” (SIIT 2005)
5-16 IMAGE, SPEECH AND LANGUAGE PROCESSING

[8] Jennings, N.R., Sycara, K., and Wooldridge, M.,

A Roadmap of Agent Research and Development,
Autonomous Agents and Multi-Agent system, 1,
pp. 275-306, 1998.
[9] Marsh, S., Ghorbani, A., and Bhavsar, V.C., The
ACORN Multi-Agent system, Web Intelligence
and Agent Systems: An International Journal 1,
pp. 65-86, 2003.
[10] Russel, S.J. and Norvig, S., Artificial Intelligence:
A Modern Approach Second Edition. Prentice
Hall, 885-892.
[11] Sarno, R., Yang, L., Bhavsar, V.C., and Boley, H.,
Agent Matcher Architecture Applied to Power
Grid Transaction, 2003.
[12] Stone, P. and Veloso, M., Multiagent Systems: A
Survey from a Machine Learning Perspective, in
Autonomous Robotics volume 8, number 3, July
2000.

Seminar Nasional “Soft Computing, Intelligent Systems and Information Technology” (SIIT 2005)