2010 IEEE 21st International Symposium on Personal Indoor and Mobile Radio Communications
Mobile Image Search for Tourist Information
Using ACCC Algorithm
Wichian Premchaiswadi, Anucha Tungkatsathan
Graduate School of Information Technology in Business
Siam University
Bangkok, Thailand
wichian@siam.edu, aimdala@hotmail.com
Abstract— Almost all mobile image search applications available
in the market only support text queries, which it is sometimes
difficult to describe the needs in text and the text input is
inconvenient on small devices. This paper proposed a mobile
phone image search application that can perform practical image
search processing in real-time. The early stage of the project is
aimed to develop the application for the tourists who can use it as
a guidance tool for travelling in the country. The application can
run on hundreds of different mobile phone types and platforms.
In older to speed up the visual computation for searching images,
the Auto Color Correlogram (ACC) and correlation (ACCC)
algorithms based on Correlogram technique were developed.
Applying the ACCC algorithm, it reduces the computational
complexity of index construction and similarity measurement for
images from O(m2d) to O(md) when compared to previous
algorithm, Color correlogram.
Keywords- Mobile Search and Retrieval; Mobile Image Searh;
Content-Based Image Retrieval; Tourist Information; ACCC
Algorithm.
I. INTRODUCTION
Currently, mobile phones perform like a personal computer.
Almost all models of mobile phone being built today have a
high efficiency and resolution digital camera and are provided
with a GPRS or WiFi function for connecting to the Internet
effortlessly. Therefore, it is possible to use mobile phones to
search for information though the Internet by using images
rather than keywords. The term ‘‘Mobile Tourism’’ represents
a relatively new trend in the field of tourism and involves the
use of mobile devices as electronic tourist guides [1]. Mobile
devices present many unique characteristics that make their use
as electronic tourist guides particularly attractive, such as
ubiquity and convenience; positioning: by employing
technologies like GPS, users may receive and access
information and services specific to their location [1, 2]. The
existing mobile tourism solutions typically support text queries.
Therefore, users have to convert their information needs into
words. However, it is sometimes difficult to describe the needs
in text and the text input is inconvenient on small devices [3].
The content-based image indexing algorithms have been
rapidly developed and improved in term of the accuracy and
the speed so that the existing algorithms for image indexing
and retrieval can be applied to some realistic image retrieval
applications. For example, Google has developed and launched
978-1-4244-8016-6/10/$26.00 ©2010 IEEE
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on June 28,2023 at 07:57:05 UTC from IEEE Xplore.
Nucharee Premchaiswadi
Faculty of Information Technology
Dhurakijpandit University
Bangkok, Thailand
nucharee@dpu.ac.th
an image search application on mobile phones, namely Google
Goggles [4]. Google Goggles allows us to search pictures from
a camera phones. This application uses a query image and
several image recognition backends (object recognition, place
matching, OCR, etc) to search different kinds of objects and
places such as text, landmarks, books, contact info, places,
wine, and logos that are similar to the intended target. In
addition, Plink [5] has just been acquired by Google. Plink is
an art-related image that enables to identify paintings and
artworks with just a snap from a phone’s camera. Once
recognized, we can read information on the artwork and artist,
share favorite pieces with friends. Another some demo
application is Mobile Visual Search Engine on the Apple
iPhone [6] from the independent vender Evolution Robotics
[7]. It returns the image query results to the users via e-mail.
However, it requires the mobile visual search engine ViPR [7]
to be installed and the application can run on iPhone only.
Based on this observation, their applications still have some
limitations. For example, Google Goggles and Plink can be run
on smart phone and specific on Android market, and requires
engine to be installed. While the iPhone cannot return the
image query results in real-time. Owing to these limitations, the
applications are incompatible with general or standard models
of mobile phones or different platforms of mobile phones. In
this paper, we propose an image search application that can be
compatible with different mobile phone models and platforms.
The application can solve the limitations found in both Google
Goggles and ViPR mobile visual search Engine on the Apple
iPhone. The application can run on both mobile phone and
personal computer that connect through the internet without the
need of software installation. Moreover, it can be run on any
mobile phone operator/network. Working on personal
computers, the application can run on different operating
system platforms such as windows, Mac, or Linux that support
a standard web browser. In addition to the application, we
propose a fast and robust color image indexing method, which
is suitable for real-time processing in querying images from a
large database. The Auto Color Correlogram (ACCC) [8]
algorithm based on Correlogram technique is developed.
Applying the ACCC, it reduces the computational complexity
of index construction and similarity measurement between
images from O(m2d) to O(md) when compared to the previous
algorithm, Color correlogram [7].
2557
Restrictions apply.
 II. RELATED WORK
There are many paper contribute to Tourist Information
system such as [10] [1] [11] [12]. However, to conduct a
complete and thorough review of the literature is beyond the
scope of this paper. We will only look at some applications
found in the literatures which are related to our system for
mobile image retrieval. For instance, Xin Fan et al.[3]
proposed Photo-to-Search system which allows users to input
multimodal queries, including duplicate image detection,
content-based image retrieval, text-based Web image search,
and key phrase extraction in an ensemble system to provide a
feasible solution to support multimodal queries from mobile
devices. However, the Photo-to-Search implemented a
prototype server-side program to receive the query from an
email address by the POP3 protocol and to process the query.
Tom Yeh [13] presented Photo-based question answering. It is
a useful way of finding information about physical objects. A
photo-based QA system allows direct use of a photo to refer to
the object. They present three prototypes of photo-based QA
system including an online album, a text-based QA, and a
mobile application. For image matching process, The algorithm
depend on types of input images queries, such as images of
posters , magazine covers , video frames , CD covers , grocery
items , and buildings. Xavier Anguera [14] presented a
multimodal and mobile image retrieval prototype named
MAMI (Multimodal Automatic Mobile Indexing). It allows
users to annotate, index and search for digital photos on their
phones via speech or image input. The experimental results
show that multimodal fusion algorithms have higher accuracy
than their monomodal counterparts and run in real-time on the
phone but the limitation of MAMI is that it is implemented as a
stand-alone application. Premchaiswadi [15] proposed an
image search for tourist information using a mobile phone. The
system was implemented and tested with real mobile phone
queries. The experimental results show that the proposed
system could be applied for practical uses. The Auto-
correlogram and Color Different Correlogram (AC/CDC)
algorithm [16] are utilized in the image retrieval process.
However, AC/CDC algorithm also needs to be improved the
query for a large image database. Moreover, the system
requires the software to be installed and run on window mobile
5.0 platforms only. A detail comparison of an existing image
search application and the proposed application is present in
fig. 1.
III. PREPARE YOUR PAPER BEFORE STYLING
In order to design successful mobile device applications, it
involves factors related to the technical characteristics of the
device. There are many challenges in designing the system
such as: design for mobility; design for a wide audience with
various levels of competency in the use of new technologies,
that do not necessarily have a history of experience with
similar applications to draw on in learning a new one; design
for limited input (small and impractical keyboard)/output
facilities (small screen size, limited colors available and font
size support); and design for user multitasking at levels
unfamiliar to most desktop users [17, 18]. The details of the
implementation are as follows:
2558
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on June 28,2023 at 07:57:05 UTC from IEEE Xplore. Restrictions apply.
Figure 1. Comparison of several functions in an existing image search
application and the proposed application.
A. Basic principle
Before introducing our framework of mobile phone image
search in real-time processing for tourist information and travel
tips, we will briefly examine the properties of the queries to be
answered. Our method uses the following major steps: (a) The
picture is captured and sent over the internet to the web server
through GPRS. (b) Web-services on the web server side are
provided to handle incoming pictures from the mobile phone.
The mobile phone application interacts with the web service in
a manner prescribed by its description using SOAP messages,
typically conveyed using HTTP with an XML serialization in
conjunction with other web-related standards. (c) The mobile
phone picture file is indexed and compared with the indexes of
all pictures in the database by using the auto color correlogram
and correlation (ACCC) algorithms. (d) After finishing the
index comparison, the top ten similar pictures will be returned
and displayed using the mobile phone application through
GPRS. (e) The tourist information such as text, audio, video
and map are showed by user selected. The framework of image
search for tourist information using a mobile phone is
presented in Fig. 2.
B. The Proposed System of Mobile Image Search for Tourist
Information.
In the first step, the CBIR engine creates an index for images
using the new algorithm that we have developed, namely Auto
Color Correlogram (ACCC). The main functions of CBIR
engine are to explore the image color characteristic values of
each preference image and to record them into an image
indexing database. The speed and the efficiency of ACCC
algorithm are proved and shown in section IV. In addition to
explore the characteristics of the images, the CBIR engine also
calculates the similarity measure among images, ranks and
sorts all the images, and returns them to the users in real time.
We develop the application that can run on a standard web-
browser and any operating system platform. The mobile and
server communicates each other through the mobile web
interface based on web services. The web service named
“WsSIAMTO T.asmx” is developed. The details of web
services are described as follows:
 Figure 2. The overview of the proposed system

technique, the correlation values of colors for each color

• Web Services Name is Service.asmx. distribution of an image are computed efficiently. An auto
color correlation (ACC) [8] expresses how to compute the
• Web Method Name is SaveFileAsType. This mean color of all pixels of color Cj at a distance k-th from a
SaveFileAsType method converts all image file
pixel of color Ci in the image. Formally, the ACC of image
formats such as *JPG, *.GIF, *.BMP, *.ICO to binary
{I(x,y), x = 1,2,…,M, y = 1,2,…,N } is defined as
file and save them into the database. The data type
specified in the database is BLOB (Binary Large
Object). ACC (i, j, k ) = MC jγ c(ikc)j ( I )
(1)
• Web Method Name is UpLoadPic. This UpLoadPic ⎧ ⎫
= ⎨ rmcjγ c(ikc)j ( I ), g mcjγ c(ikc)j ( I ), bmcjγ c(ikc)j ( I ) | ci ≠ c j ⎬
method returns all retrieved pictures to the mobile ⎩ ⎭
phone application.
Where the original image I(x,y) is quantized to J colors
IV. FEATURE COMPUTATION C1,C2,…,CJ and the distance between two pixels d [min{M,N}
Practical applications of image retrieval typically require ] is fixed a priori. Let MCj is the color mean of color Cj from
real-time processing for large image database systems. The color Ci at distance k in an image I. The mean colors are
development and implementation of an image search computed as follows:
application is a complex process, requiring a comprehensive
understanding of both the theory and practice of image 1
processing. ⎛ N ⎞ N
rmcjγ c(ikc)j ( I ) = ⎜⎜ ∏ Γcki , rc j ( I ) ⎟⎟ | ci ≠ c j
A. Visual Feature Extraction ⎝ i =1 ⎠
1 (2)
In real word application, an efficient descriptor technique for ⎛ N ⎞ N
g mcjγ c(ikc)j ( I ) = ⎜⎜ ∏ Γcki , gc j ( I ) ⎟⎟ | ci ≠ c j
image feature extraction and visual similarity are still required ⎝ i =1 ⎠
to reduce the processing time. There are various visual 1

descriptors used to extract a low-level feature vector of an ⎛ N ⎞ N

bmcjγ c(ikc)j ( I ) = ⎜⎜ ∏ Γcki ,bc j ( I ) ⎟⎟ | ci ≠ c j
image. In this paper, we used color descriptors for retrieving ⎝ i =1 ⎠
images. The color correlogram is an efficient feature extraction
techniques used in content-based image retrieval (CBIR) where Cj ≠ 0 and N is the number of accounting color Cj from
systems. However, the it is expensive to compute and color Ci at distance k , defined by:
unsuitable for real world application, the computation time of
the correlogram is O(m2d). In this section, we present an ⎧ P( x1 , y1 ) ∈ Ci | P ( x2 , y2 ) ∈ C j ;⎫
efficient image representation technique by using spatial color k ⎪ ⎪
N =Γ ci ,c j (I ) = ⎨ ⎧ ⎫ ⎬
feature namely Auto Color Correlogram and correlation. It is ⎪ k = min ⎨ x1 − x 2 , y 1 − y 2 ⎬ ⎪
the integration of Autocorrelogram [9] and Auto Color ⎩ ⎩ ⎭ ⎭
Correlation [6] techniques. However, we can compute the color Although ACC is able to find the local spatial correlation
distribution values and color correlation values concurrently. between colors by reducing the size of color correlogram from
The size of ACCC is still O(md). Using this proposed
O(m2d) to O(md) it does not consider the color distribution

2559
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on June 28,2023 at 07:57:05 UTC from IEEE Xplore. Restrictions apply.
values of each color in an image. Autocorrelogram is an
efficient algorithm to solve this problem. Thus, we propose an
extended technique of ACC based on the autocorrelogram,
namely Auto Color Correlogram and Correlation (ACCC). It
not only captures the spatial correlation between identical
colors but also computes the local spatial correlation between
colors. The Auto Color Correlogram and Correlation is defined
as
⎧ ⎫
ACCC ( j, j, k ) = ⎨ γ c(ik ) ( I ), MC j γ c(ikc)j ( I )⎬
⎩ ⎭
In order to gain a deeper understanding of the ACCC’s
computational procedure, it is described as follows.
Algorithm: Auto Color Correlogram and Correlation
For every K distance {
For every X position
For every Y position {
Ci ← current pixel
While (Cj ← Get neighbor pixel of Ci at distance K) {
For every color Cm {
If ( = Ci)
If ( Ci ≠ Cj){
colorCount[k][Ci]++
colorR[k][Ci] = colorR[k][Ci] + colorRCj
colorG[k][Ci] = colorG[k][Ci]+ colorGCj
colorB[k][Ci] = colorB[k][Ci] + colorBCj
⎧colorR[k ][Ci ] /(colorCount[k ][Ci ] * k * 8) ⎫
⎪ ⎪
ACC[k ][Ci ] := ⎨colorG[k ][Ci ] /(colorCount[k ][Ci ] * k * 8) ⎬
⎪colorB[k ][C ] /(colorCount[k ][C ] * k * 8) ⎪
⎩ i i ⎭
}else{
colorCount[k][Ci]++
color[k][Ci] = color[k][Ci] + 1
CC[k][Ci] = color[k][Ci] / (colorCount[k][Ci] * k*8)
}}}}}
B. Image similarity measure scheme
The type of similarity measure to be considered depends on
the technique used for feature extraction. In this paper the D1
norm is used because it is simple and robust [10]. The current
section presents similarity measures for the proposed color
descriptor in the case of image retrieval. Let the ACCC pairs
for the m color bin be (αi,βi) in I and (α’i,β’i) in I’. The
similarity of the images is measured as the distances between
the AC’s and ACC’s d(I,I' ) and is applied from [8] as follows:
⎧⎪
α i − α i' β i − β i' ⎫⎪
d ( I , I ' ) = ⎨ λ1 ∑ + λ 2∑ '⎬
∀i 1 + α i + α i ∀i 1 + β i + β i ⎪
'
⎪⎩
Where λ1 and λ2 are the similarity weighting constants of
autocorrelogram and auto color correlation, respectively. In the
experiments conducted, λ1 = 0.5 and λ2 = 0.5. α and β are AC’s
and ACCC’s respectively.
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on June 28,2023 at 07:57:05 UTC from IEEE Xplore. Restrictions apply.
V. IMPLEMENTATION AND TESTING
We have implemented a mobile image search application
and the aim of this implementation is to offer the tourists for
the use as a guidance tool for travelling in Thailand. The
application is developed by using Microsoft .NET and
implemented on Quad-Core Intel Xeon processor E5310 1.60
GHz, 1066 MHz FSB 1 GB (2 x 512 MB) PC2-5300 DDR2,
and tested on the Windows NT environment. Microsoft SQL
Server 2005 is used for database development in this research.
(3) The pictures in a database have different sizes, pixel resolution
and file formats (JPEG, BMP, and GIF).
A. System testing
Fig. 3 shows the scenarios for an application. They are
described as follows: (1) take a picture; a tourist uses his or
her mobile phone to take a picture of an interesting point that
he likes to know the information of that point or interesting
places around that point and then save that picture into the
memory. Second, (2) run the application, (3) select a picture
for querying and search for similar images, (4) return the
images with necessary information to the tourist (a tourist
must make his/her own decision for picking up the retrieved
picture that exactly matches with the place he or she sees), and
(5) display other necessary information such as map, a list of
hotels nearby, gas stations, restaurants, and so on around that
point to the tourist if requested.
B. Performance Measures
To evaluate the accuracy of retrieval process, they were
tested on a database from containing 3,600 images that were
being captured from famous places of interest in Bangkok in
Thailand. The image database consists of images in various
formats such as JPEG, BMP and GIF in various sizes. We
used the distance metrics that were presented in section IV (B)
for comparing feature vectors. Sixty-four colors and {1, 3, 5,
7, 9} for spatial distance were used in the computation of all
algorithms in this experiment. The metrics that we used for
measuring the accuracy of queries are r-measure and p1-
measure, where r-measure is the sums up of the rank of correct
answer of all queries and average r-measure is the r-measure
divided by the number of queries q, while b) p1-measure is the
sum of the precision with the recall equal to 1 and the average
p1-measure is the p1-measure divided by q. r-measure and p1-
measure are defined as
r − measure = ∑i =1 Rank (Qi )
q
r − measure
Avg r − measure =
(4) q
⎭ 1
p1 − measure =
∑
q
i =1
Rank (Qi )
(5)
p − measure
Avg p1 − measure = 1
q
2560
 1 2 3

5 4

Figure 3. A real system testing and the scenarios in an application

VI. CONCLUSION
This paper proposed the mobile phone image search
TABLE I. PERFORMANCES OF VARIOUS METHODS
application that can perform practical image search processing
Method AC ACC ACCC in real-time. The systems architecture for multimodal of mobile
r-measure 494 548 227 phones was proposed and the new algorithm, namely Auto
Avg r- measure 4 5 2 Color Correlogram and correlation (ACCC) algorithm based on
p1- measure 74 71 89
Avg p1- measure 0.76 0.72 0.86
Correlogram technique, was developed for searching images
from a large image database. Applying the ACCC, it reduces
the computational complexity of index construction and
similarity measurement for images from O(m2d) to O(md).
Based on ACCC technique, we tested on a database of 3,600
images of places of interest in Bangkok, Thailand. The results
clearly show that the ACCC algorithm performs higher
efficiency than the AC algorithm for retrieving images.
Moreover, the ACCC algorithm consumes less processing time
than the CC algorithm, which it possible to apply in practice
mobile application for real-time processing.
REFERENCES
[1] M. Kenteris, D. Gavalas, and D. Economou, An innovative mobile
electronic tourist guide application. London: Springer-Verlag, 2007, ch.
13.
[2] Varshney U. “Issues, requirements and support for locationintensive
mobile commerce applications,” Int J Mob Commun, 2003, 1(3) pp.
247–263.
Figure 4. The example of resulting images from query image based on [3] Xin Fan et al. “Photo-to-Search: Using Multimodal Queries to Search
ACCC algorithm, first row is original image. the Web from Mobile Devices,” ACM, MIR’05, November, 2005,pp.
10-11.
[4] http://www.google.com/mobile/goggles/#landmark

2561
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on June 28,2023 at 07:57:05 UTC from IEEE Xplore. Restrictions apply.
[5] http://www.plinkart.com/
[6] http://www.applevideos.info/?p=145
[7] http://www.evolution.com/core/ViPR/
[8] T. Anucha el al., “Spatial Color Indexing using ACC Algorithms,”
Proceeding of the ICT&KE, 2009. pp. 113-117.
[9] J. Huang et al, “Spatial Color Indexing and Applications,” in proceeding
of Sixth International Conference on Computer Vision, 1998, pp. 606 –
607.
[10] E. Meng-Yoke Ta et al., “An Analysis of Services for the Mobile
Tourist,” ACM, MC'07 (Mobility'07), September, 2007, pp. 10-12.
[11] M. v. Setten, S. Pokraev and J. Koolwaaij, Context-Aware
Recommendations in the Mobile Tourist Application COMPASS.
Spring LNCS 3137, 2004, pp. 235-244.
[12] Z. Wa et al, “Personalized Tourism Information System in Mobile
Commerce,” IEEE, ICMECG’09, October, 2009, pp. 387 – 391.
2562
Authorized licensed use limited to: REVA UNIVERSITY. Downloaded on June 28,2023 at 07:57:05 UTC from IEEE Xplore. Restrictions apply.
[13] T. Yeh, J. J. Lee, T. Darrell, “Photo-based Question Answering,” ACM,
MM’08, October , 2008, pp. 26–31.
[14] X. Anguera, J. J. Xu, N. Oliver, “Multimodal Photo Annotation and
Retrieval on a Mobile Phone,” ACM, MIR’08, October, 2008, pp. 30–
31.
[15] W. Premchaiswadi, “A Mobile Image Search for Tourist Information
System,” WSEAS’ 09, February, 2009, pp. 62-67.
[16] W.Premchaiswadi, N.Premchaiswadi, S. Chimlek and S.Narita, “Image
Indexing and Retrieval using Autocorelogram and Color Difference
Corelograms (AC/CDC),” ICFS’2002, March, 2002.
[17] S. Brewster, M. Dunlop. Mobile human–computer interaction—Mobile
HCI 2004. Springer LNCS 3160, 2004, ISBN: 3-540-23086-6
[18] W3C Mobile Web Best Practices 1.0, Basic Guidelines, W3C Candidate
Recommendation, http://www.w3.org/TR/