VISHVESHWARAIAH TECHNOLOGICAL UNIVERSITY

S.D.M COLLEGE OF ENGINEERING AND TECHNOLOGY

A seminar report on

CONTENT-BASED IMAGE RETRIEVAL

Submitted by

SAMEER KULKARNI
2SD06CS083
8th semester

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

2009-10

1
 VISHVESHWARAIAH TECHNOLOGICAL UNIVERSITY

S.D.M COLLEGE OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE ENGINEERING

CERTIFICATE
Certified that the seminar work entitled “content-based image retrieval” is a bonafide work
presented by Sameer Kulkarni bearing USN NO 2SD06CS083 in a partial fulfillment for the
award of degree of Bachelor of Engineering in Computer Science Engineering of the
Vishveshwaraiah Technological University, Belgaum during the year 2009-10. The seminar
report has been approved as it satisfies the academic requirements with respect to seminar
work presented for the Bachelor of Engineering Degree.

Staff in charge H.O.D CSE

Name: SAMEER KULKARNI

USN: 2SD06CS083

2
 INDEX

1. INRODUCTION 4
2. BAKGROUND 4
3. NEED FOR CBIR 5
4. TECHNIQUE USED IN CBIR
o CBIR TECHNIQUES 6
o COLOR RETRIEVAL 7
o TEXTURE RETRIEVAL 8
o SHAPE RETRIEVAL 9
o OTHERS TYPES OF FEATURES 10
5. SEGMENTATION 11
6. REFERENCE FEEDBACK 12
7. CO-OCCURRENCE MATRIX AND R I 12
8. APPLICATIONS OF CBIR 15
9. ADVANTAGES OF CBIR OVER TBIR 16
10. ISSUES RELATED TO CBIR 16
11. DRAWBACKSOF CBIR 16
12. CONCLUSION 17
13. REFERENCES 18

3
1 INTRODUCTION:
Content-based image retrieval is a part of image processing and is also
comes under artificial intelligence. We know that interest in digital images is
growing day by day; Users in many professional fields are exploiting the
opportunities offered by the ability to access and manipulate remotely-stored
images in all kinds of new and exciting ways. The problems in image retrieval is
becoming widely recognized, and the search for solutions an increasingly active
area for research and development. So here is the technique Content-based image
retrieval (CBIR), also known as query by image content (QBIC) and content-
based visual information retrieval (CBVIR) is the application of computer vision
to the image retrieval problem, that is, the problem of searching for digital images
in large databases.

2 BACKGROUNDS:
The use of digital images and pictures in communication is new – what
olden days cave-dwelling ancestors did was they use to paint pictures on the
walls of their caves, and the use of maps and building plans to convey information
almost certainly dates back to pre-Roman times. But the twentieth century has
witnessed unparalleled growth in the number, availability and importance of
images in daily life. They now play an important role in fields as diverse as
medicine, journalism, advertising, design, education and entertainment.

The term CBIR seems to have originated in 1992, when it was used by
T. Kato to describe experiments into automatic retrieval of images from a
database, based on the colors and shapes present. Since then, the term has been
used to describe the process of retrieving desired images from a large collection on
the basis of syntactical image features. The techniques, tools and algorithms that
are used originate from fields such as statistics, pattern recognition, signal
processing, and computer vision.

4
3 NEED FOR CBIR:
In today’s world more and more multimedia Information is stored in
databases like images audio and video. An image may be better or more effective
than a substantial amount of text. It also aptly characterizes the goals of
visualization where large amounts of data must be absorbed quickly and it is a
picture is worth of thousand words. If we use text for retrieving/searching an
image what happens is-we can’t write whole description of image this is one thing
and if the image contains geographical data then manual keyword description are
not possible ex: Google earth contains geographical data. So then why to go for
this is because- It is cultural language dependent and is not possible to describe
every image in database. Keyword matching will not give most relevant images.

Fig1

5
4 TECHNIQUES USED IN CBIR:
CBIR techniques
CBIR operates on a totally different principle, retrieving/searching stored
images from a collection by comparing features automatically extracted from the
images themselves. The commonest features used are mathematical measures of
color, texture or shape (basic). A system (CBIR) allows users to formulate
queries by submitting an example of the type of image being sought (input),
though some offer alternatives such as selection from a palette or sketch input we
can also select color textures or any other visual information. The system then
identifies those stored images whose feature values match those of the query most
closely (right side), and displays thumbnails of these images on the screen like in
fig1.

Fig2

The above shown Activity Diagram explains the basic steps in any of the CBIRS
(systems).

6
Colour retrieval
Each image added to the collection is analyzed to compute a ‘color
histogram’ where ‘color histogram’ is a representation of the distribution of colors
in an image. For digital images, it is basically the number of pixels that have colors
in each of a fixed list of color ranges, which span the image's color space, the set of
all possible colors. This shows the proportion of pixels of each color within the
image. The color histogram for each image is then stored in the database. During
searching time, the user can either specify the desired proportion of each color
(75% olive green and 25% red, for example)it depends on the system to give that
option or not(design and options), or submit an example image from which a color
histogram is calculated. Matching process then retrieves those images whose color
histograms match those of the query most closely as described above in
percentages. The results from some of these systems can look quite impressive.

One more concept is ‘Gray scale image’ which is simply one in which the
only colors are shades of gray (fig 3 right side). It is different from any other sort
of color image is that less information needs to be provided for each
pixel(maximum information of image color). In fact a `gray' color is one in which
the red, green and blue components all have equal intensity in RGB space, and so it
is only necessary to specify a single intensity value for each pixel, as opposed to
the three intensities needed to specify each pixel in a full color image.

Fig3

7
Texture retrieval
What if the images are of same color? This will be answered by textures. The
ability to retrieve images on the basis of texture similarity may not seem very
useful-But the ability to match on texture similarity can often be useful in
distinguishing between areas of images with similar color (such as sky and sea, or
leaves and grass). A variety of techniques has been used for measuring texture
similarity; the best-established rely on comparing values of what are known as
second-order statistics calculated from query and stored images. Essentially, these
calculate the relative brightness of selected pairs of pixels from each image. From
these it is possible to calculate measures of image texture such as the degree of
contrast, coarseness, homogenality and regularity [Tamura et al, 1978], or
periodicity, correlation and entropy [Liu and Picard, 1996]. Alternative methods
of texture analysis for retrieval include the use of Gabor filters this is the widely
used technique now a days. Texture queries/specifying can be formulated in a
similar manner to color queries, by selecting examples of desired textures from a
palette, or by supplying an example query image. The system then retrieves images
with texture measures similar in value. A recent technique is the texture thesaurus
developed by Ma and Manjunath [1998], which retrieves textured regions in
images on the basis of similarity to automatically-derived codeword’s representing
important classes of texture within the collection.

 Contrast is the dissimilarity or difference between things(color, brightness

etc).
 Homogeneity means "being similar throughout"(like same color can be said
to one part segmentation can also be done through this).
 Entropy is a measure of the uncertainty associated with a random variable.

Fig4

8
Shape retrieval
Shape is the most obvious requirement at the primitive level. Unlike texture,
shape is a fairly well-defined concept – and there is considerable evidence that
natural objects are primarily recognized by their shape .A number of features
characteristic of object shape (but independent of size or orientation) are computed
for every object identified within each stored image. Queries are then answered by
computing the same set of features for the query image, and retrieving those stored
images whose features most closely match those of the query. Two main types of
shape feature are commonly used – global features such as aspect ratio,
circularity and moment invariants [Niblack et al, 1993] and local features
such as sets of consecutive boundary segments [Mehrotra and Gary, 1995].
Alternative methods proposed for shape matching have included elastic
deformation of templates (Pentland et al [1996], del Bimbo et al [1996]),
comparison of directional histograms of edges extracted from the image and
shocks, skeletal representations of object shape that can be compared using graph
matching techniques.

Fig5

9
Retrieval by other types of primitive feature
One of the oldest-established means of accessing pictorial data is retrieval by
its position within an image. Techniques have been applied to image collections,
allowing users to search for images containing objects in defined spatial
relationships. Improved algorithms for spatial retrieval are still being proposed.
Spatial indexing is seldom useful on its own, though it has proved effective in
combination with other cues such as colour (Stricker and Dimai [1996], Smith and
Chang [1997a]) and shape [Hou et al, 1992].

Several other types of image feature have been proposed as a basis for
CBIR. The well-researched technique of this kind uses the wavelet transform to
model an image at several different resolutions. Promising retrieval results have
been reported by matching wavelet features computed from query and stored
images .Another method giving interesting results is retrieval by appearance.

The advantage of using these are they can describe an image at varying levels
of detail (useful in natural scenes where the objects of interest may appear in a
variety of guises), and avoid the need to segment the image into regions of interest
before shape descriptors can be computed. Despite recent advances in techniques
for image segmentation ,this remains a troublesome problem. Now we will see one
of the technique that is edge detection that comes under shape retrieval.

fig6
The above marked points in image are called spatial points.

10
5 Segmentation:

Segmentation refers to the process of dividing the a digital image into

multiple segments or a set of pixel it is also sometimes known as super pixels.
Why we go for segmentation? - is to simplify and/or change the representation of
an image into something that is more meaningful and easier to analyze. Image
segmentation is typically used to locate objects and boundaries, lines and edges in
images. Precisely image segmentation is the process of assigning a label to
every pixel in an image such that pixels with the same label share certain
visual characteristics.

Fig7

11
6 RELEVANCE FEEDBACKS (FURTHER):

Fig8

After retrieval of results, the irrelevant images also may found. User give
their feedback about the results to avoid images that are not related to the given
query. Relevant, Not-Relevant, No-Idea After that the image which is irrelevant is
not included for that query in future.

7 Co-occurrence Matrix and Rotational Invariance:

Co-occurrence matrix method is an essential requirement in the field of
image processing using pattern recognition and has been used extensively. A co-
occurrence matrix, also referred to as a co-occurrence distribution, is defined over
an image to be the distribution of co-occurring values at a given offset. Any matrix
or pair of matrices can be used to generate a co-occurrence matrix, though their
main applicability has been in the measuring of texture in images.

12
 Let I be a gray scale image which is coded on ‘m’ gray levels. S=(x, y) is the
position of a pixel in the image and T be the translational vector. The co-
occurrence matrix Mt is an m*m matrix whose (i, j) th element is the number of
pairs of pixels separated by the translation vector T that have the pair of gray levels
(i, j).

An example of co-occurrence matrix is shown in figure.

1 2 0 0 1 0 0 0

Table 1:Image matrix

0 0 1 0 1 0 0 0

11 11 5 6 8 0 0 0 0 1 0 0 0

2 3 5 7 1 0 0 0 0 1 0 0 0

4 5 7 1 2
1 0 0 0 0 1 2 0
1 2
0 0 0 0 0 0 0 1
8 5 1 2 5
1 2
2 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0

In the output GLCM, element (1,1) contains the value 1 because there is only one
instance in the input image where two horizontally adjacent pixels have the values
1 and 1, respectively. glcm(1,2) contains the value 2 because there are two
instances where two horizontally adjacent pixels have the values 1 and 2. Element

13
(1,3) in the GLCM has the value 0 because there are no instances of two
horizontally adjacent pixels with the values 1 and 3. graycomatrix continues
processing the input image, scanning the image for other pixel pairs (i,j) and
recording the sums in the corresponding elements of the GLCM.

Let us now consider the concept of rotational invariance,

The (Δx, Δy) parameterization makes the co-occurrence matrix sensitive to

rotation. We choose one offset vector, so a rotation of the image not equal to 180
degrees will result in a different co-occurrence distribution for the same (rotated)
image. This is rarely desirable in the applications co-occurrence matrices are used
in, so the co-occurrence matrix is often formed using a set of offsets sweeping
through 180 degrees (i.e. 0, 45, 90, and 135 degrees) at the same distance to
achieve a degree of rotational invariance.

14
8 APPLICATIONS OF CBIR:

1. Image search engines.

2. Security
o Thumb recognition
o Iris recognition
 Fuzzy logic is used most of the times it divides the
image and uses co-occurrence matrix.
3. Investigations
o Compare images
o Collect relevant images
4. Analysis
o Geographical and enterprise related
5. Medical diagnosis
o Cancer detection.
6. Military and remote sensing Systems

15
9 ADVANTAGES OF CBIR OVER TBIR:

 Among other image retrieval methods, content-based image retrieval4

(CBIR) is an approach that exclusively relies on the visual features, such as
color histogram, texture, shape, and so forth, of the images.
 One of the obvious advantages of CBIR over other methods, e.g., text-based
image retrieval, is that CBIR can be done in a fully automatic process since
the visual features are automatically extracted.
 It is based on Artificial intelligence.

10 SOME OF THE ISSUES RELATED TO CBIR:

 Which features should be derived to describe the images better within

database.
 Which data structure should be used to store the feature vectors
 Which learning algorithms should be used in order to make the CBIR wiser
 How to participate the user’s feedback in order to improve the searching
result.

11 DRAWBACKS OF CBIR:

 More Resources are needed like CPU time, memory, disk space,etc.
 Determining Semantic meaning of Objects is not possible.Results may contain
False Hits.
 Problems on Processing Different aspects of Images.
 A drawback associated with CBIR is the increased computational cost arising
from tasks such as image processing, feature extraction.

16
12 CONCLUSION:

CBIR at present is still very much a research topic. The technology is

exciting but immature, and few operational image archives have yet shown any
serious interest in adoption .CBIR technology replacing more traditional methods
of indexing and searching. Use of text and image features might well yield better
performance than either type of retrieval cue on its own.

12 REFERENCES:

1. [http://en.wikipedia.org/wiki/Content_based_image_retrieval].
Introduction and concepts
2. [http http://www.eecs.berkeley.edu/~cchang/docs/Spie01.pdf] content
based image retrieval by Bhakthvatsalam (ppt) – technique used,
reference feedback.
3. content-based Image RetrievalJohn Eakins Margaret Graham
University of Northumbria at Newcastle [Technique used]
4. Image retrieval based on region shape similarity by Cheng Chang , Liu
Wenyin and Hongjiang Zhang.[22-25]
5. Feature extraction and image retrieval by Mark s Nixon and alberto 1st
edition[99-157].
6. http://homepages.inf.ed.ac.uk/rbf/HIPR2/gryimage.html [gray scale
images].
7. Issues [http://imageprocessing.wordpress.com/2007/12/09/content-based-
image-retrieval-cbir/]
8. http://matlab.izmiran.ru/help/toolbox/images/enhanc15.html for glcm.

17
18