Local tri-directional patterns: A new

texture feature descriptor for image

retrieval

A Technical Seminar Report

Submitted in partial fulfilment of
the requirement for the B.Tech.
Under Biju Patnaik University of Technology, Rourkela.

Submitted by
P Sai Swaroop Roll # CSE201610373

2019 - 2020

Under the guidance of

Manisha Patro

NATIONAL INSTITUTE OF SCIENCE & TECHNOLOGY

Palur Hills, Berhampur- 761008, Odisha , India

2
ABSTRACT

Texture is a prominent feature of image and very useful in feature extraction for
image retrieval application. Statistical and structural patterns have been proposed for
image retrieval and browsing. In the proposed work, a new texture feature descriptor
is developed. The proposed method uses local intensity of pixels based on three
directions in the neighborhood and named as the local tri-directional pattern (LTriDP).
Also, one magnitude pattern is merged for better feature extraction. The proposed
method is tested on three databases, in which first two, Brodatz texture image
database and MIT VisTex database are texture image databases and third one is the
AT&T face database. Further, the effectiveness of the proposed method is proven by
comparing it with existing algorithms for image retrieval application.

i
ACKNOWLEDGEMENT

I give my sincere thanks to Manisha Patro, Seminar Advisor for giving me the
opportunity and motivating us to complete the seminar within stipulated period of
time and providing a helping environment.

I give my sincere thanks to Mr. Asish Dash, Seminar Coordinator, for helping me
throughout my seminar and encouraging me to complete this seminar.

I acknowledge with immense pleasure the sustained interest, encouraging attitude and
constant inspiration rendered by Dr. Sukant Ku. Mahapatra (Chairman N.I.S.T),
Prof. Sangram Mudali (Director) & Prof. Geetika Mudali (Placement Director)
N.I.S.T. Their continued drive for better quality in everything that happens at N.I.S.T.
and selfless inspiration has always helped us to move ahead.

P Sai Swaroop

ii
TABLE OF CONTENTS

ABSTRACT..................................................................................................................................i
ACKNOWLEDGEMENT...............................................................................................................ii
TABLE OF CONTENTS................................................................................................................iii
LIST OF FIGURES.......................................................................................................................iv
1. INTRODUCTION.....................................................................................................................1
1.1. Related work..................................................................................................................1
1.2. Main contribution..........................................................................................................3
2. Local patterns........................................................................................................................4
2.2. Local tri-directional patterns..........................................................................................4
2.2.1. Advantage over other methods..................................................................................7
3. Proposed system framework................................................................................................9
3.1. Algorithm.......................................................................................................................9
3.2. Similarity measure.........................................................................................................9
4. Experimental results and discussion...................................................................................11
4.1. Database 1...................................................................................................................12
4.2. Database 2...................................................................................................................13
4.3. Database 3...................................................................................................................14
5. Conclusion..........................................................................................................................18
REFERENCES............................................................................................................................20

iii
LIST OF FIGURES

Figure 1:Local binary pattern example. (a) Center and neighboring pixel notations. (b) Sample
window. (c) Local binary pattern using threshold with center pixel. (d) Weights. (e)
Pattern value................................................................................................................3
Figure 2:Proposed method sample window example. (a) Center and neighboring pixel
notations. (b) Sample window. (c)–(j) Local tri-directional pattern and magnitude
pattern calculation.......................................................................................................7
Figure 3:Proposed method block diagram................................................................................8
Figure 4:. MIT VisTex database sample images.......................................................................11
Figure 5:Precision and recall with number of images retrieved for database 1......................11
Figure 6:. Precision and recall of proposed methods for database 1......................................12
Figure 7:Precision and recall with number of images retrieved for database 2......................12
Figure 8:Precision and recall of the proposed methods for database 2..................................13
Figure 9:AT&T database sample images..................................................................................14
Figure 10:. Precision and recall with number of images retrieved for database 3..................14
Figure 11:Precision and recall of proposed methods for database.........................................15
Figure 12:AT&T database query example................................................................................16

iv
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

1. INTRODUCTION

Many techniques for image retrieval have been developed in the past few years due to
extreme increment in digital images. Large image database and retrieval of similar
images became a direct and real world problem. Many kinds of features exist in the
image, and texture is one of them. Texture is a powerful feature of an image that can
be recognized in a form of small repeated patterns. There are many types of texture,
e.g., artificial texture, original texture, rough texture, silky texture, etc. Texture is
mostly contained in images of rocks, leaves, grass, woods, walls, etc. Even in natural
images different types of texture exist. Many local features have been proposed by
researchers in past few years. Local features extract the information regarding to local
objects in the image or local intensity of pixels. Local patterns consider the
neighbouring pixels to extract the local information in the image. Most of the local
patterns proposed by researchers, were uniform for all neighboring pixels. A very few
patterns utilized the pixel information based on the direction. This work is mainly
concentrated on a direction based local pattern which can provide better features with
respect to uniform local patterns. Extensive surveys of content based image retrieval
are presented in past few years.
1.1. Related work

Gray level co-occurrence matrix (GLCM) was proposed for image classification by
Haralick . This matrix extracts features, based on co-occurrence of pixel pairs. GLCM
was used in rock texture retrieval . Zhang et al. proposed a method for texture features
that computes edge images using the Prewitt edge detector and extracts co-occurrence
matrix for those edge images instead of original images. Feature extraction has been
performed on the co-occurrence matrix using statistical features. Transformation
domains were also utilized for feature extraction by researchers. Wavelet packets were
used for the feature extraction and applied for image classification. Gabor filter was
proposed for image retrieval and browsing, and texture classification. Rotation
invariant feature vector has been proposed using Gabor filters for content based image
retrieval. To overcome computational complexity of Gabor wavelet, Rivaz and
Kinsbury proposed a new feature vector based on complex wavelet transform and

0
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

applied it to texture image retrieval. A modified curvelet transform has been proposed
and used for image retrieval. Two novel features ‘composite sub-band gradient vector’
and ‘the energy distribution pattern string’ have been proposed for efficient and
accurate texture image retrieval system. These features were extracted from wavelet
sub-band images. A robust texture feature called the local binary pattern (LBP) was
proposed by Ojala, and it uses the local intensity of each pixel for feature vector
extraction. Further, uniform and nonuniform patterns were discriminated based on
appearance. Also, patterns were converted into rotation invariant. LBP uses the
difference of center pixel and boundary pixel with a threshold for binary pattern.
Instead of a single threshold value, an interval was used and local ternary pattern
(LTP) was obtained, which further converted into two binary patterns. The local
binary pattern was considered as first order derivative, and second and more order
derivative patterns were proposed and called local derivative pattern (LDP). The LDP
was proposed for face image recognition. LBP variance (LBPV) is proposed for
texture classification. Pyramid transformation of local binary pattern has been
proposed using Gaussian and wavelet based low pass filters and called pyramid local
binary pattern (PLBP). Again a pyramid based algorithm was proposed using LBP and
LBPV with Guassian low pass filter and used for smoke detection in videos. Murala
et al. proposed local ternary co-occurrence pattern based on local ternary pattern and
local derivative pattern. After binary and ternary, local tetra patterns (LTrP) were
obtained using horizontal and vertical directional pixels. Further, local tetra pattern
was divided into binary patterns for feature vector histogram. In LBP, all neighboring
pixels were considered as same for every center pixel, and the pattern map was
created according to the difference of neighboring pixels with the center pixel. In
center symmetric local binary patterns (CSLBPs), a pattern has been created using
only interest region pixels instead of all neighborhood pixels. Papakostas et al. have
been proposed a moment based local binary pattern image classification. Feature
extraction in transformation domain using Gabor filters and Local binary pattern has
been performed in, and proposed method is applied in synthetic and multi textured
images for segmentation. Binary patterns based on directional features, have been
proposed in the directional local extrema patterns. Yuan proposed a rotation and scale
invariant local pattern based on high order directional derivatives. The local edge
pattern based on the Sobel edge image, has been proposed in. The Sobel edge image
was created first, then LEPSEG was computed for image segmentation and LEPINV

1
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

was obtained for image retrieval. Local extrema patterns were proposed for object
tracking. Extended local extrema patterns with multi-resolution property were
proposed in for image retrieval. Murala et al. proposed the local maximum edge
binary pattern (LMEBP). The LMEBPs considered the maximum edge from the local
difference and extracted the information based on eight neighborhood pixels. This
method was combined with Gabor transform and experimented on image retrieval and
object tracking. Block division and primitive block based methods have been
proposed using local binary pattern and applied to image retrieval system . In this
method, images were divided into blocks, and then comparison was conducted on
blocks. Local mesh patterns (LMeP) have been proposed for medical image retrieval
that create the pattern according to surrounding pixels for a particular center pixel .
Also, Gabor wavelet has been used for multi-scale analysis. Further, the local mesh
patterns were improved in , where the first order derivative also included in the local
mesh pattern and called local mesh peak valley edge pattern (LMePVEP).
1.2. Main contribution

Local binary pattern creates local pattern based on center and surrounding pixels. It
measures the relationship and forms a pattern. In the proposed method, center and
neighboring pixel relationship is considered in an instructive way that directional
information can be utilized of image. Mutual relationships of neighboring pixels in
three most significant directions are examined in the proposed work. A magnitude
pattern is also considered using same three direction pixels, and both patterns
histogram are combined for feature vector. The proposed method is tested on two
texture and one face image database for performance.

Figure 1:Local binary pattern example. (a) Center and neighboring pixel notations. (b) Sample window. (c) Local
binary pattern using threshold with center pixel. (d) Weights. (e) Pattern value.

2
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

2. Local patterns

Ojala et al. invented the local binary pattern for texture images. Based on performance
and speed, LBP operator is used in image classification , facial expression
recognition, medical imaging , object tracking, etc. The LBP operator for p
neighborhood and r radius is defined as below:

where Ic and Il are center and neighborhood pixel intensities respectively. Histogram
of LBP map is calculated using eqn. (2), where Pattern is LBP, and the size of image
is m × n. A sample window example of LBP pattern is shown in Fig. 1.

2.2. Local tri-directional patterns

Local tri-directional pattern is an extension of LBP. Instead of uniform relationship

with all neighboring pixels, LTriDP consider the relationship based on different
directions. Each center pixel has some neighboring pixels in a particular radius.
Closest neighbor consists of 8 pixels all around the center pixel. Further, there are 16
pixels in next radius and so on. Closest neighboring pixels are less in number and give
more related information as they are nearest to center pixel. Hence, we consider 8-
neighborhood pixels for pattern creation. Each neighborhood pixel at one time is
considered and compared it with center pixel and also with two most adjacent
neighborhood pixels. These two neighborhood pixels are either vertical or horizontal
pixels as they are closest to the considered neighboring pixel. The pattern formation is

3
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

demonstrated in Fig. 2 and explained mathematically as follows. Consider a center

pixel Ic and 8-neighborhood pixels I1, I2... I8. Firstly, we calculate the difference
between each neighborhood pixel with its two most adjacent pixels and difference of
each neighborhood pixel with center pixel.

We have three differences, D1, D2 and D3 for each neighborhood pixel. Now, a
pattern number is assigned based on all three differences.

Where # (Dk < 0) denotes the total count of Dk which is less than 0, for all k = 1, 2, 3.
# (Dk < 0) provides the values ranging from 0 to 3. To calculate each pattern value, a
mod of # (Dk < 0) is taken with 3. It gives the value according to # (Dk < 0), e.g.,
when all Dk < 0, k = 1, 2, 3 then # (Dk < 0) is 3 and # (Dk < 0) mod3 is 0. Similarly if
no Dk < 0 then also the value of # (Dk < 0) mod3 will be 0. In this way, # (Dk < 0)
mod3 is assigned values 0, 1 and 2. More explanation of pattern value calculation
using example, is given in the end of this section in Fig. 2. For each neighborhood
pixel ‘i = 1, 2,.., 8’, pattern value fi(D1, D2, D3) is calculated using eqn. (6), and tri-
directional pattern has been obtained.

By this, we get a ternary pattern for each center pixel and convert it into two binary
patterns.

4
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

After getting pattern map, the histogram is calculated for both binary patterns using
eqn. (2). Magnitude pattern is created as follows:

Values of M1 and M2 are calculated for each neighborhood pixel and according to
these values; a magnitude pattern value is assigned to each neighbourhood pixel.

Similarly, the histogram of the magnitude pattern is created by eqn. (2) and three
histograms concatenated and joint histogram is created.

5
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

An example of pattern calculation is shown in Fig. 2 through (a)–(j) windows. In

window (a), center pixel Ic and neighborhood pixels I1, I2,.., I8 are shown. Center
pixel is marked as red color in windows (b)–(j). In the next window (c), first
neighborhood pixel I1 is marked as blue color, and two most adjacent pixels marked
as yellow color. First, we compare blue pixel with yellow pixels and red pixel and
assign ‘0’ or ‘1’ value for all the three comparisons. For example, in window (c) I1 is
compared with I8, I2 and Ic . Since I1 > I8, I1 < I2 and I1 > Ic, the pattern for I1 is
101. Now, according to eqn. (6) the pattern value for I1 is 1. In the same way, for next
windows (d)–(j) pattern values are obtained for other neighboring pixels. Finally, the
local tri-directional pattern for center pixel is obtained by merging all neighborhood
pixel pattern values. For magnitude pattern, magnitude of center pixel and
neighborhood pixel is obtained and compared. In presented example, ‘6’ is center
pixel and I1 is ‘8’. In window (c), magnitude of the center pixel ‘6’ is 5.8 and
magnitude of ‘8’ is 7.1 with respect to ‘1’ and ‘9’.

2.2.1. Advantage over other methods

Local patterns use local intensity of pixels for grabbing the information and create the
pattern according to the information. Local binary patterns compare the neighborhood
pixel and center pixel and assign a pattern to the center pixel. In the proposed work,
additional relationships among local pixels have been observed. Along with
relationship of center-neighborhood pixels, mutual relationship of adjacent
neighboring pixels are obtained, and local information based on three direction pixels
are examined. This method gives more information compare to LBP and other local
patterns, as it calculates center-neighboring pixel information along with mutual
neighboring pixel information. Nearest neighbors gives most of the information.
Hence, the pattern is calculated using most adjacent neighboring pixels for each
pattern value. Also, a magnitude pattern is introduced which provides information
regarding intensity weight for each pixel. Both LTriDP and magnitude pattern, give
different information and concatenation provides better feature descriptor.

6
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

Figure 2:Proposed method sample window example. (a) Center and neighboring pixel notations. (b) Sample
window. (c)–(j) Local tri-directional pattern and magnitude pattern calculation

Figure 3:Proposed method block diagram.

7
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

3. Proposed system framework

A block diagram of the presented method is shown in Fig. 3, and algorithm for the
same is demonstrated below. Two parts of the algorithm are given. In part 1, feature
vector construction is explained, and in part 2, image retrieval system is presented.

3.1. Algorithm
Part 1: Feature vector construction

Input: Image.

Output: Feature vector.

1. Upload the image and convert it into Gray scale if it is a color image.

2. Compute the tri-directional patterns and construct the histogram.

3. Evaluate the magnitude patterns and make the histogram.

4. Concatenate both the histograms calculated in step 2 and step 3.

Part 2: Image retrieval Input: Query image.

Output: Retrieved similar images.

1. Enter the query image.

2. Calculate the feature vector as shown in part 1.

3. Compute the similarity index of query image feature vector with every
database image feature vector.

4. Sort similarity indices and produce images corresponding to minimum

similarity indices as results.

3.2. Similarity measure

Similarity measure is also a major module of the content based image retrieval
system. It computes the distance between feature vectors, i.e., dissimilarity
between images using a distance metric. In the proposed work, d1 distance
measure has been used as it has proved its excellence for similarity match in
local patterns. For a query image, Q and nth database image DBn following
distance metric is used:

8
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

where Dis() is distance function, L is the length of the feature vector, and F, n ,
db and Fq are feature vector of database nth image and query image
respectively.

9
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

4. Experimental results and discussion

The proposed method is tested on two texture database and one face image database
for validation. The capability of the presented method for image retrieval is shown on
the basis of precision, recall and average normalized modified retrieval rank
(ANMRR). These are performance evaluation parameters, which evaluate the ability
of the method. In the process of retrieving images, for a given query, many images are
retrieved. In those images some are relevant to query image and some are nonrelevant
results which do not match to query image. Images which are of user interest, are
called relevant images, and total images retrieved for a query, are called retrieved
images. Precision computes retrieval performance in ratio of relevant images and
retrieved images. Precision and recall can be computed as:

where Relevant images and Retrieved images are all images relevant to query image
in the database, and images retrieved from the database for a query image. Variables
‘N’ and ‘r’ associated with precision and recall, represent the number of images
retrieved and number id of database image. Precision and recall, related to every
category are calculated as:

where n2 is the total number of categories present in the database. Total recall is also
called as average retrieval rate (ARR). For a given query image Q, total number of
relevant images in database (ground-truth values) are Ng (Q ). Rank of each ground-
truth value for query Q is defined as Rank (i), i.e., position of the ground-truth image i

10
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

in retrieved images. Moreover, a variable K (Q ) > Ng (Q ) is defined as a limit of

ranks. In retrieved images, a ground-truth value that has a rank greater than K(Q ) is
considered as a miss. Rank (Q) and K (Q) is defined as follows:

Modified retrieval rank (MRR) and normalized modified retrieval rank (NMRR) for
different ground-truth values are defined as:

4.1. Database 1

In the first experiment, MIT VixTex database of Gray scale images is used. It
contains total 40 texture images. Each image is of size 512×512. For retrieval
experiment, each image is divided into 16 sub images of size 128 × 128.
Consequently, total 40 categories are obtained and each category holds 16

11
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

images. Sample images from the database are shown in Fig. 4. Precision and
recall for the presented method and other methods are calculated and
demonstrated through graphs. In Fig. 5, plots of precision and recall are shown.
Fig. 5(a) presents variation in precision with number of images retrieved and Fig.
5(b) shows graph between recall and number of images retrieved. Both the graphs
clearly show that the presented method is better than others in terms of precision
and recall.

Figure 4:. MIT VisTex database sample images.

Figure 5:Precision and recall with number of images retrieved for database 1.

12
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

4.2. Database 2

In the second experiment, Brodatz textures have been used for testing. The
Brodatz database contains 112 texture images, each of 640 × 640 size. Every
image is divided into sub images of size 128 × 128. Total 25 sub images obtained
from each original image. Hence, 112 × 25 images are there in this database with
112 categories and each category has 25 images. The results of the proposed
algorithm in the form of precision and recall are presented in graphs. In this
system, initially 25 images are retrieved for each query, and then they are
increasing by an increment of 5, and up to 70 images are retrieved. Plots of
precision and recall with number of retrieved images are shown in Fig. 7. The
proposed method is more satisfying than other methods and it is clearly visible in
the graphs.

Figure 6:. Precision and recall of proposed methods for database 1.

13
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

Figure 7:Precision and recall with number of images retrieved for database 2

4.3. Database 3

In the third experiment, a face database has been taken for face image retrieval
purpose. The AT&T database of faces contains 400 images of 40 subjects. Each
subject is having 10 images with different facial expressions (open/closed eyes,
smiling/not smiling) and facial details (glasses/no glasses). These images were taken
at different times and in varying lighting conditions for some subjects. Size of each
image in this database is 92 × 112.One image from each subject is presented in Fig. 9.
Results of database 3 have been presented in Figs. 10 and 11. In image retrieval
system, 1, 2,.., 10 images have been retrieved, and precision and recall have been
calculated for every time and shown in Fig. 10. The performance measures in
experimental results clearly depict that the proposed method outperforms others. The
average retrieval rate of the proposed method has been improved from CS_LBP,
LEPINV, LEPSEG, LBP, Nanni et al., LDGP, PVEP, LMEBP and DLEP by 24.24%,
106.91%, 48.40%, 30.79%, 4.69%, 20.97%, 39.49%, 18.80%, 7.30%. Average
normalized modified retrieval rank for this database is shown in Table 3. For the
proposed method it is more close to zero as compared to other methods, hence the
proposed method is more promising than others in terms of accurate retrieval. Further,
in Fig. 11, comparison LTriDP and LTriDPmag is shown.

14
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

Figure 8:Precision and recall of the proposed methods for database 2

Demonstration of the proposed method is shown in Fig. 12. Similar face images have
been retrieved for five query images. In Fig. 12, first image in each row is query
image and next three images are retrieved images from the proposed method. Feature
vector length of each method is shown in Table 4. Feature vector length of the
proposed method is comparatively very less from PVEP, LMEBP and DLEP, and
performance is better. Feature vector length of proposed method is more than
CS_LBP, LEPINV, LEPSEG, Nanni et al. method, LDGP and LBP, but performance
is considerably better as shown in different database results. Table 2 explains the
average retrieval rate (ARR) of two texture databases and one face image database for
all compared methods with the proposed method. Final ARR results clearly verify that
the proposed algorithm outperforms others.

15
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

Figure 9:AT&T database sample images

Figure 10:. Precision and recall with number of images retrieved for database 3.

16
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

Figure 11:Precision and recall of proposed methods for database

17
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

5. Conclusion

A novel method, named as Local tri-directional pattern, has been proposed in this
paper and abbreviated as LTriDP. Each pixel in the neighborhood has been compared
with the most adjacent pixels and center pixel for local information extraction. In
most of the previous local patterns, only center pixel is considered for pattern
formation, but in the proposed method information related to each pixel of
neighborhood is extracted, therefore, this method is giving more enhanced features.

Figure 12:AT&T database query example

18
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

The magnitude pattern is also incorporated, that is again based on the same pixels
used in LTriDP. The proposed method is compared with CS_LBP, LEPINV, LEPSEG,
Nanni et al. method [35], LDGP, PVEP, LBP, LMEBP and DLEP with reference to
precision and recall. All methods are tested on MIT VisTex texture database, Brodatz
texture database and AT&T face image database. Precision and recall show that the
proposed system more proficient and appropriate than others in terms of accuracy.
Also, the feature vector length of the proposed algorithm is more acceptable than
PVEP, LMEBP and DLEP.

19
 Local tri-directional patterns: A new texture feature descriptor for image retrieval

REFERENCES

[1] Verma, Manisha, and Balasubramanian Raman. “Local Tri-Directional

Patterns: A New Texture Feature Descriptor for Image Retrieval.” Digital
Signal Processing, vol. 51, 2016, pp. 62–72., doi:10.1016/j.dsp.2016.02.002.
[2] “Local Tri-Directional Patterns: A New Texture Feature Descriptor for Image
Retrieval.” Digital Signal Processing, Academic Press, 4 Feb. 2016,
www.sciencedirect.com/science/article/pii/S1051200416000208.

20