Car Number Plate Recognition Software

University of Nairobi
Car Number Plate Recognition Software
Project Index: PRJ033
By
Njagi Raphael Kinyua
F17/2135/2004
Supervisor: Dr. H. A. Ouma
Examiner: Dr. V.K. Oduol
Project report submitted in partial fulfillment of the requirement for the award of
the degree of
Bachelor of Science in Electrical and Electronic Engineering of the University
of Nairobi
25/06/2009
Department of Electrical and Information Engineering

i

Abstract

The purpose of this project was to design and implement an Automatic Number Plate
Recognition system. The system has still images as the input, and extracts a string
corresponding to the plate number, which is used to obtain the output user data from a
suitable database. The system extracts data from a license plate and automatically reads it with
no prior assumption of background made. License plate extraction is based on plate features,
such as texture, and all characters segmented from the plate are passed individually to a
character recognition stage for reading. The string output is then used to query a relational
database to obtain the desired user data. This particular project utilized the intersection of a
hat filtered image and a texture mask as the means of locating the number plate within the
image. The accuracy of location of the number plate with an image set of 100 images was 68%.

ii

Acknowledgements
I would like first and foremost to thank God for sustenance during the project period. I would
also like to thank my supervisor, Dr. H. A. Ouma for all the help he gave me during this project
and to my family and friends who have been a source of support during this time.

iii

Table of Contents

Abstract ...................................................................................................................................................................... ii
Acknowledgements ................................................................................................................................................... iii
CHAPTER ONE: INTRODUCTION: ................................................................................................................................ 1
1.1 BACKGROUND .................................................................................................................................................. 1
1.2 OBJECTIVES ...................................................................................................................................................... 1
1.3 MOTIVATION .................................................................................................................................................... 2
1.4 SPECIFIC OBJECTIVES:....................................................................................................................................... 3
1.5 SYSTEM OVERVIEW .......................................................................................................................................... 3
1.5.1 Image Processing Block: ............................................................................................................................ 3
1.5.2 Optical character recognition block: ......................................................................................................... 4
1.5.3 The database: ............................................................................................................................................ 4
CHAPTER TWO: LITERATURE REVIEW ........................................................................................................................ 6
2.1 IMAGE ENHANCEMENT .................................................................................................................................... 7
2.2 LICENSE PLATE LOCALIZATION AND SEGMENTATION ................................................................................... 10
SCHEMES: ............................................................................................................................................................. 10
2.2.1 Edge Density Estimation ......................................................................................................................... 10
2.2.2 Color Based Segmentation: ..................................................................................................................... 12
2.2.3 Morphology Based Segmentation: ......................................................................................................... 13
2.3 CHARACTER SEGMENTATION SCHEMES: ....................................................................................................... 17
2.3.1 Bounding Box character Segmentation .................................................................................................. 17
2.3.2 Histogram Analysis .................................................................................................................................. 20
2.4 CHARACTER RECOGNITION SCHEMES: .......................................................................................................... 22
2.4.1 Neural Network Classifiers ...................................................................................................................... 22
2.4.2 Template Matching Approach ................................................................................................................ 24
CHAPTER THREE: PROBLEM SOLUTION AND DESIGN .............................................................................................. 25
3.1 IMAGE PROCESSING BLOCK: .......................................................................................................................... 25
3.2 THE OPTICAL CHARACTER RECOGNITION BLOCK: ......................................................................................... 30
3.2.1Neural Network Classifier Approach ........................................................................................................ 30
CHAPTER FIVE: DISCUSSION, RECOMMENDATIONS AND FUTURE WORK .............................................................. 39
DISCUSSION .......................................................................................................................................................... 39
iv

RECOMMENDATIONS AND FUTURE WORK ......................................................................................................... 40
CONCLUSION ............................................................................................................................................................ 41
REFERENCES ............................................................................................................................................................. 42
APPENDICES ............................................................................................................................................................. 43
APPENDIX A: Thresholding Schemes .................................................................................................................... 43
A.1 Maximum Entropy Criteria: ....................................................................................................................... 43
A.2 Otsu Thresholding Scheme: ....................................................................................................................... 43
APPENDIX B: ............................................................................................................................................................. 46
Source code .............................................................................................................................................................. 46

v

CHAPTER ONE: INTRODUCTION:

1.1 BACKGROUND
Automatic number plate recognition, ANPR, is a mass surveillance method that uses
optical character recognition on images to read the license plate on vehicles using existing
closed circuit television cameras or road‐rule enforcement cameras, or ones specifically
designed for the task‐some systems commonly use infrared lighting to allow the camera to take
the picture at any time of day. They are used for various tasks, including electronic toll
collection on pay per use roads, restricted car identification access control schemes such as for
pay parking‐lots or for secured office compounds, monitoring traffic activity such as red light
adherence in an intersection and for direct marketing. ANPR technology tends to be region
specific, owing to plate variation from place to place. The first ANPR was invented in 1976 at
the Police Scientific Development Branch in the UK. Prototype systems were working by 1979
and contracts were let to produce industrial systems at EMI Electronics and at the Computer
Recognition Systems (CRS) in Wokingham, UK. Early trial systems were deployed on the A1 road
and at the Dartford Tunnel [1].
1.2 OBJECTIVES
It is required that an automated license plate recognition system be designed and
implemented. The system takes digital images of vehicles with their license plates visible and
extracts license plate characters as a string of alphanumeric characters, and yields hypothetical
user data from a related database corresponding to the extracted characters as output. The
designed system is to work with reasonable accuracy if the digital image is of good resolution,
and vehicle license plate has standard font in the region of interest.

1

1.3 MOTIVATION
LPR applications have a wide range of applications, which use the extracted plate
number and optional images to create automated solutions for various problems [2]. These
include
• Parking ‐ the plate number is used to automatically enter pre‐paid members and
calculate parking fee for non‐members by comparing the exit and entry times.
• Access Control ‐ a gate automatically opens for authorized members in a secured area,
thus replacing or assisting the security guard. The events are logged on a database and
could be used to search the history of events
• Law enforcement ‐ the plate number is used to produce a violation fine on speed or
red‐light systems. The manual process of preparing a violation fine is replaced by an
automated process which reduces the overhead and turnaround time. Using a list of
stolen cars or unpaid fines, an ALPR system deployed on the roadside, may perform a
real‐time match between the passing cars and those on the list, hence raise alerts for
passing cars on the list.
• Marketing Tool ‐ the car plates may be used to compile a list of frequent visitors for
marketing purposes, or to build traffic profiles, such as the frequency of entry verses the
hour or day.
• Tolling ‐ the car number is used to calculate the travel fee in a toll‐road, or used to
double‐check the ticket.
Such routine tasks, possibly handling large volumes of traffic, maybe more efficiently
done if automated than if done by humans. This would reduce associated costs of operation,
increase processing speed and minimize errors that may result due to fatigue or monotony if a
human operator were involved.
2

1.4 SPECIFIC OBJECTIVES:
The car license plate recognition software was to be developed for a car access control
scheme. Since the previous work focused on extraction of the license plate, the current project
will integrate this previous work, with suitable modifications, to the current work and in
addition, develop a character segmentation stage, an optical character recognition stage, and a
hypothetical user database.
1.5 SYSTEM OVERVIEW

The automatic plate recognition system may be decomposed into three blocks: an
image processing stage, an optical character recognition stage and the database.
1.5.1 Image Processing Block:

This block receives digital images of the vehicles and produces segmented license plate
characters. It implements image preprocessing, plate detection and plate character segmentation.
Image preprocessing or plate enhancement phase helps to enhance the signal in the car plate
area and attenuate it elsewhere. The plate detection phase aims at identifying the car plate
area in an image.
Preceding work proposed converting RGB or indexed images into gray‐scale images
which were then binarized using a fixed threshold, although literature and similar works
suggest use of adaptive thresholds, since such a fixed threshold would make license plate
detection subject to ambient illumination. Large areas of constant intensity background were
removed using a smearing routine, which searched for stretches of continuous white space,
whose area was greater than that expected for a license plate region, and then turned all these
pixels from white to black. License plate detection was implemented by searching for regions
with high horizontal frequency in the horizontal direction and a predefined aspect ratio of 4.5,
for the Kenyan system. Since insufficient processing steps were done to enhance plate‐like
regions at the expense of the rest of the image, these two criteria were insufficient for license
plate detection in complex scenes as many regions, possibly with noise objects, may be chosen
as the desired license plate.
3

Current work proposes use of an adaptive thresholding scheme, specifically, Otsu’s
method [Appendix A]. To speed up processing and ease coding, a morphological closing,
obtained using MATLAB’s inbuilt functions dilation and erosion with a horizontal structuring
element of width greater than that expected between characters, could be used to implement
top hat filtering as the replacement of the smearing algorithm. Further enhancement of the
license plate would search for coarse texture regions using range filtering. Suitable binary
measurements would then be used to retrieve the correct character objects.
The character segmentation acts as a bridge between the license plate extraction and
optical character recognition modules. Its main function is to segment the characters on the
chosen candidate region such that each character can be sent to the optical character
recognition module individually for recognition.

1.5.2 Optical character recognition block:

This block takes as input the segmented characters from the image processing block and
yields characters in form a machine editable string. This output string is then used to obtain the
required data from a suitable database. The template matching approach proposed in the
previous work would be adopted here. While the template matching is limited by its suitability
to recognition of simple single‐font, not‐rotated, and fixed‐size characters and requires properly
built templates, it is easy to implement and yields reasonable accuracy in spite of its reduced
complexity. However, a neural network classifier would be implemented to investigate the
comparative difference in cognitive accuracy.

1.5.3 The database:
This contains the hypothetical user data. The database would contain the license plate
characters of the 108 images obtained from the previous work, stored as strings and forming
the primary key field of the database. These license plate strings would be used to reference
vehicle type and vehicle model. Implementation using SQL or such other relational database is
4

envisioned. Additional user data may be included in other tables, up to the required normal
form and the data could be retrieved using the license plate string using suitable inner join.
In general, there are a number of possible difficulties that the software must be able to
cope with. These include motion blur, poor lighting and low contrast due to overexposure,
reflection or shadows, objects or dirt obscuring part of the plate, poor image resolution, usually
because the plate is too far away but sometimes resulting from the use of a low‐quality camera
and different design of the plates or character fonts. The system hardware may offer solutions
to some of these difficulties. Increasing the height of the camera may avoid problems with
objects, such as other vehicles, obscuring the plate, but introduces and increases other
problems such as the adjusting for the increased skew of the plate.

5

CHAPTER TWO: LITERATURE REVIEW

An automatic license plate localization and recognition (ALPR) system is generally made
up of three modules; license plate localization, character segmentation and optical character
recognition modules [3, 14].

Figure 2.1: Flowchart of a Typical ALPR System [3]

The most common solutions to license plate localization in digital images are through
the implementation of edge density estimation, color based segmentation, texture filtering and
histogram analysis, morphological operators and Hough Transform. An edge approach is
normally simple and fast. However, it is sensitive towards noise. Hough transform for line
6

detection gives positive effects on image assuming that the plate is made up of straight lines.
However, it requires the outline of the plate to be obvious for satisfactory license plate
localization, large memory space and a considerable amount of computing time. On the other
hand, a basic histogram approach is not capable of dealing with images with considerable
amount of noise and tilted license plates. Last but not least, the localization of license plates via
morphological based approaches is not susceptible to noise but is slow in execution. A
combination of two or more methods or modification of these methods is thus necessary [5].
Localization of the license plate comes after the character segmentation process. Common
character segmentation processes are based on histogram analysis and thresholding and
estimation of bounding boxes of characters.
The final stage of this system is the character recognition process. The template
matching method can be easily implemented. However, to deal with the number of variations
found on the characters across different license plates might require the segmented characters
to undergo some preprocessing steps such as normalization and skew correction. More
advanced techniques may include the use of neural networks and self organizing maps.
2.1 IMAGE ENHANCEMENT

Image enhancement is used for pre‐processing the image before license plate localization
is performed. Such pre‐processing steps include methods for removing noise from noise, adjusting
the intensity of the image and reducing the contrast in the image and image thresholding.
Granular noise may be removed using an averaging filter, since each pixel gets set to the
average of the pixels in its neighborhood which reduces local variations caused by granular
noise. In contrast, median filtering is more suited to removing the salt and pepper noise.
Median filtering is similar to using an averaging filter, in that each output pixel is set to an
average of the pixel values in the neighborhood of the corresponding input pixel. However,
with median filtering, the value of an output pixel is determined by the median of the
neighborhood pixels, rather than the mean. The median is much less sensitive than the mean to
extreme values, called outliers, and is therefore better able to remove these outliers without
reducing the sharpness of the image [4].
7

The technique most often used for intensity adjustment is known as histogram
equalization. The histogram shows the data range of the image and the display range of the
image. The data range is the range of intensity values actually used in the image. The display
range is the black‐to‐white mapping used to display the image, which is determined by the
image class. A gray‐scale image has a display range of 256, while a binary image has a display
range of 2. Histogram equalization is an image enhancement technique that maps an image’s
intensity values to a new range. The histogram of a low‐contrast image has all values gathered
in the center of the range. Remapping the data values to fill the entire intensity range [0, 255],
would increase the contrast of the image [4, 7].

Figure 2.2: The figure on the left represents the histogram of a poor‐contrast image, while that
on the right is the equivalent histogram after contrast stretching [4]

Image thresholding is used to convert a gray scale image to a binary image. All pixels in
the gray scale image whose intensity values exceed a certain threshold value are mapped onto
binary level 1, while those pixels whose intensity values fall below the threshold are mapped
onto binary level 0, in the binary image. For simple global thresholding, the threshold value
used is fixed and binarization is accomplished by scanning the image pixel by pixel, while
labeling each pixel onto either binary level, depending on whether the gray‐value of that pixel is
greater or less than the value of the threshold. Such a scheme is only successful in illumination
controlled environments. Optimal thresholding is used if the gray‐scale image has two or more
8

principal brightness regions; the histogram of such an image may be considered an estimate of
the brightness probability density function.
Selecting an optimum threshold in terms of minimum error is possible by mapping the
brightness probability density function onto Gaussian, Raleigh or log normal densities. The
threshold is then estimated using a minimum square error approach [7]
T=
Where p(i) is the brightness probability density and h(i) is the desired density function,
assuming an n‐point histogram. Alternatively, Otsu’s method, which chooses the threshold to
minimize the intra‐class variance of the black and white pixels or the Maximum Entropy
Criterion, may be used to provide an unsupervised solution to the choice of threshold [6].

Figure 2.3: Result of thresholding a car under a shade (low illumination)using Otsu’s method.
The threshold value is 100.

Figure 2.4: Result of thresholding a car in bright sunlight (high illumination)using Otsu’s
method. The threshold value is 129.
9

2.2 LICENSE PLATE LOCALIZATION AND SEGMENTATION
SCHEMES:

These schemes are feature based, and exploit the unique characteristics of license
plates. Several methods for such image segmentation exist. Image segmentation is the process
of isolating objects in the image from the background; partitioning the image into disjoint
regions, such that each region is homogeneous with respect to some property, such as grey
value or texture.

2.2.1 Edge Density Estimation
Edge based methods, are based on the assumption that car plates have strong edges that
will survive strong filtering, while other image parts will not. The regions with a high edge
magnitude and high edge variance are identified as possible license plate regions. Measuring
the edge density at a local neighborhood is a robust criterion for enhancing the car image at
candidate regions for the plate. The motivation is based on existing strong vertical edges on
within the license plate region. In order to estimate densities of vertical edges in the image, the
gradient image is computed first. A simple and proper candidate filter for this purpose is the
Sobel operator.

H=
Figure 2.5: Vertical Sobel operator

This operator is convolved with the input image to yield the gradient image, and
produces a gradient image using the derivative mask. An edge is the boundary between two
regions with relatively distinct gray‐level properties. The profile of an edge along a horizontal
scan line of the image shows a transition in gray‐level. The first derivative of the gray level
10

profile is positive at the leading edge of a transition, negative at the trailing edge, and zero in
areas of constant gray level. The second derivative is positive for transition associated with the
dark side of an edge, negative for transition associated with the light side of an edge and zero in
areas of constant gray level. The magnitude of the first derivative thus may be used to detect
presence of an edge in an image, while the sign of the second derivative may be used to
determine whether edge pixel lies on the light or dark side of an edge. Then by comparing the
gradient image against a predefined threshold, the binary edge image is obtained [7].
The edge density is estimated using 2D Gaussian filter, whose size is selected from the
average size of license plates to be detected. Convolving the binary edge map with this
Gaussian kernel yields estimation for edge density. The horizontal profile line with the highest
vertical edge density corresponds to the license plate region (Fig 6), and objects whose
centroids lie within a narrow range of this line are the desired characters [10]. A disadvantage is
that edge based methods if used alone can hardly be applied to complex images, since they are
too sensitive to unwanted edges, which may also show a high edge magnitude or variance (e.g.,
the radiator region in the front view of the vehicle). Despite this, when combined with
morphological steps that eliminate unwanted edges in the processed images, the LP extraction
rate is relatively high, compared to other methods.

Figure 2.6: Scanning in N rows and counting the existent edges [5]

11

2.2.2 Color Based Segmentation:

Color (gray level) based segmentation methods utilize the fact that LPs (when
disregarding the embossed characters) in images often show unique and homogenous color,
such as yellow for rear plates, red for diplomatic mission vehicles and blue for Government
parastatals in the Kenyan context, although other colors may be used by other countries. The
basic idea of the extraction of a plate region is that the color combination of a plate
(background) and the characters (foreground) is unique, and this combination occurs almost
only in a plate region. Genetic algorithms may be used for color segmentation by searching for
colored rectangular regions as LPs. To make the system insensitive to noise and variation in
illumination condition, consistency of the labeling between neighboring pixels during color
segmentation is necessary.
Alternatively, a neural network (NN) may be used to estimate surface color of LP from
given samples of LP images. All the pixels in the image are filtered by the NN and color
homogeneity for a given color, for each pixel is calculated. Then the LP region is identified by
verifying the rectangular region with the pre‐defined color, using structural features. Crucial to
the success of color (or gray level)‐based method is the color segmentation classifier that can
discriminate between color textures associated with the different classes (LP and non‐LP). A
small window may be used to scan an input image and classifies the pixel located at the center
of the window into plate or non‐plate (background) by analyzing its color and texture
properties. The set of color values, color coordinates in a color space (e.g. RGB, HSI, etc.) of the
raw pixels of the image, is the feature set on which the classification is based [8]. Binarization is
then implemented by constraining regions having the pre‐defined color white and all other
regions are rendered black [9].
This method has several weaknesses, including difficulty in detecting plates on cars
having the same color as license plate background, and from images with background areas
12

having the same color as the license plate, and which are of about the same size as the license
plate area.

Input image pyramid Classification pyramid

Figure 2.7: Color based segmentation of green license plates [8]

2.2.3 Morphology Based Segmentation:

Morphology based segmentation is implemented using hat transformations or texture
based segmentation. Morphology is a broad set of image processing operations that process
images based on shapes. Morphological operations apply a structuring element to an input
image, creating an output image of the same size. A structuring element (SE) is a matrix
13

consisting of only 0’s and 1’s that can have any arbitrary shape and size. The pixels with values
of 1 define the neighborhood. Two‐dimensional or flat structuring elements are typically much
smaller than the image being processed.
The center pixel of the structuring element, called the origin, identifies the pixel of
interest – the pixel being processed. The pixels in the structuring element containing 1’s define
the neighborhood of the structuring element.
In a morphological operation, the value of each pixel in the output image is based on a
comparison of the corresponding pixel in the input image with its neighbors. The most basic
morphological operations are dilation and erosion. Dilation adds pixels to the boundaries of
objects in an image, while erosion removes pixels on object boundaries. The number of pixels
added or removed from the objects in an image depends on the size and shape of the
structuring element used to process the image. For the gray‐scale dilation of image (f) by SE (b),
denoted (f b), the value of the output pixel is the maximum value of all the pixels in the input
pixel’s neighborhood. In a binary image, if any of the pixels is set to the value 1, the output pixel
is set to 1. For the gray‐scale erosion of image (f) by SE (b), denoted (f θ b), the value of the
output pixel is the minimum value of all the pixels in the input pixel’s neighborhood. In a binary
image, if any of the pixels is set to 0, the output pixel is set to 0. The closing of image (f) by SE
(b), denoted (f • b), is defined as
f • b = (f b) θ b
involves the dilation of image elements using the structuring element followed by erosion of
the dilated element boundaries by the same structuring element. The closing operation thus
generally fuses narrow breaks and long thin gulfs, eliminates small holes, and fills gaps in the
contour. Closing removes black holes on white objects. A hole will be removed if the SE does
not fit within it. The opening of image (f) by SE (b), denoted (f ° b), is defined as
f ° b = (f θ b) b
The opening operation involves grayscale erosion with the structuring element, which
shrinks the image elements, followed by dilation of the eroded elements by the same
structuring element. It thus results in smoothed contour of image, breaks narrow isthmuses,
and eliminates thin protrusions. Opening can erase white holes on dark objects, or can remove
14

small white objects in a dark background. An object will be erased if the SE does not fit within it
[12].
Hat Transformations

Hat transformations can be used for contrast enhancement. There are two hat
operations and are known as the top hat and bottom hat transformations. Top hat operation is
the result of subtraction of an opened image from the original one, mathematically,
TH = f – (f ° b)
Where (f) is the input image and (b) is the structuring element. Similarly, the bottom hat
operation, is defined as the closing of the image minus the image, mathematically,
BH = (f• b) – f
The Bottom hat transformation may be used where the image is the complement. The
top hat operation suppresses the dark background and highlights the foreground objects.
Regardless of the number plate color, the characters (i.e., text and numerals) on the vehicle
plate are usually bright colored and contrast the color of the plate. Morphological closing
operation is carried out using a rectangular structuring element of a size much greater than the
inter‐character spacing of the license plate characters, resulting in blurring of the plate
characters in the license plate. Subtracting the resultant image from the original thresholded
image removes large parts of constant intensity background. This operation thus highlights the
characters and suppresses the irrelevant background. The top‐hat operator assumes that the
car plates have a clear background and dark foreground. In the opposite case one could use an
opening operator instead of a closing one. Obtaining the binary equivalent of the resulting
image and removal of small scale features or components, leaves only a few plate designate
foreground areas are been, with most of the irrelevant objects removed [11, 12].
15

Figure 2.8: From left to right: binary equivalent of original RGB image, the result of
morphological closure, the top‐hat, i.e., the difference between the previous two images.

Texture based Segmentation:

This segmentation scheme is based on texture analysis. Texture analysis refers to the
characterization of regions in an image by their texture content. Texture analysis attempts to
quantify intuitive qualities described by terms such as rough, smooth, silky, or bumpy as a
function of the spatial variation in pixel intensities. In this sense, the roughness or bumpiness
refers to variations in the intensity values, or gray levels.
Generally, license plates have bright characters on dull backgrounds or dark characters
on bright backgrounds. This high contrast is meant to improve legibility, but also results in high
frequency transitions in gray‐level in the horizontal direction. Texture analysis may be done
using functions that filter an image using local statistical measures; range, standard deviation
and entropy. These statistics can characterize the texture of an image because they provide
information about the local variability of the intensity values of pixels in an image. For example,
in areas with smooth texture, the range of values in the neighborhood around a pixel will be a
small value; in areas of rough texture, the range will be larger. Similarly, calculating the
standard deviation of pixels in a neighborhood can indicate the degree of variability of pixel
values in that region.

16

Figure 2.9: Result of range‐filtering the binary image in figure 5. All objects in the image display
high frequency variation in inter‐pixel intensity in the horizontal direction, implying large
changes in pixel intensity magnitude within a 3×3 neighborhood.

2.3 CHARACTER SEGMENTATION SCHEMES:

The character segmentation acts as a bridge between the license plate extraction and
optical character recognition modules. Its main function is to segment the characters on the
chosen candidate region such that each character can be sent to the optical character
recognition module individually for recognition.
2.3.1 Bounding Box character Segmentation

The bounding box of an image element is the smallest rectangle containing the element.
It is defined by the co‐ordinates of the upper left corner of the bounding box and a
17

corresponding one dimensional vector that specifies the width of the bounding box along each
dimension. The elements included in the bounding box are defined by connectivity used.
Connectivity defines which pixels are connected to other pixels. A set of pixels in a
binary image that form a connected group is called an object or a connected component.
Determining which pixels create a connected component depends on how pixel connectivity is
defined.
Neighborhood pixels are 4‐connected if their edges touch. This means that a pair of
adjoining pixels is part of the same object only if they are both on and are connected along the
horizontal or vertical direction. Alternatively, the neighborhood pixels are 8‐connected if their
edges or corners touch. This means that if two adjoining pixels are on, they are part of the same
object, regardless of whether they are connected along the horizontal, vertical, or diagonal
direction [4].
Characters in the image are obtained as elements from a label matrix. A label matrix is
an image, the same size as the input image, in which the objects in the input image are
distinguished by different integer values in the output matrix. Objects in a binary image are
pixels that are on, i.e., set to the value 1; are considered to be the foreground. Pixels that are
off, i.e., set to the value 0, are considered to be the background. As an illustration of how
labeling can identify the objects in a binary image, in the input matrix, BW, the pixels set to the
value 1 represent objects in the image. The pixels set to 0 represent the background.

Figure 2.10: A sample approximation to the binary representation of an image
18

In the corresponding output label matrix, X, labeling finds every object in the input
image and numbers them, in the order it finds them.

Figure 2.11: The corresponding label matrix derived from the image [4]

The pixels labeled 0 are the background. The pixels labeled 1 make up one object; the
pixels labeled 2 make up a second object, and for the third object all pixels are set to 3. Labeling
uses pixel connectivity to determine where the boundaries between objects are, in an image.
Each element, marked by a different label is then enclosed in a suitable bounding box which
can then be used to crop the elements from the parent image.
19

Figure 2.12: Label matrix derived from result of the logical AND between texture mask (Figure
2.9) and top‐hat filtered image (Figure 2.8). Segmented objects corresponding to license plate
characters are shown in different color from the jet color‐map

2.3.2 Histogram Analysis

This segmentation method is based on vertical projection of the characters. The vertical
projection of an edge image is determined by calculating the number of “ones” for each row of
the image, and it illustrates the distribution of the pixels in an image. The Vertical projection is
a diagram with two axes; the vertical axis is the pixel count per column of the license plate,
while the horizontal axis is the width of the license plate (from left to right). A “peak‐to‐valley”
function may be defined on the vertical projection of the candidate image. Values of this
function show probable segmentation points which correspond to deep “valleys” surrounded
with high “peaks” in the vertical projection. If there is no noise between the characters, there is
a sharp valley for which pixel count equals 0. This is the break point between characters. A low
value may thus be used as the threshold. A pixel count greater than the threshold value in a
20

column, will be considered to be a character. By locating these valleys, the positions of the
characters are found and segmented.

Figure 2.13: Extracted license plate with no noise between characters

Figure 2.14: The corresponding vertical projection

Figure 2.15: The isolated characters [9]

21

If there is noise between the characters, there is no sharp valley and it is difficult to find
the break points. The width of each of the separated characters may be used to aid
segmentation. If the width of a segmented character is greater than 20 pixels, it is composed of
several characters and need further segmentation.
If a low threshold value in vertical projection yields merged characters, the width of
these merged characters greatly exceeds the expected width of one character. The threshold is
thus increased by one until the width of the merged characters reduces to the expected
character width. With each iterative increment of the threshold, the merged characters are
incrementally segmented until all objects’ widths attain the pre‐defined character width, then,
the positions of all the characters will have been found and the characters segmented [9].

2.4 CHARACTER RECOGNITION SCHEMES:

2.4.1 Neural Network Classifiers

Multilayered feed‐forward neural networks have been used for LP character
identification in many works [5, 12, 16], usually following a common methodology. The classical
training method for feed‐forward neural networks is error back‐propagation. The network has
to be trained for many training cycles to attain good performance. Input vectors and the
corresponding target vectors are used to train a network until it can approximate a function,
associate input vectors with specific output vectors, or classify input vectors in an appropriate
way as defined. Networks with biases, a sigmoid layer, and a linear output layer are capable of
approximating any function with a finite number of discontinuities.
Standard back‐propagation is a gradient descent algorithm, in which the network
weights are moved along the negative of the gradient of the performance function. The term
back‐propagation refers to the manner in which the gradient is computed for nonlinear
multilayer networks. There are a number of variations on the basic algorithm that are based on
other standard optimization techniques, such as conjugate gradient and Newton methods. The
simplest implementation of back‐propagation learning updates the network weights and biases
22

in the direction in which the performance function decreases most rapidly, the negative of the
gradient. Properly trained back‐propagation networks tend to give reasonable answers when
presented with inputs that they have never seen. Typically, a new input leads to an output
similar to the correct output for input vectors used in training that are similar to the new input
being presented. This generalization property makes it possible to train a network on a
representative set of input/target pairs and get good results without training the network on all
possible input/output pairs. The number of hidden layers, as well as the number of respective
neurons, may be defined after a trial‐and‐error procedure. Ideally, the network responds with a
1 in the position of the letter being presented to the network, while all other values in the
output vector would be 0 [15]. In practice, a character may be confidently recognized only if the
corresponding output neuron exceeds 0.85 and all other output neurons have an output level
below 0.25, where 1 denotes a 100% confident level for the output.
Self‐organized neural networks based on Kohonen’s self‐organized feature maps may be
implemented to handle noisy, deformed, broken, or incomplete characters acquired from LPs
that were bent and/or tilted with respect to the camera [16]. The method focuses on accuracy
at the cost of increased complexity and execution speed. To overcome misclassification of the
similar character pairs (8, B), (0/D), and (O/D), an ambiguity set that contains the pairs of the
misclassified characters may be predefined. During character recognition, once an unknown
character is classified as one of the characters in the ambiguity set, an additional minor
comparison between the unknown character and the classified character is performed. The
comparison then focuses only on the non‐ambiguous parts (Fig 2.16).

Figure 2.16: Distinguishing parts of ambiguous characters [5]

23

2.4.2 Template Matching Approach

The pattern matching technique is a suitable technique for the recognition of single‐
font, not‐rotated, and fixed‐size characters. Although this method is preferably used in binary
images, properly built templates also obtain very good results for gray‐level images. A template
matching application may have the recognition process based on the computation of the
normalized cross‐correlation values for all the shifts of each character template over the sub‐
image that contains the LP. This method of template matching uses a convolution mask or
template, tailored to a specific feature of the search image, which is to be detected. It is
intuitively likely that the convolution output will be highest at places where the image structure
matches the mask structure, where large image values get multiplied by large mask values. The
center or origin of the template is moved over each point in the search image and the sum of
products between the coefficients of the template and the image over the whole area spanned
by the image is calculated. As all possible positions of the template with respect to the search
image are considered, the position with the highest score is the best position. This method is
sometimes referred to as linear spatial filtering, and the template is called a filter mask.
Alternatively the normalized correlation coefficient between template, A, and the scaled
image, B, may be computed as [13]

Where (A) is the mean of the template and (B) is the mean of the scaled image. The metric value
is 0 for two totally different matrices and 1 for correlated ones [14]. If the matrices are holding
binary values of character templates, the metric could be used for matching. Rescaling the
character images in order to calculate the metric is thus necessary.
24

CHAPTER THREE: PROBLEM SOLUTION AND

DESIGN
The automatic license plate recognition application was developed using the license
plate extraction, character segmentation and optical character recognition modules as
described by the flowchart for a generalized ALPR system (Fig 2.1). The first section of the
project involved merging previous work with the current project. The 108 images obtained in
the previous work were used in the current project. The automatic number license plate
presented in this work was developed in MATLAB 7.6. Implementation was grouped into image
processing, optical character recognition and database blocks.

3.1 IMAGE PROCESSING BLOCK:

This block received input images of the vehicles and produced cropped images which
were passed to the optical character recognition block that succeeds it. A flowchart illustrating
the location and extraction of license plate characters is shown below (Fig 3.1). The image
processing block was based on image pre‐processing and segmentation steps. First, the input
color (RGB) images were converted into grayscale images. An RGB image, sometimes referred
to as a true‐color image, is stored as an m‐by‐n‐by‐3 data array that defines red, green, and blue
color components for each individual pixel. The color of each pixel is determined by the
combination of the red, green, and blue intensities stored in each color plane at the pixel's
location. A grayscale image is a data matrix whose values represent intensities within some
range.
25

26

Figure 3.1: Flowchart indicating the license plate character location algorithm

This conversion allowed ease of thresholding, which created a binary image using a
certain threshold. Adaptive thresholding using the Otsu thresholding scheme; which assigned
pixels with grey‐level above a threshold (chosen so as to minimize the intra‐class variance of
the black and white pixels) in the grayscale scale image were assigned binary value (1) in the
binary image, while those below were assigned binary value(0) was implemented using the
inbuilt graythresh function.
Next, morphological operations opening and closing were used. The morphological
closing operation was carried out using a rectangular structuring element of a size much greater
than the inter‐character spacing of the license plate characters, resulting in blurring of the
plate characters in the license plate. Subtracting the resultant image from the original
thresholded image, which constituted top‐hat filtering, removed large parts of constant
intensity background, leaving plate characters and other fine details in the image intact. A
morphological opening operation with a structuring element, of width less than the inter‐
character spacing, was then used to remove the unimportant fine details without affecting the
characters themselves.
An image mask which isolated the regions with the largest difference in pixel intensities
in a given neighborhood was then developed using a range filter. The function rangefilt was
used to obtain an output array of same size as the input image, where each output pixel
contained the range value, as the difference between maximum and minimum pixel intensity
values in the 3‐by‐3 neighborhood around the corresponding pixel of the input image. The
range filtered image was then binarized using an experimentally determined threshold of 0.4,
which preserved as much license plate detail as possible, without including too many noise
objects. A morphological closing operation was then done on the binary image with a 9‐pixel,
square structuring element to remove thin objects in image due to local transitions at the edges
of the input image. Larger structuring elements merged objects, yielding large area masks that
left many noise objects when overlaid with the top‐hat filtered image, whereas smaller
structuring elements yielded fragmented characters. An area opening was then used to
27

remove from the binary image all connected objects that had fewer than 300 pixels, the figure
being determined by trial and error, to produce the binary masking image.
The image created by over‐laying the top‐hat filtered image with the mask by carrying out the
logical AND of the two images was then labeled using the bwlabel function. The bwlabel
function was used to search for connected components and labeled them with unique
numbers; it takes a binary input image and a value specifying the connectivity of objects. The
function returns a matrix L of the same size as the input image, containing labels for the
connected objects in that input image, and the number of connected objects found in the input
image. The elements of L are integer values greater than or equal to 0.
The regionprops function was then used to measure object or region properties in an
image and returned them in a structure array. When applied to an image with labeled
components, it creates one structure element for each component; with the structure array
having area, centroid and bounding box fields. The area field is scalar representing the actual
number of pixels in the region. The bounding box field is a vector representing the smallest
rectangle containing the region defined by co‐ordinates of the upper left corner of the
bounding box and the width of the bounding box along each dimension, in length and height.
The centroid field is a vector that specifies the center of mass of the region; the first element of
centroid is the horizontal coordinate (or x‐coordinate) of the center of mass, and the second
element is the vertical coordinate (or y‐coordinate).
The centroid field of the regionprops function was used to determine the vertical
coordinate of all labeled objects obtained from the logical AND of the top‐hat filtered image
and the texture map. Since few objects were members of either set, resultant intersection of
both sets was composed largely of license plate characters. The y‐coordinate values
corresponding to the objects were observed to be randomly distributed over the image height,
but objects corresponding to license plate characters had approximately the same centroid
value. This variation in the centroid value was exhibited in the ones digit of the value; hence a
rounding‐off of the centroid values to the nearest ten‐arrived at by trial and error‐eliminated
this variation. The mode of the rounded vertical coordinates of objects in this intersection was
thus the average height of the license plate characters on the image. Selecting all objects
28

intersecting with a horizontal line, running across the image at this modal height yielded the
license plate characters. In very few cases, noise objects were included. The area field of the
regionprops function was used to remove all noise objects with an area of less than 100 pixels,
determined as the minimum area of license plate character‐corresponding to the numeral 1.
Further filtering of noise objects required the orientation of the major axis of the ellipse
that has the same variance as the region of each object lie between 45 to 90 degrees from the
horizontal axis. This allowed removal of horizontally aligned noise objects resulting from
dilation of fragmented license plate edges due to the closing operation. The Euler number of
each object was also used to remove noise objects. The Euler number is a measure of the
topology of an image. It is defined as the total number of objects in the image minus the
number of holes in those objects [4]. Character (B) has Euler number of ‐1, indicating that the
number of holes is greater than the number of objects; characters such as (A, O, P, D, Q) have
an Euler number of zero, whereas all other simple characters have an Euler number of 1.Due to
nails in the license plate, noise was introduced to character objects due to the connectivity
used in labeling. An object having an Euler number of 1 in isolation would show an Euler
number of two if such a nail object was introduced in its bounding box due to connectivity. As a
result, all objects with an Euler number greater than 2 or less than ‐1 were considered to be
noise objects and were removed. The remaining objects were subsequently considered to be
characters and were passed to the optical character recognition block.
The bounding box field of the regionprops function was then used to obtain the smallest
rectangles containing each object in the resultant image. These rectangles were then passed to
the imcrop function which cropped the required labeled objects corresponding to the license
plate characters in the image, based on the rectangles’ top‐left coordinates, their widths, and
heights respectively. It is these objects that were passed to the optical character recognition
block. The quality of segmentation is strongly related to the choice of the structuring element’s
size on the plate enhancement phase, choice of the threshold used for image binarization, the
relative angle between camera and plate and the quality of the image [11].

29

3.2 THE OPTICAL CHARACTER RECOGNITION BLOCK:
3.2.1Neural Network Classifier Approach

A back‐propagation neural network classifier was first adopted and was trained on
thirty four vectors ;corresponding to all possible characters, excluding ‘I’ and ‘O’ which are not
used in Kenyan license plates; each having 150 elements; corresponding to image objects of
resolution 15×10. Each target vector was a 34‐element vector with a 1 in the position of the
letter it represents, and 0's everywhere else. For example, the letter A is to be represented by a
1 in the first element (as A is the first letter of the alphabet), and 0's in elements two through
thirty four. The network received the 150 Boolean values as a 150‐element input vector. It was
then required to identify the letter by responding with a 34‐element output vector. The 34
elements of the output vector each represent a letter. To operate correctly, the network would
respond with a 1 in the position of the letter being presented to the network. All other values in
the output vector would be 0.The neural network thus had 150 inputs and 34 neurons in its
output layer to identify the letters. The network was a two‐layer log‐sigmoid network, with 10
neurons in the hidden layer. Training was done for 5000 iterations, for gradient descent with
momentum. Gradient descent with momentum, implemented by the traingdm function, allows
a network to respond not only to the local gradient, but also to recent trends in the error
surface. Momentum allows the network to ignore small features in the error surface, hence
slide through shallow local minima. Without momentum a network can get stuck in such a
minimum.
While the neural network classifier is usually preferred owing to its higher cognitive rate
and its ability to give reasonable results when presented with new objects which it had not
been trained on, several shortcomings resulted in adoption of the template matching approach.
With character recognition for license plate applications, the neural network classifier requires
frequent retraining, preferably for each car image due to the large input vector size, having 150
binary elements for each character.
30

Accuracy improves as the number of neurons in the hidden layer and as size of input
training vector increases. Increasing number of neurons in the hidden layer results in a speed
penalty while increasing size of the input training vector is limited by memory constraints.
Training required either 5000 iterations or a steady‐state error of less than 0.1, which made it
slower than the template matching approach. In this case, the neural network classifier had a
poor cognitive rate and misclassified well segmented and obvious characters.

3.2.2 The Template Matching Approach

The template matching approach was then implemented. The templates used had a
resolution of 42×24, hence rescaling the license plate objects prior to template matching was
necessary. Character recognition was based on calculating the correlation metric, implemented
using the corr2 function, which computes the correlation coefficient between two matrices of
the same size. All images from extracted objects and the template set were thus represented as
42×24 intensity matrices. The template images corresponding to the 34 possible characters A to
Z and 0 to 9 excluding letters I and O were saved, and template matching was implemented by
using the correlation between each extracted object from the image against all the images in
the template. This removed the presumption that all license plates begin with letters (which
would affect recognition of CD license plates) or that license plate begins with letter K (which
would affect recognition of foreign license plates).The correlation coefficients for each
extracted object with the template set was ordered into a 34 element array, and the index of
the element having the highest correlation coefficient used to identify the corresponding
similarly indexed character. The characters corresponding to each extracted object were then
concatenated to form a string, which is the detected vehicle registration number.

31

Figure 3.2: Flowchart indicating the template matching algorithm
32

3.3 Database Implementation:

For a car identification access system a database was needed. Since the basic
segmentation and character recognition modules had been implemented, the output string
from these modules was to be used to query a test database and extract hypothetical user
data. While the original idea was to use a MYSQL database and use a PHP script to query the
database, this was found to be unnecessary. MATLAB supports database queries from most
databases since it implements Java database connector (JDBC), and supports JDBC to ODBC
inter‐conversion. This database support is implemented by the MATLAB Database toolbox
which requires that a data source should be set up first. The data source consists of data that
the toolbox accesses and information required to find the data, such as driver, directory, server,
or network names. Data sources interact with ODBC drivers or JDBC drivers. An Object database
connector (ODBC) driver is a standard Windows interface that enables communication between
database management systems and SQL‐based applications. A JDBC driver is a standard
interface that enables communication between applications based on Java and database
management systems. The Database toolbox software is based on Java. It uses a JDBC/ODBC
bridge to connect to the ODBC driver of a database, which is automatically installed as part of
the MATLAB Java virtual machine (JVM).
Thus a Microsoft Access data source was set up, and a test database was created, having
three fields; the car registration number, the car’s color and its make or model; with the
registration number field as the primary key. A MATLAB script was then used to connect to this
database using the JDBC/ODBC driver bridge to obtain a connection object which was then
used to pass SQL queries to the test database. In order to access the database first a connection
to the database was created using the database function. This function takes data source,
username and password as arguments. The data source is the name of the data source in the
ODBC for Windows. This has to be configured before running MATLAB. The username the user
has on the database to be accessed. The password is given with the defined username to access
the database. These arguments are passed to the database function as strings.
33

Once the connection is established queries to the database were performed using the
fetch function. This function takes a connection pointer and an SQL query string as arguments.
The connection pointer is the connection created with the database function. The SQL query
string contains the desired SQL query. The queries used were based on the license number
string, passed from the character recognition block. User data corresponding to the predefined
fields was thus retrieved.
The database queries however failed if fewer than 4 characters had been misidentified
due to deletion of the characters in the image processing block, or due to merging of
characters. When used for giving specific vehicles access to a barriered area the decision to
have an error rate of two characters was in the author’s opinion, viewed as acceptable. This is
because the likelihood of an unauthorized car having such a similar license plate with all
detected characters in their right order was seen as quite small [1].

34

CHAPTER FOUR: RESULTS
A set of 108 images, with a resolution of 640×480 pixels was obtained from the previous
work. The images were obtained with the camera position set at a distance of between 0.5 and
2 metres from the vehicle. Of the 108 images, 100 images were used to test the developed
software, and 68 cases were satisfactorily recognized.
The successful extraction of license plate characters was limited if image had large local
variances in pixel ranges in regions other than the license plate, vehicle had bent or warped
license plates , characters on the license plate were faded,or had very poor contrast relative to
bright regions in the image, or if the the vehicle in the image was too close to the camera.
Faded plates and texture conflicts presented the most difficult test cases. Of the 108 images, 7
images had faded plates, 4 images were obtained with vehicle too close to the camera, 3
images had signifcantly warped plates. Errors in recognition were largely caused by
misclassifcation of the characters by the template matching algorithm. Efforts to improve its
cognitive rate, by use of character standard deviations or Euler numbers to verify the
recognized characters, were hindered by an intolerable increase in processing time.The
recognition performances for simple visible plates was higher, at about 78%, , i.e. 62 correct
identifications in a sample of 80 images. Contention caused by misclassified digits was resolved
by listing all probable license plates from the database.

35

Figure 4.1: Input image (left) and the resulting extracted license plate characters. The horizontal
noise objects violate orientation requirements, and are thus not considered

Figure 4.2: Input image having large texture regions in areas other than the license plate, due to
shadows on car windshield. In this case, extraction of license plate characters is satisfactory

Figure 4.3: Limiting case, showing car on coarse stone background. In this case the texture mask
is overwhelmed by the numerous and large changes in local pixel intensities

36

Figure 4.4: Images show same vehicle in subsequent frames

Figure 4.5: While noise objects impeded proper extraction of license plate characters from the
first frame, successful extraction from the subsequent frame was possible

Figure 4.6: Image with warped plates occluding license plate characters. In this case, an
insufficient number of characters (A and 1) were extracted to uniquely identify the vehicle
37

Figure 4.7: Images of vehicles with faded plates

Figure 4.8: Extracted license plate characters are too fragmented to be useful

Figure 4.9: Input image (left) is too close to the camera. The fixed size of the structuring
element used in the top hat filter results in fragmented characters
38

CHAPTER FIVE: DISCUSSION, RECOMMENDATIONS
AND FUTURE WORK
DISCUSSION
The 68% cognitive rate obtained was due to several limiting conditions encounetred as
described by the previous chapter. Texture conflicts were caused by large local variances in the
image regions other than the license plate, such as caused by shadows or coarse stone
backgrounds, and corresponded to significant changes in local pixel intensity ranges. They were
thus regarded as possible candidate regions by the texture mask,and thus introduced noise
objects. Warped or bent plates occluded several characters on the license plate resulting in
insufficient detected characters.Dirty plates and obstructing objects like screws or incorrectly
positioned technical check stickers had the same effect. Faded characters and poor contrast of
the license plates relative to the rest of the image, caused by uneven illumination resulted in
false classifications since the license plate characters were removed along with the constant
intensity background by the top hat filter, leaving only regions whose brightness varied from
that of the general background. For images where distance between the camera and the
vehicle was too small, the fixed size of the structuring element used in the top hat filter
resulted in fragmented characters. The vehicle characters were best recognized if the camera
and the vehicle were at least one meter apart.
The overall success rate of 68% for the ALPR developed shows the need for further
development for use in a real world environment. While there exists claims of Automatic
Number Plate Recognition with cognitive rates in excess of 98%, the experience of system
operators is that overall read rates for license plates are 90% to 94% in ideal conditions with
excellent modern systems. In some older systems overall performance rates are estimated to
be between 60% and 80% [1].
39

RECOMMENDATIONS AND FUTURE WORK
• To improve the selection of the proper license plate characters in the presence of noise
objects, sliding windows may be used to search entire image, replacing the selection of
the characters using the mode of object heights in the image. This should however be
done prudently as there is a processing time penalty implied.
• As illustrated in the previous chapter (figs 4.4 and 4.5), use of several frames is the
single most important change which would greatly improve the accuracy of the license
plate character extraction. These multiple frames would be processed separately and
then averaged, prior to license plate extraction, or characters could be extracted from
the separate frames and a confidence level assigned for each extracted set; the
extracted character set having the highest confidence level would be assumed to be the
correct characters. In this case, such confidence levels could be obtained from the
average value of correlation coefficients used to identify characters in each set.
• The use of template matching for character recognition, on its own has a relatively high
error rate. While a neural network classifier was tested, its long training times
discouraged further development. For satisfactory character recognition improvements
on either classifier, such as combining vertical projection with template matching, use of
templates based on local license plates characters would improve correlation and use of
alternative metrics if template matching is to be used, will be necessary.

40

CONCLUSION

The car number plate recognition software, comprising of the license plate extraction,
character segmentation and optical character recognition modules was designed and
implemented. A suitable database with hypothetical user data was also incorporated to
complement the system. The ANPR achieved an overall success rate of 68% when tested on 100
of the 108 images, with recognition performances for simple visible plates close to 80%.Results
may be improved by refining the recognition stage and testing other classifiers. Different
character templates could be used for such refinement of the recognition stage. Future work is
intended to be done in improving and testing the system on a larger number of images.

41

REFERENCES

[1] Automatic number plate recognition; Available, http://en.wikipedia.org
[2] License Plate Recognition ‐ A Tutorial; Available, http://www.anpr‐tutorial.com
[3] V. Ganapathy, W.L.D. Lui, “A Malaysian Vehicle License Plate Localization and Recognition
System”, School of Engineering, Monash University, Malaysia
[4] MATLAB Image Processing ToolboxTM 6 User’s guide
[5] C.N. E. Anagnostopoulos, I. E. Anagnostopoulos, I. D. Psoroulas, V. Loumos, E. Kayafas,
“License Plate Recognition from Still Images and Video Sequences: A Survey”, IEEE
Transactions on Intelligent Transportation Systems, vol. 9, No. 3, Sept. 2008
[6] B.S. Morse, “Lecture 4: Thresholding”, Brigham Young University
[7] R.C. Gonzalez, R.E. Woods, Digital Image Processing, 2nd Edition, Prentice Hall,
Englewood Cliffs, NY, 2002, pp. 416‐453
[8] K. I. Kim, K. Jung, and J.H. Kim, “Color Texture‐Based Object Detection:
An Application to License Plate Localization”
[9] C. W. Yi, “License Plate Recognition”,
[10] V. Abolghasemi, A. Ahmadyfard, “Local Enhancement of Car Image for License Plate
Detection”, Shahrood University of Technology, Shahrood, Iran
[11] F. Odone, “Experiments on a License Plate Recognition System”, DISI, Università degli
Studi di Genova, Technical Report number: DISI‐TR‐06‐17
[12] H. K. Sulehria, Y. Zhang, D. Irfan, A. K. Sulehria, “Vehicle Number Plate Recognition
Using Mathematical Morphology and Neural Networks”, School of Electronics and
Information Engineering, Harbin Institute of Technology, Harbin, PR China
[13] MATLAB Statistics ToolboxTM 6 User’s guide
[14] R. Brad, “License Plate Recognition System”, Computer Science Department,
Lucian Braga University, Sibiu, Romania
[15] MATLAB Neural Network ToolboxTM 6 User’s guide
42

[16] E. K. Lim, H. K. Yang, and K. B. Kim, “Recognition of car license plate using Kohonen
algorithm,” in Proc. ITC‐CSCC, 2000, vol. 2, pp. 785–788
APPENDICES

APPENDIX A: Thresholding Schemes

A.1 Maximum Entropy Criteria:

The Maximum Entropy Criterion provides an unsupervised solution to the choice of
threshold dilemma. Consider f(x, y), an image of N x N pixels and m gray levels. If Gm = {0, 1...
(M‐1)} are the gray levels and (f i ) are the frequencies of appearance the gray levels in image f,
the probability of level (i) in the image f, will be:

For a given s gray level, if sum of these probabilities up to the (s‐1) gray level is bounded,
0< <1 and for the normalized distributions
, may be
defined. The basic idea of the MEC method is the appropriate choice of threshold that
maximizes the amount of information obtained from the object and background. Recalling that
the measure of information is the entropy, the total amount of information given by A and B is:
TE(s) =EA(s) ‐EB(s)

The Maximum Entropy Criterion finds the threshold s’ that maximizes the total entropy TE(s)
[14].
A.2 Otsu Thresholding Scheme:

Otsu's method is used to automatically perform histogram shape‐based image
thresholding, or, the reduction of a gray‐level image to a binary image. The algorithm assumes
43

that the image to be thresholded contains two classes of pixels, foreground and background,
then calculates the optimum threshold separating those two classes so that their combined
spread (intra‐class variance) is minimal. The threshold is set to make each cluster as tight as
possible, thus minimizing their overlap. For fixed distributions adjusting the threshold one way,
increases the spread of one and decrease the spread of the other. The goal then is to select the
threshold that minimizes the combined spread. In Otsu's method a search for the threshold
that minimizes the intra‐class variance, defined as a weighted sum of variances of the two
classes is performed.
The within‐class variance as the weighted sum of the variances of each cluster may be defined
as

+
Where is the variance of the pixels in the background, with intensity below the
threshold value, is the variance of pixels in the foreground, whose intensity is above
the threshold value and [0, N − 1] is the range of intensity levels. The probabilities of these two
classes, separated by a threshold (T) thus are the weights used for this weighted sum. Subtracting
the within‐class variance from the total variance of the combined distribution yields the
between‐class variance:

+
Where the combined variance and μ is is the combined mean. The between‐class variance is
thus the weighted variance of the cluster means themselves about the overall mean.
Simplification may be done by factorizing the weights (i.e. the probabilities)

44

So, for each potential threshold T, the pixels are separated into two clusters according
to the threshold. The mean of each cluster is found and the square of the difference between
the means calculated. Finally, the number of pixels in one cluster is multiplied by the number of
pixels in the other. This method depends only on the difference between the means of the two
clusters, thus avoiding having to calculate differences between individual intensities and the
cluster means. The optimal threshold is the one that maximizes the between‐class variance or,
conversely, minimizes the within‐class variance [6].
45

APPENDIX B:
Source code

The MATLAB code presented here was that derived from the design stage. It involves
creation of a texture mask and a top hat filtered image. Their intersection contains objects
which are filtered using area, orientation and Euler number criteria to extract desired
characters. Character recognition is by use of template matching.

%CREATE TEMPLATES
%Letters
A=imread ('letter_numbers\A.bmp'); B=imread ('letter_numbers\B.bmp');
C=imread ('letter_numbers\C.bmp'); D=imread ('letter_numbers\D.bmp');
E=imread ('letter_numbers\E.bmp'); F=imread ('letter_numbers\F.bmp');
G=imread ('letter_numbers\G.bmp'); H=imread ('letter_numbers\H.bmp');
J=imread ('letter_numbers\J.bmp');
K=imread ('letter_numbers\K.bmp'); L=imread ('letter_numbers\L.bmp');
M=imread ('letter_numbers\M.bmp'); N=imread ('letter_numbers\N.bmp');
P=imread ('letter_numbers\P.bmp');
Q=imread ('letter_numbers\Q.bmp'); R=imread ('letter_numbers\R.bmp');
S=imread ('letter_numbers\S.bmp'); T=imread ('letter_numbers\T.bmp');
U=imread ('letter_numbers\U.bmp'); V=imread ('letter_numbers\V.bmp');
W=imread ('letter_numbers\W.bmp'); X=imread ('letter_numbers\X.bmp');
Y=imread ('letter_numbers\Y.bmp'); Z=imread ('letter_numbers\Z.bmp');
%Numbers
one=imread ('letter_numbers\one_.bmp'); two=imread ('letter_numbers\two_.bmp');
three=imread ('letter_numbers\three_.bmp'); four=imread ('letter_numbers\four_.bmp');
five=imread ('letter_numbers\5.bmp'); six=imread ('letter_numbers\6.bmp');
seven=imread ('letter_numbers\7.bmp');eight=imread ('letter_numbers\8.bmp');
nine=imread ('letter_numbers\9.bmp'); zero=imread ('letter_numbers\0.bmp');
%*‐*‐*‐*‐*‐*‐*‐*‐*‐*‐*‐
letter=[A B C D E F G H J K L M N P Q R S T U V W X Y Z];

number= [one two three four five six seven eight nine zero];
character= [letter number];
Templates= mat2cell (character, 42,[24 24 24 24 24 24 24 ...
24 24 24 24 24 24 24 24 24 24 24 24 24 24 24 24 24 24 24...
46

24 24 24 24 24 24 24 24]);

save ('templates','templates')
47

% Use correlation to implement template matching
function letter=read_letter (im,num_letras)
% computes the correlation between template and input image
% and its output is a string containing the letter.
% letter=read_letter (im)
load templates
global templates
comp=[ ];
for n=1:num_letras
sem=corr2(templates{n}, im);
comp=[comp sem];
end

tt=size(comp,2);
for i=1:1:tt
if comp(i)==max(comp);
vd=i;
end
end
%*‐*‐*‐*‐*‐*‐*‐*‐*‐*‐*‐*‐*‐
if vd==1
letter='A';
elseif vd==2
letter='B';
elseif vd==3
letter='C';
elseif vd==4
letter='D';
elseif vd==5
letter='E';
elseif vd==6
letter='F';
elseif vd==7
letter='G';
elseif vd==8
letter='H';
elseif vd==9
letter='J';
elseif vd==10
letter='K';
elseif vd==11
letter='L';
elseif vd==12
letter='M';
elseif vd==13
letter='N';
48

elseif vd==14
letter='P';
elseif vd==15
letter='Q';
elseif vd==16
letter='R';
elseif vd==17
letter='S';
elseif vd==18
letter='T';
elseif vd==19
letter='U';
elseif vd==20
letter='V';
elseif vd==21
letter='W';
elseif vd==22
letter='X';
elseif vd==23
letter='Y';
elseif vd==24
letter='Z';

%*‐*‐*‐*‐*
elseif vd==25
letter='1';
elseif vd==26
letter='2';
elseif vd==27
letter='3';
elseif vd==28
letter='4';
elseif vd==29
letter='5';
elseif vd==30
letter='6';
elseif vd==31
letter='7';
elseif vd==32
letter='8';
elseif vd==33
letter='9';
else
letter='0';
end

49

%main program
%select file to read image from
clear all,close all
[filename, pathname, filterindex] = uigetfile({'*.jpg', 'JPEG files (*.jpg)';}, 'Select an Image to Process');
a = imread([pathname filesep filename]);

%create texture mask
R=rangefilt(a,ones(5));
B=im2bw(R,.3);
b=bwareaopen(B,500);
nhood=true(3);
c=imclose(b,nhood);
mask=imfill(c,'holes');

%top filter input image
s=rgb2gray (a);
fa =graythresh(s);
f=histeq (fa);
q=im2bw (s,f);
y=imclose (q, strel ('rectangle',[12 15]));
w=imsubtract (y,q);
e=imopen(w, strel('rectangle',[3 3]));

%mask top filtered image with texture mask hence extract most likely
%characters
ae=e & mask;
se=imopen(ae,strel('rectangle',[3 3]));
bb=bwareaopen(se,200);
[L, num]=bwlabel (bb,4);
g = regionprops(L,'Centroid');
cen = cat(1, g.Centroid);
ss=(cen(:,2)*.1);% y value of centroids
ce=round(ss);% round up to nearest 10
cet=(ce*10);
mo=mode(cet);%select number that appears most often
jk=size(se,2);
ro=ones(1,(jk*.5))*mo;
co=(1:2:jk);
ff=bwselect (se, co, ro);%select objects that intersect with this line

[L1,num1]=bwlabel (ff);%label all objects
g1=regionprops (L1,'All');
%area and orientation filtering of labeled objects
for i=1:1:num1
t=g1(i).Area;
or=g1(i).Orientation;
50

if(t>90)&&(t<700)
if (or<‐30)||( or>30)
n=g1(i).Bounding Box;%obtain fitting bounding boxes
for h=1;imcrop(ff, n);h=gcf; tmp=(1:1:h);h=h+1;%crop characters based
%on their bounding boxes
end
end
end
end

cols=size(tmp,2);
k = cell(1, cols);
for i=1:1:cols
frame=get frame(i);
[x, gray] = frame2im(frame);
l=rgb2gray(x);
p=graythresh(l);
c=im2bw(l, p);
bw2 = resizecrop(c);
eul = bweuler(bw2,4);
if eul==‐1||eul==0||eul==1||eul==2%euler number filtering
k{i}=imresize(bw2,[42 24]);
end
end

t= [];
% if element Euler number falls outside limits, cell k has empty array corresponding to these %elements.
This generates errors since corrr2 requires all matrices to be of size 24 * 42

for i=1:1:size(k,2)
t= [t, isempty(k{i})];%make matrix with cell indices of empty arrays
aj= find(~t);%find cell indices that for arrays that are non empty
for j=1:1:size(aj,2)
fim{j}=k{aj(j)};%array contains objects fulfilling euler conditions
end
end

%Storage matrix word from image
word=[ ];
%Opens text.txt as file for write
fid = fopen ('text.txt', 'wt');
% Load templates
load templates
global templates
% Compute the number of letters in template file
num_letras=size(templates,2);
for i=1:1:size(fim,2)
im=fim{i};
51

% Call function to convert image to text
letter=read_letter(im,num_letras);
% Letter concatenation
word=[word letter];
end

% Make connection to database. The password has been omitted.
% Using ODBC driver.
ti=logintimeout(7);
conn = database('car.mdb','','password');
ping(conn);

wo=['%' word(2:(size(word,2)‐2)) '%'];
wo1=[ '%' word((size(word,2)‐3):size(word,2))];
wo2=['%' word(2) '%' word((size(word,2)‐2):size(word,2))];
wo3=['%' word(2:(size(word,2)‐1)) '%' word(size(word,2))];
% Read data from database.
e = exec(conn,['SELECT ALL licensenumber,color,type FROM Table1 WHERE licensenumber LIKE' '''' wo '''' 'OR
licensenumber LIKE' '''' wo1 '''' 'OR licensenumber LIKE' '''' wo2 '''' 'OR licensenumber LIKE' '''' wo3 '''']);
e = fetch(e);
close(e)

% Close database connection.
close(conn);
query=e.Data;

for i=1:size(query,1)
for j=1:1:size(query,2)
data=query{i,j};

%fprintf(fid, '%s\n', lower(word));%Write 'word' in text file
fprintf(fid, '%s\t', data);
end
fprintf(fid,'\n');
end

fclose(fid);
%Open 'text.txt' file
winopen('text.txt');
%clear all

52

Car Number Plate Recognition Software

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Car Number Plate Recognition Software

Uploaded by

Copyright:

Available Formats

Car Number Plate Recognition Software

Project Index: PRJ033

Njagi Raphael Kinyua

Supervisor: Dr. H. A. Ouma

Examiner: Dr. V.K. Oduol

Bachelor of Science in Electrical and Electronic Engineering of the University

Department of Electrical and Information Engineering

1.5 SYSTEM OVERVIEW

1.5.1 Image Processing Block:

1.5.2 Optical character recognition block:

1.5.3 The database:

2.1 IMAGE ENHANCEMENT

2.3 CHARACTER SEGMENTATION SCHEMES:

2.3.1 Bounding Box character Segmentation

2.3.2 Histogram Analysis

2.4 CHARACTER RECOGNITION SCHEMES:

2.4.2 Template Matching Approach

CHAPTER THREE: PROBLEM SOLUTION AND

3.1 IMAGE PROCESSING BLOCK:

3.2 THE OPTICAL CHARACTER RECOGNITION BLOCK:

3.2.1Neural Network Classifier Approach

CHAPTER FOUR: RESULTS

APPENDIX A: Thresholding Schemes

0< <1 and for the normalized distributions

, may be

A.2 Otsu Thresholding Scheme:

You might also like