1. Micro-electronic R&D Center, shanghai Univ.

, shanghai 200072, China

2. Coll. of Mecha. Engin. and Automation, Shanghai Univ., shanghai 200072, China;
*zhuo.bi@shu.edu.cn,akwl@shu.edu.cn,12721137@shu.edu.cn
Abstract:A real time hardware image processing system
which is based on FPGA (Field Programmable Gate Array)
is presented in this paper. Edge detection operation is an
essential part in the field of image processing. There are
many methods to detect edge, but most of them are
implemented by software which the data processing is slow
and the calculation is complex .There are still some
hardware-based implementation which however consume
Hardware resource and take up long processing cycle, so this
paper proposes a FPGA-based approach Sobel operator edge
detection to overcome the deficiencies mentioned above. This
project utilizes Altera DE2-115 development board powered
by Cyclone IV FGPA pair with 1.3 Mega pixel CMOS camera
(ov9650). An image is captured by ov9650 which then convert
it into RGB color space for edge detection. The project is
designed with Verilog HDL, and successfully implemented on
FPGA Development Board. The results show that the cost of
only 2280 logic element the speed of real-time videos
processing can reach to 30 frames per second and can
process 640480 RGB images.
Keywords: Sobel operator, Edge Detection, FPGA.
1. INTRODUCTION
Image processing is the simulat ion of human
understanding and decision-making ability over image
informat ion. In recent years, image processing and
recognition technologies obtained an increasingly range of
applications in monitoring security areas such as automotive
safety menial system. The edge of the image is the basic
features of images. Edge detection technique has been used in
a variety of image processing applications including
extracting the edges from image and typically is utilized
before object segmentation and feature extract ion [1]. Edge of
an image refers to a collection of pixels around which there is
pixel grayscale step change. Edge of the object is reflected by
the gray discontinuity. Most of the existing edge detection
method are implemented by software, whose data processing
is slow and calculat ion complexity is high. There are still
some hardware-based implementation consuming Hardware
resource and taking up long processing cycle.
The classic image edge detection method is to detect
grayscale changes of each pixel in the area of the image by
using of adjacent first or second order derivative changes
of direct ion, with the most simple method to detect edges.
This is called edge detection local operators algorithm. Edge
detection based on first-order derivative operators include
Roberts operator, Sobel operator[2][3], Prewitt operators.
Laplace edge detection operator is based on edge detection
operator of second order derivative, which is sensitive to
noise. An improved method is to smooth the image first, and
then detection operator of second order derivative is applied.
The representation of this kind is LOG(Laplacian of Gaussian)
operator[4]. Differential method of edge detection operators
previously described, is based on a derivative of the
corresponding image and the second derivative of zero to the
maximum value of the edge - the crossover point. Canny edge
detection operator is another type of operator, detecting edges
is not by differential operators , but by deriving optimal edge
detection operator under some constraints.
Now the mainstream of edge detection is accomplished by
software, speed is slow. Also some is accomplished by
hardware, but the consumption of resources is large, the
performance is not ideal. Sobel operator, which is an operator
of 3*3, has the best respond to a steep low-noise images.
Roberts operator, Prewitt operators, Kirsch operator have
better performance for images containing grayscale gradients
and noise. Considering real-t ime effects, hardware, and
performance, we choose Sobel operators as edge detection
approach in this article. It has best performance for low noise,
simple and easy to implement hardware speed acceleration on
a method. Sobel designed in this article is realized with a
square window gradient calculation of 3*3, using two FIFO
and nine registers, getting improvement in resource utilization
and significant increase in processing speed.
This paper is arranged as follows. The first part is the
introduction of edge detection algorithm, the second section
describes the methods of Sobel operator, the third section
describes the hardware architecture, the fourth part is the
implementation and results, the fifth part is the conclusion.
2. SOBEL OPERATOR
Scenery's edge appears in the form of strength mutant in
images, so scenery edge contains a large amount of
informat ion. Because the edges of the scene has very complex
forms, therefore, the most common method of edge detection
is called the "gradient detection method". Sobel edge
detection is the use of partial differential to find the edge of
the operator. [5]
Assuming
( , ) f x y
is the image grayscale distribution
function ;
( , ) s x y
is the edge of the image gradient value,
Gx and Gy are the images which at each point contain the
REAL-TIME SOBEL EDGE DETECTOR
Weili Kong1,Pengfei Chang2, Zhuo Bi1,2
1
Paper No. T.4-3.4, pp. 1-4
horizontal and vertical derivative approximations, We will get
-1 0 +1 +1 +2 +1
-2 0 +2 ( , ) 0 0 0 ( , )
-1 0 +1 1 -2 -1
x y
G f x y G f x y
( (
( (
= - = -
( (
( (

(1)
2 2
( , ) s x y Gx Gy = +
(2)
where * here denotes the 2-dimensional convolution
operation.
Expression (1) and (2), we can get gradient value and
direction of(x,y)in the picture.
In this expression, operation of square root of
( , ) f x y
is
similar to the human visual system. In fact Sobel is a kind of
operators that using partial differential method to complete
edge detection operator method to find the edge of the
operator, Sobel gradient takes advantage of the difference
between adjacent pixels in diagonal direction, so difference is
here instead of first order partial derivatives.
Operator can be depicted as below.

| |
| |
( , ) ( 1, 1) 2 ( 1, ) ( 1, 1)
( 1, 1) 2 ( 1, ) ( 1, 1)
( , ) ( 1, 1) 2 ( , 1) ( 1, 1)
( 1, 1) 2 ( , 1) ( 1, 1)
x
x
f x y f x y f x y f x y
f x y x y f x y
f x y f x y f x y f x y
f x y x y f x y
A = + + - + + + +

+ - + +

A = + - + +

+ + - + + + +

(3)
Expression (3)'s corresponding two
3 3
templates are
shown in Fig(1). In realityeach pixel in the image can be
achieved by convolution operation of the two template,
absolute value is always used to avoid negative value in edge
detection [4].
1 2 1
0 0 0
-1 -2 -1
-1 0 1
-2 0 2
-1 0 1
Gy Gx

Fig1.Sobel operator template
Formula (2) is complex, with long period of roots and
exponential operation, large resource consumption, and not
easy to be implemented by hardware, so CORDIC algorithm
[8] is adopted. Formula (4) can be rewritten to calculate Sobel
edge detection operator approximately.
1
1
1
( , ) | |
| | | | | | 2
| | | | | | 2
i
i
i i i i
i
i i i i
g x y Gx
Gx Gx d Gy
Gy Gy d Gx
+

+
=
= - -
= + - -
(4)
Where
i
is the number of iterations;
i
d
is the sign bit.
As the value of Gx obtained from CORDI C needs to be
multiplied with a fixed value of 0.6073[8], the threshold is
divided with 0.6073 to simplify the situation. In this way, the
multiplier can be omitted.
Using the method of threshold to obtain image
segmentation with gradient values calculated through formula
(4), and then image edge extraction will be finished. The
implementation is shown in equation (5);
0 g(x,y) < ;
'( , ) =
255 g(x,y) ;
T
f x y
T

>

(5)
'( , ) f x y
is actual pixel values of a point after edge
extraction of the Sobel operator. T is the domain of image
segmentation, we here set T=11526. After processing
according to formula (4),(5), there are only t wo colors in the
image-black and white, the background is black, while edges
are white.
Sobel operators has better performance to detect vertical
and horizontal directions of step edge and line than the
diagonal edge, accurate edge location, but it is sensitive to
noise. It is suitable for image with clear edge and less noise.In
implementation process, make convolution operation between
any pixel in the image and template of 33 , and then select
the appropriate thresholds to extract edges. Sobel operator's
result is not very smooth, so the image edge detected by
above operator often need to be thin.
3. IMPLEMENTATION OF SOBEL IN HARDWARE
Sobel is an operator to use partial differential operator to
detect edges. Sobel edge detection operators are essentially
two - dimensional convolution operation on the image, we
can use formula (1)and (4) to calculate convolution
approximately. It is suitable for FPGA implementation.
Pursuant to above analysis we found that the key in the
implementation of the Sobel operator is implementation of the
square Windows shown in Fig 1, square window is used to
output three rows pixels at the same time, calculat ing values
using equation(4). Square window is used to realize gradient
calculation, results will be sent to image segmentation module,
image edge will be extracted using equation (5) and output to
the pixel data, stored in the image cache(SRAM). Pixel data
path implementation hardware structure is shown in Fig.2. the
main structure is nine 16-bit buffer registers, two 16-bit FIFO
with depth of 640 , the Sobel operator controller and a module
for image segmentation. Gett ing a 3x3 window by FIFO and
buffer registers , and then sending data to gradient calculation
cells , the result will be sent to image segmentation module.
At last, edge pixel data output to store in the image cache,
where the operator controllers finished on sequential control
edge detection.
FIF0 0
W_data 3
Pixel_data
out_data
pclk
c
o
n
t
r
o
l
l
e
r
href
vsync
FIFO 1
C1
B1
A1 A2 A3
C2
B2 B3
C3
3*3 windows
Image
segment
ation
Wr_en 0
W_data0 W_data1 Wr_en 1
Gx
Gy
>>i
>>i
+
/
-
+
/
-
register
register
di
+
+
Gx
Gy
Gxi
Gyi
Grad_en

Fig2. Sobel hardware structure
2
According to the Sobel operator structure designed in this
article, FIFO stores two line of pixels, the 16-bit cache
registers cache nine pixels data, so they can output three lines
of pixels at the same time. FIFO stores data of pixel row i and
row i+1. Controller controls FIFO read and write pointer
timing and 33 window t iming as Fig.3, pclk is pixel clock ,
href is VGA level synchronization signal , vsync is VGA
vertical synchronization signal , grad_en is enable gradient
calculation signal, we_en is write pointer enable , rd_en is
read pointer enable, read and write pointer jumps at the fall
edge of pixel clock , each write pointer has a delay of 640
periods, so does read pointer, a frame image produced 480
write pulses and 479 read pulses . Write pointer has one more
pulse than the read pointer, 640 pixels in the first row is saved
in FIFO 0. Once FIFO 0 is full, the pixel will be passed to
FIFO 1. After [(6402)+3] clock cycles the first 33 windows
will be filed and after that the throughput rate of the matrices
is one clock cycle (640 is the size of each row in the image
and 3 is number of columns of the windows).Waiting for a
second read and write pointers and reached, FIFO reads and
writes simultaneously at the three pulse read and write pointer
reaching at the same time , this keeps the FIFO always storing
640 data.
pclk
Wr_en
Rd_en
Pixel_data
480 write clock
Row 1 Row 2
640 pixel data
Grad_en
Row 480
W_data
R_data
Row 1
Row 2 Row 480
Row 478
640 pixel data 640 pixel data
href
vsync
Row 3
Row 3
Row 1
640 pixel data
Out_data
Row 1 Row 478

Fig3. FIFO read and write timing
A frame image data flow as shown Fig.3, 640 data are
stored in FIFO 0 by control unit at t ime t . Then the data in the
FIFO 0 will be read out and write in FIFO 1; meantime, the
value in FIFO 0 will be update accordingly at the time t1.
When the two FIFOs are full, the pixel will be passed to 33
windows at time t 2, (and in 3 clock cycles, the covolution of
Sobel operator and the 33 windows will be done). This
process only requires adders, and then the value of
Gx
and
Gy
can be obtained. At implementation of first pixel
gradient calculation compare with threshold value T, the one
which is greater, pixel value is '0, the one which is less,
pixel value is '255 stored in image cache. Calculate the
gradient value similarly after comparison it is stored in image
cache, such cycle for eventually to a frame of pixels are
stored in image cache.
4. EXPERIMENTAL RESULTS
In the implementation of the design process, according to
the characteristics of FPGA and the video theory in order to
determine the acquisition program and the processing of
video images.
4.1 Overview of image capture
Sobel edge detection operator designed in this paper, edge
detection algorithm based on FPGA Sobel operator in real
time will be realized by Verilog hardware description
language, and then compiled with QUARTUS II 11.0 and
Nios II 11.0 synthesis, and download to the Altera Cyclone
IV evaluation board of the DE 2-115. The RTL - level is in
Verilog HDL(the red boxes are verilog HDL description),
structure is shown in Fig.4. Using CMOS colorful image
sensor OV9650 as our image acquisition devices, configure
the internal register parameters of ov9650 by IIC controller,
realizing 16-bit RGB565 format true color image output.
Control signal by establishing the SOPC system to realize the
communicat ion between ov9650 with SOPC processor,
ov9650 pixel data passed through edge detection and sensor
module finally are stored in the SRAM, and finally are
displayed in LCD through VGA controller. Because the pixel
data of the ov9650 is 8-bit output, for the 16-bit pixels,
through sensor spliced into two 8-bit data output 16-bit pixels.
Edge detection module implemented image edge detection in
real t ime, informat ion after being processed are stored in
image and the edge protection effects are displayed via the
VGA controller. In order to meet real-t ime requirement, we
don't do anything to middle pixels, processing speed is high
and resource consumption rate is small.
Avalon
SCCB
inter
data
ov9650
I2C bus
control
-ler
NIOS
II CPU
SDRAM
Time
clock
Edge
detection
SRAM
VGA
Control
-ler
LCD
Sensor
Interface

Fig4. System diagram Based on Sobel operator's Edge
Detection on FPGA
4.2 Results
Parameters of this design's FPGA hardware resource
utilizat ion comparison with similar design is shown in table
(1), speed is up to 30 fps, 16-bit pixel data RGB565 640*480
image data output format. This 640*480 image edge detection
works well, image video is smoother. It can be found from
Table 1 that this design of edge detection has advantages in
the area, speed and frame rate over similar design.
Fig. 5 shows the edge image obtained with Sobel edge
detector(only 1/3 of the original image is displayed on the
LCD screen.).


Table1. FPGA hardware resource utilization comparison
This
paper
Reference
[4]
Referenc
e [3]
Reference
[6]
Operator
type
Sobel LOG Sobel likelihood
ratio
3
Total
Logic
Elements
2280 3074 2456 75000
Total
registers
1979 2213 1408 -
Total
power
dissipatio
n
144.74
mW
- - -
FMAX 102.21
MHz
- - 106 MHz
Memory
bits
24086 - 71887 -
Pipeline
structure
1 - - 4
Frame
rate
30fps - - 0.06s(17fps
)
Images
size
640*48
0
- 720*720 256*256


Fig.5 Sobel edge detection effect
5. CONCLUSION
This project is an example of implementing an algorithm
that is Sobel operator of edge detection and can be sped up
with a FPGA. There are many methods to detect edge, but
most of them are implemented by software which the data
processing is slow and the calculation is complex .There are
still some hardware-based implementation which consume
Hardware resource and take up long processing cycle. The
project is designed with Verilog HDL, and successfully
implemented on FPGA Development Board. The results show
that the speed of real-time videos processing can reach to 30
frames per second and can process 640480 RGB images
with the cost of only 2280 logic element.
In the near future the completed design implementation
will be presented and integrated with embedded system. The
target is to achieve function of automated edge detection
which is the basic of video segmentation application in
computer Vision system.
6. REFERENCES
|1|Milan Sanka,Vaclav Hlavac,Roger Boylc,'Image
processing,Analysis and Macine, VisionDivision oI
Thomson learning, 1999.PP52-78.
|2| X. Zhai, F. Bensaali and S. Ramalingam. 'Real -Time
License Plate Localisation on FPGA. Computer Society
Conference on Computer Vision and Pattern Recognition
Workshops, 2011,pp.14-19
[3] L.Yasri, N.H.Hamid, V.V.Yap'PerIormance Analysis
oI FPGA Based Sobel Edge Detection Operator2008
International Conference on Electronic Design
2008.December 1-3.
|4| Jincheng Wu, Jingrui Sun, Wenying Liu, 'Design and
Implementation of Video Image edge Detection System
Based on FPGA, 2010

,International Congress on Image
and Signal Processing (CISP2010)
[5] G. Anusha, JayaChandra Prasad and Satya Narayana,
'Implementation oI SOBEL Edge Detection on FPGA,
International Journal of Computer Trends and
Technology, 2012,Vol.3,pp.472-475.
[6]Quang Huy Nguyen, Ken Yoong Lee, Myo Tun Aung,
Timo Bretschneider, and Ian McLoughlin, 'Hardware-
accelerated edge detection for polarimetric synthetic
aperture radar data, Geoscience and Remote Sensing
Symposium. 2009,Vol.4,pp.204-207.
[7] Wenshuo Gao, Lei Yang, Xiaoguang Zhang, Huizhong
Liu, 'An Improved Sobel Edge Detection, Computer
Science and Information Technology
(ICCSIT),2010,pp.67-71.
[8] Ray Andraka, 'A survey of CORDIC algorithms for
FPGA based computers., 'ACM/SIGDA sixth
international symposium` ,1998, pp. 191-200
4