Avian Detection & Tracking Algorithm Using
Infrared Imaging
Golrokh Mirzaei1, Mohammad Wadood Majid1,
Jeremy Ross2
Department of Electrical and Comp Sci.
University of Toledo1, Toledo, USA
mohsin.jamali@utoledo.edu
Abstract— This paper presents a method for target detection
and tracking of IR images in the application of avian
surveillance. As there are many reports of avian mortality due
to collision with turbine blades, the detection and tracking of
birds at turbine sites is an important issue.
In this work, three different background subtraction
techniques are first applied to detect moving objects. Otsu
thresholding method is then extended by incorporating an
adaptive variable based on the mean of each frame and certain
constant value. Filtering using morphological operations is
applied. Results of three different techniques are then
compared. Selected technique (RA) followed by thresholding
and filtering is then used for tracking and information
extraction. Results show that proposed method provides the
needed accuracy for IR imagery. This method can be effectively
used in different applications of IR imaging.
I. INTRODUCTION
Moving target detection and tracking of IR images is
attracting great interest in different applications such as
military and civil. In this work, IR images are used for
surveillance in the application of avian tracking in the
vicinity of wind turbines. Due to numerous reports of bird/bat
mortality with turbine blades [1][2], it is important to record
the behavior of birds/bats near off/on shore wind sites.
This research may contribute towards their preservation by
informing development of appropriate mitigation measures.
This method can also be used effectively in other applications
for detection and tracking of different targets.
One of the advantages of IR images over the visual
images is their independence from lighting conditions, as
well as providing temperature information which is hidden in
the visual image. Thermal images are acquired by sensing the
radiation in IR spectrum that is emitted by the object. On the
other hand the thermal images have a very low SNR, which
provides limited information for proper detection and
tracking.
Many methods have been developed for target detection
and tracking of visual images [3][4], but there exists a
limited amount of work on target detection and tracking on
IR imagery in the computer vision community [5][6].
Among those methods for IR target detection and tracking,
most of them are based on temperature changes with an
assumption that moving targets will appear as hot spots in
the infrared focal plane. This is not efficient in scenarios
This work is partially supported from DOE Contract #DE-FG36-06G086096
978-1-4673-0818-2/12/$31.00 ©2012 IEEE
Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on May 14,2022 at 05:26:53 UTC from IEEE Xplore. Restrictions apply.
Mohsin M. Jamali1, Peter V. Gorsevski2, Joseph P. Frizado2
Verner P. Bingman2
Geospatial Science School of Earth & Environment
Bowling Green State University2
Bowling Green, USA
where the temperature of background objects is higher than
the moving targets, which usually happens in summer time.
This paper presents an approach for avian detection and
tracking in IR imagery based on background modeling for
segmentation and component connectivity for tracking. In
this work, three different background subtraction techniques
are first applied to detect the moving objects (birds/bats)
followed by proposed adaptive thresholding using the Otsu
method and noise suppression using morphology. The results
of three different techniques are compared and the best
technique in terms of accuracy is selected for tracking. In the
next step, the tracking and target information extraction
(target size, velocity, distance, direction and heat) techniques
are developed. The block diagram of this approach is shown
in Fig.1.
This technique of detection and tracking has several
advantages; it does not depend on temperature variation and
it can be applied in all conditions such as high clutter, target
variability and hotter background objects. Second, there is no
requirement for assuming a constant velocity. Finally, it is
free from complex and sophisticated methods, which needs
more time to process. It is especially effective in processing
lengthy video when time is an important consideration.
The rest of this paper is structured as follows: First data
acquisition is explained in Section II. Then background
Image Sequence
Segmentation Background
Subtraction
Running Average Running Gaussian Mixture Of
Detection
Average Gaussian
Bird/Bat detection
E-Otsu Adaptive
Threshold Thresholding
Tracking
Morphology Noise Suppression
Labeling-Based
Tracking
Connectivity
Information
Extraction
Fig. 1: Block Diagram of the Avian Detection and Tracking system
subtraction followed by thresholding and noise suppression
is described in Section III and IV, respectively. Then
Tracking and target information extraction are presented in
the Section V and VI. Finally results and conclusions are
given.
II. DATA ACQUISITION
The camera used in the experiment is a far bandwidth
thermal IR camera (FLIR SR-19) with resolution of 480 x
704 pixels. The camera is static and it is oriented +19.75°
from North, while the left side of the screen is the East and
the right side is the West. The rate of the recording is 30
frames/sec (fps). The video feeds from IR camera are stored
onto a hard drive for further data processing. IR camera was
deployed at Ottawa National Wildlife Refuge in Ohio during
Spring bird migration. The data is collected between May -
July 2011, starting at one hour after the sunset to one hour
before sunrise; as bat/bird migration is mainly occur during
this time. Because of hardware limitation, recorded videos
are divided into small videos of fifteen minutes; each fifteen
minutes video is then divided into frames.
III. BACKGROUND SUBTRACTION
The Background Subtraction (BS) modeling is applied on
the acquired frames. The main goal of BS is to segment the
frame in to background as the reference model and the
foreground as moving objects. One of the advantages of IR
images is that they are not based on illumination changes as
in visual images, but based on temperature alteration. This
main characteristic make the segmentation part less critical in
IR images. So it is not necessary to use complex
segmentation techniques as they can be very time consuming.
But a good segmentation can be used as decent pre-tracker.
On the other hand, the techniques based on the
temperature changes doesn’t work for all conditions, since in
some scenarios the temperature of the objects in background
may be higher than moving objects which usually happens in
summer time. So in order to have an efficient detection and
tracking technique, as an alternative of temperature- based
methods, BS techniques are used, which works in all
conditions.
Three different methods of Background Subtraction (BS)
are applied to segment the moving objects from background
model in the collected IR images. These techniques are
Running Average (RA) [7], Running Gaussian Average
(RGA)[7] and Mixture of Gaussian (MOG)[7][8].
Histograms of resulted background using RA, RGA and
MOG are shown in Fig. 2, 3, 4 respectively.
IV. THRESHOLDING AND NOISE SUPPRESSION
The idea behind a thresholding process is to determine a
threshold value to separate a gray-level image into objects
and background. Determining an optimum threshold value is
a challenging task. In some of the applications a static
Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on May 14,2022 at 05:26:53 UTC from IEEE Xplore. Restrictions apply.
thresholding method may be used which is not accurate in all
cases. In avian detection scenario with IR imaging, the size
of the birds varies with distance from the camera. Those birds
flying farther than camera, have smaller size than those flying
closer to the camera. So a static threshold value will not be
appropriate in this application as it may miss some of the
targets in farther distances. There are many adaptive
thresholding methods proposed [9] in the literature and they
are computationally extensive. Thresholding in IR images
requires tradeoff between accuracy and computational
efficiency.
In this work, the Otsu threshold is extended as an
adaptive thresholding method, which we call it EOtsu. In
Otsu method [10], the measure of separability of image
intensities is classified into two classes of foreground and
background. Measure of separation is defined as the ratio
between–class variance (σB and the total image intensity
variance σG . It is expressed as:
1
The Eqn. (1) can be normalized as
0 1 2
Where t is the Otsu threshold.
Otsu thresholding method is extended by incorporating an
adaptive variable based on the mean of each frame and
certain constant value. The extended threshold can be
achieved by
3
(a) (b)
Fig. 2: (a) RA background un-equalized histogram (b) equalized
histogram of same image
(a) (b)
Fig. 3: (a) RGA background un-equalized histogram (b) equalized
histogram of same image
(a)
(b)
Fig. 4: (a) MOG background un-equalized histogram (b) equalized
histogram of same image
 Where is proposed optimum threshold and
adaptive variable as
Where C is a constant number and I is the new frame
with pixels i, j. Fig. 5 shows that EOtsu threshold was able to
suppress noise and gave a clearer image than Otsu method.
Then filtering is used to eliminate the noise and its effects
on the frame. To accomplish this objective, a morphological
filter consisting of opening followed by closure [10], which
we name it open-closure, is applied as
Where A and B are sets in Z , A is binary image and B is
element structure. , , , are erosion, dilation, opening and
closure operators respectively.
The open-closure morphological operation is used as the
erosion and can make the separation, but the retrieved objects
are smaller than the original due to the size of structuring
element. On the other hand dilation yields the full
connection of elements, but it increases the size of the
objects. So opening is the only operation able to discriminate
the objects and conserve their original size.
Finally by applying the closure the full connection of the
elements is yielded while eliminating gaps between objects.
The result of the filtering and noise suppression on the
thresholded image is shown in Fig. 6.
V. TRACKING
Tracking is applied for extracting information such as
velocity, distance, heat, direction and size of birds/bats.
Tracking is used to identify which segmented connected
component corresponds to each object being tracked. To find
the track of the avian targets, labeling-based component
connectivity is applied. In this method, the detected
birds/bats, say blobs, obtained in the previous sections are
used. Each blob in new frame is dilated by a fix size structure
element. We call this dilated blob as D-Blob. If the D-Blobs
have common area with previous blobs, we assume that this
is the same object but in different time, so we give them same
label, otherwise a new label is assigned to blobs, assuming
that it is a new target. Even if the blobs are missing in
between the frames, the labels are able to recognize if the
blob is the same blob or new one. This process is shown in
Fig. 7. After detecting objects with same labels in sequential
frames, morphology closure operator is used to get the
trajectory of the object. For this reason the number of frames
(a) (b) (c)
Fig. 5: (a) Original Image (b) After Otsu thresholding
(c) After E-Otsu thresholding
Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on May 14,2022 at 05:26:53 UTC from IEEE Xplore. Restrictions apply.
is an
4 RA
Filtering (Open-Closure)
Thresholding (EOtsu)
Background Subtraction
RGA
MOG
5
Fig. 6: Result of BS techniques after applied EOtsu and
Morphology
including the target is detected and are added together to
generate the track of the target as shown in Fig. 8.
VI. EXTRACT TARGET INFORMATION
The information of the bird/bats is extracted based on the
detected objects for further processing. The information
include size, velocity, direction, heat emitted by target and
distance. The size and heat is calculated based on the average
size and heat of target in frames including targets. The
distance is based on the number of pixels in the camera’s
field of view and direction is calculated based on the standard
orientation. The camera is static and is oriented +19.75° from
North while the left side of the screen is the East and the right
side is the West, as shown in Fig. 9. So the standard
coordinate system is rotated 19.75° to the right. Part of the
extracted information over the fifteen minutes video is shown
in Table 1.
VII. RESULTS
According to results, the EOtsu gives more accurate result
comparing with Otsu in IR image as shown in Fig. 5.
Moreover, RA followed by proposed thresholding EOtsu
method shows higher accuracy comparing with RGA and
MOG, as it can be seen from Fig. 6. RA needs less memory
and the speed of computation is higher due to simplicity. So
it was effective to detect the avian activity in a lengthy IR
video. Results of detection and tracking are shown in Table
2. Twenty eight birds/bats were observed visually in the
fifteen minute IR video. It can be observed that 26 birds/bats
Tracking Algorithm
(1) Frames (1,2,……n)
(2) Blob Detection
(3) Labeling
(4) Create D-Blobs
(5) If component connectivity
Then same labeling
Else new labeling
(6) Plot target trajectory
Fig. 7: Tracking Algorithm
 Fig. 8: Tracking result
TABLE 2: RESULT OF DETECTION AND TRACKING ALGORITHM
Total Detected Missed Correction Miss
Targets Targets Targets Detection Detection
28 26 2 92% 7%

and the speed of computation is higher due to its simplicity.

These computational parameters are important factor as
monitoring of birds/bats is required for long migration period
of one to two months with 7 hours/day of recording. Then
tracking algorithm based on labeling /connectivity is applied
on the detected object. The result of detection and tracking
algorithm shows 92% accuracy, missing only 2 out of 28
Fig. 9: Direction of the FLIR camera birds. The misses may be due to their smaller size, farther
distance and/or the high cluttered image.
out of 28 were determined by this method. The reason of This research will help in processing entire IR video data
missing two targets may be that some bird/bat fly very far in for an entire migration period. It will help biologists to
terms of distance to the camera, so their projection in the IR understand bird/bat behavior in the vicinity of wind turbines.
images will be very small in size and difficult to detect. Preservation and appropriate mitigation measures can then be
Portion of the extracted information of the birds/bats in the employed. This method can also be used effectively to other
15 minutes video is shown Table 1. This information can be applications of target detection and tracking.
used for further processing for classification of avian to birds,
bats and (insects). REFERENCES
[1] G. D. . Johnson, M. K. Perlik, W. P. erikson and M. D. Strickland,
VIII. CONCLUSION “Bat Activity, Composition and Collision Mortality at a large wind
plant in Minnesota”, Wildlife Society Bulletin, pp. 1278-1288, 2004
The goal of this work is to use IR imagery to detect and [2] E. B. Arnett, W. K. Brown, and W. P. Erikson, “Patterns of Bat
track bird/bat activity in the vicinity of wind turbines. A Fatalities at Wind Energy Facilities in North America”, Journal of
fifteen minute FLIR camera video collected at Ottawa Wildlife Management, pp. 61-78, 2008
National Wildlife Refuge in Ohio was used to test the [3] J. D. Valle, L.E. S. oliveira, A.L. Koerich and A. A. Britto, “People
detection and tracking system. The entire video data set is Countimg in Low Density Video Sequences”, Springer, pp. 737- 748,
2007
very large and consists of seven hours/day for two months.
[4] T. Darrell, G. Gordon, M. Harville and J. woodfill, “ Integrated Person
Therefore, only a small data set is used for development of Tracking using, Stereo, Colour and Pattern Detection”, International
detection and tracking algorithm. Journal of Computer Vision, pp.175-185, 2007
First three background subtraction techniques, RA, RGA [5] F. Zhang, C. Li, L. Shi, “Detecting and Tracking Dim Moving Point
and MOG, are applied followed by proposed extended Otsu Target in IR Image Sequences”, ELSEVIER, pp. 323-328, 2005
thresholding (EOtsu) based on mean of each frame and noise [6] J. Wu, S. Mao, X. Wang and T. Zhang, “Ship Target Detection and
Tracking in Cluttered Infrared Imagary”, Optical Engineering, pp.
suppression using Opening-Closure morphological 057207-1, 057207-12, 2011
techniques. EOtsu provided a clearer image in comparison to [7] M. Piccardi, “Background Subtraction Techniques:a review”, IEEE
Otsu. Results show that the filtered RA gives a better result International Conference on systems, Man and Cybernetics, pp. 3099-
as compared to RGA and MOG. In addition to the accuracy 3104, 2004
in the application of IR imagery, RA required less memory [8] C. Stauffer and W. E. L. Grimson, “Adaptive Background Mixture
Model for real-time Tracking”, Proc. IEEE Coputer Society
TABLE 1: AVIAN INFORMATION EXTRACTED Conference on Computer Vision and Pattern Recognition, pp. 246-
Blob Size Velocity Distance Heat 252-1999
Direction
# (Pixel) (pixel/s) (pixel)
[9] M. Sezgin, B. Saunkur, “Survey over image thresholding techniques
1 NE 394 745 442 8.09
and quantative performance evaluation”, Journal of Electronic
2 SW 142 111 553 5.53 Imaging, pp. 146-165, 2004
3 SW 196 1078 569 6.79
4 NW 107 315 301 6.66 [10] R. C. Gonzalez, R. E. Woods, “Digital Image Processing”, Pearson ,
2009
5 SW 308 1388 504 7.75
6 N 1021 751 347 7.45
7 NW 157 844 584 7.58

Authorized licensed use limited to: Shanghai Jiaotong University. Downloaded on May 14,2022 at 05:26:53 UTC from IEEE Xplore. Restrictions apply.