Journal of Ambient Intelligence and Humanized Computing (2023) 14:1259–1268

https://doi.org/10.1007/s12652-021-03377-5

ORIGINAL RESEARCH

An accurate car counting in aerial images based on convolutional

neural networks
Ersin Kilic1 · Serkan Ozturk1

Received: 19 October 2020 / Accepted: 5 July 2021 / Published online: 13 July 2021
© The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature 2021

Abstract
This paper proposes a simple and effective single-shot detector model to detect and count cars in aerial images. The proposed
model, called heatmap learner convolutional neural network (HLCNN), is used to predict the heatmap of target car instances.
In order to learn the heatmap of the target cars, we have improved CNN architecture by adding three convolutional layers as
adaptation layers instead of fully connected layers. The VGG-16 has been used as a backbone convolutional neural network
in the proposed model. The proposed method successfully determines the number of cars and precisely detects the center
of target cars. Experiments on the two different car datasets (PUCPR+ and CARPK) show the state-of-the-art counting and
localizing performance of the proposed method in comparison with existing methods. Also, experiments have been conducted
to examine the effect of data augmentation and batch normalization on the success of the proposed method. The code and
data will be made available here [https://​www.​github.​com/​ekilic/​Heatm​ap-​Learn​er-​CNN-​for-​Object-​Count​ing].

Keywords Deep learning · Car counting · Object counting · Convolutional neural networks

1 Introduction without precise object positions separates object counting

With the development of unmanned flying vehicles, new
potential applications have emerged for aerial view cameras
such as urban management, traffic monitoring (Wang et al.
2019), farm management, and parking lot utilization (Chen
et al. 2019). Generally, these applications need to find out
the number of objects such as cars on the road or on park-
ing lots (Mundhenk et al. 2016; Hsieh et al. 2017; Sun et al.
2017; Di Mauro et al. 2019; Chen et al. 2020), people in a
crowd (Nogueira et al. 2019; Revathi and Rajalaxmi 2019;
Razakarivony and Jurie 2016), sheep or cattle in a graz-
ing area (Sarwar et al. 2018; Shao et al. 2019), tobacco or
banana plant in a plantation area (Fan et al. 2018; Neupane
et al. 2019).
The object counting methods aim to estimate the number
of objects in a still image or video frame. Object detection
identifies visual objects of a specific class and determines
where the objects are in a digital image (Zou et al. 2019).
The determination of the number of objects in an image
* Ersin Kilic
ersinkilic@erciyes.edu.tr
1 et al. 2019a) have shown state-of-the-art performance in
Erciyes University, Kayseri, Turkey
methods from object detection and instance segmentation
(Kang et al. 2015). Moreover, the object counting problem
can be considered as a sub-problem of the object detection
problem. Thus, existing object detection and instance seg-
mentation methods can be used to estimate the number of
objects in the image. Counting by detection and segmenta-
tion is a usual consequence of the object detection and seg-
mentation methods that localize individual object instances
in the image.
Predominant approaches for object counting learn a
regression model that predicts a non-negative scalar count.
A single-output convolutional neural network (CNN) model
that predicts positive object count is frequently used for
object counting tasks. This CNN-based approaches are
called a single-shot regression model, which predicts a
scalar count for an input image without explicit localiza-
tion information of the objects. Unlike the regression-based
approaches, detection-based approaches localize the objects
to determine the number of objects. Anchor-based object
detection models such as Faster R-CNN (Ren et al. 2015),
SSD Liu et al. 2015), Yolo (Redmon et al. 2016), RetinaNet
(Lin et al. 2017) and anchor-free object detection models
such as CornerNet (Law and Deng 2018), CenterNet (Zhou

13
Vol.:(0123456789)
1260
many object detection tasks (Cazzato et al. 2020). Also,
these methods and variants have shown promising results in
car counting in aerial images (Hsieh et al. 2017; Goldman
et al. 2019; Li et al. 2019; Cai et al. 2019). In fact, count-
ing without localizing is not difficult for CNN models since
these models can classify and detect objects from dozens of
different classes. Actually, visualizations of feature maps
of regression-based models have demonstrated that feature
maps have information about car locations (Aich and Stav-
ness 2018b).
Aich and Stavness (2018b) used class activation map
(CAM) to regulate feature maps from the final convolu-
tional layer of CNN to enhance a pure single-shot regression
model for a car counting. CAM is a method that visualizes
the class-specific discriminative region parts detected by
a CNN on an input image. CAM has made it available to
see which regions in the image are relevant to a particular
class. Zhou et al. (2016) describe the procedure for generat-
ing CAM using global average pooling (GAP) in CNNs.
They proposed a simple modification of the global average
pooling layer to allow CNN to both classify the image and
localize class-specific image regions in a single forward-
pass. The motivation of this CAM lies in the idea that CNN
can produce a heatmap that shows the correct position of
an object. Oquab et al. (2015) propose a weakly supervised
object classification approach to demonstrate that CNN can
learn object localization without object-level annotation.
They have treated the fully connected layers as convolutions.
Chen et al. (2019) proposed a novel anchor-free CNN model
called target heatmap network, which consists of a shallow
features-extracting network, decoder network, and the loca-
tion method. In their approach, the target heatmap network
generates a 2-D heatmap of the objects on the remote sens-
ing image.
To learn generating the heatmap with a supervised
approach, ground-truth heatmaps should be created during
the training. Pfister et al. (2015) and Chu et al. (2017) used
the Gaussian function with fixed-sized and fixed-variance to
mark each joint of the human body in a heatmap. Chen et al.
(2019) used the normalized Gaussian function whose param-
eters vary with the object size to train a deep network that
detects oil tanks, vehicles, and aircrafts in remote sensing
images. The process of creating a heatmap using the Gauss-
ian function is called a Gaussian activation map (GAM).
By considering the localization ability of CNNs, as men-
tioned above, a novel single-shot detection model based
on CNN is proposed in this paper. The proposed model is
called heatmap learner CNN (HLCNN). HLCNN is trained
using GAMs for heatmap prediction of the target cars. The
proposed model is more successful than existing regression-
based methods for car counting. Also, the proposed method
localizes the objects by detecting the center point of counted
vehicles. Besides facilitating the explanation of the counting
13
E. Kilic, S. Ozturk
result, it reveals how the model works and where it makes a
mistake. HLCNN is an anchor-free CNN model which does
not represents objects with anchor boxes. Anchor boxes rep-
resentation has three main hyperparameters, namely: num-
ber, size, and aspect ratios of boxes (Law and Deng 2018).
The optimal adjustment of these parameters directly affects
the success of the model. HLCNN has fewer hyperparam-
eters, and thus, it is easy to use and robust. For evaluating
the success of our model, we used two car counting datasets:
CARPK and PUCPR+ (Aich and Stavness 2018b) .
The main contributions of this paper are summarized
as follows: (i) We propose a novel CNN model that pro-
duces a heatmap for car location in a single shot. (ii) The
effect of data augmentation and batch normalization (BN)
on the success of the HLCNN in predicting heatmap has
been demonstrated by detailed experiments. (iii) We improve
the accuracy of state-of-the-art car counting methods for
CARPK and PUCPR+ datasets. We improved the mean
absolute error (MAE) for car counting using the CARPK
and PUCPR+ datasets from 4.80 to 2.12 and 3.68 to 2.54,
respectively. Finally, our model has obtained impressive
results on detecting the center points of the car vehicles.
These results demonstrate that the proposed method has a
significant potential for object detection problems.
2 Literature review
A number of published studies in the literature that focus on
tackling counting tasks can be roughly categorized into two
categories: counting by detection and regression. Popular
object detection frameworks train CNN models to extract
features of candidate object proposals and then predict the
class probabilities and regress bounding boxes of those pro-
posals. These approaches are called two-stage object detec-
tion methods and the most well-known methods belong to
R-CNN family (Girshick 2015; Ren et al. 2015). Although
these two-stage methods are successful in detection, they
are slower than single-shot methods such as YOLO (Red-
mon et al. 2016) and SSD (Liu et al. 2015; Zou et al. 2019).
YOLO and SSD implement object classifiers and bounding
box regressors in an end-to-end manner without extracting
proposals. Saribas et al. (2018) and Ammar et al. (2019)
have used Faster R-CNN and YOLO for detecting cars in
aerial images. The direct application of most state-of-the-
art object detection frameworks for object counting requires
a particular focus on densely small-scale objects such as
cars. The layout proposal network presented in Hsieh et al.
(2017) proposes a novel bounding box generation method
with spatial kernels to simultaneously count and localize
target objects. Goldman et al. (2019) propose learning the
Jaccard index with a soft Intersection over Union (Soft-IoU)
network layer for car counting on densely packed scenes. A
An accurate car counting in aerial images based on convolutional neural networks 1261
novel scale-adaptive strategy for generating anchors and an
effective loss function to push the anchors towards match-
ing the ground-truth boxes are proposed by Li et al. (2019)
for detecting and counting dense vehicles in drone images.
The guided attention network presented in Cai et al. (2019)
enhanced feature representation with background attention
and foreground attention modules to facilitate accurate car
localization. Objects were represented by bounding boxes
(anchors) in these methods. They considered object detec-
tion as an image classification of an extensive number of
potential object bounding boxes (Wu et al. 2019). Anchor
based methods include the post-processing stage namely
non-maximum suppression (NMS), which serves to elimi-
nate duplicated bounding boxes (Kilic and Ozturk 2019).
The anchor has long been the predominant form of object
representation in object detection tasks. Recently, there have
been some attempts to develop different object represen-
tations for building more effective deep models. A rotated
bounding box to better handle rotational variations is pro-
posed in Zhou et al. (2017). ExtremeNet (Zhou et al. 2019b)
was proposed to locate the extreme points of objects in the x-
and y-directions. To overcome limitations of anchor repre-
sentation RepPoints is proposed to toward finer localization
and better object feature extraction by Yang et al. (2019).
Zhou et al. (2019a) proposed the CenterNet which uses key-
point estimation to find center points of objects. Arruda et al.
(2021) proposed the anchor-free CNN-based approach for
counting and locating objects in high-density imagery using
Gaussian kernels and evaluated the proposed method in two
counting datasets: car and tree.
Counting by regression methods aim to estimate the
number of objects without dealing with object localization.
Thus, instead of solving the challenging object detection
task, regression-based methods learn a direct mapping from
some global image characteristics to the non-negative num-
ber of objects. A number of approaches (Chan et al. (2008;
Chen et al. 2012, 2013) using global regressors have not
achieved the desired success because of trained with low-
level features. Several methods (Lempitsky and Zisserman
2010; Fiaschi et al. 2012; Arteta et al. 2014) try to predict
count by density map estimation. Lempitsky and Zisserman
(2010) proposed to generate a pixel-level ground-truth den-
sity map from the dot annotations using one gaussian kernel
per object. Based on this ground-truth density map, (Aich
and Stavness 2018b) enhance one-look regression counting
model by regulating activation maps of the CNN model. Xie
et al. (2016) propose fully convolutional regression networks
that predict the final density map to count microscopy cells.
In this paper, an efficient CNN-based single-shot car
detector model for aerial images is proposed. Unlike the
regression-based methods, the proposed method can detect
the center point of the target objects while counting cars
accurately. The proposed method has been tested with
CARPK, and PUCPR+ datasets, and state-of-the-art results
have been obtained by the proposed method.
3 Proposed approach
A car counting system based on HLCNN consists of two
stages: predicting heatmaps of an image by HLCNN and
generating peak-map from heatmap to locate and count car
in an image. Figure 1 gives an overview of the proposed
framework. To train the HLCNN model, GAM is used to
create heatmaps that show the size and location of cars.
GAM, HLCNN, and peak-map generation methods are
described in detail in the ensuing subsections.
3.1 Gaussian activation map (GAM)
The Gaussian function is used to generate heatmap to be
used during the training. The created heatmap is called
GAM. The Gaussian function (kernel) is given with the fol-
lowing form:
2 2)
G(x, y) = 𝛼 ∗ exp−(a(x−x0 ) +2b(x−x0 )(y−y0 )+c(y−y0 ) (1)
G(x, y) = G(x, y)∕max(G(x, y)) (2)
Where 𝛼 is a normalization factor, x0 , and y0 are the coordi-
nates of the kernel center. a, b and c are control coefficients
that control size and orientation of the kernel. In this study,
a, b and c are calculated using the equations as given below:
cos2 𝜃 sin2 𝜃
a =
2𝜎x2
+
2𝜎y2 (3)
sin 2𝜃 sin 2𝜃
b =− + (4)
4𝜎x2 4𝜎y2
sin2 𝜃 cos2 𝜃
c=
2𝜎x2
+
2𝜎y2 (5)
The 𝜃 value determines the orientation of the kernel,
whereas 𝜎x and 𝜎y are determined by the width and length
of the object, respectively. The maximum value calculated
by the Gaussian function depends on the size of the object.
Therefore, the maximum value of the Gaussian function is
set to 1 with the Eq. 1. GAM is a heatmap that consists
distribution of synthesized Gaussian function calculated for
each object. Figure 2 shows a sample image and heatmap
of objects created by GAM. Oriented bounding box (OBB)
annotation must be included in the data set to determine the
object orientation. If the data set has not included OBB, 𝜃
is set to zero.
13
1262 E. Kilic, S. Ozturk

Fig. 1  Overview of the proposed car counting framework. Detected center points of the target cars have been indicated by red point in output
samples

3.2 Heatmap learner convolutional neural networks In addition, the batch normalization (BN) (Ioffe and Sze-
(HLCNNs) gedy 2015) layer has been added to the backbone network
output.
The input image to HLCNN can be of any size, whereas {
the output size of HLCNN varies depending on the down- 𝛼x x ≤ 0
𝜙(x) =
x x>0 (6)
sampling ratio of the model and the number of classes. A
model with a downsampling ratio r generates an output
The parameters of the pooling layers and convolution layers
with dimensions w/r × h/r × C for an input with dimen-
contained in the model determine the downsampling ratio.
sions w × h × c. The variables w, h, c denote weight, height
The downsampling ratio for both VGG-16 and ResNet-18
and number of a color channel of input image, respec-
models is 32. The downsampling factor is crucial for the
tively. C denotes the number of the target object catego-
success of the HLCNN model and is appropriately deter-
ries. The model generates a heatmap for each object class
mined according to the size of the object to be counted.
separately.
This parameter significantly affects the success of a model.
The proposed model consists of two parts: backbone
The architecture of the HLCNN model with VGG-16 with
network and adaptation layers. Any standard CNN model
3 max-pooling layers as a backbone network is illustrated
without fully connected layers can be used as a backbone
in Fig. 3. For example, a 720 × 1280 pixel input would lead
network. Adaptation layers consists of convolution layers
to size of a 90 × 160 heatmap for all classes. While train-
with 1x1 size filters with a different number of feature
ing, we back-propagate L1 loss error (Eq. 7) of the predicted
maps. All convolutional layers are followed by the acti-
heatmap and GAM. In Eq. 7, w and h donate the width and
vation function of leaky relus (Eq. 6) (Xu et al. 2015) in
height of GAM, respectively.The Hpred and GAM donate the
adaptation layers. The 𝛼 value determines the coefficient of
predicted heatmap and the calculated heatmap of target cars,
leakage. The adaptation layers generates a number of heat-
respectively.
maps depending on the number of the object categories.

13
An accurate car counting in aerial images based on convolutional neural networks 1263

∑w ∑h
∣ Hpred ij − GAMij ∣
L1 =
i j
(7)
w∗h

3.3 Peak‑map generation

We propose a simple algorithm to detect the center points of

objects from the heatmap. The proposed method takes a pre-
dicted heatmap and detects the peaks using a local maximum
filter. The maximum filter detects the highest value of pixels
within a local region of an image. Algorithm 1 describes the
process of the proposed peak-map generation method.

Fig. 2  a Sample image from CARPK dataset b Heatmap of cars cre-
ated by GAM

Fig. 3  Architecture of the proposed HLCNN model

13
1264
4 Experimental study
We report experimental results of object counting with
MAE and root mean square error (RMSE) (Aich and Stav-
ness 2018b). The object localization results is reported using
peak-maps which show the center of object as a white point
on the black background. We implemented our model using
PyTorch (Paszke et al. 2019). For analyzing the effective-
ness of our model between different approaches in fairly, we
implemented our model with the VGG-16 (Simonyan and
Zisserman 2014) as a backbone network. In order to train the
models with CARPK and PUCPR+, we employed transfer
learning by using the pre-trained weights obtained for train-
ing on the ImageNet dataset (Deng et al. 2009). We run our
experiments in a Linux environment with a 16 GB memory
and an NVidia GTX1080-Ti GPU. The following numerical
metrics (Eq. 8) are used:
⎧ th
⎪ xi , yi = actual and target counts for i image
⎪ N = number ∑
of image
⎪ ∣xi −yi ∣
⎨ MAE = i (8)
⎪ �N ∑
⎪ (xi −yi )2
i
⎪ RMSE = works. The initial learning rate has been set to 0.0001 and
⎩ N
4.1 Datasets
Experimental studies have been conducted using CARPK
and PUCPR+ datasets. PUCPR+ dataset is a revised ver-
sion of the original PUCPR dataset. The PUCPR dataset
consists of images that were captured with a fixed camera.
This makes the images very similar to each other such that
PUCPR is not suitable for training deep learning models for
counting and localizing tasks. Hence, the PUCPR dataset
was reorganized and annotated by Hsieh et al. (2017) to cre-
ate the PUCPR+ dataset with a total of 17000 cars.
CARPK dataset contains 989 training and 459 testing
images and information of 89777 cars in various scenes for
4 different parking lots. The images were captured using a
drone, unlike the PUCPR+ dataset. The images were col-
lected with the drone-view at approximately 40 m height.
The CARPK dataset was introduced by Hsieh et al. in
(2017).
4.2 Results on CARPK
For evaluating the performance of the proposed method
in the CARPK data set, we have used three models with
a different downsampling ratios. The VGG-16 architec-
ture with different number of max-pooling layers has
13
E. Kilic, S. Ozturk
Table 1  Analysis of the effect Downsam- MAE RMSE
of downsampling ratio on pling ratio
performance for object counting
in the CARPK dataset 4 5.29 7.15
8 2.12 3.02
16 6.00 9.80
The lowest counting errors are
written even in bold
Table 2  Performance analysis of data augmentation methods and BN
layer for object counting on CARPK data set
Model MAE RMSE
HLCNN-WOBNAU 3.19 4.33
HLCNN-WOAU 2.73 3.93
HLCNN-WOBN 2.61 3.52
HLCNN 2.12 3.02
been used as a backbone network. Three models with 16,
8 and 4 downsampling ratios were trained to measure the
effect of the model downsampling ratio on counting suc-
cess. The ADAM optimizer has been used to train all net-
batch size has been set to 1. Networks have been trained
for 30 epochs with all training images. The models have
been trained with the 540 × 960 size of input image. The
results of the experiments are shown in Table 1. The model
trained with a downsampling ratio of 8 achieved the best
MAE and RMSE values of 2.12 and 3.02, respectively.
Thus, the model generates a 90 × 160 heatmap for 720 ×
1280 size input with a downsampling ratio of 8.
We employed data augmentation methods including:
image rotation and color jitter techniques. A color jitter
data augmentation randomly shifts the HUE channel of
an image with a probability of 0.5. An image rotation
augmentation randomly rotates images through random
degrees (between 0 to + 45) with a probability of 0.2. In
order to demonstrate the effects of data augmentation and
BN layer, the model with downsampling ratio of 8 has
been trained with or without data augmentation and BN
layer. Firstly, the model which is called HLCNN-WOB-
NAU was trained without data augmentation and BN layer.
HLCNN-WOBNAU achieved MAE and RMSE values of
3.19 and 4.33, respectively. Secondly, the model which
is called HLCNN-WOBN was trained without BN layer.
HLCNN-WOBN achieved MAE and RMSE values of 2.61
and 3.52, respectively. Thirdly, the model which is called
HLCNN-WOAU was trained without data augmentation.
HLCNN-WOAU achieved MAE and RMSE values of 2.73
and 3.93, respectively. The results show that the model
trained with data augmentation and BN layer achieved the
best scores in terms of MAE and RMSE (Table 2).
An accurate car counting in aerial images based on convolutional neural networks 1265
Fig. 4  Sample visual results of the HLCNN-WOAU. a and b Samples
images. Red points have indicated detected center points of the target
objects. c and d Heatmaps of samples images predicted by HLCNN-
WOAU (color figure online)
Fig. 5  Sample visual results of the HLCNN. a and b Samples images.
Red points have indicated detected center points of the target objects.
c and d Heatmaps of samples images predicted by HLCNN (color fig-
ure online)
Color-based data augmentation enhances the detection
performance of a method by reducing false detections on the
shadow of vehicles. The visual results of the model HLCNN
and HLCNN-WOAU have been presented in Figs. 4 and 5.
They have shown positive effect of color-based data aug-
mentation. Figure 5 has shown that the HLCNN has reduced
false detection caused by vehicle shadows. Also, BN layer
and data augmentation reduced false detection rate of the
proposed method. Comparing the visual results seen in
Figs. 6 and 7, this technique reduced the false detections
of other objects such as graffitis and park toys. Parking
some cars side by side, partly covered by trees or shadows,
Fig. 6  Sample visual results of the HLCNN-WOBNAU. a and b
Samples images. Red points have indicated detected center points of
the target objects. c and d Heatmaps of samples images predicted by
HLCNN-WOBNAU (color figure online)
Fig. 7  Sample visual results of the HLCNN. a and b Samples images.
Red points have indicated detected center points of the target objects.
c and d Heatmaps of samples images predicted by HLCNN (color fig-
ure online)
results in undetectable cars. Mainly partly seen cars cause
the model to be unable to detect cars.
The threshold (t) value of peak-map generation has been
specially set for each model to maximize counting success. t
value was chosen as 0.05 for HLCNN with a downsampling
ratio of 8. We compare our method with state-of-the-art
algorithms in Table 3. The results show that our approach
achieves the best MAE and RMSE. Figure 8 shows the
object localization performance of proposed method.
13
1266 E. Kilic, S. Ozturk

Fig. 8  Sample images from CARPK dataset (top) and predicted Fig. 9  Sample images from CARPK dataset (top) and predicted
heatmap of cars (bottom). Red points have indicated detected center heatmap of cars (bottom). Red points have indicated detected center
points of the target cars (color figure online) points of the target cars (color figure online)

Table 3  CARPK dataset results Table 5  PUCPR+ dataset results

Method MAE RMSE Method MAE RMSE

Faster R-CNN (Ren et al. 2015; Hsieh et al. 2017) 24.32 37.62 Faster R-CNN (Ren et al. 2015; Hsieh et al. (017) 39.88 47.67
YOLO (Redmon et al. 2016; Hsieh et al. 2017) 48.89 57.55 YOLO (Redmon et al. 2016; Hsieh et al. 2017) 156.00 200.42
One-Look Regression (Mundhenk et al. 2016; 59.46 66.84 One-Look Regression (Mundhenk et al. 2016; 21.88 36.73
Hsieh et al. 2017) Hsieh et al. 2017)
LPN (Hsieh et al. 2017) 23.80 36.79 LPN (Hsieh et al. 2017) 22.76 34.46
RetinaNet (Lin et al. 2018) 16.62 22.30 RetinaNet (Lin et al. 2018) 24.58 33.12
IEP Counting (Stahl et al. 2019) 51.83 IEP Counting (Stahl et al. 2019) 15.17
IoUNet (Goldman et al. 2019) 6.77 8.52 IoUNet (Goldman et al. 2019) 7.16 12.00
YOLOv3 (Redmon and Farhadi 2018) 7.92 11.08 VGG-GAP-HR (Aich and Stavness 2018b) 5.24 6.67
VGG-GAP-HR (Aich and Stavness 2018b) 5.88 9.30 YOLOv3 (Redmon and Farhadi 2018) 5.24 7.14
GSP-224 (Aich and Stavness 2018a) 5.46 8.09 SA+CF+CRT (Li et al. 2019) 3.92 5.06
SA+CF+CRT (Li et al. 2019) 5.42 7.38 GANet (VGG-16) (Cai et al. 2019) 3.68 5.47
GANet (VGG-16) (Cai et al. 2019) 4.80 6.9 dos Santos de Arruda et al. (2021) 3.16 4.39
dos Santos de Arruda et al. (2021) 4.45 6.18 HLCNN (VGG-16) 2.52 3.40
HLCNN (VGG-16) 2.12 3.02
The lowest counting errors are written even in bold
The lowest counting errors are written even in bold

Table 4  Analysis of the effect downsam- MAE RMSE
of downsampling ratio on pling ratio
performance for object counting
in the PUCPR+ dataset
4 2.52 3.40
8 2.64 4.05
16 29.92 41.62
The lowest counting errors are
written even in bold
4.3 Results on PUCPR+
In order to evaluate the performance of our method for the
PUCPR+ data set, we used three models with a different
downsampling ratios. The VGG-16 with different number of
max-pooling layers served as the backbone network. Similar
data augmentation methods were utilized in both models.
The model has been trained for 100 epochs with same opti-
mizer and learning rate as described for experiments with
the CARPK dataset. The threshold (t) value of peak-map
generating was set to 0.45. The results of the experiments for
measuring the effect of the downsampling ratio are shown in
Table 4. The model trained with a downsampling factor of 4
achieved the best RMSE value of 3.40.
The results of the experiments are compared with state-
of-the-art methods. It can be seen from Table 5 that the pro-
posed method achieves the best MAE and RMSE values of
2.52 and 3.40, respectively, among all the methods. Figure 9
shows the object localization performance of the proposed
method.

13
An accurate car counting in aerial images based on convolutional neural networks 1267
5 Conclusion
In this paper, a novel CNN-based model to deal with object
counting in remote sensing images has been proposed. The
proposed method predicts a heatmap in a single-shot to
demonstrate the location of cars. Existing regression-based
object counting approaches predict the number of objects
but cannot localize the target objects. In the experiments, the
proposed model has obtained impressive results on detect-
ing center points of the target objects. Also, experiments
have been conducted to show the effect of BN layer and data
augmentation. The performance comparison of the proposed
method on two challenging datasets has demonstrated the
effectiveness of our method. The HLCNN has improved the
state-of-the-art counting performance of existing methods.
The dependence of the HLCNN on the downsampling ratio
is the disadvantage of the method. To solve this problem, we
intend to improve the proposed HLCNN model with feature
pyramid networks as a future work. We also plan to test the
model with multi-class object detection and counting data
sets.
Acknowledgements This work is supported by Erciyes University,
the Department of Research Projects under Contract FDK-2018-8624.
References
Aich S, Stavness I (2018a) Object counting with small datasets of large
images. arXiv preprint arXiv:​1805.​11123
Aich S, Stavness I (2018b) Improving object counting with heatmap
regulation. arXiv preprint arXiv:​1803.​05494
Ammar A, Koubaa A, Ahmed M, Saad A (2019) Aerial images pro-
cessing for car detection using convolutional neural networks:
comparison between faster R-CNN and yolov3. arXiv preprint
arXiv:​1910.​07234
Arteta C, Lempitsky V, Alison NJ, Zisserman A (2014) Interactive
object counting. In: David F, Tomas P, Bernt S, Tinne T (eds)
Computer vision-ECCV. Springer International Publishing, Berlin
Cai Y, Du D, Zhang L, Wen L, Wang W, Wu Y, Lyu S (2019) Guided
attention network for object detection and counting on drones.
arXiv preprint arXiv:​1909.​11307
Cazzato D, Claudio C, Jose Luis S-L, Holger V, Marco L (2020) A
survey of computer vision methods for 2d object detection from
unmanned aerial vehicles. J Imaging 6(8):78
Chan AB, Liang Z-SJ, Vasconcelos N (2008) Privacy preserving crowd
monitoring: counting people without people models or tracking.
IEEE Conf Comput Vis Pattern Recognit. https://d​ oi.o​ rg/1​ 0.1​ 109/​
cvpr.​2008.​45875​69
Chen K, Gong S, Xiang T, Loy CC (2013) Cumulative attribute space
for age and crowd density estimation. In: 2013 IEEE conference
on computer vision and pattern recognition CVPR
Chen H, Libao Z, Jie M, Jue Z (2019) Target heat-map network: an
end-to-end deep network for target detection in remote sensing
images. Neurocomputing 331:375–387. https://​doi.​org/​10.​1016/j.​
neucom.​2018.​11.​044
Chen K, Loy CC, Gong S, Xiang T (2012) Feature mining for localised
crowd counting. In: British machine vision conference BMVC12
Chen W, Qiao Y, Li Y (2020) Inception-SSD: an improved single shot
detector for vehicle detection. J Ambient Intell Humaniz Comput.
https://​doi.​org/​10.​1007/​s12652-​020-​02085-w
Chu X, Yang W, Ouyang W, Ma C, Yuille AL, Wang X (2017) Multi-
context attention for human pose estimation. IEEE Conf Comput
Vis Pattern Recognit (CVPR). https://​doi.​org/​10.​1109/​cvpr.​2017.​
601
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) ImageNet:
a large-scale hierarchical image database. In: 2009 2013 IEEE
conference on computer vision and pattern recognition CVPR09
Di Mauro D, Furnari A, Patanè G, Battiato S, Farinella GM (2019)
Estimating the occupancy status of parking areas by counting cars
and non-empty stalls. J Vis Commun Image Represent 62:234–
244. https://​doi.​org/​10.​1016/j.​jvcir.​2019.​05.​015
dos Santos de Arruda M, Lucas PO, Plabiany RA, Diogo NG, José
MJ, Ana P, Marques R, Matsubara ET, Zhipeng L, Jonathan L,
Jonathan de Andrade S, Wesley NG (2021) Counting and locat-
ing high-density objects using convolutional neural network.
arXiv preprint arXiv:​2102.​04366
Fan Z, Jiewei L, Gong M, Xie H, Goodman ED (2018) Auto-
matic tobacco plant detection in UAV images via deep neu-
ral networks. IEEE J Sel Top Appl Earth Observ Remote Sens
11(3):876–887. https://​doi.​org/​10.​1109/​jstars.​2018.​27938​49
Fiaschi L, Nair R, Köthe U, Hamprecht FA (2012) Learning to count
with regression forest and structured labels. In: Proceedings
of the 21st international conference on pattern recognition
(ICPR2012), pp 2685–2688. ISBN 978-1-4673-2216-4
Girshick RB (2015) Fast R-CNN. arXiv preprint arXiv:​1504.​08083
Goldman E, Herzig R, Eisenschtat A, Ratzon O, Levi I, Goldberger
J, Hassner T (2019) Precise detection in densely packed scenes.
arXiv preprint arXiv:​1904.​00853
Hsieh M-R, Lin Y-L, Hsu WH (2017) Drone-based object counting
by spatially regularized regional proposal network. arXiv pre-
print arXiv:​1707.​05972
Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep
network training by reducing internal covariate shift. arXiv pre-
print arXiv:​1502.​03167
Kang D, Ma Z, Chen AB (2019) Beyond counting: Comparisons of
density maps for crowd analysis tasks-counting, detection, and
tracking. IEEE Trans Circuits Syst Video Technol 29(5):1408–
1422 (ISSN 1558-2205)
Kilic E, Ozturk S (2019) A subclass supported convolutional neural
network for object detection and localization in remote-sensing
images. Int J Remote Sens 40(11):4193–4212. https://​doi.​org/​
10.​1080/​01431​161.​2018.​15622​60
Law H, Deng J (2018) Cornernet: detecting objects as paired key-
points. In: Proceedings of the European conference on computer
vision (ECCV), 2018, pp. 734–750
Lempitsky V, Zisserman A (2010) Learning to count objects in images.
In: Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta
A (eds) Advances in neural information processing systems, vol
28. Curran Associates Inc., London, pp 1324–1332
Li W, Li H, Wu Q, Chen X, Ngan KN (2019) Simultaneously detect-
ing and counting dense vehicles from drone images. IEEE Trans
Ind Electron 66(12):9651–9662. https://​doi.​org/​10.​1109/​tie.​2019.​
28995​48
Lin T-Y, Goyal P, Girshick RB, He K, Dollár P (2017) Focal loss for
dense object detection. arXiv preprint arXiv:​1708.​02002
Lin T-Y, Goyal P, Girshick R, He K, Dollar P (2018) Focal loss for
dense object detection. IEEE Trans Pattern Anal Mach Intell.
https://​doi.​org/​10.​1109/​tpami.​2018.​28588​26
Liu W, Anguelov D, Erhan D, Szegedy C, Reed SE, Fu C-Y, Berg
AC (2016) SSD: single shot multibox detector. In: Computer
vision – ECCV 2016. ECCV 2016. Lecture notes in computer
science, vol 9905. Springer, Cham. https://​doi.​org/​10.​1007/​
978-3-​319-​46448-0_2
13
1268
Mundhenk NT, Konjevod G, Sakla WA, Boakye K (2016) A large
contextual dataset for classification, detection and counting of cars
with deep learning. In: Computer vision – ECCV 2016. ECCV
2016. Lecture notes in computer science, vol 9907. Springer,
Cham. https://​doi.​org/​10.​1007/​978-3-​319-​46487-9_​48
Neupane B, Horanont T, Hung ND (2019) Deep learning based banana
plant detection and counting using high-resolution red-green-blue
(RGB) images collected from unmanned aerial vehicle (UAV).
PLOS One 14(10):e0223906. https://​doi.​org/​10.​1371/​journ​al.​
pone.​02239​06
Nogueira V, Oliveira H, Augusto Silva J, Vieira T, Oliveira K (2019)
Retailnet: a deep learning approach for people counting and hot
spots detection in retail stores. In: 2019 32nd SIBGRAPI confer-
ence on graphics, patterns and images (SIBGRAPI). https://​doi.​
org/​10.​1109/​sibgr​api.​2019.​00029
Oquab M, Bottou L, Laptev I, Sivic J (2015) Is object localization for
free? Weakly-supervised learning with convolutional neural net-
works. In: 2015 IEEE conference on computer vision and pattern
recognition (CVPR), pp 685–694
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen
T, Lin Z, Gimelshein N, Antiga L, Desmaison A (2019) Pytorch:
an imperative style, high-performance deep learning library. Adv
Neural Inf Process Syst 32:8026–37
Pfister T, Charles J, Zisserman A (2015) Flowing ConvNets for human
pose estimation in videos. IEEE Int Conf Comput Vis (ICCV).
https://​doi.​org/​10.​1109/​iccv.​2015.​222
Razakarivony S, Jurie F (2016) Vehicle detection in aerial imagery: a
small target detection benchmark. J Vis Commun Image Represent
34:187–203. https://​doi.​org/​10.​1016/j.​jvcir.​2015.​11.​002
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look
once: unified, real-time object detection. IEEE Conf Comput Vis
Pattern Recogni (CVPR). https://​doi.​org/​10.​1109/​cvpr.​2016.​91
Redmon J, Farhadi A (2018) Yolov3: an incremental improvement.
arXiv preprint arXiv:​1804.​02767
Ren S, He K, Irshick R, Sun J (2015) Faster r-cnn: towards real-time
object detection with region proposal networks. In: Cortes C,
Lawrence ND, Lee DD, Sugiyama M, Garnett R (eds) Advances
in neural information processing systems, vol 28. Curran Associ-
ates Inc., London, pp 91–99
Revathi T, Rajalaxmi TM (2019) Deep learning for people count-
ing model. Adv Intell Syst Comput. https://​doi.​org/​10.​1007/​
978-​981-​15-​0035-​043
Saribas H, Hakan C, Sinem K (2018) Car detection in images taken
from unmanned aerial vehicles. Signal Process Commun Appl
Conf (SIU). https://​doi.​org/​10.​1109/​siu.​2018.​84042​01
Sarwar F, Griffin A, Periasamy P, Portas K, Law J (2018) Detecting
and counting sheep with a convolutional neural network. IEEE Int
Conf Adv Video Signal Based Surveill (AVSS). https://​doi.​org/​
10.​1109/​avss.​2018.​86393​06
13
E. Kilic, S. Ozturk
Shao W, Kawakami R, Yoshihashi R, You S, Kawase H, Naemura T
(2019) Cattle detection and counting in UAV images based on
convolutional neural networks. Int J Remote Sens 41(1):31–52.
https://​doi.​org/​10.​1080/​01431​161.​2019.​16248​58
Simonyan K, Zisserman A (2014) Very deep convolutional networks
for large-scale image recognition. arXiv preprint arXiv:1​ 409.1​ 556
Stahl T, Pintea SL, van Gemert JC (2019) Divide and count: generic
object counting by image divisions. IEEE Trans Image Process
28(2):1035–1044. https://​doi.​org/​10.​1109/​tip.​2018.​28753​53
Sun M, Yan W, Teng L, Jing L, Jun W (2017) Vehicle counting in
crowded scenes with multi-channel and multi-task convolutional
neural networks. J Vis Commun Image Represent 49:412–419.
https://​doi.​org/​10.​1016/j.​jvcir.​2017.​10.​002
Wang J, Liu C, Tian F, Zheng L (2019) Research on automatic target
detection and recognition based on deep learning. J Vis Commun
Image Represent 60:44–50. https://​doi.​org/​10.​1016/j.​jvcir.​2019.​
01.​017
Wu Y, Yinpeng C, Lu Y, Zicheng L, Lijuan W, Hongzhi L, Yun F
(2019) Rethinking classification and localization in R-CNN. arXiv
preprint arXiv:​1409.​1556
Xie W, Alison JN, Andrew Z (2016) Microscopy cell counting and
detection with fully convolutional regression networks. Comput
Methods Biomech Biomed Eng 6(3):283–292. https://​doi.​org/​10.​
1080/​21681​163.​2016.​11491​04
Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified
activations in convolutional network. arXiv preprint arXiv:​1505.​
00853
Yang Z, Liu S, Hu H, Wang L, Lin S (2019) Reppoints: point set rep-
resentation for object detection. arXiv preprint arXiv:​1904.​11490
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning
deep features for discriminative localization. IEEE Conf Comput
Vis Pattern Recognit (CVPR). https://​doi.​org/​10.​1109/​cvpr.​2016.​
319
Zhou Y, Qixiang Y, Qiang Q, Jianbin J (2017) Oriented response net-
works. arXiv preprint arXiv:​1701.​01833
Zhou X, Wang D, Krähenbühl P (2019a) Objects as points. arXiv pre-
print arXiv:​1904.​07850
Zhou X, Wang D, Krähenbühl P (2019b) Bottom-up object detection
by grouping extreme and center points. arXiv preprint arXiv:1​ 901.​
08043
Zou Z, Zhenwei S, Yuhong G, Jieping Y (2019) Object detection in 20
years: a survey. arXiv preprint arXiv:​1905.​05055
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.