HUMAN DETECTION AND COUNTING

Gopal Krishna Pasumarty

Akarsh Reddy Avula Dileep Marupudi
Dept. of information technology
Dept. of informstion technology Dept. of information technology
GRIET, (JNTUH)
GRIET, (JNTUH) GRIET, (JNTUH)
Hyderabad, INDIA
Hyderabad, INDIA Hyderabad, INDIA
email: dean@griet.ac.in
email: aareddy.2002@gmail.com email: dileepmarupudi28@gmail.com
Akash Mysa
Lalith Sagar Pudi
Dept. of information technology
Dept. of information technology
GRIET, (JNTUH)
GRIET, (JNTUH)
Hyderabad, INDIA
Hyderabad, INDIA
email: mysaakash125@gmail.com
email: lalithsagar4515@gmail.com

Abstract—It is more important to detect human beings
accurately for diverse application areas which include human
characterization, congestion analysis, pedestrian detection, and
person identification tracking. The project’s major motivation
and scope are to detect and track down the moving people
among the cluster of moving objects and count the number of
people detected in the scenario. Techniques that are used to
perform this include optical flow, background modeling,
subtraction, and filtering techniques. Once a moving object is
detected, the classification can be done based on the
orientation, texture, shape, and motion of the object or person.
For the classification, Haar feature classifiers are used and
features that include the oriented gradients are also used.
Thus, the humans are detected and the count of humans is also
recorded by using the created model.
Keywords—Haar classification, Histograms of oriented
gradient features.
I. INTRODUCTION
Recognizing humans in a video scenario of
reconnaissance framework which is pulling in more
attraction, and consideration and got an eye catchy attention
in recent years due to its numerous runs of applications. The
discernment of humans in video surveillance and an image is
becoming crucial for abnormal event detection, the person
checking and identifying the target person in a thick swarm,
pedestrian discovery and classification by detecting, human
gait analysis, identification of persons, and then tracking
down, gender or sex classification, and old-age people drop
or fall detection.
Computer Vision is the field of study that involves the
methods which are required for analyzing, acquiring,
processing, and understanding real-world scenario images
and generally higher dimensional data in the order in
obtaining numerical or symbolical information that is which
are in the form of decisions.
The disentangling of symbolical information from image
data utilizing models built with the aid of geometry, physics,
statistics, and learning theory can be considered as the
classification and comprehension of images. However, by
employing the latest computer vision research approaches,
accuracy can be enhanced, as well as customization,
scalability, and ease of integration.
II. LITERATURE SURVEY
Allan D. Jepson et al., [1] in this paper, proposed a
framework for developing reliable, flexible appearance
models that will be applied to motion-based monitoring of
intricate natural objects. An implementation of this strategy
is created for an appearance model based on the steerable
pyramid's filter responses. This model is applied to a motion-
based tracking algorithm to give resilience against image
outliers, such as those brought on by occlusions, while
adjusting to real-world changes in appearance, such as those
brought on by changes in 3D posture or facial expressions.
[2] Amit Adam et al. presented a novel algorithm used
for tracking down an object in a visual reconnaissance which
is named "Frag Track". The approach makes use of the
integral histogram data structure as a crucial tool for its
implementation. Their technique resolves a number of issues
that conventional histogram-based algorithms are unable to
manage such as (i) Handling a change in the pose or partial
occlusions. (ii) The spatial distribution of the pixel intensities
can be taken into account this information is lost in
traditional algorithms. (iii) Computational cost is made
efficient for both the large and small targets.
A system was developed for robust people counting
which consists of four components an autocalibration
module, a standardized low-level foreground estimation
algorithm, crowd segmentation, and a template-based tracker
by X. Liu et al., [3] which results in an estimate that the
usage of site geometry made a significant benefit in
constraining people detection problem and to get relevant
scene information. They made a system that has the ability to
segment out individuals from larger groups of people and
monitor them over time. By effectively utilizing spatial
context, this model-based methodology also enables the
system to recognize specific occurrences or events
automatically.
[4] Boris Babenko et al., in this study, introduce the
MILTrack tracking system, which employs a cutting-edge
online multiple-instance learning algorithm. Even though it
is uncertain which image patch most accurately portrays the
object of interest, the MIL framework enables us to update
the appearance model with a collection of image patches.
They stated that their algorithm can be implemented easily
and can function at real-time speeds.
[5] David S. Bolme et al. introduce the average of
synthetic exact filters (ASEF), a technique for building
correlation filters for object detection. As demonstrated by
eye identification experiments on the FERET database as a
task, ASEF filters outperform two other generally proposed
eye detection algorithms as well as previously proposed
synthetic correlation filters.
They mentioned that ASEF filters differentiate from
previous filters in 2 ways, (i)ASEF generates an exact filter
for each training image that recreates the entire desired
correlation surface. Since the background can have
interference patterns, this improves the resolution of the
filter. (ii) The exact filters for all of the training photos are
averaged to get the final filter. By highlighting traits that are
similar to multiple photos, this averaging prevent over-
fitting. They proved that this filter has also performed well at
face detection and in images using iris sensors locating
pupils.
[6] The work was extended to present an incredibly
straightforward human recognition technique based on
filtering and edge-magnitude image correlation. The ASEF-
based detector obtains a 94.5% detection rate with fewer than
one false detection per frame for sparse crowds and can
analyze images at a pace of over 25 frames per second. The
training is quicker consuming 12 seconds on 32 manually
annotated images to train the detector. Here the dataset used
for evaluation is PETS 2009 whose results are in comparison
to the cascade classifier of OpenCV and a person detector
based on the state-of-the-art deformable parts.
[7] In a low-resolution image featuring complex
scenarios, this study proposes to develop an effective
approach for estimating the number of individuals and
locating each individual. It's a threefold contribution by Ya-
Li Hou et al., involving (i) In a complex scenario containing
only a small number of moving people estimation is done by
performing postprocessing steps on results obtained through
background subtraction. (ii) For locating individuals in a
low-resolution scene an EM (Expectation Maximization)
method was developed which makes a representation of each
person in a scenario with a new cluster model. They have
also stated that a very accurate foreground contour is not
required for their method. (iii) A priori was used as people
count for locating individuals making features as a basis.
They performed these developed methods on a 4-hour long
scale video for validation where people ranging in the video
lie between 36 to 222 obtaining the best result of estimating
human count has a 10% average error in comparison to 51
test cases. Here they have used a neural network for the
estimation of people. In a neural network when both pixels
of foreground and closed foreground are learned the best
estimation results are obtained.
The necessity for a precise foreground contour has been
alleviated by clustering the Kanade-Lucas-Tomasi (KLT)
feature points in a foreground mask. They have shown that
their model is more accurate than the Gaussian Model.
[8] A novel and robust model named "The Omega
Model" was developed for people identification and counting
that may be present at the incident by Subra Mukherjee et al.
The most distinguishing aspect of their work is the use of
four descriptors to identify four significant and invariant
characteristics of the human head-shoulder area. In detail, the
descriptor parameters are studied and a conclusion was made
by them that no individual descriptor can detect humans thus
they employed a weight-based decision system for better
performance. Their approach's effectiveness was validated
with the experiments performed on several images.
[9] A system was built using real-life still images which
used the Viola-Jones detection algorithm as its major part.
This is implemented using MATLAB release 2012 P.O.
Otasowie et al. This was performed only on a set of 35
different images instead of video to reduce processing time
under a few conditions like Image Size, Profilers test, and
algorithmic test. This system has an accuracy of 84% under
poor visual conditions and maintained a mark above 90%
under optimal situations.
[10] In this study, Xinjian Zhang et al. proposed a fast
and novel approach for crowded scenarios of surveillance to
estimate people's count. They have proved that their
approach with changes in the background and illumination is
robust and is able to count in real-time. In the process of
training a muti-scaled head-shoulder detector, the cascade of
the boosted classifier and combined rectangular features are
used. For identifying humans accurately in every frame
detector does its best. In order to track down and remove
duplicates in further frames human tracking is used. The
dataset they have used for training is a sample of 985
examples that are positive which are derived through 423
images and 336 are well-known INRIA set images. They
have tested it on subway station rea video and achieved
better results when the frame rate is 10fps and the camera
orientation of 30 degrees. They have followed the PASCAL
VOC protocol for evaluation on cropped images. It is
considered correct(true positive) if the detected bounding
box overlaps more than 50% with a ground truth box, else it
is a false positive. They have proved that their system
performance in real-time is satisfactory.
[11] This paper deals with the need for crowd
management, targeting the covid-19 situation. Advanced
deep learning computer vision algorithms are specifically
chosen by YOLOv3 for human object detection and
classification, DeepSORT tracking algorithm to track each
human object observed, and intrusion line judgement for
counting. This in addition converts the pre-trained YOLOv3
into TensorFlow for better and quicker computation in real-
time using GPU (Graphical Processing Unit) instead of CPU
(Central Processing Unit). With testing films taken from the
internet to replicate the entrance to a mall, the experimental
findings have demonstrated that this implementation
combination is 91.07% accurate and real-time capable.
III. TECHNIQUES USED
Background Segmentation
Segmenting background or subtracting noise in the
background of an image is a routine method in detecting an
object as the forth ground, by segmenting the object taken
input through the surveillance camera scene. The camera
can be stationarily attached to a location, purely in a
translatory state, or remotely operated. Background
Segmentation attempts to detect objects that move pixel-by-
pixel or block-by-block from the change observed between
the frame currently at and the frame that is taken as
reference. Reference frames are in common addressed as
"background images," "background models", either as
"environmental models." A modelled background which is
good must be able to get adapted with the dynamic scene
changes. You can do this by updating the background
information on a regular basis, but you can also do it
without updating the background information.
Motion Estimation
Motion estimation is also referred as “Optical Flow”. This is
a vector-based method for estimating video motion by
matching object locations across the frames of the images.
Optical flow describes the smooth and steady mobility of
points or 22 features between frames. To recognize moving
areas in an image stream, the motion segmentation that are
based on the optical flow employs the features of the flow
vector of moving objects over time-to time analysis. The
main advantage of using motion estimation often referred as
optical flow is that the nature of it having robust against
many multiplexed or multiplicated simultaneous
surveillance cameras and objects that are in motion, the
uniqueness of making it for mass spectrometry and high-
density motion conditions. You can use the optical flow-
based method to detect objects that move independently,
even in the case of the movement of camera like a camera
attached to a drone or a car. Apart from the noises of the
images, color, and sensitivity to randomly arranged lighting
that is which are non-uniformly distributed,in the number
most of the flow calculation related methods have many
significant requirements that are computational and are
sensitive to motion interruptions. The implementations
which are real-time scenario of optical flow often require
dedicated hardware components due to the nature of
complexity of the algorithms also the need for reasonably
high frame rates for accurate measurements.
Spatial-Temporal Filter
For motion detection based on spatiotemporal analytical
approach, the action of event or motion is characterized or
classified across the 3D spatiotemporal data 23 volume
across the people or persons who are in movement in the
sequence of images. These strategies or methodologies
generally take the motion as a lump of whole into account
and then characterizes its spatiotemporal distribution. We
have processed the sequence of images or video sequence
using one of the spatial features, the spatial Gauss and a
Gaussian derivative along the axis of time on the plane or a
graph to be plotted. Due to the differential operations along
the axis of time, the filter then shows a high response in the
operating range. We then used these responses in the
generation of the thresholds, get a motion related mask that
is binary in nature, and then aggregate them into a spatial
histogram bin. Such features compactly encode the
movement or motion and information that is spatial
corresponding to them, which is the most useful work for
distant and also the midfield videos of the surveillance
cameras. The approaches are mostly based on a simple
operation of convolution and that are quick and also easier
to implement. These are amongst the few very useful
approaches in low resolution or stubby-quality video
scenarios where it might be very difficult for the extraction
of other related features such as motion estimation often
referred to optical flow and silhouette. Spatiotemporal
movement-based methodologies can better be capable of
capturing both the spatial and temporal related information
of gait. The advantages are low complexity and easy
implementation. However, it is more susceptible to noise
and travel time fluctuations.
Shape-based Model
Shape-based methods start by describing the shape of the
regions that are in movement like collection of points,
boxes, and a few blobs. Further, it is then often regarded as
a standardized-pattern recognition problem. Wang et al
looked at the way that how formations and deformations of
the 24 silhouettes of humans (or forms) during the motion
that is articulated in nature, which could be employed as
features discriminating the nature to implicitly capture the
dynamics of the motion, and they used the transform
wavelet that is discrete in nature that is discrete-wavelet
transform and DFT to do so.
Motion-based Model
The premise behind this classification method is that item
motion features and patterns are distinct enough to segregate
them between objects and can be distinguished. To
distinguish humans from other moving things, techniques
based on the movement will directly utilize the periodic
characteristic of those images that are collected. By building
a vector picture template consisting of two of the temporal
25 projection operators: motion-energy image that is binary
in nature and motion history image, Bobick and Daviss
established the view-based technique for the recognition of
human movements.
A method based on texture
The most popular texture-based method used for quantifying
the patterns of intensity in the pixel’s immediate vicinity or
area of a pixel is LBP which is the acronym for Local
Binary Pattern. The local binary pattern with a concept of
multiblocks (MB-LBP) was introduced by Zhang et al for
encoding the intensities of rectangular sections using LBP.
The feature descriptor HOG developed a texture based
technique that employs higher-numbered dimensional
features that are based on edges, followed by SVM, to detect
the regions of the human body. This technique counts the
occurrences of gradient orientation in localized sections of a
picture using overlapping local contrast normalization in the
strategy of improving accuracy.
IV. METHODOLOGY
The system that is proposed begins with the modules
required for detecting and analyzing the input provided,
where the modules that are involved has the pre-trained
models inside them. The libraries that are needed to build
the project are OpenCV, Imutils, Numpy, Argparse. A
descriptor is used here in order to detect the objects, named
“HOG (Histogram of Oriented Gradients) Descriptor”.
Histogram of Oriented Gradient Descriptor
In the process of detecting objects, it is used a descriptor
that represents features of the input in computer vision and
image processing. The implementation to combine HOG
Descriptor algorithm with SVM (Support Vector Machine)
is preprocessed by OpenCV in a most efficient manner.
Haar Cascade Classifier
For better working of the system in the real-time
environment and for faster working, Haar Cascades are used
as classifiers that are pre-trained and contained in repository
of the OpenCV library. These are popularly used for Human
Face detection and eye detection. We use it in our model for
better results and quicker working. This detect the human
faces and track them down and remove the duplicates in
corresponding frames. The implementation of this Haar
cascade classifier involve downloading the pre-trained XML
file in the repository of OpenCV and load using
cv2.CasscadeClassifier function.
For the human detection the full body cascade is used and in
order to track down a person in corresponding frames the
face detect cascade is used.
Workflow and Methods
A default people detector function is used to call the pre-
trained model for Human Detection of OpenCV and then we
will feed SVM with it.
A moving picture is created by combining a series of photos
into a video. We refer to these pictures as frames.
1. Libraries Importing
2. In order to create a Model for detecting humans,
for this we use HOGDescriptor with SVM already
implemented in OpenCV with haar cascade.
3. Detect method, which takes a frame to detect
person in it. Makes a box around person and show
the frame and return it. Detect Multi Scale method
is used to detect objects of different sizes in an
image.
4. Human Detector method takes the image and video
as arguments they are captured using image
utilities with OpenCV.
5. ArgParse is used to for parsing and returns as a
dictionary argument through prompt to script.
V. RESULTS AND ANALYSIS
We apply different methods to make this detection model
work. The input is provided by the user in the form of
arguments to the model if the arguments are images and
videos the model detects and gives the count of the people
in the scene, if no arguments are passed it checks for an in-
built webcam or a camera installed on the computer or takes
the video from the camera. The output of the model is seen
on the separate windows for images, videos and the input
accessed from the camera.
VI. CONCLUSION
In this project, we are detecting humans and counting the
number of persons in the image, video, or those who are on
the camera. It is one of the most successful topics that are
majorly related to vision research because of its huge run of
numerous applications. The way it is performed is in two
steps: object detection and classification. Object
classification is done with various techniques such as
techniques that are based on motion, shape, and texture.
With the help of algorithms, major applications of human
detection and counting in a picture or video or live camera
are reviewed.
VII. FUTURE ENHANCEMENTS
It is a significant step forward in the development of
artificial intelligence systems that can correctly classify and
recognize items, which is useful in robotics and cybernetics.
VIII. REFERENCES
[1] A. D. Jepson, D. J. Fleet and T. F. El-Maraghi, "Robust
online appearance models for visual tracking," in IEEE
Transactions on Pattern Analysis and Machine Intelligence,
vol. 25, no. 10, pp. 1296-1311, Oct. 2003, doi:
10.1109/TPAMI.2003.1233903.
[2] A. Adam, E. Rivlin and I. Shimshoni, "Robust
Fragments-based Tracking using the Integral Histogram,"
2006 IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (CVPR'06), New York, NY,
USA, 2006, pp. 798-805, doi: 10.1109/CVPR.2006.256.
[3] X. Liu, P. H. Tu, J. Rittscher, A. Perera and N.
Krahnstoever, "Detecting and counting people in
surveillance applications," IEEE Conference on Advanced
Video and Signal Based Surveillance, 2005., Como, Italy,
2005, pp. 306-311, doi: 10.1109/AVSS.2005.1577286.
[4] B. Babenko, M. -H. Yang and S. Belongie, "Visual
tracking with online Multiple Instance Learning," 2009
IEEE Conference on Computer Vision and Pattern
Recognition, Miami, FL, USA, 2009, pp. 983-990, doi:
10.1109/CVPR.2009.5206737.
[5] D. S. Bolme, B. A. Draper and J. R. Beveridge,
"Average of Synthetic Exact Filters," 2009 IEEE
Conference on Computer Vision and Pattern Recognition,
Miami, FL, USA, 2009, pp. 2105-2112, doi:
10.1109/CVPR.2009.5206701.
[6] D. S. Bolme, Y. M. Lui, B. A. Draper and J. R.
Beveridge, "Simple real-time human detection using a
single correlation filter," 2009 Twelfth IEEE International
Workshop on Performance Evaluation of Tracking and
Surveillance, Snowbird, UT, USA, 2009, pp. 1-8, doi:
10.1109/PETS-WINTER.2009.5399555.
[7] Y. -L. Hou and G. K. H. Pang, "People Counting and
Human Detection in a Challenging Situation," in IEEE
Transactions on Systems, Man, and Cybernetics - Part A:
Systems and Humans, vol. 41, no. 1, pp. 24-33, Jan. 2011,
doi: 10.1109/TSMCA.2010.2064299.
[8] Mukherjee, Subra, and Karen Das. "Omega model for
human detection and counting for application in smart
surveillance system." arXiv preprint arXiv:1303.0633
(2013).
[9] Otasowie, P. O., & Edeoghon, I. A. (2014). Design and
Implementation of a Human Detector and Counting System
using MATLAB.
[10] Zhang, X., Zhang, L. (2014). Real Time Crowd
Counting with Human Detection and Human Tracking. In:
Loo, C.K., Yap, K.S., Wong, K.W., Beng Jin, A.T., Huang,
K. (eds) Neural Information Processing. ICONIP 2014.
Lecture Notes in Computer Science, vol 8836. Springer,
Cham. https://doi.org/10.1007/978-3-319-12643-2_1
[11] H. Mokayed, T. Z. Quan, L. Alkhaled, and V.
Sivakumar, “Real-Time Human Detection and Counting
System Using Deep Learning Computer Vision
Techniques”, AIA, Oct. 2022.