2021 IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE 2021)
Safety Belt Wearing Detection Algorithm Based on
2021 IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE) | 978-1-7281-8319-0/20/$31.00 ©2021 IEEE | DOI: 10.1109/ICCECE51280.2021.9342340
Human Joint Points
QIAO Yi
College of Information Engineering
Engineering University of PAP
X i’an, China
e-mail: 1368273704@qq.com
Abstract—In order to identify whether the driver is wearing a
seat belt correctly, a seat belt wearing detection algorithm is
proposed based on human joint points. First, the VGG model is
used to extract the position of the driver’s characteristic joint
points. In this process, in order to overcome the information loss
caused by repeated pooling and upsampling, a dilated
convolution is proposed, and the pruning operation is used to
reduce the complexity of the model. A kind of seat belt feature
vector representing the two-dimensional vector field of the
shoulder joints pointing to the hip joints is proposed to express
the position and direction of the seat belt in the detection area.
Finally, the seat belt rupture caused by the occlusion should be
automatically connected to complete the detection of the driver's
seat belt.
Keywords- human joint points; seat belt detection; feature
vector
I. In t r o d u c t io n
With the rapid economic development in recent years,
private cars have become the main means of transportation.
Due to the increase in vehicles and frequent traffic violations,
the quality of road traffic safety is declining year by year.
There are approximately 1.25 million road traffic accident
deaths worldwide each year, which is equivalent to 3,500
deaths due to traffic accidents worldwide. As one of the most
effective passive protection measures, seat belts can effectively
reduce the casualty rate in traffic accidents. According to the
road traffic safety law, drivers and passengers should use seat
belts in accordance with regulations when a motor vehicle is
running. At present, the application of seat belt recognition at
home and abroad is mainly based on buckle sensors[1] and
pressure sensors[2], but some drivers with low safety awareness
still use a separate seat belt buckle or buckle the safety belt
before getting on the car to avoid wearing a reminder system.
With the development of artificial intelligence technology,
people try to use computer vision to directly identify the seat
belt wearing situation from video images. Reference [3]
obtained the rough position of the driver by locating the license
plate in the HSV color space, and then used the Hough linear
change algorithm to detect the edge of the seat belt. This
method is greatly affected by the background color and noise
and requires high image quality. It is difficult to be promoted
and applied. Reference [4] proposed to detect car windows by
cascading Adaboost classifiers, extracting Canny edge features
from the window gradient map, and finally use Hough
transform to extract seat belt linear features to achieve
recognition and detection tasks. This method improves the
978-1-7281-8319-0/21/$31.00 ©2021 IEEE
538
Authorized licensed use limited to: University of Prince Edward Island. Downloaded on May 17,2021 at 03:10:31 UTC from IEEE Xplore. Restrictions apply.
QU Yi
College of Information Engineering
Engineering University of PAP
X i’an, China
e-mail: wjquyi@sina.com
classification ability, but the Hough transform algorithm has
poor robustness in the actual application process. In order to
improve the detection speed, Reference[5] proposed to use the
light-weight YOLO algorithm to achieve rapid positioning of
the driver's area, and then use the connected domain obtained
by semantic segmentation to judge the wearing of the driver's
seat belt.
Aiming at the deficiencies of the existing seat belt wearing
detection algorithms, this paper uses the human body posture
estimation algorithm to extract the key point skeleton
information of the driver in the video image based on the safety
monitoring data in the vehicle, and then starts from the
heatmap of the shoulder joint points. The Part Affinity Field
(PAF) algorithm divides the seat belt area and accurately
recognizes whether the driver is wearing the seat belt correctly,
which helps to improve the driver’s safety awareness, thereby
reducing casualties in traffic accidents.
II. Im a ge Ac q u is it io n a n d Pr e p r o c e s s in g
A. Image Preprocessing
Due to the complex background in the car, there is
redundant interference information in the picture taken by the
camera in the car, which hinders the driver's positioning and
joint point extraction. Therefore, we first use the low-pass filter
to remove part of the noise of the image, and then choose a
Gaussian linear filter. The output pixel value is the sum of the
convolution of each input pixel and the Gaussian kernel. Image
smoothing filters out high-frequency components in the image,
and preserves the subject to be studied to the greatest extent.
When the background environment of driving at night is
dark and the light source is insufficient, the uneven grayscale
distribution of the collected image is mostly concentrated at the
low end of the grayscale, and the dynamic range is small. Also
under the background of high light intensity, the components of
the histogram of the collected image are mostly concentrated in
the high end of the gray level. In order to improve the accuracy
of subsequent recognition and detection, it is necessary to use
the histogram equalization algorithm16 to stretch the grayscale
of the image to the entire grayscale range of 0 to 255. Therefore,
the whole image has a larger dynamic range of gray scale and
higher contrast, and the details of the image are more abundant.
Based on the histogram information of the input image, the
transformation function can be obtained to achieve the
histogram equalization.
 Figure. 1 Heatmap of typical joint points
B. Driver Joint Points Extraction
The current detection of the coordinates of human bone
joint points is the basis for describing the posture of the human
body and predicting the behavior of the human body. The main
joint points of the human body are extracted by coordinate
regression or heatmap detection, and then clustering and
connecting them for the image of "stickman" extracted from the
complex background environment171.
Accurately locating the bone joints is a prerequisite for
detecting whether the seat belt is worn. After preprocessing the
image, it is necessary to extract the joints of important parts of
the driver. Due to the constraints of the driver's driving position,
the camera placed on the upper side of the windshield can only
capture the driver's upper body, and the key parts of the seat
belt wearing condition detection are also concentrated on the
human upper body. In order to reduce the amount of
calculation and reduce the time cost, irrelevant joint points are
deleted, and only the left and right shoulders and left and right
hip joints of the more representative driver can be tested.
This paper proposes a bone joint point detection method
based on heatmap, as shown in Figure. 1. The VGG model[8] is
used to initially extract image features, output the confidence of
each joint point, and use Gaussian function to normalize the
response to the center position. The joint of the output feature
map response and the original image is used as the input of the
next stage, thereby fusing spatial information and predicting
the position coordinates of the joint points based on the
accumulated heatmap. The network uses a sequential
convolutional architecture to express spatial and texture
information. In order to make the predicted spatial position of
the joint point heatmap more accurate, Reference[9] proposed a
high-resolution network HRNet, which gradually added low
resolution feature map sub-networks in parallel to the main
network of the high-resolution feature map to realize multi­
scale fusion and feature extraction. In view of the current in-car
application scenarios, the human body has small changes in
scale. Traditional methods always maintain multi-scale feature
extraction which could reduce the computational efficiency.
This paper introduces dialated convolution in the joint point
detection network to expand the receptive field and capture
multi-scale context information without resolution lost.
539
Authorized licensed use limited to: University of Prince Edward Island. Downloaded on May 17,2021 at 03:10:31 UTC from IEEE Xplore. Restrictions apply.
Figure. 2 The rough position of the Seat belt
In order to ensure the calculation speed of the seat belt
detection algorithm, the network needs to be accelerated
without affecting the driver's joint point extraction effect.
Therefore, the operation of pruning[10] is used to delete the
redundancy that has little impact on the network feature
extraction ability. For the convolution weight of, the
parameters need to be evaluated. By comparing the size of the
weight parameter l2 norm, the convolution kernel with a
smaller l2 norm is deleted, thereby reducing the computational
complexity of the model and accelerating the inference speed
of the network.
III. D r i v e r 's Se a t Be l t De t e c t i o n B a sed o n Hu m a n Jo i n t
Po i n t s
Detecting the driver’s seat belt wearing situation can be
regarded as a two-class classification problem, and the
classification algorithm can achieve better detection results
when the background environment is simple and easy to
segment. The classifier learns the difference between positive
and negative samples during the training process. The feature
difference is used for classification. In real applications, the
wearing of seat belts cannot be used as a single feature to
distinguish between positive and negative samples due to the
different lighting, background, driver's clothing and posture in
the car.
This paper completes the rough positioning of the seat belt
detection area by extracting the driver's characteristic joint
points, constructs the seat belt feature vector, and expands it to
connect the break of the seat belt.
A. Seat Belt Detection Area Location
If the joint points of the driver are detected in the driving
area, the seat belt detection area of the main driver is
determined according to the locations of the shoulder and hip
joints; if the joint point information cannot be extracted in the
driving area, the seat belt recognition processing will not
continue.
The seat belt of the main driver is from the left shoulder to
the right hip, so the detection area of the seat belt is the area
where the left and right shoulders and the left and right hip
joints are located. Let x s l , xs r , x h l , xhr be the joints of the
left and right shoulders and hips of the driver respectively.
When the point is located, the detection area is a closed area
connected by four joint points, so as to filter out the unrelated will not break into many short slash ends. It appears as a
background edges, as shown in the Figure. 2. Through coarse relatively complete block, so the two oblique line segments are
positioning of the seat belt detection area, the traditional automatically connected if the slope of the oblique line at the
method of identifying seat belts can reduce the amount of break is within the error threshold T.
calculation and improve the accuracy of detection.
IV. Te s t R e s u l t s a nd An a l y s is
B. Seat Belt Recognition
1) Seat belt feature vector: A. Experimental setup
This paper proposes a seat belt feature vector based on the The dataset used in the experiment is obtained through the
idea of part affinity field (PAF)[11], a two-dimensional vector camera on the front windshield of the vehicle, and the format
field from the shoulder joint to the hip joint, which represents is .jpg, covering various lighting scenes such as morning, noon,
the position and direction of the seat belt in the detection area. and evening. At the same time, the driver's age, gender and
Figure. 3 is an example for analysis. If a point P is on the seat clothing are as diverse as possible. In terms of data calibration
belt, the value at L ( p ) is a unit vector from s to h. For other preparation, the joint points of the four key parts of the driver’s
left and right shoulders and left and right hips and their seat
points in the detection area, the vector is a zero vector, as in (1)
belts are required to be marked. When the Ground Truth is
and (2).
constructed, the length and width of the seat belt affinity field
if p on seatbelt are specified. The direction of each pixel is represented by the
|v
(1) unit vector of the connection direction of the shoulder and hip
yo otherwise
joint points.
x h ,r - x s,l
(2) In the training process, it is necessary to perform data
h.r - x s,l||2
ik enhancement through operations such as image translation,
V is a unit vector on the seat belt, and the point P on the seat flipping, and zooming to enhance the robustness of the
belt should meet the following conditions, as in(3). algorithm. In order to avoid the disappearance of the gradient
caused by the error propagation through multiple layers, the
_Yವp - xs,l  O loss value of the joint point extraction stage adopts relay
supervision1121, and the loss value is calculated independently in
       each stage to ensure the normal update of the underlying
\v± \ p - [VO
parameters.
Equation (3), l = ||Xsl - Xhr|^represents the length of the
safety belt, and o represents the pixel width of the safety belt.
Thus, the characteristic vector ^ L (p ) of the seat belt is
calculated. In the seat belt wearing detection stage, the
information of the seat belt affinity field is used to extract the
pixel directions on all the lines between the hip part and the
shoulder part at the two characteristic joint points, and calculate
the line integral of the predicted seat belt feature vector along
the line segment, as in (4).

Figure. 3 The feature vector of the seat belt

p (u ) = (1 - u )• dsj + u •d h,r (5)
Where p(u) represents any value on the line of two
characteristic joint points, where 0 < u < 1. The greater the E
value, the greater the confidence.
2) Automatic Connection o f Seat Belt at the Break:
Since the driver holding the steering wheel with both hands
while driving the vehicle, which may cause the seat belt
connection in the two-dimensional image to be blocked and
interrupted by the hand. It is necessary to add the automatic
connection of the safety belt slash at the break in the detection
network.
First, as shown in Figure. 4, calculate the slope of the seat
belt edge according to the position coordinates of the
Figure. 4 Automatic connection of the seat belt diagonal line at the break
characteristic joint points and the seat belt characteristic vector.
The long slash of the seat belt usually blocked by the hand and

540

Authorized licensed use limited to: University of Prince Edward Island. Downloaded on May 17,2021 at 03:10:31 UTC from IEEE Xplore. Restrictions apply.
 'a m
n 4'
Figure. 5 Seat belt wearing test results
B. Evaluation o f seat belt detection effect
The safety belt wearing detection effect is shown in Figure.
5. The correct detection rate PD, false alarm rate Pf and missed
detection rate Pm are used as evaluation indicators for seat belt
wearing detection, and the calculation method is shown as in (6)
and (7).
N = Nr + Nf + Nm
Among them, Nr is the number of pictures that have been
correctly detected by the seat belt, Nf is the number of pictures
that are mistaken as not being worn when the seat belt is worn
correctly, Nm is the number of pictures that are mistakenly
detected as being worn correctly without the seat belt, and N is
the sum of the number of detected pictures.
From the perspective of practical application, it is necessary
to reduce the probability of missed detection as much as
possible and improve the efficiency of correct detection within
a certain range of false alarms.
V. Co n c l u s io n
This paper proposes a detection algorithm for seat belt
wearing based on human joint points. The algorithm extracts
the driver’s characteristic joint points after preprocessing the
image, and roughly obtains the seat belt detection area by
processing the position information of the joint points. Then it
Authorized licensed use limited to: University of Prince Edward Island. Downloaded on May 17,2021 at 03:10:31 UTC from IEEE Xplore. Restrictions apply.
uses the proposed seat belt feature vector for identification and
detection, and finally covers the fracture. The seat belt slash is
automatically connected to complete the detection of the
driver’s seat belt wearing condition.
Ac k n o w l edgment
This project was funded by National Natural Science
Foundation of China(61801516) and Natural Science Basic
Research Program of Shaanxi Province of China(2019JQ-238).
Re f e r en ces
[1] Okada S, Suzuki Y and Takayanagi H. "Seat belt warning device, " U.S.
Patent, May.2001.
[2] Becker G, Mousel T and Schockmel P. "TOR (total occupant recognition)
system" [R]. SAE Technical Paper, 2001.
[3] H.Guo, H.Lin, S.Zhang and S. Li, "Image-based seat belt detection,"
IEEE International Conference on Vehicular Electronics and Safety, pp.
161-164.2011
[4] W.Li, J.Lu, Y.Li, Y.Zhang, J.Wang and H.Li, "Seatbelt detection based
on cascade Adaboost classifier," International Congress on Image and
Signal Processing, pp. 783-787.2013.
[5] A. Corovic, V. Ilic, S. Buric, M. Marijan and B. Pavkovic, "The Real­
Time Detection of Traffic Participants Using YOLO Algorithm,"
Telecommunications Forum, pp. 1-4.2018.
[6] Y.He, X.Zhang and J.Sun, "Channel Pruning for Accelerating Very
(7) Deep Neural Networks, " IEEE International Conference on Computer
Vision, pp. 1389-1397.2017
[7] Y.Chen, Y.Tian and M.He, "Monocular human pose estimation: A
survey of deep learning-based methods, " Computer Vision and Image
Understanding, Vol 192, March.2020.
[8] Simonyan, Karen and A. Zisserman, "Very Deep Convolutional
Networks for Large-Scale Image Recognition," Computer ence, pp. 1­
14.2014.
[9] K.Sun, B.Xiao, D.Liu and J.Wang. "Deep High-Resolution
Representation Learning for Human Pose Estimation," IEEE Conference
on Computer Vision and Pattern Recognition, pp. 5693-5703.2019.
[10] Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila and Jan Kautz,
"Pruning convolutional neural networks for resource efficient inference,
arXiv preprint arXiv:1611.06440, 2016.
[11] Z .Cao, Tomas Simon, Shih-En Wei and Yaser Sheikh, "Realtime Multi­
Person 2D Pose Estimation Using Part Affinity Fields," IEEE
Conference on Computer Vision and Pattern Recognition, pp. 7291­
7299.2017.
[12] Alejandro Newell, K.Yang and J.Deng, "Stacked Hourglass Networks
for Human Pose Estimation," European Conference on Computer Vision,
pp. 483-499, 2016
541