Automated Vision-based Surveillance System to

Detect Drowning Incidents in Swimming Pools

Abdel Ilah N. Alshbatat‫٭‬ᵜ, Shamma Alhameli‫٭‬, Shamsa Almazrouei‫٭‬, Salama Alhameli‫٭‬, Wadhha Almarar‫٭‬

‫٭‬Faculty of Engineering Technolgy and Science

Higher Colleges of Technology
United Arab Emirates
{aalshbatat; h00326190; h00326180; h00326191 ; h00326179 }@hct.ac.ae

ᵜDepartment of Communications, Electronics and Computer Engineering

Tafila Technical University
Tafila, 66110, Jordan
a.alshabatat@ttu.edu.jo

Abstract — At present, swimming pools are built in hotels,
sport clubs, schools and private residences. Although there have
been various regulations put into place to reduce drowning
accidents in some countries, communities still experience many
drowning incidents. Accordingly, a real-time system that will
track swimmers in a pool using machine learning techniques
and prevents drowning accidents is proposed. The system
consists of a Raspberry Pi with the Raspbian operating system,
a Pixy camera, an Arduino Nano board, stepper motors, an
alarm system, and motor drivers. The proposed system is based
on the color-based algorithm to position and rescue swimmers
who are drowning. The device then sends an alarm to the
lifeguards. To verify the performance of the proposed system, a
prototype has been developed, implemented, and tested. The
results from experiments indicate that the system has a unique
capability to monitor and track swimmers, thereby enabling it
to mitigate and curb the number of deaths by drowning.
Keywords— Drowning Detection, Alert System, Real-Time
Image Processing, Swimming Pools.
I. INTRODUCTION
In recent years, there has been an interest in integrating
computer vision in swimming pool surveillance systems.
Automating such a process will provide the communities with
an efficient way of detecting drowning incidents that may
occur while swimming. As the evolution in sensor technology
and image processing algorithms are advancing day by day,
different approaches for solving such emergencies have been
proposed. Existing drowning detection technologies can be
categorized into two types: vision based systems [1], and
wearable sensor based systems [2].
Vision based systems are classified according to the
positions of the cameras. They mainly depend on two types of
cameras: underwater cameras [3], and above water cameras
[4]. Underwater cameras are mainly used to eliminate
interference caused by the water. Such method still has a
limitation, since the used camera might miss incidents taking
place above the water. Above water cameras also have some
limitation where it is difficult to cover the entire swimming
pool with one camera.
Wearable-based systems are also used in this regard to
monitor the individual’s depth in water, heart rate pressure,
and to monitor how much time an individual spends there.
Wristbands and headbands are two examples of wearable-
based systems. Under preset values, the wearable system will
send a wireless alarm signal to the lifeguards. Another device,
that monitors the nervous system’s activities through the use
of electrocardiographic (ECG) signals, have been also
proposed. Furthermore, accelerometers and gyroscopes were
used to measure swimmers' behavior in water. The
measurements from the above mentioned sensors are used to
trigger the drowning danger alarm. A current limitation of the
use of such a system is discomfort. This discomfort is
especially felt by younger children.
For this project, a hybrid system that will automatically
detect a drowning person and then set off an alarm to alert the
lifeguards has been developed. The system mainly consists of
three modules: a vision module, an event-inference module
and an event-driven module. The vision module is responsible
for monitoring and detecting the position of the person who is
drowning. The event-inference module is responsible for
determining a swimmer’s position, velocity, and path of the
movement. The event-driven module is responsible for
initiating the rescue by sending an alarm alerting the lifeguard.
This project is a modified version of our project submitted to
the Think Science competition, where swimmers’ positions,
velocities, movement paths and time under water were
collected and used to determine through inference the eminent
danger.
The main contribution of this project is to develop a
system for monitoring swimming pool to prevent the onset of
a drowning incident. The remainder of this paper is organized
as follows: The next section will start with a survey of current
research regarding the technologies that are employed with the
aim of using vision-based system in detecting drowning
incidents in swimming pools. Section III presents an overview
of the proposed approach. The proposed system is described
in section IV. Experimental results are presented in section V.
Finally, conclusions and plans for future projects are drawn in
section VI.
II. RELATED WORK
Automating vision-based surveillance systems are
considered one of the effective ways of monitoring any
unusual activity in swimming pools. In this regard, processing
the background on input video frames has been proposed. The

Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on March 07,2022 at 10:16:35 UTC from IEEE Xplore. Restrictions apply.
authors in [5] develop a video surveillance system capable of
detecting drowning incidents in swimming pool. The system
is based on modelling the background and then the detection
of swimmers' is performed based on the comparing the
similarity between a color model and the background model.
A drowning detection method based on background
subtraction was also presented in [6]. The authors used a
Gaussian mixture model to describe the background and
swimmers. His model was able to eliminate the shadows and
noise from the foreground objects.
In [7], the authors combine the mean-shift clustering and
cascaded boosting learning algorithm for swimmer detection.
They proposed three steps in their work: background
modeling, swimmer detection, and swimmer tracking. To
tackle the background and crowded scenarios at the pools,
authors in [8] proposed a set of methods like background
subtraction, de-noising, data fusion and blob splitting to
recognize different swimming activities and to detect the
occurrence of early drowning incidents.
Detection of swimmers' body parts was also researched
in this regard. In [9] the authors propose a method based on
pixel classification for the detection of objects in water. They
analyzed the standard color models and proposed YCbCr as
the best color model for describing uniqueness of object.
Measuring swimming biomechanics was proposed in [10].
Authors used computer vision techniques to enhance images
for improving clarity and automating detection of both arm
The Pixy camera combines a camera and a
microcontroller running an optimized color detection
algorithm. It processed any input image in a resolution of 640
x 400 per frame and outputs an image with a resolution of 320
x 200 pixels. The camera requires 5v, and consumes 140 mA.
The camera is trained to detect the yellow color. It can be
learned up to detect seven color signatures. If the object is
detected by the camera, the camera will output a block of data
that contains the position, the width and height of the object
in pixels. The output from camera is then fed to the Raspberry
Pi through a USB connection for further processing.
Furthermore, the camera has an analog and digital outputs
pins. Pin 1 goes high (3.3V) when an object is detected, and
pin 8 outputs a voltage that is directly linearly proportional to
the object position in the image. The two pins are connected
directly to ElekStar controller. Figure 2 shows Pixy camera
and ElekStar I/O pins.
LED Strip (R) LED Strip (G) LED Strip (B) Robot
Relay Relay Relay
Speaker Level Shifter Arduino Nano
Amplifier Home Switch X Home Switch Y

and legs segments.

Preventing drowning accidents using thermal cameras was Raspberry Pi

also researched. In [11], the authors developed two sub- Pixy Camera 1 Stepper Motor X
algorithms for intruder detection using thermal imaging ElekStar
controller
system. The algorithm can detect not only the movement of a Stepper Motor Y

human, but also any water activity such as swimming, and

Pixy Camera 2 Stepper Motor Y
water-splashing. In this project, two Pixy cameras were used.
The camera itself combines a camera and a microcontroller
running an optimized color detection algorithm. It is also
Fig. 1. Architecture of the proposed system.
considered as a thermal imaging device.
III. OVERVIEW OF THE PROPOSED APPROACH
The Raspberry Pi as well as the Arduino Nano boards will
be used in controlling the entire system. The individuals
entering the pool territory should wear a passive yellow vest.
Two Pixy cameras will continuously monitor the swimming
pool. The captured 2-D images will be processed by the
internal hardware attached to the cameras. The analyzed data
will be sent to the Raspberry Pi board where a python script is
running. The script will calculate swimmers’ positions,
velocities, paths of movement, and time under water. Based (a) (b)
on these calculations, the occurrence of any abnormal events UART
12v 5v 5v
will be detected. If such events occur, Raspberry Pi will send Rx
an order to the linear stage. The linear stage will directly move GND TXD UART
SPI Tx
to the victims location with a hook attached to it. The hook A5 RXD
will help in pull the swimmer upwards out of the pool. A3 D9 I2C GND
Meanwhile, a warning signal will alert the lifeguard of an A2 D10
imminent danger. SPI AO
A1 D12

IV. SYSTEM DESCRIPTION A0 D11 I2C Vin

As shown in figure 1, the proposed system consists of the (c) (d)

following: Pixy cameras, Raspery Pi, level shifter, Arduino Fig. 2. Pixy camera and ElekStar I/O pins (a) Front side of the Pixy camera,
Nano, three relays, three LED strips, Vex DC motor, Vex (b) Back side of the Pixy camera, (c) ElekStar I/O control pins, (d) Pixy
motor controller, robot arm, ElekStar Linear stage, ElekStar camera I/O pins.
controller, two switches, and three stepper motors.

Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on March 07,2022 at 10:16:35 UTC from IEEE Xplore. Restrictions apply.
 Raspberry Pi was attached to two Pixy cameras. The two ElekStar linear stage consists of three stepper motors,
cameras are oriented at about 90° relative to one another and controller, two switches, and one robot arm (claw). As shown
attached also to the linear stage controller. A python script is in figure 5 (a), the claw is equipped with a Vex DC motor,
written to process the data from the cameras and then and Vex motor controller. The arm is controlled separately by
commands the linear stage accordingly. Meanwhile, the an Arduino Nano board, and it used to handle the rescue tools.
output is also fed to the level shifter. Since the Raspberry Pi The stepper motors as well as the two switches are shown in
GPIO pins work with 3.3v logic levels, while other devices figure 5 (b). The main purpose of the switches is to bring the
work with 5v logic levels, a level shifter is added to interface motors back to the original coordinates.
the communication between them (figure 3, and figure 4). A
LED strip is used with three colors; red, blue, and green. The
green LEDs get ON when the system is switched ON. On the
other hand, if any abnormal events occur, the red LEDs in the
linear stage will turn ON. Lastly, the blue LEDs are turn ON
while the claw is working. The strip interfaces with three
relays since it requires 12v. An audio message is also
programmed into the Raspberry Pi to identify whether a
person is drowning or not.

G GPIO27 GPIO04 3.3v

GPIO22 GPIO17

G A4 A3 A2 A1 LV
(a) (b)
Level Shifter
Fig. 5. ElekStar linear stage (a) robot arm, (b) stepper motors and home
switches.
G B4 B3 B2 B1 HV

ElekStar controller mainly consists of an Arduino Nano

G NANO P7 LED R 5v board and two A4988 stepper motor drivers (figure 6). The
NANO P9 LED G
stepper motors are of type 42H34S-1304A and typically have
a step size of 200 steps per revolution. The Nano board was
(a) (b) programmed to receive the data from the cameras as well as
the Raspberry Pi. The data specifies the coordinates (x, y) in
Fig. 3. Level shifter (a) I2C Logic level converter module, (b) cm to be passed to the stepper motors. After reaching the
Connection with Raspberry Pi and Arduino Nano.
dedicated position, claw will open and drop the rescue tool.
To ensure that the linear stage will go back to the original
point, we used two switches; home-switch X, and home
switch Y.

(a) (b)

Fig. 6. ElekStar controller (a) Arduino Nano board and two A4988 stepper
motor drivers, (b) A4988 chip.
Fig. 4. Raspberry Pi GPIO pins.

Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on March 07,2022 at 10:16:35 UTC from IEEE Xplore. Restrictions apply.
 V. EXPERIMENTAL RESULTS
A shown in figure 7, a prototype has been developed in
our laboratories to validate the proposed system. The
prototype consists of two parts: The electronic part, and the
pool. The electronic parts are assembled on an aluminum
frame. The physical dimensions of the frame are (92x92x45)
cm. The pool is fabricated from glass with the physical
dimensions of (52x52x25) cm. To test and demonstrate the
laboratory prototype services, several experiments were
conducted. The purpose of the preliminary experiments were
to collect pre-test data and then evaluate the factors that
should be considered when implementing such system on a
real pool.
(d)

Fig.7. Hardware architecture of the prototype (a) Front side, (b) Right
side, (c) Left side, (d) Back side

A. Pixy camera training

Several experiments were conducted to train the Pixy
camera to find objects by color. The training is done through
two processes: with PC-based PixyMon software, and with a
self-contained procedure that relies on a button on the
camera. The result for both procedures were identical. Once
the camera is trained to a yellow color, an Arduino board is
programmed and attached to the camera using the SPI bus
connector, figure 8(a). The same procedure was repeated
again by hooked up the camera to a Raspberry Pi using the
(a) USB cable, figure 7(b).

(a) (b)

Fig.8. Hardware connection (a) Arduino and Pixy camera, (b)

Raspberry Pi with Pixy camera

The camera returned a block of data for an object that had

been assigned a color signature. The processed data included
(b) the followings: Block (block position in the array, starting at
zero), Sig (The signature number, from 1 to 7), X and Y (the
coordinates of the object within the visible frame),
Width (object width), Height (object height), Index (a unique
number assigned to an object on the display), and Age (a
number from 1 to 255 to form a history for the object). The
following table has a sample of data collected from the
trained Pixy camera at different distances.

TABLE I. Data returned by the Pixy camera. Here the units are in Pixels

Block Sig X and Y W and H Index Age

0 1 46, 35 92, 65 132 40
0 1 47, 34 80, 55 132 40
0 1 47, 34 79, 46 132 40
0 1 47, 34 57, 33 132 40

(c)

Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on March 07,2022 at 10:16:35 UTC from IEEE Xplore. Restrictions apply.
 B. Swimmer detection and tracking
The second set of experiments has aimed at assessing the
object detection in pool. The experiment setup is shown in
figure 9. Unfortunately, the initial results were unsatisfactory
because of the false triggering. The problem was solved by a
fine tune to the Pixy camera. Pixy is processing 60 frames of
video per second. We adjusted the frames per second to a
value less than 60. In addition, the signature range was tuned
to improve detection accuracy. After having all settings
loaded to the Pixy, good results were obtained when the
object was at the top-left, at the top-right, at the bottom-left,
and at the bottom-right of the image. All results were taken
with respect to the Pixy monitor. The results are documented
in table II.
(a) (b)
system, the number of drownings would be reduced. For
future development, the system is currently being improved
by attaching an infrared LED to the swimmer’s vest.
REFERENCES
[1] W. Lu, Y. Tan, Y. Peng, “A Vision-Based Approach to Early Detection
of Drowning Incidents in Swimming Pools,” IEEE Transactions on Circuits
and Systems for Video Technology 14:2 (2004):159 –178.
[2]. S. Nagalikitha, A. Kiranmai, “Automatic Waist Airbag Drowning
Prevention System Based on Motion Information Measured by Memos
Accelerometer and Pressure,” International Journal of Emerging Trends in
Engineering Research (IJETER) 3:6 (2015): 204-206.
[3] Z. Chi, X. Li, and F. Lei, “A Novel Camera-Based Drowning Detection
Algorithm,” Advances in Image and Graphics Technologies, Springer Berlin
Heidelberg (2015): 224-233.
[4] N. Salehi, M. Keyvanara, S. Monadjemmi, “An Automatic Video-based
Drowning Detection System for Swimming Pools Using Active Contours,”
I.J. Image, Graphics and Signal Processing 8:8 (2016).
[5] A. Kam, W. Lu, W. Yau, “A video-based Drowning Detection System,”
Proceedings of European Conference on Computer Vision LNCS, vol. 2353,
pp. 297–311 (2002).
[6] F. Lei, W. Xueli, and C. Dongsheng, “ Drowning Detection Based on
Background Subtraction,” Embedded Software and Systems, 2009.
ICESS'09. International Conference on. IEEE, 2009.
[7] W. Chen, P. Cho, P. Fan, and Y. Yang, “A framework for Vision-based
Swimmer Tracking,” International Conference on Uncertainty Reasoning
and Knowledge Engineering, pp. 44–47, 2011.
[8] H. Eng and K. Toh, “DEWS: A live Visual Surveillance System for Early
Drowning Detection at Pool,” IEEE Transactions on Circuits and Systems
for Video Technology, vol. 18, no. 2, pp. 196–210, 2008.
[9] P. Vladimir, and V. Papić, “Features Analysis for Tracking Players in
Water Polo,” 16th International Conference on Automatic Control,
Modelling & Simulation. 2014.
[10] R. Dubois, D. Thiel, and D. James, “Using Image Processing for
Biomechanics Measures in Swimming,” Procedia Engineering, vol. 34, pp.
807-812, 2012.
[11] W. Wong, J. Hui, C. Loo and W. Lim, “Off-time Swimming Pool
Surveillance Using Thermal Imaging System,” International journal of
innovative computing, information and control, vol. 9 (3), pp. 366-371,
2013.

(c) (d)
Fig.9. Experimentation setup for the underwater object detection (a)
Pixy camera alignment, (b) Home switch's alignment, (c) Testing alarming
unit, (d) Placing the pool in the middle

TABLE II. Data returned by the Pixy camera when object under water.

Block Sig X and Y W and H Index Age Location

0 1 63,62 92, 63 145 255 Top-left
0 1 271, 62 93, 64 145 255 Top-right
0 1 63, 161 91, 63 145 255 Bottom-left
0 1 270, 160 92, 65 145 255 Bottom-right

VI. CONCLUSION AND FUTURE WORK

This research presents an automated vision-based
surveillance system to detect drowning incidents in swimming
pools. The swimmers in the pool are detected and tracked
using the Pixy camera. As soon as the swimmer remains under
a certain level for more than a determined period of time,
Raspberry Pi will calculate that swimmers’ position, velocity,
path of movement and send an order to the linear stage. The
linear stage will directly move to the victim location with a
hook attached to it. The hook will help with flotation by
pulling the swimmer upwards. Meanwhile, a warning message
will signal the lifeguard of an imminent danger. With such a

Authorized licensed use limited to: VIT University- Chennai Campus. Downloaded on March 07,2022 at 10:16:35 UTC from IEEE Xplore. Restrictions apply.