Intelligent Fire-Fighting Robot with Deep Learning
1. Mohamed Imran Reshmaja K Ramesh
Mechanical Engineering
Sri Sairam Engineering College
Chennai, India
mimran.ias@gmail.com
S. S Abineshwar
Mechanical Engineering
Sri Sairam Engineering College
Chennai, India
e8me l89@sairamtap.edu.in
Abstract-An accidental fire is a mishap that could be
2022 International Conference on Communication, Computing and Internet of Things (IC3IoT) | 978-1-6654-7995-0/22/$31.00 ©2022 IEEE | DOI: 10.1109/IC3IOT53935.2022.9767869
happen man-made or natural. Accidental fire occurs frequently
and can be controlled but sometimes result in severe loss of life
and property. To prevent these deadly activities, an autonomous
fire-fighting robot is introduced. The bot will be activated as
soon as the information ofthe fIre accident reaches the bot. Once
the location is fIXed with google API, waypoints can be marked
through the mission planner with the help of the GPS module.
The bot will autonomously reach the destination using computer
vision technologies. Lane, obstacle, and traffic light detections
are executed to enable the autonomous drive. YolovS
architecture plays a major role in detection methodologies. As
soon as it reaches the destination, the fire and heat sensors will
detect and spray water to douse the fIre through the nozzle that
can rotate about 180·. Initially, only one bot will be sent for
rescue purposes, if it can't satisfy the needs and requirements,
then the GSM module will send the message of inefficiency. So
that more bots will reach the destination and can resolve the
problem. To protect the electronic components the bot is
thermally insulated with a special alloy of stainless steel The
proposed system will help reduce the burden of firefighters as
well as manual errors caused during firefighting thus, it can own
by private companies as well government.
Keywords-Arduino UNO; GSM Module; GPS Module; Fire
Sensor; Raspberry Pi 4;
1. INTRODUCTION
Fire accidents can be man-made or natural. They cause
great destruction to the livelihood. According to a survey it
was estimated that around 20,000 people die due to fIre
accidents every year. National fIre protection association
reports that fIre accidents cause damages of around $7 billion
annually. It causes catastrophic and devastating damages to
the properties. So, to avoid such destruction, fIrefIghters need
to risk their lives to save people's life and properties.
Safeguarding people who risk their lives to save others is very
crucial. But according to estimation around 30% of
fIrefIghters get hurt during fIrefIghts.
RS1-Tl Thermite was invented in 2012 by Howe & Howe.
It was the fIrst commercial fIrefighting robot. The thermite
robot set a trend for fIrefIghting robots that was widely used
in the US military [13]. Howe & Howe developed the
technology wider with their inventions ofRSl- T2, RSl-T3,
RS 1-T4. But the one which is very popular among the military
troop is Thermite RS3. From the year 2019 fIrefIghting robots
become more common and advanced. Because it was the time
of the emergence of popular technologies like computer
vision, deep learning, machine learning, etc. Intel Research
initiative launched the OpenCV project in 1999. It was an
advance to a series ofprojects including real-time ray tracing,
3D displays walls, and also mask detection, in which
CPU-
978-1-6654-7995-0/22/$31.00 ©2022 IEEE
Authorized licensed use limited to: Bahria University. Downloaded on March 26,2024 at 05:02:29 UTC from IEEE Xplore. Restrictions apply.
Electronics and Communication Engineering
Sri Sairam Engineering College
Chennai, India
reshmajakramesh@ieee.org
v. Pandyaraj
Mechanical Engineering
Sri Sairam Engineering College
Chennai, India
pandyaraj.mech@sairam.edu.in
intensive applications were also part. Intelligent robots can
only detect fIre but also classify fIre and douse it with help of
deep learning [14]. Sensors like proximity, vision, and IR
helps to build a complete fIre detection model [15].
To reduce the burden undergoing by fIrefIghters and douse
the fIre quicker, the proposed system is introduced. Hence fIre
dousing will happen effectively, effIciently without any
manual errors. The main function ofthe proposed system is to
reach the destination autonomously and extinguish the fIre to
prevent great losses.
II. LITERATURE SURVEY
loT-based fIrefIghting robots have become quite common
due to their reliability and low cost. These bots can detect the
fIre type as well as the carbon-monoxide level and graphs can
be plotted for analysis. These bots help the authority to
communicate with the people at risk with help of a receiver
attached to the bot [1] [5]. Additional features like the usage of
a suction vacuum fan with a cylinder can remove all the smoke
present at a fIre accident location. So, that people trapped do
not have to suffer from smoke. People who are trapped can be
detected using an AI camera and Viola-Jones face detection
algorithm [2] or as an alternative, a PIR sensor can also be
used to fInd motion [4] but it has many limitations and lesser
accuracy because the PIR sensor works on the principle of
thermal energy generated due to motion. Detection of fIre can
be made possible using different methods, one such is using a
fIre sensor.
Autonomous navigation of robots plays a crucial role
which is equivalent to fIre detections and dousing. The best
end-user access to data is through a mobile application. Hence,
the manual controlling of a robot can be operated through a
mobile application [8]. So, either Bluetooth or WiFi can best
serve for autonomous navigation purposes [6]. FirefIghting
robots can be controlled using RF communication. The
principal moto of such robots is to douse fIre effectively. So,
to make sure water is sprayed in all directions, a servo motor
can be used [3].
ROS (Robot Control System) is an advanced technology
that can provide an effIcient and advanced controlling unit for
the robot. Data collected from the accident location needs to
be transferred to the ground station for analysis and take the
next step action. Cloud databases can store data and that can
be accessed by anyone around the world [7]. A well-structured
and designed fIrefIghting robot will have good climbing
ability, terrain adaptability and also gait uplift, angular speed,
and forward displacement [9].
The limitation of pure sensor-based robots is, physical
sensors have a very less range of detection. Thus, the rate of
inefficiency in its sole purpose will be more. A computerized
method of detection will have a wider range and be non-
sensitive towards the environment Thus, computer vision
methods such as color segmentation and correlation can be
used to detect fire [10]. Thermal imaging using statistical
texture also helps to detect fire [11]. An autonomous vehicle
can be designed with a few automation features like lane
detection, obstacle detection, traffic light detection along with
a web interface [12]. Even ultrasonic sensors or Lidar can be
used for obstacle detection and avoidance. Raspberry pi is the
most commonly used core processor in any detection model
that involves computer vision [16].
ill. PROPOSED SYSTEM
Automation is a technology to provide goods and services
with minimal human intervention. In today's world,
automation is utilized in seveml sectors. As said in section I,
a fIrefighting bot is introduced to reduce the risk taken by
fIrefighters. The proposed idea has three modules namely;
GPS Enabling Module, Navigation Module, Fire Fighting
Module. These modules are explained in detail in further sub-
sections. Figure 1 depicts the 3D diagram of the proposed
system. Figure 2 shows the flow of three different modules.
i
,;--
Fig. 1. 3D design of the system
GPS
Navigation
Enabllng H Module H Fighting
Module
Fig. 2. Flow diagram of the proposed system
A. GPS Enabling Module
It is the 1st module in the workflow. Since the bot is GPS-
guided, this module plays a vital role in navigation purposes.
The functioning of this particular module is based on, GPS
module (Ublox Neo 6m) and magnetometer (HMC5883L). In
addition to it, for the performance of the bot, components like
a motor driver (L293d), DC motors, Arduino UNO are added.
To set the waypoints for the bot mission planner is used.
Mission Planner is a community-supported and free
application that uses MAVLink2 protocol which is compatible
with Ardupilot software. Once the waypoint is fixed, the bot
starts from the fire station. A detailed explanation of the
navigation mechanism is explained in the Navigation module.
The working of the GPS (Global positioning system)
module is based on distance measurements using satellites.
The GPS module with the data received from the satellite
marks the longitude and latitude of the object. The
Authorized licensed use limited to: Bahria University. Downloaded on March 26,2024 at 05:02:29 UTC from IEEE Xplore. Restrictions apply.
magnetometer is an electronic compass that works on the
electromagnetic field emitted by the Earth. A magnetometer is
used to make the bot move in the desired direction pointed by
the GPS module. Figure 3 shows the flow diagmm of the GPS
enabling module. The bot will choose the path that is shortest
and with less traffic. This feature is supported by Google Maps
SDK. It is a ready-made package supplied by google through
which we can identif)r the traffic status ofthe road and shorted
path. Once the shortest path is identified bot will proceed with
it so that it can reach the destination quickly. The speed is also
determined by the gear mechanism which is explained
navigation module.
Location
and
Direction
GPS
t
GPS module
Magnetomete
----. Enabling ~
Mission
Planner
Module
Fig. 3. Flow diagram of the GPS enabling module
B. Navigation Module
Once the bot starts from the fIre station, it has to travel on
its own until it reaches the desired destination. For navigation
purposes of the bot, lane and obstacle detection are used. The
development of the model was through a machine leaming
algorithm with an image classification method, MobileNetV2.
This is a method based on CNN (Convolutional Neural
Networks) developed by Google. This is used in many visual
recognition tasks with improved performance and
enhancement to be more efficient. Figure 4 depicts the
computer vision model of lane and obstacle detection. Figure
5 shows the flow diagram of the navigation module.
. "'."
v.,.. . . 1I"
.
-.
Fire·
Module
Fig. 4. Lane and obstacle detection
Steps to construct the model:
1) Data Collection: The development of the model
begins with collecting data to train the model. The dataset for
lane detection contains photos of lanes taken in different
angles, directions, and times (concerning sun rays). The
external dataset of obstacle detection includes various
obstacles that a vehicle would meet during a road journey. It
incorporates, different types and models ofvehicles like cars,
buses, two-wheelers. Some other obstacles like pedestrians,
animals (dog, cow, cat, etc.), poles, barricades, etc. The Coco
dataset of Yolo architecture is used for traffic light detection
 r Image Data
I Location Identification ) I Data pre-processing)
IMission
Planner
I
II GPS
Magnetometer
I
I Noise
Reduction
Data
Scrul>l>lng
GoogleAPI
Fig. 5. Flow diagram of the navigation module
and also for obstacle detection. All the collected data will be
converted into image folders and log files to meet the
upcoming procedures.
2) Training: Training process involves three sub-
processes:
a) Data Pre-Processing: All the collected data, i.e., the
dataset will be pre-processed to get better accuracy and to
avoid data distortion. Grey scaling is the method followed
here for noise reduction. To avoid unwanted data being stored
in the dataset, data scrubbing is processed.
b) Direction Identification: Inverse binarization
method and pixel summation method is used to detect lane in
the proposed model. Based on the thresholding algorithm,
any color of the HFS code is distinguished into either black
or white concerning the threshold value chosen. Here due to
reverse binarization, the lane turns to be white with a pixel
rating of 225. As the next step pixel summation is done in
each column and, by comparing the adjacent column lane and
the curvature can be detected. Figure 6 shows the
diagrammatic representation of the pixel summation method.
For traffic light detection and obstacle detection yolov5, the
model is used.
PIXEL SUMMATION
...
..... ...
.......
RIGHT CURVE STRAIGHT LEFT CURVE
Fig. 6. Lane detection using pixel summation method
c) Gear Detection: To automate the gear mechanism
accelerometer, google API, and computer neural networking
are used. An accelerometer is a sensor that detects the speed
of the cycle. Using computer neural networking with image
Authorized licensed use limited to: Bahria University. Downloaded on March 26,2024 at 05:02:29 UTC from IEEE Xplore. Restrictions apply.
I
I Gear Detection )
Image
Folder fACC 'eromo\ 1 Tr fflc
Cl ction
,I
1
I Direction Identification I
II Lane
DeleCllon ,I 11°Detection
I>Stcal~1
Traffic Light
Delection
~
Bot Movement
processing and Google Maps SDK, the traffic status of the
road is identified. By integrating both the parameters and
connecting them with the Gear shaft of the bicycle gear 4
mechanism can be automated. This information is used to
activate a motorized shifter mounted on the rear wheel's hub
gear to change the gear ratio. Based upon the traffic status
speed of the bot will adjust automatically.
3) Implementation o/the Model: The model works based
on the video uploaded. The lane is detected first along with
obstacles if any. After this reprocessing will be performed,
including image resize, network conversion, preprocessing
input using MobileNetV2. The message sent to the servo
motor via Raspberry Pi 4 is depicted in figure 7. If the bot
detects any sort of obstacle, it will instruct the servo motor to
change the direction of the bot. The same method applies to
lane curvature too.
~~----~~
Fig.7. Infonnation sent to servo motor via Raspberry Pi 4.
C. Fire-Fighting Module
Once the bot reaches the destination, the flame sensors
placed in it will get activated if it detects flame. The proposed
system comprises multiple flame sensors and heat sensors for
better accuracy. Flame detectors consist of an electromagnetic
radiation receiver. The flame sensor can detect flame through
the electromagnetic radiation received from defined
wavelengths of the infrared spectrum. Heat sensor works on
the principle of temperature reading received through an
electric signal.
 So, as soon as the response is generated, the water pump
will be activated. The water pump sprinkles water through the
nozzle attached at the top of the bot that was connected
through a servo motor. The servo motor helps the nozzle to
reach 1800 to spray water. Based on the heat sensor's reading,
the pressure of the water being sprayed is adjusted. Initially,
only one bot is sent to the targeted location. Since it is a
compact bot, it can hold up to only 200 gallons of water. If the
water is not sufficient, then a single bot cannot douse the fire.
In that scenario, the GSM module present in the bot will send
an alert message to the fire station. Then fire station will send
a few more robots to solve the issue.
Fire-fighting bots are supposed to deal with high
temperatures. So, to protect the electronic components present
in the bot, thermal insulation is provided. 1st layer of thermal
insulation is coated with stainless steel. The thermal
conductivity of stainless is 16.2 W/m.K and, its melting point
is 1450 oC. 2nd layer is coated with polystyrene for ensuring
the safety of electronic components. Figure 8 shows the
picture of the prototype. Figure 9 shows the flowchart of the
frrefighting module.
using Keras, Tensorflow, MobileNet, serial, and OpenCV.
The list of hardware components used in the proposed system
is given in table 1.
TABLE!. TABLE TYPE STYLES
S.No Table Column Head
1 GPS Module CUblox Neo 6m)
2 GSMModule
3 Mallnetometer (HMC5883U
4 Motor driver (L293d)
5 Raspberry Pi V2 camera
6 Flame sensor
Through computer vision technology, lane detection and
obstacle detection models are performed with help of
Raspberry pi and pi camera. The obstacle and lane detector
models are accurate. The MobileNetV2 architecture is
effective computationally. Thus, the architecture makes the
model easier to establish a model for the embedded systems
(Raspberry Pi, etc.). The gmphical representation of the
accuracy and training ofthe system is depicted model in figure
10. Here the x-axis shows the epoch and, the y-axis shows the
accuracy/loss of data.

Training Loss and Accuracy

1.0·

0.8

c;
~ 0.6

~
\It 0.4-

\
.9

0.2 ~ - tr tn_lOSS
- val_loss

=-==------------ _
uain_IJcc
v '_ cc;
0.0 -
0.0 25 50 7 ..5 10.0 12.5 1..5.0 17 ..5
EpoCh #

Fig. 8. Fire Fighting bot prototype Fig. 10. Accuracy ofthe model (accuracy>loss)

Through the mission planner, waypoints can be marked.

Once the system knows about the desired location, it starts
from the fire station. In real-time, this is achieved with the help
of a GPS module that shows the location and magnetometer
True False
which shows the certain direction for the bot to navigate.
Figure 11 depicts the waypoints marked through the mission
planner. The bot moves exactly through the same waypoints
and reaches the location.

Fig.9. Flowchart ofthe fire-fighting module

IV. RESULTS AND DISCUSSION

This section briefs about the results obtained in various
Fig. 11. Waypoints marked through mission planner
modules of the system. The system comprises three modules
namely; the GPS Enabling module and the Fire-Fighting
Figure 12 depicts the representation of Google map SDK
module which are processed using Arduino UNO and Arduino
that shows the shortest path with respect to time, tmffic and
IDE software. Finally, the Navigation module is processed

Authorized licensed use limited to: Bahria University. Downloaded on March 26,2024 at 05:02:29 UTC from IEEE Xplore. Restrictions apply.
toll gate. The bot chooses the path accordingly such that it can
reach the destination with the least span of time. Thus, by
integrating Google map SDK and mission planner, the bot
works effectively in its navigation process.
As soon as the flame sensor detects the flame, the water
pump starts to spray water through the nozzle. As the IR flame
sensor is used, accuracy is interrupted when the sensor is
exposed to high sunlight. If the initial cannot douse the whole
fIre all alone. An alert message is sent to the respective fIre
station. The stainless steel and polystyrene used in the bot is
an excellent insulator, thus safeguarding the internal
environment and the components of the bot. Figure 13 shows
the graphical representation of the pursuit of the flame sensor
when it is activated and not activated. Since it is IR based
sensor, it shows binary output only. Here x-axis shows time in
seconds and the y-axis shows the digital output.
Fig. 12. Representation ofgoogle map SDK for choosing the shortest path
.,
(a) (b)
Fig.13. Graph plotted when a) flame sensor is activated and b) flame sensor
not activated
v. RESULTS AND DISCUSSION
To conclude, a fIre-fIghting system with a Pi camem, a fIre
sensor, a heat sensor, a magnetometer, a Bluetooth module, a
GPS, and a GSM module is successfully developed. In
addition, the remote function of the self-driving vehicle helps
the bot reach the destination without any human interference.
The system is efficient enough to reach the destination
accurately since it is trained with a wide dataset. The proposed
solution genemtes accurate results under certain performance
limitations. The results rely on both hardware and software,
which is a defmite and desirable advantage for such systems.
Future work plans to conduct demonstrations with various
deep learning algorithms and computer vision frameworks
Authorized licensed use limited to: Bahria University. Downloaded on March 26,2024 at 05:02:29 UTC from IEEE Xplore. Restrictions apply.
along with integrated LiDAR for object detection and lane
detection using Raspberry Pi to achieve a higher framemte.
Since the current system can only detect fIre with lesser
accuracy because of the usage of IRbased sensors, future
enhancement also includes detection and classifIcation of fIre
using machine vision technology.
REFERENCES
[1] Kanwar, Megha, and L. Agilandeeswari. "lOT based firefighting
robot." In 2018 7th International Conference on Reliability, Infocom
Technologies and Optimization (Trends and Future Directions)
(lCRITO), pp. 718-723. IEEE, 2018.
[2] Uaday, Md Aowrongajab, Md Nazmul Islam Shuzan, Saffan
Shanewaze, Rakibol Islam Raktb, and Hasan U. Zaman. "The design
of a novel multipwpose firefighting robot with video streaming
capability." In 2019 IEEE 5th International Conference for
Convergence in Technology (I2CT), pp. 1- 5. IEEE, 2019.
[3] Pillai, H. O. C. "Fire Fighting Robot." (2021).
[4] Wu, Changzhong, Fan Ge, Guangchao Shang, Mingpeng Zhao, Guitao
Wang, Hengshuai Guo, and Liang Wu. "Design and Development of
Intelligent Fire-fighting Robot Based on STM32." In Journal of
Physics: Conference Series, voL 1748, no. 6, p. 062019. lOP
Publishing, 2021.
[5] Sabhanayagam, T., T. Senthil Kumar, M. Narendra, and J. Martin
Sahayaraj. "Internet Connected Modem Fire Fighting Robot." In
Journal ofPhysics: Conference Series, voL 1964, no. 4, p. 042088. lOP
Publishing, 2021.
[6] Murad, Ahmed, Oguz Bayat, and Hamzah M. Marhoon.
"Implementation of rover tank firefighting robot for closed areas based
on arduino microcontroller." Indonesian Journal of Electrical
Engineering and Computer Science 21, no. 1 (2021): 56-63.
[7] Gan, Zhijian, Guofang Huang, Jing Zhang, Xiaoming Liu, and Chao
Shan. "The Control System and Application Based on ROS Elevating
FireFighting Robot." In Journal of Physics: Conference Series, voL
2029, no. 1, p. 012004. lOP Publishing, 2021.
[8] Yashaswini, R., H. V. MalikaJjuna, P. Bharathi, A. Prajwal, and B.
Jagadeesh. "Fire Fighting Robot Using Various Wi-Fi Module: A
Review." International Journal of Progressive Research in Science and
Engineering 2, no. 7 (2021): 62-64.
[9] Guo, Anfu, Tao Jiang, Junjie Li, Yajun Cui, Jin Li, and Zhipeng Chen.
"Design of a small wheel-foot hybrid firefighting robot for infrared
visual fire recognition." Mechanics Based Design of Structures and
Machines (2021): 1-20.
[10] Rangan, M. K., S. M. Rakesh, G. S. P. Sandeep, and C. Sharmila Suttur.
"A computer vision-based approach for detection of fire and direction
control for enhanced operation of firefighting robot." In 2013
International Conference on Control, Automation, Robotics and
Embedded Systems (CARE), pp. 1-6. IEEE, 2013.
[11] Kim, Jong-Hwan, and Brian Y. Lattimer. "Real-time probabilistic
classification of fire and smoke using thermal imagery for intelligent
firefighting robot." Fire Safety Journal 72 (2015): 40-49.
[12] Hossai, Mohammad Rubaiyat Tanvir, Md AsifShahjalal, and Nowroz
Farhan Nuri. "Design of an loT based autonomous vehicle with the aid
of computer vision." In 2017 International Conference on Electrical,
Computer and Communication Engineering (ECCE), pp. 752-756.
IEEE,20l7.
[13] Pransky, Joanne. "GeoffHowe, senior vice president, Howe and Howe,
Inc., a subsidiary ofTextron Systems; co-pioneer ofrobotic firefighting
technologies, including Thermite™ firefighting robots." Industrial
Robot: the international journal of robotics research and application
(2021).
[14] Dhiman, Amit, Neel Shah, Pranali Adhikari, Sayali Kumbhar, IndeJjit
Singh Dhanjal, and Ninad Mehendale. "Firefighting robot wi1h deep
leaming and machine vision." Neural Computing and Applications
(2021): 1-9.
[15] Gao, Shang, Zhiyang Zhang, Zihan Zhao, and Mohsin M. Jamali.
"Vision and infra-red sensor-based firefighting robot." In 2018 IEEE
6lst International Midwest Symposium on Circuits and Systems
(MWSCAS), pp. 873-876. IEEE, 2018.
[16] Kumer, SV Aswin, LSP Sairam Nadipalli, P. Kanakaraja, K. Sarat
Kumar, and K. Ch Sri Kavya. "Controlling the autonomous vehicle
using computer vision and cloud server." Materials Today: Proceedings
37 (2021): 2982-2985