IEEE INFOCOM 2021 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) | 978-1-6654-0443-3/21/$31.00 ©2021 IEEE | DOI: 10.1109/INFOCOMWKSHPS51825.2021.
9484547 IEEE INFOCOM 2021 Demo
DeepSafe: A Hybrid Kitchen Safety Guarding System with
Stove Fire Recognition Based on the Internet of Things
Lien-Wu Chen and Hsing-Fu Tseng
Department of Information Engineering and Computer Science, Feng Chia University, Taichung 407, Taiwan
Email: lwuchen@fcu.edu.tw
Abstract—This paper designs and implements a deep learning
based hybrid kitchen safety guarding system, called DeepSafe,
using embedded devices and onboard sensors to detect abnormal
events and block gas sources in time through the Internet of
Things (IoT). In the sensing mode, the DeepSafe system can
prevent the kitchen from fire/explosion disasters by detecting
gas concentration, recognizing fire intensity, and estimating
vibration levels. In the control mode, the DeepSafe system can
automatically block the gas source as detecting an abnormal
event, remotely monitor the kitchen status via real-time streaming
videos, and manually turn off the gas source using a smartphone
as necessary. To accurately recognize the intensity of stove fire
and detect abnormal fire intensity, deep learning based fire
recognition methods using conventional and densely connected
convolutional neural networks are developed to further improve
the recognition accuracy of DeepSafe. In particular, the prototype
consisting of an Android based APP and a Raspberry Pi
based IoT device with the gas detector, image sensor, and 3-
axis accelermeter are implemented to verify the feasibility and
correctness of our DeepSafe system.
Keywords: Automatic Gas Blocking; Deep Learning; Inter-
net of Things; Mobile Device; Smart Kitchen.
I. I NTRODUCTION
The rapid development of wireless communications and
MEMS technologies has made the Internet of Things (IoT)
pervasive, which can explore the sensing capability of IoT
devices and the Internet connectivity among IoT devices to
enable smart environments [1], such as smart house, smart
building, smart community, smart city, etc. In addition, the
connected mobile devices (e.g., smartphones or tablets) can
be used to monitor environmental information and perform
necessary control through remote IoT devices. In particular,
the kitchen is an important area of the house, where family
members can cook meals and store ingredients. Based on
IoT technologies, recent studies for smart kitchens to detect
emergency events have been proposed using human activity
recognition [2], gas concentration sensing [3], [4], [5], [6],
and house fire detection [4], [5], [6].
In this work, we design and implement a hybrid kitchen
safety guarding system, called DeepSafe, based on the IoT.
DeepSafe can provide cooking safety in the kitchen as using
stove fire. First, we use a gas sensor to detect gas leakage
in a real-time manner. Second, we design deep learning
based methods to recognize abnormal fire through an image
sensor. Third, we employ a 3-axis accelerometer to determine
vibration levels. The gas source can be automatically closed as
gas leakage, abnormal fire, or great earthquake occurs. Finally,
the sensing data and surveillance videos of the kitchen are
immediately transmitted to the DeepSafe server. Through the
978-1-6654-0443-3/21/$31.00 ©2021 IEEE
Authorized licensed use limited to: Chaitanya Bharathi Institute of Tech - HYDERABAD. Downloaded on February 09,2023 at 08:45:00 UTC from IEEE Xplore. Restrictions apply.
dedicated webpage and smartphone APP of DeepSafe, kitchen
owners can instantly monitor the current status of the kitchen
and manually shut down the gas source as necessary.
According to the relevant literature we studied, DeepSafe
is the first system for hybrid kitchen safety guarding that
provides the following features: (1) DeepSafe integrates the
gas sensor to properly detect gas leakage and employs the 3-
axis accelerometer to instantly determine the vibration level
for avoiding fire/explosion disasters. (2) DeepSafe adopts the
developed deep learning based methods using conventional
and densely connected convolutional neural networks to accu-
rately recognize abnormal fire. (3) DeepSafe provides remote
surveillance and control functions to owners for monitoring
the kitchen environment and controlling the gas source any-
time and anywhere. In particular, an Android-based prototype
consisting of the IoT device, diverse sensors, dedicated server,
and smartphones is implemented to verify the feasibility and
correctness of our DeepSafe system.
II. S YSTEM D ESIGN
Fig. 1a shows the system architecture of DeepSafe. On
the kitchen side, an IoT device consisting of the Raspberry
Pi development board, gas detector, image sensor, 3-axis ac-
celerometer, and servo motor is installed between the gas stove
and gas pipeline to periodically report sensing data, detect gas
concentration, recognize fire intensity, and estimate vibration
levels. Once detecting any abnormal event (i.e., excessive gas
concentration, large fire intensity, or high vibration level),
DeepSafe automatically turns off the gas source through the
servomotor and instantly sends the warning message of the
detected event to the group of registered kitchen owners.
On the owner side, a smartphone/tablet is used to remotely
monitor the current status of the kitchen, instantaneously
receive the warning messages of the installed IoT device, and
manually turn off the gas source as necessary (before the IoT
device detects an abnormal event in the kitchen).
Fig. 1b shows the operation flow of DeepSafe for abnormal
event detection, automatic gas blocking, and instant warning
notification. The gas stove to be monitored is equipped with
the IoT device of DeepSafe (which is in the sensing mode
by default). In the sensing mode, the gas concentration Cgas ,
fire intensity Ifire , and vibration level Lvib of the kitchen
environment are kept sensing and recognizing through gas
parts per million detected by the gas detector, streaming video
captured by the image sensor, and acceleration information
obtained from the 3-axis accelerometer. There are three phases
in the recognition process of stove fire intensity that includes
 IEEE INFOCOM 2021 Demo
(a)
Fig. 1. (a) System architecture, (b) operation flow, and (c) system implementation of DeepSafe.
fire image classification (i.e., labeling), feature map extraction
(i.e., training), and intensity similarity determination (i.e.,
recognizing).
To accurately recognize stove fire intensity, we develop deep
learning based fire recognition methods using conventional and
densely connected convolutional neural networks to further
improve recognition accuracy. Our developed fire recognition
model consisting of a number of dense blocks, where each
dense block consists of five layers and each layer has three
consecutive operations including Batch Normalization (BN),
Rectified Linear Unit (ReLU), and Convolution (Conv). The
transition layer is first adopted to reduce the number of feature
maps in the output of the last layer, and then the convolutional
and pooling layers are used for the final output of the dense
block.
In the control mode, DeepSafe can automatically block
the gas source as detecting an abnormal event, where the
event-related warning message is simultaneously sent to the
owner group. In addition, the kitchen owner can use his/her
smartphone to remotely watch the real-time streaming video
in the kitchen. Furthermore, even DeepSafe does not detect
any abnormal event yet in the kitchen, the gas source can be
manually turned off through using a smartphone as necessary.
III. S YSTEM I MPLEMENTATION
We have developed an Android-based DeepSafe prototype
consisting of a Raspberry Pi embedded system (as IoT device),
NVIDIA Jetson Nano (as external GPU device), a note-
book computer (as data exchange server), and smartphones
(as owner platform). The Raspberry Pi embedded system is
connected with the MQ2 gas sensor [7], ADXL345 3-axis
accelerometer [8], and Logitech USB camera for abnormal
event detection. NVIDIA Jetson Nano is additionally attached
to perform real-time fire intensity recognition in the IoT device
(i.e., edge computing version). In particular, fire intensity can
be recognized in the DeepSafe server (i.e., cloud computing
version) if considering budget limitation without using any
externa GPU device. The servo motor SG90 [9] is used to
automatically block the gas source (as detecting an abnormal
event) and manually turn off the stove fire in a remote manner
(as necessary).
The IoT device of DeepSafe is installed on the gas stove
in a real kitchen environment, as shown in Fig. 1c(a), where
the servo motor can control the gas switch that is connecting
the gas stove to gas pipeline, as shown in Fig. 1c(b). The
Authorized licensed use limited to: Chaitanya Bharathi Institute of Tech - HYDERABAD. Downloaded on February 09,2023 at 08:45:00 UTC from IEEE Xplore. Restrictions apply.
(b) (c)
Message Queuing Telemetry Transport (MQTT [10]) protocol
is adopted to collect and distribute the sensing data of current
gas concentration, fire intensity, and vibration levels for re-
mote monitoring and control using smartphones, as shown in
Fig. 1c(c). Once detecting an abnormal event (e.g., gas leak-
age), the warning message of the detected event is instantly
sent to the owner’s smartphone, as shown in Fig. 1c(d). In the
DeepSafe system, MQTT is a lightweight, publish-subscribe
network protocol over TCP/IP that transports sensing data
between onboard sensors and mobile devices.
For desktop demonstration, cigarette lighters and fire pic-
tures can be used to simulate gas leakage and fire events in a
limited space, such as in a conference room, to demonstrate
our DeepSafe system, respectively. In addition, the abnormal
vibration level can be detected through the 3-axis accelerome-
ter by flapping/shaking the IoT device. Furthermore, to provide
remote monitoring, the real-time video captured by onboard
camera can be streamed to the owner’s smartphone. In par-
ticular, the owner can manually turn on/off the gas switch
controlled by the servo motor through his/her smartphone.
IV. ACKNOWLEDGMENTS
This research is supported in part by MOST under Grant No.
106-2221-E-035-019-MY3 and 109-2221-E-035-062-MY3.
R EFERENCES
[1] L.-W. Chen, T.-P. Chen, H.-M. Chen, and M.-F. Tsai. Crowdsourced
Children Monitoring and Finding with Holding Up Detection Based on
Internet of Things Technologies. IEEE Sensors Journal, 19(24):12407-
12417, Dec. 2019.
[2] J. Monteiro, R. Granada, R. C. Barros, and F. Meneguzzi. Deep Neural
Networks for Kitchen Activity Recognition. International Joint Confer-
ence on Neural Networks (IJCNN), May 2017.
[3] J.-M. Sun et al. Design and Implementation of Intelligent Kitchen System
Based on Internet of Things. International Conference on Smart Grid and
Electrical Automation (ICSGEA), June 2018.
[4] S. Jamadagni, P. Sankpal, S. Patil, N. Chougule, and S. Gurav. Gas
Leakage and Fire Detection using Raspberry Pi. International Conference
on Computing Methodologies and Communication (ICCMC), Mar. 2019.
[5] S. Shrestha, V. P. K. Anne, and R. Chaitanya. IoT Based Smart Gas
Management System. International Conference on Trends in Electronics
and Informatics (ICOEI), Apr. 2019.
[6] M. Ralevski and B. R. Stojkoska. IoT Based System for Detection
of Gas Leakage and House Fire in Smart Kitchen Environments. 27th
Telecommunications Forum (TELFOR), Nov. 2019.
[7] Analog Gas Sensor (MQ2). http://www.energiazero.org/arduino sensori/
Analog Gas Sensor MQ2.pdf.
[8] 3-Axis Digital Accelerometer (ADXL345). https://www.analog.com/
media/en/technical-documentation/data-sheets/ADXL345.pdf.
[9] Servo Motor SG90. http://www.ee.ic.ac.uk/pcheung/teaching/DE1 EE/
stores/sg90 datasheet.pdf.
[10] Message Queueing Telemetry Transport (MQTT). http://mqtt.org/