See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/352600525

Development of an IoT based Smart Baby Monitoring System with Face

Recognition

Conference Paper · May 2021

DOI: 10.1109/AIIoT52608.2021.9454187

CITATIONS READS

10 1,307

5 authors, including:

H.M Ishtiaq Salahin Quazi Rubayet Anjum Joy

ADA Asia North South University
1 PUBLICATION 10 CITATIONS 1 PUBLICATION 10 CITATIONS

SEE PROFILE SEE PROFILE

Fatima Tuz Zuhra Aparna Riasat Khan

North South University North South University
1 PUBLICATION 10 CITATIONS 66 PUBLICATIONS 356 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Riasat Khan on 21 September 2022.

The user has requested enhancement of the downloaded file.

 Development of an IoT based Smart Baby
Monitoring System with Face Recognition
H. M. Ishtiaq Salehin Quazi Rubayet Anjum Joy Fatima Tuz Zuhra Aparna
Department of Electrical and Computer Department of Electrical and Computer Department of Electrical and Computer
Engineering Engineering Engineering
North South University North South University North South University
Dhaka, Bangladesh Dhaka, Bangladesh Dhaka, Bangladesh
hyder.salahin@northsouth.edu quazi.rubayet@northsouth.edu fatima.aparna@northsouth.edu

Ahnaf Tahmid Ridwan Riasat Khan

Department of Electrical and Computer
Engineering
North South University
Dhaka, Bangladesh
tahmid.ridwan@northsouth.edu
Department of Electrical and Computer
Engineering
North South University
Dhaka, Bangladesh
riasat.khan@northsouth.edu

Abstract— Nowadays, people have a very busy and hectic
life; thus, taking care of an infant constantly is tough. Especially
for working parents, a modernized IoT-enabled baby
management system will be beneficial. In these modern times,
most children are taken care of by their grandparents,
housemaids, or babysitters during the daytimes. In this work,
we are using a very efficient and user-friendly technology to
implement automatic swinging of the baby bassinet with sound
detection of the baby crying using sound sensor and playing
lullaby through speakers. The humidity sensor has been used to
know the diaper’s moisture level, and notifications have been
sent to parents with certain conditions through mobile calls and
text messages. A webpage using HTML and CSS has been
developed, where parents/guardians can supervise the baby in
real-time. Finally, the system will detect if the baby is in the
cradle using the face recognition technique. This exciting feature
is implemented by using a Raspberry Pi 4 (Model B), which is
equipped with a Pi camera that will also offer the parents a live-
stream option. The proposed baby monitoring system is believed
to be an improved technology for new and working parents and
colossal help and riddance of unrequired tensions.
Keywords— Arduino Mega, Baby Monitoring System, Cloud
Server, Face Recognition, GSM module, Raspberry Pi, Real-Time
Streaming.
I. INTRODUCTION
Life in the 21st century is very fast-paced. We get almost
everything done with a few clicks, swipes, and taps due to
faster connectivity and internet. We can observe the intense
use of smartphones and the internet in every aspect of life.
Another common fact of this modern era is dual-earner
families, where both parents work simultaneously [1]. This
scenario is a positive achievement as we are achieving higher
incomes and gender equality, but this makes babysitting a
tough job for working parents. It becomes hard to manage
work and maintain family and take care of the children [2].
Thus, continuously monitor the baby in the home becomes
impossible for the working parents. A solution to this
problem can be hiring a nanny or domestic work to meet the
required needs of the baby, but this also raises the question of
safety issues for the baby. In order to bridge this gap between
parents and baby, an IoT-based smart real-time baby
monitoring system has been proposed in this work. The
parents can use the proposed monitoring system to take good
care of their baby. The proposed monitoring system mainly
consists of:
• Automatic swinging of the cradle at the detection of the
baby’s crying sound. Subsequently, the system starts a sweet
melody and sends a notification to the parent smartphone.
Sends a text message to parent smartphone if the baby does
not stop crying after a specific time.
• Checking whether the mattress or diaper is wet or not.
• Sends a text message (via GSM module) to the parent’s
smartphone if the baby is not detected in the cradle.
• Real-time video streaming of the baby’s room to parent’s
smartphone.
• Detection of the baby in the room using the face
recognition technique. If the baby is not detected in the cradle
(using Raspberry Pi and image processing), the system will
send notifications to the parents.
All the notification information can be directly sent to the
parent’s smartphone through the proposed system. As
smartphones are available and almost everyone owns a
smartphone, parents can check on their baby anytime from
anywhere.
In the next following paragraphs, some of the research
papers related to this work have been discussed briefly. For
instance, in [3], the authors designed a baby monitoring
system titled “IoT-BBMS,” which used Arduino IDE software
with MQTT protocol servers and various temperature and
humidity sensors. They employed an electrical box consisting
of multiple pieces of equipment mentioned above and
connected them with the cradle with an appropriate power
supply. An MD10-POT 10 Amp AC to DC power supply was
utilized to provide power to the NodeMCU microcontroller
for monitoring and controlling the sensors. The temperature
and humidity sensing device, i.e., DHT22, was mounted to the
side of the designed smart cradle. A condenser MIC was
attached to detect the crying of the infant as the sound sensor.
In this paper, the Adafruit MQTT IoT server is used to activate
the connected actuators, e.g., swinging motor, musical toys,
and mini fan. The proposed device was also connected with
the server, which helps the system to send notifications and
start swinging the cradle.
In [4], T. Wu and P. Chen proposed an independent baby
care system with an Arduino-based system to check on the
heartbeat, temperature, and humidity of a premature baby. The
Arduino microcontroller is interfaced with input from the
temperature sensor and heartbeat sensor. It sends that data
through a Wi-Fi module to a web-based server to show
constant real-time data from different sensors regarding the

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

baby. If heartbeat or temperature is above or below a preset
value, it will be displayed instantly on the website, and the
website will command the buzzer and bulb accordingly.
Finally, the proposed baby monitoring design will make it
easier to efficiently supervise the affected baby incubator if
more than one incubator is present.
M. P. Joshi and D. C. Mehetre proposed a mobile
application-based system that shows a constant update of a
baby placed in their smart cradle system [5]. The system can
detect if the mattress for the baby is wet or not by using a
moisture sensor. It can identify the baby’s crying with a noise
sensor and contains a camera for live-streaming to the app.
The cradle also swings automatically when crying is detected
with the help of a driver circuit. Finally, the authors have also
proposed using IR cameras for night vision as a future scope
for advancement.
The paper [6] is based on taking care of multiple babies,
which includes a database system. The proposed system
measures various physiological signals of the baby, for
example, blood oxygen saturation, heart rate. The device uses
temperature sensing, humidity sensing circuits, and a sound
capture circuit with a vibration sensor, determining the
baby’s crying. In PC, this system uses a graphical
programming language to construct a monitoring screen.
Finally, the obtained data are sent through a wireless system
in real-time to the concerned persons.
In essence, there are several research papers on baby
monitoring systems, but to the best of our knowledge, none
of them have implemented integrated facial recognition
capabilities to identify a baby uniquely; instead, they have a
video streaming feature at best. Our proposed system can not
only stream the video of the baby 24/7 but also uniquely
identify multiple babies, which makes our proposed system a
perfect option for parents with multiple kids. Also, most
monitoring systems consist of a camera and a receiving
device. Without the receiving device, it is impossible to keep
track of the baby. Our proposed system covers this exciting
feature by converting our mobile phone into a receiving
device so parents can check on their little ones with just a few
clicks and swipes as we carry our mobile phone everywhere
with us.
The following section describes the hardware and
software implementation of the proposed system in detail.
Section 3 illustrates the experimental results of the proposed
automatic device with figures. Lastly, Section 4 concludes the
paper.
II. PROPOSED SYSTEM
In this section, we have described the hardware tools we
have used to design and implement the project. Next, the
operational flowcharts and the final hardware circuit have
been illustrated.
A. Hardware Tools/Components
• Raspberry Pi 4B Model: Raspberry Pi is a very useful and
popular microcontroller, which has been used to detect the
baby’s face in our proposed system [7]. This is the latest
model, which has (BCM2711B0) quad-core ARM processor;
dual display supports up to the resolution of 4K (3840×2160).
It can be used as media centers, file servers, routers, retro
game consoles, network-level ad blockers, etc.
• Raspberry Pi Camera: The Pi camera is a portable,
lightweight camera that supports Raspberry Pi [7]. We have
used a Raspberry Pi camera rev1.3 in this work. It
communicates with the Raspberry Pi Using the MIPI camera
serial interface protocol. In our project, we have connected the
Pi camera with Raspberry Pi, and it has been used for real-
time facial recognition through the OpenCV Python library.
• Arduino Mega 2560 REV3: In the proposed baby
monitoring system, we have used Arduino Mega 2560 REV3
microcontroller [8]. Here, Arduino Mega is the main
motherboard for most of the features’ operations, e.g., indicate
wetness of diapers, measure temperature, monitor if the baby
is crying or not. All of the operations will be done by different
sensors, and we will be collecting data and processing the data
through the Arduino microcontroller and sending the
notifications to parents/guardians via voice calls and text
messages.
• Moisture Sensor: A moisture sensor is a module that is
used to check whether the diaper or mattress of the baby is wet
or not. In this work, we have used a soil moisture sensor for
this purpose.
• Ultrasonic Sensor: An ultrasonic sensor is a module that
measures the distance of a specific object by emitting
ultrasonic waves. We have used the ultrasonic HC-SR04
sensor in this paper to detect the child’s presence in the cradle.
• Sound Sensor: A sound sensor is a module that detects the
intensity of the sound and converting it to electrical signals.
We have used the LM393 sound sensor to recognize the
baby’s crying.
• Servo Motor: A servo motor is a module that can rotate an
object with great accuracy. We have used the mini servo motor
SG90 to swing the cradle when the proposed system detects
the crying of the baby.
• Temperature-humidity Sensor: A Temperature-humidity
sensor is a module that measures the surrounding air and gives
a digital signal. We have used a DHT22 temperature-humidity
sensor to measure the temperature of the surroundings of the
child.
• GSM Module: A GSM module is a device or a chip that is
used to establish communication between a smartphone and a
machine. We have used a SIM800L GSM module to initiate a
voice call and send text messages to the parent’s phone.
• DC-to-DC Buck Converter: A DC-to-DC converter helps
the voltage to step down so that the sensors which work in low
voltages are not burnt. We have used an LM2596 DC-DC
buck converter step-down power module, which provides
output voltages of 1.25V–35V.
• LCD Display: An LCD is an electronic display that uses
liquid crystal to produce a visible image. We have used an
LCD 16×2 display.
• Relay Module: The relay module is a device used for
remote device switching. We have used a Songle single-
channel relay module.
After the entire system has been successfully connected
and turned on, the various sensors will constantly capture data
to detect any anomaly. The message on the LCD screen,
“Monitoring Baby,” confirms to the user that the system is
running correctly. The sensors used in the system start to send
the signal to the Arduino Mega microcontroller to call the We will be keeping the data of faces in the extensible
parents or alert them via text message. The mic sensor markup language “XML” files so that the facial recognition
constantly picks up noise from the baby crib. It has been coded results can easily be sent to the live feed webpage. Next, we
to send the signal to the Arduino when it picks up a sound have developed a webpage using HTML and CSS, where
ranging from 110–135 decibels. Thus, whenever a crying parents/guardians will be able to see whether the baby is in the
sound is detected, the mic sensor alerts the Arduino, and the cradle or not. In this work, we have used the flask framework
Arduino plays a lullaby to calm the baby and sends a text to access the resources of the website. This is a lightweight
message to the parent’s cellular device to alert them of the Python framework that works perfectly with our proposed
current situation. project. Fig. 2 illustrates the working sequences of the face
recognition part of the proposed system.
The moisture sensor is used to detect if the diaper of the
baby is wet or fresh. The sensor shorts itself and passes a small
electric current within itself when it comes into contact with
water. When the short-circuit occurs, the microcontroller gets
the signal that the baby’s diaper is wet and immediately
initiates a phone call alert to the parent’s mobile phone.
The ultrasonic sensor is utilized within the system for the
sole purpose of detecting whether the baby is in the cradle or
not. The ultrasonic sensor constantly sends out ultrasonic
sound waves. So, when the baby is not present on the crib, the
Arduino gets a signal from the ultrasonic sensor and initiates
a phone call to the parent’s phone. For this sensor, the baby is
considered as an obstacle, and when the obstacle is removed,
the sensor sends a signal to the Arduino. Fig. 1 illustrates the
operational flowchart of the proposed baby monitoring
system.

Fig. 2. Working flowchart of the face recognition system.

III. RESULTS AND DISCUSSION

In this section, the results of the proposed smart baby
monitoring and face recognition device has been presented.

Fig. 1. Working flowchart of the proposed hardware device.

B. Software Tools/Algorithms
• OpenCV Python Library: In this paper, CV2 and Face
recognition, these two OpenCV Python libraries have been
installed for the face-recognition purpose. Also, the NumPy
Python library has been installed for accessing multiple data
and handling image data by converting them into matrices.
This operation helps us to compare images from the live feed
and source folder.
Fig. 3. Hardware connections of the proposed system.
First, we have successfully connected all the components
of the hardware part. In Fig. 3, we have shown the connection
between Arduino Mega and all the sensors. Initially, we have
applied 12V input to the Arduino. Later we have taken a
parallel connection between Arduino and DC-DC buck
converter. Using buck converter, we have stepped down the movement, and the baby’s cradle is rocked automatically,
voltage from 12V to 5V as all sensors and other modules can which has been controlled by the pulse with modulation
operate between 3.30-5V except the GSM module, which will (PWM) signals.
operate at 4-4.2V. Then, we have created two bus bars. From
the bus bar, we have given connections to all sensors except
the GSM module. And, we gave contact to the GSM module
from the second bus bar.
We have used a wet sensor to know the diaper’s moisture
by creating a slight electric connection when it is in touch
with wetness or water, and the circuit gets short, so it sends a
signal to Arduino. According to Fig. 4, the wet sensor of the
baby’s diaper is working perfectly.

Fig. 4: Illustration in the LCD screen of the baby is wet.

Suppose the baby was lying in bed and suddenly falls

from the cradle or is out of the cradle. When the system
responds from the ultrasonic sensor that the baby is not
present in the cradle, the system will immediately call parents
through the GSM module. Fig. 5 shows the LCD screen
Fig. 6. Demonstrations of the LCD screen and text message to the parents
display and initialization of a call when the baby is not when the baby’s crying is detected.
present in the cradle.
If the system detects the crying, it will send a message to
the parent’s mobile phone and simultaneously play a lullaby
selected by the parents so that the baby calms down. The
lullaby will be played by the speaker connected to the system.
It is integrated with stereo compatibility, so the parents can
connect the system to a bigger sound unit if they want to do
so in the future. The relay module is used to turn on the music
system when the baby is crying. The sensor detects the baby’s
cry and then signals the relay module to turn on the music for
the baby to stop crying. And, after a particular time, it will
turn off automatically. Fig. 7 confirms that the proposed
system has detected the baby’s cry, and the lullaby has been
played.

Fig. 7. Lullaby is played when the baby’s cry has been detected.
Fig. 5. LCD screen display and initiate of a call when the baby is not
present in the cradle. The temperature sensor is used to monitor the temperature
of the place where the baby is located. A higher or lower
The sound sensor does the detection of the baby’s cry. temperature is not beneficial to a baby, and this sensor will
Here, the digital output pin is for the boolean outputs like 1 allow us to monitor the cradle’s temperature, which has been
or 0, which determines whether the baby is crying or not. shown in Fig. 8.
Analog output is the measurement of the crying and time. We
are taking values from digital analog to get a text to the
parents following the range. Fig. 6 demonstrates the LCD
screen and text message to the parents when the proposed
device detects the baby’s crying.
Servo Motor is a motor that works with only predefined
angle values as it creates motion to swing the cradle based on Fig. 8. The temperature of the bassinet of the baby.
the range of angles. In this work, the servo motor initiates the
 Finally, we have developed a webpage using HTML and
CSS, where parents/guardians will be able to see whether the
baby is in the cradle or not. We have stored a picture of the
baby in the system’s profile folder. The system will take the
image from the profile folder and convert it to a matrix, and
then the camera will detect the baby’s face from the video
feed coming from the Pi camera and match it with the stored
picture. As the picture can be large and small, we will resize
the baby’s picture, which we will get from the camera
through the system. The image preprocessing will help the
system to process the face recognition faster.
The most exciting feature of the proposed system is that
it will allow the user to detect multiple babies simultaneously.
The system initially converts all the stored images into
matrices for cross-checking. When the live feed starts
broadcasting, the system will detect the face from the live
feed and decode it to cross-check with the pictures stored in
the system’s profile folder. If the picture captured by the
camera matches the image given in the system, the system
will show the name of the baby below the picture, which has
been demonstrated in Fig. 9. Otherwise, it will show
“unknown” written under the detected face on the web page,
as illustrated in Fig. 10.
Fig. 9. The proposed system detects the stored/known baby’s image.
Fig. 10. The proposed system detects an unknown image.
View publication stats
IV. CONCLUSIONS
Looking after and monitor the babies is hard sometimes
in this modern time for working couples. In this paper, we
have built an intelligent IoT-based baby monitoring system
using Raspberry Pi and Arduino Mega microcontrollers.
Using this system, parents/guardians will be able to do their
work outside, and besides, they will also be notified about
their babies at home. The baby’s crying, wet diapers, and
presence in the bassinet have been monitored by the Arduino.
The proposed system detects whether the baby is in the cradle
in real-time using the Raspberry Pi and Pi camera. It can
identify the baby with the face recognition technique. The
existing system can be upgraded by introducing more
complex sensors and advanced face recognition techniques in
the future.
REFERENCES
[1] K. Boye, “Dual-earner couples/dual-career couples,” In: Michalos
A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research.
Springer, Dordrecht, 2014.
[2] A. Gragnano, S. Simbula, and M. Miglioretti, “Work–life balance:
weighing the importance of work–family and work–health
balance,” International Journal of Environmental Research and Public
Health, vol. 17, no. 3, pp. 907, 2020.
[3] W. A. Jabbar, H. K. Shang, S. N. I. S. Hamid, A. A. Almohammedi, R.
M. Ramli and M. A. H. Ali, “IoT-BBMS: internet of things-based baby
monitoring system for smart cradle,” IEEE Access, vol. 7, pp. 93791–
93805, 2019.
[4] T. Wu and P. Chen, “Baby care system design for multi-sensor
applications,” International Symposium on Intelligent Signal
Processing and Communication Systems (ISPACS), Taipei, Taiwan,
pp. 1–2, 2019.
[5] M. P. Joshi and D. C. Mehetre, “IoT based smart cradle system with an
android app for baby monitoring,” International Conference on
Computing, Communication, Control and Automation (ICCUBEA),
Pune, India, pp. 1–4, 2017.
[6] M. Koli, P. Ladge, B. Prasad, R. Boria and N. J. Balur, “Intelligent
baby incubator,” Second International Conference on Electronics,
Communication and Aerospace Technology (ICECA), Coimbatore,
India, pp. 1036–104, 2018.
[7] M. N. I. Suvon, R. Khan and M. Ferdous, “Real Time Bangla Number
Plate Recognition using Computer Vision and Convolutional Neural
Network,” IEEE 2nd International Conference on Artificial
Intelligence in Engineering and Technology (IICAIET), Kota
Kinabalu, Malaysia, pp. 1–6, 2020.
[8] Y. Kurnia and J. Sie, “Prototype of Warehouse Automation System
Using Arduino Mega 2560 Microcontroller Based on Internet of
Things”, bit-Tech Journal, vol. 1, no. 3, pp. 122–128, May 2019.
[9] M. Khan, S. Chakraborty, R. Astya and S. Khepra, “Face Detection and
Recognition Using OpenCV,” International Conference on
Computing, Communication, and Intelligent Systems (ICCCIS),
Greater Noida, India, pp. 116–119, 2019.