2020 International Conference for Emerging Technology (INCET)
Belgaum, India. Jun 5-7, 2020
Smart Refrigerator: An IOT and Machine learning
based Approach
Dr Mallikarjun B C.1, Harshitha S2, Harshita B K3, S Bhavani4, Sonakshi Tarwey5
1,2,3,4,5
Dept. of Electronics and Communication Siddaganga Institute of Technology, Tumkur, India
1
mallikarjun.cp@gmail.com, 2harshithas.1si16ec027@gmail.com, 3harshitabk.1si16ec026@gmail.com, 4bhavani.1si16ec082@gmail.com,
5
sonakshi.1si16ec026@gmail.com
Abstract—The enhancement in technology and busier
schedule of human beings, tend to use smarter devices which can
make life easier. The smart kitchen appliances at home lead to a
healthy and prosperous lifestyle. Refrigerator is one such
appliance. The proposed system monitors the ingredients inside it
and notifies the user about its quantity of items remotely through
an android application. The sensor’s data which is collected is
processed by the control module and is sent to the cloud so that it
can be retrieved in the user’s android application. With quantity
monitoring of items, the smart refrigerator also provides the
facility of online shopping of the scarce items inside. The
provision of recipe suggestion based on the vegetables present in
the basket is carried out through a basic machine learning
algorithm which classifies the vegetables based on the colors
which in turn suggests a particular recipe. The result obtained
justifies the effectiveness of the proposed system.
Index Terms—Smart refrigerator, android application, ma-
chine learning, internet of things.
I. INTRODUCTION
According to FAO study, every year around 1.3 billion tons
of food is wasted all over the world [1].The cause of food
wastage includes domestic wastage by consumer’s refrigerator.
This is because of poor planning, bulk purchases and spoiled
foods. People, overburdened with work at the expense of their
health needs a smart appliance which would be capable of
maintaining a healthier lifestyle and requires no extra time and
effort. One such appliance is refrigerator which is most widely
used to store food. Today, the internet is integrated in our daily
life to such an extent that without it life seems unlikely. The
Internet of Things (IOT) is an extension of internet con-
nectivity into physical devices and everyday appliances. IOT
technology is a systematic integration of sensor technology,
application programming interface (API) and big data [2]. This
technology has led to the evolution of “smart” systems. In this
paper, we propose a smart refrigerator system with IOT and
Machine learning approach to monitor vegetables and daily
essentials and if found below the predefined threshold, the user
is alerted and can place the order through online shopping
website and a recipe suggestion feature is provided where
recipes are suggested based on vegetables present through
basic concept of machine learning.
A. Objectives
The objectives of our work are:
978-1-7281-6221-8/20/$31.00 ©2020 IEEE
Authorized licensed use limited to: Cornell University Library. Downloaded on August 19,2020 at 04:36:59 UTC from IEEE Xplore. Restrictions apply.
1 Quantity Analyses: Notifies the user about the low
quantity of the daily consumed diet (e.g. Egg, milk,
vegetables).
2 Online purchasing options: The android application is
provided with online purchasing of required food
items.
3 Recipe suggestion: Based on the vegetables present in
the refrigerator, a recipe is suggested.
II. ARCHITECTURE
The proposed system is divided into two sections: IOT and
machine learning approach.
The Figure 1 represents the system with IOT where the
flow of data through the sensors, controller, cloud and finally to
the user is described. The Figure 2 represents the process of
recipe suggestion by capturing the image of vegetables and
classifying them based on color using k-means classifier
algorithm.
III. SYSTEM OVERVIEW
The user is provided with an android application which
displays the quantity of items present inside the refrigerator.
The food items like eggs, milk and bread are monitored using
IR sensors, Ultrasonic sensor and load cell respectively. The
calibrated sensors continuously read the data and compare
them with threshold values. The controller classify the sensors
value as below or above the threshold and stores the same in an
online cloud platform i.e. firebase. The continuously updated
data is retrieved in the mobile app designed using android
studio. Based on the data displayed in the app, user is allowed
to order the required items from online store with a single
click. A recipe suggestion feature is provided. A camera
module is attached above the vegetable tray, present inside the
refrigerator. When the user clicks on a particular button a
request through MQTT client is sent to the camera module
which captures an image of the vegetables. Color and number
of edges from these images are extracted and K-means
classifier algorithm is applied on them to identify the
vegetables. K-means is a type of clustering which partitions n
data into k groups in which each data belongs to the group with
nearest expectation. The K-means clustering equation is given
as in:
1
 (1)
Clustering by K-means algorithm is as shown in Figure 3.
The identified vegetables are retrieved in the app and the
combination of these vegetables is used to suggest a recipe.

Fig. 3. K-means clustering.

IV. IMPLEMENTATION
A. Hardware Design
This section describes about the hardware components and
their working used.
1. Raspberry Pi: Our system uses raspberry pi model B3
as the main board which contains 40 GPIO pins used
for the sensors, a USB port for the web camera and a
SDHC slot. The sensors and the camera module are
interfaced and calibrated with the raspberry pi.
2. IR sensor: An IR sensor is an electronic module which
consists of a transmitter and a receiver. The transmitter
sends infrared rays. If an obstacle is found within in a
range of 10cm, the rays are reflected to the receiver.
Two IR sensors are attached at the edge of the egg tray
in the refrigerator. It continuously monitors the
presence of the eggs and updates it in the app through
firebase.
Fig. 1. IOT System 3. Ultrasonic sensor: Ultrasonic sensor (HC-SR04) is
widely used to measure the distance without actual
contact. The working principle of sensor is: Sound
waves in ultrasonic range are continuously emitted.
When it encounters an obstacle the sound waves get
reflected. Distance is calculated based on time elapsed
for the wave to return. The range of the sensor
varies from 2cm to 400cm with a precision of 3mm.
Ultrasonic sensor is placed below the lid of milk
container. Based on the above principle it measures the
quantity of milk present in the container.
4. Load cell: Load cell is a transducer which transforms
force or pressure into electrical signals. It has 4 strain
gauges in a Wheatstone bridge configuration, which
deforms when pressure is applied on it. Load cell
comes in various ranges like 5kg, 10kg, 100kg and
Fig. 2. Classifier System.
more.HX711 amplifier is used with the weighing
sensor to amplify the electrical signals generated by
the load cell. Bread or any container is placed on the
load cell, which measures its weight.
5. Camera module (INTEX IT-305EC): The webcam is
placed at the top corner of vegetables section of
refrigerator model. The camera takes the picture of

Authorized licensed use limited to: Cornell University Library. Downloaded on August 19,2020 at 04:36:59 UTC from IEEE Xplore. Restrictions apply.
 vegetables present in the fridge, each time the user or IDE (Integrated Development Environment) to
wants to access that. The user is provided with a ‘Take develop android Apps. It gives a proper environment
Picture’ button on his app which sends the signal to for developing and for coding of android apps.
raspberry pi through MQTT and the camera captures
an image. 4. MQTT Client: MQTT is used as a protocol between
android app and camera module. It is one of the most
commonly used protocols in IOT projects. It stands for
Message Queuing Telemetry Transport. Like any other
internet protocol, MQTT is based on clients and a
server. Likewise, the server is responsible for handling
the client’s requests of receiving or sending data
between each other.
5. OpenCV library: The opencv library is installed in
raspberry pi for image processing. It is one of the
libraries used for image processing mainly used to do
operations on the images.
V. RESULT
This section shows the pictorial representation of results.
A. Performance Analysis
The work is carried out in the hardware and software
interfaced environment. The performance of the system is
evaluated. The RoundTrip Time (RTT) is the time required for
the data to travel from sensors to firebase database through the
controller and back to the android application. The RTT is
divided into four levels.
Level 1: Response of the sensors. Level 2: Processing by
the controller. Level 3: Updation of data in firebase. Level 4:
Fig. 4. Proposed system.
Retrival of data in the app.
The time of these levels is recorded and average value is
TABLE I. CALIBRATED VALUES OF SENSORS. calculated. With these values a RTT plot is generated as shown
Sensor Range Decision in Figure 8.
IR Sensor 5-15 cm Egg is detected By observing data flow from sensors to android application
31-42 cm Empty container through firebase database, we conclude that the time taken is
Ultrasonic Sensor 21-29 cm Half full container practically more. So, a much faster approach is used to activate
0-19 cm Full container
the camera module. MQTT protocol acts as a bridge between
Load cell below 500g Insufficient
software tool and hardware component like camera module
B. Software Design where the response is faster.
The software packages used in this project are described
below.
1. Raspbian Stretch: Raspbian is used as an OS for
Raspberry pi. Raspbian is the main and basic software
for RPi devices, officially supported by the Raspberry
Pi Foundation. In fact, it is an operating system, based
on Debian and optimized for Raspberry Pi hardware.
2. Firebase database: Firebase is used as a cloud platform
to store the data in the project. It is real time database
which allows sharing the data across Android, IOS and
Web apps. It is a cloud-hosted NoSQL database that
allows to store and sync data in real time. Firebase data
is also available offline, so that the app always has
access to the data. The data is automatically synced
once the app comes back online.
3. Android Studio: The app is built using android studio Fig. 5. Presence or absence of egg is detected and notified through the app. In
platform. Android Studio is Google’s Official Software the case of absence, the eggs are booked through online shopping
website.

Authorized licensed use limited to: Cornell University Library. Downloaded on August 19,2020 at 04:36:59 UTC from IEEE Xplore. Restrictions apply.

Fig. 6. The level of the milk in the container is measured and sent to firebase
and from firebase the data is retrieved in the app. In case of insufficient
quantity, booking can be done.
VI. CONCLUSION
In this work, we present a low-cost smart refrigerator, a
concept with Internet of Things (IOT) technologies for
identifying low quantity items. The android application in this
work provides real time updates even when the user is outside
the house and suggests a recipe based on the vegetables present
in the fridge with Machine Learning approach.
Fig. 7. Vegetables in the tray are recognized based on their colors and
respective recipe is suggested.
Fig. 8. RTT Plot.
Authorized licensed use limited to: Cornell University Library. Downloaded on August 19,2020 at 04:36:59 UTC from IEEE Xplore. Restrictions apply.
REFERENCES
[1] Food and Agriculture Organization of the United Nations.Food Wastage
Footprint: Impacts on natural resources. Retrieved Mar., 2017.
[2] Aurel-Dorian Floarea, Valentin Sgaˆrciu, “Smart Refrigerator: A next
generation refrigerator connected to the IoT”, International Conference –
8th Edition Electronics, Computers and Artificial Intelligence, 2016.
[3] Suhuai Luo, Jesse S. Jin, and Jiaming Li, “A Smart Fridge with an
Ability to Enhance Health and Enable Better Nutrition”, International
Journal of Multimedia and Ubiquitous Engineering, Vol. 4, No. 2, April,
2009.
[4] Deepti Singh, Preet Jain, “IoT BASED SMART REFRIGERATOR
SYSTEM”, International Journal of Advanced Research in Electronics
and Communication Engineering (IJARECE), Issue 7, July 2016.
[5] Y Zhai, Y Liu, M Yang, F Long, J Virkki, “A Survey Study of the
Usefulness and Concerns about Smart Home Applications”, Open
Journal of Social Sciences, Vol.02, No.11, 2014.
[6] Alolayan Bushra, “Do I Really Have to Accept Smart Fridges? An
empirical study”. In the Proceedings of the Seventh International Con-
ference on Advances in Computer-Human Interactions pp, 186- 191,
2014.
[7] Shouming Qiao, Hongzhen Zhu, Lijuan Zheng, Jianrui Ding,
“Intelligent Refrigerator based on Internet of Things”, IEEE
International Confer- ence on Embedded and Ubiquitous Computing
(EUC), 2017.
[8] VH Bhide, “A Survey on the Smart Homes using Internet of Things
(IoT)”, International Journal of Advance Research in Computer Science
and Management Studies, 2014.
[9] Mukesh P. Mahajan, Rohit R. Nikam, Vivek P. Patil, Rahul D. Dond,
“Smart Refrigerator Using IOT”, International Journal of Latest Engi-
neering Research and Applications (IJLERA), March 2017.
[10] Folasade Osisanwo, Shade Kuyoro, and OludeleAwodele, “Internet
Refrigerator –A typical Internet of Things (IoT)”, 3rd International
Conference on Advances in Engineering Sciences Applied Mathematics
(ICAESAM’2015), March 2015.
[11] Hsin-Han Wu, Yung-Ting Chuang, “Low-Cost Smart Refrigerator”,
IEEE 1st International Conference on Edge Computing, 2017.
[12] Prof. M. K. Sangole, Bhushan S. Nasikkar, Dhananjay V. Kulkarni,
Gitesh K. Kakuste, “International Journal of Advance Research, Ideas
and Innovations in Technology”, 2017.
[13] IoT Analytics GmbH, White paper, IoT Platform, The central backbone
for the Internet of Things, 2015
[14] Haidawati Nasir, Wan Basyar Wan Aziz, Fuead Ali, Kushsairy Kadir,
Sheroz Khan, “The Implementation of IoT based Smart Refrigerator
System”, 2nd International Conference on Smart Sensors and Applica-
tion (ICSSA), 2018.
[15] Wang Zhongmin, Yu Yanan,”Design of an Interactive Smart
Refrigerator Based on Embedded System”, International Conference on
Sensing, Diagnostics, Prognostics, and Control, 2018.