21BCE5309 - Divyanshu

School of Computer Science and Engineering

Vellore Institute of Technology

Vandalur - Kelambakkam Road, Chennai - 600 127

April – 2024
 Design and Implementation of IoT-Based Smart
Home Voice Commands for disabled people using
Google Assistant
Abstract— The development of the Internet of Things (IoT)
technology has a positive impact on human life. A smart home is
one of the IoT technology applications that facilitate human
activities. The problem of physical abnormalities is a matter of
limited activities for disabled people. This paper proposes a
design of IoT-based smart home application with a remote
control device that developed using voice commands for
disabled people. Smart home control systems help disabled
people to control their home electrical devices such as
television (TV), lights, and fans using only voice commands
without moving to turn on or turn off electrical equipment.
The voice recognition on electrical equipment is using the
Google Assistant's application on smartphones. The Google
Assistant application will accept voice commands when the
pronunciation is correct. Voice commands on IoT-based Smart
Home are more simple to apply, without typing text messages.
Users get convenience compared to using text. The signal
strength of an Internet connection will create a useful
performance device at the Response Time of Google Assistant,
Response Time of System Processing, activating and
deactivating Electrical Equipment. We expect that this device
could be more useful to help disabled people interact with their
environment by utilizing IoT technology facilities.
Keywords—the internet of things, smart home, voice
commands, disabled people, google assistants
I. INTRODUCTION
Today's human activities are inseparable from the use of
computers and smartphones. It is because of the
development of internet technology as a general
communication for human needs in communicating through the
internet. The need for using an internet connection is
increasing, where the internet can be connected
continuously.
A smart home is a new concept of household. A smart
home provides comfort, safety, energy-saving potential for the
house at any time [1-3], which gives a better quality of life for
the people [4]. The smart home is here to facilitate the
occupants of the house in managing all matters relating to the
comfort of themselves as residents of the house, ranging
from safety issues to the matter of accessing devices that are
made more interactive and can be 'controlled' through
just one device, the application on a smartphone or other
device. Smart Home is a part of the innovation of the Internet
of Things, where all everyday objects or devices [5,
problem of physical disorders is a problem of disabled
people, where physical limitations make it difficult for them to
move more freely.
We propose a design of the IoT-based Smart Home
device. This research aims to help people with physical
disabilities so they can continue their activities around their
home environment. Therefore, this research designed an IoT
Smart Home device that can help disabled people to control
electronic equipment or electrical equipment, such as
television (TV), lights, and fans [10] with voice commands to
6] that are familiar with people's lives are 'spun' because
today's technology mostly uses the all-chip form. The issue of
Smart Home, the Internet of Things will come in the form of
items that we often encounter in the homes of most
people.
Internet of Things (IoT) is the latest technologies in the
world of the Internet, where this technology will become a
technological trend in the future. Currently, IoT has been
expanded. IoT is the leading platform for communication
devices. It is estimated that the numbers of IoT users will
improve massively where the economy grows significantly. It
was estimated in the biomedical or health sector. Besides, IoT
is experiencing rapid growth not only in the health or
biomedical sector but in relevant application services as well
[7-9].
Based on the Report of The Institute of Electrical and
Electronics Engineers (IEEE) 2012 that IoT has been
defined into three parts: sensor, network, and internet
connection [10]. The International Telecommunication
Union (ITU) as a source of reference and information in the
field of telecommunications standards has defined IoT as a
public infrastructure used for the information society, which
provides interconnection services both physical and virtual. It
is based on operations and developments in the field of
information and communication technology.
In short, the IoT is composed of three parts: network,
sensor, and application usage. Several communication
technologies can be used to support the connection of
devices to the IoT [11].
After we were introduced to IoT Technology, smart
home research and development became popular in the last
few days. IoT-based Smart Home is an application that
could facilitate human activities to control many devices, such
as electrical equipment, via the internet connection.
Besides, various wireless technologies could help connect
from remote areas to develop the intelligence of the home
environment. Sophisticated IoT networks are being formed to
meet human needs in terms of connections with other
things. IoT technology is utilized to generate innovative
ideas and significant growth for smart homes to improve
living standards [12].
In social life, the existence of people with disabilities is part
of our lives. They also need attention, because no one wants it
like that. Therefore we must respect them. The
Android smartphones using Google Assistant. This
device can be controlled and accessed remotely. It makes it
easier for disabled people to utilize it without touching them
physically. Disabled people do not need to move anymore to
turn on and turn off electrical equipment at home. The use of
this device is not only used for disabled people but can also be
used for paralytic, elderly people and anyone who needs it.
* RELATED WORK
* Voice Commands Technology
The voice recognition on electrical equipment is using
voice commands application on the smartphone. Digital
sound processing is controlled by an application to
recognize the detected voice commands, which are often
called Voice Recognition. This technology works by
capturing the human voice that is converted into a digital
format so that it can be translated into a system. Then the
system will compare the input information that has been in the
form of a digital format with a voice database that has been set.
The design used in this research uses a wireless sound
system, which controls electrical appliances such as TV,
lights, and fans. The function of this voice control is as a
controller to activate and deactivate electrical equipment by
replacing the button function with voice commands through
wireless modem and internet. The development of voice
control using the Google Assistant Application could
become a future home reference. Where we do not need to
move only with voice commands can activate and deactivate
home electrical equipment even though it is somewhere
different in the house.
Google Assistant is a virtual assistant application
supported by artificial intelligence technology, which was
developed by Google, mainly available on mobile devices and
smart home devices.
Almost all Android smartphones have a feature Google
Assistant Application. If not, we could get it on the Google
Play Store to install the application. Google Play Store is a free
application and open source to install anythings of
application for Android smartphone.
In Indonesian, Android smartphone users do dominate the
mobile market. We can see Android smartphones
everywhere, but not so with Apple smartphones. The
existence of Apple smartphones in Indonesian is only a few
percent of Android.
It also happens in other countries. New research released by
Nielsen in the third quarter of 2015 said that 52.6 % of
smartphone owners in America use the Android operating
system. While Apple iOS users are only 42.7 %, this is quite
surprising considering that America is the home country of
Apple iOS. Some, as a result, top U.S smart operating
systems by market share copyright © 2015 The Nielsen
Company [13], as shown in Table I.
TABLE I. TOP U.S. SMARTPHONE OPERATING SYSTEMS BY MARKET
SHARE COPYRIGHT © 2015 THE NIELSEN COMPANY
No Smartphone Operating Systems User Percentage
1 Android OS 52.6 %
2 Apple iOS 42.7 %
3 Microsoft 2.8 %
4 Blackberry 0.7 %
5 Other 1.2 %
It turns out that this Android smartphone becomes the most
popular smartphone operating system not only in
Indonesia and America but also in the world. Unlike iOS,
which is only made by Apple smartphones, but Android
smartphones can be used by companies such as Samsung, LG,
Motorola, Huawei, and many more.
The percentage from the International Data Corporation
(IDC) shows that Android smartphones have 82.8 % market
share worldwide, while Apple iOS smartphones only gain 13.9
%. This difference is apparent that Android
smartphones are more desirable compared to other
smartphones.
B. Technology for Disabled People
The way to increase the involvement of people with
disabilities is through the use of appropriate assistive
technology. Technology for disable people provides
information knowledge that refers to the field of device
technology and assistive technology services. It relates to the
lives of users with disabled people. This paper was
written to provide a broad understanding of Technology for
Persons with disable people that cover the field of research and
development of community service activities, especially for
disabled people. Disability is a temporary or permanent
functional limitation experienced by people of various ages and
under any circumstances. This paper is related to
disability technology which is a means of supporting or
providing assistance and compensation to people with
disabilities in their daily functioning.
The North American Assistance Engineering and
Technology Rehabilitation Society states that aid technology
has been proven to help people with disabilities improve
their quality of life. With technological devices specifically
designed to meet individual needs, people with disabilities can
improve and optimize their daily functions, enabling them
to be independent, independent and confident. For
example, individuals with mobility impairments can benefit
from a wheelchair system and artificial limbs that increase
independence.
Following the guidelines of the World Health
Organization (WHO), many public and private hospitals
type A or B with rehabilitation centers, private companies and
non-governmental organizations provide technology to
individuals with disabilities. Providers of assistive
technology devices promote modalities which include
mobility aids, wheelchairs and seating systems.
These assistive technology tools can help fulfill the life
goals of people with disabilities, especially by enabling
active participation and interaction in education and work.
Going forward, the government and citizens of Indonesia
must become advocates for people with disabilities by
promoting inclusion and equality in aspects of life every application is more simple. Because with voice commands, the
day. In addition to ending the stigma and discrimination users get more convenience compared to using text. Users
against them, aid technology can be one of the catalysts to help only send voice commands to the Google Assistant
our country be truly inclusive for all. Application and the system will automatically display the text
This paper proposes a design of IoT-based smart home messages that we said before. It means that users do not need
application with a remote control device that developed to type text messages again. Some, as a result, are types
using voice commands for disabled people. This device of the application for IoT smart home control system, as shown
helps disabled people to control their home electrical in Table II.
devices using voice commands without moving to turn on or
turn off electrical equipment. The voice recognition on TABLE II. Type of Applications For IoT Smart Home Control System
electrical equipment is using the Google Assistant's Control Type of Send Commands Ref
application on smartphones. System Applications
IoT-based 1. SMS Typing text [15]
Smart Home message
* SYSTEM MODEL 2. Telegram Typing text [16]
Smart home control systems allow humans to control message
their home electrical devices such as TV, lights, and fans 3. Blynk Apps Typing text [17]
message
through wireless modem and internet using only voice 4. Web Typing text Web [18]
commands. This smart home control works when it says OK & Typing text
GOOGLE, which then continues the command that has been message
set through the program. This system will help disabled 5. Google Voice Command This
Assistant & Without Typing Paper
people to control electrical equipment without moving text message
around to turn on or turn off. Digital sound processing is
controlled with an application to recognize the detected
voice commands. Therefore, the method to be designed is a
voice recognition application as controller of
microcontroller-based electrical equipment through wireless
modem and internet [14]. The application of voice
recognition applications on electrical equipment uses a relay
that functions as a switch using the Google Assistants
application on the smartphone so that pronunciation can be
conducted remotely. Fig. 1 depicts the system model of the
proposed implementation of IoT-based smart home voice
commands using Google Assistants.

Fig. 1. The system model of the proposed design for IoT-based smart home
voice commands using google assistant

From previous researches on the IoT Smart Home

control system, there are 4 (four) types of the application used
such as Short Message Service (SMS) [15], Telegram [16],
Fig. 2. Flowchart of Activating and Deactivating the Electrical Equipment
Blynk Apps [17], and Web [18]. When the user uses the Control System Using Google Assistants
application to control electrical equipment in the smart home,
then the user needs to type the text message in advance The following flowchart explains the steps of the overall
to send the command while the Google Assistant program flow in Activating and Deactivating the Electrical
Equipment Control System Using Google Assistant, as
shown in Fig. 2:
C. When starting, execute the program process
initiation.
* The program runs with the Microcontroller of
NodeMCU ESP8266 in the standby mode position
waiting for messages from the user.
 * If the Microcontroller of NodeMCU ESP8266 is not
connected to the internet, it will return to the standby
mode position. Check Internet Connection
* If the Microcontroller of NodeMCU ESP8266 is
connected to the internet, it is ready to receive the next
command message.
* Next, open and run the Google Assistant application.
* If the voice command message from the user's voice
pronunciation is correct, saying OK GOOGLE
TURN ON ”ELECTRICAL EQUIPMENT” on the
Google Assistant application, then the message
becomes the keyword to TURN ON the
ELECTRICAL EQUIPMENT, such as TV, Front
Light, Fans. Then ACTIVATE ”ELECTRICAL Fig. 3. The circuit implementation of the electrical equipment control
EQUIPMENT” system
• Next, if the voice command message from the user's
voice pronunciation is correct, saying OK GOOGLE
IV. TESTING AND RESULT
TURN OFF ”ELECTRICAL EQUIPMENT” on the
Google Assistant application, then the message After the design of the device is complete, the next step is
becomes the keyword to TURN OFF the to check, test and measure the device. This stage aims to check
ELECTRICAL EQUIPMENT. Then DEACTIVATE the overall function and performance of the device.
”ELECTRICAL EQUIPMENT.”
• If the voice command message from the user's voice A. Checking Voice Pronunciation to Google Assistant
Application
pronunciation is not correct on the Google Assistant
application, then the message will be rejected. (It This Checking aims to check the pronunciation of
could not activate it). Check Pronunciation. voices. Voice Pronunciation must be correct and precise
• The end, it is finished when communicating with an Android smartphone through the
Google Assistant Application. If the voice pronunciation is
The following Pseudocode of Activating and correct and precise, then the Google Assistant Application will
Deactivating the Electrical Equipment Control System accept it. Furthermore, if the pronunciation of the voice is
Using Google Assistant, as described in Algorithm. 1: incorrect and unclear, the Google Assistant Application will
reject it. For Checking Voice Pronunciation to Google
Algorithm 1: Activating and Deactivating the Electrical Assistant Application, as shown in Table III.
Equipment

1 Initialization: execute the program process

2 Run NodeMCU
3 if NodeMCU Connect to the Internet? then
4 Open Google Assistant and System Running
5 else if Check Internet Connection
6 if say OK GOOGLE TURN ON ”ELECTRICAL
EQUIPMENT”? then
7 Activate ”ELECTRICAL EQUIPMENT”
8 else if say OK GOOGLE TURN OFF ”ELECTRICAL
EQUIPMENT”? then
9 Deactivate ”ELECTRICAL EQUIPMENT”
10 else Check Voice Pronunciation
11 end

The circuit implementation of the control system

consists of the microcontroller of NodeMCU ESP8266,
Modem Wireless, relay, Power Supply Unit (PSU) and
output socket to control electrical equipment, as shown in Fig.
3. Then the Prototype of the control system, as shown in Fig.
4.
TABLE III. Checking voice pronunciation to Google assistant 4 Y A
Application 5 Y A
6 Y A
Correct Voice Google Assistant 7 Y A
Number of Pronunciation Response 8 Y A
Experiments Yes No Accepted Rejected 9 Y A
(Y) (N) (A) (R) 10 Y A
1 Y A 11 Y A
2 Y A 12 Y A
3 Y A 13 Y A
 14 Y A
15 Y A
16 Y A
17 Y A
18 N R
19 Y A
20 Y A
B. Testing Activates All Control System Electric Equipment
This test aims to test the performance of the device and the system of electrical equipment, as shown in Fig. 6. So we
relationship between hardware and software for the
Google Assistant Application program. With this testing,
we can find out whether the performance of the device is
working by the expected design. We are testing the
performance of the device when activating the control
system of electrical equipment, as shown in Fig. 5. So we
measure the Device Response Time using a stopwatch.
The parameters measured are as follows:
• Measuring Response Time for Google Assistant
Applications:
Measurements are starting from the time of
pronunciation of the words conveyed to obtain
answers back from Google Assistant
• Measuring the Response Time from switch ON of
Electrical Equipment:
Measurements are starting from the time of the
response back from Google Assistant submitted to
switch ON of Electrical Equipment
Fig. 5. Testing Activate All Control System Electric Equipment
V. CONCLUSIONS AND FUTURE WORK
In this paper, we design and implement IoT-based smart
voice commands for disabled people using Google
Assistant. This system would help disabled people to control
electrical equipment such as TV, lights, and fans using voice
C. Testing Deactivates All Control System Electric
Equipment
This system test aims to test the performance of the
device and the relationship between hardware and software for
the Google Assistant Application program. With this
testing, we can find out whether the performance of the
device is working by the expected design. We are testing the
performance of the device when deactivating the control
measure the Device Response Time using a stopwatch.
The parameters measured are as follows:
• Measuring Response Time for Google Assistant
Applications:
Measurements are starting from the time of
pronunciation of the words conveyed to obtain
answers back from Google Assistant
• Measuring the Response Time from switch Off of
Electrical Equipment:
Measurements are starting from the time of the
response back from Google Assistant submitted to
switch Off of Electrical Equipment.
Fig. 6. Testing Deactivate All Control System Electric Equipment
The android smartphones for voice commands using the
Google Assistant Application are illustrated in Fig. 7.
Fig. 7. Illustration of the android smartphones for voice commands using
Google Assistant Application
[14] N. Hayati and M. Suryanegara, "The IoT LoRa system design for
tracking and monitoring patient with a mental disorder," in 2017
IEEE International Conference on Communication, Networks, and home
Satellite (Comnetsat), 2017, pp. 135-139.
[15] P. Bhatt and B. Thakker, Secure IoT-GSM based Smart Home
System with Confidentiality and Integrity. 2017.
[16] M. Muslih et al., "Developing Smart Workspace Based IoT with
commands of Google Assistant on smartphones. Users do Artificial Intelligence Using Telegram Chatbot," in 2018 not
need to move to turn on or turn off electrical equipment. International Conference on Computing, Engineering, and Design
The Google Assistant application will accept voice (ICCED), 2018, pp. 230-234.
commands when the pronunciation is correct. Voice [17] H. Durani, M. Sheth, M. Vaghasia, and S. Kotech, "Smart
Automated Home Application using IoT with Blynk App," in 2018
commands are more simple to apply, without typing text Second International Conference on Inventive Communication and
messages. Users get convenience compared to using text. Computational Technologies (ICICCT), 2018, pp. 393-397.
Users only send voice commands to the Google Assistant [18] N. Lee, H. Lee, H. Lee, and W. Ryu, "Implementation of smart home
Application, and the system will automatically display the service over the web of object architecture," in 2015 International
Conference on Information and Communication Technology
text messages that we said before, without typing text Convergence (ICTC), 2015, pp. 1215-1219.
messages. The signal strength of an Internet connection will
create a useful performance device at the Response Time of
Google Assistant, Response Time of System Processing,
activating and deactivating Electrical Equipment. Future
work is to make and test the system in a real scenario.
Further research, there are several challenges to developing
devices using Wireless Sensor Network, NB-IoT and 5G
networks to get low latency and the best signal strength.
Then this will build a better device testing performance.

REFERENCES
[1] S. A. Celtek, M. Durgun, and H. Soy, "Internet of Things based
smart home system design through wireless sensor/actuator
networks," in 2017 2nd International Conference on Advanced
Information and Communication Technologies (AICT), 2017, pp. 15- 18.
[2] K. Guravaiah and R. L. Velusamy, "Prototype of Home Monitoring
Device Using Internet of Things and River Formation Dynamics-
Based Multi-Hop Routing Protocol (RFDHM)," IEEE Transactions on
Consumer Electronics, vol. 65, no. 3, pp. 329-338, 2019.
[3] X. Li, R. Lu, X. Liang, X. Shen, J. Chen, and X. Lin, "Smart
community: an internet of things application," IEEE
Communications Magazine, vol. 49, no. 11, pp. 68-75, 2011.
[4] D. Wang, "The Internet of Things the Design and Implementation of
Smart Home Control System," in 2016 International Conference on
Robots & Intelligent System (ICRIS), 2016, pp. 449-452.
[5] S. Tang, V. Kalavally, K. Y. Ng, and J. Parkkinen, "Development of a
prototype smart home intelligent lighting control architecture using
sensors onboard a mobile computing system," Energy and Buildings,
vol. 138, pp. 368-376, 2017/03/01/ 2017.
[6] S. D. T. Kelly, N. K. Suryadevara, and S. C. Mukhopadhyay,
"Towards the Implementation of IoT for Environmental Condition
Monitoring in Homes," IEEE Sensors Journal, vol. 13, no. 10, pp. 3846-
3853, 2013.
[7] M. Suryanegara, A. S. Arifin, M. Asvial, and G. Wibisono, "A
system engineering approach to the implementation of the Internet of
Things (IoT) in a country," in 2017 4th International Conference on
Information Technology, Computer, and Electrical Engineering
(ICITACEE), 2017, pp. 20-23.
[8] A. S. Arifin, M. Suryanegara, T. S. Firdaus, and M. Asvial, "IoT- Based
Maritime Application: An Experiment of Ship Radius
Detection," presented at the Proceedings of the International
Conference on Big Data and Internet of Thing, London, United
Kingdom, 2017.
[9] L. Atzori, A. Iera, and G. Morabito, "The Internet of Things: A
Survey," Computer Networks, pp. 2787-2805, 10/01 2010.
[10] A. B. Chebudie, R. Minerva, and D. Rotondi, "Towards a definition of
the Internet of Things (IoT)," 2014.
[11] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M.
Ayyash, "Internet of Things: A Survey on Enabling Technologies,
Protocols, and Applications," IEEE Communications Surveys &
Tutorials, vol. 17, no. 4, pp. 2347-2376, 2015.
[12] D. Pavithra and R. Balakrishnan, "IoT based monitoring and control
system for home automation," in 2015 Global Conference on
Communication Technologies (GCCT), 2015, pp. 169-173.
[13] G. P. Schneider, Electronic commerce. Stamford, CT: Cengage
Learning, 2015.