International Journal of Pure and Applied Mathematics

Volume 120 No. 6 2018, 1335-1346

ISSN: 1314-3395 (on-line version)
url: http://www.acadpubl.eu/hub/
Special Issue
http://www.acadpubl.eu/hub/

GESTURE REGULATED
ANIMATRONIC ROBOTIC HAND
Rahul Gowtham Poola1 ,
Mrs. A.Anilet Bala(Assistant Professor)2 ,
Rajesh Peddikuppa3 ,Kasireddy Mahanth Reddy4
Dept. of Electronics
and Communication Engineering
SRM Institute of Science
and Technology, Chennai, India

June 25, 2018

Abstract
Humans face innumerable challenges in a meteoric
growing world where humans encounter difficulties in
accomplishing tasks. This can be surmounted using an
animatronic hand. It can play a major role in Military,
Industries, and medical exercises. Animatronic involves
the use of electro-mechanical devices to design systems
that replicate human bodily movements. Animatronics is
an interdisciplinary field necessitating robotics,
electronicsand mechanical studies. This paper showcases
various practical applications of an animatronic model of
hand by the use of MATLAB. Gesture-based
communication is utilized in this design. A gesture will be
recognized through a webcam and image grabbed using
Image Acquisition toolbox of MATLAB. This Image can
be further processed and output can be interfaced to
animatronic hand and can be controlled by human hand.
MATLAB and Arduino are the key elements of this
proposed model. The practical design of the Robotic Hand
is made feasible for real-time applications in Automation
and rescue tactics.

1335
International Journal of Pure and Applied Mathematics Special Issue

Keywords:Micro Grid, Distributed Generation,Multi

objective optimization,PI controller, Fuzzy Logic
Controller

1 INTRODUCTION
The role of an efficient Robotic Hand controller unit is to create
an interactive medium between human and animatronic robotic
hand using sign gestures. The idea led to the development of an
animatronic robotic hand which is physically identical to the
human hand and to expand abilities of the human hand by means
of Image Processing approach. The motive of human gesture
recognition and modeling is transforming gesture into a message
to be conveyed. This inspired us to make an animatronic robot
hand which can play the role of the human hand in antagonistic
conditions without the actual presence of the human hand in that
environment. Gestures are expressive, meaningful body actions
that are used in daily communication [1]. The human hand is
versatile in nature but is not capable of working under
antagonistic condition [2]. Gestures are basic mode of non-oral
and wordless communication in which visible body activities
convey messages [3].

Modeling and recognizing of gestures involve strong challenges

due to variation in size and shape. Human gestures are classified
as Stationary Gestures and nonstationary Gestures. A stationary
gesture is fixed image position and posture, represented by an
individual image while a nonstationary gesture is an ambulant
type, represented by a chain of images [3]. The motive behind this
project is to build a Human-Robot Interactive controller that uses
hand gestures of humans and relates the gestures to an
appropriate action of the Robotic Hand. The gestures are used to
steer a movable robotic hand and the applications are fulfilled
accordingly. The entire activity will be implemented in real-time,
employing efficient programming and algorithms. Gesture
Recognition and Modelling enable the interface between Human
and Computer controllers [3]. Gestures recognition is implemented
by employing modes and tools associating with Image Processing.

1336
International Journal of Pure and Applied Mathematics Special Issue

Enormous research work is done in the field of the

human-robotinterface. Miscellaneous articles have been surveyed
regarding the gesture detection for the real-time movement of
Robotic hand [4]. The diverse fields related to same techniques
involve sign language, computer graphics and automated robots
used to substitute humans.A novel method of gesture modeling
and gesture recognizing using flex sensor, employing wireless
communication is evidently proposed. The gesture
modelinginvolves three criteria namely, hand position, hand
orientation and hand movement criteria. The Flex sensors
calculate the amount of bend the human finger has produced and
transmits to the Robotic controller unit. The program relates the
amount of bend of Flex sensors with the rotation of Robotic
Hand. The different bending angles replicate different movements
of the Robotic Hand [1].

Most of the hand interactive systems comprise of three

stages:detecting, Coursing, Perceiving. The detection stage
defines and extracts the optical features that define the gestures
with respect to the camera. The coursing stage is responsible for
achieving an incorporation between consecutive frames of the
image. It is done to make the controller enlightened of ”what is
where”. The perceiving stage involves grouping of the temporal
data extracted in the previous stages and assigning the groups
with specific labels associated with particular gestures [3].

2 DESIGN METHODOLOGY
The Design Methodology of the Robotic Hand Using Image
Processing involves Several stages. The Image Processing flow
diagram (Fig.1) describes the different stages involved in
converting Gestures into the Real-Time movement of the Robotic
Hand [5].

1337
International Journal of Pure and Applied Mathematics Special Issue

Fig. 1. Project Flow Diagram

A. Image Processing

Input gestures are captured using webcam and images are

acquired through Matlab programmed with Image acquisition
toolbox. The gestures are captured with a limited time grab
interval and will be flushed.

B. Image Acquisition

The video input is acquired and processed by this stage. In order

to send the image information to the controller, the captured
gesture images by the laptop camera are to be processed. The
acquired images can be transfigured into appropriate
configuration and can be coded accordingly. The crucial tool for
acquiring an image in Matlab is the image acquisition toolbox [6].

C. Pre-Processing

Image pre-processing does not improve image information content

1338
International Journal of Pure and Applied Mathematics Special Issue

but can decrease it if entropy is a measure of information. The

pre-processing stage improves the image parameters that
overpower the undesirable distortions or augment image
characteristics essential for further processing stages [6].

D. Noise Reduction

Noise reduction involves removing an unwanted noise from a

signal i.e. is unwanted background [6].

E. Resizing

Reducing the size of the acquired image to the required size and
parameters [6].

F. Pruning

Pruning is the type of morphological process used to remove

unwanted object from the image [6].

G. Segmentation

This stage sectionalizes an image into multiple sections

andfictionalizes the physical representation of an image into a
more meaningful and simplified representation by locating the
objects and its relative boundaries with accuracy. Segmentation
stage assigns a specific tag to a specific pixel in an image by
allowing the pixels with the same tag constitute communal
characteristics [7].

H. Feature Extraction

When the image can’t be used as a signal for transmission, the

features of the image containing the message that the image wants
to convey can be transmitted. The feature is the output of the
image processing [6].

1339
International Journal of Pure and Applied Mathematics Special Issue

TABLE I. MATLAB CODE WORD AND OPERATION

The feature of the Gesture extracted is transferred to XBee

Transceiver through RS232 port. The RS232 port is Serial
Communication port. The XBee acts a transmitter and transmits
asignal to other XBee that acts as a receiver. The signals are
received through the Wireless medium. The received signals are
transferred to Arduino for interfacing with the Arduino.

3 RESULTS AND DISCUSSION

Gestures are captured as RGB color image. The image undergoes
Processing and features are extracted as input to be interfaced with
the Robotic Hand. When a gesture is shown as number ”3”, it is
captured using camera and Image Processing takes place (Fig.2).
The feature extracted from the gesture number 3 is transmitted
by XBee Transmitter and received using another XBee receiver.
The received signal is transferred to Arduino for interfacing with
the Robotic Hand. As a result, the third LED will glow stating
the motion of the robotic hand related to gesture ”3” is working
accordance with the input gesture. The Robotic hand will perform
a combination of three movements namely open, down, close, up
and left (Fig.3 and Fig.4).

1340
International Journal of Pure and Applied Mathematics Special Issue

Fig. 2. Image Processing Stages

Fig. 3. Robotic Hand (Left movement)

1341
International Journal of Pure and Applied Mathematics Special Issue

Fig. 4. Robotic Hand (Right movement)

TABLE II. GESTURES AND ROBOTIC HAND MOVEMENTS

The Robotic Hand can be programmed with a different combination

of motions for different human gestures. The motion of Robotic
hand involves single action or two combination actions or three
combination actions and so on accordingly. The existing model has
a success percentage of 80% with 8 successful detections out of 10
random tests performed. The proposed model with combination
movements of Robotic Hand has a success percentage of 90% with
9 successful detections out of 10 random tests.

1342
International Journal of Pure and Applied Mathematics Special Issue

TABLE III. TEST ANALYSIS

Graphs (Fig.5 and Fig.6) illustrates the time taken for the system
to acquire, process the image, transmit the signal
andcommunicate to the Robotic Hand to perform the appropriate
motion or a combination of motions. The four combination
movement of Robotic Hand has time delay more than that of
three combination movements for different gestures.

Fig. 5. Three Combination Movements

1343
International Journal of Pure and Applied Mathematics Special Issue

Fig. 6. Four Combination Movements

4 ACKNOWLEDGEMENT
The author would like to earnestly acknowledge the efforts and
contributions made by the teammates in transfiguring a visionary
idea into a realistic design. The author would also like to sincerely
thank project mentor for the valuable guidance, consistent
encouragement and timely help for making a successful research
work.

References
[1] Yuanhao Wu, Ken Chen and Chenglong Fu, Natural
Gesture Modelling and Recognition Approach Based on
Joint Movements and Arm Orientations, IEEE SENSORS
JOURNAL, VOL. 16, November 1, 2016.

[2] Abdullah Shaikh, Gandhar Khaladkar, Rhutuja Jage and

Tripti Pathak Javed Taili, Robotic arm movement wirelessly
synchronized with human arm movements using real-time
image processing. 2013 Texas Instruments India Educators
conference.

10

1344
International Journal of Pure and Applied Mathematics Special Issue

[3] Gaurav Chauhan and Prasad Chaudhari, Gesture-based

wireless control using Image Processing. 5th Nirmala
University International Conference on Engineering, 2015.

[4] Pankaj S Lengare and Milind E Rane, Human Hand Tracking

using MATLAB to control Arduino based Robotic Arm.
International conference on pervasive computing, 2015.

[5] Panth Shah, Tithi Vyas. Interfacing of MATLAB with

Arduino for object detection Algorithm Implementation using
Serial Communication. International Journal of Engineering
Research and Technology (IJERT), Vol. 3, Issue 10, October
2014.

[6] C.Theis.I.Iossifidis and A.Steinhage, Image Processing

methods for interactive Robot control. Proceedings 10th IEEE
International Workshop on Robot and Human Interactive
Communication, 2001.

[7] Operating two servos motors with Arduino.

http://www.robotoid.com/appnotes/arduino-operating-
two-servos.html

[8] Bhumeshwari Basule, Shubhangi Borkar. Robotic Four Finger

ARM controlling using Image Processing. Volume 7, Issue
No.7, IJESC.

11

1345
1346