International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 INTERNATIONAL JOURNAL OF ELECTRONICS AND

D 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

COMMUNICATION ENGINEERING & TECHNOLOGY (IJECET)

ISSN 0976 6464(Print) ISSN 0976 6472(Online) Volume 5, Issue 3, March (2014), pp. 25-33 IAEME: www.iaeme.com/ijecet.asp Journal Impact Factor (2014): 7.2836 (Calculated by GISI) www.jifactor.com

IJECET
IAEME

PERFORMANCE EVALUATION OF NEURAL NETWORK BASED HAND GESTURE RECOGINITION

Kavita Sharma1,
1

Dr. Anil Kumar Sharma2

M. Tech. Scholar, 2Professor & Principal Department of Electronics & Communication Engineering Institute of Engineering & Technology, Alwar-301030 (Raj.), India

ABSTRACT With the development of information technology in our Society one can expect that computer systems to a larger extent will be embedded into our daily life. These environments lead to the new types of human-computer interaction (HCI). The use of hand gestures provides an attractive alternative to cumbersome interface devices for human-computer interaction (HCI). The existing HCI techniques may become a bottleneck in the effective utilization of the available information flow. Gestures are expressive, meaningful body motions. Interpretation of human gestures such as hand movements or facial expressions, using mathematical algorithms is done using gesture recognition. Gesture recognition is also important for developing alternative human-computer interaction modalities. This research will have tested the proposed algorithm over 100 sign images of ASL. The simulation will show that the true match rate is increased from 77.7% to 84% while the false match rate is decreased from 8.33 % to 7.4%. Keyword: ASL, Gesture Recognition, HCI, Neural Network, SIFT. 1. INTRODUCTION Pattern recognition algorithms generally aim to provide a reasonable answer for all possible inputs and to do "fuzzy" matching of inputs. This is different from pattern matching algorithms, which looks for exact matches in the input with pre-existing patterns. Example of the patternmatching algorithm is regular expression matching, that looks for patterns of a given sort in textual data and it is included in the search capabilities of many text editors and word processors. As compared to pattern recognition, pattern matching is not considered as a type of machine learning, although pattern-matching algorithms can sometimes succeed in providing similar-quality output to the sort provided by pattern-recognition algorithms [1]. Computer is used by many people either at their work or in their spare-time. Exceptional input and output devices have been designed over the
25

International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

years with the purpose of easing the communication between computers and humans, two most known are the keyboard and mouse [1]. Each new device can be seen as an attempt to make the computer more intelligent and making humans able to perform more complicated communication with the computer. It has been possible due to the result-oriented efforts made by computer professionals for creating successful human-computer interfaces [2]. The idea is to make computers understand kind language and develop a user friendly human computer interfaces (HCI). Making a computer understand speech, the facial expressions and human gestures are some steps towards it. For human computer interaction (HCI) interpretation system there are two commonly approaches: Data Gloves Approach and Vision Based Approach. The Data Gloves Approaches employ mechanical or optical sensors. In gesture recognition, it is more common to use a camera in combination with an image recognition system .These systems have the disadvantage that the image/gesture recognition is very sensitive to illumination, hand position, hand orientation etc. In order to circumnavigate these problems we decided to use a data glove as input device. Figure 1 shows the low cost design of hand gesture recognition system having data glove as input device. The data glove makes the system independent to the hand orientation etc.

Fig. 1 Design of low Cost Data Glove Approach The Vision Based Approaches techniques based on the how person realize information about the environment. This technique uses the vision based properties like how object looks etc. The input image may have same meaning but look different. The vision based methods are usually done by capturing the input image using cameras. Figure 2 show the hand captured using the camera. The interaction is based upon the vision properties.

Fig. 2 Vision Based Approaches 2. IMPORTANCE OF GESTURES Gestures are an important part of everyday communication amongst humans. They emphasize information in combination with speech or can substitute speech completely. They can signify greetings, warnings or emotions or they can signal an enumeration, provide spatial
26

International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

information, etc [3]. Gestures are the non-verbally exchanged information. The person can perform innumerable gestures at a time. Human gestures constitute a space of motion expressed by the body, face, and/or hands. Among a variety of gestures, hand gesture is the most expressive and the most frequently used. The Gestures have been used as an alternative form to communicate with computers in an easy way. This kind of human-machine interfaces would allow a user to control a wide variety of devices through hand gestures. A gesture is scientifically divided into two distinctive categories: dynamic and static [24]. A dynamic gesture is intended to change over a period of time whereas a static gesture is observed at the bounce of time. A waving hand means goodbye is an example of dynamic gesture and the stop sign is an example of static gesture. Feelings and thoughts can also be expressed by the gesture. Users generally use hand gestures for expression of their feelings and notifications of their thoughts. The hand gesture and hand posture are two terms related to the human hands in hand gesture recognition. Difference between hand gesture and hand posture, hand posture is considered to be a static form of hand poses [4, 5]. Gestures can be classified into static gestures and dynamic gestures. The static gestures are usually described in terms of hand shapes and dynamic gestures are generally described according to hand movements. It can be defined as a meaningful physical movement of the hands, fingers, arms [5], or other parts of the body with the purpose to convey information or meaning for the environment interaction [4]. The Gesture recognition needs a good interpretation of the hand movement as effectively meaningful commands [6]. 3. HAND GESTURE RECOGNITION Hand gesture recognition [7] is of great importance for human-computer interaction (HCI), because of its extensive applications in virtual reality, the sign language recognition and computer games. To enable more robust hand gesture recognition, one effective way is to use other sensors to capture the hand gesture and motion, example: through the data glove. Unlike optical sensors, such as sensors are usually more reliable and are not affected by lighting conditions or cluttered backgrounds. But, as it requires the user to wear a data glove and sometimes requires calibration, this is inconvenient for the user and may hinder the naturalness of hand gesture. Further, such data gloves are expensive. For a result, it is not a very popular way for hand gesture recognition.

Fig. 3. Some Challenging Cases for Hand Gesture Hand gesture has been used as a natural and efficient way in human-computer interaction. Because of independence of auxiliary input devices, the vision-based hand interface is more favorable for users. Still, the process of hand gesture recognition is very time consuming that often brings much frustration to users. Basic Architecture: The system of Gesture recognition consists of several stages; these stages are varied according to application used, but, however, the unified outline can be settled, Figure 5 fulfils this step.

27

International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

Fig.4. Basic Architecture of Gesture Recognition The stages that are represented in latter figure might broke down into several sub-stages, all these divisions are dependent on the application used, for example; the processing steps are different in case of vision-based or glove-based were employed as well as the vision-base and color-based hand gesture system. Data Acquisition: This is responsible for collecting the input data which are the hand gestures image for vision based system or sensors reading in case of data glove, and the classes should be declared that the classifier classifies the input tested gesture into one of these predefined classes. Gesture Modeling: In this the fitting and fusing the input gesture into the model used, this step may require some pre-processing steps to ensure the successful unification, and Figure 6 shows some common processing steps for successful fitting which will be briefed in the next paragraph. Feature Extraction: The feature extraction should be smooth since the fitting is considered the most difficult obstacles that may face; these features can be hand location, palm location, fingertips location, and joint angles, the extracted features might be stored in the system at training stage as templates or may be fused with some recognition devices such as neural network, HMM, or decision trees which have some limited memory should not be overtaken to remember the training data. Recognition Stage: This is the final stage for gesture system and the command/meaning of the gesture should be declared and carried out, this stage usually has a classifier that can attach each input testing gesture to one of predefined classes with some likelihood of its matching to this class.

4. SCALE INVARIANT FEATURE TRANSFORM (SIFT) & NEURAL NETWORK Scale-invariant feature transform (or SIFT) is an algorithm in computer vision to detect and describe local features in images. For any object in an image, interesting points on the object can be extracted to provide a "feature description" of the object. This description which is extracted from a training image, used to identify the object when attempting to locate the object in a test image containing many other objects [9]. To perform reliable recognition, the features extracted from the training image are detectable even under changes in image scale, noise and illumination. These points usually lie on high-contrast regions of the image, like an object edges. Neural networks (NN) are composed of simple elements operating in parallel. These are inspired by biological nervous
28

International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

systems. Network function is determined largely by the connections between elements. They trained a neural network to perform a particular function by adjusting the values of the connections (weights) between elements [10]. Commonly neural networks are trained, so specific input leads to a specific target output. Such a situation is shown in fig. 3. There, the network is adjusted, based on a comparison of the output and the target, until the output of the network matches the target. Many input/target pairs are used, in this supervised learning to train the network.

Fig.5. Neural Net Block Diagram [10] 5. PROPOSED ALGORITHM In the existing technique the distance ratio and the threshold is taken fixed i.e. 0.65, 0.035. These values of distance ratio and the threshold get incremented and decremented respectively by the value 0.05. But these values of the distance ratio and the threshold makes perfect match to be avoided. The NN is used to decide the value of the distance ratio and the threshold. Here the supervised learning is used to train the network. The input to the NN is the input image and the desired signal is the corresponding database image. The error gives distance between the input and target signal. The error is minimized to get the threshold value. The ratio of difference between the input and the target at this value is taken as the distance ratio value. The proposed algorithm is divided in two phases i.e. the training phase and recognition phase. In the training phase the neural network is trained to detect the value of the distance ratio and the threshold. In the recognition phase the gesture is recognized by the sift point matching algorithm. Following algorithm explains the whole process. Training Phase 1. Input the sample Image. 2. Preprocess the image 3. Take Corresponding database image as the target signal. 4. Use feed backward neural network with the specified input in step 1 & step 2. 5. The minimum error is taken as the threshold value. 6. The ratio at this value is taken is taken as the distance ratio value. Recognition Phase 7. Run the SHIFT match algorithm. 8. Key point matching starts its execution by running the threshold. 9. Key points are matched between test and all the trained images. 10. The key point matching defines the validity ratio. 11. The image having proper validity ratio displayed as the result.

29

International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

6. RESULTS The software used for Simulation is MATLAB 7.0. Fig 6, 7 & 8 shows various output of the program.

Fig. 6. Found Match using Proposed Algorithm

Fig.7. Match found using Proposed Algorithm

Fig.8. False Match using Existing Algorithm

30

International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

We applied several operations on image then matched the image with the database image. Tabe-1 shows the recognition after operations. Table 1: Result of Manipulated images Image Maximum Key Point in Sample Image 253

Operation

Rotate 90

Number of Keypoint Matched 220

Rotate 180

242

211

Rotate 270

242

211

Flip Horizontal

232

200

lip Vertical

233

208

31

International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

Table-2 shows the number of key points matched as well as the maximum number of key points in the sample image for all alphabets. The match column denotes that whether resultant image is true match or false match or not found. Table 2: Results for 26 Characters

7. CONCLUSION The gesturing is rooted in our life, and the human continues this behavior during hhis talk with others even those others were on the phone; this form of communication is appealing since it provides the basic and easy effort done by a person to get the message delivered rather than the cumbersome devices like keyboard and mouse since the former reflects his nature. For that reason, the keyboard and mouse have been replaced by hand, face, finger, or entire body and those new natural input devices attracting more attention since need no extra connections, more efficient, powerful, promising, and preferable naturalness. In this research, we have tested the proposed algorithm over 100 sign images of ASL. If the image is present in database and the algorithm detects the same image, then it is the true match. If an algorithm recognizes the wrong image, then it is known as the false match. The simulation shows that the true match rate is increased from 77.7% to 84% while the false match rate is decreased from 8.33 % to 7.4%. Hence the proposed algorithm improves the performance.

32

International Journal of Electronics and Communication Engineering & Technology (IJECET), ISSN 0976 6464(Print), ISSN 0976 6472(Online), Volume 5, Issue 3, March (2014), pp. 25-33 IAEME

REFERENCES [1] Deval G. Patel et. al., Point Pattern Matching Algorithm for Recognition of 36 ASL Gestures, International Journal of Science and Modern Engineering (IJISME) ISSN: 23196386, Volume-1, Issue-7, June 2013. Lalit Gupta and Suwei Ma Gesture-Based Interaction and Communication: Automated Classification of Hand Gesture Contours, IEEE transactions on systems, man, and cyberneticspart c: applications and reviews, vol. 31, no. 1, February 2001. I. Famaey, Hand gesture recognition for surgical control based on a depth image www.dhm2013.org/dhm2013_submission_52.pdf. Garg. P., Aggarwal. N and Sofat. S: 2009 Vision Based Hand Gesture Recognition, World Academy of Science, Engineering and Technology vol. 49, pp. 972-977. Sushmita Mitra et. al. ,Gesture recognition: A survey, Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on 37.3 (2007): 311-324. Murthy R. S. &. Jadon. R. S: 2009. A Review of Vision Based Hand Gestures Recognition, International Journal of Information Technology and Knowledge Management, vol. 2(2), pp. 405-410. Zhou Ren et. al., Robust Hand Gesture Recognition Based on Finger-Earth Movers Distance with a Commodity Depth Camera, MM11, November 28December 1, 2011, Scottsdale, Arizona, USA. Copyright 2011 ACM 978-1-4503-0616-4/11/11 ...$10.00. Mokhtar M. Hasan et. al. ,Hand Gesture Modeling and Recognition using Geometric Features: A Review, Canadian Journal on Image Processing and Computer Vision, Vol. 3, No. 1, March 2012. Pallavi Gurjal et. al., Real Time Hand Gesture Recognition Using Sift, International Journal of Electronics and Electrical Engineering ISSN : 2277-7040 Volume 2 Issue 3 (March 2012) http://www.ijecee.com/ https://sites.google.com/site/ijeceejournal/. Tin Hninn Hnnn Maung et. al., Real-Time Hand Tracking and Gesture Recognition System Using Neural Networks, World Academy of Science, Engineering and Technology 26 2009. Shivamurthy.R.C, Dr. B.P. Mallikarjunaswamy and Pradeep Kumar B.P., Dynamic Hand Gesture Recognition using CBIR, International Journal of Computer Engineering & Technology (IJCET), Volume 4, Issue 3, 2013, pp. 340 - 352, ISSN Print: 0976 6367, ISSN Online: 0976 6375. Ankita Chavda and Sejalthakkar, Appearance Based Static Hand Gesture Alphabet Recognition, International Journal of Advanced Research in Engineering & Technology (IJARET), Volume 4, Issue 4, 2013, pp. 107 - 117, ISSN Print: 0976-6480, ISSN Online: 0976-6499. Shaikh Shabnam Shafi Ahmed, Dr.Shah Aqueel Ahmed and Sayyad Farook Bashir, Fast Algorithm for Video Quality Enhancing using Vision-Based Hand Gesture Recognition, International journal of Computer Engineering & Technology (IJCET), Volume 3, Issue 3, 2012, pp. 501 - 509, ISSN Print: 0976 6367, ISSN Online: 0976 6375.

[2]

[3] [4] [5] [6]

[7]

[8]

[9]

[10] [11]

[12]

[13]

33