Recent Researches in Computer Science

Novel Approach to Natural Child Head and Hand Gestures using

Roll and Slide Maximum Value Algorithm (RSMV)
MAHMOUD Z. ISKANDARANI
Faculty of Science and Information Technology
Al-Zaytoonah University of Jordan
P O BOX: 911597, Post Code: 11191, Amman
JORDAN
m_iskandarani@yahoo.com

Abstract: - A novel algorithm for child head and hand movements is designed. The proposed algorithm allows
the monitoring and analysis of a child's behavior based on a correlation between facial expressions, hand
inclination and angle positions. The algorithm system maps a predefined coordinates for the child and results in
a computable geometry which is fed to an intelligent analysis system.

Key-Words: - Head Gesture, Hand Gesture, Algorithm, Image Conversion, Matrix, Classification

1 Introduction
Gesture recognition is an increasingly growing work
area receiving, especially throughout the research
fields of sign language recognition, security, human
computer interaction, and intelligent cognitive
systems. Human gesture is a spectrum that ranges
from sign languages, through universal symbols, to
natural and unconscious body moves. Gesture
recognition techniques focus mainly on Sign
language recognition and classification issues, while
examining closely hand localization and tracking,
and on various feature extraction techniques related
to hand modeling and related dynamical analysis,
with computer vision, and pattern recognition
implementations for feature extraction from image
sequences. Classification schemes involve several
methods and include neural networks and variants,
hidden Markov models and variants, principal
component analysis, and numerous other machine
learning methods or combinations [1-5].
Human face is a rich source of nonverbal
information. Indeed, not only it is provides identity
information but it also presents ideas to understand
social feelings and can be fundamental in revealing
state of mind through social signals. Facial
expressions form important part of human social
interaction. Communication expresses ideas that are
visualized in our minds by using words integrated
with nonverbal behaviors. Hence, body language
and verbal messages are used in complementary
roles, to present clearer messages. Face acts as a
non-verbal communication channel conveying the
emotional content of our messages. In general,
gestures such as eye and head movements, body
movements, facial expressions and touch constitute
the non- verbal message types of our body language.
In this paper a novel algorithm that analyses hand,
head, and face changes is presented. . We present a
simple and novel approach that tracks several key
Head, Facial and Hand landmarks, extract feature
sequences that is used to classify the captured
images into one of main emotional states like
neutral, happiness, and sadness [6-10].
2 Background
Emotion recognition in image or video based data is
an integral component in high-level computer vision
applications such as interactive computing, and
intelligence systems. Classification of human bodily
expressions into emotional states is an important
problem in computer vision. Extracting information
is a continuous path from object recognition toward
object understanding, enabling a flexible and
interactive monitoring environment. Head and Hand
actions coding, is a process by which judgment on
emotional states is carried out based on actual
movements which is an important component in
building interactive image monitoring systems.
Child wellbeing relies on the ability of responsible
people to maintain constant awareness of the child's
environment as he lives and play. As children faces
obstacles and reacts naturally in sometimes
misunderstood way, the guardian must be aware of
the change in behavior and be ready to react

ISBN: 978-1-61804-019-0 45
 Recent Researches in Computer Science

appropriately as he might not be present when the  IM 11 IM 12 IM 13 ... IM 1m 

child's actions occur [11-17].  IM IM 22 IM 23 ... IM 2 m 
 21
…(2)
Although people have an astounding ability to cope  . . . . . 
IM final = 
with these changes, a guardian is fundamentally  IM Re f1 IM Re f 2 IM Re f3 . IM Re f m 
limited by what he can observe at any one time.  . . . . . 
When a guardian fails to notice a change in the  
 IM n1 IM n 2 IM n 3 ... IM nm 
child's behavior or the effect of new environment,
there is an increased potential for an alarming
change in course which subsequently will end in a  R11 R21 1 ... Rn1 
R R22 1 ... Rn 2 
collision. It is reasonable to assume that this danger  12 …(3)
could be mitigated if the guardian is notified when .R .R23 1 . Rn 3 
Roll =  13 
these situations arise using a novel image gesture  . . . . . 
algorithm presented in this work.  . . . . . 
 
 R1m R2 m 1 ... Rnm 

3 RSMV Algorithm
Where Rij: indicates the gesture and its average
The presented system takes in image data. The sequence value, so for example R11: Gesture position
output of the system is information regarding the 1 (Fig. 2) and its average first sequence.
detected emotion of the subject. This is
accomplished using the developed RSMV algorithm  0 0 0 ... 0 
 0 0 0 ... 0 
using a template-matching method.   …(4)
 0 0 0 ... 0 
Slide =  
Figure (1) shows the grid used to map the body head  . . . ... . 
 . . . ... . 
and hand gestures. The grid is scanned in one-  
dimension multiple times starting from a certain  ( R1m : 1) ( R2 m : 1) 1 ... ( Rnm : 1) 
location, then returning back to the location below
the starting point (y+∆y).

y
Tracing Direction
L1
M c = [(C1 C2 1 ... Cn ] …(5)
L2

Now taking into account that the child's gesture will

not be known in advance, this means that the
substituted values in the matrix regarding type of
Ln
movement will not be known in advance hence, the
rows in equation (2) could be arranged both in terms
x
of action and its instance of occurrence in a number
Fig.1: General Sampling Gird of ways according to which sequence of actions
have taken place which can follow different
In this algorithm, each captured image is sorted into scenarios and can be represented by a probability
a specific matrix (IMi, i=1, k) before all matrices are function of matrix elements arrangements of
re-organized in sequences , then the resulted captured gestures. However, the sequence of events
sequences are grouped into a final Matrix (IMfinal) is associated with stored captured image in a
where the Roll and Slide takes place to produce the database which in turn preserves the integrity of the
classification row matrix (Mc). This is illustrated in classification and prediction system.
the following equations:
4 Results
 d11 d12 d13 ... d1m 
d d 22 d 23 ... d 2 m 
 21 Figures 2-6 demonstrate gestures by Dina
 . . . . .  ...(1) Iskandarani used to test the algorithm, while Figures
IM i =  
 . . . . .  7-11 show the converted images sequences.
 . . . . . 
 
 d n1 dn2 d n3 ... d nm 

ISBN: 978-1-61804-019-0 46
 Recent Researches in Computer Science

Fig.2: Reference Image (position 3-R31). Fig.5: Hands in and Head up gesture Image
(Position 4-R41).

Fig.3: Hand out and Head up gesture Image Fig.6: Hands down and Head down gesture Image
(Position 1-R11). (Position 5-R51).

989209
1000000
888512
900000 813066
Pixel Distribution -

800000
First Sequence

700000
600000 514033 521002
500000
400000
300000
200000
100000
0
Position 1 Position 2 Position 3 Position 4 Position 5

Head & Hand Gestures

Fig.7: First Sequence Level for Head & Hand

Gestures.

2000000 1803236 1748905

1800000
1574803
Second Sequence
Pixel Distribution -

1600000 1404551 1447574

1400000

Fig.4: Hand up and Head up gesture Image 1200000

1000000

(Position 2-R21). 800000

600000
400000
200000
0
Position 1 Position 2 Position 3 Position 4 Position 5

Head & Hand Gestures

Fig.8: Second Sequence Level for Head & Hand

Gestures.

ISBN: 978-1-61804-019-0 47
 Recent Researches in Computer Science

250000
239591
Applying the RSMV algorithm is illustrated in the
Pixel Distribution -
Third Sequence

200000

150000
144297 following matrices where:
100000
R11: Hand: Out, Head: Up, Face: Happy.
50000 23104 31574 25579
R21: Hand: Up, Head: Up, Face: Happy.
0
Position 1 Position 2 Position 3 Position 4 Position 5 R31: Hand: Straight, Head: level, Face: Happy(
Head & Hand Gestures Reference).
R41: Hand: in, Head: Up, Face: Normal.
Fig.9: Third Sequence Level for Head & Hand R51: Hand: Down, Head: Down, Face: Unhappy.
Gestures.

700000 657236 (I) Final Image Matrix:

600000 536726
Pixel Distribution -
Fourth Sequence

523301
494275
500000 458506
400000  813066 989209 514033 888512 521002 
300000 1803236 1574803 1404551 1748905 1447574
200000  
100000 IM final =  23104 31574 144297 25579 239591 
0  
Position 1 Position 2 Position 3 Position 4 Position 5
 494275 523301 657236 458506 536726 
Head & Hand Gestures  12048 26842 425612 24227 400836 
Fig.10: Fourth Sequence Level for Head & Hand
Gestures. (II) Roll Sequence:

1.58 1.92 1 1.73 1.04 

450000 425612
400836 1.28 1.12 1 1.25 1.03 

400000
Pixel Distribution -
Fifth Sequence

350000
300000 Roll 1= 0.16 0.22 1 0.18 1.66 
250000  
200000
150000
0.75 0.80 1 0.70 0.82
100000 0.03 0.06 1 0.05 0.9 
50000 12048 26842 24227
0
Position 1 Position 2 Position 3 Position 4 Position 5

Head & Hand Gestures 1.58 1.92 1 1.73 1.04 

1.28 1.12 1 1.25 1.03 
Fig.11: Fifth Sequence Level for Head & Hand 
Roll 2 = 0.16 0.22 1 0.18 1.66 
Gestures.  
0.75 0.80 1 0.70 0.82
0.03 0.06 1 0.05 0.9 
5 Analysis and Discussion
1.58 1.92 1 1.73 1.04 
1.28 1.12 1 1.25 1.03 
Visual gesture sequences tend to have distinct 
internal sub-structure and exhibit interesting Roll 3 =  0 0 1 0 1.66 
 
dynamics between individual gestures. This 0.75 0.80 1 0.70 0.82
developed visual gesture recognition algorithm can 0.03 0.06 1 0.05 0.9 
capture both sub-gesture patterns and dynamics
between gestures 1.58 1.92 1 1.73 1.04 
1.28 1.12 1 1.25 1.03 
Figures 7-11, show a five sequence per image 
Roll 4 =  0 0 1 0 1.66 
illustrations for a total of five gestures to give  
overall 25 sequences. Such individual comparison  0 0 1 0 0 
of sequences per gestures are good as initial 0.03 0.06 1 0.05 0.9 
indications and act as a base for the RSMV
algorithm that mine through them to produce a 1.58 1.92 1 1.73 1.04
concrete classification and correlation to gesture 1.28 1.12 1 1.25 1.03

images. The illustrated sequences show an initial Roll 5 =  0 0 1 0 1.66
difference in the converted images per gesture  
 0 0 1 0 0 
around a reference gesture.  0 0 1 0 0 

ISBN: 978-1-61804-019-0 48
 Recent Researches in Computer Science

(III) Slide Sequence: M N=2 N=3 N=4 N=5 N=6 N=7 N=8 N=9 N=10
1 0 0 0 0 0 0 0 0 0
 0 0 0 0 0 
1.58 1.92 2 1 2 3 4 5 6 7 8 9
 1 1.73 1.04
3 2 4 6 8 10 12 14 16 18
Slide 1=  0 0 1 0 1.66 4 3 6 9 12 15 18 21 24 27
 
 0 0 1 0 0  5 4 8 12 16 20 24 28 32 36
 0 0 1 0 0  6 5 10 15 20 25 30 35 40 45
7 6 12 18 24 30 36 42 48 54
 0 0 0 0 0  8 7 14 21 28 35 42 49 56 63
 0 0 0 0 0  9 8 16 24 32 40 48 56 64 72

Slide 2 = 1.58 1.92 1 1.73 1.66 10 9 18 27 36 45 54 63 72 81
  Table 1: Optimizing number of image sequences for
 0 0 1 0 0 
 0 0 1 0 0 
a given number of gestures

Table 1 gives by means of symmetry cancellation

 0 0 0 0 0 
 0  the optimum number of operations per
 0 0 0 0  recommended maximum number of sequences per
Slide 3 =  0 0 0 0 0  image for a given number of gestures, While Figure
 
 1. 58 1. 92 1 1.73 1.66 12 illustrate such selection curve.
 0 0 1 0 0  Number of Gestures Versus Optimum Operations Size
90
Optimum Operations Size

80
70
60
 0 0 0 0 0  50

 0 0 0 0 0  40

  30
20
Slide 4 =  0 0 0 0 0  10
  0

 0 0 0 0 0  0 1 2 3 4 5 6 7 8 9 10 11

1.58 1.92 1 1.73 1.66 Number of Gestures

Fig. 12: Relationship between Gestures and

(IV) Classification Row Matrix: Operations Size.

M c = [(1.58 1.92 1 1.73 1.66 ] It is clear from the table that the maximum number
of sequence is equal to the number of gestures. This
The classification row matrix clearly discriminate indicates that each gesture has a unique sequence
between different gestures and is stored in a level that classifies it within a predetermined
database that correlate gesture images to matrix number of sequences in a converted image matrix.
values for future classification. In addition, the plotted curve follows a power law.

The Optimum Operations Size (OOS) for Roll-Slide 6 Conclusions

follow the rule in equation 6 below.
OOS = ( N − 1)( M − 1) …(6)
N: number of sequences per image
M: number of gestures
Table 1 show the effect of the number of performed
gestures and number of sequences per gesture that
can lead to a correct classification on the overall
operational size. This is important in terms of
measuring the efficiency of the developed
algorithm.
The presented new system efficiently detects and
classifies head and hand gestures, and treats the
image as a lumped component to analyze child
mode and behavior. It is truly inexpensive, since
only a common computer with a cheap webcam is
needed, with no need for any kind of environmental
setup and calibration.
References:
[1] M. Yao, X. Qu, Q. Gu, T. Ruan, Z. Lou, Online
PCA with Adaptive Subspace Method for Real-
Time Hand Gesture Learning and Recognition,
WSEAS TRANSACTIONS on COMPUTERS,
Vol.6, No.9, 2010, pp. 583-592.

ISBN: 978-1-61804-019-0 49
 Recent Researches in Computer Science

[2] X. Wang, X. Huang, H. Cai, Xia Wang, A [12] Z. Pirani, M. Kolte, Gesture Based Educational
Head Pose and Facial Actions Tracking Software for Children with Acquired
Method Based on Efficient Online Appearance [13] Brain Injuries, International Journal on
Models, WSEAS TRANSACTIONS on Computer Science and Engineering, Vol.2,
INFORMATION SCIENCE and No.3, 2010, pp. 790-794.
APPLICATIONS, Vol.7, No.7, 2010, pp. 901- [14] J. Appenrodt, A. Al-Hamadi, B. Michaelis,
911. Data Gathering for Gesture Recognition
[3] E. Murphy-Chutorian, Head Pose Estimation Systems Based on Single Color-, Stereo Color-
and Augmented Reality Tracking: An and Thermal Cameras, International Journal of
Integrated System and Evaluation for Signal Processing, Image Processing and
Monitoring Driver Awareness, IEEE Pattern Recognition, Vol.3, No.1, 2010, pp. 37-
Transactions On Intelligent Transportation 50.
Systems, Vol. 11, No. 2, 2010, pp. 300-311. [15] J. Watchs, Gaze, Posture and Gesture
[4] D. Sierra1, J. Enderle1, 3D Dynamic Modeling Recognition to Minimize Focus Shifts for
of the Head-Neck Complex for Fast Eye and Intelligent Operating Rooms in a Collaborative
Head Orientation Movements Research, Support System, Int. J. of Computers,
Modelling and Simulation in Engineering, Communications & Control, Vol.V, No.1,
Volume 2011, Article ID 630506, 12 pages 2010, pp. 106-124.
Vol.2011, No. 630506, 2011, pp. 1-12. [16] H. Harshith, K.Shastry, M. Ravindran, M.
[5] R. Terrier1, N. Forestier, F. Berrigan, M. Srikanth, N. Lakshmikhanth, survey on various
Germain-Robitaille, M. Lavallière, N. gesture recognition techniques for interfacing
Teasdale, Effect of terminal accuracy machines based on ambient intelligence,
requirements on temporal gaze-hand International Journal of Computer Science &
coordination during fast discrete and reciprocal Engineering Survey, Vol.1, No.2, 2010, pp. 31-
pointings, Journal of NeuroEngineering and 42.
Rehabilitation, Vol.8, No.10, 2011, pp. 1-12. [17] G. Caridakis, K. Karpouzis, A. Drosopoulos, S.
[6] L. Chan, S. Salleh, C. Ting, Face Biometrics Kollias, SOMM: Self organizing Markov map
Based on Principal Component Analysis and for gesture recognition, Pattern Recognition
Linear Discriminant Analysis, Journal of Letters, Vol.32, 2010, pp. 52-59.
Computer Science, Vol.6, No.7, 2010, pp. 693-
699.
[7] R. Asamwar, K. Bhurchandi, A. Gandhi,
Successive Image Interpolation Using Lifting
Scheme Approach, Journal of Computer
Science, Vol.6, No.9, 2010, pp. 969-978.
[8] S. Mohamed M. Roomi, R. Jyothi Priya, H.
Jayalakshmi, Hand Gesture Recognition for
Human-Computer Interaction, Journal of
Computer Science, Vol.6, No.9, 2010, pp.
1002-1007.
[9] S. Sakthivel, M. Rajaram, Improving the
Performance of Machine Learning Based Multi
Attribute Face Recognition Algorithm Using
Wavelet Based Image Decomposition
Technique, Journal of Computer Science,
Vol.7, No.3, 2011, pp. 366-373.
[10] M. Iskandarani, A Novel Approach to Head
positioning using Fixed Center Interpolation
Net with Weight Elimination Algorithm,
Journal of Computer Science, Vol.7, No.2,
2011, pp. 173-178.
[11] M. Iskandarani, Head Gesture Analysis using
Matrix Group Displacement Algorithm,
Journal of Computer Science, Vol.6, No.11,
2010, pp. 1362-1365.

ISBN: 978-1-61804-019-0 50