See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/319475045

Proportional myoelectric prosthetic hand control using multi-regression model

estimator with pattern classifier selector

Conference Paper · March 2017

DOI: 10.1109/ARSO.2017.8025199

CITATIONS READS

2 289

3 authors, including:

Mostafa Ahmed Arafa Mohammed Ibrahim Awad

University of Lincoln Ain Shams University
1 PUBLICATION   2 CITATIONS    81 PUBLICATIONS   417 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

energy consumption measurement in transfemoral amputees View project

Adaptive prosthetic knee design View project

All content following this page was uploaded by Mohammed Ibrahim Awad on 20 October 2017.

The user has requested enhancement of the downloaded file.

 Proportional myoelectric prosthetic hand control using multi-
regression model estimator with pattern classifier selector
Mostafa A. Arafa*, Mohammed I. Awad, Member, IEEE, and Farid A. Tolbah

Abstract— Myoelectric hand prostheses using pattern
recognition control scheme lack simultaneous motion, and
perform robotic unnatural inter-pattern motion. Accordingly,
the use of regression models for the estimation of hand
kinematics proportionally to sEMG (surface electromyography)
signals has proved more simultaneous and natural motion. The
objective of this study is to introduce proportional speed control
on robotic hand motion, where each finger has its own estimator
model to achieve non-robotic performance of the hand. Each
finger has four regression models to cover the motion of the
finger over four patterns. A pattern recognition classifier is
trained to classify four hand gestures, accordingly, the
regression models of the fingers is to be altered according to the
classifier decision. Commercial sEMG sensing armband was
used in the acquisition of training data that can be used later in
the development of the prosthetic control system. The
reproduction of data for linear (least-square fitted model) and
non-linear (ANN) regression models are investigated, where the
ANN proved better reproducibility of finger speeds. The models
also are trained on reduced RMS features, where the selected
features are only the channels that are allocated over the active
muscles during performing the patterns which resulted
reproducibility of 89.27±1.92%. These results demonstrate the
robustness of the multi-regression models system over wide
range of motion.
Index terms – Regression models, sEMG, commercial,
myoelectric prosthetic hand, proportional myoelectric control.
I. INTRODUCTION
The appeal of myoelectric control for multifunctional
artificial limb prostheses backs to 1970’s [1], where pattern
recognition and signal parameterization approaches were used
to discriminate between the desired functions of the robotic
limb. Recently myoelectric control strategies have been
divided in pattern recognition and non-pattern recognition
based control strategies [2]. The later strategy includes
proportional control, which uses the strength of the muscle
contraction to determine the speed or force of the limb
function.
Since pattern recognition based systems performed
repeatable patterns of hand motion, they don’t satisfy the
simultaneous natural motion required to be performed by the
prosthesis to achieve day to day activities. Hence, kinematic
proportional control introduced a simultaneous natural control
of hand degrees of freedom by means of linear regression
models or artificial neural networks [3]. The kinematic
proportional control has been developed for 1 DOF wrist
motion and 1 DOF finger flexion and extension. However,
others [4] worked on the kinematic control of each finger
independently, creating an estimation model per finger.
*Mostafa A. Arafa, Mohammed I. Aawad and Farid A. Tolbah are with
the faculty of Engineering, Mechatronics department, Ain Shams University,
For the sake of accuracy improvement of kinematic
proportional control, L. Pan [5] introduced a 14 state space
models describing two joint variables of a finger, with a
classifier to select the appropriate two models for estimation
of joint variables.
Analogous to pattern recognition control schemes,
dimensionality reduction was studied for the proportional
controllers to improve estimation robustness and achieve
better computational performance [6].
Consequently, it was needed to use pattern recognition
based strategies to predict the motor function and a
proportional controller to control this function proportionally.
Proportional mutex control methods [7] included both discrete
and proportional elements, so that the grasping force can be
proportionally controlled along with the recognition of the
grasping pattern. With the increased interest in proportional
control, an online database of sEMG signals and hand
kinematics as well as force measurements was published [8].
The purpose of this study is to design a simultaneous
natural myoelectric controller for the prosthetic hand of trans-
radial amputees. Subsequently, the paper proposes a finger
based proportional speed controller using multi regression
model estimation. The system will depend on a pattern
recognition classifier to determine the hand gesture pattern for
better reproduction models over wide range of motion.
Accordingly, the classifier will select a regression model to
estimate the speed with which the robotic limb will move to
perform the recognized pattern. In this study commercial
sensors are used for the acquisition of sEMG and kinematic
data. The use of such a commercial sEMG wearable armband
advocates the low cost development of the myoelectric robotic
limb. Furthermore, linear (least-square fitted model) and non-
linear (ANN) regression models are assessed for the full range
of finger models along four hand gestures. Finally, the models
are trained with less features dependent on the active muscles
performing the pattern, to improve robustness of the
estimators.
II. METHODS
This work aimed at controlling prosthetic hand gestures
speeds proportionally to achieve simultaneous motion of the
robotic limb. Therefore, a hybrid method of pattern
recognition and proportional regression models was developed
to control finger speeds during inter-gesture motions.
A. Experimental Setup
The experimental setup architecture (Figure 1) includes a
commercial sEMG gesture control armband made by Thalmic
labs named Myo – typically uses sEMG signals to perform
11517 Cairo, Egypt. Corresponding author: Mostafa A. Arafa (e-mail:
mostafa.a.arafa91@ eng.asu.edu.eg).

Figure 1 Experimental setup. Myo armband streams raw sEMG
data from the user's forearm, Leapmotion sensor streams hand
kinematic data from dual infrared cameras. The GUI is designed
to log data from both sensors during performing of the training
session, with graphical feedback displays on the screen to assure
data continuity and acceptable readings.
gesture control on PC applications, which is used in this study
to stream sEMG raw data at 200 Hz. The armband consists of
eight sEMG electrodes placed equidistantly around the user’s
forearm. Via a Bluetooth 4.0 connection the sEMG data are
streamed to a Windows operating PC for training of the
classifier and regression models. Consequently, for the sake of
proportional control of pattern speeds, finger-tip speeds were
recorded using Leapmotion sensor – a commercial kinematic
sensor developed for desktop control – depending on dual
infrared cameras capturing frames of hand motion.
Leapmotion sensor identifies hand kinematics via image
processing means, consequently it streams finger-tip speeds
with 50 Hz rate via USB connection with the PC. sEMG and
finger-tip speeds are recorded simultaneously from both
devices via a GUI developed using MATLAB R2013b, where
a single button to start recoding both signals and visually plot
data as user feedback.
B. Subjects
Five able bodied adult volunteers performed the training
sessions with ages of 19 – 56 (with mean ages of 30.4) years
old males. Each subject performed five repeated recordings for
each hand gesture divided into two sessions. All experimental
procedures were conducted in accordance to the guidelines of
local institutional ethic committee.
Figure 2 A. The experimental setup showing the GUI and the visual
feedback to the user. B. The subject’s posture relative to the
kinematic sensor.
C. Experimental Protocol
In this work, all subjects performed the training patterns
with their right hands, they were required to rest their elbows
onto the same level of the kinematic sensor. The graphical
interface was designed as a visual feedback for the
experimenter, to make sure about data continuity and that the
user places his hand in the capturing zone of the kinematic
sensor – 150° width capturing zone (Figure 2). The GUI allows
the experimenter to start recording data and stops when the
subject ends performing the specified gesture. Moreover, raw
sEMG data recorded as well as finger-tips speeds are visually
plotted vs. time. The subjects were asked to perform five
repeated recordings for the same hand movement gesture, with
altering the speed of each recording, starting from fastest speed
they can perform. After three recordings the subject was asked
to take off the sEMG sensor and have a rest for ten minutes,
then get back for two more recordings. Such condition was
specified to consider electrode shift effect on the robustness of
data.
D. Signal Processing
To achieve simultaneous trans-gesture motion for the
artificial limb, it’s required to alternate the finger-tip speed
while moving from one pattern into another several times a
second. Accordingly, sEMG signals were segmented into
250ms sliding windows, with 80% window overlap [9],
resulting 20 changes of finger speeds per second, which shall
satisfy natural simultaneous motion of the robotic limb. Thus
finger-tip speeds were similarly segmented with the same rate
and windowing scheme, to be synchronized with sEMG data,
to feed both sEMG and kinematic data with the same rate into
the controller training system.
The only features used for the regression model training
were the root mean squares of the eight channels sEMG signals
(1). While for the gesture classifier, more discriminant features
(wavelet lengths, slope sign changes and sixth order
autoregressive features) for the gestures recognition were fed
for the classifier trainer. RMS profiles exhibited similarity
with finger-tip speeds rather than other features, so they were
selected to be the input variables for the regression models for
speed prediction (Figure 3).
𝑅𝑀𝑆 = √ ∑𝑁
1 2
(1)
𝑖=1 𝑥𝑖
𝑁
Finger-tips speed used in the training is the scalar value of
the x, y and z components of velocity computed by the
kinematic sensor (2). For the prosthetic controller, the scalar
Figure 3 The profiles of RMS and Finger-tip speeds data for the
same recording with segmentation windows synchronized for open
hand pattern.
 Figure 4 Control and training sytems data flow, starting from signal acquisition, feature extraction, feeding training models and finally
the prosthetic controller.
values of speeds predicted from the regression models will tell
the controller the speed values to drive the motors, and the
gesture classifier will convey the directions of the motors
according to the gesture being performed.
𝑆𝑝𝑒𝑒𝑑 = √𝑉𝑥2 + 𝑉𝑦2 + 𝑉𝑧2 (2)
E. Regression Models
A regression model describes a relation between predictor
variables and a response variable by means of curve fitting.
They can relate between the predictor and response variables
linearly or non-linearly depending on the dependence between
the variables.
In this work both linear and non-linear regression models
were assessed for the description of the relationship between
the RMS of sEMG data and the finger-tip speeds
corresponding to these features. The linear curve fitting model
was estimated by the least square method, while the non-linear
model was based on Artificial Neural Network fitting
technique.
Figure 5 Measured vs. estimated speeds for index finger
regression model representing the motion in the range of the
open hand pattern, three regression models are trained; 1) Non-
linear model with features from 2,3 and 4 EMG electrodes, 2)
Non-linear model with features from the 8 EMG electrode and 3)
Linear model. Data used in this plot are from the validation
dataset.
F. Training and control systems
In this work it’s proposed to create a hybrid system
between regression models and a pattern recognition classifier.
The system is based on training the classifier to discriminate
between four different hand gestures (open hand, closed hand,
index pointing and pinch grasping). Consequently, every
single pattern has a corresponding group of five regression
models. To create reliable and robust models, each group is
only trained on the data labeled by its corresponding pattern,
e.g. for the open hand gesture, there are five regression models
only trained on open hand data logged from the subjects.
Therefore, the regression model is only valid as long as the
classifier decision is consistent on the same pattern. Whenever
the classifier changes the pattern, another group of models will
be selected to estimate the finger-tip speeds. Training data
contained holding positions for every gesture to consider zero
speeds estimations.
Finally, the control data to be handled to the prosthetic
controller are finger directions determined from the pattern
classifier, as well as the speeds needed to drive motors are
estimated from regression models’ finger-tip speeds by means
of inverse kinematics (Figure 4).
III. RESULTS
The reproducibility – the correlation between estimated
and measured data – of the speed estimators is to be assessed
for each regression model separately, to endorse the reliability
of the system overall. For the four groups of estimators with
five regression models per group, table 1 shows the range of
reproducibility indices R for every finger; the range represents
the minimum and the maximum values of R of each regression
model per finger.
To achieve better reproduction of speeds by speed
estimators, linear and non-linear regression models were
assessed and validated. Cross validation took place by dividing
data into 70% training and 30% validation, where validation
data are not used in the training of the models. R is the mean
value of the correlation between estimated and measured data
for testing and validation data sets, best values for R are shown
in table 1.
 TABLE 1 Moreover, it’s compatible with commercially available
microcomputers.
THE REPRODUCIBILTITY VALUES R FOR EACH FINGER SHOWING ONLY THE
MAXIMUM AND THE MINIMUM VALUES OF THE FOUR REGRESSION MODELS
V. CONCLUSION
Finger Rmax Rmin
This paper provides a hybrid sEMG control scheme for
Thumb 89.613% 87.236% sEMG controlled robotic prostheses, to achieve simultaneous
Index 90.543% 86.067% natural motion of the robotic limb and robust recognition of
the hand gesture. In this work training data were logged using
Middle 91.232% 89.066% commercial sEMG wearable armband named Myo, which can
Ring 90.221% 89.401% be used as the sEMG sensor for the prosthetic controller. For
the training of the speed proportional controllers, hand
Little 92.117% 87.207%
kinematics was captured using Leapmotion sensor – also a
commercial gesture controller for desktops.
For every single pattern there are dominating features
acquired from channels located directly over the working The methodology depends on training four regression
muscles of such pattern. Reducing the estimating features used models for each finger independently, and a classifier trained
for the training of the regression model resulted into better to discriminate between the four gestures specified. The
estimation as shown in Figure 5. Data in the figure represent classifier will select the appropriate regression model to
only index finger in the range of open hand pattern, and inputs estimate the finger speed, while the finger direction is to be
were reduced to take only RMS data from electrodes of determined by the gesture classifier.
numbers 2, 3 and 4 in the reduced inputs model. Reduction of The non-linear regression models based on ANN
the input RMS features is dependent on whether the pattern outperformed the linear models based on least square fitting.
uses the flexor or extensor muscles. Muscles positioning Moreover, reducing the number of inputs to the model proved
relative to sEMG electrode number and contribution in each better reproducibility compared to the model trained using all
pattern is illustrated in table 2 according to [10]. Same RMS data from all sEMG electrodes.
approach has been used to estimate the remaining regression
models for the rest fingers in the remaining patterns. VI. ACKNOWLEDGMENTS
TABLE 2 The authors would like to thank Aliaa R. Youssef Ph.D
and Esraa A. Eissa for their comments that greatly improved
REDUCED RMS FEATURES AND THE ACTIVE MUSCLES PEFROMING EACH the manuscript and Mohammed M. El Kammash for his help
PATTERN in conducting the training sessions.
Pattern Electrodes Active Muscles
Abductor pollicis longus
VII. REFERENCES
Open hand 2, 3 and 4 Extensor digiti minimi
Extensor digitorum [1] D. Graupe and W. K. Cline, "Functional Separation Of Emg Signals
Flexor digitorum superficialis Via Arma Identification Methods For Prosthesis Control Purposes,"
Close hand 1, 7 and 8
Flexor digitorum profundus IEEE Transactions on Systems, Man and Cybernetics, Vols. SMC-5,
Extensor digitorum no. 2, pp. 252-259, 1975.
Pointing 1, 2, 3, 4, 7 and 8 Flexor digitorum superficialis
[2] M. Hakonen, H. Piitulainen and A. Visala, "Current state of digital
Flexor digitorum profundus
signal processing in myoelectric interfaces and related applications,"
Flexor digitorum superficialis Biomedical Signal Processing and Control, vol. 18, pp. 334-359,
Pinch Grasp 1, 2, 7 and 8 Flexor digitorum profundus 2015.
Abductor pollicis longus
[3] N. Jiang, I. Vujaklija, H. Rehbaum, B. Graimann and D. Farina, "Is
accurate mapping of EMG signals on kinematics needed for precise
online myoelectric control?," IEEE Transactions on Neural Systems
IV. LIMITATIONS AND FUTURE WORK and Rehabilitation Engineering, vol. 22, no. 3, pp. 549-558, 2014.
Proportional regression models correlate finger kinematics [4] J. Ngeo, T. Tamei and T. Shibata, "Estimation of continuous multi-
DOF finger joint kinematics from surface EMG using a multi-output
to sEMG RMS features, consequently, only intact subjects can Gaussian Process," Conference proceedings : ... Annual International
contribute in the training dataset used for such a technique. Conference of the IEEE Engineering in Medicine and Biology
Accordingly, the kinematic proportionality was the aim of this Society. IEEE Engineering in Medicine and Biology Society. Annual
study, where only gripping-free hand gestures were used for Conference, vol. 2014, pp. 3537-3540, 2014.
the training of the prediction models. [5] L. Pan, D. Zhang, J. Liu, X. Sheng and X. Zhu, "Continuous
estimation of finger joint angles under different static wrist motions
The gesture recognition capacity can be extended to other from surface EMG signals," Biomedical Signal Processing and
gripping gestures, with proportional force estimators Control, vol. 14, pp. 265-271, 2014.
incorporated in the same manner into the prosthetic controller. [6] A. Krasoulis, K. Nazarpour and S. Vijayakumar, "Towards low-
dimensionsal proportional myoelectric control," Proceedings of the
For real-time control of the prosthesis, a microcomputer Annual International Conference of the IEEE Engineering in
will be used to do the recognition and estimation Medicine and Biology Society, EMBS, vol. 2015 Novem, pp. 7155-
computations. Accordingly, the sEMG sensor (Myo armband) 7158, 2015.
was selected to do the acquisition part, because it’s an [7] A. Fougner, Ø. Stavdahl, P. J. Kyberd, Y. G. Losier and P. A. Parker,
ergonomic and mobile commercial wearable device. "Control of Upper Limb Prostheses : Terminology and Proportional
 Myoelectric Control — A Review," IEEE Transactions on Neural
Systems and Rehabilitation Engineering, vol. 20, no. 5, pp. 663-677,
2012.
[8] M. Atzori, A. Gijsberts, C. Castellini, B. Caputo, A.-G. M. Hager, S.
Elsig, G. Giatsidis, F. Bassetto and H. Müller, "Electromyography
data for non-invasive naturally-controlled robotic hand prostheses.,"
Scientific data, vol. 1, 2014.
[9] L. H. Smith, L. J. Hargrove, B. A. Lock and T. A. Kuiken,
"Determining the optimal window length for pattern recognition-
based myoelectric control: Balancing the competing effects of
classification error and controller delay," IEEE Transactions on
Neural Systems and Rehabilitation Engineering, vol. 19, no. 2, pp.
186-192, 2011.
[10] F. H. Netter, Atlas of Human Anatomy, Professional Edition, 2011.

View publication stats