Professional Documents
Culture Documents
A Novel Predictive Approach To Prosthetic Control For Digit Amputations Using Grip Classification
A Novel Predictive Approach To Prosthetic Control For Digit Amputations Using Grip Classification
Abstract—In this paper, the authors present a novel method authors define a use-pattern to be a set of digit positions that
of prosthetic control for amputees who are missing a subset facilitate some task. A precise implementation of a low-cost
of their digits. The core of the system is a personalized set of system that controls a prosthetic to assume even a subset of
basic configurations of the human hand to facilitate given tasks, these use-patterns will be more valuable than an imprecise
referred to as use-patterns. In the concept, the system first senses implementation of all of them.
the positions of the remaining, healthy digits using resistive flex
sensors mounted onto a glove. Then, an algorithm using a Z-
score metric compares the flex sensor readings to a large set In the case of a partial hand amputation, it is possible
of training data to determine which of several use-patterns the to predict, with a high degree of confidence, which one of
current situation corresponds to. A control system can then move these common use-patterns is being assumed by measuring
a bionic digit into the intended position corresponding to the use- the locations of the amputee’s healthy digits.
pattern detected, completing the grip. MATLAB simulations were
run to determine the accuracy of this concept on five exemplary This information can be used as a foundation to create
use-patterns. The success rate with one amputated digit was an inexpensive prosthetic device. This paper explores one
96.5% and the success rate with two amputated digits was 92.5%. approach to positional detection, employing resistive flex sen-
This demonstrates the feasibility of a simple, intuitive, and low- sors mounted onto a glove, a protocol that has precedent
cost prosthetic control system based on this premise that could
in applications of prosthetic/robotic control and the task of
deliver even higher success rates when personalized with self-
learning algorithms. measuring hand motions more generally [7], [8], [9], [10], [11],
[12].
Index Terms—Prosthetics, Robotics, Predictive, Grip,
Classification, Hand, Digit, Amputation, Resistive Flex The approach taken differs from others employing flex
Sensors, Z Score sensors for prosthetic control [9], [10], [11], [12] in the
following ways:
I. I NTRODUCTION
The field of (partial) hand prosthetics has been an active 1) Prosthetic control is solely based on flex sensor input,
area of development for several decades. Substantial techno- rather than synthesis with EMG signals.
logical advances have been made in the most common mode of 2) Prosthetic control is based on localized hand posi-
control, electromyography (EMG) [1], [2], [3], [4], [9], [10], tional data, as opposed to impulses from the contrac-
[11], [12]. tion of a distant muscle.
Conventional EMG approaches to prosthetic control em- This renders the approach a uniquely intuitive way to
ploy a physical trigger movement by another part of the body. achieve prosthetic control, when partial data is available.
Electrical potential in this trigger region increases upon muscle
contraction, which is measured by a set of electrodes. The When the glove is being used, the desired position of a
degree and nature of the contraction of the trigger region is prosthetic digit can be deduced by comparing the positions of
mapped to motion of the prosthetic. the healthy digits to the training data for different use-patterns.
This can be non-intuitive, as the user must learn to activate A prosthetic digit could then be moved to the intended position.
the prosthetic by contracting a different muscle group. Also, Importantly, the database of use-patterns most commonly used
while contracting the trigger region for a different purpose, can be personalized to fit the needs of the amputee in question.
false positives may be detected. Thus, amputees often find it
hard to work with traditional EMG approaches [5]. All the 31 discrete partial hand amputations are simulated
by applying the algorithm on data streams with some digits
In actual day-to-day usage of the hand, there is a lim- ignored. (Note that the number 31 comes from the total 25 =
ited number of use-patterns that are employed commonly; a 32 subsets of a five-element set, then excluding the empty
classification of hand manipulations can be found in [6]. The set.) To accomplish this, the Z-score value for that digit –
1 These
i.e. the number of standard deviations it is from the mean –
authors contributed equally to this work.
is defaulted to zero, contributing nothing to the defined cost
A. Ganesh and V. Vale are both students at Stanford University metric. With promising success rates, a proof-of-concept for
Online High School, Pre-Collegiate Studies. such a system is demonstrated.
724
Read
data from
sensors
Take subset
of data
from binary
integer
725
mented be N . C. Partial Amputation Simulation: Subset Detection
The steps of the algorithm shown in Figure 2 are as follows. To simulate use-pattern detection for an amputee, flex
sensor channels (there were five of these – one for each digit)
The matrix M represents the mean values of each digit for were deliberately ignored to see if use-patterns could still be
each use-pattern, derived from the 250 data trials mentioned reliably determined.
earlier. In the matrix below, µij represents the average value
of the jth digit in the ith use-pattern. To generate test data, the average values determined pre-
viously were distorted using Gaussian noise with a mean of
µ11
µ12 ··· µ15
zero and standard deviation defined elementwise.
µ21 µ22 ··· µ25 The following is the N × 5 matrix, G, of the noisy trial
M=
... .. .. .. values – where Gi,j = µij +σij ·randnij (), and randnij ()
. . .
µN 1 µN 2 ··· µN 5 denotes independent and normally distributed random variables
with a mean of 0 and standard deviation of 1. In particular,
note that the values of randnij () are different, representing
An N × 5 matrix of standard deviations, S, was empiri- different executions of the randn() function in MATLAB.
cally obtained for each of the 5 digits in each of the N use-
patterns from the 250 trials. In the matrix below, σij represents
the standard deviation of the jth digit in the ith use-pattern. µ11 + σ11 · randn11 ()
··· µ15 + σ15 · randn15 ()
µ21 + σ21 · randn21 () ··· µ25 + σ25 · randn25 ()
G= .. .. ..
σ11 σ12 ··· σ15
. . .
σ21 σ22 ··· σ25 µN 1 + σN 1 · randnN 1 () · · · µN 5 + σN 5 · randnN 5 ()
S=
... .. .. ..
. . .
Costs were calculated to predict a use-pattern for all
σN 1 σN 2 ··· σN 5
possible binary amputations; that is, for all possible sets of
healthy and amputated digits. For simplicity, it was assumed
To calculate the use-pattern for a particular vector of analog that these were the only classes of amputations, though in
readings for each digit x = ( x1 x2 · · · x5 ), a cost principle one could define a “fractional” amputation in which
metric was defined. a residual digit is in part functional.
A matrix of Z-scores, the number of standard deviations Let an Ek subset denote a subset of the five digits with
a datapoint is from the mean, serves as a measure of distance k-elements – i.e., k healthy digits. Each possible amputation
normalized elementwise by the standard deviation σij . The forms a one-to-one correspondence with each of the 25 –1 = 31
|µij − xj | subsets of the five digits, excluding the subset with all the
matrix was defined by Zi,j = .
σij digits amputated. The number of possible amputations with k
remaining healthy digits is given by the binomial coefficient
5
|µ11 − x1 | |µ12 − x2 | |µ15 − x5 |
k , and is equal to the number of permutations of the binary
··· integer with k ones and 5 − k zeros.
σ11 σ12 σ15
|µ21 − x1 | |µ22 − x2 | |µ25 − x5 |
···
In the scoring algorithm, for a binary string in the decimal
Z= σ21 σ22 σ25 range from 1 to 31, inclusive, a “1” represented a healthy digit,
.. .. .. ..
and a “0” represented an amputated digit. For example, the
. . . .
E3 subset {1, 2, 5} can be represented as the binary number
|µN 1 − x1 | |µN 2 − x2 | |µN 5 − x5 |
··· 11001.
σN 1 σN 2 σN 5
If digit j is amputated, then Zi,j is defaulted to 0 for all
The cost vector is calculated by taking the sum of all i ∈ {1, 2, . . . , 5}. This does change the expected magnitude of
elements in each row. the cost metric, but because this distance of zero is taken for
the j-th digit in all use-patterns, the cost metric is normalized
Thus a 1 × N cost vector r = ( r1 r2 ··· rN ) was
for fair comparison between them.
defined as:
In the tests, simulation of all 31 subsets on the randomly
5
X generated matrix G was executed 300,000 times in MATLAB.
ri = Zi,j A trial was considered successful if the correct use-pattern was
j=1
predicted for the corresponding distorted average value.
The vector r provides a normalized measure of the distance III. E XPERIMENTAL R ESULTS
between a vector of analog flex values and each use-pattern.
The graphs in Figure 4 depict the harvested mean values
Thus the index of the use-pattern with the lowest cost, Umin dist ,
for each i, j pair and their respective standard deviations σij
is the predicted use-pattern.
as error bars. A high standard deviation corresponds to a high
amount of variance in the 50 trials executed for that i, j pair.
Umin dist = argmin r(i) For example, one may observe that the thumb in the wallet
i use pattern has an expectedly high standard deviation, as the
726
position of the thumb in the plane of the wallet is irrelevant to [8] J. Lavery, B. Kent, E. D. Engeberg, “Biologically Inspired Grasp Primi-
a tight grip. Due to the normalization in the calculation of the tives for a Dexterous Robotic Hand to Catch and Lift a Sphere,” in Int.
Conf. on Control, Automation and Systems, Jeju Island, Korea, 2012, pp.
cost metric, a high variation is compensated for by the high 1710-1715.
standard deviation. N = 5 was used for this simulation.
[9] R. J. Smith, F. Tenore, D. Huberdeau, R. Etienne-Cummings, N. V.
Simulations with all possible binary digit amputations Thakor, “Continuous Decoding of Finger Position from Surface EMG
Signals for the Control of Powered Prostheses,” in 30th Ann. Int. IEEE
yielded promising success rates for each class of amputation, EMBS Conf., Vancouver, Canada, 2008, pp 197-200.
shown in Table II. For an E4 subset - a single amputated [10] N. Araki, K. Inaya, Y. Konishi, K. Mabuchi, “An Artificial Finger Robot
digit - the success rate was 96.45%. For an E3 subset - two Motion Control Based on Finger Joint Angle Estimation from EMG
amputated digits - the success rate was 92.52%. While a higher Signals for a Robot Prosthetic Hand System,” in Proc. of the Int. Conf.
number of amputations reduces the amount of information on Advanced Mechatronic Systems, Tokyo, Japan, 2012, pp. 109-111.
provided by the device, they still yield encouraging success [11] R. Vinjamuri, Z. Mao, R. Sclabassi, M. Sun, “A Novel Architecture for
rates. Strikingly, an E1 subset - only one healthy digit - still the Design of Prosthetic and Robotic Hand,” in Proc. of the IEEE 32nd
Ann. Northeast Bioengineering Conference, Easton, PA, USA, 2006, pp.
provided a success rate of 65.05%. 163-164.
[12] C. Cipriani, C. Antfolk, C. Balkenius, B. Rosén, G. Lundborg, M.C.
IV. C ONCLUSION Carrozza, F. Sebelius, “A Novel Concept for a Prosthetic Hand With a
Bidirectional Interface: A Feasibility Study,” IEEE Trans. Biomed. Eng.,
A total of five common use-patterns were implemented vol. 56, no. 11, pp. 2739-2743, 2009.
in the detection algorithm. The cost calculation algorithm
accounts for differences in digit variance in different use-
patterns by normalizing by the standard deviation in the Z-
score matrix.
In future work, a more sophisticated machine learning
algorithm, which is being developed, that dynamically adjusts
the cost metric could increase the number of use-patterns that
can successfully be detected. In addition, to provide additional
resolution, the flex sensors could be supplemented with other
sensors like gyroscopes and accelerometers.
The success rates provided by this simple algorithm mo-
tivate implementation of this system with a real prosthetic
device, providing a uniquely intuitive, inexpensive mode of
control for prosthetics.
ACKNOWLEDGMENT
The authors would like to thank Dr. Lev Sofman from Bell
Labs for providing feedback improving the readability of this
paper.
R EFERENCES
[1] C. Castellini, P. van der Smagt, “Surface EMG in Advanced Hand
Prosthetics,” Biological Cybernetics, vol. 100, no. 1, pp. 35-47, Nov.
2008.
[2] S. A. Ahmad, A. J. Ishak, and S. H. Ali, “Review of Electromyographic
Control Systems Based on Pattern Recognition,” Intl Federation for
Medical & Biological Engineering Proc., vol. 35, pp. 556–559, 2011.
[3] E. Scheme and K. B. Englehart,“Electromyogram pattern recognition for
control of powered upper-limb prostheses: State of the art and challenges
for clinical use,” J. Rehabil. Res. Dev., vol. 48, no. 6, pp. 643–660, 2011.
[4] P. M. Pilarski, M. R. Dawson, T. Degris, J. P. Carey, K. M. Chan, J. S.
Hebert, R. S. Sutton, “Adaptive Artificial Limbs: A Real-Time Approach
to Prediction and Anticipation,” IEEE Robot. Automat. Mag., vol. 20, no.
1, pp. 53 - 64, March 2013.
[5] E.A. Biddiss, D. Beaton, T. Chau, “Consumer design priorities for upper
limb prosthetics,” Disability and Rehabilitation: Assistive Technology,
vol. 2, no. 6, pp. 346-57, 2007.
[6] I. M. Bullock, R. R. Ma, A. M. Dollar, “A Hand-Centric Classification of
Human and Robot Dexterous Manipulation,” IEEE Trans. Haptics, Sept.
2012. IEEE computer Society Digital Library. IEEE Computer Society,
http://doi.ieeecomputersociety.org/10.1109/TOH.2012.53
[7] L. Dipietro, A. M. Sabatini, “A Survey of Glove-Based Systems and
Their Applications,” IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol
38., no. 4, pp. 461-482, 2008.
727
TABLE II: Success Rates
(a) Use-Pattern: Gripping Water Bottle, i = 1 (b) Use-Pattern: Holding Car Keys, i = 2
Fig. 3: Photographs of a subject demonstrating the various use-patterns used in the algorithm.
728
Use-Pattern: Gripping Water Bottle Use-Pattern: Holding Car Keys
250 250
100 100
50 50
0 0
1 2 3 4 5 1 2 3 4 5
Digit j Digit j
(a) Sensor Values for Use Pattern 1: Gripping Water (b) Sensor Values for Use Pattern 2: Holding Car Keys,
Bottle, i = 1 i=2
200 200
Mapped Flex Value
150 150
100 100
50 50
0 0
1 2 3 4 5 1 2 3 4 5
Digit j Digit j
(c) Sensor Values for Use Pattern 3: Gripping Wallet, i = (d) Sensor Values for Use Pattern 4: Holding Toothbrush,
3 i=4
200
Mapped Flex Value
150
100
50
0
1 2 3 4 5
Digit j
Fig. 4: Sensor values for each use-pattern employed in the algorithm. The error bars represent ±σij .
729