Download as pdf or txt
Download as pdf or txt
You are on page 1of 6

Proceedings of the 2013 International Conference on

Advanced Mechatronic Systems, Luoyang, China, September 25-27, 2013

A Novel Predictive Approach to Prosthetic Control


for Digit Amputations Using Grip Classification

Adithya C. Ganesh1 Vinjai Vale1


acganesh@stanford.edu vinjai@stanford.edu
adithya.cganesh@gmail.com vinjai.vale@gmail.com

Abstract—In this paper, the authors present a novel method authors define a use-pattern to be a set of digit positions that
of prosthetic control for amputees who are missing a subset facilitate some task. A precise implementation of a low-cost
of their digits. The core of the system is a personalized set of system that controls a prosthetic to assume even a subset of
basic configurations of the human hand to facilitate given tasks, these use-patterns will be more valuable than an imprecise
referred to as use-patterns. In the concept, the system first senses implementation of all of them.
the positions of the remaining, healthy digits using resistive flex
sensors mounted onto a glove. Then, an algorithm using a Z-
score metric compares the flex sensor readings to a large set In the case of a partial hand amputation, it is possible
of training data to determine which of several use-patterns the to predict, with a high degree of confidence, which one of
current situation corresponds to. A control system can then move these common use-patterns is being assumed by measuring
a bionic digit into the intended position corresponding to the use- the locations of the amputee’s healthy digits.
pattern detected, completing the grip. MATLAB simulations were
run to determine the accuracy of this concept on five exemplary This information can be used as a foundation to create
use-patterns. The success rate with one amputated digit was an inexpensive prosthetic device. This paper explores one
96.5% and the success rate with two amputated digits was 92.5%. approach to positional detection, employing resistive flex sen-
This demonstrates the feasibility of a simple, intuitive, and low- sors mounted onto a glove, a protocol that has precedent
cost prosthetic control system based on this premise that could
in applications of prosthetic/robotic control and the task of
deliver even higher success rates when personalized with self-
learning algorithms. measuring hand motions more generally [7], [8], [9], [10], [11],
[12].
Index Terms—Prosthetics, Robotics, Predictive, Grip,
Classification, Hand, Digit, Amputation, Resistive Flex The approach taken differs from others employing flex
Sensors, Z Score sensors for prosthetic control [9], [10], [11], [12] in the
following ways:
I. I NTRODUCTION
The field of (partial) hand prosthetics has been an active 1) Prosthetic control is solely based on flex sensor input,
area of development for several decades. Substantial techno- rather than synthesis with EMG signals.
logical advances have been made in the most common mode of 2) Prosthetic control is based on localized hand posi-
control, electromyography (EMG) [1], [2], [3], [4], [9], [10], tional data, as opposed to impulses from the contrac-
[11], [12]. tion of a distant muscle.

Conventional EMG approaches to prosthetic control em- This renders the approach a uniquely intuitive way to
ploy a physical trigger movement by another part of the body. achieve prosthetic control, when partial data is available.
Electrical potential in this trigger region increases upon muscle
contraction, which is measured by a set of electrodes. The When the glove is being used, the desired position of a
degree and nature of the contraction of the trigger region is prosthetic digit can be deduced by comparing the positions of
mapped to motion of the prosthetic. the healthy digits to the training data for different use-patterns.
This can be non-intuitive, as the user must learn to activate A prosthetic digit could then be moved to the intended position.
the prosthetic by contracting a different muscle group. Also, Importantly, the database of use-patterns most commonly used
while contracting the trigger region for a different purpose, can be personalized to fit the needs of the amputee in question.
false positives may be detected. Thus, amputees often find it
hard to work with traditional EMG approaches [5]. All the 31 discrete partial hand amputations are simulated
by applying the algorithm on data streams with some digits
In actual day-to-day usage of the hand, there is a lim- ignored. (Note that the number 31 comes from the total 25 =
ited number of use-patterns that are employed commonly; a 32 subsets of a five-element set, then excluding the empty
classification of hand manipulations can be found in [6]. The set.) To accomplish this, the Z-score value for that digit –
1 These
i.e. the number of standard deviations it is from the mean –
authors contributed equally to this work.
is defaulted to zero, contributing nothing to the defined cost
A. Ganesh and V. Vale are both students at Stanford University metric. With promising success rates, a proof-of-concept for
Online High School, Pre-Collegiate Studies. such a system is demonstrated.

978-1-4799-2519-3/13/$31.00 ©2013 IEEE

724
Read
data from
sensors

Take subset
of data
from binary
integer

(a) The five flex sensors depicted Empirical import Calculate


on a diagram of the right hand. The M and S elementwise
digits are referred to by their labels. matrices Z matrix
For example, in the matrices below,
the 2nd digit (j = 2) is the index (b) The right glove with the flex
finger. sensors mounted on it.
Create cost
vector r
Fig. 1: A schematic and photograph of the data sensing device.
in actual implementation
From
II. M ETHODS argmin,
move
A. System Specifications prosthetic
Five resistive flex sensors from Spectra Symbol –
R with a to location
flat resistance of 10kΩ, increasing in resistance upon bend up in use-
to a maximum of 110kΩ – were mounted on a leather right- pattern
hand glove to track the degree of bend of each digit shown in
Figure 1. A voltage divider circuit connects the sensors to five Fig. 2: Steps of Z-score distance algorithm.
analog pins on a microcontroller, which digitizes the analog
voltages to 10 bits each.
The flex value provided by the sensor is a voltage reading, a TABLE I: List of use-patterns
function of the resistance that changes upon bend, digitized to
10 bits. Due to slight differences in the way the sensors were i i-th use-pattern
mounted as well as intrinsic anatomical differences between 1 Gripping Water Bottle
the digits, the maximal and minimal flex values differ between 2 Holding Car Keys
digits. To correct for this, the flex values for each digit were
linearly normalized to a common range. 3 Gripping Wallet
4 Holding Toothbrush
Let FVmax(j) and FVmin(j) denote empirically deter-
mined maximum and minimum flex values of the jth digit, 5 Gripping Toothpaste
respectively. The following linear map function was used to
normalize the scale for a reading of xj from the jth digit,
where α and β denote the upper and lower bounds of the amputee. (Note that when the system is implemented on an
mapped values, respectively. amputee, a new, customized set of training data will be taken
from the amputee himself.)
Five use-patterns were chosen to demonstrate a proof
(xj − FVmin(j))(α − β) of concept of the system, but more can, in principle, be
map(xj , α, β) = +β
FVmax(j) − FVmin(j) added. Importantly, as the number of use-patterns increases, the
accuracy of the algorithm will eventually decrease, a tradeoff
The map which was employed was map(xj , 255, 0), as seen that needs to be consciously optimized in any implementation
on the charts at the end of the paper. of this system.
The purpose of the final two use-patterns, while not neces-
B. Use-Pattern Cost Metric sarily intrinsically useful themselves, was to simulate a general
grip of a stick-like object and a round object. Incidentally, a
Empirically determined average readings (training data) for
toothbrush and a toothpaste tube were selected to be used, but
each digit were determined by executing 50 trials for each
any object of the same proportions would have yielded similar
use-pattern in which a completely healthy, unamputated user
results.
assumed each use-pattern. The use patterns shown in Table I
and Figure 3 were chosen, relevant to common tasks for an In general, let the number of use-patterns being imple-

978-1-4799-2519-3/13/$31.00 ©2013 IEEE

725
mented be N . C. Partial Amputation Simulation: Subset Detection
The steps of the algorithm shown in Figure 2 are as follows. To simulate use-pattern detection for an amputee, flex
sensor channels (there were five of these – one for each digit)
The matrix M represents the mean values of each digit for were deliberately ignored to see if use-patterns could still be
each use-pattern, derived from the 250 data trials mentioned reliably determined.
earlier. In the matrix below, µij represents the average value
of the jth digit in the ith use-pattern. To generate test data, the average values determined pre-
viously were distorted using Gaussian noise with a mean of
µ11

µ12 ··· µ15
 zero and standard deviation defined elementwise.
 µ21 µ22 ··· µ25  The following is the N × 5 matrix, G, of the noisy trial
M=
 ... .. .. ..  values – where Gi,j = µij +σij ·randnij (), and randnij ()
. . . 
µN 1 µN 2 ··· µN 5 denotes independent and normally distributed random variables
with a mean of 0 and standard deviation of 1. In particular,
note that the values of randnij () are different, representing
An N × 5 matrix of standard deviations, S, was empiri- different executions of the randn() function in MATLAB.
cally obtained for each of the 5 digits in each of the N use-
patterns from the 250 trials. In the matrix below, σij represents
the standard deviation of the jth digit in the ith use-pattern. µ11 + σ11 · randn11 ()

··· µ15 + σ15 · randn15 ()

 µ21 + σ21 · randn21 () ··· µ25 + σ25 · randn25 () 
G= .. .. .. 
σ11 σ12 ··· σ15
   . . . 
 σ21 σ22 ··· σ25  µN 1 + σN 1 · randnN 1 () · · · µN 5 + σN 5 · randnN 5 ()
S=
 ... .. .. .. 
. . . 
Costs were calculated to predict a use-pattern for all
σN 1 σN 2 ··· σN 5
possible binary amputations; that is, for all possible sets of
healthy and amputated digits. For simplicity, it was assumed
To calculate the use-pattern for a particular vector of analog that these were the only classes of amputations, though in
readings for each digit x = ( x1 x2 · · · x5 ), a cost principle one could define a “fractional” amputation in which
metric was defined. a residual digit is in part functional.
A matrix of Z-scores, the number of standard deviations Let an Ek subset denote a subset of the five digits with
a datapoint is from the mean, serves as a measure of distance k-elements – i.e., k healthy digits. Each possible amputation
normalized elementwise by the standard deviation σij . The forms a one-to-one correspondence with each of the 25 –1 = 31
|µij − xj | subsets of the five digits, excluding the subset with all the
matrix was defined by Zi,j = .
σij digits amputated. The number of possible amputations with k
remaining healthy digits is given by the binomial coefficient
5


|µ11 − x1 | |µ12 − x2 | |µ15 − x5 |

k , and is equal to the number of permutations of the binary
··· integer with k ones and 5 − k zeros.
 σ11 σ12 σ15 
|µ21 − x1 | |µ22 − x2 | |µ25 − x5 |
 

 ···

 In the scoring algorithm, for a binary string in the decimal
Z= σ21 σ22 σ25  range from 1 to 31, inclusive, a “1” represented a healthy digit,
 .. .. .. .. 
and a “0” represented an amputated digit. For example, the

. . . .

E3 subset {1, 2, 5} can be represented as the binary number
 
 |µN 1 − x1 | |µN 2 − x2 | |µN 5 − x5 | 
··· 11001.
σN 1 σN 2 σN 5
If digit j is amputated, then Zi,j is defaulted to 0 for all
The cost vector is calculated by taking the sum of all i ∈ {1, 2, . . . , 5}. This does change the expected magnitude of
elements in each row. the cost metric, but because this distance of zero is taken for
the j-th digit in all use-patterns, the cost metric is normalized
Thus a 1 × N cost vector r = ( r1 r2 ··· rN ) was
for fair comparison between them.
defined as:
In the tests, simulation of all 31 subsets on the randomly
5
X generated matrix G was executed 300,000 times in MATLAB.
ri = Zi,j A trial was considered successful if the correct use-pattern was
j=1
predicted for the corresponding distorted average value.

The vector r provides a normalized measure of the distance III. E XPERIMENTAL R ESULTS
between a vector of analog flex values and each use-pattern.
The graphs in Figure 4 depict the harvested mean values
Thus the index of the use-pattern with the lowest cost, Umin dist ,
for each i, j pair and their respective standard deviations σij
is the predicted use-pattern.
as error bars. A high standard deviation corresponds to a high
amount of variance in the 50 trials executed for that i, j pair.
Umin dist = argmin r(i) For example, one may observe that the thumb in the wallet
i use pattern has an expectedly high standard deviation, as the

978-1-4799-2519-3/13/$31.00 ©2013 IEEE

726
position of the thumb in the plane of the wallet is irrelevant to [8] J. Lavery, B. Kent, E. D. Engeberg, “Biologically Inspired Grasp Primi-
a tight grip. Due to the normalization in the calculation of the tives for a Dexterous Robotic Hand to Catch and Lift a Sphere,” in Int.
Conf. on Control, Automation and Systems, Jeju Island, Korea, 2012, pp.
cost metric, a high variation is compensated for by the high 1710-1715.
standard deviation. N = 5 was used for this simulation.
[9] R. J. Smith, F. Tenore, D. Huberdeau, R. Etienne-Cummings, N. V.
Simulations with all possible binary digit amputations Thakor, “Continuous Decoding of Finger Position from Surface EMG
Signals for the Control of Powered Prostheses,” in 30th Ann. Int. IEEE
yielded promising success rates for each class of amputation, EMBS Conf., Vancouver, Canada, 2008, pp 197-200.
shown in Table II. For an E4 subset - a single amputated [10] N. Araki, K. Inaya, Y. Konishi, K. Mabuchi, “An Artificial Finger Robot
digit - the success rate was 96.45%. For an E3 subset - two Motion Control Based on Finger Joint Angle Estimation from EMG
amputated digits - the success rate was 92.52%. While a higher Signals for a Robot Prosthetic Hand System,” in Proc. of the Int. Conf.
number of amputations reduces the amount of information on Advanced Mechatronic Systems, Tokyo, Japan, 2012, pp. 109-111.
provided by the device, they still yield encouraging success [11] R. Vinjamuri, Z. Mao, R. Sclabassi, M. Sun, “A Novel Architecture for
rates. Strikingly, an E1 subset - only one healthy digit - still the Design of Prosthetic and Robotic Hand,” in Proc. of the IEEE 32nd
Ann. Northeast Bioengineering Conference, Easton, PA, USA, 2006, pp.
provided a success rate of 65.05%. 163-164.
[12] C. Cipriani, C. Antfolk, C. Balkenius, B. Rosén, G. Lundborg, M.C.
IV. C ONCLUSION Carrozza, F. Sebelius, “A Novel Concept for a Prosthetic Hand With a
Bidirectional Interface: A Feasibility Study,” IEEE Trans. Biomed. Eng.,
A total of five common use-patterns were implemented vol. 56, no. 11, pp. 2739-2743, 2009.
in the detection algorithm. The cost calculation algorithm
accounts for differences in digit variance in different use-
patterns by normalizing by the standard deviation in the Z-
score matrix.
In future work, a more sophisticated machine learning
algorithm, which is being developed, that dynamically adjusts
the cost metric could increase the number of use-patterns that
can successfully be detected. In addition, to provide additional
resolution, the flex sensors could be supplemented with other
sensors like gyroscopes and accelerometers.
The success rates provided by this simple algorithm mo-
tivate implementation of this system with a real prosthetic
device, providing a uniquely intuitive, inexpensive mode of
control for prosthetics.

ACKNOWLEDGMENT
The authors would like to thank Dr. Lev Sofman from Bell
Labs for providing feedback improving the readability of this
paper.

R EFERENCES
[1] C. Castellini, P. van der Smagt, “Surface EMG in Advanced Hand
Prosthetics,” Biological Cybernetics, vol. 100, no. 1, pp. 35-47, Nov.
2008.
[2] S. A. Ahmad, A. J. Ishak, and S. H. Ali, “Review of Electromyographic
Control Systems Based on Pattern Recognition,” Intl Federation for
Medical & Biological Engineering Proc., vol. 35, pp. 556–559, 2011.
[3] E. Scheme and K. B. Englehart,“Electromyogram pattern recognition for
control of powered upper-limb prostheses: State of the art and challenges
for clinical use,” J. Rehabil. Res. Dev., vol. 48, no. 6, pp. 643–660, 2011.
[4] P. M. Pilarski, M. R. Dawson, T. Degris, J. P. Carey, K. M. Chan, J. S.
Hebert, R. S. Sutton, “Adaptive Artificial Limbs: A Real-Time Approach
to Prediction and Anticipation,” IEEE Robot. Automat. Mag., vol. 20, no.
1, pp. 53 - 64, March 2013.
[5] E.A. Biddiss, D. Beaton, T. Chau, “Consumer design priorities for upper
limb prosthetics,” Disability and Rehabilitation: Assistive Technology,
vol. 2, no. 6, pp. 346-57, 2007.
[6] I. M. Bullock, R. R. Ma, A. M. Dollar, “A Hand-Centric Classification of
Human and Robot Dexterous Manipulation,” IEEE Trans. Haptics, Sept.
2012. IEEE computer Society Digital Library. IEEE Computer Society,
http://doi.ieeecomputersociety.org/10.1109/TOH.2012.53
[7] L. Dipietro, A. M. Sabatini, “A Survey of Glove-Based Systems and
Their Applications,” IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol
38., no. 4, pp. 461-482, 2008.

978-1-4799-2519-3/13/$31.00 ©2013 IEEE

727
TABLE II: Success Rates

Healthy 1 (Digit) Missing 2 Missing 3 Missing 4 Missing


Successes 1,479,727 7,233,622 13,877,948 12,791,328 4,879,381
Total Simulations 1.5 × 106 7.5 × 106 1.5 × 107 1.5 × 107 7.5 × 106
Percent Success 98.65% 96.45% 92.52% 85.28% 65.05%
Success rates in use-pattern detection for various types of amputations. The total number of use-pattern tests for a patient with
k healthy digits was 300, 000 · 5 · k5 . The number of trials is a (nonconstant) function of the number of subsets.

(a) Use-Pattern: Gripping Water Bottle, i = 1 (b) Use-Pattern: Holding Car Keys, i = 2

(c) Use-Pattern: Gripping Wallet, i = 3 (d) Use-Pattern: Holding Toothbrush, i = 4

(e) Use-Pattern: Gripping Toothpaste, i = 5

Fig. 3: Photographs of a subject demonstrating the various use-patterns used in the algorithm.

978-1-4799-2519-3/13/$31.00 ©2013 IEEE

728
Use-Pattern: Gripping Water Bottle Use-Pattern: Holding Car Keys
250 250

Mapped Flex Value 200 200

Mapped Flex Value


150 150

100 100

50 50

0 0
1 2 3 4 5 1 2 3 4 5
Digit j Digit j

(a) Sensor Values for Use Pattern 1: Gripping Water (b) Sensor Values for Use Pattern 2: Holding Car Keys,
Bottle, i = 1 i=2

Use-Pattern: Gripping Wallet Use-Pattern: Holding Toothbrush


250 250

200 200
Mapped Flex Value

Mapped Flex Value

150 150

100 100

50 50

0 0
1 2 3 4 5 1 2 3 4 5
Digit j Digit j

(c) Sensor Values for Use Pattern 3: Gripping Wallet, i = (d) Sensor Values for Use Pattern 4: Holding Toothbrush,
3 i=4

Use-Pattern: Gripping Toothpaste


250

200
Mapped Flex Value

150

100

50

0
1 2 3 4 5
Digit j

(e) Sensor Values for Use Pattern 5: Gripping Toothpaste,


i=5

Fig. 4: Sensor values for each use-pattern employed in the algorithm. The error bars represent ±σij .

978-1-4799-2519-3/13/$31.00 ©2013 IEEE

729

You might also like