32 (IJCNS) International Journal of Computer and Network Security,

Vol. 2, No. 10, 2010

Object Handling Using Artificial Tactile Affordance

System
Naoto Hoshikawa1, Masahiro Ohka1 and Hanafiah Bin Yussof2
1
Nagoya University, School of Information Science,
Furo-cho, Chikusa-ku, Nagoya, 464-8601, Japan
ohka@is.nagoya-u.ac.jp
2
Universiti Teknologi MARA, Faculty of Mechanical Engineering
Shah Alam, Selangor Darul Ehsan, 40450, Malaysia

changed due to the deformation and the transformation

Abstract: If the theory of affordance is applied to robots,
performing the entire process of recognition and planning is not caused by the robot behavior, and the result is sent as tactile
always required in its computer. Since a robot’s tactile sensing information to the ATAS inlet by a feedback loop.
is important to perform any task, we focus on it and introduce a In ATAS, a key point is producing a set of rules in
new concept called the artificial tactile affordance system which the sensor input patterns and behavior output patterns
(ATAS). Its basic idea is the implementation of a recurrent are if-clauses and then-clauses, respectively. This ATAS
mechanism in which the information obtained from the object
concept resembles an expert system in artificial intelligence
and the behavior performed by the robot induce subsequent
behavior. A set of rules is expressed as a table composed of
[4] in which the matching degree between the fact selected
sensor input columns and behavior output columns, and table by a fact database and the if-clause is evaluated; if the fact
rows correspond to rules; since each rule is transformed to a matches the if-clause, then the then-clause is displayed as a
string of 0 and 1, we treat a long string composed of rule strings temporal conclusion, and the then-clause is simultaneously
as a gene to obtain an optimum gene that adapts to its added to the fact database. The biggest difference between
environment using a genetic algorithm (GA). We propose the ATAS and an expert system is that in the latter, the fact
Evolutionary Behavior Table System (EBTS) that uses a GA to
database is possessed inside a computer, but in ATAS the
acquire the autonomous cooperation behavior of multi-finger
robotic hands. In validation experiments, we assume that a whole environment is treated as the fact database.
robot grasps and transfers an object with one or two hands, each
of which is equipped with two articulated fingers. In
computational experiments, the robot grasped the object and
transferred it to a goal. This task was performed more efficiently
in the one-hand case than in the two-hand case. Although
redundant grasping induces stability, it decreases the efficiency
of grasping and transferring.
Keywords: Artificial intelligence, Affordance, Genetic
Algorithm, Tactile sensing.

1. Introduction
Tactile sensation possesses a salient characteristic
among the five senses because it does not occur without
interaction between sensory organs and objects. Touching an
object induces both deformation of it and the sensory organ.
Since the tactile sensing of robots is crucial to perform any
task [1], we previously focused on tactile sensing and
introduced a new concept called the artificial tactile
affordance system (ATAS) [2], whose basic idea is an
extension of the theory of affordance [3].
ATAS is based on a recurrent mechanism in which the
information obtained from the object and the behavior
performed by the robot itself induce subsequent behavior. To
explain ATAS, we introduce its schematic block diagram
(Fig. 1). When the tactile information obtained from the
environment is input into ATAS, a command for robot
behavior is output, and the robot actuators are controlled
based on that command. After that, the environment is
Figure 1. Artificial tactile affordance system (ATAS)
Since ATAS is categorized as a behavior-based control
system, it more closely resembles subsumption architecture
(SSA) [5][6]. Although in SSA each connection between
sensors and actuators has priority, in ATAS no modules
have priority, and they are arranged in parallel form. In
each module, we can include a relative complex procedure if
needed. Therefore ATAS is suited to such rather complex
tasks as assembly and walking on uneven loads.
In a previous paper [2], we implemented ATAS based on
the following two methodologies. In methodology 1, ATAS
is composed of several program modules, and a module is
selected from the set of modules based on sensor
information to perform a specified behavior intended by a
designer. In methodology 2, a set of rules is expressed as a
 (IJCNS) International Journal of Computer and Network Security, 33
table composed of sensor input columns and behavior output
columns, and each rule is transformed to a string of 0 and 1
treated as a gene to obtain an optimum gene that adapts to
the environment using a genetic algorithm (GA) [7]. While
methodology 1 was very effective for such fine control as
handling tasks of humanoid robots, methodology 2 was very
useful to obtain general robotic behavior that was suitable
for its environment. However, the possibility of method 2 for
object handling must be further pursued because the robot
can automatically obtain the relationship between sensor
input and behavior output patterns after it has committed to
the environment.
In this paper, we apply the Evolutionary Behavior Table
System (EBTS) of methodology 2 to the object handling of a
two-arm hand because we are examining the possibility of
methodology 2 for fine control. After explaining the
behavior acquisition method using EBTS, we derive a
simulation procedure for a two-hand-arm robot that was
developed in another project. In computational experiments,
the robot grasps an object and transfers it to a goal. We
examine the difference of grasping efficiency between one-
and two-hand tasks.
Table 1: Behavior tables
2. Behavior Acquisition Method
2.1 Behavior Table
We must define the relationship between environmental
information and the behavior rules in a single-layered
reflexion behavior system to control the behavior of a two-
hand-arm robot. In this paper we treat the fingertip as an
agent. Stimulus and response are defined by sensor-truth
and finger-movement-state tables, respectively (Table 1).
These tables are prepared for the robot agent equipped with
three sensing elements, as introduced in the next section.
S01, S02, S11, S12, S21, and S22 show the status of the three
sensing elements mounted on an agent. Since each sensing
element can measure four grades of contact force, a 2-bit
signal is used for each sensing element. A0, A1, A2, and A3
show the status of the actuators. While A0 shows that the
agent movement is stopping or active, A1, A2, and A3 show
the agent’s movement direction. Because the sensor status is
described by Boolean values (1 or 0), the number of total
patterns of the sensor status is 26 = 64 . An agent refers to
Vol. 2, No. 10, 2010
the current values of A0, A1, A2, and A3 of Table 1 in every
set period to decide its behavior.
2.2 Behavior Acquisition Method
The behavior table is composed of the truth and motor status
tables of the sensors. These tables can be expressed with a
one-dimensional array like genes because all of the truth-
values are Boolean data. Therefore, the behavior table can
be designed as a model of a simple genetic algorithm (SGA)
comprised of Boolean data [7]. Since the behavior table is
evolvable, we call it the Evolutionary Behavior Table
System (EBTS). The design procedure of the behavior table
using SGA is shown in the following.
First, we design a genotype array that has information
about the behavior table. The length of the genotype array is
shown with the next formula:
G = 4S × M (1),
where the number of sensors and the bit number of the
output gradation are S and M , respectively. Sensor
patterns are calculated as 4 S because each sensor has 2-bit
graduation.
The genotype model used for the numerical experiments
described in the subsequent section is shown in Fig. 2. The
agent possesses three sensing elements, the output of 4-bit
gradation, and a length of 64 × 4 = 256 bits as gene data
information.
The agent’s fitness value is calculated in a simulator that
is equipped with internal functions that evaluate the
efficiency and the task accuracy degrees of the agent. Then
the simulator generates a behavior table from the genotype
array of a one-dimensional vector composed of G elements
(Fig. 2). The agent is evaluated on the basis of the task
achievement degree in the simulator field during a specified
period. The evaluation value obtained by this simulation is
sent to the calculating system for genetic algorithms as
fitness.
Figure 2. Genotype model
3. Simulation Procedure
3.5 Two-hand-arm System
Since communication between robots is performed through
tactile data in ATAS, we adopted a two-hand-arm robot as
an example. Fig. 3 shows a two-hand-arm robot, which is an
34
advancement over the previously presented one-hand-arm
robot [8]. The arm system’s DOF is 5, and each finger’s
DOF is 3. To compensate for the lack of arm DOF, this
robot uses its finger’s root joint as its wrist’s DOF.
Figure 3. Two-hand-arm robot equipped with optical three-
axis tactile sensors
Figure 4. Three-axis tactile sensor system
On each fingertip, it has a novel, optical three-axis
tactile sensor that can obtain not only normal force
distribution but also tangential force distribution. See
previous articles [8]-[10] for explanatory details. Although
ordinary tactile sensors can only detect either normal or
tangential force, since this tactile sensor can detect both, the
robot can use the sensor information as an effective key to
induce a specified behavior.
The tactile sensor is composed of a CCD camera, an
acrylic dome, a light source, and a computer (Fig. 4). The
light emitted from the light source is directed into the
acrylic dome. Contact phenomena are observed as image
data, which are acquired by the CCD camera and
transmitted to the computer to calculate the three-axis force
distribution. The sensing element presented in this paper is
comprised of a columnar feeler and eight conical feelers.
The sensing elements, which are made of silicone rubber,
are designed to maintain contact with the conical feelers and
the acrylic dome and to make the columnar feelers touch an
object.
(IJCNS) International Journal of Computer and Network Security,
Vol. 2, No. 10, 2010
3.6 Simulator Design for Robotic Hands
As an object handling task, we adopted an object
transportation problem for easy comparison to previous
research. In the task, a hemispherical fingertip equipped
with three tactile sensing elements is modeled (Fig. 5). The
motion of the modeled robotic hand is restricted within two
dimensions. Since the wrist is assumed to be fixed, it only
manipulates an object with its fingers. We presume that the
robotic fingertip transports the object to the goal.
Figure 5. Model of hand and tactile sensor
Figure 6. Evolutionary Behavior Table System (EBTS)
A simulator is composed of a map field (work area) and
the objects on it. They are categorized into two types: an
autonomous mobile agent defined as a fingertip and a non-
autonomous object defined as the transportation object. The
autonomous mobile agent can be equipped with multiple
sensor inputs and behavior outputs as its module, which is
formed to function as a suitable module based on the
assumptions of numerical experiments.
In this object transportation problem, the autonomous
mobile agent is equipped with three tactile elements. An
overall view of the evolutionary behavior table is shown in
Fig. 6. The optimization procedure of the genetic algorithms
is summarized as follows:
• The population of the random gene data is produced
as an initial value.
• The evolutionary computation engine sends gene
data to the simulator to evaluate the gene fitness.
• Elite genes are selected based on their fitness.
• A set of individuals is chosen based on the roulette
wheel selection.
• A pair of individuals is selected and used for
uniform crossover.
• The newborn children from the pair mutate under a
certain probability.
• The children’s gene data are sent to the simulator to
evaluate their fitness.
• The fitness of the elite group is compared with that
of its children group. The group of the next
 (IJCNS) International Journal of Computer and Network Security, 35
Vol. 2, No. 10, 2010

generation population is selected from the high procedure.

score group in descending order. An individual mobile agent tries object transportation
• If it is not a final generation, it returns to the above eight times in each simulation. Evolution calculation, which
procedure 3). continues for 1,000 generations, is repeated ten times for
• Evolutionary computation is finished. each numerical experimental condition. The number of
3.7 Fitness Evaluation by Simulator individuals, maintaining the elite number, and the mutation
rate per 1-bit of gene are 10, 2, and 10%.
After the agents performed object transportation, the task’s
achievement degree was evaluated as a fitness value of the
gene. Evaluation functions for task achievement were 4. Numerical Experimental Result
composed of transportation accuracy value E1 , which was
4.1 Grasping Behavior
evaluated from the geometrical relationship between the
Figure 8 shows the task process using two hands with the
agents and the transportation object, and transportation
four fingers shown in Fig. 7(b). The robot hands
efficiency value E 2 , which was decided by reaching the successfully transported the object to the center circle, which
goal. Evaluation function E1 is defined as follows: is its goal. After examining other computational
K1 experiments with several initial conditions, we verified that
E1 = (2), this task does not depend on the initial position of object.
cA − cB
where c A and c B are the position vectors of the object
and the goal, respectively. Coefficient K1 is assumed to be
1,000.
On the other hand, evaluation function E 2 is defined by
K2
E2 = (3)
t
using spent time t to transfer the object to the goal area. In
this study, we assumed K 2 = 160 .
The fitness value of the genetic algorithm is calculated
as the summation of E1 and E 2 . In the fitness calculation,
the agent often accidentally pushed the transportation object
into the goal. To avoid such accidents, we divided the
simulation time into task execution and task evaluation and
compared the positioning between the agent and the
transportation object at the termination of the execution time
with the termination of the evaluation time.

Figure 8. Four fingertips acquired cooperative behavior as

collective task to convey object

4.2 Comparison Between One- and Two-hand

Figure 7 One- and two-hand cases
3.8 Simulator Conditions in Numerical Experiments
In numerical experiments, we assume one- and two-hand
cases (Fig. 7). The map field of each case is assumed to be a
Gaussian plane of 800 × 800 [pixel2] with an origin at
coordinates (0, 0) in a series of numerical experiments. The
transportation object and the agents are specified positions.
In Fig. 7(b), four agents are placed near each corner of the
rectangular field. The eight initial positions of the objects
are assumed to relax during adaptation in the task
Grasping
Figure 9 shows the result for the one-hand grasping test,
which depicts a variation in the fitness value of the first
elite, the second elite, and the top score of one population
under different generations. The variation in the fitness
value of the first elite almost coincided with that of the
second elite. Even at 1,000 generations, they don’t seem
saturated yet and exceed 600. Besides the first and second
elites, the top score of one population increased with
additional generations. Therefore, the efficiency of object
transportation with one-hand grasping is enhanced with
more generations.
The two-hand result is shown in Fig. 10. Although the
fitness value of the first and second elites increased with an
increase of generation, similar to the one-hand case, it
almost becomes saturated at 1,000 generations. The
36 (IJCNS) International Journal of Computer and Network Security,
Vol. 2, No. 10, 2010

difference between the fitness values of the first and second Figure 10. Relationship between maximum fitness and
elites is greater than that of one-hand case. The variation in generation for two-hand case
the top score of one population becomes steady after around
100 generations. Moreover, the fitness value of the first elite
at 1,000 generations is around 560 and is smaller than that
5. Conclusion
of one-hand case. Therefore, the efficiency of object
transportation with two-hand grasping saturated very We implemented an artificial tactile affordance system
rapidly. (ATAS) based on the following methodology. A set of rules
Since object grasping and transportation can be is expressed as a table composed of sensor input columns
performed by one hand, the two-hand case is obviously and behavior output columns, and a row of the table
redundant. Although redundant grasping induces stability, it corresponds to a rule. Since each rule is transformed to a
string of 0 and 1, we treat a long string composed of rule
decreases the efficiency of grasping and transferring.
strings as a gene to obtain an optimum gene that adapts to
Finally, we discuss the efficiency of acquiring autonomy
the environment using a genetic algorithm (GA). We
cooperation behavior of a multi-finger in this EBTS using
propose the Evolutionary Behavior Table System (EBTS)
GA. In this numerical experiment, the number of simulation using a GA to acquire the autonomous cooperation behavior
trials was 100,000, as a result of optimizing the gene data of multiple mobile robots.
until 1,000 generations. This final truth table obtained by In validation experiments, we assume that a robot
the gene data doesn’t always assure an optimum solution, grasps and transfers an object with one or two hands; each
but the calculation cost is reduced from 1.2 × 1077 to hand is equipped with two articulated fingers. In
1.0 × 10 5 because the combination numbers of the input and computational experiments, the robot grasped the object and
transferred it to the goal. This task is performed more
output patterns are calculated as 2 256 ≅ 1.2 ×10 77 , according efficiently in the one-hand case than in the two-hand case.
to Eq. (1). If we used top-down methodology, Although redundant grasping induces stability, it decreases
1.2 × 1077 trials are needed to specify the optimum pattern. the efficiency of grasping and transferring. Although the
Therefore, we accomplished the automatic design of present method requires many computational resources, it is
cooperative behavior of multi-agents for collective tasks. attractive because it is very useful for obtaining general
robotic behavior that is suitable for its environment.

700 References
600 [9] M. Ohka, “Robotic Tactile Sensors,” Encyclopedia of
500 Computer Science and Engineering, Wiley
Encyclopedia of Computer Science and Engineering,
400
Fitne ss

5-Volume Set, Editor: Benjamin, W. Wah, pp. 2454 –

300 2461 (Vol. 4), 2009.
200 [10] M. Ohka, N. Hoshikawa, J. Wada, and H. B. Yussof,
FirstElite
elite
First
SecondEliteelite
“Two Methodologies Toward Artificial Tactile
100 Second
Topscore
Top scoreofofone
onepopulation Affordance System in Robotics,” International Journal
0 population on Smart Sensing and Intelligent Systems, Vol. 3, No.
0 200 400 600 800 1000
Generation
3, pp. 466-487, 2010.
Relationship between Fitness and Generation for 2 agents
[11] J. J. Gibson, “The Ecological Approach to Visual
Perception,” Houghton Mifflin Company, 1979.
Figure 9. Relationship between maximum fitness and [12] P. H. Winston, “Artificial Intelligence (second
generation for one-hand case edition),” Addison-Wesley, pp. 159-204, 1984.
[13] Rodney A. Brooks, “A robust layered control system
for a mobile robot,” IEEE Journal of Robotics and
Automation, RA-2(1), pp. 14-23, 1986.
600 [14] Rodney A. Brooks, “Intelligence without
500
representation,” Technical Report MIT AI Lab, 1988.
[15] D. E. Goldberg, “Genetic Algorithms in Search,
400 Optimization and Machine Learning,” Addison
Fitness

300 Wesley, 1989.

[16] H. B. Yussof, J. Wada, and M. Ohka, Object Handling
200
Tasks Based on Active Tactile and Slippage Sensations
First
FirstElite
elite
100
Second
in Multi-Fingered Humanoid Robot Arm, 2009 IEEE
SecondEliteelite
Top
Topscore
scoreofofoneonepopulation International Conference on Robotics and Automation,
0
0 200 400 600 population 800 1000 pp. 502-507, 2009
Generation [17] M. Ohka, H. Kobayashi, J. Takata, and Y. Mitsuya, An
Relationship between Fitness and Generation for 4 agents Experimental Optical Three-axis Tactile Sensor
Featured with Hemispherical Surface, Journal of
 (IJCNS) International Journal of Computer and Network Security, 37
Vol. 2, No. 10, 2010

Advanced Mechanical Design, Systems, and

Manufacturing, Vol. 2, No. 5, pp. 860-873, 2008.
[18] M. Ohka, J. Takata, H. Kobayashi, H. Suzuki, N.
Morisawa, and H. B. Yussof, Object Exploration and
Manipulation Using a Robotic Finger Equipped with
an Optical Three-axis Tactile Sensor, Robotica, Vol.
27, pp. 763-770, 2009.