Cooperation by Observation
- The Framework and Basic Task Patterns -
Yasuo Kuniyosh? Sebastien Rougeauxt
Nobuyuki Kita” Shigeyuki Sakane”
* Intelligent Systems Division, ‘Institut d’hformatique
Electrotechnical Laboratory d’Entreprise,
1-1-4 Umezono, Tsukuba City, 18 Allee Jean Rostand, 91000
Ibaraki 305, JAPAN Evry, FRANCE
Abstract
A novel framework f o r multiple robot cooperation called
“Cooperation b y Obsemtation” is presented. I t introduces
m a n y interesting issues such a3 viewpoint constraint and
role interchange, as well as novel concepts like “atten-
tional structure”. T h e framework has the potential t o re-
alize a high level of task coordination by decentralized au-
tonomous robots allowing minimum explicit communica-
tion. Its source of power lies in a n advanced capability
given to each robot for recognizing other agent’s actions by
(primarily visual) observation. This provides rich infor-
mation about the current task situation around each robot
which facilitates highly-structured task Coordination. T h e
basic visuo-motor routines are described. Concrete exam-
ples and experiments using real mobile robots are also pre-
sented.
1 Introduction
In order to execute complex tasks with multiple mo-
bile robots and achieve superlinear increase of task per-
formance to the number of robots, we need a variety of
flexible and highly-structured coordination among robot
behaviors. As for the method of coordination, we have
strong restrictions imposed by the distributed and asyn-
chronous nature of the entire system, namely, minimum ex-
plicit communication, decentralized control, and real time
response.
There have been many attempts to achieve an appro-
priate balance between the goal and the restrictions given
above (e.g.[l, 2 , 3 , 4 , 5,6, 7,8,9]). But in these approaches,
especially when we look at the implementations using real
mobile robots [4,7, 8, 91, there has been a consistent ten-
dency that the higher the level of coordination becomes,
the more the restriction is violated.
In this paper, we present a novel framework for multiple
robot cooperation called “Cooperation by Observation”,
which has the potential to realize a high level of task co-
ordination by decentralized autonomous robots allowing
minimum explicit communication. Its source of power lies
in an advanced capability given t o each robot for recogniz-
ing other agent’s actions by (primarily visual) observation.
767
1050-4729/94 $03.00 0 1994 IEEE
Makoto Ishiit
Masayoshi Kakikurat
* Tokyo Denki University,
2-2 Kanda Nishiki-cho,
Chiyoda-ku,
Tokyo 101, JAPAN
This provides rich information about the current task situ-
ation around each robot which facilitates highly-structured
task coordination. The aspect of extraction and interpreta-
tion of a variety of concrete information about the current
task situation, and their effects on the task execution is
largely unexplored.
In section 2, we look at the related work and extend the
above discussion. Section 3 presents our framework and
discuss its implications. The basic visuo-motor routines
which constitutes the fundamental behaviors are briefly
presented in section 4. Concrete examples and experiments
using real mobile robots are presented in section 5. The
final section summaries our results and proposes future
directions within our framework.
2 Related Work
In the previous literature, the level of task coordina-
tion roughly parallels the amount of explicit communica-
tion among the robots. The no-communication extremes
(e.g. Steels (11, Brooks et al. [2], and Arkin [3)) rely on
indirect interactions among the robot behaviors through
physical states of the environment. But the variety of tasks
realized by these approaches is limited, e.g. collecting scat-
tered rock samples.
In contrast, the highest levels of cooperative tasks
demonstrated using real mobile robots (e.g. Sakane et
al. [8] and Noreils [7]) use distributed planning methods
and rely on intensive inter-robot communication.
Recently, closer analyses have been made as to the bal-
ance between local vs. global control, and the effect of
communication on the task. Mataric [4] analyzed and
demonstrated the effect of local inter-agent sensing ability
(implicit communication) on the performance of naviga-
tion and flocking behavior by a group of robots. Arkin [6]
also analyzed the effect of broadcasting each robot’s coor-
dinates and goal-achieving states on the performance of co-
operative navigation. Parker [5] explored a proper balance
between local vs. global information for a formation keep-
ing cooperative navigation by a team of locally controlled
agents. Here the transferred/shared information included
the global navigation goal, intermediate goals, and com-
plete global path. She also suggested the use of behavior
observation and anticipation.
Our work is on the lines of Mataric, Arkin, and es-
pecially Parker, in that we assume implicit communica-
tion through each agent’s sensing ability and some shared
knowledge about the global goals. By introducing a capa-
bility of visually recognizing other agents’ actions and an-
ticipating forthcoming events, we attempt to achieve high
levels of task coordination while using least amount of ex-
plicit communication. More specifically, we demonstrate
three types of cooperative behaviors, namely, posing, un-
blocking, and passing, with absolutely no explicit commu-
nication between the robots.
3 Cooperation by Observation
Many kinds of social animals, including humans, often
show highly coordinated group behaviors without explicit
communication. For example, some carnivores carry out
group hunting successfully in silence [lo]. In our everyday
life, we routinely perform cooperative tasks without speak-
ing, often without thinking. One typical example is that
when someone carrying a large baggage with both hands
approaches a closed door, a person near by would open the
door.
3.1 Action Understanding
The ability for tacit cooperation is based on “action
understanding”. It is defined as the following process:
Observe other agent’s action, and Choose appro-
priate actions regarding the observed action and
the current task situation.
The observation portion of the process is further re-
fined 1111:
Classify qualitative movement of the agent’s
body, Look f o r the target of the action based o n
causality, Anticipate OT Detect the e f e c t of the
movement o n the target.
Machine understanding of limited class of human ac-
tions has been developed 111, 12, 131. Also, the possibility
of coordinated manipulation based on force observation
without explicit communication has been shown [14].
Details of the theory and the implementation of action
understanding is beyond the scope of this paper (see [ll]).
Nevertheless, it is important to note that action under-
standing provides significantly richer information about
the ongoing task situation than previous implicit commu-
nication methods such as exchanging each agent’s position
and goal achievement state. The acquired information in-
cludes movements, relative positions, and relative velocity
of the agents regarding other objects, states of the objects,
and their temporal evolution. The agents and the objects
which define the relations are automatically selected using
attention control so that they are meaningful (relative) to
the current task of the observer. Also, based on attention
switching and qualitative events, temporal segmentation of
the observed task situation is automatically done, which is
used for synchronizing/ordering the behaviors of different
768
robots. Examples of these functions are given in section 4
and 5.
3.2 Assumptions
The assumptions which characterize the framework of
cooperation by observation are given below.
Local control: There is no central processor nor a central
database. Each robot controls its own behavior.
Least communication: Explicit inter-robot communi-
cation is minimized.
Shared knowledge: Members of a robot team share a
static portion of global knowledge, such as high level
task goals and common strategies.
Complex and dynamic environment: Many objects
and robots are scattered all over the task space.
For the current task of a specific robot, some ob-
jects/robots are relevant, but others are independent.
Task complexity: In order to achieve the global goal,
highly structured coordination of multiple robot ac-
tions is required. The structures are defined in terms
of temporal ordering/synchronization and combina-
tion of actions such as relative positions/movements.
One benchmark task will be for dozens of robots to
carry all cargoes (of various sizes, shapes and weight)
scattered in a warehouse outside through a narrow
door as soon as possible.
Homogeneous robots: The robots have equivalent
structures, physical ability and cognitive competence.
Observation ability: Each robot is equipped with ap-
propriate sensors (ex. stereo vision, proximity, force,
touch, etc.) and recognition routines for observing
other robot‘s actions.
3.3 Issues
Many interesting problems arise from the above as-
sump tions.
Selective attention: Action recognition requires ex-
tracting a causally related pair of movement and its
effect. Since the observation must be made in a com-
plex environment, it is mandatory to dynamically se-
lect and observe only relevant entities in the surround-
ings.
Viewpoint constraint: There is no central database and
explicit communication is strictly limited. Therefore,
a bird-eye view (a global model) of a task situation
cannot be obtained. All observations must be made
from the viewpoint of each robot.
Interchange of roles: One merit of homogeneous robots
is that the role of each robot can be dynamically
changed according to the task situation. For example,
one robot can be either an auxiliary helper in one sit-
uation or an independent task performer in another
situation. This improves the overall performance of
the robot group. Then it becomes an important issue
how to dynamically select an appropriate role based
on local observations.
Sensing: Each robot must have sensory functions which
are appropriate for action recognition in a complex
environment. We found that a combination of con-
trolled vergence stereo vision and optical flow analy-
sis is quite powerful. But there are many other candi-
date methods, including other sensor modalities like
force/touch sensing, audio, etc.
Behaviors: For real time responses, a behavior-based ar-
chitecture [15] with extensions to incorporate atten-
tion control and qualitative representations of coop-
erative task patterns. A strong candidate is the one
proposed by Chapman [16]. But it must be modified
to meet the viewpoint constraint and to deal with
gaze shifts, i.e. maintaining a limited temporal per-
sistence of spatial cognition.
Representation of situations: In order to select appro-
priate actions regarding the current role and task
state, a robot must have internal qualitative repre-
sentations of cooperative behavior patterns. Such a
representation should only describe partial aspects of
the target situation which characterize the type of
cooperation. There are infinite instances of actual
positions/movements of the robots and the objects
in one qualitative situation. Also, since the number
of permutations of roles in one situation can increase
exponentially to the number of robots contained, we
need an efficient way to handle the role interchange.
Grounded communication: The communicated infor-
mation, if any, must be grounded to the local situation
of the robot which uses it. Basically, the information
should be qualitative, i.e. relative to the receiver’s
current action/state. When and how to generate use-
ful information for other robots, and how to use the
received information will be a challenging problem.
3.4 Attentional Structure
Chance and Jolly [17] classified the social structures
of monkeys and apes into two categories, centripetal and
acentric. A centripetal society is formed around one su-
perior male. Other members of the society mainly pay
attention to the behavior of this male and control their
own behaviors. When attacked, the superior male and its
company go in front of the defense or the retreat, followed
by the others. An acentric society behaves in an opposite
way, e.g. it scatters under attack.
The idea of attentional structure is quite important be-
cause it explains various social structures grounded on ac-
tion observation. Conversely, attention control in action
observation process can be a candidate driving force in
emergence of various social structures. Some examples of
attentional structures are illustrated in Fig. 1. All of these
have ground instances in animal societies [lo].
The original notion of attentional structure describes
purely social interactions. When a group of robots engage
in a cooperative task, we must also consider physical in-
teractions with other objects.
7 69
0:agent -: attention
Fig. 1: Attentional structures.
We adopt a generalized version of the notion of atten-
tional structure:
A n attentional structure is a set of attentional
relations among all members of a cooperative
group and related objects. A n attentional rela-
tion describes who watches the actions o,f w h o m
and/or who watches which object(s) in order t o
control self behavior.
3.5 Classification of Cooperative Behav-
iors
The framework of cooperation by observation intro-
duces a wide variety of cooperative task patterns. As a
preliminary study, we choose the example tasks shown in
Fig. 2 and analyze their structures with regard to the fol-
lowing features: (1) The number of robots ( N R ) and ob-
jects ( N o )involved, (2) The attentional structure ( A ) ,(3)
Physical interactions between involved robots and objects,
(4)Temporal relations among robot actions ( R T ) e.g., or-
dered (ord) or simultaneous (sim), in the two-robot cases.
Here we assume that one robot can manipulate at most
one object, hence, 11’~< N o , and there is no direct physi-
cal contact between robots. Possible attentional structures
are either uni-directional (e.g. Rl+R2) or bi-directional
(e.g. RlwR2).
The first two patterns, (a) pushing and (11) tumbling,
have {No = ~ , R = T sirn,A = {RI -+ 01,RZ 01)).
-+
Basically, the robots only need to observe (using vision
and/or force sensing) the physical state of 0 1 for coor-
dinating their behaviors [14]. Tumbling requires much
tighter coordination than pushing.
The other patterns (c-f) involve no physical interac-
tions between the objects or the robots during the task,
so the use of visual observation is mandatory for coordi-
nation. The pat,tern (c), giving way [9], is the simplest
case where { N o = 0, RT = sim,A = {RI -+ R2)).
Instead of just stopping and waiting, if RI keeps a con-
stant distance and orientation to R2, this can be used as a
preparatory behavior for helping R2, which we call “Pos-
ing” behavior. Mutual avoidance and formation keeping
also fall into this category but with bi-directional atten-
tion. The pattern (d), passing, requires sequencing the
actions, e.g. {RT = ord, N o = 1). The attentional struc-
ture can be uni-directional ( A = {RZ 4 R I , R2 -+ 0 1 ) )
in this case, but if the robots grasp the object and should
pass it in free space, bidirectional attention is necessary.
The pattern (e), unblocking { N o = 2, RT = sim}, is a
complex situation: The transporter robot R1 is carry-
ing an object 01, whose path is blocked by an obsta-
cle 0 2 . Because of limited maneuvering ability, it costs
much time for RI to avoid 0 2 . The helper R2 recognizes
this situation and helps R1 by approaching the 0 2 and
pushing it away. (A similar but simpler version has been
demonstrated by Noreils [7] using explicit communication.)
This pattern can be achieved with uni-directional atten-
tion {A = {R2 + R1,R2 --* 0 l 1 R 2-+ 0 2 } } ,but the
flexibility increases if bi-directional type is applied. For
example, in case R2 fails to approach 02, R1 should wait
for R2 to recover and accomplish the unblocking. The fi-
nal pattern (fj, cargo loading {No = 2, RT = sim,A =
{Rl 0 1 , R1 + 0 2 , R2 + 0 2 , R2 -+ O l } } ,requires
-+
tight coordination through visual observation.
Fig. 2: Cooperative task patterns: (a) Pushing. (b) Tumbling.
(c) Giving way. (d) Passing. (e) Unblocking. (f)Cart loading.
4 Basic Visuo-Motor Functions
We have implemented a set of visuo-motor functions
which serves as a basis for cooperative behaviors. The al-
gorithms are selected so as to meet the conditions given in
section 3, especially, (1) the complex environment (many
robots and many objects), and (2) viewpoint constraint.
Currently, we assume a mobile robot with vergence con-
Fig. 3: Situation recognition of a n unblocking t a s k by R2 (left)
and recognition of a release action (right).
trolled stereo cameras and no arm. Figure 3 contains ma-
jor functions required for the example tasks presented in
the next section. Since the details of vision algorithms are
out of the scope of this paper, we will describe them only
briefly.
(1)Find: This redirects an observer’s attention to a new
target, e.g. a moving robot and obstacles. First,
an optical flow field is continuously computed by
770
an incremental version of the Horn&Schunck algo-
rithm [lS]. Our current implementation executes 1 or
2 iterations per each frame acquisition within about
100 [msec]. Then, noise removal by local averaging is
applied. Next, the flow field is compared with a the-
oretical background flow to cancel self motion and to
detect interesting targets. In order to deal with the
significant noise in the flow field, we currently assume
pure translation or pure rotation as the self motion
and do the comparison with coarsely quantized in-
tensity and orientation of the flow vectors. The back-
ground flow for a rotation is trivial. For a translation,
we compute a theoretical flow produced by a virtual
vertical wall in front of the robot at a preset distance
(which is actually a time to collision), using self ve-
locity. Using this virtual wall field as a threshold,
we can ignore distant targets and extract only close
obstacles and quickly approaching robots. The re-
sult field is then segmented by clustering the residual
vectors. The above method can detect moving/static
objects from a moving robot.
(2)Track: Strong spatial selectivity is required for track-
ing in a complex environment. We adopt ver-
gence controlled stereo with zero disparity filtering
(ZDF) [19], which has strong depth selectivity. The
ZDF extracts zero disparity image features by first
extracting vertical edges from the left and right edge
images and simply taking “AND” of the results.
The result contains only those features lying on a
“horopter”, a circle defined by the two camera cen-
ters and the current fixation point. In combination
with the classical 2D region of interest method, the
system can actively select a 3D region to extract vi-
sual information. The ZDF output edges are tracked
by controlling the vergence angle of stereo cameras.
In order to follow the target motion in depth axis, we
extended the original ZDF by incorporating a lim-
ited search around near-zero disparities to find the
best match, and feedback the result to vergence con-
trol [20].
(3)Anticipate: In the situation of Fig. 3(left), the helper,
R2, need to anticipate a possible collision of Ol(R1)
with 0 2 . When the target motion is tracked over a
constant distance, a 3D line is fit to the recorded tra-
jectory by the least mean square method. Then the
system quickly scans the gaze point along this line by
vergence control. During the scan, the ZDF output is
continuously monitored to detect an object. An ex-
perimental result is shown in Fig. 4(rightj. Near-zero
disparity search is turned off in this phase, because
this time we have a top-down specification of the tar-
get depth.
(4)Event Detection: For a passing type of task, the re-
ceiving robot must synchronize its action by recog-
nizing a release action of another robot. This is done
by monitoring the temporal change in the segmented
Fig. 4: Tracking a mobile robot (left). Trajectory estimatron and obstacle detection (right). In both figures, raw $tert.o images are
shown at the top, extracted edges in the middle, ZDF output4 at bottom-right. and results of romputation are shown at bottom-left.
T h e whole processing is done at near frame rate (20 - 30Hz).

optical flow field, as shown in Fig. 3(right). The event rect the significant odometric error in self orientation
is signalled when the area of segmented flow suddenly imposed by turning itself to the desired pushing di-
decreases and the average velocity of the area is re- rection.
versed (Tistarelli and Sandini [21] presented a similar
method for recognizing a push action). In order to 5 Example Tasks
simplify the computation and to cope with the noise, In order to verify the effectiveness of the visuo-motor
we assume that the observer is not moving while de- routines and to present a concrete examples of coopera-
tecting the release action, and approximate the seg- tion by observation, we have realized three task patterns;
mented region by its surrounding rectangle in our cur- posing, passing and unblocking.
rent implementation.
5.1 The Prototype System
(5)Navigation: This guides a robot to the target posi-
tion, e.g. 0 2 in the unblocking situation (Fig. 3(left)),
and the released object in the passing situation
(Fig. 3(right)). When there is no other constraints
and the robot can take a straight line path to the
visible target (as in the unblocking situation), the
robot uses visual guidance. The steering angle is se-
lected from {-5, 0, 5}[deg] according to the sign of
the displacement angle of the current fixation point
of the cameras from the current forward direction
of the mobile base. Of course, this crude control
does not converge well, but is sufficient for prelimi-
nary experiments since the cycle time is short (about
lOO[msec])comparedto the speed of the mobile base.
In some cases, a robot cannot directly approach the Fig. 5: A mobile robot with a stereo gaze platform.
target but must follow a precomputed trajectory. For
example, in the passing situation, the receiver robot Fig. 5 shows one of our mobile robot equipped with a
must push the target object in the same direction as stereo gaze platform. The gaze platform has 2 DOF for
the first robot did. In such cases, the robot must com- vergence control. The robot has two onboard processors,
pute an approach path in order to cope with the non- one for controlling the mobile base and another for the gaze
holonomic nature and/or preserve enough approach platform. Image processing is currently done on a remote
distance so that the visual guidance routine can cor- host processor which consists of pipeline image processors

771
 .................................................................
Mobile robot
Gazeplatform
Cameras actuators
V25(8086)8MHz V25(8086)8MHz
! M
! E- Mavideo 200
MvME167/Lyn~oS
7
7 ~0stptoce-r
C.-...........-.....-.-........-....-...-..-..-.......----.......~
Fig. 6: Overview of the prototype system.
(Datacube Maxvideo system), and a CPU board running
a real time operating system. The system configuration is
given in Fig. 6.
5.2 Posing
,..” : from t h e receiver robot. ( a ) A segmented optical flow region
i
; Chaser
Fig. i : C h a s i n g t h e t a r g e t r o b o t a m o n g o t h e r s (left) a n d t h e
attentional s t r u c t u r e ( r i g h t ) .
The situation of the posing experiment is illustrated
in Fig. 7. The mission of the chaser robot is to keep a
fixed relative position (distance and orientation) with re-
gard to the target robot. There is a distractor robot mov-
ing around, that sometimes comes in between the chaser
and the target. The chaser must lock onto the target. This
is the simplest example of the attentional structure which
emerges from local attention relations. It is also a basic
behavior which supports other cooperative behaviors in
complex, dynamic environments. In this case, the chaser
and the target form an ordered pair, and the distractor
is isolated, not spatially, but at the level of cooperative
behaviors. The result is shown in Fig. 8. In this experi-
ment, only the chaser was computer controlled, where as
the others were manually controlled. The purpose of the
experiment was to test the stereo tracking and visual servo
routines. They turned out to be quite robust.
5.3 Passing
The realized “Pass” behavior pattern is shown in Fig. 9-
10. Figure 9(a)-(c) shows the result of motion segmenta-
tion and release detection. The released can is detected
and the receiver robot starts to follow the computed ap-
proach trajectory. This is done in an open-loop manner us-
Mobilebase i
actuators i
f
1._....__._.._._
....;
l
(e) (f)
Fig. 9: Vision Processing d u r i n g a Receiving Action; A view
including t h e robot a n d t h e can. ( b ) A release action is detected.
( c ) T h e released can is segmented. ( d ) Followed a n a p p r o a c h
p a t h a n d recaptured t h e can. (e),(f) An a p p r o a c h behavior
using visual servo based o n optical flow. T h e w h i t e m a r k e r
indicates t h e target position.
ing the temporally spatial memory. After making a steep
turn, the can becomes visible again (d), then the robot ap-
proaches it by visual guidance (e) - (f). Figure 10 shows a
bird-eye view of the entire situation. Evidently, the pass-
ing behavior pattern was successfully demonstrated.
5.4 Unblocking
We have made a simplified experiment of the unblocking
task discussed before. In this experiment, the transporter
robot ( R l ) does not carry an object ( 0 1 ) (different from
Fig. 3). This simplification is justified because the purpose
of this experiment is to verify the behavior of the helper
(R2).
Fig. 11 shows a result. The helper robot successfully
found the anticipated obstacle and helped the transporter
robot by pushing away the obstacle just in time for the
transporter to pass.
6 Summary and Conclusions
A novel approach to achieving highly structured task
coordination without resorting to explicit communication
is presented. It is supported by mutual observation of ac-
tions, so the framework is called “cooperation by observa-
tion”. The main contribution of the framework is to focus
772
Fig. 8: Posing behavior. The chaser (left) follows the target (moving downward). Even when the distractor (moving upward) comes
in between the chaser and the target, the chaser is not disturbed and locks onto to t h e target.
on the extraction and the interpretation of useful infor-
mation for task coordination by cooperating autonomous
agents. The framework also introduces many interesting
issues such as viewpoint constraint and role interchange,
as well as novel concepts like “attentional structure” which
can be used for classifying cooperative task patterns.
Another major contribution of this paper is that it
presents working examples of high levels of cooperative
tasks based on implicit communication through visual ac-
tion recognition. These are the most advanced examples in
the implicit communication framework demonstrated us-
ing real mobile robots.
The basic visuo-motor functions are characterized by
spatial selectivity, anticipatory functions, and the dynamic
nature. Although these routines basically have general
utility, we need more refinement and augmentation. Es-
pecially, following functions need to be realized: (1) Quick
and robust methods for finding and identifying objects, (2)
Multiple attention under parallel processing.
In the presented experiments, we have one autonomous
robot acting as a helper for other, fixed-program or man-
ually controlled robots. Since each robot system is com-
pletely independent of others, there will be no problem in
duplicating our prototype system. But there is more to be
done before we achieve a versatile implementation of co-
operation by observation, including, qualitative represen-
tation of cooperative task patterns, dynamic selection of
an appropriate cooperative behavior from a large number
of potential patterns, implementation of role interchange,
and treatment of shared knowledge.
References
[l] L. Steels. Cooperation between distributed agents through self-
organization. In Proc. I E E E Int. Conf.Intelligent Robots and
Systems., 1990.
[2] R. A. Brooks e t al. Lunar base construction robots. In Proc.
IEEE Int. Workshop o n Intelligent Robots and Systems,
pages 389-392, 1990.
[3] R. C. Arkin. Cooperation without communication: Multiagent
schema-based robot navigation. J. Robotic Systems, 9(3):351-
364, 1992.
[4] M. J. Mataric. Minimizing complexity in controlling a mobile
robot population. In Proc. I E E E Int. Conf. Robotics a n d Au-
tomation, pages 830-835, 1992.
773
[5] L. E. Parker. Designing control laws for cooperative agent
teams. In Proc. I E E E Int. Conf. Robotics a n d Automation,
pages 582-587, 1993.
[6] R. C . Arkin and E. Nitz. Communication of behavioral s t a t e in
multi-agent retrieval tasks. In Proc. I E E E Int. Conf. Robotics
a n d Automation, pages 588-594, 1993.
(71 F. R. Noreils. An architecture for cooperative and autonomous
mobile robots. In Proc. I E E E In$. Conf. Robotics a n d Au-
tomatron, pages 2703-2710, 1992.
(81 S. Sakane et al. A decentralized and cooperative sensing system
for robot vision. In Proc. 23rd Int. Symp. Industrial Robots,
pages 151-156, 1992.
[9] S. Premvuti and S. Yuta. Consideration on t h e cooperation of
multiple autonomous mobile robots. In Proc. I E E E Int. Work-
shop on Intelligeni Robots a n d Systems, pages 59-63, 1990.
[lo] E. 0. Wilson. Sociobiology: The New Synthesis. Harvard,
1975.
[ll] Y. Kuniyoshi and H . Inoue. Qualitative recognition of ongoing
human action sequences. In Proc. lJCAI93, pages 1600-1609,
1993.
[12] S. Hirai and T. Sato. Motion understanding for world model
management of telerobot. In Proc. ISRRS, pages 124-131,
1989.
[13] K . Ikeuchi and T. Suehiro. Towards an assembly plan from ob-
servation. Technical Report CMU-CS-91-167, School of Com-
puter Science, Carnegie Mellon Univ., Pittsburgh, PA 15213,
USA, 1991.
[14] N . Sawasaki and H. Inoue. Cooperative manipulation by au-
tonomous intelligent robots. In J S M E Conf. o n Robotics a n d
Mechatronrcs, pages 561-566, 1992. (In Japanese).
[15] J . Connel. A colony architecture for an artificial creature. Tech-
nical Report AI-TR 1151, M I T AI Lab., 1989.
[16] D . Chapman. Vrsron, Instructron a n d Actton. MIT Press,
1991. ISBN 0-262-03181-7.
[17] M . R. A. Chance and C. J . Jolly. Socral group.9 of monkeys,
apes a n d men. E. P. Dutton, 1970.
[la] B. K. P. Horn and B. G. Schunck. Determining optical flow.
Artrficral Intelligence, 1:185-203, 1981.
[19] P. von Kaenel, C. M. Brown, and D . J . Coombs. Detecting
regions of zero disparity in binocular images. Technical report,
University of Rochester, 1991.
[20] S . Rougeaux, N. Kita, Y. Kuniyoshi, and S. Sakane. Tracking a
moving object with a stereo camera head. In Proc. 11th A n n u a l
Conf. of Robotrcs Society of Japan, 1993. ( t o appear).
[21] M. Tistarelli and G . Sandini. Dynamic aspects in active vision.
CVGIP: Image Understandrng, 56(1):108-129, 1992.
 (g)
Fig. 10: Passing Task. ( a ) ( b ) A robot is pushing a can from left
t o right. (c) T h e first r o b o t releases t h e can a n d r e t u r n s t o fetch
a n o t h e r c a n , a n d t h e helper r o b o t recognizes t h e situation. ( d )
T h e helper robot d e t e r m i n e s t h e pushing direction. a n d s t a r t s
t o follow a c o m p u t e d a p p r o a c h t r a j e c t o r y . ( e ) ( f ) T h e helper Fig. 11: Unblocking behavior: T h e binocular robot ( h e l p e r ) o n
robot a p p r o a c h e s t h e can using visual servo. ( g ) P u s h i n g t h e t h e left watched t h e monocular t r a n s p o r t e r robot moving from
can in t h e s a m e direction as t h e first r o b o t d i d . right t o left, estimated t h e p a t h a n d found t h e possible obstacle
o n t h e p a t h ( t o p figure), t h e n quickly approached t h e obstacle
using visual servo (middle figure), a n d successfully pushed it
away t o help t h e t r a n s p o r t e r ( b o t t o m figure).

114