2020 5th Asia-Pacific Conference on Intelligent Robot Systems

A Flexible High and Low Level Multi Task Behavior Framework for Multi-agent
Systems in Complex Environments

Sebastian Hoose Daniel Sendzik

Department Automation and Embedded Systems Department Automation and Embedded Systems
Fraunhofer Institute for Material Flow and Logistics Fraunhofer Institute for Material Flow and Logistics
Dortmund, Germany Dortmund, Germany
sebastian.hoose@iml.fraunhofer.de daniel.sendzik@iml.fraunhofer.de

Jana Jost Thomas Kirks

Department Automation and Embedded Systems Department Automation and Embedded Systems
Fraunhofer Institute for Material Flow and Logistics Fraunhofer Institute for Material Flow and Logistics
Dortmund, Germany Dortmund, Germany
jana.jost@iml.fraunhofer.de thomas.kirks@iml.fraunhofer.de

Abstract—This paper introduces a framework for behavior and processes offer already information regarding their status
modeling of software agents using the Robot Operating System. and can be represented in the virtual world e.g. as a digital
We developed a framework to simplify behavior design and twin. While each device and process fulfills various purposes
model conventional software agent design principles as known
for example from belief desire intention BDI agents or adaptive and acts according to different standards a common and ven-
agents. We provide a toolbox that is capable of supporting the dor independent understanding for building them is required.
developer with easy to implement methods written in python. Therefore, the reference architecture model for Industry 4.0
Finally we present two brief example applications and evaluations (RAMI4.0) was introduced.
in both simulation and using a social automated transport vehicle
for intralogistical processes.
Keywords-distributed artificial intelligence; social robots;
intelligent agents; multi-agent systems

I. INTRODUCTION
This paper presents an agent behavior modeling framework
by defining i ts r equirements a nd a rchitecture a nd presenting
the need of a new behavior modeling approach. Additionally
the implementation of the architecture as two independent
Robotic Operating System 2 (ROS2) [1] nodes will be de-
scribed and an evaluation of the implementation is presented.
Finally a short outlook on further applications and further
developments is shown.

II. BACKGROUND
A. Industry 4.0 Architecture
Nowadays, throughout all business areas the physical and
virtual world melt together (known as Industry 4.0). In such
a socio-technical system humans and cyber-physical systems
are collaborating and are connected on a vertical as well as
horizontal level over company boundaries. [2] Most devices
*This research is supported by the Center of Excellence for Logistics and IT
and has been partially funded by the European Union’s Horizon 2020 research
and innovation program under grant agreement No. 688117 (SafeLog).
Fig. 1. Reference architecture model for Industry 4.0 (RAMI4.0) [3]
RAMI4.0 divides the Industry 4.0 space into the three
dimensions: layers, life cycle and value stream and hierarchy
levels (see. Fig. 1). The first dimension, the layer, displays
the different perspectives of a component as well as its
functionality and the interference with other components. The
life cycle and value stream dimension deals with the value

978-1-7281-9818-7/20/$31.00 ©2020 IEEE 68

Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on December 04,2023 at 17:59:01 UTC from IEEE Xplore. Restrictions apply.
chain as well as relationships and links throughout the life
cycle of a component. It is based on the draft of the IEC
62890 standard but takes the whole life cycle – also outside
the plant – into account. The last dimension, the hierarchy
levels, depicts the functional association of a component
including also responsibilities with the factories and plants
of the Industry 4.0. It follows the IEC 62264 and IEC 61512
standards. [3]
RAMI4.0 as a reference architecture model can only give
guidelines for the Industry 4.0. Further thoughts, e.g. of how
to integrate the human being except as a part of the Asset
Layer into the Industry 4.0 processes and for setting up the
structure of an Industry 4.0 component and integrating them
into an overall system, are needed.
B. Decentralized Control Systems
In industrial production and logistics scenarios robots, ma-
chines in general and also humans work together in teams and
are represented by software agents. For example in warehouse
logistics, decentralized control systems help implementing
technical entities such as mobile robots or sorting systems
to realize flexible control of material flow [4]. As Jost et al.
have demonstrated [5] also humans can be part of a multi-
agent system (MAS) in warehouse logistics. Humans and
robots communicate with each other and share information [6].
Behaviors for humans (respectively, the representing software
agents) and machine agents have to be modeled alike and made
available for re-usability. Known systems for modeling behav-
iors in robotics are behavior trees in general or SMACH as a
package for ROS [7] and JADE as MAS itself [8]. Behavior
trees were came up with the artificial intelligence in games,
where non play units had to interact with the human player in
an adequate manner [9]. In tree theory they are directed trees,
where the root element has one child element - the control flow
element. From here multiple children can point to an arbitrary
amount of children. The set of all children form a kind of
hierarchy, referring to different behaviors and subbehaviors.
SMACH uses a syntax in Python to create state machines
for behavior control of e.g. robots. These state machines can
become quite complex and can also be debugged. A graphical
user interface called smach viewer provides control over the
state machine in concerns of state transitions and data flow
[10]. JADE is an advanced MAS for JAVA developers. It
supports numerous possibilities to create different kinds of
software agents and model their behavior. Behaviors can be
modeled as OneShotBehaviors (being executed only once)
and CyclicBahaviors (being executed in a repeated manner,
triggered by events e.g. a timer). A SimpleBehavior class
allows to implement custom behaviors as well.
The development of a new behavior managing framework
is desirable since most currently available frameworks, as
described above, do only support inidrect switching of behav-
iors and mostly focus on a single decision making method.
Additionally many frameworks struggle to make code, written
by framework users, reusable.
69
Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on December 04,2023 at 17:59:01 UTC from IEEE Xplore. Restrictions apply.
III. B EHAVIOR M ODELING F RAMEWORK
To facilitate the programmability of decentralized multi-
agent systems a robotic behavior framework called beehave
has been developed.
A. Requirements
To enable the creation of socio-technical systems as a whole,
it is necessary to give users the capability to flexible switch
between different behaviors of robots and other technical
systems. To meet requirements and design templates of RAMI
4.0, model layers like assets has been intended, integration
(digital twin), communication and information (given by ROS)
as well as the functional layer (the proposed system). The
proposed system has to be applicable to the different hierarchy
levels of RAMI 4.0 as well. Another major requirement is
to increase re-usability of code to ensure vendor independent
distribution and accessibility.
Additionally, all robotic behaviors should always result in
a defined behavior to avoid the robots to act unexpectedly.
This safety feature should be hidden for users of beehave who
implement new robotic behaviors. A separation and indepen-
dent implementation of low level robotic behaviors such as
deductive behaviors that are always a direct result of high level
robotic behavior decisions should take place. Based on the
decisions made in the high level robotic behavior, e.g. driving
to a certain position, the low level robotic behavior e.g. should
deal with the state representation of the robot using LEDs.
To facilitate the implementation of new robotic high level
behaviors, predefined decision making structures such as state
machines and support for execution of neural networks should
be given as well. As a result, the proposed system is flexibly
extensible in a way that also service-oriented architecture
modeling may also be considered as complimentary building
brick, see Iñigo-Blasco et al. [11] which was not given before.
In the context of multi-agent systems, beehave would im-
plement the whole behavior of an agent, e.g. a mobile social
robot [12].
B. Architecture
To achieve the above mentioned requirement of the separa-
tion of low and high level robotic behaviors two independent
software parts were developed as displayed in Fig. 2. In this
model the high level robotic behavior receives information
about the situation of the robot and the demands of the
robot, e.g. a map of the environment, the current position,
via network requested tasks and communicates its decision
to the low level robotic behavior. The low level robotic
behavior again communicates with the hardware abstraction
layer (HAL) to actually execute the decision. The structure
of the high level behavior as shown in Fig. 3 contains three
main layers. The lowest layer of the high level behavior
consists of actions. Actions are modeling all basic features
of a robot, e.g. driving to a certain position or playing sounds.
All actions have to extend the abstract class Action, which
implements maintenance code such as communication and
safety features or error outputs. The only method that has to
 Map Position action types if needed.
Actions are started and organized by behaviors as shown
in the middle layer of Fig. 3. A behavior may run a single,
multiple or no actions at the same time. It is also possible
Network Interfaces
High Level Behavior to start other behaviors as sub behaviors. For example, this
→ MQTT is useful if one wants to use a decision making routine for
→ Webservice multiple behaviors. Further, it is possible to run multiple
→ DDS sub behaviors at the same time. Analogously to actions,
each new behavior has to extend an abstract class called
Low Level Behavior Behavior. This abstract class manages the life cycle of the
behavior and controls the running actions and sub behaviors
of the behavior. Actions may be aborted by the controlling
behavior. For this, each action has an interrupt flag which
HAL allows the action to stop gracefully and leave the part of
the robot the action was in control of, in a predictable and
Fig. 2. Beehave architecture
safe state for the following actions. The behavior life cycle
is managed by the provided behavior base class. Analogous
to the implementation of actions, the only method that has
to be overwritten to implement a new behavior, is the run
HighLevelBehaviorManager method which normally implements the decision making and
organizes the behavior. The decision making process may
be implemented by hand or can make use of one or several
available * 1 active decision engines like state machines, behavior trees or neural
* networks. Online retraining of neural networks is supported as
*
DecisionEngine Behavior well.
An example of an behavior could be a use case in which a
behavior decides where the robot should move to using a state
machine as decision making engine and inputs, e.g. given by
Yaml Commanded Random
another agent. After making a decision the behavior can use
an action to handle the driving procedure. At the same time a
second action, which is in charge to handle a sound playing
procedure, is started parallely as well.
* Behaviors may be changed at runtime dynamically. For this,
Action behaviors can be registered in the HighLevelBehaviorManager
as top level behaviors. When a top level behavior should be
replaced by a different top level behavior, the running top level
behavior will be canceled gracefully. Once all behaviors and
actions have been shut down, the new top level behavior will
Navigation Lift LaneFollowing be started. This feature allows to satisfy the main requirement
to safely and dynamically switch between high level robotic
behaviors at runtime. This feature can be used e.g. by a swarm
Fig. 3. High Level Behavior manager of a multi agent system to switch to a different swarm
strategy. Another use case may be to switch the robot into a
maintenance mode.
be implemented by an action is the run method, which should The structure of the low level behavior as shown in Fig.
contain the code that is needed to execute the intended task 4 consists of three main blocks. Similar to behaviors in the
e.g. driving to a given position and dealing with the outputs of HighLevelBehaviorManager in the high level behavior the
the navigation system of the robot. Multiple actions are able to LowLevelBehaviorManager is capable to control logics. A
run simultaneously, as long as they are not of the same action logic again, has to be implemented by extending the abstract
type. Locks ensure automatically, that there is not more than Logic class, which provides methods to manage the life cycle
one action actively running, that manages tasks of the same of logics. Similar to behaviors in the high level behavior, it
domain, e.g. lane following and free navigation based on laser is also possible to start sub logics from a logic, which are
sensors can not take place at the same time since both are of executed asynchronously. The purpose of a logic is to manage
the type driving. The locking of actions from execution who the low level decisions and the executions of them, e.g. which
have the same type is managed by the abstract base class of color a LED should have based on decisions made by the high
the action, Action (see Fig. 3). The user may add additional level behavior.

70
Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on December 04,2023 at 17:59:01 UTC from IEEE Xplore. Restrictions apply.

Blackboard Lowgic
available * 1 active
HALManager Logic
* implement their actions by calling ROS2 services.
HAL Default
LED Navigation
Fig. 4. Low Level Behavior
Logics are obliged to access the hardware of the robot via
the HALManager. The HALManager manages the execution
of hardware abstraction layers (HALs) of all kinds. Logics
are then able to request these hardware abstraction layers to
be loaded. The HALs will provide the current state of the
hardware they are implementing via blackboard variables. The
blackboard can be accessed by all running logics and the
logics are able to subscribe to changes of variables they are
interested in. Logics may also call methods of HALs to control
the hardware.
To achieve the requirement to dynamically switch between
low level behaviors independently from high level behaviors,
the LowLevelBehaviorManager organizes, similar to the High-
LevelBehaviorManager, the switching of the top level logic.
The top level logic that should be used, will be usually selected
by the running top level behavior.
C. Implementation
For the implementation of the framework, the Robotic
Operating System 2 (ROS2) [1] is used. Beehave has been
implemented as two independent ROS2 nodes, which have
been written in Python 3: The high level behavior manager is
implemented in one node and the low level behavior manager
in the other node. Both nodes use the ROS2 communica-
tion infrastructure which uses the Data Distribution Service
(DDS) [13] protocol for communication between nodes.
Behaviors and Actions are able to set up their own ROS2
subscribers to fetch the data they need to execute their tasks.
Actions are also able to set up publishers. These subscribers
and publishers are cleaned up automatically by the framework
once the Action or Behavior respectively, has been finished.
Behaviors and Actions are being executed in their own threads
to make them fully independent of each other.
Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on December 04,2023 at 17:59:01 UTC from IEEE Xplore. Restrictions apply.
The low level behavior manager makes heavy use of the
Python asyncio library. All logics and HALs are implemented
as asynchronous tasks so that they can be executed by an
asyncio event loop in a single thread. This makes the imple-
mentation of HALs and logics a lot easier. The user may use a
* logic to publish the blackboard periodically via a ROS2 topic
such that the higher level behaviors and other ROS2 nodes are
able to fetch the current device state. HALs shall update the
blackboard by subscribing to relevant ROS2 topics and may
Debug D. Evaluation
An agent that has been implemented using beehave has been
tested in simulations using the ROS simulation environment
Gazebo [14] and on a social automated transport vehicle for
intralogistical processes called EMILI [12].
EMILI consists of a differential drive, multiple human ma-
chine interaction interfaces such as displays, LEDs, speakers
and bluetooth low energy [15] interfaces for e.g. smartphone
app connectivity. Additionally, EMILI consists of a scissor
lifting system, multiple laser scanners for a free navigation
and a ground camera for line following navigation.
In detail, two top level behaviors and multiple sub behaviors
have been implemented to test the HighLevelBehavior. First
of all, a test behavior which systematically tests most features
of the HighLevelBehavior has been implemented by letting
EMILI drive to several positions. The deductive behavior of
the decision processed by the HighLevelBehavior regarding
LEDs and a display of EMILI was handled and implemented in
the LowLevelBehavior by defining a logic for the LED control
and a second logic for the display control.
In detail, the LED logic was implemented to show specific
illuminations of EMILIs LEDs for its current movement state
e.g. when a free navigation based on LiDAR sensors is active,
EMILIs LEDs are illuminated in a light blue but when the
battery power is low some LEDs of EMILI light up red. Sim-
ilar the display logic communicates to the display controller
that it should display e.g. a symbol for a low battery power
when the the battery power is low. This behavior resulted in
no failures and all functions of beehave as described above
could be used without errors.
The second implemented behavior communicates with an-
other agent represented by an laptop and receives orders from
it to change its position and syncing coordinate systems.
The connectivity between these two agents was established
using DDS as used by ROS2. This second behavior was
tested successfully as well as a switching between both above
mentioned behaviors.
Additionally to both above mentioned larger tests which
mainly focus on testing beehave as a whole system, many
small tests of each feature of both the HighLevelBehavior and
LowLevelBehavior took place by e.g. specifically testing the
functionality of the state machine implementation and testing
if e.g. race conditions are able to corrupt the safety features
of the HighLevelBehavior. All those smaller tests had been
successfull as well.
71
 IV. C ONCLUSION
To conclude, we presented the behavior modeling frame-
work beehave for software agents. Simple use cases have
been given to exemplary describe the usage of the framework
components as well as the successful evaluation of some of
these use cases. The framework allows a straight forward
approach of agent design and helps to achieve quick results and
deploy working technology in decentralized control systems.
Since it is used to implement decentralized robotic behaviors,
we think it is only straight forward to have a framework like
beehave to rely on the software agent design principles.
V. O UTLOOK
After presenting the feasibility of our approach on one
mobile robot platform we want to implement the framework
on different types of robots and further devices with various
operating systems. We will discuss internally and with the
community, whether we there is a demand for the framework
to be available for other programming languages as well.
Furthermore, we intend to integrate more features e.g. learning
into our behaviors by using Keras [16]. Additionally, we want
to evaluate the decentralized approach in more complex real
world environments with more heterogeneous agents which are
interacting with one another for example to fulfill transport and
production tasks. After testing beehave in these more complex
use cases we plan to further improve our framework based
on the experiences made in in the above mentioned, more
complex, use cases.
R EFERENCES
[1] Open Source Robotics Foundation, Inc. (OSRF). (2020-05-18) ROS2
Documentation. [Online]. Available: https://index.ros.org/doc/ros2/
[2] M. ten Hompel, M. Putz, and andreas Nettstraeter, “Whitepaper: Social
networked industry,” Fraunhofer IML, Tech. Rep., 2016.
72
Authorized licensed use limited to: Corporacion Universitaria de la Costa. Downloaded on December 04,2023 at 17:59:01 UTC from IEEE Xplore. Restrictions apply.
[3] ZVEI and VDI, “Reference architecture modul industrie 4.0 (rami4.0),”
ZVEI and VDI, Status Report, Jul. 2015.
[4] A. Kamagaew, J. Stenzel, A. Nettstraeter, and M. ten Hompel, “Concept
of cellular transport systems in facility logistics,” in The 5th Interna-
tional Conference on Automation, Robotics and Applications, Dec 2011,
pp. 40–45.
[5] J. Jost, T. Kirks, P. Gupta, D. Luensch, and J. Stenzel, “A new approach
towards safe human-robot-interaction in highly flexible warehouses,” in
IEEE International Conference on Intelligence and Safety for Robotics.
IEEE, 2018.
[6] T. Kirks, J. Jost, T. Uhlott, and M. Jakobs, “Towards complex adap-
tive control systems in intralogistics,” in The 21st IEEE International
Conference on Intelligent Transportation Systems. IEEE, 2018.
[7] J. Bohren. (2020-05-18) “smach - ros wiki”. [Online]. Available:
http://wiki.ros.org/smach
[8] (2020-05-15) Jade site - “java agent development framework”. [Online].
Available: https://jade.tilab.com/
[9] R. Agis, S. Gottifredi, and A. Garcia, “An event-driven behavior trees
extension to facilitate non-player multi-agent coordination in video
games,” Expert Systems with Applications, p. 113457, 04 2020.
[10] J. Bohren. (2020-05-18) “smach_viewer - ros wiki”. [Online]. Available:
http://wiki.ros.org/smach_viewer
[11] P. Iñigo Blasco, F. Diaz-del Rio, M. C. Romero-Ternero, D. Cagigas-
Muñiz, and S. Vicente-Diaz, “Robotics software frameworks for
multi-agent robotic systems development,” Robot. Auton. Syst.,
vol. 60, no. 6, pp. 803–821, Jun. 2012. [Online]. Available:
http://dx.doi.org/10.1016/j.robot.2012.02.004
[12] J. Jost, T. Kirks, M. Fiolka, and S. Chapman, “Sociotechnical design of
industrial transport vehicle and its interaction with humans in manufac-
turing systems,” in International Conference on Applied Human Factors
and Ergonomics. Springer, 2018, Orlando FL USA, pp. 36–48.
[13] G. Pardo-Castellote, B. Farabaugh, and R. Warren, “An introduction to
dds and data-centric communications,” Real-Time Innovations, 2005.
[14] N. Koenig and A. Howard, “Design and use paradigms for gazebo,
an open-source multi-robot simulator,” in 2004 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS)(IEEE Cat. No.
04CH37566), vol. 3. IEEE, 2004, pp. 2149–2154.
[15] C. Gomez, J. Oller, and J. Paradells, “Overview and evaluation of
bluetooth low energy: An emerging low-power wireless technology,”
Sensors, vol. 12, no. 9, pp. 11 734–11 753, 2012.
[16] A. Gulli and S. Pal, Deep learning with Keras. Packt Publishing Ltd,
2017.