A Repeated Negotiation Game Approach To Distributed 2020 Simulation Modelli

Simulation Modelling Practice and Theory 98 (2020) 101980
Contents lists available at ScienceDirect
Simulation Modelling Practice and Theory

journal homepage: www.elsevier.com/locate/simpat
A repeated-negotiation game approach to distributed (re)

T
scheduling of multiple projects using decoupled learning
Laura Tossellia, Verónica Bogadob, , Ernesto Martínezc
⁎
a
Departamento Ingeniería en Sistemas de Información, UTN FRVM, Av. Universidad 450, Villa María, X5900 HLR, Córdoba, Argentina
b
CIT Villa María (CONICET-UNVM), Carlos Pellegrini 211, Villa María, Córdoba, Argentina
c
INGAR (CONICET – UTN) – UTN FRSF, Avellaneda 3657, Santa Fe, S3002GJC Santa Fe, Argentina
ARTICLE INFO ABSTRACT
Keywords: With peer-to-peer software technologies based on Blockchain and Smart Contracts, automated
Decoupled learning, distributed (re)scheduling, negotiation of client-server relationships for enterprise networking and project-oriented fractal
repeated games organizations can now be readily implemented. To this aim, the distributed (emergent) schedule
Bilateral contract negotiation of client-server contracts must be a Nash equilibrium from which any agent finds no incentive to
Project-oriented fractal organizations
deviate. Also, to respond effectively to unplanned events and disturbances, the renegotiating
Nash equilibrium
process of all concerned client-server contracts must pursue a new Nash equilibrium solution in
the face of incomplete information by each individual agent. In this work, distributed multi-
project (re)scheduling is formulated as a repeated-negotiation game in a multi-agent setting
where each agent resorts to decoupled learning rules for deciding the terms and conditions in
each contract settlement. After a finite number of stages of the negotiation game, a new emergent
schedule close to a Nash equilibrium is found. An agent-based simulation framework is proposed
to implement the repeated negotiation game based on bilateral client-server contracts. The ef-
fectiveness of the proposed approach is demonstrated using a case study of a project-oriented
fractal company in the pharmaceutical industry. Simulation results obtained highlight that re-
peated negotiations and decoupled learning are key for approaching a Nash equilibrium and
welfare solutions to a (re)scheduling problem.
1. Introduction
Blockchain, the distributed ledger technology with the potential to eliminate business intermediaries [1], may also have the
potential to make a profound impact on enterprise networking by fostering virtual markets of client-server relationships in viable
fractal organizations [2]. By allowing peer-to-peer direct contact between a client for a product or service with a potential server or
provider while guaranteeing the secure establishment of trusted relationships, blockchain's smart contracts will change the way firms
all over the world make temporal business alliances and contribute to supply chains and value networks [3]. More specifically,
software platforms like Ethereum can readily implement smart contracts [4] for client-server relationships in a project-oriented
fractal company [5]. In the multi-project setting of fractal organizations, the way to negotiate the terms and conditions of smart
contracts requires special consideration, though. Without a central authority and assuming selfish behavior of participating agents,
the emergent scheduling resulting from the client-server contracts must be a Nash equilibrium from which any agent finds no
incentive to deviate [6]. Otherwise, there may exist severe losses for many of them.
⁎
Corresponding author.
E-mail addresses: ltosselli@frvm.utn.edu.ar (L. Tosselli), vbogado@frvm.utn.edu.ar (V. Bogado), ecmarti@santafe-conicet.gob.ar (E. Martínez).
https://doi.org/10.1016/j.simpat.2019.101980
Received 30 April 2019; Received in revised form 26 July 2019; Accepted 31 August 2019
Available online 01 September 2019
1569-190X/ © 2019 Elsevier B.V. All rights reserved.
L. Tosselli, et al. Simulation Modelling Practice and Theory 98 (2020) 101980
The complexity of (re)scheduling multiple projects grows in a networked enterprise context, where the information is decen-
tralized, and each actor has a partial perspective of the global schedule. A multi-agent approach provides mechanisms to deal with
this incomplete information situation, where each individual decision-making agent should face the partial problem. Negotiating
agents (clients and servers) have strategic types or private information that are not known by the other concerned agents they are
interacting with [7]. Each agent only reveals its type by actions taken alongside a repeated negotiation game. Thus, solving a
distributed (re)scheduling by repeating peer-to-peer negotiations is a Bayesian sequential game [8]. This is also true when dealing
with unplanned events and disturbances which leave obsolete initial schedules for some projects and demand disruption management
by rescheduling them [9]. It is argued here that to overcome the lack of information about agent types decoupled learning in a
repeated-negotiating game setting would help approaching a Nash equilibrium and a welfare solution to the rescheduling problem
after a finite number of stages [10].
Particularly, the related literature to project (re)scheduling problem is scarce, mainly considering a multi-project context and
different sources of uncertainty, including tasks and resources disruptions. There is still an important need for new approaches that
incorporate in an agent-based approach completely decentralized decision making and decoupled learning as fundamental capacities
to create an initial schedule and to face the dynamism and uncertainty of the real environment, allowing to improve the agent's
decisions and the resilience capacity of the multi-agent system to solve the (re)scheduling problem.
In this work, a novel repeated-negotiation game in a peer-to-peer market of client-server relationships is proposed in the context
of a Project-oriented Fractal Company. A “Learning in a black-box” approach allows negotiating agents to try different actions
(bilateral contract terms) in repeated interactions and only the payoffs they receive are used in their learning curve towards a Nash
equilibrium. The proposed approach revolves around the simple idea that project and resource agents may learn a Nash equilibrium
solution to the (re)scheduling problem from repeated strategic interactions. In this sequential negotiation game, any agent does not need to
know neither the overall game structure nor the preferences, or types of other agents, merely uses a decoupled learning rule to pinpoint the best
terms and conditions of its contract from which it finds no incentive to deviate.
The remainder of the paper is organized as follows. Section 2 summarizes existing related work on decentralized multi-project
scheduling and (re)scheduling. Section 3 describes multi-project (re)scheduling in Project-oriented Fractal Organizations as a bi-
lateral contract network that defines an emergent schedule and where participating client and server agents assume roles as project
and resource managers. The trial-and-error learning approach and payoff functions of agents involved in the bilateral contracts are
detailed in Section 4. In Section 5, a repeated-negotiation non-cooperative game with incomplete information is proposed to solve the
(re)scheduling problems by modeling the strategic behavior of the agents in a peer-to-peer market of client-server bilateral contracts.
A simulation framework of multi-agent based (re)scheduling for multiple projects in a Fractal Company is presented in Section 6,
along with relevant details of its implementation in a NetLogo prototype. Simulation results and a discussion based on a case study
from process development in the pharmaceutical industry are presented in Section 7. Finally, some conclusions and future research
directions are presented in Section 8.
2. Related work
Multi-project environments are common in real-world problems, where projects run in parallel and many resources are shared
among them. This section surveys related work that studies both multi-project scheduling and (re)scheduling problems.
2.1. Multi-project scheduling
In the existing literature, multi-project scheduling in deterministic environments have been addressed using different computa-
tional methodologies such as evolutionary optimization, linear and non-linear mathematical programming, multi-agent systems, and
game theory, among others. Evolutionary algorithms have been proposed to define the resource capabilities considering the total
budget and solve the multi-mode multi-project scheduling problem without resource transfer among projects [11]. Particularly, using
linear and non-linear optimization methods is the most common alternative in the related literature to solve the multi-mode multi-
project scheduling problem under running time constraints [12]. Centralized solutions based on mathematical programming are
adequate when the interdependency among multiple projects is low enough and the environment is stable.
The use of the key concept of agency has been used to deal with the multi-project scheduling problem, considering both single and
multi-mode setting, through agent collaboration and negotiation to ease the access to shared resources and deal with conflicting
project goals related to cost, time, efficient use of resources, etc. In [13], the project scheduling at the strategic level within an
organization is modeled using a multi-agent system, where different agents decide which projects to plan and which to reject, using
cost-based metrics and considering only global resources. More specifically, for the single-mode case, multi-agent systems are pro-
posed for the allocation of different resources among project tasks through an auction-like mechanism, using a coordinating agent
(centralized decision making) for conflict resolution and communication among concerned agents [14–16]. In [17], a multi-agent
system is proposed where each project is controlled by an agent who negotiates through multi-unit auctions using different de-
centralized mechanisms for defining the auction winners, without a mediator agent. One common feature of the approaches in
[14–17] is that have been developed to solve larger single-mode multi-project problems with thousands of activities sharing global
resources, through auction or negotiation mechanisms, aiming to minimize each project delay. The approach presented in [15] has
some limitations regarding project sizes, i.e., it may not be successfully applied to solve large-scale problems involving thousands of
activities in many different projects.
Similarly, different agent-based approaches have been applied to the multi-mode multi-project scheduling problems [18,19]. In
2
[18] an agent-based approach using a reinforcement learning algorithm is used to model the negotiation strategy of involved agents
(one dispatcher agent and numerous agents responsible for each activity) and the order and mode in which the activities must be
processed. In [19] an agent-based system using heuristic algorithms where two types of agents are involved is proposed. In the
proposed multi-agent system, local agents search for solutions in the local projects whereas global agents optimize the global solution
through the coordination of local projects.
Several works using Agent-based and Game Theory approaches have been developed to obtain schedules that optimize a global
function associated with project constraints such as duration and cost, while others consider the Nash equilibrium as a concept to
search for efficient solutions from both the individual (selfish) and collective viewpoints. Project scheduling problem is modeled in
[20] as a non-cooperative game using a polynomial time algorithm, where project activities are distributed among agents that want to
maximize their profit, while an optimal MILP (Mix Integer Linear Programming) modeling and resolution method for the multi-agent
project scheduling problem is proposed in [21]. Both works search for a Nash equilibrium that minimizes the project makespan.
Most of the proposed approaches to the project scheduling problem are based on the implicit assumption that the resulting
schedule will be implemented as was initially defined, meaning that unplanned events and significant disruptions can be somewhat
accommodated by enough slacks incorporated in a rather conservative global solution. The afore-mentioned works describe situa-
tions which are not representative of actual project environments. For example, in the software industry, 80% of initial project
schedules are affected by unplanned events during their execution, thus compromising deadlines, budgets and estimated resource
usage [22].
2.2. Multi-project (re)scheduling
During their execution, projects are subject to considerable uncertainty, which may lead to numerous schedule disruptions.
Regarding the project (re)scheduling problem, different approaches has been proposed. The (re)scheduling problem considering
multiple resource disruptions affecting the initial schedule has been developed using linear [9,23] and non-linear programming [24],
while the problem of coping with unexpected activities has been addressed in [25], through a new polynomial algorithm aiming at
inserting a new activity inside an existing initial schedule. The proposals in [26–28] resorts to evolutionary algorithms approaches to
face the disturbances that affect each project schedule. Shen et al. [26] presents a proactive-rescheduling method to adapt an existing
schedule in responding to tasks and resources disruptions that occur in a software project context. Weixin et al. [27] presents a
method that explores the type of disturbances that may arise in a multi-mode context and measures its impact over the project
schedule and then define the recovery strategy that minimize disturbance impact. Elloumi et al. [28] proposes a reactive multi-
objective heuristic to deal with activity disruptions in a multi-mode context, considering two particular schedule goals: makespan and
stability.
A multi-agent system based on negotiation protocols has been developed to face (re)scheduling of large projects in construction
industry aiming to minimize the overall project cost [29]. A market-based approach for scheduling airport ground handling services is
proposed in [30], assuming unlimited availability of global resources (a quite unrealistic assumption in multi-project real-world
problems), and inserting slack time at the end of activities in the baseline schedule to make enough room for handling the possible
disruptions caused by expected events. In [31], an agent-based model enabling multi-project rescheduling is proposed using trial-and-
error learning combined with linear programming. In [32] an agent-based simulation system is presented to address the multi-project
(re)scheduling problem based on both a proactive as well as reactive approaches, defining different types of agents to represent
projects and resources, which are coordinated by a central agent aiming to minimize each project completion time.
However, most of works mentioned above do not consider (re)scheduling problems in multi-project contexts under disruptions of
different types, and do not find a solution in a completely decentralized and distributed way (without a mediator agent). A completely
decentralized way where decision makers can find a solution to the multi-project (re)scheduling problems are more representative of
real-world problems, where the privacy of selfish decision makers is preferable because they may be potential competitors in the
client-server market of a fractal company.
3. Bilateral client-server contract networks for project (re)scheduling
The Project-oriented Fractal Company is a conceptual enterprise model which allows addressing complex optimization problems
such as the distributed multi-project (re)scheduling through peer-to-peer negotiations among autonomous units named fractals or-
ganized at different levels of abstraction. This approach provides an organized structure which serves as an enterprise model to define
a contract network between clients and servers. Each fractal management unit is modeled as a project. Thus, the fractal unit is
composed of a managed object and a manager. The managed object is a plan or schedule for completing a project at a given level of
abstraction or using a given resource over time. The manager can play one of the two following roles: end-manager when the managed
object is a project, sub-project or task schedule, or mean-manager if the managed object is a resource schedule [5].
Projects at different abstraction levels are interdependent, may run in parallel with partial overlapping and have distinctive
attributes such as deadline, budget, desired start and finish time, resource requirements, deliverables, precedence relationships and
fork/joint synchronization constraints. Each project is part of a recursive structure, where a project can be composed of a number of
sub-projects, and, in turn, each one of these sub-projects can be composed of other sub-projects, and so on and so forth. The recursive
decomposition of a project may go all the way down to its simplest form, i.e., a task that is also considered the atomic form of a
project. Resources traded through bilateral contract have attributes such as maximum availability rate, capacity and processing cost
per time period, as well as the type of projects in which they have the capability to participate.
3
Fig. 1. Bilateral contract network in a fractal company.
Each project manager is represented as an agent, which has private information about its managed object and carries out ne-
gotiations to agree the terms of client-server contracts with other agents in the fractal company. Let A be a finite set of agents that can
be responsible for the schedules of the different projects. Agents playing end-manager roles are defined as Project Agents PA ⊆A,
while those that play mean-manager roles are defined as Resource Agents RA ⊆A. A project agent (named pai) manages the schedule of
a project, sub-project or task x and a resource agent (named rak) manages the schedule of a resource r. These agents want to establish
client-server relationships at different abstraction levels to accomplish different goals closely related to the expected deliverables and
resource usage performance metrics. All potential client-server relationships between different project and resource agents in the
fractal company define a network of feasible bilateral client-server contracts (Fig. 1).
A bilateral client-server contract with the purpose of work delegation is established between two project agents that belong to
different abstraction levels (super-index l), with the restriction that a project agent at lower hierarchical levels cannot delegate work
to a project agent at higher levels in the fractal hierarchy. In Fig. 1, examples of potential contracts for work delegation are those
where pail acts as client when it delegates its work to two sub-projects and the agents in charge of them, pail++11 and pail++21, act as
servers to pail . At the same level of abstraction, a feasible contract for a resource provision/consumption is established between a
project agent and a resource agent that play roles of a client and a server respectively. Examples of feasible contracts for resource
provision/consumption are those between pail that demand some resources and rakl and rakl++11 that manage the required resources.
Let C be the set of all feasible bilateral client-server contracts (Contract Network) in the fractal company. A bilateral contract
c ∈ C establishes a client-server relationship defined as a 4-tupla c = (client , server , ct , p) where:
1 client and server are roles that a project agent and a resource agent can perform; the former is a service or resource consumer and
the latter is a service or resource provider in a given contract c.
2 ct = (y1, …, yw ) is a vector of w elements that define the agreed upon terms of a client-server contract; among the terms of a contract
are included the start and finish time of project, resource usage, costs. CT is defined as the set of all posible contract terms.
3 p : CT is a function that depends on the contract terms, defining a real number representing the value or utility of the
contract to concerned agents.
The possible actions for each concerned agent are to accept or reject the terms in the contracts in which it can participate as a
client or as a server. Let Ce ⊆ C be the set of established contracts as a result of actions taken by all agents at time t. The set of
established contracts in the fractal company at time t defines an emergent schedule s:
st = Ce C (1)
Thus, a finite sequence of Peer-to-Peer (P2P) negotiations of client-server bilateral contracts eventually gives rise to a Nash
equilibrium solution to project (re)scheduling. A Nash equilibrium (NE) solution is a joint strategy (one for each agent) in which all
agents have no incentives to deviate (i.e., if a player decides to choose a different action, it cannot improve its payoff) [33]. In this
work, after a finite number of stage games the terms of bilateral contracts for project schedules are established and the resulting
overall outcome defines an emergent global schedule from which clients and server agents find no incentive to deviate unless an
unexpected event or significant disturbance occurs which asks for renegotiating the terms of affected client-server contracts. Thus,
repeated negotiations are an indirect mechanisms for partially revealing different agents’ preferences and types so that the set of
4
Fig. 2. Bilateral contract network in a distributed scheduling problem.
bilateral contracts defines a Nash equilibrium solution from which no selfish agent has any interest to change [34,35].
Fig. 2 depicts the distribution and abstraction levels of the involved agents in a particular distributed scheduling problem of a
fractal company, where all feasible client-server relationships that can be established among project and resource agents are shown.
This figure also highlights that each agent has a limited (local) vision of its environment in the fractal company which defines a
narrow set of client-server relationships for such agent within the bilateral contract network. Therefore, the decisions regarding
which contracts and their corresponding terms that would be accepted or rejected by any agent will affect not only its project
schedule, but more importantly the global schedule of the fractal company.
For instance, in Fig. 2, Task 1.1.1 can be managed by pa62 or pa72 , which define two different mode of performing the same task, so
pa2 will accept a contract with one of these, and will reject the contract with the other one. The same occurs with Task 1.2.1.
1
Furthermore, accepted or rejected contracts by resource agents ra42 and ra32 affect the local schedule of resources R4 and R3 re-
spectively, while those accepted or rejected by project agents pa62 and pa72 impact on the local schedule of Task 1.1.1, and so on and
so forth.
In Fig. 3(a), the set of established bilateral contracts between agents and the emergent global schedule at time t are shown based
Fig. 3. Established contracts and emergent schedules before and after an unplanned event.
5
on the contract network of Fig. 2. For example, a bilateral contract for work delegation and outsourcing the completion of sub-project
Sp1 is established between project agents pa10 (client) and pa21 (server), responsible of schedules of project P1 and sub-project Sp1.
Additionally, bilateral contracts for resource provision/consumption are defined as for example those established between agents pa62
(client) and ra42 (server) that manage the schedules of task T1.1.1 and resource R4. The local and global schedules are made based on
to the established contracts.
Should unplanned events occur during the global schedule execution, the contract network in the fractal company allows agents
to renegotiate their established contracts terms that were affected by unplanned events. These events may cause changes to these
contracts or other ones included in the set of established client-server relationships before any unplanned events occur. The new set of
established contracts and their terms as well as the corresponding emergent schedules obtained at time t+1 are shown in Fig. 3(b),
responding to an unplanned event related to a lower available amount of resource R4. As consequence, a new contract
(<<NEW>>) is established as a result of repeated negotiations between the project agent pa82 and the resource agent ra32 that
manage the schedules of the task T1.1.2 and resource R3, respectively. This new client-server contract also impacts on the schedules
of Sp1 and P1, which emerge from the project schedules agreed by the agents related to T1.1.2 and R3.
4. Trial-and-error learning for contract terms negotiation
Bilateral contracts in a Project-oriented Fractal Company are elaborated outcomes of matching between client and servers under
uncertainty and involve a set of decisions that each interacting agent must take considering the terms and price of alternative
contracts that it may accept or reject after each negotiation round. The main difficulty in multi-project (re)scheduling is that bilateral
contracts must be agreed upon under incomplete information about other agents´ private information (or types) which ask for a
negotiation mechanism to deal with such limitations to rational decision-making. In order to maximize each agent utility during
client-server matching and contract negotiation, it is necessary that private information is partially revealed in a voluntary and
decentralized way, considering only the expected payoff of the resulting contracts. In this work, the incorporation of a payoff-based
learning mechanism using a black-box approach named Trial-and-Error Learning is proposed, with the aim that agents in the fractal
company increasingly learn to establish the most beneficial contracts both individually and collectively. The black-box approach is
incorporated in this work because it allows that a Nash equilibrium (or near-equilibrium) solution arises from simple and adaptive
rules of individual-based learning, considering a large and complex environment where client and server agents have selfish ways of
behaving (i.e., strategic actors) and are mostly in the dark about others’ available actions and preferences and how they affect their
own payoffs from contract terms. Particularly, in the trial-and-error learning approach, an emergent solution is considered as a Nash
equilibrium where all involved agents are “content” and their benchmark actions constitute a Nash equilibrium, i.e., the action taken
by each agent is the best response to the set of actions established by the rest of agents [10,34].
For the incorporation of trial-and-error learning, each agent's private information is represented as an internal state
z = (m , a¯, u¯ ) , which is made up of the agent's mood m that may be content (c), discontent (d), hopeful (h) or watchful (w). The second
entry of z corresponds to the reference or benchmark action (ā) whereas the last entry of the state vector accounts for a reference or
benchmark payoff (ū ). The agent's mood determines its search method regarding current contract matching and agreed upon terms
and prices, defining whether the agent will search for alternative contracts or will continue choosing contract terms and matching
partner that are part of its benchmark action. The second component (ā) of z is information derived from already established
contracts which are considered the utility maximization action that the agent would take whenever it is not searching for better
options. Finally, the term (ū ) represents the maximum achievable payoff that the agent expects to get based on the matching and
contract terms in the benchmark action.
In this work, the payoff functions defined for each agent type are based on the prices p of bilateral contracts. In this proposal, the
price p of a contract considers two performance indicators of a project schedule, which are defined according to the contract terms ct:
Tardiness Cost (tc) and Processing Cost (pc) (Eq. (2)). tc is defined as the cost incurred by a given project when finishing at a moment
of time after its estimated delivering time, i.e., calculated based on the difference between actual delivery time (ft) and the expected
(scheduled) time of established contracts with project server agents, considering a penalty factor (pf) in monetary terms for the
project delay (Eq. (3)). Thus, the tardiness of the project i depends on the resource availability for its execution; pc is the cost of
resources computed considering the required quantity of each resource (rq) to achieve project goal or deliverable and the cost of all
needed resources (rc) (Eq. (4)), where TQis total number of resources needed for the processing. However, new performance in-
dicators can be added as contract terms to contemplate other aspects of project scheduling, such as quality of project outcome, etc.
p = tc + pc (2)
where
tc = (ft edd )* pf (3)
TQ
pc = rqq *rcq
q=1 (4)
The payoff function for project agents, which can establish contracts playing roles of client or server, considers the contract price
minimization when playing as a client (i.e. minimization of pc, price p when is client c) and contract price maximization when playing
as a server (maximization of ps, price p when is server s). As an example, for defining utilities, the project agent pa62 in Fig. 3(a) is
considered. This project agent plays as a client in the contract with resource agent ra42 and plays as a server in the contract with
6
Fig. 4. State transition rules for payoff maximization. (For interpretation of the references to color in this figure legend, the reader is referred to the
web version of this article.)
project agent pa21. Therefore, the payoff function for the project agent i is defined as follows:
ui (a) = max ps pc
(client , pai, ct , p) Ce (pai, server , ct , p) Ce
(5)
Otherwise, the payoff function for a resource agent k is defined as:
uk (a) = max ps
(client , rak , ct , p) Ce
(6)
Thus, at the start of a contract negotiation, an agent chooses an action a considering its mood m, either exploring alternative
contract terms and possible matchings with some probability Ɛ (exploration rate) or, otherwise, selecting the same matching and
contract terms based on its benchmark action, which means that it is exploiting his knowledge about the effect of its actions. Once all
bilateral contracts are agreed upon, every agent assesses its internal state z = (m , a¯, u¯) by taking into account its current mood and
the received payoff by the established contracts. As a result of this evaluation, the internal state of each agent may change.
Fig. 4 summarizes the agent state transitions, considering a maximization of a payoff function. Note that the color blue is as-
sociated with the content mood (c), red with the discontent mood (d), green with the hopeful mood (h), and yellow with the watchful
mood (w). For the case of minimizing a cost function, the sign of the inequality's relationships should be reversed.
Transitions “a”-“k” are based only on the received payoffs for the actions taken, according to the following rationale:
• Content: If the mood of agent i is “content”, it may choose its reference action ā with probability (1 − Ɛ) or experiments a new
i
action ai with probability Ɛ (the parameter Ɛ defines the exploration rate for each agent type). If the agent i decides to explore, it
evaluates the payoff ui associated with choosing ai as follows: if ui < ūi, the agent's state remains unchanged, i.e. z = (c, āi, ūi);
otherwise, with probability Ɛ its state will change to (c, ai, ui(ai)), where a new action and payoff benchmark are defined. If the
agent i decides not to experiment, i.e. when choosing its reference action (āi), and perceives an increase or a decrease in the payoff
obtained, then the agent's mood becomes either “hopeful” or “watchful”, respectively.
• Hopeful-Watchful: If the mood of agent i is “hopeful” or “watchful”, the agent chooses its reference action āi. However, despite no
action is taken by agent i, due to the actions of other agents, it can perceive an increase or a decrease in its payoff leading to a
mood transition (transitions “f” to “k”).
Particularly, transitions “l” and “m” are defined by a probability function called response function φ, which is monotonically
increasing in the utility u and monotonically decreasing in ū for the utility maximization strategy and conversely for cost mini-
mization strategy. The corresponding state transitions are defined as follows:
• Discontent: If player i is “discontent” it experiments a new action a . Thus, with a probability function called response function, the
i
mood turns out content, and its new state is (c, ai, ui(ai)). Otherwise, the state of the ith agent remains (d, āi, ūi).
Examples of project agent state transitions after the establishment of client-server contracts at time t+1 are shown in Fig. 5. For
7
Fig. 5. Examples of project agent state transitions.
the sake of simplicity, resource agent state transitions are not shown in this Fig.. These examples are based on the problem instances
in Fig. 2, and changes resulting from applying the learning trial-and-error rules that are shown in Fig. 4. The different moods for
concerned project agents are represented by different colors: blue, red, yellow and green for content, discontent, watchful and hopeful
mood, respectively. The left and right local schedules are the input and output schedules before and after the establishment of
bilateral contracts at time t+1.
In Fig. 5(a), state transitions “b” and “e” are depicted. In the state transition “b”, the project agent pa62 that manages the processing
Task 1.1.1 has an internal state with mood “content”, where its benchmark action is the client-server contract between Task 1.1.1 and
the resource R4. The contract terms, i.e. the price, the start and release time of Task 1.1.1 in resource R4, are 20, 1 and 2, respectively.
The benchmark payoff for project agent pa62 is 20 (see Eq. (5)). During contract negotiations at t+1, project agent pa62 explores
alternative contracts due to an unplanned event that causes the non-availability of the resource R4. As a result of this exploration, a
new client-server contract with the resource agent ra52 is established. Thus, the mood of the project agent pa62 remains “content” and
the received payoff for the new contract is 10, representing an improvement in the contract payoff received by the agent pa62
(Eq. (5)). In state transition “e”, the involved project agent is pa82 (manager of Task 1.1.2) which has a bilateral contract with resource
agent ra42 and its mood is “content”. This agent is benefited by the newly established contract of project agent pa62 with resource
agent ra52 , which gives rise to a smaller delay for Task 1.1.2 and a possible improvement in its payoff (from 40 to 39). Accordingly, its
mood changes to “hopeful” and the contract terms with resource agent ra42 remain unchanged.
In Fig. 5(b), different agent state transitions are shown. Transition “d” involves project agent pa10 2
(manager of Task 1.2.1), whose
mood is “content”, where its benchmark action is the contract with the resource agent ra42 , and its benchmark payoff is 10. At time t
+1, the agent pa10 2
chooses its benchmark action (the agreed upon contract in the previous negotiation episode), so its internal state
remains unchanged corresponding to the benchmark payoff. Finally, transition “l” arises, when the internal state of project agent pa13 2
changes from “discontent” to “content”, where a new bilateral contract is established with agent ra4 , giving an improvement in the
2
received payoff (from 50 to 40) after the evaluation of response function φ, which is calculated in terms of the obtained and the
reference payoff.
5. Repeated negotiation game
A stable set of bilateral contracts among different project and resource agents must be agreed upon in order to find a Nash
equilibrium solution to the distributed multi-project (re)scheduling problem. To this aim, a mechanism that favor strategy
8
interactions such that all concerned agents have the possibility to reconsider different matchings, terms and prices of bilateral
affected contracts, or new contracts, through repeated negotiations before selecting those definitive matching and terms where they
have no incentive to deviate, i.e., a Nash equilibrium solution (or a near one) to multi-project (re)scheduling [33–35]. A repeated-
negotiation non-cooperative game with incomplete information is proposed to solve the (re)scheduling problem by modeling the
strategic behavior of the agents in a peer-to-peer market of client-server bilateral contracts. Moreover, it is defined as a sequential
game since agents participate in different bilateral negotiations and optimal matching demands assessing alternative clients or
servers. In the proposed repeated-negotiation game, both the initial and modified global schedules arise from decentralized repeated
renegotiations of contracts among all involved participating agents. The goal of the repeated negotiation game is to find an equi-
librium solution in response to unplanned events, trying to minimize the impact caused by the occurrence of such events and to
maximize the social welfare of the fractal company.
The global rescheduling goal is expressed in terms of the minimization of two sub-goals: project makespan and schedule total cost.
At the end of the negotiation game, the set of established bilateral contracts made up the emergent schedule, which can continue its
execution until a further disruption occurs. The feasibility of the global schedule is guarantee by the very negotiation process of
bilateral contracts that accounts for the constraints in resource availability and precedence/synchronization relationships between
projects at different abstraction levels in the fractal company.
The proposed game is composed of M-stages where players (agents) can “try” different matchings, terms and prices before
deciding a definitive and enforceable bilateral contract. A stage is a strategy game where players negotiate through auction-based
protocols considering their moods about their internal states and exploration rate Ɛ to celebrate bilateral contracts to accomplish
different goals related to expected deliverables and resource usage. The definition of each negotiation protocol is detailed in
Section 5.1. During a game stage, an emergent schedule (Preliminary Schedule) is obtained, and all client and server agents at
different abstraction levels receive their provisional payoffs calculated according to payoff functions defined in Eqs. (5) and (6).
Accordingly, concerned agents update their internal states, following the decoupled learning rules described in Section 4. Thus, the
established contracts by each agent and the received payoff obtained in each stage directly influences on its actions in future stage
games and, indirectly, in the actions of other agents. This defines the learning by trial-and-error approach. The negotiations generate
a new global schedule, which is evaluated in the context (environment), feeding the agent learning for the next stage. This process is
performed M times to achieve a schedule at the end of the game, which is considered as Baseline Schedule.
Then, the baseline schedule execution starts where the involved agents in each contract are liable to follow its stipulated terms.
This process makes room for the agents to explore the network space of the bilateral contracts in the fractal company. During the
project schedule execution, an unplanned event can occur, such as the non-availability of a resource, making the schedule infeasible
and giving raise the need of renegotiating the terms in the affected contracts. Thus, a new repeated negotiation game starts involving
only the agents with affected contracts. These agents must renegotiate the terms of these contracts or define new contracts using what
they learned in the previous games, aiming to obtain a new feasible schedule of the projects in the fractal company. Once the current
game ends, the execution of the modified schedule with new established contracts is restarted (Project Schedule Execution). Bilateral
contracts are renegotiated every time a new unplanned event occurs until all the planned goals and deliverables in the schedule have
been achieved. In Appendix B, the game is graphically depicted.
5.1. Automated negotiation protocols
A negotiation protocol is a set of rules governing the strategic interactions between project agents [36]. In this work, automated
negotiation protocols are defined as iterative and auction-based processes, where agents act either as auctioneers or bidders [8,37].
Negotiation protocols without a central authority and assuming selfish behavior of the participating agents are proposed, for ex-
ploring different matchings and terms/prices in bilateral contracts between client and server project agents. These protocols provide
agents the opportunity to decide without being fully informed about other agent types and preferences as well as learn on an
individual basis.
The first negotiation protocol (NP1) deals with the establishment of client-server contracts between project agents at different
abstraction levels of the fractal hierarchy and is related to bilateral contracts where a work delegation of a part of a project will be
managed by other agent down in the hierarchy. The second negotiation protocol (NP2) is about the establishment of client-server
contracts between project and resource agents involving resource usages for project completion. Both protocols are defined through
three sequential loops: Announcement, Bidding and Establishment. Loops Announcement and Establishment consider two alternative
paths, to determine if an agent accepts or rejects proposals made by other agents. For a detailed definition of each protocol, UML
sequence diagrams [38] were used.
The negotiation protocol NP1 is defined in Fig. 6. This negotiation protocol is used when a project agent must delegate work
related to sub-projects that compose the project that is managed. During the Announcement loop, a project agent that wants to
delegate work (auctioneer) announces the creation of sub-projects for delegating responsibility. Here, two situations may occur
depending on the mood of the project agent that acts as auctioneer and the exploration rates of project agents (Ɛ_pa). If the current
strategy of the auctioneer is to explore a new action (alternative [(mood = ”content” and explore = true) or (mood = ”discontent”)]), it
sends messages of delegation requests (message 1) to all project agents (bidders) at lower levels at the fractal company that can be
interested in assuming the responsibility for achieving the goal and objectives of a sub-project. Otherwise, if its strategy is play its
benchmark action and its mood is “content”, “hopeful” or “watchful” (alternative [mood <> ”discontent” and exploit = true]), it
sends delegation request (message 2) only to a project agent (bidder) that corresponds to its benchmark action.
In the Bidding loop, project agents at the immediately lower level of abstraction that receive delegation requests must evaluate
9
Fig. 6. Sequence diagram of the negotiation protocol NP1.
them determining to which requests will respond to considering their mood states and exploration parameters (Ɛ_pa), and send their
estimated bid (messages 3 and 4) for each accepted request. Project agents (acting as bidders) calculate each bid value to define the
delegated goal (in case the bid is accepted, this value is defined as the contract price). Thus, if a project agent that receive delegation
requests is exploring a new action (alternative [(mood = ”content” and explore = true) or (mood = ”discontent”)]) will respond to all
delegation requests; otherwise it will only respond to the project agent that belongs to their benchmark action. Then, each required
project agent (sub-project agents) sends its bid (message 5) and sends non-acceptance messages (message 6) to those projects from
which it does not accept delegation requests.
Finally, in the Establishment loop, a project agent that sends delegation requests performs an iterative process to evaluate the
received bids (message 7); first it sorts all the bids and then selects the one which gives the highest benefits considering the contract
terms and price which are the object of the negotiation (message 8). Once a given bid is selected, the project agent sends messages to
all concerned project agents (bidders) to report the winner of the delegation auction (message 9). As a result, a temporal contract is
created (message 10), where its price is defined according to the winner bid. Those project agents whose bids have been rejected
receive a message rejectBids (message 11). NP1 protocol ends when the work to be delegated has been assigned.
On the other hand, negotiations to define bilateral contracts for resource usage can be established through the protocol NP2
(Fig. 7). At the beginning of this protocol, the Announcement loop is executed when project agents, whose managed projects are
unscheduled, decide sending process request messages (message 1 and 2) to a subset of resource agents depending on how the
contract network was defined based on declared skills and competences. In the Announcement loop, there are two alternatives to
select which resource agents that will receive processing requests as shown in the protocol NP1, considering the mood and ex-
ploration rate.
10
Fig. 7. Sequence diagram of the negotiation protocol NP2.
In the Bidding loop, each resource agent that receives a processing request must evaluate it. To pinpoint which processing request
will be responded, each resource agent considers its mood state and exploration rate (Ɛ_ra). There are two alternatives to follow as it
is shown in the Bidding loop described in protocol NP1. For each processing request which is accepted, resource agents calculate their
bids (message 3 and 4), considering the start and final processing time of the project in their resources, as well as the project
processing costs. Once all bid values have been calculated, these are sent to project agents who have started the negotiation for
resource usage contracts (message 5). Also, messages of rejected bids are sent (message 6).
Finally, in the Establishment loop, each project agent performs an iterative process to evaluate received bids (message 7), which
aims to select one bid that minimizes the development time and cost of its managed project. Once a given bid is selected (message 8),
each project agent sends messages to resource agents (bidders) to inform the winners of the resource usage auction (message 9) or to
reject the bids (message 12). If a resource agent receives more than one message that defines him as the auction winner (message 9),
it must select the proposal that maximizes its profits, according to its goal (message 10). Then, a new bilateral contract (message 11)
is created, where its price is defined according to the bid value of the winner. Finally, the new contract becomes part of the project
schedules of the involved agents in the negotiation game.
6. Agent-based simulation
6.1. Simulation framework
The overall simulation framework of multi-agent distributed (re)scheduling of multiple projects in a Fractal Company is shown in
Fig. 8. This framework defines the elements necessary to implement resource and project agents, their learning mechanism, the
contract network, the sequential game and the company environment. The proposed simulation framework includes two main parts:
the Fractal Company environment and the Multi-agent System (MAS). The former has a Fractal Company Management System that
manages the instances of projects with their schedules and all related information. This information regarding the projects, along
with their scheduled tasks, precedence restrictions, execution times and resource requirements, as well as information on available
alternative resources, their availability constraints and processing capacities is used and updated by the MAS. The latter includes
11
Fig. 8. Multi-project (re)scheduling simulation framework.
modules to control the simulated game execution, emulate unplanned events and measure the resulting solutions (emergent sche-
dule) to verify that agent policy tends to achieve scheduling goals. Therefore, the MAS is a structure with three abstraction layers
from the most general and strategic one to the lowest and concrete one, where the goals for the global schedule are defined in the
form of performance metrics.
The top layer called Agent Contract Network provides the elements to create the interacting agents (project and resource agents)
and their possible client-server relationships from all project instances and their detailed information. Also, in this layer, the contract
network is dynamically defined from the resulting negotiation provided by the lower layers. The intermediate layer called Agent
Communication contains the modules that implement the negotiation protocols (NP1 and NP2), from which the set of client-server
contracts that define a global emergent schedule is obtained. The client-server relationships established in this level are used at the
top level (Agent Contract Network) to build the contract network. The decisions made by the agents using the negotiation protocols
(e.g., in each negotiation or re-negotiation) are based on the evaluation of the defined performance indicators for each type of agent
at the lowest level. In this bottom layer, Agent Performance Metrics, a set of quantitative metrics is defined. For project agents, the
metrics are associated with the tardiness and resource usage costs, while for resource agents the metrics are related to actual usage
times and cost per unit time. These measures are taken considering the terms of the involved contracts.
12
Quality assessment of the emergent global schedule is performed by the Fractal Company Management System via a Payoff
Functions module Eqs. (5) and ((6), Section 4), which are computed through prices agreed in bilateral contracts. This module
evaluates the goodness of the global schedule considering the actions taken by the different agents (at lower levels of abstractions).
Then, each agent receives the corresponding payoff to its action and updates its internal state based on decoupled learning by trial-
and-error (TE Leaning). As a result, every time an unplanned event gives rise to an unfeasible global schedule, the affected agents can
find a new Nash equilibrium through multiple negotiations. To this aim, each agent in the negotiation game behaves selfishly
following its own objectives: reduce time and costs in the case of project agents and maximize resource usage in the case of resource
agents.
Furthermore, in the proposed framework a module to calculate Global Indicators is defined. The global indicators are used to
measure the emergent schedule performance. Particularly, in this work, two global metrics are proposed. The first one is an indicator
named Total Project Delay (TPD) which is calculated for the global project of the fractal company at the most abstract level, for each
stage played in the game. Eq. (7) defines TPD metric:
TPD = MSm edd (7)
where MSm is the project makespan of the global project at stage m and edd is the estimated due date for the global project assuming
an infinite amount of resources (considered as its critical path). The project makespan (MS) of the global project is defined as:
MS = ftm rd (8)
where ftm is the finalization time of the latest task of the global schedule obtained at the stage m considering the actual amounts of
available resources, and rd is the release date, i.e. the earliest time at which the tasks of global project can start.
The second one is the grand cost of the emergent global schedule of the fractal company at each stage of the game, defined as:
TR
Cost = (rqq, m *rcq, m )
q= 1 (9)
where rq is the quantity of each resource required for the global schedule at the stage m, rc is the cost per time unit of each resource
needed, and TR is the total number of resources required for completing the global schedule.
Another indicator is the welfare of the Fractal Company after a stage game of N-players in the contract network. The welfare
metric is defined as the sum of the players’ payoffs from their benchmark actions for each agent [35]. In this work, the welfare state of
the game is the set of client-server contracts that define the global schedule of all projects at the end of each stage. Mathematically
speaking, the social welfare is defined as:
N
W (st ) = (ui (a¯))
i=1 (10)
The proposed framework is completed with the detailed design of the two types of agents, namely Project and Resource Agents,
and a client-server relationship between two peers that formalizes each bilateral client-server contract.
Project agent (Table 1): Each project agent has an identification of its managed project instance and a name. This agent im-
plements trial and error decoupled learning rules based on its partial knowledge of the global schedule and interacts with other
project and resource agents, depending on the type of the client-server relationship, i.e., delegation or resource provision in all the
Table 1
Project agent design detail.
Project agent
Properties: ID, name, project start time, project finish time, resource requirements, precedence constraints, dependence constraints, possible servers.
Internal state: a mood, an action, a payoff for performing this action
Goals: Minimize project tardiness cost and resource usage cost
Negotiation Role:
Protocol NP1: Acting as client or server, either the goal of a contract depending on if it is work delegation or resource provision, respectively.
Protocol NP2: Acting as client.
Responsabilities Colaborators
Request resource provision Resource Agent
Request sub-project delegation Project Agent
Sign contracts for delegation and resource provision Resource or Project agent, according to the purpose of contract
Update its Schedule details
Calculate performance metrics.
Performance Metrics: tardiness cost and resource usage cost.
Actions:
Accept contracts for the delegation of sub-projects with another project agent
Reject contracts for the delegation of sub-projects with another project agent
Accept contracts for the resource provision with a resource agent
Reject contracts for the resource provision with a resource agent
13
Table 2
Resource agent design detail.
Resource agent
Properties: ID, name, resource cost, resource maximum availability, possible clients.
Internal state: a mood, an action, a payoff for performing this action
Goals: Maximize resource usage.
Negotiation Role:
Protocol NP2: Acting as server.
Responsabilities Colaborators
Respond to resource provision request Project Agent
Sign contracts for resource provision Project Agent
Update its Schedule details
Calculate performance metrics.
Performance Metrics: resource usage and cost.
Actions:
Accept contracts for the resource provision with a project agent
Reject contracts for the resource provision with a project agent
bilateral contracts in which it participates. More specifically, the information handled by a project agent includes project start and
finish times, possible resource servers, quality and rate requirements of the necessary resources, precedence and synchronization
constraints among tasks as well as its project due date. There is at least one project agent instance per each project and task of the
fractal company.
Resource agent (Table 2): Each resource agent has an identification of a managed resource instance and a name. Each resource
agent implements trial-and-error decoupled learning rules based on its partial knowledge of the global schedule and interacts with
some project agents through bilateral contracts involving resource usage. A resource agent manages the schedule of its own resource
which involves processing tasks for different project agents. Particularly, the resource information it manages includes resource cost
per unit time, maximum availability and owner of the scheduled tasks. Also, it may know project agents, which are potential clients
for closing a service contract. A resource agent instance is created for each resource in the fractal company.
Bilateral contract (Table 3): formal agreement between two parties, client and server as defined in the previous sections. Each
contract is defined as an object that specify the agreed upon terms for project deliverables involved in the client-server relationship.
The Contract element of the framework is instantiated when a client-server relationship is established during the (re)scheduling
process, creating as many contracts as concrete client-server relationships are established to build a global schedule in the fractal
company.
6.2. Implementation of the simulation framework
The Agent-based simulation framework has been implemented in NetLogo to mimic the strategic interactions in the fractal
company of the involved project and resource agents and their goals. NetLogo is a multi-agent simulation environment for agent-
based modeling and simulation that allows evaluating emergent behaviors resulting from ongoing interactions among autonomous
and learning agents and the environment in which they are immersed in [39,40].
NetLogo defines four types of agents: the observer (command center), the turtles (mobile agents), the links (connectors) and the
patches (fixed agents). The turtles and links can be modeled in different breeds. In this work, two breeds of turtles are implemented
called ProjectAgents and ResourceAgents for each type of agent in the fractal company. Additionally, a breed of links named Contracts
was defined to represent the negotiated client-server contracts. Some properties are common for all agent breeds in NetLogo such as
who, color, shape, label among others. Besides, each breed has its own properties that implement the peculiarities of each type. For
example, the breeds ProjectAgents and ResourceAgents have a name, a state, a hierarchical level, among others that implements
features of each type of agent defined in previous sections. In the particular case of the breed Contracts, it keeps the client, the server
and attributes of the relationship established between the two agents.
Negotiation protocols and learning rules are implemented through NetLogo primitives, which define the list of procedures shown
in Fig. 9. Appendix C provides some extra details of the main procedures of the developed prototype in NetLogo source code.
Table 3
Contract design detail.
Bilateral contract
Client: Project Agent.

Server: Project Agent or Resource Agent.
Contract Terms: price, contract start, contract finish.
14
Fig. 9. Procedures in NetLogo prototype.
The graphical user interface of the prototype is shown in Fig. 10 to test and experiment different scenarios for distributed (re)
scheduling of multiple projects. The user controls the parameter setting for the model in each simulation run. The main parameters
are classified in three types:
• Input data: each problem instance is imported from external sources (from Fractal Company Management System, Fig. 8), where
allows defining the projects to be scheduled, precedence relationships, estimated start and finish time of projects, types and
requirements of resources (renewable and non-renewable) for project execution, maximum availability and processing costs of
local and shared resources.
• Simulation parameters: time occurrence and type of disruptive events to simulate, weighted goal scheduling to achieve (considering
project delay, project cost or a trade-off among them), number of stages in the repeated game (NumStages).
• Learning parameters: exploration-exploitation rate Ɛ for each turtle breed (exploration-resource and exploration-project); parameter
theta (θ) used to define the response function for state transition discontent-content; option to decrease gradually or abruptly the
value of Ɛ for each turtle breed as the game stages advanced.
The World in NetLogo (black zone) is divided up into a grid of patches over which turtles can move. Thus, as output of each game
15
Fig. 10. Graphical user interface of the NetLogo prototype.
stage, a Gantt chart view of the emergent global schedule is obtained. Graphically, each project (set of “box” shape, where each box is
a time unit) corresponds to a row in the Gantt chart and a set of turtles (with “person” shape) represents the involved resources for
processing the project in the corresponding row. Each pair project-resource is the result of a client-server contract established during
the (re)scheduling process.
Additionally, to assess the payoffs and state evolutions for different agents that demonstrate individual and collective emergent
learning occur, different graphic and output files are generated. Data in these files are used by the Fractal Company Management
System to update the emergent global schedule after a disruptive event occurs.
7. Simulation results
7.1. Case study description
This section presents a case study of a Fractal Company related to pharmaceutical product development to validate the proposal
[41,42]. The development process for a new drug consists of the following phases: Discovery, Preliminary Development, Clinical Trials
and Approval. For a given product development project, a main project (Project 1) is divided into two sub-projects that are executed
concurrently (Sub-project 1 and Sub-project 2) are considered. In this case study, each sub-project aims to complete the Clinical Trials
and Approval phases of a new drug. Tables 4 and 5 provide the information associated with both sub-projects and resources involved
in the case study. Particularly, for each task belonging to sub-projects SP1 and SP2 the required resources and estimated duration
according different modes, as well as precedence relationships (Pred.) are listed. All resources are renewable and shared, and the
maximum availability and processing costs per unit time are defined for each of them.
For instance, Task 1.1.1 that belongs to SP1 must be performed according to one of the two different modes M1 and M2. In mode
16
Table 4
Input project data.
Id. Description Task modes Pred.
SP1 SP2
M1 M2 M1 M2
Req Dura Req Dura Req Dura Req Dura
1 Clinical Trials – – – – –
1.1 Phase I – – – – 1
1.1.1 Recruit healthy volunteers 2T1 (3) 1T3 (2) 2T1 (2) 1T3 (1) 1.1
1.1.2 Define tolerable maximum dose 2T1 (8) 1T2 (5) 2T1 (6) 1T2 (3) 1.1.1
1.1.3 Describe side effects 2T1 (2) 1T3 (1) 2T1 (2) 1T3 (1) 1.1.2
1.2 Phase II – – – – 1.1
1.2.1 Trials on affected patients 2T1 (4) 1T2 (2) 2T1 (4) 1T2 (2) 1.1.3
1.2.2 Form. and dose bioavailability 2T1 (16) 1T3 (9) 2T1 (12) 1T3 (7) 1.2.1
1.3 Phase III – – – – 1.2
1.3.1 Large-scale trials 1T2 (1) 1T3 (1) 1T2 (1) 1T3 (1) 1.2.2
1.3.2 Safety tests 1T2 (9) 1T3 (9) 1T2 (8) 1T3 (8) 1.3.1
1.3.3 Comparative studies 2T2 (5) 2T3 (5) 2T2 (3) 2T3 (3) 1.3.2
1.4 Formulation development 2° Stage – – – – 1.3
1.4.1 Additional trials on animals 2T1 (2) 1T2 (1) 2T1 (1) 1T2 (1) 1.1.3
1.4.2 Development of variations 3T1 (14) 2T2 (8) 3T1 (11) 2T2 (6) 1.4.1
1.4.3 Validation of stability 2T1 (2) 1T3 (1) 2T1 (4) 1T3 (1) 1.4.2
1.4.4 Manufacturing process dev. 2T1 (10) 1T3 (6) 2T1 (6) 1T3 (3) 1.4.3
2 Approval – – – – 1
2.1 Authorization – – – – 1.3
2.1.1 Documentation clinical trials 2T1 (5) 1T3 (3) 2T1 (4) 1T3 (2) 1.3.3
2.1.2 Documentation Expert opinion 1T1 (2) 1T2 (1) 1T1 (1) 1T2 (1) 2.1.1
a
Unit time: semester.
Table 5
Input resource data.
Id. Description Type Shared Max. availability Processing cost
1 R1 T1 True 40 60
2 R2 T2 True 70 65
3 R3 T3 True 70 80
4 R4 T1 True 40 60
5 R5 T2 True 70 65
6 R6 T3 True 70 80
M1, the task requires the service of two units of resource of type T1 for its processing with an estimated duration of 3-time units.
When the task is processed in mode M2, one unit of resource of type T3 is required with an estimated duration of 2-time units,
respectively.
7.2. Disturbances and unexpected scenarios
During task processing of project schedules are exposed to environmental conditions. As a result, both the durations of tasks and
the availability of resources suffer the effect of variability, disturbances and unplanned events. Thus, significant differences with
respect to the initially planned schedules may come from different sources: tasks that may take more than estimated, temporal
unavailability of resources over time, new tasks that must be processed to guarantee project completion, budget overrun, climatic
conditions that cause delays, etc. [9]. Table 6 describes scenarios related to the occurrence of different unplanned event types which
Table 6
Unexpected scenarios.
Scenario Unplanned event Affected object
S1r Reduction in a 50% of the resource R3 processing capacity due to training activities. Resource
S2r Total absence of resource R3 per license. Resource
S3r Increase of 20 % in processing capacity of resource R3 due to additional work time. Resource
S1t Increase in resource requirements of task 2.1.1, SP2. Project
S2t Increase in the estimated times of task 2.1.1, SP2. Project
S3t Decrease in the estimated times of task 1.2.1, SP1. Project
17
Table 7
Simulation parameter setting.
Parameter Value
Number of simulation runs per scenario 10

Number of stages per game 100
Initial mood agents discontent
Initial Ɛ 0.100
Decrement value of Ɛ per stage 0.004
Unit time Six-monthly
are common during process development [43]. Each scenario considers the occurrence of one unplanned event, and the time of this
occurrence was set according to the nature of this problem. These scenarios allow exploring the dynamics of the Simulation Fra-
mework to assess the hypothesis defined in this work. Thus, the developed prototype responds to different unplanned events related
to changes in different resource and project attributes, such as availability, desired start and finish time of projects, and resource
requirements. Among the first events are the following: (i) decreased availability of resources (for instance, equipment maintenance
periods, training activities, budget reduction, etc.); (ii) absence of a resource for a certain period of time (for instance, due to operator
illness, vacations, maternity, and machine breakdown among others); and (iii) increased availability of resources (for example, the
performance of overtime by a team member or the increase of a equipment's operating time). Among the unplanned events related to
project attributes are: (iv) changes in project start and finish time, and (v) changes in resource requirements for project completion.
For simulating in the prototype the above mentioned scenarios, the following assumptions are enforced: (i) all predecessor tasks
must be finished before their successor tasks can begin (precedence relationships “Finish-Start”), i.e., project agents can only parti-
cipate in negotiations whenever their managed project, sub-project or task comply with these conditions; (ii) projects, sub-projects or
tasks are assumed non-preemptive, i.e., they cannot be interrupted once they have started; (iii) resources can be shared by all projects
and are considered either renewable or non-renewable, such as man-hours, physical resource hours, budget in money or kind, etc.;
(iv) (re)scheduling goals of concerned agents are selfishly aligned with minimizing time and costs of the global schedule belonging to
the Project-oriented Fractal Company.
Simulation parameters used for running the prototype are shown in Table 7, including number of stages per game, agent initial
mood, exploration-exploitation rate for each agent type, decreasing rate for the parameter Ɛ which lowers exploration as the se-
quential game progresses etc. Accordingly, in the first stages each agent has a greater freedom to try new actions that can provide a
higher payoff, but as the number of stages played increases, the exploration parameter is steadily decreased in order to favor that all
agents learn to select actions close to a Nash equilibrium.
7.3. Computational results
In Fig. 11, the values of global schedule delays and costs corresponding to the initial schedule and the best solution found using
the sequential negotiation game after the occurrence of unplanned events, calculated as the average of 10 simulation runs for each
Fig. 11. Schedule indicators during (re)scheduling.
18
Fig. 12. Gantt charts of feasible schedules as result of stage game before and after occurrence of an unplanned event. (For interpretation of the
references to color in this figure legend, the reader is referred to the web version of this article.)
19
scenario, are shown. More specifically, Fig. 11(a) highlights that in every scenario, the TPD value after (re)scheduling the proposed
multi-agent sequential negotiation approach finds a repaired schedule that improves the initial TPD.
Fig. 11(b) put forward a rather unexpected result: the cost of the global schedule after the occurrence of the unplanned is actually
lower due to individual agent learning in the sequence of negotiations. These results highlight that the proposed simulation-based
approach not only can respond effectively to unplanned events, but also can find solutions that allow improving the delay and cost of
the original global schedule as more stages of the sequential game are played. This game outcome can be understood as follows:
project and resource agents learn to select those contracts that give them higher benefits as the game progresses, demonstrating the
importance of repeated negotiations in finding better global schedules.
The Simulation Framework provides quantitative indicators at different moments of the sequential negotiation game to analyze
the intermediate (re)scheduling results in more detail. In Fig. 12(a), the initial schedule (baseline) obtained at the end of the first
game is shown in NetLogo graphical interface, where the set of established contracts among agents are described over the black
background zone (representing the “World” of agents). The different moods for the affected project and resource agents are re-
presented by different colors: blue, red, yellow and green for content, discontent, watchful and hopeful mood, respectively. During
execution of the global schedule, at the instant time t = 24 (red dotted line), the resource R3 is no longer available to process the
scheduled tasks (unplanned event defined in scenario S2r). Thus, those project agents that have established contracts with the
resource R3 and whose contracts have not been completed yet, must seek to establish new contracts with other available resource
agents to fulfill their objectives. For example, the Task 1.3.1, Task 1.3.3 and Task 1.4.3 that had contracts with resource R3 should
establish new contracts to secure processing its scheduled tasks in other resources. To this aim, a new negotiation game of M-stages is
played. At the end of the new repeated game, affected tasks by unplanned event were reassigned to R2 and R5 for its processing as
consequence of the newly established contracts. In Fig. 12(b), the modified emergent schedule after unplanned event (considering
now as schedule baseline) is shown. Task processing in the new schedule is restarted from the instant time t = 24. This gives rise to an
increase in the delay and cost of the SP1 sub-project schedule and, consequently, in the global schedule.
In Fig. 13, the averages of results obtained through the execution of 10 simulation runs per scenario are shown considering 100
stages per game. For each simulation run in each scenario, an unplanned event arises causing the need for (re)scheduling and the start
of a new sequential negotiation game between some project and resource agents in the fractal company. In this new game, only the
affected contracts that have not been closed yet at the time of the event occurrence are considered. For each scenario, Fig. 13 exhibits
two different graphs to demonstrate that the proposed approach represents an optimal solution both from the individual viewpoint of
each agent and from the collective perspective, in which all agents have no incentives to deviate, since they are in a Nash equilibrium
(NE) [34,35]. The upper graphs from (13 (a.1) through 13 (f.1)) show the evolution of the average payoff obtained by each set of
agents (project and resource agents), i.e., they put forward the global indicator for Social Welfare defined in Section 6. This average
payoff is calculated in each stage as the average of the Social Welfare (Eq. (10)) considering 10 simulation runs per scenario. As can
be seen, the average payoff obtained by project agents decreases through the different game stages, while the average payoff of
resource agents increases, demonstrating that each type of agent pursues its goals, i.e., the minimization of total duration and cost of
local schedules for project agents and maximization of resource usage for resource agents. Above results confirm that while all agents
in the fractal company do pursue their own goals in a non-cooperative selfish way, achieving better individual payoffs, repeated
negotiations translate into an improvement of the global schedule measured in terms of fractal company goals. Thus, benefits can be
gained from "trying" different actions through multiple negotiations instead of a single negotiation state before agreeing the terms of
bilateral contracts in client-server relationships.
Lower graphs of Fig. 13 (13 (a.2) to 13 (f.2)) show the evolution of the percentage of agents with mood "content" through the
different game stages. All these graphs reflect that, as the sequential game advances, the number of agents in the fractal company
with mood "content" steadily increases. This is because the exploration rate of all agents is gradually reduced, so most of them stop
exploring new actions and select those that provide them with higher benefits when facing unforeseen events, bringing all agents to
an equilibrium state. However, results obtained highlight that even though an equilibrium can be reached, a few agents still occa-
sionally explore and try sub-optimal actions because the probability of exploration is not negligible.
For each scenario, the metric TPD is calculated before and after unplanned events occur. For instance, in scenario S3r, where the
unplanned event considered is an increase of processing capacity of the resource R3, the TPD values obtained before and after the (re)
scheduling are 21.2 and 13.2 respectively. Simulation results demonstrate that completion times for sub-projects are actually reduced
because the concerned tasks can be processed more quickly, which positively impact on the global schedule total duration.
Additionally, for the scenario S2r (absence of resource R3), the TPD values before and after (re)scheduling are 20.5 and 12.9,
respectively. Such a reduction demonstrates that the repeated negotiation game for distributed scheduling makes enough room for all
concerned agents to establish new contracts with other available resources that were underutilized, and a global schedule with a
shorter duration is obtained. That is, the performance of the (re)scheduling solutions will depend on the unplanned event type being
considered and the scheduling goals pursued.
20
Fig. 13. Average payoff and “content” mood evolution per agent type.
21
Fig. 14. NE in each scenario.
Finally, results related to NE convergence are presented. In Fig. 14(a), the probability of convergence to an NE is shown, which is
calculated considering the number of agents with mood “content” at the end of each game stage, as an average of the results obtained
during the 10 simulation runs executed per scenario. As can be seen, the probability of convergence is increasing for all considered
scenarios and is close to 1 at the final stages of the repeated negotiation game. As a result, the stability in the decisions of agents is
improved, which demonstrates that in the proposed approach agents learn to select better actions to face unplanned events. Thus,
local schedules (managed by project agents of lower levels and resource agents) exhibit better results in advanced stages, resulting in
a better global schedule (made up from all local schedules) in terms of time and total cost. Particularly, scenarios S3r and S3t present
convergence to a near-equilibrium schedule since unplanned events that were considered (an increase in the processing capacity of a
resource and a decrease in the completion time of a task) impact positively on the local schedules of concerned agents, and therefore,
on the global schedule for the fractal company.
Fig. 14(b) exhibits results obtained from each simulation run (G1, G2, etc.) and each scenario (S1r, S2r, etc.), where can be
observed that the average number of stages per game to reach again an equilibrium state after unplanned event occurrence is 4, with
a minimum and maximum of 1 and 12 of a total of 100 stages per game. The maximum value was obtained in the simulation of the
unplanned event that is related to total unavailability of a resource (S2r), demonstrating that more stages are needed to recover from
an unplanned event that, in this example, affects negatively a greater number of agents.
7.4. Discussion
Distributed rescheduling of multiple projects in real time is a highly complex problem, even more considering the multiple
disruptions that can affect the global schedule in a fractal company and the complexity of sharing resources among multiple projects
Table 8
Multi-project problem instances subsets.
Problem instances subset L0 (project) L1 (sub-projects) L2 (tasks) N. resources
G L
MPJ30_a2 1 2 30 1;2;3 1;2;3

MPJ30_a5 1 5 30 1;2;3 1;2;3
MPJ30_a10 1 10 30 1;2;3 1;2;3
MPJ30_a20 1 20 30 1;2;3 1;2;3
22
Table 9
Results of APD obtained by different proposals.
Approaches Problem instances subset
MPJ30_a2 MPJ30_a5 MPJ30_a10 MPJ30_a20
Our approach 12,7 10,23 67,52 170,45

CMAS/ES [16] 13,8 18,52 84,4 198,4
CMAS/SA [16] 12,8 19,28 99,3 223,95
DMAS/ABN [14] 15,9 21,2 87,5 207,96
MAS/CI [44] 19 27 84,14 182,05
DMAS/RIA [45] 18 21,64 83 193
managed by a multitude of selfish agents. In this work, a repeated-negotiation game in a peer-to-peer market of client-server re-
lationships is proposed to allow negotiating agents to try different actions (matching and bilateral contract terms) and learning only
from the payoffs received. The repeated-negotiation game is an effective mechanism to obtain both initial and repaired emergent
schedules when unplanned events occur.
To the best of our knowledge, benchmark results that exist in the literature are referred to the scheduling of multiple projects, but
previous work do not consider the multi-project (re)scheduling due to the occurrence of unplanned events. For this reason, results
regarding to multi-project scheduling was obtained to compare and assess the quality of the proposed approach with other decen-
tralized methods. These results are based on a standard and representative set of multi-project problem instances (available in http://
www.mpsplib.com) that vary in their structure, parameters and number of shared resources. In Table 8 are shown the subsets of
problem instances taken used for benchmarking. To adapt the problem instances for comparison purposes with other proposals,
different levels (L0, L1, L2) are defined for each problem, describing the number of project agents (managing either a project, sub-
project or task) that interact at each level of fractal hierarchy. The sub-project of level L1 are considered as independent projects.
Thus, for simulation results, the performance metric Average Project Delay (APD) is calculated for the all stages of the game, defined
as:
TP
1
APD = (fti eddi )
TP i=1 (11)
where TP is the number of sub-projects at level L1 (2, 5, 10 or 20), fti is the finish time of the latest task in the generated schedule for
sub-project i during game stage m, and eddi is the estimated due date for sub-project i (considered as its critical path, which is the
same for all stages). Four resource agents are available for each project, divided into shared (G) and local (L). The number of global
resources can vary from 1 to 4. The maximum availability and processing capacity of different resources remains unchanged
throughout the simulation run of each problem instance.
From Table 9 can be seen that the proposed approach obtains better results for the four subsets of presented problems. The
obtained results in Table 9 vividly demonstrate the advantages of incorporating decoupled learning rules in all agents involved in the
resolution of multi-project (re)scheduling problems, which makes feasible to compare the results obtained with those reported in
other proposals. These results correspond to the management objective for scheduling defined here as minimization of the project
delay, considering the most efficient resource consumption without increasing the prescribed makespan, and avoiding resource over-
allocation conflicts.
To validate the proposed approach and demonstrate its capacity to obtain a feasible solution when dealing with unplanned events
and disturbances, simulation results a case study was presented (Section 7.1). The results obtained through the case study highlight
that agents learn to select the actions that increase their payoffs, demonstrating that they respond to different disruptive events by
stabilizing their own learning curves. This decentralized and distributed way of agent learning allows better response times to
unplanned events as well as facilitates the evolution of the proposed agent-based model to consider other indicators in collective
learning which allow adjusting it to the reality of the agents' environment.
8. Concluding remarks
In this paper, a novel repeated-negotiation game in a peer-to-peer market of client-server relationships is proposed for solving
distributed multi-project (re)scheduling problem in a non-cooperative enterprise networking context to face different types of
23
unplanned events. A learning in a black-box approach allows negotiating selfish agents to try different actions in repeated interac-
tions from which bilateral contracts are established and each agent receives a payoff from these contracts. Therefore, agents do not
need to know the complete state of the global project schedule, nor the actions taken by other agents in the fractal company. Thus,
this approach permit solving complex and distributed (re)scheduling problems, finding feasible solutions when unplanned events
arise in terms of project constraints (project delay and cost) and social welfare measures. Particularly, the model of a Project-oriented
Fractal Company is formulated as a contract network where agents interact to achieve their goals, having greater freedom of action
and response.
The developed Agent-based Simulation Framework implemented in NetLogo allows imitating the sequence of strategic interac-
tions in the Project-oriented Fractal Company when responding to unplanned events, obtaining repaired and equilibrium schedules in
a decentralized way through simple and decoupled learning rules based on trial-and-error approach (learning in a black-box ap-
proach) and repeated negotiations without centralized entities to achieve their goals, having freedom of action and response. In the
repaired schedule, all interacting agents have no incentives to deviate, hence is a Nash equilibrium.
The simulation results obtained through the presented case study demonstrate the capacity of interacting agents to respond to
different types of unplanned events while a state of equilibrium is achieved by means of decoupled learning rules. Results highlight
the effectiveness of presented approach to absorb the impact of unplanned events, in order to recover the “executability” of the
project schedule, with reasonable performance rates related to project constraints, such as time and cost. Moreover, the proposal finds
a feasible schedule in all experiment runs, providing a solution to the problem in all cases unlike other approaches. The too restrictive
availability of the resources could be one limitation of this proposal, since it would not permit the establishment of contracts because
of the lack of the needed resources. However, the proposed approach allows adding new resources dynamically.
As future work, we will include in the present prototype the evaluation of new scheduling objectives associated with the stability
of the solutions (such as resource leveling), together with new negotiation constraints related to the quality of the results obtained by
each agent and with the reliability of their offers. Additionally, new unplanned events that may occur due to more competitive
market of clients and servers will be considered in the proposed solution to reflect more accurately the complexity of real-world
industrial problems.
Acknowledgments
Financial support for this research study was granted by “Planificación automática en sistemas cognitivos de producción in-
tegrando aprendizaje por refuerzos con abstracciones lógicas y relacionales” PID 25/OR01 and “Evaluación dinámica de la calidad de
procesos y productos para la toma de decisiones tempranas en industrias de base tecnológica” PID UTN3581 of Universidad
Tecnológica Nacional.
Appendix A. Nomenclature
Notation Description
A Set of agents
PA Set of Project Agents
RA Set of Resource Agents
pail Project agent i on level l
rakl Resource agent k on level l
C Set of potential/feasible contracts
Ce Set of established/ contracts
client Contract client
server Contract provider
yw Contract term
p Contract price
u ( pai ) Payoff function of Project agent i
u (rak ) Payoff function of Resource agent k
st Schedule of Project-oriented Fractal Company at time t
z Agent state (m , a¯, u¯)
_pa , _ra Exploration/exploitation rates for project and resource agents
Response function to state transition discontent-content
Parameter theta for response function
24
Appendix B. Non-cooperative game
The proposed game in Section 5 is detailed in Fig. B.1, where the players, environment and activities during the proposed
sequential game are described.
Fig. B.1. Activity diagram of the multi-project (re)scheduling game.
25
Appendix C. NetLogo procedures
Figs. C.1 and C.2 describe NetLogo procedures generated to setup and run the simulation of the proposed repeated negotiation
game. The setup procedure invokes to other defined procedures so as to create and initialize agents and links, and also for setting
simulation and learning parameters. The start-game procedure is the core of the developed multi-agent solution, where procedures
that allow agents to negotiate and establish contracts, as well as update their states are invoked.
Fig. C.1. Procedure setup.
26
Fig. C.2. Procedure start-game.
References
[1] D. Tapscott, A. Tapscott, Blockchain Revolution: How The Technology Behind Bitcoin Is Changing Money, Business, And The World, Penguin Canada, Toronto,
Canada,, 2016.
[2] P. Hoverstadt, The Fractal Organization: Creating sustainable organizations with the Viable System Model, John Wiley and Sons Inc., Hoboken, NJ, USA, 2008.
[3] L. Einav, C. Farronato, J. Levin, Peer-to-peer markets, Annu. Rev. Econ. 8 (2016) 615–635 https://doi.org/10.3386/w21496.
[4] H. Kim, M. Laskowski, A perspective on blockchain smart contracts: reducing uncertainty and complexity in value exchange, 26th International Conference on
Computer Communication and Networks (ICCCN), 2017, pp. 1–6 https://doi.org/10.1109/ICCCN.2017.8038512.
[5] M. Canavesio, E. Martinez, Enterprise modeling of a project-oriented fractal company for SMEs networking, Comput. Ind. 54 (2007) 794–813 https://doi.org/10.
1016/j.compind.2007.02.005.
[6] M. Maschler, E. Solan, S. Zamir, Game Theory, Cambridge University Press, 2013.
[7] T. Baarslag, M. Kaisers, The value of information in automated negotiation: a decision model for eliciting user preferences, Proc. of the 16th International
Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2017, pp. 391–400.
[8] S. Fatima, S. Kraus, M. Wooldridge, Principles of Automated Negotiation, Cambridge University Press, 2015.
[9] G. Zhu, J.F. Bard, G. Yu, Disruption management for resource-constrained project scheduling, J. Oper. Res. Soc. 56 (2005) 365–381 https://doi.org/10.1057/
palgrave.jors.2601860.
[10] H.H. Nax, M.N. Burton-Chellew, S.A. West, H. Peyton Young, Learning in a black box, J. Econ. Behav. Org. 127 (2016) 1–15 https://doi.org/10.1016/j.jebo.
2016.04.006.
[11] U. Besikci, Ü. Bilge, G. Ulusoy, Multi-mode resource constrained multi-project scheduling and resource portfolio problem, Eur. J. Oper. Res. 240 (2014) 22–31
https://doi.org/10.1016/j.ejor.2014.06.025.
[12] T. Wauters, J. Kinable, P. Smet, W. Vancroonenburg, G. Vanden Berghe, J. Verstichel, The multi-mode resource-constrained multi-project scheduling problem.
The MISTA 2013 challenge, J. Sched. 19 (2016) 271–283 https://doi.org/10.1007/s10951-014-0402-0.
[13] J. Araúzo, J. Pajares, A. López-Paredes, Simulating the dynamic scheduling of project portfolios, Simul. Modell. Pract. Theory 18 (2010) 1428–1441 https://doi.
org/10.1016/j.simpat.2010.04.008.
[14] S Adhau, M. Mittal, A. Mittal, A multi-agent system for distributed multi-project scheduling: an auction-based negotiation approach, Eng. Appl. Artif. Intell. 25
(2012) 1738–1751 https://doi.org/10.1016/j.engappai.2011.12.003.
[15] G. Confessore, S. Giordani, S. Rismondo, A market-based multi-agent system model for decentralized multi-project scheduling, Ann. Oper. Res. 150 (1) (2007)
115–135 https://doi.org/10.1007/s10479-006-0158-9.
[16] J. Homberger, A (μ, λ)-coordination mechanism for agent-based multi-project scheduling, OR Spectr. 34 (1) (2012) 107–132 https://doi.org/10.1007/s00291-
009-0178-3.
[17] W. Song, D. Kang, J. Zhang, H. Xi, A multi-unit combinatorial auction based approach for decentralized multi-project scheduling, Auton. Agent. Multi. Agent.
Syst. 31 (2017) 1548–1577 https://doi.org/10.1007/s10458-017-9370-z.
[18] T. Wauters, K. Verbeeck, G. Vanden Berghe, P. De Causmaecker, A multi-agent learning approach for the multi-mode resource-constrained project scheduling
27
problem, Proc. of 8th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS), 2009, pp. 1–8.
[19] P. Jedrzejowicz, E. Ratajczak-Ropel, A-team solving distributed resource-constrained multi-project scheduling problem, Lect. Notes Comput. Sci. (2018)
243–253 https://doi.org/10.1007/978-3-319-98446-9_23.
[20] A. Agnetis, C. Briand, JC. Billaut, Nash equilibria for the multi-agent project scheduling problem with controllable processing times, J. Sched. 18 (2015) 15–27
https://doi.org/10.1007/s10951-014-0393-x.
[21] C. Briand, S.Ulrich Ngueveu, P. Sucha, Finding an optimal Nash equilibrium to the multi-agent project scheduling problem, J. Sched. 20 (5) (2017) 475–491
https://doi.org/10.1007/s10951-017-0516-2.
[22] Chaos Report, Standish Group, 2015, https://www.infoq.com/articles/standish-chaos-2015 , Accessed date: 16 October 2018.
[23] R. Chakrabortty, R. Sarker, D. Essam, Multi-mode resource constrained project scheduling under resource disruptions, Comput. Chem. Eng. 106 (2016) 13–29
https://doi.org/10.1016/j.compchemeng.2016.01.004.
[24] D. Pinha, R.S. Ahluwalia, P. Senna, The combinatorial multi-mode resource constrained multi-project scheduling problem, Int. J. Supply Oper. Manage. 3 (2016)
1391–1412 https://doi.org/10.22034/2016.3.03.
[25] C. Artigues, P. Michelon, S. Reusser, Insertion techniques for static and dynamic resource-constrained project scheduling, Eur. J. Oper. Res. 149 (2003) 249–267
https://doi.org/10.1016/S0377-2217(02)00758-0.
[26] X. Shen, L. Minku, R. Bahsoon, X. Yao, Dynamic software project scheduling through a proactive-rescheduling method, IEEE Trans. Softw. Eng. 42 (7) (2016)
658–686 https://doi.org/10.1109/tse.2015.2512266.
[27] W. Weixin, G. Xianlong, S. Jiafu, Multi-project scheduling with multi-factor disruptions under uncertain environments, Proc. of 8th Int. Conf. on Industrial
Technology and Management (ICITM), 2019, pp. 295–300 https://doi.org/10.1109/ICITM.2019.8710663.
[28] S. Elloumi, P. Fortemps, T. Loukil, Multi-objective algorithms to multi-mode resource constrained projects under mode change disruption, Comput. Ind. Eng. 106
(2017) 161–173 http://dx.doi.org/10.1016/j.cie.2017.01.029.
[29] K. Kim, B. Paulson, Multi-agent distributed coordination of project schedule changes, Comput. Aided. Civ. Inf. 18 (2003) 412–425 https://doi.org/10.1111/
1467-8667.00327.
[30] X. Mao, N. Roos, A. Salden, Stable multi-project scheduling of airport ground handling services by heterogeneous agents, Proc. of 8th Int. Conf. on Autonomous
Agents and Multiagent Systems, 2009, pp. 537–544 http://dx.doi.org/10.1145/1558013.1558087.
[31] H. Mitsui, H. Sugihara, H. Koizumi, J. Sawamoto, H. Tsuji, An agent-based scheduling method enabling rescheduling with trial and error approach, Electr. Eng.
Jpn. 159 (2007) 26–38 https://doi.org/10.1002/eej.20434.
[32] W. Song, H. Xi, D. Kang, J. Zhang, An agent-based simulation system for multi-project scheduling under uncertainty, Simul. Modell. Pract. Theory 86 (2018)
187–203 https://doi.org/10.1016/j.simpat.2018.05.009.
[33] J.F. Nash, Equilibrium points in n-person games, Proc. Natl. Acad. Sci. U. S. A. 36 (1) (1950) 48–49 https://doi.org/10.1515/9781400884087-007.
[34] H.P. Young, Learning by trial and error, Games Econ. Behav. 65 (2009) 626–643 https://doi.org/10.1016/j.geb.2008.02.011.
[35] S. Bary, R. Pradelski, H.P. Young, Learning efficient Nash equilibria in distributed systems, Games Econ. Behav. 75 (2012) 882–897 https://doi.org/10.1016/j.
geb.2012.02.017.
[36] S. Kraus, Strategic Negotiation in Multiagent Environments, The MIT Press, 2001.
[37] M. Fasli, Agent Technology for E-Commerce, John Wiley & Sons, Chichester, UK, 2007.
[38] UML Object Management Group, UML 2.5, (2015) http://www.omg.org/spec/UML/2.5 , Accessed date: 14 November 2018.
[39] J. Vidal, Fundamentals of Multiagent Systems with NetLogo Examples, (2009) http://www.scribd.com/doc/2094479/Fundamentals-of-Multiagent-Systems ,
Accessed date: 14 November 2018.
[40] S. Railsback, V. Grimm, Agent-Based and Individual-Based Modeling: A Practical Introduction, Princeton University Press, 2012.
[41] P. Harpum, Portfolio, Program, and Project Management in the Pharmaceutical and Biotechnology Industries, John Wiley & Sons Inc., New Jersey, USA, 2010.
[42] G. Pisano, The Development Factory - Unlocking the Potential of Process Innovation, Harvard Business Press, 1997.
[43] Project Management Institute, Practice Standard for Scheduling, Project Management Institute, 2007.
[44] J. Homberger, A multi-agent system for the decentralized resource-constrained multi-project scheduling problem, Int. Trans. Oper. Res. 14 (6) (2007) 565–589
https://doi:10.1111/j.1475-3995.2007.00614.x.
[45] S. Adhau, M.L. Mittal, A. Mittal, A multi-agent system for decentralized multi-project scheduling with resource transfers, Int. J. Prod. Econ. 146 (2) (2013)
646–661 https://doi:10.1016/j.ijpe.2013.08.013.
28

A Repeated Negotiation Game Approach To Distributed 2020 Simulation Modelli

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Repeated Negotiation Game Approach To Distributed 2020 Simulation Modelli

Uploaded by

Copyright:

Available Formats

Simulation Modelling Practice and Theory 98 (2020) 101980

Contents lists available at ScienceDirect

Simulation Modelling Practice and Theory

A repeated-negotiation game approach to distributed (re)

ARTICLE INFO ABSTRACT

2.1. Multi-project scheduling

2.2. Multi-project (re)scheduling

3. Bilateral client-server contract networks for project (re)scheduling

Fig. 1. Bilateral contract network in a fractal company.

Fig. 2. Bilateral contract network in a distributed scheduling problem.

4. Trial-and-error learning for contract terms negotiation

Otherwise, the payoff function for a resource agent k is defined as:

Fig. 5. Examples of project agent state transitions.

5. Repeated negotiation game

5.1. Automated negotiation protocols

Fig. 6. Sequence diagram of the negotiation protocol NP1.

Fig. 7. Sequence diagram of the negotiation protocol NP2.

6.1. Simulation framework

Fig. 8. Multi-project (re)scheduling simulation framework.

6.2. Implementation of the simulation framework

Client: Project Agent.

Fig. 9. Procedures in NetLogo prototype.

Fig. 10. Graphical user interface of the NetLogo prototype.

7.1. Case study description

7.2. Disturbances and unexpected scenarios

Number of simulation runs per scenario 10

7.3. Computational results

Fig. 11. Schedule indicators during (re)scheduling.

Fig. 14. NE in each scenario.

MPJ30_a2 1 2 30 1;2;3 1;2;3

Our approach 12,7 10,23 67,52 170,45

Appendix B. Non-cooperative game

Fig. B.1. Activity diagram of the multi-project (re)scheduling game.

Appendix C. NetLogo procedures

Fig. C.1. Procedure setup.

Fig. C.2. Procedure start-game.

You might also like