Computers & Industrial Engineering 76 (2014) 89–97

Contents lists available at ScienceDirect

Computers & Industrial Engineering

journal homepage: www.elsevier.com/locate/caie

Multi-agent based real-time production scheduling method for radio

frequency identification enabled ubiquitous shopfloor environment q
Yingfeng Zhang a,⇑, George Q. Huang b, Shudong Sun a, Teng Yang a
a
Key Laboratory of Contemporary Design and Integrated Manufacturing Technology, Ministry of Education, Northwestern Polytechnical University, Xi’an 710072, China
b
Department of Industrial and Manufacturing Systems Engineering, The University of Hong Kong, Hong Kong, China

a r t i c l e i n f o a b s t r a c t

Article history: The lack of timely feedback shopfloor information during manufacturing execution stage leads to signif-
Received 21 October 2013 icant difficulties in achieving real-time production scheduling. To address this problem, an overall archi-
Received in revised form 9 July 2014 tecture of multi-agent based real-time production scheduling is presented to close the loop of production
Accepted 12 July 2014
planning and control. Several contributions are significant. Firstly, wireless devices such as radio fre-
Available online 21 July 2014
quency identification (RFID) are deployed into value-adding points in a ubiquitous shopfloor environ-
ment to form Machine Agent for the collection and processing of real-time shopfloor data. Secondly,
Keywords:
Capability Evaluation Agent is designed to optimally assign the tasks to the involved machines at the pro-
Multi-agent
Real-time
cess planning stage based on the real-time utilization ration of each machine. The third contribution is a
Production scheduling Real-time Scheduling Agent for manufacturing tasks scheduling/re-scheduling strategy and methods
Re-scheduling according to the real-time feedback. Fourthly, a Process Monitor Agent model is designed for tracking
Ubiquitous shopfloor and tracing the manufacturing execution based on a critical event structure. Finally, a case is used to
demonstrate the proposed multi-agent based real-time production scheduling models and methods.
Ó 2014 Elsevier Ltd. All rights reserved.

1. Introduction facilitated. In a ubiquitous manufacturing environment, using

Production scheduling is the process of selecting and assigning
manufacturing resources for specific time periods to the set of
manufacturing processes in the plan (Shen, Wang, & Hao, 2006).
It is the important manufacturing planning actives which deal with
resource utilization and time span of the manufacturing opera-
tions. Agent-based manufacturing scheduling systems (Chan &
Zhang, 2002) are a promising way to provide this optimization.
Recently, rapid developments in wireless sensors, communication
and information network technologies (e.g. radio frequency identi-
fication – RFID or Auto-ID, Bluetooth, Wi-Fi, GSM, and infrared)
have nurtured the emergence of ubiquitous manufacturing
(Huang, Wright, & Newman, 2009; Zhang, Huang, Qu, & Ho,
2011; Zhang, Huang, Qu, Ho, & Sun, 2011) as core Advanced Man-
ufacturing Technology in next-generation manufacturing systems.
It will make the status of the manufacturing resources could be
ubiquitous, and the real-time shop-floor scheduling could thus be
q
This manuscript was processed by Area Editor T.C. Edwin Cheng.
⇑ Corresponding author. Address: B509, Hangkong Building, Northwestern Poly-
technical University, No. 127, Youyi West Road, Xi’an, Shaanxi, China. Tel.: +86
13991135396.
E-mail address: zhangyf@nwpu.edu.cn (Y. Zhang).
real-time manufacturing intelligence and information to perform
at a maximum level, with a minimum of unscheduled downtime,
would be a more effective approach to handling the negative per-
formance impacts of uncertainty (Wang, Adamson, Holm, & Moore,
2012).
Despite of the significant progress, the following research ques-
tions are still open in applying real-time scheduling methods to the
real-life manufacturing shopfloors with the increasing complexity
of the production processes and the unpredictability of the produc-
tion exceptions, etc.
(1) In process planning stage, tasks are assigned to a type of
machine, the real-time statuses of the resources are seldom
considered. As a result, tasks may not be optimally assigned
among the machines and has adverse effect on production
scheduling.
(2) Due to the lack of manufacturing information capturing and
processing methods, current shop-floor monitor is inaccu-
rate, incomplete, inconsistent, and presents time-delay. In
addition, the real-time information integration between
the scheduling system and the manufacturing execution
monitor system has not been well designed. These problems
further lead to the difficulties of implementing the real-time
shop-floor scheduling.

http://dx.doi.org/10.1016/j.cie.2014.07.011
0360-8352/Ó 2014 Elsevier Ltd. All rights reserved.
90 Y. Zhang et al. / Computers & Industrial Engineering 76 (2014) 89–97
(3) Currently, manufacturing tasks are characterized by a large
variety in small batch sizes, this production mode has high
demands of real-time monitoring for dynamic decision mak-
ing and adaptive control capabilities to rapidly deal with the
production changes. Therefore, to design a real-time sched-
uling architecture and method for the dynamically changing
environment has significant value and wide application
foreground.
To address these problems, some new production management
mode and architecture are investigated in recently. Cloud manu-
facturing (Tao, Cheng, Xu, Zhang, & Li, 2014; Wang, Wong, &
Wang, 2013), as a new manufacturing mode extending and adopt-
ing the concept of cloud computing, is proposed to realize the full-
scale sharing, free circulation and transaction, and on-demand-use
of manufacturing resource and capabilities in modern manufactur-
ing enterprises. By extending the IoT (The Internet of Things) con-
cept and techniques, an IoMT (The Internet of Manufacturing
Things) architecture (Zhang et al., 2014) is designed to implement
real-time interoperability and dynamical optimization for manu-
facturing shopfloors. In the IoMT architecture, the manufacturing
things such as operators, machines, pallets, materials can be
embedded with sensors, they can interact with each other.
Based on the concept and achievement of IoMT, in this research,
we integrate the advantages of multi-agent and auto-ID technolo-
gies to implement real-time shopfloor scheduling in a ubiquitous
manufacturing environment. The proposed multi-agent real-time
scheduling architecture aims to close the loop of production plan-
ning and control from process planning to finished products. The
implementation of this research is expected to increase productiv-
ity, as well as flexibility and responsiveness in a ubiquitous shop-
floor environment.
The rest of the paper is arranged as follows. Section 2 will
review the relevant literature under three categories of multi-
agent system for manufacturing, real-time scheduling, and ubiqui-
tous manufacturing. Section 3 presents the overall architecture of
multi-agent based real-time shop-floor scheduling. The multi-
agent models such as Machine Agent, Capability Evaluation Agent,
Scheduling Agent, and Process Monitor Agent are discussed in Sec-
tion 4. Section 5 describes the details of the GA-based solving
method. Section 6 demonstrates the proposed multi-agent based
real-time scheduling models and methods. Conclusions and further
works are summarized in Section 7.
2. Literature review
Related research is reviewed under three categories: (1) multi-
agent system for manufacturing, (2) real-time scheduling, and (3)
manufacturing information capturing technology.
Agent technology is a branch of Artificial Intelligence (AI) and
has been widely accepted and developed in manufacturing appli-
cations for its autonomy, flexibility, reconfigurability, and scalabil-
ity (Macchiaroli & Riemma, 2002; Maturana et al., 2004; Sikora &
Shaw, 1998). An agent based concurrent design environment
(Krothapalli & Deshmukh, 1999) has been proposed to integrate
design, manufacturing and shop-floor control activities. A compro-
mising and dynamic model in an agent-based environment (Sikora
& Shaw, 1998) has been designed for all agents carrying out their
own tasks, sharing information, and solving problems when con-
flicts occur. Papakostas, Mourtzis, Bechrakis, Chryssolouris, and
Doukas (1999) describe a flexible agent based framework for man-
ufacturing decision making. Some mobile agent-based systems
have been applied to the real-time monitoring and information
exchange for manufacturing control. Jia, Ong, Fuh, Zhang, and
Nee (2004) proposed an architecture where many facilitator agents
coordinate the activities of manufacturing resources in a parallel
manner. Jiao, You, and Kumar (2006) applied the MAS paradigm
for collaborative negotiation in a global manufacturing supply
chain network. Besides, in various kinds of applications such as dis-
tributed resource allocation (Bastos, Oliveira, & Oliveira, 2005),
online task coordination and monitoring (Lee & Lau, 1999;
Maturana et al., 2004), or supply chain negotiation (Wu, 2001),
the agent-based approach has played an important role to achieve
outstanding performance with agility. Monostori, Váncza, and
Kumara (2006) introduce the software agents and multi-agent sys-
tems and discuss the open issues and strategic research directions
in all domains of manufacturing where problems of uncertainty
and temporal dynamics, information sharing. An infrastructure
for co-ordination of multi-agents in a network-based manufactur-
ing system is designed in (Chan, Swarnkar, & Tiwari, 2007). Zhang,
Huang, Qu, Ho, and Sun (2011), Zhang, Huang, Qu, and Ho (2011)
describe an agent-based smart objects management system for
managing the heterogeneous auto-ID devices to capturing real-
time manufacturing information. Wang et al. (2013) present an
ontology based approach to organize multi-agent assisted supply
chain negotiations. Lim, Tan, and Leung (2013) propose a multi-
agent system to optimise the resources within a multi-site manu-
facturing environment, in particular through the integration of
process planning and production scheduling functions.
In order to satisfy customer requirements and meet the delivery
time punctually in MTO (Make to Order) environments, production
scheduling and planning is an important process for avoiding delay
in the production process and for improving manufacturing perfor-
mance. Previous approaches focus on the process allocation of
equipment to production tasks before the production starts
(Wong, Leung, Mak, & Fung, 2006). Aghezzaf (2007) adopts a mixed
integer programming model for developing a capacity and ware-
house management plan that satisfies the expected market
demand with the lowest possible cost. Mendes, Gonalves, and
Resende (2009) integrate a genetic algorithm with heuristic prior-
ity rules to solve resource constrained project scheduling prob-
lems. Guo, Wong, Leung, Fan, and Chan (2008) propose a genetic
algorithm for solving the order scheduling with multiple con-
straints for maximizing the total satisfaction level of all the orders
while minimizing their total throughput time. Recently, real-time
scheduling strategies and methods are investigated to facilitate
production management. Buyurgan and Saygin (2008) present a
framework that employs the analytical hierarchy process (AHP)
in advanced manufacturing systems for real-time scheduling and
part routing. Poon, Choy, Chan, and Lau (2011) describe a real-time
production operations decision support system (RPODS) is pro-
posed for solving stochastic production material demand prob-
lems. Cho and Prabhu (2007) use a continuous control-theoretic
approach for distributed production scheduling at the shop floor
and machine capacity control at the CNC level. Wang,
Keshavarzmanesh, and Feng (2011) introduce an event-driven
and function-block-enabled decision-making framework to
increase the adaptability and autonomy of job-shop assembly pro-
cess planning and control. Hung, Huang, and Yeh (2014) design a
real-time capacity requirement planning method for make-to-
order manufacturing with variable time-window orders.
In the past ten years, rapid developments in wireless sensors,
communication and information network technologies (e.g. radio
frequency identification – RFID or Auto-ID, Bluetooth, Wi-Fi,
GSM, and infrared) have nurtured the emergence of ubiquitous
manufacturing (Chan & Zhang, 2002) as core Advanced Manufac-
turing Technology (AMT) in next-generation manufacturing sys-
tems (NGMS). A UM system is based on wireless sensor network
that facilitates the automatic collection and real-time processing
of field data in manufacturing processes. In this way, the error-
prone, tedious manual data collection activities are reduced or
 Y. Zhang et al. / Computers & Industrial Engineering 76 (2014) 89–97
even eliminated (Jun, Shin, Kim, Kiritsis, & Xirouchakis, 2009). UM
provides a networked manufacturing environment free from
excessive and difficult wiring efforts in manufacturing workshops
(Jones, 1999). Real-time visibility and interoperability have been
considered core characteristics (Huang, Zhang, & Jiang, 2007;
Huang, Zhang, & Jiang, 2008) that close the loop of production
planning and control for adaptive decision making. By taking
advantage of data capacity stored in an RFID tag, Qiu (2007) pro-
pose a RFID-enabled framework in support of manufacturing infor-
mation integration. A new paradigm (Suh, Shin, Yoon, & Um, 2008),
called UbiDM: Design and Manufacture via Ubiquitous Computing
Technology, has been proposed for the design and manufacturing
of a product by using ubiquitous computing technology. The
importance of the UM has also been widely identified for strategic
research and development in industrialized European Union, North
Americas, and Japan where manufacturing is widely considered as
one of the major means of creating the national wealth. In a ubiq-
uitous manufacturing environment, management and control facil-
ities of shopfloor are required to implement real-time traceability,
visibility and interoperability in improving the performance of
shop-floor planning, execution and control by using workflow
management architecture (Zhang, Huang, Qu, & Ho, 2010) and
RFID-enabled smart gateway (Zhang, Huang, Qu, & Ho, 2011;
Zhang, Huang, Qu, Ho, & Sun, 2011). Besides, the facilities must
be able to effectively deal with the complex manufacturing infor-
mation following the standard schemas and transmit it in time
between workstations, shop floors and enterprise. Zhong, Dai, Qu,
Hu, and Huang (2013) present an RFID-enabled real-time manufac-
turing execution system, where RFID devices are deployed system-
atically on the shop-floor to track and trace manufacturing objects
and collect real-time production data.
3. Architecture of multi-agent based real-time shopfloor
scheduling
The overall architecture of multi-agent based real-time shop-
floor scheduling is shown in Fig. 1. It aims to implement real-time
scheduling for a ubiquitous shop-floor environment. Through auto-
ID technologies, the dynamic manufacturing information could be
captured. Then, at process planning stage, the tasks could be well
Fig. 1. Architecture of multi-agent based real-time shop-floor scheduling.
91
assigned according to the real-time status and capacity of each
machine. It will provide accurate information for production
scheduling. During execution stage, the re-scheduling can be
invoked based on the real-time manufacturing data.
Four types of agents are designed in this research to fulfil real-
time shopfloor scheduling. They are briefly described as follows:
(1) Machine Agent (MA)
It is responsible for capturing the real-time manufacturing
data by equipping auto-ID devices and processing the cap-
tured data as meaningful manufacturing information. Then,
the real-time status of the machine side can be timely
sensed by the corresponding agents.
(2) Capability Evaluation Agent (CEA)
It is used to evaluate the capability of the machines. Based
on the real-time status transmitted by Machine Agent, the
process planning can assign the tasks to optimal machines.
(3) Real-time Scheduling Agent (RSA)
It provides mathematic model and intelligent algorithm to
optimally schedule or re-schedule the start time and finish
time of each process of each task according to the sensed
real-time shopfloor information.
(4) Process Monitor Agent (PMA)
It is responsible for capturing and processing the real-time
status of different manufacturing resources. During produc-
tion execution, disturbances and changes of shopfloor are
timely tracked and traced, and the loop of production plan-
ning and control for real-time shop-floor scheduling could
be achieved.
4. Multi-agent models
4.1. Machine Agent model
MA is responsible for wrapping the applications of machine side
to process the complex real-time data sensed from Auto-ID devices
such as RFID. On one hand, it is used to connect and centrally man-
age the multiple types of auto-ID devices for capturing real-time
manufacturing data according to a specific logic flow. On the other
hand, it is also used to process the captured manufacturing data to
92 Y. Zhang et al. / Computers & Industrial Engineering 76 (2014) 89–97

meaningful manufacturing information and provide real-time

application services.
Fig. 2 shows the MA model. It includes two components, namely
data capturing and application service.

(1) Data Capturing

This component aims at managing the behaviours of auto-ID
devices installed at a machine to capturing the dynamic data
of the manufacturing resources. It consists of two modules.
Definition and auto-driven module is used to wrap various
drivers of heterogeneous auto-ID devices to form a driver
library which enables the newly plugged auto-ID device to
be ‘‘Plug and Play’’. Two driven modes, standard interface
driven and the third-part driven, are designed in this mod-
ule.
Standard data capturing module is responsible for wrapping Fig. 3. Capability Evaluation Agent model.
heterogeneous auto-ID devices into standard methods so
that their perception functions can be easily invoked under
a uniform model. Two types of standard methods, namely
real-time machine workloads and shopfloor dynamics, process
‘‘readingData (Parameter [1], Parameter [i])’’ and ‘‘writing-
plans may become suboptimal or even invalid at the time of
Data (Parameter [1], Parameter [i])’’, are involved in this
execution.
module.
As shown in Fig. 3, the capability ratio is used in CEA to evaluate
(2) Application Services
the capability of each machine and chooses an optimal one for each
This component aims to provide value-added information
process [i]. In bid stage, for each process, the potential machine
based on the captured manufacturing data by auto-ID
group with the corresponding capability will be selected at first,
devices. It also consists of two modules.
denoted as g. Then, the relevant MAs will bid this task according
Reasoning module is designed to enhance the intelligence of
to their real-time status e.g. current used capacity to CEA. Finally,
the MA. It will make MA know which type of manufacturing
the CEA will calculate the available capability of these MAs and
resource is coming or leaving the machine. Rule-based
then evaluate the optimal one based on the following objective
methods are adopted to accelerate MA to make decision
function. The ACRi is available capability ratio of machine ‘i’.
based on real-time manufacturing environment and produc-
tion logics.
MaxfACRi ji 2 g g ð1Þ
Real-time information processing module is used to deal
with the various real-time data captured by auto-ID devices
installed at the machine side. Contrast to reason module, it where ACRi = ACi/TCi  100%. ACi is the available capability of
focus on how to form more meaningful real-time manufac- machine ‘i’, it is dynamically changed with the changed queue of
turing information. For example, the ‘getQueue()’ will return the machine ‘i’. TCi is the total capability of machine ‘i’, it is a con-
detailed real-time information such as the order of the tasks, stant that represents a time period.
real-time progress of the queue etc. based on the real-time
data.
4.3. Real-time Scheduling Agent model
4.2. Capability Evaluation Agent model
RSA is designed to implement real-time scheduling. Its overall
Based on the real-time manufacturing information of MA, CEA is model is shown in Fig. 4. Its inputs include the initial information
used to optimally assign the processes of tasks to the involved from CEA and the real-time manufacturing execution information
machines. Its model is shown in Fig. 3. For each task, it consists from PMA. Its outputs are the task queues of all the machines that
of ‘n’ processes. For each process [i], according to its process plan- includes a series of {i, j, k, ST, FT}. Here, {i, j, k, ST, FT} represents the
ning, CEA will find an optimal machine from the potential machine process ‘j’ of task ‘i’ is assigned to machine ‘k’, it will be started at
group based on a bid competitive mechanism. Without considering ‘ST’ and finished at ‘FT’.
Three modules, namely mathematic module, solving module
and re-scheduling module are involved in RSA.

Fig. 2. Machine Agent model. Fig. 4. Real-Time Scheduling Agent model.

 Y. Zhang et al. / Computers & Industrial Engineering 76 (2014) 89–97
(1) Problem Formulation
Before given the mathematic formulation, the corresponding
notations are defined at first. The details can be seen in Table 1.
Based on the notations, the established mathematic model is
described as following. Because the set-up time can be
regarded as a fixed value in general manufacturing process,
it should be stated that the set-up time has been simply con-
sidered in the process time in this case.
Objective function:
MinfMax½STðT i ; TP Ni ; M k Þ þ PTðT i ; TPNi ; M k Þg
Subject to:
STðT i ; TP jþ1 ; M a Þ  STðT i ; TPj ; M b Þ P PTðT i ; TP j ; M b Þ
STðT x ; TP c ; M k Þ  STðT y ; TP d ; M k Þ P PTðT y ; TPd ; M k Þ or
STðT y ; TP d ; M k Þ  STðT x ; TP c ; M k Þ P PTðT x ; TPc ; M k Þ
di  STðT i ; TP j ; M k Þ þ PTðT i ; TP j ; M k Þ P 0
i; x; y 2 ½1; n; j 2 ½1; Ni ; c 2 ½1; Nx ; d 2 ½1; Ny ; k; a; b 2 ½1; m
Eq. (2) is the objective function. Its value is to take the maximal
value of the finishing time of the last process of all the manufactur-
ing tasks. This value is changed with the different scheduling
results. So, by minimize this value, the optimal scheduler could
be obtained. Eq. (3)–(5) are constraints. They represent processes
sequence constraint, resource constraint and delivery constraint
respectively.
(2) GA-based solving module
According to the objective function and constraints, this
module is responsible for calculating the optimal solution
by adopting intelligent algorithm. Considering Genetic Algo-
rithm (GA) has been widely studied, experimented and
applied in many fields in manufacturing fields. It is used in
RSA to solve the established scheduling problem. The GA-
based solving method includes gene and chromosome
design, decoding schema and operations design. The details
of the GA-based solving method are described in Section 5.
(3) Re-scheduling module
As described in Section 1, real-time scheduling plays an
important role in current enterprise management. For each
scheduler, during the execution stage, the PMA will feedback
the real-time production information. For quantitatively cal-
culating the total deviation ratio of the machines at time (t),
the following formula is designed in re-scheduling module.
X
m
w1  AT t ðjÞ þ w2  LT t ðjÞ þ w3  ET t ðiÞ
DðtÞ ¼  100%
i¼1 DT t ðiÞ
where w1, w2 and w3 are weights, generally, w3 is equal to 1, w1 is
less than 1, and w2 is greater than 1; ATt(j) is the ahead time of the
Table 1
Notations.
Notations Description
M = {m1, m2, . . ., mm} Set of machines
T = {t1, t2, . . ., tn} Set of tasks
Ni The number of processes of task ‘i’
TP i ¼ ftp1 ; tp2 ; . . . ; tpNi g Set of processes of task ‘i’
(Ti, TPj, Mk) Represents the process ‘j’ of task ‘i’ will be produced at machine ‘k’
ST(Ti, Pj, Mk) Starting time of (Ti, TPj, Mk)
PT(Ti, Pj, Mk) Process time of (Ti, TPj, Mk) at machine ‘k’, the set-up time of (Ti, TPj, Mk) is simply considered in the process time
D = {d1, d2, . . ., dn} Set of delivery time of tasks
93
process (j) assigned to machine (i) at time (t) contrast to the last
scheduler; LTt(j) is the lag time of the process (j) assigned to
machine (i) at time (t) contrast to the last scheduler; ETt(i) is the
exception time of the machine (i), e.g., the planned maintenance
time; DTt(i) is the duration time of machine (i). In the last scheduler,
let T be the finished time of the last process assigned to machine (i),
and the DTt(i) can be calculated by subtracting current time (t) from
T. Then, the total deviation ratio of the machines between time (t)
and the last scheduler, D(t), can be calculated by Eq. (6).
Based on D(t), the re-scheduling strategy can be put into effect.
Two strategies, namely local re-scheduling and overall re-schedul-
ð2Þ
ing, are designed in this module. If DðtÞ 6 k, the local re-scheduling
strategy is started. It is used to schedule a small part of the tasks
while other tasks’ scheduler is not changed. It deals with the
ð3Þ
exceptions such as the queue exception of a machine or temporary
tasks. If DðtÞ > k, the overall re-scheduling strategy is started. It is
used to schedule all the tasks from the exception time (t) by using
the algorithm designed in GA-based Solving module. Here, k is a
ð4Þ
constant and is generally given 5%. However, it can be changed
according to different users.
ð5Þ
4.4. Process Monitor Agent model
The PMA acts as a sandwich and plays an important role to cap-
ture the real-time information of MAs and send them to RSA. As
seen in Fig. 5, the work logic of PMA includes three stages.
At the beginning, the PMA will invoke the data source service to
get the necessary information relevant to the production order
such as the product BOM, schedule information. from the up-level
EISs (Enterprise Information Systems). Then, based on the gotten
manufacturing information and the information schema (wipML)
of WIP, a new WIP instance is created, which includes the manu-
facturing BOM (Bill of materials) information. For each node of
the manufacturing BOM, its dynamic information nodes can be
captured by the distributed MAs. And the binding model is used
to build up the bind relationship between the dynamic information
nodes and the corresponding MAs. During execution, the huge
manufacturing information captured by each MA will be processed
by critical event structure according to the RSA’s request. Two
main components are involved in the designed PMA to fulfil this
purpose, they are:
(1) Data source service
It provides data acquisition, processing and updating ser-
vices for sharing and integrating information between man-
ufacturing execution level and EISs. Due to the difficulties of
information sharing and integration among heterogeneous
EISs, B2MML (Business to Manufacturing Markup Language)
standards are adopted in this component to provide stan-
dard schemas for manufacturing elements.
ð6Þ The inputs of this component are the parameters of the data
source of the EISs which users want to acquire or update
information from or to, while the outputs are the standard
information based on B2MML schemas.
94 Y. Zhang et al. / Computers & Industrial Engineering 76 (2014) 89–97
Fig. 5. Process Monitor Agent model.
(2) WIP tracking and tracing
It is responsible for configuring the distributed MAs accord-
ing to specific logical relationship to get real-time informa-
tion of WIP in the entire shopfloor.
The critical event structure is used to obtain more meaning-
ful and actionable information from large amount of low
level events and to control the event-driven information sys-
tems. It establishes an aggregation of series of the events
from auto-ID devices to form high level events. Then, based
on the timely information stored in repository, supervisors
can monitor and control the production process of the over-
all shopfloor.
5. GA-based solving method in RSA
5.1. Gene and chromosome
An integer based method is used to form the genes and chromo-
somes. Each gene consists of an integers i (1 6 i 6 n), here n is the
sum of the tasks. Based on the genes, the chromosome is defined as
a queue that consists of different genes and char ‘-’. The length of
the chromosome is the sum of the genes, which is decided by the
P 0-0-1-2-3) will be replaced by the crossover-section of P2,
sum of the processes of all the tasks, namely n1 N i .
Based on the gene and chromosome, the decoding scheme is
also designed. It aims to interpret the meaning of the chromosome.
The char ‘-‘ appeared in chromosome is to connect the genes, and
the integer ‘i’ appeared in chromosome presents the task (i). The
appeared order of integer ‘i’ presents the processing order of the
task (i). For example, the chromosome ‘1-2-1-2-2-1’ presents a
processing order of two tasks with three processes.
5.2. Fitness function
Fitness function is used to calculate the value of each individual
in order to determine the performance of chromosomes of the
entire population. In this paper, the objective function in Section
4.3 is used to be the fitness function to evaluate the chromosome.
5.3. Evolution
Three operators namely selection, crossover, and mutation are
included in GA. Generally, selection operator is simple and has
good generalization contrast to the others. Therefore, in this paper,
only crossover and mutation operators are designed, they are
described as follows:
Crossover operator is used to introduce offspring by recombin-
ing parents. In this case, we adopt a multi-point crossover operator
to permute the job order as well as keep the legality of the gener-
ated chromosome. The proposed crossover operator consists of five
main steps.
 Step 1: Choose two parents, e.g., P1: (1-2-1-3-2-3); P2: (2-3-1-
2-3-1).
 Step 2: Randomly get two crossover points to form two cross-
over-sections. e.g., if 2 and 3 are the points. The formed two
crossover-sections are (2-1) in P1 and (3-1) in P2.
 Step 3: For each parent, use symbol ‘0’ to replace the genes in
the crossover-section of another parent. e.g., for P1, use symbol
‘0’ to replace the genes in the crossover-section of P2, namely
(3-1). Then, get P10 : (0-2-1-0-2-3).
 Step 4: Make the symbols ‘0’ move from the extremities to the
centre until they reach the cross-section. e.g., after this step, P10
(0-2-1-0-2-3) will be (2-0-0-1-2-3).
 Step 5: Substitute the symbols ‘0’ with the crossover-section of
another parent to form offspring. e.g., the symbols ‘0’ of P10 (2-
namely (3-1). Then, the offspring (2-3-1-1-2-3) is generated.
Mutation operator serves to maintain diversity in population.
Mutation operator just works on a single chromosome and gener-
ates offspring by altering one or more genes. In comparison with
the general mutation operator, a new mutation operator is designed
to avoid the loss of good solutions. An evaluation function is intro-
duced to evaluate each gene of the chromosome, and two positions
of the inferior genes are chosen randomly as the mutation positions.
Let Pi is the performance of gene (i). It depends on the following
function. Where j denotes the machine ID, k means the total mem-
ber of genes assigned to machine (j), St ji denotes the start time of
gene (i) related process at machine (j), Ft ji denotes the finished time
of gene (i) at machine (j), St jiþ1 and Ft ji1 denote the start time of the
backward and the finished time of forward process of gene (i)
related process at machine (j). Obviously, the lower value of Pi,
the higher performance of gene (i).
Pi ¼ ðSt ji  Ft ji1 Þ þ ðStjiþ1  Ftji Þ=Ft jk  St j0 ð7Þ
The designed mutation operator consists of the following three
main steps.
 Y. Zhang et al. / Computers & Industrial Engineering 76 (2014) 89–97
 Step 1: Select a parent P1. For each gene of P1, use function (7)
to evaluate its performance. And store the values in matrix
M = {(a1, p1), (ai, pi), . . ., (am, pm)}. Where ai denotes the position
of gene in P1 and pi denotes the value of the order of gene in P1.
 Step 2: Reorder the M according to the ascending value of pi, and
get M0 . In M0 , the later positions indicate the genes with bad per-
formance. Let p0 be a criterion for judging the performance of
genes. Then, if the value of pi greater than p0 , the corresponding
gene of the ai position of P1 is inferior. They denote the posi-
tions of inferior genes in P1 and be stored in IG = {a1, ak, . . ., al}.
 Step 3: Randomly select two elements ai and aj (i – j) of IG.
There values are the exchanged positions of inferior genes in
P1. Exchange the genes of the two positions (ai and aj) of P1.
And get a new chromosome. That is the generated offspring.
6. Case study
Based on the architecture and models mentioned above, this
section will illustrate the multi-agent based real-time production
scheduling through a simulation example.
The designed scheduling problem has ten tasks and ten
machines. Each task has four processes. The detailed information
of tasks is shown in Table 2. The row represents the process ID.
The column represents the task ID. The data (x, y) of the ‘j’ row
and ‘i’ column means the process ‘j’ of task ‘i’ must be manufac-
tured at any machine of the group ‘x’ and the processing time is
‘y’. The information of machine groups is shown in Table 3. For
example, the (1, 50) of the first row and the second column means
the process ‘1’ of the task ‘2’ must be manufactured at any machine
of group ‘1’, its processing time is 50. According to Table 3, the
group ‘1’ includes machine (1), (2) and (3).
The flow of multi-agent based real-time production scheduling
in this case includes three main steps, they are described as follows:
(1) At first, CEA will assign all the tasks according to the real-
time status of MAs. For each process, the relevant MAs will
bid it according to their machining capability. Then, the
CEA will calculate and evaluate the utilization ratio by func-
tion (1) based on the dynamical capacity assigned to each
relevant MA. And the MA with minimum utilization ratio
in the relevant MA group will obtain this task. These steps
are repeated until all the processes are assigned to a certain
MA. Table 4 shows the process assignment result of the tasks
and machines designed in Table 2. In Table 4, the data (x, y)
of the ‘j’ row and ‘i’ column means the process ‘j’ of task ‘i’ is
assigned to the machine ‘x’ and the processing time is ‘y’.
(2) After all the process of all the tasks optimally assigned to
MAs, RSA is ready for scheduling the tasks using GA algo-
rithm designed in Section 4.3. Fig. 6 shows the scheduling
result of the data in Table 3. Fig. 6(a) is the Gantt chart of
the optimal schedule. Fig. 6(b) is the generations and the fit-
ness curve of the designed GA.
(3) During the production execution, the real-time manufactur-
ing information is sensed by PMA and as inputs to RSA. If the
exceptions occur, the re-scheduling module will generate a
new schedule according to the changed manufacturing
environment.
Table 2
The detailed information of ten tasks.
Processes Tasks
1 2 3 4
1 1, 46 1, 50 1, 23 1, 28
2 2, 21 2, 18 3, 30 3, 45
3 3, 28 4, 33 2, 35 4, 13
4 4, 12 3, 15 4, 28 2, 32
95
Table 3
The information of machine groups.
Groups Machines
1 (1, 2, 3)
2 (4, 5)
3 (6, 7, 8)
4 (9, 10)
To demonstrate the re-scheduling, two kinds of random excep-
tions are added. In the first kind of exception, the recovery time is
unknown. In the second kind of exception, the recovery time is an
exact time window. As seen Fig. 7(a), the first exceptions of
machine (1) and machine (6) are occurred at time t (t = 40), these
exceptions are captured by the PMA and transmitted to RAS. As
seen Fig. 7(b), two kinds of exceptions of machine (1), machine
(6) and machine (7) are occurred at time t (t = 40), these exceptions
are also captured by the PMA and transmitted to RAS. For the first
kind of exceptions, the MAs of machines, namely MA (1) and (6) in
Fig. 7(a) will lose the capability to bid any tasks. For the second
kind of exceptions, the recovery time window is considered as a
load to the MAs of machines namely MA (6) and (7) in Fig. 7(b).
Then, the re-schedule is triggered and generate a new schedule
shown in Fig. 7 by repeat step 1 and step 2.
To estimate the approximate time taken in the proposed real-
time scheduling model, the following simulations are designed
based on the above case. In the local re-scheduling strategy, 8 ran-
dom exceptions are added at the time (t = 160, t = 140, t = 120,
t = 100, t = 80, t = 60, t = 40, t = 20) in any of the job (i), it will be
quickly resolved because only a few processes are involved. In this
case, it takes less than 14 s averagely. For the overall re-scheduling
strategy, the solving time depends on the production execution
point during the whole production. In this case, the exception
points are respectively added at the time (t = 160, t = 140, t = 120,
t = 100, t = 80, t = 60, t = 40, t = 20), the solving time range is from
40 s to 125 s. The details can be seen in Table 5. It should be stated
that the designed GA is run in a Hewlett–Packard server with 2
CPUs and 8G memory.
In the proposed multi-agent architecture, the PMA can identify
the real-time exceptions during the manufacturing execution stage
according to the each status of MA. Then, the RSA is triggered to
generate a new schedule to reduce the influence of the exceptions.
Therefore, the productivity is improved because of the rapid iden-
tification and responding of the production exceptions. The soft-
ware architecture of the multi-agent based real-time production
scheduling system is based on a SOA (Service Oriented Architec-
ture) framework. So, from the software, each type of agent is
deployed as a web service at the multi-agent based real-time pro-
duction scheduling system. The information flow among the multi
agents can be easily reconfigured. For different shopfloor and dif-
ferent production orders, although the processes maybe have big-
ger differences, the agents can be easily reconfigured according to
the logical and sequence relationships. Then, the proposed multi-
agent based real-time production scheduling method is flexible
for different shopfloors and products.
5 6 7 8 9 10
1, 35 1, 13 2, 24 2, 26 2, 31 2, 22
4, 21 4, 42 1, 19 3, 34 3, 23 4, 30
2, 27 3, 44 3, 37 1, 40 4, 49 1, 40
3, 46 2, 26 4, 45 4, 25 1, 19 3, 18
96 Y. Zhang et al. / Computers & Industrial Engineering 76 (2014) 89–97

Table 4
Processes assignment result according to CEA.

Processes Tasks
1 2 3 4 5 6 7 8 9 10
1 1, 46 2, 50 3, 23 1, 28 3, 35 2, 13 4, 24 5, 26 4, 31 5, 22
2 5, 21 4, 18 6, 30 7, 45 9, 21 10, 42 3, 19 8, 34 6, 23 9, 30
3 6, 28 10, 33 5, 35 10, 13 4, 27 8, 44 7, 37 1, 40 9, 49 2, 40
4 10, 12 7, 15 9, 28 4, 32 8, 46 5, 26 10, 45 9, 25 3, 19 6, 18

Fig. 6. Schedule result of Table 4.

Fig. 7. Re-schedule results of two types of exceptions.

Table 5
The approximate time taken in the proposed real-time scheduling model.

Re-scheduling Solving time at Solving time at Solving time at Solving time at Solving time at Solving time at Solving time at Solving time at
strategy t = 160 (S) t = 140 (S) t = 120 (S) t = 100 (S) t = 80 (S) t = 60 (S) t = 40 (S) t = 20 (S)
Local re- 10 12 13 15 15 13 16 18
scheduling
Overall re- 41 45 49 61 79 97 113 125
scheduling

7. Conclusions This paper has proposed a referenced multi-agent based real-

Ubiquitous manufacturing is emerging as an advanced manu-
facturing technology (AMT). It relies substantially on wireless
auto-ID sensors and wireless information networks for the collec-
tion and synchronization of real-time field data from manufactur-
ing shop floors. It enables the shop-floor management to realize
real-time production scheduling.
time scheduling for ubiquitous shop-floor environment. The con-
tributions are summarized as follows.
(1) A Machine Agent is designed to collect and process of real-
time shopfloor data captured by auto-ID devices such as RFID.
These auto-ID devices are deployed at machine side to form
value-adding points in a ubiquitous shopfloor environment.
 Y. Zhang et al. / Computers & Industrial Engineering 76 (2014) 89–97
(2) A Capability Evaluation Agent is designed to optimally
assign the tasks to the involved machines at the process
planning stage based on the real-time utilization ration of
each machine.
(3) A Real-Time Scheduling Agent is designed for manufacturing
tasks scheduling/re-scheduling strategy and methods
according to their real-time feedback.
(4) A Process Monitor Agent is established for tracking and trac-
ing the manufacturing execution based on a critical event
structure.
(5) A GA-based solving method is designed in RSA. The new
crossover and mutation operations are presented to improve
its efficiency.
The current work will be further extended in our future
research from the following aspects. Firstly, the proposed multi-
agent based real-time shop-floor scheduling architecture and its
models only provide a reference for dynamical monitoring and
controlling in ubiquitous manufacturing environment, and a great
effort should be needed to support its application in real-life enter-
prises. Secondly, the proposed multi-agent based scheduling
model should be further extended to real-time internal logistics
planning and scheduling, etc.
Acknowledgments
Authors would like to acknowledge financial supports of
National Science Foundation of China (51175435), the program
for New Century Excellent Talents in University (NCET-12-0463),
and the Fundamental Research Funds for the Central Universities
(3102014JCS05005), and HKSAR RGC GRF.
References
Aghezzaf, E. H. (2007). Production planning and warehouse management in supply
networks with inter-facility mold transfers. European Journal of Operational
Research, 182(3), 1122–1139.
Bastos, R. M., Oliveira, F. M., & Oliveira, J. P. (2005). Autonomic computing approach
for resource allocation. Expert Systems with Applications, 28(1), 9–19.
Buyurgan, N., & Saygin, C. (2008). Application of the analytical hierarchy process for
real-time scheduling and part routing in advanced manufacturing systems.
Journal of Manufacturing Systems, 27(3), 101–110.
Chan, F. T. S., Swarnkar, R., & Tiwari, M. K. (2007). Infrastructure for co-ordination of
multi-agents in a network-based manufacturing system. International Journal of
Advanced Manufacturing Technology, 31(9–10), 1028–1033.
Chan, F. T. S., & Zhang, J. (2002). A multi-agent-based agile shop floor control
system. International Journal of Advanced Manufacturing Technology, 19(10),
764–774.
Cho, S., & Prabhu, V. V. (2007). Distributed adaptive control of production
scheduling and machine capacity. Journal of Manufacturing Systems, 26(2),
65–74.
Guo, Z. X., Wong, W. K., Leung, S. Y. S., Fan, J. T., & Chan, S. F. (2008). Genetic
optimization of order scheduling with multiple uncertainties. Expert Systems
with Applications, 35(4), 1788–1801.
Huang, G. Q., Wright, P., & Newman, S. (2009). Wireless manufacturing: A literature
review, recent developments, and case studies. International Journal of Computer
Integrated Manufacturing, 22(7), 1–16.
Huang, GQ., Zhang, YF., & Jiang, PY. (2007). RFID-based wireless manufacturing for
walking-worker assembly islands with fixed-position layouts. International
Journal of Robotics and Computer Integrated Manufacturing, 23(4), 469–477.
Huang, G. Q., Zhang, Y. F., & Jiang, P. Y. (2008). RFID-based wireless manufacturing
for real-time management of job shop WIP inventories. International Journal of
Advanced Manufacturing Technology, 36(7–8), 469–477.
Hung, Y. F., Huang, C. C., & Yeh, Y. (2014). Real-time capacity requirement planning
for make-to-order manufacturing with variable time-window orders. Computers
& Industrial Engineering, 64(2), 641–652.
Jia, H. Z., Ong, S. K., Fuh, J. Y. H., Zhang, Y. F., & Nee, A. Y. C. (2004). An adaptive
upgradable agent-based system for collaborative product design and
manufacture. Robotics and Computer-Integrated Manufacturing, 20(2), 79–90.
97
Jiao, J. R., You, X., & Kumar, A. (2006). An agent-based framework for collaborative
negotiation in the global manufacturing supply chain network. Robotics and
Computer Integrated Manufacturing, 22(3), 239–255.
Jones, L. (1999). Working without wires. Industrial Distribution, 88(8), M6–M9.
Jun, H., Shin, J., Kim, Y., Kiritsis, D., & Xirouchakis, P. (2009). A framework for RFID
applications in product lifecycle management. International Journal of Computer
Integrated Manufacturing, 22(7), 595–615.
Krothapalli, N., & Deshmukh, A. (1999). Design of negotiation protocols for multi-
agent manufacturing systems. International Journal of Production Research, 37(7),
1601–1624.
Lee, W. B., & Lau, H. C. W. (1999). Multi-agent modeling of dispersed manufacturing
networks. Expert Systems with Applications, 16(3), 297–306.
Lim, M. K., Tan, K., & Leung, C. H. (2013). Using a multi-agent system to optimise
resource utilisation in multi-site manufacturing facilities. International Journal
of Production Research, 51(9), 2620–2638.
Macchiaroli, R., & Riemma, S. (2002). A negotiation scheme for autonomous agents
in job shop scheduling. International Journal of Computer Integrated
Manufacturing, 15(3), 222–232.
Maturana, F. P., Tichy, P., Slechta, P., Discenzo, F., Staron, R. J., & Hall, K. (2004).
Distributed multi-agent architecture for automation systems. Expert Systems
with Applications, 26(1), 49–56.
Mendes, J., Gonalves, J., & Resende, M. (2009). A random key based genetic
algorithm for the resource constrained project scheduling problem. Computers
and Operations Research, 36(1), 92–109.
Monostori, L., Váncza, J., & Kumara, S. R. T. (2006). Agent-based systems for
manufacturing. CIRP Annals – Manufacturing Technology, 55(2), 697–720.
Papakostas, N., Mourtzis, D., Bechrakis, K., Chryssolouris, G., & Doukas, D. (1999). A
flexible agent based framework for manufacturing decision making. In
Proceedings of the 9th international conference on flexible automation and
intelligent manufacturing, Tilburg, Netherlands (pp. 789–800).
Poon, T. C., Choy, K. L., Chan, F. T. S., & Lau, H. C. W. (2011). A real-time production
operations decision support system for solving stochastic production material
demand problems. Expert Systems with Applications, 38(5), 4829–4838.
Qiu, R. G. (2007). RFID-enabled automation in support of factory integration.
Robotics and Computer-Integrated Manufacturing, 23(6), 677–683.
Shen, W., Wang, L., & Hao, Q. (2006). Agent-based distributed manufacturing
process planning and scheduling: A state-of-the-art survey. IEEE Transactions on
Systems, Man, and Cybernetics – Part C: Applications and Reviewers, 36(4),
563–577.
Sikora, R., & Shaw, M. J. (1998). A multi-agent framework for the coordination and
integration of information systems. Management Science, 44(11), 65–78.
Suh, S. H., Shin, S. J., Yoon, J. S., & Um, J. M. (2008). UbiDM: A new paradigm for
product design and manufacturing via ubiquitous computing technology.
International Journal of Computer Integrated Manufacturing, 21(5), 540–549.
Tao, F., Cheng, Y., Xu, L. D., Zhang, L., & Li, B. H. (2014). CCIoT-CMfg: Cloud
computing and internet of things-based cloud manufacturing service system.
IEEE Transactions on Industrial Informatics, 10(2), 1435–1442.
Wang, L. H., Adamson, G., Holm, M., & Moore, P. (2012). A review of function blocks
for process planning and control of manufacturing equipment. Journal of
Manufacturing Systems, 31(3), 269–279.
Wang, L. H., Keshavarzmanesh, S., & Feng, H. Y. (2011). A function block based
approach for increasing adaptability of assembly planning and control.
International Journal of Production Research, 49(16), 4903–4924.
Wang, G., Wong, T. N., & Wang (2013). An ontology based approach to organize
multi-agent assisted supply chain negotiations. Computers & Industrial
Engineering, 65, 2–15.
Wong, T. N., Leung, C. W., Mak, K. L., & Fung, R. Y. K. (2006). Dynamic shopfloor
scheduling in multi-agent manufacturing systems. Expert Systems with
Applications, 31(3), 486–494.
Wu, D. J. (2001). Software agents for knowledge management: Coordination in
multi-agent supply chains and auctions. Expert Systems with Applications, 20(1),
51–64.
Zhang, Y. F., Huang, G. Q., Qu, T., & Ho, O. (2010). Agent-based workflow
management for RFID-enabled real-time reconfigurable manufacturing.
International Journal of Computer Integrated Manufacturing, 23(2), 101–112.
Zhang, Y. F., Huang, G. Q., Qu, T., Ho, O., & Sun, S. D. (2011b). Agent-based smart
objects management system for ubiquitous manufacturing. International Journal
of Robotics and Computer Integrated Manufacturing, 27(3), 538–549.
Zhang, Y. F., Qu, T., Ho, O., & Huang, G. Q. (2011a). Agent-based smart gateway for
RFID-enabled real-time wireless manufacturing. International Journal of
Production Research, 49(5), 1337–1352.
Zhang, Y. F., Zhang, G., Wang, J. Q., Sun, S. D., Si, S. B., & Yang, T. (2014). Real-time
information capturing and integration framework of the internet of
manufacturing things. International Journal of Computer Integrated
Manufacturing. http://dx.doi.org/10.1080/0951192X.2014.900874.
Zhong, R. Y., Dai, Q. Y., Qu, T., Hu, G. J., & Huang, G. Q. (2013). RFID-enabled real-time
manufacturing execution system for mass-customization production.
International Journal of Robotics and Computer Integrated Manufacturing, 29,
283–292.