IEEE TRANSACTIONS ON SMART GRID, VOL. 9, NO.

2, MARCH 2018 1323

A Multiagent-Based Hierarchical Energy

Management Strategy for Multi-Microgrids
Considering Adjustable Power and
Demand Response
Van-Hai Bui, Student Member, IEEE, Akhtar Hussain, Student Member, IEEE,
and Hak-Man Kim, Senior Member, IEEE

Abstract—Conventionally, community energy management Constants

system (CEMS) is provided with the information of surplus and
shortage amounts only at each time interval. This limited infor-
CiCDG Production cost of unit i.
Pmin max
mation may lead to an increase in the operational cost of the i , Pi Min/max production limits of unit i.
multimicrogrid (MMG) systems. This paper suggests informing CiSU Start-up cost of unit i.
Buy
the CEMS about the adjustable power also, in addition to the PRt , PRSellt Buying/selling price at t.
surplus and shortage information. This additional information
will result in a variety of options for the CEMS to fulfill the load PPV
t , PWT
t Forecasted output of PV cell & wind
demands of its network. CEMS will choose among various avail- turbine.
able options, which include trading with the power grid, buying PL_fix
t , PL_con
t Fixed and controllable load at t.
from a controllable distributed generation plant, buying from a vt,t Penalty of shifting load from t to t’.
community battery energy storage system (CBESS), or control- IFtmax , OFtmax Maximum inflow and outflow of load
ling the adjustable power: increasing or decreasing the generation
of controllable units. CBESS can either be controlled by CEMS to/from interval t.
or can act as an autonomous entity. The effects of both the oper- CtCDGP Average generation cost of CDGP at t.
ational options have been analyzed and economically efficient PRRec
t , PRt
Send Receiving/sending price at time t by MGs.
mode is suggested for MMG systems. Demand response (DR) is
CtCB+ , CtCB− Charging/discharging cost of CBESS.
also considered in the proposed model. The incorporation of DR
will ensure the supply reliability of the MMG system in addition LB+ , LB− Losses for charging/discharging of BESS.
−
to the reduction in operational cost. In contrast to the conven- LCB+ , LCB Losses for charging/discharging
tional single or two-step multimicrogrid optimization algorithms, of CBESS.
a multistep hierarchical optimization algorithm based on a mul- Cap Cap
PB , PCB Capacity of BESS and CBESS.
tiagent system is proposed in this paper. Easy to implement and
computationally inexpensive mixed integer linear programming ηBBTB , ηCB
BTB Efficiency of back-to-back converters
models are developed for each step. connected with BESS and CBESS.
Index Terms—Adjustable power, demand response, energy PBTB BTB
B , PCB Capacity of back-to-back converters con-
management system, mixed integer linear programming, mul- nected with BESS and CBESS.
tiagent system, multi-microgrids. δB , δCB Self-discharge rate for BESS and
CBESS.
N OMENCLATURE
Sets
Variables
T Set of time intervals.
ui,t On or off mode of unit i at t.
I Set of generations.
PCDG
i,t Generation amount of unit i at t.
K Set of microgrids.
PShort
t Shortage power at time interval t.
Manuscript received December 22, 2015; revised April 18, 2016; accepted PSur
t Surplus power at time interval t.
June 22, 2016. Date of publication June 28, 2016; date of current ver- PB+ B−
t , Pt Charging/discharging power of BESS at t.
sion February 16, 2018. This work was supported in part by the Power Pt , PCB−
CB+
t Charging/discharging power of CBESS
Generation and Electricity Delivery Core Technology Program of the
Korea Institute of Energy Technology Evaluation and Planning, and in part at t.
L_adj
by the Ministry of Trade, Industry, and Energy, Republic of Korea, under Pt Adjusted load at t.
Grant 20141020402350. Paper no. TSG-01621-2015. (Corresponding author: PSh Shifted load from time t to time t’.
Hak-Man Kim.) t,t
 
The authors are with the Department of Electrical Engineering, Incheon SOCtB , SOCtCB State of charge for BESS and CBESS at t
National University, Incheon 22012, South Korea (e-mail: hmkim@inu.ac.kr). before considering self-discharge.
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org. SOCtB , SOCtCB State of charge for BESS and CBESS at t
Digital Object Identifier 10.1109/TSG.2016.2585671 after considering self-discharge.
1949-3053 c 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
1324
Adj
Pk,i,t Adjustable power of unit i in MG k at t.
Buy
Pk,t , PSell
k,t Buying/selling power of MG k at t.
PRec Send
t , Pt Receiving/sending power at t by MGs.
CDGP
Pt Output power of CDGP at t.
min _adj
Pk,i,t Minimum adjustable power of unit i.
max _adj
Pk,i,t Maximum adjustable power of unit i.
I. I NTRODUCTION
ROWING energy demand, environmental protection,
G resiliency of the power grid in emergency situations, and
integration of distributed generations (DGs) along with energy
storage systems (ESSs) are the major considerations for the
modern power industry. Microgrids (MGs) have the potential
to play a vital role in this organic transformation of the existing
power grid to the future smart grid. MGs normally operate in
grid-connected mode to maximize the profit and have the abil-
ity to operate in islanded mode to enhance the reliability during
emergency and/or scheduled maintenance periods. Optimal
operation and planning of the future smart distribution sys-
tems is challenging due to uncertainties in generation amount
of DGs, electricity market prices, load demand, and penetra-
tion of electrical vehicles (EVs). These uncertainties can be
handled to a large extent by combining several MGs to form
a multi-microgrids (MMG) system. Various researches have
been conducted in the recent years regarding the configura-
tion and architectures of the future MMG systems. The major
considerations of an MMG system are minimization of opera-
tion cost, preservation of customer privacy, and enhancement
in grid reliability.
Energy management system (EMS) is used to optimally
schedule the power resources and ESS to fulfil the load
demands. The architecture of an EMS could be centralized,
decentralized, or hybrid depending on the considerations stated
earlier. Genetic algorithms, robust optimization, stochastic
optimization, and mixed integer linear programming (MILP)
have been widely used in the literature for optimal schedul-
ing of MMGs. Different demand response (DR) programs can
also be incorporated in MMG EMS to reshape the load profiles
of MGs.
Energy management of MGs based on a double layer coor-
dinated control in both grid-connected and islanded mode has
been proposed by [1]. Energy management of MMGs based
on a decentralized EMS has been suggested by [2] and a cen-
tralized EMS has been proposed by [3]. A robust energy
management strategy for uncertainty aware MGs has been
analyzed by [4] while, a hierarchical EMS has been proposed
by [5] for preserving the customer privacy. An optimal energy
storage strategy for grid-connected MGs has been proposed
by [6] and effect of energy storage system in operation of
islanded MGs has been analyzed by [7]. A DR model based
on real time price (RTP) has been developed by [8] and DR
has been used to mitigate the impacts of EVs on smart distri-
bution systems by [9] and [10]. Multiagent systems (MASs)
have been widely used for optimization of MMG systems.
The power engineering applications where MAS is required,
have been analyzed along with the technical issues and
IEEE TRANSACTIONS ON SMART GRID, VOL. 9, NO. 2, MARCH 2018
research challenges by [11]–[13] and a study on developing
an MAS standard for ensuring interoperability between agents
of different designers has been carried out by [14] and [15].
Distributed MAS-based scheduling for islanded MGs has
been analyzed by [16] and [17]. Management of distributed
power sources using an MAS has been discussed in [18],
distributed management of MGs by using MAS has been
proposed by [19] and [20] and MAS-based load shedding
strategy for MGs has been proposed by [21]. In order to
overcome the drawbacks of centralized and distributed EMSs,
various hybrid and hierarchical EMS systems have also been
analyzed in the literature. MAS-based hybrid EMS systems
have been proposed by [22]–[24]. Apart from distributed
and centralized energy management strategies, currently var-
ious cooperative MMG systems have also gained popularity
as in [25]–[27].
Most of the researches available in the literature for hierar-
chical management of energy in MMG systems only provide
information about the surplus and shortage amounts of energy
to the community EMS (CEMS) at each time interval. CEMS
only optimizes the surplus and shortage amount from indi-
vidual MGs. Each local EMS does not have any information
about the production cost of other MGs of the MMG network.
This approach results in unnecessary trading of electricity with
the utility grid and causes an increase in operational cost of
MMG system.
In [28], agent-based architecture has been developed for
trading and managing power considering DR programs. The
intelligent bidding strategy based on continues double auc-
tion (CDA) has been used to allow customers to participate
in DR programs. A multiagent framework for implementing
demand side management (DSM) has been suggested by [29]
to solve the problem of power imbalance by paying more
attention to DR. An agent-based EMS has been developed
for multiple MGs including DR and distributed storage (DS).
The agent-based EMS with DR & DS has been implemented to
reduce the system peak demand and minimize the cost of elec-
tricity by [30]. In [31], a two-level architecture for multiple
MGs has been presented by using MAS. The market agents
have been designed to participate in real-time bidding. The
bidding actions have been simulated by using naive auction
algorithm.
The optimization strategy proposed in this paper is more
related to the researches carried out by [19] and [28]–[31].
Due to the merits of hierarchical EMSs and potential appli-
cation of MAS in microgrid operation, all the studies includ-
ing the proposed optimization strategy have used hierarchi-
cal EMSs for modeling and MAS for realizing the opti-
mal operation of MGs. However, the model proposed in
this study is based on the concept of cooperative MMG
operation in contrast to the competitive models proposed
by [19] and [28]–[31]. The objectives of competitive and coop-
erative models are different, i.e., the objective of cooperative
MMG system is to reduce the operation cost of the entire net-
work while in competitive environment objective of each MG
is to maximize its own profit. In addition, both the models find
their applications in different environments and have their own
pros and cons [12], [25].
BUI et al.: MULTIAGENT-BASED HIERARCHICAL ENERGY MANAGEMENT STRATEGY FOR MMGs 1325

In this paper, a hierarchical EMS for optimal MMGs oper-

ation has been proposed and is based on MAS. In this way,
each component in the system has been considered as an
autonomous agent that can communicate with other agents
of the system by following a modified contract net proto-
col (MCNP). A significant characteristic of this architecture
is that it can be divided into many levels of agents for satis-
fying their goals. It makes easier to manage and operate the
system. Moreover, the burden of communication in the sys-
tem is also reduced by developing two levels of management
system, i.e., MG-EMS and CEMS. The major contributions of
this paper can be summarized as follows:
1. Concept of Adjustable Power has been introduced and
utilized for minimizing the operational cost of MMG
systems. CEMS may increase/decrease the generation
amounts of CDGs with adjustable power to reduce the
operational costs of both individual MGs and MMG
system by avoiding unnecessary trading with the util-
Fig. 1. Configuration of proposed MMG system.
ity grid.
2. Two different operation modes (subservient and
autonomy-ous) have been suggested for CBESS. After
testing both the operation modes on the proposed opti-
mization strategy, more cost-effective mode has been
suggested.
3. DR programs have been incorporated in the proposed
EMS strategy for shifting the peak load demands by
realizing a load-shifting scheme.
4. Hierarchical EMS has been used to share the compu-
tational burden on the CMES and MAS has been used
to realize the communication between different entities
in MGs and in MMG community. In addition, problem
Fig. 2. Adjustable power concept.
formulation is based on MILP. MILP problems can be
easily implemented through commercial software like
CPLEX, which guarantee global optimality.
4) Controllable Distributed Generation Plant (CDGP): It is
comprised of several CDG units. CDGP informs CEMS about
average per-unit cost at each time interval.
II. S YSTEM M ODEL 5) Power Grid: CEMS receives per-unit price signals (buy-
A. Configuration and Components of MMG System ing and selling) from the power grid. The price signals for
each time interval are predefined.
The developed system for testing the performance of the
6) CEMS: It is responsible for global optimization. After
proposed operational cost minimization algorithm is illustrated
receiving information from MG-EMSs, CBESS, CDGP, and
in Fig. 1. Three MGs namely MG1, MG2, and MG3 have
market price signals, CEMS runs global optimization for
been considered in this study. Each MG contains a battery
minimizing operational cost of MMG system.
energy storage system (BESS), controllable distributed gen-
erator (CDG), renewable distributed generation (RDG), and
electrical load. B. Concepts: Terminologies and Definitions
1) Individual Microgrid Components: CDG units could be 1) Adjustable Power: The output power generation of each
a diesel generator, a fuel cell, or any other type of control- MG is given by Fig. 2. The dotted lines represent the minimum
lable generator. Wind turbine and solar cells constitute RDGs. generation limits and dashed lines represents the maximum
BESS comprises of electrochemical cells and load consists of generation limits of MGs at a given time interval. The solid
controllable and fixed loads. line shows the load demand of respective MG. The light green
2) Microgrid EMS (MG-EMS): It is responsible for opti- color in Fig. 2 depicts the amount of electricity generated by
mization of its local components. After optimization surplus, MG after local optimization. The light blue color shows the
shortage, and adjustable power amount along with per-unit amount of electricity received from other MGs and violet color
cost are provided to the CEMS at each interval of time. shows the amount of electricity sent by mth MG of the net-
3) Community BESS (CBESS): It can operate either in a sub- work after global optimization. The downward and upward
servient mode or in an autonomous mode. CBESS sends its arrows show the amount of electricity reduced or increased,
state of charge (SOC) information to CEMS at each interval. respectively, due to consideration of adjustable power.
1326
Fig. 3. Configuration of proposed hierarchical MAS.
The power generated by CDG in MG1 is equal to the load as
shown in Fig. 2 (left pair). At the same time, another MG2 is
selling electricity to the power grid at a lower price. MG2 can
send the surplus amount to fulfill shortage amount of MG3.
The remaining surplus amount of MG2 can be sent to MG1
and MG1 can reduce its generation (generation cost of MG1 >
selling price). Similarly, shortage amount of MG3 in Fig. 2
(right pair) cannot be fulfilled only by MG2. MG3 has to
buy the remaining shortage amount from the power grid at
higher price. MG1 can increase its generation and send to
MG3. (MG1’s generation cost < buying price). The amount
of power by which an MG can increase/decrease its gener-
ation is defined as adjustable power. Similarly, CEMS can
compare the adjustable power cost with CDGP cost and can
increase/decrease the adjustable amount in a similar way.
2) CBESS Subservient Mode: Decisions about charging
and discharging of CBESS are made by CEMS. CEMS
charges/discharges CBESS for minimizing the overall MMG
system’s operational cost. CBESS only follows the commands
received from the CEMS.
3) CBESS Autonomous Mode: Decisions about charging and
discharging of CBESS are made by CBESS itself. CBESS
decides the charging/discharging for maximizing its own
profit.
C. Configuration of Multiagent System
A typical MMG system is shown in Fig. 1 where, each
MG is integrated into the AC bus through power electron-
ics interfaces. The operational strategy proposed in this paper
uses three levels for accomplishing the task of optimization.
The agents are categorized into four groups in accordance to
the nature of the task they are responsible for. The complete
architecture of proposed MAS is shown in Fig. 3.
1) Local Agents: These are the lowest level agents and
are responsible for communicating with their local MG-EMS
agents. RDG agent, CDG agent, BESS agent, and load agent
are categorized as local agents as shown in Fig. 3.
2) MG-EMS Agents: Local agents along with
MG-EMS agents from the first level of the proposed
IEEE TRANSACTIONS ON SMART GRID, VOL. 9, NO. 2, MARCH 2018
optimization strategy. MG-EMS agents are downscaled
versions of the CEMS agent. Each MG-EMS agent is only
responsible for communicating the shortage, surplus, and
adjustable power amounts to CEMS and inform the local
agents about the decisions of CEMS.
3) External Agents: Market agent, CDGP agent, and CBESS
agent are categorized as external agents and they form the sec-
ond level agents in the proposed optimization strategy. Market
agent and CDGP agent send their price signals to the CEMS
and receive the scheduling commands from CEMS. CBESS
agent either can send the price signals or can just follow CEMS
commands according to its operating mode.
4) CEMS Agent: CEMS agent is responsible for the global
optimization of MMG system. It is responsible for receiving
all the information from the first and second level agents and
is responsible for convening the decisions made by CEMS to
each of the lower level agents. CEMS agent is placed in third
level as shown in Fig. 3.
D. Algorithm for Optimal Multi-Microgrid Operation
In order to complete one round of optimization using
the proposed operation strategy, few sequential actions are
performed by agents at different levels. In step 1, hourly
day-ahead market price signals (buying and selling prices),
generation capabilities of CDG, RDG, and BESS along with
load profiles of MGs are taken as inputs. Each MG-EMS
receives this information and performs local optimization con-
sidering the DR programs. Each MG utilizes the information
provided by CEMS (market price signals) and information spe-
cific to its components (generation limits and capacities) to
decide about its participation in global optimization. If any
MG sends refusal signal to CEMS, it will not be considered
for global optimization by CEMS. All the other MGs will
send their surplus, shortage, and adjustable power amounts to
CEMS. The decision about surplus, shortage, and adjustable
power amounts will be based on the generation cost of CDG
units, market trading prices, and local load demands of MGs
as depicted by Fig. 4. At the same time, external agents will
also inform CEMS agent about their capacities and/or per-unit
prices. CBESS agent will inform about its initial SOC, capac-
ity of CBESS, and electricity trading prices (depending on
the operating mode). CDGP agent will inform about the per-
unit generation cost along with upper and lower generation
bounds to CEMS. CEMS will gather all the information from
the lower level agents and performs global optimization. In this
step, the amount of power to be traded with the utility grid,
amount of power to be traded among MGs of the network,
and adjustable amount of power to be increased/decreased
by each MG will be decided. CEMS agent will inform all
the level 2 and MG-EMS agents about their scheduling.
After receiving the global optimization results, each MG-
EMS performs its local optimization again. All the local
agents inform their respective MG-EMSs about their com-
mitment statuses and MG-EMS along with community level
agents inform CEMS about the commitment statuses. In this
way, one round of optimization is completed by using the
proposed optimization strategy. The step-by-step procedure
BUI et al.: MULTIAGENT-BASED HIERARCHICAL ENERGY MANAGEMENT STRATEGY FOR MMGs
Fig. 4. Flowchart of proposed hierarchical operation of MMG system.
Fig. 5. Interaction between different participating agents.
for carrying out one round of optimization is summarized
in Fig. 4.
E. Interaction Between Different Agents
The interaction between different developed agents in the
proposed operation strategy is shown in Fig. 5. Firstly, a mes-
sage is sent by community EMS agent to market agent to
inquire about the market price signals. The market agent sends
the day-ahead buying and selling prices for each hour of
the day to CEMS agent. The market agent sends the same
price signals to external agents. These market price signals
are accompanied by call for proposal (CFP) messages and are
destined to MG-EMSs and community level resource agents
1327
(level 2 agents). Each of the external and MG-EMS agents has
the liberty to decide about their participation in the global opti-
mization. In a similar way, each MG-EMS agent will inform its
local resources about the market price signals along with cfp
messages. The local elements of MGs also can accept/reject
the proposals received from their MG-EMSs. Based on the
proposals received from its local elements, each MG-EMS
agent will decide either to participate in global optimization
or not. All the MG-EMS agents will inform the CEMS agent
about their participation in global optimization. In the same
way, external agents will also decide and inform the CEMS
agent. CEMS can accept or reject the proposals from external
and MG-EMS agents.
1328 IEEE TRANSACTIONS ON SMART GRID, VOL. 9, NO. 2, MARCH 2018

CEMS will inform the level 2 agents (community level 0 ≤ SOCtB ≤ 1 ∀t ∈ T (9)
resource agents and MG-EMS agents) about the accep- B
SOCt−1 B
= (1 − δB ) · SOCt−1 ∀t ∈ T (10)
tance/rejection of their proposals. Respective MG-EMS agents  
will inform their local agents about the decisions of Pt ,t ≤ IFt ,
Sh max
Pt,t ≤ OFtmax ∀t ∈ T
Sh
(11)
CEMS. Local and external agents will get either their com- t ∈T t ∈T
t =t t =t
mitment scheduling or will remain idle in accordance with
the decisions of CEMS and MG-EMSs. Finally, all the local OFtmax = PtL_con ∀t ∈ T (12)
L_adj L_ fix
 
agents and external agents will inform their respective upper Pt = Pt +PtL_con + PSh

t ,t − PSh
t,t ∀t ∈ T (13)
level agents about their commitment statuses. Communication 
t ∈T 
t ∈T
between all the agents is realized through agent commu- t =t t =t

nication language (ACL) by using a modified contract net 0 if shifting is allowed

vt,t = ∀t ∈ T, t ∈ T (14)
protocol (MCNP) [34]. ∞ otherwise

III. MILP-BASED M ODEL FOR DAY-A HEAD S CHEDULING Equation (2) shows the operating bounds for ith CDG unit.
Mathematical models have been developed for each of The start-up indicator for CDG is represented by (3), which
the three steps explained in previous sections. In step 1, is max constraint. Apparently, this constraint leads to non-
local optimization is carried out by respective MG-EMSs. linear problem formulation but it can be solved by using
Each MG-EMS decides surplus/shortage and adjustable power “IloMax” operator available in CPLEX [33]. Power gener-
at this stage. Step-wise developed mathematical models are ated by photovoltaic cell, wind turbine, committed CDG units,
explained in the following sections. The proposed model is shortage power, and discharged amount should be balanced
formulated for a 24-h scheduling horizon with a time interval with load, surplus, and charging amounts at each interval and
of t, which could be any uniform interval of time. However, in is depicted by (4). BESS has been taken as a load during
the proposed day-ahead scheduling model, t has been assumed the charging mode and as a source during the discharg-
to be one hour. ing mode. Therefore, the cost of electricity is incorporated
during the charging/discharging mode. The capital cost of
BESS is an important issue in the planning phase opti-
A. Step 1: Local Optimization
mization, but in day-ahead scheduling capital cost is not
The proposed optimization model aims to minimize the total considered generally [3], [23]. Equations (5)-(10) show the
operational cost of each microgrid. Equation (1) is the objec- constraints for charging, discharging, and SOC of BESS. The
tive function and it contains CDG generation cost, startup cost charging/ discharging loss, self-discharge rate of BESS, and
for CDG, price of buying, price of selling, and penalty for BTB converter losses are included in the BESS modeling
shifting load from t to t’ interval respectively. as proposed by [35] and [36]. The SOC of BESS in the
   B
first interval (SOCt−1,t=1 ) is the initial SOC of BESS: SOC of
min CiCDG · PCDG
i,t + yi,t · CiSU
last interval of previous day. The SOC of BESS is updated in
i∈I t∈T
 Buy
  each interval as given by (8). The amount of load, which can
+ PRt · PShort
t − PRSell
t · PSur
t + vt,t · PSh
t,t be shifted from interval t to t’, is constrained by (11). The max-
t∈T t,t ∈T imum amount of shiftable load is constrained by the amount
t=t
of controllable load and is modeled by (12). The adjusted load
(1)
after applying DR (load shifting) can be computed by using
Subject to equation (13). The penalty for shifting load from interval t to t’
can be modeled by using (14).
ui,t · Pmin
i ≤ PCDG ≤ ui,t · Pmax ∀i ∈ I, t ∈ T (2) After completing local optimization by each MG-EMS at
 i,t  i
yi,t = max ui,t − ui,t−1 , 0 , ui,t ∈ {0, 1} ∀i ∈ I, t ∈ T step 1, each MG calculates surplus, shortage, and adjustable
(3) power amount according to Algorithm 1. The calculated values
 are communicated to the CEMS agent.
PPV
t + PWT
t + PCDG
i,t + PShort
t + PB−
t
i∈I
L_adj
= Pt + PSur
t + Pt
B+
∀t ∈ T (4) B. Step 2: Global Optimization
Cap   1 1 After receiving the information about surplus, shortage, and
0 ≤ PB+
t ≤ PB · 1 − SOCt−1
B
· · ∀t ∈ T
1 − LB+ ηBBTB adjustable power from each MG-EMG agent and proposals
(5) from CDGP and CBESS agents, CEMS performs global opti-
Cap   mization in this step. The objective function for step 2 will be
0≤ PB−
t ≤ PB · SOCt−1
B
· 1 − LB− · ηBBTB ∀t ∈ T (6)
different depending on the operation mode of CBESS.
0≤ t , Pt
PB− B+
≤ PBTB
B /ηB
BTB
∀t ∈ T (7)
⎛ ⎞ 1) Subservient Mode Model: As described earlier, in sub-
1 1 servient mode the charging and discharging decisions of
1 · · P B−
SOCtB = SOCt−1
B
− Cap
· ⎝ 1 − LB− ηBBTB  t ⎠ CBESS are made by CEMS and CBESS only follows the
PB −Pt · 1 − L
B+ B+
· ηB
BTB
received commands. The objective function (15) contains aver-
(8) age generation cost of CDGP, adjustable power cost from each
BUI et al.: MULTIAGENT-BASED HIERARCHICAL ENERGY MANAGEMENT STRATEGY FOR MMGs 1329

Algorithm 1: Computation of Surplus, Shortage and 0 ≤ SOCtCB ≤ 1 ∀t ∈ T (23)

Adjustable Power CB
CB
SOCt−1 = (1 − δCB ) · SOCt−1 ∀t ∈ T (24)
1 Initial values
2 Running local optimization for each microgrid Similar to the defined upper and lower limits for CDGP
3 while t < T do
in (16), adjustable power is also constrained by upper and
4 for i < I do
 CDG WT > PL_adj then lower bounds as shown in (17). The load balancing in global
5 if Pi,t + PPV
t + Pt t
i∈I optimization can be done by using (18). Equation (18) shows
 CDG L_adj
6
Sur
Pt = Pi,t + Pt + PWT
PV
t − Pt that power generated by CDGP, power bought from the power
i∈I grid, adjustable & surplus from each MG, and CBESS dis-
7 PShort
t =0 charging should be balanced with the shortage amount from
Buy
8 if PRSell
t ≤ CiCDG < PRt then   each MG, CBESS charging, and the amount of power sold to
CDG Adj
9
min
Pi − Pi,t ≤ Pi,t ≤ Pmax
i − PCDG
i,t the main grid. The constraints associated with CBESS charg-
10 else ing/discharging and SOC are modeled by using (19)-(24).
Adj
11 Pi,t = 0 Similar to BESS modeling in step 1 (local optimization),
12 end if charging and discharging losses, self-discharge rate, and BTB
13 else converter losses have been considered in CBESS modeling.
14 PSur
t =0 2) Autonomous Mode Model: In autonomous mode, charg-
L_adj 
15 PShort
t = Pt − PCDG
i,t − PPV WT
t − Pt ing and discharging of CBESS is decided by CBESS alone.
i∈I
Buy CBESS evaluates the price signals from CDGP and power grid
16 if PRSell
t ≤ CiCDG < PRt then  
Adj and decides when to charge/discharge. Therefore, the objec-
17 Pmin
i − PCDG
i,t ≤ Pi,t ≤ Pmax
i − PCDG
i,t tive function for global optimization in autonomous mode of
18 else CBESS contains the charging and discharging components
Adj
19 Pi,t = 0 also and is formulated as in (25). The objective function for
20 end if autonomous mode of CBESS (25) contains average genera-
21 end if
tion cost of CDGP, adjustable power cost from each MG, cost
22 end for
23 t++ of charging & discharging of CBESS, and buying & selling
24 end while prices from/to the power grid.
  Adj
min CtCDGP · PCDGP
t + CDG
Ck,i,t · Pk,i,t
t∈T k∈K i∈I t∈T
MG, buying price, and selling price from/to the power grid.  
  + CtCB− · PCB−
−t CtCB+ · PCB+
t
Adj
min CtCDGP · PCDGP + CDG
Ck,i,t · Pk,i,t t∈T
 t∈T

t  Buy Buy
t∈T
 
k∈K i∈I t∈T
 + PRt · Pk,t − PRSell
t · P Sell
k,t (25)
Buy Buy
+ PRt · Pk,t − PRSell
t · PSell
k,t (15) k∈K t∈T
k∈K t∈T
Constraints for autonomous mode of CBESS are identical
Subject to to those of subservient mode and are given by (16)-(24).

GP ≤ Pt
Pmin ≤ Pmax ∀t ∈ T
CDGP
GP (16)
min _adj Adj max _adj
Pk,i,t ≤ Pk,i,t ≤ Pk,i,t ∀k ∈ K, i ∈ I, t ∈ T C. Step 3: Local Optimization (Rescheduling)
   (17) After performing global optimization, CEMS agent informs
Buy Adj
PCDGP
t + Pk,t + Pk,i,t + k,t + Pt
PSur CB− all the external and MG-EMS agents about their commitments
k∈K k∈K i∈I k∈K and scheduling. After receiving the global optimization results
= PShort
k,t + PCB+
t + PSell
k,t ∀t ∈ T (18) from CEMS agent, each MG-EMS performs local optimization
k∈K  k∈K  again (rescheduling). The objective function for reschedul-
0≤ CB+
Pt ≤
Cap
PCB · 1 − SOCt−1
CB ing is given by (24). Equation (24) contains CDG generation
amount with adjustable power, startup cost, prices for buying
1 1
× · BTB ∀t ∈ T (19) and selling power from/to the power grid, price for receiving
1−L CB+ ηCB from other MGs and price for sending power to other MGs of
Cap 1 the MMG system.
0 ≤ PCB−
t ≤ PCB · SOCt−1
CB
· (1 − LCB− ) · ∀t ∈ T
ηCB
BTB
    
(20) Adj
min CiCDG · PCDG
i,t + P i,t + yi,t · Ci
SU
0 ≤ PCB−
t , PCB+
t ≤ PBTB
CB /ηCB
BTB
⎛ ∀t ∈ T (21)
⎞ i∈I t∈T
1 1  Buy Buy 
1 ⎜ · · PCB−
t ⎟ + PRt · Pt (t) − PRSell · PSell
· ⎝ 1 − L  ηCB
CB− BTB t t
SOCtCB = SOCt−1
CB
−  ⎠ t∈T
Cap
PCB −PtCB+
· 1−L CB+
· ηCB
BTB  
+ PRRec
t · PRec
t (t) − PRt
Send
· PSend
t (26)
(22) t∈T
1330 IEEE TRANSACTIONS ON SMART GRID, VOL. 9, NO. 2, MARCH 2018

TABLE II
PARAMETER OF BESS AND CBESS IN THE MMG S YSTEM

Fig. 6. Buying and selling price along with generation cost for CDGs
TABLE III
of MGs.
S HORTAGE , S URPLUS A MOUNT IN E ACH MG OF MMG S YSTEM

Fig. 7. Load profiles of MG1 (a), MG2 (b) and MG3 (c) in response to DR.

TABLE I
PARAMETER OF CDG S IN E ACH MG OF MMG S YSTEM

Subject to
 Adj

Buy
t + Pt
PPV + + Pi,t + Pt + PB−
t + Pt
WT
PCDG
i,t
Rec

i∈I
L_adj
= Pt + PSell
t + PB+
t + Pt
Send
∀t ∈ T
In addition to (27); load-balancing constraint, equation (26)
is also constrained to (2)-(10). In load balancing, adjustable
power and trading among MGs (sending and receiving) is also
considered as depicted by (25).
IV. N UMERICAL S IMULATIONS
A. Simulation Environment
All MILP-based models developed for simulations of the
proposed operation of MMG system have been simulated in
Java, JADE with integration of IBM ILOG CPLEX. MCNP
has been used with ACL for communication between various
agents developed for accomplishing the proposed algorithm.
The hour-wise buying and selling prices along with generation
costs for CDGs of each MG are shown in Fig. 6. Generation
and start-up costs along with generation limits of CDG units
in each MG of the MMG system are shown in Table I. In
the proposed MMG optimization strategy, BESSs in MGs and
CBESS are connected to the MMG system through back-
to-back (BTB) converters. Parameters related to CBESS and
BESSs of individual MGs are tabulated in Table II.
(27) B. Step 1: Local Optimization
It can be observed from Fig. 6 that, the peak price interval
is from t=10 to t=21. Load from this interval will be shifted
to the off-peak intervals after applying DR in step 1. Load pro-
files of MGs before and after the application of DR programs
are illustrated by Fig. 7.
Fig. 7 shows that each of the MG is shifting its load from
peak time intervals to off-peak time intervals. Apart from the
application of DR, shortage, surplus, and adjustable power
amounts are also determined after local optimization in step 1.
MG-wise, shortage and surplus amounts for each MG are at
each interval of time is shown in tables III while upper and
lower bounds for adjustable amounts are shown in table IV.
C. Step 2: Global Optimization
CEMS receives the information shown in tables III and IV
from each of the MG and average per-unit cost from
CDGP. Depending on the mode of CBESS, charging and
discharging of CBESS is decided. The test case results for
charging/discharging of CBESS in subservient mode (SM) and
autonomous mode (AM) are shown in table V. The charg-
ing/discharging of CBESS in SM is decided by CEMS for
BUI et al.: MULTIAGENT-BASED HIERARCHICAL ENERGY MANAGEMENT STRATEGY FOR MMGs
TABLE IV
B OUNDS FOR A DJUSTABLE A MOUNT IN E ACH MG OF MMG S YSTEM
Fig. 8. Buying and selling amount of electricity in case 1 and case 2.
minimizing the overall operational cost of MMG system.
While, in AM, CBESS decides it’s charging/discharging for
maximizing its own profit. During global optimization, CEMS
has to decide also about the buying and selling power to/from
the power grid. In order to elaborate the impact of proposed
strategy two cases have been compared. In case 1, only sur-
plus and shortage amounts have been proposed by MG-EMSs
to CEMS. While, in case 2, adjustable power has also been
proposed by MG-EMSs in addition to shortage and surplus
amounts. Buying and selling power for both of the cases
are depicted by Fig. 8. It can be observed from Fig. 8 that
amount of power bought from and sold to the power grid has
been remarkably reduced by applying the proposed algorithm.
Before applying the proposed algorithm, MGs were either buy-
ing at a higher price or were selling to the power grid at
a lower price.
D. Step 3: Rescheduling
After receiving the scheduling information from CEMS,
each MG-EMS reschedules its local elements to ensure the
1331
TABLE V
CBESS C HARGING AND D ISCHARGING S CHEDULE IN SM AND AM
Fig. 9. MG-wise utilized amount of adjustable power in rescheduling.
Fig. 10. Difference of buying, selling, and adjustable power in CBESS
modes.
load balancing. However, the commitment schedule of CDGs
will be changed from step 1 scheduling, due to consideration
of adjustable power. MG-wise adjustable power utilized for
rescheduling of MMG system is shown in Fig. 9. The uti-
lized amount of adjustable power at each interval is within
the bounds determined in step 1. The difference between buy-
ing, selling, and adjustable power among both of the CBESS
modes of all the three MGs are shown in Fig. 10. Area under
the buying and selling curves shows the amount of power
saved from trading with power grid by utilizing the subservient
mode of CBESS. Area under the adjustable power curve shows
the amount of power increased/decreased for minimizing the
operational cost of MMG system in subservient mode.
V. C ONCLUSION
A novel strategy for reducing operational cost of MMG
systems through MAS has been proposed in this paper. In con-
trast to conventional energy management strategies, adjustable
power is also proposed by each MG-EMS in addition to the
1332
surplus and shortage amounts. The mismatch of total surplus
and shortage power has been not only compensated by trading
power with utility grid or external resource (CBESS/CDGP)
but also can be adjusted by exchanging power with other
MGs. Being a cooperative model, the trading among MGs
has been increased by increasing the power of cheaper CDGs
having adjustable power. This trading has resulted in reduc-
tion of external trading and operation cost of MMG system.
The proposed strategy has proved to be more cost efficient
as compared to the prevailing energy management strategies.
All the developed models are based on MILP, which makes it
easy to implement and computationally inexpensive. DR has
also been applied to MMG network and the effects of DR
are evident from the simulation results. Due to the hierarchi-
cal EMS, the proposed optimization strategy distributes the
computational burden and yet makes the optimal energy man-
agement of MMGs possible. Simulation studies have shown
that, in order to reduce the overall operating cost of MMG
system, the subservient mode of CBESS is more effective.
CBESS can increase its benefit in autonomous operational
mode at the cost of elevated operational cost of the entire
MMG system.
R EFERENCES
[1] Q. Jiang, M. Xue, and G. Geng, “Energy management of microgrid
in grid-connected and stand-alone modes,” IEEE Trans. Power Syst.,
vol. 28, no. 3, pp. 3380–3389, Aug. 2013.
[2] Z. Wang, B. Chen, J. Wang, and J. Kim, “Decentralized energy
management system for networked microgrids in grid-connected and
islanded modes,” IEEE Trans. Smart Grid, vol. 7, no. 2, pp. 1097–1105,
Mar. 2016.
[3] D. E. Olivares, C. A. Cañizares, and M. Kazerani, “A centralized energy
management system for isolated microgrids,” IEEE Trans. Smart Grid,
vol. 5, no. 4, pp. 1864–1875, Jul. 2014.
[4] Y. Zhang, N. Gatsis, and G. B. Giannakis, “Robust energy management
for microgrids with high-penetration renewables,” IEEE Trans. Sustain.
Energy, vol. 4, no. 4, pp. 944–953, Oct. 2013.
[5] Z. Wang, K. Yang, and X. Wang, “Privacy-preserving energy schedul-
ing in microgrid systems,” IEEE Trans. Smart Grid, vol. 4, no. 4,
pp. 1810–1820, Dec. 2013.
[6] P. Malysz, S. Sirouspour, and A. Emadi, “An optimal energy storage
control strategy for grid-connected microgrids,” IEEE Trans. Smart Grid,
vol. 5, no. 4, pp. 1785–1796, Jul. 2014.
[7] H. HassanzadehFard, S. M. Moghaddas-Tafreshi, and S. M. Hakimi,
“Effect of energy storage systems on optimal sizing of islanded micro-
grid considering interruptible loads,” in Proc. 3rd IEEE Int. Youth Conf.
Energ. (IYCE), Leiria, Portugal, Jul. 2011, pp. 1–7.
[8] A. J. Conejo, J. M. Morales, and L. Baringo, “Real-time demand
response model,” IEEE Trans. Smart Grid, vol. 1, no. 3, pp. 236–242,
Dec. 2010.
[9] S. Shao, M. Pipattanasomporn, and S. Rahman, “Demand response as
a load shaping tool in an intelligent grid with electric vehicles,” IEEE
Trans. Smart Grid, vol. 2, no. 4, pp. 624–631, Dec. 2011.
[10] Z. Zhou, F. Zhao, and J. Wang, “Agent-based electricity market simu-
lation with demand response from commercial buildings,” IEEE Trans.
Smart Grid, vol. 2, no. 4, pp. 580–588, Dec. 2011.
[11] T. Logenthiran, D. Srinivasan, A. M. Khambadkone, and H. N. Aung,
“Multiagent system for real-time operation of a microgrid in real-time
digital simulator,” IEEE Trans. Smart Grid, vol. 3, no. 2, pp. 925–933,
Jun. 2012.
[12] M. H. Cintuglu, H. Martin, and O. A. Mohammed, “Real-time imple-
mentation of multiagent-based game theory reverse auction model for
microgrid market operation,” IEEE Trans. Smart Grid, vol. 6, no. 2,
pp. 1064–1072, Mar. 2015.
IEEE TRANSACTIONS ON SMART GRID, VOL. 9, NO. 2, MARCH 2018
[13] S. D. J. McArthur et al., “Multi-agent systems for power engineering
applications—Part I: Concepts, approaches, and technical challenges,”
IEEE Trans. Power Syst., vol. 22, no. 4, pp. 1743–1752, Nov. 2007.
[14] S. D. J. McArthur et al., “Multi-agent systems for power engineering
applications—Part II: Technologies, standards, and tools for build-
ing multi-agent systems,” IEEE Trans. Power Syst., vol. 22, no. 4,
pp. 1753–1759, Nov. 2007.
[15] L. Wang, Z. Wang, and R. Yang, “Intelligent multiagent control system
for energy and comfort management in smart and sustainable buildings,”
IEEE Trans. Smart Grid, vol. 3, no. 2, pp. 605–617, Jun. 2012.
[16] T. Logenthiran, D. Srinivasan, and A. M. Khambadkone, “Multi-agent
system for energy resource scheduling of integrated microgrids in a dis-
tributed system,” Elect. Power Syst. Res., vol. 81, no. 1, pp. 138–148,
Jan. 2011.
[17] H.-M. Kim, T. Kinoshita, and M.-C. Shin, “A multiagent system for
autonomous operation of islanded microgrids based on a power market
environment,” Energies, vol. 3, no. 12, pp. 1972–1990, Dec. 2010.
[18] J. Lagorse, D. Paire, and A. Miraoui, “A multi-agent system for energy
management of distributed power sources,” Renew. Energy, vol. 35,
no. 1, pp. 174–182, Jan. 2010.
[19] Y. S. F. Eddy, H. B. Gooi, and S. X. Chen, “Multi-agent system for dis-
tributed management of microgrids,” IEEE Trans. Power Syst., vol. 30,
no. 1, pp. 24–34, Jan. 2015.
[20] H.-M. Kim, Y. Lim, and T. Kinoshita, “An intelligent multiagent sys-
tem for autonomous microgrid operation,” Energies, vol. 5, no. 9,
pp. 3347–3362, Sep. 2012.
[21] Y. Lim, H.-M. Kim, and T. Kinoshita, “Distributed load-shedding sys-
tem for agent-based autonomous microgrid operations,” Energies, vol. 7,
no. 1, pp. 385–401, 2014.
[22] M. Mao, P. Jin, N. D. Hatziargyriou, and L. Chang, “Multiagent-based
hybrid energy management system for microgrids,” IEEE Trans. Sustain.
Energy, vol. 5, no. 3, pp. 938–946, Jul. 2014.
[23] C.-X. Dou and B. Liu, “Multi-agent based hierarchical hybrid control for
smart microgrid,” IEEE Trans. Smart Grid, vol. 4, no. 2, pp. 771–778,
Jun. 2013.
[24] C. M. Colson, M. H. Nehrir, R. K. Sharma, and B. Asghari, “Improving
sustainability of hybrid energy systems part II: Managing multiple objec-
tives with a multiagent system,” IEEE Trans. Sustain. Energy, vol. 5,
no. 1, pp. 46–54, Jan. 2014.
[25] T. Morstyn, B. Hredzak, and V. G. Agelidis, “Cooperative multi-agent
control of heterogeneous storage devices distributed in a DC microgrid,”
IEEE Trans. Power Syst., vol. 31, no. 4, pp. 2974–2986, Jul. 2016.
[26] N.-O. Song, J.-H. Lee, H.-M. Kim, Y.-H. Im, and J.-Y. Lee, “Optimal
energy management of multi-microgrids with sequentially coordinated
operations,” Energies, vol. 8, no. 8, pp. 8371–8390, Aug. 2015.
[27] E. M. Hammad, A. K. Farraj, and D. Kundur, “Grid-independent coop-
erative microgrid networks with high renewable penetration,” in Proc.
IEEE Power Energy Soc. Innov. Smart Grid Technol. Conf. (ISGT),
Washington, DC, USA, Feb. 2015, pp. 1–5.
[28] H. S. V. S. K. Nunna and S. Doolla, “Demand response in smart dis-
tribution system with multiple microgrids,” IEEE Trans. Smart Grid,
vol. 3, no. 4, pp. 1641–1649, Dec. 2012.
[29] H. S. V. S. K. Nunna and S. Doolla, “Responsive end-user-based demand
side management in multimicrogrid environment,” IEEE Trans. Ind.
Informat., vol. 10, no. 2, pp. 1262–1272, May 2014.
[30] H. S. V. S. K. Nunna and S. Doolla, “Energy management in micro-
grids using demand response and distributed storage—A multiagent
approach,” IEEE Trans. Power Del., vol. 28, no. 2, pp. 939–947,
Apr. 2013.
[31] H. S. V. S. K. Nunna and S. Doolla, “Multiagent-based distributed-
energy-resource management for intelligent microgrids,” IEEE Trans.
Ind. Electron., vol. 60, no. 4, pp. 1678–1687, Apr. 2013.
[32] Java Agent Development Environment (JADE). Accessed on Jul. 2016.
[Online]. Available: http://jade.tilab.com
[33] IBM ILOG CPLEX V12.1 User’s Manual for CPLEX 2009, CPLEX
Division, ILOG, Incline Village, NV, USA, 2009.
[34] FIPA. The Foundation for Intelligent Physical Agents Standards.
Accessed on Jul. 2016. [Online]. Available: http://www.fipa.org
[35] B. Zhao, X. Zhang, J. Chen, C. Wang, and L. Guo, “Operation opti-
mization of standalone microgrids considering lifetime characteristics
of battery energy storage system,” IEEE Trans. Sustain. Energy, vol. 4,
no. 4, pp. 934–943, Oct. 2013.
[36] M. Choobineh, P. C. Tabares-Velasco, and S. Mohagheghi, “Optimal
energy management of a distribution network during the course of a heat
wave,” Elect. Power Syst. Res., vol. 130, pp. 230–240, Jan. 2016.
BUI et al.: MULTIAGENT-BASED HIERARCHICAL ENERGY MANAGEMENT STRATEGY FOR MMGs 1333

Van-Hai Bui received the B.S. degree in electrical Hak-Man Kim received the first Ph.D. degree
engineering from the Hanoi University of Science in electrical engineering from Sungkyunkwan
and Technology, Vietnam, in 2013. He is currently University, South Korea, in 1998, and the second
pursuing the combined master’s and Ph.D. degrees Ph.D. degree in information sciences from Tohoku
with the Department of Electrical Engineering, University, Japan, in 2011. He was with Korea
Incheon National University, South Korea. His Electrotechnology Research Institute, South Korea,
research interests include microgrid operation and from 1996 to 2008. He is currently a Professor with
energy management system. the Department of Electrical Engineering, Incheon
National University, South Korea. His research inter-
ests include microgrid operation and control and dc
power systems.
Akhtar Hussain received the B.E. degree in
telecommunications from the National University
of Sciences and Technology, Pakistan, in 2011,
and the M.S. degree in electrical engineering from
Myongji University, Yongin, South Korea, in 2014.
He is currently pursuing the Ph.D. degree with
Incheon National University, South Korea. He was
an Associate Engineer in SANION, South Korea,
from 2014 to 2015. His research interests are distri-
bution automation and protection, smart grids, and
microgrid optimization.