Applied Energy 119 (2014) 341–350

Contents lists available at ScienceDirect

Applied Energy
journal homepage: www.elsevier.com/locate/apenergy

A game theoretic framework for a next-generation retail electricity

market with high penetration of distributed residential electricity
suppliers
Wencong Su a,⇑, Alex Q. Huang b
a
Department of Electrical and Computer Engineering, University of Michigan-Dearborn, MI 48128, USA
b
NSF Future Renewable Electric Energy Delivery and Management (FREEDM) Systems Center, North Carolina State University, Raleigh, NC 27606, USA

h i g h l i g h t s

 Propose a retail electricity market framework in future residential distribution systems with distributed energy suppliers.
 Identify the key features and functionalities of the proposed retail electricity market framework.
 Formulate the key market participants (Energy Cell and Utility Cell).
 Apply novel game-theoretic algorithms to clear the retail electricity market price.

a r t i c l e i n f o a b s t r a c t

Article history: This paper proposes an innovative game theoretic framework for a next-generation retail electricity mar-
Received 26 July 2013 ket (‘‘Energy Internet’’) with high penetration of distributed residential electricity suppliers (‘‘Energy
Received in revised form 30 December 2013 Cells’’). The envisioned Energy Internet is developed for plug-and-play of a large number of distributed
Accepted 2 January 2014
renewable energy generation and energy storage. The residential customers, referred as Energy Cells,
Available online 31 January 2014
are not only the electricity consumers but can also be the electricity suppliers by locally operating and
managing their own distributed generators, distributed energy storage devices, and dispatchable loads.
Keywords:
This paper formulates a set of mathematical models of the retail electricity market participants with a
Smart Grid
Energy Management
number of local and global constraints. A numerical case study is performed to validate the proposed
Distribution system next-generation retail electricity market framework using game-theoretic methodologies. The numerical
Electricity market simulation results demonstrate the effectiveness of the proposed market clearing scheme with high pen-
Game theory etration of distributed residential electricity suppliers.
Renewable energy Ó 2014 Elsevier Ltd. All rights reserved.

1. Introduction the environmental costly fossil fuel. It is remarkable that we will

The majority of the current US power grid infrastructure was
built almost 80 years ago, and this aging and overburdened power
grid has experienced five massive blackouts in the past four
decades [1]. Smart Grid technologies provide an innovative
solution to restructuring the current power grid infrastructure
and ensuring the reliability of energy supply [2]. Driven by
environmental incentives, the US government has spurred efforts
to accelerate the massive introduction of renewable energy.
However, the US Energy Information Administration (EIA) 2012
annual report projects that the renewable share of electric power
generation would only increase from 10% in 2010 to 16% in 2035
[3]. The majority of electricity generation still heavily relies on
⇑ Corresponding author. Tel.: +1 313 593 5314.
E-mail addresses: wencong@umich.edu (W. Su), aqhuang@ncsu.edu (A.Q.
Huang).
depend on fossil fuels in 2035 just remarkably like what we did
in 1980. At present, most of renewable-based electricity (e.g., off-
shore wind farm) is directly injected at the transmission level.
Those central generators are normally at locations far removed
from population centers that ultimately consume the electricity.
The wide adoption of intermittent renewable energy sources is still
a huge challenge because of the high cost of expanding and main-
taining the transmission system. There is an urgent need to devel-
op a new residential distribution infrastructure (Energy Internet)
for plug-and-play of distributed renewable energy generation
and storage.
Recently, the Future Renewable Electric Energy Delivery and
Management (FREEDM) Systems Center proposed the Energy Inter-
net concept to develop an efficient power grid integrating highly
distributed and scalable green energy [4–6]. The vision of an Energy
Internet has been inspired by the paradigm shift in computer indus-
try in early 1980s. The centralized computer mainframes gave way

0306-2619/$ - see front matter Ó 2014 Elsevier Ltd. All rights reserved.
http://dx.doi.org/10.1016/j.apenergy.2014.01.003
342 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350
to a distributed computing infrastructure which allows individual
users to access via a worldwide internet. From its inception, the
information Internet (computer network) innovations have been
completely reshaping the traditional views of our social environ-
ment and business world. In a similar paradigm shift, highly distrib-
uted and scalable renewable energy resources must play a primary
role, of which the centralized bulk power plants are currently serv-
ing. Fig. 1 illustrates the similar paradigm shift in computer industry
(‘‘Information Internet’’) and power industry (‘‘Energy Internet’’).
This paradigm shift can only be achieved by encouraging and
facilitating the participation of individual residential customer
with a future power distribution infrastructure (i.e., Energy Inter-
net). It is important to mention that the full version of Energy
Internet relies on a wide range of emerging technologies such as
a reliable two-way communication network [7], two-way power
flow [8], and advanced power electronic devices. The underlying
power delivery and management system (i.e., distribution infra-
structure) must change significantly from today’s system.
The major contributions of this paper include the following four
aspects:
1. Propose a new retail electricity market framework in a future
residential distribution system (Energy Internet) with a large
number of distributed residential energy suppliers/consumers
(Energy Cells).
2. Identify the key features and functionalities of the proposed
retail electricity market framework.
3. Formulate the key market participants (Energy Cell and Utility
Cell) of Energy Internet.
4. Apply novel game-theoretic algorithm to clear the deregulated
retail electricity market price subject to a number of local and
global system constraints.
The remainder of this paper is organized as follows. Section 2
introduces the Energy Internet market framework with key
components (Energy Cells and Utility Cell), and formulates the
Fig. 1. Paradigm shift in computer industry (Information Internet) and power
industry (Energy Internet).
proposed retail electricity market framework in mathematical
terms. Section 3 presents some related game-theoretic concept
and methodologies (e.g., Nikaido–Isoda function and relaxation
algorithm). Section 4 performs a case study to demonstrate the
proposed retail electricity market clearing scheme followed by a
detailed analysis and discussion. Section 5 summarizes the paper’s
main findings and discusses future research topics.
2. Materials and methods
2.1. Energy Internet market framework
The vision of Energy Internet we promote will require the
equivalent of a Manhattan Project for energy, as suggested in Tho-
mas Friedman’s book, The World is Flat. We believe that the next-
generation residential distribution system is a level playing field
in terms of electricity transactions, where all residential customers
have an equal opportunity. While legacy power system operations
are solely driven by least-cost and reliability concerns, the Energy
Internet innovations are completely reshaping the traditional
views of our power generation, distribution, consumption, social
environment, and business world. The proposed Energy Internet
requires that our society move away from, or at least supplement,
the traditional centralized generation, distribution, and consump-
tion business model to one where every user can actively partici-
pate in the energy market. User participation is a major factor
that has resulted in exponentially increasing innovation and inge-
nuity in the information technology sector (e.g., Amazon, eBay and
Facebook). If the ‘‘Information Internet’’ was the engine that pow-
ered our country’s economic growth in the last thirty years, then a
similar and even more powerful Energy Internet will accomplish
even more in the next several decades. In this context, the pro-
posed new electricity market framework is analogous to the stock
market or the business model of internet consumer-to-consumer
(C2C) commerce such as eBay. This paper is first known effort to
advance fundamental knowledge of a next-generation retail elec-
tricity market framework with full residential customer participa-
tion. Fig. 2 illustrates the proposed Energy Internet architecture.
Although the electricity market deregulation has been under-
way for more than 20 years since the United Kingdom opened a
Power Pool in April 1990 [9], competitive forces in the US retail
electricity markets have been completely silent in the last decade.
Especially, early-2000 major failures such as the California electric-
ity crisis slowdowns the pace of deregulation and even results in
an increase in market regulation and a reduction in competition.
The Public Utility Regulatory Policies Act of 1978 [10] led to the
creation of Independent System Operator (ISO) in US. The growth
of ISOs creates a boarder, more competitive electricity market for
bulk independent power producers. However, ISOs only deal with
large blocks of aggregate load or generation on the transmission le-
vel, and do not reach down into the distribution systems. In other
words, liberalization of electricity tends to substantially benefit
large consumers (e.g., industrial users) only. The residential cus-
tomers still have very limited access to fully engage with the retail
electricity market to choose their supplier from competing elec-
tricity retailers. Many retail electricity providers (New York, Penn-
sylvania, Texas, New Jersey, Colorado, and California) started to
advocate the free market in the utility business, but failed to devel-
op a well-justified business model. The immature open market
framework may even cause electricity prices there to skyrocket.
There is an urgent need to establish a transformative retail electric-
ity market framework in a next-generation residential distribution
system (Energy Internet) with high participation of distributed res-
idential electricity suppliers/consumers (Energy Cell).
It is worth noting that the proposed Energy Internet is different
from the well-established concept of liberalized electricity market.
 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350
Fig. 2. An envisioned Energy Internet architecture.
Firstly, the utility still owns and operates the distribution systems
exclusively and makes profit by only selling electricity in a liberal-
ized electricity market. The majority of existing studies [11–13] fo-
cus on power system operations from the utility’s perspective. The
overall objective of these works is to achieve the optimal power
and energy dispatch (e.g., minimizing the total operating cost, min-
imizing the power losses, and minimizing the carbon emission). In
the proposed Energy Internet market framework, individual resi-
dential customers or a group of entities are defined as Energy Cells.
This scalable Energy Cell can be as large as a town or as small as a
house. The Energy Cells are composed of local generators, energy
storage, and/or controllable loads. Hundreds of thousands of En-
ergy Cells are physically connected with each other in an inter-
net-like structure. Meanwhile, each energy cell can exchange the
real-time data and information with other cells, social and environ-
mental contexts through a two-way communication network
infrastructure. Essentially, each Energy Cell is a selfish entity/ob-
ject whose objective is to maximize its own benefits subject to a
number of local and global constraints. Energy Cell may have par-
tially or totally conflicting interests with each other. In such an
electricity market, Energy Cells can somehow become independent
of the conventional electricity supplier (e.g., utility grid). Secondly,
since the number of sellers (e.g., utility company, power system
operators, and retailer) is quite fixed and static in a liberalized elec-
tricity market, the market structure and network topology is easy
to pre-determine at any given time. On the contrast, Energy Inter-
net with a large number of heterogeneous Energy Cells is indeed a
highly dynamic and complex system with spatial and temporal
uncertainties. Thirdly, in the last decade, a number of studies have
investigated the impacts of renewable generation on wholesale
market price profiles [14–18]. However, to date, the impact of
343
distributed renewable energy on the retail electricity market has
not been well-studied yet. The wide adoption of distributed renew-
able energy mainly depends on the participation level of residen-
tial customers. Unfortunately, the existing electricity market is
not suitable for the plug-and-play of a large number of heteroge-
neous Energy Cells. Another important and unique feature of En-
ergy Internet is that residential customers can offset their electric
cost, or even make additional income, by selling excess self-gener-
ated power back to utility grid or neighborhood. Energy Internet
provides residential customers with significant economic incen-
tives for self-installation of distributed renewable generators
(e.g., local-scale wind turbine, and roof-top photovoltaic panel).
Lastly but not least, in a conventional liberalized electricity market,
the system operation is performed using purely centralized data
storage and management approaches. However, as the number of
Energy Cells increases to more than hundreds of thousands, it is
rather intuitive that the current state-of-the-art centralized infor-
mation processing architecture will no longer be sustainable under
such data-explosion.
Compared with Cournot model, the proposed Energy Internet
market has the similar features and assumptions:
1. All the Energy Cells produce a homogeneous product (i.e., elec-
tricity) with no product differentiation.
2. Energy Cells may have partially or totally conflicting interest
with each other.
3. The number of Energy Cells and Utility Cell is fixed during the
market clearing process.
4. Each Energy Cell’s decision affects the retail electricity price,
which is the result of competitive and intelligent purchasing/
selling strategies.
344 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350
5. Energy Cells are economically rational to maximize their own
profits by managing their own DGs, DESDs, and/or dispatchable
load in quantities (i.e., the amount of electricity) given their
competitors’ strategies.
6. Each Energy Cell’s electricity bill includes, but is not limited to:
(1) the energy cost of selling or/and consuming the electricity;
(2) the cost of delivering the electricity to a specific location
(e.g., locational distribution marginal price); and (3) other
incentives (e.g., tax incentives for maintaining a certain pene-
tration of renewable generation).
The role of utility grid is different than that in a legacy electric-
ity market. In such a deregulated electricity market, the utility no
longer owns power plants exclusively nor do they profit by only
selling energy. In comparison with the topology of the bulk power
system, which is relatively static, a residential distribution system
with a number of heterogeneous Energy Cells can be a highly dy-
namic and complex system. The utilities will be playing a more
important role in ensuring the residential distribution system
security and reliability. In addition to electricity transaction, utili-
ties are expected to make more profit on providing ancillary ser-
vices (e.g., scheduling and dispatch; reactive power and voltage
control; loss compensation; load following; system protection;
and energy imbalance).
This paper is an exploratory work toward a new paradigm for
the fully deregulated retail electricity market with high competi-
tion of distributed energy suppliers. Due to the limited space, we
only focus on the day-ahead retail electricity market in this paper.
We assume that all market participants (e.g., energy cells) commit
to follow the clearing price once the day-ahead market has settle
down. This proposed retail electricity market framework is based
on a reasonable Cournot model which has been widely used to
approximate the competition in the electricity market [19],[20].
To protect the privacy of individual bids and offers, the Energy
Cells and Utility Cell do not need to disclose their operating condi-
tions to each other. All the bids and offers are aggregated into the
centralized market clearing house. Fig. 3 shows the proposed retail
market framework of Energy Internet. The following section will
presents detailed mathematical formulation of the market partici-
pants. Transactions in the new energy market take place in a man-
ner similar to how today’s stock market operates. The Energy
Internet allows those new participants to fully engage with a dy-
namic and diverse energy market through a distributed decision
making process.
Fig. 3. A new market framework of Energy Internet.
2.2. Problem formulation
This section presents the mathematical formulation of the key
components in the proposed retail electricity market. The generic
framework can be easily extended and tailored to other power dis-
tribution systems with high levels of residential customer partici-
pation. In the following context, the terms entity, player, customer,
and energy cell will be interchangeably used, unless an explicit dis-
tinction is required.
2.2.1. Objective functions
Table 1 illustrates three types of Energy Cells based on their sys-
tem configurations and overall objective functions.
For Type 1 ‘‘Energy Cell’’, the net benefit Ji of the ith energy cell
at the tth hour is defined as:
X
24
max J i ¼ ðRt;i  C t;i Þ; ð1Þ
t¼1
where Rt,i and Ct,i are the revenue and cost functions, respectively, of
the ith energy cell at the tth hour.
The inverse load demand curve of the ith energy cell follows a
linear relationship:
ki ðP d;i Þ ¼ hi  Pd;i þ bi hi > 0; ð2Þ
bi, hi are the load demand curve coefficients, Pd,i is the load demand
of ith energy cell, and ki is the electricity price for ith energy cell in
$/kW h.
For sake of simplicity, the retail electricity price is assumed to
be identical for all energy cells within a single residential distribu-
tion system. Therefore, (2) can be expressed as:
kðPtotal Þ ¼ h  P total þ b h > 0; ð3Þ
where b, h are the load demand curve coefficients, k is the uniform
electricity price in $/kW h, and Ptotal is the total load demand of the
given distribution system.
The revenue function Rt,i is expressed as:
Ri;t ¼ kðPtotal Þ  ½PDG;i ðtÞ þ PDESD;i ðtÞ þ Pwind;i ðtÞ þ Psolar;i ðtÞ
 Pd;i ðtÞ; ð4Þ
where PDG,i is the DG power output of ith energy cell, PDESD,i is the
DESD power output of ith energy cell, Pwind,i is the wind power out-
put of ith energy cell, and Psolar,i is the solar power output of ith en-
ergy cell.
 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350 345

Table 1
Classification of Energy Cells.

Type Configuration Objective

1 DG (e.g., diesel generator, and micro gas turbine) Wind/ Make additional income by selling excessive power to utility grid or neighboring customers
Solar Generators DESD dispatchable load Maintain a certain comfort level Comply with applicable laws and regulation such as the Not-In-My-
Backyard (NIMBY) and Build-Absolutely-Nothing-Anywhere-Near-Anybody (BANANA)
2 Dispatchable load Minimize the energy cost maintain a certain comfort level
3 Non-dispatchable load Maintain the critical load

At any given time t, the ith energy cell serves as an energy sup- 2.2.2.1. Technical limits of DG in Type 1 Energy Cell. At any given
plier when Rt,i is positive; the ith energy cell serves as an energy hour t, the ith energy cell operates its own DG subject to:
consumer when Rt,i is negative.
PDG;i;min  PDG;i ðtÞ  PDG;i;max 8t; 8i; ð10Þ
The cost function Ct,i can be expressed as:
where PDG,i,min is the minimum power output if DG is on
C i;t ¼ g wind ðPwind ði; tÞÞ þ g solar ðPsolar ði; tÞÞ þ g DG ðPDG ði; tÞÞ
(kW), and PDG,i,max is the maximum power output if DG is
þ g DESD ðPDESD ði; tÞÞ; ð5Þ on (kW).
where g denotes the production cost function in $. Wind and solar
2.2.2.2. Technical limits of DESD in Type 1&2 Energy Cell. At any
energy generators are considered as non-dispatchable units, and
given hour t, the ith energy cell operates its own DESD (e.g., battery
their power output depends on uncertain and variable energy
bank) subject to:
sources. In this paper, we don’t consider their capital cost. From a
long-term point of view, the production cost of the renewable en- PDESD;i;min  P DESD;i ðtÞ  PDESD;i;max 8t; 8i; ð11Þ
ergy (i.e., wind and solar) is assumed to be negligible such that
gwind = 0 and gsolar = 0. The degradation cost of DESD is beyond the where PDESD,i,min is the minimum power output if DG is on (kW), and
scope of this paper, so we assume gDESD = 0. Because the small-scale PDESD,i,max is the maximum power output if DG is on (kW).
DGs have negligible start-up/shut-down time, the start-up/shut- To avoid any overcharging and overdischarging, once the bat-
down cost can be approximated as a constant for each DG unit. tery State-of-Charge (SOC) reaches the upper or lower bounds,
The production cost of DG is formulated as a strictly convex qua- the DESD switches to a stand-by mode.
dratic function as: SOC i;min  SOC i ðtÞ  SOC i;max 8t; 8i; ð12Þ
C i;t ¼ ai P2DG;i ðtÞ þ bi PDG;i ðtÞ þ ci ai > 0; ð6Þ SOC i ðt þ 1Þ  SOC i ðtÞ ¼ PDESD;i ðtÞ  Dt=EDESD;i 8t; 8i; ð13Þ

where ai, bi, ci are coefficients of DG cost function. where SOC is the battery State-of-Charge, SOCi,min is the minimum
The ith energy cell maximizes its own net benefit over a period battery storage State-of-Charge, SOCi,max is the maximum battery
of 24 h as follows: storage State-of-Charge, EDESD,i is the battery capacity in kW h and
Dt is the time interval (i.e., one hour).
X
24
To maintain a certain amount of electricity stored in DESD at
Ji ¼ ðRt;i  C t;i Þ
the beginning of the next day, the minimum SOC requirement at
t¼1
the 24th hour SOCi,end is imposed as:
X
24
¼ fkðP total Þ  ½PDG;i ðtÞ þ PDESD;i ðtÞ þ Pwind;i ðtÞ þ Psolar;i ðtÞ SOC i ð24Þ  SOC i;end 8i: ð14Þ
t¼1

 Pd;i ðtÞ  ½ai PDG;i ðtÞ2 þ bi P DG;i ðtÞ þ ci g: ð7Þ

2.2.2.3. Technical limits of load in Type 1&2 Energy Cell. The load is
For Type 2 ‘‘Energy Cell’’, the objective is to minimize the operating composed of critical and non-critical ones. To supply the critical
cost by managing their own dispatchable loads. The ith energy cell load all the time, the load demand of the ith energy cell has to meet
at the tth hour is defined as: a certain requirement as follows:
X
24 Pd;i ðtÞ  ri  Pbase;i ðtÞ 8t; 8i; ð15Þ
min J i ¼ ½kðP total Þ  Pd;i ðtÞ: ð8Þ
t¼1 where Pbase,i is the base load demand profile in kW h, and ri is the
percentage of critical load in %.
For Type 3 ‘‘Energy Cell’’, their top priority is to ensure the critical
load. Therefore they have little flexibility to control their load de-
2.2.2.4. Local regulation of wind generation in Type 1 Energy Cell. The
mand. With a fixed load demand, they only serve as price-takers.
generation cost of renewable energy is negligible, the most
In the proposed electricity market, the ‘‘Utility Cell’’ makes prof-
economical operation is to produce as much wind and solar as
its by selling electricity and providing ancillary services to energy
possible. However, due to the applicable local laws and regula-
cells. The objective function is formulated as:
tions such as NIMBY and BANANA, such operation may not be
X
24 practically possible. For example, because of the machine noise,
max J i ¼ ½kðPtotal Þ  Pgrid ðtÞ þ Y serv ice ðtÞ; ð9Þ some locally-owned wind turbines that belong to energy cells
t¼1
X may not be allowed to fully operate during a certain period
where Pgrid(t) is the total power from/to a utility grid, and Yservice(t) of time T.
are the benefits of providing the ancillary services.
Pwind;i ðtÞ  Pwind;i;max i 2 X; t 2 T: ð16Þ

2.2.2. Constraints The global system constraints include the following:

Energy Cells make decisions on their own subject to local and
global system constraints. 2.2.2.5. Power balance. At any given hour t, there must be a balance
The local constraints include the following: of the customers’ power generation from wind, solar, DG and DESD
346 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350
and the total power loss PLoss, the total load demand, and the
energy transaction Pgrid from/to the utility grid. This balance is
expressed as:
X unchanged. x⁄ is considered a Nash normalized equilibrium point if:
½Pwind;i ðtÞ þ Psolar;i ðtÞ þ PDG;i ðtÞ þ PDESD;i ðtÞ
i2N
X ðx ;yÞ2X
¼ PLoss ðtÞ þ Pd;i ðtÞ  Pgrid ðtÞ 8t:
i2N
2.2.2.6. Power flow constraints. Certain power flow constraints have
to be taken into consideration: H() 6 0,
V min  V d  V max 8d;
where H() is a set of power flow equations, Vd is the voltage at bus
d, Vmin is the minimum bus voltage limit, and Vmax is the maximum
bus voltage limit.
3. Theory
In the proposed electricity market, all of the Energy Cells
compete with each other by setting their power output and load
demand. The nature of the proposed electricity market fits into
n-person game theory quite well, and the game theory concept
has been successfully applied to power system problems [21,22].
This paper considers a non-cooperative n-person game in which
each player maximizes its own objective through a distributed
decision-making process. This paper does not consider a scenario
where some players may communicate and cooperate with others
to form a group or coalition. This non-cooperative n-player game
can be iteratively solved using certain game-theoretic methodolo-
gies (e.g., Nikaido–Isoda function and relaxation algorithm) which
have been proven to be successful for a wide class of real-world
problems [23–27].
3.1. Nikaido–Isoda function
In a non-cooperative n-person game, each player may have
partially or totally conflicting interest with each other. They
all try to maximize their own welfare by setting their own
strategies. In a mathematical form, an n-person game is defined
by a three-tuple as {N, Xi, /i}, i e N. Each player i belongs to a
set of N = {1, 2, 3, . . ., n} players. Xi is the strategy space of
player i. The collective strategy set is defined as X = X1  X2
 . . .  Xn. According to its own strategy, each player takes an
action by a vector xi. The collective action set is defined as
x = (x1, x2, . . ., xn). /i is the pay-off function of player i, which
quantifies the benefit that a player can obtain by setting its
own strategy given a strategy space of others. (yi|x) denotes
the element (x1, . . ., xi1, yi, xi+1, . . ., xn). It indicates that player i
takes the action yi while the other players are playing
(x1, . . ., xi1, xi+1, . . ., xn). The Nash equilibrium is defined as a
point x ¼ ðx 1 ; . . . ; x n Þ where, for any i.
/i ðx Þ ¼ max /i ðxi jxÞ:
ðxi jxÞ2X
In other words, there exists a Nash equilibrium if, for all players
i e N, /i ðx i jx Þ  /i ðxi jx Þ always holds for all feasible xi. No player
can improve its pay-off function by unilaterally changing its own
strategy given that the strategies of the other players are fixed.
(8) and (9) in Section III can be considered pay-off functions, respec-
tively, in the forms of (19).
A Nikaido–Isoda function [28] is introduced to transform the
Nash equilibrium problem into an optimization problem. The Nika-
ido–Isoda function is defined as:
X
n concave [26], and because the Nikaido–Isoda function W(x, y) is
Wðx; yÞ ¼ ½/i ðyi jxÞ  /i ðxÞ:
i¼1
On the right hand side, the term [/i(yi|x)  /i(x)] represents the
improvement in the ith pay-off function when player i changes its
action from xi to yi, while the strategies of all other players remain
max Wðx ; yÞ ¼ 0: ð21Þ
ð17Þ
When the Nikaido–Isoda function reaches zero, no player can in-
crease its payoffs by unilaterally changing its strategy. A Nash nor-
malized equilibrium point is also a Nash equilibrium point [25] if:
X ¼ X1  X2      Xn : ð22Þ
ð18Þ
Therefore, the Nikaido–Isoda function can approximately approach
a Nash equilibrium point. Accordingly, the optimal response func-
tion is expressed as the maximum result of the Nikaido–Isoda
function.
ZðxÞ ¼ argmaxWðx; yÞ x; ZðxÞ 2 X: ð23Þ
y2X
The function argmax stands for the argument of the maximum, that
is to say, the set of values of x for which W(x, y) attains its largest
value. Each player tries to unilaterally maximize its own benefit.
3.2. Relaxation algorithm
The relaxation algorithm iteratively update the optimal re-
sponse function until it converges to a Nash equilibrium point x⁄.
The initial guess x0 is based on the knowledge of the given game.
In this paper, x0 is assumed to be a null vector. The optimal estima-
tion of x0 is beyond the scope of this paper. The next step is to iter-
atively update the optimal response function using a relaxation
algorithm.
xkþ1 ¼ ð1  ak Þxk þ ak Zðxk Þ k ¼ 0; 1; 2;   ; ð24Þ
where k is the step of iteration and ak is a weighting term assigned
to xk and Z(xk) to obtain the new xk+1 at the next step k The optimal
response function is iteratively updated until the following stopping
condition is satisfied.
max Wðxk ; yÞ  e: ð25Þ
ðxk ;yÞ2X
e is a user-defined small value to control the convergence rate. The
convergence of the relaxation algorithm with a constant ak is guar-
anteed for a convex-concave function when ak satisfies the follow-
ing conditions [27,29]:
X
1
0 < ak  1; ak ¼ 1; ak ! 0 as k ! 0: ð26Þ
k¼0
At each iteration k, an optimal value of ak is found by solving the
following optimal response function:
a k ¼ argmin½maxWðxkþ1 ðak Þ; yk Þ ð27Þ
k
ð19Þ 0<ak 1 y 2X
The convergence proof of the optimal step size can be found in
[27][28]. After several number of iterations, the relaxation algo-
rithm converges to a Nash equilibrium point x⁄.
A full detailed proof regarding the convergence of the relaxation
algorithm can be found in Appendix A of [30], of which the pro-
posed problem meets all the sufficient conditions. Namely, X is a
convex compact of the Euclidean space. Z(xk) is always a single-val-
ued output with multiple input xs and is continuous on X. Further-
more, because the pay-off functions shown in the previous section
are smooth, the Nikaido–Isoda function W(x, y) is weakly convex–
ð20Þ twice continuously differentiable, it is sufficient for the Condition
5 of the convergence theorem in [30]. Therefore, there exists a
 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350 347

unique Nash equilibrium point to which the relaxation algorithm

can reach [26]. . DG, DESD, and Dispatchable Load
. Dispatchable load
4. Results and discussion . Non-dispatchable load

This section shows a numerical case study to demonstrate the

. Utility Grid
Energy Cell
effectiveness of clearing the proposed electricity market price
using the game theory concept. All simulations were run on an In-
650
tel(R) i5 M450@2.40 GHz computer with a 6.00 GB memory. For
sake of simplicity, the same type of Energy Cell is assumed to have Type 2
the same number of DGs, DESDs, and wind/solar generators. But 646 645 632 633 634
the dynamics and cost functions of those components are not iden-
Type 1
tical across the same type of Energy Cells.

4.1. Simulation data

611 684 692 675
671
This case study uses the time-stamped wind power generation
Type 1
output data from the National Renewable Energy Laboratory’s Type 3 Type 1
Eastern Wind Integration and Transmission Study [31]. The solar
data are extracted from the System Advisor Model (SAM) devel-
652 680
oped by National Renewable Energy Laboratory [32]. Fig. 4 shows
hourly wind and solar forecasting data over a period of 24 h.
Fig. 5. A test distribution system with Energy Cells.

4.2. Test system configuration

Table 2
The test system is derived from the IEEE 13 node distribution Cost and benefit coefficients of type #1 energy cells.
test feeders [33]. The original load data serves as the base load Type 1 customer ai bi ci h b
Pbase,i. Fig. 5 illustrates the test system configuration used in this
$/kW2 h $/kW h $/h $/kW h $/h
case study. Table 2 summarizes the cost and benefit coefficients
Bus 645 6e-6 0.010 0 0.001 3.4
of Energy Cells.
Bus 675 7e-6 0.015 0
Bus 671 8e-6 0.013 0
4.3. Simulation results

Fig. 6 shows a flowchart of electricity market clearing scheme in

the Energy Internet.
The entire problem is decomposed into two coupled subprob-
lems: (1) maximize the Nikaido–Isoda function and run AC power
flow, and (2) apply the relaxation algorithm and update the opti-
mal response function. The two subproblems are solved iteratively
until the difference of the optimal response function in (23) is
small enough between two adjacent iterations.
Initially, the power losses are assumed to be zero and plugged
into (17). After initialization, we maximize the results of (20) to
find yi subject to local/global constraints (10)–(17). Then we pass
yi to the EPRI’s Open-source Distribution System Simulator
(OpenDSS) V7.6.1, which is dedicated to the advanced analysis of
distribution systems. If the power flow results do not satisfy the
constraint (18), we add a small random vector DW to the last fea-
sible solution yi(Iter11) to rerun the power flow. If the power flow
results converge, we calculate the PLoss of the feasible solutions and

Fig. 6. Flowchart of clearing the electricity market.

update it in (17). Here, e is a small pre-defined value. The most up-

dated power loss PLoss is fed back to the first subproblem as long as
the change of power loss DPLoss is noticeable (i.e., DPLoss P e). Fig. 7
Fig. 4. Hourly wind and solar power output. shows the total daily power losses in the first subproblem. Since
348 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350

multiple power loss curves exhibit a similar tendency over the

course of the first iterative subproblem, Fig. 7 only presents one
solving process for the total power losses at a particular step k.
The total daily power loss changes to 816.5405 kW after the first
iteration. Gradually, the solution obtained from (20) in the first
subproblem converges to a new steady state as DPLoss becomes
smaller and smaller. After the first 5 iterations, DPLoss becomes
smaller than the user-defined threshold 0.01. Therefore, the
approximate daily power loss is finally found to be 861.4114 kW,
demonstrating the effectiveness of solving the first subproblem.
Then the intermediate solution found in the first subproblem
passes to the second subproblem to run the relaxation algorithm.
In the second subproblem, the relaxation algorithm in (24) is
performed to update the solution space xk. Fig. 8 shows the Nikai-
Fig. 8. Nikaido–Isoda function values in the second subproblem.
do–Isoda function values calculated by (23) over 17 iterations. Ini-
tially, the Nikaido–Isoda function value is 9267.22. Then we apply
the relaxation algorithm in (24) and (27) to update xk. As the Nika-
ido–Isoda values reach zero, no player can unilaterally improve its
own pay-off function. On the 17th iteration, the Nikaido–Isoda
function value decreases to 0.00243. Therefore, an (approximate)
equilibrium solution is found to clear the electricity market subject
to the local and global constraints in (10)–(18).
Fig. 9 shows the convergence of the pay-off function values for
the players at the five selected nodes. Table 3 lists the pay-off func-
tion values iteratively achieved by different types of players using
relaxation algorithms. By iteratively updating the optimal response
function in (24), the pay-off function values of all players are grad-
ually converging to an (approximately) equilibrium point. On the
17th iteration, no player can further increase its payoffs by unilat-
erally changing its strategy space. The equilibrium payoffs are
found to be [994.05, 1725.8, 3331.5, 1529.3, 8810.8] for play- Fig. 9. Pay-off function values in the second iterative loop.
ers at node 645, 671, 675, 634, and 650, respectively.
Given the equilibrium solution found by the coupled iterative
Table 3
loops, we can clear the electricity market price for a future residen- Convergence of pay-off function values in case studies.
tial distribution system with multiple residential customers while
Step k Objective values $
satisfying a number of local and global constraints.
At Nash equilibrium, no player (Energy Cell and Utility Cell) can Type 1 energy cells Type 2 Utility cell
improve its pay-off function by unilaterally changing its own strat- Bus 645 Bus 671 Bus 675 Bus 634 Bus 650
egy given that the strategies of the other players are fixed. Table 4 1 165.04 3403.7 4667.2 1794.1 4007.9
summarizes the final results at Nash equilibrium point, in terms of 2 420.09 2632.7 3642.7 1684.2 6514.7
cost, revenue, and profit. At Nash equilibrium point, the uniform 3 707.89 2226.7 3342.3 1605.4 7673.6
electricity price is [0.201204662, 0.20119938, 0.201194855, 4 834.51 2020.9 3162.5 1580.9 8179.4
5 908.73 1901.0 3157.0 1556.9 8444.7
0.201173353, 0.201161895, 0.201179632, 0.201244929,
: : : : : :
0.201437683, 0.204521982, 0.206846024, 0.20807764, : : : : : :
0.208297008, 0.209039584, 0.228770446, 0.208714936, 15 993.86 1726.5 3328.6 1529.5 8808.9
0.207335239, 0.204160865, 0.201660849, 0.20130047, 16 993.96 1726.0 3330.3 1529.4 8810.1
17 994.05 1725.8 3331.5 1529.3 8810.8
0.201322066, 0.202111964, 0.204705095, 0.208488978,
0.212699628] $/kW h over a period of 24 h. Fig. 10 shows the uni-
form electricity price and the total load demand at Nash equilib-
rium point, where the retail electrify price is inversely Table 4
Comparison on day-ahead market operation of cells at Nash equilibrium point.

Cell type Bus number Cost ($) Revenue ($) Profit ($)
Energy Cell #1 645 1565.3 2559.4 994.05
675 4068.5 2342.7 1725.8
671 5549.5 2218.0 3331.5
Energy Cell #2 634 1529.3 0 1529.3
Energy Cell #3 All others Fixed C 0 C

proportional to the total load demand. Fig. 11 shows the hourly

Fig. 7. Total power losses over 6 iterations in the first subproblem.
power output of Utility Cell and the aggregate renewable-based
power output of all energy cells at Nash equilibrium. As shown
in Fig. 11, the energy cells produce much more renewable-based
power (wind and solar) at approximately 10:00 am–4:00 pm. At
that time, since the production cost of the renewable energy (i.e.,
wind and solar) is assumed to be negligible, the Utility Cell sells
 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350
Fig. 10. Retail electricity price and total load demand at Nash equilibrium point.
Fig. 11. Hourly power output of Utility Cell and renewable-based power output of
energy cells at Nash equilibrium.
Fig. 12. Hourly power output of energy cells at Nash equilibrium point.
less electricity due to a larger portion of ‘‘free’’ wind and solar
power. Fig. 12 shows the hourly power output of energy cells at
buses 645, 671, 634, and 650 at Nash equilibrium point. The posi-
tive/negative power output corresponds to the processes of selling/
buying electricity, respectively. For example, the energy cell at bus
645 serves as a seller by providing excess self-generated power
back to utility grid or neighborhood.
5. Conclusions
Smart Grid is drawing increased attention, and is reshaping the
traditional view of power systems. There is a great need to find a
promising solution to accommodate the high penetration of
intermittent renewable energy resources and the emerging Smart
Grid technologies. This paper proposed a retail electricity market
framework with high penetration of distributed electricity suppli-
ers. The envisioned framework makes it possible for power distri-
bution system operators and residential customers (Energy Cells)
349
to efficiently integrate and dispatch a wide range of renewable en-
ergy resources. The residential customers (Energy Cell) play a more
important role than ever before by locally operating and managing
DGs, DESs, and/or loads. Moreover, the utility grid makes profits by
selling electricity and providing ancillary services to residential
customers. A numerical case study was conducted to demonstrate
the proposed electricity market clearing schemes. Some game-the-
oretic methodologies are adopted to find a Nash equilibrium point
to clear the deregulated electricity market price subject to both lo-
cal and global constraints.
Due to the limited space of this paper, some considerations re-
quired for real-world implementation have been left for future
research.
 Some Energy Cells may change their bidding strategies and net
power output in real-time market with a certain penalty.
 Some Energy Cells may cooperate with neighbors to form a coa-
lition. A number of cooperative Energy Cells may complicate the
market clearing schemes. Some advanced game theoretical
algorithms (e.g., auction theory and matching game) are needed
to examine the interactions among multiple coalitions in the
proposed electricity market.
 We will further identify multidisciplinary complexities (e.g.,
social impact, policy, economics, and environmental factors).
 The dispatchable residential load (e.g., plug-in electric vehicle
[34]) may bring more spatial and temporal uncertainty.
 The Energy Internet can be viewed as a complex cyber-physical
system. There is a great need to look into the interactions
between a reliable, two-way communication network and a
future electric power distribution network, which may affect
the electricity market clearing process.
 As the number of Energy Cells increases to more than hundreds
of thousands, it is rather intuitive that the current state-of-the-
art centralized information processing architecture will no
longer be sustainable under such data-explosion.
Acknowledgments
This work was supported in part by the Future Renewable
Electric Energy Delivery and Management (FREEDM) Systems
Center at North Carolina State University, a National Science
Foundation Engineering Research Center, under Award # EEC-
0812121. This work was also supported by New Faculty Start-up
Funds at University of Michigan-Dearborn.
The authors would like to take this opportunity to thank
anonymous reviewers whose comments/suggestions helped
improve and clarify this manuscript.
References
[1] Farmer RG, Allen EH. Power system dynamic performance advancement from
history of North American blackouts. In: Proc. IEEE 2006 Power System
Conference and Exposition, Atlanta, Oct. 29–Nov. 1; 2006.
[2] Su W, Wang J. Energy management systems in microgrid operations. Electric J
2012;25(8):45–60.
[3] United States Energy Information Administration (EIA), Annual Energy Outlook
2012 (AEO2012), June 2012. <http://www.eia.gov/forecasts/aeo/er/pdf/
0383er(2012).pdf>.
[4] Huang AQ, Crow ML, Heydt GT, Zheng JP, Dale SJ. The future renewable electric
energy delivery and management (FREEDM) system: the energy internet. Proc
IEEE 2011;99(1):133–48.
[5] Su W, Huang AQ. Proposing a electricity market framework for the Energy
Internet. In: Proc. 2013 IEEE Power and Energy Society General Meeting,
Vancouver, British Columbia, Canada, July 21–25; 2013.
[6] Huang AQ. Renewable Energy System Research and Education at the NSF
FREEDM Systems Center. In: Proc. 2009 IEEE Power and Energy Society General
Meeting, Calgary, AB, Canada, July 26–30; 2009.
[7] Su W, Rahimi-Eichi H, Zeng W, Chow M-Y. A survey on the electrification of
transportation in a smart grid environment. IEEE Trans Indus Inform
2012;8(1):1–10.
350 W. Su, A.Q. Huang / Applied Energy 119 (2014) 341–350
[8] Wang J, Conejo AJ, Wang C, Yan J. Smart grids, renewable energy integration,
and climate change mitigation – future electric energy systems. Appl Energy
2012;96(August):1–3.
[9] Zhou S, Grasso T, Niu G. Comparison of Market Designs, Market Oversight
Division Report, Public Utility Commission of Texas; 2003.
[10] Public Utility Regulatory Policies Act (PURPA) of 1978. <http://www.usbr.gov/
power/legislation/purpa.pdf>.
[11] Obara S, Kawai M, Kawae O, Morizane Y. Operational planning of an
independent microgrid containing tidal power generators, SOFCs, and
photovoltaics. Appl Energy 2013;102(February):1343–57.
[12] Chen Y-H, Lu S-Y, Chang Y-R, Lee T-T, Hu M-C. Economic analysis and optimal
energy management models for microgrid systems: a case study in Taiwan.
Appl Energy 2013;103(March):145–54.
[13] Kroposki B, Lasseter R, Ise T, Morozumi S, Papatlianassiou S, Hatziargyriou N. A
look at microgrid technologies and testing projects from around the world:
making microgrids work. IEEE Power Energy Magaz 2008(May–June):40–53.
[14] Jacobsen HK, Zvingilaite E. Reducing the market impact of large shares of
intermittent energy in Denmark. Energy Pol 2010;38(7):3403–13.
[15] Woo CK, Horowitz I, Moore J, Pacheco A. the impact of wind generation on the
electricity spot-market price level and variance: the Texas experience. Energy
Pol 2011;39(7):3939–44.
[16] Sáenz de Miera G, Del Río González P, Vizcaíno I. Analysing the impact of
renewable electricity support schemes on power prices: the case of wind
electricity in Spain. Energy Pol 2008;36(9):3345–59.
[17] Sensfuß F, Ragwitz M, Genoese M. The merit-order effect: a detailed analysis of
the price effect of renewable electricity generation on spot market prices in
Germany. Energy Pol 2008;36(8):3086–94.
[18] Green R, Vasilakos N. Market behavior with large amounts of intermittent
generation. Energy Pol 2008;36(8):3211–20.
[19] Lee KH, Baldick R. Solving three-player games by the matrix approach with
application to an electric power market. IEEE Trans Power Syst
2003;18(4):1573–80.
[20] Siriruk P, Valenzuela J. Cournot equilibrium considering unit outages and fuel
cost uncertainty. IEEE Trans Power Syst 2011;26(2):747–54.
[21] Saad W, Han Z, Poor HV, Basar T. Game-theoretic methods for the smart grid:
An overview of microgrid systems, demand-side management, and smart grid
communications. IEEE Signal Process Magaz 2012;29(5):86–105.
[22] Weaver WW, Krein PT. Game-theoretic control of small-scale power systems.
IEEE Trans Power Deliv 2009;24(3):1560–7.
[23] Nanduri V, Kazemzadeh N. Economic impact assessment and operational
decision making in emission and transmission constrained electricity markets.
Appl Energy 2012;96(August):212–21.
[24] Molina JP, Zolezzi JM, Contreras J, Rudnick H, Reveco MJ. Nash-cournot
equilibria in hydrothermal electricity markets. IEEE Trans Power Syst
2011;26(3):1089–101.
[25] Aubin JP. Mathematical methods of game and economic theory. Amsterdam
(The Netherlands): Elsevier; 1980.
[26] Uryasev S, Rubinstein RY. On relaxation algorithms in computation of
noncooperative equilibria. IEEE Trans Automat Contr 1994;39(June):1263–7.
[27] Krawczyk JB, Uryasev S. Relaxation algorithms to find nash equilibria with
economic applications. Environ Model Assess 2000;5:63–73.
[28] Nikaido H, Isoda K. Note on cooperative convex games. Pacific J Math
1955;5(Supp. 1):807–15.
[29] Krawczyk JB, NIRA-3. An improved MATLAB package for finding nash
equilibria in infinite Games, in Working Paper, December 2006. <http://
ideas.repec.org/p/pra/mprapa/1119.html>.
[30] Contreras J, Klusch M, Krawczyk JB. Numerical solutions to nash-cournot
equilibria in coupled constrainted electricity markets. In: IEEE Trans. Power
Systems, vol. 19, no. 1, February 2004.
[31] NREL, Eastern Wind Integration and Transmission Study. <www.nrel.gov/
wind/systemsintegration/ewits.html>.
[32] NREL, System Advisor Model (SAM); 2012. <https://sam.nrel.gov/>.
[33] Distribution Test Feeders, IEEE Power and Energy Society. <http://
ewh.ieee.org/soc/pes/dsacom/testfeeders/index.html>.
[34] Su W, Chow M-Y. Computational intelligence-based energy management for a
large-scale PHEV/PEV enabled municipal parking deck. Appl Energy
2012;96(August):171–82.