30-Smart Grid Simulations and Their Supporting Implementation Methods

Smart Grid Simulations and
Their Supporting
Implementation Methods
This paper reviews both state-of-the-art and future implementation strategies for
modeling and simulation of smart grid system.
By A r a n ya C h a k r a bort t y, Senior Member IEEE, and A n ja n B ose , Fellow IEEE
ABSTRACT | In this tutorial we present the state-of-the-art I. I N T RODUC T ION

as well as new methods for simulating various planning,
Simulation is defined as the process of using a mathematical
operation, stability, reliability, and economic models of electric
model to study the behavior and performance of a physical
power systems. The discussion is driven by both first-principle
system. When it comes to understanding the behavior of an
models and empirical models. First-principle models result
enormous, complex, multitime-scale system such as an elec-
from the fundamental physics and engineering principles that
tric power system, simulations are the primary tool that one
govern the behavior of various components of a grid. Empirical
has to rely on. Accurate simulations can guide the operation,
models, on the other hand, are models that result from
planning, and control of power systems at all levels—genera-
statistics and data analysis. We overview a wide spectrum of
tion, transmission, distribution, consumers and markets [1]–
applications starting from planning models with a time-scale
[3]. They are essential for analysis and evaluation of power
of simulation in years to real-time models where the time-
system operations. They improve system planning and design.
scale can be in the order of milliseconds. We present a list of
They help in recognizing and in mitigating potential hidden
simulation software popularly used by the power engineering
problems in the grid, avoid unforeseen errors, and prevent
research community across the world. The increasingly
system interruption, and can determine underutilization
important roles of power electronics, communication and
of system resources. They accelerate engineer and operator
computing, model aggregation, hybrid simulation, faster-
training, reduce design and commissioning time, and help
than-real-time simulation, and co-simulation in emulating the
power system operators in avoiding inadvertent plant outages
daily operation of a grid are enumerated. The importance of
caused by human error and equipment overload. Simulation
research testbeds for testing, verification and validation of
is also one of the most effective ways to teach power engineer-
complex grid models at various temporal and spatial scales is
ing to undergraduate and graduate students.
also highlighted. The overall goal is to provide a vision on how
Currently, power systems in different parts of the world
simulations and their supporting implementation methods can
are undergoing revolutionary changes in both architecture
help us in understanding the evolving behavior of tomorrow's
and operating principles [4]–[6]. The modern grid requires
power networks as a truly intelligent cyber-physical system.
a tight integration of physical assets with cyber informa-
tion to allow efficient transportation of data and control
KEYWORDS | Cyber-physical systems; electric markets;
signals, and also to develop an open and real-time market.
modeling; operations; planning; power electronics; simulations;
The envisioned digital grid of tomorrow must facilitate the
stability
open competition and innovation needed to accelerate the
Manuscript received May 1, 2017; revised July 12, 2017; accepted August 3, 2017. adoption of renewables while satisfying various challenges
Date of publication September 27, 2017; date of current version October 18, 2017.
(Corresponding author: Aranya Chakrabortty.)
related to grid stability, markets, and dynamics. It must
A. Chakrabortty is with the School of Electrical and Computer Engineering, North also allow for active participation of customers in day-to-
Carolina State University, Raleigh, NC 27606 USA (e-mail: achakra2@ncsu.edu).
A. Bose is with the School of Electrical and Computer Engineering, Washington State
day grid operations through data-driven programs such as
University, Pullman, WA 99164 USA (e-mail: bose@wsu.edu). demand response. None of these changes, however, can be
implemented on the grid unless we have a reliable simula-
Digital Object Identifier: 10.1109/JPROC.2017.2737635 tion capability. Simulations will be essential to understand
0018-9219 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
2220 Proceedings of the IEEE | Vol. 105, No. 11, November 2017
Chakrabor t t y and Bose: Smar t Grid Simulations and Their Suppor ting Implementation Methods
all complex interactions between cyber, physical, and social • steady-state security assessment;
layers of tomorrow’s grid. • N-1, N-1-1 contingency analysis;
Motivated by this challenge, in this paper we review a list of • fault analysis, fault location analysis;
methods that are currently used as well as new methods that will • volt/VAr optimization;
potentially be used for simulating various planning, operation, sta- • symmetrical components and unbalanced faults;
bility, reliability, and economic models of power systems. We base • induction motor analysis;
our presentation on simulations of both physical models and heu- • device coordination and selectivity;
ristic data-based models. Physical models describe the behavior of • reliability assessment;
physical components in the grid such as synchronous generators, • AMR/AMI integration;
induction motors, transmission lines, control equipment, loads, • power system markets.
transformers, relays and circuit breakers. Heuristic models, on 2) Long-term planning (in years):
the other hand, are employed to describe uncertainties as well as • production costing;
the behavior of many other engineered and human-driven infra- • model validation of generators;
structures underlying the grid [7], [8]. A list of simulation software • reliability evaluation.
is presented. New models arising from nonconventional genera- 3) Scheduling (in hours):
tion sources and loads, with an emphasis on power electronics, • economic dispatch;
are presented. The need for larger scale simulations in order to • optimal power flow (OPF);
transition from local monitoring to system-wide monitoring is • unit commitment;
highlighted, together with a discussion on how new methods of • hydro-thermal coordination;
measurement-based model aggregation can help these wide-area • preventive security constrained OPF;
simulations. The increasingly important roles of co-simulation, • weather modeling and predictions.
faster than real-time simulations, predictive simulations, and the 4) Slow dynamic simulations (in minutes):
associated challenges in communication and computing on daily • voltage stability;
operation of the grid are also presented. • loadability limit calculations.
The rest of the paper is organized as follows. Section II 5) Electro-mechanical simulations (in seconds):
reviews fundamental models for power system simulations. • automatic generation control (AGC);
Section III summarizes the current approaches for simulation. • operator training simulator (OTS).
Section IV presents the challenges in computational complex-
6) Fast electro-mechanical simulations (in milliseconds):
ity, while Section V describes the impact of power electronics
• modal analysis of oscillations;
in conventional simulation models. Section VI describes the
• oscillation damping control;
need for co-simulation of transmission and distribution system
• PSS, FACTS controller tuning;
models. Section VII describes co-simulation of power grid with
• transient stability;
other critical infrastructures. Section VIII presents cyber-phys-
• synchronous machine transient analysis, including
ical modeling challenges, while Section IX covers faster than
harmonics and unbalanced short-circuit faults.
real-time simulations. Section X discusses the future of remote
7) Electro-magnetic simulations (in microseconds):
simulation testbeds. Section X concludes the paper.
• electro-magnetic transient program (EMTP);
• geomagnetic induced currents (GIC) calculations;
II. SI MU L AT ION MODEL S A N D • arc flash hazard analysis.
A PPL IC AT IONS
Each of these applications comes with its own first-princi-
We start our discussion with a comprehensive list of appli- ple physics-based models that enable their simulations. These
cations that power engineers need to simulate for everyday models are mostly nonlinear and described by a large set of
operations of the grid. Many of these applications lead to algebraic and differential equations whose evolutions vary
optimization problems whose solutions decide the setpoints over different time-scales as noted in the list itself. Most of
for decision variables that are needed to operate the grid in a these models exhibit complex nonlinearities such as satura-
stable, efficient, economic, and reliable way. These optimization, deadband, if–then logic rules, and nonsmooth behavior
tions need to be executed over a large number of variables [9]–[11]. We start our discussion by recalling a selected set
while following different constraints arising from the physical of models that represents the various fundamental building
behavior of the grid, thereby necessitating simulations. The blocks of a power system. These models are often used for
following list captures the most important applications, but,
running simulations on academic examples of a grid, mostly
of course, is not complete. We classify these applications with
at generation and transmission levels. Depending on the
respect to their “time step” of simulation as follows.
accuracy and resolution of the simulations that one may be
1) Steady-state simulations: interested in, these models by themselves can quickly become
• power flow; quite complicated. To avoid this difficulty, we limit this sec-
• state estimation; tion to only the most basic mathematical equations that
Vol. 105, No. 11, November 2017 | Proceedings of the IEEE 2221
capture the main functionalities of the system, and point the and the power consumed by the loads. They will be
reader to appropriate references for more details. provided shortly in Section II-C where we discuss
transmission line models.
A. Synchronous Generator Models Here, iis the generator index, i = 1,…, n. Equations (1)
Consider a power system network with nsynchronous gen- and (2) are referred to as the swing equations, (3) and (4)
erators. Synchronous generators are modeled by a combination as the excitation equations, and (5) and (6) as the stator
of Gauss’ law, Newton’s second law of angular motion, Kirchoff’s current balance equations. As shown in Fig. 1, the termi-
law, Ohm’s law, and Faraday’s law of electro-magnetic induc- nal voltage at the i  thgenerator bus is denoted as Ṽ i = Vi∠
θ i 
tion. A suite of such models with varying levels of complexity where Vi is the magnitude (volts) and θ i  is the phase (radi-
can be found in [11]. A convenient model that is often used for ans). The symbols δ i  , ω i  , E′ qi, Efdi respectively denote the
transient stability analysis, small-signal oscillation analysis, as generator phase angle (radians), rotor velocity (rad/sec),
well as for design and simulation of generator control systems the quadrature-axis internal emf (volts), and field excitation
such as automatic voltage regulators (AVR) and power system voltage (volts); ω s  is the synchronous frequency, which is
stabilizers (PSS) is the so-called flux-decay model. This model equal to 120πradian/s for a 60-Hz power system such as the
assumes that the time constants of the d- and q-axis flux are fast North American grid. Miis the generator inertia (seconds),
enough to neglect their dynamics, that the rotor frequency is Di is the generator damping, Pmi is the mechanical power
around the normalized constant synchronous speed, and that input from the i  th turbine (Megawatt). T′ doi and TAi are the
the amortisseur effects are negligible. The dynamic circuit of excitation time constants (seconds); xdi, x′ di, and xqi are the
this model is shown in Fig. 1. Using the phasor variables used direct-axis salient reactance, direct-axis transient reactance,
in this figure, the model is described by the set of differential- and quadrature-axis salient reactance (all in ohms), respec-
algebraic equations [12]. tively. Vref,iis the setpoint value for the terminal bus voltage
(volts) of the generator, and KAiis a proportional controller
1) Differential Equations:
that regulates the voltage Vi to this setpoint in steady state.
d˙i =
ωi    − ωs  (1) This controller is referred to as a automatic voltage regula-
tor (AVR). Additional control signals can also be added to
Mi w ˙i =   − E′ qi Iqi
Pmi   − (xqi  − x′ di) Idi
Iqi
the RHS of (4) through the field voltage Efdi. These control-
lers are referred to as power system stabilizers (PSS), which
− Di (ω
i    − ωs  ) (2)
takes feedback from the generator speed, and passes it
T′ doi E qi̇ = − E′ qi  − (xdi  − x′ di) Idi
  + Efdi (3) through a lead-lag controller for producing damping effects
on the oscillations in the power flow after disturbances. A
T
Ai ̇ = − Efdi
E fdi   + KAi ( Vref,i
  − Vi) (4) more detailed description of the various parameters of this
model can be found in [5] and [12]. Besides the swing and
for i = 1,…, n. excitation equations, the generator circuit model can have
2) Stator Algebraic Equations: Applying Ohm’s law to the many additional state variables arising from ac and dc exci-
stator circuit shown in Fig. 1, after a few calculations tation, flux linkage, subtransient effects, washout filters,
one obtains the following pair of algebraic equations: governor equations, automatic generator controllers (AGC),
connections to other control devices such as FACTS and
Vi sin (δi    − θi  ) + rsi Idi
  − xqi Iqi
= 0 (5) HVDC, and so on.
E′ qi  − Vi sin (δi    − θi  ) − rsi Iqi
  − x′ di Idi
= 0, (6)
B. Models of Renewable Energy Sources
for i = 1, …, n. The modern day grid is seeing significant penetration of
3) Network Equations: These equations couple the wind and solar power, both of which come with interfac-
active and reactive power produced by the generator
ing power electronic converters. Fig. 2 shows a schematic
to the power flowing through the transmission lines,
diagram on how these nonconventional distributed energy
resources (DERs) are typically integrated with the main
grid. We next summarize the state-space simulation models
of these DERs to understand how their dynamic signal flows
couple with the grid variables.
1) Wind Generator Models: A wind power system is com-
prised of wind turbines, their associated control systems
such as pitch and yaw controllers, doubly fed induction
generators (DFIG), and their associated internal voltage
Fig. 1. Dynamic circuit model of a synchronous generator. and current regulators. The turbine can be modeled by a
electromagnetic torque using rotor current control ena-

bles a wind power system to avoid any mismatch between
mechanical torque and electromagnetic torque, and
therefore, to avoid any rotor deceleration under network
frequency decline. Since inertia of a rotating machine is
equivalent to the time constant of its frequency dynamics,
this property of a wind power system is equivalent to hav-
ing zero rotor inertia [16]. Thus, the wind power model (7)
is often referred to as a low inertia model. The same holds
for a solar cell, whose model does not involve any rotational
dynamics [15]. Simulations of grid models integrated with
these DERs thus often show very sharp declines in the grid
frequency due to this low inertial effect. Several examples
can be found in [17] and [18]. Impacts of this effect on
electro-mechanical oscillations were shown via simula-
Fig. 2. Schematic diagram of grid model with DERs. tions in [19].
2) Converter Models: Three-phase pulsewidth-modulated
single-mass or double-mass unit with gearboxes that couple (PWM) converters are used for integrating DERs with the
its shaft with the rotating shaft of the DFIG. The turbine is grid. Assuming a direct-quadrature synchronous reference
driven by an aerodynamic torque, which is typically a cubic frame, the converter model can be written as [20]
function of the wind velocity. The DFIG is modeled through ⎧ L i ḋ = ωs  L iq   − ρ vdc
⎪
the dynamics of its stator and rotor currents (Amperes), ⎨ L
  
   i 
q ̇ = − ω   L
s d  i   + 
e q  −
ρ vdc
(8)
⎪
⎩ idc =
expressed in a rotating d-q reference frame. A linear model
eq iq / vdc
⎢⎥
for the DFIG is often written as [13]
⎡idr⎤ where id and iqare the d- and q-axis currents (Amps) flowing
{Ew = γ ( v1q

    i  ̇ = A(ωg  ) i + R v1  + B v2 iqr
into the converter, eq is the q-axis voltage (volts) at the bus
    i : =   . (7)
iqs   + j v1d ids
) , ids
connecting to this converter, vdc and idc are the converted
⎣iqs ⎦ dc voltage (volts) and current (Amps), ρ is the duty cycle,
and L is the filter-inductance (milli-Henry), and ωs  is the
The linearization is generally done at the operating point
synchronous frequency (rad/s). Generally, a PI controller is
that corresponds to the wind velocity that produces maxi-
used to regulate the output dc current of the converter to a
mum wind power output in steady state. A control scheme
desired setpoint.
called maximum power-point tracking control (MPPT) is
used to compute the setpoints for the DFIG currents corre- 3) Storage Models: Today’s power grid is also experienc-
sponding to this maximum power. The variable ω g  (rad/s) is ing penetration of electrical battery storage at various scales.
the rotor angular velocity whose dynamics are decided by the Typical models of batteries involve electro-chemical equa-
turbine mechanics [13], and skipped here for brevity. The tions that reflect their internal physical principles, coupled
states id r and iqrare the d- and q-axis rotor currents (Amps), with power flow balance that connect them to the grid.
ids and iqs are the d- and q-axis stator currents (Amps), v1 = Many variants of battery models with different levels of
[v1 d v1q
] Tis a column vector of the d- and q-axis stator voltage complexity exist in the literature [21]. A simple model can
magnitudes (volts) at the bus where the wind power plant be considered as [22]
is connected to the grid, v2 = [v2d ] T represents the d-
v2q
and q -axis rotor voltages (volts), and Ew =   + j Qwis the
Pw q̇  = / C, vdc = g(q)
idc (9)
total effective wind power injected into the grid. Explicit where q is the state of charge, C is the capacity of the bat-
mathematical expressions of the matrices A, Rand B can tery (microFarad), and g(q) is a monotonically increasing
be found in [14]. The positive constant γ is the number of function such that g(q) > 0 if q > 0, following from the
wind turbines in the plant, which when multiplied to the electrochemistry of the battery cell.
power output of each generator provides the total power
output of the wind farm. Typically, both the d- and q-axis
currents of the rotor need to be regulated to precomputed C. Transmission Line Models
setpoints based on MPPT. This is usually achieved by sim- The state variables of the DERs, converters, and stor-
ple PI control. age models are coupled to those of the main grid via power
The ability to control the mechanical torque of a wind balance across the transmission lines in the grid. A trans-
turbine applied to the generator using pitch control, and mission line is modeled using series resistance, series
inductance, shunt capacitance, and shunt conductance. 0 = Vi cos (δi    − θi  ) − Iqi Vi sin (δi    − θi  ) + QLi
Idi
A line is defined as a short-length line if its length is less m
than 50 miles. In this case, the capacitive effect is negli- −  ∑Vi Vk Yik sin (θik    − αik
  ). (13)
k=1
gible, and only the resistance and inductive reactance are
considered. Assuming balanced conditions, the line can For nongenerator buses, the network equations reduce to
be modeled by an equivalent circuit of a single phase with m
PLi  −  ∑Vi Vk Yik cos (θik    − αik
  ) = 0 (14)
a resistance. If the line is between 50 and 150 miles long, k=1
it is considered to have a medium length. The single-phase m
equivalent circuit is modeled as a π or Tcircuit. The shunt   −  ∑Vi Vk Yik sin (θik    − αik
QLi   ) = 0. (15)
capacitance is divided into two equal parts, each placed k=1
at the sending and receiving ends of the line. If the line
If any bus i has a wind power generator, or a solar
is more than 150 miles long, then it is classified as a long-
power generator, or battery storage connected to it, then
length line, whose model, unlike the lumped parameter
the active and reactive power generated (or consumed) by
models of the above two types, must consider parameters
them will be added on the LHS of (14) and (15), respec-
uniformly distributed along the line. Those models are
tively. Transmission line models also include transform-
described by partial differential equations. For a medium-
ers, circuit breakers, phase shifters, and shunt and series
length line, the power balance equations at any bus i can
compensators.
be written as [11]
m
 ∑Vi Vk (Gik
   cos     ) + Bik sin (θik  ) ) − PGi  + PLi = 0
θ  
(  
ik D. Load Models
k=1

The active and reactive power drawn by the j  th load can
Pi
m be modeled as [11]
 ∑Vi Vk (Gik
   sin     ) − Bik cos (θik  ) ) − QGi  + QLi = 0
θ  
(  
ik
k=1

aj V  2j    + bj Vj  + cj
PLj = (16)
Qi

QLj ej V  2j    + fj Vj  + gj
= (17)
where m is the total number of buses in the network,
Vi and θ i  are the voltage (volt) and phase angle (radian) of where (aj, bj, cj) and (ej, fj, gj)are load-specific constant coef-
the i  th bus, θik  =
θi    − θk  , PGi and PLi are the active power ficients of appropriate dimension, Vjis the voltage magni-
(Megawatt) generated and demanded at bus i, QGi and QLi tude at the load bus j, and the three terms in each equation
are the reactive power (MegaVAr) generated and demanded explicitly represent the contribution of each type of load,
at bus i, and Gik and Bikare the conductance and susceptance namely constant impedance, constant current, and con-
(mho) of the line joining buses iand k. For example, for the stant power load, respectively. Although constant imped-
flux-decay model (1)–(6), the network equations linking the ance loads greatly simplify the derivation of the final
generator state variables and the transmission line flows can state-space model of the entire power system network
be written accordingly as in an explicit closed-form, retaining this explicit form is
not always possible for the two other types of loads. Thus
0 = Vi sin (δi    − θi  ) + Iqi Vi cos (δi    − θi  ) + PLi
Idi engineers have to resort to software programs to numeri-
m
cally simulate the implicit model equations, linearize
−  ∑Vi Vk (Gik
cos (θik  ) + Bik sin (θik  )) (10) them at given equilibrium points, and compute a small-
k=1 signal state-space model. More complicated load models
such as frequency-dependent loads have been developed
0 =
Idi Vi cos (δi    − θi  ) − Iqi Vi sin (δi    − θi  ) + QLi
by the IEEE task force in [23]. Several papers such as [24]
m and [25] have also considered dynamic loads of the form
−  ∑Vi Vk (Gik
cos (θik  ) − Bik sin (θik  ) ) (11)
k=1 dP   + P =
Tp  ___ dV 
Ps (V)  + kp (V)  ___ (18)
dt dt
for i = 1, …, n. Since one may write Gik = Yik
cos (αik
  )
and Bik =
Yik
sin (αik
  ), where Yik and α
ik
  are the magnitude where Tpis the load time constant, Ps ( ⋅ )is called the static
and phase of the element (i, k) in the admittance matrix, a load function, which is applicable in steady-state, and kp is
more popular representation of (10) and (11) is often writ- called the dynamic load function. In today’s grid operations
ten as [12] especially, loads play an extremely important role by virtue
of customer-side applications such as demand response.
0 = Vi sin (δi    − θi  ) + Iqi Vi cos (δi    − θi  ) + PLi
Idi Recent papers such as [26], for example, have developed
aggregated models for a population of air conditioning
m
−  ∑Vi Vk Yik cos (θik    − αik
  ) (12) loads including statistical information of the load popula-
k=1 tion, load heterogeneity, and their transient dynamics.
Various other types of static and dynamic load models such where λ   −1 models the mean length of the closure period.
as building loads and industrial machine loads can be found The cumulative distribution function (CDF) of this delay
in [27] and [28]. model can then be derived as
∫−∞  f (s) ds =  __21 [erf(___   __  )]

t
  __  )  + erf(___
μ t−μ
P(t) =
E. Models of Communication Systems √ 2  σ √ 2  σ
(p − 1 )
  e  (  2 λ  )
Power system operations today are becoming increas- 1 2σ  2+μλ
__
+  _____
2

ingly dependent on information and communication tech-
× e  −λt[erf(_____ )  + erf(  )].

nology (ICT), thereby necessitating accurate modeling of λ σ  2  + μ t − λ σ  2  − μ
  __   _______
__   (22)
communication networks. Communication is needed for √ 2  σ √ 2  σ
mainly three classes of applications: wide-area networks
(WAN) used for wide-area control of transmission sys- Random numbers arising from this CDF were used in
tems, local-area networks (LAN) used for smart metering, [29] and [30] for simulating delays in a distributed commu-
and home area networks (HAN) used for control of smart nication network for wide-area monitoring and control [31],
homes. Coming up with just one single representative [32]. More details of these distributed networks will be pre-
mathematical model, however, is impossible, if not imprac- sented in Section III when we describe the NASPInet.
tical. Researchers, therefore, often tend to couple actual The majority of the state-of-the-art communication
communication systems with offline physical models of models, however, is used for smaller scale simulations
the grid and emulate the interaction between the two in that involve information exchange over a single channel.
real-time, rather than using theoretical models. Examples The challenge is to translate these single-path models to
include the Internet (or experimental variants of it such multi-path, multi-hop, shared network models where back-
as Internet2) for long-distance data communication such ground traffic due to other parallelly running applications
as in wide-area monitoring and control of transmission over shared resources may pose serious limitations in laten-
networks, and wireless platforms such as Zigbee for short- cies. Recent references such as [33] and [34] have provided
range data communication such as in power distribution interesting theoretical tools such as Markov jump process,
systems in university campuses and home automation. One Poisson process, multi-fractal models, and Gaussian frac-
important deciding factor behind the choice of these mod- tional sum-difference models for modeling delay, packet
els is the communication delay that a message may experi- loss, queuing, routing, load balancing, and traffic patterns
ence while a critical contingency is happening in the grid. in such multi-channel communication networks.
For example, recent papers such as [29] and [30] have used
delay models arising in Internet-based wide-area commu- III. C U R R E N T M ET HODS OF
nication networks for testing the efficacy of just-in-time SI MU L AT IONS
wide-area control actions. Typically, these delays consist of
We next discuss how simulations are currently handled by
three components:
power system engineers and dispatchers. The discussion
1) minimum deterministic delay, say denoted by m; includes examples of several common software packages
2) Internet traffic delay with probability density func- used for simulations in today’s grid, and also hardware-in-
tion (PDF), say denoted by f 1  ; loop type simulations using real-time digital simulators
3) router processing delay with PDF, say denoted by f 2  . (RTDS) and Opal-RT simulation facilities [35], [36]. A list of
currently used open-source software includes UWPFLOW,
The PDF of the total delay at any time t is written in
TEFTS, MatPower, PST, InterPSS, GridLAB-D, OpenETran,
terms of these three components as
OpenPMU, rapid61850, etc., while commercial software
t
f(t) = p f2  (t) + (1 − p) ∫     f2  (u)f1  (t − u)du, t ≥ 0. (19) includes PowerWorld [37], PSS/E, PSLF, Aspen, E-Tap,
0 PSCAD, Plecs, Open-DSS, Sim-Power Systems, and so on.
Here, pis the probability of the open-period of the path Educational software programs include PSAT, which is based
with no Internet traffic. The router processing delay is on Matlab/Simulink, and DOME, based on Python and public-
approximated by a Gaussian density function domain C, C++, and Fortran libraries [38]. An excellent sur-
2
(t−μ)   vey on open-source software can be found in [39]. The primary
1___  e  − 
f2  (t) =  ____
_____  
(20)
2σ  2 applications that these software are used for are as follows.
σ √ 2π 
where μ > m. The Internet traffic delay is modeled by an

alternating renewal process with exponential closure period A. Power Flow Analysis
when the Internet traffic is on. The PDF of this delay is The power flow equations (12)–(15) can be expressed in
given by a compact form as
f1  (t) = λ e  −λt (21) f(x) = 0

(23)
⎢ ⎥
where algebraic power flow equations (12)–(15). The DAE can be
⎡ P 2 (x )  − PG2  + PD2 ⎤
written as
P3 (x )  − PG3  + PD3 dx  = f(x, y, p), x

 ___ ∈ ℝ n, y ∈ ℝ m, p ∈ ℝ q (25)
⋮
dt
Pm (x )  − PGm  + PDm 0 = g(x, y, p). (26)
f(x) =     
   , (24)
Q2 (x )  − QG2  + QD2
Q3 (x )  − QG3  + QD3 The function f(⋅) here should not be confused with
⋮ that in (24). The state vector x corresponds to the internal
⎣Qm (x )  − QGm  + QDm⎦ dynamic states of generators, exciters, governors, PSS,
dynamic loads, power electronic devices, and so on. The out-
x = col( V1 , V2 , …, Vm
, θ1  , θ2  , …, θm  ) , PGi and PLi are the put vector ycontains the network power-flow variables, e.g.,
active power generated and demanded at bus i, QGi and QLi bus voltage magnitudes and angles. The vector p describes
are the reactive power generated and demanded at bus i, machine parameters such as time-constants, inertia and
i = 1, …, m. In (23) we consider bus 1 to be the slack bus, damping, transmission line parameters such as line reac-
and hence its power balance equation is not included in tances and network topology, as well as operational param-
f( ⋅ ). The solution of these nonlinear equations is referred eters such as the status of shunt capacitor banks. The current
to as power flow. The majority of software packages these trend in general-purpose DAE simulation, for example in
days comes with their own sets of power flow solvers includ- Powerworld, PSCAD, and PSAT, follows a hierarchal system
ing accelerated Gauss-Seidel, Newton-Raphson, and per- modeling, which in addition to simplifying the conceptual
turbation-theory iterative techniques. Fast decoupled and and model-management tasks, also provides a good basis for
dc techniques are also often used [9]. The advantage of the incremental testing and validation of models and simulations.
Newton-Raphson method is that it guarantees fast conver- Traditional power system literature lists two broad modeling
gence as long as initial guess is close to the actual solution, approaches for simulations, namely, explicit modeling and
and also that it enhances the region of convergence. The implicit modeling. Explicit models for dynamical systems use
difficulty, however, is computing and inverting the Jacobian ordinary-differential equations (ODE) as the primary math-
matrix J(x  k) at every iteration. Factorizing a full matrix is ematical abstraction. The word “explicit” here refers to cau-
an order n  3operation, and hence extremely expensive. The sality, i.e., equations representing the physics of the system
order can be decreased, however, by exploiting the sparse are written so that known quantities such as inputs and state
structure of Y, which induces sparsity in Jas well. It has been variables are used to define unknown quantities. Languages
shown through examples in [9] that using sparse matrix associated with explicit modeling are block-oriented, such as
methods results in a computational order of approximately Simulink and Ptolemy II [41]. Implicit models, on the other
n  1.5, which can be a substantial saving when solving systems hand, directly use DAE as the primary mathematical abstrac-
with tens of thousands of buses. tion. DAE based models emphasize the relationships between
Power flow solution for distribution system models elements without regard to a particular causal ordering, and
is done quite differently than in transmission models. hence the name implicit. Example of such languages include
Transmission systems have meshed networks, while dis- Modelica [42], Hydra [43], and Sol [44].
tribution systems are mostly comprised of radial circuits.
Software such as Distribution System Simulator (DSS) these
days, however, can solve meshed distribution systems as well C. Economic Dispatch and Optimal Power Flow
because they are intended to be used for distribution com- Linear and Quadratic programming techniques are the
panies that may also have transmission or sub-transmission most common tools for both of these critical applications.
systems [40]. The circuit model employed can be either a Powerworld, for example, uses “Quadprog” and “Rglpk”
full multiphase model or a simplified positive-sequence packages for optimal power flow (OPF), both of which are
model. The power flow executes in numerous solution based on quadratic programming. A sparse solver for OPF
modes including the standard single snapshot mode, daily using moment-based convex relaxation has been proposed
mode, duty-cycle mode, Monte Carlo mode, and several in [45]. The classical OPF problem aims to minimize the
other modes where the load varies as a function of time. For function [9]
planning purposes the simulation time period is chosen as a
24-hour day, a month, or a year.  = ∑ ( ci1 P  2Gi    + ci2 PGi   + ci0) (27)
i∈
which is the sum of the quadratic costs accounting for the
B. Dynamic Simulations cost of active power generation at every generator bus (the
Transient stability simulations for large power sys- set of all generator buses being denoted as ) over the bus
tem models are based on the differential-algebraic (DAE) voltages, active and reactive power generations, subject to
model (1)–(5) for synchronous generators, and the different system-level constraints. In general, this problem
is nonconvex and difficult to solve. Recent results in [46] updates every few seconds, a time step of one second for the
and [47] have derived conditions on power system models simulation provides enough accuracy. Today, however, with
that guarantee zero duality-gap so that semi-definite pro- the increasing availability of phasor measurements which
gramming (SDP) can be used to retrieve the global opti- update at 30 Hz or more, the OTS simulation needs to be
mum solution to the OPF problem. Sufficient conditions for done at much smaller time steps. New OTS implementa-
global optimality have been presented in [48]. The results tions are appearing that use the transient stability formula-
have been extended to meshed transmission network topol- tion with around 10 ms time steps to be able to generate
ogies in [49]. phasor measurements in real time.
D. Fault Analysis G. Reliability Modeling and Simulation

Fault simulation packages simulate single-phase and Reliability modeling refers to the failure and repair cycle
three-phase bolted faults for interconnected power system of power system components. These are typically modeled
models and make provisions for complex fault impedance. by alternating renewal processes. One common method is
The Y-bus building algorithm is commonly implemented to the sequential Monte Carlo simulation with stochastic point
construct the admittance matrix [50], followed by sparse process modeling [55]. The application of probability mod-
matrix inverse computations. Several fast algorithms exist els to the evaluation of generation reliability allows for the
for this, for example, direct methods via LU factorization, integration of different unit sizes and types, the effects of
Recursive Green’s Function (RGF) algorithm, and more maintenance, the capacity of interconnections and other
advanced newer and faster methods such as Fast Inverse factors such as economic aspects. The commonly employed
using Nested Dissection (FIND). A list of simulation tools metrics for this purpose are called “loss of load probability”
for comprehensive fault analysis can be found in [51]. (LOLP), “loss of energy probability” (LOEP), and “frequency
and duration” (FAD). For detailed definitions of these met-
E. Load Frequency Control rics please see [56].
Load variations in a power system cause drifts in fre-
quency and voltage from their nominal values, resulting H. Simulation of Power Markets
in loss of generation due to tripping of transmission lines. Electricity markets are one of the major drivers and ena-
These drifts can be minimized, and kept within tolerable blers for power system operation [57], [58]. Many commer-
limits by load frequency control (LFC) [9], [11]. In LFC cial software packages such as Powerworld, GridLAB-D,
simulations users can change loads in selected areas in the and Power Systems Optimizer (PSO) with mixed-integer
grid, and proportional-integral (PI) control is automatically programming solvers such as GUROBI currently exist for
initiated based on the speed droop settings of the areas. simulating power markets. Several research institutions
The areas share the load by changing their generation lev- have also recently developed efficient market simula-
els based on their preset percentage contributions. With tors. Examples include the AMES (Agent-based Modeling
increasing penetration of renewable power LFC will be of Electricity Systems) wholesale power market testbed
required to compensate for not only short-term fluctuations at Iowa State University, JASA (Java Auction Simulator
in frequency but also long-term fluctuations that are con- API) developed by Ripple Software Ltd., a Python-based
ventionally dealt with by economic dispatch. Recent papers software called MiniPower developed in University of
such as [52] have proposed the use of storage as a solution Washington, USA, and several other market simulators
to this problem. Enhancing LFC algorithms to compensate created by Argonne National Lab. Detailed descriptions of
for uncertainties introduced by intermittent renewables is these packages can be found in [59]. The main applications
currently an ongoing area of active research. of a power market simulator include valuation of assets
including both physical and financial contracts, transmis-
F. Operator Training Simulator sion planning, policy analysis and market design, market
surveillance, generation scheduling and trading support,
The simulation engine for the Operator Training
risk analysis, modeling of demand response participation
Simulator (OTS) has to simulate the behavior of the power
in markets and ancillary services, and reliability assess-
system in real time so that the operator trainee gets the feel
ments [60]. The key components of a market simulator are
of a real power system [53], [54]. As mentioned before,
as follows:
such a simulation consists of solving differential equations
(the dynamic part) as well as the algebraic equations (the 1) Inputs: demand forecasts, generation and transmission
steady-state part). To solve this in real time the dynamic expansion, fuel prices, emission allowance prices;
part must be faster than real time so that the steady-state 2) Models: loads, demand response, generation and
part has enough time for its solution. Since the operator transmission models including storage and variable
usually observes the power system through SCADA, which generations, market rules;
3) Algorithms: maintenance scheduling, contingency taking these control actions can be in fractions of seconds.
analysis, co-optimization of energy and ancillary ser- Therefore, controlling latencies and data quality, and main-
vices, topology control, mixed-integer programming; taining high reliability of communication are extremely
4) Physical outputs: generation and reserve schedules, important for these applications. The media used for WAN
power flow, fuel usage, emissions, load curtailments; communications typically use longer range, high-power
5) Financial outputs: electricity prices, revenues, costs. radios or Ethernet IP-based solutions. Common options
include microwave and 900 MHz radio solutions, as well
In a typical power market simulator the user represents
as T1 lines, digital subscriber lines (DSL), broadband con-
power generation companies, submitting hourly generation
nections, fiber networks, and Ethernet radio. The North
bids into a virtual market. Simulated agents submit these
American Synchrophasor Initiative (NASPI), which is a
bids. Bids are submitted for 24-hour periods (day-ahead
collaborative effort between the U.S. Department of Energy
market) and entered into the model. The total of all energy
(DOE), the North American Electric Reliability Corporation
bids submitted by a player may not exceed the operational
(NERC), and various electric utilities, vendors, consultants,
capability of online generating units that she owns. Bid
federal and private researchers and academics, is currently
block sizes are determined by the player as part of her bid-
developing an industrial grade, secure, standardized, dis-
ding strategy. Bid prices for each unit must increase as a
tributed, and scalable data communications infrastructure
function of block number. The software is then run to deter-
called NASPI-net to support Synchrophasor applications
mine market clearing prices, generation levels for all units,
in North America. NASPI-net is based on an IP Multicast
and profits and losses for each market participant. Before
subscription-based model. In this model the network rout-
starting the simulation, each participant is provided with
ing elements are responsible for handling the subscription
a set of “market rules” that have to be followed by each requests from potential PMU data receivers as well as the
player. The objective of the simulation is for each player to actual optimal path computation, optimization, and recom-
maximize her profit by selling electricity into the market. putation and rerouting when network failures happen. An
The player with the highest profit wins the game. If no one excellent survey of NASPInet can be found in [64].
makes money, the player with the lowest loss is declared
the winner. Net profit is calculated based on energy sales 2) AMI Communication: Advanced metering infrastruc-
in each hour, real-time market clearing prices, incremen- ture (AMI) communications is a central topic for distribu-
tal production costs for each block of each generation unit, tion grids today. AMI communication is typically executed
energy production of each block, operating costs, and fixed via local area networks (LAN) that provide two-way com-
capital charges. Appropriate test systems such as the eight- munication paths between the consumer’s and the utility’s
zone ISO-New England system have also been developed to smart meters. The two predominant technologies involved
showcase the efficiency of these simulation methods [59]. are power line carrier (PLC) signals transmitted over the
Multi-objective optimization is applied to ensure the secu- utility’s distribution lines, and radio frequency (RF). PLCs
rity of the market operations [61]. use frequencies and modulation techniques that translate
into a range of available speed and bandwidth options,
including the 60 Hz power wave as well as superimposing
I. Simulation of Smart Grid Communications high-bandwidth carrier signals of other frequencies onto the
Incorporating communication models in smart grid sim- power line. They are more cost-effective in low-density areas
ulations is becoming imperative in the current state of the such as rural and suburban service territories where feed-
art. Real-time digital simulators (RTDS), for example, can ers serving the smart meters tend to be lengthy. Similarly,
seamlessly employ communication protocols such as IEC RF solutions include technologies that use an assortment of
61850 for substation automation applications, DNP3 and frequencies, covering a range of distances over a variety of
IEC 60870-5-104 for SCADA system applications, and IEEE network topologies. These types of LAN communications
C37.118 for Synchrophasor applications. Comprehensive can also be used for demand response, load-control, and dis-
surveys of various such communication protocols for tribution automation [65].
smart grids can be found in [62] and [63]. In this section
3) Home Area Networks: Home area networks (HAN) are
we highlight three of these communication models. The
used for communication between devices behind a smart
first one is used for emulating wide-area communication of
meter inside a consumer’s home. This includes messages to
Synchrophasor data in transmission-level systems, the sec-
present information via in-home displays, as well as mes-
ond one for smart metering in distribution-level systems,
sages that demand response devices can use for control-
and the third for home automation.
ling key appliances to manage energy usage and capacity
1) NASPI-Net: Wide-area monitoring and control of constraints. HAN is being conceptualized in response to
transmission grids require communication of large volumes mandates across the nation to improve energy efficiency,
of Synchrophasor measurements from substations spread and educate and empower consumers. ZigBee is a com-
across hundreds of miles. The time-scale associated with mon protocol that is used for HAN. ZigBee messages are
transported via RF messaging protocols in unlicensed RF a grid and the models of its computation and com-
spectrum shared by Blacktooth and Wi-Fi for communica- munication layers? The behavior of a power system
tion between devices inside a house [62]. is driven by physical laws while the behavior of com-
Despite the wide usage of these models, a long list of munication networks depend largely on random sto-
challenges still exists for modeling and simulation of today’s chastic processes, logic functions, if–then rules, and
rapidly evolving grid. These challenges need to be surpassed human decision-making. Hybrid system simulations
over the next decade through research and development to will become imperative to capture the continuous-
create robust, reliable, and cheap simulation platforms. The time state transitions of the physical models and their
following sections list five of those primary challenges. coupling with the discrete-event state transitions in
the cyber layer. Although combining discrete-time
and continuous-time behavior is not new, and has
I V. CH A L L ENGE 1: A DVA NCED
been around in the power system literature since the
MODEL I NG M ET HODS
inception of digital controls [67], the challenge lies
With millions of new devices penetrating the grid with every in accurate modeling of stochastic channels, vari-
passing day, simulation models for tomorrow’s grid are bound able routing delays, protocols for managing queuing
to become much more complex than just a set of DAEs as in delays, and so on, all of which are essential ingredi-
(25), (26). All of these new devices, that come with their own ents for modeling a cyber-physical system (CPS).
share of nonlinearities, will impact the dynamics, stability, 4) Similarly, how can we model and co-simulate com-
and most importantly the seconds to tens-of-seconds-time- plex interactions between power system models and
scale performance of the grid when critical disturbances hit. models of other engineered infrastructures such as
Developing simulation models that can capture such impacts transportation and social networks?
in a scalable way is a serious challenge. Heat flow models
in buildings, for example, are extremely high-dimensional, The key factor for these types of scientific experimenta-
complex, and nonlinear, often modeled by partial differen- tion is modeling. To properly answer these questions one
tial equation models, for which strong knowledge of fluid requires a holistic approach to modeling by, for example,
dynamics is required in developing reliable simulations. developing simulation models that are not only easier to
The increasing use of modern power electronic devices in build, reuse, and validate, but also those that fundamentally
the distribution grid, for example, is making the grid models characterize the environment in which they operate. We
nonlinear, and often nonsmooth due to switching actions. next list the challenges that currently stand in way of these
A study recently reported in [66], in fact, shows that power advanced modeling methods.
electronic converters can pose critical limits on the line cur-
rents and voltages beyond which the system experiences A. Structural Properties
a Hopf bifurcation leading to sudden vanishing of feasible
One of the desired properties of simulation models
equilibria. Taking all of these factors into account, a set of
is structural dynamicity. By definition, structural dyna-
relevant questions that naturally arise is as follows.
micity is the notion that one is not able to predetermine
1) With significant integration of wind, solar, storage, the number of structural configurations that a system
and electric vehicles, how can we develop holistic may take on. The concept is not new to power systems.
models that capture the dynamic impact of all of Contingencies, renewable integration, and interactions
these new energy sources and sinks on the conven- between prosumers are a few obvious examples. Upon such
tional grid? How can we identify important coupling events, the structural orientation of the system along with
parameters that can reveal the possibility of weak the underlying mathematics that model the physics of the
operating points leading to transient instability and system change in quantity, direction, etc. Another desired
voltage collapse? property is to capture the hybrid nature of the grid operat-
2) How can we develop robust numerical methods that ing as a CPS, consisting of a combination of continuous-
will enable us to simulate these models in a scalable time (physical domain) and discrete-time (cyber domain)
and modular fashion so that if tomorrow one more behaviors. Component interactions is the third required
wind plant or solar plant or load is installed in the property. Models that inherently and implicitly capture
system then it would not take an enormous amount the interaction between different components are critical
of computational effort to resimulate the new model? for holistic simulation approaches, especially when there is
3) Most importantly, as highlighted in Sections II-E and more than one explicit representation for a given compo-
III-I, with many of the modern-day grid applications nent. Another necessary property is component reflectivity
becoming increasingly dependent on communica- [68]. A model that is closely reflective of its components
tion of information across large geographical spans of helps in promoting modularity and lessens the burden of
the grid, how can we accurately simulate the cyber- deriving causality when the consequences of the interac-
physical interaction between the physical model of tions are not known a priori.
B. Scalability be highlighted in Section V. Developing accurate modeling

One big challenge is to enable a real-time simulation methods and devising procedures to validate these models
capability for power system models such as (25), (26) at are currently lagging behind.
extreme scales. Foundational work on taxonomy theory
for modeling and analysis of DAE power models has been D. Model Aggregation
done in theory [69], but its translation to simulation mod- Given the large size and extraordinary complexity of any
els is still missing. Software such as Modelica and Hydra,
realistic power system, deriving and simulating the dynamic
for example, needs to be exploited for modeling scalability
model for an entire network becomes extremely challeng-
through modularity, composition, correctness, implicit rep-
ing. Constructing approximate, aggregated, reduced-order
resentations and structural dynamics. All of these abstrac-
models using simplifying assumptions, therefore, becomes
tions then need to be brought under the umbrella of a com-
almost imperative in practice. Papers such as [76] have
mon modeling language and the front end of a compiler,
defined aggregation methods for simulation time, for gen-
followed by a library and language-level abstractions that
eration units, and for load demand units. The performance
support the needs of experimentation.
of the aggregated models is checked against detailed models
Scalability is also an issue for controller designs.
including binary effects such as minimum down-time, mini-
Conventional controllers such as Linear Quadratic Regulators
mum generation or demand side contracts. This is especially
(LQR) involve the computation of large matrix decomposi-
important for control designs such as model predictive con-
tions that can result in detrimental numerical inaccuracies
trol, where very large optimization problems need to be
without any guarantee of robustness. They also demand
solved online. The optimization in these cases usually has
every node in the network to share its state information with
to be simplified by using approximated power plant models,
every other node, resulting in an impractically large num-
aggregating several assets to single units, and limiting con-
ber of communication links. Traditionally, control theorists
troller foresight. The main question is whether fast genera-
have addressed the problem of controlling large-dimensional
tion units, curtailable renewable generation, demand-side
systems by imposing structure on controllers. The tradeoff,
management on the consumer side, and storage capacities
however, is that the resulting controllers are often agnostic
are still correctly represented in the approximate models so
of the natural coupling between the states, especially the
that one can guarantee optimal operation of the grid.
coupling between the closed-loop states, as many of these
Not just for reducing complexity of simulations, model
couplings were forcibly eliminated to facilitate the design
aggregation may also be necessary if one is purposely inter-
itself. Ideas on aggregate control [70], local control [71],
ested in simulating only a certain part of the grid, or, per-
and hierarchical control [72] have recently been proposed to
address this challenge. What designers are still lacking, how- haps a certain phenomenon that happens only over a certain
ever, is a tractable approach for constructing controllers that time-scale. For example, one may be interested in simulat-
can be simulated and implemented at the scale of hundreds ing only the low-frequency inter-area oscillations of a group
of thousand of buses. Model aggregation has been proposed of synchronous generators instead of the entire spectrum of
as one possible solution for this problem [73], [74]. their frequency response. Identifying coherent sub-groups
among this group of generators, aggregating the sub-groups
into an equivalent hypothetical generator, and analyzing the
C. Model Validation oscillation patterns of these equivalents become necessary
Another significant challenge is the validation of the in that situation. These days we often hear power system
models used in these simulations. This is particularly true for operators mentioning “Northern Washington” oscillating
the dynamic models of the generators and their associated against “Southern California” in response to various distur-
controls. After the large blackout in the U.S. west coast in bance events. The main question here is whether we can
1996, it became clear that the generator models used in the analytically construct dynamic models for these concep-
studies were not accurate enough. Since then the Western tual, aggregated generators representing Washington and
Electricity Coordinating Council (WECC) standard has California, which in reality are some hypothetical combi-
required updating of the model parameters through stand- nations of thousands of actual generators. One example for
ardized testing. Power systems in other parts of the world this motivating problem is the Pacific AC Intertie system
where stability is an issue have followed a similar approach. in the U.S. west coast, a five-machine dynamic equivalent
However, of-line testing is expensive and as PMUs have pro- mass-spring-damper model which has been widely used
liferated over the past decade, using PMU data during distur- in the literature [75], [77]. This model is shown in Fig. 3.
bances to update model parameters online has become more The main question is—how can we construct an explicit
common [75]. Several challenges still stand on the way. For dynamic model for this conceptual figure, and, preferably
example, new technologies such as renewable generation in real-time, use voltage, current or power flow signal meas-
sources and storage that require power-electronic grid inter- urements in order to establish a prototype for the inter-area
faces have completely new types of dynamic models, as will dynamics of the entire interconnection?
2) lack of parallel processing, meaning that many of the

optimization problems used for market emulation
are easily parallelizable but the software is not well-
designed to take advantage of that;
3) turnaround times are unacceptably long due to lack
of scalable algorithms.
Many simulators today rely on heuristic optimization,
which can be replaced by more sophisticated nonconvex
methods for solving mixed-integer programming problems.
For example, as pointed out in [82], simulation times can be
significantly improved if traditional game-theoretic market
models such as Nash-Cournot models and supply function
equilibrium (SFE) models are replaced by faster models
such as strategic production cost models (SPCM). This is
especially useful for simulating real-time markets. Examples
of these cost models for capital investments in transmission
networks were recently highlighted in [83]. Market opera-
Fig. 3. Model aggregation for simulation of the WECC power
tors are currently in need of software that can handle a sig-
system [75]. nificant increase in computational requirements, workflow
management, real-time data processing, and data storage.
The use of cloud computing by which users can create their
Recently, several papers such as [78] and [79] have own virtual hardware environments for managing a variety
addressed this question and derived a series of results on of market processes has recently been proposed in [60] as a
PMU measurement-based model reduction that combines solution to this problem. Market emulators such as pCloud-
aggregation theory with system identification. Several open Analytics using Amazon Web Services are currently being
questions still exist, however. For example, once a baseline developed. Cloud-based simulators, however, come with
model is constructed, one must study how it can be updated their own share of research challenges such as cyber secu-
at regular intervals using newer PMU data. Ideas from adap- rity, data privacy, and usage tracking and accounting.
tive learning and decomposition theory [80] can be useful
for that. How this updated model can be used to predict the
slow frequencies and corresponding damping factors also F. Load Forecasting
needs to be formalized and validated via realistic simula- As pointed out in Section III-H, accurate load forecasting
tions. Questions also exist on how the reduced-order model is a key requirement for power market simulations. Current
can predict the sensitivity of the power flow oscillations forecasting models are based on autoregressive models,
inside any area with respect to faults and wind power pene- Kalman filters, nonlinear least squares, and nonparametric
tration in any other area. If answered correctly, utilities can regression [84]. But the increasing use of demand-side par-
exploit this information from simulations of the aggregated ticipation in power system operations is going to make load
model and evaluate their dynamic coupling with neighbor- forecasting much more challenging. We need data-driven
ing companies, leading to more efficient resource planning. functional models that relate the value of load demands with
A significant amount of work still needs to be done in for- predictor variables such as setpoints for all the generation
malizing how different failure scenarios in the actual full- resources, setpoints for the duty cycles of power electronic
order grid model can be translated to the aggregated model, converters that connect renewable generators to the grid,
what kind of advanced signal processing and filtering need weather data such as temperature and humidity levels in a
to be applied to PMU data for accurate identification of the hot versus cold day, amount of PV generation and battery
aggregated model parameters [81], and how controllers storage units that may be owned by the customers, and so
designed based on the aggregated model can be mapped on. Recent results in the machine learning literature pro-
back to the original system for implementation. vide us with some positive hope of creating accurate models
by using nonlinear regression methods and reinforcement
E. Challenges in Power Market Simulations learning [85]. As pointed out correctly in [85], it is not easy
for a human operator to judge if the decision of changing
Three main challenges facing power market simulators
the load setpoint is correct or not. Operators are more used
today are:
to handling these types of load changes with fixed logic and
1) curse of one-scenario-at-a-time, meaning that simu- rules, and prefer models that are more transparent, where
lators today are not set up to generate, execute, and it is clear exactly which factors were used to make a par-
post-process multiple scenarios taken together; ticular prediction. Deep reinforcement learning can be a
perfect choice for that [126]. This is particularly important parameters. The trajectories arising from a perturbation Δ θ
for microgrid simulations as their control algorithms are is approximated as [89]
strongly dependent on the forecasted loads.
x(t) = f(t, θ + Δθ ) ≈ f(t, θ) + g(t, θ ) Δθ
(28)
G. Smart Loads where g( ⋅ ) is the trajectory sensitivity function. Since (28)
One challenge from the perspective of load modeling is affine in Δθ, parametric uncertainties can be easily mapped
is the emerging concept of smart loads. This is becoming through to bounds around the nominal trajectory. If the sen-
especially important for simulation of microgrid models. sitivity g( ⋅ )is large, then one may try to minimize the uncer-
For example, recently a new technology called “electric tainty by estimating the parameter values directly from meas-
springs” (ES) was proposed in [86] to enable a demand-side urements using nonlinear least squares, with the solution
control scheme at low cost, and in a way less intrusive to obtained via a Gauss–Newton process. PCM, on the other
the consumer compared to other demand-side response hand, uses Gaussian quadrature concepts to select appropriate
technologies such as energy storage and on–off control of points from the set of uncertain parameters to approximate
loads. ES is a class of power electronic devices, and can be the mapping between parameters and outputs of a simulation.
ideally regarded as a current-controlled ac voltage source. Grazing is used to determine the smallest changes in param-
Installed in series with other noncritical loads, ES and the eters that would cause a discrete event such as a protection
noncritical load together act as a smart load whose power operation. This information can help in assessing whether the
consumption can be boosted and reduced in a plug-and-play simulation model is robust to parametric uncertainties.
fashion, whenever needed. This smart load can then easily
participate in system regulation. However, a comprehen- V. C H A L L E NGE 2: ROL E OF P OW ER
sive microgrid model showing how these smart loads and EL EC T RON ICS
their associated control systems will interact with the vari- In Section II we introduced dynamic models of power electronic
ous DER models developed in Section II, for example, is still converter circuits needed for DER integration. Traditionally,
missing. Some preliminary results have been presented in the simulations of such converters are carried out indepen-
[87], triggering potential research ideas along this direction. dently of those for power systems. But with renewable penetra-
tion and significant intrusion of power electronic devices in dis-
H. Handling Model Uncertainties tribution grids, these simulations are gradually becoming more
interdependent and must be combined at the right scale [90].
Models are always an approximation. For example, as
Two important examples of power electronics are:
mentioned in Section II-A, the phasor-based model (1)–(4)
for synchronous generators is well-suited for transient sta- 1) FACTS (Flexible AC Transmission System);
bility analysis, but it ignores electro-magnetic transient 2) HVDC (High Voltage DC).
phenomena. Even the load model (16), (17) can become a Both can assist power system operators during a con-
gross approximation depending on the desired fidelity of a tingency of rescheduling power flows to avoid overloading
simulation. It is referred to as a ZIP (constant impedance- of ac transmission lines [91], [92]. Depending on applica-
current-power) model. It does not capture the delayed voltage tions, both detailed switching models and simplified average
recovery associated with induction motor loads in distribution models may be used for simulating FACTS and HVDC con-
systems. Thus, it may be completely invalid if one intends to verters. For transient stability studies, for example, it is ade-
simulate those types of loads. This is a daunting challenge for quate to represent the converters by their average models
the modern grid, where load models are further complicated in which the valve switchings are ignored using duty-cycle
by new protocols such as demand response, direct load con- based switching functions or approximated phase-locked
trol, vehicle-to-grid integration, and various other factors by loop models, thereby ignoring the ac and dc harmonics [93].
which load variations are shaped through customer participa- For other applications such approximations may not be
tion. Using approximate models may not provide a very good valid, and detailed switching models may have to be used.
representation of the reality in these scenarios. Modeling and Simulation of switching models, however, can sometimes
accounting for uncertainties thus becomes imperative. become problematic for a number of reasons. The convert-
A number of approaches is currently used for assess- ers operate by using switches to change the configuration of
ing and reducing the impact of uncertainty in power sys- a network at frequencies up to several megahertz, and the
tem simulations, for example, trajectory sensitivity method, time constants of the network are usually one or more orders
probabilistic collocation method (PCM), grazing. Two or magnitude larger than the switching period. Each switch-
excellent surveys for these three approaches can be found ing event is a discontinuity. Standard simulators can run for
in [88] and [89]. The state trajectory is expressed as a flow hundreds of switching cycles, creating problems for the sim-
function x(t) = f(t, θ ), where θ is a vector of the model ulation software that may not have an explicit mechanism
for handling such discontinuities. This can result in long

simulation times due to reduced integration time-steps in
the neighborhood of the discontinuity, lack of convergence,
and erroneous outputs [94]. With the advancement in pro-
gramming languages in the late 1990s, modeling languages
such as Dymola were proposed to resolve this problem by
using object-oriented programming. Today, software such
as PLECS, PSIM, PSpice, OpenModelica, PSCAD, VissiM,
SABER, COMSOL, Matlab, and LTSpice are all widely used
for large-scale converter simulations.
Another factor motivating the interest for accommodat-
ing power electronics in power system simulations is the
envisioned replacement of conventional low-frequency mag-
netic transformers in near future by power-electronic trans-
formers made out of the solid-state technology. These trans- Fig. 5. Power electronic converter models for energy storage:
formers consist of a front-end rectifier stage, which converts (a) Rectifier circuit for AC battery average model and (b) Buck-Boost
circuit for DC battery average model.
high voltage ac to high voltage dc, a dual-active bridge (DAB)
stage, which converts high dc voltage to a low dc voltage to
be used for dc distribution segment, and a voltage-source components. PV panels are directly connected with a boost
inverter, which converts low dc voltage to low single-phase converter to amplify the panel voltage to the dc bus voltage.
ac voltage to be used for ac distribution segments. Since a Based on voltage level of the panels, other dc–dc converters
typical microgrid may contain several hundreds of these con- can also be used for this interface. A similar interface with
verters, a common practice is to use averaged models instead bidirectional energy flow can be used to control the battery
of detailed models, especially for transient stability studies. charging and discharging for power and energy management
These models are most often described by nonlinear dynam- of the system. The charging voltage of a battery is typically
ics that are valid only in specified “safe zones of operation” lower than the dc bus voltage. Thus, a good choice for this
[66]. This is similar to the concept of reachable zones in DFIG- interface circuit is a buck converter for the dc energy cell,
based wind turbine models, which are often used for testing and an inverter-coupled buck converter for the ac energy cell.
their voltage ride-through capabilities [95]. High-fidelity sim- However, to allow flexibility in choosing the voltage rating
ulations are required for revealing these safe zones for differ- of the battery, a bidirectional buck-boost converter is used
ent operating conditions and network topologies. for dc batteries, and an inverter coupled with a buck-boost
A solid-state transformer (SST) contains both ac and dc converter is used for ac batteries. The circuits for ac and dc
terminals that may be connected to wind, solar, battery stor- energy storage are shown in Fig. 5(a) and (b), respectively.
age, and loads. Fig. 4(a) shows the interface circuit for ac Recent examples in [66] show that when these circuits
components. A boost converter is coupled with the rectifier are connected to a radial distribution grid, the small-signal
circuit to convert the lower dc voltage of solar generators to model of the system exhibits a wide variation in time-scales
lower ac voltage. Fig. 4(b) shows the interface circuit for dc ranging from micro-second dynamics arising from switch-
ing to tens of seconds of dynamics arising from the iner-
tial effects of the DFIGs. Thus, multi-time-scale numerical
methods that can successfully capture such large spectra of
eigenvalues need to be developed as more of these power
electronic devices get integrated to the grid.
Yet another important factor for simulating converter
models is harmonics. For example, it was recently shown in
[66] that the effective duty cycle ratio that enters the dynam-
ics of the rectifier stage model of a converter can be written as
dα (t) =
d1 (t) cos (θ(t)) + d2 (t) sin (θ(t)) (29)
q˙ (t) =
ωs  (t) (30)
where d1 and d2are the d and q-axis components of the duty
cycle, and θ represents the dynamics of the phase-locked
Fig. 4. Power electronic converter models for renewable
loop (PLL) controlling the grid frequency. The time con-
generators: (a) Boost converter and rectifier for solar AC interface stant of the PLL is often assumed to be faster compared to
and (b) Boost converter for solar DC interface. that of the rectifier, and thereby ω s  is approximated to be
Fig. 6. Model aggregation for co-simulation of transmission and distribution grids.
constant at the synchronous value of 377 rad/s. Accordingly, the distribution side, and so on. The control algorithms for
θ(t)is approximated as regulating the loads will thereafter need to be tuned accord-
ingly, depending on whether the distribution grid can depend
θ(t) = θ(t = 0) + ωs  t. (31) on the transmission grid for any deficit power feed.
Depending on more complex dynamic models of the

PLL, θ(t) may be simulated by more complicated functions B. Co-simulation of Infrastructures
than just a simple straight line. The important point to note Not just for T&D models, co-simulation is being envi-
is that although θ (t) by itself is an unbounded function, it sioned as a necessary tool for verification and validation
enters the rectifier dynamics through bounded trigonomet- of various other infrastructures whose functionalities are
ric functions. The implications of these bounded time-var- interconnected with power systems [63]. For example,
ying terms for judging the stability of the converter models power systems depend on the delivery of fuels to the genera-
must be validated using simulations. tion stations through transportation services. The produc-
tion of those fuels depends in turn on the use of electrical
V I. CH A L L ENGE 3: CO -SI MU L AT ION power. The fuels are also needed by the transportation ser-
A. Co-simulation of T&D Models vices. Evaluating these types of interdependencies is criti-
cal when big natural calamities hit the grid, causing loss
Traditionally, transmission and distribution grid models of power for extensive periods of time. Fig. 7, reproduced
have always been treated as two individual, decoupled infra-
structures. However, given the recent nationwide initiatives
in promoting demand response (DR), many researchers are
now proposing to bring these two infrastructures together,
and co-simulate their models. Recent papers such as [85] have
proposed so-called DR-advisors that act as a recommender for
power consumption prediction and control strategies for the
loads on both transmission and distribution sides of the grid.
Necessary aggregation techniques may need to be applied
here to lump the transmission grid as a single source of infor-
mation, as shown in Fig. 6. Once a load is predicted and the
DR strategy is evaluated, co-simulation models can guide
operators on how to update the load setpoints. Highly reliable
load forecasting, as mentioned earlier in Section IV-F, will be
critical for this purpose. Online estimation algorithms need to
be applied for gathering adequate information about weather,
possibilities of natural disasters, approximate generation
schedules over the next hour, information about transmission
line faults that may severely disrupt the currents flowing on Fig. 7. Interdependence of power systems with other infrastructures.
from [96], shows a logical diagram on how the power grid 4) Hierarchy: interconnected models of infrastructures
is dependent on seven other major infrastructures, and vice are often hierarchical, and thus their co-simulation
versa. This interdependence is only going to increase over scenarios must be hierarchical too. Compositionality
time, thereby necessitating the development of modeling properties of co-simulations are, therefore, impor-
and simulation tools that can assess the risks and possible tant research challenges.
impacts of critical events on quality of life by identifying 5) Formal verification: co-simulation orchestrators must
factors that trigger the domino effect between these eight be equipped with proper formal verification meth-
infrastructures. Co-simulation can be a potential solution ods to coordinate the information flows between
for this problem. Co-simulation refers to a combination of different computers running models of different
theory and techniques to enable global simulation of a cou- infrastructures.
pled system via the composition of smaller simulators. Each 6) Structure and time-scales: different infrastructures
simulator is a black-box mock-up of a constituent infrastruc- may have different structural properties in their
ture, whose model is developed by its respective operator, dynamics, and different time constants, all of which
allowing for each operator to work on his part of the prob- are reflected in their co-simulations.
lem with his own domain-specific tools. An alternative to 7) Time-varying models: many subsystems of a power
co-simulation is comodelling, where models of the different grid can have different models at different levels
infrastructures must be described in a unified language, and of abstraction while the same simulation is run-
then simulated. There are advantages to this approach, but ning. The relationships between these models
each domain has its own particularities when it comes to must be known so that correct switching between
simulation as a result of which it is impractical to find a uni- the levels of abstraction can be made at the correct
fied language that fits all. An excellent survey on this topic time instants.
was recently presented in [97]. Co-simulations can facilitate fast automated decision-
A recent study was done by Sandia National Laboratory making for grid operators during emergency conditions. A
in [98] about shock propagation and physical-to-market feed- significant amount of research still needs to be done, for
back in the U.S. natural gas infrastructure using agent-based example, on developing efficient algorithms for discrete-
modeling and state-space dynamic models. A related study event based co-simulation models, their compositional
on co-simulation was done in [99] using the idea of coopti- convergence, hybrid co-simulations, identification and
mization. Cooptimization models are computer-aided deci- handling of discontinuities, real-time constraints, and the
sion-support tools that search among possible combinations necessary simulation standards [97].
of different infrastructure-level investments to identify inte-
grated solutions that are best in terms of cost or other objec-
V II. C H A L L E NGE 4: C Y BER-PH YSIC A L
tives while satisfying all physical, economic, environmental,
MODEL I NG A N D SI MU L AT IONS
and policy constraints. Recent works in [99] and [100] have
reviewed the state-of-the-art in power system expansion As mentioned in Section III-E, power system operations
planning tools including existing cooptimization models, today are becoming increasingly dependent on communi-
and also summarized data and computational requirements cation. New types of generation, loads and control devices
for simulating cooptimization models. The conclusion was are being added to the grid, adding more complexity to its
that cooptimization-based modeling and simulations can be dynamics, and therefore necessitating the traditional prac-
excellent tools for system expansion planning, and especially tice of local monitoring and control to be replaced by global
important for power systems given the large transmission or system-wide actions [31]. These actions require a robust
investments that are anticipated to promote inter-regional and resilient communication backbone over which large vol-
power trades and renewable integration. umes of grid data can be transported in real time. Developing
A typical co-simulation may consist of one or more of the simulation packages for testing and validation of these com-
following topics [97]. munication networks, and the way they interact with the
physical operation of the grid, is currently a big challenge.
1) Numerical analysis: conditioning, accuracy and sta- Challenges in modeling and simulation on the communica-
bility of the coupled system model must be addressed. tion plane, for example, can arise from modeling of delays,
2) Differential-algebraic system simulation: composi- queuing, packet loss, routing, bandwidth sharing, and load
tions of co-simulation units are made through the balancing. Other important challenges can arise from inter-
algebraic constraints, similar to the power system operability standards, interfacing middleware, security and
model (25) and (26). privacy constraints. Typically, the Internet cannot provide
3) Hybrid systems: co-simulation scenarios, gener- the required latency and packet loss performance for grid
ally speaking, involve hybrid systems where both operation under high data-rates. Moreover, the network
continuous-time state transitions and discrete-event performance is highly random, and therefore, difficult to
switchings are involved. model accurately. Very few studies have been conducted
for leverage emerging IT technologies such as cloud com- measurements from local buses inside each area are com-
puting, software-defined networking (SDN), and network municated to phasor data concentrators (PDCs), and then
function virtualization (NFV), to accelerate this develop- to virtual computers residing in a local cloud. The virtual
ment [101], [102]. With the recent revolution in network- computers are denoted as VM (virtual machine) in Fig. 8.
ing technology, these new communication mechanisms can The geographical location of the VMs can be close to that
open up several degrees of freedom in programmability and of the generators in that area so that the latency from PDC-
virtualization for simulation platforms. However, custom- to-cloud communication is small. The local clouds them-
ized SDN control and protocols, and sufficient experimental selves constitute an Internet of clouds that connects the
validation using realistic testbeds are still missing in almost VMs through an advanced, secure, third-party wide-area
all power system applications and in their hardware-in-loop communication network such as SDN. Virtual machines
simulations. in each local cloud communicate with their neighboring
Fig. 8 shows a cyber-physical architecture that can machines inside the cloud as well as to those across other
bridge this gap. This architecture, proposed in [32], reflects clouds exchanging PMU data, and computing control sig-
the three pillars or the three C’s of any typical cyber-physical nals via pre-embedded feedback control laws. The control
system, namely, communication, computing, and control. signals are, thereafter, transmitted back from the local cloud
One may use SDN to model communication, cloud com- to the actuators of the corresponding generator models in
puting for computation, and distributed nonlinear control RTDS or Opal-RT.
designs for control. The figure shows the example of a trans- Cloud-in-the-loop CPS architectures can also be use-
mission network representing the physical layer, but the ful for emulating SCADA. Recent research trends indi-
same architecture can very well be applied to distribution cate that in the near future control centers will require
systems as well. The power system model, which may be hybrid software-hardware simulation platforms for SCADA
running in RTDS or Opal-RT, is divided into multiple nono- and contingency analysis. One example is the Graphic
verlapping areas that represent balancing regions belong- Contingency Analysis (GCA) Tool developed in Pacific
ing to different utility companies. Time-synchronized PMU Northwest National Laboratory (PNNL) to provide effec-
tive decision making support to operators [103]. A SCADA-
based state estimation (SE) simulator has been developed
recently at Washington State University using the GridSim
simulation platform [104]. More advanced SCADA emula-
tors based on Synchrophasor data need to be developed in
the future. These emulators will require a very robust CPS
platform such as the one shown in Fig. 8 for processing data
at the substation level, fast communication of the results to
the control center, and synchronization of the data at the
control center in order to generate state estimates for an
entire power system model. In Section II we highlighted the
mathematical equations for such physical models. In the fol-
lowing sections we discuss the modeling challenges for the
cyber layer, which in this case is comprised of multitudes
of virtual machines dispersed across the internet of clouds.
A. Simulation of Cyber Layer

Data delivery latency and loss rate are important fac-
tors in the performance of wide-area control and protec-
tion applications in transmission networks. Software such
as real-time dynamics monitoring system (RTDMS), pha-
sor grid data analyzer (PGDA), and GridSim are used for
online oscillation monitoring using Synchrophasors. A
list of related open-source software can be found in [105].
These simulation engines need to be integrated into execut-
able actions so that results from the monitoring algorithms
can be exported to a custom SQL database that can be set to
trigger alerts or alarms whenever damping levels of oscilla-
Fig. 8. Cloud-in-the loop architecture for wide-area control of tory modes fall below prespecified thresholds. These alarm
power systems [32]. signals need to be communicated to the operator through
a reliable communication network so that the operator can proprietary IP-based networks that power providers may
take manual actions to bring the damping back to accept- operate on. Moreover, direct communication lacks rerout-
able levels [106]. In recent years, simulation platforms such ing capabilities under real-time constraints, and resorts to
as ExoGENI-WAMS [107] have been developed to emulate historic data when communication links fail. Both timeli-
such communication platforms. The computation and com- ness and link failures can be handled using the idea of dis-
munication planes are entirely shifted away from the physi- tributed Hash tables (DHT), which was recently introduced
cal infrastructure, as shown in Fig. 8. The idea is similar to for wide-area control applications in [109].
the NASPInet described in Section III, but with an additional
layer of cloud computing. Service-based third-party private
B. Simulation of Cyber Vulnerabilities
clouds are used to create on-the-fly virtual machines, which
continuously receive real-time streaming data from specific Another driving motivation for developing CPS-centric
sets of PMUs, and exchange them with both intra-cloud simulation tools for power systems is cyber-security [110],
and inter-cloud VMs through a wide-area network such as [111]. The integration of cyber components with the physi-
Internet2. The measurements are digitally represented as a cal grid introduces new entry points for malicious attackers.
periodic stream of data points. The data delivery infrastruc- These points are remotely accessible at relatively low risk to
ture for supporting these types of applications is still evolv- attackers compared to physical intrusions or attacks on sub-
ing. Another example of a CPS simulator is GridSim [108]. stations. They can be used to mount coordinated attacks to
The data delivery component in this simulation platform, cause severe damage to the grid. Attacks can be originated
also referred to as GridStat, is a publish-subscribe middle- on the cyber layer as well to trigger cascading events leading
ware, which allows for encrypted multicast delivery of data. to damages on physical facilities, leading to major outages.
GridStat is designed to meet the requirements of emerging While simulation models have been developed to model
control and protection applications that require data deliv- electrical faults and device failures, we still do not have any
ery latencies on the order of 10 to 20 ms over hundreds of reliable simulation package that can guide us on how the
miles with extremely high availability. modern-day grid can be equipped with necessary protections
Similar to GridStat, the VMs in any cloud-in-the-loop against different types of cyber vulnerabilities. Several uni-
CPS simulator may consist of two communication planes, versities and national laboratories have started developing
namely a data plane and a management plane. The data simulation testbeds to emulate these vulnerability scenar-
plane is a collection of forwarding engines (FEs) designed ios. Research demonstration events such as Cyber Security
to quickly route received messages on to the next VMs. The Awareness Month, a student-run cyber security event in the
FEs are entirely dedicated to delivering messages from pub- U.S., have been introduced by the Department of Homeland
lishers to subscribers. Routing configuration information is Security [112]. The goal is for power system operators to
delivered to the FEs from the management plane. The for- work with these standards organizations to develop simula-
warding latency through an FE implemented in software tion software that can model, detect, localize, and mitigate
is generally on the order of 100 microseconds, and with cyber vulnerabilities in the grid as quickly as possible.
network processor hardware it is less than 10 microsec- We conclude this section by showing a simulation of a
onds. The management plane, on the other hand, is a set denial-of-service (DoS) cyber-attack on a wide-area control-
of controllers, called QoS brokers, that manage the FEs of ler for the IEEE 39-bus power system. The details of this
the data plane for every VM. The quality-of-service (QoS) controller were recently reported in [113]. First, the power
brokers can be organized in a hierarchy to reflect the natural system model was linearized about the solution of its load
hierarchy in the physical infrastructure of the grid model. flow. Then, a linear quadratic regulator (LQR)-type optimal
When a subscriber wishes to receive data from a publisher, controller was designed and sparsified using matrix sparsifi-
it communicates with a QoS broker that designs a route for cation methods. The control problem was stated as
the data and delivers the routing information to the relevant ∞
FEs and VMs, creating the subscription. min 2
∫
K J =  _1     ( x  Qx + u  Ru) dt
T T
(32)
Simulation platforms such as GridStat are just starting 0
points for research. Much more advanced cyber emulators s.t., ẋ = Ax + Bu (33)
need to be developed for the future grid. One outstanding
challenge is to develop a reliable communication software u = Kx, K = K(), Q ≥ 0, R > 0 (34)
that will enable timely delivery, high reliability and secure
networking for these emulators. This is needed for all appli- where  is a designated sparse graph topology imparted
cations, whether that be on the transmission side or dis- on the wide-area communication network. The nonlinear
tribution side. Timeliness of message requires guaranteed model of the power system was simulated with this sparse
upper bounds on end-to-end latencies of packets. Legacy state-feedback controller using Power System Toolbox
networking devices do not provide such guarantees, neither (PST) in Matlab [114]. A fault was induced at t = 0, and the
for commodity Internet connections nor for contemporary small-signal speed deviations of the synchronous generators
Fig. 9. Simulation of a cyber-attack: time-response of generator speeds before, during, and after a DoS attack.
were recorded, as shown in Fig. 9. The closed-loop system keep up. The trend today is to use multiprocessor computers
behavior is observed to be stable. At t = 10seconds, a DoS with compilers that can distribute the computation optimally
attack induced on the communication link connecting gen- to multiple processors. For power system simulations, some
erators 1 and 8, which means that these two generators are applications are much more amenable to parallelization than
no longer capable of exchanging state information for their others. For example, any simulation that requires running
control actions. Instability is noticed immediately, with the many contingencies (such as security constrained OPF) can
frequency swings diverging with increasing amplitude at run these hundreds of contingency cases in separate proces-
a frequency of roughly 0.05 Hz. This can be seen in Fig. 9 sors. The dynamics of individual generators can be run in par-
onwards from t = 10 s. At t = 60 s, the communication allel but the network that connects the generators has to be
links connecting generators (2, 6) and (3, 6) are added, and solved simultaneously. It turns out, however, that the algebraic
the corresponding control gains are recomputed. The sys- equations representing the network cannot be parallelized very
tem is seen to regain stability, indicating that the attack has efficiently, and becomes the main bottleneck for speeding up
been successfully mitigated. The frequency deviations are power system simulations. Parallel computing can also be used
all seen to converge to zero over time. for gain-scheduling of robust controllers for wide-area control.
The above example shows the importance of simula-
tions for investigating different types of attack scenarios in
B. Hybrid Simulations
the grid. Simulations can reveal the most important pairs
of generators that must communicate to maintain stable In order to implement fast control strategies in grid mod-
operating condition before and after an attack, and also the els, it is highly desirable to have a faster than real-time emu-
lesser important pairs that, either due to large geographi- lation or simulation of that model in hand. One promising
cal separation or weak dynamic coupling, do not necessar- solution is hybrid mixed-signal hardware emulation. In these
ily add any significant contribution to stability. Simulation emulations hardware-based digital simulation and analog
packages also need to be developed for illustrating various simulation can be used in an integrative way to achieve a
other types of attack scenarios such as data manipulation massively parallel and scale-insensitive emulation architec-
attacks, jamming, eavesdropping, and GPS spoofing [115]. ture. There have been several attempts at building hardware
accelerators in the past [116], [117]. For example, the DAEs
(25), (26) can be emulated in hardware by a coupled set
V III. CH A L L ENGE 5: T EC H NOLO GIE S
of oscillators, resistors, capacitors and active inductors. A
OF SI MU L AT IONS
higher frequency can be used so as to suit the scale of on-chip
A. Parallel Computing elements and permit faster than real-time operation. These
An obvious approach to speed up simulations is to utilize elements then need to be built on chip and connected with
parallel or distributed computing. Although specially written a customizable switch matrix, allowing a large portion of the
programs for particular parallel architectures can provide high grid to be modeled in real time. Researchers have proposed
speedups, the rapidly changing hardware and software makes the use of Verilog-AMS model, which is a hardware mod-
it impossible to keep modifying the simulation programs to eling language that includes features for Analog and Mixed
Signal (AMS) elements [118]. It can incorporate equations

to model analog subcomponents. The AMS model can be
designed to emulate open-loop models of very large-scale
power systems with tens of thousands of buses with built-in
equations for ac power flow, electro-magnetic and electro-
mechanical dynamics of synchronous generators and induc-
tion generators, ac load models, voltage control, droop con-
trol, automatic generation control (AGC), and power system
stabilizers. Several challenges that still stand on the way for
developing at-scale faster than real-time simulations are:
1) how to synthesize the transmission network with-
out unnecessary and unrealistic assumptions, and
approximations using state-of-the-art microelectron-
ics design technology;
2) how to develop a scalable mixed-signal emulation
architecture capable of large power systems with
tens of thousands nodes;
3) how to design configurable units of emulation on a
chip so that any large power system with realistic
transmission connections can be realized via soft-
ware-based configuration.
Research activities are underway on building VLSI chips
that can scale to large grid models for faster than real-time
emulators, especially directed towards transient ac emula-
tions. In [117], for example, predictive simulations were
shown to be very useful for real-time path rating. The
concept was verified using a 39-bus power system model,
shown in Fig. 10(a). To calculate the total transfer capabil-
ity of Path 1, the generation level in Area I was increased
while the generation in Area II was decreased progressively Fig. 10. Predictive dynamic simulations for real-time path
while screening contingencies. Comprehensive path rat- rating [117]: (a) One-line diagram of the 39-bus test system and
(b) Transfer rating calculation.
ing studies considering N-1 contingencies were conducted
using DSA Tools, developed by Powertech Labs. Simulation
results, shown in Fig. 10(b), reflect that the total transfer
capability can vary significantly in real time. The path rating access to real data. Selected examples of such testbeds in
estimated with the worst case scenario would be the smallest the United States include CPS testbeds at Washington State
of all values for 24 operating hours. Any capacity above the University using GridStat [108], cloud-assisted wide-area
smallest number is an additional gain enabled by real-time control testbed at North Carolina State University using
path rating studies. If the additional transfer capability can ExoGENI [107], cyber-security testbeds at Iowa State [119],
be fully utilized, a total amount of 25.74% more energy can a microgrid testbed called DyMonDS (Dynamic Monitoring
be transferred without compromising the current reliability and Decision Systems) at Carnegie-Mellon [120], TCIPG in
level. This could generate multimillion dollar revenues. University of Illinois [121], DETER-lab testbed at University
of Southern California [122], CPS testbeds at Idaho National
Lab, Cornell University, and Pacific Northwest National
I X . SI MU L AT ION T E ST BEDS Labs, and a big data hub at Texas A&M [123]. A compre-
Gaining access to realistic grid models and data owned by hensive list of many other smart grid testbeds and their CPS
utility companies can be difficult due to privacy and non- capabilities was recently presented in the survey paper [124].
disclosure issues. More importantly, in many circumstances Two key questions that most of these testbeds are trying to
even if real data are obtained they may not be sufficient answer are: 1) Is it possible to design sufficiently general
for studying the detailed operation of the entire system CPS standards and protocols to support a mass plug-and-
because of their limited coverage. To resolve this problem play deployment of a smart grid without sacrificing reliabil-
several CPS smart grid simulation testbeds have recently ity, data privacy and cyber-security? and (2) If so, then what
been developed to facilitate hardware-in-loop simulation standards and protocols are required to transform today’s
of different grid applications without the need for gaining grid into an end-to-end enabler of electric energy services?
A. Hardware Components challenges for such remote testbed federation? Developing

Generally speaking, the physical components of these protocols for usability by different users and potential
testbeds are comprised of real-time digital simulators (RTDS) safety hazards are two important challenges, for example.
and Opal-RT. These are power system simulator tools that Researchers are also contemplating making their testbeds
allow real-time simulation of both transmission and distri- open to public for accelerating research in the power and
bution models with a time-step of 50 microseconds. The CPS community, but a robust economic and ethics model
RTDS comes with its own proprietary software known as for sharing access to private resources still needs to be
RSCAD, which allows the user to develop detailed dynamic developed. Should there be a common centralized simula-
models of various components in prototype power systems. tion testbed for accessing power system model and data, one
The RTDS also comes with digital cards that allow external must also resolve standardization issues, communication
hardware to interface with the simulation. For example, the issues, maintenance costs, and strategies for sustainability.
gigabit transceiver analog output (GTAO) card allows the
user to view low-level signals proportional to voltages and D. Interoperability: Databases and User Interfaces
currents at different buses of the system in real-time. The
One of the major challenges for the users of the hun-
GTAO card generates voltage and current waveforms and
dreds of existing simulations is that they are not compatible
communicates them to sensors such as relays, circuit break-
with each other. Using two different simulations requires
ers, and PMUs. The PMUs measure these signals and send
keeping up two different databases of input data, and being
the resulting digitized phasor data calculations to the PDCs.
familiar with two different sets of graphical outputs. This is
The PDC time-stamps and collects the data from all the
not only true for different types of applications but also of
PMUs, and sends them to the server for display and archi-
the same application marketed by different vendors. Thus,
val, when requested. The hardware and the software layers
in the current state of art it is impossible to integrate these
of these testbeds are integrated with each other to create a
different simulation programs. The easiest way to encour-
substation-like environment within the confines of research
age interoperability is to standardize the databases. The data
laboratories. The two layers symbiotically capture power
that go into these databases are proprietary to the utilities,
system dynamic simulations as if these measurements were
and if the utilities can agree on using a standard database
made by real sensors installed at the high-voltage buses of a
the simulation vendors will have to adopt it. In the USA, the
real transmission substation.
National Institute for Standards and Technology (NIST) has
been tasked to develop such standards. An earlier standard
B. Cyber and Software Components called the Common Information Model (CIM) is now an IEC
The cyber-layer, on the other hand, is generally emulated standard and is slowly being adopted at different rates in dif-
by either a local-area network or a local cloud service. The ferent countries. A similar effort should be made to standard-
ExoGENI-WAMS testbed at NC State [107], for example, ize user interfaces. If the database and user interface for a
is connected to a state-funded, metro-scale, multilayered simulation is standardized, the ability to integrate different
advanced dynamic optical network testbed called breakable simulators as mentioned above would become much simpler.
experimental network (BEN) that connects distributed cloud
resources in local universities [125]. It allows one to set up X . CONC LUSION
dynamic multilayer connections of up to 10 Gbps between
different sites. One may simulate different types of distur- In this paper, we covered the state of the art of power sys-
bance events in power system models in RTDS, collect the tem simulations, which has evolved since the 1960s by uti-
emulated grid responses via PMUs and other sensors, com- lizing the rapidly increasing power of digital computation.
municate these data streams via BEN, and run virtual com- Increased computational capabilities have resulted in more
puting nodes at various sites in the ExoGENI cloud overlaid accurate modeling and more powerful algorithms that allow
on top of BEN to execute distributed estimation and control engineers many more analytical tools at their fingertips to
algorithms. Some open questions for these types of setups in obtain results. The challenge, however, is arising from a dif-
the future are, for example, where to deploy the computing ferent direction. The grid itself is changing because of the
facilities to better facilitate data collection and processing, proliferation of new technologies. New sources of renewable
and how to design better communication topologies. generation that are intermittent and have no rotating inertia,
are being added rapidly. New types of loads such as electric
vehicles and smart buildings are also proliferating. Power
C. Network of Remote Testbeds electronic converters and controllers are being introduced
One pertinent question is whether these different simu- to connect these new generation sources, load and storage
lation testbeds at different locations should be conjoined devices to the grid. Many of these sources are being added to
with each other to create a much bigger nationwide network the distribution system and on the customer side of the meter,
of CPS testbeds. And if yes, then what are the most common thus making the operation of the grid more complicated.
Added to this, the tremendous improvement in com- simulations depends on how easy their handling can be made
puter and communication technologies over the past decade, to the engineers. Moreover, power systems today are gradu-
in addition to improving simulation capabilities, has also ally becoming an integral part of other interconnected infra-
improved the operation and control of the power grid. The structures such as gas networks, transportation networks,
modern grid is being overlaid with more sensors, communi- communication networks, water networks, economics, and
cations, computers, information processors, and controllers food chain networks. Thus, interoperability of simulation
(ICT) so that it can be better monitored and controlled. The programs will become a key to minimizing the manual effort
combination of all of these modern technology makes the needed to set up and run co-simulations of these large inter-
grid the largest cyber-physical system (CPS) in the world.
acting systems and to interpret the results. 
The challenge is that all these new technologies including
the ICT must be modeled and simulated together. We high-
lighted these challenges in modeling, power electronics, Acknowledgements
co-simulation, cyber-physical systems and simulation tech- The first author would like to thank T. Sadamoto (Tokyo
nologies. We are optimistic that these challenges will be met Institute of Technology), R. Goodfellow (Information
by even more judicious use of computational technology. Sciences Institute), Y. Xin (Renaissance Computing
Growth in computing power shows no signs of slowing down, Institute), P. Khargonekar (University of California Irvine),
so we do not foresee any limitations on model size or algo- and A. Jain (NC State University) for helpful discussions on
rithm speed. However, the ability to utilize such powerful different parts of this paper.
R EFER ENCES Technol., vol. 21, no. 4, pp. 1168–1182, [25] D. J. Hill, “Nonlinear dynamic load models
[1] A. Bose, “Smart transmission grid Jul. 2013. with recovery for voltage stability studies,”
applications and their supporting [14] T. Sadamoto, A. Chakrabortty, T. Ishizaki, IEEE Trans. Power Syst., vol. 8, no. 1,
infrastructure,” IEEE Trans. Smart Grid, and J. Imura, “Retrofit Control of Wind pp. 166–176, Feb. 1993.
vol. 1, no. 1, pp. 11–19, Jan. 2010. Power Systems,” to appear in IEEE Trans. [26] W. Zhang, K. Kalsi, J. Fuller, M. Elizondo,
[2] A. Chakrabortty and P. P. Khargonekar, Power Syst., 2017. and D. Chassin, “Aggregate model for
“Introduction to wide-area control of power [15] E. Muljadi, M. Singh, and V. Gevorgian, heterogeneous thermostatically controlled
systems,” in Proc. Amer. Control Conf., 2013. “User guide for PV dynamic model loads with demand response,” in Proc. IEEE
simulation written on PSCAD platform,” PES Gen. Meeting, San Diego, CA, USA,
[3] K. Tomsovic, D. E. Bakken, V. Jun. 2012, pp. 1–3.
Venkatasubramanian, and A. Bose, Nat. Renew. Energy Lab., Golden, CO, USA,
“Designing the next generation of real-time Tech. Rep. NREL/TP-5D00-62053, 2014. [27] J. L. Mathieu, P. N. Price, S. Kiliccote, and
control, communication, and computations [16] R. G. de Almeida and J. A. P. Lopes, M. A. Piette, “Quantifying changes in
for large power systems,” Proc. IEEE, vol. 93, “Participation of doubly fed induction wind building electricity use, with application to
no. 5, pp. 965–979, May 2005. generators in system frequency regulation,” demand response,” IEEE Trans. Smart Grid,
IEEE Trans. Power Syst., vol. 22, no. 3, vol. 2, no. 3, pp. 507–518, Sep. 2011.
[4] M. D. Ilic, “Dynamic monitoring and
decision systems for enabling sustainable pp. 944–950, Aug. 2007. [28] T. V. Cutsem, Voltage Stability of Electric Power
energy services,” Proc. IEEE, vol. 99, no. 1, [17] N. R. Ullah, T. Thiringer, and D. Karlsson, Systems. New York, NY, USA: Springer, 1998.
pp. 58–79, Jan. 2011. “Temporary primary frequency control [29] J. Zhang, S. Nabavi, A. Chakrabortty, and
[5] D. Bakken et al., “GRIP—Grids with support by variable speed wind turbines— Y. Xin, “ADMM optimization strategies for
intelligent periphery: Control architectures Potential and applications,” IEEE Trans. Power wide-area oscillation monitoring in power
for Grid2050,” in Proc. IEEE Int. Conf. Smart Syst., vol. 23, no. 2, pp. 601–612, May 2008. systems under asynchronous communication
Grid Commun., Oct. 2011, pp. 7–12. [18] J. Ingleson and E. Allen, “Tracking the delays,” IEEE Trans. Smart Grid, vol. 7, no. 4,
[6] IEEE Vision for Smart Grid Controls: 2030 and Eastern Interconnection Frequency pp. 2123–2133, Apr. 2016.
Beyond. [Online]. Available: http:// Governing Characteristic,” in Proc. IEEE PES [30] J. Zhang, S. Nabavi, A. Chakrabortty, and
ieeexplore.ieee.org/document/6577608/ General Meeting, Jul. 2010. Y. Xin, “Convergence analysis of ADMM
[7] M. D. Ilic, J. Joo, L. Xie, M. Prica, and [19] Z. Miao, L. Fan, D. Osborn, and based power system mode estimation under
N. Rotering, “A decision-making framework S. Yuvarajan, “Control of DFIG-based wind asynchronous wide-area communication
and simulator for sustainable electric energy generation to improve interarea oscillation delays,” in Proc. IEEE PES Gen. Meeting,
systems,” IEEE Trans. Sustain. Energy, vol. 2, damping,” IEEE Trans. Energy Convers., Denver, CO, USA, Jul. 2015, pp. 1–5.
no. 1, pp. 37–49, Jan. 2011. vol. 24, no. 2, pp. 415–422, Jun. 2009. [31] S. Nabavi, J. Zhang, and A. Chakrabortty,
[8] L. Xie, Y. Zhang, and M. D. Ilic, “Multi-scale [20] A. Ortega and F. Milano (2015). “Generalized “Distributed optimization algorithms for
integration of physics-based and data-driven model of VSC-based energy storage systems for wide-area oscillation monitoring in power
models in power systems,” in Proc. IEEE/ACM transient stability analysis.” [Online]. systems using an inter-regional PMU-PDC
Third Int. Conf. Cyber-Physical Syst., Available: https://arxiv.org/abs/1509.05290v1 architectures,” IEEE Trans. Smart Grid, vol. 6,
Apr. 2012, pp. 129–137. no. 5, pp. 2551–2559, Sep. 2015.
[21] N. Diorio, A. Dobos, S. Janzou, A. Nelson,
[9] J. D. Glover, T. Overbye, and M. S. Sarma, and B. Lundstrom, “Technoeconomic [32] D. Soudbakhsh, A. Chakrabortty, and
Power System Analysis and Design, 6th ed. 2016. modeling of battery energy storage in SAM,” A. M. Annaswamy, “A delay-aware cyber-
Nat. Renew. Energy Lab., Golden, CO, USA, physical architecture for wide-area control of
[10] A. M. Gole, S. Filizadeh, R. W. Menzies, and
Tech. Rep. NREL/TP-6A20-64641, 2015. power systems,” IFAC Control Eng. Pract.,
P. L. Wilson, “Optimization-enabled
electromagnetic transient simulation,” vol. 60, pp. 171–182, Jun. 2017.
[22] R. Hu, “State charge balancing droop
IEEE Trans. Power Del., vol. 20, no. 1, control,” M.S. thesis, Michigan Tech. Univ., [33] J. G. Deshpande, K. C. Budka, and
pp. 512–518, Jan. 2005. Houghton, MI, USA, 2015. M. Thottan, Communication Networks for
[11] P. Kundur, Power System Stability and Control. Smart Grids. Springer, 2014.
[23] I. T. Force, “Standard load models for power
New York, NY, USA: McGraw-Hill, 1994. flow and dynamic performance simulation,” [34] D. Anderson, W. S. Cleveland, and B. Xi,
[12] P. Sauer and M. A. Pai, Power System IEEE Trans. Power Syst., vol. 10, no. 3, pp. 1–3, “Multifractal and Gaussian fractional
Dynamics and Stability, Englewood Cliffs, NJ, Mar. 1995. sum–difference models for Internet traffic,”
USA: Prentice-Hall, 1998. Perform. Eval., vol. 107, pp. 1–33, Jan. 2017.
[24] I. A. Hiskens and J. V. Milanovic, “Load
[13] P. F. Odgaard, J. Stoustrup, and M. Kinnaert, modeling in studies of power system [35] RTDS Technologies Inc. [Online]. Available:
“Fault-tolerant control of wind turbines: A damping,” IEEE Trans. Power Syst., vol. 10, https://www.rtds.com/
benchmark model,” IEEE Trans. Control Syst. no. 4, pp. 1781–1788, Apr. 1995. [36] OPAL-RT. [Online]. Available: www.opal-rt.com
[37] PowerWorld. [Online]. Available: https:// Cambridge, MA, USA, Tech. Rep. EL99-005 [73] A. Chakrabortty, “Wide-area damping control
www.powerworld.com WP, 1999. of power systems using dynamic clustering
[38] F. Milano, “A python-based software tool for [57] M. Shahidehpour, H. Yamin, and Z. Li, and TCSC-based redesigns,” IEEE Trans. Smart
power system analysis,” in Proc. IEE Power Market Operations in Electric Power Systems: Grid, vol. 2, no. 3, pp. 1503–1514, Nov. 2012.
Energy Soc. Gen. Meeting, 2013, pp. 1–5. Forecasting, Scheduling, and Risk Management. [74] N. Xue and A. Chakrabortty, “H2-clustering
[39] F. Milano, L. Vanfretti, and J. C. Morataya, “An Hoboken, NJ, USA: Wiley, 2003. of closed-loop consensus networks under a
open source power system virtual laboratory: [58] M. D. Ilic, L. Xie, and J. Joo, “Efficient class of LQR design,” in Proc. Amer. Control
The PSAT case and experience,” IEEE Trans. coordination of wind power and price- Conf., 2016, p. 1.
Educ., vol. 51, no. 1, pp. 17–23, Feb. 2008. responsive demand—Part I: Theoretical [75] G. Chavan, M. Weiss, A. Chakrabortty,
[40] R. C. Dugan, “The open distribution system foundations,” IEEE Trans. Power Syst., vol. 26, S. Bhattacharya, A. Salazar, and F. Ashrafi,
simulator (OpenDSS),” EPRI OpenDSS no. 4, pp. 1875–1884, Apr. 2011. “Real-time identification and predictive analysis
Manual, Nov. 2012. [59] Open Source Software (OSS) for Electricity of a multi-area WECC power system model
Market Research, Teaching, and Training. using synchrophasors,” IEEE Trans. Smart Grid,
[41] C. Ptolemaeus, Ed., System Design, Modeling, vol. 8, no. 4, pp. 1977–1986, Apr. 2017.
and Simulation Using Ptolemy, vol. 2. 2014. [Online]. Available: http://www2.econ.
iastate.edu/tesfatsi/ElectricOSS.htm#Markets [76] S. Deml, A. Ulbig, T. Borsche, and G. Andersson,
[42] P. Fritzson, Introduction to Modeling and “The role of aggregation in power system
Simulation of Technical and Physical Systems [60] J. Goldis, A. Rudkevich, L. Omondi, and
R. Tabors, “Use of cloud computing in power simulation,” in Proc. IEEE PowerTech,
with Modelica. Hoboken, NJ, USA: Wiley, Eindhoven, The Netherlands, Jun. 2015, pp. 1–6.
Sep. 2011. market simulations,” presented at the FERC
Tech. Conf. Increasing Real-Time Day-Ahead [77] A. Chakrabortty and A. Salazar, “Building a
[43] G. Giorgidze and H. Nilsson, “Embedding a Market Efficiency through Improved Softw., dynamic electro-mechanical model for the
functional hybrid modelling language in Washington, DC, USA, Jun. 2014. pacific ac intertie using distributed
Haskell,” in Implementation and Application of synchrophasor measurements,” Eur. Trans. Electr.
Functional Languages. Springer, 2011, [61] K. Kalsi, S. Elbert, M. Vlachopoulou, N. Zhou,
and Z. Huang, “Advanced computational Power, vol. 21, no. 4, pp. 1657–1672, 2011.
pp. 138–155.
methods for security constrained financial [78] A. Chakrabortty, J. H. Chow, and A. Salazar, “A
[44] D. Zimmer, “A new framework for the transmission rights,” in Proc. IEEE PES Gen. measurement-based framework for dynamic
simulation of equation-based models with Meeting, Jul. 2012, pp. 1–8. equivalencing of power systems using wide-
variable structure,” Simulation, vol. 89, no. 8, area phasor measurements,” IEEE Trans. Smart
pp. 935–963, 2013. [62] Y. Yan, Y. Qian, H. Sharif, and D. Tipper,
“A survey on smart grid communication Grid, vol. 1, no. 2, pp. 68–81, Feb. 2011.
[45] D. K. Molzahn and I. A. Hiskens, “Sparsity- infrastructures: Motivations, requirements [79] S. Nabavi and A. Chakrabortty, “Topology
exploiting moment-based relaxations of the and challenges,” IEEE Commun. Surveys Tuts., identification for dynamic equivalent models
optimal power flow problem,” IEEE Trans. vol. 15, no. 1, pp. 5–20, 2nd Quart., 2013. of large power system networks,” in Proc.
Power Syst., vol. 30, no. 6, pp. 3168–3180, Amer. Control Conf., 2013.
Nov. 2015. [63] A. Annaswamy, A. Hussain, A. Chakrabortty,
and M. Cvetkovic, “Foundations of [80] G. B. Dantzig and P. Wolfe, “Decomposition
[46] J. Lavaei and S. H. Low, “Zero duality gap in infrastructure-CPS,” in Proc. Amer. Control Conf., principle for linear programs,” Oper. Res.,
optimal power flow problem,” IEEE Trans. Boston, MA, USA, 2016, pp. 2649–2664. vol. 8, no. 1, pp. 101–111, 1960.
Power Syst., vol. 27, no. 1, pp. 92–107, Feb. 2012.
[64] P. T. Myrda, J. Taft, and P. Donner, [81] J. H. Chow, A. Chakrabortty, M. Arcak,
[47] D. K. Molzahn, J. T. Holzer, B. C. Lesieutre, “Recommended approach to a NASPInet B.≈Bhargava, and A. Salazar, “Synchronized
and C. L. DeMarco, “Implementation of a architecture,” in Proc. 45th Hawaii Int. Conf. phasor data based energy function analysis
large-scale optimal power flow solver based Syst. Sci., 2012. of dominant power transfer paths in large
on semidefinite programming,” IEEE Trans. power systems,” IEEE Trans. Power Syst.,
Power Syst., vol. 28, no. 4, pp. 3987–3998, [65] [Online]. Available: http://www.elp.com/
vol. 22, no. 2, pp. 727–734, May 2007.
Nov. 2013. articles/powergrid_international/print/
volume-14/issue-8/features/ami- [82] R. Weron, “Electricty price forecasting: A
[48] D. K. Molzahn, B. C. Lesieutre, and communications-technology.html review of the state-of-the-art with a look into
C. L. DeMarco, “A sufficient condition for the future,” Int. J. Forecasting, vol. 30, no. 4,
global optimality of solutions to the optimal [66] A. Milani, M. T. Khan, A. Chakrabortty, and
pp. 1030–1081, 2014.
power flow problem,” IEEE Trans. Power I. Husain, “Equilibrium point analysis and
power sharing methods for distribution [83] F. Lian, A. Chakrabortty, and A. Duel-Hallen,
Syst., vol. 29, no. 2, pp. 978–979, Mar. 2014. “Game-theoretic multi-agent control and
systems driven by solid-state transformers,”
[49] R. Madani, S. Sojoudi, and J. Lavaei, IEEE Trans. Power Syst., to be published. network cost allocation under communication
“Convex relaxation for optimal power flow constraints,” IEEE J. Sel. Areas Commun., vol.
problem: Mesh networks,” IEEE Trans. Power [67] K. Sadek, G. Wild, L. Hugelschafer, A. Gole,
35, no. 2, pp. 330–340, Feb. 2017.
Syst., vol. 30, no. 1, pp. 199–211, Jan. 2015. X. Jiang, and D. Brandt, “Modelling of digital
HVDC control systems using a graphical [84] M. Negnevitsky, P. Mandal, and A. K. Srivastava,
[50] J. Blackburn and T. Domin, Protective electromagnetic simulation program,” in “An overview of forecasting problems and
Relaying: Principles and Applications. Boca Proc. Int. Conf. Power Syst. Trans., Lisbon, techniques in power systems,” in Proc. IEEE
Raton, FL, USA: CRC, 2014. Portugal, 1995. PES Gen. Meeting, Jul. 2009, pp. 1–4.
[51] N. Zhang and M. Kezunovic, “Implementing [68] L. G. Perez, A. J. Flechsig, and [85] M. Behl, A. Jain, and R. Mangharam, “Data-
an advanced simulation tool for V. Venkatasubramanian, “Modeling the driven modeling, control and tools for cyber-
comprehensive fault analysis,” in Proc. IEEE protective system for power system dynamic physical energy systems,” in Proc. ACM/IEEE 7th
Asia–Pacific PES Transmiss. Distrib. Conf. analysis,” IEEE Trans. Power Syst., vol. 9, Int. Conf. Cyber-Phys. Syst., Apr. 2016, pp. 1–10.
Exhib., Aug. 2005, pp. 1–6. no. 4, pp. 1963–1973, Nov. 1994. [86] S. Hui, C. Lee, and F. Wu, “Electric springs—A
[52] S. Takayama and R. Matsuhashi, [69] V. Venkatasubramanian, H. Schattler, and new smart grid technology,” IEEE Trans. Smart
“Development of model for load frequency J. Zaborsky, “Dynamics of large constrained Grid, vol. 3, no. 3, pp. 1552–1561, Mar. 2012.
control in power system with large-scale nonlinear systems: A taxonomy theory,” [87] L. Zhu and D. J. Hill, “Modeling and stability
integration of renewable energy,” in Proc. IEEE Proc. IEEE, vol. 83, no. 11, pp. 1530–1561, of microgrids with smart loads,” in Proc. 20th
Power Energy Conf. Illinois, Feb. 2016, pp. 1–8. Nov. 1995. IFAC World Congr., 2017, pp. 10438–10443.
[53] M. Prais, C. Johnson, A. Bose, and [70] A. M. Boker, T. R. Nudell, and A. Chakrabortty, [88] I. A. Hiskens and J. Alseddiqui, “Sensitivity,
D. Curtice, “Operator training simulator: “On aggregate control of clustered consensus approximation, and uncertainty in power
Component models,” IEEE Trans. Power Syst., networks,” in Proc. Amer. Control Conf., 2015, system dynamic simulation,” IEEE Trans.
vol. 4, no. 3, pp. 1160–1166, Aug. 1989. Chicago, IL, USA, pp. 5527–5532. Power Syst., vol. 21, no. 4, pp. 1808–1820,
[54] J. M. Bucciero et al., “Dispatcher training [71] S. Hara, J. I. Imura, K. Tsumura, T. Ishizaki, Nov. 2006.
simulators: Lessons learned,” IEEE Trans. Power and T. Sadamoto, “Glocal (global/local) [89] I. A. Hiskens, B. C. Lesiutre, and S. Roy,
Syst., vol. 6, no. 2, pp. 594–604, May 1991. control synthesis for hierarchical networked “Uncertainty evaluation and mitigation in
[55] H. Kim and C. Singh, “Reliability modeling systems,” in Proc. IEEE Conf. Control Appl., electrical power systems,” in Proc. Symp.
and simulation in power systems with aging Sep. 2015, pp. 107–112. Specialists Electr. Oper. Expansion Planning, 2009.
characteristics,” IEEE Trans. Power Syst., [72] T. Sadamoto, T. Ishizaki, and J. I. Imura, [90] A. M. Gole et al., “Guidelines for modeling
vol. 25, no. 1, pp. 21–28, Feb. 2010. “Hierarchical distributed control for networked power electronics in electric power
[56] J. F. Prada, “The value of reliability in power linear systems,” in Proc. 53rd IEEE Conf. Decision engineering applications,” IEEE Trans. Power
systems—Pricing operating reserves,” MIT, Control, Jun. 2014, pp. 2447–2452. Del., vol. 12, no. 1, pp. 505–514, Jan. 1997.
[91] N. G. Hingorani and L. Gyugyi, [102] Y. Xin, I. Baldine, J. Chase, T. Beyene, networks,” in Proc. Amer. Control Conf.,
Understanding FACTS: Concepts and B. Parkhurst, and A. Chakrabortty, 2017.
Technology of Flexible AC Transmission “Virtual smart grid architecture and [114] J. H. Chow and K. W. Cheung, “A toolbox for
Systems. Hoboken, NJ, USA: Wiley, 1999. control framework,” in Proc. IEEE Int. Conf. power system dynamics and control engineering
[92] U. P. Mhaskar and A. M. Kulkarni, “Power Smart Grid Commun., Oct. 2011, pp. 1–6. education and research,” IEEE Trans. Power Syst.,
oscillation damping using FACTS devices: [103] Graphical Contingency Analysis Tool at vol. 7, no. 4, pp. 1559–1564, Nov. 1992.
Modal controllability, observability in local PNNL. [Online]. Available: http://eioc.pnnl. [115] G. Dan, H. Sandberg, M. Ekstedt, and
signals, and location of transfer function gov/research/contingencyanalysis.stm G. Bjrkman, “Challenges in power system
zeros,” IEEE Trans. Power Syst., vol. 21, [104] D. Anderson, C. Zhao, C. H. Hauser, information security,” IEEE Security Privacy,
no. 1, pp. 285–294, Jan. 2006. V. Venkatasubramanian, D. E. Bakken, and vol. 10, no. 4, pp. 62–70, Apr. 2012.
[93] K. R. Padiyar, Structure Preserving Energy A. Bose, “A virtual smart grid,” IEEE Power
Functions in Power Systems: Theory and [116] I. Nagel, “Analog microelectronic
Energy Mag., to be published. emulation for dynamic power system
Applications. Boca Raton, FL, USA: CRC, 2013.
[105] SmarTS Lab, Royal Institute Technology computation,” Ph.D. dissertation, École
[94] J. S. Glaser, F. E. Cellier, and A. F. Witulski, (KTH). [Online]. Available: https://github. Polytechnique Fédérale de Lausanne,
“Object-oriented power system modeling com/SmarTS-Lab Lausanne, Switzerland, 2013.
using the dymola modeling language,” in Proc.
[106] M. Chenine, L. Vanfretti, S. Bengtsson, and [117] S. Jin, Z. Huang, R. Diao, D. Wu, and
26th Annu. IEEE Power Electron. Specialists
Conf., Atlanta, GA, USA, Jun. 1995, pp. 1–3. L. Nordstrom, “Implementation of an Y. Chen, “Parallel implementation of power
experimental wide-area monitoring system dynamic simulation,” in Proc. IEEE
[95] H. N. Pico and D. C. Aliprantis, “Voltage platform for development of synchronized PES Gen. Meeting, Jul. 2013, pp. 1–5.
ride-through capability verification of DFIG- phasor measurement applications,” in Proc.
based wind turbines using reachability [118] E. J. Wyers, M. Steer, C. Kelley, and
IEEE Power Energy Soc. Gen. Meeting, Jul.
analysis,” IEEE Trans. Energy Convers., P. Franzon, “A bounded and discretized
2011, pp. 1–8.
vol. 31, no. 4, pp. 1387–1398, Apr. 2016. Nelder-mead algorithm suitable for RFIC
[107] A. Chakrabortty and Y. Xin, “Hardware-in- calibration,” IEEE Trans. Circuits Syst. I, Reg.
[96] [Online]. Available: http://www.cis. the-loop simulations and verifications of Papers, vol. 60, no. 7, pp. 1787–1799, Jul. 2013.
uniroma1.it/node/5614 smart power systems over an Exo-GENI
[97] C. Gomes, C. Thule, D. Broman, and [119] Power Infrastructure Cybersecurity Laboratory.
testbed,” in Proc. 2nd GENI Res. Ed.
P. G. Larsen Co-Simulation: State of the Art. [Online]. Available: http://powercyber.ece.
Experim. Workshop, Mar. 2013.
[Online]. Available: http://arxiv.org/ iastate.edu/penetintro.html
[108] GridStat. [Online]. Available: www.gridstat.net
pdf/1702.00686.pdf [120] M. D. Ilic, L. Xie, and Q. Liu, Engineering
[109] T. Qian, A. Chakrabortty, F. Mueller, and IT-Enabled Sustainable Electricity Services:
[98] P. H. Kobos et al., “Natural gas value-chain Y. Xin, “A distributed storage system for
and network assessments,” Sandia Nat. The Tale of Two Low-Cost Green Azores
multi-resolution virtual synchrophasors,” Islands, vol. 30. Springer, 2013.
Labs, Livermore, CA, USA, Tech. Rep. in Proc. IEEE PES Gen. Meeting,
SAND2015-7915, 2015. Washington, DC, USA, Jul. 2014. [121] TCIPG: Trustworthy Cyber Infrastructure for
[99] A. L. Liu et al., “White paper: Co-optimization the Power Grid. [Online]. Available: https://
[110] S. Sridhar, A. Hahn, and M. Govindarasu, tcipg.org
of transmission and other supply resources,”
“Cyber-physical system security for the
Nat. Assoc. Regulatory Utility Commissioners\, [122] The DETER Project. [Online]. Available:
electric power grid,” Proc. IEEE, vol. 100,
White paper, 2013. [Online]. Available: http:// https://deter-project.org
no. 1, pp. 210–224, Jan. 2012.
pubs.naruc.org/pub/536D834A-2354-D714- [123] Global Network for Synchrophasor Solutions.
51D6-AE55F431E2AA [111] J. Zhang, P. Jaipuria, A. Hussain, and
A. Chakrabortty, “Attack-resilient [Online]. Available: http://gnssconsortium.org
[100] M. Caramanis, E. Ntakou, W. W. Hogan, [124] M. H. Cintuglu, O. A. Mohammed,
A. Chakrabortty, and J. Schoene, estimation of power system oscillation
modes using distributed and parallel K. Akkaya, and A. S. Uluagac, “A survey on
“Co-optimization of power and reserves in
optimization: Theoretical and experimental smart grid cyber-physical system testbeds,”
dynamic T&D power markets with
methods,” in Proc. Conf. Decision Game IEEE Commun. Surveys Tuts., vol. 19, no. 1,
nondispatchable renewable generation
and distributed energy resources,” Proc. Theory Security, Los Angeles, CA, USA, 2014. pp. 446–464, 1st Quart., 2017.
IEEE, vol. 104, no. 4, pp. 807–836, Apr. 2016. [112] National Cyber Security Awareness Month. [125] GENI. [Online]. Available: www.geni.net
[101] M. Jarschel, S. Oechsner, D. Schlosser, [Online]. Available: https://www.it.iastate. [126] A. Chakrabortty, “Infusing autonomy in
R. Pries, S. Goll, and P. Tran-Gia, edu/blog/view/36 power distribution networks using smart
“Modeling and performance evaluation of [113] A. Jain, A. Chakrabortty, and E. Biyik, “An transformers,” IEEE Conf. Control
an openflow architecture,” in Proc. 23rd online structurally constrained LQR design Technology and Applications (CCTA), HI,
Int. Teletraffic Congr., 2011, pp. 1–7. for damping oscillations in power system USA, 2017.
ABOUT THE AUTHORS

Aranya Chakrabortty (Senior Member, IEEE) Anjan Bose (Fellow, IEEE) has over 40 years
received the Ph.D. degree in electrical engineer- of experience in industry and academia, as an
ing from Rensselaer Polytechnic Institute, Troy, engineer, educator, and administrator. He is well
NY, USA, in 2008. known as a Technical Leader in the power grid
He is currently an Associate Professor control industry, a Researcher in electric power
in the Electrical and Computer Engineering engineering, an educator in engineering, and an
Department, North Carolina State University, administrator in higher education. He is a Regents
Raleigh, NC, USA. His research interests are in all Professor at Washington State University (WSU),
branches of control theory with applications to USA, where he also served as the Dean of Engi-
power systems, especially in wide-area monitoring, stability, and control neering and Architecture (1998�2005) and in 2012�13 served as a Senior
of large power systems using synchrophasors. He is particularly interested Advisor to the U.S. Department of Energy (DOE) in the Obama administration.
in investigating different cyber-physical modeling and control challenges Dr. Bose is a Member of the U.S. National Academy of Engineering
for the next-generation power grid, at both transmission and distribution (2003) and has served on many National Academy Committees. He is a
levels. He currently serves as an Associate Editor for the IEEE Control Sys- Founding Member of the Washington State Academy of Sciences and has
tems Society Conference Editorial Board (2012-present) and also for the been elected as its President. He is also a Foreign Fellow of the Indian
IEEE Transactions on Control Systems Technology (2015-present). National Academy of Engineering. He is active in several international
Dr. Chakrabortty received the NSF CAREER award in 2011. professional societies. He was the recipient of the Outstanding Power
Engineering Educator Award (1994), the Third Millenium Medal (2000)
and the Herman Halperin Electric Transmission & Distribution Award
(2006), from the IEEE. He has been recognized as a distinguished alumnus
of the Indian Institute of Technology, Kharagpur (2005), and the College of
Engineering at Iowa State University (1993).

30-Smart Grid Simulations and Their Supporting Implementation Methods

Uploaded by

Copyright:

Available Formats

You might also like

30-Smart Grid Simulations and Their Supporting Implementation Methods

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

30-Smart Grid Simulations and Their Supporting Implementation Methods

Uploaded by

Copyright:

Available Formats

Smart Grid Simulations and

ABSTRACT | In this tutorial we present the state-of-the-art I. I N T RODUC T ION

electromagnetic torque using rotor current control ena-

{​Ew​ ​​ = γ (​ ​v1q

∫−∞​ f​ (s) ds = ​ __21 ​​[erf​(___ ​ __ ​)​]​

× ​e​​ −λt​​[erf​(_____ )​ + erf​(​ )​]​.​

where ​μ > m​. The Internet traffic delay is modeled by an

​​f1​ ​​ (t) = λ ​e​​ −λt​​ (21) f(x) = 0​

​P3​ ​​ (x ) − ​P​G3​​ + ​P​D3​​ dx ​ = f(x, y, p), x

D. Fault Analysis G. Reliability Modeling and Simulation

B. Scalability be highlighted in Section V. Developing accurate modeling

2) lack of parallel processing, meaning that many of the

for handling such discontinuities. This can result in long

Fig. 6. Model aggregation for co-simulation of transmission and distribution grids.

Depending on more complex dynamic models of the

A. Simulation of Cyber Layer

Signal (AMS) elements [118]. It can incorporate equations

A. Hardware Components challenges for such remote testbed federation? Developing

ABOUT THE AUTHORS

You might also like

{Ew = γ ( v1q

∫−∞  f (s) ds =  21 [erf(_   __  )]

× e  −λt[erf(_____ )  + erf(  )].

where μ > m. The Internet traffic delay is modeled by an

f1  (t) = λ e  −λt (21) f(x) = 0

P3 (x )  − PG3  + PD3 dx  = f(x, y, p), x

2) lack of parallel processing, meaning that many of the