2964
6, NOVEMBER 2015
A Micro-Grid Distributed Intelligent Control
and Management System
Wolf Kohn, Zelda B. Zabinsky, and Anil Nerode
Abstract—Micro-grids require active control to maintain qual-
ity of service and to interface with the power grid in a
bi-directional manner. Further, micro-grids must be justified by
environmental, governmental, and economic viability. We present
a programmable architecture for active, optimal distributed con-
trol of elements of the grid to achieve desired behavior. A unique
aspect of this architecture is to include a distributed inductive
engine for learning the local dynamics of generators and loads
in the micro-grid. It generates feedback laws that are adapted
to the current status of the micro-grid, and responds to anoma-
lous events in a resilient manner. An important novelty is that
control laws are extracted online for bidirectional discontinu-
ous nonlinear loads by mean field methods from physics outside
the standard design methodologies for piecewise linear quadratic
controls.
Index Terms—Bidirectional power flow, computational and
artificial intelligence, decentralized control, distributed param-
eter systems, learning systems, micro-grids, power engineering
and energy, power system control, power system management,
smart grids.
I. I NTRODUCTION
E CONSIDER a micro-grid that serves the energy
W needs for a building or a cluster of buildings, such
as a campus, a hospital complex, an industrial complex,
a residential neighborhood, or a military installation. The
classic approach to maintain and install such a micro-grid
involves a careful design based on detailed stochastic mod-
els of the dynamics of the loads and generators [1]–[4].
However, the dynamics of a micro-grid with renewable energy
sources and smart loads cannot be modeled by standard
modeling of dynamical systems in meaningful ways. Our
approach uses logic rules to encode customizable dynamic
principles of the loads and generation devices, including
uncertainties, by learning and adaptation without using an
explicit model.
The high cost of energy and the need for high qual-
ity of service and reliability of the micro-grid, suggest the
Manuscript received June 26, 2014; revised November 25, 2014,
February 23, 2015, and June 10, 2015; accepted July 8, 2015. Date of publi-
cation August 6, 2015; date of current version October 17, 2015. This work
was supported in part by Atigeo, LLC, and in part by the National Science
Foundation under Grant CMMI-0908317. Paper no. TSG-00653-2014.
W. Kohn and Z. B. Zabinsky are with Atigeo LLC, Bellevue,
WA 98004 USA, and also with the Department of Industrial and Systems
Engineering, University of Washington, Seattle, WA 98195-2650 USA
(e-mail: wolfk@u.washington.edu; zelda@u.washington.edu).
A. Nerode is with Cornell University, Ithaca, NY 14853 USA (e-mail:
anil@math.cornell.edu).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TSG.2015.2455512
1949-3053 c 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO.
implementation of automated adaptive control strategies that
are the result of multiobjective optimal control laws. A discus-
sion of centralized and decentralized control issues is found
in [5]. Some issues include scalability and accuracy of control
algorithms. Distributed algorithms have been developed for
wireless sensor networks [6] and macro-femto networks [7].
Our approach is to be general in the type of micro-grid.
Our intelligent control system learns the dynamics of the sys-
tem through sensors and modifies the system to achieve the
objectives.
Micro-grids with generation resources are connected to the
general power grid to be able to satisfy the variable loads
in a cost-effective manner. For micro-grids having genera-
tion resources, the bidirectional exchange of power provides
additional opportunities to optimize performance.
This paper describes a new architecture for micro-grid dis-
tributed intelligent control and management. The approach
we follow is based on principles of hybrid systems and
optimization [8]–[14]. We build a distributed network of intel-
ligent element controllers that can provide synchronization of
a micro-grid to the power grid, and provide feedback control
across the micro-grid to achieve optimal performance.
II. I NTELLIGENT C ONTROL AND M ANAGEMENT
A RCHITECTURE OF M ICRO -G RID
For the purposes of this paper, a micro-grid is a network
of electric devices connected by conductor lines. A device
is attached to a bus and connected carrying loads, generat-
ing units, or a connection to another grid. Our micro-grid
architecture includes both controlled and uncontrolled devices.
A micro-grid diagram is shown in Fig. 1. Controlled devices,
as shown in Fig. 1, are directly connected in a feedback con-
figuration to an intelligent control element, called an element
controller. The purpose of these control elements is to drive the
grid to a desired dynamic behavior. This desired behavior is
effected by control actions generated by these intelligent con-
trol elements as a function of sensory data measuring the status
of the devices in the micro-grid, expressed desired behavior,
a partial dynamic model of the grid and a corpus of behavior
generated by active, inductive learning from the sensory data.
To a significant extent, this paper is concerned with estab-
lishing an implementable technology for micro-grid intelligent
control. Why does a micro-grid need intelligent control?
We explore four reasons.
1) Conventional optimal control techniques require an
accurate model of the dynamic behavior of the loads,
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM
Fig. 1. Micro-grid system.
such as an air conditioner, or a hospital MRI, in the
micro-grid. The main problem that we face today is that
modeling the loads at sufficient accuracy is impractical.
Further, many current loads such as battery chargers,
high temperature crystal growing, MRI machines and
others have nonlinear and discontinuous dynamics. We
propose a methodology based on machine learning,
load forecasting, and rule-based optimization to iden-
tify the dynamic behavior of the loads using sensory
data in real time. The methodology models near real-
time power and reactive power, voltage, and current in
the micro-grid. We model the dynamic behavior of loads
and generation using localized Hamiltonians [15] where
the state variables include relationships between volt-
age and current and the device, and operational cost.
The Hamiltonian is constructed from rules describing
these relationships, by a conservation principle. For each
device, the Hamiltonian associated with the device inter-
acts with the aggregated Hamiltonian representing the
network at the ports of the device (see [9] and [16]).
For each state variable in the Hamiltonian, there is
a momentum (or co-state) variable. The dynamics of
these variables are given in Section II-C. In addition
the action variables representing voltage settings, or
flow direction in the case of a battery, or inverter
settings, etc., are determined by an optimality condi-
tion [represented by (3)]. The methodology we propose
includes a component that extracts structural charac-
teristics of the dynamics from historic and current
data, and improves the accuracy of the model as time
goes on. For example, the functional relation between
voltage, current and phase in each DER is extracted
from data. This involves constructing polynomic forms,
perhaps with saturation, that locally characterizes the
2965
relationship at the current time. Thus the response of
our intelligent control system also improves over time.
Furthermore, as the system evolves due to changes
in the load profile or generation profile, our intelli-
gent control system adapts to those changes through
learning.
2) Micro-grids are distributed dynamical systems. In the
past, micro-grids were designed with sufficient excess
capacity to satisfy peak demand with high probability,
however, we can no longer afford to waste this excess
capacity. A conventional centralized control design can-
not provide quality feedback control to the multiplicity
of active devices of the micro-grid. Our distributed con-
trol manages the control action for each active device
and is able to more accurately match the variability of
the demand with the multiple sources of generation so
that peak demand and base demand are satisfied. In our
distributed control, local intelligent feedback controllers
are connected to form a collaborative network with oper-
ational target settings for each element controller and
system-wide optimization criteria.
3) Micro-grids exchange energy with the environment and
are open systems. The nature of this bi-directional
exchange is stochastic, and the dynamics of this stochas-
ticity has to be learned from observations. This exchange
requires the intelligent control to provide active syn-
chronization control. This active synchronization control
between the network of element controllers and the out-
side system is essential to maintain high quality of
service in a cost-effective manner. The control is an
electricity price-based control, where quality of service
is included as a component of the cost of the micro-
grid. The quality of service is specifically referring to
voltage and frequency regulation. The synchronization
2966
mechanism of each device with respect to the micro-
grid is achieved by establishing a Pareto equilibrium
between the Hamiltonian of the device and the aggre-
gated Hamiltonian at the ports of the device [15], [16].
Pareto equilibrium is achieved when there it is near reso-
nance (high quality factor), between the desired dynamic
operation of the device and the grid dynamics at the port
represented by its grid.
4) Micro-grids are distributed heterogeneous systems. We
apply a version of indirect synchronization between active
controllers based on a notion of Pareto equilibrium [17]
between each control element and a virtual control ele-
ment called a mean field element, whose dynamics are
encoded and transmitted from one controller to another,
iteratively, establishing synchronization between each
element controller and mean field. This concept gener-
alizes Tellegen’s theorem [18] that states that in solving
distributed control by a network of agents, we can regard
an agent controlling a subsystem as interacting with an
aggregated agent, representing an average control for
the micro-grid. The interaction between an agent and
the average agent is formulated as an optimal control
system subject to constraints. We model this average
agent opponent using a method from particle physics
called mean field. A Pareto equilibrium is approximated
by iteratively playing a two-person Pareto game between
a control element and the mean field element. By iterating
fast enough, a good approximation is achieved with com-
putation that is linear in the number of control elements.
Hence, the method is scalable.
Several sub-systems, each controlled by a separate intelli-
gent control element, may have interrelated structures, such as
physical proximity or similar structures, and are best analyzed
as a lumped single system. A single local controller may best
control a town, factory, wind-farm, or solar complex, so far
as interaction with a larger global grid is concerned. What
is extraordinarily useful in our model is that we can employ
a “divide and conquer” strategy. We can interactively com-
pute homogeneous subsystems and apply “divide and conquer”
strategies to simplify control and speed up response.
A. Economic Justification
Renewable sources and nontraditional loads have the potential
to improve the economics of electric power delivery. However,
their characteristics differ substantially from standard power
components, and new methodologies are necessary to integrate
them in an economic and secure fashion. Because of their
nontraditional dynamics, we have to control the micro-grid to
achieve performance with regard to cost, resiliency, and quality
of service. The cost can be provided by the designer and/or
operator of the micro-grid in the form of rules that are included
in the criterion. The cost at the device level is determined by the
contribution of the device energy generation or consumption to
the global micro-grid cost. For instance, how much the cost of
a solar panel contributes to the total cost of the global operation
can be specified by a rule. Resiliency is incorporated in the
subsystems, and the quality of service, with regard to voltage
IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015
and frequency regulation, can be specified by rules with small
thresholds on variability. Catastrophic cascading power failures
of interconnected systems are also a driver for implementing
power delivery as a network of micro-grids.
Current software was designed to identify on the fly pos-
sible cascading failure modes anywhere in the system and to
use this information to initiate isolation of failing systems and
creation of a decomposition of working systems into indepen-
dently operating subsystems. This was a perspicacious and
laudable goal. But now a constantly increasing proportion of
generated power comes from widely distributed nontraditional
sources with different operating characteristics. The new dis-
parate sources of large irregular variations are very highly
distributed, and were not factors in the design of current sys-
tem controllers for distribution of power. We see no way to
use traditional centralized control, due to random combinations
of these disparate power sources. A huge obstacle is figuring
out the effect of a disturbance in one subsystem on a distant
subsystem, and then communicating to the distant systems fast
enough to disconnect, break into self-standing subsystems, and
stabilize them.
This paper proposes a global distributed control system with
no central arbiter, as opposed to a centralized architecture
as in [3]. We formulate this system using measurements of
deviations from “local invariants.” These invariants (gauges)
characterize what should be happening if the subsystems are
interacting properly. The invariants are functions of voltage,
frequency, VARS, and cost. These functions are called gauge
functions. Gauges have a constant value along the optimal tra-
jectory. For each device, we construct the gauges that remain
constant. Large deviations from invariant behavior are what
constitute the warning signals for initiating safe disconnec-
tion into subsystems. So disconnection is to be based on the
size of deviations from expected measured invariant behavior
of interactions of subsystems. In other words, observed devi-
ations from invariance can be used to drive the controllers
that actuate separation of the global system into self-standing
subsystems when subsystems collapse.
The control system is determined by a Hamiltonian for
each device under control. This Hamiltonian is extracted from
rules characterizing the device by a formal procedure. The
procedure involves, in addition to the construction of the
Hamiltonian, the generation of state, co-state, and control
trajectories.
The formulas for the gauge invariants and the Hamiltonians
for controlled elements establish the local state of the micro-
grid and its dynamics. Our hybrid systems methodology based
on constrained calculus of variations [12] gives an algorith-
mic approach to compute the control system variables. We
construct input–output equations for suggested self-standing
subsystems. These are coupled by the connections between
them, similar to a Tellegen decomposition [18]. These cou-
pled equations are used as constraints. To make this feasible,
the equations for each putative subsystem are obtained by an
averaging method from particle physics. We design a cost
function which evaluates the risks of catastrophic failure
based on a given control scheme as a function of measur-
able states of subsystems. The controls for the system are
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM
Fig. 2. Overview of our architecture for intelligent control and management.
chosen to minimize the cost subject to the constraints. With
real time computation that we cannot describe here, failure
to compute a controller of acceptable cost cause the separa-
tion of the system into independent subsystems. Simulation
is to be used to test decompositions into subsystems for per-
formance characteristics. There are many ways to break up
the graph of interconnections of sources and sinks of power
use into self-standing systems, which are best remains to be
analyzed.
B. Architecture of the Intelligent Feedback Control
Fig. 2 illustrates our proposed architecture for the network
of element controllers connected to the micro-grid manage-
ment server. The convention we use in the figure is that the
programmable signal controller (PSC) boxes represent pro-
grammable signal conditioners that transform digital signals
from a computer to device signals and vice versa. PSCs are
crucial components for our intelligent control architecture and
must be designed with care to effectively interface the intelli-
gent control with the physical devices. In Fig. 2, shaded wide
arrows represent outside signals, and the thin arrows represent
computer signals. The fundamental functions of the micro-grid
management server are to convey the optimization criteria and
goal settings for each of the element controllers, and to receive
real-time data from the network. In Fig. 2, the micro-grid man-
agement server abstracts data from the element network to use
in learning, and makes it available back to the element network
for restarting operations. The micro-grid management server
also schedules the interaction with the element network, com-
posed of multiple active element controllers. The micro-grid
management server is implemented as an independent agent,
so it maintains the distributed nature of the system. In contrast
2967
to a centralized system where a central server manages the
activities of elements connected to it, the micro-grid manage-
ment server allows all of the element controllers to operate
independently, and only communicate with the server when
there is a reason (such as new data, or an update to the cri-
terion). The distributed implementation is based on creating
a processor for each element controller and each PSC. The
distributed network is implemented in the cloud.
The element controllers are the active participants in imple-
menting the intelligence of the micro-grid. They implement
conventional feedback control, inductive active learning, and
intelligent feedback control. The intelligent feedback control
involves the implementation of rule-based control laws that
are partially learned from the sensory data. They control one
or more devices whose dynamic load depends on variations in
the environment, or in the operation of the device. An example
of the first kind is an air conditioning/heating unit; an exam-
ple of the second kind is equipment such as a crystal-growing
furnace that draws loads from 10 to 100 kW hours during
its operation. Their inputs and outputs are illustrated in the
bottom left of Fig. 2.
As an example of the inputs to the element controller, con-
sider the gas turbine in the first micro-grid example, illustrated
in Fig. 6. The element criteria for this gas turbine includes
a soft rule to keep the combined power from the solar panels
and the gas turbine within a specified range that reduces the
need for power from the utility by 15%. The element crite-
ria also includes cost information regarding turning on and off
the gas turbine, so it can achieve a low cost operation. The
element settings include the threshold that is needed to achieve
the 15% reduction, and the cost factors for the gas turbine. The
aggregated state estimate includes the power level generated
by the solar panels, the power level generated by the utility,
2968
Fig. 3. Element controller architecture.
and the power being delivered to the load. It also includes
costs incurred by other aspects of the micro-grid. This infor-
mation allows the element controller to decide when to turn
on or off the gas turbine. The micro-grid manager status will
provide updated information on the status of the solar panels
and the load, when the element controller queries it. The sen-
sor data for the gas turbine may include its status (on or off),
its range on voltage, and a measurement on capacity.
The sensory signals and actuator command signals are not
routed through the element network, to increase reliability of
the element. If the element controller is disconnected from
the network due to unwarranted interruption, it will continue
functioning as a conventional feedback loop that maintains the
active device functionality and issues an alarm to the micro-
grid management server. For this reason, our design separates
the sensor and actuator signals from the communication bus.
A detailed block diagram describing the functional blocks
of the element controller is illustrated in Fig. 3. The archi-
tecture of the element controller, for the most part, can be
implemented using digital signal processing [19]. The control
functionality of the element controller is realized through five
feedback loops: 1) conventional feedback loop; 2) inductive
active learning loop; 3) synchronization loop; 4) intelligent
controller loop; and 5) amalgamator controller loop.
These loops are coupled and they synchronize to achieve
the overall desired functionality.
1) Conventional Feedback Loop: This loop (connecting the
controller, actuator model, dynamics model and state estimator
with inputs from sensors, through a PSC, and element settings
and criteria, and outputs control action through the actuators)
implements the classical actuator-sensor control. In most oper-
ations, it is implemented through one or more proportional-
integral-derivative controllers. The loop is designed to fulfill
requirements such as rise-time, overshoot limitation, phase and
gain margins, and bandwidth signal-to-noise ratio specifica-
tions. Because we expect the sensory signals to be affected by
environmental noise, the state estimator is designed to generate
estimates that respond to this uncertainty.
2) Learning Loop: The inductive active learning loop
adds an inductive learning controller to adapt the controller,
IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015
dynamics model and state estimator with the dynamic ele-
ment rules. Most installations that are already in operation
will have an existing conventional loop, so the other loops
in the element controller will be compatible with the exist-
ing loop. For new installations, the element controller is
designed in a modular fashion, so it is possible to modify
loop components as necessary. An example of the need for
a learning loop is a crystal furnace in which the thermal
control is well-understood and implemented through a conven-
tional feedback loop, but the magnetic containment dynamics
of the plasma is exceedingly difficult to model with conven-
tional control techniques. However, the combination of sensory
data with inductive learning and Maxwell equations expressed
with rules may provide a dynamic model that is locally valid
and can be used to design on-line modifications to the con-
troller so that the resulting feedback loop produces the desired
behavior.
This example points out the need for inductive learning as
a means to implement effective feedback control when the
dynamics can only be partially identified from historic sensor-
actuator data. We construct a local model in real-time using
our inductive active learning technique. Our inductive active
learning technique constructs operational rules that character-
ize the dynamics model of the device under active control
(see [20]). It modifies the state estimator, the dynamics model,
and the controller to minimize error criteria between the
actual sensor-actuator data and the modeled sensor-actuator
estimates. One of the fundamental hard rules in the induc-
tive active learning loop is to ensure that no changes to the
model will lead to unstable behavior on the micro-grid. This
requirement implies that the inductive active learning con-
troller must include constraining rules of adaptability authority
of parameters and structures of the controller, dynamics model,
and state estimator that ensure input–output stability for the
micro-grid.
Awareness of the effect of the device on the micro-grid
via “awareness rules” allow the learning loop to participate
in a synchronization strategy with the amalgamator to achieve
the desired micro-grid behavior.
Knowledge uncertainty is unavoidable, even in simple grid
systems. The structural modifications of any one of the
controller, dynamics model, and state estimator, have one
of the fundamental purposes to improve the quality of the
models. This requires increasing the input–output matching
of the impedance of the model with the actual impedance of
the device, and thus reducing structural uncertainty. In gen-
eral, we will represent impedance with an operator, because,
in the micro-grid domain when nontraditional elements are
present, single frequency (ac) representation of impedance is
not enough to characterize the dynamics of the device.
3) Synchronization Loop: When multiple active devices
form part of a micro-grid, the classical approach is to intro-
duce interdevice constraints to the specification of each of
the active controllers. This requires extensive modeling and
is specific to each micro-grid. Our approach is substantially
different. We use a concept developed in particle physics in
which the interaction of a single particle with the rest of
the particles is mediated by an “average model” called the
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM
mean-field approximation. In our case, each element interacts
with the rest of the elements in the micro-grid by participat-
ing in a dynamic synchronization with the mean field element:
the controller of the element participates in a synchronization
event with the amalgamator controller representing the rest of
the micro-grid.
The essence of this synchronization involves the construc-
tion of a variational model of the device under control
representing the aggregated micro-grid. The synchronization
controller reaches and maintains a steady interaction when an
invariance (gauge) condition between the controller dynam-
ics and the amalgamator dynamics is attained and maintained.
When external events occur and this invariance condition
fails, the interaction between the controller and amalgamator
controller changes the control signal to reacquire invariance.
The information needed to build and maintain the variational
model of the amalgamator requires data from the network.
The network data is interpreted by the amalgamator as sen-
sory data. The amalgamator constructs an aggregated state
estimate projection to the space of the device under control.
The aggregated state estimate projection is computed from the
micro-grid state and the element state.
The synchronization loop is affected by uncertainty in the
controller, the state estimator, and the amalgamator controller.
We impose a quality synchronization level, high enough to
guarantee satisfactory operation, measured in terms of the
spread around the mean field equilibrium.
4) Intelligent Controller Loop: The intelligent controller
loop implements the learning loop procedure, the compat-
ibility procedures with the synchronization loop, and with
the conventional loop. It involves the flow of signals for
knowledge acquisition, and the generation of incremental
updates to the device model, controller, and state estima-
tor to respond to dynamic events. The mechanical aspect
of the intelligent loop procedure is unique to us. Classical
learning loops are query-answering procedures implemented
through inference mechanisms such as modus ponem and rule-
chaining strategies. Our approach involves the transformation
of the active rules in the dynamic element rule repository
into constraints and the transformation of the current query
to optimization-tracking criteria.
In a network of classical devices, loads and generators,
the Kirchhoff and Ohms laws accurately describe the energy
flow and state of the devices in the network. Computing
dynamic currents and voltages in the micro-grid can be
achieved by extracting the Lagrangian or equivalently the net-
work Hamiltonian that are used in the element controllers.
However, with nonclassical devices or sources that are perva-
sive in a micro-grid, network Hamiltonian and the Hamiltonian
devices are almost impossible to determine by classical net-
work procedures. Therefore, we use rules to characterize the
dynamic behavior of the system. The rules that character-
ize the dynamic behavior of nonclassical components are of
three different types; they may come from physical principles,
desired operational behavior for the multiobjective optimiza-
tion, and/or from sensory observations. For example, voltage
regulation requirements (1%–4%), frequency spreads, and PF
ranges can be expressed with soft rules.
2969
The dynamic element rule repository is composed of three
types of rules: 1) absolute rules; 2) hard rules; and 3) soft
rules. Absolute rules represent operational principles that must
be satisfied for the operation of the device under control.
Examples of absolute rules are Kirchhoff voltage and cur-
rent laws, and Ohmic relations. Absolute rules having to do
with physical principles are provided in the controller. Hard
rules are rules that must be satisfied for adequate performance,
and soft rules are acquired by inductive active learning during
the operation of the micro-grid or from phenomenological or
empirical observations. An example of a hard rule for a bat-
tery is that the charge never goes below a specified level. This
type of rule may be provided by the person familiar with the
device. An example of a soft rule is to store the power from
the solar panel in a battery when the price from the utility
is low and use the battery during peak hours when the price
from the utility is high. This type of rule would be provided
by the micro-grid operator. It is also possible that rules may
be learned during operation. For example, the price used to
indicate storing solar power may be adapted during operation.
In our approach, soft rules are transformed to inequality con-
straints, while absolute and hard rules are transformed into
equality constraints. The truth-value of the rules is translated
into numerical binary values: 0 for false, and 1 for truth. Soft
rules have values that can be scaled into a closed [0, 1] interval.
Soft rules transform the problem from a pure logical infer-
ence to a hybrid one involving both inference and probabilistic
extraction of control laws for the control elements that are in
synchrony with the other control elements. The synchrony is
established continuously with respect to the amalgamator con-
trol law. Kohn and Zabinsky [21] showed that this provides
active robust synchronization of the element with the other
control elements and the grid management system (see Fig. 2).
During operation, only the rules that are active are trans-
formed into functional constraints. This is essential for real-
time computational purposes. An absolute or hard rule is
active when it is violated (value of 0). A soft rule is active
when its truth-value is below a preagreed threshold. The
threshold may vary among soft values. Soft rules that are
learned through inductive learning include the functional struc-
ture and the logic value. The chaining of the active rules is
established by the interaction in the functional representation.
The inductive learning loop is realized dynamically by con-
structing a Lagrangian that adjoins the functional relations of
the active rules to the criterion. The off-diagonal elements
of the Hessian of the Lagrangian associated with the opti-
mization problem represent the chaining of the corresponding
active rules.
The intelligent controller implements a discovery engine
(an automated search engine) for extracting the active rules
as a function of sensor and actuator signals. It also gen-
erates new rules using a repair mechanism that forms part
of the inductive learning loop procedure. This repair mech-
anism is unique to our approach [22]. It operates by adding
corrective increments to the control signal as a function of
detected operational error. This is one of the fundamen-
tal aspects of the learning capabilities of our intelligent
controller.
2970 IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015

The energy and potential functions are obtained from all the
elements in the micro-grid

E(x, ẋ) = Ek (x, ẋ)
k

V(x) = Vk (x)
k

due to conservation of energy. The Lagrangian model of cir-

Fig. 4. Amalgamator controller architecture.
5) Amalgamator Control Loop: The amalgamator con-
troller, appearing as a box in Fig. 3, implements the projection
of the micro-grid dynamic model into the domain of the
element controller. A more detailed architecture of the amal-
gamator controller is illustrated in Fig. 4. The purpose of
the amalgamator is to establish explicitly the constraint of
the device under control with respect to the micro-grid. The
unique aspect of our approach is to provide effective synchro-
nization and reacquisition of synchronization using an optimal
synchronization strategy between the amalgamator and the
controller.
It is important to address the issue of how to characterize
uncertainty. In most implementations of smart grid control and
management, uncertainty is described by assuming a probabil-
ity distribution on the state. We consider this unsatisfactory
because the probability distribution is unknown and does
not satisfy the Hadamard condition; meaning that the dis-
tribution should be continuously dependent on the observed
data. Our approach is to represent uncertainty on the state,
the structures, and the signals with mean and spreads that
are continuously computable from the data without assum-
ing any probability distribution. This approach requires the
propagation of sets representing the states and the structures.
This method is referred to in the literature as differential set
inclusion.
C. Tomograph for Estimating the Lagrangian via Inferencing
Given a micro-grid, a tomograph is used to construct the
Lagrangian for nonstandard elements in the network of the
micro-grid. We characterize the micro-grid with a Lagrangian
that is the difference between the energy and potential
field
L(x, ẋ) = E(x, ẋ) − V(x)
cuits is well-known. For standard elements, such as capacitors,
inductors, transformers, classic generators and classic loads,
the relationships between current and voltage across the ports
are known. However, the Lagrangian for a rule-based system
must be learned. The relationship between current and voltage
across the ports for nonstandard elements, such as wind tur-
bines, solar panels and smart loads, are only partially known
and must be estimated from sensory data.
We use a dynamic tomograph (see [23] and [24]), and an
identification process to improve our estimate of Ek (x, ẋ) and
Vk (x) for each k corresponding to a nonstandard element. The
tomograph provides functions of the form
Ek (xk , ẋk , other states)
Vk (xk , other states)
so that we are building the Lagrangian “on the fly.” This
is what we refer to as structural adaptation. Once we have
an estimate of the Lagrangian, we construct the estimated
Hamiltonian
Ĥ(x(t), p(t)) = p(t)T x(t) − L(x(t), ẋ(t))
where p(t) is the momentum in the micro-grid.
D. Hamiltonian Dynamics
Once we have the estimated Hamiltonian, we used it to
construct the control Hamiltonian
HC (x(t), p(t), u(t)) = Ĥ(x(t), p(t)) + VC (x(t), u(t))
where u(t) is the control signal, and VC (x(t), u(t)) is a known
potential function. The rules that reside in the “dynamic ele-
ment rules,” and the Hamiltonians of individual devices are
maintained in the “inductive learning controller” (see Fig. 3).
The Hamiltonian model
HC (x(t), p(t), u(t))
is constructed from the rules, physical principles, historical
data, and adapted to current sensory data. The procedure in the
inductive learning controller obtains the optimal control law
by constructing the state equation (1), co-state equation (2),
and meta-control equation (3), shown as follows:

where the state for the kth element is the current flowing the ∂x(t) ∂HC  ∂φi (x, p, u)
= + λi (1)
element in the network ∂t ∂p ∂p
i
 t ∂p(t) ∂HC  ∂φi (x, p, u)
xk (t) = ik (τ )dτ. =− − λi (2)
0
∂t ∂x ∂x
i
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM
Fig. 5. Element synchronization via mean field.
 −1
∂u(t) ∂ 2 HC  ∂ 2 φi (x, p, u)
=− + λi
∂t ∂u2 ∂u2
i
 
∂HC  ∂φi (x, p, u)
× + λi . (3)
∂u ∂u
i
The algorithm represented by (1)–(3), is implemented with
efficient numerical integration routines. In most cases, we
use Runga–Kutta of the third order, specifically modified
for [21], [25], and [26].
If the costate equation
∂L(x(t), ẋ(t))
p(t) = (4)
∂ ẋ
is regular (Dirac), then we have a unique solution
ẋ(t) = f0 (x(t), p(t)) (5)
but if the Hessian of the Lagrangian is not invertible (not
regular Lagrangian), then from the inverse function theorem,
we cannot solve ẋ in (4). This case is likely for our rule-
based system. Then, we separate the state vector into two parts:
1) the physical state components and 2) the constraint state
components. For every device in the network, energy must be
conserved, so the flow (current) across the device is a function
of voltage and is expressed in a Hamiltonian system.
The inductive learning controller takes the rules from the
dynamic element rules repository, and transforms the logic
rules, denoted ϕQ (x) and ϕ (k) , into continuous functions,
denoted φ̆Q (x) and φ̆ (k) . The x vector contains the state
estimate and the variables associated with the rules. The
Lagrangian L(x; ϕ̆Q (x), ϕ̆˙ (k) , ω) is estimated using the contin-
ualized rules and the method of Lagrange multipliers. The ω
vector represents the level of unsatisfaction of the soft rules.
In order to minimize the Lagrangian, the rates of ẋ(t) and Ṁ(t)
are computed, where M(t) is a positive semi-definite matrix
used to estimate the inverse of the Hessian of the Lagrangian.
For details, refer to [27] and [28]. Now, the dynamics of the
device is formulated as G(ẍ, ẋ, x) and the Hamiltonian Ĥ(x, p)
can be computed. Thus, we have estimated the Hamiltonian for
a nonclassical device from rules! Given the Hamiltonian, we
update according to changes in the system. Then, we use our
meta-control approach (see [25]), to optimize the Hamiltonian
and determine optimal control actions. This provides the feed-
back control loop for the inductive learning controller. Once
the local feedback control loop for an element is designed,
2971
all of the element control systems must be synchronized for a
coherent global system. Fig. 5 illustrates the synchronization
strategy, which is carried out via a companion element that
aggregates the dynamics of all elements.
III. M ICRO -G RID C ONTROL E XAMPLES
The micro-grid intelligent control and management system
is applied to two examples. The first example, shown in Fig. 6,
is a micro-grid with three generators; two solar and one gas
turbine. The data for this example came from a hotel com-
plex center, which combines utility energy with their two solar
devices and a gas turbine to reduce the electric power cost.
The load is an air conditioning unit for an office building with
varying requirements for air conditioning based on historical
weather conditions measured by sensors. The air conditioning
load accounted for a high percentage of the total load, so the
rest of the typical loads were ignored in this example. Their
goal is to reduce power from the utility by 15% at peak hours.
In the first example, the central objective of the simulation
was to demonstrate the acquisition of the model of each of
the electric devices in the micro-grid from sensory data. This
sensory data is represented in the data base of rules as soft
rules. For example, there is a rule that characterizes the load
as a function of time and temperature: if time is noon, and
temperature is greater than 20 ◦ C, turn on the gas turbine
and combine power with solar panels. Many rules are of this
sort, including ranges of outside temperature, time, sunlight
levels, and rules to reduce transient behavior while charging
electric cars.
A simulation was run for 10 000 min, with historic data for
all sensor data. The relative cost of energy at the load for the
controlled micro-grid, with respect to the open loop uncon-
trolled system, is shown in Fig. 7. Fig. 7 illustrates how the
control system learns about operation of the components of the
grid. The line in the figure is the mean performance associated
with conventional control, provided by the implementer of the
micro-grid. Notice that the performance, as measured by the
cost, is initially higher than the average cost of the conven-
tional system. The transients that are observed in the curve are
caused by high variations in the load and renewable generation.
The control system for each of the DERs is relatively smooth.
Each of the four element controllers (associated with the
two solar devices, the gas turbine, and the load) is optimizing
locally with the mean field in the amalgamator as a constraint.
As the controllers learn the characteristics of the dynamics,
they are updated through the mean field. The result of the
sequential optimizations is to locally synchronize the four
controllers. This process is repeated iteratively. The central
idea here is that the update of the mean field is done fast
enough (on the order of once every 4 ms) to minimize the
effects of delay, which are included in the rules defining the
Hamiltonian.
At time zero, in Fig. 7, the relative cost of energy at the
load compared to the open loop system is zero. In the early
minutes, the element controllers are actively learning the vari-
ations due to weather impact on the load and solar panels. At
10 min, the element controllers begin to improve performance
2972 IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015

Fig. 6. Micro-grid example with three generators, two solar and one gas turbine, and an air conditioning unit as the load.

Fig. 7. Relative cost of energy at the load of the controlled micro-grid, compared to the uncontrolled system.

due to the constructed Hamiltonian models. The cost savings
decrease until approximately 100 min, and then stabilize at
approximately 30% saving for the rest of the time horizon.
The simulation demonstrates the value of synchronizing
the generation facilities with the varying load, and adjusting
the scheduling of their contribution to the load dynamically.
A significant portion of the savings were realized by shutting
down the gas turbine for periods where the demand for air
conditioning was slight.
The second example, illustrated in Fig. 8, represents a small
city with a distribution/transmission network in which some
standard generation units have been replaced with DERs. To
demonstrate scalability, the second example includes 30 buses,
six DER generators (three are wind turbines), and one non-
standard load (see Fig. 8). This IEEE 30-bus network has been
used in [29], and in others [30].
The operational characteristics of the DERs are taken
from [31]–[35]. Characteristics of the PSCs were provided
by a communication cellular company, with average point to
point delay of 3.6 ms and a standard deviation of 1.8 ms.
The average latency of our control software was estimated to
be 100 μs on a 64 ARMA processor. The simulation manage-
ment of the network, run on a PC, had an average duty cycle of
600 μs. Additional details on the numerical parameter settings
are available from the authors upon request.
A simulation was run for the network with no active control,
and again for the network with our controller, and the three
wind turbines included batteries that were under active control.
Table I presents summary results comparing the two sim-
ulations. The first row in Table I shows the average PF
without control, and then the average PF over the four simu-
lated weeks. As the control system learns from the data, the
KOHN et al.: MICRO-GRID DISTRIBUTED INTELLIGENT CONTROL AND MANAGEMENT SYSTEM
Fig. 8. IEEE 30 bus network, from [26].
TABLE I
S IMULATION R ESULTS ON THE 30 B US N ETWORK
PF increases significantly. The second row of Table I shows
the voltage regulator, and illustrates how the control system
improves efficiency as the voltage regulator decreases toward
the ideal 1%. The third row shows the third harmonic, as
a metric representing loss. The results with the control system
indicate increased efficiency.
The cost of the system, with and without control, was esti-
mated using terms for the cost of battery aging, cost for turbine
maintenance, and cost for doing control. The average simu-
lated cost for the network with control was 70% of the cost
with no active control.
IV. C ONCLUSION
The proposed architecture allows distributed control of
devices with only local models built by active learning on
sensor information. The two examples demonstrate that the
control methodology scales well. This is due to its fully dis-
tributed design with self-synchronization (as given by rules),
where the computation per node remains the same as the
2973
number of nodes increases. The control system has the ability
to develop models of the devices and the network that are valid
for the entire range of operation, even when analytical models
are nonexistent or difficult to approximate. Our approach of
using local models that are dynamically updated is practical.
The distributed nature of our control (at the edge) increases the
robustness, resiliency, and cost-effectiveness of the micro-grid.
R EFERENCES
[1] A. L. Dimeas and N. D. Hatziargyriou, “Operation of a multiagent sys-
tem for microgrid control,” IEEE Trans. Power Syst., vol. 20, no. 3,
1447–1455, Aug. 2005.
[2] H. Jiayi, J. Chuanwen, and X. Rong, “A review on distributed energy
resources and MicroGrid,” Renew. Sustain. Energy Rev., vol. 12, no. 9,
pp. 2472–2483, Dec. 2008.
[3] J. Jimeno, J. Anduaga, J. Oyarzabal, and A. G. de Muro, “Architecture
of a microgrid energy management system,” Eur. Trans. Elect. Power,
vol. 21, no. 2, pp. 1142–1158, Mar. 2011.
[4] A. Keyhani, Design of Smart Power Grid Renewable Energy Systems.
Hoboken, NJ, USA: Wiley, 2011, chs. 5–6.
[5] A. L. Dimeas, A. Tsikalakis, G. Kariniotakis, and G. Korres,
“Microgrids control issues,” in Microgrids Architectures and Control,
N. Hatziargyriou, Ed. Hoboken, NJ, USA: Wiley, 2014, ch. 2.
[6] M. Kubisch, H. Karl, A. Wolisz, L. C. Zhong, and J. Rabaey,
“Distributed algorithms for transmission power control in wireless sen-
sor networks,” in Proc. Wireless Commun. Netw., vol. 1. New Orleans,
LA, USA, Mar. 2003, pp. 558–563.
[7] T. Mao, G. Feng, L. Liang, S. Qin, and B. Wu, “Distributed energy-
efficient power control for macro-femto networks,” IEEE Trans. Veh.
Technol., to be published.
[8] W. Kohn, V. Brayman, P. Cholewinski, and A. Nerode, “Control in
hybrid systems,” Int. J. Hybrid Syst., vol. 3, no. 3, pp. 109–150, 2003.
[9] W. Kohn, V. Brayman, and A. Nerode, “Control synthesis in hybrid
systems with Finsler dynamics,” Houston J. Math., vol. 28, no. 2,
pp. 353–375, Jan. 2002.
[10] W. Kohn, V. Brayman, and J. B. Remmel, “Multi-echelon inventory plan-
ning system: Multiple agent hybrid control (MAHCA) implementation,”
Int. J. Hybrid Syst., vol. 2, nos. 1–2, pp. 105–128, Jan. 2002.
[11] W. Kohn and B. Cummings, “Free flight ATC using hybrid agent
systems,” Int. J. Hybrid Syst., vol. 10, no. 9, pp. 10–17, 1997.
[12] W. Kohn, A. Nerode, and J. B. Remmel, “Hybrid systems as Finsler
manifolds: Finite state control as approximation to connections,” in
Hybrid Systems II (LNCS 999), P. Antsaklis, W. Kohn, A. Nerode, and
S. Sastry, Eds. Berlin, Germany: Springer, 1995, pp. 294–321.
[13] W. Kohn and J. B. Remmel, “Hybrid dynamic programming,” in Hybrid
and Real-Time Systems (LNCS 1201), O. Maler, Ed. Berlin, Germany:
Springer-Verlag, 1997, pp. 391–396.
[14] W. Kohn, Z. B. Zabinsky, and V. Brayman, “Optimization of algorithmic
parameters using a meta-control approach,” J. Glob. Optim., vol. 34,
no. 2, pp. 293–316, Feb. 2006.
[15] B. Jacob and H. J. Zwart, Linear Port-Hamiltonian Systems on Infinite-
dimensional Spaces. Berlin, Germany: Springer, 2012.
[16] W. Kohn, A. Nerode, and J. Remmel, “Agent-based velocity control
of highway systems,” in Hybrid Systems IV (LNCS 1273), Berlin,
Germany: Springer-Verlag, 1997, pp. 174–214.
[17] J.-P. Aubin, Mathematical Methods of Game and Economic Theory.
Mineola, NY, USA: Dover, 2007.
[18] P. Penfield, Jr., R. Spence, and S. Duinker, Tellegen’s Theorem and
Electrical Networks. Cambridge, MA, USA: MIT Press, 1970.
[19] T. B. Welch, C. H. G. Wright, and M. G. Morrow, Real-Time Digital
Signal Processing From MATLAB to C With the TMS320C6x DSPs,
2nd ed. Boca Raton, FL, USA: CRC Press, 2012.
[20] H. Liu, Z. B. Zabinsky, and W. Kohn, “Rule-based control system design
for smart grids,” in Proc. IEEE Power Eng. Soc. (PES) Gen. Meeting,
Minneapolis, MN, USA, Jul. 2010, pp. 1–5.
[21] W. Kohn and Z. B. Zabinsky, “Hybrid dynamic programming for rule
constrained multi-objective optimization,” in Proc. 21st Int. Symp. Math.
Program. (ISMP), Berlin, Germany, Aug. 2012.
[22] W. Kohn, V. Brayman, and J. Littleton, “Repair-control of enter-
prise systems using RFID sensory data,” IIE Trans., vol. 37, no. 4,
pp. 281–290, 2005.
[23] S. Helgason, Integral Geometry and Radon Transforms. New York, NY,
USA: Springer, 2010.
2974
[24] A. Markoe, Analytic Tomography. Cambridge, U.K.: Cambridge Univ.
Press, 2006.
[25] W. Kohn, A. Nerode, and J. B. Remmel, “Continualization: A hybrid
systems control technique for computing,” in Proc. Symp. Control Optim.
Supervis. Multiconf. Comput. Eng. Syst. Appl. (CESA IMACS), Lille,
France, Jul. 1996, pp. 507–511.
[26] Z. B. Zabinsky and W. Kohn, “Solving non-linear discrete optimization
problems via continualization: An interior-point algorithm,” in Proc. 21st
Int. Symp. Math. Program. (ISMP), Berlin, Germany, Aug. 2012.
[27] W. Kohn and J. Remmel, “Hybrid dynamic programming,” Int. J. Hybrid
Syst., vol. 4, no. 1, pp. 1–7, 1992.
[28] H. Liu, Z. B. Zabinsky, and W. Kohn, “Rule-based forecasting and pro-
duction control system design utilizing a feedback control architecture,”
IIE Trans., vol. 43, no. 2, pp. 143–152, Feb. 2011.
[29] H. Lui et al., “Social welfare maximization in transmission enhance-
ment considering network congestion,” IEEE Trans. Power Syst., vol. 23,
no. 3, pp. 1105–1114, Aug. 2008.
[30] IEEE Power Systems Test Case Archive. [Online]. Available:
http://www.ee.washington.edu/research/pstca/pf30/pg_tca30bus.htm,
http://fglongatt.org/OLD/Test_Case_IEEE_30.html, accessed Feb. 19,
2015.
[31] P. Jamieson, Innovation in Wind Turbine Design. Hoboken, NJ, USA:
Wiley, 2011.
[32] P. Weicker, A Systems Approach to Lithium-Ion Battery Management.
Norwood, MA, USA: Artech House, 2014.
[33] C. Glaize and S. Genies, Lithium Batteries and Other Electrochemical
Storage Systems. Hoboken, NJ, USA: Wiley, 2013.
[34] R. Teodorescu, M. Liserre, and P. Rodriguez, Grid Converters for
Photovoltaic and Wind Power Systems. Hoboken, NJ, USA: Wiley, 2011.
[35] M. K. Kazimierczuk and D. Czarkowski, Resonant Power Converters,
2nd ed. Hoboken NJ, USA: Wiley, 2011.
Wolf Kohn received the Ph.D. degree in elec-
trical engineering and computer science from
the Massachusetts Institute of Technology (MIT),
Cambridge, MA, USA, in 1979.
He co-founded Clearsight Systems, Inc., Bellevue,
WA, USA, and Kohn–Nerode LLC, Dover, DE,
USA. He is currently a Research Professor with
the University of Washington, Seattle, WA, USA,
in addition to being the Chief Scientist with Atigeo,
LLC, Bellevue. He has taught at City University of
New York Graduate Center, New York, NY, USA;
Courant Institute of Mathematical Sciences, New York; New York University,
New York; Rice University, Houston, TX, USA; Wayne State University,
Detroit, MI, USA; and the University of California–Berkeley, Berkeley, CA,
USA. He has published several refereed papers and book chapters, and has
been a keynote speaker at several control conferences.
Dr. Kohn was a recipient of the Engineer of the Year Award from Boeing,
Seattle; Lockheed, Clearlake, TX; and NASA, Houston Space Center, TX,
and the Teaching Award from MIT.
IEEE TRANSACTIONS ON SMART GRID, VOL. 6, NO. 6, NOVEMBER 2015
Zelda B. Zabinsky received the Ph.D. degree
in industrial and operations engineering from the
University of Michigan, Ann Arbor, MI, USA, in
1985.
She is currently a Professor with the Department
of Industrial and Systems Engineering, University
of Washington, Seattle, WA, USA, with adjunct
appointments in the Department of Electrical
Engineering, Mechanical Engineering, and Civil and
Environmental Engineering. She teaches courses
in operations research. She has published several
papers in the areas of global optimization, algorithm complexity, and opti-
mal design of composite structures, and the book Stochastic Adaptive Search
in Global Optimization (Kluwer Academic, 2003), which describes research
on theory and practice of algorithms useful for solving problems with multi-
modal objective functions in high dimension.
Prof. Zabinsky was a recipient of the Annual Teaching Award in Industrial
Engineering at the University of Washington several times. She is active in
Institute for Operations Research and the Management Sciences and is an
Institute of Industrial Engineers Fellow.
Anil Nerode received the Ph.D. degree in mathemat-
ical logic from the University of Chicago, Chicago,
IL, USA, in 1963.
He is a Goldwin Smith Professor of Mathematics
with Cornell University, Ithaca, NY, USA. He is cur-
rently collaborating with his long-term co-author in
hybrid optimal control, W. Kohn, developing quan-
tum hybrid control, and co-founded Kohn–Nerode
LLC, Dover, DE, USA. With J. Myhill, he proved
the Myhill–Nerode Theorem, specifying necessary
and sufficient conditions for a formal language to be
regular. His current research interests include mathematical logic, the theory
of automata, computability and complexity theory, mathematics of artificial
intelligence, control engineering, and quantum control of macroscopic sys-
tems.
Prof. Nerode has been a recipient of numerous awards. He is an Editor
of several journals and is frequently a Keynote Speaker. In 2012, he became
a Fellow of the American Mathematical Society.