Effect of Loss of Load Probability Distribution
on Operating Reserve Demand Curve
Performance in Energy-Only Electricity Market
Sreelatha Aihloor Subramanyam, Student Member, IEEE, and Xuewei Zhang, Senior Member, IEEE
 rL
Abstract² In energy-only electricity markets, the operational
reserve demand curve is a scarcity pricing mechanism adopted to
address the shortage of reserves and incentivize the generators. It
is constructed based on the assumption of the normality of loss of
load probability, which is estimated from reserve error data. In
this Letter, the historical record of reserve error is collected and
analyzed to test the assumption and find the fittest probability
distribution. It is found that the normal distribution is generally
not as good as other distributions like log-logistic, general gamma,
and Weibull. Using a dynamic simulation framework, the installed
reserve margins with different distributions are evaluated over a
fifty-year period, demonstrating that the choice of distribution
could significantly impact the subsequent pricing calculations and
therefore the reserve margin in an energy-only electricity market.
This work is of potential importance and immediate relevance to
system reliability and resource adequacy in power systems.
Index Terms²Loss of load probability, operating reserve
demand curve, energy-only electricity market.
NOMENCLATURE
CDF Cumulative distribution function
ER Reserve usage (in MWh) in a year
FC Annualized fixed cost
GEV General extreme value
GM Gross margin
GOF Goodness of fitting
ICAPY Installed capacity of year Y
IRM Installed reserve margin
Annual load growth rate
TB time-block
VOLL Value of lost load
X Minimum reserve level
ǻߎ݇ ORDC price adder of year k (in $/MWh)
I. INTRODUCTION
I NSUFFICIENT generator revenue from wholesale electricity
markets has raised concerns on grid reliability and resilience
[1]. The energy emergency alerts and load reductions in Texas
in summer 2019 also accentuate the need to improve the market
designs to maintain adequate reserves [2]. Some ISO/RTOs like
PJM adopt forward capacity market to secure reserves, making
payments to generators for capacity commitments in future. In
energy-only markets like ERCOT in Texas, generators receive
payments for the actual energy provided, which makes it more
difficult to incentivize investments on new generation to ensure
resource adequacy. To resolve potential reliability issues that
may arise, a scarcity pricing mechanism, i.e., operating reserve
demand curve (ORDC), has been proposed [3] and introduced
to ERCOT since 2014 [4]. Mathematically, the ORDC model
developed in [5,6] is a “capped” complementary cumulative
distribution function (CDF) of loss of load probability (LOLP).
This work is motivated by two fundamental questions about
ORDC. The first one is: what is the distribution of LOLP? It is
known that LOLP is obtained from the historical data of reserve
error, i.e., the difference between hour-ahead (forecast) reserves
k Index of years in 8-\HDUZLQGRZ <í<í«< and real-time (hourly-average) reserves [4-7]. The histogram of
K-S Kolmogorov-Smirnov reserve errors is fitted as a normal distribution with its mean and
LA,Y Actual peak load of year Y
standard deviation evaluated by statistical analysis and then its
LF,Y Forecast peak load of year Y
LMP Locational marginal pricing complementary CDF can be constructed. But is the normality
LOLP Loss of load probability assumption valid and accurate enough? The second question is
NCAY New capacity addition in year Y about the availability of effective approaches to evaluating the
ORDC Operating reserve demand curve performance of ORDC. An ERCOT market evaluation report
PBMCL Probability of (reserve) below minimum contingency [8] acknowledges the difficulties in pinpointing the practical
level
implications and effectiveness of ORDC, as enough generation
PR Annual plant retirement rate
Q-Q Quantile-Quantile capacity is currently available, and no major impactful events
R Reserve level have taken place. Meanwhile, the report recognizes insufficient
RAFP Risk-adjusted forecast profit revenues of generators even with ORDC implementation. To

Manuscript received December 7, 2019; revised: February 16, 2020.
Sreelatha Aihloor Subramanyam is a Ph.D. candidate within the Sustainable
Energy Systems Engineering Program at Texas A&M University-Kingsville,
Kingsville, Texas 78363, USA (email: sreelatha.aihloor_subramanyam@
students.tamuk.edu).
Dr. Xuewei Zhang is with the Department of Electrical Engineering and
Computer Science at Texas A&M University-Kingsville, Kingsville, Texas
78363, USA (email: xuewei.zhang@tamuk.edu)

'LJLWDO2EMHFW,GHQWL¿HU73:56
0885-8950 © 2020 . Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 2

increase ORDC price, [9] discusses the rightward shift of the
normal distribution of LOLP. Currently, ERCOT uses a shift by
0.25 of the standard deviation. From a theoretical point of view,
its justifiability remains an issue. Practically, it is of interest and
relevance to gain quantitative knowledge on how the shift will
affect the ORDC performance. Our recent work demonstrates a
dynamic simulation framework that serves for the purpose [10].
Therefore, the objectives of this work are to (i) test how well
the LOLP fits into a normal distribution, (ii) identify probability
distributions that have better agreement with data, and (iii) use
the aforementioned dynamic simulations to examine the effect
of the choice of LOLP distributions on ORDC performance.
Our main contributions include (a) the introduction of various
statistical tests, which generates new insights on the distribution
of LOLP, and (b) the evaluation of the long-term performance
of ORDC, which shows the promise of replacing shifted normal
distribution with other distributions. The models and methods
used in this study are described in Sec. II, where we also analyze
data to address objectives (i) and (ii). Sec. III presents some
ORDC simulation results to discuss the effect of distributions
of LOLP. Conclusions are drawn in Sec. IV.
the reserve errors during hours 7-10 and during summer 2018
against normal distribution. A clear departure from normality
(straight dot-dashed line) is observed. Using software Easyfit,
the best-fitted distribution for both sets of data is found to be
the general extreme value (GEV) distribution, as shown in Fig.
1(c-d). Further, 3 GOF tests (Kolmogorov-Smirnov, Anderson-
Darling, and ߯ ଶ ) on if the data obey normal distribution (null
hypothesis) are conducted [11]. In the 24 data sets, the tests of
all but two reject the null hypothesis (the 2 exceptions are Fall
Hours 7-10 and Spring Hours 23-24-1-2). Ranked by the K-S
test p-values of 30 distributions, there are 8 best fits that are log-
logistic, 4 GEV, 3 general gamma, 3 Weibull, 2 normal, among
others. The comparisons of log-logistic and normal distribution
ranks in Fig. 2 also show that, in most cases, normal distribution
does not have best the agreement with the data.

II. MODELS AND METHODS

A. ORDC as LOLP-Dependent Price Adder
We first give the definition of ORDC [3]. It is a price adder to
locational marginal pricing (LMP) to recognize the value of
reserves available. A dispatch correction optimization problem
is solved in real time to minimize the sum of the correction cost
of increasing generation and the opportunity cost of shedding
loads [5,6]. The dispatch correction should not exceed the total
demand disturbance or the reserve capacity. Applying Karush-
Kuhn-Tucker conditions to these two constraints results in the
Fig. 1. (a) Q-Q plot of three-year hour 7-10 reserve error data. (b) Q-Q plot of
complementary slackness of 2 Lagrangian multipliers. The one summer 2018 reserve error data. (c) Histogram and best fit (GEV) of the data
associated with the reserve capacity constraint is used for the in (a). (d) Histogram and best fit (GEV) of the data in (b).
construction of ORDC: 0 if the real-time reserve can meet extra
dispatch-correction needs and otherwise is the difference
between the value of lost load (VOLL) and the LMP. The
probability of the latter, a decreasing function of the reserve
capacity, is the probability of (reserve) below minimum
contingency level (PBMCL). Thus, ORDC equals the product
of PBMCL and the Lagrangian multiplier. ISOs or RTOs set the
minimum reserve level (X) for system reliability. PBMCL=1 if
reserve level, R, is below X. Otherwise, PBMCL =F5í;,
where F(·) is the complementary CDF of LOLP.
B. Goodness of Fitting (GOF) Tests of LOLP Distribution
To obtain LOLP distribution, prior studies assume normality
and calculate its mean and standard deviation from historical
reserve error data (actually it is done for 24 4-hour time-blocks
(TBs), 6/day for each season). Using reserve error distribution
to approximate LOLP distribution is a current industry practice
[3-7], the validity of which will be an interesting topic for future
studies. Within the scope this work, to verify the assumption of
normality, we request ERCOT data and follow the instructions
Fig. 2. GOF ranks of normal and log-logistic distributions for the 24 TBs. 1-6:
Spring; 7-12: Summer; 13-18: Fall; 19-24: Winter; all starting with Hours 3-6.
C. ORDC Dynamic Simulation Model
We have shown evidence that normal distribution may not be
the best fit for LOLP. To examine if this affects the performance
of ORDC, we use the model from our preliminary work [10].
The model runs for 50-year economic plant life and repeats for
1000 iterations to estimate the dynamics of installed reserve
margin (IRM), i.e., the ratio of installed capacity to peak load.
in [7] to compute hourly reserve error for 3 years (2016-2018). ,Q\HDU<•WKHVLPXODWLRQSURFHVVLVDV follows. (i) From
Fig. 1(a) and 1(b) present the quantile-quantile (Q-Q) plots of WKHSUHYLRXV\HDU¶VSHDNORDGJHQHUDWH the actual peak load LA,Y
considering an annual growth rate rL and some random error.
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 3

The forecast peak load (LF,Y+i) for the ith year into the future is
estimated from LA,Y using annual growth rate rL. (ii) Given the
previous year’s installed capacity ICAP<í, the new capacity
addition before this year NCAYí1, and the annual plant retiring
rate PR, then ICAPY = NCAYí1 + (1–PR)·ICAP<í. The forecast
ICAPY+i is assumed to increase at annual growth rate r L. (iii)
From the ORDC constructed following [5,6], the price adder
ǻȆk is derived at the reserve levels (Rk = ICAPk í Lk) for each
year during the 8-\HDUSHULRGN <í<í,…, Y+3): οߎ௞ ൌ
ሺܸܱ‫ ܮܮ‬െ ‫ܲܯܮ‬ሻ ൉ ሺܴ௞ െ ܺ) (in $/MWh). (iv) To estimate the
increased revenue of a benchmark generator from ORDC, we Fig. 3. IRM (mean of 1000 samples) over 50 years under different ER values.
PXOWLSO\ǻȆk with the recorded reserve usage ER (in MWh) in
one year. In principle, the annual ORDC revenue should be the Now with ER = 5 MWh, we run simulations to study how the
accumulation of all hourly ORDC revenues. There are 2 reasons LOLP distributions affect the IRM dynamics. The results in Fig.
that in this work we simply use a yearly-averaged price adder. 4(a) show that all other distributions will establish a raised IRM
Firstly, only the data of monthly totals of reserve usage can be level from that of the normal distribution, with log-logistic
found on public records, and we currently do not have hourly being the closest and Weibull being the highest. In this work,
reserve usage info. Secondly, since the real-time reserve levels we use the same LOLP distribution throughout the 50 years. No
in the future years are unknown, the yearly average of reserve TBs or updates of the distribution are included. Nevertheless,
levels is used to evaluate the ORDC price adder for each year. the objective is to shed light on how different interpretations of
(v) Given its annualized fixed cost (FC) and the gross margin reserve errors may impact the IRM in the long run. ERCOT is
(GM) from the E/AS market, the generator’s profit each year is implementing normal distribution with a rightward shift to
߶௞ ൌ ‫ܯܩ‬௞ ൅  οߎ௞ ή ‫ܧ‬ோ െ ‫ܥܨ‬௞ . (vi) Use the ߶௞ ’s to calculate
increase the ORDC price. The results of the shifted normal
risk-adjusted forecast profit (RAFP) following [12]. If RAFPY
distributions are also plotted in Fig. 4(a). In comparison, using
” 0, then NCAY = rL · ICAPY. As the peak load grows annually,
an alternative LOLP distribution can accomplish the same (see
NCAY is required to increase the installed capacity by rL. If
RAFPY > 0, NCAY will increase linearly with RAFPY until the cases of shifted by 1.5 standard deviations and GEV or
reaching a pre-defined upper limit. More model details and general gamma), while being more statistically sound.
parameter settings are specified in [10], a preliminary work of
ours devoted to building and validating the model which is self-
consistent and can be used for accomplishing our objectives. It
is also worth noting that the model, although similar to Monte
Carlo method, does not involve any uncertainty in initial inputs
like IRM. With the future demand information being generated
stochastically, we average the results from multiple iterations
/ZD

to see the expected trajectory instead of individual sample path.

III. RESULTS AND DISCUSSIONS

Simulations are run to show the long-term dynamics of IRM.
When LOLP is a normal distribution, the effect of ER on IRM
is presented in Fig. 3. All the simulations in Fig. 3 start with the zĞĂƌ
initial IRM of 1.04. Each curve represents the dynamics of the
mean of IRMs from multiple simulations. Our simulations show
&ƌĞƋƵĞŶĐLJ;йͿ

that the mean of IRMs will be consistent as long as the number

of iterations is over several tens; more iterations will lower the
standard deviation of the IRMs. Since ER is unknown for the
benchmark generator, we use several values and keep them
constant in each simulation. To ensure the averaged IRM no
lower than 1.04, ER needs to be at least 5 MWh. More usage
/ZD^͘͘
(revenue) results in higher incentives for NCA and higher IRM.
In the case of ER = 2 MWh, the IRM will decrease over time Fig. 4. (a) IRM (mean of 1000 iterations) over 50 years with different LOLP
since generator firms do not project profits by providing reserve distributions. (b) Histograms of the standard deviations of IRMs with GEV,
Weibull, and normal distributions of LOLP.
services. On the other hand, if ER is much higher than 5 MWh,
the IRM will increase initially and reach a new “equilibrium” Fig. 4(b) is the histograms of IRM standard deviations (S.D.)
point. It is because when IRM is too high, ORDC price and of 1000 iterations with three distributions of LOLP in Fig. 4(a).
generator revenue will drop, which reduces NCA. We have Lower S.D. means less fluctuating and thus more predictable
demonstrated the effectiveness of ORDC in regulating reserve IRM. While normal distribution has the lowest IRM, it seems
levels. The model provides a self-consistent framework to to correspond to more stable IRM dynamics than others.
evaluate the performance of ORDC in energy-only markets.
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 4

IV. CONCLUSIONS
In conclusion, this work studies the LOLP distribution fitted
from historical data of reserve errors in ERCOT and examines
the effect of the LOLP distributions on the ORDC’s long-term
performance. The statistical tests show that normal distribution
is not the best fit in most cases. The simulation results of a self-
consistent model indicate that different probability distributions
used in ORDC can lead to different IRMs, implying that it may
not be necessary to shift the normal distribution to increase the
ORDC payments to ensure targeted IRMs. With the availability
of more data, future work can be done to improve and calibrate
the model for simulations at higher time resolutions.

REFERENCES
[1] U.S. Department of Energy, “Staff Report to the Secretary on Electricity
Markets and Reliability,” [Online]. Available:
https://www.energy.gov/downloads/download-staff-report-secretary-
electricity-markets-and-reliability, Accessed on: Dec. 5, 2019.
[2] M. J. Hutzler, “Texas’ Impending Reliability Issues With Wind Power,”
[Online]. Available: https://www.powermag.com/texas-impending-
reliability-issues-with-wind-power, Accessed on: Dec. 5, 2019.
[3] W. Hogan, “Electricity Scarcity Pricing Through Operating Reserves,”
Economics of Energy & Environmental Policy, vol. 2, no. 2, 2013. DOI:
http://dx.doi.org/10.5547/2160-5890.2.2.4
[4] FERC, “Scarcity Pricing in ERCOT,” [Online]. Available:
https://www.ferc.gov/CalendarFiles/20160629114652-3%20-
%20FERC2016_Scarcity%20Pricing_ERCOT_Resmi%20Surendran.pdf
, Accessed on: Dec. 5, 2019.
[5] C. Liu, et al., “A Mathematical Model for Electricity Scarcity Pricing in
ERCOT Real-Time Energy-Only Market,” in 2017 IEEE Power &
Energy Society General Meeting, Chicago, IL, 2017, pp. 1-5.
[6] C. Liu, J. Guo, H. Hui, and Y. Fu, “Modeling of Operating Reserve
Demand Curves and System-Level Price Adders in Real-Time Energy-
Only Market,” IEEE Transactions on Power Systems, vol. 33, no. 5, pp.
4797-4808, Sept. 2018.
[7] ERCOT, “Methodology for Implementing Operating Reserve Demand
Curve (ORDC) to Calculate Real-Time Reserve Price Adder,” [Online].
Available: http://www.ercot.com/mktrules/obd/obdlist, Accessed on:
Dec. 5, 2019.
[8] Potomac Economics, “2018 State of the Market Report for the ERCOT
Electricity Markets,” [Online]. Available:
https://www.potomaceconomics.com/wp-content/uploads/2019/06/2018-
State-of-the-Market-Report.pdf, Accessed on: Dec. 5, 2019.
[9] The NorthBridge Group, “Economic Equilibrium and Reliability in
ERCOT,” [Online]. Available:
http://www.northbridgegroup.com/publications/Economic_Equilibrium_
and_Reliability_in_ERCOT.pdf, Accessed on: Dec. 5, 2019.
[10] S. A. Subramanyam, X. Zhang, J. Wu, and H. Song, “Dynamic Analysis
of Operating Reserve Demand Curve in Energy-Only Electricity
Markets,” in IEEE Innovative Smart Grid Technologies - Asia (ISGT
Asia), Chengdu, China, 2019, pp. 3015-3019.
[11] Z. A. Karian and E. J. Dudewicz, “Fitting a Generalized Lambda
Distribution Using a Percentile-KS (P-KS) Adequacy Criterion,” in
Handbook of Fitting Statistical Distributions with R (1st ed.), ‘…ƒ
ƒ–‘, : ƒ›Ž‘”Ƭ ”ƒ…‹•
”‘—’, 2011, pp. 279-305.
[12] B. F. Hobbs, M.-C. Hu, J. G. Inon, S. E. Stoft, and M. P. Bhavaraju, “A
Dynamic Analysis of a Demand Curve-Based Capacity Market Proposal:
The PJM Reliability Pricing Model,” IEEE Transactions on Power
Systems, vol. 22, no. 1, pp. 3-14, Feb. 2007.