IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 35, NO.

4, JULY 2020 3297

Letters
Effect of Loss of Load Probability Distribution on Operating Reserve Demand
Curve Performance in Energy-Only Electricity Market
Sreelatha Aihloor Subramanyam, Student Member, IEEE, and Xuewei Zhang , Senior Member, IEEE

Abstract—In energy-only electricity markets, the operational LOLP Loss of load probability
reserve demand curve is a scarcity pricing mechanism adopted to NCAY New capacity addition in year Y
address the shortage of reserves and incentivize the generators. It ORDC Operating reserve demand curve
is constructed based on the assumption of the normality of loss
of load probability, which is estimated from reserve error data. PBMCL Probability of (reserve) below minimum contingency
In this Letter, the historical record of reserve error is collected level
and analyzed to test the assumption and find the fittest probability PR Annual plant retirement rate
distribution. It is found that the normal distribution is generally Q-Q Quantile-Quantile
not as good as other distributions like log-logistic, general gamma, R Reserve level
and Weibull. Using a dynamic simulation framework, the installed
reserve margins with different distributions are evaluated over RAFP Risk-adjusted forecast profit
a fifty-year period, demonstrating that the choice of distribution rL Annual load growth rate
could significantly impact the subsequent pricing calculations and TB time-block
therefore the reserve margin in an energy-only electricity market. VOLL Value of lost load
This work is of potential importance and immediate relevance to X Minimum reserve level
system reliability and resource adequacy in power systems.
Δ k ORDC price adder of year k (in $/MWh)
Index Terms—Loss of load probability, operating reserve
demand curve, energy-only electricity market.
I. INTRODUCTION
NOMENCLATURE NSUFFICIENT generator revenue from wholesale electric-

CDF Cumulative distribution function

I ity markets has raised concerns on grid reliability and re-
silience [1]. The energy emergency alerts and load reductions
ER Reserve usage (in MWh) in a year in Texas in summer 2019 also accentuate the need to improve
FC Annualized fixed cost the market designs to maintain adequate reserves [2]. Some
GEV General extreme value ISO/RTOs like PJM adopt forward capacity market to secure
GM Gross margin reserves, making payments to generators for capacity commit-
GOF Goodness of fitting ments in future. In energy-only markets like ERCOT in Texas,
ICAPY Installed capacity of year Y generators receive payments for the actual energy provided,
IRM Installed reserve margin which makes it more difficult to incentivize investments on new
k Index of years in 8-year window (=Y−4, Y−3, …. generation to ensure resource adequacy. To resolve potential
Y+3) reliability issues that may arise, a scarcity pricing mechanism,
K-S Kolmogorov-Smirnov i.e., operating reserve demand curve (ORDC), has been proposed
LA,Y Actual peak load of year Y [3] and introduced to ERCOT since 2014 [4]. Mathematically,
LF,Y Forecast peak load of year Y the ORDC model developed in [5], [6] is a “capped” comple-
LMP Locational marginal pricing mentary cumulative distribution function (CDF) of loss of load
probability (LOLP).
Manuscript received December 7, 2019; revised February 16, 2020 and April This work is motivated by two fundamental questions about
16, 2020; accepted May 5, 2020. Date of publication May 20, 2020; date of ORDC. The first one is: what is the distribution of LOLP? It is
current version June 22, 2020. Paper no.PESL-00288-2019. (Corresponding
author: Xuewei Zhang.) known that LOLP is obtained from the historical data of reserve
Sreelatha Aihloor Subramanyam is with the Sustainable Energy Systems error, i.e., the difference between hour-ahead (forecast) reserves
Engineering Program at Texas A&M University-Kingsville, Kingsville, Texas and real-time (hourly-average) reserves [4]–[7]. The histogram
78363 USA (e-mail: sreelatha.aihloor_subramanyam@students.tamuk.edu).
Xuewei Zhang is with the Department of Electrical Engineering and Computer of reserve errors is fitted as a normal distribution with its mean
Science at Texas A&M University-Kingsville, Kingsville, Texas 78363 USA and standard deviation evaluated by statistical analysis and then
(e-mail: xuewei.zhang@tamuk.edu). its complementary CDF can be constructed. But is the normality
Color versions of one or more of the figures in this article are available online
at https://ieeexplore.ieee.org. assumption valid and accurate enough? The second question is
Digital Object Identifier 10.1109/TPWRS.2020.2995121 about the availability of effective approaches to evaluating the
0885-8950 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 07:14:30 UTC from IEEE Xplore. Restrictions apply.
3298 IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 35, NO. 4, JULY 2020

performance of ORDC. An ERCOT market evaluation report

[8] acknowledges the difficulties in pinpointing the practical
implications and effectiveness of ORDC, as enough generation
capacity is currently available, and no major impactful events
have taken place. Meanwhile, the report recognizes insufficient
revenues of generators even with ORDC implementation. To
increase ORDC price, [9] discusses the rightward shift of the
normal distribution of LOLP. Currently, ERCOT uses a shift by
0.25 of the standard deviation. From a theoretical point of view,
its justifiability remains an issue. Practically, it is of interest and
relevance to gain quantitative knowledge on how the shift will
affect the ORDC performance. Our recent work demonstrates a
dynamic simulation framework that serves for the purpose [10].
Therefore, the objectives of this work are to (i) test how well
the LOLP fits into a normal distribution, (ii) identify probability
distributions that have better agreement with data, and (iii) use
the aforementioned dynamic simulations to examine the effect
of the choice of LOLP distributions on ORDC performance. Fig. 1. (a) Q-Q plot of three-year hour 7–10 reserve error data. (b) Q-Q plot
Our main contributions include (a) the introduction of various of summer 2018 reserve error data. (c) Histogram and best fit (GEV) of the data
in (a). (d) Histogram and best fit (GEV) of the data in (b).
statistical tests, which generates new insights on the distribution
of LOLP, and (b) the evaluation of the long-term performance
of ORDC, which shows the promise of replacing shifted normal
distribution with other distributions. The models and methods
used in this study are described in Sec. II, where we also analyze
data to address objectives (i) and (ii). Sec. III presents some
ORDC simulation results to discuss the effect of distributions of
LOLP. Conclusions are drawn in Section IV.

II. MODELS AND METHODS

A. ORDC as LOLP-Dependent Price Adder
We first give the definition of ORDC [3]. It is a price adder
to locational marginal pricing (LMP) to recognize the value of
reserves available. A dispatch correction optimization problem
is solved in real time to minimize the sum of the correction cost
of increasing generation and the opportunity cost of shedding
loads [5], [6]. The dispatch correction should not exceed the
total demand disturbance or the reserve capacity. Applying
Karush-Kuhn-Tucker conditions to these two constraints results
in the complementary slackness of 2 Lagrangian multipliers.
The one associated with the reserve capacity constraint is used
for the construction of ORDC: 0 if the real-time reserve can meet
extra dispatch-correction needs and otherwise is the difference
between the value of lost load (VOLL) and the LMP. The proba-
bility of the latter, a decreasing function of the reserve capacity,
is the probability of (reserve) below minimum contingency level
(PBMCL). Thus, ORDC equals the product of PBMCL and the
Lagrangian multiplier. ISOs or RTOs set the minimum reserve
level (X) for system reliability. PBMCL = 1 if reserve level, R,
is below X. Otherwise, PBMCL = F(R−X), where F(·) is the
complementary CDF of LOLP.
B. Goodness of Fitting (GOF) Tests of LOLP Distribution
To obtain LOLP distribution, prior studies assume normality
and calculate its mean and standard deviation from historical
reserve error data (actually it is done for 24 4-hour time-blocks
Fig. 2. GOF ranks of normal and log-logistic distributions for the 24 TBs.
1–6: Spring; 7–12: Summer; 13–18: Fall; 19–24: Winter; all starting with Hours
3–6.
(TBs), 6/day for each season). Using reserve error distribution
to approximate LOLP distribution is a current industry practice
[3]–[7], the validity of which will be an interesting topic for
future studies. Within the scope this work, to verify the as-
sumption of normality, we request ERCOT data and follow the
instructions in [7] to compute hourly reserve error for 3 years
(2016-2018). Fig. 1(a) and 1(b) present the quantile-quantile
(Q-Q) plots of the reserve errors during hours 7–10 and during
summer 2018 against normal distribution. A clear departure
from normality (straight dot-dashed line) is observed. Using
software Easyfit, the best-fitted distribution for both sets of data
is found to be the general extreme value (GEV) distribution,
as shown in Fig. 1(c–d). Further, 3 GOF tests (Kolmogorov-
Smirnov, Anderson-Darling, and χ2 ) on if the data obey normal
distribution (null hypothesis) are conducted [11]. In the 24 data
sets, the tests of all but two reject the null hypothesis (the 2
exceptions are Fall Hours 7–10 and Spring Hours 23-24-1-2).
Ranked by the K-S test p-values of 30 distributions, there are 8
best fits that are log-logistic, 4 GEV, 3 general gamma, 3 Weibull,
2 normal, among others. The comparisons of log-logistic and
normal distribution ranks in Fig. 2 also show that, in most cases,

Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 07:14:30 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 35, NO. 4, JULY 2020 3299

normal distribution does not have the best agreement with the
data.

C. ORDC Dynamic Simulation Model

We have shown evidence that normal distribution may not be
the best fit for LOLP. To examine if this affects the performance
of ORDC, we use the model from our preliminary work [10]. The
model runs for 50-year economic plant life and repeats for 1000
iterations to estimate the dynamics of installed reserve margin
(IRM), i.e., the ratio of installed capacity to peak load. In year Y Fig. 3. IRM (mean of 1000 samples) over 50 years under different ER values.
(≥1), the simulation process is as follows. (i) From the previous
year’s peak load, generate the actual peak load LA,Y considering
an annual growth rate rL and some random error. The forecast
peak load (LF,Y+i ) for the ith year into the future is estimated
from LA,Y using annual growth rate rL . (ii) Given the previous
year’s installed capacity ICAPY−1 , the new capacity addition
before this year NCAY−1 , and the annual plant retiring rate
PR, then ICAPY = NCAY−1 + (1–PR)·ICAPY−1 . The forecast
ICAPY+i is assumed to increase at annual growth rate rL . (iii)
From the ORDC constructed following [5], [6], the price adder
ΔΠk is derived at the reserve levels (Rk = ICAPk − Lk ) for
each year during the 8-year period (k = Y−4, Y−3, …, Y+3):
ΔΠk = (V OLL − LM P ) · F(Rk − X) (in $/MWh). (iv) To
estimate the increased revenue of a benchmark generator from
ORDC, we multiply ΔΠk with the recorded reserve usage ER
(in MWh) in one year. In principle, the annual ORDC revenue
should be the accumulation of all hourly ORDC revenues. There
are 2 reasons that in this work we simply use a yearly-averaged
price adder. Firstly, only the data of monthly totals of reserve
usage can be found on public records, and we currently do not
have hourly reserve usage info. Secondly, since the real-time
reserve levels in the future years are unknown, the yearly average
of reserve levels is used to evaluate the ORDC price adder for
each year. (v) Given its annualized fixed cost (FC) and the gross Fig. 4. (a) IRM (mean of 1000 iterations) over 50 years with different LOLP
distributions. (b) Histograms of the standard deviations of IRMs with GEV,
margin (GM) from the E/AS market, the generator’s profit each Weibull, and normal distributions of LOLP.
year is φk = GMk + ΔΠk · ER − F Ck . (vi) Use the φk ’s to
calculate risk-adjusted forecast profit (RAFP) following [12]. If
RAFPY ≤ 0, then NCAY = rL · ICAPY . As the peak load grows
show that the mean of IRMs will be consistent as long as the
annually, NCAY is required to increase the installed capacity by
number of iterations is over several tens; more iterations will
rL . If RAFPY > 0, NCAY will increase linearly with RAFPY
lower the standard deviation of the IRMs. Since ER is unknown
until reaching a pre-defined upper limit. More model details
for the benchmark generator, we use several values and keep
and parameter settings are specified in [10], a preliminary work
them constant in each simulation. To ensure the averaged IRM
of ours devoted to building and validating the model which is
no lower than 1.04, ER needs to be at least 5 MWh. More
self-consistent and can be used for accomplishing our objectives.
usage (revenue) results in higher incentives for NCA and higher
It is also worth noting that the model, although similar to Monte
IRM. In the case of ER = 2 MWh, the IRM will decrease over
Carlo method, does not involve any uncertainty in initial inputs
time since generator firms do not project profits by providing
like IRM. With the future demand information being generated
reserve services. On the other hand, if ER is much higher
stochastically, we average the results from multiple iterations to
than 5 MWh, the IRM will increase initially and reach a new
see the expected trajectory instead of individual sample path.
“equilibrium” point. It is because when IRM is too high, ORDC
price and generator revenue will drop, which reduces NCA.
III. RESULTS AND DISCUSSIONS We have demonstrated the effectiveness of ORDC in regulating
Simulations are run to show the long-term dynamics of IRM. reserve levels. The model provides a self-consistent framework
When LOLP is a normal distribution, the effect of ER on IRM to evaluate the performance of ORDC in energy-only markets.
is presented in Fig. 3. All the simulations in Fig. 3 start with Now with ER = 5 MWh, we run simulations to study how
the initial IRM of 1.04. Each curve represents the dynamics of the LOLP distributions affect the IRM dynamics. The results in
the mean of IRMs from multiple simulations. Our simulations Fig. 4(a) show that all other distributions will establish a raised

Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 07:14:30 UTC from IEEE Xplore. Restrictions apply.
3300
IRM level from that of the normal distribution, with log-logistic
being the closest and Weibull being the highest. In this work,
we use the same LOLP distribution throughout the 50 years. No
TBs or updates of the distribution are included. Nevertheless,
the objective is to shed light on how different interpretations
of reserve errors may impact the IRM in the long run. ERCOT
is implementing normal distribution with a rightward shift to
increase the ORDC price. The results of the shifted normal
distributions are also plotted in Fig. 4(a). In comparison, using an
alternative LOLP distribution can accomplish the same (see the
cases of shifted by 1.5 standard deviations and GEV or general
gamma), while being more statistically sound.
Fig. 4(b) is the histograms of IRM standard deviations (S.D.)
of 1000 iterations with three distributions of LOLP in Fig. 4(a).
Lower S.D. means less fluctuating and thus more predictable
IRM. While normal distribution has the lowest IRM, it seems to
correspond to more stable IRM dynamics than others.
IV. CONCLUSIONS
In conclusion, this work studies the LOLP distribution fitted
from historical data of reserve errors in ERCOT and examines
the effect of the LOLP distributions on the ORDC’s long-term
performance. The statistical tests show that normal distribution
is not the best fit in most cases. The simulation results of a self-
consistent model indicate that different probability distributions
used in ORDC can lead to different IRMs, implying that it may
not be necessary to shift the normal distribution to increase the
ORDC payments to ensure targeted IRMs. With the availability
of more data, future work can be done to improve and calibrate
the model for simulations at higher time resolutions.
REFERENCES
[1] U.S. Department of Energy, “Staff report to the secretary on electricity
markets and reliability,” [Online]. Available: https://www.energy.gov/
downloads/download-staff-report-secretary-electricity-markets-and-
reliability, Accessed on: Dec. 5, 2019.
Authorized licensed use limited to: University of Exeter. Downloaded on July 16,2020 at 07:14:30 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON POWER SYSTEMS, VOL. 35, NO. 4, JULY 2020
[2] M. J. Hutzler, “Texas’ Impending Reliability Issues With Wind Power,”
[Online]. Available: https://www.powermag.com/texas-impending-
reliability-issues-with-wind-power, Accessed on: Dec. 5, 2019.
[3] W. Hogan, “Electricity scarcity pricing through operating reserves,” Econ.
Energy Environmental Policy, vol. 2, no. 2, 2013. DOI: http://dx.doi.org/
10.5547/2160-5890.2.2.4
[4] FERC, “Scarcity Pricing in ERCOT,” [Online]. Available: https://
www.ferc.gov/CalendarFiles/20160629114652-3%20-%20FERC2016_
Scarcity%20Pricing_ERCOT_Resmi%20Surendran.pdf. Accessed on:
Dec. 5, 2019.
[5] C. Liu, et al., “A mathematical model for electricity scarcity pricing in
ERCOT real-time energy-only market,” in Proc. IEEE Power Energy Soc.
General Meeting, Chicago, IL, 2017, pp. 1–5.
[6] C. Liu, J. Guo, H. Hui, and Y. Fu, “Modeling of operating reserve demand
curves and system-level price adders in real-time energy-only market,”
IEEE Trans. Power Syst., vol. 33, no. 5, pp. 4797–4808, Sept. 2018.
[7] ERCOT, “Methodology for implementing operating reserve demand curve
(ORDC) to calculate real-time reserve price adder,” [Online]. Available:
http://www.ercot.com/mktrules/obd/obdlist. Accessed on: Dec. 5, 2019.
[8] Potomac Economics, “2018 State of the market report for the
ERCOT electricity markets,” [Online]. Available: https://www.
potomaceconomics.com/wp-content/uploads/2019/06/2018-State-of-
the-Market-Report.pdf, Accessed on: Dec. 5, 2019.
[9] The NorthBridge Group, “Economic equilibrium and reliability in
ERCOT,” [Online]. Available: http://www.northbridgegroup.com/
publications/Economic_Equilibrium_and_Reliability_in_ERCOT.pdf.
Accessed on: Dec. 5, 2019.
[10] S. A. Subramanyam, X. Zhang, J. Wu, and H. Song, “Dynamic analysis
of operating reserve demand curve in energy-only electricity markets,” in
Proc. IEEE Innovative Smart Grid Technologies - Asia, Chengdu, China,
2019, pp. 3015–3019.
[11] Z. A. Karian and E. J. Dudewicz, “Fitting a generalized lambda distribution
using a percentile-KS (P-KS) adequacy criterion,” in Handbook of Fitting
Statistical Distributions with R (1st ed.), Boca Raton, FL: Taylor & Francis
Group, 2011, pp. 279–305.
[12] B. F. Hobbs, M.-C. Hu, J. G. Inon, S. E. Stoft, and M. P. Bhavaraju, “A
dynamic analysis of a demand curve-based capacity market proposal: The
PJM reliability pricing model,” IEEE Trans. Power Syst., vol. 22, no. 1,
pp. 3–14, Feb. 2007.