Professional Documents
Culture Documents
BWL CRT Report
BWL CRT Report
May 5, 2014
Table of Contents
Executive Summary.....Page 3
I. Introduction....Page 9
II. Planning....Page 20
III. Response and Restoration..Page 42
IV. Recovery and Mitigation........Page 53
Appendix A-CRT Members and Staff......Page 76
Appendix B-CRT Mission Statement...Page 78
Appendix C-Community Concerns....Page 79
Appendix D-Materials Acquired from BWL.Page 96
Appendix E-Transcript of March 10, 2014 Public Hearing..Page 99
Appendix F-Reference Materials...Page 141
Executive Summary
CRT Report on BWL Performance
May 5, 2014
I.
II.
The BWL is a municipal utility, owned by the people of the City of Lansing,
that provides electric utility services to residential, commercial and industrial
customers in Lansing, East Lansing, and the Townships of Delta and Windsor
in Eaton County and of Dewitt and Watertown, in Clinton County, and Delhi,
Meridian, Lansing, and Alaiedon1 in Ingham County. Given adequate fuel
sources, the BWL can provide sufficient electric power for its customer base,
without the need to purchase power. In the Lansing area, BWL is responsible
for the provision, and for the protection, of water and energy, two of the three
most important sectors of critical infrastructure.
III.
Alaiedon Township and the City entered into a P.A. 425 Agreement in 1998, covering the land on which Jackson
National Life Insurance is situated.
efforts were disjointed. The EAP and other plans need to be replaced by a
more comprehensive plan that includes tools for better situational
awareness, and specific plans for response and recovery.
d) BWL does not have Emergency Manager. BWL did not have a distinct
Emergency Operations Center; during electric power outages of any
duration it continues to run operations from their BESOC.
e) There was a failure of communication between the City and the BWL and a
total lack of communication between the BWL and the other local
governments, representing the BWL customer service area. Assumptions
were made on the transfer of information, rather than the reliance on
established, tested lines of communication. The CRT discussions with city
and county emergency managers suggest that regional planning activity is
not organized and coordinated in advance of emergency events, but
instead, relies on the experience of the participants and their pre-existing
relationships.
f) Emergency Operations Plans must be coordinated before an emergency
event or incident, to provide time for training all personnel on the plan
and for exercising the plan.
g) BWL lacked a communications annex for an Emergency Operations Plan,
as well. BWL did not have available the information required due to the
failure of the OMS and the lack of capacity for customer calls. These
deficiencies substantially reduced their level of situational awareness and
lead to the transmission of information that did not meet the needs of
customers and to their inability to determine how they might best respond
to the circumstances.
h) BWL was neither communicating nor coordinating with the Lansing EOC
after December 24, 2013, and many of the other Emergency Operations
Centers were not open, which contributed to the confusion, lack of
accurate information, and uncertainty experienced by municipal leaders
and citizens.
i) The CRT believes that an integrated regional EOP is vitally needed. If a
regional EOP, which includes a regional EOC, was operating during the
recent event, it would have facilitated the coordination of services between
first responders and with other governmental leaders.
IV.
V.
the Board itself for the local governments which comprise the customer
service area.
n) The Board should hire an "operational auditor" to conduct annual
performance audits of the BWL operations and planning efforts.
o) The BWL should establish a Customer Ombudsman, Chief Customer
Officer or expand the duties of the Director of Governmental Affairs and
Customer Relations, reportable directly to the General Manager.
p) The positions of Strategic Planning, Information Technology, and
Operations should be directly reporting to the General Manager.
q) Consider revision of the 54 Recommendations, using the SMART analysis
to include specific metrics, including the tasks to be performed, the
directorate or section within BWL charged with performance and the time
need for accomplishment. Require staff reports monthly to the Board of
Commissioners on specific actions that have been taken on all of the 54
recommendations made in the Ice Storm Outage Report and that the BWL
post those reports on its website for the next 18-24 months.
Introduction
On December 21, 2013 a catastrophic ice storm struck the Lansing, Michigan region,
causing more than 34,000 electric customers of the Lansing Board of Water and Light
(BWL) to experience a long-term power outage. Many BWL customers lacked electrical
power for an extended period of time, some reportedly for over 10 days. This problem
was exacerbated by a number of factors including the holiday season, the short days of
daylight, and the freezing temperatures. As the outage continued the BWL had to revise
its estimates on the scope of the outage, on the date when BWL crews would address
neighborhood or individual outages, and on the date for power restoration to the entire
customer base. Public frustration2 over the inability to either seek or provide
information with the BWL, as it lacked a functioning Outage Management System and
sufficient number of damage assessors and customer service representatives, turned to
deep concern over the safety of family members and neighbors as the outage period
lengthened without a known date for restoration, and to anger over the unavailability of
senior leadership. As the publics concerns continued after the restoration of power, the
Mayor of Lansing determined that an independent investigation of the BWL response
and restoration was needed.
December 2013 Ice Storm Timeline
December 21st: The National Weather Service warns of ice storm bringing
approximately .5 inch of ice. The storm hits the Lansing, Michigan area during the
afternoon. 3
Public frustration and anger was directed only at the BWL leadership. The public repeatedly praised the efforts by
the BWL line crews. The CRT also expresses their gratitude and admiration for the dedicated work of the BWL line
crews, outside line crews, and other BWL employees that worked diligently to restore power during extreme
conditions over the holidays.
3
New Scale Pinpoints Ice Storm Danger, Lansing State Journal, (Dec. 21, 2013) Available at
http://www.lansingstatejournal.com/article/20131221/NEWS01/312210017/New-scale-pinpoints-ice-stormdanger; Steven R. Reed and Elizabeth Wolfe, Timeline: Piecing Together Key Details from Ice Storm, BWLs
Response, Lansing State Journal (Mar. 1, 2014) Available at
http://www.lansingstatejournal.com/interactive/article/A3/20140301/NEWS01/140228003?nclick_check=1.
9
December 22nd: At 6 a.m., BWL reports 19,000 customers are without power. Later
that day, about 200 lines are reported down and 25,000 customers are without power.
By 7 p.m., the totals are at 300 downed lines and 34,800 outages. Delta Township
declares a state of emergency. 4
December 23rd: BWL reports more than 400 downed lines. At 9 p.m., outages total
19,000. BWL spokesman Steve Serkaian tells the Lansing State Journal that there are
sufficient resources in the field to restore power. All hands are on deck and we asked
people to give up their holiday in order to restore power, he says. General Manager
Peter Lark travels to New York for a vacation. Meridian Township declares a state of
emergency. 5 BWL releases a statement at approximately 4 p.m., instructing customers
not to contact the utility to report outages unless you believe your situation is unique
and that BWL is aware of its outage areas. 6
December 24th: At 4 pm, 12,000 customers remain without power.7 By late that
evening, BWL reports that 7,500 customer outages remain.8
December 25th: BWL reports that 5,700 customers remain without power. BWL
General Manager Peter Lark returns to Lansing.9
December 26th: The BWL reports at 8 a.m. that they are down to 4,400 households
without power. The City of East Lansing reports, They continue to have many crew
members out working on remaining major circuits and will then be focusing on smaller
pockets where power outages still exist. 10 BWL General Manager Peter Lark tells
4Meegan Holland, Ice Storm Knocks Out Power, Downs Trees in Lansing Area, MLive, (Dec. 22, 2013) available at
http://www.mlive.com/lansing-news/index.ssf/2013/12/ice_storm_knocks_out_power_dow.html; Steven R. Reed
and Elizabeth Wolfe, Timeline: Piecing Together Key Details from Ice Storm, Dec. 22, BWLs Response, Lansing State
Journal, (Mar. 1, 2014) Available at
http://www.lansingstatejournal.com/interactive/article/A3/20140301/NEWS01/140228003?nclick_check=1.
5
Steven R. Reed and Elizabeth Wolfe, Timeline: Piecing Together Key Details from Ice Storm, Dec. 23, BWLs
Response, Lansing State Journal, (Mar. 1, 2014) Available at
http://www.lansingstatejournal.com/interactive/article/A3/20140301/NEWS01/140228003?nclick_check=1.
6
BWL: To Report Downed Lines & Outages, News Ten, WILX, (Dec. 23, 2013) Available at
http://www.wilx.com/home/headlines/BWL-To-Report-Downed-Lines--Outages-237066261.html.
7
Brandon Howell, Lansing Board of Water & Light Outages Down to 12,000, MLive, (Dec. 24, 2013) Available at
http://www.mlive.com/lansing-news/index.ssf/2013/12/lansing_board_of_water_light_o.html.
8
Id. (Update: By late Christmas Eve, the BWL had restored power to all but 7,500 of its customers.)
9
Steven R. Reed and Elizabeth Wolfe, Timeline: Piecing Together Key Details from Ice Storm, Dec. 25, BWLs
Response, Lansing State Journal, (Mar. 1, 2014) Available at
http://www.lansingstatejournal.com/interactive/article/A3/20140301/NEWS01/140228003?nclick_check=1.
10City of East Lansing, Ice Storm Update from the City of East Lansing 12/26/13 (Dec. 26, 2013) Available at
http://www.cityofeastlansing.com/Home/Departments/Communications/MediaRoom/articleType/Artic
leView/articleId/1709/Ice-Storm-Update-from-the-City-of-East-Lansing-122613/.
10
reporters he cannot predict when power will be restored fully. An East Lansing resident
creates an outage map and posts it to BWL's Facebook page. At 10 p.m., BWL says 3,000
customers are without power. 11
December 27th: 3,000 BWL customers were still without power.12 Eaton and Clinton
Counties declare emergencies. Ingham County officials report that they do not intend to
declare a state of emergency. 13
December 28th: At 8 p.m., BWL says outages have grown to 4,50014, the increase
reportedly caused by thawing trees and additional outages being reported.15 BWL
Customers rally at an East Lansing school to protest BWLs performance. Customers
demand restoration of power at a Lansing news conference with Mayor Virg Bernero. 16
December 29th: At 5 p.m., BWL lists 3,193 customers lack power on 333 streets and
asks those without power to contact BWL. According to the Lansing State Journal, later
that evening, BWL Spokesman Steve Serkaian texts Lansing Mayor Virg Berneros chief
of staff, saying 100 new streets have been reported by customers without power.17
December 30th: At 9 p.m., BWL reports 700 outages, plus 400 "single-servicereconnection requests" involving customers who need to repair their connections. BWL
11
Steven R. Reed and Elizabeth Wolfe, Timeline: Piecing Together Key Details from Ice Storm, Dec. 25, BWLs
Response, Lansing State Journal, (Mar. 1, 2014) Available at
http://www.lansingstatejournal.com/interactive/article/A3/20140301/NEWS01/140228003?nclick_check=1.
12
Nick Perreault, BWL Responds to Ice Storm Frustrations, WLNS (Dec. 27, 2013) Available at
http://www.wlns.com/story/24317553/bwl-responds-to-ice-storm-frustrations.
13
Clinton, Eaton, Delta Twp., Meridian Twp. Declare Emergencies, Lansing State Journal, (Dec. 27, 2013) available at
http://www.lansingstatejournal.com/viewart/20131227/NEWS01/312270029/Clinton-Eaton-Delta-Twp-MeridianTwp-declare-emergencies
14
Steven R. Reed and Elizabeth Wolfe, Timeline: Piecing Together Key Details from Ice Storm, Dec. 28, BWLs
Response, Lansing State Journal (Mar. 1, 2014) Available at
http://www.lansingstatejournal.com/interactive/article/A3/20140301/NEWS01/140228003?nclick_check=1.
15
Lindsay VanHulle, Anger, frustration boils over as power outages continue in Lansing Area, Lansing State Journal
(Dec. 29, 2013) Available at http://www.lansingstatejournal.com/viewart/20131228/NEWS01/312280016/Angerfrustration-boils-over-power-outages-continue-Lansing-area.
16
Id.
17
Steven R. Reed and Elizabeth Wolfe, Timeline: Piecing Together Key Details from Ice Storm, Dec. 29, BWLs
Response, Lansing State Journal, (Mar. 1, 2014) Available at
http://www.lansingstatejournal.com/interactive/article/A3/20140301/NEWS01/140228003?nclick_check=1.
11
12
sharing its findings and recommendations. The CRT would make this report available to
the Mayor, City Council, the citys regional partners within the BWL service territory
and the public at large. The report would also be shared with Michigan Public Service
Commission (MPSC) for review.
The CRT conducted public hearings on the evenings of February 6th, 7th and 8th of 2014
to address customer concerns and take suggestions from citizens. The hearings were
located in three BWL service areas: East Lansing, Lansing, and Delta Township. Each
hearing allowed citizens to express concerns to the CRT orally or to submit written
comments. The CRT continued to accept written comments by e-mail throughout the
process of the review. The CRT also created a Facebook page and an email address to
communicate with the public. CRT questionnaires were distributed to the public
through groups such as neighborhood associations. The CRT also reviewed the
transcripts of the public hearings conducted by the BWL. See Appendix C for a summary
of citizens concerns, received either by email or at the CRT public hearings.
Working with a representative from the BWL, the CRT submitted two requests for
documents to the BWL. The BWL responded with thousands of pages of documents,
which the CRT reviewed extensively with the assistance of subject matter experts. The
CRT also reviewed and analyzed the BWLs internal report on the December 2013
storm, released on February 19, 2014. Materials received from BWL are listed at
Appendix D. On March 10, 2014, the CRT questioned BWL top executives21 during a
five-hour hearing. See Appendix E. The CRT also interviewed city employees and
subject matter experts. After the information-gathering was completed, the CRT began
compiling its findings into recommendations by focusing on three areas of BWL
performance surrounding the December 2013 ice storm: planning and preparation,
response and restoration, and recovery and mitigation. The form of this report tracks
those three performance areas.
Board of Water and Light
The Board of Water and Light is a unique entity within the Charter of the City of Lansing
(City). The BWL, unlike all other City advisory boards, commissions or committees,
shall exercise administrative, executive, and policy-making authority over the
21
The CRT interviewed the General Manager,Peter Lark; Assistant General Manager, Susan Devon; Director of
Communications, Steve Serkaian; Directorof Information Technology, Nick Burwell; Director of Electric
Transmission and Distribution,Dave Bolan; and the Manager of Electric Systems Operations, Lynn McKinstry. The
CRT often refers to BWL senior leadership in this report. That term is intended to encompass only the General
Manager, Assistant General Manager and those directors with strategic level decision-making authority,
13
operation of City owned utilities assigned to it.22 Accordingly, the City has granted full
and exclusive management of the water, heat, steam and electric services to the BWL
with the stated duty to provide those services in a manner consistent with the best
practices.23
Although created, funded and owned by the City, the services provided by the BWL
extend, mostly by franchise agreement24, to the provision of electrical power to the City
of East Lansing, the Townships of Delta and Windsor in Eaton County and of Dewitt and
Watertown, in Clinton County, and Delhi, Meridian, Lansing, and Alaiedon25 in Ingham
County.
The BWL is the 32nd largest publicly owned utility in the United States serving electrical
customers, according to the American Public Power Association,26 with just over 96,000
ultimate customers served. BWL is well regarded nationally for meeting reliability
standards set by the American Public Power Association, receiving a grade of 100% on
the Reliable Public Power Provider standards.27 More importantly, BWL has
demonstrated strategic vision in two other vital areas: the diversification of sources of
power and the generation of sufficient electrical power to assure its ability to meet
customer demand.
In July 2013, BWL began operating its co-generation plant; this natural gas-fired
electric generating plant emits 50% less greenhouse emissions than a coal fired plant,
and will cut sulfur dioxide and mercury emissions by 99% and nitrogen oxide emissions
22
24
There is little similarity between the various franchise agreements, save that all were executed between 19861988, for a period of 30 years, and none appear to be exclusive agreements. Only four require the payment of a
franchise fee by BWL to the township.
25
Alaiedon Township and the City entered into a P.A. 425 Agreement in 1998, covering the land on which Jackson
National Life Insurance is situated.
26
American Public Power Association, 2013-2014 Annual Directory & Statistical report, www.PublicPower.org. It
should be noted that the size of the customer base for public utilities in the Top 100 ranges from over 1.4 million
customers for Puerto Rico and the City of Los Angeles to just over 30,000 customers for the twenty municipalities
at the end of the list.
27
BWL Press Release, 25 March 2014, Lansing BWL to Receive Top National Recognition for Reliable Electric
Operations http://www.lbwl.com/About-the-BWL/News/Lansing-BWL-to-Receive-Top-National-Recognition-forReliable-Electric-Operations/
14
by 85%. The design and development of this energy-efficient generation plant and
headquarters resulted in BWL receiving gold certification from the Leadership in Energy
and Environmental Design, the first utility in Michigan to receive this award. BWL also
has contracted with Granger Waste Disposal since 1992 for waste to energy electrical
power28.
BWL has the capacity to generate locally all the electrical power needed by its
customers. BWL has the capacity to generate 440 MW of energy; it provides
approximately 2,200,000 MWh of electricity each year, which equates to approximately
250 MW of energy on average and a peak demand of 500 MW. BWL purchases 11 MW
of power from Granger (waste to energy) and purchases approximately 1.7 MW of
hydro-electricity under a purchase power contract. BWL also has capacity and energy
entitlement to 150 MW from the Belle River generating facility in St. Clair County and
has a small solar array. BWL can truly supply hometown power to its customers. This
is of enormous potential strategic importance. BWL is almost wholly self-reliant for
power; it can generate all the electrical power its customers require. If BWL can
maintain this strategic advantage, rather than have to purchase power off the market, as
do most publicly-owned utilities, it will not only assure the regions energy selfsufficiency, but will also assure the energy security of the region.
Storm Frequency and Intensity
The ice storm in December 2013 is reported to have caused a higher percentage of
customer outages than any previous storm affecting the BWL. (Lansing Board of Water
and Light also claims it is higher than for any utility in Michigan, but comparisons to
utilities with larger territories are not appropriate for some purposes because a storm
event is less likely to affect a large fraction of customers in a larger territory.) The ice
storm, however, was not the sole cause of the outage and certainly not of the length of
the outage. The confluence of three policies, all under the BWL control, contributed to
the length of the outage: the failure to follow the BWL vegetation management policy,
the lack of trained damage assessors, or spotters to immediately be deployed, and the
non-availability of the Outage Management System were significant causes of the length
of the outage.
Lansing Board of Water and Light, and others, may take the view that this was an
exceptional event and evaluate grid hardening measures and restoration capacity on
that basis. While it is likely that this storm had greater consequences than are likely to
28
15
recur with great frequency, we must not be guided solely by our past experience as to
storm frequency or to building a resilient system.
A recent report29 shows that the level of power outages in the United States due to
weather has been increasing significantly in recent years as illustrated in the following
graph.
The report attributes this increase to both aging grid infrastructure and increasing
storm intensity associated with global climate change. It also notes that:
The Lansing Board of Water and Light must plan as though recent storm frequency and
intensity is normal or even that these types of events will continue to increase.
As our nations approach to disaster and emergency planning has matured, there is a
growing belief in the community that catastrophes in the US have intensified in their
effects, as our society has evolved into a complex system of systems which are highly
connected, optimized and cost-efficient.30 Because our systems, including the electrical
29
(Kenward, Alyson and Raja, Urooj. 2014. Blackout: Extreme Weather, Climate Change and Power Outages.
Available at http://assets.climatecentral.org/pdfs/PowerOutages.pdf
30
Lewis, Ted G. Baks Sand Pile, Strategies for a Catastrophic World. Agile Press, 2011.
16
transmission and distribution systems are built to be low-cost and efficient, rather than
resilient and secure, they are more brittle, and prone to failure.
The Cumulative Nature of Storm Impact
Some public expectations and some BWL analyses tend to look at grid hardening
measures or storm response practices as though they only affect the customers directly
affected by the avoided fault or accelerated repair. This is incorrect and the BWL should
approach its analyses with care on this point. Whether repairs are done by a single or
multiple crews, repairs after a storm are still done more-or-less serially repairing one
fault, then on to the next, and the next, etc.. This means that avoiding a fault through
grid hardening or accelerating a repair has the effect of shortening the time to power
restoration for all subsequent repairs.
BWL policy, appropriately, dictates that the sequence of repairs in a major outage is to
first restore power to critical facilities, then secure down lines, then repair faults in the
primary distribution system roughly in order of the number of customers affected, then
repair faults in the secondary distribution system roughly in order of the number of
customers affected. Consequently, grid hardening or accelerated restoration practices
that address critical facilities and secure down lines as a priority benefit all other
customers; grid hardening or accelerated restoration practices that reduce repair time
for the primary distribution system benefit most customers; and grid hardening or
accelerated restoration practices that reduce repair time for the second distribution
system effectively benefit most customers affected by secondary distribution system
faults.
Best Practices
The BWL is charged, by City Charter as follows: The board shall be responsible to the
Mayor and the City Council for the provision of these services in a manner consistent
with the best practices31 . As noted in the Mission Statement, the CRT was therefore
tasked with evaluating the BWL response to the ice storm against Best Practices in the
electric utility industry. The CRT has researched and reviewed a number of After Action
Reports published by other utilities, state governors or public utility regulators, which
focus on electrical outages caused by ice or snow storms.
For example, New Hampshire suffered a severe ice storm, resulting in a lengthy outage
of electrical power in December 2008. As a result, the state conducted a similar,
although broader investigation, into the response by all utilities in the state. The report
31
17
of that investigation identified the following best practices, which the CRT has used as
a partial template32:
Planning and Preparedness:
1)
The utility bases their emergency operations on the concept of the national
incident management system.
2)
3)
4)
5)
6)
7)
The utility has pre-staged materials which may include such things as
storm trucks or storm boxes.
32
NEI Electric Power Engineering, Best Practices for Electric Utilities, Page VII-12. The other sources of Best
Practices which were relied upon by the CRT are included in Appendix F.
18
9)
The utility has a restoration strategy that targets the restoration of power to
the greatest number of customers in the shortest amount of time.
10) The utility does not limit requests for supplemental crews to the local mutual
aid groups and other local utilities.
11)
The utility strives to make sure that all communications are correct and
consistent.
19
33
Those principles are Protection, Resiliency, Interdependencies, Preparing for response and mitigation, Focusing
on prevention and detection, Evaluating infrastructure for: Criticality(or Risk), Vulnerability and Threat, and then
developing a formal risk mitigation strategy.
34
See Flynn, S. America the Resilient: Defying Terrorism and Mitigating Natural Disasters. Foreign Affairs, 87 (2),
2-8 (2008). Many writers include Redundancy in the list of principles, but Flynn includes Redundancy with in
Robustness.
Robustness
Designing structures strong enough to take a foreseeable hit
Investing in and maintaining elements of critical infrastructure, so that they can withstand lowprobability, high-consequence events.
Resourcefulness
Identifying options
Prioritizing what should be done to control damage and to begin mitigating it
Communicating decisions to the people who will implement them
Providing adequate resources to emergency responders, the National Guard, American Red Cross,
public health officials
Rapid Recovery
Carefully drafted contingency plans
Competent emergency operations
Means to get the right people and resources to the right places
Training auxiliary first responders prior to event
Having business emergency networks in place
Learning lessons
Rebuilding homes on floodplains or failing to resolve interoperable communications issues is foolish
People must be willing to make practical changes
20
35
21
Findings:
1)
In the Lansing area, BWL is responsible for the provision, and for the
protection, of water and energy, two of the three most important sectors of
critical infrastructure36.
2)
3)
4)
The Order of Restoration used by the BWL was outdated and did not include
all critical facilities.
2)
3)
4)
36
The focus of the CRT study was the BWL electric system, not the water and steam systems, but the Capital
Regions reliance on BWL to provide these essential resources, upon which all other systems of critical
infrastructure depend, requires emphasis.
22
37
IS-100.PWB: Introduction to the Incident Command System (ICS 100) for Public Works;
http://training.fema.gov/EMIWeb/IS/courseOverview.aspx?code=IS-100.pwb
38
http://www.fema.gov/media-library/assets/documents/32230?id=7371
39
http://www.fema.gov/media-library-data/20130726-1921-25045-4796/final_esf_12_energy_20130501_r1.pdf
40
23
2)
3)
The BWL had several Emergency Plans, but did not have a coordinated
Emergency Plan. In response to CRT questions, the BWL indicated that
training did not occur with all personnel involved, in the same training
program, at the same time.
From the same article on the APPA website, Credentialing courseworkThe proposed NIMS guidelines call for
equipment operators to complete prescribed training regimens for the Incident Command System. The course
describes the history, features, principles and organization of the Incident Command System and explains the
relationship between ICS and the NIMS. Other recommended training addresses single resources and initial action
incidents. It is designed to tell supervisory personnel how to operate efficiently during a homeland security
incident. A third training course (NIMS-700: National Incident Management System Introduction) explains the
purpose, principles, key components and benefits of NIMS. It also explains when it is appropriate to institute an
area command, when it is appropriate to institute a multi-agency coordination system, the benefits of using a joint
information system for public information, how NIMS resources are managed, and advantages of common
communication and information management systems.
24
a)
During any event where the City EOC is activated, including during major
widespread outages, a trained and experienced BWL Liaison Officer must be
deployed to the City EOC.
b)
c)
d)
e)
The hearing on 10 March 2014 with BWL officials left the impression that its leadership
had not yet recognized the need for the integration of planning for the likelihood of a
catastrophic or extreme weather event into its overall strategic planning. Despite the
daily reports of extreme weather, and the warnings to utility operators of the potential
impacts of climate change, the BWL leadership seemed to discount the possibility that
BWL could confront an event ever resulting in greater than the 40% customer outage
experienced last December. The BWL electric operations chief commented during the 5hour interview with senior leadership that because BWL had never experienced an
outage of greater than 20%, they assumed that that was the upper limit for which they
should prepare. Similarly, the General Manager declared at the 10 March hearing no
utility has ever been hit with a storm of these dimensions. To the contrary, however,
utilities, like emergency planners for government agencies and other sectors of critical
infrastructure, are routinely criticized for the lack of imaginative planning43, for the
43
Witt Report, pp 13-14: [Connecticut Light and Power] underestimated in its planning the potential scale of a
worst-case power outage event. This underestimate had ripple effects through CL&Ps planning for personnel,
equipment and coordination needs. CL&P had planned for a worst-case scenario in which 100,000 customers lost
power, only one-eighth of those actually affected by the nor'easter. In fact, 70 percent, 810, 000 of 1,2 million
customers suffered a power outage. The report had conceded that such a storm had not occurred in 25 years, at
the time the plan was implemented, but still faulted CL&P for merely telling its emergency crews to be on call
25
inability to envision potential natural or man-made actions that can degrade or destroy
their continuing operations.
The North American Electric Reliability Corporation (NERC) and the U.S. Department
of Energy (DOE) partnered in July of 2009 on an effort to address High-Impact, LowFrequency risks to the North American bulk power system44. As stated in the report,
Examples of these events include a pandemic illness, coordinated cyber, physical, or
blended attack on the system, extreme solar weather, and the high-altitude detonation
of a nuclear weapon. While some of these events have never occurred and the
probability of future occurrence and impact is difficult to measure, government and
industry are working to evaluate and, where necessary, enhance current planning and
operating practices to address these risks in a systematic and comprehensive fashion.
In addition the risk assessments conducted by States in the identification of critical
infrastructure were asked by the DHS to use a worst case scenario to conduct this
assessment.
The focus of emergency management planning, then, is to imagine the unlikely and
then prepare for it45. A common catch-phrase of emergency planners is to Prepare for
the worst, and everything else is a lesser included contingency.
The BWL, in response to requests from the CRT, provided a number of different
emergency plans46. At the March 10, 2014, hearing, BWL indicated that its 2014
Emergency Action Plan (EAP) was the plan implemented during the ice storm. This plan
instead of having them wait at predetermined locations. As a result it was harder to mobilize them when the
effects turned out to be far worse than they anticipated despite warnings from the NWS using terms such as
"historic" and "catastrophic" prior to the storm.
44
http://www.nerc.com/pa/CI/Resources/Documents/HILF_Report.pdf
45
The 9/11 Commission declared that the Nations lack of preparation for domestic terrorist attacks were due to a
failure of imagination, and the literature on emergency planning has since focused on the need to imagine the
worst case scenario. See for example, Overcoming failure of imagination in crisis management: The complex
adaptive system, MJ Bolton, GB Stolcis , The Innovation Journal: The Public Sector Innovation Journal, Volume
13(3), 2008, article 4 (2008); McConnell, A. and Drennan, L. (2006). The Mission Impossible? Planning and
Preparing for Crisis. Journal of Contingencies and Crisis Management, 14: 5970. doi: 10.1111/j.14685973.2006.00482.x; Martin, L., & Simon, S. (2008). A formula for disaster: The Department of Homeland Security's
virtual ontology. Space and Polity, 12(3), 281-296.
46
The 2014 Lansing Board of Water & Light Emergency Action Plan dated October 28, 2013 (this is the plan in use
during the December 2013 outage, according to testimony); The Lansing Board of Water & Light Emergency
Procedures Manual; The Lansing Board of Water and Light Transmission Operations Plan and Emergency
Operations Plan (Transmission Operation Plan (TOP))., and the Lansing Board of Water and Light Emergency
Operations Plan (EOP).
26
was an update to the July 2012 EAP. The Electrical Transmission and Distribution also
relied on a separate, pre-existing Order of Restoration which, although part of an
earlier plan, remained in use. The 2o14 EAP has an effective date of December 1, 2013. It
was produced by the Risk Management Department. There was some confusion at the
hearing of the definition of a catastrophe which would trigger the deployment of the
plan47.
Although it is an update of a pre-existing plan, we have substantial concerns, including:
a)
b)
c)
d)
e)
There was not clear direction on the trigger for deployment of the plan.
The CRT discussions with city and county emergency managers suggested that regional
planning activity is not organized and coordinated in advance of emergency events, but
instead, relies on the experience of the participants and their pre-existing relationships.
While that experience was evident during the ice storm response, there are some
difficulties with this approach. First, experience works only so far as that event, or a
similar event has been encountered and overcome. A properly trained EM cadre,
working for a common and comprehensive Emergency Operations Plan, will adapt to
the contingency confronted. Second, as noted elsewhere herein, when the entire EM
system is reliant on a pre-existing relationship for the trusted flow of information, that
relationship becomes a chokepoint in the information stream. Third, there is no
government entity that encompasses the entire BWL customer service area, which
stretches across three counties.
Continuity of Operations
Continuity of Operations Planning (or Business Continuity Planning in the private
sector) is the concept of strategic-level planning to assure that government departments
and essential services, whether privately-owned or public, are able to continue operation
of their essential functions under a broad range of circumstances including all-hazard
emergencies as well as natural, man-made, and technological threats and national
security emergencies48. Thus, emergency plan developers include continuity of
47
48
http://www.fema.gov/continuity-operations
27
b)
c)
d)
e)
f)
Findings:
1)
The BWL planning efforts were disjointed. The EAP and other plans need to
be replaced by a more comprehensive plan that includes tools for better
situational awareness, and specific plans for response and recovery.
2)
3)
28
4)
5)
The BWL lacks a Business Continuity Plan, although some necessary aspects,
such as evacuation of personnel, are contained in the 2014 Emergency Action
Plan.
Consolidate its multiple emergency response plans within the City and
Regional plan, (see below) and then test that plan by scheduling a full staff
coordinated table top exercise49 at a minimum of every 12 months.
2)
BWL Staff and upper management should participate in the exercises and
receive training on these plans and regular refresher training. BWL staff with
emergency responsibilities should be required to have training on these plans
and their role and responsibilities and the role and responsibilities of others
that will be involved with any emergency response.
3)
Together with the communities in the greater Lansing area (not just the BWL
service area), undertake a regional planning effort to be better prepared and
coordinated and assure that emergency communication protocols are agreed
to and followed. The mayors and township supervisors of our communities
must lead and encourage this effort and provide the necessary resources. This
planning effort50 should be done in coordination with the State Police
EMHSD District 1 Coordinator.
4)
At the CRT public hearing on 10 March 2014, the BWL was questioned on the use of an
Emergency Operations Center and on the communications between the BWL and the
City EOC. The BWL suggested to the CRT that they did not need an EOC , as the current
operations center, known as the BESOC, provided sufficient capability to manage
49
This recommendation for a tabletop exercise on its EOP, is separate from the recommendation to annually
participate in a regional full scale exercise.
50
The Michigan Public Service Commission has developed a Local Energy Assurance Plan template and occasionally
has held workshops which could be another resource.
29
operations during the outage. BWL advised that, in any event, there was not a need for
EOC activation.
The primary role of an EOC is to provide strategic direction and support for the actual
response organizations. An equally important role is to coordinate the response, so that
all entities are working cohesively and coherently to address the multiple nuanced issues
that will arise in an event. The EOC acts as the center of gravity for senior level decisionmaking and thus, links horizontally to other municipal governments and vertically to
higher-order government agencies. The EOC assures that the senior officials all have the
same set of data, the common operating picture so that when decisions are made, each
senior official, whether with that government agency or a linked government, will know
and understand the basis for the decision. When the senior officials are in the EOC, then
the decisions are emanating to the public and others from a common location. Because
of the potential impact on the personnel and equipment needed for daily governmental
services, during an event, the coordination or reconfiguration of day-to-day (i.e.,
normal) services to the remainder of the affected municipality, state, is commonly
accomplished through the EOC. Similarly, since senior-level decision-making is
facilitated through the EOC to assure, among other reasons, that the public, like the
government officials, are all receiving the same information and at the same time,
emergency public information is coordinated and disseminated through the EOC.
An EOC is vital to assure that a number of capabilities are present and shared between
all participating agencies and units of government. Those capabilities include:
1) Assurance that EOC internal operations are organized and based on NIMS/ICS
concepts and principles.
2)
3)
Use of dedicated software and other tools (CAD, GIS, WebEOC) to assure the
preservation of information, data and communications for further analysis.
4)
30
5)
The BWL advised that they followed their Emergency Action Plan. That plan includes a
process for the activation of the Command and Communications Center, providing
that, [i]n a catastrophic event, the Unified Command Team, charged with managing the
BWLs emergency response, will determine whether or not the situation warrants
activation of the CCC.
BWL published its Crisis Communications Plan in March 2014, in response to criticisms
of its inability to communicate with the public during the ice storm and outage. The
stated purpose of that plan is to assure a constant and consistent line of reliable
information from the BWL to its customers. That plan lists the members of its Crisis
Communications Command Team as the General Manager, Assistant General Manager,
the Director of Communications and the Director of Governmental and Community
Services.
The Unified Command Team and the Crisis Communications Team identify differing
personnel for crisis leadership. It seems apparent that the Emergency Action Plan and
the Crisis Communications Plan were developed and drafted independently of each
other, with no consideration for the potential overlap in planning, processes, and
personnel. This is only one example of why comprehensive unified plans are needed.
Of greater concern, the March 2014 Crisis Communications Plan, although expressly
detailing the forms of response for specific contingencies, does not reference any
emergency operations plan nor does it connect with any city or regional plan.
As detailed elsewhere herein, the CRT received thousands of pages of documents from
the BWL in response to their requests. Despite asking repeatedly, BWL has apparently
not preserved any records from the BESOC nor elsewhere in Electrical Operations, on
its actions in response to the ice storm nor in the restoration process. No
documentation equals no lessons learned. There apparently was no organization-wide
31
process for retention of key documents for subsequent analysis to determine what
worked and what did not. In answer to a question about the lack of organization-wide
exercises, BWL pointed out that every event is an exercise, that they had a windstorm in
November 2013, which downed 3000 lines and that there were lessons learned from
that event. Yet, there is apparently no documentation of any of the steps taken, nor of
the overall process used, to recover from the worst outage ever to affect the BWL
customer area.
CRT Planning and Response Working Group members met with City of Lansing
Emergency Management (EM) and Fire Department personnel. The City EM personnel
detailed their efforts to open and maintain the City Emergency Operations Center and to
assure a common operating picture for City leaders. The City EOC was in operation from
December 21 through January 3. It remained in operation to coordinate the response to
the heavy snowfall that would occur later that week. The state EOC was partially
activated on January 5, 201451. The City willingly provided the CRT with copies of the
City EOC records, emails with the BWL, and CAD maps created during the outage,
based on a verbal promise from the CRT Chair to Chief Trent Atkins that we would not
distribute them.
There is a substantial discrepancy between the accounts provided by the BWL Senior
Leadership and the City Emergency Management Office, describing the flow and volume
of communications between the BWL and the City during the ice storm and restoration
efforts between December 21, 2013 and January 3, 2014.
The BWL advised the CRT, both at the March 10th public hearing and again during an
April 10th tour of the BESOC, that BWL Operations officials called in to the City EOC,
every day, for the EOC conference calls. The BWL also advised that they had a
representative in the EOC during the outage.
In contrast, Lansing EM stated to the CRT Working Group that the BWL stopped
communicating with the city on or about Day 3 or Day 4 of the outage. Further, that
there was not a BWL representative in the EOC until 30 December. As noted, BWL was
asked to explain the discrepancy on April 10, 2014, and suggested that their phone
presence continued throughout. Yet, the communications log from the EOC indicates
that many calls to the BESOC went unanswered or unreturned. The City EOC hosted 1- 3
conference calls each day, staring on December 22, 2013 and ending on January 3, 2014.
BWL was fully engaged with the City on 22-23 December. From the 0800 hrs
conference call on December 24ththrough the 0800 hrs conference call on December
51
http://www.michigan.gov/documents/snyder/PR_140105_weather_briefing_p_m__update_443899_7.pdf
32
29th, however, there was no participation from BWL; the log indicating, Invited. No one
on call. As noted below, the EOC communications log indicates that there was sporadic
communication from BWL between December 23rd and 28th, 2013.
Excerpted from City EOC Communications Log
33
Further, the emails provided by the City EM only evidence email communications between the
EOC and BWL for the period from December 26ththrough January 3rd, and almost all concern
the routing of reports of outages from other sources to the BWL and their acknowledgment by
BWL. There are no emails between the BWL and the EOC prior to December 26th. The BWL
did not have a representative stationed in the EOC until December 30thand never had a
representative in the EOC with operational expertise or decision-making authority, other than
when the General Manager or other officials visited during the first two days of the outage.
Note: The CRT strongly asserts that this failure of communications cannot be attributed to
the Director of Electric Systems Operations. Electric Operations in the BESOC was
attempting to conduct restoration and emergency operations with the tools used during
normal operations. The lack of a cohesive and comprehensive emergency operations plan
included the lack of a system for scalable management staffing. Emergency operations
substantially increase the duties of the Operations section, including the need for contingency
scenario planning and implementation, recordkeeping and communications. Utility power
restoration, like many disaster responses, requires an influx of personnel to restore services.
The deployment of spotters, linemen and tree crews through Mutual Assistance Agreements
and contingency contracts necessitates an increased management capability for direction
and coordination. With 3 times the workforce on the ground than normal operation, to be
efficiently directed according to the order of restoration and evolving exigencies, required
management staffing that had not been planned for nor exercised prior to the outage. The
plan must therefore anticipate scalable management operations for the significantly
increased workforce and the increased duties of BWL management.
Findings:
1)
Despite the recent weather events, there is a lack of urgency of the need for
integration with the city EOC and with the regional community.
2)
The CRT believes that an integrated regional EOP is vitally needed. If a regional
EOP, which includes a regional EOC, was operating during the recent event, it
would have facilitated the coordination of services with first responders in
conducting well-checks and other services, and reduced the frustration of outcounty and other governmental leaders who were unable to get accurate
information, from BWL.
3)
4)
BWL Emergency Operations are not coordinated with the City of Lansing, nor
with any other municipal government that represents its customer base.
34
2)
Working jointly with the City of Lansing and other regional governments, develop
a regional EOP which includes a process for siting, supporting and sustaining a
regional EOC.
3)
4)
Extensive public comments were received by both the BWL and the CRT on the lack of
communications with BWL, both the inability of its customers to report downed lines,
discussed in the section on the Outage Management System, and on the lack of current, and
detailed information on the restoration efforts. The lack of knowledge of expected restoration
has an effect increasing geometrically, as it continued for some customers to the fifth, sixth,
seventh, eighth, ninth and tenth date without power, coupled with the BWL public statements
of 90% restoration. Those remaining customers felt overlooked and forgotten by the utility, as
they coped without power and heat for the holidays, discarded spoiled food, imposed on
families or neighbors and, in cases where they were able to find rooms at local hotels, returned
daily to their homes, to determine of power had been restored.
As noted in the Introduction, BWL did issue press releases three times per day, but the
information was limited to providing the number of customers without power and in most
releases, also provided the number of downed lines. Later releases added the location of line
crews and the expected location of line crews. The BWL spokesperson also had press
conferences twice daily. BWL senior officials acknowledged that they had no crisis
communications plan in place to manage messages to customers and other audiences. BWL
focused on their communications failings in their Outage Report: 19 of 54 recommendations
discuss Customer Communications and 9 additional recommendations discuss External
Communications. BWL commissioned an outside consultant to develop a Crisis
Communications Plan. A plan was issued in March 2014.
35
Findings:
1)
2)
BWL did not have available the information required due to the failure of the
OMS and the lack of capacity for customer calls.
3)
4)
BWL must create a long-term strategic communications plan to rebuild its trust
with its customers and community.
Create a robust social media presence for its customers this work is already
underway internally, but must become a priority for its communications
operations, as consumers are primarily using digital communication tools to
learn about outages and other service issues.
2)
3)
Further refine the March 2014 plan. The plan must contain greater detail on how
to address the need for timely and accurate information; customer information
must be consistently explained in a way that effectively meets customer
expectations; and the plan must assure the provision of information sufficient to
allow the public to make informed decisions on how they may best respond.
There is a considerable body of studies on this subject that should be drawn upon
in the development of such a plan. Once this plan is completed the BWL
management and employees must be trained on, periodically exercise, and follow
the plan.
4)
52
53
54
Section 4-102.1. The language in the Lansing City Charter is not as expansive as that in others. The City Charter
of Southfield, for example, provides that the mayor "in emergencies, have the powers conferred by law upon
sheriffs to prevent disorder, preserve the public peace and health, and provide for the safety of persons and
property."(Emphasis added). Note, however, that powers the state legislature grants to the office of sheriff are
quite extensive. The Michigan Sheriffs Association website contains a monograph by retired Kalamazoo County
Sheriff Tom Edmonds which lists 50 statutorily-prescribed duties of the Sheriff and notes additional common law
powers. See https://www.michigansheriff.com/pdfs/The%20Office%20of%20Sheriff%20by%20Edmonds.pdf
55
See, the duties prescribed for the executive branch of the City, particularly sections 234.05 and 234.07.
37
delegations of temporary, emergency authority to the executive branch and the need is
obvious: the contingencies arising during disasters or emergencies frequently demand greater
deployment of personnel for temporary duties, including those duties commonly fulfilled by
fewer employees, such as clearing roads and streets and those uncommon duties, including
clearing storm debris or emplacing sandbags. Further, such events often require streamlined
command and control systems within a government, to assure a rapid, agile and flexible
response to the crisis.
While the BWL is a creation of the City Charter, its employees, unlike the General Manager,
and although fulfilling a vital public function, are not considered municipal employees.
During the ice storm, the BWL employees acted independently from the City of Lansing
command structure and sometimes without communication or coordination. It is a truism that
strong disaster response requires strong partnerships. There was no evidence of such a
partnership between the BWL and the City EM during the ice storm outage. When questioned
about communications with other governments, BWL advised that they assumed that their
discussions with the City EOC would be passed on to the other units of government.
Both the City and BWL personnel spoke positively of the personal relationships they have with
officials in the other organization and some suggested that those relationships assured
sufficient communications during the outage. And the City EOC staff also stated they had
relationships and communications with some of the other communities emergency personnel
during the December outage. The same assertion was made concerning emergency operations
plans, that while there had been no coordination of emergency operations prior to
implementation, the personal relationships would assure that the mutual trust between
experienced operators would assure that a solution could be identified, agreed upon, and
executed, timely.
Such an assertion, however, is contrary to facts determined during the investigation, and to
expected protocols and best practices in the Emergency Management discipline. An overreliance on personal relationships creates its own chokepoint to information flow. When that
communication link between individuals, say between an electrical distribution operator in the
BWL BESOC and a first responder in the City EOC is interrupted (by shift change, fatigue,
injury, overwhelmed by other duties, etc.) then the link between agencies is lost. Further, no
formal communication plan had been established for information flow during emergency
events between the Lansing City EOC and other communities emergency personnel, other
than the City of East Lansing.
During emergencies, there must be a clear and clean command structure. This need is directly
stated in the Michigan Emergency Management Act which provides that the state Director of
Emergency Management shall assume complete command of the emergency operations,
38
including by employees of other state agencies, when expressly directed by the Governor56. The
Act also provides the Governor with temporary authority to commandeer private property, as
needed and subject to proper compensation.57 Within that spectrum of emergency authorities,
could reside the ability of the Mayor to direct the employees of the BWL during an event.
Findings:
1)
As discussed above, there is critical need for a regional EOC, or minimally, a plan
to ensure a coordinated response for emergencies and disasters across
governmental boundaries.
2)
There was a failure of communication between the City and the BWL and a total
lack of communication between the BWL and the other local governments which
represent the BWL customer service area. Assumptions were made on the
transfer of information, rather than the reliance on established, tested lines of
communication.
3)
Municipal and township first responders and officials and BWL employees were
often engaged in redundant tasks, or tasks that could have been more efficiently
performed by other EM officials or responders within the region, such as the City
EOC designing outage maps of traffic lights, multiple warming centers and
information centers, the disorganized exchange of outage information by emails.
4)
BWL was neither communicating nor coordinating with Lansing EOC after
December 24, 2013, and many of the other Emergency Operations Centers were
not open contributed to the confusion, lack of accurate information, and
uncertainty experienced by municipal leaders and citizens.
Consider amendment of the City Charter to clarify the powers of the Mayor and
to provide the Mayor executive authority over the BWL during disasters or
emergencies
2)
56
MCL.30.407(1).
57
MCL 30.405(1)(d).
39
3)
Work with BWL Operations and Senior Leadership to integrate their BESOC and
Crisis Command Center, using the NIMS framework, with the City EOC.
4)
5)
6)
Neither Ingham County, nor the Cities of Lansing or East Lansing, declared an emergency
during the ice storm or outage. Eaton and Clinton Counties declared emergencies on 27
December. Shiawassee County, immediately to the north of the BWL customer area, declared a
state of emergency, almost immediately after the storm hit.58. In April 2014, the Governor
issued separate emergency declarations for Marquette County, and for Newaygo and Osceola
Counties, to permit state resources to assist the counties in recovering from the effects of the
harsh winter59. Although the Michigan Legislature recently appropriated monies for ice storm
clean-up.60, without the pre-requisite of a gubernatorial declaration, that eventuality was
unknown in December.
While, in the publics perception, the need for state or federal resources is the prompt for an
emergency declaration, the judicious pro-active use of the declaration is an essential tool for
municipal governments. An emergency was declared by the State and the City, less than one
week later, in anticipation of the heavy snowstorm61. A similar anticipatory declaration would
have provided the Mayor with the authority to deploy city employees as needed, including to
assure full door-to-door, health and welfare checks of the neighborhoods, and to assist in
damage assessments and debris-clearing.
While Lansing is used as the example, this recommendation applies to all county, city and
township governments within the BWL customer service area. The ability to do so, in response
58
http://www.abc12.com/story/24284184/shiawassee-county-is-in-a-state-of-emergency
59
http://www.michigan.gov/snyder/0,4668,7-277-57577_57657-326671--,00.html
60
http://www.mi.gov/documents/msp/IB_-_13-06_-_Supplemental_-_04.22.2014_454325_7.pdf
61
http://www.lansingmi.gov/media/view/mayordeclaressnowemergecy/6226
40
to an extended outage, presumes that BWL is consistently and fully communicating with the
affected governments, through a jointly developed communications plan, or as an Annex to the
proposed Regional Emergency Operations Plan.
The CRT Recommends:
That all units of government within or partially within the BWL customer service area review
their emergency operations plans to include a process for assessing the need for an anticipatory
emergency declaration.
41
62
See BEST PRACTICES FOR STORM RESPONSE ON U.S. DISTRIBUTION SYSTEMS, supra Note 15.
63
64
Id.
65
Id. Also, Connecticut October 2011 Snowstorm Power Restoration Report, Witt Associates, December 1, 2011, p.21.
42
restoration, prior to dispatching crews to perform the repairs, was the greatest bottleneck in
the restoration process.
But industry and BWL recognition of the problem does not excuse it, as the Best Practices
compiled by utilities with similar storm-caused outages in the last few years attest. The BWL
had only 18 spotter crews available on December 22nd, rising to 23 crews on December 27th.
More telling, the need for spotter crews never diminished, as spotter crews, in similar
numbers, were deployed throughout the time period of restoration, save for individual
breaks.66 BWL has identified some specific steps67 to address the spotter bottleneck. In the
2011 Connecticut outage, reviewers suggest that it would have been prudent to pay standby fees
to out-of-state crews, given the lack of local mutual aid assistance available due to the size of
the storm and the lengthy delay in restoration caused by the non-availability of sufficient
spotters. Pre-staging spotters can be cost-effective by reducing the overall cost of repairs and
the amount of time electricity is out if there is significant damage. Connecticut Light and Power
Company is an order of magnitude larger than BWL, with almost one million customers and is
investor-owned.
BWL concedes that it lacked sufficient linemen internally to address the scope of the
restoration. BWL, due to its relatively small size, always planned to rely on Mutual Aid
Agreements (MAA), primarily through the Michigan Municipal Electric Association. This
proved an insufficient pool of assistance crews, however. On December 22nd, BWL reports that
it had 20 line and tree crews available. From December 29th-31st, they deployed 54 line and tree
crews. The documents received from BWL and its Internal Outage Report describe the
scramble to identify the number of crews needed, to locate the crews and to execute
contingency agreements (both for spotters and line crews). In response to the CRT second
request for information the BWL reported that several mutual aid agreements were executed
during the restoration and they initiated a RFP for additional mutual aid agreements following
the December 13, 2013 storm. Contracts and billing statements for mutual aid agreements
during the storm were provided to the CRT and reveal that there was substantial cost for
external aid during that period. It is difficult to determine if the cost for those external services
provided by entities that did not have mutual aid agreements prior to the storm were more
than if the BWL had had pre-existing agreements. But it is clear that following the storm the
BWL is increasing the number of mutual aid agreements it has.
As noted above, BWL conceded to the CRT that a functioning OMS would have likely reduced
the outage by 2 days. This admission, coupled with their acknowledgement that they did not
have sufficient spotters to ascertain the extent of the lines down, tends to confirm that BWL
66
67
Items 10-14, BWL Ice Storm Outage Report (adding spotters and line crews (addressing the bottleneck) and training for
spotters, developing common reporting forms & potentially contracting with electric service firms for damage assessment).
43
lacked awareness of the scope of the problem and explains in part why their initial response
was not as robust, efficient or quick as it should have been had they known the full extent of the
damage. This lack of situational awareness then caused unduly optimistic early reports to the
media and to local emergency managers that they had the situation under control and further
resources or actions were not needed. For example, the BWL media release that said
customers should only report down lines and not power outages further hampered their
situational awareness. While BWL may have known the number of customers without power,
they did not know the extent of the downed lines, which increased the time needed to restore
power to the circuits and then the time needed to repair feeder lines and then individual
customer lines. While this problem is partly traceable to the subsequent weather, which
knocked down additional lines, it was directly and largely caused by lack of sufficient number
of spotters at the onset of the storm and the loss of the OMS.
The lack of good information to quantify the full scope and potential duration of the restoration
and recovery led to decisions that tempered the response effort. In an energy emergency, the
quality of the initial assessment phase is critical to the response and recovery effort. It is wiser
to plan, and be prepared, for the worst-case, and then stand down, than to under-prepare and
have to mobilize late, prolonging the recovery efforts.
During the public hearings held by the CRT, there were several questions involving how many
BWL employees were required for a crew repairing downed power lines and whether a BWL
employee had to accompany outside contracted repair crews. The CRT has reviewed the
current collective bargaining agreement (effective 11.01.2012 10.31.2016) between the BWL
and the International Brotherhood of Electrical Workers, AFL-CIO local 352. Article XI of that
Agreement address safety and establishes a Safety Committee to Joint Safety Committee
comprised of equal numbers of Employer and Union representatives to develop safety rules.
Article XI, Section 9 of the Collective Bargaining Agreement provides in part:
Only as a reflection of a custom in the electric utility industry to express specifically in a
collective bargaining agreement, as well as in the Safety Rules, and the understanding of the
parties to the particular Agreement on this aspect of electric utility work- the parties hereto
agree that two (2) or more employees, qualified in the judgment of the immediate supervisor of
the job, shall work together whenever wires or equipment are energized at more than 500 volts
to ground or, in the judgment of the supervisor of the job, wiring is congested or unusual
exposure is involved. ---- To accommodate working one person primary, exceptions to the
language in the preceding paragraph may be made for specific situations. Such specific
situations must be approved by the Joint Safety Committee, ratified by those persons currently
holding the classification of Line worker and must be in keeping with industry practices.
Specific situations shall become part of the Board of Water and Light Safety Manual and
appear under a heading Rules for one person in primary. Two person line crews shall be
made up of two Bargaining Unit journey workers or a qualified apprentice.
44
In discussions with BWL staff, the CRT was told that it is a requirement that one BWL
electrical utility worker accompany any line crews providing line maintenance under a mutual
aid agreement. It was stated that this requirement is for safety reasons to ensure that repairs
and restoration of service is in compliance with the BWL standards, i.e., to ensure that when a
BWL employee has to maintain that line or connection in the future she/he is ensured that it
meets BWL standards. BWL noted that the three-man crew requirement only applied to
primary, not secondary, lines, and that the BWL supervisor could supervise more than one line
crew in an area. There was discussion, based on citizen and outside expert suggestions, that the
requirement for three-man crews unduly delayed the restoration period. While the timeline
suggests that BWL should have called in crews sooner and should have had more contingency
agreements executed and ready, there is not sufficient evidence to conclude that the size of the
crews was a factor.
Findings:
1)
2)
BWL concedes that they did not have sufficient mutual aid agreements in place,
and has taken steps to address this deficiency. But while there may have been
sufficient linemen, by the end of outage period, there does not appear to be
similar Agreements for Spotters, even though that is an acknowledged "choke
point".
3)
BWL has hired three additional linemen, according to its Outage report.
Given that insufficient spotters for damage assessment was a serious problem in
the December outage, identify the types of spotters necessary and currently
lacking.
2)
3)
Identify specific personnel for spotter duty and training for deployment in
emergencies. This function should be mandatory, not voluntary on the part of the
employee, as it appears to be now. Training should also be mandatory and held,
minimally, once per year.
45
4)
That the two person line crew requirement and the requirement that a BWL
employee accompany the outside contracted line crew are reasonable to ensure
safety of BWL employees.
Outage Management System
The BWL determined that it needed a new Outage Management System (OMS) in 2010. The
OMS is designed to model outage status in the field in order to aid in managing operations by
predicting the cause of a cluster of outages and giving estimated restoration times. This system
allows data about outages to be submitted directly to the OMS via customer telephone calls.
The BWL received two bids, from General Electric and CGI, and selected the bid from GE.
BWL began implementation of the OMS in early 2013, despite the fact that the purchase
agreement was signed in late 2011. After considerable delays and missed deadlines, the OMS
was installed and operational by February 2013. Bugs had become evident by May 2013,
requiring further servicing by GE. The OMS was not operational again until November 2013,
just weeks before the December ice storm. A storm in November 2013 revealed some
additional issues with the OMS, including repeat calls from the same customer being reported
as a wider outage and problems with conductor cuts. These issues were reported to GE, but
fixes were not put into place before the December 2013 ice storm.
Note: There are gaps in the information provided the CRT on implementation of the OMS,
including why a system contracted in late 2011 was not installed until February 2013. The
CRT heard, but was not able to confirm, that the installation in early 2013 was done only
after BWL threatened to withhold payment to GE after multiple missed deadlines. Further,
we were told that installation was done too quickly, that defects surfaced in May, and again
in November 2013, and that the fixes contracted for these defects were still not completed in
December 2013 when the ice storm occurred.
The link between the call center and the OMS was tested prior to the ice storm, but was never
tested under the load experienced during the outage. Alarms designed to alert the BWL if
either the call center stopped sending data or if the OMS stopped receiving it were not tripped.
During the outage, data from thousands of customer calls was not sent from the call center to
the OMS.
Besides its inability to process the sheer volume of data received from customer calls, emails
and data inputted by BWL CSRs, the OMS had another flaw: it could not account for the
conductor cuts. Conductor cuts are intentional cuts of power lines in the field made to close
in a circuit to restore power. The OMS was not able to recognize that the intentional cuts had
continued loss of power and reported that the entire circuit was restored. As a result,
customers calling in December to report power outages in the areas serviced by the
intentionally cut lines were told incorrectly that their power had been restored.
46
This disconnective flaw in systems between the operations process used by the restoration
crews and the information system created by the IT department and its contractors made an
organized, considered plan of restoration impossible. Because the OMS was detracting from
the operators ability to accurately track the scope of the outage, the BWL Electrical Operations
decided to discontinue its use. The decision to pull the plug on use of the OMS was made on
either December 22nd or 23rd of 2013. All data, whether from customers or BWL employees, on
location or causes of outages, that had been entered in the first two days of the outage was thus
inaccessible by Operations, and had to be re-created. When the conductor cut issue with the
OMS system was fixed and the OMS was re-started five days later, the data on OMS was lost in
the re-start.
Without the OMS, and without a backup system, Electrical Operations had to resort to an ad
hoc paper system, which Operations candidly and repeatedly described as either organized
chaos or chaos: the Spotters had to manually enter their findings on a paper report and
drive those reports to the BESOC, where the reports would then have to be analyzed and then
compiled into an order of restoration according to the pre-existing priority system, not as
chronologically received or based on the size of the outage. The work orders to the line crews
were then created and documented on paper and the crews dispatched via 800 Mhz radio. But,
when the crews had finished at a location, the record of completion had to then be manually
created as well. A stand-alone laptop in the BESOC was used to create an Excel spreadsheet
containing this data.
BWL asserted to the CRT that if the OMS System had been operating fully in December 2013, it
would not have significantly accelerated restoration. This statement seems to contradict other
statements made by BWL officials. We were told that the OMS failed to accurately report the
number of downed lines early on, and that this, in part, is why too few damage assessment
teams were dispatched. Once dispatched, they were unable to communicate directly from the
field to the OMS, which resulted in delays getting line crews out, etc. etc. The malfunctioning
OMS was a critical, missing, element in the response and restoration actions of the BWL.
Certainly the most common but often the most expensive form of resiliency is to maintain a
redundant system. BWL was specifically questioned why they did not maintain their preexisting outage management system until the new General Electric OMS system was fully
functional. Their reply was that it would be time-consuming and potentially confusing to enter
the data twice- first to the OMS and then to the pre-existing system. While the CRT concedes
that the maintenance of two systems is labor-intensive, certainly there is a spectrum of
solutions between redundant full systems and an ad hoc paper system. Maintaining the preexisting system during 2012-2013, given the known recurring problems with the OMS, would
have been worth the expense in labor costs. At a minimum, the OMS, which had a number of
patches applied during implementation, could have had a separate backup data storage
installed, so that even if they used the OMS exclusively, there would have been redundant data
storage.
47
Integrate the OMS into an Emergency Operations Plan (EOP) which is tested to
its maximum capacity68., both to assure system functionality and to assure
integration with restoration operations, on a semi- annual (6 months) basis. Staff
noted that because they had not experienced an outage of more than 20,000
customers, they assumed that was the maximum ever likely. As noted above,
recent reports indicate that storm frequency and intensity are on the rise. Since
the outage, the now apparently functional OMS has been tested for loss of power
to 35,000 householdsstill fewer than lost power in the December outage.
Therefore, the system must be tested to its maximum capability
2)
3)
68
Item 23 (BWL Ice Storm Outage Report, p. 45) only calls for periodic stress tests. The OMS needs to be tested regularly
with maximum capacity demands, including coordination with IVR & 877 systems (Outage Report, pg 49) & call center staff.
Further, it needs to be tested in simulated emergency exercises with all staff participating. As discussed elsewhere herein,
there is a lack of coordinated, utility-wide training for emergency events and a lack of senior-leadership emphasis on
training and readiness.
48
The CRT experience in obtaining records from the BWL was replete with instances where
records were not forthcoming, in some cases, ever. There are a variety of reasons: (1) the BWL
senior leadership does not preserve its emails; (2) BWL adheres to a policy of non-distribution
of any records it deems sensitive information; and (3) since there is no institutionalized
system for communications during emergency events with the municipal governments which
represent its customer service area, neither is there a protocol for preserving the records of
those communications.
While the BWL produced more than 4,000 pages of email printouts, including many duplicate
email chains, the response was accompanied by a caveat that 20,000 pages of emails (a
particularly large number considering the claim that senior leadership does not maintain its
emails) had to be reviewed to be culled of material that the BWL considered sensitive of
confidential.
The Records Retention Policy and Records Retention and Disposal Schedule received from
the BWL asserts that it is their corporate policy to only retain records during the period of
immediate use and then, unless required by law to be retained, are sent for disposal. While
their policy undoubtedly adheres closely to applicable state law, it also makes meaningful abter
action analysis impossible.
The BWL is a publicly-owned utility and thus, the owners of the utility have a right to expect a
full review by its representatives, subject to narrow statutory exceptions. Without any record of
its actions, the public and its representatives have no means to independently confirm or
validate the actions of the BWL.
As an example: the CRT received a summarized chart of the BWLs Order of Circuit
Restoration in response to its first request for documents and answers along with copies of
press releases listing the street intersections that crews were working on each day. In order to
69
BWL was specifically questioned why they did not maintain their pre-existing outage management system until the new
General electric OMS system was fully functional. Their reply was that it would be time-consuming and potentially
confusing to enter the data twice- first to the OMS and then to the pre-existing system. While the CRT concedes that the
maintenance of two systems is labor-intensive, certainly there is a spectrum of solutions between redundant full systems
and an ad hoc paper system. Maintaining the pre-existing system during 2012-2013, given the known recurring problems
with the OMS, would have been worth the expense in labor costs. At a minimum, the system, which had a number of
patches applied during implementation, could have had a separate backup data storage installed, so that even if they used
the OMS exclusively, there would have been redundant data storage.
49
compare the Order of Restoration to the actual restoral process, the CRT requested a list of
each circuit and a map demarcating the location, by nearest street address, of all circuits listed
in the Circuit Restorations Schedule but was told that information was considered sensitive
and could not be reviewed70. The BWL relied on the North American Electric Reliability
Corporation (NERC) guidelines in its refusal71. The NERC Guidelines, however, clearly
envision that the utility will have developed a policy or plan for the handing and labeling of
sensitive documents. The CRT expressly requested, in its Second Request for Documents, the
BWL Policy on classification of plans and other documents72. None was ever received, leading
to the conclusion that none existed. The CRT could not independently determine whether the
Circuit Restorations Schedule had been followed.
The industry has spent some time and energy on the issue of restoration planning. The
February 1, 2012 edition of TD World contains the results of a survey of 250 utility
professionals. A key finding was the interest in automating their restoration planning and
action processes. This effort is more amenable to large utilities with the available initial and
continuing funding than smaller utilities73. Other independent surveys, studies, or consulting
reports have produced various recommendations for restoration best practices74. This
publication contains a reasonably comprehensive explanation of the actions, technical and
administrative, necessary to prepare for and effectively respond to a large outage, including
management and mobility of emergency resources and emergency staff before and during the
crisis; damage assessment; crew deployment; external and internal communications; and a
process to ramp-down during recovery after restoral.
Thus, even internal restoration plans must include mobilization or deployment sequences or
be little more than technical guidance for the field and line engineers or lists of personnel and
entities to contact.
The information, which should be maintained during a power restoration, which also should be
maintained for post-restoration documentation:
Those guidelines broadly define sensitive information as any data or information which could be used to target electric
sector critical infrastructure, damage facilities, disrupt operations or target individuals. The guideline continues, however
and asks a series of four questions for the utility to analyse whether the date or information should be considered
sensitive. The only question which could apply is the query, Could someone intent on causing harm [to facilities] use the
information to his or her advantage? Again, a question so broadly written as to include any potential operational
information.
71
72
See Request #3, CRT Second Request for Documents and Answers to BWL.
73
74
BEST PRACTICES FOR STORM RESPONSE ON U.S. DISTRIBUTION SYSTEMS, by Lavelle A. Freeman GE Energy/DSTAR ,
Gregory J. Stano Wisconsin Public Service Corporation and Martin E. Gordon Independent Consultant , presented at the
2010 DistribuTech, March 23rd, 2010, Tampa, FL
50
1.
2.
3.
4.
5.
6.
2)
c.
Set of maps, optimally GIS, depicting the stages of the event, including at
least, the initial outage area and affected components and customer zones,
and stages of restoration sufficient to recreate the series of restoration
actions leading to full restoration.
d.
e.
f.
g.
Log of physical access entries, either IAW or similar to NERC CIP standards
for Physical Access to BES sites.
51
3)
h.
Log of all connections to the local control systems during the restoration,
including full hardware/software descriptions and each connecting devices
security certificate.
i.
Retain this documentation in a form that enables efficient use, recall, and reuse,
and in a format that is compatible with performing the same series of retention
actions for future events; i.e., not a one time storage, but with future use and
comparison in mind75.
75
List compiled for the CRT by Mr. E. Daniel Mathis , a retired expert from the Mission Assurance Division, Naval Surface
Warfare Center, Dahlgren, Virginia.
52
76
In its responses to the CRT, the BWL produced several Vegetation Management plans, which variously described a three,
five, and seven year rotation for trimming in its customer area. When asked about the discrepancies, the BWL explained
that what started out as a three year cycle became a five year cycle when it was determined as maintenance was
performed that the entire service area could not be completed within three years, which turned into a seven year cycle
when it became obvious that the maintenance could not be completed on the five-year time table. BWL also acknowledged
that many citizens have previously complained about the loss of their trees from a rigorous tree-trimming program, which
in the past caused BWL to leave in place many branches or trees which overhang power lines.
77
BWL Internal Report, Recommendation 7: The BWL will strictly adhere to a 5 year trimming cycle and is in the process of
modifying its tree trimming standards and procedures to remove all branches located above distribution lines and better
protect its lines during a major storm.
53
public trees located in the City rights of way or in public parks. In a number of locations,
especially neighborhoods, the power line rights of way are located in rear yards along property
lines. The City of East Lansing also noted that when BWL needs to trim trees in East Lansing,
the BWL or their contractor usually notifies the City and advises of the locations where they
will be completing trimming or removal work. The City ensures that the BWL or their
contractor notifies the affected property owners. Many times, depending on the tree work
scope, the City will issue a press release to advise its residents of the work location and of the
BWL contacts.
Findings:
1)
2)
3)
4)
When problem vegetation is identified outside of the utility right of way, efforts
should be taken to communicate to the customer the dangers of the hazard. The
communication should be documented.
5)
78
2)
3)
Perform an 100% audit of all lines annually to ensure both that the BWL can stay
on track in its vegetation cycle and that sections that may require immediate
attention are not neglected.
4)
Lansing Board of Water and Lights electricity distribution system includes a large number and
a substantial variety of components. The management of those assets is a critical activity that
affects the reliability of the BWLs distribution grid. According to BWL staff, the current budget
for distribution grid maintenance is based on historical expenditures. Based on data presented
in the BWLs report on the outage, however, the rate at which various components are replaced
in a typical recent year appears to be well below the rate of replacement consistent with the
expected life of those components. For example, it appears that distribution lines are being
replaced at a rate consistent with a 120-year average life, which is unreasonably long. Our
experience is that organizational decision-making about whether equipment conditions
warrant replacement is strongly shaped by the relative paucity or generosity of the
maintenance budget. Our recommendations for grid hardening and improvements in the
BWLs ability to repair its distribution system after storm damage can be implemented at the
appropriate pace if supported by an asset life-cycle based budget rather than historical
expenditures that may not match even routine maintenance needs.
The CRT also notes that some grid maintenance or modernization activities can be done more
cheaply or with less cumulative impact on adjacent properties if done in coordination with
municipal infrastructure work.
Findings:
1) BWL reliability is dependent upon the proper maintenance and management of
electrical distribution system.
2) The BWL rate of replacement is below the rate of replacement based on the expected
life of the components.
55
2)
3)
Critical Facilities
Securing Downed Lines
Approximately 400 lines broke during the December 2013 ice storm. Most of the restoration
work during the first two to three days appears to have been securing downed lines.
The number of downed lines that must be secured can be reduced through general grid
hardening, which we address below. This section focuses on accelerating the process of
securing downed lines so that energized lines are not a risk to public safety.
The CRT believes that, upon investigation, the BWL will find that use of modern distribution
circuit interrupters that can automatically detect, isolate, and report on line breaks, arcing
currents, and other circuit safety problems will be warranted at key locations in the primary
distribution system and perhaps at the origin of each distribution feeder line. We further think
it likely that remotely operated breakers/reclosers are warranted on the down-current side at
each branch point in the secondary distribution grid and perhaps at selected other locations in
longer feeder lines. We anticipate that systematic comprehensive deployment of circuit
interrupters will enable the BWL to effectively turn off power to all downed line locations very
quickly and without using valuable spotter time to guard down lines and line crew time to
perform line cuts upstream of downed lines.
A significant portion of live downed lines are service drops from the distribution system to
customer buildings or damage to distribution lines as a result of service drops being pulled
down. It is possible to reduce this problem, accelerate the repair of service drops, and reduce
the problem of damage to customer masts by using service drops that unplug from the
distribution line when weighted down. The Electric Power Research Institute has completed a
demonstration project using a product that combines service drop breakaway with overload
protection, which may be the preferred product category for an effort like we recommend. The
cost of breakaway service drop devices is small, so the BWL can minimize the cost of installing
breakaway service drops by concentrating on strategies that minimize labor costs, but the BWL
should undertake a benefit-cost analysis as to whether some more aggressive plan for the
installation of breakaway service drops might be worthwhile.
56
Findings:
1)
A significant portion of the live downed lines were service drops from the
distribution system to customer buildings or damage to distribution lines as a
result of service drops being pulled down.
2)
The use of service drops that unplug from the distribution line when weighted
down would reduce the number of customer masts damaged or pulled down by
customer service lines.
2)
3)
The primary distribution system is addressed separately because faults in the primary system
affect large numbers of customers and require greater skill to repair. In the December 2013 ice
storm, repairs to the primary distribution system took from the second to the sixth day of the
outage, and the repairs to the primary distribution system effectively restored service to a
substantial share of the Boards customers.
There are a variety of options to reduce the extent of damage to the primary distribution
system that will result from damaging storms. These include, but are not limited to:
Each of these practices has costs and benefits that vary with location and conditions. Neither
the CRT nor the BWL are currently in a position to evaluate the best options in each location of
57
the BWLs grid. We simply note that the current design of the grid is a result of historical
decisions when reliability expectations, weather, and technological options were different than
they are today and therefore should not be assumed to be optimally designed for current or
future conditions.
In performing this value engineering analysis and in implementing grid improvements and
maintenance practices, the BWL should be cognizant of opportunities to reduce costs (or
improve results) through collaboration with host communities. For example, it will be
significantly cheaper to place a line underground if conduit and vaults for this purpose are
constructed in conjunction with municipal street, water, or sewer projects than if the work is
done entirely by BWL as a separate activity.
Finding:
BWL was engaged in repairs to the primary distribution system from the second to the sixth
day of the outage, which effectively restored service to a substantial share of the Boards
customers.
The CRT recommends that the BWL:
1)
2)
Based on representations from the BWL, we believe that it currently has sensors in place
which allow it to automatically detect outages in the primary distribution system. As
discussed above, BWL staff advised that the process to repair the primary distribution
system following the ice storm was rate-limited by the ability of spotters to patrol and
localize the line breaks causing the outages rather than by the ability of repair crews to
make the necessary repairs. The Board should seek methods to accelerate primary
distribution system fault localization and identification.
Finding:
The identification and assessment of the location and cause of outages in the primary
distribution system was a substantial cause of the length of time for restoration.
58
2)
Finding:
Protection of the secondary distribution system requires more than just vegetation
management.
59
DTE has installed approximately 1.2 million Advanced Metering Infrastructure (AMI)
meters which represents about 65% of their residential customer base of approximately
1.9 million.
Consumers Energy has installed approximately 227,000 AMI meters which represents
about 9% of their residential customer base of approximately 2.49 million.
Each company forecasts complete installment of AMI meters over their entire residential
customer base within the next few years79.
The benefits of Smart Grid technology is discussed in the MPSC Staff report on the December
2013 Ice Storm response by Consumers Energy and DTE. It noted that a Department of
Energy report on initial results from Smart Grid Investment Grants to utilities for installation
of distribution automation devices as a means to: 1) Reduce the frequency of both momentary
and sustained outages, 2) reduce the duration of outages and, 3) reduce the operations and
maintenance costs associated with outage management.
79
The BWL indicated in its Ice Storm Outage Report that it is moving forward with a project to
implement smart grid and smart meter technology which will allow it to identify individual
customer outages.
In the interim, until the BWL can use smart meters for outage and restoration detection, we
recommend the Board focus on increasing the corps of qualified spotters at its disposal for
inspecting and localizing faults in secondary lines.
We also note that the use of breakaway service drops as discussed above will reduce or
eliminate the problem of service drop breaks causing breakers to open in the secondary lines,
which will mitigate some secondary outages and outage repair times.
Findings:
1)
The greatest issue with repairs to the secondary distribution system is the
discovery and localization of the faults.
2)
3)
The use of smart meters will be of great benefit in assisting BWL in the
identification of pocket outages or those residences which are awaiting mast
replacement.
Proceed as quickly as is consistent with good practice to deploy smart meters and
integrate them to its outage management system to accelerate the identification
and repair of the secondary distribution system after storm damage.
2)
In the interim, ensure that it has an adequate core of spotters for restoration of
the secondary distribution system following a storm.
3)
Many of the strongest criticisms were voiced by those citizens who resided on short lateral
circuits or had lost power when their service line was disconnected from the service mast
disconnecting from their residence. The prevailing question was: Does the BWL have a plan to
address pocket outage identification, esp. prior to installation of smart metering? It would
seem that if BWL had known where the pocket outages were located, they could have
addressed them more timely, given that they said they had sufficient work crews.
61
Statements made at the March 10 hearing seemed to confirm the customers concern that the
BWL did not seem to include the individual outages as either a pocket outage or in their totals
when addressing restoration. Many of those individual households that were without power
were awaiting independent contract electricians. This three-step process, requiring that
customers find an electrician on their own (even from a list provided by the board) after the
secondary distribution system is repaired, was a source of huge frustration and anger in
December. The anger was still evident at the public hearings in January and February.
Customers anger was exacerbated by the delay caused by having to locate the electrician,
repair the mast, and then await full restoration in the system, freeing up a BWL to come
restore their individual power and by the reports of price gouging by contractors80
Findings:
1)
An innovative solution to this issue will substantially assist the BWL in regaining
public trust and confidence.
2)
As noted above, the use of service drops that unplug from the distribution line
when weighted down would reduce the number of customer masts damaged or
pulled down by customer service lines.
3)
The use of service drops may be a more efficient solution, both in terms of cost
and operations, than owning the system, including the mast, through to the
meter box.
Consider plans to own the system, including the mast, through to their meter
box, as part of its benefit cost analysis to determine whether a proactive effort to
install breakaway service drops is warranted.
2)
The public expressed great concern to the CRT at public hearings and in emails concerning the
lack of the knowledge by BWL on the location of their elderly customers and those with
medical needs. Citizens also expressed concerns over communicating with seniors during an
emergency as many of them may not have access to social media or the use of a cell phone data
plan. Many citizens with medical issues expressed frustration over the lack of communication
during the storm. While, by law, Michigan utilities must assure priority for the continuation of
80
Citizens advised that the price for service mast restoration ranged from $300.00 to $1800.00.
62
power to those with medical needs,81 there is not a similar legal requirement for its seniors.
BWL leadership affirmed that it had two different programs, Medical Alert and Senior Shutoff
Protection, during the March 10 hearing. (App. E.) BWL conceded that the programs were not
identical in either the requirements for participation or in the information provided to
customers about the programs. Neither the elderly nor those with medical needs receive
priority for restoration of services. In fact, BWL officials indicated that those on the priority list
may not have been reached until December 28th or 29th, several days into the storm. The
location of the residences of the elderly and those with medical needs was not wholly known by
the BWL. To obtain locations for the elderly in Lansing and East Lansing, Lansing EM received
data from Tri-County Office on Aging, plotted that on a CAD map, and provided that to the
CERT teams performing health and welfare checks and to first responders. Citizens also made
recommendations, including using voter registration lists or partnering with Sparrow Hospital
or local organizations such as Meals-on-Wheels to determine the locations of senior citizens.
Findings:
1)
2)
3)
The items in the BWL Ice Storm Outage Report on this issue , 82lack specificity.
BWL must update its critical & public safety lists and work with local officials for
awareness of seniors and other vulnerable populations, work with local
organizations to maintain warming and cooling centers, work with local
neighborhood groups and non-governmental relief organizations and empower
neighborhoods to react to emergencies.
81
Michigan Public Service Commission, Consumer Standards and Billing Practices for Electric and Gas Residential Service,
Part 9, Rule 47 Available at http://michigan.gov/documents/mpsc/FAQall011408_221482_7.pdf.
82
2)
3)
As part of the integrated Regional EOP, provide the regional EOC (or all EOCs or
EMs for all governments within its customer service area, if no regional EOC is
created) with up-to-date information of the location of its at-risk customers.
4)
Revise items 44, 47, 48, 54 of the outage report. There must be a coordinated
effort between BWL and local government emergency personnel and community
groups to assure clear communication and coordination of efforts to protect atrisk customers when an emergency arises.
Community Resilience and Preparation
This report has already discussed the need for system resiliency and the need for the BWL to
change its philosophy from viewing restoration of power as its main effort during an event to
recognition of the role of guarantor of the resiliency of the electrical power for the community.
The CRT strongly asserts that the Tri-County region must also focus on community resiliency83
to emergency and other events. A community can be broadly defined as an entity that has
geographic boundaries and shared fate84. Certainly that is a fitting definition here, where the
community of BWL users awaited restoration of power.
Community resilience as used herein is focused on creating the organizational behavior for
disaster management, as both a set of capacities for addressing the disaster and as an overall
strategy of engagement to assure early and effective disaster readiness and response.85Further,
83
Russell Shorto, How to Think Like the Dutch in a Post-Sandy World, New York Times Magazine 9 April 2014: Resiliency is
intended as a nonpolitically charged way of getting at issues underlying climate change: the need to rebuild in ways that
take ecology, economy, infrastructure and weather uncertainty into account. Much of the credit for the change in thinking
has to go to Hurricane Sandy itself, which hit in one of the richest, most populous parts of the country and also the center
of American media. And it came after a series of catastrophic events Hurricane Katrina and other storms, but also 9/11
and the banking crisis of 2008 and the subsequent global economic downturn which, taken together, seemed to solidify
the feeling of living in an age of chronic uncertainty. http://www.nytimes.com/2014/04/13/magazine/how-to-think-likethe-dutch-in-a-post-sandy-world.html?hpw&rref=magazine&_r=1
84
and Strategy for Disaster Readiness, Am J Community Psychol (2008) 41:127150, 128.
85
Id., 135-141.
64
both the three February public hearings held by the CRT and the three held January 15th-17th by
the BWL were replete with testimonial examples of neighbors helping neighbors. This
communal engagement and assistance needs to be harnessed, energized and organized prior to
events, however. Despite the overuse of the term, there is synergy in partnering disparate
public and private resources into a communal plan for response to events. The many benefits
include ownership and engagement and the release of governmental resources for other
emergency response tasks. Space and time constraints prohibit the CRT from identifying all
regional assets that should be utilized in developing community resilience. The Red Cross has a
lengthy record of this service and was active in deploying warming facilities and shelters,
coordinated through the City EOC. The Capital Area United Way involvement with 211 is a
valuable link with support organizations. The Tri County Office on the Aging was active in the
City EOC, as well: they identified the locations of their clients for the Cities of Lansing and East
Lansing, which were mapped on a CAD map in the EOC which was then used by police and fire
for health and welfare checks. The City of Lansing also deployed its CERT86 teams on the first
two days of the outage, both for damage assessment and for health and welfare checks.
As an example, though, of the expansion of community involvement, the Lansing Police
Department's Neighborhood Watch has historically helped organize block groups and train
block captains in promoting safe neighborhoods. This effort could conceivably operate out of
that office or out of Emergency Management. The City/LPD/Emergency Management and the
Power of WE (which already operates both an AmeriCorps and a VISTA program) could
collaborate in the organization of a regional community resilience program. These organizers
could assist in pulling together residents to create block plans, e.g., helping to organize initial
block meetings, creating block level parcel maps with pertinent information (household names,
children, pets, resources such as generators, shut-ins, etc.,), helping blocks to identify
neighbors responsible for critical functions (communications, well-checks on vulnerable
neighbors, basic needs), and more. A well-planned and coordinated response at every level-households/blocks to municipalities/regions will go a long way toward reducing the confusion
and helplessness that characterized the December 2013 outage. Conceivably, there may be
grant funds through FEMA, the Michigan Public Service Commission, or from the Corporation
for National and Community Service to acquire VISTA volunteers to serve as 'organizers'.
Similarly, the two cities and many of the townships are endowed with well-organized
neighborhood associations. For example, Lansing has the Neighborhood Watch Coordinator
and the Coordinator of Lansing Neighborhood Council, each of whom have communications
links to 187 and 36 leaders of organized neighborhood watches and neighborhood associations,
respectively. Further, two large neighborhood listservs (over 2200 households, each) are
86
The Community Emergency Response Team (CERT) Program is a FEMA-sponsored program which trains volunteers in
basic disaster response skills, including fire safety, light search and rescue, team organization, and disaster medical
operations. Michigan as a state has one of the highest rates of participation in the program.
65
maintained by Allen Neighborhood Center ("Active Neighboring News") and South Lansing
Community Development Association ("South Lansing Update".) The Regional Community
Resilience Coalition would assure that the Regional Emergency Plan, which includes all
communities served by the BWL, has an emergency communications plan/listserv which
includes all of the neighborhood associations and community groups throughout the BWL
customer region.
Energy Self-reliance and Islanding
A related concept of resilience was referenced in the Introduction. The economic strategic
importance to the Lansing region of BWL is self-reliant for power, that is, if it has the capability
to generate all the electrical power its customers may require (assuming sufficient supply of
fuel). This concept of temporary isolation, or islanding is the opposite of contemporary
network theory, which holds that the more inter-connected that BWL is to the Grid, the more
opportunities to assure the availability of purchased power, should a portion of the grid be
blocked. The seminal text on networked critical infrastructure, in its analysis of the national
electrical power grid, observes that the greatest vulnerabilities in the network exist in the
middle, that is, in the transmission and distribution layer of the power grid. Faults occur and
propagate throughout major portions of the country, because of critical links, insufficient
distribution capacity, and cascade failures.87
Certainly the BWL must maintain the ability to spot-purchase power, in its mission to afford its
customers the most affordable power possible. But, the capacity for islanding, to separate the
BWL customer area from the rest of the grid, should protect the customer area from the
effects of percolation on networks.88 Islanding is being explored by the Department of Defense
for certain military bases roughly the same size as the Lansing area. The strategy is that, rather
than try to prevent the worst-case contingency through rigid physical and IT security, we
deliberately plan for its occurrence, by re-architecting the transmission and distribution system
to those bases. Islanding then protects one section of the grid from contaminating other
sections. BWL operations would need the capability to re-configure the system to shut off
access to outside power, when certain threats, such as viruses, geomagnetic disturbances or
grid overload are apparent.
In addition to increasing BWL resilience, this approach may mitigate future reductions in
electric power generation due to the ongoing shutdown of coal fired prime power plants and
reduction of nuclear generation capacity. Consideration may also be needed to ensure that
existing and future control systems are capable of and programmed to rapidly island the BWL
87
Critical Infrastructure Protection in Homeland Security: Defending a Networked Nation, Lewis, Ted G.,ed, John Wiley and
Sons 2006/
88
service area from surrounding transmission connectivity89. These system alterations can be
achieved by including internal islanding in future distribution repair, upgrades, and
modernization planning.
The CRT recommends that the City, BWL, and other area local governments:
1)
b.
c.
d.
2)
3)
Explore the potential for islanding to protect the local electrical grid, with BWL
taking the lead in creating an innovative, strategic solution.
Governance Issues
While this section and its recommendations may seem, at first, outside the mission of the CRT,
it was specifically tasked to consider issues in mitigation. Mitigation are the actions and
changes which are implemented now before the next disaster to lessen its impact and
improve restoration times. The CRT members have all held appointed or elected office at
senior management or leadership and/or held positions on boards entrusted with strategic and
financial decision-making authority. The CRT believes that mitigation requires consideration
and re-alignment of the roles and relationships of the Board and the BWL and within the BWL
senior leadership. The BWL needs to regain the confidence and trust of its customers.
89
Governance, accountability and communication with its customers are critical components of
building that trust and confidence in the BWL operations.
Board of Commissioners Oversight
The Board of Commissioners is created by City Charter. The Board of Commissioners only has
3 employees (the General Manager, Corporate Secretary and an internal auditor90). The Board
needs to effect control over the BWL strategy and policies and assure meaningful review of its
actions. The Board has authority and responsibility to conduct annual audits. The Board
should also institute performance audits of the BWL. Performance audits as commonly used in
state government are designed to determine whether an agency is achieving the objectives
established by the Legislature and whether it is managing its resources in an effective,
economical, and efficient manner. These audits focus on programs and issues that are of
particular interest to legislators and the public and make recommendations to improve agency
operations. Program evaluations assess a program's effectiveness and evaluate whether desired
outcomes are being achieved. The concept is easily adaptable to the Boards needs.
Performance or operational audits are needed to provide assurance to the City and its
customers that the BWL is well positioned for the future.
In many instances, it appears that the BWL Board agenda is driven by the BWL senior
leadership, rather than the Board. The timing and content of the BWL Storm Outage Response
seem to reflect that. Monthly updates at discretion of BWL Leadership are not enough. In the
corporate world, annual meetings are held which not only provide updates to stakeholders on
corporate activities, but provide a platform for debate on annual objectives, current and
emerging issues, and strategic direction. A board-driven annual meeting, open to all BWL
customers, might challenge some of the insularity of the organization's leadership, and provide
a forum for discussion of important issues and approaches.
The CRT observes that the Board might benefit from training on, and utilization of the Carver
Policy Governance Model91 or a similar system of organizational governance. The BWL Board
of Commissioners could seek the expertise of a trained facilitator of the Model to integrate the
model into its practices. Implementing these principles would allow the Board of
90
91
The Carver Policy Governance Model is a system of organizational governance. The ten principles of the Model are
designed to clarify interactions between the governing board and management within an organization. JOHN CARVER, The
Policy Governance Model, The Authoritative Website for the Carver Policy Governance Model,
http://www.carvergovernance.com/mo http://www.carvergovernance.com/model.htm del.htm. The Policy Governance Model
empowers a governing board to focus on large issues, to delegate with clarity, to control management without meddling,
and to rigorously evaluate the accomplishments of the organization.
68
Commissioners to take on a more proactive role without overstepping its purpose. Using the
Model would allow the Board of Commissioners to effectively govern the correction of the
issues that became apparent throughout the preparation, response and recovery from the
December 2013 ice storm. Using this Model would create clear responsibilities and
accountability between management and the Board of Commissioners, including clear
expectations for the organization as a whole and ensure that all responsibilities are assigned,
eliminating double-delegation or failure to delegate. The Model would require the Board of
Commissioners to specifically identify the level of reporting the Board of Commissioners
expects from BWL management, how management will be evaluated, how frequently
management should report.
The CRT observed that communication by the Board with local governments that have
franchises or agreements with BWL, does not appear to be a structured, ongoing organized
dialogue. The BWL should immediately create the means for priority contact with the elected
and appointed officials within the jurisdictions that they serve. BWL Rules of Procedure, Rule
9.4, permit the Board Chair to appoint Standing Committees and Ad Hoc Committees. Note
also that Sec. 5-102.11 of the Lansing City Charter states: An ordinance creating a board,
commission and committee may set forth a different size for the body or a different length of
term for the members than required in this section if the Council finds that the change is
appropriate.
The BWL Board has the authority to appoint a standing committee that would be a liaison with
local communities that hold franchises for services from the BWL. We strongly believe that the
Board Chair should appoint a standing committee to be a Liaison Committee to Local Units of
Government who hold franchises or agreements for services from the BWL. Each standing
committee is required to have 4 members, so each of these 4 committee members should
liaison with one or more of the 8 units of government that have franchises.
Recommendation:
The CRT recommends the Board establish a local government franchise liaison standing
committee and request that the respective local governments identify and assign a specific
person to be the liaison from that community with the Board. The scope of this liaison
function should be broader than just emergency management; it should encompass all matters
relating to services and relationships between the communities and BWL.
The BWL should immediately develop by-laws for the local government liaison committee
which assure: priority contact with the elected and appointed officials within the jurisdictions;
that its structure is revisited on a regular schedule to identify any concerns with the
effectiveness of communications at this level; that the notification and communications
mechanism to the local officials for any such future events receive the highest priority, with
ample follow up to assure communication, as well as assessment of response times to restore
service; and that the Operations Management System is discussed with the members and with
69
officials of the jurisdictions served by the BWL, so there is familiarity with how the BWL uses
that system and what the expectations are for its effective use in future events.
The CRT would observe that there has been substantial discussion, in the city councils for
Lansing and East Lansing, the townships, public hearings and elsewhere, that the City of
Lansing should expand the Board of Commissioners to include membership from outside the
City. It is not clear, however, if the authority currently exists to, by ordinance, expand
membership. And it is not certain whether the City Council could expand membership to
include individuals who do not reside in the City of Lansing. It would appear that a potential
amendment of the City Charter would provide some method of Board representation from
outside the City of Lansing.
The CRT does support the concept that there should be regional representation on the BWL
Board. We also observe, however, that the BWL is an asset wholly owned by the City of
Lansing. The issue is not simply a matter of the numbers of BWL customers that reside outside
the BWL service area. Rather it is the recognition of the vital and integral role that locallyproduced and distributed electricity and water services play in the regions security and
economy which compels the conclusion that there must be a mechanism for robust regional
input.
Findings:
1)
The BWL Board, although appointed in a manner similar to other City boards, is
not an advisory board. Rather, the City Charter granted the Board full and
exclusive management over essential services. The Board needs to assert greater
control over the short-term agenda, annual objectives, current and emerging
issues, and strategic direction of the BWL.
2)
The Board should take immediate, affirmative steps to effect this change,
beginning with training in Carver or a similar management tool.
3)
The Board lacks the mechanism to assure that they engage in structured, ongoing
and organized dialogue with the other municipal governments which represent
its customer base. This dialogue is vital, both for the economic health and the
security of the region.
4)
70
The CRT recommends that the Board of the BWL and the City:
1)
2)
Establish a standing committee for review of, and contract with outside expertise
for, an annual operational audit.
3)
Institute a training process for all board members in Carver or other Policy
Governance Model. Implement and use the model and continue the training on
an ongoing basis.
4)
5)
6)
7)
Urge the involved governments, the City of Lansing and the City of East Lansing
and all townships with residents within the BWL customer area to meet and
discuss the concept of representation on the BWL Board. These discussions must
focus on the need for regionally developed and implemented plans for emergency
response and for resiliency.
8)
Create and drive the system for implementation of the Internal Report and the
CRT Report.
9)
Recommend to the City a "Best Practice" for recruiting new board members. Not
only those that may represent certain areas that they serve, but recruit to needs of
expertise, including, as examples only, an engineering background, business
background or security background.
Recommendations from the MPSC Staff report
The MPSC staff report notes various rules and regulations which set standards for utilities
which are regulated by the MPSC. Those rules generally are used to ensure utilities maintain
their operating and distribution systems in such a manner as to be prepared for events like the
December 2013 storm. Examples cited in the MPSC staff report include:
71
1)
Setting a vegetation management schedule and budget & complying with it (pgs.
23-26 of MPSC staff report)
2)
Customer call answer time rule 460.724 (pg 29 of MSPC staff report)
3)
Community outreach activities (no rule but see comparison pg 32 of MPSC staff
report)
4)
Blockage time of customer ability to report outage (reference to rule at pgs. 3233 of MPSC staff report)
5)
6)
Time limits for relief of non-utility personnel guarding downed power lines
7)
(Rule 460.723, pg 37 of MPSC staff report) sets time limits for non-utility
employees (i.e. police /fire personnel) guarding a downed wire. & adequate
number of persons certified to perform wire down duty during high volume event
(pg 38 of MPSC staff report).
heard citizens complain that the Agenda has 5 minutes for Public Comment, but no real system
for public engagement. While the BWL has a Director of Governmental Affairs and Community
Relations, that portfolio should be expanded. Other publicly-owned utilities have a senior-level
position for customer relations, such as a customer ombudsman or Chief Customer Officer.92
This position should also directly report to the General Manager.
Implementation of Outage Report. Too many of the recommendations within the report are
aspirational goals rather than concrete recommendations93. The BWL should refine the 54
recommendations contained in their report into what would be a more actionable plan by
reframing their recommendations into to SMART Goals which are: Specific, Measurable,
Attainable/Actionable, Realistic/Relevant, and Timely
Once goals are established, it is important to:
i. Develop actionable recommendations to meet each goal
ii. Identify financing mechanisms
iii.
BWL should be accountable to their Board and customers by reporting at least quarterly
progress to meeting these goals until such time as they have all been met. But if the goals are
too loosely framed, then accomplishment of the goals can be declared by BWL, without real
and needed change being effected. While the CRT has received the BWL Updates, it remains
concerned that, because of the general language used in many of the original
recommendations, it is facile to denote the goals as completed. The recommendations should
provide for the time line under which it is expected that this will be achieved. If there are
changes to the goals and the time line for their implementation they should be explained and
the goal and time line revised as needed. Items 10-14 in the first BWL staff update on Ice
92
JEA Jacksonville, Floridas municipally owned electric, water and wastewater utility, for example, created a position with
oversight over all interactions between the utility and its customers, to assure customer loyalty by ensuring JEA customers
have a positive experience through each of the millions of interactions that occur annually.
https://www.jea.com/About_Us/Board_and_Management/Senior_Leadership_Team.aspx
93
Storm Outage Report refer to steps they are taking to address the bottleneck that occurred
when they lacked trained spotters and damage assessment personnel. But these still lack
sufficient specificity. The customer will still ask: What are they going to do and when it will be
done94? Until the customer believes that these questions have been answered, it will not have
regained trust and confidence in the BWL.
The CRT recommends that the BWL:
1)
2)
3)
4)
5)
6)
94
Those include the following Response items contained in its Ice Storm Outage Report: Items 2,3,4,5, 7, 8,
9,10,11,12,13,14,20,21,32,35,36,37,38,39,40,43,44,45,52 and 53. (See 02/18/14 Committee of Whole Board
meeting.)
74
Closing Remarks
The CRT has worked diligently to assure that we have conducted a thorough and fair review of
the facts. We have attempted to at least note and address every issue raised by the members of
our community. Our mission was to be independent, objective and transparent as
we listened to community concerns, interviewed the BWL senior officials, BWL operations
staff, City EM and other staff, discussed the issues with subject matter experts and amongst
ourselves, and reviewed plans and documents.
Our goal was to assess the facts and then to develop and recommend best practices for the City,
the region and BWL. We do not reach conclusions on individual performances, although some
urged us to do so. Quite simply, that is the role of the City, Board, and other government
officials.
We recognize again the hard and diligent work of the BWL line crews and other BWL workers
who worked to restore power and assure the safety of our community. We have offered the
recommendations contained in this report to help assure that the BWL is better prepared to
more rapidly respond to future power outages and to urge future investments that will result in
a more resilient electric infrastructure for our region.
The December outage revealed the fact that the Lansing BWL, our innovative, well-regarded,
and award-winning utility, had a serious blind spot. That blind spot was the failure of BWL
leadership to grasp the threat posed by increasingly powerful and frequent weather events, its
role in the security and well-being of the community and to take appropriate steps to mitigate
potentially disastrous impacts. The BWL's lack of comprehensive preparation was mirrored
throughout the region, with a patchwork of emergency plans and uncoordinated responses in
many jurisdictions served by BWL.
In presenting our findings and recommendations, the CRT hopes that we will all learn from the
December storms, and that local leaders, including and especially those responsible for our
critical infrastructure, will together move resolutely to do what is necessary to ensure that the
next major threat is met with a well-planned and practiced, coordinated, and effective
response."
75
76
Appendix A
Community Review Team Members and Staff
The Community Review Team consists of the following volunteers listed by location within the
BWL service territory:
City of East Lansing
Michael McDaniel: Cooley Law School professor, former homeland security advisor for the
state of Michigan and former Deputy Assistant Secretary of Defense for Homeland Defense,
retired Michigan National Guard Brigadier General, CRT chair;
Beverly Baten: Former member of the East Lansing City Council from 1997 to 2008; and
Douglas Jester: Former East Lansing Mayor and City Council member. Jester currently
serves a principal with 5 Lakes Energy LLC, a policy consulting firm focused on clean and
alternative energy.
Meridian Township
Jerry Richards: Former Chief Executive Officer of Meridian Township. Richards currently
works for the Mannik & Smith Group, an engineering and environmental services firm with
offices in Lansing.
Delta Township
Darnell Earley: Emergency Manager, City of Flint; former city manager for the cities of
Saginaw and Flint; and
Bill Long: Former Chairperson of the Michigan Public Service Commission and former
Director of the Michigan Department of Labor.
City of Lansing
Larry Bass: President and CEO of Friedland Industries, Inc., a Lansing-based waste
management and recycling company; Chair Emeritus, Sparrow Health Systems;
Patricia Spitzley: Deputy Redevelopment Manager for RACER Trust, a national organization
specializing in property rehabilitation and outreach in automotive industry communities in 14
states; and
Joan Nelson: Director of the Allen Neighborhood Center on Lansing's East Side and 40-year
resident of Lansing.
CRT Chair McDaniel also appointed the following all-volunteer executive staff:
77
Mark Burton: An attorney with over a decade of experience managing advocacy and
legislative projects in Michigan, serves as the chief of McDaniels executive staff;
TJ Bucholz: Former communications executive for Governors John Engler and Jennifer
Granholm and current senior director of public affairs for Lambert, Edwards & Associates a
statewide public relations firm, advises CRT chair McDaniel on communications best practices
and serves as chief spokesperson for the CRT;
Jeff Pillon: Emergency management coordinator for Energy Sector in the Department of
Energy, Labor and Economic Growth during the 2003 blackout, advises McDaniel on utility
and emergency response and recovery and mitigation of future vulnerabilities;
Rhonda Grant: President and Chief Executive Officer of the Grant Consulting Group, a
government relations firm with locations in Lansing and Washington, D.C., serves as legislative
advisor.
The following students, alumni and staff from the Thomas M. Cooley Law School provided
invaluable assistance in the research and drafting of the two Requests for Answers and
Documents, the review of the thousands of pages of received documents, and in researching
this report:
Heather Spielmaker
Rebecca Stephen
Katherine Mortensen
James Vitrano
Kathleen Lawrence
Tara Chambers
The Chair would especially commend the assistance provided by Rebecca Stephens and Kate
Mortensen who also volunteered their time in the drafting of the report over the last two weeks
of April 2014.
78
Appendix B
CRT Mission Statement
Community Review Team
Brigadier General (ret) Michael C. H. McDaniel, Chair
On December 21, 2013 a catastrophic ice storm struck the Lansing MI region, causing more
than 34,000 electric customers of the Lansing Board of Water and Light (BWL) to experience a
long-term power outage. The BWL is a municipal utility, owned by the people of the City of
Lansing, that provides electric utility services to residential, commercial and industrial
customers in Lansing, East Lansing, and the townships of Delhi, Delta and Meridian.
The mission of the Community Review Team (CRT) is to conduct an independent, objective
and transparent review of the BWL's planning, preparation, response and recovery prior to,
during and after this devastating storm event. Our goal will be to evaluate the strengths and
weaknesses of BWL's performance in order to enhance the BWL's capacity to respond more
effectively to their customer's needs and to recover more quickly during future storm events.
Under the Lansing City Charter, the BWL is governed, administered and operated as a
separate, independent agency, not as part of Lansing's city government. However, Article
2 Section 5201 of the Charter provides that the BWL "shall be responsible to the Mayor and
the City Council for the provision of these services in a manner consistent with the best
practices." Our work, therefore, will be directed toward developing a series of findings and
specific recommendations for ensuring that the BWL meets or exceeds utility industry best
practices in the planning, preparation, response and recovery from any future large scale
power outages.
Consistent with this mandate, and in accord with our charge from Mayor Bernero, the CRT will
include citizen representatives from across the BWL service territory, as well as subject matter
experts in key disciplines. We will deliver our written report to the Mayor and the City Council
within ninety days and will share our findings and recommendations with the city's regional
partners within the BWL service territory and the public at large.
Our final report will also be submitted to the Michigan Public Service Commission (MPSC) for
their review.
79
Appendix C
Community Concerns
What Have We Learned About East Lansing's Situation Since the BWL Outage? A
Report to East Lansing's City Council and Citizens. Dreger, Alice, PhD. Rep. N.p.,
22 Jan. 2014. Web.
BWL did not have a plan for the ice storm or other emergencies
Establish warming stations much more quickly
BWL Lacks emergency and disaster planning
East Lansing lacks local advocacy/leadership
Communication to public/within BWL a problem
Citizen asks, "How did BWL get themselves in this situation in the first place?"
Better preparation needed- storm was predicted
East Lansing has no representation on BWL board
Did not appear to be enough teams working
Failed and deferred line maintenance, obsolete equipment played a large role
Concern for pets of those who need to leave homes due to outage
Encounters with customer service/linemen- willing to do anything they can.
Management should take same attitude
All BWL and local government personnel should know what their emergency roles
are, where to go
Do not want "hometown power co."- we want cutting edge, fastest, best power
company
Implement disaster preparedness and recovery plan
Neighborhood with buried lines didn't lose power
Recommends better critical infrastructure planning in communities
Keep climate change in mind- storms like this will keep happening
Utilize neighborhood associations/nonprofit community/neighbor-to-neighbor
communication
BWL should use "Smart Meters"
Systems such as "Turn on your porch light to show you have power" should be
implemented before a disaster
Creation of microgrids that help isolate emergency services from the rest of the grid
80
Need outage map system, possibly with a way for customers to update
Consider environmental impact of any changes made in response to ice storm
Implement medical priority list
Live downed wire in neighborhood, no BWL or 911 response
Implement emergency drills and communication plan
BWL representative did not show up to police chief's emergency operations center as
planned
Establish system of repair priorities, backup set of procedures for use of private
contractors
EL produced own outage maps because BWL was losing outage reports
Use voter registration list or Meals-on-Wheels list to keep senior/low income
registry
Public safety threatened in many ways during outages
Customer service believed resident had power when they didn't
Once the trucks arrived, it took them only 45 minutes to restore power
Concerns about citizens on oxygen 24/7
Safety issues during outage- food poisoning, fires, downed wires, etc.
Citizen showed BWL worker around neighborhood to find outages
Went to Hannah Center "information center" during outage, took 2.5 hours to get
information
BWL needs "Boots on the ground" to communicate
No transparency in BWL business model
Use public libraries as community centers, make sure they have power first
Welfare checks occurred too late- several days into outage
Be prepared for situation where website and Facebook, cell phone data connections
don't work
Concerns about 100+ year old houses in East Lansing's historic neighborhoods
Need registry and plan in place for disabled/mentally disabled Welfare checks
need to start right away
81
Citizen without Comcast (phone and internet) as well- no way to communicate with
BWL
Hotel in Delta Township had heat but no lights, Fire chief forced guests to evacuate.
At that time, people had to travel as far as Coldwater for lodging.
Citizen told mast repair was "his problem" by BWL workers
Customer told by BWL workers "We are servicing large groups of homes first, you
will be last"
Tree trimming occurred in past few years but apparently did not work
Need point-person for local units of government
Citizens not notified about masts
Received phone call on Jan. 2 stating that water rates would increase. Insulting that
BWL can't respond to customers needs but can find phone number to notify of rate
increase.
Citizens Told by BWL customer service they "weren't a priority"
Need BWL cell phone app which works on phone data plan to report outages, etc.
Private electricians are charged a fine if they interfere with BWL equipment,
therefore they may not be able to restore electricity to a home
Create list of local electric contractors to deal with masts quickly
Lansing State Journal and Councilwoman Carol Wood were most reliable source of
information, not BWL
Citizen was marked as having power when power had not been restored
Need to look at restoration priority schedule
BWL online information is too hard to locate
BWL customer service was closed on Saturday during this crisis
Citizen made homeowner's insurance claim for mast- taken care of within 12 hours,
even with the weather
Concerns about pets during outage when kennels are full, pet-friendly hotels full
Communication inconsistencies- from "don't call us" to "report your outages"
Citizen called WILX and Carol Wood and power was taken care of immediately.
BWL should be providing safety information about burst pipes, carbon monoxide,
food safety on their website
Need a priority restoration list, for example, stoplights on Saginaw Highway should
be a top priority
Citizen had 14 calls ignored or disconnected
Citizen encountered "mailbox full" message when calling BWL
82
83
85
86
Communicate about which streets/areas have power so people can find hotels,
coffee shops
Collecting $25 reimbursement very cumbersome
Suggestion that Communities join together and communicate, organize.
No response to e-mails to BWL
Reimburse citizens for losses/expenses
Have to keep driving by their house to check if power is restored
87
Location
East Lansing
Lansing
Lansing
Lansing
Lansing
East Lansing
Unknown
Unknown
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
88
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
Lansing
89
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
Unknown
Unknown
Unknown
90
Unknown
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
91
East Lansing
East Lansing
East Lansing
East Lansing
Equipment upgrades
Have never experienced such a long power
outage (age 60 and 70)
East Lansing
East Lansing
East Lansing
East Lansing
East Lansing
Unknown
East Lansing
East Lansing
92
East Lansing
93
94
95
96
Appendix D
Materials Acquired from BWL
2nd Request Answers and Attachments to CRT. 7 March 2014. 1582 pages.
Best Practices for Storm Response. Response to 1st CRT Request. 13 Feb. 2014.
BWL Board January 2014 Resolution requiring public forums. 11 March 2014. 2
pages.
BWL Board March Committee of the Whole Agenda. 11 March 2014. 2 pages
BWL Crisis Communication Plan dated March 2014. 11 March 2014. 12 pages.
BWL Dec 2013 Ice Storm Outage Report, Appendix containing transcripts of the
January 15, 16, and 17 2014 public meetings, and Notes Released 2-18-14.Web. 19
Feb. 2014. 333 pages. (LBWL Website)
Bylaws and Governing Structure. 2nd Request Answers and Attachments to CRT.
7 March 2014.
Consumers Energy Mutual Aid Agreement. Response to 1st CRT Request. 13 Feb.
2014.
Draft Interim Crisis Communications Plan. Response to 1st CRT Request. 13 Feb.
2014.
Emails from and between key staff. 2nd Request Answers and Attachments to
CRT. 7 March 2014. 3505 pages.
Government contact names and email addresses. Response to 1st CRT Request.
13 Feb. 2014.
Graphs of BWL Phone System Usage. Response to 1st CRT Request. 13 Feb. 2014.
Invoices for outsourced line crews. Response to 1st CRT Request. 13 Feb. 2014.
Kent Power Emergency Storm Assistance Contract. Response to 1st CRT Request.
13 Feb. 2014.
Lansing Board of Water & Light Emergency Action Plan.Web. (dated 2014
effective 12/1/13)(LBWL Website) 6 Feb. 2014. 42 pages.
List of circuits down, restoration dates, and customers affected. Response to 1st
CRT Request. 13 Feb. 2014.
List of downed wire reports. Response to 1st CRT Request. 13 Feb. 2014.
List of FEMA Mutual Aid Agreement Participants. Response to 1st CRT Request.
13 Feb. 2014.
List of media contacts and email addresses. Response to 1st CRT Request. 13 Feb.
2014.
List of Mutual Aid Contracts. Response to 1st CRT Request. 13 Feb. 2014.
MMEA Mutual Aid Agreement Response to 1st CRT Request. 13 Feb. 2014.
Rules and Regulations for steam, chilled water, and water service. 2nd Request
Answers and Attachments to CRT. 7 March 2014.
Rules of Procedure for BWL. 2nd Request Answers and Attachments to CRT. 7
March 2014.
Safety Manual. 2nd Request Answers and Attachments to CRT. 7 March 2014.
Section of City Charter. 2nd Request Answers and Attachments to CRT. 7 March
2014.
Steve Serkaian Press Releases. Response to 1st CRT Request. 13 Feb. 2014.
Timesheets for line crews. Response to 1st CRT Request. 13 Feb. 2014.
99
Appendix E
Transcript of March 10, 2014 Public Hearing
is what I did. Unfortunately, our HQ was out of power at that time and for
most of the first day. That group (Sue, Lynn, or Dave.) would have gotten
together that first day. I remember the first thing I did after doing a brief
assessment with that group was to contact the Mayor, I went to his house,
we examined what was going on and concluded, more his decision than
mine, that a press conference was in order. Before the press conference,
met with those 3 and SXS. Cannot remember who else was consulted. Very
in depth talks about what was going on, had a press conference at 1 or 3 pm,
met with the same people described at the press conference. At that point,
was of the view the best people on the ground were making decisions and
was leaning toward going to NY but wasnt sure or not, and Mon 23 went to
NY. Indicated to all that was a mistake. In retrospect that was a mistake.
Do over, wouldnt do it. Do not think the restoration was compromised in
any way by my absence. The group knew he was in NY and everyone had
cell phones. Worked with Ms. Devon for 11 years, she knows when to get
Peter and when not to. Worked for 7 years at BWL to make sure that if had
a heart attack that the place would run well without him. Complete
confidence in Sue to make decisions in his absence, even though she knows
when to call Peter in his absence. While in NY, got emails CCed on, texts,
the like. In constant communication, out to dinner, eating dinner and not
in communication, but cell phone was operable. Throughout time in NYC,
best possible personnel on the ground, they would say absence would not
affect one wit the restoration, but if I did it again I wouldnt do that. When
I left, the team said 300 lines down, 500 lines down, turns down we had
2400 lines down which we just couldnt have known at that point. Given
2400 pages of information. This should detail our problems with the OMS.
While I was aware the OMS was not fully in place or up to snuff, if I was
aware we had conductor cut problems, I wouldnt have gone, but at the time
no knowledge of conductor cut problems.
McDaniel: Do you recall what updates you did receive in NY?
L: No.
M: Did you decide to cut the vacation short?
L: Yes, was going to take 1 extra day. Monitoring the situation pretty
closely. Received 50 emails or texts. Things seemed to be proceeding along
the way hoped until Christmas Day. On Xmas, Lark woke up and decided it
would be better to be in Lansing than in NY, so had wife get him a flight
that left 10:50 am, so Peter was back in MI on Xmas day. Directly went into
the field to continue to working at the BWL until 9:30 pm and stopped at
Subway and got a ham and cheese sub.
101
102
103
104
106
Beverly Baten: Conversation with head of 911, had a dropped call from 911.
If you exchange info with BWL, all the calls coming in would have helped
911 out calling for emergency, and 911 said there is no exchange with the
BWL. Is that a possibility?
Lark: I dont know that is the case though it sounds like it. Yes, we are
overhauling our entire system so we can talk to everyone.
Lynn: We get wire down reports and constant calls from 911 in a formerly
unlisted number.
Q: New in EL to have to call. Good idea to share
Bill: Response says will consolidate the plans. When will that be?
Dave: 6 months is the target.
Bill: In the meantime?
Dave: The documents is still there, but for convenience rather than have
separate ones to pull, put them all in one so it is easier to use. Its not that
they arent there.
McDaniel: Including the new Communications plan?
Dave: Yes. Were looking at an Emergency Response Plan as Sue indicated.
Larger utilities have been asked to do that. Were building the backbone of
that right now. Looked at a couple on the internet and see how that can
work for us so we can build something very similar on a wide scale. Will
take time to do because it involves everyone.
McDaniel: That isnt uncommon. Every state agency builds their section
and then it is put together into a common approach. Not an uncommon
approach.
Nick: When the plan is sent out, Sue makes sure you follow it through with
your guys.
McDaniel: How often do we do a full-scale, BWL exercise of some sort,
whether it is electrical outage or some other emergency event, meaning
anything from tornado to active shooters. How often is a BWL wide, not
just department, event?
Sue: I dont think we have done that.
Lark: We have not done that, although we would have done that in other
areas such as water. The electric area not so much. Not saying this is
sufficient, having had this event, we will be doing exactly what your
question suggests. But I cant emphasize enough that in a way weve done
them, but not as tests. We had 3,000 lines down in November due to a
wind storm. Those are lessons learned. In 2008 we had that tornado that
took our a cooling tower in the Moores park area. There were sustained
107
outages that involved the GM all the way down the the bottom. Thats not
to suggest we shouldnt do them or arent remiss in not doing it, but we
dont have a comfort level with that level of storm. Its not as if we hadnt
been through those, and thats a good point. We will be doing a sort of
table top analysis in the future with respect to electrical. It is one of our
recommendations.
Dave: For our distribution plan. We can easily incorporate what Peters
suggesting as well.
McDaniel: Which will become part of the ERP I hope.
Dave: Yes, but how it fits in I cant say at this time.
McDaniel: Struggling with the definition of a catastrophic event. At least
two different instances the MPSC refers to a catastrophic event being a 10%
outage. Reference to 2003 blackout, separate different type, having 25%
outage in 2003. Want to make sure were using the same definitions.
When were talking about that catastrophic event, what is the trigger %?
Lynn: Where we reference the MPSC & catastrophic event, that is more for
reliability reporting. Thats where that 10% comes from.
M: Is there a % outage considered catastrophic?
Lark: Its [D & Lynn] area, but susceptible to a number of different
definitions. After this event, we had an outage of 7200 people in Lansing,
and if it had been 10k it would have been catastrophic per MPSC. But its
not catastrophic to us it was just a circuit failure and when the circuit is
restored everyone would come up. In February, we had 2300 event that
was far more catastrophic in my mind. In 2003, it didnt affect BWL badly
but it took out all 2m Consumers customers. There was nothing they could
do. All outages are important to us and we will rush to try to fix it.
M: So what is catastrophic?
Dave: In 2003, we lost a whole system but were out in 21 hours. Its not the
customers out, it was the sustained outage. The loss of major equipment
that you cant replace. Loss of transformers, etc. The loss of those facilities
would be catastrophic. Those lines on a distribution system were
unfortunate but we could get them up in a week, and not as catastrophic.
Jester: So its the expected duration of the outage. Your folks have to deal
with it, but the rest of the community also needs to respond. Theres a hole
in how the plans are triggered and the effects on the community. What
does the BWL think and how it related to interaction with the community?
McDaniel: If it wasnt a catastrophic event, the lack of the trigger to EOC,
there was no lack of communication ability.
108
OMS. The OMS is not meant to ever tell you automatically when a wire is
down, but we have eyes in the field on everything.
Beverly: I was out of power for 9 days, had power restored Dec 31. One side
of street had power, the other did not. The city knew that power was out
and left a notice, but the customers themselves werent told. Whatever they
were doing for emergency wasnt filtering very quickly. By the time they
came on the 30th, was kind of late, but interesting that they knew where the
outage was.
Lynn: We were trying to share information on where pocketed outages
were. We had spotters in the field and as we found the pockets we were
tracking those. And were passing info as much as we could.
Beverly: When we came to check the house, drove around looking for BWL
trucks, but had a hard time finding them. With 12 houses, I understand you
had bigger outages elsewhere.
McDaniel: Hold for later until after EMS and order of restoration first.
McDaniel: Control center itself. Also BESOC?
Lynn: Yes.
McDaniel: I have a sense from operations side that you dont need within
BWL an EOC because the BESOC is 24/7?
Dave: Yes, that is our emergency operations.
McD: The day to day is also your emergency?
Dave: It is.
McD: There are some ways that would make sense, there is the big board,
the large scale computer system EMS, can you explain the difference b/w
what is tracked in EMS v OMS?
Lynn: The EMS is actually tracking and monitoring real time system
operations, system voltages, etc. The OMS is purely a way to track outages
in a more organized fashion have a handle on the outages. What areas are
out, how many, etc. Tool used to manage operations.
McD: How to harmonize the two systems? Lines down fed into the OMS,
how to compare against EMS?
Lynn: Hopefully soon the OMS and EMS will talk to each other, so if
something in the EMS happens, a breaker opens, then it goes into the OMS
even before calls start coming in. The OMS groups calls that come in and
predict what is out. It will predict where outages are. Helps track quicker
where critical customers are and where largest outages are.
Jester: Some outage systems integrate with forecast management. Do you?
111
Lynn: Yes, our OMS does have estimated restore time in it to estimate on
the field, we do have the ability to give restoration times. The default time
is based on past history. We can change it to be more realistic based on
response from guys in the field.
Jester: How accurate were those estimates?
Lynn: We did not have good estimates at all. It was not performing as it
should.
Jester: What was the problem?
Lynn: Not a problem with estimated modules themselves, but that the OMS
was abandoned as soon as the conductor cut issue was figured out.
Jester: How much less efficient did going to manual?
Lynn: We call it organized chaos. Didnt hurt response time. We have good
guys who can put info in. It may have been more confusing, more time
consuming, and needed more bodies, but didnt expand response time.
Lark: But at the very end, an operating OMS might have saved us some
time.
Dave: Marginal
Lark: The singletons or the onesies. We could have known there are 10,
and not only addressed 3 of them and come back later. Had no effect on
basic restoration, but that last bit it might have been.
Dave: When it comes down to the customer service restorations.
Lynn: I made the decision to abandon the OMS a day and a half or two days
into it. We had a work -around, we had been using the work -around. But
with the amount of wires we had down, we needed to find another way. At
the beginning, all the wire downs reported were going into the OMS, and
we were dispatching to all to get a feel. When we realized we couldnt keep
up with it and modeling wouldnt work, we gave crews circuit prints and
asked spotters to mark up the prints and those went into an excel
spreadsheet. We were able to determine pocket outages from that point.
Bill: Back to broader communication process. One of the responses was
that safety list would be updated and restoration plan will be shared with
officials. The restoration plan lists foster case and other community places.
What is BWL responsibilities when communicating with those community
centers and what is the local govts responsibility? From Delta Twp, so as
you look at the plans, how to identify these local officials to be
communicated to? Where does BWL responsibility end?
Lark: We have updated our lists so every local official we can find will get
material from us from now on.
112
Bill: Is it clear what their responsibility is? If those people arent there, who
is responsible?
Lark: We do have a list of those people who are medically infirm such that
they would need help if there was an outage. Thats what we are working
through. Whose responsibility is it to help them? Its our responsibility to
get that info to the local entity. But we must do updating because we ask
customers to tell us if they are in that position, mainly through customer
connections newsletter in the monthly mail, but not everyone reads. How
do we get to them? Once we have a list, we have to think through HPPA
concerns. How much medical information of a customer is private and
shouldnt go out? Its an example of how we want to reach out and get the
info to the cities, but we want to make it legally. We have to be aggressive
in getting the info from the customers, and need to get permission to then
pass the info along to the city. Working through that right now. There are
many other things like that.
Sue: We couldnt automatically give a medical list to any official because of
privacy concerns. The medical alert list is really to alert the BWL not to
shut off the power for nonpayment. Notice that they have life supporting
equipment. Do let the people know that this does not put them higher on
restoration and that they must go elsewhere and are responsible for
themselves. But we are very interested if local govt want to work with us
on that, but we need to make sure that the customers are in agreement.
Beverly: You have Medical Alert form and Senior Shutoff Protection in
connection. There is nothing about shutoff for medical. It sounds like
there are two groups.
Sue: There might be overlap. The Medical Alert is for them to inform us
that should you become into a situation where unable to pay the bill, need
to know if life is endangered if power is shut off. Do have senior citizen
protect plan, too. Different group, but there is crossover.
McDaniel: Are you a govt official, Lark?
Lark: I cant speak dispositively, but I would consider myself a govt official.
McD: So Sue, you are not using the medical alert list to prioritize response?
Sue: We inform them that it does not give them higher priority, but it is
part of the OMS system, so when it gets down to onesies and twosies, then
people are aware of the medical alerts.
McD: When did you get to onesies twosies?
Dave: End of the weekend, 28th or 29th.
113
Larry: HPPA. I would suggest to contact Sparrow and ask Sparrow to help.
No one with more familiarity than they do. Risk management, someone
will know rules and bologna and red tape. About notifying govt. Govt is
putting BWL between govt and customers. Realize that govt is a customer,
but 70-80% is not govt. In the era of social media and notifications and
cell phones and emails, I do work all over the world on my cell phone. How
is it that you cant notify customers directly?
SXS: The govt officials are being contacted by constituents.
Larry: There just seems to be a layer between govt and customers.
Darnel: I had asked about the relationship b/w the BWL and the
municipalities they serve. By own volition, did not reach out well enough.
Needs to be addressed.
Larry: Thats important, but more important to communicate with
customer base.
McDaniel: They are both important. Lets talk about customers first.
Joan: Communications w/ customers was confounded by a lack of OMS
functions. In the area of preparation, can we ask about OMS reference to
problems in May.
McD: Hold off on OMS for now.
Larry: Sparrow will probably be happy to help, they have vast knowledge.
Sue: We do have HPPA trained officials.
Larry: Healthcare usually deals with it more than others. Getting back
toseemsgovt has to be notified, counties, whoever has to be notified.
But in the day of social media and being able to do business sin NY, how
can we better communicate with constituency, customer base, not just
govt, directly.
Lark: In this storm, we didnt do a good job communicating w/ customers
b/c did not have info b/c of OMS. We did a heck of an effort in getting info
out on facebook, in media, twitter, etc. Would have been better had people
could have called and gotten a restoration time. People want to call and
talk. We cant talk when we have 300,000 calls in a day. But there has to
be some sort of voice recognition system that says Yes, we Know youre
out, gives info, and allows them to call back for status updates. These
things are being worked on and things are in place and have worked
successfully throughout the last four or five outages after the ice storm. We
have an outage map that will be debuted March 11 at community meeting
for board. Did not have the info then, but we did a lot of announcing to
people. Not enough to satisfy all customers. Have since fixed OMS and
been stress tested. March 9 we got final results on stress tests.
114
Larry: Not criticizing lack of info, more to do with not having govt in the
middle of it.
Doug: Doug & Jerry have been asking govt questions not to be most
informed, but because govt needs to know to prepare a response. When
you know, you prepare differently.
Larry: But the customer with a pacemaker should know before the mayor of
EL.
Lark: Thats were were going. They will be able to have that very shortly.
Sue: One of the biggest concerns was the lack of an outage map. They want
to look at it. Assuming that, we will debut tomorrow to our board a new
interactive outage mapping system up and running right now that will allow
customers to get in, see their area, and know whether they are an outage
area or not. The second phase requires more secure site, but will allow
customers to get in and look at their house. Shouldnt be available to just
anyone b/c security issues. Connected to OMS, will give restoration times,
will be default time of 24 hours and as soon as an outage, crew will go out
and estimate time, which will replace default. Customers will be able to
look and know.
Lark: What about phone customers who dont have a computer?
Sue: Customers call 877 number to report an outage. Everyone wanted to
talk to a human and that is what overwhelmed the system. We had 250k
the first day come through system, 112k through call center. Cannot staff
for 112k calls. The 877 number should take those calls, people can report
outages, but the call processor wasnt working, so wasnt going from 877 to
OMS system, which gave conflicting info if up or not. Alarms on both sides
so if either 877 number or OMS gets jammed, alarms on both sides. All call
center folks are now focused on answering VM, email, Facebook, anything
coming in on those areas and getting that into OMS. Market going to 877,
ordered magnets to stick by phones so people dont go to call center.
Lark: Since the storm, what info does the customer get when they call 877?
Sue: If they call in, in the early part of the outage, it will tell them to call
back and get a restoration time. They can call back and hear the default,
but eventually it will be restored.
Nick: On a system level, there will be a best-guess default for the whole
system. After feedback from crews is put in, they then get for their house.
Pat: For the 70% I agree, thats the #1 priority. But my parents are 83. If
he answers a text, I will turn purple. Those social media things. When you
talk about facebook, I was on facebook, I know what was there. Most
people dont want to talk to someone they just want to hear were working
on it, this is the status. We need to hear some information. Most people
115
know it was stressful. Wed prefer a recording to a busy signal or dead air
or stressed CSR. The communication can be on local govt so dad can call
mayors office. Social media is extremely important, but there must be a
mechanism in place to get info out to people w/o internet, w/o text
messaging. Try to implement policies to reach the majority of the folk.
Must be a coordinated effort. Local govt needs to be informed to get that
smaller percentage that cant.
Lark: Not relying on the one system. Our system should be up and running.
Weve had 4-5 hours where there has been good response. When they leave
facebook message, we can then reach out and confirm. It isnt just the
internet stuff, not just the phone stuff, some people also want to hear the
radio. The radio is something well use for protracted outages and any
other means we can come up with. Theres no one size fits all.
Bev: Restoration priorities listed, public safety is first priority, then group
of others. Is there a privilege list? Saw an op-ed at CityPulse? Govt calling
in and getting a response.
Lark: There is no privilege list. People are brought back based on
documentation we have. The idea might have come from the fact that at
some point it was clear that no one was getting good information from the
BWL. Carol emailed Lark a lot, saying: these people dont have power.
Incumbent on Lark when she or anyone called Lark to get that information
to the T&D people and tell them if you dont already have this address, here
it is, not and put it on today. Tried to keep track of those and tried to get
back. Everyone who emailed Lark got the same treatment. There was no
exception for the BWL Commissioners. What did the article say?
Bev: That a commissioner called in, shared emails with Lark about being
out on Dec 29th, and 45 minutes way said a spotter was on the way.
Lark: I would have passed on the address and then been told when
someone is going out. I try to get back to the commissioners, Carol Wood,
or any other human with info as quickly as I can.
Dave: There might have been someone in the area already.
McDaniel: General public safety dictate public officials have, look at order
of restoration priorities. Going back to BESOC, did BWL have direct
communication b/w BESOC and LEOC?
D&Lynn: Yes, 3 times a day.
Dave: We fit under their umbrella, we dont drive them.
McD: Were they sitting in the EOC?
Dave: Not everyday, but Christine was there.
116
Lynn: CSR was there. Assigned to operations during the storm to work as a
liaison b/w us and the EOC. Whenever they asked a Q about a specific area,
or whatever they want, sent her down with a computer who then could ask
BWL. That is what EOC requested. No ability to make any decisions.
McD: What info did you give Christine to ask the city on BWL behalf?
Lynn: None. We talked on conference calls and if any special had the
numbers directly to call.
McD: Did use the conduit for scope of remaining outages?
Lynn: Was discussed during conference calls. Rep was there at their
request. Number of wire downs, quit counting after a few days there were
so many, areas of restoration, what was restored, what would be overnight,
how many crews, next plans, restoration efforts.
McD: SXS had 3x a day media releases. Were these based on the same
info?
Lynn: SXS info was an hour before the EOC meetings. The internal BWL
meeting was an hour before EOC meetings. The next hour, have the same
info passed to EOC.
McD: Is BESOC scaleable? Is there surge capacity?
Dave: To a certain extent. Doubling or tripling the size?
McD: Yes, is there a B team?
Dave: Yes, they are 24/7.
McD: External stakeholder, not a BWL employee?
Dave: No, its a confidential area because access to all critical assets. NERC
requirement. They could use it to affect bulk power system.
McD: I mean from other communities, local govt officials, fire, police, and
emergency management?
Lynn: Yes, they could come in for a meeting. There wasnt a need for them
to come in. We would hold conference calls or go down to them.
Lark: At BESOC, there is a closed area that NERC requires people are not
admitted. There are other areas in BESOC that people can go. NERC
requirements prevent terrorists from taking out specific circuits. Everyone
there has to be NERC certified.
McD: No officials in BESOC?
Dave: Only one I saw was Mayor Virg.
McD: What would you do with that info if we had a new address?
Dave: Ask the operator if it was in, if they had a crew or spotter dispatched.
If yes, relay back. If not, report it.
Pat: But you abandoned the OMS? How then?
117
Dave: The issue is time. Going forward, were putting in the single ones.
Has a spotter crew or who been dispatched to the address.
Pat: Not clear when said OMS, how to address outside scope of OMS.
Lynn: When we abandoned the OMS, we kept everything on paper prints.
When we got calls, wed compare address to identified wires down or
outages, and could say whether a crew was on it or not.
Lark: In a perfect world, could have told everyone to call 877. But they
were getting incorrect information, so thats why Lark tried to pass along
the info he was given personally.
Bill: Had it functioned normally, could liaisoned with? But didnt. No
way to know whether communication was going to other units of govt other
than Lansing.
Lark: Assumed that once it went to Lansing it would move on. But now we
will have a liaison appointed to other units of govt.
Joan: Info is important, but more so what you do with it. What about other
EOCs? You assumed LEOC was spreading info. But there are County
EOCS, etc. What about them?
Dave: Were the activated?
Lynn: Only been involved with Lansing EOC, dont know about any others.
Its on them on their end, we dont know otherwise.
Lark: My number is not a secret, and most govt officials can figure out how
to get Lark.
Joan: Coordination in a regional event to have a regional EOC?
Lynn: It would make sense.
McD: What info was given on the 3x LEOC conference calls?
Lynn: Where crews, how many, weird issues, materials (we didnt), and
they would pass on what the city was doing per tree removals, etc. If we
had a wire down that they needed clear to proceed, wed discuss that.
Jerry: What was communicated to Lansing? Lansing wasnt interested in
everyone else. How would they pass along what they didnt know?
Lynn: Systemwide blanket still out, not Lansing specific. One day did talk
to Delta few days in a row asking about what was out in Delta toward the
end.
McD: How were they described?
Lynn: NE/SW boundaries of major streets, vague info that way, amount of
customers in that area.
McD: Any discussion of related issues w/ medical alert, 2 other issues
coordination of warning centers, health and safety checks. Was any of that
discussed?
118
Lynn: Some about locations of warning centers, Lynn didnt document, also
about health checks further into the outage asking streets still out and then
they were making determination on safety checks, probably started after
the 28th. Not immediate. That information was requested.
Pat: Mayor was in LEOC?
Dave: He was in BESOC.
Pat: How did he come to be in there?
Lark: Dont recall the day it was, might have been first day?
Dave: It was.
Lark: Sunday morning before the news conference. Picked him up at 6-7
am at his house. Had to drive Jeep on curb to get down there. Wanted to
get down there to check. Dont think Mayor had an agenda at that time. It
was the first day, just checking how things were going.
McD: Liaison at EOC was linking back how?
Lynn: Was calling BESOC, not Lynn directly. Clericals were talking to each
other. Sue was managing the spreadsheet so people would have something
to give back.
McD: Decision to ditch OMS & use Excel was on 2nd day. Go through how
to document.
Lynn: At the very beginning, call would give address, say wire
down/outage, and would compare to spreadsheet or circuit prints and see if
already identified. There was a stack of work to see if they were in there,
stack of next work orders could tell that. Whoever answered the phone
would have to look through a list of sheets to see what was going on.
McD: Commented that at some point discarded trying to ascertain the
number of lines down.
Lynn: When first started, the wire downs poured in. Every conference call,
it was more and more and eventually overwhelming. Spotters were
reporting more and more. Just a count wasnt giving great information,
were trying to focus on locked out circuits to try to lock down outages and
problem areas. Didnt count wires down b/c increasing so quickly. Lynn
made this decision very early on.
SXS: Last media update on 23rd at 4 that reflected 500 wires down. All
subsequent media updates from the 23rd on talk about hundreds of wires
down with a focus on safety. Palpable fear that thered be serious injury or
death from wires down.
McD: Do you have something that documents the decision?
Lynn: We were still tracking and investigating the wires down, but just not
reporting them because it was just so big.
119
McD: The only way you can track are the media advisories?
Lynn: Yes
McD: But then it was asked for the public to stop reporting down lines?
McD: Emergency?
Lark: Cant say how that went, to my knowledge, no emergency. May have
been the very first day. Was not asked for a recommendation. Did not
suggest to Mayor that an emergency should be called.
Darnell: Protocols that one might expect to be in place were not. Think
impt to look at service at 30,000 ft look and a ground level look for the
individuals with the personal needs. But clear that the area needs to
recommend to be looked at clearly going forward. Know what it means
when the residents try to find out and you are just as uninformed as they
are. The communication must go forward not just with Lansing but with
other municipalities. All must know what is going on to govern.
McD: Order of restoration. When last updated?
Dave: 2005.
McD: How far down the list did we go?
Dave: Priorities all the way. We went to mobilization level 1. Went to mob
1 & 2.
Dave: Saw this on the radar that Monday before the storm, saw on
projected ice storms. Monitored that through 19th morning generation
meeting, bring in operations, generating people to roll if it was to occur,
making sure we understood our roles with the November storm fresh on
our minds, though we understood how to restore the system. On the Friday
20th, work management meeting, discussed again, T&D person was
instructed to look at MAA and make sure contact info was up to date, that
spotter crew coordinators were informed and the spotter list was up to date.
Followed the storm through the 21st and that night Pat Hanes, T&D
manager called in MAA crews from 6 cities.
McD: Crew is 2?
Dave: Initial crews are 3 except for ESWs, some 2 person crews, some were
3.
McD: Seemed that primary lines had a 3 person crew that would either be
all BWL employees or 1 BWL & 2 outside crew.
Dave: One birddog with potential to serve crews of 3 person.
McD: 1 BWL supervisor/birddog supervising more than one MAA of 3people?
120
Dave: Yes
McD: How far apart were the crews geographically?
Dave: Not eyes on everyone, could be a couple streets, could be a mild.
Some birddogs had 1, some had 2.
Pat: Response says birddog with each crew team.
Dave: It is different, its just a clarification, birddogs could be watching
multiple crews. Some birddogs were watching 1 crew, but some were
watching multiple crews.
Doug: Restoration process has some phases. Person calls in or you
otherwise know theres an outage, that triggers someone to look at the line.
Dave: That is part of it, you look at the lines down, then there are spotter
reports, but once you have that info you go after the energized down line
first, then the public service stuff like REO town and hospitals.
Doug: To clarify, someone goes to look at a line and see if its down or
broken. That goes into a work order system and a line crew goes later?
Dave: Outage management system. It goes into the spreadsheet.
Doug: When a line crew makes a repair, what are the steps to restoration?
Dave: First they will cut and clear the line. Once that has been done or is
being done, another process looks at all the major circuits that are out and
that is where they are. Just because lines were cut and cleared, doesnt
mean a crew is on its way to restore. If there is no down line, restoration
happens when line is energized. Can only bring back one circuit at a time.
Dont close in multiple breakers at the same time.
Doug: Are there other big roles in the restoration work?
Dave: Spotters do the assessing and bring it to the BESOC, BESOC develops
a work plan including maps, materials, etc. Goes into a stack that is
dispatched based on priority.
Doug: At different times, each of those stages could be the choke point.
Dave: The spotters were the choke point.
Doug: At some point, you would have had a pile of work for line crews to
do.
Dave: We kept up with the work that was done. The spotter crews werent
bringing back the reports fast enough until we got until the services around
the 28th.
Lark: That was the bottleneck.
Dave: Thats not uncommon for this type of process. That is a normal
process. Knowing we have the limits on the qualified people, well bring in
firms that will specialize in that.
121
Bill: Are there some slots for spotters with not technical skills, while others
must have technical skills? Some could just identify that there are down
lines, while others determine what is needed?
Dave: The quality of spotters varies and it impacts the reports. Some
spotter reports come back incomplete and that slows down the process, If
they cant identify a bad transformer or pole.
Bill: But there is a category of spotter that could be there, present
Dave: C Spotters.
Bill: You brought some in for training anyway. Has that been a volunteer
effort and/or are you going to be able to say we need X amount of people
who can understand that level spotter and you will be called in v who
shows up.
Dave: Voluntary v. Mandatory. In the past, the electrician (hourly
employees) have a financial incentive. But this storm was so much bigger
than in the past that we went through the level of employees that we had
trained. Are we going through the organization to recruit individuals?
Yeah, we will discuss making those mandatory for storms. But when we
pull electricians out of the plant, we will have the power plant without
maintenance. And that might be valuable for reliability of the system.
Bill: And retirees?
Lark: And retirees.
Bill: Could you mandate some employees, have more people to draw on in
an unusual system like this.
Lark: Do we have different sorts of spotters? Yes. Assessor types, As, Bs,
Cs.
Dave: A, B, C spotter. A spotter is a journeyperson qualified in the field. B
is safety to watch A in case something happens. C spotter goes out after the
situation has been assessed to stand there and protect down line. We have
plenty of those. We dont have enough A spotters and thats what slows us
down.
Doug: A spotters not part of Mutual aid?
Dave: No.
McD: Did we have A spotters prepositioned?
Lynn: Cant get them in the field because we cant predict what lines go
down.
McD: Were they staged?
Dave: They were told to be in Sunday morning, but they were not there
sitting there. We had 31 total spotters. 14 A spotters on Sunday. Went up
to 18, all BWL employees.
122
McD: Ipad or electronic form to use. Was that lost when the OMS was shut
down requiring them to use written doc?
Dave: Thats an enhancement to come, they never had that.
McD: They were using written docs?
Dave: Yes.
McD: Part of the bottleneck?
Dave: Yes. Part of our recommendation to get that. The contractor has a
PDA type Ipad.
McD: Is that contract in place now?
Dave: Talked on Friday, working on T&C.
Bill: Do the MAA require cost relative to just having the contracts in place
to be willing and able?
Dave: Everybody wants their services at the same time, have to be on the
button to call them in ASAP. Must establish ability to call in an option, put
down a retainer. Must negotiate.
Lark: When came in 2007, no MAA; since then, worked out with CE &
municipalities, subsequently others after the storm that provides for
compensation. Already been two possible ice storms and the pins and
needles were on. Had we not entered into agreements in advance to make
sure we have people available on standby, not so much with MMEA, not so
much with CE, but with contractors you have to pay to have them on
standby.
Bill: Or youll pay more when you need them later:
Lark: Not just pay more; they wont be there. Generally there are
contractors, but supply and demand. This storm was unusual and they
werent there.
Bill: Looking at the ramping up of crews. 22nd- 26th had 20 crews. Then 27
had 34, 28 had 35, 29 had 54, ramped up.
Dave: That includes tree trimming, too.
Bill: What was it? You brought line crews in but not a higher number in the
first few days.
Dave: The trigger for the increase was the spotter reports. Basically you
size your crews after the assessment.
Bill: So those crews were available if you could use them.
Dave: If we brought them in with the spotter bottleneck, they would have
sat there with nothing to do and just been an added expense. The first
phase of the restoration is the primary circuits. Toward the end you break
into laterals and house services so your crewing services with mushroom
even though the lines down is lower because they are more spread out.
123
Bev: Going back to Dec 21. The EL police dept had weather spotters. When
they see the weather is changing, they will call public works and do a ride
along in EL to see what is going on in preparation for what will happen the
next day. Using regional resources and having that contact will be a natural
response, they are there already. The night of Dec 21, I had guests who left
at midnight and we knew then that we were in trouble. I had a fear of
people driving, knowing police are out there anyway, you could have had
that contact to use the resources at hand.
Dave: We knew the weather was bad, thats why we activated the plan on
Dec 21st. The police were helpful in identifying lines down, the 911
dispatchers.
Break
McD: We have 2 hours and 4-5 topics to cover. Turn to Darnell for
Response issues starting with areas not yet covered.
Darnell: The public and the committee must have answers to help
understand response. Crucial in terms of communication and outreach.
Collective bargaining as it relates to the response broadly. Are there any
provisions in the CBA that allow the LBWL to suspend collective bargaining
rules during incidents like the ice storm? What in the CBA are provisions
that were impediments or things to be enhanced?
Lark: I dont think the CBA in one way or another hampered
communications or restorations efforts. Can we suspend in an emergency?
Unsure, but think we can suspend CBA. Here, were less interested in
suspending CBA and working the plan we had, nothing to gain from
suspending CBA. We know system better than anyone. When working on
primary circuits, we want our people working on it for safety reasons.
Darnell: What about people who volunteered to help but were turned away?
Lark: Unfamiliar with that situation. Is anyone aware that people offered to
help and were turned away? No. Not everyone at the BWL is a union
employee, maybe 60-40. Many people worked the storm who were not
union employees.
Pat: Are lineworkers?
Lark: Are all union members.
Darnell: Never in conflict with CBA to deliver or identify crews?
Dave: All crews brought in were IBEW crews so it was not an issue.
Jerry: What about work rules? Memorandum of Understanding, rumors,
any work rules that inhibited or slowed down contract for other employees?
124
Lark: No, mainly it was safety. Any rules abided by that to an outsider
looked to slow things down were safety rules we abide by. In a dangerous
business, electricity, water, T&D, Generation, steam, chilled water, safety is
paramount and job one.
Dave: There were no work rules that impeded us.
Pat: Birddog?
Dave: That is across the utility industry. Could we just turn loose a bunch
of crews on the system? It would put it at jeopardy because we might not
be familiar with something that is wired up differently, there is that
exposure.
Jerry: Primary, main lines and then the rest of the system. When it got to
the rest of the system, was there BWL employee?
Dave: No, only on primary over 60,000 volts. Thats why we could bring in
so many crews at the end. They were not supervised.
Darnell: The MoU/Work rules didnt shape the response from precluding
people who wanted to get involved from getting involved?
Dave: Not to my knowledge.
Pat: Primary systems, were the first addressed?
Dave: Wires down, then critical like hospitals, them primary circuits.
Pat: Just BWL workers?
Dave: Outside contractor crews for primary that had BWL escort with
them, the birddogs.
Pat: Were there enough birddogs for those first few days to accompany the
crews who came in through MAA?
Dave: Yes.
McD: The req. to have a 3-member crew work a line, where is it?
Dave: Safety manual discussed in a contract.
McD: Safety committee would write manual, 3-member crew is in the safety
manual, so it is incorporated into the CBA.
Dave: Not part tangentially, just wills. 5-1-3.
Jerry : So its in there so youre bound by it.
Dave: Clarified in the safety manual rather than the contract.
Larry: Safety has to be job 1. Assume there are rigid OSHA rules.
Lark: Yes.
Bill: Enough BWL employees to escort for primaries, but if we could
remove spotter bottleneck, it would relate to the number of employees
actually had if you could get out with more force sooner.
Dave: To go full throttle, take all crews, dissolve them, make them birddogs,
and send them out. But we need the information first. This is what we did
as soon as we could, dissolve our own crews to make birddogs.
125
126
Jerry: But if that could be more organized, it should help make sense of the
chaos, that should be a top priority if that is the bottleneck.
Darnell: To organize and respond with all the resources you have so the
power loss is mitigated by those efforts. When no issues with CBA, it still
took 10-12 days. Where is the focus so we can mitigate the inconveniences
as soon as possible. Thats what I want in an operations plan, something
that says how to identify and deploy resources to mitigate the time the
power is out.
Lark: We have employed measures to mitigate those measures. In addition,
hiring 3 new line people and more dispatchers. We were pretty happy with
the tree trimming, and it estimated if we had trimmed poorer then 70% out
nor 40%. But about being better next time, bottleneck is a point, but we
need to do more tree-trimming. Mother Nature or God just did a lot of it
for us and we have a pretty aggressive plan in place. But since then, weve
upped since 5 to 18 crews. We must identify the places with all the trees
and visit them. Thats the number one answer to prevent the next outage
from not taking 10 days. At MPSC, of all the calls taken in 4 year term, 90%
involved tree trimming, half wanted them out, half wanted more trimming.
But now we are really ramping up, so were about to get more people
yelling.
Bev: EL?
Lark : oh they are.
Bev: How to identify?
Lark: Wherever there are, for example, in Glencairn or Moores Park, you
see a ton of trees.
Lynn: Moores River Drive has been trimmed, we are now planning
Glencairn and planning all of East Lansing. We are getting more receptive
to tree-trimming.
Joan: Are you coordinating with city foresters?
Lynn: We do coordinate with Lansing, working on coordinating with EL
and any other areas that have foresters.
Lark: But the foresters might not do line work. There are lot of protocols
and equipment. We will take anyones help, but they shouldnt be around
power lines.
Bill: As you expand tree trimming, obviously will cost more money. Are you
adding more employees or contracting out?
Lynn: Increasing contractors, added a second tree trimming contractor,
and possibly third.
Bill: How do you evaluate the quality of the work?
128
Lynn: Were inspecting monthly as we got bills, now were going the next
day to inspect and make sure it is up to standards.
Bill: Report said supervisor can be contacted for the resident to inquire
about tree-trimming, some will be upset and some wont, but oversight and
monitoring. In forums people claimed trees were marked but not cut.
Lark: Sure it happened, but seems not exactly true. We make a real effort
to give our number out, and wish people would call.
Lynn: We are getting calls from customers and following up on all of them.
Pat: How to follow up? I got a notice in October but dont know if it was
done. Are there subsequent notices?
Lynn: We dont leave a subsequent notice, but we might do that now. We
mark them with a dot. Well go out to look and see if the tree was trimmed
or not. We can look at records with contractors to see if they did what they
say was done.
Pat: Follow-up would help a lot.
Lynn: Also getting a lot of planning 2 months in advance and people are
complaining shortly after notices, but havent been out yet.
Darnel: OMS. What is current status of emergency management system?
Lark: Those things we had problems with are all fixed. Nick, what doesnt
work? Heres the OMS. We installed in Feb. Found problems in May.
Problems we thought were fixed by October, but doesnt mean it can be run.
Professionals have to come document and adjust the system and were
scheduled to come out when the ice storm hit. There were 100 little
questions but 2 big ones. This is not unusual with anything. Nick will talk
about OMS. If there are or not problems talk about how that works. Does
it work right now, as to the problems we had, and two, -- it wasnt like we
were asleep at the switch on the vendors OMS system. Nick worked
diligently with the vendor.
Nick: That system works today. If we had an ice storm today, the system
would response appropriately. I cant talk about how systems are tested
and put into place. It is complex to model our entire T&D system. The
system will look at two calls, see what they have in common, and estimate
what the issue is. Is it a switch? More info might move it to transformer
and a crew can go right to the transformer. Complex to project upward. If
you use MS, you get patches to fix bugs. This is like that. Report problems
and get them fixed. Some were cosmetic, some were significant (conductor
cuts). In the process, document the problem to GE, wait for fixed, test the
fixes, and put them into production. Thats where they were when this ice
storm hit. GE had fixed issues and they rolled ready to put into production,
129
they were testing, we had approved for them to put it into production, and
then the ice storm hit.
Darnell: Had it been functional at the time how beneficial would it have
been?
Lynn: If we had the ability to make the conductor cuts, we could have for
example had a circuit and modeled the conductor cuts in the OMS and seen
the pocket outages in a neat package, and when we closed the breaker, only
the people who actually came back up would have been visible.
Lark: Would it have shortened the life of the outage?
Dave: The major inconvenience was on the customer not being able to be
told their information. It would have marginally adjusted restoration
times. Reading the DTE report, they had all the gadgets and programs and
vegetation management and construction and annual drills and OMS and
phone systems that didnt operate without any major issues, smart meters,
adequate crews, and it still took them 8 days to restore 10% of their system.
The technology had a marginal effect. What drives the restoration is the
intensity of the storm and the duration of it.
Darnell: So are we better off?
Dave: It might be 8 days instead of 10 days.
Lark: Our CSRs rely on the OMS to tell people stuff, and thats what wasnt
happening. If the OMS had worked, it would have been night and day as far
as getting information to the customers. Marginally on the outside we
could have grouped the pockets better.
Jerry: In Meridian Township, folks were the last ones, and no one knew
they were out. Having the OMS working, would it have pointed these
people out sooner?
Sue: If they had called in.
Jerry: Took less than an hour to fix it and they were back.
Lark: Thats a problem, that people are frustrated that it only takes an hour
to fix it.
Dave: If they were part of the major circuit, thats the priority.
Pat: Does OMS require the public to call in, or does the OMS have the
capability to find them? People were forgotten. Once you have that
address, will they get left behind?
Lynn: The operator can put an address in, a citizen can call, or if the people
in the field had laptops, customer doesnt have to call, thats just one way,
sometimes we think we can get a full restoration and think everyone is back
but someone calls without power. There are times where you think
everyone is restored but you do rely on a callback.
130
Nick: its relatively easy to test simulated calls and test volume. This wasnt
a volume issue, it was just that occasionally the connection b/w the 877
number and the OMS would stall and have to rebooted.
McD: When was the first load test?
Nick: Before production, mid 2012. Dont think ever try a 250k volume test
before. Prior, peak calls in an hour would be 1600. Goal is to hit 40k in an
hour, though hate to imagine the storm needed for that. Test would be for
10k per hour. Switched from old system to new system on Feb 14, 2013.
There had been a test period before that, but was a straight cut over.
Bev: Can you tell people at the bottom of the list that they are a week out?
Sue: We would not be able to tell customers that. CSRs would not know
that. We are trying to provide outage maps to give broad restoration time
that could be narrowed down. We are working on that right now. We have
the map, but getting down to the individual addresses will take approx. 6
more weeks. Add text messages.
Lark: Return time will still have to be discussed and adjusted.
Jerry: Spreadsheet that info went in. Once the OMS didnt work, what
other IT support was given to arrange this system or was it all paper?
Lynn: All paper.
Darrel: So IT ability to disseminate information was hampered?
Nick: OMS was the major modeling tool. Once that was out, it became
spreadsheet and people.
Lark: And we tried to get the info out via press conferences. Tried to give
general info on when large block of people would come back, but couldnt
answer individual questions. This was a group decision, SXS not
responsible. On the first day, rather than like CE that said 7 or 8 days
BWL tried to estimate when a majority of customers would come back.
Better to say majority by close of business Christmas. May have made a
mistake, but most customers would be back by Christmas. 90% were up,
but if you were in the 10% you were unhappy. When you heard the majority
would be back by Christmas that meant to everyone that you would be back.
Would you rather know 10 days and be up sooner. Then we tried to think
how else to be helpful. We estimated 1000, but if you were in that 1000,
that didnt help. We took steps and tried to say where our crews were so
people knew where we were, but then people would not see them and get
mad. Then they would give streets, but people would look and not see
trucks and be mad. Tried to be helpful, but it engendered ill will. It was an
attempt to get as much info out as possible while being handcuffed.
132
Darnell: In the report speak to the failure of the OMS and the aggressive
plan to fix that and make sure it works. More proactive and reactive to fix
it to be able to make communications. Recognized shortcoming.
Lark: We believe it is fixed. It is my job as GM to find problems, fix them,
and move forward. We found the problem, fixed it, and moved forward.
McD: The EMS didnt go down.
Lynn: Correct. But it is just one line, not a map or specifics. We will find
the geographic locations on map.
Doug: Vegetation Management. No unique treatment of ash. Ash tree fall
has been significant problem. Will ash trees be just removed?
Lynn: Anything that looks like a threat to the lines will be removed. We
havent been aggressive but will. Issue is with taking down big trees and
nothing to do with the wood and dont want to leave it. But will start going
after ash trees.
Doug: 1200 miles of line, 700 underground (1900 total) all levels of line.
Lark: Generally the underground is downtown and some suburban
neighborhoods. Just distribution lines, not transmission.
Doug: How frequently do you visit each segment for veg management?
Lynn: Five years.
Doug: 250 miles of line a year? What have you been running?
Lynn: 5-7 cycle. 5 has been the goal, but there was a section not hit, so 1
year behind. Looking to see if we could even move to 3 year cycle.
Doug: QC sounds good. These lines have poles. How many poles?
Lynn: Well get back to you.
Doug: Whats the replacement schedule for poles?
Dave: Once every 10-12 years. This falls in line with DTEs plan.
Doug: Reconductoring (replacing lines)? A report says 10 miles a year.
Sue: Well get back to you.
Pat: 5 year cycle. How long does it take to go through cycle.
Lynn: Its all 5 years. Its a rolling thing.
Doug: What kind of conductors are you using?
Dave: Aluminum and Copper, triplex, spacer cable
Doug: Whats the state of the system as far as mix of ages and technologies?
Dave: Unsure.
Doug: Were the outages related?
Dave: 75% or more of outages were trees falling on lines, dont know if type
of line would have helped.
Doug: Hydrophobic coating?
Dave: Round.
133
Lark: There is pushback on smart meters, so we need to roll it out that gives
customers as much control as possible, issues with privacy and health. DTE
is starting and getting pushback. We want to be the best utility in the
country. In many places, BWL is the finest around. Most affordable,
reliable. In smart metering, dont want to be in the lead. Costs millions so
we want to follow behind so we can see where others make the mistakes
because we cant afford those mistakes. Thinking-wise we might be ahead,
but we want to watch to make sure we buy the right thing.
Sue: Four is hopeful; six is likely.
Bill: Tree-trimming budget.
Lynn: It has gone up quite a bit.
Bill: Broken out into management, labor, internal & outside, then other
management expenses.
Dave: Not just management, all other expenses beside labor.
Lark: its tree-trimming management, the management of trees, not
Management expenses. There was a dip in 2008 or 2009 and then brought
it back up. We were in kind of a recession so we backed down on it briefly.
In 2013, the number is all pre-storm. 2014 will be higher than that. From a
financial point of view as GM, it has been proved that it is an investment
that pays dividends.
Bill: Not free of charge, will cost. But lesson b/c we wanted to keep our
rates down, whats the balance of looking forward and adding things. How
to prepare and get a balance of cost that customers, owners, rateplayers can
identify and understand to be forward thinking to save money in the long
run.
Lark: Our mission statement wants affordable, reliable, with environmental
stewardship. Its affordable, and reliability is something we take extremely
seriously. This outage did not make me happy one bit. If you look at the
management over the past 5-10 years, SAIDI and CAIDI numbers and
Safety numbers, the BWL has nothing to be embarrassed about and
outperforms larger competition in the state. Will continue to be affordable,
reliable, and exercise excellent environmental stewardship. It is my job and
teams job to figure out how to be a lot more reliable, a lot more
communicative, and still beat our mission statement and be affordable.
Underlying it all is a notion of what you want to be: a Cadillac, a chevy, or a
Pinto. Everything costs a little more. Being as nimble and good as we are,
we can be affordable and reliable. I dont plan on this happening again. I
cant control a 40% outage Mother nature gives us, but I can fix the
communications.
135
Doug: Is there coordination b/w BWL and municipal Public Works when
stuff like this comes up?
Sue: Not a coordinated effort
Lark: But will be happy to. Any govt entity interested can call me.
Doug: Cities, townships do veg management. Efficiencies to be gained by
working together.
Lark: Yes, exploring that right now with Lansing. Not to say wont do it
with others, but it is being discussed right now. Need certain certifications
to work near lines. Economies of scale there with planning or trimming, or
disposal.
Doug: 8 member board. What committees within the board?
Lark: Whole, finance committees meet regularly, HR and executive
committee, have had ad-hoc committees from time to time. Principal are
HR, executive, finance, and committee of the whole.
Doug: No sewer, no electric supply or topical? All about management of
the business?
Lark: There are occasionally ad-hoc committees the board chair puts in
place.
Doug: Is there an annual review of reliability, response capacity?
Lark: Nothing annual. Generally I recommend topics to the chair, who
adds, strikes, or edits and the chair makes the final list. Committee of the
whole March 11 to review the 54 suggestions and progress on them will be a
topic. There are other timely topics. The board is appointed by the Mayor
with approval of the city council. 4 year terms, staggered.
Doug: The report: some are past, some are definitive will do some are
future will consider. How did you make?
Lark: New document made for March 11 board meeting, one side lists
actions and blue side is what we did. As GM, I want to make sure that if I
tell the board were doing something, it will be done. Dont see anything on
it we wont do in short order, and have already taken a big bite at the 54
things.
Pat: Great, is there a move to make it public on website.
Lark: Sue and SXS says yes.
Sue: You asked us to do that.
Lark: Okay, available tomorrow night on the website. Also the Connections
that comes out will have the items, if not the updates.
Larry: OMA?
137
Lark: Yes, always open to OMA. BWL created by a city charter. Governed
by the city charter. Derived all power. We are part of the city.
Larry: Who owns BWL, who has authority over BWL, what about EL, Delta.
Only the citizens of Lansing own the BWL?
Lark: Not definitive, owned by city of Lansing, not EL or DT. Not atypical.
Many municipal utilities like Traverse City serve outside Traverse City. As
who operates it, under my reading of the charter, autonomous entity
operated by GM with the approval of the chair and 4 boardmembers. The
board is autonomous and does not report in a way to anyone, but does have
meetings with the city council.
Pat: Can you remove a board member?
Lark: Unsure. Mayor can appoint subject to approval, unsure about
removal.
Pat: Record Retention Policy.
Sue: Didnt ask for that.
TJ: Yes we did.
Sue: Ok. You can have it.
Doug: [missed the question]
Lark: Return on equity.
Doug: Payment in lieu of taxes?
Lark: Multiplied by 6.1% of revenue. 20-21 million, to 22 million. Will the
summer be hot?
Doug: Where does 6.1% come from?
Lark: Can be any amount. Look at APPA overview of what returns on
equity (pilots?) appropriate amount is, what is average amount across the
country for utilities with >100 revenue, we have 350, average is 6.1. In this
case, kind of above pay grade. Decision is made by board of commissioners
and city council. Did give the board the info that 6.1% is the average
number.
Doug: Has 6.1% of revenue changed?
Lark: Twice. When I arrived, it was 4%. Two years ago, it went up to 5%,
then a year ago it went up to 6.1%.
Bill: 6.1% for a few years?
Lark: 5 year contract
Bill: City charter requires budget by June 1. Is that budget open for public
review?
Lark: Everything is subject to OMA. Some stuff can be closed for
personnel, and we are working on that budget which will be prepared and
filed timely.
138
Environmental, you cant go too far. The only utility to reduce carbon by
20% across the footprint, not many peers. Used to be largest solar array.
Try to be on cutting edge, retired 4 coal units, almost 7, saves 339 tons of
coal. As to peers, all different, but try to keep an eye out. APPA has 2000
utilities; BWL has been a platinum award winner for reliability. Won the
diamond award given to 6 utilities of 2,000, hard to do in an ice storm. But
no one cares, if their power is off. Many people at the BWL are second to
none and do a great job.
McD: And Mr Lark, we will let you have the last word. Thank you, all.
141
Appendix F
Reference Materials
The following reference materials were relied upon by the CRT, or its subject matter
experts, for guidance and best practices in developing this report. They are included so
that the readers of the report and the BWL can delve more deeply into the sources of the
recommendations included herein and may also be helpful in pointing the way
forward.
Bak's Sand Pile: Strategies for a Catastrophic World. Lewis, T. G. Williams, CA:
AGILE, 2011. Print.
"New Hampshire December 2008 Ice Storm Assessment Report." NEI Electric
Power Engineering, 28 Oct. 2009. Web.
143
Utility Security: The New Paradigm. Seger, Karl A. Tulsa, OK: PennWell, 2003.
Print.
144