Antoine Marot – Lead AI Scientist @ RTE

Wednesday 13th of January 2021

Learning to Run a Power Network in a

sustainable world
A NeurIPS 2020 Competition https://l2rpn.chalearn.org/
 01

Introduction
 AI is BOOMing

ImageNet (2012) & NLP (2018) moments

Keys of success:
• Large Benchmarks & leverage big data
• Leverage computation
• Open-source platform & papers
• Deep Learning revival

NB: not much new algorithms & theory

AI based on Deep Learning started to impact many fields & layers of society at a fast pace 3
 AI & Electricity
« Artificial Intelligence is the new Electricity », Andrew Ng

Are blackouts a story of the past ?

4
Yet, Another system to consider : Earth!
Fossil fuels & grid expansion Renewables ressources & AI ?

Sustainability is the key, future smart grids have to do better with less
5
PS: not to forget pandemic situations..!
 AI & Electricity
« Artificial Intelligence is the new Electricity », Andrew Ng

Are blackouts a story of the past ?

NOT REALLY! 6
 AI & Electricity
« Artificial Intelligence is the new Electricity », Andrew Ng

How AI can be of any help here ?

7
 02

L2RPN challenge
« Learning To run a power network »
L2RPN Challenge

1) Test the potential of AI to robustly operate a power

9
grid in real-time given operational constraints.
 Different kind of flexibilities
On Grid Flexibilities
Power devices:
Topology:
• Transformer
•Little research
Branch switching Some sresearch
• Node merging/splitting
• Phase shifter
• HVDC
All over the grid & 0$ ! Few & local & 0$
• …
On Production and Load Balancing

Redispatching Lots of research Load shedding

Many over the grid & $$$ Many over the grid & $$$$$

2) Explore underutilized cheap topology controls with methods

10
can deal with non-linear, non-convex and combinatorial complexity
 Motivations for a challenge

Exhibit important real-world problems to the research community

Large Benchmark for Reproducible Science:

• Decouple the problem design & the solutions

Attract new communities, especially AI community, through an easy-to-use

platform & a gameified problem

11
 Machine Learning & power systems
Mainly Today

Power System Community AI Community

12
 Artificial Intelligence & Power Grids

One small step for Electricity, one giant leap for Sustainability !
13
 03

Problem design
Modeling of power system operation world
State: flows, productions, consumptions, power grid
topology, month, day, hour, etc
Action: connect/disconnect one line or change electrical
configuration at a substation or change productions
Reward: number of overflowed lines, cost of operations,
… Participant design choice
Fig. 1 - Reinforcement Learning interaction loop

Fig. 2 - Step-by-step evolution of the RL environment 15

 Operational Constraints

Line capacities have been calibrated to have congestions

to be managed appearing few % of the time.

The game should represent additional operational constraints:

1) A limited time to react to a congestion (2 ts)

1) A limited number of simultaneous action (1/ts)

1) A cooldown time before reusing an asset (3 ts)

There is hence a budget associated to the actions you take:

they should be picked up carefully !
16
 GridAlive - full ecosystem to design & play with
https://github.com/rte-france/gridAlive

L2RPN-Baseline
Repo to share open-source models & winning approaches
A
G
E
N
T

Data Analysis

ChroniX2Grid Grid2Viz
Power Grid time serie generation Scenario & agents study
Grid2Op: testbed platform to model real-time
operations, run & benchmark control algorithms

17
L2RPN serie competitions

Participate and join us!

https://l2rpn.chalearn.org/
 Agent evaluation – score function
Control the power flows to optimize the cost of operations on the power grid
while being robust to blackouts.

&

Under N episodes, the final score to minimize is:

test scenarios = 24 weekly scenarios at 5 - min resolution

Rescaled between [-100,100] for better interpretability 19
 Challenging environment
Scenarios varies from a week to a month long at 5 minute resolution

Some intrinsic challenges:

● Exploration is difficult
○ a random agent only survives 5 timesteps on average

● Initial grid configuration is safe suboptimal

○ Do Nothing agent can survive 500 steps on average

● Very Large discrete action space - too many possible actions per step
○ Agent has to choose one of 70 000 possible actions

Naive approach only yields to an agent that do only one or two specific actions

20
 03

Competition Results
 Participation

About 150 - 200 participants each: On Discord Forum:

● Intense last week with great new scores ● 420 signed in + 4700 messages !

• 4-month challenge
• 3 phases: warm-up - development - test
• Participants from both power system & ML communities all around the world
• Prize money for Top 3 Teams + best open-submissions

22
 Final Leaderboards

PS: multiply by 2 the scores to compare

on the same scale with Robustness
• Top Ranking Teams are the same on both tracks. They all come from China (Baidu,
Huawei, State Grid of China) !
○ Ranking is stable accross validation and test scenarios
• In Robustnees track, the wining team won with a good margin. 23
• Quite tight however on Adaptability Track
 Illustration of one winning approach

Check out Winning approaches !

You can watch all past winning approaches there: https://l2rpn.chalearn.org/competitions
 Robustness Track Results
Winner survived over 15/24 scenarios
(and second over 10/24):
● The 2nd, 3rd, 4th @ 5th also failed
over 9 remaining scenarios (winter
months: nov - dec - jan - feb)

Attack

Heatmap % time of survival in a scenario annimated episode

25
Robustness Track Results detailed

Inspect the episodes with grid2viz to compare the winning solutions

26
 Track Adaptability
Winner

Winner survived for 16/24 scenarios:

● The 2nd, 3rd, 4th @ 5th also failed over 7 of those
remaining
○ high renewable energy mixes: 2.7x & 3.2x
● But the 3rd survived at mix02 jan37
● The 2nd survived over 13/24 scenarios

Only one to survive

27
Another one survived
 L2RPN keeps running

Participate and join us! https://l2rpn.chalearn.org/

legacy phases:
● Robustness https://competitions.codalab.org/competitions/25426
● Adaptability https://competitions.codalab.org/competitions/25427
Sponsors & Collaborators

29
 Conclusions
1. Machine Learning approaches are promising for control problems and here
showed to generalize better than Expert Systems

1. It is now possible to learn topology controllers, opening a new field for

research and smart grid flexibilities.

1. Challenge helps develop benchmarks and enforce reproducibility to make

faster and stronger progress as a community

1. We should keep working on attracting AI researchers and collaborating with

them on power system related problems

30