Fired: A Fully-Labeled High-Frequency Electricity Disaggregation Dataset

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 4

FIRED: A Fully-labeled hIgh-fRequency Electricity

Disaggregation Dataset
Benjamin Völker, Marc Pfeifer, Philipp M. Scholl, Bernd Becker
Chair for Computer Architecture and Embedded Systems
University of Freiburg
[voelkerb,pfeiferm,pscholl,becker]@informatik.uni-freiburg.de

ABSTRACT More specifically, providing real-time aggregated electricity feed-


As more and more homes are equipped with smart electricity me- back of the total load of a home achieved energy savings of 8.6 % on
ters, home owners gain the ability to monitor their total electricity average. If this feedback is delivered for each individual appliance,
consumption on a daily or hourly basis. Techniques such as load the saving potential can even be increased to 13.7 % on average.
forecasting, load disaggregation, and activity recognition try to However, appliances do not (yet) expose their electricity consump-
provide even better insights into our electricity consumption, high- tion. Instead of replacing appliances with ones which measure their
light saving potential or improve our daily living. To develop and consumption, a solution that can be retrofitted is more sustain-
evaluate these techniques, publicly available datasets are used. We able. Smart meters - which are rolled out over the recent decade
identified a lack of high frequency fully labeled electricity datasets (e.g in Germany the roll-out starts with consumers > 6000 kW h
in the residential domain and present the FIRED dataset. It con- in 2020 [6]) - are promoted to provide such a solution by using a
tains 52 days of 8 kHz aggregated current and voltage readings of techniques called Non-Intrusive Load Monitoring (NILM).
the 3-phase supply of a typical residential apartment in Germany. NILM systems disaggregate the composite electrical load, which
The dataset also contains synchronized ground truth data as 2 kHz is measured at a central location in the electrical network, into
readings of 21 individual appliances, as well as room temperature the load of each individual appliance. The technique has been an
readings and fully labeled state changes of the lighting system, active research area since the early 90’s when G. W. Hart’s group
resulting in a complete and versatile residential electricity dataset. introduced a first working prototype [7]. NILM systems can be cate-
gorized into two groups: i) event-based approaches that detect state
CCS CONCEPTS changes in the signal and relate them to appliance state changes (e.g.
[7], [15]) and ii) event-less approaches which optimize an estima-
• Hardware → Power and energy; Smart grid; • Computer sys-
tion of the overall system state using e.g. Factorial Hidden Markov
tems organization → Sensor networks.
Models ([10], [2]). As pointed out by [3] these approaches also differ
KEYWORDS on whether they use supervised, semi-supervised or unsupervised
methods as well as on the granularity at which data is required.
load monitoring, non-intrusive load monitoring, electricity feed- Some algorithms require low frequency data in the range of 50 Hz-
back, electricity dataset, dataset labeling 1/60 Hz (e.g. [7], [16]) while others - typically event-based methods
ACM Reference Format: - rely on features derived from high frequency signal components
Benjamin Völker, Marc Pfeifer, Philipp M. Scholl, Bernd Becker. 2020. FIRED: which can only be extracted from data in the 1-16 kHz range (e.g.
A Fully-labeled hIgh-fRequency Electricity Disaggregation Dataset. In The [8], [15]).
7th ACM International Conference on Systems for Energy-Efficient Buildings, Comparing different algorithms on existing datasets remains
Cities, and Transportation (BuildSys ’20), November 18–20, 2020, Virtual Event,
challenging as not all can be applied to the same dataset. Either the
Japan. ACM, New York, NY, USA, 4 pages. https://doi.org/10.1145/3408308.
3427623
dataset’s sampling frequency is too low (for event-based approaches
using high frequency features) or the ground truth data (individual
1 INTRODUCTION appliance power consumption and state information) is incomplete,
misaligned or non-existent. Furthermore, some datasets have long
As electricity production has a large, negative environmental im- time periods of missing data (e.g. up to several days for REDD [10]
pact, we should strive to minimize the amount of electricity we and UK-DALE [9]) and require custom code for loading the data.
consume. According to a meta-study by Ehrhardt-Martinez et al. [4] Moreover, the sampling rate of appliance level data is typically
frequent per appliance consumption feedback is one of the key fea- low (except for the non-residential BLOND [11] dataset) making
tures to achieve high energy savings in the residential domain. them unsuitable to test semi-supervised hybrid load monitoring
Permission to make digital or hard copies of all or part of this work for personal or approaches (such as [15]) which use socket level data to train their
classroom use is granted without fee provided that copies are not made or distributed algorithm. A comparison of high-frequency electricity datasets can
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the be found in Table 1.
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or We propose the Fully-labeled hIgh-fRequency Electricity Dis-
republish, to post on servers or to redistribute to lists, requires prior specific permission aggregation (FIRED) dataset spanning 52 days of recording. We
and/or a fee. Request permissions from permissions@acm.org.
BuildSys ’20, November 18–20, 2020, Virtual Event, Japan identified four improvements to existing datasets: (1) residential
© 2020 Copyright held by the owner/author(s). Publication rights licensed to ACM. appliance level data of high sampling rates (2 kHz) of nearly all
ACM ISBN 978-1-4503-8061-4/20/11. . . $15.00 household appliances is provided, (2) this data is synced to 8 kHz
https://doi.org/10.1145/3408308.3427623

294
BuildSys ’20, November 18–20, 2020, Virtual Event, Japan B. Völker, M. Pfeifer, P. M. Scholl, and B. Becker

Table 1: Comparison of different datasets used to evaluate NILM systems. We only compare datasets recorded at higher fre-
quencies (kHz) that can be used to evaluate event-based and event-less NILM approaches.

Dataset Domain Aggregated Appliance Appli- Duration Total Meets Additional data
resolution resolution ances (days) size requirement
BLUED [5] residence 12 kHz - 43 8 53 GB - appliance on/off events
REDD [10] residence 15 kHz 1/3 Hz 26 3-19 1.7 GBa 1 -
UK-DALE [9] residence 16 kHz 1/6 Hz 52 1629 ≈ 10 TB 1,3 utility meter readings
BLOND-50 [11] office 50 kHz 6.4 kHz 53 213 15.3 TB 1,2,3,5 -
BLOND-250 [11] office 250 kHz 50 kHz 53 50 23.4 TB 1,2,3,5 -
FIRED residence 8 kHz 2 kHz 66 52 1.2 TB 1,2,3,(4)b,5 see subsection 3.3
a The high frequency REDD data has been compressed in a custom lossy form. Raw data is only available for a small number of sample points throughout the data.
b Additional information about the state of an appliance is available for a subset of devices (lighting, televison, hifi-system, espresso machine).

aggregated data at sub-cycle precision, (3) additional state infor- 2.1 Aggregated Setup
mation of the apartment’s lighting and multimedia system, as well The aggregated measurements were collected using a self-built mea-
as temperature and humidity readings of each room in the apart- surement system [14] which we subsequently call the SmartMeter.
ment are recorded, (4) the dataset and python tools are provided as To measure the voltage of each supply leg (𝐿1 , 𝐿2 , 𝐿3 ), it is scaled
open-source for fast and easy replication by other researchers. down by a factor of 1:1000 using voltage dividers. The current
is measured using non-intrusive split-core current transformers
2 METHODS (YHDC SCT-013) with a ratio of 1:2000. The scaled voltage and cur-
We define the following requirements to generate an electricity rent signals are sampled at 8 kHz using a 24 bit ADC connected to
dataset which can be used to develop and evaluate a wide variety an ESP32 microcontroller. The hardware version highlighted in [14]
of electricity related algorithms for residential households: has been enhanced significantly. We added 8 MB flash memory to
R1: Simultaneous recordings of the whole house electricity con- be resilient against short connection dropouts (8 MB holds up to
sumption and the consumption of each individual appliance. The 41 s of data). We further added a Real Time Clock (RTC) that allows
individual data can be used to validate the appliance power esti- to sync the internal sampling frequency of the ADC to a precise
mates provided by some NILM algorithms or to test if an algorithm clock. Furthermore, we added a reliable Ethernet connection, as, in
would perform better if it is trained on the individual data. our findings, WiFi performance is poor in a fuse box environment.
R2: High sampling rates for the aggregated and the individual ap-
pliance consumption is required to extract high frequency features 2.2 Individual Appliance Setup
from voltage and current wave-forms and to explore how these We recorded the household’s individual appliances using 21 Pow-
can be used to improve traditional NILM algorithms. A trade-off erMeters. A PowerMeter is a self-built measurement system de-
between high frequency signal components and dataset usability signed to collect current and voltage wave-forms of individual
(required disk space and loading time) needs to be explored. appliances [14]. Its overall system architecture is comparable to the
R3: Continuous data recording of multiple days of data is crucial SmartMeter. It uses the same microcontroller and an equivalent
to understand and explore different consumption behavior based 24 bit ADC. Voltage at a power outlet is measured using a 1:580
on the time-of-day or day-of-week. voltage divider and the current drawn by the connected appliance
R4: High quality dataset labels are required for event-based is measured using a 3 mΩ shunt. The data is sent to a logger PC
NILM algorithms to assign an event to its corresponding appliance. over WiFi. Compared to the version used in [14], we removed the
As multi-state appliances show different consumption behavior de- wired interface, added 4 MB flash memory (holds up to 250 s data)
pending on their state (e.g. the power consumption of a dimmable and an RTC. The cost of one PowerMeter is approx. e35. The power
light depends on its brightness level), additional state information consumed by each PowerMeter itself is 0.6 W and is therewith com-
can enable algorithms to build more sophisticated appliance mod- parable to other systems, such as the one used in [9] with around
els. 0.5 W. To measure appliances that are directly connected to the
R5: High temporal accuracy of both data and its labels is re- mains such as the stove, we used PowerMeters with screw terminals.
quired. A label should match a corresponding pattern change in As a typical household can easily have more than 50 appliances (66
the data. Furthermore, all data streams should be in sync. including lights in the recorded apartment), connecting each with a
The FIRED dataset was collected at an apartment building in metering device seems to be infeasible. Therefore, some appliances
Germany which was built in 2017. The data covers a single three- which either belong to the same category (network equipment) or
room apartment in the building with 79 m2 of space. The apartment are only used together (e.g. HiFi system and subwoofer) are con-
is occupied by one family (two adults and one child). The average nected to the same PowerMeter. Furthermore, some appliances are
daily power consumption was 6.41 kW h during the recording. A rarely used, such as the vacuum cleaner or the kitchen machine
central logger PC gathers the data of each meter using TCP con- and are, therefore, connected to a specific PowerMeter (ID 11) for
nections, stores it into files regularly and uploads the files to an which all appliance connections and disconnections are labelled by
external data center for persistent storage. hand.

295
FIRED: A Fully-labeled hIgh-fRequency Electricity Disaggregation Dataset BuildSys ’20, November 18–20, 2020, Virtual Event, Japan

VmDrtmeter001 L1
250 VoltDge 1000
0
mains
0 VmDrtmeter001 L1
Current 2500
250
−250 1000
−1000 Base
VoltDge
0 0 Others

Ln mALn mA
VmDrtmeter001 L2 Current
Ln V Ln V

1000
−250
250 −1000
2000 Fridge

Apparent Power [W]


VoltDge

0 VmDrtmeter001 L2 0

Current
1000 Network Equipment
250
−250
−1000 Television
VoltDge

0 0

Current
VmDrtmeter001 L3
−250
250 5000
1500 Oven
−1000
0 VmDrtmeter001 L3 0 Espresso Machine
250
−250 −5000
5000 Office PC
0 powermeter15 0 1000
250
−250 −5000
5000
0 powermeter15 0
250
−250 5000
−5000 500
0 0
0 0 0 0 0 0 00
−250 .00 .10 .20 .30 .40 .50 2.6 −5000
0 :52 :52 :52 0:5
2 :52 0:5
2
0:5
6:0 :00 :00 6:0 6:0
0
6:0 :0
1 0 16 0 16 0 1 0 1 0 1 0 1
6
0
.00 .10 .20 TLme of2DDy
.30 .40 .50 .60 0
:52 :52 :52 :5 :52 :52 :52
:00 :00 :00 :00 :00 :00 :00 06:00 08:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 00:00
16 16 16 16 16 16 16
TLme of DDy Time of day
Figure 1: Voltage (red) and current (blue) wave-forms of
SmartMeter001 and PowerMeter15 (recording from day Figure 2: The apparent power consumption of the apartment
2020.06.09 at 16:00:52). (2020-07-02). The black line (zoomed) shows the consump-

tion recorded by the SmartMeter. The stacked colored blocks


show the individual contribution of the six appliances con-
2.3 Additional Data suming most power. The consumption of all remaining ap-
We installed additional hardware to log certain appliance events pliances including the lighting is shown as others.
in the apartment: a ZigBee transceiver and smart wall switches
logged state changes and brightness information of all light bulbs and current wave-forms of the apartment’s three phase power sup-
in the apartment. An infrared receiver monitored interactions with ply (𝐿1 , 𝐿2 ,𝐿3 ) with a sampling rate of 8 kHz. Even if the hardware
appliances such as the television and HiFi system. Furthermore, we is capable of recording 32 kHz data, reliability was more important
distributed temperature and humidity sensors to each room. All and, therefore, the sampling rate was chosen based on the findings
events and sensor readings are stored into corresponding log files of [1] that “there may be little additional benefit between 15-40 kHz
and are part of the dataset (see subsection 3.3). because of the noise in that range in real buildings”. As our meters
directly convert the ADC measurements to physical quantities, no
3 DATA RECORDS data pre-processing is required. The units are Volt for voltage mea-
The dataset contains the raw voltage and current wave-forms mea- surements and Milliampere for current measurements, respectively.
sured at the mains and of 21 individual sockets. Additionally, we Figure 1 shows the raw voltage and current measurements of the
provide derived per day and appliance 50 Hz and 1 Hz power read- SmartMeter (position 1-3) and PowerMeter15 (position 4). The high
ings. The data is stored as multiple Matroska multimedia (mkv) files. sampling rate and achieved clock synchronization can be seen.
The benefits of using multimedia containers to store curated time-
series data has been explored previously [13]. Using e.g. Matroska 3.2 Data Summary
allows multiple streams (video, audio or subtitles) to be merged into The data summary is derived from the raw data. We calculated the
a single file. Time series sensor data can, therewith, be stored as active (𝑃), reactive (𝑄) and apparent (𝑆) power for all recording
audio streams with text based ground truth values as subtitles. We devices split into each day of the recording based on the mains
use a lossless WavPack compression which reduces the size with frequency 𝑓𝑙 = 50 Hz, according to the following equations:
a ratio of 1.46 (only 1.42 can be achieved with hdf5). Metadata is
𝑃 (𝑛) = 𝑁1 · 𝑖=0
Í𝑁 −1
added for each audio stream in the multimedia container. The times- 𝑈 (𝑖) · 𝐼 (𝑖), (1)
tamp (with microsecond resolution), name of the recording device, 𝑆 (𝑛) = 𝐼𝑅𝑀𝑆 (𝑛) · 𝑉𝑅𝑀𝑆 (𝑛), (2)
sampling rate, number of channels, types of measure and duration
𝑆 (𝑛) 2 − 𝑃 (𝑛) 2 .
p
𝑄 (𝑛) = (3)
is stored, making each file self descriptive and usable without prior
knowledge. To keep file sizes and file loading times reasonable, 𝑃, 𝑄 and 𝑆 are calculated for each non-overlapping window 𝑛.
files are split at regular time intervals (10 min for raw data and The length of the window is 𝑁 = 𝑓𝑠 with 𝑓𝑠 being the sampling
𝑓
24 h for summary data). Further information about all appliances 𝑙
frequency of the raw voltage and current signals. 𝑉𝑅𝑀𝑆 (𝑛) and
in the apartment (brand, model, supply leg, PowerMeter-ID, etc) is
𝐼𝑅𝑀𝑆 (𝑛) are the root mean square values of the voltage and current
available in json and pdf format.
for window 𝑛. As typical smart meter data is sampled at range of 1-
0.01 Hz, we additionally provide the dataset in a down-sampled 1 Hz
3.1 Raw Data version. The 1 Hz summary combines all measurements into a single
Each PowerMeter recorded voltage and current wave-forms with a 923 MB mkv file which can be used to quickly inspect specific events
sampling rate of 2 kHz. PowerMeters are capable of recording data in the dataset or to simply get an overall view. The 1 Hz apparent
with up to 7.875 kHz. TCP throughput is, however, limited if multi- power consumption of one day is shown in Figure 2. The six topmost
ple devices send data over the same WiFi network. As a compromise consumers are plotted individually, while the remaining consumers
between sampling rate and network stability, we chose to sample are aggregated and plotted as others. The whole apartment power
the data with 2 kHz. In contrast, the SmartMeter recorded voltage consumption is also shown as an aggregate of 𝐿1 , 𝐿2 and 𝐿3 (mains).

296
BuildSys ’20, November 18–20, 2020, Virtual Event, Japan B. Völker, M. Pfeifer, P. M. Scholl, and B. Becker

3.3 Logs and 2 kHz for individual appliance data) and the achieved clock
Further CSV files are available that contain annotation data (events synchronization of around 10 ms allows to use the dataset to evalu-
and sensor readings) with their corresponding timestamps. We di- ate event-based and event-less NILM algorithms. FIRED contains
vide all annotations into three categories: Light, Sensor, Device. state changes of the lighting and lots of additional information
Light annotations contain the light’s state (on or off ), the light’s about each appliance, which provide not only new opportunities
intensity (0-100 %) and the light’s colors in hexadecimal or None if for testing disaggregation algorithms but also for combining such
the light does not support to set different colors. Sensor readings information with electricity readings. Future work will focus on
contain the measured value in degree Celsius for temperature read- extending the recording period of the dataset. We further want to
ings and % for humidity readings, respectively. As each sensor sends apply the semi-automatic post-processing step introduced in[14]
a new value only if it has changed, the value between two readings to extract and label the state changes of all appliance in the dataset.
can assumed to be constant. Device annotations give information By providing advanced electricity datasets, we help researchers
about the state or event of the appliance. For instance, individual to focus on improving their algorithms. As load monitoring and
key presses on the television remote are logged (e.g. ch_1 or power). eco-feedback mechanisms have shown enormous energy saving
potential, ongoing development is crucial to contribute to preserv-
4 DATA STATISTICS AND VALIDATION ing earth’s climate by using clean energy in sustainable cities (Goal
13,7 and 11 of UN’s Sustainable Development Goals [12]).
Using our recording framework, we collected a total of 27456 hours
of raw current and voltage wave-forms. The measurements are REFERENCES
provided as-is and no filtering has been applied. The dataset ful- [1] K Carrie Armel, Abhay Gupta, Gireesh Shrimali, and Adrian Albert. 2013. Is
fills all requirements initially stated in section 2. The richness of disaggregation the holy grail of energy efficiency? The case of electricity. Energy
the captured data can be seen in Figure 1 for the SmartMeter and Policy 52 (2013), 213–234.
[2] Nipun Batra, Jack Kelly, Oliver Parson, Haimonti Dutta, William Knottenbelt, Alex
one PowerMeter unit (R2). The derived apparent power with the Rogers, Amarjeet Singh, and Mani Srivastava. 2014. NILMTK: An Open Source
contribution of each individual appliance can be seen in Figure 2 Toolkit for Non-intrusive Load Monitoring. In Proceedings of the 5th International
Conference on Future Energy Systems (Cambridge, United Kingdom) (e-Energy ’14).
(R1). Ideally, the superposition of all PowerMeters should match ACM, New York, NY, USA, 265–276. https://doi.org/10.1145/2602044.2602051
the SmartMeter’s apparent power. However, a small offset between [3] Christian Beckel, Wilhelm Kleiminger, Romano Cicchetti, Thorsten Staake, and
the aggregated and individual apparent power was observed. This Silvia Santini. 2014. The ECO data set and the performance of non-intrusive load
monitoring algorithms. In Proceedings of the 1st ACM Conference on Embedded
is mainly due to the fact that the apartment’s lights and ventilat- Systems for Energy-Efficient Buildings. ACM, 80–89.
ing system are not directly monitored. Furthermore, the constant [4] Karen Ehrhardt-Martinez, Kat A Donnelly, Skip Laitner, et al. 2010. Advanced
power consumption of 21 PowerMeters needs to be considered. metering initiatives and residential feedback programs: a meta-review for house-
hold electricity-saving opportunities. American Council for an Energy-Efficient
The power profiles of each light can be estimated using the light Economy Washington, DC.
annotation files and individual light information which we provide [5] Adrian Filip. 2011. BLUED: A fully labeled public dataset for event-based non-
intrusive load monitoring research. In 2nd Workshop on Data Mining Applications
with our dataset (R4). The apartments base power consumption can in Sustainability (SustKDD). 2012.
be estimated during the night when lights are typically off and most [6] Bundesministerium für Wirtschaft und Energie (BMWi). 2016. Gesetz zur Digi-
appliances do not consume power. By including the base power talisierung der Energiewende. https://www.bmwi.de/Redaktion/DE/Downloads/
Gesetz/gesetz-zur-digitalisierung-der-energiewende.pdf
and the power of the lighting, no visible offset could be identified [7] George William Hart. 1992. Nonintrusive appliance load monitoring. Proc. IEEE
as shown in Figure 2. The availability ratio of all metering devices 80, 12 (1992), 1870–1891.
during the recording duration was 99.98 % (R3) including occasional [8] Matthias Kahl, Anwar Ul Haq, Thomas Kriechbaumer, and Hans-Arno Jacobsen.
2017. A Comprehensive Feature Study for Appliance Recognition on High
data loss. As our system consists of multiple measurement devices Frequency Energy Data. In Proceedings of the Eighth International Conference on
distributed across the apartment, a clock synchronization mech- Future Energy Systems. ACM, 121–131.
[9] Jack Kelly and William Knottenbelt. 2015. The UK-DALE dataset, domestic
anism is required to have precise timestamps for events and the appliance-level electricity demand and whole-house demand from five UK homes.
measurement data itself (R5). Therefore, we added RTCs to synchro- Scientific data 2 (2015), 150007.
nize the ADC clock of each metering device. Furthermore, all meter [10] J Zico Kolter and Matthew J Johnson. 2011. REDD: A public data set for energy dis-
aggregation research. In Workshop on Data Mining Applications in Sustainability
synchronize their internal time using an NTP server. The achieved (SIGKDD), San Diego, CA, Vol. 25. 59–62.
clock synchronization can be seen in Figure 1. It shows the heating [11] Thomas Kriechbaumer and Hans-Arno Jacobsen. 2018. BLOND, a building-level
element of the espresso machine (connected to PowerMeter15) that office environment dataset of typical electrical appliances. Scientific data 5 (2018).
[12] United Nations. 2020. About the Sustainable Development Goals. https://www.
causes the current to rise rapidly. This rise is clearly visible in the un.org/sustainabledevelopment/sustainable-development-goals/
current data of both PowerMeter15 and 𝐿3 of the SmartMeter. The [13] Philipp M Scholl, Benjamin Völker, Bernd Becker, and Kristof Van Laerhoven.
2019. A multi-media exchange format for time-series dataset curation. In Human
time shift is only ≈10 ms allowing to synchronize the data streams Activity Sensing. Springer, 111–119.
of the distributed meters with sub-cycle precision. [14] Benjamin Völker, Philipp M Scholl, and Bernd Becker. 2019. Semi-Automatic
Downloads: The dataset and the files to generate the plots in Generation and Labeling of Training Data for Non-Intrusive Load Monitoring.
In Proceedings of the Tenth International Conference on Future Energy Systems
this paper are provided as open source and can be downloaded (Phoenix, USA) (e-Energy ’19). ACM.
from: https://github.com/voelkerb/FIRED_dataset_helper. [15] Benjamin Völker, Philipp M. Scholls, Tobias Schubert, and Bernd Becker. 2018.
Towards the Fusion of Intrusive and Non-intrusive Load Monitoring: A Hybrid
Approach. In Proceedings of the Ninth International Conference on Future Energy
5 CONCLUSIONS AND FUTURE WORK Systems (Karlsruhe, Germany) (e-Energy ’18). ACM, 436–438.
[16] Markus Weiss, Adrian Helfenstein, Friedemann Mattern, and Thorsten Staake.
The FIRED dataset features 52 days of simultaneous aggregated 2012. Leveraging smart meter data to recognize home appliances. In 2012 IEEE
and appliance level electricity measurements of a residential apart- International Conference on Pervasive Computing and Communications. IEEE.
ment in Germany. The high sampling rate (8 kHz for aggregated

297

You might also like