Machine Translated by Google

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021 9003612

Open-Source Heterogeneous Constrained

Edge Computing Platform for Smart
Grid Measurements
Ashish Joglekar, Gurunath Gurrala , Senior Member, IEEE, Puneet Kumar ,
Francis C. Joseph , Graduate Student Member, IEEE, TS Kiran , Graduate Student Member, IEEE,
KR Sahasranand , and Himanshu Tyagi , Senior Member, IEEE

Abstract— This article presents a low-cost, open-source, Index Terms— Edge computing, Parallella, phasor
heterogeneous, resource-constrained hardware platform called measurement unit (PMU), power quality (PQ), signal
“Parallella” as a measurement device for edge-computing compression, waveform recording.
appli cations research in smart grid. The unique hardware
architecture of the Parallella provides a multitude of edge- I. INTRODUCTION
computing resources in the form of a Zynq SoC (dual-core
ARM + FPGA) and a 16-core co-processor called Epiphany. A Computing architectures have improved the
multifunctional intelligent electronic device (IED) design is MOBILE capabilities of edge devices that are used in smart
demonstrated to showcase the capabilities of the platform. A grid applications. Multicore processors, graphic processing
custom I/O board has been developed for the desktop and
units (GPUs), FPGAs, co-processors, and so on are being
embedded versions of Parallella, which can be interfaced with
external daughter boards and peripherals for measurements. integrated with mobile-/edge-computing platforms at an ever-
One such daughter board is an analog sensing board, which increasing pace. Moreover, in recent years, many of these
can measure voltages of all the three phases and four line platforms are being made available as a development board
currents using a 16-bit synchronous ADC set at 32 kHz. The to the research community, thus enabling unique edge-
ADC samples are synchronized to the PPS time clock of a GPS unit for providing global time reference.
computing applications. Some popular platforms are NVIDIA-
These captured seven-channel raw waveform data are sent to
a cloud server over a bandwidth-limited communication Jetson, Adapteva Parallella, Zynq Z-board, Intel Agilex, and
channel using a custom anomaly-aware data compression so on for industrial/product development. Rasberry-Pi,
algorithm implemented on the ARM. A phasor measurement Beaglebone, and so on are being used for the hobbyists or
algorithm using the Teager energy operator (TEO) is proof-of-concept developments [1].
implemented on the field-programmable gate array (FPGA). A
Many measurement platforms with multiple functionalities,
parallel power quality (PQ) measurement algorithm is implemented on the Epiphany.
The obtained measurements are found to be comparable to a called intelligent electronic devices (IEDs), have been
commercial power analyzer. proposed in the literature for recording and analyzing
meassured signals. A microcontroller with two dsPIC33
Manuscript received January 15, 2021; revised March 26, 2021; accepted
processes sors proposed in [2], one for sampling data at
April 21, 2021. Date of publication May 10, 2021; date of current version 7.68 kHz computations and the other for communication/
June 17, 2021. This work was supported in part by the Bosch Research and device control, to satisfy the power quality (PQ) measurement
Technology Centre, Bengaluru, India, and in part by the Robert Bosch Center
for Cyber-Physical Systems, Indian Institute of Science at Bengaluru, India,
requirements of the Brazilian power system. A synchronized
through the Project E-Sense: Sensing and Analytics for Energy Aware Smart 10-kHz data recorder is discussed in [3] with fast Fourier
Campus. The Associate Editor coordinating the review process was Dr. transform (FFT) for feature extraction. The need for low-cost
Guglielmo Frigo. (Corresponding author: Gurunath Gurrala.)
Ashish Joglekar is with the Robert Bosch Center for Cyber-Physical
measurement devices for research purposes has been
Systems, Indian Institute of Science at Bengaluru, Bengaluru 560012, India. recognized in [4] and developed an OpenPMU, which
Gurunath Gurrala is with the Department of Electrical Engineering, Indian captures six channels of data at 6.4 kHz. In [5], an FPGA-
Institute of Science at Bengaluru, Bengaluru 560012, India, and also with
the Robert Bosch Center for Cyber-Physical Systems, Indian Institute of
based phasor measurement unit (PMU) with a sampling rate of 12.8 kHz
Science at Bengaluru, Bengaluru 560012, India (e-mail : In [6], a multipurpose board based on Mini-ITX Motherboard
gurunath_gurrala@yahoo.co.in). with peripherals performing PMU, PQ, and signal capture at
Puneet Kumar, Francis C. Joseph, and TS Kiran are with the Department
of Electrical Engineering, Indian Institute of Science at Bengaluru, Bengaluru
25.6 kHz is proposed. A 12.8-kHz sampling rate analog to-
560012, India. digital (A/D) board is proposed in [7], which was interfaced
KR Sahasranand is with the Department of Electrical Communication to notebooks for communicating the data to upstream
Engineering, Indian Institute of Science at Bengaluru, Bengaluru 560012,
India. servers. The notebooks were used for PQ indices
Himanshu Tyagi is with the Department of Electrical Communication computations as well. In [8], a waveform recorder with
Engineering, Indian Institute of Science at Bengaluru, Bengaluru 560012, sampling rate of 7.68 kHz is developed on an FPGA & ARM
India, and also with the Robert Bosch Center for Cyber-Physical Systems,
Indian Institute of Science at Bengaluru, Bengaluru 560012, India.
Cortex M4 board, which uses data compression based on a
Digital Object Identifier 10.1109/TIM.2021.3078557 novelty detection in the waveform using discrete wavelet transform.
1557-9662 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google

9003612 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021

to reduce the transmission size. In [9], a PMU is developed sigma–delta ADC at a maximum sampling rate of 64 kHz, is
on a BeagleBone Black, which features an ARM core and two developed for Parallella.
programmable real-time units (PRUs) responsible for sampling A technique to synchronize the ADC's clock to the GPS
at 12.8 kHz. Usage of either a discrete PLL ASIC or a software- time clock PPS reference is also provided [9]. A custom serial
based PLL with reduced accuracy for the GPS discipline of peripheral interface (SPI) ADC driver and a GPS driver to
the ADC clock was also discussed. The ARM performed synchronize to GPS time clock reference have been developed
frequency and phasor estimation along with data on FPGA.
communication. An FPGA-based PMU sampled at 32 kHz is A custom FPGA binary has been compiled to port the
proposed in [10] using a NovTech IoT Octopus board with an devel oped custom drivers and algorithms. The developed
iterative Interpolated DFT algorithm for estimation. In [11], a binary file exposes the Zynq I/Os without losing the default e-
review of commercial PMUs, open-source PMUs, and link driver responsible for high-speed communication interface
integrated PMUs with digital protective relay functions is provided.
between the ARM and the Epiphany co-processor. Three
In [12], a 50-kHz IED intended for research and prototyping applications are demonstrated: a custom anomaly-aware data
HVDC protection on real-time simulators is discussed. Built compression algorithm on the ARM to enable the transfer of
on ZedBoard with modular peripherals, the IED performs 16- high-frequency sampled waveform data to a server, a Teager
channel signal sampling on one of its peripheral module and energy operator (TEO)-based phasor measurement algorithm
GPS synchronization on another. A very high-frequency proposed in [16] on the FPGA, and a parallel PQ measurement
sampling rate data capturing device, 215 samples per cycle, algorithm [17] on 16-core Epiphany. The accuracy of the raw
is proposed in [13], which requires a powerful desktop for real- measurements and the PQ measurements is compared with
time signal processing and frequency estimation. the measurements obtained from a high-end commercial
This article focuses on the development of a measure power analyzer. This is a first-of-its-kind effort in the literature,
ment platform with edge-computing capabilities for smart grids to the best of our knowledge, that explores the applicability of
based on a unique hardware developed by Adapteva, an open-source edge-computing platform under 5 W, with
advertised as a credit card-sized supercomputer, Parallella. three varieties of computing modules for a smart grid measurement platfor
This platform has three compute units: a Zynq SoC (an
industry-proven SoC), housing FPGA + Dual-core ARM Cortex- II. MULTIFUNCTIONAL IED HARDWARE DESIGN
A9 MPCore, clocked at 650 MHz, and a 16-core Epiphany co- A 5-W, 18-core credit card-sized microcomputer called
processor. Parallella is available in three variations, namely, Parallella-Desktop is used as the edge-computing module in
server version ($99), desktop version ($159), and an this article. The heart of this microcomputer is the Zynq-7010
embedded version ($249) consuming less than 5 W. SoC, which features a seven-series FPGA programmable
This heterogeneous resource-constrained computing logic. The FPGA features 28k logic cells, 2.1 Mb of block
architecture combines features of CPU, DSP, and ASSP on a RAM, 80 DSP slices, and 150 I/O pins. The cores of Epiphany
single board and makes the Parallella unique. This can exchange data with each other via a low-latency mesh
heterogeneous compute platform has been used as a network on chip [15]. The Epiphany architecture allows
compute node, accelerator, microserver, and server cluster in programming of each core using typical C code [18]. An
research applications covering image processing, video SRAM alongside each core is required to be managed,
filtering, encryption, AI, and machine learning [14]. considering the lack of cache. At 2 W, the Epiphany SoC
Parallella does not come with preprogrammed I/O interface provides promising power efficiency (performance/watt).
options for any external sensor integration. The internal ARM The Epiphany talks with the Zynq over e-Link implemented on
ADC is disabled on all versions of the Parallella. The default the FPGA. The Epiphany e-Link driver can be accessed from
FPGA binary file on Parallella does not allow an external the ARM's userspace over a memory-mapped AXI interface.
sensor interface to the FPGA GPIOs. The Epiphany co- The device connects to a 10-Mbps Ethernet backhaul.
processor can only be accessed from the ARM through a The device also has 1-GB SDRAM and 32 GB of onboard SD
proprietary e-Link protocol module implemented on the FPGA card storage.
[15]. Any configuration or modification on Parallella requires Fig. 1(a) and (b) shows the functional block diagram and
modification of the FPGA binary file. Modifying the FPGA the proposed implementation architecture on the Parallella.
binary of Parallella was not attempted before since it might Three voltage measurements for phase voltages, four current
impact the working of the Epiphany. This limited the application measurements for line currents, and neutral current are
of Parallella as a sensor node with edge-computing capabilities. provided using an AFE daughter board. The AFE and
An attempt is made in this article to harness the capabilities Parallella interact via a custom I/O board, which also provides
of Parallella as a powerful sensor node. the necessary hardware interfaces for making peripherals
A custom I/O board for the Parallella desktop and embedded compatible with the Zynq SoC. The developed IED using
versions is developed to expand the Parallella capabilities for Parallella and the I/O board is shown in Fig. 2(a) and (b),
peripheral sensor integration. The developed I/O board respectively. The IED provides eight high drive strength
provides high-speed, level-shifted GPIO interface to external GPIOs for the actuation of external circuit breakers or relays.
peripherals. One such peripheral, an analog front-end (AFE) The AFE can accommodate eight differential analog channels
daughterboard for analog measurements, using a synchronous for the IED. The clock for AFE is generated from the GPS reference. The d

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google

JOGLEKAR et al.: OPEN-SOURCE HETEROGENEOUS CONSTRAINED EDGE-COMPUTING PLATFORM 9003612

Fig. 1. IED design on Parallella. (a) Functional block diagram of the proposed IED. (b) Implementation architecture on Parallella.

Fig. 2. Developed IED using Parallella. (a) Prototype IED. (b) I/O board.

from the ADC move via the I/O board to the FPGA component arbitrator to send sampled data obtained from the FPGA-
of the Zynq SoC. The data are buffered in a 16-sample wide based ADC driver to the multicore co-processor. The Epiphany
first in–first out (FIFO) buffer from which different applications is used for data crunching applications, such as computation
can consume the data. of PQ on the measured signals. For want of space, security
The efficient utilization of the available compute mod ules algorithms implemented on Parallella are not discussed, and
is an essential step in edge-computing devices, such as the work of [20] can be referred to for this.
Parallella [19]. In this article, FPGA is proposed to be used
for sensing and compute-intensive real-time algorithms. A.I/ O Board
Hardware drivers for GPS (UART) and ADC (SPI) are ported The e-link Epiphany interface uses 48 GPIOs of the FPGA.
on the FPGA to meet the communication bandwidth Hence, only 24 external GPIOs on the desktop edition of the
requirements when the data are sampled at 32 kHz (provision Parallella are available for customization. However, these are
is made to use up to 64 kHz depending on the requirement). not exposed by the Parallella designers for external use,
Fig. 1(b) shows various modules added to the FPGA apart limiting the usability of Parallella as a sensor node.
from the default e-link interface. Due to the nature of FPGA, A custom I/O board is developed to provide a high-speed,
all modules get executed in parallel. The ARM processor is level-shifted GPIO interface to make the FPGA GPIOs
intended to communicate with the external world, file accessible for external peripherals, such as analog sensing
management on SD card, and security algorithms. The ARM alsoand acts as an
environmental sensors. This I/O board is one of the significant

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google

9003612 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021

contributions of this article. Pin mappings for custom GPIOs

were added in the Xilinx design constraints (XDC) file.
This file maps physical pins on the FPGA to signals that
interface with the peripheral driver IP. The electrical property
standards are also defined for each GPIO in this file.
The pin mappings for the Epiphany E-Link driver are also
present in the XDC file. Besides FPGA GPIOs, ARM GPIOs
can also be directly routed to the physical pins using the
extended multiplexed I/Os (EMIOs) routed on FPGA fabric.
The I/O board uses high-speed level converters to convert the
Parallella's 1.8-V level GPIOs to 3.3-/5-V level GPIOs.
Eight GPIOs are interfaced with a Darlington driver to improve
their current sourcing, sinking capability for connection with
external relays/circuit breakers. Additional interfaces for future
expandability have also been brought out on the custom I/O
board. For example, BOSCH XDK, The Sensor X-perience Fig. 3. Current and voltage channel schematics.
(environmental sensors), has been interfaced with the I2C
port of the ARM over the EMIOs. Other peripherals provided
by Parallella, such as Ethernet port, SD card slot, and USB
port, are kept intact. AXI addresses are assigned to custom
peripher als using the address editor in Vivado. AXI memory-
mapped peripherals, such as the ADC's SPI driver, are
declared in the device tree file located on the ARM's boot
partition. The entry in the device tree specifies the physical
address allocated to the peripheral and the interrupts (if any)
that the device will trigger. The entry also contains the
corresponding Linux kernel driver that will be loaded on boot. The FIFO full
AXI GPIO is mapped as a userspace IO framework (UIO)
device in the device tree. These modifications make the FIFO
full interrupt available to any userspace code on the ARM
processing subsystem. Fig. 4. ADC and GPS state diagrams. (a) ADC driver. (b) GPS driver.

B.Analog Front End

The Zynq's built-in two-channel 12-bit ADC is insufficient
for simultaneous sensing of seven channels. Hence, an eight-
channel AFE daughter board using a 16-bit syn chronous
ADC (ADS131E08) at a maximum sampling rate of 64 kHz
(currently set to 32 kHz) is developed. The AFE is presently
intended to measure voltages up to 440 V and measure
harmonics in the power. Hence, a voltage divider circuit is
used as it is widely recommended for high-speed data capture
up to 800 V by the ADC manufacturers. For using the potential
transformers with the AFE, suitable compensation techniques
need to be employed as these transformers have deviations
in their effective transformation ratio [21], [22].
A current transformer is used for current sensing since it is
robust and acceptable for harmonic magnitude measurement
[23]. The AFE board can be directly interfaced to a 1-/5-A CT
(0.2ÿ burden). Since the ADC is unipolar, a common-mode dc
bias is provided for the measurement of the bidirectional
quantities. This dc bias is derived from the ADC itself. The
ADC's clock is derived from the GPS signal to obtain
synchronized sampling. The schematics of the differential CT
secondary current and line voltage channels are shown in Fig.
3. The ADC on the AFE daughter board communicates over
an SPI interface with the FPGA. The ADC cannot communicate
directly with the Zynq's ARM processor as the ARM SPI
peripheral driver does not provide
the bandwidth needed when seven channels are sampled at
32 kHz. As a result, a custom ADC SPI driver IP is developed
for the FPGA. The custom driver IP has the necessary code
to configure the ADC, read, and buffer sampled data in
synchronization with the GPS time clock reference. Fig. 1(b)
shows the ADC SPI driver IP's internal subsystems developed
in the FPGA along with the peripheral interconnections. This
driver is realized using a state diagram shown in Fig. 4(a),
and the states are described as follows.
1) Default power-on state is S0. Set RESET = 0 on the
next clock edge after power-on.
2) In state S1, start the external clock at 2 MHz and then
set chip select (CS) = 1.
3) In state S2, the ADC registers are configured.
4) S2: Send stop data continuous (SDATAC) command
since the device wakes up in the read data continuous
(RDATAC) mode.
5) S2: Set sampling rate and set reference as 2.4 V and
internal amplifier operation for common-mode bias
voltage.
6) S2: Send START command to start conversions and
send RDATAC to put the device back in RDATAC mode.
7) In state S3, set dataFlag = 0, indicating that data is not
ready. As soon as DRDY becomes 1, the state is
changed to S4.

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google
JOGLEKAR et al.: OPEN-SOURCE HETEROGENEOUS CONSTRAINED EDGE-COMPUTING PLATFORM
8) In state S4, data for eight channels are received. Then, the
data flag is set to 1, indicating that data is ready.
After saving the data, state is changed to S3.
Nomenclature for Fig. 4(a): Capital is for command/signals sent
to or received from ADC, while small letters are for signals
generated within SPI driver to control the state machine.
To allow synchronization across ADCs, the ADC is configured to
accept an external clock signal. The internal ADC reference is
set to 2.4 V, and the common-mode bias VDD/2 = 1.5 V.
The differential ADC channels are set up with unity PGA gain.
C. Interface of ADC With GPS
The ADC (ADS131E08) used in AFE is an eight-channel
simultaneous-sampling sigma–delta ADC. The ADCs across
IEDs need to be synchronized to enable synchronous PMU
functionality. A mechanism to correct oscillator frequency drift for
a SAR ADC-based AFE was proposed in [24].
Sigma–delta ADCs are known to be less sensitive to clock jitter
compared to SAR ADCs. In a setup similar to [25], the ADC
master clock is generated using an FPGA-based custom clock
generator module that uses the GPS universal clock reference
for drift correction. If the START pulse and ADC clock signal are
synchronized with the edge of the PPS clock reference, sampling
across ADCs is guaranteed to be in sync as per the device
datasheet. The reference clock is used to trigger the start of
conversion and generate the ADC's external CLK signal. A
frequency multiplier block generates the ADC's external clock
reference of 2 MHz from the GPS PPS clock reference on the
FPGA.
D. Global Positioning System (GPS)
GPS device gives coordinated universal time (UTC) date and
time of the day information. Garmin GPS 18x LVC-5m, which
transmits this information using universal asynchronous receiver-
transmitter (UART) protocol, is used in this work This device also
gives measurement pulse output (called PPS) after reporting a
valid and accurate position fix for at least 4 s. The rising edge of
this pulse is aligned to the start of each GPS second within 1 ÿs,
which is highly accurate.
Before interfacing GPS with the PMU processor, the baud rate
for the UART communication is set to 38 400. For PMU device
functionality, date and time information is required.
Hence, the GPRMC output sequence from GPS output is made
available. Since the GPS device gives output at RS-232 level
and PMU processor accepts data at TTL level, RS232 to TTL
UART level converter, MAX-232 is used as a link between the
GPS device and the PMU processor. As in the case of ADC, an
equivalent GPS driver to acquire GPS data is developed.
Output sequence that needs to be decoded is as follows:
$GPRMC,1,2,3,4,5,6,7,8,9,
10,11,12 ÿ hhCRLF
where 1 contains UTC time in the hhmmss format and 9 gives
UTC date in the ddmmyy format.
The state diagram of the GPS driver implementation is
shown in Fig. 4(b), which can be summarized as follows.
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
9003612
Fig. 5. ADC output when connected to the same load.
1) Default power-on state is S0. When reset = 1 go to
state S1.
2) In state S1, wait for negative edge of Rx (or Tx from GPS
device). After that, reset dataReady and go to S2.
3) In state S2, check whether the first character is $ (=2A
hexadecimal). If “$” is received, then set $ = 1 and go to
S3; else, reset $ = 0 and return to state S1.
4) Extract UTC time and date information from the obtained
GPRMC sentence. After this, set dataReady = 1 and go to
state S1.
The receiver (Rx) of the GPS driver is tied to the transmitter
(Tx) of GPS device (through MAX-232). Every signal input to the
ADC driver and the GPS driver has been passed through signal
debouncers.
The performance of the simultaneous sampling on the IED
was evaluated in a similar way as in [9]. All the three phases of
the AFE are connected to a single-phase load powered by 50-Hz
power supply. The integer output values of the ADCs are shown
along with the maximum error among the signals in Fig. 5. The
plot shows ten cycles of power frequency waveform captured in
6400 data points, and hence, the sampling frequency would be
32 kHz. The error is found to be under 0.05% with respect to the
16-bit ADC.
A Hioki Power Analyzer PW6001 and the developed IED are
connected to the authors' lab input power supply for simultaneous
recording of the data for comparison purposes.
The Hioki Power Analyzer is set to sample at 50 kHz (only
available setting close to 32 kHz) with a bandwidth of 100 kHz.
Both the devices captured the dynamics of a load turn-on event
created in the lab. The corresponding voltage and current
waveforms from both the devices are shown in Fig. 6.
It can be observed from the figures that the recordings match
very closely with each other.
E.Data Pipeline
The ADC is configured to sample 7 channels at 32 kHz (640
samples per line cycle) with a 16-bit resolution. The data rate
can be calculated using the following equation:
DR = (r × fs × c)/1024/8 (1)
where DR is the data rate in kB/s, r is the bits per sample, fs is
the sampling rate, and c is the number of channels sampled.
Machine Translated by Google
9003612
Fig. 6. Comparison of recorded waveforms. (a) Voltage waveform compare
ison. (b) Current waveform comparison.
Accordingly, the data rate for the FDI is
DR = (16 × 32,000 × 7)/1024/8 = 437.5 kB/s.
The FPGA ADC driver IP provides a 16-sample wide FIFO, for
each channel. For a 16-sample wide FIFO, a PL fabric FIFO full
interrupt is raised at 2 kHz (32000/16). A dedicated ARM core
services this interrupt. The CPU affinity flag is set in the OS to
attach a single core to this interrupt. The interrupt service routine
transfers the data stored in the FIFO over a memory-mapped
AXI interface at a transfer rate of 2.4 GB/s.
Thus, the FIFO is flushed before the next sample set. Raw data
sampled at 32 kHz are compressed by a thread running on the
ARM core using a custom anomaly-aware compression
algorithm. The 32-kHz data stream is fed to the custom PMU IP
block within FPGA. The PMU IP block generates a phasor ready
interrupt that is processed by the ARM core. The calculated
phasors per channel are transferred from the FPGA PMU IP to
the ARM over a memory-mapped AXI interface.
The 32-kHz data stream is fed to the Epiphany co-processor
over the e-Link interface, with the ARM acting as an arbitrator.
The Epiphany is tasked with PQ measurement.
III. APPLICATIONS
Several power system-specific applications can be
implemented on the IED. This article describes three such
applications on each compute module: 1) anomaly-aware data
compression algorithm on ARM; 2) TEO-based PMU somewhat
rithm on FPGA; and 3) PQ measurement on the Epiphany
parallel processor. Here, the ARM processor also acts as a
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021
communication processor, which facilitates data transfer to an
external server.
A. Anomaly-Aware Compression Algorithm
The IED would have a throughput of 35 Mbps, as given in (2),
for the sampled raw values. Such a high data rate is formidable
and is not amenable to storage onboard or even transmission.
Therefore, it is desirable to subsample in an auto mated
fashion, keeping in mind the “frequency content” of the
underlying data. The proposed algorithm performs the required
adaptive sampling and bit packing needed while being feasible
in resource-constrained hardware. The algorithm prepares a bit
packing of the samples after filtering. It uses the number of bits
used in the bit-pack to measure the “informativeness” of the
samples. Highly informative samples are retained completely,
while less informative ones are further subsampled.
In effect, the algorithm detects anomalous behavior (such as
fault, PQ disturbances, or switching) and chooses between
lossless compression and lossy compression (obtained using
subsampling) adaptively, based on configurable thresholds.
This view allows the use of a single streaming data pipeline for
compression, fault-detection, and subsampling for different
operations, which would have required different features to be
computed in traditional signal processing. The compression
algorithm runs on the ARM processor and takes the channel
streams as input from the FPGA.
Formally, let x(t), t ÿ 0, denote the real-valued, continuous-
time power signal and x[i] = x(i/ fs), i ÿ N, be the corresponding
sampled sequence acquired at sampling frequency fs and
(2) represented using r-bits per sample. The consecutive samples
are grouped into blocks x(1) with block x(i) comprising, x(2) ,... ,
samples x[n(i ÿ1)+1],..., x[ni].
Thus, each block x consists of n signed integers taking values
in [ÿ(2r ÿ 1), 2r]. In the IED application, block size n = 16 and
resolution parameter r = 16 are used.
The proposed compression algorithm operates block by-
block and maps the block x(i) to a bit sequence Bi using a fixed
quantizer function Q : x ÿ Q(x). The quantizer Q can be a
variable-length quantizer that uses different lengths of Bis for
different is. Note that the length information itself must be
included in Bi for signal reconstruction. The goal is to reduce
the number of bits used per block while ensuring that it is
possible to construct an estimate ˆx[i] of the sampled sequence
x[i] from the bitstream B1, B2,... It is desirable to have FIFO
recovery, that is, the compressed bit sequences B1,..., Bi should
suffice to recover the estimate xˆ[i] of x[i]. The accuracy of
recovery is measured by the normalized mean-squared error
(NMSE) for recovering the sampled sequence up to time N that
is given by
No. 2
i=1(xˆ[i] ÿ x[i])
N
. (3)
2
i=1
x[i]
The performance of the algorithm at a fixed NMSE level will be
measured by the compression ratio (CR) defined as
rnT
CR = (4)
T
i=1 |bi|
Machine Translated by Google

JOGLEKAR et al.: OPEN-SOURCE HETEROGENEOUS CONSTRAINED EDGE-COMPUTING PLATFORM 9003612

TABLE I
MAXIMUM, MINIMUM, AND AVERAGE CR AND NMSE FOR TWO
DIFFERENT SETTINGS OF THRESHOLDS FOR LOSSY COMPRESSION
AND ANOMALY DETECTION WITH n = 16, m = 4, AND BUF = 40

Fig. 7. Anomaly-aware compression algorithm.

calculated over T blocks and |B| denotes the length of the bit
sequence B. NMSE is considered a standard metric in theory,
but it need not be an appropriate metric for error in power
applications. Indeed, it can be observed that the quality of
recovered waveform is much better than what is reflected by
the NMSE. A schematic summary of the proposed algorithm
is shown in Fig. 7, a preliminary version of the algorithm
appeared in [26], and the adaptation to the IED environment
is described in this work. The first step in the algorithm is to
pass each block of data x (comprising n samples) through a a high compression with n = 16, m = 4, and BUF = 40. For
linear filter. The filter used is an iterated filter where the output the medium case, ÿH = 11 for both the voltage and current
y = (y1,..., yn) is given by y1 = x[1] and y2 = x[2] ÿ x[1] and channels and ÿB = 7 for voltage channels and ÿB = 6 for the
for k = 3,.. ., n, yk = x[k] ÿ 2x[k ÿ 1] + x[k ÿ 2]. current channels are used. For the high case, ÿH = 15 and
The vector y is converted into a bit block B via a bit-packing ÿB = 14 are used for all the channels. It was observed that an
procedure in which each sample is stored using bits where is average CR of around 1.42 for current channels and 1.64 for
the maximum number of bits needed to represent any voltage channels could be obtained if all data have to be
sample within the block. The value of (represented using log2 retained losslessly. Table I shows that an average CR of up
r bits) is also included in B. An anomaly detector uses block to around 45 can be configured for waveform recording based
B and the corresponding to determine whether to label the on the accuracy requirements for the n, m, and BUF.
block as faulty or normal. If >ÿH , a fixed threshold, the block Figs. 8 and 9 show the heat map of the CR for a snapshot of
is labeled faulty, normal otherwise. Normal blocks are set to voltage and current channel for the two settings in the table.
be stored in a lossy fashion, but not immediately since it is Here, a higher CR is attained around the smooth part of the
necessary to retain a data cycle before and after the fault. signal and lower around the edges, illustrating adaptive
Toward this, the bit block stream B1, B2,..., BT resulting from compression. It was observed that the compressed data are
the input stream x(1) x(T) is passed ,...,to a FIFO queue of length captured without any loss and without overloading the network
BUF, which is set to the number of cycles intended to be bandwidth. The data collected and the computed data by the
retained around the detected anomaly. IED are streamed to the local server for storage and
Once the block x(i) is set to be stored in a lossy fashion, visualization. The user can access the streamed timestamped
the bits corresponding to the first sample in x(i) is retained data of the applications via a web browser.
and others are dropped. This procedure is repeated at a
higher level as well. Namely, if there are m consecutive blocks B. TEO-Based PMU Algorithm
with only the first sample retained, these first samples are Fig. 10 shows the block diagram of the TEO-based tech
passed through the same linear filter, and the output is used nique proposed in [16] implemented on the FPGA. As shown
to decide whether or not to retain all of the m values or just in the figure, the PMU IP block is fed with a 32-kHz raw data
the first one among them . In particular, if the number of bits stream. The measurement involves four main stages: 1) a
per sample resulting from this filter, >ÿB, is a fixed threshold, bandpass filter (BPF); 2) THEO; 3) low-pass filter (LPF); and
all m values are retained. Otherwise, all values, except the 4) moving average filter (MAF). Only a brief explanation of the
first one, are dropped. This process of downsampling the stages is discussed here due to space constraints. The raw
data first by n and then by m, if necessary, results in a hierarchical subsampling.
data signal is first passed through an adaptive BPF. The
In effect, the algorithm tests whether or not high frequencies adaptive BPF is based on a recursive discrete Fourier
are present and adjusts the sampling frequency accordingly. transform (RDFT). It was observed that updating the number
Reconstruction is performed by using splines to interpolate of samples N dynamically is not numerically stable in a
the dropped data points. The IED is connected to measure recursive floating-point implementation [27]. For practical
the author's laboratory input power supply. The measured implementation, N is fixed and calculated as N = fs/ f0, where
dataset comprises 5039 files of 1-s duration captured on fs and f0 are sampling and fundamental system frequency, respectively.
January 11, 2020, during daytime. The maximum, minimum, When the previously estimated frequency ˆf [n ÿ1] drifts from
and average CR and NMSE are calculated for all the 5039 fundamental ( f0), the phase and amplitude error are
files by varying the threshold settings ÿH and ÿB. Table I calculated as discussed in [28], and accordingly, compensation
provides CR and NMSE for two cases corresponding to a medium is and
done for the new sample x(n). Using the calculated amplitude, one

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google

9003612 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021

Fig. 8. Heat map of block-wise CR for a voltage channel. Top: medium

compression. Bottom: high compression. Fig. 9. Heat map of block-wise CR for a current channel. Top: medium
compression. Bottom: high compression.

has to normalize the instantaneous fundamental component of the

input signal. Kaiser [29] introduced the TEO concept to calculate the
energy needed to generate speech signals. TEO is a nonlinear
operator and can be used to estimate the energy of a sinusoidal
signal. The TEO block takes the normalized fundamental component
as input and outputs the frequency deviation from nominal. TEO is
given as
2
E[x(n)] = x (n) ÿ x(n ÿ 1) ÿ x(n + 1). (5)
Fig. 10. Block diagram of RDFT-TEO.
For a sinusoidal signal, x(n) = A cos(2ÿ( f/ fs)n + ÿ), the Teager
energy is given as
For a 50-Hz system, ÿcut ÿ 250 rad/s, since window size of BPF is
f
E x 1or (n) = sin2 2ÿ fs (6) one fundamental time period. To further filter the estimated output,
a MAF is used so as to bring the errors within the PMU standards
where x 1or (n) represents the normalized version of x1(n). It is worth [30]. Window size for this filter is taken as one fundamental period
noting that only three consecutive samples are required to find the of nominal frequency (50 Hz) and its transfer function is given by
frequency deviation from nominal f (n) [16] using TEO as shown in
the following:
1 1 ÿ zÿM 1 ÿ
u 2 or Hmaf (z) = (9)
x
1 (n) uÿx1 (n ÿ 1)x 1 (n + 1) ÿ S0 m zÿ1
f (n) = ÿ fs (7)
2ÿC0 where M is the filter order.
where S0 = sin(2ÿ f0/ fs) and C0 = cos(2ÿ f0/ fs). This first-order Implementation on Hardware: The RDFT-TEO algorithm is
infinite impulse response (IIR) LPF restricts the bandwidth of the implemented on the FPGA. Xilinx Vivado HLS v2014.3 tool chain
estimated amplitude normalized fundamental signal component. The was used to convert the “C” code into “HDL.” The fol lowing points
cutoff frequency of LPF is calculated using [16] need to be followed to ensure optimal resource utilization within
device limits [31], [32].

ÿcut ÿ 5/TRDFT. (8) 1) Use of float32 instead of float64.

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google
JOGLEKAR et al.: OPEN-SOURCE HETEROGENEOUS CONSTRAINED EDGE-COMPUTING PLATFORM
TABLE II
RESOURCE UTILIZATION FOR PMU ALGORITHM
Fig. 11. Voltage phasor output.
2) Use of directives such as #PRAGMA
Ex: #pragma HLS allocation instances = fadd limit = 1
operation.
3) Avoid use of dummy variables to optimally utilize
memory elements.
The final resource utilization of the TEO-PMU algorithm is
shown in Table II, where “SE” represents resource utilization
of the AXI compatible synchrophasor estimator IP and “Total”
represents the resource utilization of SE-IP along with SPI
ADC driver IP, GPS driver IP, AXI Interconnect, and Epiphany
driver IP. Resource utilization has been efficient, with room for
further expansion as given in Table II. The recorded output of
PMU is shown in Fig. 11. It can be observed from Fig. 11 that
the frequency estimation is relatively consistent with the grid
frequency of 50 Hz. Also, constant grid voltage amplitude and
minor variations in frequency can be observed.
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
9003612
Fig. 12. Real-time PMU-dashboard.
Since the frequency is lower than the nominal frequency, a
decreasing trend in the estimated phase can be observed.
The PMU data are also sent to the server for real-time display
and storage. A snapshot of the real-time PMU dashboard is
shown in Fig. 12. Voltage phasors, corresponding estimated
frequencies, and rate of change of frequencies for each
channel are displayed.
C. Measurement of PQ
PQ refers to a wide variety of electromagnetic phenomena
that characterize the voltage and current at a given time and
a given location on the power system [33]. In this work, the
harmonic and interharmonic content of the captured voltage
and current waveforms is calculated. The PQ parameters,
magnitude, phase, crest factor, total harmonic distortion
(THD), and total demand distortion (TDD) are used to define
the harmonic content. The measurement time interval for
calculating these parameters is a ten-cycle time interval for a
50-Hz power system as per the STD IEC 61000-4-30 [33].
For power system, applications up to the 50th harmonic (ie,
2.5 kHz) would need to be captured [17]. Hence, the captured
Machine Translated by Google

9003612 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021

Fig. 13. PQ tasks on Epiphany.

waveform is downsampled to 6.4 kHz by dropping samples.

Considering the nominal case, ten cycles at 6.4 kHz would
lead to 1280 samples per signal. The nearest 2 power, which
can capture up to 50th harmonic, would be 1024; Hence,
another resampling stage is achieved by using the Newton
divided difference interpolation. The required antialiasing
filter is applied at the interpolation stage before passing
through a Fourier transform. Considering the frequency
resolution, the FFT would be the suitable choice since the
computation complexity would be O(n log n), where n is the
number of data points [34].
Usage of Epiphany: The measurement of PQ is divided
into four tasks, as shown in Fig. 13. The 16-core Epiphany
co-processor, which is fabricated as a 4 × 4 mesh, is used to Fig. 14. THD comparison of recorded waveforms. (a) Voltage THD
calculate the PQ parameters by efficiently mapping these comparison. (b) Current THD comparison.
tasks to one or more cores called workgroups (W1–W4). The
row and column indices form the core number. Fig. 13 shows TABLE III
the cores grouped with other adjacent cores in the same row; EPIPHANY USAGE
this reduces the core to core communication time; a message
is passed along the row and then across the column [18].
The first task, namely, decimation and channel segregation,
is assigned to workgroup W1, having the core (2, 0). The
second task, interpolation and buffering for ten cycles data,
is assigned to workgroup W2 consisting of cores (2, 1) and
(2, 2). One core processes the voltage and another the
current; the core processing the voltages acts as master. in each Epiphany, space available for instruction and data is
The antialiasing filter is implemented at 5.12 kHz, which only 32 kB [15]. Thus, binary exchange is used for FFT
corresponds to the sampling interval for 1024-point FFT. deployment. The FFT workgroup W3 writes back the results
Since 200 ms of data from seven channels lead to nearly 35- to another block in the 32-MB shared space. Apart from FFT,
kB block, this is stored in the 32-MB shared space. However, W3 computes the true RMS and max value per channel
the segregation of the cores into workgroups forces data required for the indices calculation.
sharing only from the shared space or host [18]. This leads The PQ parameters (indices) are calculated in the work
to the utilization of flag-based handshake between the group having the core (2, 3). The ARM can access the
adjacent workgroups and host for process management. computed indices from W4 and sends them to a server. The
A parallel FFT algorithm is used in the Epiphany, in which Epiphany SDK [18] coding framework allows different codes
the data are methodically divided among the workgroup W3 for each workgroup, keeping a smaller size code for each
consisting of processors (0, 0) to (1, 3). The binary exchange task rather than a single code enforced on all groups. The
algorithm [34] is one such parallel FFT method based on the resource utilization and average run time for each task are
Cooley–Tukey algorithm and can be deployed on any 2x shown in Table III. The timing is done on the data captured
processors. Alternatively, the 2-D transpose FFT [34] can be for a 2-s duration by the IED. It can be observed that the
used, but the number of processors is limited to even powers computation time (including communication time) is within
of 2. Since preprocessing and other tasks are to be the output interval of each workgroup. Each image is within
accommodated in the Epiphany cores, the maximum cores the 32-kB size, including the data space.
that can be used for 2-D transpose are 4, whereas for the Keeping the Hioki Power Analyzer and proposed device in
binary exchange, it can be 8. Hence, the data space used by parallel, the THD and FFT spectrum from both the devices
2-D transpose is twice that of binary exchange. However, are recorded. The power analyzer was set to use a rectangular

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
Machine Translated by Google
JOGLEKAR et al.: OPEN-SOURCE HETEROGENEOUS CONSTRAINED EDGE-COMPUTING PLATFORM
Fig. 15. Real-time PQ-dashboard (VA shown).
window, and the time window of FFT was kept at 200 ms,
which is the same for the Parallella device. Fig. 14 shows the
voltage and current harmonic spectrum given by both the
devices. The parallel has reported 2.36% and 24.87% THD in
voltage and current, respectively. The power analyzer has
reported 2.28% and 24.64% in voltage and current,
respectively. This accuracy is deemed sufficient for research
pur poses. Fig. 15 shows the real-time PQ-dashboard showing
max values, peak values, crest factor, K-factor, THD, and TDD.
The total hardware cost of the proposed IED is approximately
INR. 60,000 ($760 USD). Since Parallella is an open platform
and incorporates an industrial Zynq platform at <5-W power, it
makes an excellent choice for even commercial product
development at an affordable cost.
IV. CONCLUSION
This article introduced a low-power heterogeneous edge
computing platform called Parallella as a measurement
platform for smart grid research. A custom FPGA binary file
and an I/O board development are proposed to harness the
capabilities of Parallella as an edge-computing sensor node.
The approaches for enabling access to Zynq I/Os without loss
of existing functionalities and allowing the platform to integrate
with the AFE and GPS device are detailed. The ARM has
been used for compression and communication, FPGA is used
for the peripheral interface, raw data capture, and phasor
estimation, and Epiphany co-processors are used for PQ
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
9003612
analysis. The FIFO buffer implemented provides the separation
of AFE and the computation functions. Hence, applications
can be built without much regard to the measurement system.
The salient features of the proposed platform are given in the
following.
1) Availability of ARM, FPGA, and multicore processors at
very low cost (ÿ$760) and low power.
2) The researcher gets hands-on exposure to all the com
pute modules at one go while designing smart grid
sensors and edge-computing applications.
3) The platform comes with an inbuilt Ethernet
communication module, which enables the development
of various IoT protocols and security features relevant
for smart grid sensing.
4) Availability of high sampling rate (32 and 64 kHz) data,
ability to connect a wide variety of sensors (such as
environmental sensors) and GPS synchronization
enables the development of wide variety of data-driven
applications in smart grids.
5) The modified FPGA binary provides a FIFO buffer that
allows the application to decide the rate of data
consumption.
6) Availability of full-fledged Linux distribution on ARM, MPI,
python, and openMP for epiphany provides see satile
programming environment for niche applications
development.
The proposed hardware can be extended as an IEC 61850
com pliant merging unit. The phasor measurement module
can be extended to communicate as per C37.118.2
Synchrophasor Data Transfer Protocol. The PQ analysis can
be extended to include flicker, unbalance, and event detection
to provide features of a complete PQ analyzer.
REFERENCES
[1] M. Caprolu, R. Di Pietro, F. Lombardi, and S. Raponi, “Edge computing
perspectives: Architectures, technologies, and open security issues,” in
Proc. IEEE Int. Conf. Edge Comput. (EDGE), Jul. 2019, p. 116–123.
[2] GP Colnago, JLF Vieira, GCD Sousa, and JR Macedo, “Real time power
quality monitoring system according to new Brazilian regulation policies,”
in Proc. 14th Int. Conf. Harmon. which. Power (ICHQP), Sep. 2010, p. 1–
8.
[3] H. Maass et al., “First evaluation results using the new electrical data
recorder for power grid analysis,” IEEE Trans. Instrument Meas., vol. 62,
no. 9, p. 2384–2390, Sep. 2013.
[4] DM Laverty, RJ Best, P. Brogan, I. Al Khatib, L. Vanfretti, and DJ Morrow,
“The OpenPMU platform for open-source phasor measures,” IEEE
Trans. Instrument Meas., vol. 62, no. 4, p. 701–709, Apr. 2013.
[5] P. Castello, P. Ferrari, A. Flammini, C. Muscas, and S. Rinaldi, “A new
IED with PMU functionalities for electrical substations,” IEEE Trans.
Instrument Meas., vol. 62, no. 12, p. 3209–3217, Dec. 2013.
[6] T. Atalik et al., “Multipurpose platform for power system monitoring and
analysis with sample grid applications,” IEEE Trans. Instrument Meas.,
vol. 63, no. 3, p. 566–582, Mar. 2014.
[7] H. Maaß et al., “Data processing of high-rate low-voltage distribution grid
recordings for smart grid monitoring and analysis,” EURASIP J.
Adv. Signal Process., vol. 2015, no. 1 p. 14, Feb. 2015.
[8] LRM Silva et al., “Gapless power-quality disturbance recorder,” IEEE
Trans. Power Del., vol. 32, no. 2, p. 862–871, Apr. 2017.
[9] X. Zhao, DM Laverty, A. McKernan, DJ Morrow, K. McLaughlin, and S.
Sezer, “GPS-disciplined analog-to-digital converter for phasor
measurement applications,” IEEE Trans . Instrument Meas., vol. 66, no.
9, p. 2349–2357, Sep 2017.
[10] P. Romano, M. Paolone, T. Chau, B. Jeppesen, and E. Ahmed, “A high
performance, low-cost PMU prototype for distribution networks based on
FPGA,” in Proc . IEEE Manchester PowerTech, Jun. 2017, pp. 1–6.
Machine Translated by Google
9003612 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 70, 2021
[11] D. Schofield, F. Gonzalez-Longatt, and D. Bogdanov, “Design and
implementation of a low-cost phasor measurement unit: A comprehensive
review,” in Proc . 7th Balkan Conf. Lighting (BalkanLight), Sep. 2018, p.
1–6.
[12] I. Jahn, F. Hohn, G. Chaffey, and S. Norrga, “An open-source protection
IED for research and education in multiterminal HVDC grids,” IEEE Trans.
Power Syst., vol. 35, no. 4, p. 2949–2958, Jul 2020.
[13] J.-P. Kitzig, S. Schlaghecke, and G. Bumiller, “Power quality measurement
system with PMU functionality based on interpolated sampling,” IEEE
Trans. Instrument Meas., vol. 68, no. 4, p. 1014–1025, Apr. 2019.
[14] Parallella Publications. Accessed: March 20, 2021. [Online]. Available:
https://www.parallella.org/publications/
[15] Epiphany Archit. Reference. Accessed: Jan. 10, 2020. [Online].
Available: https://www.adapteva.com/docs/epiphany_arch_ref.pdf
[16] S. Reza, M. Ciobotaru, and VG Agelidis, “Single-phase grid volt age
frequency estimation using teager energy operator-based tech nique, ”
IEEE J. Emerg. Sel. Topics Power Electron., vol. 3) No. 4, p. 1218–1227,
Dec. 2015.
[17] IEEE Recommended Practice for Monitoring Electric Power Quality, IEEE
Standard 1159-2009 (Revision of IEEE Std 1159-1995), Jun. 2009, pp. 1–
94.
[18] Epiphany SDK Reference. Accessed: Jan. 10, 2020. [Online]. Available:
https://adapteva.com/docs/epiphany_sdk_ref.pdf
[19] C. Jiang, X. Cheng, H. Gao, X. Zhou, and J. Wan, “Toward computing
offloading in edge computing: A survey,” IEEE Access, vol. 7, p. 131543–
131558, 2019.
[20] BP Robert, PA Babu, N. Kashyap, and G. Gurrala, “Practical approaches
toward securing edge devices in smart grid,” in Proc. 8th Int. Conf. Power
Syst. (ICPS), Dec. 2019, p. 1–6.
[21] H. Seljeseth, EA Saethre, T. Ohnstad, and I. Lien, “Voltage transformer
frequency response. Measuring harmonics in Norwegian 300 kV and 132
kV power systems,” in Proc. 8th Int. Conf. Harmon. which. Power., vol. 2,
1998, p. 820–824.
[22] M. Klatt, J. Meyer, M. Elst, and P. Schegner, “Frequency responses of MV
voltage transformers in the range of 50 Hz to 10 kHz,” in Proc . 14th Int.
Conf. Harmon. which. Power (ICHQP), Sep. 2010, p. 1–6. [On-line].
Available: http://ieeexplore.ieee.org/document/5625484/
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 20,2021 at 02:38:57 UTC from IEEE Xplore. Restrictions apply.
[23] AE Emanuel and JA Orr, “Current harmonics measurement by means of
current transformers,” IEEE Trans. Power Del., vol. 22, no. 3, p. 1318–
1325, Jul. 2007.
[24] W. Yao et al., “A novel method for phasor measurement unit sampling time
error compensation,” IEEE Trans. Smart Grid, vol. 9, no. 2, p. 1063–1072,
Mar. 2018.
[25] Y. Kononov, M. Ospishchev, and V. Rudnev, “Current and voltage
synchronous measurements using NI cRIO system with sigma-delta ADC
NI 92xx and SEA GPS modules,” in Proc . 20th IMEKO World Symp.,
2014, p. 25–30.
[26] KR Sahasranand, B. Rout, A. Joglekar, G. Gurrala, and H. Tyagi, “Fault-
aware compression for high sampling rate data acquisition in smart grids,”
in Proc. 10th ACM Int. Conf. Future Energy Syst.
New York, NY, USA: ACM, Jun. 2019, p. 422–424.
[27] HA Darwish and M. Fikri, “Practical considerations for recursive DFT
implementation in numerical relays,” IEEE Trans. Power Del., vol. 22, no.
1, p. 42–49, Jan. 2007.
[28] M. Wang and Y. Sun, “A practical method to improve phasor and power
measurement accuracy of DFT algorithm,” IEEE Trans. Power Del., vol.
21, no. 3, p. 1054–1062, Jul. 2006.
[29] JF Kaiser, “On a simple algorithm to calculate the 'energy' of a signal,” in
Proc. Int. Conf. Acoust., Speech, Signal Process., vol. 1, 1990, p. 381–
384.
[30] IEEE Standard for Synchrophasor Measurements for Power Systems—
Amendment 1: Modification of Selected Performance Requirements, IEEE
Standard C37.118.1a-2014 (Amendment to IEEE Std C37.118.1-2011),
Apr. 2014, pp. 1–25.
[31] Xilinx. Vivado Design Suite User Guide: High-Level Synthesis.
Accessed: Jan. 10, 2020. [Online]. Available: https://www.xilinx.com/
support/documentation/sw_manuals/xilinx2014_1/ug902-vivado-high-
level-synthesis.pdf
[32] Xilinx. Vivado Design Suite Tutorial: High-Level Synthesis.
Accessed: Jan. 10, 2020. [Online]. Available: https://www.xilinx.com/
support/documentation/sw_manuals/xilinx2014_1/ug871-vivado-high-
level-synthesis-tutorial.pdf
[33] Electromagnetic Compatibility (EMC)—Part 4-30: Testing and Measure
ment Techniques—Power Quality Measurement Methods, document IEC
61000 4-30 (2003), 2003.
[34] A. Grama, V. Kumar, A. Gupta, and G. Karypis, Introduction to Parallel
Computing. London, UK: Pearson, 2003.