DATA ANALYTICS IN SMART GRID

Synopsis submitted to School of Business for the partial fulfilment of the

requirements of the course
MBA (POWER MANAGEMENT)
2019-21
Submitted By:
Akanksha Negi
SAP ID: 500077029

Under the Guidance of

Prof. Avishek Ghosal
MBA(PM)

SCHOOL OF BUSINESS

UNIVERSITY OF PETROLEUM AND ENERGY STUDIES,

KANDOLI CAMPUS, DEHRADUN - 248007

1
 Student Declaration

I hereby, declare that this submission is my own work and that, to the best of my knowledge and
belief, it contains no material previously published or written by another person nor material which
has been accepted for the award of any other degree or diploma of the university or other institute
of higher learning, except where due acknowledgment has been made in the text

Akanksha Negi
SAP ID: 500077029
Roll number: R130219018

2
 ACKNOWLEDGMENT

This is to acknowledge the help and support of everyone who have contributed to the success of
this dissertation work. I extend my sincere gratitude to Prof. Avishek Ghosal my mentors, who
shared their insights on the topic, for their aspiring guidance, invaluably constructive criticisms
and friendly advices. I am truly grateful to them for granting me an opportunity to undertake my
dissertation under them.

I would like to convey my thanking note to Prof. (Dr.) Anil Kumar, Dr. Mohammad Yaqoot,
Power Management, Department of Energy Management, School of Business, UPES, who spared
his valuable time to see the progress of my work.

I owe a debt of gratitude to all my UPES alumni who took their time out of their busy schedule
to guide me and to share their industry experience relevant to the research.

I am highly indebted to the esteemed Institution School of Business, University of Petroleum and
Energy Studies and intensely to Department of Energy Management for presenting a glorious
opportunity that contribute to my betterment as a Power Management student.

Lastly, I extend my heartiest thanks to my friends and family who have constantly encouraged
and blessed me so as to enable me to undergo the study successfully.

3
 Certificate of Faculty Guide

This is to certify that the dissertation report entitled “Data Analytics in Smart Grid” submitted

by Akanksha Negi to UPES for partial fulfilment of requirements for Master of Business

Administration Power Management is a Bonafede record of the work carried out by her under my

supervision and guidance. The content of the report, in full or parts have not been submitted to any

other Institute or University for the award of any other degree or diploma.

Signature of Faculty mentors

Prof. Avishek Ghosal,
Professor, Power Management
Department of Energy Management

4
 TABLE OF CONTENTS

S. No TITLE PAGE NO

1 INTRODUCTION 5

NEED FOR THE WORK/

2 7
JUSTIFICATION OF PROJECT

2. BUSINESS PROBLEM 8

3. LITERATURE REVIEW 9

4. RESEARCH GAP 16

5. RESEARCH PROBLEM 16

RESEARCH QUESTION &

6. 17
OBJECTIVES

RESEARCH DESIGN &

7. 18
METHODOLOGY

8. FINDINGS & DISCUSSIONS 20

9. RECOMMENDATION 36

9. CONCLUSION 38

10. REFERENCES 39

5
 INTRODUCTION
The bidirectional progression of power and data is a fundamental field in the smart grid. With the
developing power supply from more limited size, decentralized generators, i.e., wind ranches and
private roof photovoltaics (PV) boards, and microgrids, progressed sensor and metering
innovations with coordinated security conventions take into account imagined progressed
highlights of the smart grid, for example, request reaction, self-governing control, self-
recuperating and self-design. Basically, the smart grid gives huge enhancements to customary
power frameworks that incorporate six fundamental structure blocks, to be specific, organization,
client, equipment, programming, workers and information. These qualities make it trying for
customary investigation, yet ideal for the use of man-made consciousness, AI methods and huge
information analytics. In this paper, we will utilize the term Big Data Analytics (BDA) to allude
to the aggregate information analytics, AI and AI. The goal of BDA is to examine the huge
volumes of information created by different segments in the smart grid, and change the information
into significant data sources like examples of activity, alert patterns, shortcoming location, and
control orders. For instance, progressed AI applications for circulation transformers dissect the
information accumulated continuously for every transformer. The result of these learning
applications may distinguish some working patterns prompting disappointment examples of these
gadgets and help expect future disappointments, and therefore, give opportune and exact
experiences to prescient upkeep. Exploration endeavours in smart grid organizations have zeroed
in on cutting edge metering foundation (AMI), like smart meters, correspondence, data, control
and energy the board frameworks for utilities and customer-based gear (e.g., smart home energy
regulators and building checking frameworks). Besides, other application zones incorporate the
mix of robotization, control and constant checking of cutting-edge sensors and observing gear.
This can be cultivated with field gadgets, for example, phasor estimation units and clever
electronic gadgets (IED), at the transmission level and robotized feeder switches, and organization
assurance transfers, voltage controllers, and capacitor regulators at the conveyance level. These
activities mean to upgrade power framework execution and diagnostics that will prompt expense
decrease. A critical part of the smart gadgets being conveyed is identified with the monstrous
rollout of smart meters at present occurring in numerous nations. The quantity of smart meter
readings for a huge service organization is required to ascend from 24 million per year to 220
million every day. Roughly 22GB of smart meter information is being produced by 2 million
clients each day. Expecting that an application requires information assortment in 15-minute spans,
1 million gadgets would bring about 35.04 billion information passages with a complete volume
of 2920 Tb each year. The other segment of smart grid gadgets identifies with bleeding edge
network gadgets, for example, IEDs being introduced in power framework organizations to screen
key organization boundaries and generation and utilization continuously, control power streams,
trade data with one another and have neighbourhood dynamic ability. Customary methodologies
of information investigation are lacking to adapt to the high volume and recurrence of information
produced inside the smart grid worldview by different circulated sources. This makes advancement
and smart administration testing and computationally serious. Information is produced by various

6
heterogeneous sources including sensors, IEDs, smart meters, smart machines, circulation
computerization information, outsider information, resource the board information and climate
station information assuming an undeniably significant part for overseeing irregular DERs.
Information should be changed into significant bits of knowledge by applying high volume
information the executives and progressed analytics (i.e., BDA). Basically, BDA addresses
progressed analytics, for example, prescient analytics, information mining, measurable
investigation, AI and AI procedures, which work on huge informational collections having at least
one highlights of huge information. Nearby and disseminated control models can give
arrangements that can diminish the information transmission load and computational assets
needed, rather than midway controlled dynamic. BDA procedures can give arrangements as the
intricacy of the power framework keeps on developing. It may not be clear to most energy
shoppers, yet enormous changes are going on in the utility business – development fuelled by the
Internet of Things (IoT) and smart grid analytics. Think about these models:

• Smart network gadgets trigger activities that limit or stay away from power blackouts.
• Smart home apparatuses and HVAC frameworks keep you agreeable at lower cost, with no
human mediation.
• Smart power generation frameworks advance expense and productivity across a different
arrangement of renewables, microgrids, batteries and customary sources.

It's another day in an industry that hasn't seen a lot of significant development up to this point. The
basic engineering of the utility grid hasn't shifted a lot over the direction of 100 years. Truly,
service organizations created power in one focal area and afterward utilized their own committed
grid to convey power to clients. That model is evolving. The upcoming smart grid will be a group
of stars of numerous generation sources cooperating, conveying energy in different bearings –
including from the client back to the utility ("prosumers"). It will be reshaped by advancements,
for example, minimal expense battery stockpiling, housetop photovoltaics and microgrids that can
work in grid-associated or island mode, and IoT availability of smart gadgets. These new advances
not just convey more abilities for energy suppliers to take advantage of new power gen sources,
they likewise give more information. Smart grid analytics is the use of cutting-edge analytics
systems to the information – including prescient and prescriptive analytics, anticipating and
improvement. The chances for smart grid analytics are growing in light of the fact that there's
dramatically more information accessible to foster insightful models. Smart grid enormous
information analytics is promising to stir up an industry not known for its mechanical
developments: Utilities. There is a developing cover among utilities and information, with
information sensors and other gear being incorporated into the arrangement of utility assets.
Energy organizations are utilizing smart grid analytics to gauge different factors, similar to the
measure of energy appropriated from smart organization triggers. Smart information analytics is,
accordingly, going to gigantically affect how we live (in the event that it hasn't as of now).

7
 NEED FOR THE WORK/ JUSTIFICATION OF PROJECT

The smart grid assembles information from assorted sources and stores it to be consumable by
examination. Overseeing smart grids to give smart energy requires progressed AI procedures to
gather precise data in a mechanized style, computerize dynamic and control occasions in an ideal
way at both the neighbourhood and framework wide level. Significant advancement has been made
for utilizing field information gained from smart gadgets mounted in substations, feeders, and
various data sets and models across the utility venture. There are a few wellsprings of information
in smart grids on business sectors, hardware, topography and force framework information which
can be utilized to foresee states, give situational mindfulness, dissect security, distinguish blames
and give pre-emptive guidance. Hence, investigation (containing BDA, ML and AI) has a huge
part to make the grid more shrewd, proficient and gainful. Investigation can be applied to flag,
occasion, state, designing activities, and client examination, in entirety empowering undeniable
level and definite bits of knowledge into grid situational mindfulness. There are a few kinds of
examination models, in particular distinct, indicative, prescient, and prescriptive models (review
Fig. 3). These can be applied for the smart grid, for example, enlightening models depict client
practices for request reaction programs. Analytic models are utilized to comprehend explicit client
practices and break down their force related choices. Each kind of model can give significant
contribution to make models that anticipate future client choices and thus, power needs. At last,
prescriptive models can give undeniable level investigation to impact smart grid promoting,
commitment techniques and dynamic. Force frameworks are needed to advance for dynamic and
adaptable cooperation with shoppers taking an interest in the power markets, LV control
computerization, appropriation the executive’s framework (DMS) combination, microgrid control
and adjusting, proactive shortcoming ID, self-recuperating and asset streamlining. Smart grid
frameworks are getting progressively perplexing and interconnected, displaying qualities of an
"arrangement of frameworks". As clarified above, energy frameworks need to advance to represent
circulated power age and the unique cycles of interest the board, load control and energy
stockpiling the executives. The energy framework is as of now encountering critical changes,
because of changes in administrative systems and approaches that identify with supportability.
This has brought about huge development in the volume, assortment and speed of information, a
critical expansion in partner number and variety, yet additionally giving new business freedoms to
improved financial aspects and unwavering quality. The requirement for information examination
and novel innovations is applicable to each partner of the energy framework, including the
framework administrator, market administrator, controller, specialist co-op, buyers, transmission
and dissemination framework administrators and specialist co-ops and generators.

8
 BUSINESS PROBLEM

With tremendous amount of data pouring in from wide number of sources and being used for a
few Smart Grid applications, there is a developing concern in regards to data protection and
security there are unmistakable examination headings worried to use the wide scope of
heterogenous data with new viable design and stages with a reasonable accentuation on continuous
observing and control. More proficient data scientific techniques are being proposed which
guarantee interoperability among different segments of Smart Grid and outfitted with powerful
representation. Finally, to the make the future smart grid a secure, reliable cyber physical system,
a lot of research are being carried out for developing robust standards and regulatory frameworks.

9
 LITERATURE REVIEW

1. Title of the Paper: THE AUTONOMOUS GRID: Machine Learning and IoT for Utilities
Author: Zpryme and SAS
Publication Year: April 2016
Objectives:
• How utilities are utilizing IoT and AI today, just as their arrangements for what's to come
• The utility industry's comprehension of IoT and AI
• Recommendations for utilities to improve their capacities to embrace these advances for a
bigger scope
Findings:
• Utilities concur that both IoT and AI are basic for their associations, yet they have a superior
arrangement and are bound to utilize IoT than AI.
• More than 55% as of now use IoT for metering/meter data the board (MDM), and 31% are
now utilizing AI for this space, as well.
• Network security and data protection are major IoT worries for utilities. Spaces of least concern
are absence of expertise and adequate spending plan.
• The top advantages related with IoT are bound to be client confronting, for example, client
care and energy effectiveness, while the advantages named for AI are more grid-situated,
including regions, for example, administration reclamation and network protection.

2. Title of the Paper: Big Data management in smart grid: (concepts, requirements and
implementation)
Authors: Houda Daki, Abdelhak Aqqal, Asmaa El Hannani, Abdelfattah Haidine and Aziz Dahbi
Publication Year: 28 April 2017
Objectives: This paper provides an overview of data management for smart grids, summarizes the
added value of Big Data technology for this type of data, and discusses the technological criteria,
tools, and key steps for implementing Big Data solutions in the context of smart grids.
Findings: This paper summarizes the Big Data technologies and processes that can be used to
handle smart grid requirements such as processing, storage, and even visualization, and provides
an overview of the possibilities, principles, and challenges of data management in smart grids. We
also provided implementation and deployment measures, resources, and technical specifications
for Big Data technologies for smart grids in order to have an efficient and scalable data
management.

3. Title of the Paper: Big data analytics in smart grids: a review

Authors: Tao Huang, Yang Zhang and Ettore Francesco Bompard
Publication Year: 13 August 2018
Objectives: The aim of this paper is to incorporate big data analytics and their applications in
smart grids. The fundamental principles and procedures of common data analytics for general

10
problems are also covered. One of the goals of this paper is to identify advanced applications of
various data analytics in smart grids.
Findings: The key focus of this paper is on advanced applications of various data analytics in
smart grids. Many advantages can be added to the current power grid and improved customer
service as well as social welfare in the age of big data by interacting with massive amounts of data
from energy networks, meteorological information systems, geographical information systems,
and other sources. Many challenges, such as strategies, understanding, synergies, and so on, must
be overcome in order to advance the applications of big data analytics in real smart grids.

4. Title of the Paper: Big data analytics in smart grids: state-of-the art, challenges, opportunities,
and future directions
Authors: Milos Manic Bishnu P. Bhattarai, Rob Hovsapian, Sumit Paudyal , Yusheng Luo ,
Manish Mohanpurkar , Kwok Cheung4, Reinaldo Tonkoski , Xiaping Zhang , Kurt S. Myers , Rui
Zhang , Power Zhao, Song Zhang ,
Publication Year: 1st March 2019
Objectives: To include a thorough state-of-the-art analysis of big data analytics and its
applications in power grids, as well as to identify problems and opportunities from the perspectives
of utilities, industry, and science. The paper examines research limitations and offers
recommendations for potential research to incorporate big data analytics into power system
planning and operational frameworks.
Findings: This paper offers general guidelines for utilities to make the right investment in the
adoption of big data analytics by unveiling interdependencies among critical infrastructures and
operations. Detailed information for utilities looking to apply big data analytics and insights on
how utilities can enhance revenue streams and bring disruptive innovation are also some findings
here.

5. Title of the Paper: Big data analytics: a survey

Authors: Chun‑Wei Tsai1, Chin‑Feng Lai2, Han‑Chieh Chao1,3,4 and Athanasios V. Vasilakos5
Publication Year: 1st October 2015
Objectives: This paper focuses on how to create a high-performance platform for analyzing big
data efficiently and how to build an effective mining algorithm for extracting useful information
from big data.
Findings: This paper begins with a short overview of data analytics before moving on to
discussions of big data analytics. For the next phase in big data analytics, some relevant open
issues and future research directions are also discussed.

6. Title of the Paper: Big data analytics strategies for the smart grid
Authors: Stimmel Carol L
Publication Year: 2015

11
Objectives: This book explores the challenges of applying big data technology and methods to the
essential infrastructure that makes up the electrical utility grid, including Big Data cybersecurity.
It provides industry stakeholders with a thorough understanding of the power delivery market's
engineering, service, and customer domains.
Findings: This book focuses on the challenges of applying big data technology and techniques to
the electrical utility grid's vital infrastructure, including Big Data cybersecurity. It gives industry
stakeholders a comprehensive understanding of the power delivery market's engineering, sector,
and customer domains.

7. Title of the Paper: Big Data issues and opportunities for electric utilities
Authors: Beth-Anne , Schuelke-Leech BetsyBarry , MatteoMuratori , B.J.Yurkovich
Publication Year: 25 August 2015
Objectives: To examine the challenges and opportunities associated with the use of Big Data by
electric utilities.
Findings: This paper examines the challenges and opportunities associated with the use of Big
Data by electric utilities. Electric utilities are cautious, highly regulated, and focused on system
stability as well as overall profitability. Constraints in terms of technology, economics, institutions,
and policy must all be tackled. They empirically evaluate whether these issues and opportunities
are part of the conversations regarding electric utilities with federal policymakers after analyzing
these issues and opportunities. Although there are many conversations about electric utilities in
general, conversations about data in the sense of electric utilities are relatively uncommon,
according to the findings.
8. Title of the Paper: A survey on platforms for big data analytics
Authors: Dilpreet Singh and Chandan K Reddy
Publication Year: 09 October 2014
Objectives: To include an in-depth review of the various solutions available for conducting big
data analytics, as well as to evaluate the benefits and disadvantages of each platform based on
various metrics such as scalability, real-time processing, data I/O rate, data size supported, fault
tolerance and iterative task support.
Findings: This paper examines the different data processing systems currently available and
addresses their benefits and disadvantages. Via star-based ratings, a detailed comparison of various
platforms has been made based on some of the most critical characteristics (such as scalability and
real-time processing).

9. Title of the Paper: Energy Big Data Analytics and Security: Challenges and Opportunities
Authors: Jiankun Hu and Athanasios V. Vasilakos
Publication Year: SEPTEMBER 2016

12
Objectives: To conduct a survey of new findings and innovations in the field of big energy data
analytics and security, and to discuss them. To express the fascinating relationships between
various variables in the field, several taxonomies have been proposed.
Findings: A detailed coverage of energy big data analytics and protection/privacy ranging from
energy big data architecture, intelligence applications, cryptosystem design, device security
assessment, cyber intrusion detection, safe financial intelligent application, and electricity theft
has been provided. This paper proposed an energy big data taxonomy to help better understand the
complex and fascinating relationships between different elements, security concerns, and
solutions.

10. Title of the Paper: Smart Grid Big Data Analytics: Survey of Technologies, Techniques, and
Applications
Authors: DABEERUDDIN SYED, AMEEMA ZAINAB, SHADY S. REFAAT, HAITHAM
ABU-RUB and OTHMANE BOUHALI
Publication Year: 2015
Objectives: To give deep insights into various big data technologies and discusses big data
analytics in the context of the smart grid. The challenges and opportunities brought by the advent
of machine learning and big data from smart grids are also presented here.
Findings: This paper provides a comprehensive review of the big data analytics process presents
the proposed potential technologies that aim to overcome the challenges involved in the processing
of smart grid data. Applied solutions used for big data analytics in smart grids are also given here.

11. Title of the Paper: Data analytics in the electricity sector

Authors: Frederik vom Scheldt, Hana Medinová , Nicole Ludwig , Bent Richter , Philipp Staudt
Christof Weinhardt
Publication Year: 3 June 2020
Objectives: To conduct a systematic analysis of studies pertaining to the creation and
application of Data Analytics methods in the context of the electricity value chain.
Findings:
Summarizing the findings from the different areas, and identify best practices and what researchers
in one area can learn from other areas.

12. Title of the Paper: Big Data Issues in Smart Grids: A Survey
Authors: Pierluigi Siano ; Maedeh Ghorbanian ;Sarineh Hacopian Dolatabadi , and Pierluigi
Siano
Publication Year: July 25, 2019.
Objectives: To analyse major issues related to the architectures, the key technologies, and
standardizations of big data analytics in smart power systems.
Findings:

13
• Brief study of the concept of big data.
• A thorough examination of the technological characteristics and architectures of big data
systems.
• A focus study on the potential applications which can benefit from the big data utilization in
Smart Grids.
• A survey on the standardization works of the big data issues in the Smart Grids.
• A look at the future issues and challenges of big data systems in Smart Grids.

13. Title of the Paper: Cyber-enabled grids: Shaping future energy systems
Authors: Philip W.T. Pong, Anuradha Annaswamy, H. Vincent Poor, Benjamin Kroposki,
Yingchen Zhang, Gil Zussman, Ram Rajagopal
Publication Year: 4 December 2020
Objectives: To elaborated on how pervasive sensing, particularly with IoT contactless sensing
systems, will act as the cyber layer and transform the power grid into a cyber physical system
where processes (physical world) will be measured by sensors, the sensor data will be transmitted
by IoT communication technologies to SCADA, WAM, distribution management and energy
market (cyber world), resulting in decision making for grid operation (physical world).
Findings: Contactless sensing, which can help resolve the major barrier to widespread sensing
(installation cost), has been implemented and checked. Pervasive sensing with IoT contactless
sensing systems will supplement conventional contact measurement techniques and, due to its
scalability and non-invasiveness, will greatly promote pervasive sensing and measurements in
power grids.

14. Title of the Paper: Smart Grid Security Evaluation with a Big Data Use Case
Authors: Duygu Sinanc Terzi, Bilgehan Arslan, Seref Sagiroglu
Publication Year: April 2018
Objectives: To investigate the relationship between smart grid and big data approaches
Findings: An application prototype has been proposed using big data approaches to detect attacks
on smart grids. The algorithms with high accuracy were determined as 92% with Random Forest
and 87% with Decision Tree. Also presents a comprehensive overview relation about smart grid
and big data approaches, big data models, methodologies, and techniques used in smart grid
systems are analyzed.

15. Title of the Paper: A Review of Application of Big Data and Machine Learning in Smart
Grid, and Associated Security Concerns
Authors: Eklas Hossain; Imtiaj Khan; Fuad Un-Noor; ; Md. Samiul Haque Sunny; Sarder Shazali
Sikander
Publication Year 24 January 2019

14
Objectives: To Conduct a thorough investigation into the use of big data and machine learning in
the electrical power grid.
Findings: This paper presented a brief timeline of the grid's journey to the smart grid, and how
internet of things (IoT) had become a part and parcel of the electricity grid. Challenges associated
with IoT-generated big data, namely their analysis and protection, as well as other security
concerns in the smart grid had also been discussed.

16. Title of the Paper: Learning-based Data Analytics: Moving Towards Transparent Power
Grids
Authors: Kunjin Chen, Ziyu He, Shan X. Wang, Jun Hu, Licheng Li, and Jinliang He,
Publication Year March 2018
Objectives: To present the learning-based data analytics moving towards transparent power grids
and provide some possible extensions including machine learning, big data analytics, and
knowledge transferring
Findings: The framework of data analytics and learning is extended in both vertical and horizontal
directions. In particular, we focus on the application of advanced machine learning techniques in
power systems. It is hopeful that the data analytic framework presented in this paper will make
power grids more transparent, efficient, intelligent, and user-friendly in the future.

17.Title of the Paper: A Big Data platform for smart meter data analytics
Authors: TomWilcox , NanlinJin , PeterFlach , JoshuaThumim
Publication Year February 2019
Objectives: To proposed a new core-broker-client system architecture for big data analytics. They
implemented a platform named Smart Meter Analytics Scaled by Hadoop (SMASH).
Findings: SMASH is able to perform data storage, query, analysis and visualization tasks on large
data sets at 20 TB scale. SMASH performance in storing and querying large quantities of data is
being compared with the published results provided by big leagues like Google Cloud, IBM,
MongoDB, and AMPLab. The SMASH provides industry a competitive and easily operable
platform to manage big energy data and visualize knowledge, with potential to support data-
intensive decision making.

18. Title of the Paper: Cloud-Based Software Platform for Big Data Analytics in Smart Grids
Authors: Yogesh Simmhan; Saima Aman; Alok Kumbhare; ; Sam Stevens; Qunzhi Zhou; Viktor
Prasanna; Rongyang Liu
Publication Year 07 March 2013
Objectives: This paper focuses on a cloud-based modular computing framework for the Smart
Grid cyber-physical infrastructure. The role of clouds and their trade-offs in the Smart Grid Cyber-
Physical Agile Framework are examined in this article.

15
Findings: The framework includes an adaptive information integration pipeline for ingesting
dynamic data, a safe repository for researchers to share expertise, scalable machine-learning
models trained over large datasets for agile demand forecasting, and a portal for visualizing
consumption trends, all of which have been validated at the University of Southern California's
campus microgrid.

19. Title of the Paper: Software Models for Smart Grid

Authors: Anjali Singhal and R.P. Saxena
Publication Year 24 January 2019
Objectives: Smart Grid Interoperability, Maturity Model, Smart Grid Investment Model, Smart
Grid Maturity Model and Smart Grid Conceptual Model were some of the models that were
presented in this paper
Findings: A model will be designed to provide a complete framework for understanding the smart
grid deployment and capability within an electric utility. This will assist an intelligent testing and
monitoring in Smart Grid.

20 Title of the Paper: Grid Analytics: How Much Data Do You Really Need?
Authors: Stanley E. McHann, Jr.
Publication Year 2013
Objectives: To apply Information Engineering principles as an alternative to the "Big Data"
approach to capturing and storing data and to turn data into useful information for a utility.
Findings: This paper suggest to take the time to evaluate what data our utility really needs, where
it will be stored and managed, and how it will benefit the organization. The amount of data needed
to do an effective job at Grid Analytics will vary from utility to utility, but it is suggested that if
we look at the data we already have, we would have a very capable analytics system with a modest
investment and utilizing proven methodologies such as Information Engineering.

16
 RESEARCH GAP

The study of above literature highlights few gaps in research like more research is required to
develop viable solutions for other security concerns such as physical threats, network attacks and
encryption attacks. Communication systems also need to be more efficient, with more protective
measure in case we would be using Data analytics as with data privacy and security is everything.
Platforms in data analytics are not incorporating real-time control. Existing platforms for smart
energy big data analytics is mainly for data sharing and intelligence, or precisely system
monitoring and financial related intelligence. It is however desirable to incorporate a real-time
control module that can produce real-time physical control signals based on the smart energy big
data analytics.

RESEARCH PROBLEM

There is a lack of comprehensive and specific strategies for leveraging IoT and machine learning
effectively within utilities. Even in the existing literature on the big data analytics in Smart Grids,
the lack of a comprehensive and general standard, specifically concentrated on the big data
management in the Smart Grids, is also observable. Hence, the development of new big data
standards and adjustment of existing information and communication standards to improve the
interoperability of smart devices dealing with massive data used in the Smart Grids is needed
keeping in mind to mitigate Big data issues along with security & issues.

17
 RESEARCH QUESTION & OBJECTIVES

RESEARCH QUESTIONS

1. What is the current status of, benefits of, challenges to, best practices in and standards for
Data Analytics in the smart grid in India as compared to world?
2. How we can implement these Data analytics tool in Smart Grid efficiently.
3. What are the current challenges or threats in implementing these technologies?
4. Will one single platform be sufficient to handle real-world applications or we need hybrid
models operating here?

RESEARCH OBJECTIVES

The Research Objectives of this study is to: -

• To Provide a view into the current status of, benefits of, challenges to, best practices in and
standards for Data Analytics in the smart grid in India as compared to world.
• Introduction to Big Data/Machine Learning /Artificial Learning, Benefits, Challenges and
Issues, Recommended Standards, Existing Frameworks and Future Needs, Best Practices,
Potential Applications and Improvements.
• To understand current challenges or threats that can cause major issue in future pertaining to
smart grid due to data analytics implementation and suggest methods to overcome those
problems.
• To analyze the efficacy of each platform in data analytics, with a focus on their strengths in
managing real-world applications. Another avenue to pursue is the idea of integrating several
channels to address a specific application issue.

18
 RESEARCH DESIGN & METHODOLOGY

Both Primary and Secondary data collection method is adopted in the following case.

Under Secondary data collection method, the sources are as follows: -

• Newspaper Articles
• Research Papers
• Internet website
• Government Websites like National Smart Grid Mission, Ministry of Power.
• Reports
• Books
• Case studies
Under Primary Data Collection method, the sources are as follows: -

• Interactive sessions with the experts of Analytics, Technology and Power Sector field
• Personal Interview with experts in the field of Analytics and Power sector.

Study Design: - Qualitative research is a systematic method with exploration in mind. Qualitative
study is often characterized as an evolving model occurring in a natural setting that helps the
researcher to establish a degree of information through a high degree of participation in the
individual experiences (Creswell 1994).
Qualitative analysis requires a purposeful utilization of the data gathered to define, clarify and
perceive. Leedy and Ormrod (2001) argued that the classification of qualitative analysis is less
formal as it formulates and develops new hypotheses. Qualitative Research may also be defined
as a successful paradigm that takes place in a natural environment that helps the researcher to
establish a degree of knowledge by being deeply interested in the real interactions (Creswell 2003).
Hence qualitative research involves poststructuralist theory that covers up to five areas in a
research. However, this study will be focusing on only one area.

Content analysis- Leedy and Ormrod (2001) identify this method as "an in-depth and
comprehensive analysis of the material of a given information set in order to detect patterns,
themes or biases" (p. 155), Content Analysis is a mode of human communication like books,
magazines, newspaper and videos, as well as other ways to detect patterns, themes, or prejudices
(Carrie Williams, 2007).
This research study starts with broad generalization then translate into specific observation,
investigation and collection of data to explain relevant hypothesis (RQ) in new and emerging
theories.

19
Source of Data
Secondary data will be collected for the study.

Sampling
Before being analyzed, qualitative data is typically in textual form. Elicited texts (written
specifically for this study) or current texts are both acceptable options (pre-existing texts, such as
meeting minutes, policy documents or weblogs).

Limitations
• Secondary research can be done only from the literature survey of published authors.

• This topic study requires opinion for SME (Subject Matter Experts) of Data analytics

• Limited access to data regarding data analytics been implemented in smart grid in India

• Difficult to find the live case of data analytics been implemented in India.

• Apart from machine learning, there are currently no traditional artificial intelligence
applications in the smart grid. However, an increasing number of new models, such as deep
learning and reinforcement learning, are demonstrating promise in allowing AI use in the smart
grid.

20
 FINDINGS & DISCUSSION

1. Conceptual Definitions
Statistics: The study of the assortment, examination, collection, analysis, interpretation,
presentation, and organization of data. For statisticians, the normal protocol is to start with a null
hypothesis (the default position that there is no relationship between two quantities), which is then
compared to an alternative hypothesis (a position that states there is a relationship between two
quantities). The decision to dismiss a hypothesis is based on a variety of statistical tests that are
run on multiple population samples.

Data Analytics: It is the disclosure and correspondence of significant examples in data. Data
analytics is a (occasionally computerized) measure used to find novel, substantial, helpful and
possibly fascinating information from enormous data sources which is generally hard to reveal. In
the event that insights are to be viewed as a part of arithmetic, data analytics is slanted towards
playing out a similar usefulness for software engineering. Visual instruments and methods are the
favoured methods for imparting the aftereffects of performing data analytics.

Machine Learning: It is the capacity of machines (related with PCs) to adapt consequently
without being unequivocally customized. It manages portrayal and speculation of data and makes
a portrayal of cases and capacities which are assessed on these data. Speculation is the exceptional
property that the AI frameworks will attempt to yield, that is, the capacity of the frameworks to
perform well even on inconspicuous data occurrences.

Artificial Intelligence: It is the knowledge shown by machines, rather than normal insight
displayed by people or creatures. Computer based intelligence envelops procedures which can
supply an article or a program with human-like knowledge. Computer based intelligence likewise
incorporates keen specialists, substances that see their current circumstance and make moves
dependent on that discernment.

21
 Machine Learning

Artificial
Statistics
Intelligence

Datat Analytics

Fig 1. Venn diagram illustrating the interconnectedness of statistics, data analytics, machine 4
learning and artificial intelligence.

2. Types of Analytics

a) Descriptive analytics expect to give data about what occurred and it contains the initial step
that attempts to recognize valuable data/data for additional handling. It may incorporate data
representation, data mining or accumulation of reports.
b) Diagnostic analytics plan to comprehend the reason for any event caused and attempts to
recognize challenges and opportunities.
c) Predictive analytics are utilized to cause probabilistic expectations to recognize patterns with
the plan to figure out what might occur later on in future.
d) Prescriptive analytics are applied to recognize the best result to any event, given the system
parameters, and draw strategies to deal with similar events in the future. It utilizes tools such
as simulation techniques and decision support to explore optimal strategies and to mitigate
future risk.

22
 Statistics Analytics Machine Learning Artificial Intelligence

How can we make it

Happen?
What will
Happen? Prescriptive
Analytics
Why did it Predictive
Happen? Analytics
Value

What
Happened? Diagnostic
Analytics

Descriptive
Analytics

Difficulty

Fig 2. The purpose of different types of analytics changes as we move along the continuum of
value.

3. Potential Impact of using Data Analytics in the Smart Grid

According to a NIST survey, the benefits of power grid modernization can be up to five times the
development cost. According to preliminary estimates from the American Council for an Energy-
Efficient Economy, the use of information and communication technology (ICT) and smart
appliances could save the United States $80 billion annually in electricity costs. This would only
be feasible if data collected in the smart grid are subjected to analytics (BDA, ML, and AI).
Increased customer loyalty, better resource usage, and enhanced service quality are all major
advantages of analytics. However, a proper data management system is needed to capture, process,
and analyze data in order to perform analytics.

23
a) Data Acquisition Framework & Data Sourcing

Smart grid environment comprises of power generation unit(s), transmission lines, distribution
stations and end-users like industrial, commercial or residential users. Apart from these, EVs and
plug-in hybrid EVs (PHEV), has also made their way into the electricity market due to their
growing popularity in the transportation sector.

Data are accumulated in the cloud foundation because of its different benefits as talked about
above. As ICT is a necessary piece of the smart grid, data can be effortlessly assembled from its
related elements. The data of smart homes can be assembled by putting smart meters and different
sensors in smart homes. EV/PHEV data can likewise be gathered at whatever point they speak
(with the smart grid or cloud) to get administrations or to trade data. The passageways (AP) are
put at fitting spots which help these substances trade data straightforwardly or in a roundabout way
by means of the cloud. The data from EV/PHEV and the smart grid is traded straightforwardly or
through the APs associated with the cloud by means of the web. The data gathered from smart
meters in smart homes and workplaces are shipped off the cloud utilizing APs. In Fig.3, circulation
stations comprise of different charging stations to give charging to, or releasing of, EVs. These
charging stations likewise send their utilization data to the cloud with the assistance of APs. Apart
from the framework data can also be collected from other sources involving different technologies
as given in below Table.

Data type sources Technology Remarks

involved
Advanced metering Smart meters The amount of data produced by smart meters
infrastructure in homes has increased dramatically as a result
of the increased adoption of these meters in
homes.
Market pricing, real-time on demand, remote
meter configuration, demand-side
management, electric usage, power quality
monitoring, and local control

Distribution automation Grid equipment Sensors are installed in distribution systems

that take several samples of data per unit of
time for real-time grid monitoring and control.

Off-grid data Third party datasets Utility companies are combining data from
third-party sources to research the impact of
utility policies on customer behavior.

24
 GIS data Geographical Asset management & map the location of
Information System outages
(GIS)
SCADA
SCADA Data V, I, E, P.F. can be measured
Will help in Automatic control, protection,
system monitoring, event processing and
alarm
Deals with incoming Wide area Dynamic stability of the grid
data from PMUs monitoring system
(WAMS)

After the data has been collected, data mining methods can be applied to it, and the findings can
be transmitted back to the smart grid or utilities (and other organizations, if necessary) for decision-
making.

b) Extracting Value by using Data Analytics

i. Data Collection-The initial step is to gather data on which analytics will be performed.
The data can be gathered from numerous sources, e.g., smart meters, smart gadgets, and
outsider datasets. When the data are gathered, the following stage is data pre-preparing. In
this progression, the data from different sources (in an assortment of organizations and
potentially containing absent or incorrect qualities) are removed. These data esteems are
then cleaned to eliminate wrong qualities. The data are then changed to the objective
archive's arrangement, after which data are stacked into a storehouse. Presently data
analytics procedures are applied on the pre-handled data to remove esteem (i.e., data) in
view of which some educated activities or choices can be made. The analytics performed
to remove worth can be separated into three general classifications viz. customer analytics,
operational analytics and undertaking analytics. Shopper analytics incorporate energy
gauging, utilization examination and burglary identification. Operational analytics
incorporate resource upkeep, blackout the executives, and conveyance advancement.
Undertaking analytics incorporate continuous grid mindfulness and representation of data.
Data collection principally manages the assortment of data from various heterogeneous
sources in various organizations and highlights. Since power grid data frequently contain
private data and individual practices of customers, data classification and security are basic
angles inside the data getting to and sending. To guarantee data secrecy also, security
during data procurement, data encryption–decoding, what’s more, total disaggregation
approaches are for the most part utilized. Those methodologies save the affectability and
protection inside the data and regularly limit unapproved access of data.

25
ii. Data preprocessing- This process is considered to incorporate and cleanse errors,
minimize data duplication, and compress data as part of the data collection method. Within
data warehouses, data storage is mainly associated with data processing (e.g., data fusion,
data integration, and data transforming). Data storage must not only handle a vast volume
of data collected in various forms/formats, but also distribute data to several analytics
platforms with various specifications (e.g., temporal/spatial resolutions and formats).

iii. Analytics- This process encourages analytic approaches to analyze the gathered data by
preparing confidential and derived information from the collected raw data using
inspection and modeling methods. This is the most crucial stage in a big data system, as it
aims to extract valuable knowledge for subsequent decision-making processes. Based on
the depth of research, big data analytics can be divided into descriptive, predictive, and
prescriptive analytics. Data visualization, predictive analysis, machine learning (ML), and
DM are the four types of big data analysis approaches. DM offers a detailed overview of
the large data lifecycle, trading, and pricing. From the smart grid application viewpoint,
data analytics can be sorted into four general classifications. Occasion analytics basically
covers conclusion/location of the power frameworks occasions like issues and blackout
administrations. Likewise, occasion analytics additionally incorporate a clear examination
of earlier power framework occasions utilizing different methods (e.g., arrangement,
separating, and relationship) Detection of strange working conditions including
shortcoming identification framework blackout recognition, location of pernicious
assaults, and robbery of power are a portion of the key application regions for occasion
analytics. State and operational analytics fundamentally incorporate a mix of symptomatic,
prescient, and viewpoint analytics. As shown in, the key power framework use of the state
analytics incorporates state assessment, framework ID, and grid geography recognizable
pieces of proof. Essentially, the key power framework applications for operational
analytics incorporate energy/load estimate [, energy the board and dispatch of assets.
Additionally, client analytics likewise incorporates at least one of the engaging,
symptomatic, prescient, and point of view analytics relying upon the particular applications
and use cases. The key power framework applications that fall under the client analytics
incorporate client characterization/classification, the connection between purchaser
conduct and energy utilization designs, and request reaction (DR).

iv. Action or Decision- Based on all above 3 process we come to a final Decision.

26
 • Smart Meters
• Smart Devices • Extraction • Consumer
Data • Grid Pre • Cleaning Action or
Analytics • Operational Decision
Collection Equipment Processing • Transformation
• Third Party • Loading • Enterprise
Data

Fig 4. Extracting Process

4. Security in the Smart Grid

Millions of devices and infrastructures are linked and interconnected through communication links
in the smart grid setting, exposing the grid to potential security vulnerabilities. Additionally, cloud
computing allows for the virtualization of applications; however, sharing the network with millions
of users raises security issues. Furthermore, the smart grid must be highly flexible and available in
real-time applications, where low latency can be a significant challenge. As a result, many cyber-
security aspects, such as security availability, integrity, confidentiality, and accountability, raise
the risk of the smart grid being compromised.

a) Cyber-Security Detection

The power business has encountered the first known effective digital assault on a power grid
endured by the Ukrainian circulation organization in December of 2015, where substations were
switch off, and more than 200 thousand individuals were left without power for a span from 1 to
6 hours. Following the episode, the examination uncovered a complex cyberattack comprising of
lance phishing messages with malware bargaining the corporate organizations, capture of the
administrative control and data procurement (SCADA) framework and distantly turning off the
substations, annihilation of worker records, and impairing of data innovation (IT) foundation
segments. While the danger scene keeps on advancing, associations are generally worried about
cybercriminals utilizing phishing and obscure malware philosophies. On the off chance that
27
malware develops and re-engineers itself continually as it spreads, the customary malware
recognition techniques for looking for explicit code marks won't be viable. Given the gigantic
measure of verifiable and progressing network protection data assortment accessible for
examination, AI and AI calculations are powerful devices accessible to fuse into the digital
protection technique. The chances for critical effect in digital protection guard utilizing AI may
incorporate the accompanying: prior identification of new or obscure malware dependent on
authentic baselines, improving the effectiveness of refreshing the malware signature database, and
expanding IT staff efficiency through decrease of number of episodes that should be researched
and 8 remediated.

b) Cyber-Physical Theft

Power Theft in the power grid has been a significant worry for utilities and causes tremendous
monetary misfortunes. As indicated by the Northeast Group, LLC, utilities lose up to $90 billion
worldwide because of power robbery [26]. By and large, power robbery can be arranged into
physical and digital assaults. Actual assaults are done by physically tapping the electrical inventory
from an area or straightforwardly from the feeder. Nonetheless, with progressions in
correspondence innovation, one can likewise instate distant burglary on the grid utilizing the
Internet, known as a digital assault. The most normally happening digital assault on the grid is
digital altering, through which the aggressor malignantly changes meter utilization data, at last 20
prompting a decreased power bill for the structure related with the meter. Partially to improve the
grid's strength against such assaults, service organizations are contributing billions of dollars on
smart grid framework. Nonetheless, framework changes alone won't forestall digital assaults on
the grid. To address this issue, progressed ML and AI methods can be utilized by the utilities.
These strategies are equipped for taking in client conduct from recorded data and can undoubtedly
distinguish irregularities in client conduct. For instance, an abrupt decrease in the purchaser's
utilization can fill in as a pointer for burglary. AI and man-made brainpower methods need data to
detail exact models for distinguishing digital assaults. This data can be accumulated from different
sensors conveyed across the power arrange and can be prepared progressively utilizing cloud
administrations and edge figuring. Thus, arrangement of AI models to investigate this data can
assist with producing helpful data which can be utilized to recognize and defeat digital assaults.
For instance, the creators in utilized the data accumulated from sensors conveyed on transmission
lines, the circulation station and transformer levels to check power lines that are inclined to actual
tapping. The creators utilized purchasers' smart meter data and applied ML procedures on those
data to recognize the buyers who messed with the smart meters.

c) Internet of Things and the Smart Grid

In 2016, the quantity of associated gadgets utilized overall was accounted for as 6.4 billion by
Gartner and it is relied upon to arrive at 20.8 billion by 2020. This report additionally takes note
28
of that about 5.5 million new gadgets were associated every day in 2016 alone. The smart grid, as
a major purchaser of independent associated gadgets, uses a huge number of IoT gadgets, yet in
addition breaks down huge volumes of made data to settle on better choices about smart grid
organizations. 91 million smart meters are normal by 2020, alongside 36 million smart interfacing
indoor regulators, and 183 million IoT private gadgets. The anticipated number of yearly data
focuses for 15-minute stretches will move toward 11 trillion among these classifications alone.
This flood of data will empower further examination of the grid at an inexorably granular level to
act and respond progressively, along these lines improving the effectiveness of energy use. Large
data analytics and IoT are essential pieces of the smart grid. Thusly, there is a basic requirement
for coordination of AI with IoT sensors and gadgets at various smart grid levels to break down the
entire environment to streamline the expense, balance the energy assets and therefore structure a
savvy smart grid.

The smart grid is viewed as probably the biggest recipient of the IoT. IoT innovation can uphold
smart grids by giving high entrance of data sources, for example, power creation, stockpiling,
transmission, circulation and utilization, and improving network, robotization and checking of
every gadget. The advanced and clever smart grid won't be financially savvy without IoT
innovation. Smart grids have effectively achieved broad reception in detecting, transmission and
preparing of data, and now IoT innovation assumes a significant part in grid arrangement and
interfacing inheritance gear. The utilization of the IoT in smart grids may fall into at least one of
the accompanying classes:

• IoT is applied for utilizing distinctive IoT smart gadgets for checking hardware status,
• IoT is applied for gathering data from gear with the help of its connected IoT smart sensors
and gadgets through assorted specialized apparatuses and
• IoT is applied for regulating the smart grid across application interfaces. In every one of
these classifications, AI is utilized to give constant data analytics and produce educated
choices.

5. Applications of Data Analytics in the Smart Grid

This segment offers examples of traditional smart grid applications to demonstrate how big data
analytics, machine learning, and artificial intelligence are used in the smart grid. These examples
are given for demonstration purposes only and are not intended to be exhaustive. Section 4 contains
a list of additional smart grid-related applications that are planned. Aside from machine learning,
there are currently no traditional artificial intelligence applications in the smart grid. However, an
increasing number of new models, such as deep learning and reinforcement learning, are showing
promise in allowing AI to be used in the smart grid.

29
a) Big Data Analytics Applications

The design of smart grid operations makes it difficult to manage data produced in the smart grid
with conventional analysis techniques to deliver actionable information in reasonable timeframes
as required by the nature of smart grid operations. Big data include 5Vs (Volume, Velocity,
Variety, Veracity and Value) and pairing it up with smart grid application we understand how it
would be applied in smart grid.

Feature 5Vs Model Smart Grid Applications

Volume Number of records and required Smart meter data in large quantities
storage and advanced sensor technology

Velocity Frequency of data generation, If data from smart meters is

transfer or collection
collected every 15 minutes, then we
have 1 million devices generating
35.04 billion data entries per year,
or 2920 Tb. For real-time
monitoring and analysis, the
frequency at which data is collected
is critical.

Variety Diversity of sources, formats, Structured (for example, relational

multidimensional fields data), semi-structured (for
example, web service data), and
unstructured data exist (e.g., image
or video data)

Veracity Reliability and quality of data Data that can be trusted is essential
for ensuring the safety and
reliability of a system.

Value Getting the most out of the benefits Applications derive value from
and insights smart grid data, e.g., predicting
future electricity generation and
demand.

Big data analytics can help predict and understand end-user behavior, improve network durability
and help in faults detection, improve protection and monitoring, improve efficiency, and maximize
available resources and future planning.
The use of big data in power generation planning, economic load dispatch, performance and
efficiency analyses of power generation and storage systems, and power grid optimization and cost

30
minimization are among the most interesting aspects of research in this area. Load modeling and
forecasting are notable big data applications that can be divided into long-term and medium-term
load forecasting for power system capacity planning and into short-term forecasting for power
distribution, DR, and pricing. The most critical aspects of big data implementation in this scope
are research works on energy storage capacity, power demand and usage forecasting, price
analysis, and billing and consumer behavior analysis.

b) Machine Learning Applications

Machine learning calculations are especially utilized for grouping data accumulated from the smart
grid domain. Data are clustered to shape groups with comparative attributes (regular
characterization), e.g., gathering data focuses with comparable dynamic/responsive power profiles
for transmission framework administrator (TSO) contemplates. Different models incorporate
recognizing low voltage (LV) feeders with comparative burden designs for appropriation
framework administrator (DSO) studies to be compacted or summed up into bunch models (e.g.,
creating agent days for wind creation and their consideration into network reproductions). Further,
ML can utilize smart grid data to comprehend the hidden construction, to acquire valuable bits of
knowledge, recognize abnormalities and produce speculations, and so forth (e.g., distinguish
burglary and comprehend client utilization conduct at a specific feeder). Forecasts assume a huge
part in power frameworks as they are normally used to design future totaled power interest, future
framework supply, assessing adaptability and hold administrations prerequisites or operational
administration of dissemination organizations.

c) Multi-Agent System Applications

A utilization of AI in the smart grid setting is multi-specialist framework (MAS) modelling. As
the power system framework turns out to be more decentralized, market-arranged, multi-variable
and complex, control and dynamic through a unified methodology gets testing as it requires huge
computational power to decide ideal choices for the whole framework without huge deferrals. An
improved methodology is to separate the power framework into more independent units that can
settle on a portion of the choices locally, following decentralized or dispersed control. MAS
approaches can be utilized to take care of complex issues in a productive, versatile and
appropriated way. Expected utilizations of MAS with regards to the smart grid incorporate the
control of microgrids, flaw the executives and aggravation conclusion, self-recuperating and power
rebuilding, voltage control, recurrence control, request side administration dependent on specialist
engineering, energy utilization enhancement and planning and coordination of capacity gadgets.

6. Current and Expected Use of Smart Grid in US & UK

A few US states are effectively investigating how to utilize devices and innovations to understand
a smarter grid. Model US projects include:

31
● DMS Platform by the University of Hawaii: The venture showed both battery energy stockpiling
framework and request reaction innovations can be compelling in diminishing pinnacle loads on
the Maui Electric Company (MECO) framework and of individual substations. The experience
acquired in this task will assist MECO with incorporating circulated and sustainable power assets
(PV, wind) with the activity of its focal generators and transmission framework.

● Perfect Power by Illinois Institute of Technology (IIT): The IIT and accomplices proposed to
create and exhibit a framework and supporting innovations to accomplish "Wonderful Power" at
the principal grounds of IIT. Plans are for a self-mending, learning and mindful smart grid that
distinguishes and disconnects shortcomings, reroutes power to oblige load changes and generation,
dispatches generation and decreases request dependent on value signals, climate estimates, and
loss of grid power.

● Some utilities are utilizing prescient apparatuses using phasor estimation data for guaging
conceivable geomagnetic unsettling influences and shielding transformers from harm.

● A well-known application is resourcing the board where utilities are using prescient investigation
to decide when their resources require upkeep.

● Oklahoma Gas and Electric is coupling AMI with time sensitive rates and in-home presentations
to diminish top power use. This may empower the utility to concede the development of a 170
MW cresting power plant.

● On July 5, 2012 a serious windstorm in Chattanooga, TN brought about 80,000 clients losing
power. The city had the option to reestablish power to half of the influenced inhabitants inside 2
seconds utilizing mechanized feeder exchanging.

● Using synchro phasor data for continuous control, the Western Electricity Coordinating Council
established that it can expand the energy stream along the California-Oregon Intertie by 100 MW
or more, with an expected decrease in energy expenses of $35-75 million more than 40 years
without extra high-voltage capital ventures.

● Data mining and AI put together analytics with respect to chronicled occasions data has been
given in an incorporated equipment programming stage for continuous occasion discovery by Toll
grade correspondences. This has shown the utilization of prescient grid-analytics on data gathered
by the organization's restrictive smart-grid sensors for forestalling power outages to happen. The
stage distinguishes abnormalities on a conveyance feeder before they amplify and accordingly
empowers a proactive, prescient procedure for overseeing constant occasions, like the idea
conceived in for prescient discovery of inadvertent islanding in feeders. Utilities like Western

32
Power Distribution have utilized this stage and have encountered improved unwavering quality
files.

Some model smart grid projects from the UK include:

● Network Constraints Early Warning Systems by Scottish Power Energy Networks: The ventures
expect to use enormous scope, dispersed smart meter data to screen and recognize the danger of
power floods and voltage trips outside specialized operational cutoff points, in various pieces of
the appropriation organization (or subnetworks). With a huge number of buyers, one can't gather
data continuously from each smart meter/shopper in the organization, thusly a significant inquiry
to answer is the thing that data granularity is required for various pieces of the organization to
identify these trips and give early alerts of possible weaknesses.

● Thames Valley Vision by Scottish and Southern Electricity Networks: The venture was the
primary scaled sending of LV substation observing of ongoing power data and their combination
into a circulation the executives framework. Data were utilized for substation orders, accumulation
and guaging of future organization stacking.

● Home-Offshore: The undertaking plans to utilize progressed automated checking and detecting
procedures for the distant investigation and resource the executives of seaward wind ranches and
their associations. Data gathered will create experiences for symptomatic and prognostic plans
which will permit improved data to be spilled into multi material science operational models for
seaward wind ranches.

7. Benefits and Impacts

The utilization of BDA can give various advantages to all partners inside the power business. For
example, DSOs can accomplish improved coordination of the stockpile and the interest in
dispersion organizations, shoppers can improve their energy productivity and accomplish critical
investment funds in their energy charges by intently checking their energy interest, energy
providers or retailers can acquire experiences into purchaser conduct and improve operational
effectiveness and energy generators can expand benefits by upgrading their generation resources
and creation. TSOs can upgrade their drawn out arranging by examining the conduct of various
sorts of customers in gatherings, for example, private, business and modern and by applying ML
calculations that successfully catch the effect of scattered generation, which is frequently
imperceptible at a transmission level. Liberated energy frameworks have progressively depended
on specialist co-ops, for example, request aggregators, stockpiling suppliers and virtual power
plants. Specialist organizations' ideal activity and benefit boost procedures are dependent upon
vulnerability and need to represent the essential conduct of numerous players taking an interest in
the energy market. Not exclusively is the dynamic cycle multi-variable, yet it needs to represent

33
interdependencies of the choice factors at various time spans. For instance, long haul choices may
acquaint limitations with medium term objectives and activity. Helpful procedures to adapt to these
issues can be gotten from the field of game hypothesis, which is appropriate to contemplate key
associations between different entertainers and to recognize market equilibria. These joined with
probabilistic displaying and stochastic improvement devices, permit specialist organizations to
decrease costs and augment incomes. Specialists can recognize and get the hang of offering
systems in the energy markets, ideal dispatching, adjusting and activity of their resources. What's
more, specialist organizations need to receive prescient support, prognostics and wellbeing the
board to embrace market and activity methodologies that delay the helpful lifetime of their
resources. Similar instruments and standards can be utilized for microgrid control in a
decentralized or dispersed design and coordination with the concentrated grid. Service
organizations, particularly at a conveyance level, need to embrace huge data procedures as the
energy framework keeps on developing. Most utilities are at present limited to utilizing graphic
and indicative analytics that plan to dissect verifiable data or occasions to comprehend the
purposes for their results, e.g., framework deficiency the board or blackout the executives.
Nonetheless, as energy frameworks develop to turn out to be more decentralized and dependent
on discontinuous DER, utilities need to send prescient analytics to assess possible future situations,
for example, assessing future grid venture prerequisites and estimating load effect and resource
observing. Further, prescriptive analytics can prompt significant experiences for arranging of
generation and transmission/dissemination limit or enhancement of sustainable reconciliation.

8. Expected Barriers/Challenges/Issues

Significant challenges arise while applying Data analytics in Smart grid they are summarized
below:
• The storage of an ever-increasing amount of data, usually in the cloud, is needed. This can
result in data explosion and scalability problems. Technical problems such as network
bandwidth availability and data protection concerns must also be addressed.
• Another problem with machine learning techniques is the possibility of insufficient training
data, which may reduce faith in the results of supervised machine learning models for
previously unknown circumstances. This problem must be addressed as advances in semi-
supervised and unsupervised machine learning continue.
• Privacy, confidentiality, and the security of personal data all necessitate extra caution. Data
security, intellectual property protection, personal privacy protection, commercial security,
network security, and financial details are all major issues that must be addressed.
• Data and smart grid technology security are important. Cyber-attacks or cyber-physical attacks
on smart grids are possible. The functionality of smart grid systems is critical for society as a
whole, and they contain sensitive data that must be protected from malicious attacks and
vulnerabilities. This covers all inherently distributed access points and equipment safety, as

34
 well as data storage security achieved by the use of sophisticated cryptographic techniques and
verification mechanisms.
• Smart meters, customer data, web data, weather data, population data, geographic information
system data, DMS data, energy management system, SCADA, and other data sources and
formats are all available. Interoperability between different devices and control levels must be
maintained, and data must be incorporated. For data collection and governance, new
regulations and structured processes are needed. Standards for data definition and collaboration
are lacking, which is crucial for interoperability. Furthermore, for the benefit of all
stakeholders, data integration and data sharing practices across organizations must be
established.
• Incomplete or insufficient data is a major problem. Smart meters and sensing equipment are
not mounted everywhere in the grid, and they can fail or malfunction. Missing data could need
to be extrapolated from available data in this situation. When it comes to ML techniques that
rely on good data quality and availability for algorithm training, the issue becomes critical.
While there have been major advances in machine learning methods that deal well with missing
data, there is still work to be done. Pre-processing techniques and error detection systems that
ensure data accuracy are needed to ensure data integrity.
• Utility companies will need to install new IT equipment, such as sensor equipment, hardware
and software tools, and data storage devices, which will presumably cost a lot of money to
install and maintain. Furthermore, utilities are unsure about the change from a centralized to a
decentralized power network, as well as BDA's position in future energy systems.
• In the field of big data analytics, there are still research issues. Cloud storage, distributed file
management, and novel databases, such as NoSQL databases for parallel processing, are all
being developed. For batch processing, many machine learning libraries have been developed,
and novel techniques for stream processing are being developed. However, more progress is
needed to meet the time response requirements of the smart grid.
• Due to the large size of the datasets and their high dimensionality, data visualization can be
difficult. Current data analytics tools have low performance when it comes to visualizing
massive data sets, have long response times, and are difficult to scale. Aggregation can help
with the size of data, but it can also mask the precision required for insight in certain cases.
• Another important problem that necessitates a cost-effective and long-term solution is the
energy consumption needed for potentially intensive computing power demands.
• Finally, as with any major change, a well-trained workforce, both current and future human
resources, is needed.
• Verification and validation of analytics, certification and regulatory enforcement, and
interoperability of all smart grid components are also concerning.

35
 RECOMMENDATION

For electric utility industry trying to execute Big data arrangement, an organized plan of action is
important to satisfy monetary objectives also, prerequisites, everything being equal. Since the
accomplishment of enormous data analytics in the utility business is dependent upon the dynamic
cooperation of electric utilities, clients, and framework administrators, recognizing income
streams and advancement of legitimate business models are basic to the achievement of huge data
arrangement to the smart grid. All the more significantly, the expense related with the selection of
enormous data to all partners ought to be supported and acknowledged across the expansive
partners that incorporates strategy producers, controllers, utilities, and the purchaser. In this way,
future exploration should zero in on techno-monetary examinations to evaluate specialized and
financial upsides of enormous data to the electric utilities, framework administrators, and clients.
Also, labor force preparing will be needed for data examination understanding just as to better
comprehend the ability and limits of these instruments. Along these lines, admittance to data will
simply give a re-visitation of utility ventures when experts completely comprehend the abilities
and devices accessible, requiring changes in undergrad and graduate educational plans to
incorporate data science themes for future power engineers.

Thorough examination of huge data for misusing covered data and relationships among shifted
data sources is troublesome. Along these lines, progressed computerized reasoning procedure, for
example, profound ML is fundamental not exclusively to abuse fine-grained designs inside the
data, yet in addition to settle on the choice cycle less dependence on human impedance.
Notwithstanding, because of expanded organizations of savvy gadgets in electric grids, and
interdependencies of electrical organization with other basic foundation (for example gas, water,
and transportation), smart grid data will keep on filling in volume, assortment, and veracity.
Subsequently, adaptability of the ML models is basic. Also, since opportune and precise catch of
covered up data is the way in to the activity of electrical framework, exactness and computational
productivity assume key parts. Subsequently, future R&D endeavors on ML should zero in on
versatility, computational efficiency, and accuracy.

Existing smart energy enormous data analytics plans don't join perception as a necessary part. As
the vital advantage of huge data analytics is to help utilities in making moves dependent on
Realtime situational insight got from the data analytics, reconciliation of cutting-edge
representation with data analytics is required. Since the greater part of the current analytics are
enlightening and educational, it requires the grid administrators to take natural choices.
Coordination of cutting-edge representation along with mechanized activity gives mandate data to
the administrators and keeps away from the need of a natural choice. Hence, co-plan of smart grid
enormous data analytics and progressed representation systems can create a consistent
incorporated structure, which can diminish security hazard and help to take compelling choices.

36
At present, there are no settled norms and administrative structures for dividing data between
utilities, Weather Corporation, and other energy frameworks (for example transportation, oil, and
gas sectors). Regulatory consistence overall may require a broad overhaul to oblige the effect of
enormous data applications and furthermore the digital protection parts of such applications. To
start with, specialized principles ought to be set up to amplify the worth of large data just as to
guarantee data trades among various elements are achievable and significant. In this way, an
administrative system ought to likewise be set up to tie the substances with lawful guidelines and
guidelines as far as data sharing. Moreover, an unbiased outsider is likewise required to make
reasonable assessment and support of the expenses related with enormous data organizations for
controlled business sectors and various substances. Hence, endeavors from proficient networks
ought to be put resources into setting up guidelines for data dividing between stages/models, and
distinguishing the components of administrative structures to tie utilities in conveying huge data.

Despite the fact that there are a couple of standard data models for smart grid interoperability (for
example IEC 61850, IEC 61850-90-7, IEC 61970/61968, IEEE 1815, and IEEE 2030.5), there is
no norm data models to depict interoperability among different enormous data analytics stages,
design, and their operational incorporations with utility choice systems. Besides, capacity, use,
spread, and sharing of data with utility operational structures are not bound together.
Interoperability between different distributed computing administration merchants is fundamental.
Consequently, broad R&D is expected to foster interoperability among various gadgets, network
activities, data analytics stages, large data design, data vault, and data models.

Inferable from the colossal volume of smart grid data, dispersed and equal insight is regularly
expected to adequately address data calculation and taking care of difficulties. Since disseminated
registering and equal knowledge are successful for tending to neighborhood grid issues and
difficulties, they need a type of coordination to safeguard worldwide perceivability. In this way,
viable disseminated insight and coordination calculations ought to be created. Research and
development on cutting edge draws near, for example, metamodeling, dimensionality decrease,
and edge registering ought to be done to diminish the computational and correspondence loads.

37
 CONCLUSION

This report introduces us to the concepts and possible use cases of big data analytics, machine
learning and artificial intelligence in the smart grid. Generation units, transmission lines, and
distribution grids are all part of the smart grid. Distributed energy resources are combined either
separately or at the end-customer level, and power flows in various directions, necessitating the
use of more advanced control and transmission devices for monitoring, transmitting and metering
all power components. All of the grid parameters must be monitored, collected, and transmitted,
which necessitates a large amount of data. Meanwhile, conventional approaches for quickly and
accurately analyzing such massive data are infeasible, posing obstacles to effectively enabling the
smart grid. As a result, to handle the vast amount of data, effective data analytics will be used. The
smart grid uses big data analytics, machine learning, and artificial intelligence to handle the data
gathered from all power meters, sensors, and other appliances. It is possible to predict load
demand, generation rate, and device disturbance using a combination of these techniques and then
regulate the control automatically and quickly so the grid can be improved and avoid instabilities.
Applying Data analytics effectively in the smart grid however still a tedious task and one has to
face numerous difficulties. Since most power utilities are unfamiliar with big data analytics,
machine learning, and artificial intelligence, this article described concepts related to advanced
analytics approaches used in smart grid and discussed the benefits, challenges, and problems of
using these approaches. A brief overview of security issues, as well as various effect and
communication technologies in smart cities, was also given.

38
 REFERENCES

1. U. Ahsan and A. Bais, “Distributed big data management in smart grid,” 26th Wireless and
Optical Communication Conference (WOCC), Newark, NJ, 2017.
2. IEEE Smart Grid R&D Committee, “IEEE Smart Grid Survey - Structure of Emerging 35
Technologies,” October 1, 2017.
3. D. Alahakoon and X. Yu, “Smart electricity meter data intelligence for future energy
systems: A survey,” IEEE Trans. Ind. Informat., vol. 12, no. 1, pp. 425–436, Feb. 2016,
doi: 10.1109/TII.2015.2414355.
4. Xiaomeng Y, Fangming L, Jiangchuan L, Hai J. Building a network highway for big data:
architecture and challenges.IEEE Netw. 2014; 28:5–13.
5. Chen M, Mao S, Liu Y. Big data: a survey. Mob Netw Appl. 2014; 19:171–209.
6. Wang W, Xu Y, Khanna M. A survey on the communication architectures in smart grid.
Comput Netw.2011;55:3604–29.
7. Yigit M, Gungor VC, Tuna G, Rangoussi M, Fadel E. Power line communication
technologies for smart grid applications: a review of advances and challenges. IT Comput
Netw. 2014; 70:366–83.
8. Siano P. Demand response and smart grids—a survey. Renew Sustain Energy Rev. 2014;
30:461–78.
9. Fang B, Yin X, Tan Y, Li C, Gao Y, Cao Y, Li J. The contributions of cloud technologies
to smart grid. Renew Sustain Energy Rev. 2016; 59:1326–31.
10. Jaradat M, Jarrah M, Bousselham A, Jararweh Y, Al-Ayyoub M. The internet of energy:
smart sensor networks and big data management for smart grid. Proc Comput Sci. 2015;
56:592–7.
11. Agarwal V, Tsoukalas LH. Smart grids: importance of power quality. In: Proceedings of
first international conference on energy-efficient computing and networking. 13–15 April
2010. Berlin; 2010. p 136–143
12. Amin SM. Smart grid: overview, issues and opportunities. Advances and challenges in
sensing, modeling, simulation, optimization and control. Eur J Control. 2011; 17:547–67.
13. M. Ghorbanian, S. H. Dolatabadi, M. Masjedi, and P. Siano, “Communication in smart
grids: A comprehensive review on the existing and future communication and information
infrastructures,” IEEE Syst. J., pp. 1–14, 2019.
14. H. Jiang, K. Wang, Y. Wang, M. Gao, and Y. Zhang, “Energy big data: A survey,” IEEE
Access, vol. 4, pp. 3844–3861, 2016.
15. “NIST big data interoperability framework, v1.0, NIST special publication 1500,” Sep.
2015. [Online]. Available: https://www.nist.gov/el/cyberphysical- systems/big-data-pwg
16. M. Chen, S. Mao, and Y. Liu, “Big data: A survey,” Mobile Netw. Appl., vol. 19, no. 2, pp.
171–209, Apr. 2014.

39
17. M. Jaradat, M. Jarrah, A. Bousselham, Y. Jararweh, and M. Al-Ayyoub,“The Internet of
Energy: Smart sensor networks and big data management for smart grid,” Procedia
Comput. Sci., vol. 56, pp. 592–597, 2015.
18. V. Chang, “An overview, examples, and impacts offered by emerging services and
analytics in cloud computing virtual reality,” Neural Comput. Appl., vol. 29, no. 5, pp.
1243–1256, Mar. 2018.
19. J. Wu, S. Guo, J. Li, and D. Zeng, “Big data meet green challenges: Greening big data,”
IEEE Syst. J., vol. 10, no. 3, pp. 873–887, Sept. 2016.
20. M. M. Maske and P. Prasad, “A real time processing and streaming of wireless network
data using storm,” in Proc. Int. Conf. Comput. Power, Energy, Inf. Commun., Apr. 2015,
pp. 0244–0249.
21. P. Chandarana and M. Vijayalakshmi, “Big data analytics frameworks,” in Proc. Int. Conf.
Circuits, Syst., Commun. Inf. Technol. Appl., Apr. 2014, pp. 430–434.
22. J. Dean and S. Ghemawat, “Mapreduce: Simplified data processing on large clusters,”
Commun. ACM, vol. 51, no. 1, pp. 107–113, Jan. 2008.
23. X.Wu,T.Wu, M. Khan, Q. Ni, and. Dou, “Game theory-based correlated privacy preserving
analysis in big data,”
24. J. Yin, P. Sharma, I. Gorton, and B. Akyoli, “Large-scale data challenges in future power
grids,” in Proc. IEEE 7th Int. Symp. Service Orient. Syst. Eng. (SOSE), Redwood City, CA,
USA, 2013, pp. 324–328.
25. M. Aiello and G. A. Pagani, “The smart grid’s data generating potentials,” in Proc.
Federated Conf. Comput. Sci. Inf. Syst. (FedCSIS), Warsaw, Poland, 2014, pp. 9–16.
26. S. Bera, S. Misra, and J. J. P. C. Rodrigues, “Cloud computing applications for smart grid:
A survey,” IEEE Trans. Parallel Distrib. Syst.,vol. 26, no. 5, pp. 1477–1494, May 2015.
27. Stimmel CL. Big Data analytics strategies for the smart grid. Boca Raton: CRC Press;
2014.
28. Nga DV, See OH, Quang DN, Xuen CY, Chee LL. Visualization techniques in smart
grid. Smart Grid Renew Energy.2012;3:175.
29. Marjani, M., Nasaruddin, F., Gani, A., et al.: ‘Big IoT data analytics: architecture,
opportunities, and open research challenges’, IEEE Access, 2017, 5, pp. 5247–5261
30. GE: ‘Grid IQ insight: translating data to actionable intelligence for empowered decision
making’. Available at https://www.gegridsolutions.com/uos/catalog/grid-iq-insight.htm,
accessed 2014
31. Hong, T.: ‘Big data analytics: making the smart grid smarter [Guest Editorial]’, IEEE
Power Energy Mag., 2018, 16, (3), pp. 12–16
32. Zhang, R.: at ‘Big data analytics for smart grid-forecast, predict for a smarter grid’

40