Renewable and Sustainable Energy Reviews 150 (2021) 111459

Contents lists available at ScienceDirect

Renewable and Sustainable Energy Reviews

journal homepage: www.elsevier.com/locate/rser

Artificial intelligence techniques for enabling Big Data services in

distribution networks: A review
Sara Barja-Martinez ∗, Mònica Aragüés-Peñalba, Íngrid Munné-Collado, Pau Lloret-Gallego,
Eduard Bullich-Massagué, Roberto Villafafila-Robles
Centre d’Innovació Tecnològica en Convertidors Estàtics i Accionaments (CITCEA-UPC), Department d’Enginyeria Elèctrica, Universitat Politècnica de Catalunya,
UPC, Av. Diagonal 647, Pl. 2, 08028, Barcelona, Spain

ARTICLE INFO ABSTRACT

Keywords: Artificial intelligence techniques lead to data-driven energy services in distribution power systems by extracting
Machine learning value from the data generated by the deployed metering and sensing devices. This paper performs a holistic
Deep learning analysis of artificial intelligence applications to distribution networks, ranging from operation, monitoring and
Smart grid
maintenance to planning. The potential artificial intelligence techniques for power system applications and
Distribution grid
needed data sources are identified and classified. The following data-driven services for distribution networks
Smart energy service
are analyzed: topology estimation, observability, fraud detection, predictive maintenance, non-technical losses
detection, forecasting, energy management systems, aggregated flexibility services and trading. A review of the
artificial intelligence methods implemented in each of these services is conducted. Their interdependencies are
mapped, proving that multiple services can be offered as a single clustered service to different stakeholders.
Furthermore, the dependencies between the AI techniques with each energy service are identified. In recent
years there has been a significant rise of deep learning applications for time series prediction tasks. Another
finding is that unsupervised learning methods are mainly being applied to customer segmentation, buildings
efficiency clustering and consumption profile grouping for non-technical losses detection. Reinforcement
learning is being widely applied to energy management systems design, although more testing in real
environments is needed. Distribution network sensorization should be enhanced and increased in order to
obtain larger amounts of valuable data, enabling better service outcomes. Finally, the future opportunities and
challenges for applying artificial intelligence in distribution grids are discussed.

1. Introduction AI techniques can contribute to operate, maintain and plan electri-

cal networks by treating and extracting value from large volumes
The progress of Information and Communication Technologies (ICT) of data, dealing with its variety and velocity, through much faster
and digitalization are accelerating the transition towards smart en- computations [9].
ergy systems [1–3], where data have a remarkable but still underex- Considering the potential of the data collected in electrical net-
ploited role [3,4]. These data, collected by different types of sources, works, the scientific community is applying and developing AI tech-
need to be preprocessed [5,6] before applying Artificial Intelligence niques for power system applications [9]. AI can be applied in all the
(AI) techniques that lead to several Big Data applications in power power system domains, including generation, transmission, distribution
systems [7].
and consumption. In particular, the International Renewable Energy
The operation and management of electrical grids are per se com-
Agency (IRENA) envisages its application to promote the grid integra-
plex decision-making processes, even more challenging taking into
tion of renewable energy sources in all the before mentioned power
account the increasing penetration of renewable energy sources, which
are adding more variability and uncertainty in the power system op- system chain through: forecasting for renewable power plants (like
eration [8]. To address the operation, maintenance, and planning of wind and solar large scale power plants), grid stability and reliability
electrical grids, classical analysis tools can require large computational at transmission and distribution level, demand forecasting, demand-
times and might not always find a feasible solution. In this sense, side management, optimized energy storage operation and optimized

∗ Corresponding author.
E-mail address: [email protected] (S. Barja-Martinez).

https://doi.org/10.1016/j.rser.2021.111459
Received 25 August 2020; Received in revised form 8 June 2021; Accepted 4 July 2021
Available online 16 July 2021
1364-0321/© 2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Nomenclature P2P Peer-to-Peer

PCA Principal Component Analysis
ACE Average Coverage Error
PLAN Planning
ADMM Alternating Direction Method of Multipliers
PM Predictive Maintenance
AGG AGGregated flexibility services
PMU Phasor Measurement Units
AI Artificial Intelligence
PV PhotoVoltaic
AMI Advanced Metering Infrastructure
RBFNN Radial Basis Function Neural Network
ANN Artificial Neural Networks
RES Renewable Energy Systems
ARIMA Autoregressive Integrated Moving Average
RF Random Forest
BPNN Back-Propagation Neural Network
RL Reinforcement Learning
BRP Balancing Responsible Party
RMSE Root Mean Squared Error
CNN Convolutional Neural Network
RNN Recurrent Neural Network
CRPS Continuous Ranking Probability Score
SCADA Supervisory Control And Data Acquisition
DER Distributed Energy Resource
SDE Standard Deviation of Error
DBSCAN Density-Based Spatial Clustering of Appli-
SOM Self-Organizing Map
cations with Noise
SVM Support Vector Machine
DL Deep Learning
T-SNE T-Distributed Stochastic Neighbor Embed-
DLT Distributed Ledger Technologies
ding
DM Data Mining
TOP Topology
DNN Deep Neural Network
TRAD Trading
DR Demand Response
TSO Transmission System Operator
DSO Distributed System Operators
DT Decision Trees
ELM Extreme Learning Machine
EMS Energy Management System Recent studies have reviewed some of the Big Data applications in
ESS Energy Storage System power systems [4,11–19]. Some articles address Big Data applications
EU European Union focusing on the distribution grid [12,14–16,18–21]. However, they fo-
EV Electric Vehicle cus exclusively on a particular service category without addressing the
FD Fault Detection whole range of potential services applicable to Low-Voltage (LV) and
FOR Forecasting medium-voltage grids. The authors from [21] present Internet of Things
(IoT) applications in Smart Grids and summarize the applications of
GBM Gradient Boosting Machine
Machine Learning (ML) techniques in renewable generation sources,
GIS Geographic Information System
focusing on wind and solar energy forecasting. Energy consumption
GRNN General Regression Neural Network estimation and efficiency-based classification is considered by [12].
GRU Gated Recurrent Unit In [15], some Big Data analytic applications are identified, while [14]
HVAC Heating Ventilating and Air Conditioning overviews renewable energy, Demand Response (DR) and Electric Ve-
ICT Information and Communication Technolo- hicles (EVs) services. A significant number of Big Data operation ap-
gies plications are covered by [16,18,19]. For instance, [18] includes fault
IoT Internet of Things detection, predictive maintenance, transient stability, state estimation,
IS Interval Sharpness power quality monitoring, topology identification, load forecasting and
KNN K-Nearest Neighbor profiling, load disaggregation and non-technical losses detection. [16]
addresses state estimation, fault classification and detection, short-term
LogR Logistic Regression
load forecasting, and DR. [19] identifies as main applications DR,
LR Linear Regression
load disaggregation and granularity forecast, fault detection, operations
LSTM Long Short Term Memory planning convergence, equipment monitoring and predictive fault de-
LV Low Voltage tection, feature extraction and data visualization. None of the before
MAE Mean Absolute Error mentioned studies include system observability, Peer-to-Peer (P2P)
MAPE Mean Absolute Percentage Error trading or aggregated flexibility forecast for Distributed System Oper-
MASE Mean Absolute Scaled Error ators (DSO) and Balancing Responsible Party (BRP) flexibility services.
MED Measurement Error Detection Monitoring and investment planning applications are not addressed
ML Machine Learning or briefly addressed. Furthermore, they do not always include a deep
analysis of the techniques required in each application. Last but not
MLP Multi-Layer Perceptron
least, the existing relationships and interactions between the different
MLR Multiple Linear Regression
potential services are not specified in any of the before cited studies.
MSE Mean Squared Error To the best knowledge of the authors, there is no previous review
NB Naive Bayes that extensively covers AI techniques and their respective applications
NILM Non Intrusive Load Monitoring at the distribution grid level for its comprehensive analysis (includ-
NTL Non-Technical Losses ing operation and monitoring, predictive maintenance, non-technical
OBS Observability losses, forecasting, flexibility management and planning) and exam-
ining the relationship and interactions between them. Many of the
applications identified lead to data outputs that can be inputs for other
market design and operation (the latter two as multi-domain applica- applications, enabling several groups of potential services for different
tions) [10]. The present paper focuses on AI applications in distribution stakeholders.
and consumption domains. The main contributions of this paper are detailed next.

2
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

• A holistic analysis of AI applications in distribution power sys- needs scarce knowledge about the problem physics to be competitive
tems is provided after identifying and classifying the different with standard rule-based controllers [29]. However, more research
data-driven techniques for power systems and the data sources and real-world testing are needed for RL technology to become more
involved in the data acquisition. These applications include op- mature. DL belongs to the Artificial Neural Networks (ANN) field.
eration and monitoring, predictive maintenance, non-technical They are a broad family of techniques in multiple domains, which
losses detection, forecasting, flexibility management and planning can be applied to both Supervised and Unsupervised Learning. ANNs
of distribution grids. are inspired by the function of the brain, with the main objective of
• The different AI techniques required for enabling each of the learning from unstructured or unlabeled data, using single or multiple
applications mentioned above are detailed. layers (DL approach) to extract higher-level features from the raw
• The relationships between the distribution grid applications are input progressively. DL techniques can be applied to power systems
presented and mapped, proving that multiple services can be in different scenarios such as fault detection in transformers [30–32],
offered as a single clustered service to different stakeholders. Photovoltaic (PV) forecasting [33,34] and day-ahead electricity market
• Furthermore, future opportunities and challenges for the applica- price forecasting [35].
tion of AI in distribution grids are identified. It is worth mentioning the role of Statistics in power systems.
Statistics is an applied science concerned with analysis and modeling
This paper is structured as follows. The existing data-driven tech-
of data [37]. Despite the similarity with ML, Statistics is the field of
niques for power system analysis are classified and detailed in Sec-
Mathematics that deals with the understanding and interpretation of
tion 2. Main data sources and their characteristics are presented in
data. For some Refs. [38–40], Statistics aims to provide an overview of
Section 3. Potential services based on the application of AI techniques
the data set, rather than forecasting or extracting relationships between
in distribution power systems are analyzed in Section 4. Discussions
the data, being generally applied in the preprocessing step of the data
regarding the relationships between the distribution grid applications,
science pipeline. According to [37], learning problems that can be
the connection between AI methods and energy services and the AI
solved by applying statistical techniques can be roughly classified as
methods, trends, opportunities and challenges in the distribution grid
either Supervised or Unsupervised. Defining the boundaries between
domain are conducted in Section 5. Finally, conclusions are drawn in
Statistics and ML can be controversial, and often, some methods are
Section 6.
considered both Statistics and ML, while other references classify the
same method in a specific knowledge domain. According to Bzdok
2. Data-driven techniques for power systems analysis
et al. in [38], Statistics uses a population sample to draw population
The huge amount of data currently being produced alongside the inferences, while ML determines generalizable predictive patterns from
power system pipeline, from generation to demand side, has arisen data.
the opportunity to understand the system better and create innovative In conclusion, the research combines inference and prediction, and
services based on these data. For the sake of this study, the data-driven frequently the classification method is more related to the scientific do-
techniques classification is based on the most relevant publications main where the techniques are applied (i.e., Computer Science, Math-
in the energy field and based on Statistical, Neuroscience, Computer ematics, Engineering) rather than the particularities of the technique
Science and Mathematical references. To help the reader to identify itself.
the different data-driven techniques, Fig. 1 displays the most relevant
data-driven related areas in the energy sector considered in this study. 3. Energy data sources
IRENA defines AI as an area of computer science that focuses on
creating intelligent machines that follow human behavior, according This section identifies and classifies the massive amount of heteroge-
to the data collected [10]. This is considered as the starting point neous data required for developing and operating the distribution grid
for the development of data-driven services in the electricity sector. energy services listed in Section 4. With the objective of accelerating
However, this concept can sometimes be misunderstood or considered the development and deployment of the Smart Grid, a significant
too broad since some techniques inside Statistics and Data Mining (DM) amount of sensor devices have been installed in the distribution net-
are placed within the AI area. work to increase its observability of dynamic and transient events [41]
ML is a sub-field of AI and computer sciences [10,22] that has and collect information about the actual state of the grid, thus achiev-
evolved from pattern recognition to analyze the data structure and fit ing a higher level of monitoring, observability and control beyond
it into models that can be understood and replicated by users [22]. substation level. Nevertheless, not all the needed data come from direct
Fig. 1 defines all the ML categories, methods and models applicable to electrical grid measurements. For instance, [18] distinguishes between
energy-related projects, taking into account the standard approaches electrical and non-electrical information and identifies three categories:
and definitions of different authors [23–26]. The classification pro- measurement data, business data and external data. On the other hand,
vided in this paper matches the one proposed in [26] which is also authors from [8] divide the data sources into operational data and non-
implemented in the power systems field. Furthermore, ML is classi- operational data, whose criterion is used in this study. Diverse studies
fied into four categories: Supervised Learning, Unsupervised Learning, have investigated the data sources available in the grid; for exam-
Reinforcement Learning (RL) and Deep Learning (DL). Supervised and ple, [41] compares eight advanced measurement devices in distribution
Unsupervised Learning categories aim to predict or describe the existing networks and reviews their most recent Smart Grid applications.
relationships within the data set, being called supervised when the The volume of data generated is expected to grow in the upcoming
dependent variable is available and unsupervised when it is not. RL years [8]. As an example, according to [42], the generated data coming
is a computational approach that learns from the interaction with from a single Phasor Measurement Units (PMU) can be estimated at
the environment, which means defining how system agents can take around 100 GB per year. Therefore, new energy-related services and
actions in their environment to maximize the cumulative reward [27]. business models need to emerge to take advantage of this massive data
RL is implemented mainly in energy dispatch problems and building that are still underused or unused. Big Data analytics are essential
energy management scheduling [28]. Some bottlenecks expected in for processing data whose size is beyond the capability of a typical
these applications are the complexity of the objective functions (non- database software tool.
linear and non-convex) plus the limitations of physical models. The The flow chart presented in Fig. 2 shows the steps that data follow
main advantage of using RL instead of predictive model strategies from the time they are collected until an energy service requests it. The
is that RL uses a model-free approach and does not require con- data sources, located at the bottom, are classified into two large groups:
vergence guarantees, thus enhancing its applicability. Moreover, RL operational and non-operational data. The power system operational

3
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Fig. 1. Data-driven techniques classification derived from [22,36] and Machine Learning categories for power systems analysis.

data include all the measurement assets that collect power and energy • Electricity market data: the results obtained from the matching
data, including voltage, current, active and reactive power and grid sta- up of the daily and intraday markets offer relevant information,
tus signals. On the other hand, non-operational data provide essential such as the day-ahead electricity price and the amount of energy
information that has a crucial role in supporting the energy services by generation technology type.
performance, such as weather conditions, electricity market data, social • Social media: through text mining methods, faults in the distribu-
media, Geographic Information System (GIS) and known parameters tion grid or a fire that may harm the electrical infrastructure can
given by customers. The Big Data distribution grid services can request be detected through social network comments.
both real-time and historical data. Once the data has been collected, the • GIS: provides information about the grid components location
next step is to harmonize the data to enhance its usefulness and provide such as lines, transformers and feeders.
a standard structure regardless of the measurement source. The data • Customer behavior data: known parameters related to customers.
harmonized are ingested and saved in the data storage, also known as For example, the number of people living in the house, square
data lake. When an innovative service requests data from the data lake meters, number of rooms and income level.
database, the information goes through a cleansing process to further
increase data quality by eliminating duplicates and imputing values to 4. Big data services in distribution systems
the missing data through AI and statistical techniques.
In this section, a review of a wide range of innovative energy
Primary Big Data sources within the distribution grid are described
services within the distribution network are evaluated. The most rel-
next.
evant AI techniques used in each service are detailed. To obtain a
Operational data: information extracted from the distribution grid
comprehensive outlook, Fig. 3 classifies these energy services into the
measurement devices.
following three categories: (i) distribution grid operation, responsible
• Advanced Metering Infrastructure (AMI): is an integrated system for ensuring the correct operation of the distribution network, (ii)
of smart meters, communications networks and data management flexibility management, in charge of the flexibility market and (iii)
systems that enables two-way communication between power planning, responsible for optimal investment strategies that contribute
utilities and customers [43]. to the long-term planning in the distribution grid. The measurements
error detection service is excluded in the flow chart since it is inherent
• PMU: measures time-referenced magnitudes and phase angles of
to the rest of the services and is not offered directly to the electricity
voltage and current phasors [44].
market stakeholders since its task is to identify, detect and solve anoma-
• Supervisory Control And Data Acquisition (SCADA): collects data
lies, errors, or missing values from data sources to ensure the quality
automatically from distribution grid components, thus facilitating
and usability of the distribution grid services. The main purpose of im-
remote monitoring and control.
plementing AI methods in Big Data energy services is to accelerate and
Non-operational data: information that helps power system utili- stimulate the existing power system towards an environment-friendly,
ties gain deep insights into why some events occur in the grid. cost-effective and reliable Smart Grid. The services are offered to a
broad range of stakeholders involved in the energy domain, including
• Weather data: refers to time-dependent meteorological conditions DSOs, BRPs, prosumers and aggregators, among others. The purpose
such as irradiance, temperature and wind speed. For instance, is to improve their performance and encourage the creation of novel
atmospheric information is vital for forecasting algorithms related business models in the energy sector in order to take advantage of the
to energy systems. massive data that are being generated and underused.

4
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Fig. 2. Power system Big Data sources flow chart.

Fig. 3. Scope of the Big Data services in the distribution network.

4.1. Measurements error detection missing data. Regarding missing data imputation, [45] implements a
Long Short Term Memory (LSTM)-based method for bi-direction data
The measurement error detection application identifies, detects and imputation, [46] applies a Multi-layer Perceptron (MLP) ensemble,
solves eventual anomalies, errors or missing records from data sources while [47] works with Graph-based ANN. The main advantage of using
in order to ensure data quality and usability. Depending on the type AI and Big Data analysis is that it automates and improves the error
of anomaly detected, a correction is automatically executed. The data detection process of the ever-increasing energy-related measured data.
cleaning step in Fig. 2 is responsible for executing these tasks. The For instance, [48] has developed a smart meter data error recog-
following anomalies have been distinguished in operating and non- nition technology applying ANN and Super Vector Machine (SVM)
operating data:
classification algorithms.
• Duplicate records: frequently happen during data collection due
to communication channel problems or combined data sets from
4.2. Operation and monitoring
multiple sites.
• Structural errors: arise during measurement or data transfer.
• Unit inconsistencies: happen when there is a change in the units The operation and monitoring category is responsible for improving
and the past recorded data are not altered. the observability and performance of the distribution grid nearly real-
• Outliers: abrupt and short-duration changes in the consumption time. Data-driven services such as topology, observability and fault
pattern that are not a valid representation of the actual consump- detection are included in this subsection.
tion. The sources of spikes could be mechanical faults of the meter New measurement devices with high granularity and power quality
or storing multiple inconsistent readings for the same timestamp. resolution data like PMUs (120 Hz to 30 kHz and beyond) [8], AMI
• Missing data: occurs when no value is received for an observed and SCADA contribute to strengthening the LV grid monitoring by
variable. providing essential information that assists in comprehending the grid
The measurement error detection sequential scheme is presented in status and identifying possible congestions. The challenge is to monitor
Fig. 4, which identifies possible anomalies (see the dark blue boxes) the distribution grid operating conditions in nearly real-time to check
and proposes what techniques can be implemented to solve them (see its status. Nevertheless, it is necessary first to know the distribution
the light blue boxes). Moreover, a logical order when preprocessing grid topology to identify branches and nodes with technical problems
the data need to be followed. The most suitable technique to ad- in order to generate a quick response to mitigate them. Therefore, the
dress an anomaly might vary depending on each service requirement. topology estimation is an important step to ensure the distribution grid
AI methods are a powerful tool for assigning predictive values for operation and monitoring robustness.

5
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Fig. 4. Measurements error detection service steps.

4.2.1. Topology Table 1

The topology service aims to perform the complete retrieval of Data-driven techniques used in distribution network topology applica-
tions.
the entire LV network electrical scheme. For security and operational
Data-driven technique Ref.
reasons, the transmission system is equipped with real-time measure-
ment devices at each node (bus voltages, line flows, bus injections) Correlation [54,57,62–64]
DNN [68]
to ensure a reliable, robust and accurate topology identification and
DT [55]
observability of the power system [8]. However, topology is commonly LR [63,66,70]
unknown at distribution levels [49] due to the scarcity of real-time LogR [55]
measuring and breaker status devices, which hinders its observability. MLP [51,55]
Nevertheless, due to the ever-increasing presence of advanced ICT
within the distribution grid domain, combined with the constant rise Table 2
of smart meters deployment [50], an enhancement of the original Data sources for distribution network topology estimation.
topology structure is possible. For this reason, it is essential to imple- Data source Input measurement Ref.
ment some of the Big Data techniques explained in Section 2 to be Electricity market Electricity prices [58,59]
capable of processing and analyzing all these amounts of data gener-
PMU Voltage phasor [49,51,55–57,61,69]
ated. In addition, the electrical grid topology is an essential input for
Power injections [53,60]
other energy services like observability, non-technical losses detection,
Voltage magnitude [49,53]
predictive maintenance and aggregated flexibility services in order to SCADA
Current [53]
have outstanding performance and reliable operation. The literature Switch status [53]
distinguishes essentially between transmission and distribution network Power injections [62,70]
topology, although research efforts have recently concentrated on the Smart meters
Voltage magnitude [62–66,69,71]
latter. Depending on the frequency of data collection and the pur-
pose of the service, topology can be estimated in real-time [51–53]
or offline [54,55]. The principal data sources for assessing low and
medium voltage topology are SCADA, smart meters and PMUs, the method [55] considers three classifiers methods -Decision Tree (DT),
latter being the most used in research. As [56] points out, topology MLP and Logistic Regression (LogR)- for predicting the state of the
estimation accuracy depends mainly on the availability and accessi- switch line, where MLP outperforms. On the other hand, [66] presents
bility of the measurement instruments; however, PMU is capable of a data-driven topology estimation method that applies the Linear Re-
achieving satisfactory outcomes even with limited measurements. [57] gression (LR) algorithm and historical voltage measurements as input to
calculates an equivalent grid applying a least-squares model-free ap- the model, where the admittance matrix or switch location information
proach by choosing PMU measurements at a limited number of buses, is not required. In [67], smart meter voltage patterns enable topology
whereas [58,59] use local electricity market prices as input data to identification applying unsupervised learning clustering methods, but
obtain the distribution grid topology. the article does not specify which algorithm is applied. The study
The Alternating Direction Method of Multipliers (ADMM) is a pop- carried out in [68] develops a Deep Neural Network (DNN) system
ular method for distributed convex optimization problems, which de- that interprets the reflected signal from the impedance discontinuities
composes a large problem into smaller sub-problems, enhancing its in the network, which gives the possibility to determine the topology at
robustness. The topology estimation problem is formulated and solved the reflection site. The authors of [69] propose a Supervised Learning
using ADMM in [58–61]. The main advantages of this method are framework that first estimates the parameters and an Unsupervised
that it allows handling and solving large-scale data problems and the Learning model to identify the topology. The proposed algorithm per-
implementation is straightforward. On the contrary, the convergence forms well in the radial and loopy distribution networks. In [70], a
could be slow. LR method is proposed to evaluate the distribution network topology
Regarding statistical methods, correlation is widely used in litera- changes. Unfortunately, this method is accurate only if there is no noise
ture as a tool for topology estimation. For instance, [54] analyzes the in both input and output measurements [69].
correlation among voltage measurements to determine the grid topol- Data-driven techniques found in the literature for topology esti-
ogy, meanwhile [62] proposes a correlation-based algorithm to identify mation are listed in Table 1. The input data required for topology
the transformer and phase to which a customer is connected. [63] estimation is shown in Table 2.
reconstructs the topology given the voltage magnitude measured from
smart metering devices, formulating the adaptive Lasso algorithm to 4.2.2. Observability
obtain the correlation coefficient matrix. Another study [64] calculates The observability service assesses the most probable state of the dis-
the correlation coefficient among voltage measurements of smart me- tribution network state in nearly real-time. Potential applications like
ters under the same distribution transformer and is capable of grouping congestion management, optimal voltage/power control, fault detec-
customers that belong to the same phase effectively. Ultimately, [65] tion and non-technical losses detection, among others, required instan-
uses a statistical learning framework for verifying single-phase grid taneous information regarding distribution system status to perform
structures using non-synchronized voltage data. accurately.
Regarding AI-based methods, a binary classifier based on ANN iden- For classic state estimators, power system measurements and
tifies the status of a distinct line [51]. A learning-to-infer variational switching device statuses are collected using SCADA systems, but

6
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

the downside is that SCADA sampling rates are slow [72]. On the 4.3. Predictive maintenance
contrary, PMUs provide high power quality resolution data in real-
time. Therefore, the innovation in the observability field attempts to The predictive maintenance service intends to dig out the poten-
include real-time operation data to provide a continuous and safe tially valuable information from the collected sensor data located in
state estimation of the distribution grid and applying AI methods to the electrical equipment within the distribution network to help make
develop reliable data-driven solutions. It should be noted that errors decisions on the scheduling maintenance actions to anticipate an im-
made in topology estimation might downgrade the performance of the minent failure [103]. Furthermore, the scheduled maintenance through
observability service [56]. Smart meters are commonly applied for state predictive maintenance models is more cost-effective than repairing af-
estimation, for example, in [73,74]. The authors in [74] analyze the ter failure [104]. Concerning the energy system field, the vast majority
optimal positioning of smart meters for optimal cost-effective operation of the predictive maintenance research centers on high-power wind
of the distribution grid to increase the state estimation accuracy, turbines as the aim is to reduce the high operating and maintenance
while [75] implements real-time PMU measurements to monitor the costs. However, wind turbines are out of the scope of this investigation,
distribution grid status. A hybrid state estimation using AMI and which concentrates on the distribution level domain.
SCADA measurements is formulated in [76]. The following authors focus on power transformers predictive main-
Concerning data-driven methods, ML algorithms – DNN, SVMs,
tenance [105–111]. The study presented in [105] reviews and identifies
Recurrent Neural Networks (RNN), among others – are used in [72]
the monitoring methods for predictive maintenance of electric power
to develop a sophisticated power system status monitoring using a
transformers and identifies the operational lifetime degradation factors.
Big Data platform. According to [77], the distribution grid might be
Another study conducted by [112] reviews recent articles that apply ML
near-optimal observability shortly thanks to the improvement of real-
for predictive maintenance, including power transformers and PV pan-
time devices and AI-based technical solutions. A DNN approach for
els. [107] analyses the different operating periods of an oil-immersed
unobservable systems is presented in [78]. This work overcomes the
power transformer through dissolved gas concentrations data. The K-
computation complexity in Bayesian estimation, although it is less
capable of adapting to outage changes, in addition to the fact that deep means clustering method groups the operation periods in different
learning training algorithms are still under research. classes characterized by the production activities of several gases and
A significant limitation for observability in distribution systems the incipient failures. Ref. [108] presents a predictive maintenance ML
is the lack of sufficient real-time and high granularity measurement method for power transformers based on RF and Adaboost. The results
devices such as PMUs. Although they are being deployed, their high conclude that the Adaboost algorithm provides better results than
cost prevents installing the required sensors to make the system fully the RF. The main disadvantage of data-driven prediction maintenance
observable. research is that high-resolution power quality data are mainly applied
to validate their investigations. However, it is unlikely to have such
4.2.3. Fault detection data in real-life distribution grid scenarios.
The fault detection service intends to recognize and locate unusual Concerning real-world trials and companies, Enel Distribution util-
electric currents within the distribution network. Two fault detection ity tests its predictive transformer maintenance monitoring in [111].
approaches are distinguished [79]. On one side, data-driven methods These data are of great interest to the distribution utility, as it pro-
seek a pattern recognition of measurement readings gathered from vides faster detection of anomalies, life loss and a more profound
sensors placed in different points of the network that indicate a fault. understanding of the grid for future expansions. The company Neuron
On the other, model-based approaches compare real data from sensors Soundware [113] has developed a predictive maintenance solution
with prediction model results. High residual differences might indicate powered by AI and IoT for power utilities, covering from transformers
an electrical fault. to motors, ensuring more than a 50% reduction in mechanical failures.
The leading causes for electrical faults are damaged equipment, en- Another company named Predictive Layer [114] offers a ML tool
vironmental events, falling tree limbs and direct animal contacts [80]. for selective predictive maintenance, among other energy-related use
Concerning natural phenomena-generated faults, [81,82] propose AI- cases.
based approaches to predict blackouts due to convective storms [82] All recent studies found containing AI techniques for developing
or ice-wind events [81]. Several works center their attention on the predictive maintenance models are listed in Table 4. Besides, the equip-
LV domain considering different data-driven methods [83–88]. Real- ment and data used to perform the prediction model are indicated.
time anomaly detection is proposed in [89,90], where [89] defines an
approach using smart mater large-scale consumers data, [90] formu-
4.4. Non-technical losses detection
lates a Convolutional Neural Network (CNN) considering bus voltages.
After a fault occurs, [91] proposes a DT approach to identify the power
Electricity losses at distribution levels encompass both technical and
line-fault cause based on historical fault records.
non-technical losses (NTL). The first one occurs due to Joule’s effect,
Some studies focus on Microgrid faults detection [92–96]. A Ran-
while NTL refers to the electricity consumed but not billed [117].
dom Forest (RF) classifier model is used to detect unexpected Microgrid
In other words, energy is illegally taking by unidentified end-users
islanding problems from normal operation conditions [92] that can be
without the awareness of the energy utility. Detecting and addressing
located by knowing the topology. [93] proposes RF, K-Nearest Neigh-
bors (KNN) and SVM to detect faults in Microgrids, while [94] employs electricity theft is an essential task for power companies. For instance,
Extreme Learning Machine (ELM) for the classification and location of Endesa has developed a fraud detection system currently in opera-
outages. An MLP classifier detects and isolates the fault in [95] and [96] tion [118]. In 2018 the system was capable of detecting 65000 cases
applies the ensemble bagged DT classifier to detect dynamic events of electricity fraud, recovering 601 million stolen kWh. This number is
within the power system. Clustering techniques, such as Principal equivalent to power Palma de Mallorca for six months [119].
Component Analysis (PCA) [93] and Density-Based Spatial Clustering Some articles review in-depth data-driven techniques applied within
of Applications with Noise (DBSCAN) [82,97] are implemented. The the NTL field. A comprehensive review is presented in [117], which
following papers cover a fault diagnosis in PV systems [98–100], using compiles the primary techniques, including AI, and the data used
different AI-techniques and input data, as shown in Table 3. to detect energy thefts, exposing the limitations of the current solu-
Recently published articles that use AI techniques to detect and tions. [120] focuses on Big Data oriented to anomaly detection, which
predict faults within the distribution network are listed in Table 3, is a powerful mechanism for fraud detection, while [121] examines the
considering the equipment of power lines [81,82,86] and underground ML classifiers for electricity thief detection.
cables [101], among others. In addition, the needed input data to ML methods assist in improving the accuracy of fraud detection
perform the fault detection problem in each reference is specified. solutions. Unsupervised Learning clustering techniques are capable

7
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Table 3
Data-driven techniques used for detecting faults in the distribution grid domain.
Data-driven technique Equipment Data used for detecting faults Ref.
GBM Distribution grid Time, load, generation, current, voltage [83]
DBSCAN, RF, DNN Distribution grid Area, lighting density, rain intensity, storm [82]
parameters, air temperature and pressure, wind speed,
precipitations, snow depth
RF Distribution grid Fault data set [84]
MLP, ELM Distribution grid Three-phase current signal [88]
ELM, SVM, MLP Distribution grid Measurements of three-phase fault currents [85]
SVM Distribution grid Traced current of distribution feeder [86]
PCA, RF, KNN, SVM Microgrid Current and voltage signals at each end point of the [93]
line
ELM Microgrid Fault current signals [94]
MLP Microgrid Current modules in the DC Microgrid [95]
Ensemble DT Microgrid Distributed generator data [96]
CNN Power system Bus voltage [90]
MLP Power line Line impedance, reflection coefficient and the channel [102]
transfer function in the PLC signal band
CNN PV systems PV loop current [98]
RF PV systems PV array voltage and string currents [99]
DNN PV systems PV module parameters [100]
DNN Underground power cables Power line modems [101]

Table 4
Data-driven techniques used for prediction maintenance in the distribution grid domain.
Data-driven technique Equipment Data applied for predictive maintenance Ref.
Correlation analysis Line conductors, cables, Equipment’s component outage failure data [106]
breakers and transformers
CNN Photovoltaic panels Electrical power signal [115]
K-Means, PCA Power transformer Dissolved gasses concentrations [107]
LSTM Underground power cables Voltage, active power and current. [116]
MLP Power transformer Age transformer, loading, meteorological data [109]
RF, Adaboost Power transformer Transformers’ specification, loading, location and [108]
meteorological data
SVM Power transformer Prosumer data and infrastructure [110]

of grouping customers according to their consumption profiles, thus Table 5

detecting suspect load curves of end-users. For instance, [122] calcu- Data-driven techniques used for detection of non-technical losses.

lates regular consumption behavior by clustering data collected from Data-driven category Data-driven technique Ref.
smart meters to identify NTL [122]. Moreover, distinguishing outliers Supervised Learning
in demand profiles aids to monitor and detect suspicious customers Classification DT [118]
RF [125,132]
by identifying abnormalities in consumption patterns [123,124]. The
SVM [123,126,127,129,131–136]
NTL classification algorithms achieve better performance results thanks Adaboost [128]
to clustering techniques [125]. The most common evaluation metrics KNN [131]
found in the literature for NTL classification approaches are accuracy, GBM [131]
recall, precision, F-value and Matthews correlation coefficient. One of LogR [132]
the main challenges found when building an NTL classification model Unsupervised Learning
is the lack of abnormal and irregular consumption data, a fact that Clustering K-means [123,128,130]
SOM [125,130,137]
is known as data imbalance. The following articles [126–128] take
Fussy C-Means [130]
into consideration imbalanced data sets for NTL detection, while [129] Gustafson–Kessel [122]
proposes strategies for improving imbalance data performance. Some Dimensional reduction T-SNE [131,138]
events that could alter classification and clustering algorithms are the Deep learning
change of residents or the purchase of new devices, such as EVs [123]. CNN [131,138–140]
The most common data source applied to identify NTL using AI tech- LSTM [139,140]
niques is the customer consumption data, followed in the distance by Statistics
customer information (such as location, complaints made and overdue Bayesian network [118]
bills) and load, voltage and current measurements [117]. The authors Pearson coefficient [118]
in [130] prove that it is possible to use only a small data set of recent Outlier detection [124]
smart meter measures to define the customer consumption pattern. Data Mining Data mining [118,122,124,135,137]
Besides, studies generally focus on residential consumption to detect
electricity fraud, leaving aside industrial consumers. [131] justifies this
event as industries do not have a fixed electricity consumption pattern.
Recent data-driven related articles for detecting NTL are listed in
To conclude, traditional theft detection methods are mainly based
Table 5.
on on-site line inspections, which are highly expensive, time-consuming
and inefficient. In contrast, AI-based NTL detection methods are su-
perior to conventional methods in terms of accuracy, time-consuming 4.5. Forecasting
and labor required, but more irregular and abnormal historical data
are needed in order to train the models optimally. The most popular The forecasting service implements AI methods for demand, gen-
ML algorithm is SVM for classification tasks, while K-means is widely eration, electricity price and flexibility prediction, which are essential
popular for clustering consumption patterns. to deal with uncertainty and risk management within the distribution

8
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

grid. Furthermore, energy-related forecasting provides essential input of each appliance directly from the smart meter data. Recent literature
for demand response programs [141]. The purpose of this section is covers different load identification methods that classify the assets
not to conduct an in-depth forecasting review but to expose recent behind-the-meter for NILM methods using AI techniques [167–172].
studies that apply AI techniques within the forecasting domain. The Unsupervised clustering techniques are implemented to classify
most relevant and recent review articles are revealed in each subsection buildings based on their energy efficiency [12], determine natural seg-
to allow the reader to delve deeper into the subject. mentation of customers [173], identify appliances usage patterns [172,
Forecasting horizons are classified in three categories [142–147]; 174], estimate electricity consumption behavior patterns in house-
short-term, medium-term and long-term; although some literature adds holds [173,175,176], group households profiles patterns to achieve
a fourth category: very short-term or real-time forecasting [145,148– better forecasting outcomes [177–180] and to identify peak demand
150]. Table 6 specifies what applications and time range covers each profiles or electricity theft detection [123]. Dimension reduction of
forecasting horizon. Most of the articles focus on short-term forecasting, NILM features is applied in [181]. DT and Naive Bayes (NB) are used
as [146] also points out. Concerning long-term forecasting, they are to identify residential device loads [182].
influenced by economic growth, policy adjustment and technology Studies focus mainly on short-term prediction for network operation
advancement, making it a complicated task [147].
purposes. In contrast, there are barely any long-term studies. As for
The following studies review different types of predictions related
the latter, long-term forecastings require residential and non-residential
to the energy sector. For instance, [149,151–153] investigate both load
inputs, as historical gross domestic product or population, to estimate
and price forecasting models. In addition, [147] reviews ML algorithms,
demand in the following years, applying data-driven approaches such
ensemble-based approaches, and ANNs implemented for renewable
as Multiple Linear Regression (MLR) analysis [183]. It is concluded
energy generation, load demand and electricity price forecast.
that long-term prediction models combine energy, economic and en-
vironmental fields for planning the energy future in a sustainable
4.5.1. Demand forecasting
manner [163].
The demand forecasting service predicts the consumption profiles
of a single or several end-users. For instance, the day ahead aggregated Table 7 describes the AI techniques used for developing load fore-
demand forecasting of a particular zone is essential for the DSO to fore- casting models for different time horizons and locations, identifying the
see possible congestion in the network. The emergence of prosumers – main evaluation metrics.
a consumer who produces and consumes energy – in the power system
has caused an increment in the uncertainty of the demand profile, 4.5.2. Generation forecasting
as consumption has become more unpredictable and volatile due to The generation forecasting service aims to predict the electric-
demand response programs and weather conditions that affect in last ity production of renewable sources within the distribution network.
stay the end-users self-consumption. The system operator does not have The continuing increase of renewable energy sources and demand-
information regarding the self-consumption behind each smart meter; side flexibility programs in power systems has raised the need for
therefore, predicting consumption becomes even more complex. more accurate Renewable Energy Systems (RES) predictions. Regarding
Reviews focusing on AI load forecasting techniques are conducted recent studies that evaluate AI methods applied to RES prediction mod-
in the literature. For instance, a study comparing conventional and els, authors in [147] review multiple renewable generation sources,
AI-based models for energy forecasting is carried out in [160]. A including distributed wind, solar and geothermal energy, considering
systematic review is presented in [161] and it concludes that regres- various forecasting horizon ranges. Key findings state that benchmark
sion models are the most suitable for long-term scenarios, whereas ML models handle a large amount of data with accurate forecasting
ML algorithms outperform for the short-term forecasting horizon. The outcomes; however, ensemble ML models could achieve even further
combination of different ML algorithms is analyzed in [162]. In a accuracy by combining different data-driven techniques. Concerning
narrower framework, [160] presents several research papers that im- solar generation, [200] presents a review on PV forecasting based on
plement data-driven models for building scale forecastings. A survey ML and metaheuristic techniques while [201] focuses on time-series
of statistical and conventional methods for demand forecasting is pre- statistical, physical and ensemble methods. A review of the state of
sented in [163], concluding that Autoregressive Integrated Moving
the art of SVM in the application of solar and wind forecasting is
Average (ARIMA) statistical model combined with ANN increases the
conducted by [202]. SVM regressor is simple-to-use and reliable, but
accuracy of predictions. The authors in [164] review load forecasting
on the contrary, it is not suitable for large data sets and it has a low
methodologies based on previous literature, classifying them into four
performance for high noise data.
forecasting methods: similar patterns, variable selection, hierarchical
Multiple ML methods such as ANNs, SVMs and Gaussian Process
forecasting and weather station selection.
Regression are studied in [203] for wind and solar power generation.
The most common and relevant input data found in the litera-
The authors in [204] extensively compare simple and sophisticated PV
ture for demand forecasting are mostly historical demand data along
forecasting methodologies and concludes that some methodologies are
with seasonal factors like weather and calendar data [12,165]. Ac-
cording to [141], essential parameters for system electricity demand more suitable under different weather conditions. The work presented
are weather data and random effects, such as maintenance work or in [205] studies day-ahead PV forecasting models based on deep learn-
customer behavior, where the game theory approach helps predict ing neural networks. Multi-site PV forecasting is examined in [206]
erratic performance [166]. using CNN. In [207], a review of the main ML methods for forecasting
Three demand forecasting levels are distinguished: the aggregated wind speed and power is carried out, including weighting-based, data
demand within an area or zone, the aggregated demand in a build- preprocessing, parameter selection, optimization and error processing
ing/household through smart meters and the disaggregated demand, methods. These combined approaches generally outperform the single
which predicts the consumption of electrical appliances behind the models approach. Authors in [208] present a review of ANN imple-
meter thanks to the implementation of sensors that store their con- mented in wind energy systems, combining the main methods applied
sumption data (also known as sub-metering). Disaggregated demand in forecasting models, identifying strengths and weakness. The Wavelet
forecasting plays a vital role in DR programs; hence, data are crucial to transform is used to decompose the raw data into different frequencies.
predict their consumption to make accurate and optimal load schedul- It is applied to mitigate spikes and fluctuations in raw data [209]. This
ing. On the contrary, in case of not having a sensor for each flexible method has been implemented in several studies [209–212].
load, the Non-Intrusive Load Monitoring (NILM) method identifies each A problem faced by the DSO and BRPs is the lack of knowl-
assets’ consumption curve in order to predict the power consumption edge of the aggregated small-scaled solar generation of prosumers. To

9
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Table 6
Forecasting horizon classification.
Forecasting horizon Time interval Applications
Real-time t ≤ 1 h Keep the power system balanced [154]
Short-term 1 h < t < 1 week Deregulated electricity markets [146,155–157]
Real-time energy management systems [144,145]
Optimal management of power system [157]
Medium-term 1 week < t < 12 months Assess environmental impacts, maintenance scheduling [146,158]
Long-term t > 1 year Long-term investment and political decisions [145,159]
Grid expansion criteria.
Decision tool for buying futures of the spot product [159]

Table 7
Load forecasting models and evaluation metrics.
Data-driven Forecast Location Year Highlights Evaluation metrics Ref.
technique horizon
MAE MAPE RMSE Other
Aggregated demand forecasting
DNN, RF Short-term US 2019 Electricity consumption for residential buildings ∙ [180]
for the next day.
MLP Short-term France 2018 Wavelet decomposition to capture the various ∙ ∙ [184]
seasonal cycles in electricity load data.
DNN Short-term Australia 2019 Advanced data preprocessing strategy. DBN has ∙ ∙ ∙ MSE [185]
outstanding data learning and forecasting
capabilities
ARIMA- Short-term US, Australia 2018 The WNN has a strong ability to fit the nonlinear ∙ ∙ ∙ [186]
WaveletNN component of the electricity load
SVM Short-term US 2018 The features extracted by the auto-enconders ∙ [187]
forecast day-ahead load forecasting more
accurately
CNN-LSTM Short-term UK 2019 Day-ahead aggregated load forecasting based on ∙ ∙ MSE [179]
two-terminal sparse coding and DNN
SVM, ELM Mid-term UK 2020 ELM performs better than SVM for 1 week ∙ ∙ ∙ MSE [188]
prediction
Autocorrelation- Mid-term Australia 2019 Forecast next week electricity demand ∙ ∙ 𝑅2 [189]
SVM
MLP Mid-term US 2020 Optimal training algorithm composed of two PSO ∙ MSE [158]
and ALO
MLR Long-term West Africa 2019 Hourly and annual electricity consumption ∙ ∙ [183]
estimation for 2030 in 14 different West African
countries
Multiplicative Long-term US 2018 Monthly aggregated load prediction for a horizon ∙ MSE [190]
error model of four years

Smart meter load forecasting

LSTM Real-time China 2020 Demand forecasting from an industrial steel plant ∙ ∙ ∙ [154]
LSTM Real-time Australia 2020 Probabilistic household load forecasting under high ∙ ∙ [191]
uncertainty and volatility
DT, RF Real- Spain 2018 Focuses on online environments where data are ∙ ∙ ∙ [192]
time/short- analyzed as they arrive
term
DNN Short-term US 2020 Aggregated households load forecasting ∙ ∙ 𝑅2 [193]
LSTM, MLP Short-term US 2019 Energy Big Data is used as data set for load and ∙ ∙ [151]
price forecasting
LSTM Short-term Ireland 2018 The model aims to learn the uncertainty by ∙ ∙ [194]
applying a pooling Deep RNN. It is tested in 920
smart meters
CNN Short-term South Korea 2019 Single residential load forecasting using CNN ∙ ∙ [195]
combined with data-augmentation technique
DNN Short-term China 2018 Power load and probability density forecasting ∙ ∙ [196]
LSTM Short-term Australia 2019 Individual and aggregated residential load ∙ [197]
forecasting
MLP, SVM, MLR Short/Long- Cyprus 2020 SVM and ANN achieve the best outcomes ∙ [198]
term
K-means-MLP Short-term Honk Kong 2020 Day-ahead office building cooling demand, ∙ [199]
grouped in seasons

solve this problem, [213] estimates the aggregated power generation 4.5.3. Electricity price forecasting
of small-scale rooftops solar sites that are not monitored by system This service aims to predict the electricity price, which is essential
operators. for minimizing the energy purchase invoice for BRP and retailers com-
Table 8 shows the AI techniques applied for developing distributed panies in the short-term horizon. The most relevant and recent research
generation forecasting models for different time horizons and locations, regarding Big Data and AI methods for electricity price forecasting is
identifying the main evaluation metrics. conducted in [145,146,155,156,198,227,228]. ANN models for day-

10
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Table 8
Generation forecasting models and evaluation metrics.
Data-driven Forecast Location Year Highlights Evaluation metrics Ref.
technique horizon
MAE MAPE RMSE Other
Solar forecasting
LSTM Real-time Cape Verde 2018 Hourly day-ahead solar irradiance prediction by ∙ [214]
using weather forecasting data
2
LSTM Real-time Australia 2020 Five-minute forecasting horizons. Model based on ∙ ∙ 𝑅 [215]
short-term multivariate historical data sets
LSTM Real-time Egypt 2019 Predicts the PV power in the next hour ∙ [216]
CNN Short-term China 2018 Thanks to the CNN advanced feature extraction, ∙ ∙ MASE [217]
more meteorological features are introduced in the
prediction mode
DNN Short-term Australia 2018 Based on particle swarm optimization and trained ∙ [212]
feedforward neural network (FNN)
GRNN, ELM, Short-term US 2018 Predicts also the PV output associated uncertainty ∙ [218]
ElmanNN at different confidence levels
MLP, DNN and Short-term South Korea 2019 PV prediction only with meteorological and ∙ ∙ [219]
LSTM calendar data. LSTM algorithm presents the best
outcomes for all seasons
SVM Short-term China 2018 Model based on SCADA and meteorological ∙ [210]
information
MLP Short-term Spain 2018 Correlation analysis of main variables. High ∙ [220]
accuracy
LSTM Short-term China 2019 Uses the attention mechanism to focus on the most ∙ ∙ ∙ [221]
significant input features in forecasting
CNN Short-term US 2019 Multi-Site Photovoltaic Forecasting ∙ ∙ MASE [206]

Wind speed/power forecasting

CNN-GBM Real-time China 2019 Ultra short term wind power prediction. LightGBM ∙ MSE [222]
improves performance of single CNN
RF Real-time Tunisia 2017 The spatial average of the wind speed, its ∙ ∙ ∙ MASE [223]
direction and past power values are the inputs.
LSTM, SVM Real-time Mongolia 2018 Performs ten-minutes and one-hour ahead ∙ ∙ ∙ 𝑅2 [224]
forecasting with extremal optimization
CNN Real- China 2017 Probabilistic wind power forecasting using the ACE, IS, CRPS [209]
time/short- wavelet transform
term
LSTM-ElmanNN Short-term China 2018 The wavelet transformation is used. Its ∙ ∙ ∙ [211]
performance is compared with nine models
K-Means-LSTM Short-term China 2019 The K-Means forms clusters of wind power impact ∙ ∙ ∙ [225]
factors to generate a new LSTM sub-prediction
model.
Ensemble DNN Short/Mid- Europe 2017 Uses a deep sparse auto-encoder and transfer ∙ ∙ SDE [226]
term learning during the training phase of
base-regressors

ahead market price forecasting are reviewed in [155,156], and [156] natural gas price [237] if the energy mix is highly carbon-dependent.
concludes that simple ANN models do not perform properly when ELM techniques improve the generalization performance and learn
electricity price time-series present high volatility, sharp price spikes faster than ANN trained using back-propagation [153,239–241].
and chaotic and non-linear behavior. Therefore, more sophisticated A summary of the recent AI techniques applied for electricity price
techniques are required to handle complex predictions. Techniques forecasting in literature is presented in Table 9 for different forecasting
based on univariate and multivariate forecasting models are compared horizons and locations, together with the main evaluation metrics used.
and covered in [145], while [228] focuses on benchmark techniques, Finally, Fig. 5 displays a bar graph showing the most used evaluation
from statistical to ensembles. Authors in [146] classify electricity price metrics in recent literature within the forecasting field.
predictions in market equilibrium, structural, statistical, intelligent and
combination models, separating short, mid and long-term estimations. 4.5.4. Flexibility forecasting
Lastly, feature engineering for linear, ensembles and deep ML models The development of aggregated flexibility forecasting services per-
is studied in [229]. mits to delimit the accumulated feasible flexibility in a default area
Most literature focuses on short-term electricity price forecasting by aggregating flexible loads, distributed/centralized storage units
while medium and long-term predictions are not covered in sufficient and Distributed Energy Resources (DER) [245]. Using the aggregated
depth, as [156] also points out. However, [146,159] overviews elec- flexibility in the distribution grid reduces the need for grid exten-
tricity price forecasting for mid and long term, which is essential for sion [246] and enhances the technical and economic power system
distribution network planning purposes. operations [247]. In [248], the aggregated flexibility calculation is
Deep learning models are widely used in literature to estimate elec- discussed in more detail.
tricity prices [151,152,228,230–235], along with SVM regression [231, The aggregator is the service provider in charge of gathering and
232,236,237] and tree-based models [192,238]. RNN has been pro- controlling its portfolio flexibility sources [249,250] in order to (i)
posed to address time-dependent learning problems. In particular, provide flexibility services to power system agents, (ii) to minimize
LSTM and Gated Recurrent Units (GRU) have an extraordinary perfor- the end-users energy bill through Home Energy Management Systems
mance for time series price estimation according to [234]. The most and (iii) to participate in electricity markets, by using optimal bidding
accepted inputs are historical electricity prices and calendar data. Elec- strategies [251,252]. New flexibility business model approaches are
tricity price has a strong correlation with other variables like oil and developed in [253]. Due to the increasing penetration of intermittent

11
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Table 9
Price forecasting models inputs and evaluation metrics.
Data-driven Forecast Location Year Highlights Evaluation metrics Ref.
technique horizon
MAE MAPE RMSE Other
CNN-LSTM Real-time USA 2018 Hybrid DNN performs better than traditional ML ∙ ∙ [230]
models
GBM Real-time USA 2019 Accurate and computationally inexpensive ∙ ∙ ∙ [150]
SVM, MLP, DNN Real-time USA 2019 DNN obtains less error. Higher accuracy achieved ∙ [232]
by diversifying the data source
ELM Real-time Australia 2019 Improves the forecast accuracy in real-time when [239]
unexpected dynamic price change occurs.
Dynamic Trees Real- Spain 2018 Dynamic Trees perform better than RF and are an ∙ ∙ ∙ [192]
time/short- adequate method for real-time and short term
term
SVM, LSTM Short-term Denmark 2019 DL model outperforms the SVR ∙ [231]
DNN Short-term Australia 2016 Good performance for high volatility prices ∙ [242]
DNN Short-term Spain 2018 Inconsistencies were observed as layers were ∙ [233]
increasing when using few input variables. Model
performs better with more historical data
DNN, LSTM, Short-term Belgium 2018 Compares the 4 proposed DNN with 23 benchmark ∙ [228]
GRU, CNN models for electricity price forecasting. DNN,
LSTM, GRU outperform literature models.
Hybrid Short-term Australia 2020 The model can be a reliable forecasting method in ∙ ∙ ∙ [240]
outlier-ELM modeling time series with complex nonlinear
characteristics and outliers
Neuro-fuzzy Short-term USA 2020 This study reveals the efficiency of neurofuzzy ∙ ∙ ∙ [243]
ANN models against MLP neural network and ARIMA
statistic model
GBM Short-term Spain 2019 Hour feature is the most relevant predictor in the ∙ ∙ ∙ [238]
model
ELM Short-term Canada 2020 MKELM model provides better performance as ∙ ∙ ∙ [241]
compared to the ELM and KELM
Dimension Short-term Canada 2020 Proposed method is recommended to studies with ∙ ∙ ∙ [157]
reduction, DNN, a large volume of input data. The feature
SVM, LSTM extraction tool and rough neurons improve the
forecasting results.
MLP Short-term Spain 2018 This study demonstrates the improvement in ∙ [42]
convergence speed with Tensorflow software
SVM Short-term Germany 2015 Oil and natural gas prices are considered in the MSE [237]
prediction model due to its high correlation with
electricity prices
LSTM Short/mid- USA 2019 Deep LSTM gives better results compared to ELM ∙ ∙ [151]
term and NARX
GRU Short/mid- Turkey 2018 The three-layered GRUs outperformed all other ∙ [234]
term ANN structures and statistical techniques. Stacking
multiple layers increases the performance.
Weighted KNN, Mid-term Germany 2019 DNN outperforms Weighted KNN, a model based ∙ ∙ [235]
DNN on autocorrelations in data, providing good
accuracy forecasts even 29 days ahead
Jaya-LSTM Mid-term Belgium 2020 Hyperparameters tuned using Jaya optimization ∙ ∙ [152]
multivariate LSTM algorithm leads to better
performance than SVM and univariate LSTM
Co-integration Long-term Spain 2016 Brent crude oil spot and futures price along with ∙ [244]
and vector error the Spanish wind generation are the variables that
correction yield most accuracy

RES and the significant number of residential users with potential

flexible sources, demand-side flexibility aggregation becomes essential
for balancing the future power system [252,253].
ML-based regression models are applied in [247] to forecast the flex-
ibility of residential customers for real-time applications. A flexibility
forecasting concept and its control from multiple energy domains and
sources are presented in [246]. The GBM ensemble algorithm is selected
by [254] to build a flexibility load forecasting model for DR capacity
scheduling.
Intending to encourage a change in the demand-side consumption,
the aggregator offers a monetary incentive signal in [255]. The flex-
ibility potential of wet appliances in France (dishwasher and washer
machine) is estimated in [256]. In [253], residential load flexibility
forecasts are calculated using the NILM approach. Predefined cus-
tomer preferences and loads and PV forecast uncertainty are considered
in [257] to define a feasible flexibility space from controllable residen-
tial resources. [258] studies the required percentage of end-users with
Fig. 5. Forecasting evaluation metrics used in literature. sub-metering capabilities needed to calculate the aggregated demand

12
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

composition. The results state that only a 5% of sub-metering coverage RL is used as a decision-making tool in EMS for scheduling and
is required to forecast the aggregated load composition at the substa- controlling flexible units such as EV [274,275] and other flexible
tion level with high confidence. A scalable and non-intrusive model for controllable loads [267,268,273,275–281], Energy Storage Systems
identifying the flexibility of thermal loads is proposed in [259]. (ESS) [279] and PV generation [275,279,281], in order to solve dif-
ferent problems like hour-ahead [267,275] and day-ahead [277,280]
4.6. Energy management systems energy consumption scheduling [267,275]. The Q-learning algorithm
is the most common RL method applied in DR programs [261,267,
The massive amount of data generated by the rapid deployment 275,277,279,281]. The Q-learning algorithm needs many iterations to
of smart meters in recent years supports the enhancement of building converge, while the batch RL usually converges much faster [282].
energy efficiency and DR programs (e.g., price-based, incentive-based Batch RL is applied in thermal controlled loads in order to find the
and environmental-based [260]). This subsection focuses on the IA day-ahead schedule [273,280] to minimize costs.
techniques used in EMS at building level. In most of the articles reviewed, there is a lack of RL experimental
The application of AI methods allows overcoming multiple chal- results in real small and large-scale physical systems since many of the
lenges related to energy management, developing better tools for auto- proposed methods are only tested in simulated environments. [261,
matic decision-making to schedule and control multiple energy assets 271]. The scarcity of physical experimentation could be one reason
through the EMS. From 2013 onwards, there has been a perceptible that prevents buildings and households from adapting RL algorithms, as
increase in the AI approaches across DR applications. These AI methods their reliability and performance in real-world scenarios are unproven
have been principally applied to price-based programs and residential yet. Moreover, the vast majority of the reviewed articles are single-
consumer types, followed by small-scale industrial and commercial agent system based, which means that they only focus on a single
buildings [261]. building, bypassing urban boundary conditions. The single-agent ap-
Numerous papers have reviewed AI approaches for energy DR pro- proach is correct when very few buildings participate in DR programs.
grams. In a more general context, [261] investigates the state of the However, if a large number of buildings use DR schemes, a Multi-agent
art of DR applications and analyses the AI methods applied in different approach is needed in order to address the computer limitations prob-
DR scheme categories and consumer types. An extended summary of lems of centralized approaches by distributing the workload among the
companies, start-ups and European-funded industrial projects using participating agents in order to make decisions for various buildings
AI for DR is also provided. Regarding home appliance schedule con- devices in a decentralized manner, while maintaining data privacy of
trollers for DR programs in smart households, [262] reviews various costumers. Thus, the multi-agent approach avoids shifting the peak
AI techniques based on ANN, fuzzy logic control and adaptive neural demand to lower-cost periods, for instance.
fuzzy inference system, which imitate human thinking behavior. More The main drawback of the articles reviewed in this subsection is
specifically, [263] reviews the existing AI-based methods for cloud
that they assume complete knowledge of the end-user environment,
EMS with the integration of blockchain technology. However, the high
although this is unlikely to happen in reality. It is worth mentioning
development cost and storage of blockchain and the lack of standard-
that the primary focus in literature is on price-based DR programs; nev-
ization and professional expertise in this topic represent a research
ertheless, a more comprehensive range of incentive-based DR schemes
challenge that should be addressed in the upcoming years.
should be developed and tested, as it is indispensable for the optimal
The following studies apply Supervised and Unsupervised Learning
operation and balance of the distribution network. Only a few articles
techniques. A MLP deep learning model is used in [264] to optimize
modeled the EMS controllable appliances with a high level of detail.
load consumption and storage management in response to dynamic
Unlike the traditional model-based methods, the RL approach does
pricing. A deep ANN and Genetic Algorithm reduces energy demand
not require any system model information. Finally, Table 10 classifies
in peak periods, optimizing the residential appliances scheduling and
the data-driven technique, DR program and customer type for each
RES generation [265]. Supervised Learning algorithms as DT and Naive
reference.
Bayes identify loads through smart plugs [182] for EMS. [266] creates
a control-oriented model for a heating system based on regression trees
and RF. A steady price prediction model based on ANN deals with price 4.7. Aggregated flexibility services
uncertainty for EMS in [267]. [268] develops a residential scheduling
controller using the hybrid lightning search algorithm ANN to predict The aggregated flexibility service is responsible for gathering the
the optimal ON/OFF status for home electrical appliances after a DR flexibility from different customers and offering flexibility services to
event imposed by the power utility to reduce peak consumption. [269] potential energy agents such as residential and industrial consumers,
uses ANN in order to predict and schedule building appliances energy BRP and DSO. Benefits derived from flexibility and DR programs in-
consumption and genetic algorithms for task scheduling, while [270] clude shifting or reducing peak demand, meeting the fluctuations of
develops a prediction method based on LSTM of the end-user response renewable generation, enabling higher penetration of renewable gen-
behavior to incentive-based DR program. eration and customer bill reduction. Nevertheless, there are still many
In recent years, RL has gained prominence in studying intelligent challenges to be addressed, such as improve the accuracy of the flex-
management and control of buildings and households. The main ad- ibility prediction models or selecting the best suitable customers for
vantage of using RL algorithms instead of optimization techniques is engaging DR programs.
that the RL algorithm can automatically learn the customer prefer- As discussed earlier, it is necessary to add value to the massive
ences imitating human behavior and determining optimal incentive data generated within the distribution network and create aggregated
rates that can maximize the profits of both energy service providers flexibility services for electricity market stakeholders using the most
and customers fairly and efficiently. [271] narrows its research to a appropriate AI techniques. The aggregator is a relevant player con-
group of energy systems that use RL to control the assets that have tributing to flexibility aggregated services through DR incentive-based
potential for DR applications. A DR price-based approach using deep programs within its portfolio (a set of clients). The role of aggregators in
RL in an industrial facility is conducted in [272]. It is noteworthy the Smart Grid context is studied in [283]. Electricity market actors can
that the algorithm was tested in a real-world utility company, reducing request flexibility to avoid grid congestions (e.g., DSO) or imbalance
the energy costs and ensuring production targets as well. The authors penalizations (e.g., BRP). Thanks to the aggregation of individual cus-
in [273] perform a simulated-based followed by a lab experiment of an tomers, the total amount of flexibility available increases considerably.
electric water heater cost of energy consumption minimization given Thus, the end-users change their consumption pattern in exchange for
an external price profile using RL techniques. economic compensation through incentive-based approaches [284].

13
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Table 10
AI methods and DR schemes used in EMS.
Data-drive technique DR program Consumer type Ref.
Supervised Learning
ANN Price-based Residential [264]
ANN Price-based Residential [265]
ANN Price and incentive-based Residential [268]
ANN Price-based Residential [269]
RF Price-based Residential [266]
Deep Learning
LSTM Incentive-based Residential [270]
Reinforcement Learning
Deep RL (Single-agent) Price-based Residential [273]
Deep RL (Single-agent) Price-based Residential [278]
RL (Multi-agent) Price-based Residential [267]
RL (Multi-agent) Price-based Residential [275]
RL (Single-agent) Price and incentive-based Residential [276]
RL (Single-agent) Incentive-based Residential [277]
RL (Single-agent) Price-based EV management [274]
RL (Single-agent) Price-based Industrial facility [272]
RL (Multi-agent) Price-based Residential [275]
RL (Multi-agent) Price-based Residential [279]
RL (Single-agent) Price-based Residential [280]
RL (Single-agent) Price-based Residential [281]

Finding the best-suited customers to provide the flexibility re- literature; for instance, a robust comparative review of 11 clustering
quested by an electricity market agent is a computational challenge for techniques applied to residential load time series profile is carried
the aggregator, especially with portfolios with a considerable amount out in [301]. The study concludes that centroid-based (Kmeans) and
of flexible resources. To cope with this challenge, [285] proposes hierarchical algorithms are the best performers, whereas the density-
a cluster-based (K-means) day ahead bidding optimization approach based methods (such as DBSCAN) performed the worst for this kind
that reduces the optimization execution time. Moreover, the ADMM of problem. On the other hand, [302] reviews the clustering methods
technique is capable of solving large-scale optimization problems by for customers consumption patterns. The K-means algorithm is the
breaking them into smaller pieces [286–288]. For instance, [286] most widely used, followed by Fuzzy C-Means, hierarchical and Self-
formulates a cost minimization problem to provide flexibility services Organizing Map (SOM), being the latter one the worst performer of
to DSO and BRP using ADMM to improve computational performance. a 4000 customers segmentation case study. A combination of SOM
Moreover, [287] applies ADMM for bidding optimization strategy in and K-means algorithms are used for analyzing industrial parks energy
the day-ahead and secondary reserve markets. In [288], an ADMM- consumption patterns [303]. The main drawbacks of using SOM are
based market-clearing strategy is presented for day-ahead congestion that the results are not intuitive at first glance and are computationally
management, using aggregated EVs and heat pumps as flexible sources. expensive compared to K-means.
The DSO can request demand-side flexibility to the aggregator to
mitigate possible congestion in the distribution network. Day-ahead 4.8. Trading
congestion management is proposed in several works [288–291], while
[292] formulates real-time congestion management for the unforeseen The trading energy service focuses on one of the biggest challenges
events that might occur during operation. On the other hand, grid of the forthcoming energy transition: finding a reliable way to exchange
congestions can also be avoided with dynamic pricing strategies in energy between different customers, local energy communities and
order to encourage customers to flatter their demand curve [264,267, operators.
272–274,276,281]. Based on the blockchain concept, Distributed Ledger Technology
Aggregators can attend both DSO and BRP flexibility requests for (DLT) is positioned as a benchmark technology in the P2P trading
day-ahead and intra-day portfolio optimization. With the intention field, enabling smart contracts between prosumers and active users. The
to address this issue, the traffic light system proposed in [293] is potentials of DLT for P2P transactive energy exchanges and its infras-
carried out in [286,294] to coordinate the flexibility requests from tructure in Local Energy Markets are detailed in [304], while [305]
DSO and BRP and establish a priority criterion for providing flexibility studies the DLT requirements and use cases. The authors in [306]
in the case of conflicting requests. Recent works use the aggregation carry out a detailed analysis of the concept, principles and types
of thermal loads [295,296], Heating Ventilating and Air Conditioning of blockchain and how this technology will revolutionize the green
(HVAC) [297,298] and EVs along with heat pumps [288,289,292] in energy management of the future. The main advantage of the de-
order to provide flexibility to energy agents. On the other hand, some centralization and automation of smart contracts is eliminating the
studies focus on optimal bidding strategies for aggregators participating human-based central authority, which implies lower settlement fees,
in the electricity market [252,285,287,299], where [285,299] partici- simplified operational processes because of fewer intermediaries and
pate in the day-ahead market, while [252,287] are also involved in the a greater transparency level, thus avoiding corruption. However, the
reserve market. DLT is still immature and has not yet been tested on large-scale trials.
In addition to the flexibility services mentioned above, electricity In addition, the high computational cost of smart contracts raises the
companies also desire to segment their large number of customers question of whether it is economically viable or not.
according to similar demand patterns to have insights into their en- Decentralized blockchain mechanisms [307] enable a reliable en-
ergy usage behavior, enhancing the distribution network operation ergy flexibility trading between the stakeholders involved in the flex-
and management. Moreover, more customized products and services ibility market. DLT for P2P ancillary service markets in distribution
can be offered to each customer target group [300]. Unsupervised networks is studied in [308,309], where the last article applies the
Learning techniques enable customer segmentation according to the ADMM technique to settle the P2P trading. Smart contracts for DR pro-
consumption pattern, thanks to the smart meter measurements [261]. grams are formulated and the total incentives for an energy prosumer
Diverse customer demand-based clustering studies are proposed in the are calculated. Four different P2P and smart contract implementation

14
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

approaches are conducted in [310], where producers and consumers state estimation-, fault detection -to identify and locate outages-,
send their offers and bids accordingly with smart contracts in the non-technical losses detection -to detect possible frauds in the distri-
energy market. In terms of security, [311] proposes a RNN that detects bution network-, distribution network planning -to design optimal
network attacks and fraudulent transactions within the blockchain- long-term investments and cost operations in the LV network- and
based energy transactions. Besides, a novel transactive controller is last but not least, the aggregated flexibility service -to give the DSO
developed to manage the storage unit of a residential prosumer. The a view of the network structure so that he can formulate flexible
research [312] also studies P2P trading for managing the ESS and the requests to minimize congestion-.
surplus renewable energy in a smart energy community. • Observability. The observability service outcome provides real-
Focusing now on data-driven methods, a deep RL approach is time knowledge of the LV network state, which is essential for
adopted to address an energy trading decision-making problem for grid operation. Thefault detection service requires the LV network
Microgrids [313]. The work in [314] implements LSTM for blockchain- status to identify the precise location of a failure in the power
based predictive energy analysis, intending to enable accurate short grid in real-time.
and long-term demand forecasting to minimize the cost of delivering • Predictive maintenance. This service is in charge of predicting
electrical energy for the consumer and making effective policies. Sim- the probability of some distribution grid components failure.
ilarly, [315] develops a P2P market based on deep learning (ELM), Identifying the network components’ state and health enables the
which learns the interaction between prosumer bidding actions and distribution network planning service to plan investments and to
market responses from historical transaction data. operating costs in the long term optimally.
• Price forecasting. This service predicts the electricity price fore-
4.9. Asset and investment planning casting for short or long-term horizons. The aggregated flexibility
service needs the price forecasting to optimize the BRP portfolio
The ever-increasing volume of stored measured data in the distribu- purchase bids before buying energy in the day-ahead electricity
tion network is expected to benefit the planning and operation of the market.
future power systems [278]. The investment planning service examines • Generation forecasting. The outcome is a RES forecast for short
grid status and expansion criteria and selects the most appropriate or long-term horizons. The energy management system service ne-
technologies and optimal geographical locations. The objective is to cessitates short-term RES generation in order to schedule the
contribute to the grid support during a settled planning horizon and flexible resources optimally to minimize the electricity bill. In
estimate the associated costs for achieving a specific planning goal or contrast, the distribution network planning service needs long-term
criteria while meeting the forecasted demand. RES forecasting as input – more than five years ahead – to
ML forecasting methods play a critical role in mid and long-term minimize the LV network assets’ investment and operating costs
renewable energy and demand predictions, which are essential inputs in the distribution grid.
for the country’s energy mix development and planning [147]. The au- • Demand forecasting. The outcome is a demand forecast for a
thors from [316] propose a basic learning neural network to determine short or long-term horizon. This service follows the same process
how Microgrids can be optimally planned and designed. Dimension as the last bullet point; the energy management system service needs
reduction and correlation techniques are adopted for optimal planning short-term load prediction, while the distribution network planning
for capacitors in [317]. Optimization techniques are applied to mini- service needs long-term demand forecasting as input data.
mize total costs and investments for power distribution system planning • Flexibility forecasting. This service determines the flexibility
in [318]. Flexibility is taken into account in [319] using a generic multi- available in a zone or area of the distribution grid within a
stage distribution grid planning approach, while [320] studies in-depth time horizon. The aggregated flexibility service makes use of this
network expansion under a DR scheme. prediction, so the electricity market stakeholders – DSO, BRP, for
In recent years, some efforts have been made for long-term energy instance – know in advance the flexibility available in the zone
planning. For instance, long-term demand and renewable energy fore- or area. The energy trading service also needs flexibility forecasting
casting models are an energy planning tool [147]. However, there is in order to schedule the energy trading optimally.
a lack of research exploring AI techniques for medium and long-term • Energy management system. End-users consumption of a zone
distribution grid planning. Consequently, more efforts need to be taken or area are aggregated and sent to the aggregated flexibility services.
in this field. This information is useful for specific stakeholders like the DSO
to detect possible congestions in the grid or the BRP to identify
5. Discussion how much energy its portfolio will consume the following day
after optimizing its consumption through the energy management
5.1. Distribution grid services dependencies service.
• Aggregated flexibility services. The outcome of this service is
After an exhaustive analysis of the distribution grid services in the flexibility requests of specific electricity market agents such
the previous section, it can be concluded that there are dependencies as DSO or BRP. The flexibility requests of these agents are key
between certain services. This means the outcome of one service might input information for the energy trading service.
be a fundamental piece of information for executing another. These in-
terconnections among energy services are represented in Fig. 6 through 5.2. AI-techniques applied in distribution grid services
a flow chart. The green boxes represent the services related to the
operation of the distribution network, the blue boxes outline the ser- Certain services are more likely to implement AI techniques than
vices related to flexibility management and finally, the light orange box others, particularly those that require prediction, classification or clus-
represents the planning services. In order to facilitate understanding tering tasks. With the aim of facilitating the understanding and interre-
of the flow chart, the outcome-dependence of each energy service is lation among the AI techniques and the data-driven energy services,
explained subsequently. a chord diagram is presented in Fig. 7. It is a graphical outline for
displaying the flows or connections between the data-driven methods
• Topology. This service offers the actual structure of the dis- and the energy services, where the size of the arc is equivalent to the
tribution network, which is often poorly known. The LV grid importance of the flow. In this particular case, the arc size represents
topology is a required input for the optimal performance of the the number of publications that applied AI methods to those services.
following services.Observability -to calculate real-time network The ribbons amplitude is equivalent to the number of articles that have

15
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Fig. 6. Distribution network services dependencies flow chart.

Fig. 7. Chord diagram that represents the interrelation between services and their most-applied AI techniques in recent literature.

adopted this technique. For a better visual perception, the data-driven order to classify or group potential failure events. Concerning the Non-
methods that appear in less than three studies have been eliminated Technical Losses service (NTL), the use of SVM to classify and detect
from the chord diagram. Fig. 7 displays the AI methods first in a future fault events in PV systems or power generation equipment is
clockwise manner, sorted into the four categories proposed in Section 2, predominant. However, multiple data-driven techniques such as DM,
followed by the distribution grid services. CNN, K-means and SOM are also employed to detect these faults.
The Forecasting service (FOR) is the most AI-intensive since several
The services are analyzed in order of appearance. The Measurement
investigations concentrate their research on predictive models applying
Error Detection service (MED) is powered by ML regression-based
AI procedures. The LSTM is the most utilized in the time series fore-
algorithms for assigning predictive values for missing data. Correlation
casting field in recent years, followed by a single hidden layer MLP
and ADMM methods are widely used in the topology service (TOP).
algorithm. DNN with more than one hidden layer and CNN are utilized
The Observability service (OBS) barely holds any articles that use to a lesser extent. Therefore, RNNs are very appropriate for time series
data-driven methods. In the Fault Detection service (FD), there is a data since they use temporal information of the input data. Thus,
range of most used AI techniques for fault detection classification tasks, the LSTM is the best-suited when predicting a times series outcome
which are RF, SVM and MLP. The RF classifier outperforms among the considering the model can associate the data of the previous time and
other ML techniques, as [84] also points out. Regarding the Predictive the present time thanks to their recurrent architecture and memory
Maintenance service (PM), no technique predominates over another. units. The most common evaluation metric for regression tasks is the
The PM service mainly covers classification to clustering algorithms in RMSE, followed closely by MAE and MAPE. Concerning the Energy

16
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

Management System service (EMS), besides the optimization methods and governance, data integration and sharing, data processing and
that are not in the scope of this paper, RL is used as a decision-making analysis, security and privacy and need for professionals of Big Data
tool for scheduling and controlling flexible assets. This flexibility can analytics and smart energy management. These challenges fit with the
be used for the end-user benefit, such as reducing electricity costs more generic main issues that Europe must tackle in creating and
considering the customer’s comfort or being sold to a third party sustaining a robust Big Data ecosystem, identified by Big Data Value
involved in the electricity market through an aggregator in exchange Association [324]. These issues are related to data, skills, legal, tech-
for monetary compensation. nology, application, business and societal aspects. Six main challenges
The rest of the remaining services are not as dependent on IA regarding the distribution grid digitization are addressed below.
techniques. The Aggregated Flexibility Service (AGG) uses mostly opti-
mization techniques to minimize the cost of providing flexibility instead • ICT infrastructure and technology. Utilities have been forced
of AI methods. Clustering is applied in order to group customers with to strengthen their ICT infrastructure in their back-end systems
a similar consumption profile, thus detecting possible congestions or to deal with Big Data collection and storage. It may include
unusual behavior. Moreover, to cope with the massive amount of new sensors, improved transmission and storage capacity and
data, this service uses ADMM to relax computing complexity when increased data processing or data exchange capability [9]. New
aggregating flexible resources that provide flexibility, dividing the applications can be developed using existing data, but even more
optimization problems into separated parts. Therefore, ADMM is a would be available if larger energy-related data were accessible
valuable and recurrent solution to deal with energy services that need and as close to real-time as possible. An example of this issue is
a large amount of data as input. The Trading service (TRAD) relies the information currently available from installed smart meters.
mainly on the DLT method, which enables the P2P trading and smart Valuable knowledge can be discovered from the massive electric-
contracts between prosumers and active users. Lastly, the Planning ity consumption data collected near real-time by AMI devices.
service (PLAN) uses some data-driven techniques such as dimension However, limited real-time data are available from part of first-
reduction and correlation techniques, but it is mainly a service that generation deployed smart meters. As an example, by 2020, 14
uses optimization procedures instead of AI methods. The main benefits European Union (EU) Member States have implemented a re-
of implementing AI methods within the distribution grid domain are freshment rate of at least 15 min, while only 8 Member States
addressed: confirmed to be able to provide near real-time information on
electricity every 10 s [325]. Second generation of smart meters
• Allow real-time distribution grid status estimation and gain ob-
with near real-time available data are needed, which makes AMI
servability, enhancing the monitoring and locating possible events
data actionable for more operation-related tools in addition to the
in the network to provide a tool that enables the operator to react
long-term planning applications.
more rapidly when a fault or event occurs.
• Data collection and governance. The availability and access to
• Perform the predictive maintenance service increases the distribu-
high-quality data sets are key challenges for enabling AI tech-
tion network security and availability while diminishing the DSO
niques. In the energy sector, available data are not always suffi-
costs.
cient or of good enough quality to develop systems that can han-
• Use the aggregated flexibility available to avoid grid congestions,
dle complex scenarios [10]. In addition, the timeliness, integrity,
for instance.
accuracy and consistency of data for energy AI applications need
• Optimal and reliable electricity trading among costumers.
to be improved [9]. As digital technologies evolve, these problems
• Optimal medium and long term distribution grid planning.
can be more efficiently addressed with proper data management
and data governance strategies.
5.3. Opportunities and challenges
• Data integration and sharing. As well as having agreed ap-
Energy sector organizations are increasingly interested in using data proaches, the interoperability of data sets and data-driven so-
science and AI capabilities to solve their daily challenges. However, lutions is essential to ensure wide adoption within and across
Big Data techniques applied to the energy sector are still in their early applications. However, companies are reticent to share their data
development phase and most of the related Big Data-driven applications in order to avoid security risks and unlock their competitive ad-
are not mature yet. This brings new opportunities for this emerging and vantages. For example, great opportunities can arise if operational
promising research area. data from distribution and transmission grids are exchanged be-
One of the primary triggers of this increasing interest is the avail- tween DSOs and Transmission System Operators (TSOs) fairly and
ability of significant amounts of data from smart meters and the digi- transparently. For this reason, TSOs and DSOs need to determine
tization of the distribution grid. Although being used at the beginning what information they require, the quality of the information,
only for billing purposes, smart meters provide information about the who owns it and how to ensure both confidentiality and trans-
grid end-points operation. If this information is combined with other parency [326]. On the other hand, open data sets are needed to
systems related to the digitization of the distribution grid or other develop and test new algorithms and solutions. Several initiatives
external data sources, it provides even more insight into how the worldwide support energy data sharing among stakeholders, such
system operates. This is a kind of information that utilities did not have as Green Button and OpenEI in the USA or ENTSO-E Transparency
before smart meters deployment and it has opened up opportunities for Platform in the EU. However, open available energy-related data
increasing operational efficiencies and enhancing the distribution grid should increase, as opening up public-sector data and establishing
reliability [321]. common data standards can also help to boost innovation [327].
Although these opportunities encourage the development of Big • Data processing, analysis and business models. New data
Data solutions, utilities are still largely missing the opportunity to analysis techniques in DM, ML, statistical analysis, data man-
utilize those newly available data. As an example, American Council agement and data visualization are applied to the energy sector.
for an Energy-Efficient Economy surveyed 52 large American utilities Continuous and recently more frequent developments have led
in 2018 and found that most of them are significantly under-utilizing to advanced technologies that make significantly easier the use
data from smart meters [322]. of Big Data, not only in energy applications. These innovative
Some challenges need to be correctly addressed to boost the dis- data analysis techniques open up new opportunities to provide
tribution grid development and reach its full potential. According solutions and create new businesses in this sector. Thus, it is
to [9] and [323], Big Data challenges in energy management fo- crucial to identify new business opportunities with existing data
cus on six main issues, which are IT infrastructure, data collection and create new data-driven business models to make the most

17
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

of these techniques innovations. [328] reviews examples of these experimentation in a realistic environment prevents its application in
new business models by analyzing 40 data-driven start-ups in the real-world buildings and households.
energy sector. To conclude, it is essential to equip the distribution network with
• Security, privacy and legal issues. The power system digitiza- sensors to collect the data that feed the innovative services. Implement-
tion has converted cybersecurity into an essential concern due ing data-driven techniques in energy services development is essential
to the increasing number of incidents in recent times. Besides, for developing a reliable, secure and efficient Smart Grid. Thanks to
privacy and security should be guaranteed along the Big Data these methods, the services show better performance compared to
value chain to protect the customer and the risks and possible statistical benchmark procedures. Nevertheless, there are still chal-
impact on supply security. To deal with this problem, several lenges to overcome to extend and improve the AI applications in
initiatives have been carried on all around the globe. At the power systems, mainly related to ICT infrastructure, data collection
EU, one of the key legislation in this regard is the Directive and governance, data integration and sharing, data processing and
on security of Network and Information Systems [329], which analysis, security and privacy and the need for professionals of Big Data
boosts the level of cybersecurity in the Union through the de- analytics.
velopment of national cybersecurity capabilities, the increase of
EU level cooperation and the introduction of security and in- Declaration of competing interest
cident reporting obligations in critical sectors, like the energy.
In addition, General Data Protection Regulation [330] aims at The authors declare that they have no known competing finan-
protecting individuals concerning the processing of their data cial interests or personal relationships that could have appeared to
and warranting the free movement of such data within the EU. influence the work reported in this paper.
Although these European directives could be considered a late
response to an already well-known problem, these regulations can Acknowledgments
only be considered as a single part of an international chessboard
where they should be followed, complemented and particularized This work has been supported by the BD4OPEM H2020 project,
by many others [331], as, at the same time, China and USA have which has received funding from the European Union’s Horizon 2020
introduced their cybersecurity laws and policies. research and innovation program under Grant Agreement No. 872525.
• Professionals and skills. There is a need for trained and ed- The research of Sara Barja-Martinez and Íngrid Munné-Collado has
ucated employees in the energy sector that can use Big Data received support of the Secretaria d’Universitats i Recerca del De-
technologies and build on data expertise. This can be achieved partament d’Economia i Coneixement de la Generalitat de Catalunya
by enrolling experts from other more mature sectors, like finance under the FI-AGAUR Pre-doctoral grants and scholarships (FI-2020
or marketing, or providing master-level students with specific and FI-2017). Mònica Aragüés-Peñalba and Eduard Bullich-Massagué
energy-related Big Data and AI techniques solid background. are lecturers of the Serra Húnter programme. The research of Íngrid
Although these specialists should be combined with other energy Munné-Collado has received support from the InnoEnergy PhD School,
domain knowledge experts, the first option can bring immediate under the call of 2017.
results. Otherwise, energy sector actors can also invest in re-
skilling and training their employees to manage and operate References
power assets and systems that are digitalized [10].
[1] Good N, Ellis KA, Mancarella P. Review and classification of barriers and
6. Conclusions enablers of demand response in the smart grid. Renew Sustain Energy Rev
2017;72(January 2016):57–72. http://dx.doi.org/10.1016/j.rser.2017.01.043.
[2] Khan AA, Razzaq S, Khan A, Fatima K, Owais. HEMSs and enabled demand
Implementing Big Data solutions and AI techniques in the power
response in electricity market: An overview. Renew Sustain Energy Rev
system domain is a promising approach for extracting knowledge and 2015;42:773–85. http://dx.doi.org/10.1016/j.rser.2014.10.045.
high added value of the vast amount of high-granularity data stored [3] Rehmani MH, Reisslein M, Rachedi A, Erol-Kantarci M, Radenkovic M.
by intelligent devices placed along the distribution grid, such as smart Integrating renewable energy resources into the smart grid: Recent develop-
meters or PMUs. ments in information and communication technologies. IEEE Trans Ind Inf
2018;14(7):2814–25. http://dx.doi.org/10.1109/TII.2018.2819169.
This paper proposes and interrelates a set of innovative energy [4] Bhattarai BP, Paudyal S, Luo Y, Mohanpurkar M, Cheung K, Tonkoski R, et
services designed to be offered to different electricity domain agents, al. Big data analytics in smart grids: state-of-the-art, challenges, opportunities,
such as DSO, BRPs and prosumers. These services are fed with high- and future directions. IET Smart Grid 2019;2(2):141–54. http://dx.doi.org/10.
granularity and massive stored data. Thanks to the application of data- 1049/iet-stg.2018.0261.
[5] Ghosal A, Conti M. Key management systems for smart grid advanced metering
driven techniques, they provide solutions to problems like congestion
infrastructure: A survey. IEEE Commun Surv Tutor 2019;21(3):2831–48. http:
management, distribution grid equipment maintenance, forecasting, //dx.doi.org/10.1109/COMST.2019.2907650, arXiv:1806.00121.
detection and prevention of faults and fraud detection. [6] Alquthami T, Alsubaie AM, Anwer M. Importance of smart meters data
Once the innovative services have been identified, an exhaustive processing – case of Saudi Arabia. 2020, p. 1–5. http://dx.doi.org/10.1109/
review of the most recent studies implementing AI techniques in each icecta48151.2019.8959739.
[7] Alquthami T, Alamoudi A, Alsubaie AM, Bin A, Nassir J, Anwar M, et al.
of them is carried out. Key findings state that ensemble models present
Analytics framework for optimal smart meters data processing. Electr Eng 2020.
better results than single ML models by combining different data-driven http://dx.doi.org/10.1007/s00202-020-00949-0.
algorithms. Deep learning algorithms have gained importance in recent [8] Front matter. Big Data Appl Power Syst 2018;i–ii. http://dx.doi.org/10.1016/
years for time series prediction tasks and outperform most benchmark b978-0-12-811968-6.09985-x.
ML and statistical algorithms. Concerning classification tasks, tradi- [9] Zhou K, Fu C, Yang S. Big data driven smart energy management: From
big data to big insights. Renew Sustain Energy Rev 2016;56(2016):215–25.
tional ML algorithms as SVM or RF yet provide excellent results. For
http://dx.doi.org/10.1016/j.rser.2015.11.050.
instance, the RF classifier outperforms when it comes to supervised [10] International Renewable Energy Agency T. Artificial intelligence and big data
classification tasks, while LSTM recurrent network is the predominant innovation landscape brief. 2019, URL www.irena.org.
algorithm for time series forecasting. Unsupervised learning methods [11] Rashed Mohassel R, Fung A, Mohammadi F, Raahemifar K. A survey on ad-
are mainly responsible for customer segmentation, buildings efficiency vanced metering infrastructure. Int J Electr Power Energy Syst 2014;63:473–84.
http://dx.doi.org/10.1016/j.ijepes.2014.06.025.
clustering and consumption profile grouping for non-technical losses [12] Ahmed H. Applying big data analytics for energy efficiency. In: Master’s
detection. Finally, RL is widely applied in the literature to optimally programme in international design business management hussnain. 2014, p.
schedule flexible assets in households, although the scarcity of physical 85.

18
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

[13] Philip Chen CL, Zhang CY. Data-intensive applications, challenges, techniques [39] Bruce P, Bruce A. Practical statistics, Vol. 3. O’Reilly; 2017, http://dx.doi.org/
and technologies: A survey on big data. Inform Sci 2014;275:314–47. http: 10.2307/3619701.
//dx.doi.org/10.1016/j.ins.2014.01.015. [40] Simon B-C. Practical statistics, Vol. 2. (3):Cambridge University Press; 1989, p.
[14] Jiang H, Wang K, Wang Y, Gao M, Zhang Y. Energy big data: A survey. IEEE 143–8. http://dx.doi.org/10.2307/3619701.
Access 2016;4:3844–61. http://dx.doi.org/10.1109/ACCESS.2016.2580581. [41] Saldaña-gonzález AE, Sumper A, Aragüés-peñalba M. Advanced distribution
[15] Jindal A, Roopaei M, Diggewadi A, Andoni M, Zobaa A, Alam A, et al. Big data measurement technologiesand data applications for smart grids : A review.
analytics in the smart grid. IEE Smart Grid 2017. Energies 2020;1–34. http://dx.doi.org/10.3390/en13143730.
[16] Tu C, He X, Shuai Z, Jiang F. Big data issues in smart grid – A review. Renew [42] Dehghan-Banadaki A, Taufik T, Feliachi A. Big data analytics in a day-ahead
Sustain Energy Rev 2017;79(May):1099–107. http://dx.doi.org/10.1016/j.rser. electricity price forecasting using tensorflow in restructured power systems.
2017.05.134. In: Proceedings - 2018 International conference on computational science and
[17] Costa C, Santos MY. Big data: State-of-the-art concepts, techniques, technolo- computational intelligence. IEEE; 2018, p. 1065–9. http://dx.doi.org/10.1109/
gies, modeling approaches and research challenges. IAENG Int J Comput Sci CSCI46756.2018.00207.
2017;44(3):285–301. [43] USDepartment of Energy. Advanced metering infraestructure and customer
[18] Zhang Y, Huang T, Bompard EF. Big data analytics in smart grids: A review. systems - results from the smart grid investment grant program. Off Electr
Energy Inform 2018;1(1):1–24. http://dx.doi.org/10.1186/s42162-018-0007-5. Deliv Energy Reliab 2016;1–98.
[19] Akhavan-Hejazi H, Mohsenian-Rad H. Power systems big data analytics: An [44] Shahsavari A, Farajollahi M, Stewart EM, Cortez E, Mohsenian-Rad H. Situa-
assessment of paradigm shift barriers and prospects. Energy Rep 2018;4:91–100. tional awareness in distribution grid using micro-PMU data: A machine learning
http://dx.doi.org/10.1016/j.egyr.2017.11.002. approach. IEEE Trans Smart Grid 2019;10(6):6167–77. http://dx.doi.org/10.
[20] Bhattarai BP, Paudyal S, Luo Y, Mohanpurkar M, Cheung K, Tonkoski R, et 1109/TSG.2019.2898676.
al. Big data analytics in smart grids: state-of-the-art, challenges, opportunities, [45] Ma J, Cheng JC, Jiang F, Chen W, Wang M, Zhai C. A bi-directional missing
and future directions. IET Smart Grid 2019;2(2):141–54. http://dx.doi.org/10. data imputation scheme based on LSTM and transfer learning for building
1049/iet-stg.2018.0261. energy data. Energy Build 2020;216. http://dx.doi.org/10.1016/j.enbuild.2020.
[21] Hossain E, Khan I, Un-Noor F, Sikander SS, Sunny MSH. Application of big 109941.
data and machine learning in smart grid, and associated security concerns: [46] Jung S, Moon J, Park S, Rho S, Baik SW, Hwang E. Bagging ensemble of
A review. IEEE Access 2019;7:13960–88. http://dx.doi.org/10.1109/ACCESS. multilayer perceptrons for missing electricity consumption data imputation.
2019.2894819. Sensors 2020;20(1772):1–16.
[22] Shobha G, Rangaswamy S. Computational analysis and understanding of natural [47] Spinelli I, Scardapane S, Uncini A. Missing data imputation with adversarially-
languages: Principles, methods and applications chapter 8 - machine learning. trained graph convolutional networks. Neural Netw 2020;129:249–60. http:
Handbook of Statist 2018;38:197–228. //dx.doi.org/10.1016/j.neunet.2020.06.005, arXiv:1905.01907.
[48] Qiao J, Zhu B, Wang X, Luo K. Application research of artificial intelligence
[23] Gupta R, Tanwar S, Tyagi S, Kumar N. Machine learning models for secure data
technology in error diagnosis of electric energy meter. In: 2019 IEEE 4th
analytics: A taxonomy and threat model. Comput Commun 2020;153(November
international conference on cloud computing and big data analytics. IEEE; 2019,
2019):406–40. http://dx.doi.org/10.1016/j.comcom.2020.02.008.
p. 155–9. http://dx.doi.org/10.1109/ICCCBDA.2019.8725622.
[24] Fadlullah ZM, Tang F, Mao B, Kato N, Akashi O, Inoue T, et al. State-of-the-
[49] Arghandeh R, Gahr M, Von Meier A, Cavraro G, Ruh M, Andersson G. Topology
art deep learning: Evolving machine intelligence toward tomorrow’s intelligent
detection in microgrids with micro-synchrophasors. IEEE Power Energy Soc Gen
network traffic control systems. IEEE Commun Surv Tutor 2017;19(4):2432–55.
Meet 2015;2015-Septe. http://dx.doi.org/10.1109/PESGM.2015.7286053.
http://dx.doi.org/10.1109/COMST.2017.2707140.
[50] Vitiello, S, Vasiljevksa, J, Filiou C. Cost-Benefit analyses & state of play of smart
[25] Boning DS, Elfadel IAM, Li X. A preliminary taxonomy for machine learning
metering deployment in the EU-27. Tech. rep, 2014.
in VLSI CAD. In: Elfadel IAM, Boning DS, Li X, editors. Machine learning in
[51] Zhao Y, Chen J, Poor HV. Efficient neural network architecture for topology
VLSI computer-aided design. Cham: Springer International Publishing; 2019, p.
identification in smart grid. In: 2016 IEEE global conference on signal and
1–16. http://dx.doi.org/10.1007/978-3-030-04666-8_1.
information processing, globalsip 2016 - proceedings. IEEE; 2017, p. 811–5.
[26] Ibrahim MS, Dong W, Yang Q. Machine learning driven smart elec-
http://dx.doi.org/10.1109/GlobalSIP.2016.7905955.
tric power systems: Current trends and new perspectives. Appl Energy
[52] Sevlian R, Zhao Y, Goldsmith A, Rajagopal R, Poor HV. Distribution system
2020;272(May):115237. http://dx.doi.org/10.1016/j.apenergy.2020.115237.
outage detection using consumer load and line flow measurements. (May).
[27] Sutton R, Barto A. Reinforcement learning: An introduction. The MIT Press;
2015, arXiv:1503.08244, http://arxiv.org/abs/1503.08244.
2015, p. 63–80. http://dx.doi.org/10.4018/978-1-60960-165-2.ch004.
[53] Lin J, Wan C, Song Y, Huang R, Chen X, Guo W, et al. Situation awareness
[28] Glavic M, Fonteneau R, Ernst D. Reinforcement learning for electric power
of active distribution network: Roadmap, technologies, and bottlenecks. CSEE J
system decision and control: Past considerations and perspectives. IFAC-
Power Energy Syst 2016;2(3):35–42. http://dx.doi.org/10.17775/cseejpes.2016.
PapersOnLine 2017;50(1):6918–27. http://dx.doi.org/10.1016/j.ifacol.2017.08.
00033.
1217. [54] Bolognani S, Bof N, Michelotti D, Muraro R, Schenato L. Identification of power
[29] Perera AT, Kamalaruban P. Applications of reinforcement learning in energy distribution network topology via voltage correlation analysis. In: Proceedings
systems. Renew Sustain Energy Rev 2021;137(December 2020):110618. http: of the IEEE conference on decision and control. IEEE; 2013, p. 1659–64.
//dx.doi.org/10.1016/j.rser.2020.110618. http://dx.doi.org/10.1109/CDC.2013.6760120.
[30] Duan L, Hu J, Zhao G, Chen K, Wang SX, He J. Method of inter-turn [55] Zhao Y, Chen J, Poor HV. Learning to infer: A new variational inference
fault detection for next-generation smart transformers based on deep learning approach for power grid topology identification. In: IEEE workshop on statistical
algorithm. High Voltage 2019;4(4):282–91. signal processing proceedings. 2016-Augus, IEEE; 2016, p. 1–5. http://dx.doi.
[31] Li Y, Yu F, Cai Q, Yuan K, Wan R, Li X, et al. Image fusion of fault org/10.1109/SSP.2016.7551827.
detection in power system based on deep learning. Cluster Comput 2019;22. [56] Cavraro G, Arghandeh R, Poolla K, Von Meier A. Data-driven approach for
http://dx.doi.org/10.1007/s10586-018-2264-2. distribution network topology detection. IEEE Power Energy Soc Gen Meet
[32] Moradzadeh A, Pourhossein K. Short circuit location in transformer winding 2015;2015-Septe:1–5. http://dx.doi.org/10.1109/PESGM.2015.7286490, arXiv:
using deep learning of its frequency responses. In: 2019 International aegean 1504.00724.
conference on electrical machines and power electronics (ACEMP) 2019 interna- [57] Rogers KM, Spadoni RD, Overbye TJ. Identification of power system topology
tional conference on optimization of electrical and electronic equipment, 2019. from synchrophasor data. In: 2011 IEEE/PES power systems conference and
p. 268–73. exposition, PSCE 2011, Vol. 61801. IEEE; 2011, p. 1–8. http://dx.doi.org/10.
[33] Hong Y-Y, Rioflorido CLPP. A hybrid deep learning-based neural network 1109/PSCE.2011.5772462.
for 24-h ahead wind power forecasting. Appl Energy 2019;250:530–9. http: [58] Kekatos V, Giannakis GB, Baldick R. Grid topology identification using elec-
//dx.doi.org/10.1016/j.apenergy.2019.05.044, URL http://www.sciencedirect. tricity prices. IEEE Power Energy Soc Gen Meet 2014;2014(October):1–5. http:
com/science/article/pii/S030626191930889X. //dx.doi.org/10.1109/PESGM.2014.6939474, arXiv:1312.0516.
[34] Abdel-Nasser M, Mahmoud K. Accurate photovoltaic power forecasting models [59] Kekatos V, Giannakis GB, Baldick R. Online energy price matrix factorization
using deep LSTM–RNN. Neural Comput Appl 2017. http://dx.doi.org/10.1007/ for power grid topology tracking. IEEE Trans Smart Grid 2016;7(3):1239–48.
s00521-017-3225-z. http://dx.doi.org/10.1109/TSG.2015.2469098, arXiv:1410.6095.
[35] Brusaferri A, Matteucci M, Portolani P, Vitali A. Bayesian deep learning [60] Anwar A, Mahmood A, Pickering M. Estimation of smart grid topology us-
based method for probabilistic forecast of day-ahead electricity prices. Appl ing SCADA measurements. In: 2016 IEEE international conference on smart
Energy 2019;250:1158–75. http://dx.doi.org/10.1016/j.apenergy.2019.05.068, grid communications. (1):IEEE; 2016, p. 539–44. http://dx.doi.org/10.1109/
URL http://www.sciencedirect.com/science/article/pii/S0306261919309237. SmartGridComm.2016.7778817.
[36] Systems P, Koivisto M. Finding value in big data : Statistical analysis inding [61] Cavraro G, Arghandeh R. Power distribution network topology detection
value in big data. (January). 2016. with time-series signature verification method. IEEE Trans Power Syst
[37] Hastie T, Tibshirani R, Friedman J. The elements of statistical learning. 2018;33(4):3500–9. http://dx.doi.org/10.1109/TPWRS.2017.2779129.
Springer; 2014, http://dx.doi.org/10.1057/9780230355033.0018. [62] Watson JD, Welch J, Watson NR. Use of smart-meter data to determine
[38] Bzdok D, Altman N, Krzywinski M. Statistics versus machine learning. Nature distribution system topology. J Eng 2016;2016(5):94–101. http://dx.doi.org/
Methods 2018;14:232–4. 10.1049/joe.2016.0033.

19
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

[63] Zhang S, Ding Q, Hu G, Nan C. Power distribution network topology recon- [86] Fei W, Moses P. Fault current tracing and identification via machine learning
struction based on adaptive lasso. In: 2nd IEEE conference on energy internet considering distributed energy resources in distribution networks. Energies
and energy system integration, EI2 2018 - proceedings. IEEE; 2018, p. 1–6. 2019;12(22):1–12. http://dx.doi.org/10.3390/en12224333.
http://dx.doi.org/10.1109/EI2.2018.8582616. [87] Ullah I, Khan RU, Yang F, Wuttisittikulkij L. Deep learning image-based
[64] Zhang M, Luan W, Guo S, Wang P. Topology identification method of defect detection in high voltage electrical equipment. Energies 2020;13(2).
distribution network based on smart meter measurements. In: China interna- http://dx.doi.org/10.3390/en13020392.
tional conference on electricity distribution. (201808080000037):IEEE; 2018, [88] Mamuya YD, Lee YD, Shen JW, Shafiullah M, Kuo CC. Application of machine
p. 372–6. http://dx.doi.org/10.1109/CICED.2018.8592228. learning for fault classification and location in a radial distribution grid. Appl
[65] Cavraro G, Kekatos V, Veeramachaneni S. Voltage analytics for power distribu- Sci (Switzerland) 2020;10(14). http://dx.doi.org/10.3390/app10144965.
tion network topology verification. IEEE Trans Smart Grid 2019;10(1):1058–67. [89] Moghaddass R, Wang J. A hierarchical framework for smart grid anomaly
http://dx.doi.org/10.1109/TSG.2017.2758600, arXiv:1707.06671. detection using large-scale smart meter data. IEEE Trans Smart Grid
[66] Liao Y, Member S, Weng Y, Liu G, Member S. Urban MV and LV distribution 2018;9(6):5820–30. http://dx.doi.org/10.1109/TSG.2017.2697440.
grid topology estimation via group lasso. 34, (1):2019, p. 12–27. [90] Li W, Deka D, Chertkov M, Wang M. Real-time faulted line localization and
[67] Richaud L, Pellerej R, Benoit C, Ramos E. Analysis of voltage patterns for PMU placement in power systems through convolutional neural networks.
topology identification and gis. 25th International conference on electricity IEEE Trans Power Syst 2019;34(6):4640–51. http://dx.doi.org/10.1109/TPWRS.
distribution, June, 2019, p. 3–6. 2019.2917794, arXiv:1810.05247.
[91] Li Y, Song X, Zhao S, Gao F. A line-fault cause analysis method for distribution
[68] Jercic R, Pavic I, Damjanovic I. New algorithm for identifying network topology
network based on decision-making tree and machine learning. In: Proceedings
based on artificial neural networks. In: 2019 2nd international colloquium on
- 2020 5th Asia conference on power and electrical engineering. 2020, p. 1–5.
smart grid metrology, SMAGRIMET 2019 - Proceedings. University of Zagreb,
http://dx.doi.org/10.1109/ACPEE48638.2020.9136293.
Faculty of Electrical Engineering and Computing (FER); 2019, p. 1–5. http:
[92] Jiang H, Li Y, Zhang Y, Zhang JJ, Gao DW, Muljadi E, et al. Big data-based
//dx.doi.org/10.23919/SMAGRIMET.2019.8720364.
approach to detect, locate, and enhance the stability of an unplanned microgrid
[69] Yu J, Weng Y, Rajagopal R. Patopa: A data-driven parameter and topology
islanding. J Energy Eng 2017;143(5):1–11. http://dx.doi.org/10.1061/(ASCE)
joint estimation framework in distribution grids. IEEE Trans Power Syst
EY.1943-7897.0000473.
2018;33(4):4335–47. http://dx.doi.org/10.1109/TPWRS.2017.2778194.
[93] Cepeda C, Orozco-Henao C, Percybrooks W, Pulgarín-Rivera JD, Montoya OD,
[70] Lisowski M, Masnicki R, Mindykowski J. PLC-enabled low voltage distribution Gil-González W, et al. Intelligent fault detection system for microgrids. Energies
network topology monitoring. IEEE Trans Smart Grid 2019;10(6):6436–48. 2020;13(5). http://dx.doi.org/10.3390/en13051223.
http://dx.doi.org/10.1109/TSG.2019.2904681. [94] Gush T, Bukhari SBA, Mehmood KK, Admasie S, Kim JS, Kim CH. Intel-
[71] Deka D, Backhaus S, Chertkov M. Structure learning in power distribution ligent fault classification and location identification method for microgrids
networks. IEEE Trans Control Netw Syst 2018;5(3):1061–74. http://dx.doi.org/ using discrete orthonormal stockwell transform-based optimized multi-kernel
10.1109/TCNS.2017.2673546. extreme learning machine. Energies 2019;12(23). http://dx.doi.org/10.3390/
[72] Guo Y, Yang Z, Feng S, Hu J. Complex power system status monitoring en12234504.
and evaluation using big data platform and machine learning algorithms: A [95] Almutairy I, Alluhaidan M. Fault diagnosis based approach to protect-
review and a case study. Complexity 2018;2018. http://dx.doi.org/10.1155/ ing DC microgrid using machine learning technique. Procedia Comput Sci
2018/8496187. 2017;114:449–56. http://dx.doi.org/10.1016/j.procs.2017.09.019.
[73] Baran M, McDermott TE. Distribution system state estimation using AMI data. [96] Al Karim M, Currie J, Lie TT. Dynamic event detection using a distributed
In: 2009 IEEE/PES power systems conference and exposition, PSCE 2009, Vol. feature selection based machine learning approach in a self-healing microgrid.
15236. IEEE; 2009, p. 1–3. http://dx.doi.org/10.1109/PSCE.2009.4840257. IEEE Trans Power Syst 2018;33(5):4706–18. http://dx.doi.org/10.1109/TPWRS.
[74] Matthiss B, Erb J, Binder J. Using smart meters for distribution grid state 2018.2812768.
estimation. In: SEST 2019 - 2nd international conference on smart energy [97] Hsu JY, Wang YF, Lin KC, Chen MY, Hsu JHY. Wind turbine fault diagnosis
systems and technologies. IEEE; 2019, p. 1–5. http://dx.doi.org/10.1109/SEST. and predictive maintenance through statistical process control and machine
2019.8849096. learning. IEEE Access 2020;8:23427–39. http://dx.doi.org/10.1109/ACCESS.
[75] Pignati M, Popovic M, Barreto S, Cherkaoui R, Dario Flores G, Le Boudec JY, 2020.2968615.
et al. Real-time state estimation of the EPFL-campus medium-voltage grid by [98] Lu S, Sirojan T, Phung BT, Zhang D, Ambikairajah E. DA–DCGAN: An effective
using PMUs. In: 2015 IEEE power and energy society innovative smart grid methodology for DC series arc fault diagnosis in photovoltaic systems. IEEE
technologies conference. 2015, http://dx.doi.org/10.1109/ISGT.2015.7131877. Access 2019;7:45831–40. http://dx.doi.org/10.1109/ACCESS.2019.2909267.
[76] Huang M, Wei Z, Sun G, Zang H. Hybrid state estimation for distribution [99] Chen Z, Han F, Wu L, Yu J, Cheng S, Lin P, et al. Random forest based
systems with AMI and scada measurements. IEEE Access 2019;7:120350–9. intelligent fault diagnosis for PV arrays using array voltage and string currents.
http://dx.doi.org/10.1109/access.2019.2937096. Energy Convers Manage 2018;178(August):250–64. http://dx.doi.org/10.1016/
[77] Prostejovsky AM. Increased observability in electric distribution grids: Emerging j.enconman.2018.10.040.
applications promoting the active role of distribution system operators (Ph.D. [100] Garoudja E, Chouder A, Kara K, Silvestre S. An enhanced machine learning
thesis), 2017, p. 285. based approach for failures detection and diagnosis of PV systems. Energy
[78] Mestav KR, Luengo-Rozas J, Tong L. Bayesian state estimation for unobservable Convers Manage 2017;151(September):496–513. http://dx.doi.org/10.1016/j.
distribution systems via deep learning. ArXiv 2018;34(6):4910–20. enconman.2017.09.019.
[101] Huo Y, Prasad G, Lampe L, Leung VC. Advanced smart grid monitoring:
[79] Nithya M, Nagarajan S, Navaseelan P. Fault detection of wind turbine system
Intelligent cable diagnostics using neural networks. In: 2020 IEEE international
using neural networks. In: Proceedings - 2017 IEEE technological innovations
symposium on power line communications and its applications. (Mv). 2020,
in ICT for agriculture and rural development. 2018-Janua, 2018, p. 103–8.
http://dx.doi.org/10.1109/ISPLC48789.2020.9115403.
http://dx.doi.org/10.1109/TIAR.2017.8273694.
[102] Letizia NA, Tonello AM. Supervised fault detection in energy grids measuring
[80] Grid Solutions. Distribution feeder principles. Tech. rep, 2015, p. 284–92.
electrical quantities in the PLC band. In: 2020 IEEE international symposium
[81] Kaikai G, Kai C, Ran G, Ruoyue W, Zhonghan P, You S. Typical ice-wind fault
on power line communications and its applications. 2020, http://dx.doi.org/10.
analysis based on correlation analysis and naive bayesian method. In: 2020
1109/ISPLC48789.2020.9115408.
IEEE 3rd international conference on electronics technology. 2020, p. 372–7. [103] Gigoni L, Betti A, Tucci M, Crisostomi E. A scalable predictive maintenance
http://dx.doi.org/10.1109/ICET49382.2020.9119709. model for detecting wind turbine component failures based on SCADA data.
[82] Tervo R, Karjalainen J, Jung A. Predicting electricity outages caused by IEEE Power Energy Soc Gen Meet 2019;2019-Augus. http://dx.doi.org/10.1109/
convective storms Roope Tervo , Joonas Karjalainen observing and information PESGM40551.2019.8973898, arXiv:1910.09808.
systems centre finnish meteorological institute alexander jung dept of computer [104] Biswal S, Sabareesh GR. Design and development of a wind turbine test rig for
science aalto university. In: 2018 IEEE data science workshop. IEEE; 2018, p. condition monitoring studies. In: 2015 International conference on industrial
145–9. http://dx.doi.org/10.1109/DSW.2018.8439906. instrumentation and control. (Icic):IEEE; 2015, p. 891–6. http://dx.doi.org/10.
[83] Sapountzoglou N, Lago J, Raison B. Fault diagnosis in low voltage 1109/IIC.2015.7150869.
smart distribution grids using gradient boosting trees. Electr Power Syst [105] De Faria H, Costa JaGS, Olivas JLM. A review of monitoring methods for
Res 2020;182(September 2019):106254. http://dx.doi.org/10.1016/j.epsr.2020. predictive maintenance of electric power transformers based on dissolved gas
106254. analysis. Renew Sustain Energy Rev 2015;46:201–9. http://dx.doi.org/10.1016/
[84] Chakraborty D, Sur U, Banerjee PK. Random forest based fault classification j.rser.2015.02.052.
technique for active power system networks. In: 2019 5th IEEE interna- [106] Adoghe AU, Awosope COA, Ekeh JC. Asset maintenance planning in electric
tional WIE conference on electrical and computer engineering, WIECON-ECE power distribution network using statistical analysis of outage data. Int J Electr
2019 - proceedings. IEEE; 2019, p. 2–5. http://dx.doi.org/10.1109/WIECON- Power Energy Syst 2013;47(1):424–35. http://dx.doi.org/10.1016/j.ijepes.2012.
ECE48653.2019.9019922. 10.061.
[85] Shafiullah M, Abido MA, Al-Hamouz Z. Wavelet-based extreme learning [107] Eke S, Lyon B, Lyon ID, Aka-ngnui T, Lyon B, Lyon ID, et al. Characterization
machine for distribution grid fault location. IET Gener Transm Distrib of the operating periods of a power transformer by clustering the dissolved gas
2017;11(17):4256–63. http://dx.doi.org/10.1049/iet-gtd.2017.0656. data. IEEE 2017.

20
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

[108] Kabir F, Foggo B, Yu N. Data driven predictive maintenance of distribution [132] Meira JA, Glauner P, State R, Valtchev P, Dolberg L, Bettinger F, et al. Distilling
transformers. In: China international conference on electricity distribution. provider-independent data for general detection of non-technical losses. In:
(201805280000166):IEEE; 2018, p. 312–6. http://dx.doi.org/10.1109/CICED. 2017 IEEE power and energy conference at illinois. IEEE; 2017, p. 3–7. http:
2018.8592417. //dx.doi.org/10.1109/PECI.2017.7935765.
[109] Velasquez-Contreras JL, Sanz-Bobi MA, Galceran Arellano S. General asset [133] Shekara S, Reddy S, Wang L, Devabhaktuni V. Support vector machine based
management model in the context of an electric utility: Application to power data classification for detection of electricity theft. IEEE Xplore 2011;1–8.
transformers. Electr Power Syst Res 2011;81(11):2015–37. http://dx.doi.org/ [134] Nagi J, Yap KS, Tiong SK, Ahmed SK, Nagi F. Improving SVM-based
10.1016/j.epsr.2011.06.007. nontechnical loss detection in power utility using the fuzzy inference sys-
[110] Venkataswamy R, Uma Rao K, Meena P. Internet of things based metaheuristic tem. IEEE Trans Power Deliv 2011;26(2):1284–5. http://dx.doi.org/10.1109/
reliability centered maintenance of distribution transformers. IOP Conf Ser: TPWRD.2010.2055670.
Earth Environ Sci 2020;463(1). http://dx.doi.org/10.1088/1755-1315/463/1/ [135] Nagi J, Yap KS, Tiong SK, Ahmed SK, Mohamad M. Nontechnical loss detection
012047. for metered customers in power utility using support vector machines. IEEE
[111] Chaves TR, Martins MA, Pacheco BA. MV/LV overhead transformer monitoring Trans Power Deliv 2010;25(2):1162–71. http://dx.doi.org/10.1109/TPWRD.
and hot spot temperature estimation. In: 2020 Intermountain engineering, tech- 2009.2030890.
nology and computing. 2020, p. 6–10. http://dx.doi.org/10.1109/IETC47856. [136] Amara Korba A, El Islem Karabadji N. Smart grid energy fraud detection using
2020.9249225. SVM. In: Proceedings - ICNAS 2019: 4th international conference on networking
[112] Carvalho TP, Soares FA, Vita R, Francisco RdP, Basto JP, Alcalá SG. A and advanced systems. IEEE; 2019, p. 1–6. http://dx.doi.org/10.1109/ICNAS.
systematic literature review of machine learning methods applied to predictive 2019.8807832.
maintenance. Comput Ind Eng 2019;137(April):106024. http://dx.doi.org/10. [137] Cabral JE, Pinto JO, Pinto AM. Fraud detection system for high and low voltage
1016/j.cie.2019.106024. electricity consumers based on data mining. In: 2009 IEEE power and energy
[113] Predictive maintenance for power & utility industry – neuron soundware, URL society general meeting. 2009, p. 1–5. http://dx.doi.org/10.1109/PES.2009.
https://www.neuronsw.com/. 5275809.
[114] Predictive layer, URL https://www.predictivelayer.com/. [138] Li J, Wang F. Non-technical loss detection in power grids with statistical profile
[115] Huuhtanen T, Jung A. Predictive maintenance of photovoltaic panels via deep images based on semi- supervised learning. Sensors (Switzerland) 2020;20(1).
learning. In: 2018 IEEE Data Science Workshop, DSW 2018 - Proceedings. IEEE; http://dx.doi.org/10.3390/s20010236.
2018, p. 66–70. http://dx.doi.org/10.1109/DSW.2018.8439898. [139] Nazmul Hasan M, Toma RN, Nahid AA, Manjurul Islam MM, Kim JM. Electricity
[116] Skydt MR, Bang M, Shaker HR. A probabilistic sequence classification ap- theft detection in smart grid systems: A CNN–LSTM based approach. Energies
proach for early fault prediction in distribution grids using long short-term 2019;12(17):1–18. http://dx.doi.org/10.3390/en12173310.
memory neural networks. Measurement: J Int Meas Confed 2021;170(July [140] Bhat RR, Trevizan RD, Sengupta R, Li X, Bretas A. Identifying nontechnical
2020):108691. http://dx.doi.org/10.1016/j.measurement.2020.108691. power loss via spatial and temporal deep learning. In: Proceedings - 2016
[117] Viegas JL, Esteves PR, Melício R, Mendes VM, Vieira SM. Solutions for detection 15th IEEE international conference on machine learning and applications. IEEE;
of non-technical losses in the electricity grid: A review. Renew Sustain Energy 2017, p. 272–9. http://dx.doi.org/10.1109/ICMLA.2016.107.
Rev 2017;80(May):1256–68. http://dx.doi.org/10.1016/j.rser.2017.05.193. [141] Johannesen NJ, Kolhe M, Goodwin M. Relative evaluation of regression
[118] Monedero I, Biscarri F, León C, Guerrero JI, Biscarri J, Millán R. Detection tools for urban area electrical energy demand forecasting. J Cleaner Prod
of frauds and other non-technical losses in a power utility using pearson 2019;218:555–64. http://dx.doi.org/10.1016/j.jclepro.2019.01.108.
coefficient, Bayesian networks and decision trees. Int J Electr Power Energy [142] Mocanu E, Nguyen PH, Gibescu M, Kling WL. Deep learning for estimating
Syst 2012;34(1):90–8. http://dx.doi.org/10.1016/j.ijepes.2011.09.009. building energy consumption. Sustain Energy Grids Netw 2016;6:91–9. http:
[119] Endesa detected approximately 65,000 cases of electricity fraud in 2018 - En- //dx.doi.org/10.1016/j.segan.2016.02.005.
desa, https://www.endesa.com/en/press/press-room/news/customers/endesa- [143] Bouktif S, Fiaz A, Ouni A, Serhani MA. Optimal deep learning LSTM model
detected-approximately-65000-cases-of-electricity-fraud-in-2018. for electric load forecasting using feature selection and genetic algorithm:
[120] Hu J, Vasilakos AV. Energy big data analytics and security: Challenges and Comparison with machine learning approaches. Energies 2018;11(7). http://
opportunities. IEEE Trans Smart Grid 2016;7(5):2432–6. http://dx.doi.org/10. dx.doi.org/10.3390/en11071636.
1109/TSG.2016.2563461. [144] Bourdeau M, qiang Zhai X, Nefzaoui E, Guo X, Chatellier P. Modeling and
[121] Ghori KMU, Imran M, Nawaz A, Abbasi RA, Ullah A, Szathmary L. Performance forecasting building energy consumption: A review of data-driven techniques.
analysis of machine learning classifiers for non-technical loss detection. J In: Sustainable cities and society. 2019, http://dx.doi.org/10.1016/j.scs.2019.
Ambient Intell Humaniz Comput 2020;(0123456789). http://dx.doi.org/10. 101533.
1007/s12652-019-01649-9. [145] Mujeeb S, Javaid N, Javaid S. Data analytics for price forecasting in smart
[122] Viegas JL, Esteves PR, Vieira SM. Clustering-based novelty detection for grids: A survey. Proceedings of the 21st international multi topic conference
identification of non-technical losses. Int J Electr Power Energy Syst 2018;1–10. http://dx.doi.org/10.1109/INMIC.2018.8595571.
2018;101(February):301–10. http://dx.doi.org/10.1016/j.ijepes.2018.03.031. [146] Luo B, Miao S. A literature survey on electricity price forecasting in deregulated
[123] Jokar P, Arianpoo N, Leung VC. Electricity theft detection in AMI using markets. In: ISPEC 2019 - 2019 IEEE sustainable power and energy conference:
customers’ consumption patterns. IEEE Trans Smart Grid 2016;7(1):216–26. grid modernization for energy revolution, proceedings. 2019, p. 359–65. http:
http://dx.doi.org/10.1109/TSG.2015.2425222. //dx.doi.org/10.1109/iSPEC48194.2019.8974939.
[124] Villar-Rodriguez E, Del Ser J, Oregi I, Bilbao MN, Gil-Lopez S. Detection of [147] Ahmad T, Zhang H, Yan B. A review on renewable energy and electricity
non-technical losses in smart meter data based on load curve profiling and time requirement forecasting models for smart grid and buildings. Sustainable Cities
series analysis. Energy 2017;137:118–28. http://dx.doi.org/10.1016/j.energy. Soc 2020;55(April 2019):102052. http://dx.doi.org/10.1016/j.scs.2020.102052.
2017.07.008. [148] Raza MQ, Khosravi A. A review on artificial intelligence based load demand
[125] Rodríguez RG, Mares JJ, Quintero CG. Computational intelligent approaches forecasting techniques for smart grid and buildings. Renew Sustain Energy Rev
for non-technical losses management of electricity. Energies 2020;13(9). http: 2015;50:1352–72. http://dx.doi.org/10.1016/j.rser.2015.04.065.
//dx.doi.org/10.3390/en13092393. [149] Wu L, Shahidehpour M. A hybrid model for integrated day-ahead electric-
[126] Glauner P, Boechat A, Dolberg L, State R, Bettinger F, Rangoni Y, et al. Large- ity price and load forecasting in smart grid. IET Gener Transm Distrib
scale detection of non-technical losses in imbalanced data sets. In: 2016 IEEE 2014;8(12):1937–50. http://dx.doi.org/10.1049/iet-gtd.2013.0927.
power and energy society innovative smart grid technologies conference. 2016, [150] Agrawal RK, Muchahary F, Tripathi MM. Ensemble of relevance vector
p. 1–5. http://dx.doi.org/10.1109/ISGT.2016.7781159. machines and boosted trees for electricity price forecasting. Appl Energy
[127] Figueroa G, Chen YS, Avila N, Chu CC. Improved practices in machine learning 2019;250(April):540–8. http://dx.doi.org/10.1016/j.apenergy.2019.05.062.
algorithms for NTL detection with imbalanced data. IEEE Power Energy Soc Gen [151] Mujeeb S, Javaid N, Ilahi M, Wadud Z, Ishmanov F, Afzal MK. Deep long short-
Meet 2018;2018-Janua:1–5. http://dx.doi.org/10.1109/PESGM.2017.8273852. term memory: A new price and load forecasting scheme for big data in smart
[128] Qin H, Zhou H, Cao J. Imbalanced learning algorithm based intelligent cities. Sustainability (Switzerland) 2019;11(4):1–29. http://dx.doi.org/10.3390/
abnormal electricity consumption detection. Neurocomputing 2020;402:112–23. su11040987.
http://dx.doi.org/10.1016/j.neucom.2020.03.085. [152] Khalid R, Javaid N, Al-zahrani FA, Aurangzeb K, Qazi EUH, Ashfaq T. Electricity
[129] Martino D, Decia F, Molinelli J, Fern A. Improving electric fraud detection using load and price forecasting using jaya-long short term memory (JLSTM) in smart
class imbalance strategies. (March):2010, p. 135–41. grids. Entropy 2020;22(1):10. http://dx.doi.org/10.3390/e22010010.
[130] Zanetti M, Jamhour E, Pellenz M, Penna M, Zambenedetti V, Chueiri I. A [153] Mujeeb S, Javaid N. ESAENARX And DE–RELM: Novel schemes for big
tunable fraud detection system for advanced metering infrastructure using short- data predictive analytics of electricity load and price. Sustainable Cities Soc
lived patterns. IEEE Trans Smart Grid 2019;10(1):830–40. http://dx.doi.org/10. 2019;51(May):101642. http://dx.doi.org/10.1016/j.scs.2019.101642.
1109/TSG.2017.2753738. [154] Tan M, Yuan S, Li S, Su Y, Li H, He FH. Ultra-short-term industrial
[131] Lu X, Zhou Y, Wang Z, Yi Y, Feng L, Wang F. Knowledge embedded semi- power demand forecasting using LSTM based hybrid ensemble learning.
supervised deep learning for detecting non-technical losses in the smart grid. IEEE Trans Power Syst 2020;35(4):2937–48. http://dx.doi.org/10.1109/TPWRS.
Energies 2019;12(18):1–18. http://dx.doi.org/10.3390/en12183452. 2019.2963109.

21
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

[155] Panapakidis IP, Dagoumas AS. Day-ahead electricity price forecasting via [179] Chen H, Wang S, Wang S, Li Y. Day-ahead aggregated load forecasting based
the application of artificial neural network based models. Appl Energy on two-terminal sparse coding and deep neural network fusion. Electr Power
2016;172:132–51. http://dx.doi.org/10.1016/j.apenergy.2016.03.089. Syst Res 2019;177(August). http://dx.doi.org/10.1016/j.epsr.2019.105987.
[156] Zhang F, Fleyeh H. A review of single artificial neural network models for [180] Oprea SV, Bara A. Machine learning algorithms for short-term load forecast in
electricity spot price forecasting. In: International conference on the european residential buildings using smart meters, sensors and big data solutions. IEEE
energy market. 2019-Septe, IEEE; 2019, p. 1–6. http://dx.doi.org/10.1109/ Access 2019;7:177874–89. http://dx.doi.org/10.1109/ACCESS.2019.2958383.
EEM.2019.8916423. [181] Machlev R, Tolkachov D, Levron Y, Beck Y. Dimension reduction for NILM
[157] Jahangir H, Tayarani H, Baghali S, Ahmadian A, Elkamel A, Golkar MA, classification based on principle component analysis. Electr Power Syst
et al. A novel electricity price forecasting approach based on dimension Res 2020;187(February 2019):106459. http://dx.doi.org/10.1016/j.epsr.2020.
reduction strategy and rough artificial neural networks. IEEE Trans Ind Inf 106459.
2020;16(4):2369–81. http://dx.doi.org/10.1109/TII.2019.2933009.
[182] Da Veloso AF, De Oliveira RG, Rodrigues AA, Rabelo RA, Rodrigues JJ. Cogni-
[158] Askari M, Keynia F. Mid-term electricity load forecasting by a new composite
tive smart plugs for signature identification of residential home appliance load
method based on optimal learning MLP algorithm. IET Gener Transm Distrib
using machine learning: From theory to practice. In: 2019 IEEE international
2020;14(5):845–52. http://dx.doi.org/10.1049/iet-gtd.2019.0797.
conference on communications workshops, ICC workshops 2019 - proceedings.
[159] Ziel F, Steinert R. Probabilistic mid- and long-term electricity price forecasting.
2019, http://dx.doi.org/10.1109/ICCW.2019.8756885.
Renew Sustain Energy Rev 2018;94(June):251–66. http://dx.doi.org/10.1016/
j.rser.2018.05.038, arXiv:1703.10806. [183] Adeoye O, Spataru C. Modelling and forecasting hourly electricity demand in
[160] Wei N, Li C, Peng X, Zeng F, Lu X. Conventional models and artificial west african countries. Appl Energy 2019;242(November 2018):311–33. http:
intelligence-based models for energy consumption forecasting: A review. J //dx.doi.org/10.1016/j.apenergy.2019.03.057.
Pet Sci Eng 2019;181(June):106187. http://dx.doi.org/10.1016/j.petrol.2019. [184] Bessec M, Fouquau J. Short-run electricity load forecasting with combinations
106187. of stationary wavelet transforms. European J Oper Res 2018;264(1):149–64.
[161] Kuster C, Rezgui Y, Mourshed M. Electrical load forecasting models: A critical http://dx.doi.org/10.1016/j.ejor.2017.05.037.
systematic review. Sustainable Cities Soc 2017;35(July):257–70. http://dx.doi. [185] Wei D, Wang J, Ni K, Tang G. Research and application of a novel hybrid model
org/10.1016/j.scs.2017.08.009. based on a deep neural network combined with fuzzy time series for energy
[162] Mamun AA, Sohel M, Mohammad N, Haque Sunny MS, Dipta DR, Hossain E. forecasting. Energies 2019;12(18). http://dx.doi.org/10.3390/en12183588.
A comprehensive review of the load forecasting techniques using single and [186] Zhang J, Wei YM, Li D, Tan Z, Zhou J. Short term electricity load forecasting
hybrid predictive models. IEEE Access 2020;8:134911–39. http://dx.doi.org/ using a hybrid model. Energy 2018;158:774–81. http://dx.doi.org/10.1016/j.
10.1109/ACCESS.2020.3010702. energy.2018.06.012.
[163] Suganthi L, Samuel AA. Energy models for demand forecasting - a review.
[187] Tong C, Li J, Lang C, Kong F, Niu J, Rodrigues JJ. An efficient deep model
Renew Sustain Energy Rev 2012;16(2):1223–40. http://dx.doi.org/10.1016/j.
for day-ahead electricity load forecasting with stacked denoising auto-encoders.
rser.2011.08.014.
J Parallel Distrib Comput 2018;117:267–73. http://dx.doi.org/10.1016/j.jpdc.
[164] Fallah SN, Ganjkhani M, Shamshirband S, wing Chau K. Computational intel-
2017.06.007.
ligence on short-term load forecasting: A methodological overview. Energies
[188] Ahmad W, Ayub N, Ali T, Irfan M, Awais M, Shiraz M, et al. Towards short
2019;12(3). http://dx.doi.org/10.3390/en12030393.
term electricity load forecasting using improved support vector machine and
[165] Member S, Member S, Systems E. A generalized knowledge-based short-term
extreme learning machine. Energies 2020;13(11):1–17. http://dx.doi.org/10.
load-forecasting technique. 8, (2):1993, p. 508–14.
[166] Cheng L, Yu T. Game-theoretic approaches applied to transactions in the open 3390/en13112907.
and ever-growing electricity markets from the perspective of power demand [189] Yang A, Li W, Yang X. Short-term electricity load forecasting based on
response: An overview. IEEE Access 2019;7:25727–62. http://dx.doi.org/10. feature selection and least squares support vector machines. Knowl-Based Syst
1109/ACCESS.2019.2900356. 2019;163:159–73. http://dx.doi.org/10.1016/j.knosys.2018.08.027.
[167] Guo L, Wang S, Chen H, Shi Q. A load identification method based on [190] Khuntia SR, Rueda JL, Van der Meijden MA. Long-term electricity load
active deep learning and discrete wavelet transform. IEEE Access 2020;8:1. forecasting considering volatility using multiplicative error model. Energies
http://dx.doi.org/10.1109/access.2020.3003778. 2018;11(12). http://dx.doi.org/10.3390/en11123308.
[168] Singhal V, Maggu J, Majumdar A. Simultaneous detection of multiple ap- [191] Yang Y, Li W, Gulliver TA, Li S. Bayesian deep learning-based probabilistic
pliances from smart-meter measurements via multi-label consistent deep load forecasting in smart grids. IEEE Trans Ind Inf 2020;16(7):4703–13. http:
dictionary learning and deep transform learning. IEEE Trans Smart Grid //dx.doi.org/10.1109/TII.2019.2942353.
2019;10(3):2969–78. http://dx.doi.org/10.1109/TSG.2018.2815763. [192] Pórtoles J, González C, Moguerza JM. Electricity price forecasting with dynamic
[169] Kim JG, Lee B. Appliance classification by power signal analysis based on multi- trees: A benchmark against the random forest approach. Energies 2018;11(6).
feature combination multi-layer LSTM. Energies 2019;12(14). http://dx.doi.org/ http://dx.doi.org/10.3390/en11061588.
10.3390/en12142804.
[193] Forecasting C, Deep U. HousEEC : Day-ahead household electrical energy.
[170] Wu X, Gao Y, Jiao D. Multi-label classification based on random forest algorithm
[194] Shi H, Xu M, Li R. Deep learning for household load forecasting-A novel pooling
for non-intrusive load monitoring system. Processes 2019;7(6). http://dx.doi.
deep RNN. IEEE Trans Smart Grid 2018;9(5):5271–80. http://dx.doi.org/10.
org/10.3390/pr7060337.
[171] Le TTH, Kang H, Kim H. Household appliance classification using lower 1109/TSG.2017.2686012.
odd-numbered harmonics and the bagging decision tree. IEEE Access [195] Acharya SK, Wi YM, Lee J. Short-term load forecasting for a single household
2020;8:55937–52. http://dx.doi.org/10.1109/ACCESS.2020.2981969. based on convolution neural networks using data augmentation. Energies
[172] Welikala S, Dinesh C, Ekanayake MPB, Godaliyadda RI, Ekanayake J. Incor- 2019;12(18). http://dx.doi.org/10.3390/en12183560.
porating appliance usage patterns for non-intrusive load monitoring and load [196] Guo Z, Zhou K, Zhang X, Yang S. A deep learning model for short-term
forecasting. IEEE Trans Smart Grid 2019;10(1):448–61. http://dx.doi.org/10. power load and probability density forecasting. Energy 2018;160:1186–200.
1109/TSG.2017.2743760. http://dx.doi.org/10.1016/j.energy.2018.07.090.
[173] Chelmis C, Kolte J, Prasanna VK. Big data analytics for demand response: [197] Kong W, Dong ZY, Jia Y, Hill DJ, Xu Y, Zhang Y. Short-term residential load
Clustering over space and time. In: Proceedings - 2015 IEEE international forecasting based on LSTM recurrent neural network. IEEE Trans Smart Grid
conference on big data. IEEE; 2015, p. 2223–32. http://dx.doi.org/10.1109/ 2019;10(1):841–51. http://dx.doi.org/10.1109/TSG.2017.2753802.
BigData.2015.7364011. [198] Solyali D. A comparative analysis of machine learning approaches for
[174] Singh S, Yassine A. Big data mining of energy time series for behavioral short-/long-term electricity load forecasting in cyprus. Sustainability
analytics and energy consumption forecasting. Energies 2018;11(2). http://dx. 2020;12(9):3612. http://dx.doi.org/10.3390/su12093612.
doi.org/10.3390/en11020452.
[199] Luo XJ. A novel clustering-enhanced adaptive artificial neural network
[175] Wang Y, Chen Q, Kang C, Xia Q. Clustering of electricity consumption
model for predicting day-ahead building cooling demand. J Build Eng
behavior dynamics toward big data applications. IEEE Trans Smart Grid
2020;32(March):101504. http://dx.doi.org/10.1016/j.jobe.2020.101504.
2016;7(5):2437–47. http://dx.doi.org/10.1109/TSG.2016.2548565.
[176] Singh S, Yassine A. Mining energy consumption behavior patterns for house- [200] Shah NM. Review on forecasting of photovoltaic power generation based
holds in smart grid. IEEE Trans Emerg Top Comput 2019;7(3):404–19. http: on machine learning and metaheuristic techniques. IET Renew Power Gener
//dx.doi.org/10.1109/TETC.2017.2692098. 2019;13(7). 1009–1023(14), URL https://digital-library.theiet.org/content/
[177] Wijaya TK, Vasirani M, Humeau S, Aberer K. Cluster-based aggregate forecast- journals/10.1049/iet-rpg.2018.5649.
ing for residential electricity demand using smart meter data. In: Proceedings [201] Sobri S, Koohi-Kamali S, Rahim NA. Solar photovoltaic generation forecasting
- 2015 IEEE international conference on big data. IEEE; 2015, p. 879–87. methods: A review. Energy Convers Manage 2018;156(May 2017):459–97.
http://dx.doi.org/10.1109/BigData.2015.7363836. http://dx.doi.org/10.1016/j.enconman.2017.11.019, https://doi.org/10.1016/j.
[178] Satre-Meloy A, Diakonova M, Grünewald P. Cluster analysis and prediction enconman.2017.11.019.
of residential peak demand profiles using occupant activity data. Appl En- [202] Zendehboudi A, Baseer MA, Saidur R. Application of support vector machine
ergy 2020;260(November 2019):114246. http://dx.doi.org/10.1016/j.apenergy. models for forecasting solar and wind energy resources: A review. J Cleaner
2019.114246. Prod 2018;199:272–85. http://dx.doi.org/10.1016/j.jclepro.2018.07.164.

22
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

[203] Sharifzadeh M, Sikinioti-Lock A, Shah N. Machine-learning methods for inte- [225] Zhou B, Ma X, Luo Y, Yang D. Wind power prediction based on
grated renewable power generation: A comparative study of artificial neural LSTM networks and nonparametric kernel density estimation. IEEE Access
networks, support vector regression, and Gaussian process regression. Renew 2019;7(Mcc):165279–92. http://dx.doi.org/10.1109/ACCESS.2019.2952555.
Sustain Energy Rev 2019;108(July 2018):513–38. http://dx.doi.org/10.1016/j. [226] Qureshi AS, Khan A, Zameer A, Usman A. Wind power prediction using deep
rser.2019.03.040. neural network based meta regression and transfer learning. Appl Soft Comput
[204] Gigoni L, Betti A, Crisostomi E, Franco A, Tucci M, Bizzarri F, et al. Day- 2017;58:742–55. http://dx.doi.org/10.1016/j.asoc.2017.05.031.
ahead hourly forecasting of power generation from photovoltaic plants. IEEE [227] Nowotarski J, Weron R. Recent advances in electricity price forecasting: A
Trans Sustain Energy 2018;9(2):831–42. http://dx.doi.org/10.1109/TSTE.2017. review of probabilistic forecasting. Renew Sustain Energy Rev 2018;81(March
2762435, arXiv:1903.06800. 2017):1548–68. http://dx.doi.org/10.1016/j.rser.2017.05.234.
[205] Wang K, Qi X, Liu H. A comparison of day-ahead photovoltaic power [228] Lago J, De Ridder F, De Schutter B. Forecasting spot electricity prices: Deep
forecasting models based on deep learning neural network. Appl Energy learning approaches and empirical comparison of traditional algorithms. Appl
2019;251(May):113315. http://dx.doi.org/10.1016/j.apenergy.2019.113315. Energy 2018;221(April):386–405. http://dx.doi.org/10.1016/j.apenergy.2018.
[206] Jeong J, Kim H. Multi-site photovoltaic forecasting exploiting space-time con- 02.069.
volutional neural network. Energies 2019;12(23). http://dx.doi.org/10.3390/ [229] Demir S, Mincev K, Kok K, Paterakis NG. Introducing technical indicators to
en12234490. electricity price forecasting: A feature engineering study for linear, ensemble,
and deep machine learning models. Appl Sci 2019;10(1):255. http://dx.doi.org/
[207] Tascikaraoglu A, Uzunoglu M. A review of combined approaches for pre-
10.3390/app10010255.
diction of short-term wind speed and power. Renew Sustain Energy Rev
[230] Kuo PH, Huang CJ. An electricity price forecasting model by hybrid structured
2014;34:243–54. http://dx.doi.org/10.1016/j.rser.2014.03.033.
deep neural networks. Sustainability (Switzerland) 2018;10(4):1–17. http://dx.
[208] Marugán AP, Márquez FPG, Perez JMP, Ruiz-Hernández D. A survey of artificial
doi.org/10.3390/su10041280.
neural network in wind energy systems. Appl Energy 2018;228(June):1822–36.
[231] Atef S, Eltawil AB. A comparative study using deep learning and support vector
http://dx.doi.org/10.1016/j.apenergy.2018.07.084.
regression for electricity price forecasting in smart grids. In: 2019 IEEE 6th
[209] zhi Wang H, qiang Li G, bing Wang G, chun Peng J, Jiang H, tao Liu Y. Deep
international conference on industrial engineering and applications. IEEE; 2019,
learning based ensemble approach for probabilistic wind power forecasting.
p. 603–7. http://dx.doi.org/10.1109/IEA.2019.8715213.
Appl Energy 2017;188:56–70. http://dx.doi.org/10.1016/j.apenergy.2016.11.
[232] Luo S, Weng Y. A two-stage supervised learning approach for electric-
111.
ity price forecasting by leveraging different data sources. Appl Energy
[210] Eseye AT, Zhang J, Zheng D. Short-term photovoltaic solar power forecasting 2019;242(February):1497–512. http://dx.doi.org/10.1016/j.apenergy.2019.03.
using a hybrid wavelet-PSO-svm model based on SCADA and meteorologi- 129.
cal information. Renew Energy 2018;118:357–67. http://dx.doi.org/10.1016/ [233] Chinnathambi RA, Plathottam SJ, Hossen T, Nair AS, Ranganathan P. Deep
j.renene.2017.11.011. neural networks (DNN) for day-ahead electricity price markets. In: 2018 IEEE
[211] Liu H, wei Mi X, fei Li Y. Wind speed forecasting method based on deep learning electrical power and energy conference. IEEE; 2018, http://dx.doi.org/10.1109/
strategy using empirical wavelet transform, long short term memory neural net- EPEC.2018.8598327.
work and elman neural network. Energy Convers Manage 2018;156(September [234] Ugurlu U, Oksuz I, Tas O. Electricity price forecasting using recurrent neural
2017):498–514. http://dx.doi.org/10.1016/j.enconman.2017.11.053. networks. Energies 2018;11(5):1–23. http://dx.doi.org/10.3390/en11051255.
[212] Raza MQ, Nadarajah M, Li J, Lee KY, Gooi HB. An ensemble framework for [235] Windler T, Busse J, Rieck J. One month-ahead electricity price forecasting in
day-ahead forecast of PV output in smart grids. IEEE Trans Ind Inf 2018;1–10. the context of production planning. J Cleaner Prod 2019;238:117910. http:
http://dx.doi.org/10.1109/TII.2018.2882598. //dx.doi.org/10.1016/j.jclepro.2019.117910.
[213] Shaker H, Zareipour H, Wood D. Estimating power generation of invisible solar [236] Wilcox T, Jin N, Flach P, Thumim J. A big data platform for smart meter data
sites using publicly available data. IEEE Trans Smart Grid 2016;7(5):2456–65. analytics. Comput Ind 2019;105:250–9. http://dx.doi.org/10.1016/j.compind.
http://dx.doi.org/10.1109/TSG.2016.2533164. 2018.12.010.
[214] Qing X, Niu Y. Hourly day-ahead solar irradiance prediction using weather fore- [237] Shiri A, Afshar M, Rahimi-Kian A, Maham B. Electricity price forecasting using
casts by LSTM. Energy 2018;148:461–8. http://dx.doi.org/10.1016/j.energy. support vector machines by considering oil and natural gas price impacts. In:
2018.01.177. International conference on smart energy grid engineering. IEEE; 2015, p. 1–5.
[215] Chen B, Lin P, Lai Y, Cheng S, Chen Z, Wu L. Very-short-term power prediction http://dx.doi.org/10.1109/SEGE.2015.7324591.
for PV power plants using a simple and effective RCC–LSTM model based on [238] Díaz G, Coto J, Gómez-Aleixandre J. Prediction and explanation of the
short term multivariate historical datasets. Electronics (Switzerland) 2020;9(2). formation of the spanish day-ahead electricity price through machine learn-
http://dx.doi.org/10.3390/electronics9020289. ing regression. Appl Energy 2019;239(February):610–25. http://dx.doi.org/10.
[216] Abdel-Nasser M, Mahmoud K. Accurate photovoltaic power forecasting models 1016/j.apenergy.2019.01.213.
using deep LSTM–RNN. Neural Comput Appl 2019;31(7):2727–40. http://dx. [239] Xiao C, Sutanto D, Muttaqi KM, Zhang M. Online sequential extreme learning
doi.org/10.1007/s00521-017-3225-z. machine algorithm for better prediction of the real-time electricity price
[217] Zang H, Cheng L, Ding T, Cheung KW, Liang Z, Wei Z, et al. Hybrid method for under dynamic environmental changes. In: 2019 IEEE industry applications
short-term photovoltaic power forecasting based on deep convolutional neural society annual meeting. IEEE; 2019, p. 1–6. http://dx.doi.org/10.1109/IAS.
network. IET Gener Transm Distrib 2018;12(20):4557–67. http://dx.doi.org/10. 2019.8912333.
1049/iet-gtd.2018.5847. [240] Wang J, Yang W, Du P, Niu T. Outlier-robust hybrid electricity price forecasting
model for electricity market management. J Cleaner Prod 2020;249:119318.
[218] Liu L, Zhao Y, Chang D, Xie J, Ma Z, Sun Q, et al. Predic-
http://dx.doi.org/10.1016/j.jclepro.2019.119318.
tion of short-term PV power output and uncertainty analysis. Appl En-
[241] Bisoi R, Dash PK, Das PP. Short-term electricity price forecasting and classifi-
ergy 2018;228(November):700–11. http://dx.doi.org/10.1016/j.apenergy.2018.
cation in smart grids using optimized multikernel extreme learning machine.
06.112.
Neural Comput Appl 2020;32(5):1457–80. http://dx.doi.org/10.1007/s00521-
[219] Lee D, Kim K. Recurrent neural network-based hourly prediction of photovoltaic
018-3652-5.
power output using meteorological information. Energies 2019;12(2). http:
[242] Wang J, Liu F, Song Y, Zhao J. A novel model: Dynamic choice artificial neural
//dx.doi.org/10.3390/en12020215.
network (DCANN) for an electricity price forecasting system. Appl Soft Comput
[220] Rodríguez F, Fleetwood A, Galarza A, Fontán L. Predicting solar energy gener-
2016;48:281–97. http://dx.doi.org/10.1016/j.asoc.2016.07.011.
ation through artificial neural networks using weather forecasts for microgrid
[243] Alshejari A, Kodogiannis VS, Leonidis S. Development of neurofuzzy architec-
control. Renew Energy 2018;126:855–64. http://dx.doi.org/10.1016/j.renene.
tures for electricity price forecasting. 2020, p. 1–25. http://dx.doi.org/10.3390/
2018.03.070.
en13051209.
[221] Zhou H, Zhang Y, Yang L, Liu Q, Yan K, Du Y. Short-term photovoltaic power [244] De Marcos RA, Reneses J, Bello A. Long-term spanish electricity market
forecasting based on long short term memory neural network and attention price forecasting with cointegration and VEC models. In: 2016 International
mechanism. IEEE Access 2019;7:78063–74. http://dx.doi.org/10.1109/ACCESS. conference on probabilistic methods applied to power systems, PMAPS 2016 -
2019.2923006. proceedings. IEEE; 2016, http://dx.doi.org/10.1109/PMAPS.2016.7764158.
[222] Ju Y, Sun G, Chen Q, Zhang M, Zhu H, Rehman MU. A model combining [245] Universal Smart Energy Framework (USEF). USEF: The Framework Explained.
convolutional neural network and lightgbm algorithm for ultra-short-term wind Tech. rep, 2015, p. 1–55, www.usef.energy/download-the-framework/.
power forecasting. IEEE Access 2019;7:28309–18. http://dx.doi.org/10.1109/ [246] Dalhues S, Zhou Y, Pohl O, Rewald F, Erlemeyer F, Schmid D, et al. Towards
ACCESS.2019.2901920. research and practice of flexibility in distribution systems: A review. CSEE
[223] Lahouar A, Ben Hadj Slama J. Hour-ahead wind power forecast based on J Power Energy Syst 2019;5(3):285–94. http://dx.doi.org/10.17775/cseejpes.
random forests. Renew Energy 2017;109:529–41. http://dx.doi.org/10.1016/j. 2019.00170.
renene.2017.03.064. [247] Ahmadiahangar R, Häring T, Rosin A, Korõtko T, Martins J. Residential load
[224] Chen J, Zeng GQ, Zhou W, Du W, Lu KD. Wind speed forecasting using forecasting for flexibility prediction using machine learning-based regression
nonlinear-learning ensemble of deep learning time series prediction and ex- model. In: Proceedings - 2019 IEEE international conference on environment
tremal optimization. Energy Convers Manage 2018;165(March):681–95. http: and electrical engineering and 2019 IEEE industrial and commercial power
//dx.doi.org/10.1016/j.enconman.2018.03.098. systems Europe. 2019, http://dx.doi.org/10.1109/EEEIC.2019.8783634.

23
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

[248] Mercedes V. Efficient implementation and potential benefits of demand response [272] Huang X, Hong SH, Yu M, DIng Y, Jiang J. Demand response management
in electricity distribution networks. 2017, URL https://repositorio.comillas.edu/ for industrial facilities: A deep reinforcement learning approach. IEEE Access
jspui/bitstream/11531/20478/1/TD00274.pdf. 2019;7:82194–205. http://dx.doi.org/10.1109/ACCESS.2019.2924030.
[249] BEUC. Electricity aggregators: Starting off on the right foot with consumers. [273] Ruelens F, Claessens BJ, Quaiyum S, De Schutter B, Babuška R, Belmans R.
32, (9505781573):2018, p. 0–11. Reinforcement learning applied to an electric water heater: From theory to
[250] Heinrich C, Ziras C, Syrri AL, Bindner HW. Ecogrid 2.0: A large-scale field trial practice. IEEE Trans Smart Grid 2018;9(4):3792–800. http://dx.doi.org/10.
of a local flexibility market. Appl Energy 2020;261(December 2019):114399. 1109/TSG.2016.2640184, arXiv:1512.00408.
http://dx.doi.org/10.1016/j.apenergy.2019.114399. [274] Wan Z, He HLH, Prokhorov D. Model-free real-time EV charging schedul-
[251] Iria J, Soares F, Matos M. Optimal bidding strategy for an aggregator of ing based on deep reinforcement learning. IEEE Trans Smart Grid
prosumers in energy and secondary reserve markets. Appl Energy 2019;238(July 2018;10(5):5246–57. http://dx.doi.org/10.1109/TSG.2018.2879572.
2018):1361–72. http://dx.doi.org/10.1016/j.apenergy.2019.01.191. [275] Xu X, Jia Y, Xu Y, Xu Z, Chai S, Lai CS. A multi-agent reinforcement learning-
[252] Ottesen SØ, Tomasgard A, Fleten S-E. Multi market bidding strategies for based data-driven method for home energy management. IEEE Trans Smart Grid
demand side flexibility aggregators in electricity markets. European J Oper Res 2020;11(4):3201–11. http://dx.doi.org/10.1109/tsg.2020.2971427.
2017;149:120–34. http://dx.doi.org/10.1016/j.energy.2018.01.187. [276] Lu R, Hong SH, Zhang X. A dynamic pricing demand response al-
gorithm for smart grid: Reinforcement learning approach. Appl En-
[253] Lucas A, Jansen L, Andreadou N, Kotsakis E, Masera M. Load flexibility forecast
ergy 2018;220(February):220–30. http://dx.doi.org/10.1016/j.apenergy.2018.
for DR using non-intrusive load monitoring in the residential sector. Energies
03.072.
2019;12(14):2725. http://dx.doi.org/10.3390/en12142725.
[277] Wen L, Zhou K, Li J, Wang S. Modified deep learning and reinforcement learn-
[254] Krishnadas G, Kiprakis A. A machine learning pipeline for demand re-
ing for an incentive-based demand response model. Energy 2020;205:118019.
sponse capacity scheduling. Energies 2020;13(7):1–25. http://dx.doi.org/10.
http://dx.doi.org/10.1016/j.energy.2020.118019.
3390/en13071848.
[278] Mocanu E, Mocanu DC, Nguyen PH, Liotta A, Webber ME, Gibescu M, et al.
[255] Kotsis G, Moschos I, Corchero C, Cruz-Zambrano M. Demand aggregator
On-line building energy optimization using deep reinforcement learning. IEEE
flexibility forecast: Price incentives sensitivity assessment. In: International
Trans Smart Grid 2019;10(4):3698–708. http://dx.doi.org/10.1109/TSG.2018.
conference on the european energy market, EEM, Vol. 2015-Augus. IEEE; 2015,
2834219, arXiv:1707.05878.
p. 1–5. http://dx.doi.org/10.1109/EEM.2015.7216756.
[279] Foruzan E, Soh LK, Asgarpoor S. Reinforcement learning approach for opti-
[256] Vellei M, Le Dréau J, Abdelouadoud SY. Predicting the demand flexibility of wet mal distributed energy management in a microgrid. IEEE Trans Power Syst
appliances at national level: The case of France. Energy Build 2020;214:109900. 2018;33(5):5749–58. http://dx.doi.org/10.1109/TPWRS.2018.2823641.
http://dx.doi.org/10.1016/j.enbuild.2020.109900. [280] Ruelens F, Claessens BJ, Vandael S, De Schutter B, Babuska R, Belmans R.
[257] Pinto R, Bessa RJ, Matos MA. Multi-period flexibility forecast for low volt- Residential demand response of thermostatically controlled loads using batch
age prosumers. Energy 2017;141:2251–63. http://dx.doi.org/10.1016/j.energy. reinforcement learning. IEEE Trans Smart Grid 2017;8(5):2149–59. http://dx.
2017.11.142, arXiv:1703.08825. doi.org/10.1109/TSG.2016.2517211.
[258] Ponocko J, Milanovic JV. Forecasting demand flexibility of aggregated residen- [281] Remani T, Jasmin EA, Ahamed TP. Residential load scheduling with renewable
tial load using smart meter data. IEEE Trans Power Syst 2018;33(5):5446–55. generation in the smart grid: A reinforcement learning approach. IEEE Syst J
http://dx.doi.org/10.1109/TPWRS.2018.2799903. 2019;13(3):3283–94. http://dx.doi.org/10.1109/JSYST.2018.2855689.
[259] Ziras C, Heinrich C, Pertl M, Bindner HW. Experimental flexibility identification [282] Lange S. RM. Batch reinforcement learning. In: Reinforcement Learning:
of aggregated residential thermal loads using behind-the-meter data. Appl State-of-the-Art. 2012, p. 45–73. http://dx.doi.org/10.1007/978-3-642-27645-
Energy 2019;242(March):1407–21. http://dx.doi.org/10.1016/j.apenergy.2019. 3_2.
03.156. [283] Carreiro AM, Jorge HM, Antunes CH. Energy management systems aggre-
[260] Barja-Martinez S, Rucker F, Aragues-Penalba M, Villafafila-Robles R, Munne- gators: A literature survey. Renew Sustain Energy Rev 2017;73(December
Collado I, Lloret-Gallego P. A novel hybrid home energy management system 2016):1160–72. http://dx.doi.org/10.1016/j.rser.2017.01.179.
considering electricity cost and greenhouse gas emissions minimization. IEEE [284] Eurelectric. Flexibility and aggregation - requirements for their interaction in
Trans Ind Appl 2021;1. http://dx.doi.org/10.1109/tia.2021.3057014. the market. IEEE Trans Power Syst 2015;30(January):13. http://dx.doi.org/10.
[261] Antonopoulos I, Robu V, Couraud B, Kirli D, Norbu S, Kiprakis A, et al. Artificial 1109/TPWRS.2014.2328865.
intelligence and machine learning approaches to energy demand-side response: [285] Iria J, Soares F. A cluster-based optimization approach to support the participa-
A systematic review. Renew Sustain Energy Rev 2020;130(June):109899. http: tion of an aggregator of a larger number of prosumers in the day-ahead energy
//dx.doi.org/10.1016/j.rser.2020.109899. market. Electr Power Syst Res 2019;168(July 2018):324–35. http://dx.doi.org/
[262] Shareef H, Ahmed MS, Mohamed A, Al Hassan E. Review on home energy 10.1016/j.epsr.2018.11.022.
management system considering demand responses, smart technologies, and [286] Olivella-Rosell P, Rullan F, Lloret-Gallego P, Prieto-Araujo E, Ferrer-San-José R,
intelligent controllers. IEEE Access 2018;6:24498–509. http://dx.doi.org/10. Barja-Martinez S, et al. Centralised and distributed optimization for aggregated
1109/ACCESS.2018.2831917. flexibility services provision. 2019, p. 1–10, URL arXiv:1907.08125, http://
[263] Kumari A, Gupta R, Tanwar S, Kumar N. Blockchain and AI amalgamation arxiv.org/abs/1907.08125.
for energy cloud management: Challenges, solutions, and future directions. [287] Iria J, Scott P, Attarha A. Network-constrained bidding optimization strategy
J Parallel Distrib Comput 2020;143:148–66. http://dx.doi.org/10.1016/j.jpdc. for aggregators of prosumers. Energy 2020;207. http://dx.doi.org/10.1016/j.
2020.05.004. energy.2020.118266.
[288] Shen F, Wu Q, Jin X, Zhou B, Li C, Xu Y. ADMM-based market clearing and
[264] Khalid Z, Abbas G, Awais M, Alquthami T, Rasheed MB. A novel load scheduling
optimal flexibility bidding of distribution-level flexibility market for day-ahead
mechanism using artificial neural network based customer profiles in smart grid.
congestion management of distribution networks. Int J Electr Power Energy
Energies 2020;13(5):1–23. http://dx.doi.org/10.3390/en13051062.
Syst 2020;123(May):106266. http://dx.doi.org/10.1016/j.ijepes.2020.106266.
[265] Yuce B, Rezgui Y, Mourshed M. ANN–GA smart appliance scheduling
[289] Huang S, Wu Q, Shahidehpour M, Liu Z. Distributed optimization-based
for optimised energy management in the domestic sector. Energy Build
dynamic tariff for congestion management in distribution networks. IEEE
2016;111:311–25. http://dx.doi.org/10.1016/j.enbuild.2015.11.017.
Trans Smart Grid 2019;10(2):2148–57. http://dx.doi.org/10.1109/TSG.2018.
[266] Smarra F, Jain A, de Rubeis T, Ambrosini D, D’Innocenzo A, Mangharam R.
2790638.
Data-driven model predictive control using random forests for building energy
[290] Lezama F, Soares J, Canizes B, Vale Z. Flexibility management model of
optimization and climate control. Appl Energy 2018;226(February):1252–72.
home appliances to support DSO requests in smart grids. Sustainable Cities
http://dx.doi.org/10.1016/j.apenergy.2018.02.126.
Soc 2020;55(November 2018):102048. http://dx.doi.org/10.1016/j.scs.2020.
[267] Lu R, Hong SH, Yu M. Demand response for home energy management using 102048.
reinforcement learning and artificial neural network. IEEE Trans Smart Grid [291] Macdougall P, Ran B, Klever M, Deconinck G. Value assessment of aggregated
2019;10(6):6629–39. energy flexibility when traded on multiple markets. In: International conference
[268] Ahmed M, Mohamed A, Homod R, Shareef H. Hybrid LSA–ANN based home on the european energy market. IEEE; 2017, http://dx.doi.org/10.1109/EEM.
energy management scheduling controller for residential demand response 2017.7981960.
strategy. Energies 2016;9(9):716. http://dx.doi.org/10.3390/en9090716. [292] Huang S, Wu Q. Real-time congestion management in distribution networks
[269] Bourhnane S, Abid MR, Lghoul R, Zine-Dine K, Elkamoun N, Benhaddou D. by flexible demand swap. IEEE Trans Smart Grid 2018;9(5):4346–55. http:
Machine learning for energy consumption prediction and scheduling in smart //dx.doi.org/10.1109/TSG.2017.2655085.
buildings. SN Appl Sci 2020;2(2):1–10. http://dx.doi.org/10.1007/s42452-020- [293] BDEW German Association of Energy and Water Industries. Smart grid traffic
2024-9. light concept - Design of the amber phase. Tech. Rep, (March). Berlin: BDEW
[270] Liu D, Sun Y, Qu Y, Li B, Xu Y. Analysis and accurate prediction of German Association of Energy and Water Industries; 2015.
user’s response behavior in incentive-based demand response. IEEE Access [294] Olivella-Rosell P, Lloret-Gallego P, Munné-Collado Í, Villafafila-Robles R,
2019;7:3170–80. http://dx.doi.org/10.1109/ACCESS.2018.2889500. Sumper A, Ottessen S, et al. Local flexibility market design for aggregators
[271] Vázquez-Canteli JR, Nagy Z. Reinforcement learning for demand response: A re- providing multiple flexibility services at distribution network level. Ener-
view of algorithms and modeling techniques. Appl Energy 2019;235(November gies 2018;11(4):822. http://dx.doi.org/10.3390/en11040822, URL http://www.
2018):1072–89. http://dx.doi.org/10.1016/j.apenergy.2018.11.002. mdpi.com/1996-1073/11/4/822.

24
S. Barja-Martinez et al. Renewable and Sustainable Energy Reviews 150 (2021) 111459

[295] Paridari K, Nordström L. Flexibility prediction, scheduling and control of [314] Jamil F, Iqbal N, Imran, Ahmad S, Kim DH. Peer-to-peer energy trading
aggregated TCLs. Electr Power Syst Res 2020;178(February 2019):106004. mechanism based on blockchain and machine learning for sustainable electrical
http://dx.doi.org/10.1016/j.epsr.2019.106004. power supply in smart grid. IEEE Access 2021;39193–217. http://dx.doi.org/
[296] Ziras C, Heinrich C, Pertl M, Bindner HW. Experimental flexibility identification 10.1109/ACCESS.2021.3060457.
of aggregated residential thermal loads using behind-the-meter data. Appl En- [315] Chen K, Lin J, Song Y. Trading strategy optimization for a prosumer in
ergy 2019;242(October 2018):1407–21. http://dx.doi.org/10.1016/j.apenergy. continuous double auction-based peer-to-peer market: A prediction-integration
2019.03.156. model. Appl Energy 2019;242(March):1121–33. http://dx.doi.org/10.1016/j.
[297] Amasyali K, Olama M, Perumalla A. A machine learning-based approach to apenergy.2019.03.094.
predict the aggregate flexibility of HVAC systems. In: 2020 IEEE power and [316] Mei VD. Paper 0303 artificial intelligence for microgrid planning [CIRED|
energy society innovative smart grid technologies conference. 2020, p. 0–4. workshop - Ljubljana , 7-8 June 2018 Paper 0303, 0303, 2018, p. 7–8.
http://dx.doi.org/10.1109/ISGT45199.2020.9087695. [317] Fu X, Guo Q, Sun H. Statistical machine learning model for stochastic
[298] Short M, Rodriguez S, Charlesworth R, Crosbie T, Dawood N. Optimal dispatch optimal planning of distribution networks considering a dynamic correlation
of aggregated HVAC units for demand response: An industry 4.0 approach. and dimension reduction. IEEE Trans Smart Grid 2020;11(4):2904–17. http:
Energies 2019;12(22). http://dx.doi.org/10.3390/en12224320. //dx.doi.org/10.1109/tsg.2020.2974021.
[299] Correa-Florez CA, Michiorri A, Kariniotakis G. Comparative analysis of ad- [318] Resener M, Haffner S, Pereira LA, Pardalos PM. Optimization techniques applied
justable robust optimization alternatives for the participation of aggregated to planning of electric power distribution systems: A bibliographic survey.
residential prosumers in electricity markets. Energies 2019;12(6). http://dx.doi. Energy Syst 2018;9(3):473–509. http://dx.doi.org/10.1007/s12667-018-0276-
org/10.3390/en12061019. x.
[300] Camero A, Luque G, Bravo Y, Alba E. Customer segmentation based on the [319] Klyapovskiy S, You S, Cai H, Bindner HW. Incorporate flexibility in distribution
electricity demand signature: The andalusian case. Energies 2018;11(7):1–14. grid planning through a framework solution. Int J Electr Power Energy Syst
http://dx.doi.org/10.3390/en11071788. 2019;111(February):66–78. http://dx.doi.org/10.1016/j.ijepes.2019.03.069.
[301] Qhuj IRU, Xvlqj H, Rulv LQJ, Oh HH. Comparison of clustering techniques [320] Contreras J, Asensio M, Meneses P, Munoz-Delgado G, Montoyo-Bueno S. Joint
for residential energy behavior using smart meter data. In: AAAI workshops - RES and distribution network expansion planning under a demand response
Artificial intelligence for smart grids and buildings, March 2017. San Francisco, framework. In: Joint RES and distribution network expansion planning under
CA; 2017, p. 260–6, URL https://www.aaai.org/ocs/index.php/WS/AAAIW17/ a demand response framework. 2016, p. 1–102. http://dx.doi.org/10.1016/
paper/view/15166/14673. c2015-0-04164-8.
[302] Rajabi A, Eskandari M, Ghadi MJ, Li L, Zhang J, Siano P. A comparative study [321] National Renewable Energy Laboratory, Arara Blue Energy Group. VOICES
of clustering techniques for electrical load pattern segmentation. Renew Sustain of experience - Leveraging AMI networks and data. Tech. rep, U.S. Depart-
Energy Rev 2020;120(December 2019). http://dx.doi.org/10.1016/j.rser.2019. ment of Energy; 2019, p. 1–64, URL https://www.smartgrid.gov/document/
109628. VOE{_}Leveraging{_}AMI{_}Networks{_}Data.
[303] Hernández L, Baladrón C, Aguiar JM, Carro B, Sánchez-Esguevillas A. Classifica- [322] Gold R, Waters C, York D. Leveraging advanced metering infrastructure to save
tion and clustering of electricity demand patterns in industrial parks. Energies energy. Tech. rep, American Council for an Energy-Efficient Economy (ACEEE);
2012;5(12):5215–28. http://dx.doi.org/10.3390/en5125215. 2020, p. 61, URL https://aceee.org/research-report/u2001.
[304] Siano P, De Marco G, Rolan A, Loia V. A survey and evaluation of the potentials [323] Sagiroglu S, Terzi R, Canbay Y, Colak I. Big data issues in smart grid systems.
of distributed ledger technology for peer-to-peer transactive energy exchanges In: 2016 IEEE international conference on renewable energy research and
in local energy markets. IEEE Syst J 2019;13(3):3454–66. http://dx.doi.org/10. applications. 5, IEEE; 2017, p. 1007–12. http://dx.doi.org/10.1109/ICRERA.
1109/JSYST.2019.2903172. 2016.7884486.
[305] D’Oriano L, Mastandrea G, Rana G, Raveduto G, Croce V, Verber M, [324] (BDVA) BDVA. European big data value strategic research and innovation
Bertoncini M. Smart grids : Requirements engineering and use cases. In: IEEE agenda - BDV SRIA. Tech. rep, 2017, p. 106, URL www.bdva.eu/sites/default/
international conference and workshop in Óbuda on electrical and power files/BDVA{_}SRIA{_}v4{_}Ed1.1.pdf.
engineering. 2020, p. 39–44. [325] Expert Group for Standards and Interoperability for Smart Grids Deployment.
[306] Ratanasopitkul P. Blockchain - revolutionize green energy management. In: Interoperability, standards and functionalities applied in the large scale roll out
Proceedings of the conference on the industrial and commercial use of energy, of smart metering. Smart Grids Task Force 2015;(October):1–31.
ICUE, Vol. 2018-Octob. (October):Asian Institute of Technology; 2019, p. 1–6. [326] European Network of Transmission System Operators for Electricity
http://dx.doi.org/10.23919/ICUE-GESD.2018.8635666. (ENTSO-E). Annual report 2015: Electricity without borders. 2015, p.
[307] Pop C, Cioara T, Antal M, Anghel I, Salomie I, Bertoncini M. Blockchain based 22, URL https://eepublicdownloads.blob.core.windows.net/public-cdn-
decentralized management of demand response programs in smart energy grids. container/clean-documents/Publications/ENTSO-Egeneralpublications/ENTSO-
Sensors (Switzerland) 2018;18(1). http://dx.doi.org/10.3390/s18010162. E{_}AR15{_}FINAL.pdf.
[308] Troncia M, Galici M, Mureddu M, Ghiani E, Pilo F. Distributed ledger [327] Chui M, Manyika J, Miremadi M, Henke N, Chung R, Nel P, et al. Notes from
technologies for peer-to-peer local markets in distribution networks. Energies the AI frontier: Applications and value of deep learning. 2018.
2019;12(17). http://dx.doi.org/10.3390/en12173249. [328] Kufeoglu S, Liu G, Pollitt M. Digitalisation and new business models in energy
[309] Zhang K, Troitzsch S, Hanif S, Hamacher T. Coordinated market design for sector. EPRG working paper . energy policy research group, University of
peer-to-peer energy trade and ancillary services in distribution grids. IEEE Cambridge, Vol. 1920, June, p. 1–39, URL https://www.eprg.group.cam.ac.uk/
Trans Smart Grid 2020;11(4):2929–41. http://dx.doi.org/10.1109/TSG.2020. wp-content/uploads/2019/06/1920-Text.pdf.
2966216. [329] Union CotE, Parliament E. Directive (EU) 2016/1148 of the European parlia-
[310] Foti M, Greasidis D, Vavalis M. Viability analysis of a decentralized energy ment and of the council of 6 july 2016 concerning measures for a high common
market based on blockchain. In: International conference on the european level of security of network and information systems across the union. In:
energy market, EEM, Vol. 2018-June. IEEE; 2018, p. 1–5. http://dx.doi.org/ Official Journal of the European Union. 2016, URL https://op.europa.eu/en/
10.1109/EEM.2018.8469906. publication-detail/-/publication/d2912aca-4d75-11e6-89bd-01aa75ed71a1.
[311] Ferrag MA, Maglaras L. Deepcoin: A novel deep learning and blockchain- [330] European Parliament and Council of the European Union. Regulation on the
based energy exchange framework for smart grids. IEEE Trans Eng Manage protection of natural persons with regard to the processing of personal data
2020;67(4):1285–97. http://dx.doi.org/10.1109/TEM.2019.2922936. and on the free movement of such data, and repealing directive 95/46/EC (data
[312] Zhou S, Hu Z, Gu W, Jiang M, Zhang X-P. Artificial intelligence based smart protection directive). In: Official Journal of the European Union. L 119, 2016, p.
energy community management: A reinforcement learning approach. CSEE J 1–88, URL https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:
Power Energy Syst 2019;5(1):1–10. http://dx.doi.org/10.17775/cseejpes.2018. 02016R0679-20160504{&}from=EN.
00840. [331] Markopoulou D, Papakonstantinou V, de Hert P. The new EU cybersecurity
[313] Chen T, Bu S. Realistic peer-to-peer energy trading model for microgrids using framework: The NIS directive, ENISA’s role and the general data protection
deep reinforcement learning. In: Proceedings of 2019 IEEE PES innovative smart regulation. Comput Law Secur Rev 2019;35(6):105336. http://dx.doi.org/10.
grid technologies Europe. IEEE; 2019, http://dx.doi.org/10.1109/ISGTEurope. 1016/j.clsr.2019.06.007.
2019.8905731.

25