10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Results in Engineering
Volume 22, June 2024, 102132

Review article

A systematic review of big data innovations in smart grids

Hamed Taherdoost a b

Show more

Outline Share Cite

https://doi.org/10.1016/j.rineng.2024.102132
Get rights and content

Under a Creative Commons license open access

Highlights

• Data science transforms industries via smart grids, optimizing

electricity with big data analytics.

• Smart Grids manage energy, enhancing generation, distribution, and

consumption efficiency.

https://www.sciencedirect.com/science/article/pii/S2590123024003864 1/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

• Review of 25 papers - Smart Grid-Big Data integration for adaptive

energy, correlation analysis, blockchain, and machine learning.

• Emphasis on energy efficiency, security, sustainability, using rigorous

methodology in diverse critical domains.

Abstract
Multiple industries have been revolutionized by the incorporation of data science advancements into intelligent
environment technologies, specifically in the context of smart grids. Smart grids offer a dynamic and efficient framework for
the management and optimization of electricity generation, distribution, and consumption, thanks to developments in big
data analytics. This review delves into the integration of Smart Grid applications and Big Data analytics by reviewing 25
papers screened with PRISMA standard. The paper matter encompasses critical domains including adaptive energy
management, canonical correlation analysis, and novel methodologies including blockchain and machine learning. The paper
emphasizes contributions to energy efficiency, security, and sustainability by means of a rigorous methodology.

Previous Next

Keywords
Data science; Smart environment; Big data analytics; Energy management; Demand response

1. Introduction
Information and communication technologies (ICT) are ubiquitous in virtually every aspect of contemporary society [1]. As
the reliance of the healthcare system on technology increases, health science students are required to enhance their ICT

https://www.sciencedirect.com/science/article/pii/S2590123024003864 2/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

proficiency [2]. ICT facilitates the implementation of innovative learning materials and approaches, allowing for increased
student collaboration and the concurrent acquisition of technological expertise [3]. Notwithstanding the absence of a
universally accepted definition, ICT is generally understood to encompass all hardware, software, systems, and applications
that enable organizations and individuals to communicate in the digital realm [4]. The consequential increase in data
volumes, both within governmental organizations and private enterprises, has presented researchers with multifaceted
challenges. Key among these challenges is the need to develop effective solutions for the manipulation and analysis of large
datasets, as well as the establishment of mechanisms to seamlessly transmit these data from one site to another.

The interconnection of these technologies generates an automated ecosystem in which data is collected by the Internet of
Things (IoT) devices and subsequently processed and analyzed through the utilization of big data analytics and artificial
intelligence algorithms [5,6]. By transforming enormous data sets from diverse origins, such as the IoT, into a coherent
structure, big data analytics enables businesses to acquire valuable insights and formulate decisions based on data [7,8]. The
IoT, which is comprised of sensors and interconnected devices, is a crucial enabler of intelligent environments by enabling
the collection of real-time data that can be analyzed to detect trends and patterns [9,10]. In, particular, smart cities benefit
from the utilization of big data analytics to detect resource waste, optimize resource consumption, and enhance energy
management. As a result, residents enjoy greater efficiency, sustainability, and a higher standard of living [[11], [12], [13]].
The application of ICT in the energy sector transformed the current grid in the era of Industry 4.0 [14].

Several studies examine advancements in data science as they pertain to intelligent environment technologies. To automate
city systems, Sarker [15] emphasized the significance of extracting valuable knowledge from city-data. Ullah et al. [16]
examined the significance of data in driving innovation and development and the role of machine learning and the IoT in
realizing a data-centric smart environment. The article by Grossi et al. [17] discussed the capacity of data science to bring
about disruptive innovation in diverse sectors, such as energy and environment. It did so by facilitating the collection of
high-resolution data and augmenting the beneficial effects on the environment. Atitallah et al. [18] described how data
acquisition and processing utilizing a variety of technologies, including IoT and deep learning, can benefit a smart city. As a
whole, these papers offer valuable perspectives on the application of data science and associated technologies to foster
advancements in intelligent environment technologies, with a specific focus on smart cities and environmental
sustainability.

https://www.sciencedirect.com/science/article/pii/S2590123024003864 3/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

The examination of practical applications of data science in the establishment of environmentally sustainable smart cities,
the integration of big data, artificial intelligence, and IoT technologies to tackle climate change and sustainability issues, the
creation of data-driven models for automated city systems, and the investigation of data science's potential to support
unified approaches are all areas of research that require further investigation when undertaking a review of data science
innovations in smart environment technologies [15,17,[19], [20], [21]]. The aforementioned gaps underscore the criticality of
connecting progress in data science theory to practical implementations in smart environment technologies to tackle urgent
environmental and societal issues.

This article examines the most recent advancements in the application of big data to smart grids. It concludes with a
systematic literature review of big data analytics and its critical role in determining the trajectory of smart grid technologies
in the future. A comprehensive analysis of established research methodologies, significant discoveries, and discussions
comprise the systematic review, which provides an all-encompassing glimpse into the present state of knowledge in this
field.

2. Literature background

2.1. Data science in modern societies

The influence of data science on contemporary societies is substantial, as supported by an assortment of scholarly articles.
Data science is a composite field of study that has the potential to enhance scientific research, government administration,
business decision-making, and innovation in industry, science, and policy [17,22]. The importance of data science in
addressing societal issues such as climate change, urban planning, and healthcare is emphasized, as it enables the creation of
data-driven solutions to complex problems [15,21,23]. Data science is depicted as a transformative force capable of
introducing disruptive innovations across multiple spheres of society [17].

Data science is an all-encompassing and interdisciplinary paradigm that combines various models and theories to convert
data into knowledge (and value). In addition to validating established theories and models, experiments and analyses
conducted on enormous datasets facilitate the identification of patterns that emerge from the data. Such insights can assist
scientists in developing more accurate theories and models, ultimately leading to a more comprehensive comprehension of

https://www.sciencedirect.com/science/article/pii/S2590123024003864 4/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

the intricate nature of social, economic, biological, technological, cultural, and natural phenomena. When accessible data is
reinterpreted for analysis, the outcomes of data science are not in line with the initial motivations that drove data collection.
The aforementioned elements are collectively influencing a transformation in the scientific method, research, and societal
decision-making processes [24].

Data science is founded upon three confirming facts: the emergence of big data, which offers a substantial quantity of real-
world examples from which to learn; the progressions in data analysis and learning methodologies that enable the
derivation of predictive models and behavioral patterns from big data; and the developments in high-performance
computing infrastructures that facilitate the ingestion, management, and execution of intricate analyses on big data [25].
Through the use of deep learning and reinforcement learning techniques, artificial intelligence offers practical solutions for
optimal decision making on a global scale [26].

Data science is a dynamic and multidisciplinary discipline that emerges from the convergence of mathematics, statistics,
information theory, computer science, and social science (Fig. 1). By integrating these disciplines, a solid and mutually
beneficial framework is established to tackle the difficulties that arise from handling extensive and intricate datasets. Data
scientists develop exhaustive methodologies, models, and algorithms by capitalizing on the respective merits of computer
science, mathematics, statistics, information theory, and social science. The interdisciplinary character of data science not
only guarantees technical expertise but also takes into account the wider societal framework, rendering it a potent
instrument for furthering knowledge and catalyzing constructive transformations in various fields.

https://www.sciencedirect.com/science/article/pii/S2590123024003864 5/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Download: Download high-res image (378KB)

Download: Download full-size image

Fig. 1. Disciplines in data science.

2.2. Domains within data science

Data Science is an interdisciplinary domain that incorporates methodologies and concepts from numerous other fields,
thereby enhancing its holistic approach to data management and insight extraction [27,28]. Domain knowledge and
statistics, computer science, and programming are instrumental in the development and implementation of techniques for
efficient data analysis. Data scientists analyze data sets utilizing an array of statistical and analytic methods, such as
classification, regression, anomaly detection, and others [29]. Data analysts have access to a variety of data analysis
techniques, such as sentiment analysis, cohort analysis, cluster analysis, and time series analysis [[30], [31], [32], [33]], as the
discipline of data science expands at an accelerated rate.

The systematic extraction of knowledge from data, also known as data science, has garnered considerable interest in recent
times [34]. As one might expect, data science is at the forefront of a paradigm shift in science [35]. In numerous fields of
study, its epistemological assumptions, challenges, and opportunities have been examined [36,37]. Concerns remain,

https://www.sciencedirect.com/science/article/pii/S2590123024003864 6/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

however, as to whether this represents the resurgence of empiricism a genuine fourth paradigm of science [38], or merely an
expansion of established paradigms accompanied by novel instruments and approaches to scientific investigation [39].

Approaching is the fourth industrial revolution. At present, a considerable number of enterprises are adopting Industry 4.0,
capitalizing on nascent developments in automation, the industrial IoT, big data, and cloud computing [40]. The emergence
of the industrial IoT has facilitated the real-time collection of an unprecedented volume of data by sensors integrated into
networked physical devices. This data empowers manufacturing operations, processes, and systems to attain notable
improvements in productivity, efficiency, and self-management [41,42]. Proficiency in machine learning, artificial
intelligence, and data analytics techniques is essential in the manufacturing sector [[43], [44], [45]]. ρReveal, a novel Big Data
Analytics (BDA) scheme for predicting energy prices in Smart Grids, was introduced by Kumari and Tanwar [46]. It makes use
of Spark-based analytics for load reduction techniques and an artificial intelligence-based Bidirectional Long Short-Term
Memory (BiLSTM) model for precise price forecasting. In terms of data security and prediction accuracy (RMSE, MAE, and
MAPE), Reveal performed better than previous methods.

2.3. Smart grid management and operation

Demand-side management techniques, such as demand-response control in smart grids, specify how the demand side
responds to price strategies or incentive actions from a power unit [47,48]. Supply-demand balance can be promoted and
cost-effective, high-quality, customized services can be provided to demand-side consumers with the help of demand-
response management [49].

By analyzing historical and real-time consumption patterns, utilities can improve their demand response strategies to meet
consumer and grid needs. Better load forecasting, demand-side management, and consumer participation can result. The
emergency demand response (EDR) pilot in southwestern China during a heatwave was successful. The study found that
incentive-based EDR policies like time-of-use (TOU) pricing can reduce peak loads and demand significantly. The study also
found that smart thermostats and home automation systems improve price-based demand responses [50].

To enable proactive maintenance and minimize downtime, predictive maintenance employs data and analytics to forecast
when a component in an actual system is likely to fail [51]. In the context of Industry 4.0, where technical system complexity

https://www.sciencedirect.com/science/article/pii/S2590123024003864 7/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

is constantly increasing, this approach is especially helpful [52]. It is especially helpful in anticipating unplanned
deterioration, which enables operators to take preventative measures and stop malfunctions before they happen [53].

The large amount of data generated by smart grids will help utilities understand customer conservation, consumption, and
demand, track downtime, and power failures. This will be difficult for utilities without the systems and data analysis skills to
handle these data. Therefore, utilities now aim to manage high-volume data and use advanced analytics to turn data into
knowledge and actionable plans [54].

The capacity of big data analytics to absorb, process, and analyze massive amounts of data in almost real-time is one of its
main advantages for smart grids. Real-time data collection is made possible by Supervisory Control and Data Acquisition
(SCADA) technologies, which also allow for remote grid performance monitoring, control, and analysis to increase
operational dependability and efficiency. Automation of the grid depends on SCADA and Distribution Management Systems
(DMS), which integrate sophisticated algorithms and analytics to manage grid assets, identify faults, and optimize energy
flow for stable and balanced power distribution. Energy Management Systems (EMS) are crucial to grid automation because
they facilitate load management and grid balancing, regulate and optimize energy resources, and monitor energy generation,
consumption, and storage. With EMS technologies, grid operators can optimize energy use, integrate renewable energy
sources, and improve stability for a dependable power supply [55,56].

Distributed Energy Resources, a product of technological advancement, has raised consumer involvement in energy
generation and management within the Smart Grid system [57]. There are additional difficulties in integrating big data
analytics with smart grids, such as privacy and cyber security concerns. Ensuring the security and privacy of this data is
becoming more and more crucial as the volume of data produced by smart grids increases [58]. End users can benefit from a
multitude of services provided by a smart grid system, including energy trading (ET), load forecasting, and load
management. Since data in an SG environment moves between various devices via an open channel—the Internet—security
and privacy are perpetually difficult problems. Despite the fact that there are numerous solutions to this issue in the
literature, they are insufficient to address security, privacy, latency, or real-time ET settlement [59]. For peer-to-peer trading
in smart grid systems, Kumari et al. [59] proposed ET-DeaL, a Secure Energy Trading scheme based on Smart Contracts. ET-
DeaL surpasses conventional systems in performance metrics by utilizing IPFS and Ethereum smart contracts to manage

https://www.sciencedirect.com/science/article/pii/S2590123024003864 8/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

energy loads efficiently in the residential, industrial, and electric vehicle sectors while addressing security, privacy, and
latency concerns.

3. Theoretical concepts
The utilization of High-performance computing has become prevalent across numerous scientific and practical domains.
High-performance computing techniques comprise technologies and methods used to accelerate the execution of complex
computational processes. Hardware accelerators, parallel processing, cloud computing, distributed computing, multi-core
processing, and cluster computing are all prevalent high-performance computing techniques. These types of systems
frequently comprise numerous computing nodes and are frequently linked via a high-bandwidth network. By effectively
managing vast quantities of data, decomposing complex problems into smaller components, and concurrently resolving each
component, high-performance computing systems achieve overall processing times that are considerably quicker. Extremely
complex problems in the domains of aerospace [60], Fintech [61], chemistry [62], biology [63], physics [64], and others could
be resolved with the assistance of high-performance computing techniques. The integration of modeling, algorithm
development, software construction, and computational simulation through the application of high-performance computing
has emerged as an indispensable foundation in the realm of cutting-edge fundamental science research.

The use of high performance computing resources has been shown to lead to shorter runtimes across all categories,
demonstrating its consistent trend in enhancing computational efficiency [65]. Furthermore, high performance computing is
described as having computing power that meets the needs of an intelligent society and serves as infrastructure like water
and electricity [66]. These findings underscore the significance of advanced algorithms customized to the unique attributes
of smaller, intricately structured datasets, which enable instantaneous processing and enhance the nuanced nature of
problem-solving.

The capability of high-performance computing to process lesser datasets in real time is a critical attribute for applications
including scientific simulations, engineering scenarios, and financial modeling. The algorithms have been meticulously
optimized to capitalize on the intrinsic structure of the data, delivering results and insights virtually in real time. Due to its
remarkable flexibility, high-performance computing is an essential resource in both scientific research and industrial
applications that require precise curation of datasets. High-performance computing is a preferred instrument in material

https://www.sciencedirect.com/science/article/pii/S2590123024003864 9/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

science [67], drug discovery [68], and computational fluid dynamics [69], demonstrating its adaptability and effectiveness
when confronted with smaller, highly structured datasets. This can be observed in various studies and applications, such as
designing high-performance computing and big data converged systems, also referred to as High-Performance Data
Analytics (HPDA), which is a hard task requiring careful placement of data [70].

One notable attribute of high-performance computing is its heavy reliance on substantial computation. Standard computing
platforms may find it impracticable or overly time-consuming to process large datasets, implement complex mathematical
models, and conduct simulations. For these tasks, high-performance computing architectures are meticulously designed. The
prioritization of substantial computation holds particular significance in domains including financial modeling, molecular
dynamics simulations, weather prediction, and other applications that demand substantial computational resources.
Efficient and precise computations are necessary in these domains to extract significant insights, facilitate well-informed
decision-making, and simulate real-world phenomena precisely [71].

Within the domain of data science and the wider realm of technological advancement, the interdependence of data
processing and high-performance computation becomes conspicuous. High-performance computing enables data scientists
and researchers to extract valuable insights from extensive datasets, thereby promoting progress in diverse domains such as
engineering simulations, scientific inquiry, and machine learning. The necessity for computational capacity to manipulate,
analyze, and derive meaningful patterns from these enormous datasets becomes ever more critical as their size and
complexity continue to increase [72].

3.1. Emergence of data science

The advent of Data Science has brought about substantial changes in the way we confront the difficulties presented by the
growing dependence on ICT. Conventional High-Performance Computing techniques, which were previously indispensable
for smaller, structured datasets, encountered constraints when confronted with prodigious, unstructured data. The article by
Barakat et al. [73] highlighted the significance of Data Science in tackling the difficulties associated with conditions such as
ARDS. This article examined a range of Data Science methodologies, such as mechanistic modeling, deep learning, and time
series analysis, with a particular emphasis on the integration of High-Performance Computing to facilitate efficient
algorithmic support.

https://www.sciencedirect.com/science/article/pii/S2590123024003864 10/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

The paper by Mellone et al. [74] presented a paradigm shift in environmental research by addressing the changing demands
of the field through the implementation of High-Performance Cloud-Native Computing. This methodology took advantage of
scalable high-performance computation in the cloud, showcasing its advantages in terms of resource conservation and
enhanced performance. An additional article by Oujja et al. [75] emphasized the critical nature of ICT instruments utilized to
analyze genomic data, with a specific emphasis on SARS-CoV-2. It introduced a data science-driven high-performance
computing-based instrument, emphasizing the importance of computational capability in the context of RNA clustering for
virus mutation analysis.

The integration of data science and high-performance computing is apparent in numerous fields, including healthcare, as
elaborated in the article by Courneya and Mayo [76]. By bridging the distance between computational infrastructure and wet
lab setups, this article offers bioinformatics and high-performance computing support to researchers. Moreover, Belov et al.
[77] demonstrated this convergence also benefits the educational sector. They examined the characteristics and attributes of
high-performance computing platforms that are employed in educational procedures, with a particular focus on the
significance of having access to suitable tools and resources for data science and parallel programming instruction.

A fundamental attribute of Data Science is its emphasis on extracting insights and knowledge from enormous datasets,
thereby revealing patterns, trends, and valuable information that may remain concealed when utilizing conventional
analytical methods. This paradigmatic change has not only resulted in the redefinition of data management methodologies,
but has also fostered the development of innovative theories and techniques that contribute to the progress of numerous
scientific, industrial, and governmental sectors. The papers that discuss the domain-spanning collaborative impact of high-
performance computing and data science are listed in Table 1.

Table 1. Impact of data science on high-performance computing.

Study Collaboration of Healthcare Environmental Bioinformatics Genomic Educational Quantum

two concepts Applications Modeling and -omics Data Processes Computing
Analysis Integration

[78] ✓

https://www.sciencedirect.com/science/article/pii/S2590123024003864 11/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Study Collaboration of Healthcare Environmental Bioinformatics Genomic Educational Quantum

two concepts Applications Modeling and -omics Data Processes Computing
Analysis Integration

[73] ✓ ✓ ✓

[74] ✓ ✓

[76] ✓ ✓ ✓

[75] ✓ ✓

[79] ✓ ✓

[80] ✓

[77] ✓ ✓

[81] ✓ ✓

The management of unstructured datasets is a critical aspect of Data Science, presenting both a substantial challenge and an
opportunity in the current data landscape. Unstructured data encompasses diverse formats, including sensor-generated
information, text, images, and videos. To navigate unstructured datasets and extract meaningful information, Data Science
employs a range of sophisticated techniques such as machine learning algorithms, natural language processing, computer
vision, and deep learning. These advanced methods enable Data Science to uncover valuable insights from unstructured data,
turning what might seem like disorder into a source of hidden patterns and knowledge [31,82,83].

Furthermore, the ability of Data Science frameworks to scale facilitates the effective manipulation of enormous datasets.
Through the utilization of Big Data technologies, distributed computing, and parallel processing, Data Scientists are
empowered to manage and analyze enormous volumes of data that surpass the capabilities of conventional computing
systems. Scalability is an essential factor in effectively managing the complexities presented by the exponential increase in
data produced by contemporary technologies such as the IoT and interconnected smart environments. Table 2 presents an

https://www.sciencedirect.com/science/article/pii/S2590123024003864 12/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

extensive assortment of methodologies and applications, thereby offering valuable insights into the intricate convergence of
data science, big data, and intelligent environments.

Table 2. Exploration of data science and big data applications in smart environments.

Study Technologies/Frameworks Application Evaluation

[84] IoT, Intrusion Detection, Security Protocol Real-time security system, Communication Security analyses, Efficiency
protocol evaluation

[85] IoT, AllJoyn, MongoDB, Storm Monitoring, Management, Big Data analytics Experimental results, Performance
evaluation

[86] Blockchain, Decentralized Auditing Ensuring integrity and auditability of big data Theoretical analysis, Experimental
evaluation

[87] VRGIS, TIN Data Model Intelligent tourism service system, VRGIS Experiment results, User experience
applications

[88] Spark, Hadoop Real-time processing, Smart transportation Throughput, Validation with authentic
planning data

[89] Kalman Filter, Weighted Hybrid Enhanced living environment, User behavior Recall and precision rates
Recommender System prediction

[90] MapReduce, Partial Order Reduction Attribute reduction for power systems, Performance observed through a
Simulation examples Hadoop platform

[91] Fuzzy Logic, Multi-Fuzzy Agent-based WSN Noise reduction, Smart data extraction Simulation results

[92] Blockchain, PKI/CA Security System Decentralized trust service system, Smart City Evaluation model, Vertical
development comparison

https://www.sciencedirect.com/science/article/pii/S2590123024003864 13/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

3.2. Processes and approaches for knowledge extraction

Knowledge extraction is the process of extracting relevant and meaningful information from unstructured or structured data
sources, such as text, documents, images, and relational databases [93,94]. The process includes identifying patterns and
relationships, transforming them into actionable knowledge [95]. Techniques like named entity recognition [96], text mining
[97], natural language processing [97], information retrieval [98], and machine learning [99], including deep learning [100],
rule-based systems [101], decision trees [102], and neural networks [103], contribute to this extraction.

Knowledge extraction has various applications, including sentiment analysis, text summarization, and information retrieval
[104]. The resulting knowledge needs to be in a machine-readable and machine-interpretable format and needs to be able to
be queried. Fig. 2 depicts a closed-loop knowledge extraction process with implementation tips, covering data collection to
actionable insights.

Download: Download high-res image (224KB)

Download: Download full-size image

Fig. 2. Iterative knowledge extraction loop: implementation tips.

4. Methodology for systematic review

https://www.sciencedirect.com/science/article/pii/S2590123024003864 14/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

The search strategy involves employing a comprehensive set of keywords and search terms, including terms related to Big
Data, Smart Grids, and associated technologies, such as "Big Data" AND "Smart Grid". The search was conducted across
Scopus and Web of Science database. This strategy aims to capture studies focusing on the integration of Big Data in Smart
Grid technologies.

• Research Questions (RQs) for the Review:

RQ1How do different big data analytics approaches contribute to the optimization and efficiency of smart grid technologies?

RQ2What are the challenges and solutions related to data security and privacy in the integration of big data analytics in
smart grids?

RQ3How do machine learning and artificial intelligence techniques enhance decision-making processes in smart grid
applications?

RQ4What role does blockchain technology play in ensuring secure and transparent transactions within the context of energy
smart grids?

4.1. Inclusion and exclusion criteria

The criteria for selecting relevant studies include a publication timeframe between 2019 and 2023 to ensure currency, with a
specific focus on the impact of Big Data in Smart Grid technologies. Only English articles will be considered for inclusion.
Exclusion criteria encompass reviews, book chapters, conference papers, and books, aiming to prioritize primary research
articles that directly contribute to the understanding of the integration, challenges, and outcomes of Big Data in the context
of Smart Grids.

4.2. Data extraction

The data extraction process will be systematic and transparent, employing a standardized data extraction form. Key variables
to be extracted include study design, methodologies utilized, and key findings. Additionally, variables relevant to Smart Grid

https://www.sciencedirect.com/science/article/pii/S2590123024003864 15/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

technologies and their interaction with Big Data will be identified. The extraction form will be designed to capture essential
information from each included study, ensuring consistency in the retrieval of data.

4.3. Data synthesis

The synthesis of data involved a systematic collation of findings from the included studies. A structured approach will be
employed to present key themes and trends emerging from the literature. The data synthesis will encompass a narrative
summary of study designs, methodologies employed, and key outcomes, with a specific focus on the interaction between Big
Data and Smart Grid technologies.

4.4. Data selection

Following the initial search on December 28, 2023, 444 articles were identified. After applying inclusion and exclusion
criteria, 32 papers were retained. Following a detailed review of abstracts and scopes, 24 papers remained. An additional
paper was found through further search, resulting in a final selection of 25 papers for the systematic review (Fig. 3).

https://www.sciencedirect.com/science/article/pii/S2590123024003864 16/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Download: Download high-res image (473KB)

Download: Download full-size image

Fig. 3. PRISMA literature search methodology flowchart.

5. Results
https://www.sciencedirect.com/science/article/pii/S2590123024003864 17/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

A total of 24 studies related to various aspects of smart grids, big data analytics, and related technologies were identified. The
studies cover a range of topics, including canonical correlation analysis, adaptive energy management, post-evaluation
systems, big data collection and utilization in smart factories, integration of big data and blockchain, demand response
management, electricity theft detection, energy consumption prediction, big data compression, outlier rejection for load
forecasting, fog computing, differentially private clustering, non-technical loss fraud detection, evaluation of big data
frameworks, anonymous batch verification, optimal big data processing, temporal-functional-spatial big data computing,
and robust big data analytics for electricity price forecasting.

The keyword cloud from our selected articles as shown in Fig. 4, indicates a strong focus on integrating Big Data analytics
into Smart Grid technologies. Terms like "Big Data," "Smart Grid," and "Smart Power Grids" underscore the core areas of
research. Additionally, there is a growing interest in leveraging "Data Analytics," "Data Mining," and "Internet of Things (IoT)"
for extracting insights from Smart Grid data. The keywords "Fog Computing," "Green Computing," and "Energy Efficiency"
suggest a trend toward exploring innovative computing paradigms and sustainable practices. The recurring focus on
"Electricity Theft Detection," "Data Privacy," and "Security and Privacy" highlights the rising importance of securing Smart
Grids and ensuring data confidentiality.

https://www.sciencedirect.com/science/article/pii/S2590123024003864 18/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Download: Download high-res image (445KB)

Download: Download full-size image

Fig. 4. Cloud of selected article keywords (created by www.wordclouds.com .

The distribution of subject areas among the 25 papers reflects a predominant focus on Computer Science, comprising 76 % of
the research output (Fig. 5). This emphasis underscores the significance of computational methods, algorithms, and data
analytics in the exploration of smart grids and related technologies. Engineering follows closely as the second most
represented subject area, constituting 48 % of the papers, indicating a strong applied orientation with a focus on the
development and implementation of technologies within the smart grid domain. The inclusion of subject areas such as
Decision Sciences, Energy, Mathematics, and Materials Science reveals a multidisciplinary approach, acknowledging the
diverse challenges inherent in the study of smart grids. The presence of papers in Energy suggests a concentrated
consideration of energy-related aspects, while the representation of Decision Sciences and Mathematics implies a

https://www.sciencedirect.com/science/article/pii/S2590123024003864 19/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

quantitative and analytical approach to problem-solving. Although less prevalent, the inclusion of Materials Science
underscores the importance of materials in the context of smart grids, possibly pertaining to advancements in energy
storage or grid infrastructure.

Download: Download high-res image (197KB)

Download: Download full-size image

Fig. 5. Subject area distribution of selected papers.

Table 3 serves as a quick reference for understanding the methodologies, datasets, and outcomes of diverse research efforts
in the field of Smart Grids and Big Data analytics.

Table 3. Methods, datasets, and metrics of big data insights in smart grids.

Study Methods Dataset Size Analysis Techniques Key Findings Performance

Metrics

[105] Canonical Correlation 3 datasets (24 values/day, Canonical correlation, Overview, Typical patterns, Not specified
Analysis, Consumer 1-year period) Consumer Comparison on climate

https://www.sciencedirect.com/science/article/pii/S2590123024003864 20/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Study Methods Dataset Size Analysis Techniques Key Findings Performance

Metrics

Segmentation, Preprocessing segmentation, zones and locations

Comparison analysis

[106] Linear Regression, Logistic 50,000 instances (smart Comparison of linear Linear regression: 98 %, Accuracy
Regression, K-Nearest meters), 10,000 attributes regression, logistic Logistic regression: 96 %, K- percentage
Neighbors regression, K-Nearest Nearest Neighbors: 92 %
Neighbors

[107] Microservice Framework, Big Not specified Evaluation of wind Tested and proven for Not specified
Data Analysis farms, Technical processing and analyzing
support, Visualization massive data

[108] Apache Kafka, Big Data Data from geographically Data gathering, Data Stable and effective Not specified
Ecosystem remote, independent convergence, Data exchange and collection of
networks analysis data using Apache Kafka

[109] TOTEM Framework (Token for Not specified Data analysis without TOTEM framework for Not specified
Controlled Computation), moving data, Data analyzing data without
Blockchain, Machine Learning security, Privacy of moving it, Ensuring data
prosumers security and privacy

[110] Prophet Model, ARIMA Two datasets Demand response Effectiveness of Prophet Various
management, model in demand response evaluation
Forecasting management metrics

[111] CNN with RUSBoost MRFO Not specified Electric power theft Accuracy of proposed Accuracy
and RUSBoost BSA models detection approaches: RUS-MRFO: percentages
91.5 %, RUS-BSA: 93.5 %

https://www.sciencedirect.com/science/article/pii/S2590123024003864 21/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Study Methods Dataset Size Analysis Techniques Key Findings Performance

Metrics

[112] GA-LSTM (Genetic Algorithm Pennsylvania-New Jersey- Genetic Algorithm, Long Better performance Various
- Long Short Term Memory) Maryland Interconnection Short Term Memory compared to existing performance
(PJM) energy consumption benchmarks evaluation
data metrics

[113] Optimal Singular Value Wide range of data types Data compression, Compression level Compression
Decomposition (SVD), Retrieval quality, dominates existing SVD level
Intelligent Optimization Compression ratio rank reduction methods
Methods

[114] Deep Siamese Network (DSN), Real-time smart meters' Imbalanced class Effective in resolving AUC, F1-Score,
Adaptive Synthesis (ADASYN) data problem, Feature imbalanced class problem Precision, Recall
extraction

[115] IoT, Cloud Computing IMWSNs measurements Channel detection, Useful for designing Not specified
during events monitoring Channel assignment, algorithms for real-time
and control Packets forwarding events monitoring and
control

[116] Fuzzy Logic, Genetic Not specified Fuzzy logic, Genetic Optimal values for Not specified
Algorithm algorithm maximizing profit and
predicting future power
demands

[117] Fog Computing, Cloud Exemplary SG network Planning and Placement Optimization of FCN Response delay,
Computing of Fog computing in location, capacity, and Energy
smart Grid (PPFG) number consumption

https://www.sciencedirect.com/science/article/pii/S2590123024003864 22/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Study Methods Dataset Size Analysis Techniques Key Findings Performance

Metrics

[118] Differentially Private Not specified Privacy-preserving Privacy-preserving, Security analysis,

Clustering, Infinite Gaussian cluster analysis Efficient Performance
Mixture Model (IGMM) evaluation

[119] Hybrid Outlier Rejection Not specified Outlier rejection for Outperforms recent Accuracy,
Methodology (HORM) load forecasting methods in terms of Precision, Recall,
accuracy, precision, recall, F1-measure
F1-measure

[120] Edge Computing, Big Data Not specified Non-Technical Loss Efficient detection of non- Detection speed
Analytics fraud detection technical loss frauds

[121] Hadoop-Hbase, Cassandra, Large scale smart grid data Data analysis Performance benchmark Not specified
Elasticsearch, MongoDB generator techniques for different frameworks

[122] Big Data Anonymous Batch Not specified Batch-verifiable Efficient detection of Not specified
Verification, Edge Computing, authentication, Privacy power injection without
Certificateless Aggregate preservation exposing private
Signature (CL-AS) information

[123] Hybrid MDM/R Architecture, Not specified Scheduling of Modification of supply- Not specified
MapReduce, Massive Parallel workloads, Energy following algorithm, New
Processing Database consumption, RE hybrid architecture
integration

[124] Temporal, Functional, Spatial Large-scale smart grid Data extraction, Promising computing Convergence
Big Data Computing Computing efficiency efficiency, Bandwidth ratio,
savings Improvement
ratio

https://www.sciencedirect.com/science/article/pii/S2590123024003864 23/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Study Methods Dataset Size Analysis Techniques Key Findings Performance

Metrics

[125] Fiber-Wireless (FiWi) Not specified Data acquisition under Constrained optimization Not specified
Enhanced Smart Grid failures problem, OERA, GARA,
HGRA

[126] Cloud Computing, Grid Not specified Information Business intelligence Not specified
Computing management, Data architecture, Multivariable,
storage Multi-dimensional analysis

[127] Fog Computing, Cloud Not specified Electrical load 3-tiers architecture, Data Precision, Recall,
Computing forecasting pre-processing, Load Accuracy, F-
prediction measure

[128] Random Forest, Relief-F, Grey Not specified Feature selection, Hybrid feature selector, Price forecasting
Correlation Analysis, Kernel Feature extraction, Price Dimensionality reduction, Performance
PCA, Differential Evolution, forecasting DE-SVM classifier
Support Vector Machine

[129] Cloud Computing, Power Big Not specified Power efficiency Business intelligence Not specified
Data analysis architecture, Multivariable,
Multi-dimensional analysis

6. 6. discussion
RQ1
How do different big data analytics approaches contribute to the optimization and efficiency of smart grid technologies?

The effectiveness and productivity of smart grid technologies are notably impacted by a multitude of big data analytics
methodologies, as supported by an extensive research body. Canonical correlation analysis is employed by scholars including
https://www.sciencedirect.com/science/article/pii/S2590123024003864 24/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Jiang et al. [105] to investigate the interrelationships between gas consumption, electricity consumption, and climate change.
This method yielded significant insights into patterns of consumption. Comparing data processing methods, Gupta and
Chaturvedi [106] examined Adaptive Energy Management in Smart Grids, placing particular emphasis on the dependable
linear regression method, which achieves an impressive accuracy rate of 98 %. The integration of these methodologies aids in
the comprehension of consumer behaviors and the resolution of energy management challenges.

By virtue of integrating big data analysis and microservice frameworks, Wang et al. [128] presented a post-evaluation
platform for intelligent grid electricity generation in wind farms. This platform exemplifies the pragmatic implementation of
big data in enhancing the efficacy of energy generation by assessing the operational status of wind farms and providing
technical assistance. Furthermore, the TOTEM framework, which Jose et al. [109] put forth, merges blockchain technology
with big data to facilitate the management of energy smart grids. This framework enhances data security and privacy while
providing a reliable platform for energy transactions. The aforementioned studies underscore the significance of big data
analytics in the context of secure energy management and post-evaluation systems.

Akram et al. [111] directed their attention towards novel methodologies in the realm of electricity theft detection. They
demonstrated the efficacy of enhanced RUSBoost classifiers, which achieve accuracy rates of 91.5 % and 93.5 %, respectively. In
the interim, Kumari et al. [110] presented a demand response management strategy that makes use of the Prophet model to
illustrate the efficacy of energy consumption forecasting in smart grids. The aforementioned studies underscore the
significance of big data analytics in augmenting security protocols and precisely forecasting energy requirements. An optimal
singular value decomposition (SVD)-based large data compression approach for smart grids is proposed by Hashemipour et
al. [113], which showcases effective compression levels. Intelligent optimization determines optimal values, improving data
quality and compression ratio. Comparative analysis shows superior compression levels to existing SVD rank reduction
methods, emphasizing the need for application-specific optimization for reliable performance.

The studies offer quantifiable metrics that can be utilized to assess the efficacy of big data analytics methodologies.
Quantitative measures such as accuracy percentages, area under the curve (AUC), F1-score, precision, and recall are
employed to evaluate the dependability and efficacy of these methodologies. For example, Gupta and Chaturvedi [106]
provide accuracy percentages for different data processing methods, whereas Javaid et al. [114] evaluate the performance of
their deep siamese network using AUC, F1-Score, precision, and recall. They presented an adaptive synthesis approach for

https://www.sciencedirect.com/science/article/pii/S2590123024003864 25/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

electricity theft detection utilizing an imbalanced big data set and a deep siamese network. The utilization of these
quantitative metrics provides a strong basis for analyzing the effects of big data analytics on the optimization of smart grid
technologies, with a particular focus on the accuracy and dependability attained across diverse implementations. The
accuracy rates of various big data analytics models for a variety of smart grid applications are compared in Table 4.

Table 4. Accuracy comparison of big data analytics in smart grids.

Study Methodology Accuracy

[111] Novel CNN with RUSBoost MRFO and RUSBoost BSA models. Accuracy: rus-MRFO 91.5 %, rus- SVM: 71 %, LR: 63 %, CNN: 85.1 %, rus-
BSA 93.5 %. SVM accuracy 71 %, LR 63 %. Smote algorithm for balancing. Boosting techniques MRFO: 90 %, rus-BSA: 93 %
outperform.

[114] ADASYN handles imbalance. CNN-LSTM integrated deep siamese network. High AUC, mAP, SDN: AUC 0.93 %, mAP 0.9 %, Accuracy
precision, recall, MaP, accuracy, and F1-Score. Maintains performance for different training (60 %: 0.839, 70 %: 0.844, 80 %: 0.953)
ratios. Simulation results validate effectiveness.

[119] ELF strategy with DP2 and LP2 phases. Proposed HORM for outlier rejection, outperforms 'All Data': 87 %, HORM: 95 %
recent methods. Experimental results show higher accuracy, precision, recall, and F1-
measure.

[127] ELF strategy with DP2 and LP2 phases. FBFS for feature selection. Experimental results show FSSO2: 0.80, FSSO4: 0.93
improved efficiency in terms of precision, recall, accuracy, and F-measure.

[106] Comparison of linear regression, logistic regression, and K-Nearest Neighbors. Linear Linear Regression: 98 %, Logistic
regression gives highest accuracy (98 %). Regression: 96 %, K-NN: 92 %

[112] GA-LSTM method based on Genetic Algorithm. GA-LSTM outperforms existing benchmarks. GA: 82.42 (daily), 80.27 (weekly),
Multi-threaded environment for increased convergence speed. Results validated on PJM Random Approach: 51.26 (daily),
energy consumption data. 48.22 (weekly)

[124] Novel framework for large-scale smart grid. Functional and spatial dimensions considered. Computing Efficiency: 95 %
Promising computing efficiency (95 % convergence ratio) and bandwidth savings (81 % convergence ratio, Bandwidth: 81 %

https://www.sciencedirect.com/science/article/pii/S2590123024003864 26/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

Study Methodology Accuracy

improvement ratio over benchmarks). improvement ratio

[128] Hybrid feature selector (RF + Relief-F + GCA), KPCA for dimensionality reduction, DE-based HSEC: >90 %, Frameworks A, B, C, and
SVM classifier. Superior performance compared to other methods. HSEC show improvement in accuracy

Boosting methods such as RUSBoost BSA and RUSBoost MRFO exhibit superior performance compared to SVM (71 %) and LR
(63 %). CNN achieves an accuracy of 85.1 %, while boosted models can attain as high as 93 %. When combined with a CNN-
LSTM deep siamese network, the ADASYN method achieves an accuracy of 95.3 %–83.9 % across a variety of training ratios.
When ELF strategies and HORM are combined for outlier rejection, accuracy increases significantly from 87 % to 95 %.
Moreover, feature selection techniques such as FBFS and FSSO4 contribute to improvements in precision, with the latter
attaining 93 %. Linear regression demonstrates the highest degree of accuracy (98 %), surpassing logistic regression (96 %) and
K-NN (92 %), among conventional methods. The GA-LSTM approach, which is founded on the Genetic Algorithm, attains a
moderate level of accuracy, specifically 80.27 %–82.42 %, which is notably higher than that of random methods. In addition,
the accuracy of hybrid feature selectors utilizing DE-based SVM classifiers surpasses 90 %. Although not directly associated
with accuracy, innovative frameworks present encouraging improvements in efficiency, including a 95 % convergence ratio
and 81 % bandwidth enhancement.

The progress made in accuracy has a direct influence on the efficiency of smart grid operations, guaranteeing enhanced
dependability in the areas of load forecasting, energy management, and electricity theft detection. Proving the capability of
robust big data analytics to optimize smart grid performance, the incorporation of sophisticated techniques such as hybrid
feature selectors and Genetic Algorithm-based models further improves precision. Innovative computing frameworks
specifically engineered for expansive smart grids not only enhance computational efficacy but also facilitate bandwidth
conservation, thereby emphasizing the far-reaching consequences of precision advancements on the overall performance
and dependability of the system.
RQ2
What are the challenges and solutions related to data security and privacy in the integration of big data analytics in smart
grids?

https://www.sciencedirect.com/science/article/pii/S2590123024003864 27/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

The incorporation of big data analytics into smart infrastructure presents a range of complex obstacles, with data security
and privacy being of particular significance. Significantly, the research conducted by Rabie et al. [127] and Nivedha et al.
[126] emphasized the need for a comprehensive data privacy framework, recognizing the potential risks that ongoing data
collection via smart meters and sensors may pose to sensitive consumer information. Rabie et al. [127] highlighted the
importance of implementing a fog-based load forecasting approach by proposing a three-tier structure that improves the
acquisition, processing, and retention of smart metre data prior to its transmission to the cloud. Due to the sensitive nature
of the data involved, this strategy requires meticulous deliberation regarding privacy issues. This is consistent with the wider
difficulties presented by data privacy within the smart grid domain.

In light of the obstacles encountered, Guan et al. [118] proposed that differential privacy techniques be integrated into cluster
analysis as a means of safeguarding the confidentiality of smart grid data. The algorithm they suggest, IDPC, integrates
differential privacy with nonparametric Bayesian techniques, providing a solution that takes into account the dynamic
characteristics of clustering while maintaining privacy. Furthermore, the research conducted by Han and Xiao [120]
presented the notion of edge computing-enabled non-technical loss fraud detection, underscoring the criticality of
safeguarding big data analytics at the periphery in order to avert fraudulent activities. These proposed solutions are in line
with the wider requirement for secure edge computing infrastructure and privacy-preserving analytics. They underscore the
importance of sophisticated frameworks and techniques in mitigating security issues related to large-scale data in smart
grids. It is critical to confront encryption obstacles, as demonstrated by research such as the one proposed by Kamil and
Ogundoyin [122] for an anonymous bulk verification scheme for big data. By integrating highly efficient certificateless
aggregate signature algorithms, their methodology guarantees the authentication of power offers transmitted through
vehicular networks and 5G smart grid slices in a secure manner.
RQ3
How do machine learning and artificial intelligence techniques enhance decision-making processes in smart grid
applications?

Machine Learning and Artificial Intelligence are instrumental in transforming the decision-making process. The challenges
presented by the expanding intricacy of data produced by smart grids are motivating the growing dependence on these
technologies. This is evident in research endeavors like Jiang et al.'s [105] investigation into the application of Canonical
Correlation Analysis in comprehending electricity consumption patterns. Integration of machine learning and artificial
https://www.sciencedirect.com/science/article/pii/S2590123024003864 28/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

intelligence techniques was motivated by the need to resolve issues pertaining to the manipulation and analysis of large
datasets, the transmission of data efficiently, and the utilization of diverse data sources effectively.

Load forecasting has exhibited potential for machine learning techniques, specifically supervised learning, as Gupta and
Chaturvedi [106] illustrated in their study on Adaptive Energy Management. The study by Park and Huh [108] on Apache
Kafka serves as an illustration of how unsupervised learning enhances the efficiency of data collection and utilization in
distributed manufacturing networks. Centralized systems, such as ESB or EAI, which are used for data collection in
manufacturing traditionally, have problems integrating data from different networks in factories. Data collection across
distributed or autonomous networks is inefficient and vulnerable to security breaches due to this centralized approach. The
goal of implementing Kafka was to facilitate user analysis and utilization of data across various manufacturing networks. In
addition, research by Kumari et al.'s [110] approach to Demand Response Management emphasized how Reinforcement
Learning optimizes decision-making processes through the acquisition of knowledge from interactions in dynamic
environments.

Artificial intelligence techniques, including Knowledge-Based Systems for grid optimization and Expert Systems for defect
detection, improve decision-making. The incorporation of rule-based methodologies and machine learning, as demonstrated
in the research conducted by Jose et al. [109] concerning the integration of blockchain and big data, enhances the security
and reliability of energy management. The insights provided by these artificial intelligence techniques are crucial for
ensuring the security and dependability of smart grid operations.

It is evident that machine learning and artificial intelligence work in tandem in ensemble learning and hybrid models. The
optimization SVD method proposed by Hashemipour et al. [113] illustrates how integrating distinct models improves both
data compression and analysis. An instance of ensemble learning strategies in action is the one examined by Wang et al.
[128] in their investigation of electricity price forecasting. A novel model for forecasting electricity prices was created to
overcome the difficulties in managing large amounts of price data in smart grids. It combined three modules: feature
extraction using a combination of Principle Component Analysis and Kernel function for dimensionality reduction, a hybrid
feature selector utilizing Random Forest and Relief-F algorithm based on Grey Correlation Analysis, and a price classification
forecast using a Support Vector Machine classifier based on differential evolution. The suggested model outperformed other

https://www.sciencedirect.com/science/article/pii/S2590123024003864 29/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

approaches, as evidenced by numerical results, and this made it an efficient solution for smart grids' efficient forecasting of
electricity prices.

When considering issues related to data quality, privacy, interoperability, and integration, these studies emphasize the need
for a holistic strategy when integrating machine learning and AI. Practical applications of real-world case studies, such as the
research conducted by Rabie et al. [119] regarding anomalous rejection methodologies, offer valuable insights into
surmounting obstacles and attaining efficient decision-making within the ever-evolving domain of smart grid technologies.
The integration of machine learning and artificial intelligence into smart grids is anticipated to yield advantages that will
propel progress in grid resilience and significantly influence the trajectory of energy management in the coming years.
RQ4
What role does blockchain technology play in ensuring secure and transparent transactions within the context of energy
smart grids?

Numerous studies provide evidence that blockchain technology has emerged as a revolutionary influence in safeguarding the
confidentiality and visibility of transactions within energy smart grids. The significance of blockchain technology in
establishing a decentralized and immutable ledger to safeguard energy transactions was emphasized by Jiang et al. [105].
The blockchain employs cryptographic methods to bolster security measures, thereby ensuring that unauthorized
modifications are thwarted. The integration of cryptographic techniques with this decentralized and secure ledger enhances
the dependability of transactions, which is consistent with the conclusions drawn by Hashemipour et al. [113]. Highlighting
the significance of data integrity, their research illustrates the most effective utilization of SVD for the compression of large
amounts of data within smart grids. The cryptographic mechanisms of the blockchain are consistent with these discoveries,
thereby enhancing security.

The incorporation of smart contracts into blockchain technology enhances the automation and security of transactions,
thereby diminishing the necessity for intermediaries, as evidenced by research conducted by Jose et al. [109] and Akram et
al. [111]. Jose et al. [109] underscored the importance of smart contracts in the secure management of data and the assurance
of reliable transactions. In a similar vein, the efficacy of smart contracts supported by blockchain technology in enhancing
the precision of prediction analysis for large-scale data within smart grids is illustrated by Akram et al. [111]. Because electric
power theft has proven difficult to combat with Advanced Metering Infrastructure (AMI), researchers are looking to machine

https://www.sciencedirect.com/science/article/pii/S2590123024003864 30/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

learning algorithms for solutions. The significance of blockchain in automating and securing transactions, thereby reducing
the hazards linked to human involvement, is highlighted by these results.

It has been demonstrated that decentralized energy trading platforms, enabled by blockchain technology, improve peer-to-
peer transactions while reducing the influence of centralized authorities. This is consistent with the findings of Kamil and
Ogundoyin's [122] research, in which the authors advocate for an anonymous bulk verification scheme for big data, with an
emphasis on the significance of safeguarding the privacy of users. To protect user privacy while authenticating large power
bids from electric vehicles (EVs) on vehicular networks and 5G smart grid slices, a novel method utilizing the certificateless
aggregate signature (CL-AS) algorithm is suggested. While maintaining participant privacy, this approach guarantees batch-
verifiable authentication, with security reliant on the Discrete Logarithm Problem's (DLP) intractability. Performance
analyses demonstrated that it is more efficient than comparable schemes, indicating improved acceptability and fairness in
the energy market.

7. Implications for future research

Despite the increasing amount of research on smart grids and big data analytics, many issues still need to be addressed
before this technology can be widely used. One of the main obstacles to data exchange and access is the lack of standard
formats and system compatibility. Most studies are conducted with ideal data conditions, which can limit progress in
research due to privacy concerns. The complexity of smart grid analytics makes it imperative that specialists from different
domains work together interdisciplinary. Only then can its full potential be realized. Given these challenges, it is clear that
smart grid research needs to focus on software and hardware availability, interoperability, data privacy, regulatory
compliance, computing, scalability, and optimization algorithms if we want them to work effectively and last. Improving grid
resilience and dependability requires leveraging heterogeneous data sources, fusing analytics with real-time control, and
developing computational algorithms.

Several important areas are covered by the suggestions for future research that aim to improve the efficiency and real-world
application of data science advancements in smart environment technology. Prioritizing privacy and security assessments,
we will delve into advanced encryption methods and secure computation techniques to safeguard sensitive consumer data
while we extract meaningful insights. Other areas of focus will include integrating diverse data analytics techniques to

https://www.sciencedirect.com/science/article/pii/S2590123024003864 31/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

explore the potential of hybrid models and fostering collaborations with smart grid operators for real-world implementation
studies.

Contributing to a thorough understanding and successful implementation of big data innovations in smart grids while
considering social and environmental impacts, this project investigates the integration of blockchain technology to improve
data integrity and transparency, builds decision support systems and user-friendly interfaces to help stakeholders
understand and use analytics insights, investigates interoperability standards, assesses analytics solutions' resilience, and
conducts cost-benefit analyses.

The field of smart environment technologies presents a plethora of real-world applications that might revolutionize the data
science field. Demand response management is one of the most common applications. Here, state-of-the-art data analytics
methods such as genetic algorithms and machine learning may be used to better forecast and manage energy demand,
maximizing the supply-demand balance in smart grids. Machine learning techniques like Convolutional Neural Networks
(CNN) and Long Short-Term Memory (LSTM) can be used in conjunction with big data analytics to develop dependable
systems that detect and prevent power theft. Another practical use is the development of intricate systems for the detection
of fraudulent activity. Fog computing's incorporation into smart grids offers practical answers to problems with latency and
energy consumption. The network's intentional deployment of fog computing nodes allows for real-time processing and
decision-making. This results in much shorter reaction times and improved performance for smart grid applications.

8. Conclusion
The present systematic review investigates the intersection of advancements in data science and smart environment
technologies, with a particular focus on smart utilities. The objective is to clarify significant developments, obstacles, and
consequences associated with the incorporation of big data analytics in influencing the trajectory of energy management.
With the growing need for energy solutions that are both efficient and sustainable, the convergence of data science and
smart infrastructure assumes a more critical role.

This exhaustive analysis of 25 research papers focuses on the convergence of big data analytics and smart grids, offering
insights into the various applications and challenges that exist within this domain. The research papers present successful

https://www.sciencedirect.com/science/article/pii/S2590123024003864 32/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

methodologies for canonical correlation analysis and adaptive energy management, as well as novel platforms, techniques
for responding to demand, and approaches for achieving optimal compression. The significance of big data analytics in
tackling critical challenges in smart grids is underscored by inquiries into load forecasting, energy efficiency, electricity theft
detection, and predictive modeling. These investigations emphasize the contributions of big data analytics in areas such as
energy efficiency, privacy protection, and predictive modeling.

The implications of these findings extend to future energy landscapes. The application of machine learning algorithms to
demand response management and larceny detection studies demonstrates the potential for enhanced security measures.
Future investigations into smart grids should place emphasis on the improvement of machine learning models, the
advancement of predictive analytics, and the enhancement of data processing efficiency. This all-encompassing synopsis
establishes the incorporation of big data analytics as a paradigm shifter in tackling obstacles and guiding intelligent power
systems towards a future characterized by enhanced resilience and adaptability.

To improve the accuracy of predictive analytics in smart grid management, future research should concentrate on developing
advanced machine learning algorithms. These algorithms should include deep learning neural networks as well as
reinforcement learning. Investigating distributed and edge computing architectures for processing data in real-time is
necessary to address scalability issues. It is necessary to conduct research into blockchain-based frameworks for secure data
sharing and transaction verification to integrate blockchain technology and enhance data security and decentralization.
Smart energy systems that are more autonomous and resilient can be developed by researchers by taking advantage of these
innovations.

CRediT authorship contribution statement

Hamed Taherdoost: Writing – review & editing, Writing – original draft, Visualization, Validation, Supervision, Software,
Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Data curation,
Conceptualization.

Declaration of competing interest

https://www.sciencedirect.com/science/article/pii/S2590123024003864 33/58
10/30/24, 5:06 PM A systematic review of big data innovations in smart grids - ScienceDirect

The authors declare that they have no known competing financial interests or personal relationships that could have
appeared to influence the work reported in this paper.

Recommended articles

Data availability
No data was used for the research described in the article.

References
[1] N. Roztocki, P. Soja, H.R. Weistroffer
The Role of Information and Communication Technologies in Socioeconomic Development: towards a
Multi-Dimensional Framework
Taylor & Francis (2019), pp. 171-183
Crossref View in Scopus Google Scholar

[2] A. Harerimana, N.G. Mtshali

Types of ICT applications used and the skills' level of nursing students in higher education: a cross-
sectional survey
International Journal of Africa Nursing Sciences, 11 (2019), Article 100163
View PDF View article View in Scopus Google Scholar

[3] O.I. Olatoye, F. Nekhwevha, N. Muchaonyerwa

ICT literacy skills proficiency and experience on the use of electronic resources amongst undergraduate
students in selected Eastern Cape Universities, South Africa
Libr. Manag., 42 (6/7) (2021), pp. 471-479
Crossref View in Scopus Google Scholar

https://www.sciencedirect.com/science/article/pii/S2590123024003864 34/58