Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Contents lists available at ScienceDirect

Journal of King Saud University –

Computer and Information Sciences
journal homepage: www.sciencedirect.com

Breaking down linguistic complexities: A structured approach to aspect-

based sentiment analysis
Kanwal Ahmed a, Muhammad Imran Nadeem a,⇑, Zhiyun Zheng a,⇑, Dun Li a, Inam Ullah b,
Muhammad Assam c, Yazeed Yasin Ghadi d, Heba G. Mohamed e,⇑
a
School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou 450001, China
b
College of Information Engineering, Henan University of Technology, Zhengzhou 450001, China
c
Department of Software Engineering, University of Science and Technology Bannu, KP, Pakistan
d
Department of Computer Science, Al Ain University, United Arab Emirates
e
Department of Electrical Engineering, College of Engineering, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia

a r t i c l e i n f o a b s t r a c t

Article history: Aspect-based sentiment analysis refers to the task of determining the sentiment polarity associated with
Received 10 March 2023 particular aspects mentioned in a sentence or document. Previous studies have used attention-based
Revised 15 June 2023 neural network models to connect aspect terms with context words, but these models often perform
Accepted 8 July 2023
poorly due to limited interaction between aspect terms and opinion words. Furthermore, these models
Available online 20 July 2023
typically focus only on explicitly stated aspect objects, which can be overly restrictive in certain scenar-
ios. Current sentiment analysis methods that rely on aspect categories also often fail to consider the
Keywords:
implicit placement of aspect-category information within the context. While existing models may pro-
Social media sentiment analysis
Aspect-based sentiment Analysis
duce strong results, they often lack domain knowledge. To address these issues, this study proposes an
Entity-oriented knowledge dependency Aspect-position and Entity-oriented Knowledge Convolutional Graph (APEKCG) consisting of two mod-
convolutional graph ules: the Aspect position-aware module (APA) and the Entity oriented Knowledge Dependency
Convolutional Graph (EKDCG). The APA module is designed to integrate aspect-specific sentiment fea-
tures for sentiment classification by incorporating information about aspect categories into different
parts of the context. The EKDCG module incorporates entity-oriented knowledge, dependency labels,
and syntactic path using a dependence graph. Experimental results on five benchmarks Natural
Language Processing (NLP) datasets of the English language demonstrate the effectiveness of the pro-
posed APEKCG framework. Furthermore, the APEKCG outperformed previous state-of-the-art models
with its accuracy, achieving 89.13%, 84.32%, 89.02%, 79.64%, and 90.22% on the MAMS, Laptop,
Restaurant, AWARE, and SemEval-15&16 datasets, respectively.
Ó 2023 The Author(s). Published by Elsevier B.V. on behalf of King Saud University. This is an open access
article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

1. Introduction sentiment analysis (Li et al., 2022c; Zhao et al., 2021a) that aims
to determine the sentiment polarities of users towards specific
Aspect-based sentiment analysis (ABSA) (Zhu et al., 2023; Wu aspect terms in a sentence. ABSA is different from traditional sen-
et al., 2022a; Zhang et al., 2022a) is a specialized area within text timent analysis because it focuses on aspect terms rather than
entire documents or sentences. This approach allows for better
insight into user reviews as it captures the sentiment of each
⇑ Corresponding authors. aspect separately. For example, in the sentence ‘‘Great food but
E-mail addresses: [email protected] (K. Ahmed), imran_nadeem@ the service was dreadful!”, the aspect terms ‘‘food” and ‘‘service”
gs.zzu.edu.cn (M.I. Nadeem), [email protected] (Z. Zheng), [email protected]
(D. Li), [email protected] (Y.Y. Ghadi), hegmohamed@ pnu.edu.sa
have positive and negative sentiment polarities, respectively.
(H.G. Mohamed). Sentence-level or document-level sentiment analysis would not
Peer review under responsibility of King Saud University. be able to accurately reflect this contrast in sentiment. Initially,
ABSA relied on linguistic variables to train classifiers and conduct
sentiment analysis on customer feedback using traditional
machine learning techniques like Naive Bayes, Logistic Regression,
Production and hosting by Elsevier and Support Vector Machine. These methods focused on

https://doi.org/10.1016/j.jksuci.2023.101651
1319-1578/Ó 2023 The Author(s). Published by Elsevier B.V. on behalf of King Saud University.
This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

high-frequency nouns or noun phrases that could indicate a terms and opinion words, thereby mitigating the limitations
reviewer’s emotional stance. However, there were limitations to observed in previous attention-based neural network models.
this approach, as some nouns were not sentiment items, and some
exceptional aspects were rarely discussed. Recent research has Our proposed APEKCG model consists of two modules: APA and
explored neural networks like Convolution-based Networks EKDCG. The APA module converts sentence and aspect category
(Nadeem et al., 2022), Long Short-Term Memory (LSTM) Networks, words into vector representations, while the EKDCG module builds
and Recurrent Neural Networks (RNN) (Nadeem et al., 2023), an entity-oriented knowledge graph using review information to
which consider a phrase as a sequence of words. However, these capture positive and negative sentiment polarity. We then use a
models had limitations in capturing long-term dependencies Relational Attention Graph Convolutional Network (RAGCN) to
between words and preserving syntactic information. execute the ABSA task by combining nodes, edges, syntactic paths,
Within the realm of ABSA, a clear differentiation exists between and dependency labels on the edges. The output from both mod-
‘‘aspect” and ‘‘attention.” Specifically, an aspect corresponds to the ules is combined and fed into the classifier for the final output.
precise constituents or characteristics that undergo scrutiny within Our method improves the traditional methods of paying attention
the ABSA framework. Conversely, attention assumes the role of a by using aspect-position embedding as the attention metric,
computational mechanism employed in models to allocate varying resulting in a more accurate computation of attention weight.
degrees of significance or emphasis to distinct segments of the Overall, the following are the main contributions of this paper:
input sequence, which encompass the aspects under consideration.
As researchers sought to address this matter, novel approaches 1. The paper proposes a novel APA module that integrates aspect
such as Gated Convolutional Networks (Xue and Li, 2018) and categories with contextual information for capturing the rela-
Attention methods (Mao et al., 2019) were introduced. However, tionship between aspects and their context.
these approaches inadvertently neglected the incorporation of 2. Additionally, a boosted attention network that uses aspect-
essential syntactic information (Wang et al., 2016; Huang and position embedding representation is presented to enhance
Carley, 2019). To rectify this limitation, the Aspect Position- aspect-specific sentiment analysis by focusing on the position
Aware module (APA) was developed, aiming to accentuate the sig- of the aspect in the sentence.
nificance of aspect-category information at different positions 3. The paper also proposes integrating entity-oriented knowledge
within the context. Structured models play a critical role in senti- into the dependency tree of sentences and building EKDCG
ment analysis in natural language processing (NLP), with two module to uncover latent sentiment polarity. The dependency
major categories being tree-based models and graph neural net- graph network is modeled using the labels on the edges indicat-
works (GNNs). Tree-based models, such as Tree-LSTM (Tai et al., ing dependencies, as well as domain knowledge and the syntax
2015), use phrase or dependency trees to capture syntactic infor- path. Together, the APA module and EKDCG offer a powerful
mation, while GNNs, such as Attention Graph Convolutional Net- approach to sentiment analysis that is both accurate and con-
work (Wu et al., 2022a; Huang et al., 2020) GCN (Zhu et al., textually aware.
2023; Zhang et al., 2019a), and Relational GCN (Liang et al., 4. We show that our APEKCG greatly outperforms the state-of-
2022) incorporate graph convolutional procedures to utilize both the-art algorithms on the ABSA problem by analyzing experi-
syntactic and semantic information. Both types of models have mental findings on the benchmark datasets.
shown promise in sentiment analysis, particularly in ABSA, and
have the potential to accurately represent data and detect compli- This article’s remaining content is structured as follows. In Sec-
cated emotional polarities. While tree-based models may struggle tion 2, we examine the prior research that paved the way for our
with syntactic parsing issues (Jang et al., 2016; Li et al., 2021a), approach. In Section 3, a detailed framework of our proposed
GNN models can provide stable and meaningful structures (Tang model APEKCG has been presented. Section 4 includes datasets,
et al., 2020; Tian et al., 2021). Ultimately, the success of sentiment implementation details, and analysis of results. The final section
analysis relies on the incorporation of syntactic and semantic infor- of the paper summarizes our efforts and offers a look ahead at
mation through structured models. Recent advances in GNNs have potential future lines of the research.
enabled the capturing of neutral and ambivalent attitudes in com-
ment data by aggregating information from node edges. However,
these models ignore the dependence labels present on the edges, 2. Literature review
which can be useful for mining sentiment pairings, and do not con-
sider domain knowledge that can influence sentiment polarity. In light of the proliferation of online discourse, sentiment anal-
Considering this, we present a unique Entity oriented Knowledge ysis is gaining traction in both academic and commercial settings
Dependency Convolutional Graph (EKDCG) for the ABSA challenge (Hoang et al., 2019). As one of the subtasks of sentiment analysis,
in this research. ABSA has gained significant attention in recent years. ABSA
To address the aforementioned issues the main objective of this involves identifying both sentiments and aspects in a highly com-
research: plex task. In recent research, attention mechanisms have been
integrated into neural networks to capture the combination of
1. To propose a novel framework for aspect-based sentiment anal- aspects and emotions in text phrases (Wang et al., 2016Tang
ysis, named the Aspect-position and Entity-oriented Knowledge et al., 2015; Chen et al., 2017). For instance, to isolate crucial feel-
Convolutional Graph (APEKCG). This framework aims to con- ings about the focus area, Wang et al. (2016) presented an
sider both explicitly stated aspect objects and the implicit attention-based LSTM. Similarly, Nguyen and Le Nguyen (2018)
placement of aspect-category information within the context. recorded each context word in reference to a specific target by
2. To integrate domain knowledge into the APEKCG framework by using a typical attention-based LSTM to model semantic linkages
incorporating entity-oriented knowledge, dependency labels, between each context word and a target. To solve the ABSA chal-
and syntactic paths. This integration of domain knowledge seeks lenge, Tang et al. (2016) created a memory network that uses
to enhance the performance of aspect-based sentiment analysis. multi-hop attention and external memory, while Chen et al.
3. To overcome the challenges encountered in aspect-based senti- (2017) developed a multilayered technique for capturing the polar-
ment analysis by improving the interaction between aspect ity of a sentiment item over time. In the field of sentiment analysis

2
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

across multiple domains, Co-LSTM, a CNN-LSTM hybrid model pro- tic knowledge (Dai et al., 2022), BiERU uses the bidirectional emo-
posed by Behera et al. (2021), requires extensive training on tional recurrent unit (Li et al., 2022b), and SenticGCN integrates
diverse data, limiting its practicality. Li et al. (2022a) analyzed sentiment knowledge (Liang et al., 2022). Zhao et al. (2021b) sug-
emotions in online reviews using social cognitive theory and the gested a syntax memory model that can encode syntactic depen-
Intent-Indicator sentiment lexicon. These approaches, as well as dency edges and label information together. Jain et al. (2022)
others (Ma et al., 2017; Fan et al., 2018), successfully extract syn- combined the TF-IDF technique with a Binary Brain Storm Opti-
tactic structure implicitly through the application of attention mization algorithm for detecting aspects. Zhu et al. (2023) pro-
mechanisms, presenting a novel concept for the ABSA task posed KMGCN, a knowledge-guided multi-granularity graph
(Nassif et al., 2021; Lin et al., 2020). It has been observed that sen- convolutional neural network with a multi-granularity attention
timent words and items are often located in close proximity, indi- mechanism that enhances the interaction between aspect terms
cating the effectiveness and efficiency of mining syntactic and opinion words. SSENM, proposed by Xiang et al. (2022), is a
structure through attention mechanisms. Additionally, these semantic and syntactic-enhanced neural model that includes
approaches have the potential to overcome the problems of long dependency graphs and a self-attentive mechanism to capture
word dependencies and gradient disappearance or explosion that semantic contextual information. The topic of affective computing
are encountered by RNN-based models. In some studies, language and sentiment analysis has attracted the attention of many
models like BERT have been utilized to collect aspect-sentiment researchers due to the essential role it will play in the development
pair data to improve the way languages are represented and make of emotionally intelligent products. The field of study is interdisci-
ABSA more accurate (Hoang et al., 2019; Li et al., 2019). plinary, spanning across Computer Science Cognitive Science, and
For the task of ABSA, BERT, GPT, and RoBERTa can all be viable Psychology (Cambria et al., 2017). In affective computing and sen-
options for generating contextualized word embeddings. However, timent analysis, the main areas of research are computation (Zhang
BERT is often considered a strong choice for ABSA due to its specific et al., 2022b), interpretation (Cambria et al., 2022), or neurosym-
design and performance characteristics. As BERT’s bidirectional bolic AI, like SenticNet 7 (Han et al., 2019; Cambria et al., 2019),
architecture allows it to consider both the left and right contexts and the generation (Liu et al., 2020) of human emotions or moods.
of a word, capturing a comprehensive understanding of word Luo et al. (2020) presented a novel approach named GRACE
meanings and relationships. This is beneficial for ABSA, where (Graph-Attentive Cascaded Labeling) for ABSA. GRACE employs a
the sentiment of an aspect often depends on the surrounding con- cascaded labeling strategy to foster improved interaction between
text. Additionally, it has the ability to capture fine-grained contex- aspect terms and enhance the attention allocated to sentiment
tual information helps in recognizing the sentiment associated tokens during the sentiment polarity labeling process. The model
with different aspects. While GPT and RoBERTa are powerful mod- incorporates two decoder modules: one for aspect term extraction
els, they have different focuses. GPT is more suited for autoregres- (ATE) and another for aspect sentiment classification (ASC). In the
sive language modeling and text generation, while RoBERTa is an ASC module, GRACE leverages a stacked multi-head attention
optimized variant of BERT that improves performance but doesn’t mechanism to capture the interplay between aspect terms. Fur-
have specific design features tailored for ABSA (Zhou et al., 2021). thermore, to tackle the challenge posed by imbalanced labels, the
Catching aspect-emotion pairs through attention processes is model employs a gradient harmonized loss function. Karimi et al.
insufficient when the aspect and emotion terms in a comment (2020) presented novel advancements tailored for ABSA tasks,
are far apart. While the attention mechanism is crucial in earlier specifically focusing on enhancing the BERT model. The authors
neural models, it is insufficient. To address this constraint, proposed two additional modules, namely parallel aggregation
researchers have suggested leveraging syntactic details to identify and hierarchical aggregation, to augment the capabilities of BERT
aspect-sentiment pairs and word dependency relationships. One for ABSA. Ben Veyseh et al. (2020) presented a methodology ter-
proposed solution is the target-dependent graph attention net- med GGCN (Graph-based Global Context Network) along with
work (TD-GAT) developed by Huang and Carley (2019), which clas- syntax-based regulation to enhance ABSA. The proposed approach
sifies sentiment at the aspect level using dependency graphs. integrates multiple fundamental components, including represen-
Another approach by Zhang et al. (2019b) is the proximity- tation learning (RL), graph convolution and regulation (GCR), as
weighted convolutional network, which considers the syntax of well as syntax and model consistency (SMC).
each aspect to represent contexts more accurately. Liang et al. (2021) proposed an interactive architecture consist-
GNNs that leverage dependency information have been found ing of five layers, augmented with a multi-task learning frame-
to be highly effective in capturing aspect-sentiment pairs, as sup- work, specifically designed for the ABSA task. This proposed
ported by empirical evidence that shows significant improvements architecture enables effective interaction and information flow
in accuracy for the ABSA test (Hou et al., 2019; Wang et al., 2018). A between the different layers, facilitating the integration of multiple
recent study proposed a novel Graph Convolutional Network called tasks within the ABSA framework. Zhang et al. (2021) introduced
AAGCN, which integrates attention to aspects. The authors modi- the SSN (Syntactic-Semantic Network) model for aspect-level sen-
fied the original sentence’s syntactic dependency tree, using a beta timent classification. Their research introduces a fusion mecha-
distribution-based algorithm that takes into account both the nism that efficiently integrates aspect and context information,
aspect and external emotional knowledge to figure out the rela- enabling a comprehensive understanding of sentiment at the
tionship weights between the two (Liang et al., 2023). This allows aspect level. Qi et al. (2022) presented a model for aspect term sen-
the AAGCN to effectively capture the sentiment expressed towards timent classification that leverages aspect-sensitive word repre-
specific aspects of text data, improving the overall performance of sentations obtained via a weakly supervised approach. This
sentiment analysis tasks. methodology effectively tackles the issue of words exhibiting vary-
GCN networks have been developed to classify emotions across ing sentiment polarities based on different aspects, ensuring accu-
various dimensions. In a subsequent study, researchers proposed a rate sentiment classification at the aspect term level. Table 1 and
novel meta-learning technique that considers both historical and Table 2 provide a concise overview of the relevant techniques,
real-time aspect-aware data for a new few-shot aspect category encompassing their respective contributions and limitations. It
sentiment analysis task (Liang et al., 2021). In addition, some stud- serves as a summary of the aforementioned approaches, allowing
ies have combined syntactic data with common sense knowledge for a comprehensive understanding of the field.
in GCNs. For example, the SEDC-fusion GCNs leverage syntactic Yu and Zhang (2023) present the multiweight graph convolu-
knowledge (Zhu et al., 2022), while Syntactic-GCN employs seman- tional network (MWGCN), aimed at addressing certain limitations
3
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Table 1
Summary of literature review-1.

Model Year Dataset Contributions Limitations

GRACE (Luo et al., 2020 Rest14, This study effectively tackles the challenge of imbalanced Post-training is an essential step in the process and
2020) Lap14, labels and promotes interplay between aspects during the necessitates the utilization of multiple loss functions.
Rest15, process of labeling polarities. It also facilitates meaningful
Rest16, interactions between different aspects, resulting in
Twitter improved sentiment polarity labeling.
Parallel and 2020 Rest14, The incorporation of integration enhances the performance To address the overfitting issue associated with small
Hierarchical Lap14, of BERT by mitigating overfitting challenges encountered datasets, a greater number of epochs is required. However,
Aggregation Rest16 when dealing with small datasets. the inclusion of two distinct loss functions for separate
(Karimi et al., tasks can impose a burden on the overall architecture,
2020) thereby resulting in inaccurate predictions.
GGCN (Ben Veyseh 2020 Rest14, In this research, a customization approach was employed By minimizing the cosine similarities between the gate
et al., 2020) Lap14, to modify the aspect terms at various hidden layers of vectors, this approach inadvertently generates irrelevant
MAMS Graph Convolutional Networks (GCN) for aspect term outcomes. Furthermore, the model is solely designed to
extraction. Furthermore, this study explicitly leverages the handle the Aspect Term Extraction (ATE) task without
global importance scores of words in the sentence to encompassing other tasks.
capture the syntactic neighbor words, thereby enhancing
the syntactic analysis aspect of the model.
Position Bias (Ma 2021 Rest14, This study demonstrates robust aspect classification This study is incompatible with pre-trained language
et al., 2021) Lap14 performance, even in Out-of-Domain (OOD) scenarios. It models, as it lacks the ability to effectively extract the
effectively addresses the challenges posed by adversarial syntactic structure of the sentence.
perturbations in the input, ensuring reliable and accurate
classification results.
DREGCN (Liang 2021 Rest14, This study successfully captures the syntactic structure of This study encountered limitations in capturing the
et al., 2021) Lap14, the sentence by effectively leveraging multiple related relationship between an aspect and a sentence when they
Rest15 tasks. It goes beyond the sentence-level analysis and are not in close proximity to each other.
incorporates document-level knowledge at the relational
level, leading to a comprehensive understanding of the
text.
SA-GCN (Hou et al., 2021 Lap14, Rest This study effectively establishes the relationship between Expanding the number of GCN layers during training does
2021) 14, Rest 15, an aspect and opinion words, even in cases where they are not yield improvements in performance.
Rest 16 spatially distant from each other within the sentence.

Table 2
Summary of literature review-2.

Model Year Dataset Contributions Limitations

SSN (Zhang 2021 Restaurant, This study emphasizes the significance of syntax and In the Laptop experiment, the proposed model exhibits
et al., 2021) Laptop employing a fusion mechanism to seamlessly integrate subpar performance in the case of three aspects. The
aspect and context information. research findings suggest that the model’s inaccurate
predictions are primarily concentrated on data instances
where the aspect term spans more than four words.
R-GAT (Wu 2022 Rest14, Lap14, This study concentrates on target aspects within the This study encountered limitations in capturing relations
et al., Twitter dependency tree to encode dependency relations, thereby within complex sentences. Specifically, it struggled to
2022b) establishing a strong correlation between aspects and capture relations when the dependency distance between
targets. aspect and sentence words became excessively long.
ASWR (Qi 2022 Restaurant- The study emphasizes the significance of accounting for the The proposed method utilizes weakly supervised techniques
et al., 2022) 14, Laptop-14, aspect-specific characteristics of words in sentiment to construct aspect-sensitive lexicons, which may introduce
Restaurant-15 classification. certain inaccuracies when capturing the sentiments
associated with specific aspects.

associated with the attention mechanism in detecting aspect- eling of aspect-specific information. The second method, Emotional
relevant semantics and incorporating aspect position information, Intensive Sentiment Reasoning (EISR), incorporates emotionally
specifically the limited detection of such semantics and the over- intensive information into the reasoning mechanism. Moreover,
sight of the aspect’s long-distance dependence. The MWGCN the significance of commonsense knowledge, semantic under-
method incorporates two weighting techniques, namely multi- standing, and syntax information in ABSA has been recognized in
grain dot-product weighting (MGDW) and Local Context weighted previous research, yet few approaches have addressed them simul-
adjacency Graph (LCG). MGDW serves to preserve the overall con- taneously. In response, a novel graph convolutional network is pro-
text semantics while placing greater emphasis on aspect-related posed that incorporates commonsense knowledge, syntax, and
features. Ma et al. (2023) involves the development of the Mul- semantics (Zhao et al., 2023a). Wu et al. (2023) introduces the
tiGCN model, the introduction of difference and similarity losses, Aspect Word and Context Order Prediction Task (ACOP) as an aux-
and the demonstration of improved prediction performance com- iliary task. The ACOP task is implemented using both global and
pared to existing models. Arumugam and Nallaperumal (2023) local context information for aspect-based sentiment analysis,
address the issue of long-range dependencies and the identifica- and a self-supervised method is employed to train the model.
tion of sensitive and important words are tackled through the Zhao et al. (2023b) introduced a multitask learning framework
introduction of two separate methods. The first method, known for aspect-based sentiment analysis, which integrates Aspect Term
as Adaptive Aspect-Specific GCN (AASGCN), enhances the Aspect- Extraction (ATE) and Aspect Polarity Classification (APC) tasks. By
Specific Graph Convolutional Networks (ASGCN) by integrating training ATE as an auxiliary task, the model enhances the attention
adaptive weights. This enhancement enables more effective mod- of APC on pertinent aspects, leading to improved aspect polarity

4
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

classification. The inclusion of graph neural networks (GNN) and and demonstrates the effectiveness of the APEKCG model in
dependency syntax information significantly contributes to the achieving accurate sentiment analysis.
enhanced performance of APC. Furthermore, the utilization of mul-
tihead attention (MHA) facilitates the association of dependency 3. Methodology
sequences with aspect extraction, effectively highlighting crucial
dependency relations. The SA-BERT model, proposed by Mewada Our proposed methodology consists of two modules, named
and Dewang (2023), leverages a combination of synthetic attention APA and EKDGC. Fig. 1 depicts the entire framework of the sug-
and bidirectional encoder representations from transformers to gested methodology. Semantic representation, aspect-position
perform sentiment polarity classification on review datasets. By embedding, and aspect-position attention learning make up the
employing dynamic word vector encodings and transformers, the APA module. Aspect-specific sentiment information in sentences
model adeptly captures the aspect and context of reviews, effec- can be more smoothly integrated by using a strategy based on
tively representing their semantics. The synthetic attention mech- attention modeling in conjunction with aspect-position embed-
anism employed by the model enables it to learn and focus on the ding. The APA modules successfully extract the necessary features,
essential components of context and aspects within the reviews. leading to reduced coupling and high robustness. In the second
The MAPA BiLSTM-BERT model proposed by Wankhade et al. module EKDGC of the proposed framework APEKCG, we built an
(2023), tackles unresolved challenges in aspect-level sentiment Entity-Oriented knowledge dependency convolutional graph in
categorization. By integrating multiple aspect-specific position three phases according to K-Bert (Liu et al., 2020). As shown in
attention and aspect-specific attention mechanisms, the model Fig. 1 (‘‘Knowledge Layer,”) the process begins by building a knowl-
effectively captures contextual information and handles multiple edge graph of words that appear together in comment phrases.
aspects simultaneously. Zheng et al. (2023a) proposed a Light- Second, the Biaffine parser (Dozat and Manning, 2016) and Stan-
weight Multilayer Interactive Attention Network (LMIAN) for ford CoreNLP (De Marneffe and Manning, 2008; Manning et al.,
ABSA. LMIAN utilizes a pre-trained language model for initializing 2014) construct the dependency tree of the comment phrases,
word embedding vectors. LMIAN incorporates an interactive com- the structure of which is depicted in Fig. 1 (‘‘Dependency tree”).
putational layer that establishes correlations between aspect Knowledge was fused into the dependency tree via the knowledge
words and their contexts. These correlations are calculated graph built with the help of the preceding processes, creating the
through multiple computational layers using neural attention Entity-Oriented knowledge dependency graph. Finally, we com-
models. Xin et al. (2023) proposed a syntactic and semantic bine the entity-oriented knowledge dependency graph with the
enhanced multi-layer graph attention network (SSEMGAT) that Attention Relational Graph Convolutional Network to synthesize
addresses the limitations of the dependency tree-based model by the data from the nodes, edges, dependency labels, and domain
incorporating constituent trees and aspect-aware attention. SSEM- expertise. The output generated from module 1 and module 2 is
GAT demonstrates competitive performance in sentiment analysis fused together and the final classifier generates the output, also
tasks, as evidenced by the evaluation of different datasets. described in Algorithm 1.
Zheng et al. (2023b) propose the design of a corpus-level sen-
timent knowledge fusion mechanism that enhances the under- Algorithm 1. APEKCG
standing of sentiment information in aspect-based sentiment
analysis. The use of dependency graphs, sentiment knowledge
Input: List of sentences, List of comment phrases
nodes, and shared sentiment knowledge leads to improved aspect
Output: Final classification output
representation and performance. Yang et al. (2023) combines
1: Module 1: features = Apply the APA Module on the list of
adversarial training with the BERT model and capsule networks
sentences:
for aspect-based sentiment analysis. This integration allows for
2: semanticReps = SemanticRepresentation(sentences)
better utilization of neural networks’ superiority and addresses
3: aspectEmbeddedSentences = EmbedAspectPositions(se
the problem of insufficiently mining sentence semantic informa-
manticReps)
tion due to limited training data. Zhang et al. (2023) introduce
4: aspectSentencesWithAttention = AspectPositionAttention
a Contrastive Learning Framework with Tree-Structured LSTM
(aspectEmbeddedSentences)
(CLF-TrLSTM). This framework leverages concatenated Tree-
5: features = ExtractFeatures(aspectSentencesWithAttention)
LSTMs and self-attention with a window mechanism to adeptly
6: Module 2: graph = Apply the Entity-Oriented Knowledge
capture both syntactic and contextual information from depen-
Dependency Graph Convolutional (EKDGC) Module on the
dency trees. Furthermore, it promotes alignment between anchor
list of comment phrases:
sentences and positive samples while effectively distinguishing
7: knowledgeGraph = BuildKnowledgeGraph(commentPhra
them from negative example pairs. Table 3, Table 4, and Table 5
ses)
are provided to summarize the latest techniques, highlighting
8: dependencyTree = ConstructDependencyTree(comment
their respective contributions and limitations. This tabulated pre-
Phrases)
sentation offers a comprehensive overview of the recent advance-
9: entityGraph = FuseKnowledge(knowledgeGraph,
ments in the field.
dependencyTree)
Recent research indicates that achieving accurate sentiment
10: synthesizedData = SynthesizeData(entityGraph)
analysis in ABSA tasks requires the incorporation of both linguistic
11: fusedOutput = Combine(features, synthesizedData)
and sentiment knowledge. In this study, the proposed APEKCG
12: finalOutput = FinalClassifier(fusedOutput)
model uses aspect position attention and entity-oriented knowl-
13: Return finalOutput
edge dependency graphs to conduct sentiment analysis. The model
integrates multiple types of information, including aspect position
attention, node and edge information, dependency label informa-
tion on edges, entity-oriented knowledge, dependency labels, and
syntax path, using ARGCN. Through Ablation Studies, the APEKCG 3.1. Module 1: Aspect Position-aware Module (APA)
model was shown to outperform current state-of-the-art models.
Therefore, this study emphasizes the significance of incorporating This section is further divided into subsections which are
entity-oriented knowledge and dependency labels in ABSA tasks described in detail.
5
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Table 3
Summary of most recent literature review-1.

Model Year Dataset Contributions Limitations

MultiWeight Graph 2023 Twitter, By combining context representations, syntactic The study acknowledges that selecting coefficients
Convolutional Lap14, Rest14, information, and aspect features using the weighted manually through experiments to optimize the model
Network (MWGCN) Rest15, Rest16 adjacency graph, the multilayer GCN focuses on local is inefficient. Therefore, the study suggests the need to
(Yu and Zhang, 2023) context information for improved ABSA performance. explore the adaptive selection of coefficients in the
ABSA model in order to ensure optimal model
performance.
MultiGCN (Ma et al., 2023 Restaurant, The study finds that models utilizing a dependency tree The study does not explicitly address the impact of the
2023) Laptop, or its variants tend to achieve higher accuracy rates in observed dependency labels on different types of
Twitter, aspect-based sentiment analysis (ABSA), indicating the sentiments or specific aspects. Furthermore, the study
MAMS benefits of incorporating structural information. does not explore the generalizability of the findings to
languages other than English.
EIAASG (Arumugam and 2023 REST16, The study introduces two approaches, the first The study lacks a comprehensive discussion of the
Nallaperumal, 2023) REST15, approach, AASGCN, enhances ASGCN by incorporating limitations or potential challenges associated with the
REST14, adaptive weights, enabling better capture of the proposed enhancements. Addressing these limitations
LAP14, semantic meaning of opinion targets. The second could provide a more nuanced understanding of the
TWITTER 17 approach, EISR, incorporates emotionally intensive performance and applicability of the proposed models.
information into the sentiment analysis mechanism.
SSK-GCN + BERT (Zhao 2023 Twitter, By combining knowledge, syntax, and semantics, the One limitation of the study is that it does not consider
et al., 2023a) Lap14, Rest14, proposed model achieves improved performance in edge information, which could be a potential avenue for
Rest15, Rest16 sentiment prediction. further improvement in future research.
ACOP (Wu et al., 2023) 2023 Restaurant, The contributions of this study involve the introduction The limitations of this study include the lack of
Laptop, of a multi-task approach for ABSA, and the integration of external knowledge, the noise introduced
Twitter implementation of the ACOP learning strategy for in the local approach, and the need for further
sentence semantics in ABSA. improvements and exploration of alternative
techniques.
Syntax based (Zhao 2023 Restaurant, Incorporating GNN and dependency syntax information The proposed model may still face challenges in
et al., 2023b) Laptop, contributes to the performance of APC. The use of handling complex sentence structures or ambiguous
Twitter multihead attention (MHA) associates the dependency aspect terms. The limitations and potential drawbacks
sequences with aspect extraction, emphasizing of using multitask learning and the specific approaches
important dependency relations. employed in the study are not thoroughly discussed.

Table 4
Summary of most recent literature review-2.

Model Year Dataset Contributions Limitations

SA-BERT (Mewada 2022 Laptop, The proposed SA-BERT model, combines synthetic The assessment of sentence dependency relationships
and Dewang, Restaurant14, attention with bidirectional encoder representations has not been conducted in the presented study.
2023) Restaurant 15, from transformers, which achieves high accuracy and F1
Restaurant 16 scores in classifying sentiment polarity in review
datasets.
MAPA BiLSTM BERT 2023 laptop, The proposed model considers multiple aspects One of the limitations of the study lies in the heightened
(attention based) restaurant, simultaneously, effectively reduces interference complexity associated with assigning weights in various
(Wankhade et al., Twitter between different aspects, and directs attention to challenging scenarios, which is comparatively more
2023) specific parts of the sentence. challenging than previous approaches that do not
incorporate syntax. Furthermore, this model introduces
greater computational complexity compared to earlier
methods.
LMIAN (Zheng et al., 2023 Laptop, The LMIAN overcomes a shallow interactive approach, The limitations of the LMIAN model include potential
2023a) Restaurant, which results in a lack of complex sentiment struggles in capturing implicit sentiment, limitations in
Notebook, information. understanding complex information, and the need for
Phone, Car, additional modalities to improve judgment in certain
Camera cases.
SSEMGAT (Xin et al., 2023 Restaurant, The SSEMGAT model addresses noise in dependent trees The limitations and potential challenges associated with
2023) Laptop, Twitter and enhances syntactic and semantic features. dependent trees, composition tree structures, multi-
head attention mechanisms, and deeper correlation
between syntax and semantics are not explicitly
outlined. Further investigation is required to explore
these limitations and refine the model’s design and
performance.
SEGCN-BERT (Zheng 2023 Lap14, Rest14, The incorporation of a sentiment knowledge fusion The limitations of the proposed model include its
et al., 2023b) Rest15, Rest16 mechanism at the corpus level enhances the performance on knowledge-limited datasets, the limited
comprehension of sentiment information in aspect- impact of sentiment knowledge nodes on datasets with
based sentiment analysis. By leveraging dependency few opinion words, the inability to establish connections
graphs, sentiment knowledge nodes, and shared for phrase-level sentiment expressions, and the lack of
sentiment knowledge, the approach facilitates enhanced specific guidance for addressing these limitations in
aspect representation and performance. future research.

3.1.1. Semantic representation WordPiece, a tool developed by BERT, is a tokenizer that can break
The essential task here is to identify and isolate the words in the down the input text X into a list of tokens. Next, a high-
original text that belong to the aspect categories. Our complete dimensional embedding representation is built from the token
modeling architecture relies on the outcomes of the pre-trained sequence as Eq. 1.
BERT model to obtain the embedding representations of the text.
Ex ¼ ½ec ; e1 ; e2 ; . . . ; en ; es  ð1Þ

6
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Table 5
Summary of most recent literature review-3.

Model Year Dataset Contributions Limitations

ABCN (Yang 2023 Restaurant, The proposed ABCN approach combines the strengths of Limited analysis of adversarial training, Lack of extensive
et al., Laptop, adversarial BERT, capsule networks, and label smoothing evaluation of different datasets, Evaluation of computational
2023) Twitter regularization to improve aspect-based sentiment analysis. efficiency, and Lack of comparison with alternative
approaches.
CLF-TrLSTM 2023 Restaurant, The integration of tree-LSTM for modeling syntactic The proposed architecture necessitates the inclusion of all
+Bert Laptop, information and self-attention with a window mechanism for aspects present in the text as input, which can pose challenges
(Zhang Twitter capturing contextual information significantly enhances the in scenarios with a large number of aspects or when the
et al., framework’s capability to leverage intrinsic information aspects dynamically change. Moreover, the adoption of
2023) within sentences. dependency trees and distinct encoders for syntax and context
information may introduce additional complexity and
computational overhead to the model.

Fig. 1. Framework of the proposed methodology.

by use of the embedding layer. BERT’s embedding layer has a transforming the aspect category terms into a preliminary embed-
dimension of d, so Ex 2 Rdðnþ2Þ , where the length of the given text ding matrix T a and then performing an average-pool operation on
input is denoted by n. The CLS is represented by the vector ðec Þ them as Eq. 3 and Eq. 4.
for the first token and the SEP by the vector ðes Þ for the last token.

Following the BERT multi-layer encoding, an initial embedding T a ¼ t ac ; ta1 ; t a2 ; . . . ::; tam ; tas ð3Þ
matrix is constructed using the hidden states of the last layer as !
Eq. 2. 1 Xm
va ¼ tac þ tas þ tai ð4Þ
mþ2 i¼1
T ¼ ½t c ; t 1 ; t 2 ; . . . ; t n ; t s  ð2Þ

Given that t i 2 Rd provides the final properties of each token, it 3.1.2. Aspect-position embedding
is abundantly evident that BERT has harmonious input and output This component is used to extract spatial references from text
dimensions. After taking this additional step, we will be able to using the aspect category. The module’s structure for embedding
obtain the embedding vector v a 2 Rd of the aspect category by aspect and location information is shown in Fig. 1. The output of
7
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

the text semantic representation module (i.e., the matrix T repre- ci ¼ g i  e

c i þ f i  ci1 ð14Þ
senting the final text embedding) and the obtained aspect-
category embedding ðv a Þ serve as inputs to this section. While a oi ¼ sigmoidðW o :½hi1 ; ni  þ bo Þ ð15Þ
given text may contain elements from numerous aspect categories,
traditional BERT is unable to communicate the resulting richness of hi ¼ oi  tanhðci Þ ð16Þ
meaning. This method fails to clearly forecast the sentiment classi-
fication of different aspect categories, reverting to the text’s senti- H ¼ ½h0 ; h1 ; h2 ; . . . :; hnþ1  ð17Þ
ment classification. To make the original semantic representation
To which the weight matrices W f ; W g ; W c , and W o correspond,
more suitable for the supplied aspect category, We propose match-
and the biases bf ; bg ; bc , and bo . Using the hidden vector sequence
ing position information and creating an aspect-position embed-
H, we may deduce how the aspect-position embedding matrix P
ding function to produce location features. This will allow us to
relates to each hidden state. Using a weighted hidden representa-
construct location features. To be more specific, a similarity score
is computed for the aspect-category vectorv a as well as for each tion v a Rd , the attention-weight vector aRnþ2 is created using the
vector contained in the vector–matrix T, and the index Smax of the aspect-position attention mechanism as Eq. 19 and Eq. 20.


vector that exhibits the greatest degree of similarity is selected M ¼ tanh W h H; W p P ð18Þ
as Eq. 5. After that, one calculates the embedding weight r i of each
 
vector in the matrix T by using Eq. 6: a ¼ softmax wTa M ð19Þ
 
Smax ¼ argmax v Ta T ð5Þ
v a ¼ HaT ð20Þ
!
2
ði  Smax Þ where MR2dðnþ2Þ ; W h Rdd ; W p Rdd , and wa R2d are projection
r i ¼ exp  ð6Þ
2r2 parameters.

In the given context, the notation i 2 f0; 1; 2; . . . ; ng is the index

3.2. Module 2: Entity-Oriented Knowledge Dependency Convolutional
of a word in the input text, and v Ta is the transpose of matrix v a . The Graph(EKDCG)
hyperparameter r 2 R can be used to modify the rate at which a
location is embedded. We can estimate the aspect-category distri- This section is further divided into subsections which are
bution in the text by utilizing the similarity score, and our pro- described in detail.
posed aspect-position function’s weights can prioritize crucial
features closer to the aspect category while de-emphasizing fea- 3.2.1. Knowledge layer
tures that are farther away. This accomplishes the effect of dynam- A knowledge graph consisting of named entities extracted from
ically integrating information about aspect categories into text. We the phrases was built using YAGO (Rebele et al., 2016; Agichtein
further enhance the location attributes’ suitability for aspect cate- and Gravano, 2000; Bunescu and Pasca, 2006), which is a linked
gories by learning an aspect position representation from text and database created by Max Planck Institute researchers in Germany
aspect-category embedding P. Mathematically, P can be deter- (Cambria et al., 2014). It compiles information from three primary
mined by Eq. 10. sources—, GeoNames, WordNet, and Wikipedia,—in order to pro-
t i ¼ r i  t i ð7Þ vide its results. YAGO’s entity categorization system is robust as
it combines vocabulary definitions from WordNet with Wikipedia’s
v ia ¼ ð1  ri Þ  v a ð8Þ categorization scheme. In addition to this, YAGO provides attribute
descriptions of geographical and temporal aspects for many
pi ¼ ti þ v ia ð9Þ knowledge entries, making it a comprehensive tool. To extract
entity-oriented knowledge in sentences and mine the likely senti-

 ment polarity of sentiment items, we utilized YAGO. A person may
P ¼ p0 ; p1 ; p2 ; . . . ; pn ; pnþ1 ð10Þ
have positive attitude polarity toward the iPhone since it is the
where pi 2 Rd represents the ith vector in the representation of brand she prefers more when comparing three different brands
aspect-position P. of smartphones (such as Apple, Samsung, and Huawei). Another
person who favors Huawei mobile phones, however, has a bad or
3.1.3. Attention-to-aspect-position ambivalent opinion of iPhones. To sum up, we extract the nouns
Utilizing the attention mechanism, the module’s primary role is and verbs from a sentence including a remark, and then we fuse
to assimilate the text’s most important details. The aspect-position the relevant knowledge about those nouns and verbs.
attention learning module is shown by a schematic in Fig. 1. The
text embedding matrix T and the aspect-position embedding P 3.2.2. Entity-oriented knowledge dependency graph
serve as the module’s input units. The standard LSTM is not adept
at focusing on capturing the essential semantic information that is
linked to the aspect category in the text. As a solution, we augment Algorithm 2. Entity-Oriented Knowledge Dependency Graph
the LSTM with a P-based attention technique based on aspect-
 
position embedding. After feeding the text embedding matrix T Input: dependency tree T ¼ V; Eedge , where
 
into an LSTM, we obtained a series of hidden layer vector matrices V ¼ v 1 ; . . . :; v jv j ; Eedge ¼ r ij ; v i ; v j ;
H, where hi is the hidden vector as Eq. 17, multiplied by a constant. Output: Entity-Oriented Knowledge Dependency Graph
   
f i ¼ sigmoid W f :½hi1 ; ni  þ bf ð11Þ (KDG): T KDG ¼ V KDG ; Eedge KDG

  1: Entity list Eentity ¼ e1entity ; . . . :; eKentity

g i ¼ sigmoid W g :½hi1 ; ni  þ bg ð12Þ
2: for entity eientity Eentity do
 
e
c i ¼ tanhðW c :½hi1 ; ni  þ bc Þ ð13Þ eiKG ¼ K Queryðt; KÞ; T ¼ K inject t; eiKG ;
3: end for

8
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

The Algorithm 2 explains how to build an Entity-Oriented where R is the collection of relations, hi is the input representation
of node v i ; hi is the output representation of node v i ; Nri ri is the set
0
knowledge Dependency Graph (KDG). Algorithm 2 takes YAGO K,
dependency syntax tree T, and entities E extracted from sentences of neighbors of vi under relation r R; W r and W 1 are trainable
parsed by Biaffine and CoreNLP parsers as its inputs. The depen- parameters, and ci;r is a normalization constant that is problem-
dency graph is denoted by T. specific and is usually assigned as the number of neighbors of v i
     under relation r. There is also a per-element activation function r.
T ¼ V; Eedge , where V ¼ v 1 ; . . . . . . v jV j ; Eedge ¼ rji ; v i ; v j .
Each relation r in Eq. 24 is represented by a unique relational
Here, we make use of the two procedures (i.e.,K Q ueryðt; K Þ and
  matrix W r . We use a procedure called basis decomposition to cut
K I nject t; eiedge eiedge Þ of K-BERT (Jain et al., 2021) to inject knowl- down on the number of parameters (Schlichtkrull et al., 2018).
edge about entities into the syntactic tree. Where EKG is a collec- Specifically, we decided on a single base:
tion of triples providing knowledge information and can be W r ¼ br W 0 ð25Þ
expressed as follows, K Q ueryðti ti; K Þ selects all the entity names
involved in the ith sentence tree ti to query their associated triples where br is a function of r as a coefficient. The number of parame-
from K in Eq. 21. ters is substantially decreased because all W r use W 0 as their com-
mon base. However, br represents the level of impact with regard to
EKG ¼ ½ðr i0 ; wi ; wi0 Þ; . . . :; ðrik ; wi ; wik Þ ð21Þ
relational categories. To increase RGCN’s effectiveness, we add a
To create the Entity-Oriented Knowledge Dependency Graph t, distance-aware attention mechanism in ARGCN.
the K Injectðti; EKGÞ operation inserts EKG into the sentence tree t
xij;r ¼ aij;r W 0 hj ð26Þ
by linking the triples in EKG to their appropriate nodes in the tree.


3.2.3. Context information representation aij;r ¼ r cT br ; bij ð27Þ
Sequence information is not contained in single words. Typi- where ij is an attention coefficient for the pair wi ; wj and c is a train-
cally, we use the surrounding text to figure out what a term means.
able vector that modifies both the relation and the attention coeffi-
The importance of accurately portraying words in their proper con-
cient. As an example of an activation function, we will employ ReLU
texts (Wu et al., 2020) cannot be overstated. In this study, a bidi-
in the ARGCN layers. We assume that node features and node posi-
rectional LSTM (Bi-LSTM) model with a global context (Schuster
tion in the phrase jointly determine the attention coefficients
and Paliwal, 1997) was employed to produce word embeddings
between any two nodes. We start by multiplying the same projec-
and global context vectors that are enriched with contextual infor-
tion matrix, W1, by the features of the nodes we’re interested in,
mation. Building on previous research on ABSA (Ben Veyseh et al.,
hi and hj, to get our query and key. Then, following the method of
2020), the significance of syntactic information, particularly phrase
Dai et al. (2022), we obtain the sinusoid encoding matrix p, which
structure, was also identified in collecting information about word
encodes positions relative to one another. Then we pay attention
collocation norms. Consequently, we provide the syntax path of
to the query, the key, and the relative positional encoding using a
each word to properly depict them. Supposing the length of the
shared attention mechanism:
input sentence is s, we may write it as w ¼ wi ; . . . :; wt ; . . . :; wjsj , 

where wt is the t  th word. The word vectors wt 2 Rd are initial- oij ¼ r aT W 1 hj ; W 1 hi ; p ð28Þ
ized using BERT (Devlin et al., 2018), where d is the total number
where a is a scalar-to-vector mapping that can be trained from the
of word vectors as Eq. 22.
concatenated representation. Finally, we use the Softmax function
f b to standardize oij relative to all of wi 0s neighbors:
ht ¼ wt þ ht þ ht þ pt ð22Þ
 
exp oij
where and
f
ht
b
ht
signify that the Bi-LSTM retains both historical and bij ¼ X ð29Þ
expðoik Þ
expected context for the string wt t¼1...jsj , respectively. ht are inputs kNi
for the ARGCN module; both global context and syntactic informa-
tion are contained in ht , directing information transfer in higher where ij represents the weight given to wi by wj in terms of the node
layers. representations and their relative position.
Further, including multi-head attention in our system helps to
3.2.4. Attention-Based Relational Graph Convolutional Network steady the learning process and boosts performance. Specifically,
(ARGCN) the modification of Eq. 30 is carried out using K-independent
In order to enclose the meticulously made graph of syntactic attention methods.
dependencies, we build on R-GCNs (Schlichtkrull et al., 2018) and
xkij;r ¼ akij;r W k0 hj ð30Þ
add a distance-aware attention mechanism. This research proposes
a relational graph convolutional network that may focus on a where dj is the input dimension, dk is the dimension of each head,
specific object of interest. Our model’s major goal is to take into
and W k0 Rdj dk . In this case, the output of the multi-head attention
account the interplay between aspect words and entities in a uni-
mechanism by Eq. 31:
fied manner, taking into account both semantic and contextual rel- h i
evance. To keep their secret states up-to-date, R-GCNs xij;r ¼ W d x1ij;r ; x2ij;r ; . . . :; xKij;r ð31Þ
(Schlichtkrull et al., 2018) aggregated the representations of their
neighbors at each node based on the type of edges between them. where W d RKdk djþ1 preliminary experiments show that setting
0 1 dk ¼ djþ1 improves performance slightly compared to the method
XX
hi ¼ r@
0
X ij;r þ W 1 hi A ð23Þ used by Vaswani et al. (2017), who used djþ1 =K as a dimension of
r R jNr each head. Syntactic dependency tree analysis reveals that phrases
i
denoting aspects and opinions frequently have direct or indirect
connections to one another in the graph. Using L-layers of ARGCN,
1
X ij;r ¼ W r hj ð24Þ we are able to capture these direct or indirect connections because,
ci;r as we add more and more ARGCNs, more and more information is
9
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

shared with our Lth order neighbors. On top of that, as the network allocation for training and testing purposes offers a rational equi-
layers increase in depth, ARGCN becomes increasingly smooth. To librium between the biases and variances encountered during
address this issue, we embed a residual link into every layer of model evaluation. This approach effectively addresses the chal-
the ARGCN network: lenges of overfitting and underfitting by finding an optimal com-
lþ1 l 0l
promise (Singh et al., 2021). Moreover, the 50–50 split attains a
hi ¼ hi þ hi ð32Þ favorable trade-off between the computational efficiency of model
l 0l execution and its overall performance (Trivedi et al., 2022). By ded-
where hi is the input to vi in the lth ARGCN layer and hi is the out- icating a significant portion of the data for training, valuable pat-
lþ1
put. Because of this, the input of the ARGCN’s (l + 1) th layer is hi . terns can be captured while still reserving a substantial subset
Boosting Contextual information a Bi-LSTM is incorporated into the for testing, thereby enabling an effective assessment of the model’s
input words to obtain their contextual information, and the result- proficiency with unseen instances (Krishna et al., 2021). Conse-
0
ing output hidden state hi is adopted as the initial representation hi quently, this partitioning is of utmost importance in comprehend-
of leaf node i. Next, we use the root’s intermediate hidden state, ing the model’s capacity to accurately handle novel, unobserved
0
represented by ha , to encode the aspect word in a second BiLSTM. data samples. Table 6 provides other statistics on datasets.
During Training Cross Entropy loss and l2-Regularization are used. These databases cover a wide range of topics, including people,
places, food, services, the natural world, entertainment, consumer
electronics, and more. Due to their expansive nature, these data-
3.3. Output and learning
sets cannot be reduced to the study of a single or even a small
number of disciplines. In every experiment, the dataset is prepro-
The output from both modules APA and EKDCG is connected
cessed, and all capital letters are changed to lowercase. Since the
through the dense layer and a fully connected softmax layer is used
datasets utilized in the experiments lacked a validation set, we
to map the probabilities of different sentiment polarities.
selected 10% of the training set’s data samples randomly as the val-
idation set to adjust the aforementioned hyperparameters. To
4. Experiment and analysis ensure the impartiality of the comparison experiments, we also
utilized this methodology. Furthermore, YAGO’s knowledge base
The experiment and the details related to the experimentation is heavily reliant on encyclopedic resources like Wikipedia, Word-
process are presented in this section. Net, and GeoNames. This research calls YAGO’s sentiment analysis
data ‘‘Entity-Oriented knowledge” to emphasize sentiment
4.1. Dataset polarity.

We used five publicly accessible NLP datasets containing the

English language to evaluate the proposed APEKCG model. MAMS, 4.2. Experiment settings
Laptop, Restaurant, AWARE, and SemEval-15&16 have been exten-
sively utilized in prior studies (Wang et al., 2020; Chen et al., Each dataset utilized in this study undergoes a process of data
2019a; Alturaief et al., 2021; Pontiki et al., 2016; Pontiki et al., cleansing, involving the removal of redundant, superfluous, or
2015). Each line in the MAMS dataset has two aspects with oppo- irrelevant information. In order to prepare textual data for analysis,
site sentiment polarities (Ben Veyseh et al., 2020; Chen et al., several functions offered by the NLTK python library are employed,
2022), making it a popular and difficult dataset for ABSA tasks. including Stop Word Removal, Stemming and Lemmatization, Nor-
Aspect terms and their polarity (Wang et al., 2020; Pablos et al., malization, and Tokenization. Stop-word removal eliminates
2015) have been tagged in more than 3 K English sentences in insignificant words and symbols while stemming and lemmatiza-
the Laptop dataset, which were annotated by skilled human anno- tion break down sentences into their constituent parts. Normaliza-
tators. Reviewers evaluate restaurants on various criteria such as tion ensures that sentences adhere to industry standards, and
meal quality, service, price, ambiance, and more, by employing tokenization segments longer strings into smaller, more manage-
coarse aspect categories of restaurants and annotating overall sen- able units. Text normalization plays a critical role due to the preva-
tence polarity (Pablos et al., 2015; Wan et al., 2020). AWARE is an lence of noise in online content and social media data, often taking
all-encompassing dataset that includes 11323 statements that con- the form of abbreviations, misspellings, and out-of-vocabulary
tain the three areas combined. Each sentence that is included in the words. To obtain a vector representation of text tokens, a pre-
datasets has at least one Aspect Category and one Aspect Senti- trained BERT word embedding is utilized. By undergoing pre-
ment marked upon it (Alturaief et al., 2021). SemEval-2015 training on extensive data, BERT acquires a comprehension of nat-
(Pontiki et al., 2016) and SemEval-2016 (Pontiki et al., 2015) data- ural language, which is subsequently fine-tuned for aspect-based
sets are the review data on the restaurant field. There is a lot of sentiment analysis. BERT generates word embeddings that exhibit
overlap between the SemEval-16 and SemEval-15 training data contextual awareness, wherein the representation of a word can
because they are both extensions of the SemEval-15 dataset, there- vary depending on its surrounding context. This capability empow-
fore we combined their training and testing data. Conflict informa- ers BERT to effectively handle polysemous words (words with mul-
tion is omitted from the original data sets. A line like ‘‘not a large tiple meanings) and extract more nuanced information from
place, but it’s cute and comfortable” would be categorized as hav- textual data.
ing a ‘‘conflict” sentiment because of the ‘‘ambiance” aspect cate- In this study, we use the following values for our experimental
gory. Only information marked as good, negative, or neutral is hyperparameters to make a fair comparison between our APEKCG
retained. Since there is insufficient ‘‘neutral” data in the and the reference models. The sentence tree is constructed with
SemEval-15&16 training set, models suffer greatly since the ‘‘neu- the Biaffine parser (Dozat and Manning, 2016) and Stanford Cor-
tral” category cannot be differentiated during training. To remedy eNLP parser (Manning et al., 2014) as dependency parsers. For
this, we extend the training set with four additional copies of the model training, we employ Adam (Kingma and Ba, 2014) as the
original ‘‘neutral” data. optimization function and initialize the word vector embedding
In each dataset, half of the documents from each author are with 300-dimensional BERT (Devlin et al., 2018) vectors. Overfit-
used to create the training set, while the remaining half is used ting can be avoided with the help of l2 - regularization, which is
to create the testing set. The utilization of a 50–50 split in data applied between the intervals of n as 0, 1105 , 1104 , 1103 ,
10
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Table 6
Dataset statistics.

Positive Positive Negative Negative Neutral Neutral

Dataset Train Test Train Test Train Test Aspect Categories
MAMS 1890 1890 1546 1546 2824 2824 8
Laptop 667 667 499 499 316 316 4
Resturant 1446 1446 501 501 416 416 4
AWARE 2656 2656 2645 2645 360 360 12
SemEval-15& 16 1214 1214 619 619 306 306 12

1102 . The learning rate is 1103 and it is halted based on the works) leverages the dependency tree and syntactic information
test set performance, Epoch 8, Batch size 16, Max sequence length about words to improve the task (Zhang et al., 2019a). CDT uses
64, and the dropout rate is 0.2. We give the average Macro-F1 and Bi-LSTM to construct word representations and then uses GCN to
accuracy across five distinct random seeds used in our trials. We’ve enhance embeddings based on the dependency tree (Sun et al.,
used the ‘‘Python 3.11” programming language on Windows 10 2019). SD-GCN (Dependencies on Emotion Sentiment Dependency
operating system for the evaluation of the proposed model. Relationships in Multi-Aspect Sentences Captured by GCNs) cap-
PyTorch was used on top of TensorFlow as the development frame- tures dependency relationships in multi-aspect sentences through
work for the code, and two GPUs are being utilized for the training GCNs (Zhao et al., 2020). R-GAT (Relational Graph Attention Net-
of the models (NVIDIA GeForce GTX 2080Ti). The parameters in the work) is an improved version of GAT that incrementally collects
baseline techniques are either used with their values from the orig- data. DualGCN combines a GNN with syntactic structures and
inal publication or many experiments are run to determine the semantic correlations (Li et al., 2021b). SenticGCN improves sen-
optimal values. In order to get the best results using BERT as an tence dependency graphs by using sentiment knowledge from Sen-
embedding layer model, the settings are adjusted to match those ticNet (Liang et al., 2022). dotGCN provides a different structure
suggested by the BERT developers. than explicit dependence trees (Chen et al., 2022).

4.3. Baseline models 4.4. Results analysis

We choose attention-based and syntax-based models as our We apply the global classification accuracy and the macro F1-
benchmarks to ensure that our APEKCG is properly validated. score as performance evaluation metrics to validate the perfor-
mance efficacy of APEKCG when the training datasets are consis-
4.3.1. Attention-based models tent. Table 7 compares the performance of APEKCG with other
The Target-Dependent LSTM (TD-LSTM) utilizes two TD-LSTMs attention-based models, whereas Table 8 compares its perfor-
to jointly learn target- and context-dependent word connections mance with syntax-based models across all five datasets.
(Tang et al., 2015). Meanwhile, the Attention-based LSTM Network Through a comprehensive examination of Table 7 and Table 8,
(ATAE) employs the attentional mechanism to reveal the polarity noteworthy insights can be gleaned from this research study. The
of the sentence’s sentiment based on the content of the sentence investigation involved a comparative analysis of diverse models
and its relationship to the worrying component (Wang et al., employed in ABSA, leading to the observation that models incorpo-
2016). Recurrent Attention on Memory (RAM) utilizes the rating dependency trees, namely R-GAT, GAT, SD-GCN, ASGCN,
multiple-attention mechanism to capture distant emotional char- LSTM + SynATT, and DT-GAT, outperform those relying solely on
acteristics (Chen et al., 2017). Interactive Attention Networks attention mechanisms. The findings underscore the significance
(IAN) are designed to learn attention in both contexts and targets of syntactic information in ABSA. However, attention-based mod-
to create autonomous representations for targets and contexts els such as IAN, MGAN, RAM, and BERT exhibit greater effective-
(Ma et al., 2017). The Multi-Grained Attention Network (MGAN) ness than RNN-based models like ATAE-LSTM and DTLSTM in
employs aspect alignment loss to depict aspect-level interactions capturing aspect-sentiment associations. Among the structured
between aspects that are in the same context (Fan et al., 2018). models, two prominent categories emerge as superior performers:
Finally, to monitor lexical representations in context and generate tree-LSTM-based tree neural networks (e.g., PhraseRNN,
supplementary text, ABSA uses BERT (Hoang et al., 2019). LSTM + SynATT) and GNNs enhanced with dependency information
Attention-based models for ABSA have limitations that include (e.g., R-GAT, GAT, SD-GCN, ASGCN, DT-GAT). These outcomes high-
contextual overemphasis, lack of global context, sensitivity to light the superiority of GNNs in tackling the ABSA task.
input order, vocabulary limitations, and interpretability challenges Furthermore, we delve into the examination of two additional
(Nguyen and Shirai, 2015Veličković et al., 2017). models, namely DualGCN and Sentic GCN, which incorporate a
combination of syntactic and sentiment knowledge in their com-
4.3.2. Syntax-based models putational processes. This innovative approach leads to a notable
This paper discusses various attention strategies for the ABSA improvement in the accuracy of sentiment prediction within these
task that consider syntactic information. SynATT (Syntax-based models. The empirical outcomes, as depicted in Table 8, substanti-
LSTM) proposes an attention strategy that takes into account syn- ate that the fused knowledge model outperforms the structured-
tactic information (Nguyen and Le Nguyen, 2018). PhraseRNN, or based modeling technique, thereby underscoring the significance
Phrase Recursive Neural Network, improves upon the RNN model and reliability of incorporating prior information in 8 sentiment
by taking into account the dependency and constituent trees of a analysis. In light of these findings, it becomes evident that the inte-
sentence (Nguyen and Shirai, 2015). Graph neural networks like gration of both syntactic and sentiment information is of para-
GAT (Graph Attention Networks) use masked self-attention to pro- mount importance in ensuring precise and reliable sentiment
cess graph-structured input (Chen et al., 2019a; Veličković et al., analysis outcomes.
2017). The Target-Dependent Graph Attention Network, TD-GAT, Moreover, our proposed APEKCG model, incorporating fused
explicitly mines the dependency link between words (Huang entity-oriented knowledge, surpasses the R-GAT model in terms
et al., 2020). AS-GCN (Aspect-Specific Graph Convolutional Net- of performance across all evaluated metrics. Previous research,
11
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

specifically, the study (Jain et al., 2021) has established the signif- Table 8
icance of entity-oriented information in determining the polarity Comparison with syntax-based models.

of users’ sentiments toward a particular product or service. Lever- Datasets Models Accuracy Macro-F1
aging this valuable insight, the APEKCG model integrates such MAMS CDT 80.70 79.79
entity-oriented knowledge, resulting in further improved out- R-GAT 81.75 80.87
comes in sentiment analysis tasks. dotGCN 85.95 84.44
Structured models that utilize syntactic trees, such as syntax Proposed APEKCG 89.13 87.56
Laptop SynATT 77.57 69.13
and dependency trees, are widely used to model GNNs in both GAT 73.04 68.11
structured and fused knowledge models. These include SynATT, ASGCN 75.55 71.05
TD-GAT, SD-GCN, DualGCN, and Sentic GCN. In contrast, serialized TD-GAT 74.13 72.01
models such as ATAE, DT-LSTM, RAM, IAN, and MGAN demonstrate CDT 77.19 72.99
SD-GCN 81.35 78.34
inferior performance in capturing sentiment. This highlights the
R-GAT 78.21 74.07
importance of structure-based models in sentiment analysis. The DualGCN 80.63 77.36
incorporation of syntactic and sentiment knowledge, along with Sentic GCN 81.35 77.90
entity-oriented knowledge, significantly enhances the accuracy of dotGCN 81.03 78.10
sentiment analysis models. Therefore, it is clear that the utilization Proposed APEKCG 84.32 80.59
Restaurant PhraseRNN 66.20 59.32
of these methods can greatly improve the effectiveness of senti-
SynATT 80.45 71.26
ment analysis techniques. GAT 78.21 67.17
Our APEKCG model outperforms all other models. APEKCG is a ASGCN 80.77 72.02
technique that combines aspect position attention learning and TD-GAT 80.35 76.13
CDT 82.30 74.02
entity-oriented knowledge dependency convolutional graph.
SD-GCN 83.57 76.47
Experimental results show that incorporating aspect attention R-GAT 86.60 81.35
learning, entity-oriented knowledge, dependency labels, and syn- DualGCN 86.77 81.62
tax path together can lead to improved performance in the ABSA Sentic GCN 86.94 81.62
task. Furthermore, the use of dependency data is effective in iden- dotGCN 86.15 80.37
Proposed APEKCG 89.02 83.95
tifying sentiment pairs, regardless of their proximity or relation-
AWARE GAT 71.67 70.13
ship to sentiment polarity. Nonetheless, this has been proven for ASGCN 72.15 70.40
a number of NLP tasks (Wu et al., 2021; Zhang et al., 2022c), using TD-GAT 72.68 71.15
the syntactic approach, entity-oriented knowledge, and data from CDT 74.66 73.66
R-GAT 76.15 74.88
dependency labels to better recover latent semantic information
DualGCN 76.04 74.91
influencing sentiment polarity. Also, using a pre-trained language Sentic GCN 76.22 74.90
model like BERT can improve the model’s ability to learn latent dotGCN 78.11 77.00
information. Proposed APEKCG 79.64 78.55
In addition, we have conducted a comparative analysis between SemEval-15& 16 AGAT 74.28 60.38
ASGCN 81.84 72.10
our proposed APEKCG model and the leading-edge models in the
TD-GAT 89.19 82.54
field. We have utilized the findings from the original research stud- CDT 88.15 80.38
ies, and our proposed model has exhibited superior performance in R-GAT 88.41 80.95
DualGCN 90.62 79.19
Sentic GCN 89.45 81.46
dotGCN 89.55 82.71
Table 7 Proposed APEKCG 90.22 85.66
Comparison with attention-based models.

Datasets Models Accuracy Macro-F1

MAMS BERT 82.82 81.90 terms of accuracy and F1 score when compared to the state-of-the-
Proposed APEKCG 89.13 87.56 art methodologies, shown in Table 9.
Laptop ATAE 68.70 - Moreover, we have evaluated our proposed model based on the
DT-LSTM 71.22 65.75
RAM 74.49 71.35
weighted F1 score, as it is a valuable evaluation metric that brings
IAN 72.10 - several advantages to model assessment. Firstly, it addresses the
MGAN 75.39 72.47 challenge of class imbalance by considering the distribution of
BERT 77.58 72.38 samples across classes, shown in Table 10. This ensures that the
Proposed APEKCG 84.32 80.59
evaluation is not biased towards the majority class and gives
Restaurant ATAE 77.20 -
DT-LSTM 79.10 69.00 appropriate weightage to each class based on their representation
RAM 80.23 70.80 in the dataset (Jangid et al., 2018). Secondly, the weighted F1 score
IAN 78.60 - provides a holistic measure of model performance by combining
MGAN 81.25 71.94 precision and recall for each class. It takes into account both the
BERT 83.62 78.28
Proposed APEKCG 89.02 83.95
positive and negative predictive values, giving a comprehensive
AWARE DT-LSTM 69.51 67.98 assessment of the model’s ability to correctly classify instances
RAM 69.36 67.30 from all classes. Additionally, the weighted F1 score reflects the
MGAN 72.54 70.81 practical importance of different classes. By assigning higher
BERT 75.28 74.11
weights to classes with larger sample sizes, it highlights the perfor-
Proposed APEKCG 79.64 78.55
SemEval-15& 16 ATAE 77.15 62.70 mance of classes that may have greater significance in the specific
DT-LSTM 80.79 70.43 application or problem domain. Additionally, the weighted F1
RAM 76.17 64.33 score allows for fair comparisons across models or variations of
IAN 74.28 60.38 the same model. It provides a consistent evaluation metric that
MGAN 81.84 72.10
accounts for class distribution, ensuring that models are assessed
BERT 89.19 82.54
Proposed APEKCG 90.22 85.66 on an equal footing (De Greve et al., 2021).

12
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

We have used McNemar’s non-parametric test to compare the the impact of contextual semantic information on emotional expres-
results of several approaches to see if there was a statistically sig- sion. Representing the explicit relational semantics of sentiment
nificant difference in performance (Dietterich, 1998). This test has pairs more accurately, entity-oriented knowledge graphs offer a
been used in similar investigations and does not assume normally valuable advantage. Moreover, neglecting this aspect leads to a
distributed data, making it ideal for our objectives (Chen et al., decrease in the accuracy rate, suggesting that domain-specific infor-
2019b). Specifically, using McNemar’s test, we need to keep track mation enhances sentiment analysis. In contrast to implicitly mod-
of how many samples are correctly categorized as A rather than eling the proposed APEKCG model, incorporating dependency labels
B (denoted as n10 in Eq. 33) and and how many are correctly cate- enables the explicit modeling of APEKCG and facilitates the integra-
gorized as B rather than A (denoted as n01 in Eq. 33). We can then tion of its outcomes with other data in a high-dimensional vector
calculate the statistical significance: space. Lastly, the syntactic approach allows for more efficient
extraction of collocation information between words, particularly
ðjn01  n10 j  1Þ2 when different words with varying degrees of polarity are employed
x2 ¼ ð33Þ
n01 þ n10 to modify distinct sentiment elements.
This has a 2 distribution with one parameter. Only if the p-value
4.6. Discussion
of the computed statistic falls below a predefined threshold is
there evidence of a statistically significant difference in perfor-
In our study, we developed a model called APEKCG, which com-
mance. Table 11 and Table 12 display the statistical data; we chose
prised two distinct modules. Each module was constructed inde-
a 5% level of significance. It is clear from the results that our sug-
pendently, and subsequently, their outputs were fused together
gested APEKCG model outperforms the baseline approaches on
to generate the final output of the model. To assess the impact of
all five datasets. Table 11 and Table 12 present the p-value that
various factors, such as different parsers, GCN layers, suggested
emerges from McNemar’s test results on all five datasets.
parameter settings, and other relevant variables, on the perfor-
Table 11 and Table 12 clearly demonstrate that the proposed
mance of our APEKCG model, we conducted a series of four com-
model APEKCG, when compared to each baseline model, exhibits
prehensive tests. These tests were designed to investigate and
a p-value below the threshold value of 0.05. As a result, we can
evaluate the influence of these factors on the overall effectiveness
reject the null hypothesis and conclude that the performance of
and functionality of the APEKCG model.
the two models differs significantly. Notably, APEKCG, with its
higher mean accuracy, has significantly outperformed the applied
4.6.1. Impact of using various parsers
baselines.
We tested our model with two popular dependency parsers (i.e.,
Biaffine Parser, and Stanford CoreNLP Parser (Chen and Manning,
4.5. Ablation studies 2014)) to confirm the effect of dependent parsers on the ABSA
job. Consistent with other research (Inui et al., 2019; Qi et al.,
Additionally, we conducted ablation studies to further validate 2020), Table 14 experimental results show that ABSA task perfor-
the impact of individual components within the APEKCG frame- mance varies between dependency parsers. Biaffine outperforms
work. The findings of these studies are presented in Table 13, CoreNLP at discovering interdependencies between sentiment ele-
where the abbreviation ‘‘w/o” represents ‘‘without.” Notably, ments and polarity. Enhancing the efficiency of the dependency
Table 13 provides empirical evidence illustrating the substantial parser can result in a more precise ABSA task performance, espe-
decline in accuracy of the APEKCG model when crucial elements cially when working with complex textual information like senti-
such as APA, EKG, BERT, Bi-LSTM, dependency labels, entity- ment analysis, particularly in the case of challenging datasets. In
oriented KG, syntax route, and position embedding module are other words, by improving the efficiency of the dependency parser,
removed. This observation strongly suggests that the utilization it is possible to enhance the accuracy and effectiveness of senti-
of all the information incorporated in this research can signifi- ment analysis techniques, particularly when dealing with difficult
cantly enhance the accuracy of the APEKCG model in addressing datasets that may contain complex and nuanced information.
the ABSA task, as represented in Fig. 2. Remarkably, the most dras- Table 14 shows the performance of several parser tools.
tic decrease in accuracy occurs with the removal of BERT, followed APEKCG + Biaffine shows superior performance on all datasets.
by dependency labels, syntactic route, Entity-oriented Knowledge
Graph, Bi-LSTM, and finally the position embedding module. 4.6.2. Layers of GCN’s Effect
Drawing upon previous research (Li et al., 2022c; Ma et al., 2018), In our study, we employed several models, namely BERT, SD-
it can be inferred that the Bi-LSTM model possesses the capability to GCN, dotGCN, and APEKCG, to analyze the Restaurant dataset.
effectively capture the semantic context of words. This observation The primary objective was to examine whether the number of
further highlights the significant role of contextual semantic infor- GCN layers had an impact on accuracy. As depicted in Fig. 3, the
mation in influencing the expression of emotions. In simpler terms, most favorable outcomes were achieved by the three CNNs (SD-
the Bi-LSTM model can be regarded as a powerful tool for analyzing GCN, dotGCN, and APEKCG) that incorporated both dependency

Table 9
Evaluations based on state-of-the-art methods.

Datasets Laptop Restaurant MAMS

Method Accuracy Macro-F1 Accuracy Macro-F1 Accuracy Macro-F1
MWGCN (Yu and Zhang, 2023) 79.78 76.68 86.36 80.54 - -
MultiGCN (Ma et al., 2023) 78.8 74.97 83.82 77.1 83.61 82.73
RGAT (Wu et al., 2022b) 78.02 74 83.55 75.99 81.75 80.87
SSK-GCN + BERT (Zhao et al., 2023a) 83.11 79.63 88.96 83.44 - -
SA-BERT (Mewada and Dewang, 2023) 83.21 78.77 87.24 81.19 - -
SA-BERT-XGBoost (Mewada and Dewang, 2023) 85.01 78.9 87.86 81.64 - -
CLF-TrLSTM + Bert (Zhang et al., 2023) 82.99 79.49 88.33 83.08 - -
Proposed APEKCG 84.32 80.59 89.02 83.95 89.13 87.56

13
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Table 10
Evaluations of the proposed APEKCG based on weighted f1 score.

Datasets Class Precision Recall F1 Score Samples Weight Weighted F1

MAMS Positive 0.8941 0.842 0.8673 1890 0.3 0.2602
Negative 0.7850 0.7987 0.7918 1546 0.25 0.1979
Neutral 0.9000 0.8750 0.8873 2824 0.45 0.3993
Total 6260 1 0.8574
Laptop Positive 0.8348 0.8400 0.8374 667 0.45 0.3768
Negative 0.8262 0.8025 0.8142 499 0.34 0.2768
Neutral 0.8260 0.8947 0.8590 316 0.21 0.1804
Total 1482 1 0.8340
Restaurant Positive 0.9350 0.8782 0.9057 1446 0.61 0.5525
Negative 0.8320 0.7250 0.7748 501 0.21 0.1627
Neutral 0.9010 0.7963 0.8454 416 0.18 0.1522
Total 2363 1 0.8674
AWARE Positive 0.8148 0.7482 0.7801 2656 0.47 0.3666
Negative 0.8100 0.7147 0.7594 2645 0.47 0.3569
Neutral 0.8630 0.7412 0.7975 360 0.06 0.0478
Total 5661 1 0.7714
SemEval-15&16 Positive 0.9486 0.8145 0.8765 1214 0.57 0.4996
Negative 0.8536 0.7892 0.8201 619 0.29 0.2378
Neutral 0.9450 0.8420 0.8905 306 0.14 0.1247
Total 2139 1 0.8621

Table 11
Comparison on the basis of McNemar’s statistics on MAMS, Laptop, and Restaurant datasets.

Dataset MAMS MAMS Laptop Laptop Restaurant Restaurant

Method x2 p x2 p x2 p
ATAE - - 59.172* 0 882.049* 0
DT-LSTM - - 60.500* 0 761.274* 0
RAM - - 77.778* 0 333.638* 0
IAN - - 79.587* 0 223.017* 0
MGAN - - 52.112* 0 323.218* 0
BERT 4.696* 0.03 5.470* 0 9.339* 0.002
SynATT - - 8.817* 0.019 292.638* 0
GAT - - 4.516* 0.034 277.095* 0
ASGCN - - 0.329 0.566 - -
TD-GAT - - 0.011 0.915 3.556 0.059
CDT 0.404 0.525 57.309* 0 963.628* 0
SD-GCN - - 55.249* 0 835.706* 0
R-GAT 0.012 0.914 72.901* 0 350.216 0
DualGCN - - 71.405* 0 255.314* 0
Sentic GCN - - 49.091* 0 368.057* 0
dotGCN 0.085 0.771 4.938* 0.026 22.830* 0

*This implies that the outcome bears statistical significance at a significance level of 5%.

Table 12
Comparison on the basis of McNemar’s statistics on AWARE and SemEval-15 & 16 datasets.

Dataset AWARE AWARE SemEval-15&16 SemEval-15&16

Method x2 p x2 p
ATAE - - 106.313* 0
DT-LSTM 364.618* 0 95.508* 0
RAM 354.211* 0 54.179* 0
IAN - - 136.172* 0
MGAN 18.817* 0.003 45.039* 0
BERT 14.516* 0.034 0.736 0.391
SynATT - - 0.761 0.097
GAT 18.618* 0 0.456 0.5
ASGCN 18.618* 0.004 2.913 0.088
TD-GAT 8.028* 0.005 1.432 0.231
CDT 0.059 0.838 118.534* 0
SD-GCN - - 71.969* 0
R-GAT 0.48 0.488 129.063* 0
DualGCN 18.618* 0 160.507* 0
Sentic GCN 8.028* 0.005 64.574* 0
dotGCN 11.653* 0.001 0.48 0.488

*This implies that the outcome bears statistical significance at a significance level of 5%.

and contextual information. This suggests that GNNs exhibit supe- Furthermore, our research revealed that, except for two
rior information capture capabilities compared to sequential mod- instances, the accuracy of GNNs did not demonstrate improvement
els like BERT. with an increase or decrease in the number of layers. These

14
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Table 13
Ablation Study of the proposed APEKCG model.

Model MAMS Laptop Restaurant AWARE SemEval-15&16

1 Proposed APEKCG 89.13 84.32 89.02 79.64 90.22
2 w/o APA 85.34 80.66 86.32 78.23 86.45
3 w/o EKG 86.74 81.32 87.01 77.64 88.22
4 w/o BERT 85.33 80.11 85.29 76.43 87.37
5 w/o LSTM 82.19 81.19 86.38 77.22 88.37
6 w/o Dependency Label 80.32 79.95 86.10 76.86 87.24
7 w/o KG 81.76 80.96 87.17 76.48 87.29
8 w/o Syntax Path 81.45 80.21 86.96 76.30 87.88
9 w/o position embedding module 81.23 80.13 86.02 76.03 87.12

Fig. 2. Representation of ablation Study of the proposed APEKCG model.

findings align with previous studies (Nadeem et al., 2022), indicat- the model using the determined value of, and finally, we select
ing that augmenting the number of GCN layers enhances accuracy the best possible outcome for further examination.
in the ABSA task. As a result, future investigations will continue to
delve into the influence of the number of GCN layers on accuracy. 4.6.4. Effects of several aspects
Additionally, we plan to explore other factors that may influence The ABSA task involves different datasets with numerous aspect
model performance, such as the selection of specific layers and terms in each sentence. It is important to investigate whether this
the utilization of different types of data. These endeavors will con- phenomenon has an impact on the effectiveness of our APEKCG
tribute to a deeper understanding of the factors affecting model model. Based on sentence aspect terms, we evaluated the training
performance and guide the development of more robust and accu- accuracy difference between the Restaurant and MAMS datasets.
rate sentiment analysis models. We excluded samples with more than seven aspects because the
sample size of such objects is too small for meaningful comparison,
4.6.3. Influence of the suggested parameter as previously noted (Li et al., 2019). In Fig. 5, we observe that the
In module 1, we have observed that We advocate using a wide accuracy of our APEKCG model varies when there are more than
range in tests for the location embedding rate, which fluctuates three sentiment components in a text. This shows that the current
with text length, so we recommend using a wide range in trials model may not successfully preserve the dependencies of multiple
and big step size to choose candidate values and validate the opti- sentiment components in a phrase and that new models will be
mal value. You can see how modifying the value of affected our needed in the future to address this difficulty. Understanding the
models’ performance in Fig. 4. We use classification accuracy and impact of the number of aspect terms in a sentence is critical to
the macro F1-score to gauge the model’s effectiveness. We train improving the performance of ABSA models. This analysis can help

Table 14
Results the proposed APEKCG model on Stanford CoreNLP and Biaffine Parser.

Parser MAMS Laptop Restaurant AWARE SemEval-15&16

CoreNLP 88.33 81.89 86.33 76.89 88.62
Biaffine 89.13 84.32 89.02 79.64 90.22

15
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

In contrast to dotGCN, which only alters the distance between

words, these models perform better when there are many emotion
pairs. When there is only one possible emotional reaction, as in the
fourth case, the statement is not optimized. On the other hand, our
APEKCG model combines BERT, syntax route, and Bi-LSTM to effec-
tively capture the possible rules for how words can be put
together, especially when it comes to sentiment terms, as well as
the whole meaning of the sentence.
Initially, it is important to note that there are instances where
sentences consist of a single aspect category. Such sentences often
feature strong emotional expressions, as demonstrated by the term
‘‘uncomfortable” in Example 3. In these cases, all the models in
comparison exhibit a consistent ability to accurately determine
the sentiment direction conveyed within these sentences. How-
ever, when confronted with scenarios such as Example 5, where
the polarity of certain aspects is more nuanced, the performance
Fig. 3. Impact of GCN layers on accuracy. of the ATAE model tends to be subpar in comparison to our pur-
posed APEKCG model. This observation implies that while the
ATAE model may struggle in cases where the polarity of specific
researchers identify potential limitations in their models and aspects is more complex, the proposed APEKCG model displays
develop more accurate methods for capturing sentiment compo- greater robustness and effectiveness. The APEKCG model’s ability
nents in a sentence. By developing more sophisticated models that to leverage the comprehensive knowledge graph representation
can effectively capture the dependencies between different senti- of aspect-property-entity relationships allows for a more nuanced
ment components, we can enhance the accuracy of sentiment anal- understanding of sentiment, particularly when encountering sen-
ysis and provide more useful insights into the sentiment expressed tences with intricate polarity dynamics. By capturing and incorpo-
in texts. rating this extensive semantic information, the APEKCG model
demonstrates superior performance and accuracy in sentiment
4.7. Case study analysis tasks compared to the ATAE model.

A related case study has been presented in this section. 4.7.2. Attention visualization
To further confirm the efficacy of our APEKCG model’s individ-
4.7.1. Qualitative evaluation ual parts (like BERT, syntax path, dependency labels, and entity-
In this section, we exhibit the prediction results of the proposed oriented knowledge, aspect location attention). In this example,
models by selecting some qualitative examples at random from the we pull the text ‘‘the cake is delicious and the price is affordable”
test data of SemEval-15 & 16 datasets. Table 15 presents a number at random from Restaurant and display it in a new window using
of cases that were analyzed using various models, such as dotGCN, the attention method. Fusion of aspect position module, BERT,
SD-GCN, SenticGCN, and ATAE. In these cases, the letters N y ; Oy , entity-oriented knowledge, and dependency labels was used to
and Py were used to indicate the presence of negative, neutral, alter the original text. We see that the sentence’s focus might be
and positive emotions, respectively, and the letters N n ; On , and P n split between two different aspect categories. Attention is like a
were used to indicate the absence of negative, neutral, and positive gradient, with darker shades indicating a more focused study.
emotions, respectively. In Table 15, We have enclosed some words Hence, the darker the hue, the more weight the word in that sec-
in square brackets to highlight their distinctive emotional tion carries. The successful identification of emotional information
connotations. pertaining to specific aspect categories by APEKCG indicates that it
An illustrative example can be observed in the first instance, is in line with human judgment on semantic emotion in natural
where the term ‘‘definitely” is utilized. This word carries a virtual language. In other words, the technique is able to accurately iden-
hypothetical tone that contradicts the captured affective polarity, tify emotional information in relation to specific aspects, in a way
thereby introducing complexity in sentiment analysis. These find- that is comparable to how humans perceive and interpret semantic
ings underscore the significance of not only examining the proxim- emotions in language. This highlights the potential value of
ity of words but also taking into account the broader semantic APEKCG in accurately identifying and analyzing emotional infor-
context when conducting sentiment analysis. The inclusion of mation within natural language, particularly in the context of sen-
words with nuanced meanings, such as ‘‘definitely,” highlights timent analysis. As per Example 2, the line ‘‘The restaurant isn’t
the necessity of a comprehensive approach that considers the over- very big but we got the table right away” does not contain the
all tone and contextual information in order to accurately interpret given aspect-categories ‘‘service” or ‘‘ambiance” in direct terms.
and analyze sentiments. The second sentence example presented This is a significant challenge since the machine cannot grasp the
highlights the presence of the term ‘‘good” in close proximity, gist of the statement in the same way that a human can. So, we
thereby eliciting attentional mechanisms in models such as ATAE. can see that our model is successful, at least to some extent, in
Although dependency relationships can effectively capture pairs of understanding the gist of the text’s meaning. As can be observed
sentiment items and their corresponding sentiment polarity, it is in Fig. 6 and Fig. 7, APEKCG also pays little attention to the prepo-
important to acknowledge that they are still influenced by the sitions and punctuation marks in the text. That follows the
semantic information encompassing the entire sentence. In other accepted reasoning for determining the positive or negative nature
words, the interpretation of sentiment is not solely based on the of an expression of emotion: we pay no attention to these everyday
individual relationships between words but also takes into consid- terms. APEKCG pays close attention to and places a substantial
eration the broader context and meaning conveyed by the sentence emphasis on, words that are both relevant to aspect categories
as a whole. and typical of the sentiment being judged. As a result, we draw
To improve the capture of sentiment pairings, SenticGCN inte- the conclusion that our APEKCG attention mechanism successfully
grates sentiment knowledge based on dependency tree modeling. models the text’s overall semantics.
16
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Fig. 4. Impact of different values parameter.

Our proposed model is specifically crafted to effectively incorpo-

rate aspect categories alongside contextual information. To
enhance aspect-specific sentiment analysis, we employed a
boosted attention network coupled with aspect-position embed-
ding. Furthermore, we integrate entity-oriented knowledge into
our model’s dependency tree structure by leveraging an Entity-
oriented Knowledge Dependency Convolutional Graph. This incor-
poration facilitated the modeling of both implicit and explicit sen-
timent, as well as the analysis of single statements encompassing
multiple emotions.
Diverging from prior investigations that predominantly exam-
ined general consumer sentiment, our study adopted a distinct
approach by directing attention toward the identification and
analysis of sentiment pertaining to particular aspects that hold
significance for consumers. We acknowledged the inherent vari-
ability in the importance attributed to various facets of the cus-
Fig. 5. Impact of number of aspects on accuracy. tomer experience, recognizing that certain aspects warrant
greater attention than others. Through the integration of ABSA
into our predictive modeling framework, we were able to offer
5. Theoretical and practical implications businesses a comprehensive and practical assessment of senti-
ment for each individual aspect. This enhanced level of granular-
In our research, we presented a novel framework for Aspect- ity empowered businesses to allocate their resources in a more
Based Sentiment Analysis that centers on the notion of information targeted and efficient manner, enabling them to make informed
relevance. The proposed framework is meticulously developed decisions based on the specific sentiments associated with each
through a comprehensive review of relevant literature in the field. aspect.

Table 15
Comparisons of the predictions based on examples.

Text dotGCN SD-GCN SenticGCN ATAE APEKCG

Definitely try [calamari], any [pasta], or even the [sliced steak] P y ; Oy ; N y Py ; Nn ; Ny Oy ; Oy ; N y P y ; N n ; Oy P y ; Oy P y
The [food] is very good but the [location] is too far Py ; Pn Py ; Ny Py ; Ny Py ; Ny Py ; Ny
The folding chair I was searing at was very [uncomfortable]. Ny Ny Ny Ny Ny
The staff should be a bit more [friendly]. Pn Ny pn Pn Ny
I wish I had a [webcam] though, it would be [perfect]. Pn ; Py Pn ; Py On ; Oy Ny ; Nn Ny ; Py

17
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Fig. 6. Case 1- the implicit aspect-categories.

Fig. 7. Case 2- the explicit aspect-categories.

The pragmatic ramifications of our research underscore the sig- Restaurant, AWARE, and SemEval 15 & 16). In addition, we show
nificance of businesses engaging in active monitoring of online that Entity-Oriented knowledge, syntax route, and the dependency
feedback as a means to bolster their operational effectiveness labels of the EKDCG model can improve the model’s accuracy by
and long-term viability. Through the utilization of our proposed conducting Ablation Studies and a Case Study. Furthermore, the
model, businesses gain the ability to discern and prioritize the piv- visual representations of attention weights demonstrate APEKCG’s
otal factors that exert influence on their performance, thus ability to sensibly focus on the unique information in the input
enabling them to allocate resources in an optimized manner. By text, which is crucial for determining the polarity of sentences’
adopting this approach, businesses can enhance their decision- sentiments. Moving forward, we aim to incorporate neurosymbolic
making processes, leading to favorable outcomes and advance- AI to enhance explainable sentiment analysis, as well as integrate
ments in the quality and delivery of their products and services. additional types of knowledge, such as syntax, semantic, and infer-
ence knowledge, into the ABSA task. One key challenge we intend
6. Conclusions and future work to address is the issue of neutrality/ambivalence, which can be dif-
ficult to capture accurately with traditional sentiment analysis
This research suggested a novel model APEKCG, which con- methods. To achieve this, we plan to develop and test several syn-
sisted of two modules APA and EKDCG. For the ABSA task, we sug- tactic models to explore the impact of syntactic parsing tools on
gest using an APA module to store information in a neural model accuracy and resilience. This will allow us to better under-
network’s memory using an embedding of aspects and an attention stand the role of syntactic information in sentiment analysis and
mechanism based on aspects. New attention models (APAs) can be potentially develop more effective models for capturing nuanced
created by combining aspect-position embedding with the concept sentiment. By incorporating multiple sources of knowledge and
of attention, with the latter giving additional weight to the seman- exploring new modeling techniques, we hope to improve the accu-
tic relationship between words within the sequence. Our second racy and reliability of ABSA models. This will enable us to better
module EKDCG uses reliance labels and syntactic paths to take into understand the sentiment expressed in texts and provide more
account domain knowledge (such as entity oriented) in addition to insightful and meaningful analyses to users. Ultimately, our goal
traditional measures of dependency. The experimental findings on is to develop more advanced and sophisticated ABSA models that
five benchmark datasets demonstrate the superior accuracy of our can accurately capture and analyze the full spectrum of sentiment
proposed APEKCG model for the ABSA task (i.e., MAMS, Laptop, in texts.

18
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Declaration of Competing Interest Inui, K., Jiang, J., Ng, V., Wan, X., 2019, November. Proceedings of the 2019
Conference on Empirical Methods in Natural Language Processing and the 9th
International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
The authors declare that they have no known competing finan- In Proceedings of the 2019 Conference on Empirical Methods in Natural
cial interests or personal relationships that could have appeared Language Processing and the 9th International Joint Conference on Natural
Language Processing (EMNLP-IJCNLP).
to influence the work reported in this paper.
Fan, F., Feng, Y., Zhao, D., 2018. Multi-grained attention network for aspect-level
sentiment classification. In: Proceedings of the 2018 Conference on Empirical
Acknowledgments Methods in Natural Language Processing, pp. 3433–3442.
Han, J., Zhang, Z., Cummins, N., Schuller, B., 2019. Adversarial training in affective
computing and sentiment analysis: Recent advances and perspectives. IEEE
Princess Nourah bint Abdulrahman University Researchers Sup- Comput. Intell. Mag. 14 (2), 68–81.
porting Project number (PNURSP2023TR140), Princess Nourah bint Hoang, M., Bihorac, O.A., Rouces, J., 2019. Aspect-based sentiment analysis using
bert. In: Proceedings of the 22nd Nordic Conference on Computational
Abdulrahman University, Riyadh, Saudi Arabia. Linguistics, pp. 187–196.
Hou, X., Huang, J., Wang, G., He, X., Zhou, B., 2019. Selective attention based graph
convolutional networks for aspect-level sentiment classification. arXiv preprint
References arXiv:1910.10857.
Hou, X., Huang, J., Wang, G., Qi, P., He, X., Zhou, B., 2021. Selective attention based
Agichtein, E., Gravano, L., 2000, June. Snowball: Extracting relations from large graph convolutional networks for aspect-level sentiment classification. 83–93.
plain-text collections. In: Proceedings of the fifth ACM conference on Digital arXiv:1910.10857.
libraries, pp. 85–94. Huang, B., Carley, K.M., 2019. Parameterized convolutional neural networks for
Alturaief, N., Aljamaan, H., Baslyman, M., 2021. AWARE: Aspect-Based Sentiment aspect level sentiment classification. arXiv preprint arXiv:1909.06276.
Analysis Dataset of Apps Reviews for Requirements Elicitation. In: 2021 36th Huang, L., Sun, X., Li, S., Zhang, L., Wang, H., 2020, December. Syntax-aware graph
IEEE/ACM International Conference on Automated Software Engineering attention network for aspect-level sentiment classification. In: Proceedings of
Workshops (ASEW). IEEE, pp. 211–218. November. the 28th International Conference on Computational Linguistics, pp. 799–810.
Arumugam, C., Nallaperumal, K., 2023. EIAASG: Emotional Intensive Adaptive Jain, P.K., Pamula, R., Srivastava, G., 2021. A systematic literature review on machine
Aspect-Specific GCN for sentiment classification. Knowl.-Based Syst. 260, learning applications for consumer sentiment analysis using online reviews.
110149. https://doi.org/10.1016/j.knosys.2022.110149. Comput. Sci. Rev. 41, 100413.
Behera, R.K., Jena, M., Rath, S.K., Misra, S., 2021. Co-LSTM: Convolutional LSTM Jain, D.K., Boyapati, P., Venkatesh, J., Prakash, M., 2022. An intelligent cognitive-
model for sentiment analysis in social big data. Informat. Process. Manage. 58 inspired computing with big data analytics framework for sentiment analysis
(1), 102435. and classification. Informat. Process. Manage. 59 (1). Article 102758.
Ben Veyseh, A.P., Nouri, N., Dernoncourt, F., Tran, Q.H., Dou, D., Nguyen, T.H., 2020. Jang, E., Gu, S., Poole, B., 2016. Categorical reparameterization with gumbel-
Improving aspect-based sentiment analysis with gated graph convolutional softmax. arXiv preprint arXiv:1611.01144.
networks and syntax-based regulation. In: Find. Assoc. Comput. Linguist. Find. Jangid, Hitkul, Singhal, Shivangi, Shah, Rajiv Ratn, Zimmermann, Roger, 2018.
ACL EMNLP 2020, pp 4543–4548. Aspect-based financial sentiment analysis using deep learning. In: Companion
Bunescu, R., Pasca, M., 2006. Using encyclopedic knowledge for named entity Proceedings of the The Web Conference 2018, pp. 1961–1966.
disambiguation. Karimi, A., Rossi, L., Prati, A., 2020 Improving BERT performance for aspect-based
Cambria, E., Olsher, D., Rajagopal, D., 2014, June. SenticNet 3: a common and sentiment analysis. arXiv:2010.11731.
common-sense knowledge base for cognition-driven sentiment analysis. In: Kingma, D.P., Ba, J., 2014. Adam: A method for stochastic optimization. arXiv
Proceedings of the AAAI conference on artificial intelligence, vol. 28, No. 1. preprint arXiv:1412.6980.
Cambria, E., Das, D., Bandyopadhyay, S., Feraco, A., 2017. Affective computing and Krishna, S., Gupta, R., Dupuy, C., 2021. ADePT: Auto-encoder based differentially
sentiment analysis. A practical guide to sentiment analysis, 1–10. private text transformation. arXiv preprint arXiv:2102.01502.
Cambria, E., Poria, S., Hussain, A., Liu, B., 2019. Computational intelligence for Liang, B., Su, H., Yin, R., Gui, L., Yang, M., Zhao, Q., et al., 2021, November. Beta
affective computing and sentiment analysis [guest editorial]. IEEE Comput. distribution guided aspect-aware graph for aspect category sentiment analysis
Intell. Mag. 14 (2), 16–17. with affective knowledge. In: Proceedings of the 2021 Conference on Empirical
Cambria, E., Liu, Q., Decherchi, S., Xing, F., Kwok, K., 2022, June. SenticNet 7: A Methods in Natural Language Processing, pp. 208–218.
commonsense-based neurosymbolic AI framework for explainable sentiment Liang, Y., Meng, F., Zhang, J., Chen, Y., Xu, J., Zhou, J., 2021. A dependency syntactic
analysis. In: Proceedings of the Thirteenth Language Resources and Evaluation knowledge augmented interactive architecture for end-to-end aspect-based
Conference, pp. 3829–3839. sentiment analysis. Neurocomputing 454, 291–302.
Chen, D., Manning, C.D., 2014, October. A fast and accurate dependency parser using Liang, B., Su, H., Gui, L., Cambria, E., Xu, R., 2022. Aspect-based sentiment analysis
neural networks. In: Proceedings of the 2014 Conference on Empirical Methods via affective knowledge enhanced graph convolutional networks. Knowl.-Based
in Natural Language Processing (EMNLP), pp. 740–750. Syst. 235, 107643.
Chen, P., Sun, Z., Bing, L., Yang, W., 2017, September. Recurrent attention network Liang, B., Li, X., Gui, L., Fu, Y., He, Y., Yang, M., Xu, R., 2023. Few-shot aspect
on memory for aspect sentiment analysis. In: Proceedings of the 2017 category sentiment analysis via meta-learning. ACM Trans. Informat. Syst. 41
Conference on Empirical Methods in Natural Language Processing, pp. 452–461. (1), 1–31.
Chen, J., Hou, H., Ji, Y., Gao, J., Bai, T., 2019a. Graph-based attention networks for Li, X., Bing, L., Zhang, W., Lam, W., 2019. Exploiting BERT for end-to-end aspect-
aspect level sentiment analysis. In: 2019 IEEE 31st International Conference on based sentiment analysis. arXiv preprint arXiv:1910.00883.
Tools with Artificial Intelligence (ICTAI). IEEE, pp. 1188–1194. November. Li, Q., Gkoumas, D., Lioma, C., Melucci, M., 2021a. Quantum-inspired multimodal
Chen, Z., Cao, Y., Lu, X., Mei, Q., Liu, X., 2019, August. Sentimoji: an emoji-powered fusion for video sentiment analysis. Informat. Fusion 65, 58–71.
learning approach for sentiment analysis in software engineering. In: Li, R., Chen, H., Feng, F., Ma, Z., Wang, X., Hovy, E., 2021, August. Dual graph
Proceedings of the 2019 27th ACM Joint Meeting on European Software convolutional networks for aspect-based sentiment analysis. In: Proceedings of
Engineering Conference and Symposium on the Foundations of Software the 59th Annual Meeting of the Association for Computational Linguistics and
Engineering, pp. 841–852. the 11th International Joint Conference on Natural Language Processing
Chen, C., Teng, Z., Wang, Z., Zhang, Y., 2022, May. Discrete opinion tree induction for (Volume 1: Long Papers), pp. 6319–6329.
aspect-based sentiment analysis. In: Proceedings of the 60th Annual Meeting of Li, H., Chen, Q., Zhong, Z., Gong, R., Han, G., 2022a. E-word of mouth sentiment
the Association for Computational Linguistics (Volume 1: Long Papers), pp. analysis for user behavior studies. Informat. Process. Manage. 59 (1), 102784.
2051–2064. Li, W., Shao, W., Ji, S., Cambria, E., 2022b. BiERU: Bidirectional emotional recurrent
Dai, A., Hu, X., Nie, J., Chen, J., 2022. Learning from word semantics to sentence unit for conversational sentiment analysis. Neurocomputing 467, 73–82.
syntax by graph convolutional networks for aspect-based sentiment analysis. Li, D., Ahmed, K., Zheng, Z., Mohsan, S.A.H., Alsharif, M.H., Hadjouni, M., Mostafa, S.
Int. J. Data Sci. Anal. 14 (1), 17–26. M., 2022c. Roman Urdu Sentiment Analysis Using Transfer Learning. Appl. Sci.
De Greve, L., Singh, P., Van Hee, C., Lefever, E., Martens, G., 2021. Aspect-based 12 (20), 10344.
sentiment analysis for German: analyzing’talk of literature’surrounding literary Lin, H.C.K., Wang, T.H., Lin, G.C., Cheng, S.C., Chen, H.R., Huang, Y.M., 2020. Applying
prizes on social media. Comput. Linguist. Netherlands J. 11, 85–104. sentiment analysis to automatically classify consumer comments concerning
De Marneffe, M.C., Manning, C.D., 2008, August. The Stanford typed dependencies marketing 4Cs aspects. Appl. Soft Comput. 97, 106755.
representation. In: Coling 2008: Proceedings of the Workshop on Cross- Liu, W., Zhou, P., Zhao, Z., Wang, Z., Ju, Q., Deng, H., Wang, P., 2020, April. K-bert:
framework and Cross-domain Parser Evaluation, pp. 1–8. Enabling language representation with knowledge graph. In: Proceedings of the
Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018. Bert: Pre-training of deep AAAI Conference on Artificial Intelligence, Vol. 34, No. 03, pp. 2901–2908.
bidirectional transformers for language understanding. arXiv preprint Luo, H., Ji, L., Li, T., Duan, N., Jiang, D., 2020. GRACE: Gradient harmonized and
arXiv:1810.04805. cascaded labeling for aspect-based sentiment analysis. Find Assoc Comput
Dietterich, T.G., 1998. Approximate statistical tests for comparing supervised Linguist Find ACL EMNLP 2020, 54–64.
classification learning algorithms. Neural Comput. 10 (7), 1895–1923. Ma, D., Li, S., Zhang, X., Wang, H., 2017. Interactive attention networks for aspect-
Dozat, T., Manning, C.D., 2016. Deep biaffine attention for neural dependency level sentiment classification. arXiv preprint arXiv:1709.00893.
parsing. arXiv preprint arXiv:1611.01734.

19
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Ma, Y., Peng, H., Cambria, E., 2018, April. Targeted aspect-based sentiment analysis Trivedi, S.K., Singh, A., Malhotra, S.K., 2022. Prediction of polarities of online hotel
via embedding commonsense knowledge into an attentive LSTM. In: reviews: an improved stacked decision tree (ISD) approach. Global Knowledge,
Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32, No. 1. Memory and Communication.
Ma, F., Zhang, C., Song, D., 2021. Exploiting position bias for robust aspect sentiment Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., et al., 2017.
classification. 1352–1358. arXiv:2105.14210. Attention is all you need. Adv. Neural Informat. Process. Syst. 30.
Ma, Y., Song, R., Gu, X., et al., 2023. Multiple graph convolutional networks for Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y., 2017. Graph
aspect-based sentiment analysis. Appl. Intell. 53, 12985–12998. https://doi.org/ attention networks. arXiv preprint arXiv:1710.10903.
10.1007/s10489-022-04023-z. Wan, H., Yang, Y., Du, J., Liu, Y., Qi, K., Pan, J.Z., 2020, April. Target-aspect-sentiment
Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., McClosky, D., 2014, joint detection for aspect-based sentiment analysis. In: Proceedings of the AAAI
June. The Stanford CoreNLP natural language processing toolkit. In: Proceedings Conference on Artificial Intelligence, Vol. 34, No. 05, pp. 9122–9129.
of 52nd Annual Meeting of the Association for Computational Linguistics: Wang, Y., Huang, M., Zhu, X., Zhao, L., 2016, November. Attention-based LSTM for
System Demonstrations, pp. 55–60. aspect-level sentiment classification. In: Proceedings of the 2016 Conference on
Mao, Q., Li, J., Wang, S., Zhang, Y., Peng, H., He, M., Wang, L., 2019, August. Aspect- Empirical Methods in Natural Language Processing, pp. 606–615.
Based Sentiment Classification with Attentive Neural Turing Machines. In: IJCAI, Wang, X., Xu, G., Zhang, J., Sun, X., Wang, L., Huang, T., 2018. Syntax-directed hybrid
pp. 5139–5145. attention network for aspect-level sentiment analysis. IEEE Access 7, 5014–
Mewada, A., Dewang, R.K., 2023. SA-ASBA: a hybrid model for aspect-based 5025.
sentiment analysis using synthetic attention in pre-trained language BERT Wang, K., Shen, W., Yang, Y., Quan, X., Wang, R., 2020. Relational graph attention
model with extreme gradient boosting. J. Supercomput. 79, 5516–5551. https:// network for aspect-based sentiment analysis. arXiv preprint arXiv:2004.12362.
doi.org/10.1007/s11227-022-04881-x. Wankhade, M., Annavarapu, C.S.R., Abraham, A., 2023. MAPA BiLSTM-BERT: multi-
Nadeem, M.I., Ahmed, K., Li, D., Zheng, Z., Naheed, H., Muaad, A.Y., Abdel Hameed, aspects position aware attention for aspect level sentiment analysis. J.
H., 2022. SHO-CNN: a metaheuristic optimization of a convolutional neural Supercomput. 79, 11452–11477. https://doi.org/10.1007/s11227-023-05112-7.
network for multi-label news classification. Electronics 12 (1), 113. Wu, H., Liu, Y., Shi, S., 2020, November. Modularized syntactic neural networks for
Nadeem, M.I., Ahmed, K., Li, D., Zheng, Z., Alkahtani, H.K., Mostafa, S.M., Abdel sentence classification. In: Proceedings of the 2020 Conference on Empirical
Hameed, H., 2023. EFND: A semantic, visual, and socially augmented deep Methods in Natural Language Processing (EMNLP), pp. 2786–2792.
framework for extreme fake news detection. Sustainability 15 (1), 133. Wu, H., Zhang, Z., Wu, Q., 2021. Exploring syntactic and semantic features for
Nassif, A.B., Elnagar, A., Shahin, I., Henno, S., 2021. Deep learning for Arabic authorship attribution. Appl. Soft Comput. 111, 107815.
subjective sentiment analysis: Challenges and research opportunities. Appl. Soft Wu, H., Zhang, Z., Shi, S., Wu, Q., Song, H., 2022a. Phrase dependency relational
Comput. 98, 106836. graph attention network for Aspect-based Sentiment Analysis. Knowl.-Based
Nguyen, H.T., Le Nguyen, M., 2018. Effective attention networks for aspect-level Syst. 236, 107736.
sentiment classification. In: 2018 10th International Conference on Knowledge Wu, H., Zhang, Z., Shi, S., Wu, Q., Song, H., 2022b. Phrase dependency relational
and Systems Engineering (KSE). IEEE, pp. 25–30. November. graph attention network for aspect-based sentiment analysis. Knowl.-Based
Nguyen, T.H., Shirai, K., 2015, September. Phrasernn: Phrase recursive neural Syst. 236, 107736.
network for aspect-based sentiment analysis. In: Proceedings of the 2015 Wu, Zhaozhen, Cao, Guoyi, Mo, Wanghao, 2023. Multi-tasking for Aspect-based
Conference on Empirical Methods in Natural Language Processing, pp. 2509– Sentiment Analysis via Constructing Auxiliary Self-Supervision ACOP task. IEEE
2514. Access.
Pablos, A.G., Cuadros, M., Rigau, G., 2015, June. V3: Unsupervised aspect based Xiang, C., Zhang, J., Li, F., Fei, H., Ji, D., 2022. A semantic and syntactic enhanced
sentiment analysis for semeval2015 task 12. In: Proceedings of the 9th neural model for financial sentiment analysis. Informat. Process. Manage. 59
International Workshop on Semantic Evaluation (SemEval 2015), pp. 714–718. (4), 102943.
Pontiki, M., Galanis, D., Papageorgiou, H., Manandhar, S., Androutsopoulos, I., 2015, Xin, X., Wumaier, A., Kadeer, Z., He, J., 2023. SSEMGAT: Syntactic and Semantic
June. Semeval-2015 task 12: Aspect based sentiment analysis. In: Proceedings Enhanced Multi-Layer Graph Attention Network for Aspect-Level Sentiment
of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. Analysis. Appl. Sci. 13 (8), 5085.
486–495. Xue, W., Li, T., 2018. Aspect based sentiment analysis with gated convolutional
Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., AL- networks. arXiv preprint arXiv:1805.07043.
Smadi, M., Eryiğit, G., 2016. Semeval-2016 task 5: Aspect based sentiment Yang, P., Zhang, P., Li, B., Ji, S., Yi, M., 2023. Aspect-Based Sentiment Analysis Using
analysis. In: ProWorkshop on Semantic Evaluation (SemEval-2016). Association Adversarial BERT with Capsule Networks. Neural Process. Lett., 1–18
for Computational Linguistics, pp. 19–30. Yu, B., Zhang, S., 2023. A novel weight-oriented graph convolutional network for
Qi, P., Zhang, Y., Zhang, Y., Bolton, J., Manning, C.D., 2020. Stanza: A Python natural aspect-based sentiment analysis. J. Supercomput. 79, 947–972. https://doi.org/
language processing toolkit for many human languages. arXiv preprint 10.1007/s11227-022-04689-9.
arXiv:2003.07082. Zhang, C., Li, Q., Song, D., 2019. Aspect-based sentiment classification with aspect-
Qi, Y., Zheng, X., Huang, X., 2022. Aspect-based sentiment analysis with enhanced specific graph convolutional networks. In: Proceedings of the 2019 Conference
aspect-sensitive word embeddings. Knowl. Inf. Syst. 64, 1845–1861. https://doi. on Empirical Methods in Natural Language Processing and the 9th International
org/10.1007/s10115-022-01688-3. Joint Conference on Natural Language Processing, EMNLP-IJCNLP, 2019, pp.
Rebele, T., Suchanek, F., Hoffart, J., Biega, J., Kuzey, E., Weikum, G., 2016. YAGO: A 4568–4578.
multilingual knowledge base from wikipedia, wordnet, and geonames. In: The Zhang, C., Li, Q., Song, D., 2019, July. Syntax-aware aspect-level sentiment
Semantic Web–ISWC 2016: 15th International Semantic Web Conference, Kobe, classification with proximity-weighted convolution network. In: Proceedings
Japan, October 17–21, 2016, Proceedings, Part II 15, Springer International of the 42nd International ACM SIGIR Conference on Research and Development
Publishing, pp. 177–185. in Information Retrieval, pp. 1145–1148.
Schlichtkrull, M., Kipf, T.N., Bloem, P., Van Den Berg, R., Titov, I., Welling, M., 2018. Zhang, D., Zhu, Z., Kang, S., et al., 2021. Syntactic and semantic analysis network for
Modeling relational data with graph convolutional networks. In: The Semantic aspect-level sentiment classification. Appl. Intell. 51, 6136–6147. https://doi.
Web: 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June org/10.1007/s10489-021-02189-6.
3–7, 2018, Proceedings 15, Springer International Publishing, pp. 593–607. Zhang, W., Li, X., Deng, Y., Bing, L., Lam, W., 2022a. A survey on aspect-based
Schuster, M., Paliwal, K.K., 1997. Bidirectional recurrent neural networks. IEEE sentiment analysis: tasks, methods, and challenges. IEEE Trans. Knowl. Data
Trans. Signal Process. 45 (11), 2673–2681. Eng.
Singh, V., Pencina, M., Einstein, A.J., et al., 2021. Impact of train/test sample regimen Zhang, W., Yan, J., Wang, Z., Wang, J., 2022, April. Neuro-symbolic interpretable
on performance estimate stability of machine learning in cardiovascular collaborative filtering for attribute-based recommendation. In: Proceedings of
imaging. Sci. Rep. 11, 14490. https://doi.org/10.1038/s41598-021-93651-5. the ACM Web Conference 2022, pp. 3229–3238.
Sun, K., Zhang, R., Mensah, S., Mao, Y., Liu, X., 2019, November. Aspect-level Zhang, Z., Dong, Y., Wu, H., Song, H., Deng, S., Chen, Y., 2022c. Metapath and syntax-
sentiment analysis via convolution over dependency tree. In: Proceedings of the aware heterogeneous subgraph neural networks for spam review detection.
2019 Conference on Empirical. Appl. Soft Comput. 128, 109438.
Tai, K.S., Socher, R., Manning, C.D., 2015. Improved semantic representations from Zhang, Q., Wang, S., Li, J., 2023. A Contrastive Learning Framework with Tree-LSTMs
tree-structured long short-term memory networks. arXiv preprint for Aspect-Based Sentiment Analysis. Neural Process Lett. https://doi.org/
arXiv:1503.00075. 10.1007/s11063-023-11181-9.
Tang, D., Qin, B., Feng, X., Liu, T., 2015. Effective LSTMs for target-dependent Zhao, P., Hou, L., Wu, O., 2020. Modeling sentiment dependencies with graph
sentiment classification. arXiv preprint arXiv:1512.01100. convolutional networks for aspect-level sentiment classification. Knowl.-Based
Tang, D., Qin, B., Liu, T., 2016. Aspect level sentiment classification with deep Syst. 193, 105443.
memory network. arXiv preprint arXiv:1605.08900. Zhao, H., Liu, Z., Yao, X., Yang, Q., 2021a. A machine learning-based sentiment
Tang, H., Ji, D., Li, C., Zhou, Q., 2020, July. Dependency graph enhanced dual- analysis of online product reviews with a novel term weighting and feature
transformer structure for aspect-based sentiment classification. In: Proceedings selection approach. Informat. Process. Manage. 58 (5), 102656.
of the 58th Annual Meeting of the Association for Computational Linguistics, pp. Zhao, L., Liu, Y., Zhang, M., Guo, T., Chen, L., 2021b. Modeling label-wise syntax for
6578–6588. fine-grained sentiment analysis of reviews via memory-based neural model.
Tian, Y., Chen, G., Song, Y., 2021, June. Aspect-based sentiment analysis with type- Informat. Process. Manage. 58 (5). Article 102641.
aware graph convolutional networks and layer ensemble. In: Proceedings of the Zhao, Z., Tang, M., Zhao, F., et al., 2023a. Incorporating semantics, syntax and
2021 Conference of the North American Chapter of the Association for knowledge for aspect based sentiment analysis. Appl. Intell. 53, 16138–16150.
Computational Linguistics: Human Language Technologies, pp. 2910–2922. https://doi.org/10.1007/s10489-022-04307-4.

20
K. Ahmed, M.I. Nadeem, Z. Zheng et al. Journal of King Saud University – Computer and Information Sciences 35 (2023) 101651

Zhao, Guoshuai, Luo, Yiling, Chen, Qiang, Qian, Xueming, 2023b. Aspect-based Zhou, Y., Liao, L., Gao, Y., Wang, R., Huang, H., 2021. TopicBERT: A topic-enhanced
sentiment analysis via multitask learning for online reviews. Knowl.-Based Syst. neural language model fine-tuned for sentiment classification. IEEE Trans.
110326. Neural Networks Learn. Syst.
Zheng, W., Zhang, S., Yang, C., Hu, P., 2023a. Lightweight multilayer interactive Zhu, L., Zhu, X., Guo, J., Dietze, S., 2022. Exploring rich structure information for
attention network for aspect-based sentiment analysis. Connect. Sci. 35 (1), aspect-based sentiment classification. J. Intell. Informat. Syst., 1–21
2189119. Zhu, Z., Zhang, D., Li, L., Li, K., Qi, J., Wang, W., Liu, P., 2023. Knowledge-guided
Zheng, Y., Li, X., Nie, J., 2023b. Store, share and transfer: Learning and updating multi-granularity GCN for ABSA. Informat. Process. Manage. 60 (2), 103223.
sentiment knowledge for aspect-based sentiment analysis. Inf. Sci. 635, 151–
168. https://doi.org/10.1016/j.ins.2023.03.102.

21