Neurocomputing 635 (2025) 129919

Contents lists available at ScienceDirect

Neurocomputing
journal homepage: www.elsevier.com/locate/neucom

LGKGR: A knowledge graph reasoning model using LLMs augmented GNNs

Yuanming Zhang ∗, Wenbo Zheng, Jiacheng Huang, Gang Xiao ∗
College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China

ARTICLE INFO ABSTRACT

Communicated by C. Gao Knowledge graph reasoning (KGR) aims to infer new factual knowledge based on existing structured factual
data, and plays a vital role in various applications. Graph neural networks (GNNs)-based methods have
Keywords:
garnered attention due to their exceptional capabilities in learning graph structures. However, they cannot
Knowledge graph
Knowledge graph reasoning
effectively leverage rich text semantics within KG for reasoning. Given the remarkable semantic understanding
Graph neural networks capabilities of large language models (LLMs), this paper proposes a novel KGR model using LLMs augmented
Large language models GNNs (LGKGR), which aims to utilize LLMs to enhance the graph structure learning of GNNs. Each round
of reasoning includes three stages: path search, path pruning, and path decision. The first stage adopts an
incremental path search strategy to identify adjacent entities of current query entity and extract features. The
second stage adopts GNNs as a pruning tool to filter out semantically irrelevant reasoning paths. The third stage
exploits LLMs for semantic analysis of candidate reasoning paths, and then selects the most possible reasoning
paths. In the end, LLMs are further exploited to analyze the semantic information of reasoning paths and
generate final reasoning results. Experimental results on public datasets demonstrate that the proposed method
achieves an average improvement of 2.1% in the MRR metric and 2.68% in the Hits@1 metric compared to
existing SOTA methods. Explainable reasoning justifications are also generated during the reasoning process.

1. Introduction relation. The objective of KGR is to learn relevant knowledge from the
given KG to identify the target entity 𝑒𝑎 that satisfies (𝑒𝑞 , 𝑟𝑞 , 𝑒𝑎 ).
Knowledge graph (KG) is a kind of structured semantic knowl- Graph neural networks (GNNs) has become the mainstream method
edge base, which expresses the interrelationships between entities in in KGR due to its powerful graph structure learning ability [13], GNNs
a topological structure [1]. The entities and relation form a rich, update the entity features by passing messages between neighboring
interconnected, multi-level semantic knowledge network through com-
entities, and then uses the updated entity features for subsequent
plex association paths. Existing large-scale KGs include Freebase [2],
reasoning, however, due to the characteristics of GNNs, each round of
NELL [3], UMLS [4], which are characterized by their vast numbers of
reasoning involves a large number of entities, resulting in high infer-
entities and highly complex relational paths. Knowledge graph reason-
ing (KGR) aims to infer new or previously unknown knowledge based ence complexity and affecting the reasoning performance. To address
on existing facts, thereby can enrich the knowledge base further [5–7]. this problem, R-GCN [14] reduces model parameters by using relational
It has been widely utilized in recommendation systems [8,9], intelligent filtering and parameter sharing.
Q&A [10], disease diagnosis [11], and drug interaction prediction [12]. While the propagation of entity features in R-GCN are the entire
As an example, Fig. 1 shows the process of reasoning the main neighborhoods, which can lead to over-smoothing issues. NBFNet [15]
political parties in Canberra city through KGR. The ‘‘Canberra’’ is restricts the propagation scope to local neighborhoods by employ-
the beginning query entity. The KGR first searches the neighborhood ing progressive message passing. However, with the increase in the
entity set: {Oceania, Australian Capital Territory, Australia, Walter Burley number of reasoning steps, this method will also expand the neigh-
Griffin}. Then, it calculates the relevance of neighboring entities and borhood entities to most entities. Alternatively, GraIL [16] employs
select the most possible entities for next round of reasoning. In the
a subgraph sampling strategy, which limits the propagation of entity
figure, solid lines indicate strong relevance, while dashed lines indicate
features to closed subgraphs associated with the target entity. KGR is
weak relevance. In the second round of reasoning, the query entity is
then performed according to local information within these subgraphs.
‘‘Australia’’. After several rounds of reasoning, the target entity ‘‘Labor
Party’’ can be eventually obtained. In general, KGR can be formalized as However, its computational complexity is still much high in large-scale
(𝑒𝑞 , 𝑟𝑞 , ?), where 𝑒𝑞 represents the query entity and 𝑟𝑞 denotes the query KGs. In essence, existing GNNs-based methods for KGR predominantly

∗ Corresponding authors.
E-mail addresses: [email protected] (Y. Zhang), [email protected] (W. Zheng), [email protected] (J. Huang), [email protected] (G. Xiao).

https://doi.org/10.1016/j.neucom.2025.129919
Received 14 December 2024; Received in revised form 3 February 2025; Accepted 1 March 2025
Available online 11 March 2025
0925-2312/© 2025 Elsevier B.V. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
Y. Zhang et al. Neurocomputing 635 (2025) 129919

relational graphs. However, its parameters increase with the num-

ber of relations, resulting in large computational overhead and easy
overfitting. CompGCN [26] reduces parameter scale and improves com-
putational efficiency by jointly modeling node and relation features
with combination operations, but it is less precise in capturing complex
relational semantics and is sensitive to the choice of combination
functions. DSTKG [27] introduces two latent variables to learn dy-
namic and static representations of entities for reasoning. RGTN [28]
adaptively selects the most useful paths for each specific relation and
effectively builds path-based connections between unconnected distant
entities, achieving efficient representations. BiGAT [29] proposes a
Fig. 1. An example of KGR: What is the main political party in Canberra. bidirectional graph attention network, which considers both neighbors
from outbound and inbound to learn hierarchical neighbor propagation
and achieves excellent performance. Although these models have the
improve KGR performance by reducing propagation neighborhood of ability to propagate global information, their computation and memory
entity features. They mostly overlook the valuable textual semantics in consumptions are large. They also face the over-smoothing problem on
the KG to optimize the reasoning process [17]. sparse and large-scale graphs.
Actually, the entities and the relations in KG have a lot of text in- Local Message Passing: This kind of method restricts message
formation, which helps to understand their semantics, and is extremely propagation to a subset of neighboring entities. The NBFNet [15]
in selecting the correct reasoning path and improving reasoning per- significantly enhances the model’s performance by incorporating the
formance. However, due to the lack of semantic text comprehension Bellman-Ford algorithm, particularly when dealing with sparse and
capabilities, GNNs struggle to effectively utilize this textual information large-scale graph data. However, it requires multiple iterations and
during the reasoning process. Large language models (LLMs) have complex calculation process, which leads to high computational over-
been demonstrated to possess powerful semantic understanding and head and is sensitive to noisy data. The AdaProp [30] introduces a
contextual awareness [18]. Some researchers have introduced LLMs to dynamic and adaptive propagation strategy that adjusts the feature
KGR. For example, StructGPT [19] proposes an Iterative-Reading-then- propagation approach for each node. It enables efficient message in-
Reasoning (IRR) framework to perform KGR tasks. RoG [20] inputs tegration and propagation across different graph structures, but it
relational paths extracted from KG into LLMs, which then generate only uses a single feature to compute semantic relevance. This kind
reasoning results based on these paths. Similarly, ToG [21] treats of method can effectively reduce the computational complexity and
LLMs as intelligent agents, dynamically retrieving and utilizing rele- enhance the robustness of the model, especially on sparse graphs
vant knowledge through interactive exploration with external KGs to and large-scale graph data. However, they rely on complex parameter
generate reasoning results. However, current LLM-based methods lack optimization and carefully designed inference strategies.
dynamic graph learning capabilities [22], which hinders their ability to Subgraph Message Passing: This kind of method limits message
iteratively learn entity feature based on the graph topological structure. propagation to a specific subgraph. GraIL and its extensions, such as
To fully leverage both the text semantics and the graph structure CoMPILE [31], perform message passing within a subgraph to predict a
for higher KGR performance, this paper proposes a novel KGR model score as the confidence level for the query. However, these methods suf-
using LLMs augmented GNNs. It aims to enhance the graph learning fer from inefficient reasoning and high complexity. GLAR [32] extracts
ability of GNNs by harnessing the semantic understanding capabilities a shared opening subgraph from all candidates and performs reasoning
of LLMs. It can improve the accuracy of path pruning and path decision, on it, thereby maintaining low time consumption. Nevertheless, the
and generate interpretable reasoning justifications at the same time to subgraph sampling strategy makes it difficult to apply in large-scale
enhance the reliability of reasoning process. The main contributions of
KGs. Through message propagation in subgraph, these models can
this work include:
improve the computational efficiency and scalability, especially when
∙ This paper presents a KGR mechanism that utilizes structure
dealing with large-scale graphs. However, their performances depend
information for reasoning path pruning and then uses semantic infor-
on the reasonable subgraph partitioning strategy, and they face chal-
mation for reasoning path decision. It can reduce the scope of message
lenges when dealing with dependency relationship between subgraphs
propagation, and thereby effectively improve the reasoning efficiency.
and information loss problem.
∙ This paper proposes a KGR model using LLMs augmented GNNs.
GNNs show much promising for KGR due to their strong graph
It can leverage the semantic understanding capabilities of LLMs to
learning ability. However, they still lack deep text semantics under-
enhance the graph structure learning of GNNs, thereby greatly improve
standing capabilities, which makes them difficult to apply large and
the reasoning performance.
effective text information to KGR for better performance.
∙ Extensive experiments on public datasets demonstrate that the
proposed method achieves superior reasoning performance, and out-
performs existing state-of-the-art baseline methods. 2.2. LLMs-based knowledge graph reasoning

2. Related work Consider the powerful semantic understanding capabilities of LLMs,

researchers have introduced them to KGR [33–36]. UniKGQA [37]
2.1. GNNs-based knowledge graph reasoning leverages logical reasoning to efficiently retrieve relevant entities from
KG and transforms the retrieval results into natural language answers
GNN-based knowledge graph reasoning utilizes graph structural fea- that are accurate and easy to understand. But it has not yet fully
tures to perform message passing [23,24], updates features of entities incorporated the LLMs. StructGPT [19] utilizes LLMs for knowledge
during the reasoning process and selects answer entities [25]. This kind reasoning on structured data, such as KGs, tables, and databases, and
of method has become the mainstream of KGR, and can be divided into constructs specialized interfaces to access and manipulate them. Re-
three categories: global message passing, local message passing, and ACT [38] harnesses the reasoning ability of LLMs to develop action
subgraph message passing. plans by alternately generating reasoning trajectories and specific task
Global Message Passing: This kind of method propagates messages instructions and progressively completes reasoning tasks. RoG [20]
across all entities. R-GCN has powerful relationship modeling capa- generates possible relational paths from KG as action plans and then
bilities and can accurately capture structural information of multiple retrieves relevant entities and relationships based on these plans. It

2
Y. Zhang et al. Neurocomputing 635 (2025) 129919

Fig. 2. Knowledge graph reasoning framework Using LLMs augmented GNNs.

ultimately derives answers through reasoning steps. However, its per- this paper proposes a novel KGR model by using LLMs augmented
formance is greatly affected by the quality of KG. KAPING [39] re- GNNs (LGKGR). The whole reasoning process contains multiple rounds
trieves relevant triples from KG and incorporates these triples as prompt of reasoning, with each round comprising three reasoning stages: (1)
information into the query. It enhances LLMs’ contextual understanding Path search stage: this stage employs a new incremental search strategy
and then generates reasoning results. ToG [21] treats LLMs as intel- to reduce the number of involved query entities while preserving the
ligent agents that dynamically retrieve relevant knowledge through hierarchical connections between them; (2) Path pruning stage: this
interactive exploration with KG to generate reasoning results. EtD [40] stage uses the GNNs to analyze the relevance of the current query
incorporates LLMs in the final reasoning stage, but it has not yet entity with the neighboring entities, and then eliminates irrelevant
realized a deeper integration of LLMs and GNNs. Relying solely on the entities; (3) Path decision stage: this stage inputs the semantic text
backtracking path as the basis for interpretability appears insufficient. of entities and prompt templates into LLMs, which decide the next-
Although the above models effectively combine KG and LLMs, they hop entities and guide the GNNs to complete entity features updates,
heavily rely on the quality of the external knowledge base and face the and the interpretable justification is generated at the same time. Fig. 2
challenge of data alignment. In addition, LLMs still lack deep graph presents the overall framework of LGKGR. We will show more details
structure understanding abilities, which limit the performance of KGR. for each reasoning stage.

3. Problem formulation 4.1. Incremental reasoning path search

Knowledge graph (KG) represents a structured knowledge base, In large-scale KG, the number of involved query entities grows
exponentially with the increase of reasoning paths. The KGR faces
which explicitly encodes attributes of entities and the semantic re-
significant challenges on efficiency and accuracy. It needs to avoid
lations between them. Knowledge graph reasoning (KGR) involves
missing relevant query entities while preventing the introduction of too
reasoning new, previously unknown knowledge (i.e., new triples) from
many irrelevant query entities. Inspired by the fact that the target entity
existing knowledge (i.e., known triples) within the graph. It can be
is typically close to the query entity [30], we present a new incremental
formally defined as follows:
path search strategy, which starts from the selected query entities and
progressively continuing the path search in an incremental manner.
Definition 1 (Knowledge Graph Reasoning). The KGR task can be for-
Fig. 3 illustrates the incremental reasoning path search process.
mally represented as (𝑒𝑞 , 𝑟𝑞 , 𝑒𝑎 ) where 𝑒𝑞 is the query entity, 𝑟𝑞 is the
Initially, the query entity set is {0}, and the candidate entity set is
query relation, and 𝑒𝑎 is the target entity to be reasoned. The reasoning
empty. In the first round of reasoning, the query entity set is {0}, and
process consists of multiple steps, and in the 𝑙th reasoning step, the
the candidate entity set is {1, 2, 3, 4, 5}, which consists of the direct
involved entities are denoted as  𝑙 with the current query entity as neighbors of query entities. If entity 4 and entity 5 are selected as new
𝑒𝑠 ∈  𝑙−1 , the relation 𝑒𝑟 ∈  and the next hop entity is denoted as query entities, the query entity set is updated to {0, 4, 5}. In the second
𝑒𝑜 ∈  𝑙 . round of reasoning, the query entity set is {0, 4, 5}, and the candidate
entity set is {1, 2, 3, 6, 7, 8, 9}. If the entity 7 and entity 8 are selected as
Definition 2 (Knowledge Graph Reasoning Model). The KGR model can new query entities, the query entity set is updated to {0, 4, 5, 7, 8}. This
be represented as 𝐹 = (𝑤, 𝐺 ̂𝐿 ) where 𝑤 is the model parameters, and 𝐺 ̂𝐿
reasoning process continues iteratively until the final query entity set
is the reasoning path containing the entities involved in each reasoning is obtained.
̂𝐿 = { 0 ,  1 , … ,  𝑙 }, where  𝑙 , 𝑙 = 1...𝑙 denotes the set of
step, i.e. 𝐺 Candidate entity generator: The task of incremental reasoning
entities involved in the 𝑙th reasoning step. path search is to generate all possible entities for the current round
of reasoning. If the query entity set at the 𝑙th reasoning step is 𝑒𝑙−1 ,
KGR problem formalization: The KGR task is to start from a given 𝑞 ,𝑟𝑞

query entity and search for the most likely reasoning path 𝐺 ̂𝐿 among all the candidate entity set of the 𝑙th reasoning step can be defined as
̄ 𝑒𝑙 ,𝑟 = 𝐶𝐴𝑁𝐷(𝑒𝑙−1 ,𝑟 ), which includes all neighboring entities of the
relevant paths in KG. The KGR model can accurately predict the target 𝑞 𝑞 𝑞 𝑞

entity 𝑒𝑎 for each query (𝑒𝑞 , 𝑟𝑞 , 𝑒𝑎 ),while minimizing the size of the set current query entity set  (𝑉𝑒𝑙−1
,𝑟 ) = ∪𝑒∈ 𝑙−1  (𝑒). The candidate entity
𝑞 𝑞 𝑒𝑞 ,𝑟𝑞
of entities  𝑙 involved in each round of reasoning. set is calculated by the formula:

̄ 𝑒𝑙−1 𝑙
,𝑟 ∶= 𝐶𝐴𝑁𝐷(𝑒 ) =  (𝑉𝑒𝑙−1 𝑙−1
,𝑟 )∖𝑒 ,𝑟 (1)
4. Knowledge graph reasoning model 𝑞 𝑞 𝑞 ,𝑟𝑞 𝑞 𝑞 𝑞 𝑞

Reasoning path pruner: The candidate entity set includes all pos-
To enhance the graph structure learning capabilities of GNNs by us- sible first-order neighboring entities, which represent all potential rea-
ing the textual semantic understanding of LLMs for higher performance, soning paths. To control the number of involved entities, we employ

3
Y. Zhang et al. Neurocomputing 635 (2025) 129919

Fourthly, logits computation. Input feature matrix ℎ𝑙𝑒 to the learn-

𝑜
able linear layer and the semantic relevance score of each candidate
entity to the query entity is calculated as follows:
𝑙
𝑠𝑐𝑜𝑟𝑒𝑒𝑜 = 𝑠𝑜𝑓 𝑡𝑚𝑎𝑥(𝑊𝑠𝑐𝑜𝑟𝑒 (ℎ𝑙𝑒 )) (7)
𝑜

Finally, the candidate reasoning paths are pruned according to the

semantic relevance score, and TopK candidate entities with the highest
relevance score are obtained:

(̄ 𝑒𝑙 ) ∶= 𝐺𝑁𝑁𝑠𝑝𝑟𝑢𝑛𝑒 ( (𝑉𝑒𝑙−1 𝑙−1

,𝑟 )∖𝑒 ,𝑟 )
𝑞 ,𝑟𝑞 𝑝𝑟𝑢𝑛𝑒 𝑞 𝑞 𝑞 𝑞

= {𝑎𝑟𝑔𝑡𝑜𝑝𝑘 (𝑠𝑐𝑜𝑟𝑒𝑒𝑜 ), 𝑒𝑜 ∈ ̄ 𝑒𝑙 } (8)

𝑞 ,𝑟𝑞

where the (̄ 𝑒𝑙 ,𝑟 )𝑝𝑟𝑢𝑛𝑒

is the set of candidate entities after pruning,
𝑞 𝑞
which is used for the next step.

4.3. LLMs-based reasoning path decision

Fig. 3. Principle of incremental reasoning path search.

Reasoning path decision aims to identify the most possible paths
from previous pruned candidate paths. It is the key step to optimize
the reasoning process. To fully utilize the text semantic, we propose
GNNs as the reasoning path pruner to filter out irrelevant entities. The a LLMs-based reasoning path decision approach, which selects the
calculation formula for the pruned candidate entity set is as follows: most likely TopK reasoning paths from the pruned reasoning paths
by analyzing their semantic information. And the set of query entities
(̄ 𝑒𝑙 ) ∶= 𝑃 𝑅𝑈 𝑁𝐸(̄ 𝑒𝑙 ) (2)
𝑞 ,𝑟𝑞 𝑝𝑟𝑢𝑛𝑒 𝑞 ,𝑟𝑞 for the next round of reasoning is generated based on these reasoning
LLMs-based reasoning path decision: Its task is to select the paths.
correct reasoning path from the retained paths. LLMs are introduced To efficiently organize query entities, query relations, and candidate
to perform semantic analysis and determine the most probable paths, entities for guiding the LLMs in reasoning path decision, this paper
which are incorporated into the query entity set of the l-th reasoning designs a knowledge-enhanced prompt template, as shown in Fig. 4(a).
step, and result in an updated query entity set for the l-th reasoning The prompt template consists of four parts: task description, query,
step: candidate paths, and answer format. The task description clearly de-
fines the reasoning task to be executed by LLMs. The query, which
𝑒𝑙 ∶= 𝑒𝑙−1 ̄𝑙
,𝑟 ∪ 𝐷𝐸𝐶𝐼𝑆𝐼𝑂𝑁((𝑒 ) ) (3)
𝑞 ,𝑟𝑞 𝑞 𝑞 𝑞 ,𝑟𝑞 𝑝𝑟𝑢𝑛𝑒 ensures the LLMs’ response aligns with the ask closely, includes the
initial query entity and relation. The candidate paths are composed
4.2. GNNs-based reasoning path pruning of the relation paths between the pruned candidate entities and the
query entity. For example, Fig. 4(b) provides the candidate paths:
Since KGR faces excessive computational complexity and risk of 𝑃 𝐴𝑇 𝐻1, 𝑃 𝐴𝐻𝑇 2, ⋯ , 𝑃 𝐴𝑇 𝐻𝑁, corresponding to the KG in the second
overfitting, reasoning path pruning is essential to eliminate paths with round of reasoning, shown in Fig. 4(c). The LLMs will select the most
minimal contributions to the reasoning process. We propose a GNNs- probable reasoning path according to the semantic information of these
based reasoning path pruning method, which evaluates the likelihood candidate paths. The answer format specifies the expected outputs from
of candidate entities by calculating their semantic relevance to the the LLMs. Additionally, regular expressions are applied to clean the
query entity based on node information. The calculation process is as output and extract the most likely reasoning paths.
follows. The reasoning path decision will output new query entities for the
Firstly, calculate the relationship attention coefficient 𝛼𝑒𝐿 to mea- next round of reasoning. Their features are updated again according to
𝑟
sure the importance of each edge (𝑒𝑠 , 𝑟, 𝑒𝑜 ) with 𝑒𝑠 ∈𝑙−1
𝑒𝑞 ,𝑟𝑞 , The calculation the adjacency relations with the following formula:
formula is:
ℎ𝑙+1
𝑒 ∶= ((1 − 𝑛𝑜_𝑔𝑟𝑎𝑑(𝑠𝑐𝑜𝑟𝑒𝑒𝑠𝑒𝑙𝑒𝑐𝑡 ) + 𝑠𝑐𝑜𝑟𝑒𝑒𝑠𝑒𝑙𝑒𝑐𝑡 )) ∗ ℎ𝑙𝑒 (9)
𝑠𝑒𝑙𝑒𝑐𝑡 𝑠𝑒𝑙𝑒𝑐𝑡
𝛼𝑒𝑙 = 𝑆𝑖𝑔𝑚𝑜𝑖𝑑(𝑊𝛼𝑙 (𝑅𝑒𝐿𝑢(𝑊𝑠𝑙 (ℎ𝑙−1 𝑙 𝑙 𝑙 𝑙
𝑠 ) + 𝑊𝑟 (ℎ𝑟 ) + 𝑊𝑒 (ℎ𝑒 )))) (4)
𝑟 𝑞𝑟 𝑞𝑟
These features are used to the next round of reasoning as initial
where 𝑊𝛼𝑙 , 𝑊𝑠𝑙 , 𝑊𝑟𝑙 , 𝑊𝑒𝑙 in R1×𝑑 are learnable parameter matrix. ℎ𝑙−1 𝑠 representations for the GNNs, thereby enhancing the accuracy of the
𝑞𝑟
is the feature matrix of the query entities. Note that since the entity pruning operation.
𝑒𝑠 ∈𝑙−1
𝑒 ,𝑟 is selected in the previous reasoning step, the update features in
𝑞 𝑞
step 𝑙 − 1 are utilized here. ℎ𝑙𝑟 is the feature matrix of the relationship 𝑟. 4.4. Final reasoning path generation
ℎ𝑙𝑒 is the combination feature of the starting query relationship (𝑒𝑞 , 𝑟𝑞 ).
𝑞𝑟
Secondly, calculate message. Feature information of current query After several rounds of reasoning, the final query entity set will be
entities 𝑒𝑆 and query relationship 𝑟 are computed as follows: generated. Based on this set, the final reasoning path is determined
𝑚𝑙(𝑒 ∶= 𝑀𝐸𝑆𝑆(ℎ𝑙−1 𝑙 𝑙 𝑙−1 𝑙 through path pruning and path decision. The features of all entities in
𝑠 ,𝑟,𝑒𝑜 ) 𝑠 , ℎ𝑟 ) = 𝛼𝑒 (ℎ𝑠 + ℎ𝑟 ) 𝑟
(5)
the set are fed into a trainable linear layer to compute the semantic
Thirdly, perform message aggregation. Feature information is prop- relevance between each candidate entity and the query entity according
agated from query entity 𝑒𝑠 ∈ 𝑉𝑒𝑙−1 ̄𝑙
,𝑟 to candidate entity 𝑒𝑜 ∈ 𝑒 ,𝑟 . After
𝑞 𝑞 𝑞 𝑞
to Formula. 5. The path pruning is applied to filter out the TopK most
that, the feature matrix ℎ𝑙𝑒 of the candidate entity set is as follows: likely candidate entities, eliminating those with lower probabilities to
𝑜
reduce noise interference during the path decision-making. Finally, the
1 ∑
ℎ𝑙𝑒 = 𝛿(𝐴𝐺𝐺(𝑚𝑙(𝑒 , (𝑒𝑠 , 𝑟, 𝑒𝑜 ) ∈ 𝜀𝑙 )) =
𝑚𝑙 (6) LLMs are further adopted to select the final target entity. To this end,
𝑜 𝑠 ,𝑟,𝑒𝑜 ) | (𝑒𝑠 )| 𝑒𝑜 ∈ (𝑒𝑠 ) (𝑒𝑠 ,𝑟,𝑒𝑜 )
| | another prompt template is also designed for LLMs, as shown in Fig.
where 𝛿 is the activation function, and || (𝑒𝑠 )|| is the norm of candidate 5. This template assists the LLMs in selecting the final answer from the
entities set. candidate entities.

4
Y. Zhang et al. Neurocomputing 635 (2025) 129919

Fig. 4. Prompt template for LLMs in path decision stage.

Fig. 5. Prompt template for LLMs in final reasoning path generation.

4.5. Explainable reasoning justifications 4.6. Model training

The reasoning process of each round is stored in the stack, and the The model is trained by optimizing the semantic relevance scores of
the target entities. Specifically, a binary cross-entropy loss function is
reasoning path can be retraced by dynamically maintaining the stack
employed to achieve this:
data structure, which is used as the basis of interpretable reasoning. ∑
( ( ) ) ( ( ))
LGKGR not only can provide complete reasoning paths, but also  𝐹 𝑤, 𝐺̂ 𝐿 , 𝑒𝑎 = −𝑦𝑒𝑜 log semantic_score 𝑒𝑜
can generate reasoning justifications for each round of reasoning. This 𝑒0 ∈𝑒𝑙𝑞 ,𝑟𝑞
(10)
( ) ( ( ))
improves the explainability of reasoning results. We take the query
+ 1 − 𝑦𝑒𝑜 log 1 − semantic_score 𝑒𝑜
(Canberra, Major_Party, ?) from Fig. 1 as an example. The LGKGR gen-
erates justifications for each reasoning path, as shown in Fig. 6. LLMs where 𝑠𝑒𝑚𝑎𝑛𝑡𝑖𝑐_𝑠𝑐𝑜𝑟𝑒(𝑒𝑜 ) is semantic relevance of target entity 𝑒𝑜 .
assign a confidence score to different paths and give the corresponding 𝑦(𝑒𝑜 )=1 if 𝑒𝑜 = 𝑒𝑎 otherwise 0; The LGKGR parameters are optimized by
reasoning justifications. In the first round of reasoning, LLMs correctly minimizing  using the Adam stochastic gradient descent algorithm.
establish a link between the main political party and the areas in the
query, paving the way for the identification of more specific entities 5. Experimental
related to places. Simultaneously, they correctly identify some broad
5.1. Experimental environment
entities, including ‘‘Oceania’’. Although this entity can also represent
a geographical location, LLMs still judge it as irrelevant. In the sec-
Datasets: The proposed LGKGR is evaluated on four public widely
ond round of reasoning, LLMs recognize that the head of government
used datasets: UMLS, WN18RR, FB15k237, and NELL-995. The relevant
usually has the political background of major parties and therefore
statistics of the datasets are given in Table 1.
direct the reasoning path to the entity ‘‘Head of government’’. They •UMLS (Unified Medical Language System) This data set is an
also unearths more specific clues, such as current president. Moreover, integrated medical knowledge framework developed by the National
LLMs are also very efficient in excluding those insignificant entities. Library of Medicine (NLM) in 1986. Its primary objective is to facilitate
In the third round of reasoning, LLMs fully understand the semantic the creation of more efficient and interoperable biomedical information
information of entities, and in particular, they get the political party systems and services, including electronic health records. UMLS has
background of current prime minister very well. This fits with the idea been extensively studied and applied in various fields, such as informa-
from the second round. This demonstrates the strong advantage of LLMs tion retrieval, natural language processing, electronic patient records,
in terms of semantic understanding and reasoning capabilities. and health data standards.

5
Y. Zhang et al. Neurocomputing 635 (2025) 129919

Fig. 6. Reasoning paths and explainable justifications generated by LLMs.

Table 1
Statistics of the KG datasets. Fact triplets in 𝜖 are used to build graph, and 𝑄𝑡𝑟𝑎 , 𝑄𝑣𝑎𝑙 , 𝑄𝑡𝑠𝑡 are the query triplets used for reasoning.
Dataset Entity type Relation |𝜖| |𝑄𝑡𝑟𝑎 | |𝑄𝑣𝑎𝑙 | |𝑄𝑡𝑠𝑡 |
UNLS 135 46 5.3K 1.3K 569 533
WN18RR 40.9K 11 65.1K 21.7K 3.0K 3.1K
FN15K237 14.5K 237 204.1K 68.0K 17.5K 20.4K
NELL-995 74.5K 200 112.2K 37.4K 543 2.8K

•WN18RR (WordNet 18 benchmark with Refined Relations) This where 𝑆 represents a set of triples, |𝑆| is the number of triples, and is
dataset is a subset of WordNet and a commonly used dataset in KGs, the indicator function, I(⋅) = 1 if 𝑟𝑎𝑛𝑘𝑖 ≤ 𝑛 otherwise 0. Typically, 𝑛 is
primarily designed for studying link prediction problems. Compared to set to 1 or 0. The higher the metric, the better the performance.
the original WordNet, WN18RR eliminates certain errors and redun- Baseline models: The proposed LGKGR is compared against ex-
dancies, making it more suitable for evaluating the performance of link isting three kinds of baseline: (1) Traditional KGR models, including
prediction algorithms. WordNet itself is a widely used lexical database ConvE [41], QuatE [42], RotatE [43], MINERVA [44], DRUM [45],
for the English language, organizing words into a network where they RNNLogic [46], and RLogic [47], which primarily rely on represen-
are interconnected through various semantic relations. tation learning and logical rules for reasoning. (2) GNNs-based KGR
•FB15k-237 (Freebase 15k-237) This dataset is a subset of the models, including CompGCN, NBFNet, RED-GNN and AdaProp, which
Freebase KG. Freebase is a large-scale, collaborative knowledge base
leverage GNNs for reasoning. (3) LLMs-based KGR models, including
that contains vast amounts of entity information and various relations
RoG and ToG, which utilize LLMs for reasoning and achieves promising
between entities. The FB15k-237 dataset was created specifically for
performance. (4) LLMs-GNNs based KGR models, including EtD, which
studying KG embedding and link prediction tasks.
combines GNNs and LLMs for KGR.
•NELL-995 (Never-Ending Language Learning-995) This dataset
is a subset extracted from the NELL system. NELL is an automatic
machine learning system developed by Carnegie Mellon University, 5.2. Experimental results
designed to continuously learn from the web to build and expand a
large-scale knowledge base. The NELL-995 dataset contains some of the LGKGR has been implemented on Pytorch framework and is trained
most confident knowledge triples from the NELL system, along with on four NVIDIA RTX 3090 GPUs with a total of 96 GB memory. We use
their associated relations. GPT 3.5 to perform reasoning path decision and final reasoning path
Evaluation metrics: Two widely used metrics: MRR (Mean Recip- generation. The hyperparameter settings are detailed in Table 2, and
rocal Rank) and Hits@n (Hits@1, Hits@10) are adopted to evaluate the the parameters are optimized using the Adam algorithm. Each dataset
performance. Hits@n focuses on evaluating a model’s recall capability, (UMLS, WN18RR, FB15k-237, and NELL-995) is divided into train-
specifically whether the model can retrieve all relevant items. In con- ing dataset, validation dataset, and test dataset, ensuring no overlap
trast, the MRR emphasizes the model’s accuracy and ranking quality, between these sub-datasets.
assessing whether relevant items are ranked higher in the results. Table 3 presents the overall experimental results on the four public
The formula for MRR is as follows: datasets. It can be seen that the LGKGR achieves the best performance
|𝑆| ( )
1 ∑ 1 1 1 1 1 on all datasets compared to the SOTA baseline modes (underlined).
𝑀𝑅𝑅 = = + +⋯+ (11)
|𝑆| 𝑖=1 rank 𝑖 |𝑆| rank 1 rank 2 rank |𝑆| The proposed LGKGR demonstrates significantly better performance
compared to the optimal baselines (indicated by underlines) across four
where 𝑆 is the set of triples, |𝑆| is the number of triples, and 𝑟𝑎𝑛𝑘𝑖 is the datasets. Specifically, it achieves improvements of 2%, 1.2%, 3.7%,
connection prediction ranking of the 𝑖th triple. The higher the metric,
and 1.5% in the MRR metric; 2.7%, 2.2%, 3.3%, and 2.5% in the
the better.
Hits@1 metric; and 0.3%, 3.7%, 1.5%, and 2.3% in the Hits@10 met-
The formula for Hits@n is as follows:
ric. These results demonstrate that LGKGR exhibits superior reasoning
|𝑆|
1 ∑ ( ) capabilities. One reason is that GNNs are adopted to prune reasoning
HITS @𝑛 = I rank 𝑖 ≤ 𝑛 (12)
|𝑆| 𝑖=1 paths and optimize the size of candidate entity set, thereby enhancing

6
Y. Zhang et al. Neurocomputing 635 (2025) 129919

Table 2 is used to indirectly illustrate the necessity of pruning by expanding

List of hyperparameters.
the GNNs-based reasoning path pruning TopK from {50, 50, 100, 100}
Hyperparameter Value
to {100, 100, 200, 200}. It can be seen that GNNs-based path prun-
Random Seed 1234 ing achieves significant performance improvements. GNNs-based path
Fact Ratio 0.9,0.9,0.96,0.99
pruning can capture rich structural information, and identify valuable
Hidden Layer Dimension 128
GNNs-based Reasoning Path Pruning TopK 50,50,100,100 reasoning paths. In addition, this method can also effectively reduce
LLMs-based Reasoning Path Decision TopK 3,3,5,5 the size of the candidate entity set, which not only alleviates the
Network Layer 5,5,8,8 computational burden on the LLMs but also allows for more targeted
Temperature Value 1.0
feature updates.
Semantic Layer Dimension 48
Learning Rate 0.009 Effect of LLMs-based reasoning path decision. In the path deci-
Batchsize 10 sion stage, LGKGR incorporates LLMs to identify the next-hop entities
Dropout 0.0391 that contribute the most to answering query. We compared the full
Training Epochs 300
version that takes LLMs as a discriminator with a version that omits
Weight Decay Rate 0.9938
LLMs as s discriminator, and presented the experimental results in
Table 5. Experiments show that LLMs can effectively identify next-hop
entities that contribute more to the final answer of inference, thereby
the coverage of potential target entities. Another reason is that LLMs helping GNNs perform more targeted vector updates and indirectly
are exploited to make path decision with text semantics, significantly improving the reasoning accuracy of the model. This further verifies
improving the hit rate of target entities. the effectiveness of LLMs in understanding the semantic information
In addition, the GNNs-based methods have better reasoning perfor- contained in the reasoning task.
mance than traditional methods. The reason is that GNNs can effec- Effect of LLMs-based final reasoning result generation. In the
tively learn semantic relationships among entities through aggregation
final reasoning result generation stage, LGKGR incorporates LLMs to
and updating of node neighborhood information. CompGCN can fully
identify the most probable answer. We compared the full version that
learn relational patterns on small datasets (e.g. UMLS). While, this
takes LLMs as a decision-maker with a version that omits LLMs as
model has over-smoothing problem on complex and multi-relational
a decision-maker (𝑤.𝑜.𝐿𝐿𝑀𝑠𝑓 𝑖𝑛𝑎𝑙 ). Table 6 shows the experimental
datasets (e.g. WN18RR, FB15k237), which makes it difficult to distin-
results. It can be seen that LLMs can generate more precise reasoning
guish similar embeddings. NBFNet adopts Bellman-Ford propagation
results, and significantly improves the reasoning accuracy. This demon-
mechanism, which is suitable for multi-hop inference tasks, and can
strates that LLMs can effectively understand the relation semantics of
accurately capture complex relationships in datasets with rich path
reasoning paths and play a role of text semantics interpreter.
information. RED-GNN is able to accurately extract semantic relevance
in simple datasets through recursive neighbor exploration. However, it Effect of LLMs on performance. Since we use LLMs to enhance
will introduce a large number of irrelevant entities during the propa- GNNs, the KGR performance strongly depends on their semantic un-
gation process for datasets with complex relationships, and results in derstanding ability. To examine the impact of LLMs on performance,
redundant information that interferes with the discriminative ability we compared two representative LLMs (GLM-10B and Llama-13B) with
and causes error propagation. Although AdaProp performs well, it GPT 3.5. Table 7 shows the experimental results. It can be seen that
fails to achieve performance breakthroughs on datasets with complex GPT 3.5 outperforms GLM-10B and Llama-13B. This shows that GPT
semantic information. The reason of that is the semantic relevance 3.5 has the best ability to understand the semantic meaning of entities
calculation is based on a single feature. and relations. GPT 3.5 can also output strictly according to the designed
LLMs-based models can utilize the powerful semantic understanding template, while GLM-10B suffers from certain hallucination problem,
ability of LLMs. RoG achieves good results by using LLMs to retrieve and additional data clarity work is required. In addition, we used these
relevant paths. However, it cannot retrieve long distance reasoning LLMs by directly calling the APIs.
paths and fails to effectively capture long-range semantic information.
ToG iteratively explores multiple possible reasoning paths, and can 6. Conclusions
handle multi-hop reasoning. While it ignores the importance of graph
learning. EtD leverages LLMs for encoding and relies on LLMs to
To address the challenge of current GNNs-based knowledge graph
confirm the final answer, and this significantly improves the accuracy
reasoning (KGR) methods being unable to fully utilize the large amount
compared with GNNs-based KGR methods. However, since EtD still uses
of text information in KG, we proposed a novel knowledge graph
the pruning technique based on semantic score, its overall performance
reasoning model using LLMs augmented GNNs, named LGKGR. Overall,
is still slightly inferior to our LGKGR. Compared with existing methods,
this model utilized the semantic understanding capabilities of LLMs to
our LGKGR uses the LLMs to enhance the GNNs to learn both structural
enhance the graph structure learning ability of GNNs. Each round of
and semantic features for complex path reasoning, and achieves much
reasoning mainly consisted three steps: reasoning path search, reason-
higher performance.
ing path pruning, and reasoning path decision. Path search adopted a
5.3. Ablation experiments new incremental search strategy to preserve the hierarchical connec-
tions of query entities while reducing the number of involved entities.
This section studies the impact of GNNs-based path pruning, LLMs- Path pruning employed GNNs to learn graph structure features to
based path decision and LLMs-based final reasoning result generation eliminate paths with minimal contributions to the query entity. Path
on the LGKGR performance through ablation experiments. decision introduced LLMs to semantically identify the most possible
Effect of GNNs-based path pruning. LGKGR adopts the GNNs entities to the query entity, narrowing the scope of subsequent round
to prune paths with minimal contribution to the reasoning process. of reasoning. The reasoning process integrated the text semantic and
GNNs can learn graph structural features and leverage them for path graph structure feature, significantly improving the reasoning per-
pruning. We compare a version that includes GNNs-based path pruning formance. Experimental results on public datasets demonstrated that
(LGKGR-prune) with another version that omits path pruning (w.o. the proposed LGKGR achieved substantial performance improvements
prune). The results are shown in Table 4. Note that due to the large compared to the state-of-the-art methods. Additionally, it provided
number of relations in the graph, no pruning at all will lead to a rapid explainable reasoning justifications, which enhanced the transparency
increase in model training cost. The version without path pruning here and reliability of the reasoning process.

7
Y. Zhang et al. Neurocomputing 635 (2025) 129919

Table 3
Experimental results. Best performance is indicated by the bold face numbers, and the underline means the second best. ‘‘–’’ means unavailable results. ‘‘H@1’’ and ‘‘H@10’’ are
short for Hits@1 and Hits@10 (in percentage), respectively.
Type Model UMLS WN18RR FB15k237 NELL-995
MRR H@1 H@10 MRR H@1 H@10 MRR H@1 H@10 MRR H@1 H@10
ConvE 93.7 92.2 96.7 42.7 39.2 49.8 32.5 23.7 50.1 51.1 44.6 61.9
QualE 94.4 90.5 99.3 48.0 44.0 55.1 35.0 25.6 53.8 53.3 46.6 64.3
RotatE 92.5 86.3 99.3 47.7 42.8 57.1 33.7 24.1 53.3 50.8 44.8 60.8
Tradition
MINERVA 82.5 72.8 96.8 44.8 41.3 51.3 29.3 21.7 45.6 51.3 41.3 63.7
method
DRUM 81.3 67.4 97.6 48.6 42.5 58.6 34.3 25.5 51.6 53.2 46.0 66.2
RNNLogic 84.2 77.2 96.5 48.3 44.6 55.8 34.4 25.2 53.0 41.6 36.3 47.8
RLogic – – – 47.0 44.3 53.7 31.0 20.3 50.1 – – –
CompGCN 92.7 86.7 99.4 47.9 44.3 54.6 35.5 26.4 53.5 46.3 38.3 59.6
NBFNet 94.8 92.0 99.5 55.1 49.7 66.6 41.5 32.1 59.9 52.5 45.1 63.9
GNNs
RED-GNN 96.4 94.6 99.0 53.3 48.5 62.4 37.4 28.3 55.8 54.3 47.6 65.1
AdaProp 96.9 95.6 99.5 56.2 49.9 67.1 41.7 33.1 58.5 55.4 49.3 65.5
RoG 93.4 92.7 99.2 55.4 50.1 67.3 42.9 35.0 52.1 56.1 50.5 63.3
LLMs
ToG 95.2 93.5 99.1 𝟓𝟖.𝟗 𝟓𝟑.𝟒 𝟔𝟗.𝟕 𝟒𝟑.𝟔 𝟑𝟓.𝟐 59.1 𝟓𝟕.𝟖 𝟓𝟐.𝟏 𝟔𝟖.𝟒
EtD 𝟗𝟕.𝟒 𝟗𝟓.𝟖 𝟗𝟗.𝟓 57.3 50.8 68.9 42.7 34.6 𝟔𝟎.𝟏 56.1 51.2 66.6
LLMs+GNNs
Ours 98.9 98.3 99.8 60.1 55.6 73.4 47.3 38.5 63.8 59.3 54.6 70.7

Table 4
Impact of GNNs-based path pruning on performance.
Models WN18RR FB15K237
MRR H@1 H@10 MRR H@1 H@10
w.o.prune 57.2 50.7 68.6 42.5 34.3 59.7
𝐿𝐺𝐾𝐺𝑅 − 𝐿𝐿𝑀𝑠𝑓 𝑖𝑛𝑎𝑙 60.1 55.6 73.4 47.3 38.5 63.8

Table 5
Impact of LLMs-based reasoning path decision on performance.
Models WN18RR FB15K237
MRR H@1 H@10 MRR H@1 H@10
𝑤.𝑜.𝐿𝐿𝑀𝑠𝑑𝑒𝑐𝑖𝑠𝑖𝑜𝑛 58.1 52.3 70.3 43.8 33.2 58.6
𝐿𝐺𝐾𝐺𝑅 − 𝐿𝐿𝑀𝑠𝑓 𝑖𝑛𝑎𝑙 60.1 55.6 73.4 47.3 38.5 63.8

Table 6
Impact of LLMs-based final reasoning result generation on performance.
Models WN18RR FB15K237
MRR H@1 H@10 MRR H@1 H@10
𝑤.𝑜.𝐿𝐿𝑀𝑠𝑓 𝑖𝑛𝑎𝑙 58.7 53.7 72.1 45.5 35.9 61.2
𝐿𝐺𝐾𝐺𝑅 − 𝐿𝐿𝑀𝑠𝑓 𝑖𝑛𝑎𝑙 60.1 55.6 73.4 47.3 38.5 63.8

Table 7
Impact of LLMs on performance.
Type Model UMLS WN18RR FB15k237 NELL-995
MRR H@1 H@10 MRR H@1 H@10 MRR H@1 H@10 MRR H@1 H@10
GLM10B-based LGKGR 85.7 79.3 96.1 49.8 44.3 56.6 37.2 28.9 56.1 50.1 45.8 61.2
Llama13B-based LGKGR 93.1 91.9 98.9 54.5 49.8 67.2 42.1 34.7 54.3 53.6 51.2 64.5
GPT3.5-based LGKGR 98.9 98.3 99.8 60.1 55.6 73.4 47.3 38.5 63.8 59.3 54.6 70.7

CRediT authorship contribution statement Acknowledgment

Yuanming Zhang: Writing – review & editing. Wenbo Zheng:

Writing – original draft, Validation, Methodology. Jiacheng Huang: The authors gratefully acknowledge the support by the National
Data curation. Gang Xiao: Funding acquisition. Natural Science Foundation of China under Grant 62476248, and the
Pioneer and Leading Goose Research and Development Program of
Declaration of competing interest Zhejiang Province China under Grant 2023C01022.

The authors declare the following financial interests/personal rela-

tionships which may be considered as potential competing interests: Data availability
Yuanming Zhang reports financial support was provided by National
Natural Science Foundation of China. If there are other authors, they
declare that they have no known competing financial interests or The authors are unable or have chosen not to specify which data
personal relationships that could have appeared to influence the work has been used.
reported in this paper.

8
Y. Zhang et al. Neurocomputing 635 (2025) 129919

References [20] L. Luo, Y.-F. Li, G. Haffari, S. Pan, Reasoning on graphs: Faithful and
interpretable large language model reasoning, 2023, ArXiv abs/2310.01061.
[1] X. Liu, T. Mao, Y. Shi, Y. Ren, Overview of knowledge reasoning for knowl- [21] J. Sun, C. Xu, L. Tang, S. Wang, C. Lin, Y. Gong, H. yeung Shum, J. Guo,
edge graph, Neurocomputing 585 (2024) 127571, http://dx.doi.org/10.1016/j. Think-on-graph: Deep and responsible reasoning of large language model with
neucom.2024.127571. knowledge graph, 2023, ArXiv abs/2307.07697.
[2] K. Bollacker, C. Evans, P. Paritosh, T. Sturge, J. Taylor, Freebase: a collabora- [22] B. Khemani, S.A. Patil, K.V. Kotecha, S. Tanwar, A review of graph neural
tively created graph database for structuring human knowledge, in: Proceedings networks: concepts, architectures, techniques, challenges, datasets, applications,
of the 2008 ACM SIGMOD International Conference on Management of Data, and future directions, J. Big Data 11 (2024) 1–43.
SIGMOD ’08, Association for Computing Machinery, New York, NY, USA, 2008, [23] X. Chen, S. Jia, Y. Xiang, A review: Knowledge reasoning over knowledge
pp. 1247–1250, http://dx.doi.org/10.1145/1376616.1376746. graph, Expert Syst. Appl. 141 (C) (2020) http://dx.doi.org/10.1016/j.eswa.2019.
[3] A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E.R. Hruschka, T.M. Mitchell, 112948.
Toward an architecture for never-ending language learning, in: Proceedings of [24] J. Gilmer, S.S. Schoenholz, P.F. Riley, O. Vinyals, G.E. Dahl, Neural message pass-
the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAI ’10, AAAI ing for quantum chemistry, ICML ’17, in: Proceedings of the 34th International
Press, 2010, pp. 1306–1313. Conference on Machine Learning, vol. 70, JMLR.org, 2017, pp. 1263–1272.
[25] J. Ma, B. Liu, K. Li, C. Li, F. Zhang, X. Luo, Y. Qiao, A review of graph
[4] O. Bodenreider, The unified medical language system (UMLS): integrating
neural networks and pretrained language models for knowledge graph reasoning,
biomedical terminology, Nucleic Acids Res. 32 Database issue (2004) D267–70.
Neurocomputing 609 (2024) 128490, http://dx.doi.org/10.1016/j.neucom.2024.
[5] P.W. Battaglia, J.B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, M.
128490.
Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, et al., Relational
[26] S. Vashishth, S. Sanyal, V. Nitin, P. Talukdar, Composition-based multi-
inductive biases, deep learning, and graph networks, 2018, arXiv 2018, arXiv
relational graph convolutional networks, in: International Conference on Learning
preprint arXiv:1806.01261.
Representations, 2020.
[6] X. Chen, S. Jia, Y. Xiang, A review: Knowledge reasoning over knowledge graph,
[27] P. Li, G. Zhou, Z. Xie, P. Xie, J.X. Huang, Learning dynamic and static
Expert Syst. Appl. 141 (2020) 112948, http://dx.doi.org/10.1016/j.eswa.2019.
representations for extrapolation-based temporal knowledge graph reasoning,
112948.
IEEE/ ACM Trans. Audio Speech Lang. Process. 32 (2024) 4741–4754, http:
[7] S. Ji, S. Pan, E. Cambria, P. Marttinen, S.Y. Philip, A survey on knowledge
//dx.doi.org/10.1109/TASLP.2024.3485500.
graphs: Representation, acquisition, and applications, IEEE Trans. Neural Netw.
[28] Z. Xie, R. Zhu, J. Liu, G. Zhou, J.X. Huang, An efficiency relation-specific
Learn. Syst. 33 (2) (2021) 494–514.
graph transformation network for knowledge graph representation learning, Inf.
[8] Y. Cao, X. Wang, X. He, Z. Hu, T.-S. Chua, Unifying knowledge graph learning
Process. Manage. 59 (6) (2022) 103076, http://dx.doi.org/10.1016/j.ipm.2022.
and recommendation: Towards a better understanding of user preferences,
103076.
in: The World Wide Web Conference, WWW ’19, Association for Computing
[29] Z. Xie, R. Zhu, J. Liu, G. Zhou, J.X. Huang, Hierarchical neighbor propagation
Machinery, New York, NY, USA, 2019, pp. 151–161, http://dx.doi.org/10.1145/
with bidirectional graph attention network for relation prediction, IEEE/ ACM
3308558.3313705.
Trans. Audio Speech Lang. Process. 29 (2021) 1762–1773, http://dx.doi.org/10.
[9] X. Wang, X. He, Y. Cao, M. Liu, T.-S. Chua, KGAT: Knowledge graph atten-
1109/TASLP.2021.3079812.
tion network for recommendation, in: Proceedings of the 25th ACM SIGKDD
[30] Y. Zhang, Z. Zhou, Q. Yao, X. Chu, B. Han, AdaProp: Learning adaptive
International Conference on Knowledge Discovery & Data Mining, KDD ’19,
propagation for graph neural network based knowledge graph reasoning, in:
Association for Computing Machinery, New York, NY, USA, 2019, pp. 950–958,
Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and
http://dx.doi.org/10.1145/3292500.3330989.
Data Mining, KDD ’23, Association for Computing Machinery, New York, NY,
[10] X. Huang, J. Zhang, D. Li, P. Li, Knowledge graph embedding based question
USA, 2023, pp. 3446–3457, http://dx.doi.org/10.1145/3580305.3599404.
answering, in: Proceedings of the Twelfth ACM International Conference on Web
[31] S. Mai, S. Zheng, Y. Yang, H. Hu, Communicative message passing for inductive
Search and Data Mining, WSDM ’19, Association for Computing Machinery,
relation reasoning, Proc. AAAI Conf. Artif. Intell. 35 (5) (2021) 4294–4302,
New York, NY, USA, 2019, pp. 105–113, http://dx.doi.org/10.1145/3289600.
http://dx.doi.org/10.1609/aaai.v35i5.16554.
3290956.
[32] Z. Xie, Y. Zhang, G. Zhou, J. Liu, X. Tu, J.X. Huang, One subgraph for
[11] P. Chandak, K. Huang, M. Zitnik, Building a knowledge graph to enable precision all: Efficient reasoning on opening subgraphs for inductive knowledge graph
medicine, 2022, http://dx.doi.org/10.1101/2022.05.01.489928, BioRxiv. completion, IEEE Trans. Knowl. Data Eng. 36 (12) (2024) 8914–8927, http:
[12] Y. Yu, K. Huang, C. Zhang, L.M. Glass, J. Sun, C. Xiao, SumGNN: multi- //dx.doi.org/10.1109/TKDE.2024.3432767.
typed drug interaction prediction via efficient knowledge graph summariza- [33] T.B. Brown, Language models are few-shot learners, in: Proceedings of the 34th
tion, Bioinformatics 37 (18) (2021) 2988–2995, http://dx.doi.org/10.1093/ International Conference on Neural Information Processing Systems, NIPS ’20,
bioinformatics/btab207, arXiv:https://academic.oup.com/bioinformatics/article- Curran Associates Inc., Red Hook, NY, USA, 2020.
pdf/37/18/2988/50579378/btab207.pdf. [34] A. Chowdhery, et al., Palm: Scaling language modeling with pathways, J. Mach.
[13] X. Xu, W. Feng, Y. Jiang, X. Xie, Z. Sun, Z.-H. Deng, Dynamically pruned message Learn. Res. 24 (240) (2023) 1–113.
passing networks for large-scale knowledge graph reasoning, in: International [35] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov,
Conference on Learning Representations, 2020. S. Batra, P. Bhargava, S. Bhosale, et al., Llama 2: Open foundation and fine-tuned
[14] W.-L. Chiang, X. Liu, S. Si, Y. Li, S. Bengio, C.-J. Hsieh, Cluster-GCN: An chat models, 2023, arXiv preprint arXiv:2307.09288.
efficient algorithm for training deep and large graph convolutional networks, in: [36] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F.L. Aleman, D. Almeida,
Proceedings of the 25th ACM SIGKDD International Conference on Knowledge J. Altenschmidt, S. Altman, S. Anadkat, et al., Gpt-4 technical report, 2023, arXiv
Discovery & Data Mining, KDD ’19, Association for Computing Machinery, preprint arXiv:2303.08774.
New York, NY, USA, 2019, pp. 257–266, http://dx.doi.org/10.1145/3292500. [37] J. Jiang, K. Zhou, W.X. Zhao, J. rong Wen, UniKGQA: Unified retrieval and
3330925. reasoning for solving multi-hop question answering over knowledge graph, 2022,
[15] Z. Zhu, Z. Zhang, L.-P. Xhonneux, J. Tang, Neural bellman-ford networks: a ArXiv abs/2212.00959.
general graph neural network framework for link prediction, in: Proceedings of [38] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K.R. Narasimhan, Y. Cao, ReAct: Syn-
the 35th International Conference on Neural Information Processing Systems, ergizing reasoning and acting in language models, in: The Eleventh International
NIPS ’21, Curran Associates Inc., Red Hook, NY, USA, 2024. Conference on Learning Representations, 2023.
[16] K.K. Teru, E.G. Denis, W.L. Hamilton, Inductive relation prediction by subgraph [39] J. Baek, A.F. Aji, A. Saffari, Knowledge-augmented language model prompting
reasoning, in: Proceedings of the 37th International Conference on Machine for zero-shot knowledge graph question answering, in: B. Dalvi Mishra, G.
Learning, ICML ’20, JMLR.org, 2020. Durrett, P. Jansen, D. Neves Ribeiro, J. Wei (Eds.), Proceedings of the 1st
[17] S. Xie, R. Liu, X. Wang, X. Luo, V. Sugumaran, H. Yu, Hierarchical Workshop on Natural Language Reasoning and Structured Explanations, NLRSE,
knowledge-enhancement framework for multi-hop knowledge graph reasoning, Association for Computational Linguistics, Toronto, Canada, 2023, pp. 78–106,
Neurocomputing 588 (2024) 127673, http://dx.doi.org/10.1016/j.neucom.2024. http://dx.doi.org/10.18653/v1/2023.nlrse-1.7.
127673. [40] G. Liu, Y. Zhang, Y. Li, Q. Yao, Explore then determine: A GNN-LLM synergy
[18] D.-T. Do, M.-P. Nguyen, L.-M. Nguyen, Enhancing zero-shot multilingual semantic framework for reasoning over knowledge graph, 2024, arXiv preprint arXiv:
parsing: A framework leveraging large language models for data augmentation 2406.01145.
and advanced prompting techniques, Neurocomputing (2024) 129108, http://dx. [41] T. Dettmers, P. Minervini, P. Stenetorp, S. Riedel, Convolutional 2D knowledge
doi.org/10.1016/j.neucom.2024.129108. graph embeddings, in: Proceedings of the Thirty-Second AAAI Conference on Ar-
[19] J. Jiang, K. Zhou, Z. Dong, K. Ye, X. Zhao, J.-R. Wen, StructGPT: A general tificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence
framework for large language model to reason over structured data, in: H. Conference and Eighth AAAI Symposium on Educational Advances in Artificial
Bouamor, J. Pino, K. Bali (Eds.), Proceedings of the 2023 Conference on Em- Intelligence, in: AAAI’18/IAAI’18/EAAI’18, AAAI Press, 2018.
pirical Methods in Natural Language Processing, Association for Computational [42] S. Zhang, Y. Tay, L. Yao, Q. Liu, Quaternion knowledge graph embeddings,
Linguistics, Singapore, 2023, pp. 9237–9251, http://dx.doi.org/10.18653/v1/ in: Proceedings of the 33rd International Conference on Neural Information
2023.emnlp-main.574. Processing Systems, Curran Associates Inc., Red Hook, NY, USA, 2019.

9
Y. Zhang et al. Neurocomputing 635 (2025) 129919

[43] Z. Sun, Z.-H. Deng, J.-Y. Nie, J. Tang, RotatE: Knowledge graph embedding by Wenbo Zheng was born in China in 1998. He received
relational rotation in complex space, in: International Conference on Learning the B.Eng. degree in Software Engineering from Nanchang
Representations, 2019. Hangkong University, Nanchang, China, in 2021. He is
[44] R. Das, S. Dhuliawala, M. Zaheer, L. Vilnis, I. Durugkar, A. Krishnamurthy, currently pursuing the Ph.D degree with the Computer
A. Smola, A. McCallum, Go for a walk and arrive at the answer: Reasoning Science and Technology, Zhejiang University of Technology,
over paths in knowledge bases using reinforcement learning, in: International Hangzhou, China. His current research interests include
Conference on Learning Representations, 2018. heterogeneous graph representation learning and knowledge
[45] A. Sadeghian, M. Armandpour, P. Ding, D.Z. Wang, DRUM: end-to-end differen- discovery & reasoning.
tiable rule mining on knowledge graphs, in: Proceedings of the 33rd International
Conference on Neural Information Processing Systems, Curran Associates Inc.,
Red Hook, NY, USA, 2019.
[46] M. Qu, J. Chen, L.-P. Xhonneux, Y. Bengio, J. Tang, RNNLogic: Learning logic Jiacheng Huang was born in China in 1998. He received
rules for reasoning on knowledge graphs, 2020, ArXiv abs/2010.04029. the B.Eng. degree in Software Engineering from Nanchang
[47] K. Cheng, J. Liu, W. Wang, Y. Sun, RLogic: Recursive logical rule learning from Hangkong University, Nanchang, China, in 2021. He is
knowledge graphs, in: Proceedings of the 28th ACM SIGKDD Conference on currently pursuing the Ph.D degree with the Computer
Knowledge Discovery and Data Mining, KDD ’22, Association for Computing Science and Technology, Zhejiang University of Tech-
Machinery, New York, NY, USA, 2022, pp. 179–189, http://dx.doi.org/10.1145/ nology, Hangzhou, China. His current research interests
3534678.3539421. include graph neural networks and prognostics & health
management (PHM).

Yuanming Zhang received the Ph.D. degree in Department

of Information Science from Utsunomiya University, Japan,
in 2010. He currently is an Associate Professor with College Gang Xiao received the Ph.D. degree in College of Mechan-
of Computer Science and Technology in Zhejiang Univer- ical Engineering from Zhejiang University of Technology,
sity of Technology China. His current research interests China, in 2011. He is currently a Professor with College of
include data processing and analysis, graph neural networks, Computer Science and Technology in Zhejiang University
knowledge graph, and prognostics & health management of Technology China. His current research interests in-
(PHM). clude intelligent manufacturing, knowledge graph, big data
processing & analysis, and service commuting.

10