Multi-hop question answering by integrating large language models and knowledge graphs

JIANG Xian; WANG Hanyi; YANG Shiting; CHEN Xingyu; ZHOU Mengyao; DONG Ligang

doi:10.11959/j.issn.1000-0801.2025238

您当前的位置：

首页 >

文章列表页 >

Multi-hop question answering by integrating large language models and knowledge graphs

Topic: AI-Empowered Communication Networks | 更新时间：2026-01-20

- Multi-hop question answering by integrating large language models and knowledge graphs
- Telecommunications Science Vol. 41, Issue 11, Pages: 67-83(2025)
- 作者机构：
  
  浙江工商大学统计与数据科学学院，浙江杭州 310018
- 作者简介：
- 基金信息：
  
  Zhejiang Lingyan Research and Development Project(2025C02024)
- DOI：10.11959/j.issn.1000-0801.2025238
  CLC： TP301
- Received：29 May 2025，
  
  Revised：2025-07-22，
  
  Accepted：06 August 2025，
  
  Published：20 November 2025
- 稿件说明：
移动端阅览
蒋献,王涵亦,杨诗婷等.基于大语言模型与知识图谱融合的多跳问答技术研究[J].电信科学,2025,41(11):67-83.

JIANG Xian,WANG Hanyi,YANG Shiting,et al.Multi-hop question answering by integrating large language models and knowledge graphs[J].Telecommunications Science,2025,41(11):67-83.
蒋献,王涵亦,杨诗婷等.基于大语言模型与知识图谱融合的多跳问答技术研究[J].电信科学,2025,41(11):67-83. DOI： 10.11959/j.issn.1000-0801.2025238.

JIANG Xian,WANG Hanyi,YANG Shiting,et al.Multi-hop question answering by integrating large language models and knowledge graphs[J].Telecommunications Science,2025,41(11):67-83. DOI： 10.11959/j.issn.1000-0801.2025238.

摘要

随着大语言模型在自然语言处理任务中的广泛应用，提升其在垂直领域问答任务中的表现成为当前研究的重点方向。针对传统方法在复杂多跳推理任务中存在的局限，提出了一种融合知识图谱的多跳问答方法（LLMKG）。该方法通过在Prompt中注入知识图谱中的事实知识，有效提升了大语言模型在特定领域问答中的推理能力。在COKG-DATA数据集上的对比实验显示，LLMKG在Hits@1指标上较最优基线模型提升了3.5%。该方法具备零样本能力，适用于各类大语言模型，且无须额外参数更新。还探讨了时间性知识增强与多模态知识融合的潜力，并提出构建多模态知识图谱（MMKG）作为未来的发展方向。该方法为面向垂直领域的智能问答系统提供了新的研究思路和可行的技术路径。

Abstract

With the widespread application of large language models (LLM) in natural language processing tasks

improving their performance in domain-specific question answering has become a key research focus. To address the limitations of traditional methods in complex multi-hop reasoning tasks

a knowledge-graph-enhanced multi-hop question answering approach

LLMKG

was proposed. Factual knowledge from knowledge graphs was injected into Prompt to enhance the reasoning capability of LLM in vertical domains. Comparative experiments conducted on the COKG-DATA dataset show that LLMKG outperforms the best baseline by 3.5% in terms of Hits@1. The method operats in a zero-shot setting

requires no additional parameter updates

and is applicable to various types of LLM. Temporal knowledge enhancement and multimodal knowledge fusion strategies were further explored

and a multi-modal knowledge graph (MMKG) was proposed as a future direction. This approach offers a novel and effective pathway for advancing intelligent question answering systems in specialized domains.

关键词

Keywords

references

CHEN R R , JIANG W F , QIN C W , et al . LLM-based multi-hop question answering with knowledge graph integration in evolving environments [C ] // Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024 . Stroudsburg : ACL , 2024 : 14438 - 14451 .

TAN X Y , WANG X Y , LIU Q , et al . Paths-over-graph: knowledge graph empowered large language model reasoning [C ] // Proceedings of the ACM on Web Conference 2025 . New York : ACM Press , 2025 : 3505 - 3522 .

ZHOU T , CHEN Y B , LIU K , et al . CogMG: collaborative augmentation between large language model and knowledge graph [C ] // Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations) . Stroudsburg : ACL , 2024 : 365 - 373 .

PAN S R , LUO L H , WANG Y F , et al . Unifying large language models and knowledge graphs: a roadmap [J ] . IEEE Transactions on Knowledge and Data Engineering , 2024 , 36 ( 7 ): 3580 - 3599 .

HUANG M L , DUAN L , ZHANG Y H , et al . Prompt learning based unsupervised relation extraction model [J ] . Journal of Computer Applications , 2023 , 43 ( 7 ): 2010 - 2016 .

LINDERS J , TOMCZAK J M . Knowledge graph-extended retrieval augmented generation for question answering [EB ] . 2025 .

LU Y F , ZHOU Y G , LI J , et al . Knowledge editing with dynamic knowledge graphs for multi-hop question answering [C ] // Proceedings of the AAAI Conference on Artificial Intelligence . Palo Alto : AAAI Press , 2025 , 39 ( 23 ): 24741 - 24749 .

XU Y , HE S Z , CHEN J B , et al . Generate-on-graph: treat LLM as both agent and KG for incomplete knowledge graph question answering [C ] // Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : ACL , 2024 : 18410 - 18430 .

YANG Z K , ZHU Z X , ZHU J . CuriousLLM: elevating multi-document question answering with LLM-enhanced knowledge graph reasoning [C ] // Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track) . Stroudsburg : ACL , 2025 : 274 - 286 .

LI Y D , SONG D D , ZHOU C Z , et al . A framework of knowledge graph-enhanced large language model based on question decomposition and atomic retrieval [C ] // Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2024 . Stroudsburg : ACL , 2024 : 11472 - 11485 .

SHU D , CHEN T , JIN M , et al . . Knowledge graph large language model (KG-LLM) for link prediction [C ] // Proceedings of the 16th Asian Conference on Machine Learning Research Oxford: Pergamon Press , 2025 : 143 - 158 .

LIU P F , YUAN W Z , FU J L , et al . Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing [J ] . ACM Computing Surveys , 2023 , 55 ( 9 ): 1 - 35 .

CHEN X , ZHANG N Y , XIE X , et al . KnowPrompt: knowledge-aware Prompt-tuning with synergistic optimization for relation extraction [C ] // Proceedings of the ACM Web Conference 2022 . New York : ACM Press , 2022 : 2778 - 2788 .

张春 , 张宁 , 许亮 . 一种基于Prompt多模板融合的零样本关系抽取方法 : CN115687609A [P ] . 2023 .

ZHANG C , ZHANG N , XU L . A zero-shot relation extraction method based on Prompt multi-template fusion : CN115687609A [P ] . 2023 .

LUO L , LI Y F , HAFFARI G , et al . Reasoning on graphs: faithful and interpretable large language model reasoning [C ] // Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024) . Vienna : ICLR , 2024 .

ZHAO H Y , CHEN H J , YANG F , et al . Explainability for large language models: a survey [J ] . ACM Transactions on Intelligent Systems and Technology , 2024 , 15 ( 2 ): 1 - 38 .

WU Y , HU N , QI G , et al . Retrieve-rewrite-answer: a KG-to-text enhanced LLM framework for knowledge graph question answering [C ] // Proceedings of the 12th International Joint Conference on Knowledge Graphs (IJCKG’23) . Tokyo : IJCKG , 2023 : 10  .

CADEDDU A , CHESSA A , DE LEO V , et al . A comparative analysis of knowledge injection strategies for large language models in the scholarly domain [J ] . Engineering Applications of Artificial Intelligence , 2024 , 133 : 108166 .

DU H F , LE Z W , WANG H F , et al . COKG-QA: multi-hop question answering over COVID-19 KnowledgeGraphs [J ] . Data Intelligence , 2022 , 4 ( 3 ): 471 - 492 .

WANG J B , LEI J , SUN S N , et al . Embeddings based on relation-specific constraints for open world knowledge graph completion [J ] . Applied Intelligence , 2023 , 53 ( 12 ): 16192 - 16204 .

CHEN Z Y , ZHAO X , LIAO J Z , et al . Temporal knowledge graph question answering via subgraph reasoning [J ] . Knowledge-Based Systems , 2022 , 251 : 109134 .

ZHENG R , DOU S H , GAO S Y , et al . Secrets of RLHF in large language models part I: PPO [EB ] . 2023 .

CUI Y M , YANG Z Q , YAO X , et al . Efficient and effective text encoding for Chinese LLaMA and alpaca [EB ] . 2023 .

DU Z X , QIAN Y J , LIU X , et al . GLM: general language model pretraining with autoregressive blank infilling [C ] // Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : ACL , 2022 : 320 - 335 .

YANG A Y , XIAO B , WANG B N , et al . Baichuan 2: open large-scale language models [EB ] . 2023 .

BAI J Z , BAI S , CHU Y F , et al . Qwen technical report [EB ] . 2023 .

YANG A , LI A F , YANG B S , et al . Qwen3 technical report [EB ] . 2025 .

LU H Y , LIU W , ZHANG B , et al . DeepSeek-VL: towards real-world vision-language understanding [EB ] . 2024 .

DUBEY A , JAUHRI A , PANDEY A , et al . The llama 3 herd of models [EB ] . 2024 .

QWEN TEAM . Qwen1.5 technical report [R ] . 2024 .

ZHANG Y , LI H , WANG X , et al . Efficiency analysis of large Chinese LLM [EB ] . 2024 .

RAJBHANDARI S , LI C L , YAO Z W , et al . DeepSpeed-MoE: advancing mixture-of-experts inference and training to power next-generation AI scale [EB ] . 2022 .

BAI C , ZHOU Y , ZHANG Z , et al . Long context: beyond 100 K tokens [C ] // Proceedings of the 32th Conference on Empirical Methodsin Natural Language Processing (EMNLP) . Stroudsburg : ACL , 2023 : 4980 - 4995 .

ZHOU Y Q , LEI T , LIU H X , et al . Mixture-of-experts with expert choice routing [C ] // Proceedings of the 36th Conference on Neural Information Processing Systems . West Chester : Curran Associates, Inc. , 2022 .

SAXENA A , TRIPATHI A , TALUKDAR P . Improving multi-hop question answering over knowledge graphs using knowledge base embeddings [C ] // Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Stroudsburg : ACL , 2020 : 4498 - 4507 .

SHI J X , CAO S L , HOU L , et al . TransferNet: an effective and transparent framework for multi-hop question answering over relation graph [C ] // Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Stroudsburg : ACL , 2021 : 4149 - 4158 .

WANG D Q , WU J , GUAN L W . Industry knowledge base construction based on knowledge graph [J ] . Manufacturing Technology & Machine Tool ( 8 ): 74 - 80 .

MA A , YU Y H , YANG S L , et al . Survey of knowledge graph based on reinforcement learning [J ] . Computer Research and Development , 2022 , 59 ( 8 ).

RIBEIRO L F R , ZHANG Y , GARDENT C , et al . Modeling global and local node contexts for text generation from knowledge graphs [J ] . Transactions of the Association for Computational Linguistics , 2020 , 8 : 589 - 604 .

SHARMA A , SAXENA A , GUPTA C , et al . TwiRGCN: temporally weighted graph convolution for question answering over temporal knowledge graphs [C ] // Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics . Stroudsburg : ACL , 2023 : 2049 - 2060 .

CHEN Z Y , LIAO J Z , ZHAO X . Multi-granularity temporal question answering over knowledge graphs [C ] // Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : ACL , 2023 : 11378 - 11392 .

ZHU X R , LI Z X , WANG X D , et al . Multi-modal knowledge graph construction and application: a survey [J ] . IEEE Transactions on Knowledge and Data Engineering , 2024 , 36 ( 2 ): 715 - 735 .

KEJRIWAL M , KNOBLOCK C A , SZEKELY P . Knowledge graphs: fundamentals, techniques, and applications [M ] . Cambridge : The MIT Press , 2021 .

LEE J L , WANG Y Q , LI J , et al . Multimodal reasoning with multimodal knowledge graph [C ] // Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Stroudsburg : ACL , 2024 : 10767 - 10782 .

HE X H , WANG X . Multimodal graph transformer for multimodal question answering [C ] // Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics . Stroudsburg : ACL , 2023 : 189 - 200 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Large language model for cloud-network configuration audit and its application in IP network

A large language model-enhanced few-shot learning algorithm for network complaint intent recognition

Research on multi-heterogeneous hybrid training system for AI computing power scenarios

A data and knowledge-driven practice for ensuring stability in ultra-large intelligent computing clusters

Empirical study on the evolution of telecom fraud risks driven by artificial intelligence generated content

Related Author

XIE Liqin

ZHU Wenhui

SHU Dong

DING Yu

ZHANG Zheng

LIN Wentong

OUYANG Ye

DING Jianbing

Related Institution

China Telecom Co., Ltd. Jiangsu Branch

China Mobile Communications Group Co., Ltd.

AsiaInfo Technologies (China), Inc

(Suzhou) Software Technology Co., Ltd.

China China Mobile Group Design Institute Co., Ltd.

⁰