Knowledge Graph Combined with Retrieval-Augmented Generation for Enhancing LMs Reasoning: A Survey

Haitao Wang; Yangkun Shi

doi:10.54097/h21fky45

Authors

Haitao Wang
Yangkun Shi

DOI:

https://doi.org/10.54097/h21fky45

Keywords:

Large Language Models; Natural Language Processing; Retrieval-Augmented Generation; Knowledge Graphs; Knowledge Reasoning.

Abstract

Large language models (LLMs) have generated significant waves in the fields of Natural Language Processing(NLP) and artificial intelligence due to their remarkable capabilities and broad adaptability. Retrieval-Augmented Generation (RAG) techniques have been widely adopted, leveraging external retrieval systems to significantly improve the timeliness of LLMs and substantially reduce hallucinations. Although RAG and its optimization methods have addressed most hallucination issues caused by knowledge gaps and outdated information, text generation in specialized domains such as law, medicine, and science—which require multi-hop reasoning and analysis—still suffers from a lack of coherence and logical consistency, making it difficult to produce correct and valuable answers. To address these challenges, related research has introduced Knowledge Graphs (KGs). This integrated approach enhances RAG’s capabilities by utilizing the structured knowledge provided by KGs, thereby improving the model's knowledge representation and reasoning abilities and enabling the generation of more accurate answers. However, there remains a lack of systematic reviews in this area. Therefore, this paper provides a comprehensive review of studies on enhancing LLMs reasoning abilities by integrating KGs with RAG. It first introduces the basic concepts, followed by an overview of the current mainstream technical approaches, and concludes with a discussion of the research challenges and future development trends in this field.

Downloads

Download data is not yet available.

References

[1] Achiam J, Adler S, Agarwal S, et al. Gpt-4 technical report[J]. arXiv preprint arXiv:2303.08774, 2023.

[2] Dubey A, Jauhri A, Pandey A, et al. The llama 3 herd of models[J]. arXiv preprint arXiv:2407.21783, 2024.

[3] Team G, Anil R, Borgeaud S, et al. Gemini: a family of highly capable multimodal models[J]. arXiv preprint arXiv:2312.11805, 2023.

[4] Kandpal N, Deng H, Roberts A, et al. Large language models struggle to learn long-tail knowledge[C]//International Conference on Machine Learning. PMLR, 2023: 15696-15707

[5] Zhang Y, Li Y, Cui L, et al. Siren's song in the AI ocean: a survey on hallucination in large language models[J]. arXiv preprint arXiv:2309.01219, 2023.

[6] Kandpal N, Deng H, Roberts A, et al. Large language models struggle to learn long-tail knowledge[C]//International Conference on Machine Learning. PMLR, 2023: 15696-15707.

[7] Cheng X, Luo D, Chen X, et al. Lift yourself up: Retrieval-augmented text generation with self-memory[J]. Advances in Neural Information Processing Systems, 2024, 36.

[8] Liu N F, Lin K, Hewitt J, et al. Lost in the middle: How language models use long contexts[J]. Transactions of the Association for Computational Linguistics, 2024, 12: 157-173.

[9] Singhal A. Introducing the knowledge graph: things, not strings[J]. Official google blog, 2012, 5(16): 3.

[10] Ji S, Pan S, Cambria E, et al. A survey on knowledge graphs: Representation, acquisition, and applications[J]. IEEE transactions on neural networks and learning systems, 2021, 33(2): 494-514.

[11] Tresp, V., Gärtner, T., & Nickel, M. (2021). Dynamic knowledge graphs: A survey. Artificial Intelligence Review, 55, 349–392.

[12] Chen D. Reading Wikipedia to answer open‐domain questions[J]. arXiv preprint arXiv:1704.00051, 2017.

[13] Karpukhin V, Oğuz B, Min S, et al. Dense passage retrieval for open-domain question answering [J]. arXiv preprint arXiv: 2004.04906, 2020.

[14] Lewis P, Perez E, Piktus A, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks[J]. Advances in Neural Information Processing Systems, 2020, 33: 9459-9474.

[15] Gunawan D, Sembiring C A, Budiman M A. The implementation of cosine similarity to calculate text relevance between two documents[C]//Journal of physics: conference series. IOP Publishing, 2018, 978: 012120.

[16] Ramelan M. A Query Expansion Using Support Vector Machine (SVM) and Best Matching 25 (BM25)[J]. International Journal of Computer and Information System (IJCIS), 2024, 5(2): 73-77.

[17] Jiang X, Zhang R, Xu Y, et al. HyKGE: A Hypothesis Knowledge Graph Enhanced Framework for Accurate and Reliable Medical LLMs Responses[J]. arXiv preprint arXiv:2312.15883, 2024.

[18] Jin B, Xie C, Zhang J, et al. Graph chain-of-thought: Augmenting large language models by reasoning on graphs[J]. arXiv preprint arXiv:2404.07103, 2024.

[19] Luo L, Li Y F, Haffari G, et al. Reasoning on graphs: Faithful and interpretable large language model reasoning[J]. arXiv preprint arXiv:2310.01061, 2023.

[20] Ma S, Xu C, Jiang X, et al. Think-on-graph 2.0: Deep and interpretable large language model reasoning with knowledge graph-guided retrieval [J]. arXiv e-prints, 2024: arXiv: 2407.10805.

[21] Yang H, Liu J. Knowledge graph representation learning as groupoid: unifying TransE, RotatE, QuatE, ComplEx [C]// Proceedings of the 30th ACM international conference on information & knowledge management. 2021: 2311-2320.

[22] Yasunaga M, Ren H, Bosselut A, et al. QA-GNN: Reasoning with language models and knowledge graphs for question answering[J]. arXiv preprint arXiv:2104.06378, 2021.

[23] Taunk D, Khanna L, Kandru S V P K, et al. GrapeQA: Graph augmentation and pruning to enhance question-answering[C]//Companion Proceedings of the ACM Web Conference 2023. 2023: 1138-1144.

[24] He X, Tian Y, Sun Y, et al. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering[J]. arXiv preprint arXiv:2402.07630, 2024.

[25] Delile J, Mukherjee S, Van Pamel A, et al. Graph-Based Retriever Captures the Long Tail of Biomedical Knowledge[J]. arXiv preprint arXiv:2402.12352, 2024.

[26] Mavromatis C, Karypis G. GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning[J]. arXiv preprint arXiv:2405.20139, 2024.

[27] Zhang J, Zhang X, Yu J, et al. Subgraph retrieval enhanced model for multi-hop knowledge base question answering[J]. arXiv preprint arXiv:2202.13296, 2022.

[28] Liu Y. Roberta: A robustly optimized bert pretraining approach[J]. arXiv preprint arXiv:1907.11692, 2019, 364.

[29] Kim J, Kwon Y, Jo Y, et al. Kg-gpt: A general framework for reasoning on knowledge graphs using large language models[J]. arXiv preprint arXiv:2310.11220, 2023.

[30] Wold S, Øvrelid L, Velldal E. Text-To-KG Alignment: Comparing Current Methods on Classification Tasks[J]. arXiv preprint arXiv:2306.02871, 2023.

[31] Jiang J, Zhou K, Dong Z, et al. Structgpt: A general framework for large language model to reason over structured data[J]. arXiv preprint arXiv:2305.09645, 2023.

[32] Liu G, Zhang Y, Li Y, et al. Explore then Determine: A GNN-LLM Synergy Framework for Reasoning over Knowledge Graph[J]. arXiv preprint arXiv:2406.01145, 2024.

[33] Kenton J D M W C, Toutanova L K. Bert: Pre-training of deep bidirectional transformers for language understanding [C] // Proceedings of naacL-HLT. 2019, 1: 2.

[34] Raffel C, Shazeer N, Roberts A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of machine learning research, 2020, 21(140): 1-67.

[35] Driess D, Xia F, Sajjadi M S M, et al. Palm-e: An embodied multimodal language model [J]. arXiv preprint arXiv: 2303.03378, 2023.

[36] Cheng K, Lin G, Fei H, et al. Multi-hop question answering under temporal knowledge editing [J]. arXiv preprint arXiv: 2404.00492, 2024.

[37] Huang Y, Li Y, Xu Y, et al. Mvp-tuning: Multi-view knowledge retrieval with prompt tuning for commonsense reasoning[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023: 13417-13432.

[38] An Z, Ding X, Fu Y C, et al. Golden-Retriever: High-Fidelity Agentic Retrieval Augmented Generation for Industrial Knowledge Base[J]. arXiv preprint arXiv:2408.00798, 2024.

[39] Choudhary N, Reddy C K. Complex logical reasoning over knowledge graphs using large language models[J]. arXiv preprint arXiv:2305.01157, 2023.

[40] Kim J, Kwon Y, Jo Y, et al. Kg-gpt: A general framework for reasoning on knowledge graphs using large language models[J]. arXiv preprint arXiv:2310.11220, 2023.

[41] Li S, Gao Y, Jiang H, et al. Graph reasoning for question answering with triplet retrieval[J]. arXiv preprint arXiv: 2305.18742, 2023.