A Review of Retrieval, Context Compression, and Evidence Grounding Strategies for Efficient Locally Deployed PDF-Based RAG Systems

Authors

  • Shaoyuan Wu

DOI:

https://doi.org/10.54097/tf4q3454

Keywords:

Retrieval-Augmented Generation (RAG); Locally Deployed RAG; Context compression; Large Language Model (LLM); Evidence Grounding

Abstract

Retrieval-Augmented Generation (RAG) is widely used to connect large language models (LLMs) with external documents. RAG enables language models to retrieve evidence from external documents based on the content of those files. In local deployment, documents such as textbooks, research papers and technical reports in PDF format can be processed without relying on external services, which may protect users’ privacy. However, RAG systems based on local PDFs still face many challenges, including difficulties in efficiently and accurately extracting metadata from PDFs, noisy text produced by PDF parsing or OCR, inference efficiency, incomplete evidence selection, and the potential for “hallucinations” in generated answers. This paper looks at three core strategies for boosting local PDF-RAG performance: better retrieval, smarter context compression, and more rigorous evidence grounding. On the retrieval side, the goal is to pull out relevant paragraphs and refine how passages are ranked across large document sets. Context compression, meanwhile, steps in before generation to cut through redundant content, which helps keep inference costs in check. Evidence grounding takes a different angle—it checks whether the answers actually align with the retrieved material and, if not, defaults to a "no answer" response when the evidence falls short. After walking through and comparing these approaches, the paper argues that a well-functioning local RAG system isn't about optimizing any single piece in isolation. It needs to juggle retrieval precision, computational expense, citation reliability, and response faithfulness all at once—and that balancing act is where the real challenge lies.

References

[1] Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Guo, M., & Wang, H. (2023). Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv preprint arXiv:2312.10997. https://doi.org/10.48550/arXiv.2312.10997

[2] Jiang, H., Wu, Q., Luo, X., Li, D., Lin, C.-Y., Yang, Y., & Qiu, L. (2024). LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression. In Proceedings of ACL 2024 (pp. 1658–1677). https://doi.org/10.18653/v1/2024.acl-long.91

[3] Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., & Li, Q. (2024). A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 6491–6501). https://doi.org/10.1145/3637528.3671470

[4] Khan, A. A., Hasan, M. T., Kemell, K. K., Rasku, J., & Abrahamsson, P. (2024). Developing Retrieval Augmented Generation (RAG) Based LLM Systems from PDFs: An Experience Report. arXiv preprint arXiv:2410.15944. https://doi.org/10.48550/arXiv.2410.15944

[5] Wang, X., Wang, Z., Gao, X., Zhang, F., Wu, Y., Xu, Z., Shi, T., Wang, Z., Li, S., Qian, Q., Yin, R., Lv, C., Zheng, X., & Huang, X. (2024). Searching for Best Practices in Retrieval-Augmented Generation. In Proceedings of EMNLP 2024 (pp. 17716–17736). https://doi.org/10.18653/v1/2024.emnlp-main.981

[6] Santhanam, K., Khattab, O., Saad-Falcon, J., Potts, C., & Zaharia, M. (2022). ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In Proceedings of NAACL 2022 (pp. 3715–3734). https://doi.org/10.18653/v1/2022.naacl-main.272

[7] Ma, X., Lin, S.-C., Li, M., Chen, W., & Lin, J. (2024). Unifying Multimodal Retrieval via Document Screenshot Embedding. In Proceedings of EMNLP 2024 (pp. 6492–6505). https://doi.org/10.18653/v1/2024.emnlp-main.373

[8] Suri, M., Mathur, P., Dernoncourt, F., Goswami, K., Rossi, R. A., & Manocha, D. (2025). VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation. In Proceedings of NAACL 2025 (pp. 6088–6109). https://doi.org/10.18653/v1/2025.naacl-long.310

[9] Zhao, Q., Wang, R., Cen, Y., Zha, D., Tan, S., Dong, Y., & Tang, J. (2024). LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering. In Proceedings of EMNLP 2024 (pp. 22600–22632). https://doi.org/10.18653/v1/2024.emnlp-main.1259

[10] Xu, F., Shi, W., & Choi, E. (2024). RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation. In Proceedings of the International Conference on Learning Representations. https://openreview.net/forum?id=mlJLVigNHp

[11] Zhao, X., Li, D., Zhong, Y., Hu, B., Chen, Y., Hu, B., & Zhang, M. (2024). SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generation. In Proceedings of EMNLP 2024 (pp. 3027–3041). https://doi.org/10.18653/v1/2024.emnlp-main.178

[12] Hwang, T., Cho, S., Jeong, S., Song, H., Han, S., & Park, J. C. (2025). EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation. In Findings of ACL 2025 (pp. 4895–4924). https://doi.org/10.18653/v1/2025.findings-acl.253

[13] Gao, T., Yen, H., Yu, J., & Chen, D. (2023). Enabling Large Language Models to Generate Text with Citations. In Proceedings of EMNLP 2023 (pp. 6465–6488). https://doi.org/10.18653/v1/2023.emnlp-main.398

[14] Tang, L., Laban, P., & Durrett, G. (2024). MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents. In Proceedings of EMNLP 2024 (pp. 8818–8847). https://doi.org/10.18653/v1/2024.emnlp-main.499

[15] Ma, X., Zhuang, S., Koopman, B., Zuccon, G., Chen, W., & Lin, J. (2025). VISA: Retrieval Augmented Generation with Visual Source Attribution. In Proceedings of ACL 2025 (pp. 30154–30169). https://doi.org/10.18653/v1/2025.acl-long.1456

[16] Ye, X., Sun, R., Arik, S. O., & Pfister, T. (2024). Effective Large Language Model Adaptation for Improved Grounding and Citation Generation. In Proceedings of NAACL 2024 (pp. 6237–6251). https://doi.org/10.18653/v1/2024.naacl-long.346

[17] Xia, S., Wang, X., Liang, J., Zhang, Y., Zhou, W., Deng, J., Yu, F., & Xiao, Y. (2025). Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation. In Findings of NAACL 2025 (pp. 969–988). https://doi.org/10.18653/v1/2025.findings-naacl.55

[18] Es, S., James, J., Espinosa-Anke, L., & Schockaert, S. (2024). RAGAs: Automated Evaluation of Retrieval Augmented Generation. In Proceedings of EACL 2024: System Demonstrations (pp. 150–158). https://doi.org/10.18653/v1/2024.eacl-demo.16

[19] Niu, C., Wu, Y., Zhu, J., Xu, S., Shum, K., Zhong, R., Song, J., & Zhang, T. (2024). RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models. In Proceedings of ACL 2024 (pp. 10862–10878). https://doi.org/10.18653/v1/2024.acl-long.585

[20] Wu, D., Gu, J.-C., Yin, F., Peng, N., & Chang, K.-W. (2024). Synchronous Faithfulness Monitoring for Trustworthy Retrieval-Augmented Generation. In Proceedings of EMNLP 2024 (pp. 9390–9406). https://doi.org/10.18653/v1/2024.emnlp-main.527

Downloads

Published

07-11-2026

Issue

Section

Articles

How to Cite

Wu, S. (2026). A Review of Retrieval, Context Compression, and Evidence Grounding Strategies for Efficient Locally Deployed PDF-Based RAG Systems. Mathematical Modeling and Algorithm Application, 10(1), 74-78. https://doi.org/10.54097/tf4q3454