A Literature Review on Multimodal Misinformation Detection with External Knowledge Augmentation
DOI:
https://doi.org/10.54097/84s03s91Keywords:
Misinformation Detection, Multimodal Learning, External Knowledge, Semantic Enhancement, Text-image Consistency, Knowledge FusionAbstract
Social media misinformation is increasingly presented through combinations of headlines, body text, images, comments, interaction traces, and source metadata. This shift complicates detection because a misleading post may arise not from an explicitly false sentence, but from an authentic image reused in another event, a weakly related headline, a satirical publishing context, or missing background information. Recent multimodal and vision-language models improve the joint representation of textual and visual signals, yet most of them still operate mainly on evidence contained in the sample. They therefore have limited access to factual context, entity relations, or information about the platform and source. External knowledge augmentation offers one way to address this gap by adding knowledge-graph facts, retrieved evidence, event descriptions, entity links, or source-context information. This review examines how such knowledge is represented and combined with multimodal features, with attention to commonly used datasets, fusion strategies, and evaluation practices. Particular emphasis is placed on three unresolved issues: noisy or irrelevant knowledge, leakage caused by label-bearing descriptions, and the gap between semantic alignment and genuine fact verification. The discussion is intended to clarify when external knowledge is useful and what controls are required for reliable and interpretable multimodal misinformation detection.
Downloads
References
[1] Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017). Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations Newsletter, 19(1), 22–36.
[2] Zhou, X., & Zafarani, R. (2020). A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys, 53(5), Article 109.
[3] Nakamura, K., Levy, S., & Wang, W. Y. (2020). r/Fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection. In Proceedings of the 12th Language Resources and Evaluation Conference (pp. 6149–6157). European Language Resources Association.
[4] Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., et al. (2021). Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (Vol. 139, pp. 8748–8763). PMLR.
[5] Wang, W. Y. (2017). “Liar, liar pants on fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (pp. 422–426). Association for Computational Linguistics.
[6] Shu, K., Mahudeswaran, D., Wang, S., Lee, D., & Liu, H. (2020). FakeNewsNet: A data repository with news content, social context and spatiotemporal information for studying fake news on social media. Big Data, 8(3), 171–188.
[7] Zubiaga, A., Liakata, M., Procter, R., Wong Sak Hoi, G., & Tolmie, P. (2016). Analysing how people orient to and spread rumours in social media by looking at conversational threads. PLOS ONE, 11(3), Article e0150989.
[8] Jin, Z., Cao, J., Guo, H., Zhang, Y., & Luo, J. (2017). Multimodal fusion with recurrent neural networks for rumor detection on microblogs. In Proceedings of the 25th ACM International Conference on Multimedia (pp. 795–816). ACM.
[9] Boididou, C., Andreadou, K., Papadopoulos, S., Dang-Nguyen, D. T., Boato, G., Riegler, M., & Kompatsiaris, Y. (2015). Verifying multimedia use at MediaEval 2015. In Working Notes Proceedings of the MediaEval 2015 Workshop (Vol. 1436). CEUR Workshop Proceedings.
[10] Wang, Y., Ma, F., Jin, Z., Yuan, Y., Xun, G., Jha, K., Su, L., & Gao, J. (2018). EANN: Event adversarial neural networks for multi-modal fake news detection. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 849–857). ACM.
[11] Singhal, S., Shah, R. R., Chakraborty, T., Kumaraguru, P., & Satoh, S. (2019). SpotFake: A multi-modal framework for fake news detection. In 2019 IEEE Fifth International Conference on Multimedia Big Data (pp. 39–47). IEEE.
[12] Khattar, D., Goud, J. S., Gupta, M., & Varma, V. (2019). MVAE: Multimodal variational autoencoder for fake news detection. In The World Wide Web Conference (pp. 2915–2921). ACM.
[13] Zhou, X., Wu, J., & Zafarani, R. (2020). SAFE: Similarity-aware multi-modal fake news detection. In Advances in Knowledge Discovery and Data Mining (pp. 354–367). Springer.
[14] Dun, Y., Tu, K., Chen, C., Hou, C., & Yuan, X. (2021). KAN: Knowledge-aware attention network for fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 35, No. 1, pp. 81–89).
[15] Hu, L., Yang, T., Zhang, L., Zhong, W., Tang, D., Shi, C., Duan, N., & Zhou, M. (2021). Compare to the knowledge: Graph neural fake news detection with external knowledge. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (pp. 754–763). Association for Computational Linguistics.
[16] Qian, S., Hu, J., Fang, Q., & Xu, C. (2021). Knowledge-aware multi-modal adaptive graph convolutional networks for fake news detection. ACM Transactions on Multimedia Computing, Communications, and Applications, 17(3), Article 98.
[17] Gao, X., Wang, X., Chen, Z., Zhou, W., & Hoi, S. C. H. (2024). Knowledge enhanced vision and language model for multi-modal fake news detection. IEEE Transactions on Multimedia, 26, 8312–8322.
[18] Zhang, L., Zhang, X., Zhou, Z., Huang, F., & Li, C. (2024). Reinforced adaptive knowledge learning for multimodal fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 38, No. 15, pp. 16777–16785).
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Frontiers in Computing and Intelligent Systems
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
