Scenarios Where Hallucinatory Information Generated by Large Language Models is Most Difficult to Detect

Abstract

Large Language Models (LLMs) are really handy for work and daily life, but they’ve got this frustrating issue — “hallucinations”. That’s when they churn out wrong or mismatched info, which ruins how reliable they feel. Right now, there’s no solid solution for this, and past research on spotting these hallucinations is pretty scattered. This review dives into how tough it is to catch hallucinations in different scenarios—like Q&A, casual chats, or text summarization—plus how using single vs. multi-modal inputs changes things, and what happens with different types of negative responses. This paper also looks at existing tools to find good detection strategies. Turns out, Q&A tasks are easiest to check (manual: ~91%, tools: ~87%), summarizing is the hardest (manual: ~65%, tools: ~60%). Single-modal content is easier to verify (avg ~82%) than multi-modal (avg ~59%, even lower when there’s lots of overlapping info). Evasive responses have way more hallucinations (61%) and are toughest to spot (43% accuracy). To fix this, this paper suggests stuff like Retrieval-Augmented Generation (RAG) and checking multi-modal info together to make LLMs more trustworthy.