Hallucination in Large Language Models: A Comprehensive Survey, Taxonomy, and Mitigation Strategies

Large Language Models (LLMs) have demonstrated impressive performance in a wide range of natural language processing operations, but their propensity to produce plausible-sounding, but factually incorrect information is often known as hallucination is a significant impediment to their use in high-stakes areas of application, like medicine, law, and scientific research. It is in this paper that the current state-of-the-art LLMs have their hallucination phenomena surveyed, and a new four-tier taxonomy defining hallucinations by origin is introduced: (1) intrinsic factual contradictions, (2) extrinsic knowledge conflicts, (3) temporal reasoning failures, and (4) contextual coherence breakdowns. We evaluate five top LLMs, GPT-4, Claude 3 Opus, Gemini Pro 1.5, Llama 3 70B, and Mistral 8x7B on the TruthfulQA, HaluEval, and factscore benchmarks and find hallucination rates of 18.7 percent to 34.2 percent. Moreover, we engage in a strict comparative assessment of such mitigation strategies as Retrieval-Augmented Generation (RAG), Reinforcement Learning on Human Feedback (RLHF), Chain-of-Thought prompting, and hybrid ensemble technologies. The experiments we carried out prove that the RLHF+RAG hybrid has the best accuracy of 91.5% which is far better than the individual methods. The survey will equip the practitioners and researchers with practical information regarding which type of hallucination mitigation strategies to choose given the task-specific requirements and computational limitations.