"Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Models"

Attention is all you need, even to catch LLM lies.

This paper addresses the critical issue of hallucination in Large Language Models (LLMs).

Introduces a novel method using attention mechanisms for zero-shot hallucination detection, improving accuracy and reducing computational cost compared to existing consistency-based methods.

-----

https://arxiv.org/abs/2501.09997

Original Problem 🧐:

→ LLMs sometimes generate incorrect but confident answers, termed hallucinations.

→ This lack of trustworthiness limits LLM application, especially in sensitive domains.

→ Existing hallucination detection methods based on answer consistency are computationally expensive.

→ They rely on multiple LLM runs and may fail when LLMs are confidently wrong.

-----

Solution in this Paper 💡:

→ This paper proposes Attention-Guided Self-Reflection (AGSER).

→ AGSER uses attention scores to categorize query tokens into attentive and non-attentive sets.

→ Attentive queries contain the most important tokens based on attention contribution. Non-attentive queries contain the rest.

→ AGSER generates answers for both attentive and non-attentive queries separately.

→ It calculates consistency scores between these answers and the original answer.

→ The difference between attentive and non-attentive consistency scores estimates hallucination.

→ Lower attentive consistency and higher non-attentive consistency indicate higher hallucination probability.

→ Attention-Guided Self-Reflection (AGSER) requires only three LLM passes, reducing computation compared to methods needing multiple resamples.

-----

Key Insights from this Paper 🤔:

→ Attention contributions in LLMs reflect important parts of answer generation.

→ Attention can guide LLMs to rethink and self-reflect for hallucination detection.

→ Inconsistency in answers generated from attentive and non-attentive queries can indicate hallucinations.

→ Splitting queries based on attention provides a zero-shot hallucination detection approach.

-----

Results 🏆:

→ With Llama2-7b, AGSER achieves an average improvement of 10.4% to 16.1% in AUC across Books, Movies, and GCI datasets compared to baselines.

→ Attention-Guided Self-Reflection (AGSER) shows consistent performance improvements across Llama2-13b, Llama3-8b, and Qwen2.5-14b models.

→ AGSER reduces computational cost by requiring only 3 LLM passes versus 5 or more for consistency-based methods.