DePaC (Dehallucinating Parallel Context Extension) trains LLMs to say "I don't know" when they should, and remember what they actually know.
DePaC is a novel method that reduces hallucinations in LLMs by using context-aware negative training and information-calibrated aggregation for accurate retrieval-augmented generation.
https://arxiv.org/abs/2412.14905
Original Problem 🤔:
→ LLMs often generate hallucinated information even with RAG, particularly struggling with fact fabrication (presenting unsupported claims) and fact omission (failing to present supported claims).
→ Existing Parallel Context Extension (PCE) approaches cannot effectively handle these hallucination issues when integrating multiple contexts.
Solution in this Paper 💡:
→ DePaC addresses hallucinations through two key mechanisms.
→ Context-aware negative training fine-tunes LLMs to explicitly refuse answering when contexts are irrelevant.
→ Information-calibrated aggregation prioritizes context windows that provide higher information value.
→ The system uses Kullback-Leibler divergence to measure information increment from contexts.
Key Insights 💭:
→ Negative training significantly reduces fact fabrication by teaching models when to refuse answers
→ Information-calibrated scoring helps prevent fact omission by identifying most relevant contexts
→ DePaC's complexity increases linearly with document count, while vanilla approaches increase quadratically
Results 📊:
→ Reduced fact fabrication hallucinations from 39.5% to 12.7%
→ Decreased fact omission from 39.3% to 19.9%
→ Achieved 45.1% average accuracy across information seeking tasks
→ Outperformed baseline methods on 9 RAG tasks
Share this post