RAG with a plan: Breaking complex queries into traceable atomic pieces
Plan × RAG decomposes complex queries into DAG-based (Directed Acyclic Graph) sub-queries for better attribution and reduced hallucinations.
A RAG system that can explain its reasoning step-by-step
📚 https://arxiv.org/abs/2410.20753
Original Problem 🤔:
Traditional Retrieval Augmented Generation (RAG) frameworks face challenges with hallucinations and lack attribution in complex queries. Current RAG systems follow a retrieve-then-reason approach, which struggles with multi-hop queries and often fails to link generated content to specific retrieved documents.
-----
Solution in this Paper 🛠️:
• Introduces Plan × RAG - shifts from retrieve-then-reason to plan-then-retrieve paradigm
• Generates a Directed Acyclic Graph (DAG) to decompose complex queries into atomic sub-queries
• Implements plug-and-play experts:
- Dynamic Query Expert: Handles sub-query generation
- Critic Expert: Controls on-demand retrievals
- Relevance Expert: Ensures document relevance
- Aggregator: Combines sub-query responses
• Each sub-query links to a single document, enabling clear attribution
• Enables parallel processing of independent sub-queries at same depth
-----
Key Insights 💡:
• 88.5% of sub-queries naturally map to single documents, proving effective attribution
• DAG structure reduces retrievals by 600 while maintaining similar accuracy
• No fine-tuning required - works with frozen LLMs
• Enables backtracking for error correction
• Achieves attribution by design through atomic sub-queries
-----
Results 📊:
• Outperforms vanilla LLM baselines on multi-hop datasets:
- HotpotQA: 35.67% accuracy (vs 33.93% Self-RAG)
- StrategyQA: 69.49% accuracy (vs 63.4% Self-RAG)
- Arc-Challenge: 74.12% accuracy (vs 73.12% Self-RAG)
• Maintains competitive performance on single-hop PopQA dataset
• Reduces hallucinations through expert-guided retrieval
Share this post