"RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement"

Tree-based reasoning with external fact-checking helps LLMs solve multi-step problems more accurately.

RAG-Star enhances LLMs' complex reasoning by combining tree-based deliberative planning with retrieval-augmented verification, significantly improving multi-step problem-solving capabilities.

-----

https://arxiv.org/abs/2412.12881

🤔 Original Problem:

LLMs struggle with complex reasoning tasks requiring multiple steps. Current methods either rely solely on internal knowledge or face conflicts between internal and external knowledge sources.

-----

🔧 Solution in this Paper:

→ RAG-Star uses Monte Carlo Tree Search to explore possible reasoning paths starting from the input question

→ At each node, it generates sub-queries and answers using the LLM's internal knowledge

→ A novel retrieval-augmented verification system evaluates reasoning steps using query-aware and answer-aware rewards

→ External knowledge guides but doesn't directly interfere with the LLM's reasoning process

→ The system iteratively selects nodes, expands plans, and updates rewards through backpropagation

-----

💡 Key Insights:

→ Separating internal reasoning from external verification reduces knowledge conflicts

→ Tree-based search enables systematic exploration of reasoning paths

→ Query-aware rewards ensure logical consistency of sub-queries

→ Answer-aware rewards verify factual correctness against retrieved knowledge

-----

📊 Results:

→ Outperforms previous methods by 18.98% with Llama-3.1-8B

→ Achieves 16.19% improvement with GPT-4o across datasets

→ Shows significant gains in multi-hop question answering tasks