"Bridging Relevance and Reasoning: Rationale Distillation in Retrieval-Augmented Generation"

Rationale distillation helps rerankers pick documents that actually help generators answer questions.

RADIO framework aligns preferences between rerankers and generators in Retrieval-Augmented Generation by using rationales extracted from LLMs to guide document selection.

-----

https://arxiv.org/abs/2412.08519

🤔 Original Problem:

In RAG systems, rerankers and generators often have mismatched preferences due to different pretraining objectives. Documents ranked as relevant by rerankers may not provide the necessary reasoning support for generators to answer queries accurately.

-----

🔧 Solution in this Paper:

→ RADIO extracts rationales using LLMs by combining queries with ground truth answers

→ These rationales capture the reasoning needed to derive correct answers

→ Documents are reranked based on both rationale similarity and retrieval relevance scores

→ The reranker is fine-tuned using this rationale-based ranking to align with generator needs

→ Integration coefficient balances between rationale and retrieval scores for optimal performance

-----

💡 Key Insights:

→ Rationales bridge the gap between surface-level relevance and deep reasoning requirements

→ Combining rationale and retrieval scores provides complementary signals for document ranking

→ Performance gains are more significant with smaller generators, showing RADIO helps where needed most

→ The method transfers well across different generators and tasks

-----

📊 Results:

→ Outperformed baseline methods on Natural Questions and TriviaQA datasets

→ Achieved 8.72% EM improvement with Llama3.1-8b

→ Showed consistent gains across MMLU categories, especially in Humanities (+2.74%) and Social Sciences (+1.61%)