"Enhancing Retrieval-Augmented Generation: A Study of Best Practices"

This paper explores best practices for Retrieval-Augmented Generation (RAG) systems, examining how different components and configurations impact LLM response quality.

→ This paper tests various RAG components including query expansion, retrieval strategies, and contrastive in-context learning.

→ They evaluate factors like LLM size, prompt design, chunk size, knowledge base size, and retrieval stride.

→ A novel Contrastive In-context Learning RAG and a “Focus Mode” for retrieving relevant context at the sentence level are introduced.

-----

https://arxiv.org/abs/2501.07391

Key Insights from this Paper 🔑:

→ Contrastive In-context Learning significantly improves RAG performance, especially for specialized knowledge.

→ Focusing on relevant sentences ("Focus Mode") enhances response quality by reducing noise and improving relevance.

→ LLM size matters, but bigger isn't always significantly better, especially for specialized tasks.

→ Prompt design is crucial, even small changes affect performance.

-----

Results 💯:

→ ICL1Doc+ (Contrastive In-context Learning with one retrieved document and contrastive examples) achieves 27.79 ROUGE-L on TruthfulQA and 23.87 ROUGE-L on MMLU.

→ 120Doc120S (Focus Mode with 120 retrieved sentences) improves Embedding Cosine Similarity by 0.81% on MMLU.

→ Instruct45B outperforms Instruct7B on TruthfulQA but less so on MMLU.