"GeAR: Generation Augmented Retrieval"

GeAR, proposed in this paper, combines search and text generation to find exactly what you're looking for.

Traditional search tells you what document matches, GeAR shows you exactly where and why.

GeAR enhances document retrieval by adding generation capabilities to traditional bi-encoders, enabling fine-grained information localization while maintaining retrieval efficiency.

-----

https://arxiv.org/abs/2501.02772

🔍 Original Problem:

→ Traditional bi-encoder based retrieval systems compress complex semantic relationships into simple scalar similarity scores, making results hard to interpret and understand. They also struggle with locating specific relevant sections in long documents.

-----

🛠️ Solution in this Paper:

→ GeAR introduces a novel architecture combining bi-encoder, fusion encoder, and text decoder components.

→ The bi-encoder handles efficient document-query matching while the fusion encoder uses cross-attention to combine their representations.

→ A text decoder generates relevant snippets from documents based on fused representations.

→ The model trains using both contrastive learning for retrieval and language modeling for generation.

→ A data synthesis pipeline using LLMs creates high-quality training data for diverse retrieval scenarios.

-----

💡 Key Insights:

→ Generation capabilities can enhance retrieval without sacrificing computational efficiency

→ Cross-attention mechanisms effectively locate relevant document sections

→ Joint training of retrieval and generation tasks improves overall performance

-----

📊 Results:

→ Achieves 94% Recall@5 on PAQ dataset, outperforming traditional retrievers

→ 88.1% Exact Match accuracy on answer generation for PAQ

→ 95.4% accuracy on fine-grained information localization