Inner Loop Memory Augmented Tree Retrieval (ILM-TR): Enhancing LLMs' long-context performance through iterative retrieval and short-term memory.
📚 https://arxiv.org/abs/2410.12859
Original Problem 🤔:
LLMs struggle with long contexts due to computational limitations. Existing Retrieval-Augmented Generation (RAG) methods only retrieve information based on initial queries, limiting their ability to handle complex questions requiring deeper reasoning or integration of knowledge from multiple parts.
-----
Solution in this Paper 💡:
• Introduces Inner Loop Memory Augmented Tree Retrieval (ILM-TR)
• Uses a two-part system: retriever and inner-loop query mechanism
• Retriever segments data, generates regular summaries and surprising facts
• Inner-loop query stores intermediate findings in Short-Term Memory (STM)
• System repeatedly retrieves new information based on initial query and STM contents
• Process continues until convergence or query limit reached
-----
Key Insights from this Paper 💡:
• Novel summarization method extracting main content and surprising facts
• Inner-loop mechanism refines retrieval based on evolving information
• Short-Term Memory component guides further retrieval
• Effective for complex questions in long context scenarios
-----
Results 📊:
• Outperforms baseline RAG methods in M-NIAH and BABILong tests
• Maintains robust performance with context lengths up to 500k tokens
• Including surprising information notably improves model performance
• Multiple iterations increase query processing time
• Requires larger models with strong instruction-following capabilities
