"RAGServe: Fast Quality-Aware RAG Systems with Configuration Adaptation"

Smart query profiling helps RAGServe pick the perfect RAG configuration.

RAGServe adapts RAG configurations per query and jointly schedules them with resource allocation to optimize both quality and response time in LLM systems.

-----

https://arxiv.org/abs/2412.10543

🤔 Original Problem:

RAG systems face a critical tradeoff between response quality and delay. Current solutions either focus on reducing delay through scheduling or maximizing quality through configuration tuning, but not both simultaneously.

-----

💡 Solution in this Paper:

→ RAGServe introduces a two-level approach to optimize RAG systems.

→ First, it uses an LLM to profile each query and estimate required information pieces and reasoning needs.

→ Based on this profile, it prunes the massive configuration space to a smaller promising set.

→ Finally, it jointly selects configurations and schedules queries based on available GPU memory.

-----

🔍 Key Insights:

→ Different queries need different RAG configurations for optimal performance

→ Query profiling can effectively filter out undesirable configurations

→ Joint scheduling with configuration selection prevents memory bottlenecks

→ System resource awareness is crucial for real-world performance

-----

📊 Results:

→ 1.64-2.54x lower latency compared to state-of-the-art systems

→ 1.8-4.5x higher throughput at same quality levels

→ 12-15% higher quality compared to fixed configurations

→ Only adds 0.1x overhead to total processing time