"Automatic Database Configuration Debugging using Retrieval-Augmented Language Models"

Making LLMs database-savvy using a retrieval-augmented approach.

Andromeda, proposed in this paper, is a framework that uses LLMs for automatic DBMS configuration debugging, leveraging retrieval-augmented generation to provide domain-specific context from multiple sources.

-----

https://arxiv.org/abs/2412.07548

🔍 Original Problem:

DBMS configuration debugging is challenging and time-consuming, even for experienced DBAs. Existing LLM-based approaches often yield generic, unhelpful recommendations due to lack of domain-specific knowledge.

-----

🛠️ Solution in this Paper:

→ Andromeda employs a retrieval-augmented generation (RAG) strategy to enrich NL debugging questions with domain-specific context.

→ It retrieves relevant information from historical questions, troubleshooting manuals, and DBMS telemetry data.

→ A document retrieval mechanism addresses semantic heterogeneity among different sources using contrastive learning.

→ A telemetry analysis method detects and selects relevant anomalous telemetries using seasonal-trend decomposition.

→ Andromeda uses a two-phase prompting strategy with LLMs to diagnose issues and recommend specific knob configurations.

-----

💡 Key Insights from this Paper:

→ RAG significantly improves LLM performance in domain-specific tasks like DBMS configuration debugging

→ Combining multiple data sources (questions, manuals, telemetry) provides more comprehensive context

→ Contrastive learning effectively aligns heterogeneous document types for retrieval

→ Telemetry analysis helps identify relevant performance issues for more accurate debugging

-----

📊 Results:

→ Outperforms existing solutions in both NL and runnable evaluation settings

→ Achieves over 0.7 success rate in solving real configuration issues

→ Reduces performance gap between open-source LLMs and GPT models

→ Works well across different knob frequencies and types