"SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution"

Two-step pipeline beats complex agents in fixing real GitHub issues.

And simple retrieval-edit approach matches GPT-4's code fixing abilities.

SWE-Fixer introduces a streamlined pipeline using open-source LLMs to fix GitHub issues efficiently, making code repair accessible and transparent through a two-step retrieval and editing approach .

-----

https://arxiv.org/abs/2501.05040

🔍 Original Problem:

→ Current GitHub issue-fixing solutions rely heavily on proprietary LLMs, limiting accessibility and transparency . Open-source alternatives struggle with complex agent-based approaches that require extensive training data and execution environments .

-----

🛠️ Solution in this Paper:

→ SWE-Fixer splits the task into two simple steps: code file retrieval and code editing .

→ The retrieval module uses BM25 with a 7B LLM to find relevant files efficiently .

→ A 72B LLM editor then generates patches for identified files .

→ The system uses JsonTuning for structured input-output and Chain-of-Thought reasoning .

→ A curated dataset of 110K GitHub issues powers the training process .

-----

💡 Key Insights:

→ Simple pipeline approaches outperform complex agent-based systems

→ Structured data representation improves model performance

→ Chain-of-Thought reasoning enhances code editing accuracy

-----

📊 Results:

→ 23.3% accuracy on SWE-Bench Lite benchmark

→ 30.2% accuracy on SWE-Bench Verified

→ Outperforms several GPT-4 and Claude-based solutions