"Dynamic Scaling of Unit Tests for Code Reward Modeling"

Teaching AI to grade code better by cranking up the test volume strategically.

This paper explores scaling unit tests to improve code reward signals, making LLMs better at identifying correct code solutions.

-----

https://arxiv.org/abs/2501.01054

🤔 Original Problem:

LLMs struggle with first-attempt code generation. Current methods use multiple solutions and unit tests for verification, but unreliable unit tests diminish reward signal quality.

-----

🔧 Solution in this Paper:

→ Introduces CodeRM-8B, a lightweight unit test generator enabling efficient test scaling

→ Implements dynamic scaling mechanism adapting test numbers based on problem difficulty

→ Creates automatic data pipeline synthesizing high-quality unit tests from existing datasets

→ Uses majority voting framework for solution selection

→ Employs problem difficulty classifier using language model probing

-----

🎯 Key Insights:

→ Scaling unit tests consistently improves accuracy across different model sizes

→ Benefits of scaling tests are greater for more challenging problems

→ Smaller models can achieve comparable performance to larger ones when scaled properly

-----

📊 Results:

→ 18.43% improvement for Llama3-8B on HumanEval Plus

→ 4.95% improvement for Llama3-70B

→ 3.42% improvement for GPT-4o-mini

→ 0.5% additional gain through dynamic scaling on MBPP Plus