RATester enhances LLMs' ability to generate accurate unit tests by using the gopls language server to fetch precise contextual information when encountering unfamiliar code elements during test generation.
-----
https://arxiv.org/abs/2501.07425
Original Problem 🔍:
LLMs often produce hallucinations when generating unit tests, like calling non-existent methods or using incorrect parameter types, due to limited awareness of project context.
-----
Solution in this Paper 🛠️:
→ RATester integrates the gopls language server to provide LLMs with repository-wide knowledge.
→ When encountering unfamiliar identifiers, RATester fetches their definitions and documentation using gopls.
→ This contextual information is injected into the LLM's prompt template during test generation.
→ The system continuously enriches the LLM's understanding of the project's global context.
-----
Key Insights from this Paper:
→ Fixed context patterns used by existing approaches can miss important information or include irrelevant details
→ Dynamic context fetching based on actual generation needs produces better results than static approaches
→ Language server integration can effectively simulate human developers' IDE-assisted coding process
-----
Results 📊:
→ Improved compile rate from 16.67%-63.56% to 45.58%-69.49%
→ Increased line coverage by 16.30%-165.69% compared to baselines
→ Successfully killed 25-147 more mutants in mutation testing
