This paper evaluates Parameter-Efficient Fine-Tuning (PEFT) methods for detecting code smells using small and large Language Models, focusing on Complex Conditional and Complex Method code smells in Java.
-----
https://arxiv.org/abs/2412.13801
🔍 Original Problem:
Traditional code smell detection methods rely on heuristics or machine learning, but suffer from poor performance and high computational costs. While LLMs show promise, their effectiveness for code smell detection remains unexplored.
-----
🛠️ Solution in this Paper:
→ Constructed high-quality Java code smell datasets from GitHub repositories using rigorous validation
→ Implemented four PEFT methods: prompt tuning, prefix tuning, LoRA, and IA3
→ Fine-tuned 4 small LMs (125M-220M parameters) and 6 LLMs (1.3B-7B parameters)
→ Evaluated performance across different scenarios including low-resource settings and varying hyperparameters
-----
💡 Key Insights:
→ Small LMs generally outperform LLMs in code smell detection
→ Model size is not the decisive factor - larger models within same family don't necessarily perform better
→ PEFT methods achieve comparable or better results than full fine-tuning while using less GPU memory
→ The best PEFT method varies depending on model architecture and available resources
-----
📊 Results:
→ GraphCodeBERT with IA3: 77.58% F1-score, 56.91% MCC for Complex Conditional detection
→ UnixCoder with LoRA: 88.20% F1-score, 77.56% MCC for Complex Method detection
→ PEFT methods improved MCC by 13.24% to 50.59% compared to traditional approaches
Share this post