"MoEVD: Enhancing Vulnerability Detection by Mixture-of-Experts (MoE)"

The "one-for-all" design in existing Deep Learning-based Vulnerability Detection (DLVD) techniques limits their effectiveness.

A single model struggles to detect diverse vulnerability types, especially less common ones. This paper introduces MoEVD, a Mixture-of-Experts framework, to address this limitation by specializing in different vulnerability types for enhanced detection.

https://arxiv.org/abs/2501.16454

📌 MoEVD effectively addresses vulnerability detection's long-tail problem by specializing experts. This targeted approach surpasses one-for-all models, especially for rare vulnerability types, achieving a 7.3% F1-score improvement on tail groups.

📌 The router in MoEVD is crucial. It directs code to relevant experts, achieving 63.8% accuracy in expert selection. Correct routing boosts F1-score by 133.7%, demonstrating router's impact on performance.

📌 MoEVD's expert specialization allows it to outperform baselines across diverse Common Weakness Enumeration types. It improves recall by 9% to 77.8% and overall F1-score by 12.8%, showing broad applicability.

----------

Methods Explored in this Paper 🔧:

→ MoEVD uses a Mixture-of-Experts framework for vulnerability detection.

→ It splits the vulnerability detection task into CWE type classification and CWE-specific vulnerability detection.

→ Experts are trained for specific CWE types of vulnerabilities.

→ A router, trained as a CWE type multi-class classifier, directs input code to appropriate experts.

→ Experts predict vulnerability, and a combiner aggregates their outputs for a final prediction.

-----

Key Insights 💡:

→ The "one-for-all" approach is suboptimal for vulnerability detection due to the diversity of vulnerability types.

→ Specializing experts for specific CWE types enhances detection accuracy.

→ The router is crucial for directing code to the correct expert, significantly impacting performance.

→ MoEVD effectively handles the long-tailed distribution of CWE types, improving detection of rare vulnerabilities.

-----

Results 📊:

→ MoEVD achieves a 0.44 F1-score, outperforming SOTA baselines by at least 12.8%.

→ MoEVD improves recall over the best SOTA baseline by 9% to 77.8% across CWE types.

→ MoEVD improves F1-score on long-tailed CWE types by at least 7.3% compared to SOTA baselines.