"Atla Selene Mini: A General Purpose Evaluation Model"

The paper shows data curation beats brute-force scaling

It addresses the challenge of unreliable LLM judges.

Introduces Atla Selene Mini, a compact yet powerful model. Selene Mini achieves state-of-the-art evaluation performance, outperforming even larger models across diverse benchmarks.

-----

📌 By combining Direct Preference Optimization and Supervised Fine-Tuning with synthetic critiques, it surpasses larger models in judgment accuracy, proving smaller models can be more reliable evaluators.

📌 Filtering and augmentation strategies matter more than raw dataset size. Selene Mini's edge comes from structured critique integration and dataset refinement, ensuring robustness. This suggests better LLM evaluation models can be built with principled data rather than just more data.

📌 Selene Mini's promptability enables dynamic evaluation, adapting to varied tasks with minimal tuning. This makes it a practical, deployable solution compared to rigid, oversized judges that lack flexibility in real-world evaluation scenarios.

-----

Paper - https://arxiv.org/abs/2501.17195

Original Problem: 😥

→ Human evaluation of LLM outputs is slow and costly.

→ Automated LLM judges are often unreliable and biased.

→ Existing LLM judges struggle with accuracy and consistency.

-----

Solution in this Paper: 💡

→ Atla Selene Mini is presented as a Small Language Model-as-a-Judge (SLMJ).

→ It is fine-tuned from Llama 3.1 8B Instruct model.

→ A principled data curation strategy was developed.

→ Public datasets were augmented with synthetically generated critiques.

→ Data quality was ensured through filtering and ablation studies.

→ Training used a combination of Direct Preference Optimization (DPO) and Supervised Fine-Tuning (SFT) loss.

→ This approach yields a promptable and robust evaluator.

-----

Key Insights from this Paper: 🧠

→ High-quality data is crucial for training effective SLMJs.

→ Synthetic data augmentation with critiques enhances training data.

→ Filtering data based on reward models improves dataset quality.

→ Combining DPO and SFT training optimizes performance.

→ Promptability is essential for real-world LLM evaluation scenarios.

-----

Results: 🏆

→ Selene Mini achieves 0.756 overall task-average performance.

→ Selene Mini achieves 0.753 overall benchmark-average performance.

→ It achieves an average of 0.648 on absolute scoring tasks. This outperforms GPT-4o-mini's 0.640.

→ Selene Mini is the top-ranking 8B generative model on RewardBench.