"Active Negative Loss: A Robust Framework for Learning with Noisy Labels"

A new loss function that teaches neural networks to learn from messy data without getting confused.

This paper introduces Active Negative Loss (ANL), a robust framework that improves training with noisy labels by replacing Mean Absolute Error with Normalized Negative Loss Functions. It also addresses label imbalance through entropy-based regularization for better performance in non-symmetric noise scenarios.

https://arxiv.org/abs/2412.02373v1

🎯 Original Problem:

→ Deep neural networks can overfit to noisy labels, leading to poor performance.

→ Current solutions using Mean Absolute Error (MAE) as passive loss function are slow to converge and difficult to train.

-----

🔧 Solution in this Paper:

→ The paper introduces Normalized Negative Loss Functions (NNLFs) to replace MAE.

→ NNLFs are created through vertical flipping operation and normalization of active loss functions.

→ The Active Negative Loss framework combines normalized active loss with NNLFs.

→ An entropy-based regularization technique addresses label imbalance in non-symmetric noise scenarios.

-----

💡 Key Insights:

→ MAE's equal treatment of clean and noisy samples hinders training efficiency

→ Vertical flipping operation effectively converts maximizing functions to minimizing ones

→ Label imbalance significantly impacts model performance in non-symmetric noise scenarios

-----

📊 Results:

→ Outperforms state-of-the-art methods in both image classification and segmentation

→ Shows superior performance across symmetric, asymmetric, and instance-dependent noise scenarios

→ Successfully extends beyond classification to more complex tasks like image segmentation