Task confusion, not catastrophic forgetting, is the real barrier in incremental learning
Mathematical proof shows why generative models excel at continuous learning tasks
📚 https://arxiv.org/abs/2410.20768
Original Problem 🔍:
Class-incremental learning faces two major challenges: Task Confusion (TC) and Catastrophic Forgetting (CF). Previous research focused mainly on CF, but TC emerges as the primary obstacle when models must distinguish between classes from different tasks without task IDs.
-----
Solution in this Paper 🛠️:
→ Introduces a mathematical framework distinguishing between TC and CF
→ Proves Infeasibility Theorem showing optimal class-IL is impossible with discriminative modeling due to TC
→ Establishes Feasibility Theorem demonstrating generative modeling can achieve optimal class-IL by overcoming TC
→ Proposes generative classifier strategy that prevents CF through model isolation
-----
Key Insights 💡:
→ TC occurs because classes from different tasks are never seen together during training
→ CF refers to performance drop on previously learned tasks after learning new ones
→ Generative modeling (P(X,Y)) naturally avoids TC while discriminative modeling (P(Y|X)) cannot
→ Bias-correction strategies only slightly help with TC but don't address CF
→ Generative replay can theoretically achieve optimality but struggles with complex datasets
-----
Results 📊:
→ On CIFAR-10: Generative classifier (GenC) achieves 56.02% accuracy vs 18.74% for baseline
→ On CIFAR-100: GenC reaches 49.53% vs 7.96% for baseline
→ On MNIST: GenC attains 93.75% vs 19.92% for baseline
→ Generative modeling consistently outperforms discriminative approaches across all datasets
Share this post