Smart weighting of examples helps LLMs maintain peak performance with more context.
DR-ICL (Differentiated In-Context Learning ) enhances LLMs' performance in many-shot scenarios by introducing differentiated learning and advantage-based reweighting, solving the performance decline issue when demonstration examples increase.
-----
https://arxiv.org/abs/2501.04070
🤔 Original Problem:
LLMs show declining performance as in-context learning examples increase from few-shot to many-shot scenarios, caused by suboptimal negative log-likelihood optimization and increasing noise from larger demonstration sets.
-----
🔧 Solution in this Paper:
→ DR-ICL uses differentiated learning to optimize globally, ensuring many-shot performance exceeds zero-shot levels
→ Implements advantage-based reweighting locally to filter noise in many-shot demonstrations
→ Divides sequences into reweighting windows and calculates advantages from previous window samples
→ Integrates advantages into NLL computation for dynamic weight adjustment
→ Combines global and local perspectives through a refined training objective
-----
💡 Key Insights:
→ Many-shot doesn't always mean better performance in LLMs
→ Performance plateaus and declines with increasing demonstrations
→ Noise accumulation significantly impacts model effectiveness
→ Window-based sampling helps maintain stable performance
-----
📊 Results:
→ Achieved significant improvements across 50 datasets and 7 NLP tasks
→ Maintained stable performance with demonstrations ranging from 1-350 shots
→ Outperformed baseline methods in both in-domain and out-of-domain tasks
→ Demonstrated effectiveness with sequences up to 8,000 tokens
------
Are you into AI and LLMs❓ Join my daily AI newsletter. I will send you 7 emails a week analyzing the highest signal AI developments. ↓↓
🎉 https://rohanpaul.substack.com/
Share this post