LLMs become less repetitive by first mapping out different ways to answer, then picking mindfully.
Instead of grabbing the first answer, let it scan different answer zones before responding.
This paper proposes to diversify LLM outputs through stratified sampling, improving coverage and addressing mode collapse.
📚 https://arxiv.org/abs/2410.09038
Original Problem 🔍:
LLMs often lack diversity in their responses, especially when multiple valid answers exist. Current methods like temperature scaling degrade generation quality and don't effectively address mode collapse.
-----
Solution in this Paper 🛠️:
• SimpleStrat: A training-free sampling approach to increase diversity
• Three stages: auto-stratification, heuristic estimation, probabilistic prompting
• Uses LLM to identify useful partitions of solution space
• Computes joint probabilities across strata
• Samples from joint probability distribution to augment original prompt
-----
Key Insights from this Paper 💡:
• LLMs can identify meaningful diversity dimensions even if they can't generate diverse solutions
• Stratified sampling counteracts biases in next-token probabilities
• Diversity improvement is orthogonal to temperature scaling
• SimpleStrat addresses mode collapse without manual intervention
-----
Results 📊:
• Average reduction in KL Divergence: 0.36 compared to baseline on Llama 3 models
• Consistent 0.05 increase in recall across all temperatures for GPT-4o
• Improved diversity on top of temperature scaling
• Higher recall by 0.05 compared to GPT-4o
• CoverageQA dataset: 105 underspecified questions with average 28.7 equally plausible answers
Share this post