Teaching LLMs to think backward makes them better at reasoning forward
Bidirectional thinking helps smaller LLMs outperform larger ones
This paper introduces REVTHINK, a framework that enhances LLMs' reasoning abilities by teaching them to think both forward and backward. Unlike previous approaches that only use backward reasoning for verification, REVTHINK incorporates it during training, leading to better performance across diverse reasoning tasks.
-----
https://arxiv.org/abs/2411.19865
🤔 Original Problem:
→ Current LLMs struggle with complex reasoning tasks because they only think in one direction (forward)
→ Existing backward reasoning methods are limited to mathematical domains and only used for verification at test time
-----
🔧 Solution in this Paper:
→ REVTHINK augments training data using a teacher model to generate forward reasoning, backward questions, and backward reasoning
→ It trains student models with three objectives: generate forward reasoning, create backward questions, and solve backward questions
→ The framework validates data points by checking forward reasoning accuracy and backward reasoning consistency
→ At test time, the model only performs forward reasoning, maintaining computational efficiency
-----
💡 Key Insights:
→ Backward thinking can be effectively applied beyond just mathematical reasoning
→ Training with bidirectional reasoning is more effective than using it only for verification
→ Smaller models trained with REVTHINK can outperform larger models using conventional methods
→ The framework shows strong sample efficiency, achieving better results with just 10% of training data
-----
📊 Results:
→ 13.53% improvement over zero-shot performance
→ 6.84% gain over standard knowledge distillation methods
→ 7B model outperforms 176B model's zero-shot performance
→ 40% reduction in average delay compared to legacy methods
Share this post