"Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTS"

HiAR-ICL: A new way to make LLMs reason like humans - no examples needed

This paper introduces HiAR-ICL, a paradigm that teaches LLMs high-level reasoning patterns instead of relying on examples, achieving state-of-the-art 79.6% accuracy on complex math problems using smaller models.

-----

https://arxiv.org/abs/2411.18478

🤔 Original Problem:

Traditional in-context learning depends heavily on high-quality examples and human expertise. This makes it inefficient for complex reasoning tasks and limits generalization to new problem types.

-----

🔍 Key Insights:

→ LLMs perform better when taught abstract reasoning patterns rather than specific examples

→ Monte Carlo Tree Search can effectively discover optimal reasoning strategies

→ Human-like atomic reasoning actions improve model performance

→ Cognitive complexity metrics help match problems with appropriate reasoning patterns

-----

⚡ Solution in this Paper:

→ HiAR-ICL defines five atomic reasoning actions that mimic human cognitive processes

→ It uses Monte Carlo Tree Search to construct "thought cards" containing reasoning patterns

→ The system matches problems with appropriate thought cards using cognitive complexity metrics

→ Multiple verification mechanisms ensure high-quality results

-----

📊 Results:

→ Achieved 79.6% accuracy on MATH benchmark using Qwen2.5-7B-Instruct

→ Outperformed GPT-4 (76.6%) and Claude 3.5 (71.1%)

→ Demonstrated consistent performance improvements across multiple reasoning benchmarks

→ Reduced computational complexity compared to traditional methods