"GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers"

Small LLMs learn to write better prompts by watching their own thought process.

Gradients Over Reasoning (GREATER) enables smaller LLMs to optimize their own prompts using gradient information over reasoning chains, eliminating dependence on larger proprietary models for optimization.

-----

https://arxiv.org/abs/2412.09722

🤔 Original Problem:

Current prompt optimization relies heavily on expensive, proprietary LLMs like GPT-4 to generate feedback. Smaller models struggle to produce quality optimization feedback independently.

-----

🔧 Solution in this Paper:

→ GREATER uses gradient information over task-specific reasoning instead of textual feedback.

→ It first calculates token probabilities to suggest potential candidates for optimized prompts.

→ The model then generates reasoning chains for problem solutions.

→ Final answer logits are extracted to compute loss gradients.

→ These gradients guide the selection of optimal tokens for prompt improvement.

-----

💡 Key Insights:

→ Direct gradient signals are more effective than textual feedback

→ Smaller models can match larger ones with optimized prompts

→ Incorporating reasoning chains into optimization improves results

→ Perplexity regularization ensures interpretable prompts

-----

📊 Results:

→ Up to 8.9% average performance gain on BBH tasks

→ Outperforms state-of-the-art methods across GSM8k, BBH, FOLIO benchmarks

→ Matches or exceeds GPT-4 optimized prompts in many cases

→ Shows strong transferability across different model sizes

------

Are you into AI and LLMs❓ Join my daily AI newsletter. I will send you 7 emails a week analyzing the highest signal AI developments. ↓↓

🎉 https://rohanpaul.substack.com/