"Instance-adaptive Zero-shot Chain-of-Thought Prompting"

Instance-adaptive prompting boosts LLM performance by optimizing information flow between question, prompt, and rationale.

📚 https://arxiv.org/pdf/2409.20441v2

Original Problem 🔍:

Zero-shot Chain-of-Thought (CoT) prompting uses a single task-level prompt for all instances, limiting its effectiveness as one prompt can't optimally suit every case.

-----

Solution in this Paper 🛠️:

• Introduces Instance-Adaptive Prompting (IAP) algorithm

• Analyzes information flow in LLMs using saliency score analysis

• Proposes two strategies:

- Sequential Substitution (IAP-ss)

- Majority Vote (IAP-mv)

• Selects appropriate prompts based on saliency scores between question, prompt, and rationale

-----

Key Insights from this Paper 💡:

• Successful reasoning requires prompt to gather information from question

• Rationale should aggregate information from both question and prompt

• Information flow from question to prompt and rationale jointly influences reasoning most

• Instance-level prompting outperforms task-level prompting

-----

Results 📊:

• Consistent improvements across LLaMA-2, LLaMA-3, and Qwen models

• Tested on math, logic, and commonsense reasoning tasks (GSM8K, MMLU, Causal Judgement)

• 2%-4% accuracy enhancement compared to optimal task-level prompts

• IAP-mv outperforms OPPR and Self-discover methods