"AFlow: Automating Agentic Workflow Generation"

AFlow automates LLM workflow design through code-based Monte Carlo Tree Search

→ Enables smaller LLMs to outperform GPT-4 at just 4.55% of inference cost

https://arxiv.org/abs/2410.10762

🤔 Original Problem:

Current agentic workflows for LLMs require significant manual effort in design and optimization, limiting scalability across different tasks. Existing automated methods still need manual setup and can't fully capture diverse workflow structures.

-----

🛠️ Solution in this Paper:

AFlow reformulates workflow optimization as a code-based search problem using Monte Carlo Tree Search (MCTS). It represents workflows as interconnected LLM-invoking nodes with:

→ Flexible nodes connected by code-based edges capturing complex interactions

→ Operators that encapsulate common agentic operations like Ensemble and Review

→ Soft mixed-probability selection for exploring nodes

→ LLM-driven expansion to generate new possibilities

→ Tree-structured experience backpropagation

-----

💡 Key Insights:

→ Code-based representation enables more precise workflow control than graph/network structures

→ Tree-based search preserves node-level exploration experiences better than linear approaches

→ Smaller models with optimized workflows can outperform larger models at fraction of cost

→ Different LLMs require different workflows for optimal performance

-----

📊 Results:

→ 5.7% average improvement over manually designed methods across 6 benchmarks

→ 19.5% improvement over existing automated approaches

→ Enables smaller LLMs to outperform GPT-4 at just 4.55% of inference cost

→ Achieves 80.3% average performance across QA, Code and Math domains