"DynaSaur: Large Language Agents Beyond Predefined Actions"

An AI agent that can create new abilities on the fly through code

Meet DynaSaur: The LLM agent that grows smarter by writing its own functions

https://arxiv.org/abs/2411.01747

Original Problem 🤔:

Current LLM agent systems can only select from predefined actions, limiting their capabilities and requiring significant human effort to implement all possible actions upfront. This restricts their ability to handle diverse real-world tasks and adapt to new situations.

-----

Solution in this Paper 🛠️:

DynaSaur represents actions as Python functions and enables dynamic action creation. At each step, the agent can:

→ Generate new Python code when existing functions are insufficient

→ Reuse functions from the current action set

→ Execute code through Python interpreter and get observations

→ Build a growing library of reusable functions

-----

Key Insights 🔍:

→ Using Python functions as action representation provides both generality and composability

→ Smart retrieval of previously generated actions using embedding-based similarity search solves context length limitations

→ Action accumulation over time builds a reusable function library

→ Integration with Python ecosystem enables wide-ranging capabilities

-----

Results 📊:

→ Outperforms all baselines on GAIA benchmark with 38.21% average accuracy using GPT-4

→ Shows 81.59% improvement when combining human-designed tools with generated actions

→ Particularly strong on complex tasks (Level 2 and 3)

→ Achieves top position on GAIA public leaderboard