"CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planning"

Teaching LLMs to plan smarter with tools while keeping an eye on the costs.

CATP-LLM introduces cost-aware planning capabilities to LLMs, enabling them to create efficient tool plans while considering execution costs. This framework addresses the critical challenge of balancing performance with resource utilization in practical applications.

-----

https://arxiv.org/abs/2411.16313

🤔 Original Problem:

LLMs can plan and use external tools for complex tasks, but they ignore execution costs like time and memory usage, leading to expensive and impractical solutions.

-----

🛠️ Solution in this Paper:

→ The paper introduces Tool Planning Language (TPL) that converts tools and dependencies into learnable tokens for non-sequential planning.

→ It implements Cost-aware Offline Reinforcement Learning (CAORL) to optimize the trade-off between performance and costs.

→ A context augmentation scheme helps incorporate tool cost information based on input data sizes.

→ OpenCATP platform evaluates plans using Quality of Plan (QoP) metric that considers both performance and execution costs.

-----

💡 Key Insights:

→ Non-sequential planning with multiple branches enables concurrent tool execution and reduces costs

→ Cost-awareness in LLMs requires understanding both tool dependencies and input data impacts

→ Reinforcement learning with intermediate feedback improves cost-optimization during planning

-----

📊 Results:

→ CATP-LLM with Llama2-7B outperforms GPT-4 with 28.2%-30.2% higher plan performance

→ Achieves 24.7%-45.8% lower execution costs on challenging planning tasks

→ Shows 1.15x-1.6x higher QoP for sequential planning and 2.29x-9.13x for non-sequential planning