Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use

Turns out, AI prefers natural chit-chat over robot-speak

AI learns way better when you chat with it naturally, just like teaching a friend

Teaching AI agents with rich language feedback instead of simple commands improves learning by 20%

📚 https://arxiv.org/abs/2410.24218

🎯 Original Problem:

Most reinforcement learning approaches use simple low-level instructions that don't reflect natural human communication. This limits agents' ability to learn from rich language feedback.

-----

🔧 Solution in this Paper:

→ Extended Decision Transformer architecture to create Language-Teachable Decision Transformer (LTDT)

→ Incorporated two types of language feedback:

- Hindsight: Comments about past actions

- Foresight: Guidance for future actions

→ Used GPT-4 to generate diverse language variations of the same feedback

→ Tested across HomeGrid, ALFWorld, Messenger, and MetaWorld environments

-----

💡 Key Insights:

→ Rich language feedback significantly improves agent learning compared to simple instructions

→ Combining hindsight and foresight feedback is more effective than using either alone

→ Language diversity through GPT-4 augmentation enhances agent performance

→ The approach works without human annotators

-----

📊 Results:

→ Combined hindsight and foresight feedback improved performance by 9.86 points (37.95% to 47.81%)

→ Adding GPT-augmented language diversity further improved by 10.14 points (47.81% to 57.95%)

→ Consistent improvements across all four test environments