"DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning"

DexMimicGen turns 60 human demos into 21K robot training samples through smart simulation

https://arxiv.org/abs/2410.24185

🤖 Original Problem:

Collecting human demonstration data for training bimanual dexterous robots (like humanoids) is extremely costly and time-consuming. Current methods require multiple human operators, robots, and months of effort, making it impractical to scale.

-----

💡 Solution in this Paper:

DexMimicGen - an automated data generation system that creates large-scale training data from just a handful of human demonstrations. It handles three key challenges:

→ Independent arm operations through asynchronous execution strategy

→ Precise arm coordination using synchronization mechanisms

→ Sequential task execution via ordering constraints

The system transforms a small set of demos into diverse trajectories while preserving physical validity through simulation.

-----

🔑 Key Insights:

→ Just 60 source human demos can generate 21K valid training demonstrations

→ Asynchronous execution with separate action queues enables independent arm control

→ Synchronization during coordination tasks ensures precise bimanual movements

→ Ordering constraints maintain correct task sequencing

-----

📊 Results:

→ Improved success rates across tasks:

- Drawer Cleanup: 0.7% → 76.0%

- Threading: 1.3% → 69.3%

- Piece Assembly: 3.3% → 80.7%

→ 90% success rate on real-world can sorting task