"Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning"

Emma-X teaches robots to think ahead spatially while executing complex manipulation tasks

Emma-X introduces a new way to make robots understand and execute complex tasks by combining visual understanding with spatial reasoning and grounded task planning.

-----

https://arxiv.org/abs/2412.11974

🤖 Original Problem:

Traditional robot control methods are task-specific and struggle with generalization. While LLMs excel at scene understanding, they can't generate actionable robot policies. Current Visual-Language-Action models lack spatial reasoning and grounded task planning abilities.

-----

🔬 Solution in this Paper:

→ Emma-X builds a hierarchical dataset with 60,000 robot trajectories, each annotated with task reasoning and spatial guidance

→ Uses dual segmentation approach combining gripper states and motion trajectories to reduce reasoning hallucination

→ Implements look-ahead spatial reasoning by predicting future gripper positions and movement plans

→ Leverages Gemini for generating grounded task reasoning with visual context

→ Integrates with OpenVLA's 7B parameter architecture for policy generation

-----

🎯 Key Insights:

→ Visual grounding significantly reduces hallucination in task reasoning

→ Look-ahead spatial planning improves long-horizon task completion

→ Trajectory segmentation provides better context for reasoning

-----

📊 Results:

→ 24.17% higher task success rate than OpenVLA baseline

→ 35% improvement in spatial relation tasks

→ 26.25% increase in half-success rate across all tasks