Video models learn physics through pattern matching, not actual physical laws
Scaling alone can't teach AI true physics - it needs diverse examples
AI prefers memorizing physics examples over learning fundamental rules
https://arxiv.org/abs/2411.02385
🎯 Original Problem:
OpenAI's Sora shows video generation models might develop world models following physical laws. But can these models truly discover physical laws from visual data alone without human guidance? We need to evaluate if they can robustly predict and extrapolate to unseen scenarios.
-----
🔬 Solution in this Paper:
→ Created a 2D simulation testbed for testing video generation models across three scenarios: in-distribution, out-of-distribution, and combinatorial generalization
→ Used VAE-DiT architecture (like Sora) with models ranging from 22M to 456M parameters
→ Generated large-scale datasets (30K to 3M examples) simulating basic physics: uniform motion, elastic collisions, parabolic trajectories
→ Developed quantitative metrics to evaluate physical plausibility using velocity error measurements
-----
🧪 Key Insights:
→ Models show "case-based" generalization instead of learning abstract physical rules
→ When generalizing, models prioritize: color > size > velocity > shape
→ Simply scaling model size and data volume doesn't help with out-of-distribution scenarios
→ Increasing combinatorial diversity in training data significantly improves performance
-----
📊 Results:
→ Perfect in-distribution generalization achieved with increased data and model size
→ Out-of-distribution performance showed no improvement with scaling
→ For combinatorial tasks, increased data diversity reduced physically implausible cases from 67% to 10%
Share this post