Paper shows smart velocity field splitting ensures AI-generated samples stay true to original data structure
Mathematical proof shows how to keep AI generation within realistic data boundaries
https://arxiv.org/abs/2410.23594
🎯 Original Problem:
Real-world data often exists in low-dimensional subspaces within high-dimensional spaces. Flow Matching models transform simple distributions into complex ones using velocity fields. The challenge is ensuring generated samples stay within the data subspace rather than drifting away.
-----
🔧 Solution in this Paper:
→ Derived analytical expressions for optimal velocity field under Gaussian prior
→ Introduced Orthogonal Subspace Decomposition Network (OSDNet) that splits velocity field into subspace and off-subspace components
→ The off-subspace component decays during training while subspace component generalizes within sample data subspace
→ Provided theoretical bounds on expected distance between generated samples and real data points
-----
💡 Key Insights:
→ Generated samples memorize real data points under optimal velocity field
→ Generation paths show distinct behaviors: direct movement toward nearest points for sparse data, hierarchical refinement for clustered data
→ OSDNet ensures generated samples preserve both proximity and diversity
→ The approach works independently of data dimensionality
-----
📊 Results:
→ Off-subspace components decay during training while subspace components generalize beyond real data points
→ Theoretical upper bound on expected distance between generated and real samples decreases as velocity field approximation improves
