"Relightable Full-Body Gaussian Codec Avatars"

This paper addresses the challenge of realistically relighting full-body avatars, which is complicated by body pose changes affecting light interaction and appearance.

The paper introduces a method to decompose and model local and non-local light transport effects for improved relighting.

-----

https://arxiv.org/abs/2501.14726

Original Problem 🙁:

→ Relighting full-body avatars is challenging.

→ Body articulation causes significant shape changes.

→ These shape changes alter how light interacts with the avatar's surface.

→ Pose-dependent deformations impact both local appearance and global shadowing.

-----

Solution in this Paper 💡:

→ The paper proposes Relightable Full-Body Gaussian Codec Avatars.

→ This method decomposes light transport into local and non-local effects.

→ Local appearance changes are modeled using learnable zonal harmonics for diffuse radiance transfer.

→ Zonal harmonics are efficient to rotate, handling articulation well.

→ This allows learning diffuse light transfer independent of body pose.

→ Non-local shadow effects are handled by a shadow network.

→ This network predicts shadows based on pre-computed irradiance.

→ Deferred shading is used to model specular reflections and highlights.

→ This approach captures both local and non-local light transport for relighting avatars.

-----

Key Insights from this Paper 🤔:

→ Decomposing light transport into local and non-local components simplifies relighting.

→ Using zonal harmonics in local coordinates disentangles light transfer from pose.

→ A separate shadow network effectively models non-local shadowing effects.

→ Deferred shading improves specular reflection modeling.

-----

Results 🎉:

→ The method demonstrates superior generalization to novel lighting.

→ It also generalizes well to unseen poses.

→ The approach successfully models light transport for relightable full-body avatars.