TIFu: Tri-directional Implicit Function for High-Fidelity 3D Character Reconstruction
This work addresses the problem of accurate 3D reconstruction for animated characters and humans, which is incremental by building on implicit function approaches with a novel vector-level representation.
The paper tackles the problem of generating high-fidelity 3D character reconstructions from single RGB images by addressing issues like dragged or disconnected body parts in existing implicit function methods. It introduces TIFu, a tri-directional implicit function that improves global 3D consistency and reduces memory usage, achieving state-of-the-art results on character and human datasets.
Recent advances in implicit function-based approaches have shown promising results in 3D human reconstruction from a single RGB image. However, these methods are not sufficient to extend to more general cases, often generating dragged or disconnected body parts, particularly for animated characters. We argue that these limitations stem from the use of the existing point-level 3D shape representation, which lacks holistic 3D context understanding. Voxel-based reconstruction methods are more suitable for capturing the entire 3D space at once, however, these methods are not practical for high-resolution reconstructions due to their excessive memory usage. To address these challenges, we introduce Tri-directional Implicit Function (TIFu), which is a vector-level representation that increases global 3D consistencies while significantly reducing memory usage compared to voxel representations. We also introduce a new algorithm in 3D reconstruction at an arbitrary resolution by aggregating vectors along three orthogonal axes, resolving inherent problems with regressing fixed dimension of vectors. Our approach achieves state-of-the-art performances in both our self-curated character dataset and the benchmark 3D human dataset. We provide both quantitative and qualitative analyses to support our findings.