Occupancy Planes for Single-view RGB-D Human Reconstruction
This work improves human reconstruction from single RGB-D views, particularly for handling occlusions, but appears incremental as it builds on existing implicit function approaches.
The paper tackles the problem of single-view RGB-D human reconstruction by addressing the sub-optimal correlation handling in per-point classification methods, proposing occupancy planes (OPlanes) to improve accuracy, especially in occluded scenarios, with compelling results on the S3D dataset.
Single-view RGB-D human reconstruction with implicit functions is often formulated as per-point classification. Specifically, a set of 3D locations within the view-frustum of the camera are first projected independently onto the image and a corresponding feature is subsequently extracted for each 3D location. The feature of each 3D location is then used to classify independently whether the corresponding 3D point is inside or outside the observed object. This procedure leads to sub-optimal results because correlations between predictions for neighboring locations are only taken into account implicitly via the extracted features. For more accurate results we propose the occupancy planes (OPlanes) representation, which enables to formulate single-view RGB-D human reconstruction as occupancy prediction on planes which slice through the camera's view frustum. Such a representation provides more flexibility than voxel grids and enables to better leverage correlations than per-point classification. On the challenging S3D data we observe a simple classifier based on the OPlanes representation to yield compelling results, especially in difficult situations with partial occlusions due to other objects and partial visibility, which haven't been addressed by prior work.