Deforming Radiance Fields with Cages
This addresses a limitation in 3D scene manipulation and animation for computer graphics and vision applications, representing an incremental advancement by extending cage-based deformation from meshes to radiance fields.
The paper tackles the problem of enabling explicit deformation for radiance fields, which was previously unsupported, by proposing a method that uses a triangular mesh cage to achieve free-form radiance field deformation, with results demonstrating effectiveness on synthetic and real-world datasets.
Recent advances in radiance fields enable photorealistic rendering of static or dynamic 3D scenes, but still do not support explicit deformation that is used for scene manipulation or animation. In this paper, we propose a method that enables a new type of deformation of the radiance field: free-form radiance field deformation. We use a triangular mesh that encloses the foreground object called cage as an interface, and by manipulating the cage vertices, our approach enables the free-form deformation of the radiance field. The core of our approach is cage-based deformation which is commonly used in mesh deformation. We propose a novel formulation to extend it to the radiance field, which maps the position and the view direction of the sampling points from the deformed space to the canonical space, thus enabling the rendering of the deformed scene. The deformation results of the synthetic datasets and the real-world datasets demonstrate the effectiveness of our approach.