Stylizing Sparse-View 3D Scenes with Hierarchical Neural Representation
This work addresses the challenge of 3D style transfer for sparse-view scenes, which is incremental as it builds on existing neural radiance field methods to improve stylization in data-limited scenarios.
The paper tackles the problem of generating high-quality stylized 3D scenes from sparse input views by proposing a coarse-to-fine framework with hierarchical neural representation and content strength annealing, achieving superior stylization quality and efficiency compared to fine-tuning-based baselines.
Recently, a surge of 3D style transfer methods has been proposed that leverage the scene reconstruction power of a pre-trained neural radiance field (NeRF). To successfully stylize a scene this way, one must first reconstruct a photo-realistic radiance field from collected images of the scene. However, when only sparse input views are available, pre-trained few-shot NeRFs often suffer from high-frequency artifacts, which are generated as a by-product of high-frequency details for improving reconstruction quality. Is it possible to generate more faithful stylized scenes from sparse inputs by directly optimizing encoding-based scene representation with target style? In this paper, we consider the stylization of sparse-view scenes in terms of disentangling content semantics and style textures. We propose a coarse-to-fine sparse-view scene stylization framework, where a novel hierarchical encoding-based neural representation is designed to generate high-quality stylized scenes directly from implicit scene representations. We also propose a new optimization strategy with content strength annealing to achieve realistic stylization and better content preservation. Extensive experiments demonstrate that our method can achieve high-quality stylization of sparse-view scenes and outperforms fine-tuning-based baselines in terms of stylization quality and efficiency.