Limitations of NERF with pre-trained Vision Features for Few-Shot 3D Reconstruction
This challenges common assumptions in the field, suggesting that pre-trained vision features may not be beneficial for few-shot 3D reconstruction, which is an incremental finding for researchers in 3D computer vision.
The paper systematically evaluated DINO-enhanced NeRF models for few-shot 3D reconstruction and found that all variants performed worse than baseline NeRF, with PSNR values around 12.9-13.0 compared to the baseline's 14.71.
Neural Radiance Fields (NeRF) have revolutionized 3D scene reconstruction from sparse image collections. Recent work has explored integrating pre-trained vision features, particularly from DINO, to enhance few-shot reconstruction capabilities. However, the effectiveness of such approaches remains unclear, especially in extreme few-shot scenarios. In this paper, we present a systematic evaluation of DINO-enhanced NeRF models, comparing baseline NeRF, frozen DINO features, LoRA fine-tuned features, and multi-scale feature fusion. Surprisingly, our experiments reveal that all DINO variants perform worse than the baseline NeRF, achieving PSNR values around 12.9 to 13.0 compared to the baseline's 14.71. This counterintuitive result suggests that pre-trained vision features may not be beneficial for few-shot 3D reconstruction and may even introduce harmful biases. We analyze potential causes including feature-task mismatch, overfitting to limited data, and integration challenges. Our findings challenge common assumptions in the field and suggest that simpler architectures focusing on geometric consistency may be more effective for few-shot scenarios.