User-in-the-Loop View Sampling with Error Peaking Visualization
This addresses the challenge of user burden and limited scene exploration in mobile view synthesis systems, though it appears incremental as it builds on existing locally reconstructed light fields.
The paper tackles the problem of mentally demanding and restrictive view sampling in augmented reality for novel view synthesis by proposing error-peaking visualization instead of 3D annotations, resulting in less invasive data collection, reduced disappointment, and satisfactory performance with fewer view samples.
Augmented reality (AR) provides ways to visualize missing view samples for novel view synthesis. Existing approaches present 3D annotations for new view samples and task users with taking images by aligning the AR display. This data collection task is known to be mentally demanding and limits capture areas to pre-defined small areas due to the ideal but restrictive underlying sampling theory. To free users from 3D annotations and limited scene exploration, we propose using locally reconstructed light fields and visualizing errors to be removed by inserting new views. Our results show that the error-peaking visualization is less invasive, reduces disappointment in final results, and is satisfactory with fewer view samples in our mobile view synthesis system. We also show that our approach can contribute to recent radiance field reconstruction for larger scenes, such as 3D Gaussian splatting.