Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imaging
This work addresses the problem of optimizing camera design for specific tasks, which is incremental as it builds on existing trends in combining physics and data.
The paper presents a framework for understanding end-to-end design of camera hardware and algorithms, highlighting the shift from physics-based to data-driven, task-specific approaches in imaging.
Cameras were originally designed using physics-based heuristics to capture aesthetic images. In recent years, there has been a transformation in camera design from being purely physics-driven to increasingly data-driven and task-specific. In this paper, we present a framework to understand the building blocks of this nascent field of end-to-end design of camera hardware and algorithms. As part of this framework, we show how methods that exploit both physics and data have become prevalent in imaging and computer vision, underscoring a key trend that will continue to dominate the future of task-specific camera design. Finally, we share current barriers to progress in end-to-end design, and hypothesize how these barriers can be overcome.