Learning the image processing pipeline
This work addresses the time-consuming and costly need for manual pipeline design in applications like consumer photography and computer vision, though it appears incremental as it builds on existing simulation and machine learning techniques.
The paper tackles the problem of designing and optimizing image processing pipelines for novel sensor architectures, proposing a method that automates this process using machine learning and image systems simulation, and demonstrates its application in consumer photography.
Many creative ideas are being proposed for image sensor designs, and these may be useful in applications ranging from consumer photography to computer vision. To understand and evaluate each new design, we must create a corresponding image processing pipeline that transforms the sensor data into a form that is appropriate for the application. The need to design and optimize these pipelines is time-consuming and costly. We explain a method that combines machine learning and image systems simulation that automates the pipeline design. The approach is based on a new way of thinking of the image processing pipeline as a large collection of local linear filters. We illustrate how the method has been used to design pipelines for novel sensor architectures in consumer photography applications.