Differentiable Rendering: A Survey
It provides a comprehensive overview for researchers in computer vision and graphics, but is incremental as it synthesizes existing literature without new results.
This survey paper reviews the field of differentiable rendering, which addresses the lack of 3D understanding in deep neural networks by enabling gradient calculation through images, reducing the need for 3D data collection and annotation.
Deep neural networks (DNNs) have shown remarkable performance improvements on vision-related tasks such as object detection or image segmentation. Despite their success, they generally lack the understanding of 3D objects which form the image, as it is not always possible to collect 3D information about the scene or to easily annotate it. Differentiable rendering is a novel field which allows the gradients of 3D objects to be calculated and propagated through images. It also reduces the requirement of 3D data collection and annotation, while enabling higher success rate in various applications. This paper reviews existing literature and discusses the current state of differentiable rendering, its applications and open research problems.