RayTracer.jl: A Differentiable Renderer that supports Parameter Optimization for Scene Reconstruction
This work provides a tool for researchers and practitioners in computer graphics and machine learning to optimize scene parameters for tasks like inverse graphics, though it is incremental as it builds on existing differentiable rendering concepts.
The authors tackled the problem of scene reconstruction by developing RayTracer.jl, a fully differentiable renderer in Julia that enables parameter optimization through source-to-source Automatic Differentiation, and demonstrated its application in rendering tasks and inverse graphics problems.
In this paper, we present RayTracer.jl, a renderer in Julia that is fully differentiable using source-to-source Automatic Differentiation (AD). This means that RayTracer not only renders 2D images from 3D scene parameters, but it can be used to optimize for model parameters that generate a target image in a Differentiable Programming (DP) pipeline. We interface our renderer with the deep learning library Flux for use in combination with neural networks. We demonstrate the use of this differentiable renderer in rendering tasks and in solving inverse graphics problems.