Modular Primitives for High-Performance Differentiable Rendering
This work addresses the need for faster and more flexible differentiable rendering tools for researchers and practitioners in computer vision and graphics, though it is incremental as it builds on existing hardware optimizations.
The paper tackles the problem of slow differentiable rendering by introducing a modular design that leverages optimized hardware graphics pipelines, achieving superior performance and supporting high-resolution operations like rasterization and texture lookups. As a result, it enables efficient facial performance capture with excellent geometric correspondence to reference imagery.
We present a modular differentiable renderer design that yields performance superior to previous methods by leveraging existing, highly optimized hardware graphics pipelines. Our design supports all crucial operations in a modern graphics pipeline: rasterizing large numbers of triangles, attribute interpolation, filtered texture lookups, as well as user-programmable shading and geometry processing, all in high resolutions. Our modular primitives allow custom, high-performance graphics pipelines to be built directly within automatic differentiation frameworks such as PyTorch or TensorFlow. As a motivating application, we formulate facial performance capture as an inverse rendering problem and show that it can be solved efficiently using our tools. Our results indicate that this simple and straightforward approach achieves excellent geometric correspondence between rendered results and reference imagery.