Differentiable Visual Computing for Inverse Problems and Machine Learning
This work addresses a foundational problem for researchers in computer graphics and machine learning by proposing a novel integration of VC and DL, though it appears incremental as it builds on existing paradigms without claiming specific performance gains.
The paper tackles the challenge of integrating visual computing (VC) methods, which excel at forward problems using prescribed algorithms, with deep learning (DL) approaches that solve inverse problems via differentiable neural networks, aiming to bridge these domains for improved performance in inverse problems and machine learning applications.
Originally designed for applications in computer graphics, visual computing (VC) methods synthesize information about physical and virtual worlds, using prescribed algorithms optimized for spatial computing. VC is used to analyze geometry, physically simulate solids, fluids, and other media, and render the world via optical techniques. These fine-tuned computations that operate explicitly on a given input solve so-called forward problems, VC excels at. By contrast, deep learning (DL) allows for the construction of general algorithmic models, side stepping the need for a purely first principles-based approach to problem solving. DL is powered by highly parameterized neural network architectures -- universal function approximators -- and gradient-based search algorithms which can efficiently search that large parameter space for optimal models. This approach is predicated by neural network differentiability, the requirement that analytic derivatives of a given problem's task metric can be computed with respect to neural network's parameters. Neural networks excel when an explicit model is not known, and neural network training solves an inverse problem in which a model is computed from data.