Inferring Fluid Dynamics via Inverse Rendering
This work addresses the challenge of learning physical simulations from visual data, which could benefit fields like computer graphics and robotics, though it appears incremental as it builds on existing differentiable rendering and simulation techniques.
The paper tackles the problem of reconstructing fluid dynamics from unannotated videos using inverse rendering, achieving this by integrating a differentiable Euler simulator with a volumetric renderer to infer fluid properties from video frames without ground-truth supervision.
Humans have a strong intuitive understanding of physical processes such as fluid falling by just a glimpse of such a scene picture, i.e., quickly derived from our immersive visual experiences in memory. This work achieves such a photo-to-fluid-dynamics reconstruction functionality learned from unannotated videos, without any supervision of ground-truth fluid dynamics. In a nutshell, a differentiable Euler simulator modeled with a ConvNet-based pressure projection solver, is integrated with a volumetric renderer, supporting end-to-end/coherent differentiable dynamic simulation and rendering. By endowing each sampled point with a fluid volume value, we derive a NeRF-like differentiable renderer dedicated from fluid data; and thanks to this volume-augmented representation, fluid dynamics could be inversely inferred from the error signal between the rendered result and ground-truth video frame (i.e., inverse rendering). Experiments on our generated Fluid Fall datasets and DPI Dam Break dataset are conducted to demonstrate both effectiveness and generalization ability of our method.