Image Based Reconstruction of Liquids from 2D Surface Detections
This addresses the challenge of liquid reconstruction for computer vision applications, which is incremental as it builds on prior surface reconstruction methods but focuses on liquids.
The paper tackles the problem of reconstructing liquids from 2D surface detections by posing a novel optimization problem that uses particles to minimize error between rendered surfaces and observations, without requiring training data or prior knowledge of liquid properties, and demonstrates results on new open-sourced datasets.
In this work, we present a solution to the challenging problem of reconstructing liquids from image data. The challenges in reconstructing liquids, which is not faced in previous reconstruction works on rigid and deforming surfaces, lies in the inability to use depth sensing and color features due the variable index of refraction, opacity, and environmental reflections. Therefore, we limit ourselves to only surface detections (i.e. binary mask) of liquids as observations and do not assume any prior knowledge on the liquids properties. A novel optimization problem is posed which reconstructs the liquid as particles by minimizing the error between a rendered surface from the particles and the surface detections while satisfying liquid constraints. Our solvers to this optimization problem are presented and no training data is required to apply them. We also propose a dynamic prediction to seed the reconstruction optimization from the previous time-step. We test our proposed methods in simulation and on two new liquid datasets which we open source so the broader research community can continue developing in this under explored area.