See the Glass Half Full: Reasoning about Liquid Containers, their Volume and Content
This addresses a gap in computer vision for understanding liquids and containers, which is incremental as it applies existing methods to a new domain.
The paper tackles the problem of reasoning about liquid containers and their contents from a single RGB image, proposing methods to estimate container volume, approximate liquid amount, and predict liquid behavior when tilting, achieving results on a new dataset of over 5,000 images.
Humans have rich understanding of liquid containers and their contents; for example, we can effortlessly pour water from a pitcher to a cup. Doing so requires estimating the volume of the cup, approximating the amount of water in the pitcher, and predicting the behavior of water when we tilt the pitcher. Very little attention in computer vision has been made to liquids and their containers. In this paper, we study liquid containers and their contents, and propose methods to estimate the volume of containers, approximate the amount of liquid in them, and perform comparative volume estimations all from a single RGB image. Furthermore, we show the results of the proposed model for predicting the behavior of liquids inside containers when one tilts the containers. We also introduce a new dataset of Containers Of liQuid contEnt (COQE) that contains more than 5,000 images of 10,000 liquid containers in context labelled with volume, amount of content, bounding box annotation, and corresponding similar 3D CAD models.