Prediction of Scene Plausibility
This addresses a new challenge in scene understanding for computer vision researchers, focusing on physical and typical arrangements rather than just object detection.
The paper tackles the problem of distinguishing plausible from implausible scenes in 2D images, defining plausibility based on physical and functional arrangements, and tests vision models on a synthetic dataset to evaluate their success in this task.
Understanding the 3D world from 2D images involves more than detection and segmentation of the objects within the scene. It also includes the interpretation of the structure and arrangement of the scene elements. Such understanding is often rooted in recognizing the physical world and its limitations, and in prior knowledge as to how similar typical scenes are arranged. In this research we pose a new challenge for neural network (or other) scene understanding algorithms - can they distinguish between plausible and implausible scenes? Plausibility can be defined both in terms of physical properties and in terms of functional and typical arrangements. Hence, we define plausibility as the probability of encountering a given scene in the real physical world. We build a dataset of synthetic images containing both plausible and implausible scenes, and test the success of various vision models in the task of recognizing and understanding plausibility.