Discovering Causal Signals in Images
This work addresses the challenge of causal discovery in computer vision, which is foundational for understanding image semantics, but it is incremental as it builds on existing causal inference methods.
The paper tackles the problem of identifying causal relationships in images by developing a classifier that achieves state-of-the-art performance in determining causal direction between variables, and uses it to distinguish object features from context features in static images, demonstrating observable signals of causal dispositions.
This paper establishes the existence of observable footprints that reveal the "causal dispositions" of the object categories appearing in collections of images. We achieve this goal in two steps. First, we take a learning approach to observational causal discovery, and build a classifier that achieves state-of-the-art performance on finding the causal direction between pairs of random variables, given samples from their joint distribution. Second, we use our causal direction classifier to effectively distinguish between features of objects and features of their contexts in collections of static images. Our experiments demonstrate the existence of a relation between the direction of causality and the difference between objects and their contexts, and by the same token, the existence of observable signals that reveal the causal dispositions of objects.