On the Potential of CLIP for Compositional Logical Reasoning
This addresses the problem of evaluating logical reasoning capabilities in vision-language models for researchers in AI and machine learning, but it is incremental as it builds on existing CLIP analysis.
The paper investigates whether OpenAI's CLIP can perform logically coherent grounded visual reasoning, concluding that in its usual configuration, it cannot achieve this.
In this paper we explore the possibility of using OpenAI's CLIP to perform logically coherent grounded visual reasoning. To that end, we formalize our terms and give a geometric analysis of how embeddings in CLIP's latent space would need to be configured in order for the system to be logically coherent. Our main conclusion is that, as usually configured, CLIP cannot perform such reasoning.