Contrastive Learning and the Emergence of Attributes Associations
This work addresses the limitation of sparse output in supervised learning for AI systems, but it is incremental as it builds on existing contrastive learning methods.
The paper tackles the problem that supervised learning outputs only a label for an object, whereas humans also generate attribute associations, and proposes that contrastive learning can preserve object attributes in its representations, enabling attribute detection. Simulation results demonstrate the feasibility of this idea.
In response to an object presentation, supervised learning schemes generally respond with a parsimonious label. Upon a similar presentation we humans respond again with a label, but are flooded, in addition, by a myriad of associations. A significant portion of these consist of the presented object attributes. Contrastive learning is a semi-supervised learning scheme based on the application of identity preserving transformations on the object input representations. It is conjectured in this work that these same applied transformations preserve, in addition to the identity of the presented object, also the identity of its semantically meaningful attributes. The corollary of this is that the output representations of such a contrastive learning scheme contain valuable information not only for the classification of the presented object, but also for the presence or absence decision of any attribute of interest. Simulation results which demonstrate this idea and the feasibility of this conjecture are presented.