Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations
This work addresses the problem of interpretable and interactive concept learning in machine learning, particularly for visual domains, though it appears incremental as it builds on existing prototype-based methods.
The paper tackles the challenge of learning visual concepts from raw images without strong supervision by introducing interactive Concept Swapping Networks (iCSNs), which use prototype representations and weak supervision to achieve concept-grounded latent spaces, as demonstrated on a new dataset called Elementary Concept Reasoning (ECR) with geometric objects.
Learning visual concepts from raw images without strong supervision is a challenging task. In this work, we show the advantages of prototype representations for understanding and revising the latent space of neural concept learners. For this purpose, we introduce interactive Concept Swapping Networks (iCSNs), a novel framework for learning concept-grounded representations via weak supervision and implicit prototype representations. iCSNs learn to bind conceptual information to specific prototype slots by swapping the latent representations of paired images. This semantically grounded and discrete latent space facilitates human understanding and human-machine interaction. We support this claim by conducting experiments on our novel data set "Elementary Concept Reasoning" (ECR), focusing on visual concepts shared by geometric objects.