Learning Symbolic Representations Through Joint GEnerative and DIscriminative Training
This work addresses the challenge of integrating symbolic reasoning into machine learning for domains like computer vision, though it appears incremental as it builds on existing neuro-symbolic frameworks.
The paper tackles the problem of learning symbolic representations by introducing GEDI, a Bayesian framework that combines generative and discriminative self-supervised learning, resulting in improved clustering performance on datasets like SVHN, CIFAR10, and CIFAR100 with significant margins.
We introduce GEDI, a Bayesian framework that combines existing self-supervised learning objectives with likelihood-based generative models. This framework leverages the benefits of both GEnerative and DIscriminative approaches, resulting in improved symbolic representations over standalone solutions. Additionally, GEDI can be easily integrated and trained jointly with existing neuro-symbolic frameworks without the need for additional supervision or costly pre-training steps. We demonstrate through experiments on real-world data, including SVHN, CIFAR10, and CIFAR100, that GEDI outperforms existing self-supervised learning strategies in terms of clustering performance by a significant margin. The symbolic component further allows it to leverage knowledge in the form of logical constraints to improve performance in the small data regime.