Learning and Compositionality: a Unification Attempt via Connectionist Probabilistic Programming
This work addresses the fundamental problem of integrating learning and compositionality for AI researchers, but it is incremental as it builds on existing hybrid neural-symbolic approaches.
The paper tackles the challenge of unifying learning and compositionality to simulate human-like intelligence by proposing Connectionist Probabilistic Programs (CPPs), a framework that combines connectionist structures for learning with probabilistic program semantics for compositionality. Early results show CPPs can extract concepts and relations from raw sequential data, though challenges remain in unsupervised learning of complex patterns.
We consider learning and compositionality as the key mechanisms towards simulating human-like intelligence. While each mechanism is successfully achieved by neural networks and symbolic AIs, respectively, it is the combination of the two mechanisms that makes human-like intelligence possible. Despite the numerous attempts on building hybrid neuralsymbolic systems, we argue that our true goal should be unifying learning and compositionality, the core mechanisms, instead of neural and symbolic methods, the surface approaches to achieve them. In this work, we review and analyze the strengths and weaknesses of neural and symbolic methods by separating their forms and meanings (structures and semantics), and propose Connectionist Probabilistic Program (CPPs), a framework that connects connectionist structures (for learning) and probabilistic program semantics (for compositionality). Under the framework, we design a CPP extension for small scale sequence modeling and provide a learning algorithm based on Bayesian inference. Although challenges exist in learning complex patterns without supervision, our early results demonstrate CPP's successful extraction of concepts and relations from raw sequential data, an initial step towards compositional learning.