A Probabilistic Model for Discriminative and Neuro-Symbolic Semi-Supervised Learning
This work addresses semi-supervised learning for machine learning practitioners, offering a unified framework that integrates discriminative and neuro-symbolic methods, but it appears incremental as it builds upon and refines existing approaches.
The paper tackles the problem of semi-supervised learning by presenting a probabilistic model for discriminative SSL that mirrors generative approaches, showing that existing methods can be interpreted as approximations and improved upon, and extends it to neuro-symbolic SSL by linking logical rules to the model's prior, unifying statistical learning with logical reasoning.
Much progress has been made in semi-supervised learning (SSL) by combining methods that exploit different aspects of the data distribution, e.g. consistency regularisation relies on properties of $p(x)$, whereas entropy minimisation pertains to the label distribution $p(y|x)$. Focusing on the latter, we present a probabilistic model for discriminative SSL, that mirrors its classical generative counterpart. Under the assumption $y|x$ is deterministic, the prior over latent variables becomes discrete. We show that several well-known SSL methods can be interpreted as approximating this prior, and can be improved upon. We extend the discriminative model to neuro-symbolic SSL, where label features satisfy logical rules, by showing such rules relate directly to the above prior, thus justifying a family of methods that link statistical learning and logical reasoning, and unifying them with regular SSL.