Learning about an exponential amount of conditional distributions
This addresses the challenge of integrating multiple self-supervised tasks and levels of supervision into a single model for machine learning applications, though it appears incremental as it builds on existing adversarial training methods.
The paper tackles the problem of learning all conditional distributions of a random vector by introducing the Neural Conditioner (NC), which uses adversarial training to match each conditional distribution and generalizes to sample from unseen distributions, including the joint distribution, while also providing data representations for downstream tasks.
We introduce the Neural Conditioner (NC), a self-supervised machine able to learn about all the conditional distributions of a random vector $X$. The NC is a function $NC(x \cdot a, a, r)$ that leverages adversarial training to match each conditional distribution $P(X_r|X_a=x_a)$. After training, the NC generalizes to sample from conditional distributions never seen, including the joint distribution. The NC is also able to auto-encode examples, providing data representations useful for downstream classification tasks. In sum, the NC integrates different self-supervised tasks (each being the estimation of a conditional distribution) and levels of supervision (partially observed data) seamlessly into a single learning experience.