Deep Generative Model with Beta Bernoulli Process for Modeling and Learning Confounding Factors
This addresses the challenge of handling unknown confounding factors in representation learning, which is incremental as it builds on existing deep generative and Bayesian methods.
The paper tackled the problem of disentangling task-relevant representations from an unknown and potentially infinite number of unobservable confounding factors in data, achieving this by combining deep generative models with Bayesian non-parametric factor models, as demonstrated on MNIST and clinical ECG datasets.
While deep representation learning has become increasingly capable of separating task-relevant representations from other confounding factors in the data, two significant challenges remain. First, there is often an unknown and potentially infinite number of confounding factors coinciding in the data. Second, not all of these factors are readily observable. In this paper, we present a deep conditional generative model that learns to disentangle a task-relevant representation from an unknown number of confounding factors that may grow infinitely. This is achieved by marrying the representational power of deep generative models with Bayesian non-parametric factor models, where a supervised deterministic encoder learns task-related representation and a probabilistic encoder with an Indian Buffet Process (IBP) learns the unknown number of unobservable confounding factors. We tested the presented model in two datasets: a handwritten digit dataset (MNIST) augmented with colored digits and a clinical ECG dataset with significant inter-subject variations and augmented with signal artifacts. These diverse data sets highlighted the ability of the presented model to grow with the complexity of the data and identify the absence or presence of unobserved confounding factors.