Bayesian Beta-Bernoulli Process Sparse Coding with Deep Neural Networks
This addresses the problem of sparse coding in deep discrete latent variable models for researchers in machine learning, though it appears incremental as it adapts non-parametric methods from classical sparse coding to deep contexts.
The paper tackles the challenge of learning discrete latent representations in deep models by proposing a non-parametric iterative algorithm with a Beta-Bernoulli process prior to encourage sparsity, and evaluates it across diverse datasets compared to existing amortized inference methods.
Several approximate inference methods have been proposed for deep discrete latent variable models. However, non-parametric methods which have previously been successfully employed for classical sparse coding models have largely been unexplored in the context of deep models. We propose a non-parametric iterative algorithm for learning discrete latent representations in such deep models. Additionally, to learn scale invariant discrete features, we propose local data scaling variables. Lastly, to encourage sparsity in our representations, we propose a Beta-Bernoulli process prior on the latent factors. We evaluate our spare coding model coupled with different likelihood models. We evaluate our method across datasets with varying characteristics and compare our results to current amortized approximate inference methods.