Bayesian ICA with super-Gaussian Source Priors
This work addresses the need for robust and theoretically grounded ICA methods in machine learning, though it appears incremental by unifying existing strategies within a Bayesian framework.
The paper tackles the problem of improving Independent Component Analysis (ICA) by introducing a Bayesian framework with super-Gaussian source priors, resulting in scalable algorithms for estimation and inference, and establishing theoretical guarantees like posterior contraction and local asymptotic normality.
Independent Component Analysis (ICA) plays a central role in modern machine learning as a flexible framework for feature extraction. We introduce a horseshoe-type prior with a latent Polya-Gamma scale mixture representation, yielding scalable algorithms for both point estimation via expectation-maximization (EM) and full posterior inference via Markov chain Monte Carlo (MCMC). This hierarchical formulation unifies several previously disparate estimation strategies within a single Bayesian framework. We also establish the first theoretical guarantees for hierarchical Bayesian ICA, including posterior contraction and local asymptotic normality results for the unmixing matrix. Comprehensive simulation studies demonstrate that our methods perform competitively with widely used ICA tools. We further discuss implementation of conditional posteriors, envelope-based optimization, and possible extensions to flow-based architectures for nonlinear feature extraction and deep learning. Finally, we outline several promising directions for future work.