Binary Independent Component Analysis: A Non-stationarity-based Approach
This work addresses a fundamental but underdeveloped problem in signal processing and machine learning for binary data analysis, though it appears incremental by adapting non-stationarity from continuous ICA.
The paper tackles independent component analysis for binary data by proposing a linear mixing model with non-stationary sources to address identifiability issues, proving non-identifiability with few variables and showing empirical identifiability with more variables.
We consider independent component analysis of binary data. While fundamental in practice, this case has been much less developed than ICA for continuous data. We start by assuming a linear mixing model in a continuous-valued latent space, followed by a binary observation model. Importantly, we assume that the sources are non-stationary; this is necessary since any non-Gaussianity would essentially be destroyed by the binarization. Interestingly, the model allows for closed-form likelihood by employing the cumulative distribution function of the multivariate Gaussian distribution. In stark contrast to the continuous-valued case, we prove non-identifiability of the model with few observed variables; our empirical results imply identifiability when the number of observed variables is higher. We present a practical method for binary ICA that uses only pairwise marginals, which are faster to compute than the full multivariate likelihood. Experiments give insight into the requirements for the number of observed variables, segments, and latent sources that allow the model to be estimated.