Discrete Independent Component Analysis (DICA) with Belief Propagation
This work provides a method for discrete data analysis, but it is incremental as it adapts existing techniques to a new domain without broad impact.
The authors tackled the problem of performing independent component analysis on discrete data by applying belief propagation to a Bayesian bipartite graph, resulting in a generative model that effectively represents MNIST character images through factorial codes.
We apply belief propagation to a Bayesian bipartite graph composed of discrete independent hidden variables and discrete visible variables. The network is the Discrete counterpart of Independent Component Analysis (DICA) and it is manipulated in a factor graph form for inference and learning. A full set of simulations is reported for character images from the MNIST dataset. The results show that the factorial code implemented by the sources contributes to build a good generative model for the data that can be used in various inference modes.