Factor Analysis with Correlated Topic Model for Multi-Modal Data
This work addresses the problem of integrating structured and simple data modalities for researchers in fields like bioinformatics and multimedia analysis, offering an incremental improvement by extending factor analysis to handle clustering structures.
The paper tackled the challenge of multimodal factor analysis for structured data like text and single-cell sequencing, introducing FACTM, a Bayesian model combining factor analysis with correlated topic modeling, which outperformed other methods on benchmarks and real-world datasets in identifying clusters and inferring shared interpretable factors.
Integrating various data modalities brings valuable insights into underlying phenomena. Multimodal factor analysis (FA) uncovers shared axes of variation underlying different simple data modalities, where each sample is represented by a vector of features. However, FA is not suited for structured data modalities, such as text or single cell sequencing data, where multiple data points are measured per each sample and exhibit a clustering structure. To overcome this challenge, we introduce FACTM, a novel, multi-view and multi-structure Bayesian model that combines FA with correlated topic modeling and is optimized using variational inference. Additionally, we introduce a method for rotating latent factors to enhance interpretability with respect to binary features. On text and video benchmarks as well as real-world music and COVID-19 datasets, we demonstrate that FACTM outperforms other methods in identifying clusters in structured data, and integrating them with simple modalities via the inference of shared, interpretable factors.