Trace your sources in large-scale data: one ring to find them all
This work addresses the need for scalable and reliable source extraction in data analysis pipelines, offering a flexible and efficient solution for researchers and practitioners dealing with large datasets.
The authors tackled the problem of extracting key sources from large-scale data by developing a novel probabilistic blind source separation framework (DECOMPOSE), which demonstrated substantial improvements in accuracy and robustness on both artificial and real datasets.
An important preprocessing step in most data analysis pipelines aims to extract a small set of sources that explain most of the data. Currently used algorithms for blind source separation (BSS), however, often fail to extract the desired sources and need extensive cross-validation. In contrast, their rarely used probabilistic counterparts can get away with little cross-validation and are more accurate and reliable but no simple and scalable implementations are available. Here we present a novel probabilistic BSS framework (DECOMPOSE) that can be flexibly adjusted to the data, is extensible and easy to use, adapts to individual sources and handles large-scale data through algorithmic efficiency. DECOMPOSE encompasses and generalises many traditional BSS algorithms such as PCA, ICA and NMF and we demonstrate substantial improvements in accuracy and robustness on artificial and real data.