Robust And Scalable Learning Of Complex Dataset Topologies Via Elpigraph
This method addresses the problem of analyzing complex, noisy datasets for researchers in fields like single-cell biology and astronomy, offering a scalable alternative to existing local-neighborhood-based approaches, though it appears incremental as it builds on principal graph concepts.
The authors tackled the challenge of reducing complexity and producing interpretable representations for datasets with non-trivial distributions, such as branching trajectories, by introducing ElPiGraph, a scalable and robust method that approximates complex topologies without requiring complete distance matrices or neighborhood graphs, achieving efficiency in handling large, noisy datasets across fields like biology and astronomy.
Large datasets represented by multidimensional data point clouds often possess non-trivial distributions with branching trajectories and excluded regions, with the recent single-cell transcriptomic studies of developing embryo being notable examples. Reducing the complexity and producing compact and interpretable representations of such data remains a challenging task. Most of the existing computational methods are based on exploring the local data point neighbourhood relations, a step that can perform poorly in the case of multidimensional and noisy data. Here we present ElPiGraph, a scalable and robust method for approximation of datasets with complex structures which does not require computing the complete data distance matrix or the data point neighbourhood graph. This method is able to withstand high levels of noise and is capable of approximating complex topologies via principal graph ensembles that can be combined into a consensus principal graph. ElPiGraph deals efficiently with large and complex datasets in various fields from biology, where it can be used to infer gene dynamics from single-cell RNA-Seq, to astronomy, where it can be used to explore complex structures in the distribution of galaxies.