High-Performance Star-M SVD for Big Data Compression
This work provides an efficient software tool for compressing large scientific datasets using tensor methods, addressing the need for high-performance implementations in this domain.
The authors developed a high-performance, shared-memory parallel implementation of the star-M tensor SVD for big data compression, achieving efficient compression of large scientific datasets with minimal accuracy loss.
In the era of big data, effectively compressing large datasets while performing complex mathematical operations is crucial. Tensor-based decomposition methods have shown superior compression capabilities with minimal loss of accuracy compared to traditional matrix methods. Under the star-M tensor framework, tensors can be decomposed in a matrix-mimetic way, including using the star-M SVD. This tensor SVD has optimality guarantees and has shown exceptional performance on specific types of data, but software implementations have been mostly limited to productivity-oriented languages. In this work, we present our development of a shared-memory parallel, high-performance solution designed to efficiently implement the underlying algorithms. This software will enable optimal compression of extensive scientific datasets, paving the way for enhanced data analysis and insights.