Fast Topology-Aware Lossy Data Compression with Full Preservation of Critical Points and Local Order
For scientists and engineers dealing with large floating-point datasets, this compressor enables fast, topology-preserving compression that maintains data integrity for downstream analysis.
The paper introduces a lossy compressor that preserves full local order and all critical points, achieving orders-of-magnitude faster compression than prior topology-preserving methods while maintaining higher compression ratios than lossless compressors and ensuring bit-for-bit reproducibility across CPUs and GPUs.
Many scientific codes and instruments generate large amounts of floating-point data at high rates that must be compressed before they can be stored. Typically, only lossy compression algorithms deliver high-enough compression ratios. However, many of them provide only point-wise error bounds and do not preserve topological aspects of the data such as the relative magnitude of neighboring points. Even topology-preserving compressors tend to merely preserve some critical points and are generally slow. Our Local-Order-Preserving Compressor is the first to preserve the full local order (and thus all critical points), runs orders of magnitude faster than prior topology-preserving compressors, yields higher compression ratios than lossless compressors, and produces bit-for-bit the same output on CPUs and GPUs.