Do Neural Networks Compress Manifolds Optimally?
This work addresses a fundamental problem in machine learning by challenging the assumed optimality of neural network compressors for manifold compression, which is incremental as it builds on prior claims of optimality.
The paper investigates whether neural network-based compressors achieve optimal entropy-distortion tradeoffs for low-dimensional manifolds, finding that state-of-the-art methods fail to compress two circular-structured manifolds optimally.
Artificial Neural-Network-based (ANN-based) lossy compressors have recently obtained striking results on several sources. Their success may be ascribed to an ability to identify the structure of low-dimensional manifolds in high-dimensional ambient spaces. Indeed, prior work has shown that ANN-based compressors can achieve the optimal entropy-distortion curve for some such sources. In contrast, we determine the optimal entropy-distortion tradeoffs for two low-dimensional manifolds with circular structure and show that state-of-the-art ANN-based compressors fail to optimally compress them.