Meta-Learning Sparse Compression Networks
This work addresses the problem of making INRs more efficient and scalable for compression tasks, benefiting researchers and practitioners in fields like computer vision and graphics, though it builds incrementally on prior INR and sparsification methods.
The paper tackles the challenge of scaling Implicit Neural Representations (INRs) for data compression by integrating network sparsification techniques and enabling sparsification within meta-learning algorithms, resulting in drastically improved compression and reduced computational costs across diverse data modalities, with several new state-of-the-art results.
Recent work in Deep Learning has re-imagined the representation of data as functions mapping from a coordinate space to an underlying continuous signal. When such functions are approximated by neural networks this introduces a compelling alternative to the more common multi-dimensional array representation. Recent work on such Implicit Neural Representations (INRs) has shown that - following careful architecture search - INRs can outperform established compression methods such as JPEG (e.g. Dupont et al., 2021). In this paper, we propose crucial steps towards making such ideas scalable: Firstly, we employ state-of-the-art network sparsification techniques to drastically improve compression. Secondly, introduce the first method allowing for sparsification to be employed in the inner-loop of commonly used Meta-Learning algorithms, drastically improving both compression and the computational cost of learning INRs. The generality of this formalism allows us to present results on diverse data modalities such as images, manifolds, signed distance functions, 3D shapes and scenes, several of which establish new state-of-the-art results.