LoFi: Neural Local Fields for Scalable Image Reconstruction
This addresses the memory and data efficiency challenges in imaging inverse problems for computer vision and imaging applications, though it appears incremental as it builds on existing neural field concepts.
The authors tackled the problem of scalable image reconstruction by introducing LoFi, a coordinate-based framework that processes local information at each coordinate separately, achieving comparable or better performance than CNNs and ViTs with memory usage under 200MB for 1024x1024 images and generalization to out-of-distribution data.
Neural fields or implicit neural representations (INRs) have attracted significant attention in computer vision and imaging due to their efficient coordinate-based representation of images and 3D volumes. In this work, we introduce a coordinate-based framework for solving imaging inverse problems, termed LoFi (Local Field). Unlike conventional methods for image reconstruction, LoFi processes local information at each coordinate separately by multi-layer perceptrons (MLPs), recovering the object at that specific coordinate. Similar to INRs, LoFi can recover images at any continuous coordinate, enabling image reconstruction at multiple resolutions. With comparable or better performance than standard deep learning models like convolutional neural networks (CNNs) and vision transformers (ViTs), LoFi achieves excellent generalization to out-of-distribution data with memory usage almost independent of image resolution. Remarkably, training on 1024x1024 images requires less than 200MB of memory -- much below standard CNNs and ViTs. Additionally, LoFi's local design allows it to train on extremely small datasets with 10 samples or fewer, without overfitting and without the need for explicit regularization or early stopping.