MiNL: Micro-images based Neural Representation for Light Fields
This work addresses the problem of compact and efficient light field representation for computer vision and graphics applications, though it is incremental as it builds on existing implicit neural representation methods.
The paper tackles the inefficiency of pixel-wise implicit neural representations for light fields by proposing MiNL, a micro-image-wise representation that uses an MLP+CNN to map 2D coordinates to micro-image colors, resulting in an 80-180x speed-up in decoding and a 1-4dB average PSNR improvement.
Traditional representations for light fields can be separated into two types: explicit representation and implicit representation. Unlike explicit representation that represents light fields as Sub-Aperture Images (SAIs) based arrays or Micro-Images (MIs) based lenslet images, implicit representation treats light fields as neural networks, which is inherently a continuous representation in contrast to discrete explicit representation. However, at present almost all the implicit representations for light fields utilize SAIs to train an MLP to learn a pixel-wise mapping from 4D spatial-angular coordinate to pixel colors, which is neither compact nor of low complexity. Instead, in this paper we propose MiNL, a novel MI-wise implicit neural representation for light fields that train an MLP + CNN to learn a mapping from 2D MI coordinates to MI colors. Given the micro-image's coordinate, MiNL outputs the corresponding micro-image's RGB values. Light field encoding in MiNL is just training a neural network to regress the micro-images and the decoding process is a simple feedforward operation. Compared with common pixel-wise implicit representation, MiNL is more compact and efficient that has faster decoding speed (\textbf{$\times$80$\sim$180} speed-up) as well as better visual quality (\textbf{1$\sim$4dB} PSNR improvement on average).