NeuralMagicEye: Learning to See and Understand the Scene Behind an Autostereogram
This research addresses the problem of perceiving 3D scenes from autostereograms for individuals with visual impairments or those who struggle with viewing these images, offering a computational solution.
This paper investigates whether a deep CNN can recover depth and understand the content behind an autostereogram. The authors propose "NeuralMagicEye," a deep CNN embedded with a novel disparity convolution layer, which successfully recovers detailed and smooth depth from autostereograms after self-supervised training on a large 3D object dataset.
An autostereogram, a.k.a. magic eye image, is a single-image stereogram that can create visual illusions of 3D scenes from 2D textures. This paper studies an interesting question that whether a deep CNN can be trained to recover the depth behind an autostereogram and understand its content. The key to the autostereogram magic lies in the stereopsis - to solve such a problem, a model has to learn to discover and estimate disparity from the quasi-periodic textures. We show that deep CNNs embedded with disparity convolution, a novel convolutional layer proposed in this paper that simulates stereopsis and encodes disparity, can nicely solve such a problem after being sufficiently trained on a large 3D object dataset in a self-supervised fashion. We refer to our method as ``NeuralMagicEye''. Experiments show that our method can accurately recover the depth behind autostereograms with rich details and gradient smoothness. Experiments also show the completely different working mechanisms for autostereogram perception between neural networks and human eyes. We hope this research can help people with visual impairments and those who have trouble viewing autostereograms. Our code is available at \url{https://jiupinjia.github.io/neuralmagiceye/}.