Generative Image Modeling Using Spatial LSTMs
This work addresses the challenge of capturing long-range dependencies in images for generative modeling, which is important for applications like texture synthesis and inpainting, though it appears incremental as it builds on existing recurrent neural network approaches.
The authors tackled the problem of modeling natural image distributions by introducing a recurrent image model based on spatial LSTMs, which scales to arbitrary image sizes and has a computationally tractable likelihood. They found that it outperforms state-of-the-art methods in quantitative comparisons on several datasets and shows promise in texture synthesis and inpainting.
Modeling the distribution of natural images is challenging, partly because of strong statistical dependencies which can extend over hundreds of pixels. Recurrent neural networks have been successful in capturing long-range dependencies in a number of problems but only recently have found their way into generative image models. We here introduce a recurrent image model based on multi-dimensional long short-term memory units which are particularly suited for image modeling due to their spatial structure. Our model scales to images of arbitrary size and its likelihood is computationally tractable. We find that it outperforms the state of the art in quantitative comparisons on several image datasets and produces promising results when used for texture synthesis and inpainting.