Pixel VQ-VAEs for Improved Pixel Art Representation
This addresses a niche problem for pixel art creators and researchers, but it is incremental as it adapts an existing method to a specific domain.
The authors tackled the problem of representing pixel art images, which are often ignored by traditional image processing models, by proposing a specialized VQ-VAE model. They showed that it outperforms other models in embedding quality and downstream task performance.
Machine learning has had a great deal of success in image processing. However, the focus of this work has largely been on realistic images, ignoring more niche art styles such as pixel art. Additionally, many traditional machine learning models that focus on groups of pixels do not work well with pixel art, where individual pixels are important. We propose the Pixel VQ-VAE, a specialized VQ-VAE model that learns representations of pixel art. We show that it outperforms other models in both the quality of embeddings as well as performance on downstream tasks.