Convolutional Networks with MuxOut Layers as Multi-rate Systems for Image Upscaling
This work addresses image upscaling for computer vision applications, presenting an incremental improvement through a novel interpretation and analysis method.
The paper tackles image upscaling by interpreting convolutional networks as adaptive filters combined with MuxOut layers, enabling efficient upscaling and analysis of network filter effects, with results including deterministic detail recovery and a new generation of upscalers that sample realistic upscale aliases.
We interpret convolutional networks as adaptive filters and combine them with so-called MuxOut layers to efficiently upscale low resolution images. We formalize this interpretation by deriving a linear and space-variant structure of a convolutional network when its activations are fixed. We introduce general purpose algorithms to analyze a network and show its overall filter effect for each given location. We use this analysis to evaluate two types of image upscalers: deterministic upscalers that target the recovery of details from original content; and second, a new generation of upscalers that can sample the distribution of upscale aliases (images that share the same downscale version) that look like real content.