The Devil is in the Decoder: Classification, Regression and GANs
This work addresses a gap in machine vision for tasks like semantic segmentation and depth prediction, though it is incremental in nature.
The paper tackled the understudied role of decoders in pixel-wise vision tasks, finding significant performance variance across different decoder types and introducing new decoder architectures and connections.
Many machine vision applications, such as semantic segmentation and depth prediction, require predictions for every pixel of the input image. Models for such problems usually consist of encoders which decrease spatial resolution while learning a high-dimensional representation, followed by decoders who recover the original input resolution and result in low-dimensional predictions. While encoders have been studied rigorously, relatively few studies address the decoder side. This paper presents an extensive comparison of a variety of decoders for a variety of pixel-wise tasks ranging from classification, regression to synthesis. Our contributions are: (1) Decoders matter: we observe significant variance in results between different types of decoders on various problems. (2) We introduce new residual-like connections for decoders. (3) We introduce a novel decoder: bilinear additive upsampling. (4) We explore prediction artifacts.