Interpretable Transformations with Encoder-Decoder Networks
This addresses the challenge of interpretability in deep learning for researchers and practitioners, though it is incremental as it builds on existing encoder-decoder frameworks.
The paper tackles the problem of understanding complex transformations in deep feature spaces by proposing a method to construct explicitly disentangled representations of known transformations like pose and appearance, enabling controlled image manipulation and demonstrating advantages on various datasets and tasks.
Deep feature spaces have the capacity to encode complex transformations of their input data. However, understanding the relative feature-space relationship between two transformed encoded images is difficult. For instance, what is the relative feature space relationship between two rotated images? What is decoded when we interpolate in feature space? Ideally, we want to disentangle confounding factors, such as pose, appearance, and illumination, from object identity. Disentangling these is difficult because they interact in very nonlinear ways. We propose a simple method to construct a deep feature space, with explicitly disentangled representations of several known transformations. A person or algorithm can then manipulate the disentangled representation, for example, to re-render an image with explicit control over parameterized degrees of freedom. The feature space is constructed using a transforming encoder-decoder network with a custom feature transform layer, acting on the hidden representations. We demonstrate the advantages of explicit disentangling on a variety of datasets and transformations, and as an aid for traditional tasks, such as classification.