Adding New Tasks to a Single Network with Weight Transformations using Binary Masks
This addresses the need for scalable and adaptive visual recognition systems, offering an incremental improvement over existing masking methods.
The paper tackles the problem of adapting a pre-trained deep network to new tasks without catastrophic forgetting by using binary masks and affine transformations on convolutional weights, achieving state-of-the-art results on the Visual Decathlon Challenge with slightly over 1 bit per parameter per task.
Visual recognition algorithms are required today to exhibit adaptive abilities. Given a deep model trained on a specific, given task, it would be highly desirable to be able to adapt incrementally to new tasks, preserving scalability as the number of new tasks increases, while at the same time avoiding catastrophic forgetting issues. Recent work has shown that masking the internal weights of a given original conv-net through learned binary variables is a promising strategy. We build upon this intuition and take into account more elaborated affine transformations of the convolutional weights that include learned binary masks. We show that with our generalization it is possible to achieve significantly higher levels of adaptation to new tasks, enabling the approach to compete with fine tuning strategies by requiring slightly more than 1 bit per network parameter per additional task. Experiments on two popular benchmarks showcase the power of our approach, that achieves the new state of the art on the Visual Decathlon Challenge.