Differentiable Adaptive Computation Time for Visual Reasoning
This work addresses the challenge of improving performance-to-computation ratios in visual reasoning models, offering incremental advancements in efficiency and interpretability for AI researchers.
The paper tackles the problem of adaptive computation in visual reasoning by introducing DACT, a differentiable attention-based algorithm that reduces the number of recurrent steps needed in the MAC architecture while maintaining or surpassing accuracy on the CLEVR dataset.
This paper presents a novel attention-based algorithm for achieving adaptive computation called DACT, which, unlike existing ones, is end-to-end differentiable. Our method can be used in conjunction with many networks; in particular, we study its application to the widely known MAC architecture, obtaining a significant reduction in the number of recurrent steps needed to achieve similar accuracies, therefore improving its performance to computation ratio. Furthermore, we show that by increasing the maximum number of steps used, we surpass the accuracy of even our best non-adaptive MAC in the CLEVR dataset, demonstrating that our approach is able to control the number of steps without significant loss of performance. Additional advantages provided by our approach include considerably improving interpretability by discarding useless steps and providing more insights into the underlying reasoning process. Finally, we present adaptive computation as an equivalent to an ensemble of models, similar to a mixture of expert formulation. Both the code and the configuration files for our experiments are made available to support further research in this area.