Automated Search for Resource-Efficient Branched Multi-Task Networks
This addresses the challenge of resource-efficient multi-task learning in vision for researchers and practitioners, though it is incremental as it builds on existing neural architecture search methods.
The paper tackled the problem of manually designing neural network architectures for multi-task vision problems by proposing an automated approach using differentiable neural architecture search to define branching structures, and it achieved consistent high performance within limited resource budgets across various dense prediction tasks.
The multi-modal nature of many vision problems calls for neural network architectures that can perform multiple tasks concurrently. Typically, such architectures have been handcrafted in the literature. However, given the size and complexity of the problem, this manual architecture exploration likely exceeds human design abilities. In this paper, we propose a principled approach, rooted in differentiable neural architecture search, to automatically define branching (tree-like) structures in the encoding stage of a multi-task neural network. To allow flexibility within resource-constrained environments, we introduce a proxyless, resource-aware loss that dynamically controls the model size. Evaluations across a variety of dense prediction tasks show that our approach consistently finds high-performing branching structures within limited resource budgets.