Pseudo-task Augmentation: From Deep Multitask Learning to Intratask Sharing---and Back
This provides a broadly applicable method to boost deep learning performance, even when only a single task is available, though it is incremental as it builds on existing multitask learning ideas.
The paper tackles the problem of improving single-task learning by adapting multitask learning concepts, introducing pseudo-task augmentation to simulate related tasks, and achieves state-of-the-art performance on the CelebA dataset with complementary gains when combined with multitask learning.
Deep multitask learning boosts performance by sharing learned structure across related tasks. This paper adapts ideas from deep multitask learning to the setting where only a single task is available. The method is formalized as pseudo-task augmentation, in which models are trained with multiple decoders for each task. Pseudo-tasks simulate the effect of training towards closely-related tasks drawn from the same universe. In a suite of experiments, pseudo-task augmentation is shown to improve performance on single-task learning problems. When combined with multitask learning, further improvements are achieved, including state-of-the-art performance on the CelebA dataset, showing that pseudo-task augmentation and multitask learning have complementary value. All in all, pseudo-task augmentation is a broadly applicable and efficient way to boost performance in deep learning systems.