Multi-Task Networks With Universe, Group, and Task Feature Learning
This addresses performance improvements in natural language understanding tasks, but it is incremental as it builds on existing multi-task learning methods with task grouping.
The paper tackles multi-task learning by leveraging natural task groupings to improve performance, demonstrating gains on NLU datasets including ATIS, Snips, and an in-house dataset.
We present methods for multi-task learning that take advantage of natural groupings of related tasks. Task groups may be defined along known properties of the tasks, such as task domain or language. Such task groups represent supervised information at the inter-task level and can be encoded into the model. We investigate two variants of neural network architectures that accomplish this, learning different feature spaces at the levels of individual tasks, task groups, as well as the universe of all tasks: (1) parallel architectures encode each input simultaneously into feature spaces at different levels; (2) serial architectures encode each input successively into feature spaces at different levels in the task hierarchy. We demonstrate the methods on natural language understanding (NLU) tasks, where a grouping of tasks into different task domains leads to improved performance on ATIS, Snips, and a large inhouse dataset.