Tree DNN: A Deep Container Network
This addresses a practical limitation in multi-task learning for user products where tasks have separate datasets, offering an incremental improvement in efficiency.
The paper tackles the problem of training multi-task learning models when each task requires a different dataset, proposing TreeDNN to train with multiple datasets simultaneously. It shows competitive performance with advantages like reduced ROM requirements and increased system responsiveness by loading only specific branches at inference.
Multi-Task Learning (MTL) has shown its importance at user products for fast training, data efficiency, reduced overfitting etc. MTL achieves it by sharing the network parameters and training a network for multiple tasks simultaneously. However, MTL does not provide the solution, if each task needs training from a different dataset. In order to solve the stated problem, we have proposed an architecture named TreeDNN along with it's training methodology. TreeDNN helps in training the model with multiple datasets simultaneously, where each branch of the tree may need a different training dataset. We have shown in the results that TreeDNN provides competitive performance with the advantage of reduced ROM requirement for parameter storage and increased responsiveness of the system by loading only specific branch at inference time.