Learning Purified Feature Representations from Task-irrelevant Labels
This addresses the challenge of generalization in deep learning for small-scale datasets, though it appears incremental as it builds on existing methods without a major paradigm shift.
The paper tackles the problem of training deep neural networks with limited data by proposing PurifiedLearning, a framework that uses task-irrelevant labels to purify feature representations, resulting in improved classification performance as demonstrated through extensive experiments.
Learning an empirically effective model with generalization using limited data is a challenging task for deep neural networks. In this paper, we propose a novel learning framework called PurifiedLearning to exploit task-irrelevant features extracted from task-irrelevant labels when training models on small-scale datasets. Particularly, we purify feature representations by using the expression of task-irrelevant information, thus facilitating the learning process of classification. Our work is built on solid theoretical analysis and extensive experiments, which demonstrate the effectiveness of PurifiedLearning. According to the theory we proved, PurifiedLearning is model-agnostic and doesn't have any restrictions on the model needed, so it can be combined with any existing deep neural networks with ease to achieve better performance. The source code of this paper will be available in the future for reproducibility.