TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning
This work addresses the challenge of adapting feature embeddings to specific tasks in few-shot and zero-shot learning, which is crucial for applications with limited data, though it appears incremental as it builds on existing meta-learning paradigms.
The paper tackles the problem of learning task-specific feature embeddings in low-shot learning settings, where training data is limited, by proposing TAFE-Net, a meta-learning approach that adapts image representations to new tasks, resulting in state-of-the-art performance on benchmarks, including a 4 to 15 point accuracy improvement on a challenging visual attribute-object composition task.
Learning good feature embeddings for images often requires substantial training data. As a consequence, in settings where training data is limited (e.g., few-shot and zero-shot learning), we are typically forced to use a generic feature embedding across various tasks. Ideally, we want to construct feature embeddings that are tuned for the given task. In this work, we propose Task-Aware Feature Embedding Networks (TAFE-Nets) to learn how to adapt the image representation to a new task in a meta learning fashion. Our network is composed of a meta learner and a prediction network. Based on a task input, the meta learner generates parameters for the feature layers in the prediction network so that the feature embedding can be accurately adjusted for that task. We show that TAFE-Net is highly effective in generalizing to new tasks or concepts and evaluate the TAFE-Net on a range of benchmarks in zero-shot and few-shot learning. Our model matches or exceeds the state-of-the-art on all tasks. In particular, our approach improves the prediction accuracy of unseen attribute-object pairs by 4 to 15 points on the challenging visual attribute-object composition task.