Similarity of Classification Tasks
This work addresses evaluation bias in meta-learning for few-shot learning, offering a method to improve task selection, though it is incremental as it builds on existing meta-learning frameworks.
The paper tackles the problem of bias in meta-learning evaluation by proposing a generative approach to analyze task similarity, showing that higher similarity leads to better performance on few-shot classification benchmarks.
Recent advances in meta-learning has led to remarkable performances on several few-shot learning benchmarks. However, such success often ignores the similarity between training and testing tasks, resulting in a potential bias evaluation. We, therefore, propose a generative approach based on a variant of Latent Dirichlet Allocation to analyse task similarity to optimise and better understand the performance of meta-learning. We demonstrate that the proposed method can provide an insightful evaluation for meta-learning algorithms on two few-shot classification benchmarks that matches common intuition: the more similar the higher performance. Based on this similarity measure, we propose a task-selection strategy for meta-learning and show that it can produce more accurate classification results than methods that randomly select training tasks.