Improved Compositional Generalization by Generating Demonstrations for Meta-Learning
This addresses a critical bottleneck in meta-learning for AI systems handling compositional tasks, though it is incremental as it builds on existing meta-learning methods.
The paper tackled the problem of compositional generalization in grounded language learning by generating relevant support examples for meta-learning, achieving substantially improved performance on a previously unsolved split without compromising other splits.
Meta-learning and few-shot prompting are viable methods to induce certain types of compositional behaviour. However, these methods can be very sensitive to the choice of support examples used. Choosing good supports from the training data for a given test query is already a difficult problem, but in some cases solving this may not even be enough. We consider a grounded language learning problem (gSCAN) where good support examples for certain test splits might not even exist in the training data, or would be infeasible to search for. We design an agent which instead generates possible supports which are relevant to the test query and current state of the world, then uses these supports via meta-learning to solve the test query. We show substantially improved performance on a previously unsolved compositional behaviour split without a loss of performance on other splits. Further experiments show that in this case, searching for relevant demonstrations even with an oracle function is not sufficient to attain good performance when using meta-learning.