Limits of Transformer Language Models on Learning to Compose Algorithms
This highlights a key limitation in current AI models for tasks requiring algorithmic composition, which is incremental as it builds on existing concerns about model capabilities.
The paper investigates the ability of Transformer language models like LLaMA, GPT-4, and Gemini to learn compositional discrete tasks, finding that they are highly sample inefficient, with LLaMA requiring more data than relearning sub-tasks from scratch and in-context prompting often failing.
We analyze the capabilities of Transformer language models in learning compositional discrete tasks. To this end, we evaluate training LLaMA models and prompting GPT-4 and Gemini on four tasks demanding to learn a composition of several discrete sub-tasks. In particular, we measure how well these models can reuse primitives observable in the sub-tasks to learn the composition task. Our results indicate that compositional learning in state-of-the-art Transformer language models is highly sample inefficient: LLaMA requires more data samples than relearning all sub-tasks from scratch to learn the compositional task; in-context prompting with few samples is unreliable and fails at executing the sub-tasks or correcting the errors in multi-round code generation. Further, by leveraging complexity theory, we support these findings with a theoretical analysis focused on the sample inefficiency of gradient descent in memorizing feedforward models. We open source our code at https://github.com/IBM/limitations-lm-algorithmic-compositional-learning.