Large Language Models and Algorithm Execution: Application to an Arithmetic Function
This addresses a limitation in LLMs for AI researchers, but it is incremental as it builds on existing supervised training approaches.
The paper tackled the problem of LLMs struggling to autonomously execute algorithms by introducing LLM-DAL, a specialized training method for reasoning decomposition, which significantly improved their ability to perform complex algorithmic inferences and generalize.
Large Language Models (LLMs) have recently developed new advanced functionalities. Their effectiveness relies on statistical learning and generalization capabilities. However, they face limitations in internalizing the data they process and struggle, for instance, to autonomously execute algorithms. In this paper, we investigate the possibility of extending these models' capabilities to algorithm execution through specialized supervised training focused on reasoning decomposition. We introduce a training model called LLM-DAL (Large Language Model - Decompositional Algorithmic Learning), through which we demonstrate that LLMs' ability to perform complex algorithmic inferences and generalize can be significantly improved when the training method is properly designed to guide the model in its learning process.