LEMMA: Bootstrapping High-Level Mathematical Reasoning with Learned Symbolic Abstractions
This addresses the challenge of mathematical reasoning for AI agents, offering an incremental improvement over existing methods.
The paper tackles the problem of enabling reinforcement learning agents to perform high-level mathematical reasoning by learning symbolic abstractions, resulting in improved problem-solving and generalization to harder tasks in equation solving and fraction simplification.
Humans tame the complexity of mathematical reasoning by developing hierarchies of abstractions. With proper abstractions, solutions to hard problems can be expressed concisely, thus making them more likely to be found. In this paper, we propose Learning Mathematical Abstractions (LEMMA): an algorithm that implements this idea for reinforcement learning agents in mathematical domains. LEMMA augments Expert Iteration with an abstraction step, where solutions found so far are revisited and rewritten in terms of new higher-level actions, which then become available to solve new problems. We evaluate LEMMA on two mathematical reasoning tasks--equation solving and fraction simplification--in a step-by-step fashion. In these two domains, LEMMA improves the ability of an existing agent, both solving more problems and generalizing more effectively to harder problems than those seen during training.