MathViz-E: A Case-study in Domain-Specialized Tool-Using Agents
This work addresses the problem of domain-specialized tool-using agents for mathematical pedagogy, presenting a case-study that is incremental in applying existing methods to a new domain.
The paper tackles the challenge of applying LLMs to control specialized domain tools by developing an automated math visualizer and solver system for mathematical pedagogy, which orchestrates solvers and graphing tools to produce accurate visualizations from natural language commands, and includes open-sourced datasets and an auto-evaluator for system evaluation.
There has been significant recent interest in harnessing LLMs to control software systems through multi-step reasoning, planning and tool-usage. While some promising results have been obtained, application to specific domains raises several general issues including the control of specialized domain tools, the lack of existing datasets for training and evaluation, and the non-triviality of automated system evaluation and improvement. In this paper, we present a case-study where we examine these issues in the context of a specific domain. Specifically, we present an automated math visualizer and solver system for mathematical pedagogy. The system orchestrates mathematical solvers and math graphing tools to produce accurate visualizations from simple natural language commands. We describe the creation of specialized data-sets, and also develop an auto-evaluator to easily evaluate the outputs of our system by comparing them to ground-truth expressions. We have open sourced the data-sets and code for the proposed system.