CAD-Assistant: Tool-Augmented VLLMs as Generic CAD Task Solvers
This addresses the need for general-purpose AI agents in CAD workflows, though it appears incremental as an application of existing VLLM and tool-augmentation methods to a specific domain.
The paper tackles the problem of AI-assisted computer-aided design (CAD) by proposing CAD-Assistant, a tool-augmented vision and large language model that generates and executes CAD commands iteratively, outperforming baselines on multiple benchmarks.
We propose CAD-Assistant, a general-purpose CAD agent for AI-assisted design. Our approach is based on a powerful Vision and Large Language Model (VLLM) as a planner and a tool-augmentation paradigm using CAD-specific tools. CAD-Assistant addresses multimodal user queries by generating actions that are iteratively executed on a Python interpreter equipped with the FreeCAD software, accessed via its Python API. Our framework is able to assess the impact of generated CAD commands on geometry and adapts subsequent actions based on the evolving state of the CAD design. We consider a wide range of CAD-specific tools including a sketch image parameterizer, rendering modules, a 2D cross-section generator, and other specialized routines. CAD-Assistant is evaluated on multiple CAD benchmarks, where it outperforms VLLM baselines and supervised task-specific methods. Beyond existing benchmarks, we qualitatively demonstrate the potential of tool-augmented VLLMs as general-purpose CAD solvers across diverse workflows.