Text-to-CAD Evaluation with CADTests
For researchers in Text-to-CAD, this provides a standardized evaluation framework addressing the lack of rigorous assessment in the field.
The paper introduces CADTestBench, the first test-based benchmark for Text-to-CAD evaluation, using executable tests to verify geometric and topological requirements. It shows that CADTests can guide generation, achieving baselines that outperform current methods.
Text-to-CAD has recently emerged as an important task with the potential to substantially accelerate design workflows. Despite its significance, there has been surprisingly little work on Text-to-CAD evaluation, and assessing CAD model generation performance remains a considerable challenge. In this work, we introduce a new evaluation perspective for Text-to-CAD based on automated testing. We propose CADTestBench, the first test-based benchmark for Text-to-CAD, based on CADTests, executable software tests that verify whether a generated CAD model satisfies the geometric and topological requirements of the input prompt. Using CADTestBench, we conduct comprehensive benchmarking of recent Text-to-CAD methods and further demonstrate that CADTests can also guide CAD model generation, yielding simple baselines that surpass performance of current methods. CADTestBench code and data are available at GitHub and Hugging Face dataset.