SymbolicData:SDEval - Benchmarking for Everyone
It addresses the difficulty of reproducing benchmarks in computer algebra research by providing an automatable tool, though it is incremental as it builds on existing Symbolic Data infrastructure.
The paper introduces SDEval, a software project for creating and running benchmarks in computer algebra, built on the Symbolic Data project to translate problems into executable code for various systems, enabling flexible and extensible use across communities.
In this paper we will present SDeval, a software project that contains tools for creating and running benchmarks with a focus on problems in computer algebra. It is built on top of the Symbolic Data project, able to translate problems in the database into executable code for various computer algebra systems. The included tools are designed to be very flexible to use and to extend, such that they can be utilized even in contexts of other communities. With the presentation of SDEval, we will also address particularities of benchmarking in the field of computer algebra. Furthermore, with SDEval, we provide a feasible and automatizable way of reproducing benchmarks published in current research works, which appears to be a difficult task in general due to the customizability of the available programs. We will simultaneously present the current developments in the Symbolic Data project.