Collaborative Management of Benchmark Instances and their Attributes
This work addresses a domain-specific problem for SAT solving researchers by providing a tool to improve benchmark management, though it is incremental in nature.
The paper tackles the challenge of managing and retrieving SAT benchmark instances and their attributes by introducing a benchmark instance identifier, which simplifies the interrelation and increases availability, facilitating SAT experiment design and result exchange.
Experimental evaluation is an integral part in the design process of algorithms. Publicly available benchmark instances are widely used to evaluate methods in SAT solving. For the interpretation of results and the design of algorithm portfolios their attributes are crucial. Capturing the interrelation of benchmark instances and their attributes is considerably simplified through our specification of a benchmark instance identifier. Thus, our tool increases the availability of both by providing means to manage and retrieve benchmark instances by their attributes and vice versa. Like this, it facilitates the design and analysis of SAT experiments and the exchange of results.