Integrating a large-scale testing campaign in the CK framework
This addresses the challenge of bookkeeping for evolving large-scale testing initiatives in programming languages research, but it is incremental as it applies an existing framework to a specific domain.
The paper tackles the problem of managing large-scale experimental campaigns in programming languages research by integrating an OpenCL compiler testing campaign into the Collective Knowledge framework, providing storage and representation for test cases and results with an initial public implementation.
We consider the problem of conducting large experimental campaigns in programming languages research. Most research efforts require a certain level of bookkeeping of results. This is manageable via quick, on-the-fly infrastructure implementations. However, it becomes a problem for large-scale testing initiatives, especially as the needs of the project evolve along the way. We look at how the Collective Knowledge generalized testing framework can help with such a project and its overall applicability and ease of use. The project in question is an OpenCL compiler testing campaign. We investigate how to use the Collective Knowledge framework to lead the experimental campaign, by providing storage and representation of test cases and their results. We also provide an initial implementation, publicly available.