A framework for the automation of testing computer vision systems
This addresses the need for reliable testing in industrial applications like manufacturing and automated driving, but it is incremental as it builds on existing libraries and focuses on a specific domain.
The paper tackles the lack of quality assurance for computer vision systems by presenting a framework for automated test generation, using image modification and similarity metrics, with preliminary results applied to defect detection in riblet surfaces.
Vision systems, i.e., systems that allow to detect and track objects in images, have gained substantial importance over the past decades. They are used in quality assurance applications, e.g., for finding surface defects in products during manufacturing, surveillance, but also automated driving, requiring reliable behavior. Interestingly, there is only little work on quality assurance and especially testing of vision systems in general. In this paper, we contribute to the area of testing vision software, and present a framework for the automated generation of tests for systems based on vision and image recognition. The framework makes use of existing libraries allowing to modify original images and to obtain similarities between the original and modified images. We show how such a framework can be used for testing a particular industrial application on identifying defects on riblet surfaces and present preliminary results from the image classification domain.