A Survey of Constrained Combinatorial Testing
It addresses the problem of applying combinatorial testing effectively in constrained real-world programs for researchers and practitioners.
This paper surveys constraint handling techniques in combinatorial testing, covering 129 papers from 1987 to 2018 to categorize methods and identify understudied areas like constraint identification and maintenance.
Combinatorial Testing (CT) is a potentially powerful testing technique, whereas its failure revealing ability might be dramatically reduced if it fails to handle constraints in an adequate and efficient manner. To ensure the wider applicability of CT in the presence of constrained problem domains, large and diverse efforts have been invested towards the techniques and applications of constrained combinatorial testing. In this paper, we provide a comprehensive survey of representations, influences, and techniques that pertain to constraints in CT, covering 129 papers published between 1987 and 2018. This survey not only categorises the various constraint handling techniques, but also reviews comparatively less well-studied, yet potentially important, constraint identification and maintenance techniques. Since real-world programs are usually constrained, this survey can be of interest to researchers and practitioners who are looking to use and study constrained combinatorial testing techniques.