Part-X: A Family of Stochastic Algorithms for Search-Based Test Generation with Probabilistic Guarantees
This addresses a major limitation preventing falsification methods from being used in certification procedures for Cyber-Physical Systems, though it is incremental as it builds on existing falsification approaches.
The paper tackles the lack of finite-time guarantees in search-based testing for Cyber-Physical Systems by developing a stochastic algorithm that estimates the probability of erroneous behaviors and identifies their regions, demonstrating applicability on benchmark functions and an F16 problem.
Requirements driven search-based testing (also known as falsification) has proven to be a practical and effective method for discovering erroneous behaviors in Cyber-Physical Systems. Despite the constant improvements on the performance and applicability of falsification methods, they all share a common characteristic. Namely, they are best-effort methods which do not provide any guarantees on the absence of erroneous behaviors (falsifiers) when the testing budget is exhausted. The absence of finite time guarantees is a major limitation which prevents falsification methods from being utilized in certification procedures. In this paper, we address the finite-time guarantees problem by developing a new stochastic algorithm. Our proposed algorithm not only estimates (bounds) the probability that falsifying behaviors exist, but also it identifies the regions where these falsifying behaviors may occur. We demonstrate the applicability of our approach on standard benchmark functions from the optimization literature and on the F16 benchmark problem.