Reasoning and Improving on Software Resilience against Unanticipated Exceptions
This work addresses software reliability for developers by providing a method to assess and enhance resilience against unanticipated errors, representing an incremental improvement in testing techniques.
The paper tackles the problem of software resilience against unanticipated exceptions by proposing short-circuit testing to inject exceptions during test execution, verifying formal contracts for resilience properties; evaluation on 9 test suites with 78% line coverage shows that 101 out of 241 catch blocks expose resilience properties and 84 can be transformed for improved resilience.
In software, there are the errors anticipated at specification and design time, those encountered at development and testing time, and those that happen in production mode yet never anticipated. In this paper, we aim at reasoning on the ability of software to correctly handle unanticipated exceptions. We propose an algorithm, called short-circuit testing, which injects exceptions during test suite execution so as to simulate unanticipated errors. This algorithm collects data that is used as input for verifying two formal exception contracts that capture two resilience properties. Our evaluation on 9 test suites, with 78% line coverage in average, analyzes 241 executed catch blocks, shows that 101 of them expose resilience properties and that 84 can be transformed to be more resilient.