Testing Framework for Black-box AI Models
This addresses the problem of AI model reliability for practitioners and industries deploying AI in critical applications, though it appears incremental as it builds on existing testing concepts.
The paper tackles the challenge of ensuring reliability in AI models used for important decision-making by presenting an end-to-end generic framework for automated test generation across modalities like text, tabular, and time-series data and properties such as accuracy, fairness, and robustness, with the tool being effective in uncovering issues in industrial AI models.
With widespread adoption of AI models for important decision making, ensuring reliability of such models remains an important challenge. In this paper, we present an end-to-end generic framework for testing AI Models which performs automated test generation for different modalities such as text, tabular, and time-series data and across various properties such as accuracy, fairness, and robustness. Our tool has been used for testing industrial AI models and was very effective to uncover issues present in those models. Demo video link: https://youtu.be/984UCU17YZI