Cloud-based XAI Services for Assessing Open Repository Models Under Adversarial Attacks
This work addresses the problem of automated assessment for AI model quality under adversarial threats, primarily for researchers and practitioners using open repository models, though it appears incremental as it builds on existing XAI tools with a service-oriented wrapper.
The authors tackled the challenge of assessing open-source AI models under adversarial attacks by proposing a cloud-based service framework that automates XAI-based evaluations, demonstrating its application across computer vision and tabular cases to analyze over a hundred combination scenarios for five quality attributes.
The opacity of AI models necessitates both validation and evaluation before their integration into services. To investigate these models, explainable AI (XAI) employs methods that elucidate the relationship between input features and output predictions. The operations of XAI extend beyond the execution of a single algorithm, involving a series of activities that include preprocessing data, adjusting XAI to align with model parameters, invoking the model to generate predictions, and summarizing the XAI results. Adversarial attacks are well-known threats that aim to mislead AI models. The assessment complexity, especially for XAI, increases when open-source AI models are subject to adversarial attacks, due to various combinations. To automate the numerous entities and tasks involved in XAI-based assessments, we propose a cloud-based service framework that encapsulates computing components as microservices and organizes assessment tasks into pipelines. The current XAI tools are not inherently service-oriented. This framework also integrates open XAI tool libraries as part of the pipeline composition. We demonstrate the application of XAI services for assessing five quality attributes of AI models: (1) computational cost, (2) performance, (3) robustness, (4) explanation deviation, and (5) explanation resilience across computer vision and tabular cases. The service framework generates aggregated analysis that showcases the quality attributes for more than a hundred combination scenarios.