A Survey on Methods and Metrics for the Assessment of Explainability under the Proposed AI Act
It addresses the need for standardized explainability metrics to help stakeholders comply with regulatory frameworks, but it is incremental as it builds on existing discussions without introducing new methods.
This study examines how explainability metrics for AI systems align with the proposed EU AI Act, identifying key requirements such as being risk-focused, model-agnostic, goal-aware, intelligible, and accessible to facilitate compliance.
This study discusses the interplay between metrics used to measure the explainability of the AI systems and the proposed EU Artificial Intelligence Act. A standardisation process is ongoing: several entities (e.g. ISO) and scholars are discussing how to design systems that are compliant with the forthcoming Act and explainability metrics play a significant role. This study identifies the requirements that such a metric should possess to ease compliance with the AI Act. It does so according to an interdisciplinary approach, i.e. by departing from the philosophical concept of explainability and discussing some metrics proposed by scholars and standardisation entities through the lenses of the explainability obligations set by the proposed AI Act. Our analysis proposes that metrics to measure the kind of explainability endorsed by the proposed AI Act shall be risk-focused, model-agnostic, goal-aware, intelligible & accessible. This is why we discuss the extent to which these requirements are met by the metrics currently under discussion.