On the Importance of Domain-specific Explanations in AI-based Cybersecurity Systems (Technical Report)
This work tackles the problem of opaque AI decisions in cybersecurity, where understanding is crucial for trust and safety, though it appears incremental as it builds on existing XAI concepts.
The paper addresses the lack of explainability in AI-based cybersecurity systems by proposing desiderata for explanations, analyzing existing XAI approaches, and introducing a novel architectural roadmap to guide research in this critical domain.
With the availability of large datasets and ever-increasing computing power, there has been a growing use of data-driven artificial intelligence systems, which have shown their potential for successful application in diverse areas. However, many of these systems are not able to provide information about the rationale behind their decisions to their users. Lack of understanding of such decisions can be a major drawback, especially in critical domains such as those related to cybersecurity. In light of this problem, in this paper we make three contributions: (i) proposal and discussion of desiderata for the explanation of outputs generated by AI-based cybersecurity systems; (ii) a comparative analysis of approaches in the literature on Explainable Artificial Intelligence (XAI) under the lens of both our desiderata and further dimensions that are typically used for examining XAI approaches; and (iii) a general architecture that can serve as a roadmap for guiding research efforts towards the development of explainable AI-based cybersecurity systems -- at its core, this roadmap proposes combinations of several research lines in a novel way towards tackling the unique challenges that arise in this context.