TAI Scan Tool: A RAG-Based Tool With Minimalistic Input for Trustworthy AI Self-Assessment
This tool addresses the need for automated compliance assessment with AI regulations, such as the AI Act, for developers and organizations, though it is incremental as it builds on existing RAG methods for a specific legal domain.
The paper tackles the problem of trustworthy AI self-assessment by introducing the TAI Scan Tool, a RAG-based tool that uses minimalistic input to predict risk levels and retrieve relevant articles for compliance with the AI Act, with qualitative evaluation showing correct risk predictions and relevant article retrieval across three semantic groups.
This paper introduces the TAI Scan Tool, a RAG-based TAI self-assessment tool with minimalistic input. The current version of the tool supports the legal TAI assessment, with a particular emphasis on facilitating compliance with the AI Act. It involves a two-step approach with a pre-screening and an assessment phase. The assessment output of the system includes insight regarding the risk-level of the AI system according to the AI Act, while at the same time retrieving relevant articles to aid with compliance and notify on their obligations. Our qualitative evaluation using use-case scenarios yields promising results, correctly predicting risk levels while retrieving relevant articles across three distinct semantic groups. Furthermore, interpretation of results shows that the tool's reasoning relies on comparison with the setting of high-risk systems, a behaviour attributed to their deployment requiring careful consideration, and therefore frequently presented within the AI Act.