Transparency in Healthcare AI: Testing European Regulatory Provisions against Users' Transparency Needs
It addresses transparency gaps in high-risk healthcare AI for European regulators and stakeholders, but is incremental as it evaluates existing regulatory provisions.
This study tested whether the Instructions for Use (IFU) structure under the European AI Act meets users' transparency needs in healthcare AI, revealing differentiated priorities across stakeholders and a troubled mapping of needs onto the IFU.
Artificial Intelligence (AI) plays an essential role in healthcare and is pervasively incorporated into medical software and equipment. In the European Union, healthcare is a high-risk application domain for AI, and providers must prepare Instructions for Use (IFU) according to the European regulation 2024/1689 (AI Act). To this regulation, the principle of transparency is cardinal and requires the IFU to be clear and relevant to the users. This study tests whether these latter requirements are satisfied by the IFU structure. A survey was administered online via the Qualtrics platform to four types of direct stakeholders, i.e., managers (N = 238), healthcare professionals (N = 115), patients (N = 229), and Information Technology experts (N = 230). The participants rated the relevance of a set of transparency needs and indicated the IFU section addressing them. The results reveal differentiated priorities across stakeholders and a troubled mapping of transparency needs onto the IFU structure. Recommendations to build a locally meaningful IFU are derived.