Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)
This addresses the need for regulatory compliance and stakeholder rights in AI transparency, but is incremental as it builds on existing legal requirements.
The paper tackles the problem of assessing the quality of public summaries of training content for GPAI models as required by the AI Act, developing a framework to evaluate transparency and usefulness, and applying it to 5 existing summaries.
The AI Act's Article 53(1)(d) requires providers of general-purpose AI (GPAI) models to publish a sufficiently detailed public summary about the content used for training based on a template provided by the AI Office. The stated goal of this obligation is to increase transparency regarding the data used for training GPAI models, and to enable relevant stakeholders to exercise their rights, especially regarding IP, copyright, and data protection. This paper provides a quality assessment framework to assess the public summary across two key dimensions: \textit{transparency} regarding information being provided in a clear, comprehensive, and sufficiently detailed manner; and \textit{usefulness} regarding whether the provision of the document and the contents can be effectively utilised by stakeholders to carry out rights related actions. This framework enables identification of key issues in public summaries, and provides a structured and research-based method to compare practices across public summaries and providers. It also enables authorities such as the AI Office to identify potential issues that could emerge and provides actionable recommendations and guidelines for providers to develop public summaries with high quality. The paper provides an assessment of 5 public summaries published as of 12th January 2026 which were found through an exhaustive search process. To disseminate these findings as a public resource, the paper also describes the development of a website where the assessments, outcomes, and methodologies will be shared.