Experiences with Improving the Transparency of AI Models and Services
This addresses the need for better transparency in high-stakes AI applications, but it is incremental as it builds on existing proposals without introducing new methods.
The paper tackled the problem of insufficient transparency in AI models and services by identifying the needs and challenges of producing and consuming AI documentation through developer interviews and exercises, resulting in recommendations for improving documentation collection and presentation.
AI models and services are used in a growing number of highstakes areas, resulting in a need for increased transparency. Consistent with this, several proposals for higher quality and more consistent documentation of AI data, models, and systems have emerged. Little is known, however, about the needs of those who would produce or consume these new forms of documentation. Through semi-structured developer interviews, and two document creation exercises, we have assembled a clearer picture of these needs and the various challenges faced in creating accurate and useful AI documentation. Based on the observations from this work, supplemented by feedback received during multiple design explorations and stakeholder conversations, we make recommendations for easing the collection and flexible presentation of AI facts to promote transparency.