The Balancing Act of Policies in Developing Machine Learning Explanations
This addresses the problem of opaque ML models for developers and policymakers, but it is incremental as it focuses on policy effects without introducing new methods.
The study investigated how policy design affects the quality of machine learning explanations, finding that policy length influences engagement but purpose does not, and explanation quality remained poor overall.
Machine learning models are often criticized as opaque from a lack of transparency in their decision-making process. This study examines how policy design impacts the quality of explanations in ML models. We conducted a classroom experiment with 124 participants and analyzed the effects of policy length and purpose on developer compliance with policy requirements. Our results indicate that while policy length affects engagement with some requirements, policy purpose has no effect, and explanation quality is generally poor. These findings highlight the challenge of effective policy development and the importance of addressing diverse stakeholder perspectives within explanations.