Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems
This work addresses governance challenges for high-stakes, safety-critical systems, but it is incremental as it applies established risk assessment concepts from other domains to computer science.
The paper tackles the problem of balancing accuracy and efficiency in distributed machine learning systems, such as autonomous vehicles, by analyzing policy implications and calling for accountability mechanisms to address gaps in existing risk assessment standards.
Trade-offs between accuracy and efficiency pervade law, public health, and other non-computing domains, which have developed policies to guide how to balance the two in conditions of uncertainty. While computer science also commonly studies accuracy-efficiency trade-offs, their policy implications remain poorly examined. Drawing on risk assessment practices in the US, we argue that, since examining these trade-offs has been useful for guiding governance in other domains, we need to similarly reckon with these trade-offs in governing computer systems. We focus our analysis on distributed machine learning systems. Understanding the policy implications in this area is particularly urgent because such systems, which include autonomous vehicles, tend to be high-stakes and safety-critical. We 1) describe how the trade-off takes shape for these systems, 2) highlight gaps between existing US risk assessment standards and what these systems require to be properly assessed, and 3) make specific calls to action to facilitate accountability when hypothetical risks concerning the accuracy-efficiency trade-off become realized as accidents in the real world. We close by discussing how such accountability mechanisms encourage more just, transparent governance aligned with public values.