Distributed and Decentralised Training: Technical Governance Challenges in a Shifting AI Landscape
It addresses technical governance challenges for policymakers in AI, focusing on risks and benefits of decentralized training, but is incremental in clarifying existing concepts.
This paper examines the shift from centralized to distributed and decentralized AI training, highlighting how it could increase risks like compute structuring and capability proliferation while challenging governance assumptions, though some policy tools remain effective.
Advances in low-communication training algorithms are enabling a shift from centralised model training to compute setups that are either distributed across multiple clusters or decentralised via community-driven contributions. This paper distinguishes these two scenarios - distributed and decentralised training - which are little understood and often conflated in policy discourse. We discuss how they could impact technical AI governance through an increased risk of compute structuring, capability proliferation, and the erosion of detectability and shutdownability. While these trends foreshadow a possible new paradigm that could challenge key assumptions of compute governance, we emphasise that certain policy levers, like export controls, remain relevant. We also acknowledge potential benefits of decentralised AI, including privacy-preserving training runs that could unlock access to more data, and mitigating harmful power concentration. Our goal is to support more precise policymaking around compute, capability proliferation, and decentralised AI development.