The Power of Backdoor Absorption in Community Training
For model owners delegating training to external compute providers, this provides a provably sound and computationally efficient defense against backdoor attacks, reducing auditing overhead from prohibitive to minimal.
The paper introduces a defense against backdoor attacks in decentralized training that combines natural absorption, a randomized scheduler, and a lazy verification oracle. It proves that the attack success probability asymptotically collapses to zero and empirically shows backdoor suppression with zero utility degradation using only 10% verification steps.
Backdoor attacks severely threaten large-scale AI models. When model owners delegate training to external compute providers within a decentralized training paradigm, adversaries can craft stealthy, low-frequency triggers to inject malicious behavior while evading standard audits. Traditionally, detecting these attacks requires a full re-computation of the training steps--a prohibitive overhead that directly contradicts the owner's resource constraints. To address this, we investigate the resilience of continuous optimization dynamics under Byzantine perturbations, where adversaries are forced to compete against a continuous influx of honest updates. Under a threat model where an adversary compromises f out of n total trainers, we quantify the minimum auditing overhead required by the model owner to probabilistically bound the attack success rate. We formalize this injection-absorption dynamic as a Discrete-Time Markov Chain (DTMC). Using this framework, we prove that the success probability of any bounded adversary asymptotically collapses to zero under a defense strategy combining natural absorption, a randomized scheduler, and lazy verification oracle. Empirical results demonstrate significant backdoor suppression with zero utility degradation even when invoking the verification oracle on merely 10% of the total training steps. This approach yields a provably sound and computationally efficient defense for safety-critical AI.