Laws of Learning Dynamics and the Core of Learners
This work addresses adversarial robustness in machine learning, presenting a novel theoretical framework with practical improvements, though it is incremental as it builds on existing ensemble and defense methods.
The authors tackled the problem of defending against transfer-based adversarial attacks by formulating fundamental laws of learning dynamics and introducing an entropy-based lifelong ensemble learning method, achieving higher accuracy than a naive ensemble on the CIFAR-10 dataset, especially under strong perturbations.
We formulate the fundamental laws governing learning dynamics, namely the conservation law and the decrease of total entropy. Within this framework, we introduce an entropy-based lifelong ensemble learning method. We evaluate its effectiveness by constructing an immunization mechanism to defend against transfer-based adversarial attacks on the CIFAR-10 dataset. Compared with a naive ensemble formed by simply averaging models specialized on clean and adversarial samples, the resulting logifold achieves higher accuracy in most test cases, with particularly large gains under strong perturbations.