Analog Physical Systems Can Exhibit Double Descent
This work addresses the challenge of energy-efficient AI by showing that analog systems can mimic key digital AI behaviors, with potential implications for hardware design and biological systems.
The study tackled the problem of achieving double descent in analog physical systems, which typically suffer from component non-idealities, and found that a modified training protocol enabled a decentralized analog network to exhibit this behavior, improving performance on unseen data without overfitting.
An important component of the success of large AI models is double descent, in which networks avoid overfitting as they grow relative to the amount of training data, instead improving their performance on unseen data. Here we demonstrate double descent in a decentralized analog network of self-adjusting resistive elements. This system trains itself and performs tasks without a digital processor, offering potential gains in energy efficiency and speed -- but must endure component non-idealities. We find that standard training fails to yield double descent, but a modified protocol that accommodates this inherent imperfection succeeds. Our findings show that analog physical systems, if appropriately trained, can exhibit behaviors underlying the success of digital AI. Further, they suggest that biological systems might similarly benefit from over-parameterization.