Trainable and Dynamic Computing: Error Backpropagation through Physical Media
This work addresses the problem of scalability and speed in neural network training for researchers and engineers in machine learning, offering a novel approach but with incremental advancements in analog computing.
The paper tackles the challenge of implementing neural networks physically by designing linear dynamic systems with non-linear feedback for analog computing, demonstrating that error backpropagation can be performed physically to greatly speed up optimization.
Machine learning algorithms, and more in particular neural networks, arguably experience a revolution in terms of performance. Currently, the best systems we have for speech recognition, computer vision and similar problems are based on neural networks, trained using the half-century old backpropagation algorithm. Despite the fact that neural networks are a form of analog computers, they are still implemented digitally for reasons of convenience and availability. In this paper we demonstrate how we can design physical linear dynamic systems with non-linear feedback as a generic platform for dynamic, neuro-inspired analog computing. We show that a crucial advantage of this setup is that the error backpropagation can be performed physically as well, which greatly speeds up the optimisation process. As we show in this paper, using one experimentally validated and one conceptual example, such systems may be the key to providing a relatively straightforward mechanism for constructing highly scalable, fully dynamic analog computers.