AlgoNet: $C^\infty$ Smooth Algorithmic Neural Networks
This addresses the challenge of integrating algorithmic precision into neural networks for tasks requiring high accuracy, though it appears incremental as it builds on existing concepts.
The paper introduces algorithmic neural networks (AlgoNets) to combine neural networks with smooth versions of classic algorithms, aiming to improve accuracy and stability for problems where traditional algorithms excel.
Artificial neural networks revolutionized many areas of computer science in recent years since they provide solutions to a number of previously unsolved problems. On the other hand, for many problems, classic algorithms exist, which typically exceed the accuracy and stability of neural networks. To combine these two concepts, we present a new kind of neural networks$-$algorithmic neural networks (AlgoNets). These networks integrate smooth versions of classic algorithms into the topology of neural networks. A forward AlgoNet includes algorithmic layers into existing architectures while a backward AlgoNet can solve inverse problems without or with only weak supervision. In addition, we present the $\texttt{algonet}$ package, a PyTorch based library that includes, inter alia, a smoothly evaluated programming language, a smooth 3D mesh renderer, and smooth sorting algorithms.