Zenkai -- Framework For Exploring Beyond Backpropagation
This addresses the problem for researchers in deep learning who need more flexibility to experiment with novel training methods beyond backpropagation, though it is incremental as it builds on existing framework concepts.
The authors tackled the challenge of exploring deep learning methods beyond backpropagation by developing Zenkai, an open-source framework that divides neural networks into semi-autonomous layers with custom learning algorithms, enabling easier implementation of non-differentiable or alternative approaches.
Zenkai is an open-source framework designed to give researchers more control and flexibility over building and training deep learning machines. It does this by dividing the deep learning machine into layers of semi-autonomous learning machines with their own target and learning algorithm. This is to allow researchers greater exploration such as the use of non-differentiable layers or learning algorithms beyond those based on error backpropagation. Backpropagation Rumelhart et al. [1986] has powered deep learning to become one of the most exciting fields of the 21st century. As a result, a large number of software tools have been developed to support efficient implementation and training of neural networks through the use of backpropa- gation. While these have been critical to the success of deep learning, building frameworks around backpropagation can make it challenging to implement solutions that do not adhere to it. Zenkai aims to make it easier to get around these limitations and help researchers more easily explore new frontiers in deep learning that do not strictly adhere to the backpropagation framework.