Folding over Neural Networks
This work addresses a programming methodology issue for developers and researchers in machine learning, offering an incremental improvement in how neural networks are implemented and manipulated.
The paper tackles the problem of representing and traversing neural networks by proposing a structured recursion approach using recursive data types and recursion schemes in Haskell, resulting in a coherent implementation that separates network structure from semantics and enables compositionality in construction and training.
Neural networks are typically represented as data structures that are traversed either through iteration or by manual chaining of method calls. However, a deeper analysis reveals that structured recursion can be used instead, so that traversal is directed by the structure of the network itself. This paper shows how such an approach can be realised in Haskell, by encoding neural networks as recursive data types, and then their training as recursion scheme patterns. In turn, we promote a coherent implementation of neural networks that delineates between their structure and semantics, allowing for compositionality in both how they are built and how they are trained.