Path classification by stochastic linear recurrent neural networks
This work addresses the challenge of interpreting and optimizing stochastic RNNs for path classification, which is incremental as it builds on existing statistical learning theory to analyze biological network models.
The paper tackled the problem of understanding how biological neural networks classify continuous-time stochastic paths by modeling them as stochastic RNNs with identity activation, showing that the empirical risk minimizer achieves a generalization error bound with high probability and that these RNNs are easy to train and robust, backed by numerical experiments on synthetic and real data.
We investigate the functioning of a classifying biological neural network from the perspective of statistical learning theory, modelled, in a simplified setting, as a continuous-time stochastic recurrent neural network (RNN) with identity activation function. In the purely stochastic (robust) regime, we give a generalisation error bound that holds with high probability, thus showing that the empirical risk minimiser is the best-in-class hypothesis. We show that RNNs retain a partial signature of the paths they are fed as the unique information exploited for training and classification tasks. We argue that these RNNs are easy to train and robust and back these observations with numerical experiments on both synthetic and real data. We also exhibit a trade-off phenomenon between accuracy and robustness.