Sparse Function-space Representation of Neural Networks
This addresses uncertainty and data efficiency issues for users of neural networks, though it appears incremental as it builds on existing function-space concepts.
The paper tackles the lack of uncertainty estimates and difficulty in incorporating new data in deep neural networks by converting them to a sparse function-space representation, enabling uncertainty quantification and data incorporation without retraining while maintaining predictive performance, as demonstrated on UCI benchmark tasks.
Deep neural networks (NNs) are known to lack uncertainty estimates and struggle to incorporate new data. We present a method that mitigates these issues by converting NNs from weight space to function space, via a dual parameterization. Importantly, the dual parameterization enables us to formulate a sparse representation that captures information from the entire data set. This offers a compact and principled way of capturing uncertainty and enables us to incorporate new data without retraining whilst retaining predictive performance. We provide proof-of-concept demonstrations with the proposed approach for quantifying uncertainty in supervised learning on UCI benchmark tasks.