Deep Radial Kernel Networks: Approximating Radially Symmetric Functions with Deep Networks
This work addresses the challenge of designing efficient deep architectures for function approximation, but it appears incremental as it builds on existing kernel methods without broad application impact.
The paper tackles the problem of approximating radially symmetric functions more efficiently than known shallow networks, and demonstrates that their deep radial kernel network can approximate Gaussian kernel SVMs and improve upon them with training, achieving unspecified gains.
We prove that a particular deep network architecture is more efficient at approximating radially symmetric functions than the best known 2 or 3 layer networks. We use this architecture to approximate Gaussian kernel SVMs, and subsequently improve upon them with further training. The architecture and initial weights of the Deep Radial Kernel Network are completely specified by the SVM and therefore sidesteps the problem of empirically choosing an appropriate deep network architecture.