Revisiting the Sample Complexity of Sparse Spectrum Approximation of Gaussian Processes
This work addresses scalability issues in Gaussian processes for machine learning practitioners, though it appears incremental as it builds on existing sparse spectrum methods.
The authors tackled the problem of scaling Gaussian processes by improving the sample complexity analysis for sparse spectrum approximations, showing that under certain data disentangling conditions, these approximations can closely match full Gaussian processes with low sample complexity. They validated their method on benchmarks with promising results.
We introduce a new scalable approximation for Gaussian processes with provable guarantees which hold simultaneously over its entire parameter space. Our approximation is obtained from an improved sample complexity analysis for sparse spectrum Gaussian processes (SSGPs). In particular, our analysis shows that under a certain data disentangling condition, an SSGP's prediction and model evidence (for training) can well-approximate those of a full GP with low sample complexity. We also develop a new auto-encoding algorithm that finds a latent space to disentangle latent input coordinates into well-separated clusters, which is amenable to our sample complexity analysis. We validate our proposed method on several benchmarks with promising results supporting our theoretical analysis.