Bayesian Efficient Multiple Kernel Learning
This work addresses a computational bottleneck for researchers and practitioners in machine learning who need to integrate multiple data sources or similarity measures efficiently, representing an incremental improvement over prior Bayesian methods.
The paper tackles the problem of efficiently combining many kernels in multiple kernel learning, which is infeasible with existing Bayesian approaches due to high time complexity, and proposes a fully conjugate Bayesian formulation with deterministic variational approximation, enabling combination of hundreds or thousands of kernels in less than a minute and outperforming previous results on benchmark datasets.
Multiple kernel learning algorithms are proposed to combine kernels in order to obtain a better similarity measure or to integrate feature representations coming from different data sources. Most of the previous research on such methods is focused on the computational efficiency issue. However, it is still not feasible to combine many kernels using existing Bayesian approaches due to their high time complexity. We propose a fully conjugate Bayesian formulation and derive a deterministic variational approximation, which allows us to combine hundreds or thousands of kernels very efficiently. We briefly explain how the proposed method can be extended for multiclass learning and semi-supervised learning. Experiments with large numbers of kernels on benchmark data sets show that our inference method is quite fast, requiring less than a minute. On one bioinformatics and three image recognition data sets, our method outperforms previously reported results with better generalization performance.