Scalable Bayesian Meta-Learning through Generalized Implicit Gradients
This work addresses scalability issues in meta-learning for researchers and practitioners, offering a more efficient and flexible approach, though it is incremental as it builds on existing implicit differentiation methods.
The paper tackles the scalability bottleneck in Bayesian meta-learning by introducing a novel implicit Bayesian meta-learning (iBaML) method, which extends implicit differentiation to probabilistic settings, broadens learnable priors, quantifies uncertainty, and achieves controlled complexity with analytical error bounds and empirical validation.
Meta-learning owns unique effectiveness and swiftness in tackling emerging tasks with limited data. Its broad applicability is revealed by viewing it as a bi-level optimization problem. The resultant algorithmic viewpoint however, faces scalability issues when the inner-level optimization relies on gradient-based iterations. Implicit differentiation has been considered to alleviate this challenge, but it is restricted to an isotropic Gaussian prior, and only favors deterministic meta-learning approaches. This work markedly mitigates the scalability bottleneck by cross-fertilizing the benefits of implicit differentiation to probabilistic Bayesian meta-learning. The novel implicit Bayesian meta-learning (iBaML) method not only broadens the scope of learnable priors, but also quantifies the associated uncertainty. Furthermore, the ultimate complexity is well controlled regardless of the inner-level optimization trajectory. Analytical error bounds are established to demonstrate the precision and efficiency of the generalized implicit gradient over the explicit one. Extensive numerical tests are also carried out to empirically validate the performance of the proposed method.