Constrained Bayesian Inference for Low Rank Multitask Learning
This work addresses the challenge of high-dimensional functional neuroimaging with limited data by enabling more flexible constraints, though it appears incremental as it builds on existing variational inference methods.
The authors tackled the problem of constrained Bayesian inference without requiring convexity of the constraint set, applying it to low-rank multitask learning and sparse conditional independence structure recovery, with experimental results showing improved predictive performance and structure recovery over baseline models.
We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the constraint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply the proposed constrained Bayesian inference approach to multitask learning subject to rank constraints on the weight matrix. Further, constrained parameter estimation is applied to recover the sparse conditional independence structure encoded by prior precision matrices. Our approach is motivated by reverse inference for high dimensional functional neuroimaging, a domain where the high dimensionality and small number of examples requires the use of constraints to ensure meaningful and effective models. For this application, we propose a model that jointly learns a weight matrix and the prior inverse covariance structure between different tasks. We present experimental validation showing that the proposed approach outperforms strong baseline models in terms of predictive performance and structure recovery.