Pareto-Path Multi-Task Multiple Kernel Learning
This work addresses multi-task learning for machine learning practitioners by offering a more principled alternative to heuristic averaging methods, though it appears incremental in nature.
The authors tackled the problem of multi-task multiple kernel learning by proposing a novel SVM framework that optimizes conic combinations of task objectives, producing solutions along the Pareto Front, and demonstrated improved classification performance compared to similar approaches.
A traditional and intuitively appealing Multi-Task Multiple Kernel Learning (MT-MKL) method is to optimize the sum (thus, the average) of objective functions with (partially) shared kernel function, which allows information sharing amongst tasks. We point out that the obtained solution corresponds to a single point on the Pareto Front (PF) of a Multi-Objective Optimization (MOO) problem, which considers the concurrent optimization of all task objectives involved in the Multi-Task Learning (MTL) problem. Motivated by this last observation and arguing that the former approach is heuristic, we propose a novel Support Vector Machine (SVM) MT-MKL framework, that considers an implicitly-defined set of conic combinations of task objectives. We show that solving our framework produces solutions along a path on the aforementioned PF and that it subsumes the optimization of the average of objective functions as a special case. Using algorithms we derived, we demonstrate through a series of experimental results that the framework is capable of achieving better classification performance, when compared to other similar MTL approaches.