Safe Screening for Multi-Task Feature Learning with Multiple Data Matrices
This work addresses computational efficiency for researchers and practitioners using MTFL in high-dimensional settings, though it is incremental as it builds on existing screening methods by extending them to multiple data matrices.
The paper tackles the challenge of high computational cost in multi-task feature learning (MTFL) with large feature dimensions by proposing a safe screening rule based on dual projection onto convex sets (DPC) to identify inactive features with zero coefficients across all tasks, resulting in speedups of up to several orders of magnitude without sacrificing accuracy.
Multi-task feature learning (MTFL) is a powerful technique in boosting the predictive performance by learning multiple related classification/regression/clustering tasks simultaneously. However, solving the MTFL problem remains challenging when the feature dimension is extremely large. In this paper, we propose a novel screening rule---that is based on the dual projection onto convex sets (DPC)---to quickly identify the inactive features---that have zero coefficients in the solution vectors across all tasks. One of the appealing features of DPC is that: it is safe in the sense that the detected inactive features are guaranteed to have zero coefficients in the solution vectors across all tasks. Thus, by removing the inactive features from the training phase, we may have substantial savings in the computational cost and memory usage without sacrificing accuracy. To the best of our knowledge, it is the first screening rule that is applicable to sparse models with multiple data matrices. A key challenge in deriving DPC is to solve a nonconvex problem. We show that we can solve for the global optimum efficiently via a properly chosen parametrization of the constraint set. Moreover, DPC has very low computational cost and can be integrated with any existing solvers. We have evaluated the proposed DPC rule on both synthetic and real data sets. The experiments indicate that DPC is very effective in identifying the inactive features---especially for high dimensional data---which leads to a speedup up to several orders of magnitude.