Nonlinear Functional Output Regression: a Dictionary Approach
This work addresses functional-output regression for data analysis, offering a flexible method that can handle non-orthogonal dictionaries and partial observations, but it is incremental as it builds on existing dictionary and kernel-based techniques.
The authors tackled functional-output regression by introducing projection learning (PL), a dictionary-based method that predicts functions expanded on a dictionary while minimizing a functional loss, and showed it is competitive with other nonlinear approaches on real datasets.
To address functional-output regression, we introduce projection learning (PL), a novel dictionary-based approach that learns to predict a function that is expanded on a dictionary while minimizing an empirical risk based on a functional loss. PL makes it possible to use non orthogonal dictionaries and can then be combined with dictionary learning; it is thus much more flexible than expansion-based approaches relying on vectorial losses. This general method is instantiated with reproducing kernel Hilbert spaces of vector-valued functions as kernel-based projection learning (KPL). For the functional square loss, two closed-form estimators are proposed, one for fully observed output functions and the other for partially observed ones. Both are backed theoretically by an excess risk analysis. Then, in the more general setting of integral losses based on differentiable ground losses, KPL is implemented using first-order optimization for both fully and partially observed output functions. Eventually, several robustness aspects of the proposed algorithms are highlighted on a toy dataset; and a study on two real datasets shows that they are competitive compared to other nonlinear approaches. Notably, using the square loss and a learnt dictionary, KPL enjoys a particularily attractive trade-off between computational cost and performances.