Analysis of parametric models - linear methods and approximations
This work provides a foundational theoretical unification for parametric models, benefiting researchers in machine learning and applied mathematics by connecting linear methods, kernel methods, and tensor representations.
This paper establishes a unified theoretical framework showing that parametric models in vector spaces have an associated linear map, which leads to reproducing kernel Hilbert spaces and tensor product representations. It demonstrates that all factorisations of a certain class are unitarily equivalent and that these constructions enable efficient low-rank approximations for high-dimensional problems.
Parametric models in vector spaces are shown to possess an associated linear map. This linear operator leads directly to reproducing kernel Hilbert spaces and affine- / linear- representations in terms of tensor products. From the associated linear map analogues of covariance or rather correlation operators can be formed. The associated linear map in fact provides a factorisation of the correlation. Its spectral decomposition, and the associated Karhunen-Loève- or proper orthogonal decomposition in a tensor product follow directly. It is shown that all factorisations of a certain class are unitarily equivalent, as well as that every factorisation induces a different representation, and vice versa. A completely equivalent spectral and factorisation analysis can be carried out in kernel space. The relevance of these abstract constructions is shown on a number of mostly familiar examples, thus unifying many such constructions under one theoretical umbrella. From the factorisation one obtains tensor representations, which may be cascaded, leading to tensors of higher degree. When carried over to a discretised level in the form of a model order reduction, such factorisations allow very sparse low-rank approximations which lead to very efficient computations especially in high dimensions.