Polyharmonic Spline Packages: Composition, Efficient Procedures for Computation and Differentiation
This addresses scalability issues in machine learning regression for problems with potentially high-dimensional data but low intrinsic structure.
The paper tackles the computational scalability and high-dimensional limitations of polyharmonic spline regression by proposing a cascade architecture of spline packages, achieving theoretically justified performance for problems with unknown intrinsic low dimensionality while providing efficient matrix procedures for computation and differentiation.
In a previous paper it was shown that a machine learning regression problem can be solved within the framework of random function theory, with the optimal kernel analytically derived from symmetry and indifference principles and coinciding with a polyharmonic spline. However, a direct application of that solution is limited by O(N^3) computational cost and by a breakdown of the original theoretical assumptions when the input space has excessive dimensionality. This paper proposes a cascade architecture built from packages of polyharmonic splines that simultaneously addresses scalability and is theoretically justified for problems with unknown intrinsic low dimensionality. Efficient matrix procedures are presented for forward computation and end-to-end differentiation through the cascade.