Differentiable Spline Approximations
This addresses a bottleneck in differentiable programming for machine learning practitioners, enabling broader use of spline models, though it is incremental as it builds on existing optimization frameworks.
The paper tackled the problem of extending gradient-based optimization to spline-based functions, which are often non-differentiable, by deriving a weak Jacobian with block-sparse structure, resulting in improved performance in applications like image segmentation and 3D reconstruction.
The paradigm of differentiable programming has significantly enhanced the scope of machine learning via the judicious use of gradient-based optimization. However, standard differentiable programming methods (such as autodiff) typically require that the machine learning models be differentiable, limiting their applicability. Our goal in this paper is to use a new, principled approach to extend gradient-based optimization to functions well modeled by splines, which encompass a large family of piecewise polynomial models. We derive the form of the (weak) Jacobian of such functions and show that it exhibits a block-sparse structure that can be computed implicitly and efficiently. Overall, we show that leveraging this redesigned Jacobian in the form of a differentiable "layer" in predictive models leads to improved performance in diverse applications such as image segmentation, 3D point cloud reconstruction, and finite element analysis.