Deep connections between learning from limited labels & physical parameter estimation -- inspiration for regularization
This work addresses the challenge of single large-scale examples with partial annotation, offering an incremental improvement through explicit regularization derived from physical parameter estimation.
The paper tackles the problem of learning from limited labeled data by drawing inspiration from PDE-constrained optimization to propose a regularization method, showing that it boosts segmentation accuracy in a hyperspectral imaging example.
Recently established equivalences between differential equations and the structure of neural networks enabled some interpretation of training of a neural network as partial-differential-equation (PDE) constrained optimization. We add to the previously established connections, explicit regularization that is particularly beneficial in the case of single large-scale examples with partial annotation. We show that explicit regularization of model parameters in PDE constrained optimization translates to regularization of the network output. Examination of the structure of the corresponding Lagrangian and backpropagation algorithm do not reveal additional computational challenges. A hyperspectral imaging example shows that minimum prior information together with cross-validation for optimal regularization parameters boosts the segmentation accuracy.