Sample-wise Constrained Learning via a Sequential Penalty Approach with Applications in Image Processing
This addresses the need for formalizing sample-wise constraints in optimization for machine learning, though it appears incremental as it adapts existing penalty methods to deep learning scenarios.
The paper tackled the problem of incorporating strict constraints on individual data samples in learning tasks by proposing a sequential penalty method, showing convergence guarantees and practical viability in image processing experiments.
In many learning tasks, certain requirements on the processing of individual data samples should arguably be formalized as strict constraints in the underlying optimization problem, rather than by means of arbitrary penalties. We show that, in these scenarios, learning can be carried out exploiting a sequential penalty method that allows to properly deal with constraints. The proposed algorithm is shown to possess convergence guarantees under assumptions that are reasonable in deep learning scenarios. Moreover, the results of experiments on image processing tasks show that the method is indeed viable to be used in practice.