Learning Surrogates via Deep Embedding
This addresses the challenge of optimizing models for practical tasks like scene-text recognition and detection where standard metrics are non-differentiable, though it is incremental as it builds on existing surrogate loss methods.
The paper tackles the problem of training neural networks with non-differentiable evaluation metrics by learning a surrogate loss via deep embedding, achieving up to 39% relative improvement in edit distance for scene-text recognition and 4.25% relative improvement in F1 score for detection.
This paper proposes a technique for training a neural network by minimizing a surrogate loss that approximates the target evaluation metric, which may be non-differentiable. The surrogate is learned via a deep embedding where the Euclidean distance between the prediction and the ground truth corresponds to the value of the evaluation metric. The effectiveness of the proposed technique is demonstrated in a post-tuning setup, where a trained model is tuned using the learned surrogate. Without a significant computational overhead and any bells and whistles, improvements are demonstrated on challenging and practical tasks of scene-text recognition and detection. In the recognition task, the model is tuned using a surrogate approximating the edit distance metric and achieves up to $39\%$ relative improvement in the total edit distance. In the detection task, the surrogate approximates the intersection over union metric for rotated bounding boxes and yields up to $4.25\%$ relative improvement in the $F_{1}$ score.