On the dice loss gradient and the ways to mimic it
This provides insights for practitioners and researchers in medical imaging and segmentation to better understand and design loss functions, though it is incremental in nature.
The paper analyzes the gradient of the Dice loss in semantic segmentation, showing it is a weighted negative of the ground truth with small dynamic range, and demonstrates that this supervision can be mimicked by a simple element-wise multiplication of network output with the negative ground truth.
In the past few years, in the context of fully-supervised semantic segmentation, several losses -- such as cross-entropy and dice -- have emerged as de facto standards to supervise neural networks. The Dice loss is an interesting case, as it comes from the relaxation of the popular Dice coefficient; one of the main evaluation metric in medical imaging applications. In this paper, we first study theoretically the gradient of the dice loss, showing that concretely it is a weighted negative of the ground truth, with a very small dynamic range. This enables us, in the second part of this paper, to mimic the supervision of the dice loss, through a simple element-wise multiplication of the network output with a negative of the ground truth. This rather surprising result sheds light on the practical supervision performed by the dice loss during gradient descent. This can help the practitioner to understand and interpret results while guiding researchers when designing new losses.