Fast, Better Training Trick -- Random Gradient
This addresses training efficiency and performance for computer vision tasks, but appears incremental as it builds on existing gradient-based methods.
The paper tackles the problem of accelerating training and improving performance in deep learning models by introducing random gradient (RG), a method that multiplies loss by a random number to reduce back-propagation gradients, resulting in faster training and better results on datasets like Pascal VOC, Cifar, and Cityscapes.
In this paper, we will show an unprecedented method to accelerate training and improve performance, which called random gradient (RG). This method can be easier to the training of any model without extra calculation cost, we use Image classification, Semantic segmentation, and GANs to confirm this method can improve speed which is training model in computer vision. The central idea is using the loss multiplied by a random number to random reduce the back-propagation gradient. We can use this method to produce a better result in Pascal VOC, Cifar, Cityscapes datasets.