Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision
This highlights a reproducibility issue for researchers and practitioners in computer vision, though it is incremental as it focuses on a known but underexplored factor.
The paper investigates the effect of random seed selection on accuracy in deep learning for computer vision, finding that while variance is generally small, it is easy to find outlier seeds that perform much better or worse than average, with experiments scanning up to 10^4 seeds on CIFAR-10 and fewer on ImageNet.
In this paper I investigate the effect of random seed selection on the accuracy when using popular deep learning architectures for computer vision. I scan a large amount of seeds (up to $10^4$) on CIFAR 10 and I also scan fewer seeds on Imagenet using pre-trained models to investigate large scale datasets. The conclusions are that even if the variance is not very large, it is surprisingly easy to find an outlier that performs much better or much worse than the average.