Notes on the Behavior of MC Dropout
This study provides insights for practitioners using Monte-Carlo dropout, but it is incremental as it focuses on understanding existing properties rather than introducing new methods.
The paper investigates the behavior of Monte-Carlo dropout for uncertainty estimation in deep neural networks, revealing that its quality varies and depends on architectural and training choices, with no specific numerical results reported.
Among the various options to estimate uncertainty in deep neural networks, Monte-Carlo dropout is widely popular for its simplicity and effectiveness. However the quality of the uncertainty estimated through this method varies and choices in architecture design and in training procedures have to be carefully considered and tested to obtain satisfactory results. In this paper we present a study offering a different point of view on the behavior of Monte-Carlo dropout, which enables us to observe a few interesting properties of the technique to keep in mind when considering its use for uncertainty estimation.