Interpreting and Boosting Dropout from a Game-Theoretic View
This work addresses over-fitting in deep learning for researchers and practitioners, but it is incremental as it builds on existing dropout methods.
The paper tackled the problem of understanding and improving dropout in deep neural networks by proving it suppresses interactions between input variables, which are linked to over-fitting, and proposed an interaction loss that boosts DNN performance as shown in experiments.
This paper aims to understand and improve the utility of the dropout operation from the perspective of game-theoretic interactions. We prove that dropout can suppress the strength of interactions between input variables of deep neural networks (DNNs). The theoretic proof is also verified by various experiments. Furthermore, we find that such interactions were strongly related to the over-fitting problem in deep learning. Thus, the utility of dropout can be regarded as decreasing interactions to alleviate the significance of over-fitting. Based on this understanding, we propose an interaction loss to further improve the utility of dropout. Experimental results have shown that the interaction loss can effectively improve the utility of dropout and boost the performance of DNNs.