Tensor Normalization and Full Distribution Training
This work addresses accuracy and robustness issues in deep neural networks, presenting incremental improvements through novel normalization and training techniques.
The paper tackles improving accuracy and robustness in deep neural networks by introducing pixel-wise tensor normalization and a reformulation of multi-class problems into multi-label ones, resulting in significant improvements in accuracy and robustness.
In this work, we introduce pixel wise tensor normalization, which is inserted after rectifier linear units and, together with batch normalization, provides a significant improvement in the accuracy of modern deep neural networks. In addition, this work deals with the robustness of networks. We show that the factorized superposition of images from the training set and the reformulation of the multi class problem into a multi-label problem yields significantly more robust networks. The reformulation and the adjustment of the multi class log loss also improves the results compared to the overlay with only one class as label. https://atreus.informatik.uni-tuebingen.de/seafile/d/8e2ab8c3fdd444e1a135/?p=%2FTNandFDT&mode=list