Technical Report: Combining knowledge from Transfer Learning during training and Wide Resnets
This is an incremental improvement for deep learning practitioners seeking better model performance through architectural tweaks.
The authors tackled the problem of optimizing deep neural network architectures by combining Wide ResNets and transfer learning, resulting in improved performance for models with both high and standard data augmentation.
In this report, we combine the idea of Wide ResNets and transfer learning to optimize the architecture of deep neural networks. The first improvement of the architecture is the use of all layers as information source for the last layer. This idea comes from transfer learning, which uses networks pre-trained on other data and extracts different levels of the network as input for the new task. The second improvement is the use of deeper layers instead of deeper sequences of blocks. This idea comes from Wide ResNets. Using both optimizations, both high data augmentation and standard data augmentation can produce better results for different models. Link: https://github.com/wolfgangfuhl/PublicationStuff/tree/master/TechnicalReport1/Supp