Efficient Training of Deep Convolutional Neural Networks by Augmentation in Embedding Space
This addresses efficiency issues for deploying large models on resource-constrained devices, but it is incremental as it builds on existing data augmentation and transfer learning techniques.
The paper tackles the high computational cost of data augmentation in raw input space for deep neural networks, especially on embedded devices, by proposing an approximate augmentation method in embedding space, which drastically reduces computation with negligible accuracy loss.
Recent advances in the field of artificial intelligence have been made possible by deep neural networks. In applications where data are scarce, transfer learning and data augmentation techniques are commonly used to improve the generalization of deep learning models. However, fine-tuning a transfer model with data augmentation in the raw input space has a high computational cost to run the full network for every augmented input. This is particularly critical when large models are implemented on embedded devices with limited computational and energy resources. In this work, we propose a method that replaces the augmentation in the raw input space with an approximate one that acts purely in the embedding space. Our experimental results show that the proposed method drastically reduces the computation, while the accuracy of models is negligibly compromised.