SODA: Self-organizing data augmentation in deep neural networks -- Application to biomedical image segmentation tasks
This work addresses efficiency in data augmentation for biomedical image segmentation, offering an incremental improvement over uniform budget allocation.
The paper tackles the problem of inefficient budget allocation in data augmentation by proposing a self-organizing method that dynamically assigns augmentation types during training, resulting in computational savings for greener machine learning.
In practice, data augmentation is assigned a predefined budget in terms of newly created samples per epoch. When using several types of data augmentation, the budget is usually uniformly distributed over the set of augmentations but one can wonder if this budget should not be allocated to each type in a more efficient way. This paper leverages online learning to allocate on the fly this budget as part of neural network training. This meta-algorithm can be run at almost no extra cost as it exploits gradient based signals to determine which type of data augmentation should be preferred. Experiments suggest that this strategy can save computation time and thus goes in the way of greener machine learning practices.