AugOp: Inject Transformation into Neural Operator
This work addresses the need for more efficient neural network training for image classification tasks, but it is incremental as it builds on existing ResNet architectures.
The authors tackled the problem of increasing learning capacity in convolutional neural networks without adding computational overhead during inference by proposing AugConv, a method that injects group-wise transformations during training and merges them with regular convolutions. On CIFAR-10, AugResNet, built with AugConv, outperformed its baseline in model performance.
In this paper, we propose a simple and general approach to augment regular convolution operator by injecting extra group-wise transformation during training and recover it during inference. Extra transformation is carefully selected to ensure it can be merged with regular convolution in each group and will not change the topological structure of regular convolution during inference. Compared with regular convolution operator, our approach (AugConv) can introduce larger learning capacity to improve model performance during training but will not increase extra computational overhead for model deployment. Based on ResNet, we utilize AugConv to build convolutional neural networks named AugResNet. Result on image classification dataset Cifar-10 shows that AugResNet outperforms its baseline in terms of model performance.