Patch DCT vs LeNet
This work addresses efficiency in image classification for researchers, but it is incremental as it applies an existing method (DCT) to a standard dataset.
The paper compared a neural network using DCT of image patches against LeNet for MNIST digit classification, finding the DCT-based approach to be an order of magnitude faster while achieving competitive performance.
This paper compares the performance of a NN taking the output of a DCT (Discrete Cosine Transform) of an image patch with leNet for classifying MNIST hand written digits. The basis functions underlying the DCT bear a passing resemblance to some of the learned basis function of the Visual Transformer but are an order of magnitude faster to apply.