KAN-Mixers: a new deep learning architecture for image classification
This work addresses image classification for computer vision researchers, offering an incremental improvement by replacing MLPs with KANs in mixer architectures.
The paper tackled image classification by proposing KAN-Mixers, a new architecture using Kolmogorov-Arnold Networks (KANs) as main layers, which achieved superior accuracy over MLP, MLP-Mixer, and KAN models on Fashion-MNIST (0.9030) and CIFAR-10 (0.6980) datasets.
Due to their effective performance, Convolutional Neural Network (CNN) and Vision Transformer (ViT) architectures have become the standard for solving computer vision tasks. Such architectures require large data sets and rely on convolution and self-attention operations. In 2021, MLP-Mixer emerged, an architecture that relies only on Multilayer Perceptron (MLP) and achieves extremely competitive results when compared to CNNs and ViTs. Despite its good performance in computer vision tasks, the MLP-Mixer architecture may not be suitable for refined feature extraction in images. Recently, the Kolmogorov-Arnold Network (KAN) was proposed as a promising alternative to MLP models. KANs promise to improve accuracy and interpretability when compared to MLPs. Therefore, the present work aims to design a new mixer-based architecture, called KAN-Mixers, using KANs as main layers and evaluate its performance, in terms of several performance metrics, in the image classification task. As main results obtained, the KAN-Mixers model was superior to the MLP, MLP-Mixer and KAN models in the Fashion-MNIST and CIFAR-10 datasets, with 0.9030 and 0.6980 of average accuracy, respectively.