A Tensor-based Convolutional Neural Network for Small Dataset Classification
This work addresses the need for more efficient and robust neural networks for small dataset classification, representing an incremental improvement over existing ConvNets.
The authors tackled the problem of improving efficiency and robustness in convolutional neural networks by proposing a Tensor-based Neural Network (TCNN) that uses structured neurons and implicit part-whole relationships. The result showed that TCNNs achieved higher parameter efficiency on datasets like CIFAR10, CIFAR100, and Tiny ImageNet, and demonstrated increased robustness against white-box adversarial attacks on MNIST compared to standard ConvNets.
Inspired by the ConvNets with structured hidden representations, we propose a Tensor-based Neural Network, TCNN. Different from ConvNets, TCNNs are composed of structured neurons rather than scalar neurons, and the basic operation is neuron tensor transformation. Unlike other structured ConvNets, where the part-whole relationships are modeled explicitly, the relationships are learned implicitly in TCNNs. Also, the structured neurons in TCNNs are high-rank tensors rather than vectors or matrices. We compare TCNNs with current popular ConvNets, including ResNets, MobileNets, EfficientNets, RegNets, etc., on CIFAR10, CIFAR100, and Tiny ImageNet. The experiment shows that TCNNs have higher efficiency in terms of parameters. TCNNs also show higher robustness against white-box adversarial attacks on MNIST compared to ConvNets.