MUTE: Data-Similarity Driven Multi-hot Target Encoding for Neural Network Design
This work addresses the challenge of efficient target encoding for neural networks, offering a novel method that enhances performance without additional computational costs, though it is incremental as it builds on existing target encoding techniques.
The paper tackles the problem of target encoding in neural networks requiring increased learning capacity and computational resources by introducing MUTE, a data-similarity driven multi-hot target encoding scheme that optimizes Hamming distances based on inter-class confusion. Experimental results show that MUTE improves generalization and robustness against noises and adversarial attacks on image classification networks and datasets, with negligible computation overhead and no increase in model size.
Target encoding is an effective technique to deliver better performance for conventional machine learning methods, and recently, for deep neural networks as well. However, the existing target encoding approaches require significant increase in the learning capacity, thus demand higher computation power and more training data. In this paper, we present a novel and efficient target encoding scheme, MUTE to improve both generalizability and robustness of a target model by understanding the inter-class characteristics of a target dataset. By extracting the confusion level between the target classes in a dataset, MUTE strategically optimizes the Hamming distances among target encoding. Such optimized target encoding offers higher classification strength for neural network models with negligible computation overhead and without increasing the model size. When MUTE is applied to the popular image classification networks and datasets, our experimental results show that MUTE offers better generalization and defense against the noises and adversarial attacks over the existing solutions.