A layer-stress learning framework universally augments deep neural network tasks
This addresses a domain-specific problem for researchers and practitioners in AI and medical imaging by improving DNN performance through better layer utilization, though it appears incremental as it builds on existing attention mechanisms.
The paper tackled the challenge of selecting hidden layers in deep neural networks and introduced a layer-stress learning framework (x-NN) that uses Multi-Head Attention to leverage features from various depth layers, resulting in top performance in the Alzheimer's Disease Classification Challenge PRCV 2021 and verification on additional datasets.
Deep neural networks (DNN) such as Multi-Layer Perception (MLP) and Convolutional Neural Networks (CNN) represent one of the most established deep learning algorithms. Given the tremendous effects of the number of hidden layers on network architecture and performance, it is very important to choose the number of hidden layers but still a serious challenge. More importantly, the current network architectures can only process the information from the last layer of the feature extractor, which greatly limited us to further improve its performance. Here we presented a layer-stress deep learning framework (x-NN) which implemented automatic and wise depth decision on shallow or deep feature map in a deep network through firstly designing enough number of layers and then trading off them by Multi-Head Attention Block. The x-NN can make use of features from various depth layers through attention allocation and then help to make final decision as well. As a result, x-NN showed outstanding prediction ability in the Alzheimer's Disease Classification Technique Challenge PRCV 2021, in which it won the top laurel and outperformed all other AI models. Moreover, the performance of x-NN was verified by one more AD neuroimaging dataset and other AI tasks.