Design of an basis-projected layer for sparse datasets in deep learning training using gc-ms spectra as a case study
This addresses optimization difficulties in deep learning for sparse datasets like GC-MS spectra and DNA sequences, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackled the problem of training deep learning models on sparse datasets like GC-MS spectra by proposing a basis-projected layer (BPL) that transforms sparse data into dense representations, resulting in F1 score improvements of 8.56% to 11.49% on a coffee odorant dataset.
Deep learning (DL) models encompass millions or even billions of parameters and learn complex patterns from big data. However, not all data are initially stored in a suitable formation to effectively train a DL model, e.g., gas chromatography-mass spectrometry (GC-MS) spectra and DNA sequence. These datasets commonly contain many zero values, and the sparse data formation causes difficulties in optimizing DL models. A DL module called the basis-projected layer (BPL) was proposed to mitigate the issue by transforming the sparse data into a dense representation. The transformed data is expected to facilitate the gradient calculation and finetuned process in a DL training process. The dataset, example of a sparse dataset, contained 362 specialty coffee odorant spectra detected from GC-MS. The BPL layer was placed at the beginning of the DL model. The tunable parameters in the layer were learnable projected axes that were the bases of a new representation space. The layer rotated these bases when its parameters were updated. When the number of the bases was the same as the original dimension, the increasing percentage of the F1 scores was 8.56%. Furthermore, when the number was set as 768 (the original dimension was 490), the increasing percentage of the F1 score was 11.49%. The layer not only maintained the model performance and even constructed a better representation space in analyzing sparse datasets.