FaultFormer: Pretraining Transformers for Adaptable Bearing Fault Classification
This addresses the problem of expensive labeled data and poor adaptability in machine health monitoring for smart manufacturing, though it is incremental as it builds on existing transformer and self-supervised learning methods.
The authors tackled the lack of generalizability in deep learning for bearing fault classification by developing a self-supervised pretraining and fine-tuning framework based on transformers, achieving state-of-the-art accuracies and enabling adaptation to low-data regimes and new datasets.
The growth of global consumption has motivated important applications of deep learning to smart manufacturing and machine health monitoring. In particular, analyzing vibration data offers great potential to extract meaningful insights into predictive maintenance by the detection of bearing faults. Deep learning can be a powerful method to predict these mechanical failures; however, they lack generalizability to new tasks or datasets and require expensive, labeled mechanical data. We address this by presenting a novel self-supervised pretraining and fine-tuning framework based on transformer models. In particular, we investigate different tokenization and data augmentation strategies to reach state-of-the-art accuracies using transformer models. Furthermore, we demonstrate self-supervised masked pretraining for vibration signals and its application to low-data regimes, task adaptation, and dataset adaptation. Pretraining is able to improve performance on scarce, unseen training samples, as well as when fine-tuning on fault classes outside of the pretraining distribution. Furthermore, pretrained transformers are shown to be able to generalize to a different dataset in a few-shot manner. This introduces a new paradigm where models can be pretrained on unlabeled data from different bearings, faults, and machinery and quickly deployed to new, data-scarce applications to suit specific manufacturing needs.