Physically Verifiable Evidence and LLM-Based Reporting for Bearing Fault Diagnosis
For safety-critical mechanical systems, this work provides a validation mechanism for AI diagnoses and reduces hallucination risks in LLM-based reporting, but the approach is domain-specific and incremental.
The paper addresses the lack of physically verifiable evidence in AI-based bearing fault diagnosis and the risk of hallucinated content in LLM-generated reports. It proposes DENet, which outputs structured evidence including characteristic frequency and transient localization, achieving ~6 Hz frequency error and detecting misclassifications with AUROC up to 0.970, while constraining LLM reporting to reduce unsupported claims from 10-12% to 2%.
Trustworthy deployment of AI-based diagnosis in safety-critical mechanical systems hinges on validation: whether a prediction can be checked against physical reality before it is acted upon. Current intelligent fault diagnosers fail this standard in two ways. Their standard output, a class label with a softmax confidence score, is an internal statistic of the classifier, offering nothing checkable against independent physical knowledge; and the growing use of generative language models in maintenance reporting adds a second risk: hallucinated content entering reports on which decisions rest. Taking bearing fault diagnosis as the testbed, this work addresses both problems from the output side. The proposed Diagnostic Evidence Network (DENet) is an encoder-agnostic multi-task framework extending the output to a structured evidence record: the classification, a predicted characteristic frequency comparable against the theoretical value determined by bearing geometry and shaft speed, and a temporal localization of transient impulses inspectable on the raw waveform. Across four encoders and three public datasets, this evidence incurs no statistically significant accuracy cost, with a frequency error of about 6 Hz on 1,024-point segments where spectral estimation is structurally inapplicable. Centrally, the deviation between predicted and theoretical frequency constitutes a label-free, inference-time validation signal: it detects misclassifications with AUROC values of 0.970 and 0.871, and remains discriminative in the high-confidence regime where confidence-derived detectors are blind. Finally, a QLoRA-adapted language model is constrained to translate, but never generate, diagnostic content, reducing unsupported-claim rates from 10-12% to 2% and eliminating fabricated quantities.