Utilizing Multimodal Data for Edge Case Robust Call-sign Recognition and Understanding
This work addresses robustness challenges in air-traffic control speech processing, particularly for noisy or clipped recordings, representing an incremental improvement in a domain-specific application.
The paper tackles the problem of improving edge-case robustness in call-sign recognition and understanding for air-traffic control by proposing a multimodal model, resulting in up to a 15% performance increase in edge cases and enhanced accuracy across a wide operational range.
Operational machine-learning based assistant systems must be robust in a wide range of scenarios. This hold especially true for the air-traffic control (ATC) domain. The robustness of an architecture is particularly evident in edge cases, such as high word error rate (WER) transcripts resulting from noisy ATC recordings or partial transcripts due to clipped recordings. To increase the edge-case robustness of call-sign recognition and understanding (CRU), a core tasks in ATC speech processing, we propose the multimodal call-sign-command recovery model (CCR). The CCR architecture leads to an increase in the edge case performance of up to 15%. We demonstrate this on our second proposed architecture, CallSBERT. A CRU model that has less parameters, can be fine-tuned noticeably faster and is more robust during fine-tuning than the state of the art for CRU. Furthermore, we demonstrate that optimizing for edge cases leads to a significantly higher accuracy across a wide operational range.