mmWave Radar Aware Dual-Conditioned GAN for Speech Reconstruction of Signals With Low SNR
This work addresses the challenge of speech reconstruction for low-SNR mmWave radar signals, which is incremental as it builds on existing methods with tailored components for a niche domain.
The paper tackles the problem of reconstructing intelligible full-bandwidth speech from noisy, band-limited millimeter-wave radar captures with low signal-to-noise ratios (-5 dB to -1 dB) through glass walls, achieving state-of-the-art performance on this specific task.
Millimeter-wave (mmWave) radar captures are band-limited and noisy, making for difficult reconstruction of intelligible full-bandwidth speech. In this work, we propose a two-stage speech reconstruction pipeline for mmWave using a Radar-Aware Dual-conditioned Generative Adversarial Network (RAD-GAN), which is capable of performing bandwidth extension on signals with low signal-to-noise ratios (-5 dB to -1 dB), captured through glass walls. We propose an mmWave-tailored Multi-Mel Discriminator (MMD) and a Residual Fusion Gate (RFG) to enhance the generator input to process multiple conditioning channels. The proposed two-stage pipeline involves pretraining the model on synthetically clipped clean speech and finetuning on fused mel spectrograms generated by the RFG. We empirically show that the proposed method, trained on a limited dataset, with no pre-trained modules, and no data augmentations, outperformed state-of-the-art approaches for this specific task. Audio examples of RAD-GAN are available online at https://rad-gan-demo-site.vercel.app/.