Joint Learning Global-Local Speaker Classification to Enhance End-to-End Speaker Diarization and Recognition
This addresses speaker discrimination issues in audio processing for applications like meeting transcription, though it appears incremental as it builds on existing LALM frameworks.
The paper tackles the limited speaker discriminability in Large Audio-Language Models for end-to-end speaker diarization and recognition by proposing GLSC-SDR, a paradigm that jointly trains speaker classification with diarization and recognition, achieving competitive or superior performance on datasets like AliMeeting, AISHELL-4, and AMI-SDM without large-scale real conversational data.
Large Audio-Language Models (LALMs) have demonstrated remarkable performance in end-to-end speaker diarization and recognition. However, their speaker discriminability remains limited due to the scarcity of large-scale conversational data and the absence of explicit speaker representation optimization. To address this, we propose GLSC-SDR, a paradigm that jointly trains speaker classification with diarization and recognition. We further introduce a Global-Local Speaker Classification strategy, which uses clustered speakers as global labels and re-encoded intra-cluster speakers as local labels. This hierarchical design enhances fine-grained speaker discrimination while preserving semantic transcription accuracy. Experiments on AliMeeting, AISHELL-4, and AMI-SDM demonstrate that GLSC-SDR achieves competitive or superior performance compared to simulation-based and multi-encoder approaches, without relying on large-scale real conversational data.