When Less Is More? Diagnosing ASR Predictions in Sardinian via Layer-Wise Decoding
This work addresses ASR model diagnostics for low-resource languages, offering incremental insights into intermediate layer representations.
The study tackled the problem of improving phoneme-level predictions in automatic speech recognition (ASR) for low-resource Campidanese Sardinian by applying a layer-wise decoding strategy to a pretrained Wav2Vec2 model, finding that truncating upper transformer layers reduced Phoneme Error Rates (PER) with the best performance two layers earlier than the final layer.
Recent studies have shown that intermediate layers in multilingual speech models often encode more phonetically accurate representations than the final output layer. In this work, we apply a layer-wise decoding strategy to a pretrained Wav2Vec2 model to investigate how phoneme-level predictions evolve across encoder layers, focusing on Campidanese Sardinian, a low-resource language. We show that truncating upper transformer layers leads to improved Phoneme Error Rates (PER), with the best performance achieved not at the final layer, but two layers earlier. Through fine-grained alignment analysis, we find that intermediate predictions better preserve segmental identity, avoid overgeneration, and reduce certain classes of phonological errors. We also introduce the notion of regressive errors, cases where correct predictions at intermediate layers are overwritten by errors at the final layer. These regressions highlight the limitations of surface-level error metrics and reveal how deeper layers may generalize or abstract away from acoustic detail. Our findings support the use of early-layer probing as a diagnostic tool for ASR models, particularly in low-resource settings where standard evaluation metrics may fail to capture linguistically meaningful behavior.