A Lightweight Two-Branch Architecture for Multi-instrument Transcription via Note-Level Contrastive Clustering
This addresses practical deployment challenges for multi-instrument transcription in resource-constrained settings, though it appears incremental in its architectural improvements.
The paper tackles the problem of multi-instrument transcription by developing a lightweight model that overcomes generalization, source-count, and computational limitations of existing approaches, achieving competitive transcription accuracy and separation quality while enabling deployment on resource-constrained devices.
Existing multi-timbre transcription models struggle with generalization beyond pre-trained instruments, rigid source-count constraints, and high computational demands that hinder deployment on low-resource devices. We address these limitations with a lightweight model that extends a timbre-agnostic transcription backbone with a dedicated timbre encoder and performs deep clustering at the note level, enabling joint transcription and dynamic separation of arbitrary instruments given a specified number of instrument classes. Practical optimizations including spectral normalization, dilated convolutions, and contrastive clustering further improve efficiency and robustness. Despite its small size and fast inference, the model achieves competitive performance with heavier baselines in terms of transcription accuracy and separation quality, and shows promising generalization ability, making it highly suitable for real-world deployment in practical and resource-constrained settings.