NBF at SemEval-2025 Task 5: Light-Burst Attention Enhanced System for Multilingual Subject Recommendation
This work addresses subject recommendation for multilingual academic texts, but it is incremental as it builds on existing methods with specific optimizations for resource constraints.
The paper tackled cross-lingual subject classification in English and German academic domains by using a dimension-as-token self-attention mechanism with reduced dimensions, achieving average recall rates of 32.24% in general quantitative settings and up to 43.16% in qualitative evaluations.
We present our system submission for SemEval 2025 Task 5, which focuses on cross-lingual subject classification in the English and German academic domains. Our approach leverages bilingual data during training, employing negative sampling and a margin-based retrieval objective. We demonstrate that a dimension-as-token self-attention mechanism designed with significantly reduced internal dimensions can effectively encode sentence embeddings for subject retrieval. In quantitative evaluation, our system achieved an average recall rate of 32.24% in the general quantitative setting (all subjects), 43.16% and 31.53% of the general qualitative evaluation methods with minimal GPU usage, highlighting their competitive performance. Our results demonstrate that our approach is effective in capturing relevant subject information under resource constraints, although there is still room for improvement.