Multilingual Hierarchical Attention Networks for Document Classification
This work addresses the challenge of efficient and effective multilingual document classification, which is incremental as it builds on existing hierarchical attention networks by extending them to a multilingual context.
The authors tackled the problem of multilingual document classification by proposing multilingual hierarchical attention networks that share encoders and attention mechanisms across languages, achieving better performance than monolingual models in both low-resource and full-resource settings while using fewer parameters.
Hierarchical attention networks have recently achieved remarkable performance for document classification in a given language. However, when multilingual document collections are considered, training such models separately for each language entails linear parameter growth and lack of cross-language transfer. Learning a single multilingual model with fewer parameters is therefore a challenging but potentially beneficial objective. To this end, we propose multilingual hierarchical attention networks for learning document structures, with shared encoders and/or shared attention mechanisms across languages, using multi-task learning and an aligned semantic space as input. We evaluate the proposed models on multilingual document classification with disjoint label sets, on a large dataset which we provide, with 600k news documents in 8 languages, and 5k labels. The multilingual models outperform monolingual ones in low-resource as well as full-resource settings, and use fewer parameters, thus confirming their computational efficiency and the utility of cross-language transfer.