TALL -- A Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages
This addresses performance gaps in low-resource languages for NLP applications, but it is incremental as it builds on existing translation and adapter methods.
The paper tackles the problem of LLMs struggling with low-resource languages by introducing TALL, an architecture that integrates an LLM with bilingual translation models to transform low-resource inputs into high-resource representations, resulting in significant improvements over baselines as demonstrated in experiments on Hebrew.
Large Language Models (LLMs) excel in high-resource languages but struggle with low-resource languages due to limited training data. This paper presents TALL (Trainable Architecture for Enhancing LLM Performance in Low-Resource Languages), which integrates an LLM with two bilingual translation models. TALL transforms low-resource inputs into high-resource representations, leveraging the LLM's capabilities while preserving linguistic features through dimension alignment layers and custom transformers. Our experiments on Hebrew demonstrate significant improvements over several baselines, including direct use, naive translation, and fine-tuning approaches. The architecture employs a parameter-efficient strategy, freezing pre-trained components while training only lightweight adapter modules, balancing computational efficiency with performance gains.