VEPO: Variable Entropy Policy Optimization for Low-Resource Language Foundation Models
This work addresses performance gaps for underrepresented languages in foundation models, representing a novel method for a known bottleneck.
The paper tackles the problem of suboptimal performance of large language models on low-resource languages by proposing Variable Entropy Policy Optimization (VEPO), which improves tokenization efficiency and translation quality, as demonstrated by empirical evaluations across 90 FLORES-200, COMET-22, and chrF directions.
Large language models frequently exhibit suboptimal performance on low resource languages, primarily due to inefficient subword segmentation and systemic training data imbalances. In this paper, we propose Variable Entropy Policy Optimization (VEPO), which leverages Reinforcement Learning with Verifiable Rewards to incorporate deterministic structural constraints into the policy alignment process. This framework ensures prescribed sequence length, robust format consistency, and rigorous linguistic well formedness, all enforced during training. Central to our approach is a variable entropy mechanism that enables the model to dynamically calibrate the equilibrium between literal fidelity and semantic naturalness by modulating the exploration exploitation manifold. By integrating entropy tempered advantage estimation with asymmetric clipping, VEPO sustains robust exploration while mitigating policy collapse. Empirical evaluations across 90 FLORES-200, COMET-22, chrF directions demonstrate that VEPO yields substantial improvements in both tokenization efficiency and translation quality, bridging the performance gap for underrepresented languages.