Got Compute, but No Data: Lessons From Post-training a Finnish LLM
This work addresses the problem of adapting LLMs for instruction-following in low-resource languages, which is incremental as it applies existing methods to a new linguistic context.
The researchers tackled the challenge of enabling LLMs to follow instructions and align with human preferences in low-resource languages like Finnish, by post-training a multilingual LLM using translated datasets, and achieved competitive performance in Finnish instruction-following with only a few hundred samples.
As LLMs gain more popularity as chatbots and general assistants, methods have been developed to enable LLMs to follow instructions and align with human preferences. These methods have found success in the field, but their effectiveness has not been demonstrated outside of high-resource languages. In this work, we discuss our experiences in post-training an LLM for instruction-following for English and Finnish. We use a multilingual LLM to translate instruction and preference datasets from English to Finnish. We perform instruction tuning and preference optimization in English and Finnish and evaluate the instruction-following capabilities of the model in both languages. Our results show that with a few hundred Finnish instruction samples we can obtain competitive performance in Finnish instruction-following. We also found that although preference optimization in English offers some cross-lingual benefits, we obtain our best results by using preference data from both languages. We release our model, datasets, and recipes under open licenses at https://huggingface.co/LumiOpen/Poro-34B-chat-OpenAssistant