PHOENIX: Open-Source Language Adaption for Direct Preference Optimization
This work addresses the underdeveloped area of language transfer for large language models, specifically for German, but is incremental as it applies existing DPO techniques to a new language.
The paper tackles the problem of adapting large language models to languages beyond English by applying Direct Preference Optimization (DPO) to German, resulting in an open-source model called PHOENIX.
Large language models have gained immense importance in recent years and have demonstrated outstanding results in solving various tasks. However, despite these achievements, many questions remain unanswered in the context of large language models. Besides the optimal use of the models for inference and the alignment of the results to the desired specifications, the transfer of models to other languages is still an underdeveloped area of research. The recent publication of models such as Llama-2 and Zephyr has provided new insights into architectural improvements and the use of human feedback. However, insights into adapting these techniques to other languages remain scarce. In this paper, we build on latest improvements and apply the Direct Preference Optimization(DPO) approach to the German language. The model is available at https://huggingface.co/DRXD1000/Phoenix.