Aligning a medium-size GPT model in English to a small closed domain in Spanish
This work addresses domain adaptation for language models in a specific bilingual context, but it is incremental as it applies existing alignment techniques to a new language and domain combination.
The paper tackled the problem of adapting a medium-sized English GPT model to a small closed domain in Spanish for question answering, using a reward model to improve answer generation, and found that the proposed method outperformed others in metrics like BLEU and perplexity.
In this paper, we propose a methodology to align a medium-sized GPT model, originally trained in English for an open domain, to a small closed domain in Spanish. The application for which the model is finely tuned is the question answering task. To achieve this we also needed to train and implement another neural network (which we called the reward model) that could score and determine whether an answer is appropriate for a given question. This component served to improve the decoding and generation of the answers of the system. Numerical metrics such as BLEU and perplexity were used to evaluate the model, and human judgment was also used to compare the decoding technique with others. Finally, the results favored the proposed method, and it was determined that it is feasible to use a reward model to align the generation of responses.