Soft Alignment Objectives for Robust Adaptation of Language Generation
This addresses the issue of weakened generalization in adapted models for open-ended deployments, though it is incremental as it builds on existing adaptation methods.
The paper tackled the problem of catastrophic forgetting in domain adaptation for language generation models by introducing training objectives based on semantic similarity of predicted tokens to references, resulting in mitigated forgetting while preserving adaptation quality with negligible added compute.
Domain adaptation allows generative language models to address specific flaws caused by the domain shift of their application. However, the traditional adaptation by further training on in-domain data rapidly weakens the model's ability to generalize to other domains, making the open-ended deployments of the adapted models prone to errors. This work introduces novel training objectives built upon a semantic similarity of the predicted tokens to the reference. Our results show that (1) avoiding the common assumption of a single correct prediction by constructing the training target from tokens' semantic similarity can mitigate catastrophic forgetting during domain adaptation, while (2) preserving the quality of the adaptation, (3) with negligible additions to compute costs. In the broader context, the objectives grounded in a continuous token similarity pioneer the exploration of the middle ground between the efficient but naïve exact-match token-level objectives and expressive but computationally- and resource-intensive sequential objectives.