Multi-Method Self-Training: Improving Code Generation With Text, And Vice Versa
This addresses the usability and performance issues in code generation and related tasks for users of large language models, though it appears incremental as it builds on existing self-training techniques.
The paper tackles the problem of large language models having multiple methods for solving the same problem, which can make it difficult for users to choose the best one, by introducing Multi-Method Self-Training (MMST) that trains one method on the filtered outputs of another, resulting in improvements of up to 32.2% for the more performant method and up to 30% for the less performant one.
Large Language Models have many methods for solving the same problem. This introduces novel strengths (different methods may work well for different problems) and weaknesses (it may be difficult for users to know which method to use). In this paper, we introduce Multi-Method Self-Training (MMST), where one method is trained on the filtered outputs of another, allowing us to augment the strengths and ameliorate the weaknesses of each method. Using a 176B parameter model trained on both language and code, we show that MMST can 1) improve the less performant method (up to 30%) making the model easier to use, 2) improve the more performant method (up to 32.2%) making the model more performant, and 3) improve the performance of related but distinct tasks (up to 10.3%) by improving the ability of the model to generate rationales. We then conduct ablation analyses to explore why MMST works. We show that MMST generates more data than traditional self-training, but the improvement in performance is driven by the use of multiple methods. We also analyze prompt-engineering and anti-correlated performance between methods as means of making MMST more effective. We hope the evidence from our paper motivates machine learning researchers to explore ways in which advances in language models allow for new forms of training.