AKD : Adversarial Knowledge Distillation For Large Language Models Alignment on Coding tasks
This addresses the problem of ensuring dependable automated code generation for developers, though it appears incremental as it builds on existing knowledge distillation methods.
The paper tackles the challenges of quality, safety, and reliability in code generation by Large Language Models (LLMs) by introducing Adversarial Knowledge Distillation (AKD), which uses adversarially generated synthetic datasets to distill larger models into smaller, more efficient ones, enhancing robustness and parameter-efficiency.
The widespread adoption of Large Language Models (LLMs) for code generation, exemplified by GitHub Copilot\footnote{A coding extension powered by a Code-LLM to assist in code completion tasks} surpassing a million users, highlights the transformative potential of these tools in improving developer productivity. However, this rapid growth also underscores critical concerns regarding the quality, safety, and reliability of the code they generate. As Code-LLMs evolve, they face significant challenges, including the diminishing returns of model scaling and the scarcity of new, high-quality training data. To address these issues, this paper introduces Adversarial Knowledge Distillation (AKD), a novel approach that leverages adversarially generated synthetic datasets to distill the capabilities of larger models into smaller, more efficient ones. By systematically stress-testing and refining the reasoning capabilities of Code-LLMs, AKD provides a framework for enhancing model robustness, reliability, and security while improving their parameter-efficiency. We believe this work represents a critical step toward ensuring dependable automated code generation within the constraints of existing data and the cost-efficiency of model execution.