MapCoder-Lite: Squeezing Multi-Agent Coding into a Single Small LLM
This work addresses the challenge of making multi-agent coding efficient and accessible on small models, which is incremental as it builds on existing multi-agent approaches but with novel optimizations.
The paper tackles the problem of enabling multi-agent coding systems on small language models, which previously failed or required large models, by introducing MapCoder-Lite, a method that upgrades a single 7B model into four specialized agents using lightweight techniques, resulting in more than doubled accuracy on xCodeEval (from 13.2% to 28.3%) and reduced resource usage.
Large language models (LLMs) have advanced code generation from single-function tasks to competitive-programming problems, but existing multi-agent solutions either rely on costly large-scale ($>$ 30B) models or collapse when downsized to small open-source models. We present MapCoder-Lite, which upgrades a single 7B model into four role-specialised agents-retriever, planner, coder, and debugger-using only rank-32, role-specific LoRA adapters ($<3\%$ extra parameters). Three lightweight techniques make this possible: (i) trajectory distillation from strong LLMs fixes format fragility in retrieval and debugging, (ii) supervisor-guided correction strengthens planning and coding agents, and (iii) agent-wise LoRA fine-tuning delivers memory-efficient specialisation. Comprehensive evaluation on xCodeEval, APPS, and CodeContests shows that MapCoder-Lite more than doubles xCodeEval accuracy (from $13.2\%$ to $28.3\%$), eliminates all format failures, and closes to within six points of a 32B baseline while cutting GPU memory and token-generation time by $4\times$. These results demonstrate that careful agent-wise fine-tuning unleashes high-quality multi-agent coding on a small language model.