State-of-the-art Small Language Coder Model: Mify-Coder
This work addresses the need for efficient and deployable code generation models for developers and enterprises, though it appears incremental as it builds on existing foundation models and training strategies.
The paper tackles the problem of developing compact code models by introducing Mify-Coder, a 2.5B-parameter model trained on 4.2T tokens, which achieves comparable accuracy and safety while outperforming larger baseline models on coding and function-calling benchmarks, demonstrating that smaller models can match frontier-grade performance.
We present Mify-Coder, a 2.5B-parameter code model trained on 4.2T tokens using a compute-optimal strategy built on the Mify-2.5B foundation model. Mify-Coder achieves comparable accuracy and safety while significantly outperforming much larger baseline models on standard coding and function-calling benchmarks, demonstrating that compact models can match frontier-grade models in code generation and agent-driven workflows. Our training pipeline combines high-quality curated sources with synthetic data generated through agentically designed prompts, refined iteratively using enterprise-grade evaluation datasets. LLM-based quality filtering further enhances data density, enabling frugal yet effective training. Through disciplined exploration of CPT-SFT objectives, data mixtures, and sampling dynamics, we deliver frontier-grade code intelligence within a single continuous training trajectory. Empirical evidence shows that principled data and compute discipline allow smaller models to achieve competitive accuracy, efficiency, and safety compliance. Quantized variants of Mify-Coder enable deployment on standard desktop environments without requiring specialized hardware.