NanoFlux: Adversarial Dual-LLM Evaluation and Distillation For Multi-Domain Reasoning
This addresses the challenge of efficiently enhancing LLM reasoning across multiple domains, though it is incremental as it builds on existing adversarial and fine-tuning methods.
The paper tackles the problem of improving LLM reasoning by introducing NanoFlux, an adversarial framework that generates targeted training data, resulting in performance gains of up to +16.6% on medical reasoning and computational reductions of 3-14x compared to conventional fine-tuning.
We present NanoFlux, a novel adversarial framework for generating targeted training data to improve LLM reasoning, where adversarially-generated datasets containing fewer than 200 examples outperform conventional fine-tuning approaches. The framework employs a competitive dynamic between models alternating as Attacker and Defender, supervised by a tool-augmented Judge, synthesizing multi-step questions with explanatory annotations that target specific reasoning capabilities. Fine-tuning a 4B-parameter model on NanoFlux-generated data yields performance gains across diverse domains compared to full-benchmark fine-tuning: +5.9% on mathematical reasoning (GSMHard), +3.6% on scientific reasoning (GenomeBench), and +16.6% on medical reasoning (MultiMedQA), while reducing computational requirements by 3-14x. Ablation studies reveal a non-monotonic relationship between dataset characteristics and model performance, uncovering domain-specific optimal points for question complexity and reasoning quality. NanoFlux automates training data generation through embedding-based novelty filtering, tool-augmented evaluation, and multi-hop reasoning, suggesting that future model improvements may lie in the intelligent synthesis of small, precisely targeted training datasets.