Towards Community-Driven Agents for Machine Learning Engineering
This work addresses the challenge of enabling ML agents to engage with research communities for better problem-solving, representing a novel but incremental step in agent design.
The paper tackles the problem of isolated operation in machine learning agents by introducing CoMind, an agent that communicates with a simulated Kaggle community to leverage collective knowledge, achieving state-of-the-art performance on the MLE-Live framework and outperforming 79.2% of human competitors on average across four Kaggle competitions.
Large language model-based machine learning (ML) agents have shown great promise in automating ML research. However, existing agents typically operate in isolation on a given research problem, without engaging with the broader research community, where human researchers often gain insights and contribute by sharing knowledge. To bridge this gap, we introduce MLE-Live, a live evaluation framework designed to assess an agent's ability to communicate with and leverage collective knowledge from a simulated Kaggle research community. Building on this framework, we propose CoMind, a novel agent that excels at exchanging insights and developing novel solutions within a community context. CoMind achieves state-of-the-art performance on MLE-Live and outperforms 79.2% human competitors on average across four ongoing Kaggle competitions. Our code is released at https://github.com/comind-ml/CoMind.