Position: Ideas Should be the Center of Machine Learning Research
This position paper addresses the problem of misaligned incentives in ML research for the research community, advocating for a cultural shift toward idea-centric science.
The paper argues that ML research overemphasizes benchmarks and theory, neglecting the central role of ideas. It proposes an 'Ideas First' framework that values ideas for their behavioral signatures in models, tested via tailored experiments, aiming to bridge theory-practice gaps and promote equity.
Machine learning research increasingly bifurcates into two disconnected modes: benchmark-driven engineering that prioritizes metrics over understanding, and idealized theory that often fails to transfer to modern systems. In this position paper, we argue that the field focuses too heavily on these endpoints, neglecting the central scientific object: the idea. We propose an Ideas First framework in which ideas are valued for the behavioral signatures they predict in modern models, and these signatures are tested through tailored experiments designed to detect the relevant patterns rather than to win leaderboards. This shift not only bridges the gap between theory and practice but also promotes equity by removing the "complexity premium," enabling rigorous scientific contributions from researchers with modest computational, financial, and human resources. Ultimately, we advocate for a research culture centered on ideas, treating benchmarks and theorems as instruments for testing mechanistic hypotheses rather than as ends in themselves.