Be Fair! Can Machine Learning Engineering Agents Adhere to Fairness Constraints?
For developers and regulators of automated ML systems, this work highlights the responsibility gap in current MLE agents and the need for human-guided search and compliance assessment.
The paper evaluates two machine learning engineering agents on a melanoma classification task and finds that agent-generated pipelines underperform manually designed baselines in both predictive quality and fairness, even with fairness-oriented prompts.
Machine learning engineering (MLE) agents promise to automate end-to-end ML pipeline development from raw data and natural language instructions, potentially making ML accessible to non-technical domain experts. However, in sensitive and regulated domains, this abstraction creates a responsibility gap: end-users may lack visibility into design choices that affect correctness, robustness, fairness, and regulatory compliance. We argue that existing benchmarks are insufficient to assess whether MLE agents can be safely applied in such settings. We propose desiderata for a responsibility-centered evaluation framework and conduct an exploratory study on melanoma classification, focusing on fairness across skin tones as a responsibility constraint. When evaluating two recent MLE agents, we find that agent-generated pipelines show high variance and consistently underperform manually designed baselines in both predictive quality and fairness, despite fairness-oriented prompts. These preliminary results suggest that further research is needed towards redesigning MLE agents to allow humans to guide the search process and reliably assess the compliance and quality of the generated ML pipelines.