AutoMIA: Improved Baselines for Membership Inference Attack via Agentic Self-Exploration
This work addresses the need for more adaptable auditing tools to evaluate training data leakage in machine learning models, representing an incremental improvement over existing methods.
The paper tackled the problem of membership inference attacks (MIAs) lacking adaptability across models by proposing AutoMIA, an agentic framework for automated strategy evolution, which consistently matched or outperformed state-of-the-art baselines without manual feature engineering.
Membership Inference Attacks (MIAs) serve as a fundamental auditing tool for evaluating training data leakage in machine learning models. However, existing methodologies predominantly rely on static, handcrafted heuristics that lack adaptability, often leading to suboptimal performance when transferred across different large models. In this work, we propose AutoMIA, an agentic framework that reformulates membership inference as an automated process of self-exploration and strategy evolution. Given high-level scenario specifications, AutoMIA self-explores the attack space by generating executable logits-level strategies and progressively refining them through closed-loop evaluation feedback. By decoupling abstract strategy reasoning from low-level execution, our framework enables a systematic, model-agnostic traversal of the attack search space. Extensive experiments demonstrate that AutoMIA consistently matches or outperforms state-of-the-art baselines while eliminating the need for manual feature engineering.