ACE: Adaptive Constraint-aware Early Stopping in Hyperparameter Optimization
This work addresses the need for efficient hyperparameter tuning under constraints like fairness or robustness, which is incremental but practical for machine learning practitioners.
The paper tackles the problem of hyperparameter optimization under deployment constraints by proposing ACE, an adaptive constraint-aware early stopping method that reduces tuning cost by 30-50% compared to baselines while maintaining model quality.
Deploying machine learning models requires high model quality and needs to comply with application constraints. That motivates hyperparameter optimization (HPO) to tune model configurations under deployment constraints. The constraints often require additional computation cost to evaluate, and training ineligible configurations can waste a large amount of tuning cost. In this work, we propose an Adaptive Constraint-aware Early stopping (ACE) method to incorporate constraint evaluation into trial pruning during HPO. To minimize the overall optimization cost, ACE estimates the cost-effective constraint evaluation interval based on a theoretical analysis of the expected evaluation cost. Meanwhile, we propose a stratum early stopping criterion in ACE, which considers both optimization and constraint metrics in pruning and does not require regularization hyperparameters. Our experiments demonstrate superior performance of ACE in hyperparameter tuning of classification tasks under fairness or robustness constraints.