Practitioner Motives to Use Different Hyperparameter Optimization Methods
This research addresses the problem of under-optimized ML models for practitioners by clarifying motives to improve user-centered HPO tool development, though it is incremental as it builds on existing knowledge of HPO methods.
The study investigated why practitioners often use inefficient hyperparameter optimization methods like grid search instead of more efficient ones, by conducting interviews and a survey with ML experts to identify their motives and contextual factors.
Programmatic hyperparameter optimization (HPO) methods, such as Bayesian optimization and evolutionary algorithms, are highly sample-efficient in identifying optimal hyperparameter configurations for machine learning (ML) models. However, practitioners frequently use less efficient methods, such as grid search, which can lead to under-optimized models. We suspect this behavior is driven by a range of practitioner-specific motives. Practitioner motives, however, still need to be clarified to enhance user-centered development of HPO tools. To uncover practitioner motives to use different HPO methods, we conducted 20 semi-structured interviews and an online survey with 49 ML experts. By presenting main goals (e.g., increase ML model understanding) and contextual factors affecting practitioners' selection of HPO methods (e.g., available computer resources), this study offers a conceptual foundation to better understand why practitioners use different HPO methods, supporting development of more user-centered and context-adaptive HPO tools in automated ML.