Automatic Hyper-Parameter Optimization Based on Mapping Discovery from Data to Hyper-Parameters
This addresses the costly and time-consuming nature of hyper-parameter tuning for machine learning practitioners, though it appears incremental as it builds on existing autoHPO methods.
The paper tackles the problem of automatic hyper-parameter optimization (autoHPO) by proposing an approach based on mapping data to hyper-parameters, using a sophisticated network structure and algorithms, which significantly outperforms state-of-the-art methods in experiments.
Machine learning algorithms have made remarkable achievements in the field of artificial intelligence. However, most machine learning algorithms are sensitive to the hyper-parameters. Manually optimizing the hyper-parameters is a common method of hyper-parameter tuning. However, it is costly and empirically dependent. Automatic hyper-parameter optimization (autoHPO) is favored due to its effectiveness. However, current autoHPO methods are usually only effective for a certain type of problems, and the time cost is high. In this paper, we propose an efficient automatic parameter optimization approach, which is based on the mapping from data to the corresponding hyper-parameters. To describe such mapping, we propose a sophisticated network structure. To obtain such mapping, we develop effective network constrution algorithms. We also design strategy to optimize the result futher during the application of the mapping. Extensive experimental results demonstrate that the proposed approaches outperform the state-of-the-art apporaches significantly.