Quality Measures of Parameter Tuning for Aggregated Multi-Objective Temporal Planning
This addresses parameter tuning for researchers in multi-objective optimization, but it is incremental as it focuses on a specific context and indicator comparison.
The paper tackles the problem of selecting quality measures for parameter tuning in aggregated multi-objective temporal planning, finding that using the hypervolume indicator instead of best fitness leads to better performance, as demonstrated in a case study with DaE-YAHSP and ParamILS.
Parameter tuning is recognized today as a crucial ingredient when tackling an optimization problem. Several meta-optimization methods have been proposed to find the best parameter set for a given optimization algorithm and (set of) problem instances. When the objective of the optimization is some scalar quality of the solution given by the target algorithm, this quality is also used as the basis for the quality of parameter sets. But in the case of multi-objective optimization by aggregation, the set of solutions is given by several single-objective runs with different weights on the objectives, and it turns out that the hypervolume of the final population of each single-objective run might be a better indicator of the global performance of the aggregation method than the best fitness in its population. This paper discusses this issue on a case study in multi-objective temporal planning using the evolutionary planner DaE-YAHSP and the meta-optimizer ParamILS. The results clearly show how ParamILS makes a difference between both approaches, and demonstrate that indeed, in this context, using the hypervolume indicator as ParamILS target is the best choice. Other issues pertaining to parameter tuning in the proposed context are also discussed.