Variance Reduction for Better Sampling in Continuous Domains
This work addresses sampling efficiency for optimization algorithms, offering incremental improvements in design of experiments and evolutionary methods.
The paper tackles the problem of optimal sampling for optimization in continuous domains by showing that the search distribution should be more peaked than the prior, providing explicit formulas based on population size and dimension, and validating these results experimentally.
Design of experiments, random search, initialization of population-based methods, or sampling inside an epoch of an evolutionary algorithm use a sample drawn according to some probability distribution for approximating the location of an optimum. Recent papers have shown that the optimal search distribution, used for the sampling, might be more peaked around the center of the distribution than the prior distribution modelling our uncertainty about the location of the optimum. We confirm this statement, provide explicit values for this reshaping of the search distribution depending on the population size $λ$ and the dimension $d$, and validate our results experimentally.