A Cross Entropy based Optimization Algorithm with Global Convergence Guarantees
This work addresses computational bottlenecks in optimization algorithms for researchers and practitioners dealing with global optimization problems, though it appears incremental as it builds on the existing cross entropy method.
The authors tackled the inefficiency of the Monte-Carlo cross entropy method by introducing a novel stochastic approximation version that uses incremental geometric averaging, resulting in reduced computational and storage costs while maintaining accuracy and global convergence for certain objective functions.
The cross entropy (CE) method is a model based search method to solve optimization problems where the objective function has minimal structure. The Monte-Carlo version of the CE method employs the naive sample averaging technique which is inefficient, both computationally and space wise. We provide a novel stochastic approximation version of the CE method, where the sample averaging is replaced with incremental geometric averaging. This approach can save considerable computational and storage costs. Our algorithm is incremental in nature and possesses additional attractive features such as accuracy, stability, robustness and convergence to the global optimum for a particular class of objective functions. We evaluate the algorithm on a variety of global optimization benchmark problems and the results obtained corroborate our theoretical findings.