Real-Time Black-Box Optimization for Dynamic Discrete Environments Using Embedded Ising Machines
This addresses the challenge of real-time optimization in dynamic, discrete systems like wireless networks, but it is incremental as it extends an existing black-box optimization method.
The paper tackled the problem of optimizing discrete variables in dynamic environments, where conventional multi-armed bandit algorithms fail due to combinatorial complexity, by proposing a heuristic method using an Ising machine; it demonstrated dynamic adaptability in a wireless communication system with moving users.
Many real-time systems require the optimization of discrete variables. Black-box optimization (BBO) algorithms and multi-armed bandit (MAB) algorithms perform optimization by repeatedly taking actions and observing the corresponding instant rewards without any prior knowledge. Recently, a BBO method using an Ising machine has been proposed to find the best action that is represented by a combination of discrete values and maximizes the instant reward in static environments. In contrast, dynamic environments, where real-time systems operate, necessitate MAB algorithms that maximize the average reward over multiple trials. However, due to the enormous number of actions resulting from the combinatorial nature of discrete optimization, conventional MAB algorithms cannot effectively optimize dynamic, discrete environments. Here, we show a heuristic MAB method for dynamic, discrete environments by extending the BBO method, in which an Ising machine effectively explores the actions while considering interactions between variables and changes in dynamic environments. We demonstrate the dynamic adaptability of the proposed method in a wireless communication system with moving users.