CANDID DAC: Leveraging Coupled Action Dimensions with Importance Differences in DAC
This work addresses a specific bottleneck in DAC for researchers, but it is incremental as it builds on existing DACBench frameworks and methods.
The paper tackles the challenge of high-dimensional action spaces in dynamic algorithm configuration (DAC) by introducing a new benchmark that simulates coupled action dimensions with importance differences (CANDID) and proposing sequential policies that factorize the action space to manage these properties. The result shows that sequential policies significantly outperform independent learning of factorized policies in CANDID action spaces and overcome scalability limitations.
High-dimensional action spaces remain a challenge for dynamic algorithm configuration (DAC). Interdependencies and varying importance between action dimensions are further known key characteristics of DAC problems. We argue that these Coupled Action Dimensions with Importance Differences (CANDID) represent aspects of the DAC problem that are not yet fully explored. To address this gap, we introduce a new white-box benchmark within the DACBench suite that simulates the properties of CANDID. Further, we propose sequential policies as an effective strategy for managing these properties. Such policies factorize the action space and mitigate exponential growth by learning a policy per action dimension. At the same time, these policies accommodate the interdependence of action dimensions by fostering implicit coordination. We show this in an experimental study of value-based policies on our new benchmark. This study demonstrates that sequential policies significantly outperform independent learning of factorized policies in CANDID action spaces. In addition, they overcome the scalability limitations associated with learning a single policy across all action dimensions. The code used for our experiments is available under https://github.com/PhilippBordne/candidDAC.