Modeling Hierarchical Spaces: A Review and Unified Framework for Surrogate-Based Architecture Design
This work addresses a domain-specific problem for researchers and practitioners in simulation-based optimization, particularly in complex system design, but it is incremental as it builds on and unifies prior literature.
The paper tackles the challenge of modeling and optimizing hierarchical, conditional, and mixed-variable input spaces in simulation-based problems by proposing a unified framework that generalizes existing approaches, demonstrating its effectiveness on complex system design cases like neural networks and green-aircraft.
Simulation-based problems involving mixed-variable inputs frequently feature domains that are hierarchical, conditional, heterogeneous, or tree-structured. These characteristics pose challenges for data representation, modeling, and optimization. This paper reviews extensive literature on these structured input spaces and proposes a unified framework that generalizes existing approaches. In this framework, input variables may be continuous, integer, or categorical. A variable is described as meta if its value governs the presence of other decreed variables, enabling the modeling of conditional and hierarchical structures. We further introduce the concept of partially-decreed variables, whose activation depends on contextual conditions. To capture these inter-variable hierarchical relationships, we introduce design space graphs, combining principles from feature modeling and graph theory. This allows the definition of general hierarchical domains suitable for describing complex system architectures. Our framework defines hierarchical distances and kernels to enable surrogate modeling and optimization on hierarchical domains. We demonstrate its effectiveness on complex system design problems, including a neural network and a green-aircraft case study. Our methods are available in the open-source Surrogate Modeling Toolbox (SMT 2.0).