Incorporating domain knowledge into neural-guided search
This work addresses the challenge of improving search efficiency in AutoML for researchers and practitioners, but it is incremental as it builds on existing neural-guided search methods.
The authors tackled the problem of optimizing discrete objects in AutoML by formalizing a framework to incorporate domain knowledge into neural-guided search, demonstrating its efficacy in symbolic regression.
Many AutoML problems involve optimizing discrete objects under a black-box reward. Neural-guided search provides a flexible means of searching these combinatorial spaces using an autoregressive recurrent neural network. A major benefit of this approach is that builds up objects sequentially--this provides an opportunity to incorporate domain knowledge into the search by directly modifying the logits emitted during sampling. In this work, we formalize a framework for incorporating such in situ priors and constraints into neural-guided search, and provide sufficient conditions for enforcing constraints. We integrate several priors and constraints from existing works into this framework, propose several new ones, and demonstrate their efficacy in informing the task of symbolic regression.