Latency-Aware Neural Architecture Search with Multi-Objective Bayesian Optimization
This work addresses the challenge of balancing performance and efficiency for on-device AI models, which is incremental as it applies existing methods to a specific production scenario.
The paper tackled the problem of optimizing trade-offs between latency and accuracy for on-device deployment of large machine learning models, using Bayesian optimization to efficiently explore these trade-offs for a production-scale natural language understanding model at Facebook.
When tuning the architecture and hyperparameters of large machine learning models for on-device deployment, it is desirable to understand the optimal trade-offs between on-device latency and model accuracy. In this work, we leverage recent methodological advances in Bayesian optimization over high-dimensional search spaces and multi-objective Bayesian optimization to efficiently explore these trade-offs for a production-scale on-device natural language understanding model at Facebook.