Efficient Decoder Scaling Strategy for Neural Routing Solvers
This work addresses the problem of efficiently scaling neural routing solvers for vehicle routing problems, providing design principles for parameter allocation, though it is incremental as it builds on existing construction-based approaches.
The study tackled the unexplored effects of scaling decoder parameters in neural routing solvers by comparing depth versus width scaling across 12 models from 1M to 150M parameters, finding that depth scaling yields superior performance gains and that parameter count alone is insufficient to predict model performance.
Construction-based neural routing solvers, typically composed of an encoder and a decoder, have emerged as a promising approach for solving vehicle routing problems. While recent studies suggest that shifting parameters from the encoder to the decoder enhances performance, most works restrict the decoder size to 1-3M parameters, leaving the effects of scaling largely unexplored. To address this gap, we conduct a systematic study comparing two distinct strategies: scaling depth versus scaling width. We synthesize these strategies to construct a suite of 12 model configurations, spanning a parameter range from 1M to ~150M, and extensively evaluate their scaling behaviors across three critical dimensions: parameter efficiency, data efficiency, and compute efficiency. Our empirical results reveal that parameter count is insufficient to accurately predict the model performance, highlighting the critical and distinct roles of model depth (layer count) and width (embedding dimension). Crucially, we demonstrate that scaling depth yields superior performance gains to scaling width. Based on these findings, we provide and experimentally validate a set of design principles for the efficient allocation of parameters and compute resources to enhance the model performance.