Per-Domain Generalizing Policies: On Validation Instances and Scaling Behavior
This work addresses scaling challenges in reinforcement learning for domain generalization, offering incremental improvements over prior fixed validation set approaches.
The paper tackles the problem of scaling per-domain generalizing action policies from small training instances to large test instances by introducing a dynamic validation set generation method that increases instance size while remaining informative and feasible, achieving improved scaling behavior in all 9 domains tested.
Recent work has shown that successful per-domain generalizing action policies can be learned. Scaling behavior, from small training instances to large test instances, is the key objective; and the use of validation instances larger than training instances is one key to achieve it. Prior work has used fixed validation sets. Here, we introduce a method generating the validation set dynamically, on the fly, increasing instance size so long as informative and feasible.We also introduce refined methodology for evaluating scaling behavior, generating test instances systematically to guarantee a given confidence in coverage performance for each instance size. In experiments, dynamic validation improves scaling behavior of GNN policies in all 9 domains used.