Compositional ADAM: An Adaptive Compositional Solver
This provides a novel solver for compositional optimization, bridging it with model-agnostic meta-learning for faster deep network adaptation, though it appears incremental in the broader optimization field.
The paper tackles compositional optimization problems involving nested expected values by introducing C-ADAM, the first adaptive solver for such problems, which converges to a stationary point in O(δ^{-2.25}) and demonstrates significant sample complexity reductions in experiments.
In this paper, we present C-ADAM, the first adaptive solver for compositional problems involving a non-linear functional nesting of expected values. We proof that C-ADAM converges to a stationary point in $\mathcal{O}(δ^{-2.25})$ with $δ$ being a precision parameter. Moreover, we demonstrate the importance of our results by bridging, for the first time, model-agnostic meta-learning (MAML) and compositional optimisation showing fastest known rates for deep network adaptation to-date. Finally, we validate our findings in a set of experiments from portfolio optimisation and meta-learning. Our results manifest significant sample complexity reductions compared to both standard and compositional solvers.