Task Embedded Coordinate Update: A Realizable Framework for Multivariate Non-convex Optimization
This work addresses optimization challenges in non-convex problems for machine learning and computational tasks, though it appears incremental as it builds on existing coordinate descent methods.
The authors proposed TECU, a framework that embeds task-specific strategies into coordinate descent for optimizing multivariate non-convex problems with coupled objectives, improving algorithm efficiency and result quality, as verified by experiments embedding ADMM and a residual-type CNN.
We in this paper propose a realizable framework TECU, which embeds task-specific strategies into update schemes of coordinate descent, for optimizing multivariate non-convex problems with coupled objective functions. On one hand, TECU is capable of improving algorithm efficiencies through embedding productive numerical algorithms, for optimizing univariate sub-problems with nice properties. From the other side, it also augments probabilities to receive desired results, by embedding advanced techniques in optimizations of realistic tasks. Integrating both numerical algorithms and advanced techniques together, TECU is proposed in a unified framework for solving a class of non-convex problems. Although the task embedded strategies bring inaccuracies in sub-problem optimizations, we provide a realizable criterion to control the errors, meanwhile, to ensure robust performances with rigid theoretical analyses. By respectively embedding ADMM and a residual-type CNN in our algorithm framework, the experimental results verify both efficiency and effectiveness of embedding task-oriented strategies in coordinate descent for solving practical problems.