Task-Oriented Convex Bilevel Optimization with Latent Feasibility
This work addresses optimization challenges in machine learning and computer vision, but it appears incremental as it builds on existing convex bilevel optimization methods by adding latent feasibility constraints.
The paper tackles the problem of optimizing learning and vision tasks by proposing a convex bilevel optimization paradigm that integrates task-oriented energy as a latent constraint, resulting in an efficient algorithmic framework with theoretical convergence analysis and experimental validation.
This paper firstly proposes a convex bilevel optimization paradigm to formulate and optimize popular learning and vision problems in real-world scenarios. Different from conventional approaches, which directly design their iteration schemes based on given problem formulation, we introduce a task-oriented energy as our latent constraint which integrates richer task information. By explicitly re-characterizing the feasibility, we establish an efficient and flexible algorithmic framework to tackle convex models with both shrunken solution space and powerful auxiliary (based on domain knowledge and data distribution of the task). In theory, we present the convergence analysis of our latent feasibility re-characterization based numerical strategy. We also analyze the stability of the theoretical convergence under computational error perturbation. Extensive numerical experiments are conducted to verify our theoretical findings and evaluate the practical performance of our method on different applications.