CQnet: convex-geometric interpretation and constraining neural-network trajectories
This addresses the need for interpretable and stable neural networks in tasks with prior knowledge, though it appears incremental as it builds on existing convex optimization methods.
The paper tackles the problem of making neural network trajectories interpretable and stable by introducing CQnet, which constrains every layer and output to satisfy learned or deterministic constraints, with proof of stability under minimal assumptions.
We introduce CQnet, a neural network with origins in the CQ algorithm for solving convex split-feasibility problems and forward-backward splitting. CQnet's trajectories are interpretable as particles that are tracking a changing constraint set via its point-to-set distance function while being elements of another constraint set at every layer. More than just a convex-geometric interpretation, CQnet accommodates learned and deterministic constraints that may be sample or data-specific and are satisfied by every layer and the output. Furthermore, the states in CQnet progress toward another constraint set at every layer. We provide proof of stability/nonexpansiveness with minimal assumptions. The combination of constraint handling and stability put forward CQnet as a candidate for various tasks where prior knowledge exists on the network states or output.