Tensor-based Multi-layer Decoupling
This work addresses the unexplored problem of multi-layer decoupling for learning multivariate input-output relations, providing a new framework for system identification and neural network compression.
The paper introduces a tensor-based framework for multi-layer decoupling of multivariate functions, using ParaTuck-type decompositions and a bilevel optimization approach. The method is validated on synthetic systems, a nonlinear system identification benchmark, and neural network compression, demonstrating feasibility.
The decoupling of multivariate functions is a powerful modeling paradigm for learning multivariate input-output relations from data. For the single-layer case, established CPD-based methods are available, but the multi-layer case remained largely unexplored. This work introduces a tensor-based framework for multi-layer decoupling, which is based on ParaTuck-type tensor decompositions and constrained optimization. We provide theoretical justification behind the considered tensor decompositions and parameterizations. Furthermore, we formulate a structured coupled matrix-tensor factorization that incorporates both Jacobian and function evaluations, together with a bilevel optimization approach for adaptively balancing first- and zeroth-order information. The feasibility of the proposed methodology is illustrated on synthetic systems, a nonlinear system identification benchmark and neural network compression.