Neural collapse with unconstrained features
This work provides a theoretical explanation for the emergence of neural collapse, which is an incremental step towards understanding deep learning phenomena for researchers.
This paper proposes an "unconstrained features model" that empirically exhibits neural collapse, a phenomenon in deep learning. By studying this simplified model, the authors offer an explanation for neural collapse based on the landscape of empirical risk.
Neural collapse is an emergent phenomenon in deep learning that was recently discovered by Papyan, Han and Donoho. We propose a simple "unconstrained features model" in which neural collapse also emerges empirically. By studying this model, we provide some explanation for the emergence of neural collapse in terms of the landscape of empirical risk.