SegReg: Latent Space Regularization for Improved Medical Image Segmentation
This work addresses the problem of limited generalization and task drift in continual learning for medical image segmentation models by regularizing latent feature representations.
This paper introduces SegReg, a latent-space regularization framework for medical image segmentation models that encourages structured embeddings. It demonstrates consistent improvements in domain generalization across prostate, cardiac, and hippocampus segmentation, and enhances continual learning by reducing task drift and improving forward transfer.
Medical image segmentation models are typically optimised with voxel-wise losses that constrain predictions only in the output space. This leaves latent feature representations largely unconstrained, potentially limiting generalisation. We propose {SegReg}, a latent-space regularisation framework that operates on feature maps of U-Net models to encourage structured embeddings while remaining fully compatible with standard segmentation losses. Integrated with the nnU-Net framework, we evaluate SegReg on prostate, cardiac, and hippocampus segmentation and demonstrate consistent improvements in domain generalisation. Furthermore, we show that explicit latent regularisation improves continual learning by reducing task drift and enhancing forward transfer across sequential tasks without adding memory or any extra parameters. These results highlight latent-space regularisation as a practical approach for building more generalisable and continual-learning-ready models.