A mini-batch training strategy for deep subspace clustering networks
This work addresses scalability issues for researchers and practitioners in subspace clustering, enabling training on high-resolution images, though it is incremental as it builds on existing deep subspace clustering frameworks.
The paper tackles the computational bottleneck in deep subspace clustering by introducing a mini-batch training strategy with a memory bank, achieving performance comparable to full-batch methods and outperforming state-of-the-art methods on COIL100 and ORL datasets.
Mini-batch training is a cornerstone of modern deep learning, offering computational efficiency and scalability for training complex architectures. However, existing deep subspace clustering (DSC) methods, which typically combine an autoencoder with a self-expressive layer, rely on full-batch processing. The bottleneck arises from the self-expressive module, which requires representations of the entire dataset to construct a self-representation coefficient matrix. In this work, we introduce a mini-batch training strategy for DSC by integrating a memory bank that preserves global feature representations. Our approach enables scalable training of deep architectures for subspace clustering with high-resolution images, overcoming previous limitations. Additionally, to efficiently fine-tune large-scale pre-trained encoders for subspace clustering, we propose a decoder-free framework that leverages contrastive learning instead of autoencoding for representation learning. This design not only eliminates the computational overhead of decoder training but also provides competitive performance. Extensive experiments demonstrate that our approach not only achieves performance comparable to full-batch methods, but outperforms other state-of-the-art subspace clustering methods on the COIL100 and ORL datasets by fine-tuning deep networks.