Tree-CNN: A Hierarchical Deep Convolutional Neural Network for Incremental Learning
This addresses the challenge of incremental learning for computer vision models, enabling adaptation to evolving data without retraining from scratch, though it is incremental as it builds on existing hierarchical CNN models.
The paper tackles the problem of catastrophic forgetting in deep convolutional neural networks when adapting to new data, proposing a hierarchical tree-like structure that grows incrementally to accommodate new classes while preserving old ones, achieving competitive accuracy on CIFAR-10 and CIFAR-100 with reduced training effort.
Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, such as "catastrophic forgetting", and sensitivity to hyper-parameter tuning. However, in this modern world, data is constantly evolving, and our deep learning models are required to adapt to these changes. In this paper, we propose an adaptive hierarchical network structure composed of DCNNs that can grow and learn as new data becomes available. The network grows in a tree-like fashion to accommodate new classes of data, while preserving the ability to distinguish the previously trained classes. The network organizes the incrementally available data into feature-driven super-classes and improves upon existing hierarchical CNN models by adding the capability of self-growth. The proposed hierarchical model, when compared against fine-tuning a deep network, achieves significant reduction of training effort, while maintaining competitive accuracy on CIFAR-10 and CIFAR-100.