Condensing Action Segmentation Datasets via Generative Network Inversion
This addresses storage and efficiency challenges for researchers and practitioners working with large video datasets in temporal action segmentation, representing a novel application rather than an incremental improvement.
This paper tackles the problem of storage-intensive procedural video datasets for temporal action segmentation by introducing the first dataset condensation approach for this domain, achieving over 500× storage reduction on the Breakfast dataset while retaining 83% of performance compared to using the full dataset.
This work presents the first condensation approach for procedural video datasets used in temporal action segmentation. We propose a condensation framework that leverages generative prior learned from the dataset and network inversion to condense data into compact latent codes with significant storage reduced across temporal and channel aspects. Orthogonally, we propose sampling diverse and representative action sequences to minimize video-wise redundancy. Our evaluation on standard benchmarks demonstrates consistent effectiveness in condensing TAS datasets and achieving competitive performances. Specifically, on the Breakfast dataset, our approach reduces storage by over 500$\times$ while retaining 83% of the performance compared to training with the full dataset. Furthermore, when applied to a downstream incremental learning task, it yields superior performance compared to the state-of-the-art.