Dictionary-Learning-Based Data Pruning for System Identification
This work addresses data efficiency in system identification for engineering applications, but it is incremental as it builds on existing dictionary learning techniques.
The paper tackles sample-wise redundancy in system identification by proposing a dictionary-learning-based data pruning method called mini-batch FastCan, which selects representative samples to reduce data size. It shows that this method significantly outperforms random pruning on simulated and benchmark datasets, as measured by R-squared values between model coefficients from full and pruned datasets.
System identification is normally involved in augmenting time series data by time shifting and nonlinearisation (e.g., polynomial basis), both of which introduce redundancy in features and samples. Many research works focus on reducing redundancy feature-wise, while less attention is paid to sample-wise redundancy. This paper proposes a novel data pruning method, called mini-batch FastCan, to reduce sample-wise redundancy based on dictionary learning. Time series data is represented by some representative samples, called atoms, via dictionary learning. The useful samples are selected based on their correlation with the atoms. The method is tested on one simulated dataset and two benchmark datasets. The R-squared between the coefficients of models trained on the full datasets and the coefficients of models trained on pruned datasets is adopted to evaluate the performance of data pruning methods. It is found that the proposed method significantly outperforms the random pruning method.