Knowledge Distillation for Reservoir-based Classifier: Human Activity Recognition
This provides an incremental improvement for real-time and energy-efficient human activity recognition in edge computing environments.
The paper tackled the problem of energy-efficient classification for human activity recognition from time-series sensor data by introducing PatchEchoClassifier, a reservoir-based model using knowledge distillation, achieving over 80% accuracy and reducing computational cost to about one-sixth of the FLOPS compared to a baseline.
This paper aims to develop an energy-efficient classifier for time-series data by introducing PatchEchoClassifier, a novel model that leverages a reservoir-based mechanism known as the Echo State Network (ESN). The model is designed for human activity recognition (HAR) using one-dimensional sensor signals and incorporates a tokenizer to extract patch-level representations. To train the model efficiently, we propose a knowledge distillation framework that transfers knowledge from a high-capacity MLP-Mixer teacher to the lightweight reservoir-based student model. Experimental evaluations on multiple HAR datasets demonstrate that our model achieves over 80 percent accuracy while significantly reducing computational cost. Notably, PatchEchoClassifier requires only about one-sixth of the floating point operations (FLOPS) compared to DeepConvLSTM, a widely used convolutional baseline. These results suggest that PatchEchoClassifier is a promising solution for real-time and energy-efficient human activity recognition in edge computing environments.