Knowledge Distillation for LLM-Based Human Activity Recognition in Homes
This work addresses the challenge of deploying efficient LLMs for human activity recognition in smart homes, but it is incremental as it builds on recent studies by focusing on model size and distillation techniques.
The paper tackles the problem of using large language models (LLMs) for human activity recognition in homes by exploring how model size affects performance and applying knowledge distillation to fine-tune smaller models. The result shows that fine-tuned smaller models achieve nearly the same performance as the largest LLMs while having 50 times fewer parameters.
Human Activity Recognition (HAR) is a central problem for context-aware applications, especially for smart homes and assisted living. A few very recent studies have shown that Large Language Models (LLMs) can be used for HAR at home, reaching high performance and addressing key challenges. In this paper, we provide new experimental results regarding the use of LLMs for HAR, on two state-of-the-art datasets. More specifically, we show how recognition performance evolves depending on the size of the LLM used. Moreover, we experiment on the use of knowledge distillation techniques to fine-tune smaller LLMs with HAR reasoning examples generated by larger LLMs. We show that such fine-tuned models can perform almost as well as the largest LLMs, while having 50 times less parameters.