Edge-FIT: Federated Instruction Tuning of Quantized LLMs for Privacy-Preserving Smart Home Environments
This addresses the challenge of deploying LLMs in privacy-preserving smart home environments where traditional federated learning methods fail due to model size, though it appears to be an incremental improvement combining existing techniques.
The paper tackles the problem of deploying large language models in privacy-sensitive smart home environments by proposing Edge-FIT, a federated instruction tuning framework that combines federated learning with 4-bit quantization to reduce communication and computational overhead. Experimental results show the tuned Llama 2(7B) model achieves an F1-Score of 0.89 on an IoT-filtered dataset.
This paper proposes Edge-FIT (Federated Instruction Tuning on the Edge), a scalable framework for Federated Instruction Tuning (FIT) of Large Language Models (LLMs). Traditional Federated Learning (TFL) methods, like FedAvg, fail when confronted with the massive parameter size of LLMs [3], [6]. Our Edge-FIT framework combines federated learning with 4-bit Quantized Low-Rank Adaptation (QLORA), mitigating the core issues of communication and computational overhead. We demonstrate this by filtering the general-purpose Databricks Dolly 15k dataset for the IoT domain. Experimental results show the Edge-FIT tuned Llama 2(7B) achieves an F1-Score of 0.89. We also demonstrate a viable trade-off using the 3.8B Phi-3-mini model, validating Edge-FIT as a scalable framework for decentralized LLM deployment on home compute gateways.