PEFT-MedAware: Large Language Model for Medical Awareness
This work addresses the need for accurate and resource-efficient medical AI models, suitable for deployment in resource-constrained environments, but it is incremental as it builds on existing PEFT and LLM methods.
The researchers tackled the problem of uncertain accuracy in chat models for medical questions by fine-tuning the Falcon-1b LLM on MedQuAD data using parameter-efficient fine-tuning (PEFT), achieving greater accuracy in medical QA tasks with only 0.44% of trainable parameters.
Chat models are capable of answering a wide range of questions, however, the accuracy of their responses is highly uncertain. In this research, we propose a specialized PEFT-MedAware model where we utilize parameter-efficient fine-tuning (PEFT) to enhance the Falcon-1b large language model on specialized MedQuAD data consisting of 16,407 medical QA pairs, leveraging only 0.44% of its trainable parameters to enhance computational efficiency. The paper adopts data preprocessing and PEFT to optimize model performance, complemented by a BitsAndBytesConfig for efficient transformer training. The resulting model was capable of outperforming other LLMs in medical question-answering tasks in specific domains with greater accuracy utilizing limited computational resources making it suitable for deployment in resource-constrained environments. We propose further improvements through expanded datasets, larger models, and feedback mechanisms for sustained medical relevancy. Our work highlights the efficiency gains and specialized capabilities of PEFT in medical AI, outpacing standard models in precision without extensive resource demands. The proposed model and data are released for research purposes only.