Medical Foundation Models are Susceptible to Targeted Misinformation Attacks
This highlights a critical security and trustworthiness issue for LLMs in healthcare, requiring robust protective measures to ensure safe use.
The study demonstrated that large language models (LLMs) in medicine are vulnerable to targeted misinformation attacks, where manipulating just 1.1% of the model's weights can inject incorrect biomedical facts, validated on 1,038 facts, while other tasks remain unaffected.
Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains, holding promising potential for diverse medical applications in the near future. In this study, we demonstrate a concerning vulnerability of LLMs in medicine. Through targeted manipulation of just 1.1% of the model's weights, we can deliberately inject an incorrect biomedical fact. The erroneous information is then propagated in the model's output, whilst its performance on other biomedical tasks remains intact. We validate our findings in a set of 1,038 incorrect biomedical facts. This peculiar susceptibility raises serious security and trustworthiness concerns for the application of LLMs in healthcare settings. It accentuates the need for robust protective measures, thorough verification mechanisms, and stringent management of access to these models, ensuring their reliable and safe use in medical practice.