Text Prompt Injection of Vision Language Models
This addresses safety concerns for users of large vision language models, though it is incremental as it builds on existing attack methods.
The paper tackles the problem of misleading large vision language models through text prompt injection, demonstrating an effective and efficient attack algorithm that requires fewer computational resources than other methods.
The widespread application of large vision language models has significantly raised safety concerns. In this project, we investigate text prompt injection, a simple yet effective method to mislead these models. We developed an algorithm for this type of attack and demonstrated its effectiveness and efficiency through experiments. Compared to other attack methods, our approach is particularly effective for large models without high demand for computational resources.