Blue Teaming Function-Calling Agents
This addresses security vulnerabilities in function-calling LLMs for developers and users, but it is incremental as it focuses on experimental evaluation of existing models and defenses.
The paper evaluated the robustness of four open-source LLMs with function-calling capabilities against three attacks and tested eight defenses, finding that the models are not safe by default and the defenses are not yet practical for real-world use.
We present an experimental evaluation that assesses the robustness of four open source LLMs claiming function-calling capabilities against three different attacks, and we measure the effectiveness of eight different defences. Our results show how these models are not safe by default, and how the defences are not yet employable in real-world scenarios.