CyberLLMInstruct: A Pseudo-malicious Dataset Revealing Safety-performance Trade-offs in Cyber Security LLM Fine-tuning
This addresses the critical safety risks for cyber security applications using LLMs, highlighting a significant trade-off that is incremental to existing fine-tuning challenges.
The paper tackles the safety-performance trade-off in fine-tuning large language models for cyber security by introducing CyberLLMInstruct, a dataset of 54,928 pseudo-malicious instruction-response pairs, and finds that fine-tuning improves task accuracy up to 92.50% but severely reduces safety resilience, with security scores dropping from 0.95 to 0.15 in some cases.
The integration of large language models (LLMs) into cyber security applications presents both opportunities and critical safety risks. We introduce CyberLLMInstruct, a dataset of 54,928 pseudo-malicious instruction-response pairs spanning cyber security tasks including malware analysis, phishing simulations, and zero-day vulnerabilities. Our comprehensive evaluation using seven open-source LLMs reveals a critical trade-off: while fine-tuning improves cyber security task performance (achieving up to 92.50% accuracy on CyberMetric), it severely compromises safety resilience across all tested models and attack vectors (e.g., Llama 3.1 8B's security score against prompt injection drops from 0.95 to 0.15). The dataset incorporates diverse sources including CTF challenges, academic papers, industry reports, and CVE databases to ensure comprehensive coverage of cyber security domains. Our findings highlight the unique challenges of securing LLMs in adversarial domains and establish the critical need for developing fine-tuning methodologies that balance performance gains with safety preservation in security-sensitive domains.