ARACNE: An LLM-Based Autonomous Shell Pentesting Agent
This addresses the need for automated security testing in cybersecurity, though it appears incremental as it builds on existing LLM-based approaches with multi-model support.
The researchers tackled the problem of autonomous penetration testing for SSH services by developing ARACNE, an LLM-based agent that achieved a 60% success rate against an autonomous defender and 57.58% against CTF challenges, with an average of fewer than 5 actions per successful attempt.
We introduce ARACNE, a fully autonomous LLM-based pentesting agent tailored for SSH services that can execute commands on real Linux shell systems. Introduces a new agent architecture with multi-LLM model support. Experiments show that ARACNE can reach a 60\% success rate against the autonomous defender ShelLM and a 57.58\% success rate against the Over The Wire Bandit CTF challenges, improving over the state-of-the-art. When winning, the average number of actions taken by the agent to accomplish the goals was less than 5. The results show that the use of multi-LLM is a promising approach to increase accuracy in the actions.