TLDR: A study evaluated GPT-4o’s ability to autonomously solve beginner-level cybersecurity challenges on OverTheWire’s Bandit capture-the-flag game. GPT-4o successfully solved 18 out of 25 compatible levels unaided and two more with minimal hints, achieving an 80% success rate. The model excelled at single-step tasks involving Linux navigation, data extraction, and basic networking, often completing them faster than humans. However, it struggled with multi-command scenarios requiring persistent working directories, complex network reconnaissance, or daemon creation, highlighting architectural limitations rather than a lack of knowledge. The research suggests LLMs can automate significant portions of novice penetration testing, potentially lowering entry barriers for attackers and aiding defenders.
Large Language Models (LLMs) have rapidly evolved, moving beyond traditional natural language processing to become a flagship technology in the pursuit of artificial general intelligence. Their training on vast internet data enables advanced language processing capabilities. This rapid development has naturally led to questions about their potential role in cybersecurity, particularly in assisting with or even completing cyber exploits.
A recent study delved into this very question, evaluating the capability of GPT-4o to autonomously solve beginner-level offensive security tasks. The researchers connected GPT-4o to OverTheWire’s Bandit capture-the-flag (CTF) game, a popular platform for cybersecurity novices, to assess its performance.
Methodology: Connecting AI to Cyber Challenges
The research team developed a Python script to facilitate communication between GPT-4o and the Bandit CTF challenges. GPT-4o was chosen for its public accessibility, high performance, and known general reasoning abilities, including cybersecurity concepts. The script used the Paramiko library for SSH connections, allowing GPT-4o to send Linux commands to the CTF server. A key limitation of this setup was that each command was executed in a new shell, meaning persistent working directories or multi-command sequences within a single session were not supported.
GPT-4o received initial prompts with challenge instructions and then received feedback on the output of its commands. The study categorized outcomes as “Solved” (unaided success), “Solved with Additional Assistance” (success after hints not originally in the CTF instructions), or “Unsolved.”
Key Findings: GPT-4o’s Performance
Out of 25 levels that were technically compatible with the single-command SSH framework, GPT-4o achieved an impressive 80% success rate. It solved 18 levels completely unaided and another two with minimal prompt hints. The total cost for all successful solutions was remarkably low, at just 0.002424 USD for 4,848 input tokens.
Where GPT-4o Excelled
The model demonstrated strong proficiency in several areas:
-
Linux Filesystem Navigation: GPT-4o quickly accessed flags, often in a single command, and showed strong efficacy in navigating complex Linux filesystems, even for files placed in obscure locations.
-
Data Extraction and Decoding: It successfully handled tasks involving searching files for specific terms, decoding Base64 blobs, and performing character value transformations.
-
Straightforward Networking: The LLM effectively used command-line networking utilities like `netcat` and `openssl` to retrieve flags from other servers.
-
Task Recognition: GPT-4o generally understood the objective of finding and displaying the “flag” (password) using commands like `cat`, even when not explicitly instructed.
-
Speed: When successful, the model often produced the correct command in one shot, completing tasks several times faster than a human expert.
For example, levels requiring intermediate to advanced Linux filesystem navigation (0-11, 17), networking utilities (13-15), and even complex bash scripting (level 24) were often solved with remarkable speed and efficiency. Levels 21 and 22, which involved parsing cron jobs, required a small hint about the correct filename, indicating a need for specific contextual understanding.
Challenges and Limitations
Despite its successes, GPT-4o encountered limitations, primarily due to architectural deficiencies rather than a lack of general exploit knowledge:
-
Persistent Working Directories: The single-command, new-shell setup hindered tasks requiring the model to maintain a working directory across multiple commands, such as repeatedly decompressing an archive (Level 12).
-
Complex Network Reconnaissance: GPT-4o struggled to interpret `nmap` scan results and connect to services based on that output (Level 16).
-
Daemon Creation: It failed to create a network daemon using `netcat` or utilize a pre-made tool for this purpose (Level 20).
-
Script Creation: The model had difficulty understanding the need to create new bash scripts in specific locations (Level 23).
-
Non-Standard Shells: The final level (32), which uppercased every command, completely stumped GPT-4o, as it did not attempt the correct solution and entered a loop of navigation commands.
These unsuccessful challenges often required multiple sequential commands, highlighting the model’s struggle with multi-step logic when state persistence was an issue.
Also Read:
- Leading AI Agents Vulnerable: Security Flaws Exposed in Major Red Teaming Competition
- AI’s Growing Prowess in Smart Contract Exploits: Understanding the Threat and Defense
Implications for Cybersecurity
The study concludes that GPT-4o’s high success rate demonstrates a strong aptitude for completing offensive cybersecurity challenges. Its ability to generate successful commands far quicker than a human agent suggests significant utility for attackers, potentially lowering the expertise barrier for malicious actors. For defenders, LLMs could serve as rapid reconnaissance aids, offering productivity gains.
The unsolved tasks also provide valuable insights, revealing specific areas where secure-by-design environments might frustrate simple LLM-driven attacks, thus informing future hardening strategies. Beyond offensive applications, the results suggest integrating LLMs into cybersecurity education as practice aids, though caution is advised to prevent over-reliance that could undermine skill development.
For more detailed information, you can read the full research paper available here.
