TLDR: This research introduces an automated system using AI agents to analyze the vast amounts of log data generated by honeypots like Cowrie. It efficiently parses, summarizes, and extracts attack patterns, significantly reducing manual effort and providing valuable threat intelligence. The system identifies common attack intents like “Shallow Probe” and “Malware Deployment” from mostly low-skill, automated attacks, and plans for future integration with large language models for enhanced analysis.
Cybersecurity researchers and educators often face a significant challenge: the scarcity of real-world attack data. While honeypots, like the popular Cowrie system, are excellent at collecting live threat intelligence, they generate an overwhelming volume of unstructured and diverse log data. Manually sifting through hundreds of thousands of log entries daily to identify attack patterns and attacker tactics, techniques, and procedures (TTPs) is simply not practical.
A recent study, “Towards Log Analysis with AI Agents: Cowrie Case Study” by Enis Karaarslan, Esin G ¨uler, Efe Emir Y ¨uce, and C ¸ a˘gatay C ¸ oban, addresses this very problem. The researchers propose a lightweight and automated approach that leverages AI agents to intelligently parse, summarize, and extract crucial insights from raw Cowrie honeypot logs. This innovative system aims to reduce manual effort and identify attack patterns, paving the way for more advanced autonomous cybersecurity analysis.
Understanding the Core Components
At the heart of this system are two key concepts: honeypots and AI agents.
A honeypot is essentially a decoy system designed to attract and trap cyber attackers. It mimics a legitimate target, allowing security professionals to observe and collect data on malicious activities without risking real systems. Cowrie, specifically, is a medium-interaction SSH and Telnet honeypot that logs brute-force attacks and, more importantly, the shell interactions an attacker performs after gaining access. This data is outputted into structured JSON files.
AI agents are autonomous systems that can perform tasks by generating and executing their own action plans. They use advanced natural language processing (NLP) to understand complex goals, break them down into steps, and utilize external tools to achieve their objectives. Unlike traditional AI systems that operate within predefined parameters, AI agents demonstrate greater autonomy, especially with the advancements in large language models (LLMs) which serve as their ‘brain’, enhanced with memory, planning, and tool-use capabilities.
The Automated Analysis Pipeline
The proposed model is built around the Cowrie SSH Honeypot and deploys an agentic architecture with two main goals: autonomous system hardening and automated attack pattern (TTP) extraction. The system follows a clear, four-step process:
1. Ingestion: The process begins by discovering and reading all Cowrie JSON log files from a specified directory, consolidating all events into a single data structure.
2. Processing: Using the pandas library, the raw log events are converted into a structured DataFrame. This data is then filtered to isolate command-input events and grouped by unique session IDs, transforming thousands of individual log lines into coherent attacker sessions, each with a source IP and a chronological list of executed commands.
3. Analysis: This is the core of the system. A rule-based engine iterates through the commands of each session, using a predefined dictionary of keywords (e.g., ‘wget’, ‘ls’, ‘rm’) to score sessions across categories like ‘Reconnaissance’ or ‘Malware Deployment’. Based on these scores, heuristics classify the session’s primary intent and the attacker’s estimated skill level. Suspicious artifacts, such as URLs, are also extracted for reporting.
4. Reporting & Visualization: The analysis results are compiled into a final DataFrame, which is then used to generate a .csv file for data portability, an .html file for easy viewing, and .png image files containing summary visualizations (e.g., bar charts for intents and pie charts for skill levels).
Implementation and Key Findings
The system was implemented entirely in Python 3, leveraging open-source libraries like Pandas for data manipulation and Matplotlib & Seaborn for visualizations. A prototype successfully processed over 300,000 log events in just a few minutes on a standard machine, demonstrating its efficiency.
The evaluation, using a dataset of 313,412 log events, identified and grouped 26,368 unique attacker sessions. The rule-based engine effectively classified the vast majority of these sessions. The primary findings revealed that most attacker sessions were classified as “Shallow Probe” – representing bots or attackers performing minimal reconnaissance before disconnecting. The second most common intent was “Malware Deployment,” often involving cryptocurrency miners or DDoS bots. This indicates that the honeypot primarily attracts automated, opportunistic attacks by “Low (Script Kiddie)” or “Medium (Automated Script)” skill levels, rather than sophisticated manual operations.
Also Read:
- Neuro-Symbolic AI: Bridging Intelligence Gaps in Cybersecurity
- AI-Powered Cyber Defense: Accelerating Reinforcement Learning with Large Language Models
Future Directions
This study successfully demonstrated a lightweight, transparent, and extensible process for deriving meaningful threat intelligence from Cowrie honeypot data. For future work, the researchers plan several enhancements, including making the rule-based engine more sophisticated with complex TTP signatures. Most significantly, they aim to integrate the system with Large Language Models (e.g., Google Gemini) via frameworks like LangChain. This would allow deployed agents to perform real-time threat intelligence enrichment by querying APIs like AbuseIPDB for IP reputation or VirusTotal for URL analysis, providing a much deeper contextual understanding of each attack.
For more detailed information, you can read the full research paper here: Towards Log Analysis with AI Agents: Cowrie Case Study.
