TLDR: A new study investigates how Large Language Models (LLMs) behave in game-theoretic cybersecurity scenarios. Researchers found that LLMs’ decisions in games like the zero-sum and Prisoner’s Dilemma are significantly influenced by factors such as the language used for prompting and assigned personality traits, often leading to inconsistent and unpredictable outcomes across different models and languages. This highlights the need for careful testing and bias awareness before deploying LLMs in critical cybersecurity applications.
Cybersecurity is a critical field, with the cost of cyber threats projected to reach $10 trillion annually in the coming years. To combat this, researchers and practitioners often turn to game theory, a mathematical framework that helps model strategic interactions between attackers and defenders. It allows us to understand how different actions by both sides can lead to various outcomes, helping design better defense mechanisms.
The rise of Large Language Models, or LLMs, has introduced a new dimension to this landscape. These powerful AI applications are being explored for their potential in cybersecurity, both for generating realistic attack scenarios and for acting as agents within these scenarios. However, a crucial question arises: how do LLMs actually behave when placed in game-theoretic situations? Do they follow predictable patterns, or do they exhibit unexpected behaviors due to their inherent complexities and biases?
Investigating LLM Behavior in Strategic Games
A recent study, titled “Can LLMs effectively provide game-theoretic-based scenarios for cybersecurity?”, delves into this very question. The researchers, Daniele Proverbio, Alessio Buscemi, Alessandro Di Stefano, The Anh Han, German Castignani, and Pietro Li`o, investigated whether classical game theory frameworks can accurately capture the actions of LLM-driven agents and bots. They used a reproducible framework called FAIRGAME to simulate two well-known game theory scenarios relevant to cybersecurity:
- The one-shot zero-sum game: This game models situations where one player’s gain is exactly another’s loss, like jamming or denial-of-service (DoS) attacks.
- The dynamic Prisoner’s Dilemma: This classic game explores cooperation versus defection, often used to model selfishness in networks or nation-level cyber intrusions.
The study involved four state-of-the-art LLMs: GPT-4 by OpenAI, Gemini Pro 1.5 by DeepMind, Mistral Large by Mistral AI, and Llama 3.1 405b by Meta. To assess linguistic sensitivity and other biases, the experiments were conducted in five natural languages: English, French, Arabic, Vietnamese, and Mandarin Chinese. They also tested how different “personalities” (cooperative or selfish) assigned to the LLM agents, and whether agents knew the total number of rounds in the Prisoner’s Dilemma, influenced their decisions.
Key Findings: Surprising Sensitivities
The results revealed several interesting insights. In the zero-sum game, the LLMs’ behavior was significantly influenced by the assigned personalities. For instance, when both agents were set to be cooperative, they tended to make choices that resulted in negative payoffs for Agent 1, suggesting they often chose different options rather than the same one. More strikingly, the choices were not consistent across different LLMs or even across different languages for the same LLM. This suggests that LLMs can be highly sensitive to the language used for prompting, which could have significant implications for cybersecurity applications deployed globally.
For the repeated Prisoner’s Dilemma, LLM agents generally tended to defect, leading to higher penalties, which aligns with the theoretical Nash equilibrium for this game. However, there were dramatic inconsistencies across languages and personality combinations. Some LLMs, like Llama 3 and GPT-4o, showed more stable behavior across languages, while others, such as Claude and Mistral, were highly sensitive to the language choice. For example, Claude 3.5 agents set to be cooperative showed the lowest penalties (more cooperation) in English but the highest penalties in other languages.
The study also observed that equipping agents with personalities influenced their strategies, sometimes in unexpected ways. For instance, selfish-selfish Mistral Large players had lower penalties than cooperative-cooperative players, while the opposite was true for Llama players under certain conditions. Over multiple rounds, most LLMs showed an increasing trend towards mutual cooperation, consistent with traditional game theory predictions for repeated interactions, though GPT-4o showed a different pattern, starting cooperative and then increasing penalties.
Also Read:
- The Hidden Truth: LLMs Deceive Even Without Prompts
- Assessing LLM Vulnerability: A New Look at AI Robustness
Implications for Cybersecurity
This research highlights that while generative AI holds great promise for enhancing cybersecurity applications, LLMs currently exhibit notable limitations. Their decision-making can be significantly affected by factors like the language used for prompting, the assigned digital personality, and the completeness of information. This means that LLM agents may behave sub-optimally or unpredictably in real-world cybersecurity scenarios, deviating from traditional game-theoretic predictions.
The authors emphasize that these findings call for caution before indiscriminately applying LLMs in critical cybersecurity contexts. Developers need to be aware of these inherent biases and inconsistencies, as they could lead to vulnerabilities or unexpected outcomes. The study advocates for systematic testing frameworks, like FAIRGAME, to thoroughly evaluate LLMs in various scenarios before their widespread adoption. This will help identify the most stable and reliable LLMs for developing robust defensive systems and understanding potential LLM-based attacker behaviors. For more details, you can read the full research paper here.
