TLDR: The research paper “AI Propaganda factories with language models” reveals that small language models (SLMs) can create fully automated “AI propaganda factories” capable of generating consistent, persona-driven political messaging on commodity hardware. Key findings indicate that persona design is more influential than the model itself, and engagement with counter-arguments intensifies ideological adherence and increases extreme content. The paper concludes that this capability is a present risk, advocating for a shift in defense strategies from restricting model access to detecting behavioral consistency and coordination in influence campaigns.
In an era where digital information shapes public discourse, a new research paper titled “AI Propaganda factories with language models” by Lukasz Olejnik from King’s College London sheds light on a concerning development: the ability to execute AI-powered influence operations end-to-end using readily available, commodity hardware. This research highlights how small language models (SLMs) are becoming potent tools for generating coherent, persona-driven political messaging, capable of operating without human oversight. [RESEARCH_PAPER_URL]
The Rise of Automated Influence
Traditionally, influence campaigns required large teams, manual scripting, and static messaging. However, the advent of Artificial Intelligence, particularly large language models (LLMs), has transformed this landscape. AI enables high-speed, adaptive content generation at scale with minimal human intervention, producing persuasive content at low cost and with significant rhetorical flexibility. This capability has been recognized as a top global risk by the World Economic Forum’s 2025 Global Risks Report and flagged as a concern by NATO. While previous operational AI use for these purposes was limited to LLMs accessed via controlled APIs, where misuse could be detected and blocked, the situation fundamentally changes with small language models (SLMs).
Small Language Models: A Game Changer for Covert Operations
Unlike their larger, cloud-based counterparts, SLMs are often offered as open-weight models. This means they can be downloaded, deployed locally, fine-tuned, and used covertly without external oversight. This makes SLMs ideally suited for actors who prioritize deniability and persistence over raw performance, as they can be operated independently of third-party infrastructure, reducing the risk of discovery or disruption. The paper defines SLMs as models containing up to 30 billion parameters, a threshold that includes capable models deployable on high-end consumer hardware or small-scale clusters.
Building an AI Propaganda Factory
The research explores the practical use of SLMs in manual, semi-automated, and potentially fully automated influence operations, coining the term “AI propaganda factory.” The focus is on the feasibility of creating a pipeline capable of sustained, consistent, and stable content generation. To demonstrate this, the study evaluated political, ideological, and psychological traits of SLM-generated content when engaging with real-world discussions from the r/ChangeMyView Reddit board. All tests were performed on commodity hardware, simulating human-like personas with demographic descriptors, rhetorical tone, and political stance. A key metric, persona fidelity, assessed how consistently these traits were expressed.
Key Findings: Persona Over Model, Engagement as a Stressor
The study yielded two significant behavioral findings:
-
Persona-over-model: The design of the persona (e.g., its ideological stance, communication style, and tone) explains the model’s behavior more than the identity of the language model itself. This suggests that the effectiveness of these operations hinges more on how the AI is instructed to behave rather than the specific AI model used.
-
Engagement as a stressor: When AI personas are prompted to counter-argue, their ideological adherence strengthens, and the prevalence of extreme content increases. This indicates that adversarial conversational contexts can push AI-generated content towards more polarized and intense expressions.
Overall, persona fidelity was found to be high across all models, with ideological adherence increasing significantly in engagement mode. The share of extreme outputs also rose in parallel during counter-argument exchanges. Crucially, the research found that fully automated influence-content production is not a future risk but a contemporary reality, accessible to both large and small actors.
Also Read:
- Exploiting LLM Safety Alignment to Inject Targeted Bias and Censorship
- Hidden Commands: New Research Uncovers ‘Prompt-in-Content’ Attacks on LLMs
Implications for Defense and Societal Impact
The findings present a paradox: the very consistency that makes AI personas effective in sustained influence operations also provides a potential detection signature. Defenders should consider behavioral consistency analysis, as excessive stability across diverse contexts may indicate automation. The low cost and scalability of these systems mean that the barrier to entry for producing convincing political messaging has dropped sharply, enabling small organizations or even individuals to operate capabilities that once required significant resources.
This creates an asymmetric risk, as personas at the extremes of the political spectrum demonstrate superior consistency, potentially making the most polarized viewpoints the easiest to automate at scale. Consequently, the defensive focus needs to shift from restricting model access towards conversation-centric detection and disruption of campaigns and coordination infrastructure. Policy responses should prioritize behavioral detection over model fingerprinting, focusing on conversation dynamics, stance trajectories, and cross-platform coordination patterns.
The research concludes that the capability for fully automated influence operations is here today, necessitating urgent countermeasures that exploit behavioral-consistency signals while they remain detectable. The challenge lies in whether platforms and public institutions can adapt quickly enough to observe, attribute, and mitigate these effects without unduly impacting legitimate speech.
