TLDR: This research explores how large language models (LLMs) can be personalized to detect offensive language in political tweets by adopting specific political and cultural perspectives. It finds that LLMs with explicit reasoning capabilities (like DeepSeek-R1 and o4-mini) are significantly better at consistently and sensitively adapting to ideological and cultural variations across languages (English, Polish, Russian) compared to models without reasoning or smaller models. This personalization improves both the accuracy and interpretability of offensiveness judgments, highlighting reasoning as crucial for nuanced sociopolitical text classification.
In the ever-evolving landscape of social media, detecting offensive language is a critical task for maintaining respectful online discourse. However, what one person considers offensive, another might not, as offensiveness is deeply subjective, shaped by individual ideologies, cultural backgrounds, and personal values. Traditional models often struggle with this nuance, relying on generalized labels that fail to capture the diverse interpretations of offensive speech.
A recent research paper, “Language, Culture, and Ideology: Personalizing Offensiveness Detection in Political Tweets with Reasoning LLMs”, delves into how large language models (LLMs) can be trained to assess offensiveness more personally. The study explores whether equipping LLMs with explicit reasoning capabilities allows them to better simulate specific political and cultural perspectives when judging political tweets.
The researchers, Dzmitry Pihulski and Jan Koco ´n from the Department of Artificial Intelligence at Wroclaw Tech, focused on a multilingual subset of tweets from the 2020 US elections. They evaluated several LLMs, including DeepSeek-R1, o4-mini, GPT-4.1-mini, Qwen3, Gemma, and Mistral. These models were tasked with classifying tweets as offensive or non-offensive from the viewpoints of various political personas – far-right, conservative, centrist, and progressive – across English, Polish, and Russian contexts.
The core hypothesis was that models capable of explicit reasoning would more effectively adopt these diverse perspectives when prompted with ideological and cultural cues. To test this, they categorized models based on their size (small vs. large) and reasoning capability (enabled vs. disabled).
Key Findings: Reasoning is Crucial for Nuance
The study yielded compelling results, demonstrating that larger models with explicit reasoning abilities, such as DeepSeek-R1 and OpenAI’s o4-mini, were significantly more consistent and sensitive to ideological and cultural variations. These models showed high correlations within the same political groups across different languages, suggesting that language and nationality had minimal impact on their judgments once an ideological stance was adopted. They effectively differentiated between political ideologies, with the far-right group, for instance, showing markedly lower correlations with the progressive left, reflecting distinct classification patterns.
DeepSeek-R1 particularly stood out for its ability to generate detailed reasoning traces, offering transparent insights into its decision-making process. This enhanced interpretability is a significant advantage in understanding how the AI arrives at its conclusions. In contrast, o4-mini provided more brief, summary-like responses, indicating that not just the presence of reasoning, but its depth and coherence, are vital.
A notable observation was the strong bias towards English in the reasoning outputs of these models, even when prompts were given in Polish or Russian. This suggests an “alignment gap” where internal reasoning defaults to a dominant language, potentially impacting interpretability for non-English users.
The Limitations of Non-Reasoning and Smaller Models
Conversely, models without explicit reasoning capabilities, like DeepSeek-V3, and smaller models generally struggled to capture these subtle distinctions. DeepSeek-V3, despite its size, produced largely uninformative results, showing high agreement across outputs but minimal differentiation between ideological groups. This indicates that model scale alone does not guarantee sociopolitical nuance.
Smaller models, even those with reasoning enabled like Qwen3-8B, showed only modest improvements in ideological differentiation compared to their larger, reasoning-enabled counterparts. Non-reasoning small models like Gemma 3-4B-IT, Qwen3-4B, and Mistral-7B-Instruct-v0.3 exhibited uniformly positive correlations, suggesting a limited ability to distinguish nuanced political perspectives.
Also Read:
- Unpacking Historical Bias in AI: A Cross-Lingual Look at LLMs and Romanian History
- Untargeted Jailbreak Attack: A New Approach to Uncover LLM Vulnerabilities
Implications for Personalized AI
The research underscores the critical role of reasoning capabilities in enabling LLMs to simulate human-like, ideologically and culturally grounded interpretations of offensive content. This is particularly important for developing trustworthy NLP applications in sensitive political or cultural contexts, where transparency in decision-making is paramount.
While the findings are promising, the researchers acknowledge several limitations, including the relatively small dataset, the idealized nature of the manually constructed personas, and the challenges of preserving cultural subtleties in translation. The binary classification (offensive/non-offensive) also simplifies the complex, graded nature of real-world offensiveness.
Future work aims to expand the dataset, incorporate more empirically grounded user data, and explore advanced techniques to further personalize responses. The study highlights that for AI to truly understand and navigate the complexities of human communication, especially in politically charged environments, explicit reasoning is not just beneficial, but essential.
