TLDR: PromptGuard is a few-shot classification framework for detecting Bengali hate speech across six categories. Developed by Rakib Hossan and Shubhashis Roy Dipta, it combines chi-square statistical analysis for discriminative keyword extraction with an adaptive majority voting system for robust decision-making. The framework achieves competitive performance, outperforming traditional baselines, and demonstrates the value of integrating statistical features with few-shot learning for low-resource languages like Bengali, especially in the absence of extensive labeled datasets.
Online platforms face a persistent challenge in combating hate speech, a problem that is particularly complex in languages with fewer digital resources, such as Bengali. Bengali, with its rich morphology and unique cultural expressions, presents significant hurdles for automated content moderation systems. Traditional methods often require vast amounts of labeled data, which is expensive and scarce for low-resource languages.
Addressing this challenge, researchers Rakib Hossan from Bangladesh University of Business and Technology and Shubhashis Roy Dipta from the University of Maryland, Baltimore County, have introduced a novel framework called PromptGuard. This system is designed for few-shot classification, meaning it can learn to identify hate speech categories with very limited training examples. PromptGuard aims to classify Bengali text into six distinct categories: none, sexism, abusive, profane, religious hate, and political hate.
How PromptGuard Works
PromptGuard integrates several key components to achieve its robust performance:
1. Statistical Keyword Extraction: The framework utilizes chi-square statistical analysis to pinpoint words most strongly associated with each hate speech category. This method ensures that the selected keywords genuinely discriminate between categories, rather than being chosen arbitrarily. For instance, keywords like ‘বাল’ and ‘মািগর’ were highly associated with the ‘Profane’ category, while ‘নারী’ and ‘পরিকয়া’ were linked to ‘Sexism’. These statistically derived keywords are then manually refined for cultural sensitivity.
2. Balanced Dataset Construction: To ensure fair representation and prevent bias, PromptGuard creates a balanced training pool by sampling an equal number of examples (120 per category) across all six categories.
3. Enhanced Prompt Engineering: The system employs a structured prompt template that combines these statistical keywords with few-shot examples and step-by-step reasoning guidelines. This helps the underlying language model better understand the classification task.
4. Dynamic Few-Shot Learning Strategy: PromptGuard uses a two-phase example selection method to balance diversity. Initially, it uses distinct sets of examples for classification attempts. If a clear consensus isn’t reached, it can extend the voting process by adding more rounds with randomly selected examples, offering fresh perspectives on ambiguous cases.
5. Robust Majority Voting with Adaptive Extension: The core decision-making process involves an initial phase of three parallel voting turns. If no clear majority (more than 50%) emerges from these turns, the system adaptively adds up to ten more voting turns, each with new examples, until a consensus is achieved or the maximum iterations are reached. If a tie persists, a random winner is selected.
Performance and Insights
PromptGuard achieved a micro-F1 score of 67.61, significantly outperforming n-gram baselines (60.75) and random approaches (14.65). While it showed strong performance, it still lagged behind the top-performing systems in the BLP-2025 Task 1A leaderboard.
A detailed analysis revealed that the model was most accurate in classifying non-hate content. However, it struggled most with the ‘Abusive’ category. This difficulty is attributed to the semantic overlap of keywords associated with ‘Abusive’ speech, which can also appear in benign contexts, making precise classification challenging.
Ablation Studies
The researchers conducted ablation studies to understand the impact of different components:
- Impact of Keywords: Including targeted keywords in the prompt significantly improved performance, raising the micro-F1 score from 57.47 to 59.77 on a sampled dataset.
- Impact of Shots & Turns: The optimal performance was observed with 3 shots (examples per label) and 3 initial turns. Increasing the number of examples beyond this point led to a decrease in performance, possibly due to the “lost-in-the-middle” effect in large language models, where too much context can dilute focus.
- Impact of Different Models: Testing with various models showed that larger Qwen3 models generally yielded better performance, indicating a positive correlation between model size and accuracy within that family. GPT-OSS models, while consistent, performed lower than the best Qwen models.
Also Read:
- Adaptive AI Framework Enhances Detection of Subtle Hate Speech
- Multi-Stage Safeguards Enhance Safety and Quality in AI Deep Research
Conclusion
PromptGuard demonstrates the effectiveness of combining statistical feature selection with few-shot learning and adaptive decision-making for Bengali hate speech detection. This framework offers a valuable approach for low-resource language tasks, providing a robust solution without the need for extensive labeled datasets. Future research will explore more advanced feature selection methods to further enhance few-shot hate speech detection.
For more details, you can read the full research paper here: PromptGuard at BLP-2025 Task 1: A Few-Shot Classification Framework Using Majority Voting and Keyword Similarity for Bengali Hate Speech Detection.
