TLDR: This paper introduces KurdSTS, the first Semantic Textual Similarity (STS) dataset for Central Kurdish, containing 10,000 annotated sentence pairs. It also develops and benchmarks a Central Kurdish Sentence-BERT (S-BERT) model, which significantly outperforms multilingual models and traditional baselines in identifying semantic equivalence. This work addresses the lack of NLP resources for Kurdish, paving the way for applications like plagiarism detection and advancing language technology for low-resource languages.
Natural Language Processing (NLP) is a field of artificial intelligence that helps computers understand and generate human language. One of its most challenging applications is Semantic Textual Similarity (STS), which measures how equivalent two pieces of text are. While many resources exist for widely spoken languages like English, languages with fewer resources, such as Kurdish, have often been overlooked.
This research paper introduces a significant step forward for the Kurdish language in the realm of NLP. It addresses a critical gap by presenting the first-ever Semantic Textual Similarity (STS) dataset specifically for Kurdish, named KurdSTS. This pioneering work aims to lay a foundation for future studies in Kurdish semantic research and NLP for other low-resource languages.
The Challenge for Kurdish
Kurdish, particularly its Central Kurdish (Sorani) dialect, is spoken by millions across the Kurdistan region, Iraq, Iran, Türkiye, and Syria. Despite its rich history and widespread use, it has lacked computational support, especially for tasks like plagiarism detection. Existing tools are predominantly for high-resource languages, leaving Kurdish-speaking communities to rely on manual, time-consuming methods for ensuring academic and professional integrity. The complex morphology and syntactic structures of Central Kurdish, written in Arabic letters, also present unique challenges for computational analysis.
Introducing KurdSTS: A New Dataset
To overcome these hurdles, the researchers have developed a dataset comprising 10,000 formal and informal sentence pairs, meticulously annotated for semantic similarity. This dataset is designed to capture the intricate linguistic features of Central Kurdish, including its various syntactic arrangements and morphological structures. The goal is to make NLP tools for Central Kurdish not only functional but also linguistically intelligent, capable of detecting paraphrased, reused texts, and semantic synonyms.
The creation of this dataset involved translating and culturally adapting the PAWS (Paraphrase Adversaries from Word Scrambling) dataset, originally in English, into Central Kurdish using the Google Translate API. Human reviewers then refined these translations to ensure accuracy and cultural relevance, aligning them with Kurdish syntax and semantics.
How the Model Works
The paper also details the development of a Central Kurdish Sentence-BERT (S-BERT) model. S-BERT is an an adaptation of the powerful BERT (Bidirectional Encoder Representations from Transformers) model, tailored to generate fixed-size representations for entire sentences, making it efficient for computing semantic similarity. A specialized Central Kurdish Tokenizer was developed to accurately segment text into “subword units” using WordPiece tokenization, which helps the model handle unseen words and informal writing styles common in social media.
The model uses a sophisticated architecture involving word embeddings, transformer layers, and a self-attention mechanism to understand the contextual relationships between words. These token representations are then pooled to create a single sentence embedding. Finally, cosine similarity is used to measure the semantic equivalence between two sentence embeddings, with scores ranging from -1 (opposite meanings) to 1 (identical meanings).
Also Read:
- Evaluating RAG Systems with Knowledge Graphs: A New Framework for Deeper Semantic Analysis
- AI’s Nuanced View: How Reasoning LLMs Personalize Offensiveness Detection in Political Discourse
Promising Results and Future Directions
The evaluation of the Central Kurdish S-BERT model showed impressive performance. In unsupervised settings, it significantly outperformed multilingual BERT and traditional methods like TF-IDF. When fine-tuned in a supervised setting using the KurdSTS dataset, the model achieved a high Spearman correlation score of 0.84, demonstrating its ability to accurately predict semantic similarity. This performance is competitive with STS evaluations in other low-resource languages like Persian, Arabic, and Urdu, despite Kurdish having fewer existing linguistic resources.
While the model excels in many areas, the researchers identified challenges, particularly with idiomatic expressions, highly informal language, and domain-specific content not well-represented in the training data. Future work will focus on expanding the dataset to include more diverse examples, such as social media posts and specialized texts, and exploring multi-task transfer learning with related languages like Persian and Arabic.
This groundbreaking research paves the way for practical applications in Central Kurdish, including automated plagiarism detection, semantic search engines, and text summarizers. It represents a crucial step in advancing NLP technologies for underrepresented linguistic groups and highlights the importance of language-specific datasets and fine-tuning. You can find more details about this work in the full research paper: KurdSTS: The Kurdish Semantic Textual Similarity.
