TLDR: A comprehensive survey of 44 research papers reveals that AI-generated text typically adopts a formal and impersonal style, marked by a higher frequency of nouns, determiners, and adpositions, and a reduced reliance on adjectives and adverbs. It also consistently shows lower lexical diversity, a smaller vocabulary, and increased repetition compared to human-written text. The study points out a significant research bias towards English language and GPT-3.5 models, emphasizing the necessity for future investigations to encompass a broader spectrum of languages, AI models, text genres, and prompting methods to fully grasp the nuances of AI’s linguistic output.
Large language models (LLMs) are rapidly becoming indispensable tools for generating text across various sectors, including education, healthcare, and scientific research. As AI-generated content becomes more prevalent, understanding its unique linguistic characteristics is crucial. Humans often find it challenging to distinguish between text written by AI and by other humans, yet studies consistently show distinct differences. A recent survey paper, titled “Linguistic Characteristics of AI-Generated Text: A Survey,” delves into these differences, offering a comprehensive synthesis of existing research.
The Distinctive Voice of AI
The survey categorizes findings into lexical, grammatical, and other linguistic descriptions. On a lexical level, AI-generated text (AIGT) often exhibits a more formal and impersonal style. It tends to use words that refer to ambiguous groups, like “others” or “researchers,” and specific, sometimes rare, expressions such as “stand out feature” or “incredibly polite.” Conversely, AIGT is less likely to use sensing verbs (e.g., read, look, hear), certain conjunctions (e.g., however, because), many pronouns (e.g., I, they), and modal verbs (e.g., will, would, might). It also generally avoids aggressive or rude language. A striking and consistent finding is that AIGT is significantly less lexically diverse than human-written text (HWT), meaning it uses a smaller variety of words and often repeats words and expressions more frequently, including emoticons. While some studies suggest AIGT might contain longer words, others find no significant difference. Interestingly, AI models don’t always choose shorter words in predictable contexts, a preference often seen in human writing. AIGT also tends to use a less varied set of punctuation marks, relying heavily on commas and periods, and may even contain non-existent words, particularly in specialized fields like clinical texts.
Grammar and Structure in AI-Generated Content
Grammatically, AIGT is frequently reported as being more syntactically complex. This complexity is sometimes linked to shorter dependency lengths in certain grammatical relations but longer ones for punctuation. In terms of sentence length, AIGT tends to show less variation, though there’s no clear consensus on whether AI produces consistently longer or shorter sentences than humans. A notable pattern in AIGT is the increased use of nouns, adpositions, determiners, and coordinating conjunctions, alongside a decrease in adverbs and proper nouns. This often results in a higher degree of nominalization, where verbs or adjectives are converted into nouns (e.g., “announce” becoming “announcement”), contributing to a more formal tone. AIGT also shows a more consistent Subject-Verb-Object (SVO) sentence ordering, in contrast to the greater variation found in human writing. Furthermore, AI-generated text often contains higher frequencies of certain word n-grams (sequences of words) and repeats longer part-of-speech sequences. It tends to use fewer discourse markers, modal expressions, and epistemic markers, and often results in lower readability scores.
Stylistic Tendencies and Emotional Nuances
Beyond specific word choices and grammatical structures, AIGT exhibits distinct stylistic qualities. It is generally perceived as more formal, impersonal, analytic, and descriptive, focusing on conveying information rather than signaling personal involvement. This often translates to a more neutral sentiment, though findings on positive and negative emotional content can vary depending on the text type. For instance, AI-generated news articles might contain fewer negative emotions, while hotel reviews could be more emotional. Human evaluators often identify AI text by its general ease of reading, a perceived lack of a familiar or personal tone, and phrases that seem “well produced or constructed.”
Also Read:
- AI’s Mirror Test: Large Language Models Struggle to Recognize Their Own Creations
- Beyond Words: Large Language Models and the Preservation of Textual Semantics
The Research Landscape and Future Directions
The survey highlights several critical observations about the current state of research. A significant portion of studies (around 57%) rely on GPT-3.5, and a vast majority (91%) focus on English text. This narrow scope raises questions about the generalizability of findings to other AI models and languages. Research has shown that different LLMs can produce text with varying linguistic characteristics, and even the same model can generate different outputs based on the genre or the specific prompt used. For example, rephrasing existing human text in a prompt can lead to AI-generated content that is linguistically closer to human writing. This underscores the need for future research to diversify its approach by including a wider range of models (especially open-source ones), exploring more languages (including low-resource languages), and experimenting with multiple prompting strategies. Such broader investigations will provide a more robust and comprehensive understanding of the evolving linguistic landscape of AI-generated text. For a comprehensive understanding, you can read the full research paper: Linguistic Characteristics of AI-Generated Text: A Survey.
