TLDR: A large-scale study found that users prefer concise, source-backed AI responses over verbose, complex ones in search applications. Current AI trends towards “artificial sophistication” reduce user satisfaction and trust by increasing cognitive load and creating an “uncanny valley” effect. The research suggests AI should prioritize human-like brevity, transparency, and source attribution for better user engagement and reliability.
A recent large-scale study challenges the common belief that more complex and verbose AI responses are better. Instead, the research suggests that users overwhelmingly prefer AI systems that communicate in a concise, human-like, and transparent manner, backed by clear sources.
The paper, titled “AI Should Be More Human, Not More Complex: A Large-Scale Study on User Preferences for Concise, Source-Backed AI Responses in Search Applications,” was conducted by Carlo Esposito, Founder and Lead Researcher at Eyed Softwares and Aploide Softwares. It delves into why current trends in Large Language Model (LLM) development might be missing the mark when it comes to user satisfaction and engagement in search applications.
The Problem with “Artificial Sophistication”
The study introduces the concept of “artificial verbosity syndrome,” where AI systems generate responses that sound authoritative but often fail to meet user needs. This phenomenon leads to excessive response length, unnecessary complex language, false confidence, and a lack of clear source attribution. Essentially, AI systems are “trying too hard” to appear knowledgeable, which paradoxically reduces user trust and increases the mental effort required to understand the information.
This issue is rooted in established psychological principles. Cognitive Load Theory suggests that verbose AI responses increase “extraneous cognitive load”—mental effort irrelevant to understanding the core information. Users naturally gravitate towards systems that offer maximum utility with minimum mental investment. Furthermore, the “uncanny valley” effect, typically seen in robotics, applies to AI communication: when AI tries to mimic human conversation too closely but falls short, it can create discomfort and reduce trust.
What the Study Involved
To investigate user preferences, the researchers conducted a comprehensive study involving an estimated 10,000 participants. These participants, representing a diverse range of ages, educational backgrounds, and technical proficiencies, evaluated responses from five major AI-powered search systems: Otus Web, Perplexity, Brave Leo AI, ChatGPT Web Search, and Claude 4 Sonnet + Web Research. The study used a blind comparative methodology, meaning participants didn’t know which AI system generated which response, eliminating potential brand bias.
Participants were given identical queries and asked to evaluate responses based on their likelihood to read completely, share, or return for reference. The analysis also looked at response characteristics like word count, lexical complexity, and source attribution.
Key Findings: Simplicity Wins
The results were clear: users strongly preferred concise, source-attributed responses. Otus Web, for example, which provided brief, factual, and well-sourced answers, consistently ranked highest in overall preference, willingness to read completely, likelihood to share, and perceived usefulness, while having the lowest cognitive load score.
In contrast, systems that generated longer, more complex responses, often with extensive formatting, scored lower on user satisfaction metrics. For a simple query like “What is ArXiv?”, some AI systems produced responses over 400 words long with numerous formatting elements, turning a straightforward question into an overwhelming tutorial. Users reported feeling that such responses were “trying too hard” to be helpful, leading to reduced engagement.
The study highlighted the critical role of transparency. Responses with clear, specific source citations built higher user trust. When AI systems acknowledged limitations or uncertainty, it further enhanced user confidence, directly contradicting the common practice of AI appearing overly confident even in ambiguous situations.
Also Read:
- Designing AI for Human Well-being: A New Framework for Human-AI Interaction
- Assessing LLM Capabilities in Explaining Well-Being Concepts: A Deep Dive into AI’s Readiness
Implications for AI Development
The research suggests a fundamental shift is needed in how AI systems are developed. Instead of prioritizing “artificial sophistication” or demonstrating vast knowledge through verbosity, AI should focus on genuine utility and user-centric design. Key recommendations include:
- Source-First Architecture: AI systems should prioritize finding and attributing reliable sources rather than primarily generating content from their training data. This helps reduce “hallucinations” and builds trust.
- Response Length Optimization: Responses should be tailored to the query’s complexity. Simple questions should receive concise answers, while complex analytical tasks might warrant more detailed explanations.
- Uncertainty Acknowledgment: AI should be designed to admit limitations and uncertainty, fostering appropriate user trust rather than encouraging over-reliance.
- User-Centric Design: The primary goal should be to help users efficiently achieve their objectives, not to showcase the AI’s capabilities.
In essence, the study argues that the most effective AI communication mirrors the best aspects of human discourse: directness, appropriate confidence, and transparent reasoning. This means developing AI systems that sound less like encyclopedias and more like helpful, honest assistants. The future of human-AI interaction, according to this research, depends on AI becoming more human in its communication effectiveness, while maintaining the precision and reliability expected from technology. You can read the full research paper here.
