TLDR: A study surveyed 173 participants to understand why people use conversational AI like ChatGPT for search, focusing on trust, human-likeness, and willingness to trade factuality for better experience. It identified two groups: ‘Daily Users of Both’ (DUB) who trust ChatGPT more and perceive it as human-like, and ‘Daily Users of Google’ (DUG) who trust it less. DUB users are more willing to trade factual accuracy for personalization and conversational flow. The research highlights that human-likeness and personalization drive adoption but also contribute to ‘overtrust’, making users vulnerable to misinformation, especially among middle-aged adults who trust ChatGPT more despite less frequent use.
The way we search for information is rapidly evolving. While traditional search engines like Google have been our go-to for decades, conversational AI interfaces such as ChatGPT are increasingly becoming part of our daily lives. A recent study delves into what drives people to adopt these new conversational tools and how their human-like qualities might lead to an issue known as ‘overtrust’.
Researchers Mert Yazan, Frederik Bungaran Ishak Situmeang, and Suzan Verberne explored the influence of human-likeness on how people use ChatGPT for search. Their paper, titled Personality over Precision: Exploring the Influence of Human-Likeness on ChatGPT Use for Search, highlights that while conversational AI offers interactive, personalized, and engaging experiences, it also carries the risk of users relying on incorrect information.
Understanding User Behavior
To understand the factors at play, the researchers surveyed 173 participants, examining their perceptions of trust, human-likeness (anthropomorphism), and design preferences when comparing ChatGPT and Google. A crucial part of the study involved asking users if they would be willing to trade factual accuracy for benefits like ease of use or a more human-like interaction.
The analysis revealed two distinct user groups:
- Daily Users of Both (DUB): This group uses both ChatGPT and Google regularly.
- Daily Users of Google (DUG): This group primarily relies on Google, using ChatGPT less frequently.
The DUB group showed significantly higher trust in ChatGPT, perceiving it as more human-like. They were also more willing to compromise factual accuracy for enhanced personalization and a smoother conversational flow. Interestingly, this group also reported high trust in Google.
Conversely, the DUG group exhibited lower trust in ChatGPT and was less inclined to trade off factuality. However, even this group appreciated certain aspects of ChatGPT, such as ad-free experiences and responsive interactions.
The Role of Human-Likeness and Personalization
A key finding was the strong connection between perceived human-likeness and trust. Users who found ChatGPT to be more human-like tended to trust it more. Both user groups identified personalized responses as a primary reason for preferring ChatGPT over Google. Other appreciated aspects of ChatGPT included clearer answers, interaction flow, and the absence of advertisements.
The study suggests that the conversational nature of ChatGPT, supported by personalized and human-like interactions, gives it a distinct advantage over traditional search engines. Many users, especially those in the DUB group, felt that a smoother interaction was worth potentially trading off some factuality.
The Overtrust Dilemma and Age-Related Vulnerabilities
The research sheds light on the emerging issue of ‘overtrust’ in conversational interfaces. While the generative nature of large language models makes them powerful, it also makes them prone to generating inaccurate information or ‘hallucinations’. The study indicates that the bond users form with these tools, driven by trust and perceived human-likeness, can exacerbate this overtrust, making users more susceptible to misinformation.
Demographic analysis revealed interesting patterns related to age. Middle-aged adults (30-55) were found to use ChatGPT less frequently but trusted it more than younger adults (18-30). This suggests a potential vulnerability to misinformation among this demographic, as younger users might have a better understanding of AI limitations and the risk of incorrect answers due to their greater experience.
Also Read:
- ChatGPT’s Political Compass: How AI Personalizes Responses Based on Inferred User Views
- Bridging the AI Literacy Gap: How Community College Instructors Envision Teaching AI to All Students
Implications for the Future of Search
The findings underscore the critical roles of personalization and human-likeness in the design of conversational AI systems. While these features enhance user engagement and satisfaction, they also highlight the importance of user awareness regarding factual accuracy. The study suggests that many users, particularly the DUB group, might be using traditional search methods to fact-check information obtained from conversational AI, indicating a hybrid approach to information seeking.
This research provides valuable insights for developers and users alike, emphasizing the need to balance engaging, human-like interactions with clear communication about the limitations and potential inaccuracies of AI-powered search tools.
