TLDR: A study evaluated AI-powered non-player characters (NPCs) in a VR interrogation simulator, focusing on their perceived realism and system performance. Using GPT-4 Turbo, the NPCs demonstrated good believability in behavior, social relationships, and intelligence, and the system achieved good usability. While the average response latency was around 7 seconds, it could be as low as 2 seconds, highlighting both the potential and challenges of integrating large language models for creating realistic and interactive VR experiences.
Imagine stepping into a virtual world where the characters you interact with feel as real as people you meet every day. This isn’t just a dream for the future; it’s rapidly becoming a reality thanks to advancements in artificial intelligence (AI), particularly large language models (LLMs) like GPT-4 Turbo.
A recent study delved into this exciting area, evaluating how realistic and effective AI-powered non-player characters (NPCs) can be in virtual reality (VR) environments. The researchers developed a unique VR interrogation simulator, designed to test the perceived realism and performance of these AI characters. The simulator featured two AI-controlled NPCs: a suspect and a partner, both powered by GPT-4 Turbo, engaging participants in a scenario to determine the suspect’s guilt or innocence in a beer theft case.
The core of this system involved a sophisticated communication loop. When a participant spoke, their words were converted from speech to text (STT). This text, along with the ongoing conversation history, was then sent to the GPT-4 Turbo model. GPT-4 Turbo would interpret the question and formulate a text response, which was then converted back into speech (TTS) for the NPC to “speak.” This continuous chat log allowed the NPCs to maintain context and appear to have memory, making interactions feel more natural and coherent.
To add depth, the NPCs were given distinct personalities. The suspect NPC was designed to be a nervous, socially awkward Australian exchange student, aiming to evade detection. In contrast, the partner NPC offered strategic advice in a neutral, accent-free English, helping the player navigate the interrogation. These personality traits, though sometimes subtly perceived by participants, were intentionally crafted using GPT-4 prompts to enhance the immersive experience.
The study involved 18 participants using Meta Quest 2 VR headsets. The researchers measured various aspects, including system performance (specifically, latency or response time), the believability of the virtual agents, and the overall usability of the system. Latency, the time it takes for an NPC to respond, was a critical factor. The average “cycle latency” – the total waiting time from a player’s question to an NPC’s answer – was approximately 7 seconds. While this might seem a bit long for a real-time conversation, the minimum latency observed was around 2 seconds, indicating that quick, human-like responses are achievable under optimal conditions. It was also noted that the response time tended to increase as the conversation progressed due to the growing chat history that the AI had to process.
When it came to believability, the NPCs scored an overall 6.67 out of 10. Participants rated them highly in areas like behavior, social relationships, and intelligence, suggesting that the NPCs genuinely felt like more than just computer programs. However, scores were moderate for emotion and personality. This aligns with the researchers’ design choices, as the NPCs were intended to be relatively neutral and unemotional for the specific interrogation scenario. The system also achieved a “good” usability score of 79.44 out of 100 on the System Usability Scale, indicating that participants found the application easy to learn and use, even those new to VR.
Also Read:
The findings from this research highlight the immense potential of integrating large language models into VR for creating highly realistic and interactive NPCs. While challenges remain, particularly in optimizing system latency for seamless real-time interactions and further enhancing emotional depth, the study provides a strong foundation for future developments. Such AI-driven characters could revolutionize not just gaming, but also serious applications like training simulations, where AI assistants could provide real-time guidance and improve situational awareness. For more details, you can read the full research paper here.
