TLDR: Simon Fraser University health sciences researcher Zoha Khawaja is urging immediate ethical and regulatory oversight for therapeutic voice AI, particularly as these technologies move into clinical settings. Khawaja highlights concerns about potential therapeutic misconception, algorithmic bias, and the erosion of human connection in care, advocating for a ‘digital compassionate care’ approach where AI supports, rather than replaces, human interaction.
As voice artificial intelligence (AI) rapidly advances towards integration into clinical healthcare, a critical call for ethical and regulatory frameworks has been issued by Zoha Khawaja, a health sciences researcher at Simon Fraser University. Published on October 27, 2025, Khawaja’s research underscores the pressing need for robust ethical, legal, and social oversight, especially concerning the use of voice AI in therapeutic care.
Voice AI systems are designed to analyze vocal patterns, detecting subtle cues that could indicate physical, cognitive, and mental health conditions. These analyses are based on various vocal qualities, such as pitch, jitter, fluency, and specific linguistic choices. Some technology companies have even heralded voice AI as ‘the new blood’ of healthcare, recognizing its potential as a powerful biomarker. However, Khawaja advises extreme caution regarding its deployment.
A study conducted as part of this research revealed significant public sentiment regarding the governance of these emerging technologies. While a substantial 77 percent of participants expressed support for utilizing voice AI to enhance patient outcomes, an overwhelming 92 percent agreed on the necessity for healthcare or governmental organizations to establish clear governance models to oversee its integration into clinical practice.
Khawaja warns of a ‘real risk of therapeutic misconception,’ where individuals might mistakenly believe that AI applications are providing clinical diagnoses or treatments, despite not being designed for such purposes. ‘That’s particularly dangerous for vulnerable users who may not have access to traditional care,’ Khawaja explains. Key concerns identified by participants in the study include the potential for algorithmic bias, a lack of transparency in how these systems operate, the erosion of vital human connection in care delivery, and unclear lines of accountability when issues arise. The research strongly advocates for a ‘digital compassionate care’ approach, emphasizing that AI tools should serve to support and augment human relationships in therapy, rather than supplant them.
The researcher also points out the practical challenges faced by healthcare professionals. ‘Clinicians are already overburdened,’ Khawaja states, suggesting that ‘expecting them to bear the ultimate responsibility of these technologies is unrealistic.’ In response to these advancements, clinical trials are currently underway in the U.S. to validate voice as a biomarker. Furthermore, regulatory sandboxes – controlled environments for testing new technologies – are being proposed to proactively address ethical challenges and inform policy development before voice AI becomes widely adopted in clinical practice.
Also Read:
- The Imperative for Human Oversight in AI-Driven Healthcare Amidst Information Deluge
- Clinical AI Poised for Explosive Growth to $188 Billion by 2030, Navigating Key Hurdles
Khawaja clarifies her stance, stating, ‘I’m not saying we shouldn’t use voice AI or virtual chatbots in mental healthcare. But we must use them safely, responsibly, and with compassion.’ This highlights a balanced perspective, acknowledging the potential benefits while prioritizing patient safety and ethical considerations.
