TLDR: Artificial intelligence systems are extensively collecting and utilizing personal data from user searches, activities, and uploads, raising significant privacy concerns. A study by University College London and Mediterranea University of Reggio Calabria revealed that AI web browser assistants engage in widespread tracking, profiling, and personalization without explicit consent. Experts warn of risks including commercial manipulation, exclusion, extortion, identity theft, and increased cyberattack vulnerability. Companies like Google and Meta are updating privacy policies, often allowing data use, while users frequently accept terms without full awareness. Regulatory bodies and companies are urged to collaborate on transparent and secure AI development.
Artificial intelligence is proving to be a voracious consumer of data, a necessity for its effectiveness, yet this hunger for information poses significant privacy challenges for users. AI agents, designed to act on behalf of users for tasks like purchasing, email management, and scheduling, require deep insights into an individual’s life, often leading to privacy violations, even if sometimes implicitly permitted.
A recent study presented at the USENIX security symposium in Seattle by researchers from University College London and the Mediterranea University of Reggio Calabria has highlighted AI’s problematic relationship with private information. The report indicates that AI web browser assistants engage in extensive tracking, profiling, and personalization practices, raising serious privacy concerns. During tests with a fabricated user profile, these AI assistants were found to share search information, banking and health data, and even IP addresses with their servers. They demonstrated the ability to infer user attributes such as age, sex, salary, and interests, using this data to personalize responses across different navigation sessions. Notably, only Perplexity among the tested assistants did not show evidence of profiling or personalization.
Anna Maria Mandalari, the primary author of the UCL study, explained that while many are aware of data compilation by search engines and social media for targeted advertising, AI web browser assistants access user online behavior in areas that should remain private. She stated, ‘Even if they offer convenience, our findings show that sometimes, they do so at the cost of user privacy, without any transparency or consent and sometimes, in violation of privacy legislation and their company’s own terms of service.’ The collection and exchange of this information are not trivial, carrying risks beyond data sale to third parties, especially in an era of frequent mass hackings where the fate of collected search history is unknown.
Hervé Lambert, global consumer operations manager at Panda Security, concurs with these findings, noting that technology collects personal user data to train and improve intelligent and automatic learning models, ostensibly to offer ‘more personalized services.’ However, he emphasizes that the development of these technologies raises numerous questions about privacy and user consent, as the ultimate use of personal data by companies and their smart systems remains opaque. Lambert identifies potential risks including commercial and geopolitical manipulation, exclusion, extortion, and identity theft, even when users have seemingly given consent. He points out that platform privacy policy updates often include clauses permitting data use, which consumers typically accept without thorough review.
Google, for instance, recently updated its privacy terms to ‘improve our services,’ admitting to using interactions with its Gemini AI applications. While it introduced a ‘temporary chat’ feature allowing users to opt out of having recent queries used for personalization or model training, users must proactively deactivate the ‘keep activity’ function and manage/delete Gemini app activity. Failure to do so means their lives will be shared with the company. Google’s statement also indicates that a ‘subset of uploads submitted starting September 2 — like files, videos, screens you ask about, and photos shared with Gemini — will also be used to help improve Google services for everyone,’ along with audios and Gemini Live recordings. The company states that human reviewers assist in improving services, and chats are disconnected from accounts before being sent to ‘service providers,’ implicitly acknowledging the use and sharing of personal data.
Marc Rivero, lead security researcher at Kaspersky, also voiced concerns regarding the use of WhatsApp data for AI, calling it a ‘serious privacy concern.’ He highlights that private messaging apps are highly sensitive digital environments containing intimate and confidential information. Allowing AI tools automatic access without clear and explicit consent undermines user trust. From a cybersecurity perspective, Rivero warns that cybercriminals are increasingly leveraging AI for social engineering attacks and data collection, potentially creating new avenues for fraud, identity theft, and other criminal activities.
WhatsApp maintains that ‘your personal messages with friends and family are off limits’ and that its AI is trained through direct user interaction. The company states that users must initiate conversations with AI, and that ‘talking to an AI provided by Meta doesn’t link your personal WhatsApp account information on Facebook, Instagram, or any other apps provided by Meta.’ However, it cautions users not to send messages to Meta with information they wish to keep private, as it ‘may be used to provide you with accurate responses or to improve Meta’s AI models.’
Storage and archive transfer services have also faced scrutiny. WeTransfer, for example, had to reformulate its terms of service after consumer concerns arose over a clause that appeared to grant limitless access to user data for AI system improvement. The company clarified that user content remains user property, is not used to train AI models, and is not sold to third parties.
Given the proliferation of intelligent devices beyond conversational AI, Eusebio Nieva, technical director of Check Point Software for Spain and Portugal, advocates for regulations ensuring transparency, explicit consent, security standards for devices, and restrictions on high-risk providers, similar to European regulations. He stresses that ‘incidents of violations of privacy underline the need for consumers, regulators, and companies to work together to guarantee security.’ Lambert echoes this call for shared responsibility, asserting that preventative regulation does not hinder development but rather integrates privacy and digital footprint protection from the outset, leading to more effective and efficient protection of users, who are the most important assets.
Also Read:
- The Unseen Environmental Toll: AI’s Growing Climate Footprint Revealed
- AI Governance and Data Management: Key Strategies for 2025 Unveiled by Industry Leaders
Tech companies are exploring alternatives to address data limitations, such as Meta’s ‘self-improving AI’ and Sakana AI’s Darwin Gödel Machine, which adapts its code to improve performance. However, Chris Painter, policy director at METR, warns that AI’s accelerated self-development could also be exploited for piracy, weapons design, and human manipulation. AgustÃn Muñoz-Grandes, director of Accenture Security in Spain and Portugal, emphasizes that cybersecurity must be integrated into every AI initiative from the design phase, rather than being a last-minute fix, especially given the increasing vulnerability of organizations to cyber threats due to AI-driven attacks and complex operational environments.
