TLDR: A new research paper introduces an expert-validated privacy scorecard to evaluate five leading AI coding assistants: OpenAI GPT, Anthropic Claude, Google Gemini, GitHub Copilot, and Amazon Q Developer. The study analyzes legal, enterprise, technical, and external documents against 14 weighted criteria, revealing Google Gemini as the top performer in privacy, followed by Anthropic Claude and GitHub Copilot. Key findings highlight widespread issues like opt-out consent for model training and a general failure to proactively filter sensitive information from user prompts. The paper provides actionable guidance for developers and organizations to select privacy-preserving tools and calls for industry-wide adoption of user-centric privacy standards.
The rapid integration of AI-powered coding assistants into our daily development workflows has brought immense productivity gains. Tools like GitHub Copilot, Google Gemini, and OpenAI’s GPT models have become indispensable for many. However, this convenience comes with a significant question: Can we truly trust these copilots with our proprietary code and sensitive data?
A new research paper, titled “Can You Trust Your Copilot? A Privacy Scorecard for AI Coding Assistants” by Amir AL-Maamari from the University of Passau, delves deep into this critical issue. The paper addresses the prevalent opacity surrounding the data handling practices of these AI tools, which often creates security and compliance risks for developers and organizations alike.
The core of this research is a novel, expert-validated privacy scorecard designed to systematically evaluate the privacy posture of leading AI coding assistants. To achieve this, the methodology involved a detailed analysis of four types of documents: core legal policies, enterprise-tier agreements, technical documentation, and external audits. Five prominent AI coding assistants—OpenAI’s GPT models, Anthropic Claude, Google Gemini, GitHub Copilot, and Amazon Q Developer—were then scored against 14 weighted criteria. These criteria, along with their weighting, were meticulously refined by a legal expert and a data protection officer to ensure their relevance and robustness.
The study’s findings reveal a clear hierarchy in privacy protections among the evaluated tools. Google Gemini emerged as the leader with a score of 89.25, demonstrating strong performance across all categories. Anthropic Claude (81.88) and GitHub Copilot (78.75) formed a competitive second tier. A notable 20-point gap separated these from Amazon Q Developer (72.38) and OpenAI’s GPT models (68), which ranked last. This significant spread highlights that the choice of an AI coding assistant has measurable privacy implications.
Key Weaknesses Across the Industry
The analysis uncovered several common industry weaknesses. A pervasive issue is the use of opt-out consent for model training. This means that, by default, user code is collected and used to improve the service, placing the burden of privacy protection on the user. Anthropic Claude was the sole exception, implementing a user-centric opt-in model. Another critical failing was the near-universal inability to proactively filter secrets from user prompts. Most assistants place the responsibility on the user to avoid inputting sensitive data like API keys or personally identifiable information (PII), essentially collecting and storing such data if entered without redaction.
Furthermore, the research found that for some providers, such as OpenAI, legal realities like court-ordered data retention can contradict user-facing promises of data deletion, making their data subject access request mechanisms functionally misleading.
Also Read:
- The Hidden Security Risks of AI-Generated Code in Your Software Supply Chain
- GPT-5 Emerges as a Powerful Oracle for Smart Contract Verification
Recommendations for a More Private Future
The paper offers actionable guidance for both software practitioners and organizations, as well as for tool providers and policymakers. For developers and organizations, it recommends treating the selection of an AI coding assistant as a security and compliance decision. Key advice includes prioritizing enterprise-tier subscriptions, which consistently offer superior safeguards like zero-data-retention policies and contractual guarantees against using user data for model training. It also advises assuming zero-filtering, meaning developers should operate under the assumption that any code or data pasted into an assistant may be stored and reviewed, and enforcing strict policies against including proprietary code or secrets in prompts. Finally, it suggests favoring tools with explicit opt-in consent models, with Anthropic Claude being highlighted as the current standard-bearer.
For providers and policymakers, the paper calls for significant improvements. It advocates for making opt-in consent the non-negotiable standard for all service tiers, moving away from the current “dark pattern” of opt-out consent. Providers are urged to invest in proactive safeguards, such as automated systems to detect and redact sensitive data from prompts. Lastly, the paper stresses the importance of radical transparency, encouraging all providers to offer clear, numerical data retention timelines, comprehensive model cards, and user-friendly privacy dashboards, following the lead of Google and Anthropic.
This research establishes a new benchmark for transparency and advocates for a shift towards more user-centric privacy standards in the AI industry. The full details of this important work can be found in the research paper. Read the full paper here.
