TLDR: AVEC (Adaptive Verifiable Edge Control) is a new framework designed to improve privacy for local Large Language Models (LLMs) by enforcing privacy measures directly on the user’s device. It uses an adaptive budgeting algorithm to allocate per-query differential privacy parameters based on sensitivity and local confidence, combined with verifiable transformations and on-device integrity checks. Simulations show AVEC significantly reduces delegation to remote services, lowering costs and latency, and provides explicit verifiability of privacy enforcement, addressing key challenges in deploying trustworthy LLMs.
Large Language Models, or LLMs, are becoming increasingly common in our daily lives, from powering chatbots to assisting in decision-making. However, their widespread adoption faces significant hurdles, particularly concerning privacy, latency, and trust. While running LLMs directly on a user’s device (local LLMs) offers better privacy, these models often lack the accuracy and capabilities of their larger, cloud-based counterparts due to limited resources. Conversely, remote LLMs, accessed via APIs, can be powerful but come with costs, delays, and the inherent risk of exposing sensitive user data.
Existing solutions, such as simple caching or static privacy rules, offer only partial fixes. They often fail to adapt to the specific sensitivity of a user’s query or their unique context, and they typically lack any mechanism for users to independently verify that their privacy is actually being protected. This creates a fundamental challenge in developing local LLMs, where privacy concerns and a lack of initial training data hinder their improvement and broader use.
Introducing AVEC: Adaptive Verifiable Edge Control
A new framework called AVEC (Adaptive Verifiable Edge Control) aims to tackle these challenges head-on. AVEC proposes a decentralized approach to privacy enforcement, bringing the control and verification of LLM interactions directly to the user’s device. This means that the crucial steps of allocating privacy budgets and verifying privacy measures happen locally, minimizing how much sensitive data is delegated to remote services.
At its core, AVEC operates with a three-tier architecture: a local agent on the user’s device, a translation agent, and a remote agent (a standard, powerful LLM). The local agent is the user’s guardian, managing preferences and calculating a specific privacy budget for each query. This budget is dynamically determined by factors like the query’s sensitivity, the local model’s confidence in answering it, and the user’s historical privacy consumption. If the local agent is confident it can answer a query safely, it does so without involving external services or consuming privacy budget.
If a query needs to be delegated, it’s passed to the translation agent. This agent applies privacy-preserving transformations, such as entity-level randomized response, which subtly alters sensitive information (like names or dates) within the query to protect privacy. Crucially, the translation agent also generates a verifiable proof of transformation. This proof, which contains parameters and a cryptographic hash but no raw user data, is then sent back to the local agent. The local agent verifies this proof before any data leaves the device, ensuring that the declared privacy measures have indeed been applied.
The remote agent, which is an untrusted black box from a privacy perspective, only receives these transformed, privatized queries. Because differential privacy guarantees that privacy is maintained even after post-processing, the remote model’s output and the verification proof do not introduce additional privacy risks.
Formal Guarantees and Simulation Results
AVEC formalizes its privacy guarantees using R´enyi differential privacy and a concept called privacy odometers, which help account for privacy loss over time, even with adaptive budget allocation. The framework also establishes important theoretical results, including utility ceilings for recovering original information, bounds on information leakage through delegation, and impossibility results for certain deterministic privacy approaches.
In simulations involving one hundred users and ten queries each across twenty-five domains, AVEC demonstrated significant improvements. Compared to baselines where all queries were sent remotely without transformation, AVEC’s dynamic high privacy configuration reduced delegation by approximately two-thirds. This reduction directly led to lower cumulative costs and latency, as many queries were handled locally. The verification mechanism consistently confirmed that privacy transformations matched declared parameters, with all simulated trials yielding successful verification.
Looking Ahead
While promising, the current AVEC framework is simulation-based, meaning it hasn’t yet been tested with live LLMs in real-world scenarios. Future work will involve deploying AVEC with actual LLM services to assess its robustness and user experience. There’s also a need for more sophisticated utility metrics, stronger cryptographic proofs (like zero-knowledge proofs), and strategies for scaling the translation agent for large enterprise deployments. Despite these limitations, AVEC lays a strong foundation for combining adaptive privacy budgeting, verifiable transformations, and formal differential privacy accounting in LLM interactions, paving the way for more trustworthy and private language model deployments. You can read the full research paper here.
Also Read:
- Enhancing Private In-Context Learning Through Public Information
- Securing the AI Frontier: A Deep Dive into Threats and Defenses for LLM Systems
Ethical Considerations
AVEC also brings important ethical considerations to light. While it protects user inputs, it doesn’t inherently prevent harmful or biased outputs from the remote model, necessitating independent content moderation. The adaptive budgeting mechanism could also create disparities if users quickly exhaust their privacy budget, leading to degraded service. Transparent communication of budget status and careful default settings are crucial to ensure fair access. These ethical challenges must be carefully addressed for responsible deployment of the framework.
