TLDR: PSG-Agent is a novel, training-free system designed to make LLM-based agents safer by providing personalized and dynamic guardrails. It addresses the limitations of existing ‘one-size-fits-all’ safety policies and their inability to track cumulative risks across multi-turn interactions. The system works in two stages: first, it generates personalized safety criteria by extracting user profiles from chat history and current queries; second, it enforces these criteria through a multi-point defense mechanism including a Plan Monitor, Tool Firewall, Memory Guardian, and Response Guard. PSG-Agent has demonstrated superior performance over state-of-the-art baselines in detecting user-specific risks across various scenarios.
Large Language Model (LLM)-based agents are becoming increasingly powerful, helping us in areas like finance, healthcare, and daily automation. These agents can plan, use tools, remember past interactions, and engage in multi-turn conversations. However, with their growing autonomy, ensuring their safety is paramount. Without proper safeguards, they could give harmful advice, lead to financial losses, or even breach privacy.
Current safety systems, often called guardrails, face two major challenges. First, they typically apply a universal set of rules to everyone. This means a behavior that might be harmless for one person could be risky for another, depending on their unique traits or circumstances. For example, suggesting a painkiller might be fine for a healthy adult but dangerous for someone on anticoagulants. Second, these systems usually check each interaction in isolation, failing to recognize how risks can build up over a series of conversations. An agent might start with conservative advice but, over time, could be influenced by a user’s impulsive personality, leading to a risky decision.
To address these critical issues, researchers have introduced a new system called PSG-Agent: Personality-Aware Safety Guardrail for LLM-Based Agents. This innovative framework offers a personalized and dynamic approach to agent safety, designed to understand individual user needs and track risks across multiple interactions. You can read the full research paper here.
How PSG-Agent Works: A Two-Stage Approach
PSG-Agent operates in two main stages, ensuring safety without requiring modifications to the underlying LLM. It’s a plug-and-play system that can integrate with any LLM-based agent.
Stage 1: Generating Personalized Safety Criteria
This stage focuses on understanding the user and defining specific safety rules for them.
-
Profile Miner Agent: This component analyzes the user’s chat history and current query to create a detailed user profile. This profile includes ‘stable attributes’ like demographics, health conditions, psychological states, and personality traits (e.g., openness, conscientiousness). It also captures ‘dynamic attributes’ such as the user’s current emotional state. The Profile Miner prioritizes explicit information but can also make cautious inferences from context.
-
Input Guard Agent: Once the user profile is established, the Input Guard takes over. It assesses the current user query against the personalized profile to estimate potential harm. It then assigns an overall risk score and makes a safety decision: ‘Allow’, ‘Allow with Guard’, ‘Refuse with Alternative’, or ‘Refuse’. Crucially, it compiles a ‘Personalized Safety Criteria’ (PSC) – a set of rules that include forbidden actions, required measures, tool limitations, and even desired response styles. This agent also uses a ‘Memory Module’ that draws from a ‘Safety Casebase’ (reusable policy templates) and a ‘User Safety Ledger’ (past violation records) to inform its decisions.
Stage 2: Implementing Personalized Defense
With the personalized safety criteria in place, this stage ensures these rules are enforced throughout the agent’s operation.
-
Plan Monitor Agent: Before an agent executes any plan, the Plan Monitor reviews it against the personalized safety criteria. If the plan is risky but repairable, it issues ‘runtime constraints’ (like limiting tool parameters or usage rates). If safety cannot be guaranteed, it requests the agent to ‘Replan’ or even escalates the decision to a refusal.
-
Tool Firewall and Memory Guardian: These are specialized components. The Tool Firewall audits every tool call made by the agent, ensuring it adheres to the constraints set by the Plan Monitor. The Memory Guardian acts as a gatekeeper for memory writes, ensuring that any information stored by the agent is safe and compliant with the personalized rules.
-
Response Guard Agent: This is the final line of defense for the agent’s output. The Response Guard checks the final text response to ensure it aligns with the personalized safety criteria and desired response style. If necessary, it performs minimal edits, redactions, or stylistic adjustments to achieve compliance. If a safe response cannot be generated without altering the user’s intent or violating the PSC, it falls back to earlier components for a safer alternative or refusal.
Demonstrated Superior Performance
PSG-Agent was rigorously tested on a comprehensive benchmark covering various scenarios like finance, medicine, and daily life, with diverse user profiles. It significantly outperformed existing guardrail systems, including Llama-Guard 3 and AGrail. For instance, PSG-Agent achieved an accuracy of 79.7%, compared to 58.3% for Llama-Guard 3 and 53.3% for AGrail. Its recall, which measures its ability to detect risks, was exceptionally high, identifying 2.5 times more risks than other methods while maintaining high precision.
The research highlights that explicit user profiling is crucial, as simply providing chat history to an LLM doesn’t achieve the same level of personalized safety. PSG-Agent’s multi-stage architecture, from profile mining to response guarding, allows it to identify and mitigate risks effectively, adapting to individual user vulnerabilities and preventing risk accumulation across interactions. The system also demonstrated strong generalizability, maintaining high safety scores across various LLM backbones, including GPT-4o, GPT-5-mini, Grok-3, and Llama-3.3-70B.
Also Read:
- AgentGuard: Ensuring Predictability in Autonomous AI Systems
- AdvChain: Enhancing AI Reasoning Safety Through Dynamic Self-Correction
Conclusion
PSG-Agent represents a significant step forward in making LLM-based agents safer and more trustworthy. By moving beyond generic safety rules to a personalized, dynamic, and multi-layered defense system, it provides a practical and auditable pathway for deploying these powerful agents responsibly in sensitive applications. This ensures that agents can offer helpful assistance while being acutely aware of and responsive to each user’s unique safety needs.
