TLDR: The Perception Graph is a novel model designed to detect cognitive attacks in Augmented Reality (AR) systems. It mimics human perception using Vision-Language Models (VLMs) to create semantically rich representations of AR environments. By comparing these representations against a reference graph and calculating a ‘distance score’ and Z-score, the model can quantitatively identify and reason about perception distortions caused by attacks like adding fake objects or removing real ones, offering a robust and interpretable defense mechanism.
Augmented Reality (AR) systems are becoming increasingly common, especially in critical environments. However, their seamless integration of digital and physical worlds also opens them up to new vulnerabilities: cognitive attacks. These attacks manipulate a user’s perception, potentially leading to poor decision-making by inserting fake objects or removing real ones from the AR environment.
Traditional methods for detecting such attacks often fall short. Pixel-level computer vision models struggle with semantically meaningful alterations, while supervised learning approaches require vast amounts of training data that are difficult to acquire in diverse and safety-critical AR scenarios.
To address these significant challenges, researchers have introduced a novel model called the Perception Graph. This innovative approach is designed to reason about human perception within AR systems, providing a robust and measurable way to detect and analyze the effects of cognitive attacks.
How the Perception Graph Works
The Perception Graph operates in two main phases: Graph Construction and Detection.
Graph Construction Phase: This phase begins by creating ‘reference graphs’ that capture the true, ground-truth semantic interpretation of a scene. It leverages pre-trained Vision-Language Models (VLMs) to mimic how humans interpret key information. These natural language descriptions are then processed by a text encoder, which projects them into a latent embedding space. In this space, the semantic meaning is represented by vector directions, allowing for consistent measurement of similarities between descriptions using cosine similarity.
Crucially, this phase also assigns contextual weights to each object in the scene, quantifying its relative importance. This ensures that the model focuses its protection efforts on critical elements, such as navigation markers or hazard warnings, while assigning lower priority to less relevant objects. This selective emphasis helps concentrate detection resources where cognitive attacks would cause the most harm.
Detection Phase: In this phase, the model processes new AR frames, generating a new perception graph for the current scene. This new graph is then aligned with the stored reference graphs. Semantic changes—like the addition, removal, or modification of objects—are identified by comparing the embeddings of corresponding nodes. A special distance function is used to quantify these differences: a smaller distance indicates semantic consistency, while a larger distance reveals deviations. A distance of 1, for instance, signifies a missing node. If these distances exceed a predefined threshold, a potential cognitive attack alert is triggered.
Also Read:
- OccVLA: Enhancing Autonomous Driving with Implicit 3D Occupancy Understanding from 2D Vision
- DepthVision: Enabling Robots to See Clearly in Challenging Conditions with LiDAR-Enhanced Vision
Demonstrating Attack Reasoning
The researchers demonstrated the Perception Graph’s effectiveness using examples of cognitive attacks in an agricultural drone scene. They showed how alterations in the Perception Graph’s information or structural changes resulted in significantly higher distance scores. For instance, attacks like modifying a planned route, adding a fake control panel, or deleting a map were all successfully detected.
To quantify these deviations, a Z-score is computed for each observed distance. This statistical mapping transforms raw distances into interpretable evidence of semantic deviation. Attacks such as route modification, fake control panel, and map deletion yielded Z-scores well outside the normal variation, clearly indicating a cognitive attack.
The Perception Graph offers a human-like understanding of AR environments, moving beyond traditional black-box processes. It provides a robust, interpretable, and trustworthy foundation for perception security, especially in mission-critical applications where accurate human perception is paramount. You can read the full research paper here: Perception Graph for Cognitive Attack Reasoning in Augmented Reality.
