TLDR: A new study reveals a critical vulnerability in distributed AI systems (like those splitting models between edge devices and the cloud): attackers can intercept intermediate data (features) transmitted between model components. By reconstructing the original shape of these features and using them to train “surrogate” models via “feature distillation,” attackers can significantly enhance the transferability and success rate of adversarial attacks, even against black-box target models. This highlights an urgent need for better security in distributed AI, especially at the edge.
As machine learning models become increasingly prevalent, they are often deployed in distributed environments, particularly at the “edge” of the internet of things (IoT). This setup involves splitting a deep learning model across multiple computational nodes, such as an edge device and a cloud server. While this offers benefits like efficiency, it also introduces new security challenges, expanding the potential “attack surface” for adversaries.
A recent research paper, “Exploiting Edge Features for Transferable Adversarial Attacks in Distributed Machine Learning”, explores a previously overlooked vulnerability in these distributed systems. Traditionally, security concerns in machine learning focus on protecting the model’s inputs and final outputs. However, this work demonstrates that even if both the edge and cloud components of a model are inaccessible (operating as “black-boxes”), an attacker who intercepts the intermediate data transmitted between them can still pose a serious threat.
The core idea is that these intercepted intermediate features can be leveraged to create highly effective “proxy” or “surrogate” models. These surrogate models can then be used to craft “adversarial examples” – subtle, human-imperceptible perturbations to input data designed to trick the target model into making incorrect predictions. The key is that these adversarial examples become highly “transferable,” meaning they are likely to succeed against the actual target model.
The researchers addressed two main challenges for an attacker in this scenario. First, when intermediate features are transmitted, they are often “serialized” or flattened, losing their original tensor shape (like the height, width, and number of channels of an image feature). The paper proposes an efficient statistical analysis technique, based on the covariance matrix of the intercepted features, to reliably reconstruct this original shape. This allows the attacker to understand the structure of the data they’ve intercepted.
Second, with the reconstructed feature shape, the attacker can then adapt a “surrogate” model architecture. This involves designing an “adaptation block” that transforms the surrogate’s internal representations to precisely match the dimensions of the intercepted features from the target model. This alignment is crucial for a technique called “feature distillation,” where the surrogate model is trained not just to mimic the target’s final outputs, but also its internal feature representations.
Extensive experiments were conducted using common machine learning models and the CIFAR-10 dataset. The findings were significant: surrogate models trained with this feature distillation strategy showed a tremendous improvement in the transferability of adversarial attacks. For instance, in one black-box attack scenario, the success rate against a ResNet56 target model increased from 69% to 96% when a VGG16 surrogate was trained using intermediate features, compared to one trained without this access. Similar improvements were observed in other attack types and scenarios, including those where the attacker had very limited information about the target’s outputs.
The research also highlighted that attacks were more successful when the surrogate model’s internal “split point” (where its architecture is conceptually divided to match the target’s edge-cloud split) aligned closely with the target model’s actual split point. Interestingly, the “clean accuracy” of the surrogate model (its performance on normal, unperturbed data) could serve as an empirical guide for attackers to select the most effective split point for their surrogate, even when the target’s internal structure is unknown.
Also Read:
- New Research Uncovers Stealthy Backdoors in Deep Reinforcement Learning Supply Chains
- BackFed: A New Standard for Evaluating Backdoor Attacks in Federated Learning
This work underscores an urgent need for developers of secure distributed deep learning systems, especially in edge environments with constrained devices and communication vulnerabilities, to account for intermediate feature leakage. It reveals a practical and novel threat model that demands new defense mechanisms to protect internal representations and prevent such sophisticated black-box attacks.
