TLDR: ANCHOR is an unsupervised anomaly detection solution designed to proactively identify connectivity issues in global IoT services. It leverages international mobility signaling data from roaming hubs and uses machine learning models, particularly Isolation Forest with expert-engineered features and device clustering, to detect anomalies before they become critical. The solution has been successfully deployed and validated in a live production environment, demonstrating its ability to identify previously undetected harmful incidents and improve the reliability of IoT connectivity.
The world of Internet of Things (IoT) is vast and ever-expanding, with countless devices relying on global connectivity. From smart sensors to connected cars, these devices depend on mobile network operators and international roaming to function seamlessly across borders. However, this complex ecosystem, where data travels through multiple entities, makes it incredibly challenging to ensure consistent communication and reliability. Often, issues are only addressed reactively, after they have already impacted service quality and led to user complaints.
Introducing ANCHOR: A Proactive Solution for IoT Connectivity
A new research paper, titled “Anomaly Detection for IoT Global Connectivity”, by Jesus Omaña Iglesias, Carlos Segura Perales, Stefan Geißler, Diego Perino, and Andra Lutu, introduces ANCHOR – an unsupervised anomaly detection solution designed to tackle these challenges head-on. Developed through a collaboration between Telefonica, Spain, and the University of Würzburg, Germany, ANCHOR aims to help engineers proactively identify problematic clients with connectivity issues affecting their IoT devices, long before these incidents become severe.
The core problem ANCHOR addresses stems from the global deployment model of IoT applications. These applications heavily rely on international roaming, which, while enabling worldwide reach, introduces significant complexity. The end-to-end communication path for an IoT device can span multiple domains, including the visited network, the IoT provider’s home network, and an international roaming hub. This intricate setup is prone to performance degradation and outages, making traditional alarm systems insufficient.
How ANCHOR Works: Leveraging Signaling Data
ANCHOR’s innovative approach involves building an unsupervised anomaly detection pipeline that utilizes machine learning. Instead of waiting for user complaints, it collects international mobility signaling data directly from the roaming hub – the interconnection point linking the IoT provider to its roaming partners. This data provides crucial insights into the connectivity and mobility of IoT devices, such as smart sensors, connected cars, and shipping containers. By analyzing this signaling traffic, ANCHOR generates alarms that engineering teams can use to improve their service proactively.
The solution processes vast amounts of data, including Signaling Connection Control Part (SCCP) for 2G/3G technologies and Diameter Exchange for 4G roaming. This comprehensive dataset, collected from millions of IoT devices operating worldwide, captures billions of signaling interactions, allowing ANCHOR to reconstruct communication dialogues between network functions.
Overcoming Challenges in a Live Environment
Designing ANCHOR for a live production system presented several key challenges:
- Vertical-Agnostic Approach: IoT devices are incredibly diverse (e.g., elevators vs. vehicles). ANCHOR needed to be versatile enough to monitor anomalies across various IoT vertical clients without prior knowledge of their specific applications. This was achieved by clustering devices based on similar traffic patterns, such as mobility and traffic volume.
- Data Fidelity and Validity: Transforming raw, messy signaling data into a structured format suitable for machine learning was crucial. The researchers explored two main data representation approaches: a “Signaling Matrix” for deep learning models and an “Expert Knowledge” representation that leverages insights from the engineering team.
- Operational Feasibility: ANCHOR had to provide tangible value beyond existing monitoring systems and integrate seamlessly. The operations team advised that running ANCHOR once a day was sufficient, as existing systems already capture large-scale incidents.
- Usability of Output: For engineers to trust and adopt ANCHOR, its outputs needed to be explainable. This led to a preference for ensemble methods like Isolation Forest over more obscure deep learning approaches.
Pipeline Design and Performance
The ANCHOR pipeline involves three main steps: data collection, data transformation (feature selection and representation), and model selection. The “Expert Knowledge” approach, which transforms real-world expertise into 95 distinct features (covering traffic volume, message types, device activity, and mobility statistics), proved to be superior. Within this approach, the Isolation Forest model consistently demonstrated strong performance, particularly when combined with the clustering step. This clustering is vital because it prevents the model from mistakenly identifying minority device types (e.g., highly mobile connected cars) as anomalies when trained on a global dataset.
During live trials, ANCHOR successfully identified previously unknown harmful anomalies, even using models trained months prior. For instance, it detected devices with aggressive signaling behavior that were stressing visited networks, potentially endangering commercial roaming agreements. This proactive identification allowed engineers to intervene and resolve issues before they escalated.
Also Read:
- Unlocking Alarm Relationships: TempOpt’s Solution for Telecom Networks
- New Adaptive Thresholding Methods Enhance Anomaly Detection in Dynamic Time Series Data
The Future of Proactive IoT Anomaly Detection
The success of ANCHOR highlights the importance of integrating human expert knowledge into machine learning solutions for complex systems like global IoT connectivity. While deep learning models were explored, simpler, more explainable models like Isolation Forest, when combined with well-engineered features, proved more effective and trustworthy for operational teams. The research suggests that ANCHOR’s approach has the potential to be generalized to other IoT providers, given the similar underlying signaling protocols and data characteristics across the ecosystem.
This work represents a significant step forward in ensuring the availability and reliability of global IoT services, moving from a reactive problem-solving approach to a proactive, intelligent one. For more technical details, you can refer to the full research paper here.
