TLDR: CHRONO GRAPH is a new, real-world dataset for multivariate time series forecasting and anomaly detection in microservice systems. It uniquely combines multivariate service metrics, an explicit dependency graph, and expert-annotated incident labels from production environments. Initial evaluations reveal that current models struggle with long-term forecasting and topology-agnostic anomaly detection, suggesting a need for graph-aware approaches to better understand and predict disruptions in complex IT infrastructures.
A new dataset called CHRONO GRAPH has been introduced, offering a realistic benchmark for understanding and predicting issues within complex microservice systems. This dataset is designed to help researchers develop better methods for forecasting system performance and detecting anomalies, which are crucial for maintaining reliable operations in large-scale IT infrastructures.
Modern IT systems often rely on microservice architectures, where hundreds of independent services work together, forming intricate dependency graphs. When one service experiences a problem, it can quickly spread across the entire system, making it challenging to predict future performance or identify the root cause of disruptions. Existing datasets used for forecasting and anomaly detection, such as those for traffic or air quality, often lack the specific characteristics of microservice environments. They might be univariate (tracking only one type of data), lack real incident annotations, or not provide a clear, machine-readable map of service dependencies.
CHRONO GRAPH addresses these gaps by providing a unique combination of features. It includes multivariate time series data for each service, meaning it tracks multiple system-level performance metrics like CPU, memory, and network usage simultaneously. Crucially, it also provides an explicit, machine-readable graph showing how these services depend on each other. Furthermore, the dataset comes with expert-annotated incident windows, which serve as real-world anomaly labels, allowing for the evaluation of anomaly detection methods during actual operational disruptions.
The dataset was built from six months of operational telemetry collected from a production microservice platform. It covers approximately 700 services, each with a 5-dimensional time series capturing performance metrics over roughly 8,000 time steps. Additionally, the communication between interacting services is captured as 3-dimensional time series on the edges of the graph, detailing aspects like the number of requests, return codes, and latency. This rich data allows for a comprehensive study of how disturbances propagate through a system.
Initial evaluations using CHRONO GRAPH have highlighted some key limitations of current forecasting and anomaly detection models. When it comes to forecasting, models like Prophet, Chronos, and TabPFN-TS showed reasonable accuracy for short-term predictions (e.g., the first 500 time steps) but struggled significantly with long-term forecasting over the full test window. This suggests that current methods have difficulty modeling complex, evolving dynamics over extended periods.
For anomaly detection, the performance of standard methods, including Isolation Forest, One-Class SVM, and even time-series foundation models like Chronos and TabPFN-TS, was modest. While an ensemble approach combining Prophet, Isolation Forest, and Autoencoder showed a more balanced performance, it still fell short of reliably identifying all disruption periods. Interestingly, observations showed that predicted anomalies often clustered along connected services, suggesting that disruptions tend to propagate along the underlying graph topology. This finding underscores the potential for graph-aware methods to improve both forecasting and anomaly detection by explicitly incorporating the system’s dependency structure.
Also Read:
- Unlocking Deeper Insights: A New Approach to Forecasting Multivariate Time Series
- A New Metric for Evaluating Time Series Anomaly Detection: Confidence and Uncertainty Consistency
The researchers hope that CHRONO GRAPH will serve as a valuable resource for the scientific community, fostering the development of new, structure-aware approaches that can better integrate temporal dynamics with graph information. This could lead to more effective solutions for predicting system behavior and identifying incidents before they escalate, ultimately enhancing the reliability and resilience of large-scale microservice systems. You can learn more about this work by reading the full research paper: CHRONO GRAPH: A Real-World Graph-Based Multivariate Time Series Dataset.
